2024-09-11T22:01:15.828 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T22:01:15.856 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T22:01:15.950 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901415' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgXxEz8DSEM0LJcQ0p5QmBt9HnnwaXvxhVAbu7qehi75qfnpF8CG1pGg8ni2l12nZUhGgY90goJ4PJOHkbxKN4= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC70QINxM94Tf1I8hh2l/1mT2+62QbJ/aX+INDlJHXzqha5eKm7ZVz5YVAL/jl/BrDu8NtnxKT/1O6VC6cI9dHg= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDf4kGJCc1Hq//x5AzNjbLUgWZ+XCqmR0sMZdMQW9mXwFzZ4a4LF1zbKzfKsacYNczkyb4YGGK8VJ82iwadti9w= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T22:01:15.950 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-11T22:01:15.951 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-11T22:01:15.951 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T22:01:15.953 INFO:teuthology.task.internal:Checking packages... 2024-09-11T22:01:15.975 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-11T22:01:15.976 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T22:01:15.976 INFO:teuthology.packaging:ref: None 2024-09-11T22:01:15.976 INFO:teuthology.packaging:tag: None 2024-09-11T22:01:15.976 INFO:teuthology.packaging:branch: squid 2024-09-11T22:01:15.976 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:01:15.976 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-11T22:01:16.216 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-11T22:01:16.218 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T22:01:16.223 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T22:01:16.224 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T22:01:16.342 INFO:teuthology.task.internal:Saving configuration 2024-09-11T22:01:16.363 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T22:01:16.405 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T22:01:16.432 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 21:56:02.671191', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBgXxEz8DSEM0LJcQ0p5QmBt9HnnwaXvxhVAbu7qehi75qfnpF8CG1pGg8ni2l12nZUhGgY90goJ4PJOHkbxKN4='} 2024-09-11T22:01:16.457 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 21:56:02.670023', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC70QINxM94Tf1I8hh2l/1mT2+62QbJ/aX+INDlJHXzqha5eKm7ZVz5YVAL/jl/BrDu8NtnxKT/1O6VC6cI9dHg='} 2024-09-11T22:01:16.481 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 21:56:02.668574', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDf4kGJCc1Hq//x5AzNjbLUgWZ+XCqmR0sMZdMQW9mXwFzZ4a4LF1zbKzfKsacYNczkyb4YGGK8VJ82iwadti9w='} 2024-09-11T22:01:16.482 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T22:01:16.493 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-11T22:01:16.493 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-11T22:01:16.493 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-11T22:01:16.493 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T22:01:16.695 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa0aabd3be0>, signals=[15]) 2024-09-11T22:01:16.695 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T22:01:16.705 INFO:teuthology.task.internal:Opening connections... 2024-09-11T22:01:16.705 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T22:01:16.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:01:16.791 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2024-09-11T22:01:16.792 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:01:16.869 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-09-11T22:01:16.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:01:16.944 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T22:01:16.953 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-09-11T22:01:16.972 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-09-11T22:01:16.972 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-09-11T22:01:17.030 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-09-11T22:01:17.030 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-09-11T22:01:17.030 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-09-11T22:01:17.030 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-09-11T22:01:17.031 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-09-11T22:01:17.031 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-09-11T22:01:17.031 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T22:01:17.031 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-09-11T22:01:17.031 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-09-11T22:01:17.031 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T22:01:17.031 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-09-11T22:01:17.031 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T22:01:17.031 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T22:01:17.031 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T22:01:17.032 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-09-11T22:01:17.055 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-09-11T22:01:17.074 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-09-11T22:01:17.074 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-09-11T22:01:17.132 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-09-11T22:01:17.132 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-09-11T22:01:17.132 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-09-11T22:01:17.132 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-09-11T22:01:17.132 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-09-11T22:01:17.132 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-09-11T22:01:17.132 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T22:01:17.132 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-09-11T22:01:17.133 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-09-11T22:01:17.133 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T22:01:17.133 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-09-11T22:01:17.133 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T22:01:17.133 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T22:01:17.133 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T22:01:17.133 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-09-11T22:01:17.156 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-09-11T22:01:17.175 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-09-11T22:01:17.176 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-09-11T22:01:17.233 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2024-09-11T22:01:17.233 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2024-09-11T22:01:17.233 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2024-09-11T22:01:17.233 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2024-09-11T22:01:17.233 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2024-09-11T22:01:17.233 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2024-09-11T22:01:17.233 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T22:01:17.234 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-09-11T22:01:17.234 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2024-09-11T22:01:17.234 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T22:01:17.234 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2024-09-11T22:01:17.234 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T22:01:17.234 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T22:01:17.234 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T22:01:17.234 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-09-11T22:01:17.256 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T22:01:17.267 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T22:01:17.307 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T22:01:17.307 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-09-11T22:01:17.309 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2024-09-11T22:01:17.311 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-09-11T22:01:17.329 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T22:01:17.369 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T22:01:17.370 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-09-11T22:01:17.372 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2024-09-11T22:01:17.374 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-09-11T22:01:17.403 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T22:01:17.503 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T22:01:17.520 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-11T22:01:17.521 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-11T22:01:17.521 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-11T22:01:17.521 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T22:01:17.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:01:17.538 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-11T22:01:17.596 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-503.el9.x86_64 2024-09-11T22:01:17.596 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.14.0-503.el9.x86_64 2024-09-11T22:01:17.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2024-09-11T22:01:19.873 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-11T22:01:24.858 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 20 MB 00:03 2024-09-11T22:01:30.850 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-11T22:01:32.650 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-11T22:01:36.429 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 23 MB 00:03 2024-09-11T22:01:42.243 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-09-11T22:01:44.299 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T22:01:44.300 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T22:01:44.399 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:01:44.411 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-09-11T22:01:44.412 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-11T22:01:44.412 INFO:teuthology.orchestra.run.smithi005.stdout:Install 4 Packages 2024-09-11T22:01:44.412 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:01:44.412 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 79 M 2024-09-11T22:01:44.412 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 126 M 2024-09-11T22:01:44.412 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-09-11T22:01:44.794 INFO:teuthology.orchestra.run.smithi005.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 249 kB/s | 23 kB 00:00 2024-09-11T22:01:47.086 INFO:teuthology.orchestra.run.smithi005.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.6 MB/s | 16 MB 00:02 2024-09-11T22:01:48.903 INFO:teuthology.orchestra.run.smithi005.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.9 MB/s | 29 MB 00:04 2024-09-11T22:01:50.728 INFO:teuthology.orchestra.run.smithi005.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 5.7 MB/s | 35 MB 00:06 2024-09-11T22:01:50.728 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:01:50.728 INFO:teuthology.orchestra.run.smithi005.stdout:Total 12 MB/s | 79 MB 00:06 2024-09-11T22:01:51.436 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-09-11T22:01:51.852 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-09-11T22:01:51.853 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-09-11T22:01:52.860 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-09-11T22:01:52.860 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-09-11T22:01:55.449 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-09-11T22:01:56.106 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T22:01:56.195 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T22:01:59.024 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T22:01:59.156 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T22:02:04.281 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T22:02:04.324 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:02:09.469 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:02:48.808 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:02:48.850 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:02:49.186 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:02:49.186 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T22:02:49.186 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T22:02:49.186 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T22:02:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T22:02:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:02:50.015 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-09-11T22:02:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T22:02:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T22:02:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T22:02:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T22:02:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:02:50.016 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-09-11T22:02:50.368 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T22:02:50.451 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T22:02:50.451 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T22:02:50.451 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T22:02:50.451 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T22:02:50.452 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T22:02:50.452 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-11T22:02:50.452 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T22:02:50.452 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T22:02:50.511 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T22:02:50.511 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T22:02:50.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-09-11T22:02:50.542 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-09-11T22:02:50.543 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-09-11T22:02:50.543 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-09-11T22:02:50.543 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 82.el9 2024-09-11T22:02:50.544 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-09-11T22:02:50.544 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T22:02:50.544 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-09-11T22:02:50.544 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8274670 2024-09-11T22:02:50.544 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-09-11T22:02:50.545 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T22:02:50.545 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T22:02:50.545 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T22:02:50.545 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T22:02:50.546 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-09-11T22:02:50.546 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-09-11T22:02:50.546 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T22:02:50.546 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-09-11T22:02:50.546 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-09-11T22:02:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:02:50.547 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T22:02:50.547 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T22:02:50.547 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T22:02:50.548 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-09-11T22:02:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:02:50.548 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T22:02:50.550 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T22:02:50.550 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 5.14.0-505.el9.x86_64 2024-09-11T22:02:50.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T22:02:51.191 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-09-11T22:02:52.603 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T22:02:52.633 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-09-11T22:02:52.644 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-09-11T22:02:52.659 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.rVErhASCTb 2024-09-11T22:02:52.660 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.rVErhASCTb 2024-09-11T22:02:52.726 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.rVErhASCTb 2024-09-11T22:02:52.872 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.rVErhASCTb is 6KB 2024-09-11T22:02:52.921 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.rVErhASCTb 2024-09-11T22:02:52.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T22:02:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T22:02:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T22:02:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T22:02:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T22:02:53.000 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T22:02:53.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-09-11T22:03:23.136 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T22:03:23.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-11T22:03:23.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:03:41.501 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-09-11T22:03:50.502 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-11T22:03:50.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:04:50.506 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T22:05:02.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-11T22:05:02.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:05:02.772 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-11T22:05:03.120 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-11T22:05:03.120 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-11T22:05:03.121 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-11T22:05:03.138 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-505.el9.x86_64 2024-09-11T22:05:03.139 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T22:05:03.139 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T22:05:03.139 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T22:05:04.140 DEBUG:teuthology.parallel:result is None 2024-09-11T22:05:04.140 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T22:05:04.162 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T22:05:04.163 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T22:05:04.171 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T22:05:04.183 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T22:05:04.213 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T22:05:04.401 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T22:05:05.140 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T22:05:05.140 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T22:05:05.143 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T22:05:05.145 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T22:05:05.469 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T22:05:06.052 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T22:05:06.052 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T22:05:06.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:05:06.070 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T22:05:06.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:05:06.088 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T22:05:06.091 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T22:05:06.113 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T22:05:06.150 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:05:06.170 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:05:06.170 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:05:06.192 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:05:06.206 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:05:06.213 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T22:05:06.215 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T22:05:06.723 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T22:05:06.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T22:05:06.726 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T22:05:06.729 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T22:05:06.759 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T22:05:07.080 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T22:05:07.080 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T22:05:07.083 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T22:05:07.085 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T22:05:07.104 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:05:07.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:05:07.239 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:05:07.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:05:07.369 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:05:07.369 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T22:05:07.438 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:05:07.481 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:05:07.538 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:05:07.623 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:05:07.665 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:05:07.665 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T22:05:07.733 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:05:07.773 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T22:05:07.901 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:05:07.942 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T22:05:07.989 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:05:07.989 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T22:05:08.058 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-09-11T22:05:08.060 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2024-09-11T22:05:08.063 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-09-11T22:05:08.098 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:05:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:05:08.139 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T22:05:08.651 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T22:05:08.659 INFO:teuthology.task.internal:Starting timer... 2024-09-11T22:05:08.659 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T22:05:08.700 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T22:05:08.749 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-11T22:05:08.749 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-09-11T22:05:08.806 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-09-11T22:05:08.807 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2024-09-11T22:05:08.863 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2024-09-11T22:05:08.865 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2024-09-11T22:05:08.919 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2024-09-11T22:05:08.921 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T22:05:08.921 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-09-11T22:05:08.951 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-09-11T22:05:08.951 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2024-09-11T22:05:08.975 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2024-09-11T22:05:08.975 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2024-09-11T22:05:09.005 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2024-09-11T22:05:09.005 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive'} 2024-09-11T22:05:09.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:05:09.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:05:09.035 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:05:09.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:05:09.063 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T22:05:09.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:05:09.093 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T22:05:09.093 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-09-11T22:05:09.126 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2024-09-11T22:05:09.155 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2024-09-11T22:05:09.191 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T22:05:09.200 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T22:05:09.214 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T22:05:09.216 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi159.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T22:09:10.953 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2024-09-11T22:09:10.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-11T22:09:10.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:09:11.039 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-11T22:09:11.127 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-11T22:09:11.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-11T22:09:11.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:09:11.210 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-09-11T22:09:11.286 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-11T22:09:11.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-11T22:09:11.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T22:09:11.363 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-09-11T22:09:11.455 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-09-11T22:09:11.456 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T22:09:11.473 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T22:09:11.474 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T22:09:11.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:09:11.477 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T22:09:11.477 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:09:11.479 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T22:09:11.479 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T22:09:11.505 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T22:09:11.512 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T22:09:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T22:09:11.528 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T22:09:11.528 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T22:09:11.545 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T22:09:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-09-11T22:09:11.555 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2024-09-11T22:09:11.559 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-09-11T22:09:11.567 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2024-09-11T22:09:11.571 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2024-09-11T22:09:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T22:09:11.581 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T22:09:11.583 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2024-09-11T22:09:11.591 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T22:09:11.596 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T22:09:11.598 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T22:09:11.613 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T22:09:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:09:11.649 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:09:11.649 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-09-11T22:09:11.649 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.649 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.649 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.649 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.654 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:09:11.658 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:09:11.658 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-09-11T22:09:11.658 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.658 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.658 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.658 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.669 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-09-11T22:09:11.672 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T22:09:11.672 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-09-11T22:09:11.672 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.672 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.673 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.673 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T22:09:11.673 INFO:teuthology.run_tasks:Running task install... 2024-09-11T22:09:11.681 DEBUG:teuthology.task.install:project ceph 2024-09-11T22:09:11.681 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-11T22:09:11.681 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-11T22:09:11.682 INFO:teuthology.task.install:Using flavor: default 2024-09-11T22:09:11.687 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T22:09:11.687 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T22:09:11.687 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T22:09:11.688 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:09:11.690 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T22:09:11.690 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:09:11.691 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-11T22:09:11.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:09:11.831 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T22:09:11.831 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T22:09:11.840 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T22:09:11.840 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T22:09:11.846 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-11T22:09:11.846 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-11T22:09:11.941 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T22:09:11.941 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:09:11.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T22:09:11.951 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T22:09:11.951 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:09:11.951 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T22:09:11.957 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T22:09:11.957 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:09:11.957 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T22:09:11.968 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T22:09:11.968 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T22:09:11.977 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T22:09:11.977 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T22:09:11.983 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-11T22:09:11.983 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T22:09:12.036 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T22:09:12.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T22:09:12.049 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T22:09:12.113 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T22:09:12.119 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T22:09:12.124 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T22:09:12.178 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-09-11T22:09:12.180 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-09-11T22:09:12.183 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2024-09-11T22:09:12.185 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2024-09-11T22:09:12.188 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2024-09-11T22:09:12.189 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2024-09-11T22:09:12.410 INFO:teuthology.orchestra.run.smithi172.stdout:45 files removed 2024-09-11T22:09:12.415 INFO:teuthology.orchestra.run.smithi159.stdout:45 files removed 2024-09-11T22:09:12.422 INFO:teuthology.orchestra.run.smithi005.stdout:45 files removed 2024-09-11T22:09:12.437 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T22:09:12.442 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T22:09:12.451 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-11T22:09:12.995 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 300 kB/s | 84 kB 00:00 2024-09-11T22:09:12.998 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-11T22:09:13.026 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-11T22:09:13.280 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-11T22:09:13.282 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-11T22:09:13.326 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-09-11T22:09:13.546 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-11T22:09:13.558 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-11T22:09:13.602 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-11T22:09:14.152 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-11T22:09:14.329 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-11T22:09:15.216 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.2 MB 00:01 2024-09-11T22:09:16.062 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-11T22:09:17.047 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 13 MB/s | 20 MB 00:01 2024-09-11T22:09:20.917 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 20 MB 00:04 2024-09-11T22:09:21.279 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 6.5 MB 00:01 2024-09-11T22:09:22.698 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-11T22:09:22.838 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-11T22:09:23.759 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-11T22:09:24.415 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-11T22:09:25.276 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-11T22:09:25.602 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-11T22:09:26.960 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-11T22:09:27.739 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-11T22:09:29.457 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-11T22:09:30.991 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-11T22:09:31.870 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:09:31.870 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:09:31.884 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T22:09:31.885 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T22:09:31.886 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T22:09:31.887 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T22:09:31.889 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T22:09:31.892 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T22:09:31.893 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T22:09:31.961 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-09-11T22:09:31.987 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-11T22:09:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-09-11T22:09:31.987 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-11T22:09:31.987 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-09-11T22:09:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T22:09:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T22:09:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T22:09:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T22:09:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T22:09:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T22:09:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T22:09:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T22:09:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T22:09:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T22:09:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T22:09:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T22:09:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T22:09:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T22:09:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 8 Packages 2024-09-11T22:09:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:09:32.001 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 226 M 2024-09-11T22:09:32.001 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-09-11T22:09:32.783 INFO:teuthology.orchestra.run.smithi005.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 46 kB/s | 6.5 kB 00:00 2024-09-11T22:09:33.100 INFO:teuthology.orchestra.run.smithi005.stdout:(2/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-11T22:09:33.133 INFO:teuthology.orchestra.run.smithi005.stdout:(3/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.5 MB/s | 897 kB 00:00 2024-09-11T22:09:33.159 INFO:teuthology.orchestra.run.smithi005.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 2.4 MB/s | 145 kB 00:00 2024-09-11T22:09:33.195 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-09-11T22:09:33.251 INFO:teuthology.orchestra.run.smithi005.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-11T22:09:33.268 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:09:33.269 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:09:33.280 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T22:09:33.281 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T22:09:33.282 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T22:09:33.283 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T22:09:33.284 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T22:09:33.287 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T22:09:33.287 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T22:09:33.349 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-09-11T22:09:33.373 INFO:teuthology.orchestra.run.smithi172.stdout:====================================================================================== 2024-09-11T22:09:33.373 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-09-11T22:09:33.373 INFO:teuthology.orchestra.run.smithi172.stdout:====================================================================================== 2024-09-11T22:09:33.373 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-09-11T22:09:33.373 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T22:09:33.374 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T22:09:33.375 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2024-09-11T22:09:33.376 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T22:09:33.377 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T22:09:33.378 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T22:09:33.379 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T22:09:33.380 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T22:09:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T22:09:33.382 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T22:09:33.383 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T22:09:33.384 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T22:09:33.385 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout:Installing weak dependencies: 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout:====================================================================================== 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout:Install 148 Packages 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 8 Packages 2024-09-11T22:09:33.386 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:09:33.387 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 226 M 2024-09-11T22:09:33.387 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-09-11T22:09:33.520 INFO:teuthology.orchestra.run.smithi005.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-11T22:09:33.576 INFO:teuthology.orchestra.run.smithi005.stdout:(7/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 5.4 MB/s | 2.4 MB 00:00 2024-09-11T22:09:33.835 INFO:teuthology.orchestra.run.smithi005.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 18 MB/s | 21 MB 00:01 2024-09-11T22:09:33.868 INFO:teuthology.orchestra.run.smithi172.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 52 kB/s | 6.5 kB 00:00 2024-09-11T22:09:33.885 INFO:teuthology.orchestra.run.smithi005.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 505 kB/s | 25 kB 00:00 2024-09-11T22:09:34.393 INFO:teuthology.orchestra.run.smithi172.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 897 kB 00:00 2024-09-11T22:09:34.485 INFO:teuthology.orchestra.run.smithi172.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.5 MB/s | 145 kB 00:00 2024-09-11T22:09:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 17 MB/s | 17 MB 00:01 2024-09-11T22:09:34.569 INFO:teuthology.orchestra.run.smithi005.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.5 MB/s | 228 kB 00:00 2024-09-11T22:09:34.628 INFO:teuthology.orchestra.run.smithi005.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 465 kB/s | 27 kB 00:00 2024-09-11T22:09:34.704 INFO:teuthology.orchestra.run.smithi005.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 420 kB/s | 32 kB 00:00 2024-09-11T22:09:34.803 INFO:teuthology.orchestra.run.smithi005.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 7.4 MB/s | 752 kB 00:00 2024-09-11T22:09:34.878 INFO:teuthology.orchestra.run.smithi005.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-11T22:09:34.954 INFO:teuthology.orchestra.run.smithi005.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-11T22:09:35.054 INFO:teuthology.orchestra.run.smithi005.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.0 MB/s | 506 kB 00:00 2024-09-11T22:09:35.077 INFO:teuthology.orchestra.run.smithi172.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 4.0 MB/s | 2.4 MB 00:00 2024-09-11T22:09:35.219 INFO:teuthology.orchestra.run.smithi172.stdout:(5/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 3.7 MB/s | 5.4 MB 00:01 2024-09-11T22:09:35.321 INFO:teuthology.orchestra.run.smithi005.stdout:(18/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 6.1 MB/s | 11 MB 00:01 2024-09-11T22:09:35.446 INFO:teuthology.orchestra.run.smithi005.stdout:(19/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-11T22:09:35.483 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:09:35.484 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T22:09:35.488 INFO:teuthology.orchestra.run.smithi005.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 271 kB/s | 45 kB 00:00 2024-09-11T22:09:35.498 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-11T22:09:35.500 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-11T22:09:35.500 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-11T22:09:35.502 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-11T22:09:35.503 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-11T22:09:35.506 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-11T22:09:35.507 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-11T22:09:35.510 INFO:teuthology.orchestra.run.smithi172.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 3.5 MB/s | 1.5 MB 00:00 2024-09-11T22:09:35.513 INFO:teuthology.orchestra.run.smithi005.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-11T22:09:35.539 INFO:teuthology.orchestra.run.smithi005.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.1 MB/s | 162 kB 00:00 2024-09-11T22:09:35.569 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-11T22:09:35.581 INFO:teuthology.orchestra.run.smithi005.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.7 MB/s | 321 kB 00:00 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-11T22:09:35.585 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-11T22:09:35.586 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-11T22:09:35.587 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-11T22:09:35.588 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-11T22:09:35.588 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-11T22:09:35.588 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-11T22:09:35.588 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-11T22:09:35.588 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-09-11T22:09:35.588 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-11T22:09:35.588 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-11T22:09:35.589 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T22:09:35.590 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-11T22:09:35.591 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T22:09:35.592 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T22:09:35.593 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T22:09:35.594 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T22:09:35.595 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T22:09:35.596 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T22:09:35.597 INFO:teuthology.orchestra.run.smithi159.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout:Installing weak dependencies: 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout:Install 148 Packages 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 8 Packages 2024-09-11T22:09:35.598 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:09:35.599 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 226 M 2024-09-11T22:09:35.599 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-11T22:09:35.606 INFO:teuthology.orchestra.run.smithi005.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 4.4 MB/s | 301 kB 00:00 2024-09-11T22:09:35.665 INFO:teuthology.orchestra.run.smithi005.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.2 MB/s | 100 kB 00:00 2024-09-11T22:09:35.690 INFO:teuthology.orchestra.run.smithi005.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-11T22:09:35.861 INFO:teuthology.orchestra.run.smithi172.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 7.3 MB/s | 4.7 MB 00:00 2024-09-11T22:09:36.114 INFO:teuthology.orchestra.run.smithi159.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 45 kB/s | 6.5 kB 00:00 2024-09-11T22:09:36.157 INFO:teuthology.orchestra.run.smithi005.stdout:(27/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 21 MB/s | 49 MB 00:02 2024-09-11T22:09:36.182 INFO:teuthology.orchestra.run.smithi005.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 347 kB/s | 171 kB 00:00 2024-09-11T22:09:36.207 INFO:teuthology.orchestra.run.smithi005.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 638 kB/s | 32 kB 00:00 2024-09-11T22:09:36.233 INFO:teuthology.orchestra.run.smithi005.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.8 MB/s | 146 kB 00:00 2024-09-11T22:09:36.244 INFO:teuthology.orchestra.run.smithi172.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 8.5 MB/s | 21 MB 00:02 2024-09-11T22:09:36.297 INFO:teuthology.orchestra.run.smithi159.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 4.8 MB/s | 897 kB 00:00 2024-09-11T22:09:36.308 INFO:teuthology.orchestra.run.smithi005.stdout:(31/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 4.8 MB/s | 3.1 MB 00:00 2024-09-11T22:09:36.345 INFO:teuthology.orchestra.run.smithi172.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 252 kB/s | 25 kB 00:00 2024-09-11T22:09:36.397 INFO:teuthology.orchestra.run.smithi159.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.4 MB/s | 145 kB 00:00 2024-09-11T22:09:36.633 INFO:teuthology.orchestra.run.smithi005.stdout:(32/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 8.4 MB/s | 3.6 MB 00:00 2024-09-11T22:09:36.714 INFO:teuthology.orchestra.run.smithi159.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 7.5 MB/s | 2.4 MB 00:00 2024-09-11T22:09:36.720 INFO:teuthology.orchestra.run.smithi172.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 12 MB/s | 11 MB 00:00 2024-09-11T22:09:36.725 INFO:teuthology.orchestra.run.smithi005.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 539 kB/s | 49 kB 00:00 2024-09-11T22:09:36.820 INFO:teuthology.orchestra.run.smithi172.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.2 MB/s | 228 kB 00:00 2024-09-11T22:09:36.825 INFO:teuthology.orchestra.run.smithi005.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 164 kB/s | 16 kB 00:00 2024-09-11T22:09:36.912 INFO:teuthology.orchestra.run.smithi172.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 294 kB/s | 27 kB 00:00 2024-09-11T22:09:36.919 INFO:teuthology.orchestra.run.smithi005.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.0 MB/s | 285 kB 00:00 2024-09-11T22:09:36.934 INFO:teuthology.orchestra.run.smithi159.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 6.9 MB/s | 1.5 MB 00:00 2024-09-11T22:09:36.988 INFO:teuthology.orchestra.run.smithi172.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 423 kB/s | 32 kB 00:00 2024-09-11T22:09:37.011 INFO:teuthology.orchestra.run.smithi005.stdout:(36/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 8.2 MB/s | 765 kB 00:00 2024-09-11T22:09:37.112 INFO:teuthology.orchestra.run.smithi005.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 182 kB/s | 18 kB 00:00 2024-09-11T22:09:37.163 INFO:teuthology.orchestra.run.smithi172.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 4.2 MB/s | 752 kB 00:00 2024-09-11T22:09:37.187 INFO:teuthology.orchestra.run.smithi005.stdout:(38/156): ceph-mgr-modules-core-19.1.1-298.gb53 284 kB/s | 249 kB 00:00 2024-09-11T22:09:37.237 INFO:teuthology.orchestra.run.smithi005.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 637 kB/s | 80 kB 00:00 2024-09-11T22:09:37.238 INFO:teuthology.orchestra.run.smithi172.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-11T22:09:37.297 INFO:teuthology.orchestra.run.smithi172.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-11T22:09:37.346 INFO:teuthology.orchestra.run.smithi005.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2024-09-11T22:09:37.355 INFO:teuthology.orchestra.run.smithi172.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 8.5 MB/s | 506 kB 00:00 2024-09-11T22:09:37.388 INFO:teuthology.orchestra.run.smithi005.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-11T22:09:37.423 INFO:teuthology.orchestra.run.smithi159.stdout:(6/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 15 MB/s | 21 MB 00:01 2024-09-11T22:09:37.447 INFO:teuthology.orchestra.run.smithi005.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-11T22:09:37.490 INFO:teuthology.orchestra.run.smithi159.stdout:(7/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 3.6 MB/s | 5.4 MB 00:01 2024-09-11T22:09:37.530 INFO:teuthology.orchestra.run.smithi005.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-11T22:09:37.574 INFO:teuthology.orchestra.run.smithi159.stdout:(8/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 7.4 MB/s | 4.7 MB 00:00 2024-09-11T22:09:37.622 INFO:teuthology.orchestra.run.smithi005.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-09-11T22:09:37.648 INFO:teuthology.orchestra.run.smithi005.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-11T22:09:37.657 INFO:teuthology.orchestra.run.smithi159.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 302 kB/s | 25 kB 00:00 2024-09-11T22:09:37.698 INFO:teuthology.orchestra.run.smithi005.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-11T22:09:37.764 INFO:teuthology.orchestra.run.smithi172.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-11T22:09:37.798 INFO:teuthology.orchestra.run.smithi005.stdout:(47/156): ceph-mgr-diskprediction-local-19.1.1- 4.7 MB/s | 7.4 MB 00:01 2024-09-11T22:09:37.824 INFO:teuthology.orchestra.run.smithi005.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.2 MB/s | 215 kB 00:00 2024-09-11T22:09:37.864 INFO:teuthology.orchestra.run.smithi172.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 452 kB/s | 45 kB 00:00 2024-09-11T22:09:37.939 INFO:teuthology.orchestra.run.smithi172.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.8 MB/s | 136 kB 00:00 2024-09-11T22:09:38.023 INFO:teuthology.orchestra.run.smithi172.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 162 kB 00:00 2024-09-11T22:09:38.115 INFO:teuthology.orchestra.run.smithi172.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 321 kB 00:00 2024-09-11T22:09:38.199 INFO:teuthology.orchestra.run.smithi172.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.5 MB/s | 301 kB 00:00 2024-09-11T22:09:38.249 INFO:teuthology.orchestra.run.smithi172.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-11T22:09:38.299 INFO:teuthology.orchestra.run.smithi172.stdout:(25/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T22:09:38.366 INFO:teuthology.orchestra.run.smithi159.stdout:(10/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 18 MB/s | 17 MB 00:00 2024-09-11T22:09:38.441 INFO:teuthology.orchestra.run.smithi159.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.0 MB/s | 228 kB 00:00 2024-09-11T22:09:38.508 INFO:teuthology.orchestra.run.smithi159.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 406 kB/s | 27 kB 00:00 2024-09-11T22:09:38.541 INFO:teuthology.orchestra.run.smithi172.stdout:(26/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-11T22:09:38.558 INFO:teuthology.orchestra.run.smithi159.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 637 kB/s | 32 kB 00:00 2024-09-11T22:09:38.590 INFO:teuthology.orchestra.run.smithi005.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 139 kB/s | 107 kB 00:00 2024-09-11T22:09:38.634 INFO:teuthology.orchestra.run.smithi159.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 9.8 MB/s | 752 kB 00:00 2024-09-11T22:09:38.692 INFO:teuthology.orchestra.run.smithi159.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 163 kB 00:00 2024-09-11T22:09:38.708 INFO:teuthology.orchestra.run.smithi172.stdout:(27/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 5.3 MB/s | 17 MB 00:03 2024-09-11T22:09:38.733 INFO:teuthology.orchestra.run.smithi172.stdout:(28/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 891 kB/s | 171 kB 00:00 2024-09-11T22:09:38.751 INFO:teuthology.orchestra.run.smithi159.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 127 kB 00:00 2024-09-11T22:09:38.758 INFO:teuthology.orchestra.run.smithi172.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 639 kB/s | 32 kB 00:00 2024-09-11T22:09:38.784 INFO:teuthology.orchestra.run.smithi172.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.8 MB/s | 146 kB 00:00 2024-09-11T22:09:38.799 INFO:teuthology.orchestra.run.smithi005.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 943 kB/s | 943 kB 00:01 2024-09-11T22:09:38.818 INFO:teuthology.orchestra.run.smithi159.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 7.4 MB/s | 506 kB 00:00 2024-09-11T22:09:38.841 INFO:teuthology.orchestra.run.smithi005.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 603 kB/s | 689 kB 00:01 2024-09-11T22:09:39.094 INFO:teuthology.orchestra.run.smithi159.stdout:(18/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 6.6 MB/s | 11 MB 00:01 2024-09-11T22:09:39.166 INFO:teuthology.orchestra.run.smithi005.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 38 kB/s | 22 kB 00:00 2024-09-11T22:09:39.210 INFO:teuthology.orchestra.run.smithi159.stdout:(19/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-11T22:09:39.227 INFO:teuthology.orchestra.run.smithi159.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 338 kB/s | 45 kB 00:00 2024-09-11T22:09:39.277 INFO:teuthology.orchestra.run.smithi159.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-11T22:09:39.303 INFO:teuthology.orchestra.run.smithi159.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.1 MB/s | 162 kB 00:00 2024-09-11T22:09:39.328 INFO:teuthology.orchestra.run.smithi159.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.2 MB/s | 321 kB 00:00 2024-09-11T22:09:39.342 INFO:teuthology.orchestra.run.smithi172.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-11T22:09:39.362 INFO:teuthology.orchestra.run.smithi159.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-11T22:09:39.387 INFO:teuthology.orchestra.run.smithi159.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.7 MB/s | 100 kB 00:00 2024-09-11T22:09:39.409 INFO:teuthology.orchestra.run.smithi172.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 3.7 MB/s | 249 kB 00:00 2024-09-11T22:09:39.412 INFO:teuthology.orchestra.run.smithi159.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-11T22:09:39.438 INFO:teuthology.orchestra.run.smithi005.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 56 kB/s | 33 kB 00:00 2024-09-11T22:09:39.479 INFO:teuthology.orchestra.run.smithi159.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-11T22:09:39.484 INFO:teuthology.orchestra.run.smithi172.stdout:(33/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 4.9 MB/s | 3.6 MB 00:00 2024-09-11T22:09:39.501 INFO:teuthology.orchestra.run.smithi172.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 535 kB/s | 49 kB 00:00 2024-09-11T22:09:39.543 INFO:teuthology.orchestra.run.smithi172.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 280 kB/s | 16 kB 00:00 2024-09-11T22:09:39.563 INFO:teuthology.orchestra.run.smithi159.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 18 MB/s | 3.1 MB 00:00 2024-09-11T22:09:39.568 INFO:teuthology.orchestra.run.smithi172.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.2 MB/s | 285 kB 00:00 2024-09-11T22:09:39.580 INFO:teuthology.orchestra.run.smithi159.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 317 kB/s | 32 kB 00:00 2024-09-11T22:09:39.583 INFO:teuthology.orchestra.run.smithi005.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 400 kB/s | 313 kB 00:00 2024-09-11T22:09:39.610 INFO:teuthology.orchestra.run.smithi172.stdout:(37/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 437 kB/s | 18 kB 00:00 2024-09-11T22:09:39.613 INFO:teuthology.orchestra.run.smithi159.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.8 MB/s | 146 kB 00:00 2024-09-11T22:09:39.661 INFO:teuthology.orchestra.run.smithi172.stdout:(38/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-11T22:09:39.786 INFO:teuthology.orchestra.run.smithi172.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 455 kB/s | 80 kB 00:00 2024-09-11T22:09:39.878 INFO:teuthology.orchestra.run.smithi172.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-11T22:09:39.937 INFO:teuthology.orchestra.run.smithi172.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-11T22:09:39.962 INFO:teuthology.orchestra.run.smithi172.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-11T22:09:39.967 INFO:teuthology.orchestra.run.smithi005.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-09-11T22:09:40.021 INFO:teuthology.orchestra.run.smithi172.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-11T22:09:40.022 INFO:teuthology.orchestra.run.smithi159.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 18 MB/s | 7.4 MB 00:00 2024-09-11T22:09:40.062 INFO:teuthology.orchestra.run.smithi172.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-09-11T22:09:40.084 INFO:teuthology.orchestra.run.smithi005.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-09-11T22:09:40.088 INFO:teuthology.orchestra.run.smithi172.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-11T22:09:40.105 INFO:teuthology.orchestra.run.smithi159.stdout:(32/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 6.8 MB/s | 3.6 MB 00:00 2024-09-11T22:09:40.130 INFO:teuthology.orchestra.run.smithi172.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-11T22:09:40.131 INFO:teuthology.orchestra.run.smithi159.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.2 MB/s | 249 kB 00:00 2024-09-11T22:09:40.155 INFO:teuthology.orchestra.run.smithi172.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-11T22:09:40.156 INFO:teuthology.orchestra.run.smithi159.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 980 kB/s | 49 kB 00:00 2024-09-11T22:09:40.190 INFO:teuthology.orchestra.run.smithi159.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 279 kB/s | 16 kB 00:00 2024-09-11T22:09:40.205 INFO:teuthology.orchestra.run.smithi172.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 8.9 MB/s | 689 kB 00:00 2024-09-11T22:09:40.215 INFO:teuthology.orchestra.run.smithi159.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 4.8 MB/s | 285 kB 00:00 2024-09-11T22:09:40.242 INFO:teuthology.orchestra.run.smithi005.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-11T22:09:40.257 INFO:teuthology.orchestra.run.smithi172.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 9.2 MB/s | 943 kB 00:00 2024-09-11T22:09:40.265 INFO:teuthology.orchestra.run.smithi159.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 9.9 MB/s | 765 kB 00:00 2024-09-11T22:09:40.281 INFO:teuthology.orchestra.run.smithi172.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-11T22:09:40.291 INFO:teuthology.orchestra.run.smithi159.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 242 kB/s | 18 kB 00:00 2024-09-11T22:09:40.307 INFO:teuthology.orchestra.run.smithi172.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-09-11T22:09:40.340 INFO:teuthology.orchestra.run.smithi172.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-11T22:09:40.365 INFO:teuthology.orchestra.run.smithi172.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-09-11T22:09:40.424 INFO:teuthology.orchestra.run.smithi172.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-09-11T22:09:40.468 INFO:teuthology.orchestra.run.smithi159.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 395 kB/s | 80 kB 00:00 2024-09-11T22:09:40.491 INFO:teuthology.orchestra.run.smithi172.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-09-11T22:09:40.508 INFO:teuthology.orchestra.run.smithi172.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-11T22:09:40.550 INFO:teuthology.orchestra.run.smithi172.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-11T22:09:40.575 INFO:teuthology.orchestra.run.smithi172.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-09-11T22:09:40.617 INFO:teuthology.orchestra.run.smithi172.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-11T22:09:40.650 INFO:teuthology.orchestra.run.smithi172.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-11T22:09:40.667 INFO:teuthology.orchestra.run.smithi172.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 487 kB/s | 24 kB 00:00 2024-09-11T22:09:40.734 INFO:teuthology.orchestra.run.smithi005.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-09-11T22:09:40.760 INFO:teuthology.orchestra.run.smithi005.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 202 kB/s | 160 kB 00:00 2024-09-11T22:09:40.927 INFO:teuthology.orchestra.run.smithi159.stdout:(40/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 15 MB/s | 49 MB 00:03 2024-09-11T22:09:40.960 INFO:teuthology.orchestra.run.smithi159.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 388 kB/s | 191 kB 00:00 2024-09-11T22:09:40.994 INFO:teuthology.orchestra.run.smithi159.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-11T22:09:41.052 INFO:teuthology.orchestra.run.smithi159.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-11T22:09:41.110 INFO:teuthology.orchestra.run.smithi172.stdout:(62/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 10 MB/s | 49 MB 00:04 2024-09-11T22:09:41.111 INFO:teuthology.orchestra.run.smithi159.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.2 MB 00:00 2024-09-11T22:09:41.126 INFO:teuthology.orchestra.run.smithi172.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 100 kB/s | 47 kB 00:00 2024-09-11T22:09:41.136 INFO:teuthology.orchestra.run.smithi159.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-11T22:09:41.160 INFO:teuthology.orchestra.run.smithi172.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 594 kB/s | 292 kB 00:00 2024-09-11T22:09:41.161 INFO:teuthology.orchestra.run.smithi159.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-11T22:09:41.177 INFO:teuthology.orchestra.run.smithi172.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-11T22:09:41.187 INFO:teuthology.orchestra.run.smithi159.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-11T22:09:41.213 INFO:teuthology.orchestra.run.smithi159.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-11T22:09:41.218 INFO:teuthology.orchestra.run.smithi005.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 685 kB/s | 662 kB 00:00 2024-09-11T22:09:41.319 INFO:teuthology.orchestra.run.smithi172.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 903 kB/s | 188 kB 00:00 2024-09-11T22:09:41.343 INFO:teuthology.orchestra.run.smithi005.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 42 kB/s | 24 kB 00:00 2024-09-11T22:09:41.413 INFO:teuthology.orchestra.run.smithi159.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 534 kB/s | 107 kB 00:00 2024-09-11T22:09:41.419 INFO:teuthology.orchestra.run.smithi172.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-11T22:09:41.480 INFO:teuthology.orchestra.run.smithi159.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-11T22:09:41.486 INFO:teuthology.orchestra.run.smithi172.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 15 MB/s | 4.9 MB 00:00 2024-09-11T22:09:41.496 INFO:teuthology.orchestra.run.smithi159.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 260 kB/s | 22 kB 00:00 2024-09-11T22:09:41.528 INFO:teuthology.orchestra.run.smithi172.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-11T22:09:41.547 INFO:teuthology.orchestra.run.smithi159.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-09-11T22:09:41.553 INFO:teuthology.orchestra.run.smithi172.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-11T22:09:41.564 INFO:teuthology.orchestra.run.smithi159.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-09-11T22:09:41.597 INFO:teuthology.orchestra.run.smithi159.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-11T22:09:41.614 INFO:teuthology.orchestra.run.smithi159.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-11T22:09:41.620 INFO:teuthology.orchestra.run.smithi172.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-11T22:09:41.640 INFO:teuthology.orchestra.run.smithi159.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 266 kB/s | 11 kB 00:00 2024-09-11T22:09:41.679 INFO:teuthology.orchestra.run.smithi172.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-11T22:09:41.681 INFO:teuthology.orchestra.run.smithi159.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-11T22:09:41.731 INFO:teuthology.orchestra.run.smithi159.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-11T22:09:41.760 INFO:teuthology.orchestra.run.smithi005.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 566 kB/s | 581 kB 00:01 2024-09-11T22:09:41.773 INFO:teuthology.orchestra.run.smithi159.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-11T22:09:41.811 INFO:teuthology.orchestra.run.smithi005.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 80 kB/s | 47 kB 00:00 2024-09-11T22:09:41.812 INFO:teuthology.orchestra.run.smithi172.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-11T22:09:41.824 INFO:teuthology.orchestra.run.smithi159.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 491 kB/s | 24 kB 00:00 2024-09-11T22:09:41.854 INFO:teuthology.orchestra.run.smithi172.stdout:(74/156): postgresql-13.16-1.el9.x86_64.rpm 3.0 MB/s | 1.6 MB 00:00 2024-09-11T22:09:41.857 INFO:teuthology.orchestra.run.smithi159.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-11T22:09:41.880 INFO:teuthology.orchestra.run.smithi172.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 915 kB/s | 61 kB 00:00 2024-09-11T22:09:41.883 INFO:teuthology.orchestra.run.smithi159.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 807 kB/s | 47 kB 00:00 2024-09-11T22:09:41.925 INFO:teuthology.orchestra.run.smithi159.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-09-11T22:09:41.947 INFO:teuthology.orchestra.run.smithi172.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-11T22:09:41.975 INFO:teuthology.orchestra.run.smithi159.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 773 kB/s | 38 kB 00:00 2024-09-11T22:09:41.980 INFO:teuthology.orchestra.run.smithi172.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-11T22:09:42.000 INFO:teuthology.orchestra.run.smithi159.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-11T22:09:42.006 INFO:teuthology.orchestra.run.smithi172.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-11T22:09:42.081 INFO:teuthology.orchestra.run.smithi172.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-11T22:09:42.106 INFO:teuthology.orchestra.run.smithi172.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 152 kB/s | 19 kB 00:00 2024-09-11T22:09:42.131 INFO:teuthology.orchestra.run.smithi172.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T22:09:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-09-11T22:09:42.169 INFO:teuthology.orchestra.run.smithi005.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 354 kB/s | 292 kB 00:00 2024-09-11T22:09:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-11T22:09:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-11T22:09:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-11T22:09:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 720 kB/s | 42 kB 00:00 2024-09-11T22:09:42.316 INFO:teuthology.orchestra.run.smithi172.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-11T22:09:42.342 INFO:teuthology.orchestra.run.smithi172.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-11T22:09:42.351 INFO:teuthology.orchestra.run.smithi159.stdout:(66/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-11T22:09:42.384 INFO:teuthology.orchestra.run.smithi172.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-11T22:09:42.386 INFO:teuthology.orchestra.run.smithi005.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 67 kB/s | 38 kB 00:00 2024-09-11T22:09:42.409 INFO:teuthology.orchestra.run.smithi172.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-09-11T22:09:42.434 INFO:teuthology.orchestra.run.smithi159.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-11T22:09:42.434 INFO:teuthology.orchestra.run.smithi172.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-11T22:09:42.493 INFO:teuthology.orchestra.run.smithi172.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-09-11T22:09:42.495 INFO:teuthology.orchestra.run.smithi005.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 257 kB/s | 188 kB 00:00 2024-09-11T22:09:42.501 INFO:teuthology.orchestra.run.smithi159.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 13 MB/s | 910 kB 00:00 2024-09-11T22:09:42.551 INFO:teuthology.orchestra.run.smithi172.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-09-11T22:09:42.552 INFO:teuthology.orchestra.run.smithi159.stdout:(69/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.0 MB/s | 3.0 MB 00:01 2024-09-11T22:09:42.585 INFO:teuthology.orchestra.run.smithi159.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-11T22:09:42.618 INFO:teuthology.orchestra.run.smithi172.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-11T22:09:42.652 INFO:teuthology.orchestra.run.smithi159.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-11T22:09:42.677 INFO:teuthology.orchestra.run.smithi172.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-09-11T22:09:42.702 INFO:teuthology.orchestra.run.smithi159.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-11T22:09:42.736 INFO:teuthology.orchestra.run.smithi172.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-09-11T22:09:42.811 INFO:teuthology.orchestra.run.smithi172.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.0 MB/s | 304 kB 00:00 2024-09-11T22:09:42.878 INFO:teuthology.orchestra.run.smithi172.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 870 kB/s | 58 kB 00:00 2024-09-11T22:09:42.886 INFO:teuthology.orchestra.run.smithi159.stdout:(73/156): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 6.0 MB 00:00 2024-09-11T22:09:42.936 INFO:teuthology.orchestra.run.smithi159.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-11T22:09:42.995 INFO:teuthology.orchestra.run.smithi159.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-11T22:09:43.037 INFO:teuthology.orchestra.run.smithi172.stdout:(99/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-11T22:09:43.062 INFO:teuthology.orchestra.run.smithi172.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 522 kB/s | 96 kB 00:00 2024-09-11T22:09:43.072 INFO:teuthology.orchestra.run.smithi159.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 3.1 MB/s | 248 kB 00:00 2024-09-11T22:09:43.129 INFO:teuthology.orchestra.run.smithi172.stdout:(101/156): python3-numpy-1.20.1-5.el9.x86_64.rp 5.0 MB/s | 5.1 MB 00:01 2024-09-11T22:09:43.129 INFO:teuthology.orchestra.run.smithi159.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 456 kB/s | 26 kB 00:00 2024-09-11T22:09:43.145 INFO:teuthology.orchestra.run.smithi172.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-09-11T22:09:43.162 INFO:teuthology.orchestra.run.smithi172.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 507 kB/s | 64 kB 00:00 2024-09-11T22:09:43.179 INFO:teuthology.orchestra.run.smithi159.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-11T22:09:43.213 INFO:teuthology.orchestra.run.smithi172.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T22:09:43.238 INFO:teuthology.orchestra.run.smithi159.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-11T22:09:43.247 INFO:teuthology.orchestra.run.smithi172.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-09-11T22:09:43.264 INFO:teuthology.orchestra.run.smithi172.stdout:(106/156): lua-devel-5.4.4-4.el9.x86_64.rpm 166 kB/s | 22 kB 00:00 2024-09-11T22:09:43.281 INFO:teuthology.orchestra.run.smithi172.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 581 kB/s | 19 kB 00:00 2024-09-11T22:09:43.288 INFO:teuthology.orchestra.run.smithi159.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-11T22:09:43.306 INFO:teuthology.orchestra.run.smithi172.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.4 MB/s | 26 kB 00:00 2024-09-11T22:09:43.322 INFO:teuthology.orchestra.run.smithi172.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 3.3 MB/s | 49 kB 00:00 2024-09-11T22:09:43.348 INFO:teuthology.orchestra.run.smithi172.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T22:09:43.373 INFO:teuthology.orchestra.run.smithi172.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-11T22:09:43.412 INFO:teuthology.orchestra.run.smithi005.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.0 MB/s | 4.9 MB 00:01 2024-09-11T22:09:43.423 INFO:teuthology.orchestra.run.smithi172.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-11T22:09:43.438 INFO:teuthology.orchestra.run.smithi159.stdout:(81/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-09-11T22:09:43.465 INFO:teuthology.orchestra.run.smithi172.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-11T22:09:43.491 INFO:teuthology.orchestra.run.smithi172.stdout:(114/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-09-11T22:09:43.497 INFO:teuthology.orchestra.run.smithi159.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-11T22:09:43.507 INFO:teuthology.orchestra.run.smithi172.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-09-11T22:09:43.533 INFO:teuthology.orchestra.run.smithi172.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 799 kB/s | 14 kB 00:00 2024-09-11T22:09:43.554 INFO:teuthology.orchestra.run.smithi005.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-09-11T22:09:43.555 INFO:teuthology.orchestra.run.smithi159.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-11T22:09:43.558 INFO:teuthology.orchestra.run.smithi172.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 479 kB/s | 32 kB 00:00 2024-09-11T22:09:43.583 INFO:teuthology.orchestra.run.smithi172.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.5 MB/s | 172 kB 00:00 2024-09-11T22:09:43.606 INFO:teuthology.orchestra.run.smithi159.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-11T22:09:43.617 INFO:teuthology.orchestra.run.smithi172.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-11T22:09:43.648 INFO:teuthology.orchestra.run.smithi159.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-11T22:09:43.650 INFO:teuthology.orchestra.run.smithi172.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-11T22:09:43.662 INFO:teuthology.orchestra.run.smithi005.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-09-11T22:09:43.676 INFO:teuthology.orchestra.run.smithi172.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-11T22:09:43.701 INFO:teuthology.orchestra.run.smithi172.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-11T22:09:43.706 INFO:teuthology.orchestra.run.smithi159.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-11T22:09:43.760 INFO:teuthology.orchestra.run.smithi172.stdout:(123/156): libarrow-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2024-09-11T22:09:43.765 INFO:teuthology.orchestra.run.smithi159.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-11T22:09:43.785 INFO:teuthology.orchestra.run.smithi172.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-09-11T22:09:43.810 INFO:teuthology.orchestra.run.smithi172.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-11T22:09:43.824 INFO:teuthology.orchestra.run.smithi159.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-11T22:09:43.835 INFO:teuthology.orchestra.run.smithi172.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-11T22:09:43.861 INFO:teuthology.orchestra.run.smithi172.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-11T22:09:43.874 INFO:teuthology.orchestra.run.smithi159.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-11T22:09:43.886 INFO:teuthology.orchestra.run.smithi172.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-11T22:09:43.911 INFO:teuthology.orchestra.run.smithi172.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T22:09:43.916 INFO:teuthology.orchestra.run.smithi159.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-09-11T22:09:43.937 INFO:teuthology.orchestra.run.smithi172.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T22:09:43.962 INFO:teuthology.orchestra.run.smithi172.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-11T22:09:44.004 INFO:teuthology.orchestra.run.smithi172.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-11T22:09:44.029 INFO:teuthology.orchestra.run.smithi172.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-11T22:09:44.054 INFO:teuthology.orchestra.run.smithi172.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-09-11T22:09:44.080 INFO:teuthology.orchestra.run.smithi172.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-11T22:09:44.105 INFO:teuthology.orchestra.run.smithi172.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-11T22:09:44.124 INFO:teuthology.orchestra.run.smithi159.stdout:(91/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.3 MB/s | 4.9 MB 00:02 2024-09-11T22:09:44.131 INFO:teuthology.orchestra.run.smithi172.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-11T22:09:44.156 INFO:teuthology.orchestra.run.smithi172.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-09-11T22:09:44.166 INFO:teuthology.orchestra.run.smithi159.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-11T22:09:44.181 INFO:teuthology.orchestra.run.smithi172.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-11T22:09:44.206 INFO:teuthology.orchestra.run.smithi172.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-11T22:09:44.231 INFO:teuthology.orchestra.run.smithi172.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-11T22:09:44.257 INFO:teuthology.orchestra.run.smithi172.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-11T22:09:44.282 INFO:teuthology.orchestra.run.smithi172.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-11T22:09:44.304 INFO:teuthology.orchestra.run.smithi005.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 183 kB/s | 137 kB 00:00 2024-09-11T22:09:44.333 INFO:teuthology.orchestra.run.smithi159.stdout:(93/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-09-11T22:09:44.341 INFO:teuthology.orchestra.run.smithi172.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.0 MB/s | 2.0 MB 00:00 2024-09-11T22:09:44.358 INFO:teuthology.orchestra.run.smithi172.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-09-11T22:09:44.359 INFO:teuthology.orchestra.run.smithi159.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 230 kB/s | 44 kB 00:00 2024-09-11T22:09:44.392 INFO:teuthology.orchestra.run.smithi159.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-11T22:09:44.408 INFO:teuthology.orchestra.run.smithi172.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-11T22:09:44.417 INFO:teuthology.orchestra.run.smithi159.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 610 kB/s | 36 kB 00:00 2024-09-11T22:09:44.429 INFO:teuthology.orchestra.run.smithi005.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 895 kB/s | 910 kB 00:01 2024-09-11T22:09:44.433 INFO:teuthology.orchestra.run.smithi172.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-09-11T22:09:44.443 INFO:teuthology.orchestra.run.smithi159.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-11T22:09:44.475 INFO:teuthology.orchestra.run.smithi172.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-11T22:09:44.493 INFO:teuthology.orchestra.run.smithi159.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-11T22:09:44.544 INFO:teuthology.orchestra.run.smithi159.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.9 MB/s | 96 kB 00:00 2024-09-11T22:09:44.586 INFO:teuthology.orchestra.run.smithi159.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 1.8 MB/s | 304 kB 00:00 2024-09-11T22:09:44.611 INFO:teuthology.orchestra.run.smithi159.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-09-11T22:09:44.636 INFO:teuthology.orchestra.run.smithi159.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-11T22:09:44.695 INFO:teuthology.orchestra.run.smithi159.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-09-11T22:09:44.709 INFO:teuthology.orchestra.run.smithi172.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-11T22:09:44.745 INFO:teuthology.orchestra.run.smithi159.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-11T22:09:44.759 INFO:teuthology.orchestra.run.smithi172.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 8.2 MB/s | 2.3 MB 00:00 2024-09-11T22:09:44.771 INFO:teuthology.orchestra.run.smithi159.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-11T22:09:44.796 INFO:teuthology.orchestra.run.smithi159.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-11T22:09:44.818 INFO:teuthology.orchestra.run.smithi172.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 700 kB/s | 41 kB 00:00 2024-09-11T22:09:44.893 INFO:teuthology.orchestra.run.smithi172.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 3.2 MB/s | 249 kB 00:00 2024-09-11T22:09:44.896 INFO:teuthology.orchestra.run.smithi159.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-11T22:09:44.930 INFO:teuthology.orchestra.run.smithi159.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-09-11T22:09:44.946 INFO:teuthology.orchestra.run.smithi005.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-09-11T22:09:44.951 INFO:teuthology.orchestra.run.smithi172.stdout:(153/156): librados2-19.1.1-298.gb53b6d3e.el9.x 6.3 MB/s | 3.4 MB 00:00 2024-09-11T22:09:44.955 INFO:teuthology.orchestra.run.smithi159.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-09-11T22:09:44.993 INFO:teuthology.orchestra.run.smithi172.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-11T22:09:45.005 INFO:teuthology.orchestra.run.smithi159.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.5 MB/s | 67 kB 00:00 2024-09-11T22:09:45.031 INFO:teuthology.orchestra.run.smithi159.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-11T22:09:45.035 INFO:teuthology.orchestra.run.smithi172.stdout:(155/156): NetworkManager-libnm-1.49.90-1.el9.x 5.5 MB/s | 1.8 MB 00:00 2024-09-11T22:09:45.056 INFO:teuthology.orchestra.run.smithi159.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 385 kB/s | 49 kB 00:00 2024-09-11T22:09:45.106 INFO:teuthology.orchestra.run.smithi159.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-11T22:09:45.131 INFO:teuthology.orchestra.run.smithi159.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-11T22:09:45.155 INFO:teuthology.orchestra.run.smithi005.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 120 kB/s | 87 kB 00:00 2024-09-11T22:09:45.165 INFO:teuthology.orchestra.run.smithi159.stdout:(115/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-09-11T22:09:45.190 INFO:teuthology.orchestra.run.smithi159.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-09-11T22:09:45.216 INFO:teuthology.orchestra.run.smithi159.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-11T22:09:45.239 INFO:teuthology.orchestra.run.smithi005.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.1 MB/s | 1.0 MB 00:00 2024-09-11T22:09:45.241 INFO:teuthology.orchestra.run.smithi159.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-11T22:09:45.274 INFO:teuthology.orchestra.run.smithi159.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-11T22:09:45.300 INFO:teuthology.orchestra.run.smithi159.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-11T22:09:45.325 INFO:teuthology.orchestra.run.smithi159.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-11T22:09:45.335 INFO:teuthology.orchestra.run.smithi172.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 6.2 MB/s | 2.4 MB 00:00 2024-09-11T22:09:45.343 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:09:45.343 INFO:teuthology.orchestra.run.smithi172.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-11T22:09:45.359 INFO:teuthology.orchestra.run.smithi159.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-11T22:09:45.392 INFO:teuthology.orchestra.run.smithi159.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 322 kB/s | 11 kB 00:00 2024-09-11T22:09:45.418 INFO:teuthology.orchestra.run.smithi159.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-11T22:09:45.443 INFO:teuthology.orchestra.run.smithi159.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 927 kB/s | 23 kB 00:00 2024-09-11T22:09:45.469 INFO:teuthology.orchestra.run.smithi159.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 772 kB/s | 19 kB 00:00 2024-09-11T22:09:45.493 INFO:teuthology.orchestra.run.smithi159.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-09-11T22:09:45.544 INFO:teuthology.orchestra.run.smithi159.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-11T22:09:45.569 INFO:teuthology.orchestra.run.smithi159.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T22:09:45.595 INFO:teuthology.orchestra.run.smithi159.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-11T22:09:45.620 INFO:teuthology.orchestra.run.smithi159.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-11T22:09:45.662 INFO:teuthology.orchestra.run.smithi159.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-11T22:09:45.679 INFO:teuthology.orchestra.run.smithi159.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 973 kB/s | 16 kB 00:00 2024-09-11T22:09:45.704 INFO:teuthology.orchestra.run.smithi159.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-11T22:09:45.729 INFO:teuthology.orchestra.run.smithi159.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-11T22:09:45.746 INFO:teuthology.orchestra.run.smithi159.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 673 kB/s | 28 kB 00:00 2024-09-11T22:09:45.763 INFO:teuthology.orchestra.run.smithi159.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 918 kB/s | 31 kB 00:00 2024-09-11T22:09:45.789 INFO:teuthology.orchestra.run.smithi159.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T22:09:45.814 INFO:teuthology.orchestra.run.smithi159.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-09-11T22:09:45.839 INFO:teuthology.orchestra.run.smithi005.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 90 kB/s | 61 kB 00:00 2024-09-11T22:09:45.839 INFO:teuthology.orchestra.run.smithi159.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-11T22:09:45.865 INFO:teuthology.orchestra.run.smithi159.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-11T22:09:45.898 INFO:teuthology.orchestra.run.smithi159.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-09-11T22:09:45.924 INFO:teuthology.orchestra.run.smithi159.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-11T22:09:45.949 INFO:teuthology.orchestra.run.smithi159.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-09-11T22:09:45.982 INFO:teuthology.orchestra.run.smithi159.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-11T22:09:45.997 INFO:teuthology.orchestra.run.smithi005.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 323 kB/s | 245 kB 00:00 2024-09-11T22:09:46.007 INFO:teuthology.orchestra.run.smithi159.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-11T22:09:46.058 INFO:teuthology.orchestra.run.smithi159.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-11T22:09:46.217 INFO:teuthology.orchestra.run.smithi159.stdout:(148/156): postgresql-server-13.16-1.el9.x86_64 1.6 MB/s | 5.8 MB 00:03 2024-09-11T22:09:46.248 INFO:teuthology.orchestra.run.smithi005.stdout:(77/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.6 MB/s | 6.0 MB 00:01 2024-09-11T22:09:46.325 INFO:teuthology.orchestra.run.smithi159.stdout:(149/156): librados2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-11T22:09:46.459 INFO:teuthology.orchestra.run.smithi159.stdout:(150/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 7.9 MB/s | 3.2 MB 00:00 2024-09-11T22:09:46.526 INFO:teuthology.orchestra.run.smithi159.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 7.5 MB/s | 2.3 MB 00:00 2024-09-11T22:09:46.576 INFO:teuthology.orchestra.run.smithi159.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 348 kB/s | 41 kB 00:00 2024-09-11T22:09:46.578 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-09-11T22:09:46.601 INFO:teuthology.orchestra.run.smithi159.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 3.2 MB/s | 249 kB 00:00 2024-09-11T22:09:46.615 INFO:teuthology.orchestra.run.smithi005.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 42 kB/s | 26 kB 00:00 2024-09-11T22:09:46.640 INFO:teuthology.orchestra.run.smithi005.stdout:(79/156): python3-jinja2-2.11.3-6.el9.noarch.rp 310 kB/s | 248 kB 00:00 2024-09-11T22:09:46.643 INFO:teuthology.orchestra.run.smithi159.stdout:(154/156): NetworkManager-libnm-1.49.90-1.el9.x 5.7 MB/s | 1.8 MB 00:00 2024-09-11T22:09:46.729 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-09-11T22:09:46.729 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-09-11T22:09:46.752 INFO:teuthology.orchestra.run.smithi159.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-11T22:09:46.804 INFO:teuthology.orchestra.run.smithi005.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 35 kB/s | 19 kB 00:00 2024-09-11T22:09:46.877 INFO:teuthology.orchestra.run.smithi159.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 4.5 MB/s | 1.4 MB 00:00 2024-09-11T22:09:46.882 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:09:46.882 INFO:teuthology.orchestra.run.smithi159.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-11T22:09:47.224 INFO:teuthology.orchestra.run.smithi005.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 60 kB/s | 35 kB 00:00 2024-09-11T22:09:47.449 INFO:teuthology.orchestra.run.smithi005.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 206 kB/s | 172 kB 00:00 2024-09-11T22:09:47.924 INFO:teuthology.orchestra.run.smithi005.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-09-11T22:09:48.000 INFO:teuthology.orchestra.run.smithi005.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 323 kB/s | 250 kB 00:00 2024-09-11T22:09:48.119 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-11T22:09:48.267 INFO:teuthology.orchestra.run.smithi005.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 275 kB/s | 222 kB 00:00 2024-09-11T22:09:48.273 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-11T22:09:48.273 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-11T22:09:48.543 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-09-11T22:09:48.543 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-09-11T22:09:48.575 INFO:teuthology.orchestra.run.smithi005.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 73 kB/s | 42 kB 00:00 2024-09-11T22:09:48.634 INFO:teuthology.orchestra.run.smithi005.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 109 kB/s | 77 kB 00:00 2024-09-11T22:09:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 302 kB/s | 269 kB 00:00 2024-09-11T22:09:49.284 INFO:teuthology.orchestra.run.smithi005.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 224 kB/s | 159 kB 00:00 2024-09-11T22:09:49.485 INFO:teuthology.orchestra.run.smithi005.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 328 kB/s | 279 kB 00:00 2024-09-11T22:09:49.827 INFO:teuthology.orchestra.run.smithi005.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 76 kB/s | 51 kB 00:00 2024-09-11T22:09:49.952 INFO:teuthology.orchestra.run.smithi005.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 80 kB/s | 54 kB 00:00 2024-09-11T22:09:50.093 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-11T22:09:50.093 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-11T22:09:50.394 INFO:teuthology.orchestra.run.smithi005.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 73 kB/s | 42 kB 00:00 2024-09-11T22:09:50.594 INFO:teuthology.orchestra.run.smithi005.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 69 kB/s | 44 kB 00:00 2024-09-11T22:09:50.978 INFO:teuthology.orchestra.run.smithi005.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-11T22:09:51.320 INFO:teuthology.orchestra.run.smithi005.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 239 kB/s | 221 kB 00:00 2024-09-11T22:09:51.403 INFO:teuthology.orchestra.run.smithi005.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 44 kB/s | 36 kB 00:00 2024-09-11T22:09:51.620 INFO:teuthology.orchestra.run.smithi005.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 75 kB/s | 47 kB 00:00 2024-09-11T22:09:52.062 INFO:teuthology.orchestra.run.smithi005.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 58 kB 00:00 2024-09-11T22:09:52.237 INFO:teuthology.orchestra.run.smithi005.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 332 kB/s | 304 kB 00:00 2024-09-11T22:09:52.354 INFO:teuthology.orchestra.run.smithi005.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 131 kB/s | 96 kB 00:00 2024-09-11T22:09:52.638 INFO:teuthology.orchestra.run.smithi005.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 79 kB/s | 22 kB 00:00 2024-09-11T22:09:52.763 INFO:teuthology.orchestra.run.smithi005.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 91 kB/s | 64 kB 00:00 2024-09-11T22:09:52.822 INFO:teuthology.orchestra.run.smithi005.stdout:(104/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 77 kB/s | 45 kB 00:00 2024-09-11T22:09:52.872 INFO:teuthology.orchestra.run.smithi005.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 4.9 MB/s | 548 kB 00:00 2024-09-11T22:09:52.897 INFO:teuthology.orchestra.run.smithi005.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 857 kB/s | 19 kB 00:00 2024-09-11T22:09:52.932 INFO:teuthology.orchestra.run.smithi005.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-09-11T22:09:52.956 INFO:teuthology.orchestra.run.smithi005.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-11T22:09:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-11T22:09:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T22:09:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-11T22:09:53.308 INFO:teuthology.orchestra.run.smithi005.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-11T22:09:53.391 INFO:teuthology.orchestra.run.smithi005.stdout:(113/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-09-11T22:09:53.425 INFO:teuthology.orchestra.run.smithi005.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.2 MB/s | 839 kB 00:00 2024-09-11T22:09:53.450 INFO:teuthology.orchestra.run.smithi005.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-11T22:09:53.467 INFO:teuthology.orchestra.run.smithi005.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 977 kB/s | 14 kB 00:00 2024-09-11T22:09:53.484 INFO:teuthology.orchestra.run.smithi005.stdout:(117/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 471 kB/s | 43 kB 00:00 2024-09-11T22:09:53.509 INFO:teuthology.orchestra.run.smithi005.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-11T22:09:53.543 INFO:teuthology.orchestra.run.smithi005.stdout:(119/156): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-09-11T22:09:53.568 INFO:teuthology.orchestra.run.smithi005.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-11T22:09:53.594 INFO:teuthology.orchestra.run.smithi005.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-11T22:09:53.627 INFO:teuthology.orchestra.run.smithi005.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-09-11T22:09:53.652 INFO:teuthology.orchestra.run.smithi005.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-11T22:09:53.678 INFO:teuthology.orchestra.run.smithi005.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-09-11T22:09:53.703 INFO:teuthology.orchestra.run.smithi005.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-11T22:09:53.729 INFO:teuthology.orchestra.run.smithi005.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-11T22:09:53.754 INFO:teuthology.orchestra.run.smithi005.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-11T22:09:53.779 INFO:teuthology.orchestra.run.smithi005.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-11T22:09:53.804 INFO:teuthology.orchestra.run.smithi005.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-11T22:09:53.838 INFO:teuthology.orchestra.run.smithi005.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-11T22:09:53.880 INFO:teuthology.orchestra.run.smithi005.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-09-11T22:09:53.905 INFO:teuthology.orchestra.run.smithi005.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 864 kB/s | 58 kB 00:00 2024-09-11T22:09:53.930 INFO:teuthology.orchestra.run.smithi005.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 664 kB/s | 16 kB 00:00 2024-09-11T22:09:53.964 INFO:teuthology.orchestra.run.smithi005.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-11T22:09:53.989 INFO:teuthology.orchestra.run.smithi005.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-11T22:09:54.015 INFO:teuthology.orchestra.run.smithi005.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-11T22:09:54.040 INFO:teuthology.orchestra.run.smithi005.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-11T22:09:54.082 INFO:teuthology.orchestra.run.smithi005.stdout:(138/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-11T22:09:54.107 INFO:teuthology.orchestra.run.smithi005.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 883 kB/s | 59 kB 00:00 2024-09-11T22:09:54.132 INFO:teuthology.orchestra.run.smithi005.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-11T22:09:54.158 INFO:teuthology.orchestra.run.smithi005.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-11T22:09:54.183 INFO:teuthology.orchestra.run.smithi005.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-11T22:09:54.208 INFO:teuthology.orchestra.run.smithi005.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-11T22:09:54.233 INFO:teuthology.orchestra.run.smithi005.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-11T22:09:54.259 INFO:teuthology.orchestra.run.smithi005.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-11T22:09:54.284 INFO:teuthology.orchestra.run.smithi005.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-11T22:09:54.309 INFO:teuthology.orchestra.run.smithi005.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-11T22:09:54.385 INFO:teuthology.orchestra.run.smithi005.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-11T22:09:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-11T22:09:54.719 INFO:teuthology.orchestra.run.smithi005.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 7.9 MB/s | 3.4 MB 00:00 2024-09-11T22:09:54.765 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-09-11T22:09:54.777 INFO:teuthology.orchestra.run.smithi005.stdout:(151/156): NetworkManager-1.49.90-1.el9.x86_64. 5.9 MB/s | 2.3 MB 00:00 2024-09-11T22:09:54.819 INFO:teuthology.orchestra.run.smithi005.stdout:(152/156): NetworkManager-libnm-1.49.90-1.el9.x 8.7 MB/s | 1.8 MB 00:00 2024-09-11T22:09:54.853 INFO:teuthology.orchestra.run.smithi005.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 3.2 MB/s | 249 kB 00:00 2024-09-11T22:09:54.870 INFO:teuthology.orchestra.run.smithi005.stdout:(154/156): NetworkManager-team-1.49.90-1.el9.x8 270 kB/s | 41 kB 00:00 2024-09-11T22:09:54.872 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T22:09:54.911 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T22:09:54.912 INFO:teuthology.orchestra.run.smithi005.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-11T22:09:54.979 INFO:teuthology.orchestra.run.smithi005.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-09-11T22:09:54.986 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:09:54.986 INFO:teuthology.orchestra.run.smithi005.stdout:Total 9.8 MB/s | 226 MB 00:22 2024-09-11T22:09:55.173 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T22:09:55.216 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T22:09:55.241 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T22:09:55.285 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T22:09:55.614 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T22:09:55.733 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:09:55.855 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:09:55.892 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:09:55.984 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:09:56.022 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:09:56.171 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T22:09:56.235 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T22:09:56.251 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-09-11T22:09:56.263 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:09:56.367 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:09:56.397 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-09-11T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-09-11T22:09:56.443 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:09:56.465 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:09:56.486 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:09:56.548 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-11T22:09:56.639 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T22:09:56.762 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T22:09:56.934 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:09:56.970 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T22:09:57.008 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T22:09:57.032 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T22:09:57.085 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T22:09:57.093 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:09:57.221 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T22:09:57.315 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:09:57.398 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:09:57.425 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T22:09:57.525 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:09:57.634 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:09:57.655 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:09:57.684 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:09:57.749 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T22:09:57.775 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:09:57.805 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:09:57.843 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T22:09:57.880 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T22:09:57.915 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T22:09:57.956 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T22:09:57.988 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T22:09:58.011 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T22:09:58.041 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T22:09:58.047 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:09:58.073 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T22:09:58.117 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T22:09:58.133 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:09:58.149 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T22:09:58.156 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:09:58.177 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T22:09:58.202 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:09:58.224 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:09:58.246 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T22:09:58.248 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:09:58.300 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T22:09:58.339 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T22:09:58.382 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T22:09:58.442 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:09:58.458 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T22:09:58.493 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T22:09:58.536 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T22:09:58.565 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T22:09:58.607 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:09:58.635 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T22:09:58.688 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T22:09:58.731 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T22:09:58.737 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-09-11T22:09:58.737 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-09-11T22:09:58.762 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T22:09:58.837 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:09:58.904 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T22:09:58.932 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:09:58.956 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T22:09:59.159 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:09:59.261 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T22:09:59.358 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T22:09:59.387 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T22:09:59.430 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T22:09:59.485 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T22:09:59.504 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T22:09:59.560 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T22:09:59.569 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T22:09:59.591 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T22:09:59.606 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T22:09:59.635 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T22:09:59.673 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T22:09:59.673 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T22:09:59.687 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T22:09:59.696 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T22:09:59.736 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T22:09:59.774 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T22:09:59.836 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T22:09:59.866 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T22:09:59.910 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T22:09:59.910 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T22:09:59.973 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T22:09:59.988 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T22:10:00.021 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T22:10:00.027 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T22:10:00.065 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T22:10:00.096 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T22:10:00.101 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T22:10:00.132 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T22:10:00.168 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T22:10:00.173 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T22:10:00.215 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T22:10:00.225 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T22:10:00.251 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T22:10:00.302 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T22:10:00.308 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T22:10:00.352 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T22:10:00.454 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T22:10:00.491 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T22:10:00.519 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T22:10:00.638 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T22:10:00.772 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T22:10:00.877 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T22:10:00.913 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T22:10:00.952 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T22:10:00.978 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T22:10:01.003 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T22:10:01.060 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T22:10:01.102 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T22:10:01.146 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T22:10:01.176 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T22:10:01.219 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T22:10:01.229 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T22:10:01.257 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T22:10:01.278 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T22:10:01.298 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T22:10:01.306 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T22:10:01.340 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T22:10:01.380 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T22:10:01.420 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T22:10:01.449 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T22:10:01.458 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T22:10:01.493 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T22:10:01.520 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T22:10:01.534 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T22:10:01.575 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T22:10:01.583 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T22:10:01.610 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T22:10:01.656 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T22:10:01.688 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T22:10:01.697 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T22:10:01.724 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T22:10:01.748 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T22:10:01.770 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T22:10:01.780 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T22:10:01.806 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T22:10:01.850 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T22:10:01.892 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T22:10:01.892 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T22:10:01.948 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T22:10:02.031 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T22:10:02.211 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T22:10:02.252 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T22:10:02.357 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T22:10:02.394 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T22:10:02.434 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T22:10:02.459 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T22:10:02.484 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T22:10:02.562 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T22:10:02.683 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T22:10:02.686 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T22:10:02.728 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T22:10:02.741 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T22:10:02.762 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T22:10:02.769 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T22:10:02.803 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T22:10:02.852 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T22:10:02.861 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T22:10:02.891 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T22:10:02.917 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T22:10:02.921 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T22:10:02.956 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T22:10:02.961 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T22:10:02.989 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T22:10:03.031 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T22:10:03.057 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T22:10:03.065 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T22:10:03.098 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T22:10:03.152 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T22:10:03.195 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T22:10:03.235 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T22:10:03.283 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T22:10:03.312 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T22:10:03.340 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T22:10:03.395 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T22:10:03.398 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T22:10:04.045 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T22:10:04.133 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T22:10:04.160 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T22:10:04.183 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T22:10:04.225 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T22:10:04.333 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T22:10:04.397 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T22:10:04.442 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T22:10:04.529 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T22:10:04.561 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T22:10:04.746 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T22:10:04.792 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T22:10:04.895 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T22:10:05.346 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-09-11T22:10:05.392 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T22:10:05.450 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-11T22:10:05.490 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T22:10:05.518 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T22:10:05.539 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-11T22:10:05.551 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T22:10:05.557 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T22:10:05.651 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-11T22:10:05.685 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T22:10:05.709 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-11T22:10:05.743 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-11T22:10:05.791 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-11T22:10:05.862 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T22:10:05.906 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:10:05.997 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:10:06.028 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T22:10:06.072 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T22:10:06.146 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-11T22:10:06.252 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:10:06.361 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:10:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:10:06.479 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:10:06.508 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:10:06.549 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T22:10:06.576 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:10:06.647 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:10:06.677 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T22:10:06.684 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-11T22:10:06.741 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-11T22:10:06.768 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:10:06.875 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:10:06.903 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:10:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T22:10:06.966 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-11T22:10:06.996 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:10:07.014 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T22:10:07.021 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-11T22:10:07.042 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T22:10:07.075 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T22:10:07.216 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:10:07.381 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-11T22:10:07.385 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T22:10:07.430 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:10:07.519 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-11T22:10:07.529 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:10:07.560 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T22:10:07.596 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T22:10:07.635 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:10:07.712 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:10:07.930 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-11T22:10:08.046 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-11T22:10:08.076 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T22:10:08.108 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:10:08.152 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-11T22:10:08.179 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:10:08.187 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-11T22:10:08.210 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T22:10:08.228 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-11T22:10:08.304 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-11T22:10:08.354 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-11T22:10:08.394 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-11T22:10:08.438 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-11T22:10:08.479 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-11T22:10:08.514 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-11T22:10:08.586 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-11T22:10:08.646 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-11T22:10:08.676 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-11T22:10:08.719 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-11T22:10:08.799 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-11T22:10:08.838 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-11T22:10:08.883 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-11T22:10:08.910 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-11T22:10:08.993 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-11T22:10:09.022 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:10:09.042 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-11T22:10:09.057 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:10:09.121 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-11T22:10:09.267 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-11T22:10:09.321 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-11T22:10:09.607 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:10:09.644 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:10:09.727 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:10:09.727 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T22:10:09.727 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T22:10:09.727 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:09.762 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:10:09.883 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-11T22:10:09.968 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-11T22:10:10.002 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-11T22:10:10.063 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-11T22:10:10.100 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-11T22:10:10.148 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-11T22:10:10.320 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-11T22:10:10.379 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-11T22:10:10.435 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-11T22:10:10.510 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-11T22:10:10.538 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:10:10.554 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-11T22:10:10.581 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:10:10.591 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-11T22:10:10.639 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-11T22:10:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-11T22:10:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-11T22:10:10.776 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-11T22:10:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-11T22:10:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-11T22:10:11.003 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-11T22:10:11.110 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-11T22:10:11.132 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:10:11.144 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-11T22:10:11.177 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:10:11.185 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-11T22:10:11.226 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-11T22:10:11.252 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:10:11.253 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T22:10:11.253 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T22:10:11.253 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-11T22:10:11.286 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:10:11.357 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-11T22:10:11.492 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-11T22:10:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-11T22:10:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-11T22:10:11.621 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-11T22:10:11.660 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-11T22:10:11.700 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-11T22:10:11.730 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-11T22:10:11.764 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-11T22:10:11.797 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-11T22:10:11.847 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-11T22:10:11.890 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-11T22:10:11.980 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-11T22:10:12.030 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-11T22:10:12.060 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-11T22:10:12.169 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-11T22:10:12.221 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-11T22:10:12.998 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-11T22:10:13.046 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-11T22:10:13.087 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-11T22:10:13.199 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-11T22:10:13.287 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-11T22:10:13.338 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-11T22:10:13.429 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-11T22:10:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-11T22:10:13.665 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-11T22:10:13.704 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-11T22:10:13.785 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-11T22:10:14.489 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-11T22:10:14.616 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-11T22:10:15.945 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-11T22:10:16.020 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-11T22:10:16.048 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-11T22:10:16.081 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-11T22:10:16.408 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-11T22:10:16.451 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:10:16.544 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-11T22:10:16.585 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-11T22:10:16.627 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-11T22:10:17.132 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-11T22:10:17.172 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:10:17.243 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-11T22:10:17.273 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-11T22:10:19.657 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:10:19.686 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:10:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-11T22:10:20.315 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:10:20.404 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-11T22:10:20.404 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T22:10:20.404 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T22:10:20.404 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:20.441 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:10:21.591 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:10:21.591 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2024-09-11T22:10:21.591 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2024-09-11T22:10:21.592 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2024-09-11T22:10:21.592 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2024-09-11T22:10:21.592 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2024-09-11T22:10:21.592 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /root 2024-09-11T22:10:21.592 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2024-09-11T22:10:21.592 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:21.772 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:10:21.790 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:10:21.790 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:21.790 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T22:10:21.790 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:23.237 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:10:23.237 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2024-09-11T22:10:23.237 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2024-09-11T22:10:23.237 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2024-09-11T22:10:23.237 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2024-09-11T22:10:23.238 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2024-09-11T22:10:23.238 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /root 2024-09-11T22:10:23.238 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2024-09-11T22:10:23.238 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:23.275 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:10:23.302 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:10:23.302 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:23.302 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T22:10:23.302 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:10:23.302 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:10:23.302 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:23.424 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:10:23.440 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:10:23.440 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:23.440 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T22:10:23.440 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:23.525 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:10:23.549 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:10:23.549 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:23.549 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T22:10:23.550 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:10:23.550 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:10:23.550 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:23.993 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:10:24.017 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:10:24.017 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:24.017 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T22:10:24.017 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:10:24.017 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:10:24.017 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:24.131 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T22:10:24.188 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T22:10:24.243 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T22:10:24.430 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T22:10:24.485 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T22:10:24.563 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T22:10:24.660 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T22:10:24.809 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T22:10:24.844 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T22:10:24.875 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T22:10:24.901 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:10:24.911 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T22:10:24.929 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:10:24.929 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:24.929 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T22:10:24.929 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:10:24.929 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:10:24.929 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:25.152 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:10:25.176 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:10:25.176 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:25.176 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T22:10:25.176 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:10:25.176 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:10:25.176 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:25.379 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:10:25.410 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:10:25.441 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T22:10:25.504 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T22:10:25.533 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:10:25.611 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:10:25.635 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:10:25.635 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:25.635 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T22:10:25.635 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:10:25.635 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:10:25.636 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:25.750 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T22:10:25.807 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T22:10:25.871 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T22:10:26.067 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T22:10:26.112 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T22:10:26.190 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T22:10:26.280 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T22:10:26.426 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T22:10:26.479 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T22:10:26.510 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T22:10:26.538 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T22:10:26.586 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:10:26.634 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:10:26.989 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:10:27.020 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:10:27.052 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T22:10:27.114 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T22:10:27.143 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:10:27.604 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:10:27.635 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:10:27.740 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:10:27.905 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T22:10:27.937 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:10:27.961 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:10:27.961 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:27.961 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T22:10:27.961 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:10:27.961 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:10:27.961 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:28.013 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:10:28.025 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:10:28.123 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T22:10:28.169 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T22:10:28.194 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:10:28.228 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:10:29.095 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T22:10:29.132 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:10:29.156 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:10:29.156 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:29.156 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T22:10:29.156 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:10:29.156 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:10:29.157 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:29.193 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:10:29.207 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:10:29.220 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:10:29.231 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:10:29.231 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:29.231 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T22:10:29.231 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:29.283 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:10:29.307 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:10:29.307 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:29.307 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T22:10:29.307 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:10:29.307 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:10:29.307 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:29.322 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:10:29.485 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T22:10:29.514 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:10:29.540 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:10:29.540 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:29.540 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T22:10:29.540 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:10:29.540 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:10:29.540 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:29.590 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:10:29.602 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:10:29.617 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:10:29.641 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:10:29.641 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:29.641 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T22:10:29.641 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:10:29.641 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:10:29.642 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:29.699 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T22:10:29.747 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T22:10:30.669 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T22:10:30.692 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:10:30.716 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:10:30.716 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:30.717 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T22:10:30.717 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:10:30.717 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:10:30.717 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:30.760 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:10:30.783 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:10:30.783 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:30.783 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T22:10:30.783 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:30.827 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:10:30.851 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:10:30.851 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:30.851 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T22:10:30.851 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:10:30.851 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:10:30.851 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:31.161 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:10:31.185 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:10:31.185 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:31.185 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T22:10:31.185 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:10:31.185 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:10:31.185 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-11T22:10:32.658 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-09-11T22:10:32.658 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-09-11T22:10:32.658 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-09-11T22:10:32.658 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-09-11T22:10:32.658 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-09-11T22:10:32.658 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-09-11T22:10:32.659 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-09-11T22:10:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:10:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-11T22:10:32.874 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:32.874 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T22:10:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:34.212 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T22:10:34.253 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T22:10:34.294 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T22:10:34.343 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T22:10:34.370 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T22:10:34.418 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:10:34.446 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-11T22:10:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T22:10:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:10:34.446 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T22:10:34.446 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:34.555 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T22:10:34.608 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T22:10:34.639 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T22:10:34.669 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T22:10:34.685 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:10:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-11T22:10:34.710 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:34.710 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T22:10:34.710 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:10:34.710 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T22:10:34.710 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:34.714 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T22:10:35.177 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:10:35.202 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-11T22:10:35.202 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:35.202 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T22:10:35.202 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:10:35.203 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T22:10:35.203 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:35.308 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-11T22:10:35.323 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:10:35.362 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:10:35.366 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-11T22:10:35.383 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:10:35.439 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-11T22:10:35.642 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-11T22:10:35.696 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-11T22:10:35.723 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T22:10:35.764 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T22:10:35.776 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-11T22:10:35.805 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T22:10:35.837 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T22:10:35.864 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T22:10:35.875 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-11T22:10:36.034 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-11T22:10:36.044 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T22:10:36.088 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-11T22:10:36.095 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T22:10:36.125 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T22:10:36.127 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-11T22:10:36.154 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T22:10:36.155 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-11T22:10:36.192 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T22:10:36.232 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T22:10:36.325 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T22:10:36.366 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T22:10:36.448 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T22:10:36.478 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T22:10:36.503 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T22:10:36.529 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T22:10:36.529 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:10:36.541 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:10:36.579 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-11T22:10:36.611 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-11T22:10:36.622 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:10:36.622 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:10:36.684 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-11T22:10:36.706 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:10:36.718 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:10:36.731 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T22:10:36.737 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:10:36.737 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:10:36.757 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:10:36.798 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:10:36.819 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:10:36.907 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:10:36.908 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:10:37.000 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:10:37.071 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:10:37.657 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T22:10:37.737 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T22:10:37.778 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T22:10:37.825 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-11T22:10:37.858 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T22:10:37.862 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:10:37.890 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T22:10:37.915 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T22:10:37.941 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T22:10:37.941 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:10:38.034 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:10:38.034 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:10:38.101 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:10:38.126 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T22:10:38.131 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:10:38.132 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:10:38.311 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:10:38.311 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:10:38.395 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:10:38.467 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:10:38.881 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-11T22:10:38.912 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:10:39.023 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-11T22:10:39.191 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-11T22:10:39.232 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:10:39.257 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-11T22:10:39.257 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:39.258 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T22:10:39.258 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:10:39.258 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T22:10:39.258 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:39.316 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:10:39.328 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-11T22:10:39.429 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-11T22:10:39.488 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-11T22:10:40.467 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-11T22:10:40.493 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:10:40.518 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-11T22:10:40.519 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:40.519 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T22:10:40.519 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:10:40.519 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T22:10:40.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:40.561 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:10:40.585 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-11T22:10:40.585 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:40.586 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T22:10:40.586 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:40.636 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:10:40.661 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-11T22:10:40.661 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:40.662 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-11T22:10:40.662 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:10:40.662 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-11T22:10:40.662 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:40.978 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:10:41.004 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-11T22:10:41.004 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T22:10:41.004 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T22:10:41.004 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:10:41.004 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T22:10:41.004 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:10:42.654 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T22:10:42.655 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T22:10:42.656 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T22:10:42.657 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T22:10:42.658 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T22:10:42.659 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T22:10:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T22:10:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T22:10:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T22:10:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T22:10:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T22:10:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T22:10:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T22:10:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T22:10:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T22:10:42.660 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T22:10:42.661 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T22:10:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T22:10:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T22:10:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T22:10:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T22:10:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T22:10:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T22:10:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T22:10:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T22:10:42.662 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T22:10:42.663 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T22:10:42.664 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T22:10:42.665 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T22:10:42.666 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T22:10:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T22:10:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T22:10:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T22:10:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T22:10:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T22:10:42.667 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T22:10:44.246 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:10:44.247 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T22:10:44.247 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T22:10:44.247 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T22:10:44.247 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T22:10:44.247 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T22:10:44.247 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T22:10:44.247 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T22:10:44.247 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:10:44.247 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:10:44.247 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T22:10:44.248 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T22:10:44.249 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T22:10:44.249 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T22:10:44.249 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T22:10:44.249 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T22:10:44.249 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T22:10:44.249 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T22:10:44.249 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T22:10:44.261 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T22:10:44.261 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T22:10:44.261 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T22:10:44.261 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T22:10:44.261 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T22:10:44.261 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T22:10:44.261 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T22:10:44.261 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T22:10:44.261 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T22:10:44.261 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T22:10:44.262 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T22:10:44.263 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T22:10:44.264 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T22:10:44.265 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T22:10:44.266 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T22:10:44.267 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T22:10:44.268 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T22:10:44.269 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T22:10:44.270 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T22:10:44.271 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T22:10:44.271 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T22:10:44.271 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T22:10:44.271 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T22:10:44.271 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T22:10:45.117 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-09-11T22:10:45.118 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:45.119 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.120 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.121 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:45.121 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T22:10:45.121 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T22:10:45.121 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T22:10:45.121 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T22:10:45.121 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T22:10:45.121 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T22:10:45.121 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T22:10:45.121 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.122 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T22:10:45.123 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T22:10:45.124 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T22:10:45.125 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T22:10:45.126 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T22:10:45.127 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T22:10:45.128 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T22:10:45.129 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T22:10:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T22:10:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T22:10:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T22:10:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T22:10:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T22:10:45.130 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T22:10:45.131 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:10:45.131 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-09-11T22:10:45.547 DEBUG:teuthology.parallel:result is None 2024-09-11T22:10:45.866 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-11T22:10:45.916 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-11T22:10:45.957 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-11T22:10:45.989 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-11T22:10:46.016 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-11T22:10:46.231 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-11T22:10:46.271 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-11T22:10:46.302 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-11T22:10:46.331 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-11T22:10:46.378 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-11T22:10:46.848 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:10:46.848 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T22:10:46.849 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:46.850 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:46.851 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T22:10:46.852 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.853 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T22:10:46.854 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T22:10:46.855 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T22:10:46.856 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T22:10:46.857 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T22:10:46.858 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T22:10:46.859 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.860 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.861 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:46.861 INFO:teuthology.orchestra.run.smithi159.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T22:10:46.861 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T22:10:46.861 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T22:10:46.861 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T22:10:46.861 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T22:10:46.861 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T22:10:46.861 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T22:10:46.862 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:10:46.862 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-11T22:10:46.990 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:10:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:10:47.048 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-11T22:10:47.250 DEBUG:teuthology.parallel:result is None 2024-09-11T22:10:47.959 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-11T22:10:48.064 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-11T22:10:48.113 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-11T22:10:48.216 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-11T22:10:48.251 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-11T22:10:48.277 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-11T22:10:48.303 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-11T22:10:48.303 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:10:48.403 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-11T22:10:48.403 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:10:48.496 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-11T22:10:48.521 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-11T22:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:10:48.527 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:10:48.714 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-11T22:10:48.714 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:10:48.798 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-11T22:10:48.875 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:10:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:10:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-11T22:10:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-11T22:10:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-11T22:10:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-11T22:10:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-11T22:10:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-11T22:10:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-11T22:10:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-11T22:10:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-11T22:10:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-11T22:10:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-11T22:10:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-11T22:10:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-11T22:10:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-11T22:10:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-11T22:10:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-11T22:10:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-11T22:10:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-11T22:10:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-11T22:10:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-11T22:10:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-11T22:10:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-11T22:10:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-11T22:10:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-11T22:10:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-11T22:10:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-11T22:10:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-11T22:10:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-11T22:10:54.482 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-11T22:10:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-11T22:10:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:57.210 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-09-11T22:10:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:57.210 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.211 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.212 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-09-11T22:10:57.213 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T22:10:57.214 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T22:10:57.215 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.16-1.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T22:10:57.216 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-11T22:10:57.217 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T22:10:57.218 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T22:10:57.219 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T22:10:57.220 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T22:10:57.221 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-11T22:10:57.222 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T22:10:57.223 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-11T22:10:57.223 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:10:57.223 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-09-11T22:10:57.758 DEBUG:teuthology.parallel:result is None 2024-09-11T22:10:57.759 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:10:57.917 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T22:10:57.941 INFO:teuthology.orchestra.run.smithi005.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T22:10:57.942 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T22:10:57.942 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T22:10:57.944 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:10:58.087 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T22:10:58.114 INFO:teuthology.orchestra.run.smithi159.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T22:10:58.114 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T22:10:58.114 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T22:10:58.116 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:10:58.258 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T22:10:58.285 INFO:teuthology.orchestra.run.smithi172.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-11T22:10:58.285 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-11T22:10:58.286 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-11T22:10:58.287 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T22:10:58.288 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:10:58.288 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T22:10:58.321 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:10:58.322 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T22:10:58.355 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:10:58.355 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T22:10:58.385 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T22:10:58.386 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:10:58.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T22:10:58.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T22:10:58.484 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:10:58.485 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T22:10:58.515 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T22:10:58.583 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:10:58.583 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T22:10:58.610 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T22:10:58.675 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T22:10:58.675 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:10:58.675 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T22:10:58.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T22:10:58.775 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:10:58.775 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T22:10:58.804 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T22:10:58.873 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:10:58.873 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T22:10:58.899 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T22:10:58.965 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T22:10:58.965 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:10:58.965 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T22:10:58.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T22:10:59.066 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:10:59.066 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T22:10:59.097 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T22:10:59.166 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:10:59.166 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T22:10:59.192 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T22:10:59.257 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T22:10:59.351 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-11T22:10:59.351 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:10:59.352 INFO:tasks.cephadm:Cluster fsid is b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:10:59.352 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T22:10:59.352 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.159', 'mon.c': '172.21.15.172'} 2024-09-11T22:10:59.352 INFO:tasks.cephadm:First mon is mon.a on smithi005 2024-09-11T22:10:59.352 INFO:tasks.cephadm:First mgr is x 2024-09-11T22:10:59.352 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T22:10:59.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2024-09-11T22:10:59.384 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2024-09-11T22:10:59.410 DEBUG:teuthology.orchestra.run.smithi172:> sudo hostname $(hostname -s) 2024-09-11T22:10:59.434 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-11T22:10:59.435 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:10:59.575 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-11T22:10:59.714 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:10:59.715 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T22:10:59.716 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-11T22:10:59.716 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T22:11:00.083 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 22:11 /home/ubuntu/cephtest/cephadm 2024-09-11T22:11:00.084 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T22:11:00.461 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 22:11 /home/ubuntu/cephtest/cephadm 2024-09-11T22:11:00.461 DEBUG:teuthology.orchestra.run.smithi172:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T22:11:00.832 INFO:teuthology.orchestra.run.smithi172.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 11 22:11 /home/ubuntu/cephtest/cephadm 2024-09-11T22:11:00.832 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T22:11:00.853 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T22:11:00.869 DEBUG:teuthology.orchestra.run.smithi172:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T22:11:00.903 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-11T22:11:00.903 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T22:11:00.906 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T22:11:00.912 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-11T22:11:02.498 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:11:02.512 INFO:teuthology.orchestra.run.smithi172.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:11:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:11:55.036 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-11T22:11:55.037 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:11:55.037 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T22:11:55.037 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2024-09-11T22:11:55.037 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T22:11:55.037 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-11T22:11:55.037 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-11T22:12:11.272 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:12:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:12:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T22:12:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2024-09-11T22:12:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T22:12:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:12:11.272 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:12:14.384 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-11T22:12:14.384 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-11T22:12:14.385 INFO:teuthology.orchestra.run.smithi172.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-11T22:12:14.385 INFO:teuthology.orchestra.run.smithi172.stdout: "repo_digests": [ 2024-09-11T22:12:14.385 INFO:teuthology.orchestra.run.smithi172.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-11T22:12:14.385 INFO:teuthology.orchestra.run.smithi172.stdout: ] 2024-09-11T22:12:14.385 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-11T22:12:14.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2024-09-11T22:12:14.443 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2024-09-11T22:12:14.476 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph 2024-09-11T22:12:14.503 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2024-09-11T22:12:14.532 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2024-09-11T22:12:14.562 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /etc/ceph 2024-09-11T22:12:14.586 INFO:tasks.cephadm:Writing seed config... 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-11T22:12:14.588 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-11T22:12:14.589 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-11T22:12:14.590 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T22:12:14.591 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-11T22:12:14.592 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:12:14.592 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T22:12:14.610 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b99b16b2-708a-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-11T22:12:14.610 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T22:12:14.653 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T22:12:14.696 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T22:12:14.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid b99b16b2-708a-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:12:14.908 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-11T22:12:14.909 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', 'b99b16b2-708a-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2024-09-11T22:12:14.909 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T22:12:14.909 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2024-09-11T22:12:14.933 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.2.2 2024-09-11T22:12:14.933 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2024-09-11T22:12:14.933 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2024-09-11T22:12:14.941 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T22:12:14.941 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T22:12:14.948 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T22:12:14.948 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-11T22:12:14.955 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-09-11T22:12:14.961 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-09-11T22:12:14.961 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-09-11T22:12:14.961 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2024-09-11T22:12:14.986 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.2.2 2024-09-11T22:12:14.986 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T22:12:14.986 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2024-09-11T22:12:14.986 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2024-09-11T22:12:14.993 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T22:12:14.993 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T22:12:14.999 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T22:12:14.999 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-11T22:12:15.006 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-09-11T22:12:15.013 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-09-11T22:12:15.013 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-09-11T22:12:15.013 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2024-09-11T22:12:15.013 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:15.013 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 140187546326016 on /run/cephadm/b99b16b2-708a-11ef-bcea-c7b262605968.lock 2024-09-11T22:12:15.013 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 140187546326016 acquired on /run/cephadm/b99b16b2-708a-11ef-bcea-c7b262605968.lock 2024-09-11T22:12:15.014 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2024-09-11T22:12:15.014 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2024-09-11T22:12:15.014 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-11T22:12:15.017 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.5 metric 100 2024-09-11T22:12:15.017 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2024-09-11T22:12:15.020 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T22:12:15.020 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-11T22:12:15.020 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1791sec hoplimit 64 pref medium 2024-09-11T22:12:15.023 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T22:12:15.023 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T22:12:15.023 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T22:12:15.023 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-11T22:12:15.023 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-11T22:12:15.023 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T22:12:15.024 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-09-11T22:12:15.024 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-09-11T22:12:15.024 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-11T22:12:15.024 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T22:12:15.025 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:12:15.465 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T22:12:15.465 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-11T22:12:15.465 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T22:12:15.465 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-11T22:12:15.465 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T22:12:15.465 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-11T22:12:15.465 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T22:12:16.403 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T22:12:16.403 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-11T22:12:16.404 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T22:12:17.284 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2024-09-11T22:12:17.285 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2024-09-11T22:12:18.165 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQDBFeJmaI9eIxAAm+it+ATttXk1lcvsUAm58A== 2024-09-11T22:12:19.095 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQDCFeJm/fk6HRAAUJJn/Ia/IgU2CV78UoKgOQ== 2024-09-11T22:12:19.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQDDFeJmLeKlFxAARxr/M9vT5ZpTxeBkngw1GQ== 2024-09-11T22:12:19.977 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2024-09-11T22:12:20.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T22:12:20.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-11T22:12:20.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:20.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T22:12:20.866 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T22:12:20.866 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-09-11T22:12:20.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:20.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T22:12:20.866 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:12:20.866 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2024-09-11T22:12:21.771 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2024-09-11T22:12:22.103 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T22:12:22.266 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T22:12:22.447 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b99b16b2-708a-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-b99b16b2-708a-11ef-bcea-c7b262605968.target. 2024-09-11T22:12:22.448 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b99b16b2-708a-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-b99b16b2-708a-11ef-bcea-c7b262605968.target. 2024-09-11T22:12:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.a 2024-09-11T22:12:22.720 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T22:12:22.919 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b99b16b2-708a-11ef-bcea-c7b262605968.target.wants/ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b99b16b2-708a-11ef-bcea-c7b262605968@.service. 2024-09-11T22:12:22.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:22 smithi005 systemd[1]: Starting Ceph mon.a for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:12:23.297 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:23 smithi005 podman[30556]: 2024-09-11 22:12:23.064106784 +0000 UTC m=+0.016890466 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:12:23.297 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:23 smithi005 podman[30556]: 2024-09-11 22:12:23.178787978 +0000 UTC m=+0.131571650 container create d26d9b9dda2c7c76b8472b6003fc107f2080026e418e5c7e0d54697cd5766f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-11T22:12:23.496 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:12:23.496 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-11T22:12:23.504 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:12:23.504 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-11T22:12:23.504 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-11T22:12:23.504 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T22:12:23.505 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2024-09-11T22:12:23.505 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2024-09-11T22:12:23.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:23 smithi005 podman[30556]: 2024-09-11 22:12:23.406126983 +0000 UTC m=+0.358910660 container init d26d9b9dda2c7c76b8472b6003fc107f2080026e418e5c7e0d54697cd5766f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-11T22:12:23.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:23 smithi005 podman[30556]: 2024-09-11 22:12:23.409525504 +0000 UTC m=+0.362309177 container start d26d9b9dda2c7c76b8472b6003fc107f2080026e418e5c7e0d54697cd5766f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T22:12:23.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:23 smithi005 ceph-mon[30575]: mkfs b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:23.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:23 smithi005 ceph-mon[30575]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T22:12:23.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:23 smithi005 bash[30556]: d26d9b9dda2c7c76b8472b6003fc107f2080026e418e5c7e0d54697cd5766f5c 2024-09-11T22:12:23.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:23 smithi005 systemd[1]: Started Ceph mon.a for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:12:24.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T22:12:24.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:24.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T22:12:24.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:24.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2024-09-11T22:12:24.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.513212s) 2024-09-11T22:12:24.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T22:12:24.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T22:12:24.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:24.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2024-09-11T22:12:24.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T22:12:24.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T22:12:24.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.887+0000 7fed7a0ee640 1 Processor -- start 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.888+0000 7fed7a0ee640 1 -- start start 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.888+0000 7fed7a0ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed74107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.888+0000 7fed7a0ee640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed74108160 con 0x7fed74107790 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.888+0000 7fed737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed74107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.888+0000 7fed737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed74107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32822/0 (socket says 172.21.15.5:32822) 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.888+0000 7fed737fe640 1 -- 172.21.15.5:0/767189259 learned_addr learned my addr 172.21.15.5:0/767189259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.888+0000 7fed737fe640 1 -- 172.21.15.5:0/767189259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed74108940 con 0x7fed74107790 2024-09-11T22:12:24.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.890+0000 7fed737fe640 1 --2- 172.21.15.5:0/767189259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed74107b90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fed64009820 tx=0x7fed6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b6746e1854488c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.891+0000 7fed72ffd640 1 -- 172.21.15.5:0/767189259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed64037070 con 0x7fed74107790 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.891+0000 7fed72ffd640 1 -- 172.21.15.5:0/767189259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed64032440 con 0x7fed74107790 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.891+0000 7fed72ffd640 1 -- 172.21.15.5:0/767189259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed64032740 con 0x7fed74107790 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.891+0000 7fed7a0ee640 1 -- 172.21.15.5:0/767189259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 msgr2=0x7fed74107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.891+0000 7fed7a0ee640 1 --2- 172.21.15.5:0/767189259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed74107b90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fed64009820 tx=0x7fed6402a270 comp rx=0 tx=0).stop 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.891+0000 7fed7a0ee640 1 -- 172.21.15.5:0/767189259 shutdown_connections 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.891+0000 7fed7a0ee640 1 --2- 172.21.15.5:0/767189259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed74107b90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.891+0000 7fed7a0ee640 1 -- 172.21.15.5:0/767189259 >> 172.21.15.5:0/767189259 conn(0x7fed74102fa0 msgr2=0x7fed741053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.891+0000 7fed7a0ee640 1 -- 172.21.15.5:0/767189259 shutdown_connections 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.892+0000 7fed7a0ee640 1 -- 172.21.15.5:0/767189259 wait complete. 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.892+0000 7fed7a0ee640 1 Processor -- start 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.892+0000 7fed7a0ee640 1 -- start start 2024-09-11T22:12:24.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.892+0000 7fed7a0ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed7419af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.892+0000 7fed7a0ee640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed74108ee0 con 0x7fed74107790 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.892+0000 7fed737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed7419af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.892+0000 7fed737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed7419af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32832/0 (socket says 172.21.15.5:32832) 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.892+0000 7fed737fe640 1 -- 172.21.15.5:0/877020741 learned_addr learned my addr 172.21.15.5:0/877020741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.893+0000 7fed737fe640 1 -- 172.21.15.5:0/877020741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7419b440 con 0x7fed74107790 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.893+0000 7fed737fe640 1 --2- 172.21.15.5:0/877020741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed7419af00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fed640090b0 tx=0x7fed640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.893+0000 7fed717fa640 1 -- 172.21.15.5:0/877020741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed64041070 con 0x7fed74107790 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.893+0000 7fed717fa640 1 -- 172.21.15.5:0/877020741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6403c7a0 con 0x7fed74107790 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.893+0000 7fed7a0ee640 1 -- 172.21.15.5:0/877020741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed7419b6d0 con 0x7fed74107790 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.893+0000 7fed717fa640 1 -- 172.21.15.5:0/877020741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed64037040 con 0x7fed74107790 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.893+0000 7fed7a0ee640 1 -- 172.21.15.5:0/877020741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed7419e3c0 con 0x7fed74107790 2024-09-11T22:12:24.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.893+0000 7fed717fa640 1 -- 172.21.15.5:0/877020741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fed64044070 con 0x7fed74107790 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.893+0000 7fed717fa640 1 -- 172.21.15.5:0/877020741 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fed6403d3f0 con 0x7fed74107790 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.894+0000 7fed7a0ee640 1 -- 172.21.15.5:0/877020741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed74109530 con 0x7fed74107790 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.896+0000 7fed717fa640 1 -- 172.21.15.5:0/877020741 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fed64046450 con 0x7fed74107790 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.958+0000 7fed7a0ee640 1 -- 172.21.15.5:0/877020741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fed7419e910 con 0x7fed74107790 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.958+0000 7fed717fa640 1 -- 172.21.15.5:0/877020741 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fed64046630 con 0x7fed74107790 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.959+0000 7fed7a0ee640 1 -- 172.21.15.5:0/877020741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 msgr2=0x7fed7419af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.959+0000 7fed7a0ee640 1 --2- 172.21.15.5:0/877020741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed7419af00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fed640090b0 tx=0x7fed640046c0 comp rx=0 tx=0).stop 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.959+0000 7fed7a0ee640 1 -- 172.21.15.5:0/877020741 shutdown_connections 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.959+0000 7fed7a0ee640 1 --2- 172.21.15.5:0/877020741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed74107790 0x7fed7419af00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.959+0000 7fed7a0ee640 1 -- 172.21.15.5:0/877020741 >> 172.21.15.5:0/877020741 conn(0x7fed74102fa0 msgr2=0x7fed7418f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.959+0000 7fed7a0ee640 1 -- 172.21.15.5:0/877020741 shutdown_connections 2024-09-11T22:12:24.435 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:23.959+0000 7fed7a0ee640 1 -- 172.21.15.5:0/877020741 wait complete. 2024-09-11T22:12:24.436 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2024-09-11T22:12:24.436 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T22:12:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T22:12:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: monmap epoch 1 2024-09-11T22:12:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: last_changed 2024-09-11T22:12:20.315558+0000 2024-09-11T22:12:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: created 2024-09-11T22:12:20.315558+0000 2024-09-11T22:12:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: min_mon_release 19 (squid) 2024-09-11T22:12:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: election_strategy: 1 2024-09-11T22:12:24.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-11T22:12:24.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: fsmap 2024-09-11T22:12:24.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T22:12:24.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: mgrmap e1: no daemons active 2024-09-11T22:12:24.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:24 smithi005 ceph-mon[30575]: from='client.? 172.21.15.5:0/877020741' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T22:12:25.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:25.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T22:12:25.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:25.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-11T22:12:25.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T22:12:25.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T22:12:25.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T22:12:25.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.838+0000 7fb2524d8640 1 Processor -- start 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.838+0000 7fb2524d8640 1 -- start start 2024-09-11T22:12:25.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.838+0000 7fb2524d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.838+0000 7fb2524d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c108160 con 0x7fb24c107790 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.838+0000 7fb24bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.838+0000 7fb24bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32834/0 (socket says 172.21.15.5:32834) 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.838+0000 7fb24bfff640 1 -- 172.21.15.5:0/3205387585 learned_addr learned my addr 172.21.15.5:0/3205387585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.838+0000 7fb24bfff640 1 -- 172.21.15.5:0/3205387585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c108940 con 0x7fb24c107790 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.839+0000 7fb24bfff640 1 --2- 172.21.15.5:0/3205387585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c107b90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009820 tx=0x7fb23c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cee28113b58e8e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.839+0000 7fb24affd640 1 -- 172.21.15.5:0/3205387585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb23c037070 con 0x7fb24c107790 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.839+0000 7fb24affd640 1 -- 172.21.15.5:0/3205387585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23c032440 con 0x7fb24c107790 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.839+0000 7fb24affd640 1 -- 172.21.15.5:0/3205387585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb23c032740 con 0x7fb24c107790 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.840+0000 7fb2524d8640 1 -- 172.21.15.5:0/3205387585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 msgr2=0x7fb24c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.840+0000 7fb2524d8640 1 --2- 172.21.15.5:0/3205387585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c107b90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb23c009820 tx=0x7fb23c02a270 comp rx=0 tx=0).stop 2024-09-11T22:12:25.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.840+0000 7fb2524d8640 1 -- 172.21.15.5:0/3205387585 shutdown_connections 2024-09-11T22:12:25.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.840+0000 7fb2524d8640 1 --2- 172.21.15.5:0/3205387585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c107b90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:25.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.840+0000 7fb2524d8640 1 -- 172.21.15.5:0/3205387585 >> 172.21.15.5:0/3205387585 conn(0x7fb24c102fa0 msgr2=0x7fb24c1053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:25.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.840+0000 7fb2524d8640 1 -- 172.21.15.5:0/3205387585 shutdown_connections 2024-09-11T22:12:25.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.840+0000 7fb2524d8640 1 -- 172.21.15.5:0/3205387585 wait complete. 2024-09-11T22:12:25.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.841+0000 7fb2524d8640 1 Processor -- start 2024-09-11T22:12:25.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.841+0000 7fb2524d8640 1 -- start start 2024-09-11T22:12:25.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.841+0000 7fb2524d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c07e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:25.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.841+0000 7fb2524d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c108ee0 con 0x7fb24c107790 2024-09-11T22:12:25.506 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.841+0000 7fb24bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c07e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.841+0000 7fb24bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c07e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32846/0 (socket says 172.21.15.5:32846) 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.841+0000 7fb24bfff640 1 -- 172.21.15.5:0/838060923 learned_addr learned my addr 172.21.15.5:0/838060923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.841+0000 7fb24bfff640 1 -- 172.21.15.5:0/838060923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c07e910 con 0x7fb24c107790 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.842+0000 7fb24bfff640 1 --2- 172.21.15.5:0/838060923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c07e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0090b0 tx=0x7fb23c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.842+0000 7fb2497fa640 1 -- 172.21.15.5:0/838060923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb23c041070 con 0x7fb24c107790 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.842+0000 7fb2497fa640 1 -- 172.21.15.5:0/838060923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb23c03c7a0 con 0x7fb24c107790 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.842+0000 7fb2524d8640 1 -- 172.21.15.5:0/838060923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c07cb00 con 0x7fb24c107790 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.842+0000 7fb2497fa640 1 -- 172.21.15.5:0/838060923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb23c037040 con 0x7fb24c107790 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.842+0000 7fb2524d8640 1 -- 172.21.15.5:0/838060923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c07d040 con 0x7fb24c107790 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.842+0000 7fb2497fa640 1 -- 172.21.15.5:0/838060923 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb23c044070 con 0x7fb24c107790 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.842+0000 7fb2497fa640 1 -- 172.21.15.5:0/838060923 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb23c03d3f0 con 0x7fb24c107790 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.843+0000 7fb2524d8640 1 -- 172.21.15.5:0/838060923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24c109530 con 0x7fb24c107790 2024-09-11T22:12:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.845+0000 7fb2497fa640 1 -- 172.21.15.5:0/838060923 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb23c046450 con 0x7fb24c107790 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.905+0000 7fb2524d8640 1 -- 172.21.15.5:0/838060923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fb24c07d340 con 0x7fb24c107790 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.908+0000 7fb2497fa640 1 -- 172.21.15.5:0/838060923 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7fb23c046630 con 0x7fb24c107790 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.908+0000 7fb2497fa640 1 -- 172.21.15.5:0/838060923 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb23c056070 con 0x7fb24c107790 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.909+0000 7fb2524d8640 1 -- 172.21.15.5:0/838060923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 msgr2=0x7fb24c07e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.909+0000 7fb2524d8640 1 --2- 172.21.15.5:0/838060923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c07e3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0090b0 tx=0x7fb23c0046c0 comp rx=0 tx=0).stop 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.909+0000 7fb2524d8640 1 -- 172.21.15.5:0/838060923 shutdown_connections 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.909+0000 7fb2524d8640 1 --2- 172.21.15.5:0/838060923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24c107790 0x7fb24c07e3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.909+0000 7fb2524d8640 1 -- 172.21.15.5:0/838060923 >> 172.21.15.5:0/838060923 conn(0x7fb24c102fa0 msgr2=0x7fb24c18f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.909+0000 7fb2524d8640 1 -- 172.21.15.5:0/838060923 shutdown_connections 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:24.909+0000 7fb2524d8640 1 -- 172.21.15.5:0/838060923 wait complete. 2024-09-11T22:12:25.508 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2024-09-11T22:12:25.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:25 smithi005 ceph-mon[30575]: from='client.? 172.21.15.5:0/838060923' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T22:12:25.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:25 smithi005 ceph-mon[30575]: from='client.? 172.21.15.5:0/838060923' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T22:12:26.601 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:26 smithi005 ceph-mon[30575]: from='client.? 172.21.15.5:0/3450157579' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:12:26.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.969+0000 7f98ca917640 1 Processor -- start 2024-09-11T22:12:26.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.969+0000 7f98ca917640 1 -- start start 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.970+0000 7f98ca917640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c4106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.970+0000 7f98ca917640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98c4107010 con 0x7f98c4106640 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.970+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c4106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.970+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c4106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32862/0 (socket says 172.21.15.5:32862) 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.970+0000 7f98c3fff640 1 -- 172.21.15.5:0/4054119353 learned_addr learned my addr 172.21.15.5:0/4054119353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.970+0000 7f98c3fff640 1 -- 172.21.15.5:0/4054119353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c41077f0 con 0x7f98c4106640 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.971+0000 7f98c3fff640 1 --2- 172.21.15.5:0/4054119353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c4106a40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009820 tx=0x7f98b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6409249199113e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.971+0000 7f98c2ffd640 1 -- 172.21.15.5:0/4054119353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4037070 con 0x7f98c4106640 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.971+0000 7f98c2ffd640 1 -- 172.21.15.5:0/4054119353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98b4032900 con 0x7f98c4106640 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.971+0000 7f98c2ffd640 1 -- 172.21.15.5:0/4054119353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4030850 con 0x7f98c4106640 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.972+0000 7f98ca917640 1 -- 172.21.15.5:0/4054119353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 msgr2=0x7f98c4106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:26.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.972+0000 7f98ca917640 1 --2- 172.21.15.5:0/4054119353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c4106a40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009820 tx=0x7f98b402a270 comp rx=0 tx=0).stop 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.972+0000 7f98ca917640 1 -- 172.21.15.5:0/4054119353 shutdown_connections 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.972+0000 7f98ca917640 1 --2- 172.21.15.5:0/4054119353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c4106a40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.972+0000 7f98ca917640 1 -- 172.21.15.5:0/4054119353 >> 172.21.15.5:0/4054119353 conn(0x7f98c4101db0 msgr2=0x7f98c4104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.972+0000 7f98ca917640 1 -- 172.21.15.5:0/4054119353 shutdown_connections 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.972+0000 7f98ca917640 1 -- 172.21.15.5:0/4054119353 wait complete. 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.972+0000 7f98ca917640 1 Processor -- start 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.973+0000 7f98ca917640 1 -- start start 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.973+0000 7f98ca917640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.973+0000 7f98ca917640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98c410aba0 con 0x7f98c4106640 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.973+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.973+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c419b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32878/0 (socket says 172.21.15.5:32878) 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.973+0000 7f98c3fff640 1 -- 172.21.15.5:0/3450157579 learned_addr learned my addr 172.21.15.5:0/3450157579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:26.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.973+0000 7f98c3fff640 1 -- 172.21.15.5:0/3450157579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c419bd70 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.973+0000 7f98c3fff640 1 --2- 172.21.15.5:0/3450157579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c419b830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009940 tx=0x7f98b40326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.974+0000 7f98c17fa640 1 -- 172.21.15.5:0/3450157579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4042070 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.974+0000 7f98c17fa640 1 -- 172.21.15.5:0/3450157579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f98b4032be0 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.974+0000 7f98ca917640 1 -- 172.21.15.5:0/3450157579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c419c000 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.974+0000 7f98c17fa640 1 -- 172.21.15.5:0/3450157579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98b4037040 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.974+0000 7f98ca917640 1 -- 172.21.15.5:0/3450157579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c419ecf0 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.974+0000 7f98c17fa640 1 -- 172.21.15.5:0/3450157579 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f98b4045070 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.974+0000 7f98c17fa640 1 -- 172.21.15.5:0/3450157579 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f98b4047a80 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.975+0000 7f98ca917640 1 -- 172.21.15.5:0/3450157579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c410ad20 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:25.977+0000 7f98c17fa640 1 -- 172.21.15.5:0/3450157579 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f98b4032d90 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:26.037+0000 7f98ca917640 1 -- 172.21.15.5:0/3450157579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f98c402d9c0 con 0x7f98c4106640 2024-09-11T22:12:26.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:26.037+0000 7f98c17fa640 1 -- 172.21.15.5:0/3450157579 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f98b403d430 con 0x7f98c4106640 2024-09-11T22:12:26.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:26.038+0000 7f98ca917640 1 -- 172.21.15.5:0/3450157579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 msgr2=0x7f98c419b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:26.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:26.038+0000 7f98ca917640 1 --2- 172.21.15.5:0/3450157579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c419b830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f98b4009940 tx=0x7f98b40326e0 comp rx=0 tx=0).stop 2024-09-11T22:12:26.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:26.038+0000 7f98ca917640 1 -- 172.21.15.5:0/3450157579 shutdown_connections 2024-09-11T22:12:26.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:26.038+0000 7f98ca917640 1 --2- 172.21.15.5:0/3450157579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c4106640 0x7f98c419b830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:26.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:26.038+0000 7f98ca917640 1 -- 172.21.15.5:0/3450157579 >> 172.21.15.5:0/3450157579 conn(0x7f98c4101db0 msgr2=0x7f98c4102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:26.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:26.038+0000 7f98ca917640 1 -- 172.21.15.5:0/3450157579 shutdown_connections 2024-09-11T22:12:26.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:26.038+0000 7f98ca917640 1 -- 172.21.15.5:0/3450157579 wait complete. 2024-09-11T22:12:26.611 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2024-09-11T22:12:27.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:26 smithi005 systemd[1]: Stopping Ceph mon.a for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:12:28.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:27 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a[30571]: 2024-09-11T22:12:27.850+0000 7f1c980d4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:12:28.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:27 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a[30571]: 2024-09-11T22:12:27.850+0000 7f1c980d4640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T22:12:28.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:27 smithi005 podman[30816]: 2024-09-11 22:12:27.923103618 +0000 UTC m=+1.145718665 container died d26d9b9dda2c7c76b8472b6003fc107f2080026e418e5c7e0d54697cd5766f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:12:32.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:31 smithi005 podman[30816]: 2024-09-11 22:12:31.820751894 +0000 UTC m=+5.043366939 container cleanup d26d9b9dda2c7c76b8472b6003fc107f2080026e418e5c7e0d54697cd5766f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T22:12:32.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:31 smithi005 bash[30816]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a 2024-09-11T22:12:33.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:33 smithi005 podman[30827]: 2024-09-11 22:12:33.774991758 +0000 UTC m=+5.848981412 container remove d26d9b9dda2c7c76b8472b6003fc107f2080026e418e5c7e0d54697cd5766f5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:12:34.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:33 smithi005 systemd[1]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T22:12:34.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:33 smithi005 systemd[1]: Stopped Ceph mon.a for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:12:34.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:33 smithi005 systemd[1]: Starting Ceph mon.a for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:12:34.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:34 smithi005 podman[30924]: 2024-09-11 22:12:34.12371984 +0000 UTC m=+0.017117727 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-11T22:12:35.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:35 smithi005 podman[30924]: 2024-09-11 22:12:35.177164304 +0000 UTC m=+1.070562219 container create 32a280f5e920c55b3b7da4dd51dedf00d06635c2600b39019fbde98ad374d00a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-11T22:12:36.456 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T22:12:36.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 podman[30924]: 2024-09-11 22:12:36.376402159 +0000 UTC m=+2.269800032 container init 32a280f5e920c55b3b7da4dd51dedf00d06635c2600b39019fbde98ad374d00a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T22:12:36.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 podman[30924]: 2024-09-11 22:12:36.380538935 +0000 UTC m=+2.273936810 container start 32a280f5e920c55b3b7da4dd51dedf00d06635c2600b39019fbde98ad374d00a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T22:12:36.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: pidfile_write: ignore empty --pid-file 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: load: jerasure load: lrc 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: RocksDB version: 7.9.2 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Git sha 0 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: DB SUMMARY 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: DB Session ID: Q5J9J1BOM91QLYZYH9B0 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: CURRENT file: CURRENT 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-11T22:12:36.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106378 ; 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.error_if_exists: 0 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.create_if_missing: 0 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.paranoid_checks: 1 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.env: 0x5561fc9be940 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.fs: PosixFileSystem 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.info_log: 0x5561fdf096e0 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T22:12:36.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.statistics: (nil) 2024-09-11T22:12:36.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.use_fsync: 0 2024-09-11T22:12:36.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_log_file_size: 0 2024-09-11T22:12:36.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T22:12:36.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T22:12:36.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T22:12:36.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T22:12:36.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.allow_fallocate: 1 2024-09-11T22:12:36.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T22:12:36.685 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T22:12:36.685 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.use_direct_reads: 0 2024-09-11T22:12:36.685 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T22:12:36.685 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T22:12:36.685 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.db_log_dir: 2024-09-11T22:12:36.685 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.wal_dir: 2024-09-11T22:12:36.685 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.write_buffer_manager: 0x5561fdf18dc0 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T22:12:36.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.unordered_write: 0 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.row_cache: None 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.wal_filter: None 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T22:12:36.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.two_write_queues: 0 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.wal_compression: 0 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.atomic_flush: 0 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.log_readahead_size: 0 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T22:12:36.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_background_jobs: 2 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_background_compactions: -1 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_subcompactions: 1 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T22:12:36.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_open_files: -1 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_background_flushes: -1 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Compression algorithms supported: 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: kZSTD supported: 0 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: kXpressCompression supported: 0 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: kBZip2Compression supported: 0 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: kLZ4Compression supported: 1 2024-09-11T22:12:36.690 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: kZlibCompression supported: 1 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: kSnappyCompression supported: 1 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.merge_operator: 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_filter: None 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T22:12:36.691 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5561fdf09320) 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x5561fdefd1f0 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2024-09-11T22:12:36.692 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2024-09-11T22:12:36.693 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T22:12:36.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compression: NoCompression 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.num_levels: 7 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T22:12:36.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T22:12:36.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T22:12:36.697 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.arena_block_size: 1048576 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T22:12:36.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.inplace_update_support: 0 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T22:12:36.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.bloom_locality: 0 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.max_successive_merges: 0 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.ttl: 2592000 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.enable_blob_files: false 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.min_blob_size: 0 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T22:12:36.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.blob_file_starting_level: 0 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ed5aa92a-e329-47ac-9fa9-cf11998032d5 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726092756412618, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726092756413908, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99274, "index_size": 316, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14564, "raw_average_key_size": 50, "raw_value_size": 91349, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726092756, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ed5aa92a-e329-47ac-9fa9-cf11998032d5", "db_session_id": "Q5J9J1BOM91QLYZYH9B0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-11T22:12:36.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726092756414028, "job": 1, "event": "recovery_finished"} 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5561fdf2ae00 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: DB pointer 0x5561fe026000 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T22:12:36.702 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 100.88 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 100.88 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-11T22:12:36.703 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 117.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 24.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T22:12:36.704 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 24.06 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x5561fdefd1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: mon.a@-1(???) e1 preinit fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: mon.a@-1(???).mds e1 new map 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: mon.a@-1(???).mds e1 print_map 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout: btime 2024-09-11T22:12:23:446697+0000 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T22:12:36.705 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 bash[30924]: 32a280f5e920c55b3b7da4dd51dedf00d06635c2600b39019fbde98ad374d00a 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 systemd[1]: Started Ceph mon.a for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: monmap epoch 1 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: last_changed 2024-09-11T22:12:20.315558+0000 2024-09-11T22:12:36.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: created 2024-09-11T22:12:20.315558+0000 2024-09-11T22:12:36.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: min_mon_release 19 (squid) 2024-09-11T22:12:36.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: election_strategy: 1 2024-09-11T22:12:36.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-11T22:12:36.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: fsmap 2024-09-11T22:12:36.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T22:12:36.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:36 smithi005 ceph-mon[30943]: mgrmap e1: no daemons active 2024-09-11T22:12:37.552 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.929+0000 7fdb3b689640 1 Processor -- start 2024-09-11T22:12:37.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.930+0000 7fdb3b689640 1 -- start start 2024-09-11T22:12:37.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.930+0000 7fdb3b689640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb34107b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:37.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.930+0000 7fdb3b689640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb341080e0 con 0x7fdb34107710 2024-09-11T22:12:37.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.930+0000 7fdb39425640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb34107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.930+0000 7fdb39425640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb34107b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41810/0 (socket says 172.21.15.5:41810) 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.930+0000 7fdb39425640 1 -- 172.21.15.5:0/41034185 learned_addr learned my addr 172.21.15.5:0/41034185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.930+0000 7fdb39425640 1 -- 172.21.15.5:0/41034185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb34108910 con 0x7fdb34107710 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.932+0000 7fdb39425640 1 --2- 172.21.15.5:0/41034185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb34107b10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009820 tx=0x7fdb2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b275b02bcfb07b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.932+0000 7fdb33fff640 1 -- 172.21.15.5:0/41034185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb2c037070 con 0x7fdb34107710 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.933+0000 7fdb33fff640 1 -- 172.21.15.5:0/41034185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdb2c032900 con 0x7fdb34107710 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.933+0000 7fdb33fff640 1 -- 172.21.15.5:0/41034185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb2c030850 con 0x7fdb34107710 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.933+0000 7fdb3b689640 1 -- 172.21.15.5:0/41034185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 msgr2=0x7fdb34107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.933+0000 7fdb3b689640 1 --2- 172.21.15.5:0/41034185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb34107b10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009820 tx=0x7fdb2c02a270 comp rx=0 tx=0).stop 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.933+0000 7fdb3b689640 1 -- 172.21.15.5:0/41034185 shutdown_connections 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.933+0000 7fdb3b689640 1 --2- 172.21.15.5:0/41034185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb34107b10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.933+0000 7fdb3b689640 1 -- 172.21.15.5:0/41034185 >> 172.21.15.5:0/41034185 conn(0x7fdb34102ec0 msgr2=0x7fdb341052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:37.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.933+0000 7fdb3b689640 1 -- 172.21.15.5:0/41034185 shutdown_connections 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.933+0000 7fdb3b689640 1 -- 172.21.15.5:0/41034185 wait complete. 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.934+0000 7fdb3b689640 1 Processor -- start 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.934+0000 7fdb3b689640 1 -- start start 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.934+0000 7fdb3b689640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb3419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.934+0000 7fdb3b689640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3410bcc0 con 0x7fdb34107710 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.934+0000 7fdb39425640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb3419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.934+0000 7fdb39425640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb3419b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41812/0 (socket says 172.21.15.5:41812) 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.934+0000 7fdb39425640 1 -- 172.21.15.5:0/3817125772 learned_addr learned my addr 172.21.15.5:0/3817125772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.935+0000 7fdb39425640 1 -- 172.21.15.5:0/3817125772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3419bc80 con 0x7fdb34107710 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.935+0000 7fdb39425640 1 --2- 172.21.15.5:0/3817125772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb3419b740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009940 tx=0x7fdb2c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.935+0000 7fdb327fc640 1 -- 172.21.15.5:0/3817125772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb2c042070 con 0x7fdb34107710 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.935+0000 7fdb327fc640 1 -- 172.21.15.5:0/3817125772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdb2c0308c0 con 0x7fdb34107710 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.935+0000 7fdb3b689640 1 -- 172.21.15.5:0/3817125772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3419bf10 con 0x7fdb34107710 2024-09-11T22:12:37.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.935+0000 7fdb327fc640 1 -- 172.21.15.5:0/3817125772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb2c037040 con 0x7fdb34107710 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.935+0000 7fdb3b689640 1 -- 172.21.15.5:0/3817125772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3419ec00 con 0x7fdb34107710 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.935+0000 7fdb327fc640 1 -- 172.21.15.5:0/3817125772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb2c045070 con 0x7fdb34107710 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.936+0000 7fdb327fc640 1 -- 172.21.15.5:0/3817125772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdb2c0463b0 con 0x7fdb34107710 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.936+0000 7fdb3b689640 1 -- 172.21.15.5:0/3817125772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb3410be40 con 0x7fdb34107710 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.939+0000 7fdb327fc640 1 -- 172.21.15.5:0/3817125772 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdb2c0327b0 con 0x7fdb34107710 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:36.998+0000 7fdb3b689640 1 -- 172.21.15.5:0/3817125772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fdb3419f300 con 0x7fdb34107710 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:37.000+0000 7fdb327fc640 1 -- 172.21.15.5:0/3817125772 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fdb2c0465b0 con 0x7fdb34107710 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:37.000+0000 7fdb3b689640 1 -- 172.21.15.5:0/3817125772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 msgr2=0x7fdb3419b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:37.000+0000 7fdb3b689640 1 --2- 172.21.15.5:0/3817125772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb3419b740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009940 tx=0x7fdb2c032ce0 comp rx=0 tx=0).stop 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:37.001+0000 7fdb3b689640 1 -- 172.21.15.5:0/3817125772 shutdown_connections 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:37.001+0000 7fdb3b689640 1 --2- 172.21.15.5:0/3817125772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb34107710 0x7fdb3419b740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:37.001+0000 7fdb3b689640 1 -- 172.21.15.5:0/3817125772 >> 172.21.15.5:0/3817125772 conn(0x7fdb34102ec0 msgr2=0x7fdb34103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:37.001+0000 7fdb3b689640 1 -- 172.21.15.5:0/3817125772 shutdown_connections 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:37.001+0000 7fdb3b689640 1 -- 172.21.15.5:0/3817125772 wait complete. 2024-09-11T22:12:37.559 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T22:12:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:12:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2024-09-11T22:12:37.574 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T22:12:37.575 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T22:12:37.821 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.x 2024-09-11T22:12:37.821 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-11T22:12:37.970 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b99b16b2-708a-11ef-bcea-c7b262605968.target.wants/ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b99b16b2-708a-11ef-bcea-c7b262605968@.service. 2024-09-11T22:12:38.203 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:38 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3817125772' entity='client.admin' 2024-09-11T22:12:38.496 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:38 smithi005 podman[31154]: 2024-09-11 22:12:38.423468995 +0000 UTC m=+0.333891714 container start 894bca1446d0ed668ea805ec0fdbde6cd05ce713329f0bd6a02014d4acfa0890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T22:12:38.496 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:38 smithi005 bash[31154]: 894bca1446d0ed668ea805ec0fdbde6cd05ce713329f0bd6a02014d4acfa0890 2024-09-11T22:12:38.513 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:12:38.513 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-11T22:12:38.521 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:12:38.521 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-11T22:12:38.521 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-11T22:12:38.522 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T22:12:38.532 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:12:38.533 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-11T22:12:38.540 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:12:38.540 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-11T22:12:38.540 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-11T22:12:38.540 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T22:12:38.541 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2024-09-11T22:12:38.541 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2024-09-11T22:12:38.777 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:38 smithi005 systemd[1]: Started Ceph mgr.x for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:12:38.778 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:38 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:38.555+0000 7f17ee8d0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:12:38.778 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:38 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:38.617+0000 7f17ee8d0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:12:38.778 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:38 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:38.775+0000 7f17ee8d0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:12:39.051 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:38 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:38.847+0000 7f17ee8d0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:12:39.051 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:39 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:39.005+0000 7f17ee8d0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:12:39.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:39 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/565356608' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T22:12:39.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:39 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:39.157+0000 7f17ee8d0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:12:39.694 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:39 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:39.489+0000 7f17ee8d0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:12:39.694 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:39 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:39.553+0000 7f17ee8d0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:12:39.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:39.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:12:39.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:12:39.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T22:12:39.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T22:12:39.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T22:12:39.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T22:12:39.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:39.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T22:12:39.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:12:39.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T22:12:39.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T22:12:39.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T22:12:39.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T22:12:39.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T22:12:39.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T22:12:39.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:39.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T22:12:39.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T22:12:39.709 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T22:12:23:446697+0000", 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T22:12:39.710 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:12:39.711 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T22:12:23.447334+0000", 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.979+0000 7fa384441640 1 Processor -- start 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.979+0000 7fa384441640 1 -- start start 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.980+0000 7fa384441640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c10a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.980+0000 7fa384441640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c07be30 con 0x7fa37c1081b0 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.980+0000 7fa3821dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c10a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.980+0000 7fa3821dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c10a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41832/0 (socket says 172.21.15.5:41832) 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.980+0000 7fa3821dd640 1 -- 172.21.15.5:0/531940839 learned_addr learned my addr 172.21.15.5:0/531940839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.980+0000 7fa3821dd640 1 -- 172.21.15.5:0/531940839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c10aae0 con 0x7fa37c1081b0 2024-09-11T22:12:39.712 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.982+0000 7fa3821dd640 1 --2- 172.21.15.5:0/531940839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c10a5a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa378009820 tx=0x7fa37802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc03e8bc593d90a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.982+0000 7fa3811db640 1 -- 172.21.15.5:0/531940839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa378037070 con 0x7fa37c1081b0 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.982+0000 7fa3811db640 1 -- 172.21.15.5:0/531940839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa378032900 con 0x7fa37c1081b0 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.982+0000 7fa3811db640 1 -- 172.21.15.5:0/531940839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa378030850 con 0x7fa37c1081b0 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.983+0000 7fa384441640 1 -- 172.21.15.5:0/531940839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 msgr2=0x7fa37c10a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.983+0000 7fa384441640 1 --2- 172.21.15.5:0/531940839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c10a5a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa378009820 tx=0x7fa37802a270 comp rx=0 tx=0).stop 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.983+0000 7fa384441640 1 -- 172.21.15.5:0/531940839 shutdown_connections 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.983+0000 7fa384441640 1 --2- 172.21.15.5:0/531940839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c10a5a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.983+0000 7fa384441640 1 -- 172.21.15.5:0/531940839 >> 172.21.15.5:0/531940839 conn(0x7fa37c101db0 msgr2=0x7fa37c104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.983+0000 7fa384441640 1 -- 172.21.15.5:0/531940839 shutdown_connections 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.983+0000 7fa384441640 1 -- 172.21.15.5:0/531940839 wait complete. 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.983+0000 7fa384441640 1 Processor -- start 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.983+0000 7fa384441640 1 -- start start 2024-09-11T22:12:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.984+0000 7fa384441640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.984+0000 7fa384441640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c10bb20 con 0x7fa37c1081b0 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.984+0000 7fa3821dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.984+0000 7fa3821dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41840/0 (socket says 172.21.15.5:41840) 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.984+0000 7fa3821dd640 1 -- 172.21.15.5:0/565356608 learned_addr learned my addr 172.21.15.5:0/565356608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.984+0000 7fa3821dd640 1 -- 172.21.15.5:0/565356608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c19bc90 con 0x7fa37c1081b0 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.984+0000 7fa3821dd640 1 --2- 172.21.15.5:0/565356608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c19b750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa378031000 tx=0x7fa3780326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.984+0000 7fa3737fe640 1 -- 172.21.15.5:0/565356608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa378042070 con 0x7fa37c1081b0 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.984+0000 7fa3737fe640 1 -- 172.21.15.5:0/565356608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa378032be0 con 0x7fa37c1081b0 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.984+0000 7fa384441640 1 -- 172.21.15.5:0/565356608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c19bf20 con 0x7fa37c1081b0 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.985+0000 7fa3737fe640 1 -- 172.21.15.5:0/565356608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa378037040 con 0x7fa37c1081b0 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.985+0000 7fa384441640 1 -- 172.21.15.5:0/565356608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c19ec10 con 0x7fa37c1081b0 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.985+0000 7fa3737fe640 1 -- 172.21.15.5:0/565356608 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa378045070 con 0x7fa37c1081b0 2024-09-11T22:12:39.714 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.985+0000 7fa3737fe640 1 -- 172.21.15.5:0/565356608 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa378047b20 con 0x7fa37c1081b0 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.986+0000 7fa384441640 1 -- 172.21.15.5:0/565356608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa37c108610 con 0x7fa37c1081b0 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:38.988+0000 7fa3737fe640 1 -- 172.21.15.5:0/565356608 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa378032d90 con 0x7fa37c1081b0 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:39.050+0000 7fa384441640 1 -- 172.21.15.5:0/565356608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa37c19ede0 con 0x7fa37c1081b0 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:39.050+0000 7fa3737fe640 1 -- 172.21.15.5:0/565356608 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fa37803d430 con 0x7fa37c1081b0 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:39.051+0000 7fa384441640 1 -- 172.21.15.5:0/565356608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 msgr2=0x7fa37c19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:39.051+0000 7fa384441640 1 --2- 172.21.15.5:0/565356608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c19b750 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa378031000 tx=0x7fa3780326e0 comp rx=0 tx=0).stop 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:39.052+0000 7fa384441640 1 -- 172.21.15.5:0/565356608 shutdown_connections 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:39.052+0000 7fa384441640 1 --2- 172.21.15.5:0/565356608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37c1081b0 0x7fa37c19b750 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:39.052+0000 7fa384441640 1 -- 172.21.15.5:0/565356608 >> 172.21.15.5:0/565356608 conn(0x7fa37c101db0 msgr2=0x7fa37c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:39.052+0000 7fa384441640 1 -- 172.21.15.5:0/565356608 shutdown_connections 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:39.052+0000 7fa384441640 1 -- 172.21.15.5:0/565356608 wait complete. 2024-09-11T22:12:39.715 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2024-09-11T22:12:40.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:39 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:39.751+0000 7f17ee8d0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:12:41.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:40 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:40.680+0000 7f17ee8d0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:12:41.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:40 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:40.750+0000 7f17ee8d0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:12:41.644 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:41 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:41.518+0000 7f17ee8d0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:12:41.644 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:41 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:41.643+0000 7f17ee8d0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:12:42.032 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:41 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:41.808+0000 7f17ee8d0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:12:42.032 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:41 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:41.874+0000 7f17ee8d0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:12:42.032 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:42.031+0000 7f17ee8d0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:12:42.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:42.103+0000 7f17ee8d0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:12:42.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:42.166+0000 7f17ee8d0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:12:42.623 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:42.333+0000 7f17ee8d0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:12:42.624 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:42.403+0000 7f17ee8d0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:12:42.624 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:42.473+0000 7f17ee8d0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:12:42.624 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:42.560+0000 7f17ee8d0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:12:42.624 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:42.622+0000 7f17ee8d0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:12:42.965 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:42.685+0000 7f17ee8d0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:12:43.820 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:43.516+0000 7f17ee8d0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:12:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: Activating manager daemon x 2024-09-11T22:12:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: mgrmap e2: x(active, starting, since 0.00303491s) 2024-09-11T22:12:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: from='mgr.14100 172.21.15.5:0/2017944388' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:12:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: from='mgr.14100 172.21.15.5:0/2017944388' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:12:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: from='mgr.14100 172.21.15.5:0/2017944388' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:12:43.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: from='mgr.14100 172.21.15.5:0/2017944388' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:12:43.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: from='mgr.14100 172.21.15.5:0/2017944388' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:12:43.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: Manager daemon x is now available 2024-09-11T22:12:43.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: from='mgr.14100 172.21.15.5:0/2017944388' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:12:43.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: from='mgr.14100 172.21.15.5:0/2017944388' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:12:43.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: from='mgr.14100 172.21.15.5:0/2017944388' entity='mgr.x' 2024-09-11T22:12:43.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: from='mgr.14100 172.21.15.5:0/2017944388' entity='mgr.x' 2024-09-11T22:12:43.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:43 smithi005 ceph-mon[30943]: from='mgr.14100 172.21.15.5:0/2017944388' entity='mgr.x' 2024-09-11T22:12:45.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:45 smithi005 ceph-mon[30943]: mgrmap e3: x(active, since 1.00721s) 2024-09-11T22:12:45.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:45 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/47721412' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T22:12:45.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:45.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:12:45.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:12:45.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T22:12:45.623 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:12:45.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T22:12:45.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-11T22:12:23:446697+0000", 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T22:12:45.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T22:12:45.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T22:12:23.447334+0000", 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.776+0000 7fc434f57640 1 Processor -- start 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.776+0000 7fc434f57640 1 -- start start 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.776+0000 7fc434f57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43010a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.776+0000 7fc434f57640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc43010a970 con 0x7fc430109fa0 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.777+0000 7fc42e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43010a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.777+0000 7fc42e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43010a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51490/0 (socket says 172.21.15.5:51490) 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.777+0000 7fc42e59c640 1 -- 172.21.15.5:0/3601244 learned_addr learned my addr 172.21.15.5:0/3601244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:45.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.777+0000 7fc42e59c640 1 -- 172.21.15.5:0/3601244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43010aaf0 con 0x7fc430109fa0 2024-09-11T22:12:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.778+0000 7fc42e59c640 1 --2- 172.21.15.5:0/3601244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43010a3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc420009820 tx=0x7fc42002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fc0e130e121b5e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.778+0000 7fc42d59a640 1 -- 172.21.15.5:0/3601244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc420037070 con 0x7fc430109fa0 2024-09-11T22:12:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.778+0000 7fc42d59a640 1 -- 172.21.15.5:0/3601244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc420032900 con 0x7fc430109fa0 2024-09-11T22:12:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.778+0000 7fc42d59a640 1 -- 172.21.15.5:0/3601244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc420030850 con 0x7fc430109fa0 2024-09-11T22:12:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.779+0000 7fc434f57640 1 -- 172.21.15.5:0/3601244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 msgr2=0x7fc43010a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.779+0000 7fc434f57640 1 --2- 172.21.15.5:0/3601244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43010a3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc420009820 tx=0x7fc42002a270 comp rx=0 tx=0).stop 2024-09-11T22:12:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.779+0000 7fc434f57640 1 -- 172.21.15.5:0/3601244 shutdown_connections 2024-09-11T22:12:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.779+0000 7fc434f57640 1 --2- 172.21.15.5:0/3601244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43010a3a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.779+0000 7fc434f57640 1 -- 172.21.15.5:0/3601244 >> 172.21.15.5:0/3601244 conn(0x7fc43007cfe0 msgr2=0x7fc43007d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:45.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.779+0000 7fc434f57640 1 -- 172.21.15.5:0/3601244 shutdown_connections 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.779+0000 7fc434f57640 1 -- 172.21.15.5:0/3601244 wait complete. 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.780+0000 7fc434f57640 1 Processor -- start 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.780+0000 7fc434f57640 1 -- start start 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.780+0000 7fc434f57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.780+0000 7fc434f57640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc43010df80 con 0x7fc430109fa0 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.780+0000 7fc42e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.780+0000 7fc42e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51498/0 (socket says 172.21.15.5:51498) 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.780+0000 7fc42e59c640 1 -- 172.21.15.5:0/47721412 learned_addr learned my addr 172.21.15.5:0/47721412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.780+0000 7fc42e59c640 1 -- 172.21.15.5:0/47721412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43019de60 con 0x7fc430109fa0 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.780+0000 7fc42e59c640 1 --2- 172.21.15.5:0/47721412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43019d920 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc420009940 tx=0x7fc420032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.781+0000 7fc41f7fe640 1 -- 172.21.15.5:0/47721412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc420042070 con 0x7fc430109fa0 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.781+0000 7fc41f7fe640 1 -- 172.21.15.5:0/47721412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc4200308c0 con 0x7fc430109fa0 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.781+0000 7fc434f57640 1 -- 172.21.15.5:0/47721412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43019e0f0 con 0x7fc430109fa0 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.781+0000 7fc41f7fe640 1 -- 172.21.15.5:0/47721412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc420037040 con 0x7fc430109fa0 2024-09-11T22:12:45.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.781+0000 7fc434f57640 1 -- 172.21.15.5:0/47721412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4301a0de0 con 0x7fc430109fa0 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.782+0000 7fc41f7fe640 1 -- 172.21.15.5:0/47721412 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7fc420045070 con 0x7fc430109fa0 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.782+0000 7fc434f57640 1 -- 172.21.15.5:0/47721412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc43010a420 con 0x7fc430109fa0 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.782+0000 7fc41f7fe640 1 --2- 172.21.15.5:0/47721412 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fc418038e60 0x7fc41803b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.782+0000 7fc41f7fe640 1 -- 172.21.15.5:0/47721412 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc42006bde0 con 0x7fc430109fa0 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.782+0000 7fc42dd9b640 1 --2- 172.21.15.5:0/47721412 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fc418038e60 0x7fc41803b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.783+0000 7fc42dd9b640 1 --2- 172.21.15.5:0/47721412 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fc418038e60 0x7fc41803b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4240098c0 tx=0x7fc424006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:44.787+0000 7fc41f7fe640 1 -- 172.21.15.5:0/47721412 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc42002ed90 con 0x7fc430109fa0 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.029+0000 7fc434f57640 1 -- 172.21.15.5:0/47721412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc4301a1460 con 0x7fc430109fa0 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.029+0000 7fc41f7fe640 1 -- 172.21.15.5:0/47721412 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7fc42003c760 con 0x7fc430109fa0 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.031+0000 7fc434f57640 1 -- 172.21.15.5:0/47721412 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fc418038e60 msgr2=0x7fc41803b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.031+0000 7fc434f57640 1 --2- 172.21.15.5:0/47721412 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fc418038e60 0x7fc41803b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc4240098c0 tx=0x7fc424006db0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.031+0000 7fc434f57640 1 -- 172.21.15.5:0/47721412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 msgr2=0x7fc43019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.031+0000 7fc434f57640 1 --2- 172.21.15.5:0/47721412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43019d920 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc420009940 tx=0x7fc420032ce0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.032+0000 7fc434f57640 1 -- 172.21.15.5:0/47721412 shutdown_connections 2024-09-11T22:12:45.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.032+0000 7fc434f57640 1 --2- 172.21.15.5:0/47721412 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fc418038e60 0x7fc41803b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.032+0000 7fc434f57640 1 --2- 172.21.15.5:0/47721412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc430109fa0 0x7fc43019d920 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:45.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.032+0000 7fc434f57640 1 -- 172.21.15.5:0/47721412 >> 172.21.15.5:0/47721412 conn(0x7fc43007cfe0 msgr2=0x7fc430107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:45.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.032+0000 7fc434f57640 1 -- 172.21.15.5:0/47721412 shutdown_connections 2024-09-11T22:12:45.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:45.032+0000 7fc434f57640 1 -- 172.21.15.5:0/47721412 wait complete. 2024-09-11T22:12:45.632 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2024-09-11T22:12:46.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:46 smithi005 ceph-mon[30943]: mgrmap e4: x(active, since 2s) 2024-09-11T22:12:46.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:46 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1154743276' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T22:12:46.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.024+0000 7ff46238e640 1 Processor -- start 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.024+0000 7ff46238e640 1 -- start start 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.024+0000 7ff46238e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c105e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.024+0000 7ff46238e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c106410 con 0x7ff45c105a40 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.025+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:46.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.025+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c105e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51510/0 (socket says 172.21.15.5:51510) 2024-09-11T22:12:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.025+0000 7ff45bfff640 1 -- 172.21.15.5:0/1075820732 learned_addr learned my addr 172.21.15.5:0/1075820732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.025+0000 7ff45bfff640 1 -- 172.21.15.5:0/1075820732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c108a20 con 0x7ff45c105a40 2024-09-11T22:12:46.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff45bfff640 1 --2- 172.21.15.5:0/1075820732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c105e40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009820 tx=0x7ff44c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d61018121d220e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff45b7fe640 1 -- 172.21.15.5:0/1075820732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c037070 con 0x7ff45c105a40 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff45b7fe640 1 -- 172.21.15.5:0/1075820732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff44c032900 con 0x7ff45c105a40 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff45b7fe640 1 -- 172.21.15.5:0/1075820732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c030850 con 0x7ff45c105a40 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff46238e640 1 -- 172.21.15.5:0/1075820732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 msgr2=0x7ff45c105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff46238e640 1 --2- 172.21.15.5:0/1075820732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c105e40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009820 tx=0x7ff44c02a270 comp rx=0 tx=0).stop 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff46238e640 1 -- 172.21.15.5:0/1075820732 shutdown_connections 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff46238e640 1 --2- 172.21.15.5:0/1075820732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c105e40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff46238e640 1 -- 172.21.15.5:0/1075820732 >> 172.21.15.5:0/1075820732 conn(0x7ff45c1015f0 msgr2=0x7ff45c103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff46238e640 1 -- 172.21.15.5:0/1075820732 shutdown_connections 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.026+0000 7ff46238e640 1 -- 172.21.15.5:0/1075820732 wait complete. 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.027+0000 7ff46238e640 1 Processor -- start 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.027+0000 7ff46238e640 1 -- start start 2024-09-11T22:12:46.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.027+0000 7ff46238e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.027+0000 7ff46238e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c10bc50 con 0x7ff45c105a40 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.027+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.027+0000 7ff45bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51512/0 (socket says 172.21.15.5:51512) 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.027+0000 7ff45bfff640 1 -- 172.21.15.5:0/1154743276 learned_addr learned my addr 172.21.15.5:0/1154743276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.028+0000 7ff45bfff640 1 -- 172.21.15.5:0/1154743276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c19bcc0 con 0x7ff45c105a40 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.028+0000 7ff45bfff640 1 --2- 172.21.15.5:0/1154743276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c19b780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0097f0 tx=0x7ff44c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.028+0000 7ff459ffb640 1 -- 172.21.15.5:0/1154743276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c037040 con 0x7ff45c105a40 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.028+0000 7ff46238e640 1 -- 172.21.15.5:0/1154743276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c19bf50 con 0x7ff45c105a40 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.028+0000 7ff459ffb640 1 -- 172.21.15.5:0/1154743276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff44c040070 con 0x7ff45c105a40 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.028+0000 7ff459ffb640 1 -- 172.21.15.5:0/1154743276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff44c03a390 con 0x7ff45c105a40 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.028+0000 7ff46238e640 1 -- 172.21.15.5:0/1154743276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c19ec40 con 0x7ff45c105a40 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.029+0000 7ff459ffb640 1 -- 172.21.15.5:0/1154743276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7ff44c03b430 con 0x7ff45c105a40 2024-09-11T22:12:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.029+0000 7ff459ffb640 1 --2- 172.21.15.5:0/1154743276 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7ff448038f20 0x7ff44803b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.029+0000 7ff46238e640 1 -- 172.21.15.5:0/1154743276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c105ec0 con 0x7ff45c105a40 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.029+0000 7ff459ffb640 1 -- 172.21.15.5:0/1154743276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff44c06c0d0 con 0x7ff45c105a40 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.029+0000 7ff453fff640 1 --2- 172.21.15.5:0/1154743276 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7ff448038f20 0x7ff44803b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.030+0000 7ff453fff640 1 --2- 172.21.15.5:0/1154743276 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7ff448038f20 0x7ff44803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4440098c0 tx=0x7ff444006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.034+0000 7ff459ffb640 1 -- 172.21.15.5:0/1154743276 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff44c03a5e0 con 0x7ff45c105a40 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.212+0000 7ff46238e640 1 -- 172.21.15.5:0/1154743276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff45c19ee10 con 0x7ff45c105a40 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.213+0000 7ff459ffb640 1 -- 172.21.15.5:0/1154743276 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+427 (secure 0 0 0) 0x7ff44c060030 con 0x7ff45c105a40 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.216+0000 7ff46238e640 1 -- 172.21.15.5:0/1154743276 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7ff448038f20 msgr2=0x7ff44803b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.216+0000 7ff46238e640 1 --2- 172.21.15.5:0/1154743276 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7ff448038f20 0x7ff44803b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff4440098c0 tx=0x7ff444006db0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.216+0000 7ff46238e640 1 -- 172.21.15.5:0/1154743276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 msgr2=0x7ff45c19b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.216+0000 7ff46238e640 1 --2- 172.21.15.5:0/1154743276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c19b780 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0097f0 tx=0x7ff44c030d80 comp rx=0 tx=0).stop 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.216+0000 7ff46238e640 1 -- 172.21.15.5:0/1154743276 shutdown_connections 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.216+0000 7ff46238e640 1 --2- 172.21.15.5:0/1154743276 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7ff448038f20 0x7ff44803b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.216+0000 7ff46238e640 1 --2- 172.21.15.5:0/1154743276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff45c105a40 0x7ff45c19b780 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:46.767 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.216+0000 7ff46238e640 1 -- 172.21.15.5:0/1154743276 >> 172.21.15.5:0/1154743276 conn(0x7ff45c1015f0 msgr2=0x7ff45c102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:46.767 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.216+0000 7ff46238e640 1 -- 172.21.15.5:0/1154743276 shutdown_connections 2024-09-11T22:12:46.767 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:46.216+0000 7ff46238e640 1 -- 172.21.15.5:0/1154743276 wait complete. 2024-09-11T22:12:46.767 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2024-09-11T22:12:47.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:47 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3030035492' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T22:12:47.670 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: ignoring --setuser ceph since I am not root 2024-09-11T22:12:47.671 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: ignoring --setgroup ceph since I am not root 2024-09-11T22:12:47.671 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:47.669+0000 7f071fbff040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:12:47.953 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:47.728+0000 7f071fbff040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:12:47.953 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:47.882+0000 7f071fbff040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:12:47.953 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:47.952+0000 7f071fbff040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.163+0000 7f945ae2e640 1 Processor -- start 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.163+0000 7f945ae2e640 1 -- start start 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.163+0000 7f945ae2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f9454107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.163+0000 7f945ae2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9454108170 con 0x7f94541077a0 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.163+0000 7f9458bca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f9454107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.163+0000 7f9458bca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f9454107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51528/0 (socket says 172.21.15.5:51528) 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.163+0000 7f9458bca640 1 -- 172.21.15.5:0/972513618 learned_addr learned my addr 172.21.15.5:0/972513618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.163+0000 7f9458bca640 1 -- 172.21.15.5:0/972513618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9454108950 con 0x7f94541077a0 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.164+0000 7f9458bca640 1 --2- 172.21.15.5:0/972513618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f9454107ba0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9448009820 tx=0x7f944802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e06bca2207c1c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.164+0000 7f94537fe640 1 -- 172.21.15.5:0/972513618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9448037070 con 0x7f94541077a0 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.164+0000 7f94537fe640 1 -- 172.21.15.5:0/972513618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9448032900 con 0x7f94541077a0 2024-09-11T22:12:48.172 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.164+0000 7f94537fe640 1 -- 172.21.15.5:0/972513618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9448030850 con 0x7f94541077a0 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.165+0000 7f945ae2e640 1 -- 172.21.15.5:0/972513618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 msgr2=0x7f9454107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.165+0000 7f945ae2e640 1 --2- 172.21.15.5:0/972513618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f9454107ba0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9448009820 tx=0x7f944802a270 comp rx=0 tx=0).stop 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.165+0000 7f945ae2e640 1 -- 172.21.15.5:0/972513618 shutdown_connections 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.165+0000 7f945ae2e640 1 --2- 172.21.15.5:0/972513618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f9454107ba0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.165+0000 7f945ae2e640 1 -- 172.21.15.5:0/972513618 >> 172.21.15.5:0/972513618 conn(0x7f9454102fd0 msgr2=0x7f94541053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.165+0000 7f945ae2e640 1 -- 172.21.15.5:0/972513618 shutdown_connections 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.165+0000 7f945ae2e640 1 -- 172.21.15.5:0/972513618 wait complete. 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.166+0000 7f945ae2e640 1 Processor -- start 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.166+0000 7f945ae2e640 1 -- start start 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.166+0000 7f945ae2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f945407e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.166+0000 7f945ae2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f945410bd00 con 0x7f94541077a0 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.166+0000 7f9458bca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f945407e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.166+0000 7f9458bca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f945407e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51538/0 (socket says 172.21.15.5:51538) 2024-09-11T22:12:48.173 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.166+0000 7f9458bca640 1 -- 172.21.15.5:0/3030035492 learned_addr learned my addr 172.21.15.5:0/3030035492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.166+0000 7f9458bca640 1 -- 172.21.15.5:0/3030035492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945407e910 con 0x7f94541077a0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.166+0000 7f9458bca640 1 --2- 172.21.15.5:0/3030035492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f945407e3d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94480097f0 tx=0x7f9448030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.167+0000 7f9451ffb640 1 -- 172.21.15.5:0/3030035492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9448037040 con 0x7f94541077a0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.167+0000 7f9451ffb640 1 -- 172.21.15.5:0/3030035492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9448040070 con 0x7f94541077a0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.167+0000 7f945ae2e640 1 -- 172.21.15.5:0/3030035492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945407cb00 con 0x7f94541077a0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.167+0000 7f9451ffb640 1 -- 172.21.15.5:0/3030035492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f944803a390 con 0x7f94541077a0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.167+0000 7f945ae2e640 1 -- 172.21.15.5:0/3030035492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945407cfe0 con 0x7f94541077a0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.168+0000 7f9451ffb640 1 -- 172.21.15.5:0/3030035492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f944803a5e0 con 0x7f94541077a0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.168+0000 7f9451ffb640 1 --2- 172.21.15.5:0/3030035492 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f9444038f70 0x7f944403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.168+0000 7f9451ffb640 1 -- 172.21.15.5:0/3030035492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f944806c2e0 con 0x7f94541077a0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.168+0000 7f9453fff640 1 --2- 172.21.15.5:0/3030035492 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f9444038f70 0x7f944403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.168+0000 7f945ae2e640 1 -- 172.21.15.5:0/3030035492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f945410be80 con 0x7f94541077a0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.168+0000 7f9453fff640 1 --2- 172.21.15.5:0/3030035492 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f9444038f70 0x7f944403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f94400098c0 tx=0x7f9440006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:48.174 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.173+0000 7f9451ffb640 1 -- 172.21.15.5:0/3030035492 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9448043760 con 0x7f94541077a0 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.400+0000 7f945ae2e640 1 -- 172.21.15.5:0/3030035492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f945407d6f0 con 0x7f94541077a0 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.527+0000 7f9451ffb640 1 -- 172.21.15.5:0/3030035492 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f944806c040 con 0x7f94541077a0 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.528+0000 7f9451ffb640 1 -- 172.21.15.5:0/3030035492 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f944802fd40 con 0x7f94541077a0 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.530+0000 7f945ae2e640 1 -- 172.21.15.5:0/3030035492 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f9444038f70 msgr2=0x7f944403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.530+0000 7f945ae2e640 1 --2- 172.21.15.5:0/3030035492 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f9444038f70 0x7f944403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f94400098c0 tx=0x7f9440006db0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.530+0000 7f945ae2e640 1 -- 172.21.15.5:0/3030035492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 msgr2=0x7f945407e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.530+0000 7f945ae2e640 1 --2- 172.21.15.5:0/3030035492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f945407e3d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f94480097f0 tx=0x7f9448030d80 comp rx=0 tx=0).stop 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.530+0000 7f945ae2e640 1 -- 172.21.15.5:0/3030035492 shutdown_connections 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.530+0000 7f945ae2e640 1 --2- 172.21.15.5:0/3030035492 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f9444038f70 0x7f944403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.530+0000 7f945ae2e640 1 --2- 172.21.15.5:0/3030035492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94541077a0 0x7f945407e3d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.530+0000 7f945ae2e640 1 -- 172.21.15.5:0/3030035492 >> 172.21.15.5:0/3030035492 conn(0x7f9454102fd0 msgr2=0x7f9454103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.530+0000 7f945ae2e640 1 -- 172.21.15.5:0/3030035492 shutdown_connections 2024-09-11T22:12:48.175 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:47.530+0000 7f945ae2e640 1 -- 172.21.15.5:0/3030035492 wait complete. 2024-09-11T22:12:48.255 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:48 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:48.106+0000 7f071fbff040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:12:48.256 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:48 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:48.255+0000 7f071fbff040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:12:48.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:48 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3030035492' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T22:12:48.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:48 smithi005 ceph-mon[30943]: mgrmap e5: x(active, since 4s) 2024-09-11T22:12:48.789 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:48 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:48.591+0000 7f071fbff040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:12:48.789 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:48 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:48.655+0000 7f071fbff040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:12:49.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:48 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:48.852+0000 7f071fbff040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.555+0000 7fb470c91640 1 Processor -- start 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.555+0000 7fb470c91640 1 -- start start 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.555+0000 7fb470c91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.555+0000 7fb470c91640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c107010 con 0x7fb46c106640 2024-09-11T22:12:49.447 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.555+0000 7fb46a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.555+0000 7fb46a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51560/0 (socket says 172.21.15.5:51560) 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.555+0000 7fb46a59c640 1 -- 172.21.15.5:0/2923912435 learned_addr learned my addr 172.21.15.5:0/2923912435 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.556+0000 7fb46a59c640 1 -- 172.21.15.5:0/2923912435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c1077f0 con 0x7fb46c106640 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.556+0000 7fb46a59c640 1 --2- 172.21.15.5:0/2923912435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c106a40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009820 tx=0x7fb45c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336fb6320886f883 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.557+0000 7fb46959a640 1 -- 172.21.15.5:0/2923912435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c037070 con 0x7fb46c106640 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.557+0000 7fb46959a640 1 -- 172.21.15.5:0/2923912435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb45c032900 con 0x7fb46c106640 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.557+0000 7fb46959a640 1 -- 172.21.15.5:0/2923912435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c030850 con 0x7fb46c106640 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.557+0000 7fb470c91640 1 -- 172.21.15.5:0/2923912435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 msgr2=0x7fb46c106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.557+0000 7fb470c91640 1 --2- 172.21.15.5:0/2923912435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c106a40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009820 tx=0x7fb45c02a270 comp rx=0 tx=0).stop 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.557+0000 7fb470c91640 1 -- 172.21.15.5:0/2923912435 shutdown_connections 2024-09-11T22:12:49.448 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.557+0000 7fb470c91640 1 --2- 172.21.15.5:0/2923912435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c106a40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:49.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.558+0000 7fb470c91640 1 -- 172.21.15.5:0/2923912435 >> 172.21.15.5:0/2923912435 conn(0x7fb46c101db0 msgr2=0x7fb46c104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:49.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.558+0000 7fb470c91640 1 -- 172.21.15.5:0/2923912435 shutdown_connections 2024-09-11T22:12:49.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.558+0000 7fb470c91640 1 -- 172.21.15.5:0/2923912435 wait complete. 2024-09-11T22:12:49.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.558+0000 7fb470c91640 1 Processor -- start 2024-09-11T22:12:49.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.558+0000 7fb470c91640 1 -- start start 2024-09-11T22:12:49.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.558+0000 7fb470c91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c07ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:49.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.558+0000 7fb470c91640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c10aba0 con 0x7fb46c106640 2024-09-11T22:12:49.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.559+0000 7fb46a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c07ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:49.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.559+0000 7fb46a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c07ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51568/0 (socket says 172.21.15.5:51568) 2024-09-11T22:12:49.449 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.559+0000 7fb46a59c640 1 -- 172.21.15.5:0/968462049 learned_addr learned my addr 172.21.15.5:0/968462049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.559+0000 7fb46a59c640 1 -- 172.21.15.5:0/968462049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c19e4b0 con 0x7fb46c106640 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.559+0000 7fb46a59c640 1 --2- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c07ba60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0097f0 tx=0x7fb45c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.559+0000 7fb45b7fe640 1 -- 172.21.15.5:0/968462049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c042070 con 0x7fb46c106640 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.559+0000 7fb45b7fe640 1 -- 172.21.15.5:0/968462049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb45c032ca0 con 0x7fb46c106640 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.559+0000 7fb45b7fe640 1 -- 172.21.15.5:0/968462049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb45c037040 con 0x7fb46c106640 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.559+0000 7fb470c91640 1 -- 172.21.15.5:0/968462049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c19e680 con 0x7fb46c106640 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.559+0000 7fb470c91640 1 -- 172.21.15.5:0/968462049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c19eaa0 con 0x7fb46c106640 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.560+0000 7fb45b7fe640 1 -- 172.21.15.5:0/968462049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb45c045070 con 0x7fb46c106640 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.560+0000 7fb45b7fe640 1 --2- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fb454038f70 0x7fb45403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.560+0000 7fb45b7fe640 1 -- 172.21.15.5:0/968462049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb45c06bf00 con 0x7fb46c106640 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.560+0000 7fb469d9b640 1 -- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fb454038f70 msgr2=0x7fb45403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1194241380 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.560+0000 7fb469d9b640 1 --2- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fb454038f70 0x7fb45403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.561+0000 7fb470c91640 1 -- 172.21.15.5:0/968462049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46c10ad20 con 0x7fb46c106640 2024-09-11T22:12:49.450 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.565+0000 7fb45b7fe640 1 -- 172.21.15.5:0/968462049 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb45c06b920 con 0x7fb46c106640 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.761+0000 7fb469d9b640 1 -- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fb454038f70 msgr2=0x7fb45403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1194241380 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.761+0000 7fb469d9b640 1 --2- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fb454038f70 0x7fb45403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.788+0000 7fb470c91640 1 -- 172.21.15.5:0/968462049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb46c19f140 con 0x7fb46c106640 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.788+0000 7fb45b7fe640 1 -- 172.21.15.5:0/968462049 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb45c03ccd0 con 0x7fb46c106640 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.790+0000 7fb470c91640 1 -- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fb454038f70 msgr2=0x7fb45403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.790+0000 7fb470c91640 1 --2- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fb454038f70 0x7fb45403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.790+0000 7fb470c91640 1 -- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 msgr2=0x7fb46c07ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.790+0000 7fb470c91640 1 --2- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c07ba60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0097f0 tx=0x7fb45c030ec0 comp rx=0 tx=0).stop 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.790+0000 7fb470c91640 1 -- 172.21.15.5:0/968462049 shutdown_connections 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.790+0000 7fb470c91640 1 --2- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7fb454038f70 0x7fb45403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.790+0000 7fb470c91640 1 --2- 172.21.15.5:0/968462049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb46c106640 0x7fb46c07ba60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:49.451 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.790+0000 7fb470c91640 1 -- 172.21.15.5:0/968462049 >> 172.21.15.5:0/968462049 conn(0x7fb46c101db0 msgr2=0x7fb46c102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:49.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.791+0000 7fb470c91640 1 -- 172.21.15.5:0/968462049 shutdown_connections 2024-09-11T22:12:49.452 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:48.791+0000 7fb470c91640 1 -- 172.21.15.5:0/968462049 wait complete. 2024-09-11T22:12:49.452 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-09-11T22:12:49.452 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2024-09-11T22:12:49.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:49 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/968462049' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T22:12:50.044 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:49 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:49.784+0000 7f071fbff040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:12:50.044 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:49 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:49.856+0000 7f071fbff040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:12:50.722 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:50 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:50.601+0000 7f071fbff040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:12:50.722 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:50 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:50.721+0000 7f071fbff040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:12:51.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:50 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:50.869+0000 7f071fbff040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:12:51.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:50 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:50.933+0000 7f071fbff040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:12:51.388 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:51.089+0000 7f071fbff040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:12:51.388 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:51.159+0000 7f071fbff040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:12:51.388 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:51.221+0000 7f071fbff040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:12:51.388 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:51.387+0000 7f071fbff040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:12:51.672 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:51.455+0000 7f071fbff040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:12:51.673 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:51.524+0000 7f071fbff040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:12:51.673 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:51.610+0000 7f071fbff040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:12:51.673 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:51.671+0000 7f071fbff040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:12:52.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:51.733+0000 7f071fbff040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:12:52.820 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:52.566+0000 7f071fbff040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:12:52.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: Active manager daemon x restarted 2024-09-11T22:12:52.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: Activating manager daemon x 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: mgrmap e6: x(active, starting, since 0.00318579s) 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: Manager daemon x is now available 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:12:52.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:12:52.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:52 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:12:54.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:12:54.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-11T22:12:54.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T22:12:54.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:12:54.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.837+0000 7f4f3f518640 1 Processor -- start 2024-09-11T22:12:54.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.837+0000 7f4f3f518640 1 -- start start 2024-09-11T22:12:54.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.837+0000 7f4f3f518640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f38106440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:54.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.837+0000 7f4f3f518640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3807c690 con 0x7f4f38106040 2024-09-11T22:12:54.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.837+0000 7f4f3d2b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f38106440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:54.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.837+0000 7f4f3d2b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f38106440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51570/0 (socket says 172.21.15.5:51570) 2024-09-11T22:12:54.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.837+0000 7f4f3d2b4640 1 -- 172.21.15.5:0/2009490210 learned_addr learned my addr 172.21.15.5:0/2009490210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:54.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.838+0000 7f4f3d2b4640 1 -- 172.21.15.5:0/2009490210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f38106980 con 0x7f4f38106040 2024-09-11T22:12:54.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.838+0000 7f4f3d2b4640 1 --2- 172.21.15.5:0/2009490210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f38106440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009820 tx=0x7f4f3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5374a497cff2b824 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:54.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.838+0000 7f4f2ffff640 1 -- 172.21.15.5:0/2009490210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f34037070 con 0x7f4f38106040 2024-09-11T22:12:54.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.839+0000 7f4f2ffff640 1 -- 172.21.15.5:0/2009490210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f34032900 con 0x7f4f38106040 2024-09-11T22:12:54.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.839+0000 7f4f2ffff640 1 -- 172.21.15.5:0/2009490210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f34030850 con 0x7f4f38106040 2024-09-11T22:12:54.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.839+0000 7f4f3f518640 1 -- 172.21.15.5:0/2009490210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 msgr2=0x7f4f38106440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:54.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.839+0000 7f4f3f518640 1 --2- 172.21.15.5:0/2009490210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f38106440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009820 tx=0x7f4f3402a270 comp rx=0 tx=0).stop 2024-09-11T22:12:54.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.839+0000 7f4f3f518640 1 -- 172.21.15.5:0/2009490210 shutdown_connections 2024-09-11T22:12:54.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.839+0000 7f4f3f518640 1 --2- 172.21.15.5:0/2009490210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f38106440 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.839+0000 7f4f3f518640 1 -- 172.21.15.5:0/2009490210 >> 172.21.15.5:0/2009490210 conn(0x7f4f38101e50 msgr2=0x7f4f38104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:54.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.839+0000 7f4f3f518640 1 -- 172.21.15.5:0/2009490210 shutdown_connections 2024-09-11T22:12:54.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.839+0000 7f4f3f518640 1 -- 172.21.15.5:0/2009490210 wait complete. 2024-09-11T22:12:54.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.840+0000 7f4f3f518640 1 Processor -- start 2024-09-11T22:12:54.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.840+0000 7f4f3f518640 1 -- start start 2024-09-11T22:12:54.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.840+0000 7f4f3f518640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f3819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:54.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.840+0000 7f4f3f518640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f38107a80 con 0x7f4f38106040 2024-09-11T22:12:54.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.840+0000 7f4f3d2b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f3819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:54.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.840+0000 7f4f3d2b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f3819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51584/0 (socket says 172.21.15.5:51584) 2024-09-11T22:12:54.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.840+0000 7f4f3d2b4640 1 -- 172.21.15.5:0/2712607798 learned_addr learned my addr 172.21.15.5:0/2712607798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:54.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.841+0000 7f4f3d2b4640 1 -- 172.21.15.5:0/2712607798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3819dea0 con 0x7f4f38106040 2024-09-11T22:12:54.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.841+0000 7f4f3d2b4640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f3819d960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f34032ce0 tx=0x7f4f34032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:54.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.841+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f34042070 con 0x7f4f38106040 2024-09-11T22:12:54.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.841+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4f340308c0 con 0x7f4f38106040 2024-09-11T22:12:54.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.841+0000 7f4f3f518640 1 -- 172.21.15.5:0/2712607798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3819e130 con 0x7f4f38106040 2024-09-11T22:12:54.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.841+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f34037040 con 0x7f4f38106040 2024-09-11T22:12:54.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.841+0000 7f4f3f518640 1 -- 172.21.15.5:0/2712607798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f381a0e20 con 0x7f4f38106040 2024-09-11T22:12:54.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.842+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f4f34045070 con 0x7f4f38106040 2024-09-11T22:12:54.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.842+0000 7f4f2e7fc640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 0x7f4f2403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:54.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.842+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 --> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f2403bb40 con 0x7f4f24038f70 2024-09-11T22:12:54.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.842+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4f3406c1c0 con 0x7f4f38106040 2024-09-11T22:12:54.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.842+0000 7f4f3cab3640 1 -- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 msgr2=0x7f4f2403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1194241380 2024-09-11T22:12:54.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:49.842+0000 7f4f3cab3640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 0x7f4f2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:12:54.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:50.043+0000 7f4f3cab3640 1 -- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 msgr2=0x7f4f2403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1194241380 2024-09-11T22:12:54.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:50.043+0000 7f4f3cab3640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 0x7f4f2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:12:54.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:50.444+0000 7f4f3cab3640 1 -- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 msgr2=0x7f4f2403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1194241380 2024-09-11T22:12:54.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:50.444+0000 7f4f3cab3640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 0x7f4f2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T22:12:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:51.245+0000 7f4f3cab3640 1 -- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 msgr2=0x7f4f2403b430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1194241380 2024-09-11T22:12:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:51.245+0000 7f4f3cab3640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 0x7f4f2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T22:12:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:52.570+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f4f3406ac80 con 0x7f4f38106040 2024-09-11T22:12:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:52.570+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 msgr2=0x7f4f2403b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:12:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:52.570+0000 7f4f2e7fc640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 0x7f4f2403b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.570+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f4f3403cd00 con 0x7f4f38106040 2024-09-11T22:12:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.570+0000 7f4f2e7fc640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4f2403c8a0 0x7f4f2403ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.570+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 --> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f34038e50 con 0x7f4f2403c8a0 2024-09-11T22:12:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.571+0000 7f4f3cab3640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4f2403c8a0 0x7f4f2403ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.571+0000 7f4f3cab3640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4f2403c8a0 0x7f4f2403ec90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4f28003980 tx=0x7f4f28006e10 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:54.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.572+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 <== mgr.14116 v2:172.21.15.5:6800/3503484776 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f4f34038e50 con 0x7f4f2403c8a0 2024-09-11T22:12:54.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f3f518640 1 -- 172.21.15.5:0/2712607798 --> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4f38106440 con 0x7f4f2403c8a0 2024-09-11T22:12:54.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f2e7fc640 1 -- 172.21.15.5:0/2712607798 <== mgr.14116 v2:172.21.15.5:6800/3503484776 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f4f38106440 con 0x7f4f2403c8a0 2024-09-11T22:12:54.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f3f518640 1 -- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4f2403c8a0 msgr2=0x7f4f2403ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:54.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f3f518640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4f2403c8a0 0x7f4f2403ec90 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4f28003980 tx=0x7f4f28006e10 comp rx=0 tx=0).stop 2024-09-11T22:12:54.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f3f518640 1 -- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 msgr2=0x7f4f3819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:54.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f3f518640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f3819d960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f34032ce0 tx=0x7f4f34032d10 comp rx=0 tx=0).stop 2024-09-11T22:12:54.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f3f518640 1 -- 172.21.15.5:0/2712607798 shutdown_connections 2024-09-11T22:12:54.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f3f518640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4f2403c8a0 0x7f4f2403ec90 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f3f518640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:6800/1194241380,v1:172.21.15.5:6801/1194241380] conn(0x7f4f24038f70 0x7f4f2403b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f3f518640 1 --2- 172.21.15.5:0/2712607798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f38106040 0x7f4f3819d960 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:54.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.579+0000 7f4f3f518640 1 -- 172.21.15.5:0/2712607798 >> 172.21.15.5:0/2712607798 conn(0x7f4f38101e50 msgr2=0x7f4f38191c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:54.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.580+0000 7f4f3f518640 1 -- 172.21.15.5:0/2712607798 shutdown_connections 2024-09-11T22:12:54.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:53.580+0000 7f4f3f518640 1 -- 172.21.15.5:0/2712607798 wait complete. 2024-09-11T22:12:54.156 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2024-09-11T22:12:54.156 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2024-09-11T22:12:54.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:54 smithi005 ceph-mon[30943]: Found migration_current of "None". Setting to last migration. 2024-09-11T22:12:54.751 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:54 smithi005 ceph-mon[30943]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T22:12:54.751 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:54 smithi005 ceph-mon[30943]: mgrmap e7: x(active, since 1.00529s) 2024-09-11T22:12:54.751 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:54 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:12:54.751 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:54 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:12:55.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.543+0000 7f4e0f6ed640 1 Processor -- start 2024-09-11T22:12:55.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.544+0000 7f4e0f6ed640 1 -- start start 2024-09-11T22:12:55.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.544+0000 7f4e0f6ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e08106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.544+0000 7f4e0f6ed640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08106fd0 con 0x7f4e08106600 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.544+0000 7f4e0d489640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e08106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.544+0000 7f4e0d489640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e08106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54606/0 (socket says 172.21.15.5:54606) 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.544+0000 7f4e0d489640 1 -- 172.21.15.5:0/1909611501 learned_addr learned my addr 172.21.15.5:0/1909611501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.544+0000 7f4e0d489640 1 -- 172.21.15.5:0/1909611501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e08107800 con 0x7f4e08106600 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.545+0000 7f4e0d489640 1 --2- 172.21.15.5:0/1909611501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e08106a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009820 tx=0x7f4e0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3471cf0b66075c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.545+0000 7f4dfffff640 1 -- 172.21.15.5:0/1909611501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e04037070 con 0x7f4e08106600 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.545+0000 7f4dfffff640 1 -- 172.21.15.5:0/1909611501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e04032900 con 0x7f4e08106600 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.545+0000 7f4dfffff640 1 -- 172.21.15.5:0/1909611501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e04030850 con 0x7f4e08106600 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.546+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1909611501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 msgr2=0x7f4e08106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.546+0000 7f4e0f6ed640 1 --2- 172.21.15.5:0/1909611501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e08106a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009820 tx=0x7f4e0402a270 comp rx=0 tx=0).stop 2024-09-11T22:12:55.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.546+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1909611501 shutdown_connections 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.546+0000 7f4e0f6ed640 1 --2- 172.21.15.5:0/1909611501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e08106a00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.546+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1909611501 >> 172.21.15.5:0/1909611501 conn(0x7f4e08101db0 msgr2=0x7f4e081041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.546+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1909611501 shutdown_connections 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.546+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1909611501 wait complete. 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.546+0000 7f4e0f6ed640 1 Processor -- start 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4e0f6ed640 1 -- start start 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4e0f6ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e0819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4e0f6ed640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0810abb0 con 0x7f4e08106600 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4e0d489640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e0819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4e0d489640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e0819b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54612/0 (socket says 172.21.15.5:54612) 2024-09-11T22:12:55.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4e0d489640 1 -- 172.21.15.5:0/1985118422 learned_addr learned my addr 172.21.15.5:0/1985118422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:55.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4e0d489640 1 -- 172.21.15.5:0/1985118422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0819bd60 con 0x7f4e08106600 2024-09-11T22:12:55.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4e0d489640 1 --2- 172.21.15.5:0/1985118422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e0819b820 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4e040097f0 tx=0x7f4e04030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:55.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4dfe7fc640 1 -- 172.21.15.5:0/1985118422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e04042070 con 0x7f4e08106600 2024-09-11T22:12:55.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4dfe7fc640 1 -- 172.21.15.5:0/1985118422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e04032ca0 con 0x7f4e08106600 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1985118422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0819bff0 con 0x7f4e08106600 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.547+0000 7f4dfe7fc640 1 -- 172.21.15.5:0/1985118422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e04037040 con 0x7f4e08106600 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.548+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1985118422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0819ece0 con 0x7f4e08106600 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.549+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1985118422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0810ad30 con 0x7f4e08106600 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.549+0000 7f4dfe7fc640 1 -- 172.21.15.5:0/1985118422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f4e04045070 con 0x7f4e08106600 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.549+0000 7f4dfe7fc640 1 --2- 172.21.15.5:0/1985118422 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4df4038e60 0x7f4df403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.549+0000 7f4dfe7fc640 1 -- 172.21.15.5:0/1985118422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4e0406bd70 con 0x7f4e08106600 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.549+0000 7f4e0cc88640 1 --2- 172.21.15.5:0/1985118422 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4df4038e60 0x7f4df403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.549+0000 7f4e0cc88640 1 --2- 172.21.15.5:0/1985118422 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4df4038e60 0x7f4df403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4df80098c0 tx=0x7f4df8006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.554+0000 7f4dfe7fc640 1 -- 172.21.15.5:0/1985118422 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e0403d430 con 0x7f4e08106600 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.748+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1985118422 --> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f4e0802da30 con 0x7f4df4038e60 2024-09-11T22:12:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.754+0000 7f4dfe7fc640 1 -- 172.21.15.5:0/1985118422 <== mgr.14116 v2:172.21.15.5:6800/3503484776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4e0802da30 con 0x7f4df4038e60 2024-09-11T22:12:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.756+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1985118422 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4df4038e60 msgr2=0x7f4df403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.756+0000 7f4e0f6ed640 1 --2- 172.21.15.5:0/1985118422 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4df4038e60 0x7f4df403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4df80098c0 tx=0x7f4df8006db0 comp rx=0 tx=0).stop 2024-09-11T22:12:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.756+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1985118422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 msgr2=0x7f4e0819b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.756+0000 7f4e0f6ed640 1 --2- 172.21.15.5:0/1985118422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e0819b820 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4e040097f0 tx=0x7f4e04030ec0 comp rx=0 tx=0).stop 2024-09-11T22:12:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.756+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1985118422 shutdown_connections 2024-09-11T22:12:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.756+0000 7f4e0f6ed640 1 --2- 172.21.15.5:0/1985118422 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f4df4038e60 0x7f4df403b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.756+0000 7f4e0f6ed640 1 --2- 172.21.15.5:0/1985118422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e08106600 0x7f4e0819b820 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.756+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1985118422 >> 172.21.15.5:0/1985118422 conn(0x7f4e08101db0 msgr2=0x7f4e08102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.756+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1985118422 shutdown_connections 2024-09-11T22:12:55.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:54.756+0000 7f4e0f6ed640 1 -- 172.21.15.5:0/1985118422 wait complete. 2024-09-11T22:12:55.865 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:55 smithi005 ceph-mon[30943]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T22:12:55.865 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:55 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:12:55.865 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:55 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:12:55.865 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:55 smithi005 ceph-mon[30943]: mgrmap e8: x(active, since 2s) 2024-09-11T22:12:56.516 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:56.329+0000 7f068c724640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:22:12:56] ENGINE Error in HTTPServer.serve 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: Traceback (most recent call last): 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self._connections.run(self.expiration_interval) 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self._run(expiration_interval) 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:12:56.517 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: s = self.context.wrap_socket( 2024-09-11T22:12:56.518 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:12:56.518 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: return self.sslsocket_class._create( 2024-09-11T22:12:56.518 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:12:56.518 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self.do_handshake() 2024-09-11T22:12:56.518 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:12:56.518 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self._sslobj.do_handshake() 2024-09-11T22:12:56.518 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:12:56.518 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:12:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T22:12:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.673+0000 7fca9d751640 1 Processor -- start 2024-09-11T22:12:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.673+0000 7fca9d751640 1 -- start start 2024-09-11T22:12:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.673+0000 7fca9d751640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca98107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.673+0000 7fca9d751640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca98108170 con 0x7fca981077a0 2024-09-11T22:12:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.673+0000 7fca96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca98107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.674+0000 7fca96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca98107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54624/0 (socket says 172.21.15.5:54624) 2024-09-11T22:12:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.674+0000 7fca96ffd640 1 -- 172.21.15.5:0/710596902 learned_addr learned my addr 172.21.15.5:0/710596902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.674+0000 7fca96ffd640 1 -- 172.21.15.5:0/710596902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca98108950 con 0x7fca981077a0 2024-09-11T22:12:56.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.674+0000 7fca96ffd640 1 --2- 172.21.15.5:0/710596902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca98107ba0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fca88009820 tx=0x7fca8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478586c1008f06c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.675+0000 7fca967fc640 1 -- 172.21.15.5:0/710596902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca88037070 con 0x7fca981077a0 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.675+0000 7fca967fc640 1 -- 172.21.15.5:0/710596902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fca88032900 con 0x7fca981077a0 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.675+0000 7fca967fc640 1 -- 172.21.15.5:0/710596902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca88030850 con 0x7fca981077a0 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.675+0000 7fca9d751640 1 -- 172.21.15.5:0/710596902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 msgr2=0x7fca98107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.675+0000 7fca9d751640 1 --2- 172.21.15.5:0/710596902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca98107ba0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fca88009820 tx=0x7fca8802a270 comp rx=0 tx=0).stop 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.675+0000 7fca9d751640 1 -- 172.21.15.5:0/710596902 shutdown_connections 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.675+0000 7fca9d751640 1 --2- 172.21.15.5:0/710596902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca98107ba0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.675+0000 7fca9d751640 1 -- 172.21.15.5:0/710596902 >> 172.21.15.5:0/710596902 conn(0x7fca98102fd0 msgr2=0x7fca981053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.675+0000 7fca9d751640 1 -- 172.21.15.5:0/710596902 shutdown_connections 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.675+0000 7fca9d751640 1 -- 172.21.15.5:0/710596902 wait complete. 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.676+0000 7fca9d751640 1 Processor -- start 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.676+0000 7fca9d751640 1 -- start start 2024-09-11T22:12:56.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.676+0000 7fca9d751640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca9819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.676+0000 7fca9d751640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca9810bd00 con 0x7fca981077a0 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.676+0000 7fca96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca9819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.676+0000 7fca96ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca9819d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54630/0 (socket says 172.21.15.5:54630) 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.676+0000 7fca96ffd640 1 -- 172.21.15.5:0/710596183 learned_addr learned my addr 172.21.15.5:0/710596183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.677+0000 7fca96ffd640 1 -- 172.21.15.5:0/710596183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9819deb0 con 0x7fca981077a0 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.677+0000 7fca96ffd640 1 --2- 172.21.15.5:0/710596183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca9819d970 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fca880097f0 tx=0x7fca88030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.677+0000 7fca94ff9640 1 -- 172.21.15.5:0/710596183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca88037040 con 0x7fca981077a0 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.677+0000 7fca94ff9640 1 -- 172.21.15.5:0/710596183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fca88040070 con 0x7fca981077a0 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.677+0000 7fca9d751640 1 -- 172.21.15.5:0/710596183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9819e140 con 0x7fca981077a0 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.677+0000 7fca94ff9640 1 -- 172.21.15.5:0/710596183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca8803a390 con 0x7fca981077a0 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.677+0000 7fca9d751640 1 -- 172.21.15.5:0/710596183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca981a0e30 con 0x7fca981077a0 2024-09-11T22:12:56.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.678+0000 7fca94ff9640 1 -- 172.21.15.5:0/710596183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fca8803a5e0 con 0x7fca981077a0 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.678+0000 7fca94ff9640 1 --2- 172.21.15.5:0/710596183 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fca84038fc0 0x7fca8403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.678+0000 7fca9d751640 1 -- 172.21.15.5:0/710596183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca98107c20 con 0x7fca981077a0 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.678+0000 7fca94ff9640 1 -- 172.21.15.5:0/710596183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fca8806c5f0 con 0x7fca981077a0 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.678+0000 7fca8ffff640 1 --2- 172.21.15.5:0/710596183 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fca84038fc0 0x7fca8403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.679+0000 7fca8ffff640 1 --2- 172.21.15.5:0/710596183 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fca84038fc0 0x7fca8403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fca800098c0 tx=0x7fca80006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.683+0000 7fca94ff9640 1 -- 172.21.15.5:0/710596183 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca8803b7c0 con 0x7fca981077a0 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.863+0000 7fca9d751640 1 -- 172.21.15.5:0/710596183 --> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fca981041a0 con 0x7fca84038fc0 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.864+0000 7fca94ff9640 1 -- 172.21.15.5:0/710596183 <== mgr.14116 v2:172.21.15.5:6800/3503484776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fca981041a0 con 0x7fca84038fc0 2024-09-11T22:12:56.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.866+0000 7fca9d751640 1 -- 172.21.15.5:0/710596183 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fca84038fc0 msgr2=0x7fca8403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.866+0000 7fca9d751640 1 --2- 172.21.15.5:0/710596183 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fca84038fc0 0x7fca8403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fca800098c0 tx=0x7fca80006db0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.866+0000 7fca9d751640 1 -- 172.21.15.5:0/710596183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 msgr2=0x7fca9819d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.866+0000 7fca9d751640 1 --2- 172.21.15.5:0/710596183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca9819d970 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fca880097f0 tx=0x7fca88030d80 comp rx=0 tx=0).stop 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.866+0000 7fca9d751640 1 -- 172.21.15.5:0/710596183 shutdown_connections 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.866+0000 7fca9d751640 1 --2- 172.21.15.5:0/710596183 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fca84038fc0 0x7fca8403b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.866+0000 7fca9d751640 1 --2- 172.21.15.5:0/710596183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca981077a0 0x7fca9819d970 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.866+0000 7fca9d751640 1 -- 172.21.15.5:0/710596183 >> 172.21.15.5:0/710596183 conn(0x7fca98102fd0 msgr2=0x7fca98103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.866+0000 7fca9d751640 1 -- 172.21.15.5:0/710596183 shutdown_connections 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:55.866+0000 7fca9d751640 1 -- 172.21.15.5:0/710596183 wait complete. 2024-09-11T22:12:56.528 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2024-09-11T22:12:56.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-mon[30943]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:56.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:56 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:12:57.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: Generating public/private rsa key pair. 2024-09-11T22:12:57.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: Your identification has been saved in /tmp/tmp2ugxiv3y/key 2024-09-11T22:12:57.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: Your public key has been saved in /tmp/tmp2ugxiv3y/key.pub 2024-09-11T22:12:57.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: The key fingerprint is: 2024-09-11T22:12:57.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: SHA256:nCPIJ7SIJAh/2/hnlDJIW1Gfr0zzIlmFgPsA4UlpkqU ceph-b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: The key's randomart image is: 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: +---[RSA 3072]----+ 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: |. o+o.oo | 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: |o.o++... o o | 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: |o.Eo*... + . | 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: |o. * Xo. o o | 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: |. . X *oS + . | 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: | + =.* + | 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: | . = + . | 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: | o . . | 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: | | 2024-09-11T22:12:57.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: +----[SHA256]-----+ 2024-09-11T22:12:57.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-mon[30943]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:57.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-mon[30943]: [11/Sep/2024:22:12:56] ENGINE Bus STARTING 2024-09-11T22:12:57.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-mon[30943]: [11/Sep/2024:22:12:56] ENGINE Serving on https://172.21.15.5:7150 2024-09-11T22:12:57.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-mon[30943]: [11/Sep/2024:22:12:56] ENGINE Error in HTTPServer.serve 2024-09-11T22:12:57.873 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-11T22:12:57.873 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:12:57.873 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:12:57.874 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-11T22:12:57.875 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:12:57.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-mon[30943]: [11/Sep/2024:22:12:56] ENGINE Serving on http://172.21.15.5:8765 2024-09-11T22:12:57.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-mon[30943]: [11/Sep/2024:22:12:56] ENGINE Bus STARTED 2024-09-11T22:12:57.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:12:57.875 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:57 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:12:57.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.932+0000 7fd60522f640 1 Processor -- start 2024-09-11T22:12:57.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.932+0000 7fd60522f640 1 -- start start 2024-09-11T22:12:57.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.932+0000 7fd60522f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60010a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.932+0000 7fd60522f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd60007be30 con 0x7fd6001081b0 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.933+0000 7fd5fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60010a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.933+0000 7fd5fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60010a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54640/0 (socket says 172.21.15.5:54640) 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.933+0000 7fd5fed9d640 1 -- 172.21.15.5:0/2839004381 learned_addr learned my addr 172.21.15.5:0/2839004381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.933+0000 7fd5fed9d640 1 -- 172.21.15.5:0/2839004381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60010aae0 con 0x7fd6001081b0 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.934+0000 7fd5fed9d640 1 --2- 172.21.15.5:0/2839004381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60010a5a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009820 tx=0x7fd5f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cc2e20dca19932 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.934+0000 7fd5fdd9b640 1 -- 172.21.15.5:0/2839004381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f0037070 con 0x7fd6001081b0 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.934+0000 7fd5fdd9b640 1 -- 172.21.15.5:0/2839004381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5f0032900 con 0x7fd6001081b0 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.934+0000 7fd5fdd9b640 1 -- 172.21.15.5:0/2839004381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f0030850 con 0x7fd6001081b0 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.934+0000 7fd60522f640 1 -- 172.21.15.5:0/2839004381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 msgr2=0x7fd60010a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.934+0000 7fd60522f640 1 --2- 172.21.15.5:0/2839004381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60010a5a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0009820 tx=0x7fd5f002a270 comp rx=0 tx=0).stop 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.935+0000 7fd60522f640 1 -- 172.21.15.5:0/2839004381 shutdown_connections 2024-09-11T22:12:57.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.935+0000 7fd60522f640 1 --2- 172.21.15.5:0/2839004381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60010a5a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.935+0000 7fd60522f640 1 -- 172.21.15.5:0/2839004381 >> 172.21.15.5:0/2839004381 conn(0x7fd600101db0 msgr2=0x7fd600104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.935+0000 7fd60522f640 1 -- 172.21.15.5:0/2839004381 shutdown_connections 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.935+0000 7fd60522f640 1 -- 172.21.15.5:0/2839004381 wait complete. 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.935+0000 7fd60522f640 1 Processor -- start 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.936+0000 7fd60522f640 1 -- start start 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.936+0000 7fd60522f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.936+0000 7fd60522f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd60010bb20 con 0x7fd6001081b0 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.936+0000 7fd5fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.936+0000 7fd5fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54654/0 (socket says 172.21.15.5:54654) 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.936+0000 7fd5fed9d640 1 -- 172.21.15.5:0/2100775558 learned_addr learned my addr 172.21.15.5:0/2100775558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.936+0000 7fd5fed9d640 1 -- 172.21.15.5:0/2100775558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60019bc90 con 0x7fd6001081b0 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.936+0000 7fd5fed9d640 1 --2- 172.21.15.5:0/2100775558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60019b750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0031000 tx=0x7fd5f00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:57.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.936+0000 7fd5e7fff640 1 -- 172.21.15.5:0/2100775558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f0042070 con 0x7fd6001081b0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.937+0000 7fd5e7fff640 1 -- 172.21.15.5:0/2100775558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5f0032be0 con 0x7fd6001081b0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.937+0000 7fd60522f640 1 -- 172.21.15.5:0/2100775558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60019bf20 con 0x7fd6001081b0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.937+0000 7fd5e7fff640 1 -- 172.21.15.5:0/2100775558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5f0037040 con 0x7fd6001081b0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.937+0000 7fd60522f640 1 -- 172.21.15.5:0/2100775558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60019ec10 con 0x7fd6001081b0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.937+0000 7fd5e7fff640 1 -- 172.21.15.5:0/2100775558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fd5f0045070 con 0x7fd6001081b0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.938+0000 7fd60522f640 1 -- 172.21.15.5:0/2100775558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd600108610 con 0x7fd6001081b0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.938+0000 7fd5e7fff640 1 --2- 172.21.15.5:0/2100775558 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fd5e8038f70 0x7fd5e803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.938+0000 7fd5e7fff640 1 -- 172.21.15.5:0/2100775558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd5f006bfe0 con 0x7fd6001081b0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.938+0000 7fd5fe59c640 1 --2- 172.21.15.5:0/2100775558 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fd5e8038f70 0x7fd5e803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.938+0000 7fd5fe59c640 1 --2- 172.21.15.5:0/2100775558 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fd5e8038f70 0x7fd5e803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40098c0 tx=0x7fd5f4006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:56.943+0000 7fd5e7fff640 1 -- 172.21.15.5:0/2100775558 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5f0042210 con 0x7fd6001081b0 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.118+0000 7fd60522f640 1 -- 172.21.15.5:0/2100775558 --> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fd600103110 con 0x7fd5e8038f70 2024-09-11T22:12:57.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.271+0000 7fd5e7fff640 1 -- 172.21.15.5:0/2100775558 <== mgr.14116 v2:172.21.15.5:6800/3503484776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd600103110 con 0x7fd5e8038f70 2024-09-11T22:12:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.273+0000 7fd60522f640 1 -- 172.21.15.5:0/2100775558 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fd5e8038f70 msgr2=0x7fd5e803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.273+0000 7fd60522f640 1 --2- 172.21.15.5:0/2100775558 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fd5e8038f70 0x7fd5e803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40098c0 tx=0x7fd5f4006db0 comp rx=0 tx=0).stop 2024-09-11T22:12:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.273+0000 7fd60522f640 1 -- 172.21.15.5:0/2100775558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 msgr2=0x7fd60019b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.273+0000 7fd60522f640 1 --2- 172.21.15.5:0/2100775558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60019b750 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0031000 tx=0x7fd5f00326e0 comp rx=0 tx=0).stop 2024-09-11T22:12:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.273+0000 7fd60522f640 1 -- 172.21.15.5:0/2100775558 shutdown_connections 2024-09-11T22:12:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.273+0000 7fd60522f640 1 --2- 172.21.15.5:0/2100775558 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fd5e8038f70 0x7fd5e803b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.273+0000 7fd60522f640 1 --2- 172.21.15.5:0/2100775558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6001081b0 0x7fd60019b750 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.273+0000 7fd60522f640 1 -- 172.21.15.5:0/2100775558 >> 172.21.15.5:0/2100775558 conn(0x7fd600101db0 msgr2=0x7fd600102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.273+0000 7fd60522f640 1 -- 172.21.15.5:0/2100775558 shutdown_connections 2024-09-11T22:12:57.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:57.273+0000 7fd60522f640 1 -- 172.21.15.5:0/2100775558 wait complete. 2024-09-11T22:12:58.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:58 smithi005 ceph-mon[30943]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:12:58.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:58 smithi005 ceph-mon[30943]: Generating ssh key... 2024-09-11T22:12:59.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3VqOLIT0AnXgerFrQeZlc+weRbovvmJ85lBu54krdQF9sfJrMrQfKxcxwHmQghkdwXbdtI5WHKAS2T8aHJB3kZ+EZchbGYI2vSBmrpGUy+RroMWsQZ23MPNe3TJw+7TI2v2+Gw/MxaMfgBA5CWpzVd6Ky6vE+bpUlEO7KYqsFFL9noaU+jqjvtgoWyCfwsFCKHbIxLZlSwMPpMdZFmvGq3Qjanj7omzqkbxGHNCmEsy1HWYNOmtrAo4SF43ZohRz+/Jh+UeiicywCw3kQaWArQCCj7T8OTgRB1UMQXAVHz5wQ6xLp2xDMJkEk5q95cRqKUv2k0nHv0kwGRMmd+m4npE01YoWn2taZcheEC6GqhvJeAfXXUBqer2Hs2f4/Lj49hyaQUNadW0bk1F/TwwacrAGfTksepH9Fe5y1TSIoqZrFy/i/dUc9LX5cN2G+dMCMofAlCL2xlC6gghAoowpiOLNZQsHcm9cmMjpGMn15u6UHSvkkAsQFnuVdlJe8Gm0= ceph-b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:12:59.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.297+0000 7f7b3feb3640 1 Processor -- start 2024-09-11T22:12:59.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.297+0000 7f7b3feb3640 1 -- start start 2024-09-11T22:12:59.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.297+0000 7f7b3feb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b38107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:59.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.297+0000 7f7b3feb3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b38108170 con 0x7f7b381077a0 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.298+0000 7f7b3dc4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b38107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.298+0000 7f7b3dc4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b38107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54662/0 (socket says 172.21.15.5:54662) 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.298+0000 7f7b3dc4f640 1 -- 172.21.15.5:0/337537366 learned_addr learned my addr 172.21.15.5:0/337537366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.298+0000 7f7b3dc4f640 1 -- 172.21.15.5:0/337537366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b38108950 con 0x7f7b381077a0 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.300+0000 7f7b3dc4f640 1 --2- 172.21.15.5:0/337537366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b38107ba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009820 tx=0x7f7b3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adcd987c727d685 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.300+0000 7f7b3cc4d640 1 -- 172.21.15.5:0/337537366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b30037070 con 0x7f7b381077a0 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.300+0000 7f7b3cc4d640 1 -- 172.21.15.5:0/337537366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7b30032900 con 0x7f7b381077a0 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.300+0000 7f7b3cc4d640 1 -- 172.21.15.5:0/337537366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b30030850 con 0x7f7b381077a0 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 -- 172.21.15.5:0/337537366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 msgr2=0x7f7b38107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 --2- 172.21.15.5:0/337537366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b38107ba0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009820 tx=0x7f7b3002a270 comp rx=0 tx=0).stop 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 -- 172.21.15.5:0/337537366 shutdown_connections 2024-09-11T22:12:59.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 --2- 172.21.15.5:0/337537366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b38107ba0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 -- 172.21.15.5:0/337537366 >> 172.21.15.5:0/337537366 conn(0x7f7b38102fd0 msgr2=0x7f7b381053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 -- 172.21.15.5:0/337537366 shutdown_connections 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 -- 172.21.15.5:0/337537366 wait complete. 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 Processor -- start 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 -- start start 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b3819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3810bd00 con 0x7f7b381077a0 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3dc4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b3819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3dc4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b3819d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54670/0 (socket says 172.21.15.5:54670) 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3dc4f640 1 -- 172.21.15.5:0/1177806600 learned_addr learned my addr 172.21.15.5:0/1177806600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3dc4f640 1 -- 172.21.15.5:0/1177806600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3819deb0 con 0x7f7b381077a0 2024-09-11T22:12:59.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3dc4f640 1 --2- 172.21.15.5:0/1177806600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b3819d970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7b300097f0 tx=0x7f7b30030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b36ffd640 1 -- 172.21.15.5:0/1177806600 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b30037040 con 0x7f7b381077a0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b36ffd640 1 -- 172.21.15.5:0/1177806600 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7b30040070 con 0x7f7b381077a0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 -- 172.21.15.5:0/1177806600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3819e140 con 0x7f7b381077a0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b36ffd640 1 -- 172.21.15.5:0/1177806600 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b3003a390 con 0x7f7b381077a0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.301+0000 7f7b3feb3640 1 -- 172.21.15.5:0/1177806600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b381a0e30 con 0x7f7b381077a0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.302+0000 7f7b36ffd640 1 -- 172.21.15.5:0/1177806600 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7b3003b430 con 0x7f7b381077a0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.303+0000 7f7b3feb3640 1 -- 172.21.15.5:0/1177806600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b3810be80 con 0x7f7b381077a0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.303+0000 7f7b36ffd640 1 --2- 172.21.15.5:0/1177806600 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f7b20038fc0 0x7f7b2003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.303+0000 7f7b36ffd640 1 -- 172.21.15.5:0/1177806600 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7b3006c470 con 0x7f7b381077a0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.303+0000 7f7b3d44e640 1 --2- 172.21.15.5:0/1177806600 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f7b20038fc0 0x7f7b2003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.303+0000 7f7b3d44e640 1 --2- 172.21.15.5:0/1177806600 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f7b20038fc0 0x7f7b2003b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0098c0 tx=0x7f7b2c006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.308+0000 7f7b36ffd640 1 -- 172.21.15.5:0/1177806600 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b30040ce0 con 0x7f7b381077a0 2024-09-11T22:12:59.081 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.486+0000 7f7b3feb3640 1 -- 172.21.15.5:0/1177806600 --> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7b381041c0 con 0x7f7b20038fc0 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.487+0000 7f7b36ffd640 1 -- 172.21.15.5:0/1177806600 <== mgr.14116 v2:172.21.15.5:6800/3503484776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7b381041c0 con 0x7f7b20038fc0 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.489+0000 7f7b3feb3640 1 -- 172.21.15.5:0/1177806600 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f7b20038fc0 msgr2=0x7f7b2003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.489+0000 7f7b3feb3640 1 --2- 172.21.15.5:0/1177806600 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f7b20038fc0 0x7f7b2003b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0098c0 tx=0x7f7b2c006db0 comp rx=0 tx=0).stop 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.489+0000 7f7b3feb3640 1 -- 172.21.15.5:0/1177806600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 msgr2=0x7f7b3819d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.489+0000 7f7b3feb3640 1 --2- 172.21.15.5:0/1177806600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b3819d970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7b300097f0 tx=0x7f7b30030d80 comp rx=0 tx=0).stop 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.489+0000 7f7b3feb3640 1 -- 172.21.15.5:0/1177806600 shutdown_connections 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.489+0000 7f7b3feb3640 1 --2- 172.21.15.5:0/1177806600 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f7b20038fc0 0x7f7b2003b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.489+0000 7f7b3feb3640 1 --2- 172.21.15.5:0/1177806600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b381077a0 0x7f7b3819d970 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.489+0000 7f7b3feb3640 1 -- 172.21.15.5:0/1177806600 >> 172.21.15.5:0/1177806600 conn(0x7f7b38102fd0 msgr2=0x7f7b38103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.489+0000 7f7b3feb3640 1 -- 172.21.15.5:0/1177806600 shutdown_connections 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:58.489+0000 7f7b3feb3640 1 -- 172.21.15.5:0/1177806600 wait complete. 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:12:59.082 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T22:12:59.083 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2024-09-11T22:12:59.681 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:12:59 smithi005 ceph-mon[30943]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:00.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:00 smithi005 ceph-mon[30943]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:01.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:01 smithi005 ceph-mon[30943]: Deploying cephadm binary to smithi005 2024-09-11T22:13:04.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2024-09-11T22:13:04.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.488+0000 7fcd102a9640 1 Processor -- start 2024-09-11T22:13:04.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.488+0000 7fcd102a9640 1 -- start start 2024-09-11T22:13:04.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.488+0000 7fcd102a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd08107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.488+0000 7fcd102a9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd08108170 con 0x7fcd081077a0 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.488+0000 7fcd0e045640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd08107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.488+0000 7fcd0e045640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd08107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54674/0 (socket says 172.21.15.5:54674) 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.488+0000 7fcd0e045640 1 -- 172.21.15.5:0/2605115367 learned_addr learned my addr 172.21.15.5:0/2605115367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.489+0000 7fcd0e045640 1 -- 172.21.15.5:0/2605115367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd08108950 con 0x7fcd081077a0 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.489+0000 7fcd0e045640 1 --2- 172.21.15.5:0/2605115367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd08107ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009820 tx=0x7fcd0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01bbe2ffa59f9a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.490+0000 7fcd0d043640 1 -- 172.21.15.5:0/2605115367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd04037070 con 0x7fcd081077a0 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.490+0000 7fcd0d043640 1 -- 172.21.15.5:0/2605115367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcd04032900 con 0x7fcd081077a0 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.490+0000 7fcd0d043640 1 -- 172.21.15.5:0/2605115367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd04030850 con 0x7fcd081077a0 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.490+0000 7fcd102a9640 1 -- 172.21.15.5:0/2605115367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 msgr2=0x7fcd08107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:04.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.490+0000 7fcd102a9640 1 --2- 172.21.15.5:0/2605115367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd08107ba0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009820 tx=0x7fcd0402a270 comp rx=0 tx=0).stop 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.490+0000 7fcd102a9640 1 -- 172.21.15.5:0/2605115367 shutdown_connections 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.490+0000 7fcd102a9640 1 --2- 172.21.15.5:0/2605115367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd08107ba0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.490+0000 7fcd102a9640 1 -- 172.21.15.5:0/2605115367 >> 172.21.15.5:0/2605115367 conn(0x7fcd08102fd0 msgr2=0x7fcd081053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.491+0000 7fcd102a9640 1 -- 172.21.15.5:0/2605115367 shutdown_connections 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.491+0000 7fcd102a9640 1 -- 172.21.15.5:0/2605115367 wait complete. 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.491+0000 7fcd102a9640 1 Processor -- start 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.491+0000 7fcd102a9640 1 -- start start 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.491+0000 7fcd102a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd0819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.491+0000 7fcd102a9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0810bd00 con 0x7fcd081077a0 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.492+0000 7fcd0e045640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd0819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:04.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.492+0000 7fcd0e045640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd0819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54684/0 (socket says 172.21.15.5:54684) 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.492+0000 7fcd0e045640 1 -- 172.21.15.5:0/2399779481 learned_addr learned my addr 172.21.15.5:0/2399779481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.492+0000 7fcd0e045640 1 -- 172.21.15.5:0/2399779481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0819bc80 con 0x7fcd081077a0 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.492+0000 7fcd0e045640 1 --2- 172.21.15.5:0/2399779481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd0819b740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097f0 tx=0x7fcd04030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.492+0000 7fccff7fe640 1 -- 172.21.15.5:0/2399779481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd04037040 con 0x7fcd081077a0 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.492+0000 7fccff7fe640 1 -- 172.21.15.5:0/2399779481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcd04040070 con 0x7fcd081077a0 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.492+0000 7fcd102a9640 1 -- 172.21.15.5:0/2399779481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0819bf10 con 0x7fcd081077a0 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.492+0000 7fccff7fe640 1 -- 172.21.15.5:0/2399779481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcd0403a390 con 0x7fcd081077a0 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.492+0000 7fcd102a9640 1 -- 172.21.15.5:0/2399779481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0819ec00 con 0x7fcd081077a0 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.493+0000 7fccff7fe640 1 -- 172.21.15.5:0/2399779481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fcd0403a5e0 con 0x7fcd081077a0 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.493+0000 7fcd102a9640 1 -- 172.21.15.5:0/2399779481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd0810be80 con 0x7fcd081077a0 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.494+0000 7fccff7fe640 1 --2- 172.21.15.5:0/2399779481 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fccf4038fc0 0x7fccf403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.494+0000 7fccff7fe640 1 -- 172.21.15.5:0/2399779481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcd0406c5c0 con 0x7fcd081077a0 2024-09-11T22:13:04.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.494+0000 7fcd0d844640 1 --2- 172.21.15.5:0/2399779481 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fccf4038fc0 0x7fccf403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.494+0000 7fcd0d844640 1 --2- 172.21.15.5:0/2399779481 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fccf4038fc0 0x7fccf403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fccf80098c0 tx=0x7fccf8006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.498+0000 7fccff7fe640 1 -- 172.21.15.5:0/2399779481 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd0406bf50 con 0x7fcd081077a0 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:12:59.680+0000 7fcd102a9640 1 -- 172.21.15.5:0/2399779481 --> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) -- 0x7fcd08104240 con 0x7fccf4038fc0 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.594+0000 7fccff7fe640 1 -- 172.21.15.5:0/2399779481 <== mgr.14116 v2:172.21.15.5:6800/3503484776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fcd08104240 con 0x7fccf4038fc0 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.596+0000 7fcd102a9640 1 -- 172.21.15.5:0/2399779481 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fccf4038fc0 msgr2=0x7fccf403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.596+0000 7fcd102a9640 1 --2- 172.21.15.5:0/2399779481 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fccf4038fc0 0x7fccf403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fccf80098c0 tx=0x7fccf8006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.597+0000 7fcd102a9640 1 -- 172.21.15.5:0/2399779481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 msgr2=0x7fcd0819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.597+0000 7fcd102a9640 1 --2- 172.21.15.5:0/2399779481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd0819b740 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097f0 tx=0x7fcd04030d80 comp rx=0 tx=0).stop 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.597+0000 7fcd102a9640 1 -- 172.21.15.5:0/2399779481 shutdown_connections 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.597+0000 7fcd102a9640 1 --2- 172.21.15.5:0/2399779481 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fccf4038fc0 0x7fccf403b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.597+0000 7fcd102a9640 1 --2- 172.21.15.5:0/2399779481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd081077a0 0x7fcd0819b740 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.597+0000 7fcd102a9640 1 -- 172.21.15.5:0/2399779481 >> 172.21.15.5:0/2399779481 conn(0x7fcd08102fd0 msgr2=0x7fcd08103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:04.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.597+0000 7fcd102a9640 1 -- 172.21.15.5:0/2399779481 shutdown_connections 2024-09-11T22:13:04.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:03.597+0000 7fcd102a9640 1 -- 172.21.15.5:0/2399779481 wait complete. 2024-09-11T22:13:04.434 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2024-09-11T22:13:04.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:04 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:13:04.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:04 smithi005 ceph-mon[30943]: Added host smithi005 2024-09-11T22:13:04.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:04 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:13:05.720 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T22:13:05.720 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.656+0000 7f221f297640 1 Processor -- start 2024-09-11T22:13:05.720 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.656+0000 7f221f297640 1 -- start start 2024-09-11T22:13:05.720 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.656+0000 7f221f297640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221807cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.656+0000 7f221f297640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f221807d0e0 con 0x7f221807e660 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.657+0000 7f221d033640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221807cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.657+0000 7f221d033640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221807cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52026/0 (socket says 172.21.15.5:52026) 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.657+0000 7f221d033640 1 -- 172.21.15.5:0/3567466984 learned_addr learned my addr 172.21.15.5:0/3567466984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.657+0000 7f221d033640 1 -- 172.21.15.5:0/3567466984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221807d260 con 0x7f221807e660 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f221d033640 1 --2- 172.21.15.5:0/3567466984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221807cb10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2210009820 tx=0x7f221002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce61140bd55553a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f2217fff640 1 -- 172.21.15.5:0/3567466984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210037070 con 0x7f221807e660 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f2217fff640 1 -- 172.21.15.5:0/3567466984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2210032900 con 0x7f221807e660 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f2217fff640 1 -- 172.21.15.5:0/3567466984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210030850 con 0x7f221807e660 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f221f297640 1 -- 172.21.15.5:0/3567466984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 msgr2=0x7f221807cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f221f297640 1 --2- 172.21.15.5:0/3567466984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221807cb10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2210009820 tx=0x7f221002a270 comp rx=0 tx=0).stop 2024-09-11T22:13:05.721 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f221f297640 1 -- 172.21.15.5:0/3567466984 shutdown_connections 2024-09-11T22:13:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f221f297640 1 --2- 172.21.15.5:0/3567466984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221807cb10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f221f297640 1 -- 172.21.15.5:0/3567466984 >> 172.21.15.5:0/3567466984 conn(0x7f2218102f30 msgr2=0x7f2218105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f221f297640 1 -- 172.21.15.5:0/3567466984 shutdown_connections 2024-09-11T22:13:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.658+0000 7f221f297640 1 -- 172.21.15.5:0/3567466984 wait complete. 2024-09-11T22:13:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.659+0000 7f221f297640 1 Processor -- start 2024-09-11T22:13:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.659+0000 7f221f297640 1 -- start start 2024-09-11T22:13:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.659+0000 7f221f297640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.659+0000 7f221f297640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f221810dd50 con 0x7f221807e660 2024-09-11T22:13:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.659+0000 7f221d033640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.659+0000 7f221d033640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52040/0 (socket says 172.21.15.5:52040) 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.659+0000 7f221d033640 1 -- 172.21.15.5:0/1285585688 learned_addr learned my addr 172.21.15.5:0/1285585688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.660+0000 7f221d033640 1 -- 172.21.15.5:0/1285585688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221819dea0 con 0x7f221807e660 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.660+0000 7f221d033640 1 --2- 172.21.15.5:0/1285585688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221819d960 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2210031000 tx=0x7f22100326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.660+0000 7f22167fc640 1 -- 172.21.15.5:0/1285585688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210042070 con 0x7f221807e660 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.660+0000 7f22167fc640 1 -- 172.21.15.5:0/1285585688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2210032be0 con 0x7f221807e660 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.660+0000 7f221f297640 1 -- 172.21.15.5:0/1285585688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221819e130 con 0x7f221807e660 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.660+0000 7f22167fc640 1 -- 172.21.15.5:0/1285585688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210037040 con 0x7f221807e660 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.660+0000 7f221f297640 1 -- 172.21.15.5:0/1285585688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22181a0e20 con 0x7f221807e660 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.661+0000 7f22167fc640 1 -- 172.21.15.5:0/1285585688 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2210045070 con 0x7f221807e660 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.661+0000 7f22167fc640 1 --2- 172.21.15.5:0/1285585688 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2200038f70 0x7f220003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.661+0000 7f221f297640 1 -- 172.21.15.5:0/1285585688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f221807cb90 con 0x7f221807e660 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.661+0000 7f22167fc640 1 -- 172.21.15.5:0/1285585688 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f221006bfb0 con 0x7f221807e660 2024-09-11T22:13:05.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.661+0000 7f221c832640 1 --2- 172.21.15.5:0/1285585688 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2200038f70 0x7f220003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.661+0000 7f221c832640 1 --2- 172.21.15.5:0/1285585688 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2200038f70 0x7f220003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f220c0098c0 tx=0x7f220c006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.666+0000 7f22167fc640 1 -- 172.21.15.5:0/1285585688 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f221002fe90 con 0x7f221807e660 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.856+0000 7f221f297640 1 -- 172.21.15.5:0/1285585688 --> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2218104230 con 0x7f2200038f70 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.860+0000 7f22167fc640 1 -- 172.21.15.5:0/1285585688 <== mgr.14116 v2:172.21.15.5:6800/3503484776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2218104230 con 0x7f2200038f70 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.862+0000 7f221f297640 1 -- 172.21.15.5:0/1285585688 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2200038f70 msgr2=0x7f220003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.862+0000 7f221f297640 1 --2- 172.21.15.5:0/1285585688 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2200038f70 0x7f220003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f220c0098c0 tx=0x7f220c006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.862+0000 7f221f297640 1 -- 172.21.15.5:0/1285585688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 msgr2=0x7f221819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.862+0000 7f221f297640 1 --2- 172.21.15.5:0/1285585688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221819d960 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2210031000 tx=0x7f22100326e0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.862+0000 7f221f297640 1 -- 172.21.15.5:0/1285585688 shutdown_connections 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.862+0000 7f221f297640 1 --2- 172.21.15.5:0/1285585688 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2200038f70 0x7f220003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.862+0000 7f221f297640 1 --2- 172.21.15.5:0/1285585688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221807e660 0x7f221819d960 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.862+0000 7f221f297640 1 -- 172.21.15.5:0/1285585688 >> 172.21.15.5:0/1285585688 conn(0x7f2218102f30 msgr2=0x7f2218103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.863+0000 7f221f297640 1 -- 172.21.15.5:0/1285585688 shutdown_connections 2024-09-11T22:13:05.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:04.863+0000 7f221f297640 1 -- 172.21.15.5:0/1285585688 wait complete. 2024-09-11T22:13:05.725 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2024-09-11T22:13:05.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:05 smithi005 ceph-mon[30943]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:05.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:05 smithi005 ceph-mon[30943]: Saving service mon spec with placement count:5 2024-09-11T22:13:05.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:05 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:13:07.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.353+0000 7f619c8b4640 1 Processor -- start 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.353+0000 7f619c8b4640 1 -- start start 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.353+0000 7f619c8b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f6198107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.353+0000 7f619c8b4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6198108170 con 0x7f61981077a0 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.353+0000 7f619659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f6198107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.353+0000 7f619659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f6198107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52050/0 (socket says 172.21.15.5:52050) 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.353+0000 7f619659c640 1 -- 172.21.15.5:0/2414112122 learned_addr learned my addr 172.21.15.5:0/2414112122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.354+0000 7f619659c640 1 -- 172.21.15.5:0/2414112122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6198108950 con 0x7f61981077a0 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.354+0000 7f619659c640 1 --2- 172.21.15.5:0/2414112122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f6198107ba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6188009820 tx=0x7f618802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5ac0a1ced05a94 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.355+0000 7f6195d9b640 1 -- 172.21.15.5:0/2414112122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6188037070 con 0x7f61981077a0 2024-09-11T22:13:07.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.355+0000 7f6195d9b640 1 -- 172.21.15.5:0/2414112122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6188032900 con 0x7f61981077a0 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.355+0000 7f6195d9b640 1 -- 172.21.15.5:0/2414112122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6188030850 con 0x7f61981077a0 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.355+0000 7f619c8b4640 1 -- 172.21.15.5:0/2414112122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 msgr2=0x7f6198107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.355+0000 7f619c8b4640 1 --2- 172.21.15.5:0/2414112122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f6198107ba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6188009820 tx=0x7f618802a270 comp rx=0 tx=0).stop 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.355+0000 7f619c8b4640 1 -- 172.21.15.5:0/2414112122 shutdown_connections 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.355+0000 7f619c8b4640 1 --2- 172.21.15.5:0/2414112122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f6198107ba0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.355+0000 7f619c8b4640 1 -- 172.21.15.5:0/2414112122 >> 172.21.15.5:0/2414112122 conn(0x7f6198102fd0 msgr2=0x7f61981053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.355+0000 7f619c8b4640 1 -- 172.21.15.5:0/2414112122 shutdown_connections 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.355+0000 7f619c8b4640 1 -- 172.21.15.5:0/2414112122 wait complete. 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.356+0000 7f619c8b4640 1 Processor -- start 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.356+0000 7f619c8b4640 1 -- start start 2024-09-11T22:13:07.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.356+0000 7f619c8b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f619807e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.356+0000 7f619c8b4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f619810bd00 con 0x7f61981077a0 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.356+0000 7f619659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f619807e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.356+0000 7f619659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f619807e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52066/0 (socket says 172.21.15.5:52066) 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.356+0000 7f619659c640 1 -- 172.21.15.5:0/3203893985 learned_addr learned my addr 172.21.15.5:0/3203893985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.357+0000 7f619659c640 1 -- 172.21.15.5:0/3203893985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619807e910 con 0x7f61981077a0 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.357+0000 7f619659c640 1 --2- 172.21.15.5:0/3203893985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f619807e3d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f61880097f0 tx=0x7f6188030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.357+0000 7f618f7fe640 1 -- 172.21.15.5:0/3203893985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6188037040 con 0x7f61981077a0 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.357+0000 7f618f7fe640 1 -- 172.21.15.5:0/3203893985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6188040070 con 0x7f61981077a0 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.357+0000 7f619c8b4640 1 -- 172.21.15.5:0/3203893985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619807cb00 con 0x7f61981077a0 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.357+0000 7f618f7fe640 1 -- 172.21.15.5:0/3203893985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618803a390 con 0x7f61981077a0 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.357+0000 7f619c8b4640 1 -- 172.21.15.5:0/3203893985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619807cfe0 con 0x7f61981077a0 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.358+0000 7f618f7fe640 1 -- 172.21.15.5:0/3203893985 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6188032900 con 0x7f61981077a0 2024-09-11T22:13:07.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.358+0000 7f619c8b4640 1 -- 172.21.15.5:0/3203893985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6198107c20 con 0x7f61981077a0 2024-09-11T22:13:07.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.358+0000 7f618f7fe640 1 --2- 172.21.15.5:0/3203893985 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f6184038f70 0x7f618403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:07.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.358+0000 7f618f7fe640 1 -- 172.21.15.5:0/3203893985 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f618806c450 con 0x7f61981077a0 2024-09-11T22:13:07.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.359+0000 7f618ffff640 1 --2- 172.21.15.5:0/3203893985 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f6184038f70 0x7f618403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:07.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.359+0000 7f618ffff640 1 --2- 172.21.15.5:0/3203893985 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f6184038f70 0x7f618403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f61800098c0 tx=0x7f6180006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:07.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.363+0000 7f618f7fe640 1 -- 172.21.15.5:0/3203893985 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f618803a630 con 0x7f61981077a0 2024-09-11T22:13:07.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.558+0000 7f619c8b4640 1 -- 172.21.15.5:0/3203893985 --> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6198104280 con 0x7f6184038f70 2024-09-11T22:13:07.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.562+0000 7f618f7fe640 1 -- 172.21.15.5:0/3203893985 <== mgr.14116 v2:172.21.15.5:6800/3503484776 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6198104280 con 0x7f6184038f70 2024-09-11T22:13:07.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.564+0000 7f619c8b4640 1 -- 172.21.15.5:0/3203893985 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f6184038f70 msgr2=0x7f618403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:07.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.564+0000 7f619c8b4640 1 --2- 172.21.15.5:0/3203893985 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f6184038f70 0x7f618403b430 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f61800098c0 tx=0x7f6180006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.564+0000 7f619c8b4640 1 -- 172.21.15.5:0/3203893985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 msgr2=0x7f619807e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:07.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.564+0000 7f619c8b4640 1 --2- 172.21.15.5:0/3203893985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f619807e3d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f61880097f0 tx=0x7f6188030d80 comp rx=0 tx=0).stop 2024-09-11T22:13:07.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.564+0000 7f619c8b4640 1 -- 172.21.15.5:0/3203893985 shutdown_connections 2024-09-11T22:13:07.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.564+0000 7f619c8b4640 1 --2- 172.21.15.5:0/3203893985 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f6184038f70 0x7f618403b430 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.564+0000 7f619c8b4640 1 --2- 172.21.15.5:0/3203893985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61981077a0 0x7f619807e3d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:07.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.564+0000 7f619c8b4640 1 -- 172.21.15.5:0/3203893985 >> 172.21.15.5:0/3203893985 conn(0x7f6198102fd0 msgr2=0x7f6198103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:07.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.564+0000 7f619c8b4640 1 -- 172.21.15.5:0/3203893985 shutdown_connections 2024-09-11T22:13:07.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:06.564+0000 7f619c8b4640 1 -- 172.21.15.5:0/3203893985 wait complete. 2024-09-11T22:13:07.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:07 smithi005 ceph-mon[30943]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:07.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:07 smithi005 ceph-mon[30943]: Saving service mgr spec with placement count:2 2024-09-11T22:13:07.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:07 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:13:08.873 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.822+0000 7f2c323ac640 1 Processor -- start 2024-09-11T22:13:08.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.822+0000 7f2c323ac640 1 -- start start 2024-09-11T22:13:08.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.823+0000 7f2c323ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:08.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.823+0000 7f2c323ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c107010 con 0x7f2c2c106640 2024-09-11T22:13:08.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.823+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:08.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.823+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52082/0 (socket says 172.21.15.5:52082) 2024-09-11T22:13:08.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.823+0000 7f2c2bfff640 1 -- 172.21.15.5:0/4175017838 learned_addr learned my addr 172.21.15.5:0/4175017838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:08.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.823+0000 7f2c2bfff640 1 -- 172.21.15.5:0/4175017838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c1077f0 con 0x7f2c2c106640 2024-09-11T22:13:08.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.824+0000 7f2c2bfff640 1 --2- 172.21.15.5:0/4175017838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c106a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009820 tx=0x7f2c1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0346eb4a3d215ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:08.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.824+0000 7f2c2b7fe640 1 -- 172.21.15.5:0/4175017838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c1c037070 con 0x7f2c2c106640 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.824+0000 7f2c2b7fe640 1 -- 172.21.15.5:0/4175017838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c1c032900 con 0x7f2c2c106640 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.824+0000 7f2c2b7fe640 1 -- 172.21.15.5:0/4175017838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c1c030850 con 0x7f2c2c106640 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.825+0000 7f2c323ac640 1 -- 172.21.15.5:0/4175017838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 msgr2=0x7f2c2c106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.825+0000 7f2c323ac640 1 --2- 172.21.15.5:0/4175017838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c106a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009820 tx=0x7f2c1c02a270 comp rx=0 tx=0).stop 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.825+0000 7f2c323ac640 1 -- 172.21.15.5:0/4175017838 shutdown_connections 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.825+0000 7f2c323ac640 1 --2- 172.21.15.5:0/4175017838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c106a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.825+0000 7f2c323ac640 1 -- 172.21.15.5:0/4175017838 >> 172.21.15.5:0/4175017838 conn(0x7f2c2c101db0 msgr2=0x7f2c2c104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.825+0000 7f2c323ac640 1 -- 172.21.15.5:0/4175017838 shutdown_connections 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.825+0000 7f2c323ac640 1 -- 172.21.15.5:0/4175017838 wait complete. 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.825+0000 7f2c323ac640 1 Processor -- start 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.825+0000 7f2c323ac640 1 -- start start 2024-09-11T22:13:08.875 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c323ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c07ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c323ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c10aba0 con 0x7f2c2c106640 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c07ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c07ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52090/0 (socket says 172.21.15.5:52090) 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c2bfff640 1 -- 172.21.15.5:0/3403197170 learned_addr learned my addr 172.21.15.5:0/3403197170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c2bfff640 1 -- 172.21.15.5:0/3403197170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c19e490 con 0x7f2c2c106640 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c2bfff640 1 --2- 172.21.15.5:0/3403197170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c07ba60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0097f0 tx=0x7f2c1c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c29ffb640 1 -- 172.21.15.5:0/3403197170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c1c042070 con 0x7f2c2c106640 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c29ffb640 1 -- 172.21.15.5:0/3403197170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2c1c032ca0 con 0x7f2c2c106640 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c323ac640 1 -- 172.21.15.5:0/3403197170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c19e660 con 0x7f2c2c106640 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c29ffb640 1 -- 172.21.15.5:0/3403197170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c1c037040 con 0x7f2c2c106640 2024-09-11T22:13:08.876 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.826+0000 7f2c323ac640 1 -- 172.21.15.5:0/3403197170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c19ea80 con 0x7f2c2c106640 2024-09-11T22:13:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.827+0000 7f2c29ffb640 1 -- 172.21.15.5:0/3403197170 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2c1c045070 con 0x7f2c2c106640 2024-09-11T22:13:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.827+0000 7f2c323ac640 1 -- 172.21.15.5:0/3403197170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2c10ad20 con 0x7f2c2c106640 2024-09-11T22:13:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.828+0000 7f2c29ffb640 1 --2- 172.21.15.5:0/3403197170 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2c18038fc0 0x7f2c1803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.828+0000 7f2c29ffb640 1 -- 172.21.15.5:0/3403197170 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2c1c06c020 con 0x7f2c2c106640 2024-09-11T22:13:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.828+0000 7f2c23fff640 1 --2- 172.21.15.5:0/3403197170 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2c18038fc0 0x7f2c1803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.828+0000 7f2c23fff640 1 --2- 172.21.15.5:0/3403197170 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2c18038fc0 0x7f2c1803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2c140098c0 tx=0x7f2c14006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:07.832+0000 7f2c29ffb640 1 -- 172.21.15.5:0/3403197170 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c1c045360 con 0x7f2c2c106640 2024-09-11T22:13:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.011+0000 7f2c323ac640 1 -- 172.21.15.5:0/3403197170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f2c2c106ac0 con 0x7f2c2c106640 2024-09-11T22:13:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.013+0000 7f2c29ffb640 1 -- 172.21.15.5:0/3403197170 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f2c1c03c6a0 con 0x7f2c2c106640 2024-09-11T22:13:08.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.015+0000 7f2c323ac640 1 -- 172.21.15.5:0/3403197170 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2c18038fc0 msgr2=0x7f2c1803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.015+0000 7f2c323ac640 1 --2- 172.21.15.5:0/3403197170 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2c18038fc0 0x7f2c1803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2c140098c0 tx=0x7f2c14006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.015+0000 7f2c323ac640 1 -- 172.21.15.5:0/3403197170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 msgr2=0x7f2c2c07ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.015+0000 7f2c323ac640 1 --2- 172.21.15.5:0/3403197170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c07ba60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c0097f0 tx=0x7f2c1c030ec0 comp rx=0 tx=0).stop 2024-09-11T22:13:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.015+0000 7f2c323ac640 1 -- 172.21.15.5:0/3403197170 shutdown_connections 2024-09-11T22:13:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.015+0000 7f2c323ac640 1 --2- 172.21.15.5:0/3403197170 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f2c18038fc0 0x7f2c1803b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.015+0000 7f2c323ac640 1 --2- 172.21.15.5:0/3403197170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c106640 0x7f2c2c07ba60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.015+0000 7f2c323ac640 1 -- 172.21.15.5:0/3403197170 >> 172.21.15.5:0/3403197170 conn(0x7f2c2c101db0 msgr2=0x7f2c2c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.016+0000 7f2c323ac640 1 -- 172.21.15.5:0/3403197170 shutdown_connections 2024-09-11T22:13:08.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:08.016+0000 7f2c323ac640 1 -- 172.21.15.5:0/3403197170 wait complete. 2024-09-11T22:13:09.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:09 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3403197170' entity='client.admin' 2024-09-11T22:13:09.134 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:09 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:13:10.130 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.327+0000 7fe2c890d640 1 Processor -- start 2024-09-11T22:13:10.130 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.328+0000 7fe2c890d640 1 -- start start 2024-09-11T22:13:10.130 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.328+0000 7fe2c890d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.130 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.328+0000 7fe2c890d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c4107010 con 0x7fe2c4106640 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.328+0000 7fe2c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.328+0000 7fe2c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52098/0 (socket says 172.21.15.5:52098) 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.328+0000 7fe2c259c640 1 -- 172.21.15.5:0/1953315914 learned_addr learned my addr 172.21.15.5:0/1953315914 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.328+0000 7fe2c259c640 1 -- 172.21.15.5:0/1953315914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c41077f0 con 0x7fe2c4106640 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.329+0000 7fe2c259c640 1 --2- 172.21.15.5:0/1953315914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4106a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009e80 tx=0x7fe2b402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2321e510803f092 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.329+0000 7fe2c159a640 1 -- 172.21.15.5:0/1953315914 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b4036070 con 0x7fe2c4106640 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.329+0000 7fe2c159a640 1 -- 172.21.15.5:0/1953315914 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b40318e0 con 0x7fe2c4106640 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.329+0000 7fe2c159a640 1 -- 172.21.15.5:0/1953315914 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b4007a00 con 0x7fe2c4106640 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.330+0000 7fe2c890d640 1 -- 172.21.15.5:0/1953315914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 msgr2=0x7fe2c4106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.330+0000 7fe2c890d640 1 --2- 172.21.15.5:0/1953315914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4106a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009e80 tx=0x7fe2b402a7a0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.330+0000 7fe2c890d640 1 -- 172.21.15.5:0/1953315914 shutdown_connections 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.330+0000 7fe2c890d640 1 --2- 172.21.15.5:0/1953315914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4106a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.131 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.330+0000 7fe2c890d640 1 -- 172.21.15.5:0/1953315914 >> 172.21.15.5:0/1953315914 conn(0x7fe2c4101db0 msgr2=0x7fe2c4104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.330+0000 7fe2c890d640 1 -- 172.21.15.5:0/1953315914 shutdown_connections 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.330+0000 7fe2c890d640 1 -- 172.21.15.5:0/1953315914 wait complete. 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.331+0000 7fe2c890d640 1 Processor -- start 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.331+0000 7fe2c890d640 1 -- start start 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.331+0000 7fe2c890d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4191820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.331+0000 7fe2c890d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c410aba0 con 0x7fe2c4106640 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.331+0000 7fe2c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4191820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.331+0000 7fe2c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4191820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52106/0 (socket says 172.21.15.5:52106) 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.331+0000 7fe2c259c640 1 -- 172.21.15.5:0/4224817471 learned_addr learned my addr 172.21.15.5:0/4224817471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.331+0000 7fe2c259c640 1 -- 172.21.15.5:0/4224817471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c4191d60 con 0x7fe2c4106640 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.332+0000 7fe2c259c640 1 --2- 172.21.15.5:0/4224817471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4191820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe2b402acc0 tx=0x7fe2b4031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.332+0000 7fe2bb7fe640 1 -- 172.21.15.5:0/4224817471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b4036050 con 0x7fe2c4106640 2024-09-11T22:13:10.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.332+0000 7fe2bb7fe640 1 -- 172.21.15.5:0/4224817471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2b4037040 con 0x7fe2c4106640 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.332+0000 7fe2c890d640 1 -- 172.21.15.5:0/4224817471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c418ff50 con 0x7fe2c4106640 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.332+0000 7fe2bb7fe640 1 -- 172.21.15.5:0/4224817471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2b403b920 con 0x7fe2c4106640 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.332+0000 7fe2c890d640 1 -- 172.21.15.5:0/4224817471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c4190490 con 0x7fe2c4106640 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.333+0000 7fe2bb7fe640 1 -- 172.21.15.5:0/4224817471 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe2b4044020 con 0x7fe2c4106640 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.333+0000 7fe2c890d640 1 -- 172.21.15.5:0/4224817471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c410ad20 con 0x7fe2c4106640 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.333+0000 7fe2bb7fe640 1 --2- 172.21.15.5:0/4224817471 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fe2b0038f70 0x7fe2b003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.333+0000 7fe2bb7fe640 1 -- 172.21.15.5:0/4224817471 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe2b4042020 con 0x7fe2c4106640 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.333+0000 7fe2c1d9b640 1 --2- 172.21.15.5:0/4224817471 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fe2b0038f70 0x7fe2b003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.334+0000 7fe2c1d9b640 1 --2- 172.21.15.5:0/4224817471 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fe2b0038f70 0x7fe2b003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0098c0 tx=0x7fe2ac006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.342+0000 7fe2bb7fe640 1 -- 172.21.15.5:0/4224817471 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe2b404b020 con 0x7fe2c4106640 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.530+0000 7fe2c890d640 1 -- 172.21.15.5:0/4224817471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fe2c4195550 con 0x7fe2c4106640 2024-09-11T22:13:10.133 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.533+0000 7fe2bb7fe640 1 -- 172.21.15.5:0/4224817471 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fe2b4039d90 con 0x7fe2c4106640 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.535+0000 7fe2c890d640 1 -- 172.21.15.5:0/4224817471 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fe2b0038f70 msgr2=0x7fe2b003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.535+0000 7fe2c890d640 1 --2- 172.21.15.5:0/4224817471 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fe2b0038f70 0x7fe2b003b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0098c0 tx=0x7fe2ac006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.535+0000 7fe2c890d640 1 -- 172.21.15.5:0/4224817471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 msgr2=0x7fe2c4191820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.535+0000 7fe2c890d640 1 --2- 172.21.15.5:0/4224817471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4191820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe2b402acc0 tx=0x7fe2b4031cc0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.535+0000 7fe2c890d640 1 -- 172.21.15.5:0/4224817471 shutdown_connections 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.535+0000 7fe2c890d640 1 --2- 172.21.15.5:0/4224817471 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7fe2b0038f70 0x7fe2b003b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.535+0000 7fe2c890d640 1 --2- 172.21.15.5:0/4224817471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4106640 0x7fe2c4191820 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.535+0000 7fe2c890d640 1 -- 172.21.15.5:0/4224817471 >> 172.21.15.5:0/4224817471 conn(0x7fe2c4101db0 msgr2=0x7fe2c4102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.535+0000 7fe2c890d640 1 -- 172.21.15.5:0/4224817471 shutdown_connections 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:09.535+0000 7fe2c890d640 1 -- 172.21.15.5:0/4224817471 wait complete. 2024-09-11T22:13:10.134 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2024-09-11T22:13:10.424 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:10 smithi005 ceph-mon[30943]: from='mgr.14116 172.21.15.5:0/3140049375' entity='mgr.x' 2024-09-11T22:13:10.425 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:10 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4224817471' entity='client.admin' 2024-09-11T22:13:11.250 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:11 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3543208847' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T22:13:11.530 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: ignoring --setuser ceph since I am not root 2024-09-11T22:13:11.531 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: ignoring --setgroup ceph since I am not root 2024-09-11T22:13:11.531 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:11.381+0000 7f178f021040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:13:11.531 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:11.441+0000 7f178f021040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:13:11.820 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:11.601+0000 7f178f021040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:13:11.820 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:11.672+0000 7f178f021040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:13:11.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.508+0000 7efcff2fb640 1 Processor -- start 2024-09-11T22:13:11.957 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.508+0000 7efcff2fb640 1 -- start start 2024-09-11T22:13:11.957 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.509+0000 7efcff2fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf810a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:11.957 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.509+0000 7efcff2fb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efcf807be80 con 0x7efcf8108170 2024-09-11T22:13:11.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.509+0000 7efcfd097640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf810a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:11.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.509+0000 7efcfd097640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf810a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52118/0 (socket says 172.21.15.5:52118) 2024-09-11T22:13:11.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.509+0000 7efcfd097640 1 -- 172.21.15.5:0/4225151369 learned_addr learned my addr 172.21.15.5:0/4225151369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:11.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.509+0000 7efcfd097640 1 -- 172.21.15.5:0/4225151369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf810aaa0 con 0x7efcf8108170 2024-09-11T22:13:11.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.510+0000 7efcfd097640 1 --2- 172.21.15.5:0/4225151369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf810a560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009820 tx=0x7efcf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2708797c1e9e3d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:11.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.510+0000 7efceffff640 1 -- 172.21.15.5:0/4225151369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcf4037070 con 0x7efcf8108170 2024-09-11T22:13:11.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.510+0000 7efceffff640 1 -- 172.21.15.5:0/4225151369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf4032920 con 0x7efcf8108170 2024-09-11T22:13:11.958 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.510+0000 7efceffff640 1 -- 172.21.15.5:0/4225151369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcf4030810 con 0x7efcf8108170 2024-09-11T22:13:11.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.510+0000 7efcff2fb640 1 -- 172.21.15.5:0/4225151369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 msgr2=0x7efcf810a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:11.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.510+0000 7efcff2fb640 1 --2- 172.21.15.5:0/4225151369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf810a560 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009820 tx=0x7efcf402a270 comp rx=0 tx=0).stop 2024-09-11T22:13:11.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.510+0000 7efcff2fb640 1 -- 172.21.15.5:0/4225151369 shutdown_connections 2024-09-11T22:13:11.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.511+0000 7efcff2fb640 1 --2- 172.21.15.5:0/4225151369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf810a560 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:11.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.511+0000 7efcff2fb640 1 -- 172.21.15.5:0/4225151369 >> 172.21.15.5:0/4225151369 conn(0x7efcf8101db0 msgr2=0x7efcf81041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:11.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.511+0000 7efcff2fb640 1 -- 172.21.15.5:0/4225151369 shutdown_connections 2024-09-11T22:13:11.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.511+0000 7efcff2fb640 1 -- 172.21.15.5:0/4225151369 wait complete. 2024-09-11T22:13:11.959 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.511+0000 7efcff2fb640 1 Processor -- start 2024-09-11T22:13:11.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.511+0000 7efcff2fb640 1 -- start start 2024-09-11T22:13:11.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.511+0000 7efcff2fb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:11.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.511+0000 7efcff2fb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efcf810bb80 con 0x7efcf8108170 2024-09-11T22:13:11.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.511+0000 7efcfd097640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:11.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.512+0000 7efcfd097640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52134/0 (socket says 172.21.15.5:52134) 2024-09-11T22:13:11.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.512+0000 7efcfd097640 1 -- 172.21.15.5:0/3543208847 learned_addr learned my addr 172.21.15.5:0/3543208847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:11.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.512+0000 7efcfd097640 1 -- 172.21.15.5:0/3543208847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf819bcc0 con 0x7efcf8108170 2024-09-11T22:13:11.960 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.512+0000 7efcfd097640 1 --2- 172.21.15.5:0/3543208847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf819b780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efcf40097f0 tx=0x7efcf40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:11.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.512+0000 7efcee7fc640 1 -- 172.21.15.5:0/3543208847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcf4037040 con 0x7efcf8108170 2024-09-11T22:13:11.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.512+0000 7efcee7fc640 1 -- 172.21.15.5:0/3543208847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf4030c40 con 0x7efcf8108170 2024-09-11T22:13:11.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.512+0000 7efcff2fb640 1 -- 172.21.15.5:0/3543208847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf819bf50 con 0x7efcf8108170 2024-09-11T22:13:11.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.512+0000 7efcee7fc640 1 -- 172.21.15.5:0/3543208847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcf403a5d0 con 0x7efcf8108170 2024-09-11T22:13:11.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.512+0000 7efcff2fb640 1 -- 172.21.15.5:0/3543208847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf819ec40 con 0x7efcf8108170 2024-09-11T22:13:11.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.513+0000 7efcee7fc640 1 -- 172.21.15.5:0/3543208847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efcf403b3f0 con 0x7efcf8108170 2024-09-11T22:13:11.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.513+0000 7efcff2fb640 1 -- 172.21.15.5:0/3543208847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf81085d0 con 0x7efcf8108170 2024-09-11T22:13:11.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.513+0000 7efcee7fc640 1 --2- 172.21.15.5:0/3543208847 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7efce4038f70 0x7efce403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:11.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.513+0000 7efcee7fc640 1 -- 172.21.15.5:0/3543208847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efcf406c460 con 0x7efcf8108170 2024-09-11T22:13:11.961 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.514+0000 7efcfc896640 1 --2- 172.21.15.5:0/3543208847 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7efce4038f70 0x7efce403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.514+0000 7efcfc896640 1 --2- 172.21.15.5:0/3543208847 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7efce4038f70 0x7efce403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efce80098c0 tx=0x7efce8006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.518+0000 7efcee7fc640 1 -- 172.21.15.5:0/3543208847 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcf403b6e0 con 0x7efcf8108170 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:10.749+0000 7efcff2fb640 1 -- 172.21.15.5:0/3543208847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7efcf819ee10 con 0x7efcf8108170 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.219+0000 7efcee7fc640 1 -- 172.21.15.5:0/3543208847 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7efcf403a810 con 0x7efcf8108170 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.220+0000 7efcee7fc640 1 -- 172.21.15.5:0/3543208847 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7efcf4049ca0 con 0x7efcf8108170 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.222+0000 7efcff2fb640 1 -- 172.21.15.5:0/3543208847 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7efce4038f70 msgr2=0x7efce403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.222+0000 7efcff2fb640 1 --2- 172.21.15.5:0/3543208847 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7efce4038f70 0x7efce403b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efce80098c0 tx=0x7efce8006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.222+0000 7efcff2fb640 1 -- 172.21.15.5:0/3543208847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 msgr2=0x7efcf819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.222+0000 7efcff2fb640 1 --2- 172.21.15.5:0/3543208847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf819b780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efcf40097f0 tx=0x7efcf40327b0 comp rx=0 tx=0).stop 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.222+0000 7efcff2fb640 1 -- 172.21.15.5:0/3543208847 shutdown_connections 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.222+0000 7efcff2fb640 1 --2- 172.21.15.5:0/3543208847 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7efce4038f70 0x7efce403b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:11.962 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.222+0000 7efcff2fb640 1 --2- 172.21.15.5:0/3543208847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcf8108170 0x7efcf819b780 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:11.963 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.222+0000 7efcff2fb640 1 -- 172.21.15.5:0/3543208847 >> 172.21.15.5:0/3543208847 conn(0x7efcf8101db0 msgr2=0x7efcf8102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:11.963 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.222+0000 7efcff2fb640 1 -- 172.21.15.5:0/3543208847 shutdown_connections 2024-09-11T22:13:11.963 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:11.222+0000 7efcff2fb640 1 -- 172.21.15.5:0/3543208847 wait complete. 2024-09-11T22:13:12.082 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:11.830+0000 7f178f021040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:13:12.082 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:11.984+0000 7f178f021040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:13:12.395 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:12 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:12.328+0000 7f178f021040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:13:12.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:12 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3543208847' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T22:13:12.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:12 smithi005 ceph-mon[30943]: mgrmap e9: x(active, since 18s) 2024-09-11T22:13:12.648 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:12 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:12.394+0000 7f178f021040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:13:12.649 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:12 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:12.608+0000 7f178f021040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:13:13.233 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.394+0000 7f11f16c8640 1 Processor -- start 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.394+0000 7f11f16c8640 1 -- start start 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.394+0000 7f11f16c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec10a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.394+0000 7f11f16c8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec07c6f0 con 0x7f11ec1081f0 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.395+0000 7f11eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec10a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.395+0000 7f11eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec10a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47976/0 (socket says 172.21.15.5:47976) 2024-09-11T22:13:13.234 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.395+0000 7f11eaffd640 1 -- 172.21.15.5:0/2693992633 learned_addr learned my addr 172.21.15.5:0/2693992633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.395+0000 7f11eaffd640 1 -- 172.21.15.5:0/2693992633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec10ab20 con 0x7f11ec1081f0 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.396+0000 7f11eaffd640 1 --2- 172.21.15.5:0/2693992633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec10a5e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009820 tx=0x7f11dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3003e6f3e089411 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.396+0000 7f11e9ffb640 1 -- 172.21.15.5:0/2693992633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11dc037070 con 0x7f11ec1081f0 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.396+0000 7f11e9ffb640 1 -- 172.21.15.5:0/2693992633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11dc032920 con 0x7f11ec1081f0 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.396+0000 7f11e9ffb640 1 -- 172.21.15.5:0/2693992633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11dc030810 con 0x7f11ec1081f0 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.397+0000 7f11f16c8640 1 -- 172.21.15.5:0/2693992633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 msgr2=0x7f11ec10a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.397+0000 7f11f16c8640 1 --2- 172.21.15.5:0/2693992633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec10a5e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009820 tx=0x7f11dc02a270 comp rx=0 tx=0).stop 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.397+0000 7f11f16c8640 1 -- 172.21.15.5:0/2693992633 shutdown_connections 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.397+0000 7f11f16c8640 1 --2- 172.21.15.5:0/2693992633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec10a5e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.397+0000 7f11f16c8640 1 -- 172.21.15.5:0/2693992633 >> 172.21.15.5:0/2693992633 conn(0x7f11ec101e50 msgr2=0x7f11ec104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:13.235 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.397+0000 7f11f16c8640 1 -- 172.21.15.5:0/2693992633 shutdown_connections 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.397+0000 7f11f16c8640 1 -- 172.21.15.5:0/2693992633 wait complete. 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.398+0000 7f11f16c8640 1 Processor -- start 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.398+0000 7f11f16c8640 1 -- start start 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11f16c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec107a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11f16c8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec10bb40 con 0x7f11ec1081f0 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47990/0 (socket says 172.21.15.5:47990) 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11eaffd640 1 -- 172.21.15.5:0/501232444 learned_addr learned my addr 172.21.15.5:0/501232444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11eaffd640 1 -- 172.21.15.5:0/501232444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec107fb0 con 0x7f11ec1081f0 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11eaffd640 1 --2- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec107a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0097f0 tx=0x7f11dc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11d3fff640 1 -- 172.21.15.5:0/501232444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11dc037040 con 0x7f11ec1081f0 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11d3fff640 1 -- 172.21.15.5:0/501232444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11dc030cc0 con 0x7f11ec1081f0 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11f16c8640 1 -- 172.21.15.5:0/501232444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ec1061a0 con 0x7f11ec1081f0 2024-09-11T22:13:13.236 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11d3fff640 1 -- 172.21.15.5:0/501232444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11dc03a6b0 con 0x7f11ec1081f0 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.399+0000 7f11f16c8640 1 -- 172.21.15.5:0/501232444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ec1066e0 con 0x7f11ec1081f0 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.401+0000 7f11d3fff640 1 -- 172.21.15.5:0/501232444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f11dc03a8a0 con 0x7f11ec1081f0 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.401+0000 7f11f16c8640 1 -- 172.21.15.5:0/501232444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ec108650 con 0x7f11ec1081f0 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.401+0000 7f11d3fff640 1 --2- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f11d4038fc0 0x7f11d403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.401+0000 7f11d3fff640 1 -- 172.21.15.5:0/501232444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f11dc06c810 con 0x7f11ec1081f0 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.401+0000 7f11ea7fc640 1 -- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f11d4038fc0 msgr2=0x7f11d403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3503484776 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.401+0000 7f11ea7fc640 1 --2- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f11d4038fc0 0x7f11d403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.410+0000 7f11d3fff640 1 -- 172.21.15.5:0/501232444 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11dc042e00 con 0x7f11ec1081f0 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.602+0000 7f11ea7fc640 1 -- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f11d4038fc0 msgr2=0x7f11d403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3503484776 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.602+0000 7f11ea7fc640 1 --2- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f11d4038fc0 0x7f11d403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.636+0000 7f11f16c8640 1 -- 172.21.15.5:0/501232444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f11ec106960 con 0x7f11ec1081f0 2024-09-11T22:13:13.237 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.636+0000 7f11d3fff640 1 -- 172.21.15.5:0/501232444 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f11dc03ace0 con 0x7f11ec1081f0 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.638+0000 7f11f16c8640 1 -- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f11d4038fc0 msgr2=0x7f11d403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.638+0000 7f11f16c8640 1 --2- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f11d4038fc0 0x7f11d403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.638+0000 7f11f16c8640 1 -- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 msgr2=0x7f11ec107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.638+0000 7f11f16c8640 1 --2- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec107a70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0097f0 tx=0x7f11dc0327b0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.638+0000 7f11f16c8640 1 -- 172.21.15.5:0/501232444 shutdown_connections 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.638+0000 7f11f16c8640 1 --2- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f11d4038fc0 0x7f11d403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.638+0000 7f11f16c8640 1 --2- 172.21.15.5:0/501232444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11ec1081f0 0x7f11ec107a70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.638+0000 7f11f16c8640 1 -- 172.21.15.5:0/501232444 >> 172.21.15.5:0/501232444 conn(0x7f11ec101e50 msgr2=0x7f11ec102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.638+0000 7f11f16c8640 1 -- 172.21.15.5:0/501232444 shutdown_connections 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:12.638+0000 7f11f16c8640 1 -- 172.21.15.5:0/501232444 wait complete. 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-09-11T22:13:13.238 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2024-09-11T22:13:13.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:13 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/501232444' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T22:13:13.636 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:13 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:13.546+0000 7f178f021040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:13:13.636 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:13 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:13.617+0000 7f178f021040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:13:14.641 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:14.370+0000 7f178f021040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:13:14.641 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:14.490+0000 7f178f021040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:13:14.932 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:14.640+0000 7f178f021040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:13:14.932 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:14.703+0000 7f178f021040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:13:14.932 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:14.860+0000 7f178f021040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:13:14.932 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:14.931+0000 7f178f021040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:13:15.229 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:14.994+0000 7f178f021040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:13:15.229 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:15.160+0000 7f178f021040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:13:15.229 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:15.228+0000 7f178f021040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:13:15.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:15.298+0000 7f178f021040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:13:15.509 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:15.384+0000 7f178f021040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:13:15.509 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:15.446+0000 7f178f021040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:13:15.509 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:15.507+0000 7f178f021040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:13:16.588 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:16.334+0000 7f178f021040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:13:16.588 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: Active manager daemon x restarted 2024-09-11T22:13:16.588 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: Activating manager daemon x 2024-09-11T22:13:16.588 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T22:13:16.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: mgrmap e10: x(active, starting, since 0.00328509s) 2024-09-11T22:13:16.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:13:16.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:13:16.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:13:16.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:13:16.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:13:16.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: Manager daemon x is now available 2024-09-11T22:13:16.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:13:16.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:13:16.589 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:13:18.318 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:17.969+0000 7f16fa335640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:22:13:17] ENGINE Error in HTTPServer.serve 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: Traceback (most recent call last): 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self._connections.run(self.expiration_interval) 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self._run(expiration_interval) 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: s = self.context.wrap_socket( 2024-09-11T22:13:18.319 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:13:18.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: return self.sslsocket_class._create( 2024-09-11T22:13:18.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:13:18.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self.do_handshake() 2024-09-11T22:13:18.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:13:18.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self._sslobj.do_handshake() 2024-09-11T22:13:18.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:13:18.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:13:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:13:18.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-11T22:13:18.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T22:13:18.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T22:13:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-11T22:13:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.718+0000 7f18e00e6640 1 Processor -- start 2024-09-11T22:13:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.718+0000 7f18e00e6640 1 -- start start 2024-09-11T22:13:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.718+0000 7f18e00e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d810a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.718+0000 7f18e00e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18d807bd70 con 0x7f18d8108270 2024-09-11T22:13:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.718+0000 7f18dde82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d810a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.718+0000 7f18dde82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d810a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48006/0 (socket says 172.21.15.5:48006) 2024-09-11T22:13:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.718+0000 7f18dde82640 1 -- 172.21.15.5:0/1855364373 learned_addr learned my addr 172.21.15.5:0/1855364373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.719+0000 7f18dde82640 1 -- 172.21.15.5:0/1855364373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d810aba0 con 0x7f18d8108270 2024-09-11T22:13:18.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.719+0000 7f18dde82640 1 --2- 172.21.15.5:0/1855364373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d810a660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009820 tx=0x7f18d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d05f8b37206d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.719+0000 7f18dce80640 1 -- 172.21.15.5:0/1855364373 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18d4037070 con 0x7f18d8108270 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.719+0000 7f18dce80640 1 -- 172.21.15.5:0/1855364373 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d4032920 con 0x7f18d8108270 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.720+0000 7f18dce80640 1 -- 172.21.15.5:0/1855364373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18d4030810 con 0x7f18d8108270 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.720+0000 7f18e00e6640 1 -- 172.21.15.5:0/1855364373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 msgr2=0x7f18d810a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.720+0000 7f18e00e6640 1 --2- 172.21.15.5:0/1855364373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d810a660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f18d4009820 tx=0x7f18d402a270 comp rx=0 tx=0).stop 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.720+0000 7f18e00e6640 1 -- 172.21.15.5:0/1855364373 shutdown_connections 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.720+0000 7f18e00e6640 1 --2- 172.21.15.5:0/1855364373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d810a660 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.720+0000 7f18e00e6640 1 -- 172.21.15.5:0/1855364373 >> 172.21.15.5:0/1855364373 conn(0x7f18d8101db0 msgr2=0x7f18d8104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.720+0000 7f18e00e6640 1 -- 172.21.15.5:0/1855364373 shutdown_connections 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.720+0000 7f18e00e6640 1 -- 172.21.15.5:0/1855364373 wait complete. 2024-09-11T22:13:18.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.721+0000 7f18e00e6640 1 Processor -- start 2024-09-11T22:13:18.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.721+0000 7f18e00e6640 1 -- start start 2024-09-11T22:13:18.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.721+0000 7f18e00e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:18.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.721+0000 7f18e00e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18d810bb60 con 0x7f18d8108270 2024-09-11T22:13:18.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.721+0000 7f18dde82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:18.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.721+0000 7f18dde82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48018/0 (socket says 172.21.15.5:48018) 2024-09-11T22:13:18.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.721+0000 7f18dde82640 1 -- 172.21.15.5:0/582776173 learned_addr learned my addr 172.21.15.5:0/582776173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:18.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.721+0000 7f18dde82640 1 -- 172.21.15.5:0/582776173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d819dea0 con 0x7f18d8108270 2024-09-11T22:13:18.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.722+0000 7f18dde82640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d819d960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18d4030420 tx=0x7f18d4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:18.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.722+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18d4037040 con 0x7f18d8108270 2024-09-11T22:13:18.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.722+0000 7f18e00e6640 1 -- 172.21.15.5:0/582776173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d819e130 con 0x7f18d8108270 2024-09-11T22:13:18.332 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.722+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18d4040070 con 0x7f18d8108270 2024-09-11T22:13:18.332 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.722+0000 7f18e00e6640 1 -- 172.21.15.5:0/582776173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d81a0e20 con 0x7f18d8108270 2024-09-11T22:13:18.332 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.722+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18d403a6e0 con 0x7f18d8108270 2024-09-11T22:13:18.332 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.723+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f18d403bb20 con 0x7f18d8108270 2024-09-11T22:13:18.332 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.723+0000 7f18ceffd640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 0x7f18c403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:18.332 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.723+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 --> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f18c403bb90 con 0x7f18c4038fc0 2024-09-11T22:13:18.332 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.723+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f18d406c650 con 0x7f18d8108270 2024-09-11T22:13:18.332 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.723+0000 7f18dd681640 1 -- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 msgr2=0x7f18c403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3503484776 2024-09-11T22:13:18.332 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.723+0000 7f18dd681640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 0x7f18c403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.924+0000 7f18dd681640 1 -- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 msgr2=0x7f18c403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3503484776 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:13.924+0000 7f18dd681640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 0x7f18c403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:14.325+0000 7f18dd681640 1 -- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 msgr2=0x7f18c403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3503484776 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:14.325+0000 7f18dd681640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 0x7f18c403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:15.126+0000 7f18dd681640 1 -- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 msgr2=0x7f18c403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3503484776 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:15.126+0000 7f18dd681640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 0x7f18c403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:16.338+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f18d403ad60 con 0x7f18d8108270 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:16.338+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 msgr2=0x7f18c403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:16.338+0000 7f18ceffd640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 0x7f18c403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.339+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f18d404bd10 con 0x7f18d8108270 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.339+0000 7f18ceffd640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18c403ca80 0x7f18c403ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:18.333 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.339+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f18d403de70 con 0x7f18c403ca80 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.339+0000 7f18dd681640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18c403ca80 0x7f18c403ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.340+0000 7f18dd681640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18c403ca80 0x7f18c403ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18c8003980 tx=0x7f18c8006e10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.341+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f18d403de70 con 0x7f18c403ca80 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.348+0000 7f18e00e6640 1 -- 172.21.15.5:0/582776173 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f18d81086d0 con 0x7f18c403ca80 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.348+0000 7f18ceffd640 1 -- 172.21.15.5:0/582776173 <== mgr.14148 v2:172.21.15.5:6800/3591749770 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f18d81086d0 con 0x7f18c403ca80 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 -- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18c403ca80 msgr2=0x7f18c403ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18c403ca80 0x7f18c403ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f18c8003980 tx=0x7f18c8006e10 comp rx=0 tx=0).stop 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 -- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 msgr2=0x7f18d819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d819d960 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18d4030420 tx=0x7f18d4032670 comp rx=0 tx=0).stop 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 -- 172.21.15.5:0/582776173 shutdown_connections 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18c403ca80 0x7f18c403ee70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:6800/3503484776,v1:172.21.15.5:6801/3503484776] conn(0x7f18c4038fc0 0x7f18c403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 --2- 172.21.15.5:0/582776173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8108270 0x7f18d819d960 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:18.335 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 -- 172.21.15.5:0/582776173 >> 172.21.15.5:0/582776173 conn(0x7f18d8101db0 msgr2=0x7f18d81029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:18.335 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 -- 172.21.15.5:0/582776173 shutdown_connections 2024-09-11T22:13:18.335 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:17.349+0000 7f18e00e6640 1 -- 172.21.15.5:0/582776173 wait complete. 2024-09-11T22:13:18.335 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2024-09-11T22:13:18.335 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2024-09-11T22:13:18.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:18 smithi005 ceph-mon[30943]: mgrmap e11: x(active, since 1.00568s) 2024-09-11T22:13:18.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:18 smithi005 ceph-mon[30943]: [11/Sep/2024:22:13:17] ENGINE Bus STARTING 2024-09-11T22:13:18.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:18.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:18 smithi005 ceph-mon[30943]: [11/Sep/2024:22:13:17] ENGINE Serving on https://172.21.15.5:7150 2024-09-11T22:13:18.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:18 smithi005 ceph-mon[30943]: [11/Sep/2024:22:13:17] ENGINE Error in HTTPServer.serve 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-11T22:13:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:13:18.572 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-11T22:13:18.572 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:13:18.572 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-11T22:13:18.572 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:13:18.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:18 smithi005 ceph-mon[30943]: [11/Sep/2024:22:13:18] ENGINE Serving on http://172.21.15.5:8765 2024-09-11T22:13:18.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:18 smithi005 ceph-mon[30943]: [11/Sep/2024:22:13:18] ENGINE Bus STARTED 2024-09-11T22:13:19.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T22:13:19.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.802+0000 7fe500b58640 1 Processor -- start 2024-09-11T22:13:19.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.803+0000 7fe500b58640 1 -- start start 2024-09-11T22:13:19.624 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.803+0000 7fe500b58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.803+0000 7fe500b58640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc107010 con 0x7fe4fc106640 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.803+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.803+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48084/0 (socket says 172.21.15.5:48084) 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.803+0000 7fe4fa59c640 1 -- 172.21.15.5:0/201629684 learned_addr learned my addr 172.21.15.5:0/201629684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.804+0000 7fe4fa59c640 1 -- 172.21.15.5:0/201629684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc1077f0 con 0x7fe4fc106640 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.804+0000 7fe4fa59c640 1 --2- 172.21.15.5:0/201629684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc106a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009820 tx=0x7fe4ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9d72b80d37af2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.805+0000 7fe4f959a640 1 -- 172.21.15.5:0/201629684 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4ec037070 con 0x7fe4fc106640 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.805+0000 7fe4f959a640 1 -- 172.21.15.5:0/201629684 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4ec032920 con 0x7fe4fc106640 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.805+0000 7fe4f959a640 1 -- 172.21.15.5:0/201629684 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4ec030810 con 0x7fe4fc106640 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.805+0000 7fe500b58640 1 -- 172.21.15.5:0/201629684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 msgr2=0x7fe4fc106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.805+0000 7fe500b58640 1 --2- 172.21.15.5:0/201629684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc106a40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009820 tx=0x7fe4ec02a270 comp rx=0 tx=0).stop 2024-09-11T22:13:19.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.806+0000 7fe500b58640 1 -- 172.21.15.5:0/201629684 shutdown_connections 2024-09-11T22:13:19.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.806+0000 7fe500b58640 1 --2- 172.21.15.5:0/201629684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc106a40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.806+0000 7fe500b58640 1 -- 172.21.15.5:0/201629684 >> 172.21.15.5:0/201629684 conn(0x7fe4fc101db0 msgr2=0x7fe4fc104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:19.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.806+0000 7fe500b58640 1 -- 172.21.15.5:0/201629684 shutdown_connections 2024-09-11T22:13:19.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.806+0000 7fe500b58640 1 -- 172.21.15.5:0/201629684 wait complete. 2024-09-11T22:13:19.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.806+0000 7fe500b58640 1 Processor -- start 2024-09-11T22:13:19.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.806+0000 7fe500b58640 1 -- start start 2024-09-11T22:13:19.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.807+0000 7fe500b58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc191820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.807+0000 7fe500b58640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4fc10aba0 con 0x7fe4fc106640 2024-09-11T22:13:19.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.807+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc191820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.807+0000 7fe4fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc191820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48092/0 (socket says 172.21.15.5:48092) 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.807+0000 7fe4fa59c640 1 -- 172.21.15.5:0/3360545895 learned_addr learned my addr 172.21.15.5:0/3360545895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.807+0000 7fe4fa59c640 1 -- 172.21.15.5:0/3360545895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4fc191d60 con 0x7fe4fc106640 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.807+0000 7fe4fa59c640 1 --2- 172.21.15.5:0/3360545895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc191820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec030560 tx=0x7fe4ec032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.807+0000 7fe4eb7fe640 1 -- 172.21.15.5:0/3360545895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4ec037040 con 0x7fe4fc106640 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.807+0000 7fe500b58640 1 -- 172.21.15.5:0/3360545895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4fc18ff50 con 0x7fe4fc106640 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.808+0000 7fe500b58640 1 -- 172.21.15.5:0/3360545895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4fc190490 con 0x7fe4fc106640 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.808+0000 7fe4eb7fe640 1 -- 172.21.15.5:0/3360545895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4ec040070 con 0x7fe4fc106640 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.808+0000 7fe4eb7fe640 1 -- 172.21.15.5:0/3360545895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4ec03a6d0 con 0x7fe4fc106640 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.808+0000 7fe4eb7fe640 1 -- 172.21.15.5:0/3360545895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fe4ec03a870 con 0x7fe4fc106640 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.809+0000 7fe500b58640 1 -- 172.21.15.5:0/3360545895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4dc005290 con 0x7fe4fc106640 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.809+0000 7fe4eb7fe640 1 --2- 172.21.15.5:0/3360545895 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe4e4038f00 0x7fe4e403b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:19.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.809+0000 7fe4eb7fe640 1 -- 172.21.15.5:0/3360545895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe4ec06c440 con 0x7fe4fc106640 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.809+0000 7fe4f9d9b640 1 --2- 172.21.15.5:0/3360545895 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe4e4038f00 0x7fe4e403b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.809+0000 7fe4f9d9b640 1 --2- 172.21.15.5:0/3360545895 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe4e4038f00 0x7fe4e403b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00098c0 tx=0x7fe4f0006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.814+0000 7fe4eb7fe640 1 -- 172.21.15.5:0/3360545895 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4ec041270 con 0x7fe4fc106640 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.908+0000 7fe4eb7fe640 1 -- 172.21.15.5:0/3360545895 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe4ec045460 con 0x7fe4fc106640 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:18.999+0000 7fe500b58640 1 -- 172.21.15.5:0/3360545895 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fe4dc000bc0 con 0x7fe4e4038f00 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.041+0000 7fe4eb7fe640 1 -- 172.21.15.5:0/3360545895 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fe4dc000bc0 con 0x7fe4e4038f00 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.043+0000 7fe500b58640 1 -- 172.21.15.5:0/3360545895 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe4e4038f00 msgr2=0x7fe4e403b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.043+0000 7fe500b58640 1 --2- 172.21.15.5:0/3360545895 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe4e4038f00 0x7fe4e403b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe4f00098c0 tx=0x7fe4f0006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.043+0000 7fe500b58640 1 -- 172.21.15.5:0/3360545895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 msgr2=0x7fe4fc191820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.043+0000 7fe500b58640 1 --2- 172.21.15.5:0/3360545895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc191820 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec030560 tx=0x7fe4ec032670 comp rx=0 tx=0).stop 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.043+0000 7fe500b58640 1 -- 172.21.15.5:0/3360545895 shutdown_connections 2024-09-11T22:13:19.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.043+0000 7fe500b58640 1 --2- 172.21.15.5:0/3360545895 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe4e4038f00 0x7fe4e403b3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.043+0000 7fe500b58640 1 --2- 172.21.15.5:0/3360545895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc106640 0x7fe4fc191820 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:19.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.043+0000 7fe500b58640 1 -- 172.21.15.5:0/3360545895 >> 172.21.15.5:0/3360545895 conn(0x7fe4fc101db0 msgr2=0x7fe4fc102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:19.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.043+0000 7fe500b58640 1 -- 172.21.15.5:0/3360545895 shutdown_connections 2024-09-11T22:13:19.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:19.043+0000 7fe500b58640 1 -- 172.21.15.5:0/3360545895 wait complete. 2024-09-11T22:13:19.629 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2024-09-11T22:13:19.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:19.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:19 smithi005 ceph-mon[30943]: mgrmap e12: x(active, since 2s) 2024-09-11T22:13:19.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:19 smithi005 ceph-mon[30943]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:19.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:19.873 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:20.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:20 smithi005 ceph-mon[30943]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:20.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:20 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:21.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$twx7cb/lZQ8mg6vWPzuPZeeYFfoNETLKYX1CcCXI8XOn.FeeRSoB6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726092800, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T22:13:21.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.060+0000 7f7301594640 1 Processor -- start 2024-09-11T22:13:21.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.060+0000 7f7301594640 1 -- start start 2024-09-11T22:13:21.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.060+0000 7f7301594640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc106460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:21.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.060+0000 7f7301594640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc07c660 con 0x7f72fc106060 2024-09-11T22:13:21.418 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.060+0000 7f72faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc106460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.060+0000 7f72faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc106460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48102/0 (socket says 172.21.15.5:48102) 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.060+0000 7f72faffd640 1 -- 172.21.15.5:0/1776606944 learned_addr learned my addr 172.21.15.5:0/1776606944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.061+0000 7f72faffd640 1 -- 172.21.15.5:0/1776606944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc1069a0 con 0x7f72fc106060 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.062+0000 7f72faffd640 1 --2- 172.21.15.5:0/1776606944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc106460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009820 tx=0x7f72ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e940ac7185b1f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.062+0000 7f72fa7fc640 1 -- 172.21.15.5:0/1776606944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ec037070 con 0x7f72fc106060 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.062+0000 7f72fa7fc640 1 -- 172.21.15.5:0/1776606944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec032920 con 0x7f72fc106060 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.062+0000 7f72fa7fc640 1 -- 172.21.15.5:0/1776606944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ec030810 con 0x7f72fc106060 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.062+0000 7f7301594640 1 -- 172.21.15.5:0/1776606944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 msgr2=0x7f72fc106460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.062+0000 7f7301594640 1 --2- 172.21.15.5:0/1776606944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc106460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009820 tx=0x7f72ec02a270 comp rx=0 tx=0).stop 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.062+0000 7f7301594640 1 -- 172.21.15.5:0/1776606944 shutdown_connections 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.062+0000 7f7301594640 1 --2- 172.21.15.5:0/1776606944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc106460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.062+0000 7f7301594640 1 -- 172.21.15.5:0/1776606944 >> 172.21.15.5:0/1776606944 conn(0x7f72fc101ed0 msgr2=0x7f72fc1042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:21.419 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.062+0000 7f7301594640 1 -- 172.21.15.5:0/1776606944 shutdown_connections 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.063+0000 7f7301594640 1 -- 172.21.15.5:0/1776606944 wait complete. 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.063+0000 7f7301594640 1 Processor -- start 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.063+0000 7f7301594640 1 -- start start 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.063+0000 7f7301594640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.063+0000 7f7301594640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc107ae0 con 0x7f72fc106060 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.063+0000 7f72faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.063+0000 7f72faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48104/0 (socket says 172.21.15.5:48104) 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.063+0000 7f72faffd640 1 -- 172.21.15.5:0/3868523045 learned_addr learned my addr 172.21.15.5:0/3868523045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.064+0000 7f72faffd640 1 -- 172.21.15.5:0/3868523045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc19de40 con 0x7f72fc106060 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.064+0000 7f72faffd640 1 --2- 172.21.15.5:0/3868523045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc19d900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0097f0 tx=0x7f72ec0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.064+0000 7f72f8ff9640 1 -- 172.21.15.5:0/3868523045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ec037040 con 0x7f72fc106060 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.064+0000 7f72f8ff9640 1 -- 172.21.15.5:0/3868523045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec030cc0 con 0x7f72fc106060 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.064+0000 7f72f8ff9640 1 -- 172.21.15.5:0/3868523045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ec03a6b0 con 0x7f72fc106060 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.064+0000 7f7301594640 1 -- 172.21.15.5:0/3868523045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc19e0d0 con 0x7f72fc106060 2024-09-11T22:13:21.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.064+0000 7f7301594640 1 -- 172.21.15.5:0/3868523045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc1a0dc0 con 0x7f72fc106060 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.065+0000 7f72f8ff9640 1 -- 172.21.15.5:0/3868523045 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f72ec03baf0 con 0x7f72fc106060 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.065+0000 7f7301594640 1 -- 172.21.15.5:0/3868523045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72fc106460 con 0x7f72fc106060 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.065+0000 7f72f8ff9640 1 --2- 172.21.15.5:0/3868523045 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f72e8038fc0 0x7f72e803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.066+0000 7f72f8ff9640 1 -- 172.21.15.5:0/3868523045 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f72ec06c630 con 0x7f72fc106060 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.066+0000 7f72f3fff640 1 --2- 172.21.15.5:0/3868523045 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f72e8038fc0 0x7f72e803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.066+0000 7f72f3fff640 1 --2- 172.21.15.5:0/3868523045 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f72e8038fc0 0x7f72e803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f72e40098c0 tx=0x7f72e4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.070+0000 7f72f8ff9640 1 -- 172.21.15.5:0/3868523045 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72ec042e10 con 0x7f72fc106060 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.277+0000 7f7301594640 1 -- 172.21.15.5:0/3868523045 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f72fc19e5b0 con 0x7f72e8038fc0 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.526+0000 7f72f8ff9640 1 -- 172.21.15.5:0/3868523045 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f72fc19e5b0 con 0x7f72e8038fc0 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.528+0000 7f7301594640 1 -- 172.21.15.5:0/3868523045 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f72e8038fc0 msgr2=0x7f72e803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.528+0000 7f7301594640 1 --2- 172.21.15.5:0/3868523045 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f72e8038fc0 0x7f72e803b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f72e40098c0 tx=0x7f72e4006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.529+0000 7f7301594640 1 -- 172.21.15.5:0/3868523045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 msgr2=0x7f72fc19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:21.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.529+0000 7f7301594640 1 --2- 172.21.15.5:0/3868523045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc19d900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0097f0 tx=0x7f72ec0327b0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.529+0000 7f7301594640 1 -- 172.21.15.5:0/3868523045 shutdown_connections 2024-09-11T22:13:21.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.529+0000 7f7301594640 1 --2- 172.21.15.5:0/3868523045 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f72e8038fc0 0x7f72e803b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.529+0000 7f7301594640 1 --2- 172.21.15.5:0/3868523045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72fc106060 0x7f72fc19d900 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:21.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.529+0000 7f7301594640 1 -- 172.21.15.5:0/3868523045 >> 172.21.15.5:0/3868523045 conn(0x7f72fc101ed0 msgr2=0x7f72fc191d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:21.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.529+0000 7f7301594640 1 -- 172.21.15.5:0/3868523045 shutdown_connections 2024-09-11T22:13:21.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:20.529+0000 7f7301594640 1 -- 172.21.15.5:0/3868523045 wait complete. 2024-09-11T22:13:21.422 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2024-09-11T22:13:23.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T22:13:23.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.287+0000 7f672a3cb640 1 Processor -- start 2024-09-11T22:13:23.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.287+0000 7f672a3cb640 1 -- start start 2024-09-11T22:13:23.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.288+0000 7f672a3cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f6724107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.288+0000 7f672a3cb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6724108170 con 0x7f67241077a0 2024-09-11T22:13:23.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.288+0000 7f6723fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f6724107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.288+0000 7f6723fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f6724107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47982/0 (socket says 172.21.15.5:47982) 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.288+0000 7f6723fff640 1 -- 172.21.15.5:0/3403329158 learned_addr learned my addr 172.21.15.5:0/3403329158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.288+0000 7f6723fff640 1 -- 172.21.15.5:0/3403329158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6724108950 con 0x7f67241077a0 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.289+0000 7f6723fff640 1 --2- 172.21.15.5:0/3403329158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f6724107ba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6714009820 tx=0x7f671402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740d2685c6857405 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.289+0000 7f67237fe640 1 -- 172.21.15.5:0/3403329158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6714037070 con 0x7f67241077a0 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.289+0000 7f67237fe640 1 -- 172.21.15.5:0/3403329158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6714032920 con 0x7f67241077a0 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.289+0000 7f67237fe640 1 -- 172.21.15.5:0/3403329158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6714030810 con 0x7f67241077a0 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.290+0000 7f672a3cb640 1 -- 172.21.15.5:0/3403329158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 msgr2=0x7f6724107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.290+0000 7f672a3cb640 1 --2- 172.21.15.5:0/3403329158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f6724107ba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6714009820 tx=0x7f671402a270 comp rx=0 tx=0).stop 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.290+0000 7f672a3cb640 1 -- 172.21.15.5:0/3403329158 shutdown_connections 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.290+0000 7f672a3cb640 1 --2- 172.21.15.5:0/3403329158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f6724107ba0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.290+0000 7f672a3cb640 1 -- 172.21.15.5:0/3403329158 >> 172.21.15.5:0/3403329158 conn(0x7f6724102fd0 msgr2=0x7f67241053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:23.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.290+0000 7f672a3cb640 1 -- 172.21.15.5:0/3403329158 shutdown_connections 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.290+0000 7f672a3cb640 1 -- 172.21.15.5:0/3403329158 wait complete. 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.290+0000 7f672a3cb640 1 Processor -- start 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.290+0000 7f672a3cb640 1 -- start start 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f672a3cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f672419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f672a3cb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f672410bd00 con 0x7f67241077a0 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f6723fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f672419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f6723fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f672419b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47984/0 (socket says 172.21.15.5:47984) 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f6723fff640 1 -- 172.21.15.5:0/3553403829 learned_addr learned my addr 172.21.15.5:0/3553403829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f6723fff640 1 -- 172.21.15.5:0/3553403829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672419bcb0 con 0x7f67241077a0 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f6723fff640 1 --2- 172.21.15.5:0/3553403829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f672419b770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67140097f0 tx=0x7f67140327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f6721ffb640 1 -- 172.21.15.5:0/3553403829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6714037040 con 0x7f67241077a0 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f6721ffb640 1 -- 172.21.15.5:0/3553403829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6714030cc0 con 0x7f67241077a0 2024-09-11T22:13:23.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f672a3cb640 1 -- 172.21.15.5:0/3553403829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672419bf40 con 0x7f67241077a0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f6721ffb640 1 -- 172.21.15.5:0/3553403829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f671403a640 con 0x7f67241077a0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.291+0000 7f672a3cb640 1 -- 172.21.15.5:0/3553403829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672419ec30 con 0x7f67241077a0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.292+0000 7f6721ffb640 1 -- 172.21.15.5:0/3553403829 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f671403a940 con 0x7f67241077a0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.292+0000 7f672a3cb640 1 -- 172.21.15.5:0/3553403829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f672410be80 con 0x7f67241077a0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.292+0000 7f6721ffb640 1 --2- 172.21.15.5:0/3553403829 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6710038fc0 0x7f671003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.292+0000 7f6721ffb640 1 -- 172.21.15.5:0/3553403829 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f671406c630 con 0x7f67241077a0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.293+0000 7f671bfff640 1 --2- 172.21.15.5:0/3553403829 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6710038fc0 0x7f671003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.293+0000 7f671bfff640 1 --2- 172.21.15.5:0/3553403829 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6710038fc0 0x7f671003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f670c0098c0 tx=0x7f670c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.297+0000 7f6721ffb640 1 -- 172.21.15.5:0/3553403829 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f671403ba40 con 0x7f67241077a0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.390+0000 7f6721ffb640 1 -- 172.21.15.5:0/3553403829 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f6714044400 con 0x7f67241077a0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.476+0000 7f672a3cb640 1 -- 172.21.15.5:0/3553403829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f672419ee00 con 0x7f67241077a0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.476+0000 7f6721ffb640 1 -- 172.21.15.5:0/3553403829 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f6714040de0 con 0x7f67241077a0 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.478+0000 7f672a3cb640 1 -- 172.21.15.5:0/3553403829 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6710038fc0 msgr2=0x7f671003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.478+0000 7f672a3cb640 1 --2- 172.21.15.5:0/3553403829 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6710038fc0 0x7f671003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f670c0098c0 tx=0x7f670c006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.478+0000 7f672a3cb640 1 -- 172.21.15.5:0/3553403829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 msgr2=0x7f672419b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:23.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.478+0000 7f672a3cb640 1 --2- 172.21.15.5:0/3553403829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f672419b770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f67140097f0 tx=0x7f67140327b0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.478+0000 7f672a3cb640 1 -- 172.21.15.5:0/3553403829 shutdown_connections 2024-09-11T22:13:23.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.478+0000 7f672a3cb640 1 --2- 172.21.15.5:0/3553403829 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6710038fc0 0x7f671003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.478+0000 7f672a3cb640 1 --2- 172.21.15.5:0/3553403829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67241077a0 0x7f672419b770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:23.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.478+0000 7f672a3cb640 1 -- 172.21.15.5:0/3553403829 >> 172.21.15.5:0/3553403829 conn(0x7f6724102fd0 msgr2=0x7f6724103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:23.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.479+0000 7f672a3cb640 1 -- 172.21.15.5:0/3553403829 shutdown_connections 2024-09-11T22:13:23.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:22.479+0000 7f672a3cb640 1 -- 172.21.15.5:0/3553403829 wait complete. 2024-09-11T22:13:23.193 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T22:13:23.193 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-11T22:13:23.200 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T22:13:23.200 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-11T22:13:23.200 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-11T22:13:23.200 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T22:13:23.202 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2024-09-11T22:13:23.202 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:23.202 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005.front.sepia.ceph.com:8443/ 2024-09-11T22:13:23.202 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2024-09-11T22:13:23.202 INFO:teuthology.orchestra.run.smithi005.stdout: Password: n2a7nc2pn5 2024-09-11T22:13:23.202 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:23.203 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config directory 2024-09-11T22:13:23.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:23 smithi005 ceph-mon[30943]: mgrmap e13: x(active, since 6s) 2024-09-11T22:13:23.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:23 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3553403829' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T22:13:24.475 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.604+0000 7f5acb94a640 1 Processor -- start 2024-09-11T22:13:24.475 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.604+0000 7f5acb94a640 1 -- start start 2024-09-11T22:13:24.475 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.604+0000 7f5acb94a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac4107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:24.475 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.604+0000 7f5acb94a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4108170 con 0x7f5ac41077a0 2024-09-11T22:13:24.475 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.604+0000 7f5ac96e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac4107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:24.475 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.604+0000 7f5ac96e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac4107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47996/0 (socket says 172.21.15.5:47996) 2024-09-11T22:13:24.475 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.604+0000 7f5ac96e6640 1 -- 172.21.15.5:0/1220106982 learned_addr learned my addr 172.21.15.5:0/1220106982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:24.475 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.605+0000 7f5ac96e6640 1 -- 172.21.15.5:0/1220106982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac4108950 con 0x7f5ac41077a0 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.605+0000 7f5ac96e6640 1 --2- 172.21.15.5:0/1220106982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac4107ba0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009820 tx=0x7f5ac002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e12a586dc5525a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.605+0000 7f5abbfff640 1 -- 172.21.15.5:0/1220106982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ac0037070 con 0x7f5ac41077a0 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.606+0000 7f5abbfff640 1 -- 172.21.15.5:0/1220106982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac0032920 con 0x7f5ac41077a0 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.606+0000 7f5abbfff640 1 -- 172.21.15.5:0/1220106982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ac0030810 con 0x7f5ac41077a0 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.606+0000 7f5acb94a640 1 -- 172.21.15.5:0/1220106982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 msgr2=0x7f5ac4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.606+0000 7f5acb94a640 1 --2- 172.21.15.5:0/1220106982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac4107ba0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009820 tx=0x7f5ac002a270 comp rx=0 tx=0).stop 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.606+0000 7f5acb94a640 1 -- 172.21.15.5:0/1220106982 shutdown_connections 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.606+0000 7f5acb94a640 1 --2- 172.21.15.5:0/1220106982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac4107ba0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.606+0000 7f5acb94a640 1 -- 172.21.15.5:0/1220106982 >> 172.21.15.5:0/1220106982 conn(0x7f5ac4102fd0 msgr2=0x7f5ac41053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.606+0000 7f5acb94a640 1 -- 172.21.15.5:0/1220106982 shutdown_connections 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.606+0000 7f5acb94a640 1 -- 172.21.15.5:0/1220106982 wait complete. 2024-09-11T22:13:24.476 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.607+0000 7f5acb94a640 1 Processor -- start 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.607+0000 7f5acb94a640 1 -- start start 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.607+0000 7f5acb94a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.607+0000 7f5acb94a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac410bd00 con 0x7f5ac41077a0 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.607+0000 7f5ac96e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.607+0000 7f5ac96e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac419d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48008/0 (socket says 172.21.15.5:48008) 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.607+0000 7f5ac96e6640 1 -- 172.21.15.5:0/3726547621 learned_addr learned my addr 172.21.15.5:0/3726547621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.607+0000 7f5ac96e6640 1 -- 172.21.15.5:0/3726547621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac419de90 con 0x7f5ac41077a0 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.608+0000 7f5ac96e6640 1 --2- 172.21.15.5:0/3726547621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac419d950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0030560 tx=0x7f5ac0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.608+0000 7f5aba7fc640 1 -- 172.21.15.5:0/3726547621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ac0037040 con 0x7f5ac41077a0 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.608+0000 7f5acb94a640 1 -- 172.21.15.5:0/3726547621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac419e120 con 0x7f5ac41077a0 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.608+0000 7f5aba7fc640 1 -- 172.21.15.5:0/3726547621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac0040070 con 0x7f5ac41077a0 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.608+0000 7f5aba7fc640 1 -- 172.21.15.5:0/3726547621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ac003a740 con 0x7f5ac41077a0 2024-09-11T22:13:24.477 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.608+0000 7f5acb94a640 1 -- 172.21.15.5:0/3726547621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac41a0e10 con 0x7f5ac41077a0 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.609+0000 7f5aba7fc640 1 -- 172.21.15.5:0/3726547621 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f5ac003bb90 con 0x7f5ac41077a0 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.609+0000 7f5acb94a640 1 -- 172.21.15.5:0/3726547621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac410be80 con 0x7f5ac41077a0 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.609+0000 7f5aba7fc640 1 --2- 172.21.15.5:0/3726547621 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ab0039060 0x7f5ab003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.609+0000 7f5aba7fc640 1 -- 172.21.15.5:0/3726547621 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5ac006c790 con 0x7f5ac41077a0 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.609+0000 7f5ac8ee5640 1 --2- 172.21.15.5:0/3726547621 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ab0039060 0x7f5ab003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.610+0000 7f5ac8ee5640 1 --2- 172.21.15.5:0/3726547621 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ab0039060 0x7f5ab003b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40098c0 tx=0x7f5ab4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.614+0000 7f5aba7fc640 1 -- 172.21.15.5:0/3726547621 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ac0042940 con 0x7f5ac41077a0 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.884+0000 7f5acb94a640 1 -- 172.21.15.5:0/3726547621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f5ac41a0fe0 con 0x7f5ac41077a0 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.886+0000 7f5aba7fc640 1 -- 172.21.15.5:0/3726547621 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) ==== 153+0+0 (secure 0 0 0) 0x7f5ac004a200 con 0x7f5ac41077a0 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.889+0000 7f5acb94a640 1 -- 172.21.15.5:0/3726547621 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ab0039060 msgr2=0x7f5ab003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.889+0000 7f5acb94a640 1 --2- 172.21.15.5:0/3726547621 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ab0039060 0x7f5ab003b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5ab40098c0 tx=0x7f5ab4006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:24.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.889+0000 7f5acb94a640 1 -- 172.21.15.5:0/3726547621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 msgr2=0x7f5ac419d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.889+0000 7f5acb94a640 1 --2- 172.21.15.5:0/3726547621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac419d950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0030560 tx=0x7f5ac0032670 comp rx=0 tx=0).stop 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.889+0000 7f5acb94a640 1 -- 172.21.15.5:0/3726547621 shutdown_connections 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.889+0000 7f5acb94a640 1 --2- 172.21.15.5:0/3726547621 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ab0039060 0x7f5ab003b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.889+0000 7f5acb94a640 1 --2- 172.21.15.5:0/3726547621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ac41077a0 0x7f5ac419d950 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.889+0000 7f5acb94a640 1 -- 172.21.15.5:0/3726547621 >> 172.21.15.5:0/3726547621 conn(0x7f5ac4102fd0 msgr2=0x7f5ac4103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.889+0000 7f5acb94a640 1 -- 172.21.15.5:0/3726547621 shutdown_connections 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-11T22:13:23.889+0000 7f5acb94a640 1 -- 172.21.15.5:0/3726547621 wait complete. 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:24.479 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T22:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T22:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2024-09-11T22:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2024-09-11T22:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-11T22:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2024-09-11T22:13:24.519 INFO:tasks.cephadm:Fetching config... 2024-09-11T22:13:24.519 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:13:24.519 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T22:13:24.536 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T22:13:24.537 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:13:24.537 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T22:13:24.593 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T22:13:24.593 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:13:24.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T22:13:24.660 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T22:13:24.660 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:13:24.660 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T22:13:24.717 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T22:13:24.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3VqOLIT0AnXgerFrQeZlc+weRbovvmJ85lBu54krdQF9sfJrMrQfKxcxwHmQghkdwXbdtI5WHKAS2T8aHJB3kZ+EZchbGYI2vSBmrpGUy+RroMWsQZ23MPNe3TJw+7TI2v2+Gw/MxaMfgBA5CWpzVd6Ky6vE+bpUlEO7KYqsFFL9noaU+jqjvtgoWyCfwsFCKHbIxLZlSwMPpMdZFmvGq3Qjanj7omzqkbxGHNCmEsy1HWYNOmtrAo4SF43ZohRz+/Jh+UeiicywCw3kQaWArQCCj7T8OTgRB1UMQXAVHz5wQ6xLp2xDMJkEk5q95cRqKUv2k0nHv0kwGRMmd+m4npE01YoWn2taZcheEC6GqhvJeAfXXUBqer2Hs2f4/Lj49hyaQUNadW0bk1F/TwwacrAGfTksepH9Fe5y1TSIoqZrFy/i/dUc9LX5cN2G+dMCMofAlCL2xlC6gghAoowpiOLNZQsHcm9cmMjpGMn15u6UHSvkkAsQFnuVdlJe8Gm0= ceph-b99b16b2-708a-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T22:13:24.793 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3VqOLIT0AnXgerFrQeZlc+weRbovvmJ85lBu54krdQF9sfJrMrQfKxcxwHmQghkdwXbdtI5WHKAS2T8aHJB3kZ+EZchbGYI2vSBmrpGUy+RroMWsQZ23MPNe3TJw+7TI2v2+Gw/MxaMfgBA5CWpzVd6Ky6vE+bpUlEO7KYqsFFL9noaU+jqjvtgoWyCfwsFCKHbIxLZlSwMPpMdZFmvGq3Qjanj7omzqkbxGHNCmEsy1HWYNOmtrAo4SF43ZohRz+/Jh+UeiicywCw3kQaWArQCCj7T8OTgRB1UMQXAVHz5wQ6xLp2xDMJkEk5q95cRqKUv2k0nHv0kwGRMmd+m4npE01YoWn2taZcheEC6GqhvJeAfXXUBqer2Hs2f4/Lj49hyaQUNadW0bk1F/TwwacrAGfTksepH9Fe5y1TSIoqZrFy/i/dUc9LX5cN2G+dMCMofAlCL2xlC6gghAoowpiOLNZQsHcm9cmMjpGMn15u6UHSvkkAsQFnuVdlJe8Gm0= ceph-b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:13:24.805 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3VqOLIT0AnXgerFrQeZlc+weRbovvmJ85lBu54krdQF9sfJrMrQfKxcxwHmQghkdwXbdtI5WHKAS2T8aHJB3kZ+EZchbGYI2vSBmrpGUy+RroMWsQZ23MPNe3TJw+7TI2v2+Gw/MxaMfgBA5CWpzVd6Ky6vE+bpUlEO7KYqsFFL9noaU+jqjvtgoWyCfwsFCKHbIxLZlSwMPpMdZFmvGq3Qjanj7omzqkbxGHNCmEsy1HWYNOmtrAo4SF43ZohRz+/Jh+UeiicywCw3kQaWArQCCj7T8OTgRB1UMQXAVHz5wQ6xLp2xDMJkEk5q95cRqKUv2k0nHv0kwGRMmd+m4npE01YoWn2taZcheEC6GqhvJeAfXXUBqer2Hs2f4/Lj49hyaQUNadW0bk1F/TwwacrAGfTksepH9Fe5y1TSIoqZrFy/i/dUc9LX5cN2G+dMCMofAlCL2xlC6gghAoowpiOLNZQsHcm9cmMjpGMn15u6UHSvkkAsQFnuVdlJe8Gm0= ceph-b99b16b2-708a-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T22:13:24.846 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3VqOLIT0AnXgerFrQeZlc+weRbovvmJ85lBu54krdQF9sfJrMrQfKxcxwHmQghkdwXbdtI5WHKAS2T8aHJB3kZ+EZchbGYI2vSBmrpGUy+RroMWsQZ23MPNe3TJw+7TI2v2+Gw/MxaMfgBA5CWpzVd6Ky6vE+bpUlEO7KYqsFFL9noaU+jqjvtgoWyCfwsFCKHbIxLZlSwMPpMdZFmvGq3Qjanj7omzqkbxGHNCmEsy1HWYNOmtrAo4SF43ZohRz+/Jh+UeiicywCw3kQaWArQCCj7T8OTgRB1UMQXAVHz5wQ6xLp2xDMJkEk5q95cRqKUv2k0nHv0kwGRMmd+m4npE01YoWn2taZcheEC6GqhvJeAfXXUBqer2Hs2f4/Lj49hyaQUNadW0bk1F/TwwacrAGfTksepH9Fe5y1TSIoqZrFy/i/dUc9LX5cN2G+dMCMofAlCL2xlC6gghAoowpiOLNZQsHcm9cmMjpGMn15u6UHSvkkAsQFnuVdlJe8Gm0= ceph-b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:13:24.858 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3VqOLIT0AnXgerFrQeZlc+weRbovvmJ85lBu54krdQF9sfJrMrQfKxcxwHmQghkdwXbdtI5WHKAS2T8aHJB3kZ+EZchbGYI2vSBmrpGUy+RroMWsQZ23MPNe3TJw+7TI2v2+Gw/MxaMfgBA5CWpzVd6Ky6vE+bpUlEO7KYqsFFL9noaU+jqjvtgoWyCfwsFCKHbIxLZlSwMPpMdZFmvGq3Qjanj7omzqkbxGHNCmEsy1HWYNOmtrAo4SF43ZohRz+/Jh+UeiicywCw3kQaWArQCCj7T8OTgRB1UMQXAVHz5wQ6xLp2xDMJkEk5q95cRqKUv2k0nHv0kwGRMmd+m4npE01YoWn2taZcheEC6GqhvJeAfXXUBqer2Hs2f4/Lj49hyaQUNadW0bk1F/TwwacrAGfTksepH9Fe5y1TSIoqZrFy/i/dUc9LX5cN2G+dMCMofAlCL2xlC6gghAoowpiOLNZQsHcm9cmMjpGMn15u6UHSvkkAsQFnuVdlJe8Gm0= ceph-b99b16b2-708a-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T22:13:24.899 INFO:teuthology.orchestra.run.smithi172.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3VqOLIT0AnXgerFrQeZlc+weRbovvmJ85lBu54krdQF9sfJrMrQfKxcxwHmQghkdwXbdtI5WHKAS2T8aHJB3kZ+EZchbGYI2vSBmrpGUy+RroMWsQZ23MPNe3TJw+7TI2v2+Gw/MxaMfgBA5CWpzVd6Ky6vE+bpUlEO7KYqsFFL9noaU+jqjvtgoWyCfwsFCKHbIxLZlSwMPpMdZFmvGq3Qjanj7omzqkbxGHNCmEsy1HWYNOmtrAo4SF43ZohRz+/Jh+UeiicywCw3kQaWArQCCj7T8OTgRB1UMQXAVHz5wQ6xLp2xDMJkEk5q95cRqKUv2k0nHv0kwGRMmd+m4npE01YoWn2taZcheEC6GqhvJeAfXXUBqer2Hs2f4/Lj49hyaQUNadW0bk1F/TwwacrAGfTksepH9Fe5y1TSIoqZrFy/i/dUc9LX5cN2G+dMCMofAlCL2xlC6gghAoowpiOLNZQsHcm9cmMjpGMn15u6UHSvkkAsQFnuVdlJe8Gm0= ceph-b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:13:24.912 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T22:13:25.012 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:24 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3726547621' entity='client.admin' 2024-09-11T22:13:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.693+0000 7fb1e4070640 1 -- 172.21.15.5:0/1678077195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1dc0ffec0 msgr2=0x7fb1dc1002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.693+0000 7fb1e4070640 1 --2- 172.21.15.5:0/1678077195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1dc0ffec0 0x7fb1dc1002c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009880 tx=0x7fb1d802a5d0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.693+0000 7fb1e4070640 1 -- 172.21.15.5:0/1678077195 shutdown_connections 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.693+0000 7fb1e4070640 1 --2- 172.21.15.5:0/1678077195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1dc0ffec0 0x7fb1dc1002c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.693+0000 7fb1e4070640 1 -- 172.21.15.5:0/1678077195 >> 172.21.15.5:0/1678077195 conn(0x7fb1dc0fbcd0 msgr2=0x7fb1dc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.694+0000 7fb1e4070640 1 -- 172.21.15.5:0/1678077195 shutdown_connections 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.694+0000 7fb1e4070640 1 -- 172.21.15.5:0/1678077195 wait complete. 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.694+0000 7fb1e4070640 1 Processor -- start 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.694+0000 7fb1e4070640 1 -- start start 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.694+0000 7fb1e4070640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1dc0ffec0 0x7fb1dc198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.694+0000 7fb1e4070640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1dc1098d0 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.694+0000 7fb1e1e0c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1dc0ffec0 0x7fb1dc198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.694+0000 7fb1e1e0c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1dc0ffec0 0x7fb1dc198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48026/0 (socket says 172.21.15.5:48026) 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.694+0000 7fb1e1e0c640 1 -- 172.21.15.5:0/659009796 learned_addr learned my addr 172.21.15.5:0/659009796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.695+0000 7fb1e1e0c640 1 -- 172.21.15.5:0/659009796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1dc199480 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.695+0000 7fb1e1e0c640 1 --2- 172.21.15.5:0/659009796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1dc0ffec0 0x7fb1dc198f40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8004f60 tx=0x7fb1d80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.695+0000 7fb1d2ffd640 1 -- 172.21.15.5:0/659009796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1d803f070 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.695+0000 7fb1d2ffd640 1 -- 172.21.15.5:0/659009796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1d8003ad0 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.695+0000 7fb1e4070640 1 -- 172.21.15.5:0/659009796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1dc199710 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.695+0000 7fb1d2ffd640 1 -- 172.21.15.5:0/659009796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1d8035030 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.695+0000 7fb1e4070640 1 -- 172.21.15.5:0/659009796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1dc19c400 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.696+0000 7fb1d2ffd640 1 -- 172.21.15.5:0/659009796 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fb1d8042070 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.696+0000 7fb1d2ffd640 1 --2- 172.21.15.5:0/659009796 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb1c8039060 0x7fb1c803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.696+0000 7fb1d2ffd640 1 -- 172.21.15.5:0/659009796 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb1d806cf70 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.696+0000 7fb1e160b640 1 --2- 172.21.15.5:0/659009796 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb1c8039060 0x7fb1c803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.696+0000 7fb1e4070640 1 -- 172.21.15.5:0/659009796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1dc10a050 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.697+0000 7fb1e160b640 1 --2- 172.21.15.5:0/659009796 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb1c8039060 0x7fb1c803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0098c0 tx=0x7fb1cc006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:25.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.701+0000 7fb1d2ffd640 1 -- 172.21.15.5:0/659009796 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1d804c350 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.888+0000 7fb1e4070640 1 -- 172.21.15.5:0/659009796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fb1dc192100 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.891+0000 7fb1d2ffd640 1 -- 172.21.15.5:0/659009796 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fb1d8039e00 con 0x7fb1dc0ffec0 2024-09-11T22:13:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.893+0000 7fb1e4070640 1 -- 172.21.15.5:0/659009796 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb1c8039060 msgr2=0x7fb1c803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.893+0000 7fb1e4070640 1 --2- 172.21.15.5:0/659009796 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb1c8039060 0x7fb1c803b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0098c0 tx=0x7fb1cc006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.894+0000 7fb1e4070640 1 -- 172.21.15.5:0/659009796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1dc0ffec0 msgr2=0x7fb1dc198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.894+0000 7fb1e4070640 1 --2- 172.21.15.5:0/659009796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1dc0ffec0 0x7fb1dc198f40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8004f60 tx=0x7fb1d80048f0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.894+0000 7fb1e4070640 1 -- 172.21.15.5:0/659009796 shutdown_connections 2024-09-11T22:13:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.894+0000 7fb1e4070640 1 --2- 172.21.15.5:0/659009796 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb1c8039060 0x7fb1c803b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.894+0000 7fb1e4070640 1 --2- 172.21.15.5:0/659009796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1dc0ffec0 0x7fb1dc198f40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.894+0000 7fb1e4070640 1 -- 172.21.15.5:0/659009796 >> 172.21.15.5:0/659009796 conn(0x7fb1dc0fbcd0 msgr2=0x7fb1dc0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.894+0000 7fb1e4070640 1 -- 172.21.15.5:0/659009796 shutdown_connections 2024-09-11T22:13:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:25.894+0000 7fb1e4070640 1 -- 172.21.15.5:0/659009796 wait complete. 2024-09-11T22:13:26.968 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T22:13:26.968 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T22:13:27.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:26 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/659009796' entity='client.admin' 2024-09-11T22:13:27.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:27.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:27.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:13:27.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:27.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:13:27.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:27.219 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.654+0000 7fa45aa7d640 1 -- 172.21.15.5:0/189055177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa454100510 msgr2=0x7fa454100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.654+0000 7fa45aa7d640 1 --2- 172.21.15.5:0/189055177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa454100510 0x7fa454100910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa448009880 tx=0x7fa44802a5d0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.654+0000 7fa45aa7d640 1 -- 172.21.15.5:0/189055177 shutdown_connections 2024-09-11T22:13:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.654+0000 7fa45aa7d640 1 --2- 172.21.15.5:0/189055177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa454100510 0x7fa454100910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.654+0000 7fa45aa7d640 1 -- 172.21.15.5:0/189055177 >> 172.21.15.5:0/189055177 conn(0x7fa4540fbc80 msgr2=0x7fa4540fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.654+0000 7fa45aa7d640 1 -- 172.21.15.5:0/189055177 shutdown_connections 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.654+0000 7fa45aa7d640 1 -- 172.21.15.5:0/189055177 wait complete. 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.655+0000 7fa45aa7d640 1 Processor -- start 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.655+0000 7fa45aa7d640 1 -- start start 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.655+0000 7fa45aa7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa454100510 0x7fa454196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.655+0000 7fa45aa7d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa454106620 con 0x7fa454100510 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.655+0000 7fa458819640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa454100510 0x7fa454196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.655+0000 7fa458819640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa454100510 0x7fa454196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48042/0 (socket says 172.21.15.5:48042) 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.655+0000 7fa458819640 1 -- 172.21.15.5:0/2949935624 learned_addr learned my addr 172.21.15.5:0/2949935624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.656+0000 7fa458819640 1 -- 172.21.15.5:0/2949935624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4541972e0 con 0x7fa454100510 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.656+0000 7fa458819640 1 --2- 172.21.15.5:0/2949935624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa454100510 0x7fa454196da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa448004f60 tx=0x7fa4480048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.656+0000 7fa451ffb640 1 -- 172.21.15.5:0/2949935624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa44803f070 con 0x7fa454100510 2024-09-11T22:13:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.656+0000 7fa451ffb640 1 -- 172.21.15.5:0/2949935624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa448004d40 con 0x7fa454100510 2024-09-11T22:13:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.656+0000 7fa45aa7d640 1 -- 172.21.15.5:0/2949935624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa454197570 con 0x7fa454100510 2024-09-11T22:13:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.656+0000 7fa451ffb640 1 -- 172.21.15.5:0/2949935624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa448035030 con 0x7fa454100510 2024-09-11T22:13:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.656+0000 7fa45aa7d640 1 -- 172.21.15.5:0/2949935624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45419a220 con 0x7fa454100510 2024-09-11T22:13:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.657+0000 7fa451ffb640 1 -- 172.21.15.5:0/2949935624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fa448042070 con 0x7fa454100510 2024-09-11T22:13:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.657+0000 7fa451ffb640 1 --2- 172.21.15.5:0/2949935624 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa444039060 0x7fa44403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.657+0000 7fa45aa7d640 1 -- 172.21.15.5:0/2949935624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa454106da0 con 0x7fa454100510 2024-09-11T22:13:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.657+0000 7fa451ffb640 1 -- 172.21.15.5:0/2949935624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa44806ce00 con 0x7fa454100510 2024-09-11T22:13:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.657+0000 7fa453fff640 1 --2- 172.21.15.5:0/2949935624 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa444039060 0x7fa44403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.658+0000 7fa453fff640 1 --2- 172.21.15.5:0/2949935624 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa444039060 0x7fa44403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa4400098c0 tx=0x7fa440006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:27.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.663+0000 7fa451ffb640 1 -- 172.21.15.5:0/2949935624 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4480303a0 con 0x7fa454100510 2024-09-11T22:13:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.841+0000 7fa45aa7d640 1 -- 172.21.15.5:0/2949935624 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa45402d940 con 0x7fa444039060 2024-09-11T22:13:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.845+0000 7fa451ffb640 1 -- 172.21.15.5:0/2949935624 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa45402d940 con 0x7fa444039060 2024-09-11T22:13:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.847+0000 7fa45aa7d640 1 -- 172.21.15.5:0/2949935624 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa444039060 msgr2=0x7fa44403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.847+0000 7fa45aa7d640 1 --2- 172.21.15.5:0/2949935624 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa444039060 0x7fa44403b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa4400098c0 tx=0x7fa440006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.847+0000 7fa45aa7d640 1 -- 172.21.15.5:0/2949935624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa454100510 msgr2=0x7fa454196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.847+0000 7fa45aa7d640 1 --2- 172.21.15.5:0/2949935624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa454100510 0x7fa454196da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa448004f60 tx=0x7fa4480048f0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.847+0000 7fa45aa7d640 1 -- 172.21.15.5:0/2949935624 shutdown_connections 2024-09-11T22:13:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.847+0000 7fa45aa7d640 1 --2- 172.21.15.5:0/2949935624 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa444039060 0x7fa44403b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.847+0000 7fa45aa7d640 1 --2- 172.21.15.5:0/2949935624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa454100510 0x7fa454196da0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.847+0000 7fa45aa7d640 1 -- 172.21.15.5:0/2949935624 >> 172.21.15.5:0/2949935624 conn(0x7fa4540fbc80 msgr2=0x7fa45406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.847+0000 7fa45aa7d640 1 -- 172.21.15.5:0/2949935624 shutdown_connections 2024-09-11T22:13:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:27.847+0000 7fa45aa7d640 1 -- 172.21.15.5:0/2949935624 wait complete. 2024-09-11T22:13:28.630 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi159 2024-09-11T22:13:28.631 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:13:28.631 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.conf 2024-09-11T22:13:28.651 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:13:28.651 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:13:28.709 INFO:tasks.cephadm:Adding host smithi159 to orchestrator... 2024-09-11T22:13:28.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch host add smithi159 2024-09-11T22:13:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:29.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:28 smithi005 ceph-mon[30943]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:29.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:28 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:29.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:28 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:13:29.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:28 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:13:29.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:28 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:13:29.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:28 smithi005 ceph-mon[30943]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-11T22:13:29.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:28 smithi005 ceph-mon[30943]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:13:29.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:28 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:29.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:28 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:29.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:28 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.393+0000 7f01fc304640 1 -- 172.21.15.5:0/4109203115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f4102810 msgr2=0x7f01f4102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.393+0000 7f01fc304640 1 --2- 172.21.15.5:0/4109203115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f4102810 0x7f01f4102bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009880 tx=0x7f01f002a670 comp rx=0 tx=0).stop 2024-09-11T22:13:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.393+0000 7f01fc304640 1 -- 172.21.15.5:0/4109203115 shutdown_connections 2024-09-11T22:13:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.393+0000 7f01fc304640 1 --2- 172.21.15.5:0/4109203115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f4102810 0x7f01f4102bf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.393+0000 7f01fc304640 1 -- 172.21.15.5:0/4109203115 >> 172.21.15.5:0/4109203115 conn(0x7f01f40fc1b0 msgr2=0x7f01f40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.393+0000 7f01fc304640 1 -- 172.21.15.5:0/4109203115 shutdown_connections 2024-09-11T22:13:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.393+0000 7f01fc304640 1 -- 172.21.15.5:0/4109203115 wait complete. 2024-09-11T22:13:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.393+0000 7f01fc304640 1 Processor -- start 2024-09-11T22:13:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.394+0000 7f01fc304640 1 -- start start 2024-09-11T22:13:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.394+0000 7f01fc304640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f4102810 0x7f01f4100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.394+0000 7f01fc304640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01f410bcb0 con 0x7f01f4102810 2024-09-11T22:13:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.394+0000 7f01fa0a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f4102810 0x7f01f4100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.394+0000 7f01fa0a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f4102810 0x7f01f4100470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48052/0 (socket says 172.21.15.5:48052) 2024-09-11T22:13:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.394+0000 7f01fa0a0640 1 -- 172.21.15.5:0/2974634057 learned_addr learned my addr 172.21.15.5:0/2974634057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:29.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.394+0000 7f01fa0a0640 1 -- 172.21.15.5:0/2974634057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f4102360 con 0x7f01f4102810 2024-09-11T22:13:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.394+0000 7f01fa0a0640 1 --2- 172.21.15.5:0/2974634057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f4102810 0x7f01f4100470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009230 tx=0x7f01f0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.395+0000 7f01eb7fe640 1 -- 172.21.15.5:0/2974634057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01f003f070 con 0x7f01f4102810 2024-09-11T22:13:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.395+0000 7f01eb7fe640 1 -- 172.21.15.5:0/2974634057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01f003bc80 con 0x7f01f4102810 2024-09-11T22:13:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.395+0000 7f01fc304640 1 -- 172.21.15.5:0/2974634057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f41009b0 con 0x7f01f4102810 2024-09-11T22:13:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.395+0000 7f01eb7fe640 1 -- 172.21.15.5:0/2974634057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01f0035030 con 0x7f01f4102810 2024-09-11T22:13:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.395+0000 7f01fc304640 1 -- 172.21.15.5:0/2974634057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f4100e70 con 0x7f01f4102810 2024-09-11T22:13:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.396+0000 7f01eb7fe640 1 -- 172.21.15.5:0/2974634057 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f01f0042070 con 0x7f01f4102810 2024-09-11T22:13:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.396+0000 7f01fc304640 1 -- 172.21.15.5:0/2974634057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f406a6f0 con 0x7f01f4102810 2024-09-11T22:13:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.396+0000 7f01eb7fe640 1 --2- 172.21.15.5:0/2974634057 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f01e00390c0 0x7f01e003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.396+0000 7f01eb7fe640 1 -- 172.21.15.5:0/2974634057 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f01f006ccc0 con 0x7f01f4102810 2024-09-11T22:13:29.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.396+0000 7f01f989f640 1 --2- 172.21.15.5:0/2974634057 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f01e00390c0 0x7f01e003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:29.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.396+0000 7f01f989f640 1 --2- 172.21.15.5:0/2974634057 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f01e00390c0 0x7f01e003b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01e40098c0 tx=0x7f01e4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.401+0000 7f01eb7fe640 1 -- 172.21.15.5:0/2974634057 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01f00433b0 con 0x7f01f4102810 2024-09-11T22:13:29.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:29.578+0000 7f01fc304640 1 -- 172.21.15.5:0/2974634057 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}) -- 0x7f01f41019a0 con 0x7f01e00390c0 2024-09-11T22:13:30.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:29 smithi005 ceph-mon[30943]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:13:30.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:29 smithi005 ceph-mon[30943]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:13:31.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:30 smithi005 ceph-mon[30943]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:32.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:31 smithi005 ceph-mon[30943]: Deploying cephadm binary to smithi159 2024-09-11T22:13:33.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.475+0000 7f01eb7fe640 1 -- 172.21.15.5:0/2974634057 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f01f41019a0 con 0x7f01e00390c0 2024-09-11T22:13:33.476 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi159' with addr '172.21.15.159' 2024-09-11T22:13:33.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.477+0000 7f01fc304640 1 -- 172.21.15.5:0/2974634057 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f01e00390c0 msgr2=0x7f01e003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:33.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.477+0000 7f01fc304640 1 --2- 172.21.15.5:0/2974634057 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f01e00390c0 0x7f01e003b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01e40098c0 tx=0x7f01e4006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:33.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.477+0000 7f01fc304640 1 -- 172.21.15.5:0/2974634057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f4102810 msgr2=0x7f01f4100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:33.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.477+0000 7f01fc304640 1 --2- 172.21.15.5:0/2974634057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f4102810 0x7f01f4100470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009230 tx=0x7f01f0003d40 comp rx=0 tx=0).stop 2024-09-11T22:13:33.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.477+0000 7f01fc304640 1 -- 172.21.15.5:0/2974634057 shutdown_connections 2024-09-11T22:13:33.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.477+0000 7f01fc304640 1 --2- 172.21.15.5:0/2974634057 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f01e00390c0 0x7f01e003b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:33.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.477+0000 7f01fc304640 1 --2- 172.21.15.5:0/2974634057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01f4102810 0x7f01f4100470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:33.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.477+0000 7f01fc304640 1 -- 172.21.15.5:0/2974634057 >> 172.21.15.5:0/2974634057 conn(0x7f01f40fc1b0 msgr2=0x7f01f41055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:33.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.477+0000 7f01fc304640 1 -- 172.21.15.5:0/2974634057 shutdown_connections 2024-09-11T22:13:33.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:33.477+0000 7f01fc304640 1 -- 172.21.15.5:0/2974634057 wait complete. 2024-09-11T22:13:34.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T22:13:34.394 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:34.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:34.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:34 smithi005 ceph-mon[30943]: Added host smithi159 2024-09-11T22:13:34.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:13:34.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:34.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.817+0000 7fdaf6cda640 1 -- 172.21.15.5:0/1305460855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf0102730 msgr2=0x7fdaf0102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.817+0000 7fdaf6cda640 1 --2- 172.21.15.5:0/1305460855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf0102730 0x7fdaf0102b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009810 tx=0x7fdad802a540 comp rx=0 tx=0).stop 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.817+0000 7fdaf6cda640 1 -- 172.21.15.5:0/1305460855 shutdown_connections 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.817+0000 7fdaf6cda640 1 --2- 172.21.15.5:0/1305460855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf0102730 0x7fdaf0102b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.817+0000 7fdaf6cda640 1 -- 172.21.15.5:0/1305460855 >> 172.21.15.5:0/1305460855 conn(0x7fdaf0077310 msgr2=0x7fdaf00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.817+0000 7fdaf6cda640 1 -- 172.21.15.5:0/1305460855 shutdown_connections 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.817+0000 7fdaf6cda640 1 -- 172.21.15.5:0/1305460855 wait complete. 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.818+0000 7fdaf6cda640 1 Processor -- start 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.818+0000 7fdaf6cda640 1 -- start start 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.818+0000 7fdaf6cda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf0102730 0x7fdaf0199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.818+0000 7fdaf6cda640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf010bc80 con 0x7fdaf0102730 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.818+0000 7fdaf4a76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf0102730 0x7fdaf0199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.818+0000 7fdaf4a76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf0102730 0x7fdaf0199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38378/0 (socket says 172.21.15.5:38378) 2024-09-11T22:13:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.818+0000 7fdaf4a76640 1 -- 172.21.15.5:0/3101112235 learned_addr learned my addr 172.21.15.5:0/3101112235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.818+0000 7fdaf4a76640 1 -- 172.21.15.5:0/3101112235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf0199950 con 0x7fdaf0102730 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.819+0000 7fdaf4a76640 1 --2- 172.21.15.5:0/3101112235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf0102730 0x7fdaf0199410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdad80045e0 tx=0x7fdad8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.819+0000 7fdaedffb640 1 -- 172.21.15.5:0/3101112235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdad8030e70 con 0x7fdaf0102730 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.819+0000 7fdaedffb640 1 -- 172.21.15.5:0/3101112235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad80318e0 con 0x7fdaf0102730 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.819+0000 7fdaf6cda640 1 -- 172.21.15.5:0/3101112235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf0199be0 con 0x7fdaf0102730 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.819+0000 7fdaedffb640 1 -- 172.21.15.5:0/3101112235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdad80049c0 con 0x7fdaf0102730 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.819+0000 7fdaf6cda640 1 -- 172.21.15.5:0/3101112235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf019a040 con 0x7fdaf0102730 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.820+0000 7fdaf6cda640 1 -- 172.21.15.5:0/3101112235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf0069f90 con 0x7fdaf0102730 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.820+0000 7fdaedffb640 1 -- 172.21.15.5:0/3101112235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fdad8039040 con 0x7fdaf0102730 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.820+0000 7fdaedffb640 1 --2- 172.21.15.5:0/3101112235 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdadc0390c0 0x7fdadc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.820+0000 7fdaedffb640 1 -- 172.21.15.5:0/3101112235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdad806d2b0 con 0x7fdaf0102730 2024-09-11T22:13:34.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.820+0000 7fdaeffff640 1 --2- 172.21.15.5:0/3101112235 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdadc0390c0 0x7fdadc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:34.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.821+0000 7fdaeffff640 1 --2- 172.21.15.5:0/3101112235 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdadc0390c0 0x7fdadc03b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdae00098c0 tx=0x7fdae0006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:34.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:34.825+0000 7fdaedffb640 1 -- 172.21.15.5:0/3101112235 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdad8040030 con 0x7fdaf0102730 2024-09-11T22:13:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.007+0000 7fdaf6cda640 1 -- 172.21.15.5:0/3101112235 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdaf019a770 con 0x7fdadc0390c0 2024-09-11T22:13:35.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.008+0000 7fdaedffb640 1 -- 172.21.15.5:0/3101112235 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fdaf019a770 con 0x7fdadc0390c0 2024-09-11T22:13:35.009 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:35.009 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}] 2024-09-11T22:13:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.010+0000 7fdaf6cda640 1 -- 172.21.15.5:0/3101112235 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdadc0390c0 msgr2=0x7fdadc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.010+0000 7fdaf6cda640 1 --2- 172.21.15.5:0/3101112235 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdadc0390c0 0x7fdadc03b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdae00098c0 tx=0x7fdae0006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.010+0000 7fdaf6cda640 1 -- 172.21.15.5:0/3101112235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf0102730 msgr2=0x7fdaf0199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.010+0000 7fdaf6cda640 1 --2- 172.21.15.5:0/3101112235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf0102730 0x7fdaf0199410 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdad80045e0 tx=0x7fdad8004610 comp rx=0 tx=0).stop 2024-09-11T22:13:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.010+0000 7fdaf6cda640 1 -- 172.21.15.5:0/3101112235 shutdown_connections 2024-09-11T22:13:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.010+0000 7fdaf6cda640 1 --2- 172.21.15.5:0/3101112235 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdadc0390c0 0x7fdadc03b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.010+0000 7fdaf6cda640 1 --2- 172.21.15.5:0/3101112235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaf0102730 0x7fdaf0199410 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:35.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.010+0000 7fdaf6cda640 1 -- 172.21.15.5:0/3101112235 >> 172.21.15.5:0/3101112235 conn(0x7fdaf0077310 msgr2=0x7fdaf01078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.010+0000 7fdaf6cda640 1 -- 172.21.15.5:0/3101112235 shutdown_connections 2024-09-11T22:13:35.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:35.010+0000 7fdaf6cda640 1 -- 172.21.15.5:0/3101112235 wait complete. 2024-09-11T22:13:35.626 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi172 2024-09-11T22:13:35.627 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:13:35.627 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.conf 2024-09-11T22:13:35.647 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:13:35.647 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:13:35.705 INFO:tasks.cephadm:Adding host smithi172 to orchestrator... 2024-09-11T22:13:35.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch host add smithi172 2024-09-11T22:13:35.958 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:36.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:35 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:36.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:35 smithi005 ceph-mon[30943]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:13:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.440+0000 7fcb94264640 1 -- 172.21.15.5:0/3622544395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c107660 msgr2=0x7fcb8c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.440+0000 7fcb94264640 1 --2- 172.21.15.5:0/3622544395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c107660 0x7fcb8c107a40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009880 tx=0x7fcb8802a5d0 comp rx=0 tx=0).stop 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.440+0000 7fcb94264640 1 -- 172.21.15.5:0/3622544395 shutdown_connections 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.440+0000 7fcb94264640 1 --2- 172.21.15.5:0/3622544395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c107660 0x7fcb8c107a40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.440+0000 7fcb94264640 1 -- 172.21.15.5:0/3622544395 >> 172.21.15.5:0/3622544395 conn(0x7fcb8c0fd290 msgr2=0x7fcb8c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.440+0000 7fcb94264640 1 -- 172.21.15.5:0/3622544395 shutdown_connections 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.440+0000 7fcb94264640 1 -- 172.21.15.5:0/3622544395 wait complete. 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.441+0000 7fcb94264640 1 Processor -- start 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.441+0000 7fcb94264640 1 -- start start 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.441+0000 7fcb94264640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c107660 0x7fcb8c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.441+0000 7fcb94264640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c10bc90 con 0x7fcb8c107660 2024-09-11T22:13:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.441+0000 7fcb92000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c107660 0x7fcb8c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.441+0000 7fcb92000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c107660 0x7fcb8c1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38386/0 (socket says 172.21.15.5:38386) 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.441+0000 7fcb92000640 1 -- 172.21.15.5:0/1861113062 learned_addr learned my addr 172.21.15.5:0/1861113062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.441+0000 7fcb92000640 1 -- 172.21.15.5:0/1861113062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c197600 con 0x7fcb8c107660 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.442+0000 7fcb92000640 1 --2- 172.21.15.5:0/1861113062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c107660 0x7fcb8c1970c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009230 tx=0x7fcb88003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.442+0000 7fcb82ffd640 1 -- 172.21.15.5:0/1861113062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb8803f070 con 0x7fcb8c107660 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.442+0000 7fcb82ffd640 1 -- 172.21.15.5:0/1861113062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb8803bc80 con 0x7fcb8c107660 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.442+0000 7fcb94264640 1 -- 172.21.15.5:0/1861113062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c197890 con 0x7fcb8c107660 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.442+0000 7fcb82ffd640 1 -- 172.21.15.5:0/1861113062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb88035030 con 0x7fcb8c107660 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.442+0000 7fcb94264640 1 -- 172.21.15.5:0/1861113062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c19b990 con 0x7fcb8c107660 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.443+0000 7fcb82ffd640 1 -- 172.21.15.5:0/1861113062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fcb88042070 con 0x7fcb8c107660 2024-09-11T22:13:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.443+0000 7fcb82ffd640 1 --2- 172.21.15.5:0/1861113062 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcb78039060 0x7fcb7803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.443+0000 7fcb82ffd640 1 -- 172.21.15.5:0/1861113062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcb8806ccd0 con 0x7fcb8c107660 2024-09-11T22:13:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.443+0000 7fcb917ff640 1 --2- 172.21.15.5:0/1861113062 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcb78039060 0x7fcb7803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.443+0000 7fcb94264640 1 -- 172.21.15.5:0/1861113062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8c102cd0 con 0x7fcb8c107660 2024-09-11T22:13:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.443+0000 7fcb917ff640 1 --2- 172.21.15.5:0/1861113062 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcb78039060 0x7fcb7803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0098c0 tx=0x7fcb7c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.448+0000 7fcb82ffd640 1 -- 172.21.15.5:0/1861113062 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb8804c350 con 0x7fcb8c107660 2024-09-11T22:13:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:36.630+0000 7fcb94264640 1 -- 172.21.15.5:0/1861113062 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}) -- 0x7fcb8c107af0 con 0x7fcb78039060 2024-09-11T22:13:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:37 smithi005 ceph-mon[30943]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:37 smithi005 ceph-mon[30943]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi172", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:39.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:39.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:39.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:39.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:39.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:13:39.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:13:39.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:13:39.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-11T22:13:39.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: Deploying cephadm binary to smithi172 2024-09-11T22:13:39.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:13:39.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:13:39.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:39.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:39.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:39 smithi005 ceph-mon[30943]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:39 smithi005 ceph-mon[30943]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:13:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.476+0000 7fcb82ffd640 1 -- 172.21.15.5:0/1861113062 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fcb8c107af0 con 0x7fcb78039060 2024-09-11T22:13:40.478 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi172' with addr '172.21.15.172' 2024-09-11T22:13:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.479+0000 7fcb94264640 1 -- 172.21.15.5:0/1861113062 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcb78039060 msgr2=0x7fcb7803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.479+0000 7fcb94264640 1 --2- 172.21.15.5:0/1861113062 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcb78039060 0x7fcb7803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0098c0 tx=0x7fcb7c006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.479+0000 7fcb94264640 1 -- 172.21.15.5:0/1861113062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c107660 msgr2=0x7fcb8c1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.479+0000 7fcb94264640 1 --2- 172.21.15.5:0/1861113062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c107660 0x7fcb8c1970c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009230 tx=0x7fcb88003d40 comp rx=0 tx=0).stop 2024-09-11T22:13:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.479+0000 7fcb94264640 1 -- 172.21.15.5:0/1861113062 shutdown_connections 2024-09-11T22:13:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.479+0000 7fcb94264640 1 --2- 172.21.15.5:0/1861113062 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcb78039060 0x7fcb7803b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.479+0000 7fcb94264640 1 --2- 172.21.15.5:0/1861113062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb8c107660 0x7fcb8c1970c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.479+0000 7fcb94264640 1 -- 172.21.15.5:0/1861113062 >> 172.21.15.5:0/1861113062 conn(0x7fcb8c0fd290 msgr2=0x7fcb8c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.479+0000 7fcb94264640 1 -- 172.21.15.5:0/1861113062 shutdown_connections 2024-09-11T22:13:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:40.479+0000 7fcb94264640 1 -- 172.21.15.5:0/1861113062 wait complete. 2024-09-11T22:13:41.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T22:13:41.364 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:41.730 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:41 smithi005 ceph-mon[30943]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:41.730 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:41.730 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:41 smithi005 ceph-mon[30943]: Added host smithi172 2024-09-11T22:13:41.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:13:41.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:41.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.812+0000 7f2eb5b92640 1 -- 172.21.15.5:0/4064849389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb0107660 msgr2=0x7f2eb0107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.812+0000 7f2eb5b92640 1 --2- 172.21.15.5:0/4064849389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb0107660 0x7f2eb0107a40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009880 tx=0x7f2ea002a670 comp rx=0 tx=0).stop 2024-09-11T22:13:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.812+0000 7f2eb5b92640 1 -- 172.21.15.5:0/4064849389 shutdown_connections 2024-09-11T22:13:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.812+0000 7f2eb5b92640 1 --2- 172.21.15.5:0/4064849389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb0107660 0x7f2eb0107a40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.812+0000 7f2eb5b92640 1 -- 172.21.15.5:0/4064849389 >> 172.21.15.5:0/4064849389 conn(0x7f2eb00fb930 msgr2=0x7f2eb00fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.812+0000 7f2eb5b92640 1 -- 172.21.15.5:0/4064849389 shutdown_connections 2024-09-11T22:13:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.812+0000 7f2eb5b92640 1 -- 172.21.15.5:0/4064849389 wait complete. 2024-09-11T22:13:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.813+0000 7f2eb5b92640 1 Processor -- start 2024-09-11T22:13:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.813+0000 7f2eb5b92640 1 -- start start 2024-09-11T22:13:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.813+0000 7f2eb5b92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb0107660 0x7f2eb0100bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.813+0000 7f2eb5b92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb010bc50 con 0x7f2eb0107660 2024-09-11T22:13:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.813+0000 7f2eaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb0107660 0x7f2eb0100bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.813+0000 7f2eaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb0107660 0x7f2eb0100bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57982/0 (socket says 172.21.15.5:57982) 2024-09-11T22:13:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.813+0000 7f2eaf7fe640 1 -- 172.21.15.5:0/2490960340 learned_addr learned my addr 172.21.15.5:0/2490960340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.814+0000 7f2eaf7fe640 1 -- 172.21.15.5:0/2490960340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb0102ab0 con 0x7f2eb0107660 2024-09-11T22:13:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.814+0000 7f2eaf7fe640 1 --2- 172.21.15.5:0/2490960340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb0107660 0x7f2eb0100bc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009230 tx=0x7f2ea0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.814+0000 7f2eacff9640 1 -- 172.21.15.5:0/2490960340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ea003f070 con 0x7f2eb0107660 2024-09-11T22:13:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.814+0000 7f2eacff9640 1 -- 172.21.15.5:0/2490960340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ea003bc80 con 0x7f2eb0107660 2024-09-11T22:13:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.814+0000 7f2eb5b92640 1 -- 172.21.15.5:0/2490960340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb0101160 con 0x7f2eb0107660 2024-09-11T22:13:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.814+0000 7f2eacff9640 1 -- 172.21.15.5:0/2490960340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ea0035030 con 0x7f2eb0107660 2024-09-11T22:13:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.814+0000 7f2eb5b92640 1 -- 172.21.15.5:0/2490960340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb0101620 con 0x7f2eb0107660 2024-09-11T22:13:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.815+0000 7f2eacff9640 1 -- 172.21.15.5:0/2490960340 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f2ea0042070 con 0x7f2eb0107660 2024-09-11T22:13:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.815+0000 7f2eb5b92640 1 -- 172.21.15.5:0/2490960340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb0107ac0 con 0x7f2eb0107660 2024-09-11T22:13:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.815+0000 7f2eacff9640 1 --2- 172.21.15.5:0/2490960340 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2e98039060 0x7f2e9803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.815+0000 7f2eacff9640 1 -- 172.21.15.5:0/2490960340 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2ea006cda0 con 0x7f2eb0107660 2024-09-11T22:13:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.815+0000 7f2eaeffd640 1 --2- 172.21.15.5:0/2490960340 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2e98039060 0x7f2e9803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:41.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.816+0000 7f2eaeffd640 1 --2- 172.21.15.5:0/2490960340 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2e98039060 0x7f2e9803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40098c0 tx=0x7f2ea4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:41.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:41.820+0000 7f2eacff9640 1 -- 172.21.15.5:0/2490960340 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ea004c350 con 0x7f2eb0107660 2024-09-11T22:13:42.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.013+0000 7f2eb5b92640 1 -- 172.21.15.5:0/2490960340 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2eb0102ce0 con 0x7f2e98039060 2024-09-11T22:13:42.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.015+0000 7f2eacff9640 1 -- 172.21.15.5:0/2490960340 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f2eb0102ce0 con 0x7f2e98039060 2024-09-11T22:13:42.016 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:13:42.017 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}, {"addr": "172.21.15.172", "hostname": "smithi172", "labels": [], "status": ""}] 2024-09-11T22:13:42.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.017+0000 7f2eb5b92640 1 -- 172.21.15.5:0/2490960340 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2e98039060 msgr2=0x7f2e9803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:42.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.017+0000 7f2eb5b92640 1 --2- 172.21.15.5:0/2490960340 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2e98039060 0x7f2e9803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40098c0 tx=0x7f2ea4006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:42.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.017+0000 7f2eb5b92640 1 -- 172.21.15.5:0/2490960340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb0107660 msgr2=0x7f2eb0100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:42.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.017+0000 7f2eb5b92640 1 --2- 172.21.15.5:0/2490960340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb0107660 0x7f2eb0100bc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009230 tx=0x7f2ea0003d40 comp rx=0 tx=0).stop 2024-09-11T22:13:42.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.017+0000 7f2eb5b92640 1 -- 172.21.15.5:0/2490960340 shutdown_connections 2024-09-11T22:13:42.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.017+0000 7f2eb5b92640 1 --2- 172.21.15.5:0/2490960340 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2e98039060 0x7f2e9803b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:42.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.017+0000 7f2eb5b92640 1 --2- 172.21.15.5:0/2490960340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2eb0107660 0x7f2eb0100bc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:42.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.017+0000 7f2eb5b92640 1 -- 172.21.15.5:0/2490960340 >> 172.21.15.5:0/2490960340 conn(0x7f2eb00fb930 msgr2=0x7f2eb00fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:42.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.017+0000 7f2eb5b92640 1 -- 172.21.15.5:0/2490960340 shutdown_connections 2024-09-11T22:13:42.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:42.017+0000 7f2eb5b92640 1 -- 172.21.15.5:0/2490960340 wait complete. 2024-09-11T22:13:42.696 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T22:13:42.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T22:13:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:13:43.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:42 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:43.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:42 smithi005 ceph-mon[30943]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:13:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.420+0000 7f46b5eec640 1 -- 172.21.15.5:0/297427012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46b0102730 msgr2=0x7f46b0102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.420+0000 7f46b5eec640 1 --2- 172.21.15.5:0/297427012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46b0102730 0x7f46b0102b10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009c10 tx=0x7f46a802aa10 comp rx=0 tx=0).stop 2024-09-11T22:13:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.420+0000 7f46b5eec640 1 -- 172.21.15.5:0/297427012 shutdown_connections 2024-09-11T22:13:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.420+0000 7f46b5eec640 1 --2- 172.21.15.5:0/297427012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46b0102730 0x7f46b0102b10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.420+0000 7f46b5eec640 1 -- 172.21.15.5:0/297427012 >> 172.21.15.5:0/297427012 conn(0x7f46b0077310 msgr2=0x7f46b00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.420+0000 7f46b5eec640 1 -- 172.21.15.5:0/297427012 shutdown_connections 2024-09-11T22:13:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.420+0000 7f46b5eec640 1 -- 172.21.15.5:0/297427012 wait complete. 2024-09-11T22:13:43.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.420+0000 7f46b5eec640 1 Processor -- start 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.421+0000 7f46b5eec640 1 -- start start 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.421+0000 7f46b5eec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46b0102730 0x7f46b01971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.421+0000 7f46b5eec640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46b010bc80 con 0x7f46b0102730 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.421+0000 7f46af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46b0102730 0x7f46b01971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.421+0000 7f46af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46b0102730 0x7f46b01971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58002/0 (socket says 172.21.15.5:58002) 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.421+0000 7f46af7fe640 1 -- 172.21.15.5:0/1547380883 learned_addr learned my addr 172.21.15.5:0/1547380883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.421+0000 7f46af7fe640 1 -- 172.21.15.5:0/1547380883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b01976f0 con 0x7f46b0102730 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.421+0000 7f46af7fe640 1 --2- 172.21.15.5:0/1547380883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46b0102730 0x7f46b01971b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f46a8005dd0 tx=0x7f46a8030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.422+0000 7f46acff9640 1 -- 172.21.15.5:0/1547380883 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46a8034070 con 0x7f46b0102730 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.422+0000 7f46acff9640 1 -- 172.21.15.5:0/1547380883 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a802f8c0 con 0x7f46b0102730 2024-09-11T22:13:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.422+0000 7f46b5eec640 1 -- 172.21.15.5:0/1547380883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b0197980 con 0x7f46b0102730 2024-09-11T22:13:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.422+0000 7f46acff9640 1 -- 172.21.15.5:0/1547380883 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46a8004520 con 0x7f46b0102730 2024-09-11T22:13:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.422+0000 7f46b5eec640 1 -- 172.21.15.5:0/1547380883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b0197d20 con 0x7f46b0102730 2024-09-11T22:13:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.423+0000 7f46acff9640 1 -- 172.21.15.5:0/1547380883 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f46a8037020 con 0x7f46b0102730 2024-09-11T22:13:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.423+0000 7f46b5eec640 1 -- 172.21.15.5:0/1547380883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b0069f90 con 0x7f46b0102730 2024-09-11T22:13:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.423+0000 7f46acff9640 1 --2- 172.21.15.5:0/1547380883 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f469c039060 0x7f469c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.423+0000 7f46acff9640 1 -- 172.21.15.5:0/1547380883 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f46a806cf40 con 0x7f46b0102730 2024-09-11T22:13:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.423+0000 7f46aeffd640 1 --2- 172.21.15.5:0/1547380883 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f469c039060 0x7f469c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.423+0000 7f46aeffd640 1 --2- 172.21.15.5:0/1547380883 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f469c039060 0x7f469c03b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f46980098c0 tx=0x7f4698006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:43.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.428+0000 7f46acff9640 1 -- 172.21.15.5:0/1547380883 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46a803f070 con 0x7f46b0102730 2024-09-11T22:13:43.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.606+0000 7f46b5eec640 1 -- 172.21.15.5:0/1547380883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f46b0198510 con 0x7f46b0102730 2024-09-11T22:13:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.833+0000 7f46acff9640 1 -- 172.21.15.5:0/1547380883 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f46a802c040 con 0x7f46b0102730 2024-09-11T22:13:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-09-11T22:13:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.835+0000 7f46b5eec640 1 -- 172.21.15.5:0/1547380883 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f469c039060 msgr2=0x7f469c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.835+0000 7f46b5eec640 1 --2- 172.21.15.5:0/1547380883 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f469c039060 0x7f469c03b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f46980098c0 tx=0x7f4698006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.835+0000 7f46b5eec640 1 -- 172.21.15.5:0/1547380883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46b0102730 msgr2=0x7f46b01971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.835+0000 7f46b5eec640 1 --2- 172.21.15.5:0/1547380883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46b0102730 0x7f46b01971b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f46a8005dd0 tx=0x7f46a8030c30 comp rx=0 tx=0).stop 2024-09-11T22:13:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.835+0000 7f46b5eec640 1 -- 172.21.15.5:0/1547380883 shutdown_connections 2024-09-11T22:13:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.835+0000 7f46b5eec640 1 --2- 172.21.15.5:0/1547380883 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f469c039060 0x7f469c03b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.835+0000 7f46b5eec640 1 --2- 172.21.15.5:0/1547380883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46b0102730 0x7f46b01971b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.835+0000 7f46b5eec640 1 -- 172.21.15.5:0/1547380883 >> 172.21.15.5:0/1547380883 conn(0x7f46b0077310 msgr2=0x7f46b01054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.835+0000 7f46b5eec640 1 -- 172.21.15.5:0/1547380883 shutdown_connections 2024-09-11T22:13:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:13:43.835+0000 7f46b5eec640 1 -- 172.21.15.5:0/1547380883 wait complete. 2024-09-11T22:13:44.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:43 smithi005 ceph-mon[30943]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:44.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:43 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1547380883' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T22:13:44.475 INFO:tasks.cephadm:Adding mon.a on smithi005 2024-09-11T22:13:44.475 INFO:tasks.cephadm:Adding mon.b on smithi159 2024-09-11T22:13:44.476 INFO:tasks.cephadm:Adding mon.c on smithi172 2024-09-11T22:13:44.476 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi159:172.21.15.159=b;smithi172:172.21.15.172=c' 2024-09-11T22:13:44.718 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T22:13:44.764 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-11T22:13:45.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.230+0000 7f63ce09f640 1 -- 172.21.15.172:0/3681704925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c81004e0 msgr2=0x7f63c81008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.230+0000 7f63ce09f640 1 --2- 172.21.15.172:0/3681704925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c81004e0 0x7f63c81008e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009880 tx=0x7f63b802a5d0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.230+0000 7f63ce09f640 1 -- 172.21.15.172:0/3681704925 shutdown_connections 2024-09-11T22:13:45.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.230+0000 7f63ce09f640 1 --2- 172.21.15.172:0/3681704925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c81004e0 0x7f63c81008e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.230+0000 7f63ce09f640 1 -- 172.21.15.172:0/3681704925 >> 172.21.15.172:0/3681704925 conn(0x7f63c80fbc90 msgr2=0x7f63c80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:45.260 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.230+0000 7f63ce09f640 1 -- 172.21.15.172:0/3681704925 shutdown_connections 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.230+0000 7f63ce09f640 1 -- 172.21.15.172:0/3681704925 wait complete. 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.231+0000 7f63ce09f640 1 Processor -- start 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.231+0000 7f63ce09f640 1 -- start start 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.231+0000 7f63ce09f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c81004e0 0x7f63c80fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.231+0000 7f63ce09f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81065f0 con 0x7f63c81004e0 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.232+0000 7f63c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c81004e0 0x7f63c80fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.232+0000 7f63c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c81004e0 0x7f63c80fdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.172:56470/0 (socket says 172.21.15.172:56470) 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.232+0000 7f63c77fe640 1 -- 172.21.15.172:0/79076940 learned_addr learned my addr 172.21.15.172:0/79076940 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.232+0000 7f63c77fe640 1 -- 172.21.15.172:0/79076940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c80fe040 con 0x7f63c81004e0 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.232+0000 7f63c77fe640 1 --2- 172.21.15.172:0/79076940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c81004e0 0x7f63c80fdb00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f63b8004f60 tx=0x7f63b80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:45.261 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.232+0000 7f63c4ff9640 1 -- 172.21.15.172:0/79076940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63b803f070 con 0x7f63c81004e0 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.232+0000 7f63c4ff9640 1 -- 172.21.15.172:0/79076940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b8004d40 con 0x7f63c81004e0 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.232+0000 7f63ce09f640 1 -- 172.21.15.172:0/79076940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c80fc230 con 0x7f63c81004e0 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.232+0000 7f63c4ff9640 1 -- 172.21.15.172:0/79076940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63b8035030 con 0x7f63c81004e0 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.232+0000 7f63ce09f640 1 -- 172.21.15.172:0/79076940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c80fc770 con 0x7f63c81004e0 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.233+0000 7f63c4ff9640 1 -- 172.21.15.172:0/79076940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f63b8042070 con 0x7f63c81004e0 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.233+0000 7f63c4ff9640 1 --2- 172.21.15.172:0/79076940 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f63b0039060 0x7f63b003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.234+0000 7f63c4ff9640 1 -- 172.21.15.172:0/79076940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f63b806d0a0 con 0x7f63c81004e0 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.234+0000 7f63ce09f640 1 -- 172.21.15.172:0/79076940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63c8106d70 con 0x7f63c81004e0 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.234+0000 7f63c6ffd640 1 --2- 172.21.15.172:0/79076940 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f63b0039060 0x7f63b003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.234+0000 7f63c6ffd640 1 --2- 172.21.15.172:0/79076940 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f63b0039060 0x7f63b003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0098c0 tx=0x7f63bc006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:45.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.239+0000 7f63c4ff9640 1 -- 172.21.15.172:0/79076940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63b8040260 con 0x7f63c81004e0 2024-09-11T22:13:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:44 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1547380883' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T22:13:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:44 smithi005 ceph-mon[30943]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:13:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:44 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:44 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:44 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:44 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:44 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi172", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:13:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:44 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:13:45.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:44 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:13:45.439 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.438+0000 7f63ce09f640 1 -- 172.21.15.172:0/79076940 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi159:172.21.15.159=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}) -- 0x7f63c8190250 con 0x7f63b0039060 2024-09-11T22:13:45.445 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.444+0000 7f63c4ff9640 1 -- 172.21.15.172:0/79076940 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f63c8190250 con 0x7f63b0039060 2024-09-11T22:13:45.445 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mon update... 2024-09-11T22:13:45.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.446+0000 7f63ce09f640 1 -- 172.21.15.172:0/79076940 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f63b0039060 msgr2=0x7f63b003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.446+0000 7f63ce09f640 1 --2- 172.21.15.172:0/79076940 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f63b0039060 0x7f63b003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0098c0 tx=0x7f63bc006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.446+0000 7f63ce09f640 1 -- 172.21.15.172:0/79076940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c81004e0 msgr2=0x7f63c80fdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:45.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.446+0000 7f63ce09f640 1 --2- 172.21.15.172:0/79076940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c81004e0 0x7f63c80fdb00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f63b8004f60 tx=0x7f63b80048f0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.446+0000 7f63ce09f640 1 -- 172.21.15.172:0/79076940 shutdown_connections 2024-09-11T22:13:45.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.446+0000 7f63ce09f640 1 --2- 172.21.15.172:0/79076940 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f63b0039060 0x7f63b003b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.446+0000 7f63ce09f640 1 --2- 172.21.15.172:0/79076940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c81004e0 0x7f63c80fdb00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:45.447 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.446+0000 7f63ce09f640 1 -- 172.21.15.172:0/79076940 >> 172.21.15.172:0/79076940 conn(0x7f63c80fbc90 msgr2=0x7f63c806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:45.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.446+0000 7f63ce09f640 1 -- 172.21.15.172:0/79076940 shutdown_connections 2024-09-11T22:13:45.448 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:45.446+0000 7f63ce09f640 1 -- 172.21.15.172:0/79076940 wait complete. 2024-09-11T22:13:46.050 DEBUG:teuthology.orchestra.run.smithi159:mon.b> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T22:13:46.054 DEBUG:teuthology.orchestra.run.smithi172:mon.c> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T22:13:46.056 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:13:46.056 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:13:46.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:45 smithi005 ceph-mon[30943]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:46.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:45 smithi005 ceph-mon[30943]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T22:13:46.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:45 smithi005 ceph-mon[30943]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:13:46.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:45 smithi005 ceph-mon[30943]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:13:46.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:46.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:46.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:46.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:46.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:13:46.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:13:47.107 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:13:47.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:46 smithi005 ceph-mon[30943]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:13:47.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:46 smithi005 ceph-mon[30943]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi159:172.21.15.159=b;smithi172:172.21.15.172=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:13:47.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:46 smithi005 ceph-mon[30943]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi159:172.21.15.159=b;smithi172:172.21.15.172=c;count:3 2024-09-11T22:13:47.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:46 smithi005 ceph-mon[30943]: Deploying daemon mon.c on smithi172 2024-09-11T22:13:48.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:47 smithi005 ceph-mon[30943]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:48.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.510+0000 7f036b91b640 1 -- 172.21.15.172:0/3902466800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03640ffe40 msgr2=0x7f0364100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.525 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.510+0000 7f036b91b640 1 --2- 172.21.15.172:0/3902466800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03640ffe40 0x7f0364100220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0360009880 tx=0x7f036002a5d0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.510+0000 7f036b91b640 1 -- 172.21.15.172:0/3902466800 shutdown_connections 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.510+0000 7f036b91b640 1 --2- 172.21.15.172:0/3902466800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03640ffe40 0x7f0364100220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.510+0000 7f036b91b640 1 -- 172.21.15.172:0/3902466800 >> 172.21.15.172:0/3902466800 conn(0x7f03640fb8c0 msgr2=0x7f03640fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.510+0000 7f036b91b640 1 -- 172.21.15.172:0/3902466800 shutdown_connections 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.510+0000 7f036b91b640 1 -- 172.21.15.172:0/3902466800 wait complete. 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.511+0000 7f036b91b640 1 Processor -- start 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.511+0000 7f036b91b640 1 -- start start 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.511+0000 7f036b91b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03640ffe40 0x7f0364195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.511+0000 7f036b91b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f036410ab20 con 0x7f03640ffe40 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.512+0000 7f03696b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03640ffe40 0x7f0364195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.512+0000 7f03696b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03640ffe40 0x7f0364195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.172:56482/0 (socket says 172.21.15.172:56482) 2024-09-11T22:13:48.526 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.512+0000 7f03696b7640 1 -- 172.21.15.172:0/3655353099 learned_addr learned my addr 172.21.15.172:0/3655353099 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.512+0000 7f03696b7640 1 -- 172.21.15.172:0/3655353099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03641098d0 con 0x7f03640ffe40 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.512+0000 7f03696b7640 1 --2- 172.21.15.172:0/3655353099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03640ffe40 0x7f0364195d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0360009230 tx=0x7f0360003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.512+0000 7f03567fc640 1 -- 172.21.15.172:0/3655353099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f036003f070 con 0x7f03640ffe40 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.512+0000 7f036b91b640 1 -- 172.21.15.172:0/3655353099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03641962d0 con 0x7f03640ffe40 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.512+0000 7f03567fc640 1 -- 172.21.15.172:0/3655353099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f036003bd40 con 0x7f03640ffe40 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.512+0000 7f03567fc640 1 -- 172.21.15.172:0/3655353099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0360035030 con 0x7f03640ffe40 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.512+0000 7f036b91b640 1 -- 172.21.15.172:0/3655353099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0364196790 con 0x7f03640ffe40 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.513+0000 7f036b91b640 1 -- 172.21.15.172:0/3655353099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0364101320 con 0x7f03640ffe40 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.513+0000 7f03567fc640 1 -- 172.21.15.172:0/3655353099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0360042070 con 0x7f03640ffe40 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.513+0000 7f03567fc640 1 --2- 172.21.15.172:0/3655353099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0350039060 0x7f035003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.513+0000 7f03567fc640 1 -- 172.21.15.172:0/3655353099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f036006ccc0 con 0x7f03640ffe40 2024-09-11T22:13:48.527 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.514+0000 7f0368eb6640 1 --2- 172.21.15.172:0/3655353099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0350039060 0x7f035003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:48.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.514+0000 7f0368eb6640 1 --2- 172.21.15.172:0/3655353099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0350039060 0x7f035003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f03580098c0 tx=0x7f0358006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:48.528 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.519+0000 7f03567fc640 1 -- 172.21.15.172:0/3655353099 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0360073080 con 0x7f03640ffe40 2024-09-11T22:13:48.797 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.796+0000 7f036b91b640 1 -- 172.21.15.172:0/3655353099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f03641090c0 con 0x7f03640ffe40 2024-09-11T22:13:48.798 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.797+0000 7f03567fc640 1 -- 172.21.15.172:0/3655353099 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f0360030d60 con 0x7f03640ffe40 2024-09-11T22:13:48.798 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:13:48.798 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":1,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","modified":"2024-09-11T22:12:20.315558Z","created":"2024-09-11T22:12:20.315558Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T22:13:48.799 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 1 2024-09-11T22:13:48.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.799+0000 7f036b91b640 1 -- 172.21.15.172:0/3655353099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0350039060 msgr2=0x7f035003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.799+0000 7f036b91b640 1 --2- 172.21.15.172:0/3655353099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0350039060 0x7f035003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f03580098c0 tx=0x7f0358006db0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.799+0000 7f036b91b640 1 -- 172.21.15.172:0/3655353099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03640ffe40 msgr2=0x7f0364195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:48.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.799+0000 7f036b91b640 1 --2- 172.21.15.172:0/3655353099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03640ffe40 0x7f0364195d90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0360009230 tx=0x7f0360003d80 comp rx=0 tx=0).stop 2024-09-11T22:13:48.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.799+0000 7f036b91b640 1 -- 172.21.15.172:0/3655353099 shutdown_connections 2024-09-11T22:13:48.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.799+0000 7f036b91b640 1 --2- 172.21.15.172:0/3655353099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0350039060 0x7f035003b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.799+0000 7f036b91b640 1 --2- 172.21.15.172:0/3655353099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03640ffe40 0x7f0364195d90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:48.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.799+0000 7f036b91b640 1 -- 172.21.15.172:0/3655353099 >> 172.21.15.172:0/3655353099 conn(0x7f03640fb8c0 msgr2=0x7f03640fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:48.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.800+0000 7f036b91b640 1 -- 172.21.15.172:0/3655353099 shutdown_connections 2024-09-11T22:13:48.801 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:48.800+0000 7f036b91b640 1 -- 172.21.15.172:0/3655353099 wait complete. 2024-09-11T22:13:49.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:48 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/3655353099' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:13:50.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:49 smithi005 ceph-mon[30943]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:50.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:50 smithi172 systemd[1]: Started Ceph mon.c for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:13:50.540 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:13:50.540 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:13:50.805 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:13:55.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:13:55.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: mon.a calling monitor election 2024-09-11T22:13:55.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:55.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: mon.c calling monitor election 2024-09-11T22:13:55.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:55.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: monmap epoch 2 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: last_changed 2024-09-11T22:13:49.991644+0000 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: created 2024-09-11T22:12:20.315558+0000 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: min_mon_release 19 (squid) 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: election_strategy: 1 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: fsmap 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:13:55.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: mgrmap e13: x(active, since 38s) 2024-09-11T22:13:55.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: overall HEALTH_OK 2024-09-11T22:13:55.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:55.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:55.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:55.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:13:55.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: mon.a calling monitor election 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: mon.c calling monitor election 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: monmap epoch 2 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: last_changed 2024-09-11T22:13:49.991644+0000 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: created 2024-09-11T22:12:20.315558+0000 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: min_mon_release 19 (squid) 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: election_strategy: 1 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: fsmap 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:13:55.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: mgrmap e13: x(active, since 38s) 2024-09-11T22:13:55.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: overall HEALTH_OK 2024-09-11T22:13:55.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:55.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:55.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:13:55.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:13:55.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:13:55.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.726+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2778481292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0003530 msgr2=0x7fa8e00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:55.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.726+0000 7fa8fbb78640 1 --2- 172.21.15.172:0/2778481292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0003530 0x7fa8e00059c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa8f41016c0 tx=0x7fa8f002c490 comp rx=0 tx=0).stop 2024-09-11T22:13:55.728 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.727+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2778481292 shutdown_connections 2024-09-11T22:13:55.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.727+0000 7fa8fbb78640 1 --2- 172.21.15.172:0/2778481292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0003530 0x7fa8e00059c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:55.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.727+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2778481292 >> 172.21.15.172:0/2778481292 conn(0x7fa8f40fc280 msgr2=0x7fa8f40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:55.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.727+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2778481292 shutdown_connections 2024-09-11T22:13:55.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.727+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2778481292 wait complete. 2024-09-11T22:13:55.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8fbb78640 1 Processor -- start 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8fbb78640 1 -- start start 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8fbb78640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0003530 0x7fa8f41972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8fbb78640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f41977f0 0x7fa8f419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8fbb78640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4108690 con 0x7fa8e0003530 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8fbb78640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4108510 con 0x7fa8f41977f0 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8f9113640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f41977f0 0x7fa8f419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8f9113640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f41977f0 0x7fa8f419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44708/0 (socket says 172.21.15.172:44708) 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8f9113640 1 -- 172.21.15.172:0/2740313020 learned_addr learned my addr 172.21.15.172:0/2740313020 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8f9914640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0003530 0x7fa8f41972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8f9113640 1 -- 172.21.15.172:0/2740313020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f41977f0 msgr2=0x7fa8f419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8f9113640 1 -- 172.21.15.172:0/2740313020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f41977f0 msgr2=0x7fa8f419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:13:55.731 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.728+0000 7fa8f9113640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f41977f0 0x7fa8f419bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:13:55.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.729+0000 7fa8f9113640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f41977f0 0x7fa8f419bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:13:55.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.729+0000 7fa8f9914640 1 -- 172.21.15.172:0/2740313020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f41977f0 msgr2=0x7fa8f419bb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:13:55.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.729+0000 7fa8f9914640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f41977f0 0x7fa8f419bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:55.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.729+0000 7fa8f9914640 1 -- 172.21.15.172:0/2740313020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f4197e10 con 0x7fa8e0003530 2024-09-11T22:13:55.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.729+0000 7fa8f9914640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0003530 0x7fa8f41972b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0005f30 tx=0x7fa8f0033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:55.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.729+0000 7fa8e6ffd640 1 -- 172.21.15.172:0/2740313020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa8f0038040 con 0x7fa8e0003530 2024-09-11T22:13:55.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.730+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2740313020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f419c0c0 con 0x7fa8e0003530 2024-09-11T22:13:55.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.730+0000 7fa8e6ffd640 1 -- 172.21.15.172:0/2740313020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f0032e00 con 0x7fa8e0003530 2024-09-11T22:13:55.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.730+0000 7fa8e6ffd640 1 -- 172.21.15.172:0/2740313020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa8f0004a00 con 0x7fa8e0003530 2024-09-11T22:13:55.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.730+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2740313020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f419c680 con 0x7fa8e0003530 2024-09-11T22:13:55.733 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.731+0000 7fa8e6ffd640 1 -- 172.21.15.172:0/2740313020 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fa8f0043020 con 0x7fa8e0003530 2024-09-11T22:13:55.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.731+0000 7fa8e6ffd640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa8d8038e00 0x7fa8d803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:55.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.731+0000 7fa8e6ffd640 1 -- 172.21.15.172:0/2740313020 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa8f006e6d0 con 0x7fa8e0003530 2024-09-11T22:13:55.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.731+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2740313020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f4101cc0 con 0x7fa8e0003530 2024-09-11T22:13:55.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.731+0000 7fa8f9113640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa8d8038e00 0x7fa8d803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:55.734 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.732+0000 7fa8f9113640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa8d8038e00 0x7fa8d803b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009100 tx=0x7fa8e8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:55.737 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.737+0000 7fa8e6ffd640 1 -- 172.21.15.172:0/2740313020 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8f004a020 con 0x7fa8e0003530 2024-09-11T22:13:55.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.968+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2740313020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa8f4198610 con 0x7fa8e0003530 2024-09-11T22:13:55.969 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.968+0000 7fa8e6ffd640 1 -- 172.21.15.172:0/2740313020 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fa8f004cbc0 con 0x7fa8e0003530 2024-09-11T22:13:55.969 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:13:55.969 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":2,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","modified":"2024-09-11T22:13:49.991644Z","created":"2024-09-11T22:12:20.315558Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T22:13:55.970 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 2 2024-09-11T22:13:55.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.970+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa8d8038e00 msgr2=0x7fa8d803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:55.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.970+0000 7fa8fbb78640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa8d8038e00 0x7fa8d803b2c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009100 tx=0x7fa8e8008040 comp rx=0 tx=0).stop 2024-09-11T22:13:55.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.970+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0003530 msgr2=0x7fa8f41972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:55.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.970+0000 7fa8fbb78640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0003530 0x7fa8f41972b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0005f30 tx=0x7fa8f0033680 comp rx=0 tx=0).stop 2024-09-11T22:13:55.971 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.970+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2740313020 shutdown_connections 2024-09-11T22:13:55.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.970+0000 7fa8fbb78640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa8d8038e00 0x7fa8d803b2c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:55.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.970+0000 7fa8fbb78640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8f41977f0 0x7fa8f419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:55.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.970+0000 7fa8fbb78640 1 --2- 172.21.15.172:0/2740313020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0003530 0x7fa8f41972b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:55.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.970+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2740313020 >> 172.21.15.172:0/2740313020 conn(0x7fa8f40fc280 msgr2=0x7fa8f406a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:55.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.971+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2740313020 shutdown_connections 2024-09-11T22:13:55.972 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:55.971+0000 7fa8fbb78640 1 -- 172.21.15.172:0/2740313020 wait complete. 2024-09-11T22:13:56.320 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:56 smithi172 ceph-mon[35461]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:56.320 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:56 smithi172 ceph-mon[35461]: Deploying daemon mon.b on smithi159 2024-09-11T22:13:56.320 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:56 smithi172 ceph-mon[35461]: from='client.? 172.21.15.172:0/2740313020' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:13:56.321 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:56 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:56.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:56 smithi005 ceph-mon[30943]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:56.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:56 smithi005 ceph-mon[30943]: Deploying daemon mon.b on smithi159 2024-09-11T22:13:56.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:56 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/2740313020' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:13:56.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:56 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:13:57.607 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:13:57.608 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:13:57.862 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:13:58.298 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:13:58 smithi172 ceph-mon[35461]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:58.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.303+0000 7f757a212640 1 -- 172.21.15.172:0/2421471549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7574106650 msgr2=0x7f7574106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:58.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.303+0000 7f757a212640 1 --2- 172.21.15.172:0/2421471549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7574106650 0x7f7574106a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7564009880 tx=0x7f756402a5d0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.303+0000 7f757a212640 1 -- 172.21.15.172:0/2421471549 shutdown_connections 2024-09-11T22:13:58.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.303+0000 7f757a212640 1 --2- 172.21.15.172:0/2421471549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7574106650 0x7f7574106a30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.303+0000 7f757a212640 1 -- 172.21.15.172:0/2421471549 >> 172.21.15.172:0/2421471549 conn(0x7f75740fc280 msgr2=0x7f75740fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:58.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.303+0000 7f757a212640 1 -- 172.21.15.172:0/2421471549 shutdown_connections 2024-09-11T22:13:58.304 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.303+0000 7f757a212640 1 -- 172.21.15.172:0/2421471549 wait complete. 2024-09-11T22:13:58.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.304+0000 7f757a212640 1 Processor -- start 2024-09-11T22:13:58.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.304+0000 7f757a212640 1 -- start start 2024-09-11T22:13:58.306 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.304+0000 7f757a212640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574106650 0x7f75741962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.304+0000 7f757a212640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7574196810 0x7f757419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.304+0000 7f757a212640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f757410ac80 con 0x7f7574196810 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.304+0000 7f757a212640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f757410ab00 con 0x7f7574106650 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f75737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574106650 0x7f75741962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f75737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574106650 0x7f75741962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44722/0 (socket says 172.21.15.172:44722) 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f75737fe640 1 -- 172.21.15.172:0/3128970757 learned_addr learned my addr 172.21.15.172:0/3128970757 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f7572ffd640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7574196810 0x7f757419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f75737fe640 1 -- 172.21.15.172:0/3128970757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574106650 msgr2=0x7f75741962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f75737fe640 1 -- 172.21.15.172:0/3128970757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574106650 msgr2=0x7f75741962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f75737fe640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574106650 0x7f75741962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f75737fe640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574106650 0x7f75741962d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f7572ffd640 1 -- 172.21.15.172:0/3128970757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574106650 msgr2=0x7f75741962d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:13:58.307 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f7572ffd640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574106650 0x7f75741962d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f7572ffd640 1 -- 172.21.15.172:0/3128970757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7574196e30 con 0x7f7574196810 2024-09-11T22:13:58.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f7572ffd640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7574196810 0x7f757419aba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f756800b3d0 tx=0x7f756800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:58.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f7570ff9640 1 -- 172.21.15.172:0/3128970757 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7568011630 con 0x7f7574196810 2024-09-11T22:13:58.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f757a212640 1 -- 172.21.15.172:0/3128970757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757419b0e0 con 0x7f7574196810 2024-09-11T22:13:58.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f7570ff9640 1 -- 172.21.15.172:0/3128970757 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7568011cb0 con 0x7f7574196810 2024-09-11T22:13:58.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f7570ff9640 1 -- 172.21.15.172:0/3128970757 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f7568010df0 con 0x7f7574196810 2024-09-11T22:13:58.308 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.305+0000 7f757a212640 1 -- 172.21.15.172:0/3128970757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757419b6a0 con 0x7f7574196810 2024-09-11T22:13:58.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.306+0000 7f7570ff9640 1 -- 172.21.15.172:0/3128970757 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f7568018020 con 0x7f7574196810 2024-09-11T22:13:58.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.306+0000 7f757a212640 1 -- 172.21.15.172:0/3128970757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f757419b9b0 con 0x7f7574196810 2024-09-11T22:13:58.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.307+0000 7f7570ff9640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f755c038e00 0x7f755c03b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:13:58.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.307+0000 7f7570ff9640 1 -- 172.21.15.172:0/3128970757 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f756804bcc0 con 0x7f7574196810 2024-09-11T22:13:58.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.307+0000 7f75737fe640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f755c038e00 0x7f755c03b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:13:58.309 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.307+0000 7f75737fe640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f755c038e00 0x7f755c03b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f75640096c0 tx=0x7f7564005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:13:58.313 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.312+0000 7f7570ff9640 1 -- 172.21.15.172:0/3128970757 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7568052080 con 0x7f7574196810 2024-09-11T22:13:58.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:13:58 smithi005 ceph-mon[30943]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:13:58.540 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.539+0000 7f757a212640 1 -- 172.21.15.172:0/3128970757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7574197630 con 0x7f7574196810 2024-09-11T22:13:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.540+0000 7f7570ff9640 1 -- 172.21.15.172:0/3128970757 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f756801f020 con 0x7f7574196810 2024-09-11T22:13:58.541 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:13:58.541 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":2,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","modified":"2024-09-11T22:13:49.991644Z","created":"2024-09-11T22:12:20.315558Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T22:13:58.541 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 2 2024-09-11T22:13:58.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 -- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f755c038e00 msgr2=0x7f755c03b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:58.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f755c038e00 0x7f755c03b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f75640096c0 tx=0x7f7564005a40 comp rx=0 tx=0).stop 2024-09-11T22:13:58.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 -- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7574196810 msgr2=0x7f757419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:13:58.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7574196810 0x7f757419aba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f756800b3d0 tx=0x7f756800b890 comp rx=0 tx=0).stop 2024-09-11T22:13:58.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 -- 172.21.15.172:0/3128970757 shutdown_connections 2024-09-11T22:13:58.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f755c038e00 0x7f755c03b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7574196810 0x7f757419aba0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 --2- 172.21.15.172:0/3128970757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574106650 0x7f75741962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:13:58.543 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 -- 172.21.15.172:0/3128970757 >> 172.21.15.172:0/3128970757 conn(0x7f75740fc280 msgr2=0x7f75740fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:13:58.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 -- 172.21.15.172:0/3128970757 shutdown_connections 2024-09-11T22:13:58.544 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:13:58.542+0000 7f757a212640 1 -- 172.21.15.172:0/3128970757 wait complete. 2024-09-11T22:13:59.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:13:58 smithi159 systemd[1]: Started Ceph mon.b for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:14:00.112 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T22:14:00.113 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T22:14:00.363 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:14:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.992+0000 7f1ca81ed640 1 -- 172.21.15.172:0/3434716299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c8c003530 msgr2=0x7f1c8c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.992+0000 7f1ca81ed640 1 --2- 172.21.15.172:0/3434716299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c8c003530 0x7f1c8c0059c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1ca01016c0 tx=0x7f1c9c02c150 comp rx=0 tx=0).stop 2024-09-11T22:14:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.992+0000 7f1ca81ed640 1 -- 172.21.15.172:0/3434716299 shutdown_connections 2024-09-11T22:14:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.992+0000 7f1ca81ed640 1 --2- 172.21.15.172:0/3434716299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c8c003530 0x7f1c8c0059c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.994 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.992+0000 7f1ca81ed640 1 -- 172.21.15.172:0/3434716299 >> 172.21.15.172:0/3434716299 conn(0x7f1ca00fc280 msgr2=0x7f1ca00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.992+0000 7f1ca81ed640 1 -- 172.21.15.172:0/3434716299 shutdown_connections 2024-09-11T22:14:03.995 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.992+0000 7f1ca81ed640 1 -- 172.21.15.172:0/3434716299 wait complete. 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.993+0000 7f1ca81ed640 1 Processor -- start 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.993+0000 7f1ca81ed640 1 -- start start 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.993+0000 7f1ca81ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c8c003530 0x7f1ca0102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.993+0000 7f1ca81ed640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0102750 0x7f1ca0102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.993+0000 7f1ca81ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca01a3510 0x7f1ca01a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca81ed640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca0108690 con 0x7f1ca01a3510 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca81ed640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca0108510 con 0x7f1ca0102750 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca81ed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca0108810 con 0x7f1c8c003530 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5f89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c8c003530 0x7f1ca0102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5f89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c8c003530 0x7f1ca0102210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:44728/0 (socket says 172.21.15.172:44728) 2024-09-11T22:14:03.997 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5f89640 1 -- 172.21.15.172:0/1427522421 learned_addr learned my addr 172.21.15.172:0/1427522421 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5f89640 1 -- 172.21.15.172:0/1427522421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c8c003530 msgr2=0x7f1ca0102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5f89640 1 -- 172.21.15.172:0/1427522421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c8c003530 msgr2=0x7f1ca0102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5f89640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c8c003530 0x7f1ca0102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5788640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0102750 0x7f1ca0102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca678a640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca01a3510 0x7f1ca01a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5f89640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c8c003530 0x7f1ca0102210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5788640 1 -- 172.21.15.172:0/1427522421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0102750 msgr2=0x7f1ca0102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5788640 1 -- 172.21.15.172:0/1427522421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0102750 msgr2=0x7f1ca0102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5788640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0102750 0x7f1ca0102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca5788640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0102750 0x7f1ca0102bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca678a640 1 -- 172.21.15.172:0/1427522421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c8c003530 msgr2=0x7f1ca0102210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca678a640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c8c003530 0x7f1ca0102210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.998 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca678a640 1 -- 172.21.15.172:0/1427522421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0102750 msgr2=0x7f1ca0102bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:14:03.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca678a640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0102750 0x7f1ca0102bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:03.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.994+0000 7f1ca678a640 1 -- 172.21.15.172:0/1427522421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca01a5fd0 con 0x7f1ca01a3510 2024-09-11T22:14:03.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.995+0000 7f1ca678a640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca01a3510 0x7f1ca01a5900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800bd50 tx=0x7f1c9800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:03.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.995+0000 7f1c96ffd640 1 -- 172.21.15.172:0/1427522421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98019070 con 0x7f1ca01a3510 2024-09-11T22:14:03.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.995+0000 7f1c96ffd640 1 -- 172.21.15.172:0/1427522421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c98014920 con 0x7f1ca01a3510 2024-09-11T22:14:03.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.995+0000 7f1c96ffd640 1 -- 172.21.15.172:0/1427522421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c98012990 con 0x7f1ca01a3510 2024-09-11T22:14:03.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.995+0000 7f1ca81ed640 1 -- 172.21.15.172:0/1427522421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca01a62c0 con 0x7f1ca01a3510 2024-09-11T22:14:03.999 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.995+0000 7f1ca81ed640 1 -- 172.21.15.172:0/1427522421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca01a67b0 con 0x7f1ca01a3510 2024-09-11T22:14:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.996+0000 7f1c96ffd640 1 -- 172.21.15.172:0/1427522421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f1c9801c070 con 0x7f1ca01a3510 2024-09-11T22:14:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.996+0000 7f1c96ffd640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1c840394f0 0x7f1c8403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.996+0000 7f1ca81ed640 1 -- 172.21.15.172:0/1427522421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca01097c0 con 0x7f1ca01a3510 2024-09-11T22:14:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.996+0000 7f1c96ffd640 1 -- 172.21.15.172:0/1427522421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1c9804bee0 con 0x7f1ca01a3510 2024-09-11T22:14:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.997+0000 7f1ca5f89640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1c840394f0 0x7f1c8403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:04.000 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:03.997+0000 7f1ca5f89640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1c840394f0 0x7f1c8403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c005fd0 tx=0x7f1c9c005d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:04.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.002+0000 7f1c96ffd640 1 -- 172.21.15.172:0/1427522421 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c98013c30 con 0x7f1ca01a3510 2024-09-11T22:14:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.245+0000 7f1ca81ed640 1 -- 172.21.15.172:0/1427522421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1ca01a6b10 con 0x7f1ca01a3510 2024-09-11T22:14:04.247 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.246+0000 7f1c96ffd640 1 -- 172.21.15.172:0/1427522421 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f1c98027090 con 0x7f1ca01a3510 2024-09-11T22:14:04.248 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:04.248 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:14:04.248 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: mon.c calling monitor election 2024-09-11T22:14:04.248 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: mon.a calling monitor election 2024-09-11T22:14:04.248 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.248 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:14:04.248 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.248 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:04.248 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: monmap epoch 3 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: last_changed 2024-09-11T22:13:58.895295+0000 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: created 2024-09-11T22:12:20.315558+0000 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: min_mon_release 19 (squid) 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: election_strategy: 1 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-11T22:14:04.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-11T22:14:04.250 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-09-11T22:14:04.250 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: fsmap 2024-09-11T22:14:04.250 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:14:04.250 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: mgrmap e13: x(active, since 47s) 2024-09-11T22:14:04.250 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: overall HEALTH_OK 2024-09-11T22:14:04.250 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:04.250 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:04.250 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:04.250 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:04.250 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:14:04.251 INFO:teuthology.orchestra.run.smithi172.stdout:{"epoch":3,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","modified":"2024-09-11T22:13:58.895295Z","created":"2024-09-11T22:12:20.315558Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:3300","nonce":0},{"type":"v1","addr":"172.21.15.172:6789","nonce":0}]},"addr":"172.21.15.172:6789/0","public_addr":"172.21.15.172:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T22:14:04.251 INFO:teuthology.orchestra.run.smithi172.stderr:dumped monmap epoch 3 2024-09-11T22:14:04.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 -- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1c840394f0 msgr2=0x7f1c8403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:04.251 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1c840394f0 0x7f1c8403b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c005fd0 tx=0x7f1c9c005d50 comp rx=0 tx=0).stop 2024-09-11T22:14:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 -- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca01a3510 msgr2=0x7f1ca01a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca01a3510 0x7f1ca01a5900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1c9800bd50 tx=0x7f1c9800be40 comp rx=0 tx=0).stop 2024-09-11T22:14:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 -- 172.21.15.172:0/1427522421 shutdown_connections 2024-09-11T22:14:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1c840394f0 0x7f1c8403b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca01a3510 0x7f1ca01a5900 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0102750 0x7f1ca0102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 --2- 172.21.15.172:0/1427522421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c8c003530 0x7f1ca0102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 -- 172.21.15.172:0/1427522421 >> 172.21.15.172:0/1427522421 conn(0x7f1ca00fc280 msgr2=0x7f1ca006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 -- 172.21.15.172:0/1427522421 shutdown_connections 2024-09-11T22:14:04.252 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:04.248+0000 7f1ca81ed640 1 -- 172.21.15.172:0/1427522421 wait complete. 2024-09-11T22:14:04.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:04.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:14:04.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: mon.c calling monitor election 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: mon.a calling monitor election 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T22:14:04.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: monmap epoch 3 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: last_changed 2024-09-11T22:13:58.895295+0000 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: created 2024-09-11T22:12:20.315558+0000 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: min_mon_release 19 (squid) 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: election_strategy: 1 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: fsmap 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: mgrmap e13: x(active, since 47s) 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: overall HEALTH_OK 2024-09-11T22:14:04.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:04.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:04.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:04.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:04.809 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T22:14:04.809 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T22:14:05.077 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:05.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:04 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:05.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:04 smithi172 ceph-mon[35461]: from='client.? 172.21.15.172:0/1427522421' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:14:05.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:04 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:05.198 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:04 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:05.198 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:04 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/1427522421' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:14:05.198 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:04 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.516+0000 7f1495a12640 1 -- 172.21.15.5:0/702249290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1490102750 msgr2=0x7f1490102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.516+0000 7f1495a12640 1 --2- 172.21.15.5:0/702249290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1490102750 0x7f1490102b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1480009880 tx=0x7f148002a670 comp rx=0 tx=0).stop 2024-09-11T22:14:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.516+0000 7f1495a12640 1 -- 172.21.15.5:0/702249290 shutdown_connections 2024-09-11T22:14:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.516+0000 7f1495a12640 1 --2- 172.21.15.5:0/702249290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1490102750 0x7f1490102b30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.516+0000 7f1495a12640 1 -- 172.21.15.5:0/702249290 >> 172.21.15.5:0/702249290 conn(0x7f14900fc1b0 msgr2=0x7f14900fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.516+0000 7f1495a12640 1 -- 172.21.15.5:0/702249290 shutdown_connections 2024-09-11T22:14:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.516+0000 7f1495a12640 1 -- 172.21.15.5:0/702249290 wait complete. 2024-09-11T22:14:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.517+0000 7f1495a12640 1 Processor -- start 2024-09-11T22:14:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.517+0000 7f1495a12640 1 -- start start 2024-09-11T22:14:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.517+0000 7f1495a12640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1490102750 0x7f1490197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.517+0000 7f1495a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1490197980 0x7f149019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.517+0000 7f1495a12640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f149019c250 0x7f149019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f1495a12640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f149010bcc0 con 0x7f1490197980 2024-09-11T22:14:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f1495a12640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f149010bb40 con 0x7f1490102750 2024-09-11T22:14:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f1495a12640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f149010be40 con 0x7f149019c250 2024-09-11T22:14:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1490197980 0x7f149019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1490197980 0x7f149019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47628/0 (socket says 172.21.15.5:47628) 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148e7fc640 1 -- 172.21.15.5:0/3741874452 learned_addr learned my addr 172.21.15.5:0/3741874452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148effd640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1490102750 0x7f1490197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148f7fe640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f149019c250 0x7f149019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148effd640 1 -- 172.21.15.5:0/3741874452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1490102750 msgr2=0x7f1490197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148effd640 1 -- 172.21.15.5:0/3741874452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1490102750 msgr2=0x7f1490197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148effd640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1490102750 0x7f1490197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148effd640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1490102750 0x7f1490197440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148e7fc640 1 -- 172.21.15.5:0/3741874452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f149019c250 msgr2=0x7f149019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148e7fc640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f149019c250 0x7f149019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148e7fc640 1 -- 172.21.15.5:0/3741874452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1490102750 msgr2=0x7f1490197440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148e7fc640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1490102750 0x7f1490197440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148e7fc640 1 -- 172.21.15.5:0/3741874452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14901a3fc0 con 0x7f1490197980 2024-09-11T22:14:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.518+0000 7f148e7fc640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1490197980 0x7f149019bd10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f148400b520 tx=0x7f148400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.519+0000 7f1494a10640 1 -- 172.21.15.5:0/3741874452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484004230 con 0x7f1490197980 2024-09-11T22:14:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.519+0000 7f1494a10640 1 -- 172.21.15.5:0/3741874452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14840043d0 con 0x7f1490197980 2024-09-11T22:14:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.519+0000 7f1495a12640 1 -- 172.21.15.5:0/3741874452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14901a41f0 con 0x7f1490197980 2024-09-11T22:14:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.519+0000 7f1494a10640 1 -- 172.21.15.5:0/3741874452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484003d60 con 0x7f1490197980 2024-09-11T22:14:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.519+0000 7f1495a12640 1 -- 172.21.15.5:0/3741874452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14901a4700 con 0x7f1490197980 2024-09-11T22:14:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.520+0000 7f1494a10640 1 -- 172.21.15.5:0/3741874452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f148400e070 con 0x7f1490197980 2024-09-11T22:14:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.520+0000 7f1494a10640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f14780394f0 0x7f147803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.520+0000 7f1494a10640 1 -- 172.21.15.5:0/3741874452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f148404ba80 con 0x7f1490197980 2024-09-11T22:14:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.520+0000 7f1495a12640 1 -- 172.21.15.5:0/3741874452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1490102bb0 con 0x7f1490197980 2024-09-11T22:14:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.520+0000 7f148effd640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f14780394f0 0x7f147803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.521+0000 7f148effd640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f14780394f0 0x7f147803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1480005ab0 tx=0x7f1480005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.525+0000 7f1494a10640 1 -- 172.21.15.5:0/3741874452 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1484014020 con 0x7f1490197980 2024-09-11T22:14:05.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.705+0000 7f1495a12640 1 -- 172.21.15.5:0/3741874452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f14901a4990 con 0x7f1490197980 2024-09-11T22:14:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.706+0000 7f1494a10640 1 -- 172.21.15.5:0/3741874452 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f148400ed70 con 0x7f1490197980 2024-09-11T22:14:05.707 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:14:05.707 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-09-11T22:14:05.707 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:14:05.707 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] 2024-09-11T22:14:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.708+0000 7f1495a12640 1 -- 172.21.15.5:0/3741874452 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f14780394f0 msgr2=0x7f147803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.708+0000 7f1495a12640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f14780394f0 0x7f147803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1480005ab0 tx=0x7f1480005a20 comp rx=0 tx=0).stop 2024-09-11T22:14:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.708+0000 7f1495a12640 1 -- 172.21.15.5:0/3741874452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1490197980 msgr2=0x7f149019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.708+0000 7f1495a12640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1490197980 0x7f149019bd10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f148400b520 tx=0x7f148400b9e0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.708+0000 7f1495a12640 1 -- 172.21.15.5:0/3741874452 shutdown_connections 2024-09-11T22:14:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.708+0000 7f1495a12640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f14780394f0 0x7f147803b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.708+0000 7f1495a12640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f149019c250 0x7f149019c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.708+0000 7f1495a12640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1490197980 0x7f149019bd10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.708+0000 7f1495a12640 1 --2- 172.21.15.5:0/3741874452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1490102750 0x7f1490197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.708+0000 7f1495a12640 1 -- 172.21.15.5:0/3741874452 >> 172.21.15.5:0/3741874452 conn(0x7f14900fc1b0 msgr2=0x7f1490105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.709+0000 7f1495a12640 1 -- 172.21.15.5:0/3741874452 shutdown_connections 2024-09-11T22:14:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:05.709+0000 7f1495a12640 1 -- 172.21.15.5:0/3741874452 wait complete. 2024-09-11T22:14:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:14:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: mon.c calling monitor election 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: mon.a calling monitor election 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: monmap epoch 3 2024-09-11T22:14:06.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: last_changed 2024-09-11T22:13:58.895295+0000 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: created 2024-09-11T22:12:20.315558+0000 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: min_mon_release 19 (squid) 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: election_strategy: 1 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: fsmap 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: mgrmap e13: x(active, since 47s) 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: overall HEALTH_OK 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:06.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='client.? 172.21.15.172:0/1427522421' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T22:14:06.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:06.463 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T22:14:06.463 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:14:06.463 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T22:14:06.491 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:14:06.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:14:06.558 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:14:06.559 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T22:14:06.587 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:14:06.587 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:14:06.652 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:14:06.653 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T22:14:06.680 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:14:06.680 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:14:06.746 INFO:tasks.cephadm:Adding mgr.x on smithi005 2024-09-11T22:14:06.746 INFO:tasks.cephadm:Adding mgr.y on smithi159 2024-09-11T22:14:06.746 INFO:tasks.cephadm:Adding mgr.z on smithi172 2024-09-11T22:14:06.746 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi005=x;smithi159=y;smithi172=z' 2024-09-11T22:14:06.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: mon.b calling monitor election 2024-09-11T22:14:06.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: mon.b calling monitor election 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: mon.c calling monitor election 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: mon.a calling monitor election 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: monmap epoch 3 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: last_changed 2024-09-11T22:13:58.895295+0000 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: created 2024-09-11T22:12:20.315558+0000 2024-09-11T22:14:06.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: min_mon_release 19 (squid) 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: election_strategy: 1 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: fsmap 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: mgrmap e13: x(active, since 49s) 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: overall HEALTH_OK 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:06.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.024 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:14:07.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: mon.b calling monitor election 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: mon.b calling monitor election 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: mon.c calling monitor election 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: mon.a calling monitor election 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: monmap epoch 3 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: last_changed 2024-09-11T22:13:58.895295+0000 2024-09-11T22:14:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: created 2024-09-11T22:12:20.315558+0000 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: min_mon_release 19 (squid) 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: election_strategy: 1 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: fsmap 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: mgrmap e13: x(active, since 49s) 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: overall HEALTH_OK 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:14:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.083 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.083 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.083 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.083 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.083 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:14:07.083 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:14:07.083 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:07.083 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: mon.b calling monitor election 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: mon.b calling monitor election 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: mon.c calling monitor election 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: mon.a calling monitor election 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: monmap epoch 3 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:14:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: last_changed 2024-09-11T22:13:58.895295+0000 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: created 2024-09-11T22:12:20.315558+0000 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: min_mon_release 19 (squid) 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: election_strategy: 1 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: 1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.c 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: 2: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.b 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: fsmap 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: mgrmap e13: x(active, since 49s) 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: overall HEALTH_OK 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:14:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.114 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:14:07.114 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:14:07.114 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:07.114 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:07.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.562+0000 7f4e128bf640 1 -- 172.21.15.172:0/1970558001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e0c1021e0 msgr2=0x7f4e0c1045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:07.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.562+0000 7f4e128bf640 1 --2- 172.21.15.172:0/1970558001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e0c1021e0 0x7f4e0c1045d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009880 tx=0x7f4dfc02a5d0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.563+0000 7f4e128bf640 1 -- 172.21.15.172:0/1970558001 shutdown_connections 2024-09-11T22:14:07.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.563+0000 7f4e128bf640 1 --2- 172.21.15.172:0/1970558001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e0c1021e0 0x7f4e0c1045d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.586 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.563+0000 7f4e128bf640 1 -- 172.21.15.172:0/1970558001 >> 172.21.15.172:0/1970558001 conn(0x7f4e0c0fbda0 msgr2=0x7f4e0c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.563+0000 7f4e128bf640 1 -- 172.21.15.172:0/1970558001 shutdown_connections 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.563+0000 7f4e128bf640 1 -- 172.21.15.172:0/1970558001 wait complete. 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.563+0000 7f4e128bf640 1 Processor -- start 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.563+0000 7f4e128bf640 1 -- start start 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.563+0000 7f4e128bf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0c1021e0 0x7f4e0c196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.563+0000 7f4e128bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e0c1974a0 0x7f4e0c19c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e128bf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e0c197920 0x7f4e0c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e128bf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c1077a0 con 0x7f4e0c1974a0 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e128bf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c107620 con 0x7f4e0c197920 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e128bf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c107920 con 0x7f4e0c1021e0 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0c1021e0 0x7f4e0c196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e0bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0c1021e0 0x7f4e0c196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52140/0 (socket says 172.21.15.172:52140) 2024-09-11T22:14:07.587 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e0bfff640 1 -- 172.21.15.172:0/3205714170 learned_addr learned my addr 172.21.15.172:0/3205714170 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e0b7fe640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e0c1974a0 0x7f4e0c19c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e10e5c640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e0c197920 0x7f4e0c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e0bfff640 1 -- 172.21.15.172:0/3205714170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e0c197920 msgr2=0x7f4e0c19eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e0bfff640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e0c197920 0x7f4e0c19eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e0bfff640 1 -- 172.21.15.172:0/3205714170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e0c1974a0 msgr2=0x7f4e0c19c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e0bfff640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e0c1974a0 0x7f4e0c19c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e0bfff640 1 -- 172.21.15.172:0/3205714170 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c19f360 con 0x7f4e0c1021e0 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e10e5c640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e0c197920 0x7f4e0c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e0bfff640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0c1021e0 0x7f4e0c196f60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc000c00 tx=0x7f4dfc004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e097fa640 1 -- 172.21.15.172:0/3205714170 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc03f030 con 0x7f4e0c1021e0 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e097fa640 1 -- 172.21.15.172:0/3205714170 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dfc003ce0 con 0x7f4e0c1021e0 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e128bf640 1 -- 172.21.15.172:0/3205714170 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0c19f5f0 con 0x7f4e0c1021e0 2024-09-11T22:14:07.588 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e097fa640 1 -- 172.21.15.172:0/3205714170 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc035070 con 0x7f4e0c1021e0 2024-09-11T22:14:07.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.564+0000 7f4e128bf640 1 -- 172.21.15.172:0/3205714170 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0c19fba0 con 0x7f4e0c1021e0 2024-09-11T22:14:07.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.565+0000 7f4e097fa640 1 -- 172.21.15.172:0/3205714170 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f4dfc042030 con 0x7f4e0c1021e0 2024-09-11T22:14:07.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.565+0000 7f4e097fa640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4df40394f0 0x7f4df403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:07.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.565+0000 7f4e097fa640 1 -- 172.21.15.172:0/3205714170 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4dfc06c9d0 con 0x7f4e0c1021e0 2024-09-11T22:14:07.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.565+0000 7f4e128bf640 1 -- 172.21.15.172:0/3205714170 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0c190560 con 0x7f4e0c1021e0 2024-09-11T22:14:07.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.566+0000 7f4e0b7fe640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4df40394f0 0x7f4df403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:07.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.566+0000 7f4e0b7fe640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4df40394f0 0x7f4df403b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c198480 tx=0x7f4e00009190 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:07.589 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.570+0000 7f4e097fa640 1 -- 172.21.15.172:0/3205714170 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4dfc04c350 con 0x7f4e0c1021e0 2024-09-11T22:14:07.750 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.748+0000 7f4e128bf640 1 -- 172.21.15.172:0/3205714170 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi159=y;smithi172=z", "target": ["mon-mgr", ""]}) -- 0x7f4e0c0626c0 con 0x7f4df40394f0 2024-09-11T22:14:07.750 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:07 smithi172 ceph-mon[35461]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T22:14:07.750 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:07 smithi172 ceph-mon[35461]: Reconfiguring daemon mon.a on smithi005 2024-09-11T22:14:07.750 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:07 smithi172 ceph-mon[35461]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:07.750 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:07.755 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.753+0000 7f4e097fa640 1 -- 172.21.15.172:0/3205714170 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4e0c0626c0 con 0x7f4df40394f0 2024-09-11T22:14:07.755 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mgr update... 2024-09-11T22:14:07.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.755+0000 7f4e128bf640 1 -- 172.21.15.172:0/3205714170 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4df40394f0 msgr2=0x7f4df403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:07.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.755+0000 7f4e128bf640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4df40394f0 0x7f4df403b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c198480 tx=0x7f4e00009190 comp rx=0 tx=0).stop 2024-09-11T22:14:07.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.755+0000 7f4e128bf640 1 -- 172.21.15.172:0/3205714170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0c1021e0 msgr2=0x7f4e0c196f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:07.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.755+0000 7f4e128bf640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0c1021e0 0x7f4e0c196f60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc000c00 tx=0x7f4dfc004430 comp rx=0 tx=0).stop 2024-09-11T22:14:07.757 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.755+0000 7f4e128bf640 1 -- 172.21.15.172:0/3205714170 shutdown_connections 2024-09-11T22:14:07.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.755+0000 7f4e128bf640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4df40394f0 0x7f4df403b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.755+0000 7f4e128bf640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e0c197920 0x7f4e0c19eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.755+0000 7f4e128bf640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e0c1974a0 0x7f4e0c19c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.755+0000 7f4e128bf640 1 --2- 172.21.15.172:0/3205714170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0c1021e0 0x7f4e0c196f60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:07.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.755+0000 7f4e128bf640 1 -- 172.21.15.172:0/3205714170 >> 172.21.15.172:0/3205714170 conn(0x7f4e0c0fbda0 msgr2=0x7f4e0c0fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:07.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.756+0000 7f4e128bf640 1 -- 172.21.15.172:0/3205714170 shutdown_connections 2024-09-11T22:14:07.758 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:14:07.756+0000 7f4e128bf640 1 -- 172.21.15.172:0/3205714170 wait complete. 2024-09-11T22:14:08.051 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:07 smithi159 ceph-mon[35534]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T22:14:08.051 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:07 smithi159 ceph-mon[35534]: Reconfiguring daemon mon.a on smithi005 2024-09-11T22:14:08.051 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:07 smithi159 ceph-mon[35534]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:08.051 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:07 smithi005 ceph-mon[30943]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T22:14:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:07 smithi005 ceph-mon[30943]: Reconfiguring daemon mon.a on smithi005 2024-09-11T22:14:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:07 smithi005 ceph-mon[30943]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:14:08.354 DEBUG:teuthology.orchestra.run.smithi159:mgr.y> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T22:14:08.357 DEBUG:teuthology.orchestra.run.smithi172:mgr.z> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.z.service 2024-09-11T22:14:08.359 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T22:14:08.359 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:14:08.359 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T22:14:08.380 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T22:14:08.380 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-09-11T22:14:08.440 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T22:14:08.440 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:08.440 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-11T22:14:08.440 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:08.440 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:08.440 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-11 22:13:25.424301286 +0000 2024-09-11T22:14:08.440 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-11 22:08:40.309748815 +0000 2024-09-11T22:14:08.440 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-11 22:08:40.309748815 +0000 2024-09-11T22:14:08.440 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-11 22:08:40.309748815 +0000 2024-09-11T22:14:08.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-11T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-11T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000136878 s, 3.7 MB/s 2024-09-11T22:14:08.513 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T22:14:08.573 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-09-11T22:14:08.631 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T22:14:08.632 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:08.632 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-11T22:14:08.632 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:08.632 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:08.632 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-11 22:13:25.456300605 +0000 2024-09-11T22:14:08.632 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-11 22:08:40.626742076 +0000 2024-09-11T22:14:08.633 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-11 22:08:40.626742076 +0000 2024-09-11T22:14:08.633 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-11 22:08:40.626742076 +0000 2024-09-11T22:14:08.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T22:14:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-11T22:14:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-11T22:14:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000121126 s, 4.2 MB/s 2024-09-11T22:14:08.700 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T22:14:08.759 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-09-11T22:14:08.817 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T22:14:08.817 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:08.817 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-11T22:14:08.817 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:08.817 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:08.817 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-11 22:13:25.490299882 +0000 2024-09-11T22:14:08.817 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-11 22:08:40.956735060 +0000 2024-09-11T22:14:08.817 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-11 22:08:40.956735060 +0000 2024-09-11T22:14:08.818 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-11 22:08:40.956735060 +0000 2024-09-11T22:14:08.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T22:14:08.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:08.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:08.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:08 smithi005 ceph-mon[30943]: Reconfiguring mon.b (monmap changed)... 2024-09-11T22:14:08.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:14:08.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:14:08.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:08.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:08 smithi005 ceph-mon[30943]: Reconfiguring daemon mon.b on smithi159 2024-09-11T22:14:08.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:08 smithi005 ceph-mon[30943]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi159=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:08.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:08 smithi005 ceph-mon[30943]: Saving service mgr spec with placement smithi005=x;smithi159=y;smithi172=z;count:3 2024-09-11T22:14:08.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:08.846 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-11T22:14:08.846 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-11T22:14:08.846 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000116423 s, 4.4 MB/s 2024-09-11T22:14:08.847 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T22:14:08.905 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-09-11T22:14:08.962 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T22:14:08.962 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:08.962 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-11T22:14:08.962 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:08.962 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:08.963 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-11 22:13:25.519299265 +0000 2024-09-11T22:14:08.963 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-11 22:08:41.280728172 +0000 2024-09-11T22:14:08.963 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-11 22:08:41.280728172 +0000 2024-09-11T22:14:08.963 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-11 22:08:41.280728172 +0000 2024-09-11T22:14:08.964 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T22:14:08.984 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:08.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:08.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:08 smithi159 ceph-mon[35534]: Reconfiguring mon.b (monmap changed)... 2024-09-11T22:14:08.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:14:08.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:14:08.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:08.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:08 smithi159 ceph-mon[35534]: Reconfiguring daemon mon.b on smithi159 2024-09-11T22:14:08.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:08 smithi159 ceph-mon[35534]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi159=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:08.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:08 smithi159 ceph-mon[35534]: Saving service mgr spec with placement smithi005=x;smithi159=y;smithi172=z;count:3 2024-09-11T22:14:08.985 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-11T22:14:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-11T22:14:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.00011312 s, 4.5 MB/s 2024-09-11T22:14:09.029 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T22:14:09.086 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-11T22:14:09.086 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T22:14:09.103 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T22:14:09.103 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2024-09-11T22:14:09.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:09.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:09.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:08 smithi172 ceph-mon[35461]: Reconfiguring mon.b (monmap changed)... 2024-09-11T22:14:09.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:14:09.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:14:09.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:09.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:08 smithi172 ceph-mon[35461]: Reconfiguring daemon mon.b on smithi159 2024-09-11T22:14:09.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:08 smithi172 ceph-mon[35461]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi159=y;smithi172=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:09.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:08 smithi172 ceph-mon[35461]: Saving service mgr spec with placement smithi005=x;smithi159=y;smithi172=z;count:3 2024-09-11T22:14:09.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:09.158 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T22:14:09.158 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:09.159 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-11T22:14:09.159 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:09.159 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:09.159 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-11 22:13:58.269464105 +0000 2024-09-11T22:14:09.159 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-11 22:08:52.739221056 +0000 2024-09-11T22:14:09.159 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-11 22:08:52.739221056 +0000 2024-09-11T22:14:09.159 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-11 22:08:52.739221056 +0000 2024-09-11T22:14:09.159 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T22:14:09.223 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-11T22:14:09.223 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-11T22:14:09.223 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000288554 s, 1.8 MB/s 2024-09-11T22:14:09.225 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T22:14:09.282 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2024-09-11T22:14:09.337 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T22:14:09.338 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:09.338 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-11T22:14:09.338 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:09.338 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:09.338 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-11 22:13:58.269464105 +0000 2024-09-11T22:14:09.338 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-11 22:08:53.072214795 +0000 2024-09-11T22:14:09.338 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-11 22:08:53.072214795 +0000 2024-09-11T22:14:09.338 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-11 22:08:53.072214795 +0000 2024-09-11T22:14:09.339 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T22:14:09.401 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-11T22:14:09.401 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-11T22:14:09.401 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000192838 s, 2.7 MB/s 2024-09-11T22:14:09.403 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T22:14:09.459 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2024-09-11T22:14:09.515 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T22:14:09.515 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:09.515 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-11T22:14:09.515 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:09.515 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:09.516 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-11 22:13:58.269464105 +0000 2024-09-11T22:14:09.516 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-11 22:08:53.394208742 +0000 2024-09-11T22:14:09.516 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-11 22:08:53.394208742 +0000 2024-09-11T22:14:09.516 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-11 22:08:53.394208742 +0000 2024-09-11T22:14:09.516 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T22:14:09.578 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-11T22:14:09.578 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-11T22:14:09.578 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000172608 s, 3.0 MB/s 2024-09-11T22:14:09.579 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T22:14:09.635 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2024-09-11T22:14:09.690 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T22:14:09.691 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:09.691 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-11T22:14:09.691 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:09.692 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:09.692 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-11 22:13:58.270464086 +0000 2024-09-11T22:14:09.692 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-11 22:08:53.714202726 +0000 2024-09-11T22:14:09.692 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-11 22:08:53.714202726 +0000 2024-09-11T22:14:09.692 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-11 22:08:53.714202726 +0000 2024-09-11T22:14:09.692 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T22:14:09.754 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-11T22:14:09.754 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-11T22:14:09.754 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000192944 s, 2.7 MB/s 2024-09-11T22:14:09.756 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T22:14:09.812 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-09-11T22:14:09.812 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T22:14:09.829 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T22:14:09.829 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-09-11T22:14:09.885 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T22:14:09.885 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:09.885 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-11T22:14:09.885 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:09.885 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:09.885 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-11 22:13:49.311273829 +0000 2024-09-11T22:14:09.886 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-11 22:08:49.775014730 +0000 2024-09-11T22:14:09.886 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-11 22:08:49.775014730 +0000 2024-09-11T22:14:09.886 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-11 22:08:49.775014730 +0000 2024-09-11T22:14:09.886 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T22:14:09.950 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-11T22:14:09.950 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-11T22:14:09.950 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000273445 s, 1.9 MB/s 2024-09-11T22:14:09.951 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T22:14:10.008 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-09-11T22:14:10.064 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T22:14:10.065 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:10.065 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-11T22:14:10.065 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:10.065 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:10.065 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-11 22:13:49.312273804 +0000 2024-09-11T22:14:10.065 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-11 22:08:50.102006324 +0000 2024-09-11T22:14:10.065 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-11 22:08:50.102006324 +0000 2024-09-11T22:14:10.065 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-11 22:08:50.102006324 +0000 2024-09-11T22:14:10.065 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T22:14:10.128 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-11T22:14:10.128 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-11T22:14:10.128 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000207433 s, 2.5 MB/s 2024-09-11T22:14:10.129 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T22:14:10.185 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-09-11T22:14:10.242 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T22:14:10.242 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:10.242 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-11T22:14:10.242 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:10.242 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:10.242 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-11 22:13:49.312273804 +0000 2024-09-11T22:14:10.242 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-11 22:08:50.423998046 +0000 2024-09-11T22:14:10.242 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-11 22:08:50.423998046 +0000 2024-09-11T22:14:10.242 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-11 22:08:50.423998046 +0000 2024-09-11T22:14:10.243 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T22:14:10.305 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:10 smithi172 ceph-mon[35461]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:10.305 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:10.305 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:10.305 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:10 smithi172 ceph-mon[35461]: Reconfiguring mon.c (monmap changed)... 2024-09-11T22:14:10.305 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:14:10.305 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:14:10.306 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:10.306 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:10 smithi172 ceph-mon[35461]: Reconfiguring daemon mon.c on smithi172 2024-09-11T22:14:10.306 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-11T22:14:10.307 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-11T22:14:10.307 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000199378 s, 2.6 MB/s 2024-09-11T22:14:10.308 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T22:14:10.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:10 smithi005 ceph-mon[30943]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:10.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:10.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:10.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:10 smithi005 ceph-mon[30943]: Reconfiguring mon.c (monmap changed)... 2024-09-11T22:14:10.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:14:10.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:14:10.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:10.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:10 smithi005 ceph-mon[30943]: Reconfiguring daemon mon.c on smithi172 2024-09-11T22:14:10.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:10 smithi159 ceph-mon[35534]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:10.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:10.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:10.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:10 smithi159 ceph-mon[35534]: Reconfiguring mon.c (monmap changed)... 2024-09-11T22:14:10.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T22:14:10.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T22:14:10.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:10.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:10 smithi159 ceph-mon[35534]: Reconfiguring daemon mon.c on smithi172 2024-09-11T22:14:10.364 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-09-11T22:14:10.419 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T22:14:10.419 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T22:14:10.419 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-11T22:14:10.420 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:14:10.420 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T22:14:10.420 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-09-11 22:13:49.312273804 +0000 2024-09-11T22:14:10.420 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-09-11 22:08:50.739989923 +0000 2024-09-11T22:14:10.420 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-09-11 22:08:50.739989923 +0000 2024-09-11T22:14:10.420 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-09-11 22:08:50.739989923 +0000 2024-09-11T22:14:10.420 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T22:14:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-09-11T22:14:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-09-11T22:14:10.483 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000193951 s, 2.6 MB/s 2024-09-11T22:14:10.485 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T22:14:10.541 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2024-09-11T22:14:10.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T22:14:10.801 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:11.589 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:11.590 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:11 smithi172 ceph-mon[35461]: Deploying daemon mgr.z on smithi172 2024-09-11T22:14:11.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:11.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:11.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:11.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:11.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:11.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:11.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:11.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:14:11.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:14:11.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:14:11.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:11.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:11 smithi005 ceph-mon[30943]: Deploying daemon mgr.z on smithi172 2024-09-11T22:14:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:14:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:14:11.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:14:11.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:11.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:11 smithi159 ceph-mon[35534]: Deploying daemon mgr.z on smithi172 2024-09-11T22:14:12.500 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:12 smithi172 systemd[1]: Starting Ceph mgr.z for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:14:12.802 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:12 smithi172 podman[37249]: 2024-09-11 22:14:12.500154226 +0000 UTC m=+0.106875394 container create 8053c59d603345bde68ef28c84c7d2eac27f0698a9cd23fb14dbc36810baac40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:14:12.803 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:12 smithi172 podman[37249]: 2024-09-11 22:14:12.408936833 +0000 UTC m=+0.015658002 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:14:12.803 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:12 smithi172 podman[37249]: 2024-09-11 22:14:12.710663619 +0000 UTC m=+0.317384787 container init 8053c59d603345bde68ef28c84c7d2eac27f0698a9cd23fb14dbc36810baac40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T22:14:12.803 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:12 smithi172 podman[37249]: 2024-09-11 22:14:12.714131921 +0000 UTC m=+0.320853095 container start 8053c59d603345bde68ef28c84c7d2eac27f0698a9cd23fb14dbc36810baac40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T22:14:12.803 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:12 smithi172 ceph-mgr[37268]: -- 172.21.15.172:0/107650719 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f33a89b0e0 con 0x55f33a879000 2024-09-11T22:14:12.803 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:12 smithi172 bash[37249]: 8053c59d603345bde68ef28c84c7d2eac27f0698a9cd23fb14dbc36810baac40 2024-09-11T22:14:13.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:12 smithi172 systemd[1]: Started Ceph mgr.z for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:14:13.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:12.835+0000 7fc427998040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:14:13.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:12.896+0000 7fc427998040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:14:13.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:13.044+0000 7fc427998040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:14:13.323 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:14:13.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2024-09-11T22:14:13.394 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:13.110+0000 7fc427998040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:14:13.394 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:13.256+0000 7fc427998040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:14:13.394 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:13.394+0000 7fc427998040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:14:13.603 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:14:14.026 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:13 smithi005 ceph-mon[30943]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:14.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:13 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:14.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:13 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:14.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:13 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:14.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:13 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:14:14.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:13 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:14:14.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:13 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:14:14.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:13 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:14.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:13 smithi005 ceph-mon[30943]: Deploying daemon mgr.y on smithi159 2024-09-11T22:14:14.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.025+0000 7fe5be7c0640 1 -- 172.21.15.5:0/4018529168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b81037c0 msgr2=0x7fe5b810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.025+0000 7fe5be7c0640 1 --2- 172.21.15.5:0/4018529168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b81037c0 0x7fe5b810a050 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009ca0 tx=0x7fe5b002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.025+0000 7fe5be7c0640 1 -- 172.21.15.5:0/4018529168 shutdown_connections 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.025+0000 7fe5be7c0640 1 --2- 172.21.15.5:0/4018529168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b81037c0 0x7fe5b810a050 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.025+0000 7fe5be7c0640 1 --2- 172.21.15.5:0/4018529168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b8102e00 0x7fe5b8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.025+0000 7fe5be7c0640 1 --2- 172.21.15.5:0/4018529168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5b8101c00 0x7fe5b8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.025+0000 7fe5be7c0640 1 -- 172.21.15.5:0/4018529168 >> 172.21.15.5:0/4018529168 conn(0x7fe5b80fd390 msgr2=0x7fe5b80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.025+0000 7fe5be7c0640 1 -- 172.21.15.5:0/4018529168 shutdown_connections 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.025+0000 7fe5be7c0640 1 -- 172.21.15.5:0/4018529168 wait complete. 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.026+0000 7fe5be7c0640 1 Processor -- start 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.026+0000 7fe5be7c0640 1 -- start start 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.026+0000 7fe5be7c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b8101c00 0x7fe5b8075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.026+0000 7fe5be7c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5b8102e00 0x7fe5b80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.026+0000 7fe5be7c0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b81037c0 0x7fe5b8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.026+0000 7fe5be7c0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b810d8d0 con 0x7fe5b8101c00 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.026+0000 7fe5be7c0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b810d750 con 0x7fe5b81037c0 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.026+0000 7fe5be7c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b810da50 con 0x7fe5b8102e00 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.026+0000 7fe5b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b8101c00 0x7fe5b8075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b8101c00 0x7fe5b8075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45230/0 (socket says 172.21.15.5:45230) 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b7fff640 1 -- 172.21.15.5:0/912938821 learned_addr learned my addr 172.21.15.5:0/912938821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b77fe640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5b8102e00 0x7fe5b80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5bcd5d640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b81037c0 0x7fe5b8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b7fff640 1 -- 172.21.15.5:0/912938821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5b8102e00 msgr2=0x7fe5b80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b7fff640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5b8102e00 0x7fe5b80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b7fff640 1 -- 172.21.15.5:0/912938821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b81037c0 msgr2=0x7fe5b8074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b7fff640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b81037c0 0x7fe5b8074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b7fff640 1 -- 172.21.15.5:0/912938821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b80753e0 con 0x7fe5b8101c00 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b7fff640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b8101c00 0x7fe5b8075f90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800d950 tx=0x7fe5a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b57fa640 1 -- 172.21.15.5:0/912938821 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8017070 con 0x7fe5b8101c00 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5be7c0640 1 -- 172.21.15.5:0/912938821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b81a1fc0 con 0x7fe5b8101c00 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b57fa640 1 -- 172.21.15.5:0/912938821 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a8009c30 con 0x7fe5b8101c00 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5b57fa640 1 -- 172.21.15.5:0/912938821 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800f600 con 0x7fe5b8101c00 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.027+0000 7fe5be7c0640 1 -- 172.21.15.5:0/912938821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b81a2420 con 0x7fe5b8101c00 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.028+0000 7fe5b57fa640 1 -- 172.21.15.5:0/912938821 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fe5a801a070 con 0x7fe5b8101c00 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.028+0000 7fe5be7c0640 1 -- 172.21.15.5:0/912938821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b8108f30 con 0x7fe5b8101c00 2024-09-11T22:14:14.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.028+0000 7fe5b57fa640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe5a0039290 0x7fe5a003b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.029+0000 7fe5b57fa640 1 -- 172.21.15.5:0/912938821 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe5a804c020 con 0x7fe5b8101c00 2024-09-11T22:14:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.029+0000 7fe5b77fe640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe5a0039290 0x7fe5a003b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.029+0000 7fe5b77fe640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe5a0039290 0x7fe5a003b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8075d10 tx=0x7fe5ac00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.034+0000 7fe5b57fa640 1 -- 172.21.15.5:0/912938821 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5a801bcd0 con 0x7fe5b8101c00 2024-09-11T22:14:14.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:13.707+0000 7fc427998040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:14:14.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:13.767+0000 7fc427998040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:14:14.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:13.951+0000 7fc427998040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:14:14.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-mon[35461]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:14.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:14.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:14.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:14.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:14:14.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:14:14.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:14:14.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:14.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:13 smithi172 ceph-mon[35461]: Deploying daemon mgr.y on smithi159 2024-09-11T22:14:14.181 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:13 smithi159 ceph-mon[35534]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:14.182 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:13 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:14.182 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:13 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:14.182 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:13 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:14.182 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:13 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:14:14.182 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:13 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T22:14:14.182 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:13 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:14:14.182 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:13 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:14.182 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:13 smithi159 ceph-mon[35534]: Deploying daemon mgr.y on smithi159 2024-09-11T22:14:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:14.217+0000 7fe5be7c0640 1 -- 172.21.15.5:0/912938821 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe5b8062630 con 0x7fe5a0039290 2024-09-11T22:14:14.818 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:14 smithi159 systemd[1]: Starting Ceph mgr.y for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:14:15.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:14 smithi005 ceph-mon[30943]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:15.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:14 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:14:15.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:14 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:14:15.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:14 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:15.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:14 smithi159 ceph-mon[35534]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:15.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:14 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:14:15.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:14 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:14:15.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:14 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:15.078 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:14 smithi159 podman[36884]: 2024-09-11 22:14:14.817415565 +0000 UTC m=+0.109936409 container create 51f3b0a7d5903c045a526fcacb1f47a216dadb6bb7a3405c19288e04f36d5e6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:14:15.079 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:14 smithi159 podman[36884]: 2024-09-11 22:14:14.723251438 +0000 UTC m=+0.015772290 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:14:15.079 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 podman[36884]: 2024-09-11 22:14:15.078266612 +0000 UTC m=+0.370787455 container init 51f3b0a7d5903c045a526fcacb1f47a216dadb6bb7a3405c19288e04f36d5e6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T22:14:15.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:14 smithi172 ceph-mon[35461]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:15.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:14 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:14:15.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:14 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:14:15.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:14 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:15.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:14.828+0000 7fc427998040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:14:15.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:14.895+0000 7fc427998040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:14:15.329 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 podman[36884]: 2024-09-11 22:14:15.081748358 +0000 UTC m=+0.374269215 container start 51f3b0a7d5903c045a526fcacb1f47a216dadb6bb7a3405c19288e04f36d5e6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-11T22:14:15.329 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 ceph-mgr[36903]: -- 172.21.15.159:0/3858471499 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e70c4d10e0 con 0x55e70c4ae800 2024-09-11T22:14:15.329 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 bash[36884]: 51f3b0a7d5903c045a526fcacb1f47a216dadb6bb7a3405c19288e04f36d5e6d 2024-09-11T22:14:15.329 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 systemd[1]: Started Ceph mgr.y for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:14:15.329 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:15.202+0000 7f3e28b4a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:14:15.329 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:15.263+0000 7f3e28b4a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:14:15.815 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:15.421+0000 7f3e28b4a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:14:15.815 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:15.489+0000 7f3e28b4a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:14:15.815 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:15.654+0000 7f3e28b4a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:14:15.816 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:15.815+0000 7f3e28b4a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:14:15.904 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:15 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:15.630+0000 7fc427998040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:14:15.904 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:15 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:15.760+0000 7fc427998040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:14:15.904 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:15 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:15.903+0000 7fc427998040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:14:16.155 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:15 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:15.967+0000 7fc427998040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:14:16.155 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:16.124+0000 7fc427998040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:14:16.399 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:16 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:16.140+0000 7f3e28b4a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:14:16.399 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:16 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:16.202+0000 7f3e28b4a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:14:16.399 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:16 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:16.399+0000 7f3e28b4a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:14:16.400 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:16 smithi159 ceph-mon[35534]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:16.400 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.400 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.400 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.400 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.400 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:16.423 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:16.194+0000 7fc427998040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:14:16.423 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:16.255+0000 7fc427998040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:14:16.423 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:16.421+0000 7fc427998040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:14:16.424 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-mon[35461]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:16.424 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.424 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.424 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.424 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.424 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:16 smithi005 ceph-mon[30943]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:16.700 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:16.489+0000 7fc427998040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:14:16.701 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:16.558+0000 7fc427998040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:14:16.701 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:16.641+0000 7fc427998040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:14:16.701 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:16.699+0000 7fc427998040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:14:17.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:16.758+0000 7fc427998040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:14:17.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:14:17.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:14:17.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:17.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: Reconfiguring daemon mgr.x on smithi005 2024-09-11T22:14:17.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1877775904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8ab7175-48c6-4c65-82eb-620196b29548"}]: dispatch 2024-09-11T22:14:17.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1877775904' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8ab7175-48c6-4c65-82eb-620196b29548"}]': finished 2024-09-11T22:14:17.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T22:14:17.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:17.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:17 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3780758955' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:14:17.579 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:17.291+0000 7f3e28b4a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:14:17.579 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:17.357+0000 7f3e28b4a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:14:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: Reconfiguring daemon mgr.x on smithi005 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1877775904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8ab7175-48c6-4c65-82eb-620196b29548"}]: dispatch 2024-09-11T22:14:17.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1877775904' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8ab7175-48c6-4c65-82eb-620196b29548"}]': finished 2024-09-11T22:14:17.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T22:14:17.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:17.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:17 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3780758955' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:14:17.860 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:14:17.539+0000 7fc427998040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:14:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: Reconfiguring mgr.x (unknown last config time)... 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: Reconfiguring daemon mgr.x on smithi005 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1877775904' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e8ab7175-48c6-4c65-82eb-620196b29548"}]: dispatch 2024-09-11T22:14:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1877775904' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e8ab7175-48c6-4c65-82eb-620196b29548"}]': finished 2024-09-11T22:14:17.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T22:14:17.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:17.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:17 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3780758955' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:14:18.327 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:18.072+0000 7f3e28b4a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:14:18.327 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:18.186+0000 7f3e28b4a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:14:18.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:18.388+0000 7fe5b57fa640 1 -- 172.21.15.5:0/912938821 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7fe5a800f7a0 con 0x7fe5b8101c00 2024-09-11T22:14:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-mon[35534]: Standby manager daemon z started 2024-09-11T22:14:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:14:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:14:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:14:18.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:14:18.580 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:18.326+0000 7f3e28b4a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:14:18.580 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:18.387+0000 7f3e28b4a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:14:18.580 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:18.536+0000 7f3e28b4a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:14:18.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:18 smithi005 ceph-mon[30943]: Standby manager daemon z started 2024-09-11T22:14:18.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:18 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:14:18.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:18 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:14:18.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:18 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:14:18.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:18 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:14:18.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:18 smithi172 ceph-mon[35461]: Standby manager daemon z started 2024-09-11T22:14:18.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:18 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:14:18.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:18 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:14:18.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:18 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:14:18.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:18 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.172:0/3657833258' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:14:18.883 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:18.604+0000 7f3e28b4a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:14:18.883 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:18.663+0000 7f3e28b4a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:14:18.883 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:18.818+0000 7f3e28b4a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:14:18.884 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:18.883+0000 7f3e28b4a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:14:19.148 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:18.949+0000 7f3e28b4a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:14:19.148 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:19 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:19.030+0000 7f3e28b4a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:14:19.148 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:19 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:19.088+0000 7f3e28b4a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:14:19.148 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:19 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:19.147+0000 7f3e28b4a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:14:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:19 smithi159 ceph-mon[35534]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:19 smithi159 ceph-mon[35534]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T22:14:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:19 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:14:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:19 smithi005 ceph-mon[30943]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:19 smithi005 ceph-mon[30943]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T22:14:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:14:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:19 smithi172 ceph-mon[35461]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:19 smithi172 ceph-mon[35461]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-11T22:14:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:19 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:14:20.329 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:14:19 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:14:19.934+0000 7f3e28b4a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:14:20.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:20.395+0000 7fe5b57fa640 1 -- 172.21.15.5:0/912938821 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fe5a801a3b0 con 0x7fe5b8101c00 2024-09-11T22:14:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:20 smithi005 ceph-mon[30943]: Standby manager daemon y started 2024-09-11T22:14:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:20 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:14:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:20 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:14:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:20 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:14:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:20 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:14:20.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:20 smithi159 ceph-mon[35534]: Standby manager daemon y started 2024-09-11T22:14:20.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:20 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:14:20.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:20 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:14:20.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:20 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:14:20.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:20 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:14:20.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:20 smithi172 ceph-mon[35461]: Standby manager daemon y started 2024-09-11T22:14:20.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:20 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:14:20.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:20 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:14:20.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:20 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:14:20.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:20 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.159:0/1919170820' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:14:21.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:21 smithi005 ceph-mon[30943]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:21.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:21 smithi005 ceph-mon[30943]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-11T22:14:21.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:21 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:14:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:21 smithi159 ceph-mon[35534]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:21 smithi159 ceph-mon[35534]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-11T22:14:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:21 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:14:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:21 smithi172 ceph-mon[35461]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:21 smithi172 ceph-mon[35461]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-11T22:14:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:21 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:14:23.451 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:23 smithi005 ceph-mon[30943]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:23 smithi159 ceph-mon[35534]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:23 smithi172 ceph-mon[35461]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:24.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:24.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:24.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:24.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:24.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:24.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:24.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:24.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:24.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:24.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:24.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:24.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:24.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:24.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:24.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:24.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:24.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:24.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:26.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:25 smithi005 ceph-mon[30943]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:26.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T22:14:26.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:26.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:25 smithi159 ceph-mon[35534]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:26.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T22:14:26.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:26.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:25 smithi172 ceph-mon[35461]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:26.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T22:14:26.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:27.283 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:26 smithi005 ceph-mon[30943]: Deploying daemon osd.0 on smithi005 2024-09-11T22:14:27.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:26 smithi159 ceph-mon[35534]: Deploying daemon osd.0 on smithi005 2024-09-11T22:14:27.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:26 smithi172 ceph-mon[35461]: Deploying daemon osd.0 on smithi005 2024-09-11T22:14:28.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:27 smithi005 ceph-mon[30943]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:28.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:27 smithi159 ceph-mon[35534]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:28.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:27 smithi172 ceph-mon[35461]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:30.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:29 smithi005 ceph-mon[30943]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:30.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:29 smithi159 ceph-mon[35534]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:30.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:29 smithi172 ceph-mon[35461]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:32.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:31 smithi005 ceph-mon[30943]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:32.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:31 smithi159 ceph-mon[35534]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:32.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:31 smithi172 ceph-mon[35461]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:33.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:32 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:33.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:32 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:33.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:32 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:33.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:32 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:33.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:32 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:33.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:32 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:33.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:32 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:33.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:32 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:33.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:32 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:34.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:33 smithi005 ceph-mon[30943]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:33 smithi159 ceph-mon[35534]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:34.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:33 smithi172 ceph-mon[35461]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:36 smithi005 ceph-mon[30943]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:36 smithi159 ceph-mon[35534]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:36.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:36 smithi172 ceph-mon[35461]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:36.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:36.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:36.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:36.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:36.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:36.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.885+0000 7fe5b57fa640 1 -- 172.21.15.5:0/912938821 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe5b8062630 con 0x7fe5a0039290 2024-09-11T22:14:36.887 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.889+0000 7fe5be7c0640 1 -- 172.21.15.5:0/912938821 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe5a0039290 msgr2=0x7fe5a003b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe5a0039290 0x7fe5a003b750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8075d10 tx=0x7fe5ac00b390 comp rx=0 tx=0).stop 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 -- 172.21.15.5:0/912938821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b8101c00 msgr2=0x7fe5b8075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b8101c00 0x7fe5b8075f90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800d950 tx=0x7fe5a800de10 comp rx=0 tx=0).stop 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 -- 172.21.15.5:0/912938821 shutdown_connections 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe5a0039290 0x7fe5a003b750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5b81037c0 0x7fe5b8074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5b8102e00 0x7fe5b80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 --2- 172.21.15.5:0/912938821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b8101c00 0x7fe5b8075f90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 -- 172.21.15.5:0/912938821 >> 172.21.15.5:0/912938821 conn(0x7fe5b80fd390 msgr2=0x7fe5b8106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 -- 172.21.15.5:0/912938821 shutdown_connections 2024-09-11T22:14:36.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:14:36.890+0000 7fe5be7c0640 1 -- 172.21.15.5:0/912938821 wait complete. 2024-09-11T22:14:37.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:37.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:37.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:37.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:37.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:37.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:37.527 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T22:14:37.529 INFO:tasks.cephadm:Deploying osd.1 on smithi159 with /dev/vg_nvme/lv_4... 2024-09-11T22:14:37.530 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T22:14:37.553 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:37.553 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:37.554 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:37.769 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:14:38.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:38 smithi005 ceph-mon[30943]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:38 smithi159 ceph-mon[35534]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:38 smithi172 ceph-mon[35461]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:38.770 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:14:38 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0[41953]: 2024-09-11T22:14:38.465+0000 7fd3da5ef640 -1 osd.0 0 log_to_monitors true 2024-09-11T22:14:39.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:39 smithi005 ceph-mon[30943]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T22:14:39.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:39 smithi159 ceph-mon[35534]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T22:14:39.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:39 smithi172 ceph-mon[35461]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T22:14:40.089 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:14:40.113 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_4 2024-09-11T22:14:40.408 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:14:40.529 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:40 smithi159 ceph-mon[35534]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:40.529 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:40 smithi159 ceph-mon[35534]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T22:14:40.529 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:40 smithi159 ceph-mon[35534]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T22:14:40.529 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:40 smithi159 ceph-mon[35534]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:14:40.529 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:40.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:40 smithi005 ceph-mon[30943]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:40.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:40 smithi005 ceph-mon[30943]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T22:14:40.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:40 smithi005 ceph-mon[30943]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T22:14:40.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:40 smithi005 ceph-mon[30943]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:14:40.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:40 smithi172 ceph-mon[35461]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:40 smithi172 ceph-mon[35461]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T22:14:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:40 smithi172 ceph-mon[35461]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T22:14:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:40 smithi172 ceph-mon[35461]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:14:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.890+0000 7f0894b5e640 1 -- 172.21.15.159:0/993463579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0890101cf0 msgr2=0x7f0890102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.890+0000 7f0894b5e640 1 --2- 172.21.15.159:0/993463579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0890101cf0 0x7f0890102170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0884009900 tx=0x7f088402a410 comp rx=0 tx=0).stop 2024-09-11T22:14:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.890+0000 7f0894b5e640 1 -- 172.21.15.159:0/993463579 shutdown_connections 2024-09-11T22:14:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.890+0000 7f0894b5e640 1 --2- 172.21.15.159:0/993463579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08901026b0 0x7f0890108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.890+0000 7f0894b5e640 1 --2- 172.21.15.159:0/993463579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0890101cf0 0x7f0890102170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.890+0000 7f0894b5e640 1 --2- 172.21.15.159:0/993463579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0890100af0 0x7f0890100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:40.893 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.890+0000 7f0894b5e640 1 -- 172.21.15.159:0/993463579 >> 172.21.15.159:0/993463579 conn(0x7f08900fc280 msgr2=0x7f08900fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.890+0000 7f0894b5e640 1 -- 172.21.15.159:0/993463579 shutdown_connections 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.890+0000 7f0894b5e640 1 -- 172.21.15.159:0/993463579 wait complete. 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f0894b5e640 1 Processor -- start 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f0894b5e640 1 -- start start 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f0894b5e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0890100af0 0x7f0890196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f0894b5e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0890101cf0 0x7f0890197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f0894b5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08901026b0 0x7f089019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f0894b5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f089010c750 con 0x7f08901026b0 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f0894b5e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f089010c5d0 con 0x7f0890100af0 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f0894b5e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f089010c8d0 con 0x7f0890101cf0 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f088e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0890100af0 0x7f0890196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f088e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0890100af0 0x7f0890196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:33322/0 (socket says 172.21.15.159:33322) 2024-09-11T22:14:40.894 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f088e59c640 1 -- 172.21.15.159:0/2503308627 learned_addr learned my addr 172.21.15.159:0/2503308627 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f088ed9d640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08901026b0 0x7f089019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f088dd9b640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0890101cf0 0x7f0890197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f088e59c640 1 -- 172.21.15.159:0/2503308627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0890101cf0 msgr2=0x7f0890197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f088e59c640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0890101cf0 0x7f0890197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f088e59c640 1 -- 172.21.15.159:0/2503308627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08901026b0 msgr2=0x7f089019e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f088e59c640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08901026b0 0x7f089019e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.891+0000 7f088e59c640 1 -- 172.21.15.159:0/2503308627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089019ea00 con 0x7f0890100af0 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.892+0000 7f088e59c640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0890100af0 0x7f0890196dd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f088000d950 tx=0x7f088000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.892+0000 7f087f7fe640 1 -- 172.21.15.159:0/2503308627 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880017020 con 0x7f0890100af0 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.892+0000 7f0894b5e640 1 -- 172.21.15.159:0/2503308627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089006a7f0 con 0x7f0890100af0 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.892+0000 7f087f7fe640 1 -- 172.21.15.159:0/2503308627 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0880009c30 con 0x7f0890100af0 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.892+0000 7f087f7fe640 1 -- 172.21.15.159:0/2503308627 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088000f740 con 0x7f0890100af0 2024-09-11T22:14:40.895 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.892+0000 7f0894b5e640 1 -- 172.21.15.159:0/2503308627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089006ac40 con 0x7f0890100af0 2024-09-11T22:14:40.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.893+0000 7f0894b5e640 1 -- 172.21.15.159:0/2503308627 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0890100ef0 con 0x7f0890100af0 2024-09-11T22:14:40.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.894+0000 7f087f7fe640 1 -- 172.21.15.159:0/2503308627 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f088001a020 con 0x7f0890100af0 2024-09-11T22:14:40.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.894+0000 7f087f7fe640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f08780a2b20 0x7f08780a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:14:40.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.895+0000 7f087f7fe640 1 -- 172.21.15.159:0/2503308627 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f08800ccd70 con 0x7f0890100af0 2024-09-11T22:14:40.896 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.895+0000 7f088dd9b640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f08780a2b20 0x7f08780a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:14:40.897 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.895+0000 7f088dd9b640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f08780a2b20 0x7f08780a4fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f08840314e0 tx=0x7f0884000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:14:40.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:40.899+0000 7f087f7fe640 1 -- 172.21.15.159:0/2503308627 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0880021070 con 0x7f0890100af0 2024-09-11T22:14:41.070 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:14:40 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0[41953]: 2024-09-11T22:14:40.762+0000 7fd3d6572640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T22:14:41.070 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:14:40 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0[41953]: 2024-09-11T22:14:40.765+0000 7fd3ce392640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:14:41.097 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:14:41.096+0000 7f0894b5e640 1 -- 172.21.15.159:0/2503308627 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f089006af00 con 0x7f08780a2b20 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:41.407 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.408 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' 2024-09-11T22:14:41.408 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:14:41.408 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:14:41.408 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:41.460 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:14:41.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:14:41.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:41.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:14:41.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T22:14:41.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:41.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:41.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:14:41.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:41.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:14:41.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:41.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432]' entity='osd.0' 2024-09-11T22:14:41.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:14:41.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:14:41.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:42.527 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:42 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:14:42.527 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:42 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:14:42.527 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:42 smithi159 ceph-mon[35534]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:42.527 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:42 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi005 2024-09-11T22:14:42.527 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:42 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-11T22:14:42.527 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:42 smithi159 ceph-mon[35534]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:42.527 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:42 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:42.527 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:42 smithi159 ceph-mon[35534]: osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] boot 2024-09-11T22:14:42.527 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:42 smithi159 ceph-mon[35534]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T22:14:42.527 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:42 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:42.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:42 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:14:42.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:42 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:14:42.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:42 smithi005 ceph-mon[30943]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:42.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:42 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi005 2024-09-11T22:14:42.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:42 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-11T22:14:42.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:42 smithi005 ceph-mon[30943]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:42.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:42 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:42.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:42 smithi005 ceph-mon[30943]: osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] boot 2024-09-11T22:14:42.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:42 smithi005 ceph-mon[30943]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T22:14:42.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:42 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:42 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:14:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:42 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:14:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:42 smithi172 ceph-mon[35461]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T22:14:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:42 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi005 2024-09-11T22:14:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:42 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-11T22:14:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:42 smithi172 ceph-mon[35461]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:14:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:42 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:42 smithi172 ceph-mon[35461]: osd.0 [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] boot 2024-09-11T22:14:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:42 smithi172 ceph-mon[35461]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T22:14:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:42 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:14:44.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:44 smithi005 ceph-mon[30943]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:44.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:44 smithi005 ceph-mon[30943]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T22:14:44.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:44 smithi005 ceph-mon[30943]: from='client.? 172.21.15.159:0/1311891098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a379e10-b214-42db-8b31-399b39b6cac1"}]: dispatch 2024-09-11T22:14:44.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:44 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a379e10-b214-42db-8b31-399b39b6cac1"}]: dispatch 2024-09-11T22:14:44.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:44 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a379e10-b214-42db-8b31-399b39b6cac1"}]': finished 2024-09-11T22:14:44.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:44 smithi005 ceph-mon[30943]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T22:14:44.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:44 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:14:44.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:44 smithi005 ceph-mon[30943]: from='client.? 172.21.15.159:0/2694106090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:14:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:44 smithi159 ceph-mon[35534]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:44 smithi159 ceph-mon[35534]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T22:14:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:44 smithi159 ceph-mon[35534]: from='client.? 172.21.15.159:0/1311891098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a379e10-b214-42db-8b31-399b39b6cac1"}]: dispatch 2024-09-11T22:14:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:44 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a379e10-b214-42db-8b31-399b39b6cac1"}]: dispatch 2024-09-11T22:14:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:44 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a379e10-b214-42db-8b31-399b39b6cac1"}]': finished 2024-09-11T22:14:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:44 smithi159 ceph-mon[35534]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T22:14:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:44 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:14:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:44 smithi159 ceph-mon[35534]: from='client.? 172.21.15.159:0/2694106090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:14:44.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:44 smithi172 ceph-mon[35461]: pgmap v41: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:44.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:44 smithi172 ceph-mon[35461]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T22:14:44.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:44 smithi172 ceph-mon[35461]: from='client.? 172.21.15.159:0/1311891098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a379e10-b214-42db-8b31-399b39b6cac1"}]: dispatch 2024-09-11T22:14:44.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:44 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a379e10-b214-42db-8b31-399b39b6cac1"}]: dispatch 2024-09-11T22:14:44.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:44 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a379e10-b214-42db-8b31-399b39b6cac1"}]': finished 2024-09-11T22:14:44.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:44 smithi172 ceph-mon[35461]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T22:14:44.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:44 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:14:44.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:44 smithi172 ceph-mon[35461]: from='client.? 172.21.15.159:0/2694106090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:14:46.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:46 smithi005 ceph-mon[30943]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:46 smithi159 ceph-mon[35534]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:46.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:46 smithi172 ceph-mon[35461]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:48.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:48 smithi159 ceph-mon[35534]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:48.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:48 smithi005 ceph-mon[30943]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:48.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:48 smithi172 ceph-mon[35461]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:50.490 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:50 smithi159 ceph-mon[35534]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:50.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:50 smithi005 ceph-mon[30943]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:50 smithi172 ceph-mon[35461]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:51.484 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T22:14:51.484 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:51.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T22:14:51.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:51.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T22:14:51.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:14:52.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:52 smithi005 ceph-mon[30943]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:52.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:52 smithi005 ceph-mon[30943]: Deploying daemon osd.1 on smithi159 2024-09-11T22:14:52.571 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:52 smithi159 ceph-mon[35534]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:52.571 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:52 smithi159 ceph-mon[35534]: Deploying daemon osd.1 on smithi159 2024-09-11T22:14:52.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:52 smithi172 ceph-mon[35461]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:52.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:52 smithi172 ceph-mon[35461]: Deploying daemon osd.1 on smithi159 2024-09-11T22:14:54.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:54 smithi005 ceph-mon[30943]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:54.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:54 smithi159 ceph-mon[35534]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:54.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:54 smithi172 ceph-mon[35461]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:56.396 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:56 smithi159 ceph-mon[35534]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:56.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:56 smithi005 ceph-mon[30943]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:56.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:56 smithi172 ceph-mon[35461]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:58.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:58 smithi005 ceph-mon[30943]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:58.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:58.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:58.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:14:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:58.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:58 smithi159 ceph-mon[35534]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:58.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:58.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:58.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:14:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:58.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:58 smithi172 ceph-mon[35461]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:14:58.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:14:58.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:14:58.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:14:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:00 smithi005 ceph-mon[30943]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:00 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:00 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:00 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:00 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:00.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:00 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:00 smithi159 ceph-mon[35534]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:00 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:00 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:00 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:00 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:00 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:00.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:00 smithi172 ceph-mon[35461]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:00.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:00 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:00.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:00 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:00.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:00 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:00.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:00 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:00.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:00 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:01.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:01 smithi159 ceph-mon[35534]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:01.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:01 smithi172 ceph-mon[35461]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:01.784 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.784+0000 7f087f7fe640 1 -- 172.21.15.159:0/2503308627 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f089006af00 con 0x7f08780a2b20 2024-09-11T22:15:01.784 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 1 on host 'smithi159' 2024-09-11T22:15:01.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.786+0000 7f0894b5e640 1 -- 172.21.15.159:0/2503308627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f08780a2b20 msgr2=0x7f08780a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:01.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.786+0000 7f0894b5e640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f08780a2b20 0x7f08780a4fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f08840314e0 tx=0x7f0884000bc0 comp rx=0 tx=0).stop 2024-09-11T22:15:01.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.786+0000 7f0894b5e640 1 -- 172.21.15.159:0/2503308627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0890100af0 msgr2=0x7f0890196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:01.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.786+0000 7f0894b5e640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0890100af0 0x7f0890196dd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f088000d950 tx=0x7f088000de10 comp rx=0 tx=0).stop 2024-09-11T22:15:01.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.786+0000 7f0894b5e640 1 -- 172.21.15.159:0/2503308627 shutdown_connections 2024-09-11T22:15:01.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.786+0000 7f0894b5e640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f08780a2b20 0x7f08780a4fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:01.787 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.786+0000 7f0894b5e640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08901026b0 0x7f089019e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.786+0000 7f0894b5e640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0890101cf0 0x7f0890197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.786+0000 7f0894b5e640 1 --2- 172.21.15.159:0/2503308627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0890100af0 0x7f0890196dd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.786+0000 7f0894b5e640 1 -- 172.21.15.159:0/2503308627 >> 172.21.15.159:0/2503308627 conn(0x7f08900fc280 msgr2=0x7f08901050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:15:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.787+0000 7f0894b5e640 1 -- 172.21.15.159:0/2503308627 shutdown_connections 2024-09-11T22:15:01.788 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:01.787+0000 7f0894b5e640 1 -- 172.21.15.159:0/2503308627 wait complete. 2024-09-11T22:15:01.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:01 smithi005 ceph-mon[30943]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:02.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:02 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:02.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:02 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:02.413 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:02 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:02.455 DEBUG:teuthology.orchestra.run.smithi159:osd.1> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T22:15:02.458 INFO:tasks.cephadm:Deploying osd.2 on smithi172 with /dev/vg_nvme/lv_4... 2024-09-11T22:15:02.458 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-11T22:15:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:02 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:02 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:02 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:02.711 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:15:02.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:02 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:02.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:02 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:02.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:02 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:03.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:03 smithi172 ceph-mon[35461]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:03.629 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:03 smithi159 ceph-mon[35534]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:03.629 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1[40011]: 2024-09-11T22:15:03.427+0000 7f8ca24c7640 -1 osd.1 0 log_to_monitors true 2024-09-11T22:15:03.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:03 smithi005 ceph-mon[30943]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:04.505 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:04 smithi172 ceph-mon[35461]: from='osd.1 [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:15:04.505 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:04 smithi172 ceph-mon[35461]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:15:04.818 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:04 smithi159 ceph-mon[35534]: from='osd.1 [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:15:04.818 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:04 smithi159 ceph-mon[35534]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:15:04.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:04 smithi005 ceph-mon[30943]: from='osd.1 [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:15:04.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:04 smithi005 ceph-mon[30943]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T22:15:05.162 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:15:05.191 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_4 2024-09-11T22:15:05.445 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:15:05.474 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T22:15:05.474 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='osd.1 [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:15:05.474 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T22:15:05.474 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:05.474 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:15:05.475 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:05.475 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi159 2024-09-11T22:15:05.475 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.475 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.475 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:05.475 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi159 to 17088M 2024-09-11T22:15:05.475 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.475 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:05.475 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:05.476 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:05 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T22:15:05.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='osd.1 [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:15:05.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T22:15:05.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi159 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi159 to 17088M 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:05.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T22:15:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='osd.1 [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:15:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T22:15:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:15:05.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:05.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi159 2024-09-11T22:15:05.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:05.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi159 to 17088M 2024-09-11T22:15:05.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:05.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:05.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:05 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.880+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3418808672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda9c101d10 msgr2=0x7fda9c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.880+0000 7fdaa2c8a640 1 --2- 172.21.15.172:0/3418808672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda9c101d10 0x7fda9c102190 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fda88009900 tx=0x7fda8802a410 comp rx=0 tx=0).stop 2024-09-11T22:15:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.880+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3418808672 shutdown_connections 2024-09-11T22:15:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.880+0000 7fdaa2c8a640 1 --2- 172.21.15.172:0/3418808672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda9c1026d0 0x7fda9c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.880+0000 7fdaa2c8a640 1 --2- 172.21.15.172:0/3418808672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda9c101d10 0x7fda9c102190 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:05.887 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.880+0000 7fdaa2c8a640 1 --2- 172.21.15.172:0/3418808672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c100b10 0x7fda9c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.880+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3418808672 >> 172.21.15.172:0/3418808672 conn(0x7fda9c0fc2a0 msgr2=0x7fda9c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.880+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3418808672 shutdown_connections 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.880+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3418808672 wait complete. 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa2c8a640 1 Processor -- start 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa2c8a640 1 -- start start 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa2c8a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda9c100b10 0x7fda9c104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa2c8a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c101d10 0x7fda9c1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa2c8a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda9c1026d0 0x7fda9c103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa2c8a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c10c7e0 con 0x7fda9c101d10 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa2c8a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c10c660 con 0x7fda9c1026d0 2024-09-11T22:15:05.888 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa2c8a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c10c960 con 0x7fda9c100b10 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa0a26640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda9c100b10 0x7fda9c104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa0a26640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda9c100b10 0x7fda9c104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:52104/0 (socket says 172.21.15.172:52104) 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa0a26640 1 -- 172.21.15.172:0/3904286622 learned_addr learned my addr 172.21.15.172:0/3904286622 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa1227640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda9c1026d0 0x7fda9c103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa0a26640 1 -- 172.21.15.172:0/3904286622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda9c1026d0 msgr2=0x7fda9c103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa0a26640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda9c1026d0 0x7fda9c103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa0a26640 1 -- 172.21.15.172:0/3904286622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c101d10 msgr2=0x7fda9c1030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa0a26640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c101d10 0x7fda9c1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.881+0000 7fdaa0a26640 1 -- 172.21.15.172:0/3904286622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c103ef0 con 0x7fda9c100b10 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.882+0000 7fdaa0a26640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda9c100b10 0x7fda9c104aa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fda9000d950 tx=0x7fda9000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.882+0000 7fda99ffb640 1 -- 172.21.15.172:0/3904286622 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90017020 con 0x7fda9c100b10 2024-09-11T22:15:05.889 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.882+0000 7fda99ffb640 1 -- 172.21.15.172:0/3904286622 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda90009c30 con 0x7fda9c100b10 2024-09-11T22:15:05.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.882+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3904286622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c1a20c0 con 0x7fda9c100b10 2024-09-11T22:15:05.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.882+0000 7fda99ffb640 1 -- 172.21.15.172:0/3904286622 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9000f740 con 0x7fda9c100b10 2024-09-11T22:15:05.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.882+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3904286622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c1a2620 con 0x7fda9c100b10 2024-09-11T22:15:05.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.883+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3904286622 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c1a28b0 con 0x7fda9c100b10 2024-09-11T22:15:05.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.884+0000 7fda99ffb640 1 -- 172.21.15.172:0/3904286622 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fda9001a020 con 0x7fda9c100b10 2024-09-11T22:15:05.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.885+0000 7fda99ffb640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fda8c0a2b70 0x7fda8c0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:05.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.885+0000 7fda99ffb640 1 -- 172.21.15.172:0/3904286622 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fda900cd400 con 0x7fda9c100b10 2024-09-11T22:15:05.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.886+0000 7fda9bfff640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fda8c0a2b70 0x7fda8c0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:05.890 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.886+0000 7fda9bfff640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fda8c0a2b70 0x7fda8c0a5030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda880057c0 tx=0x7fda88031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:15:05.891 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:05.889+0000 7fda99ffb640 1 -- 172.21.15.172:0/3904286622 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda90021070 con 0x7fda9c100b10 2024-09-11T22:15:06.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:06.072+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3904286622 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fda9c1a2a80 con 0x7fda8c0a2b70 2024-09-11T22:15:06.079 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1[40011]: 2024-09-11T22:15:05.785+0000 7f8c9e44a640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T22:15:06.079 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:05 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1[40011]: 2024-09-11T22:15:05.788+0000 7f8c9626a640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:15:06.618 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:06 smithi172 ceph-mon[35461]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:15:06.618 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:06 smithi172 ceph-mon[35461]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T22:15:06.618 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:06.618 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:06.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:06 smithi172 ceph-mon[35461]: from='osd.1 ' entity='osd.1' 2024-09-11T22:15:06.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:06 smithi172 ceph-mon[35461]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:15:06.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:15:06.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:15:06.619 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:06.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:06 smithi005 ceph-mon[30943]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:15:06.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:06 smithi005 ceph-mon[30943]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T22:15:06.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:06.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:06.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:06 smithi005 ceph-mon[30943]: from='osd.1 ' entity='osd.1' 2024-09-11T22:15:06.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:06 smithi005 ceph-mon[30943]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:15:06.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:15:06.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:15:06.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:06 smithi159 ceph-mon[35534]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:15:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:06 smithi159 ceph-mon[35534]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T22:15:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:06 smithi159 ceph-mon[35534]: from='osd.1 ' entity='osd.1' 2024-09-11T22:15:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:06 smithi159 ceph-mon[35534]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:15:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:15:06.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:15:06.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:07.426 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:07 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:15:07.426 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:07 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:15:07.426 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:07.426 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:07 smithi172 ceph-mon[35461]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:07.426 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:07 smithi172 ceph-mon[35461]: osd.1 [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] boot 2024-09-11T22:15:07.426 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:07 smithi172 ceph-mon[35461]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T22:15:07.426 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:07.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:07 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:15:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:07 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:15:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:07 smithi159 ceph-mon[35534]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:07 smithi159 ceph-mon[35534]: osd.1 [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] boot 2024-09-11T22:15:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:07 smithi159 ceph-mon[35534]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T22:15:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:07.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:07 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:15:07.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:07 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:15:07.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:07.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:07 smithi005 ceph-mon[30943]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-11T22:15:07.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:07 smithi005 ceph-mon[30943]: osd.1 [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] boot 2024-09-11T22:15:07.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:07 smithi005 ceph-mon[30943]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T22:15:07.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:15:08.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:08 smithi172 ceph-mon[35461]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T22:15:08.938 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:08 smithi172 ceph-mon[35461]: from='client.? 172.21.15.172:0/503641539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d23d4d6-1f97-4d0e-9c99-ae865e04ca24"}]: dispatch 2024-09-11T22:15:08.939 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:08 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d23d4d6-1f97-4d0e-9c99-ae865e04ca24"}]: dispatch 2024-09-11T22:15:08.939 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:08 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d23d4d6-1f97-4d0e-9c99-ae865e04ca24"}]': finished 2024-09-11T22:15:08.939 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:08 smithi172 ceph-mon[35461]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T22:15:08.939 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:09.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:08 smithi005 ceph-mon[30943]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T22:15:09.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:08 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/503641539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d23d4d6-1f97-4d0e-9c99-ae865e04ca24"}]: dispatch 2024-09-11T22:15:09.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:08 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d23d4d6-1f97-4d0e-9c99-ae865e04ca24"}]: dispatch 2024-09-11T22:15:09.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:08 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d23d4d6-1f97-4d0e-9c99-ae865e04ca24"}]': finished 2024-09-11T22:15:09.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:08 smithi005 ceph-mon[30943]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T22:15:09.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:09.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:08 smithi159 ceph-mon[35534]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T22:15:09.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:08 smithi159 ceph-mon[35534]: from='client.? 172.21.15.172:0/503641539' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d23d4d6-1f97-4d0e-9c99-ae865e04ca24"}]: dispatch 2024-09-11T22:15:09.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:08 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d23d4d6-1f97-4d0e-9c99-ae865e04ca24"}]: dispatch 2024-09-11T22:15:09.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:08 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d23d4d6-1f97-4d0e-9c99-ae865e04ca24"}]': finished 2024-09-11T22:15:09.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:08 smithi159 ceph-mon[35534]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T22:15:09.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:09 smithi005 ceph-mon[30943]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:09 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/2564210678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:15:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:09 smithi159 ceph-mon[35534]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:09 smithi159 ceph-mon[35534]: from='client.? 172.21.15.172:0/2564210678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:15:10.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:09 smithi172 ceph-mon[35461]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:10.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:09 smithi172 ceph-mon[35461]: from='client.? 172.21.15.172:0/2564210678' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:15:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:11 smithi005 ceph-mon[30943]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:11 smithi159 ceph-mon[35534]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:12.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:11 smithi172 ceph-mon[35461]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:14.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:13 smithi005 ceph-mon[30943]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:14.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:13 smithi159 ceph-mon[35534]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:14.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:13 smithi172 ceph-mon[35461]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:15.950 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:15 smithi172 ceph-mon[35461]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:16.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:15 smithi005 ceph-mon[30943]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:16.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:15 smithi159 ceph-mon[35534]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:17.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T22:15:17.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:17.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T22:15:17.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:17.081 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T22:15:17.081 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:18.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:17 smithi005 ceph-mon[30943]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:18.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:17 smithi005 ceph-mon[30943]: Deploying daemon osd.2 on smithi172 2024-09-11T22:15:18.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:17 smithi159 ceph-mon[35534]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:18.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:17 smithi159 ceph-mon[35534]: Deploying daemon osd.2 on smithi172 2024-09-11T22:15:18.083 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:17 smithi172 ceph-mon[35461]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:18.083 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:17 smithi172 ceph-mon[35461]: Deploying daemon osd.2 on smithi172 2024-09-11T22:15:20.041 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:19 smithi172 ceph-mon[35461]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:19 smithi005 ceph-mon[30943]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:20.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:19 smithi159 ceph-mon[35534]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:22.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:21 smithi005 ceph-mon[30943]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:22.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:21 smithi159 ceph-mon[35534]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:22.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:21 smithi172 ceph-mon[35461]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:23 smithi159 ceph-mon[35534]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:24.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:23 smithi172 ceph-mon[35461]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:23 smithi005 ceph-mon[30943]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:25 smithi159 ceph-mon[35534]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:26.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:26.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:26.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:25 smithi172 ceph-mon[35461]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:25 smithi005 ceph-mon[30943]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.670 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.669+0000 7fda99ffb640 1 -- 172.21.15.172:0/3904286622 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fda9c1a2a80 con 0x7fda8c0a2b70 2024-09-11T22:15:26.671 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 2 on host 'smithi172' 2024-09-11T22:15:26.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.671+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3904286622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fda8c0a2b70 msgr2=0x7fda8c0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:26.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.671+0000 7fdaa2c8a640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fda8c0a2b70 0x7fda8c0a5030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fda880057c0 tx=0x7fda88031400 comp rx=0 tx=0).stop 2024-09-11T22:15:26.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.671+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3904286622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda9c100b10 msgr2=0x7fda9c104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:26.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.671+0000 7fdaa2c8a640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda9c100b10 0x7fda9c104aa0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fda9000d950 tx=0x7fda9000de10 comp rx=0 tx=0).stop 2024-09-11T22:15:26.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.671+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3904286622 shutdown_connections 2024-09-11T22:15:26.673 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.671+0000 7fdaa2c8a640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fda8c0a2b70 0x7fda8c0a5030 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.671+0000 7fdaa2c8a640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda9c1026d0 0x7fda9c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.671+0000 7fdaa2c8a640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda9c101d10 0x7fda9c1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.671+0000 7fdaa2c8a640 1 --2- 172.21.15.172:0/3904286622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda9c100b10 0x7fda9c104aa0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.672+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3904286622 >> 172.21.15.172:0/3904286622 conn(0x7fda9c0fc2a0 msgr2=0x7fda9c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:15:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.672+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3904286622 shutdown_connections 2024-09-11T22:15:26.674 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:15:26.672+0000 7fdaa2c8a640 1 -- 172.21.15.172:0/3904286622 wait complete. 2024-09-11T22:15:26.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:26 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:26.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:26 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:26.992 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:26 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:27.247 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:26 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:27.247 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:26 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:27.247 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:26 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:27.255 DEBUG:teuthology.orchestra.run.smithi172:osd.2> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T22:15:27.257 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2024-09-11T22:15:27.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T22:15:27.281 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:27.281 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:27.281 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:27.512 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:15:27.831 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:27 smithi172 ceph-mon[35461]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:27.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:27 smithi005 ceph-mon[30943]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:28.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:27 smithi159 ceph-mon[35534]: pgmap v70: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:29.361 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:28 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2[40430]: 2024-09-11T22:15:28.887+0000 7ff27b84b640 -1 osd.2 0 log_to_monitors true 2024-09-11T22:15:30.031 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:29 smithi172 ceph-mon[35461]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:30.031 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:29 smithi172 ceph-mon[35461]: from='osd.2 [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:15:30.031 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:29 smithi172 ceph-mon[35461]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:15:30.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:29 smithi005 ceph-mon[30943]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:30.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:29 smithi005 ceph-mon[30943]: from='osd.2 [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:15:30.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:29 smithi005 ceph-mon[30943]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:15:30.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:29 smithi159 ceph-mon[35534]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:30.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:29 smithi159 ceph-mon[35534]: from='osd.2 [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:15:30.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:29 smithi159 ceph-mon[35534]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T22:15:30.473 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:15:30.498 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2024-09-11T22:15:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:15:31.098 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='osd.2 [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi172 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:31.100 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:31.100 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:30 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T22:15:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T22:15:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='osd.2 [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:15:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:15:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi172 2024-09-11T22:15:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-11T22:15:31.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:31.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:31.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:30 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T22:15:31.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T22:15:31.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:31.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='osd.2 [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:15:31.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:15:31.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi172 2024-09-11T22:15:31.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:31.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi172 to 17088M 2024-09-11T22:15:31.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:31.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:31.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:30 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.418+0000 7fad74cd9640 1 -- 172.21.15.5:0/1219642526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad70102690 msgr2=0x7fad70108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:31.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.418+0000 7fad74cd9640 1 --2- 172.21.15.5:0/1219642526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad70102690 0x7fad70108f20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad68009ca0 tx=0x7fad6802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.418+0000 7fad74cd9640 1 -- 172.21.15.5:0/1219642526 shutdown_connections 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.418+0000 7fad74cd9640 1 --2- 172.21.15.5:0/1219642526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad70102690 0x7fad70108f20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.418+0000 7fad74cd9640 1 --2- 172.21.15.5:0/1219642526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad70101cd0 0x7fad70102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.418+0000 7fad74cd9640 1 --2- 172.21.15.5:0/1219642526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad70100ad0 0x7fad70100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.418+0000 7fad74cd9640 1 -- 172.21.15.5:0/1219642526 >> 172.21.15.5:0/1219642526 conn(0x7fad700fc280 msgr2=0x7fad700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.418+0000 7fad74cd9640 1 -- 172.21.15.5:0/1219642526 shutdown_connections 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.418+0000 7fad74cd9640 1 -- 172.21.15.5:0/1219642526 wait complete. 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad74cd9640 1 Processor -- start 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad74cd9640 1 -- start start 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad74cd9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad70100ad0 0x7fad7010b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad74cd9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad70101cd0 0x7fad70109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad74cd9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad70102690 0x7fad70109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad74cd9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad7010d880 con 0x7fad70102690 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad74cd9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fad7010d700 con 0x7fad70100ad0 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad74cd9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fad7010da00 con 0x7fad70101cd0 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad6ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad70102690 0x7fad70109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad6ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad70102690 0x7fad70109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49746/0 (socket says 172.21.15.5:49746) 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad6ed9d640 1 -- 172.21.15.5:0/2143457124 learned_addr learned my addr 172.21.15.5:0/2143457124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad6dd9b640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad70101cd0 0x7fad70109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.419+0000 7fad6e59c640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad70100ad0 0x7fad7010b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad6ed9d640 1 -- 172.21.15.5:0/2143457124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad70101cd0 msgr2=0x7fad70109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad6ed9d640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad70101cd0 0x7fad70109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad6ed9d640 1 -- 172.21.15.5:0/2143457124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad70100ad0 msgr2=0x7fad7010b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad6ed9d640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad70100ad0 0x7fad7010b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad6ed9d640 1 -- 172.21.15.5:0/2143457124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7010a8a0 con 0x7fad70102690 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad6ed9d640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad70102690 0x7fad70109fe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fad680098d0 tx=0x7fad6802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad5f7fe640 1 -- 172.21.15.5:0/2143457124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6802fc30 con 0x7fad70102690 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad5f7fe640 1 -- 172.21.15.5:0/2143457124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad68003e50 con 0x7fad70102690 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad5f7fe640 1 -- 172.21.15.5:0/2143457124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad680305c0 con 0x7fad70102690 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad74cd9640 1 -- 172.21.15.5:0/2143457124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad701a1fe0 con 0x7fad70102690 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.420+0000 7fad74cd9640 1 -- 172.21.15.5:0/2143457124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad701a2490 con 0x7fad70102690 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.421+0000 7fad74cd9640 1 -- 172.21.15.5:0/2143457124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad70107e70 con 0x7fad70102690 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.422+0000 7fad5f7fe640 1 -- 172.21.15.5:0/2143457124 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fad68003970 con 0x7fad70102690 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.423+0000 7fad5f7fe640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fad580a2af0 0x7fad580a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.423+0000 7fad6e59c640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fad580a2af0 0x7fad580a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.423+0000 7fad5f7fe640 1 -- 172.21.15.5:0/2143457124 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7fad680eec40 con 0x7fad70102690 2024-09-11T22:15:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.424+0000 7fad6e59c640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fad580a2af0 0x7fad580a4fb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad70101b30 tx=0x7fad6000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:15:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.428+0000 7fad5f7fe640 1 -- 172.21.15.5:0/2143457124 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad680bd370 con 0x7fad70102690 2024-09-11T22:15:31.610 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2[40430]: 2024-09-11T22:15:31.240+0000 7ff2777ce640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T22:15:31.610 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2[40430]: 2024-09-11T22:15:31.244+0000 7ff26f5ee640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:15:31.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:31.612+0000 7fad74cd9640 1 -- 172.21.15.5:0/2143457124 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fad70062630 con 0x7fad580a2af0 2024-09-11T22:15:32.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:31 smithi005 ceph-mon[30943]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:32.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:31 smithi005 ceph-mon[30943]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:15:32.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:31 smithi005 ceph-mon[30943]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T22:15:32.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:31 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:32.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:31 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:32.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:31 smithi005 ceph-mon[30943]: from='osd.2 ' entity='osd.2' 2024-09-11T22:15:32.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:31 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:15:32.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:31 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:15:32.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:31 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-mon[35461]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-mon[35461]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:15:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-mon[35461]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T22:15:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-mon[35461]: from='osd.2 ' entity='osd.2' 2024-09-11T22:15:32.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:15:32.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:15:32.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:31 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:32.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:31 smithi159 ceph-mon[35534]: pgmap v73: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:32.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:31 smithi159 ceph-mon[35534]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:15:32.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:31 smithi159 ceph-mon[35534]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T22:15:32.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:31 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:32.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:31 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:32.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:31 smithi159 ceph-mon[35534]: from='osd.2 ' entity='osd.2' 2024-09-11T22:15:32.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:31 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:15:32.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:31 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:15:32.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:31 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:33.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:32 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:15:33.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:32 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:15:33.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:32 smithi172 ceph-mon[35461]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:15:33.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:32 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:33.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:32 smithi172 ceph-mon[35461]: osd.2 [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] boot 2024-09-11T22:15:33.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:32 smithi172 ceph-mon[35461]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T22:15:33.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:32 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:32 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:15:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:32 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:15:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:32 smithi005 ceph-mon[30943]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:15:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:32 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:32 smithi005 ceph-mon[30943]: osd.2 [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] boot 2024-09-11T22:15:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:32 smithi005 ceph-mon[30943]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T22:15:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:32 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:33.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:32 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:15:33.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:32 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:15:33.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:32 smithi159 ceph-mon[35534]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:15:33.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:32 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:33.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:32 smithi159 ceph-mon[35534]: osd.2 [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] boot 2024-09-11T22:15:33.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:32 smithi159 ceph-mon[35534]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T22:15:33.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:32 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:15:33.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:33 smithi005 ceph-mon[30943]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:33.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:15:33.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T22:15:33.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:33 smithi005 ceph-mon[30943]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T22:15:33.971 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:15:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:33 smithi172 ceph-mon[35461]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:15:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T22:15:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:33 smithi172 ceph-mon[35461]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T22:15:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:15:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:33 smithi159 ceph-mon[35534]: pgmap v76: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-11T22:15:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:15:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-11T22:15:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:33 smithi159 ceph-mon[35534]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T22:15:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:15:35.070 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:15:34 smithi005 sudo[46381]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:15:35.070 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:15:34 smithi005 sudo[46381]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:35.070 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:15:34 smithi005 sudo[46381]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:35.071 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:15:34 smithi005 sudo[46381]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:35.315+0000 7fad5f7fe640 1 -- 172.21.15.5:0/2143457124 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fad680b92e0 con 0x7fad70102690 2024-09-11T22:15:35.542 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:35 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:15:35.542 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:35 smithi159 ceph-mon[35534]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T22:15:35.542 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:35 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2765730584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35db3b3e-4a85-4b80-9df0-581f33718744"}]: dispatch 2024-09-11T22:15:35.542 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:35 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2765730584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35db3b3e-4a85-4b80-9df0-581f33718744"}]': finished 2024-09-11T22:15:35.543 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:35 smithi159 ceph-mon[35534]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T22:15:35.543 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:35 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:35.543 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:35 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2270659769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:15:35.570 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:15:35 smithi005 sudo[46430]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:15:35.570 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:15:35 smithi005 sudo[46430]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:35.570 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:15:35 smithi005 sudo[46430]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:35.570 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:15:35 smithi005 sudo[46430]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:35.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:35 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:15:35.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:35 smithi005 ceph-mon[30943]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T22:15:35.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:35 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2765730584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35db3b3e-4a85-4b80-9df0-581f33718744"}]: dispatch 2024-09-11T22:15:35.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:35 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2765730584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35db3b3e-4a85-4b80-9df0-581f33718744"}]': finished 2024-09-11T22:15:35.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:35 smithi005 ceph-mon[30943]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T22:15:35.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:35 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:35.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:35 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2270659769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:15:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:35 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:15:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:35 smithi172 ceph-mon[35461]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T22:15:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:35 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2765730584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35db3b3e-4a85-4b80-9df0-581f33718744"}]: dispatch 2024-09-11T22:15:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:35 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2765730584' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35db3b3e-4a85-4b80-9df0-581f33718744"}]': finished 2024-09-11T22:15:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:35 smithi172 ceph-mon[35461]: osdmap e21: 4 total, 3 up, 4 in 2024-09-11T22:15:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:35 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:35 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2270659769' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:15:35.797 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:35 smithi159 sudo[43993]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:15:35.797 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:35 smithi159 sudo[43993]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:35.797 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:35 smithi159 sudo[43993]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:35.798 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:35 smithi159 sudo[43993]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:36.079 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:35 smithi159 sudo[43996]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:15:36.079 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:35 smithi159 sudo[43996]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:36.079 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:35 smithi159 sudo[43996]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:36.079 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:15:35 smithi159 sudo[43996]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:36.321 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:36 smithi172 ceph-mon[35461]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:36.321 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:36.322 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:36 smithi172 ceph-mon[35461]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T22:15:36.322 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:36.322 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:36 smithi172 sudo[44508]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:15:36.322 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:36 smithi172 sudo[44508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:36.322 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:36 smithi172 sudo[44508]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:36.322 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:36 smithi172 sudo[44508]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:36 smithi005 ceph-mon[30943]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:36 smithi005 ceph-mon[30943]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T22:15:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:36 smithi159 ceph-mon[35534]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:36 smithi159 ceph-mon[35534]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-11T22:15:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:36.610 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:36 smithi172 sudo[44511]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:15:36.610 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:36 smithi172 sudo[44511]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:36.610 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:36 smithi172 sudo[44511]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:36.610 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:15:36 smithi172 sudo[44511]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:37.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:36 smithi005 sudo[46811]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:15:37.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:36 smithi005 sudo[46811]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:37.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:36 smithi005 sudo[46811]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:37.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:36 smithi005 sudo[46811]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:37.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:37 smithi005 sudo[46816]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:15:37.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:37 smithi005 sudo[46816]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:37.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:37 smithi005 sudo[46816]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:37.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:37 smithi005 sudo[46816]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:37.743 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:37 smithi172 sudo[44514]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:15:37.743 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:37 smithi172 sudo[44514]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:37.743 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:37 smithi172 sudo[44514]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:37.743 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:37 smithi172 sudo[44514]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:37 smithi172 sudo[44517]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:15:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:37 smithi172 sudo[44517]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:37 smithi172 sudo[44517]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:37 smithi172 sudo[44517]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:38.246 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:37 smithi159 sudo[43999]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T22:15:38.246 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:37 smithi159 sudo[43999]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:38.246 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:37 smithi159 sudo[43999]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:38.246 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 sudo[43999]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:38.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:38.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:38 smithi005 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:15:38.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:38 smithi005 ceph-mon[30943]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:38.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:38 smithi005 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:15:38.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:15:38.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:15:38.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:15:38.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:15:38.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:15:38.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:38 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:15:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 sudo[44002]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T22:15:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 sudo[44002]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T22:15:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 sudo[44002]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T22:15:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 sudo[44002]: pam_unix(sudo:session): session closed for user root 2024-09-11T22:15:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 ceph-mon[35534]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:15:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 ceph-mon[35534]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 ceph-mon[35534]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:15:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:15:38.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:15:38.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:15:38.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:15:38.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:15:38.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:38 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:15:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:38 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:38 smithi172 ceph-mon[35461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:15:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:38 smithi172 ceph-mon[35461]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:38 smithi172 ceph-mon[35461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:15:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:38 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:15:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:38 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:15:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:38 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:15:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:38 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:15:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:38 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:15:38.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:38 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:15:39.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:39 smithi005 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:15:39.465 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:39 smithi005 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:15:39.465 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:39 smithi005 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:15:39.465 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:15:39.465 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:15:39.465 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:15:39.465 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:39 smithi005 ceph-mon[30943]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:15:39.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:39 smithi159 ceph-mon[35534]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:15:39.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:39 smithi159 ceph-mon[35534]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:15:39.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:39 smithi159 ceph-mon[35534]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:15:39.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:15:39.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:15:39.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:15:39.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:39 smithi159 ceph-mon[35534]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:15:39.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:39 smithi172 ceph-mon[35461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:15:39.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:39 smithi172 ceph-mon[35461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:15:39.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:39 smithi172 ceph-mon[35461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T22:15:39.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:15:39.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:15:39.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:15:39.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:39 smithi172 ceph-mon[35461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T22:15:40.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:40 smithi005 ceph-mon[30943]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:40.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:40 smithi159 ceph-mon[35534]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:40 smithi172 ceph-mon[35461]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:42.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:42 smithi005 ceph-mon[30943]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:42.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:42 smithi159 ceph-mon[35534]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:42 smithi172 ceph-mon[35461]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:43.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:43 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T22:15:43.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:43 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:43.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:43 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T22:15:43.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:43 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:43 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T22:15:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:43 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:44.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:44 smithi005 ceph-mon[30943]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T22:15:44.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:44 smithi005 ceph-mon[30943]: Deploying daemon osd.3 on smithi005 2024-09-11T22:15:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:44 smithi159 ceph-mon[35534]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T22:15:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:44 smithi159 ceph-mon[35534]: Deploying daemon osd.3 on smithi005 2024-09-11T22:15:44.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:44 smithi172 ceph-mon[35461]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-11T22:15:44.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:44 smithi172 ceph-mon[35461]: Deploying daemon osd.3 on smithi005 2024-09-11T22:15:46.454 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:46 smithi005 ceph-mon[30943]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T22:15:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:46 smithi159 ceph-mon[35534]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T22:15:46.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:46 smithi172 ceph-mon[35461]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-11T22:15:48.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:48 smithi005 ceph-mon[30943]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T22:15:48.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:48 smithi159 ceph-mon[35534]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T22:15:48.613 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:48 smithi172 ceph-mon[35461]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-11T22:15:50.432 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:50 smithi005 ceph-mon[30943]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T22:15:50.432 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:50.432 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:50.432 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:50.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:50 smithi159 ceph-mon[35534]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T22:15:50.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:50.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:50.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:50 smithi172 ceph-mon[35461]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-11T22:15:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:52.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:52 smithi005 ceph-mon[30943]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:15:52.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:52 smithi159 ceph-mon[35534]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:15:52.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:52 smithi172 ceph-mon[35461]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:15:53.740 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:53 smithi005 ceph-mon[30943]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:15:53.740 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:53 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:53.740 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:53 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:53.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:53 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:53.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:53 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:53.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:53 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.778+0000 7fad5f7fe640 1 -- 172.21.15.5:0/2143457124 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fad70062630 con 0x7fad580a2af0 2024-09-11T22:15:53.783 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2024-09-11T22:15:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.780+0000 7fad74cd9640 1 -- 172.21.15.5:0/2143457124 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fad580a2af0 msgr2=0x7fad580a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.780+0000 7fad74cd9640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fad580a2af0 0x7fad580a4fb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fad70101b30 tx=0x7fad6000a2d0 comp rx=0 tx=0).stop 2024-09-11T22:15:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.780+0000 7fad74cd9640 1 -- 172.21.15.5:0/2143457124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad70102690 msgr2=0x7fad70109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.780+0000 7fad74cd9640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad70102690 0x7fad70109fe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fad680098d0 tx=0x7fad6802f840 comp rx=0 tx=0).stop 2024-09-11T22:15:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.780+0000 7fad74cd9640 1 -- 172.21.15.5:0/2143457124 shutdown_connections 2024-09-11T22:15:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.780+0000 7fad74cd9640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fad580a2af0 0x7fad580a4fb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.780+0000 7fad74cd9640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad70102690 0x7fad70109fe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.780+0000 7fad74cd9640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad70101cd0 0x7fad70109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.780+0000 7fad74cd9640 1 --2- 172.21.15.5:0/2143457124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad70100ad0 0x7fad7010b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.780+0000 7fad74cd9640 1 -- 172.21.15.5:0/2143457124 >> 172.21.15.5:0/2143457124 conn(0x7fad700fc280 msgr2=0x7fad70105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:15:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.781+0000 7fad74cd9640 1 -- 172.21.15.5:0/2143457124 shutdown_connections 2024-09-11T22:15:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:15:53.781+0000 7fad74cd9640 1 -- 172.21.15.5:0/2143457124 wait complete. 2024-09-11T22:15:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:53 smithi159 ceph-mon[35534]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:15:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:53 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:53 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:53 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:53 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:53 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:53 smithi172 ceph-mon[35461]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-11T22:15:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:53 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:53 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:53 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:53 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:53 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:54.526 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T22:15:54.528 INFO:tasks.cephadm:Deploying osd.4 on smithi159 with /dev/vg_nvme/lv_3... 2024-09-11T22:15:54.528 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T22:15:54.784 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:15:54.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:54.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:54.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:54.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:54.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:54.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:15:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:55 smithi005 ceph-mon[30943]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:55 smithi005 ceph-mon[30943]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T22:15:55.820 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:15:55 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3[48410]: 2024-09-11T22:15:55.558+0000 7fbad6957640 -1 osd.3 0 log_to_monitors true 2024-09-11T22:15:56.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:55 smithi159 ceph-mon[35534]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:55 smithi159 ceph-mon[35534]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T22:15:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:55 smithi172 ceph-mon[35461]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:55 smithi172 ceph-mon[35461]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T22:15:57.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:56 smithi005 ceph-mon[30943]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T22:15:57.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:56 smithi005 ceph-mon[30943]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T22:15:57.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:56 smithi005 ceph-mon[30943]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:15:57.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:56 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:57.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:56 smithi005 ceph-mon[30943]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:15:57.072 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:56 smithi159 ceph-mon[35534]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T22:15:57.072 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:56 smithi159 ceph-mon[35534]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T22:15:57.072 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:56 smithi159 ceph-mon[35534]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:15:57.072 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:56 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:57.072 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:56 smithi159 ceph-mon[35534]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:15:57.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:56 smithi172 ceph-mon[35461]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T22:15:57.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:56 smithi172 ceph-mon[35461]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T22:15:57.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:56 smithi172 ceph-mon[35461]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:15:57.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:56 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:57.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:56 smithi172 ceph-mon[35461]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:15:57.375 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:15:57.400 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_3 2024-09-11T22:15:57.469 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3[48410]: 2024-09-11T22:15:57.214+0000 7fbad28da640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T22:15:57.469 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3[48410]: 2024-09-11T22:15:57.221+0000 7fbaca6fa640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:15:57.663 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:15:57.939 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:57.939 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T22:15:57.939 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:57.939 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:57.939 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' 2024-09-11T22:15:57.939 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] boot 2024-09-11T22:15:57.940 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: osdmap e24: 4 total, 4 up, 4 in 2024-09-11T22:15:57.940 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:57.940 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:57.940 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:57.940 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:57.940 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:57.940 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:57.940 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:57.940 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:57.940 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T22:15:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' 2024-09-11T22:15:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] boot 2024-09-11T22:15:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: osdmap e24: 4 total, 4 up, 4 in 2024-09-11T22:15:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 100 MiB used, 268 GiB / 268 GiB avail 2024-09-11T22:15:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T22:15:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835]' entity='osd.3' 2024-09-11T22:15:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: osd.3 [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] boot 2024-09-11T22:15:58.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: osdmap e24: 4 total, 4 up, 4 in 2024-09-11T22:15:58.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:15:58.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:58.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:58.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:58.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:15:58.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:58.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:58.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:15:58.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:15:58.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.128+0000 7fae13709640 1 -- 172.21.15.159:0/2150999784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae0c101d10 msgr2=0x7fae0c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:58.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.128+0000 7fae13709640 1 --2- 172.21.15.159:0/2150999784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae0c101d10 0x7fae0c102190 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009900 tx=0x7fadfc02a410 comp rx=0 tx=0).stop 2024-09-11T22:15:58.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.128+0000 7fae13709640 1 -- 172.21.15.159:0/2150999784 shutdown_connections 2024-09-11T22:15:58.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.128+0000 7fae13709640 1 --2- 172.21.15.159:0/2150999784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae0c1026d0 0x7fae0c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:58.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.128+0000 7fae13709640 1 --2- 172.21.15.159:0/2150999784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae0c101d10 0x7fae0c102190 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:58.145 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.128+0000 7fae13709640 1 --2- 172.21.15.159:0/2150999784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0c100b10 0x7fae0c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.128+0000 7fae13709640 1 -- 172.21.15.159:0/2150999784 >> 172.21.15.159:0/2150999784 conn(0x7fae0c0fc2a0 msgr2=0x7fae0c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.128+0000 7fae13709640 1 -- 172.21.15.159:0/2150999784 shutdown_connections 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.128+0000 7fae13709640 1 -- 172.21.15.159:0/2150999784 wait complete. 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae13709640 1 Processor -- start 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae13709640 1 -- start start 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae13709640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0c100b10 0x7fae0c104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae13709640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae0c101d10 0x7fae0c1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae13709640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae0c1026d0 0x7fae0c103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae13709640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c10c790 con 0x7fae0c1026d0 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae13709640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c10c610 con 0x7fae0c101d10 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae13709640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae0c10c910 con 0x7fae0c100b10 2024-09-11T22:15:58.146 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae10ca4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae0c101d10 0x7fae0c1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae10ca4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae0c101d10 0x7fae0c1030f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:32858/0 (socket says 172.21.15.159:32858) 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae10ca4640 1 -- 172.21.15.159:0/3445302831 learned_addr learned my addr 172.21.15.159:0/3445302831 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae11ca6640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae0c1026d0 0x7fae0c103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.129+0000 7fae114a5640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0c100b10 0x7fae0c104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae10ca4640 1 -- 172.21.15.159:0/3445302831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0c100b10 msgr2=0x7fae0c104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae10ca4640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0c100b10 0x7fae0c104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae10ca4640 1 -- 172.21.15.159:0/3445302831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae0c1026d0 msgr2=0x7fae0c103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae10ca4640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae0c1026d0 0x7fae0c103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae10ca4640 1 -- 172.21.15.159:0/3445302831 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0c103ef0 con 0x7fae0c101d10 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae114a5640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0c100b10 0x7fae0c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae10ca4640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae0c101d10 0x7fae0c1030f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009230 tx=0x7fadfc0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae027fc640 1 -- 172.21.15.159:0/3445302831 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc03f030 con 0x7fae0c101d10 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae027fc640 1 -- 172.21.15.159:0/3445302831 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadfc003ce0 con 0x7fae0c101d10 2024-09-11T22:15:58.147 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae13709640 1 -- 172.21.15.159:0/3445302831 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0c1a2060 con 0x7fae0c101d10 2024-09-11T22:15:58.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae027fc640 1 -- 172.21.15.159:0/3445302831 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadfc035070 con 0x7fae0c101d10 2024-09-11T22:15:58.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.130+0000 7fae13709640 1 -- 172.21.15.159:0/3445302831 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0c1a2510 con 0x7fae0c101d10 2024-09-11T22:15:58.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.131+0000 7fae13709640 1 -- 172.21.15.159:0/3445302831 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0c1a2740 con 0x7fae0c101d10 2024-09-11T22:15:58.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.132+0000 7fae027fc640 1 -- 172.21.15.159:0/3445302831 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fadfc042030 con 0x7fae0c101d10 2024-09-11T22:15:58.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.133+0000 7fae027fc640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fadf80a2b80 0x7fadf80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:15:58.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.133+0000 7fae027fc640 1 -- 172.21.15.159:0/3445302831 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7fadfc0ee310 con 0x7fae0c101d10 2024-09-11T22:15:58.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.133+0000 7fae114a5640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fadf80a2b80 0x7fadf80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:15:58.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.134+0000 7fae114a5640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fadf80a2b80 0x7fadf80a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae0c101b70 tx=0x7fae08005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:15:58.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.137+0000 7fae027fc640 1 -- 172.21.15.159:0/3445302831 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fadfc0bc470 con 0x7fae0c101d10 2024-09-11T22:15:58.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:15:58.321+0000 7fae13709640 1 -- 172.21.15.159:0/3445302831 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fae0c06af10 con 0x7fadf80a2b80 2024-09-11T22:15:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:59 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:15:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:59 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:15:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:59 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi005 2024-09-11T22:15:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:59 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-11T22:15:59.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:59 smithi005 ceph-mon[30943]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T22:15:59.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:15:59.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:15:59.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:15:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:59 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:15:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:59 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:15:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:59 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi005 2024-09-11T22:15:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:59 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-11T22:15:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:59 smithi159 ceph-mon[35534]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T22:15:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:15:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:15:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:15:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:15:59.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:59 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:15:59.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:59 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:15:59.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:59 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi005 2024-09-11T22:15:59.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:59 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-11T22:15:59.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:59 smithi172 ceph-mon[35461]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T22:15:59.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:15:59.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:15:59.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:15:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:00.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:00 smithi159 ceph-mon[35534]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:16:00.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:00 smithi159 ceph-mon[35534]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:00.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:00 smithi159 ceph-mon[35534]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T22:16:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:00 smithi005 ceph-mon[30943]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:16:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:00 smithi005 ceph-mon[30943]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:00 smithi005 ceph-mon[30943]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T22:16:00.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:00 smithi172 ceph-mon[35461]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:16:00.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:00 smithi172 ceph-mon[35461]: pgmap v96: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:00.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:00 smithi172 ceph-mon[35461]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T22:16:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:01 smithi159 ceph-mon[35534]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T22:16:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:01 smithi159 ceph-mon[35534]: from='client.? 172.21.15.159:0/4048903135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810e9588-d06b-493e-be95-d78591ab283a"}]: dispatch 2024-09-11T22:16:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:01 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810e9588-d06b-493e-be95-d78591ab283a"}]: dispatch 2024-09-11T22:16:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:01 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "810e9588-d06b-493e-be95-d78591ab283a"}]': finished 2024-09-11T22:16:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:01 smithi159 ceph-mon[35534]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T22:16:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:01 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:01.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:01 smithi005 ceph-mon[30943]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T22:16:01.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:01 smithi005 ceph-mon[30943]: from='client.? 172.21.15.159:0/4048903135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810e9588-d06b-493e-be95-d78591ab283a"}]: dispatch 2024-09-11T22:16:01.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:01 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810e9588-d06b-493e-be95-d78591ab283a"}]: dispatch 2024-09-11T22:16:01.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:01 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "810e9588-d06b-493e-be95-d78591ab283a"}]': finished 2024-09-11T22:16:01.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:01 smithi005 ceph-mon[30943]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T22:16:01.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:01 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:01.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:01 smithi172 ceph-mon[35461]: osdmap e27: 4 total, 4 up, 4 in 2024-09-11T22:16:01.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:01 smithi172 ceph-mon[35461]: from='client.? 172.21.15.159:0/4048903135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810e9588-d06b-493e-be95-d78591ab283a"}]: dispatch 2024-09-11T22:16:01.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:01 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "810e9588-d06b-493e-be95-d78591ab283a"}]: dispatch 2024-09-11T22:16:01.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:01 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "810e9588-d06b-493e-be95-d78591ab283a"}]': finished 2024-09-11T22:16:01.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:01 smithi172 ceph-mon[35461]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T22:16:01.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:01 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:02.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:02 smithi005 ceph-mon[30943]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:02.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:02 smithi005 ceph-mon[30943]: from='client.? 172.21.15.159:0/3348569462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:16:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:02 smithi159 ceph-mon[35534]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:02 smithi159 ceph-mon[35534]: from='client.? 172.21.15.159:0/3348569462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:16:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:02 smithi172 ceph-mon[35461]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:02 smithi172 ceph-mon[35461]: from='client.? 172.21.15.159:0/3348569462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:16:04.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:04 smithi005 ceph-mon[30943]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-11T22:16:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:04 smithi159 ceph-mon[35534]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-11T22:16:04.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:04 smithi172 ceph-mon[35461]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-11T22:16:06.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:06 smithi005 ceph-mon[30943]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T22:16:06.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:06 smithi159 ceph-mon[35534]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T22:16:06.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:06 smithi172 ceph-mon[35461]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T22:16:08.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:08 smithi159 ceph-mon[35534]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-11T22:16:08.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:08 smithi005 ceph-mon[30943]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-11T22:16:08.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:08 smithi172 ceph-mon[35461]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-11T22:16:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:10 smithi005 ceph-mon[30943]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T22:16:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T22:16:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:10 smithi159 ceph-mon[35534]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T22:16:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T22:16:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:10 smithi172 ceph-mon[35461]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T22:16:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T22:16:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:11.431 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:11 smithi159 ceph-mon[35534]: Deploying daemon osd.4 on smithi159 2024-09-11T22:16:11.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:11 smithi005 ceph-mon[30943]: Deploying daemon osd.4 on smithi159 2024-09-11T22:16:11.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:11 smithi172 ceph-mon[35461]: Deploying daemon osd.4 on smithi159 2024-09-11T22:16:12.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:12 smithi005 ceph-mon[30943]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:16:12.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:12 smithi159 ceph-mon[35534]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:16:12.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:12 smithi172 ceph-mon[35461]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T22:16:14.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:14 smithi159 ceph-mon[35534]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:16:14.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:14 smithi005 ceph-mon[30943]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:16:14.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:14 smithi172 ceph-mon[35461]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-11T22:16:16.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:16 smithi159 ceph-mon[35534]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:16.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:16.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:16.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:16 smithi005 ceph-mon[30943]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:16 smithi172 ceph-mon[35461]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:18.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:18 smithi005 ceph-mon[30943]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:18 smithi159 ceph-mon[35534]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:18 smithi172 ceph-mon[35461]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:19 smithi159 ceph-mon[35534]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:19 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:19 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:19 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:19 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:19 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:19.898 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 4 on host 'smithi159' 2024-09-11T22:16:19.898 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.897+0000 7fae027fc640 1 -- 172.21.15.159:0/3445302831 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fae0c06af10 con 0x7fadf80a2b80 2024-09-11T22:16:19.900 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 -- 172.21.15.159:0/3445302831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fadf80a2b80 msgr2=0x7fadf80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fadf80a2b80 0x7fadf80a5040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fae0c101b70 tx=0x7fae08005f90 comp rx=0 tx=0).stop 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 -- 172.21.15.159:0/3445302831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae0c101d10 msgr2=0x7fae0c1030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae0c101d10 0x7fae0c1030f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009230 tx=0x7fadfc0042f0 comp rx=0 tx=0).stop 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 -- 172.21.15.159:0/3445302831 shutdown_connections 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fadf80a2b80 0x7fadf80a5040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae0c1026d0 0x7fae0c103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae0c101d10 0x7fae0c1030f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 --2- 172.21.15.159:0/3445302831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0c100b10 0x7fae0c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 -- 172.21.15.159:0/3445302831 >> 172.21.15.159:0/3445302831 conn(0x7fae0c0fc2a0 msgr2=0x7fae0c06a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 -- 172.21.15.159:0/3445302831 shutdown_connections 2024-09-11T22:16:19.901 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:16:19.899+0000 7fae13709640 1 -- 172.21.15.159:0/3445302831 wait complete. 2024-09-11T22:16:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:19 smithi005 ceph-mon[30943]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:20.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:19 smithi172 ceph-mon[35461]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:20.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:19 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:20.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:19 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:20.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:19 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:20.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:19 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:20.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:19 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:20.597 DEBUG:teuthology.orchestra.run.smithi159:osd.4> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T22:16:20.600 INFO:tasks.cephadm:Deploying osd.5 on smithi172 with /dev/vg_nvme/lv_3... 2024-09-11T22:16:20.600 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-11T22:16:20.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:20 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:20.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:20 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:20.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:20 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:20.852 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:16:20.973 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:20 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:20.973 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:20 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:20.974 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:20 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:21.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:20 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:21.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:20 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:21.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:20 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:22.020 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:21 smithi159 ceph-mon[35534]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:22.021 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:21 smithi159 ceph-mon[35534]: from='osd.4 [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:16:22.021 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:21 smithi159 ceph-mon[35534]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:16:22.021 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:16:21 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:16:21.768+0000 7fcd458e7640 -1 osd.4 0 log_to_monitors true 2024-09-11T22:16:22.189 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:21 smithi172 ceph-mon[35461]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:22.189 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:21 smithi172 ceph-mon[35461]: from='osd.4 [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:16:22.189 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:21 smithi172 ceph-mon[35461]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:16:22.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:21 smithi005 ceph-mon[30943]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:22.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:21 smithi005 ceph-mon[30943]: from='osd.4 [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:16:22.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:21 smithi005 ceph-mon[30943]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T22:16:23.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:22 smithi159 ceph-mon[35534]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T22:16:23.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:22 smithi159 ceph-mon[35534]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T22:16:23.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:22 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:23.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:22 smithi159 ceph-mon[35534]: from='osd.4 [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:16:23.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:22 smithi159 ceph-mon[35534]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:16:23.080 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:16:22 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:16:22.913+0000 7fcd4186a640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T22:16:23.080 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:16:22 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:16:22.918+0000 7fcd3d692640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:16:23.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:22 smithi172 ceph-mon[35461]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T22:16:23.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:22 smithi172 ceph-mon[35461]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T22:16:23.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:22 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:23.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:22 smithi172 ceph-mon[35461]: from='osd.4 [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:16:23.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:22 smithi172 ceph-mon[35461]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:16:23.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:22 smithi005 ceph-mon[30943]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T22:16:23.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:22 smithi005 ceph-mon[30943]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T22:16:23.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:22 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:23.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:22 smithi005 ceph-mon[30943]: from='osd.4 [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:16:23.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:22 smithi005 ceph-mon[30943]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:16:23.503 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:16:23.530 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_3 2024-09-11T22:16:23.784 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:16:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:16:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T22:16:24.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:24.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:24.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:24.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:24.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:24.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:24.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:23 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.213+0000 7f404b454640 1 -- 172.21.15.172:0/981455415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4044104fd0 msgr2=0x7f40441073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:16:24.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.213+0000 7f404b454640 1 --2- 172.21.15.172:0/981455415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4044104fd0 0x7f40441073c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4034009900 tx=0x7f403402a410 comp rx=0 tx=0).stop 2024-09-11T22:16:24.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.213+0000 7f404b454640 1 -- 172.21.15.172:0/981455415 shutdown_connections 2024-09-11T22:16:24.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.213+0000 7f404b454640 1 --2- 172.21.15.172:0/981455415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4044107900 0x7f4044109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:24.236 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.213+0000 7f404b454640 1 --2- 172.21.15.172:0/981455415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4044104fd0 0x7f40441073c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:24.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.213+0000 7f404b454640 1 --2- 172.21.15.172:0/981455415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40441026a0 0x7f4044104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:24.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.213+0000 7f404b454640 1 -- 172.21.15.172:0/981455415 >> 172.21.15.172:0/981455415 conn(0x7f40440fc280 msgr2=0x7f40440fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:16:24.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.213+0000 7f404b454640 1 -- 172.21.15.172:0/981455415 shutdown_connections 2024-09-11T22:16:24.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.213+0000 7f404b454640 1 -- 172.21.15.172:0/981455415 wait complete. 2024-09-11T22:16:24.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f404b454640 1 Processor -- start 2024-09-11T22:16:24.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f404b454640 1 -- start start 2024-09-11T22:16:24.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f404b454640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40441026a0 0x7f4044197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:16:24.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f404b454640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4044104fd0 0x7f4044198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:16:24.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f404b454640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4044107900 0x7f404419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:16:24.237 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f404b454640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f404410d930 con 0x7f4044104fd0 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f404b454640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f404410d7b0 con 0x7f4044107900 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f404b454640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f404410dab0 con 0x7f40441026a0 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f40491f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40441026a0 0x7f4044197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f40491f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40441026a0 0x7f4044197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:49020/0 (socket says 172.21.15.172:49020) 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f40491f0640 1 -- 172.21.15.172:0/639647199 learned_addr learned my addr 172.21.15.172:0/639647199 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.214+0000 7f40499f1640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4044107900 0x7f404419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f40489ef640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4044104fd0 0x7f4044198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f40491f0640 1 -- 172.21.15.172:0/639647199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4044107900 msgr2=0x7f404419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f40491f0640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4044107900 0x7f404419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f40491f0640 1 -- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4044104fd0 msgr2=0x7f4044198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:16:24.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f40491f0640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4044104fd0 0x7f4044198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f40491f0640 1 -- 172.21.15.172:0/639647199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404419fc10 con 0x7f40441026a0 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f40489ef640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4044104fd0 0x7f4044198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f40491f0640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40441026a0 0x7f4044197fe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f404000b4a0 tx=0x7f404000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f403a7fc640 1 -- 172.21.15.172:0/639647199 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040014070 con 0x7f40441026a0 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f404b454640 1 -- 172.21.15.172:0/639647199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404419ff00 con 0x7f40441026a0 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f403a7fc640 1 -- 172.21.15.172:0/639647199 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4040010af0 con 0x7f40441026a0 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f403a7fc640 1 -- 172.21.15.172:0/639647199 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4040017d10 con 0x7f40441026a0 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.215+0000 7f404b454640 1 -- 172.21.15.172:0/639647199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40441a03d0 con 0x7f40441026a0 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.216+0000 7f404b454640 1 -- 172.21.15.172:0/639647199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40440fdd70 con 0x7f40441026a0 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.217+0000 7f403a7fc640 1 -- 172.21.15.172:0/639647199 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4040017070 con 0x7f40441026a0 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.218+0000 7f403a7fc640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f40300a2bb0 0x7f40300a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:16:24.239 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.218+0000 7f403a7fc640 1 -- 172.21.15.172:0/639647199 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4101+0+0 (secure 0 0 0) 0x7f40400cd330 con 0x7f40441026a0 2024-09-11T22:16:24.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.218+0000 7f40489ef640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f40300a2bb0 0x7f40300a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:16:24.240 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.219+0000 7f40489ef640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f40300a2bb0 0x7f40300a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f403402aa00 tx=0x7f4034000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:16:24.241 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.222+0000 7f403a7fc640 1 -- 172.21.15.172:0/639647199 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f404009b300 con 0x7f40441026a0 2024-09-11T22:16:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:16:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T22:16:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:24.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:24.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:24.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:24.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:24.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:24.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:23 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:24.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:16:24.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: osdmap e30: 5 total, 4 up, 5 in 2024-09-11T22:16:24.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:24.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:24.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:24.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:24.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:24.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:24.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:23 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:24.420 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:24.419+0000 7f404b454640 1 -- 172.21.15.172:0/639647199 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4044062630 con 0x7f40300a2bb0 2024-09-11T22:16:25.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:24 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:16:25.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:24 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:16:25.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:24 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi159 2024-09-11T22:16:25.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:24 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-11T22:16:25.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:24 smithi172 ceph-mon[35461]: osd.4 [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] boot 2024-09-11T22:16:25.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:24 smithi172 ceph-mon[35461]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T22:16:25.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:25.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:16:25.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:16:25.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:24 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:16:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:24 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:16:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:24 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi159 2024-09-11T22:16:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:24 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-11T22:16:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:24 smithi005 ceph-mon[30943]: osd.4 [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] boot 2024-09-11T22:16:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:24 smithi005 ceph-mon[30943]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T22:16:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:16:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:16:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:25.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:24 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:16:25.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:24 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:16:25.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:24 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi159 2024-09-11T22:16:25.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:24 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-11T22:16:25.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:24 smithi159 ceph-mon[35534]: osd.4 [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] boot 2024-09-11T22:16:25.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:24 smithi159 ceph-mon[35534]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T22:16:25.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:16:25.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:16:25.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:16:25.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:26.144 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:25 smithi172 ceph-mon[35461]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:26.144 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:25 smithi172 ceph-mon[35461]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:16:26.144 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:25 smithi172 ceph-mon[35461]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T22:16:26.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:25 smithi159 ceph-mon[35534]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:26.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:25 smithi159 ceph-mon[35534]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:16:26.230 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:25 smithi159 ceph-mon[35534]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T22:16:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:25 smithi005 ceph-mon[30943]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 357 GiB / 358 GiB avail 2024-09-11T22:16:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:25 smithi005 ceph-mon[30943]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:16:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:25 smithi005 ceph-mon[30943]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T22:16:28.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:27 smithi005 ceph-mon[30943]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:28.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:27 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/847528171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e0bf89e-0062-4032-8aec-6079003315a3"}]: dispatch 2024-09-11T22:16:28.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:27 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e0bf89e-0062-4032-8aec-6079003315a3"}]: dispatch 2024-09-11T22:16:28.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:27 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e0bf89e-0062-4032-8aec-6079003315a3"}]': finished 2024-09-11T22:16:28.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:27 smithi005 ceph-mon[30943]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T22:16:28.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:27 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:28.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:27 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/161084451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:16:28.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:27 smithi159 ceph-mon[35534]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:28.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:27 smithi159 ceph-mon[35534]: from='client.? 172.21.15.172:0/847528171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e0bf89e-0062-4032-8aec-6079003315a3"}]: dispatch 2024-09-11T22:16:28.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:27 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e0bf89e-0062-4032-8aec-6079003315a3"}]: dispatch 2024-09-11T22:16:28.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:27 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e0bf89e-0062-4032-8aec-6079003315a3"}]': finished 2024-09-11T22:16:28.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:27 smithi159 ceph-mon[35534]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T22:16:28.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:27 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:28.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:27 smithi159 ceph-mon[35534]: from='client.? 172.21.15.172:0/161084451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:16:28.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:27 smithi172 ceph-mon[35461]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:28.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:27 smithi172 ceph-mon[35461]: from='client.? 172.21.15.172:0/847528171' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e0bf89e-0062-4032-8aec-6079003315a3"}]: dispatch 2024-09-11T22:16:28.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:27 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0e0bf89e-0062-4032-8aec-6079003315a3"}]: dispatch 2024-09-11T22:16:28.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:27 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0e0bf89e-0062-4032-8aec-6079003315a3"}]': finished 2024-09-11T22:16:28.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:27 smithi172 ceph-mon[35461]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T22:16:28.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:27 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:28.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:27 smithi172 ceph-mon[35461]: from='client.? 172.21.15.172:0/161084451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:16:30.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:29 smithi005 ceph-mon[30943]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:30.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:29 smithi159 ceph-mon[35534]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:30.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:29 smithi172 ceph-mon[35461]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:32.311 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:31 smithi172 ceph-mon[35461]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:32.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:31 smithi005 ceph-mon[30943]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:32.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:31 smithi159 ceph-mon[35534]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:33 smithi172 ceph-mon[35461]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:34.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:33 smithi005 ceph-mon[30943]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:33 smithi159 ceph-mon[35534]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:36.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:35 smithi172 ceph-mon[35461]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:36.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:35 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T22:16:36.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:35 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:36.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:35 smithi172 ceph-mon[35461]: Deploying daemon osd.5 on smithi172 2024-09-11T22:16:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:35 smithi005 ceph-mon[30943]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:35 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T22:16:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:35 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:35 smithi005 ceph-mon[30943]: Deploying daemon osd.5 on smithi172 2024-09-11T22:16:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:35 smithi159 ceph-mon[35534]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:35 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T22:16:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:35 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:35 smithi159 ceph-mon[35534]: Deploying daemon osd.5 on smithi172 2024-09-11T22:16:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:37 smithi172 ceph-mon[35461]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:38.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:37 smithi005 ceph-mon[30943]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:38.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:37 smithi159 ceph-mon[35534]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:40.249 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:39 smithi172 ceph-mon[35461]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:39 smithi005 ceph-mon[30943]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:40.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:39 smithi159 ceph-mon[35534]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:42.221 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:41 smithi172 ceph-mon[35461]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:42.221 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:42.221 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:42.221 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:42.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:41 smithi005 ceph-mon[30943]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:42.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:42.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:42.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:41 smithi159 ceph-mon[35534]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:44.082 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:43 smithi172 ceph-mon[35461]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:44.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:43 smithi005 ceph-mon[30943]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:44.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:43 smithi159 ceph-mon[35534]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:45.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:45.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:45.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:45.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:45.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:45.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:45 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:45.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:45 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:45.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:45 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:45.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:45 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:45.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:45 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:45.594 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:45 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:45.594 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:45 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:45.594 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:45 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:45.594 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:45 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:45.594 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:45 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:45.635 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.633+0000 7f403a7fc640 1 -- 172.21.15.172:0/639647199 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4044062630 con 0x7f40300a2bb0 2024-09-11T22:16:45.635 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 5 on host 'smithi172' 2024-09-11T22:16:45.637 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 -- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f40300a2bb0 msgr2=0x7f40300a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:16:45.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f40300a2bb0 0x7f40300a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f403402aa00 tx=0x7f4034000bc0 comp rx=0 tx=0).stop 2024-09-11T22:16:45.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 -- 172.21.15.172:0/639647199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40441026a0 msgr2=0x7f4044197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:16:45.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40441026a0 0x7f4044197fe0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f404000b4a0 tx=0x7f404000b960 comp rx=0 tx=0).stop 2024-09-11T22:16:45.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 -- 172.21.15.172:0/639647199 shutdown_connections 2024-09-11T22:16:45.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f40300a2bb0 0x7f40300a5070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:45.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4044107900 0x7f404419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:45.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4044104fd0 0x7f4044198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:45.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 --2- 172.21.15.172:0/639647199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40441026a0 0x7f4044197fe0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:45.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 -- 172.21.15.172:0/639647199 >> 172.21.15.172:0/639647199 conn(0x7f40440fc280 msgr2=0x7f4044103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:16:45.638 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 -- 172.21.15.172:0/639647199 shutdown_connections 2024-09-11T22:16:45.639 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:16:45.636+0000 7f404b454640 1 -- 172.21.15.172:0/639647199 wait complete. 2024-09-11T22:16:46.302 DEBUG:teuthology.orchestra.run.smithi172:osd.5> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T22:16:46.305 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2024-09-11T22:16:46.305 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T22:16:46.472 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:46 smithi172 ceph-mon[35461]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:46.472 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:46 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:46.472 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:46 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:46.472 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:46 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:46.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:46 smithi005 ceph-mon[30943]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:46.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:46 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:46.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:46 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:46.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:46 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:46.573 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:16:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:46 smithi159 ceph-mon[35534]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:46 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:16:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:46 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:46 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:48.111 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:16:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:16:47.750+0000 7f6e11e9d640 -1 osd.5 0 log_to_monitors true 2024-09-11T22:16:48.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:48 smithi005 ceph-mon[30943]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:48.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:48 smithi005 ceph-mon[30943]: from='osd.5 [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:16:48.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:48 smithi005 ceph-mon[30943]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:16:48.517 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:48 smithi172 ceph-mon[35461]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:48.517 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:48 smithi172 ceph-mon[35461]: from='osd.5 [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:16:48.517 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:48 smithi172 ceph-mon[35461]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:16:48.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:48 smithi159 ceph-mon[35534]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:48.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:48 smithi159 ceph-mon[35534]: from='osd.5 [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:16:48.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:48 smithi159 ceph-mon[35534]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T22:16:49.235 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:16:49.262 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2024-09-11T22:16:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:16:49.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:49 smithi005 ceph-mon[30943]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T22:16:49.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:49 smithi005 ceph-mon[30943]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T22:16:49.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:49 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:49.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:49 smithi005 ceph-mon[30943]: from='osd.5 [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:16:49.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:49 smithi005 ceph-mon[30943]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:16:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:49 smithi159 ceph-mon[35534]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T22:16:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:49 smithi159 ceph-mon[35534]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T22:16:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:49 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:49 smithi159 ceph-mon[35534]: from='osd.5 [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:16:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:49 smithi159 ceph-mon[35534]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:16:49.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:49 smithi172 ceph-mon[35461]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T22:16:49.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:49 smithi172 ceph-mon[35461]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T22:16:49.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:49 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:49.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:49 smithi172 ceph-mon[35461]: from='osd.5 [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:16:49.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:49 smithi172 ceph-mon[35461]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:16:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.037+0000 7faf2696b640 1 -- 172.21.15.5:0/3812011712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20076220 msgr2=0x7faf20074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:16:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.037+0000 7faf2696b640 1 --2- 172.21.15.5:0/3812011712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20076220 0x7faf20074680 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faf10009900 tx=0x7faf1002a410 comp rx=0 tx=0).stop 2024-09-11T22:16:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.037+0000 7faf2696b640 1 -- 172.21.15.5:0/3812011712 shutdown_connections 2024-09-11T22:16:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.037+0000 7faf2696b640 1 --2- 172.21.15.5:0/3812011712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf20109a80 0x7faf2010bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 --2- 172.21.15.5:0/3812011712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf20074c50 0x7faf200750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 --2- 172.21.15.5:0/3812011712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20076220 0x7faf20074680 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 -- 172.21.15.5:0/3812011712 >> 172.21.15.5:0/3812011712 conn(0x7faf200fd390 msgr2=0x7faf200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:16:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 -- 172.21.15.5:0/3812011712 shutdown_connections 2024-09-11T22:16:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 -- 172.21.15.5:0/3812011712 wait complete. 2024-09-11T22:16:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 Processor -- start 2024-09-11T22:16:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 -- start start 2024-09-11T22:16:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20074c50 0x7faf2019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:16:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf20076220 0x7faf2019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:16:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf20109a80 0x7faf201a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:16:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf2010fb00 con 0x7faf20074c50 2024-09-11T22:16:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faf2010f980 con 0x7faf20076220 2024-09-11T22:16:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.038+0000 7faf2696b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faf2010fc80 con 0x7faf20109a80 2024-09-11T22:16:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20074c50 0x7faf2019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:16:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20074c50 0x7faf2019a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57630/0 (socket says 172.21.15.5:57630) 2024-09-11T22:16:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1ffff640 1 -- 172.21.15.5:0/4133120115 learned_addr learned my addr 172.21.15.5:0/4133120115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:16:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf24f08640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf20109a80 0x7faf201a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:16:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1ffff640 1 -- 172.21.15.5:0/4133120115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf20109a80 msgr2=0x7faf201a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:16:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1ffff640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf20109a80 0x7faf201a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1ffff640 1 -- 172.21.15.5:0/4133120115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf20076220 msgr2=0x7faf2019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1ffff640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf20076220 0x7faf2019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1ffff640 1 -- 172.21.15.5:0/4133120115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf201a1d80 con 0x7faf20074c50 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1ffff640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20074c50 0x7faf2019a150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faf10031610 tx=0x7faf10004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1d7fa640 1 -- 172.21.15.5:0/4133120115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1003f030 con 0x7faf20074c50 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1d7fa640 1 -- 172.21.15.5:0/4133120115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf10003ad0 con 0x7faf20074c50 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf2696b640 1 -- 172.21.15.5:0/4133120115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf201a2010 con 0x7faf20074c50 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf1d7fa640 1 -- 172.21.15.5:0/4133120115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10035070 con 0x7faf20074c50 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.039+0000 7faf2696b640 1 -- 172.21.15.5:0/4133120115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf201a2570 con 0x7faf20074c50 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.040+0000 7faf2696b640 1 -- 172.21.15.5:0/4133120115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf20108800 con 0x7faf20074c50 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.042+0000 7faf1d7fa640 1 -- 172.21.15.5:0/4133120115 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7faf10042030 con 0x7faf20074c50 2024-09-11T22:16:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.042+0000 7faf1d7fa640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faf080a2bb0 0x7faf080a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:16:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.042+0000 7faf1d7fa640 1 -- 172.21.15.5:0/4133120115 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4253+0+0 (secure 0 0 0) 0x7faf100ee980 con 0x7faf20074c50 2024-09-11T22:16:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.042+0000 7faf1f7fe640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faf080a2bb0 0x7faf080a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:16:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.043+0000 7faf1f7fe640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faf080a2bb0 0x7faf080a5070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf140058c0 tx=0x7faf14005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:16:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.046+0000 7faf1d7fa640 1 -- 172.21.15.5:0/4133120115 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf100bc8c0 con 0x7faf20074c50 2024-09-11T22:16:50.110 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:16:49 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:16:49.673+0000 7f6e0de20640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T22:16:50.110 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:16:49 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:16:49.681+0000 7f6e05c40640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:16:50.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:16:50.231+0000 7faf2696b640 1 -- 172.21.15.5:0/4133120115 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7faf20062630 con 0x7faf080a2bb0 2024-09-11T22:16:50.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:50.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:16:50.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi172 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='osd.5 ' entity='osd.5' 2024-09-11T22:16:50.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:16:50.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:16:50.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:50.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: osd.5 [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] boot 2024-09-11T22:16:50.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T22:16:50.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:50 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:50.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:50.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:16:50.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T22:16:50.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi172 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='osd.5 ' entity='osd.5' 2024-09-11T22:16:50.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:16:50.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:16:50.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:50.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: osd.5 [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] boot 2024-09-11T22:16:50.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T22:16:50.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:50 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 447 GiB / 447 GiB avail 2024-09-11T22:16:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:16:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T22:16:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi172 2024-09-11T22:16:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi172 to 8544M 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='osd.5 ' entity='osd.5' 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: osd.5 [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] boot 2024-09-11T22:16:50.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T22:16:50.612 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:50 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:16:51.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:51 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:16:51.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:51 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:16:51.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:51 smithi005 ceph-mon[30943]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:16:51.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:51 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:16:51.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:51 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:16:51.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:51 smithi159 ceph-mon[35534]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:16:51.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:51 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:16:51.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:51 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:16:51.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:51 smithi172 ceph-mon[35461]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:16:52.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:52 smithi005 ceph-mon[30943]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:16:52.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:52 smithi005 ceph-mon[30943]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T22:16:52.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:52 smithi159 ceph-mon[35534]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:16:52.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:52 smithi159 ceph-mon[35534]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T22:16:52.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:52 smithi172 ceph-mon[35461]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:16:52.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:52 smithi172 ceph-mon[35461]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T22:16:53.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:53 smithi005 ceph-mon[30943]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T22:16:53.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:53 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3501359335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f11a15a6-a7a8-4a42-ba15-20867f32d7ec"}]: dispatch 2024-09-11T22:16:53.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:53 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3501359335' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f11a15a6-a7a8-4a42-ba15-20867f32d7ec"}]': finished 2024-09-11T22:16:53.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:53 smithi005 ceph-mon[30943]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T22:16:53.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:53 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:16:53.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:53 smithi159 ceph-mon[35534]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T22:16:53.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:53 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3501359335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f11a15a6-a7a8-4a42-ba15-20867f32d7ec"}]: dispatch 2024-09-11T22:16:53.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:53 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3501359335' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f11a15a6-a7a8-4a42-ba15-20867f32d7ec"}]': finished 2024-09-11T22:16:53.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:53 smithi159 ceph-mon[35534]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T22:16:53.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:53 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:16:53.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:53 smithi172 ceph-mon[35461]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T22:16:53.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:53 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3501359335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f11a15a6-a7a8-4a42-ba15-20867f32d7ec"}]: dispatch 2024-09-11T22:16:53.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:53 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3501359335' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f11a15a6-a7a8-4a42-ba15-20867f32d7ec"}]': finished 2024-09-11T22:16:53.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:53 smithi172 ceph-mon[35461]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T22:16:53.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:53 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:16:54.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:54 smithi005 ceph-mon[30943]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:16:54.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:54 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/361387962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:16:54.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:54 smithi005 ceph-mon[30943]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T22:16:54.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:16:54.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:54 smithi159 ceph-mon[35534]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:16:54.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:54 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/361387962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:16:54.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:54 smithi159 ceph-mon[35534]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T22:16:54.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:16:54.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:54 smithi172 ceph-mon[35461]: pgmap v136: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:16:54.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:54 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/361387962' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:16:54.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:54 smithi172 ceph-mon[35461]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T22:16:54.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:16:56.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:56 smithi005 ceph-mon[30943]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:16:56.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:56 smithi159 ceph-mon[35534]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:16:56.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:56 smithi172 ceph-mon[35461]: pgmap v139: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:16:58.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:16:58 smithi005 ceph-mon[30943]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-11T22:16:58.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:16:58 smithi159 ceph-mon[35534]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-11T22:16:58.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:16:58 smithi172 ceph-mon[35461]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-09-11T22:17:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:00 smithi005 ceph-mon[30943]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-11T22:17:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:00 smithi159 ceph-mon[35534]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-11T22:17:00.612 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:00 smithi172 ceph-mon[35461]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-11T22:17:02.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:02 smithi005 ceph-mon[30943]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:02.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:02 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T22:17:02.458 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:02 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:02 smithi159 ceph-mon[35534]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:02 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T22:17:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:02 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:02 smithi172 ceph-mon[35461]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:02 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T22:17:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:02 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:03.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:03 smithi005 ceph-mon[30943]: Deploying daemon osd.6 on smithi005 2024-09-11T22:17:03.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:03 smithi159 ceph-mon[35534]: Deploying daemon osd.6 on smithi005 2024-09-11T22:17:03.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:03 smithi172 ceph-mon[35461]: Deploying daemon osd.6 on smithi005 2024-09-11T22:17:04.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:04 smithi005 ceph-mon[30943]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:04 smithi159 ceph-mon[35534]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:04.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:04 smithi172 ceph-mon[35461]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:06.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:06 smithi005 ceph-mon[30943]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:06.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:06 smithi159 ceph-mon[35534]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:06.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:06 smithi172 ceph-mon[35461]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:08.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:08 smithi005 ceph-mon[30943]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:08.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:08.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:08.573 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:08.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:08 smithi159 ceph-mon[35534]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:08.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:08.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:08.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:08.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:08 smithi172 ceph-mon[35461]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:08.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:08.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:08.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:10 smithi005 ceph-mon[30943]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:10 smithi159 ceph-mon[35534]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:10 smithi172 ceph-mon[35461]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:11.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.464+0000 7faf1d7fa640 1 -- 172.21.15.5:0/4133120115 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faf20062630 con 0x7faf080a2bb0 2024-09-11T22:17:11.466 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2024-09-11T22:17:11.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 -- 172.21.15.5:0/4133120115 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faf080a2bb0 msgr2=0x7faf080a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:17:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faf080a2bb0 0x7faf080a5070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faf140058c0 tx=0x7faf14005850 comp rx=0 tx=0).stop 2024-09-11T22:17:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 -- 172.21.15.5:0/4133120115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20074c50 msgr2=0x7faf2019a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:17:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20074c50 0x7faf2019a150 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faf10031610 tx=0x7faf10004270 comp rx=0 tx=0).stop 2024-09-11T22:17:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 -- 172.21.15.5:0/4133120115 shutdown_connections 2024-09-11T22:17:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faf080a2bb0 0x7faf080a5070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf20109a80 0x7faf201a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:11.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf20076220 0x7faf2019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:11.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 --2- 172.21.15.5:0/4133120115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20074c50 0x7faf2019a150 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:11.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 -- 172.21.15.5:0/4133120115 >> 172.21.15.5:0/4133120115 conn(0x7faf200fd390 msgr2=0x7faf201059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:17:11.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 -- 172.21.15.5:0/4133120115 shutdown_connections 2024-09-11T22:17:11.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:17:11.467+0000 7faf2696b640 1 -- 172.21.15.5:0/4133120115 wait complete. 2024-09-11T22:17:12.213 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T22:17:12.215 INFO:tasks.cephadm:Deploying osd.7 on smithi159 with /dev/vg_nvme/lv_2... 2024-09-11T22:17:12.216 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T22:17:12.434 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:12 smithi005 ceph-mon[30943]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:12.435 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:12 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:12.435 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:12 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:12.435 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:12 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:12.473 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:17:12.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:12 smithi159 ceph-mon[35534]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:12.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:12 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:12.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:12 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:12.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:12 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:12.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:12 smithi172 ceph-mon[35461]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:12.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:12 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:12.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:12 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:12.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:12 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:13.311 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:17:12 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:17:12.970+0000 7fe597fe0640 -1 osd.6 0 log_to_monitors true 2024-09-11T22:17:13.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:13 smithi005 ceph-mon[30943]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T22:17:13.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:13 smithi159 ceph-mon[35534]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T22:17:13.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:13 smithi172 ceph-mon[35461]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T22:17:14.506 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:14 smithi159 ceph-mon[35534]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:14.507 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:14 smithi159 ceph-mon[35534]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T22:17:14.507 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:14 smithi159 ceph-mon[35534]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T22:17:14.507 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:14 smithi159 ceph-mon[35534]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:17:14.507 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:14 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:14.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:14 smithi172 ceph-mon[35461]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:14.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:14 smithi172 ceph-mon[35461]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T22:17:14.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:14 smithi172 ceph-mon[35461]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T22:17:14.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:14 smithi172 ceph-mon[35461]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:17:14.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:14 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:14.760 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:17:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:17:14.330+0000 7fe593f63640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T22:17:14.761 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:17:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:17:14.341+0000 7fe58fd8b640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:17:14.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:14 smithi005 ceph-mon[30943]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:14.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:14 smithi005 ceph-mon[30943]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T22:17:14.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:14 smithi005 ceph-mon[30943]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T22:17:14.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:14 smithi005 ceph-mon[30943]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:17:14.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:14 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:15.310 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:17:15.336 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_2 2024-09-11T22:17:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:17:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T22:17:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi005 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:15.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.581 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:17:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:17:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T22:17:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi005 2024-09-11T22:17:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:15.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:15.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:15.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-11T22:17:15.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:15.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:15.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:17:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: osdmap e42: 7 total, 6 up, 7 in 2024-09-11T22:17:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-09-11T22:17:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi005 2024-09-11T22:17:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:15.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:15.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:15.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-11T22:17:15.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:15.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:15.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:15.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:16.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.043+0000 7feee93fd640 1 -- 172.21.15.159:0/4036967599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4100b10 msgr2=0x7feee4100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:17:16.085 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.043+0000 7feee93fd640 1 --2- 172.21.15.159:0/4036967599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4100b10 0x7feee4100f10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feed4009900 tx=0x7feed402a410 comp rx=0 tx=0).stop 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.043+0000 7feee93fd640 1 -- 172.21.15.159:0/4036967599 shutdown_connections 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.043+0000 7feee93fd640 1 --2- 172.21.15.159:0/4036967599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee41026d0 0x7feee4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.043+0000 7feee93fd640 1 --2- 172.21.15.159:0/4036967599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee4101d10 0x7feee4102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.043+0000 7feee93fd640 1 --2- 172.21.15.159:0/4036967599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4100b10 0x7feee4100f10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.043+0000 7feee93fd640 1 -- 172.21.15.159:0/4036967599 >> 172.21.15.159:0/4036967599 conn(0x7feee40fc2a0 msgr2=0x7feee40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.043+0000 7feee93fd640 1 -- 172.21.15.159:0/4036967599 shutdown_connections 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.043+0000 7feee93fd640 1 -- 172.21.15.159:0/4036967599 wait complete. 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee93fd640 1 Processor -- start 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee93fd640 1 -- start start 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee93fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee4100b10 0x7feee4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:17:16.086 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee93fd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4101d10 0x7feee41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee93fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee41026d0 0x7feee419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee93fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feee410c790 con 0x7feee4100b10 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee93fd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feee410c610 con 0x7feee4101d10 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee93fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feee410c910 con 0x7feee41026d0 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee27fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4101d10 0x7feee41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee27fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4101d10 0x7feee41984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:60768/0 (socket says 172.21.15.159:60768) 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.044+0000 7feee27fc640 1 -- 172.21.15.159:0/94634120 learned_addr learned my addr 172.21.15.159:0/94634120 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee37fe640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee41026d0 0x7feee419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee27fc640 1 -- 172.21.15.159:0/94634120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee41026d0 msgr2=0x7feee419f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee27fc640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee41026d0 0x7feee419f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:16.087 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee2ffd640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee4100b10 0x7feee4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee27fc640 1 -- 172.21.15.159:0/94634120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee4100b10 msgr2=0x7feee4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee27fc640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee4100b10 0x7feee4197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee27fc640 1 -- 172.21.15.159:0/94634120 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee419fbb0 con 0x7feee4101d10 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee37fe640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee41026d0 0x7feee419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee27fc640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4101d10 0x7feee41984c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feed800d770 tx=0x7feed800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feecbfff640 1 -- 172.21.15.159:0/94634120 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8017070 con 0x7feee4101d10 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feecbfff640 1 -- 172.21.15.159:0/94634120 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feed8005500 con 0x7feee4101d10 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee93fd640 1 -- 172.21.15.159:0/94634120 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee419fea0 con 0x7feee4101d10 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feecbfff640 1 -- 172.21.15.159:0/94634120 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8004e00 con 0x7feee4101d10 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.045+0000 7feee93fd640 1 -- 172.21.15.159:0/94634120 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee41a0300 con 0x7feee4101d10 2024-09-11T22:17:16.088 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.046+0000 7feee93fd640 1 -- 172.21.15.159:0/94634120 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee41a0530 con 0x7feee4101d10 2024-09-11T22:17:16.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.047+0000 7feecbfff640 1 -- 172.21.15.159:0/94634120 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7feed8004fa0 con 0x7feee4101d10 2024-09-11T22:17:16.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.048+0000 7feecbfff640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7feecc0a2bb0 0x7feecc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:17:16.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.048+0000 7feecbfff640 1 -- 172.21.15.159:0/94634120 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7feed80cddb0 con 0x7feee4101d10 2024-09-11T22:17:16.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.048+0000 7feee2ffd640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7feecc0a2bb0 0x7feecc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:17:16.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.049+0000 7feee2ffd640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7feecc0a2bb0 0x7feecc0a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feed4031610 tx=0x7feed4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:17:16.089 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.052+0000 7feecbfff640 1 -- 172.21.15.159:0/94634120 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feed809ba20 con 0x7feee4101d10 2024-09-11T22:17:16.227 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:16.226+0000 7feee93fd640 1 -- 172.21.15.159:0/94634120 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7feee406af50 con 0x7feecc0a2bb0 2024-09-11T22:17:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:16 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:17:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:16 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:17:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:16 smithi159 ceph-mon[35534]: osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] boot 2024-09-11T22:17:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:16 smithi159 ceph-mon[35534]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T22:17:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:16 smithi159 ceph-mon[35534]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:17:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:17:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:17:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:16 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:17:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:16 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:17:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:16 smithi172 ceph-mon[35461]: osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] boot 2024-09-11T22:17:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:16 smithi172 ceph-mon[35461]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T22:17:16.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:16.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:16 smithi172 ceph-mon[35461]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:17:16.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:17:16.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:17:16.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:16.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:16 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:17:16.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:16 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:17:16.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:16.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:16 smithi005 ceph-mon[30943]: osd.6 [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] boot 2024-09-11T22:17:16.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:16 smithi005 ceph-mon[30943]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T22:17:16.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:17:16.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:16 smithi005 ceph-mon[30943]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:17:16.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:17:16.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:17:16.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:17.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:17 smithi172 ceph-mon[35461]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T22:17:17.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:17 smithi172 ceph-mon[35461]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:17.618 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:17 smithi159 ceph-mon[35534]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T22:17:17.618 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:17 smithi159 ceph-mon[35534]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:17.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:17 smithi005 ceph-mon[30943]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T22:17:17.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:17 smithi005 ceph-mon[30943]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:19 smithi005 ceph-mon[30943]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:19 smithi005 ceph-mon[30943]: from='client.? 172.21.15.159:0/2895057963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7d70b5c-cb1b-4e19-b185-595a94380ee6"}]: dispatch 2024-09-11T22:17:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:19 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7d70b5c-cb1b-4e19-b185-595a94380ee6"}]: dispatch 2024-09-11T22:17:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:19 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7d70b5c-cb1b-4e19-b185-595a94380ee6"}]': finished 2024-09-11T22:17:19.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:19 smithi005 ceph-mon[30943]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T22:17:19.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:19.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:19 smithi005 ceph-mon[30943]: from='client.? 172.21.15.159:0/3312577637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:17:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:19 smithi159 ceph-mon[35534]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:19 smithi159 ceph-mon[35534]: from='client.? 172.21.15.159:0/2895057963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7d70b5c-cb1b-4e19-b185-595a94380ee6"}]: dispatch 2024-09-11T22:17:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:19 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7d70b5c-cb1b-4e19-b185-595a94380ee6"}]: dispatch 2024-09-11T22:17:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:19 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7d70b5c-cb1b-4e19-b185-595a94380ee6"}]': finished 2024-09-11T22:17:19.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:19 smithi159 ceph-mon[35534]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T22:17:19.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:19 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:19.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:19 smithi159 ceph-mon[35534]: from='client.? 172.21.15.159:0/3312577637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:17:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:19 smithi172 ceph-mon[35461]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:19 smithi172 ceph-mon[35461]: from='client.? 172.21.15.159:0/2895057963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7d70b5c-cb1b-4e19-b185-595a94380ee6"}]: dispatch 2024-09-11T22:17:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:19 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7d70b5c-cb1b-4e19-b185-595a94380ee6"}]: dispatch 2024-09-11T22:17:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:19 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7d70b5c-cb1b-4e19-b185-595a94380ee6"}]': finished 2024-09-11T22:17:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:19 smithi172 ceph-mon[35461]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T22:17:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:19 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:19 smithi172 ceph-mon[35461]: from='client.? 172.21.15.159:0/3312577637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:17:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:21 smithi005 ceph-mon[30943]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:21 smithi159 ceph-mon[35534]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:21.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:21 smithi172 ceph-mon[35461]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:23 smithi005 ceph-mon[30943]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:23 smithi159 ceph-mon[35534]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:23 smithi172 ceph-mon[35461]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:25.492 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:25 smithi159 ceph-mon[35534]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:25 smithi005 ceph-mon[30943]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:25 smithi172 ceph-mon[35461]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:27 smithi159 ceph-mon[35534]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:27 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T22:17:27.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:27 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:27.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:27 smithi159 ceph-mon[35534]: Deploying daemon osd.7 on smithi159 2024-09-11T22:17:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:27 smithi005 ceph-mon[30943]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:27 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T22:17:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:27 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:27 smithi005 ceph-mon[30943]: Deploying daemon osd.7 on smithi159 2024-09-11T22:17:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:27 smithi172 ceph-mon[35461]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:27 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T22:17:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:27 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:27 smithi172 ceph-mon[35461]: Deploying daemon osd.7 on smithi159 2024-09-11T22:17:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:29 smithi159 ceph-mon[35534]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:29 smithi005 ceph-mon[30943]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:29 smithi172 ceph-mon[35461]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:31.798 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:31 smithi159 ceph-mon[35534]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:31 smithi005 ceph-mon[30943]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:31 smithi172 ceph-mon[35461]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:33.807 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:33 smithi159 ceph-mon[35534]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:33.807 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:33.807 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:33.807 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:33 smithi005 ceph-mon[30943]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:33 smithi172 ceph-mon[35461]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:35.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:35 smithi159 ceph-mon[35534]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:35 smithi005 ceph-mon[30943]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:35 smithi172 ceph-mon[35461]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:36.613 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.612+0000 7feecbfff640 1 -- 172.21.15.159:0/94634120 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7feee406af50 con 0x7feecc0a2bb0 2024-09-11T22:17:36.613 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 7 on host 'smithi159' 2024-09-11T22:17:36.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 -- 172.21.15.159:0/94634120 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7feecc0a2bb0 msgr2=0x7feecc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:17:36.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7feecc0a2bb0 0x7feecc0a5070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feed4031610 tx=0x7feed4000bc0 comp rx=0 tx=0).stop 2024-09-11T22:17:36.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 -- 172.21.15.159:0/94634120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4101d10 msgr2=0x7feee41984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:17:36.615 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4101d10 0x7feee41984c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feed800d770 tx=0x7feed800dc30 comp rx=0 tx=0).stop 2024-09-11T22:17:36.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 -- 172.21.15.159:0/94634120 shutdown_connections 2024-09-11T22:17:36.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7feecc0a2bb0 0x7feecc0a5070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:36.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feee41026d0 0x7feee419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:36.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feee4101d10 0x7feee41984c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:36.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 --2- 172.21.15.159:0/94634120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feee4100b10 0x7feee4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:36.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 -- 172.21.15.159:0/94634120 >> 172.21.15.159:0/94634120 conn(0x7feee40fc2a0 msgr2=0x7feee406a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:17:36.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 -- 172.21.15.159:0/94634120 shutdown_connections 2024-09-11T22:17:36.616 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:17:36.615+0000 7feee93fd640 1 -- 172.21.15.159:0/94634120 wait complete. 2024-09-11T22:17:37.294 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.295 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.295 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:37.295 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:37.295 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.295 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:37.295 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.295 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:36 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:37.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:36 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.322 DEBUG:teuthology.orchestra.run.smithi159:osd.7> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T22:17:37.326 INFO:tasks.cephadm:Deploying osd.8 on smithi172 with /dev/vg_nvme/lv_2... 2024-09-11T22:17:37.326 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-11T22:17:37.354 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.354 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.354 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:37.354 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:37.354 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.354 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:37.355 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.355 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:36 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:37.581 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:17:37.952 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:37 smithi172 ceph-mon[35461]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:38.201 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:37 smithi159 ceph-mon[35534]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:38.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:37 smithi005 ceph-mon[30943]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:38.976 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:38 smithi172 ceph-mon[35461]: from='osd.7 [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:17:38.976 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:38 smithi172 ceph-mon[35461]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:17:39.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:38 smithi159 ceph-mon[35534]: from='osd.7 [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:17:39.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:38 smithi159 ceph-mon[35534]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:17:39.079 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:17:38 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:17:38.862+0000 7f7257034640 -1 osd.7 0 log_to_monitors true 2024-09-11T22:17:39.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:38 smithi005 ceph-mon[30943]: from='osd.7 [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:17:39.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:38 smithi005 ceph-mon[30943]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T22:17:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:39 smithi159 ceph-mon[35534]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:39 smithi159 ceph-mon[35534]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T22:17:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:39 smithi159 ceph-mon[35534]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T22:17:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:39 smithi159 ceph-mon[35534]: from='osd.7 [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:17:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:39 smithi159 ceph-mon[35534]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:17:40.234 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:39 smithi172 ceph-mon[35461]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:40.234 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:39 smithi172 ceph-mon[35461]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T22:17:40.234 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:39 smithi172 ceph-mon[35461]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T22:17:40.234 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:40.234 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:39 smithi172 ceph-mon[35461]: from='osd.7 [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:17:40.235 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:39 smithi172 ceph-mon[35461]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:17:40.236 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:17:40.266 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_2 2024-09-11T22:17:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:39 smithi005 ceph-mon[30943]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:39 smithi005 ceph-mon[30943]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T22:17:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:39 smithi005 ceph-mon[30943]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T22:17:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:39 smithi005 ceph-mon[30943]: from='osd.7 [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:17:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:39 smithi005 ceph-mon[30943]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:17:40.559 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:17:40.829 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:17:40.369+0000 7f7252fb7640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T22:17:40.830 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:17:40.376+0000 7f724add7640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:17:41.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.080+0000 7f30f1267640 1 -- 172.21.15.172:0/2476163352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ec104f90 msgr2=0x7f30ec107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:17:41.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.080+0000 7f30f1267640 1 --2- 172.21.15.172:0/2476163352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ec104f90 0x7f30ec107380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f30e0009900 tx=0x7f30e002a410 comp rx=0 tx=0).stop 2024-09-11T22:17:41.083 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.080+0000 7f30f1267640 1 -- 172.21.15.172:0/2476163352 shutdown_connections 2024-09-11T22:17:41.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.080+0000 7f30f1267640 1 --2- 172.21.15.172:0/2476163352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec1078c0 0x7f30ec109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:41.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.080+0000 7f30f1267640 1 --2- 172.21.15.172:0/2476163352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ec104f90 0x7f30ec107380 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:41.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.080+0000 7f30f1267640 1 --2- 172.21.15.172:0/2476163352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec102660 0x7f30ec104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:41.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.080+0000 7f30f1267640 1 -- 172.21.15.172:0/2476163352 >> 172.21.15.172:0/2476163352 conn(0x7f30ec0fc280 msgr2=0x7f30ec0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:17:41.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.080+0000 7f30f1267640 1 -- 172.21.15.172:0/2476163352 shutdown_connections 2024-09-11T22:17:41.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.080+0000 7f30f1267640 1 -- 172.21.15.172:0/2476163352 wait complete. 2024-09-11T22:17:41.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.081+0000 7f30f1267640 1 Processor -- start 2024-09-11T22:17:41.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.081+0000 7f30f1267640 1 -- start start 2024-09-11T22:17:41.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.081+0000 7f30f1267640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ec102660 0x7f30ec101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:17:41.084 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.081+0000 7f30f1267640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec104f90 0x7f30ec100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:17:41.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.081+0000 7f30f1267640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec1078c0 0x7f30ec100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:17:41.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.081+0000 7f30f1267640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec10d8d0 con 0x7f30ec1078c0 2024-09-11T22:17:41.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30f1267640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec10d750 con 0x7f30ec104f90 2024-09-11T22:17:41.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30f1267640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec10da50 con 0x7f30ec102660 2024-09-11T22:17:41.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ec102660 0x7f30ec101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:17:41.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ec102660 0x7f30ec101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:58726/0 (socket says 172.21.15.172:58726) 2024-09-11T22:17:41.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eaffd640 1 -- 172.21.15.172:0/180034149 learned_addr learned my addr 172.21.15.172:0/180034149 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:17:41.085 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30ea7fc640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec104f90 0x7f30ec100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:17:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eb7fe640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec1078c0 0x7f30ec100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:17:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eaffd640 1 -- 172.21.15.172:0/180034149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec104f90 msgr2=0x7f30ec100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:17:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eaffd640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec104f90 0x7f30ec100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eaffd640 1 -- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec1078c0 msgr2=0x7f30ec100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:17:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eaffd640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec1078c0 0x7f30ec100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:17:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eaffd640 1 -- 172.21.15.172:0/180034149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec101410 con 0x7f30ec102660 2024-09-11T22:17:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eb7fe640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec1078c0 0x7f30ec100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:17:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30eaffd640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ec102660 0x7f30ec101fc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00ea80 tx=0x7f30dc00ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:17:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30d3fff640 1 -- 172.21.15.172:0/180034149 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00ce40 con 0x7f30ec102660 2024-09-11T22:17:41.086 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30f1267640 1 -- 172.21.15.172:0/180034149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec1a2140 con 0x7f30ec102660 2024-09-11T22:17:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30d3fff640 1 -- 172.21.15.172:0/180034149 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30dc005020 con 0x7f30ec102660 2024-09-11T22:17:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30d3fff640 1 -- 172.21.15.172:0/180034149 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc011760 con 0x7f30ec102660 2024-09-11T22:17:41.087 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.082+0000 7f30f1267640 1 -- 172.21.15.172:0/180034149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec1a26d0 con 0x7f30ec102660 2024-09-11T22:17:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.083+0000 7f30f1267640 1 -- 172.21.15.172:0/180034149 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30ec0fdd70 con 0x7f30ec102660 2024-09-11T22:17:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.085+0000 7f30d3fff640 1 -- 172.21.15.172:0/180034149 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f30dc0051d0 con 0x7f30ec102660 2024-09-11T22:17:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.085+0000 7f30d3fff640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f30d40a2c00 0x7f30d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:17:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.085+0000 7f30d3fff640 1 -- 172.21.15.172:0/180034149 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5430+0+0 (secure 0 0 0) 0x7f30dc0ceaa0 con 0x7f30ec102660 2024-09-11T22:17:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.085+0000 7f30ea7fc640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f30d40a2c00 0x7f30d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:17:41.088 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.086+0000 7f30ea7fc640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f30d40a2c00 0x7f30d40a50c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f30e0005ab0 tx=0x7f30e0000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:17:41.089 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.088+0000 7f30d3fff640 1 -- 172.21.15.172:0/180034149 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30dc016030 con 0x7f30ec102660 2024-09-11T22:17:41.262 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:17:41.261+0000 7f30f1267640 1 -- 172.21.15.172:0/180034149 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f30ec062630 con 0x7f30d40a2c00 2024-09-11T22:17:41.262 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:17:41.263 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T22:17:41.263 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:41.263 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:41.263 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi159 2024-09-11T22:17:41.263 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.263 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.263 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:41.264 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:41.264 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:41.264 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-11T22:17:41.264 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.264 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:41.264 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:41.265 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.265 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:40 smithi172 ceph-mon[35461]: from='osd.7 ' entity='osd.7' 2024-09-11T22:17:41.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:17:41.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T22:17:41.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:41.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:41.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi159 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:40 smithi005 ceph-mon[30943]: from='osd.7 ' entity='osd.7' 2024-09-11T22:17:41.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:17:41.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T22:17:41.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:41.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:41.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi159 2024-09-11T22:17:41.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:41.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:41.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:17:41.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-11T22:17:41.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:41.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:17:41.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:41.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:40 smithi159 ceph-mon[35534]: from='osd.7 ' entity='osd.7' 2024-09-11T22:17:42.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:17:42.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:17:42.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:42.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:42.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: osd.7 [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] boot 2024-09-11T22:17:42.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T22:17:42.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:42.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:17:42.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:17:42.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:17:42.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:41 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:17:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:17:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: osd.7 [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] boot 2024-09-11T22:17:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T22:17:42.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:42.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:17:42.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:17:42.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:17:42.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:41 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:17:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:17:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-09-11T22:17:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: osd.7 [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] boot 2024-09-11T22:17:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T22:17:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:17:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:17:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:17:42.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:17:42.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:41 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:43.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:42 smithi172 ceph-mon[35461]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T22:17:43.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:42 smithi005 ceph-mon[30943]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T22:17:43.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:42 smithi159 ceph-mon[35534]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T22:17:44.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:43 smithi005 ceph-mon[30943]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:44.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:43 smithi005 ceph-mon[30943]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T22:17:44.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:43 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/3753821476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12a4fca4-4f15-41b1-8776-6722e1692b25"}]: dispatch 2024-09-11T22:17:44.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:43 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12a4fca4-4f15-41b1-8776-6722e1692b25"}]: dispatch 2024-09-11T22:17:44.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:43 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12a4fca4-4f15-41b1-8776-6722e1692b25"}]': finished 2024-09-11T22:17:44.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:43 smithi005 ceph-mon[30943]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T22:17:44.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:43 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:17:44.328 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:43 smithi172 ceph-mon[35461]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:44.328 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:43 smithi172 ceph-mon[35461]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T22:17:44.328 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:43 smithi172 ceph-mon[35461]: from='client.? 172.21.15.172:0/3753821476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12a4fca4-4f15-41b1-8776-6722e1692b25"}]: dispatch 2024-09-11T22:17:44.328 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:43 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12a4fca4-4f15-41b1-8776-6722e1692b25"}]: dispatch 2024-09-11T22:17:44.328 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:43 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12a4fca4-4f15-41b1-8776-6722e1692b25"}]': finished 2024-09-11T22:17:44.328 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:43 smithi172 ceph-mon[35461]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T22:17:44.329 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:43 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:17:44.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:43 smithi159 ceph-mon[35534]: pgmap v172: 1 pgs: 1 unknown; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:44.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:43 smithi159 ceph-mon[35534]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T22:17:44.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:43 smithi159 ceph-mon[35534]: from='client.? 172.21.15.172:0/3753821476' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12a4fca4-4f15-41b1-8776-6722e1692b25"}]: dispatch 2024-09-11T22:17:44.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:43 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "12a4fca4-4f15-41b1-8776-6722e1692b25"}]: dispatch 2024-09-11T22:17:44.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:43 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "12a4fca4-4f15-41b1-8776-6722e1692b25"}]': finished 2024-09-11T22:17:44.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:43 smithi159 ceph-mon[35534]: osdmap e51: 9 total, 8 up, 9 in 2024-09-11T22:17:44.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:43 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:17:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:44 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/2238503487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:17:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:44 smithi005 ceph-mon[30943]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T22:17:45.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:44 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:17:45.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:44 smithi159 ceph-mon[35534]: from='client.? 172.21.15.172:0/2238503487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:17:45.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:44 smithi159 ceph-mon[35534]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T22:17:45.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:44 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:17:45.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:44 smithi172 ceph-mon[35461]: from='client.? 172.21.15.172:0/2238503487' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:17:45.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:44 smithi172 ceph-mon[35461]: osdmap e52: 9 total, 8 up, 9 in 2024-09-11T22:17:45.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:44 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:17:46.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:45 smithi005 ceph-mon[30943]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:46.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:45 smithi159 ceph-mon[35534]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:46.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:45 smithi172 ceph-mon[35461]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 251 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:48.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:47 smithi005 ceph-mon[30943]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:48.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:47 smithi159 ceph-mon[35534]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:48.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:47 smithi172 ceph-mon[35461]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:50.215 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:49 smithi172 ceph-mon[35461]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:50.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:49 smithi005 ceph-mon[30943]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:50.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:49 smithi159 ceph-mon[35534]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:51 smithi172 ceph-mon[35461]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:52.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:51 smithi005 ceph-mon[30943]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:52.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:51 smithi159 ceph-mon[35534]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:53.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:52 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T22:17:53.135 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:52 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:53.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:52 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T22:17:53.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:52 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:53.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:52 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-11T22:17:53.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:52 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:17:54.214 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:53 smithi172 ceph-mon[35461]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:54.215 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:53 smithi172 ceph-mon[35461]: Deploying daemon osd.8 on smithi172 2024-09-11T22:17:54.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:53 smithi005 ceph-mon[30943]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:54.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:53 smithi005 ceph-mon[30943]: Deploying daemon osd.8 on smithi172 2024-09-11T22:17:54.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:53 smithi159 ceph-mon[35534]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:54.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:53 smithi159 ceph-mon[35534]: Deploying daemon osd.8 on smithi172 2024-09-11T22:17:56.268 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:55 smithi172 ceph-mon[35461]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:56.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:55 smithi005 ceph-mon[30943]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:56.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:55 smithi159 ceph-mon[35534]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:58.091 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:57 smithi172 ceph-mon[35461]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:58.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:57 smithi005 ceph-mon[30943]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:58.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:57 smithi159 ceph-mon[35534]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:17:59.313 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:59.313 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:59.313 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:59.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:59.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:59.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:59.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:17:59.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:17:59.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:00.219 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:17:59 smithi172 ceph-mon[35461]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:00.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:17:59 smithi005 ceph-mon[30943]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:00.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:17:59 smithi159 ceph-mon[35534]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:02.262 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:01 smithi172 ceph-mon[35461]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:02.262 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:01 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:02.262 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:01 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:02.262 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:01 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:02.263 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:01 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:02.263 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:01 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:02.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:01 smithi005 ceph-mon[30943]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:02.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:01 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:02.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:01 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:02.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:01 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:02.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:01 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:02.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:01 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:02.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:01 smithi159 ceph-mon[35534]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:02.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:01 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:02.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:01 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:02.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:01 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:02.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:01 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:02.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:01 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:02.613 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.612+0000 7f30d3fff640 1 -- 172.21.15.172:0/180034149 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f30ec062630 con 0x7f30d40a2c00 2024-09-11T22:18:02.613 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 8 on host 'smithi172' 2024-09-11T22:18:02.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 -- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f30d40a2c00 msgr2=0x7f30d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:02.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f30d40a2c00 0x7f30d40a50c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f30e0005ab0 tx=0x7f30e0000bc0 comp rx=0 tx=0).stop 2024-09-11T22:18:02.615 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 -- 172.21.15.172:0/180034149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ec102660 msgr2=0x7f30ec101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:02.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ec102660 0x7f30ec101fc0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00ea80 tx=0x7f30dc00ef40 comp rx=0 tx=0).stop 2024-09-11T22:18:02.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 -- 172.21.15.172:0/180034149 shutdown_connections 2024-09-11T22:18:02.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f30d40a2c00 0x7f30d40a50c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:02.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec1078c0 0x7f30ec100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:02.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec104f90 0x7f30ec100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:02.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 --2- 172.21.15.172:0/180034149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ec102660 0x7f30ec101fc0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:02.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 -- 172.21.15.172:0/180034149 >> 172.21.15.172:0/180034149 conn(0x7f30ec0fc280 msgr2=0x7f30ec103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:18:02.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 -- 172.21.15.172:0/180034149 shutdown_connections 2024-09-11T22:18:02.616 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:02.615+0000 7f30f1267640 1 -- 172.21.15.172:0/180034149 wait complete. 2024-09-11T22:18:03.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:02 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:03.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:02 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:03.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:02 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:03.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:02 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:03.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:02 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:03.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:02 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:03.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:02 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:03.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:02 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:03.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:02 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:03.428 DEBUG:teuthology.orchestra.run.smithi172:osd.8> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.8.service 2024-09-11T22:18:03.430 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2024-09-11T22:18:03.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T22:18:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:18:04.098 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:03 smithi005 ceph-mon[30943]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:04.256 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:03 smithi172 ceph-mon[35461]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:04.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:03 smithi159 ceph-mon[35534]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:04.610 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:18:04 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:18:04.254+0000 7f9395072640 -1 osd.8 0 log_to_monitors true 2024-09-11T22:18:05.262 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:04 smithi005 ceph-mon[30943]: from='osd.8 [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:18:05.262 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:04 smithi005 ceph-mon[30943]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:18:05.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:04 smithi159 ceph-mon[35534]: from='osd.8 [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:18:05.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:04 smithi159 ceph-mon[35534]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:18:05.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:04 smithi172 ceph-mon[35461]: from='osd.8 [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:18:05.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:04 smithi172 ceph-mon[35461]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-11T22:18:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:06 smithi005 ceph-mon[30943]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:06 smithi005 ceph-mon[30943]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T22:18:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:06 smithi005 ceph-mon[30943]: from='osd.8 [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:18:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:06 smithi005 ceph-mon[30943]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T22:18:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:06 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:06 smithi005 ceph-mon[30943]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:18:06.293 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:06 smithi172 ceph-mon[35461]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:06.293 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:06 smithi172 ceph-mon[35461]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T22:18:06.293 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:06 smithi172 ceph-mon[35461]: from='osd.8 [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:18:06.293 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:06 smithi172 ceph-mon[35461]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T22:18:06.293 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:06 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:06.293 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:06 smithi172 ceph-mon[35461]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:18:06.294 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:18:06 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:18:06.011+0000 7f9390ff5640 -1 osd.8 0 waiting for initial osdmap 2024-09-11T22:18:06.294 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:18:06 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:18:06.019+0000 7f938ce1d640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:18:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:06 smithi159 ceph-mon[35534]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:06 smithi159 ceph-mon[35534]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-11T22:18:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:06 smithi159 ceph-mon[35534]: from='osd.8 [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:18:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:06 smithi159 ceph-mon[35534]: osdmap e53: 9 total, 8 up, 9 in 2024-09-11T22:18:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:06 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:06 smithi159 ceph-mon[35534]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:18:06.624 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:18:06.656 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2024-09-11T22:18:06.916 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:18:07.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi172 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:07.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:07.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:18:07.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T22:18:07.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:07.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:07.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi172 2024-09-11T22:18:07.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:07.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:07.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:07.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:07.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:07.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:18:07.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: osdmap e54: 9 total, 8 up, 9 in 2024-09-11T22:18:07.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:07.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:07.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi172 2024-09-11T22:18:07.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:07.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:07.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:07.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:07.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:07.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.406+0000 7f03e11f9640 1 -- 172.21.15.5:0/2627918010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03dc1078c0 msgr2=0x7f03dc109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.406+0000 7f03e11f9640 1 --2- 172.21.15.5:0/2627918010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03dc1078c0 0x7f03dc109d10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009ca0 tx=0x7f03d402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:18:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.406+0000 7f03e11f9640 1 -- 172.21.15.5:0/2627918010 shutdown_connections 2024-09-11T22:18:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.406+0000 7f03e11f9640 1 --2- 172.21.15.5:0/2627918010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03dc1078c0 0x7f03dc109d10 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.406+0000 7f03e11f9640 1 --2- 172.21.15.5:0/2627918010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03dc104f90 0x7f03dc107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.406+0000 7f03e11f9640 1 --2- 172.21.15.5:0/2627918010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03dc102660 0x7f03dc104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.406+0000 7f03e11f9640 1 -- 172.21.15.5:0/2627918010 >> 172.21.15.5:0/2627918010 conn(0x7f03dc0fc280 msgr2=0x7f03dc0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:18:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.406+0000 7f03e11f9640 1 -- 172.21.15.5:0/2627918010 shutdown_connections 2024-09-11T22:18:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.406+0000 7f03e11f9640 1 -- 172.21.15.5:0/2627918010 wait complete. 2024-09-11T22:18:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.407+0000 7f03e11f9640 1 Processor -- start 2024-09-11T22:18:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.407+0000 7f03e11f9640 1 -- start start 2024-09-11T22:18:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03e11f9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03dc102660 0x7f03dc19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03e11f9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03dc104f90 0x7f03dc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03e11f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03dc1078c0 0x7f03dc1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03e11f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc10d860 con 0x7f03dc1078c0 2024-09-11T22:18:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03e11f9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc10d6e0 con 0x7f03dc104f90 2024-09-11T22:18:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03e11f9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc10d9e0 con 0x7f03dc102660 2024-09-11T22:18:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03db59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03dc1078c0 0x7f03dc1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:18:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03db59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03dc1078c0 0x7f03dc1a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49524/0 (socket says 172.21.15.5:49524) 2024-09-11T22:18:07.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03db59e640 1 -- 172.21.15.5:0/1295658437 learned_addr learned my addr 172.21.15.5:0/1295658437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03db59e640 1 -- 172.21.15.5:0/1295658437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03dc102660 msgr2=0x7f03dc19a0d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03db59e640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03dc102660 0x7f03dc19a0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03db59e640 1 -- 172.21.15.5:0/1295658437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03dc104f90 msgr2=0x7f03dc19a610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03db59e640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03dc104f90 0x7f03dc19a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03db59e640 1 -- 172.21.15.5:0/1295658437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc1a1d00 con 0x7f03dc1078c0 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03db59e640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03dc1078c0 0x7f03dc1a1690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f03d4031ae0 tx=0x7f03d4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03c3fff640 1 -- 172.21.15.5:0/1295658437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4003c30 con 0x7f03dc1078c0 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03c3fff640 1 -- 172.21.15.5:0/1295658437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d4003dd0 con 0x7f03dc1078c0 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.408+0000 7f03e11f9640 1 -- 172.21.15.5:0/1295658437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc1a1f90 con 0x7f03dc1078c0 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.409+0000 7f03c3fff640 1 -- 172.21.15.5:0/1295658437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d402fcb0 con 0x7f03dc1078c0 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.409+0000 7f03e11f9640 1 -- 172.21.15.5:0/1295658437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc1a2440 con 0x7f03dc1078c0 2024-09-11T22:18:07.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.410+0000 7f03e11f9640 1 -- 172.21.15.5:0/1295658437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03dc1093b0 con 0x7f03dc1078c0 2024-09-11T22:18:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.411+0000 7f03c3fff640 1 -- 172.21.15.5:0/1295658437 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f03d4004050 con 0x7f03dc1078c0 2024-09-11T22:18:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.411+0000 7f03c3fff640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03c40a2bb0 0x7f03c40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.412+0000 7f03dad9d640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03c40a2bb0 0x7f03c40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:18:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.412+0000 7f03c3fff640 1 -- 172.21.15.5:0/1295658437 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5858+0+0 (secure 0 0 0) 0x7f03d40efa20 con 0x7f03dc1078c0 2024-09-11T22:18:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.412+0000 7f03dad9d640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03c40a2bb0 0x7f03c40a5070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009a60 tx=0x7f03cc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:18:07.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.416+0000 7f03c3fff640 1 -- 172.21.15.5:0/1295658437 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03d40bd310 con 0x7f03dc1078c0 2024-09-11T22:18:07.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:07.601+0000 7f03e11f9640 1 -- 172.21.15.5:0/1295658437 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f03dc062630 con 0x7f03c40a2bb0 2024-09-11T22:18:08.278 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: osd.8 [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] boot 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:18:08.279 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:08 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:08.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:18:08.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:18:08.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-11T22:18:08.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:08.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:08.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: osd.8 [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] boot 2024-09-11T22:18:08.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T22:18:08.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:08.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T22:18:08.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:18:08.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:18:08.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:08 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:08.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:18:08.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:18:08.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi172 to 5696M 2024-09-11T22:18:08.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-11T22:18:08.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:08.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: osd.8 [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] boot 2024-09-11T22:18:08.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: osdmap e55: 9 total, 9 up, 9 in 2024-09-11T22:18:08.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:18:08.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: osdmap e56: 9 total, 9 up, 9 in 2024-09-11T22:18:08.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:18:08.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:18:08.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:08 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:09.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:09 smithi005 ceph-mon[30943]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:18:09.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:09 smithi005 ceph-mon[30943]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T22:18:09.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:09 smithi159 ceph-mon[35534]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:18:09.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:09 smithi159 ceph-mon[35534]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T22:18:09.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:09 smithi172 ceph-mon[35461]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:18:09.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:09 smithi172 ceph-mon[35461]: osdmap e57: 9 total, 9 up, 9 in 2024-09-11T22:18:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:10 smithi005 ceph-mon[30943]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:10 smithi005 ceph-mon[30943]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T22:18:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:10 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1274250364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7758f2e-e665-4964-a5cf-f5298315934b"}]: dispatch 2024-09-11T22:18:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:10 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1274250364' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7758f2e-e665-4964-a5cf-f5298315934b"}]': finished 2024-09-11T22:18:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:10 smithi005 ceph-mon[30943]: osdmap e59: 10 total, 9 up, 10 in 2024-09-11T22:18:10.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:10 smithi159 ceph-mon[35534]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:10 smithi159 ceph-mon[35534]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T22:18:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:10 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1274250364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7758f2e-e665-4964-a5cf-f5298315934b"}]: dispatch 2024-09-11T22:18:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:10 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1274250364' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7758f2e-e665-4964-a5cf-f5298315934b"}]': finished 2024-09-11T22:18:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:10 smithi159 ceph-mon[35534]: osdmap e59: 10 total, 9 up, 10 in 2024-09-11T22:18:10.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:10 smithi172 ceph-mon[35461]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:10 smithi172 ceph-mon[35461]: osdmap e58: 9 total, 9 up, 9 in 2024-09-11T22:18:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:10 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1274250364' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a7758f2e-e665-4964-a5cf-f5298315934b"}]: dispatch 2024-09-11T22:18:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:10 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1274250364' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a7758f2e-e665-4964-a5cf-f5298315934b"}]': finished 2024-09-11T22:18:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:10 smithi172 ceph-mon[35461]: osdmap e59: 10 total, 9 up, 10 in 2024-09-11T22:18:10.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:11.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:11 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/888000270' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:18:11.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:11 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/888000270' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:18:11.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:11 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/888000270' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:18:12.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:12 smithi005 ceph-mon[30943]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:12.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:12 smithi159 ceph-mon[35534]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:12.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:12 smithi172 ceph-mon[35461]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:14.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:14 smithi005 ceph-mon[30943]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-09-11T22:18:14.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:14 smithi159 ceph-mon[35534]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-09-11T22:18:14.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:14 smithi172 ceph-mon[35461]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-09-11T22:18:16.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:16 smithi005 ceph-mon[30943]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-11T22:18:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:16 smithi159 ceph-mon[35534]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-11T22:18:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:16 smithi172 ceph-mon[35461]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-11T22:18:18.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:18 smithi005 ceph-mon[30943]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:18:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:18 smithi159 ceph-mon[35534]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:18:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:18 smithi172 ceph-mon[35461]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T22:18:19.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T22:18:19.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:19 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:19 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T22:18:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:19 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:19 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-11T22:18:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:19 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:20 smithi005 ceph-mon[30943]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-11T22:18:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:20 smithi005 ceph-mon[30943]: Deploying daemon osd.9 on smithi005 2024-09-11T22:18:20.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:20 smithi159 ceph-mon[35534]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-11T22:18:20.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:20 smithi159 ceph-mon[35534]: Deploying daemon osd.9 on smithi005 2024-09-11T22:18:20.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:20 smithi172 ceph-mon[35461]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-11T22:18:20.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:20 smithi172 ceph-mon[35461]: Deploying daemon osd.9 on smithi005 2024-09-11T22:18:22.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:22 smithi005 ceph-mon[30943]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-11T22:18:22.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:22 smithi159 ceph-mon[35534]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-11T22:18:22.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:22 smithi172 ceph-mon[35461]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-11T22:18:24.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:24 smithi005 ceph-mon[30943]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:18:24.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:24 smithi159 ceph-mon[35534]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:18:24.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:24 smithi172 ceph-mon[35461]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-11T22:18:26.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:26 smithi005 ceph-mon[30943]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:26.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:26.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:26.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:26.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:26 smithi159 ceph-mon[35534]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:26.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:26 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:26.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:26 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:26.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:26 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:26.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:26 smithi172 ceph-mon[35461]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:26 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:26 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:26 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:28.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:28 smithi159 ceph-mon[35534]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:28.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:28 smithi172 ceph-mon[35461]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:28.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:28 smithi005 ceph-mon[30943]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.113+0000 7f03c3fff640 1 -- 172.21.15.5:0/1295658437 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f03dc062630 con 0x7f03c40a2bb0 2024-09-11T22:18:30.117 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2024-09-11T22:18:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 -- 172.21.15.5:0/1295658437 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03c40a2bb0 msgr2=0x7f03c40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03c40a2bb0 0x7f03c40a5070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009a60 tx=0x7f03cc005f90 comp rx=0 tx=0).stop 2024-09-11T22:18:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 -- 172.21.15.5:0/1295658437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03dc1078c0 msgr2=0x7f03dc1a1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03dc1078c0 0x7f03dc1a1690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f03d4031ae0 tx=0x7f03d4031ee0 comp rx=0 tx=0).stop 2024-09-11T22:18:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 -- 172.21.15.5:0/1295658437 shutdown_connections 2024-09-11T22:18:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03c40a2bb0 0x7f03c40a5070 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03dc1078c0 0x7f03dc1a1690 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03dc104f90 0x7f03dc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 --2- 172.21.15.5:0/1295658437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03dc102660 0x7f03dc19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 -- 172.21.15.5:0/1295658437 >> 172.21.15.5:0/1295658437 conn(0x7f03dc0fc280 msgr2=0x7f03dc101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:18:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 -- 172.21.15.5:0/1295658437 shutdown_connections 2024-09-11T22:18:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:18:29.115+0000 7f03e11f9640 1 -- 172.21.15.5:0/1295658437 wait complete. 2024-09-11T22:18:30.120 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:29 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.120 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:29 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.120 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:29 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:30.120 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:29 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:30.121 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:29 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.121 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:29 smithi005 ceph-mon[30943]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:30.121 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:29 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:30.121 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:29 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.121 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:29 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.123 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.9.service 2024-09-11T22:18:30.124 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:29 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.124 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:29 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.124 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:29 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:30.125 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:29 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:30.125 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:29 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.125 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:29 smithi159 ceph-mon[35534]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:30.125 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:29 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:30.125 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:29 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.125 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:29 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.125 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:29 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.125 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:29 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:29 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:30.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:29 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:30.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:29 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:29 smithi172 ceph-mon[35461]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:30.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:29 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:30.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:29 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.126 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:29 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:30.128 INFO:tasks.cephadm:Deploying osd.10 on smithi159 with /dev/vg_nvme/lv_1... 2024-09-11T22:18:30.128 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T22:18:30.386 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:18:31.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:18:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:18:31.211+0000 7f7d3c4ba640 -1 osd.9 0 log_to_monitors true 2024-09-11T22:18:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:31 smithi005 ceph-mon[30943]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:31 smithi005 ceph-mon[30943]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T22:18:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:31 smithi159 ceph-mon[35534]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:31 smithi159 ceph-mon[35534]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T22:18:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:31 smithi172 ceph-mon[35461]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:31 smithi172 ceph-mon[35461]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-11T22:18:32.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:32 smithi005 ceph-mon[30943]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T22:18:32.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:32 smithi005 ceph-mon[30943]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T22:18:32.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:32 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:32.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:32 smithi005 ceph-mon[30943]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:18:32.821 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:18:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:18:32.482+0000 7f7d3843d640 -1 osd.9 0 waiting for initial osdmap 2024-09-11T22:18:32.821 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:18:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:18:32.493+0000 7f7d34265640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:18:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:32 smithi159 ceph-mon[35534]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T22:18:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:32 smithi159 ceph-mon[35534]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T22:18:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:32 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:32 smithi159 ceph-mon[35534]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:18:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:32 smithi172 ceph-mon[35461]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-11T22:18:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:32 smithi172 ceph-mon[35461]: osdmap e60: 10 total, 9 up, 10 in 2024-09-11T22:18:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:32 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:32 smithi172 ceph-mon[35461]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-11T22:18:33.075 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-11T22:18:33.101 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_1 2024-09-11T22:18:33.349 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:18:33.772 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:33.773 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:18:33.773 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T22:18:33.773 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:33.773 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:33.773 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi005 2024-09-11T22:18:33.777 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.777 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.777 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.777 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.777 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.778 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.778 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-11T22:18:33.778 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.778 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:33.778 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:33.778 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:33 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi005 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:33.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:33.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:33 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.855+0000 7f765e145640 1 -- 172.21.15.159:0/4087896272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7658107900 msgr2=0x7f7658109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:33.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.855+0000 7f765e145640 1 --2- 172.21.15.159:0/4087896272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7658107900 0x7f7658109d50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7650009900 tx=0x7f765002a410 comp rx=0 tx=0).stop 2024-09-11T22:18:33.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.855+0000 7f765e145640 1 -- 172.21.15.159:0/4087896272 shutdown_connections 2024-09-11T22:18:33.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.855+0000 7f765e145640 1 --2- 172.21.15.159:0/4087896272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7658107900 0x7f7658109d50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:33.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.855+0000 7f765e145640 1 --2- 172.21.15.159:0/4087896272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658104fd0 0x7f76581073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:33.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.855+0000 7f765e145640 1 --2- 172.21.15.159:0/4087896272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76581026a0 0x7f7658104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:33.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.855+0000 7f765e145640 1 -- 172.21.15.159:0/4087896272 >> 172.21.15.159:0/4087896272 conn(0x7f76580fc2a0 msgr2=0x7f76580fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:18:33.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.855+0000 7f765e145640 1 -- 172.21.15.159:0/4087896272 shutdown_connections 2024-09-11T22:18:33.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.855+0000 7f765e145640 1 -- 172.21.15.159:0/4087896272 wait complete. 2024-09-11T22:18:33.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.855+0000 7f765e145640 1 Processor -- start 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.856+0000 7f765e145640 1 -- start start 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.856+0000 7f765e145640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76581026a0 0x7f7658198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.856+0000 7f765e145640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658104fd0 0x7f7658198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.856+0000 7f765e145640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658107900 0x7f765819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.856+0000 7f765e145640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f765810d990 con 0x7f7658107900 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.856+0000 7f765e145640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f765810d810 con 0x7f76581026a0 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.856+0000 7f765e145640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f765810db10 con 0x7f7658104fd0 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.856+0000 7f76577fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76581026a0 0x7f7658198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.856+0000 7f76577fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76581026a0 0x7f7658198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53206/0 (socket says 172.21.15.159:53206) 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.856+0000 7f76577fe640 1 -- 172.21.15.159:0/891630773 learned_addr learned my addr 172.21.15.159:0/891630773 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f7657fff640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658107900 0x7f765819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f7656ffd640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658104fd0 0x7f7658198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:18:33.859 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f76577fe640 1 -- 172.21.15.159:0/891630773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658104fd0 msgr2=0x7f7658198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f76577fe640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658104fd0 0x7f7658198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f76577fe640 1 -- 172.21.15.159:0/891630773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658107900 msgr2=0x7f765819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f76577fe640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658107900 0x7f765819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f76577fe640 1 -- 172.21.15.159:0/891630773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765819fc70 con 0x7f76581026a0 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f7656ffd640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658104fd0 0x7f7658198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f76577fe640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76581026a0 0x7f7658198040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f764800ba50 tx=0x7f764800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f7654ff9640 1 -- 172.21.15.159:0/891630773 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648019070 con 0x7f76581026a0 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f765e145640 1 -- 172.21.15.159:0/891630773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765819ff60 con 0x7f76581026a0 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f7654ff9640 1 -- 172.21.15.159:0/891630773 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f764800cbe0 con 0x7f76581026a0 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f7654ff9640 1 -- 172.21.15.159:0/891630773 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76480076e0 con 0x7f76581026a0 2024-09-11T22:18:33.860 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.857+0000 7f765e145640 1 -- 172.21.15.159:0/891630773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76581a0490 con 0x7f76581026a0 2024-09-11T22:18:33.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 279 MiB used, 804 GiB / 805 GiB avail 2024-09-11T22:18:33.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-11T22:18:33.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: osdmap e61: 10 total, 9 up, 10 in 2024-09-11T22:18:33.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:33.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:33.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi005 2024-09-11T22:18:33.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:33.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-11T22:18:33.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:33.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:33.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:33 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:33.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.858+0000 7f765e145640 1 -- 172.21.15.159:0/891630773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76580fdd90 con 0x7f76581026a0 2024-09-11T22:18:33.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.859+0000 7f7654ff9640 1 -- 172.21.15.159:0/891630773 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f764801c070 con 0x7f76581026a0 2024-09-11T22:18:33.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.860+0000 7f7654ff9640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f76400a2bb0 0x7f76400a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:33.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.860+0000 7f7654ff9640 1 -- 172.21.15.159:0/891630773 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f76480cddf0 con 0x7f76581026a0 2024-09-11T22:18:33.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.860+0000 7f7656ffd640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f76400a2bb0 0x7f76400a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:18:33.863 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.861+0000 7f7656ffd640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f76400a2bb0 0x7f76400a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7658199560 tx=0x7f764c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:18:33.864 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:33.864+0000 7f7654ff9640 1 -- 172.21.15.159:0/891630773 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f764809af00 con 0x7f76581026a0 2024-09-11T22:18:34.040 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:34.040+0000 7f765e145640 1 -- 172.21.15.159:0/891630773 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7658062630 con 0x7f76400a2bb0 2024-09-11T22:18:34.700 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:34 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:18:34.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:34 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:18:34.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:34 smithi159 ceph-mon[35534]: osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] boot 2024-09-11T22:18:34.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:34 smithi159 ceph-mon[35534]: osdmap e62: 10 total, 10 up, 10 in 2024-09-11T22:18:34.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:34.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:34 smithi159 ceph-mon[35534]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:18:34.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:18:34.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:18:34.701 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:34.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:34 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:18:34.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:34 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:18:34.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:34 smithi005 ceph-mon[30943]: osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] boot 2024-09-11T22:18:34.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:34 smithi005 ceph-mon[30943]: osdmap e62: 10 total, 10 up, 10 in 2024-09-11T22:18:34.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:34.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:34 smithi005 ceph-mon[30943]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:18:34.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:18:34.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:18:34.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:34 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:18:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:34 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:18:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:34 smithi172 ceph-mon[35461]: osd.9 [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] boot 2024-09-11T22:18:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:34 smithi172 ceph-mon[35461]: osdmap e62: 10 total, 10 up, 10 in 2024-09-11T22:18:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:18:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:34 smithi172 ceph-mon[35461]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:18:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:18:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:18:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:35 smithi005 ceph-mon[30943]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:35 smithi005 ceph-mon[30943]: osdmap e63: 10 total, 10 up, 10 in 2024-09-11T22:18:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:35 smithi159 ceph-mon[35534]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:35 smithi159 ceph-mon[35534]: osdmap e63: 10 total, 10 up, 10 in 2024-09-11T22:18:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:35 smithi172 ceph-mon[35461]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 306 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:35 smithi172 ceph-mon[35461]: osdmap e63: 10 total, 10 up, 10 in 2024-09-11T22:18:37.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:37 smithi159 ceph-mon[35534]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:37.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:37 smithi159 ceph-mon[35534]: from='client.? 172.21.15.159:0/3856825446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e3cad92-29cf-43a6-84c0-4533de103ce8"}]: dispatch 2024-09-11T22:18:37.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:37 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e3cad92-29cf-43a6-84c0-4533de103ce8"}]: dispatch 2024-09-11T22:18:37.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:37 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e3cad92-29cf-43a6-84c0-4533de103ce8"}]': finished 2024-09-11T22:18:37.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:37 smithi159 ceph-mon[35534]: osdmap e64: 11 total, 10 up, 11 in 2024-09-11T22:18:37.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:37.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:37 smithi159 ceph-mon[35534]: from='client.? 172.21.15.159:0/2335453067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:18:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:37 smithi005 ceph-mon[30943]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:37 smithi005 ceph-mon[30943]: from='client.? 172.21.15.159:0/3856825446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e3cad92-29cf-43a6-84c0-4533de103ce8"}]: dispatch 2024-09-11T22:18:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:37 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e3cad92-29cf-43a6-84c0-4533de103ce8"}]: dispatch 2024-09-11T22:18:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:37 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e3cad92-29cf-43a6-84c0-4533de103ce8"}]': finished 2024-09-11T22:18:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:37 smithi005 ceph-mon[30943]: osdmap e64: 11 total, 10 up, 11 in 2024-09-11T22:18:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:37.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:37 smithi005 ceph-mon[30943]: from='client.? 172.21.15.159:0/2335453067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:18:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:37 smithi172 ceph-mon[35461]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:37 smithi172 ceph-mon[35461]: from='client.? 172.21.15.159:0/3856825446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e3cad92-29cf-43a6-84c0-4533de103ce8"}]: dispatch 2024-09-11T22:18:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:37 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2e3cad92-29cf-43a6-84c0-4533de103ce8"}]: dispatch 2024-09-11T22:18:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:37 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2e3cad92-29cf-43a6-84c0-4533de103ce8"}]': finished 2024-09-11T22:18:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:37 smithi172 ceph-mon[35461]: osdmap e64: 11 total, 10 up, 11 in 2024-09-11T22:18:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:37 smithi172 ceph-mon[35461]: from='client.? 172.21.15.159:0/2335453067' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:18:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:39 smithi005 ceph-mon[30943]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:39 smithi159 ceph-mon[35534]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:39 smithi172 ceph-mon[35461]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:41 smithi005 ceph-mon[30943]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:41.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:41 smithi159 ceph-mon[35534]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:41.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:41 smithi172 ceph-mon[35461]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:43.719 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:43 smithi159 ceph-mon[35534]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:43 smithi005 ceph-mon[30943]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:43.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:43 smithi172 ceph-mon[35461]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:45.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:45 smithi005 ceph-mon[30943]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:45.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T22:18:45.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:45 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:45.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:45 smithi005 ceph-mon[30943]: Deploying daemon osd.10 on smithi159 2024-09-11T22:18:45.824 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:45 smithi159 ceph-mon[35534]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:45.824 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:45 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T22:18:45.824 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:45 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:45.824 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:45 smithi159 ceph-mon[35534]: Deploying daemon osd.10 on smithi159 2024-09-11T22:18:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:45 smithi172 ceph-mon[35461]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:45 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-11T22:18:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:45 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:45 smithi172 ceph-mon[35461]: Deploying daemon osd.10 on smithi159 2024-09-11T22:18:47.743 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:47 smithi159 ceph-mon[35534]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:47 smithi005 ceph-mon[30943]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:47 smithi172 ceph-mon[35461]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:49 smithi159 ceph-mon[35534]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:49.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:49 smithi005 ceph-mon[30943]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:49.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:49 smithi172 ceph-mon[35461]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:51.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:51 smithi159 ceph-mon[35534]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:51.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:51.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:51.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:51 smithi005 ceph-mon[30943]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:51.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:51 smithi172 ceph-mon[35461]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:51.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:51.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:51.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:53.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:53 smithi005 ceph-mon[30943]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:53 smithi159 ceph-mon[35534]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:53 smithi172 ceph-mon[35461]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:54.237 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.236+0000 7f7654ff9640 1 -- 172.21.15.159:0/891630773 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7658062630 con 0x7f76400a2bb0 2024-09-11T22:18:54.237 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 10 on host 'smithi159' 2024-09-11T22:18:54.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 -- 172.21.15.159:0/891630773 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f76400a2bb0 msgr2=0x7f76400a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:54.239 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f76400a2bb0 0x7f76400a5070 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7658199560 tx=0x7f764c008040 comp rx=0 tx=0).stop 2024-09-11T22:18:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 -- 172.21.15.159:0/891630773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76581026a0 msgr2=0x7f7658198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76581026a0 0x7f7658198040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f764800ba50 tx=0x7f764800bf10 comp rx=0 tx=0).stop 2024-09-11T22:18:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 -- 172.21.15.159:0/891630773 shutdown_connections 2024-09-11T22:18:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f76400a2bb0 0x7f76400a5070 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7658107900 0x7f765819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658104fd0 0x7f7658198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 --2- 172.21.15.159:0/891630773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76581026a0 0x7f7658198040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 -- 172.21.15.159:0/891630773 >> 172.21.15.159:0/891630773 conn(0x7f76580fc2a0 msgr2=0x7f7658102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:18:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.238+0000 7f765e145640 1 -- 172.21.15.159:0/891630773 shutdown_connections 2024-09-11T22:18:54.240 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-11T22:18:54.239+0000 7f765e145640 1 -- 172.21.15.159:0/891630773 wait complete. 2024-09-11T22:18:54.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:54.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:54.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:54.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:54 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:54.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:54.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:54.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:54 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:18:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:54 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:54.954 DEBUG:teuthology.orchestra.run.smithi159:osd.10> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.10.service 2024-09-11T22:18:54.956 INFO:tasks.cephadm:Deploying osd.11 on smithi172 with /dev/vg_nvme/lv_1... 2024-09-11T22:18:54.956 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-11T22:18:55.210 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:18:56.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:56 smithi172 ceph-mon[35461]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:56.426 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:56 smithi159 ceph-mon[35534]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:56.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:56 smithi005 ceph-mon[30943]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:56.695 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:18:56 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:18:56.638+0000 7f250fab7640 -1 osd.10 0 log_to_monitors true 2024-09-11T22:18:57.326 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:57 smithi172 ceph-mon[35461]: from='osd.10 [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:18:57.327 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:57 smithi172 ceph-mon[35461]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:18:57.522 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:57 smithi159 ceph-mon[35534]: from='osd.10 [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:18:57.522 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:57 smithi159 ceph-mon[35534]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:18:57.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:57 smithi005 ceph-mon[30943]: from='osd.10 [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:18:57.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:57 smithi005 ceph-mon[30943]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-11T22:18:58.042 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-09-11T22:18:58.069 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi172:vg_nvme/lv_1 2024-09-11T22:18:58.323 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:18:58.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:18:58.253+0000 7f250ba3a640 -1 osd.10 0 waiting for initial osdmap 2024-09-11T22:18:58.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:18:58.263+0000 7f2507862640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:18:58.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:58.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T22:18:58.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T22:18:58.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:58.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='osd.10 [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:58.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:58 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.340 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:58.340 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T22:18:58.340 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T22:18:58.340 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:58.340 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='osd.10 [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:18:58.340 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:18:58.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:58.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:58.341 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:58 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:18:58.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-11T22:18:58.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: osdmap e65: 11 total, 10 up, 11 in 2024-09-11T22:18:58.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:58.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='osd.10 [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:18:58.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-11T22:18:58.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:18:58.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:58.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:18:58.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:58 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:18:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.818+0000 7f5d0fb77640 1 -- 172.21.15.172:0/4077111570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d08102e00 msgr2=0x7f5d08103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.818+0000 7f5d0fb77640 1 --2- 172.21.15.172:0/4077111570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d08102e00 0x7f5d08103280 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5d04009900 tx=0x7f5d0402a410 comp rx=0 tx=0).stop 2024-09-11T22:18:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.818+0000 7f5d0fb77640 1 -- 172.21.15.172:0/4077111570 shutdown_connections 2024-09-11T22:18:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.818+0000 7f5d0fb77640 1 --2- 172.21.15.172:0/4077111570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d081037c0 0x7f5d0810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:58.835 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.818+0000 7f5d0fb77640 1 --2- 172.21.15.172:0/4077111570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d08102e00 0x7f5d08103280 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.818+0000 7f5d0fb77640 1 --2- 172.21.15.172:0/4077111570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d08101c00 0x7f5d08102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.818+0000 7f5d0fb77640 1 -- 172.21.15.172:0/4077111570 >> 172.21.15.172:0/4077111570 conn(0x7f5d080fd390 msgr2=0x7f5d080ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:18:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.818+0000 7f5d0fb77640 1 -- 172.21.15.172:0/4077111570 shutdown_connections 2024-09-11T22:18:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.818+0000 7f5d0fb77640 1 -- 172.21.15.172:0/4077111570 wait complete. 2024-09-11T22:18:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0fb77640 1 Processor -- start 2024-09-11T22:18:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0fb77640 1 -- start start 2024-09-11T22:18:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0fb77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d08101c00 0x7f5d08197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0fb77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d08102e00 0x7f5d081984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0fb77640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d081037c0 0x7f5d0819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:58.836 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0fb77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0810d8e0 con 0x7f5d08102e00 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0fb77640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0810d760 con 0x7f5d081037c0 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0fb77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0810da60 con 0x7f5d08101c00 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0d913640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d08101c00 0x7f5d08197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0d913640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d08101c00 0x7f5d08197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:42564/0 (socket says 172.21.15.172:42564) 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0d913640 1 -- 172.21.15.172:0/2449995246 learned_addr learned my addr 172.21.15.172:0/2449995246 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0e114640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d081037c0 0x7f5d0819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0d913640 1 -- 172.21.15.172:0/2449995246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d081037c0 msgr2=0x7f5d0819f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0d913640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d081037c0 0x7f5d0819f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0d913640 1 -- 172.21.15.172:0/2449995246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d08102e00 msgr2=0x7f5d081984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0d913640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d08102e00 0x7f5d081984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0d913640 1 -- 172.21.15.172:0/2449995246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0819fb90 con 0x7f5d08101c00 2024-09-11T22:18:58.837 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.819+0000 7f5d0e114640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d081037c0 0x7f5d0819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.820+0000 7f5d0d913640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d08101c00 0x7f5d08197f60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00d770 tx=0x7f5cfc00dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.820+0000 7f5cfaffd640 1 -- 172.21.15.172:0/2449995246 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc017070 con 0x7f5d08101c00 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.820+0000 7f5cfaffd640 1 -- 172.21.15.172:0/2449995246 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cfc009c30 con 0x7f5d08101c00 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.820+0000 7f5cfaffd640 1 -- 172.21.15.172:0/2449995246 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc003da0 con 0x7f5d08101c00 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.820+0000 7f5d0fb77640 1 -- 172.21.15.172:0/2449995246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0819fe80 con 0x7f5d08101c00 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.820+0000 7f5d0fb77640 1 -- 172.21.15.172:0/2449995246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d081a0400 con 0x7f5d08101c00 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.821+0000 7f5d0fb77640 1 -- 172.21.15.172:0/2449995246 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d08109050 con 0x7f5d08101c00 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.822+0000 7f5cfaffd640 1 -- 172.21.15.172:0/2449995246 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5cfc003f40 con 0x7f5d08101c00 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.822+0000 7f5cfaffd640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5cf40a2c30 0x7f5cf40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.823+0000 7f5cfaffd640 1 -- 172.21.15.172:0/2449995246 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6414+0+0 (secure 0 0 0) 0x7f5cfc0ce600 con 0x7f5d08101c00 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.823+0000 7f5d0d112640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5cf40a2c30 0x7f5cf40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.823+0000 7f5d0d112640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5cf40a2c30 0x7f5cf40a50f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d040096c0 tx=0x7f5d04031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:18:58.838 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:58.826+0000 7f5cfaffd640 1 -- 172.21.15.172:0/2449995246 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cfc09bcf0 con 0x7f5d08101c00 2024-09-11T22:18:59.003 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:18:59.003+0000 7f5d0fb77640 1 -- 172.21.15.172:0/2449995246 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5d08062630 con 0x7f5cf40a2c30 2024-09-11T22:18:59.253 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:59 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi159 2024-09-11T22:18:59.254 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:59 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-11T22:18:59.254 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:59 smithi172 ceph-mon[35461]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:18:59.254 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:59 smithi172 ceph-mon[35461]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T22:18:59.254 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:59.254 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:59.254 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:18:59.254 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:18:59.254 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:18:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:59 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi159 2024-09-11T22:18:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:59 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-11T22:18:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:59 smithi005 ceph-mon[30943]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:18:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:59 smithi005 ceph-mon[30943]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T22:18:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:18:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:18:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:18:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:18:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:59 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi159 2024-09-11T22:18:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:59 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-11T22:18:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:59 smithi159 ceph-mon[35534]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-11T22:18:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:59 smithi159 ceph-mon[35534]: osdmap e66: 11 total, 10 up, 11 in 2024-09-11T22:18:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:18:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T22:18:59.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T22:18:59.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:18:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:00.552 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:00 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:19:00.552 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:00 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:19:00.552 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:00 smithi172 ceph-mon[35461]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:19:00.552 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:00 smithi172 ceph-mon[35461]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:19:00.552 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:00 smithi172 ceph-mon[35461]: osd.10 [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] boot 2024-09-11T22:19:00.553 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:00 smithi172 ceph-mon[35461]: osdmap e67: 11 total, 11 up, 11 in 2024-09-11T22:19:00.553 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:00 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:19:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:00 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:19:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:00 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:19:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:00 smithi005 ceph-mon[30943]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:19:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:00 smithi005 ceph-mon[30943]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:19:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:00 smithi005 ceph-mon[30943]: osd.10 [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] boot 2024-09-11T22:19:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:00 smithi005 ceph-mon[30943]: osdmap e67: 11 total, 11 up, 11 in 2024-09-11T22:19:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:00 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:19:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:00 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:19:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:00 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:19:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:00 smithi159 ceph-mon[35534]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-09-11T22:19:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:00 smithi159 ceph-mon[35534]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi172:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:19:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:00 smithi159 ceph-mon[35534]: osd.10 [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] boot 2024-09-11T22:19:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:00 smithi159 ceph-mon[35534]: osdmap e67: 11 total, 11 up, 11 in 2024-09-11T22:19:00.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:00 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:19:01.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:01 smithi005 ceph-mon[30943]: osdmap e68: 11 total, 11 up, 11 in 2024-09-11T22:19:01.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:01 smithi159 ceph-mon[35534]: osdmap e68: 11 total, 11 up, 11 in 2024-09-11T22:19:01.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:01 smithi172 ceph-mon[35461]: osdmap e68: 11 total, 11 up, 11 in 2024-09-11T22:19:02.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:02 smithi005 ceph-mon[30943]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:02.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:02 smithi005 ceph-mon[30943]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T22:19:02.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:02 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/3775988851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "684e017e-bc9b-4fc5-afd8-89a3199c5653"}]: dispatch 2024-09-11T22:19:02.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:02 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "684e017e-bc9b-4fc5-afd8-89a3199c5653"}]: dispatch 2024-09-11T22:19:02.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:02 smithi005 ceph-mon[30943]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "684e017e-bc9b-4fc5-afd8-89a3199c5653"}]': finished 2024-09-11T22:19:02.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:02 smithi005 ceph-mon[30943]: osdmap e70: 12 total, 11 up, 12 in 2024-09-11T22:19:02.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:02 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:02.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:02 smithi005 ceph-mon[30943]: from='client.? 172.21.15.172:0/339826043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:19:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:02 smithi159 ceph-mon[35534]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:02 smithi159 ceph-mon[35534]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T22:19:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:02 smithi159 ceph-mon[35534]: from='client.? 172.21.15.172:0/3775988851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "684e017e-bc9b-4fc5-afd8-89a3199c5653"}]: dispatch 2024-09-11T22:19:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:02 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "684e017e-bc9b-4fc5-afd8-89a3199c5653"}]: dispatch 2024-09-11T22:19:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:02 smithi159 ceph-mon[35534]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "684e017e-bc9b-4fc5-afd8-89a3199c5653"}]': finished 2024-09-11T22:19:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:02 smithi159 ceph-mon[35534]: osdmap e70: 12 total, 11 up, 12 in 2024-09-11T22:19:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:02 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:02.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:02 smithi159 ceph-mon[35534]: from='client.? 172.21.15.172:0/339826043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:19:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:02 smithi172 ceph-mon[35461]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:02 smithi172 ceph-mon[35461]: osdmap e69: 11 total, 11 up, 11 in 2024-09-11T22:19:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:02 smithi172 ceph-mon[35461]: from='client.? 172.21.15.172:0/3775988851' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "684e017e-bc9b-4fc5-afd8-89a3199c5653"}]: dispatch 2024-09-11T22:19:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:02 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "684e017e-bc9b-4fc5-afd8-89a3199c5653"}]: dispatch 2024-09-11T22:19:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:02 smithi172 ceph-mon[35461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "684e017e-bc9b-4fc5-afd8-89a3199c5653"}]': finished 2024-09-11T22:19:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:02 smithi172 ceph-mon[35461]: osdmap e70: 12 total, 11 up, 12 in 2024-09-11T22:19:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:02 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:02.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:02 smithi172 ceph-mon[35461]: from='client.? 172.21.15.172:0/339826043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T22:19:03.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:03 smithi172 ceph-mon[35461]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:19:03.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:03 smithi172 ceph-mon[35461]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T22:19:03.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:03 smithi005 ceph-mon[30943]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:19:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:03 smithi005 ceph-mon[30943]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T22:19:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:04.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:03 smithi159 ceph-mon[35534]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-11T22:19:04.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:03 smithi159 ceph-mon[35534]: osdmap e71: 12 total, 11 up, 12 in 2024-09-11T22:19:04.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:03 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:05 smithi172 ceph-mon[35461]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-11T22:19:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:05 smithi005 ceph-mon[30943]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-11T22:19:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:05 smithi159 ceph-mon[35534]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 138 KiB/s, 0 objects/s recovering 2024-09-11T22:19:07.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:07 smithi172 ceph-mon[35461]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T22:19:07.998 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:07 smithi159 ceph-mon[35534]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T22:19:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:07 smithi005 ceph-mon[30943]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T22:19:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:09 smithi172 ceph-mon[35461]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-11T22:19:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:09 smithi005 ceph-mon[30943]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-11T22:19:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:09 smithi159 ceph-mon[35534]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-11T22:19:10.823 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T22:19:10.823 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:10 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:11.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T22:19:11.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:10 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:11.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-11T22:19:11.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:10 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:11 smithi172 ceph-mon[35461]: Deploying daemon osd.11 on smithi172 2024-09-11T22:19:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:11 smithi172 ceph-mon[35461]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T22:19:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:11 smithi005 ceph-mon[30943]: Deploying daemon osd.11 on smithi172 2024-09-11T22:19:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:11 smithi005 ceph-mon[30943]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T22:19:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:11 smithi159 ceph-mon[35534]: Deploying daemon osd.11 on smithi172 2024-09-11T22:19:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:11 smithi159 ceph-mon[35534]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-11T22:19:13.724 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:13 smithi172 ceph-mon[35461]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:13.872 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:13 smithi005 ceph-mon[30943]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:14.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:13 smithi159 ceph-mon[35534]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:15 smithi172 ceph-mon[35461]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:16.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:15 smithi005 ceph-mon[30943]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:16.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:15 smithi159 ceph-mon[35534]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:17 smithi172 ceph-mon[35461]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:17 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:18.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:17 smithi005 ceph-mon[30943]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:18.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:18.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:18.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:17 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:18.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:17 smithi159 ceph-mon[35534]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:18.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:18.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:18.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:17 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:20.057 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:19 smithi172 ceph-mon[35461]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:19 smithi005 ceph-mon[30943]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:20.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:19 smithi159 ceph-mon[35534]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:20.101 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.101+0000 7f5cfaffd640 1 -- 172.21.15.172:0/2449995246 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5d08062630 con 0x7f5cf40a2c30 2024-09-11T22:19:20.101 INFO:teuthology.orchestra.run.smithi172.stdout:Created osd(s) 11 on host 'smithi172' 2024-09-11T22:19:20.103 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 -- 172.21.15.172:0/2449995246 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5cf40a2c30 msgr2=0x7f5cf40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5cf40a2c30 0x7f5cf40a50f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5d040096c0 tx=0x7f5d04031400 comp rx=0 tx=0).stop 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 -- 172.21.15.172:0/2449995246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d08101c00 msgr2=0x7f5d08197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d08101c00 0x7f5d08197f60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00d770 tx=0x7f5cfc00dc30 comp rx=0 tx=0).stop 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 -- 172.21.15.172:0/2449995246 shutdown_connections 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5cf40a2c30 0x7f5cf40a50f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d081037c0 0x7f5d0819f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d08102e00 0x7f5d081984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 --2- 172.21.15.172:0/2449995246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d08101c00 0x7f5d08197f60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 -- 172.21.15.172:0/2449995246 >> 172.21.15.172:0/2449995246 conn(0x7f5d080fd390 msgr2=0x7f5d08106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 -- 172.21.15.172:0/2449995246 shutdown_connections 2024-09-11T22:19:20.104 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:20.103+0000 7f5d0fb77640 1 -- 172.21.15.172:0/2449995246 wait complete. 2024-09-11T22:19:20.825 DEBUG:teuthology.orchestra.run.smithi172:osd.11> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.11.service 2024-09-11T22:19:20.827 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-11T22:19:20.827 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T22:19:21.098 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:21.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:21 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:21 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:21 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:21 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:21 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:21 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:21 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:21 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:21 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:21 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:21 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:21 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.658+0000 7fa6b15c1640 1 -- 172.21.15.5:0/54791152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6ac100ad0 msgr2=0x7fa6ac100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.658+0000 7fa6b15c1640 1 --2- 172.21.15.5:0/54791152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6ac100ad0 0x7fa6ac100ed0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009900 tx=0x7fa69c02a410 comp rx=0 tx=0).stop 2024-09-11T22:19:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 -- 172.21.15.5:0/54791152 shutdown_connections 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 --2- 172.21.15.5:0/54791152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ac102690 0x7fa6ac108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 --2- 172.21.15.5:0/54791152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac101cd0 0x7fa6ac102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 --2- 172.21.15.5:0/54791152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6ac100ad0 0x7fa6ac100ed0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 -- 172.21.15.5:0/54791152 >> 172.21.15.5:0/54791152 conn(0x7fa6ac0fc280 msgr2=0x7fa6ac0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 -- 172.21.15.5:0/54791152 shutdown_connections 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 -- 172.21.15.5:0/54791152 wait complete. 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 Processor -- start 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 -- start start 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6ac100ad0 0x7fa6ac197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.659+0000 7fa6b15c1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ac101cd0 0x7fa6ac1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6b15c1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102690 0x7fa6ac19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6b15c1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac10c750 con 0x7fa6ac100ad0 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6b15c1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac10c5d0 con 0x7fa6ac101cd0 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6b15c1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac10c8d0 con 0x7fa6ac102690 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6ac100ad0 0x7fa6ac197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6ac100ad0 0x7fa6ac197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45622/0 (socket says 172.21.15.5:45622) 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6aaffd640 1 -- 172.21.15.5:0/2802018887 learned_addr learned my addr 172.21.15.5:0/2802018887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6ab7fe640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102690 0x7fa6ac19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6aa7fc640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ac101cd0 0x7fa6ac1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6aaffd640 1 -- 172.21.15.5:0/2802018887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102690 msgr2=0x7fa6ac19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6aaffd640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102690 0x7fa6ac19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6aaffd640 1 -- 172.21.15.5:0/2802018887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ac101cd0 msgr2=0x7fa6ac1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6aaffd640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ac101cd0 0x7fa6ac1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6aaffd640 1 -- 172.21.15.5:0/2802018887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac19fad0 con 0x7fa6ac100ad0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6aaffd640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6ac100ad0 0x7fa6ac197ea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa69c031610 tx=0x7fa69c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa693fff640 1 -- 172.21.15.5:0/2802018887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c03f030 con 0x7fa6ac100ad0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa693fff640 1 -- 172.21.15.5:0/2802018887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa69c003ce0 con 0x7fa6ac100ad0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6b15c1640 1 -- 172.21.15.5:0/2802018887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac19fd60 con 0x7fa6ac100ad0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa693fff640 1 -- 172.21.15.5:0/2802018887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69c035070 con 0x7fa6ac100ad0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.660+0000 7fa6b15c1640 1 -- 172.21.15.5:0/2802018887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac1a01d0 con 0x7fa6ac100ad0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.661+0000 7fa6b15c1640 1 -- 172.21.15.5:0/2802018887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ac1a0400 con 0x7fa6ac100ad0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.662+0000 7fa693fff640 1 -- 172.21.15.5:0/2802018887 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa69c042030 con 0x7fa6ac100ad0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.663+0000 7fa693fff640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa6940a2c30 0x7fa6940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.663+0000 7fa6aa7fc640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa6940a2c30 0x7fa6940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.663+0000 7fa693fff640 1 -- 172.21.15.5:0/2802018887 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7fa69c0eef50 con 0x7fa6ac100ad0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.664+0000 7fa6aa7fc640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa6940a2c30 0x7fa6940a50f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0005fd0 tx=0x7fa6a0005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.667+0000 7fa693fff640 1 -- 172.21.15.5:0/2802018887 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa69c0bc490 con 0x7fa6ac100ad0 2024-09-11T22:19:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.842+0000 7fa6b15c1640 1 -- 172.21.15.5:0/2802018887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa6ac1a0760 con 0x7fa6ac100ad0 2024-09-11T22:19:21.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.843+0000 7fa693fff640 1 -- 172.21.15.5:0/2802018887 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7fa69c0bbd90 con 0x7fa6ac100ad0 2024-09-11T22:19:21.844 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 -- 172.21.15.5:0/2802018887 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa6940a2c30 msgr2=0x7fa6940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa6940a2c30 0x7fa6940a50f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0005fd0 tx=0x7fa6a0005850 comp rx=0 tx=0).stop 2024-09-11T22:19:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 -- 172.21.15.5:0/2802018887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6ac100ad0 msgr2=0x7fa6ac197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6ac100ad0 0x7fa6ac197ea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa69c031610 tx=0x7fa69c0043b0 comp rx=0 tx=0).stop 2024-09-11T22:19:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 -- 172.21.15.5:0/2802018887 shutdown_connections 2024-09-11T22:19:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa6940a2c30 0x7fa6940a50f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6ac102690 0x7fa6ac19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6ac101cd0 0x7fa6ac1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:21.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 --2- 172.21.15.5:0/2802018887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6ac100ad0 0x7fa6ac197ea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 -- 172.21.15.5:0/2802018887 >> 172.21.15.5:0/2802018887 conn(0x7fa6ac0fc280 msgr2=0x7fa6ac069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 -- 172.21.15.5:0/2802018887 shutdown_connections 2024-09-11T22:19:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:21.845+0000 7fa6b15c1640 1 -- 172.21.15.5:0/2802018887 wait complete. 2024-09-11T22:19:22.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:22 smithi172 ceph-mon[35461]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:22.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:22 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2802018887' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:19:22.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:22 smithi005 ceph-mon[30943]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:22.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:22 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2802018887' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:19:22.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:22 smithi159 ceph-mon[35534]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:22.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:22 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2802018887' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:19:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726093139,"num_in_osds":12,"osd_in_since":1726093141,"num_remapped_pgs":0} 2024-09-11T22:19:22.773 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:19:22 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:19:22.688+0000 7f0f383d4640 -1 osd.11 0 log_to_monitors true 2024-09-11T22:19:23.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:23 smithi172 ceph-mon[35461]: from='osd.11 [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:19:23.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:23 smithi172 ceph-mon[35461]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:19:23.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:23 smithi005 ceph-mon[30943]: from='osd.11 [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:19:23.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:23 smithi005 ceph-mon[30943]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:19:23.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:23 smithi159 ceph-mon[35534]: from='osd.11 [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:19:23.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:23 smithi159 ceph-mon[35534]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-11T22:19:23.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T22:19:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:24.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:24 smithi005 ceph-mon[30943]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:24.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:24 smithi005 ceph-mon[30943]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T22:19:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:24 smithi005 ceph-mon[30943]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T22:19:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:24 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:24 smithi005 ceph-mon[30943]: from='osd.11 [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:19:24.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:24 smithi005 ceph-mon[30943]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:19:24.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.401+0000 7f9e8450d640 1 -- 172.21.15.5:0/1558737619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7c100ad0 msgr2=0x7f9e7c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:24.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.401+0000 7f9e8450d640 1 --2- 172.21.15.5:0/1558737619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7c100ad0 0x7f9e7c100ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009900 tx=0x7f9e6c02a410 comp rx=0 tx=0).stop 2024-09-11T22:19:24.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 -- 172.21.15.5:0/1558737619 shutdown_connections 2024-09-11T22:19:24.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 --2- 172.21.15.5:0/1558737619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7c102690 0x7f9e7c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 --2- 172.21.15.5:0/1558737619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e7c101cd0 0x7f9e7c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 --2- 172.21.15.5:0/1558737619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7c100ad0 0x7f9e7c100ed0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 -- 172.21.15.5:0/1558737619 >> 172.21.15.5:0/1558737619 conn(0x7f9e7c0fc280 msgr2=0x7f9e7c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 -- 172.21.15.5:0/1558737619 shutdown_connections 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 -- 172.21.15.5:0/1558737619 wait complete. 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 Processor -- start 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 -- start start 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e7c100ad0 0x7f9e7c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7c101cd0 0x7f9e7c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7c102690 0x7f9e7c19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c10c7d0 con 0x7f9e7c102690 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c10c650 con 0x7f9e7c100ad0 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.402+0000 7f9e8450d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c10c950 con 0x7f9e7c101cd0 2024-09-11T22:19:24.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e82aaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7c102690 0x7f9e7c19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e82aaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7c102690 0x7f9e7c19f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45640/0 (socket says 172.21.15.5:45640) 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e82aaa640 1 -- 172.21.15.5:0/3243463537 learned_addr learned my addr 172.21.15.5:0/3243463537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e822a9640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e7c100ad0 0x7f9e7c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e81aa8640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7c101cd0 0x7f9e7c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e82aaa640 1 -- 172.21.15.5:0/3243463537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7c101cd0 msgr2=0x7f9e7c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e82aaa640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7c101cd0 0x7f9e7c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e82aaa640 1 -- 172.21.15.5:0/3243463537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e7c100ad0 msgr2=0x7f9e7c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e82aaa640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e7c100ad0 0x7f9e7c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e82aaa640 1 -- 172.21.15.5:0/3243463537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c19fbd0 con 0x7f9e7c102690 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e82aaa640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7c102690 0x7f9e7c19f560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400eaa0 tx=0x7f9e7400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e737fe640 1 -- 172.21.15.5:0/3243463537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74004d20 con 0x7f9e7c102690 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e737fe640 1 -- 172.21.15.5:0/3243463537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e74005020 con 0x7f9e7c102690 2024-09-11T22:19:24.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e8450d640 1 -- 172.21.15.5:0/3243463537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c19fec0 con 0x7f9e7c102690 2024-09-11T22:19:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e737fe640 1 -- 172.21.15.5:0/3243463537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e740116d0 con 0x7f9e7c102690 2024-09-11T22:19:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.403+0000 7f9e8450d640 1 -- 172.21.15.5:0/3243463537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c1a03f0 con 0x7f9e7c102690 2024-09-11T22:19:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.404+0000 7f9e8450d640 1 -- 172.21.15.5:0/3243463537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7c1a06b0 con 0x7f9e7c102690 2024-09-11T22:19:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.407+0000 7f9e737fe640 1 -- 172.21.15.5:0/3243463537 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9e740051d0 con 0x7f9e7c102690 2024-09-11T22:19:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.407+0000 7f9e737fe640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9e680a2bb0 0x7f9e680a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.408+0000 7f9e822a9640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9e680a2bb0 0x7f9e680a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.408+0000 7f9e737fe640 1 -- 172.21.15.5:0/3243463537 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6847+0+0 (secure 0 0 0) 0x7f9e740ceff0 con 0x7f9e7c102690 2024-09-11T22:19:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.408+0000 7f9e822a9640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9e680a2bb0 0x7f9e680a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c005ab0 tx=0x7f9e6c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:24.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.410+0000 7f9e737fe640 1 -- 172.21.15.5:0/3243463537 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e74016030 con 0x7f9e7c102690 2024-09-11T22:19:24.471 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:24 smithi172 ceph-mon[35461]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:24.471 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:24 smithi172 ceph-mon[35461]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T22:19:24.471 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:24 smithi172 ceph-mon[35461]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T22:19:24.471 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:24 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:24.471 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:24 smithi172 ceph-mon[35461]: from='osd.11 [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:19:24.472 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:24 smithi172 ceph-mon[35461]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:19:24.472 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:19:24 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:19:24.115+0000 7f0f34357640 -1 osd.11 0 waiting for initial osdmap 2024-09-11T22:19:24.472 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:19:24 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:19:24.126+0000 7f0f3017f640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T22:19:24.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:24 smithi159 ceph-mon[35534]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:24.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:24 smithi159 ceph-mon[35534]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-11T22:19:24.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:24 smithi159 ceph-mon[35534]: osdmap e72: 12 total, 11 up, 12 in 2024-09-11T22:19:24.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:24 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:24.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:24 smithi159 ceph-mon[35534]: from='osd.11 [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:19:24.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:24 smithi159 ceph-mon[35534]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]: dispatch 2024-09-11T22:19:24.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.608+0000 7f9e8450d640 1 -- 172.21.15.5:0/3243463537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f9e7c1a0920 con 0x7f9e7c102690 2024-09-11T22:19:24.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.608+0000 7f9e737fe640 1 -- 172.21.15.5:0/3243463537 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f9e7409c490 con 0x7f9e7c102690 2024-09-11T22:19:24.609 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 -- 172.21.15.5:0/3243463537 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9e680a2bb0 msgr2=0x7f9e680a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9e680a2bb0 0x7f9e680a5070 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c005ab0 tx=0x7f9e6c000bc0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 -- 172.21.15.5:0/3243463537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7c102690 msgr2=0x7f9e7c19f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7c102690 0x7f9e7c19f560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400eaa0 tx=0x7f9e7400eea0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 -- 172.21.15.5:0/3243463537 shutdown_connections 2024-09-11T22:19:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9e680a2bb0 0x7f9e680a5070 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e7c102690 0x7f9e7c19f560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e7c101cd0 0x7f9e7c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 --2- 172.21.15.5:0/3243463537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e7c100ad0 0x7f9e7c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 -- 172.21.15.5:0/3243463537 >> 172.21.15.5:0/3243463537 conn(0x7f9e7c0fc280 msgr2=0x7f9e7c06a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 -- 172.21.15.5:0/3243463537 shutdown_connections 2024-09-11T22:19:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:24.610+0000 7f9e8450d640 1 -- 172.21.15.5:0/3243463537 wait complete. 2024-09-11T22:19:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:19:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T22:19:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3243463537' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:25.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:25.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.465 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726093139,"num_in_osds":12,"osd_in_since":1726093141,"num_remapped_pgs":0} 2024-09-11T22:19:25.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:19:25.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T22:19:25.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:25.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3243463537' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:25.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:25.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:25.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi172", "root=default"]}]': finished 2024-09-11T22:19:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: osdmap e73: 12 total, 11 up, 12 in 2024-09-11T22:19:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3243463537' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:25.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:26.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T22:19:26.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:26 smithi159 ceph-mon[35534]: purged_snaps scrub starts 2024-09-11T22:19:26.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:26 smithi159 ceph-mon[35534]: purged_snaps scrub ok 2024-09-11T22:19:26.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:26 smithi159 ceph-mon[35534]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:26.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:26 smithi159 ceph-mon[35534]: Detected new or changed devices on smithi172 2024-09-11T22:19:26.481 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:26 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-11T22:19:26.481 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:26 smithi159 ceph-mon[35534]: osd.11 [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] boot 2024-09-11T22:19:26.481 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:26 smithi159 ceph-mon[35534]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T22:19:26.481 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:26 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:26.495 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:26 smithi005 ceph-mon[30943]: purged_snaps scrub starts 2024-09-11T22:19:26.495 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:26 smithi005 ceph-mon[30943]: purged_snaps scrub ok 2024-09-11T22:19:26.495 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:26 smithi005 ceph-mon[30943]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:26.495 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:26 smithi005 ceph-mon[30943]: Detected new or changed devices on smithi172 2024-09-11T22:19:26.495 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:26 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-11T22:19:26.496 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:26 smithi005 ceph-mon[30943]: osd.11 [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] boot 2024-09-11T22:19:26.496 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:26 smithi005 ceph-mon[30943]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T22:19:26.496 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:26 smithi172 ceph-mon[35461]: purged_snaps scrub starts 2024-09-11T22:19:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:26 smithi172 ceph-mon[35461]: purged_snaps scrub ok 2024-09-11T22:19:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:26 smithi172 ceph-mon[35461]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-09-11T22:19:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:26 smithi172 ceph-mon[35461]: Detected new or changed devices on smithi172 2024-09-11T22:19:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:26 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi172 to 4272M 2024-09-11T22:19:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:26 smithi172 ceph-mon[35461]: osd.11 [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] boot 2024-09-11T22:19:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:26 smithi172 ceph-mon[35461]: osdmap e74: 12 total, 12 up, 12 in 2024-09-11T22:19:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:26 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:19:26.735 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:27.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:27 smithi005 ceph-mon[30943]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T22:19:27.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.235+0000 7fe1856a4640 1 -- 172.21.15.5:0/1585917270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe180100b10 msgr2=0x7fe180100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:27.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.235+0000 7fe1856a4640 1 --2- 172.21.15.5:0/1585917270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe180100b10 0x7fe180100f10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe170009900 tx=0x7fe17002a410 comp rx=0 tx=0).stop 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.235+0000 7fe1856a4640 1 -- 172.21.15.5:0/1585917270 shutdown_connections 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.235+0000 7fe1856a4640 1 --2- 172.21.15.5:0/1585917270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1801026d0 0x7fe180108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 --2- 172.21.15.5:0/1585917270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe180101d10 0x7fe180102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 --2- 172.21.15.5:0/1585917270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe180100b10 0x7fe180100f10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 -- 172.21.15.5:0/1585917270 >> 172.21.15.5:0/1585917270 conn(0x7fe1800fc2a0 msgr2=0x7fe1800fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 -- 172.21.15.5:0/1585917270 shutdown_connections 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 -- 172.21.15.5:0/1585917270 wait complete. 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 Processor -- start 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 -- start start 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe180100b10 0x7fe180104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe180101d10 0x7fe1801030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801026d0 0x7fe180103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe18010c840 con 0x7fe1801026d0 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe18010c6c0 con 0x7fe180101d10 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe1856a4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe18010c9c0 con 0x7fe180100b10 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe17f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801026d0 0x7fe180103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe17f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801026d0 0x7fe180103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45658/0 (socket says 172.21.15.5:45658) 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.236+0000 7fe17f7fe640 1 -- 172.21.15.5:0/4236509819 learned_addr learned my addr 172.21.15.5:0/4236509819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17effd640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe180100b10 0x7fe180104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe177fff640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe180101d10 0x7fe1801030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17f7fe640 1 -- 172.21.15.5:0/4236509819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe180100b10 msgr2=0x7fe180104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17f7fe640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe180100b10 0x7fe180104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17f7fe640 1 -- 172.21.15.5:0/4236509819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe180101d10 msgr2=0x7fe1801030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17f7fe640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe180101d10 0x7fe1801030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17f7fe640 1 -- 172.21.15.5:0/4236509819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe180103ef0 con 0x7fe1801026d0 2024-09-11T22:19:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17effd640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe180100b10 0x7fe180104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:19:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe177fff640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe180101d10 0x7fe1801030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:19:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17f7fe640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801026d0 0x7fe180103630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe17800eaa0 tx=0x7fe17800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17cff9640 1 -- 172.21.15.5:0/4236509819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17800cdb0 con 0x7fe1801026d0 2024-09-11T22:19:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17cff9640 1 -- 172.21.15.5:0/4236509819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe178005020 con 0x7fe1801026d0 2024-09-11T22:19:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe1856a4640 1 -- 172.21.15.5:0/4236509819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1801a2110 con 0x7fe1801026d0 2024-09-11T22:19:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe17cff9640 1 -- 172.21.15.5:0/4236509819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1780116b0 con 0x7fe1801026d0 2024-09-11T22:19:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.237+0000 7fe1856a4640 1 -- 172.21.15.5:0/4236509819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1801a2650 con 0x7fe1801026d0 2024-09-11T22:19:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.238+0000 7fe1856a4640 1 -- 172.21.15.5:0/4236509819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1801a28e0 con 0x7fe1801026d0 2024-09-11T22:19:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.240+0000 7fe17cff9640 1 -- 172.21.15.5:0/4236509819 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe178005560 con 0x7fe1801026d0 2024-09-11T22:19:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.240+0000 7fe17cff9640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe16c0a2b80 0x7fe16c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.240+0000 7fe17effd640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe16c0a2b80 0x7fe16c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.241+0000 7fe17cff9640 1 -- 172.21.15.5:0/4236509819 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe1780ceed0 con 0x7fe1801026d0 2024-09-11T22:19:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.241+0000 7fe17effd640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe16c0a2b80 0x7fe16c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe17002aa00 tx=0x7fe170000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.245+0000 7fe17cff9640 1 -- 172.21.15.5:0/4236509819 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe178016030 con 0x7fe1801026d0 2024-09-11T22:19:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.420+0000 7fe1856a4640 1 -- 172.21.15.5:0/4236509819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe1801a2ab0 con 0x7fe1801026d0 2024-09-11T22:19:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.421+0000 7fe17cff9640 1 -- 172.21.15.5:0/4236509819 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fe17809c2d0 con 0x7fe1801026d0 2024-09-11T22:19:27.421 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.422+0000 7fe1856a4640 1 -- 172.21.15.5:0/4236509819 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe16c0a2b80 msgr2=0x7fe16c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.422+0000 7fe1856a4640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe16c0a2b80 0x7fe16c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe17002aa00 tx=0x7fe170000bc0 comp rx=0 tx=0).stop 2024-09-11T22:19:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.422+0000 7fe1856a4640 1 -- 172.21.15.5:0/4236509819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801026d0 msgr2=0x7fe180103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.422+0000 7fe1856a4640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801026d0 0x7fe180103630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe17800eaa0 tx=0x7fe17800ef60 comp rx=0 tx=0).stop 2024-09-11T22:19:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.422+0000 7fe1856a4640 1 -- 172.21.15.5:0/4236509819 shutdown_connections 2024-09-11T22:19:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.423+0000 7fe1856a4640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe16c0a2b80 0x7fe16c0a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.423+0000 7fe1856a4640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1801026d0 0x7fe180103630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.423+0000 7fe1856a4640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe180101d10 0x7fe1801030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.423+0000 7fe1856a4640 1 --2- 172.21.15.5:0/4236509819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe180100b10 0x7fe180104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.423+0000 7fe1856a4640 1 -- 172.21.15.5:0/4236509819 >> 172.21.15.5:0/4236509819 conn(0x7fe1800fc2a0 msgr2=0x7fe18006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.423+0000 7fe1856a4640 1 -- 172.21.15.5:0/4236509819 shutdown_connections 2024-09-11T22:19:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:27.423+0000 7fe1856a4640 1 -- 172.21.15.5:0/4236509819 wait complete. 2024-09-11T22:19:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:27 smithi159 ceph-mon[35534]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T22:19:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:27 smithi172 ceph-mon[35461]: osdmap e75: 12 total, 12 up, 12 in 2024-09-11T22:19:28.211 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:28 smithi005 ceph-mon[30943]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:28.211 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:28 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4236509819' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:19:28.245 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1726093165,"num_in_osds":12,"osd_in_since":1726093141,"num_remapped_pgs":0} 2024-09-11T22:19:28.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:19:28.511 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:28 smithi172 ceph-mon[35461]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:28.511 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:28 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4236509819' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:19:28.551 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:28.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:28 smithi159 ceph-mon[35534]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:28.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:28 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4236509819' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T22:19:29.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.044+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2382484524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d810c170 msgr2=0x7f65d810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:29.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.044+0000 7f65ddb8e640 1 --2- 172.21.15.5:0/2382484524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d810c170 0x7f65d810e610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009ca0 tx=0x7f65d002a950 comp rx=0 tx=0).stop 2024-09-11T22:19:29.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.044+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2382484524 shutdown_connections 2024-09-11T22:19:29.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.044+0000 7f65ddb8e640 1 --2- 172.21.15.5:0/2382484524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d810c170 0x7f65d810e610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.044+0000 7f65ddb8e640 1 --2- 172.21.15.5:0/2382484524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8103660 0x7f65d810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.044+0000 7f65ddb8e640 1 --2- 172.21.15.5:0/2382484524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65d8102d40 0x7f65d8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.044+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2382484524 >> 172.21.15.5:0/2382484524 conn(0x7f65d80fc7a0 msgr2=0x7f65d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.044+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2382484524 shutdown_connections 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.045+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2382484524 wait complete. 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.045+0000 7f65ddb8e640 1 Processor -- start 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.045+0000 7f65ddb8e640 1 -- start start 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.045+0000 7f65ddb8e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8102d40 0x7f65d81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.045+0000 7f65ddb8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8103660 0x7f65d8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.045+0000 7f65ddb8e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65d810c170 0x7f65d819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.045+0000 7f65ddb8e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8111fb0 con 0x7f65d8103660 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.045+0000 7f65ddb8e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8111e30 con 0x7f65d810c170 2024-09-11T22:19:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.045+0000 7f65ddb8e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8112130 con 0x7f65d8102d40 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8103660 0x7f65d8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8103660 0x7f65d8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45680/0 (socket says 172.21.15.5:45680) 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d6ffd640 1 -- 172.21.15.5:0/2346046692 learned_addr learned my addr 172.21.15.5:0/2346046692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d77fe640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8102d40 0x7f65d81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d7fff640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65d810c170 0x7f65d819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d6ffd640 1 -- 172.21.15.5:0/2346046692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8102d40 msgr2=0x7f65d81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d6ffd640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8102d40 0x7f65d81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d6ffd640 1 -- 172.21.15.5:0/2346046692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65d810c170 msgr2=0x7f65d819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d6ffd640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65d810c170 0x7f65d819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d6ffd640 1 -- 172.21.15.5:0/2346046692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d819d380 con 0x7f65d8103660 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d7fff640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65d810c170 0x7f65d819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d6ffd640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8103660 0x7f65d8198910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00b3f0 tx=0x7f65cc00b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d4ff9640 1 -- 172.21.15.5:0/2346046692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc015020 con 0x7f65d8103660 2024-09-11T22:19:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d4ff9640 1 -- 172.21.15.5:0/2346046692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65cc011d10 con 0x7f65d8103660 2024-09-11T22:19:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65d4ff9640 1 -- 172.21.15.5:0/2346046692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc010e60 con 0x7f65d8103660 2024-09-11T22:19:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2346046692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d819d670 con 0x7f65d8103660 2024-09-11T22:19:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.046+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2346046692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81a4f50 con 0x7f65d8103660 2024-09-11T22:19:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.047+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2346046692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d8103120 con 0x7f65d8103660 2024-09-11T22:19:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.049+0000 7f65d4ff9640 1 -- 172.21.15.5:0/2346046692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f65cc018020 con 0x7f65d8103660 2024-09-11T22:19:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.049+0000 7f65d4ff9640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f65c00a2b80 0x7f65c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.049+0000 7f65d77fe640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f65c00a2b80 0x7f65c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.049+0000 7f65d4ff9640 1 -- 172.21.15.5:0/2346046692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f65cc0ce9c0 con 0x7f65d8103660 2024-09-11T22:19:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.050+0000 7f65d77fe640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f65c00a2b80 0x7f65c00a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f65c800abe0 tx=0x7f65c800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:29.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.052+0000 7f65d4ff9640 1 -- 172.21.15.5:0/2346046692 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65cc01f020 con 0x7f65d8103660 2024-09-11T22:19:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.229+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2346046692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65d8199730 con 0x7f65d8103660 2024-09-11T22:19:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.230+0000 7f65d4ff9640 1 -- 172.21.15.5:0/2346046692 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18362 (secure 0 0 0) 0x7f65d8199730 con 0x7f65d8103660 2024-09-11T22:19:29.230 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:29.231 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:19:26.118838+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2346046692 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f65c00a2b80 msgr2=0x7f65c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f65c00a2b80 0x7f65c00a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f65c800abe0 tx=0x7f65c800a2d0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2346046692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8103660 msgr2=0x7f65d8198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8103660 0x7f65d8198910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00b3f0 tx=0x7f65cc00b7f0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2346046692 shutdown_connections 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f65c00a2b80 0x7f65c00a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f65d810c170 0x7f65d819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8103660 0x7f65d8198910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 --2- 172.21.15.5:0/2346046692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65d8102d40 0x7f65d81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2346046692 >> 172.21.15.5:0/2346046692 conn(0x7f65d80fc7a0 msgr2=0x7f65d80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2346046692 shutdown_connections 2024-09-11T22:19:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:29.232+0000 7f65ddb8e640 1 -- 172.21.15.5:0/2346046692 wait complete. 2024-09-11T22:19:30.054 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T22:15:32.851002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T22:19:30.055 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T22:19:30.314 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:30.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:30 smithi005 ceph-mon[30943]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:30.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:30 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2346046692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:19:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:30 smithi159 ceph-mon[35534]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:30 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2346046692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:19:30.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:30 smithi172 ceph-mon[35461]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:30.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:30 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2346046692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:19:30.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.817+0000 7fdd15447640 1 -- 172.21.15.5:0/3288469153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10101de0 msgr2=0x7fdd101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:30.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.817+0000 7fdd15447640 1 --2- 172.21.15.5:0/3288469153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10101de0 0x7fdd101021c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009900 tx=0x7fdd0002a410 comp rx=0 tx=0).stop 2024-09-11T22:19:30.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.817+0000 7fdd15447640 1 -- 172.21.15.5:0/3288469153 shutdown_connections 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.817+0000 7fdd15447640 1 --2- 172.21.15.5:0/3288469153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd10106f10 0x7fdd1010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.817+0000 7fdd15447640 1 --2- 172.21.15.5:0/3288469153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd10102790 0x7fdd101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.817+0000 7fdd15447640 1 --2- 172.21.15.5:0/3288469153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10101de0 0x7fdd101021c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.817+0000 7fdd15447640 1 -- 172.21.15.5:0/3288469153 >> 172.21.15.5:0/3288469153 conn(0x7fdd100fd8c0 msgr2=0x7fdd100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.817+0000 7fdd15447640 1 -- 172.21.15.5:0/3288469153 shutdown_connections 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.817+0000 7fdd15447640 1 -- 172.21.15.5:0/3288469153 wait complete. 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd15447640 1 Processor -- start 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd15447640 1 -- start start 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd15447640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd10101de0 0x7fdd101983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd15447640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd10102790 0x7fdd10198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd15447640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10106f10 0x7fdd1019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:30.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd15447640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10111f70 con 0x7fdd10106f10 2024-09-11T22:19:30.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd15447640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd10111df0 con 0x7fdd10102790 2024-09-11T22:19:30.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd15447640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd101120f0 con 0x7fdd10101de0 2024-09-11T22:19:30.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10106f10 0x7fdd1019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:30.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd07fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd10102790 0x7fdd10198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:30.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10106f10 0x7fdd1019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45706/0 (socket says 172.21.15.5:45706) 2024-09-11T22:19:30.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd0effd640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd10101de0 0x7fdd101983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:30.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.818+0000 7fdd0f7fe640 1 -- 172.21.15.5:0/649653555 learned_addr learned my addr 172.21.15.5:0/649653555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:30.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd07fff640 1 -- 172.21.15.5:0/649653555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd10101de0 msgr2=0x7fdd101983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:30.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd07fff640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd10101de0 0x7fdd101983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd07fff640 1 -- 172.21.15.5:0/649653555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10106f10 msgr2=0x7fdd1019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd07fff640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10106f10 0x7fdd1019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd07fff640 1 -- 172.21.15.5:0/649653555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd1019d380 con 0x7fdd10102790 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd0effd640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd10101de0 0x7fdd101983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd0f7fe640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10106f10 0x7fdd1019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd07fff640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd10102790 0x7fdd10198910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800d720 tx=0x7fdcf800dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd0cff9640 1 -- 172.21.15.5:0/649653555 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8017070 con 0x7fdd10102790 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd15447640 1 -- 172.21.15.5:0/649653555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd1019d670 con 0x7fdd10102790 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd0cff9640 1 -- 172.21.15.5:0/649653555 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcf800de30 con 0x7fdd10102790 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd0cff9640 1 -- 172.21.15.5:0/649653555 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80047a0 con 0x7fdd10102790 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.819+0000 7fdd15447640 1 -- 172.21.15.5:0/649653555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd101a4f50 con 0x7fdd10102790 2024-09-11T22:19:30.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.820+0000 7fdd15447640 1 -- 172.21.15.5:0/649653555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd101021c0 con 0x7fdd10102790 2024-09-11T22:19:30.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.822+0000 7fdd0cff9640 1 -- 172.21.15.5:0/649653555 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdcf80099d0 con 0x7fdd10102790 2024-09-11T22:19:30.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.822+0000 7fdd0cff9640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdcfc0a2b80 0x7fdcfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:30.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.822+0000 7fdd0effd640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdcfc0a2b80 0x7fdcfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:30.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.823+0000 7fdd0cff9640 1 -- 172.21.15.5:0/649653555 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fdcf80cec80 con 0x7fdd10102790 2024-09-11T22:19:30.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.823+0000 7fdd0effd640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdcfc0a2b80 0x7fdcfc0a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009230 tx=0x7fdd000058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:30.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:30.826+0000 7fdd0cff9640 1 -- 172.21.15.5:0/649653555 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdcf8020020 con 0x7fdd10102790 2024-09-11T22:19:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.002+0000 7fdd15447640 1 -- 172.21.15.5:0/649653555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd1010a150 con 0x7fdd10102790 2024-09-11T22:19:31.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.003+0000 7fdd0cff9640 1 -- 172.21.15.5:0/649653555 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7fdcf809bfe0 con 0x7fdd10102790 2024-09-11T22:19:31.003 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-11T22:19:31.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 -- 172.21.15.5:0/649653555 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdcfc0a2b80 msgr2=0x7fdcfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:31.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdcfc0a2b80 0x7fdcfc0a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009230 tx=0x7fdd000058d0 comp rx=0 tx=0).stop 2024-09-11T22:19:31.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 -- 172.21.15.5:0/649653555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd10102790 msgr2=0x7fdd10198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:31.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd10102790 0x7fdd10198910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800d720 tx=0x7fdcf800dbe0 comp rx=0 tx=0).stop 2024-09-11T22:19:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 -- 172.21.15.5:0/649653555 shutdown_connections 2024-09-11T22:19:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdcfc0a2b80 0x7fdcfc0a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10106f10 0x7fdd1019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd10102790 0x7fdd10198910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 --2- 172.21.15.5:0/649653555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd10101de0 0x7fdd101983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 -- 172.21.15.5:0/649653555 >> 172.21.15.5:0/649653555 conn(0x7fdd100fd8c0 msgr2=0x7fdd100febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 -- 172.21.15.5:0/649653555 shutdown_connections 2024-09-11T22:19:31.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:31.005+0000 7fdd15447640 1 -- 172.21.15.5:0/649653555 wait complete. 2024-09-11T22:19:31.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:31 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/649653555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:19:31.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:31 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/649653555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:19:32.108 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:31 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/649653555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:19:32.109 INFO:tasks.cephadm:Adding mds.a on smithi005 2024-09-11T22:19:32.109 INFO:tasks.cephadm:Adding mds.d on smithi005 2024-09-11T22:19:32.110 INFO:tasks.cephadm:Adding mds.g on smithi005 2024-09-11T22:19:32.110 INFO:tasks.cephadm:Adding mds.j on smithi005 2024-09-11T22:19:32.110 INFO:tasks.cephadm:Adding mds.b on smithi159 2024-09-11T22:19:32.110 INFO:tasks.cephadm:Adding mds.e on smithi159 2024-09-11T22:19:32.110 INFO:tasks.cephadm:Adding mds.h on smithi159 2024-09-11T22:19:32.110 INFO:tasks.cephadm:Adding mds.k on smithi159 2024-09-11T22:19:32.110 INFO:tasks.cephadm:Adding mds.c on smithi172 2024-09-11T22:19:32.110 INFO:tasks.cephadm:Adding mds.f on smithi172 2024-09-11T22:19:32.110 INFO:tasks.cephadm:Adding mds.i on smithi172 2024-09-11T22:19:32.110 INFO:tasks.cephadm:Adding mds.l on smithi172 2024-09-11T22:19:32.110 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l' 2024-09-11T22:19:32.370 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:19:32.491 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:32 smithi172 ceph-mon[35461]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:32.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:32 smithi005 ceph-mon[30943]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:32.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:32 smithi159 ceph-mon[35534]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:32.882 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.857+0000 7f1253371640 1 -- 172.21.15.172:0/2253059127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f124c1026a0 msgr2=0x7f124c104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.857+0000 7f1253371640 1 --2- 172.21.15.172:0/2253059127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f124c1026a0 0x7f124c104a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1248009900 tx=0x7f124802a410 comp rx=0 tx=0).stop 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.857+0000 7f1253371640 1 -- 172.21.15.172:0/2253059127 shutdown_connections 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.857+0000 7f1253371640 1 --2- 172.21.15.172:0/2253059127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f124c107900 0x7f124c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.857+0000 7f1253371640 1 --2- 172.21.15.172:0/2253059127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f124c104fd0 0x7f124c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.857+0000 7f1253371640 1 --2- 172.21.15.172:0/2253059127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f124c1026a0 0x7f124c104a90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.857+0000 7f1253371640 1 -- 172.21.15.172:0/2253059127 >> 172.21.15.172:0/2253059127 conn(0x7f124c0fc2a0 msgr2=0x7f124c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.857+0000 7f1253371640 1 -- 172.21.15.172:0/2253059127 shutdown_connections 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.857+0000 7f1253371640 1 -- 172.21.15.172:0/2253059127 wait complete. 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f1253371640 1 Processor -- start 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f1253371640 1 -- start start 2024-09-11T22:19:32.883 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f1253371640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f124c1026a0 0x7f124c102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f1253371640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f124c104fd0 0x7f124c100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f1253371640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f124c107900 0x7f124c100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f1253371640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f124c10d970 con 0x7f124c1026a0 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f1253371640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f124c10d7f0 con 0x7f124c104fd0 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f1253371640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f124c10daf0 con 0x7f124c107900 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f125190e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f124c107900 0x7f124c100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f125190e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f124c107900 0x7f124c100b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:36420/0 (socket says 172.21.15.172:36420) 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f125190e640 1 -- 172.21.15.172:0/2774673503 learned_addr learned my addr 172.21.15.172:0/2774673503 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.858+0000 7f125090c640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f124c104fd0 0x7f124c100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f125110d640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f124c1026a0 0x7f124c102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f125190e640 1 -- 172.21.15.172:0/2774673503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f124c104fd0 msgr2=0x7f124c100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:32.884 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f125190e640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f124c104fd0 0x7f124c100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f125190e640 1 -- 172.21.15.172:0/2774673503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f124c1026a0 msgr2=0x7f124c102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f125190e640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f124c1026a0 0x7f124c102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f125190e640 1 -- 172.21.15.172:0/2774673503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c101450 con 0x7f124c107900 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f125190e640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f124c107900 0x7f124c100b90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f124400eaa0 tx=0x7f124400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f12427fc640 1 -- 172.21.15.172:0/2774673503 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124400cdb0 con 0x7f124c107900 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f12427fc640 1 -- 172.21.15.172:0/2774673503 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1244005020 con 0x7f124c107900 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f1253371640 1 -- 172.21.15.172:0/2774673503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c1a2130 con 0x7f124c107900 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f12427fc640 1 -- 172.21.15.172:0/2774673503 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12440116b0 con 0x7f124c107900 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.859+0000 7f1253371640 1 -- 172.21.15.172:0/2774673503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c1a26c0 con 0x7f124c107900 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.860+0000 7f1253371640 1 -- 172.21.15.172:0/2774673503 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124c0fdd90 con 0x7f124c107900 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.861+0000 7f12427fc640 1 -- 172.21.15.172:0/2774673503 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1244005560 con 0x7f124c107900 2024-09-11T22:19:32.885 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.862+0000 7f12427fc640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12380a2b80 0x7f12380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:32.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.862+0000 7f12427fc640 1 -- 172.21.15.172:0/2774673503 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f12440ceea0 con 0x7f124c107900 2024-09-11T22:19:32.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.862+0000 7f125110d640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12380a2b80 0x7f12380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:32.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.862+0000 7f125110d640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12380a2b80 0x7f12380a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1248031610 tx=0x7f1248005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:32.886 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:32.866+0000 7f12427fc640 1 -- 172.21.15.172:0/2774673503 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1244016030 con 0x7f124c107900 2024-09-11T22:19:33.042 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.042+0000 7f1253371640 1 -- 172.21.15.172:0/2774673503 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}) -- 0x7f124c062730 con 0x7f12380a2b80 2024-09-11T22:19:33.050 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.050+0000 7f12427fc640 1 -- 172.21.15.172:0/2774673503 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f124c062730 con 0x7f12380a2b80 2024-09-11T22:19:33.050 INFO:teuthology.orchestra.run.smithi172.stdout:Scheduled mds.all update... 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 -- 172.21.15.172:0/2774673503 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12380a2b80 msgr2=0x7f12380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12380a2b80 0x7f12380a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1248031610 tx=0x7f1248005fd0 comp rx=0 tx=0).stop 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 -- 172.21.15.172:0/2774673503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f124c107900 msgr2=0x7f124c100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f124c107900 0x7f124c100b90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f124400eaa0 tx=0x7f124400ef60 comp rx=0 tx=0).stop 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 -- 172.21.15.172:0/2774673503 shutdown_connections 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12380a2b80 0x7f12380a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f124c107900 0x7f124c100b90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f124c104fd0 0x7f124c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 --2- 172.21.15.172:0/2774673503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f124c1026a0 0x7f124c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 -- 172.21.15.172:0/2774673503 >> 172.21.15.172:0/2774673503 conn(0x7f124c0fc2a0 msgr2=0x7f124c103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:33.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 -- 172.21.15.172:0/2774673503 shutdown_connections 2024-09-11T22:19:33.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-09-11T22:19:33.052+0000 7f1253371640 1 -- 172.21.15.172:0/2774673503 wait complete. 2024-09-11T22:19:33.765 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.a.service 2024-09-11T22:19:33.768 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.d.service 2024-09-11T22:19:33.811 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.g.service 2024-09-11T22:19:33.853 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.j.service 2024-09-11T22:19:33.896 DEBUG:teuthology.orchestra.run.smithi159:mds.b> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.b.service 2024-09-11T22:19:33.899 DEBUG:teuthology.orchestra.run.smithi159:mds.e> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.e.service 2024-09-11T22:19:33.942 DEBUG:teuthology.orchestra.run.smithi159:mds.h> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.h.service 2024-09-11T22:19:33.985 DEBUG:teuthology.orchestra.run.smithi159:mds.k> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.k.service 2024-09-11T22:19:34.028 DEBUG:teuthology.orchestra.run.smithi172:mds.c> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.c.service 2024-09-11T22:19:34.030 DEBUG:teuthology.orchestra.run.smithi172:mds.f> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.f.service 2024-09-11T22:19:34.073 DEBUG:teuthology.orchestra.run.smithi172:mds.i> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.i.service 2024-09-11T22:19:34.116 DEBUG:teuthology.orchestra.run.smithi172:mds.l> sudo journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.l.service 2024-09-11T22:19:34.159 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-11T22:19:34.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:19:34.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:34.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:19:34.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-09-11T22:19:34.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:34.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:34.200 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:34.201 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:34.201 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:34.201 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:34.201 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:34.201 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:34 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:34.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:34.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:19:34.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-09-11T22:19:34.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:34.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:34.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:34.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:34.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:34.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:34.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:34.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:34 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:19:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-09-11T22:19:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:19:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:34.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:19:34.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:34.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:34.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:34.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:34 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:34.418 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.924+0000 7f114e5d0640 1 -- 172.21.15.5:0/2149775986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11481082e0 msgr2=0x7f114810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.924+0000 7f114e5d0640 1 --2- 172.21.15.5:0/2149775986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11481082e0 0x7f114810ec50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1140009cd0 tx=0x7f114002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:19:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.924+0000 7f114e5d0640 1 -- 172.21.15.5:0/2149775986 shutdown_connections 2024-09-11T22:19:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.924+0000 7f114e5d0640 1 --2- 172.21.15.5:0/2149775986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11481082e0 0x7f114810ec50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.924+0000 7f114e5d0640 1 --2- 172.21.15.5:0/2149775986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1148103760 0x7f1148107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.924+0000 7f114e5d0640 1 --2- 172.21.15.5:0/2149775986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1148102db0 0x7f1148103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.924+0000 7f114e5d0640 1 -- 172.21.15.5:0/2149775986 >> 172.21.15.5:0/2149775986 conn(0x7f11480fc7d0 msgr2=0x7f11480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.924+0000 7f114e5d0640 1 -- 172.21.15.5:0/2149775986 shutdown_connections 2024-09-11T22:19:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.924+0000 7f114e5d0640 1 -- 172.21.15.5:0/2149775986 wait complete. 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f114e5d0640 1 Processor -- start 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f114e5d0640 1 -- start start 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f114e5d0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1148102db0 0x7f1148100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f114e5d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1148103760 0x7f1148100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f114e5d0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11481082e0 0x7f1148101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f114e5d0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11481120b0 con 0x7f1148103760 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f114e5d0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1148111f30 con 0x7f1148102db0 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f114e5d0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1148112230 con 0x7f11481082e0 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f11477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1148103760 0x7f1148100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f11477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1148103760 0x7f1148100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59516/0 (socket says 172.21.15.5:59516) 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.925+0000 7f11477fe640 1 -- 172.21.15.5:0/2179788906 learned_addr learned my addr 172.21.15.5:0/2179788906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f114cb6d640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11481082e0 0x7f1148101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f1147fff640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1148102db0 0x7f1148100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f11477fe640 1 -- 172.21.15.5:0/2179788906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11481082e0 msgr2=0x7f1148101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f11477fe640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11481082e0 0x7f1148101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f11477fe640 1 -- 172.21.15.5:0/2179788906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1148102db0 msgr2=0x7f1148100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f11477fe640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1148102db0 0x7f1148100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f11477fe640 1 -- 172.21.15.5:0/2179788906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11481a69d0 con 0x7f1148103760 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f114cb6d640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11481082e0 0x7f1148101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f11477fe640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1148103760 0x7f1148100f50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c8c0 tx=0x7f113c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f11457fa640 1 -- 172.21.15.5:0/2179788906 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c007bb0 con 0x7f1148103760 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f11457fa640 1 -- 172.21.15.5:0/2179788906 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f113c007d50 con 0x7f1148103760 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f114e5d0640 1 -- 172.21.15.5:0/2179788906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11481a6cc0 con 0x7f1148103760 2024-09-11T22:19:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f11457fa640 1 -- 172.21.15.5:0/2179788906 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c018830 con 0x7f1148103760 2024-09-11T22:19:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.926+0000 7f114e5d0640 1 -- 172.21.15.5:0/2179788906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11481a71f0 con 0x7f1148103760 2024-09-11T22:19:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.927+0000 7f114e5d0640 1 -- 172.21.15.5:0/2179788906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1148103190 con 0x7f1148103760 2024-09-11T22:19:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.928+0000 7f11457fa640 1 -- 172.21.15.5:0/2179788906 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f113c0189f0 con 0x7f1148103760 2024-09-11T22:19:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.929+0000 7f11457fa640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f11300a2c00 0x7f11300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.929+0000 7f1147fff640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f11300a2c00 0x7f11300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.929+0000 7f11457fa640 1 -- 172.21.15.5:0/2179788906 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f113c0ceeb0 con 0x7f1148103760 2024-09-11T22:19:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.929+0000 7f1147fff640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f11300a2c00 0x7f11300a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1138005fd0 tx=0x7f1138005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:34.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:34.932+0000 7f11457fa640 1 -- 172.21.15.5:0/2179788906 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f113c0fe1b0 con 0x7f1148103760 2024-09-11T22:19:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.150+0000 7f114e5d0640 1 -- 172.21.15.5:0/2179788906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1148101f40 con 0x7f1148103760 2024-09-11T22:19:35.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:35 smithi005 ceph-mon[30943]: Deploying daemon mds.l on smithi172 2024-09-11T22:19:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.151+0000 7f11457fa640 1 -- 172.21.15.5:0/2179788906 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f113c014030 con 0x7f1148103760 2024-09-11T22:19:35.152 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:35.152 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":1,"btime":"2024-09-11T22:12:23:446697+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-11T22:19:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 1 2024-09-11T22:19:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.153+0000 7f114e5d0640 1 -- 172.21.15.5:0/2179788906 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f11300a2c00 msgr2=0x7f11300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.153+0000 7f114e5d0640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f11300a2c00 0x7f11300a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1138005fd0 tx=0x7f1138005850 comp rx=0 tx=0).stop 2024-09-11T22:19:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.153+0000 7f114e5d0640 1 -- 172.21.15.5:0/2179788906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1148103760 msgr2=0x7f1148100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.153+0000 7f114e5d0640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1148103760 0x7f1148100f50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f113c00c8c0 tx=0x7f113c00cd80 comp rx=0 tx=0).stop 2024-09-11T22:19:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.153+0000 7f114e5d0640 1 -- 172.21.15.5:0/2179788906 shutdown_connections 2024-09-11T22:19:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.153+0000 7f114e5d0640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f11300a2c00 0x7f11300a50c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.153+0000 7f114e5d0640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11481082e0 0x7f1148101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.153+0000 7f114e5d0640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1148103760 0x7f1148100f50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.153+0000 7f114e5d0640 1 --2- 172.21.15.5:0/2179788906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1148102db0 0x7f1148100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.153+0000 7f114e5d0640 1 -- 172.21.15.5:0/2179788906 >> 172.21.15.5:0/2179788906 conn(0x7f11480fc7d0 msgr2=0x7f11480fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.154+0000 7f114e5d0640 1 -- 172.21.15.5:0/2179788906 shutdown_connections 2024-09-11T22:19:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:35.154+0000 7f114e5d0640 1 -- 172.21.15.5:0/2179788906 wait complete. 2024-09-11T22:19:35.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:35 smithi159 ceph-mon[35534]: Deploying daemon mds.l on smithi172 2024-09-11T22:19:35.347 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:35 smithi172 ceph-mon[35461]: Deploying daemon mds.l on smithi172 2024-09-11T22:19:35.610 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:19:35 smithi172 systemd[1]: Starting Ceph mds.l for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:19:36.029 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:19:35 smithi172 podman[65457]: 2024-09-11 22:19:35.638145765 +0000 UTC m=+0.015762019 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:19:36.029 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:19:35 smithi172 podman[65457]: 2024-09-11 22:19:35.749586547 +0000 UTC m=+0.127202837 container create 8e270a63289f4365aecd50fea679e2a0f0da5ec696acb623f7b38f469413bf65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:19:36.029 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:19:36 smithi172 podman[65457]: 2024-09-11 22:19:36.029180881 +0000 UTC m=+0.406797125 container init 8e270a63289f4365aecd50fea679e2a0f0da5ec696acb623f7b38f469413bf65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-11T22:19:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:36 smithi005 ceph-mon[30943]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:36 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2179788906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:36 smithi159 ceph-mon[35534]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:36 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2179788906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:36.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:36 smithi172 ceph-mon[35461]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:36.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:36 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2179788906' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:36.360 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:19:36 smithi172 podman[65457]: 2024-09-11 22:19:36.032624216 +0000 UTC m=+0.410240463 container start 8e270a63289f4365aecd50fea679e2a0f0da5ec696acb623f7b38f469413bf65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-11T22:19:36.360 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:19:36 smithi172 ceph-mds[65477]: -- 172.21.15.172:0/767251990 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563904919860 con 0x5639048f7400 2024-09-11T22:19:36.360 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:19:36 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l[65473]: starting mds.l at 2024-09-11T22:19:36.360 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:19:36 smithi172 bash[65457]: 8e270a63289f4365aecd50fea679e2a0f0da5ec696acb623f7b38f469413bf65 2024-09-11T22:19:36.361 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:19:36 smithi172 systemd[1]: Started Ceph mds.l for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:19:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:37 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.172:6832/1186865242,v1:172.21.15.172:6833/1186865242] up:boot 2024-09-11T22:19:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:37 smithi005 ceph-mon[30943]: fsmap 1 up:standby 2024-09-11T22:19:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:19:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:37.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:37 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:37.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:37 smithi005 ceph-mon[30943]: Deploying daemon mds.k on smithi159 2024-09-11T22:19:37.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:37 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.172:6832/1186865242,v1:172.21.15.172:6833/1186865242] up:boot 2024-09-11T22:19:37.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:37 smithi159 ceph-mon[35534]: fsmap 1 up:standby 2024-09-11T22:19:37.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:19:37.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:37.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:37.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:37.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:37.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:37.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:37 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:37.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:37 smithi159 ceph-mon[35534]: Deploying daemon mds.k on smithi159 2024-09-11T22:19:37.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:37 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.172:6832/1186865242,v1:172.21.15.172:6833/1186865242] up:boot 2024-09-11T22:19:37.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:37 smithi172 ceph-mon[35461]: fsmap 1 up:standby 2024-09-11T22:19:37.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:19:37.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:37.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:37.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:37.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:37.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:37.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:37 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:37.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:37 smithi172 ceph-mon[35461]: Deploying daemon mds.k on smithi159 2024-09-11T22:19:37.972 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:19:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:38.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:38 smithi005 ceph-mon[30943]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:38.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:38 smithi159 ceph-mon[35534]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:38.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:19:38 smithi159 systemd[1]: Starting Ceph mds.k for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:19:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:38 smithi172 ceph-mon[35461]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:38.714 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:19:38 smithi159 podman[65015]: 2024-09-11 22:19:38.321489479 +0000 UTC m=+0.015855093 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:19:38.714 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:19:38 smithi159 podman[65015]: 2024-09-11 22:19:38.44432456 +0000 UTC m=+0.138690212 container create 114d39de8ee6b497757e67c80715108f314c307f80f68204e13a64770afce78c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T22:19:38.714 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:19:38 smithi159 podman[65015]: 2024-09-11 22:19:38.71396173 +0000 UTC m=+0.408327336 container init 114d39de8ee6b497757e67c80715108f314c307f80f68204e13a64770afce78c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T22:19:38.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.741+0000 7fac0db4f640 1 -- 172.21.15.5:0/4162507691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08102d40 msgr2=0x7fac08103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:38.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.741+0000 7fac0db4f640 1 --2- 172.21.15.5:0/4162507691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08102d40 0x7fac08103120 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009900 tx=0x7fabf802a480 comp rx=0 tx=0).stop 2024-09-11T22:19:38.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.741+0000 7fac0db4f640 1 -- 172.21.15.5:0/4162507691 shutdown_connections 2024-09-11T22:19:38.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.741+0000 7fac0db4f640 1 --2- 172.21.15.5:0/4162507691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac0810c170 0x7fac0810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:38.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.741+0000 7fac0db4f640 1 --2- 172.21.15.5:0/4162507691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac08103660 0x7fac0810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:38.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.741+0000 7fac0db4f640 1 --2- 172.21.15.5:0/4162507691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08102d40 0x7fac08103120 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.741+0000 7fac0db4f640 1 -- 172.21.15.5:0/4162507691 >> 172.21.15.5:0/4162507691 conn(0x7fac080fc7a0 msgr2=0x7fac080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.741+0000 7fac0db4f640 1 -- 172.21.15.5:0/4162507691 shutdown_connections 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.741+0000 7fac0db4f640 1 -- 172.21.15.5:0/4162507691 wait complete. 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fac0db4f640 1 Processor -- start 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fac0db4f640 1 -- start start 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fac0db4f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac08102d40 0x7fac081009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fac0db4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08103660 0x7fac08100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fac0db4f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac0810c170 0x7fac08101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fac0db4f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac08111f70 con 0x7fac08103660 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fac0db4f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fac08111df0 con 0x7fac0810c170 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fac0db4f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fac081120f0 con 0x7fac08102d40 2024-09-11T22:19:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fabfffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08103660 0x7fac08100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fabfffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08103660 0x7fac08100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59536/0 (socket says 172.21.15.5:59536) 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fabfffff640 1 -- 172.21.15.5:0/1520679461 learned_addr learned my addr 172.21.15.5:0/1520679461 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fac07fff640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac0810c170 0x7fac08101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fac077fe640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac08102d40 0x7fac081009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fabfffff640 1 -- 172.21.15.5:0/1520679461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac08102d40 msgr2=0x7fac081009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fabfffff640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac08102d40 0x7fac081009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fabfffff640 1 -- 172.21.15.5:0/1520679461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac0810c170 msgr2=0x7fac08101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fabfffff640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac0810c170 0x7fac08101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.742+0000 7fabfffff640 1 -- 172.21.15.5:0/1520679461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac081a68f0 con 0x7fac08103660 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.743+0000 7fabfffff640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08103660 0x7fac08100ee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fabf000d750 tx=0x7fabf000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.743+0000 7fac057fa640 1 -- 172.21.15.5:0/1520679461 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0017070 con 0x7fac08103660 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.743+0000 7fac057fa640 1 -- 172.21.15.5:0/1520679461 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf0005020 con 0x7fac08103660 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.743+0000 7fac0db4f640 1 -- 172.21.15.5:0/1520679461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac081a6be0 con 0x7fac08103660 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.743+0000 7fac057fa640 1 -- 172.21.15.5:0/1520679461 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0004d00 con 0x7fac08103660 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.743+0000 7fac0db4f640 1 -- 172.21.15.5:0/1520679461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac081a7120 con 0x7fac08103660 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.744+0000 7fac0db4f640 1 -- 172.21.15.5:0/1520679461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac08103120 con 0x7fac08103660 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.745+0000 7fac057fa640 1 -- 172.21.15.5:0/1520679461 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fabf0009a30 con 0x7fac08103660 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.746+0000 7fac057fa640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fabf40a2c00 0x7fabf40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.746+0000 7fac077fe640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fabf40a2c00 0x7fabf40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.746+0000 7fac057fa640 1 -- 172.21.15.5:0/1520679461 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fabf00ce870 con 0x7fac08103660 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.746+0000 7fac077fe640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fabf40a2c00 0x7fabf40a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009230 tx=0x7fabf80058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.749+0000 7fac057fa640 1 -- 172.21.15.5:0/1520679461 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabf009b6a0 con 0x7fac08103660 2024-09-11T22:19:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.966+0000 7fac0db4f640 1 -- 172.21.15.5:0/1520679461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac081071e0 con 0x7fac08103660 2024-09-11T22:19:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.967+0000 7fac057fa640 1 -- 172.21.15.5:0/1520679461 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fabf009f9a0 con 0x7fac08103660 2024-09-11T22:19:38.968 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:38.968 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"btime":"2024-09-11T22:19:36:063192+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T22:19:38.968 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 -- 172.21.15.5:0/1520679461 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fabf40a2c00 msgr2=0x7fabf40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fabf40a2c00 0x7fabf40a50c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009230 tx=0x7fabf80058d0 comp rx=0 tx=0).stop 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 -- 172.21.15.5:0/1520679461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08103660 msgr2=0x7fac08100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08103660 0x7fac08100ee0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fabf000d750 tx=0x7fabf000dc10 comp rx=0 tx=0).stop 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 -- 172.21.15.5:0/1520679461 shutdown_connections 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fabf40a2c00 0x7fabf40a50c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac0810c170 0x7fac08101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08103660 0x7fac08100ee0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 --2- 172.21.15.5:0/1520679461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac08102d40 0x7fac081009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 -- 172.21.15.5:0/1520679461 >> 172.21.15.5:0/1520679461 conn(0x7fac080fc7a0 msgr2=0x7fac080fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 -- 172.21.15.5:0/1520679461 shutdown_connections 2024-09-11T22:19:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:38.970+0000 7fac0db4f640 1 -- 172.21.15.5:0/1520679461 wait complete. 2024-09-11T22:19:39.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:19:38 smithi159 podman[65015]: 2024-09-11 22:19:38.716984929 +0000 UTC m=+0.411350538 container start 114d39de8ee6b497757e67c80715108f314c307f80f68204e13a64770afce78c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T22:19:39.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:19:38 smithi159 ceph-mds[65034]: -- 172.21.15.159:0/3537947735 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565d9b9d860 con 0x5565d9b7b000 2024-09-11T22:19:39.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:19:38 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: starting mds.k at 2024-09-11T22:19:39.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:19:38 smithi159 bash[65015]: 114d39de8ee6b497757e67c80715108f314c307f80f68204e13a64770afce78c 2024-09-11T22:19:39.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:19:38 smithi159 systemd[1]: Started Ceph mds.k for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:19:40.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:40.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:40.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:40.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:40.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:40.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:40.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:40.068 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: Deploying daemon mds.j on smithi005 2024-09-11T22:19:40.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1520679461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:40.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6832/2975110123,v1:172.21.15.159:6833/2975110123] up:boot 2024-09-11T22:19:40.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: fsmap 2 up:standby 2024-09-11T22:19:40.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:39 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:19:40.257 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:40.257 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:40.257 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:40.257 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:40.257 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:40.257 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:40.257 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:40.257 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: Deploying daemon mds.j on smithi005 2024-09-11T22:19:40.258 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1520679461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:40.258 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6832/2975110123,v1:172.21.15.159:6833/2975110123] up:boot 2024-09-11T22:19:40.258 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: fsmap 2 up:standby 2024-09-11T22:19:40.258 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:39 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:19:40.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:40.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:40.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:40.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:40.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:40.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:40.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:40.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: Deploying daemon mds.j on smithi005 2024-09-11T22:19:40.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1520679461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:40.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6832/2975110123,v1:172.21.15.159:6833/2975110123] up:boot 2024-09-11T22:19:40.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: fsmap 2 up:standby 2024-09-11T22:19:40.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:39 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:19:41.811 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:19:41 smithi005 systemd[1]: Starting Ceph mds.j for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:19:41.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:19:42.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:41 smithi005 ceph-mon[30943]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:42.070 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:19:41 smithi005 podman[67737]: 2024-09-11 22:19:41.713068077 +0000 UTC m=+0.016490055 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:19:42.070 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:19:41 smithi005 podman[67737]: 2024-09-11 22:19:41.844856412 +0000 UTC m=+0.148278379 container create fe9c99224f8332e91febe5f727b81fa1085815ef25f374667c8eddb5168af80d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-j, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-11T22:19:42.286 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:41 smithi159 ceph-mon[35534]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:41 smithi172 ceph-mon[35461]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:42.409 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:19:42 smithi005 podman[67737]: 2024-09-11 22:19:42.106548285 +0000 UTC m=+0.409970252 container init fe9c99224f8332e91febe5f727b81fa1085815ef25f374667c8eddb5168af80d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:19:42.410 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:19:42 smithi005 podman[67737]: 2024-09-11 22:19:42.110134347 +0000 UTC m=+0.413556314 container start fe9c99224f8332e91febe5f727b81fa1085815ef25f374667c8eddb5168af80d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T22:19:42.410 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:19:42 smithi005 ceph-mds[67787]: -- 172.21.15.5:0/3073060969 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562071b97860 con 0x562071b75000 2024-09-11T22:19:42.410 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:19:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-j[67771]: starting mds.j at 2024-09-11T22:19:42.410 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:19:42 smithi005 bash[67737]: fe9c99224f8332e91febe5f727b81fa1085815ef25f374667c8eddb5168af80d 2024-09-11T22:19:42.410 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:19:42 smithi005 systemd[1]: Started Ceph mds.j for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.818+0000 7fb95fe0d640 1 -- 172.21.15.5:0/2586277581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb958100b20 msgr2=0x7fb958100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.818+0000 7fb95fe0d640 1 --2- 172.21.15.5:0/2586277581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb958100b20 0x7fb958100f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb950009900 tx=0x7fb95002a410 comp rx=0 tx=0).stop 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.818+0000 7fb95fe0d640 1 -- 172.21.15.5:0/2586277581 shutdown_connections 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.818+0000 7fb95fe0d640 1 --2- 172.21.15.5:0/2586277581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb95810e500 0x7fb958110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.818+0000 7fb95fe0d640 1 --2- 172.21.15.5:0/2586277581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb958101440 0x7fb95810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.818+0000 7fb95fe0d640 1 --2- 172.21.15.5:0/2586277581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb958100b20 0x7fb958100f00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.818+0000 7fb95fe0d640 1 -- 172.21.15.5:0/2586277581 >> 172.21.15.5:0/2586277581 conn(0x7fb9580fc7d0 msgr2=0x7fb9580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.818+0000 7fb95fe0d640 1 -- 172.21.15.5:0/2586277581 shutdown_connections 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.818+0000 7fb95fe0d640 1 -- 172.21.15.5:0/2586277581 wait complete. 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95fe0d640 1 Processor -- start 2024-09-11T22:19:42.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95fe0d640 1 -- start start 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95fe0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb958100b20 0x7fb9581a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95fe0d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb958101440 0x7fb9581a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95fe0d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb95810e500 0x7fb95819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95fe0d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9581142b0 con 0x7fb958100b20 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95fe0d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb958114130 con 0x7fb95810e500 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95fe0d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb958114430 con 0x7fb958101440 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95dba9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb958100b20 0x7fb9581a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95dba9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb958100b20 0x7fb9581a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60396/0 (socket says 172.21.15.5:60396) 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.819+0000 7fb95dba9640 1 -- 172.21.15.5:0/3314031801 learned_addr learned my addr 172.21.15.5:0/3314031801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95e3aa640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb95810e500 0x7fb95819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95d3a8640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb958101440 0x7fb9581a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95dba9640 1 -- 172.21.15.5:0/3314031801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb958101440 msgr2=0x7fb9581a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:42.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95dba9640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb958101440 0x7fb9581a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95dba9640 1 -- 172.21.15.5:0/3314031801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb95810e500 msgr2=0x7fb95819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95dba9640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb95810e500 0x7fb95819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95dba9640 1 -- 172.21.15.5:0/3314031801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95819ace0 con 0x7fb958100b20 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95e3aa640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb95810e500 0x7fb95819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95dba9640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb958100b20 0x7fb9581a03f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb950004770 tx=0x7fb950003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb956ffd640 1 -- 172.21.15.5:0/3314031801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95003f070 con 0x7fb958100b20 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb956ffd640 1 -- 172.21.15.5:0/3314031801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb950031ca0 con 0x7fb958100b20 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95fe0d640 1 -- 172.21.15.5:0/3314031801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95819af70 con 0x7fb958100b20 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb956ffd640 1 -- 172.21.15.5:0/3314031801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950035030 con 0x7fb958100b20 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.820+0000 7fb95fe0d640 1 -- 172.21.15.5:0/3314031801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9581a72c0 con 0x7fb958100b20 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.821+0000 7fb95fe0d640 1 -- 172.21.15.5:0/3314031801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95806a850 con 0x7fb958100b20 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.822+0000 7fb956ffd640 1 -- 172.21.15.5:0/3314031801 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb950042070 con 0x7fb958100b20 2024-09-11T22:19:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.823+0000 7fb956ffd640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb9400a2b80 0x7fb9400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:42.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.823+0000 7fb956ffd640 1 -- 172.21.15.5:0/3314031801 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fb9500ef470 con 0x7fb958100b20 2024-09-11T22:19:42.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.823+0000 7fb95d3a8640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb9400a2b80 0x7fb9400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:42.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.824+0000 7fb95d3a8640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb9400a2b80 0x7fb9400a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb94c005fd0 tx=0x7fb94c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:42.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:42.827+0000 7fb956ffd640 1 -- 172.21.15.5:0/3314031801 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9500bc870 con 0x7fb958100b20 2024-09-11T22:19:43.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.045+0000 7fb95fe0d640 1 -- 172.21.15.5:0/3314031801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb958100f00 con 0x7fb958100b20 2024-09-11T22:19:43.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.045+0000 7fb956ffd640 1 -- 172.21.15.5:0/3314031801 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7fb9500bc170 con 0x7fb958100b20 2024-09-11T22:19:43.047 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:43.047 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"btime":"2024-09-11T22:19:42:865800+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T22:19:43.047 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2024-09-11T22:19:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 -- 172.21.15.5:0/3314031801 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb9400a2b80 msgr2=0x7fb9400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb9400a2b80 0x7fb9400a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb94c005fd0 tx=0x7fb94c005850 comp rx=0 tx=0).stop 2024-09-11T22:19:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 -- 172.21.15.5:0/3314031801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb958100b20 msgr2=0x7fb9581a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb958100b20 0x7fb9581a03f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb950004770 tx=0x7fb950003c10 comp rx=0 tx=0).stop 2024-09-11T22:19:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 -- 172.21.15.5:0/3314031801 shutdown_connections 2024-09-11T22:19:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb9400a2b80 0x7fb9400a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb95810e500 0x7fb95819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb958101440 0x7fb9581a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 --2- 172.21.15.5:0/3314031801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb958100b20 0x7fb9581a03f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 -- 172.21.15.5:0/3314031801 >> 172.21.15.5:0/3314031801 conn(0x7fb9580fc7d0 msgr2=0x7fb9580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 -- 172.21.15.5:0/3314031801 shutdown_connections 2024-09-11T22:19:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:43.048+0000 7fb95fe0d640 1 -- 172.21.15.5:0/3314031801 wait complete. 2024-09-11T22:19:43.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:43.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:43.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:43.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:43.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:43.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:43.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: Deploying daemon mds.d on smithi005 2024-09-11T22:19:43.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.5:6834/2780062479,v1:172.21.15.5:6835/2780062479] up:boot 2024-09-11T22:19:43.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: fsmap 3 up:standby 2024-09-11T22:19:43.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:19:43.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:43 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3314031801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:43.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:43.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:43.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:43.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:43.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:43.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:43.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: Deploying daemon mds.d on smithi005 2024-09-11T22:19:43.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.5:6834/2780062479,v1:172.21.15.5:6835/2780062479] up:boot 2024-09-11T22:19:43.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: fsmap 3 up:standby 2024-09-11T22:19:43.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:19:43.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:43 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3314031801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: Deploying daemon mds.d on smithi005 2024-09-11T22:19:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.5:6834/2780062479,v1:172.21.15.5:6835/2780062479] up:boot 2024-09-11T22:19:43.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: fsmap 3 up:standby 2024-09-11T22:19:43.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:19:43.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:43 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3314031801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:44.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:44 smithi005 ceph-mon[30943]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:44 smithi159 ceph-mon[35534]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:44.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:44 smithi172 ceph-mon[35461]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:45.497 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:19:45 smithi005 systemd[1]: Starting Ceph mds.d for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:19:45.755 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:19:45 smithi005 podman[68164]: 2024-09-11 22:19:45.39936035 +0000 UTC m=+0.016847845 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:19:45.756 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:19:45 smithi005 podman[68164]: 2024-09-11 22:19:45.511457923 +0000 UTC m=+0.128945488 container create 816ef0aabc16e895a02e8b5edb0795eee92071ed5167923c772d893be98c2119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-11T22:19:45.756 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:19:45 smithi005 podman[68164]: 2024-09-11 22:19:45.755860526 +0000 UTC m=+0.373348018 container init 816ef0aabc16e895a02e8b5edb0795eee92071ed5167923c772d893be98c2119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:19:45.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:19:46.070 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:19:45 smithi005 podman[68164]: 2024-09-11 22:19:45.759006805 +0000 UTC m=+0.376494298 container start 816ef0aabc16e895a02e8b5edb0795eee92071ed5167923c772d893be98c2119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-11T22:19:46.070 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:19:45 smithi005 ceph-mds[68183]: -- 172.21.15.5:0/780481563 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557dc6c19860 con 0x557dc6bf6800 2024-09-11T22:19:46.070 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:19:45 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-d[68179]: starting mds.d at 2024-09-11T22:19:46.070 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:19:45 smithi005 bash[68164]: 816ef0aabc16e895a02e8b5edb0795eee92071ed5167923c772d893be98c2119 2024-09-11T22:19:46.070 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:19:45 smithi005 systemd[1]: Started Ceph mds.d for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:19:46.153 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:46.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:46 smithi005 ceph-mon[30943]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:46.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:46 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:46.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:46 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:46.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:46 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:46.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:46 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:46.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:46 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:46.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:46 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:46 smithi159 ceph-mon[35534]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:46 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:46 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:46 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:46 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:46 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:46 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:46.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:46 smithi172 ceph-mon[35461]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:46.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:46 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:46.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:46 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:46.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:46 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:46.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:46 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:46.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:46 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:46.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:46 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.659+0000 7f15a8a77640 1 -- 172.21.15.5:0/981115736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4100e20 msgr2=0x7f15a4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.659+0000 7f15a8a77640 1 --2- 172.21.15.5:0/981115736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4100e20 0x7f15a4101200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1594009900 tx=0x7f159402a480 comp rx=0 tx=0).stop 2024-09-11T22:19:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.659+0000 7f15a8a77640 1 -- 172.21.15.5:0/981115736 shutdown_connections 2024-09-11T22:19:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.659+0000 7f15a8a77640 1 --2- 172.21.15.5:0/981115736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15a4102ef0 0x7f15a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.659+0000 7f15a8a77640 1 --2- 172.21.15.5:0/981115736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4102570 0x7f15a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.659+0000 7f15a8a77640 1 --2- 172.21.15.5:0/981115736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4100e20 0x7f15a4101200 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.659+0000 7f15a8a77640 1 -- 172.21.15.5:0/981115736 >> 172.21.15.5:0/981115736 conn(0x7f15a40fc7d0 msgr2=0x7f15a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.659+0000 7f15a8a77640 1 -- 172.21.15.5:0/981115736 shutdown_connections 2024-09-11T22:19:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.659+0000 7f15a8a77640 1 -- 172.21.15.5:0/981115736 wait complete. 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a8a77640 1 Processor -- start 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a8a77640 1 -- start start 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a8a77640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4100e20 0x7f15a40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a8a77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15a4102570 0x7f15a40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a8a77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4102ef0 0x7f15a40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a8a77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4110fa0 con 0x7f15a4102ef0 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a8a77640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4110e20 con 0x7f15a4100e20 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a8a77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4111120 con 0x7f15a4102570 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4102ef0 0x7f15a40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4102ef0 0x7f15a40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60424/0 (socket says 172.21.15.5:60424) 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a2d9d640 1 -- 172.21.15.5:0/2121773922 learned_addr learned my addr 172.21.15.5:0/2121773922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.660+0000 7f15a259c640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4100e20 0x7f15a40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15a1d9b640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15a4102570 0x7f15a40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15a2d9d640 1 -- 172.21.15.5:0/2121773922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15a4102570 msgr2=0x7f15a40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15a2d9d640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15a4102570 0x7f15a40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15a2d9d640 1 -- 172.21.15.5:0/2121773922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4100e20 msgr2=0x7f15a40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15a2d9d640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4100e20 0x7f15a40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15a2d9d640 1 -- 172.21.15.5:0/2121773922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a41a69d0 con 0x7f15a4102ef0 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15a2d9d640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4102ef0 0x7f15a40fd700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f159c00b970 tx=0x7f159c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15937fe640 1 -- 172.21.15.5:0/2121773922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c019070 con 0x7f15a4102ef0 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15937fe640 1 -- 172.21.15.5:0/2121773922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159c014440 con 0x7f15a4102ef0 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15a8a77640 1 -- 172.21.15.5:0/2121773922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a41a6cc0 con 0x7f15a4102ef0 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15937fe640 1 -- 172.21.15.5:0/2121773922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c0075d0 con 0x7f15a4102ef0 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.661+0000 7f15a8a77640 1 -- 172.21.15.5:0/2121773922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a41a71b0 con 0x7f15a4102ef0 2024-09-11T22:19:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.662+0000 7f15a8a77640 1 -- 172.21.15.5:0/2121773922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a4101200 con 0x7f15a4102ef0 2024-09-11T22:19:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.663+0000 7f15937fe640 1 -- 172.21.15.5:0/2121773922 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f159c01c070 con 0x7f15a4102ef0 2024-09-11T22:19:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.664+0000 7f15937fe640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f158c0a2b80 0x7f158c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.664+0000 7f15a259c640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f158c0a2b80 0x7f158c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.664+0000 7f15937fe640 1 -- 172.21.15.5:0/2121773922 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f159c0ce120 con 0x7f15a4102ef0 2024-09-11T22:19:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.664+0000 7f15a259c640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f158c0a2b80 0x7f158c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f15940058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:46.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.668+0000 7f15937fe640 1 -- 172.21.15.5:0/2121773922 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f159c09aed0 con 0x7f15a4102ef0 2024-09-11T22:19:46.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.881+0000 7f15a8a77640 1 -- 172.21.15.5:0/2121773922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15a410a0a0 con 0x7f15a4102ef0 2024-09-11T22:19:46.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.881+0000 7f15937fe640 1 -- 172.21.15.5:0/2121773922 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f159c09f1d0 con 0x7f15a4102ef0 2024-09-11T22:19:46.882 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:46.882 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"btime":"2024-09-11T22:19:46:286867+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-11T22:19:46.882 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 5 2024-09-11T22:19:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 -- 172.21.15.5:0/2121773922 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f158c0a2b80 msgr2=0x7f158c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f158c0a2b80 0x7f158c0a5040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f15940058d0 comp rx=0 tx=0).stop 2024-09-11T22:19:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 -- 172.21.15.5:0/2121773922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4102ef0 msgr2=0x7f15a40fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4102ef0 0x7f15a40fd700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f159c00b970 tx=0x7f159c00bd70 comp rx=0 tx=0).stop 2024-09-11T22:19:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 -- 172.21.15.5:0/2121773922 shutdown_connections 2024-09-11T22:19:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f158c0a2b80 0x7f158c0a5040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15a4102ef0 0x7f15a40fd700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15a4102570 0x7f15a40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 --2- 172.21.15.5:0/2121773922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15a4100e20 0x7f15a40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 -- 172.21.15.5:0/2121773922 >> 172.21.15.5:0/2121773922 conn(0x7f15a40fc7d0 msgr2=0x7f15a406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 -- 172.21.15.5:0/2121773922 shutdown_connections 2024-09-11T22:19:46.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:46.883+0000 7f15a8a77640 1 -- 172.21.15.5:0/2121773922 wait complete. 2024-09-11T22:19:47.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:47 smithi005 ceph-mon[30943]: Deploying daemon mds.e on smithi159 2024-09-11T22:19:47.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:47 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.5:6836/118173744,v1:172.21.15.5:6837/118173744] up:boot 2024-09-11T22:19:47.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:47 smithi005 ceph-mon[30943]: fsmap 4 up:standby 2024-09-11T22:19:47.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:47 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:19:47.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:47 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2121773922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:47.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:47 smithi159 ceph-mon[35534]: Deploying daemon mds.e on smithi159 2024-09-11T22:19:47.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:47 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.5:6836/118173744,v1:172.21.15.5:6837/118173744] up:boot 2024-09-11T22:19:47.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:47 smithi159 ceph-mon[35534]: fsmap 4 up:standby 2024-09-11T22:19:47.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:47 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:19:47.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:47 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2121773922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:47 smithi172 ceph-mon[35461]: Deploying daemon mds.e on smithi159 2024-09-11T22:19:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:47 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.5:6836/118173744,v1:172.21.15.5:6837/118173744] up:boot 2024-09-11T22:19:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:47 smithi172 ceph-mon[35461]: fsmap 4 up:standby 2024-09-11T22:19:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:47 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:19:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:47 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2121773922' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:48.304 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:19:48 smithi159 systemd[1]: Starting Ceph mds.e for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:19:48.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:48 smithi005 ceph-mon[30943]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:48.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:48 smithi159 ceph-mon[35534]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:48.579 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:19:48 smithi159 podman[65331]: 2024-09-11 22:19:48.233627635 +0000 UTC m=+0.015935595 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:19:48.579 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:19:48 smithi159 podman[65331]: 2024-09-11 22:19:48.350907803 +0000 UTC m=+0.133215764 container create da2b9fe291c225c4aceb11433592cbafdfd1f16877d4691cad0453f1a02c1562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T22:19:48.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:48 smithi172 ceph-mon[35461]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:49.079 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:19:48 smithi159 podman[65331]: 2024-09-11 22:19:48.620651844 +0000 UTC m=+0.402959805 container init da2b9fe291c225c4aceb11433592cbafdfd1f16877d4691cad0453f1a02c1562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:19:49.079 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:19:48 smithi159 podman[65331]: 2024-09-11 22:19:48.623354705 +0000 UTC m=+0.405662669 container start da2b9fe291c225c4aceb11433592cbafdfd1f16877d4691cad0453f1a02c1562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True) 2024-09-11T22:19:49.079 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:19:48 smithi159 ceph-mds[65350]: -- 172.21.15.159:0/2528750513 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5605afbdb860 con 0x5605afbb9000 2024-09-11T22:19:49.079 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:19:48 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e[65346]: starting mds.e at 2024-09-11T22:19:49.079 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:19:48 smithi159 bash[65331]: da2b9fe291c225c4aceb11433592cbafdfd1f16877d4691cad0453f1a02c1562 2024-09-11T22:19:49.079 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:19:48 smithi159 systemd[1]: Started Ceph mds.e for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:19:49.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:19:49.907 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:50.028 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:50.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:50.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:50.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:50.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:50.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:50.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:50.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: Deploying daemon mds.f on smithi172 2024-09-11T22:19:50.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] up:boot 2024-09-11T22:19:50.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: fsmap 5 up:standby 2024-09-11T22:19:50.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:49 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:19:50.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:50.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:50.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:50.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:50.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:50.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:50.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:50.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: Deploying daemon mds.f on smithi172 2024-09-11T22:19:50.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] up:boot 2024-09-11T22:19:50.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: fsmap 5 up:standby 2024-09-11T22:19:50.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:49 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:19:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: Deploying daemon mds.f on smithi172 2024-09-11T22:19:50.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] up:boot 2024-09-11T22:19:50.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: fsmap 5 up:standby 2024-09-11T22:19:50.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:49 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:19:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.416+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2038212014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78106f10 msgr2=0x7f1d7810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.416+0000 7f1d7dc7a640 1 --2- 172.21.15.5:0/2038212014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78106f10 0x7f1d7810ea90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009ca0 tx=0x7f1d7002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:19:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.416+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2038212014 shutdown_connections 2024-09-11T22:19:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.416+0000 7f1d7dc7a640 1 --2- 172.21.15.5:0/2038212014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78106f10 0x7f1d7810ea90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.416+0000 7f1d7dc7a640 1 --2- 172.21.15.5:0/2038212014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d78102790 0x7f1d781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.416+0000 7f1d7dc7a640 1 --2- 172.21.15.5:0/2038212014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d78101de0 0x7f1d781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.416+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2038212014 >> 172.21.15.5:0/2038212014 conn(0x7f1d780fd8c0 msgr2=0x7f1d780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.416+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2038212014 shutdown_connections 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.416+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2038212014 wait complete. 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.417+0000 7f1d7dc7a640 1 Processor -- start 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.417+0000 7f1d7dc7a640 1 -- start start 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.417+0000 7f1d7dc7a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d78101de0 0x7f1d78198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.417+0000 7f1d7dc7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102790 0x7f1d781989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.417+0000 7f1d7dc7a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d78106f10 0x7f1d7819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.417+0000 7f1d7dc7a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d78111f90 con 0x7f1d78102790 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.417+0000 7f1d7dc7a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d78111e10 con 0x7f1d78106f10 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.417+0000 7f1d7dc7a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1d78112110 con 0x7f1d78101de0 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d76ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102790 0x7f1d781989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d76ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102790 0x7f1d781989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60436/0 (socket says 172.21.15.5:60436) 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d76ffd640 1 -- 172.21.15.5:0/2113562764 learned_addr learned my addr 172.21.15.5:0/2113562764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d777fe640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d78101de0 0x7f1d78198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d77fff640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d78106f10 0x7f1d7819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d76ffd640 1 -- 172.21.15.5:0/2113562764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d78101de0 msgr2=0x7f1d78198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d76ffd640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d78101de0 0x7f1d78198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d76ffd640 1 -- 172.21.15.5:0/2113562764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d78106f10 msgr2=0x7f1d7819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d76ffd640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d78106f10 0x7f1d7819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d76ffd640 1 -- 172.21.15.5:0/2113562764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7819d410 con 0x7f1d78102790 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d77fff640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d78106f10 0x7f1d7819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d76ffd640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102790 0x7f1d781989a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00d950 tx=0x7f1d6c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d74ff9640 1 -- 172.21.15.5:0/2113562764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c017070 con 0x7f1d78102790 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d74ff9640 1 -- 172.21.15.5:0/2113562764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d6c00de80 con 0x7f1d78102790 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2113562764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7819d700 con 0x7f1d78102790 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d74ff9640 1 -- 172.21.15.5:0/2113562764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00f620 con 0x7f1d78102790 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.418+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2113562764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d781a4fe0 con 0x7f1d78102790 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.419+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2113562764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d781021c0 con 0x7f1d78102790 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.420+0000 7f1d74ff9640 1 -- 172.21.15.5:0/2113562764 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f1d6c009c30 con 0x7f1d78102790 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.421+0000 7f1d74ff9640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1d600a2b80 0x7f1d600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.421+0000 7f1d777fe640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1d600a2b80 0x7f1d600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.421+0000 7f1d74ff9640 1 -- 172.21.15.5:0/2113562764 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f1d6c0ce8f0 con 0x7f1d78102790 2024-09-11T22:19:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.422+0000 7f1d777fe640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1d600a2b80 0x7f1d600a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009990 tx=0x7f1d68009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.425+0000 7f1d74ff9640 1 -- 172.21.15.5:0/2113562764 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d6c09bc50 con 0x7f1d78102790 2024-09-11T22:19:50.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.636+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2113562764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d7810a150 con 0x7f1d78102790 2024-09-11T22:19:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.637+0000 7f1d74ff9640 1 -- 172.21.15.5:0/2113562764 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f1d6c09b550 con 0x7f1d78102790 2024-09-11T22:19:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:50.638 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"btime":"2024-09-11T22:19:49:304173+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T22:19:50.638 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2113562764 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1d600a2b80 msgr2=0x7f1d600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1d600a2b80 0x7f1d600a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009990 tx=0x7f1d68009110 comp rx=0 tx=0).stop 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2113562764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102790 msgr2=0x7f1d781989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102790 0x7f1d781989a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00d950 tx=0x7f1d6c00dd50 comp rx=0 tx=0).stop 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2113562764 shutdown_connections 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1d600a2b80 0x7f1d600a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d78106f10 0x7f1d7819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d78102790 0x7f1d781989a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 --2- 172.21.15.5:0/2113562764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d78101de0 0x7f1d78198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2113562764 >> 172.21.15.5:0/2113562764 conn(0x7f1d780fd8c0 msgr2=0x7f1d780fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:50.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2113562764 shutdown_connections 2024-09-11T22:19:50.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:50.639+0000 7f1d7dc7a640 1 -- 172.21.15.5:0/2113562764 wait complete. 2024-09-11T22:19:50.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:50 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2113562764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:51.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:50 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2113562764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:51.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:50 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2113562764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:51.610 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:19:51 smithi172 systemd[1]: Started Ceph mds.f for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:19:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:51 smithi005 ceph-mon[30943]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:51 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:51 smithi159 ceph-mon[35534]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:51 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:52.096 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:51 smithi172 ceph-mon[35461]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:52.096 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:52.096 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:52.096 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:52.096 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:52.096 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:52.097 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:51 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:52.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:52 smithi172 ceph-mon[35461]: Deploying daemon mds.c on smithi172 2024-09-11T22:19:52.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:52 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.172:6834/4149321843,v1:172.21.15.172:6835/4149321843] up:boot 2024-09-11T22:19:52.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:52 smithi172 ceph-mon[35461]: fsmap 6 up:standby 2024-09-11T22:19:52.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:52 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:19:53.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:52 smithi005 ceph-mon[30943]: Deploying daemon mds.c on smithi172 2024-09-11T22:19:53.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:52 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.172:6834/4149321843,v1:172.21.15.172:6835/4149321843] up:boot 2024-09-11T22:19:53.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:52 smithi005 ceph-mon[30943]: fsmap 6 up:standby 2024-09-11T22:19:53.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:52 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:19:53.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:52 smithi159 ceph-mon[35534]: Deploying daemon mds.c on smithi172 2024-09-11T22:19:53.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:52 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.172:6834/4149321843,v1:172.21.15.172:6835/4149321843] up:boot 2024-09-11T22:19:53.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:52 smithi159 ceph-mon[35534]: fsmap 6 up:standby 2024-09-11T22:19:53.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:52 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:19:53.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:19:53.544 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:19:53 smithi172 systemd[1]: Starting Ceph mds.c for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:19:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:53.856 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:53 smithi172 ceph-mon[35461]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:53.857 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:19:53 smithi172 podman[66065]: 2024-09-11 22:19:53.446010834 +0000 UTC m=+0.015900910 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:19:53.857 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:19:53 smithi172 podman[66065]: 2024-09-11 22:19:53.562234533 +0000 UTC m=+0.132124611 container create ee25c5c264a2a21f76f5d8d259723819c3846848dcf764f358058d2fb0365aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-11T22:19:53.857 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:19:53 smithi172 podman[66065]: 2024-09-11 22:19:53.856848037 +0000 UTC m=+0.426738116 container init ee25c5c264a2a21f76f5d8d259723819c3846848dcf764f358058d2fb0365aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:19:54.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:53 smithi005 ceph-mon[30943]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:53 smithi159 ceph-mon[35534]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:54.110 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:19:53 smithi172 podman[66065]: 2024-09-11 22:19:53.860170011 +0000 UTC m=+0.430060090 container start ee25c5c264a2a21f76f5d8d259723819c3846848dcf764f358058d2fb0365aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T22:19:54.110 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:19:53 smithi172 ceph-mds[66085]: -- 172.21.15.172:0/2658124145 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564401075860 con 0x564401053000 2024-09-11T22:19:54.110 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:19:53 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c[66081]: starting mds.c at 2024-09-11T22:19:54.110 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:19:54 smithi172 bash[66065]: ee25c5c264a2a21f76f5d8d259723819c3846848dcf764f358058d2fb0365aca 2024-09-11T22:19:54.110 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:19:54 smithi172 systemd[1]: Started Ceph mds.c for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:19:54.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.182+0000 7f2cadabd640 1 -- 172.21.15.5:0/3352177671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca8103760 msgr2=0x7f2ca8107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:54.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.182+0000 7f2cadabd640 1 --2- 172.21.15.5:0/3352177671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca8103760 0x7f2ca8107ba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009930 tx=0x7f2c9802a420 comp rx=0 tx=0).stop 2024-09-11T22:19:54.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.182+0000 7f2cadabd640 1 -- 172.21.15.5:0/3352177671 shutdown_connections 2024-09-11T22:19:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.182+0000 7f2cadabd640 1 --2- 172.21.15.5:0/3352177671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca81082e0 0x7f2ca810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.182+0000 7f2cadabd640 1 --2- 172.21.15.5:0/3352177671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca8103760 0x7f2ca8107ba0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.182+0000 7f2cadabd640 1 --2- 172.21.15.5:0/3352177671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ca8102db0 0x7f2ca8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.182+0000 7f2cadabd640 1 -- 172.21.15.5:0/3352177671 >> 172.21.15.5:0/3352177671 conn(0x7f2ca80fc7d0 msgr2=0x7f2ca80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:54.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.182+0000 7f2cadabd640 1 -- 172.21.15.5:0/3352177671 shutdown_connections 2024-09-11T22:19:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.182+0000 7f2cadabd640 1 -- 172.21.15.5:0/3352177671 wait complete. 2024-09-11T22:19:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.183+0000 7f2cadabd640 1 Processor -- start 2024-09-11T22:19:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.183+0000 7f2cadabd640 1 -- start start 2024-09-11T22:19:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2cadabd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ca8102db0 0x7f2ca81985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2cadabd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca8103760 0x7f2ca8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:54.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2cadabd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81082e0 0x7f2ca819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2cadabd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8112070 con 0x7f2ca81082e0 2024-09-11T22:19:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2cadabd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8111ef0 con 0x7f2ca8103760 2024-09-11T22:19:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2cadabd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca81121f0 con 0x7f2ca8102db0 2024-09-11T22:19:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81082e0 0x7f2ca819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:54.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81082e0 0x7f2ca819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34736/0 (socket says 172.21.15.5:34736) 2024-09-11T22:19:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca7fff640 1 -- 172.21.15.5:0/1063278805 learned_addr learned my addr 172.21.15.5:0/1063278805 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca6ffd640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca8103760 0x7f2ca8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca77fe640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ca8102db0 0x7f2ca81985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca7fff640 1 -- 172.21.15.5:0/1063278805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ca8102db0 msgr2=0x7f2ca81985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca7fff640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ca8102db0 0x7f2ca81985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca7fff640 1 -- 172.21.15.5:0/1063278805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca8103760 msgr2=0x7f2ca8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca7fff640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca8103760 0x7f2ca8198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca7fff640 1 -- 172.21.15.5:0/1063278805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca819d590 con 0x7f2ca81082e0 2024-09-11T22:19:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca77fe640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ca8102db0 0x7f2ca81985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:19:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.184+0000 7f2ca7fff640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81082e0 0x7f2ca819ceb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000eb10 tx=0x7f2ca000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:54.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.185+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1063278805 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0004b20 con 0x7f2ca81082e0 2024-09-11T22:19:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.185+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1063278805 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ca0004cc0 con 0x7f2ca81082e0 2024-09-11T22:19:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.185+0000 7f2cadabd640 1 -- 172.21.15.5:0/1063278805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca819d880 con 0x7f2ca81082e0 2024-09-11T22:19:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.185+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1063278805 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0010760 con 0x7f2ca81082e0 2024-09-11T22:19:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.185+0000 7f2cadabd640 1 -- 172.21.15.5:0/1063278805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81a5160 con 0x7f2ca81082e0 2024-09-11T22:19:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.186+0000 7f2cadabd640 1 -- 172.21.15.5:0/1063278805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca806a7d0 con 0x7f2ca81082e0 2024-09-11T22:19:54.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.188+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1063278805 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2ca0005000 con 0x7f2ca81082e0 2024-09-11T22:19:54.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.189+0000 7f2ca4ff9640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2c940a2b80 0x7f2c940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:54.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.189+0000 7f2ca77fe640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2c940a2b80 0x7f2c940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:54.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.189+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1063278805 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f2ca00cf360 con 0x7f2ca81082e0 2024-09-11T22:19:54.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.189+0000 7f2ca77fe640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2c940a2b80 0x7f2c940a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009c20 tx=0x7f2c90005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:54.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.192+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1063278805 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ca00541b0 con 0x7f2ca81082e0 2024-09-11T22:19:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.406+0000 7f2cadabd640 1 -- 172.21.15.5:0/1063278805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ca8103190 con 0x7f2ca81082e0 2024-09-11T22:19:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.407+0000 7f2ca4ff9640 1 -- 172.21.15.5:0/1063278805 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5563 (secure 0 0 0) 0x7f2ca8103190 con 0x7f2ca81082e0 2024-09-11T22:19:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:54.408 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"btime":"2024-09-11T22:19:51:775929+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-11T22:19:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-09-11T22:19:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 -- 172.21.15.5:0/1063278805 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2c940a2b80 msgr2=0x7f2c940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2c940a2b80 0x7f2c940a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2c90009c20 tx=0x7f2c90005f90 comp rx=0 tx=0).stop 2024-09-11T22:19:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 -- 172.21.15.5:0/1063278805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81082e0 msgr2=0x7f2ca819ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81082e0 0x7f2ca819ceb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000eb10 tx=0x7f2ca000c2d0 comp rx=0 tx=0).stop 2024-09-11T22:19:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 -- 172.21.15.5:0/1063278805 shutdown_connections 2024-09-11T22:19:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f2c940a2b80 0x7f2c940a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81082e0 0x7f2ca819ceb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca8103760 0x7f2ca8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 --2- 172.21.15.5:0/1063278805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ca8102db0 0x7f2ca81985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 -- 172.21.15.5:0/1063278805 >> 172.21.15.5:0/1063278805 conn(0x7f2ca80fc7d0 msgr2=0x7f2ca8105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 -- 172.21.15.5:0/1063278805 shutdown_connections 2024-09-11T22:19:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:54.409+0000 7f2cadabd640 1 -- 172.21.15.5:0/1063278805 wait complete. 2024-09-11T22:19:55.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:55.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:55.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:55.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:55.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:55.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:55.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: Deploying daemon mds.h on smithi159 2024-09-11T22:19:55.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1063278805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:55.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] up:boot 2024-09-11T22:19:55.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: fsmap 7 up:standby 2024-09-11T22:19:55.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:55 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:19:55.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:55.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:55.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:55.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:55.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:55.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:55.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: Deploying daemon mds.h on smithi159 2024-09-11T22:19:55.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1063278805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] up:boot 2024-09-11T22:19:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: fsmap 7 up:standby 2024-09-11T22:19:55.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:55 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:19:55.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:55.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:55.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:55.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:55.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:55.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:55.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: Deploying daemon mds.h on smithi159 2024-09-11T22:19:55.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1063278805' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:55.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] up:boot 2024-09-11T22:19:55.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: fsmap 7 up:standby 2024-09-11T22:19:55.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:55 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:19:56.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:56 smithi159 ceph-mon[35534]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:56.329 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:19:56 smithi159 systemd[1]: Starting Ceph mds.h for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:19:56.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:56 smithi005 ceph-mon[30943]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:56.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:56 smithi172 ceph-mon[35461]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:56.727 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:19:56 smithi159 podman[65632]: 2024-09-11 22:19:56.323081033 +0000 UTC m=+0.015979762 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:19:56.727 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:19:56 smithi159 podman[65632]: 2024-09-11 22:19:56.474450789 +0000 UTC m=+0.167349517 container create 5e3191587899464e967e540688d3ba7d1baca144eafd5362f50ed72355c3b52d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-11T22:19:56.727 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:19:56 smithi159 podman[65632]: 2024-09-11 22:19:56.727201856 +0000 UTC m=+0.420100582 container init 5e3191587899464e967e540688d3ba7d1baca144eafd5362f50ed72355c3b52d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:19:57.079 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:19:56 smithi159 podman[65632]: 2024-09-11 22:19:56.730168931 +0000 UTC m=+0.423067658 container start 5e3191587899464e967e540688d3ba7d1baca144eafd5362f50ed72355c3b52d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T22:19:57.079 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:19:56 smithi159 ceph-mds[65651]: -- 172.21.15.159:0/2818337567 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563bcabf5860 con 0x563bcabd3000 2024-09-11T22:19:57.079 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:19:56 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h[65647]: starting mds.h at 2024-09-11T22:19:57.079 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:19:56 smithi159 bash[65632]: 5e3191587899464e967e540688d3ba7d1baca144eafd5362f50ed72355c3b52d 2024-09-11T22:19:57.079 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:19:56 smithi159 systemd[1]: Started Ceph mds.h for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:19:57.135 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:19:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:19:57.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.866+0000 7f7a177dd640 1 -- 172.21.15.5:0/1248849433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10074f40 msgr2=0x7f7a1010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:57.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.866+0000 7f7a177dd640 1 --2- 172.21.15.5:0/1248849433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10074f40 0x7f7a1010de80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009900 tx=0x7f7a0402a480 comp rx=0 tx=0).stop 2024-09-11T22:19:57.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.867+0000 7f7a177dd640 1 -- 172.21.15.5:0/1248849433 shutdown_connections 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.867+0000 7f7a177dd640 1 --2- 172.21.15.5:0/1248849433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a1010e3c0 0x7f7a10110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.867+0000 7f7a177dd640 1 --2- 172.21.15.5:0/1248849433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10074f40 0x7f7a1010de80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.867+0000 7f7a177dd640 1 --2- 172.21.15.5:0/1248849433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a10074620 0x7f7a10074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.867+0000 7f7a177dd640 1 -- 172.21.15.5:0/1248849433 >> 172.21.15.5:0/1248849433 conn(0x7f7a100fd8c0 msgr2=0x7f7a100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.867+0000 7f7a177dd640 1 -- 172.21.15.5:0/1248849433 shutdown_connections 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.867+0000 7f7a177dd640 1 -- 172.21.15.5:0/1248849433 wait complete. 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.867+0000 7f7a177dd640 1 Processor -- start 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.867+0000 7f7a177dd640 1 -- start start 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a177dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10074620 0x7f7a1019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a177dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10074f40 0x7f7a1019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a177dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a1010e3c0 0x7f7a1019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a177dd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a101141d0 con 0x7f7a10074620 2024-09-11T22:19:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a177dd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10114050 con 0x7f7a10074f40 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a177dd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7a10114350 con 0x7f7a1010e3c0 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15579640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10074620 0x7f7a1019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15579640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10074620 0x7f7a1019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34744/0 (socket says 172.21.15.5:34744) 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15579640 1 -- 172.21.15.5:0/1420383542 learned_addr learned my addr 172.21.15.5:0/1420383542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a14d78640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10074f40 0x7f7a1019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15d7a640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a1010e3c0 0x7f7a1019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15579640 1 -- 172.21.15.5:0/1420383542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a1010e3c0 msgr2=0x7f7a1019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15579640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a1010e3c0 0x7f7a1019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15579640 1 -- 172.21.15.5:0/1420383542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10074f40 msgr2=0x7f7a1019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15579640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10074f40 0x7f7a1019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15579640 1 -- 172.21.15.5:0/1420383542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1019f690 con 0x7f7a10074620 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15d7a640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a1010e3c0 0x7f7a1019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a14d78640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10074f40 0x7f7a1019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a15579640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10074620 0x7f7a1019a6e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000eb90 tx=0x7f7a0000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a0e7fc640 1 -- 172.21.15.5:0/1420383542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00004d20 con 0x7f7a10074620 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a0e7fc640 1 -- 172.21.15.5:0/1420383542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a0000ee90 con 0x7f7a10074620 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a177dd640 1 -- 172.21.15.5:0/1420383542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1019f980 con 0x7f7a10074620 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.868+0000 7f7a0e7fc640 1 -- 172.21.15.5:0/1420383542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a00010830 con 0x7f7a10074620 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.869+0000 7f7a177dd640 1 -- 172.21.15.5:0/1420383542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a101a7260 con 0x7f7a10074620 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.869+0000 7f7a177dd640 1 -- 172.21.15.5:0/1420383542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a10074a00 con 0x7f7a10074620 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.871+0000 7f7a0e7fc640 1 -- 172.21.15.5:0/1420383542 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7a00005000 con 0x7f7a10074620 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.872+0000 7f7a0e7fc640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f79f80a2b80 0x7f79f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.872+0000 7f7a14d78640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f79f80a2b80 0x7f79f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:19:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.872+0000 7f7a0e7fc640 1 -- 172.21.15.5:0/1420383542 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f7a000cf510 con 0x7f7a10074620 2024-09-11T22:19:57.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.872+0000 7f7a14d78640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f79f80a2b80 0x7f79f80a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a100ffec0 tx=0x7f7a04005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:19:57.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:57.878+0000 7f7a0e7fc640 1 -- 172.21.15.5:0/1420383542 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a000541b0 con 0x7f7a10074620 2024-09-11T22:19:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: Deploying daemon mds.i on smithi172 2024-09-11T22:19:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] up:boot 2024-09-11T22:19:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: fsmap 8 up:standby 2024-09-11T22:19:58.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:57 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:19:58.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.090+0000 7f7a177dd640 1 -- 172.21.15.5:0/1420383542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a1019ba40 con 0x7f7a10074620 2024-09-11T22:19:58.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.090+0000 7f7a0e7fc640 1 -- 172.21.15.5:0/1420383542 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7216 (secure 0 0 0) 0x7f7a0009c910 con 0x7f7a10074620 2024-09-11T22:19:58.091 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:19:58.091 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"btime":"2024-09-11T22:19:57:124769+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-11T22:19:58.092 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-09-11T22:19:58.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.092+0000 7f7a177dd640 1 -- 172.21.15.5:0/1420383542 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f79f80a2b80 msgr2=0x7f79f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:58.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.092+0000 7f7a177dd640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f79f80a2b80 0x7f79f80a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7a100ffec0 tx=0x7f7a04005a20 comp rx=0 tx=0).stop 2024-09-11T22:19:58.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.092+0000 7f7a177dd640 1 -- 172.21.15.5:0/1420383542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10074620 msgr2=0x7f7a1019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.092+0000 7f7a177dd640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10074620 0x7f7a1019a6e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000eb90 tx=0x7f7a0000c6a0 comp rx=0 tx=0).stop 2024-09-11T22:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.093+0000 7f7a177dd640 1 -- 172.21.15.5:0/1420383542 shutdown_connections 2024-09-11T22:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.093+0000 7f7a177dd640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f79f80a2b80 0x7f79f80a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.093+0000 7f7a177dd640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a1010e3c0 0x7f7a1019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.093+0000 7f7a177dd640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a10074f40 0x7f7a1019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.093+0000 7f7a177dd640 1 --2- 172.21.15.5:0/1420383542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a10074620 0x7f7a1019a6e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.093+0000 7f7a177dd640 1 -- 172.21.15.5:0/1420383542 >> 172.21.15.5:0/1420383542 conn(0x7f7a100fd8c0 msgr2=0x7f7a100ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.093+0000 7f7a177dd640 1 -- 172.21.15.5:0/1420383542 shutdown_connections 2024-09-11T22:19:58.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:19:58.093+0000 7f7a177dd640 1 -- 172.21.15.5:0/1420383542 wait complete. 2024-09-11T22:19:58.133 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:58.133 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:58.133 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:58.133 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:58.133 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:58.133 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:58.133 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:58.133 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: Deploying daemon mds.i on smithi172 2024-09-11T22:19:58.133 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] up:boot 2024-09-11T22:19:58.133 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: fsmap 8 up:standby 2024-09-11T22:19:58.134 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:57 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:19:58.332 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:19:58.332 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:58.332 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:58.333 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:19:58.333 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:19:58.333 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:19:58.333 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:19:58.333 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: Deploying daemon mds.i on smithi172 2024-09-11T22:19:58.333 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] up:boot 2024-09-11T22:19:58.333 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: fsmap 8 up:standby 2024-09-11T22:19:58.333 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:57 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:19:58.876 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:19:58 smithi172 systemd[1]: Starting Ceph mds.i for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:19:59.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:58 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1420383542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:59.190 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:58 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1420383542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:59.191 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:19:59 smithi172 podman[66366]: 2024-09-11 22:19:58.919710338 +0000 UTC m=+0.015886760 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:19:59.191 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:19:59 smithi172 podman[66366]: 2024-09-11 22:19:59.068775548 +0000 UTC m=+0.164951964 container create 98668205922bcfaed6877e51b1a293d7123e616b17b81764a64a8e88d8423cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T22:19:59.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:58 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1420383542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:19:59.450 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:19:59 smithi172 podman[66366]: 2024-09-11 22:19:59.321601624 +0000 UTC m=+0.417778053 container init 98668205922bcfaed6877e51b1a293d7123e616b17b81764a64a8e88d8423cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-11T22:19:59.450 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:19:59 smithi172 podman[66366]: 2024-09-11 22:19:59.324762162 +0000 UTC m=+0.420938580 container start 98668205922bcfaed6877e51b1a293d7123e616b17b81764a64a8e88d8423cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-11T22:19:59.450 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:19:59 smithi172 ceph-mds[66386]: -- 172.21.15.172:0/744949663 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d0bf7bc20 con 0x556d0bf59400 2024-09-11T22:19:59.450 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:19:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i[66381]: starting mds.i at 2024-09-11T22:19:59.450 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:19:59 smithi172 bash[66366]: 98668205922bcfaed6877e51b1a293d7123e616b17b81764a64a8e88d8423cbe 2024-09-11T22:19:59.451 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:19:59 smithi172 systemd[1]: Started Ceph mds.i for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:20:00.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:59 smithi005 ceph-mon[30943]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:00.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:00.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:00.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:00.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:20:00.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:20:00.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:19:59 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:00.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:59 smithi159 ceph-mon[35534]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:00.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:00.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:00.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:00.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:20:00.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:20:00.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:19:59 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:00.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:59 smithi172 ceph-mon[35461]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:00.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:00.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:00.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:00.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:20:00.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:20:00.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:19:59 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:00.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:20:00.957 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:00 smithi005 ceph-mon[30943]: Deploying daemon mds.g on smithi005 2024-09-11T22:20:00.957 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:00 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] up:boot 2024-09-11T22:20:00.957 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:00 smithi005 ceph-mon[30943]: fsmap 9 up:standby 2024-09-11T22:20:00.957 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:00 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:20:00.957 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:00 smithi005 ceph-mon[30943]: overall HEALTH_OK 2024-09-11T22:20:01.074 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:00 smithi159 ceph-mon[35534]: Deploying daemon mds.g on smithi005 2024-09-11T22:20:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:00 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] up:boot 2024-09-11T22:20:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:00 smithi159 ceph-mon[35534]: fsmap 9 up:standby 2024-09-11T22:20:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:00 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:20:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:00 smithi159 ceph-mon[35534]: overall HEALTH_OK 2024-09-11T22:20:01.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:00 smithi172 ceph-mon[35461]: Deploying daemon mds.g on smithi005 2024-09-11T22:20:01.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:00 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] up:boot 2024-09-11T22:20:01.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:00 smithi172 ceph-mon[35461]: fsmap 9 up:standby 2024-09-11T22:20:01.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:00 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:20:01.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:00 smithi172 ceph-mon[35461]: overall HEALTH_OK 2024-09-11T22:20:01.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.704+0000 7f9c4c679640 1 -- 172.21.15.5:0/2186654135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c441082a0 msgr2=0x7f9c4410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:01.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.704+0000 7f9c4c679640 1 --2- 172.21.15.5:0/2186654135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c441082a0 0x7f9c4410ec40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009cd0 tx=0x7f9c3402a910 comp rx=0 tx=0).stop 2024-09-11T22:20:01.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.704+0000 7f9c4c679640 1 -- 172.21.15.5:0/2186654135 shutdown_connections 2024-09-11T22:20:01.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.704+0000 7f9c4c679640 1 --2- 172.21.15.5:0/2186654135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c441082a0 0x7f9c4410ec40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:01.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.704+0000 7f9c4c679640 1 --2- 172.21.15.5:0/2186654135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c441036e0 0x7f9c44107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:01.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.704+0000 7f9c4c679640 1 --2- 172.21.15.5:0/2186654135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c44102d30 0x7f9c44103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:01.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.704+0000 7f9c4c679640 1 -- 172.21.15.5:0/2186654135 >> 172.21.15.5:0/2186654135 conn(0x7f9c440fc7d0 msgr2=0x7f9c440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:01.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.704+0000 7f9c4c679640 1 -- 172.21.15.5:0/2186654135 shutdown_connections 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.704+0000 7f9c4c679640 1 -- 172.21.15.5:0/2186654135 wait complete. 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4c679640 1 Processor -- start 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4c679640 1 -- start start 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4c679640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c44102d30 0x7f9c441985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4c679640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c441036e0 0x7f9c44198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4c679640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c441082a0 0x7f9c4419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4c679640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44112100 con 0x7f9c441082a0 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4c679640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44111f80 con 0x7f9c441036e0 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4c679640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44112280 con 0x7f9c44102d30 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4ac16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c441082a0 0x7f9c4419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4ac16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c441082a0 0x7f9c4419ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42492/0 (socket says 172.21.15.5:42492) 2024-09-11T22:20:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4ac16640 1 -- 172.21.15.5:0/3590903516 learned_addr learned my addr 172.21.15.5:0/3590903516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c49c14640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c441036e0 0x7f9c44198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4a415640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c44102d30 0x7f9c441985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4ac16640 1 -- 172.21.15.5:0/3590903516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c44102d30 msgr2=0x7f9c441985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4ac16640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c44102d30 0x7f9c441985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4ac16640 1 -- 172.21.15.5:0/3590903516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c441036e0 msgr2=0x7f9c44198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4ac16640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c441036e0 0x7f9c44198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c4ac16640 1 -- 172.21.15.5:0/3590903516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4419d550 con 0x7f9c441082a0 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.705+0000 7f9c49c14640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c441036e0 0x7f9c44198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.706+0000 7f9c4ac16640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c441082a0 0x7f9c4419ce70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9c34005cc0 tx=0x7f9c34003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.706+0000 7f9c437fe640 1 -- 172.21.15.5:0/3590903516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34031660 con 0x7f9c441082a0 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.706+0000 7f9c437fe640 1 -- 172.21.15.5:0/3590903516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c34031800 con 0x7f9c441082a0 2024-09-11T22:20:01.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.706+0000 7f9c4c679640 1 -- 172.21.15.5:0/3590903516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4419d7e0 con 0x7f9c441082a0 2024-09-11T22:20:01.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.706+0000 7f9c437fe640 1 -- 172.21.15.5:0/3590903516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3402fb40 con 0x7f9c441082a0 2024-09-11T22:20:01.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.706+0000 7f9c4c679640 1 -- 172.21.15.5:0/3590903516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c441a5120 con 0x7f9c441082a0 2024-09-11T22:20:01.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.707+0000 7f9c4c679640 1 -- 172.21.15.5:0/3590903516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c44103110 con 0x7f9c441082a0 2024-09-11T22:20:01.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.708+0000 7f9c437fe640 1 -- 172.21.15.5:0/3590903516 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9c3402fd00 con 0x7f9c441082a0 2024-09-11T22:20:01.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.709+0000 7f9c437fe640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9c2c0a2b80 0x7f9c2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:01.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.709+0000 7f9c437fe640 1 -- 172.21.15.5:0/3590903516 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f9c340f0270 con 0x7f9c441082a0 2024-09-11T22:20:01.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.709+0000 7f9c4a415640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9c2c0a2b80 0x7f9c2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:01.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.709+0000 7f9c4a415640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9c2c0a2b80 0x7f9c2c0a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c005fd0 tx=0x7f9c3c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:01.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.712+0000 7f9c437fe640 1 -- 172.21.15.5:0/3590903516 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c340bd670 con 0x7f9c441082a0 2024-09-11T22:20:01.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.936+0000 7f9c4c679640 1 -- 172.21.15.5:0/3590903516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c441a52f0 con 0x7f9c441082a0 2024-09-11T22:20:01.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.937+0000 7f9c437fe640 1 -- 172.21.15.5:0/3590903516 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8046 (secure 0 0 0) 0x7f9c340bcf70 con 0x7f9c441082a0 2024-09-11T22:20:01.939 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:01.939 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"btime":"2024-09-11T22:19:59:875749+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-11T22:20:01.939 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 10 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 -- 172.21.15.5:0/3590903516 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9c2c0a2b80 msgr2=0x7f9c2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9c2c0a2b80 0x7f9c2c0a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c005fd0 tx=0x7f9c3c005850 comp rx=0 tx=0).stop 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 -- 172.21.15.5:0/3590903516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c441082a0 msgr2=0x7f9c4419ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c441082a0 0x7f9c4419ce70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9c34005cc0 tx=0x7f9c34003f90 comp rx=0 tx=0).stop 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 -- 172.21.15.5:0/3590903516 shutdown_connections 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9c2c0a2b80 0x7f9c2c0a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c441082a0 0x7f9c4419ce70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c441036e0 0x7f9c44198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 --2- 172.21.15.5:0/3590903516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c44102d30 0x7f9c441985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 -- 172.21.15.5:0/3590903516 >> 172.21.15.5:0/3590903516 conn(0x7f9c440fc7d0 msgr2=0x7f9c44105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:01.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 -- 172.21.15.5:0/3590903516 shutdown_connections 2024-09-11T22:20:01.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:01.939+0000 7f9c4c679640 1 -- 172.21.15.5:0/3590903516 wait complete. 2024-09-11T22:20:02.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:01 smithi005 ceph-mon[30943]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:02.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:01 smithi159 ceph-mon[35534]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:02.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:01 smithi172 ceph-mon[35461]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:03.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:02 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3590903516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:03.070 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:20:02 smithi005 systemd[1]: Started Ceph mds.g for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:20:03.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:02 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3590903516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:03.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:02 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3590903516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.5:6838/2458034474,v1:172.21.15.5:6839/2458034474] up:boot 2024-09-11T22:20:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: fsmap 10 up:standby 2024-09-11T22:20:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:20:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:04.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:20:04.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:20:04.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:04.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:03 smithi005 ceph-mon[30943]: Deploying daemon mds.a on smithi005 2024-09-11T22:20:04.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:04.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.5:6838/2458034474,v1:172.21.15.5:6839/2458034474] up:boot 2024-09-11T22:20:04.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: fsmap 10 up:standby 2024-09-11T22:20:04.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:20:04.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:04.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:04.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:04.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:20:04.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:20:04.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:04.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:03 smithi159 ceph-mon[35534]: Deploying daemon mds.a on smithi005 2024-09-11T22:20:04.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:04.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.5:6838/2458034474,v1:172.21.15.5:6839/2458034474] up:boot 2024-09-11T22:20:04.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: fsmap 10 up:standby 2024-09-11T22:20:04.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:20:04.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:04.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:04.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:04.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:20:04.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:20:04.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:04.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:03 smithi172 ceph-mon[35461]: Deploying daemon mds.a on smithi005 2024-09-11T22:20:04.735 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:20:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:05.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.488+0000 7f99f7b8b640 1 -- 172.21.15.5:0/3190987357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0102790 msgr2=0x7f99f01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.488+0000 7f99f7b8b640 1 --2- 172.21.15.5:0/3190987357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0102790 0x7f99f01067e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009900 tx=0x7f99e002a410 comp rx=0 tx=0).stop 2024-09-11T22:20:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.488+0000 7f99f7b8b640 1 -- 172.21.15.5:0/3190987357 shutdown_connections 2024-09-11T22:20:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.488+0000 7f99f7b8b640 1 --2- 172.21.15.5:0/3190987357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f0106f10 0x7f99f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.488+0000 7f99f7b8b640 1 --2- 172.21.15.5:0/3190987357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0102790 0x7f99f01067e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.488+0000 7f99f7b8b640 1 --2- 172.21.15.5:0/3190987357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99f0101de0 0x7f99f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.489+0000 7f99f7b8b640 1 -- 172.21.15.5:0/3190987357 >> 172.21.15.5:0/3190987357 conn(0x7f99f00fd8c0 msgr2=0x7f99f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.489+0000 7f99f7b8b640 1 -- 172.21.15.5:0/3190987357 shutdown_connections 2024-09-11T22:20:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.489+0000 7f99f7b8b640 1 -- 172.21.15.5:0/3190987357 wait complete. 2024-09-11T22:20:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.489+0000 7f99f7b8b640 1 Processor -- start 2024-09-11T22:20:05.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.489+0000 7f99f7b8b640 1 -- start start 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.489+0000 7f99f7b8b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99f0101de0 0x7f99f01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.489+0000 7f99f7b8b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f0102790 0x7f99f0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f7b8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0106f10 0x7f99f019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f7b8b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0111f70 con 0x7f99f0106f10 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f7b8b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0111df0 con 0x7f99f0101de0 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f7b8b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01120f0 con 0x7f99f0102790 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f6128640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0106f10 0x7f99f019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f6128640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0106f10 0x7f99f019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42538/0 (socket says 172.21.15.5:42538) 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f6128640 1 -- 172.21.15.5:0/4115818556 learned_addr learned my addr 172.21.15.5:0/4115818556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f5927640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99f0101de0 0x7f99f01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f5126640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f0102790 0x7f99f0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:05.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f6128640 1 -- 172.21.15.5:0/4115818556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f0102790 msgr2=0x7f99f0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f6128640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f0102790 0x7f99f0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f6128640 1 -- 172.21.15.5:0/4115818556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99f0101de0 msgr2=0x7f99f01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f6128640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99f0101de0 0x7f99f01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f6128640 1 -- 172.21.15.5:0/4115818556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f019d480 con 0x7f99f0106f10 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f6128640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0106f10 0x7f99f019cda0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99e800cad0 tx=0x7f99e800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99e6ffd640 1 -- 172.21.15.5:0/4115818556 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e800ae20 con 0x7f99f0106f10 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99e6ffd640 1 -- 172.21.15.5:0/4115818556 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e8005020 con 0x7f99f0106f10 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f7b8b640 1 -- 172.21.15.5:0/4115818556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f019d770 con 0x7f99f0106f10 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99e6ffd640 1 -- 172.21.15.5:0/4115818556 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e800f6f0 con 0x7f99f0106f10 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.490+0000 7f99f7b8b640 1 -- 172.21.15.5:0/4115818556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f01a5050 con 0x7f99f0106f10 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.491+0000 7f99f7b8b640 1 -- 172.21.15.5:0/4115818556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f01021c0 con 0x7f99f0106f10 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.493+0000 7f99e6ffd640 1 -- 172.21.15.5:0/4115818556 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f99e80051d0 con 0x7f99f0106f10 2024-09-11T22:20:05.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.493+0000 7f99e6ffd640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f99dc0a2b80 0x7f99dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.493+0000 7f99f5927640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f99dc0a2b80 0x7f99dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.494+0000 7f99e6ffd640 1 -- 172.21.15.5:0/4115818556 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f99e80cef30 con 0x7f99f0106f10 2024-09-11T22:20:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.494+0000 7f99f5927640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f99dc0a2b80 0x7f99dc0a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00bbe0 tx=0x7f99ec00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.497+0000 7f99e6ffd640 1 -- 172.21.15.5:0/4115818556 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99e8014030 con 0x7f99f0106f10 2024-09-11T22:20:05.673 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:20:05 smithi005 podman[69427]: 2024-09-11 22:20:05.554646431 +0000 UTC m=+0.246679463 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:20:05.673 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:20:05 smithi005 podman[69427]: 2024-09-11 22:20:05.673236306 +0000 UTC m=+0.365269323 container create 539c65355df609f69253d53def6837ede4fb066b930ffd3f2ff38f2df11c6750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:20:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.711+0000 7f99f7b8b640 1 -- 172.21.15.5:0/4115818556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99f0199830 con 0x7f99f0106f10 2024-09-11T22:20:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.712+0000 7f99e6ffd640 1 -- 172.21.15.5:0/4115818556 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7f99f0199830 con 0x7f99f0106f10 2024-09-11T22:20:05.712 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:05.712 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"btime":"2024-09-11T22:20:02:888817+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-11T22:20:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2024-09-11T22:20:05.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 -- 172.21.15.5:0/4115818556 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f99dc0a2b80 msgr2=0x7f99dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:05.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f99dc0a2b80 0x7f99dc0a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00bbe0 tx=0x7f99ec00b390 comp rx=0 tx=0).stop 2024-09-11T22:20:05.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 -- 172.21.15.5:0/4115818556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0106f10 msgr2=0x7f99f019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0106f10 0x7f99f019cda0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99e800cad0 tx=0x7f99e800cf90 comp rx=0 tx=0).stop 2024-09-11T22:20:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 -- 172.21.15.5:0/4115818556 shutdown_connections 2024-09-11T22:20:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f99dc0a2b80 0x7f99dc0a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0106f10 0x7f99f019cda0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f0102790 0x7f99f0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 --2- 172.21.15.5:0/4115818556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99f0101de0 0x7f99f01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 -- 172.21.15.5:0/4115818556 >> 172.21.15.5:0/4115818556 conn(0x7f99f00fd8c0 msgr2=0x7f99f00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 -- 172.21.15.5:0/4115818556 shutdown_connections 2024-09-11T22:20:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:05.714+0000 7f99f7b8b640 1 -- 172.21.15.5:0/4115818556 wait complete. 2024-09-11T22:20:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:05 smithi005 ceph-mon[30943]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:05 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4115818556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:05 smithi159 ceph-mon[35534]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:05 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4115818556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:06.339 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:20:06 smithi005 podman[69427]: 2024-09-11 22:20:06.112638113 +0000 UTC m=+0.804671131 container init 539c65355df609f69253d53def6837ede4fb066b930ffd3f2ff38f2df11c6750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T22:20:06.339 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:20:06 smithi005 podman[69427]: 2024-09-11 22:20:06.11557343 +0000 UTC m=+0.807606447 container start 539c65355df609f69253d53def6837ede4fb066b930ffd3f2ff38f2df11c6750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:20:06.339 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:20:06 smithi005 ceph-mds[69483]: -- 172.21.15.5:0/3577820576 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558335913860 con 0x5583358f0800 2024-09-11T22:20:06.339 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:20:06 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a[69478]: starting mds.a at 2024-09-11T22:20:06.340 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:20:06 smithi005 bash[69427]: 539c65355df609f69253d53def6837ede4fb066b930ffd3f2ff38f2df11c6750 2024-09-11T22:20:06.340 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:20:06 smithi005 systemd[1]: Started Ceph mds.a for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:20:06.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:05 smithi172 ceph-mon[35461]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:06.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:05 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4115818556' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:20:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:20:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:07 smithi005 ceph-mon[30943]: Deploying daemon mds.b on smithi159 2024-09-11T22:20:07.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:07 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.5:6840/2232645952,v1:172.21.15.5:6841/2232645952] up:boot 2024-09-11T22:20:07.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:07 smithi005 ceph-mon[30943]: fsmap 11 up:standby 2024-09-11T22:20:07.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:07 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:20:07.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:07.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:07.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:20:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:20:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:07 smithi159 ceph-mon[35534]: Deploying daemon mds.b on smithi159 2024-09-11T22:20:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:07 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.5:6840/2232645952,v1:172.21.15.5:6841/2232645952] up:boot 2024-09-11T22:20:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:07 smithi159 ceph-mon[35534]: fsmap 11 up:standby 2024-09-11T22:20:07.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:07 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:20:07.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:07.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:07.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:07.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-11T22:20:07.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-11T22:20:07.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:07.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:07 smithi172 ceph-mon[35461]: Deploying daemon mds.b on smithi159 2024-09-11T22:20:07.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:07 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.5:6840/2232645952,v1:172.21.15.5:6841/2232645952] up:boot 2024-09-11T22:20:07.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:07 smithi172 ceph-mon[35461]: fsmap 11 up:standby 2024-09-11T22:20:07.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:07 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:20:08.502 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:08 smithi159 ceph-mon[35534]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:08.502 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:08 smithi159 systemd[1]: Starting Ceph mds.b for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:20:08.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:08 smithi005 ceph-mon[30943]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:08.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:08 smithi172 ceph-mon[35461]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:08.732 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:20:08.769 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:08 smithi159 podman[65945]: 2024-09-11 22:20:08.404802981 +0000 UTC m=+0.016175902 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:20:08.769 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:08 smithi159 podman[65945]: 2024-09-11 22:20:08.506172554 +0000 UTC m=+0.117545468 container create 8d154520033b68ab5a20962a8765bf187b08305f10cbfe2cd4fe25dee7f1caad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T22:20:08.769 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:08 smithi159 podman[65945]: 2024-09-11 22:20:08.742096728 +0000 UTC m=+0.353469639 container init 8d154520033b68ab5a20962a8765bf187b08305f10cbfe2cd4fe25dee7f1caad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T22:20:08.769 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:08 smithi159 podman[65945]: 2024-09-11 22:20:08.74505988 +0000 UTC m=+0.356432794 container start 8d154520033b68ab5a20962a8765bf187b08305f10cbfe2cd4fe25dee7f1caad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T22:20:08.769 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:08 smithi159 ceph-mds[65965]: -- 172.21.15.159:0/745672473 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c7ea57860 con 0x556c7ea35400 2024-09-11T22:20:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:09.079 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:08 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: starting mds.b at 2024-09-11T22:20:09.079 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:08 smithi159 bash[65945]: 8d154520033b68ab5a20962a8765bf187b08305f10cbfe2cd4fe25dee7f1caad 2024-09-11T22:20:09.079 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:08 smithi159 systemd[1]: Started Ceph mds.b for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:20:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.544+0000 7fb2a27d5640 1 -- 172.21.15.5:0/281831584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb29c101de0 msgr2=0x7fb29c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.544+0000 7fb2a27d5640 1 --2- 172.21.15.5:0/281831584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb29c101de0 0x7fb29c1021c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009900 tx=0x7fb28c02a410 comp rx=0 tx=0).stop 2024-09-11T22:20:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.544+0000 7fb2a27d5640 1 -- 172.21.15.5:0/281831584 shutdown_connections 2024-09-11T22:20:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.544+0000 7fb2a27d5640 1 --2- 172.21.15.5:0/281831584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb29c106f10 0x7fb29c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.544+0000 7fb2a27d5640 1 --2- 172.21.15.5:0/281831584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb29c102790 0x7fb29c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.544+0000 7fb2a27d5640 1 --2- 172.21.15.5:0/281831584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb29c101de0 0x7fb29c1021c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.544+0000 7fb2a27d5640 1 -- 172.21.15.5:0/281831584 >> 172.21.15.5:0/281831584 conn(0x7fb29c0fd8c0 msgr2=0x7fb29c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.544+0000 7fb2a27d5640 1 -- 172.21.15.5:0/281831584 shutdown_connections 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.544+0000 7fb2a27d5640 1 -- 172.21.15.5:0/281831584 wait complete. 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.545+0000 7fb2a27d5640 1 Processor -- start 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.545+0000 7fb2a27d5640 1 -- start start 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2a27d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb29c101de0 0x7fb29c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2a27d5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb29c102790 0x7fb29c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2a27d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb29c106f10 0x7fb29c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2a27d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c111f80 con 0x7fb29c101de0 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2a27d5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c111e00 con 0x7fb29c102790 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2a27d5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb29c112100 con 0x7fb29c106f10 2024-09-11T22:20:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb29bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb29c101de0 0x7fb29c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb29bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb29c101de0 0x7fb29c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42568/0 (socket says 172.21.15.5:42568) 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb29bfff640 1 -- 172.21.15.5:0/21877244 learned_addr learned my addr 172.21.15.5:0/21877244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb29b7fe640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb29c102790 0x7fb29c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2a0d72640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb29c106f10 0x7fb29c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb29bfff640 1 -- 172.21.15.5:0/21877244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb29c106f10 msgr2=0x7fb29c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb29bfff640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb29c106f10 0x7fb29c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb29bfff640 1 -- 172.21.15.5:0/21877244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb29c102790 msgr2=0x7fb29c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb29bfff640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb29c102790 0x7fb29c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb29bfff640 1 -- 172.21.15.5:0/21877244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29c1a6920 con 0x7fb29c101de0 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb29bfff640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb29c101de0 0x7fb29c0744f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb28c004770 tx=0x7fb28c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2997fa640 1 -- 172.21.15.5:0/21877244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c03f070 con 0x7fb29c101de0 2024-09-11T22:20:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2a27d5640 1 -- 172.21.15.5:0/21877244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29c1a6bb0 con 0x7fb29c101de0 2024-09-11T22:20:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2997fa640 1 -- 172.21.15.5:0/21877244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb28c031ca0 con 0x7fb29c101de0 2024-09-11T22:20:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.546+0000 7fb2997fa640 1 -- 172.21.15.5:0/21877244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c035030 con 0x7fb29c101de0 2024-09-11T22:20:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.547+0000 7fb2a27d5640 1 -- 172.21.15.5:0/21877244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29c1a7090 con 0x7fb29c101de0 2024-09-11T22:20:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.547+0000 7fb2a27d5640 1 -- 172.21.15.5:0/21877244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29c1021c0 con 0x7fb29c101de0 2024-09-11T22:20:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.549+0000 7fb2997fa640 1 -- 172.21.15.5:0/21877244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb28c042070 con 0x7fb29c101de0 2024-09-11T22:20:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.550+0000 7fb2997fa640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb2840a2b80 0x7fb2840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.550+0000 7fb29b7fe640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb2840a2b80 0x7fb2840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.550+0000 7fb2997fa640 1 -- 172.21.15.5:0/21877244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fb28c0ef770 con 0x7fb29c101de0 2024-09-11T22:20:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.551+0000 7fb29b7fe640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb2840a2b80 0x7fb2840a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb29c075f80 tx=0x7fb29000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.554+0000 7fb2997fa640 1 -- 172.21.15.5:0/21877244 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb28c0bc520 con 0x7fb29c101de0 2024-09-11T22:20:09.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.787+0000 7fb2a27d5640 1 -- 172.21.15.5:0/21877244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb29c075a20 con 0x7fb29c101de0 2024-09-11T22:20:09.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.787+0000 7fb2997fa640 1 -- 172.21.15.5:0/21877244 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7fb28c0c0820 con 0x7fb29c101de0 2024-09-11T22:20:09.788 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:09.788 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"btime":"2024-09-11T22:20:09:275364+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24391,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24431,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24454,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24500,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-11T22:20:09.789 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2024-09-11T22:20:09.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.789+0000 7fb2a27d5640 1 -- 172.21.15.5:0/21877244 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb2840a2b80 msgr2=0x7fb2840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:09.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.789+0000 7fb2a27d5640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb2840a2b80 0x7fb2840a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb29c075f80 tx=0x7fb29000b390 comp rx=0 tx=0).stop 2024-09-11T22:20:09.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.789+0000 7fb2a27d5640 1 -- 172.21.15.5:0/21877244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb29c101de0 msgr2=0x7fb29c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:09.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.789+0000 7fb2a27d5640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb29c101de0 0x7fb29c0744f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb28c004770 tx=0x7fb28c003c10 comp rx=0 tx=0).stop 2024-09-11T22:20:09.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.790+0000 7fb2a27d5640 1 -- 172.21.15.5:0/21877244 shutdown_connections 2024-09-11T22:20:09.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.790+0000 7fb2a27d5640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb2840a2b80 0x7fb2840a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:09.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.790+0000 7fb2a27d5640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb29c106f10 0x7fb29c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:09.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.790+0000 7fb2a27d5640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb29c102790 0x7fb29c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:09.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.790+0000 7fb2a27d5640 1 --2- 172.21.15.5:0/21877244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb29c101de0 0x7fb29c0744f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:09.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.790+0000 7fb2a27d5640 1 -- 172.21.15.5:0/21877244 >> 172.21.15.5:0/21877244 conn(0x7fb29c0fd8c0 msgr2=0x7fb29c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:09.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.790+0000 7fb2a27d5640 1 -- 172.21.15.5:0/21877244 shutdown_connections 2024-09-11T22:20:09.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:09.790+0000 7fb2a27d5640 1 -- 172.21.15.5:0/21877244 wait complete. 2024-09-11T22:20:09.922 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:09.922 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:09.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:09.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:09.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:09.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:09.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:20:09.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] up:boot 2024-09-11T22:20:09.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: fsmap 12 up:standby 2024-09-11T22:20:09.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:20:09.923 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:09 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/21877244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:20:10.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] up:boot 2024-09-11T22:20:10.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: fsmap 12 up:standby 2024-09-11T22:20:10.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:20:10.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:09 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/21877244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:10.175 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:10.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:10.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:10.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:10.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:10.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:10.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:20:10.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] up:boot 2024-09-11T22:20:10.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: fsmap 12 up:standby 2024-09-11T22:20:10.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:20:10.176 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:09 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/21877244' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:10.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-11T22:20:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:11.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:11 smithi005 ceph-mon[30943]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:11.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:11 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.625+0000 7f096d39c640 1 -- 172.21.15.5:0/3607445304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0968106a80 msgr2=0x7f0968106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.625+0000 7f096d39c640 1 --2- 172.21.15.5:0/3607445304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0968106a80 0x7f0968106e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0958009cd0 tx=0x7f095802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.625+0000 7f096d39c640 1 -- 172.21.15.5:0/3607445304 shutdown_connections 2024-09-11T22:20:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.625+0000 7f096d39c640 1 --2- 172.21.15.5:0/3607445304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0968101420 0x7f096810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.625+0000 7f096d39c640 1 --2- 172.21.15.5:0/3607445304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0968100a80 0x7f0968100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.625+0000 7f096d39c640 1 --2- 172.21.15.5:0/3607445304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0968106a80 0x7f0968106e60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.625+0000 7f096d39c640 1 -- 172.21.15.5:0/3607445304 >> 172.21.15.5:0/3607445304 conn(0x7f09680fc7a0 msgr2=0x7f09680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.625+0000 7f096d39c640 1 -- 172.21.15.5:0/3607445304 shutdown_connections 2024-09-11T22:20:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.625+0000 7f096d39c640 1 -- 172.21.15.5:0/3607445304 wait complete. 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.626+0000 7f096d39c640 1 Processor -- start 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.626+0000 7f096d39c640 1 -- start start 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.626+0000 7f096d39c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0968100a80 0x7f0968197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.626+0000 7f096d39c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0968101420 0x7f09681978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.626+0000 7f096d39c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0968106a80 0x7f096819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.626+0000 7f096d39c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0968110e90 con 0x7f0968100a80 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.626+0000 7f096d39c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0968110d10 con 0x7f0968106a80 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f096d39c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0968111010 con 0x7f0968101420 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f0966ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0968100a80 0x7f0968197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f0966ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0968100a80 0x7f0968197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57230/0 (socket says 172.21.15.5:57230) 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f0966ffd640 1 -- 172.21.15.5:0/1421738317 learned_addr learned my addr 172.21.15.5:0/1421738317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f09667fc640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0968101420 0x7f09681978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f09677fe640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0968106a80 0x7f096819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f0966ffd640 1 -- 172.21.15.5:0/1421738317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0968101420 msgr2=0x7f09681978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f0966ffd640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0968101420 0x7f09681978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f0966ffd640 1 -- 172.21.15.5:0/1421738317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0968106a80 msgr2=0x7f096819bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f0966ffd640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0968106a80 0x7f096819bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f0966ffd640 1 -- 172.21.15.5:0/1421738317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096819c340 con 0x7f0968100a80 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f09677fe640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0968106a80 0x7f096819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f0966ffd640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0968100a80 0x7f0968197390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0958033040 tx=0x7f09580039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f095ffff640 1 -- 172.21.15.5:0/1421738317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958004040 con 0x7f0968100a80 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f095ffff640 1 -- 172.21.15.5:0/1421738317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09580046c0 con 0x7f0968100a80 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f095ffff640 1 -- 172.21.15.5:0/1421738317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958046480 con 0x7f0968100a80 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f096d39c640 1 -- 172.21.15.5:0/1421738317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096819c5d0 con 0x7f0968100a80 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.627+0000 7f096d39c640 1 -- 172.21.15.5:0/1421738317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09681a3f10 con 0x7f0968100a80 2024-09-11T22:20:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.628+0000 7f096d39c640 1 -- 172.21.15.5:0/1421738317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0968100ee0 con 0x7f0968100a80 2024-09-11T22:20:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.629+0000 7f095ffff640 1 -- 172.21.15.5:0/1421738317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0958030710 con 0x7f0968100a80 2024-09-11T22:20:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.630+0000 7f095ffff640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09540a2c00 0x7f09540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.630+0000 7f09667fc640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09540a2c00 0x7f09540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.630+0000 7f095ffff640 1 -- 172.21.15.5:0/1421738317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f095802e030 con 0x7f0968100a80 2024-09-11T22:20:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.631+0000 7f09667fc640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09540a2c00 0x7f09540a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f09680fee20 tx=0x7f095000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.635+0000 7f095ffff640 1 -- 172.21.15.5:0/1421738317 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f095804b060 con 0x7f0968100a80 2024-09-11T22:20:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:11 smithi159 ceph-mon[35534]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:11 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.834+0000 7f096d39c640 1 -- 172.21.15.5:0/1421738317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f0968108ff0 con 0x7f0968100a80 2024-09-11T22:20:11.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.834+0000 7f095ffff640 1 -- 172.21.15.5:0/1421738317 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7f09580bd750 con 0x7f0968100a80 2024-09-11T22:20:11.835 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:11.835 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-09-11T22:20:11.835 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:20:11.835 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-09-11T22:20:11.835 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-09-11T22:20:11.835 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:20:11.835 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-09-11T22:20:11.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 -- 172.21.15.5:0/1421738317 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09540a2c00 msgr2=0x7f09540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09540a2c00 0x7f09540a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f09680fee20 tx=0x7f095000b410 comp rx=0 tx=0).stop 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 -- 172.21.15.5:0/1421738317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0968100a80 msgr2=0x7f0968197390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0968100a80 0x7f0968197390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0958033040 tx=0x7f09580039f0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 -- 172.21.15.5:0/1421738317 shutdown_connections 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09540a2c00 0x7f09540a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0968106a80 0x7f096819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0968101420 0x7f09681978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 --2- 172.21.15.5:0/1421738317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0968100a80 0x7f0968197390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 -- 172.21.15.5:0/1421738317 >> 172.21.15.5:0/1421738317 conn(0x7f09680fc7a0 msgr2=0x7f09680fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 -- 172.21.15.5:0/1421738317 shutdown_connections 2024-09-11T22:20:11.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:11.837+0000 7f096d39c640 1 -- 172.21.15.5:0/1421738317 wait complete. 2024-09-11T22:20:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:11 smithi172 ceph-mon[35461]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:11 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:12.502 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T22:20:12.503 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T22:20:12.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:12 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1421738317' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T22:20:12.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:12 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:12.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:12 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:12.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:12 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:12.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:12 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:20:12.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:12 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:12.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:12 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:20:12.826 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:12 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1421738317' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T22:20:12.826 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:12 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:12.826 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:12 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:12.827 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:12 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:12.827 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:12 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:20:12.827 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:12 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:12.827 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:12 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:20:12.837 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:12 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1421738317' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-11T22:20:12.837 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:12 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:12.837 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:12 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:12.838 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:12 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:12.838 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:12 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:20:12.838 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:12 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:12.838 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:12 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:20:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.888+0000 7fd78ba3c640 1 -- 172.21.15.5:0/500083664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd784102d30 msgr2=0x7fd784103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.888+0000 7fd78ba3c640 1 --2- 172.21.15.5:0/500083664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd784102d30 0x7fd784103110 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd780009930 tx=0x7fd78002a420 comp rx=0 tx=0).stop 2024-09-11T22:20:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.888+0000 7fd78ba3c640 1 -- 172.21.15.5:0/500083664 shutdown_connections 2024-09-11T22:20:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.888+0000 7fd78ba3c640 1 --2- 172.21.15.5:0/500083664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7841082a0 0x7fd78410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.888+0000 7fd78ba3c640 1 --2- 172.21.15.5:0/500083664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7841036e0 0x7fd784107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.888+0000 7fd78ba3c640 1 --2- 172.21.15.5:0/500083664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd784102d30 0x7fd784103110 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.888+0000 7fd78ba3c640 1 -- 172.21.15.5:0/500083664 >> 172.21.15.5:0/500083664 conn(0x7fd7840fc7d0 msgr2=0x7fd7840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.888+0000 7fd78ba3c640 1 -- 172.21.15.5:0/500083664 shutdown_connections 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.888+0000 7fd78ba3c640 1 -- 172.21.15.5:0/500083664 wait complete. 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd78ba3c640 1 Processor -- start 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd78ba3c640 1 -- start start 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd78ba3c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd784102d30 0x7fd784100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd78ba3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7841036e0 0x7fd784100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd78ba3c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7841082a0 0x7fd784101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd78ba3c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7841120a0 con 0x7fd784102d30 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd78ba3c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd784111f20 con 0x7fd7841082a0 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd78ba3c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd784112220 con 0x7fd7841036e0 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd7897d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd784102d30 0x7fd784100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd7897d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd784102d30 0x7fd784100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57242/0 (socket says 172.21.15.5:57242) 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.889+0000 7fd7897d8640 1 -- 172.21.15.5:0/608094643 learned_addr learned my addr 172.21.15.5:0/608094643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd788fd7640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7841036e0 0x7fd784100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd789fd9640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7841082a0 0x7fd784101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd7897d8640 1 -- 172.21.15.5:0/608094643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7841036e0 msgr2=0x7fd784100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd7897d8640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7841036e0 0x7fd784100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd7897d8640 1 -- 172.21.15.5:0/608094643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7841082a0 msgr2=0x7fd784101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd7897d8640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7841082a0 0x7fd784101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd7897d8640 1 -- 172.21.15.5:0/608094643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7841a6a60 con 0x7fd784102d30 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd789fd9640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7841082a0 0x7fd784101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd7897d8640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd784102d30 0x7fd784100990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd780009230 tx=0x7fd780004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd77a7fc640 1 -- 172.21.15.5:0/608094643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78003f030 con 0x7fd784102d30 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd77a7fc640 1 -- 172.21.15.5:0/608094643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd780003ad0 con 0x7fd784102d30 2024-09-11T22:20:13.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd78ba3c640 1 -- 172.21.15.5:0/608094643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7841a6cf0 con 0x7fd784102d30 2024-09-11T22:20:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd77a7fc640 1 -- 172.21.15.5:0/608094643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd780035070 con 0x7fd784102d30 2024-09-11T22:20:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.890+0000 7fd78ba3c640 1 -- 172.21.15.5:0/608094643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7841a7180 con 0x7fd784102d30 2024-09-11T22:20:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.891+0000 7fd78ba3c640 1 -- 172.21.15.5:0/608094643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd784103110 con 0x7fd784102d30 2024-09-11T22:20:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.893+0000 7fd77a7fc640 1 -- 172.21.15.5:0/608094643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd780042030 con 0x7fd784102d30 2024-09-11T22:20:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.893+0000 7fd77a7fc640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd7700a2b80 0x7fd7700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.893+0000 7fd788fd7640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd7700a2b80 0x7fd7700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.893+0000 7fd77a7fc640 1 -- 172.21.15.5:0/608094643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd7800ef6a0 con 0x7fd784102d30 2024-09-11T22:20:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.894+0000 7fd788fd7640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd7700a2b80 0x7fd7700a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd784102420 tx=0x7fd77400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:13.897+0000 7fd77a7fc640 1 -- 172.21.15.5:0/608094643 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7800bca00 con 0x7fd784102d30 2024-09-11T22:20:14.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:14.108+0000 7fd78ba3c640 1 -- 172.21.15.5:0/608094643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd78410a2b0 con 0x7fd784102d30 2024-09-11T22:20:14.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:14 smithi005 ceph-mon[30943]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:14.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:14 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:14 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:14 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:14 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:14 smithi172 ceph-mon[35461]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:14.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:14 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:14 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:14 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:14 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:14 smithi159 ceph-mon[35534]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:14.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:14 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:14 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:14 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:14.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:14 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.105+0000 7fd77a7fc640 1 -- 172.21.15.5:0/608094643 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7fd7800bc300 con 0x7fd784102d30 2024-09-11T22:20:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-09-11T22:20:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.108+0000 7fd78ba3c640 1 -- 172.21.15.5:0/608094643 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd7700a2b80 msgr2=0x7fd7700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.108+0000 7fd78ba3c640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd7700a2b80 0x7fd7700a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd784102420 tx=0x7fd77400b410 comp rx=0 tx=0).stop 2024-09-11T22:20:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.108+0000 7fd78ba3c640 1 -- 172.21.15.5:0/608094643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd784102d30 msgr2=0x7fd784100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.108+0000 7fd78ba3c640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd784102d30 0x7fd784100990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd780009230 tx=0x7fd780004270 comp rx=0 tx=0).stop 2024-09-11T22:20:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.108+0000 7fd78ba3c640 1 -- 172.21.15.5:0/608094643 shutdown_connections 2024-09-11T22:20:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.108+0000 7fd78ba3c640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd7700a2b80 0x7fd7700a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.108+0000 7fd78ba3c640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7841082a0 0x7fd784101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.108+0000 7fd78ba3c640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7841036e0 0x7fd784100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.108+0000 7fd78ba3c640 1 --2- 172.21.15.5:0/608094643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd784102d30 0x7fd784100990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.108+0000 7fd78ba3c640 1 -- 172.21.15.5:0/608094643 >> 172.21.15.5:0/608094643 conn(0x7fd7840fc7d0 msgr2=0x7fd784105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.109+0000 7fd78ba3c640 1 -- 172.21.15.5:0/608094643 shutdown_connections 2024-09-11T22:20:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:15.109+0000 7fd78ba3c640 1 -- 172.21.15.5:0/608094643 wait complete. 2024-09-11T22:20:15.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:15 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/608094643' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T22:20:15.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:15.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:15.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:15.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:20:15.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:15.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:15 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/608094643' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T22:20:15.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:15.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:15.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:15.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:20:15.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:15 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/608094643' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-11T22:20:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:20:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:20:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:20:15.789 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T22:20:16.051 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:16.444 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:16 smithi005 ceph-mon[30943]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:16.444 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:16 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/608094643' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T22:20:16.444 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:16 smithi005 ceph-mon[30943]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T22:20:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.534+0000 7efd291a1640 1 -- 172.21.15.5:0/843292563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24103660 msgr2=0x7efd2410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.534+0000 7efd291a1640 1 --2- 172.21.15.5:0/843292563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24103660 0x7efd2410bb50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009900 tx=0x7efd0c02a480 comp rx=0 tx=0).stop 2024-09-11T22:20:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.534+0000 7efd291a1640 1 -- 172.21.15.5:0/843292563 shutdown_connections 2024-09-11T22:20:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.534+0000 7efd291a1640 1 --2- 172.21.15.5:0/843292563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd2410c170 0x7efd2410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.534+0000 7efd291a1640 1 --2- 172.21.15.5:0/843292563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24103660 0x7efd2410bb50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.534+0000 7efd291a1640 1 --2- 172.21.15.5:0/843292563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24102d40 0x7efd24103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.534+0000 7efd291a1640 1 -- 172.21.15.5:0/843292563 >> 172.21.15.5:0/843292563 conn(0x7efd240fc7a0 msgr2=0x7efd240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.534+0000 7efd291a1640 1 -- 172.21.15.5:0/843292563 shutdown_connections 2024-09-11T22:20:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.534+0000 7efd291a1640 1 -- 172.21.15.5:0/843292563 wait complete. 2024-09-11T22:20:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd291a1640 1 Processor -- start 2024-09-11T22:20:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd291a1640 1 -- start start 2024-09-11T22:20:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd291a1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd24102d40 0x7efd241983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd291a1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24103660 0x7efd24198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd291a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2410c170 0x7efd2419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd291a1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd24111f70 con 0x7efd2410c170 2024-09-11T22:20:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd291a1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd24111df0 con 0x7efd24103660 2024-09-11T22:20:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd291a1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd241120f0 con 0x7efd24102d40 2024-09-11T22:20:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd2359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2410c170 0x7efd2419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd2359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2410c170 0x7efd2419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57256/0 (socket says 172.21.15.5:57256) 2024-09-11T22:20:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.535+0000 7efd2359e640 1 -- 172.21.15.5:0/4230204478 learned_addr learned my addr 172.21.15.5:0/4230204478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd22d9d640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd24102d40 0x7efd241983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd1bfff640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24103660 0x7efd24198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd2359e640 1 -- 172.21.15.5:0/4230204478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd24102d40 msgr2=0x7efd241983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd2359e640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd24102d40 0x7efd241983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd2359e640 1 -- 172.21.15.5:0/4230204478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24103660 msgr2=0x7efd24198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd2359e640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24103660 0x7efd24198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd2359e640 1 -- 172.21.15.5:0/4230204478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2419d3a0 con 0x7efd2410c170 2024-09-11T22:20:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd22d9d640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd24102d40 0x7efd241983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:20:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd2359e640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2410c170 0x7efd2419ccc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00cce0 tx=0x7efd1c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd20d99640 1 -- 172.21.15.5:0/4230204478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c00f450 con 0x7efd2410c170 2024-09-11T22:20:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd20d99640 1 -- 172.21.15.5:0/4230204478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd1c00fad0 con 0x7efd2410c170 2024-09-11T22:20:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd291a1640 1 -- 172.21.15.5:0/4230204478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2419d690 con 0x7efd2410c170 2024-09-11T22:20:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd20d99640 1 -- 172.21.15.5:0/4230204478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c018c10 con 0x7efd2410c170 2024-09-11T22:20:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.536+0000 7efd291a1640 1 -- 172.21.15.5:0/4230204478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd241a4f70 con 0x7efd2410c170 2024-09-11T22:20:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.537+0000 7efd291a1640 1 -- 172.21.15.5:0/4230204478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd24103120 con 0x7efd2410c170 2024-09-11T22:20:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.538+0000 7efd20d99640 1 -- 172.21.15.5:0/4230204478 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7efd1c00f5f0 con 0x7efd2410c170 2024-09-11T22:20:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.539+0000 7efd20d99640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efd100a2b80 0x7efd100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.539+0000 7efd22d9d640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efd100a2b80 0x7efd100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.539+0000 7efd20d99640 1 -- 172.21.15.5:0/4230204478 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7efd1c0cf4f0 con 0x7efd2410c170 2024-09-11T22:20:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.540+0000 7efd22d9d640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efd100a2b80 0x7efd100a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efd14005fd0 tx=0x7efd1400b5f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.543+0000 7efd20d99640 1 -- 172.21.15.5:0/4230204478 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd1c014030 con 0x7efd2410c170 2024-09-11T22:20:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:16 smithi159 ceph-mon[35534]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:16 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/608094643' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T22:20:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:16 smithi159 ceph-mon[35534]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T22:20:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:16 smithi172 ceph-mon[35461]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:16 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/608094643' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-11T22:20:16.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:16 smithi172 ceph-mon[35461]: osdmap e76: 12 total, 12 up, 12 in 2024-09-11T22:20:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:16.720+0000 7efd291a1640 1 -- 172.21.15.5:0/4230204478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efd24199750 con 0x7efd2410c170 2024-09-11T22:20:17.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.156+0000 7efd20d99640 1 -- 172.21.15.5:0/4230204478 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7efd24199750 con 0x7efd2410c170 2024-09-11T22:20:17.157 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-09-11T22:20:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.159+0000 7efd291a1640 1 -- 172.21.15.5:0/4230204478 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efd100a2b80 msgr2=0x7efd100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.159+0000 7efd291a1640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efd100a2b80 0x7efd100a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efd14005fd0 tx=0x7efd1400b5f0 comp rx=0 tx=0).stop 2024-09-11T22:20:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.159+0000 7efd291a1640 1 -- 172.21.15.5:0/4230204478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2410c170 msgr2=0x7efd2419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.159+0000 7efd291a1640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2410c170 0x7efd2419ccc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00cce0 tx=0x7efd1c00a620 comp rx=0 tx=0).stop 2024-09-11T22:20:17.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.160+0000 7efd291a1640 1 -- 172.21.15.5:0/4230204478 shutdown_connections 2024-09-11T22:20:17.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.160+0000 7efd291a1640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efd100a2b80 0x7efd100a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:17.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.160+0000 7efd291a1640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd2410c170 0x7efd2419ccc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:17.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.160+0000 7efd291a1640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24103660 0x7efd24198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:17.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.160+0000 7efd291a1640 1 --2- 172.21.15.5:0/4230204478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd24102d40 0x7efd241983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:17.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.160+0000 7efd291a1640 1 -- 172.21.15.5:0/4230204478 >> 172.21.15.5:0/4230204478 conn(0x7efd240fc7a0 msgr2=0x7efd240fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.160+0000 7efd291a1640 1 -- 172.21.15.5:0/4230204478 shutdown_connections 2024-09-11T22:20:17.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:17.161+0000 7efd291a1640 1 -- 172.21.15.5:0/4230204478 wait complete. 2024-09-11T22:20:17.531 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:17 smithi005 ceph-mon[30943]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T22:20:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:17 smithi159 ceph-mon[35534]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T22:20:17.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:17 smithi172 ceph-mon[35461]: osdmap e77: 12 total, 12 up, 12 in 2024-09-11T22:20:17.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T22:20:18.178 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:18 smithi005 ceph-mon[30943]: pgmap v277: 33 pgs: 8 creating+peering, 21 unknown, 4 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:18.178 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:18 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4230204478' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T22:20:18.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:18 smithi005 ceph-mon[30943]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:20:18.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:18 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4230204478' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T22:20:18.181 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:18 smithi005 ceph-mon[30943]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T22:20:18.209 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:18 smithi159 ceph-mon[35534]: pgmap v277: 33 pgs: 8 creating+peering, 21 unknown, 4 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:18 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4230204478' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T22:20:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:18 smithi159 ceph-mon[35534]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:20:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:18 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4230204478' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T22:20:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:18 smithi159 ceph-mon[35534]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T22:20:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:18 smithi172 ceph-mon[35461]: pgmap v277: 33 pgs: 8 creating+peering, 21 unknown, 4 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:18 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4230204478' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-11T22:20:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:18 smithi172 ceph-mon[35461]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T22:20:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:18 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4230204478' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-11T22:20:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:18 smithi172 ceph-mon[35461]: osdmap e78: 12 total, 12 up, 12 in 2024-09-11T22:20:18.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.679+0000 7fac54e99640 1 -- 172.21.15.5:0/1327990983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac50106a80 msgr2=0x7fac50106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.679+0000 7fac54e99640 1 --2- 172.21.15.5:0/1327990983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac50106a80 0x7fac50106e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fac40009930 tx=0x7fac4002a420 comp rx=0 tx=0).stop 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.679+0000 7fac54e99640 1 -- 172.21.15.5:0/1327990983 shutdown_connections 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.679+0000 7fac54e99640 1 --2- 172.21.15.5:0/1327990983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac50101420 0x7fac5010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.679+0000 7fac54e99640 1 --2- 172.21.15.5:0/1327990983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac50100a80 0x7fac50100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.679+0000 7fac54e99640 1 --2- 172.21.15.5:0/1327990983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac50106a80 0x7fac50106e60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.679+0000 7fac54e99640 1 -- 172.21.15.5:0/1327990983 >> 172.21.15.5:0/1327990983 conn(0x7fac500fc7a0 msgr2=0x7fac500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.679+0000 7fac54e99640 1 -- 172.21.15.5:0/1327990983 shutdown_connections 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.679+0000 7fac54e99640 1 -- 172.21.15.5:0/1327990983 wait complete. 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac54e99640 1 Processor -- start 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac54e99640 1 -- start start 2024-09-11T22:20:18.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac54e99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac50100a80 0x7fac50198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac54e99640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac50101420 0x7fac501989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac54e99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac50106a80 0x7fac5019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac54e99640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac50110ea0 con 0x7fac50100a80 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac54e99640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fac50110d20 con 0x7fac50101420 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac54e99640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fac50111020 con 0x7fac50106a80 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac4e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac50100a80 0x7fac50198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac4e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac50100a80 0x7fac50198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57280/0 (socket says 172.21.15.5:57280) 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac4e59c640 1 -- 172.21.15.5:0/2133543494 learned_addr learned my addr 172.21.15.5:0/2133543494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac4ed9d640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac50106a80 0x7fac5019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac47fff640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac50101420 0x7fac501989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:18.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac4e59c640 1 -- 172.21.15.5:0/2133543494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac50106a80 msgr2=0x7fac5019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac4e59c640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac50106a80 0x7fac5019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac4e59c640 1 -- 172.21.15.5:0/2133543494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac50101420 msgr2=0x7fac501989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac4e59c640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac50101420 0x7fac501989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac4e59c640 1 -- 172.21.15.5:0/2133543494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5019d410 con 0x7fac50100a80 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac4ed9d640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac50106a80 0x7fac5019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.680+0000 7fac47fff640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac50101420 0x7fac501989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.681+0000 7fac4e59c640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac50100a80 0x7fac50198460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac40009a50 tx=0x7fac40004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.681+0000 7fac477fe640 1 -- 172.21.15.5:0/2133543494 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4003f030 con 0x7fac50100a80 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.681+0000 7fac477fe640 1 -- 172.21.15.5:0/2133543494 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac40003ce0 con 0x7fac50100a80 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.681+0000 7fac54e99640 1 -- 172.21.15.5:0/2133543494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5019d6a0 con 0x7fac50100a80 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.681+0000 7fac477fe640 1 -- 172.21.15.5:0/2133543494 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40035070 con 0x7fac50100a80 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.681+0000 7fac54e99640 1 -- 172.21.15.5:0/2133543494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac501a4fe0 con 0x7fac50100a80 2024-09-11T22:20:18.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.682+0000 7fac54e99640 1 -- 172.21.15.5:0/2133543494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac501021c0 con 0x7fac50100a80 2024-09-11T22:20:18.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.683+0000 7fac477fe640 1 -- 172.21.15.5:0/2133543494 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fac40042030 con 0x7fac50100a80 2024-09-11T22:20:18.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.684+0000 7fac477fe640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fac3c0a2b80 0x7fac3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:18.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.684+0000 7fac47fff640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fac3c0a2b80 0x7fac3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:18.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.684+0000 7fac477fe640 1 -- 172.21.15.5:0/2133543494 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7fac400efa20 con 0x7fac50100a80 2024-09-11T22:20:18.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.684+0000 7fac47fff640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fac3c0a2b80 0x7fac3c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fac38006fd0 tx=0x7fac38008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:18.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.688+0000 7fac477fe640 1 -- 172.21.15.5:0/2133543494 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac400bca90 con 0x7fac50100a80 2024-09-11T22:20:18.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:18.899+0000 7fac54e99640 1 -- 172.21.15.5:0/2133543494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fac50100ee0 con 0x7fac50100a80 2024-09-11T22:20:19.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.177+0000 7fac477fe640 1 -- 172.21.15.5:0/2133543494 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fac400bc390 con 0x7fac50100a80 2024-09-11T22:20:19.178 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T22:20:19.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 -- 172.21.15.5:0/2133543494 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fac3c0a2b80 msgr2=0x7fac3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fac3c0a2b80 0x7fac3c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fac38006fd0 tx=0x7fac38008040 comp rx=0 tx=0).stop 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 -- 172.21.15.5:0/2133543494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac50100a80 msgr2=0x7fac50198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac50100a80 0x7fac50198460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac40009a50 tx=0x7fac40004270 comp rx=0 tx=0).stop 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 -- 172.21.15.5:0/2133543494 shutdown_connections 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fac3c0a2b80 0x7fac3c0a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fac50106a80 0x7fac5019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac50101420 0x7fac501989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 --2- 172.21.15.5:0/2133543494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac50100a80 0x7fac50198460 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 -- 172.21.15.5:0/2133543494 >> 172.21.15.5:0/2133543494 conn(0x7fac500fc7a0 msgr2=0x7fac50104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 -- 172.21.15.5:0/2133543494 shutdown_connections 2024-09-11T22:20:19.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:19.180+0000 7fac54e99640 1 -- 172.21.15.5:0/2133543494 wait complete. 2024-09-11T22:20:19.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:19 smithi005 ceph-mon[30943]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T22:20:19.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:19 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2133543494' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T22:20:19.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:19 smithi005 ceph-mon[30943]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T22:20:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:19 smithi159 ceph-mon[35534]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T22:20:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:19 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2133543494' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T22:20:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:19 smithi159 ceph-mon[35534]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T22:20:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:19 smithi172 ceph-mon[35461]: osdmap e79: 12 total, 12 up, 12 in 2024-09-11T22:20:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:19 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2133543494' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-11T22:20:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:19 smithi172 ceph-mon[35461]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-11T22:20:19.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:20:20.162 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:20.452 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:20 smithi005 ceph-mon[30943]: pgmap v280: 97 pgs: 38 creating+peering, 40 unknown, 19 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:20.452 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:20 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2133543494' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T22:20:20.452 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:20 smithi005 ceph-mon[30943]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T22:20:20.452 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:20 smithi005 ceph-mon[30943]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T22:20:20.452 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:20 smithi005 ceph-mon[30943]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T22:20:20.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:20 smithi159 ceph-mon[35534]: pgmap v280: 97 pgs: 38 creating+peering, 40 unknown, 19 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:20.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:20 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2133543494' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T22:20:20.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:20 smithi159 ceph-mon[35534]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T22:20:20.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:20 smithi159 ceph-mon[35534]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T22:20:20.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:20 smithi159 ceph-mon[35534]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T22:20:20.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:20 smithi172 ceph-mon[35461]: pgmap v280: 97 pgs: 38 creating+peering, 40 unknown, 19 active+clean; 577 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:20:20.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:20 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2133543494' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-11T22:20:20.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:20 smithi172 ceph-mon[35461]: osdmap e80: 12 total, 12 up, 12 in 2024-09-11T22:20:20.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:20 smithi172 ceph-mon[35461]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-11T22:20:20.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:20 smithi172 ceph-mon[35461]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-11T22:20:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.637+0000 7f5af43b6640 1 -- 172.21.15.5:0/92569924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aec100ca0 msgr2=0x7f5aec106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.637+0000 7f5af43b6640 1 --2- 172.21.15.5:0/92569924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aec100ca0 0x7f5aec106840 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009930 tx=0x7f5adc02a420 comp rx=0 tx=0).stop 2024-09-11T22:20:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.637+0000 7f5af43b6640 1 -- 172.21.15.5:0/92569924 shutdown_connections 2024-09-11T22:20:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.637+0000 7f5af43b6640 1 --2- 172.21.15.5:0/92569924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aec106f50 0x7f5aec10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.637+0000 7f5af43b6640 1 --2- 172.21.15.5:0/92569924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aec100ca0 0x7f5aec106840 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.637+0000 7f5af43b6640 1 --2- 172.21.15.5:0/92569924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5aec1002f0 0x7f5aec1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.637+0000 7f5af43b6640 1 -- 172.21.15.5:0/92569924 >> 172.21.15.5:0/92569924 conn(0x7f5aec077210 msgr2=0x7f5aec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.637+0000 7f5af43b6640 1 -- 172.21.15.5:0/92569924 shutdown_connections 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.637+0000 7f5af43b6640 1 -- 172.21.15.5:0/92569924 wait complete. 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af43b6640 1 Processor -- start 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af43b6640 1 -- start start 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af43b6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5aec1002f0 0x7f5aec101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af43b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aec100ca0 0x7f5aec1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af43b6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aec106f50 0x7f5aec101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af43b6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec111f60 con 0x7f5aec100ca0 2024-09-11T22:20:20.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af43b6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec111de0 con 0x7f5aec1002f0 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af43b6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec1120e0 con 0x7f5aec106f50 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af1951640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aec100ca0 0x7f5aec1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af1951640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aec100ca0 0x7f5aec1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57310/0 (socket says 172.21.15.5:57310) 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af1951640 1 -- 172.21.15.5:0/3588692628 learned_addr learned my addr 172.21.15.5:0/3588692628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af2953640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aec106f50 0x7f5aec101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af2152640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5aec1002f0 0x7f5aec101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af1951640 1 -- 172.21.15.5:0/3588692628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aec106f50 msgr2=0x7f5aec101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af1951640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aec106f50 0x7f5aec101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af1951640 1 -- 172.21.15.5:0/3588692628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5aec1002f0 msgr2=0x7f5aec101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af1951640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5aec1002f0 0x7f5aec101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af1951640 1 -- 172.21.15.5:0/3588692628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aec1a6970 con 0x7f5aec100ca0 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.638+0000 7f5af2152640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5aec1002f0 0x7f5aec101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:20:20.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.639+0000 7f5af1951640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aec100ca0 0x7f5aec1017b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5adc005940 tx=0x7f5adc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.639+0000 7f5ae37fe640 1 -- 172.21.15.5:0/3588692628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc03f030 con 0x7f5aec100ca0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.639+0000 7f5af43b6640 1 -- 172.21.15.5:0/3588692628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aec1a6c00 con 0x7f5aec100ca0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.639+0000 7f5ae37fe640 1 -- 172.21.15.5:0/3588692628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5adc003ce0 con 0x7f5aec100ca0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.639+0000 7f5ae37fe640 1 -- 172.21.15.5:0/3588692628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5adc035070 con 0x7f5aec100ca0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.639+0000 7f5af43b6640 1 -- 172.21.15.5:0/3588692628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aec1a7130 con 0x7f5aec100ca0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.640+0000 7f5af43b6640 1 -- 172.21.15.5:0/3588692628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aec1006d0 con 0x7f5aec100ca0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.641+0000 7f5ae37fe640 1 -- 172.21.15.5:0/3588692628 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5adc042030 con 0x7f5aec100ca0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.642+0000 7f5ae37fe640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ad80a2b80 0x7f5ad80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.642+0000 7f5af2152640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ad80a2b80 0x7f5ad80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.642+0000 7f5ae37fe640 1 -- 172.21.15.5:0/3588692628 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f5adc0ef4b0 con 0x7f5aec100ca0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.642+0000 7f5af2152640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ad80a2b80 0x7f5ad80a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80058c0 tx=0x7f5ae8005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:20.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.646+0000 7f5ae37fe640 1 -- 172.21.15.5:0/3588692628 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5adc0bc4c0 con 0x7f5aec100ca0 2024-09-11T22:20:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.825+0000 7f5af43b6640 1 -- 172.21.15.5:0/3588692628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5aec02d7c0 con 0x7f5aec100ca0 2024-09-11T22:20:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.827+0000 7f5ae37fe640 1 -- 172.21.15.5:0/3588692628 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22173 (secure 0 0 0) 0x7f5adc0bbdc0 con 0x7f5aec100ca0 2024-09-11T22:20:20.828 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:20.828 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":80,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:20:18.900590+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:20:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.829+0000 7f5af43b6640 1 -- 172.21.15.5:0/3588692628 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ad80a2b80 msgr2=0x7f5ad80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.829+0000 7f5af43b6640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ad80a2b80 0x7f5ad80a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80058c0 tx=0x7f5ae8005ec0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.829+0000 7f5af43b6640 1 -- 172.21.15.5:0/3588692628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aec100ca0 msgr2=0x7f5aec1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.829+0000 7f5af43b6640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aec100ca0 0x7f5aec1017b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5adc005940 tx=0x7f5adc0042f0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.829+0000 7f5af43b6640 1 -- 172.21.15.5:0/3588692628 shutdown_connections 2024-09-11T22:20:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.829+0000 7f5af43b6640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5ad80a2b80 0x7f5ad80a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.829+0000 7f5af43b6640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5aec106f50 0x7f5aec101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.829+0000 7f5af43b6640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5aec100ca0 0x7f5aec1017b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.829+0000 7f5af43b6640 1 --2- 172.21.15.5:0/3588692628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5aec1002f0 0x7f5aec101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.829+0000 7f5af43b6640 1 -- 172.21.15.5:0/3588692628 >> 172.21.15.5:0/3588692628 conn(0x7f5aec077210 msgr2=0x7f5aec0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.830+0000 7f5af43b6640 1 -- 172.21.15.5:0/3588692628 shutdown_connections 2024-09-11T22:20:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:20.830+0000 7f5af43b6640 1 -- 172.21.15.5:0/3588692628 wait complete. 2024-09-11T22:20:21.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:21 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] up:active 2024-09-11T22:20:21.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:21 smithi159 ceph-mon[35534]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:21.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:21 smithi159 ceph-mon[35534]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:21.488 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:21 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3588692628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:21.494 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:21 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] up:active 2024-09-11T22:20:21.494 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:21 smithi005 ceph-mon[30943]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:21.494 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:21 smithi005 ceph-mon[30943]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:21.494 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:21 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3588692628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:21.530 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:20:21.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:21 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] up:active 2024-09-11T22:20:21.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:21 smithi172 ceph-mon[35461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:21.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:21 smithi172 ceph-mon[35461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:21.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:21 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3588692628' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:21.794 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.365+0000 7f4f5d669640 1 -- 172.21.15.5:0/3608743214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f58102790 msgr2=0x7f4f581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.365+0000 7f4f5d669640 1 --2- 172.21.15.5:0/3608743214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f58102790 0x7f4f581067e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009900 tx=0x7f4f4c02a410 comp rx=0 tx=0).stop 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.365+0000 7f4f5d669640 1 -- 172.21.15.5:0/3608743214 shutdown_connections 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.365+0000 7f4f5d669640 1 --2- 172.21.15.5:0/3608743214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f58106f10 0x7f4f5810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.365+0000 7f4f5d669640 1 --2- 172.21.15.5:0/3608743214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f58102790 0x7f4f581067e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.365+0000 7f4f5d669640 1 --2- 172.21.15.5:0/3608743214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f58101de0 0x7f4f581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.365+0000 7f4f5d669640 1 -- 172.21.15.5:0/3608743214 >> 172.21.15.5:0/3608743214 conn(0x7f4f580fd8c0 msgr2=0x7f4f580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.365+0000 7f4f5d669640 1 -- 172.21.15.5:0/3608743214 shutdown_connections 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.365+0000 7f4f5d669640 1 -- 172.21.15.5:0/3608743214 wait complete. 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f5d669640 1 Processor -- start 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f5d669640 1 -- start start 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f5d669640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f58101de0 0x7f4f58198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f5d669640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f58102790 0x7f4f58198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:22.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f5d669640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f58106f10 0x7f4f5819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f5d669640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f58111fc0 con 0x7f4f58102790 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f5d669640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4f58111e40 con 0x7f4f58106f10 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f5d669640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4f58112140 con 0x7f4f58101de0 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f567fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f58102790 0x7f4f58198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f567fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f58102790 0x7f4f58198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40428/0 (socket says 172.21.15.5:40428) 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f567fc640 1 -- 172.21.15.5:0/762647109 learned_addr learned my addr 172.21.15.5:0/762647109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f577fe640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f58106f10 0x7f4f5819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f56ffd640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f58101de0 0x7f4f58198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f567fc640 1 -- 172.21.15.5:0/762647109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f58101de0 msgr2=0x7f4f58198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f567fc640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f58101de0 0x7f4f58198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f567fc640 1 -- 172.21.15.5:0/762647109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f58106f10 msgr2=0x7f4f5819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f567fc640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f58106f10 0x7f4f5819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f567fc640 1 -- 172.21.15.5:0/762647109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5819d3d0 con 0x7f4f58102790 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f56ffd640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f58101de0 0x7f4f58198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.366+0000 7f4f577fe640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f58106f10 0x7f4f5819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.367+0000 7f4f567fc640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f58102790 0x7f4f58198960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c004770 tx=0x7f4f4c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.367+0000 7f4f3ffff640 1 -- 172.21.15.5:0/762647109 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c03f070 con 0x7f4f58102790 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.367+0000 7f4f3ffff640 1 -- 172.21.15.5:0/762647109 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c031ca0 con 0x7f4f58102790 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.367+0000 7f4f5d669640 1 -- 172.21.15.5:0/762647109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5819d660 con 0x7f4f58102790 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.367+0000 7f4f3ffff640 1 -- 172.21.15.5:0/762647109 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c035030 con 0x7f4f58102790 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.367+0000 7f4f5d669640 1 -- 172.21.15.5:0/762647109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f581a4fa0 con 0x7f4f58102790 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.368+0000 7f4f5d669640 1 -- 172.21.15.5:0/762647109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f581032c0 con 0x7f4f58102790 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.369+0000 7f4f3ffff640 1 -- 172.21.15.5:0/762647109 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4f4c042070 con 0x7f4f58102790 2024-09-11T22:20:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.370+0000 7f4f3ffff640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4f400a2b80 0x7f4f400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:22.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.370+0000 7f4f56ffd640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4f400a2b80 0x7f4f400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:22.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.370+0000 7f4f3ffff640 1 -- 172.21.15.5:0/762647109 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f4f4c0ef7c0 con 0x7f4f58102790 2024-09-11T22:20:22.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.370+0000 7f4f56ffd640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4f400a2b80 0x7f4f400a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4f480058c0 tx=0x7f4f48005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:22.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.374+0000 7f4f3ffff640 1 -- 172.21.15.5:0/762647109 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f4c0bc870 con 0x7f4f58102790 2024-09-11T22:20:22.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.551+0000 7f4f5d669640 1 -- 172.21.15.5:0/762647109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f5810a150 con 0x7f4f58102790 2024-09-11T22:20:22.552 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:22 smithi005 ceph-mon[30943]: pgmap v282: 97 pgs: 30 creating+peering, 21 unknown, 46 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-09-11T22:20:22.552 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:22 smithi005 ceph-mon[30943]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T22:20:22.552 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:22 smithi005 ceph-mon[30943]: Cluster is now healthy 2024-09-11T22:20:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.552+0000 7f4f3ffff640 1 -- 172.21.15.5:0/762647109 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22173 (secure 0 0 0) 0x7f4f4c0bc170 con 0x7f4f58102790 2024-09-11T22:20:22.553 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:22.553 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":80,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:20:18.900590+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 -- 172.21.15.5:0/762647109 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4f400a2b80 msgr2=0x7f4f400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4f400a2b80 0x7f4f400a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4f480058c0 tx=0x7f4f48005d30 comp rx=0 tx=0).stop 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 -- 172.21.15.5:0/762647109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f58102790 msgr2=0x7f4f58198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f58102790 0x7f4f58198960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c004770 tx=0x7f4f4c003c10 comp rx=0 tx=0).stop 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 -- 172.21.15.5:0/762647109 shutdown_connections 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4f400a2b80 0x7f4f400a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4f58106f10 0x7f4f5819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f58102790 0x7f4f58198960 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 --2- 172.21.15.5:0/762647109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f58101de0 0x7f4f58198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 -- 172.21.15.5:0/762647109 >> 172.21.15.5:0/762647109 conn(0x7f4f580fd8c0 msgr2=0x7f4f580fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 -- 172.21.15.5:0/762647109 shutdown_connections 2024-09-11T22:20:22.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:22.555+0000 7f4f5d669640 1 -- 172.21.15.5:0/762647109 wait complete. 2024-09-11T22:20:22.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:22 smithi159 ceph-mon[35534]: pgmap v282: 97 pgs: 30 creating+peering, 21 unknown, 46 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-09-11T22:20:22.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:22 smithi159 ceph-mon[35534]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T22:20:22.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:22 smithi159 ceph-mon[35534]: Cluster is now healthy 2024-09-11T22:20:22.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:22 smithi172 ceph-mon[35461]: pgmap v282: 97 pgs: 30 creating+peering, 21 unknown, 46 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-09-11T22:20:22.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:22 smithi172 ceph-mon[35461]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-11T22:20:22.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:22 smithi172 ceph-mon[35461]: Cluster is now healthy 2024-09-11T22:20:23.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-11T22:20:23.497 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:23.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:23 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/762647109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:23.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:23 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/762647109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:23.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:23 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/762647109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:24.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.983+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1755201860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc010c170 msgr2=0x7f3bc010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:24.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.983+0000 7f3bc72d5640 1 --2- 172.21.15.5:0/1755201860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc010c170 0x7f3bc010e610 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8009ca0 tx=0x7f3bb802a950 comp rx=0 tx=0).stop 2024-09-11T22:20:24.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.983+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1755201860 shutdown_connections 2024-09-11T22:20:24.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.983+0000 7f3bc72d5640 1 --2- 172.21.15.5:0/1755201860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc010c170 0x7f3bc010e610 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.983+0000 7f3bc72d5640 1 --2- 172.21.15.5:0/1755201860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0103660 0x7f3bc010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.983+0000 7f3bc72d5640 1 --2- 172.21.15.5:0/1755201860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bc0102d40 0x7f3bc0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.983+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1755201860 >> 172.21.15.5:0/1755201860 conn(0x7f3bc00fc7a0 msgr2=0x7f3bc00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.983+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1755201860 shutdown_connections 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.983+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1755201860 wait complete. 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc72d5640 1 Processor -- start 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc72d5640 1 -- start start 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc72d5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102d40 0x7f3bc01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc72d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc0103660 0x7f3bc0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc72d5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bc010c170 0x7f3bc0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc72d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc0112030 con 0x7f3bc0103660 2024-09-11T22:20:24.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc72d5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc0111eb0 con 0x7f3bc010c170 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc72d5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc01121b0 con 0x7f3bc0102d40 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc4870640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc0103660 0x7f3bc0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc4870640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc0103660 0x7f3bc0100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40450/0 (socket says 172.21.15.5:40450) 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc4870640 1 -- 172.21.15.5:0/1354982857 learned_addr learned my addr 172.21.15.5:0/1354982857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc5872640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bc010c170 0x7f3bc0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.984+0000 7f3bc5071640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102d40 0x7f3bc01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bc4870640 1 -- 172.21.15.5:0/1354982857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102d40 msgr2=0x7f3bc01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bc4870640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102d40 0x7f3bc01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bc4870640 1 -- 172.21.15.5:0/1354982857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bc010c170 msgr2=0x7f3bc0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bc4870640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bc010c170 0x7f3bc0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bc4870640 1 -- 172.21.15.5:0/1354982857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc01a69f0 con 0x7f3bc0103660 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bc4870640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc0103660 0x7f3bc0100ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000b3f0 tx=0x7f3bb000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bb67fc640 1 -- 172.21.15.5:0/1354982857 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0015020 con 0x7f3bc0103660 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1354982857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc01a6ce0 con 0x7f3bc0103660 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bb67fc640 1 -- 172.21.15.5:0/1354982857 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bb0011d10 con 0x7f3bc0103660 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bb67fc640 1 -- 172.21.15.5:0/1354982857 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb000f2e0 con 0x7f3bc0103660 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.985+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1354982857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc01a71c0 con 0x7f3bc0103660 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.986+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1354982857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc01031a0 con 0x7f3bc0103660 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.988+0000 7f3bb67fc640 1 -- 172.21.15.5:0/1354982857 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3bb0018020 con 0x7f3bc0103660 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.988+0000 7f3bb67fc640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f3bac0a2b80 0x7f3bac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.988+0000 7f3bc5071640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f3bac0a2b80 0x7f3bac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.988+0000 7f3bb67fc640 1 -- 172.21.15.5:0/1354982857 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f3bb00ced40 con 0x7f3bc0103660 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.989+0000 7f3bc5071640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f3bac0a2b80 0x7f3bac0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009890 tx=0x7f3bbc008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:23.991+0000 7f3bb67fc640 1 -- 172.21.15.5:0/1354982857 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3bb001f020 con 0x7f3bc0103660 2024-09-11T22:20:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.205+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1354982857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3bc01071e0 con 0x7f3bc0103660 2024-09-11T22:20:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.212+0000 7f3bb67fc640 1 -- 172.21.15.5:0/1354982857 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f3bb009bdf0 con 0x7f3bc0103660 2024-09-11T22:20:24.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.214+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1354982857 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f3bac0a2b80 msgr2=0x7f3bac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:24.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.214+0000 7f3bc72d5640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f3bac0a2b80 0x7f3bac0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009890 tx=0x7f3bbc008040 comp rx=0 tx=0).stop 2024-09-11T22:20:24.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.214+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1354982857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc0103660 msgr2=0x7f3bc0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:24.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.214+0000 7f3bc72d5640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc0103660 0x7f3bc0100ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000b3f0 tx=0x7f3bb000b8b0 comp rx=0 tx=0).stop 2024-09-11T22:20:24.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.215+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1354982857 shutdown_connections 2024-09-11T22:20:24.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.215+0000 7f3bc72d5640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f3bac0a2b80 0x7f3bac0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:24.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.215+0000 7f3bc72d5640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3bc010c170 0x7f3bc0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:24.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.215+0000 7f3bc72d5640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc0103660 0x7f3bc0100ee0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:24.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.215+0000 7f3bc72d5640 1 --2- 172.21.15.5:0/1354982857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc0102d40 0x7f3bc01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:24.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.215+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1354982857 >> 172.21.15.5:0/1354982857 conn(0x7f3bc00fc7a0 msgr2=0x7f3bc00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:24.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.215+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1354982857 shutdown_connections 2024-09-11T22:20:24.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:24.215+0000 7f3bc72d5640 1 -- 172.21.15.5:0/1354982857 wait complete. 2024-09-11T22:20:24.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:24 smithi005 ceph-mon[30943]: pgmap v283: 97 pgs: 16 creating+peering, 81 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-09-11T22:20:24.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:24 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1354982857' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T22:20:24.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:24 smithi159 ceph-mon[35534]: pgmap v283: 97 pgs: 16 creating+peering, 81 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-09-11T22:20:24.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:24 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1354982857' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T22:20:24.579 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:24 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:20:24.293+0000 7f54cb2b2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T22:20:24.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:24 smithi172 ceph-mon[35461]: pgmap v283: 97 pgs: 16 creating+peering, 81 active+clean; 578 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 2 op/s 2024-09-11T22:20:24.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:24 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1354982857' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-11T22:20:24.851 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-11T22:20:24.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-11T22:20:25.117 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:25.412 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:25 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1354982857' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T22:20:25.413 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:25 smithi005 ceph-mon[30943]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:25.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:25 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1354982857' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T22:20:25.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:25 smithi159 ceph-mon[35534]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:25 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1354982857' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-11T22:20:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:25 smithi172 ceph-mon[35461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:25.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.619+0000 7f8427163640 1 -- 172.21.15.5:0/3803061601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420102790 msgr2=0x7f84201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:25.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.619+0000 7f8427163640 1 --2- 172.21.15.5:0/3803061601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420102790 0x7f84201067e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f840c009900 tx=0x7f840c02a410 comp rx=0 tx=0).stop 2024-09-11T22:20:25.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.619+0000 7f8427163640 1 -- 172.21.15.5:0/3803061601 shutdown_connections 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.619+0000 7f8427163640 1 --2- 172.21.15.5:0/3803061601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8420106f10 0x7f842010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.619+0000 7f8427163640 1 --2- 172.21.15.5:0/3803061601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420102790 0x7f84201067e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.619+0000 7f8427163640 1 --2- 172.21.15.5:0/3803061601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8420101de0 0x7f84201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.619+0000 7f8427163640 1 -- 172.21.15.5:0/3803061601 >> 172.21.15.5:0/3803061601 conn(0x7f84200fd8c0 msgr2=0x7f84200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.619+0000 7f8427163640 1 -- 172.21.15.5:0/3803061601 shutdown_connections 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.619+0000 7f8427163640 1 -- 172.21.15.5:0/3803061601 wait complete. 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.620+0000 7f8427163640 1 Processor -- start 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.620+0000 7f8427163640 1 -- start start 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.620+0000 7f8427163640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420101de0 0x7f8420198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.620+0000 7f8427163640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8420102790 0x7f84201989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.620+0000 7f8427163640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8420106f10 0x7f842019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.620+0000 7f8427163640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8420111f70 con 0x7f8420101de0 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.620+0000 7f8427163640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8420111df0 con 0x7f8420106f10 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.620+0000 7f8427163640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84201120f0 con 0x7f8420102790 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8424eff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420101de0 0x7f8420198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8424eff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420101de0 0x7f8420198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40468/0 (socket says 172.21.15.5:40468) 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8424eff640 1 -- 172.21.15.5:0/2757154336 learned_addr learned my addr 172.21.15.5:0/2757154336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f841ffff640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8420102790 0x7f84201989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8425700640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8420106f10 0x7f842019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8424eff640 1 -- 172.21.15.5:0/2757154336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8420102790 msgr2=0x7f84201989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8424eff640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8420102790 0x7f84201989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8424eff640 1 -- 172.21.15.5:0/2757154336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8420106f10 msgr2=0x7f842019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8424eff640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8420106f10 0x7f842019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8424eff640 1 -- 172.21.15.5:0/2757154336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842019d420 con 0x7f8420101de0 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8424eff640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420101de0 0x7f8420198470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f841400e9f0 tx=0x7f841400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f841dffb640 1 -- 172.21.15.5:0/2757154336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841400cd80 con 0x7f8420101de0 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f841dffb640 1 -- 172.21.15.5:0/2757154336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8414010910 con 0x7f8420101de0 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f8427163640 1 -- 172.21.15.5:0/2757154336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842019d710 con 0x7f8420101de0 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.621+0000 7f841dffb640 1 -- 172.21.15.5:0/2757154336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414018990 con 0x7f8420101de0 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.622+0000 7f8427163640 1 -- 172.21.15.5:0/2757154336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84201a4ff0 con 0x7f8420101de0 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.622+0000 7f8427163640 1 -- 172.21.15.5:0/2757154336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84201021c0 con 0x7f8420101de0 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.624+0000 7f841dffb640 1 -- 172.21.15.5:0/2757154336 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8414010ac0 con 0x7f8420101de0 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.624+0000 7f841dffb640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f84100a2c00 0x7f84100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.625+0000 7f841ffff640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f84100a2c00 0x7f84100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.625+0000 7f841dffb640 1 -- 172.21.15.5:0/2757154336 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f84140cf210 con 0x7f8420101de0 2024-09-11T22:20:25.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.625+0000 7f841ffff640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f84100a2c00 0x7f84100a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f840c009230 tx=0x7f840c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:25.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.629+0000 7f841dffb640 1 -- 172.21.15.5:0/2757154336 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f841409c2c0 con 0x7f8420101de0 2024-09-11T22:20:25.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:25.847+0000 7f8427163640 1 -- 172.21.15.5:0/2757154336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f842010a1b0 con 0x7f8420101de0 2024-09-11T22:20:26.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.224+0000 7f841dffb640 1 -- 172.21.15.5:0/2757154336 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f841409bbc0 con 0x7f8420101de0 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 -- 172.21.15.5:0/2757154336 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f84100a2c00 msgr2=0x7f84100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f84100a2c00 0x7f84100a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f840c009230 tx=0x7f840c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 -- 172.21.15.5:0/2757154336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420101de0 msgr2=0x7f8420198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420101de0 0x7f8420198470 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f841400e9f0 tx=0x7f841400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 -- 172.21.15.5:0/2757154336 shutdown_connections 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f84100a2c00 0x7f84100a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8420106f10 0x7f842019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8420102790 0x7f84201989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 --2- 172.21.15.5:0/2757154336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420101de0 0x7f8420198470 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 -- 172.21.15.5:0/2757154336 >> 172.21.15.5:0/2757154336 conn(0x7f84200fd8c0 msgr2=0x7f84200fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 -- 172.21.15.5:0/2757154336 shutdown_connections 2024-09-11T22:20:26.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:26.227+0000 7f8427163640 1 -- 172.21.15.5:0/2757154336 wait complete. 2024-09-11T22:20:26.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:26 smithi159 ceph-mon[35534]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-11T22:20:26.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:26 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2757154336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T22:20:26.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:26 smithi005 ceph-mon[30943]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-11T22:20:26.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:26 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2757154336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T22:20:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:26 smithi172 ceph-mon[35461]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-09-11T22:20:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:26 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2757154336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-11T22:20:26.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-11T22:20:27.214 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:27.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:27 smithi005 ceph-mon[30943]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T22:20:27.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:27 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2757154336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T22:20:27.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:27 smithi005 ceph-mon[30943]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T22:20:27.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:27 smithi005 ceph-mon[30943]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:27.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:27 smithi005 ceph-mon[30943]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-11T22:20:27.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:27 smithi005 ceph-mon[30943]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-11T22:20:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:27 smithi159 ceph-mon[35534]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T22:20:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:27 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2757154336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T22:20:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:27 smithi159 ceph-mon[35534]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T22:20:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:27 smithi159 ceph-mon[35534]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:27 smithi159 ceph-mon[35534]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-11T22:20:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:27 smithi159 ceph-mon[35534]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-11T22:20:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:27 smithi172 ceph-mon[35461]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-11T22:20:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:27 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2757154336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-11T22:20:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:27 smithi172 ceph-mon[35461]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-11T22:20:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:27 smithi172 ceph-mon[35461]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-11T22:20:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:27 smithi172 ceph-mon[35461]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-11T22:20:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:27 smithi172 ceph-mon[35461]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-11T22:20:27.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 -- 172.21.15.5:0/1203392052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8074ee0 msgr2=0x7f35e810dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:27.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 --2- 172.21.15.5:0/1203392052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8074ee0 0x7f35e810dee0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009900 tx=0x7f35d802a480 comp rx=0 tx=0).stop 2024-09-11T22:20:27.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 -- 172.21.15.5:0/1203392052 shutdown_connections 2024-09-11T22:20:27.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 --2- 172.21.15.5:0/1203392052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e810e420 0x7f35e8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:27.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 --2- 172.21.15.5:0/1203392052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8074ee0 0x7f35e810dee0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:27.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 --2- 172.21.15.5:0/1203392052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e80745c0 0x7f35e80749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:27.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 -- 172.21.15.5:0/1203392052 >> 172.21.15.5:0/1203392052 conn(0x7f35e80fd8c0 msgr2=0x7f35e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:27.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 -- 172.21.15.5:0/1203392052 shutdown_connections 2024-09-11T22:20:27.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 -- 172.21.15.5:0/1203392052 wait complete. 2024-09-11T22:20:27.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 Processor -- start 2024-09-11T22:20:27.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.717+0000 7f35f041c640 1 -- start start 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35f041c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e80745c0 0x7f35e819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35f041c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8074ee0 0x7f35e819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35f041c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e810e420 0x7f35e819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35f041c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8114170 con 0x7f35e8074ee0 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35f041c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8113ff0 con 0x7f35e810e420 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35f041c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81142f0 con 0x7f35e80745c0 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ed9b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8074ee0 0x7f35e819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ed9b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8074ee0 0x7f35e819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40498/0 (socket says 172.21.15.5:40498) 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ed9b7640 1 -- 172.21.15.5:0/3532117215 learned_addr learned my addr 172.21.15.5:0/3532117215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ee9b9640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e810e420 0x7f35e819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:27.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ee1b8640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e80745c0 0x7f35e819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ed9b7640 1 -- 172.21.15.5:0/3532117215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e80745c0 msgr2=0x7f35e819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ed9b7640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e80745c0 0x7f35e819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ed9b7640 1 -- 172.21.15.5:0/3532117215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e810e420 msgr2=0x7f35e819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ed9b7640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e810e420 0x7f35e819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ed9b7640 1 -- 172.21.15.5:0/3532117215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e819f600 con 0x7f35e8074ee0 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ee1b8640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e80745c0 0x7f35e819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35ed9b7640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8074ee0 0x7f35e819ab90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f35d8006010 tx=0x7f35d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.718+0000 7f35df7fe640 1 -- 172.21.15.5:0/3532117215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d803f030 con 0x7f35e8074ee0 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.719+0000 7f35df7fe640 1 -- 172.21.15.5:0/3532117215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d8003ce0 con 0x7f35e8074ee0 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.719+0000 7f35f041c640 1 -- 172.21.15.5:0/3532117215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e819f890 con 0x7f35e8074ee0 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.719+0000 7f35df7fe640 1 -- 172.21.15.5:0/3532117215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8035070 con 0x7f35e8074ee0 2024-09-11T22:20:27.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.719+0000 7f35f041c640 1 -- 172.21.15.5:0/3532117215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a71d0 con 0x7f35e8074ee0 2024-09-11T22:20:27.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.720+0000 7f35f041c640 1 -- 172.21.15.5:0/3532117215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e80749a0 con 0x7f35e8074ee0 2024-09-11T22:20:27.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.721+0000 7f35df7fe640 1 -- 172.21.15.5:0/3532117215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f35d8042030 con 0x7f35e8074ee0 2024-09-11T22:20:27.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.721+0000 7f35df7fe640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f35d40a2b80 0x7f35d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:27.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.722+0000 7f35ee1b8640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f35d40a2b80 0x7f35d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:27.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.722+0000 7f35df7fe640 1 -- 172.21.15.5:0/3532117215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f35d80ef6b0 con 0x7f35e8074ee0 2024-09-11T22:20:27.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.722+0000 7f35ee1b8640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f35d40a2b80 0x7f35d40a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009b50 tx=0x7f35e4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:27.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.725+0000 7f35df7fe640 1 -- 172.21.15.5:0/3532117215 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35d80bc820 con 0x7f35e8074ee0 2024-09-11T22:20:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:27.938+0000 7f35f041c640 1 -- 172.21.15.5:0/3532117215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f35e819b9b0 con 0x7f35e8074ee0 2024-09-11T22:20:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.243+0000 7f35df7fe640 1 -- 172.21.15.5:0/3532117215 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f35e819b9b0 con 0x7f35e8074ee0 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 -- 172.21.15.5:0/3532117215 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f35d40a2b80 msgr2=0x7f35d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f35d40a2b80 0x7f35d40a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009b50 tx=0x7f35e4005f90 comp rx=0 tx=0).stop 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 -- 172.21.15.5:0/3532117215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8074ee0 msgr2=0x7f35e819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8074ee0 0x7f35e819ab90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f35d8006010 tx=0x7f35d80042f0 comp rx=0 tx=0).stop 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 -- 172.21.15.5:0/3532117215 shutdown_connections 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f35d40a2b80 0x7f35d40a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e810e420 0x7f35e819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8074ee0 0x7f35e819ab90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 --2- 172.21.15.5:0/3532117215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f35e80745c0 0x7f35e819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 -- 172.21.15.5:0/3532117215 >> 172.21.15.5:0/3532117215 conn(0x7f35e80fd8c0 msgr2=0x7f35e80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 -- 172.21.15.5:0/3532117215 shutdown_connections 2024-09-11T22:20:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:28.246+0000 7f35f041c640 1 -- 172.21.15.5:0/3532117215 wait complete. 2024-09-11T22:20:28.511 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:28 smithi172 ceph-mon[35461]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T22:20:28.512 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] up:active 2024-09-11T22:20:28.512 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:28 smithi172 ceph-mon[35461]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T22:20:28.512 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:28 smithi172 ceph-mon[35461]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-11T22:20:28.512 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:28 smithi172 ceph-mon[35461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-11T22:20:28.512 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:28 smithi172 ceph-mon[35461]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-11T22:20:28.512 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:28 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3532117215' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T22:20:28.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:28 smithi005 ceph-mon[30943]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T22:20:28.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] up:active 2024-09-11T22:20:28.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:28 smithi005 ceph-mon[30943]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T22:20:28.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:28 smithi005 ceph-mon[30943]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-11T22:20:28.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:28 smithi005 ceph-mon[30943]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-11T22:20:28.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:28 smithi005 ceph-mon[30943]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-11T22:20:28.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:28 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3532117215' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T22:20:28.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:28 smithi159 ceph-mon[35534]: pgmap v285: 97 pgs: 97 active+clean; 579 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-11T22:20:28.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] up:active 2024-09-11T22:20:28.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:28 smithi159 ceph-mon[35534]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-11T22:20:28.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:28 smithi159 ceph-mon[35534]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-11T22:20:28.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:28 smithi159 ceph-mon[35534]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-11T22:20:28.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:28 smithi159 ceph-mon[35534]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-11T22:20:28.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:28 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3532117215' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-11T22:20:29.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-11T22:20:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:29.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:29 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3532117215' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T22:20:29.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:29 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] up:active 2024-09-11T22:20:29.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:29 smithi005 ceph-mon[30943]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T22:20:29.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:29 smithi005 ceph-mon[30943]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-11T22:20:29.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:29 smithi005 ceph-mon[30943]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-09-11T22:20:29.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:29 smithi005 ceph-mon[30943]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-11T22:20:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:29 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3532117215' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T22:20:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:29 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] up:active 2024-09-11T22:20:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:29 smithi159 ceph-mon[35534]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T22:20:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:29 smithi159 ceph-mon[35534]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-11T22:20:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:29 smithi159 ceph-mon[35534]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-09-11T22:20:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:29 smithi159 ceph-mon[35534]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-11T22:20:29.579 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:29 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:20:29.294+0000 7f54cb2b2640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-11T22:20:29.580 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:29 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:20:29.294+0000 7f54cb2b2640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-11T22:20:29.580 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:29 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:20:29.294+0000 7f54cb2b2640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-11T22:20:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:29 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3532117215' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-11T22:20:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:29 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] up:active 2024-09-11T22:20:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:29 smithi172 ceph-mon[35461]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-11T22:20:29.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:29 smithi172 ceph-mon[35461]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-11T22:20:29.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:29 smithi172 ceph-mon[35461]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:creating} 8 up:standby 2024-09-11T22:20:29.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:29 smithi172 ceph-mon[35461]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-11T22:20:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.942+0000 7fdbbdf31640 1 -- 172.21.15.5:0/4135357830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbb8103740 msgr2=0x7fdbb8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.942+0000 7fdbbdf31640 1 --2- 172.21.15.5:0/4135357830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbb8103740 0x7fdbb8107b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009930 tx=0x7fdbac02a420 comp rx=0 tx=0).stop 2024-09-11T22:20:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.943+0000 7fdbbdf31640 1 -- 172.21.15.5:0/4135357830 shutdown_connections 2024-09-11T22:20:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.943+0000 7fdbbdf31640 1 --2- 172.21.15.5:0/4135357830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb8108290 0x7fdbb810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.943+0000 7fdbbdf31640 1 --2- 172.21.15.5:0/4135357830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbb8103740 0x7fdbb8107b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.943+0000 7fdbbdf31640 1 --2- 172.21.15.5:0/4135357830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbb8102d90 0x7fdbb8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.943+0000 7fdbbdf31640 1 -- 172.21.15.5:0/4135357830 >> 172.21.15.5:0/4135357830 conn(0x7fdbb80fc7d0 msgr2=0x7fdbb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.943+0000 7fdbbdf31640 1 -- 172.21.15.5:0/4135357830 shutdown_connections 2024-09-11T22:20:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.943+0000 7fdbbdf31640 1 -- 172.21.15.5:0/4135357830 wait complete. 2024-09-11T22:20:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.943+0000 7fdbbdf31640 1 Processor -- start 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.943+0000 7fdbbdf31640 1 -- start start 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbbdf31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbb8102d90 0x7fdbb81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbbdf31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb8103740 0x7fdbb8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbbdf31640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbb8108290 0x7fdbb819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbbdf31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8112100 con 0x7fdbb8102d90 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbbdf31640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8111f80 con 0x7fdbb8108290 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbbdf31640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8112280 con 0x7fdbb8103740 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbb8102d90 0x7fdbb81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbb8102d90 0x7fdbb81985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40514/0 (socket says 172.21.15.5:40514) 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb77fe640 1 -- 172.21.15.5:0/3235145794 learned_addr learned my addr 172.21.15.5:0/3235145794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb7fff640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbb8108290 0x7fdbb819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb77fe640 1 -- 172.21.15.5:0/3235145794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb8103740 msgr2=0x7fdbb8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb6ffd640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb8103740 0x7fdbb8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb77fe640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb8103740 0x7fdbb8198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb77fe640 1 -- 172.21.15.5:0/3235145794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbb8108290 msgr2=0x7fdbb819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb77fe640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbb8108290 0x7fdbb819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb77fe640 1 -- 172.21.15.5:0/3235145794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb819d550 con 0x7fdbb8102d90 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb7fff640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbb8108290 0x7fdbb819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb77fe640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbb8102d90 0x7fdbb81985a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c920 tx=0x7fdba800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb4ff9640 1 -- 172.21.15.5:0/3235145794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8007c70 con 0x7fdbb8102d90 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb4ff9640 1 -- 172.21.15.5:0/3235145794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba8007e10 con 0x7fdbb8102d90 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbbdf31640 1 -- 172.21.15.5:0/3235145794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb819d840 con 0x7fdbb8102d90 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.944+0000 7fdbb4ff9640 1 -- 172.21.15.5:0/3235145794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8018890 con 0x7fdbb8102d90 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.945+0000 7fdbbdf31640 1 -- 172.21.15.5:0/3235145794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb81a5120 con 0x7fdbb8102d90 2024-09-11T22:20:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.945+0000 7fdbbdf31640 1 -- 172.21.15.5:0/3235145794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb8103170 con 0x7fdbb8102d90 2024-09-11T22:20:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.947+0000 7fdbb4ff9640 1 -- 172.21.15.5:0/3235145794 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fdba8018a50 con 0x7fdbb8102d90 2024-09-11T22:20:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.947+0000 7fdbb4ff9640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdba00a2b80 0x7fdba00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.948+0000 7fdbb6ffd640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdba00a2b80 0x7fdba00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.948+0000 7fdbb4ff9640 1 -- 172.21.15.5:0/3235145794 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fdba80cf390 con 0x7fdbb8102d90 2024-09-11T22:20:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.948+0000 7fdbb6ffd640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdba00a2b80 0x7fdba00a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0096c0 tx=0x7fdbac005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:29.951+0000 7fdbb4ff9640 1 -- 172.21.15.5:0/3235145794 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdba8014030 con 0x7fdbb8102d90 2024-09-11T22:20:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.164+0000 7fdbbdf31640 1 -- 172.21.15.5:0/3235145794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fdbb810a2a0 con 0x7fdbb8102d90 2024-09-11T22:20:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.259+0000 7fdbb4ff9640 1 -- 172.21.15.5:0/3235145794 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fdba809c4d0 con 0x7fdbb8102d90 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 -- 172.21.15.5:0/3235145794 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdba00a2b80 msgr2=0x7fdba00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdba00a2b80 0x7fdba00a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdbac0096c0 tx=0x7fdbac005fd0 comp rx=0 tx=0).stop 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 -- 172.21.15.5:0/3235145794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbb8102d90 msgr2=0x7fdbb81985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbb8102d90 0x7fdbb81985a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c920 tx=0x7fdba800cde0 comp rx=0 tx=0).stop 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 -- 172.21.15.5:0/3235145794 shutdown_connections 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdba00a2b80 0x7fdba00a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbb8108290 0x7fdbb819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdbb8103740 0x7fdbb8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 --2- 172.21.15.5:0/3235145794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbb8102d90 0x7fdbb81985a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 -- 172.21.15.5:0/3235145794 >> 172.21.15.5:0/3235145794 conn(0x7fdbb80fc7d0 msgr2=0x7fdbb8106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 -- 172.21.15.5:0/3235145794 shutdown_connections 2024-09-11T22:20:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:30.262+0000 7fdbbdf31640 1 -- 172.21.15.5:0/3235145794 wait complete. 2024-09-11T22:20:30.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:30 smithi005 ceph-mon[30943]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-11T22:20:30.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:30 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] up:active 2024-09-11T22:20:30.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:30 smithi005 ceph-mon[30943]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T22:20:30.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:30 smithi005 ceph-mon[30943]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T22:20:30.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:30 smithi005 ceph-mon[30943]: Cluster is now healthy 2024-09-11T22:20:30.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:30 smithi005 ceph-mon[30943]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-09-11T22:20:30.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:30 smithi005 ceph-mon[30943]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T22:20:30.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:30 smithi005 ceph-mon[30943]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-09-11T22:20:30.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:30 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3235145794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T22:20:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:30 smithi159 ceph-mon[35534]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-11T22:20:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:30 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] up:active 2024-09-11T22:20:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:30 smithi159 ceph-mon[35534]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T22:20:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:30 smithi159 ceph-mon[35534]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T22:20:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:30 smithi159 ceph-mon[35534]: Cluster is now healthy 2024-09-11T22:20:30.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:30 smithi159 ceph-mon[35534]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-09-11T22:20:30.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:30 smithi159 ceph-mon[35534]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T22:20:30.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:30 smithi159 ceph-mon[35534]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-09-11T22:20:30.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:30 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3235145794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T22:20:30.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:30 smithi172 ceph-mon[35461]: pgmap v286: 97 pgs: 97 active+clean; 580 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-11T22:20:30.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:30 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] up:active 2024-09-11T22:20:30.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:30 smithi172 ceph-mon[35461]: daemon mds.c assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-11T22:20:30.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:30 smithi172 ceph-mon[35461]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-11T22:20:30.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:30 smithi172 ceph-mon[35461]: Cluster is now healthy 2024-09-11T22:20:30.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:30 smithi172 ceph-mon[35461]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=e=up:active} 8 up:standby 2024-09-11T22:20:30.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:30 smithi172 ceph-mon[35461]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-11T22:20:30.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:30 smithi172 ceph-mon[35461]: daemon mds.c is now active in filesystem cephfs as rank 4 2024-09-11T22:20:30.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:30 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3235145794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-11T22:20:31.071 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-11T22:20:31.072 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-11T22:20:31.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:31 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3235145794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T22:20:31.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:31 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] up:active 2024-09-11T22:20:31.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:31 smithi005 ceph-mon[30943]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T22:20:31.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:31 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:31.332 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:31.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:31 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3235145794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T22:20:31.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:31 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] up:active 2024-09-11T22:20:31.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:31 smithi159 ceph-mon[35534]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T22:20:31.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:31 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:31.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:31 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3235145794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-11T22:20:31.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:31 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] up:active 2024-09-11T22:20:31.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:31 smithi172 ceph-mon[35461]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-11T22:20:31.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:31 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:20:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.819+0000 7f8d0796e640 1 -- 172.21.15.5:0/3260759512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d001027a0 msgr2=0x7f8d001067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.819+0000 7f8d0796e640 1 --2- 172.21.15.5:0/3260759512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d001027a0 0x7f8d001067f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009900 tx=0x7f8cf002a410 comp rx=0 tx=0).stop 2024-09-11T22:20:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.819+0000 7f8d0796e640 1 -- 172.21.15.5:0/3260759512 shutdown_connections 2024-09-11T22:20:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.819+0000 7f8d0796e640 1 --2- 172.21.15.5:0/3260759512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d00106f20 0x7f8d0010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.819+0000 7f8d0796e640 1 --2- 172.21.15.5:0/3260759512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d001027a0 0x7f8d001067f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.819+0000 7f8d0796e640 1 --2- 172.21.15.5:0/3260759512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d00101df0 0x7f8d001021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.819+0000 7f8d0796e640 1 -- 172.21.15.5:0/3260759512 >> 172.21.15.5:0/3260759512 conn(0x7f8d000fd8f0 msgr2=0x7f8d000ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.819+0000 7f8d0796e640 1 -- 172.21.15.5:0/3260759512 shutdown_connections 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.819+0000 7f8d0796e640 1 -- 172.21.15.5:0/3260759512 wait complete. 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d0796e640 1 Processor -- start 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d0796e640 1 -- start start 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d0796e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d00101df0 0x7f8d000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d0796e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d001027a0 0x7f8d00074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d0796e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d00106f20 0x7f8d00074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d0796e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00111fa0 con 0x7f8d001027a0 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d0796e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00111e20 con 0x7f8d00101df0 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d0796e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8d00112120 con 0x7f8d00106f20 2024-09-11T22:20:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d04f09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d001027a0 0x7f8d00074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d04f09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d001027a0 0x7f8d00074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43948/0 (socket says 172.21.15.5:43948) 2024-09-11T22:20:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d04f09640 1 -- 172.21.15.5:0/3528010748 learned_addr learned my addr 172.21.15.5:0/3528010748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d0570a640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d00101df0 0x7f8d000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d05f0b640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d00106f20 0x7f8d00074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d04f09640 1 -- 172.21.15.5:0/3528010748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d00106f20 msgr2=0x7f8d00074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d04f09640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d00106f20 0x7f8d00074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d04f09640 1 -- 172.21.15.5:0/3528010748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d00101df0 msgr2=0x7f8d000744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d04f09640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d00101df0 0x7f8d000744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.820+0000 7f8d04f09640 1 -- 172.21.15.5:0/3528010748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d001a6990 con 0x7f8d001027a0 2024-09-11T22:20:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.821+0000 7f8d04f09640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d001027a0 0x7f8d00074a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0005ab0 tx=0x7f8cf00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.821+0000 7f8cf67fc640 1 -- 172.21.15.5:0/3528010748 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf003f030 con 0x7f8d001027a0 2024-09-11T22:20:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.821+0000 7f8d0796e640 1 -- 172.21.15.5:0/3528010748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d001a6c20 con 0x7f8d001027a0 2024-09-11T22:20:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.821+0000 7f8cf67fc640 1 -- 172.21.15.5:0/3528010748 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cf0003ce0 con 0x7f8d001027a0 2024-09-11T22:20:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.821+0000 7f8cf67fc640 1 -- 172.21.15.5:0/3528010748 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0035070 con 0x7f8d001027a0 2024-09-11T22:20:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.821+0000 7f8d0796e640 1 -- 172.21.15.5:0/3528010748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d001a7150 con 0x7f8d001027a0 2024-09-11T22:20:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.822+0000 7f8d0796e640 1 -- 172.21.15.5:0/3528010748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d001021d0 con 0x7f8d001027a0 2024-09-11T22:20:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.823+0000 7f8cf67fc640 1 -- 172.21.15.5:0/3528010748 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8cf0042030 con 0x7f8d001027a0 2024-09-11T22:20:31.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.824+0000 7f8cf67fc640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f8cec0a2b80 0x7f8cec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:31.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.824+0000 7f8d0570a640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f8cec0a2b80 0x7f8cec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:31.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.824+0000 7f8cf67fc640 1 -- 172.21.15.5:0/3528010748 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f8cf00ef4f0 con 0x7f8d001027a0 2024-09-11T22:20:31.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.824+0000 7f8d0570a640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f8cec0a2b80 0x7f8cec0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009990 tx=0x7f8cfc009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:31.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:31.828+0000 7f8cf67fc640 1 -- 172.21.15.5:0/3528010748 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cf00bc500 con 0x7f8d001027a0 2024-09-11T22:20:32.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.027+0000 7f8d0796e640 1 -- 172.21.15.5:0/3528010748 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8d0010a170 con 0x7f8cec0a2b80 2024-09-11T22:20:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.053+0000 7f8cf67fc640 1 -- 172.21.15.5:0/3528010748 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8cfc0095e0 con 0x7f8cec0a2b80 2024-09-11T22:20:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 -- 172.21.15.5:0/3528010748 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f8cec0a2b80 msgr2=0x7f8cec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f8cec0a2b80 0x7f8cec0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009990 tx=0x7f8cfc009110 comp rx=0 tx=0).stop 2024-09-11T22:20:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 -- 172.21.15.5:0/3528010748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d001027a0 msgr2=0x7f8d00074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d001027a0 0x7f8d00074a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0005ab0 tx=0x7f8cf00042f0 comp rx=0 tx=0).stop 2024-09-11T22:20:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 -- 172.21.15.5:0/3528010748 shutdown_connections 2024-09-11T22:20:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f8cec0a2b80 0x7f8cec0a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d00106f20 0x7f8d00074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d001027a0 0x7f8d00074a30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 --2- 172.21.15.5:0/3528010748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d00101df0 0x7f8d000744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 -- 172.21.15.5:0/3528010748 >> 172.21.15.5:0/3528010748 conn(0x7f8d000fd8f0 msgr2=0x7f8d000fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 -- 172.21.15.5:0/3528010748 shutdown_connections 2024-09-11T22:20:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:32.055+0000 7f8d0796e640 1 -- 172.21.15.5:0/3528010748 wait complete. 2024-09-11T22:20:32.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:32 smithi005 ceph-mon[30943]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 636 B/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-11T22:20:32.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:20:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:20:32.031+0000 7f16fabb6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:20:32.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:20:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:20:32.031+0000 7f16fabb6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:20:32.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:20:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:20:32.031+0000 7f16fabb6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:20:32.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:20:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:20:32.031+0000 7f16fabb6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:20:32.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:20:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:20:32.031+0000 7f16fabb6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:20:32.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:32 smithi159 ceph-mon[35534]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 636 B/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-11T22:20:32.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:32 smithi172 ceph-mon[35461]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 636 B/s rd, 4.3 KiB/s wr, 13 op/s 2024-09-11T22:20:32.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-11T22:20:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:33.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:33 smithi005 ceph-mon[30943]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:33.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:33 smithi159 ceph-mon[35534]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:33.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:33 smithi172 ceph-mon[35461]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.624+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/827689900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4c0105ad0 msgr2=0x7fa4c0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.624+0000 7fa4c4e4e640 1 --2- 172.21.15.5:0/827689900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4c0105ad0 0x7fa4c0105eb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0009900 tx=0x7fa4b002a410 comp rx=0 tx=0).stop 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.624+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/827689900 shutdown_connections 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.624+0000 7fa4c4e4e640 1 --2- 172.21.15.5:0/827689900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4c0101b60 0x7fa4c010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.624+0000 7fa4c4e4e640 1 --2- 172.21.15.5:0/827689900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c01011c0 0x7fa4c0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.624+0000 7fa4c4e4e640 1 --2- 172.21.15.5:0/827689900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4c0105ad0 0x7fa4c0105eb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.624+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/827689900 >> 172.21.15.5:0/827689900 conn(0x7fa4c0077210 msgr2=0x7fa4c00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.624+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/827689900 shutdown_connections 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.624+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/827689900 wait complete. 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4c4e4e640 1 Processor -- start 2024-09-11T22:20:33.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4c4e4e640 1 -- start start 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4c4e4e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4c01011c0 0x7fa4c0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4c4e4e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0101b60 0x7fa4c0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4c4e4e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4c0105ad0 0x7fa4c019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4c4e4e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0111fe0 con 0x7fa4c01011c0 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4c4e4e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0111e60 con 0x7fa4c0105ad0 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4c4e4e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c0112160 con 0x7fa4c0101b60 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4be59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4c01011c0 0x7fa4c0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4be59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4c01011c0 0x7fa4c0198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43976/0 (socket says 172.21.15.5:43976) 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4be59c640 1 -- 172.21.15.5:0/1694468166 learned_addr learned my addr 172.21.15.5:0/1694468166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.625+0000 7fa4bed9d640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4c0105ad0 0x7fa4c019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4b7fff640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0101b60 0x7fa4c0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4be59c640 1 -- 172.21.15.5:0/1694468166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0101b60 msgr2=0x7fa4c0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:33.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4be59c640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0101b60 0x7fa4c0198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4be59c640 1 -- 172.21.15.5:0/1694468166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4c0105ad0 msgr2=0x7fa4c019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4be59c640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4c0105ad0 0x7fa4c019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4be59c640 1 -- 172.21.15.5:0/1694468166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c019d4f0 con 0x7fa4c01011c0 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4b7fff640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0101b60 0x7fa4c0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4be59c640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4c01011c0 0x7fa4c0198540 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0006010 tx=0x7fa4b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4b77fe640 1 -- 172.21.15.5:0/1694468166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b003f070 con 0x7fa4c01011c0 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4b77fe640 1 -- 172.21.15.5:0/1694468166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b0031c30 con 0x7fa4c01011c0 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/1694468166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c019d780 con 0x7fa4c01011c0 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4b77fe640 1 -- 172.21.15.5:0/1694468166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0035030 con 0x7fa4c01011c0 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.626+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/1694468166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c01a50c0 con 0x7fa4c01011c0 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.627+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/1694468166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c0101620 con 0x7fa4c01011c0 2024-09-11T22:20:33.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.629+0000 7fa4b77fe640 1 -- 172.21.15.5:0/1694468166 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa4b0042070 con 0x7fa4c01011c0 2024-09-11T22:20:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.630+0000 7fa4b77fe640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa4ac0a2c90 0x7fa4ac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.630+0000 7fa4b7fff640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa4ac0a2c90 0x7fa4ac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.630+0000 7fa4b77fe640 1 -- 172.21.15.5:0/1694468166 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa4b00efc70 con 0x7fa4c01011c0 2024-09-11T22:20:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.630+0000 7fa4b7fff640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa4ac0a2c90 0x7fa4ac0a5150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8006fd0 tx=0x7fa4a8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:33.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.633+0000 7fa4b77fe640 1 -- 172.21.15.5:0/1694468166 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4b0031de0 con 0x7fa4c01011c0 2024-09-11T22:20:33.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.835+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/1694468166 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa4c0062630 con 0x7fa4ac0a2c90 2024-09-11T22:20:33.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.861+0000 7fa4b77fe640 1 -- 172.21.15.5:0/1694468166 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa4c0062630 con 0x7fa4ac0a2c90 2024-09-11T22:20:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/1694468166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa4ac0a2c90 msgr2=0x7fa4ac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa4ac0a2c90 0x7fa4ac0a5150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8006fd0 tx=0x7fa4a8008040 comp rx=0 tx=0).stop 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/1694468166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4c01011c0 msgr2=0x7fa4c0198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4c01011c0 0x7fa4c0198540 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0006010 tx=0x7fa4b0003c10 comp rx=0 tx=0).stop 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/1694468166 shutdown_connections 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa4ac0a2c90 0x7fa4ac0a5150 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4c0105ad0 0x7fa4c019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4c0101b60 0x7fa4c0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 --2- 172.21.15.5:0/1694468166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4c01011c0 0x7fa4c0198540 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/1694468166 >> 172.21.15.5:0/1694468166 conn(0x7fa4c0077210 msgr2=0x7fa4c0103ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/1694468166 shutdown_connections 2024-09-11T22:20:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:33.864+0000 7fa4c4e4e640 1 -- 172.21.15.5:0/1694468166 wait complete. 2024-09-11T22:20:34.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:34 smithi005 ceph-mon[30943]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 4.3 KiB/s wr, 14 op/s 2024-09-11T22:20:34.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:34 smithi005 ceph-mon[30943]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T22:20:34.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-11T22:20:34.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:34 smithi159 ceph-mon[35534]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 4.3 KiB/s wr, 14 op/s 2024-09-11T22:20:34.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:34 smithi159 ceph-mon[35534]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T22:20:34.579 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:20:34 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:20:34.294+0000 7f54cb2b2640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-11T22:20:34.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:34 smithi172 ceph-mon[35461]: pgmap v288: 97 pgs: 97 active+clean; 583 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 4.3 KiB/s wr, 14 op/s 2024-09-11T22:20:34.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:34 smithi172 ceph-mon[35461]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-11T22:20:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.293+0000 7f613dcf6640 1 -- 172.21.15.5:0/4008658439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6138103660 msgr2=0x7f613810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.293+0000 7f613dcf6640 1 --2- 172.21.15.5:0/4008658439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6138103660 0x7f613810bb50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f612c009900 tx=0x7f612c02a480 comp rx=0 tx=0).stop 2024-09-11T22:20:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.293+0000 7f613dcf6640 1 -- 172.21.15.5:0/4008658439 shutdown_connections 2024-09-11T22:20:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.293+0000 7f613dcf6640 1 --2- 172.21.15.5:0/4008658439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f613810c170 0x7f613810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.293+0000 7f613dcf6640 1 --2- 172.21.15.5:0/4008658439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6138103660 0x7f613810bb50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.293+0000 7f613dcf6640 1 --2- 172.21.15.5:0/4008658439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6138102d40 0x7f6138103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.293+0000 7f613dcf6640 1 -- 172.21.15.5:0/4008658439 >> 172.21.15.5:0/4008658439 conn(0x7f61380fc7a0 msgr2=0x7f61380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:35.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.293+0000 7f613dcf6640 1 -- 172.21.15.5:0/4008658439 shutdown_connections 2024-09-11T22:20:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.293+0000 7f613dcf6640 1 -- 172.21.15.5:0/4008658439 wait complete. 2024-09-11T22:20:35.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.293+0000 7f613dcf6640 1 Processor -- start 2024-09-11T22:20:35.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f613dcf6640 1 -- start start 2024-09-11T22:20:35.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f613dcf6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6138102d40 0x7f61381009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:35.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f613dcf6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103660 0x7f6138100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:35.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f613dcf6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613810c170 0x7f6138101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f613dcf6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6138112030 con 0x7f6138102d40 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f613dcf6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6138111eb0 con 0x7f613810c170 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f613dcf6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61381121b0 con 0x7f6138103660 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f61377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6138102d40 0x7f61381009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f61377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6138102d40 0x7f61381009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43988/0 (socket says 172.21.15.5:43988) 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f61377fe640 1 -- 172.21.15.5:0/2225553530 learned_addr learned my addr 172.21.15.5:0/2225553530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f6136ffd640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103660 0x7f6138100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f6137fff640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613810c170 0x7f6138101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f61377fe640 1 -- 172.21.15.5:0/2225553530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103660 msgr2=0x7f6138100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f61377fe640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103660 0x7f6138100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f61377fe640 1 -- 172.21.15.5:0/2225553530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613810c170 msgr2=0x7f6138101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f61377fe640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613810c170 0x7f6138101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:35.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f61377fe640 1 -- 172.21.15.5:0/2225553530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61381a6a60 con 0x7f6138102d40 2024-09-11T22:20:35.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f6137fff640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613810c170 0x7f6138101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:35.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.294+0000 7f61377fe640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6138102d40 0x7f61381009a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f612800b4a0 tx=0x7f612800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:35.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.295+0000 7f6134ff9640 1 -- 172.21.15.5:0/2225553530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128014070 con 0x7f6138102d40 2024-09-11T22:20:35.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.295+0000 7f613dcf6640 1 -- 172.21.15.5:0/2225553530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61381a6d50 con 0x7f6138102d40 2024-09-11T22:20:35.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.295+0000 7f6134ff9640 1 -- 172.21.15.5:0/2225553530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6128010af0 con 0x7f6138102d40 2024-09-11T22:20:35.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.295+0000 7f6134ff9640 1 -- 172.21.15.5:0/2225553530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128017d10 con 0x7f6138102d40 2024-09-11T22:20:35.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.295+0000 7f613dcf6640 1 -- 172.21.15.5:0/2225553530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61381a7250 con 0x7f6138102d40 2024-09-11T22:20:35.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.296+0000 7f613dcf6640 1 -- 172.21.15.5:0/2225553530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61381031a0 con 0x7f6138102d40 2024-09-11T22:20:35.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.297+0000 7f6134ff9640 1 -- 172.21.15.5:0/2225553530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6128017070 con 0x7f6138102d40 2024-09-11T22:20:35.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.298+0000 7f6134ff9640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f61200a2c10 0x7f61200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:35.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.298+0000 7f6136ffd640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f61200a2c10 0x7f61200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:35.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.298+0000 7f6134ff9640 1 -- 172.21.15.5:0/2225553530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f61280ce320 con 0x7f6138102d40 2024-09-11T22:20:35.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.298+0000 7f6136ffd640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f61200a2c10 0x7f61200a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6138102430 tx=0x7f612c031490 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.302+0000 7f6134ff9640 1 -- 172.21.15.5:0/2225553530 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f612809b3e0 con 0x7f6138102d40 2024-09-11T22:20:35.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.504+0000 7f613dcf6640 1 -- 172.21.15.5:0/2225553530 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f6138062630 con 0x7f61200a2c10 2024-09-11T22:20:35.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:35 smithi005 ceph-mon[30943]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:35.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.510+0000 7f6134ff9640 1 -- 172.21.15.5:0/2225553530 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6138062630 con 0x7f61200a2c10 2024-09-11T22:20:35.510 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_0/47976776-147f-45b4-97c2-0a5a827440a3 2024-09-11T22:20:35.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 -- 172.21.15.5:0/2225553530 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f61200a2c10 msgr2=0x7f61200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f61200a2c10 0x7f61200a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6138102430 tx=0x7f612c031490 comp rx=0 tx=0).stop 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 -- 172.21.15.5:0/2225553530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6138102d40 msgr2=0x7f61381009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6138102d40 0x7f61381009a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f612800b4a0 tx=0x7f612800b960 comp rx=0 tx=0).stop 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 -- 172.21.15.5:0/2225553530 shutdown_connections 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f61200a2c10 0x7f61200a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613810c170 0x7f6138101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6138103660 0x7f6138100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 --2- 172.21.15.5:0/2225553530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6138102d40 0x7f61381009a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 -- 172.21.15.5:0/2225553530 >> 172.21.15.5:0/2225553530 conn(0x7f61380fc7a0 msgr2=0x7f61380fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 -- 172.21.15.5:0/2225553530 shutdown_connections 2024-09-11T22:20:35.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:35.512+0000 7f613dcf6640 1 -- 172.21.15.5:0/2225553530 wait complete. 2024-09-11T22:20:35.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:35 smithi159 ceph-mon[35534]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:35 smithi172 ceph-mon[35461]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:36.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-11T22:20:36.468 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:36.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:36 smithi005 ceph-mon[30943]: pgmap v289: 97 pgs: 97 active+clean; 584 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-09-11T22:20:36.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:36 smithi159 ceph-mon[35534]: pgmap v289: 97 pgs: 97 active+clean; 584 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-09-11T22:20:36.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:36 smithi172 ceph-mon[35461]: pgmap v289: 97 pgs: 97 active+clean; 584 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4.8 KiB/s wr, 18 op/s 2024-09-11T22:20:36.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.969+0000 7f44b29a3640 1 -- 172.21.15.5:0/3663458314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac10e4f0 msgr2=0x7f44ac110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:36.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.969+0000 7f44b29a3640 1 --2- 172.21.15.5:0/3663458314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac10e4f0 0x7f44ac110900 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009ca0 tx=0x7f44a402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:20:36.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.969+0000 7f44b29a3640 1 -- 172.21.15.5:0/3663458314 shutdown_connections 2024-09-11T22:20:36.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.969+0000 7f44b29a3640 1 --2- 172.21.15.5:0/3663458314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac10e4f0 0x7f44ac110900 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:36.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.969+0000 7f44b29a3640 1 --2- 172.21.15.5:0/3663458314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44ac101380 0x7f44ac10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:36.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.969+0000 7f44b29a3640 1 --2- 172.21.15.5:0/3663458314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44ac1009d0 0x7f44ac100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:36.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.969+0000 7f44b29a3640 1 -- 172.21.15.5:0/3663458314 >> 172.21.15.5:0/3663458314 conn(0x7f44ac0fc7d0 msgr2=0x7f44ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:36.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.970+0000 7f44b29a3640 1 -- 172.21.15.5:0/3663458314 shutdown_connections 2024-09-11T22:20:36.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.970+0000 7f44b29a3640 1 -- 172.21.15.5:0/3663458314 wait complete. 2024-09-11T22:20:36.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.970+0000 7f44b29a3640 1 Processor -- start 2024-09-11T22:20:36.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.970+0000 7f44b29a3640 1 -- start start 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.970+0000 7f44b29a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44ac1009d0 0x7f44ac1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.970+0000 7f44b29a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac101380 0x7f44ac1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.970+0000 7f44b29a3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44ac10e4f0 0x7f44ac19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.970+0000 7f44b29a3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1142b0 con 0x7f44ac101380 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.970+0000 7f44b29a3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac114130 con 0x7f44ac10e4f0 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.970+0000 7f44b29a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac114430 con 0x7f44ac1009d0 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac101380 0x7f44ac1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac101380 0x7f44ac1a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44012/0 (socket says 172.21.15.5:44012) 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44ab7fe640 1 -- 172.21.15.5:0/926053453 learned_addr learned my addr 172.21.15.5:0/926053453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44abfff640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44ac1009d0 0x7f44ac1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44b0f40640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44ac10e4f0 0x7f44ac19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:36.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44ab7fe640 1 -- 172.21.15.5:0/926053453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44ac1009d0 msgr2=0x7f44ac1a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44ab7fe640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44ac1009d0 0x7f44ac1a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44ab7fe640 1 -- 172.21.15.5:0/926053453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44ac10e4f0 msgr2=0x7f44ac19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44ab7fe640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44ac10e4f0 0x7f44ac19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44ab7fe640 1 -- 172.21.15.5:0/926053453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac19acf0 con 0x7f44ac101380 2024-09-11T22:20:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44b0f40640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44ac10e4f0 0x7f44ac19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44ab7fe640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac101380 0x7f44ac1a0940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c890 tx=0x7f44a000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44a97fa640 1 -- 172.21.15.5:0/926053453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0004220 con 0x7f44ac101380 2024-09-11T22:20:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44a97fa640 1 -- 172.21.15.5:0/926053453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a00043c0 con 0x7f44ac101380 2024-09-11T22:20:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44b29a3640 1 -- 172.21.15.5:0/926053453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac19afe0 con 0x7f44ac101380 2024-09-11T22:20:36.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44a97fa640 1 -- 172.21.15.5:0/926053453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0004a00 con 0x7f44ac101380 2024-09-11T22:20:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.971+0000 7f44b29a3640 1 -- 172.21.15.5:0/926053453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac1a72d0 con 0x7f44ac101380 2024-09-11T22:20:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.972+0000 7f44b29a3640 1 -- 172.21.15.5:0/926053453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44ac06a7d0 con 0x7f44ac101380 2024-09-11T22:20:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.974+0000 7f44a97fa640 1 -- 172.21.15.5:0/926053453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f44a00079b0 con 0x7f44ac101380 2024-09-11T22:20:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.975+0000 7f44a97fa640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f44940a2c10 0x7f44940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.975+0000 7f44abfff640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f44940a2c10 0x7f44940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.975+0000 7f44a97fa640 1 -- 172.21.15.5:0/926053453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f44a00cecf0 con 0x7f44ac101380 2024-09-11T22:20:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.975+0000 7f44abfff640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f44940a2c10 0x7f44940a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f449c009990 tx=0x7f449c009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:36.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:36.979+0000 7f44a97fa640 1 -- 172.21.15.5:0/926053453 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f44a0014030 con 0x7f44ac101380 2024-09-11T22:20:37.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.183+0000 7f44b29a3640 1 -- 172.21.15.5:0/926053453 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f44ac062630 con 0x7f44940a2c10 2024-09-11T22:20:37.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.209+0000 7f44a97fa640 1 -- 172.21.15.5:0/926053453 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f44ac062630 con 0x7f44940a2c10 2024-09-11T22:20:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 -- 172.21.15.5:0/926053453 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f44940a2c10 msgr2=0x7f44940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f44940a2c10 0x7f44940a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f449c009990 tx=0x7f449c009110 comp rx=0 tx=0).stop 2024-09-11T22:20:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 -- 172.21.15.5:0/926053453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac101380 msgr2=0x7f44ac1a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac101380 0x7f44ac1a0940 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f44a000c890 tx=0x7f44a000cc90 comp rx=0 tx=0).stop 2024-09-11T22:20:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 -- 172.21.15.5:0/926053453 shutdown_connections 2024-09-11T22:20:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f44940a2c10 0x7f44940a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44ac10e4f0 0x7f44ac19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:37.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac101380 0x7f44ac1a0940 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 --2- 172.21.15.5:0/926053453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44ac1009d0 0x7f44ac1a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 -- 172.21.15.5:0/926053453 >> 172.21.15.5:0/926053453 conn(0x7f44ac0fc7d0 msgr2=0x7f44ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 -- 172.21.15.5:0/926053453 shutdown_connections 2024-09-11T22:20:37.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:37.211+0000 7f44b29a3640 1 -- 172.21.15.5:0/926053453 wait complete. 2024-09-11T22:20:37.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:37 smithi005 ceph-mon[30943]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:37.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:37 smithi159 ceph-mon[35534]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:37.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:37 smithi172 ceph-mon[35461]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:37.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-11T22:20:38.200 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:38.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:38 smithi005 ceph-mon[30943]: pgmap v290: 97 pgs: 97 active+clean; 605 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 KiB/s wr, 20 op/s 2024-09-11T22:20:38.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:38 smithi005 ceph-mon[30943]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:38 smithi159 ceph-mon[35534]: pgmap v290: 97 pgs: 97 active+clean; 605 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 KiB/s wr, 20 op/s 2024-09-11T22:20:38.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:38 smithi159 ceph-mon[35534]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:38 smithi172 ceph-mon[35461]: pgmap v290: 97 pgs: 97 active+clean; 605 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 6.0 KiB/s wr, 20 op/s 2024-09-11T22:20:38.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:38 smithi172 ceph-mon[35461]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.676+0000 7f0351fe6640 1 -- 172.21.15.5:0/4149534484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c102570 msgr2=0x7f034c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.676+0000 7f0351fe6640 1 --2- 172.21.15.5:0/4149534484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c102570 0x7f034c1029b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0334009900 tx=0x7f033402a480 comp rx=0 tx=0).stop 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.676+0000 7f0351fe6640 1 -- 172.21.15.5:0/4149534484 shutdown_connections 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.676+0000 7f0351fe6640 1 --2- 172.21.15.5:0/4149534484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f034c102ef0 0x7f034c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.676+0000 7f0351fe6640 1 --2- 172.21.15.5:0/4149534484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c102570 0x7f034c1029b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.676+0000 7f0351fe6640 1 --2- 172.21.15.5:0/4149534484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c100e20 0x7f034c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.676+0000 7f0351fe6640 1 -- 172.21.15.5:0/4149534484 >> 172.21.15.5:0/4149534484 conn(0x7f034c0fc7d0 msgr2=0x7f034c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.676+0000 7f0351fe6640 1 -- 172.21.15.5:0/4149534484 shutdown_connections 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.676+0000 7f0351fe6640 1 -- 172.21.15.5:0/4149534484 wait complete. 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.676+0000 7f0351fe6640 1 Processor -- start 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f0351fe6640 1 -- start start 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f0351fe6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f034c100e20 0x7f034c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f0351fe6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c102570 0x7f034c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f0351fe6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c102ef0 0x7f034c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f0351fe6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f034c110fd0 con 0x7f034c102570 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f0351fe6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f034c110e50 con 0x7f034c100e20 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f0351fe6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f034c111150 con 0x7f034c102ef0 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c102570 0x7f034c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c102570 0x7f034c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44022/0 (socket says 172.21.15.5:44022) 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034affd640 1 -- 172.21.15.5:0/972775082 learned_addr learned my addr 172.21.15.5:0/972775082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034b7fe640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f034c100e20 0x7f034c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034bfff640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c102ef0 0x7f034c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034affd640 1 -- 172.21.15.5:0/972775082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c102ef0 msgr2=0x7f034c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034affd640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c102ef0 0x7f034c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034affd640 1 -- 172.21.15.5:0/972775082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f034c100e20 msgr2=0x7f034c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034affd640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f034c100e20 0x7f034c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034affd640 1 -- 172.21.15.5:0/972775082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c1a69b0 con 0x7f034c102570 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034b7fe640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f034c100e20 0x7f034c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.677+0000 7f034bfff640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c102ef0 0x7f034c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.678+0000 7f034affd640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c102570 0x7f034c0fd1c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f033402aa70 tx=0x7f0334003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.678+0000 7f0348ff9640 1 -- 172.21.15.5:0/972775082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033403f070 con 0x7f034c102570 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.678+0000 7f0348ff9640 1 -- 172.21.15.5:0/972775082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0334031c20 con 0x7f034c102570 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.678+0000 7f0351fe6640 1 -- 172.21.15.5:0/972775082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c1a6c40 con 0x7f034c102570 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.678+0000 7f0348ff9640 1 -- 172.21.15.5:0/972775082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334035030 con 0x7f034c102570 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.678+0000 7f0351fe6640 1 -- 172.21.15.5:0/972775082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c1a7120 con 0x7f034c102570 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.679+0000 7f0351fe6640 1 -- 172.21.15.5:0/972775082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f034c101200 con 0x7f034c102570 2024-09-11T22:20:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.680+0000 7f0348ff9640 1 -- 172.21.15.5:0/972775082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0334042070 con 0x7f034c102570 2024-09-11T22:20:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.681+0000 7f0348ff9640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03380a2c10 0x7f03380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.681+0000 7f034b7fe640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03380a2c10 0x7f03380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.681+0000 7f0348ff9640 1 -- 172.21.15.5:0/972775082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f03340ef890 con 0x7f034c102570 2024-09-11T22:20:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.682+0000 7f034b7fe640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03380a2c10 0x7f03380a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f033c005fd0 tx=0x7f033c005ea0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.686+0000 7f0348ff9640 1 -- 172.21.15.5:0/972775082 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03340bc840 con 0x7f034c102570 2024-09-11T22:20:38.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.889+0000 7f0351fe6640 1 -- 172.21.15.5:0/972775082 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f034c062630 con 0x7f03380a2c10 2024-09-11T22:20:38.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.895+0000 7f0348ff9640 1 -- 172.21.15.5:0/972775082 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f034c062630 con 0x7f03380a2c10 2024-09-11T22:20:38.896 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 2024-09-11T22:20:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.897+0000 7f0351fe6640 1 -- 172.21.15.5:0/972775082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03380a2c10 msgr2=0x7f03380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.897+0000 7f0351fe6640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03380a2c10 0x7f03380a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f033c005fd0 tx=0x7f033c005ea0 comp rx=0 tx=0).stop 2024-09-11T22:20:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.897+0000 7f0351fe6640 1 -- 172.21.15.5:0/972775082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c102570 msgr2=0x7f034c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.897+0000 7f0351fe6640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c102570 0x7f034c0fd1c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f033402aa70 tx=0x7f0334003b60 comp rx=0 tx=0).stop 2024-09-11T22:20:38.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.897+0000 7f0351fe6640 1 -- 172.21.15.5:0/972775082 shutdown_connections 2024-09-11T22:20:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.897+0000 7f0351fe6640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03380a2c10 0x7f03380a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.897+0000 7f0351fe6640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f034c102ef0 0x7f034c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.897+0000 7f0351fe6640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034c102570 0x7f034c0fd1c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.897+0000 7f0351fe6640 1 --2- 172.21.15.5:0/972775082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f034c100e20 0x7f034c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.897+0000 7f0351fe6640 1 -- 172.21.15.5:0/972775082 >> 172.21.15.5:0/972775082 conn(0x7f034c0fc7d0 msgr2=0x7f034c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.898+0000 7f0351fe6640 1 -- 172.21.15.5:0/972775082 shutdown_connections 2024-09-11T22:20:38.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:38.898+0000 7f0351fe6640 1 -- 172.21.15.5:0/972775082 wait complete. 2024-09-11T22:20:39.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:20:39.984 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:40.412 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:40 smithi005 ceph-mon[30943]: pgmap v291: 97 pgs: 97 active+clean; 605 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2024-09-11T22:20:40.412 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:40 smithi005 ceph-mon[30943]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.494+0000 7fea0fc87640 1 -- 172.21.15.5:0/466249814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea08100a90 msgr2=0x7fea08100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.494+0000 7fea0fc87640 1 --2- 172.21.15.5:0/466249814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea08100a90 0x7fea08100ef0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009930 tx=0x7fe9f802a420 comp rx=0 tx=0).stop 2024-09-11T22:20:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.494+0000 7fea0fc87640 1 -- 172.21.15.5:0/466249814 shutdown_connections 2024-09-11T22:20:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.494+0000 7fea0fc87640 1 --2- 172.21.15.5:0/466249814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea08101430 0x7fea0810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.494+0000 7fea0fc87640 1 --2- 172.21.15.5:0/466249814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea08100a90 0x7fea08100ef0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.494+0000 7fea0fc87640 1 --2- 172.21.15.5:0/466249814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea08106a90 0x7fea08106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.494+0000 7fea0fc87640 1 -- 172.21.15.5:0/466249814 >> 172.21.15.5:0/466249814 conn(0x7fea080fc7d0 msgr2=0x7fea080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.494+0000 7fea0fc87640 1 -- 172.21.15.5:0/466249814 shutdown_connections 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.494+0000 7fea0fc87640 1 -- 172.21.15.5:0/466249814 wait complete. 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0fc87640 1 Processor -- start 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0fc87640 1 -- start start 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0fc87640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea08100a90 0x7fea081973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0fc87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea08101430 0x7fea08197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0fc87640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea08106a90 0x7fea0819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0fc87640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea08110eb0 con 0x7fea08101430 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0fc87640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea08110d30 con 0x7fea08106a90 2024-09-11T22:20:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0fc87640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea08111030 con 0x7fea08100a90 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0d222640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea08101430 0x7fea08197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0d222640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea08101430 0x7fea08197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44048/0 (socket says 172.21.15.5:44048) 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.495+0000 7fea0d222640 1 -- 172.21.15.5:0/3000490750 learned_addr learned my addr 172.21.15.5:0/3000490750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0e224640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea08106a90 0x7fea0819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0da23640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea08100a90 0x7fea081973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0d222640 1 -- 172.21.15.5:0/3000490750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea08100a90 msgr2=0x7fea081973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0d222640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea08100a90 0x7fea081973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0d222640 1 -- 172.21.15.5:0/3000490750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea08106a90 msgr2=0x7fea0819bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0d222640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea08106a90 0x7fea0819bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0d222640 1 -- 172.21.15.5:0/3000490750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0819c380 con 0x7fea08101430 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0e224640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea08106a90 0x7fea0819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0d222640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea08101430 0x7fea08197910 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009a50 tx=0x7fe9f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fe9feffd640 1 -- 172.21.15.5:0/3000490750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f803f030 con 0x7fea08101430 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0fc87640 1 -- 172.21.15.5:0/3000490750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0819c610 con 0x7fea08101430 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fe9feffd640 1 -- 172.21.15.5:0/3000490750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f8003ce0 con 0x7fea08101430 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fe9feffd640 1 -- 172.21.15.5:0/3000490750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8035070 con 0x7fea08101430 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.496+0000 7fea0fc87640 1 -- 172.21.15.5:0/3000490750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea081a3f50 con 0x7fea08101430 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.497+0000 7fea0fc87640 1 -- 172.21.15.5:0/3000490750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea08100ef0 con 0x7fea08101430 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.498+0000 7fe9feffd640 1 -- 172.21.15.5:0/3000490750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe9f8042030 con 0x7fea08101430 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.499+0000 7fe9feffd640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe9f40a2c10 0x7fe9f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.499+0000 7fe9feffd640 1 -- 172.21.15.5:0/3000490750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fe9f80ef690 con 0x7fea08101430 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.499+0000 7fea0da23640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe9f40a2c10 0x7fe9f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.499+0000 7fea0da23640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe9f40a2c10 0x7fe9f40a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fea040058c0 tx=0x7fea04005c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.503+0000 7fe9feffd640 1 -- 172.21.15.5:0/3000490750 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9f80bc640 con 0x7fea08101430 2024-09-11T22:20:40.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:40 smithi159 ceph-mon[35534]: pgmap v291: 97 pgs: 97 active+clean; 605 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2024-09-11T22:20:40.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:40 smithi159 ceph-mon[35534]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:40 smithi172 ceph-mon[35461]: pgmap v291: 97 pgs: 97 active+clean; 605 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.1 KiB/s wr, 21 op/s 2024-09-11T22:20:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:40 smithi172 ceph-mon[35461]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:20:40.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.719+0000 7fea0fc87640 1 -- 172.21.15.5:0/3000490750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea08109050 con 0x7fea08101430 2024-09-11T22:20:40.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.719+0000 7fe9feffd640 1 -- 172.21.15.5:0/3000490750 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7fe9f80bbf40 con 0x7fea08101430 2024-09-11T22:20:40.720 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:40.720 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-11T22:20:30:263672+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:20:30.263652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:20:40.721 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-11T22:20:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 -- 172.21.15.5:0/3000490750 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe9f40a2c10 msgr2=0x7fe9f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe9f40a2c10 0x7fe9f40a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fea040058c0 tx=0x7fea04005c60 comp rx=0 tx=0).stop 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 -- 172.21.15.5:0/3000490750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea08101430 msgr2=0x7fea08197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea08101430 0x7fea08197910 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009a50 tx=0x7fe9f8004270 comp rx=0 tx=0).stop 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 -- 172.21.15.5:0/3000490750 shutdown_connections 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe9f40a2c10 0x7fe9f40a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea08106a90 0x7fea0819bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea08101430 0x7fea08197910 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 --2- 172.21.15.5:0/3000490750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea08100a90 0x7fea081973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 -- 172.21.15.5:0/3000490750 >> 172.21.15.5:0/3000490750 conn(0x7fea080fc7d0 msgr2=0x7fea080fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 -- 172.21.15.5:0/3000490750 shutdown_connections 2024-09-11T22:20:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:40.722+0000 7fea0fc87640 1 -- 172.21.15.5:0/3000490750 wait complete. 2024-09-11T22:20:41.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:41 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3000490750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:41.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:20:41.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:41 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3000490750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:41.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:41 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3000490750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:20:41.779 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.256+0000 7f18c1446640 1 -- 172.21.15.5:0/4188910712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18bc106f10 msgr2=0x7f18bc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.256+0000 7f18c1446640 1 --2- 172.21.15.5:0/4188910712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18bc106f10 0x7f18bc10ea90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009ca0 tx=0x7f18b402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.256+0000 7f18c1446640 1 -- 172.21.15.5:0/4188910712 shutdown_connections 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.256+0000 7f18c1446640 1 --2- 172.21.15.5:0/4188910712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18bc106f10 0x7f18bc10ea90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.256+0000 7f18c1446640 1 --2- 172.21.15.5:0/4188910712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18bc102790 0x7f18bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.256+0000 7f18c1446640 1 --2- 172.21.15.5:0/4188910712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18bc101de0 0x7f18bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.256+0000 7f18c1446640 1 -- 172.21.15.5:0/4188910712 >> 172.21.15.5:0/4188910712 conn(0x7f18bc0fd8c0 msgr2=0x7f18bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.256+0000 7f18c1446640 1 -- 172.21.15.5:0/4188910712 shutdown_connections 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.256+0000 7f18c1446640 1 -- 172.21.15.5:0/4188910712 wait complete. 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18c1446640 1 Processor -- start 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18c1446640 1 -- start start 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18c1446640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18bc101de0 0x7f18bc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18c1446640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18bc102790 0x7f18bc19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18c1446640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18bc106f10 0x7f18bc19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18c1446640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc111f70 con 0x7f18bc106f10 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18c1446640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc111df0 con 0x7f18bc102790 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18c1446640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1120f0 con 0x7f18bc101de0 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18bc106f10 0x7f18bc19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18bc106f10 0x7f18bc19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42926/0 (socket says 172.21.15.5:42926) 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.257+0000 7f18bb7fe640 1 -- 172.21.15.5:0/257101416 learned_addr learned my addr 172.21.15.5:0/257101416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18b3fff640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18bc102790 0x7f18bc19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18baffd640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18bc101de0 0x7f18bc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18bb7fe640 1 -- 172.21.15.5:0/257101416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18bc101de0 msgr2=0x7f18bc19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18bb7fe640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18bc101de0 0x7f18bc19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18bb7fe640 1 -- 172.21.15.5:0/257101416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18bc102790 msgr2=0x7f18bc19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18bb7fe640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18bc102790 0x7f18bc19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18bb7fe640 1 -- 172.21.15.5:0/257101416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc19f570 con 0x7f18bc106f10 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18baffd640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18bc101de0 0x7f18bc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18bb7fe640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18bc106f10 0x7f18bc19ee90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f18b4031a20 tx=0x7f18b4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18b8ff9640 1 -- 172.21.15.5:0/257101416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4003c30 con 0x7f18bc106f10 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18c1446640 1 -- 172.21.15.5:0/257101416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc19f800 con 0x7f18bc106f10 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18b8ff9640 1 -- 172.21.15.5:0/257101416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b4003dd0 con 0x7f18bc106f10 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18b8ff9640 1 -- 172.21.15.5:0/257101416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b402fcf0 con 0x7f18bc106f10 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.258+0000 7f18c1446640 1 -- 172.21.15.5:0/257101416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc1a7140 con 0x7f18bc106f10 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.259+0000 7f18c1446640 1 -- 172.21.15.5:0/257101416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18bc1021c0 con 0x7f18bc106f10 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.260+0000 7f18b8ff9640 1 -- 172.21.15.5:0/257101416 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f18b4004050 con 0x7f18bc106f10 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.261+0000 7f18b8ff9640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18a80a2c10 0x7f18a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.261+0000 7f18baffd640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18a80a2c10 0x7f18a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.261+0000 7f18b8ff9640 1 -- 172.21.15.5:0/257101416 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f18b40f0260 con 0x7f18bc106f10 2024-09-11T22:20:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.261+0000 7f18baffd640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18a80a2c10 0x7f18a80a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0058c0 tx=0x7f18ac005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.265+0000 7f18b8ff9640 1 -- 172.21.15.5:0/257101416 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18b40bd3b0 con 0x7f18bc106f10 2024-09-11T22:20:42.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.443+0000 7f18c1446640 1 -- 172.21.15.5:0/257101416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18bc1a7370 con 0x7f18bc106f10 2024-09-11T22:20:42.444 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:42 smithi005 ceph-mon[30943]: pgmap v292: 97 pgs: 97 active+clean; 629 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 23 op/s 2024-09-11T22:20:42.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.444+0000 7f18b8ff9640 1 -- 172.21.15.5:0/257101416 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22173 (secure 0 0 0) 0x7f18b402fe90 con 0x7f18bc106f10 2024-09-11T22:20:42.445 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:42.445 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":80,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:20:18.900590+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:20:42.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 -- 172.21.15.5:0/257101416 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18a80a2c10 msgr2=0x7f18a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:42.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18a80a2c10 0x7f18a80a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0058c0 tx=0x7f18ac005850 comp rx=0 tx=0).stop 2024-09-11T22:20:42.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 -- 172.21.15.5:0/257101416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18bc106f10 msgr2=0x7f18bc19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18bc106f10 0x7f18bc19ee90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f18b4031a20 tx=0x7f18b4031a50 comp rx=0 tx=0).stop 2024-09-11T22:20:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 -- 172.21.15.5:0/257101416 shutdown_connections 2024-09-11T22:20:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f18a80a2c10 0x7f18a80a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18bc106f10 0x7f18bc19ee90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18bc102790 0x7f18bc19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 --2- 172.21.15.5:0/257101416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18bc101de0 0x7f18bc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 -- 172.21.15.5:0/257101416 >> 172.21.15.5:0/257101416 conn(0x7f18bc0fd8c0 msgr2=0x7f18bc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 -- 172.21.15.5:0/257101416 shutdown_connections 2024-09-11T22:20:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:42.447+0000 7f18c1446640 1 -- 172.21.15.5:0/257101416 wait complete. 2024-09-11T22:20:42.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:42 smithi159 ceph-mon[35534]: pgmap v292: 97 pgs: 97 active+clean; 629 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 23 op/s 2024-09-11T22:20:42.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:42 smithi172 ceph-mon[35461]: pgmap v292: 97 pgs: 97 active+clean; 629 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.6 KiB/s wr, 23 op/s 2024-09-11T22:20:43.251 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T22:20:43.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:20:43.513 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:43.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:43 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/257101416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:43.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:43 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/257101416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:43.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:43 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/257101416' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 -- 172.21.15.5:0/349895745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fe41036e0 msgr2=0x7f9fe4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 --2- 172.21.15.5:0/349895745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fe41036e0 0x7f9fe4107b90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009930 tx=0x7f9fd002a420 comp rx=0 tx=0).stop 2024-09-11T22:20:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 -- 172.21.15.5:0/349895745 shutdown_connections 2024-09-11T22:20:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 --2- 172.21.15.5:0/349895745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe41082a0 0x7f9fe410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 --2- 172.21.15.5:0/349895745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fe41036e0 0x7f9fe4107b90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 --2- 172.21.15.5:0/349895745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4102d30 0x7f9fe4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 -- 172.21.15.5:0/349895745 >> 172.21.15.5:0/349895745 conn(0x7f9fe40fc7d0 msgr2=0x7f9fe40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 -- 172.21.15.5:0/349895745 shutdown_connections 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 -- 172.21.15.5:0/349895745 wait complete. 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 Processor -- start 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.026+0000 7f9feb0d7640 1 -- start start 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9feb0d7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4102d30 0x7f9fe4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9feb0d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fe41036e0 0x7f9fe4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9feb0d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe41082a0 0x7f9fe4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9feb0d7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe41120a0 con 0x7f9fe41036e0 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9feb0d7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe4111f20 con 0x7f9fe4102d30 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9feb0d7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe4112220 con 0x7f9fe41082a0 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fe41036e0 0x7f9fe4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:44.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fe41036e0 0x7f9fe4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42944/0 (socket says 172.21.15.5:42944) 2024-09-11T22:20:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe3fff640 1 -- 172.21.15.5:0/719295982 learned_addr learned my addr 172.21.15.5:0/719295982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe8e73640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4102d30 0x7f9fe4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe9674640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe41082a0 0x7f9fe4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe3fff640 1 -- 172.21.15.5:0/719295982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe41082a0 msgr2=0x7f9fe4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe3fff640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe41082a0 0x7f9fe4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe3fff640 1 -- 172.21.15.5:0/719295982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4102d30 msgr2=0x7f9fe4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe3fff640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4102d30 0x7f9fe4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe3fff640 1 -- 172.21.15.5:0/719295982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe41a6a10 con 0x7f9fe41036e0 2024-09-11T22:20:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe3fff640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fe41036e0 0x7f9fe4100ed0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0005940 tx=0x7f9fd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:44.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe1ffb640 1 -- 172.21.15.5:0/719295982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd003f030 con 0x7f9fe41036e0 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe1ffb640 1 -- 172.21.15.5:0/719295982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fd0003ce0 con 0x7f9fe41036e0 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9feb0d7640 1 -- 172.21.15.5:0/719295982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe41a6ca0 con 0x7f9fe41036e0 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.027+0000 7f9fe1ffb640 1 -- 172.21.15.5:0/719295982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0035070 con 0x7f9fe41036e0 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.028+0000 7f9feb0d7640 1 -- 172.21.15.5:0/719295982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe41a7180 con 0x7f9fe41036e0 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.029+0000 7f9feb0d7640 1 -- 172.21.15.5:0/719295982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe4103110 con 0x7f9fe41036e0 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.030+0000 7f9fe1ffb640 1 -- 172.21.15.5:0/719295982 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9fd0042030 con 0x7f9fe41036e0 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.030+0000 7f9fe1ffb640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9fd40a2c90 0x7f9fd40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.031+0000 7f9fe1ffb640 1 -- 172.21.15.5:0/719295982 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f9fd00ef400 con 0x7f9fe41036e0 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.031+0000 7f9fe8e73640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9fd40a2c90 0x7f9fd40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.031+0000 7f9fe8e73640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9fd40a2c90 0x7f9fd40a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800bbe0 tx=0x7f9fd800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:44.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.035+0000 7f9fe1ffb640 1 -- 172.21.15.5:0/719295982 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fd00bc3b0 con 0x7f9fe41036e0 2024-09-11T22:20:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.206+0000 7f9feb0d7640 1 -- 172.21.15.5:0/719295982 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9fe4062630 con 0x7f9fd40a2c90 2024-09-11T22:20:44.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.213+0000 7f9fe1ffb640 1 -- 172.21.15.5:0/719295982 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258891 (secure 0 0 0) 0x7f9fe4062630 con 0x7f9fd40a2c90 2024-09-11T22:20:44.217 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:44.218 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-11T22:20:44.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.216+0000 7f9feb0d7640 1 -- 172.21.15.5:0/719295982 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9fd40a2c90 msgr2=0x7f9fd40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.216+0000 7f9feb0d7640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9fd40a2c90 0x7f9fd40a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800bbe0 tx=0x7f9fd800b390 comp rx=0 tx=0).stop 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.216+0000 7f9feb0d7640 1 -- 172.21.15.5:0/719295982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fe41036e0 msgr2=0x7f9fe4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.216+0000 7f9feb0d7640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fe41036e0 0x7f9fe4100ed0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0005940 tx=0x7f9fd00042f0 comp rx=0 tx=0).stop 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.216+0000 7f9feb0d7640 1 -- 172.21.15.5:0/719295982 shutdown_connections 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.216+0000 7f9feb0d7640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f9fd40a2c90 0x7f9fd40a5150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.216+0000 7f9feb0d7640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9fe41082a0 0x7f9fe4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.216+0000 7f9feb0d7640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fe41036e0 0x7f9fe4100ed0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.216+0000 7f9feb0d7640 1 --2- 172.21.15.5:0/719295982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fe4102d30 0x7f9fe4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.216+0000 7f9feb0d7640 1 -- 172.21.15.5:0/719295982 >> 172.21.15.5:0/719295982 conn(0x7f9fe40fc7d0 msgr2=0x7f9fe4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.217+0000 7f9feb0d7640 1 -- 172.21.15.5:0/719295982 shutdown_connections 2024-09-11T22:20:44.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:44.217+0000 7f9feb0d7640 1 -- 172.21.15.5:0/719295982 wait complete. 2024-09-11T22:20:44.514 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:44 smithi005 ceph-mon[30943]: pgmap v293: 97 pgs: 97 active+clean; 629 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-11T22:20:44.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:44 smithi159 ceph-mon[35534]: pgmap v293: 97 pgs: 97 active+clean; 629 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-11T22:20:44.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:44 smithi172 ceph-mon[35461]: pgmap v293: 97 pgs: 97 active+clean; 629 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-11T22:20:45.062 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-11T22:20:42.443275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644219,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":275,"num_read_kb":249,"num_write":409,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":336,"ondisk_log_size":336,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376624,"kb_used_data":10736,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565200,"statfs":{"total":1151940427776,"available":1151554764800,"internally_reserved":0,"allocated":10993664,"data_stored":6898342,"data_compressed":108858,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47901,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":131,"num_read_kb":123,"num_write":80,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002580"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195820+0000","last_change":"2024-09-11T22:20:18.189744+0000","last_active":"2024-09-11T22:20:19.195820+0000","last_peered":"2024-09-11T22:20:19.195820+0000","last_clean":"2024-09-11T22:20:19.195820+0000","last_became_active":"2024-09-11T22:20:18.189398+0000","last_became_peered":"2024-09-11T22:20:18.189398+0000","last_unstale":"2024-09-11T22:20:19.195820+0000","last_undegraded":"2024-09-11T22:20:19.195820+0000","last_fullsized":"2024-09-11T22:20:19.195820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:24.152589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189094+0000","last_change":"2024-09-11T22:20:18.186727+0000","last_active":"2024-09-11T22:20:19.189094+0000","last_peered":"2024-09-11T22:20:19.189094+0000","last_clean":"2024-09-11T22:20:19.189094+0000","last_became_active":"2024-09-11T22:20:18.186414+0000","last_became_peered":"2024-09-11T22:20:18.186414+0000","last_unstale":"2024-09-11T22:20:19.189094+0000","last_undegraded":"2024-09-11T22:20:19.189094+0000","last_fullsized":"2024-09-11T22:20:19.189094+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:53.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187434+0000","last_change":"2024-09-11T22:20:18.179984+0000","last_active":"2024-09-11T22:20:19.187434+0000","last_peered":"2024-09-11T22:20:19.187434+0000","last_clean":"2024-09-11T22:20:19.187434+0000","last_became_active":"2024-09-11T22:20:18.179329+0000","last_became_peered":"2024-09-11T22:20:18.179329+0000","last_unstale":"2024-09-11T22:20:19.187434+0000","last_undegraded":"2024-09-11T22:20:19.187434+0000","last_fullsized":"2024-09-11T22:20:19.187434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:19.149970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192384+0000","last_change":"2024-09-11T22:20:18.191768+0000","last_active":"2024-09-11T22:20:19.192384+0000","last_peered":"2024-09-11T22:20:19.192384+0000","last_clean":"2024-09-11T22:20:19.192384+0000","last_became_active":"2024-09-11T22:20:18.191336+0000","last_became_peered":"2024-09-11T22:20:18.191336+0000","last_unstale":"2024-09-11T22:20:19.192384+0000","last_undegraded":"2024-09-11T22:20:19.192384+0000","last_fullsized":"2024-09-11T22:20:19.192384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:44.768932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188166+0000","last_change":"2024-09-11T22:20:18.181469+0000","last_active":"2024-09-11T22:20:19.188166+0000","last_peered":"2024-09-11T22:20:19.188166+0000","last_clean":"2024-09-11T22:20:19.188166+0000","last_became_active":"2024-09-11T22:20:18.181100+0000","last_became_peered":"2024-09-11T22:20:18.181100+0000","last_unstale":"2024-09-11T22:20:19.188166+0000","last_undegraded":"2024-09-11T22:20:19.188166+0000","last_fullsized":"2024-09-11T22:20:19.188166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:14.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180780+0000","last_change":"2024-09-11T22:20:18.190855+0000","last_active":"2024-09-11T22:20:19.180780+0000","last_peered":"2024-09-11T22:20:19.180780+0000","last_clean":"2024-09-11T22:20:19.180780+0000","last_became_active":"2024-09-11T22:20:18.190523+0000","last_became_peered":"2024-09-11T22:20:18.190523+0000","last_unstale":"2024-09-11T22:20:19.180780+0000","last_undegraded":"2024-09-11T22:20:19.180780+0000","last_fullsized":"2024-09-11T22:20:19.180780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:18.574957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189508+0000","last_change":"2024-09-11T22:20:18.199213+0000","last_active":"2024-09-11T22:20:19.189508+0000","last_peered":"2024-09-11T22:20:19.189508+0000","last_clean":"2024-09-11T22:20:19.189508+0000","last_became_active":"2024-09-11T22:20:18.198913+0000","last_became_peered":"2024-09-11T22:20:18.198913+0000","last_unstale":"2024-09-11T22:20:19.189508+0000","last_undegraded":"2024-09-11T22:20:19.189508+0000","last_fullsized":"2024-09-11T22:20:19.189508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:22.682802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.179981+0000","last_change":"2024-09-11T22:20:18.192458+0000","last_active":"2024-09-11T22:20:19.179981+0000","last_peered":"2024-09-11T22:20:19.179981+0000","last_clean":"2024-09-11T22:20:19.179981+0000","last_became_active":"2024-09-11T22:20:18.191967+0000","last_became_peered":"2024-09-11T22:20:18.191967+0000","last_unstale":"2024-09-11T22:20:19.179981+0000","last_undegraded":"2024-09-11T22:20:19.179981+0000","last_fullsized":"2024-09-11T22:20:19.179981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:52.686157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188709+0000","last_change":"2024-09-11T22:20:18.193537+0000","last_active":"2024-09-11T22:20:19.188709+0000","last_peered":"2024-09-11T22:20:19.188709+0000","last_clean":"2024-09-11T22:20:19.188709+0000","last_became_active":"2024-09-11T22:20:18.192497+0000","last_became_peered":"2024-09-11T22:20:18.192497+0000","last_unstale":"2024-09-11T22:20:19.188709+0000","last_undegraded":"2024-09-11T22:20:19.188709+0000","last_fullsized":"2024-09-11T22:20:19.188709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:13.899546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.181240+0000","last_change":"2024-09-11T22:20:18.193771+0000","last_active":"2024-09-11T22:20:19.181240+0000","last_peered":"2024-09-11T22:20:19.181240+0000","last_clean":"2024-09-11T22:20:19.181240+0000","last_became_active":"2024-09-11T22:20:18.193041+0000","last_became_peered":"2024-09-11T22:20:18.193041+0000","last_unstale":"2024-09-11T22:20:19.181240+0000","last_undegraded":"2024-09-11T22:20:19.181240+0000","last_fullsized":"2024-09-11T22:20:19.181240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:06.656169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188851+0000","last_change":"2024-09-11T22:20:18.197933+0000","last_active":"2024-09-11T22:20:19.188851+0000","last_peered":"2024-09-11T22:20:19.188851+0000","last_clean":"2024-09-11T22:20:19.188851+0000","last_became_active":"2024-09-11T22:20:18.197210+0000","last_became_peered":"2024-09-11T22:20:18.197210+0000","last_unstale":"2024-09-11T22:20:19.188851+0000","last_undegraded":"2024-09-11T22:20:19.188851+0000","last_fullsized":"2024-09-11T22:20:19.188851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:42.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192490+0000","last_change":"2024-09-11T22:20:18.190739+0000","last_active":"2024-09-11T22:20:19.192490+0000","last_peered":"2024-09-11T22:20:19.192490+0000","last_clean":"2024-09-11T22:20:19.192490+0000","last_became_active":"2024-09-11T22:20:18.190122+0000","last_became_peered":"2024-09-11T22:20:18.190122+0000","last_unstale":"2024-09-11T22:20:19.192490+0000","last_undegraded":"2024-09-11T22:20:19.192490+0000","last_fullsized":"2024-09-11T22:20:19.192490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.585035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188362+0000","last_change":"2024-09-11T22:20:18.190303+0000","last_active":"2024-09-11T22:20:19.188362+0000","last_peered":"2024-09-11T22:20:19.188362+0000","last_clean":"2024-09-11T22:20:19.188362+0000","last_became_active":"2024-09-11T22:20:18.189962+0000","last_became_peered":"2024-09-11T22:20:18.189962+0000","last_unstale":"2024-09-11T22:20:19.188362+0000","last_undegraded":"2024-09-11T22:20:19.188362+0000","last_fullsized":"2024-09-11T22:20:19.188362+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:44.872813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187420+0000","last_change":"2024-09-11T22:20:18.185893+0000","last_active":"2024-09-11T22:20:19.187420+0000","last_peered":"2024-09-11T22:20:19.187420+0000","last_clean":"2024-09-11T22:20:19.187420+0000","last_became_active":"2024-09-11T22:20:18.185592+0000","last_became_peered":"2024-09-11T22:20:18.185592+0000","last_unstale":"2024-09-11T22:20:19.187420+0000","last_undegraded":"2024-09-11T22:20:19.187420+0000","last_fullsized":"2024-09-11T22:20:19.187420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:14.725301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187741+0000","last_change":"2024-09-11T22:20:18.193943+0000","last_active":"2024-09-11T22:20:19.187741+0000","last_peered":"2024-09-11T22:20:19.187741+0000","last_clean":"2024-09-11T22:20:19.187741+0000","last_became_active":"2024-09-11T22:20:18.193558+0000","last_became_peered":"2024-09-11T22:20:18.193558+0000","last_unstale":"2024-09-11T22:20:19.187741+0000","last_undegraded":"2024-09-11T22:20:19.187741+0000","last_fullsized":"2024-09-11T22:20:19.187741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:08.736161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187152+0000","last_change":"2024-09-11T22:20:18.191815+0000","last_active":"2024-09-11T22:20:19.187152+0000","last_peered":"2024-09-11T22:20:19.187152+0000","last_clean":"2024-09-11T22:20:19.187152+0000","last_became_active":"2024-09-11T22:20:18.191483+0000","last_became_peered":"2024-09-11T22:20:18.191483+0000","last_unstale":"2024-09-11T22:20:19.187152+0000","last_undegraded":"2024-09-11T22:20:19.187152+0000","last_fullsized":"2024-09-11T22:20:19.187152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:26.432117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195936+0000","last_change":"2024-09-11T22:20:18.190758+0000","last_active":"2024-09-11T22:20:19.195936+0000","last_peered":"2024-09-11T22:20:19.195936+0000","last_clean":"2024-09-11T22:20:19.195936+0000","last_became_active":"2024-09-11T22:20:18.190431+0000","last_became_peered":"2024-09-11T22:20:18.190431+0000","last_unstale":"2024-09-11T22:20:19.195936+0000","last_undegraded":"2024-09-11T22:20:19.195936+0000","last_fullsized":"2024-09-11T22:20:19.195936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:10.927051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192252+0000","last_change":"2024-09-11T22:20:18.185191+0000","last_active":"2024-09-11T22:20:19.192252+0000","last_peered":"2024-09-11T22:20:19.192252+0000","last_clean":"2024-09-11T22:20:19.192252+0000","last_became_active":"2024-09-11T22:20:18.184906+0000","last_became_peered":"2024-09-11T22:20:18.184906+0000","last_unstale":"2024-09-11T22:20:19.192252+0000","last_undegraded":"2024-09-11T22:20:19.192252+0000","last_fullsized":"2024-09-11T22:20:19.192252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:32.294780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192490+0000","last_change":"2024-09-11T22:20:18.187137+0000","last_active":"2024-09-11T22:20:19.192490+0000","last_peered":"2024-09-11T22:20:19.192490+0000","last_clean":"2024-09-11T22:20:19.192490+0000","last_became_active":"2024-09-11T22:20:18.186770+0000","last_became_peered":"2024-09-11T22:20:18.186770+0000","last_unstale":"2024-09-11T22:20:19.192490+0000","last_undegraded":"2024-09-11T22:20:19.192490+0000","last_fullsized":"2024-09-11T22:20:19.192490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:59.917695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195850+0000","last_change":"2024-09-11T22:20:18.197137+0000","last_active":"2024-09-11T22:20:19.195850+0000","last_peered":"2024-09-11T22:20:19.195850+0000","last_clean":"2024-09-11T22:20:19.195850+0000","last_became_active":"2024-09-11T22:20:18.192810+0000","last_became_peered":"2024-09-11T22:20:18.192810+0000","last_unstale":"2024-09-11T22:20:19.195850+0000","last_undegraded":"2024-09-11T22:20:19.195850+0000","last_fullsized":"2024-09-11T22:20:19.195850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:57.680726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188422+0000","last_change":"2024-09-11T22:20:18.187568+0000","last_active":"2024-09-11T22:20:19.188422+0000","last_peered":"2024-09-11T22:20:19.188422+0000","last_clean":"2024-09-11T22:20:19.188422+0000","last_became_active":"2024-09-11T22:20:18.187254+0000","last_became_peered":"2024-09-11T22:20:18.187254+0000","last_unstale":"2024-09-11T22:20:19.188422+0000","last_undegraded":"2024-09-11T22:20:19.188422+0000","last_fullsized":"2024-09-11T22:20:19.188422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:41.724926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187455+0000","last_change":"2024-09-11T22:20:18.190376+0000","last_active":"2024-09-11T22:20:19.187455+0000","last_peered":"2024-09-11T22:20:19.187455+0000","last_clean":"2024-09-11T22:20:19.187455+0000","last_became_active":"2024-09-11T22:20:18.190079+0000","last_became_peered":"2024-09-11T22:20:18.190079+0000","last_unstale":"2024-09-11T22:20:19.187455+0000","last_undegraded":"2024-09-11T22:20:19.187455+0000","last_fullsized":"2024-09-11T22:20:19.187455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:42.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180019+0000","last_change":"2024-09-11T22:20:18.188503+0000","last_active":"2024-09-11T22:20:19.180019+0000","last_peered":"2024-09-11T22:20:19.180019+0000","last_clean":"2024-09-11T22:20:19.180019+0000","last_became_active":"2024-09-11T22:20:18.187957+0000","last_became_peered":"2024-09-11T22:20:18.187957+0000","last_unstale":"2024-09-11T22:20:19.180019+0000","last_undegraded":"2024-09-11T22:20:19.180019+0000","last_fullsized":"2024-09-11T22:20:19.180019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:44:25.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.182690+0000","last_change":"2024-09-11T22:20:18.196936+0000","last_active":"2024-09-11T22:20:19.182690+0000","last_peered":"2024-09-11T22:20:19.182690+0000","last_clean":"2024-09-11T22:20:19.182690+0000","last_became_active":"2024-09-11T22:20:18.196493+0000","last_became_peered":"2024-09-11T22:20:18.196493+0000","last_unstale":"2024-09-11T22:20:19.182690+0000","last_undegraded":"2024-09-11T22:20:19.182690+0000","last_fullsized":"2024-09-11T22:20:19.182690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:06.189806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180237+0000","last_change":"2024-09-11T22:20:18.195260+0000","last_active":"2024-09-11T22:20:19.180237+0000","last_peered":"2024-09-11T22:20:19.180237+0000","last_clean":"2024-09-11T22:20:19.180237+0000","last_became_active":"2024-09-11T22:20:18.194956+0000","last_became_peered":"2024-09-11T22:20:18.194956+0000","last_unstale":"2024-09-11T22:20:19.180237+0000","last_undegraded":"2024-09-11T22:20:19.180237+0000","last_fullsized":"2024-09-11T22:20:19.180237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:34.483291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192585+0000","last_change":"2024-09-11T22:20:18.190591+0000","last_active":"2024-09-11T22:20:19.192585+0000","last_peered":"2024-09-11T22:20:19.192585+0000","last_clean":"2024-09-11T22:20:19.192585+0000","last_became_active":"2024-09-11T22:20:18.190007+0000","last_became_peered":"2024-09-11T22:20:18.190007+0000","last_unstale":"2024-09-11T22:20:19.192585+0000","last_undegraded":"2024-09-11T22:20:19.192585+0000","last_fullsized":"2024-09-11T22:20:19.192585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:46.281705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195660+0000","last_change":"2024-09-11T22:20:18.193377+0000","last_active":"2024-09-11T22:20:19.195660+0000","last_peered":"2024-09-11T22:20:19.195660+0000","last_clean":"2024-09-11T22:20:19.195660+0000","last_became_active":"2024-09-11T22:20:18.193003+0000","last_became_peered":"2024-09-11T22:20:18.193003+0000","last_unstale":"2024-09-11T22:20:19.195660+0000","last_undegraded":"2024-09-11T22:20:19.195660+0000","last_fullsized":"2024-09-11T22:20:19.195660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:59.052966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189744+0000","last_change":"2024-09-11T22:20:18.189405+0000","last_active":"2024-09-11T22:20:19.189744+0000","last_peered":"2024-09-11T22:20:19.189744+0000","last_clean":"2024-09-11T22:20:19.189744+0000","last_became_active":"2024-09-11T22:20:18.189098+0000","last_became_peered":"2024-09-11T22:20:18.189098+0000","last_unstale":"2024-09-11T22:20:19.189744+0000","last_undegraded":"2024-09-11T22:20:19.189744+0000","last_fullsized":"2024-09-11T22:20:19.189744+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:47.088192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189019+0000","last_change":"2024-09-11T22:20:18.191005+0000","last_active":"2024-09-11T22:20:19.189019+0000","last_peered":"2024-09-11T22:20:19.189019+0000","last_clean":"2024-09-11T22:20:19.189019+0000","last_became_active":"2024-09-11T22:20:18.190681+0000","last_became_peered":"2024-09-11T22:20:18.190681+0000","last_unstale":"2024-09-11T22:20:19.189019+0000","last_undegraded":"2024-09-11T22:20:19.189019+0000","last_fullsized":"2024-09-11T22:20:19.189019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:12.903929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192034+0000","last_change":"2024-09-11T22:20:18.187920+0000","last_active":"2024-09-11T22:20:19.192034+0000","last_peered":"2024-09-11T22:20:19.192034+0000","last_clean":"2024-09-11T22:20:19.192034+0000","last_became_active":"2024-09-11T22:20:18.187526+0000","last_became_peered":"2024-09-11T22:20:18.187526+0000","last_unstale":"2024-09-11T22:20:19.192034+0000","last_undegraded":"2024-09-11T22:20:19.192034+0000","last_fullsized":"2024-09-11T22:20:19.192034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.347857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192242+0000","last_change":"2024-09-11T22:20:18.193869+0000","last_active":"2024-09-11T22:20:19.192242+0000","last_peered":"2024-09-11T22:20:19.192242+0000","last_clean":"2024-09-11T22:20:19.192242+0000","last_became_active":"2024-09-11T22:20:18.193468+0000","last_became_peered":"2024-09-11T22:20:18.193468+0000","last_unstale":"2024-09-11T22:20:19.192242+0000","last_undegraded":"2024-09-11T22:20:19.192242+0000","last_fullsized":"2024-09-11T22:20:19.192242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:59.619611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195295+0000","last_change":"2024-09-11T22:20:18.193947+0000","last_active":"2024-09-11T22:20:19.195295+0000","last_peered":"2024-09-11T22:20:19.195295+0000","last_clean":"2024-09-11T22:20:19.195295+0000","last_became_active":"2024-09-11T22:20:18.192257+0000","last_became_peered":"2024-09-11T22:20:18.192257+0000","last_unstale":"2024-09-11T22:20:19.195295+0000","last_undegraded":"2024-09-11T22:20:19.195295+0000","last_fullsized":"2024-09-11T22:20:19.195295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.478730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188080+0000","last_change":"2024-09-11T22:20:18.184549+0000","last_active":"2024-09-11T22:20:19.188080+0000","last_peered":"2024-09-11T22:20:19.188080+0000","last_clean":"2024-09-11T22:20:19.188080+0000","last_became_active":"2024-09-11T22:20:18.184075+0000","last_became_peered":"2024-09-11T22:20:18.184075+0000","last_unstale":"2024-09-11T22:20:19.188080+0000","last_undegraded":"2024-09-11T22:20:19.188080+0000","last_fullsized":"2024-09-11T22:20:19.188080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:50.095643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:39.285529+0000","last_change":"2024-09-11T22:20:16.128524+0000","last_active":"2024-09-11T22:20:39.285529+0000","last_peered":"2024-09-11T22:20:39.285529+0000","last_clean":"2024-09-11T22:20:39.285529+0000","last_became_active":"2024-09-11T22:20:16.128185+0000","last_became_peered":"2024-09-11T22:20:16.128185+0000","last_unstale":"2024-09-11T22:20:39.285529+0000","last_undegraded":"2024-09-11T22:20:39.285529+0000","last_fullsized":"2024-09-11T22:20:39.285529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.022931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.182947+0000","last_change":"2024-09-11T22:20:18.191578+0000","last_active":"2024-09-11T22:20:19.182947+0000","last_peered":"2024-09-11T22:20:19.182947+0000","last_clean":"2024-09-11T22:20:19.182947+0000","last_became_active":"2024-09-11T22:20:18.189568+0000","last_became_peered":"2024-09-11T22:20:18.189568+0000","last_unstale":"2024-09-11T22:20:19.182947+0000","last_undegraded":"2024-09-11T22:20:19.182947+0000","last_fullsized":"2024-09-11T22:20:19.182947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:41.737242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:28.258394+0000","last_change":"2024-09-11T22:20:16.119856+0000","last_active":"2024-09-11T22:20:28.258394+0000","last_peered":"2024-09-11T22:20:28.258394+0000","last_clean":"2024-09-11T22:20:28.258394+0000","last_became_active":"2024-09-11T22:20:16.119471+0000","last_became_peered":"2024-09-11T22:20:16.119471+0000","last_unstale":"2024-09-11T22:20:28.258394+0000","last_undegraded":"2024-09-11T22:20:28.258394+0000","last_fullsized":"2024-09-11T22:20:28.258394+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:49.317956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:27.255054+0000","last_change":"2024-09-11T22:20:16.115602+0000","last_active":"2024-09-11T22:20:27.255054+0000","last_peered":"2024-09-11T22:20:27.255054+0000","last_clean":"2024-09-11T22:20:27.255054+0000","last_became_active":"2024-09-11T22:20:16.115280+0000","last_became_peered":"2024-09-11T22:20:16.115280+0000","last_unstale":"2024-09-11T22:20:27.255054+0000","last_undegraded":"2024-09-11T22:20:27.255054+0000","last_fullsized":"2024-09-11T22:20:27.255054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:41.942137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180017+0000","last_change":"2024-09-11T22:20:18.184125+0000","last_active":"2024-09-11T22:20:19.180017+0000","last_peered":"2024-09-11T22:20:19.180017+0000","last_clean":"2024-09-11T22:20:19.180017+0000","last_became_active":"2024-09-11T22:20:18.183785+0000","last_became_peered":"2024-09-11T22:20:18.183785+0000","last_unstale":"2024-09-11T22:20:19.180017+0000","last_undegraded":"2024-09-11T22:20:19.180017+0000","last_fullsized":"2024-09-11T22:20:19.180017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:33.230175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.271855+0000","last_change":"2024-09-11T22:20:16.125852+0000","last_active":"2024-09-11T22:20:29.271855+0000","last_peered":"2024-09-11T22:20:29.271855+0000","last_clean":"2024-09-11T22:20:29.271855+0000","last_became_active":"2024-09-11T22:20:16.125250+0000","last_became_peered":"2024-09-11T22:20:16.125250+0000","last_unstale":"2024-09-11T22:20:29.271855+0000","last_undegraded":"2024-09-11T22:20:29.271855+0000","last_fullsized":"2024-09-11T22:20:29.271855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:34.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.191546+0000","last_change":"2024-09-11T22:20:18.185147+0000","last_active":"2024-09-11T22:20:19.191546+0000","last_peered":"2024-09-11T22:20:19.191546+0000","last_clean":"2024-09-11T22:20:19.191546+0000","last_became_active":"2024-09-11T22:20:18.184840+0000","last_became_peered":"2024-09-11T22:20:18.184840+0000","last_unstale":"2024-09-11T22:20:19.191546+0000","last_undegraded":"2024-09-11T22:20:19.191546+0000","last_fullsized":"2024-09-11T22:20:19.191546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:59.703349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:39.285009+0000","last_change":"2024-09-11T22:20:16.122173+0000","last_active":"2024-09-11T22:20:39.285009+0000","last_peered":"2024-09-11T22:20:39.285009+0000","last_clean":"2024-09-11T22:20:39.285009+0000","last_became_active":"2024-09-11T22:20:16.121847+0000","last_became_peered":"2024-09-11T22:20:16.121847+0000","last_unstale":"2024-09-11T22:20:39.285009+0000","last_undegraded":"2024-09-11T22:20:39.285009+0000","last_fullsized":"2024-09-11T22:20:39.285009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:41.284557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187924+0000","last_change":"2024-09-11T22:20:18.188030+0000","last_active":"2024-09-11T22:20:19.187924+0000","last_peered":"2024-09-11T22:20:19.187924+0000","last_clean":"2024-09-11T22:20:19.187924+0000","last_became_active":"2024-09-11T22:20:18.187503+0000","last_became_peered":"2024-09-11T22:20:18.187503+0000","last_unstale":"2024-09-11T22:20:19.187924+0000","last_undegraded":"2024-09-11T22:20:19.187924+0000","last_fullsized":"2024-09-11T22:20:19.187924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:45.494149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.270554+0000","last_change":"2024-09-11T22:20:16.128828+0000","last_active":"2024-09-11T22:20:29.270554+0000","last_peered":"2024-09-11T22:20:29.270554+0000","last_clean":"2024-09-11T22:20:29.270554+0000","last_became_active":"2024-09-11T22:20:16.128454+0000","last_became_peered":"2024-09-11T22:20:16.128454+0000","last_unstale":"2024-09-11T22:20:29.270554+0000","last_undegraded":"2024-09-11T22:20:29.270554+0000","last_fullsized":"2024-09-11T22:20:29.270554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:33.927564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188560+0000","last_change":"2024-09-11T22:20:18.193795+0000","last_active":"2024-09-11T22:20:19.188560+0000","last_peered":"2024-09-11T22:20:19.188560+0000","last_clean":"2024-09-11T22:20:19.188560+0000","last_became_active":"2024-09-11T22:20:18.193489+0000","last_became_peered":"2024-09-11T22:20:18.193489+0000","last_unstale":"2024-09-11T22:20:19.188560+0000","last_undegraded":"2024-09-11T22:20:19.188560+0000","last_fullsized":"2024-09-11T22:20:19.188560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:14.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:28.258631+0000","last_change":"2024-09-11T22:20:16.187938+0000","last_active":"2024-09-11T22:20:28.258631+0000","last_peered":"2024-09-11T22:20:28.258631+0000","last_clean":"2024-09-11T22:20:28.258631+0000","last_became_active":"2024-09-11T22:20:16.187545+0000","last_became_peered":"2024-09-11T22:20:16.187545+0000","last_unstale":"2024-09-11T22:20:28.258631+0000","last_undegraded":"2024-09-11T22:20:28.258631+0000","last_fullsized":"2024-09-11T22:20:28.258631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:41.674356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188079+0000","last_change":"2024-09-11T22:20:18.193018+0000","last_active":"2024-09-11T22:20:19.188079+0000","last_peered":"2024-09-11T22:20:19.188079+0000","last_clean":"2024-09-11T22:20:19.188079+0000","last_became_active":"2024-09-11T22:20:18.192706+0000","last_became_peered":"2024-09-11T22:20:18.192706+0000","last_unstale":"2024-09-11T22:20:19.188079+0000","last_undegraded":"2024-09-11T22:20:19.188079+0000","last_fullsized":"2024-09-11T22:20:19.188079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:17.626167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'24","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:39.286759+0000","last_change":"2024-09-11T22:20:16.128082+0000","last_active":"2024-09-11T22:20:39.286759+0000","last_peered":"2024-09-11T22:20:39.286759+0000","last_clean":"2024-09-11T22:20:39.286759+0000","last_became_active":"2024-09-11T22:20:16.127731+0000","last_became_peered":"2024-09-11T22:20:16.127731+0000","last_unstale":"2024-09-11T22:20:39.286759+0000","last_undegraded":"2024-09-11T22:20:39.286759+0000","last_fullsized":"2024-09-11T22:20:39.286759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:33.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":58,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189321+0000","last_change":"2024-09-11T22:20:18.189219+0000","last_active":"2024-09-11T22:20:19.189321+0000","last_peered":"2024-09-11T22:20:19.189321+0000","last_clean":"2024-09-11T22:20:19.189321+0000","last_became_active":"2024-09-11T22:20:18.188687+0000","last_became_peered":"2024-09-11T22:20:18.188687+0000","last_unstale":"2024-09-11T22:20:19.189321+0000","last_undegraded":"2024-09-11T22:20:19.189321+0000","last_fullsized":"2024-09-11T22:20:19.189321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:24.171126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:30.269724+0000","last_change":"2024-09-11T22:20:16.129724+0000","last_active":"2024-09-11T22:20:30.269724+0000","last_peered":"2024-09-11T22:20:30.269724+0000","last_clean":"2024-09-11T22:20:30.269724+0000","last_became_active":"2024-09-11T22:20:16.129351+0000","last_became_peered":"2024-09-11T22:20:16.129351+0000","last_unstale":"2024-09-11T22:20:30.269724+0000","last_undegraded":"2024-09-11T22:20:30.269724+0000","last_fullsized":"2024-09-11T22:20:30.269724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:09.642520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188152+0000","last_change":"2024-09-11T22:20:18.187031+0000","last_active":"2024-09-11T22:20:19.188152+0000","last_peered":"2024-09-11T22:20:19.188152+0000","last_clean":"2024-09-11T22:20:19.188152+0000","last_became_active":"2024-09-11T22:20:18.186698+0000","last_became_peered":"2024-09-11T22:20:18.186698+0000","last_unstale":"2024-09-11T22:20:19.188152+0000","last_undegraded":"2024-09-11T22:20:19.188152+0000","last_fullsized":"2024-09-11T22:20:19.188152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:04.810981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.271688+0000","last_change":"2024-09-11T22:20:16.124925+0000","last_active":"2024-09-11T22:20:29.271688+0000","last_peered":"2024-09-11T22:20:29.271688+0000","last_clean":"2024-09-11T22:20:29.271688+0000","last_became_active":"2024-09-11T22:20:16.124580+0000","last_became_peered":"2024-09-11T22:20:16.124580+0000","last_unstale":"2024-09-11T22:20:29.271688+0000","last_undegraded":"2024-09-11T22:20:29.271688+0000","last_fullsized":"2024-09-11T22:20:29.271688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:56.417577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189713+0000","last_change":"2024-09-11T22:20:18.194531+0000","last_active":"2024-09-11T22:20:19.189713+0000","last_peered":"2024-09-11T22:20:19.189713+0000","last_clean":"2024-09-11T22:20:19.189713+0000","last_became_active":"2024-09-11T22:20:18.194167+0000","last_became_peered":"2024-09-11T22:20:18.194167+0000","last_unstale":"2024-09-11T22:20:19.189713+0000","last_undegraded":"2024-09-11T22:20:19.189713+0000","last_fullsized":"2024-09-11T22:20:19.189713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:17.595672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:38.284222+0000","last_change":"2024-09-11T22:20:16.118934+0000","last_active":"2024-09-11T22:20:38.284222+0000","last_peered":"2024-09-11T22:20:38.284222+0000","last_clean":"2024-09-11T22:20:38.284222+0000","last_became_active":"2024-09-11T22:20:16.118614+0000","last_became_peered":"2024-09-11T22:20:16.118614+0000","last_unstale":"2024-09-11T22:20:38.284222+0000","last_undegraded":"2024-09-11T22:20:38.284222+0000","last_fullsized":"2024-09-11T22:20:38.284222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:05.791161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188120+0000","last_change":"2024-09-11T22:20:18.197692+0000","last_active":"2024-09-11T22:20:19.188120+0000","last_peered":"2024-09-11T22:20:19.188120+0000","last_clean":"2024-09-11T22:20:19.188120+0000","last_became_active":"2024-09-11T22:20:18.197336+0000","last_became_peered":"2024-09-11T22:20:18.197336+0000","last_unstale":"2024-09-11T22:20:19.188120+0000","last_undegraded":"2024-09-11T22:20:19.188120+0000","last_fullsized":"2024-09-11T22:20:19.188120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:55.172568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.271344+0000","last_change":"2024-09-11T22:20:16.186769+0000","last_active":"2024-09-11T22:20:29.271344+0000","last_peered":"2024-09-11T22:20:29.271344+0000","last_clean":"2024-09-11T22:20:29.271344+0000","last_became_active":"2024-09-11T22:20:16.186367+0000","last_became_peered":"2024-09-11T22:20:16.186367+0000","last_unstale":"2024-09-11T22:20:29.271344+0000","last_undegraded":"2024-09-11T22:20:29.271344+0000","last_fullsized":"2024-09-11T22:20:29.271344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.279387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.182723+0000","last_change":"2024-09-11T22:20:18.195570+0000","last_active":"2024-09-11T22:20:19.182723+0000","last_peered":"2024-09-11T22:20:19.182723+0000","last_clean":"2024-09-11T22:20:19.182723+0000","last_became_active":"2024-09-11T22:20:18.195226+0000","last_became_peered":"2024-09-11T22:20:18.195226+0000","last_unstale":"2024-09-11T22:20:19.182723+0000","last_undegraded":"2024-09-11T22:20:19.182723+0000","last_fullsized":"2024-09-11T22:20:19.182723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:44.108090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:28.249716+0000","last_change":"2024-09-11T22:20:16.122338+0000","last_active":"2024-09-11T22:20:28.249716+0000","last_peered":"2024-09-11T22:20:28.249716+0000","last_clean":"2024-09-11T22:20:28.249716+0000","last_became_active":"2024-09-11T22:20:16.122007+0000","last_became_peered":"2024-09-11T22:20:16.122007+0000","last_unstale":"2024-09-11T22:20:28.249716+0000","last_undegraded":"2024-09-11T22:20:28.249716+0000","last_fullsized":"2024-09-11T22:20:28.249716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.191271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.196179+0000","last_change":"2024-09-11T22:20:18.194112+0000","last_active":"2024-09-11T22:20:19.196179+0000","last_peered":"2024-09-11T22:20:19.196179+0000","last_clean":"2024-09-11T22:20:19.196179+0000","last_became_active":"2024-09-11T22:20:18.193812+0000","last_became_peered":"2024-09-11T22:20:18.193812+0000","last_unstale":"2024-09-11T22:20:19.196179+0000","last_undegraded":"2024-09-11T22:20:19.196179+0000","last_fullsized":"2024-09-11T22:20:19.196179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:35.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.268825+0000","last_change":"2024-09-11T22:20:16.128127+0000","last_active":"2024-09-11T22:20:29.268825+0000","last_peered":"2024-09-11T22:20:29.268825+0000","last_clean":"2024-09-11T22:20:29.268825+0000","last_became_active":"2024-09-11T22:20:16.127767+0000","last_became_peered":"2024-09-11T22:20:16.127767+0000","last_unstale":"2024-09-11T22:20:29.268825+0000","last_undegraded":"2024-09-11T22:20:29.268825+0000","last_fullsized":"2024-09-11T22:20:29.268825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:22.227358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189019+0000","last_change":"2024-09-11T22:20:18.191960+0000","last_active":"2024-09-11T22:20:19.189019+0000","last_peered":"2024-09-11T22:20:19.189019+0000","last_clean":"2024-09-11T22:20:19.189019+0000","last_became_active":"2024-09-11T22:20:18.191635+0000","last_became_peered":"2024-09-11T22:20:18.191635+0000","last_unstale":"2024-09-11T22:20:19.189019+0000","last_undegraded":"2024-09-11T22:20:19.189019+0000","last_fullsized":"2024-09-11T22:20:19.189019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:19:18.919247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.271079+0000","last_change":"2024-09-11T22:20:16.123608+0000","last_active":"2024-09-11T22:20:29.271079+0000","last_peered":"2024-09-11T22:20:29.271079+0000","last_clean":"2024-09-11T22:20:29.271079+0000","last_became_active":"2024-09-11T22:20:16.123193+0000","last_became_peered":"2024-09-11T22:20:16.123193+0000","last_unstale":"2024-09-11T22:20:29.271079+0000","last_undegraded":"2024-09-11T22:20:29.271079+0000","last_fullsized":"2024-09-11T22:20:29.271079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:07.362115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:21.953397+0000","last_change":"2024-09-11T22:20:18.199957+0000","last_active":"2024-09-11T22:20:21.953397+0000","last_peered":"2024-09-11T22:20:21.953397+0000","last_clean":"2024-09-11T22:20:21.953397+0000","last_became_active":"2024-09-11T22:20:18.199637+0000","last_became_peered":"2024-09-11T22:20:18.199637+0000","last_unstale":"2024-09-11T22:20:21.953397+0000","last_undegraded":"2024-09-11T22:20:21.953397+0000","last_fullsized":"2024-09-11T22:20:21.953397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:14.949002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":33,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:36.282660+0000","last_change":"2024-09-11T22:20:16.460955+0000","last_active":"2024-09-11T22:20:36.282660+0000","last_peered":"2024-09-11T22:20:36.282660+0000","last_clean":"2024-09-11T22:20:36.282660+0000","last_became_active":"2024-09-11T22:20:16.460486+0000","last_became_peered":"2024-09-11T22:20:16.460486+0000","last_unstale":"2024-09-11T22:20:36.282660+0000","last_undegraded":"2024-09-11T22:20:36.282660+0000","last_fullsized":"2024-09-11T22:20:36.282660+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:58.616853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189020+0000","last_change":"2024-09-11T22:20:18.196691+0000","last_active":"2024-09-11T22:20:19.189020+0000","last_peered":"2024-09-11T22:20:19.189020+0000","last_clean":"2024-09-11T22:20:19.189020+0000","last_became_active":"2024-09-11T22:20:18.196364+0000","last_became_peered":"2024-09-11T22:20:18.196364+0000","last_unstale":"2024-09-11T22:20:19.189020+0000","last_undegraded":"2024-09-11T22:20:19.189020+0000","last_fullsized":"2024-09-11T22:20:19.189020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:50.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":341,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188402+0000","last_change":"2024-09-11T22:19:02.590079+0000","last_active":"2024-09-11T22:20:19.188402+0000","last_peered":"2024-09-11T22:20:19.188402+0000","last_clean":"2024-09-11T22:20:19.188402+0000","last_became_active":"2024-09-11T22:19:02.589644+0000","last_became_peered":"2024-09-11T22:19:02.589644+0000","last_unstale":"2024-09-11T22:20:19.188402+0000","last_undegraded":"2024-09-11T22:20:19.188402+0000","last_fullsized":"2024-09-11T22:20:19.188402+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_clean_scrub_stamp":"2024-09-11T22:15:33.244685+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:45.506190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:30.269896+0000","last_change":"2024-09-11T22:20:16.116661+0000","last_active":"2024-09-11T22:20:30.269896+0000","last_peered":"2024-09-11T22:20:30.269896+0000","last_clean":"2024-09-11T22:20:30.269896+0000","last_became_active":"2024-09-11T22:20:16.116196+0000","last_became_peered":"2024-09-11T22:20:16.116196+0000","last_unstale":"2024-09-11T22:20:30.269896+0000","last_undegraded":"2024-09-11T22:20:30.269896+0000","last_fullsized":"2024-09-11T22:20:30.269896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:08.875170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180451+0000","last_change":"2024-09-11T22:20:18.193176+0000","last_active":"2024-09-11T22:20:19.180451+0000","last_peered":"2024-09-11T22:20:19.180451+0000","last_clean":"2024-09-11T22:20:19.180451+0000","last_became_active":"2024-09-11T22:20:18.192312+0000","last_became_peered":"2024-09-11T22:20:18.192312+0000","last_unstale":"2024-09-11T22:20:19.180451+0000","last_undegraded":"2024-09-11T22:20:19.180451+0000","last_fullsized":"2024-09-11T22:20:19.180451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:08.738569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:27.253672+0000","last_change":"2024-09-11T22:20:16.123772+0000","last_active":"2024-09-11T22:20:27.253672+0000","last_peered":"2024-09-11T22:20:27.253672+0000","last_clean":"2024-09-11T22:20:27.253672+0000","last_became_active":"2024-09-11T22:20:16.123403+0000","last_became_peered":"2024-09-11T22:20:16.123403+0000","last_unstale":"2024-09-11T22:20:27.253672+0000","last_undegraded":"2024-09-11T22:20:27.253672+0000","last_fullsized":"2024-09-11T22:20:27.253672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:24.741520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.186820+0000","last_change":"2024-09-11T22:20:18.191118+0000","last_active":"2024-09-11T22:20:19.186820+0000","last_peered":"2024-09-11T22:20:19.186820+0000","last_clean":"2024-09-11T22:20:19.186820+0000","last_became_active":"2024-09-11T22:20:18.190760+0000","last_became_peered":"2024-09-11T22:20:18.190760+0000","last_unstale":"2024-09-11T22:20:19.186820+0000","last_undegraded":"2024-09-11T22:20:19.186820+0000","last_fullsized":"2024-09-11T22:20:19.186820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:59.220067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:40.286284+0000","last_change":"2024-09-11T22:20:16.186407+0000","last_active":"2024-09-11T22:20:40.286284+0000","last_peered":"2024-09-11T22:20:40.286284+0000","last_clean":"2024-09-11T22:20:40.286284+0000","last_became_active":"2024-09-11T22:20:16.186105+0000","last_became_peered":"2024-09-11T22:20:16.186105+0000","last_unstale":"2024-09-11T22:20:40.286284+0000","last_undegraded":"2024-09-11T22:20:40.286284+0000","last_fullsized":"2024-09-11T22:20:40.286284+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:43.942072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.182722+0000","last_change":"2024-09-11T22:20:18.191970+0000","last_active":"2024-09-11T22:20:19.182722+0000","last_peered":"2024-09-11T22:20:19.182722+0000","last_clean":"2024-09-11T22:20:19.182722+0000","last_became_active":"2024-09-11T22:20:18.191623+0000","last_became_peered":"2024-09-11T22:20:18.191623+0000","last_unstale":"2024-09-11T22:20:19.182722+0000","last_undegraded":"2024-09-11T22:20:19.182722+0000","last_fullsized":"2024-09-11T22:20:19.182722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:33.213979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:40.286467+0000","last_change":"2024-09-11T22:20:16.119936+0000","last_active":"2024-09-11T22:20:40.286467+0000","last_peered":"2024-09-11T22:20:40.286467+0000","last_clean":"2024-09-11T22:20:40.286467+0000","last_became_active":"2024-09-11T22:20:16.119453+0000","last_became_peered":"2024-09-11T22:20:16.119453+0000","last_unstale":"2024-09-11T22:20:40.286467+0000","last_undegraded":"2024-09-11T22:20:40.286467+0000","last_fullsized":"2024-09-11T22:20:40.286467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:20.717980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.181111+0000","last_change":"2024-09-11T22:20:18.192581+0000","last_active":"2024-09-11T22:20:19.181111+0000","last_peered":"2024-09-11T22:20:19.181111+0000","last_clean":"2024-09-11T22:20:19.181111+0000","last_became_active":"2024-09-11T22:20:18.191702+0000","last_became_peered":"2024-09-11T22:20:18.191702+0000","last_unstale":"2024-09-11T22:20:19.181111+0000","last_undegraded":"2024-09-11T22:20:19.181111+0000","last_fullsized":"2024-09-11T22:20:19.181111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:48.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.191997+0000","last_change":"2024-09-11T22:20:16.186485+0000","last_active":"2024-09-11T22:20:19.191997+0000","last_peered":"2024-09-11T22:20:19.191997+0000","last_clean":"2024-09-11T22:20:19.191997+0000","last_became_active":"2024-09-11T22:20:16.186171+0000","last_became_peered":"2024-09-11T22:20:16.186171+0000","last_unstale":"2024-09-11T22:20:19.191997+0000","last_undegraded":"2024-09-11T22:20:19.191997+0000","last_fullsized":"2024-09-11T22:20:19.191997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:26.373626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:20.263658+0000","last_change":"2024-09-11T22:20:18.191361+0000","last_active":"2024-09-11T22:20:20.263658+0000","last_peered":"2024-09-11T22:20:20.263658+0000","last_clean":"2024-09-11T22:20:20.263658+0000","last_became_active":"2024-09-11T22:20:18.191036+0000","last_became_peered":"2024-09-11T22:20:18.191036+0000","last_unstale":"2024-09-11T22:20:20.263658+0000","last_undegraded":"2024-09-11T22:20:20.263658+0000","last_fullsized":"2024-09-11T22:20:20.263658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:29.124650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.270710+0000","last_change":"2024-09-11T22:20:16.188156+0000","last_active":"2024-09-11T22:20:29.270710+0000","last_peered":"2024-09-11T22:20:29.270710+0000","last_clean":"2024-09-11T22:20:29.270710+0000","last_became_active":"2024-09-11T22:20:16.187809+0000","last_became_peered":"2024-09-11T22:20:16.187809+0000","last_unstale":"2024-09-11T22:20:29.270710+0000","last_undegraded":"2024-09-11T22:20:29.270710+0000","last_fullsized":"2024-09-11T22:20:29.270710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:04.219621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189850+0000","last_change":"2024-09-11T22:20:18.182380+0000","last_active":"2024-09-11T22:20:19.189850+0000","last_peered":"2024-09-11T22:20:19.189850+0000","last_clean":"2024-09-11T22:20:19.189850+0000","last_became_active":"2024-09-11T22:20:18.182062+0000","last_became_peered":"2024-09-11T22:20:18.182062+0000","last_unstale":"2024-09-11T22:20:19.189850+0000","last_undegraded":"2024-09-11T22:20:19.189850+0000","last_fullsized":"2024-09-11T22:20:19.189850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:15.575846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'10","reported_seq":28,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:34.187418+0000","last_change":"2024-09-11T22:20:16.117335+0000","last_active":"2024-09-11T22:20:34.187418+0000","last_peered":"2024-09-11T22:20:34.187418+0000","last_clean":"2024-09-11T22:20:34.187418+0000","last_became_active":"2024-09-11T22:20:16.116977+0000","last_became_peered":"2024-09-11T22:20:16.116977+0000","last_unstale":"2024-09-11T22:20:34.187418+0000","last_undegraded":"2024-09-11T22:20:34.187418+0000","last_fullsized":"2024-09-11T22:20:34.187418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:51.744202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180505+0000","last_change":"2024-09-11T22:20:18.194069+0000","last_active":"2024-09-11T22:20:19.180505+0000","last_peered":"2024-09-11T22:20:19.180505+0000","last_clean":"2024-09-11T22:20:19.180505+0000","last_became_active":"2024-09-11T22:20:18.193765+0000","last_became_peered":"2024-09-11T22:20:18.193765+0000","last_unstale":"2024-09-11T22:20:19.180505+0000","last_undegraded":"2024-09-11T22:20:19.180505+0000","last_fullsized":"2024-09-11T22:20:19.180505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:25.047405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.268944+0000","last_change":"2024-09-11T22:20:16.460987+0000","last_active":"2024-09-11T22:20:29.268944+0000","last_peered":"2024-09-11T22:20:29.268944+0000","last_clean":"2024-09-11T22:20:29.268944+0000","last_became_active":"2024-09-11T22:20:16.460516+0000","last_became_peered":"2024-09-11T22:20:16.460516+0000","last_unstale":"2024-09-11T22:20:29.268944+0000","last_undegraded":"2024-09-11T22:20:29.268944+0000","last_fullsized":"2024-09-11T22:20:29.268944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:00.987981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189239+0000","last_change":"2024-09-11T22:20:18.189921+0000","last_active":"2024-09-11T22:20:19.189239+0000","last_peered":"2024-09-11T22:20:19.189239+0000","last_clean":"2024-09-11T22:20:19.189239+0000","last_became_active":"2024-09-11T22:20:18.189616+0000","last_became_peered":"2024-09-11T22:20:18.189616+0000","last_unstale":"2024-09-11T22:20:19.189239+0000","last_undegraded":"2024-09-11T22:20:19.189239+0000","last_fullsized":"2024-09-11T22:20:19.189239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:16.515211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:26.238455+0000","last_change":"2024-09-11T22:20:16.186498+0000","last_active":"2024-09-11T22:20:26.238455+0000","last_peered":"2024-09-11T22:20:26.238455+0000","last_clean":"2024-09-11T22:20:26.238455+0000","last_became_active":"2024-09-11T22:20:16.186214+0000","last_became_peered":"2024-09-11T22:20:16.186214+0000","last_unstale":"2024-09-11T22:20:26.238455+0000","last_undegraded":"2024-09-11T22:20:26.238455+0000","last_fullsized":"2024-09-11T22:20:26.238455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:09.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187945+0000","last_change":"2024-09-11T22:20:18.187489+0000","last_active":"2024-09-11T22:20:19.187945+0000","last_peered":"2024-09-11T22:20:19.187945+0000","last_clean":"2024-09-11T22:20:19.187945+0000","last_became_active":"2024-09-11T22:20:18.185649+0000","last_became_peered":"2024-09-11T22:20:18.185649+0000","last_unstale":"2024-09-11T22:20:19.187945+0000","last_undegraded":"2024-09-11T22:20:19.187945+0000","last_fullsized":"2024-09-11T22:20:19.187945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:42.825075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.270150+0000","last_change":"2024-09-11T22:20:16.117627+0000","last_active":"2024-09-11T22:20:29.270150+0000","last_peered":"2024-09-11T22:20:29.270150+0000","last_clean":"2024-09-11T22:20:29.270150+0000","last_became_active":"2024-09-11T22:20:16.117243+0000","last_became_peered":"2024-09-11T22:20:16.117243+0000","last_unstale":"2024-09-11T22:20:29.270150+0000","last_undegraded":"2024-09-11T22:20:29.270150+0000","last_fullsized":"2024-09-11T22:20:29.270150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:58.240535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195464+0000","last_change":"2024-09-11T22:20:18.184326+0000","last_active":"2024-09-11T22:20:19.195464+0000","last_peered":"2024-09-11T22:20:19.195464+0000","last_clean":"2024-09-11T22:20:19.195464+0000","last_became_active":"2024-09-11T22:20:18.183995+0000","last_became_peered":"2024-09-11T22:20:18.183995+0000","last_unstale":"2024-09-11T22:20:19.195464+0000","last_undegraded":"2024-09-11T22:20:19.195464+0000","last_fullsized":"2024-09-11T22:20:19.195464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:33.468000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:28.258102+0000","last_change":"2024-09-11T22:20:16.127180+0000","last_active":"2024-09-11T22:20:28.258102+0000","last_peered":"2024-09-11T22:20:28.258102+0000","last_clean":"2024-09-11T22:20:28.258102+0000","last_became_active":"2024-09-11T22:20:16.126696+0000","last_became_peered":"2024-09-11T22:20:16.126696+0000","last_unstale":"2024-09-11T22:20:28.258102+0000","last_undegraded":"2024-09-11T22:20:28.258102+0000","last_fullsized":"2024-09-11T22:20:28.258102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:02.637225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180611+0000","last_change":"2024-09-11T22:20:18.185635+0000","last_active":"2024-09-11T22:20:19.180611+0000","last_peered":"2024-09-11T22:20:19.180611+0000","last_clean":"2024-09-11T22:20:19.180611+0000","last_became_active":"2024-09-11T22:20:18.185268+0000","last_became_peered":"2024-09-11T22:20:18.185268+0000","last_unstale":"2024-09-11T22:20:19.180611+0000","last_undegraded":"2024-09-11T22:20:19.180611+0000","last_fullsized":"2024-09-11T22:20:19.180611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:15.996231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:26.240372+0000","last_change":"2024-09-11T22:20:16.128311+0000","last_active":"2024-09-11T22:20:26.240372+0000","last_peered":"2024-09-11T22:20:26.240372+0000","last_clean":"2024-09-11T22:20:26.240372+0000","last_became_active":"2024-09-11T22:20:16.127853+0000","last_became_peered":"2024-09-11T22:20:16.127853+0000","last_unstale":"2024-09-11T22:20:26.240372+0000","last_undegraded":"2024-09-11T22:20:26.240372+0000","last_fullsized":"2024-09-11T22:20:26.240372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:57.562470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189917+0000","last_change":"2024-09-11T22:20:18.195782+0000","last_active":"2024-09-11T22:20:19.189917+0000","last_peered":"2024-09-11T22:20:19.189917+0000","last_clean":"2024-09-11T22:20:19.189917+0000","last_became_active":"2024-09-11T22:20:18.195365+0000","last_became_peered":"2024-09-11T22:20:18.195365+0000","last_unstale":"2024-09-11T22:20:19.189917+0000","last_undegraded":"2024-09-11T22:20:19.189917+0000","last_fullsized":"2024-09-11T22:20:19.189917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:49.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.183116+0000","last_change":"2024-09-11T22:20:18.199344+0000","last_active":"2024-09-11T22:20:19.183116+0000","last_peered":"2024-09-11T22:20:19.183116+0000","last_clean":"2024-09-11T22:20:19.183116+0000","last_became_active":"2024-09-11T22:20:18.199016+0000","last_became_peered":"2024-09-11T22:20:18.199016+0000","last_unstale":"2024-09-11T22:20:19.183116+0000","last_undegraded":"2024-09-11T22:20:19.183116+0000","last_fullsized":"2024-09-11T22:20:19.183116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:38.322831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.271737+0000","last_change":"2024-09-11T22:20:16.122236+0000","last_active":"2024-09-11T22:20:29.271737+0000","last_peered":"2024-09-11T22:20:29.271737+0000","last_clean":"2024-09-11T22:20:29.271737+0000","last_became_active":"2024-09-11T22:20:16.121867+0000","last_became_peered":"2024-09-11T22:20:16.121867+0000","last_unstale":"2024-09-11T22:20:29.271737+0000","last_undegraded":"2024-09-11T22:20:29.271737+0000","last_fullsized":"2024-09-11T22:20:29.271737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:56.782445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187997+0000","last_change":"2024-09-11T22:20:18.187338+0000","last_active":"2024-09-11T22:20:19.187997+0000","last_peered":"2024-09-11T22:20:19.187997+0000","last_clean":"2024-09-11T22:20:19.187997+0000","last_became_active":"2024-09-11T22:20:18.186985+0000","last_became_peered":"2024-09-11T22:20:18.186985+0000","last_unstale":"2024-09-11T22:20:19.187997+0000","last_undegraded":"2024-09-11T22:20:19.187997+0000","last_fullsized":"2024-09-11T22:20:19.187997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:58.170680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:30.266351+0000","last_change":"2024-09-11T22:20:16.128579+0000","last_active":"2024-09-11T22:20:30.266351+0000","last_peered":"2024-09-11T22:20:30.266351+0000","last_clean":"2024-09-11T22:20:30.266351+0000","last_became_active":"2024-09-11T22:20:16.127864+0000","last_became_peered":"2024-09-11T22:20:16.127864+0000","last_unstale":"2024-09-11T22:20:30.266351+0000","last_undegraded":"2024-09-11T22:20:30.266351+0000","last_fullsized":"2024-09-11T22:20:30.266351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:11:32.005487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187753+0000","last_change":"2024-09-11T22:20:18.185205+0000","last_active":"2024-09-11T22:20:19.187753+0000","last_peered":"2024-09-11T22:20:19.187753+0000","last_clean":"2024-09-11T22:20:19.187753+0000","last_became_active":"2024-09-11T22:20:18.184402+0000","last_became_peered":"2024-09-11T22:20:18.184402+0000","last_unstale":"2024-09-11T22:20:19.187753+0000","last_undegraded":"2024-09-11T22:20:19.187753+0000","last_fullsized":"2024-09-11T22:20:19.187753+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:17:48.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.194010+0000","last_change":"2024-09-11T22:20:16.186372+0000","last_active":"2024-09-11T22:20:19.194010+0000","last_peered":"2024-09-11T22:20:19.194010+0000","last_clean":"2024-09-11T22:20:19.194010+0000","last_became_active":"2024-09-11T22:20:16.185990+0000","last_became_peered":"2024-09-11T22:20:16.185990+0000","last_unstale":"2024-09-11T22:20:19.194010+0000","last_undegraded":"2024-09-11T22:20:19.194010+0000","last_fullsized":"2024-09-11T22:20:19.194010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:23.296379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180237+0000","last_change":"2024-09-11T22:20:18.190670+0000","last_active":"2024-09-11T22:20:19.180237+0000","last_peered":"2024-09-11T22:20:19.180237+0000","last_clean":"2024-09-11T22:20:19.180237+0000","last_became_active":"2024-09-11T22:20:18.190339+0000","last_became_peered":"2024-09-11T22:20:18.190339+0000","last_unstale":"2024-09-11T22:20:19.180237+0000","last_undegraded":"2024-09-11T22:20:19.180237+0000","last_fullsized":"2024-09-11T22:20:19.180237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.085305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:37.282578+0000","last_change":"2024-09-11T22:20:16.117646+0000","last_active":"2024-09-11T22:20:37.282578+0000","last_peered":"2024-09-11T22:20:37.282578+0000","last_clean":"2024-09-11T22:20:37.282578+0000","last_became_active":"2024-09-11T22:20:16.117111+0000","last_became_peered":"2024-09-11T22:20:16.117111+0000","last_unstale":"2024-09-11T22:20:37.282578+0000","last_undegraded":"2024-09-11T22:20:37.282578+0000","last_fullsized":"2024-09-11T22:20:37.282578+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:36.470158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53617,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":139,"num_read_kb":130,"num_write":226,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":155114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249094,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":417048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":1054879,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972379,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":416978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201312,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":417356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430247,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1009876,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":417088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":1009161,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986229,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":416556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215162,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":892928,"data_stored":416403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":440160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":464803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":418034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:20:45.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:20:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:45.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:45 smithi005 ceph-mon[30943]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:20:45.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:45 smithi159 ceph-mon[35534]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:20:45.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:45 smithi172 ceph-mon[35461]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:20:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.811+0000 7fcc51b1d640 1 -- 172.21.15.5:0/4121633927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc4c100a80 msgr2=0x7fcc4c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.811+0000 7fcc51b1d640 1 --2- 172.21.15.5:0/4121633927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc4c100a80 0x7fcc4c100ee0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009cd0 tx=0x7fcc3402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:20:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.811+0000 7fcc51b1d640 1 -- 172.21.15.5:0/4121633927 shutdown_connections 2024-09-11T22:20:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.811+0000 7fcc51b1d640 1 --2- 172.21.15.5:0/4121633927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc4c101420 0x7fcc4c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.811+0000 7fcc51b1d640 1 --2- 172.21.15.5:0/4121633927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc4c100a80 0x7fcc4c100ee0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.811+0000 7fcc51b1d640 1 --2- 172.21.15.5:0/4121633927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c106a80 0x7fcc4c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.811+0000 7fcc51b1d640 1 -- 172.21.15.5:0/4121633927 >> 172.21.15.5:0/4121633927 conn(0x7fcc4c0fc7a0 msgr2=0x7fcc4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.811+0000 7fcc51b1d640 1 -- 172.21.15.5:0/4121633927 shutdown_connections 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.811+0000 7fcc51b1d640 1 -- 172.21.15.5:0/4121633927 wait complete. 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.812+0000 7fcc51b1d640 1 Processor -- start 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.812+0000 7fcc51b1d640 1 -- start start 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc51b1d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c100a80 0x7fcc4c197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc51b1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc4c101420 0x7fcc4c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc51b1d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc4c106a80 0x7fcc4c19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc51b1d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c110ea0 con 0x7fcc4c101420 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc51b1d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c110d20 con 0x7fcc4c106a80 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc51b1d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4c111020 con 0x7fcc4c100a80 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc4c101420 0x7fcc4c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc4c101420 0x7fcc4c1978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42950/0 (socket says 172.21.15.5:42950) 2024-09-11T22:20:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4affd640 1 -- 172.21.15.5:0/2820829724 learned_addr learned my addr 172.21.15.5:0/2820829724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4b7fe640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c100a80 0x7fcc4c197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4bfff640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc4c106a80 0x7fcc4c19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4affd640 1 -- 172.21.15.5:0/2820829724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c100a80 msgr2=0x7fcc4c197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4affd640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c100a80 0x7fcc4c197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4affd640 1 -- 172.21.15.5:0/2820829724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc4c106a80 msgr2=0x7fcc4c19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4affd640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc4c106a80 0x7fcc4c19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4affd640 1 -- 172.21.15.5:0/2820829724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4c19c330 con 0x7fcc4c101420 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4b7fe640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c100a80 0x7fcc4c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4bfff640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc4c106a80 0x7fcc4c19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc4affd640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc4c101420 0x7fcc4c1978c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc3402ae00 tx=0x7fcc34003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc48ff9640 1 -- 172.21.15.5:0/2820829724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34004240 con 0x7fcc4c101420 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc48ff9640 1 -- 172.21.15.5:0/2820829724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc340305a0 con 0x7fcc4c101420 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc48ff9640 1 -- 172.21.15.5:0/2820829724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3403a660 con 0x7fcc4c101420 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.813+0000 7fcc51b1d640 1 -- 172.21.15.5:0/2820829724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4c19c5c0 con 0x7fcc4c101420 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.814+0000 7fcc51b1d640 1 -- 172.21.15.5:0/2820829724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4c1a3f00 con 0x7fcc4c101420 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.815+0000 7fcc51b1d640 1 -- 172.21.15.5:0/2820829724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4c100ee0 con 0x7fcc4c101420 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.816+0000 7fcc48ff9640 1 -- 172.21.15.5:0/2820829724 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fcc34030ac0 con 0x7fcc4c101420 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.816+0000 7fcc48ff9640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcc380a2c10 0x7fcc380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.817+0000 7fcc4b7fe640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcc380a2c10 0x7fcc380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.817+0000 7fcc48ff9640 1 -- 172.21.15.5:0/2820829724 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fcc3402e030 con 0x7fcc4c101420 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.817+0000 7fcc4b7fe640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcc380a2c10 0x7fcc380a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006fd0 tx=0x7fcc3c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.820+0000 7fcc48ff9640 1 -- 172.21.15.5:0/2820829724 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc340098b0 con 0x7fcc4c101420 2024-09-11T22:20:45.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:45.993+0000 7fcc51b1d640 1 -- 172.21.15.5:0/2820829724 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcc4c062630 con 0x7fcc380a2c10 2024-09-11T22:20:46.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.000+0000 7fcc48ff9640 1 -- 172.21.15.5:0/2820829724 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258891 (secure 0 0 0) 0x7fcc4c062630 con 0x7fcc380a2c10 2024-09-11T22:20:46.003 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:46.004 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.002+0000 7fcc51b1d640 1 -- 172.21.15.5:0/2820829724 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcc380a2c10 msgr2=0x7fcc380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.002+0000 7fcc51b1d640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcc380a2c10 0x7fcc380a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006fd0 tx=0x7fcc3c008040 comp rx=0 tx=0).stop 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.002+0000 7fcc51b1d640 1 -- 172.21.15.5:0/2820829724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc4c101420 msgr2=0x7fcc4c1978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.002+0000 7fcc51b1d640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc4c101420 0x7fcc4c1978c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcc3402ae00 tx=0x7fcc34003ba0 comp rx=0 tx=0).stop 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.002+0000 7fcc51b1d640 1 -- 172.21.15.5:0/2820829724 shutdown_connections 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.002+0000 7fcc51b1d640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fcc380a2c10 0x7fcc380a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.002+0000 7fcc51b1d640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc4c106a80 0x7fcc4c19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.002+0000 7fcc51b1d640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc4c101420 0x7fcc4c1978c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.002+0000 7fcc51b1d640 1 --2- 172.21.15.5:0/2820829724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc4c100a80 0x7fcc4c197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.003+0000 7fcc51b1d640 1 -- 172.21.15.5:0/2820829724 >> 172.21.15.5:0/2820829724 conn(0x7fcc4c0fc7a0 msgr2=0x7fcc4c0fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:46.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.003+0000 7fcc51b1d640 1 -- 172.21.15.5:0/2820829724 shutdown_connections 2024-09-11T22:20:46.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:46.003+0000 7fcc51b1d640 1 -- 172.21.15.5:0/2820829724 wait complete. 2024-09-11T22:20:46.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:46 smithi159 ceph-mon[35534]: pgmap v294: 97 pgs: 97 active+clean; 629 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 16 op/s 2024-09-11T22:20:46.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:46 smithi172 ceph-mon[35461]: pgmap v294: 97 pgs: 97 active+clean; 629 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 16 op/s 2024-09-11T22:20:46.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:46 smithi005 ceph-mon[30943]: pgmap v294: 97 pgs: 97 active+clean; 629 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 KiB/s wr, 16 op/s 2024-09-11T22:20:46.849 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-11T22:20:44.443680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":644219,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":285,"num_read_kb":254,"num_write":416,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":339,"ondisk_log_size":339,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376652,"kb_used_data":10764,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124565172,"statfs":{"total":1151940427776,"available":1151554736128,"internally_reserved":0,"allocated":11022336,"data_stored":6922929,"data_compressed":108858,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47201,"num_objects":20,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":135,"num_read_kb":124,"num_write":64,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002601"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195820+0000","last_change":"2024-09-11T22:20:18.189744+0000","last_active":"2024-09-11T22:20:19.195820+0000","last_peered":"2024-09-11T22:20:19.195820+0000","last_clean":"2024-09-11T22:20:19.195820+0000","last_became_active":"2024-09-11T22:20:18.189398+0000","last_became_peered":"2024-09-11T22:20:18.189398+0000","last_unstale":"2024-09-11T22:20:19.195820+0000","last_undegraded":"2024-09-11T22:20:19.195820+0000","last_fullsized":"2024-09-11T22:20:19.195820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:24.152589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189094+0000","last_change":"2024-09-11T22:20:18.186727+0000","last_active":"2024-09-11T22:20:19.189094+0000","last_peered":"2024-09-11T22:20:19.189094+0000","last_clean":"2024-09-11T22:20:19.189094+0000","last_became_active":"2024-09-11T22:20:18.186414+0000","last_became_peered":"2024-09-11T22:20:18.186414+0000","last_unstale":"2024-09-11T22:20:19.189094+0000","last_undegraded":"2024-09-11T22:20:19.189094+0000","last_fullsized":"2024-09-11T22:20:19.189094+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:53.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187434+0000","last_change":"2024-09-11T22:20:18.179984+0000","last_active":"2024-09-11T22:20:19.187434+0000","last_peered":"2024-09-11T22:20:19.187434+0000","last_clean":"2024-09-11T22:20:19.187434+0000","last_became_active":"2024-09-11T22:20:18.179329+0000","last_became_peered":"2024-09-11T22:20:18.179329+0000","last_unstale":"2024-09-11T22:20:19.187434+0000","last_undegraded":"2024-09-11T22:20:19.187434+0000","last_fullsized":"2024-09-11T22:20:19.187434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:19.149970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192384+0000","last_change":"2024-09-11T22:20:18.191768+0000","last_active":"2024-09-11T22:20:19.192384+0000","last_peered":"2024-09-11T22:20:19.192384+0000","last_clean":"2024-09-11T22:20:19.192384+0000","last_became_active":"2024-09-11T22:20:18.191336+0000","last_became_peered":"2024-09-11T22:20:18.191336+0000","last_unstale":"2024-09-11T22:20:19.192384+0000","last_undegraded":"2024-09-11T22:20:19.192384+0000","last_fullsized":"2024-09-11T22:20:19.192384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:44.768932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188166+0000","last_change":"2024-09-11T22:20:18.181469+0000","last_active":"2024-09-11T22:20:19.188166+0000","last_peered":"2024-09-11T22:20:19.188166+0000","last_clean":"2024-09-11T22:20:19.188166+0000","last_became_active":"2024-09-11T22:20:18.181100+0000","last_became_peered":"2024-09-11T22:20:18.181100+0000","last_unstale":"2024-09-11T22:20:19.188166+0000","last_undegraded":"2024-09-11T22:20:19.188166+0000","last_fullsized":"2024-09-11T22:20:19.188166+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:14.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180780+0000","last_change":"2024-09-11T22:20:18.190855+0000","last_active":"2024-09-11T22:20:19.180780+0000","last_peered":"2024-09-11T22:20:19.180780+0000","last_clean":"2024-09-11T22:20:19.180780+0000","last_became_active":"2024-09-11T22:20:18.190523+0000","last_became_peered":"2024-09-11T22:20:18.190523+0000","last_unstale":"2024-09-11T22:20:19.180780+0000","last_undegraded":"2024-09-11T22:20:19.180780+0000","last_fullsized":"2024-09-11T22:20:19.180780+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:18.574957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189508+0000","last_change":"2024-09-11T22:20:18.199213+0000","last_active":"2024-09-11T22:20:19.189508+0000","last_peered":"2024-09-11T22:20:19.189508+0000","last_clean":"2024-09-11T22:20:19.189508+0000","last_became_active":"2024-09-11T22:20:18.198913+0000","last_became_peered":"2024-09-11T22:20:18.198913+0000","last_unstale":"2024-09-11T22:20:19.189508+0000","last_undegraded":"2024-09-11T22:20:19.189508+0000","last_fullsized":"2024-09-11T22:20:19.189508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:22.682802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.179981+0000","last_change":"2024-09-11T22:20:18.192458+0000","last_active":"2024-09-11T22:20:19.179981+0000","last_peered":"2024-09-11T22:20:19.179981+0000","last_clean":"2024-09-11T22:20:19.179981+0000","last_became_active":"2024-09-11T22:20:18.191967+0000","last_became_peered":"2024-09-11T22:20:18.191967+0000","last_unstale":"2024-09-11T22:20:19.179981+0000","last_undegraded":"2024-09-11T22:20:19.179981+0000","last_fullsized":"2024-09-11T22:20:19.179981+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:52.686157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188709+0000","last_change":"2024-09-11T22:20:18.193537+0000","last_active":"2024-09-11T22:20:19.188709+0000","last_peered":"2024-09-11T22:20:19.188709+0000","last_clean":"2024-09-11T22:20:19.188709+0000","last_became_active":"2024-09-11T22:20:18.192497+0000","last_became_peered":"2024-09-11T22:20:18.192497+0000","last_unstale":"2024-09-11T22:20:19.188709+0000","last_undegraded":"2024-09-11T22:20:19.188709+0000","last_fullsized":"2024-09-11T22:20:19.188709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:13.899546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.181240+0000","last_change":"2024-09-11T22:20:18.193771+0000","last_active":"2024-09-11T22:20:19.181240+0000","last_peered":"2024-09-11T22:20:19.181240+0000","last_clean":"2024-09-11T22:20:19.181240+0000","last_became_active":"2024-09-11T22:20:18.193041+0000","last_became_peered":"2024-09-11T22:20:18.193041+0000","last_unstale":"2024-09-11T22:20:19.181240+0000","last_undegraded":"2024-09-11T22:20:19.181240+0000","last_fullsized":"2024-09-11T22:20:19.181240+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:06.656169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188851+0000","last_change":"2024-09-11T22:20:18.197933+0000","last_active":"2024-09-11T22:20:19.188851+0000","last_peered":"2024-09-11T22:20:19.188851+0000","last_clean":"2024-09-11T22:20:19.188851+0000","last_became_active":"2024-09-11T22:20:18.197210+0000","last_became_peered":"2024-09-11T22:20:18.197210+0000","last_unstale":"2024-09-11T22:20:19.188851+0000","last_undegraded":"2024-09-11T22:20:19.188851+0000","last_fullsized":"2024-09-11T22:20:19.188851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:42.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192490+0000","last_change":"2024-09-11T22:20:18.190739+0000","last_active":"2024-09-11T22:20:19.192490+0000","last_peered":"2024-09-11T22:20:19.192490+0000","last_clean":"2024-09-11T22:20:19.192490+0000","last_became_active":"2024-09-11T22:20:18.190122+0000","last_became_peered":"2024-09-11T22:20:18.190122+0000","last_unstale":"2024-09-11T22:20:19.192490+0000","last_undegraded":"2024-09-11T22:20:19.192490+0000","last_fullsized":"2024-09-11T22:20:19.192490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.585035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188362+0000","last_change":"2024-09-11T22:20:18.190303+0000","last_active":"2024-09-11T22:20:19.188362+0000","last_peered":"2024-09-11T22:20:19.188362+0000","last_clean":"2024-09-11T22:20:19.188362+0000","last_became_active":"2024-09-11T22:20:18.189962+0000","last_became_peered":"2024-09-11T22:20:18.189962+0000","last_unstale":"2024-09-11T22:20:19.188362+0000","last_undegraded":"2024-09-11T22:20:19.188362+0000","last_fullsized":"2024-09-11T22:20:19.188362+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:44.872813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187420+0000","last_change":"2024-09-11T22:20:18.185893+0000","last_active":"2024-09-11T22:20:19.187420+0000","last_peered":"2024-09-11T22:20:19.187420+0000","last_clean":"2024-09-11T22:20:19.187420+0000","last_became_active":"2024-09-11T22:20:18.185592+0000","last_became_peered":"2024-09-11T22:20:18.185592+0000","last_unstale":"2024-09-11T22:20:19.187420+0000","last_undegraded":"2024-09-11T22:20:19.187420+0000","last_fullsized":"2024-09-11T22:20:19.187420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:14.725301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187741+0000","last_change":"2024-09-11T22:20:18.193943+0000","last_active":"2024-09-11T22:20:19.187741+0000","last_peered":"2024-09-11T22:20:19.187741+0000","last_clean":"2024-09-11T22:20:19.187741+0000","last_became_active":"2024-09-11T22:20:18.193558+0000","last_became_peered":"2024-09-11T22:20:18.193558+0000","last_unstale":"2024-09-11T22:20:19.187741+0000","last_undegraded":"2024-09-11T22:20:19.187741+0000","last_fullsized":"2024-09-11T22:20:19.187741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:08.736161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187152+0000","last_change":"2024-09-11T22:20:18.191815+0000","last_active":"2024-09-11T22:20:19.187152+0000","last_peered":"2024-09-11T22:20:19.187152+0000","last_clean":"2024-09-11T22:20:19.187152+0000","last_became_active":"2024-09-11T22:20:18.191483+0000","last_became_peered":"2024-09-11T22:20:18.191483+0000","last_unstale":"2024-09-11T22:20:19.187152+0000","last_undegraded":"2024-09-11T22:20:19.187152+0000","last_fullsized":"2024-09-11T22:20:19.187152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:26.432117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195936+0000","last_change":"2024-09-11T22:20:18.190758+0000","last_active":"2024-09-11T22:20:19.195936+0000","last_peered":"2024-09-11T22:20:19.195936+0000","last_clean":"2024-09-11T22:20:19.195936+0000","last_became_active":"2024-09-11T22:20:18.190431+0000","last_became_peered":"2024-09-11T22:20:18.190431+0000","last_unstale":"2024-09-11T22:20:19.195936+0000","last_undegraded":"2024-09-11T22:20:19.195936+0000","last_fullsized":"2024-09-11T22:20:19.195936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:10.927051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192252+0000","last_change":"2024-09-11T22:20:18.185191+0000","last_active":"2024-09-11T22:20:19.192252+0000","last_peered":"2024-09-11T22:20:19.192252+0000","last_clean":"2024-09-11T22:20:19.192252+0000","last_became_active":"2024-09-11T22:20:18.184906+0000","last_became_peered":"2024-09-11T22:20:18.184906+0000","last_unstale":"2024-09-11T22:20:19.192252+0000","last_undegraded":"2024-09-11T22:20:19.192252+0000","last_fullsized":"2024-09-11T22:20:19.192252+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:32.294780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192490+0000","last_change":"2024-09-11T22:20:18.187137+0000","last_active":"2024-09-11T22:20:19.192490+0000","last_peered":"2024-09-11T22:20:19.192490+0000","last_clean":"2024-09-11T22:20:19.192490+0000","last_became_active":"2024-09-11T22:20:18.186770+0000","last_became_peered":"2024-09-11T22:20:18.186770+0000","last_unstale":"2024-09-11T22:20:19.192490+0000","last_undegraded":"2024-09-11T22:20:19.192490+0000","last_fullsized":"2024-09-11T22:20:19.192490+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:59.917695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195850+0000","last_change":"2024-09-11T22:20:18.197137+0000","last_active":"2024-09-11T22:20:19.195850+0000","last_peered":"2024-09-11T22:20:19.195850+0000","last_clean":"2024-09-11T22:20:19.195850+0000","last_became_active":"2024-09-11T22:20:18.192810+0000","last_became_peered":"2024-09-11T22:20:18.192810+0000","last_unstale":"2024-09-11T22:20:19.195850+0000","last_undegraded":"2024-09-11T22:20:19.195850+0000","last_fullsized":"2024-09-11T22:20:19.195850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:57.680726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188422+0000","last_change":"2024-09-11T22:20:18.187568+0000","last_active":"2024-09-11T22:20:19.188422+0000","last_peered":"2024-09-11T22:20:19.188422+0000","last_clean":"2024-09-11T22:20:19.188422+0000","last_became_active":"2024-09-11T22:20:18.187254+0000","last_became_peered":"2024-09-11T22:20:18.187254+0000","last_unstale":"2024-09-11T22:20:19.188422+0000","last_undegraded":"2024-09-11T22:20:19.188422+0000","last_fullsized":"2024-09-11T22:20:19.188422+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:41.724926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187455+0000","last_change":"2024-09-11T22:20:18.190376+0000","last_active":"2024-09-11T22:20:19.187455+0000","last_peered":"2024-09-11T22:20:19.187455+0000","last_clean":"2024-09-11T22:20:19.187455+0000","last_became_active":"2024-09-11T22:20:18.190079+0000","last_became_peered":"2024-09-11T22:20:18.190079+0000","last_unstale":"2024-09-11T22:20:19.187455+0000","last_undegraded":"2024-09-11T22:20:19.187455+0000","last_fullsized":"2024-09-11T22:20:19.187455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:42.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180019+0000","last_change":"2024-09-11T22:20:18.188503+0000","last_active":"2024-09-11T22:20:19.180019+0000","last_peered":"2024-09-11T22:20:19.180019+0000","last_clean":"2024-09-11T22:20:19.180019+0000","last_became_active":"2024-09-11T22:20:18.187957+0000","last_became_peered":"2024-09-11T22:20:18.187957+0000","last_unstale":"2024-09-11T22:20:19.180019+0000","last_undegraded":"2024-09-11T22:20:19.180019+0000","last_fullsized":"2024-09-11T22:20:19.180019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:44:25.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.182690+0000","last_change":"2024-09-11T22:20:18.196936+0000","last_active":"2024-09-11T22:20:19.182690+0000","last_peered":"2024-09-11T22:20:19.182690+0000","last_clean":"2024-09-11T22:20:19.182690+0000","last_became_active":"2024-09-11T22:20:18.196493+0000","last_became_peered":"2024-09-11T22:20:18.196493+0000","last_unstale":"2024-09-11T22:20:19.182690+0000","last_undegraded":"2024-09-11T22:20:19.182690+0000","last_fullsized":"2024-09-11T22:20:19.182690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:06.189806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180237+0000","last_change":"2024-09-11T22:20:18.195260+0000","last_active":"2024-09-11T22:20:19.180237+0000","last_peered":"2024-09-11T22:20:19.180237+0000","last_clean":"2024-09-11T22:20:19.180237+0000","last_became_active":"2024-09-11T22:20:18.194956+0000","last_became_peered":"2024-09-11T22:20:18.194956+0000","last_unstale":"2024-09-11T22:20:19.180237+0000","last_undegraded":"2024-09-11T22:20:19.180237+0000","last_fullsized":"2024-09-11T22:20:19.180237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:34.483291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192585+0000","last_change":"2024-09-11T22:20:18.190591+0000","last_active":"2024-09-11T22:20:19.192585+0000","last_peered":"2024-09-11T22:20:19.192585+0000","last_clean":"2024-09-11T22:20:19.192585+0000","last_became_active":"2024-09-11T22:20:18.190007+0000","last_became_peered":"2024-09-11T22:20:18.190007+0000","last_unstale":"2024-09-11T22:20:19.192585+0000","last_undegraded":"2024-09-11T22:20:19.192585+0000","last_fullsized":"2024-09-11T22:20:19.192585+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:46.281705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195660+0000","last_change":"2024-09-11T22:20:18.193377+0000","last_active":"2024-09-11T22:20:19.195660+0000","last_peered":"2024-09-11T22:20:19.195660+0000","last_clean":"2024-09-11T22:20:19.195660+0000","last_became_active":"2024-09-11T22:20:18.193003+0000","last_became_peered":"2024-09-11T22:20:18.193003+0000","last_unstale":"2024-09-11T22:20:19.195660+0000","last_undegraded":"2024-09-11T22:20:19.195660+0000","last_fullsized":"2024-09-11T22:20:19.195660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:59.052966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189744+0000","last_change":"2024-09-11T22:20:18.189405+0000","last_active":"2024-09-11T22:20:19.189744+0000","last_peered":"2024-09-11T22:20:19.189744+0000","last_clean":"2024-09-11T22:20:19.189744+0000","last_became_active":"2024-09-11T22:20:18.189098+0000","last_became_peered":"2024-09-11T22:20:18.189098+0000","last_unstale":"2024-09-11T22:20:19.189744+0000","last_undegraded":"2024-09-11T22:20:19.189744+0000","last_fullsized":"2024-09-11T22:20:19.189744+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:47.088192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189019+0000","last_change":"2024-09-11T22:20:18.191005+0000","last_active":"2024-09-11T22:20:19.189019+0000","last_peered":"2024-09-11T22:20:19.189019+0000","last_clean":"2024-09-11T22:20:19.189019+0000","last_became_active":"2024-09-11T22:20:18.190681+0000","last_became_peered":"2024-09-11T22:20:18.190681+0000","last_unstale":"2024-09-11T22:20:19.189019+0000","last_undegraded":"2024-09-11T22:20:19.189019+0000","last_fullsized":"2024-09-11T22:20:19.189019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:12.903929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192034+0000","last_change":"2024-09-11T22:20:18.187920+0000","last_active":"2024-09-11T22:20:19.192034+0000","last_peered":"2024-09-11T22:20:19.192034+0000","last_clean":"2024-09-11T22:20:19.192034+0000","last_became_active":"2024-09-11T22:20:18.187526+0000","last_became_peered":"2024-09-11T22:20:18.187526+0000","last_unstale":"2024-09-11T22:20:19.192034+0000","last_undegraded":"2024-09-11T22:20:19.192034+0000","last_fullsized":"2024-09-11T22:20:19.192034+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.347857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.192242+0000","last_change":"2024-09-11T22:20:18.193869+0000","last_active":"2024-09-11T22:20:19.192242+0000","last_peered":"2024-09-11T22:20:19.192242+0000","last_clean":"2024-09-11T22:20:19.192242+0000","last_became_active":"2024-09-11T22:20:18.193468+0000","last_became_peered":"2024-09-11T22:20:18.193468+0000","last_unstale":"2024-09-11T22:20:19.192242+0000","last_undegraded":"2024-09-11T22:20:19.192242+0000","last_fullsized":"2024-09-11T22:20:19.192242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:59.619611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195295+0000","last_change":"2024-09-11T22:20:18.193947+0000","last_active":"2024-09-11T22:20:19.195295+0000","last_peered":"2024-09-11T22:20:19.195295+0000","last_clean":"2024-09-11T22:20:19.195295+0000","last_became_active":"2024-09-11T22:20:18.192257+0000","last_became_peered":"2024-09-11T22:20:18.192257+0000","last_unstale":"2024-09-11T22:20:19.195295+0000","last_undegraded":"2024-09-11T22:20:19.195295+0000","last_fullsized":"2024-09-11T22:20:19.195295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.478730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188080+0000","last_change":"2024-09-11T22:20:18.184549+0000","last_active":"2024-09-11T22:20:19.188080+0000","last_peered":"2024-09-11T22:20:19.188080+0000","last_clean":"2024-09-11T22:20:19.188080+0000","last_became_active":"2024-09-11T22:20:18.184075+0000","last_became_peered":"2024-09-11T22:20:18.184075+0000","last_unstale":"2024-09-11T22:20:19.188080+0000","last_undegraded":"2024-09-11T22:20:19.188080+0000","last_fullsized":"2024-09-11T22:20:19.188080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:50.095643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:39.285529+0000","last_change":"2024-09-11T22:20:16.128524+0000","last_active":"2024-09-11T22:20:39.285529+0000","last_peered":"2024-09-11T22:20:39.285529+0000","last_clean":"2024-09-11T22:20:39.285529+0000","last_became_active":"2024-09-11T22:20:16.128185+0000","last_became_peered":"2024-09-11T22:20:16.128185+0000","last_unstale":"2024-09-11T22:20:39.285529+0000","last_undegraded":"2024-09-11T22:20:39.285529+0000","last_fullsized":"2024-09-11T22:20:39.285529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.022931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.182947+0000","last_change":"2024-09-11T22:20:18.191578+0000","last_active":"2024-09-11T22:20:19.182947+0000","last_peered":"2024-09-11T22:20:19.182947+0000","last_clean":"2024-09-11T22:20:19.182947+0000","last_became_active":"2024-09-11T22:20:18.189568+0000","last_became_peered":"2024-09-11T22:20:18.189568+0000","last_unstale":"2024-09-11T22:20:19.182947+0000","last_undegraded":"2024-09-11T22:20:19.182947+0000","last_fullsized":"2024-09-11T22:20:19.182947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:41.737242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:28.258394+0000","last_change":"2024-09-11T22:20:16.119856+0000","last_active":"2024-09-11T22:20:28.258394+0000","last_peered":"2024-09-11T22:20:28.258394+0000","last_clean":"2024-09-11T22:20:28.258394+0000","last_became_active":"2024-09-11T22:20:16.119471+0000","last_became_peered":"2024-09-11T22:20:16.119471+0000","last_unstale":"2024-09-11T22:20:28.258394+0000","last_undegraded":"2024-09-11T22:20:28.258394+0000","last_fullsized":"2024-09-11T22:20:28.258394+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:49.317956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:27.255054+0000","last_change":"2024-09-11T22:20:16.115602+0000","last_active":"2024-09-11T22:20:27.255054+0000","last_peered":"2024-09-11T22:20:27.255054+0000","last_clean":"2024-09-11T22:20:27.255054+0000","last_became_active":"2024-09-11T22:20:16.115280+0000","last_became_peered":"2024-09-11T22:20:16.115280+0000","last_unstale":"2024-09-11T22:20:27.255054+0000","last_undegraded":"2024-09-11T22:20:27.255054+0000","last_fullsized":"2024-09-11T22:20:27.255054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:41.942137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180017+0000","last_change":"2024-09-11T22:20:18.184125+0000","last_active":"2024-09-11T22:20:19.180017+0000","last_peered":"2024-09-11T22:20:19.180017+0000","last_clean":"2024-09-11T22:20:19.180017+0000","last_became_active":"2024-09-11T22:20:18.183785+0000","last_became_peered":"2024-09-11T22:20:18.183785+0000","last_unstale":"2024-09-11T22:20:19.180017+0000","last_undegraded":"2024-09-11T22:20:19.180017+0000","last_fullsized":"2024-09-11T22:20:19.180017+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:33.230175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.271855+0000","last_change":"2024-09-11T22:20:16.125852+0000","last_active":"2024-09-11T22:20:29.271855+0000","last_peered":"2024-09-11T22:20:29.271855+0000","last_clean":"2024-09-11T22:20:29.271855+0000","last_became_active":"2024-09-11T22:20:16.125250+0000","last_became_peered":"2024-09-11T22:20:16.125250+0000","last_unstale":"2024-09-11T22:20:29.271855+0000","last_undegraded":"2024-09-11T22:20:29.271855+0000","last_fullsized":"2024-09-11T22:20:29.271855+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:34.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.191546+0000","last_change":"2024-09-11T22:20:18.185147+0000","last_active":"2024-09-11T22:20:19.191546+0000","last_peered":"2024-09-11T22:20:19.191546+0000","last_clean":"2024-09-11T22:20:19.191546+0000","last_became_active":"2024-09-11T22:20:18.184840+0000","last_became_peered":"2024-09-11T22:20:18.184840+0000","last_unstale":"2024-09-11T22:20:19.191546+0000","last_undegraded":"2024-09-11T22:20:19.191546+0000","last_fullsized":"2024-09-11T22:20:19.191546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:59.703349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":42,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:39.285009+0000","last_change":"2024-09-11T22:20:16.122173+0000","last_active":"2024-09-11T22:20:39.285009+0000","last_peered":"2024-09-11T22:20:39.285009+0000","last_clean":"2024-09-11T22:20:39.285009+0000","last_became_active":"2024-09-11T22:20:16.121847+0000","last_became_peered":"2024-09-11T22:20:16.121847+0000","last_unstale":"2024-09-11T22:20:39.285009+0000","last_undegraded":"2024-09-11T22:20:39.285009+0000","last_fullsized":"2024-09-11T22:20:39.285009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:41.284557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187924+0000","last_change":"2024-09-11T22:20:18.188030+0000","last_active":"2024-09-11T22:20:19.187924+0000","last_peered":"2024-09-11T22:20:19.187924+0000","last_clean":"2024-09-11T22:20:19.187924+0000","last_became_active":"2024-09-11T22:20:18.187503+0000","last_became_peered":"2024-09-11T22:20:18.187503+0000","last_unstale":"2024-09-11T22:20:19.187924+0000","last_undegraded":"2024-09-11T22:20:19.187924+0000","last_fullsized":"2024-09-11T22:20:19.187924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:45.494149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.270554+0000","last_change":"2024-09-11T22:20:16.128828+0000","last_active":"2024-09-11T22:20:29.270554+0000","last_peered":"2024-09-11T22:20:29.270554+0000","last_clean":"2024-09-11T22:20:29.270554+0000","last_became_active":"2024-09-11T22:20:16.128454+0000","last_became_peered":"2024-09-11T22:20:16.128454+0000","last_unstale":"2024-09-11T22:20:29.270554+0000","last_undegraded":"2024-09-11T22:20:29.270554+0000","last_fullsized":"2024-09-11T22:20:29.270554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:33.927564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188560+0000","last_change":"2024-09-11T22:20:18.193795+0000","last_active":"2024-09-11T22:20:19.188560+0000","last_peered":"2024-09-11T22:20:19.188560+0000","last_clean":"2024-09-11T22:20:19.188560+0000","last_became_active":"2024-09-11T22:20:18.193489+0000","last_became_peered":"2024-09-11T22:20:18.193489+0000","last_unstale":"2024-09-11T22:20:19.188560+0000","last_undegraded":"2024-09-11T22:20:19.188560+0000","last_fullsized":"2024-09-11T22:20:19.188560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:14.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:28.258631+0000","last_change":"2024-09-11T22:20:16.187938+0000","last_active":"2024-09-11T22:20:28.258631+0000","last_peered":"2024-09-11T22:20:28.258631+0000","last_clean":"2024-09-11T22:20:28.258631+0000","last_became_active":"2024-09-11T22:20:16.187545+0000","last_became_peered":"2024-09-11T22:20:16.187545+0000","last_unstale":"2024-09-11T22:20:28.258631+0000","last_undegraded":"2024-09-11T22:20:28.258631+0000","last_fullsized":"2024-09-11T22:20:28.258631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:41.674356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188079+0000","last_change":"2024-09-11T22:20:18.193018+0000","last_active":"2024-09-11T22:20:19.188079+0000","last_peered":"2024-09-11T22:20:19.188079+0000","last_clean":"2024-09-11T22:20:19.188079+0000","last_became_active":"2024-09-11T22:20:18.192706+0000","last_became_peered":"2024-09-11T22:20:18.192706+0000","last_unstale":"2024-09-11T22:20:19.188079+0000","last_undegraded":"2024-09-11T22:20:19.188079+0000","last_fullsized":"2024-09-11T22:20:19.188079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:01:17.626167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'24","reported_seq":68,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:39.286759+0000","last_change":"2024-09-11T22:20:16.128082+0000","last_active":"2024-09-11T22:20:39.286759+0000","last_peered":"2024-09-11T22:20:39.286759+0000","last_clean":"2024-09-11T22:20:39.286759+0000","last_became_active":"2024-09-11T22:20:16.127731+0000","last_became_peered":"2024-09-11T22:20:16.127731+0000","last_unstale":"2024-09-11T22:20:39.286759+0000","last_undegraded":"2024-09-11T22:20:39.286759+0000","last_fullsized":"2024-09-11T22:20:39.286759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:33.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":58,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189321+0000","last_change":"2024-09-11T22:20:18.189219+0000","last_active":"2024-09-11T22:20:19.189321+0000","last_peered":"2024-09-11T22:20:19.189321+0000","last_clean":"2024-09-11T22:20:19.189321+0000","last_became_active":"2024-09-11T22:20:18.188687+0000","last_became_peered":"2024-09-11T22:20:18.188687+0000","last_unstale":"2024-09-11T22:20:19.189321+0000","last_undegraded":"2024-09-11T22:20:19.189321+0000","last_fullsized":"2024-09-11T22:20:19.189321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:24.171126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:30.269724+0000","last_change":"2024-09-11T22:20:16.129724+0000","last_active":"2024-09-11T22:20:30.269724+0000","last_peered":"2024-09-11T22:20:30.269724+0000","last_clean":"2024-09-11T22:20:30.269724+0000","last_became_active":"2024-09-11T22:20:16.129351+0000","last_became_peered":"2024-09-11T22:20:16.129351+0000","last_unstale":"2024-09-11T22:20:30.269724+0000","last_undegraded":"2024-09-11T22:20:30.269724+0000","last_fullsized":"2024-09-11T22:20:30.269724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:09.642520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188152+0000","last_change":"2024-09-11T22:20:18.187031+0000","last_active":"2024-09-11T22:20:19.188152+0000","last_peered":"2024-09-11T22:20:19.188152+0000","last_clean":"2024-09-11T22:20:19.188152+0000","last_became_active":"2024-09-11T22:20:18.186698+0000","last_became_peered":"2024-09-11T22:20:18.186698+0000","last_unstale":"2024-09-11T22:20:19.188152+0000","last_undegraded":"2024-09-11T22:20:19.188152+0000","last_fullsized":"2024-09-11T22:20:19.188152+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:04.810981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.271688+0000","last_change":"2024-09-11T22:20:16.124925+0000","last_active":"2024-09-11T22:20:29.271688+0000","last_peered":"2024-09-11T22:20:29.271688+0000","last_clean":"2024-09-11T22:20:29.271688+0000","last_became_active":"2024-09-11T22:20:16.124580+0000","last_became_peered":"2024-09-11T22:20:16.124580+0000","last_unstale":"2024-09-11T22:20:29.271688+0000","last_undegraded":"2024-09-11T22:20:29.271688+0000","last_fullsized":"2024-09-11T22:20:29.271688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:56.417577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189713+0000","last_change":"2024-09-11T22:20:18.194531+0000","last_active":"2024-09-11T22:20:19.189713+0000","last_peered":"2024-09-11T22:20:19.189713+0000","last_clean":"2024-09-11T22:20:19.189713+0000","last_became_active":"2024-09-11T22:20:18.194167+0000","last_became_peered":"2024-09-11T22:20:18.194167+0000","last_unstale":"2024-09-11T22:20:19.189713+0000","last_undegraded":"2024-09-11T22:20:19.189713+0000","last_fullsized":"2024-09-11T22:20:19.189713+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:19:17.595672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:38.284222+0000","last_change":"2024-09-11T22:20:16.118934+0000","last_active":"2024-09-11T22:20:38.284222+0000","last_peered":"2024-09-11T22:20:38.284222+0000","last_clean":"2024-09-11T22:20:38.284222+0000","last_became_active":"2024-09-11T22:20:16.118614+0000","last_became_peered":"2024-09-11T22:20:16.118614+0000","last_unstale":"2024-09-11T22:20:38.284222+0000","last_undegraded":"2024-09-11T22:20:38.284222+0000","last_fullsized":"2024-09-11T22:20:38.284222+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:05.791161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188120+0000","last_change":"2024-09-11T22:20:18.197692+0000","last_active":"2024-09-11T22:20:19.188120+0000","last_peered":"2024-09-11T22:20:19.188120+0000","last_clean":"2024-09-11T22:20:19.188120+0000","last_became_active":"2024-09-11T22:20:18.197336+0000","last_became_peered":"2024-09-11T22:20:18.197336+0000","last_unstale":"2024-09-11T22:20:19.188120+0000","last_undegraded":"2024-09-11T22:20:19.188120+0000","last_fullsized":"2024-09-11T22:20:19.188120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:55.172568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.271344+0000","last_change":"2024-09-11T22:20:16.186769+0000","last_active":"2024-09-11T22:20:29.271344+0000","last_peered":"2024-09-11T22:20:29.271344+0000","last_clean":"2024-09-11T22:20:29.271344+0000","last_became_active":"2024-09-11T22:20:16.186367+0000","last_became_peered":"2024-09-11T22:20:16.186367+0000","last_unstale":"2024-09-11T22:20:29.271344+0000","last_undegraded":"2024-09-11T22:20:29.271344+0000","last_fullsized":"2024-09-11T22:20:29.271344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.279387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.182723+0000","last_change":"2024-09-11T22:20:18.195570+0000","last_active":"2024-09-11T22:20:19.182723+0000","last_peered":"2024-09-11T22:20:19.182723+0000","last_clean":"2024-09-11T22:20:19.182723+0000","last_became_active":"2024-09-11T22:20:18.195226+0000","last_became_peered":"2024-09-11T22:20:18.195226+0000","last_unstale":"2024-09-11T22:20:19.182723+0000","last_undegraded":"2024-09-11T22:20:19.182723+0000","last_fullsized":"2024-09-11T22:20:19.182723+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:44.108090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:28.249716+0000","last_change":"2024-09-11T22:20:16.122338+0000","last_active":"2024-09-11T22:20:28.249716+0000","last_peered":"2024-09-11T22:20:28.249716+0000","last_clean":"2024-09-11T22:20:28.249716+0000","last_became_active":"2024-09-11T22:20:16.122007+0000","last_became_peered":"2024-09-11T22:20:16.122007+0000","last_unstale":"2024-09-11T22:20:28.249716+0000","last_undegraded":"2024-09-11T22:20:28.249716+0000","last_fullsized":"2024-09-11T22:20:28.249716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.191271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.196179+0000","last_change":"2024-09-11T22:20:18.194112+0000","last_active":"2024-09-11T22:20:19.196179+0000","last_peered":"2024-09-11T22:20:19.196179+0000","last_clean":"2024-09-11T22:20:19.196179+0000","last_became_active":"2024-09-11T22:20:18.193812+0000","last_became_peered":"2024-09-11T22:20:18.193812+0000","last_unstale":"2024-09-11T22:20:19.196179+0000","last_undegraded":"2024-09-11T22:20:19.196179+0000","last_fullsized":"2024-09-11T22:20:19.196179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:35.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.268825+0000","last_change":"2024-09-11T22:20:16.128127+0000","last_active":"2024-09-11T22:20:29.268825+0000","last_peered":"2024-09-11T22:20:29.268825+0000","last_clean":"2024-09-11T22:20:29.268825+0000","last_became_active":"2024-09-11T22:20:16.127767+0000","last_became_peered":"2024-09-11T22:20:16.127767+0000","last_unstale":"2024-09-11T22:20:29.268825+0000","last_undegraded":"2024-09-11T22:20:29.268825+0000","last_fullsized":"2024-09-11T22:20:29.268825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:01:22.227358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189019+0000","last_change":"2024-09-11T22:20:18.191960+0000","last_active":"2024-09-11T22:20:19.189019+0000","last_peered":"2024-09-11T22:20:19.189019+0000","last_clean":"2024-09-11T22:20:19.189019+0000","last_became_active":"2024-09-11T22:20:18.191635+0000","last_became_peered":"2024-09-11T22:20:18.191635+0000","last_unstale":"2024-09-11T22:20:19.189019+0000","last_undegraded":"2024-09-11T22:20:19.189019+0000","last_fullsized":"2024-09-11T22:20:19.189019+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:19:18.919247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.271079+0000","last_change":"2024-09-11T22:20:16.123608+0000","last_active":"2024-09-11T22:20:29.271079+0000","last_peered":"2024-09-11T22:20:29.271079+0000","last_clean":"2024-09-11T22:20:29.271079+0000","last_became_active":"2024-09-11T22:20:16.123193+0000","last_became_peered":"2024-09-11T22:20:16.123193+0000","last_unstale":"2024-09-11T22:20:29.271079+0000","last_undegraded":"2024-09-11T22:20:29.271079+0000","last_fullsized":"2024-09-11T22:20:29.271079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:07.362115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:21.953397+0000","last_change":"2024-09-11T22:20:18.199957+0000","last_active":"2024-09-11T22:20:21.953397+0000","last_peered":"2024-09-11T22:20:21.953397+0000","last_clean":"2024-09-11T22:20:21.953397+0000","last_became_active":"2024-09-11T22:20:18.199637+0000","last_became_peered":"2024-09-11T22:20:18.199637+0000","last_unstale":"2024-09-11T22:20:21.953397+0000","last_undegraded":"2024-09-11T22:20:21.953397+0000","last_fullsized":"2024-09-11T22:20:21.953397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:14.949002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:41.288648+0000","last_change":"2024-09-11T22:20:16.460955+0000","last_active":"2024-09-11T22:20:41.288648+0000","last_peered":"2024-09-11T22:20:41.288648+0000","last_clean":"2024-09-11T22:20:41.288648+0000","last_became_active":"2024-09-11T22:20:16.460486+0000","last_became_peered":"2024-09-11T22:20:16.460486+0000","last_unstale":"2024-09-11T22:20:41.288648+0000","last_undegraded":"2024-09-11T22:20:41.288648+0000","last_fullsized":"2024-09-11T22:20:41.288648+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:58:58.616853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189020+0000","last_change":"2024-09-11T22:20:18.196691+0000","last_active":"2024-09-11T22:20:19.189020+0000","last_peered":"2024-09-11T22:20:19.189020+0000","last_clean":"2024-09-11T22:20:19.189020+0000","last_became_active":"2024-09-11T22:20:18.196364+0000","last_became_peered":"2024-09-11T22:20:18.196364+0000","last_unstale":"2024-09-11T22:20:19.189020+0000","last_undegraded":"2024-09-11T22:20:19.189020+0000","last_fullsized":"2024-09-11T22:20:19.189020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:50.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":341,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.188402+0000","last_change":"2024-09-11T22:19:02.590079+0000","last_active":"2024-09-11T22:20:19.188402+0000","last_peered":"2024-09-11T22:20:19.188402+0000","last_clean":"2024-09-11T22:20:19.188402+0000","last_became_active":"2024-09-11T22:19:02.589644+0000","last_became_peered":"2024-09-11T22:19:02.589644+0000","last_unstale":"2024-09-11T22:20:19.188402+0000","last_undegraded":"2024-09-11T22:20:19.188402+0000","last_fullsized":"2024-09-11T22:20:19.188402+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_clean_scrub_stamp":"2024-09-11T22:15:33.244685+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:45.506190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:30.269896+0000","last_change":"2024-09-11T22:20:16.116661+0000","last_active":"2024-09-11T22:20:30.269896+0000","last_peered":"2024-09-11T22:20:30.269896+0000","last_clean":"2024-09-11T22:20:30.269896+0000","last_became_active":"2024-09-11T22:20:16.116196+0000","last_became_peered":"2024-09-11T22:20:16.116196+0000","last_unstale":"2024-09-11T22:20:30.269896+0000","last_undegraded":"2024-09-11T22:20:30.269896+0000","last_fullsized":"2024-09-11T22:20:30.269896+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:08.875170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180451+0000","last_change":"2024-09-11T22:20:18.193176+0000","last_active":"2024-09-11T22:20:19.180451+0000","last_peered":"2024-09-11T22:20:19.180451+0000","last_clean":"2024-09-11T22:20:19.180451+0000","last_became_active":"2024-09-11T22:20:18.192312+0000","last_became_peered":"2024-09-11T22:20:18.192312+0000","last_unstale":"2024-09-11T22:20:19.180451+0000","last_undegraded":"2024-09-11T22:20:19.180451+0000","last_fullsized":"2024-09-11T22:20:19.180451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:08.738569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:27.253672+0000","last_change":"2024-09-11T22:20:16.123772+0000","last_active":"2024-09-11T22:20:27.253672+0000","last_peered":"2024-09-11T22:20:27.253672+0000","last_clean":"2024-09-11T22:20:27.253672+0000","last_became_active":"2024-09-11T22:20:16.123403+0000","last_became_peered":"2024-09-11T22:20:16.123403+0000","last_unstale":"2024-09-11T22:20:27.253672+0000","last_undegraded":"2024-09-11T22:20:27.253672+0000","last_fullsized":"2024-09-11T22:20:27.253672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:24.741520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.186820+0000","last_change":"2024-09-11T22:20:18.191118+0000","last_active":"2024-09-11T22:20:19.186820+0000","last_peered":"2024-09-11T22:20:19.186820+0000","last_clean":"2024-09-11T22:20:19.186820+0000","last_became_active":"2024-09-11T22:20:18.190760+0000","last_became_peered":"2024-09-11T22:20:18.190760+0000","last_unstale":"2024-09-11T22:20:19.186820+0000","last_undegraded":"2024-09-11T22:20:19.186820+0000","last_fullsized":"2024-09-11T22:20:19.186820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:59.220067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:40.286284+0000","last_change":"2024-09-11T22:20:16.186407+0000","last_active":"2024-09-11T22:20:40.286284+0000","last_peered":"2024-09-11T22:20:40.286284+0000","last_clean":"2024-09-11T22:20:40.286284+0000","last_became_active":"2024-09-11T22:20:16.186105+0000","last_became_peered":"2024-09-11T22:20:16.186105+0000","last_unstale":"2024-09-11T22:20:40.286284+0000","last_undegraded":"2024-09-11T22:20:40.286284+0000","last_fullsized":"2024-09-11T22:20:40.286284+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:43.942072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.182722+0000","last_change":"2024-09-11T22:20:18.191970+0000","last_active":"2024-09-11T22:20:19.182722+0000","last_peered":"2024-09-11T22:20:19.182722+0000","last_clean":"2024-09-11T22:20:19.182722+0000","last_became_active":"2024-09-11T22:20:18.191623+0000","last_became_peered":"2024-09-11T22:20:18.191623+0000","last_unstale":"2024-09-11T22:20:19.182722+0000","last_undegraded":"2024-09-11T22:20:19.182722+0000","last_fullsized":"2024-09-11T22:20:19.182722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:33.213979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:40.286467+0000","last_change":"2024-09-11T22:20:16.119936+0000","last_active":"2024-09-11T22:20:40.286467+0000","last_peered":"2024-09-11T22:20:40.286467+0000","last_clean":"2024-09-11T22:20:40.286467+0000","last_became_active":"2024-09-11T22:20:16.119453+0000","last_became_peered":"2024-09-11T22:20:16.119453+0000","last_unstale":"2024-09-11T22:20:40.286467+0000","last_undegraded":"2024-09-11T22:20:40.286467+0000","last_fullsized":"2024-09-11T22:20:40.286467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:20.717980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.181111+0000","last_change":"2024-09-11T22:20:18.192581+0000","last_active":"2024-09-11T22:20:19.181111+0000","last_peered":"2024-09-11T22:20:19.181111+0000","last_clean":"2024-09-11T22:20:19.181111+0000","last_became_active":"2024-09-11T22:20:18.191702+0000","last_became_peered":"2024-09-11T22:20:18.191702+0000","last_unstale":"2024-09-11T22:20:19.181111+0000","last_undegraded":"2024-09-11T22:20:19.181111+0000","last_fullsized":"2024-09-11T22:20:19.181111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:48.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.191997+0000","last_change":"2024-09-11T22:20:16.186485+0000","last_active":"2024-09-11T22:20:19.191997+0000","last_peered":"2024-09-11T22:20:19.191997+0000","last_clean":"2024-09-11T22:20:19.191997+0000","last_became_active":"2024-09-11T22:20:16.186171+0000","last_became_peered":"2024-09-11T22:20:16.186171+0000","last_unstale":"2024-09-11T22:20:19.191997+0000","last_undegraded":"2024-09-11T22:20:19.191997+0000","last_fullsized":"2024-09-11T22:20:19.191997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:26.373626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:20.263658+0000","last_change":"2024-09-11T22:20:18.191361+0000","last_active":"2024-09-11T22:20:20.263658+0000","last_peered":"2024-09-11T22:20:20.263658+0000","last_clean":"2024-09-11T22:20:20.263658+0000","last_became_active":"2024-09-11T22:20:18.191036+0000","last_became_peered":"2024-09-11T22:20:18.191036+0000","last_unstale":"2024-09-11T22:20:20.263658+0000","last_undegraded":"2024-09-11T22:20:20.263658+0000","last_fullsized":"2024-09-11T22:20:20.263658+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:29.124650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.270710+0000","last_change":"2024-09-11T22:20:16.188156+0000","last_active":"2024-09-11T22:20:29.270710+0000","last_peered":"2024-09-11T22:20:29.270710+0000","last_clean":"2024-09-11T22:20:29.270710+0000","last_became_active":"2024-09-11T22:20:16.187809+0000","last_became_peered":"2024-09-11T22:20:16.187809+0000","last_unstale":"2024-09-11T22:20:29.270710+0000","last_undegraded":"2024-09-11T22:20:29.270710+0000","last_fullsized":"2024-09-11T22:20:29.270710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:04.219621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189850+0000","last_change":"2024-09-11T22:20:18.182380+0000","last_active":"2024-09-11T22:20:19.189850+0000","last_peered":"2024-09-11T22:20:19.189850+0000","last_clean":"2024-09-11T22:20:19.189850+0000","last_became_active":"2024-09-11T22:20:18.182062+0000","last_became_peered":"2024-09-11T22:20:18.182062+0000","last_unstale":"2024-09-11T22:20:19.189850+0000","last_undegraded":"2024-09-11T22:20:19.189850+0000","last_fullsized":"2024-09-11T22:20:19.189850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:15.575846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'13","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:41.189580+0000","last_change":"2024-09-11T22:20:16.117335+0000","last_active":"2024-09-11T22:20:41.189580+0000","last_peered":"2024-09-11T22:20:41.189580+0000","last_clean":"2024-09-11T22:20:41.189580+0000","last_became_active":"2024-09-11T22:20:16.116977+0000","last_became_peered":"2024-09-11T22:20:16.116977+0000","last_unstale":"2024-09-11T22:20:41.189580+0000","last_undegraded":"2024-09-11T22:20:41.189580+0000","last_fullsized":"2024-09-11T22:20:41.189580+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:51.744202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180505+0000","last_change":"2024-09-11T22:20:18.194069+0000","last_active":"2024-09-11T22:20:19.180505+0000","last_peered":"2024-09-11T22:20:19.180505+0000","last_clean":"2024-09-11T22:20:19.180505+0000","last_became_active":"2024-09-11T22:20:18.193765+0000","last_became_peered":"2024-09-11T22:20:18.193765+0000","last_unstale":"2024-09-11T22:20:19.180505+0000","last_undegraded":"2024-09-11T22:20:19.180505+0000","last_fullsized":"2024-09-11T22:20:19.180505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:25.047405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.268944+0000","last_change":"2024-09-11T22:20:16.460987+0000","last_active":"2024-09-11T22:20:29.268944+0000","last_peered":"2024-09-11T22:20:29.268944+0000","last_clean":"2024-09-11T22:20:29.268944+0000","last_became_active":"2024-09-11T22:20:16.460516+0000","last_became_peered":"2024-09-11T22:20:16.460516+0000","last_unstale":"2024-09-11T22:20:29.268944+0000","last_undegraded":"2024-09-11T22:20:29.268944+0000","last_fullsized":"2024-09-11T22:20:29.268944+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:00.987981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189239+0000","last_change":"2024-09-11T22:20:18.189921+0000","last_active":"2024-09-11T22:20:19.189239+0000","last_peered":"2024-09-11T22:20:19.189239+0000","last_clean":"2024-09-11T22:20:19.189239+0000","last_became_active":"2024-09-11T22:20:18.189616+0000","last_became_peered":"2024-09-11T22:20:18.189616+0000","last_unstale":"2024-09-11T22:20:19.189239+0000","last_undegraded":"2024-09-11T22:20:19.189239+0000","last_fullsized":"2024-09-11T22:20:19.189239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:54:16.515211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:26.238455+0000","last_change":"2024-09-11T22:20:16.186498+0000","last_active":"2024-09-11T22:20:26.238455+0000","last_peered":"2024-09-11T22:20:26.238455+0000","last_clean":"2024-09-11T22:20:26.238455+0000","last_became_active":"2024-09-11T22:20:16.186214+0000","last_became_peered":"2024-09-11T22:20:16.186214+0000","last_unstale":"2024-09-11T22:20:26.238455+0000","last_undegraded":"2024-09-11T22:20:26.238455+0000","last_fullsized":"2024-09-11T22:20:26.238455+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:09.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187945+0000","last_change":"2024-09-11T22:20:18.187489+0000","last_active":"2024-09-11T22:20:19.187945+0000","last_peered":"2024-09-11T22:20:19.187945+0000","last_clean":"2024-09-11T22:20:19.187945+0000","last_became_active":"2024-09-11T22:20:18.185649+0000","last_became_peered":"2024-09-11T22:20:18.185649+0000","last_unstale":"2024-09-11T22:20:19.187945+0000","last_undegraded":"2024-09-11T22:20:19.187945+0000","last_fullsized":"2024-09-11T22:20:19.187945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:42.825075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.270150+0000","last_change":"2024-09-11T22:20:16.117627+0000","last_active":"2024-09-11T22:20:29.270150+0000","last_peered":"2024-09-11T22:20:29.270150+0000","last_clean":"2024-09-11T22:20:29.270150+0000","last_became_active":"2024-09-11T22:20:16.117243+0000","last_became_peered":"2024-09-11T22:20:16.117243+0000","last_unstale":"2024-09-11T22:20:29.270150+0000","last_undegraded":"2024-09-11T22:20:29.270150+0000","last_fullsized":"2024-09-11T22:20:29.270150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:58.240535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.195464+0000","last_change":"2024-09-11T22:20:18.184326+0000","last_active":"2024-09-11T22:20:19.195464+0000","last_peered":"2024-09-11T22:20:19.195464+0000","last_clean":"2024-09-11T22:20:19.195464+0000","last_became_active":"2024-09-11T22:20:18.183995+0000","last_became_peered":"2024-09-11T22:20:18.183995+0000","last_unstale":"2024-09-11T22:20:19.195464+0000","last_undegraded":"2024-09-11T22:20:19.195464+0000","last_fullsized":"2024-09-11T22:20:19.195464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:33.468000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:28.258102+0000","last_change":"2024-09-11T22:20:16.127180+0000","last_active":"2024-09-11T22:20:28.258102+0000","last_peered":"2024-09-11T22:20:28.258102+0000","last_clean":"2024-09-11T22:20:28.258102+0000","last_became_active":"2024-09-11T22:20:16.126696+0000","last_became_peered":"2024-09-11T22:20:16.126696+0000","last_unstale":"2024-09-11T22:20:28.258102+0000","last_undegraded":"2024-09-11T22:20:28.258102+0000","last_fullsized":"2024-09-11T22:20:28.258102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:02.637225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180611+0000","last_change":"2024-09-11T22:20:18.185635+0000","last_active":"2024-09-11T22:20:19.180611+0000","last_peered":"2024-09-11T22:20:19.180611+0000","last_clean":"2024-09-11T22:20:19.180611+0000","last_became_active":"2024-09-11T22:20:18.185268+0000","last_became_peered":"2024-09-11T22:20:18.185268+0000","last_unstale":"2024-09-11T22:20:19.180611+0000","last_undegraded":"2024-09-11T22:20:19.180611+0000","last_fullsized":"2024-09-11T22:20:19.180611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:54:15.996231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:26.240372+0000","last_change":"2024-09-11T22:20:16.128311+0000","last_active":"2024-09-11T22:20:26.240372+0000","last_peered":"2024-09-11T22:20:26.240372+0000","last_clean":"2024-09-11T22:20:26.240372+0000","last_became_active":"2024-09-11T22:20:16.127853+0000","last_became_peered":"2024-09-11T22:20:16.127853+0000","last_unstale":"2024-09-11T22:20:26.240372+0000","last_undegraded":"2024-09-11T22:20:26.240372+0000","last_fullsized":"2024-09-11T22:20:26.240372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:57.562470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.189917+0000","last_change":"2024-09-11T22:20:18.195782+0000","last_active":"2024-09-11T22:20:19.189917+0000","last_peered":"2024-09-11T22:20:19.189917+0000","last_clean":"2024-09-11T22:20:19.189917+0000","last_became_active":"2024-09-11T22:20:18.195365+0000","last_became_peered":"2024-09-11T22:20:18.195365+0000","last_unstale":"2024-09-11T22:20:19.189917+0000","last_undegraded":"2024-09-11T22:20:19.189917+0000","last_fullsized":"2024-09-11T22:20:19.189917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:49.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.183116+0000","last_change":"2024-09-11T22:20:18.199344+0000","last_active":"2024-09-11T22:20:19.183116+0000","last_peered":"2024-09-11T22:20:19.183116+0000","last_clean":"2024-09-11T22:20:19.183116+0000","last_became_active":"2024-09-11T22:20:18.199016+0000","last_became_peered":"2024-09-11T22:20:18.199016+0000","last_unstale":"2024-09-11T22:20:19.183116+0000","last_undegraded":"2024-09-11T22:20:19.183116+0000","last_fullsized":"2024-09-11T22:20:19.183116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:38.322831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:29.271737+0000","last_change":"2024-09-11T22:20:16.122236+0000","last_active":"2024-09-11T22:20:29.271737+0000","last_peered":"2024-09-11T22:20:29.271737+0000","last_clean":"2024-09-11T22:20:29.271737+0000","last_became_active":"2024-09-11T22:20:16.121867+0000","last_became_peered":"2024-09-11T22:20:16.121867+0000","last_unstale":"2024-09-11T22:20:29.271737+0000","last_undegraded":"2024-09-11T22:20:29.271737+0000","last_fullsized":"2024-09-11T22:20:29.271737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:56.782445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187997+0000","last_change":"2024-09-11T22:20:18.187338+0000","last_active":"2024-09-11T22:20:19.187997+0000","last_peered":"2024-09-11T22:20:19.187997+0000","last_clean":"2024-09-11T22:20:19.187997+0000","last_became_active":"2024-09-11T22:20:18.186985+0000","last_became_peered":"2024-09-11T22:20:18.186985+0000","last_unstale":"2024-09-11T22:20:19.187997+0000","last_undegraded":"2024-09-11T22:20:19.187997+0000","last_fullsized":"2024-09-11T22:20:19.187997+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:58.170680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:30.266351+0000","last_change":"2024-09-11T22:20:16.128579+0000","last_active":"2024-09-11T22:20:30.266351+0000","last_peered":"2024-09-11T22:20:30.266351+0000","last_clean":"2024-09-11T22:20:30.266351+0000","last_became_active":"2024-09-11T22:20:16.127864+0000","last_became_peered":"2024-09-11T22:20:16.127864+0000","last_unstale":"2024-09-11T22:20:30.266351+0000","last_undegraded":"2024-09-11T22:20:30.266351+0000","last_fullsized":"2024-09-11T22:20:30.266351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:11:32.005487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.187753+0000","last_change":"2024-09-11T22:20:18.185205+0000","last_active":"2024-09-11T22:20:19.187753+0000","last_peered":"2024-09-11T22:20:19.187753+0000","last_clean":"2024-09-11T22:20:19.187753+0000","last_became_active":"2024-09-11T22:20:18.184402+0000","last_became_peered":"2024-09-11T22:20:18.184402+0000","last_unstale":"2024-09-11T22:20:19.187753+0000","last_undegraded":"2024-09-11T22:20:19.187753+0000","last_fullsized":"2024-09-11T22:20:19.187753+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:17:48.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.194010+0000","last_change":"2024-09-11T22:20:16.186372+0000","last_active":"2024-09-11T22:20:19.194010+0000","last_peered":"2024-09-11T22:20:19.194010+0000","last_clean":"2024-09-11T22:20:19.194010+0000","last_became_active":"2024-09-11T22:20:16.185990+0000","last_became_peered":"2024-09-11T22:20:16.185990+0000","last_unstale":"2024-09-11T22:20:19.194010+0000","last_undegraded":"2024-09-11T22:20:19.194010+0000","last_fullsized":"2024-09-11T22:20:19.194010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:23.296379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:19.180237+0000","last_change":"2024-09-11T22:20:18.190670+0000","last_active":"2024-09-11T22:20:19.180237+0000","last_peered":"2024-09-11T22:20:19.180237+0000","last_clean":"2024-09-11T22:20:19.180237+0000","last_became_active":"2024-09-11T22:20:18.190339+0000","last_became_peered":"2024-09-11T22:20:18.190339+0000","last_unstale":"2024-09-11T22:20:19.180237+0000","last_undegraded":"2024-09-11T22:20:19.180237+0000","last_fullsized":"2024-09-11T22:20:19.180237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.085305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-11T22:20:42.288287+0000","last_change":"2024-09-11T22:20:16.117646+0000","last_active":"2024-09-11T22:20:42.288287+0000","last_peered":"2024-09-11T22:20:42.288287+0000","last_clean":"2024-09-11T22:20:42.288287+0000","last_became_active":"2024-09-11T22:20:16.117111+0000","last_became_peered":"2024-09-11T22:20:16.117111+0000","last_unstale":"2024-09-11T22:20:42.288287+0000","last_undegraded":"2024-09-11T22:20:42.288287+0000","last_fullsized":"2024-09-11T22:20:42.288287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:36.470158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53617,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":149,"num_read_kb":135,"num_write":233,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":160851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249094,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":872448,"data_stored":417048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":1054879,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972380,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":416978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201313,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":417356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430247,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":1009876,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":417088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822704,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":1009161,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986229,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":416556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215163,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":416520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":464630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":929792,"data_stored":464803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40384,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704768,"statfs":{"total":95995035648,"available":95953682432,"internally_reserved":0,"allocated":901120,"data_stored":418034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:20:46.854 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T22:20:46.854 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T22:20:46.855 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T22:20:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:47.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:47 smithi005 ceph-mon[30943]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:20:47.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:47 smithi159 ceph-mon[35534]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:20:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:47 smithi172 ceph-mon[35461]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:20:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.629+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2401827341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cdc102e00 msgr2=0x7f1cdc103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.629+0000 7f1ce43c9640 1 --2- 172.21.15.5:0/2401827341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cdc102e00 0x7f1cdc103280 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009900 tx=0x7f1ccc02a410 comp rx=0 tx=0).stop 2024-09-11T22:20:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.629+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2401827341 shutdown_connections 2024-09-11T22:20:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.629+0000 7f1ce43c9640 1 --2- 172.21.15.5:0/2401827341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cdc1037c0 0x7f1cdc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.629+0000 7f1ce43c9640 1 --2- 172.21.15.5:0/2401827341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cdc102e00 0x7f1cdc103280 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.629+0000 7f1ce43c9640 1 --2- 172.21.15.5:0/2401827341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cdc101c00 0x7f1cdc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.629+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2401827341 >> 172.21.15.5:0/2401827341 conn(0x7f1cdc0fd390 msgr2=0x7f1cdc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.629+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2401827341 shutdown_connections 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.629+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2401827341 wait complete. 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce43c9640 1 Processor -- start 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce43c9640 1 -- start start 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce43c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cdc101c00 0x7f1cdc197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce43c9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cdc102e00 0x7f1cdc198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce43c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cdc1037c0 0x7f1cdc19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce43c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc10d860 con 0x7f1cdc1037c0 2024-09-11T22:20:47.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce43c9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc10d6e0 con 0x7f1cdc102e00 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce43c9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc10d9e0 con 0x7f1cdc101c00 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce2966640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cdc1037c0 0x7f1cdc19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce2966640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cdc1037c0 0x7f1cdc19f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42978/0 (socket says 172.21.15.5:42978) 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.630+0000 7f1ce2966640 1 -- 172.21.15.5:0/2785483255 learned_addr learned my addr 172.21.15.5:0/2785483255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce1964640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cdc102e00 0x7f1cdc198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce2165640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cdc101c00 0x7f1cdc197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce2966640 1 -- 172.21.15.5:0/2785483255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cdc101c00 msgr2=0x7f1cdc197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce2966640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cdc101c00 0x7f1cdc197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce2966640 1 -- 172.21.15.5:0/2785483255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cdc102e00 msgr2=0x7f1cdc198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce2966640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cdc102e00 0x7f1cdc198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce2966640 1 -- 172.21.15.5:0/2785483255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc19fa80 con 0x7f1cdc1037c0 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce1964640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cdc102e00 0x7f1cdc198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce2966640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cdc1037c0 0x7f1cdc19f410 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400eaa0 tx=0x7f1cd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:47.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2785483255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400cdb0 con 0x7f1cdc1037c0 2024-09-11T22:20:47.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2785483255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cdc19fd70 con 0x7f1cdc1037c0 2024-09-11T22:20:47.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2785483255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cd4005020 con 0x7f1cdc1037c0 2024-09-11T22:20:47.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2785483255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40116b0 con 0x7f1cdc1037c0 2024-09-11T22:20:47.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.631+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2785483255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cdc1a01d0 con 0x7f1cdc1037c0 2024-09-11T22:20:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.633+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2785483255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cdc108fd0 con 0x7f1cdc1037c0 2024-09-11T22:20:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.634+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2785483255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1cd4005560 con 0x7f1cdc1037c0 2024-09-11T22:20:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.634+0000 7f1cd37fe640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1cc80a2c40 0x7f1cc80a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.635+0000 7f1ce2165640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1cc80a2c40 0x7f1cc80a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.635+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2785483255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1cd40cf780 con 0x7f1cdc1037c0 2024-09-11T22:20:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.635+0000 7f1ce2165640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1cc80a2c40 0x7f1cc80a5100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80058c0 tx=0x7f1cd8005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.638+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2785483255 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1cd4016030 con 0x7f1cdc1037c0 2024-09-11T22:20:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.882+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2785483255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f1cdc02d7c0 con 0x7f1cdc1037c0 2024-09-11T22:20:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.884+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2785483255 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f1cd409c630 con 0x7f1cdc1037c0 2024-09-11T22:20:47.886 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-09-11T22:20:47.886 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQC/F+Jm06O1NBAAArNfSr1FcF6OPyFgxU4FXg== 2024-09-11T22:20:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2785483255 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1cc80a2c40 msgr2=0x7f1cc80a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1cc80a2c40 0x7f1cc80a5100 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80058c0 tx=0x7f1cd8005850 comp rx=0 tx=0).stop 2024-09-11T22:20:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2785483255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cdc1037c0 msgr2=0x7f1cdc19f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cdc1037c0 0x7f1cdc19f410 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400eaa0 tx=0x7f1cd400ef60 comp rx=0 tx=0).stop 2024-09-11T22:20:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2785483255 shutdown_connections 2024-09-11T22:20:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1cc80a2c40 0x7f1cc80a5100 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cdc1037c0 0x7f1cdc19f410 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cdc102e00 0x7f1cdc198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 --2- 172.21.15.5:0/2785483255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cdc101c00 0x7f1cdc197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2785483255 >> 172.21.15.5:0/2785483255 conn(0x7f1cdc0fd390 msgr2=0x7f1cdc1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2785483255 shutdown_connections 2024-09-11T22:20:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:47.886+0000 7f1ce43c9640 1 -- 172.21.15.5:0/2785483255 wait complete. 2024-09-11T22:20:48.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:48 smithi005 ceph-mon[30943]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 15 op/s 2024-09-11T22:20:48.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:48 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2785483255' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T22:20:48.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:48 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2785483255' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T22:20:48.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:48 smithi159 ceph-mon[35534]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 15 op/s 2024-09-11T22:20:48.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:48 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2785483255' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T22:20:48.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:48 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2785483255' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T22:20:48.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:48 smithi172 ceph-mon[35461]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.4 KiB/s wr, 15 op/s 2024-09-11T22:20:48.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:48 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2785483255' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T22:20:48.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:48 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2785483255' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T22:20:48.709 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:20:48.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T22:20:48.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T22:20:48.748 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T22:20:48.749 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T22:20:48.749 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T22:20:49.046 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.541+0000 7fd3d4f38640 1 -- 172.21.15.5:0/4178592644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d0101de0 msgr2=0x7fd3d01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.541+0000 7fd3d4f38640 1 --2- 172.21.15.5:0/4178592644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d0101de0 0x7fd3d01021c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009900 tx=0x7fd3c002a410 comp rx=0 tx=0).stop 2024-09-11T22:20:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.541+0000 7fd3d4f38640 1 -- 172.21.15.5:0/4178592644 shutdown_connections 2024-09-11T22:20:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.541+0000 7fd3d4f38640 1 --2- 172.21.15.5:0/4178592644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0106f10 0x7fd3d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.541+0000 7fd3d4f38640 1 --2- 172.21.15.5:0/4178592644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3d0102790 0x7fd3d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.541+0000 7fd3d4f38640 1 --2- 172.21.15.5:0/4178592644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d0101de0 0x7fd3d01021c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.541+0000 7fd3d4f38640 1 -- 172.21.15.5:0/4178592644 >> 172.21.15.5:0/4178592644 conn(0x7fd3d00fd8c0 msgr2=0x7fd3d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.542+0000 7fd3d4f38640 1 -- 172.21.15.5:0/4178592644 shutdown_connections 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.542+0000 7fd3d4f38640 1 -- 172.21.15.5:0/4178592644 wait complete. 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.542+0000 7fd3d4f38640 1 Processor -- start 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.542+0000 7fd3d4f38640 1 -- start start 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.542+0000 7fd3d4f38640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 0x7fd3d0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.542+0000 7fd3d4f38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3d0102790 0x7fd3d0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3d4f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d0106f10 0x7fd3d019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3d4f38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d0111f70 con 0x7fd3d0106f10 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3d4f38640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d0111df0 con 0x7fd3d0101de0 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3d4f38640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d01120f0 con 0x7fd3d0102790 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d0106f10 0x7fd3d019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d0106f10 0x7fd3d019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42998/0 (socket says 172.21.15.5:42998) 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3ced9d640 1 -- 172.21.15.5:0/2295648184 learned_addr learned my addr 172.21.15.5:0/2295648184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3ce59c640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 0x7fd3d0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3cdd9b640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3d0102790 0x7fd3d0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3ced9d640 1 -- 172.21.15.5:0/2295648184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3d0102790 msgr2=0x7fd3d0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3ced9d640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3d0102790 0x7fd3d0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3ced9d640 1 -- 172.21.15.5:0/2295648184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 msgr2=0x7fd3d0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3ced9d640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 0x7fd3d0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3ced9d640 1 -- 172.21.15.5:0/2295648184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d019d3e0 con 0x7fd3d0106f10 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3ced9d640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d0106f10 0x7fd3d019cd00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009fd0 tx=0x7fd3c800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/2295648184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8017070 con 0x7fd3d0106f10 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/2295648184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3c8003e50 con 0x7fd3d0106f10 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3d4f38640 1 -- 172.21.15.5:0/2295648184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d019d6d0 con 0x7fd3d0106f10 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/2295648184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80106c0 con 0x7fd3d0106f10 2024-09-11T22:20:49.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.543+0000 7fd3d4f38640 1 -- 172.21.15.5:0/2295648184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d01a4fb0 con 0x7fd3d0106f10 2024-09-11T22:20:49.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.544+0000 7fd3d4f38640 1 -- 172.21.15.5:0/2295648184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d01021c0 con 0x7fd3d0106f10 2024-09-11T22:20:49.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.545+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/2295648184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd3c8003970 con 0x7fd3d0106f10 2024-09-11T22:20:49.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.546+0000 7fd3bf7fe640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd3b80a2c90 0x7fd3b80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:49.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.547+0000 7fd3ce59c640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd3b80a2c90 0x7fd3b80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:49.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.547+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/2295648184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd3c80cecc0 con 0x7fd3d0106f10 2024-09-11T22:20:49.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.547+0000 7fd3ce59c640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd3b80a2c90 0x7fd3b80a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009230 tx=0x7fd3c00058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:49.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.551+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/2295648184 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3c809bc70 con 0x7fd3d0106f10 2024-09-11T22:20:49.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.774+0000 7fd3d4f38640 1 -- 172.21.15.5:0/2295648184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd3d010a0f0 con 0x7fd3d0106f10 2024-09-11T22:20:49.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.779+0000 7fd3bf7fe640 1 -- 172.21.15.5:0/2295648184 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263191 (secure 0 0 0) 0x7fd3c80bb070 con 0x7fd3d0106f10 2024-09-11T22:20:49.784 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 -- 172.21.15.5:0/2295648184 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd3b80a2c90 msgr2=0x7fd3b80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd3b80a2c90 0x7fd3b80a5150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0009230 tx=0x7fd3c00058d0 comp rx=0 tx=0).stop 2024-09-11T22:20:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 -- 172.21.15.5:0/2295648184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d0106f10 msgr2=0x7fd3d019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d0106f10 0x7fd3d019cd00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009fd0 tx=0x7fd3c800b260 comp rx=0 tx=0).stop 2024-09-11T22:20:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 -- 172.21.15.5:0/2295648184 shutdown_connections 2024-09-11T22:20:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd3b80a2c90 0x7fd3b80a5150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3d0106f10 0x7fd3d019cd00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3d0102790 0x7fd3d0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 --2- 172.21.15.5:0/2295648184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3d0101de0 0x7fd3d0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 -- 172.21.15.5:0/2295648184 >> 172.21.15.5:0/2295648184 conn(0x7fd3d00fd8c0 msgr2=0x7fd3d00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 -- 172.21.15.5:0/2295648184 shutdown_connections 2024-09-11T22:20:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:49.782+0000 7fd3d4f38640 1 -- 172.21.15.5:0/2295648184 wait complete. 2024-09-11T22:20:50.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:50 smithi005 ceph-mon[30943]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-11T22:20:50.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:50 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2295648184' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T22:20:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:50 smithi172 ceph-mon[35461]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-11T22:20:50.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:50 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2295648184' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T22:20:50.639 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3591749770},{"type":"v1","addr":"172.21.15.5:6801","nonce":3591749770}]},"active_addr":"172.21.15.5:6801/3591749770","active_change":"2024-09-11T22:13:16.336866+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":785116270}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1577283141}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1618823752}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2380674497}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2274091215}]}]} 2024-09-11T22:20:50.645 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T22:20:50.645 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T22:20:50.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:20:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:50 smithi159 ceph-mon[35534]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-11T22:20:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:50 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2295648184' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T22:20:50.907 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:51.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.405+0000 7ff0b4484640 1 -- 172.21.15.5:0/3835847177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ac101de0 msgr2=0x7ff0ac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:51.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.405+0000 7ff0b4484640 1 --2- 172.21.15.5:0/3835847177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ac101de0 0x7ff0ac1021c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8009900 tx=0x7ff0a802a410 comp rx=0 tx=0).stop 2024-09-11T22:20:51.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.405+0000 7ff0b4484640 1 -- 172.21.15.5:0/3835847177 shutdown_connections 2024-09-11T22:20:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.405+0000 7ff0b4484640 1 --2- 172.21.15.5:0/3835847177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac106f10 0x7ff0ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.405+0000 7ff0b4484640 1 --2- 172.21.15.5:0/3835847177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac102790 0x7ff0ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.405+0000 7ff0b4484640 1 --2- 172.21.15.5:0/3835847177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ac101de0 0x7ff0ac1021c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.405+0000 7ff0b4484640 1 -- 172.21.15.5:0/3835847177 >> 172.21.15.5:0/3835847177 conn(0x7ff0ac0fd8c0 msgr2=0x7ff0ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.405+0000 7ff0b4484640 1 -- 172.21.15.5:0/3835847177 shutdown_connections 2024-09-11T22:20:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.405+0000 7ff0b4484640 1 -- 172.21.15.5:0/3835847177 wait complete. 2024-09-11T22:20:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b4484640 1 Processor -- start 2024-09-11T22:20:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b4484640 1 -- start start 2024-09-11T22:20:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b4484640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac101de0 0x7ff0ac1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b4484640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac102790 0x7ff0ac198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b4484640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ac106f10 0x7ff0ac19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b4484640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac111f90 con 0x7ff0ac106f10 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b4484640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac111e10 con 0x7ff0ac101de0 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b4484640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac112110 con 0x7ff0ac102790 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b2a21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ac106f10 0x7ff0ac19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b2a21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ac106f10 0x7ff0ac19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57200/0 (socket says 172.21.15.5:57200) 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b2a21640 1 -- 172.21.15.5:0/3819850276 learned_addr learned my addr 172.21.15.5:0/3819850276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b1a1f640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac102790 0x7ff0ac198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b2a21640 1 -- 172.21.15.5:0/3819850276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac102790 msgr2=0x7ff0ac198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b2a21640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac102790 0x7ff0ac198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b2a21640 1 -- 172.21.15.5:0/3819850276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac101de0 msgr2=0x7ff0ac1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b2a21640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac101de0 0x7ff0ac1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b2a21640 1 -- 172.21.15.5:0/3819850276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac19d380 con 0x7ff0ac106f10 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b1a1f640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac102790 0x7ff0ac198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.406+0000 7ff0b2a21640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ac106f10 0x7ff0ac19cca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400b970 tx=0x7ff0a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.407+0000 7ff0a37fe640 1 -- 172.21.15.5:0/3819850276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4019070 con 0x7ff0ac106f10 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.407+0000 7ff0b4484640 1 -- 172.21.15.5:0/3819850276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac19d670 con 0x7ff0ac106f10 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.407+0000 7ff0a37fe640 1 -- 172.21.15.5:0/3819850276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a400cb40 con 0x7ff0ac106f10 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.407+0000 7ff0a37fe640 1 -- 172.21.15.5:0/3819850276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4007650 con 0x7ff0ac106f10 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.407+0000 7ff0b4484640 1 -- 172.21.15.5:0/3819850276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac1a4f50 con 0x7ff0ac106f10 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.408+0000 7ff0b4484640 1 -- 172.21.15.5:0/3819850276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ac1021c0 con 0x7ff0ac106f10 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.409+0000 7ff0a37fe640 1 -- 172.21.15.5:0/3819850276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff0a401c070 con 0x7ff0ac106f10 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.409+0000 7ff0a37fe640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff0980a2c10 0x7ff0980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.410+0000 7ff0a37fe640 1 -- 172.21.15.5:0/3819850276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff0a40ce5b0 con 0x7ff0ac106f10 2024-09-11T22:20:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.410+0000 7ff0b2220640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff0980a2c10 0x7ff0980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.410+0000 7ff0b2220640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff0980a2c10 0x7ff0980a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff0a802a930 tx=0x7ff0a8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.414+0000 7ff0a37fe640 1 -- 172.21.15.5:0/3819850276 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0a409b670 con 0x7ff0ac106f10 2024-09-11T22:20:51.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.590+0000 7ff0b4484640 1 -- 172.21.15.5:0/3819850276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0ac10a120 con 0x7ff0ac106f10 2024-09-11T22:20:51.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.592+0000 7ff0a37fe640 1 -- 172.21.15.5:0/3819850276 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22173 (secure 0 0 0) 0x7ff0a409af70 con 0x7ff0ac106f10 2024-09-11T22:20:51.593 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:51.593 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":80,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:20:18.900590+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:20:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 -- 172.21.15.5:0/3819850276 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff0980a2c10 msgr2=0x7ff0980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff0980a2c10 0x7ff0980a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff0a802a930 tx=0x7ff0a8000bc0 comp rx=0 tx=0).stop 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 -- 172.21.15.5:0/3819850276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ac106f10 msgr2=0x7ff0ac19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ac106f10 0x7ff0ac19cca0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400b970 tx=0x7ff0a400be30 comp rx=0 tx=0).stop 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 -- 172.21.15.5:0/3819850276 shutdown_connections 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff0980a2c10 0x7ff0980a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ac106f10 0x7ff0ac19cca0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac102790 0x7ff0ac198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 --2- 172.21.15.5:0/3819850276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0ac101de0 0x7ff0ac1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 -- 172.21.15.5:0/3819850276 >> 172.21.15.5:0/3819850276 conn(0x7ff0ac0fd8c0 msgr2=0x7ff0ac0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 -- 172.21.15.5:0/3819850276 shutdown_connections 2024-09-11T22:20:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:51.595+0000 7ff0b4484640 1 -- 172.21.15.5:0/3819850276 wait complete. 2024-09-11T22:20:52.300 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T22:20:52.300 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:20:52.556 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:52.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:52 smithi172 ceph-mon[35461]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-11T22:20:52.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:52 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3819850276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:52.678 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:52 smithi005 ceph-mon[30943]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-11T22:20:52.679 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:52 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3819850276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:52.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:52 smithi159 ceph-mon[35534]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-09-11T22:20:52.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:52 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3819850276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:53.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.030+0000 7f133613d640 1 -- 172.21.15.5:0/1962940495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1330102ef0 msgr2=0x7f133010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:53.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.030+0000 7f133613d640 1 --2- 172.21.15.5:0/1962940495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1330102ef0 0x7f133010da40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1328009900 tx=0x7f132802a480 comp rx=0 tx=0).stop 2024-09-11T22:20:53.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.031+0000 7f133613d640 1 -- 172.21.15.5:0/1962940495 shutdown_connections 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.031+0000 7f133613d640 1 --2- 172.21.15.5:0/1962940495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1330102ef0 0x7f133010da40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.031+0000 7f133613d640 1 --2- 172.21.15.5:0/1962940495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1330102570 0x7f13301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.031+0000 7f133613d640 1 --2- 172.21.15.5:0/1962940495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330100e20 0x7f1330101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.031+0000 7f133613d640 1 -- 172.21.15.5:0/1962940495 >> 172.21.15.5:0/1962940495 conn(0x7f13300fc7d0 msgr2=0x7f13300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.031+0000 7f133613d640 1 -- 172.21.15.5:0/1962940495 shutdown_connections 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.031+0000 7f133613d640 1 -- 172.21.15.5:0/1962940495 wait complete. 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.031+0000 7f133613d640 1 Processor -- start 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.031+0000 7f133613d640 1 -- start start 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f133613d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1330100e20 0x7f13301985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f133613d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1330102570 0x7f1330198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f133613d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330102ef0 0x7f133019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f133613d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1330111020 con 0x7f1330100e20 2024-09-11T22:20:53.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f133613d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1330110ea0 con 0x7f1330102570 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f133613d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13301111a0 con 0x7f1330102ef0 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1330100e20 0x7f13301985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1330100e20 0x7f13301985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57214/0 (socket says 172.21.15.5:57214) 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132f7fe640 1 -- 172.21.15.5:0/3522992187 learned_addr learned my addr 172.21.15.5:0/3522992187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132effd640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1330102570 0x7f1330198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132ffff640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330102ef0 0x7f133019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132f7fe640 1 -- 172.21.15.5:0/3522992187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330102ef0 msgr2=0x7f133019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132f7fe640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330102ef0 0x7f133019ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132f7fe640 1 -- 172.21.15.5:0/3522992187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1330102570 msgr2=0x7f1330198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132f7fe640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1330102570 0x7f1330198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132f7fe640 1 -- 172.21.15.5:0/3522992187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133019d590 con 0x7f1330100e20 2024-09-11T22:20:53.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132f7fe640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1330100e20 0x7f13301985e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f132000ba50 tx=0x7f132000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132cff9640 1 -- 172.21.15.5:0/3522992187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320019070 con 0x7f1330100e20 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f133613d640 1 -- 172.21.15.5:0/3522992187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133019d880 con 0x7f1330100e20 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132cff9640 1 -- 172.21.15.5:0/3522992187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f132000cbe0 con 0x7f1330100e20 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f132cff9640 1 -- 172.21.15.5:0/3522992187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13200076e0 con 0x7f1330100e20 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.032+0000 7f133613d640 1 -- 172.21.15.5:0/3522992187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13301a5160 con 0x7f1330100e20 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.033+0000 7f133613d640 1 -- 172.21.15.5:0/3522992187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1330101280 con 0x7f1330100e20 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.035+0000 7f132cff9640 1 -- 172.21.15.5:0/3522992187 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f132001c070 con 0x7f1330100e20 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.035+0000 7f132cff9640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f131c0a2c10 0x7f131c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.035+0000 7f132effd640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f131c0a2c10 0x7f131c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.036+0000 7f132cff9640 1 -- 172.21.15.5:0/3522992187 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f13200ce570 con 0x7f1330100e20 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.036+0000 7f132effd640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f131c0a2c10 0x7f131c0a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f13300ff060 tx=0x7f1318008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:53.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.040+0000 7f132cff9640 1 -- 172.21.15.5:0/3522992187 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f132009b630 con 0x7f1330100e20 2024-09-11T22:20:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.216+0000 7f133613d640 1 -- 172.21.15.5:0/3522992187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f133010a0a0 con 0x7f1330100e20 2024-09-11T22:20:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.218+0000 7f132cff9640 1 -- 172.21.15.5:0/3522992187 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22173 (secure 0 0 0) 0x7f132009af30 con 0x7f1330100e20 2024-09-11T22:20:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:20:53.219 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":80,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:20:18.900590+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:20:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.220+0000 7f133613d640 1 -- 172.21.15.5:0/3522992187 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f131c0a2c10 msgr2=0x7f131c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.220+0000 7f133613d640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f131c0a2c10 0x7f131c0a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f13300ff060 tx=0x7f1318008040 comp rx=0 tx=0).stop 2024-09-11T22:20:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.220+0000 7f133613d640 1 -- 172.21.15.5:0/3522992187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1330100e20 msgr2=0x7f13301985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.220+0000 7f133613d640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1330100e20 0x7f13301985e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f132000ba50 tx=0x7f132000bf10 comp rx=0 tx=0).stop 2024-09-11T22:20:53.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.220+0000 7f133613d640 1 -- 172.21.15.5:0/3522992187 shutdown_connections 2024-09-11T22:20:53.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.220+0000 7f133613d640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f131c0a2c10 0x7f131c0a50d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:53.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.220+0000 7f133613d640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1330102ef0 0x7f133019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:53.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.220+0000 7f133613d640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1330102570 0x7f1330198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:53.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.220+0000 7f133613d640 1 --2- 172.21.15.5:0/3522992187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1330100e20 0x7f13301985e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:53.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.220+0000 7f133613d640 1 -- 172.21.15.5:0/3522992187 >> 172.21.15.5:0/3522992187 conn(0x7f13300fc7d0 msgr2=0x7f133006a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:53.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.221+0000 7f133613d640 1 -- 172.21.15.5:0/3522992187 shutdown_connections 2024-09-11T22:20:53.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:53.221+0000 7f133613d640 1 -- 172.21.15.5:0/3522992187 wait complete. 2024-09-11T22:20:53.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:53 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3522992187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:53.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:53 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3522992187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:53 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3522992187' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:20:54.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T22:20:54.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T22:20:54.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T22:20:54.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T22:20:54.024 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T22:20:54.024 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T22:20:54.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T22:20:54.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T22:20:54.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-11T22:20:54.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-11T22:20:54.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-11T22:20:54.027 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-11T22:20:54.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:54 smithi172 ceph-mon[35461]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-11T22:20:54.726 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:54.741 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:54.804 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:54.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:54 smithi005 ceph-mon[30943]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-11T22:20:54.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:54 smithi159 ceph-mon[35534]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-11T22:20:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.246+0000 7f68374a9640 1 -- 172.21.15.5:0/1904969603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830106a90 msgr2=0x7f6830106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.246+0000 7f68374a9640 1 --2- 172.21.15.5:0/1904969603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830106a90 0x7f6830106e70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f682c009930 tx=0x7f682c02a420 comp rx=0 tx=0).stop 2024-09-11T22:20:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.246+0000 7f68374a9640 1 -- 172.21.15.5:0/1904969603 shutdown_connections 2024-09-11T22:20:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.246+0000 7f68374a9640 1 --2- 172.21.15.5:0/1904969603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830101430 0x7f683010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.246+0000 7f68374a9640 1 --2- 172.21.15.5:0/1904969603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6830100a90 0x7f6830100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.246+0000 7f68374a9640 1 --2- 172.21.15.5:0/1904969603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830106a90 0x7f6830106e70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.246+0000 7f68374a9640 1 -- 172.21.15.5:0/1904969603 >> 172.21.15.5:0/1904969603 conn(0x7f68300fc7d0 msgr2=0x7f68300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.246+0000 7f68374a9640 1 -- 172.21.15.5:0/1904969603 shutdown_connections 2024-09-11T22:20:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.246+0000 7f68374a9640 1 -- 172.21.15.5:0/1904969603 wait complete. 2024-09-11T22:20:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f68374a9640 1 Processor -- start 2024-09-11T22:20:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f68374a9640 1 -- start start 2024-09-11T22:20:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f68374a9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830100a90 0x7f68301993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f68374a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830101430 0x7f68301998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f68374a9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6830106a90 0x7f68301a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f68374a9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6830110eb0 con 0x7f6830101430 2024-09-11T22:20:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f68374a9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6830110d30 con 0x7f6830100a90 2024-09-11T22:20:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f68374a9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6830111030 con 0x7f6830106a90 2024-09-11T22:20:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f6834a44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830101430 0x7f68301998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f6834a44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830101430 0x7f68301998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57224/0 (socket says 172.21.15.5:57224) 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f6834a44640 1 -- 172.21.15.5:0/671045060 learned_addr learned my addr 172.21.15.5:0/671045060 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.247+0000 7f6835245640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830100a90 0x7f68301993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f6835a46640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6830106a90 0x7f68301a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f6834a44640 1 -- 172.21.15.5:0/671045060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6830106a90 msgr2=0x7f68301a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f6834a44640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6830106a90 0x7f68301a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f6834a44640 1 -- 172.21.15.5:0/671045060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830100a90 msgr2=0x7f68301993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f6834a44640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830100a90 0x7f68301993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f6834a44640 1 -- 172.21.15.5:0/671045060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68301a4960 con 0x7f6830101430 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f6835a46640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6830106a90 0x7f68301a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f6834a44640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830101430 0x7f68301998e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f682000b4a0 tx=0x7f682000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f68267fc640 1 -- 172.21.15.5:0/671045060 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820014070 con 0x7f6830101430 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f68267fc640 1 -- 172.21.15.5:0/671045060 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6820010af0 con 0x7f6830101430 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68301a4b30 con 0x7f6830101430 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f68267fc640 1 -- 172.21.15.5:0/671045060 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820017d10 con 0x7f6830101430 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.248+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68301a5040 con 0x7f6830101430 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.249+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68300628c0 con 0x7f6830101430 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.251+0000 7f68267fc640 1 -- 172.21.15.5:0/671045060 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6820017070 con 0x7f6830101430 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.252+0000 7f68267fc640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f681c0a2c10 0x7f681c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.252+0000 7f6835245640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f681c0a2c10 0x7f681c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.252+0000 7f68267fc640 1 -- 172.21.15.5:0/671045060 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f68200ce670 con 0x7f6830101430 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.252+0000 7f68267fc640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] conn(0x7f681c0ac460 0x7f681c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.252+0000 7f68267fc640 1 -- 172.21.15.5:0/671045060 --> [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f682000dec0 con 0x7f681c0ac460 2024-09-11T22:20:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.252+0000 7f68267fc640 1 -- 172.21.15.5:0/671045060 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f68200ceb50 con 0x7f6830101430 2024-09-11T22:20:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.253+0000 7f6835245640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f681c0a2c10 0x7f681c0a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f682c009900 tx=0x7f682c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.253+0000 7f6835a46640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] conn(0x7f681c0ac460 0x7f681c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.253+0000 7f6835a46640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] conn(0x7f681c0ac460 0x7f681c0ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.254+0000 7f68267fc640 1 -- 172.21.15.5:0/671045060 <== osd.1 v2:172.21.15.159:6800/1131654571 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f682000dec0 con 0x7f681c0ac460 2024-09-11T22:20:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.268+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 --> [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6830062d40 con 0x7f681c0ac460 2024-09-11T22:20:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68267fc640 1 -- 172.21.15.5:0/671045060 <== osd.1 v2:172.21.15.159:6800/1131654571 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6830062d40 con 0x7f681c0ac460 2024-09-11T22:20:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 >> [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] conn(0x7f681c0ac460 msgr2=0x7f681c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68374a9640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] conn(0x7f681c0ac460 0x7f681c0ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f681c0a2c10 msgr2=0x7f681c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68374a9640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f681c0a2c10 0x7f681c0a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f682c009900 tx=0x7f682c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:20:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830101430 msgr2=0x7f68301998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68374a9640 1 --2- 172.21.15.5:0/671045060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830101430 0x7f68301998e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f682000b4a0 tx=0x7f682000b960 comp rx=0 tx=0).stop 2024-09-11T22:20:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f6835a46640 1 -- 172.21.15.5:0/671045060 reap_dead start 2024-09-11T22:20:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 shutdown_connections 2024-09-11T22:20:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 >> 172.21.15.5:0/671045060 conn(0x7f68300fc7d0 msgr2=0x7f68300fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 shutdown_connections 2024-09-11T22:20:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:55.269+0000 7f68374a9640 1 -- 172.21.15.5:0/671045060 wait complete. 2024-09-11T22:20:55.775 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:55.775 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:20:56.323 INFO:teuthology.orchestra.run.smithi005.stdout:55834574921 2024-09-11T22:20:56.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T22:20:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:56 smithi005 ceph-mon[30943]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:20:56.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:56 smithi172 ceph-mon[35461]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:20:56.631 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:56 smithi159 ceph-mon[35534]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:20:57.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.494+0000 7fdf6489c640 1 -- 172.21.15.5:0/3749334349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf60101420 msgr2=0x7fdf6010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.494+0000 7fdf6489c640 1 --2- 172.21.15.5:0/3749334349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf60101420 0x7fdf6010d950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009cd0 tx=0x7fdf5802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:20:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.494+0000 7fdf6489c640 1 -- 172.21.15.5:0/3749334349 shutdown_connections 2024-09-11T22:20:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.494+0000 7fdf6489c640 1 --2- 172.21.15.5:0/3749334349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf60101420 0x7fdf6010d950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.494+0000 7fdf6489c640 1 --2- 172.21.15.5:0/3749334349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf60100a80 0x7fdf60100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.494+0000 7fdf6489c640 1 --2- 172.21.15.5:0/3749334349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf60106a80 0x7fdf60106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.494+0000 7fdf6489c640 1 -- 172.21.15.5:0/3749334349 >> 172.21.15.5:0/3749334349 conn(0x7fdf600fc7a0 msgr2=0x7fdf600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.494+0000 7fdf6489c640 1 -- 172.21.15.5:0/3749334349 shutdown_connections 2024-09-11T22:20:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.494+0000 7fdf6489c640 1 -- 172.21.15.5:0/3749334349 wait complete. 2024-09-11T22:20:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.495+0000 7fdf6489c640 1 Processor -- start 2024-09-11T22:20:57.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.495+0000 7fdf6489c640 1 -- start start 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.495+0000 7fdf6489c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf60100a80 0x7fdf601a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.495+0000 7fdf6489c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf60101420 0x7fdf601a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.495+0000 7fdf6489c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf60106a80 0x7fdf6019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.495+0000 7fdf6489c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf60110f20 con 0x7fdf60101420 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.495+0000 7fdf6489c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdf60110da0 con 0x7fdf60100a80 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.495+0000 7fdf6489c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdf601110a0 con 0x7fdf60106a80 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf60101420 0x7fdf601a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf60101420 0x7fdf601a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57248/0 (socket says 172.21.15.5:57248) 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5dd9b640 1 -- 172.21.15.5:0/3229413627 learned_addr learned my addr 172.21.15.5:0/3229413627 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5ed9d640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf60106a80 0x7fdf6019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5e59c640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf60100a80 0x7fdf601a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5dd9b640 1 -- 172.21.15.5:0/3229413627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf60106a80 msgr2=0x7fdf6019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5dd9b640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf60106a80 0x7fdf6019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5dd9b640 1 -- 172.21.15.5:0/3229413627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf60100a80 msgr2=0x7fdf601a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5dd9b640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf60100a80 0x7fdf601a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5dd9b640 1 -- 172.21.15.5:0/3229413627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6019ace0 con 0x7fdf60101420 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5e59c640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf60100a80 0x7fdf601a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf5dd9b640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf60101420 0x7fdf601a0930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400b4a0 tx=0x7fdf5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf4f7fe640 1 -- 172.21.15.5:0/3229413627 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54014070 con 0x7fdf60101420 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf4f7fe640 1 -- 172.21.15.5:0/3229413627 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf54010af0 con 0x7fdf60101420 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf4f7fe640 1 -- 172.21.15.5:0/3229413627 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54017d10 con 0x7fdf60101420 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6019afd0 con 0x7fdf60101420 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.496+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf601a72c0 con 0x7fdf60101420 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.497+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdf6006b1f0 con 0x7fdf60101420 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.499+0000 7fdf4f7fe640 1 -- 172.21.15.5:0/3229413627 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdf54017070 con 0x7fdf60101420 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.499+0000 7fdf4f7fe640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdf480a2c90 0x7fdf480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.499+0000 7fdf4f7fe640 1 -- 172.21.15.5:0/3229413627 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fdf540ce640 con 0x7fdf60101420 2024-09-11T22:20:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.499+0000 7fdf5e59c640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdf480a2c90 0x7fdf480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.500+0000 7fdf4f7fe640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] conn(0x7fdf480ac4e0 0x7fdf480ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.500+0000 7fdf4f7fe640 1 -- 172.21.15.5:0/3229413627 --> [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf54023e50 con 0x7fdf480ac4e0 2024-09-11T22:20:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.500+0000 7fdf4f7fe640 1 -- 172.21.15.5:0/3229413627 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fdf540ceb20 con 0x7fdf60101420 2024-09-11T22:20:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.500+0000 7fdf5ed9d640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] conn(0x7fdf480ac4e0 0x7fdf480ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.500+0000 7fdf5e59c640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdf480a2c90 0x7fdf480a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006fd0 tx=0x7fdf50008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.500+0000 7fdf5ed9d640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] conn(0x7fdf480ac4e0 0x7fdf480ae900 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:57.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.501+0000 7fdf4f7fe640 1 -- 172.21.15.5:0/3229413627 <== osd.9 v2:172.21.15.5:6826/26202527 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdf54023e50 con 0x7fdf480ac4e0 2024-09-11T22:20:57.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 --> [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdf601021c0 con 0x7fdf480ac4e0 2024-09-11T22:20:57.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf4f7fe640 1 -- 172.21.15.5:0/3229413627 <== osd.9 v2:172.21.15.5:6826/26202527 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdf601021c0 con 0x7fdf480ac4e0 2024-09-11T22:20:57.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] conn(0x7fdf480ac4e0 msgr2=0x7fdf480ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:57.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] conn(0x7fdf480ac4e0 0x7fdf480ae900 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:57.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdf480a2c90 msgr2=0x7fdf480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:57.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdf480a2c90 0x7fdf480a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf50006fd0 tx=0x7fdf50008040 comp rx=0 tx=0).stop 2024-09-11T22:20:57.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf60101420 msgr2=0x7fdf601a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:57.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 --2- 172.21.15.5:0/3229413627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf60101420 0x7fdf601a0930 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400b4a0 tx=0x7fdf5400b960 comp rx=0 tx=0).stop 2024-09-11T22:20:57.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf5ed9d640 1 -- 172.21.15.5:0/3229413627 reap_dead start 2024-09-11T22:20:57.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 shutdown_connections 2024-09-11T22:20:57.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 >> 172.21.15.5:0/3229413627 conn(0x7fdf600fc7a0 msgr2=0x7fdf6006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:57.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 shutdown_connections 2024-09-11T22:20:57.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.515+0000 7fdf6489c640 1 -- 172.21.15.5:0/3229413627 wait complete. 2024-09-11T22:20:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.050+0000 7f7f38b9d640 1 -- 172.21.15.5:0/3597973849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f34106f10 msgr2=0x7f7f3410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.050+0000 7f7f38b9d640 1 --2- 172.21.15.5:0/3597973849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f34106f10 0x7f7f3410ea90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c009ca0 tx=0x7f7f2c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.050+0000 7f7f38b9d640 1 -- 172.21.15.5:0/3597973849 shutdown_connections 2024-09-11T22:20:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.050+0000 7f7f38b9d640 1 --2- 172.21.15.5:0/3597973849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f34106f10 0x7f7f3410ea90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.050+0000 7f7f38b9d640 1 --2- 172.21.15.5:0/3597973849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f34102790 0x7f7f341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.050+0000 7f7f38b9d640 1 --2- 172.21.15.5:0/3597973849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f34101de0 0x7f7f341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.050+0000 7f7f38b9d640 1 -- 172.21.15.5:0/3597973849 >> 172.21.15.5:0/3597973849 conn(0x7f7f340fd8c0 msgr2=0x7f7f340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.050+0000 7f7f38b9d640 1 -- 172.21.15.5:0/3597973849 shutdown_connections 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.050+0000 7f7f38b9d640 1 -- 172.21.15.5:0/3597973849 wait complete. 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f38b9d640 1 Processor -- start 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f38b9d640 1 -- start start 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f38b9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f34101de0 0x7f7f3419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f38b9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f34102790 0x7f7f3419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f38b9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f34106f10 0x7f7f3419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f38b9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f34111f70 con 0x7f7f34102790 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f38b9d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f34111df0 con 0x7f7f34101de0 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f38b9d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7f341120f0 con 0x7f7f34106f10 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f34102790 0x7f7f3419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f34102790 0x7f7f3419ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57276/0 (socket says 172.21.15.5:57276) 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f2bfff640 1 -- 172.21.15.5:0/190844725 learned_addr learned my addr 172.21.15.5:0/190844725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:58.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f3259c640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f34101de0 0x7f7f3419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.051+0000 7f7f32d9d640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f34106f10 0x7f7f3419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f2bfff640 1 -- 172.21.15.5:0/190844725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f34106f10 msgr2=0x7f7f3419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f2bfff640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f34106f10 0x7f7f3419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f2bfff640 1 -- 172.21.15.5:0/190844725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f34101de0 msgr2=0x7f7f3419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f2bfff640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f34101de0 0x7f7f3419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f2bfff640 1 -- 172.21.15.5:0/190844725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f3419f590 con 0x7f7f34102790 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f2bfff640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f34102790 0x7f7f3419ab20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00c890 tx=0x7f7f1c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f2b7fe640 1 -- 172.21.15.5:0/190844725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c004220 con 0x7f7f34102790 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f2b7fe640 1 -- 172.21.15.5:0/190844725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f1c0043c0 con 0x7f7f34102790 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f3419f880 con 0x7f7f34102790 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f2b7fe640 1 -- 172.21.15.5:0/190844725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0049e0 con 0x7f7f34102790 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.052+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f341a7160 con 0x7f7f34102790 2024-09-11T22:20:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.053+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7f34075780 con 0x7f7f34102790 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.054+0000 7f7f2b7fe640 1 -- 172.21.15.5:0/190844725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7f1c0079b0 con 0x7f7f34102790 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.055+0000 7f7f2b7fe640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7f200a2c90 0x7f7f200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.055+0000 7f7f3259c640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7f200a2c90 0x7f7f200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.055+0000 7f7f2b7fe640 1 -- 172.21.15.5:0/190844725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f7f1c0cf270 con 0x7f7f34102790 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.055+0000 7f7f2b7fe640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] conn(0x7f7f200ac4e0 0x7f7f200ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.055+0000 7f7f2b7fe640 1 -- 172.21.15.5:0/190844725 --> [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7f1c024ea0 con 0x7f7f200ac4e0 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.055+0000 7f7f2b7fe640 1 -- 172.21.15.5:0/190844725 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f7f1c0cf620 con 0x7f7f34102790 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.055+0000 7f7f32d9d640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] conn(0x7f7f200ac4e0 0x7f7f200ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.055+0000 7f7f3259c640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7f200a2c90 0x7f7f200a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7f24005fd0 tx=0x7f7f2400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.056+0000 7f7f32d9d640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] conn(0x7f7f200ac4e0 0x7f7f200ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.056+0000 7f7f2b7fe640 1 -- 172.21.15.5:0/190844725 <== osd.3 v2:172.21.15.5:6810/708957835 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7f1c024ea0 con 0x7f7f200ac4e0 2024-09-11T22:20:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.070+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 --> [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7f341032c0 con 0x7f7f200ac4e0 2024-09-11T22:20:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.070+0000 7f7f2b7fe640 1 -- 172.21.15.5:0/190844725 <== osd.3 v2:172.21.15.5:6810/708957835 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7f341032c0 con 0x7f7f200ac4e0 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] conn(0x7f7f200ac4e0 msgr2=0x7f7f200ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f38b9d640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] conn(0x7f7f200ac4e0 0x7f7f200ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7f200a2c90 msgr2=0x7f7f200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f38b9d640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7f200a2c90 0x7f7f200a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7f24005fd0 tx=0x7f7f2400b390 comp rx=0 tx=0).stop 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f34102790 msgr2=0x7f7f3419ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f38b9d640 1 --2- 172.21.15.5:0/190844725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f34102790 0x7f7f3419ab20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00c890 tx=0x7f7f1c00cd50 comp rx=0 tx=0).stop 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f32d9d640 1 -- 172.21.15.5:0/190844725 reap_dead start 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 shutdown_connections 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 >> 172.21.15.5:0/190844725 conn(0x7f7f340fd8c0 msgr2=0x7f7f340fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 shutdown_connections 2024-09-11T22:20:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.071+0000 7f7f38b9d640 1 -- 172.21.15.5:0/190844725 wait complete. 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.253+0000 7ff525337640 1 -- 172.21.15.5:0/562099145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff520074590 msgr2=0x7ff520074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.253+0000 7ff525337640 1 --2- 172.21.15.5:0/562099145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff520074590 0x7ff520074970 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff510009ca0 tx=0x7ff51002a950 comp rx=0 tx=0).stop 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.253+0000 7ff525337640 1 -- 172.21.15.5:0/562099145 shutdown_connections 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.253+0000 7ff525337640 1 --2- 172.21.15.5:0/562099145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520112ba0 0x7ff520114f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.253+0000 7ff525337640 1 --2- 172.21.15.5:0/562099145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff520074eb0 0x7ff520112660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.253+0000 7ff525337640 1 --2- 172.21.15.5:0/562099145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff520074590 0x7ff520074970 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.253+0000 7ff525337640 1 -- 172.21.15.5:0/562099145 >> 172.21.15.5:0/562099145 conn(0x7ff5200fd8f0 msgr2=0x7ff5200ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.254+0000 7ff525337640 1 -- 172.21.15.5:0/562099145 shutdown_connections 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.254+0000 7ff525337640 1 -- 172.21.15.5:0/562099145 wait complete. 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.254+0000 7ff525337640 1 Processor -- start 2024-09-11T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.254+0000 7ff525337640 1 -- start start 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.254+0000 7ff525337640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff520074590 0x7ff5201a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.254+0000 7ff525337640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 0x7ff5201a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff525337640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff520112ba0 0x7ff52019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff525337640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff520118660 con 0x7ff520112ba0 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff525337640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201184e0 con 0x7ff520074590 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff525337640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201187e0 con 0x7ff520074eb0 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff520112ba0 0x7ff52019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff520112ba0 0x7ff52019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57290/0 (socket says 172.21.15.5:57290) 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51f7fe640 1 -- 172.21.15.5:0/2211147462 learned_addr learned my addr 172.21.15.5:0/2211147462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51effd640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff520074590 0x7ff5201a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51e7fc640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 0x7ff5201a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51f7fe640 1 -- 172.21.15.5:0/2211147462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 msgr2=0x7ff5201a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51f7fe640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 0x7ff5201a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51f7fe640 1 -- 172.21.15.5:0/2211147462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff520074590 msgr2=0x7ff5201a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51f7fe640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff520074590 0x7ff5201a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51f7fe640 1 -- 172.21.15.5:0/2211147462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52019f1e0 con 0x7ff520112ba0 2024-09-11T22:20:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51e7fc640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff520074eb0 0x7ff5201a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:20:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff51f7fe640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff520112ba0 0x7ff52019eab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff51800b3f0 tx=0x7ff51800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff507fff640 1 -- 172.21.15.5:0/2211147462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518015020 con 0x7ff520112ba0 2024-09-11T22:20:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52019f4d0 con 0x7ff520112ba0 2024-09-11T22:20:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff507fff640 1 -- 172.21.15.5:0/2211147462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff518011d10 con 0x7ff520112ba0 2024-09-11T22:20:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.255+0000 7ff507fff640 1 -- 172.21.15.5:0/2211147462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51800f2e0 con 0x7ff520112ba0 2024-09-11T22:20:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.256+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5201ab7c0 con 0x7ff520112ba0 2024-09-11T22:20:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.256+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff520062900 con 0x7ff520112ba0 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.258+0000 7ff507fff640 1 -- 172.21.15.5:0/2211147462 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff518018020 con 0x7ff520112ba0 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.258+0000 7ff507fff640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff5080a2c10 0x7ff5080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.258+0000 7ff51effd640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff5080a2c10 0x7ff5080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.258+0000 7ff507fff640 1 -- 172.21.15.5:0/2211147462 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff5180cedd0 con 0x7ff520112ba0 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.258+0000 7ff507fff640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] conn(0x7ff5080ac460 0x7ff5080ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.258+0000 7ff507fff640 1 -- 172.21.15.5:0/2211147462 --> [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff518004450 con 0x7ff5080ac460 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.259+0000 7ff507fff640 1 -- 172.21.15.5:0/2211147462 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7ff5180cf2b0 con 0x7ff520112ba0 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.259+0000 7ff51e7fc640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] conn(0x7ff5080ac460 0x7ff5080ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.259+0000 7ff51effd640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff5080a2c10 0x7ff5080a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff510000f80 tx=0x7ff510009360 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.259+0000 7ff51e7fc640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] conn(0x7ff5080ac460 0x7ff5080ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.260+0000 7ff507fff640 1 -- 172.21.15.5:0/2211147462 <== osd.8 v2:172.21.15.172:6816/421514522 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff518004450 con 0x7ff5080ac460 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.273+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 --> [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff520062d00 con 0x7ff5080ac460 2024-09-11T22:20:58.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.274+0000 7ff507fff640 1 -- 172.21.15.5:0/2211147462 <== osd.8 v2:172.21.15.172:6816/421514522 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff520062d00 con 0x7ff5080ac460 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.274+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 >> [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] conn(0x7ff5080ac460 msgr2=0x7ff5080ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.274+0000 7ff525337640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] conn(0x7ff5080ac460 0x7ff5080ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.274+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff5080a2c10 msgr2=0x7ff5080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.274+0000 7ff525337640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff5080a2c10 0x7ff5080a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff510000f80 tx=0x7ff510009360 comp rx=0 tx=0).stop 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.274+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff520112ba0 msgr2=0x7ff52019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.274+0000 7ff525337640 1 --2- 172.21.15.5:0/2211147462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff520112ba0 0x7ff52019eab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff51800b3f0 tx=0x7ff51800b8b0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.274+0000 7ff51f7fe640 1 -- 172.21.15.5:0/2211147462 reap_dead start 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.274+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 shutdown_connections 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.274+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 >> 172.21.15.5:0/2211147462 conn(0x7ff5200fd8f0 msgr2=0x7ff5200ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.275+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 shutdown_connections 2024-09-11T22:20:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:58.275+0000 7ff525337640 1 -- 172.21.15.5:0/2211147462 wait complete. 2024-09-11T22:20:58.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.932+0000 7fb68dc93640 1 -- 172.21.15.5:0/1054447948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb68810c170 msgr2=0x7fb68810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.932+0000 7fb68dc93640 1 --2- 172.21.15.5:0/1054447948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb68810c170 0x7fb68810e610 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb680009ca0 tx=0x7fb68002a950 comp rx=0 tx=0).stop 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.932+0000 7fb68dc93640 1 -- 172.21.15.5:0/1054447948 shutdown_connections 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.932+0000 7fb68dc93640 1 --2- 172.21.15.5:0/1054447948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb68810c170 0x7fb68810e610 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.932+0000 7fb68dc93640 1 --2- 172.21.15.5:0/1054447948 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb688103660 0x7fb68810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.932+0000 7fb68dc93640 1 --2- 172.21.15.5:0/1054447948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb688102d40 0x7fb688103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.932+0000 7fb68dc93640 1 -- 172.21.15.5:0/1054447948 >> 172.21.15.5:0/1054447948 conn(0x7fb6880fc7a0 msgr2=0x7fb6880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.932+0000 7fb68dc93640 1 -- 172.21.15.5:0/1054447948 shutdown_connections 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.932+0000 7fb68dc93640 1 -- 172.21.15.5:0/1054447948 wait complete. 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.932+0000 7fb68dc93640 1 Processor -- start 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb68dc93640 1 -- start start 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb68dc93640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb688102d40 0x7fb6881009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb68dc93640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb688103660 0x7fb688100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb68dc93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb68810c170 0x7fb688101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb68dc93640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb688111f70 con 0x7fb68810c170 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb68dc93640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb688111df0 con 0x7fb688103660 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb68dc93640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb6881120f0 con 0x7fb688102d40 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb687fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb68810c170 0x7fb688101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb687fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb68810c170 0x7fb688101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57262/0 (socket says 172.21.15.5:57262) 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb687fff640 1 -- 172.21.15.5:0/471621082 learned_addr learned my addr 172.21.15.5:0/471621082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb686ffd640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb688103660 0x7fb688100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb6877fe640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb688102d40 0x7fb6881009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb687fff640 1 -- 172.21.15.5:0/471621082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb688102d40 msgr2=0x7fb6881009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb687fff640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb688102d40 0x7fb6881009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb687fff640 1 -- 172.21.15.5:0/471621082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb688103660 msgr2=0x7fb688100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb687fff640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb688103660 0x7fb688100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.933+0000 7fb687fff640 1 -- 172.21.15.5:0/471621082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6881a68b0 con 0x7fb68810c170 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.934+0000 7fb687fff640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb68810c170 0x7fb688101420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb68002ae70 tx=0x7fb68002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.934+0000 7fb684ff9640 1 -- 172.21.15.5:0/471621082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680034040 con 0x7fb68810c170 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.934+0000 7fb684ff9640 1 -- 172.21.15.5:0/471621082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb680030920 con 0x7fb68810c170 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.934+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6881a6b40 con 0x7fb68810c170 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.934+0000 7fb684ff9640 1 -- 172.21.15.5:0/471621082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb680045480 con 0x7fb68810c170 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.934+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6881a7020 con 0x7fb68810c170 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.935+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb6880628c0 con 0x7fb68810c170 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.936+0000 7fb684ff9640 1 -- 172.21.15.5:0/471621082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb680037020 con 0x7fb68810c170 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.937+0000 7fb684ff9640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb6700a2c10 0x7fb6700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.937+0000 7fb6877fe640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb6700a2c10 0x7fb6700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.937+0000 7fb684ff9640 1 -- 172.21.15.5:0/471621082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb6800f0120 con 0x7fb68810c170 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.937+0000 7fb684ff9640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] conn(0x7fb6700ac460 0x7fb6700ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:58.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.937+0000 7fb684ff9640 1 -- 172.21.15.5:0/471621082 --> [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb68002c030 con 0x7fb6700ac460 2024-09-11T22:20:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.937+0000 7fb684ff9640 1 -- 172.21.15.5:0/471621082 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb6800f0600 con 0x7fb68810c170 2024-09-11T22:20:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.937+0000 7fb686ffd640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] conn(0x7fb6700ac460 0x7fb6700ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.937+0000 7fb6877fe640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb6700a2c10 0x7fb6700a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb67800bbe0 tx=0x7fb67800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.937+0000 7fb686ffd640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] conn(0x7fb6700ac460 0x7fb6700ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.938+0000 7fb684ff9640 1 -- 172.21.15.5:0/471621082 <== osd.6 v2:172.21.15.5:6818/2494841053 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb68002c030 con 0x7fb6700ac460 2024-09-11T22:20:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 --> [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb688062d40 con 0x7fb6700ac460 2024-09-11T22:20:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb684ff9640 1 -- 172.21.15.5:0/471621082 <== osd.6 v2:172.21.15.5:6818/2494841053 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb688062d40 con 0x7fb6700ac460 2024-09-11T22:20:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] conn(0x7fb6700ac460 msgr2=0x7fb6700ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] conn(0x7fb6700ac460 0x7fb6700ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb6700a2c10 msgr2=0x7fb6700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb6700a2c10 0x7fb6700a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb67800bbe0 tx=0x7fb67800b390 comp rx=0 tx=0).stop 2024-09-11T22:20:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb68810c170 msgr2=0x7fb688101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 --2- 172.21.15.5:0/471621082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb68810c170 0x7fb688101420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb68002ae70 tx=0x7fb68002f5b0 comp rx=0 tx=0).stop 2024-09-11T22:20:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb687fff640 1 -- 172.21.15.5:0/471621082 reap_dead start 2024-09-11T22:20:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 shutdown_connections 2024-09-11T22:20:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 >> 172.21.15.5:0/471621082 conn(0x7fb6880fc7a0 msgr2=0x7fb6880fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 shutdown_connections 2024-09-11T22:20:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:57.952+0000 7fb68dc93640 1 -- 172.21.15.5:0/471621082 wait complete. 2024-09-11T22:20:58.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:20:58 smithi005 ceph-mon[30943]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-11T22:20:58.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:20:58 smithi172 ceph-mon[35461]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-11T22:20:58.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:20:58 smithi159 ceph-mon[35534]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.4 KiB/s wr, 12 op/s 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.352+0000 7fa76d858640 1 -- 172.21.15.5:0/528263454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7681027a0 msgr2=0x7fa7681067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.352+0000 7fa76d858640 1 --2- 172.21.15.5:0/528263454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7681027a0 0x7fa7681067f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009900 tx=0x7fa75c02a410 comp rx=0 tx=0).stop 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.352+0000 7fa76d858640 1 -- 172.21.15.5:0/528263454 shutdown_connections 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.352+0000 7fa76d858640 1 --2- 172.21.15.5:0/528263454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa768106f20 0x7fa76810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.352+0000 7fa76d858640 1 --2- 172.21.15.5:0/528263454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7681027a0 0x7fa7681067f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.352+0000 7fa76d858640 1 --2- 172.21.15.5:0/528263454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa768101df0 0x7fa7681021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.352+0000 7fa76d858640 1 -- 172.21.15.5:0/528263454 >> 172.21.15.5:0/528263454 conn(0x7fa7680fd8f0 msgr2=0x7fa7680ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.352+0000 7fa76d858640 1 -- 172.21.15.5:0/528263454 shutdown_connections 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.352+0000 7fa76d858640 1 -- 172.21.15.5:0/528263454 wait complete. 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.353+0000 7fa76d858640 1 Processor -- start 2024-09-11T22:20:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.353+0000 7fa76d858640 1 -- start start 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.353+0000 7fa76d858640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa768101df0 0x7fa768198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.353+0000 7fa76d858640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7681027a0 0x7fa7681988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa76d858640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa768106f20 0x7fa76819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa76d858640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa768111f80 con 0x7fa7681027a0 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa76d858640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa768111e00 con 0x7fa768101df0 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa76d858640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa768112100 con 0x7fa768106f20 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa7667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7681027a0 0x7fa7681988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa7667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7681027a0 0x7fa7681988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57312/0 (socket says 172.21.15.5:57312) 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa7667fc640 1 -- 172.21.15.5:0/2355177856 learned_addr learned my addr 172.21.15.5:0/2355177856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa7677fe640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa768106f20 0x7fa76819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa766ffd640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa768101df0 0x7fa768198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa7667fc640 1 -- 172.21.15.5:0/2355177856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa768106f20 msgr2=0x7fa76819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa7667fc640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa768106f20 0x7fa76819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa7667fc640 1 -- 172.21.15.5:0/2355177856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa768101df0 msgr2=0x7fa768198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa7667fc640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa768101df0 0x7fa768198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa7667fc640 1 -- 172.21.15.5:0/2355177856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76819d340 con 0x7fa7681027a0 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa766ffd640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa768101df0 0x7fa768198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa7667fc640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7681027a0 0x7fa7681988d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa75c006010 tx=0x7fa75c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa76c856640 1 -- 172.21.15.5:0/2355177856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c03f030 con 0x7fa7681027a0 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa76c856640 1 -- 172.21.15.5:0/2355177856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa75c003ce0 con 0x7fa7681027a0 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa76c856640 1 -- 172.21.15.5:0/2355177856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c035070 con 0x7fa7681027a0 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa76819d5d0 con 0x7fa7681027a0 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.354+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7681a4f10 con 0x7fa7681027a0 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.355+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa768062980 con 0x7fa7681027a0 2024-09-11T22:20:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.357+0000 7fa76c856640 1 -- 172.21.15.5:0/2355177856 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa75c042030 con 0x7fa7681027a0 2024-09-11T22:20:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.357+0000 7fa76c856640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa7500a2c90 0x7fa7500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.357+0000 7fa766ffd640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa7500a2c90 0x7fa7500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.357+0000 7fa76c856640 1 -- 172.21.15.5:0/2355177856 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa75c0efcc0 con 0x7fa7681027a0 2024-09-11T22:20:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.358+0000 7fa76c856640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] conn(0x7fa7500ac4e0 0x7fa7500ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:20:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.358+0000 7fa76c856640 1 -- 172.21.15.5:0/2355177856 --> [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa75c001d30 con 0x7fa7500ac4e0 2024-09-11T22:20:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.358+0000 7fa76c856640 1 -- 172.21.15.5:0/2355177856 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fa75c0f0070 con 0x7fa7681027a0 2024-09-11T22:20:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.358+0000 7fa766ffd640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa7500a2c90 0x7fa7500a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa758009b40 tx=0x7fa758005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.358+0000 7fa7677fe640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] conn(0x7fa7500ac4e0 0x7fa7500ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:20:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.358+0000 7fa7677fe640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] conn(0x7fa7500ac4e0 0x7fa7500ae900 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.359+0000 7fa76c856640 1 -- 172.21.15.5:0/2355177856 <== osd.11 v2:172.21.15.172:6824/2463160464 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa75c001d30 con 0x7fa7500ac4e0 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.373+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 --> [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa768062d80 con 0x7fa7500ac4e0 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76c856640 1 -- 172.21.15.5:0/2355177856 <== osd.11 v2:172.21.15.172:6824/2463160464 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa768062d80 con 0x7fa7500ac4e0 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 >> [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] conn(0x7fa7500ac4e0 msgr2=0x7fa7500ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76d858640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] conn(0x7fa7500ac4e0 0x7fa7500ae900 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa7500a2c90 msgr2=0x7fa7500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76d858640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa7500a2c90 0x7fa7500a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa758009b40 tx=0x7fa758005e50 comp rx=0 tx=0).stop 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7681027a0 msgr2=0x7fa7681988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76d858640 1 --2- 172.21.15.5:0/2355177856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7681027a0 0x7fa7681988d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa75c006010 tx=0x7fa75c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa7677fe640 1 -- 172.21.15.5:0/2355177856 reap_dead start 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 shutdown_connections 2024-09-11T22:20:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 >> 172.21.15.5:0/2355177856 conn(0x7fa7680fd8f0 msgr2=0x7fa7680feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:20:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 shutdown_connections 2024-09-11T22:20:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:20:59.374+0000 7fa76d858640 1 -- 172.21.15.5:0/2355177856 wait complete. 2024-09-11T22:21:00.270 INFO:teuthology.orchestra.run.smithi005.stdout:266287972383 2024-09-11T22:21:00.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-11T22:21:00.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:00 smithi005 ceph-mon[30943]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T22:21:00.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:00 smithi172 ceph-mon[35461]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T22:21:00.631 INFO:teuthology.orchestra.run.smithi005.stdout:103079215167 2024-09-11T22:21:00.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T22:21:00.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:00 smithi159 ceph-mon[35534]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-11T22:21:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.007+0000 7f36cadc7640 1 -- 172.21.15.5:0/3056383292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4106f20 msgr2=0x7f36c410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.007+0000 7f36cadc7640 1 --2- 172.21.15.5:0/3056383292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4106f20 0x7f36c410eaa0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009900 tx=0x7f36b402a410 comp rx=0 tx=0).stop 2024-09-11T22:21:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.007+0000 7f36cadc7640 1 -- 172.21.15.5:0/3056383292 shutdown_connections 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.007+0000 7f36cadc7640 1 --2- 172.21.15.5:0/3056383292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4106f20 0x7f36c410eaa0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.007+0000 7f36cadc7640 1 --2- 172.21.15.5:0/3056383292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c41027a0 0x7f36c41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.007+0000 7f36cadc7640 1 --2- 172.21.15.5:0/3056383292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36c4101df0 0x7f36c41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.007+0000 7f36cadc7640 1 -- 172.21.15.5:0/3056383292 >> 172.21.15.5:0/3056383292 conn(0x7f36c40fd8f0 msgr2=0x7f36c40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.007+0000 7f36cadc7640 1 -- 172.21.15.5:0/3056383292 shutdown_connections 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.007+0000 7f36cadc7640 1 -- 172.21.15.5:0/3056383292 wait complete. 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36cadc7640 1 Processor -- start 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36cadc7640 1 -- start start 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36cadc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4101df0 0x7f36c41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36cadc7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36c41027a0 0x7f36c4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36cadc7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4106f20 0x7f36c419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36cadc7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4111f80 con 0x7f36c4101df0 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36cadc7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4111e00 con 0x7f36c4106f20 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36cadc7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4112100 con 0x7f36c41027a0 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36c8b63640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4101df0 0x7f36c41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36c8b63640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4101df0 0x7f36c41983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57336/0 (socket says 172.21.15.5:57336) 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.008+0000 7f36c8b63640 1 -- 172.21.15.5:0/3370570913 learned_addr learned my addr 172.21.15.5:0/3370570913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c9364640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4106f20 0x7f36c419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c3fff640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36c41027a0 0x7f36c4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c8b63640 1 -- 172.21.15.5:0/3370570913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36c41027a0 msgr2=0x7f36c4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c8b63640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f36c41027a0 0x7f36c4198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c8b63640 1 -- 172.21.15.5:0/3370570913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4106f20 msgr2=0x7f36c419ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c8b63640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4106f20 0x7f36c419ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c8b63640 1 -- 172.21.15.5:0/3370570913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c419d390 con 0x7f36c4101df0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c9364640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36c4106f20 0x7f36c419ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c8b63640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4101df0 0x7f36c41983e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00ea80 tx=0x7f36bc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c1ffb640 1 -- 172.21.15.5:0/3370570913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc00ce40 con 0x7f36c4101df0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c1ffb640 1 -- 172.21.15.5:0/3370570913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36bc005020 con 0x7f36c4101df0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c419d680 con 0x7f36c4101df0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36c1ffb640 1 -- 172.21.15.5:0/3370570913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc011760 con 0x7f36c4101df0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.009+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c41a4f60 con 0x7f36c4101df0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.010+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36c4062940 con 0x7f36c4101df0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.011+0000 7f36c1ffb640 1 -- 172.21.15.5:0/3370570913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f36bc0051d0 con 0x7f36c4101df0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.012+0000 7f36c1ffb640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f36ac0a2c90 0x7f36ac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.012+0000 7f36c1ffb640 1 -- 172.21.15.5:0/3370570913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f36bc0cf920 con 0x7f36c4101df0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.012+0000 7f36c3fff640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f36ac0a2c90 0x7f36ac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.012+0000 7f36c1ffb640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] conn(0x7f36ac0ac4e0 0x7f36ac0ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.012+0000 7f36c1ffb640 1 -- 172.21.15.5:0/3370570913 --> [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36bc026ea0 con 0x7f36ac0ac4e0 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.012+0000 7f36c1ffb640 1 -- 172.21.15.5:0/3370570913 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f36bc0cfcd0 con 0x7f36c4101df0 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.012+0000 7f36c9364640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] conn(0x7f36ac0ac4e0 0x7f36ac0ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.012+0000 7f36c3fff640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f36ac0a2c90 0x7f36ac0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f36b8005d80 tx=0x7f36b8005c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.013+0000 7f36c9364640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] conn(0x7f36ac0ac4e0 0x7f36ac0ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.014+0000 7f36c1ffb640 1 -- 172.21.15.5:0/3370570913 <== osd.4 v2:172.21.15.159:6808/4176432146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36bc026ea0 con 0x7f36ac0ac4e0 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.027+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 --> [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f36c4062dc0 con 0x7f36ac0ac4e0 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.028+0000 7f36c1ffb640 1 -- 172.21.15.5:0/3370570913 <== osd.4 v2:172.21.15.159:6808/4176432146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f36c4062dc0 con 0x7f36ac0ac4e0 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.028+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 >> [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] conn(0x7f36ac0ac4e0 msgr2=0x7f36ac0ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.028+0000 7f36cadc7640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] conn(0x7f36ac0ac4e0 0x7f36ac0ae900 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.028+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f36ac0a2c90 msgr2=0x7f36ac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.028+0000 7f36cadc7640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f36ac0a2c90 0x7f36ac0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f36b8005d80 tx=0x7f36b8005c90 comp rx=0 tx=0).stop 2024-09-11T22:21:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.028+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4101df0 msgr2=0x7f36c41983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.028+0000 7f36cadc7640 1 --2- 172.21.15.5:0/3370570913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4101df0 0x7f36c41983e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00ea80 tx=0x7f36bc00ef40 comp rx=0 tx=0).stop 2024-09-11T22:21:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.028+0000 7f36c9364640 1 -- 172.21.15.5:0/3370570913 reap_dead start 2024-09-11T22:21:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.028+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 shutdown_connections 2024-09-11T22:21:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.028+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 >> 172.21.15.5:0/3370570913 conn(0x7f36c40fd8f0 msgr2=0x7f36c40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.029+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 shutdown_connections 2024-09-11T22:21:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:01.029+0000 7f36cadc7640 1 -- 172.21.15.5:0/3370570913 wait complete. 2024-09-11T22:21:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:01.262 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:01.262 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:01.262 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:01.263 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:02.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.565+0000 7f6fda271640 1 -- 172.21.15.5:0/2922497343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fd4106f10 msgr2=0x7f6fd410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:02.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.565+0000 7f6fda271640 1 --2- 172.21.15.5:0/2922497343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fd4106f10 0x7f6fd410ea90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009ca0 tx=0x7f6fcc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:21:02.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.565+0000 7f6fda271640 1 -- 172.21.15.5:0/2922497343 shutdown_connections 2024-09-11T22:21:02.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.565+0000 7f6fda271640 1 --2- 172.21.15.5:0/2922497343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fd4106f10 0x7f6fd410ea90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:02.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.565+0000 7f6fda271640 1 --2- 172.21.15.5:0/2922497343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd4102790 0x7f6fd41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:02.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.565+0000 7f6fda271640 1 --2- 172.21.15.5:0/2922497343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fd4101de0 0x7f6fd41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:02.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.565+0000 7f6fda271640 1 -- 172.21.15.5:0/2922497343 >> 172.21.15.5:0/2922497343 conn(0x7f6fd40fd8c0 msgr2=0x7f6fd40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:02.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.565+0000 7f6fda271640 1 -- 172.21.15.5:0/2922497343 shutdown_connections 2024-09-11T22:21:02.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.565+0000 7f6fda271640 1 -- 172.21.15.5:0/2922497343 wait complete. 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fda271640 1 Processor -- start 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fda271640 1 -- start start 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fda271640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fd4101de0 0x7f6fd4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fda271640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fd4102790 0x7f6fd4198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fda271640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd4106f10 0x7f6fd419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fda271640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd4111f90 con 0x7f6fd4101de0 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fda271640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd4111e10 con 0x7f6fd4102790 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fda271640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd4112110 con 0x7f6fd4106f10 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fd3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fd4101de0 0x7f6fd4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fd3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fd4101de0 0x7f6fd4198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45428/0 (socket says 172.21.15.5:45428) 2024-09-11T22:21:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fd3fff640 1 -- 172.21.15.5:0/4065549522 learned_addr learned my addr 172.21.15.5:0/4065549522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fcbfff640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fd4102790 0x7f6fd4198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.566+0000 7f6fd880e640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd4106f10 0x7f6fd419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fd3fff640 1 -- 172.21.15.5:0/4065549522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd4106f10 msgr2=0x7f6fd419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fd3fff640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd4106f10 0x7f6fd419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fd3fff640 1 -- 172.21.15.5:0/4065549522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fd4102790 msgr2=0x7f6fd4198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fd3fff640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fd4102790 0x7f6fd4198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fd3fff640 1 -- 172.21.15.5:0/4065549522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd419d500 con 0x7f6fd4101de0 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fd880e640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fd4106f10 0x7f6fd419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fd3fff640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fd4101de0 0x7f6fd4198550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400d950 tx=0x7f6fc400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fd1ffb640 1 -- 172.21.15.5:0/4065549522 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4017070 con 0x7f6fd4101de0 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd419d7f0 con 0x7f6fd4101de0 2024-09-11T22:21:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fd1ffb640 1 -- 172.21.15.5:0/4065549522 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fc400de80 con 0x7f6fd4101de0 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fd1ffb640 1 -- 172.21.15.5:0/4065549522 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc400f620 con 0x7f6fd4101de0 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.567+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd41a50d0 con 0x7f6fd4101de0 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.568+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6fd4062900 con 0x7f6fd4101de0 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.569+0000 7f6fd1ffb640 1 -- 172.21.15.5:0/4065549522 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6fc4009c30 con 0x7f6fd4101de0 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.570+0000 7f6fd1ffb640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6fc00a2c10 0x7f6fc00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.570+0000 7f6fcbfff640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6fc00a2c10 0x7f6fc00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.570+0000 7f6fd1ffb640 1 -- 172.21.15.5:0/4065549522 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6fc40cf1d0 con 0x7f6fd4101de0 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.570+0000 7f6fd1ffb640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] conn(0x7f6fc00ac460 0x7f6fc00ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.570+0000 7f6fd1ffb640 1 -- 172.21.15.5:0/4065549522 --> [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6fc00aef30 con 0x7f6fc00ac460 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.570+0000 7f6fd1ffb640 1 -- 172.21.15.5:0/4065549522 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f6fc40cf580 con 0x7f6fd4101de0 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.570+0000 7f6fcbfff640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6fc00a2c10 0x7f6fc00a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fd41087b0 tx=0x7f6fbc009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.570+0000 7f6fd880e640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] conn(0x7f6fc00ac460 0x7f6fc00ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.571+0000 7f6fd880e640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] conn(0x7f6fc00ac460 0x7f6fc00ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.572+0000 7f6fd1ffb640 1 -- 172.21.15.5:0/4065549522 <== osd.10 v2:172.21.15.159:6824/1772373268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6fc00aef30 con 0x7f6fc00ac460 2024-09-11T22:21:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.586+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 --> [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6fd4062d00 con 0x7f6fc00ac460 2024-09-11T22:21:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.586+0000 7f6fd1ffb640 1 -- 172.21.15.5:0/4065549522 <== osd.10 v2:172.21.15.159:6824/1772373268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6fd4062d00 con 0x7f6fc00ac460 2024-09-11T22:21:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 >> [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] conn(0x7f6fc00ac460 msgr2=0x7f6fc00ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fda271640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] conn(0x7f6fc00ac460 0x7f6fc00ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6fc00a2c10 msgr2=0x7f6fc00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fda271640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6fc00a2c10 0x7f6fc00a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fd41087b0 tx=0x7f6fbc009110 comp rx=0 tx=0).stop 2024-09-11T22:21:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fd4101de0 msgr2=0x7f6fd4198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fda271640 1 --2- 172.21.15.5:0/4065549522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fd4101de0 0x7f6fd4198550 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400d950 tx=0x7f6fc400dd50 comp rx=0 tx=0).stop 2024-09-11T22:21:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fd880e640 1 -- 172.21.15.5:0/4065549522 reap_dead start 2024-09-11T22:21:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 shutdown_connections 2024-09-11T22:21:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 >> 172.21.15.5:0/4065549522 conn(0x7f6fd40fd8c0 msgr2=0x7f6fd40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 shutdown_connections 2024-09-11T22:21:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:02.587+0000 7f6fda271640 1 -- 172.21.15.5:0/4065549522 wait complete. 2024-09-11T22:21:02.612 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:02 smithi172 ceph-mon[35461]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T22:21:02.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:02 smithi005 ceph-mon[30943]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T22:21:02.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:02 smithi159 ceph-mon[35534]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-11T22:21:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.330+0000 7f4ef9158640 1 -- 172.21.15.5:0/191866805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4102ef0 msgr2=0x7f4ef410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.330+0000 7f4ef9158640 1 --2- 172.21.15.5:0/191866805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4102ef0 0x7f4ef410da40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009ca0 tx=0x7f4eec02a950 comp rx=0 tx=0).stop 2024-09-11T22:21:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.330+0000 7f4ef9158640 1 -- 172.21.15.5:0/191866805 shutdown_connections 2024-09-11T22:21:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.330+0000 7f4ef9158640 1 --2- 172.21.15.5:0/191866805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4102ef0 0x7f4ef410da40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.330+0000 7f4ef9158640 1 --2- 172.21.15.5:0/191866805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ef4102570 0x7f4ef41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.330+0000 7f4ef9158640 1 --2- 172.21.15.5:0/191866805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef4100e20 0x7f4ef4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.330+0000 7f4ef9158640 1 -- 172.21.15.5:0/191866805 >> 172.21.15.5:0/191866805 conn(0x7f4ef40fc7d0 msgr2=0x7f4ef40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.330+0000 7f4ef9158640 1 -- 172.21.15.5:0/191866805 shutdown_connections 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.330+0000 7f4ef9158640 1 -- 172.21.15.5:0/191866805 wait complete. 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.331+0000 7f4ef9158640 1 Processor -- start 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.331+0000 7f4ef9158640 1 -- start start 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef9158640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ef4100e20 0x7f4ef41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef9158640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef4102570 0x7f4ef4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef9158640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4102ef0 0x7f4ef419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef9158640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4110fd0 con 0x7f4ef4102ef0 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef9158640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4110e50 con 0x7f4ef4100e20 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef9158640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4111150 con 0x7f4ef4102570 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4102ef0 0x7f4ef419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4102ef0 0x7f4ef419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45446/0 (socket says 172.21.15.5:45446) 2024-09-11T22:21:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef359e640 1 -- 172.21.15.5:0/451205030 learned_addr learned my addr 172.21.15.5:0/451205030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef2d9d640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ef4100e20 0x7f4ef41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef259c640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef4102570 0x7f4ef4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef359e640 1 -- 172.21.15.5:0/451205030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef4102570 msgr2=0x7f4ef4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef359e640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef4102570 0x7f4ef4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef359e640 1 -- 172.21.15.5:0/451205030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ef4100e20 msgr2=0x7f4ef41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef359e640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ef4100e20 0x7f4ef41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef359e640 1 -- 172.21.15.5:0/451205030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef419d390 con 0x7f4ef4102ef0 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef259c640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef4102570 0x7f4ef4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4ef359e640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4102ef0 0x7f4ef419ccb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009dc0 tx=0x7f4eec003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.332+0000 7f4edbfff640 1 -- 172.21.15.5:0/451205030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec034040 con 0x7f4ef4102ef0 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.333+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef419d620 con 0x7f4ef4102ef0 2024-09-11T22:21:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.333+0000 7f4edbfff640 1 -- 172.21.15.5:0/451205030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4eec02fe10 con 0x7f4ef4102ef0 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.333+0000 7f4edbfff640 1 -- 172.21.15.5:0/451205030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec004740 con 0x7f4ef4102ef0 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.333+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef41a4f60 con 0x7f4ef4102ef0 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.334+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ef406aad0 con 0x7f4ef4102ef0 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.335+0000 7f4edbfff640 1 -- 172.21.15.5:0/451205030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4eec037020 con 0x7f4ef4102ef0 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.335+0000 7f4edbfff640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4edc0a2c10 0x7f4edc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.336+0000 7f4ef2d9d640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4edc0a2c10 0x7f4edc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.336+0000 7f4edbfff640 1 -- 172.21.15.5:0/451205030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f4eec0effa0 con 0x7f4ef4102ef0 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.336+0000 7f4edbfff640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] conn(0x7f4edc0ac460 0x7f4edc0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.336+0000 7f4edbfff640 1 -- 172.21.15.5:0/451205030 --> [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4eec02c030 con 0x7f4edc0ac460 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.336+0000 7f4edbfff640 1 -- 172.21.15.5:0/451205030 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f4eec0f0480 con 0x7f4ef4102ef0 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.336+0000 7f4ef2d9d640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4edc0a2c10 0x7f4edc0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400abe0 tx=0x7f4ee400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.336+0000 7f4ef259c640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] conn(0x7f4edc0ac460 0x7f4edc0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.337+0000 7f4ef259c640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] conn(0x7f4edc0ac460 0x7f4edc0ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.337+0000 7f4edbfff640 1 -- 172.21.15.5:0/451205030 <== osd.7 v2:172.21.15.159:6816/3702567808 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eec02c030 con 0x7f4edc0ac460 2024-09-11T22:21:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.352+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 --> [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ef41021b0 con 0x7f4edc0ac460 2024-09-11T22:21:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.352+0000 7f4edbfff640 1 -- 172.21.15.5:0/451205030 <== osd.7 v2:172.21.15.159:6816/3702567808 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4ef41021b0 con 0x7f4edc0ac460 2024-09-11T22:21:03.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.352+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 >> [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] conn(0x7f4edc0ac460 msgr2=0x7f4edc0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:03.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.352+0000 7f4ef9158640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] conn(0x7f4edc0ac460 0x7f4edc0ae880 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:03.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.352+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4edc0a2c10 msgr2=0x7f4edc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:03.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.353+0000 7f4ef9158640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4edc0a2c10 0x7f4edc0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400abe0 tx=0x7f4ee400a2d0 comp rx=0 tx=0).stop 2024-09-11T22:21:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.353+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4102ef0 msgr2=0x7f4ef419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.353+0000 7f4ef9158640 1 --2- 172.21.15.5:0/451205030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ef4102ef0 0x7f4ef419ccb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009dc0 tx=0x7f4eec003d00 comp rx=0 tx=0).stop 2024-09-11T22:21:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.353+0000 7f4ef359e640 1 -- 172.21.15.5:0/451205030 reap_dead start 2024-09-11T22:21:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.353+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 shutdown_connections 2024-09-11T22:21:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.353+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 >> 172.21.15.5:0/451205030 conn(0x7f4ef40fc7d0 msgr2=0x7f4ef406a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.353+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 shutdown_connections 2024-09-11T22:21:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:03.353+0000 7f4ef9158640 1 -- 172.21.15.5:0/451205030 wait complete. 2024-09-11T22:21:04.113 INFO:teuthology.orchestra.run.smithi005.stdout:184683593775 2024-09-11T22:21:04.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T22:21:04.250 INFO:teuthology.orchestra.run.smithi005.stdout:287762808860 2024-09-11T22:21:04.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-11T22:21:04.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:04 smithi005 ceph-mon[30943]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T22:21:04.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:04 smithi172 ceph-mon[35461]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T22:21:04.704 INFO:teuthology.orchestra.run.smithi005.stdout:133143986234 2024-09-11T22:21:04.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T22:21:04.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:04 smithi159 ceph-mon[35534]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T22:21:05.318 INFO:teuthology.orchestra.run.smithi005.stdout:206158430252 2024-09-11T22:21:05.319 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T22:21:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.508+0000 7f1a50e82640 1 -- 172.21.15.5:0/227267750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c101380 msgr2=0x7f1a4c10dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.508+0000 7f1a50e82640 1 --2- 172.21.15.5:0/227267750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c101380 0x7f1a4c10dfb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009900 tx=0x7f1a3402a410 comp rx=0 tx=0).stop 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.508+0000 7f1a50e82640 1 -- 172.21.15.5:0/227267750 shutdown_connections 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.508+0000 7f1a50e82640 1 --2- 172.21.15.5:0/227267750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a4c10e4f0 0x7f1a4c110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.508+0000 7f1a50e82640 1 --2- 172.21.15.5:0/227267750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c101380 0x7f1a4c10dfb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.508+0000 7f1a50e82640 1 --2- 172.21.15.5:0/227267750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a4c1009d0 0x7f1a4c100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.508+0000 7f1a50e82640 1 -- 172.21.15.5:0/227267750 >> 172.21.15.5:0/227267750 conn(0x7f1a4c0fc7d0 msgr2=0x7f1a4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.508+0000 7f1a50e82640 1 -- 172.21.15.5:0/227267750 shutdown_connections 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.508+0000 7f1a50e82640 1 -- 172.21.15.5:0/227267750 wait complete. 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a50e82640 1 Processor -- start 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a50e82640 1 -- start start 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a50e82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c1009d0 0x7f1a4c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a50e82640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a4c101380 0x7f1a4c19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a50e82640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a4c10e4f0 0x7f1a4c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a50e82640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c114240 con 0x7f1a4c1009d0 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a50e82640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c1140c0 con 0x7f1a4c10e4f0 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a50e82640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c1143c0 con 0x7f1a4c101380 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c1009d0 0x7f1a4c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c1009d0 0x7f1a4c19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45472/0 (socket says 172.21.15.5:45472) 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a4a59c640 1 -- 172.21.15.5:0/1996426075 learned_addr learned my addr 172.21.15.5:0/1996426075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.509+0000 7f1a4ad9d640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a4c10e4f0 0x7f1a4c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a4a59c640 1 -- 172.21.15.5:0/1996426075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a4c101380 msgr2=0x7f1a4c19ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a4a59c640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a4c101380 0x7f1a4c19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a4a59c640 1 -- 172.21.15.5:0/1996426075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a4c10e4f0 msgr2=0x7f1a4c19dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a4a59c640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a4c10e4f0 0x7f1a4c19dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a4a59c640 1 -- 172.21.15.5:0/1996426075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c19e6f0 con 0x7f1a4c1009d0 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a4ad9d640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a4c10e4f0 0x7f1a4c19dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a4a59c640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c1009d0 0x7f1a4c19a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00d950 tx=0x7f1a3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a437fe640 1 -- 172.21.15.5:0/1996426075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c017070 con 0x7f1a4c1009d0 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c19e9e0 con 0x7f1a4c1009d0 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a437fe640 1 -- 172.21.15.5:0/1996426075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a3c009c30 con 0x7f1a4c1009d0 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a437fe640 1 -- 172.21.15.5:0/1996426075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00f6b0 con 0x7f1a4c1009d0 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.510+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c1a7350 con 0x7f1a4c1009d0 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.511+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a4c062900 con 0x7f1a4c1009d0 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.512+0000 7f1a437fe640 1 -- 172.21.15.5:0/1996426075 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1a3c0109d0 con 0x7f1a4c1009d0 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.513+0000 7f1a437fe640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1a380a2c10 0x7f1a380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.513+0000 7f1a49d9b640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1a380a2c10 0x7f1a380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.513+0000 7f1a437fe640 1 -- 172.21.15.5:0/1996426075 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1a3c0cf170 con 0x7f1a4c1009d0 2024-09-11T22:21:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.513+0000 7f1a437fe640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] conn(0x7f1a380ac460 0x7f1a380ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.513+0000 7f1a437fe640 1 -- 172.21.15.5:0/1996426075 --> [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a380aef30 con 0x7f1a380ac460 2024-09-11T22:21:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.513+0000 7f1a437fe640 1 -- 172.21.15.5:0/1996426075 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f1a3c0d6050 con 0x7f1a4c1009d0 2024-09-11T22:21:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.514+0000 7f1a4ad9d640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] conn(0x7f1a380ac460 0x7f1a380ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.514+0000 7f1a49d9b640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1a380a2c10 0x7f1a380a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1a3402a930 tx=0x7f1a34000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.514+0000 7f1a4ad9d640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] conn(0x7f1a380ac460 0x7f1a380ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.515+0000 7f1a437fe640 1 -- 172.21.15.5:0/1996426075 <== osd.0 v2:172.21.15.5:6802/26218432 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a380aef30 con 0x7f1a380ac460 2024-09-11T22:21:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.537+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 --> [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a4c062d80 con 0x7f1a380ac460 2024-09-11T22:21:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.538+0000 7f1a437fe640 1 -- 172.21.15.5:0/1996426075 <== osd.0 v2:172.21.15.5:6802/26218432 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1a4c062d80 con 0x7f1a380ac460 2024-09-11T22:21:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.538+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] conn(0x7f1a380ac460 msgr2=0x7f1a380ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.538+0000 7f1a50e82640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] conn(0x7f1a380ac460 0x7f1a380ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.538+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1a380a2c10 msgr2=0x7f1a380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.538+0000 7f1a50e82640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f1a380a2c10 0x7f1a380a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1a3402a930 tx=0x7f1a34000bc0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.538+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c1009d0 msgr2=0x7f1a4c19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.538+0000 7f1a50e82640 1 --2- 172.21.15.5:0/1996426075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a4c1009d0 0x7f1a4c19a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00d950 tx=0x7f1a3c00de10 comp rx=0 tx=0).stop 2024-09-11T22:21:05.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.538+0000 7f1a4ad9d640 1 -- 172.21.15.5:0/1996426075 reap_dead start 2024-09-11T22:21:05.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:05 smithi005 ceph-mon[30943]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T22:21:05.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.539+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 shutdown_connections 2024-09-11T22:21:05.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.539+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 >> 172.21.15.5:0/1996426075 conn(0x7f1a4c0fc7d0 msgr2=0x7f1a4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:05.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.539+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 shutdown_connections 2024-09-11T22:21:05.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.540+0000 7f1a50e82640 1 -- 172.21.15.5:0/1996426075 wait complete. 2024-09-11T22:21:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:05 smithi159 ceph-mon[35534]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T22:21:05.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:05 smithi172 ceph-mon[35461]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-11T22:21:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.934+0000 7f17ca871640 1 -- 172.21.15.5:0/3638225040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17c4074620 msgr2=0x7f17c4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.934+0000 7f17ca871640 1 --2- 172.21.15.5:0/3638225040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17c4074620 0x7f17c4074a00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009900 tx=0x7f17b402a480 comp rx=0 tx=0).stop 2024-09-11T22:21:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.934+0000 7f17ca871640 1 -- 172.21.15.5:0/3638225040 shutdown_connections 2024-09-11T22:21:05.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.934+0000 7f17ca871640 1 --2- 172.21.15.5:0/3638225040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17c410e3c0 0x7f17c4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.934+0000 7f17ca871640 1 --2- 172.21.15.5:0/3638225040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17c4074f40 0x7f17c410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.934+0000 7f17ca871640 1 --2- 172.21.15.5:0/3638225040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17c4074620 0x7f17c4074a00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.934+0000 7f17ca871640 1 -- 172.21.15.5:0/3638225040 >> 172.21.15.5:0/3638225040 conn(0x7f17c40fd8c0 msgr2=0x7f17c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.934+0000 7f17ca871640 1 -- 172.21.15.5:0/3638225040 shutdown_connections 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.934+0000 7f17ca871640 1 -- 172.21.15.5:0/3638225040 wait complete. 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17ca871640 1 Processor -- start 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17ca871640 1 -- start start 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17ca871640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17c4074620 0x7f17c419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17ca871640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17c4074f40 0x7f17c419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17ca871640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17c410e3c0 0x7f17c419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17ca871640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17c41141c0 con 0x7f17c4074f40 2024-09-11T22:21:05.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17ca871640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4114040 con 0x7f17c4074620 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17ca871640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17c4114340 con 0x7f17c410e3c0 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17c4074f40 0x7f17c419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17c4074f40 0x7f17c419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45508/0 (socket says 172.21.15.5:45508) 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17c37fe640 1 -- 172.21.15.5:0/645150668 learned_addr learned my addr 172.21.15.5:0/645150668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17c3fff640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17c4074620 0x7f17c419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17c37fe640 1 -- 172.21.15.5:0/645150668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17c410e3c0 msgr2=0x7f17c419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17c37fe640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17c410e3c0 0x7f17c419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17c37fe640 1 -- 172.21.15.5:0/645150668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17c4074620 msgr2=0x7f17c419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17c37fe640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17c4074620 0x7f17c419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17c37fe640 1 -- 172.21.15.5:0/645150668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17c419f680 con 0x7f17c4074f40 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.935+0000 7f17c3fff640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17c4074620 0x7f17c419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.936+0000 7f17c37fe640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17c4074f40 0x7f17c419ac10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b4a0 tx=0x7f17b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:05.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.936+0000 7f17c17fa640 1 -- 172.21.15.5:0/645150668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8014070 con 0x7f17c4074f40 2024-09-11T22:21:05.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.936+0000 7f17c17fa640 1 -- 172.21.15.5:0/645150668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17b8010af0 con 0x7f17c4074f40 2024-09-11T22:21:05.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.936+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17c419f970 con 0x7f17c4074f40 2024-09-11T22:21:05.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.936+0000 7f17c17fa640 1 -- 172.21.15.5:0/645150668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8017d10 con 0x7f17c4074f40 2024-09-11T22:21:05.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.936+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17c41a7250 con 0x7f17c4074f40 2024-09-11T22:21:05.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.937+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17c4062980 con 0x7f17c4074f40 2024-09-11T22:21:05.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.938+0000 7f17c17fa640 1 -- 172.21.15.5:0/645150668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f17b8017070 con 0x7f17c4074f40 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.939+0000 7f17c17fa640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f17ac0a2c10 0x7f17ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.939+0000 7f17c3fff640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f17ac0a2c10 0x7f17ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.939+0000 7f17c17fa640 1 -- 172.21.15.5:0/645150668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f17b80ce760 con 0x7f17c4074f40 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.939+0000 7f17c17fa640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] conn(0x7f17ac0ac460 0x7f17ac0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.939+0000 7f17c17fa640 1 -- 172.21.15.5:0/645150668 --> [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17ac0aef30 con 0x7f17ac0ac460 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.939+0000 7f17c17fa640 1 -- 172.21.15.5:0/645150668 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f17b80cec40 con 0x7f17c4074f40 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.939+0000 7f17c3fff640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f17ac0a2c10 0x7f17ac0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000c00 tx=0x7f17b4032c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.940+0000 7f17c8e0e640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] conn(0x7f17ac0ac460 0x7f17ac0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.940+0000 7f17c8e0e640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] conn(0x7f17ac0ac460 0x7f17ac0ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.941+0000 7f17c17fa640 1 -- 172.21.15.5:0/645150668 <== osd.5 v2:172.21.15.172:6808/1387739113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17ac0aef30 con 0x7f17ac0ac460 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.954+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 --> [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f17c4062e00 con 0x7f17ac0ac460 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.955+0000 7f17c17fa640 1 -- 172.21.15.5:0/645150668 <== osd.5 v2:172.21.15.172:6808/1387739113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f17c4062e00 con 0x7f17ac0ac460 2024-09-11T22:21:05.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.955+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 >> [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] conn(0x7f17ac0ac460 msgr2=0x7f17ac0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.955+0000 7f17ca871640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] conn(0x7f17ac0ac460 0x7f17ac0ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.955+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f17ac0a2c10 msgr2=0x7f17ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.955+0000 7f17ca871640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f17ac0a2c10 0x7f17ac0a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17b4000c00 tx=0x7f17b4032c70 comp rx=0 tx=0).stop 2024-09-11T22:21:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.955+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17c4074f40 msgr2=0x7f17c419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.955+0000 7f17ca871640 1 --2- 172.21.15.5:0/645150668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17c4074f40 0x7f17c419ac10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b4a0 tx=0x7f17b800b960 comp rx=0 tx=0).stop 2024-09-11T22:21:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.955+0000 7f17c8e0e640 1 -- 172.21.15.5:0/645150668 reap_dead start 2024-09-11T22:21:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.955+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 shutdown_connections 2024-09-11T22:21:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.955+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 >> 172.21.15.5:0/645150668 conn(0x7f17c40fd8c0 msgr2=0x7f17c40ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.956+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 shutdown_connections 2024-09-11T22:21:05.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.956+0000 7f17ca871640 1 -- 172.21.15.5:0/645150668 wait complete. 2024-09-11T22:21:06.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 -- 172.21.15.5:0/1469698704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef5c102570 msgr2=0x7fef5c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 --2- 172.21.15.5:0/1469698704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef5c102570 0x7fef5c1029b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009900 tx=0x7fef4c02a480 comp rx=0 tx=0).stop 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 -- 172.21.15.5:0/1469698704 shutdown_connections 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 --2- 172.21.15.5:0/1469698704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef5c102ef0 0x7fef5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 --2- 172.21.15.5:0/1469698704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef5c102570 0x7fef5c1029b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 --2- 172.21.15.5:0/1469698704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5c100e20 0x7fef5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 -- 172.21.15.5:0/1469698704 >> 172.21.15.5:0/1469698704 conn(0x7fef5c0fc7d0 msgr2=0x7fef5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 -- 172.21.15.5:0/1469698704 shutdown_connections 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 -- 172.21.15.5:0/1469698704 wait complete. 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 Processor -- start 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.823+0000 7fef647b7640 1 -- start start 2024-09-11T22:21:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef647b7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef5c100e20 0x7fef5c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef647b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef5c102570 0x7fef5c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef647b7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5c102ef0 0x7fef5c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef647b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c110fd0 con 0x7fef5c102570 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef647b7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c110e50 con 0x7fef5c102ef0 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef647b7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c111150 con 0x7fef5c100e20 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef61d52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef5c102570 0x7fef5c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef61d52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef5c102570 0x7fef5c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45496/0 (socket says 172.21.15.5:45496) 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef61d52640 1 -- 172.21.15.5:0/401726037 learned_addr learned my addr 172.21.15.5:0/401726037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef62d54640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5c102ef0 0x7fef5c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef62553640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef5c100e20 0x7fef5c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef61d52640 1 -- 172.21.15.5:0/401726037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef5c100e20 msgr2=0x7fef5c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef61d52640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef5c100e20 0x7fef5c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef61d52640 1 -- 172.21.15.5:0/401726037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5c102ef0 msgr2=0x7fef5c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef61d52640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef5c102ef0 0x7fef5c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef61d52640 1 -- 172.21.15.5:0/401726037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c1a69c0 con 0x7fef5c102570 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef61d52640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef5c102570 0x7fef5c0fd1c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef4c02aa70 tx=0x7fef4c003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef537fe640 1 -- 172.21.15.5:0/401726037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c03f070 con 0x7fef5c102570 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c1a6c50 con 0x7fef5c102570 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef537fe640 1 -- 172.21.15.5:0/401726037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef4c031c20 con 0x7fef5c102570 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef537fe640 1 -- 172.21.15.5:0/401726037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c035030 con 0x7fef5c102570 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.824+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c1a7130 con 0x7fef5c102570 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.825+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef5c062940 con 0x7fef5c102570 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.827+0000 7fef537fe640 1 -- 172.21.15.5:0/401726037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fef4c042070 con 0x7fef5c102570 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.827+0000 7fef537fe640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fef480a2c10 0x7fef480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.827+0000 7fef62553640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fef480a2c10 0x7fef480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.827+0000 7fef537fe640 1 -- 172.21.15.5:0/401726037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fef4c0efe10 con 0x7fef5c102570 2024-09-11T22:21:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.828+0000 7fef537fe640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] conn(0x7fef480ac460 0x7fef480ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.828+0000 7fef537fe640 1 -- 172.21.15.5:0/401726037 --> [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef4c038e90 con 0x7fef480ac460 2024-09-11T22:21:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.828+0000 7fef537fe640 1 -- 172.21.15.5:0/401726037 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fef4c0f02f0 con 0x7fef5c102570 2024-09-11T22:21:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.828+0000 7fef62553640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fef480a2c10 0x7fef480a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef580058c0 tx=0x7fef58005c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.828+0000 7fef62d54640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] conn(0x7fef480ac460 0x7fef480ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.828+0000 7fef62d54640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] conn(0x7fef480ac460 0x7fef480ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.829+0000 7fef537fe640 1 -- 172.21.15.5:0/401726037 <== osd.2 v2:172.21.15.172:6800/34708812 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef4c038e90 con 0x7fef480ac460 2024-09-11T22:21:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.843+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 --> [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef5c062dc0 con 0x7fef480ac460 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.843+0000 7fef537fe640 1 -- 172.21.15.5:0/401726037 <== osd.2 v2:172.21.15.172:6800/34708812 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fef5c062dc0 con 0x7fef480ac460 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 >> [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] conn(0x7fef480ac460 msgr2=0x7fef480ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef647b7640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] conn(0x7fef480ac460 0x7fef480ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fef480a2c10 msgr2=0x7fef480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef647b7640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fef480a2c10 0x7fef480a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef580058c0 tx=0x7fef58005c60 comp rx=0 tx=0).stop 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef5c102570 msgr2=0x7fef5c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef647b7640 1 --2- 172.21.15.5:0/401726037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef5c102570 0x7fef5c0fd1c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef4c02aa70 tx=0x7fef4c003b60 comp rx=0 tx=0).stop 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef62d54640 1 -- 172.21.15.5:0/401726037 reap_dead start 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 shutdown_connections 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 >> 172.21.15.5:0/401726037 conn(0x7fef5c0fc7d0 msgr2=0x7fef5c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 shutdown_connections 2024-09-11T22:21:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:05.844+0000 7fef647b7640 1 -- 172.21.15.5:0/401726037 wait complete. 2024-09-11T22:21:07.297 INFO:teuthology.orchestra.run.smithi005.stdout:236223201317 2024-09-11T22:21:07.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-11T22:21:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:07 smithi005 ceph-mon[30943]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:21:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:07.753 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:07.782 INFO:teuthology.orchestra.run.smithi005.stdout:34359738448 2024-09-11T22:21:07.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T22:21:07.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:07 smithi159 ceph-mon[35534]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:21:07.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:07 smithi172 ceph-mon[35461]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-11T22:21:08.178 INFO:teuthology.orchestra.run.smithi005.stdout:154618822710 2024-09-11T22:21:08.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T22:21:08.599 INFO:teuthology.orchestra.run.smithi005.stdout:317827579926 2024-09-11T22:21:08.600 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-11T22:21:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:09.057 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:09.064 INFO:teuthology.orchestra.run.smithi005.stdout:77309411398 2024-09-11T22:21:09.064 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T22:21:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.567+0000 7fa23bb35640 1 -- 172.21.15.5:0/3055605268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa234106f10 msgr2=0x7fa23410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.567+0000 7fa23bb35640 1 --2- 172.21.15.5:0/3055605268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa234106f10 0x7fa23410ea90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009ca0 tx=0x7fa22c02a8f0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.567+0000 7fa23bb35640 1 -- 172.21.15.5:0/3055605268 shutdown_connections 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.567+0000 7fa23bb35640 1 --2- 172.21.15.5:0/3055605268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa234106f10 0x7fa23410ea90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.567+0000 7fa23bb35640 1 --2- 172.21.15.5:0/3055605268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa234102790 0x7fa2341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.567+0000 7fa23bb35640 1 --2- 172.21.15.5:0/3055605268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa234101de0 0x7fa2341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.567+0000 7fa23bb35640 1 -- 172.21.15.5:0/3055605268 >> 172.21.15.5:0/3055605268 conn(0x7fa2340fd8c0 msgr2=0x7fa2340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.567+0000 7fa23bb35640 1 -- 172.21.15.5:0/3055605268 shutdown_connections 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.567+0000 7fa23bb35640 1 -- 172.21.15.5:0/3055605268 wait complete. 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa23bb35640 1 Processor -- start 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa23bb35640 1 -- start start 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa23bb35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa234101de0 0x7fa2340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa23bb35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa234102790 0x7fa234074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa23bb35640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa234106f10 0x7fa234074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa23bb35640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa234112050 con 0x7fa234102790 2024-09-11T22:21:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa23bb35640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa234111ed0 con 0x7fa234106f10 2024-09-11T22:21:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa23bb35640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa2341121d0 con 0x7fa234101de0 2024-09-11T22:21:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa2390d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa234102790 0x7fa234074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa2390d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa234102790 0x7fa234074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45536/0 (socket says 172.21.15.5:45536) 2024-09-11T22:21:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.568+0000 7fa2390d0640 1 -- 172.21.15.5:0/1490141095 learned_addr learned my addr 172.21.15.5:0/1490141095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa2390d0640 1 -- 172.21.15.5:0/1490141095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa234101de0 msgr2=0x7fa2340744f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:21:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa2390d0640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa234101de0 0x7fa2340744f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa2390d0640 1 -- 172.21.15.5:0/1490141095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa234106f10 msgr2=0x7fa234074f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:21:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa2390d0640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa234106f10 0x7fa234074f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa2390d0640 1 -- 172.21.15.5:0/1490141095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2341a6ad0 con 0x7fa234102790 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa2390d0640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa234102790 0x7fa234074a30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa22400b450 tx=0x7fa22400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa22affd640 1 -- 172.21.15.5:0/1490141095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224015020 con 0x7fa234102790 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa22affd640 1 -- 172.21.15.5:0/1490141095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa224011d70 con 0x7fa234102790 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa23bb35640 1 -- 172.21.15.5:0/1490141095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2341a6dc0 con 0x7fa234102790 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa22affd640 1 -- 172.21.15.5:0/1490141095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22400f340 con 0x7fa234102790 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.569+0000 7fa23bb35640 1 -- 172.21.15.5:0/1490141095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2341a72d0 con 0x7fa234102790 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.570+0000 7fa23bb35640 1 -- 172.21.15.5:0/1490141095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa234102240 con 0x7fa234102790 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.571+0000 7fa22affd640 1 -- 172.21.15.5:0/1490141095 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa224018020 con 0x7fa234102790 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.572+0000 7fa22affd640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa2200a2c90 0x7fa2200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.572+0000 7fa2398d1640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa2200a2c90 0x7fa2200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.572+0000 7fa22affd640 1 -- 172.21.15.5:0/1490141095 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa2240cea40 con 0x7fa234102790 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.573+0000 7fa2398d1640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa2200a2c90 0x7fa2200a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa230009670 tx=0x7fa230006bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.577+0000 7fa22affd640 1 -- 172.21.15.5:0/1490141095 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa22401f020 con 0x7fa234102790 2024-09-11T22:21:09.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.754+0000 7fa23bb35640 1 -- 172.21.15.5:0/1490141095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa234109bc0 con 0x7fa234102790 2024-09-11T22:21:09.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:09 smithi005 ceph-mon[30943]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:21:09.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.755+0000 7fa22affd640 1 -- 172.21.15.5:0/1490141095 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa22400d040 con 0x7fa234102790 2024-09-11T22:21:09.756 INFO:teuthology.orchestra.run.smithi005.stdout:103079215169 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 -- 172.21.15.5:0/1490141095 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa2200a2c90 msgr2=0x7fa2200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa2200a2c90 0x7fa2200a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa230009670 tx=0x7fa230006bd0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 -- 172.21.15.5:0/1490141095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa234102790 msgr2=0x7fa234074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa234102790 0x7fa234074a30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa22400b450 tx=0x7fa22400b910 comp rx=0 tx=0).stop 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 -- 172.21.15.5:0/1490141095 shutdown_connections 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fa2200a2c90 0x7fa2200a5150 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa234106f10 0x7fa234074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa234102790 0x7fa234074a30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 --2- 172.21.15.5:0/1490141095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa234101de0 0x7fa2340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 -- 172.21.15.5:0/1490141095 >> 172.21.15.5:0/1490141095 conn(0x7fa2340fd8c0 msgr2=0x7fa2340fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:09.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 -- 172.21.15.5:0/1490141095 shutdown_connections 2024-09-11T22:21:09.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:09.757+0000 7fa23bb35640 1 -- 172.21.15.5:0/1490141095 wait complete. 2024-09-11T22:21:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:09 smithi159 ceph-mon[35534]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:21:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:09 smithi172 ceph-mon[35461]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-11T22:21:10.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.507+0000 7f5f8dca4640 1 -- 172.21.15.5:0/275054831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88103660 msgr2=0x7f5f8810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:10.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.507+0000 7f5f8dca4640 1 --2- 172.21.15.5:0/275054831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88103660 0x7f5f8810bb50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009900 tx=0x7f5f7c02a480 comp rx=0 tx=0).stop 2024-09-11T22:21:10.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.507+0000 7f5f8dca4640 1 -- 172.21.15.5:0/275054831 shutdown_connections 2024-09-11T22:21:10.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.507+0000 7f5f8dca4640 1 --2- 172.21.15.5:0/275054831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f8810c170 0x7f5f8810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.507+0000 7f5f8dca4640 1 --2- 172.21.15.5:0/275054831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88103660 0x7f5f8810bb50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.507+0000 7f5f8dca4640 1 --2- 172.21.15.5:0/275054831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f88102d40 0x7f5f88103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.507+0000 7f5f8dca4640 1 -- 172.21.15.5:0/275054831 >> 172.21.15.5:0/275054831 conn(0x7f5f880fc7a0 msgr2=0x7f5f880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:10.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.507+0000 7f5f8dca4640 1 -- 172.21.15.5:0/275054831 shutdown_connections 2024-09-11T22:21:10.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.507+0000 7f5f8dca4640 1 -- 172.21.15.5:0/275054831 wait complete. 2024-09-11T22:21:10.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.508+0000 7f5f8dca4640 1 Processor -- start 2024-09-11T22:21:10.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.508+0000 7f5f8dca4640 1 -- start start 2024-09-11T22:21:10.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.508+0000 7f5f8dca4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f88102d40 0x7f5f881009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:10.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.508+0000 7f5f8dca4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88103660 0x7f5f88100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.508+0000 7f5f8dca4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f8810c170 0x7f5f88101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.508+0000 7f5f8dca4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f88111f70 con 0x7f5f88103660 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.508+0000 7f5f8dca4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5f88111df0 con 0x7f5f88102d40 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.508+0000 7f5f8dca4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5f881120f0 con 0x7f5f8810c170 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f86ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88103660 0x7f5f88100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f86ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88103660 0x7f5f88100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45556/0 (socket says 172.21.15.5:45556) 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f86ffd640 1 -- 172.21.15.5:0/904152475 learned_addr learned my addr 172.21.15.5:0/904152475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f877fe640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f88102d40 0x7f5f881009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f87fff640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f8810c170 0x7f5f88101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f86ffd640 1 -- 172.21.15.5:0/904152475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f8810c170 msgr2=0x7f5f88101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f86ffd640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f8810c170 0x7f5f88101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f86ffd640 1 -- 172.21.15.5:0/904152475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f88102d40 msgr2=0x7f5f881009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f86ffd640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f88102d40 0x7f5f881009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f86ffd640 1 -- 172.21.15.5:0/904152475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f881a68d0 con 0x7f5f88103660 2024-09-11T22:21:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f877fe640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f88102d40 0x7f5f881009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f86ffd640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88103660 0x7f5f88100ee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c005ab0 tx=0x7f5f7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f84ff9640 1 -- 172.21.15.5:0/904152475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c03f030 con 0x7f5f88103660 2024-09-11T22:21:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f84ff9640 1 -- 172.21.15.5:0/904152475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f7c003ce0 con 0x7f5f88103660 2024-09-11T22:21:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f8dca4640 1 -- 172.21.15.5:0/904152475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f881a6b60 con 0x7f5f88103660 2024-09-11T22:21:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f84ff9640 1 -- 172.21.15.5:0/904152475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c035070 con 0x7f5f88103660 2024-09-11T22:21:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.509+0000 7f5f8dca4640 1 -- 172.21.15.5:0/904152475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f881a7040 con 0x7f5f88103660 2024-09-11T22:21:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.510+0000 7f5f8dca4640 1 -- 172.21.15.5:0/904152475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f88103120 con 0x7f5f88103660 2024-09-11T22:21:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.512+0000 7f5f84ff9640 1 -- 172.21.15.5:0/904152475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5f7c042030 con 0x7f5f88103660 2024-09-11T22:21:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.512+0000 7f5f84ff9640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5f700a2c10 0x7f5f700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.512+0000 7f5f877fe640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5f700a2c10 0x7f5f700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.513+0000 7f5f84ff9640 1 -- 172.21.15.5:0/904152475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f5f7c0ba230 con 0x7f5f88103660 2024-09-11T22:21:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.513+0000 7f5f877fe640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5f700a2c10 0x7f5f700a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800abe0 tx=0x7f5f7800a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:10.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.517+0000 7f5f84ff9640 1 -- 172.21.15.5:0/904152475 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f7c0bc890 con 0x7f5f88103660 2024-09-11T22:21:10.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.697+0000 7f5f8dca4640 1 -- 172.21.15.5:0/904152475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f5f881071e0 con 0x7f5f88103660 2024-09-11T22:21:10.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:10 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1490141095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T22:21:10.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.698+0000 7f5f84ff9640 1 -- 172.21.15.5:0/904152475 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5f7c0bc190 con 0x7f5f88103660 2024-09-11T22:21:10.699 INFO:teuthology.orchestra.run.smithi005.stdout:266287972386 2024-09-11T22:21:10.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 -- 172.21.15.5:0/904152475 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5f700a2c10 msgr2=0x7f5f700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:10.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5f700a2c10 0x7f5f700a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800abe0 tx=0x7f5f7800a5c0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 -- 172.21.15.5:0/904152475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88103660 msgr2=0x7f5f88100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:10.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88103660 0x7f5f88100ee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c005ab0 tx=0x7f5f7c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 -- 172.21.15.5:0/904152475 shutdown_connections 2024-09-11T22:21:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5f700a2c10 0x7f5f700a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f8810c170 0x7f5f88101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f88103660 0x7f5f88100ee0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 --2- 172.21.15.5:0/904152475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5f88102d40 0x7f5f881009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:10.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 -- 172.21.15.5:0/904152475 >> 172.21.15.5:0/904152475 conn(0x7f5f880fc7a0 msgr2=0x7f5f880fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:10.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 -- 172.21.15.5:0/904152475 shutdown_connections 2024-09-11T22:21:10.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:10.700+0000 7f5f8dca4640 1 -- 172.21.15.5:0/904152475 wait complete. 2024-09-11T22:21:10.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:10 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1490141095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T22:21:10.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:10 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1490141095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T22:21:11.090 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215167 got 103079215169 for osd.3 2024-09-11T22:21:11.090 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:11.581 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:11 smithi005 ceph-mon[30943]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T22:21:11.581 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:11 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/904152475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T22:21:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:11 smithi159 ceph-mon[35534]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T22:21:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:11 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/904152475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T22:21:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.834+0000 7f0c0c993640 1 -- 172.21.15.5:0/3373567019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c08102d30 msgr2=0x7f0c08103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.834+0000 7f0c0c993640 1 --2- 172.21.15.5:0/3373567019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c08102d30 0x7f0c08103110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8009930 tx=0x7f0bf802a420 comp rx=0 tx=0).stop 2024-09-11T22:21:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.834+0000 7f0c0c993640 1 -- 172.21.15.5:0/3373567019 shutdown_connections 2024-09-11T22:21:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.834+0000 7f0c0c993640 1 --2- 172.21.15.5:0/3373567019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c081082a0 0x7f0c0810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.834+0000 7f0c0c993640 1 --2- 172.21.15.5:0/3373567019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c081036e0 0x7f0c08107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.834+0000 7f0c0c993640 1 --2- 172.21.15.5:0/3373567019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c08102d30 0x7f0c08103110 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.834+0000 7f0c0c993640 1 -- 172.21.15.5:0/3373567019 >> 172.21.15.5:0/3373567019 conn(0x7f0c080fc7d0 msgr2=0x7f0c080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.834+0000 7f0c0c993640 1 -- 172.21.15.5:0/3373567019 shutdown_connections 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.834+0000 7f0c0c993640 1 -- 172.21.15.5:0/3373567019 wait complete. 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c0c993640 1 Processor -- start 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c0c993640 1 -- start start 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c0c993640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c08102d30 0x7f0c08198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c0c993640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c081036e0 0x7f0c08198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c0c993640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c081082a0 0x7f0c0819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c0c993640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c081120a0 con 0x7f0c081082a0 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c0c993640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0c08111f20 con 0x7f0c08102d30 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c0c993640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c08112220 con 0x7f0c081036e0 2024-09-11T22:21:11.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c06d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c081082a0 0x7f0c0819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c06d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c081082a0 0x7f0c0819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43334/0 (socket says 172.21.15.5:43334) 2024-09-11T22:21:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c06d9d640 1 -- 172.21.15.5:0/2300420594 learned_addr learned my addr 172.21.15.5:0/2300420594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.835+0000 7f0c0659c640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c08102d30 0x7f0c08198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0c06d9d640 1 -- 172.21.15.5:0/2300420594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c081036e0 msgr2=0x7f0c08198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0c05d9b640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c081036e0 0x7f0c08198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0c06d9d640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c081036e0 0x7f0c08198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0c06d9d640 1 -- 172.21.15.5:0/2300420594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c08102d30 msgr2=0x7f0c08198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0c06d9d640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c08102d30 0x7f0c08198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0c06d9d640 1 -- 172.21.15.5:0/2300420594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0819d5d0 con 0x7f0c081082a0 2024-09-11T22:21:11.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0c0659c640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c08102d30 0x7f0c08198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0c06d9d640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c081082a0 0x7f0c0819cef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009fd0 tx=0x7f0c0000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0bf77fe640 1 -- 172.21.15.5:0/2300420594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00017070 con 0x7f0c081082a0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0c0c993640 1 -- 172.21.15.5:0/2300420594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0819d8c0 con 0x7f0c081082a0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0bf77fe640 1 -- 172.21.15.5:0/2300420594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c00003e50 con 0x7f0c081082a0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0bf77fe640 1 -- 172.21.15.5:0/2300420594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000106c0 con 0x7f0c081082a0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.836+0000 7f0c0c993640 1 -- 172.21.15.5:0/2300420594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c081a51a0 con 0x7f0c081082a0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.837+0000 7f0c0c993640 1 -- 172.21.15.5:0/2300420594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0806a850 con 0x7f0c081082a0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.838+0000 7f0bf77fe640 1 -- 172.21.15.5:0/2300420594 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0c00003970 con 0x7f0c081082a0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.839+0000 7f0bf77fe640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0bf00a2c10 0x7f0bf00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.839+0000 7f0c0659c640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0bf00a2c10 0x7f0bf00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.839+0000 7f0bf77fe640 1 -- 172.21.15.5:0/2300420594 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f0c000ceee0 con 0x7f0c081082a0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.839+0000 7f0c0659c640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0bf00a2c10 0x7f0bf00a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8000c00 tx=0x7f0bf8005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:11.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.842+0000 7f0bf77fe640 1 -- 172.21.15.5:0/2300420594 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c0009bd90 con 0x7f0c081082a0 2024-09-11T22:21:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:11 smithi172 ceph-mon[35461]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-11T22:21:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:11 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/904152475' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.716+0000 7efcd8125640 1 -- 172.21.15.5:0/4014263143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0101de0 msgr2=0x7efcd01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.716+0000 7efcd8125640 1 --2- 172.21.15.5:0/4014263143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0101de0 0x7efcd01021c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009ca0 tx=0x7efcc002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.716+0000 7efcd8125640 1 -- 172.21.15.5:0/4014263143 shutdown_connections 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.717+0000 7efcd8125640 1 --2- 172.21.15.5:0/4014263143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcd0106f10 0x7efcd010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.717+0000 7efcd8125640 1 --2- 172.21.15.5:0/4014263143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcd0102790 0x7efcd01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.717+0000 7efcd8125640 1 --2- 172.21.15.5:0/4014263143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0101de0 0x7efcd01021c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.717+0000 7efcd8125640 1 -- 172.21.15.5:0/4014263143 >> 172.21.15.5:0/4014263143 conn(0x7efcd00fd8c0 msgr2=0x7efcd00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.717+0000 7efcd8125640 1 -- 172.21.15.5:0/4014263143 shutdown_connections 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.717+0000 7efcd8125640 1 -- 172.21.15.5:0/4014263143 wait complete. 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.717+0000 7efcd8125640 1 Processor -- start 2024-09-11T22:21:11.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.717+0000 7efcd8125640 1 -- start start 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd8125640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0101de0 0x7efcd0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd8125640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcd0102790 0x7efcd01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd8125640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcd0106f10 0x7efcd019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd8125640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0111f70 con 0x7efcd0101de0 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd8125640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0111df0 con 0x7efcd0102790 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd8125640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efcd01120f0 con 0x7efcd0106f10 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd5ec1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0101de0 0x7efcd0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd5ec1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0101de0 0x7efcd0198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43328/0 (socket says 172.21.15.5:43328) 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd5ec1640 1 -- 172.21.15.5:0/3263617786 learned_addr learned my addr 172.21.15.5:0/3263617786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd66c2640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcd0106f10 0x7efcd019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd56c0640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcd0102790 0x7efcd01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:11.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd5ec1640 1 -- 172.21.15.5:0/3263617786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcd0106f10 msgr2=0x7efcd019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd5ec1640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcd0106f10 0x7efcd019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd5ec1640 1 -- 172.21.15.5:0/3263617786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcd0102790 msgr2=0x7efcd01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd5ec1640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcd0102790 0x7efcd01989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.980 INFO:teuthology.orchestra.run.smithi005.stdout:55834574924 2024-09-11T22:21:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd5ec1640 1 -- 172.21.15.5:0/3263617786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd019d440 con 0x7efcd0101de0 2024-09-11T22:21:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.718+0000 7efcd5ec1640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0101de0 0x7efcd0198490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efcc002ade0 tx=0x7efcc00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.719+0000 7efcceffd640 1 -- 172.21.15.5:0/3263617786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc002fa20 con 0x7efcd0101de0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.719+0000 7efcceffd640 1 -- 172.21.15.5:0/3263617786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc002fbc0 con 0x7efcd0101de0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.719+0000 7efcd8125640 1 -- 172.21.15.5:0/3263617786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd019d6d0 con 0x7efcd0101de0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.719+0000 7efcceffd640 1 -- 172.21.15.5:0/3263617786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc00467b0 con 0x7efcd0101de0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.719+0000 7efcd8125640 1 -- 172.21.15.5:0/3263617786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd01a5010 con 0x7efcd0101de0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.720+0000 7efcd8125640 1 -- 172.21.15.5:0/3263617786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd01021c0 con 0x7efcd0101de0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.721+0000 7efcceffd640 1 -- 172.21.15.5:0/3263617786 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7efcc0046950 con 0x7efcd0101de0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.721+0000 7efcceffd640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efcb80a2c90 0x7efcb80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.721+0000 7efcd56c0640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efcb80a2c90 0x7efcb80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.722+0000 7efcceffd640 1 -- 172.21.15.5:0/3263617786 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7efcc00f05c0 con 0x7efcd0101de0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.722+0000 7efcd56c0640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efcb80a2c90 0x7efcb80a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efcc4005fd0 tx=0x7efcc4005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.725+0000 7efcceffd640 1 -- 172.21.15.5:0/3263617786 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcc00bd710 con 0x7efcd0101de0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.904+0000 7efcd8125640 1 -- 172.21.15.5:0/3263617786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7efcd010a0f0 con 0x7efcd0101de0 2024-09-11T22:21:11.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.905+0000 7efcceffd640 1 -- 172.21.15.5:0/3263617786 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efcc00bd010 con 0x7efcd0101de0 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 -- 172.21.15.5:0/3263617786 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efcb80a2c90 msgr2=0x7efcb80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efcb80a2c90 0x7efcb80a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efcc4005fd0 tx=0x7efcc4005850 comp rx=0 tx=0).stop 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 -- 172.21.15.5:0/3263617786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0101de0 msgr2=0x7efcd0198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0101de0 0x7efcd0198490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efcc002ade0 tx=0x7efcc00319f0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 -- 172.21.15.5:0/3263617786 shutdown_connections 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7efcb80a2c90 0x7efcb80a5150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcd0106f10 0x7efcd019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcd0102790 0x7efcd01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 --2- 172.21.15.5:0/3263617786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcd0101de0 0x7efcd0198490 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 -- 172.21.15.5:0/3263617786 >> 172.21.15.5:0/3263617786 conn(0x7efcd00fd8c0 msgr2=0x7efcd00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 -- 172.21.15.5:0/3263617786 shutdown_connections 2024-09-11T22:21:11.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:11.908+0000 7efcd8125640 1 -- 172.21.15.5:0/3263617786 wait complete. 2024-09-11T22:21:12.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.028+0000 7f0c0c993640 1 -- 172.21.15.5:0/2300420594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0c0810a2b0 con 0x7f0c081082a0 2024-09-11T22:21:12.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.028+0000 7f0bf77fe640 1 -- 172.21.15.5:0/2300420594 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0c0009b690 con 0x7f0c081082a0 2024-09-11T22:21:12.029 INFO:teuthology.orchestra.run.smithi005.stdout:133143986236 2024-09-11T22:21:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 -- 172.21.15.5:0/2300420594 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0bf00a2c10 msgr2=0x7f0bf00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0bf00a2c10 0x7f0bf00a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8000c00 tx=0x7f0bf8005a20 comp rx=0 tx=0).stop 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 -- 172.21.15.5:0/2300420594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c081082a0 msgr2=0x7f0c0819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c081082a0 0x7f0c0819cef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009fd0 tx=0x7f0c0000b260 comp rx=0 tx=0).stop 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 -- 172.21.15.5:0/2300420594 shutdown_connections 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0bf00a2c10 0x7f0bf00a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c081082a0 0x7f0c0819cef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c081036e0 0x7f0c08198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 --2- 172.21.15.5:0/2300420594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c08102d30 0x7f0c08198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 -- 172.21.15.5:0/2300420594 >> 172.21.15.5:0/2300420594 conn(0x7f0c080fc7d0 msgr2=0x7f0c08105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 -- 172.21.15.5:0/2300420594 shutdown_connections 2024-09-11T22:21:12.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.030+0000 7f0c0c993640 1 -- 172.21.15.5:0/2300420594 wait complete. 2024-09-11T22:21:12.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:12 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3263617786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T22:21:12.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:12 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2300420594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T22:21:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.782+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/3112734373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f4100e20 msgr2=0x7fc1f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.782+0000 7fc1f8a5e640 1 --2- 172.21.15.5:0/3112734373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f4100e20 0x7fc1f4101200 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009900 tx=0x7fc1e402a480 comp rx=0 tx=0).stop 2024-09-11T22:21:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.782+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/3112734373 shutdown_connections 2024-09-11T22:21:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.782+0000 7fc1f8a5e640 1 --2- 172.21.15.5:0/3112734373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f4102ef0 0x7fc1f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.782+0000 7fc1f8a5e640 1 --2- 172.21.15.5:0/3112734373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1f4102570 0x7fc1f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.782+0000 7fc1f8a5e640 1 --2- 172.21.15.5:0/3112734373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f4100e20 0x7fc1f4101200 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.782+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/3112734373 >> 172.21.15.5:0/3112734373 conn(0x7fc1f40fc7d0 msgr2=0x7fc1f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:12.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.782+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/3112734373 shutdown_connections 2024-09-11T22:21:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.782+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/3112734373 wait complete. 2024-09-11T22:21:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.783+0000 7fc1f8a5e640 1 Processor -- start 2024-09-11T22:21:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.783+0000 7fc1f8a5e640 1 -- start start 2024-09-11T22:21:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f8a5e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1f4100e20 0x7fc1f40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f8a5e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f4102570 0x7fc1f40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f8a5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f4102ef0 0x7fc1f40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f8a5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f4111020 con 0x7fc1f4102ef0 2024-09-11T22:21:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f8a5e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f4110ea0 con 0x7fc1f4100e20 2024-09-11T22:21:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f8a5e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f41111a0 con 0x7fc1f4102570 2024-09-11T22:21:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f4102ef0 0x7fc1f40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f4102ef0 0x7fc1f40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43352/0 (socket says 172.21.15.5:43352) 2024-09-11T22:21:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f2d9d640 1 -- 172.21.15.5:0/1708964166 learned_addr learned my addr 172.21.15.5:0/1708964166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1ebfff640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f4102570 0x7fc1f40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f259c640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1f4100e20 0x7fc1f40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f2d9d640 1 -- 172.21.15.5:0/1708964166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f4102570 msgr2=0x7fc1f40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f2d9d640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f4102570 0x7fc1f40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f2d9d640 1 -- 172.21.15.5:0/1708964166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1f4100e20 msgr2=0x7fc1f40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f2d9d640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1f4100e20 0x7fc1f40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f2d9d640 1 -- 172.21.15.5:0/1708964166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f41a6b30 con 0x7fc1f4102ef0 2024-09-11T22:21:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.784+0000 7fc1f259c640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1f4100e20 0x7fc1f40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.785+0000 7fc1f2d9d640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f4102ef0 0x7fc1f40fd700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00b970 tx=0x7fc1ec00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.785+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1708964166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec019070 con 0x7fc1f4102ef0 2024-09-11T22:21:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.785+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1708964166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1ec014440 con 0x7fc1f4102ef0 2024-09-11T22:21:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.785+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/1708964166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f41a6e20 con 0x7fc1f4102ef0 2024-09-11T22:21:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.785+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1708964166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec0075d0 con 0x7fc1f4102ef0 2024-09-11T22:21:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.785+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/1708964166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f41a72d0 con 0x7fc1f4102ef0 2024-09-11T22:21:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.786+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/1708964166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f4101280 con 0x7fc1f4102ef0 2024-09-11T22:21:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.788+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1708964166 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc1ec01c070 con 0x7fc1f4102ef0 2024-09-11T22:21:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.788+0000 7fc1eb7fe640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc1e00a2c10 0x7fc1e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.789+0000 7fc1f259c640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc1e00a2c10 0x7fc1e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.789+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1708964166 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc1ec0ce4c0 con 0x7fc1f4102ef0 2024-09-11T22:21:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.789+0000 7fc1f259c640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc1e00a2c10 0x7fc1e00a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4005ab0 tx=0x7fc1e4005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.794+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1708964166 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1ec09b580 con 0x7fc1f4102ef0 2024-09-11T22:21:12.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:12 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3263617786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T22:21:12.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:12 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2300420594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T22:21:12.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:12 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3263617786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T22:21:12.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:12 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2300420594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T22:21:12.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.978+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/1708964166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc1f410a0a0 con 0x7fc1f4102ef0 2024-09-11T22:21:12.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.978+0000 7fc1eb7fe640 1 -- 172.21.15.5:0/1708964166 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc1ec09ae80 con 0x7fc1f4102ef0 2024-09-11T22:21:12.979 INFO:teuthology.orchestra.run.smithi005.stdout:206158430254 2024-09-11T22:21:12.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/1708964166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc1e00a2c10 msgr2=0x7fc1e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc1e00a2c10 0x7fc1e00a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4005ab0 tx=0x7fc1e4005a20 comp rx=0 tx=0).stop 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/1708964166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f4102ef0 msgr2=0x7fc1f40fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f4102ef0 0x7fc1f40fd700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec00b970 tx=0x7fc1ec00bd70 comp rx=0 tx=0).stop 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/1708964166 shutdown_connections 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc1e00a2c10 0x7fc1e00a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1f4102ef0 0x7fc1f40fd700 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc1f4102570 0x7fc1f40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 --2- 172.21.15.5:0/1708964166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc1f4100e20 0x7fc1f40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/1708964166 >> 172.21.15.5:0/1708964166 conn(0x7fc1f40fc7d0 msgr2=0x7fc1f406a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/1708964166 shutdown_connections 2024-09-11T22:21:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:12.980+0000 7fc1f8a5e640 1 -- 172.21.15.5:0/1708964166 wait complete. 2024-09-11T22:21:13.243 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574924 for osd.1 2024-09-11T22:21:13.244 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:13 smithi005 ceph-mon[30943]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-11T22:21:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:13 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1708964166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T22:21:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:13 smithi159 ceph-mon[35534]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-11T22:21:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:13 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1708964166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T22:21:13.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:13 smithi172 ceph-mon[35461]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-11T22:21:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:13 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1708964166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T22:21:13.909 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430252 got 206158430254 for osd.7 2024-09-11T22:21:13.909 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:14.330 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986234 got 133143986236 for osd.4 2024-09-11T22:21:14.330 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:14.410 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:14.411 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:14.411 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:14.411 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:14.412 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:14.412 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:14.898 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:14.939 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972383 got 266287972386 for osd.9 2024-09-11T22:21:14.939 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:15.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:15 smithi005 ceph-mon[30943]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:21:15.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:15 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:21:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:15 smithi159 ceph-mon[35534]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:21:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:15 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:21:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:15 smithi172 ceph-mon[35461]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:21:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:15 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:21:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.758+0000 7ffbbb90a640 1 -- 172.21.15.5:0/1471629615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb41082a0 msgr2=0x7ffbb410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.758+0000 7ffbbb90a640 1 --2- 172.21.15.5:0/1471629615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb41082a0 0x7ffbb410ec40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009930 tx=0x7ffbac02a420 comp rx=0 tx=0).stop 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.758+0000 7ffbbb90a640 1 -- 172.21.15.5:0/1471629615 shutdown_connections 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.758+0000 7ffbbb90a640 1 --2- 172.21.15.5:0/1471629615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb41082a0 0x7ffbb410ec40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.758+0000 7ffbbb90a640 1 --2- 172.21.15.5:0/1471629615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41036e0 0x7ffbb4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.758+0000 7ffbbb90a640 1 --2- 172.21.15.5:0/1471629615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb4102d30 0x7ffbb4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.758+0000 7ffbbb90a640 1 -- 172.21.15.5:0/1471629615 >> 172.21.15.5:0/1471629615 conn(0x7ffbb40fc7d0 msgr2=0x7ffbb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.758+0000 7ffbbb90a640 1 -- 172.21.15.5:0/1471629615 shutdown_connections 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.758+0000 7ffbbb90a640 1 -- 172.21.15.5:0/1471629615 wait complete. 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbbb90a640 1 Processor -- start 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbbb90a640 1 -- start start 2024-09-11T22:21:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbbb90a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb4102d30 0x7ffbb4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbbb90a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb41036e0 0x7ffbb4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbbb90a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41082a0 0x7ffbb419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbbb90a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb41120a0 con 0x7ffbb4102d30 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbbb90a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb4111f20 con 0x7ffbb41082a0 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbbb90a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb4112220 con 0x7ffbb41036e0 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbb96a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb4102d30 0x7ffbb4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbb96a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb4102d30 0x7ffbb4198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43362/0 (socket says 172.21.15.5:43362) 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbb96a6640 1 -- 172.21.15.5:0/272755636 learned_addr learned my addr 172.21.15.5:0/272755636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbb9ea7640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41082a0 0x7ffbb419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbb8ea5640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb41036e0 0x7ffbb4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbb96a6640 1 -- 172.21.15.5:0/272755636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb41036e0 msgr2=0x7ffbb4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbb96a6640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb41036e0 0x7ffbb4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbb96a6640 1 -- 172.21.15.5:0/272755636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41082a0 msgr2=0x7ffbb419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbb96a6640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41082a0 0x7ffbb419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.759+0000 7ffbb96a6640 1 -- 172.21.15.5:0/272755636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb419d520 con 0x7ffbb4102d30 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.760+0000 7ffbb96a6640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb4102d30 0x7ffbb4198570 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffba400eb90 tx=0x7ffba400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.760+0000 7ffbb27fc640 1 -- 172.21.15.5:0/272755636 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4004d20 con 0x7ffbb4102d30 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.760+0000 7ffbb27fc640 1 -- 172.21.15.5:0/272755636 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffba400ee90 con 0x7ffbb4102d30 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.760+0000 7ffbbb90a640 1 -- 172.21.15.5:0/272755636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb419d810 con 0x7ffbb4102d30 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.760+0000 7ffbb27fc640 1 -- 172.21.15.5:0/272755636 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4010830 con 0x7ffbb4102d30 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.760+0000 7ffbbb90a640 1 -- 172.21.15.5:0/272755636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb41a50f0 con 0x7ffbb4102d30 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.761+0000 7ffbbb90a640 1 -- 172.21.15.5:0/272755636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb4103110 con 0x7ffbb4102d30 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.762+0000 7ffbb27fc640 1 -- 172.21.15.5:0/272755636 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ffba4005000 con 0x7ffbb4102d30 2024-09-11T22:21:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.763+0000 7ffbb27fc640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ffb9c0a2c10 0x7ffb9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.763+0000 7ffbb8ea5640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ffb9c0a2c10 0x7ffb9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.763+0000 7ffbb27fc640 1 -- 172.21.15.5:0/272755636 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ffba40cf850 con 0x7ffbb4102d30 2024-09-11T22:21:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.763+0000 7ffbb8ea5640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ffb9c0a2c10 0x7ffb9c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40fede0 tx=0x7ffba800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.766+0000 7ffbb27fc640 1 -- 172.21.15.5:0/272755636 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffba40541b0 con 0x7ffbb4102d30 2024-09-11T22:21:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.943+0000 7ffbbb90a640 1 -- 172.21.15.5:0/272755636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ffbb41998d0 con 0x7ffbb4102d30 2024-09-11T22:21:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.944+0000 7ffbb27fc640 1 -- 172.21.15.5:0/272755636 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ffba409c800 con 0x7ffbb4102d30 2024-09-11T22:21:15.944 INFO:teuthology.orchestra.run.smithi005.stdout:287762808862 2024-09-11T22:21:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 -- 172.21.15.5:0/272755636 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ffb9c0a2c10 msgr2=0x7ffb9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ffb9c0a2c10 0x7ffb9c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40fede0 tx=0x7ffba800b410 comp rx=0 tx=0).stop 2024-09-11T22:21:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 -- 172.21.15.5:0/272755636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb4102d30 msgr2=0x7ffbb4198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb4102d30 0x7ffbb4198570 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffba400eb90 tx=0x7ffba400c6a0 comp rx=0 tx=0).stop 2024-09-11T22:21:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 -- 172.21.15.5:0/272755636 shutdown_connections 2024-09-11T22:21:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ffb9c0a2c10 0x7ffb9c0a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbb41082a0 0x7ffbb419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbb41036e0 0x7ffbb4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 --2- 172.21.15.5:0/272755636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb4102d30 0x7ffbb4198570 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 -- 172.21.15.5:0/272755636 >> 172.21.15.5:0/272755636 conn(0x7ffbb40fc7d0 msgr2=0x7ffbb4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 -- 172.21.15.5:0/272755636 shutdown_connections 2024-09-11T22:21:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:15.946+0000 7ffbbb90a640 1 -- 172.21.15.5:0/272755636 wait complete. 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/272755636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/41822388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-11T22:21:16.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-11T22:21:16.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-11T22:21:16.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-11T22:21:16.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-11T22:21:16.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-11T22:21:16.729 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:16 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-09-11T22:21:16.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/272755636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T22:21:16.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/41822388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-11T22:21:16.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-11T22:21:16.831 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:16 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-09-11T22:21:16.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/272755636' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-11T22:21:16.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:16.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/41822388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T22:21:16.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]: dispatch 2024-09-11T22:21:16.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-11T22:21:16.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-11T22:21:16.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-11T22:21:16.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]: dispatch 2024-09-11T22:21:16.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-09-11T22:21:16.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-09-11T22:21:16.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-11T22:21:16.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-11T22:21:16.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:16 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-09-11T22:21:17.155 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808860 got 287762808862 for osd.10 2024-09-11T22:21:17.155 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:17.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.537+0000 7fd5b731a640 1 -- 172.21.15.5:0/1122296677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b0101de0 msgr2=0x7fd5b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.537+0000 7fd5b731a640 1 --2- 172.21.15.5:0/1122296677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b0101de0 0x7fd5b01021c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009900 tx=0x7fd5ac02a410 comp rx=0 tx=0).stop 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.537+0000 7fd5b731a640 1 -- 172.21.15.5:0/1122296677 shutdown_connections 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.537+0000 7fd5b731a640 1 --2- 172.21.15.5:0/1122296677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b0106f10 0x7fd5b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.537+0000 7fd5b731a640 1 --2- 172.21.15.5:0/1122296677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b0102790 0x7fd5b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.537+0000 7fd5b731a640 1 --2- 172.21.15.5:0/1122296677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b0101de0 0x7fd5b01021c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.537+0000 7fd5b731a640 1 -- 172.21.15.5:0/1122296677 >> 172.21.15.5:0/1122296677 conn(0x7fd5b00fd8c0 msgr2=0x7fd5b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.537+0000 7fd5b731a640 1 -- 172.21.15.5:0/1122296677 shutdown_connections 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.537+0000 7fd5b731a640 1 -- 172.21.15.5:0/1122296677 wait complete. 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b731a640 1 Processor -- start 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b731a640 1 -- start start 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b731a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b0101de0 0x7fd5b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b731a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b0102790 0x7fd5b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b731a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b0106f10 0x7fd5b0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b731a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0111f70 con 0x7fd5b0101de0 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b731a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b0111df0 con 0x7fd5b0102790 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b731a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b01120f0 con 0x7fd5b0106f10 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b50b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b0101de0 0x7fd5b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b50b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b0101de0 0x7fd5b00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43416/0 (socket says 172.21.15.5:43416) 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b50b6640 1 -- 172.21.15.5:0/929381391 learned_addr learned my addr 172.21.15.5:0/929381391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b58b7640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b0106f10 0x7fd5b0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.538+0000 7fd5b48b5640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b0102790 0x7fd5b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5b50b6640 1 -- 172.21.15.5:0/929381391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b0106f10 msgr2=0x7fd5b0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:17.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5b50b6640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b0106f10 0x7fd5b0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5b50b6640 1 -- 172.21.15.5:0/929381391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b0102790 msgr2=0x7fd5b0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:17.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5b50b6640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b0102790 0x7fd5b0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5b50b6640 1 -- 172.21.15.5:0/929381391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b01a68c0 con 0x7fd5b0101de0 2024-09-11T22:21:17.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5b50b6640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b0101de0 0x7fd5b00744f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac006010 tx=0x7fd5ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:17.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5a67fc640 1 -- 172.21.15.5:0/929381391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac03f030 con 0x7fd5b0101de0 2024-09-11T22:21:17.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5b731a640 1 -- 172.21.15.5:0/929381391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b01a6b50 con 0x7fd5b0101de0 2024-09-11T22:21:17.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5a67fc640 1 -- 172.21.15.5:0/929381391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ac003ce0 con 0x7fd5b0101de0 2024-09-11T22:21:17.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5a67fc640 1 -- 172.21.15.5:0/929381391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac035070 con 0x7fd5b0101de0 2024-09-11T22:21:17.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.539+0000 7fd5b731a640 1 -- 172.21.15.5:0/929381391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b01a7030 con 0x7fd5b0101de0 2024-09-11T22:21:17.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.540+0000 7fd5b731a640 1 -- 172.21.15.5:0/929381391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b01021c0 con 0x7fd5b0101de0 2024-09-11T22:21:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.542+0000 7fd5a67fc640 1 -- 172.21.15.5:0/929381391 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd5ac042030 con 0x7fd5b0101de0 2024-09-11T22:21:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.542+0000 7fd5a67fc640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd59c0a2c10 0x7fd59c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.543+0000 7fd5b48b5640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd59c0a2c10 0x7fd59c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.543+0000 7fd5a67fc640 1 -- 172.21.15.5:0/929381391 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fd5ac0ef9a0 con 0x7fd5b0101de0 2024-09-11T22:21:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.543+0000 7fd5b48b5640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd59c0a2c10 0x7fd59c0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0075f80 tx=0x7fd5a000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:17.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.546+0000 7fd5a67fc640 1 -- 172.21.15.5:0/929381391 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5ac0bc1f0 con 0x7fd5b0101de0 2024-09-11T22:21:17.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.727+0000 7fd5b731a640 1 -- 172.21.15.5:0/929381391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd5b010a0f0 con 0x7fd5b0101de0 2024-09-11T22:21:17.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.727+0000 7fd5a67fc640 1 -- 172.21.15.5:0/929381391 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd5ac0c04f0 con 0x7fd5b0101de0 2024-09-11T22:21:17.728 INFO:teuthology.orchestra.run.smithi005.stdout:34359738450 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 -- 172.21.15.5:0/929381391 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd59c0a2c10 msgr2=0x7fd59c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd59c0a2c10 0x7fd59c0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0075f80 tx=0x7fd5a000b390 comp rx=0 tx=0).stop 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 -- 172.21.15.5:0/929381391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b0101de0 msgr2=0x7fd5b00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b0101de0 0x7fd5b00744f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac006010 tx=0x7fd5ac0042f0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 -- 172.21.15.5:0/929381391 shutdown_connections 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd59c0a2c10 0x7fd59c0a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b0106f10 0x7fd5b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd5b0102790 0x7fd5b0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 --2- 172.21.15.5:0/929381391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b0101de0 0x7fd5b00744f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 -- 172.21.15.5:0/929381391 >> 172.21.15.5:0/929381391 conn(0x7fd5b00fd8c0 msgr2=0x7fd5b00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 -- 172.21.15.5:0/929381391 shutdown_connections 2024-09-11T22:21:17.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:17.729+0000 7fd5b731a640 1 -- 172.21.15.5:0/929381391 wait complete. 2024-09-11T22:21:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.019+0000 7f5e5e230640 1 -- 172.21.15.5:0/2328743353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e58106a80 msgr2=0x7f5e58106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.019+0000 7f5e5e230640 1 --2- 172.21.15.5:0/2328743353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e58106a80 0x7f5e58106e60 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009930 tx=0x7f5e4802a420 comp rx=0 tx=0).stop 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.019+0000 7f5e5e230640 1 -- 172.21.15.5:0/2328743353 shutdown_connections 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.019+0000 7f5e5e230640 1 --2- 172.21.15.5:0/2328743353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e58101420 0x7f5e5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.019+0000 7f5e5e230640 1 --2- 172.21.15.5:0/2328743353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e58100a80 0x7f5e58100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.019+0000 7f5e5e230640 1 --2- 172.21.15.5:0/2328743353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e58106a80 0x7f5e58106e60 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.019+0000 7f5e5e230640 1 -- 172.21.15.5:0/2328743353 >> 172.21.15.5:0/2328743353 conn(0x7f5e580fc7a0 msgr2=0x7f5e580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.019+0000 7f5e5e230640 1 -- 172.21.15.5:0/2328743353 shutdown_connections 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.019+0000 7f5e5e230640 1 -- 172.21.15.5:0/2328743353 wait complete. 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e5e230640 1 Processor -- start 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e5e230640 1 -- start start 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e5e230640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e58100a80 0x7f5e58198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e5e230640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e58101420 0x7f5e581989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e5e230640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e58106a80 0x7f5e5819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e5e230640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58110e40 con 0x7f5e58100a80 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e5e230640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58110cc0 con 0x7f5e58101420 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e5e230640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58110fc0 con 0x7f5e58106a80 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e58100a80 0x7f5e58198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e58100a80 0x7f5e58198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43392/0 (socket says 172.21.15.5:43392) 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e577fe640 1 -- 172.21.15.5:0/41822388 learned_addr learned my addr 172.21.15.5:0/41822388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e4ffff640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e58101420 0x7f5e581989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.020+0000 7f5e57fff640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e58106a80 0x7f5e5819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e577fe640 1 -- 172.21.15.5:0/41822388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e58106a80 msgr2=0x7f5e5819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e577fe640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e58106a80 0x7f5e5819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e577fe640 1 -- 172.21.15.5:0/41822388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e58101420 msgr2=0x7f5e581989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:17.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e577fe640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e58101420 0x7f5e581989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e577fe640 1 -- 172.21.15.5:0/41822388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5819d410 con 0x7f5e58100a80 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e4ffff640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e58101420 0x7f5e581989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e577fe640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e58100a80 0x7f5e58198460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009230 tx=0x7f5e48004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e557fa640 1 -- 172.21.15.5:0/41822388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4803f030 con 0x7f5e58100a80 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e5e230640 1 -- 172.21.15.5:0/41822388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5819d6a0 con 0x7f5e58100a80 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e557fa640 1 -- 172.21.15.5:0/41822388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e48003ad0 con 0x7f5e58100a80 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e557fa640 1 -- 172.21.15.5:0/41822388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48035070 con 0x7f5e58100a80 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.021+0000 7f5e5e230640 1 -- 172.21.15.5:0/41822388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e581a4fe0 con 0x7f5e58100a80 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.022+0000 7f5e5e230640 1 -- 172.21.15.5:0/41822388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e58100ee0 con 0x7f5e58100a80 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.023+0000 7f5e557fa640 1 -- 172.21.15.5:0/41822388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5e48042030 con 0x7f5e58100a80 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.024+0000 7f5e557fa640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5e440a2c10 0x7f5e440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.024+0000 7f5e4ffff640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5e440a2c10 0x7f5e440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.024+0000 7f5e557fa640 1 -- 172.21.15.5:0/41822388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f5e480efaf0 con 0x7f5e58100a80 2024-09-11T22:21:17.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.025+0000 7f5e4ffff640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5e440a2c10 0x7f5e440a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5e581051c0 tx=0x7f5e40005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.027+0000 7f5e557fa640 1 -- 172.21.15.5:0/41822388 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e480bc350 con 0x7f5e58100a80 2024-09-11T22:21:17.806 INFO:teuthology.orchestra.run.smithi005.stdout:77309411400 2024-09-11T22:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.206+0000 7f5e5e230640 1 -- 172.21.15.5:0/41822388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5e5810a000 con 0x7f5e58100a80 2024-09-11T22:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.207+0000 7f5e557fa640 1 -- 172.21.15.5:0/41822388 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5e480c0650 con 0x7f5e58100a80 2024-09-11T22:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 -- 172.21.15.5:0/41822388 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5e440a2c10 msgr2=0x7f5e440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5e440a2c10 0x7f5e440a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5e581051c0 tx=0x7f5e40005f90 comp rx=0 tx=0).stop 2024-09-11T22:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 -- 172.21.15.5:0/41822388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e58100a80 msgr2=0x7f5e58198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:17.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e58100a80 0x7f5e58198460 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009230 tx=0x7f5e48004270 comp rx=0 tx=0).stop 2024-09-11T22:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 -- 172.21.15.5:0/41822388 shutdown_connections 2024-09-11T22:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f5e440a2c10 0x7f5e440a50d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e58106a80 0x7f5e5819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e58101420 0x7f5e581989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 --2- 172.21.15.5:0/41822388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e58100a80 0x7f5e58198460 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 -- 172.21.15.5:0/41822388 >> 172.21.15.5:0/41822388 conn(0x7f5e580fc7a0 msgr2=0x7f5e5806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 -- 172.21.15.5:0/41822388 shutdown_connections 2024-09-11T22:21:17.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:16.211+0000 7f5e5e230640 1 -- 172.21.15.5:0/41822388 wait complete. 2024-09-11T22:21:18.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-11T22:21:18.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T22:21:18.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:18 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/929381391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T22:21:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-11T22:21:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-11T22:21:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-11T22:21:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-11T22:21:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-11T22:21:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-11T22:21:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-11T22:21:18.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-11T22:21:18.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-11T22:21:18.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-11T22:21:18.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-09-11T22:21:18.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T22:21:18.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:18 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/929381391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T22:21:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-11T22:21:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [7, 11]}]': finished 2024-09-11T22:21:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-11T22:21:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-11T22:21:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-11T22:21:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-11T22:21:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-09-11T22:21:18.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-09-11T22:21:18.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-11T22:21:18.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-11T22:21:18.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-09-11T22:21:18.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: osdmap e81: 12 total, 12 up, 12 in 2024-09-11T22:21:18.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:18 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/929381391' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T22:21:18.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.693+0000 7f54e7178640 1 -- 172.21.15.5:0/558984331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0101de0 msgr2=0x7f54e01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:18.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.693+0000 7f54e7178640 1 --2- 172.21.15.5:0/558984331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0101de0 0x7f54e01021c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009900 tx=0x7f54dc02a410 comp rx=0 tx=0).stop 2024-09-11T22:21:18.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.693+0000 7f54e7178640 1 -- 172.21.15.5:0/558984331 shutdown_connections 2024-09-11T22:21:18.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.693+0000 7f54e7178640 1 --2- 172.21.15.5:0/558984331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e0106f10 0x7f54e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:18.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.693+0000 7f54e7178640 1 --2- 172.21.15.5:0/558984331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54e0102790 0x7f54e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.693+0000 7f54e7178640 1 --2- 172.21.15.5:0/558984331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0101de0 0x7f54e01021c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.693+0000 7f54e7178640 1 -- 172.21.15.5:0/558984331 >> 172.21.15.5:0/558984331 conn(0x7f54e00fd8c0 msgr2=0x7f54e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.693+0000 7f54e7178640 1 -- 172.21.15.5:0/558984331 shutdown_connections 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.693+0000 7f54e7178640 1 -- 172.21.15.5:0/558984331 wait complete. 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.693+0000 7f54e7178640 1 Processor -- start 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54e7178640 1 -- start start 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54e7178640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54e0101de0 0x7f54e00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54e7178640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0102790 0x7f54e0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54e7178640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e0106f10 0x7f54e0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54e7178640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0111fe0 con 0x7f54e0102790 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54e7178640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0111e60 con 0x7f54e0106f10 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54e7178640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0112160 con 0x7f54e0101de0 2024-09-11T22:21:18.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0102790 0x7f54e0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0102790 0x7f54e0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43436/0 (socket says 172.21.15.5:43436) 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54d7fff640 1 -- 172.21.15.5:0/3804535250 learned_addr learned my addr 172.21.15.5:0/3804535250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54e5715640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e0106f10 0x7f54e0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54e4f14640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54e0101de0 0x7f54e00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54d7fff640 1 -- 172.21.15.5:0/3804535250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54e0101de0 msgr2=0x7f54e00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54d7fff640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54e0101de0 0x7f54e00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54d7fff640 1 -- 172.21.15.5:0/3804535250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e0106f10 msgr2=0x7f54e0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54d7fff640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e0106f10 0x7f54e0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.694+0000 7f54d7fff640 1 -- 172.21.15.5:0/3804535250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e01a6990 con 0x7f54e0102790 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.695+0000 7f54d7fff640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0102790 0x7f54e0074a30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f54d000b4a0 tx=0x7f54d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.695+0000 7f54d5ffb640 1 -- 172.21.15.5:0/3804535250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0014070 con 0x7f54e0102790 2024-09-11T22:21:18.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.695+0000 7f54d5ffb640 1 -- 172.21.15.5:0/3804535250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d0010af0 con 0x7f54e0102790 2024-09-11T22:21:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.695+0000 7f54e7178640 1 -- 172.21.15.5:0/3804535250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e01a6c80 con 0x7f54e0102790 2024-09-11T22:21:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.695+0000 7f54d5ffb640 1 -- 172.21.15.5:0/3804535250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0017d10 con 0x7f54e0102790 2024-09-11T22:21:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.695+0000 7f54e7178640 1 -- 172.21.15.5:0/3804535250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e01a71c0 con 0x7f54e0102790 2024-09-11T22:21:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.696+0000 7f54e7178640 1 -- 172.21.15.5:0/3804535250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e01021c0 con 0x7f54e0102790 2024-09-11T22:21:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.697+0000 7f54d5ffb640 1 -- 172.21.15.5:0/3804535250 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f54d0017070 con 0x7f54e0102790 2024-09-11T22:21:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.698+0000 7f54d5ffb640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f54c80a2c90 0x7f54c80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.698+0000 7f54e4f14640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f54c80a2c90 0x7f54c80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.698+0000 7f54d5ffb640 1 -- 172.21.15.5:0/3804535250 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f54d00ce440 con 0x7f54e0102790 2024-09-11T22:21:18.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.698+0000 7f54e4f14640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f54c80a2c90 0x7f54c80a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f54dc000c00 tx=0x7f54dc005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:18.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.702+0000 7f54d5ffb640 1 -- 172.21.15.5:0/3804535250 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54d009b330 con 0x7f54e0102790 2024-09-11T22:21:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.878+0000 7f54e7178640 1 -- 172.21.15.5:0/3804535250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f54e010a0f0 con 0x7f54e0102790 2024-09-11T22:21:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.879+0000 7f54d5ffb640 1 -- 172.21.15.5:0/3804535250 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f54d009ac30 con 0x7f54e0102790 2024-09-11T22:21:18.880 INFO:teuthology.orchestra.run.smithi005.stdout:154618822712 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 -- 172.21.15.5:0/3804535250 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f54c80a2c90 msgr2=0x7f54c80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f54c80a2c90 0x7f54c80a5150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f54dc000c00 tx=0x7f54dc005a20 comp rx=0 tx=0).stop 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 -- 172.21.15.5:0/3804535250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0102790 msgr2=0x7f54e0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0102790 0x7f54e0074a30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f54d000b4a0 tx=0x7f54d000b960 comp rx=0 tx=0).stop 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 -- 172.21.15.5:0/3804535250 shutdown_connections 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f54c80a2c90 0x7f54c80a5150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e0106f10 0x7f54e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e0102790 0x7f54e0074a30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 --2- 172.21.15.5:0/3804535250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54e0101de0 0x7f54e00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 -- 172.21.15.5:0/3804535250 >> 172.21.15.5:0/3804535250 conn(0x7f54e00fd8c0 msgr2=0x7f54e00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:18.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 -- 172.21.15.5:0/3804535250 shutdown_connections 2024-09-11T22:21:18.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:18.881+0000 7f54e7178640 1 -- 172.21.15.5:0/3804535250 wait complete. 2024-09-11T22:21:19.198 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738450 for osd.0 2024-09-11T22:21:19.199 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:19.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:19 smithi005 ceph-mon[30943]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T22:21:19.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:19 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3804535250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T22:21:19.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.497+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2831315875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538102790 msgr2=0x7fb5381067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:19.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.497+0000 7fb53e0f7640 1 --2- 172.21.15.5:0/2831315875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538102790 0x7fb5381067e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb520009900 tx=0x7fb52002a410 comp rx=0 tx=0).stop 2024-09-11T22:21:19.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.497+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2831315875 shutdown_connections 2024-09-11T22:21:19.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.497+0000 7fb53e0f7640 1 --2- 172.21.15.5:0/2831315875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb538106f10 0x7fb53810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.497+0000 7fb53e0f7640 1 --2- 172.21.15.5:0/2831315875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538102790 0x7fb5381067e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.497+0000 7fb53e0f7640 1 --2- 172.21.15.5:0/2831315875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb538101de0 0x7fb5381021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.497+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2831315875 >> 172.21.15.5:0/2831315875 conn(0x7fb5380fd8c0 msgr2=0x7fb5380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.497+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2831315875 shutdown_connections 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.497+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2831315875 wait complete. 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.498+0000 7fb53e0f7640 1 Processor -- start 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.498+0000 7fb53e0f7640 1 -- start start 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.498+0000 7fb53e0f7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb538101de0 0x7fb5380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.498+0000 7fb53e0f7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb538102790 0x7fb538074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.498+0000 7fb53e0f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538106f10 0x7fb538074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb53e0f7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb538111f90 con 0x7fb538106f10 2024-09-11T22:21:19.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb53e0f7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb538111e10 con 0x7fb538102790 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb53e0f7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb538112110 con 0x7fb538101de0 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb537fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538106f10 0x7fb538074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb537fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538106f10 0x7fb538074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43452/0 (socket says 172.21.15.5:43452) 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb537fff640 1 -- 172.21.15.5:0/2024789605 learned_addr learned my addr 172.21.15.5:0/2024789605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb536ffd640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb538102790 0x7fb538074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb5377fe640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb538101de0 0x7fb5380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb537fff640 1 -- 172.21.15.5:0/2024789605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb538101de0 msgr2=0x7fb5380744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb537fff640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb538101de0 0x7fb5380744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb537fff640 1 -- 172.21.15.5:0/2024789605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb538102790 msgr2=0x7fb538074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb537fff640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb538102790 0x7fb538074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb537fff640 1 -- 172.21.15.5:0/2024789605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5381a6950 con 0x7fb538106f10 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb5377fe640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb538101de0 0x7fb5380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:19.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb537fff640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538106f10 0x7fb538074f70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb530009fd0 tx=0x7fb53000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb534ff9640 1 -- 172.21.15.5:0/2024789605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530011830 con 0x7fb538106f10 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2024789605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5381a6c40 con 0x7fb538106f10 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb534ff9640 1 -- 172.21.15.5:0/2024789605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb530016070 con 0x7fb538106f10 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb534ff9640 1 -- 172.21.15.5:0/2024789605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53000f2e0 con 0x7fb538106f10 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.499+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2024789605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5381a7100 con 0x7fb538106f10 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.501+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2024789605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5381021c0 con 0x7fb538106f10 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.502+0000 7fb534ff9640 1 -- 172.21.15.5:0/2024789605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb5300119d0 con 0x7fb538106f10 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.503+0000 7fb534ff9640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb5240a2c10 0x7fb5240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.503+0000 7fb5377fe640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb5240a2c10 0x7fb5240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.503+0000 7fb534ff9640 1 -- 172.21.15.5:0/2024789605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fb5300cee40 con 0x7fb538106f10 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.503+0000 7fb5377fe640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb5240a2c10 0x7fb5240a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb528005c40 tx=0x7fb52800a430 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:19.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.506+0000 7fb534ff9640 1 -- 172.21.15.5:0/2024789605 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb53009bd30 con 0x7fb538106f10 2024-09-11T22:21:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:19 smithi159 ceph-mon[35534]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T22:21:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:19 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3804535250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T22:21:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:19 smithi172 ceph-mon[35461]: osdmap e82: 12 total, 12 up, 12 in 2024-09-11T22:21:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:19 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3804535250' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T22:21:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.685+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2024789605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb53802d7c0 con 0x7fb538106f10 2024-09-11T22:21:19.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.686+0000 7fb534ff9640 1 -- 172.21.15.5:0/2024789605 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb53009b630 con 0x7fb538106f10 2024-09-11T22:21:19.687 INFO:teuthology.orchestra.run.smithi005.stdout:317827579930 2024-09-11T22:21:19.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.688+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2024789605 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb5240a2c10 msgr2=0x7fb5240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:19.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.688+0000 7fb53e0f7640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb5240a2c10 0x7fb5240a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb528005c40 tx=0x7fb52800a430 comp rx=0 tx=0).stop 2024-09-11T22:21:19.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.688+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2024789605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538106f10 msgr2=0x7fb538074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:19.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.688+0000 7fb53e0f7640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538106f10 0x7fb538074f70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb530009fd0 tx=0x7fb53000b8e0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.689+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2024789605 shutdown_connections 2024-09-11T22:21:19.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.689+0000 7fb53e0f7640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fb5240a2c10 0x7fb5240a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.689+0000 7fb53e0f7640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb538106f10 0x7fb538074f70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.689+0000 7fb53e0f7640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb538102790 0x7fb538074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.689+0000 7fb53e0f7640 1 --2- 172.21.15.5:0/2024789605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb538101de0 0x7fb5380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.689+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2024789605 >> 172.21.15.5:0/2024789605 conn(0x7fb5380fd8c0 msgr2=0x7fb5380fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:19.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.689+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2024789605 shutdown_connections 2024-09-11T22:21:19.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.689+0000 7fb53e0f7640 1 -- 172.21.15.5:0/2024789605 wait complete. 2024-09-11T22:21:19.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.682+0000 7f00bedac640 1 -- 172.21.15.5:0/3913246019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b8101360 msgr2=0x7f00b810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:19.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.682+0000 7f00bedac640 1 --2- 172.21.15.5:0/3913246019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b8101360 0x7f00b810df90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009900 tx=0x7f00a402a410 comp rx=0 tx=0).stop 2024-09-11T22:21:19.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.682+0000 7f00bedac640 1 -- 172.21.15.5:0/3913246019 shutdown_connections 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.682+0000 7f00bedac640 1 --2- 172.21.15.5:0/3913246019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00b810e4d0 0x7f00b81108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.682+0000 7f00bedac640 1 --2- 172.21.15.5:0/3913246019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b8101360 0x7f00b810df90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.682+0000 7f00bedac640 1 --2- 172.21.15.5:0/3913246019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b81009b0 0x7f00b8100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.682+0000 7f00bedac640 1 -- 172.21.15.5:0/3913246019 >> 172.21.15.5:0/3913246019 conn(0x7f00b80fc7d0 msgr2=0x7f00b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.682+0000 7f00bedac640 1 -- 172.21.15.5:0/3913246019 shutdown_connections 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.683+0000 7f00bedac640 1 -- 172.21.15.5:0/3913246019 wait complete. 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.683+0000 7f00bedac640 1 Processor -- start 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.683+0000 7f00bedac640 1 -- start start 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.683+0000 7f00bedac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00b81009b0 0x7f00b819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.683+0000 7f00bedac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b8101360 0x7f00b819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.683+0000 7f00bedac640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b810e4d0 0x7f00b819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:19.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00bedac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8114200 con 0x7f00b8101360 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00bedac640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8114080 con 0x7f00b810e4d0 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00bedac640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8114380 con 0x7f00b81009b0 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00bcb48640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00b81009b0 0x7f00b819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b8101360 0x7f00b819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00bcb48640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00b81009b0 0x7f00b819a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:48432/0 (socket says 172.21.15.5:48432) 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00bcb48640 1 -- 172.21.15.5:0/3705888616 learned_addr learned my addr 172.21.15.5:0/3705888616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00bd349640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b810e4d0 0x7f00b819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00b7fff640 1 -- 172.21.15.5:0/3705888616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00b81009b0 msgr2=0x7f00b819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00b7fff640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00b81009b0 0x7f00b819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00b7fff640 1 -- 172.21.15.5:0/3705888616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b810e4d0 msgr2=0x7f00b819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00b7fff640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b810e4d0 0x7f00b819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00b7fff640 1 -- 172.21.15.5:0/3705888616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b819f700 con 0x7f00b8101360 2024-09-11T22:21:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00bcb48640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00b81009b0 0x7f00b819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:21:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.684+0000 7f00b7fff640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b8101360 0x7f00b819ac90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f00a4005940 tx=0x7f00a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.685+0000 7f00b5ffb640 1 -- 172.21.15.5:0/3705888616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a403f030 con 0x7f00b8101360 2024-09-11T22:21:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.685+0000 7f00b5ffb640 1 -- 172.21.15.5:0/3705888616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a4003ce0 con 0x7f00b8101360 2024-09-11T22:21:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.685+0000 7f00bedac640 1 -- 172.21.15.5:0/3705888616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b819f990 con 0x7f00b8101360 2024-09-11T22:21:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.685+0000 7f00b5ffb640 1 -- 172.21.15.5:0/3705888616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4035070 con 0x7f00b8101360 2024-09-11T22:21:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.685+0000 7f00bedac640 1 -- 172.21.15.5:0/3705888616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b81a72d0 con 0x7f00b8101360 2024-09-11T22:21:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.686+0000 7f00bedac640 1 -- 172.21.15.5:0/3705888616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00b8100d90 con 0x7f00b8101360 2024-09-11T22:21:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.687+0000 7f00b5ffb640 1 -- 172.21.15.5:0/3705888616 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f00a4042030 con 0x7f00b8101360 2024-09-11T22:21:19.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.688+0000 7f00b5ffb640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f00a80a2c90 0x7f00a80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.688+0000 7f00bcb48640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f00a80a2c90 0x7f00a80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.688+0000 7f00b5ffb640 1 -- 172.21.15.5:0/3705888616 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f00a40ef7d0 con 0x7f00b8101360 2024-09-11T22:21:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.689+0000 7f00bcb48640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f00a80a2c90 0x7f00a80a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac00b480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:19.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.692+0000 7f00b5ffb640 1 -- 172.21.15.5:0/3705888616 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00a40bbf60 con 0x7f00b8101360 2024-09-11T22:21:19.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.871+0000 7f00bedac640 1 -- 172.21.15.5:0/3705888616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f00b8109660 con 0x7f00b8101360 2024-09-11T22:21:19.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.871+0000 7f00b5ffb640 1 -- 172.21.15.5:0/3705888616 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f00a40c0260 con 0x7f00b8101360 2024-09-11T22:21:19.872 INFO:teuthology.orchestra.run.smithi005.stdout:236223201321 2024-09-11T22:21:19.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 -- 172.21.15.5:0/3705888616 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f00a80a2c90 msgr2=0x7f00a80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f00a80a2c90 0x7f00a80a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac00b480 comp rx=0 tx=0).stop 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 -- 172.21.15.5:0/3705888616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b8101360 msgr2=0x7f00b819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b8101360 0x7f00b819ac90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f00a4005940 tx=0x7f00a40042f0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 -- 172.21.15.5:0/3705888616 shutdown_connections 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f00a80a2c90 0x7f00a80a5150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00b810e4d0 0x7f00b819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b8101360 0x7f00b819ac90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 --2- 172.21.15.5:0/3705888616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00b81009b0 0x7f00b819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 -- 172.21.15.5:0/3705888616 >> 172.21.15.5:0/3705888616 conn(0x7f00b80fc7d0 msgr2=0x7f00b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 -- 172.21.15.5:0/3705888616 shutdown_connections 2024-09-11T22:21:19.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:19.873+0000 7f00bedac640 1 -- 172.21.15.5:0/3705888616 wait complete. 2024-09-11T22:21:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:20 smithi005 ceph-mon[30943]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T22:21:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:20 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2024789605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T22:21:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:20 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3705888616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T22:21:20.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:20 smithi159 ceph-mon[35534]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T22:21:20.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:20 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2024789605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T22:21:20.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:20 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3705888616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T22:21:20.608 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201317 got 236223201321 for osd.8 2024-09-11T22:21:20.608 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:20.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:20 smithi172 ceph-mon[35461]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-11T22:21:20.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:20 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2024789605' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-11T22:21:20.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:20 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3705888616' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-11T22:21:21.024 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411400 for osd.2 2024-09-11T22:21:21.024 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.436+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/2259953931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c68100e20 msgr2=0x7f0c68101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:21.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.436+0000 7f0c6fd0a640 1 --2- 172.21.15.5:0/2259953931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c68100e20 0x7f0c68101200 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0c64009900 tx=0x7f0c6402a480 comp rx=0 tx=0).stop 2024-09-11T22:21:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.436+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/2259953931 shutdown_connections 2024-09-11T22:21:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.436+0000 7f0c6fd0a640 1 --2- 172.21.15.5:0/2259953931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c68102ef0 0x7f0c6810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.436+0000 7f0c6fd0a640 1 --2- 172.21.15.5:0/2259953931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c68102570 0x7f0c681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.436+0000 7f0c6fd0a640 1 --2- 172.21.15.5:0/2259953931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c68100e20 0x7f0c68101200 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.436+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/2259953931 >> 172.21.15.5:0/2259953931 conn(0x7f0c680fc7d0 msgr2=0x7f0c680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.436+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/2259953931 shutdown_connections 2024-09-11T22:21:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.436+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/2259953931 wait complete. 2024-09-11T22:21:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6fd0a640 1 Processor -- start 2024-09-11T22:21:21.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6fd0a640 1 -- start start 2024-09-11T22:21:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6fd0a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c68100e20 0x7f0c68103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6fd0a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c68102570 0x7f0c68103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6fd0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c68102ef0 0x7f0c681041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6fd0a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68110fd0 con 0x7f0c68102ef0 2024-09-11T22:21:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6fd0a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68110e50 con 0x7f0c68100e20 2024-09-11T22:21:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6fd0a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68111150 con 0x7f0c68102570 2024-09-11T22:21:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6e2a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c68102ef0 0x7f0c681041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6e2a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c68102ef0 0x7f0c681041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39374/0 (socket says 172.21.15.5:39374) 2024-09-11T22:21:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6e2a7640 1 -- 172.21.15.5:0/3962337249 learned_addr learned my addr 172.21.15.5:0/3962337249 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:21.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6daa6640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c68100e20 0x7f0c68103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.437+0000 7f0c6d2a5640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c68102570 0x7f0c68103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c6e2a7640 1 -- 172.21.15.5:0/3962337249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c68102570 msgr2=0x7f0c68103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c6e2a7640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c68102570 0x7f0c68103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c6e2a7640 1 -- 172.21.15.5:0/3962337249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c68100e20 msgr2=0x7f0c68103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c6e2a7640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c68100e20 0x7f0c68103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c6e2a7640 1 -- 172.21.15.5:0/3962337249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c681a6a30 con 0x7f0c68102ef0 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c6e2a7640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c68102ef0 0x7f0c681041d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000cad0 tx=0x7f0c6000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c5effd640 1 -- 172.21.15.5:0/3962337249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6000ae20 con 0x7f0c68102ef0 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/3962337249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c681a6d20 con 0x7f0c68102ef0 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c5effd640 1 -- 172.21.15.5:0/3962337249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c60005020 con 0x7f0c68102ef0 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c5effd640 1 -- 172.21.15.5:0/3962337249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c6000f6f0 con 0x7f0c68102ef0 2024-09-11T22:21:21.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.438+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/3962337249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c681a7210 con 0x7f0c68102ef0 2024-09-11T22:21:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.439+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/3962337249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c68101200 con 0x7f0c68102ef0 2024-09-11T22:21:21.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.441+0000 7f0c5effd640 1 -- 172.21.15.5:0/3962337249 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0c600051d0 con 0x7f0c68102ef0 2024-09-11T22:21:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.441+0000 7f0c5effd640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0c540a2c10 0x7f0c540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.442+0000 7f0c6daa6640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0c540a2c10 0x7f0c540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.442+0000 7f0c5effd640 1 -- 172.21.15.5:0/3962337249 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f0c600cf120 con 0x7f0c68102ef0 2024-09-11T22:21:21.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.442+0000 7f0c6daa6640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0c540a2c10 0x7f0c540a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c64000c00 tx=0x7f0c64005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:21.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.445+0000 7f0c5effd640 1 -- 172.21.15.5:0/3962337249 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c60014030 con 0x7f0c68102ef0 2024-09-11T22:21:21.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:21 smithi005 ceph-mon[30943]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T22:21:21.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:21 smithi159 ceph-mon[35534]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T22:21:21.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:21 smithi172 ceph-mon[35461]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T22:21:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.627+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/3962337249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0c681a7500 con 0x7f0c68102ef0 2024-09-11T22:21:21.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.628+0000 7f0c5effd640 1 -- 172.21.15.5:0/3962337249 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0c6009b8b0 con 0x7f0c68102ef0 2024-09-11T22:21:21.628 INFO:teuthology.orchestra.run.smithi005.stdout:184683593779 2024-09-11T22:21:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/3962337249 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0c540a2c10 msgr2=0x7f0c540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:21.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0c540a2c10 0x7f0c540a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0c64000c00 tx=0x7f0c64005b50 comp rx=0 tx=0).stop 2024-09-11T22:21:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/3962337249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c68102ef0 msgr2=0x7f0c681041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c68102ef0 0x7f0c681041d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000cad0 tx=0x7f0c6000cf90 comp rx=0 tx=0).stop 2024-09-11T22:21:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/3962337249 shutdown_connections 2024-09-11T22:21:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0c540a2c10 0x7f0c540a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c68102ef0 0x7f0c681041d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c68102570 0x7f0c68103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 --2- 172.21.15.5:0/3962337249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c68100e20 0x7f0c68103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/3962337249 >> 172.21.15.5:0/3962337249 conn(0x7f0c680fc7d0 msgr2=0x7f0c6806a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/3962337249 shutdown_connections 2024-09-11T22:21:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:21.630+0000 7f0c6fd0a640 1 -- 172.21.15.5:0/3962337249 wait complete. 2024-09-11T22:21:21.920 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579926 got 317827579930 for osd.11 2024-09-11T22:21:21.921 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:22.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:22 smithi005 ceph-mon[30943]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5 op/s; 146 B/s, 0 objects/s recovering 2024-09-11T22:21:22.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:22 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3962337249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T22:21:22.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:22 smithi159 ceph-mon[35534]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5 op/s; 146 B/s, 0 objects/s recovering 2024-09-11T22:21:22.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:22 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3962337249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T22:21:22.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:22 smithi172 ceph-mon[35461]: pgmap v314: 97 pgs: 3 peering, 94 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 5 op/s; 146 B/s, 0 objects/s recovering 2024-09-11T22:21:22.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:22 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3962337249' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T22:21:22.917 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593775 got 184683593779 for osd.6 2024-09-11T22:21:22.917 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:23.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:23 smithi005 ceph-mon[30943]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T22:21:23.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:23 smithi005 ceph-mon[30943]: Cluster is now healthy 2024-09-11T22:21:23.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:23 smithi159 ceph-mon[35534]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T22:21:23.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:23 smithi159 ceph-mon[35534]: Cluster is now healthy 2024-09-11T22:21:23.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:23 smithi172 ceph-mon[35461]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T22:21:23.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:23 smithi172 ceph-mon[35461]: Cluster is now healthy 2024-09-11T22:21:23.703 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822710 got 154618822712 for osd.5 2024-09-11T22:21:23.704 DEBUG:teuthology.parallel:result is None 2024-09-11T22:21:23.704 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T22:21:23.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:21:23.993 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:24.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:24 smithi005 ceph-mon[30943]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5 op/s; 148 B/s, 1 objects/s recovering 2024-09-11T22:21:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.520+0000 7f6afbc43640 1 -- 172.21.15.5:0/1400387471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4102d30 msgr2=0x7f6af4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:24.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.520+0000 7f6afbc43640 1 --2- 172.21.15.5:0/1400387471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4102d30 0x7f6af4103110 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009930 tx=0x7f6af002a420 comp rx=0 tx=0).stop 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.520+0000 7f6afbc43640 1 -- 172.21.15.5:0/1400387471 shutdown_connections 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.520+0000 7f6afbc43640 1 --2- 172.21.15.5:0/1400387471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af41082a0 0x7f6af410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.520+0000 7f6afbc43640 1 --2- 172.21.15.5:0/1400387471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af41036e0 0x7f6af4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.520+0000 7f6afbc43640 1 --2- 172.21.15.5:0/1400387471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4102d30 0x7f6af4103110 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.520+0000 7f6afbc43640 1 -- 172.21.15.5:0/1400387471 >> 172.21.15.5:0/1400387471 conn(0x7f6af40fc7d0 msgr2=0x7f6af40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.520+0000 7f6afbc43640 1 -- 172.21.15.5:0/1400387471 shutdown_connections 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.520+0000 7f6afbc43640 1 -- 172.21.15.5:0/1400387471 wait complete. 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afbc43640 1 Processor -- start 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afbc43640 1 -- start start 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afbc43640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af4102d30 0x7f6af4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afbc43640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af41036e0 0x7f6af4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:24.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afbc43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af41082a0 0x7f6af419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afbc43640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6af41120a0 con 0x7f6af41082a0 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afbc43640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4111f20 con 0x7f6af41036e0 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afbc43640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4112220 con 0x7f6af4102d30 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afa1e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af41082a0 0x7f6af419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afa1e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af41082a0 0x7f6af419ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39382/0 (socket says 172.21.15.5:39382) 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afa1e0640 1 -- 172.21.15.5:0/1528218316 learned_addr learned my addr 172.21.15.5:0/1528218316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6af91de640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af41036e0 0x7f6af4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6af99df640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af4102d30 0x7f6af4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afa1e0640 1 -- 172.21.15.5:0/1528218316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af4102d30 msgr2=0x7f6af4198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afa1e0640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af4102d30 0x7f6af4198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afa1e0640 1 -- 172.21.15.5:0/1528218316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af41036e0 msgr2=0x7f6af4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afa1e0640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af41036e0 0x7f6af4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:24.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6afa1e0640 1 -- 172.21.15.5:0/1528218316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af419d510 con 0x7f6af41082a0 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.521+0000 7f6af91de640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af41036e0 0x7f6af4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.522+0000 7f6af99df640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af4102d30 0x7f6af4198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.522+0000 7f6afa1e0640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af41082a0 0x7f6af419ce30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00eaa0 tx=0x7f6aec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.522+0000 7f6aeaffd640 1 -- 172.21.15.5:0/1528218316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec00cdb0 con 0x7f6af41082a0 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.522+0000 7f6aeaffd640 1 -- 172.21.15.5:0/1528218316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aec005020 con 0x7f6af41082a0 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.522+0000 7f6afbc43640 1 -- 172.21.15.5:0/1528218316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af419d800 con 0x7f6af41082a0 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.522+0000 7f6aeaffd640 1 -- 172.21.15.5:0/1528218316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec0116b0 con 0x7f6af41082a0 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.522+0000 7f6afbc43640 1 -- 172.21.15.5:0/1528218316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af41a50e0 con 0x7f6af41082a0 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.523+0000 7f6afbc43640 1 -- 172.21.15.5:0/1528218316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af406a850 con 0x7f6af41082a0 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.524+0000 7f6aeaffd640 1 -- 172.21.15.5:0/1528218316 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6aec005560 con 0x7f6af41082a0 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.525+0000 7f6aeaffd640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6ae00a2c10 0x7f6ae00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.525+0000 7f6af99df640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6ae00a2c10 0x7f6ae00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.525+0000 7f6aeaffd640 1 -- 172.21.15.5:0/1528218316 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f6aec0cf4f0 con 0x7f6af41082a0 2024-09-11T22:21:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.525+0000 7f6af99df640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6ae00a2c10 0x7f6ae00a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6af0000c00 tx=0x7f6af0032c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.529+0000 7f6aeaffd640 1 -- 172.21.15.5:0/1528218316 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6aec016030 con 0x7f6af41082a0 2024-09-11T22:21:24.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:24 smithi159 ceph-mon[35534]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5 op/s; 148 B/s, 1 objects/s recovering 2024-09-11T22:21:24.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:24 smithi172 ceph-mon[35461]: pgmap v315: 97 pgs: 97 active+clean; 642 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 5 op/s; 148 B/s, 1 objects/s recovering 2024-09-11T22:21:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.728+0000 7f6afbc43640 1 -- 172.21.15.5:0/1528218316 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6af410a320 con 0x7f6ae00a2c10 2024-09-11T22:21:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.735+0000 7f6aeaffd640 1 -- 172.21.15.5:0/1528218316 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259058 (secure 0 0 0) 0x7f6af0032e30 con 0x7f6ae00a2c10 2024-09-11T22:21:24.738 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-11T22:21:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.739+0000 7f6afbc43640 1 -- 172.21.15.5:0/1528218316 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6ae00a2c10 msgr2=0x7f6ae00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.739+0000 7f6afbc43640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6ae00a2c10 0x7f6ae00a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6af0000c00 tx=0x7f6af0032c70 comp rx=0 tx=0).stop 2024-09-11T22:21:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.739+0000 7f6afbc43640 1 -- 172.21.15.5:0/1528218316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af41082a0 msgr2=0x7f6af419ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.739+0000 7f6afbc43640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af41082a0 0x7f6af419ce30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00eaa0 tx=0x7f6aec00ef60 comp rx=0 tx=0).stop 2024-09-11T22:21:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.740+0000 7f6afbc43640 1 -- 172.21.15.5:0/1528218316 shutdown_connections 2024-09-11T22:21:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.740+0000 7f6afbc43640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f6ae00a2c10 0x7f6ae00a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.740+0000 7f6afbc43640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af41082a0 0x7f6af419ce30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.740+0000 7f6afbc43640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af41036e0 0x7f6af4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.740+0000 7f6afbc43640 1 --2- 172.21.15.5:0/1528218316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6af4102d30 0x7f6af4198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.740+0000 7f6afbc43640 1 -- 172.21.15.5:0/1528218316 >> 172.21.15.5:0/1528218316 conn(0x7f6af40fc7d0 msgr2=0x7f6af4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.740+0000 7f6afbc43640 1 -- 172.21.15.5:0/1528218316 shutdown_connections 2024-09-11T22:21:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:24.740+0000 7f6afbc43640 1 -- 172.21.15.5:0/1528218316 wait complete. 2024-09-11T22:21:25.466 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-11T22:21:24.457748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657181,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":633,"num_read_kb":465,"num_write":424,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181928,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":289,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381220,"kb_used_data":11088,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560604,"statfs":{"total":1151940427776,"available":1151550058496,"internally_reserved":0,"allocated":11354112,"data_stored":7169319,"data_compressed":178122,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":27,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001859"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183642+0000","last_change":"2024-09-11T22:20:18.189744+0000","last_active":"2024-09-11T22:21:18.183642+0000","last_peered":"2024-09-11T22:21:18.183642+0000","last_clean":"2024-09-11T22:21:18.183642+0000","last_became_active":"2024-09-11T22:20:18.189398+0000","last_became_peered":"2024-09-11T22:20:18.189398+0000","last_unstale":"2024-09-11T22:21:18.183642+0000","last_undegraded":"2024-09-11T22:21:18.183642+0000","last_fullsized":"2024-09-11T22:21:18.183642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:24.152589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186316+0000","last_change":"2024-09-11T22:20:18.186727+0000","last_active":"2024-09-11T22:21:18.186316+0000","last_peered":"2024-09-11T22:21:18.186316+0000","last_clean":"2024-09-11T22:21:18.186316+0000","last_became_active":"2024-09-11T22:20:18.186414+0000","last_became_peered":"2024-09-11T22:20:18.186414+0000","last_unstale":"2024-09-11T22:21:18.186316+0000","last_undegraded":"2024-09-11T22:21:18.186316+0000","last_fullsized":"2024-09-11T22:21:18.186316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:53.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186059+0000","last_change":"2024-09-11T22:20:18.179984+0000","last_active":"2024-09-11T22:21:18.186059+0000","last_peered":"2024-09-11T22:21:18.186059+0000","last_clean":"2024-09-11T22:21:18.186059+0000","last_became_active":"2024-09-11T22:20:18.179329+0000","last_became_peered":"2024-09-11T22:20:18.179329+0000","last_unstale":"2024-09-11T22:21:18.186059+0000","last_undegraded":"2024-09-11T22:21:18.186059+0000","last_fullsized":"2024-09-11T22:21:18.186059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:19.149970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184029+0000","last_change":"2024-09-11T22:20:18.191768+0000","last_active":"2024-09-11T22:21:18.184029+0000","last_peered":"2024-09-11T22:21:18.184029+0000","last_clean":"2024-09-11T22:21:18.184029+0000","last_became_active":"2024-09-11T22:20:18.191336+0000","last_became_peered":"2024-09-11T22:20:18.191336+0000","last_unstale":"2024-09-11T22:21:18.184029+0000","last_undegraded":"2024-09-11T22:21:18.184029+0000","last_fullsized":"2024-09-11T22:21:18.184029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:44.768932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186238+0000","last_change":"2024-09-11T22:20:18.181469+0000","last_active":"2024-09-11T22:21:18.186238+0000","last_peered":"2024-09-11T22:21:18.186238+0000","last_clean":"2024-09-11T22:21:18.186238+0000","last_became_active":"2024-09-11T22:20:18.181100+0000","last_became_peered":"2024-09-11T22:20:18.181100+0000","last_unstale":"2024-09-11T22:21:18.186238+0000","last_undegraded":"2024-09-11T22:21:18.186238+0000","last_fullsized":"2024-09-11T22:21:18.186238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:14.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186974+0000","last_change":"2024-09-11T22:20:18.190855+0000","last_active":"2024-09-11T22:21:18.186974+0000","last_peered":"2024-09-11T22:21:18.186974+0000","last_clean":"2024-09-11T22:21:18.186974+0000","last_became_active":"2024-09-11T22:20:18.190523+0000","last_became_peered":"2024-09-11T22:20:18.190523+0000","last_unstale":"2024-09-11T22:21:18.186974+0000","last_undegraded":"2024-09-11T22:21:18.186974+0000","last_fullsized":"2024-09-11T22:21:18.186974+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:18.574957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184449+0000","last_change":"2024-09-11T22:20:18.199213+0000","last_active":"2024-09-11T22:21:18.184449+0000","last_peered":"2024-09-11T22:21:18.184449+0000","last_clean":"2024-09-11T22:21:18.184449+0000","last_became_active":"2024-09-11T22:20:18.198913+0000","last_became_peered":"2024-09-11T22:20:18.198913+0000","last_unstale":"2024-09-11T22:21:18.184449+0000","last_undegraded":"2024-09-11T22:21:18.184449+0000","last_fullsized":"2024-09-11T22:21:18.184449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:22.682802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184120+0000","last_change":"2024-09-11T22:20:18.192458+0000","last_active":"2024-09-11T22:21:18.184120+0000","last_peered":"2024-09-11T22:21:18.184120+0000","last_clean":"2024-09-11T22:21:18.184120+0000","last_became_active":"2024-09-11T22:20:18.191967+0000","last_became_peered":"2024-09-11T22:20:18.191967+0000","last_unstale":"2024-09-11T22:21:18.184120+0000","last_undegraded":"2024-09-11T22:21:18.184120+0000","last_fullsized":"2024-09-11T22:21:18.184120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:52.686157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.191732+0000","last_change":"2024-09-11T22:21:18.191732+0000","last_active":"2024-09-11T22:21:18.191732+0000","last_peered":"2024-09-11T22:21:18.191732+0000","last_clean":"2024-09-11T22:21:18.191732+0000","last_became_active":"2024-09-11T22:21:18.191375+0000","last_became_peered":"2024-09-11T22:21:18.191375+0000","last_unstale":"2024-09-11T22:21:18.191732+0000","last_undegraded":"2024-09-11T22:21:18.191732+0000","last_fullsized":"2024-09-11T22:21:18.191732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:13.645224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185803+0000","last_change":"2024-09-11T22:20:18.193771+0000","last_active":"2024-09-11T22:21:18.185803+0000","last_peered":"2024-09-11T22:21:18.185803+0000","last_clean":"2024-09-11T22:21:18.185803+0000","last_became_active":"2024-09-11T22:20:18.193041+0000","last_became_peered":"2024-09-11T22:20:18.193041+0000","last_unstale":"2024-09-11T22:21:18.185803+0000","last_undegraded":"2024-09-11T22:21:18.185803+0000","last_fullsized":"2024-09-11T22:21:18.185803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:06.656169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184811+0000","last_change":"2024-09-11T22:20:18.197933+0000","last_active":"2024-09-11T22:21:18.184811+0000","last_peered":"2024-09-11T22:21:18.184811+0000","last_clean":"2024-09-11T22:21:18.184811+0000","last_became_active":"2024-09-11T22:20:18.197210+0000","last_became_peered":"2024-09-11T22:20:18.197210+0000","last_unstale":"2024-09-11T22:21:18.184811+0000","last_undegraded":"2024-09-11T22:21:18.184811+0000","last_fullsized":"2024-09-11T22:21:18.184811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:42.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184129+0000","last_change":"2024-09-11T22:20:18.190739+0000","last_active":"2024-09-11T22:21:18.184129+0000","last_peered":"2024-09-11T22:21:18.184129+0000","last_clean":"2024-09-11T22:21:18.184129+0000","last_became_active":"2024-09-11T22:20:18.190122+0000","last_became_peered":"2024-09-11T22:20:18.190122+0000","last_unstale":"2024-09-11T22:21:18.184129+0000","last_undegraded":"2024-09-11T22:21:18.184129+0000","last_fullsized":"2024-09-11T22:21:18.184129+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.585035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.182268+0000","last_change":"2024-09-11T22:20:18.190303+0000","last_active":"2024-09-11T22:21:18.182268+0000","last_peered":"2024-09-11T22:21:18.182268+0000","last_clean":"2024-09-11T22:21:18.182268+0000","last_became_active":"2024-09-11T22:20:18.189962+0000","last_became_peered":"2024-09-11T22:20:18.189962+0000","last_unstale":"2024-09-11T22:21:18.182268+0000","last_undegraded":"2024-09-11T22:21:18.182268+0000","last_fullsized":"2024-09-11T22:21:18.182268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:44.872813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185672+0000","last_change":"2024-09-11T22:20:18.185893+0000","last_active":"2024-09-11T22:21:18.185672+0000","last_peered":"2024-09-11T22:21:18.185672+0000","last_clean":"2024-09-11T22:21:18.185672+0000","last_became_active":"2024-09-11T22:20:18.185592+0000","last_became_peered":"2024-09-11T22:20:18.185592+0000","last_unstale":"2024-09-11T22:21:18.185672+0000","last_undegraded":"2024-09-11T22:21:18.185672+0000","last_fullsized":"2024-09-11T22:21:18.185672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:14.725301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185405+0000","last_change":"2024-09-11T22:20:18.193943+0000","last_active":"2024-09-11T22:21:18.185405+0000","last_peered":"2024-09-11T22:21:18.185405+0000","last_clean":"2024-09-11T22:21:18.185405+0000","last_became_active":"2024-09-11T22:20:18.193558+0000","last_became_peered":"2024-09-11T22:20:18.193558+0000","last_unstale":"2024-09-11T22:21:18.185405+0000","last_undegraded":"2024-09-11T22:21:18.185405+0000","last_fullsized":"2024-09-11T22:21:18.185405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:08.736161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187501+0000","last_change":"2024-09-11T22:20:18.191815+0000","last_active":"2024-09-11T22:21:18.187501+0000","last_peered":"2024-09-11T22:21:18.187501+0000","last_clean":"2024-09-11T22:21:18.187501+0000","last_became_active":"2024-09-11T22:20:18.191483+0000","last_became_peered":"2024-09-11T22:20:18.191483+0000","last_unstale":"2024-09-11T22:21:18.187501+0000","last_undegraded":"2024-09-11T22:21:18.187501+0000","last_fullsized":"2024-09-11T22:21:18.187501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:26.432117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183752+0000","last_change":"2024-09-11T22:20:18.190758+0000","last_active":"2024-09-11T22:21:18.183752+0000","last_peered":"2024-09-11T22:21:18.183752+0000","last_clean":"2024-09-11T22:21:18.183752+0000","last_became_active":"2024-09-11T22:20:18.190431+0000","last_became_peered":"2024-09-11T22:20:18.190431+0000","last_unstale":"2024-09-11T22:21:18.183752+0000","last_undegraded":"2024-09-11T22:21:18.183752+0000","last_fullsized":"2024-09-11T22:21:18.183752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:10.927051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186191+0000","last_change":"2024-09-11T22:20:18.185191+0000","last_active":"2024-09-11T22:21:18.186191+0000","last_peered":"2024-09-11T22:21:18.186191+0000","last_clean":"2024-09-11T22:21:18.186191+0000","last_became_active":"2024-09-11T22:20:18.184906+0000","last_became_peered":"2024-09-11T22:20:18.184906+0000","last_unstale":"2024-09-11T22:21:18.186191+0000","last_undegraded":"2024-09-11T22:21:18.186191+0000","last_fullsized":"2024-09-11T22:21:18.186191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:32.294780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185228+0000","last_change":"2024-09-11T22:20:18.187137+0000","last_active":"2024-09-11T22:21:18.185228+0000","last_peered":"2024-09-11T22:21:18.185228+0000","last_clean":"2024-09-11T22:21:18.185228+0000","last_became_active":"2024-09-11T22:20:18.186770+0000","last_became_peered":"2024-09-11T22:20:18.186770+0000","last_unstale":"2024-09-11T22:21:18.185228+0000","last_undegraded":"2024-09-11T22:21:18.185228+0000","last_fullsized":"2024-09-11T22:21:18.185228+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:59.917695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183169+0000","last_change":"2024-09-11T22:20:18.197137+0000","last_active":"2024-09-11T22:21:18.183169+0000","last_peered":"2024-09-11T22:21:18.183169+0000","last_clean":"2024-09-11T22:21:18.183169+0000","last_became_active":"2024-09-11T22:20:18.192810+0000","last_became_peered":"2024-09-11T22:20:18.192810+0000","last_unstale":"2024-09-11T22:21:18.183169+0000","last_undegraded":"2024-09-11T22:21:18.183169+0000","last_fullsized":"2024-09-11T22:21:18.183169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:57.680726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.182325+0000","last_change":"2024-09-11T22:20:18.187568+0000","last_active":"2024-09-11T22:21:18.182325+0000","last_peered":"2024-09-11T22:21:18.182325+0000","last_clean":"2024-09-11T22:21:18.182325+0000","last_became_active":"2024-09-11T22:20:18.187254+0000","last_became_peered":"2024-09-11T22:20:18.187254+0000","last_unstale":"2024-09-11T22:21:18.182325+0000","last_undegraded":"2024-09-11T22:21:18.182325+0000","last_fullsized":"2024-09-11T22:21:18.182325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:41.724926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187943+0000","last_change":"2024-09-11T22:20:18.190376+0000","last_active":"2024-09-11T22:21:18.187943+0000","last_peered":"2024-09-11T22:21:18.187943+0000","last_clean":"2024-09-11T22:21:18.187943+0000","last_became_active":"2024-09-11T22:20:18.190079+0000","last_became_peered":"2024-09-11T22:20:18.190079+0000","last_unstale":"2024-09-11T22:21:18.187943+0000","last_undegraded":"2024-09-11T22:21:18.187943+0000","last_fullsized":"2024-09-11T22:21:18.187943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:42.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184566+0000","last_change":"2024-09-11T22:20:18.188503+0000","last_active":"2024-09-11T22:21:18.184566+0000","last_peered":"2024-09-11T22:21:18.184566+0000","last_clean":"2024-09-11T22:21:18.184566+0000","last_became_active":"2024-09-11T22:20:18.187957+0000","last_became_peered":"2024-09-11T22:20:18.187957+0000","last_unstale":"2024-09-11T22:21:18.184566+0000","last_undegraded":"2024-09-11T22:21:18.184566+0000","last_fullsized":"2024-09-11T22:21:18.184566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:44:25.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186673+0000","last_change":"2024-09-11T22:20:18.196936+0000","last_active":"2024-09-11T22:21:18.186673+0000","last_peered":"2024-09-11T22:21:18.186673+0000","last_clean":"2024-09-11T22:21:18.186673+0000","last_became_active":"2024-09-11T22:20:18.196493+0000","last_became_peered":"2024-09-11T22:20:18.196493+0000","last_unstale":"2024-09-11T22:21:18.186673+0000","last_undegraded":"2024-09-11T22:21:18.186673+0000","last_fullsized":"2024-09-11T22:21:18.186673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:06.189806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185464+0000","last_change":"2024-09-11T22:20:18.195260+0000","last_active":"2024-09-11T22:21:18.185464+0000","last_peered":"2024-09-11T22:21:18.185464+0000","last_clean":"2024-09-11T22:21:18.185464+0000","last_became_active":"2024-09-11T22:20:18.194956+0000","last_became_peered":"2024-09-11T22:20:18.194956+0000","last_unstale":"2024-09-11T22:21:18.185464+0000","last_undegraded":"2024-09-11T22:21:18.185464+0000","last_fullsized":"2024-09-11T22:21:18.185464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:34.483291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186741+0000","last_change":"2024-09-11T22:20:18.190591+0000","last_active":"2024-09-11T22:21:18.186741+0000","last_peered":"2024-09-11T22:21:18.186741+0000","last_clean":"2024-09-11T22:21:18.186741+0000","last_became_active":"2024-09-11T22:20:18.190007+0000","last_became_peered":"2024-09-11T22:20:18.190007+0000","last_unstale":"2024-09-11T22:21:18.186741+0000","last_undegraded":"2024-09-11T22:21:18.186741+0000","last_fullsized":"2024-09-11T22:21:18.186741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:46.281705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183329+0000","last_change":"2024-09-11T22:20:18.193377+0000","last_active":"2024-09-11T22:21:18.183329+0000","last_peered":"2024-09-11T22:21:18.183329+0000","last_clean":"2024-09-11T22:21:18.183329+0000","last_became_active":"2024-09-11T22:20:18.193003+0000","last_became_peered":"2024-09-11T22:20:18.193003+0000","last_unstale":"2024-09-11T22:21:18.183329+0000","last_undegraded":"2024-09-11T22:21:18.183329+0000","last_fullsized":"2024-09-11T22:21:18.183329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:59.052966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.182200+0000","last_change":"2024-09-11T22:20:18.189405+0000","last_active":"2024-09-11T22:21:18.182200+0000","last_peered":"2024-09-11T22:21:18.182200+0000","last_clean":"2024-09-11T22:21:18.182200+0000","last_became_active":"2024-09-11T22:20:18.189098+0000","last_became_peered":"2024-09-11T22:20:18.189098+0000","last_unstale":"2024-09-11T22:21:18.182200+0000","last_undegraded":"2024-09-11T22:21:18.182200+0000","last_fullsized":"2024-09-11T22:21:18.182200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:47.088192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185806+0000","last_change":"2024-09-11T22:20:18.191005+0000","last_active":"2024-09-11T22:21:18.185806+0000","last_peered":"2024-09-11T22:21:18.185806+0000","last_clean":"2024-09-11T22:21:18.185806+0000","last_became_active":"2024-09-11T22:20:18.190681+0000","last_became_peered":"2024-09-11T22:20:18.190681+0000","last_unstale":"2024-09-11T22:21:18.185806+0000","last_undegraded":"2024-09-11T22:21:18.185806+0000","last_fullsized":"2024-09-11T22:21:18.185806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:12.903929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187818+0000","last_change":"2024-09-11T22:20:18.187920+0000","last_active":"2024-09-11T22:21:18.187818+0000","last_peered":"2024-09-11T22:21:18.187818+0000","last_clean":"2024-09-11T22:21:18.187818+0000","last_became_active":"2024-09-11T22:20:18.187526+0000","last_became_peered":"2024-09-11T22:20:18.187526+0000","last_unstale":"2024-09-11T22:21:18.187818+0000","last_undegraded":"2024-09-11T22:21:18.187818+0000","last_fullsized":"2024-09-11T22:21:18.187818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.347857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187211+0000","last_change":"2024-09-11T22:20:18.193869+0000","last_active":"2024-09-11T22:21:18.187211+0000","last_peered":"2024-09-11T22:21:18.187211+0000","last_clean":"2024-09-11T22:21:18.187211+0000","last_became_active":"2024-09-11T22:20:18.193468+0000","last_became_peered":"2024-09-11T22:20:18.193468+0000","last_unstale":"2024-09-11T22:21:18.187211+0000","last_undegraded":"2024-09-11T22:21:18.187211+0000","last_fullsized":"2024-09-11T22:21:18.187211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:59.619611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183106+0000","last_change":"2024-09-11T22:20:18.193947+0000","last_active":"2024-09-11T22:21:18.183106+0000","last_peered":"2024-09-11T22:21:18.183106+0000","last_clean":"2024-09-11T22:21:18.183106+0000","last_became_active":"2024-09-11T22:20:18.192257+0000","last_became_peered":"2024-09-11T22:20:18.192257+0000","last_unstale":"2024-09-11T22:21:18.183106+0000","last_undegraded":"2024-09-11T22:21:18.183106+0000","last_fullsized":"2024-09-11T22:21:18.183106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.478730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185685+0000","last_change":"2024-09-11T22:20:18.184549+0000","last_active":"2024-09-11T22:21:18.185685+0000","last_peered":"2024-09-11T22:21:18.185685+0000","last_clean":"2024-09-11T22:21:18.185685+0000","last_became_active":"2024-09-11T22:20:18.184075+0000","last_became_peered":"2024-09-11T22:20:18.184075+0000","last_unstale":"2024-09-11T22:21:18.185685+0000","last_undegraded":"2024-09-11T22:21:18.185685+0000","last_fullsized":"2024-09-11T22:21:18.185685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:50.095643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.338878+0000","last_change":"2024-09-11T22:20:16.128524+0000","last_active":"2024-09-11T22:21:18.338878+0000","last_peered":"2024-09-11T22:21:18.338878+0000","last_clean":"2024-09-11T22:21:18.338878+0000","last_became_active":"2024-09-11T22:20:16.128185+0000","last_became_peered":"2024-09-11T22:20:16.128185+0000","last_unstale":"2024-09-11T22:21:18.338878+0000","last_undegraded":"2024-09-11T22:21:18.338878+0000","last_fullsized":"2024-09-11T22:21:18.338878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.022931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186233+0000","last_change":"2024-09-11T22:20:18.191578+0000","last_active":"2024-09-11T22:21:18.186233+0000","last_peered":"2024-09-11T22:21:18.186233+0000","last_clean":"2024-09-11T22:21:18.186233+0000","last_became_active":"2024-09-11T22:20:18.189568+0000","last_became_peered":"2024-09-11T22:20:18.189568+0000","last_unstale":"2024-09-11T22:21:18.186233+0000","last_undegraded":"2024-09-11T22:21:18.186233+0000","last_fullsized":"2024-09-11T22:21:18.186233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:41.737242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.199693+0000","last_change":"2024-09-11T22:21:18.199693+0000","last_active":"2024-09-11T22:21:18.199693+0000","last_peered":"2024-09-11T22:21:18.199693+0000","last_clean":"2024-09-11T22:21:18.199693+0000","last_became_active":"2024-09-11T22:21:18.190954+0000","last_became_peered":"2024-09-11T22:21:18.190954+0000","last_unstale":"2024-09-11T22:21:18.199693+0000","last_undegraded":"2024-09-11T22:21:18.199693+0000","last_fullsized":"2024-09-11T22:21:18.199693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:11.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.181962+0000","last_change":"2024-09-11T22:20:16.115602+0000","last_active":"2024-09-11T22:21:18.181962+0000","last_peered":"2024-09-11T22:21:18.181962+0000","last_clean":"2024-09-11T22:21:18.181962+0000","last_became_active":"2024-09-11T22:20:16.115280+0000","last_became_peered":"2024-09-11T22:20:16.115280+0000","last_unstale":"2024-09-11T22:21:18.181962+0000","last_undegraded":"2024-09-11T22:21:18.181962+0000","last_fullsized":"2024-09-11T22:21:18.181962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:41.942137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184216+0000","last_change":"2024-09-11T22:20:18.184125+0000","last_active":"2024-09-11T22:21:18.184216+0000","last_peered":"2024-09-11T22:21:18.184216+0000","last_clean":"2024-09-11T22:21:18.184216+0000","last_became_active":"2024-09-11T22:20:18.183785+0000","last_became_peered":"2024-09-11T22:20:18.183785+0000","last_unstale":"2024-09-11T22:21:18.184216+0000","last_undegraded":"2024-09-11T22:21:18.184216+0000","last_fullsized":"2024-09-11T22:21:18.184216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:33.230175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186888+0000","last_change":"2024-09-11T22:20:16.125852+0000","last_active":"2024-09-11T22:21:18.186888+0000","last_peered":"2024-09-11T22:21:18.186888+0000","last_clean":"2024-09-11T22:21:18.186888+0000","last_became_active":"2024-09-11T22:20:16.125250+0000","last_became_peered":"2024-09-11T22:20:16.125250+0000","last_unstale":"2024-09-11T22:21:18.186888+0000","last_undegraded":"2024-09-11T22:21:18.186888+0000","last_fullsized":"2024-09-11T22:21:18.186888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:34.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185013+0000","last_change":"2024-09-11T22:20:18.185147+0000","last_active":"2024-09-11T22:21:18.185013+0000","last_peered":"2024-09-11T22:21:18.185013+0000","last_clean":"2024-09-11T22:21:18.185013+0000","last_became_active":"2024-09-11T22:20:18.184840+0000","last_became_peered":"2024-09-11T22:20:18.184840+0000","last_unstale":"2024-09-11T22:21:18.185013+0000","last_undegraded":"2024-09-11T22:21:18.185013+0000","last_fullsized":"2024-09-11T22:21:18.185013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:59.703349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:19.339554+0000","last_change":"2024-09-11T22:20:16.122173+0000","last_active":"2024-09-11T22:21:19.339554+0000","last_peered":"2024-09-11T22:21:19.339554+0000","last_clean":"2024-09-11T22:21:19.339554+0000","last_became_active":"2024-09-11T22:20:16.121847+0000","last_became_peered":"2024-09-11T22:20:16.121847+0000","last_unstale":"2024-09-11T22:21:19.339554+0000","last_undegraded":"2024-09-11T22:21:19.339554+0000","last_fullsized":"2024-09-11T22:21:19.339554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:41.284557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.189560+0000","last_change":"2024-09-11T22:20:18.188030+0000","last_active":"2024-09-11T22:21:18.189560+0000","last_peered":"2024-09-11T22:21:18.189560+0000","last_clean":"2024-09-11T22:21:18.189560+0000","last_became_active":"2024-09-11T22:20:18.187503+0000","last_became_peered":"2024-09-11T22:20:18.187503+0000","last_unstale":"2024-09-11T22:21:18.189560+0000","last_undegraded":"2024-09-11T22:21:18.189560+0000","last_fullsized":"2024-09-11T22:21:18.189560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:45.494149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184153+0000","last_change":"2024-09-11T22:20:16.128828+0000","last_active":"2024-09-11T22:21:18.184153+0000","last_peered":"2024-09-11T22:21:18.184153+0000","last_clean":"2024-09-11T22:21:18.184153+0000","last_became_active":"2024-09-11T22:20:16.128454+0000","last_became_peered":"2024-09-11T22:20:16.128454+0000","last_unstale":"2024-09-11T22:21:18.184153+0000","last_undegraded":"2024-09-11T22:21:18.184153+0000","last_fullsized":"2024-09-11T22:21:18.184153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:33.927564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186436+0000","last_change":"2024-09-11T22:20:18.193795+0000","last_active":"2024-09-11T22:21:18.186436+0000","last_peered":"2024-09-11T22:21:18.186436+0000","last_clean":"2024-09-11T22:21:18.186436+0000","last_became_active":"2024-09-11T22:20:18.193489+0000","last_became_peered":"2024-09-11T22:20:18.193489+0000","last_unstale":"2024-09-11T22:21:18.186436+0000","last_undegraded":"2024-09-11T22:21:18.186436+0000","last_fullsized":"2024-09-11T22:21:18.186436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:14.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186332+0000","last_change":"2024-09-11T22:20:16.187938+0000","last_active":"2024-09-11T22:21:18.186332+0000","last_peered":"2024-09-11T22:21:18.186332+0000","last_clean":"2024-09-11T22:21:18.186332+0000","last_became_active":"2024-09-11T22:20:16.187545+0000","last_became_peered":"2024-09-11T22:20:16.187545+0000","last_unstale":"2024-09-11T22:21:18.186332+0000","last_undegraded":"2024-09-11T22:21:18.186332+0000","last_fullsized":"2024-09-11T22:21:18.186332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:41.674356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.191937+0000","last_change":"2024-09-11T22:21:18.191937+0000","last_active":"2024-09-11T22:21:18.191937+0000","last_peered":"2024-09-11T22:21:18.191937+0000","last_clean":"2024-09-11T22:21:18.191937+0000","last_became_active":"2024-09-11T22:21:18.191218+0000","last_became_peered":"2024-09-11T22:21:18.191218+0000","last_unstale":"2024-09-11T22:21:18.191937+0000","last_undegraded":"2024-09-11T22:21:18.191937+0000","last_fullsized":"2024-09-11T22:21:18.191937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:41.654630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":161,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:19.339351+0000","last_change":"2024-09-11T22:20:16.128082+0000","last_active":"2024-09-11T22:21:19.339351+0000","last_peered":"2024-09-11T22:21:19.339351+0000","last_clean":"2024-09-11T22:21:19.339351+0000","last_became_active":"2024-09-11T22:20:16.127731+0000","last_became_peered":"2024-09-11T22:20:16.127731+0000","last_unstale":"2024-09-11T22:21:19.339351+0000","last_undegraded":"2024-09-11T22:21:19.339351+0000","last_fullsized":"2024-09-11T22:21:19.339351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:33.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":112,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186621+0000","last_change":"2024-09-11T22:20:18.189219+0000","last_active":"2024-09-11T22:21:18.186621+0000","last_peered":"2024-09-11T22:21:18.186621+0000","last_clean":"2024-09-11T22:21:18.186621+0000","last_became_active":"2024-09-11T22:20:18.188687+0000","last_became_peered":"2024-09-11T22:20:18.188687+0000","last_unstale":"2024-09-11T22:21:18.186621+0000","last_undegraded":"2024-09-11T22:21:18.186621+0000","last_fullsized":"2024-09-11T22:21:18.186621+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:24.171126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185606+0000","last_change":"2024-09-11T22:20:16.129724+0000","last_active":"2024-09-11T22:21:18.185606+0000","last_peered":"2024-09-11T22:21:18.185606+0000","last_clean":"2024-09-11T22:21:18.185606+0000","last_became_active":"2024-09-11T22:20:16.129351+0000","last_became_peered":"2024-09-11T22:20:16.129351+0000","last_unstale":"2024-09-11T22:21:18.185606+0000","last_undegraded":"2024-09-11T22:21:18.185606+0000","last_fullsized":"2024-09-11T22:21:18.185606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:09.642520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.182030+0000","last_change":"2024-09-11T22:20:18.187031+0000","last_active":"2024-09-11T22:21:18.182030+0000","last_peered":"2024-09-11T22:21:18.182030+0000","last_clean":"2024-09-11T22:21:18.182030+0000","last_became_active":"2024-09-11T22:20:18.186698+0000","last_became_peered":"2024-09-11T22:20:18.186698+0000","last_unstale":"2024-09-11T22:21:18.182030+0000","last_undegraded":"2024-09-11T22:21:18.182030+0000","last_fullsized":"2024-09-11T22:21:18.182030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:04.810981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186471+0000","last_change":"2024-09-11T22:20:16.124925+0000","last_active":"2024-09-11T22:21:18.186471+0000","last_peered":"2024-09-11T22:21:18.186471+0000","last_clean":"2024-09-11T22:21:18.186471+0000","last_became_active":"2024-09-11T22:20:16.124580+0000","last_became_peered":"2024-09-11T22:20:16.124580+0000","last_unstale":"2024-09-11T22:21:18.186471+0000","last_undegraded":"2024-09-11T22:21:18.186471+0000","last_fullsized":"2024-09-11T22:21:18.186471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:56.417577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187783+0000","last_change":"2024-09-11T22:21:18.187783+0000","last_active":"2024-09-11T22:21:18.187783+0000","last_peered":"2024-09-11T22:21:18.187783+0000","last_clean":"2024-09-11T22:21:18.187783+0000","last_became_active":"2024-09-11T22:21:18.187358+0000","last_became_peered":"2024-09-11T22:21:18.187358+0000","last_unstale":"2024-09-11T22:21:18.187783+0000","last_undegraded":"2024-09-11T22:21:18.187783+0000","last_fullsized":"2024-09-11T22:21:18.187783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:41.251531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.347132+0000","last_change":"2024-09-11T22:20:16.118934+0000","last_active":"2024-09-11T22:21:18.347132+0000","last_peered":"2024-09-11T22:21:18.347132+0000","last_clean":"2024-09-11T22:21:18.347132+0000","last_became_active":"2024-09-11T22:20:16.118614+0000","last_became_peered":"2024-09-11T22:20:16.118614+0000","last_unstale":"2024-09-11T22:21:18.347132+0000","last_undegraded":"2024-09-11T22:21:18.347132+0000","last_fullsized":"2024-09-11T22:21:18.347132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:05.791161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184096+0000","last_change":"2024-09-11T22:20:18.197692+0000","last_active":"2024-09-11T22:21:18.184096+0000","last_peered":"2024-09-11T22:21:18.184096+0000","last_clean":"2024-09-11T22:21:18.184096+0000","last_became_active":"2024-09-11T22:20:18.197336+0000","last_became_peered":"2024-09-11T22:20:18.197336+0000","last_unstale":"2024-09-11T22:21:18.184096+0000","last_undegraded":"2024-09-11T22:21:18.184096+0000","last_fullsized":"2024-09-11T22:21:18.184096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:55.172568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186091+0000","last_change":"2024-09-11T22:20:16.186769+0000","last_active":"2024-09-11T22:21:18.186091+0000","last_peered":"2024-09-11T22:21:18.186091+0000","last_clean":"2024-09-11T22:21:18.186091+0000","last_became_active":"2024-09-11T22:20:16.186367+0000","last_became_peered":"2024-09-11T22:20:16.186367+0000","last_unstale":"2024-09-11T22:21:18.186091+0000","last_undegraded":"2024-09-11T22:21:18.186091+0000","last_fullsized":"2024-09-11T22:21:18.186091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.279387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185715+0000","last_change":"2024-09-11T22:20:18.195570+0000","last_active":"2024-09-11T22:21:18.185715+0000","last_peered":"2024-09-11T22:21:18.185715+0000","last_clean":"2024-09-11T22:21:18.185715+0000","last_became_active":"2024-09-11T22:20:18.195226+0000","last_became_peered":"2024-09-11T22:20:18.195226+0000","last_unstale":"2024-09-11T22:21:18.185715+0000","last_undegraded":"2024-09-11T22:21:18.185715+0000","last_fullsized":"2024-09-11T22:21:18.185715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:44.108090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.195020+0000","last_change":"2024-09-11T22:21:18.194553+0000","last_active":"2024-09-11T22:21:18.195020+0000","last_peered":"2024-09-11T22:21:18.195020+0000","last_clean":"2024-09-11T22:21:18.195020+0000","last_became_active":"2024-09-11T22:21:18.188635+0000","last_became_peered":"2024-09-11T22:21:18.188635+0000","last_unstale":"2024-09-11T22:21:18.195020+0000","last_undegraded":"2024-09-11T22:21:18.195020+0000","last_fullsized":"2024-09-11T22:21:18.195020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:34.165195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183408+0000","last_change":"2024-09-11T22:20:18.194112+0000","last_active":"2024-09-11T22:21:18.183408+0000","last_peered":"2024-09-11T22:21:18.183408+0000","last_clean":"2024-09-11T22:21:18.183408+0000","last_became_active":"2024-09-11T22:20:18.193812+0000","last_became_peered":"2024-09-11T22:20:18.193812+0000","last_unstale":"2024-09-11T22:21:18.183408+0000","last_undegraded":"2024-09-11T22:21:18.183408+0000","last_fullsized":"2024-09-11T22:21:18.183408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:35.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.198325+0000","last_change":"2024-09-11T22:21:18.198325+0000","last_active":"2024-09-11T22:21:18.198325+0000","last_peered":"2024-09-11T22:21:18.198325+0000","last_clean":"2024-09-11T22:21:18.198325+0000","last_became_active":"2024-09-11T22:21:18.193651+0000","last_became_peered":"2024-09-11T22:21:18.193651+0000","last_unstale":"2024-09-11T22:21:18.198325+0000","last_undegraded":"2024-09-11T22:21:18.198325+0000","last_fullsized":"2024-09-11T22:21:18.198325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:42.552492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185226+0000","last_change":"2024-09-11T22:20:18.191960+0000","last_active":"2024-09-11T22:21:18.185226+0000","last_peered":"2024-09-11T22:21:18.185226+0000","last_clean":"2024-09-11T22:21:18.185226+0000","last_became_active":"2024-09-11T22:20:18.191635+0000","last_became_peered":"2024-09-11T22:20:18.191635+0000","last_unstale":"2024-09-11T22:21:18.185226+0000","last_undegraded":"2024-09-11T22:21:18.185226+0000","last_fullsized":"2024-09-11T22:21:18.185226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:19:18.919247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185965+0000","last_change":"2024-09-11T22:20:16.123608+0000","last_active":"2024-09-11T22:21:18.185965+0000","last_peered":"2024-09-11T22:21:18.185965+0000","last_clean":"2024-09-11T22:21:18.185965+0000","last_became_active":"2024-09-11T22:20:16.123193+0000","last_became_peered":"2024-09-11T22:20:16.123193+0000","last_unstale":"2024-09-11T22:21:18.185965+0000","last_undegraded":"2024-09-11T22:21:18.185965+0000","last_fullsized":"2024-09-11T22:21:18.185965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:07.362115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184596+0000","last_change":"2024-09-11T22:20:18.199957+0000","last_active":"2024-09-11T22:21:18.184596+0000","last_peered":"2024-09-11T22:21:18.184596+0000","last_clean":"2024-09-11T22:21:18.184596+0000","last_became_active":"2024-09-11T22:20:18.199637+0000","last_became_peered":"2024-09-11T22:20:18.199637+0000","last_unstale":"2024-09-11T22:21:18.184596+0000","last_undegraded":"2024-09-11T22:21:18.184596+0000","last_fullsized":"2024-09-11T22:21:18.184596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:14.949002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:19.197893+0000","last_change":"2024-09-11T22:21:18.198896+0000","last_active":"2024-09-11T22:21:19.197893+0000","last_peered":"2024-09-11T22:21:19.197893+0000","last_clean":"2024-09-11T22:21:19.197893+0000","last_became_active":"2024-09-11T22:21:18.189489+0000","last_became_peered":"2024-09-11T22:21:18.189489+0000","last_unstale":"2024-09-11T22:21:19.197893+0000","last_undegraded":"2024-09-11T22:21:19.197893+0000","last_fullsized":"2024-09-11T22:21:19.197893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:19.908801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184893+0000","last_change":"2024-09-11T22:20:18.196691+0000","last_active":"2024-09-11T22:21:18.184893+0000","last_peered":"2024-09-11T22:21:18.184893+0000","last_clean":"2024-09-11T22:21:18.184893+0000","last_became_active":"2024-09-11T22:20:18.196364+0000","last_became_peered":"2024-09-11T22:20:18.196364+0000","last_unstale":"2024-09-11T22:21:18.184893+0000","last_undegraded":"2024-09-11T22:21:18.184893+0000","last_fullsized":"2024-09-11T22:21:18.184893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:50.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184308+0000","last_change":"2024-09-11T22:19:02.590079+0000","last_active":"2024-09-11T22:21:18.184308+0000","last_peered":"2024-09-11T22:21:18.184308+0000","last_clean":"2024-09-11T22:21:18.184308+0000","last_became_active":"2024-09-11T22:19:02.589644+0000","last_became_peered":"2024-09-11T22:19:02.589644+0000","last_unstale":"2024-09-11T22:21:18.184308+0000","last_undegraded":"2024-09-11T22:21:18.184308+0000","last_fullsized":"2024-09-11T22:21:18.184308+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_clean_scrub_stamp":"2024-09-11T22:15:33.244685+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:45.506190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183638+0000","last_change":"2024-09-11T22:20:16.116661+0000","last_active":"2024-09-11T22:21:18.183638+0000","last_peered":"2024-09-11T22:21:18.183638+0000","last_clean":"2024-09-11T22:21:18.183638+0000","last_became_active":"2024-09-11T22:20:16.116196+0000","last_became_peered":"2024-09-11T22:20:16.116196+0000","last_unstale":"2024-09-11T22:21:18.183638+0000","last_undegraded":"2024-09-11T22:21:18.183638+0000","last_fullsized":"2024-09-11T22:21:18.183638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:08.875170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184311+0000","last_change":"2024-09-11T22:20:18.193176+0000","last_active":"2024-09-11T22:21:18.184311+0000","last_peered":"2024-09-11T22:21:18.184311+0000","last_clean":"2024-09-11T22:21:18.184311+0000","last_became_active":"2024-09-11T22:20:18.192312+0000","last_became_peered":"2024-09-11T22:20:18.192312+0000","last_unstale":"2024-09-11T22:21:18.184311+0000","last_undegraded":"2024-09-11T22:21:18.184311+0000","last_fullsized":"2024-09-11T22:21:18.184311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:08.738569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.197381+0000","last_change":"2024-09-11T22:21:18.196985+0000","last_active":"2024-09-11T22:21:18.197381+0000","last_peered":"2024-09-11T22:21:18.197381+0000","last_clean":"2024-09-11T22:21:18.197381+0000","last_became_active":"2024-09-11T22:21:18.189487+0000","last_became_peered":"2024-09-11T22:21:18.189487+0000","last_unstale":"2024-09-11T22:21:18.197381+0000","last_undegraded":"2024-09-11T22:21:18.197381+0000","last_fullsized":"2024-09-11T22:21:18.197381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:20.715696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.181955+0000","last_change":"2024-09-11T22:20:18.191118+0000","last_active":"2024-09-11T22:21:18.181955+0000","last_peered":"2024-09-11T22:21:18.181955+0000","last_clean":"2024-09-11T22:21:18.181955+0000","last_became_active":"2024-09-11T22:20:18.190760+0000","last_became_peered":"2024-09-11T22:20:18.190760+0000","last_unstale":"2024-09-11T22:21:18.181955+0000","last_undegraded":"2024-09-11T22:21:18.181955+0000","last_fullsized":"2024-09-11T22:21:18.181955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:59.220067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:20.349901+0000","last_change":"2024-09-11T22:20:16.186407+0000","last_active":"2024-09-11T22:21:20.349901+0000","last_peered":"2024-09-11T22:21:20.349901+0000","last_clean":"2024-09-11T22:21:20.349901+0000","last_became_active":"2024-09-11T22:20:16.186105+0000","last_became_peered":"2024-09-11T22:20:16.186105+0000","last_unstale":"2024-09-11T22:21:20.349901+0000","last_undegraded":"2024-09-11T22:21:20.349901+0000","last_fullsized":"2024-09-11T22:21:20.349901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:43.942072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186762+0000","last_change":"2024-09-11T22:20:18.191970+0000","last_active":"2024-09-11T22:21:18.186762+0000","last_peered":"2024-09-11T22:21:18.186762+0000","last_clean":"2024-09-11T22:21:18.186762+0000","last_became_active":"2024-09-11T22:20:18.191623+0000","last_became_peered":"2024-09-11T22:20:18.191623+0000","last_unstale":"2024-09-11T22:21:18.186762+0000","last_undegraded":"2024-09-11T22:21:18.186762+0000","last_fullsized":"2024-09-11T22:21:18.186762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:33.213979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":82,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:20.351585+0000","last_change":"2024-09-11T22:20:16.119936+0000","last_active":"2024-09-11T22:21:20.351585+0000","last_peered":"2024-09-11T22:21:20.351585+0000","last_clean":"2024-09-11T22:21:20.351585+0000","last_became_active":"2024-09-11T22:20:16.119453+0000","last_became_peered":"2024-09-11T22:20:16.119453+0000","last_unstale":"2024-09-11T22:21:20.351585+0000","last_undegraded":"2024-09-11T22:21:20.351585+0000","last_fullsized":"2024-09-11T22:21:20.351585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:20.717980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185497+0000","last_change":"2024-09-11T22:20:18.192581+0000","last_active":"2024-09-11T22:21:18.185497+0000","last_peered":"2024-09-11T22:21:18.185497+0000","last_clean":"2024-09-11T22:21:18.185497+0000","last_became_active":"2024-09-11T22:20:18.191702+0000","last_became_peered":"2024-09-11T22:20:18.191702+0000","last_unstale":"2024-09-11T22:21:18.185497+0000","last_undegraded":"2024-09-11T22:21:18.185497+0000","last_fullsized":"2024-09-11T22:21:18.185497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:48.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186182+0000","last_change":"2024-09-11T22:20:16.186485+0000","last_active":"2024-09-11T22:21:18.186182+0000","last_peered":"2024-09-11T22:21:18.186182+0000","last_clean":"2024-09-11T22:21:18.186182+0000","last_became_active":"2024-09-11T22:20:16.186171+0000","last_became_peered":"2024-09-11T22:20:16.186171+0000","last_unstale":"2024-09-11T22:21:18.186182+0000","last_undegraded":"2024-09-11T22:21:18.186182+0000","last_fullsized":"2024-09-11T22:21:18.186182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:26.373626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184991+0000","last_change":"2024-09-11T22:20:18.191361+0000","last_active":"2024-09-11T22:21:18.184991+0000","last_peered":"2024-09-11T22:21:18.184991+0000","last_clean":"2024-09-11T22:21:18.184991+0000","last_became_active":"2024-09-11T22:20:18.191036+0000","last_became_peered":"2024-09-11T22:20:18.191036+0000","last_unstale":"2024-09-11T22:21:18.184991+0000","last_undegraded":"2024-09-11T22:21:18.184991+0000","last_fullsized":"2024-09-11T22:21:18.184991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:29.124650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186875+0000","last_change":"2024-09-11T22:20:16.188156+0000","last_active":"2024-09-11T22:21:18.186875+0000","last_peered":"2024-09-11T22:21:18.186875+0000","last_clean":"2024-09-11T22:21:18.186875+0000","last_became_active":"2024-09-11T22:20:16.187809+0000","last_became_peered":"2024-09-11T22:20:16.187809+0000","last_unstale":"2024-09-11T22:21:18.186875+0000","last_undegraded":"2024-09-11T22:21:18.186875+0000","last_fullsized":"2024-09-11T22:21:18.186875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:04.219621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.182314+0000","last_change":"2024-09-11T22:20:18.182380+0000","last_active":"2024-09-11T22:21:18.182314+0000","last_peered":"2024-09-11T22:21:18.182314+0000","last_clean":"2024-09-11T22:21:18.182314+0000","last_became_active":"2024-09-11T22:20:18.182062+0000","last_became_peered":"2024-09-11T22:20:18.182062+0000","last_unstale":"2024-09-11T22:21:18.182314+0000","last_undegraded":"2024-09-11T22:21:18.182314+0000","last_fullsized":"2024-09-11T22:21:18.182314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:15.575846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186199+0000","last_change":"2024-09-11T22:20:16.117335+0000","last_active":"2024-09-11T22:21:18.186199+0000","last_peered":"2024-09-11T22:21:18.186199+0000","last_clean":"2024-09-11T22:21:18.186199+0000","last_became_active":"2024-09-11T22:20:16.116977+0000","last_became_peered":"2024-09-11T22:20:16.116977+0000","last_unstale":"2024-09-11T22:21:18.186199+0000","last_undegraded":"2024-09-11T22:21:18.186199+0000","last_fullsized":"2024-09-11T22:21:18.186199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:51.744202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185562+0000","last_change":"2024-09-11T22:20:18.194069+0000","last_active":"2024-09-11T22:21:18.185562+0000","last_peered":"2024-09-11T22:21:18.185562+0000","last_clean":"2024-09-11T22:21:18.185562+0000","last_became_active":"2024-09-11T22:20:18.193765+0000","last_became_peered":"2024-09-11T22:20:18.193765+0000","last_unstale":"2024-09-11T22:21:18.185562+0000","last_undegraded":"2024-09-11T22:21:18.185562+0000","last_fullsized":"2024-09-11T22:21:18.185562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:25.047405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183506+0000","last_change":"2024-09-11T22:20:16.460987+0000","last_active":"2024-09-11T22:21:18.183506+0000","last_peered":"2024-09-11T22:21:18.183506+0000","last_clean":"2024-09-11T22:21:18.183506+0000","last_became_active":"2024-09-11T22:20:16.460516+0000","last_became_peered":"2024-09-11T22:20:16.460516+0000","last_unstale":"2024-09-11T22:21:18.183506+0000","last_undegraded":"2024-09-11T22:21:18.183506+0000","last_fullsized":"2024-09-11T22:21:18.183506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:00.987981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.188802+0000","last_change":"2024-09-11T22:21:18.188802+0000","last_active":"2024-09-11T22:21:18.188802+0000","last_peered":"2024-09-11T22:21:18.188802+0000","last_clean":"2024-09-11T22:21:18.188802+0000","last_became_active":"2024-09-11T22:21:18.188441+0000","last_became_peered":"2024-09-11T22:21:18.188441+0000","last_unstale":"2024-09-11T22:21:18.188802+0000","last_undegraded":"2024-09-11T22:21:18.188802+0000","last_fullsized":"2024-09-11T22:21:18.188802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:02.295441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187676+0000","last_change":"2024-09-11T22:20:16.186498+0000","last_active":"2024-09-11T22:21:18.187676+0000","last_peered":"2024-09-11T22:21:18.187676+0000","last_clean":"2024-09-11T22:21:18.187676+0000","last_became_active":"2024-09-11T22:20:16.186214+0000","last_became_peered":"2024-09-11T22:20:16.186214+0000","last_unstale":"2024-09-11T22:21:18.187676+0000","last_undegraded":"2024-09-11T22:21:18.187676+0000","last_fullsized":"2024-09-11T22:21:18.187676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:09.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185428+0000","last_change":"2024-09-11T22:20:18.187489+0000","last_active":"2024-09-11T22:21:18.185428+0000","last_peered":"2024-09-11T22:21:18.185428+0000","last_clean":"2024-09-11T22:21:18.185428+0000","last_became_active":"2024-09-11T22:20:18.185649+0000","last_became_peered":"2024-09-11T22:20:18.185649+0000","last_unstale":"2024-09-11T22:21:18.185428+0000","last_undegraded":"2024-09-11T22:21:18.185428+0000","last_fullsized":"2024-09-11T22:21:18.185428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:42.825075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184722+0000","last_change":"2024-09-11T22:20:16.117627+0000","last_active":"2024-09-11T22:21:18.184722+0000","last_peered":"2024-09-11T22:21:18.184722+0000","last_clean":"2024-09-11T22:21:18.184722+0000","last_became_active":"2024-09-11T22:20:16.117243+0000","last_became_peered":"2024-09-11T22:20:16.117243+0000","last_unstale":"2024-09-11T22:21:18.184722+0000","last_undegraded":"2024-09-11T22:21:18.184722+0000","last_fullsized":"2024-09-11T22:21:18.184722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:58.240535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183220+0000","last_change":"2024-09-11T22:20:18.184326+0000","last_active":"2024-09-11T22:21:18.183220+0000","last_peered":"2024-09-11T22:21:18.183220+0000","last_clean":"2024-09-11T22:21:18.183220+0000","last_became_active":"2024-09-11T22:20:18.183995+0000","last_became_peered":"2024-09-11T22:20:18.183995+0000","last_unstale":"2024-09-11T22:21:18.183220+0000","last_undegraded":"2024-09-11T22:21:18.183220+0000","last_fullsized":"2024-09-11T22:21:18.183220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:33.468000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187931+0000","last_change":"2024-09-11T22:20:16.127180+0000","last_active":"2024-09-11T22:21:18.187931+0000","last_peered":"2024-09-11T22:21:18.187931+0000","last_clean":"2024-09-11T22:21:18.187931+0000","last_became_active":"2024-09-11T22:20:16.126696+0000","last_became_peered":"2024-09-11T22:20:16.126696+0000","last_unstale":"2024-09-11T22:21:18.187931+0000","last_undegraded":"2024-09-11T22:21:18.187931+0000","last_fullsized":"2024-09-11T22:21:18.187931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:02.637225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.189710+0000","last_change":"2024-09-11T22:21:18.188614+0000","last_active":"2024-09-11T22:21:18.189710+0000","last_peered":"2024-09-11T22:21:18.189710+0000","last_clean":"2024-09-11T22:21:18.189710+0000","last_became_active":"2024-09-11T22:21:18.188277+0000","last_became_peered":"2024-09-11T22:21:18.188277+0000","last_unstale":"2024-09-11T22:21:18.189710+0000","last_undegraded":"2024-09-11T22:21:18.189710+0000","last_fullsized":"2024-09-11T22:21:18.189710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:59.995647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186347+0000","last_change":"2024-09-11T22:20:16.128311+0000","last_active":"2024-09-11T22:21:18.186347+0000","last_peered":"2024-09-11T22:21:18.186347+0000","last_clean":"2024-09-11T22:21:18.186347+0000","last_became_active":"2024-09-11T22:20:16.127853+0000","last_became_peered":"2024-09-11T22:20:16.127853+0000","last_unstale":"2024-09-11T22:21:18.186347+0000","last_undegraded":"2024-09-11T22:21:18.186347+0000","last_fullsized":"2024-09-11T22:21:18.186347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:57.562470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184751+0000","last_change":"2024-09-11T22:20:18.195782+0000","last_active":"2024-09-11T22:21:18.184751+0000","last_peered":"2024-09-11T22:21:18.184751+0000","last_clean":"2024-09-11T22:21:18.184751+0000","last_became_active":"2024-09-11T22:20:18.195365+0000","last_became_peered":"2024-09-11T22:20:18.195365+0000","last_unstale":"2024-09-11T22:21:18.184751+0000","last_undegraded":"2024-09-11T22:21:18.184751+0000","last_fullsized":"2024-09-11T22:21:18.184751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:49.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185795+0000","last_change":"2024-09-11T22:20:18.199344+0000","last_active":"2024-09-11T22:21:18.185795+0000","last_peered":"2024-09-11T22:21:18.185795+0000","last_clean":"2024-09-11T22:21:18.185795+0000","last_became_active":"2024-09-11T22:20:18.199016+0000","last_became_peered":"2024-09-11T22:20:18.199016+0000","last_unstale":"2024-09-11T22:21:18.185795+0000","last_undegraded":"2024-09-11T22:21:18.185795+0000","last_fullsized":"2024-09-11T22:21:18.185795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:38.322831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187943+0000","last_change":"2024-09-11T22:20:16.122236+0000","last_active":"2024-09-11T22:21:18.187943+0000","last_peered":"2024-09-11T22:21:18.187943+0000","last_clean":"2024-09-11T22:21:18.187943+0000","last_became_active":"2024-09-11T22:20:16.121867+0000","last_became_peered":"2024-09-11T22:20:16.121867+0000","last_unstale":"2024-09-11T22:21:18.187943+0000","last_undegraded":"2024-09-11T22:21:18.187943+0000","last_fullsized":"2024-09-11T22:21:18.187943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:56.782445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.188621+0000","last_change":"2024-09-11T22:20:18.187338+0000","last_active":"2024-09-11T22:21:18.188621+0000","last_peered":"2024-09-11T22:21:18.188621+0000","last_clean":"2024-09-11T22:21:18.188621+0000","last_became_active":"2024-09-11T22:20:18.186985+0000","last_became_peered":"2024-09-11T22:20:18.186985+0000","last_unstale":"2024-09-11T22:21:18.188621+0000","last_undegraded":"2024-09-11T22:21:18.188621+0000","last_fullsized":"2024-09-11T22:21:18.188621+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:58.170680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186747+0000","last_change":"2024-09-11T22:20:16.128579+0000","last_active":"2024-09-11T22:21:18.186747+0000","last_peered":"2024-09-11T22:21:18.186747+0000","last_clean":"2024-09-11T22:21:18.186747+0000","last_became_active":"2024-09-11T22:20:16.127864+0000","last_became_peered":"2024-09-11T22:20:16.127864+0000","last_unstale":"2024-09-11T22:21:18.186747+0000","last_undegraded":"2024-09-11T22:21:18.186747+0000","last_fullsized":"2024-09-11T22:21:18.186747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:11:32.005487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186436+0000","last_change":"2024-09-11T22:20:18.185205+0000","last_active":"2024-09-11T22:21:18.186436+0000","last_peered":"2024-09-11T22:21:18.186436+0000","last_clean":"2024-09-11T22:21:18.186436+0000","last_became_active":"2024-09-11T22:20:18.184402+0000","last_became_peered":"2024-09-11T22:20:18.184402+0000","last_unstale":"2024-09-11T22:21:18.186436+0000","last_undegraded":"2024-09-11T22:21:18.186436+0000","last_fullsized":"2024-09-11T22:21:18.186436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:17:48.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185468+0000","last_change":"2024-09-11T22:20:16.186372+0000","last_active":"2024-09-11T22:21:18.185468+0000","last_peered":"2024-09-11T22:21:18.185468+0000","last_clean":"2024-09-11T22:21:18.185468+0000","last_became_active":"2024-09-11T22:20:16.185990+0000","last_became_peered":"2024-09-11T22:20:16.185990+0000","last_unstale":"2024-09-11T22:21:18.185468+0000","last_undegraded":"2024-09-11T22:21:18.185468+0000","last_fullsized":"2024-09-11T22:21:18.185468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:23.296379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185002+0000","last_change":"2024-09-11T22:20:18.190670+0000","last_active":"2024-09-11T22:21:18.185002+0000","last_peered":"2024-09-11T22:21:18.185002+0000","last_clean":"2024-09-11T22:21:18.185002+0000","last_became_active":"2024-09-11T22:20:18.190339+0000","last_became_peered":"2024-09-11T22:20:18.190339+0000","last_unstale":"2024-09-11T22:21:18.185002+0000","last_undegraded":"2024-09-11T22:21:18.185002+0000","last_fullsized":"2024-09-11T22:21:18.185002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.085305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":90,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:22.353540+0000","last_change":"2024-09-11T22:20:16.117646+0000","last_active":"2024-09-11T22:21:22.353540+0000","last_peered":"2024-09-11T22:21:22.353540+0000","last_clean":"2024-09-11T22:21:22.353540+0000","last_became_active":"2024-09-11T22:20:16.117111+0000","last_became_peered":"2024-09-11T22:20:16.117111+0000","last_unstale":"2024-09-11T22:21:22.353540+0000","last_undegraded":"2024-09-11T22:21:22.353540+0000","last_fullsized":"2024-09-11T22:21:22.353540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:36.470158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66579,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":497,"num_read_kb":346,"num_write":241,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1192,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249094,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579931,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":434362,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1052672,"data_stored":1085133,"data_compressed":42058,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":921600,"data_stored":434270,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201322,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":434626,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":970752,"data_stored":1027168,"data_compressed":42058,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593780,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":905216,"data_stored":434380,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":983040,"data_stored":1026453,"data_compressed":42058,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986238,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":435018,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215172,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":433812,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":978944,"data_stored":493714,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":966656,"data_stored":495057,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":925696,"data_stored":435326,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:21:25.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:21:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:25.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:25 smithi005 ceph-mon[30943]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1 op/s; 148 B/s, 1 objects/s recovering 2024-09-11T22:21:25.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:25.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:25 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:25.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:25 smithi005 ceph-mon[30943]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:21:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:25 smithi159 ceph-mon[35534]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1 op/s; 148 B/s, 1 objects/s recovering 2024-09-11T22:21:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:25 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:25 smithi159 ceph-mon[35534]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:21:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:25 smithi172 ceph-mon[35461]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1 op/s; 148 B/s, 1 objects/s recovering 2024-09-11T22:21:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:25 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:25 smithi172 ceph-mon[35461]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:21:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.242+0000 7fdff983d640 1 -- 172.21.15.5:0/1721096687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff4103660 msgr2=0x7fdff410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.242+0000 7fdff983d640 1 --2- 172.21.15.5:0/1721096687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff4103660 0x7fdff410bb50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8009900 tx=0x7fdfe802a480 comp rx=0 tx=0).stop 2024-09-11T22:21:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.242+0000 7fdff983d640 1 -- 172.21.15.5:0/1721096687 shutdown_connections 2024-09-11T22:21:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.242+0000 7fdff983d640 1 --2- 172.21.15.5:0/1721096687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdff410c170 0x7fdff410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.242+0000 7fdff983d640 1 --2- 172.21.15.5:0/1721096687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff4103660 0x7fdff410bb50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.242+0000 7fdff983d640 1 --2- 172.21.15.5:0/1721096687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdff4102d40 0x7fdff4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.242+0000 7fdff983d640 1 -- 172.21.15.5:0/1721096687 >> 172.21.15.5:0/1721096687 conn(0x7fdff40fc7a0 msgr2=0x7fdff40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.242+0000 7fdff983d640 1 -- 172.21.15.5:0/1721096687 shutdown_connections 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.242+0000 7fdff983d640 1 -- 172.21.15.5:0/1721096687 wait complete. 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.243+0000 7fdff983d640 1 Processor -- start 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.243+0000 7fdff983d640 1 -- start start 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.243+0000 7fdff983d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdff4102d40 0x7fdff419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.243+0000 7fdff983d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdff4103660 0x7fdff419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.243+0000 7fdff983d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff410c170 0x7fdff419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.243+0000 7fdff983d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4111fc0 con 0x7fdff410c170 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.243+0000 7fdff983d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4111e40 con 0x7fdff4102d40 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.243+0000 7fdff983d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4112140 con 0x7fdff4103660 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff410c170 0x7fdff419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff2ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdff4102d40 0x7fdff419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff410c170 0x7fdff419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39408/0 (socket says 172.21.15.5:39408) 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff37fe640 1 -- 172.21.15.5:0/902376928 learned_addr learned my addr 172.21.15.5:0/902376928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff27fc640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdff4103660 0x7fdff419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff37fe640 1 -- 172.21.15.5:0/902376928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdff4103660 msgr2=0x7fdff419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff37fe640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdff4103660 0x7fdff419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff37fe640 1 -- 172.21.15.5:0/902376928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdff4102d40 msgr2=0x7fdff419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff37fe640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdff4102d40 0x7fdff419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff37fe640 1 -- 172.21.15.5:0/902376928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff419f630 con 0x7fdff410c170 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff2ffd640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdff4102d40 0x7fdff419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff37fe640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff410c170 0x7fdff419ef50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00eb10 tx=0x7fdfec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff883b640 1 -- 172.21.15.5:0/902376928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec004b20 con 0x7fdff410c170 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff983d640 1 -- 172.21.15.5:0/902376928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff419f920 con 0x7fdff410c170 2024-09-11T22:21:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff883b640 1 -- 172.21.15.5:0/902376928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfec004cc0 con 0x7fdff410c170 2024-09-11T22:21:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff883b640 1 -- 172.21.15.5:0/902376928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec010760 con 0x7fdff410c170 2024-09-11T22:21:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.244+0000 7fdff983d640 1 -- 172.21.15.5:0/902376928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff41a7200 con 0x7fdff410c170 2024-09-11T22:21:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.245+0000 7fdff983d640 1 -- 172.21.15.5:0/902376928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff4103120 con 0x7fdff410c170 2024-09-11T22:21:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.246+0000 7fdff883b640 1 -- 172.21.15.5:0/902376928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdfec005020 con 0x7fdff410c170 2024-09-11T22:21:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.247+0000 7fdff883b640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdfdc0a2c10 0x7fdfdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.247+0000 7fdff2ffd640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdfdc0a2c10 0x7fdfdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.247+0000 7fdff883b640 1 -- 172.21.15.5:0/902376928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fdfec0cf7f0 con 0x7fdff410c170 2024-09-11T22:21:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.248+0000 7fdff2ffd640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdfdc0a2c10 0x7fdfdc0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005fd0 tx=0x7fdfe400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:26.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.251+0000 7fdff883b640 1 -- 172.21.15.5:0/902376928 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdfec09c570 con 0x7fdff410c170 2024-09-11T22:21:26.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.427+0000 7fdff983d640 1 -- 172.21.15.5:0/902376928 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdff40feac0 con 0x7fdfdc0a2c10 2024-09-11T22:21:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.433+0000 7fdff883b640 1 -- 172.21.15.5:0/902376928 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259058 (secure 0 0 0) 0x7fdff40feac0 con 0x7fdfdc0a2c10 2024-09-11T22:21:26.435 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-11T22:21:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 -- 172.21.15.5:0/902376928 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdfdc0a2c10 msgr2=0x7fdfdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdfdc0a2c10 0x7fdfdc0a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005fd0 tx=0x7fdfe400b390 comp rx=0 tx=0).stop 2024-09-11T22:21:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 -- 172.21.15.5:0/902376928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff410c170 msgr2=0x7fdff419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff410c170 0x7fdff419ef50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00eb10 tx=0x7fdfec00c2d0 comp rx=0 tx=0).stop 2024-09-11T22:21:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 -- 172.21.15.5:0/902376928 shutdown_connections 2024-09-11T22:21:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdfdc0a2c10 0x7fdfdc0a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff410c170 0x7fdff419ef50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdff4103660 0x7fdff419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 --2- 172.21.15.5:0/902376928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdff4102d40 0x7fdff419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 -- 172.21.15.5:0/902376928 >> 172.21.15.5:0/902376928 conn(0x7fdff40fc7a0 msgr2=0x7fdff40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 -- 172.21.15.5:0/902376928 shutdown_connections 2024-09-11T22:21:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:26.436+0000 7fdff983d640 1 -- 172.21.15.5:0/902376928 wait complete. 2024-09-11T22:21:27.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:21:27.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:21:27.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:26 smithi005 ceph-mon[30943]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:27.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:26 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:21:27.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:26 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:21:27.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:26 smithi159 ceph-mon[35534]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:27.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:26 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:21:27.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:26 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:21:27.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:26 smithi172 ceph-mon[35461]: from='mgr.14148 172.21.15.5:0/401561738' entity='mgr.x' 2024-09-11T22:21:27.137 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-11T22:21:24.457748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657181,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":633,"num_read_kb":465,"num_write":424,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181928,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":289,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381220,"kb_used_data":11088,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124560604,"statfs":{"total":1151940427776,"available":1151550058496,"internally_reserved":0,"allocated":11354112,"data_stored":7169319,"data_compressed":178122,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":378713401},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":27,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001859"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183642+0000","last_change":"2024-09-11T22:20:18.189744+0000","last_active":"2024-09-11T22:21:18.183642+0000","last_peered":"2024-09-11T22:21:18.183642+0000","last_clean":"2024-09-11T22:21:18.183642+0000","last_became_active":"2024-09-11T22:20:18.189398+0000","last_became_peered":"2024-09-11T22:20:18.189398+0000","last_unstale":"2024-09-11T22:21:18.183642+0000","last_undegraded":"2024-09-11T22:21:18.183642+0000","last_fullsized":"2024-09-11T22:21:18.183642+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:24.152589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186316+0000","last_change":"2024-09-11T22:20:18.186727+0000","last_active":"2024-09-11T22:21:18.186316+0000","last_peered":"2024-09-11T22:21:18.186316+0000","last_clean":"2024-09-11T22:21:18.186316+0000","last_became_active":"2024-09-11T22:20:18.186414+0000","last_became_peered":"2024-09-11T22:20:18.186414+0000","last_unstale":"2024-09-11T22:21:18.186316+0000","last_undegraded":"2024-09-11T22:21:18.186316+0000","last_fullsized":"2024-09-11T22:21:18.186316+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:53.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186059+0000","last_change":"2024-09-11T22:20:18.179984+0000","last_active":"2024-09-11T22:21:18.186059+0000","last_peered":"2024-09-11T22:21:18.186059+0000","last_clean":"2024-09-11T22:21:18.186059+0000","last_became_active":"2024-09-11T22:20:18.179329+0000","last_became_peered":"2024-09-11T22:20:18.179329+0000","last_unstale":"2024-09-11T22:21:18.186059+0000","last_undegraded":"2024-09-11T22:21:18.186059+0000","last_fullsized":"2024-09-11T22:21:18.186059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:19.149970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184029+0000","last_change":"2024-09-11T22:20:18.191768+0000","last_active":"2024-09-11T22:21:18.184029+0000","last_peered":"2024-09-11T22:21:18.184029+0000","last_clean":"2024-09-11T22:21:18.184029+0000","last_became_active":"2024-09-11T22:20:18.191336+0000","last_became_peered":"2024-09-11T22:20:18.191336+0000","last_unstale":"2024-09-11T22:21:18.184029+0000","last_undegraded":"2024-09-11T22:21:18.184029+0000","last_fullsized":"2024-09-11T22:21:18.184029+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:44.768932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186238+0000","last_change":"2024-09-11T22:20:18.181469+0000","last_active":"2024-09-11T22:21:18.186238+0000","last_peered":"2024-09-11T22:21:18.186238+0000","last_clean":"2024-09-11T22:21:18.186238+0000","last_became_active":"2024-09-11T22:20:18.181100+0000","last_became_peered":"2024-09-11T22:20:18.181100+0000","last_unstale":"2024-09-11T22:21:18.186238+0000","last_undegraded":"2024-09-11T22:21:18.186238+0000","last_fullsized":"2024-09-11T22:21:18.186238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:14.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186974+0000","last_change":"2024-09-11T22:20:18.190855+0000","last_active":"2024-09-11T22:21:18.186974+0000","last_peered":"2024-09-11T22:21:18.186974+0000","last_clean":"2024-09-11T22:21:18.186974+0000","last_became_active":"2024-09-11T22:20:18.190523+0000","last_became_peered":"2024-09-11T22:20:18.190523+0000","last_unstale":"2024-09-11T22:21:18.186974+0000","last_undegraded":"2024-09-11T22:21:18.186974+0000","last_fullsized":"2024-09-11T22:21:18.186974+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:18.574957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184449+0000","last_change":"2024-09-11T22:20:18.199213+0000","last_active":"2024-09-11T22:21:18.184449+0000","last_peered":"2024-09-11T22:21:18.184449+0000","last_clean":"2024-09-11T22:21:18.184449+0000","last_became_active":"2024-09-11T22:20:18.198913+0000","last_became_peered":"2024-09-11T22:20:18.198913+0000","last_unstale":"2024-09-11T22:21:18.184449+0000","last_undegraded":"2024-09-11T22:21:18.184449+0000","last_fullsized":"2024-09-11T22:21:18.184449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:22.682802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184120+0000","last_change":"2024-09-11T22:20:18.192458+0000","last_active":"2024-09-11T22:21:18.184120+0000","last_peered":"2024-09-11T22:21:18.184120+0000","last_clean":"2024-09-11T22:21:18.184120+0000","last_became_active":"2024-09-11T22:20:18.191967+0000","last_became_peered":"2024-09-11T22:20:18.191967+0000","last_unstale":"2024-09-11T22:21:18.184120+0000","last_undegraded":"2024-09-11T22:21:18.184120+0000","last_fullsized":"2024-09-11T22:21:18.184120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:52.686157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.191732+0000","last_change":"2024-09-11T22:21:18.191732+0000","last_active":"2024-09-11T22:21:18.191732+0000","last_peered":"2024-09-11T22:21:18.191732+0000","last_clean":"2024-09-11T22:21:18.191732+0000","last_became_active":"2024-09-11T22:21:18.191375+0000","last_became_peered":"2024-09-11T22:21:18.191375+0000","last_unstale":"2024-09-11T22:21:18.191732+0000","last_undegraded":"2024-09-11T22:21:18.191732+0000","last_fullsized":"2024-09-11T22:21:18.191732+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:13.645224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185803+0000","last_change":"2024-09-11T22:20:18.193771+0000","last_active":"2024-09-11T22:21:18.185803+0000","last_peered":"2024-09-11T22:21:18.185803+0000","last_clean":"2024-09-11T22:21:18.185803+0000","last_became_active":"2024-09-11T22:20:18.193041+0000","last_became_peered":"2024-09-11T22:20:18.193041+0000","last_unstale":"2024-09-11T22:21:18.185803+0000","last_undegraded":"2024-09-11T22:21:18.185803+0000","last_fullsized":"2024-09-11T22:21:18.185803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:06.656169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184811+0000","last_change":"2024-09-11T22:20:18.197933+0000","last_active":"2024-09-11T22:21:18.184811+0000","last_peered":"2024-09-11T22:21:18.184811+0000","last_clean":"2024-09-11T22:21:18.184811+0000","last_became_active":"2024-09-11T22:20:18.197210+0000","last_became_peered":"2024-09-11T22:20:18.197210+0000","last_unstale":"2024-09-11T22:21:18.184811+0000","last_undegraded":"2024-09-11T22:21:18.184811+0000","last_fullsized":"2024-09-11T22:21:18.184811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:42.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184129+0000","last_change":"2024-09-11T22:20:18.190739+0000","last_active":"2024-09-11T22:21:18.184129+0000","last_peered":"2024-09-11T22:21:18.184129+0000","last_clean":"2024-09-11T22:21:18.184129+0000","last_became_active":"2024-09-11T22:20:18.190122+0000","last_became_peered":"2024-09-11T22:20:18.190122+0000","last_unstale":"2024-09-11T22:21:18.184129+0000","last_undegraded":"2024-09-11T22:21:18.184129+0000","last_fullsized":"2024-09-11T22:21:18.184129+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.585035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.182268+0000","last_change":"2024-09-11T22:20:18.190303+0000","last_active":"2024-09-11T22:21:18.182268+0000","last_peered":"2024-09-11T22:21:18.182268+0000","last_clean":"2024-09-11T22:21:18.182268+0000","last_became_active":"2024-09-11T22:20:18.189962+0000","last_became_peered":"2024-09-11T22:20:18.189962+0000","last_unstale":"2024-09-11T22:21:18.182268+0000","last_undegraded":"2024-09-11T22:21:18.182268+0000","last_fullsized":"2024-09-11T22:21:18.182268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:44.872813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185672+0000","last_change":"2024-09-11T22:20:18.185893+0000","last_active":"2024-09-11T22:21:18.185672+0000","last_peered":"2024-09-11T22:21:18.185672+0000","last_clean":"2024-09-11T22:21:18.185672+0000","last_became_active":"2024-09-11T22:20:18.185592+0000","last_became_peered":"2024-09-11T22:20:18.185592+0000","last_unstale":"2024-09-11T22:21:18.185672+0000","last_undegraded":"2024-09-11T22:21:18.185672+0000","last_fullsized":"2024-09-11T22:21:18.185672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:14.725301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185405+0000","last_change":"2024-09-11T22:20:18.193943+0000","last_active":"2024-09-11T22:21:18.185405+0000","last_peered":"2024-09-11T22:21:18.185405+0000","last_clean":"2024-09-11T22:21:18.185405+0000","last_became_active":"2024-09-11T22:20:18.193558+0000","last_became_peered":"2024-09-11T22:20:18.193558+0000","last_unstale":"2024-09-11T22:21:18.185405+0000","last_undegraded":"2024-09-11T22:21:18.185405+0000","last_fullsized":"2024-09-11T22:21:18.185405+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:08.736161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187501+0000","last_change":"2024-09-11T22:20:18.191815+0000","last_active":"2024-09-11T22:21:18.187501+0000","last_peered":"2024-09-11T22:21:18.187501+0000","last_clean":"2024-09-11T22:21:18.187501+0000","last_became_active":"2024-09-11T22:20:18.191483+0000","last_became_peered":"2024-09-11T22:20:18.191483+0000","last_unstale":"2024-09-11T22:21:18.187501+0000","last_undegraded":"2024-09-11T22:21:18.187501+0000","last_fullsized":"2024-09-11T22:21:18.187501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:26.432117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183752+0000","last_change":"2024-09-11T22:20:18.190758+0000","last_active":"2024-09-11T22:21:18.183752+0000","last_peered":"2024-09-11T22:21:18.183752+0000","last_clean":"2024-09-11T22:21:18.183752+0000","last_became_active":"2024-09-11T22:20:18.190431+0000","last_became_peered":"2024-09-11T22:20:18.190431+0000","last_unstale":"2024-09-11T22:21:18.183752+0000","last_undegraded":"2024-09-11T22:21:18.183752+0000","last_fullsized":"2024-09-11T22:21:18.183752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:10.927051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186191+0000","last_change":"2024-09-11T22:20:18.185191+0000","last_active":"2024-09-11T22:21:18.186191+0000","last_peered":"2024-09-11T22:21:18.186191+0000","last_clean":"2024-09-11T22:21:18.186191+0000","last_became_active":"2024-09-11T22:20:18.184906+0000","last_became_peered":"2024-09-11T22:20:18.184906+0000","last_unstale":"2024-09-11T22:21:18.186191+0000","last_undegraded":"2024-09-11T22:21:18.186191+0000","last_fullsized":"2024-09-11T22:21:18.186191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:32.294780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185228+0000","last_change":"2024-09-11T22:20:18.187137+0000","last_active":"2024-09-11T22:21:18.185228+0000","last_peered":"2024-09-11T22:21:18.185228+0000","last_clean":"2024-09-11T22:21:18.185228+0000","last_became_active":"2024-09-11T22:20:18.186770+0000","last_became_peered":"2024-09-11T22:20:18.186770+0000","last_unstale":"2024-09-11T22:21:18.185228+0000","last_undegraded":"2024-09-11T22:21:18.185228+0000","last_fullsized":"2024-09-11T22:21:18.185228+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:59.917695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183169+0000","last_change":"2024-09-11T22:20:18.197137+0000","last_active":"2024-09-11T22:21:18.183169+0000","last_peered":"2024-09-11T22:21:18.183169+0000","last_clean":"2024-09-11T22:21:18.183169+0000","last_became_active":"2024-09-11T22:20:18.192810+0000","last_became_peered":"2024-09-11T22:20:18.192810+0000","last_unstale":"2024-09-11T22:21:18.183169+0000","last_undegraded":"2024-09-11T22:21:18.183169+0000","last_fullsized":"2024-09-11T22:21:18.183169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:57.680726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.182325+0000","last_change":"2024-09-11T22:20:18.187568+0000","last_active":"2024-09-11T22:21:18.182325+0000","last_peered":"2024-09-11T22:21:18.182325+0000","last_clean":"2024-09-11T22:21:18.182325+0000","last_became_active":"2024-09-11T22:20:18.187254+0000","last_became_peered":"2024-09-11T22:20:18.187254+0000","last_unstale":"2024-09-11T22:21:18.182325+0000","last_undegraded":"2024-09-11T22:21:18.182325+0000","last_fullsized":"2024-09-11T22:21:18.182325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:41.724926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187943+0000","last_change":"2024-09-11T22:20:18.190376+0000","last_active":"2024-09-11T22:21:18.187943+0000","last_peered":"2024-09-11T22:21:18.187943+0000","last_clean":"2024-09-11T22:21:18.187943+0000","last_became_active":"2024-09-11T22:20:18.190079+0000","last_became_peered":"2024-09-11T22:20:18.190079+0000","last_unstale":"2024-09-11T22:21:18.187943+0000","last_undegraded":"2024-09-11T22:21:18.187943+0000","last_fullsized":"2024-09-11T22:21:18.187943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:42.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184566+0000","last_change":"2024-09-11T22:20:18.188503+0000","last_active":"2024-09-11T22:21:18.184566+0000","last_peered":"2024-09-11T22:21:18.184566+0000","last_clean":"2024-09-11T22:21:18.184566+0000","last_became_active":"2024-09-11T22:20:18.187957+0000","last_became_peered":"2024-09-11T22:20:18.187957+0000","last_unstale":"2024-09-11T22:21:18.184566+0000","last_undegraded":"2024-09-11T22:21:18.184566+0000","last_fullsized":"2024-09-11T22:21:18.184566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:44:25.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186673+0000","last_change":"2024-09-11T22:20:18.196936+0000","last_active":"2024-09-11T22:21:18.186673+0000","last_peered":"2024-09-11T22:21:18.186673+0000","last_clean":"2024-09-11T22:21:18.186673+0000","last_became_active":"2024-09-11T22:20:18.196493+0000","last_became_peered":"2024-09-11T22:20:18.196493+0000","last_unstale":"2024-09-11T22:21:18.186673+0000","last_undegraded":"2024-09-11T22:21:18.186673+0000","last_fullsized":"2024-09-11T22:21:18.186673+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:06.189806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185464+0000","last_change":"2024-09-11T22:20:18.195260+0000","last_active":"2024-09-11T22:21:18.185464+0000","last_peered":"2024-09-11T22:21:18.185464+0000","last_clean":"2024-09-11T22:21:18.185464+0000","last_became_active":"2024-09-11T22:20:18.194956+0000","last_became_peered":"2024-09-11T22:20:18.194956+0000","last_unstale":"2024-09-11T22:21:18.185464+0000","last_undegraded":"2024-09-11T22:21:18.185464+0000","last_fullsized":"2024-09-11T22:21:18.185464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:34.483291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186741+0000","last_change":"2024-09-11T22:20:18.190591+0000","last_active":"2024-09-11T22:21:18.186741+0000","last_peered":"2024-09-11T22:21:18.186741+0000","last_clean":"2024-09-11T22:21:18.186741+0000","last_became_active":"2024-09-11T22:20:18.190007+0000","last_became_peered":"2024-09-11T22:20:18.190007+0000","last_unstale":"2024-09-11T22:21:18.186741+0000","last_undegraded":"2024-09-11T22:21:18.186741+0000","last_fullsized":"2024-09-11T22:21:18.186741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:46.281705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183329+0000","last_change":"2024-09-11T22:20:18.193377+0000","last_active":"2024-09-11T22:21:18.183329+0000","last_peered":"2024-09-11T22:21:18.183329+0000","last_clean":"2024-09-11T22:21:18.183329+0000","last_became_active":"2024-09-11T22:20:18.193003+0000","last_became_peered":"2024-09-11T22:20:18.193003+0000","last_unstale":"2024-09-11T22:21:18.183329+0000","last_undegraded":"2024-09-11T22:21:18.183329+0000","last_fullsized":"2024-09-11T22:21:18.183329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:59.052966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.182200+0000","last_change":"2024-09-11T22:20:18.189405+0000","last_active":"2024-09-11T22:21:18.182200+0000","last_peered":"2024-09-11T22:21:18.182200+0000","last_clean":"2024-09-11T22:21:18.182200+0000","last_became_active":"2024-09-11T22:20:18.189098+0000","last_became_peered":"2024-09-11T22:20:18.189098+0000","last_unstale":"2024-09-11T22:21:18.182200+0000","last_undegraded":"2024-09-11T22:21:18.182200+0000","last_fullsized":"2024-09-11T22:21:18.182200+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:47.088192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185806+0000","last_change":"2024-09-11T22:20:18.191005+0000","last_active":"2024-09-11T22:21:18.185806+0000","last_peered":"2024-09-11T22:21:18.185806+0000","last_clean":"2024-09-11T22:21:18.185806+0000","last_became_active":"2024-09-11T22:20:18.190681+0000","last_became_peered":"2024-09-11T22:20:18.190681+0000","last_unstale":"2024-09-11T22:21:18.185806+0000","last_undegraded":"2024-09-11T22:21:18.185806+0000","last_fullsized":"2024-09-11T22:21:18.185806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:12.903929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187818+0000","last_change":"2024-09-11T22:20:18.187920+0000","last_active":"2024-09-11T22:21:18.187818+0000","last_peered":"2024-09-11T22:21:18.187818+0000","last_clean":"2024-09-11T22:21:18.187818+0000","last_became_active":"2024-09-11T22:20:18.187526+0000","last_became_peered":"2024-09-11T22:20:18.187526+0000","last_unstale":"2024-09-11T22:21:18.187818+0000","last_undegraded":"2024-09-11T22:21:18.187818+0000","last_fullsized":"2024-09-11T22:21:18.187818+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.347857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187211+0000","last_change":"2024-09-11T22:20:18.193869+0000","last_active":"2024-09-11T22:21:18.187211+0000","last_peered":"2024-09-11T22:21:18.187211+0000","last_clean":"2024-09-11T22:21:18.187211+0000","last_became_active":"2024-09-11T22:20:18.193468+0000","last_became_peered":"2024-09-11T22:20:18.193468+0000","last_unstale":"2024-09-11T22:21:18.187211+0000","last_undegraded":"2024-09-11T22:21:18.187211+0000","last_fullsized":"2024-09-11T22:21:18.187211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:59.619611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183106+0000","last_change":"2024-09-11T22:20:18.193947+0000","last_active":"2024-09-11T22:21:18.183106+0000","last_peered":"2024-09-11T22:21:18.183106+0000","last_clean":"2024-09-11T22:21:18.183106+0000","last_became_active":"2024-09-11T22:20:18.192257+0000","last_became_peered":"2024-09-11T22:20:18.192257+0000","last_unstale":"2024-09-11T22:21:18.183106+0000","last_undegraded":"2024-09-11T22:21:18.183106+0000","last_fullsized":"2024-09-11T22:21:18.183106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.478730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185685+0000","last_change":"2024-09-11T22:20:18.184549+0000","last_active":"2024-09-11T22:21:18.185685+0000","last_peered":"2024-09-11T22:21:18.185685+0000","last_clean":"2024-09-11T22:21:18.185685+0000","last_became_active":"2024-09-11T22:20:18.184075+0000","last_became_peered":"2024-09-11T22:20:18.184075+0000","last_unstale":"2024-09-11T22:21:18.185685+0000","last_undegraded":"2024-09-11T22:21:18.185685+0000","last_fullsized":"2024-09-11T22:21:18.185685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:50.095643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.338878+0000","last_change":"2024-09-11T22:20:16.128524+0000","last_active":"2024-09-11T22:21:18.338878+0000","last_peered":"2024-09-11T22:21:18.338878+0000","last_clean":"2024-09-11T22:21:18.338878+0000","last_became_active":"2024-09-11T22:20:16.128185+0000","last_became_peered":"2024-09-11T22:20:16.128185+0000","last_unstale":"2024-09-11T22:21:18.338878+0000","last_undegraded":"2024-09-11T22:21:18.338878+0000","last_fullsized":"2024-09-11T22:21:18.338878+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.022931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186233+0000","last_change":"2024-09-11T22:20:18.191578+0000","last_active":"2024-09-11T22:21:18.186233+0000","last_peered":"2024-09-11T22:21:18.186233+0000","last_clean":"2024-09-11T22:21:18.186233+0000","last_became_active":"2024-09-11T22:20:18.189568+0000","last_became_peered":"2024-09-11T22:20:18.189568+0000","last_unstale":"2024-09-11T22:21:18.186233+0000","last_undegraded":"2024-09-11T22:21:18.186233+0000","last_fullsized":"2024-09-11T22:21:18.186233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:41.737242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.199693+0000","last_change":"2024-09-11T22:21:18.199693+0000","last_active":"2024-09-11T22:21:18.199693+0000","last_peered":"2024-09-11T22:21:18.199693+0000","last_clean":"2024-09-11T22:21:18.199693+0000","last_became_active":"2024-09-11T22:21:18.190954+0000","last_became_peered":"2024-09-11T22:21:18.190954+0000","last_unstale":"2024-09-11T22:21:18.199693+0000","last_undegraded":"2024-09-11T22:21:18.199693+0000","last_fullsized":"2024-09-11T22:21:18.199693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:11.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.181962+0000","last_change":"2024-09-11T22:20:16.115602+0000","last_active":"2024-09-11T22:21:18.181962+0000","last_peered":"2024-09-11T22:21:18.181962+0000","last_clean":"2024-09-11T22:21:18.181962+0000","last_became_active":"2024-09-11T22:20:16.115280+0000","last_became_peered":"2024-09-11T22:20:16.115280+0000","last_unstale":"2024-09-11T22:21:18.181962+0000","last_undegraded":"2024-09-11T22:21:18.181962+0000","last_fullsized":"2024-09-11T22:21:18.181962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:41.942137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184216+0000","last_change":"2024-09-11T22:20:18.184125+0000","last_active":"2024-09-11T22:21:18.184216+0000","last_peered":"2024-09-11T22:21:18.184216+0000","last_clean":"2024-09-11T22:21:18.184216+0000","last_became_active":"2024-09-11T22:20:18.183785+0000","last_became_peered":"2024-09-11T22:20:18.183785+0000","last_unstale":"2024-09-11T22:21:18.184216+0000","last_undegraded":"2024-09-11T22:21:18.184216+0000","last_fullsized":"2024-09-11T22:21:18.184216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:33.230175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186888+0000","last_change":"2024-09-11T22:20:16.125852+0000","last_active":"2024-09-11T22:21:18.186888+0000","last_peered":"2024-09-11T22:21:18.186888+0000","last_clean":"2024-09-11T22:21:18.186888+0000","last_became_active":"2024-09-11T22:20:16.125250+0000","last_became_peered":"2024-09-11T22:20:16.125250+0000","last_unstale":"2024-09-11T22:21:18.186888+0000","last_undegraded":"2024-09-11T22:21:18.186888+0000","last_fullsized":"2024-09-11T22:21:18.186888+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:34.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185013+0000","last_change":"2024-09-11T22:20:18.185147+0000","last_active":"2024-09-11T22:21:18.185013+0000","last_peered":"2024-09-11T22:21:18.185013+0000","last_clean":"2024-09-11T22:21:18.185013+0000","last_became_active":"2024-09-11T22:20:18.184840+0000","last_became_peered":"2024-09-11T22:20:18.184840+0000","last_unstale":"2024-09-11T22:21:18.185013+0000","last_undegraded":"2024-09-11T22:21:18.185013+0000","last_fullsized":"2024-09-11T22:21:18.185013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:59.703349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":126,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:19.339554+0000","last_change":"2024-09-11T22:20:16.122173+0000","last_active":"2024-09-11T22:21:19.339554+0000","last_peered":"2024-09-11T22:21:19.339554+0000","last_clean":"2024-09-11T22:21:19.339554+0000","last_became_active":"2024-09-11T22:20:16.121847+0000","last_became_peered":"2024-09-11T22:20:16.121847+0000","last_unstale":"2024-09-11T22:21:19.339554+0000","last_undegraded":"2024-09-11T22:21:19.339554+0000","last_fullsized":"2024-09-11T22:21:19.339554+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:41.284557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.189560+0000","last_change":"2024-09-11T22:20:18.188030+0000","last_active":"2024-09-11T22:21:18.189560+0000","last_peered":"2024-09-11T22:21:18.189560+0000","last_clean":"2024-09-11T22:21:18.189560+0000","last_became_active":"2024-09-11T22:20:18.187503+0000","last_became_peered":"2024-09-11T22:20:18.187503+0000","last_unstale":"2024-09-11T22:21:18.189560+0000","last_undegraded":"2024-09-11T22:21:18.189560+0000","last_fullsized":"2024-09-11T22:21:18.189560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:45.494149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184153+0000","last_change":"2024-09-11T22:20:16.128828+0000","last_active":"2024-09-11T22:21:18.184153+0000","last_peered":"2024-09-11T22:21:18.184153+0000","last_clean":"2024-09-11T22:21:18.184153+0000","last_became_active":"2024-09-11T22:20:16.128454+0000","last_became_peered":"2024-09-11T22:20:16.128454+0000","last_unstale":"2024-09-11T22:21:18.184153+0000","last_undegraded":"2024-09-11T22:21:18.184153+0000","last_fullsized":"2024-09-11T22:21:18.184153+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:33.927564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186436+0000","last_change":"2024-09-11T22:20:18.193795+0000","last_active":"2024-09-11T22:21:18.186436+0000","last_peered":"2024-09-11T22:21:18.186436+0000","last_clean":"2024-09-11T22:21:18.186436+0000","last_became_active":"2024-09-11T22:20:18.193489+0000","last_became_peered":"2024-09-11T22:20:18.193489+0000","last_unstale":"2024-09-11T22:21:18.186436+0000","last_undegraded":"2024-09-11T22:21:18.186436+0000","last_fullsized":"2024-09-11T22:21:18.186436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:14.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186332+0000","last_change":"2024-09-11T22:20:16.187938+0000","last_active":"2024-09-11T22:21:18.186332+0000","last_peered":"2024-09-11T22:21:18.186332+0000","last_clean":"2024-09-11T22:21:18.186332+0000","last_became_active":"2024-09-11T22:20:16.187545+0000","last_became_peered":"2024-09-11T22:20:16.187545+0000","last_unstale":"2024-09-11T22:21:18.186332+0000","last_undegraded":"2024-09-11T22:21:18.186332+0000","last_fullsized":"2024-09-11T22:21:18.186332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:41.674356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.191937+0000","last_change":"2024-09-11T22:21:18.191937+0000","last_active":"2024-09-11T22:21:18.191937+0000","last_peered":"2024-09-11T22:21:18.191937+0000","last_clean":"2024-09-11T22:21:18.191937+0000","last_became_active":"2024-09-11T22:21:18.191218+0000","last_became_peered":"2024-09-11T22:21:18.191218+0000","last_unstale":"2024-09-11T22:21:18.191937+0000","last_undegraded":"2024-09-11T22:21:18.191937+0000","last_fullsized":"2024-09-11T22:21:18.191937+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:41.654630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":161,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:19.339351+0000","last_change":"2024-09-11T22:20:16.128082+0000","last_active":"2024-09-11T22:21:19.339351+0000","last_peered":"2024-09-11T22:21:19.339351+0000","last_clean":"2024-09-11T22:21:19.339351+0000","last_became_active":"2024-09-11T22:20:16.127731+0000","last_became_peered":"2024-09-11T22:20:16.127731+0000","last_unstale":"2024-09-11T22:21:19.339351+0000","last_undegraded":"2024-09-11T22:21:19.339351+0000","last_fullsized":"2024-09-11T22:21:19.339351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:33.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":112,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186621+0000","last_change":"2024-09-11T22:20:18.189219+0000","last_active":"2024-09-11T22:21:18.186621+0000","last_peered":"2024-09-11T22:21:18.186621+0000","last_clean":"2024-09-11T22:21:18.186621+0000","last_became_active":"2024-09-11T22:20:18.188687+0000","last_became_peered":"2024-09-11T22:20:18.188687+0000","last_unstale":"2024-09-11T22:21:18.186621+0000","last_undegraded":"2024-09-11T22:21:18.186621+0000","last_fullsized":"2024-09-11T22:21:18.186621+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:24.171126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185606+0000","last_change":"2024-09-11T22:20:16.129724+0000","last_active":"2024-09-11T22:21:18.185606+0000","last_peered":"2024-09-11T22:21:18.185606+0000","last_clean":"2024-09-11T22:21:18.185606+0000","last_became_active":"2024-09-11T22:20:16.129351+0000","last_became_peered":"2024-09-11T22:20:16.129351+0000","last_unstale":"2024-09-11T22:21:18.185606+0000","last_undegraded":"2024-09-11T22:21:18.185606+0000","last_fullsized":"2024-09-11T22:21:18.185606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:09.642520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.182030+0000","last_change":"2024-09-11T22:20:18.187031+0000","last_active":"2024-09-11T22:21:18.182030+0000","last_peered":"2024-09-11T22:21:18.182030+0000","last_clean":"2024-09-11T22:21:18.182030+0000","last_became_active":"2024-09-11T22:20:18.186698+0000","last_became_peered":"2024-09-11T22:20:18.186698+0000","last_unstale":"2024-09-11T22:21:18.182030+0000","last_undegraded":"2024-09-11T22:21:18.182030+0000","last_fullsized":"2024-09-11T22:21:18.182030+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:04.810981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186471+0000","last_change":"2024-09-11T22:20:16.124925+0000","last_active":"2024-09-11T22:21:18.186471+0000","last_peered":"2024-09-11T22:21:18.186471+0000","last_clean":"2024-09-11T22:21:18.186471+0000","last_became_active":"2024-09-11T22:20:16.124580+0000","last_became_peered":"2024-09-11T22:20:16.124580+0000","last_unstale":"2024-09-11T22:21:18.186471+0000","last_undegraded":"2024-09-11T22:21:18.186471+0000","last_fullsized":"2024-09-11T22:21:18.186471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:56.417577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187783+0000","last_change":"2024-09-11T22:21:18.187783+0000","last_active":"2024-09-11T22:21:18.187783+0000","last_peered":"2024-09-11T22:21:18.187783+0000","last_clean":"2024-09-11T22:21:18.187783+0000","last_became_active":"2024-09-11T22:21:18.187358+0000","last_became_peered":"2024-09-11T22:21:18.187358+0000","last_unstale":"2024-09-11T22:21:18.187783+0000","last_undegraded":"2024-09-11T22:21:18.187783+0000","last_fullsized":"2024-09-11T22:21:18.187783+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:41.251531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'5","reported_seq":79,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.347132+0000","last_change":"2024-09-11T22:20:16.118934+0000","last_active":"2024-09-11T22:21:18.347132+0000","last_peered":"2024-09-11T22:21:18.347132+0000","last_clean":"2024-09-11T22:21:18.347132+0000","last_became_active":"2024-09-11T22:20:16.118614+0000","last_became_peered":"2024-09-11T22:20:16.118614+0000","last_unstale":"2024-09-11T22:21:18.347132+0000","last_undegraded":"2024-09-11T22:21:18.347132+0000","last_fullsized":"2024-09-11T22:21:18.347132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:05.791161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184096+0000","last_change":"2024-09-11T22:20:18.197692+0000","last_active":"2024-09-11T22:21:18.184096+0000","last_peered":"2024-09-11T22:21:18.184096+0000","last_clean":"2024-09-11T22:21:18.184096+0000","last_became_active":"2024-09-11T22:20:18.197336+0000","last_became_peered":"2024-09-11T22:20:18.197336+0000","last_unstale":"2024-09-11T22:21:18.184096+0000","last_undegraded":"2024-09-11T22:21:18.184096+0000","last_fullsized":"2024-09-11T22:21:18.184096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:55.172568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186091+0000","last_change":"2024-09-11T22:20:16.186769+0000","last_active":"2024-09-11T22:21:18.186091+0000","last_peered":"2024-09-11T22:21:18.186091+0000","last_clean":"2024-09-11T22:21:18.186091+0000","last_became_active":"2024-09-11T22:20:16.186367+0000","last_became_peered":"2024-09-11T22:20:16.186367+0000","last_unstale":"2024-09-11T22:21:18.186091+0000","last_undegraded":"2024-09-11T22:21:18.186091+0000","last_fullsized":"2024-09-11T22:21:18.186091+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.279387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185715+0000","last_change":"2024-09-11T22:20:18.195570+0000","last_active":"2024-09-11T22:21:18.185715+0000","last_peered":"2024-09-11T22:21:18.185715+0000","last_clean":"2024-09-11T22:21:18.185715+0000","last_became_active":"2024-09-11T22:20:18.195226+0000","last_became_peered":"2024-09-11T22:20:18.195226+0000","last_unstale":"2024-09-11T22:21:18.185715+0000","last_undegraded":"2024-09-11T22:21:18.185715+0000","last_fullsized":"2024-09-11T22:21:18.185715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:44.108090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.195020+0000","last_change":"2024-09-11T22:21:18.194553+0000","last_active":"2024-09-11T22:21:18.195020+0000","last_peered":"2024-09-11T22:21:18.195020+0000","last_clean":"2024-09-11T22:21:18.195020+0000","last_became_active":"2024-09-11T22:21:18.188635+0000","last_became_peered":"2024-09-11T22:21:18.188635+0000","last_unstale":"2024-09-11T22:21:18.195020+0000","last_undegraded":"2024-09-11T22:21:18.195020+0000","last_fullsized":"2024-09-11T22:21:18.195020+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:34.165195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183408+0000","last_change":"2024-09-11T22:20:18.194112+0000","last_active":"2024-09-11T22:21:18.183408+0000","last_peered":"2024-09-11T22:21:18.183408+0000","last_clean":"2024-09-11T22:21:18.183408+0000","last_became_active":"2024-09-11T22:20:18.193812+0000","last_became_peered":"2024-09-11T22:20:18.193812+0000","last_unstale":"2024-09-11T22:21:18.183408+0000","last_undegraded":"2024-09-11T22:21:18.183408+0000","last_fullsized":"2024-09-11T22:21:18.183408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:35.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.198325+0000","last_change":"2024-09-11T22:21:18.198325+0000","last_active":"2024-09-11T22:21:18.198325+0000","last_peered":"2024-09-11T22:21:18.198325+0000","last_clean":"2024-09-11T22:21:18.198325+0000","last_became_active":"2024-09-11T22:21:18.193651+0000","last_became_peered":"2024-09-11T22:21:18.193651+0000","last_unstale":"2024-09-11T22:21:18.198325+0000","last_undegraded":"2024-09-11T22:21:18.198325+0000","last_fullsized":"2024-09-11T22:21:18.198325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:42.552492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185226+0000","last_change":"2024-09-11T22:20:18.191960+0000","last_active":"2024-09-11T22:21:18.185226+0000","last_peered":"2024-09-11T22:21:18.185226+0000","last_clean":"2024-09-11T22:21:18.185226+0000","last_became_active":"2024-09-11T22:20:18.191635+0000","last_became_peered":"2024-09-11T22:20:18.191635+0000","last_unstale":"2024-09-11T22:21:18.185226+0000","last_undegraded":"2024-09-11T22:21:18.185226+0000","last_fullsized":"2024-09-11T22:21:18.185226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:19:18.919247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185965+0000","last_change":"2024-09-11T22:20:16.123608+0000","last_active":"2024-09-11T22:21:18.185965+0000","last_peered":"2024-09-11T22:21:18.185965+0000","last_clean":"2024-09-11T22:21:18.185965+0000","last_became_active":"2024-09-11T22:20:16.123193+0000","last_became_peered":"2024-09-11T22:20:16.123193+0000","last_unstale":"2024-09-11T22:21:18.185965+0000","last_undegraded":"2024-09-11T22:21:18.185965+0000","last_fullsized":"2024-09-11T22:21:18.185965+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:07.362115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184596+0000","last_change":"2024-09-11T22:20:18.199957+0000","last_active":"2024-09-11T22:21:18.184596+0000","last_peered":"2024-09-11T22:21:18.184596+0000","last_clean":"2024-09-11T22:21:18.184596+0000","last_became_active":"2024-09-11T22:20:18.199637+0000","last_became_peered":"2024-09-11T22:20:18.199637+0000","last_unstale":"2024-09-11T22:21:18.184596+0000","last_undegraded":"2024-09-11T22:21:18.184596+0000","last_fullsized":"2024-09-11T22:21:18.184596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:14.949002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:19.197893+0000","last_change":"2024-09-11T22:21:18.198896+0000","last_active":"2024-09-11T22:21:19.197893+0000","last_peered":"2024-09-11T22:21:19.197893+0000","last_clean":"2024-09-11T22:21:19.197893+0000","last_became_active":"2024-09-11T22:21:18.189489+0000","last_became_peered":"2024-09-11T22:21:18.189489+0000","last_unstale":"2024-09-11T22:21:19.197893+0000","last_undegraded":"2024-09-11T22:21:19.197893+0000","last_fullsized":"2024-09-11T22:21:19.197893+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:19.908801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184893+0000","last_change":"2024-09-11T22:20:18.196691+0000","last_active":"2024-09-11T22:21:18.184893+0000","last_peered":"2024-09-11T22:21:18.184893+0000","last_clean":"2024-09-11T22:21:18.184893+0000","last_became_active":"2024-09-11T22:20:18.196364+0000","last_became_peered":"2024-09-11T22:20:18.196364+0000","last_unstale":"2024-09-11T22:21:18.184893+0000","last_undegraded":"2024-09-11T22:21:18.184893+0000","last_fullsized":"2024-09-11T22:21:18.184893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:50.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":345,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184308+0000","last_change":"2024-09-11T22:19:02.590079+0000","last_active":"2024-09-11T22:21:18.184308+0000","last_peered":"2024-09-11T22:21:18.184308+0000","last_clean":"2024-09-11T22:21:18.184308+0000","last_became_active":"2024-09-11T22:19:02.589644+0000","last_became_peered":"2024-09-11T22:19:02.589644+0000","last_unstale":"2024-09-11T22:21:18.184308+0000","last_undegraded":"2024-09-11T22:21:18.184308+0000","last_fullsized":"2024-09-11T22:21:18.184308+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_clean_scrub_stamp":"2024-09-11T22:15:33.244685+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:45.506190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183638+0000","last_change":"2024-09-11T22:20:16.116661+0000","last_active":"2024-09-11T22:21:18.183638+0000","last_peered":"2024-09-11T22:21:18.183638+0000","last_clean":"2024-09-11T22:21:18.183638+0000","last_became_active":"2024-09-11T22:20:16.116196+0000","last_became_peered":"2024-09-11T22:20:16.116196+0000","last_unstale":"2024-09-11T22:21:18.183638+0000","last_undegraded":"2024-09-11T22:21:18.183638+0000","last_fullsized":"2024-09-11T22:21:18.183638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:08.875170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184311+0000","last_change":"2024-09-11T22:20:18.193176+0000","last_active":"2024-09-11T22:21:18.184311+0000","last_peered":"2024-09-11T22:21:18.184311+0000","last_clean":"2024-09-11T22:21:18.184311+0000","last_became_active":"2024-09-11T22:20:18.192312+0000","last_became_peered":"2024-09-11T22:20:18.192312+0000","last_unstale":"2024-09-11T22:21:18.184311+0000","last_undegraded":"2024-09-11T22:21:18.184311+0000","last_fullsized":"2024-09-11T22:21:18.184311+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:08.738569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.197381+0000","last_change":"2024-09-11T22:21:18.196985+0000","last_active":"2024-09-11T22:21:18.197381+0000","last_peered":"2024-09-11T22:21:18.197381+0000","last_clean":"2024-09-11T22:21:18.197381+0000","last_became_active":"2024-09-11T22:21:18.189487+0000","last_became_peered":"2024-09-11T22:21:18.189487+0000","last_unstale":"2024-09-11T22:21:18.197381+0000","last_undegraded":"2024-09-11T22:21:18.197381+0000","last_fullsized":"2024-09-11T22:21:18.197381+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:20.715696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.181955+0000","last_change":"2024-09-11T22:20:18.191118+0000","last_active":"2024-09-11T22:21:18.181955+0000","last_peered":"2024-09-11T22:21:18.181955+0000","last_clean":"2024-09-11T22:21:18.181955+0000","last_became_active":"2024-09-11T22:20:18.190760+0000","last_became_peered":"2024-09-11T22:20:18.190760+0000","last_unstale":"2024-09-11T22:21:18.181955+0000","last_undegraded":"2024-09-11T22:21:18.181955+0000","last_fullsized":"2024-09-11T22:21:18.181955+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:59.220067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":80,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:20.349901+0000","last_change":"2024-09-11T22:20:16.186407+0000","last_active":"2024-09-11T22:21:20.349901+0000","last_peered":"2024-09-11T22:21:20.349901+0000","last_clean":"2024-09-11T22:21:20.349901+0000","last_became_active":"2024-09-11T22:20:16.186105+0000","last_became_peered":"2024-09-11T22:20:16.186105+0000","last_unstale":"2024-09-11T22:21:20.349901+0000","last_undegraded":"2024-09-11T22:21:20.349901+0000","last_fullsized":"2024-09-11T22:21:20.349901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:43.942072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186762+0000","last_change":"2024-09-11T22:20:18.191970+0000","last_active":"2024-09-11T22:21:18.186762+0000","last_peered":"2024-09-11T22:21:18.186762+0000","last_clean":"2024-09-11T22:21:18.186762+0000","last_became_active":"2024-09-11T22:20:18.191623+0000","last_became_peered":"2024-09-11T22:20:18.191623+0000","last_unstale":"2024-09-11T22:21:18.186762+0000","last_undegraded":"2024-09-11T22:21:18.186762+0000","last_fullsized":"2024-09-11T22:21:18.186762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:33.213979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'6","reported_seq":82,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:20.351585+0000","last_change":"2024-09-11T22:20:16.119936+0000","last_active":"2024-09-11T22:21:20.351585+0000","last_peered":"2024-09-11T22:21:20.351585+0000","last_clean":"2024-09-11T22:21:20.351585+0000","last_became_active":"2024-09-11T22:20:16.119453+0000","last_became_peered":"2024-09-11T22:20:16.119453+0000","last_unstale":"2024-09-11T22:21:20.351585+0000","last_undegraded":"2024-09-11T22:21:20.351585+0000","last_fullsized":"2024-09-11T22:21:20.351585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:20.717980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185497+0000","last_change":"2024-09-11T22:20:18.192581+0000","last_active":"2024-09-11T22:21:18.185497+0000","last_peered":"2024-09-11T22:21:18.185497+0000","last_clean":"2024-09-11T22:21:18.185497+0000","last_became_active":"2024-09-11T22:20:18.191702+0000","last_became_peered":"2024-09-11T22:20:18.191702+0000","last_unstale":"2024-09-11T22:21:18.185497+0000","last_undegraded":"2024-09-11T22:21:18.185497+0000","last_fullsized":"2024-09-11T22:21:18.185497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:48.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186182+0000","last_change":"2024-09-11T22:20:16.186485+0000","last_active":"2024-09-11T22:21:18.186182+0000","last_peered":"2024-09-11T22:21:18.186182+0000","last_clean":"2024-09-11T22:21:18.186182+0000","last_became_active":"2024-09-11T22:20:16.186171+0000","last_became_peered":"2024-09-11T22:20:16.186171+0000","last_unstale":"2024-09-11T22:21:18.186182+0000","last_undegraded":"2024-09-11T22:21:18.186182+0000","last_fullsized":"2024-09-11T22:21:18.186182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:26.373626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184991+0000","last_change":"2024-09-11T22:20:18.191361+0000","last_active":"2024-09-11T22:21:18.184991+0000","last_peered":"2024-09-11T22:21:18.184991+0000","last_clean":"2024-09-11T22:21:18.184991+0000","last_became_active":"2024-09-11T22:20:18.191036+0000","last_became_peered":"2024-09-11T22:20:18.191036+0000","last_unstale":"2024-09-11T22:21:18.184991+0000","last_undegraded":"2024-09-11T22:21:18.184991+0000","last_fullsized":"2024-09-11T22:21:18.184991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:29.124650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186875+0000","last_change":"2024-09-11T22:20:16.188156+0000","last_active":"2024-09-11T22:21:18.186875+0000","last_peered":"2024-09-11T22:21:18.186875+0000","last_clean":"2024-09-11T22:21:18.186875+0000","last_became_active":"2024-09-11T22:20:16.187809+0000","last_became_peered":"2024-09-11T22:20:16.187809+0000","last_unstale":"2024-09-11T22:21:18.186875+0000","last_undegraded":"2024-09-11T22:21:18.186875+0000","last_fullsized":"2024-09-11T22:21:18.186875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:04.219621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.182314+0000","last_change":"2024-09-11T22:20:18.182380+0000","last_active":"2024-09-11T22:21:18.182314+0000","last_peered":"2024-09-11T22:21:18.182314+0000","last_clean":"2024-09-11T22:21:18.182314+0000","last_became_active":"2024-09-11T22:20:18.182062+0000","last_became_peered":"2024-09-11T22:20:18.182062+0000","last_unstale":"2024-09-11T22:21:18.182314+0000","last_undegraded":"2024-09-11T22:21:18.182314+0000","last_fullsized":"2024-09-11T22:21:18.182314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:15.575846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186199+0000","last_change":"2024-09-11T22:20:16.117335+0000","last_active":"2024-09-11T22:21:18.186199+0000","last_peered":"2024-09-11T22:21:18.186199+0000","last_clean":"2024-09-11T22:21:18.186199+0000","last_became_active":"2024-09-11T22:20:16.116977+0000","last_became_peered":"2024-09-11T22:20:16.116977+0000","last_unstale":"2024-09-11T22:21:18.186199+0000","last_undegraded":"2024-09-11T22:21:18.186199+0000","last_fullsized":"2024-09-11T22:21:18.186199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:51.744202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185562+0000","last_change":"2024-09-11T22:20:18.194069+0000","last_active":"2024-09-11T22:21:18.185562+0000","last_peered":"2024-09-11T22:21:18.185562+0000","last_clean":"2024-09-11T22:21:18.185562+0000","last_became_active":"2024-09-11T22:20:18.193765+0000","last_became_peered":"2024-09-11T22:20:18.193765+0000","last_unstale":"2024-09-11T22:21:18.185562+0000","last_undegraded":"2024-09-11T22:21:18.185562+0000","last_fullsized":"2024-09-11T22:21:18.185562+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:25.047405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183506+0000","last_change":"2024-09-11T22:20:16.460987+0000","last_active":"2024-09-11T22:21:18.183506+0000","last_peered":"2024-09-11T22:21:18.183506+0000","last_clean":"2024-09-11T22:21:18.183506+0000","last_became_active":"2024-09-11T22:20:16.460516+0000","last_became_peered":"2024-09-11T22:20:16.460516+0000","last_unstale":"2024-09-11T22:21:18.183506+0000","last_undegraded":"2024-09-11T22:21:18.183506+0000","last_fullsized":"2024-09-11T22:21:18.183506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:00.987981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.188802+0000","last_change":"2024-09-11T22:21:18.188802+0000","last_active":"2024-09-11T22:21:18.188802+0000","last_peered":"2024-09-11T22:21:18.188802+0000","last_clean":"2024-09-11T22:21:18.188802+0000","last_became_active":"2024-09-11T22:21:18.188441+0000","last_became_peered":"2024-09-11T22:21:18.188441+0000","last_unstale":"2024-09-11T22:21:18.188802+0000","last_undegraded":"2024-09-11T22:21:18.188802+0000","last_fullsized":"2024-09-11T22:21:18.188802+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:02.295441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187676+0000","last_change":"2024-09-11T22:20:16.186498+0000","last_active":"2024-09-11T22:21:18.187676+0000","last_peered":"2024-09-11T22:21:18.187676+0000","last_clean":"2024-09-11T22:21:18.187676+0000","last_became_active":"2024-09-11T22:20:16.186214+0000","last_became_peered":"2024-09-11T22:20:16.186214+0000","last_unstale":"2024-09-11T22:21:18.187676+0000","last_undegraded":"2024-09-11T22:21:18.187676+0000","last_fullsized":"2024-09-11T22:21:18.187676+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:09.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185428+0000","last_change":"2024-09-11T22:20:18.187489+0000","last_active":"2024-09-11T22:21:18.185428+0000","last_peered":"2024-09-11T22:21:18.185428+0000","last_clean":"2024-09-11T22:21:18.185428+0000","last_became_active":"2024-09-11T22:20:18.185649+0000","last_became_peered":"2024-09-11T22:20:18.185649+0000","last_unstale":"2024-09-11T22:21:18.185428+0000","last_undegraded":"2024-09-11T22:21:18.185428+0000","last_fullsized":"2024-09-11T22:21:18.185428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:42.825075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184722+0000","last_change":"2024-09-11T22:20:16.117627+0000","last_active":"2024-09-11T22:21:18.184722+0000","last_peered":"2024-09-11T22:21:18.184722+0000","last_clean":"2024-09-11T22:21:18.184722+0000","last_became_active":"2024-09-11T22:20:16.117243+0000","last_became_peered":"2024-09-11T22:20:16.117243+0000","last_unstale":"2024-09-11T22:21:18.184722+0000","last_undegraded":"2024-09-11T22:21:18.184722+0000","last_fullsized":"2024-09-11T22:21:18.184722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:58.240535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.183220+0000","last_change":"2024-09-11T22:20:18.184326+0000","last_active":"2024-09-11T22:21:18.183220+0000","last_peered":"2024-09-11T22:21:18.183220+0000","last_clean":"2024-09-11T22:21:18.183220+0000","last_became_active":"2024-09-11T22:20:18.183995+0000","last_became_peered":"2024-09-11T22:20:18.183995+0000","last_unstale":"2024-09-11T22:21:18.183220+0000","last_undegraded":"2024-09-11T22:21:18.183220+0000","last_fullsized":"2024-09-11T22:21:18.183220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:33.468000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187931+0000","last_change":"2024-09-11T22:20:16.127180+0000","last_active":"2024-09-11T22:21:18.187931+0000","last_peered":"2024-09-11T22:21:18.187931+0000","last_clean":"2024-09-11T22:21:18.187931+0000","last_became_active":"2024-09-11T22:20:16.126696+0000","last_became_peered":"2024-09-11T22:20:16.126696+0000","last_unstale":"2024-09-11T22:21:18.187931+0000","last_undegraded":"2024-09-11T22:21:18.187931+0000","last_fullsized":"2024-09-11T22:21:18.187931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:02.637225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.189710+0000","last_change":"2024-09-11T22:21:18.188614+0000","last_active":"2024-09-11T22:21:18.189710+0000","last_peered":"2024-09-11T22:21:18.189710+0000","last_clean":"2024-09-11T22:21:18.189710+0000","last_became_active":"2024-09-11T22:21:18.188277+0000","last_became_peered":"2024-09-11T22:21:18.188277+0000","last_unstale":"2024-09-11T22:21:18.189710+0000","last_undegraded":"2024-09-11T22:21:18.189710+0000","last_fullsized":"2024-09-11T22:21:18.189710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:59.995647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186347+0000","last_change":"2024-09-11T22:20:16.128311+0000","last_active":"2024-09-11T22:21:18.186347+0000","last_peered":"2024-09-11T22:21:18.186347+0000","last_clean":"2024-09-11T22:21:18.186347+0000","last_became_active":"2024-09-11T22:20:16.127853+0000","last_became_peered":"2024-09-11T22:20:16.127853+0000","last_unstale":"2024-09-11T22:21:18.186347+0000","last_undegraded":"2024-09-11T22:21:18.186347+0000","last_fullsized":"2024-09-11T22:21:18.186347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:57.562470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.184751+0000","last_change":"2024-09-11T22:20:18.195782+0000","last_active":"2024-09-11T22:21:18.184751+0000","last_peered":"2024-09-11T22:21:18.184751+0000","last_clean":"2024-09-11T22:21:18.184751+0000","last_became_active":"2024-09-11T22:20:18.195365+0000","last_became_peered":"2024-09-11T22:20:18.195365+0000","last_unstale":"2024-09-11T22:21:18.184751+0000","last_undegraded":"2024-09-11T22:21:18.184751+0000","last_fullsized":"2024-09-11T22:21:18.184751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:49.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185795+0000","last_change":"2024-09-11T22:20:18.199344+0000","last_active":"2024-09-11T22:21:18.185795+0000","last_peered":"2024-09-11T22:21:18.185795+0000","last_clean":"2024-09-11T22:21:18.185795+0000","last_became_active":"2024-09-11T22:20:18.199016+0000","last_became_peered":"2024-09-11T22:20:18.199016+0000","last_unstale":"2024-09-11T22:21:18.185795+0000","last_undegraded":"2024-09-11T22:21:18.185795+0000","last_fullsized":"2024-09-11T22:21:18.185795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:38.322831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.187943+0000","last_change":"2024-09-11T22:20:16.122236+0000","last_active":"2024-09-11T22:21:18.187943+0000","last_peered":"2024-09-11T22:21:18.187943+0000","last_clean":"2024-09-11T22:21:18.187943+0000","last_became_active":"2024-09-11T22:20:16.121867+0000","last_became_peered":"2024-09-11T22:20:16.121867+0000","last_unstale":"2024-09-11T22:21:18.187943+0000","last_undegraded":"2024-09-11T22:21:18.187943+0000","last_fullsized":"2024-09-11T22:21:18.187943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:56.782445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.188621+0000","last_change":"2024-09-11T22:20:18.187338+0000","last_active":"2024-09-11T22:21:18.188621+0000","last_peered":"2024-09-11T22:21:18.188621+0000","last_clean":"2024-09-11T22:21:18.188621+0000","last_became_active":"2024-09-11T22:20:18.186985+0000","last_became_peered":"2024-09-11T22:20:18.186985+0000","last_unstale":"2024-09-11T22:21:18.188621+0000","last_undegraded":"2024-09-11T22:21:18.188621+0000","last_fullsized":"2024-09-11T22:21:18.188621+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:58.170680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186747+0000","last_change":"2024-09-11T22:20:16.128579+0000","last_active":"2024-09-11T22:21:18.186747+0000","last_peered":"2024-09-11T22:21:18.186747+0000","last_clean":"2024-09-11T22:21:18.186747+0000","last_became_active":"2024-09-11T22:20:16.127864+0000","last_became_peered":"2024-09-11T22:20:16.127864+0000","last_unstale":"2024-09-11T22:21:18.186747+0000","last_undegraded":"2024-09-11T22:21:18.186747+0000","last_fullsized":"2024-09-11T22:21:18.186747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:11:32.005487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.186436+0000","last_change":"2024-09-11T22:20:18.185205+0000","last_active":"2024-09-11T22:21:18.186436+0000","last_peered":"2024-09-11T22:21:18.186436+0000","last_clean":"2024-09-11T22:21:18.186436+0000","last_became_active":"2024-09-11T22:20:18.184402+0000","last_became_peered":"2024-09-11T22:20:18.184402+0000","last_unstale":"2024-09-11T22:21:18.186436+0000","last_undegraded":"2024-09-11T22:21:18.186436+0000","last_fullsized":"2024-09-11T22:21:18.186436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:17:48.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185468+0000","last_change":"2024-09-11T22:20:16.186372+0000","last_active":"2024-09-11T22:21:18.185468+0000","last_peered":"2024-09-11T22:21:18.185468+0000","last_clean":"2024-09-11T22:21:18.185468+0000","last_became_active":"2024-09-11T22:20:16.185990+0000","last_became_peered":"2024-09-11T22:20:16.185990+0000","last_unstale":"2024-09-11T22:21:18.185468+0000","last_undegraded":"2024-09-11T22:21:18.185468+0000","last_fullsized":"2024-09-11T22:21:18.185468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:23.296379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:18.185002+0000","last_change":"2024-09-11T22:20:18.190670+0000","last_active":"2024-09-11T22:21:18.185002+0000","last_peered":"2024-09-11T22:21:18.185002+0000","last_clean":"2024-09-11T22:21:18.185002+0000","last_became_active":"2024-09-11T22:20:18.190339+0000","last_became_peered":"2024-09-11T22:20:18.190339+0000","last_unstale":"2024-09-11T22:21:18.185002+0000","last_undegraded":"2024-09-11T22:21:18.185002+0000","last_fullsized":"2024-09-11T22:21:18.185002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.085305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":90,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T22:21:22.353540+0000","last_change":"2024-09-11T22:20:16.117646+0000","last_active":"2024-09-11T22:21:22.353540+0000","last_peered":"2024-09-11T22:21:22.353540+0000","last_clean":"2024-09-11T22:21:22.353540+0000","last_became_active":"2024-09-11T22:20:16.117111+0000","last_became_peered":"2024-09-11T22:20:16.117111+0000","last_unstale":"2024-09-11T22:21:22.353540+0000","last_undegraded":"2024-09-11T22:21:22.353540+0000","last_fullsized":"2024-09-11T22:21:22.353540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:36.470158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66579,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":497,"num_read_kb":346,"num_write":241,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1192,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":199737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":207,"ondisk_log_size":207,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":249094,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579931,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":901120,"data_stored":434362,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1052672,"data_stored":1085133,"data_compressed":42058,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":921600,"data_stored":434270,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201322,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":434626,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":970752,"data_stored":1027168,"data_compressed":42058,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593780,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":905216,"data_stored":434380,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822713,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":983040,"data_stored":1026453,"data_compressed":42058,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986238,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":913408,"data_stored":435018,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215172,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36252,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708900,"statfs":{"total":95995035648,"available":95957913600,"internally_reserved":0,"allocated":921600,"data_stored":433812,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":978944,"data_stored":493714,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":966656,"data_stored":495057,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":925696,"data_stored":435326,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34650,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:21:27.141 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T22:21:27.141 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T22:21:27.141 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T22:21:27.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T22:21:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:27 smithi005 ceph-mon[30943]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:21:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:27 smithi005 ceph-mon[30943]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4 op/s; 128 B/s, 1 objects/s recovering 2024-09-11T22:21:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.931+0000 7f61797f5640 1 -- 172.21.15.5:0/938068809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61741082a0 msgr2=0x7f617410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.931+0000 7f61797f5640 1 --2- 172.21.15.5:0/938068809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61741082a0 0x7f617410ec40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f616c009cd0 tx=0x7f616c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.931+0000 7f61797f5640 1 -- 172.21.15.5:0/938068809 shutdown_connections 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.931+0000 7f61797f5640 1 --2- 172.21.15.5:0/938068809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61741082a0 0x7f617410ec40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.931+0000 7f61797f5640 1 --2- 172.21.15.5:0/938068809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61741036e0 0x7f6174107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.931+0000 7f61797f5640 1 --2- 172.21.15.5:0/938068809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6174102d30 0x7f6174103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.931+0000 7f61797f5640 1 -- 172.21.15.5:0/938068809 >> 172.21.15.5:0/938068809 conn(0x7f61740fc7d0 msgr2=0x7f61740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.931+0000 7f61797f5640 1 -- 172.21.15.5:0/938068809 shutdown_connections 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.931+0000 7f61797f5640 1 -- 172.21.15.5:0/938068809 wait complete. 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.931+0000 7f61797f5640 1 Processor -- start 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f61797f5640 1 -- start start 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f61797f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6174102d30 0x7f6174198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f61797f5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61741036e0 0x7f6174198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f61797f5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61741082a0 0x7f617419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f61797f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61741120a0 con 0x7f6174102d30 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f61797f5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6174111f20 con 0x7f61741082a0 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f61797f5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6174112220 con 0x7f61741036e0 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f6172ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6174102d30 0x7f6174198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f6172ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6174102d30 0x7f6174198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39412/0 (socket says 172.21.15.5:39412) 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f6172ffd640 1 -- 172.21.15.5:0/4041446685 learned_addr learned my addr 172.21.15.5:0/4041446685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f61737fe640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61741082a0 0x7f617419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f61727fc640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61741036e0 0x7f6174198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f6172ffd640 1 -- 172.21.15.5:0/4041446685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61741036e0 msgr2=0x7f6174198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f6172ffd640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61741036e0 0x7f6174198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f6172ffd640 1 -- 172.21.15.5:0/4041446685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61741082a0 msgr2=0x7f617419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f6172ffd640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61741082a0 0x7f617419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.932+0000 7f6172ffd640 1 -- 172.21.15.5:0/4041446685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617419d520 con 0x7f6174102d30 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.933+0000 7f6172ffd640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6174102d30 0x7f6174198570 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f616400d770 tx=0x7f616400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.933+0000 7f615bfff640 1 -- 172.21.15.5:0/4041446685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164017070 con 0x7f6174102d30 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.933+0000 7f61797f5640 1 -- 172.21.15.5:0/4041446685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617419d810 con 0x7f6174102d30 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.933+0000 7f615bfff640 1 -- 172.21.15.5:0/4041446685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6164005500 con 0x7f6174102d30 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.933+0000 7f615bfff640 1 -- 172.21.15.5:0/4041446685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6164004e00 con 0x7f6174102d30 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.933+0000 7f61797f5640 1 -- 172.21.15.5:0/4041446685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61741a50f0 con 0x7f6174102d30 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.934+0000 7f61797f5640 1 -- 172.21.15.5:0/4041446685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6174103110 con 0x7f6174102d30 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.935+0000 7f615bfff640 1 -- 172.21.15.5:0/4041446685 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6164004fa0 con 0x7f6174102d30 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.936+0000 7f615bfff640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f615c0a2c10 0x7f615c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.936+0000 7f61727fc640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f615c0a2c10 0x7f615c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.936+0000 7f615bfff640 1 -- 172.21.15.5:0/4041446685 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f61640cecd0 con 0x7f6174102d30 2024-09-11T22:21:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.936+0000 7f61727fc640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f615c0a2c10 0x7f615c0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f61740fede0 tx=0x7f6168005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:27.940+0000 7f615bfff640 1 -- 172.21.15.5:0/4041446685 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f616409ba50 con 0x7f6174102d30 2024-09-11T22:21:28.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:27 smithi172 ceph-mon[35461]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:21:28.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:27 smithi172 ceph-mon[35461]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4 op/s; 128 B/s, 1 objects/s recovering 2024-09-11T22:21:28.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:27 smithi159 ceph-mon[35534]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:21:28.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:27 smithi159 ceph-mon[35534]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4 op/s; 128 B/s, 1 objects/s recovering 2024-09-11T22:21:28.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.154+0000 7f61797f5640 1 -- 172.21.15.5:0/4041446685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f617410a2b0 con 0x7f6174102d30 2024-09-11T22:21:28.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.154+0000 7f615bfff640 1 -- 172.21.15.5:0/4041446685 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f616409b350 con 0x7f6174102d30 2024-09-11T22:21:28.155 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:28.155 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T22:21:28.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 -- 172.21.15.5:0/4041446685 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f615c0a2c10 msgr2=0x7f615c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:28.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f615c0a2c10 0x7f615c0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f61740fede0 tx=0x7f6168005f90 comp rx=0 tx=0).stop 2024-09-11T22:21:28.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 -- 172.21.15.5:0/4041446685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6174102d30 msgr2=0x7f6174198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:28.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6174102d30 0x7f6174198570 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f616400d770 tx=0x7f616400dc30 comp rx=0 tx=0).stop 2024-09-11T22:21:28.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 -- 172.21.15.5:0/4041446685 shutdown_connections 2024-09-11T22:21:28.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f615c0a2c10 0x7f615c0a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:28.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61741082a0 0x7f617419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:28.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61741036e0 0x7f6174198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:28.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 --2- 172.21.15.5:0/4041446685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6174102d30 0x7f6174198570 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:28.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 -- 172.21.15.5:0/4041446685 >> 172.21.15.5:0/4041446685 conn(0x7f61740fc7d0 msgr2=0x7f6174105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:28.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 -- 172.21.15.5:0/4041446685 shutdown_connections 2024-09-11T22:21:28.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:28.157+0000 7f61797f5640 1 -- 172.21.15.5:0/4041446685 wait complete. 2024-09-11T22:21:28.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:28 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4041446685' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T22:21:28.856 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T22:21:28.857 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:21:29.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:28 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4041446685' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T22:21:29.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:28 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4041446685' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T22:21:29.115 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:29.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.597+0000 7fe7406e1640 1 -- 172.21.15.5:0/216589293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe738101de0 msgr2=0x7fe7381021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:29.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.597+0000 7fe7406e1640 1 --2- 172.21.15.5:0/216589293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe738101de0 0x7fe7381021c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe734009900 tx=0x7fe73402a410 comp rx=0 tx=0).stop 2024-09-11T22:21:29.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.597+0000 7fe7406e1640 1 -- 172.21.15.5:0/216589293 shutdown_connections 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.597+0000 7fe7406e1640 1 --2- 172.21.15.5:0/216589293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738106f10 0x7fe73810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.597+0000 7fe7406e1640 1 --2- 172.21.15.5:0/216589293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe738102790 0x7fe7381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.597+0000 7fe7406e1640 1 --2- 172.21.15.5:0/216589293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe738101de0 0x7fe7381021c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.597+0000 7fe7406e1640 1 -- 172.21.15.5:0/216589293 >> 172.21.15.5:0/216589293 conn(0x7fe7380fd8c0 msgr2=0x7fe7380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.597+0000 7fe7406e1640 1 -- 172.21.15.5:0/216589293 shutdown_connections 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.597+0000 7fe7406e1640 1 -- 172.21.15.5:0/216589293 wait complete. 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe7406e1640 1 Processor -- start 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe7406e1640 1 -- start start 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe7406e1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe738101de0 0x7fe7380744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe7406e1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738102790 0x7fe738074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe7406e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe738106f10 0x7fe738074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe7406e1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe738111f90 con 0x7fe738106f10 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe7406e1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe738111e10 con 0x7fe738101de0 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe7406e1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe738112110 con 0x7fe738102790 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73ec7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe738106f10 0x7fe738074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73ec7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe738106f10 0x7fe738074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39422/0 (socket says 172.21.15.5:39422) 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73ec7e640 1 -- 172.21.15.5:0/2120913963 learned_addr learned my addr 172.21.15.5:0/2120913963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73dc7c640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738102790 0x7fe738074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73e47d640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe738101de0 0x7fe7380744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73ec7e640 1 -- 172.21.15.5:0/2120913963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738102790 msgr2=0x7fe738074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73ec7e640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738102790 0x7fe738074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73ec7e640 1 -- 172.21.15.5:0/2120913963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe738101de0 msgr2=0x7fe7380744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73ec7e640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe738101de0 0x7fe7380744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73ec7e640 1 -- 172.21.15.5:0/2120913963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7381a6950 con 0x7fe738106f10 2024-09-11T22:21:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.598+0000 7fe73e47d640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe738101de0 0x7fe7380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.599+0000 7fe73ec7e640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe738106f10 0x7fe738074f70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe73000eaa0 tx=0x7fe73000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.599+0000 7fe72b7fe640 1 -- 172.21.15.5:0/2120913963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73000cdb0 con 0x7fe738106f10 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.599+0000 7fe72b7fe640 1 -- 172.21.15.5:0/2120913963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe730005020 con 0x7fe738106f10 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.599+0000 7fe7406e1640 1 -- 172.21.15.5:0/2120913963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7381a6c40 con 0x7fe738106f10 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.599+0000 7fe72b7fe640 1 -- 172.21.15.5:0/2120913963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7300116b0 con 0x7fe738106f10 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.599+0000 7fe7406e1640 1 -- 172.21.15.5:0/2120913963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7381a7180 con 0x7fe738106f10 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.600+0000 7fe7406e1640 1 -- 172.21.15.5:0/2120913963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7381021c0 con 0x7fe738106f10 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.601+0000 7fe72b7fe640 1 -- 172.21.15.5:0/2120913963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe730005560 con 0x7fe738106f10 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.602+0000 7fe72b7fe640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe7240a2c90 0x7fe7240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.602+0000 7fe73e47d640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe7240a2c90 0x7fe7240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.602+0000 7fe72b7fe640 1 -- 172.21.15.5:0/2120913963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fe7300cf870 con 0x7fe738106f10 2024-09-11T22:21:29.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.602+0000 7fe73e47d640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe7240a2c90 0x7fe7240a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe734000c00 tx=0x7fe734032c70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:29.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.605+0000 7fe72b7fe640 1 -- 172.21.15.5:0/2120913963 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe730016030 con 0x7fe738106f10 2024-09-11T22:21:29.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.818+0000 7fe7406e1640 1 -- 172.21.15.5:0/2120913963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe73810a0f0 con 0x7fe738106f10 2024-09-11T22:21:29.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:29 smithi005 ceph-mon[30943]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4 op/s; 115 B/s, 1 objects/s recovering 2024-09-11T22:21:29.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.819+0000 7fe72b7fe640 1 -- 172.21.15.5:0/2120913963 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7fe73009c670 con 0x7fe738106f10 2024-09-11T22:21:29.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:29.853 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-11T22:20:30:263672+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:20:30.263652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:21:29.854 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-11T22:21:29.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 -- 172.21.15.5:0/2120913963 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe7240a2c90 msgr2=0x7fe7240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:29.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe7240a2c90 0x7fe7240a5150 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe734000c00 tx=0x7fe734032c70 comp rx=0 tx=0).stop 2024-09-11T22:21:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 -- 172.21.15.5:0/2120913963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe738106f10 msgr2=0x7fe738074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe738106f10 0x7fe738074f70 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe73000eaa0 tx=0x7fe73000ef60 comp rx=0 tx=0).stop 2024-09-11T22:21:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 -- 172.21.15.5:0/2120913963 shutdown_connections 2024-09-11T22:21:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe7240a2c90 0x7fe7240a5150 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe738106f10 0x7fe738074f70 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe738102790 0x7fe738074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 --2- 172.21.15.5:0/2120913963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe738101de0 0x7fe7380744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 -- 172.21.15.5:0/2120913963 >> 172.21.15.5:0/2120913963 conn(0x7fe7380fd8c0 msgr2=0x7fe7380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 -- 172.21.15.5:0/2120913963 shutdown_connections 2024-09-11T22:21:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:29.821+0000 7fe7406e1640 1 -- 172.21.15.5:0/2120913963 wait complete. 2024-09-11T22:21:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:29 smithi159 ceph-mon[35534]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4 op/s; 115 B/s, 1 objects/s recovering 2024-09-11T22:21:30.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:29 smithi172 ceph-mon[35461]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4 op/s; 115 B/s, 1 objects/s recovering 2024-09-11T22:21:30.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:21:30.807 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:30.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:30 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2120913963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:30 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2120913963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:30 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2120913963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.340+0000 7fc42329f640 1 -- 172.21.15.5:0/2883819320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc41c101420 msgr2=0x7fc41c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.340+0000 7fc42329f640 1 --2- 172.21.15.5:0/2883819320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc41c101420 0x7fc41c10d950 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc414009930 tx=0x7fc41402a420 comp rx=0 tx=0).stop 2024-09-11T22:21:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.340+0000 7fc42329f640 1 -- 172.21.15.5:0/2883819320 shutdown_connections 2024-09-11T22:21:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.340+0000 7fc42329f640 1 --2- 172.21.15.5:0/2883819320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc41c101420 0x7fc41c10d950 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.340+0000 7fc42329f640 1 --2- 172.21.15.5:0/2883819320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c100a80 0x7fc41c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.340+0000 7fc42329f640 1 --2- 172.21.15.5:0/2883819320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc41c106a80 0x7fc41c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.340+0000 7fc42329f640 1 -- 172.21.15.5:0/2883819320 >> 172.21.15.5:0/2883819320 conn(0x7fc41c0fc7a0 msgr2=0x7fc41c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.340+0000 7fc42329f640 1 -- 172.21.15.5:0/2883819320 shutdown_connections 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.340+0000 7fc42329f640 1 -- 172.21.15.5:0/2883819320 wait complete. 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.341+0000 7fc42329f640 1 Processor -- start 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.341+0000 7fc42329f640 1 -- start start 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.341+0000 7fc42329f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc41c100a80 0x7fc41c0fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.341+0000 7fc42329f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc41c101420 0x7fc41c0fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.341+0000 7fc42329f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c106a80 0x7fc41c0fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.341+0000 7fc42329f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c110e80 con 0x7fc41c101420 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.341+0000 7fc42329f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c110d00 con 0x7fc41c100a80 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.341+0000 7fc42329f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc41c111000 con 0x7fc41c106a80 2024-09-11T22:21:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.341+0000 7fc42083a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc41c101420 0x7fc41c0fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.341+0000 7fc42083a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc41c101420 0x7fc41c0fd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59408/0 (socket says 172.21.15.5:59408) 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42083a640 1 -- 172.21.15.5:0/277461517 learned_addr learned my addr 172.21.15.5:0/277461517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42103b640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc41c100a80 0x7fc41c0fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42183c640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c106a80 0x7fc41c0fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42083a640 1 -- 172.21.15.5:0/277461517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c106a80 msgr2=0x7fc41c0fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42083a640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c106a80 0x7fc41c0fd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42083a640 1 -- 172.21.15.5:0/277461517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc41c100a80 msgr2=0x7fc41c0fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42083a640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc41c100a80 0x7fc41c0fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42083a640 1 -- 172.21.15.5:0/277461517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41c1a6960 con 0x7fc41c101420 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42183c640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c106a80 0x7fc41c0fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42083a640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc41c101420 0x7fc41c0fd280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00d770 tx=0x7fc40c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc4127fc640 1 -- 172.21.15.5:0/277461517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c017070 con 0x7fc41c101420 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc4127fc640 1 -- 172.21.15.5:0/277461517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc40c005500 con 0x7fc41c101420 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42329f640 1 -- 172.21.15.5:0/277461517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41c1a6c50 con 0x7fc41c101420 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc4127fc640 1 -- 172.21.15.5:0/277461517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c004e00 con 0x7fc41c101420 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.342+0000 7fc42329f640 1 -- 172.21.15.5:0/277461517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41c1a7190 con 0x7fc41c101420 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.343+0000 7fc42329f640 1 -- 172.21.15.5:0/277461517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41c100ee0 con 0x7fc41c101420 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.344+0000 7fc4127fc640 1 -- 172.21.15.5:0/277461517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc40c004fa0 con 0x7fc41c101420 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.345+0000 7fc4127fc640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc4080a2c90 0x7fc4080a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.345+0000 7fc42103b640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc4080a2c90 0x7fc4080a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.345+0000 7fc4127fc640 1 -- 172.21.15.5:0/277461517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fc40c0cf050 con 0x7fc41c101420 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.346+0000 7fc42103b640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc4080a2c90 0x7fc4080a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc418005fd0 tx=0x7fc418009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.348+0000 7fc4127fc640 1 -- 172.21.15.5:0/277461517 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc40c09be50 con 0x7fc41c101420 2024-09-11T22:21:31.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.560+0000 7fc42329f640 1 -- 172.21.15.5:0/277461517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc41c10a050 con 0x7fc41c101420 2024-09-11T22:21:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.561+0000 7fc4127fc640 1 -- 172.21.15.5:0/277461517 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7fc40c09b750 con 0x7fc41c101420 2024-09-11T22:21:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:31.563 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-11T22:20:30:263672+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:20:30.263652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:21:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-11T22:21:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 -- 172.21.15.5:0/277461517 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc4080a2c90 msgr2=0x7fc4080a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc4080a2c90 0x7fc4080a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc418005fd0 tx=0x7fc418009680 comp rx=0 tx=0).stop 2024-09-11T22:21:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 -- 172.21.15.5:0/277461517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc41c101420 msgr2=0x7fc41c0fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc41c101420 0x7fc41c0fd280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00d770 tx=0x7fc40c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:21:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 -- 172.21.15.5:0/277461517 shutdown_connections 2024-09-11T22:21:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fc4080a2c90 0x7fc4080a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:31.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc41c106a80 0x7fc41c0fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc41c101420 0x7fc41c0fd280 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 --2- 172.21.15.5:0/277461517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc41c100a80 0x7fc41c0fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 -- 172.21.15.5:0/277461517 >> 172.21.15.5:0/277461517 conn(0x7fc41c0fc7a0 msgr2=0x7fc41c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 -- 172.21.15.5:0/277461517 shutdown_connections 2024-09-11T22:21:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:31.563+0000 7fc42329f640 1 -- 172.21.15.5:0/277461517 wait complete. 2024-09-11T22:21:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:31 smithi005 ceph-mon[30943]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7 op/s; 99 B/s, 0 objects/s recovering 2024-09-11T22:21:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:31 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/277461517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:32.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:31 smithi159 ceph-mon[35534]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7 op/s; 99 B/s, 0 objects/s recovering 2024-09-11T22:21:32.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:31 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/277461517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:31 smithi172 ceph-mon[35461]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7 op/s; 99 B/s, 0 objects/s recovering 2024-09-11T22:21:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:31 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/277461517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:32.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:21:32.503 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.008+0000 7f624ee52640 1 -- 172.21.15.5:0/2368479483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6248100350 msgr2=0x7f6248100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.008+0000 7f624ee52640 1 --2- 172.21.15.5:0/2368479483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6248100350 0x7f6248100730 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6238009930 tx=0x7f623802a420 comp rx=0 tx=0).stop 2024-09-11T22:21:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.008+0000 7f624ee52640 1 -- 172.21.15.5:0/2368479483 shutdown_connections 2024-09-11T22:21:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.008+0000 7f624ee52640 1 --2- 172.21.15.5:0/2368479483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6248106f50 0x7f624810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.008+0000 7f624ee52640 1 --2- 172.21.15.5:0/2368479483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6248100d00 0x7f6248106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.008+0000 7f624ee52640 1 --2- 172.21.15.5:0/2368479483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6248100350 0x7f6248100730 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.008+0000 7f624ee52640 1 -- 172.21.15.5:0/2368479483 >> 172.21.15.5:0/2368479483 conn(0x7f6248077210 msgr2=0x7f62480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.008+0000 7f624ee52640 1 -- 172.21.15.5:0/2368479483 shutdown_connections 2024-09-11T22:21:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.008+0000 7f624ee52640 1 -- 172.21.15.5:0/2368479483 wait complete. 2024-09-11T22:21:33.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f624ee52640 1 Processor -- start 2024-09-11T22:21:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f624ee52640 1 -- start start 2024-09-11T22:21:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f624ee52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6248100350 0x7f62481012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f624ee52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6248100d00 0x7f6248101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f624ee52640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6248106f50 0x7f6248101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f624ee52640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6248112000 con 0x7f6248100d00 2024-09-11T22:21:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f624ee52640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6248111e80 con 0x7f6248100350 2024-09-11T22:21:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f624ee52640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6248112180 con 0x7f6248106f50 2024-09-11T22:21:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f6247fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6248100d00 0x7f6248101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f6247fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6248100d00 0x7f6248101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59426/0 (socket says 172.21.15.5:59426) 2024-09-11T22:21:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f6247fff640 1 -- 172.21.15.5:0/3364186659 learned_addr learned my addr 172.21.15.5:0/3364186659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f624cbee640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6248100350 0x7f62481012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.009+0000 7f624d3ef640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6248106f50 0x7f6248101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f6247fff640 1 -- 172.21.15.5:0/3364186659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6248106f50 msgr2=0x7f6248101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f6247fff640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6248106f50 0x7f6248101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f6247fff640 1 -- 172.21.15.5:0/3364186659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6248100350 msgr2=0x7f62481012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f6247fff640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6248100350 0x7f62481012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f6247fff640 1 -- 172.21.15.5:0/3364186659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62481a6a20 con 0x7f6248100d00 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f624d3ef640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6248106f50 0x7f6248101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f6247fff640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6248100d00 0x7f6248101810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f623000b450 tx=0x7f623000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f6245ffb640 1 -- 172.21.15.5:0/3364186659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6230015020 con 0x7f6248100d00 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f6245ffb640 1 -- 172.21.15.5:0/3364186659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6230011d70 con 0x7f6248100d00 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f624ee52640 1 -- 172.21.15.5:0/3364186659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62481a6d10 con 0x7f6248100d00 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f6245ffb640 1 -- 172.21.15.5:0/3364186659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623000f340 con 0x7f6248100d00 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.010+0000 7f624ee52640 1 -- 172.21.15.5:0/3364186659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62481a71a0 con 0x7f6248100d00 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.011+0000 7f624ee52640 1 -- 172.21.15.5:0/3364186659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6248100730 con 0x7f6248100d00 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.012+0000 7f6245ffb640 1 -- 172.21.15.5:0/3364186659 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6230018020 con 0x7f6248100d00 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.013+0000 7f6245ffb640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f62340a2c10 0x7f62340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.013+0000 7f624cbee640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f62340a2c10 0x7f62340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.013+0000 7f6245ffb640 1 -- 172.21.15.5:0/3364186659 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f62300cec70 con 0x7f6248100d00 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.013+0000 7f624cbee640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f62340a2c10 0x7f62340a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6238000c00 tx=0x7f6238031490 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.017+0000 7f6245ffb640 1 -- 172.21.15.5:0/3364186659 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f623001f020 con 0x7f6248100d00 2024-09-11T22:21:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.193+0000 7f624ee52640 1 -- 172.21.15.5:0/3364186659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f624810a140 con 0x7f6248100d00 2024-09-11T22:21:33.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.194+0000 7f6245ffb640 1 -- 172.21.15.5:0/3364186659 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22697 (secure 0 0 0) 0x7f623000d040 con 0x7f6248100d00 2024-09-11T22:21:33.196 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:33.196 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:21:18.179555+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:21:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 -- 172.21.15.5:0/3364186659 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f62340a2c10 msgr2=0x7f62340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f62340a2c10 0x7f62340a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6238000c00 tx=0x7f6238031490 comp rx=0 tx=0).stop 2024-09-11T22:21:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 -- 172.21.15.5:0/3364186659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6248100d00 msgr2=0x7f6248101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6248100d00 0x7f6248101810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f623000b450 tx=0x7f623000b910 comp rx=0 tx=0).stop 2024-09-11T22:21:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 -- 172.21.15.5:0/3364186659 shutdown_connections 2024-09-11T22:21:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f62340a2c10 0x7f62340a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6248106f50 0x7f6248101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6248100d00 0x7f6248101810 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 --2- 172.21.15.5:0/3364186659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6248100350 0x7f62481012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 -- 172.21.15.5:0/3364186659 >> 172.21.15.5:0/3364186659 conn(0x7f6248077210 msgr2=0x7f6248105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.196+0000 7f624ee52640 1 -- 172.21.15.5:0/3364186659 shutdown_connections 2024-09-11T22:21:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:33.197+0000 7f624ee52640 1 -- 172.21.15.5:0/3364186659 wait complete. 2024-09-11T22:21:33.830 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:21:34.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:33 smithi005 ceph-mon[30943]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 1 B/s, 0 objects/s recovering 2024-09-11T22:21:34.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:33 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3364186659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:21:34.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:33 smithi159 ceph-mon[35534]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 1 B/s, 0 objects/s recovering 2024-09-11T22:21:34.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:33 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3364186659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:21:34.087 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:34.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:33 smithi172 ceph-mon[35461]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 1 B/s, 0 objects/s recovering 2024-09-11T22:21:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:33 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3364186659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:21:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.576+0000 7f4e01c53640 1 -- 172.21.15.5:0/3647841116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc06a370 msgr2=0x7f4dfc1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.576+0000 7f4e01c53640 1 --2- 172.21.15.5:0/3647841116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc06a370 0x7f4dfc1009c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4df0009900 tx=0x7f4df002a410 comp rx=0 tx=0).stop 2024-09-11T22:21:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.576+0000 7f4e01c53640 1 -- 172.21.15.5:0/3647841116 shutdown_connections 2024-09-11T22:21:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.576+0000 7f4e01c53640 1 --2- 172.21.15.5:0/3647841116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dfc104e80 0x7f4dfc105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.576+0000 7f4e01c53640 1 --2- 172.21.15.5:0/3647841116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc06a370 0x7f4dfc1009c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.576+0000 7f4e01c53640 1 --2- 172.21.15.5:0/3647841116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dfc069f00 0x7f4dfc100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.576+0000 7f4e01c53640 1 -- 172.21.15.5:0/3647841116 >> 172.21.15.5:0/3647841116 conn(0x7f4dfc077210 msgr2=0x7f4dfc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.576+0000 7f4e01c53640 1 -- 172.21.15.5:0/3647841116 shutdown_connections 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.576+0000 7f4e01c53640 1 -- 172.21.15.5:0/3647841116 wait complete. 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4e01c53640 1 Processor -- start 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4e01c53640 1 -- start start 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4e01c53640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dfc069f00 0x7f4dfc102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4e01c53640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dfc06a370 0x7f4dfc103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4e01c53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc104e80 0x7f4dfc1035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4e01c53640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc110eb0 con 0x7f4dfc104e80 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4e01c53640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc110d30 con 0x7f4dfc06a370 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4e01c53640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc111030 con 0x7f4dfc069f00 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4dfbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc104e80 0x7f4dfc1035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4dfbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc104e80 0x7f4dfc1035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59446/0 (socket says 172.21.15.5:59446) 2024-09-11T22:21:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4dfbfff640 1 -- 172.21.15.5:0/1709723387 learned_addr learned my addr 172.21.15.5:0/1709723387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4dfb7fe640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dfc069f00 0x7f4dfc102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.577+0000 7f4dfaffd640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dfc06a370 0x7f4dfc103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4dfbfff640 1 -- 172.21.15.5:0/1709723387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dfc069f00 msgr2=0x7f4dfc102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4dfbfff640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dfc069f00 0x7f4dfc102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4dfbfff640 1 -- 172.21.15.5:0/1709723387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dfc06a370 msgr2=0x7f4dfc103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4dfbfff640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dfc06a370 0x7f4dfc103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4dfbfff640 1 -- 172.21.15.5:0/1709723387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc1a6970 con 0x7f4dfc104e80 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4dfbfff640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc104e80 0x7f4dfc1035a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4df400eaa0 tx=0x7f4df400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4df8ff9640 1 -- 172.21.15.5:0/1709723387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4004d20 con 0x7f4dfc104e80 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4df8ff9640 1 -- 172.21.15.5:0/1709723387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df4005020 con 0x7f4dfc104e80 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4df8ff9640 1 -- 172.21.15.5:0/1709723387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df40116d0 con 0x7f4dfc104e80 2024-09-11T22:21:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4e01c53640 1 -- 172.21.15.5:0/1709723387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc1a6c60 con 0x7f4dfc104e80 2024-09-11T22:21:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.578+0000 7f4e01c53640 1 -- 172.21.15.5:0/1709723387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc1a7120 con 0x7f4dfc104e80 2024-09-11T22:21:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.579+0000 7f4e01c53640 1 -- 172.21.15.5:0/1709723387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dfc105330 con 0x7f4dfc104e80 2024-09-11T22:21:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.580+0000 7f4df8ff9640 1 -- 172.21.15.5:0/1709723387 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4df40051d0 con 0x7f4dfc104e80 2024-09-11T22:21:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.581+0000 7f4df8ff9640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4de40a2c10 0x7f4de40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.581+0000 7f4df8ff9640 1 -- 172.21.15.5:0/1709723387 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f4df40cf740 con 0x7f4dfc104e80 2024-09-11T22:21:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.581+0000 7f4dfb7fe640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4de40a2c10 0x7f4de40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.581+0000 7f4dfb7fe640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4de40a2c10 0x7f4de40a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009990 tx=0x7f4dec009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.584+0000 7f4df8ff9640 1 -- 172.21.15.5:0/1709723387 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4df4016030 con 0x7f4dfc104e80 2024-09-11T22:21:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.800+0000 7f4e01c53640 1 -- 172.21.15.5:0/1709723387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4dfc1a73b0 con 0x7f4dfc104e80 2024-09-11T22:21:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.800+0000 7f4df8ff9640 1 -- 172.21.15.5:0/1709723387 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7f4df409c520 con 0x7f4dfc104e80 2024-09-11T22:21:34.802 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:34.802 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-11T22:20:30:263672+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:20:30.263652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:21:34.803 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-11T22:21:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.802+0000 7f4e01c53640 1 -- 172.21.15.5:0/1709723387 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4de40a2c10 msgr2=0x7f4de40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.802+0000 7f4e01c53640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4de40a2c10 0x7f4de40a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009990 tx=0x7f4dec009110 comp rx=0 tx=0).stop 2024-09-11T22:21:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.802+0000 7f4e01c53640 1 -- 172.21.15.5:0/1709723387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc104e80 msgr2=0x7f4dfc1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:34.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.802+0000 7f4e01c53640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc104e80 0x7f4dfc1035a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4df400eaa0 tx=0x7f4df400eea0 comp rx=0 tx=0).stop 2024-09-11T22:21:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.803+0000 7f4e01c53640 1 -- 172.21.15.5:0/1709723387 shutdown_connections 2024-09-11T22:21:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.803+0000 7f4e01c53640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f4de40a2c10 0x7f4de40a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.803+0000 7f4e01c53640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc104e80 0x7f4dfc1035a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.803+0000 7f4e01c53640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4dfc06a370 0x7f4dfc103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.803+0000 7f4e01c53640 1 --2- 172.21.15.5:0/1709723387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4dfc069f00 0x7f4dfc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.803+0000 7f4e01c53640 1 -- 172.21.15.5:0/1709723387 >> 172.21.15.5:0/1709723387 conn(0x7f4dfc077210 msgr2=0x7f4dfc0fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.803+0000 7f4e01c53640 1 -- 172.21.15.5:0/1709723387 shutdown_connections 2024-09-11T22:21:34.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:34.803+0000 7f4e01c53640 1 -- 172.21.15.5:0/1709723387 wait complete. 2024-09-11T22:21:35.522 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T22:20:18.900781+0000', 'modified': '2024-09-11T22:20:30.263652+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24500, 'mds_1': 24473, 'mds_2': 24454, 'mds_3': 24452, 'mds_4': 24436}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/2458034474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 2458034474}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 2458034474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/2232645952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 2232645952}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 2232645952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6833/1186865242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6832', 'nonce': 1186865242}, {'type': 'v1', 'addr': '172.21.15.172:6833', 'nonce': 1186865242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/4149321843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 4149321843}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 4149321843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/2975110123', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 2975110123}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 2975110123}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.172:6837/990138139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 990138139}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 990138139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.159:6835/1262598700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1262598700}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1262598700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.172:6839/1937249261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 1937249261}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 1937249261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.159:6837/2315339117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 2315339117}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 2315339117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.159:6839/994288762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 994288762}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 994288762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24500, 24473, 24454, 24452, 24436]} 2024-09-11T22:21:35.522 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T22:21:35.522 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T22:21:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:35.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:35 smithi005 ceph-mon[30943]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T22:21:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:35 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1709723387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:36.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:35 smithi159 ceph-mon[35534]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T22:21:36.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:35 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1709723387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:36.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:35 smithi172 ceph-mon[35461]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-11T22:21:36.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:35 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1709723387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.271+0000 7f3852d42640 1 -- 172.21.15.5:0/3402754251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f384c10c170 msgr2=0x7f384c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.271+0000 7f3852d42640 1 --2- 172.21.15.5:0/3402754251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f384c10c170 0x7f384c10e610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3844009ca0 tx=0x7f384402a950 comp rx=0 tx=0).stop 2024-09-11T22:21:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.271+0000 7f3852d42640 1 -- 172.21.15.5:0/3402754251 shutdown_connections 2024-09-11T22:21:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.271+0000 7f3852d42640 1 --2- 172.21.15.5:0/3402754251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f384c10c170 0x7f384c10e610 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.271+0000 7f3852d42640 1 --2- 172.21.15.5:0/3402754251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f384c103660 0x7f384c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.271+0000 7f3852d42640 1 --2- 172.21.15.5:0/3402754251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c102d40 0x7f384c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.271+0000 7f3852d42640 1 -- 172.21.15.5:0/3402754251 >> 172.21.15.5:0/3402754251 conn(0x7f384c0fc7a0 msgr2=0x7f384c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.271+0000 7f3852d42640 1 -- 172.21.15.5:0/3402754251 shutdown_connections 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.271+0000 7f3852d42640 1 -- 172.21.15.5:0/3402754251 wait complete. 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.271+0000 7f3852d42640 1 Processor -- start 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f3852d42640 1 -- start start 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f3852d42640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c102d40 0x7f384c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f3852d42640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f384c103660 0x7f384c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f3852d42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f384c10c170 0x7f384c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f3852d42640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f384c112030 con 0x7f384c10c170 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f3852d42640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f384c111eb0 con 0x7f384c102d40 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f3852d42640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f384c1121b0 con 0x7f384c103660 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f38512df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f384c10c170 0x7f384c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f38512df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f384c10c170 0x7f384c101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59464/0 (socket says 172.21.15.5:59464) 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f38512df640 1 -- 172.21.15.5:0/4054742710 learned_addr learned my addr 172.21.15.5:0/4054742710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f384bfff640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f384c103660 0x7f384c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f3850ade640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c102d40 0x7f384c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f38512df640 1 -- 172.21.15.5:0/4054742710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f384c103660 msgr2=0x7f384c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f38512df640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f384c103660 0x7f384c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f38512df640 1 -- 172.21.15.5:0/4054742710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c102d40 msgr2=0x7f384c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f38512df640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c102d40 0x7f384c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f38512df640 1 -- 172.21.15.5:0/4054742710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384c1a6a00 con 0x7f384c10c170 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f3850ade640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c102d40 0x7f384c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.272+0000 7f38512df640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f384c10c170 0x7f384c101420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3844006010 tx=0x7f384402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.273+0000 7f3849ffb640 1 -- 172.21.15.5:0/4054742710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3844034040 con 0x7f384c10c170 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.273+0000 7f3849ffb640 1 -- 172.21.15.5:0/4054742710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3844003e00 con 0x7f384c10c170 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.273+0000 7f3852d42640 1 -- 172.21.15.5:0/4054742710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384c1a6c90 con 0x7f384c10c170 2024-09-11T22:21:36.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.273+0000 7f3849ffb640 1 -- 172.21.15.5:0/4054742710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38440456e0 con 0x7f384c10c170 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.273+0000 7f3852d42640 1 -- 172.21.15.5:0/4054742710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f384c1a71c0 con 0x7f384c10c170 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.275+0000 7f3849ffb640 1 -- 172.21.15.5:0/4054742710 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3844037020 con 0x7f384c10c170 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.275+0000 7f385511b740 1 Processor -- start 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.275+0000 7f385511b740 1 -- start start 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f3849ffb640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38340a2c90 0x7f38340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f385511b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa5872db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f385511b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58732f0 0x556fa587d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f385511b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa587dbb0 0x556fa587ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f385511b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556fa57d9090 con 0x556fa58729d0 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f385511b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556fa5780a70 con 0x556fa58732f0 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f385511b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556fa57d5050 con 0x556fa587dbb0 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f384bfff640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38340a2c90 0x7f38340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f3850ade640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa5872db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f3850ade640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa5872db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59480/0 (socket says 172.21.15.5:59480) 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f3850ade640 1 -- 172.21.15.5:0/3916704021 learned_addr learned my addr 172.21.15.5:0/3916704021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f3849ffb640 1 -- 172.21.15.5:0/4054742710 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f38440f0180 con 0x7f384c10c170 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f384bfff640 1 --2- 172.21.15.5:0/3916704021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58732f0 0x556fa587d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f38512df640 1 --2- 172.21.15.5:0/3916704021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa587dbb0 0x556fa587ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f3850ade640 1 -- 172.21.15.5:0/3916704021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa587dbb0 msgr2=0x556fa587ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f3850ade640 1 --2- 172.21.15.5:0/3916704021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa587dbb0 0x556fa587ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f3850ade640 1 -- 172.21.15.5:0/3916704021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58732f0 msgr2=0x556fa587d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f3850ade640 1 --2- 172.21.15.5:0/3916704021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58732f0 0x556fa587d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f3850ade640 1 -- 172.21.15.5:0/3916704021 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa578d130 con 0x556fa58729d0 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.276+0000 7f384bfff640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38340a2c90 0x7f38340a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f384c102430 tx=0x7f38400156d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.277+0000 7f3850ade640 1 --2- 172.21.15.5:0/3916704021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa5872db0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f383c00aa10 tx=0x7f383c00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f7565b4cb08087 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.277+0000 7f3832ffd640 1 -- 172.21.15.5:0/3916704021 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c015070 con 0x556fa58729d0 2024-09-11T22:21:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.277+0000 7f3832ffd640 1 -- 172.21.15.5:0/3916704021 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c00cbe0 con 0x556fa58729d0 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.277+0000 7f3832ffd640 1 -- 172.21.15.5:0/3916704021 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c0185c0 con 0x556fa58729d0 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.277+0000 7f385511b740 1 -- 172.21.15.5:0/3916704021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 msgr2=0x556fa5872db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.277+0000 7f385511b740 1 --2- 172.21.15.5:0/3916704021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa5872db0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f383c00aa10 tx=0x7f383c00aed0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.278+0000 7f385511b740 1 -- 172.21.15.5:0/3916704021 shutdown_connections 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.278+0000 7f385511b740 1 --2- 172.21.15.5:0/3916704021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa587dbb0 0x556fa587ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.278+0000 7f385511b740 1 --2- 172.21.15.5:0/3916704021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58732f0 0x556fa587d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.278+0000 7f385511b740 1 --2- 172.21.15.5:0/3916704021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa5872db0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.278+0000 7f385511b740 1 -- 172.21.15.5:0/3916704021 >> 172.21.15.5:0/3916704021 conn(0x556fa586ef60 msgr2=0x556fa586f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.278+0000 7f385511b740 1 -- 172.21.15.5:0/3916704021 shutdown_connections 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.278+0000 7f385511b740 1 -- 172.21.15.5:0/3916704021 wait complete. 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.278+0000 7f385511b740 1 -- start start 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.278+0000 7f385511b740 1 Processor -- start 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f385511b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 0x556fa587d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f385511b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58732f0 0x556fa587a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f385511b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa587dbb0 0x556fa587aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f385511b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556fa5880780 con 0x556fa58732f0 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f385511b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556fa5793f20 con 0x556fa58729d0 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f385511b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556fa5880900 con 0x556fa587dbb0 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f385511b740 10 client.0 _ll_register_callbacks cb 0x556fa575d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f384bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58732f0 0x556fa587a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f384bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58732f0 0x556fa587a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59482/0 (socket says 172.21.15.5:59482) 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f384bfff640 1 -- 172.21.15.5:0/3523166046 learned_addr learned my addr 172.21.15.5:0/3523166046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f38512df640 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa587dbb0 0x556fa587aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f3850ade640 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 0x556fa587d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f384bfff640 1 -- 172.21.15.5:0/3523166046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa587dbb0 msgr2=0x556fa587aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f384bfff640 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa587dbb0 0x556fa587aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f384bfff640 1 -- 172.21.15.5:0/3523166046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 msgr2=0x556fa587d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f384bfff640 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 0x556fa587d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f384bfff640 1 -- 172.21.15.5:0/3523166046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa554a2b0 con 0x556fa58732f0 2024-09-11T22:21:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f3850ade640 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 0x556fa587d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f384bfff640 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58732f0 0x556fa587a660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f38400187b0 tx=0x7f38400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ceb332a175042d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f382bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:21:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f382bfff640 1 -- 172.21.15.5:0/3523166046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38400178e0 con 0x556fa58732f0 2024-09-11T22:21:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f382bfff640 1 -- 172.21.15.5:0/3523166046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f3840017a80 con 0x556fa58732f0 2024-09-11T22:21:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.279+0000 7f385511b740 1 -- 172.21.15.5:0/3523166046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383c00cbe0 con 0x556fa58732f0 2024-09-11T22:21:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.280+0000 7f382bfff640 1 -- 172.21.15.5:0/3523166046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3840017c80 con 0x556fa58732f0 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.280+0000 7f385511b740 10 client.14949 fetch_fsmap learned FSMap version 27 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.280+0000 7f385511b740 1 -- 172.21.15.5:0/3523166046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556fa588bee0 con 0x556fa58732f0 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.281+0000 7f382bfff640 1 -- 172.21.15.5:0/3523166046 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c00cbe0 con 0x556fa58732f0 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.281+0000 7f382bfff640 1 -- 172.21.15.5:0/3523166046 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f384002d510 con 0x556fa58732f0 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.281+0000 7f385511b740 10 client.14949 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.281+0000 7f385511b740 10 client.14949 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.281+0000 7f385511b740 20 client.14949 populate_metadata read hostname 'smithi005' 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.281+0000 7f385511b740 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x556fa588ea60 0x556fa588fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.281+0000 7f385511b740 4 client.14949 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.281+0000 7f385511b740 1 -- 172.21.15.5:0/3523166046 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fa57e6600 con 0x556fa588ea60 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.281+0000 7f3850ade640 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x556fa588ea60 0x556fa588fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.282+0000 7f3850ade640 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x556fa588ea60 0x556fa588fed0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34507d9ccf8c5f58 server_cookie=781b61f6bb0d79d8 in_seq=0 out_seq=0 2024-09-11T22:21:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.282+0000 7f382bfff640 10 client.14949 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:21:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.282+0000 7f382bfff640 1 -- 172.21.15.5:0/3523166046 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fa57e6600 con 0x556fa588ea60 2024-09-11T22:21:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.282+0000 7f382bfff640 10 client.14949 handle_command_reply: tid=0 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.282+0000 7f385511b740 1 client.14949 shutdown 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 1 -- 172.21.15.5:0/3523166046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58732f0 msgr2=0x556fa587a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58732f0 0x556fa587a660 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f38400187b0 tx=0x7f38400046c0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 1 -- 172.21.15.5:0/3523166046 shutdown_connections 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x556fa588ea60 0x556fa588fed0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa587dbb0 0x556fa587aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58732f0 0x556fa587a660 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 1 --2- 172.21.15.5:0/3523166046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 0x556fa587d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 1 -- 172.21.15.5:0/3523166046 >> 172.21.15.5:0/3523166046 conn(0x556fa586ef60 msgr2=0x556fa5874c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f382bfff640 0 client.14949 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 1 -- 172.21.15.5:0/3523166046 shutdown_connections 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 1 -- 172.21.15.5:0/3523166046 wait complete. 2024-09-11T22:21:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.283+0000 7f385511b740 20 client.14949 trim_cache size 0 max 16384 2024-09-11T22:21:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f385511b740 1 Processor -- start 2024-09-11T22:21:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f385511b740 1 -- start start 2024-09-11T22:21:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f385511b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa588f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f385511b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa58732f0 0x556fa5869e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f385511b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa587dbb0 0x556fa5879f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f385511b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556fa588e210 con 0x556fa58729d0 2024-09-11T22:21:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f385511b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556fa588e8e0 con 0x556fa587dbb0 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f385511b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556fa588efb0 con 0x556fa58732f0 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f3850ade640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa588f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f3850ade640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa588f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59494/0 (socket says 172.21.15.5:59494) 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f3850ade640 1 -- 172.21.15.5:0/2196387688 learned_addr learned my addr 172.21.15.5:0/2196387688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f3850ade640 1 -- 172.21.15.5:0/2196387688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa58732f0 msgr2=0x556fa5869e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f3850ade640 1 --2- 172.21.15.5:0/2196387688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa58732f0 0x556fa5869e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f3850ade640 1 -- 172.21.15.5:0/2196387688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa587dbb0 msgr2=0x556fa5879f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f3850ade640 1 --2- 172.21.15.5:0/2196387688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa587dbb0 0x556fa5879f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f3850ade640 1 -- 172.21.15.5:0/2196387688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa5878bb0 con 0x556fa58729d0 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.301+0000 7f3850ade640 1 --2- 172.21.15.5:0/2196387688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa588f800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f383c011e20 tx=0x7f383c011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4a1e2b86e06b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f38327fc640 1 -- 172.21.15.5:0/2196387688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c015040 con 0x556fa58729d0 2024-09-11T22:21:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f38327fc640 1 -- 172.21.15.5:0/2196387688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c02a070 con 0x556fa58729d0 2024-09-11T22:21:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f38327fc640 1 -- 172.21.15.5:0/2196387688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383c025730 con 0x556fa58729d0 2024-09-11T22:21:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f385511b740 1 -- 172.21.15.5:0/2196387688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 msgr2=0x556fa588f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f385511b740 1 --2- 172.21.15.5:0/2196387688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa588f800 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f383c011e20 tx=0x7f383c011db0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f385511b740 1 -- 172.21.15.5:0/2196387688 shutdown_connections 2024-09-11T22:21:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f385511b740 1 --2- 172.21.15.5:0/2196387688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa587dbb0 0x556fa5879f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f385511b740 1 --2- 172.21.15.5:0/2196387688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa58732f0 0x556fa5869e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f385511b740 1 --2- 172.21.15.5:0/2196387688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa58729d0 0x556fa588f800 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f385511b740 1 -- 172.21.15.5:0/2196387688 >> 172.21.15.5:0/2196387688 conn(0x556fa586ef60 msgr2=0x556fa5878f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.302+0000 7f385511b740 1 -- 172.21.15.5:0/2196387688 shutdown_connections 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f385511b740 1 -- 172.21.15.5:0/2196387688 wait complete. 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f385511b740 1 -- start start 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f385511b740 1 Processor -- start 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f385511b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 0x556fa587f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f385511b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa58732f0 0x556fa587df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f385511b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa587dbb0 0x556fa587e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f385511b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556fa588fd50 con 0x556fa587dbb0 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f385511b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556fa588f680 con 0x556fa58729d0 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f385511b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556fa5890420 con 0x556fa58732f0 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f385511b740 10 client.0 _ll_register_callbacks cb 0x7f38400093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f38512df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa587dbb0 0x556fa587e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f38512df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa587dbb0 0x556fa587e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59500/0 (socket says 172.21.15.5:59500) 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.303+0000 7f38512df640 1 -- 172.21.15.5:0/2839755687 learned_addr learned my addr 172.21.15.5:0/2839755687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f3850ade640 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 0x556fa587f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f384bfff640 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa58732f0 0x556fa587df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f38512df640 1 -- 172.21.15.5:0/2839755687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa58732f0 msgr2=0x556fa587df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f38512df640 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa58732f0 0x556fa587df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f38512df640 1 -- 172.21.15.5:0/2839755687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 msgr2=0x556fa587f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f38512df640 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 0x556fa587f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f38512df640 1 -- 172.21.15.5:0/2839755687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa587ed40 con 0x556fa587dbb0 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f38512df640 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa587dbb0 0x556fa587e510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f384402f780 tx=0x7f38440bc7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9be66695f81145 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f382bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f382bfff640 1 -- 172.21.15.5:0/2839755687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38440d5060 con 0x556fa587dbb0 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f382bfff640 1 -- 172.21.15.5:0/2839755687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f384404e3f0 con 0x556fa587dbb0 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f385511b740 1 -- 172.21.15.5:0/2839755687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f383c02a070 con 0x556fa587dbb0 2024-09-11T22:21:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f382bfff640 1 -- 172.21.15.5:0/2839755687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f38440bd070 con 0x556fa587dbb0 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f385511b740 10 client.14955 fetch_fsmap learned FSMap version 27 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f385511b740 1 -- 172.21.15.5:0/2839755687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556fa58981c0 con 0x556fa587dbb0 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.304+0000 7f382bfff640 1 -- 172.21.15.5:0/2839755687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383c02a070 con 0x556fa587dbb0 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.305+0000 7f382bfff640 1 -- 172.21.15.5:0/2839755687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f384404d030 con 0x556fa587dbb0 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.305+0000 7f385511b740 10 client.14955 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.305+0000 7f385511b740 10 client.14955 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.305+0000 7f385511b740 20 client.14955 populate_metadata read hostname 'smithi005' 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.305+0000 7f385511b740 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x556fa58fcf30 0x556fa58fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.305+0000 7f385511b740 4 client.14955 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.305+0000 7f385511b740 1 -- 172.21.15.5:0/2839755687 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "status"}) -- 0x556fa57e0b10 con 0x556fa58fcf30 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.305+0000 7f3850ade640 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x556fa58fcf30 0x556fa58fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.305+0000 7f382bfff640 10 client.14955 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:21:36.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.305+0000 7f3850ade640 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x556fa58fcf30 0x556fa58fbc40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da8bca0cb4d07f6c server_cookie=6925114c5c92d9b6 in_seq=0 out_seq=0 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f382bfff640 1 -- 172.21.15.5:0/2839755687 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556fa57e0b10 con 0x556fa58fcf30 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f382bfff640 10 client.14955 handle_command_reply: tid=0 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 client.14955 shutdown 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 -- 172.21.15.5:0/2839755687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa587dbb0 msgr2=0x556fa587e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa587dbb0 0x556fa587e510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f384402f780 tx=0x7f38440bc7b0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 -- 172.21.15.5:0/2839755687 shutdown_connections 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x556fa58fcf30 0x556fa58fbc40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fa587dbb0 0x556fa587e510 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f382bfff640 0 client.14955 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556fa58732f0 0x556fa587df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 --2- 172.21.15.5:0/2839755687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fa58729d0 0x556fa587f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 -- 172.21.15.5:0/2839755687 >> 172.21.15.5:0/2839755687 conn(0x556fa586ef60 msgr2=0x556fa58755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 -- 172.21.15.5:0/2839755687 shutdown_connections 2024-09-11T22:21:36.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 1 -- 172.21.15.5:0/2839755687 wait complete. 2024-09-11T22:21:36.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.306+0000 7f385511b740 20 client.14955 trim_cache size 0 max 16384 2024-09-11T22:21:36.313 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:21:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:21:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-11T22:21:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24436, 2024-09-11T22:21:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:21:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:21:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:21:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 67.045351288999996, 2024-09-11T22:21:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-11T22:21:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-09-11T22:21:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T22:21:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 102.421431695 2024-09-11T22:21:36.314 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:21:36.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 -- 172.21.15.5:0/4054742710 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38340a2c90 msgr2=0x7f38340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38340a2c90 0x7f38340a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f384c102430 tx=0x7f38400156d0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 -- 172.21.15.5:0/4054742710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f384c10c170 msgr2=0x7f384c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f384c10c170 0x7f384c101420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3844006010 tx=0x7f384402f710 comp rx=0 tx=0).stop 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 -- 172.21.15.5:0/4054742710 shutdown_connections 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38340a2c90 0x7f38340a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f384c10c170 0x7f384c101420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f384c103660 0x7f384c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 --2- 172.21.15.5:0/4054742710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f384c102d40 0x7f384c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 -- 172.21.15.5:0/4054742710 >> 172.21.15.5:0/4054742710 conn(0x7f384c0fc7a0 msgr2=0x7f384c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 -- 172.21.15.5:0/4054742710 shutdown_connections 2024-09-11T22:21:36.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:36.307+0000 7f3852d42640 1 -- 172.21.15.5:0/4054742710 wait complete. 2024-09-11T22:21:37.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T22:21:37.336 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:37.708 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:37 smithi005 ceph-mon[30943]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-11T22:21:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.866+0000 7f37c37ba640 1 -- 172.21.15.5:0/2871762668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc102790 msgr2=0x7f37bc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.866+0000 7f37c37ba640 1 --2- 172.21.15.5:0/2871762668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc102790 0x7f37bc1067e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f37ac009900 tx=0x7f37ac02a410 comp rx=0 tx=0).stop 2024-09-11T22:21:37.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.866+0000 7f37c37ba640 1 -- 172.21.15.5:0/2871762668 shutdown_connections 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.866+0000 7f37c37ba640 1 --2- 172.21.15.5:0/2871762668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc106f10 0x7f37bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.866+0000 7f37c37ba640 1 --2- 172.21.15.5:0/2871762668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc102790 0x7f37bc1067e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.866+0000 7f37c37ba640 1 --2- 172.21.15.5:0/2871762668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37bc101de0 0x7f37bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.866+0000 7f37c37ba640 1 -- 172.21.15.5:0/2871762668 >> 172.21.15.5:0/2871762668 conn(0x7f37bc0fd8c0 msgr2=0x7f37bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.866+0000 7f37c37ba640 1 -- 172.21.15.5:0/2871762668 shutdown_connections 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.866+0000 7f37c37ba640 1 -- 172.21.15.5:0/2871762668 wait complete. 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c37ba640 1 Processor -- start 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c37ba640 1 -- start start 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c37ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc101de0 0x7f37bc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c37ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc102790 0x7f37bc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c37ba640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37bc106f10 0x7f37bc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c37ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc111f70 con 0x7f37bc101de0 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c37ba640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc111df0 con 0x7f37bc106f10 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c37ba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc1120f0 con 0x7f37bc102790 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c1556640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc101de0 0x7f37bc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c1556640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc101de0 0x7f37bc198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59510/0 (socket says 172.21.15.5:59510) 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c1556640 1 -- 172.21.15.5:0/4149202738 learned_addr learned my addr 172.21.15.5:0/4149202738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.867+0000 7f37c0d55640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc102790 0x7f37bc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37c1d57640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37bc106f10 0x7f37bc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37c1556640 1 -- 172.21.15.5:0/4149202738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc102790 msgr2=0x7f37bc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37c1556640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc102790 0x7f37bc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37c1556640 1 -- 172.21.15.5:0/4149202738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37bc106f10 msgr2=0x7f37bc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37c1556640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37bc106f10 0x7f37bc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37c1556640 1 -- 172.21.15.5:0/4149202738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc19d3e0 con 0x7f37bc101de0 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37c1556640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc101de0 0x7f37bc198430 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c890 tx=0x7f37b800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37b27fc640 1 -- 172.21.15.5:0/4149202738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8004220 con 0x7f37bc101de0 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37c37ba640 1 -- 172.21.15.5:0/4149202738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc19d6d0 con 0x7f37bc101de0 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37b27fc640 1 -- 172.21.15.5:0/4149202738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b80043c0 con 0x7f37bc101de0 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37b27fc640 1 -- 172.21.15.5:0/4149202738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8004a00 con 0x7f37bc101de0 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.868+0000 7f37c37ba640 1 -- 172.21.15.5:0/4149202738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc1a4fb0 con 0x7f37bc101de0 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37b27fc640 1 -- 172.21.15.5:0/4149202738 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f37b80079b0 con 0x7f37bc101de0 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c5b93740 1 Processor -- start 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c5b93740 1 -- start start 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581802799d0 0x558180279db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x558180284670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558180284bb0 0x558180286fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c5b93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5581801e0090 con 0x55818027a2f0 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c5b93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558180187a70 con 0x5581802799d0 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c5b93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5581801dc050 con 0x558180284bb0 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c1556640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x558180284670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c1556640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x558180284670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59524/0 (socket says 172.21.15.5:59524) 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c1556640 1 -- 172.21.15.5:0/2078927361 learned_addr learned my addr 172.21.15.5:0/2078927361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c1d57640 1 --2- 172.21.15.5:0/2078927361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581802799d0 0x558180279db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.871+0000 7f37c0d55640 1 --2- 172.21.15.5:0/2078927361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558180284bb0 0x558180286fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37b27fc640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f37a80a2c10 0x7f37a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37c1556640 1 -- 172.21.15.5:0/2078927361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558180284bb0 msgr2=0x558180286fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37c1556640 1 --2- 172.21.15.5:0/2078927361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558180284bb0 0x558180286fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37c1556640 1 -- 172.21.15.5:0/2078927361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581802799d0 msgr2=0x558180279db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37c1556640 1 --2- 172.21.15.5:0/2078927361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581802799d0 0x558180279db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37c1556640 1 -- 172.21.15.5:0/2078927361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558180194130 con 0x55818027a2f0 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37c0d55640 1 --2- 172.21.15.5:0/2078927361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558180284bb0 0x558180286fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37c1d57640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f37a80a2c10 0x7f37a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37b27fc640 1 -- 172.21.15.5:0/4149202738 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f37b80cefa0 con 0x7f37bc101de0 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37c1556640 1 --2- 172.21.15.5:0/2078927361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x558180284670 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f37b80cc870 tx=0x7f37b8008ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d719f3477501564c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37a77fe640 1 -- 172.21.15.5:0/2078927361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b809b1c0 con 0x55818027a2f0 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37a77fe640 1 -- 172.21.15.5:0/2078927361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b8008450 con 0x55818027a2f0 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.872+0000 7f37a77fe640 1 -- 172.21.15.5:0/2078927361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b80662c0 con 0x55818027a2f0 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 -- 172.21.15.5:0/2078927361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 msgr2=0x558180284670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c1d57640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f37a80a2c10 0x7f37a80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f37b4005fd0 tx=0x7f37b4004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:37.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 --2- 172.21.15.5:0/2078927361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x558180284670 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f37b80cc870 tx=0x7f37b8008ed0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 -- 172.21.15.5:0/2078927361 shutdown_connections 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 --2- 172.21.15.5:0/2078927361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558180284bb0 0x558180286fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 --2- 172.21.15.5:0/2078927361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x558180284670 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 --2- 172.21.15.5:0/2078927361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5581802799d0 0x558180279db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 -- 172.21.15.5:0/2078927361 >> 172.21.15.5:0/2078927361 conn(0x558180275f60 msgr2=0x558180276340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 -- 172.21.15.5:0/2078927361 shutdown_connections 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 -- 172.21.15.5:0/2078927361 wait complete. 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 -- start start 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.873+0000 7f37c5b93740 1 Processor -- start 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581802799d0 0x558180284380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x5581802816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558180284bb0 0x558180281c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c5b93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558180287780 con 0x55818027a2f0 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c5b93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55818019af20 con 0x558180284bb0 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c5b93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558180287900 con 0x5581802799d0 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c5b93740 10 client.0 _ll_register_callbacks cb 0x558180164030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c0d55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x5581802816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c0d55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x5581802816d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59536/0 (socket says 172.21.15.5:59536) 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c0d55640 1 -- 172.21.15.5:0/1921661194 learned_addr learned my addr 172.21.15.5:0/1921661194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c1556640 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581802799d0 0x558180284380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c1d57640 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558180284bb0 0x558180281c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c0d55640 1 -- 172.21.15.5:0/1921661194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581802799d0 msgr2=0x558180284380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:37.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c0d55640 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581802799d0 0x558180284380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c0d55640 1 -- 172.21.15.5:0/1921661194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558180284bb0 msgr2=0x558180281c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c0d55640 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558180284bb0 0x558180281c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c0d55640 1 -- 172.21.15.5:0/1921661194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55817ff512b0 con 0x55818027a2f0 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c1556640 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581802799d0 0x558180284380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c1d57640 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558180284bb0 0x558180281c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37c0d55640 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x5581802816d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f37ac02a930 tx=0x7f37ac0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818c5552c23c31c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.874+0000 7f37a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1921661194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac035070 con 0x55818027a2f0 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1921661194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f37ac004240 con 0x55818027a2f0 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37c5b93740 1 -- 172.21.15.5:0/1921661194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5581801ed600 con 0x55818027a2f0 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1921661194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f37ac04a070 con 0x55818027a2f0 2024-09-11T22:21:37.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37c5b93740 10 client.14967 fetch_fsmap learned FSMap version 27 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37c5b93740 1 -- 172.21.15.5:0/1921661194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558180292fb0 con 0x55818027a2f0 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1921661194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5581801ed600 con 0x55818027a2f0 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1921661194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f37ac004ab0 con 0x55818027a2f0 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37c5b93740 10 client.14967 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37c5b93740 10 client.14967 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37c5b93740 20 client.14967 populate_metadata read hostname 'smithi005' 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5581802955f0 0x558180296a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37c5b93740 4 client.14967 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.875+0000 7f37c5b93740 1 -- 172.21.15.5:0/1921661194 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581801e7b10 con 0x5581802955f0 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.876+0000 7f37c1556640 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5581802955f0 0x558180296a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.876+0000 7f37c1556640 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5581802955f0 0x558180296a60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c2ae41de7216fc5a server_cookie=a15caf1e1269ab61 in_seq=0 out_seq=0 2024-09-11T22:21:37.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.876+0000 7f37a4ff9640 10 client.14967 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1921661194 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581801e7b10 con 0x5581802955f0 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37a4ff9640 10 client.14967 handle_command_reply: tid=0 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 client.14967 shutdown 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 -- 172.21.15.5:0/1921661194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 msgr2=0x5581802816d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x5581802816d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f37ac02a930 tx=0x7f37ac0319a0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 -- 172.21.15.5:0/1921661194 shutdown_connections 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5581802955f0 0x558180296a60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558180284bb0 0x558180281c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55818027a2f0 0x5581802816d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37a4ff9640 0 client.14967 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1921661194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5581802799d0 0x558180284380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 -- 172.21.15.5:0/1921661194 >> 172.21.15.5:0/1921661194 conn(0x558180275f60 msgr2=0x55818027bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:37.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 -- 172.21.15.5:0/1921661194 shutdown_connections 2024-09-11T22:21:37.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 1 -- 172.21.15.5:0/1921661194 wait complete. 2024-09-11T22:21:37.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.877+0000 7f37c5b93740 20 client.14967 trim_cache size 0 max 16384 2024-09-11T22:21:37.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.895+0000 7f37c5b93740 1 Processor -- start 2024-09-11T22:21:37.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.895+0000 7f37c5b93740 1 -- start start 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.895+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x558180296a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.895+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55818027a2f0 0x55818027e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.895+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558180284bb0 0x55818027ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.895+0000 7f37c5b93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558180294da0 con 0x5581802799d0 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.895+0000 7f37c5b93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558180295470 con 0x558180284bb0 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.895+0000 7f37c5b93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558180295b40 con 0x55818027a2f0 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37c1556640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x558180296a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37c1556640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x558180296a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59550/0 (socket says 172.21.15.5:59550) 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37c1556640 1 -- 172.21.15.5:0/2666913133 learned_addr learned my addr 172.21.15.5:0/2666913133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37c1556640 1 -- 172.21.15.5:0/2666913133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55818027a2f0 msgr2=0x55818027e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37c1556640 1 --2- 172.21.15.5:0/2666913133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55818027a2f0 0x55818027e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37c1556640 1 -- 172.21.15.5:0/2666913133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558180284bb0 msgr2=0x55818027ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37c1556640 1 --2- 172.21.15.5:0/2666913133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558180284bb0 0x55818027ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37c1556640 1 -- 172.21.15.5:0/2666913133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558180282800 con 0x5581802799d0 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37c1556640 1 --2- 172.21.15.5:0/2666913133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x558180296a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f37b8097360 tx=0x7f37b8008ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6ec970071c96fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37a6ffd640 1 -- 172.21.15.5:0/2666913133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b809afe0 con 0x5581802799d0 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37a6ffd640 1 -- 172.21.15.5:0/2666913133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b80860c0 con 0x5581802799d0 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.896+0000 7f37a6ffd640 1 -- 172.21.15.5:0/2666913133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b809fa30 con 0x5581802799d0 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 -- 172.21.15.5:0/2666913133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 msgr2=0x558180296a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 --2- 172.21.15.5:0/2666913133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x558180296a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f37b8097360 tx=0x7f37b8008ed0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 -- 172.21.15.5:0/2666913133 shutdown_connections 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 --2- 172.21.15.5:0/2666913133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558180284bb0 0x55818027ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 --2- 172.21.15.5:0/2666913133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55818027a2f0 0x55818027e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 --2- 172.21.15.5:0/2666913133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x558180296a60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 -- 172.21.15.5:0/2666913133 >> 172.21.15.5:0/2666913133 conn(0x558180275f60 msgr2=0x55818027ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 -- 172.21.15.5:0/2666913133 shutdown_connections 2024-09-11T22:21:37.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 -- 172.21.15.5:0/2666913133 wait complete. 2024-09-11T22:21:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 -- start start 2024-09-11T22:21:37.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.897+0000 7f37c5b93740 1 Processor -- start 2024-09-11T22:21:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x55818029ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55818027a2f0 0x55818029f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c5b93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558180284bb0 0x558180298200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c5b93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5581802968e0 con 0x5581802799d0 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c5b93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558180296210 con 0x55818027a2f0 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c5b93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558180296fb0 con 0x558180284bb0 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c5b93740 10 client.0 _ll_register_callbacks cb 0x7f37ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c1556640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x55818029ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c1556640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x55818029ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59556/0 (socket says 172.21.15.5:59556) 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c1556640 1 -- 172.21.15.5:0/1273454738 learned_addr learned my addr 172.21.15.5:0/1273454738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c1556640 1 -- 172.21.15.5:0/1273454738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558180284bb0 msgr2=0x558180298200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c1556640 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558180284bb0 0x558180298200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c1556640 1 -- 172.21.15.5:0/1273454738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55818027a2f0 msgr2=0x55818029f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c1556640 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55818027a2f0 0x55818029f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c1556640 1 -- 172.21.15.5:0/1273454738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558180298a30 con 0x5581802799d0 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c1556640 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x55818029ee80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f37b8005fd0 tx=0x7f37b800a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81626096bddf3f43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:37.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1273454738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b809afe0 con 0x5581802799d0 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1273454738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f37b8097510 con 0x5581802799d0 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.898+0000 7f37c5b93740 1 -- 172.21.15.5:0/1273454738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b80860c0 con 0x5581802799d0 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1273454738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f37b809b1c0 con 0x5581802799d0 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37c5b93740 10 client.14973 fetch_fsmap learned FSMap version 27 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37c5b93740 1 -- 172.21.15.5:0/1273454738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55818029d440 con 0x5581802799d0 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1273454738 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b80860c0 con 0x5581802799d0 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1273454738 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f37b8007ca0 con 0x5581802799d0 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37c5b93740 10 client.14973 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37c5b93740 10 client.14973 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37c5b93740 20 client.14973 populate_metadata read hostname 'smithi005' 2024-09-11T22:21:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x558180303bc0 0x5581803028d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37c5b93740 4 client.14973 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.899+0000 7f37c5b93740 1 -- 172.21.15.5:0/1273454738 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "status"}) -- 0x5581801640e0 con 0x558180303bc0 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.900+0000 7f37c0d55640 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x558180303bc0 0x5581803028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.900+0000 7f37a4ff9640 10 client.14973 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.900+0000 7f37c0d55640 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x558180303bc0 0x5581803028d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3ba74377d5e0a92d server_cookie=38765b936313b05a in_seq=0 out_seq=0 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.900+0000 7f37a4ff9640 1 -- 172.21.15.5:0/1273454738 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5581801640e0 con 0x558180303bc0 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.900+0000 7f37a4ff9640 10 client.14973 handle_command_reply: tid=0 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 client.14973 shutdown 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 -- 172.21.15.5:0/1273454738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 msgr2=0x55818029ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x55818029ee80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f37b8005fd0 tx=0x7f37b800a5b0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 -- 172.21.15.5:0/1273454738 shutdown_connections 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558180284bb0 0x558180298200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x558180303bc0 0x5581803028d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55818027a2f0 0x55818029f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 --2- 172.21.15.5:0/1273454738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581802799d0 0x55818029ee80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 -- 172.21.15.5:0/1273454738 >> 172.21.15.5:0/1273454738 conn(0x558180275f60 msgr2=0x558180282d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37a4ff9640 0 client.14973 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:21:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 -- 172.21.15.5:0/1273454738 shutdown_connections 2024-09-11T22:21:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 1 -- 172.21.15.5:0/1273454738 wait complete. 2024-09-11T22:21:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c5b93740 20 client.14973 trim_cache size 0 max 16384 2024-09-11T22:21:37.906 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:21:37.906 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:21:37.906 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-11T22:21:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24452, 2024-09-11T22:21:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:21:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:21:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:21:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 69.651777760000002, 2024-09-11T22:21:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-11T22:21:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-09-11T22:21:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T22:21:37.907 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 109.252047015 2024-09-11T22:21:37.907 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:21:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c37ba640 1 -- 172.21.15.5:0/4149202738 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f37a80a2c10 msgr2=0x7f37a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c37ba640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f37a80a2c10 0x7f37a80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f37b4005fd0 tx=0x7f37b4004ab0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c37ba640 1 -- 172.21.15.5:0/4149202738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc101de0 msgr2=0x7f37bc198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.901+0000 7f37c37ba640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc101de0 0x7f37bc198430 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f37b800c890 tx=0x7f37b800cd50 comp rx=0 tx=0).stop 2024-09-11T22:21:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.902+0000 7f37c37ba640 1 -- 172.21.15.5:0/4149202738 shutdown_connections 2024-09-11T22:21:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.902+0000 7f37c37ba640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f37a80a2c10 0x7f37a80a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.902+0000 7f37c37ba640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37bc106f10 0x7f37bc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.902+0000 7f37c37ba640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37bc102790 0x7f37bc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.902+0000 7f37c37ba640 1 --2- 172.21.15.5:0/4149202738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc101de0 0x7f37bc198430 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.902+0000 7f37c37ba640 1 -- 172.21.15.5:0/4149202738 >> 172.21.15.5:0/4149202738 conn(0x7f37bc0fd8c0 msgr2=0x7f37bc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.902+0000 7f37c37ba640 1 -- 172.21.15.5:0/4149202738 shutdown_connections 2024-09-11T22:21:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:37.902+0000 7f37c37ba640 1 -- 172.21.15.5:0/4149202738 wait complete. 2024-09-11T22:21:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:37 smithi159 ceph-mon[35534]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-11T22:21:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:37 smithi172 ceph-mon[35461]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-11T22:21:38.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T22:21:38.860 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:39.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.358+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1811480644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbe4100a80 msgr2=0x7fbbe4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:39.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.358+0000 7fbbeb9d6640 1 --2- 172.21.15.5:0/1811480644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbe4100a80 0x7fbbe4100ee0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009930 tx=0x7fbbd402a420 comp rx=0 tx=0).stop 2024-09-11T22:21:39.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.358+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1811480644 shutdown_connections 2024-09-11T22:21:39.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.358+0000 7fbbeb9d6640 1 --2- 172.21.15.5:0/1811480644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbe4101420 0x7fbbe410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.358+0000 7fbbeb9d6640 1 --2- 172.21.15.5:0/1811480644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbe4100a80 0x7fbbe4100ee0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.358+0000 7fbbeb9d6640 1 --2- 172.21.15.5:0/1811480644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbbe4106a80 0x7fbbe4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.358+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1811480644 >> 172.21.15.5:0/1811480644 conn(0x7fbbe40fc7a0 msgr2=0x7fbbe40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:39.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.358+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1811480644 shutdown_connections 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.358+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1811480644 wait complete. 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbeb9d6640 1 Processor -- start 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbeb9d6640 1 -- start start 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbeb9d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbe4100a80 0x7fbbe41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbeb9d6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbbe4101420 0x7fbbe41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbeb9d6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbe4106a80 0x7fbbe419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbeb9d6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4110ea0 con 0x7fbbe4100a80 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbeb9d6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4110d20 con 0x7fbbe4101420 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbeb9d6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4111020 con 0x7fbbe4106a80 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe9772640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbe4100a80 0x7fbbe41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe9772640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbe4100a80 0x7fbbe41984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59576/0 (socket says 172.21.15.5:59576) 2024-09-11T22:21:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe9772640 1 -- 172.21.15.5:0/1893349733 learned_addr learned my addr 172.21.15.5:0/1893349733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbbe4101420 0x7fbbe41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe9f73640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbe4106a80 0x7fbbe419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe9772640 1 -- 172.21.15.5:0/1893349733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbe4106a80 msgr2=0x7fbbe419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe9772640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbe4106a80 0x7fbbe419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe9772640 1 -- 172.21.15.5:0/1893349733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbbe4101420 msgr2=0x7fbbe41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe9772640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbbe4101420 0x7fbbe41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe9772640 1 -- 172.21.15.5:0/1893349733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe419d460 con 0x7fbbe4100a80 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.359+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbbe4101420 0x7fbbe41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.360+0000 7fbbe9772640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbe4100a80 0x7fbbe41984b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c890 tx=0x7fbbe000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.360+0000 7fbbda7fc640 1 -- 172.21.15.5:0/1893349733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0004220 con 0x7fbbe4100a80 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.360+0000 7fbbda7fc640 1 -- 172.21.15.5:0/1893349733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe00043c0 con 0x7fbbe4100a80 2024-09-11T22:21:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.360+0000 7fbbda7fc640 1 -- 172.21.15.5:0/1893349733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00049e0 con 0x7fbbe4100a80 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.360+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1893349733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe419d750 con 0x7fbbe4100a80 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.360+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1893349733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe41a5030 con 0x7fbbe4100a80 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.363+0000 7fbbda7fc640 1 -- 172.21.15.5:0/1893349733 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fbbe00079b0 con 0x7fbbe4100a80 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.363+0000 7fbbeddaf740 1 Processor -- start 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.363+0000 7fbbeddaf740 1 -- start start 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.363+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0628b0 0x563d9a062c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0631d0 0x563d9a071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a0743b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d99fc9090 con 0x563d9a0628b0 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563d99f70a70 con 0x563d9a071fc0 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563d99fc5050 con 0x563d9a0631d0 2024-09-11T22:21:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9f73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0628b0 0x563d9a062c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9f73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0628b0 0x563d9a062c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59590/0 (socket says 172.21.15.5:59590) 2024-09-11T22:21:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9f73640 1 -- 172.21.15.5:0/777071628 learned_addr learned my addr 172.21.15.5:0/777071628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9772640 1 --2- 172.21.15.5:0/777071628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0631d0 0x563d9a071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbda7fc640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fbbd00a2c90 0x7fbbd00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/777071628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a0743b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9f73640 1 -- 172.21.15.5:0/777071628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0631d0 msgr2=0x563d9a071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9f73640 1 --2- 172.21.15.5:0/777071628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0631d0 0x563d9a071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9f73640 1 -- 172.21.15.5:0/777071628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 msgr2=0x563d9a0743b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9f73640 1 --2- 172.21.15.5:0/777071628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a0743b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9f73640 1 -- 172.21.15.5:0/777071628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d99f7d130 con 0x563d9a0628b0 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9772640 1 --2- 172.21.15.5:0/777071628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0631d0 0x563d9a071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbda7fc640 1 -- 172.21.15.5:0/1893349733 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fbbe00cf2c0 con 0x7fbbe4100a80 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9f73640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fbbd00a2c90 0x7fbbd00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.364+0000 7fbbe9f73640 1 --2- 172.21.15.5:0/777071628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0628b0 0x563d9a062c90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c4a0 tx=0x7fbbdc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b096c03b2ff48461 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbcf7fe640 1 -- 172.21.15.5:0/777071628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc018640 con 0x563d9a0628b0 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbcf7fe640 1 -- 172.21.15.5:0/777071628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbdc01e070 con 0x563d9a0628b0 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbcf7fe640 1 -- 172.21.15.5:0/777071628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc00a970 con 0x563d9a0628b0 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbe9f73640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fbbd00a2c90 0x7fbbd00a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc01d000 tx=0x7fbbdc017000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbeddaf740 1 -- 172.21.15.5:0/777071628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0628b0 msgr2=0x563d9a062c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/777071628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0628b0 0x563d9a062c90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c4a0 tx=0x7fbbdc00c960 comp rx=0 tx=0).stop 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbeddaf740 1 -- 172.21.15.5:0/777071628 shutdown_connections 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/777071628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a0743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/777071628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0631d0 0x563d9a071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/777071628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0628b0 0x563d9a062c90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.365+0000 7fbbeddaf740 1 -- 172.21.15.5:0/777071628 >> 172.21.15.5:0/777071628 conn(0x563d9a05ef60 msgr2=0x563d9a05f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 1 -- 172.21.15.5:0/777071628 shutdown_connections 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 1 -- 172.21.15.5:0/777071628 wait complete. 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 1 -- start start 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 1 Processor -- start 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a0628b0 0x563d9a071870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a071fc0 0x563d9a06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d9a074b90 con 0x563d9a0631d0 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563d99f83f20 con 0x563d9a0628b0 2024-09-11T22:21:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563d9a074d10 con 0x563d9a071fc0 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbeddaf740 10 client.0 _ll_register_callbacks cb 0x563d99f4d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.366+0000 7fbbe8f71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbe8f71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59606/0 (socket says 172.21.15.5:59606) 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbe8f71640 1 -- 172.21.15.5:0/3847359093 learned_addr learned my addr 172.21.15.5:0/3847359093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbe9f73640 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a071fc0 0x563d9a06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbe9772640 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a0628b0 0x563d9a071870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbe8f71640 1 -- 172.21.15.5:0/3847359093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a071fc0 msgr2=0x563d9a06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a071fc0 0x563d9a06f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbe8f71640 1 -- 172.21.15.5:0/3847359093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a0628b0 msgr2=0x563d9a071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a0628b0 0x563d9a071870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbe8f71640 1 -- 172.21.15.5:0/3847359093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d99d3a2b0 con 0x563d9a0631d0 2024-09-11T22:21:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06ebc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbbd402a940 tx=0x7fbbd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca1da5a998714ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbccff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbccff9640 1 -- 172.21.15.5:0/3847359093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4035070 con 0x563d9a0631d0 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbccff9640 1 -- 172.21.15.5:0/3847359093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fbbd4005220 con 0x563d9a0631d0 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbeddaf740 1 -- 172.21.15.5:0/3847359093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbdc01e070 con 0x563d9a0631d0 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbccff9640 1 -- 172.21.15.5:0/3847359093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbd404a070 con 0x563d9a0631d0 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbeddaf740 10 client.14985 fetch_fsmap learned FSMap version 27 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.367+0000 7fbbeddaf740 1 -- 172.21.15.5:0/3847359093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d9a0804f0 con 0x563d9a0631d0 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.368+0000 7fbbccff9640 1 -- 172.21.15.5:0/3847359093 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbdc01e070 con 0x563d9a0631d0 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.368+0000 7fbbccff9640 1 -- 172.21.15.5:0/3847359093 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbbd40035f0 con 0x563d9a0631d0 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.368+0000 7fbbeddaf740 10 client.14985 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.368+0000 7fbbeddaf740 10 client.14985 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.368+0000 7fbbeddaf740 20 client.14985 populate_metadata read hostname 'smithi005' 2024-09-11T22:21:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.368+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x563d9a082ae0 0x563d9a083f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.368+0000 7fbbeddaf740 4 client.14985 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.368+0000 7fbbeddaf740 1 -- 172.21.15.5:0/3847359093 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d99fd6600 con 0x563d9a082ae0 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.368+0000 7fbbe9772640 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x563d9a082ae0 0x563d9a083f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.369+0000 7fbbccff9640 10 client.14985 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.369+0000 7fbbe9772640 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x563d9a082ae0 0x563d9a083f50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fb6722f9f2b9de47 server_cookie=8f671bdb371241f in_seq=0 out_seq=0 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.369+0000 7fbbccff9640 1 -- 172.21.15.5:0/3847359093 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d99fd6600 con 0x563d9a082ae0 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.369+0000 7fbbccff9640 10 client.14985 handle_command_reply: tid=0 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 client.14985 shutdown 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 -- 172.21.15.5:0/3847359093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 msgr2=0x563d9a06ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06ebc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbbd402a940 tx=0x7fbbd40042f0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 -- 172.21.15.5:0/3847359093 shutdown_connections 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x563d9a082ae0 0x563d9a083f50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a071fc0 0x563d9a06f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06ebc0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbccff9640 0 client.14985 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:21:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/3847359093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a0628b0 0x563d9a071870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 -- 172.21.15.5:0/3847359093 >> 172.21.15.5:0/3847359093 conn(0x563d9a05ef60 msgr2=0x563d9a064cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 -- 172.21.15.5:0/3847359093 shutdown_connections 2024-09-11T22:21:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 1 -- 172.21.15.5:0/3847359093 wait complete. 2024-09-11T22:21:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.370+0000 7fbbeddaf740 20 client.14985 trim_cache size 0 max 16384 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbeddaf740 1 Processor -- start 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbeddaf740 1 -- start start 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0628b0 0x563d9a0831b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a06c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d9a082290 con 0x563d9a0631d0 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563d9a082960 con 0x563d9a071fc0 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563d9a083030 con 0x563d9a0628b0 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbe8f71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbe8f71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59618/0 (socket says 172.21.15.5:59618) 2024-09-11T22:21:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.388+0000 7fbbe8f71640 1 -- 172.21.15.5:0/4244653158 learned_addr learned my addr 172.21.15.5:0/4244653158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.389+0000 7fbbe9772640 1 --2- 172.21.15.5:0/4244653158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0628b0 0x563d9a0831b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.389+0000 7fbbe8f71640 1 -- 172.21.15.5:0/4244653158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0628b0 msgr2=0x563d9a0831b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.389+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/4244653158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0628b0 0x563d9a0831b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.389+0000 7fbbe8f71640 1 -- 172.21.15.5:0/4244653158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 msgr2=0x563d9a06c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.389+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/4244653158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.389+0000 7fbbe8f71640 1 -- 172.21.15.5:0/4244653158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9a05ea00 con 0x563d9a0631d0 2024-09-11T22:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.389+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/4244653158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06be20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009850 tx=0x7fbbd4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dc285088483f7f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.389+0000 7fbbceffd640 1 -- 172.21.15.5:0/4244653158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4035070 con 0x563d9a0631d0 2024-09-11T22:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.389+0000 7fbbceffd640 1 -- 172.21.15.5:0/4244653158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd40035f0 con 0x563d9a0631d0 2024-09-11T22:21:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.389+0000 7fbbceffd640 1 -- 172.21.15.5:0/4244653158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4044d50 con 0x563d9a0631d0 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 -- 172.21.15.5:0/4244653158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 msgr2=0x563d9a06be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/4244653158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06be20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009850 tx=0x7fbbd4001e70 comp rx=0 tx=0).stop 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 -- 172.21.15.5:0/4244653158 shutdown_connections 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/4244653158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a06c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/4244653158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a06be20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/4244653158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0628b0 0x563d9a0831b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 -- 172.21.15.5:0/4244653158 >> 172.21.15.5:0/4244653158 conn(0x563d9a05ef60 msgr2=0x563d9a06dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 -- 172.21.15.5:0/4244653158 shutdown_connections 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 -- 172.21.15.5:0/4244653158 wait complete. 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 -- start start 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.390+0000 7fbbeddaf740 1 Processor -- start 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0628b0 0x563d9a0861a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a0866e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbeddaf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a086c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d9a083dd0 con 0x563d9a0631d0 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563d9a083700 con 0x563d9a071fc0 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbeddaf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563d9a0844a0 con 0x563d9a0628b0 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbeddaf740 10 client.0 _ll_register_callbacks cb 0x7fbbd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe8f71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a0866e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe8f71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a0866e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59630/0 (socket says 172.21.15.5:59630) 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe8f71640 1 -- 172.21.15.5:0/2173065991 learned_addr learned my addr 172.21.15.5:0/2173065991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe9772640 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0628b0 0x563d9a0861a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe9f73640 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a086c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe8f71640 1 -- 172.21.15.5:0/2173065991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0628b0 msgr2=0x563d9a0861a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0628b0 0x563d9a0861a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe8f71640 1 -- 172.21.15.5:0/2173065991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 msgr2=0x563d9a086c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a086c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe8f71640 1 -- 172.21.15.5:0/2173065991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d9a05e810 con 0x563d9a0631d0 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe9f73640 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a086c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbe8f71640 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a0866e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4003e90 tx=0x7fbbd4003ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6014a9eb58ee09ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbccff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.391+0000 7fbbccff9640 1 -- 172.21.15.5:0/2173065991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd405c030 con 0x563d9a0631d0 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbccff9640 1 -- 172.21.15.5:0/2173065991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fbbd4005020 con 0x563d9a0631d0 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbeddaf740 1 -- 172.21.15.5:0/2173065991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd40035f0 con 0x563d9a0631d0 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbccff9640 1 -- 172.21.15.5:0/2173065991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbbd4035040 con 0x563d9a0631d0 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbeddaf740 10 client.14991 fetch_fsmap learned FSMap version 27 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbeddaf740 1 -- 172.21.15.5:0/2173065991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563d9a0ed5b0 con 0x563d9a0631d0 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbccff9640 1 -- 172.21.15.5:0/2173065991 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd40035f0 con 0x563d9a0631d0 2024-09-11T22:21:39.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbccff9640 1 -- 172.21.15.5:0/2173065991 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbbd4057070 con 0x563d9a0631d0 2024-09-11T22:21:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbeddaf740 10 client.14991 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:21:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbeddaf740 10 client.14991 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:21:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbeddaf740 20 client.14991 populate_metadata read hostname 'smithi005' 2024-09-11T22:21:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x563d9a0f1d30 0x563d9a0f0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbeddaf740 4 client.14991 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:21:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.392+0000 7fbbeddaf740 1 -- 172.21.15.5:0/2173065991 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "status"}) -- 0x563d99fd0b10 con 0x563d9a0f1d30 2024-09-11T22:21:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbe9772640 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x563d9a0f1d30 0x563d9a0f0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbccff9640 10 client.14991 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:21:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbe9772640 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x563d9a0f1d30 0x563d9a0f0a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ec85aa9ec4f945e server_cookie=3a98fc18393a27fd in_seq=0 out_seq=0 2024-09-11T22:21:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbccff9640 1 -- 172.21.15.5:0/2173065991 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563d99fd0b10 con 0x563d9a0f1d30 2024-09-11T22:21:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbccff9640 10 client.14991 handle_command_reply: tid=0 2024-09-11T22:21:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 client.14991 shutdown 2024-09-11T22:21:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 -- 172.21.15.5:0/2173065991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 msgr2=0x563d9a0866e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a0866e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4003e90 tx=0x7fbbd4003ec0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 -- 172.21.15.5:0/2173065991 shutdown_connections 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x563d9a0f1d30 0x563d9a0f0a40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d9a071fc0 0x563d9a086c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d9a0631d0 0x563d9a0866e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbccff9640 0 client.14991 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 --2- 172.21.15.5:0/2173065991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d9a0628b0 0x563d9a0861a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 -- 172.21.15.5:0/2173065991 >> 172.21.15.5:0/2173065991 conn(0x563d9a05ef60 msgr2=0x563d9a05d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 -- 172.21.15.5:0/2173065991 shutdown_connections 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.393+0000 7fbbeddaf740 1 -- 172.21.15.5:0/2173065991 wait complete. 2024-09-11T22:21:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeddaf740 20 client.14991 trim_cache size 0 max 16384 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24454, 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 72.149398476000002, 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T22:21:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 100.041874029 2024-09-11T22:21:39.401 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:21:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1893349733 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fbbd00a2c90 msgr2=0x7fbbd00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fbbd00a2c90 0x7fbbd00a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc01d000 tx=0x7fbbdc017000 comp rx=0 tx=0).stop 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1893349733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbe4100a80 msgr2=0x7fbbe41984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbe4100a80 0x7fbbe41984b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c890 tx=0x7fbbe000cd50 comp rx=0 tx=0).stop 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1893349733 shutdown_connections 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fbbd00a2c90 0x7fbbd00a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbbe4106a80 0x7fbbe419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbbe4101420 0x7fbbe41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 --2- 172.21.15.5:0/1893349733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbe4100a80 0x7fbbe41984b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1893349733 >> 172.21.15.5:0/1893349733 conn(0x7fbbe40fc7a0 msgr2=0x7fbbe4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1893349733 shutdown_connections 2024-09-11T22:21:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:39.394+0000 7fbbeb9d6640 1 -- 172.21.15.5:0/1893349733 wait complete. 2024-09-11T22:21:40.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:39 smithi005 ceph-mon[30943]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:21:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:39 smithi159 ceph-mon[35534]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:21:40.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T22:21:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:39 smithi172 ceph-mon[35461]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:21:40.348 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:40.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.843+0000 7fdcb9281640 1 -- 172.21.15.5:0/4000708873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4108300 msgr2=0x7fdcb410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:40.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.843+0000 7fdcb9281640 1 --2- 172.21.15.5:0/4000708873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4108300 0x7fdcb410ec70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009cd0 tx=0x7fdcac02a8d0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.843+0000 7fdcb9281640 1 -- 172.21.15.5:0/4000708873 shutdown_connections 2024-09-11T22:21:40.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.843+0000 7fdcb9281640 1 --2- 172.21.15.5:0/4000708873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4108300 0x7fdcb410ec70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.843+0000 7fdcb9281640 1 --2- 172.21.15.5:0/4000708873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb4103780 0x7fdcb4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.843+0000 7fdcb9281640 1 --2- 172.21.15.5:0/4000708873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb4102dd0 0x7fdcb41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.843+0000 7fdcb9281640 1 -- 172.21.15.5:0/4000708873 >> 172.21.15.5:0/4000708873 conn(0x7fdcb40fc7d0 msgr2=0x7fdcb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.843+0000 7fdcb9281640 1 -- 172.21.15.5:0/4000708873 shutdown_connections 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.843+0000 7fdcb9281640 1 -- 172.21.15.5:0/4000708873 wait complete. 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcb9281640 1 Processor -- start 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcb9281640 1 -- start start 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcb9281640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb4102dd0 0x7fdcb41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcb9281640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4103780 0x7fdcb4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcb9281640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb4108300 0x7fdcb419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcb9281640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb41120d0 con 0x7fdcb4103780 2024-09-11T22:21:40.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcb9281640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4111f50 con 0x7fdcb4108300 2024-09-11T22:21:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcb9281640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb4112250 con 0x7fdcb4102dd0 2024-09-11T22:21:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4103780 0x7fdcb4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4103780 0x7fdcb4198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59652/0 (socket says 172.21.15.5:59652) 2024-09-11T22:21:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcabfff640 1 -- 172.21.15.5:0/401283099 learned_addr learned my addr 172.21.15.5:0/401283099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcb37fe640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb4108300 0x7fdcb419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.844+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb4102dd0 0x7fdcb41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcabfff640 1 -- 172.21.15.5:0/401283099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb4102dd0 msgr2=0x7fdcb41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcabfff640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb4102dd0 0x7fdcb41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcabfff640 1 -- 172.21.15.5:0/401283099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb4108300 msgr2=0x7fdcb419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:40.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcabfff640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb4108300 0x7fdcb419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcabfff640 1 -- 172.21.15.5:0/401283099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb419d4a0 con 0x7fdcb4103780 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb4102dd0 0x7fdcb41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcabfff640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4103780 0x7fdcb4198a30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c970 tx=0x7fdc9c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcb0ff9640 1 -- 172.21.15.5:0/401283099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c007bb0 con 0x7fdcb4103780 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcb0ff9640 1 -- 172.21.15.5:0/401283099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c007d50 con 0x7fdcb4103780 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcb9281640 1 -- 172.21.15.5:0/401283099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb419d790 con 0x7fdcb4103780 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcb0ff9640 1 -- 172.21.15.5:0/401283099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00f450 con 0x7fdcb4103780 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.845+0000 7fdcb9281640 1 -- 172.21.15.5:0/401283099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb41a5070 con 0x7fdcb4103780 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.847+0000 7fdcb0ff9640 1 -- 172.21.15.5:0/401283099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdc9c004f40 con 0x7fdcb4103780 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.847+0000 7fdcbb65a740 1 Processor -- start 2024-09-11T22:21:40.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcbb65a740 1 -- start start 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b185c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b194a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b1973b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e7b0ec090 con 0x559e7b185890 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e7b093a70 con 0x559e7b194fc0 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559e7b0e8050 con 0x559e7b1861b0 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcb37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b185c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcb37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b185c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59664/0 (socket says 172.21.15.5:59664) 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcb37fe640 1 -- 172.21.15.5:0/2652179279 learned_addr learned my addr 172.21.15.5:0/2652179279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcabfff640 1 --2- 172.21.15.5:0/2652179279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b1973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.848+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/2652179279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b194a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdcb37fe640 1 -- 172.21.15.5:0/2652179279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 msgr2=0x559e7b194a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdcb37fe640 1 --2- 172.21.15.5:0/2652179279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b194a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdcb37fe640 1 -- 172.21.15.5:0/2652179279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 msgr2=0x559e7b1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdcb37fe640 1 --2- 172.21.15.5:0/2652179279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b1973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdcb37fe640 1 -- 172.21.15.5:0/2652179279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e7b0a0130 con 0x559e7b185890 2024-09-11T22:21:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdcabfff640 1 --2- 172.21.15.5:0/2652179279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdcb0ff9640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdca00a2c90 0x7fdca00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdca00a2c90 0x7fdca00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdcb37fe640 1 --2- 172.21.15.5:0/2652179279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b185c70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009cd0 tx=0x7fdcac02f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dcfea20630450c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:40.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdcb0ff9640 1 -- 172.21.15.5:0/401283099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fdc9c0cee80 con 0x7fdcb4103780 2024-09-11T22:21:40.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdca97fa640 1 -- 172.21.15.5:0/2652179279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac031df0 con 0x559e7b185890 2024-09-11T22:21:40.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdca97fa640 1 -- 172.21.15.5:0/2652179279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcac036030 con 0x559e7b185890 2024-09-11T22:21:40.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.849+0000 7fdca97fa640 1 -- 172.21.15.5:0/2652179279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac0043a0 con 0x559e7b185890 2024-09-11T22:21:40.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcbb65a740 1 -- 172.21.15.5:0/2652179279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 msgr2=0x559e7b185c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:40.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/2652179279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b185c70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdcac009cd0 tx=0x7fdcac02f490 comp rx=0 tx=0).stop 2024-09-11T22:21:40.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdca00a2c90 0x7fdca00a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdca4005fd0 tx=0x7fdca4005f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:40.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcbb65a740 1 -- 172.21.15.5:0/2652179279 shutdown_connections 2024-09-11T22:21:40.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/2652179279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b1973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/2652179279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b194a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/2652179279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b185c70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcbb65a740 1 -- 172.21.15.5:0/2652179279 >> 172.21.15.5:0/2652179279 conn(0x559e7b181f60 msgr2=0x559e7b182340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcbb65a740 1 -- 172.21.15.5:0/2652179279 shutdown_connections 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcbb65a740 1 -- 172.21.15.5:0/2652179279 wait complete. 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.850+0000 7fdcbb65a740 1 -- start start 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcbb65a740 1 Processor -- start 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b194830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b191b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b1920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e7b197b30 con 0x559e7b185890 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e7b0a6f20 con 0x559e7b194fc0 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559e7b197cb0 con 0x559e7b1861b0 2024-09-11T22:21:40.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcbb65a740 10 client.0 _ll_register_callbacks cb 0x559e7b070030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b194830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59676/0 (socket says 172.21.15.5:59676) 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb2ffd640 1 -- 172.21.15.5:0/3303046247 learned_addr learned my addr 172.21.15.5:0/3303046247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcabfff640 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b191b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb37fe640 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b1920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb2ffd640 1 -- 172.21.15.5:0/3303046247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 msgr2=0x559e7b191b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b191b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb2ffd640 1 -- 172.21.15.5:0/3303046247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 msgr2=0x559e7b1920c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b1920c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb2ffd640 1 -- 172.21.15.5:0/3303046247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e7ae5d2b0 con 0x559e7b185890 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcabfff640 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b191b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:40.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb37fe640 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b1920c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b194830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdca4011e10 tx=0x7fdca40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e9de83b07b555a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.851+0000 7fdc9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/3303046247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca401e420 con 0x559e7b185890 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/3303046247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fdca401e5c0 con 0x559e7b185890 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdcbb65a740 1 -- 172.21.15.5:0/3303046247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcac036030 con 0x559e7b185890 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/3303046247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdca401e7c0 con 0x559e7b185890 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdcbb65a740 10 client.15003 fetch_fsmap learned FSMap version 27 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdcbb65a740 1 -- 172.21.15.5:0/3303046247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e7b1a34c0 con 0x559e7b185890 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/3303046247 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcac036030 con 0x559e7b185890 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/3303046247 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdca4031a20 con 0x559e7b185890 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdcbb65a740 10 client.15003 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:21:40.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdcbb65a740 10 client.15003 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:21:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdcbb65a740 20 client.15003 populate_metadata read hostname 'smithi005' 2024-09-11T22:21:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559e7b1a5b00 0x559e7b1a6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdcbb65a740 4 client.15003 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:21:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.852+0000 7fdcbb65a740 1 -- 172.21.15.5:0/3303046247 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e7b0f9600 con 0x559e7b1a5b00 2024-09-11T22:21:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.853+0000 7fdcabfff640 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559e7b1a5b00 0x559e7b1a6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.853+0000 7fdcabfff640 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559e7b1a5b00 0x559e7b1a6f70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1f9fc32de54ca16 server_cookie=24bd7fb81722bc2c in_seq=0 out_seq=0 2024-09-11T22:21:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.853+0000 7fdc9a7fc640 10 client.15003 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:21:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/3303046247 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e7b0f9600 con 0x559e7b1a5b00 2024-09-11T22:21:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdc9a7fc640 10 client.15003 handle_command_reply: tid=0 2024-09-11T22:21:40.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 client.15003 shutdown 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 -- 172.21.15.5:0/3303046247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 msgr2=0x559e7b194830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b194830 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdca4011e10 tx=0x7fdca40046c0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 -- 172.21.15.5:0/3303046247 shutdown_connections 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559e7b1a5b00 0x559e7b1a6f70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b1920c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b191b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdc9a7fc640 0 client.15003 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/3303046247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b194830 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 -- 172.21.15.5:0/3303046247 >> 172.21.15.5:0/3303046247 conn(0x559e7b181f60 msgr2=0x559e7b187cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 -- 172.21.15.5:0/3303046247 shutdown_connections 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 1 -- 172.21.15.5:0/3303046247 wait complete. 2024-09-11T22:21:40.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.854+0000 7fdcbb65a740 20 client.15003 trim_cache size 0 max 16384 2024-09-11T22:21:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.872+0000 7fdcbb65a740 1 Processor -- start 2024-09-11T22:21:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.872+0000 7fdcbb65a740 1 -- start start 2024-09-11T22:21:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.872+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b1a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.872+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b1920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.872+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b18ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.872+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e7b1a52b0 con 0x559e7b185890 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.872+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e7b1a5980 con 0x559e7b194fc0 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.872+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559e7b1a6050 con 0x559e7b1861b0 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b1a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcb2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b1a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59678/0 (socket says 172.21.15.5:59678) 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcb2ffd640 1 -- 172.21.15.5:0/907772126 learned_addr learned my addr 172.21.15.5:0/907772126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcb37fe640 1 --2- 172.21.15.5:0/907772126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b18ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcabfff640 1 --2- 172.21.15.5:0/907772126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b1920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcb2ffd640 1 -- 172.21.15.5:0/907772126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 msgr2=0x559e7b1920d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/907772126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b1920d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcb2ffd640 1 -- 172.21.15.5:0/907772126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 msgr2=0x559e7b18ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/907772126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b18ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcb2ffd640 1 -- 172.21.15.5:0/907772126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e7b185fe0 con 0x559e7b185890 2024-09-11T22:21:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/907772126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b1a5430 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdca4005d50 tx=0x7fdca4010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42abb918255c13ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdca8ff9640 1 -- 172.21.15.5:0/907772126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca4036040 con 0x559e7b185890 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdca8ff9640 1 -- 172.21.15.5:0/907772126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca401ee40 con 0x559e7b185890 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.873+0000 7fdca8ff9640 1 -- 172.21.15.5:0/907772126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca40317e0 con 0x559e7b185890 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.874+0000 7fdcbb65a740 1 -- 172.21.15.5:0/907772126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 msgr2=0x559e7b1a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.874+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/907772126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b1a5430 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdca4005d50 tx=0x7fdca4010cb0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.874+0000 7fdcbb65a740 1 -- 172.21.15.5:0/907772126 shutdown_connections 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.874+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/907772126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b194fc0 0x559e7b18ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.874+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/907772126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b1861b0 0x559e7b1920d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.874+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/907772126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b185890 0x559e7b1a5430 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.874+0000 7fdcbb65a740 1 -- 172.21.15.5:0/907772126 >> 172.21.15.5:0/907772126 conn(0x559e7b181f60 msgr2=0x559e7b190450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.874+0000 7fdcbb65a740 1 -- 172.21.15.5:0/907772126 shutdown_connections 2024-09-11T22:21:40.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.874+0000 7fdcbb65a740 1 -- 172.21.15.5:0/907772126 wait complete. 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.874+0000 7fdcbb65a740 1 -- start start 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcbb65a740 1 Processor -- start 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b185890 0x559e7b1ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b1861b0 0x559e7b1ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcbb65a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b194fc0 0x559e7b1a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e7b1a6df0 con 0x559e7b1861b0 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e7b1a6720 con 0x559e7b185890 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcbb65a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559e7b1a74c0 con 0x559e7b194fc0 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcbb65a740 10 client.0 _ll_register_callbacks cb 0x7fdca400c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b1861b0 0x559e7b1ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b1861b0 0x559e7b1ad5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59694/0 (socket says 172.21.15.5:59694) 2024-09-11T22:21:40.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcabfff640 1 -- 172.21.15.5:0/4274125923 learned_addr learned my addr 172.21.15.5:0/4274125923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b185890 0x559e7b1ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcb37fe640 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b194fc0 0x559e7b1a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcabfff640 1 -- 172.21.15.5:0/4274125923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b194fc0 msgr2=0x559e7b1a8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcabfff640 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b194fc0 0x559e7b1a8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcabfff640 1 -- 172.21.15.5:0/4274125923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b185890 msgr2=0x559e7b1ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcabfff640 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b185890 0x559e7b1ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcabfff640 1 -- 172.21.15.5:0/4274125923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e7b1a8d10 con 0x559e7b1861b0 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcb37fe640 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b194fc0 0x559e7b1a8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b185890 0x559e7b1ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdcabfff640 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b1861b0 0x559e7b1ad5e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0cc7e0 tx=0x7fdc9c0d12d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b863cd9adf128a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.875+0000 7fdc9a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/4274125923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c01d040 con 0x559e7b1861b0 2024-09-11T22:21:40.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/4274125923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fdc9c0cca00 con 0x559e7b1861b0 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdcbb65a740 1 -- 172.21.15.5:0/4274125923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e7b0f3b10 con 0x559e7b1861b0 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/4274125923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc9c09bc00 con 0x559e7b1861b0 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdcbb65a740 10 client.15009 fetch_fsmap learned FSMap version 27 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdcbb65a740 1 -- 172.21.15.5:0/4274125923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559e7b1ac840 con 0x559e7b1861b0 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/4274125923 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559e7b0f3b10 con 0x559e7b1861b0 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/4274125923 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdc9c005230 con 0x559e7b1861b0 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdcbb65a740 10 client.15009 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdcbb65a740 10 client.15009 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdcbb65a740 20 client.15009 populate_metadata read hostname 'smithi005' 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559e7b1a5b00 0x559e7b213410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdcbb65a740 4 client.15009 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdcbb65a740 1 -- 172.21.15.5:0/4274125923 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "status"}) -- 0x559e7b0700e0 con 0x559e7b1a5b00 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.876+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559e7b1a5b00 0x559e7b213410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:40.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdcb2ffd640 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559e7b1a5b00 0x559e7b213410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67e3d5651e101101 server_cookie=bf73681fe5a74ad2 in_seq=0 out_seq=0 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdc9a7fc640 10 client.15009 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdc9a7fc640 1 -- 172.21.15.5:0/4274125923 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559e7b0700e0 con 0x559e7b1a5b00 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdc9a7fc640 10 client.15009 handle_command_reply: tid=0 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdcbb65a740 1 client.15009 shutdown 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdcbb65a740 1 -- 172.21.15.5:0/4274125923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b1861b0 msgr2=0x559e7b1ad5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b1861b0 0x559e7b1ad5e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0cc7e0 tx=0x7fdc9c0d12d0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdcbb65a740 1 -- 172.21.15.5:0/4274125923 shutdown_connections 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559e7b1a5b00 0x559e7b213410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e7b194fc0 0x559e7b1a8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e7b1861b0 0x559e7b1ad5e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdc9a7fc640 0 client.15009 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdcbb65a740 1 --2- 172.21.15.5:0/4274125923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e7b185890 0x559e7b1ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.877+0000 7fdcbb65a740 1 -- 172.21.15.5:0/4274125923 >> 172.21.15.5:0/4274125923 conn(0x559e7b181f60 msgr2=0x559e7b18ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcbb65a740 1 -- 172.21.15.5:0/4274125923 shutdown_connections 2024-09-11T22:21:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcbb65a740 1 -- 172.21.15.5:0/4274125923 wait complete. 2024-09-11T22:21:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcbb65a740 20 client.15009 trim_cache size 0 max 16384 2024-09-11T22:21:40.893 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:21:40.893 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:21:40.893 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-11T22:21:40.893 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24473, 2024-09-11T22:21:40.894 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:21:40.894 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:21:40.894 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:21:40.894 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 74.647524364999995, 2024-09-11T22:21:40.894 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-11T22:21:40.894 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-09-11T22:21:40.894 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T22:21:40.894 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 104.12204388000001 2024-09-11T22:21:40.894 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:21:40.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 -- 172.21.15.5:0/401283099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdca00a2c90 msgr2=0x7fdca00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:40.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdca00a2c90 0x7fdca00a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdca4005fd0 tx=0x7fdca4005f40 comp rx=0 tx=0).stop 2024-09-11T22:21:40.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 -- 172.21.15.5:0/401283099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4103780 msgr2=0x7fdcb4198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:40.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4103780 0x7fdcb4198a30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c970 tx=0x7fdc9c00ce30 comp rx=0 tx=0).stop 2024-09-11T22:21:40.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 -- 172.21.15.5:0/401283099 shutdown_connections 2024-09-11T22:21:40.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fdca00a2c90 0x7fdca00a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb4108300 0x7fdcb419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4103780 0x7fdcb4198a30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 --2- 172.21.15.5:0/401283099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcb4102dd0 0x7fdcb41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 -- 172.21.15.5:0/401283099 >> 172.21.15.5:0/401283099 conn(0x7fdcb40fc7d0 msgr2=0x7fdcb4105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 -- 172.21.15.5:0/401283099 shutdown_connections 2024-09-11T22:21:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:40.878+0000 7fdcb9281640 1 -- 172.21.15.5:0/401283099 wait complete. 2024-09-11T22:21:41.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T22:21:41.872 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:41.993 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:41 smithi005 ceph-mon[30943]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:21:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:41 smithi159 ceph-mon[35534]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:21:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:41 smithi172 ceph-mon[35461]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:21:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.381+0000 7ff358e9f640 1 -- 172.21.15.5:0/3908407856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3541082a0 msgr2=0x7ff35410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.381+0000 7ff358e9f640 1 --2- 172.21.15.5:0/3908407856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3541082a0 0x7ff35410ec40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009930 tx=0x7ff34c02a420 comp rx=0 tx=0).stop 2024-09-11T22:21:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.381+0000 7ff358e9f640 1 -- 172.21.15.5:0/3908407856 shutdown_connections 2024-09-11T22:21:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.381+0000 7ff358e9f640 1 --2- 172.21.15.5:0/3908407856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3541082a0 0x7ff35410ec40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.381+0000 7ff358e9f640 1 --2- 172.21.15.5:0/3908407856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3541036e0 0x7ff354107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.381+0000 7ff358e9f640 1 --2- 172.21.15.5:0/3908407856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff354102d30 0x7ff354103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.381+0000 7ff358e9f640 1 -- 172.21.15.5:0/3908407856 >> 172.21.15.5:0/3908407856 conn(0x7ff3540fc7d0 msgr2=0x7ff3540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.381+0000 7ff358e9f640 1 -- 172.21.15.5:0/3908407856 shutdown_connections 2024-09-11T22:21:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.381+0000 7ff358e9f640 1 -- 172.21.15.5:0/3908407856 wait complete. 2024-09-11T22:21:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff358e9f640 1 Processor -- start 2024-09-11T22:21:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff358e9f640 1 -- start start 2024-09-11T22:21:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff358e9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354102d30 0x7ff354198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff358e9f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3541036e0 0x7ff354198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff358e9f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3541082a0 0x7ff35419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff358e9f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3541120a0 con 0x7ff354102d30 2024-09-11T22:21:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff358e9f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff354111f20 con 0x7ff3541082a0 2024-09-11T22:21:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff358e9f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff354112220 con 0x7ff3541036e0 2024-09-11T22:21:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff35259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354102d30 0x7ff354198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff35259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354102d30 0x7ff354198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54574/0 (socket says 172.21.15.5:54574) 2024-09-11T22:21:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff35259c640 1 -- 172.21.15.5:0/3429238254 learned_addr learned my addr 172.21.15.5:0/3429238254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff351d9b640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3541036e0 0x7ff354198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff35259c640 1 -- 172.21.15.5:0/3429238254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3541036e0 msgr2=0x7ff354198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff35259c640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3541036e0 0x7ff354198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff352d9d640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3541082a0 0x7ff35419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff35259c640 1 -- 172.21.15.5:0/3429238254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3541082a0 msgr2=0x7ff35419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff35259c640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3541082a0 0x7ff35419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff35259c640 1 -- 172.21.15.5:0/3429238254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35419d510 con 0x7ff354102d30 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff351d9b640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3541036e0 0x7ff354198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.382+0000 7ff352d9d640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3541082a0 0x7ff35419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.383+0000 7ff35259c640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354102d30 0x7ff354198560 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff34400cce0 tx=0x7ff34400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.383+0000 7ff34b7fe640 1 -- 172.21.15.5:0/3429238254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344004b20 con 0x7ff354102d30 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.383+0000 7ff34b7fe640 1 -- 172.21.15.5:0/3429238254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff344004cc0 con 0x7ff354102d30 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.383+0000 7ff358e9f640 1 -- 172.21.15.5:0/3429238254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35419d800 con 0x7ff354102d30 2024-09-11T22:21:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.383+0000 7ff34b7fe640 1 -- 172.21.15.5:0/3429238254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34400f7d0 con 0x7ff354102d30 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.383+0000 7ff358e9f640 1 -- 172.21.15.5:0/3429238254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3541a50e0 con 0x7ff354102d30 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.385+0000 7ff34b7fe640 1 -- 172.21.15.5:0/3429238254 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff34400f970 con 0x7ff354102d30 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.385+0000 7ff35b278740 1 Processor -- start 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.385+0000 7ff35b278740 1 -- start start 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff35b278740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1afdddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff35b278740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afde2f0 0x55ca1afe8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff35b278740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 0x55ca1afeafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff35b278740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca1af44090 con 0x55ca1afdd9d0 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff35b278740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ca1aeeba70 con 0x55ca1afde2f0 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff35b278740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ca1af40050 con 0x55ca1afe8bb0 2024-09-11T22:21:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff352d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1afdddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff352d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1afdddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54578/0 (socket says 172.21.15.5:54578) 2024-09-11T22:21:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff352d9d640 1 -- 172.21.15.5:0/670265131 learned_addr learned my addr 172.21.15.5:0/670265131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff35259c640 1 --2- 172.21.15.5:0/670265131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afde2f0 0x55ca1afe8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff351d9b640 1 --2- 172.21.15.5:0/670265131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 0x55ca1afeafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff34b7fe640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff3400a2c90 0x7ff3400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff352d9d640 1 -- 172.21.15.5:0/670265131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 msgr2=0x55ca1afeafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff352d9d640 1 --2- 172.21.15.5:0/670265131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 0x55ca1afeafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff352d9d640 1 -- 172.21.15.5:0/670265131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afde2f0 msgr2=0x55ca1afe8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff352d9d640 1 --2- 172.21.15.5:0/670265131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afde2f0 0x55ca1afe8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff352d9d640 1 -- 172.21.15.5:0/670265131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca1aef8130 con 0x55ca1afdd9d0 2024-09-11T22:21:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff34b7fe640 1 -- 172.21.15.5:0/3429238254 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff3440cf2f0 con 0x7ff354102d30 2024-09-11T22:21:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.386+0000 7ff352d9d640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff3400a2c90 0x7ff3400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.387+0000 7ff352d9d640 1 --2- 172.21.15.5:0/670265131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1afdddb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff34c005c40 tx=0x7ff34c032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1e8e64afb43b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.387+0000 7ff348ff9640 1 -- 172.21.15.5:0/670265131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c035070 con 0x55ca1afdd9d0 2024-09-11T22:21:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.387+0000 7ff348ff9640 1 -- 172.21.15.5:0/670265131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff34c0046c0 con 0x55ca1afdd9d0 2024-09-11T22:21:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.387+0000 7ff348ff9640 1 -- 172.21.15.5:0/670265131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c003850 con 0x55ca1afdd9d0 2024-09-11T22:21:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 -- 172.21.15.5:0/670265131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 msgr2=0x55ca1afdddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff352d9d640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff3400a2c90 0x7ff3400a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff34c048000 tx=0x7ff34c045000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:42.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 --2- 172.21.15.5:0/670265131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1afdddb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff34c005c40 tx=0x7ff34c032d60 comp rx=0 tx=0).stop 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 -- 172.21.15.5:0/670265131 shutdown_connections 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 --2- 172.21.15.5:0/670265131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 0x55ca1afeafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 --2- 172.21.15.5:0/670265131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afde2f0 0x55ca1afe8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 --2- 172.21.15.5:0/670265131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1afdddb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 -- 172.21.15.5:0/670265131 >> 172.21.15.5:0/670265131 conn(0x55ca1afd9f60 msgr2=0x55ca1afda340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 -- 172.21.15.5:0/670265131 shutdown_connections 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 -- 172.21.15.5:0/670265131 wait complete. 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 -- start start 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 Processor -- start 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 0x55ca1afe8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afde2f0 0x55ca1afe57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afe8bb0 0x55ca1afe5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca1afeb780 con 0x55ca1afe8bb0 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ca1aefef20 con 0x55ca1afdd9d0 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ca1afeb900 con 0x55ca1afde2f0 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff35b278740 10 client.0 _ll_register_callbacks cb 0x55ca1aec8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.388+0000 7ff352d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afe8bb0 0x55ca1afe5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff352d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afe8bb0 0x55ca1afe5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54590/0 (socket says 172.21.15.5:54590) 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff352d9d640 1 -- 172.21.15.5:0/3670689367 learned_addr learned my addr 172.21.15.5:0/3670689367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff351d9b640 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afde2f0 0x55ca1afe57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff35259c640 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 0x55ca1afe8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff352d9d640 1 -- 172.21.15.5:0/3670689367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afde2f0 msgr2=0x55ca1afe57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff352d9d640 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afde2f0 0x55ca1afe57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff352d9d640 1 -- 172.21.15.5:0/3670689367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 msgr2=0x55ca1afe8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff352d9d640 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 0x55ca1afe8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff352d9d640 1 -- 172.21.15.5:0/3670689367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca1acb52b0 con 0x55ca1afe8bb0 2024-09-11T22:21:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff35259c640 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 0x55ca1afe8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff352d9d640 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afe8bb0 0x55ca1afe5d00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0056c0 tx=0x7ff34c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db48790f78e95e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:21:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff339ffb640 1 -- 172.21.15.5:0/3670689367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c057070 con 0x55ca1afe8bb0 2024-09-11T22:21:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff339ffb640 1 -- 172.21.15.5:0/3670689367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff34c004a70 con 0x55ca1afe8bb0 2024-09-11T22:21:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.389+0000 7ff35b278740 1 -- 172.21.15.5:0/3670689367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff34c0046c0 con 0x55ca1afe8bb0 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff339ffb640 1 -- 172.21.15.5:0/3670689367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff34c063050 con 0x55ca1afe8bb0 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff35b278740 10 client.15021 fetch_fsmap learned FSMap version 27 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff35b278740 1 -- 172.21.15.5:0/3670689367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca1aff70d0 con 0x55ca1afe8bb0 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff339ffb640 1 -- 172.21.15.5:0/3670689367 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff34c0046c0 con 0x55ca1afe8bb0 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff339ffb640 1 -- 172.21.15.5:0/3670689367 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff34c03d480 con 0x55ca1afe8bb0 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff35b278740 10 client.15021 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff35b278740 10 client.15021 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff35b278740 20 client.15021 populate_metadata read hostname 'smithi005' 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff35b278740 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ca1aff9290 0x55ca1affa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff35b278740 4 client.15021 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff35b278740 1 -- 172.21.15.5:0/3670689367 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca1af51600 con 0x55ca1aff9290 2024-09-11T22:21:42.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.390+0000 7ff35259c640 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ca1aff9290 0x55ca1affa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.391+0000 7ff339ffb640 10 client.15021 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:21:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.391+0000 7ff35259c640 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ca1aff9290 0x55ca1affa700 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7585ade0e0a026d5 server_cookie=bb11364c61c5a4b0 in_seq=0 out_seq=0 2024-09-11T22:21:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.391+0000 7ff339ffb640 1 -- 172.21.15.5:0/3670689367 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca1af51600 con 0x55ca1aff9290 2024-09-11T22:21:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.391+0000 7ff339ffb640 10 client.15021 handle_command_reply: tid=0 2024-09-11T22:21:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 client.15021 shutdown 2024-09-11T22:21:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 -- 172.21.15.5:0/3670689367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afe8bb0 msgr2=0x55ca1afe5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afe8bb0 0x55ca1afe5d00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0056c0 tx=0x7ff34c009260 comp rx=0 tx=0).stop 2024-09-11T22:21:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 -- 172.21.15.5:0/3670689367 shutdown_connections 2024-09-11T22:21:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ca1aff9290 0x55ca1affa700 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afe8bb0 0x55ca1afe5d00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afde2f0 0x55ca1afe57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 --2- 172.21.15.5:0/3670689367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 0x55ca1afe8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff339ffb640 0 client.15021 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:21:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 -- 172.21.15.5:0/3670689367 >> 172.21.15.5:0/3670689367 conn(0x55ca1afd9f60 msgr2=0x55ca1afdfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 -- 172.21.15.5:0/3670689367 shutdown_connections 2024-09-11T22:21:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 1 -- 172.21.15.5:0/3670689367 wait complete. 2024-09-11T22:21:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.392+0000 7ff35b278740 20 client.15021 trim_cache size 0 max 16384 2024-09-11T22:21:42.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35b278740 1 Processor -- start 2024-09-11T22:21:42.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35b278740 1 -- start start 2024-09-11T22:21:42.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35b278740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1affa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35b278740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afde2f0 0x55ca1afe7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35b278740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afe8bb0 0x55ca1b002dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35b278740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca1aff8a40 con 0x55ca1afdd9d0 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35b278740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ca1aff9110 con 0x55ca1afe8bb0 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35b278740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ca1aff97e0 con 0x55ca1afde2f0 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1affa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1affa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54604/0 (socket says 172.21.15.5:54604) 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35259c640 1 -- 172.21.15.5:0/1112952791 learned_addr learned my addr 172.21.15.5:0/1112952791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35259c640 1 -- 172.21.15.5:0/1112952791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afde2f0 msgr2=0x55ca1afe7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff351d9b640 1 --2- 172.21.15.5:0/1112952791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afde2f0 0x55ca1afe7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35259c640 1 --2- 172.21.15.5:0/1112952791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afde2f0 0x55ca1afe7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35259c640 1 -- 172.21.15.5:0/1112952791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afe8bb0 msgr2=0x55ca1b002dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:42.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35259c640 1 --2- 172.21.15.5:0/1112952791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afe8bb0 0x55ca1b002dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.410+0000 7ff35259c640 1 -- 172.21.15.5:0/1112952791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca1afde120 con 0x55ca1afdd9d0 2024-09-11T22:21:42.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff35259c640 1 --2- 172.21.15.5:0/1112952791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1affa700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff34400ab20 tx=0x7ff344086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5941de1c81f175e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:42.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff33bfff640 1 -- 172.21.15.5:0/1112952791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34401d040 con 0x55ca1afdd9d0 2024-09-11T22:21:42.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff33bfff640 1 -- 172.21.15.5:0/1112952791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff34409b9d0 con 0x55ca1afdd9d0 2024-09-11T22:21:42.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff33bfff640 1 -- 172.21.15.5:0/1112952791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344098040 con 0x55ca1afdd9d0 2024-09-11T22:21:42.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff35b278740 1 -- 172.21.15.5:0/1112952791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 msgr2=0x55ca1affa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:42.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff35b278740 1 --2- 172.21.15.5:0/1112952791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1affa700 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff34400ab20 tx=0x7ff344086040 comp rx=0 tx=0).stop 2024-09-11T22:21:42.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff35b278740 1 -- 172.21.15.5:0/1112952791 shutdown_connections 2024-09-11T22:21:42.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff35b278740 1 --2- 172.21.15.5:0/1112952791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afe8bb0 0x55ca1b002dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff35b278740 1 --2- 172.21.15.5:0/1112952791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afde2f0 0x55ca1afe7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff35b278740 1 --2- 172.21.15.5:0/1112952791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afdd9d0 0x55ca1affa700 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff35b278740 1 -- 172.21.15.5:0/1112952791 >> 172.21.15.5:0/1112952791 conn(0x55ca1afd9f60 msgr2=0x55ca1afe6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff35b278740 1 -- 172.21.15.5:0/1112952791 shutdown_connections 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.411+0000 7ff35b278740 1 -- 172.21.15.5:0/1112952791 wait complete. 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff35b278740 1 -- start start 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff35b278740 1 Processor -- start 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff35b278740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 0x55ca1afe2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff35b278740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afde2f0 0x55ca1afe2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff35b278740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 0x55ca1b05edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff35b278740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca1affa580 con 0x55ca1afde2f0 2024-09-11T22:21:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff35b278740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ca1aff9eb0 con 0x55ca1afdd9d0 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff35b278740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ca1affac50 con 0x55ca1afe8bb0 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff35b278740 10 client.0 _ll_register_callbacks cb 0x7ff344097cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff351d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afde2f0 0x55ca1afe2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff35259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 0x55ca1afe2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff351d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afde2f0 0x55ca1afe2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54608/0 (socket says 172.21.15.5:54608) 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff351d9b640 1 -- 172.21.15.5:0/1402182204 learned_addr learned my addr 172.21.15.5:0/1402182204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.412+0000 7ff352d9d640 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 0x55ca1b05edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff351d9b640 1 -- 172.21.15.5:0/1402182204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 msgr2=0x55ca1b05edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff351d9b640 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 0x55ca1b05edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff351d9b640 1 -- 172.21.15.5:0/1402182204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 msgr2=0x55ca1afe2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:21:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff351d9b640 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 0x55ca1afe2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff351d9b640 1 -- 172.21.15.5:0/1402182204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca1b05f470 con 0x55ca1afde2f0 2024-09-11T22:21:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff35259c640 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 0x55ca1afe2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:21:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff352d9d640 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 0x55ca1b05edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff351d9b640 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afde2f0 0x55ca1afe2dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff35406a370 tx=0x7ff33c0138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70fbd7a18858ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:21:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff339ffb640 1 -- 172.21.15.5:0/1402182204 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c021060 con 0x55ca1afde2f0 2024-09-11T22:21:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff339ffb640 1 -- 172.21.15.5:0/1402182204 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff33c011650 con 0x55ca1afde2f0 2024-09-11T22:21:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff35b278740 1 -- 172.21.15.5:0/1402182204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff34409b9d0 con 0x55ca1afde2f0 2024-09-11T22:21:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff339ffb640 1 -- 172.21.15.5:0/1402182204 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff33c011890 con 0x55ca1afde2f0 2024-09-11T22:21:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff35b278740 10 client.15027 fetch_fsmap learned FSMap version 27 2024-09-11T22:21:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff35b278740 1 -- 172.21.15.5:0/1402182204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ca1b067a60 con 0x55ca1afde2f0 2024-09-11T22:21:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.413+0000 7ff339ffb640 1 -- 172.21.15.5:0/1402182204 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff34409b9d0 con 0x55ca1afde2f0 2024-09-11T22:21:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.414+0000 7ff339ffb640 1 -- 172.21.15.5:0/1402182204 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff33c027cb0 con 0x55ca1afde2f0 2024-09-11T22:21:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.414+0000 7ff35b278740 10 client.15027 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:21:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.414+0000 7ff35b278740 10 client.15027 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:21:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.414+0000 7ff35b278740 20 client.15027 populate_metadata read hostname 'smithi005' 2024-09-11T22:21:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.414+0000 7ff35b278740 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ca1b06c630 0x55ca1b06b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.414+0000 7ff35b278740 4 client.15027 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:21:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.414+0000 7ff35b278740 1 -- 172.21.15.5:0/1402182204 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "status"}) -- 0x55ca1af4bb10 con 0x55ca1b06c630 2024-09-11T22:21:42.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.414+0000 7ff35259c640 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ca1b06c630 0x55ca1b06b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.414+0000 7ff339ffb640 10 client.15027 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.414+0000 7ff35259c640 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ca1b06c630 0x55ca1b06b340 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78610d14f5c14acb server_cookie=32f9f830a643471b in_seq=0 out_seq=0 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff339ffb640 1 -- 172.21.15.5:0/1402182204 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55ca1af4bb10 con 0x55ca1b06c630 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff339ffb640 10 client.15027 handle_command_reply: tid=0 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 client.15027 shutdown 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 -- 172.21.15.5:0/1402182204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afde2f0 msgr2=0x55ca1afe2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afde2f0 0x55ca1afe2dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff35406a370 tx=0x7ff33c0138f0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 -- 172.21.15.5:0/1402182204 shutdown_connections 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ca1b06c630 0x55ca1b06b340 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ca1afe8bb0 0x55ca1b05edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca1afde2f0 0x55ca1afe2dd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 --2- 172.21.15.5:0/1402182204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ca1afdd9d0 0x55ca1afe2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff339ffb640 0 client.15027 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:21:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 -- 172.21.15.5:0/1402182204 >> 172.21.15.5:0/1402182204 conn(0x55ca1afd9f60 msgr2=0x55ca1afd9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 -- 172.21.15.5:0/1402182204 shutdown_connections 2024-09-11T22:21:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 1 -- 172.21.15.5:0/1402182204 wait complete. 2024-09-11T22:21:42.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff35b278740 20 client.15027 trim_cache size 0 max 16384 2024-09-11T22:21:42.423 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24500, 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 83.235908804000005, 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T22:21:42.424 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 93.644224928 2024-09-11T22:21:42.425 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:21:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff358e9f640 1 -- 172.21.15.5:0/3429238254 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff3400a2c90 msgr2=0x7ff3400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff358e9f640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff3400a2c90 0x7ff3400a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff34c048000 tx=0x7ff34c045000 comp rx=0 tx=0).stop 2024-09-11T22:21:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff358e9f640 1 -- 172.21.15.5:0/3429238254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354102d30 msgr2=0x7ff354198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.415+0000 7ff358e9f640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354102d30 0x7ff354198560 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff34400cce0 tx=0x7ff34400a620 comp rx=0 tx=0).stop 2024-09-11T22:21:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.416+0000 7ff358e9f640 1 -- 172.21.15.5:0/3429238254 shutdown_connections 2024-09-11T22:21:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.416+0000 7ff358e9f640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff3400a2c90 0x7ff3400a5150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.416+0000 7ff358e9f640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff3541082a0 0x7ff35419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.416+0000 7ff358e9f640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3541036e0 0x7ff354198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.416+0000 7ff358e9f640 1 --2- 172.21.15.5:0/3429238254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff354102d30 0x7ff354198560 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.416+0000 7ff358e9f640 1 -- 172.21.15.5:0/3429238254 >> 172.21.15.5:0/3429238254 conn(0x7ff3540fc7d0 msgr2=0x7ff354105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.416+0000 7ff358e9f640 1 -- 172.21.15.5:0/3429238254 shutdown_connections 2024-09-11T22:21:42.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:42.416+0000 7ff358e9f640 1 -- 172.21.15.5:0/3429238254 wait complete. 2024-09-11T22:21:43.124 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T22:21:43.125 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T22:21:43.139 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T22:21:43.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-11T22:21:43.396 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:43.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 -- 172.21.15.5:0/841241425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faab8100d90 msgr2=0x7faab81011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 --2- 172.21.15.5:0/841241425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faab8100d90 0x7faab81011f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009900 tx=0x7faaa002a410 comp rx=0 tx=0).stop 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 -- 172.21.15.5:0/841241425 shutdown_connections 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 --2- 172.21.15.5:0/841241425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab8101730 0x7faab8101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 --2- 172.21.15.5:0/841241425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faab8100d90 0x7faab81011f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 --2- 172.21.15.5:0/841241425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab8100450 0x7faab8100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 -- 172.21.15.5:0/841241425 >> 172.21.15.5:0/841241425 conn(0x7faab80fc280 msgr2=0x7faab80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 -- 172.21.15.5:0/841241425 shutdown_connections 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 -- 172.21.15.5:0/841241425 wait complete. 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 Processor -- start 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.914+0000 7faabec7d640 1 -- start start 2024-09-11T22:21:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabec7d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab8100450 0x7faab819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabec7d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab8100d90 0x7faab819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabec7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faab8101730 0x7faab81a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabec7d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faab810fa40 con 0x7faab8101730 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabec7d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faab810f8c0 con 0x7faab8100d90 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabec7d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faab810fbc0 con 0x7faab8100450 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabd21a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faab8101730 0x7faab81a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabd21a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faab8101730 0x7faab81a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54628/0 (socket says 172.21.15.5:54628) 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabd21a640 1 -- 172.21.15.5:0/3108057267 learned_addr learned my addr 172.21.15.5:0/3108057267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faab7fff640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab8100d90 0x7faab819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabca19640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab8100450 0x7faab819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabd21a640 1 -- 172.21.15.5:0/3108057267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab8100450 msgr2=0x7faab819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabd21a640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab8100450 0x7faab819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabd21a640 1 -- 172.21.15.5:0/3108057267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab8100d90 msgr2=0x7faab819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabd21a640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab8100d90 0x7faab819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabd21a640 1 -- 172.21.15.5:0/3108057267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab81a1cf0 con 0x7faab8101730 2024-09-11T22:21:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faabd21a640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faab8101730 0x7faab81a1680 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faab000d750 tx=0x7faab000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.915+0000 7faab5ffb640 1 -- 172.21.15.5:0/3108057267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0017070 con 0x7faab8101730 2024-09-11T22:21:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.916+0000 7faab5ffb640 1 -- 172.21.15.5:0/3108057267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab0005500 con 0x7faab8101730 2024-09-11T22:21:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.916+0000 7faabec7d640 1 -- 172.21.15.5:0/3108057267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab81a1fe0 con 0x7faab8101730 2024-09-11T22:21:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.916+0000 7faab5ffb640 1 -- 172.21.15.5:0/3108057267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0004d90 con 0x7faab8101730 2024-09-11T22:21:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.916+0000 7faabec7d640 1 -- 172.21.15.5:0/3108057267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab81a24e0 con 0x7faab8101730 2024-09-11T22:21:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.917+0000 7faabec7d640 1 -- 172.21.15.5:0/3108057267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab810b310 con 0x7faab8101730 2024-09-11T22:21:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.918+0000 7faab5ffb640 1 -- 172.21.15.5:0/3108057267 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7faab0004f30 con 0x7faab8101730 2024-09-11T22:21:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.919+0000 7faab5ffb640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faaa40a2cc0 0x7faaa40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.919+0000 7faabca19640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faaa40a2cc0 0x7faaa40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.919+0000 7faab5ffb640 1 -- 172.21.15.5:0/3108057267 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7faab00cec30 con 0x7faab8101730 2024-09-11T22:21:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.919+0000 7faabca19640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faaa40a2cc0 0x7faaa40a5180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faaa8005c40 tx=0x7faaa800a780 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:43.923+0000 7faab5ffb640 1 -- 172.21.15.5:0/3108057267 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faab009b9b0 con 0x7faab8101730 2024-09-11T22:21:43.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:43 smithi005 ceph-mon[30943]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:21:44.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:43 smithi159 ceph-mon[35534]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:21:44.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:43 smithi172 ceph-mon[35461]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-11T22:21:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.113+0000 7faabec7d640 1 -- 172.21.15.5:0/3108057267 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7faab8062630 con 0x7faaa40a2cc0 2024-09-11T22:21:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.114+0000 7faab5ffb640 1 -- 172.21.15.5:0/3108057267 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7faab8062630 con 0x7faaa40a2cc0 2024-09-11T22:21:44.114 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2024-09-11T22:21:44.114 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2024-09-11T22:21:44.115 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 -- 172.21.15.5:0/3108057267 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faaa40a2cc0 msgr2=0x7faaa40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faaa40a2cc0 0x7faaa40a5180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faaa8005c40 tx=0x7faaa800a780 comp rx=0 tx=0).stop 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 -- 172.21.15.5:0/3108057267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faab8101730 msgr2=0x7faab81a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faab8101730 0x7faab81a1680 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7faab000d750 tx=0x7faab000dc10 comp rx=0 tx=0).stop 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 -- 172.21.15.5:0/3108057267 shutdown_connections 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7faaa40a2cc0 0x7faaa40a5180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faab8101730 0x7faab81a1680 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faab8100d90 0x7faab819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 --2- 172.21.15.5:0/3108057267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faab8100450 0x7faab819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 -- 172.21.15.5:0/3108057267 >> 172.21.15.5:0/3108057267 conn(0x7faab80fc280 msgr2=0x7faab80fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 -- 172.21.15.5:0/3108057267 shutdown_connections 2024-09-11T22:21:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:44.116+0000 7faabec7d640 1 -- 172.21.15.5:0/3108057267 wait complete. 2024-09-11T22:21:44.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-11T22:21:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:44 smithi005 ceph-mon[30943]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:44 smithi159 ceph-mon[35534]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:45.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:44 smithi172 ceph-mon[35461]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:45.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.676+0000 7f38ca36e640 1 -- 172.21.15.5:0/3114909166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38c4074c50 msgr2=0x7f38c40750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.676+0000 7f38ca36e640 1 --2- 172.21.15.5:0/3114909166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38c4074c50 0x7f38c40750b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009900 tx=0x7f38b802a410 comp rx=0 tx=0).stop 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.677+0000 7f38ca36e640 1 -- 172.21.15.5:0/3114909166 shutdown_connections 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.677+0000 7f38ca36e640 1 --2- 172.21.15.5:0/3114909166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38c4109a80 0x7f38c410bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.677+0000 7f38ca36e640 1 --2- 172.21.15.5:0/3114909166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38c4074c50 0x7f38c40750b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.677+0000 7f38ca36e640 1 --2- 172.21.15.5:0/3114909166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c4076220 0x7f38c4074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.677+0000 7f38ca36e640 1 -- 172.21.15.5:0/3114909166 >> 172.21.15.5:0/3114909166 conn(0x7f38c40fd390 msgr2=0x7f38c40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.677+0000 7f38ca36e640 1 -- 172.21.15.5:0/3114909166 shutdown_connections 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.677+0000 7f38ca36e640 1 -- 172.21.15.5:0/3114909166 wait complete. 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.677+0000 7f38ca36e640 1 Processor -- start 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.677+0000 7f38ca36e640 1 -- start start 2024-09-11T22:21:45.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38ca36e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38c4074c50 0x7f38c419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38ca36e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c4076220 0x7f38c419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38ca36e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38c4109a80 0x7f38c41a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38ca36e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410fb00 con 0x7f38c4074c50 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38ca36e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410f980 con 0x7f38c4109a80 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38ca36e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38c410fc80 con 0x7f38c4076220 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38c4074c50 0x7f38c419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38c4074c50 0x7f38c419a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54648/0 (socket says 172.21.15.5:54648) 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38c3fff640 1 -- 172.21.15.5:0/4115498595 learned_addr learned my addr 172.21.15.5:0/4115498595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38c37fe640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c4076220 0x7f38c419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38c3fff640 1 -- 172.21.15.5:0/4115498595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c4076220 msgr2=0x7f38c419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:45.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38c3fff640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c4076220 0x7f38c419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38c3fff640 1 -- 172.21.15.5:0/4115498595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38c4109a80 msgr2=0x7f38c41a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38c3fff640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38c4109a80 0x7f38c41a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38c3fff640 1 -- 172.21.15.5:0/4115498595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c41a1df0 con 0x7f38c4074c50 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.678+0000 7f38c3fff640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38c4074c50 0x7f38c419a1c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f38b400b450 tx=0x7f38b400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.679+0000 7f38c17fa640 1 -- 172.21.15.5:0/4115498595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4015020 con 0x7f38c4074c50 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.679+0000 7f38c17fa640 1 -- 172.21.15.5:0/4115498595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b4011d70 con 0x7f38c4074c50 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.679+0000 7f38ca36e640 1 -- 172.21.15.5:0/4115498595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c41a20e0 con 0x7f38c4074c50 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.679+0000 7f38c17fa640 1 -- 172.21.15.5:0/4115498595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b400f340 con 0x7f38c4074c50 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.679+0000 7f38ca36e640 1 -- 172.21.15.5:0/4115498595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41a2620 con 0x7f38c4074c50 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.680+0000 7f38ca36e640 1 -- 172.21.15.5:0/4115498595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c4108800 con 0x7f38c4074c50 2024-09-11T22:21:45.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.681+0000 7f38c17fa640 1 -- 172.21.15.5:0/4115498595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f38b4018020 con 0x7f38c4074c50 2024-09-11T22:21:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.681+0000 7f38c17fa640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38ac0a2c40 0x7f38ac0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.681+0000 7f38c37fe640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38ac0a2c40 0x7f38ac0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.681+0000 7f38c17fa640 1 -- 172.21.15.5:0/4115498595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f38b40ceb90 con 0x7f38c4074c50 2024-09-11T22:21:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.682+0000 7f38c37fe640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38ac0a2c40 0x7f38ac0a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f38b80314e0 tx=0x7f38b8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:45.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.686+0000 7f38c17fa640 1 -- 172.21.15.5:0/4115498595 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38b401f020 con 0x7f38c4074c50 2024-09-11T22:21:45.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.871+0000 7f38ca36e640 1 -- 172.21.15.5:0/4115498595 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f38c4062630 con 0x7f38ac0a2c40 2024-09-11T22:21:45.872 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:45 smithi005 ceph-mon[30943]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:21:45.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.883+0000 7f38c17fa640 1 -- 172.21.15.5:0/4115498595 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f38c4062630 con 0x7f38ac0a2c40 2024-09-11T22:21:45.884 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-11T22:21:45.884 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (100s) 21s ago 99s 15.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 539c65355df6 2024-09-11T22:21:45.884 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi159 running (97s) 29s ago 97s 26.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 8d154520033b 2024-09-11T22:21:45.884 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi172 running (112s) 29s ago 111s 19.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 ee25c5c264a2 2024-09-11T22:21:45.884 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (2m) 21s ago 2m 16.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 816ef0aabc16 2024-09-11T22:21:45.884 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi159 running (117s) 29s ago 117s 20.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 da2b9fe291c2 2024-09-11T22:21:45.884 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi172 running (114s) 29s ago 114s 16.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 e7e643923079 2024-09-11T22:21:45.884 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (104s) 21s ago 102s 15.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 5fd3bb2fb78d 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi159 running (109s) 29s ago 109s 20.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 5e3191587899 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi172 running (106s) 29s ago 106s 18.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 98668205922b 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (2m) 21s ago 2m 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 fe9c99224f83 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi159 running (2m) 29s ago 2m 20.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 114d39de8ee6 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi172 running (2m) 29s ago 2m 15.9M - 19.1.1-298-gb53b6d3e 54467abc55f8 8e270a63289f 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (9m) 21s ago 9m 509M - 19.1.1-298-gb53b6d3e 54467abc55f8 894bca1446d0 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi159 *:8443,8765 running (7m) 29s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 51f3b0a7d590 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi172 *:8443,8765 running (7m) 29s ago 7m 443M - 19.1.1-298-gb53b6d3e 54467abc55f8 8053c59d6033 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (9m) 21s ago 9m 51.4M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 32a280f5e920 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi159 running (7m) 29s ago 7m 39.4M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 991186ed1dcb 2024-09-11T22:21:45.885 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi172 running (7m) 29s ago 7m 39.0M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 548b0f3e09d5 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (7m) 21s ago 7m 57.0M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 81f7b02cb496 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi159 running (6m) 29s ago 6m 56.5M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 8a4b9ef87350 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi172 running (6m) 29s ago 6m 57.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 ea23f0bb2ec9 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (5m) 21s ago 5m 55.5M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 2b1ff9a6a1a2 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi159 running (5m) 29s ago 5m 41.7M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 7566b6e1a337 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi172 running (5m) 29s ago 5m 55.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 3af4b6f48dd9 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (4m) 21s ago 4m 41.4M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 e1fad748b305 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi159 running (4m) 29s ago 4m 54.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 b13e34e6b59d 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi172 running (3m) 29s ago 3m 44.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 c4ee0b999fd0 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (3m) 21s ago 3m 43.9M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 e826e430d041 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi159 running (2m) 29s ago 2m 45.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 65829637b940 2024-09-11T22:21:45.886 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi172 running (2m) 29s ago 2m 43.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 d404a7ea46a4 2024-09-11T22:21:45.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 -- 172.21.15.5:0/4115498595 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38ac0a2c40 msgr2=0x7f38ac0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:45.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38ac0a2c40 0x7f38ac0a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f38b80314e0 tx=0x7f38b8000bc0 comp rx=0 tx=0).stop 2024-09-11T22:21:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 -- 172.21.15.5:0/4115498595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38c4074c50 msgr2=0x7f38c419a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38c4074c50 0x7f38c419a1c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f38b400b450 tx=0x7f38b400b910 comp rx=0 tx=0).stop 2024-09-11T22:21:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 -- 172.21.15.5:0/4115498595 shutdown_connections 2024-09-11T22:21:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f38ac0a2c40 0x7f38ac0a5100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38c4109a80 0x7f38c41a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38c4076220 0x7f38c419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 --2- 172.21.15.5:0/4115498595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38c4074c50 0x7f38c419a1c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 -- 172.21.15.5:0/4115498595 >> 172.21.15.5:0/4115498595 conn(0x7f38c40fd390 msgr2=0x7f38c41059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 -- 172.21.15.5:0/4115498595 shutdown_connections 2024-09-11T22:21:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:45.886+0000 7f38ca36e640 1 -- 172.21.15.5:0/4115498595 wait complete. 2024-09-11T22:21:46.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:45 smithi159 ceph-mon[35534]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:21:46.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:45 smithi172 ceph-mon[35461]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:21:46.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-11T22:21:46.896 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:47.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:46 smithi005 ceph-mon[30943]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:47.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:46 smithi159 ceph-mon[35534]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:47.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:46 smithi172 ceph-mon[35461]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:47.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.409+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/1304793152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b681037c0 msgr2=0x7f0b6810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:47.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.409+0000 7f0b6d6f4640 1 --2- 172.21.15.5:0/1304793152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b681037c0 0x7f0b6810a050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0b60009ca0 tx=0x7f0b6002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:21:47.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.409+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/1304793152 shutdown_connections 2024-09-11T22:21:47.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.409+0000 7f0b6d6f4640 1 --2- 172.21.15.5:0/1304793152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b681037c0 0x7f0b6810a050 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.409+0000 7f0b6d6f4640 1 --2- 172.21.15.5:0/1304793152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b68102e00 0x7f0b68103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.409+0000 7f0b6d6f4640 1 --2- 172.21.15.5:0/1304793152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68101c00 0x7f0b68102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.409+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/1304793152 >> 172.21.15.5:0/1304793152 conn(0x7f0b680fd390 msgr2=0x7f0b680ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.409+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/1304793152 shutdown_connections 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.409+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/1304793152 wait complete. 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b6d6f4640 1 Processor -- start 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b6d6f4640 1 -- start start 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b6d6f4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68101c00 0x7f0b68075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b6d6f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b68102e00 0x7f0b680745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b6d6f4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b681037c0 0x7f0b68074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b6d6f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6810d860 con 0x7f0b68102e00 2024-09-11T22:21:47.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b6d6f4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6810d6e0 con 0x7f0b681037c0 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b6d6f4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b6810d9e0 con 0x7f0b68101c00 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b68102e00 0x7f0b680745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b68102e00 0x7f0b680745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54672/0 (socket says 172.21.15.5:54672) 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b667fc640 1 -- 172.21.15.5:0/460388429 learned_addr learned my addr 172.21.15.5:0/460388429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b677fe640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b681037c0 0x7f0b68074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b66ffd640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68101c00 0x7f0b68075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b667fc640 1 -- 172.21.15.5:0/460388429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68101c00 msgr2=0x7f0b68075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b667fc640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68101c00 0x7f0b68075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b667fc640 1 -- 172.21.15.5:0/460388429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b681037c0 msgr2=0x7f0b68074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b667fc640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b681037c0 0x7f0b68074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:47.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.410+0000 7f0b667fc640 1 -- 172.21.15.5:0/460388429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b680753e0 con 0x7f0b68102e00 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.411+0000 7f0b667fc640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b68102e00 0x7f0b680745e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00d950 tx=0x7f0b5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.411+0000 7f0b4ffff640 1 -- 172.21.15.5:0/460388429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c017070 con 0x7f0b68102e00 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.411+0000 7f0b4ffff640 1 -- 172.21.15.5:0/460388429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b5c009c30 con 0x7f0b68102e00 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.411+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/460388429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b681a1fb0 con 0x7f0b68102e00 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.411+0000 7f0b4ffff640 1 -- 172.21.15.5:0/460388429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c00f6b0 con 0x7f0b68102e00 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.411+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/460388429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b681a2460 con 0x7f0b68102e00 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.412+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/460388429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b68109000 con 0x7f0b68102e00 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.413+0000 7f0b4ffff640 1 -- 172.21.15.5:0/460388429 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0b5c0109d0 con 0x7f0b68102e00 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.414+0000 7f0b4ffff640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0b500a2b80 0x7f0b500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.414+0000 7f0b66ffd640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0b500a2b80 0x7f0b500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.414+0000 7f0b4ffff640 1 -- 172.21.15.5:0/460388429 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f0b5c0ceda0 con 0x7f0b68102e00 2024-09-11T22:21:47.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.414+0000 7f0b66ffd640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0b500a2b80 0x7f0b500a5040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0b58005fd0 tx=0x7f0b58005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:47.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.418+0000 7f0b4ffff640 1 -- 172.21.15.5:0/460388429 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b5c009de0 con 0x7f0b68102e00 2024-09-11T22:21:47.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.601+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/460388429 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f0b68062630 con 0x7f0b500a2b80 2024-09-11T22:21:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.607+0000 7f0b4ffff640 1 -- 172.21.15.5:0/460388429 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f0b68062630 con 0x7f0b500a2b80 2024-09-11T22:21:47.607 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-11T22:21:47.607 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 31s ago 2m smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi159=b;smithi159=e;smithi159=h;smithi159=k;smithi172=c;smithi172=f;smithi172=i;smithi172=l;count:12 2024-09-11T22:21:47.607 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 31s ago 7m smithi005=x;smithi159=y;smithi172=z;count:3 2024-09-11T22:21:47.608 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 31s ago 8m smithi005:172.21.15.5=a;smithi159:172.21.15.159=b;smithi172:172.21.15.172=c;count:3 2024-09-11T22:21:47.608 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 31s ago - 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/460388429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0b500a2b80 msgr2=0x7f0b500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0b500a2b80 0x7f0b500a5040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0b58005fd0 tx=0x7f0b58005850 comp rx=0 tx=0).stop 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/460388429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b68102e00 msgr2=0x7f0b680745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b68102e00 0x7f0b680745e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00d950 tx=0x7f0b5c00de10 comp rx=0 tx=0).stop 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/460388429 shutdown_connections 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0b500a2b80 0x7f0b500a5040 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b681037c0 0x7f0b68074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b68102e00 0x7f0b680745e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 --2- 172.21.15.5:0/460388429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68101c00 0x7f0b68075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/460388429 >> 172.21.15.5:0/460388429 conn(0x7f0b680fd390 msgr2=0x7f0b681061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/460388429 shutdown_connections 2024-09-11T22:21:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:47.609+0000 7f0b6d6f4640 1 -- 172.21.15.5:0/460388429 wait complete. 2024-09-11T22:21:48.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:47 smithi005 ceph-mon[30943]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:21:48.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:47 smithi005 ceph-mon[30943]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T22:21:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:47 smithi159 ceph-mon[35534]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:21:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:47 smithi159 ceph-mon[35534]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T22:21:48.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:47 smithi172 ceph-mon[35461]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:21:48.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:47 smithi172 ceph-mon[35461]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-11T22:21:48.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-11T22:21:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:48.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:48 smithi005 ceph-mon[30943]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:49.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:48 smithi159 ceph-mon[35534]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:49.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:48 smithi172 ceph-mon[35461]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.229+0000 7f9114f69640 1 -- 172.21.15.5:0/1000954320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9110102e00 msgr2=0x7f9110103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:49.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.229+0000 7f9114f69640 1 --2- 172.21.15.5:0/1000954320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9110102e00 0x7f9110103280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9104009900 tx=0x7f910402a410 comp rx=0 tx=0).stop 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.229+0000 7f9114f69640 1 -- 172.21.15.5:0/1000954320 shutdown_connections 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.229+0000 7f9114f69640 1 --2- 172.21.15.5:0/1000954320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91101037c0 0x7f911010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.229+0000 7f9114f69640 1 --2- 172.21.15.5:0/1000954320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9110102e00 0x7f9110103280 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.229+0000 7f9114f69640 1 --2- 172.21.15.5:0/1000954320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9110101c00 0x7f9110102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.229+0000 7f9114f69640 1 -- 172.21.15.5:0/1000954320 >> 172.21.15.5:0/1000954320 conn(0x7f91100fd390 msgr2=0x7f91100ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.229+0000 7f9114f69640 1 -- 172.21.15.5:0/1000954320 shutdown_connections 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.229+0000 7f9114f69640 1 -- 172.21.15.5:0/1000954320 wait complete. 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f9114f69640 1 Processor -- start 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f9114f69640 1 -- start start 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f9114f69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9110101c00 0x7f9110075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:49.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f9114f69640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9110102e00 0x7f91100745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f9114f69640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91101037c0 0x7f9110074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f9114f69640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f911010d980 con 0x7f9110101c00 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f9114f69640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f911010d800 con 0x7f9110102e00 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f9114f69640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f911010db00 con 0x7f91101037c0 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f910e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9110101c00 0x7f9110075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f910e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9110101c00 0x7f9110075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54678/0 (socket says 172.21.15.5:54678) 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f910e59c640 1 -- 172.21.15.5:0/1477648676 learned_addr learned my addr 172.21.15.5:0/1477648676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f910dd9b640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9110102e00 0x7f91100745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.230+0000 7f910ed9d640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91101037c0 0x7f9110074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f910e59c640 1 -- 172.21.15.5:0/1477648676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91101037c0 msgr2=0x7f9110074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f910e59c640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91101037c0 0x7f9110074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f910e59c640 1 -- 172.21.15.5:0/1477648676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9110102e00 msgr2=0x7f91100745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:49.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f910e59c640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9110102e00 0x7f91100745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f910e59c640 1 -- 172.21.15.5:0/1477648676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91100753e0 con 0x7f9110101c00 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f910ed9d640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91101037c0 0x7f9110074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f910e59c640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9110101c00 0x7f9110075f90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f910000b450 tx=0x7f910000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f90ff7fe640 1 -- 172.21.15.5:0/1477648676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100015020 con 0x7f9110101c00 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f90ff7fe640 1 -- 172.21.15.5:0/1477648676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9100011d70 con 0x7f9110101c00 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f9114f69640 1 -- 172.21.15.5:0/1477648676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91101a2220 con 0x7f9110101c00 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f90ff7fe640 1 -- 172.21.15.5:0/1477648676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910000f340 con 0x7f9110101c00 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.231+0000 7f9114f69640 1 -- 172.21.15.5:0/1477648676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91101a27b0 con 0x7f9110101c00 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.232+0000 7f9114f69640 1 -- 172.21.15.5:0/1477648676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9110109070 con 0x7f9110101c00 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.233+0000 7f90ff7fe640 1 -- 172.21.15.5:0/1477648676 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9100018020 con 0x7f9110101c00 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.234+0000 7f90ff7fe640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f90f80a2b80 0x7f90f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.234+0000 7f910dd9b640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f90f80a2b80 0x7f90f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:49.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.234+0000 7f90ff7fe640 1 -- 172.21.15.5:0/1477648676 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f91000ceb40 con 0x7f9110101c00 2024-09-11T22:21:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.234+0000 7f910dd9b640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f90f80a2b80 0x7f90f80a5040 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f91040314e0 tx=0x7f9104000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.238+0000 7f90ff7fe640 1 -- 172.21.15.5:0/1477648676 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f910001f020 con 0x7f9110101c00 2024-09-11T22:21:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.420+0000 7f9114f69640 1 -- 172.21.15.5:0/1477648676 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f9110062630 con 0x7f90f80a2b80 2024-09-11T22:21:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.421+0000 7f90ff7fe640 1 -- 172.21.15.5:0/1477648676 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f9110062630 con 0x7f90f80a2b80 2024-09-11T22:21:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2024-09-11T22:21:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2024-09-11T22:21:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:smithi159 172.21.15.159 2024-09-11T22:21:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:smithi172 172.21.15.172 2024-09-11T22:21:49.422 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2024-09-11T22:21:49.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 -- 172.21.15.5:0/1477648676 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f90f80a2b80 msgr2=0x7f90f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f90f80a2b80 0x7f90f80a5040 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f91040314e0 tx=0x7f9104000bc0 comp rx=0 tx=0).stop 2024-09-11T22:21:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 -- 172.21.15.5:0/1477648676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9110101c00 msgr2=0x7f9110075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9110101c00 0x7f9110075f90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f910000b450 tx=0x7f910000b910 comp rx=0 tx=0).stop 2024-09-11T22:21:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 -- 172.21.15.5:0/1477648676 shutdown_connections 2024-09-11T22:21:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f90f80a2b80 0x7f90f80a5040 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:49.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91101037c0 0x7f9110074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9110102e00 0x7f91100745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 --2- 172.21.15.5:0/1477648676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9110101c00 0x7f9110075f90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 -- 172.21.15.5:0/1477648676 >> 172.21.15.5:0/1477648676 conn(0x7f91100fd390 msgr2=0x7f9110106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 -- 172.21.15.5:0/1477648676 shutdown_connections 2024-09-11T22:21:49.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:49.423+0000 7f9114f69640 1 -- 172.21.15.5:0/1477648676 wait complete. 2024-09-11T22:21:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:49 smithi005 ceph-mon[30943]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:21:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:49 smithi159 ceph-mon[35534]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:21:50.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:49 smithi172 ceph-mon[35461]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-11T22:21:50.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-11T22:21:50.398 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:50.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:50 smithi005 ceph-mon[30943]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.929+0000 7f0cbc057640 1 -- 172.21.15.5:0/698300830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb40ffef0 msgr2=0x7f0cb41002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.929+0000 7f0cbc057640 1 --2- 172.21.15.5:0/698300830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb40ffef0 0x7f0cb41002f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009930 tx=0x7f0cb002a420 comp rx=0 tx=0).stop 2024-09-11T22:21:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.929+0000 7f0cbc057640 1 -- 172.21.15.5:0/698300830 shutdown_connections 2024-09-11T22:21:50.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.929+0000 7f0cbc057640 1 --2- 172.21.15.5:0/698300830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb4103930 0x7f0cb410a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.929+0000 7f0cbc057640 1 --2- 172.21.15.5:0/698300830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cb4102f70 0x7f0cb41033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.929+0000 7f0cbc057640 1 --2- 172.21.15.5:0/698300830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb40ffef0 0x7f0cb41002f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.929+0000 7f0cbc057640 1 -- 172.21.15.5:0/698300830 >> 172.21.15.5:0/698300830 conn(0x7f0cb40fba40 msgr2=0x7f0cb40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.929+0000 7f0cbc057640 1 -- 172.21.15.5:0/698300830 shutdown_connections 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.929+0000 7f0cbc057640 1 -- 172.21.15.5:0/698300830 wait complete. 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cbc057640 1 Processor -- start 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cbc057640 1 -- start start 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cbc057640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cb40ffef0 0x7f0cb419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cbc057640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb4102f70 0x7f0cb419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cbc057640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4103930 0x7f0cb41a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:50.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cbc057640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb410d910 con 0x7f0cb4103930 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cbc057640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb410d790 con 0x7f0cb4102f70 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cbc057640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb410da90 con 0x7f0cb40ffef0 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cba5f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4103930 0x7f0cb41a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cba5f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4103930 0x7f0cb41a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54690/0 (socket says 172.21.15.5:54690) 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cba5f4640 1 -- 172.21.15.5:0/2330354562 learned_addr learned my addr 172.21.15.5:0/2330354562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.930+0000 7f0cb95f2640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb4102f70 0x7f0cb419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0cb9df3640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cb40ffef0 0x7f0cb419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0cba5f4640 1 -- 172.21.15.5:0/2330354562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cb40ffef0 msgr2=0x7f0cb419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0cba5f4640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cb40ffef0 0x7f0cb419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0cba5f4640 1 -- 172.21.15.5:0/2330354562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb4102f70 msgr2=0x7f0cb419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0cba5f4640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb4102f70 0x7f0cb419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:50.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0cba5f4640 1 -- 172.21.15.5:0/2330354562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb41a1e80 con 0x7f0cb4103930 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0cba5f4640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4103930 0x7f0cb41a1810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00cad0 tx=0x7f0cac00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0caaffd640 1 -- 172.21.15.5:0/2330354562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00ad60 con 0x7f0cb4103930 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0caaffd640 1 -- 172.21.15.5:0/2330354562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cac005020 con 0x7f0cb4103930 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0cbc057640 1 -- 172.21.15.5:0/2330354562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb41a2170 con 0x7f0cb4103930 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0caaffd640 1 -- 172.21.15.5:0/2330354562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00f6b0 con 0x7f0cb4103930 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.931+0000 7f0cbc057640 1 -- 172.21.15.5:0/2330354562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb41a2730 con 0x7f0cb4103930 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.932+0000 7f0cbc057640 1 -- 172.21.15.5:0/2330354562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb41a2a50 con 0x7f0cb4103930 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.933+0000 7f0caaffd640 1 -- 172.21.15.5:0/2330354562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0cac005560 con 0x7f0cb4103930 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.934+0000 7f0caaffd640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0ca00a2c30 0x7f0ca00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.934+0000 7f0cb9df3640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0ca00a2c30 0x7f0ca00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.934+0000 7f0caaffd640 1 -- 172.21.15.5:0/2330354562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f0cac0cf0c0 con 0x7f0cb4103930 2024-09-11T22:21:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.934+0000 7f0cb9df3640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0ca00a2c30 0x7f0ca00a50f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0006010 tx=0x7f0cb00315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:50.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:50.938+0000 7f0caaffd640 1 -- 172.21.15.5:0/2330354562 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cac014030 con 0x7f0cb4103930 2024-09-11T22:21:51.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:50 smithi159 ceph-mon[35534]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:51.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:50 smithi172 ceph-mon[35461]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:51.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.121+0000 7f0cbc057640 1 -- 172.21.15.5:0/2330354562 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0cb41a2c20 con 0x7f0ca00a2c30 2024-09-11T22:21:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.123+0000 7f0caaffd640 1 -- 172.21.15.5:0/2330354562 <== mgr.14148 v2:172.21.15.5:6800/3591749770 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f0cb41a2c20 con 0x7f0ca00a2c30 2024-09-11T22:21:51.123 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-11T22:21:51.123 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-11T22:21:51.123 INFO:teuthology.orchestra.run.smithi005.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T22:21:51.123 INFO:teuthology.orchestra.run.smithi005.stdout:smithi172 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005M400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-11T22:21:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 -- 172.21.15.5:0/2330354562 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0ca00a2c30 msgr2=0x7f0ca00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0ca00a2c30 0x7f0ca00a50f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0006010 tx=0x7f0cb00315d0 comp rx=0 tx=0).stop 2024-09-11T22:21:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 -- 172.21.15.5:0/2330354562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4103930 msgr2=0x7f0cb41a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4103930 0x7f0cb41a1810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00cad0 tx=0x7f0cac00ced0 comp rx=0 tx=0).stop 2024-09-11T22:21:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 -- 172.21.15.5:0/2330354562 shutdown_connections 2024-09-11T22:21:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f0ca00a2c30 0x7f0ca00a50f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cb4103930 0x7f0cb41a1810 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cb4102f70 0x7f0cb419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 --2- 172.21.15.5:0/2330354562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cb40ffef0 0x7f0cb419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 -- 172.21.15.5:0/2330354562 >> 172.21.15.5:0/2330354562 conn(0x7f0cb40fba40 msgr2=0x7f0cb41087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 -- 172.21.15.5:0/2330354562 shutdown_connections 2024-09-11T22:21:51.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:51.125+0000 7f0cbc057640 1 -- 172.21.15.5:0/2330354562 wait complete. 2024-09-11T22:21:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:51 smithi005 ceph-mon[30943]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T22:21:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:51 smithi005 ceph-mon[30943]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:51.962 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-11T22:21:51.973 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T22:21:51.974 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-11T22:21:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:51 smithi159 ceph-mon[35534]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T22:21:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:51 smithi159 ceph-mon[35534]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:51 smithi172 ceph-mon[35461]: pgmap v329: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T22:21:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:51 smithi172 ceph-mon[35461]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:21:52.237 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.747+0000 7fe6e9546640 1 -- 172.21.15.5:0/2937976246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e41026a0 msgr2=0x7fe6e4104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.747+0000 7fe6e9546640 1 --2- 172.21.15.5:0/2937976246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e41026a0 0x7fe6e4104a90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009900 tx=0x7fe6d402a410 comp rx=0 tx=0).stop 2024-09-11T22:21:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.747+0000 7fe6e9546640 1 -- 172.21.15.5:0/2937976246 shutdown_connections 2024-09-11T22:21:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.747+0000 7fe6e9546640 1 --2- 172.21.15.5:0/2937976246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e4107900 0x7fe6e4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.747+0000 7fe6e9546640 1 --2- 172.21.15.5:0/2937976246 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6e4104fd0 0x7fe6e41073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.747+0000 7fe6e9546640 1 --2- 172.21.15.5:0/2937976246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e41026a0 0x7fe6e4104a90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.747+0000 7fe6e9546640 1 -- 172.21.15.5:0/2937976246 >> 172.21.15.5:0/2937976246 conn(0x7fe6e40fc2a0 msgr2=0x7fe6e40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.747+0000 7fe6e9546640 1 -- 172.21.15.5:0/2937976246 shutdown_connections 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.747+0000 7fe6e9546640 1 -- 172.21.15.5:0/2937976246 wait complete. 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.748+0000 7fe6e9546640 1 Processor -- start 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.748+0000 7fe6e9546640 1 -- start start 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e9546640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e41026a0 0x7fe6e4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e9546640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e4104fd0 0x7fe6e41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e9546640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6e4107900 0x7fe6e419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e9546640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e410d990 con 0x7fe6e41026a0 2024-09-11T22:21:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e9546640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e410d810 con 0x7fe6e4104fd0 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e9546640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e410db10 con 0x7fe6e4107900 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e41026a0 0x7fe6e4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e41026a0 0x7fe6e4198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41124/0 (socket says 172.21.15.5:41124) 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e2ffd640 1 -- 172.21.15.5:0/1488996947 learned_addr learned my addr 172.21.15.5:0/1488996947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e37fe640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6e4107900 0x7fe6e419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e27fc640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e4104fd0 0x7fe6e41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e2ffd640 1 -- 172.21.15.5:0/1488996947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6e4107900 msgr2=0x7fe6e419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e2ffd640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6e4107900 0x7fe6e419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e2ffd640 1 -- 172.21.15.5:0/1488996947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e4104fd0 msgr2=0x7fe6e41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e2ffd640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e4104fd0 0x7fe6e41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e2ffd640 1 -- 172.21.15.5:0/1488996947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e419fca0 con 0x7fe6e41026a0 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e27fc640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e4104fd0 0x7fe6e41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6e2ffd640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e41026a0 0x7fe6e4198070 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009230 tx=0x7fe6d4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6cbfff640 1 -- 172.21.15.5:0/1488996947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d403f030 con 0x7fe6e41026a0 2024-09-11T22:21:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.749+0000 7fe6cbfff640 1 -- 172.21.15.5:0/1488996947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d4003ce0 con 0x7fe6e41026a0 2024-09-11T22:21:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.750+0000 7fe6e9546640 1 -- 172.21.15.5:0/1488996947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e419ff30 con 0x7fe6e41026a0 2024-09-11T22:21:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.750+0000 7fe6cbfff640 1 -- 172.21.15.5:0/1488996947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4035070 con 0x7fe6e41026a0 2024-09-11T22:21:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.750+0000 7fe6e9546640 1 -- 172.21.15.5:0/1488996947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e41a03b0 con 0x7fe6e41026a0 2024-09-11T22:21:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.751+0000 7fe6e9546640 1 -- 172.21.15.5:0/1488996947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e40fdd90 con 0x7fe6e41026a0 2024-09-11T22:21:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.752+0000 7fe6cbfff640 1 -- 172.21.15.5:0/1488996947 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe6d4042030 con 0x7fe6e41026a0 2024-09-11T22:21:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.753+0000 7fe6cbfff640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe6cc0a2bb0 0x7fe6cc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.753+0000 7fe6e27fc640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe6cc0a2bb0 0x7fe6cc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.753+0000 7fe6cbfff640 1 -- 172.21.15.5:0/1488996947 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fe6d40ef790 con 0x7fe6e41026a0 2024-09-11T22:21:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.753+0000 7fe6e27fc640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe6cc0a2bb0 0x7fe6cc0a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8006fd0 tx=0x7fe6d8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.757+0000 7fe6cbfff640 1 -- 172.21.15.5:0/1488996947 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6d40bc6d0 con 0x7fe6e41026a0 2024-09-11T22:21:52.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.973+0000 7fe6e9546640 1 -- 172.21.15.5:0/1488996947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe6e4062f30 con 0x7fe6e41026a0 2024-09-11T22:21:52.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.974+0000 7fe6cbfff640 1 -- 172.21.15.5:0/1488996947 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3124 (secure 0 0 0) 0x7fe6d40bbfd0 con 0x7fe6e41026a0 2024-09-11T22:21:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:e27 2024-09-11T22:21:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:btime 2024-09-11T22:20:30:263672+0000 2024-09-11T22:21:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T22:21:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T22:21:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2024-09-11T22:21:52.975 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2024-09-11T22:21:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2024-09-11T22:21:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 27 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-11T22:20:18.900781+0000 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-11T22:20:30.263652+0000 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2024-09-11T22:21:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 5 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2,3,4 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24500,1=24473,2=24454,3=24452,4=24436} 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3] 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2024-09-11T22:21:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:qdb_cluster leader: 24500 members: 24500,24473,24454,24452,24436 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{0:24500} state up:active seq 4 addr [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{0:24431} state up:standby-replay seq 1 addr [v2:172.21.15.159:6832/2975110123,v1:172.21.15.159:6833/2975110123] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{1:24473} state up:active seq 9 addr [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{1:24430} state up:standby-replay seq 1 addr [v2:172.21.15.172:6834/4149321843,v1:172.21.15.172:6835/4149321843] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{2:24454} state up:active seq 8 addr [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{2:24391} state up:standby-replay seq 1 addr [v2:172.21.15.172:6832/1186865242,v1:172.21.15.172:6833/1186865242] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{3:24452} state up:active seq 11 addr [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{3:14595} state up:standby-replay seq 1 addr [v2:172.21.15.5:6840/2232645952,v1:172.21.15.5:6841/2232645952] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{4:24436} state up:active seq 10 addr [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{4:14583} state up:standby-replay seq 1 addr [v2:172.21.15.5:6838/2458034474,v1:172.21.15.5:6839/2458034474] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2024-09-11T22:21:52.979 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:52.979 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.5:6834/2780062479,v1:172.21.15.5:6835/2780062479] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.979 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.5:6836/118173744,v1:172.21.15.5:6837/118173744] compat {c=[1],r=[1],i=[1fff]}] 2024-09-11T22:21:52.979 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-11T22:21:52.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 -- 172.21.15.5:0/1488996947 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe6cc0a2bb0 msgr2=0x7fe6cc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe6cc0a2bb0 0x7fe6cc0a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8006fd0 tx=0x7fe6d8008040 comp rx=0 tx=0).stop 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 -- 172.21.15.5:0/1488996947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e41026a0 msgr2=0x7fe6e4198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e41026a0 0x7fe6e4198070 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009230 tx=0x7fe6d4004430 comp rx=0 tx=0).stop 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 -- 172.21.15.5:0/1488996947 shutdown_connections 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe6cc0a2bb0 0x7fe6cc0a5070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6e4107900 0x7fe6e419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6e4104fd0 0x7fe6e41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 --2- 172.21.15.5:0/1488996947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6e41026a0 0x7fe6e4198070 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 -- 172.21.15.5:0/1488996947 >> 172.21.15.5:0/1488996947 conn(0x7fe6e40fc2a0 msgr2=0x7fe6e41030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.976+0000 7fe6e9546640 1 -- 172.21.15.5:0/1488996947 shutdown_connections 2024-09-11T22:21:52.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:52.977+0000 7fe6e9546640 1 -- 172.21.15.5:0/1488996947 wait complete. 2024-09-11T22:21:53.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-11T22:21:53.952 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:54.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:53 smithi005 ceph-mon[30943]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:21:54.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:53 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1488996947' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T22:21:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:53 smithi159 ceph-mon[35534]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:21:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:53 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1488996947' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T22:21:54.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:53 smithi172 ceph-mon[35461]: pgmap v330: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:21:54.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:53 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1488996947' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-11T22:21:54.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.506+0000 7f41b909f640 1 -- 172.21.15.5:0/2003525880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b41078c0 msgr2=0x7f41b4109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.506+0000 7f41b909f640 1 --2- 172.21.15.5:0/2003525880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b41078c0 0x7f41b4109d10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009900 tx=0x7f41ac02a410 comp rx=0 tx=0).stop 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.506+0000 7f41b909f640 1 -- 172.21.15.5:0/2003525880 shutdown_connections 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.506+0000 7f41b909f640 1 --2- 172.21.15.5:0/2003525880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b41078c0 0x7f41b4109d10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.506+0000 7f41b909f640 1 --2- 172.21.15.5:0/2003525880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4104f90 0x7f41b4107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.506+0000 7f41b909f640 1 --2- 172.21.15.5:0/2003525880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b4102660 0x7f41b4104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.506+0000 7f41b909f640 1 -- 172.21.15.5:0/2003525880 >> 172.21.15.5:0/2003525880 conn(0x7f41b40fc280 msgr2=0x7f41b40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.506+0000 7f41b909f640 1 -- 172.21.15.5:0/2003525880 shutdown_connections 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.506+0000 7f41b909f640 1 -- 172.21.15.5:0/2003525880 wait complete. 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.507+0000 7f41b909f640 1 Processor -- start 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.507+0000 7f41b909f640 1 -- start start 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b909f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102660 0x7f41b4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:54.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b909f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4104f90 0x7f41b4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b909f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b41078c0 0x7f41b419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b909f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41b410d880 con 0x7f41b4102660 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b909f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41b410d700 con 0x7f41b4104f90 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b909f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41b410da00 con 0x7f41b41078c0 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102660 0x7f41b4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102660 0x7f41b4197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41128/0 (socket says 172.21.15.5:41128) 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b2d9d640 1 -- 172.21.15.5:0/3376438793 learned_addr learned my addr 172.21.15.5:0/3376438793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b259c640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4104f90 0x7f41b4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b359e640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b41078c0 0x7f41b419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b2d9d640 1 -- 172.21.15.5:0/3376438793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b41078c0 msgr2=0x7f41b419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b2d9d640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b41078c0 0x7f41b419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b2d9d640 1 -- 172.21.15.5:0/3376438793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4104f90 msgr2=0x7f41b4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b2d9d640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4104f90 0x7f41b4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b2d9d640 1 -- 172.21.15.5:0/3376438793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b419fb00 con 0x7f41b4102660 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b359e640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b41078c0 0x7f41b419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b2d9d640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102660 0x7f41b4197ed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f41a400ea80 tx=0x7f41a400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41abfff640 1 -- 172.21.15.5:0/3376438793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a400ce40 con 0x7f41b4102660 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41abfff640 1 -- 172.21.15.5:0/3376438793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a4005020 con 0x7f41b4102660 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.508+0000 7f41b909f640 1 -- 172.21.15.5:0/3376438793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b419fdf0 con 0x7f41b4102660 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.509+0000 7f41abfff640 1 -- 172.21.15.5:0/3376438793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4011760 con 0x7f41b4102660 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.509+0000 7f41b909f640 1 -- 172.21.15.5:0/3376438793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b41a01f0 con 0x7f41b4102660 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.510+0000 7f41b909f640 1 -- 172.21.15.5:0/3376438793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b40fdd70 con 0x7f41b4102660 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.512+0000 7f41abfff640 1 -- 172.21.15.5:0/3376438793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f41a40051d0 con 0x7f41b4102660 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.512+0000 7f41abfff640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f41a00a2bb0 0x7f41a00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.513+0000 7f41b259c640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f41a00a2bb0 0x7f41a00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.513+0000 7f41abfff640 1 -- 172.21.15.5:0/3376438793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f41a40cf720 con 0x7f41b4102660 2024-09-11T22:21:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.513+0000 7f41b259c640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f41a00a2bb0 0x7f41a00a5070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f419c005fd0 tx=0x7f419c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.516+0000 7f41abfff640 1 -- 172.21.15.5:0/3376438793 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41a4016030 con 0x7f41b4102660 2024-09-11T22:21:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.691+0000 7f41b909f640 1 -- 172.21.15.5:0/3376438793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f41b402d7c0 con 0x7f41b4102660 2024-09-11T22:21:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.693+0000 7f41abfff640 1 -- 172.21.15.5:0/3376438793 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5286 (secure 0 0 0) 0x7f41a409c660 con 0x7f41b4102660 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 82 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:fsid b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-11T22:12:23.446853+0000 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-11T22:21:18.179555+0000 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 26 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2024-09-11T22:21:54.694 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/26218432,v1:172.21.15.5:6803/26218432] [v2:172.21.15.5:6804/26218432,v1:172.21.15.5:6805/26218432] exists,up e8ab7175-48c6-4c65-82eb-620196b29548 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/1131654571,v1:172.21.15.159:6801/1131654571] [v2:172.21.15.159:6802/1131654571,v1:172.21.15.159:6803/1131654571] exists,up 8a379e10-b214-42db-8b31-399b39b6cac1 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 18 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6800/34708812,v1:172.21.15.172:6801/34708812] [v2:172.21.15.172:6802/34708812,v1:172.21.15.172:6803/34708812] exists,up 5d23d4d6-1f97-4d0e-9c99-ae865e04ca24 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 24 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/708957835,v1:172.21.15.5:6811/708957835] [v2:172.21.15.5:6812/708957835,v1:172.21.15.5:6813/708957835] exists,up 35db3b3e-4a85-4b80-9df0-581f33718744 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 31 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/4176432146,v1:172.21.15.159:6809/4176432146] [v2:172.21.15.159:6810/4176432146,v1:172.21.15.159:6811/4176432146] exists,up 810e9588-d06b-493e-be95-d78591ab283a 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 36 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6808/1387739113,v1:172.21.15.172:6809/1387739113] [v2:172.21.15.172:6810/1387739113,v1:172.21.15.172:6811/1387739113] exists,up 0e0bf89e-0062-4032-8aec-6079003315a3 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 43 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/2494841053,v1:172.21.15.5:6819/2494841053] [v2:172.21.15.5:6820/2494841053,v1:172.21.15.5:6821/2494841053] exists,up f11a15a6-a7a8-4a42-ba15-20867f32d7ec 2024-09-11T22:21:54.695 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 48 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/3702567808,v1:172.21.15.159:6817/3702567808] [v2:172.21.15.159:6818/3702567808,v1:172.21.15.159:6819/3702567808] exists,up b7d70b5c-cb1b-4e19-b185-595a94380ee6 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 55 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6816/421514522,v1:172.21.15.172:6817/421514522] [v2:172.21.15.172:6818/421514522,v1:172.21.15.172:6819/421514522] exists,up 12a4fca4-4f15-41b1-8776-6722e1692b25 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 62 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/26202527,v1:172.21.15.5:6827/26202527] [v2:172.21.15.5:6828/26202527,v1:172.21.15.5:6829/26202527] exists,up a7758f2e-e665-4964-a5cf-f5298315934b 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 67 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/1772373268,v1:172.21.15.159:6825/1772373268] [v2:172.21.15.159:6826/1772373268,v1:172.21.15.159:6827/1772373268] exists,up 2e3cad92-29cf-43a6-84c0-4533de103ce8 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 74 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.172:6824/2463160464,v1:172.21.15.172:6825/2463160464] [v2:172.21.15.172:6826/2463160464,v1:172.21.15.172:6827/2463160464] exists,up 684e017e-bc9b-4fc5-afd8-89a3199c5653 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.0 [7,11] 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.c [2,4] 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.e [2,1] 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.f [8,11] 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1a [8,4] 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.6 [4,7] 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.b [3,9] 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.13 [4,9] 2024-09-11T22:21:54.696 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.14 [4,6] 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.37 [5,8] 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/3503484776 expires 2024-09-12T22:13:16.336721+0000 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/3503484776 expires 2024-09-12T22:13:16.336721+0000 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/981960152 expires 2024-09-12T22:13:16.336721+0000 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/497465376 expires 2024-09-12T22:13:16.336721+0000 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1875833568 expires 2024-09-12T22:12:52.569019+0000 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2378444675 expires 2024-09-12T22:12:52.569019+0000 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2381319073 expires 2024-09-12T22:13:16.336721+0000 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3381830267 expires 2024-09-12T22:12:52.569019+0000 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/1194241380 expires 2024-09-12T22:12:52.569019+0000 2024-09-11T22:21:54.697 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/1194241380 expires 2024-09-12T22:12:52.569019+0000 2024-09-11T22:21:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 -- 172.21.15.5:0/3376438793 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f41a00a2bb0 msgr2=0x7f41a00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f41a00a2bb0 0x7f41a00a5070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f419c005fd0 tx=0x7f419c005850 comp rx=0 tx=0).stop 2024-09-11T22:21:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 -- 172.21.15.5:0/3376438793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102660 msgr2=0x7f41b4197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102660 0x7f41b4197ed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f41a400ea80 tx=0x7f41a400ef40 comp rx=0 tx=0).stop 2024-09-11T22:21:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 -- 172.21.15.5:0/3376438793 shutdown_connections 2024-09-11T22:21:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f41a00a2bb0 0x7f41a00a5070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f41b41078c0 0x7f41b419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b4104f90 0x7f41b4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 --2- 172.21.15.5:0/3376438793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102660 0x7f41b4197ed0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 -- 172.21.15.5:0/3376438793 >> 172.21.15.5:0/3376438793 conn(0x7f41b40fc280 msgr2=0x7f41b4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 -- 172.21.15.5:0/3376438793 shutdown_connections 2024-09-11T22:21:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:54.695+0000 7f41b909f640 1 -- 172.21.15.5:0/3376438793 wait complete. 2024-09-11T22:21:55.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:54 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3376438793' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T22:21:55.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:54 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3376438793' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T22:21:55.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:54 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3376438793' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-11T22:21:55.351 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-11T22:21:55.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:21:55.625 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:55.746 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:55 smithi005 ceph-mon[30943]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:21:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:55 smithi159 ceph-mon[35534]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:21:56.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:55 smithi172 ceph-mon[35461]: pgmap v331: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:21:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.192+0000 7f634959c640 1 -- 172.21.15.5:0/3374315235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6344102570 msgr2=0x7f63441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.192+0000 7f634959c640 1 --2- 172.21.15.5:0/3374315235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6344102570 0x7f63441029b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6338009ca0 tx=0x7f633802a950 comp rx=0 tx=0).stop 2024-09-11T22:21:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.192+0000 7f634959c640 1 -- 172.21.15.5:0/3374315235 shutdown_connections 2024-09-11T22:21:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.192+0000 7f634959c640 1 --2- 172.21.15.5:0/3374315235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6344102ef0 0x7f634410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.192+0000 7f634959c640 1 --2- 172.21.15.5:0/3374315235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6344102570 0x7f63441029b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.192+0000 7f634959c640 1 --2- 172.21.15.5:0/3374315235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6344100e20 0x7f6344101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.192+0000 7f634959c640 1 -- 172.21.15.5:0/3374315235 >> 172.21.15.5:0/3374315235 conn(0x7f63440fc7d0 msgr2=0x7f63440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.192+0000 7f634959c640 1 -- 172.21.15.5:0/3374315235 shutdown_connections 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.192+0000 7f634959c640 1 -- 172.21.15.5:0/3374315235 wait complete. 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.193+0000 7f634959c640 1 Processor -- start 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.193+0000 7f634959c640 1 -- start start 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.193+0000 7f634959c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6344100e20 0x7f63441985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.193+0000 7f634959c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6344102570 0x7f6344198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.193+0000 7f634959c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6344102ef0 0x7f634419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.193+0000 7f634959c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6344110f50 con 0x7f6344102ef0 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.193+0000 7f634959c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6344110dd0 con 0x7f6344100e20 2024-09-11T22:21:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.193+0000 7f634959c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f63441110d0 con 0x7f6344102570 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f63437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6344102ef0 0x7f634419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f63437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6344102ef0 0x7f634419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41148/0 (socket says 172.21.15.5:41148) 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f63437fe640 1 -- 172.21.15.5:0/3857207752 learned_addr learned my addr 172.21.15.5:0/3857207752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f63427fc640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6344102570 0x7f6344198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f6342ffd640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6344100e20 0x7f63441985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f63437fe640 1 -- 172.21.15.5:0/3857207752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6344102570 msgr2=0x7f6344198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f63437fe640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6344102570 0x7f6344198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f63437fe640 1 -- 172.21.15.5:0/3857207752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6344100e20 msgr2=0x7f63441985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f63437fe640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6344100e20 0x7f63441985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f63437fe640 1 -- 172.21.15.5:0/3857207752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634419d560 con 0x7f6344102ef0 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f63437fe640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6344102ef0 0x7f634419ce80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f633c00c970 tx=0x7f633c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f632bfff640 1 -- 172.21.15.5:0/3857207752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c007bb0 con 0x7f6344102ef0 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f634959c640 1 -- 172.21.15.5:0/3857207752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634419d850 con 0x7f6344102ef0 2024-09-11T22:21:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f632bfff640 1 -- 172.21.15.5:0/3857207752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c007d50 con 0x7f6344102ef0 2024-09-11T22:21:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f632bfff640 1 -- 172.21.15.5:0/3857207752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c00f450 con 0x7f6344102ef0 2024-09-11T22:21:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.194+0000 7f634959c640 1 -- 172.21.15.5:0/3857207752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63441a5130 con 0x7f6344102ef0 2024-09-11T22:21:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.195+0000 7f634959c640 1 -- 172.21.15.5:0/3857207752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6344101200 con 0x7f6344102ef0 2024-09-11T22:21:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.196+0000 7f632bfff640 1 -- 172.21.15.5:0/3857207752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f633c004f60 con 0x7f6344102ef0 2024-09-11T22:21:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.197+0000 7f632bfff640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f632c0a2b80 0x7f632c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.197+0000 7f6342ffd640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f632c0a2b80 0x7f632c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.197+0000 7f632bfff640 1 -- 172.21.15.5:0/3857207752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f633c0cedc0 con 0x7f6344102ef0 2024-09-11T22:21:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.198+0000 7f6342ffd640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f632c0a2b80 0x7f632c0a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6334005fd0 tx=0x7f633400b590 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.201+0000 7f632bfff640 1 -- 172.21.15.5:0/3857207752 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f633c014030 con 0x7f6344102ef0 2024-09-11T22:21:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.415+0000 7f634959c640 1 -- 172.21.15.5:0/3857207752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f634410a0a0 con 0x7f6344102ef0 2024-09-11T22:21:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.416+0000 7f632bfff640 1 -- 172.21.15.5:0/3857207752 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7f633c09bd00 con 0x7f6344102ef0 2024-09-11T22:21:56.417 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:56.417 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-11T22:20:30:263672+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:20:30.263652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:21:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-11T22:21:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 -- 172.21.15.5:0/3857207752 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f632c0a2b80 msgr2=0x7f632c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f632c0a2b80 0x7f632c0a5040 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6334005fd0 tx=0x7f633400b590 comp rx=0 tx=0).stop 2024-09-11T22:21:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 -- 172.21.15.5:0/3857207752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6344102ef0 msgr2=0x7f634419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6344102ef0 0x7f634419ce80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f633c00c970 tx=0x7f633c00ce30 comp rx=0 tx=0).stop 2024-09-11T22:21:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 -- 172.21.15.5:0/3857207752 shutdown_connections 2024-09-11T22:21:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f632c0a2b80 0x7f632c0a5040 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6344102ef0 0x7f634419ce80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6344102570 0x7f6344198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 --2- 172.21.15.5:0/3857207752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6344100e20 0x7f63441985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 -- 172.21.15.5:0/3857207752 >> 172.21.15.5:0/3857207752 conn(0x7f63440fc7d0 msgr2=0x7f634406a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 -- 172.21.15.5:0/3857207752 shutdown_connections 2024-09-11T22:21:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:56.419+0000 7f634959c640 1 -- 172.21.15.5:0/3857207752 wait complete. 2024-09-11T22:21:57.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:56 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3857207752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:57.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:56 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3857207752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:57.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:56 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3857207752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:57.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:21:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:57.945 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:57 smithi005 ceph-mon[30943]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T22:21:57.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.944+0000 7f09c1443640 1 -- 172.21.15.5:0/3699565691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc1036c0 msgr2=0x7f09bc10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.944+0000 7f09c1443640 1 --2- 172.21.15.5:0/3699565691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc1036c0 0x7f09bc10bbb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009900 tx=0x7f09a402a480 comp rx=0 tx=0).stop 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.944+0000 7f09c1443640 1 -- 172.21.15.5:0/3699565691 shutdown_connections 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.944+0000 7f09c1443640 1 --2- 172.21.15.5:0/3699565691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09bc10c180 0x7f09bc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.944+0000 7f09c1443640 1 --2- 172.21.15.5:0/3699565691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc1036c0 0x7f09bc10bbb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.944+0000 7f09c1443640 1 --2- 172.21.15.5:0/3699565691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09bc102da0 0x7f09bc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.944+0000 7f09c1443640 1 -- 172.21.15.5:0/3699565691 >> 172.21.15.5:0/3699565691 conn(0x7f09bc0fc7a0 msgr2=0x7f09bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.944+0000 7f09c1443640 1 -- 172.21.15.5:0/3699565691 shutdown_connections 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.944+0000 7f09c1443640 1 -- 172.21.15.5:0/3699565691 wait complete. 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09c1443640 1 Processor -- start 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09c1443640 1 -- start start 2024-09-11T22:21:57.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09c1443640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09bc102da0 0x7f09bc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09c1443640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09bc1036c0 0x7f09bc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09c1443640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc10c180 0x7f09bc19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09c1443640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc111f60 con 0x7f09bc10c180 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09c1443640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc111de0 con 0x7f09bc1036c0 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09c1443640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc1120e0 con 0x7f09bc102da0 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc10c180 0x7f09bc19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc10c180 0x7f09bc19cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41166/0 (socket says 172.21.15.5:41166) 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09bb7fe640 1 -- 172.21.15.5:0/1279182799 learned_addr learned my addr 172.21.15.5:0/1279182799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09b3fff640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09bc1036c0 0x7f09bc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.945+0000 7f09baffd640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09bc102da0 0x7f09bc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09bb7fe640 1 -- 172.21.15.5:0/1279182799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09bc102da0 msgr2=0x7f09bc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09bb7fe640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09bc102da0 0x7f09bc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09bb7fe640 1 -- 172.21.15.5:0/1279182799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09bc1036c0 msgr2=0x7f09bc1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09bb7fe640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09bc1036c0 0x7f09bc1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09bb7fe640 1 -- 172.21.15.5:0/1279182799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09bc19d320 con 0x7f09bc10c180 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09bb7fe640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc10c180 0x7f09bc19cc40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f09b400eaa0 tx=0x7f09b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09b8ff9640 1 -- 172.21.15.5:0/1279182799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b400cdb0 con 0x7f09bc10c180 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09b8ff9640 1 -- 172.21.15.5:0/1279182799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b4005020 con 0x7f09bc10c180 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09c1443640 1 -- 172.21.15.5:0/1279182799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09bc19d610 con 0x7f09bc10c180 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09b8ff9640 1 -- 172.21.15.5:0/1279182799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b40116b0 con 0x7f09bc10c180 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.946+0000 7f09c1443640 1 -- 172.21.15.5:0/1279182799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09bc1a4ef0 con 0x7f09bc10c180 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.947+0000 7f09c1443640 1 -- 172.21.15.5:0/1279182799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09bc103180 con 0x7f09bc10c180 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.948+0000 7f09b8ff9640 1 -- 172.21.15.5:0/1279182799 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f09b4005560 con 0x7f09bc10c180 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.949+0000 7f09b8ff9640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09a80a2b80 0x7f09a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.949+0000 7f09baffd640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09a80a2b80 0x7f09a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.949+0000 7f09b8ff9640 1 -- 172.21.15.5:0/1279182799 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f09b40cf240 con 0x7f09bc10c180 2024-09-11T22:21:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.949+0000 7f09baffd640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09a80a2b80 0x7f09a80a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f09ac005e10 tx=0x7f09ac005da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:57.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:57.953+0000 7f09b8ff9640 1 -- 172.21.15.5:0/1279182799 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09b4016030 con 0x7f09bc10c180 2024-09-11T22:21:58.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:57 smithi172 ceph-mon[35461]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T22:21:58.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:57 smithi159 ceph-mon[35534]: pgmap v332: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-11T22:21:58.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.166+0000 7f09c1443640 1 -- 172.21.15.5:0/1279182799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09bc02d7c0 con 0x7f09bc10c180 2024-09-11T22:21:58.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.166+0000 7f09b8ff9640 1 -- 172.21.15.5:0/1279182799 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11967 (secure 0 0 0) 0x7f09b409c180 con 0x7f09bc10c180 2024-09-11T22:21:58.167 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:58.168 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-11T22:20:30:263672+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:20:30.263652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:21:58.168 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-11T22:21:58.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 -- 172.21.15.5:0/1279182799 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09a80a2b80 msgr2=0x7f09a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:58.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09a80a2b80 0x7f09a80a5040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f09ac005e10 tx=0x7f09ac005da0 comp rx=0 tx=0).stop 2024-09-11T22:21:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 -- 172.21.15.5:0/1279182799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc10c180 msgr2=0x7f09bc19cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc10c180 0x7f09bc19cc40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f09b400eaa0 tx=0x7f09b400ef60 comp rx=0 tx=0).stop 2024-09-11T22:21:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 -- 172.21.15.5:0/1279182799 shutdown_connections 2024-09-11T22:21:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f09a80a2b80 0x7f09a80a5040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09bc10c180 0x7f09bc19cc40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09bc1036c0 0x7f09bc1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 --2- 172.21.15.5:0/1279182799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f09bc102da0 0x7f09bc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 -- 172.21.15.5:0/1279182799 >> 172.21.15.5:0/1279182799 conn(0x7f09bc0fc7a0 msgr2=0x7f09bc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 -- 172.21.15.5:0/1279182799 shutdown_connections 2024-09-11T22:21:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:58.169+0000 7f09c1443640 1 -- 172.21.15.5:0/1279182799 wait complete. 2024-09-11T22:21:58.983 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:58 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1279182799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:59.021 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:21:59.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:58 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1279182799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:59.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:58 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1279182799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:21:59.286 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:21:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.809+0000 7f82b4275640 1 -- 172.21.15.5:0/4040220453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ac100a80 msgr2=0x7f82ac100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.809+0000 7f82b4275640 1 --2- 172.21.15.5:0/4040220453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ac100a80 0x7f82ac100ee0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f829c009930 tx=0x7f829c02a420 comp rx=0 tx=0).stop 2024-09-11T22:21:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.809+0000 7f82b4275640 1 -- 172.21.15.5:0/4040220453 shutdown_connections 2024-09-11T22:21:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.809+0000 7f82b4275640 1 --2- 172.21.15.5:0/4040220453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ac101420 0x7f82ac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.809+0000 7f82b4275640 1 --2- 172.21.15.5:0/4040220453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ac100a80 0x7f82ac100ee0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.809+0000 7f82b4275640 1 --2- 172.21.15.5:0/4040220453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82ac106a80 0x7f82ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.809+0000 7f82b4275640 1 -- 172.21.15.5:0/4040220453 >> 172.21.15.5:0/4040220453 conn(0x7f82ac0fc7a0 msgr2=0x7f82ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.810+0000 7f82b4275640 1 -- 172.21.15.5:0/4040220453 shutdown_connections 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.810+0000 7f82b4275640 1 -- 172.21.15.5:0/4040220453 wait complete. 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.810+0000 7f82b4275640 1 Processor -- start 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.810+0000 7f82b4275640 1 -- start start 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b4275640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ac100a80 0x7f82ac1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b4275640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82ac101420 0x7f82ac1978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b4275640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ac106a80 0x7f82ac19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b4275640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac110e80 con 0x7f82ac106a80 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b4275640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac110d00 con 0x7f82ac100a80 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b4275640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac111000 con 0x7f82ac101420 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b2812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ac106a80 0x7f82ac19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b2812640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ac106a80 0x7f82ac19bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41198/0 (socket says 172.21.15.5:41198) 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b2812640 1 -- 172.21.15.5:0/1276456049 learned_addr learned my addr 172.21.15.5:0/1276456049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b2011640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ac100a80 0x7f82ac1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b1810640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82ac101420 0x7f82ac1978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b2812640 1 -- 172.21.15.5:0/1276456049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82ac101420 msgr2=0x7f82ac1978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b2812640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82ac101420 0x7f82ac1978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b2812640 1 -- 172.21.15.5:0/1276456049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ac100a80 msgr2=0x7f82ac1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b2812640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ac100a80 0x7f82ac1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b2812640 1 -- 172.21.15.5:0/1276456049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac19c350 con 0x7f82ac106a80 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b1810640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82ac101420 0x7f82ac1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b2812640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ac106a80 0x7f82ac19bc70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f82a400eaa0 tx=0x7f82a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82a37fe640 1 -- 172.21.15.5:0/1276456049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a400cdb0 con 0x7f82ac106a80 2024-09-11T22:21:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82a37fe640 1 -- 172.21.15.5:0/1276456049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a4005020 con 0x7f82ac106a80 2024-09-11T22:21:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82b4275640 1 -- 172.21.15.5:0/1276456049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac19c640 con 0x7f82ac106a80 2024-09-11T22:21:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.811+0000 7f82a37fe640 1 -- 172.21.15.5:0/1276456049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a40116b0 con 0x7f82ac106a80 2024-09-11T22:21:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.812+0000 7f82b4275640 1 -- 172.21.15.5:0/1276456049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1a3f20 con 0x7f82ac106a80 2024-09-11T22:21:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.813+0000 7f82b4275640 1 -- 172.21.15.5:0/1276456049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82ac100ee0 con 0x7f82ac106a80 2024-09-11T22:21:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.814+0000 7f82a37fe640 1 -- 172.21.15.5:0/1276456049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f82a4005560 con 0x7f82ac106a80 2024-09-11T22:21:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.814+0000 7f82a37fe640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f82980a2c00 0x7f82980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:21:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.815+0000 7f82b2011640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f82980a2c00 0x7f82980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:21:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.815+0000 7f82a37fe640 1 -- 172.21.15.5:0/1276456049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f82a40cf280 con 0x7f82ac106a80 2024-09-11T22:21:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.815+0000 7f82b2011640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f82980a2c00 0x7f82980a50c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f82a80058c0 tx=0x7f82a800a250 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:21:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.819+0000 7f82a37fe640 1 -- 172.21.15.5:0/1276456049 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82a4016030 con 0x7f82ac106a80 2024-09-11T22:21:59.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.994+0000 7f82b4275640 1 -- 172.21.15.5:0/1276456049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82ac108a60 con 0x7f82ac106a80 2024-09-11T22:21:59.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:21:59 smithi005 ceph-mon[30943]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:21:59.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.996+0000 7f82a37fe640 1 -- 172.21.15.5:0/1276456049 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22697 (secure 0 0 0) 0x7f82a409c1c0 con 0x7f82ac106a80 2024-09-11T22:21:59.997 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:21:59.997 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":82,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:21:18.179555+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:21:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 -- 172.21.15.5:0/1276456049 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f82980a2c00 msgr2=0x7f82980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f82980a2c00 0x7f82980a50c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f82a80058c0 tx=0x7f82a800a250 comp rx=0 tx=0).stop 2024-09-11T22:21:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 -- 172.21.15.5:0/1276456049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ac106a80 msgr2=0x7f82ac19bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:21:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ac106a80 0x7f82ac19bc70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f82a400eaa0 tx=0x7f82a400ef60 comp rx=0 tx=0).stop 2024-09-11T22:21:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 -- 172.21.15.5:0/1276456049 shutdown_connections 2024-09-11T22:21:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f82980a2c00 0x7f82980a50c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:21:59.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ac106a80 0x7f82ac19bc70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:00.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82ac101420 0x7f82ac1978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:00.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 --2- 172.21.15.5:0/1276456049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ac100a80 0x7f82ac1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:00.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 -- 172.21.15.5:0/1276456049 >> 172.21.15.5:0/1276456049 conn(0x7f82ac0fc7a0 msgr2=0x7f82ac0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:00.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 -- 172.21.15.5:0/1276456049 shutdown_connections 2024-09-11T22:22:00.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:21:59.998+0000 7f82b4275640 1 -- 172.21.15.5:0/1276456049 wait complete. 2024-09-11T22:22:00.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:21:59 smithi159 ceph-mon[35534]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:22:00.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:21:59 smithi172 ceph-mon[35461]: pgmap v333: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:22:00.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T22:20:18.900781+0000', 'modified': '2024-09-11T22:20:30.263652+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24500, 'mds_1': 24473, 'mds_2': 24454, 'mds_3': 24452, 'mds_4': 24436}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/2458034474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 2458034474}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 2458034474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/2232645952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 2232645952}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 2232645952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6833/1186865242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6832', 'nonce': 1186865242}, {'type': 'v1', 'addr': '172.21.15.172:6833', 'nonce': 1186865242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/4149321843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 4149321843}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 4149321843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/2975110123', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 2975110123}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 2975110123}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.172:6837/990138139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 990138139}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 990138139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.159:6835/1262598700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1262598700}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1262598700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.172:6839/1937249261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 1937249261}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 1937249261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.159:6837/2315339117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 2315339117}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 2315339117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.159:6839/994288762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 994288762}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 994288762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24500, 24473, 24454, 24452, 24436]} 2024-09-11T22:22:00.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T22:22:00.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T22:22:01.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:00 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1276456049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:01.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:00 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1276456049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:01.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:00 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1276456049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:01.120 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.631+0000 7f04041e7640 1 -- 172.21.15.5:0/1906703342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc102da0 msgr2=0x7f03fc103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.631+0000 7f04041e7640 1 --2- 172.21.15.5:0/1906703342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc102da0 0x7f03fc103180 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009900 tx=0x7f03f802a480 comp rx=0 tx=0).stop 2024-09-11T22:22:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 -- 172.21.15.5:0/1906703342 shutdown_connections 2024-09-11T22:22:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 --2- 172.21.15.5:0/1906703342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03fc10c180 0x7f03fc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 --2- 172.21.15.5:0/1906703342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03fc1036c0 0x7f03fc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 --2- 172.21.15.5:0/1906703342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc102da0 0x7f03fc103180 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 -- 172.21.15.5:0/1906703342 >> 172.21.15.5:0/1906703342 conn(0x7f03fc0fc7a0 msgr2=0x7f03fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 -- 172.21.15.5:0/1906703342 shutdown_connections 2024-09-11T22:22:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 -- 172.21.15.5:0/1906703342 wait complete. 2024-09-11T22:22:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 Processor -- start 2024-09-11T22:22:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 -- start start 2024-09-11T22:22:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03fc102da0 0x7f03fc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.632+0000 7f04041e7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03fc1036c0 0x7f03fc1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f04041e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc10c180 0x7f03fc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f04041e7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc111f60 con 0x7f03fc10c180 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f04041e7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc111de0 con 0x7f03fc1036c0 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f04041e7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc1120e0 con 0x7f03fc102da0 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0402784640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc10c180 0x7f03fc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0402784640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc10c180 0x7f03fc19cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48144/0 (socket says 172.21.15.5:48144) 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0402784640 1 -- 172.21.15.5:0/1708195831 learned_addr learned my addr 172.21.15.5:0/1708195831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0401782640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03fc1036c0 0x7f03fc1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0401f83640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03fc102da0 0x7f03fc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0402784640 1 -- 172.21.15.5:0/1708195831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03fc102da0 msgr2=0x7f03fc198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0402784640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03fc102da0 0x7f03fc198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0402784640 1 -- 172.21.15.5:0/1708195831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03fc1036c0 msgr2=0x7f03fc1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0402784640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03fc1036c0 0x7f03fc1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0402784640 1 -- 172.21.15.5:0/1708195831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc19d410 con 0x7f03fc10c180 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0401782640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03fc1036c0 0x7f03fc1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0401f83640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03fc102da0 0x7f03fc198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f0402784640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc10c180 0x7f03fc19cd30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f03f400eb10 tx=0x7f03f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f03f2ffd640 1 -- 172.21.15.5:0/1708195831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4004b20 con 0x7f03fc10c180 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f03f2ffd640 1 -- 172.21.15.5:0/1708195831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f4004cc0 con 0x7f03fc10c180 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f04041e7640 1 -- 172.21.15.5:0/1708195831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc19d700 con 0x7f03fc10c180 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f03f2ffd640 1 -- 172.21.15.5:0/1708195831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4010760 con 0x7f03fc10c180 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.633+0000 7f04041e7640 1 -- 172.21.15.5:0/1708195831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc1a4fe0 con 0x7f03fc10c180 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.636+0000 7f03f2ffd640 1 -- 172.21.15.5:0/1708195831 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f03f4005020 con 0x7f03fc10c180 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.636+0000 7f04065c0740 1 Processor -- start 2024-09-11T22:22:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f04065c0740 1 -- start start 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f04065c0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f69d0 0x55f8189f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f04065c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f72f0 0x55f818a01670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f04065c0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f818a03fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f04065c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f81895d090 con 0x55f8189f72f0 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f04065c0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f818904a70 con 0x55f8189f69d0 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f04065c0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f818959050 con 0x55f818a01bb0 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401f83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f72f0 0x55f818a01670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401f83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f72f0 0x55f818a01670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48160/0 (socket says 172.21.15.5:48160) 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401f83640 1 -- 172.21.15.5:0/3984589307 learned_addr learned my addr 172.21.15.5:0/3984589307 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401782640 1 --2- 172.21.15.5:0/3984589307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f69d0 0x55f8189f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401782640 1 --2- 172.21.15.5:0/3984589307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f818a03fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401f83640 1 -- 172.21.15.5:0/3984589307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 msgr2=0x55f818a03fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401f83640 1 --2- 172.21.15.5:0/3984589307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f818a03fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401f83640 1 -- 172.21.15.5:0/3984589307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f69d0 msgr2=0x55f8189f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401f83640 1 --2- 172.21.15.5:0/3984589307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f69d0 0x55f8189f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f03f2ffd640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03e80a2b80 0x7f03e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401f83640 1 -- 172.21.15.5:0/3984589307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f818911130 con 0x55f8189f72f0 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.637+0000 7f0401782640 1 --2- 172.21.15.5:0/3984589307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f818a03fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.638+0000 7f0401782640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03e80a2b80 0x7f03e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.638+0000 7f03f2ffd640 1 -- 172.21.15.5:0/1708195831 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f03f40cf740 con 0x7f03fc10c180 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.638+0000 7f0401f83640 1 --2- 172.21.15.5:0/3984589307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f72f0 0x55f818a01670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f03f8033000 tx=0x7f03f8005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d274e2d0a5ddaa22 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.638+0000 7f03e7fff640 1 -- 172.21.15.5:0/3984589307 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f803f070 con 0x55f8189f72f0 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.638+0000 7f03e7fff640 1 -- 172.21.15.5:0/3984589307 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f8004030 con 0x55f8189f72f0 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.638+0000 7f03e7fff640 1 -- 172.21.15.5:0/3984589307 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8035070 con 0x55f8189f72f0 2024-09-11T22:22:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 -- 172.21.15.5:0/3984589307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f72f0 msgr2=0x55f818a01670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f0401782640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03e80a2b80 0x7f03e80a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55f8189f42e0 tx=0x7f03ec014040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 --2- 172.21.15.5:0/3984589307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f72f0 0x55f818a01670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f03f8033000 tx=0x7f03f8005e60 comp rx=0 tx=0).stop 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 -- 172.21.15.5:0/3984589307 shutdown_connections 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 --2- 172.21.15.5:0/3984589307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f818a03fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 --2- 172.21.15.5:0/3984589307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f72f0 0x55f818a01670 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 --2- 172.21.15.5:0/3984589307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f69d0 0x55f8189f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 -- 172.21.15.5:0/3984589307 >> 172.21.15.5:0/3984589307 conn(0x55f8189f2f60 msgr2=0x55f8189f3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 -- 172.21.15.5:0/3984589307 shutdown_connections 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 -- 172.21.15.5:0/3984589307 wait complete. 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 -- start start 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.639+0000 7f04065c0740 1 Processor -- start 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f04065c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a01450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f04065c0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 0x55f8189fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f04065c0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f8189fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f04065c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f818a04780 con 0x55f8189f69d0 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f04065c0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f818917f20 con 0x55f8189f72f0 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f04065c0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f818a04900 con 0x55f818a01bb0 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f04065c0740 10 client.0 _ll_register_callbacks cb 0x55f8188e1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401f83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a01450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401f83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a01450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48162/0 (socket says 172.21.15.5:48162) 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401f83640 1 -- 172.21.15.5:0/3399658098 learned_addr learned my addr 172.21.15.5:0/3399658098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401782640 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 0x55f8189fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401f83640 1 -- 172.21.15.5:0/3399658098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 msgr2=0x55f8189fece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0402784640 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f8189fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401f83640 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f8189fece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401f83640 1 -- 172.21.15.5:0/3399658098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 msgr2=0x55f8189fe7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401f83640 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 0x55f8189fe7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401f83640 1 -- 172.21.15.5:0/3399658098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8186ce2b0 con 0x55f8189f69d0 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401782640 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 0x55f8189fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f0401f83640 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a01450 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f03f802a9a0 tx=0x7f03f8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccd2f44528223af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.640+0000 7f03e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f03e57fa640 1 -- 172.21.15.5:0/3399658098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8035070 con 0x55f8189f69d0 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f04065c0740 1 -- 172.21.15.5:0/3399658098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f8004030 con 0x55f8189f69d0 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f03e57fa640 1 -- 172.21.15.5:0/3399658098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f03f80317b0 con 0x55f8189f69d0 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f03e57fa640 1 -- 172.21.15.5:0/3399658098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03f803f040 con 0x55f8189f69d0 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f04065c0740 10 client.15099 fetch_fsmap learned FSMap version 27 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f04065c0740 1 -- 172.21.15.5:0/3399658098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f818a10010 con 0x55f8189f69d0 2024-09-11T22:22:01.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f03e57fa640 1 -- 172.21.15.5:0/3399658098 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f8004030 con 0x55f8189f69d0 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f03e57fa640 1 -- 172.21.15.5:0/3399658098 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f03f8004750 con 0x55f8189f69d0 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f04065c0740 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f04065c0740 10 client.15099 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f04065c0740 20 client.15099 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f04065c0740 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f818a12170 0x55f818a135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f04065c0740 4 client.15099 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.641+0000 7f04065c0740 1 -- 172.21.15.5:0/3399658098 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f81896a600 con 0x55f818a12170 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.642+0000 7f0401782640 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f818a12170 0x55f818a135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.642+0000 7f0401782640 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f818a12170 0x55f818a135e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9ba6dad5606ec7a server_cookie=213f64028c80d5a0 in_seq=0 out_seq=0 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.642+0000 7f03e57fa640 10 client.15099 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f03e57fa640 1 -- 172.21.15.5:0/3399658098 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f81896a600 con 0x55f818a12170 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f03e57fa640 10 client.15099 handle_command_reply: tid=0 2024-09-11T22:22:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 client.15099 shutdown 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 -- 172.21.15.5:0/3399658098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 msgr2=0x55f818a01450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a01450 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f03f802a9a0 tx=0x7f03f8031400 comp rx=0 tx=0).stop 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 -- 172.21.15.5:0/3399658098 shutdown_connections 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f818a12170 0x55f818a135e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f8189fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 0x55f8189fe7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 --2- 172.21.15.5:0/3399658098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a01450 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f03e57fa640 0 client.15099 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 -- 172.21.15.5:0/3399658098 >> 172.21.15.5:0/3399658098 conn(0x55f8189f2f60 msgr2=0x55f8189f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 -- 172.21.15.5:0/3399658098 shutdown_connections 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 1 -- 172.21.15.5:0/3399658098 wait complete. 2024-09-11T22:22:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.643+0000 7f04065c0740 20 client.15099 trim_cache size 0 max 16384 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.661+0000 7f04065c0740 1 Processor -- start 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.661+0000 7f04065c0740 1 -- start start 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.661+0000 7f04065c0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f69d0 0x55f818a135e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.661+0000 7f04065c0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f8189f72f0 0x55f8189f9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.661+0000 7f04065c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f818a01bb0 0x55f8189fba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.661+0000 7f04065c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f818a11920 con 0x55f818a01bb0 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.661+0000 7f04065c0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f818a11ff0 con 0x55f8189f69d0 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.661+0000 7f04065c0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f818a126c0 con 0x55f8189f72f0 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0402784640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f818a01bb0 0x55f8189fba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0402784640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f818a01bb0 0x55f8189fba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48176/0 (socket says 172.21.15.5:48176) 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0402784640 1 -- 172.21.15.5:0/2106752773 learned_addr learned my addr 172.21.15.5:0/2106752773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0401782640 1 --2- 172.21.15.5:0/2106752773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f8189f72f0 0x55f8189f9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0401f83640 1 --2- 172.21.15.5:0/2106752773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f69d0 0x55f818a135e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0402784640 1 -- 172.21.15.5:0/2106752773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f8189f72f0 msgr2=0x55f8189f9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0402784640 1 --2- 172.21.15.5:0/2106752773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f8189f72f0 0x55f8189f9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0402784640 1 -- 172.21.15.5:0/2106752773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f69d0 msgr2=0x55f818a135e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0402784640 1 --2- 172.21.15.5:0/2106752773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f69d0 0x55f818a135e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0402784640 1 -- 172.21.15.5:0/2106752773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8189f2d90 con 0x55f818a01bb0 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0401782640 1 --2- 172.21.15.5:0/2106752773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f8189f72f0 0x55f8189f9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f0402784640 1 --2- 172.21.15.5:0/2106752773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f818a01bb0 0x55f8189fba00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f03f40a4660 tx=0x7f03f40981c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af801fed4b7c6ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f03e77fe640 1 -- 172.21.15.5:0/2106752773 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f40b7030 con 0x55f818a01bb0 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f03e77fe640 1 -- 172.21.15.5:0/2106752773 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f409c620 con 0x55f818a01bb0 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.662+0000 7f03e77fe640 1 -- 172.21.15.5:0/2106752773 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f40b4070 con 0x55f818a01bb0 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 -- 172.21.15.5:0/2106752773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f818a01bb0 msgr2=0x55f8189fba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 --2- 172.21.15.5:0/2106752773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f818a01bb0 0x55f8189fba00 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f03f40a4660 tx=0x7f03f40981c0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 -- 172.21.15.5:0/2106752773 shutdown_connections 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 --2- 172.21.15.5:0/2106752773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f818a01bb0 0x55f8189fba00 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 --2- 172.21.15.5:0/2106752773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f8189f72f0 0x55f8189f9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 --2- 172.21.15.5:0/2106752773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f69d0 0x55f818a135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 -- 172.21.15.5:0/2106752773 >> 172.21.15.5:0/2106752773 conn(0x55f8189f2f60 msgr2=0x55f8189f9610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 -- 172.21.15.5:0/2106752773 shutdown_connections 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 -- 172.21.15.5:0/2106752773 wait complete. 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 -- start start 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.663+0000 7f04065c0740 1 Processor -- start 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f04065c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a1bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f04065c0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 0x55f818a1c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f04065c0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f818a15160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f04065c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f818a13460 con 0x55f8189f69d0 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f04065c0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f818a12d90 con 0x55f8189f72f0 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f04065c0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f818a13b30 con 0x55f818a01bb0 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f04065c0740 10 client.0 _ll_register_callbacks cb 0x7f03ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f0401f83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a1bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f0401f83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a1bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48186/0 (socket says 172.21.15.5:48186) 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f0401f83640 1 -- 172.21.15.5:0/3208050384 learned_addr learned my addr 172.21.15.5:0/3208050384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f0401782640 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 0x55f818a1c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f0401f83640 1 -- 172.21.15.5:0/3208050384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 msgr2=0x55f818a15160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f0401f83640 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f818a15160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f0401f83640 1 -- 172.21.15.5:0/3208050384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 msgr2=0x55f818a1c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f0401f83640 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 0x55f818a1c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f0401f83640 1 -- 172.21.15.5:0/3208050384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8189f2810 con 0x55f8189f69d0 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f0401f83640 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a1bc30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f03f8033000 tx=0x7f03f80052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3018e088221733f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.664+0000 7f03e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f03e57fa640 1 -- 172.21.15.5:0/3208050384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f805f060 con 0x55f8189f69d0 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f03e57fa640 1 -- 172.21.15.5:0/3208050384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f03f80035f0 con 0x55f8189f69d0 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f04065c0740 1 -- 172.21.15.5:0/3208050384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f409c620 con 0x55f8189f69d0 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f03e57fa640 1 -- 172.21.15.5:0/3208050384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03f8035070 con 0x55f8189f69d0 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f04065c0740 10 client.15105 fetch_fsmap learned FSMap version 27 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f04065c0740 1 -- 172.21.15.5:0/3208050384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f818a1a250 con 0x55f8189f69d0 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f03e57fa640 1 -- 172.21.15.5:0/3208050384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f409c620 con 0x55f8189f69d0 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f03e57fa640 1 -- 172.21.15.5:0/3208050384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f03f80318e0 con 0x55f8189f69d0 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f04065c0740 10 client.15105 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f04065c0740 10 client.15105 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f04065c0740 20 client.15105 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f04065c0740 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f818a80fc0 0x55f818a7fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f04065c0740 4 client.15105 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f04065c0740 1 -- 172.21.15.5:0/3208050384 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "status"}) -- 0x55f818964b10 con 0x55f818a80fc0 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.665+0000 7f0401782640 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f818a80fc0 0x55f818a7fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f0401782640 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f818a80fc0 0x55f818a7fcd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2eb254f6c13f0bb server_cookie=2d8de17fc5681f61 in_seq=0 out_seq=0 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f03e57fa640 10 client.15105 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f03e57fa640 1 -- 172.21.15.5:0/3208050384 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f818964b10 con 0x55f818a80fc0 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f03e57fa640 10 client.15105 handle_command_reply: tid=0 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 client.15105 shutdown 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 -- 172.21.15.5:0/3208050384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 msgr2=0x55f818a1bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a1bc30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f03f8033000 tx=0x7f03f80052d0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 -- 172.21.15.5:0/3208050384 shutdown_connections 2024-09-11T22:22:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f818a80fc0 0x55f818a7fcd0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f818a01bb0 0x55f818a15160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f8189f72f0 0x55f818a1c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 --2- 172.21.15.5:0/3208050384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f8189f69d0 0x55f818a1bc30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f03e57fa640 0 client.15105 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:22:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 -- 172.21.15.5:0/3208050384 >> 172.21.15.5:0/3208050384 conn(0x55f8189f2f60 msgr2=0x55f8189f9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 -- 172.21.15.5:0/3208050384 shutdown_connections 2024-09-11T22:22:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.666+0000 7f04065c0740 1 -- 172.21.15.5:0/3208050384 wait complete. 2024-09-11T22:22:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04065c0740 20 client.15105 trim_cache size 0 max 16384 2024-09-11T22:22:01.680 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:01.680 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:22:01.680 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-11T22:22:01.680 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24436, 2024-09-11T22:22:01.680 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:22:01.680 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:22:01.681 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:22:01.681 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 92.404805190000005, 2024-09-11T22:22:01.681 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-11T22:22:01.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-09-11T22:22:01.681 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T22:22:01.681 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 127.780886088 2024-09-11T22:22:01.681 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 -- 172.21.15.5:0/1708195831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03e80a2b80 msgr2=0x7f03e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03e80a2b80 0x7f03e80a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55f8189f42e0 tx=0x7f03ec014040 comp rx=0 tx=0).stop 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 -- 172.21.15.5:0/1708195831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc10c180 msgr2=0x7f03fc19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc10c180 0x7f03fc19cd30 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f03f400eb10 tx=0x7f03f400c2d0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 -- 172.21.15.5:0/1708195831 shutdown_connections 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f03e80a2b80 0x7f03e80a5040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc10c180 0x7f03fc19cd30 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03fc1036c0 0x7f03fc1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 --2- 172.21.15.5:0/1708195831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03fc102da0 0x7f03fc198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 -- 172.21.15.5:0/1708195831 >> 172.21.15.5:0/1708195831 conn(0x7f03fc0fc7a0 msgr2=0x7f03fc0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 -- 172.21.15.5:0/1708195831 shutdown_connections 2024-09-11T22:22:01.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:01.667+0000 7f04041e7640 1 -- 172.21.15.5:0/1708195831 wait complete. 2024-09-11T22:22:01.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:01 smithi005 ceph-mon[30943]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T22:22:02.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:01 smithi159 ceph-mon[35534]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T22:22:02.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:01 smithi172 ceph-mon[35461]: pgmap v334: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 341 B/s wr, 12 op/s 2024-09-11T22:22:02.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T22:22:02.857 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 -- 172.21.15.5:0/463841798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1218100e20 msgr2=0x7f1218101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 --2- 172.21.15.5:0/463841798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1218100e20 0x7f1218101200 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1214009900 tx=0x7f121402a480 comp rx=0 tx=0).stop 2024-09-11T22:22:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 -- 172.21.15.5:0/463841798 shutdown_connections 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 --2- 172.21.15.5:0/463841798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1218102ef0 0x7f121810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 --2- 172.21.15.5:0/463841798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1218102570 0x7f12181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 --2- 172.21.15.5:0/463841798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1218100e20 0x7f1218101200 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 -- 172.21.15.5:0/463841798 >> 172.21.15.5:0/463841798 conn(0x7f12180fc7d0 msgr2=0x7f12180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 -- 172.21.15.5:0/463841798 shutdown_connections 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 -- 172.21.15.5:0/463841798 wait complete. 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 Processor -- start 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.426+0000 7f121fcbc640 1 -- start start 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121fcbc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1218100e20 0x7f1218199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121fcbc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1218102570 0x7f12181998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121fcbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1218102ef0 0x7f12181a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121fcbc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1218110f80 con 0x7f1218102ef0 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121fcbc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1218110e00 con 0x7f1218102570 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121fcbc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1218111100 con 0x7f1218100e20 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121e259640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1218102ef0 0x7f12181a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121e259640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1218102ef0 0x7f12181a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48212/0 (socket says 172.21.15.5:48212) 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121e259640 1 -- 172.21.15.5:0/3681007904 learned_addr learned my addr 172.21.15.5:0/3681007904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121d257640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1218102570 0x7f12181998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121da58640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1218100e20 0x7f1218199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121e259640 1 -- 172.21.15.5:0/3681007904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1218100e20 msgr2=0x7f1218199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121e259640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1218100e20 0x7f1218199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121e259640 1 -- 172.21.15.5:0/3681007904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1218102570 msgr2=0x7f12181998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121e259640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1218102570 0x7f12181998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121e259640 1 -- 172.21.15.5:0/3681007904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12181a4940 con 0x7f1218102ef0 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.427+0000 7f121e259640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1218102ef0 0x7f12181a4400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f121000eaa0 tx=0x7f121000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.428+0000 7f120effd640 1 -- 172.21.15.5:0/3681007904 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121000cdb0 con 0x7f1218102ef0 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.428+0000 7f120effd640 1 -- 172.21.15.5:0/3681007904 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1210005020 con 0x7f1218102ef0 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.428+0000 7f121fcbc640 1 -- 172.21.15.5:0/3681007904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12181a4b10 con 0x7f1218102ef0 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.428+0000 7f120effd640 1 -- 172.21.15.5:0/3681007904 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12100116b0 con 0x7f1218102ef0 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.428+0000 7f121fcbc640 1 -- 172.21.15.5:0/3681007904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12181a4fd0 con 0x7f1218102ef0 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.430+0000 7f120effd640 1 -- 172.21.15.5:0/3681007904 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1210005560 con 0x7f1218102ef0 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.430+0000 7f1222095740 1 Processor -- start 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.430+0000 7f1222095740 1 -- start start 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.430+0000 7f1222095740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bec8b0 0x557bb0becc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f1222095740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bed1d0 0x557bb0bfba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f1222095740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bfbfc0 0x557bb0bfe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f1222095740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557bb0b53090 con 0x557bb0bec8b0 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f1222095740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557bb0afaa70 con 0x557bb0bed1d0 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f1222095740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557bb0b4f050 con 0x557bb0bfbfc0 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121d257640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bec8b0 0x557bb0becc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f120effd640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12040a2b80 0x7f12040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121d257640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bec8b0 0x557bb0becc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48216/0 (socket says 172.21.15.5:48216) 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121d257640 1 -- 172.21.15.5:0/1670117387 learned_addr learned my addr 172.21.15.5:0/1670117387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121d257640 1 --2- 172.21.15.5:0/1670117387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bfbfc0 0x557bb0bfe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121da58640 1 --2- 172.21.15.5:0/1670117387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bed1d0 0x557bb0bfba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121e259640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12040a2b80 0x7f12040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121d257640 1 -- 172.21.15.5:0/1670117387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bfbfc0 msgr2=0x557bb0bfe3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121d257640 1 --2- 172.21.15.5:0/1670117387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bfbfc0 0x557bb0bfe3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121d257640 1 -- 172.21.15.5:0/1670117387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bed1d0 msgr2=0x557bb0bfba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121d257640 1 --2- 172.21.15.5:0/1670117387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bed1d0 0x557bb0bfba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121d257640 1 -- 172.21.15.5:0/1670117387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb0b07130 con 0x557bb0bec8b0 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f120effd640 1 -- 172.21.15.5:0/3681007904 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f12100cf1a0 con 0x7f1218102ef0 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121d257640 1 --2- 172.21.15.5:0/1670117387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bec8b0 0x557bb0becc90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1208004650 tx=0x7f1208004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdbd26faa3da6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f1203fff640 1 -- 172.21.15.5:0/1670117387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120800be60 con 0x557bb0bec8b0 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f121e259640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12040a2b80 0x7f12040a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f121009ba20 tx=0x7f121006aa10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f1203fff640 1 -- 172.21.15.5:0/1670117387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1208005500 con 0x557bb0bec8b0 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.431+0000 7f1203fff640 1 -- 172.21.15.5:0/1670117387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12080106c0 con 0x557bb0bec8b0 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.432+0000 7f1222095740 1 -- 172.21.15.5:0/1670117387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bec8b0 msgr2=0x557bb0becc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.432+0000 7f1222095740 1 --2- 172.21.15.5:0/1670117387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bec8b0 0x557bb0becc90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1208004650 tx=0x7f1208004250 comp rx=0 tx=0).stop 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.432+0000 7f1222095740 1 -- 172.21.15.5:0/1670117387 shutdown_connections 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.432+0000 7f1222095740 1 --2- 172.21.15.5:0/1670117387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bfbfc0 0x557bb0bfe3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.432+0000 7f1222095740 1 --2- 172.21.15.5:0/1670117387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bed1d0 0x557bb0bfba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.432+0000 7f1222095740 1 --2- 172.21.15.5:0/1670117387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bec8b0 0x557bb0becc90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.432+0000 7f1222095740 1 -- 172.21.15.5:0/1670117387 >> 172.21.15.5:0/1670117387 conn(0x557bb0be8f60 msgr2=0x557bb0be9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.432+0000 7f1222095740 1 -- 172.21.15.5:0/1670117387 shutdown_connections 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.432+0000 7f1222095740 1 -- 172.21.15.5:0/1670117387 wait complete. 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f1222095740 1 -- start start 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f1222095740 1 Processor -- start 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f1222095740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bec8b0 0x557bb0bfb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f1222095740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0bf8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f1222095740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bfbfc0 0x557bb0bf9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f1222095740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557bb0bfeb90 con 0x557bb0bed1d0 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f1222095740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557bb0b0df20 con 0x557bb0bec8b0 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f1222095740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557bb0bfed10 con 0x557bb0bfbfc0 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f1222095740 10 client.0 _ll_register_callbacks cb 0x557bb0ad7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f121da58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0bf8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f121da58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0bf8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48232/0 (socket says 172.21.15.5:48232) 2024-09-11T22:22:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f121d257640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bec8b0 0x557bb0bfb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f121da58640 1 -- 172.21.15.5:0/1636778792 learned_addr learned my addr 172.21.15.5:0/1636778792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.433+0000 7f121d257640 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bfbfc0 0x557bb0bf9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f121da58640 1 -- 172.21.15.5:0/1636778792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bfbfc0 msgr2=0x557bb0bf9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f121da58640 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bfbfc0 0x557bb0bf9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f121da58640 1 -- 172.21.15.5:0/1636778792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bec8b0 msgr2=0x557bb0bfb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f121da58640 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bec8b0 0x557bb0bfb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f121da58640 1 -- 172.21.15.5:0/1636778792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb08c42b0 con 0x557bb0bed1d0 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f121da58640 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0bf8b10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f12140314e0 tx=0x7f12140043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfdcc851705ddde server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f12017fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f12017fa640 1 -- 172.21.15.5:0/1636778792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214035070 con 0x557bb0bed1d0 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f12017fa640 1 -- 172.21.15.5:0/1636778792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f1214003a00 con 0x557bb0bed1d0 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f1222095740 1 -- 172.21.15.5:0/1636778792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1208005500 con 0x557bb0bed1d0 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f12017fa640 1 -- 172.21.15.5:0/1636778792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f121404b070 con 0x557bb0bed1d0 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f1222095740 10 client.15117 fetch_fsmap learned FSMap version 27 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f1222095740 1 -- 172.21.15.5:0/1636778792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bb0c0a420 con 0x557bb0bed1d0 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f12017fa640 1 -- 172.21.15.5:0/1636778792 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1208005500 con 0x557bb0bed1d0 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f12017fa640 1 -- 172.21.15.5:0/1636778792 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f121403b430 con 0x557bb0bed1d0 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.434+0000 7f1222095740 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:22:03.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.435+0000 7f1222095740 10 client.15117 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.435+0000 7f1222095740 20 client.15117 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.435+0000 7f1222095740 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557bb0c0ca70 0x557bb0c0dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.435+0000 7f1222095740 4 client.15117 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.435+0000 7f1222095740 1 -- 172.21.15.5:0/1636778792 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bb0b60600 con 0x557bb0c0ca70 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.435+0000 7f121d257640 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557bb0c0ca70 0x557bb0c0dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.435+0000 7f121d257640 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557bb0c0ca70 0x557bb0c0dee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=32d0b86c7eb58c49 server_cookie=73acb1024e81311f in_seq=0 out_seq=0 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.435+0000 7f12017fa640 10 client.15117 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f12017fa640 1 -- 172.21.15.5:0/1636778792 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557bb0b60600 con 0x557bb0c0ca70 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f12017fa640 10 client.15117 handle_command_reply: tid=0 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 client.15117 shutdown 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 -- 172.21.15.5:0/1636778792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 msgr2=0x557bb0bf8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0bf8b10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f12140314e0 tx=0x7f12140043b0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 -- 172.21.15.5:0/1636778792 shutdown_connections 2024-09-11T22:22:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bfbfc0 0x557bb0bf9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0bf8b10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557bb0c0ca70 0x557bb0c0dee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 --2- 172.21.15.5:0/1636778792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bec8b0 0x557bb0bfb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f12017fa640 0 client.15117 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:22:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 -- 172.21.15.5:0/1636778792 >> 172.21.15.5:0/1636778792 conn(0x557bb0be8f60 msgr2=0x557bb0beecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 -- 172.21.15.5:0/1636778792 shutdown_connections 2024-09-11T22:22:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 1 -- 172.21.15.5:0/1636778792 wait complete. 2024-09-11T22:22:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.436+0000 7f1222095740 20 client.15117 trim_cache size 0 max 16384 2024-09-11T22:22:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.454+0000 7f1222095740 1 Processor -- start 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.454+0000 7f1222095740 1 -- start start 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.454+0000 7f1222095740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bec8b0 0x557bb0c0dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.454+0000 7f1222095740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bed1d0 0x557bb0bf3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.454+0000 7f1222095740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bfbfc0 0x557bb0bf3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.454+0000 7f1222095740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557bb0c0c220 con 0x557bb0bfbfc0 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.454+0000 7f1222095740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557bb0c0c8f0 con 0x557bb0bed1d0 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.454+0000 7f1222095740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557bb0c0cfc0 con 0x557bb0bec8b0 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121d257640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bfbfc0 0x557bb0bf3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121d257640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bfbfc0 0x557bb0bf3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48240/0 (socket says 172.21.15.5:48240) 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121d257640 1 -- 172.21.15.5:0/3582621398 learned_addr learned my addr 172.21.15.5:0/3582621398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121da58640 1 --2- 172.21.15.5:0/3582621398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bed1d0 0x557bb0bf3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121d257640 1 --2- 172.21.15.5:0/3582621398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bec8b0 0x557bb0c0dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121d257640 1 -- 172.21.15.5:0/3582621398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bec8b0 msgr2=0x557bb0c0dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121d257640 1 --2- 172.21.15.5:0/3582621398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bec8b0 0x557bb0c0dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121d257640 1 -- 172.21.15.5:0/3582621398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bed1d0 msgr2=0x557bb0bf3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121d257640 1 --2- 172.21.15.5:0/3582621398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bed1d0 0x557bb0bf3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121d257640 1 -- 172.21.15.5:0/3582621398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb0bf7060 con 0x557bb0bfbfc0 2024-09-11T22:22:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f121d257640 1 --2- 172.21.15.5:0/3582621398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bfbfc0 0x557bb0bf3f60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1208004da0 tx=0x7f120800b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67057f6d17cd4e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f12037fe640 1 -- 172.21.15.5:0/3582621398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120802b040 con 0x557bb0bfbfc0 2024-09-11T22:22:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f12037fe640 1 -- 172.21.15.5:0/3582621398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12080054e0 con 0x557bb0bfbfc0 2024-09-11T22:22:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.455+0000 7f12037fe640 1 -- 172.21.15.5:0/3582621398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208010ad0 con 0x557bb0bfbfc0 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 -- 172.21.15.5:0/3582621398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bfbfc0 msgr2=0x557bb0bf3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 --2- 172.21.15.5:0/3582621398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bfbfc0 0x557bb0bf3f60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1208004da0 tx=0x7f120800b9d0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 -- 172.21.15.5:0/3582621398 shutdown_connections 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 --2- 172.21.15.5:0/3582621398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bfbfc0 0x557bb0bf3f60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 --2- 172.21.15.5:0/3582621398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bed1d0 0x557bb0bf3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 --2- 172.21.15.5:0/3582621398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bec8b0 0x557bb0c0dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 -- 172.21.15.5:0/3582621398 >> 172.21.15.5:0/3582621398 conn(0x557bb0be8f60 msgr2=0x557bb0be3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 -- 172.21.15.5:0/3582621398 shutdown_connections 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 -- 172.21.15.5:0/3582621398 wait complete. 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 -- start start 2024-09-11T22:22:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.456+0000 7f1222095740 1 Processor -- start 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f1222095740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bec8b0 0x557bb0c165e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f1222095740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0c0f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f1222095740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bfbfc0 0x557bb0c0fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f1222095740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557bb0c0dd60 con 0x557bb0bed1d0 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f1222095740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557bb0c0d690 con 0x557bb0bfbfc0 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f1222095740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557bb0c0e430 con 0x557bb0bec8b0 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f1222095740 10 client.0 _ll_register_callbacks cb 0x7f1214009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121da58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0c0f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121da58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0c0f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48248/0 (socket says 172.21.15.5:48248) 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121da58640 1 -- 172.21.15.5:0/2967624398 learned_addr learned my addr 172.21.15.5:0/2967624398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121d257640 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bfbfc0 0x557bb0c0fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121d257640 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bec8b0 0x557bb0c165e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121da58640 1 -- 172.21.15.5:0/2967624398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bec8b0 msgr2=0x557bb0c165e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121da58640 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bec8b0 0x557bb0c165e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121da58640 1 -- 172.21.15.5:0/2967624398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bfbfc0 msgr2=0x557bb0c0fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121da58640 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bfbfc0 0x557bb0c0fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121da58640 1 -- 172.21.15.5:0/2967624398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb0c103e0 con 0x557bb0bed1d0 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.457+0000 7f121d257640 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bfbfc0 0x557bb0c0fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f121da58640 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0c0f960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1214005940 tx=0x7f121403bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10d3a85d2c8f412 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f12017fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f12017fa640 1 -- 172.21.15.5:0/2967624398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214035040 con 0x557bb0bed1d0 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f12017fa640 1 -- 172.21.15.5:0/2967624398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f1214003a00 con 0x557bb0bed1d0 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f1222095740 1 -- 172.21.15.5:0/2967624398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12080054e0 con 0x557bb0bed1d0 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f12017fa640 1 -- 172.21.15.5:0/2967624398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f121403b640 con 0x557bb0bed1d0 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f1222095740 10 client.15123 fetch_fsmap learned FSMap version 27 2024-09-11T22:22:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f1222095740 1 -- 172.21.15.5:0/2967624398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557bb0c14e70 con 0x557bb0bed1d0 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f12017fa640 1 -- 172.21.15.5:0/2967624398 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12080054e0 con 0x557bb0bed1d0 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f12017fa640 1 -- 172.21.15.5:0/2967624398 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f12140046c0 con 0x557bb0bed1d0 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f1222095740 10 client.15123 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f1222095740 10 client.15123 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f1222095740 20 client.15123 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f1222095740 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557bb0c7bb10 0x557bb0c7a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f1222095740 4 client.15123 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.458+0000 7f1222095740 1 -- 172.21.15.5:0/2967624398 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "status"}) -- 0x557bb0b5ab10 con 0x557bb0c7bb10 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.459+0000 7f121d257640 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557bb0c7bb10 0x557bb0c7a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.459+0000 7f121d257640 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557bb0c7bb10 0x557bb0c7a820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3bd464107a9a1ef0 server_cookie=964beb15cb409c12 in_seq=0 out_seq=0 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.459+0000 7f12017fa640 10 client.15123 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:22:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.459+0000 7f12017fa640 1 -- 172.21.15.5:0/2967624398 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557bb0b5ab10 con 0x557bb0c7bb10 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.459+0000 7f12017fa640 10 client.15123 handle_command_reply: tid=0 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.459+0000 7f1222095740 1 client.15123 shutdown 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.459+0000 7f1222095740 1 -- 172.21.15.5:0/2967624398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 msgr2=0x557bb0c0f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.459+0000 7f1222095740 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0c0f960 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1214005940 tx=0x7f121403bcf0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f1222095740 1 -- 172.21.15.5:0/2967624398 shutdown_connections 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f1222095740 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557bb0c7bb10 0x557bb0c7a820 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f1222095740 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557bb0bfbfc0 0x557bb0c0fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f1222095740 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557bb0bed1d0 0x557bb0c0f960 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f12017fa640 0 client.15123 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f1222095740 1 --2- 172.21.15.5:0/2967624398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557bb0bec8b0 0x557bb0c165e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f1222095740 1 -- 172.21.15.5:0/2967624398 >> 172.21.15.5:0/2967624398 conn(0x557bb0be8f60 msgr2=0x557bb0bf3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f1222095740 1 -- 172.21.15.5:0/2967624398 shutdown_connections 2024-09-11T22:22:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f1222095740 1 -- 172.21.15.5:0/2967624398 wait complete. 2024-09-11T22:22:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f1222095740 20 client.15123 trim_cache size 0 max 16384 2024-09-11T22:22:03.466 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:22:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-11T22:22:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24452, 2024-09-11T22:22:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:22:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:22:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:22:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 95.210535457000006, 2024-09-11T22:22:03.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-11T22:22:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-09-11T22:22:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T22:22:03.467 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 134.81080411600001 2024-09-11T22:22:03.467 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f121fcbc640 1 -- 172.21.15.5:0/3681007904 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12040a2b80 msgr2=0x7f12040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f121fcbc640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12040a2b80 0x7f12040a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f121009ba20 tx=0x7f121006aa10 comp rx=0 tx=0).stop 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f121fcbc640 1 -- 172.21.15.5:0/3681007904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1218102ef0 msgr2=0x7f12181a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f121fcbc640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1218102ef0 0x7f12181a4400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f121000eaa0 tx=0x7f121000ef60 comp rx=0 tx=0).stop 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f121fcbc640 1 -- 172.21.15.5:0/3681007904 shutdown_connections 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f121fcbc640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f12040a2b80 0x7f12040a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f121fcbc640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1218102ef0 0x7f12181a4400 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f121fcbc640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1218102570 0x7f12181998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f121fcbc640 1 --2- 172.21.15.5:0/3681007904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1218100e20 0x7f1218199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.460+0000 7f121fcbc640 1 -- 172.21.15.5:0/3681007904 >> 172.21.15.5:0/3681007904 conn(0x7f12180fc7d0 msgr2=0x7f12180fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.461+0000 7f121fcbc640 1 -- 172.21.15.5:0/3681007904 shutdown_connections 2024-09-11T22:22:03.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:03.461+0000 7f121fcbc640 1 -- 172.21.15.5:0/3681007904 wait complete. 2024-09-11T22:22:04.025 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:03 smithi005 ceph-mon[30943]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T22:22:04.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:03 smithi159 ceph-mon[35534]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T22:22:04.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:03 smithi172 ceph-mon[35461]: pgmap v335: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-11T22:22:04.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T22:22:04.669 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.184+0000 7ff179009640 1 -- 172.21.15.5:0/3380090561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff174101200 msgr2=0x7ff174101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.184+0000 7ff179009640 1 --2- 172.21.15.5:0/3380090561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff174101200 0x7ff174101660 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff168009900 tx=0x7ff16802a410 comp rx=0 tx=0).stop 2024-09-11T22:22:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.184+0000 7ff179009640 1 -- 172.21.15.5:0/3380090561 shutdown_connections 2024-09-11T22:22:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.184+0000 7ff179009640 1 --2- 172.21.15.5:0/3380090561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff174101ba0 0x7ff17410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.184+0000 7ff179009640 1 --2- 172.21.15.5:0/3380090561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff174101200 0x7ff174101660 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.184+0000 7ff179009640 1 --2- 172.21.15.5:0/3380090561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff174105ad0 0x7ff174105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.184+0000 7ff179009640 1 -- 172.21.15.5:0/3380090561 >> 172.21.15.5:0/3380090561 conn(0x7ff174077210 msgr2=0x7ff1740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.184+0000 7ff179009640 1 -- 172.21.15.5:0/3380090561 shutdown_connections 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.184+0000 7ff179009640 1 -- 172.21.15.5:0/3380090561 wait complete. 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff179009640 1 Processor -- start 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff179009640 1 -- start start 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff179009640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff174101200 0x7ff1740fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff179009640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff174101ba0 0x7ff1740fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff179009640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff174105ad0 0x7ff1740fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff179009640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff174110eb0 con 0x7ff174105ad0 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff179009640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff174110d30 con 0x7ff174101200 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff179009640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff174111030 con 0x7ff174101ba0 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff17359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff174105ad0 0x7ff1740fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff17359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff174105ad0 0x7ff1740fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48272/0 (socket says 172.21.15.5:48272) 2024-09-11T22:22:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.185+0000 7ff17359e640 1 -- 172.21.15.5:0/3992523606 learned_addr learned my addr 172.21.15.5:0/3992523606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff172d9d640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff174101200 0x7ff1740fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff17259c640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff174101ba0 0x7ff1740fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff17359e640 1 -- 172.21.15.5:0/3992523606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff174101ba0 msgr2=0x7ff1740fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff17359e640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff174101ba0 0x7ff1740fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff17359e640 1 -- 172.21.15.5:0/3992523606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff174101200 msgr2=0x7ff1740fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff17359e640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff174101200 0x7ff1740fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff17359e640 1 -- 172.21.15.5:0/3992523606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1741a6880 con 0x7ff174105ad0 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff17359e640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff174105ad0 0x7ff1740fd110 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00d720 tx=0x7ff16c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff15bfff640 1 -- 172.21.15.5:0/3992523606 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c017070 con 0x7ff174105ad0 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff15bfff640 1 -- 172.21.15.5:0/3992523606 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff16c00de30 con 0x7ff174105ad0 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff179009640 1 -- 172.21.15.5:0/3992523606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1741a6b70 con 0x7ff174105ad0 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff15bfff640 1 -- 172.21.15.5:0/3992523606 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c004d90 con 0x7ff174105ad0 2024-09-11T22:22:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.186+0000 7ff179009640 1 -- 172.21.15.5:0/3992523606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1741a7060 con 0x7ff174105ad0 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.188+0000 7ff15bfff640 1 -- 172.21.15.5:0/3992523606 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ff16c004f30 con 0x7ff174105ad0 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17b3e2740 1 Processor -- start 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17b3e2740 1 -- start start 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c49d0 0x5619f28c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5619f282b090 con 0x5619f28c52f0 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5619f27d2a70 con 0x5619f28c49d0 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619f2827050 con 0x5619f28cfbb0 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff172d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff172d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28cf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48278/0 (socket says 172.21.15.5:48278) 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff172d9d640 1 -- 172.21.15.5:0/497600308 learned_addr learned my addr 172.21.15.5:0/497600308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff15bfff640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff15c0a2b80 0x7ff15c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17259c640 1 --2- 172.21.15.5:0/497600308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff172d9d640 1 -- 172.21.15.5:0/497600308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 msgr2=0x5619f28d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff172d9d640 1 --2- 172.21.15.5:0/497600308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28d1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff172d9d640 1 -- 172.21.15.5:0/497600308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c49d0 msgr2=0x5619f28c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff172d9d640 1 --2- 172.21.15.5:0/497600308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c49d0 0x5619f28c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff172d9d640 1 -- 172.21.15.5:0/497600308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619f27df130 con 0x5619f28c52f0 2024-09-11T22:22:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17259c640 1 --2- 172.21.15.5:0/497600308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.189+0000 7ff17359e640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff15c0a2b80 0x7ff15c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff15bfff640 1 -- 172.21.15.5:0/3992523606 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff16c0ce9b0 con 0x7ff174105ad0 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff172d9d640 1 --2- 172.21.15.5:0/497600308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28cf670 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff164010370 tx=0x7ff164010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f64cb680765b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff1597fa640 1 -- 172.21.15.5:0/497600308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164016070 con 0x5619f28c52f0 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff1597fa640 1 -- 172.21.15.5:0/497600308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff164004d20 con 0x5619f28c52f0 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff1597fa640 1 -- 172.21.15.5:0/497600308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164005480 con 0x5619f28c52f0 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff17b3e2740 1 -- 172.21.15.5:0/497600308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 msgr2=0x5619f28cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff17359e640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff15c0a2b80 0x7ff15c0a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff16c06e280 tx=0x7ff16c0c5d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/497600308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28cf670 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff164010370 tx=0x7ff164010830 comp rx=0 tx=0).stop 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff17b3e2740 1 -- 172.21.15.5:0/497600308 shutdown_connections 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/497600308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28d1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/497600308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28cf670 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/497600308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c49d0 0x5619f28c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.190+0000 7ff17b3e2740 1 -- 172.21.15.5:0/497600308 >> 172.21.15.5:0/497600308 conn(0x5619f28c0f60 msgr2=0x5619f28c1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 1 -- 172.21.15.5:0/497600308 shutdown_connections 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 1 -- 172.21.15.5:0/497600308 wait complete. 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 1 -- start start 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 1 Processor -- start 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 0x5619f28cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5619f28d2780 con 0x5619f28c49d0 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5619f27e5f20 con 0x5619f28c52f0 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619f28d2900 con 0x5619f28cfbb0 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17b3e2740 10 client.0 _ll_register_callbacks cb 0x5619f27af030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48282/0 (socket says 172.21.15.5:48282) 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.191+0000 7ff17259c640 1 -- 172.21.15.5:0/489236679 learned_addr learned my addr 172.21.15.5:0/489236679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff172d9d640 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 0x5619f28cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff17259c640 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff17259c640 1 -- 172.21.15.5:0/489236679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 msgr2=0x5619f28ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff17259c640 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff17259c640 1 -- 172.21.15.5:0/489236679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 msgr2=0x5619f28cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff17259c640 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 0x5619f28cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff17259c640 1 -- 172.21.15.5:0/489236679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619f259c2b0 con 0x5619f28c49d0 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff172d9d640 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 0x5619f28cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff17259c640 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28cf470 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff168005c40 tx=0x7ff1680056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba22572560ac55f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff156ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff156ff5640 1 -- 172.21.15.5:0/489236679 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168043030 con 0x5619f28c49d0 2024-09-11T22:22:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff156ff5640 1 -- 172.21.15.5:0/489236679 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff168003c70 con 0x5619f28c49d0 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff17b3e2740 1 -- 172.21.15.5:0/489236679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff164004d20 con 0x5619f28c49d0 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff156ff5640 1 -- 172.21.15.5:0/489236679 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff168035040 con 0x5619f28c49d0 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff17b3e2740 10 client.15135 fetch_fsmap learned FSMap version 27 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff17b3e2740 1 -- 172.21.15.5:0/489236679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5619f28de040 con 0x5619f28c49d0 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.192+0000 7ff156ff5640 1 -- 172.21.15.5:0/489236679 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff164004d20 con 0x5619f28c49d0 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.193+0000 7ff156ff5640 1 -- 172.21.15.5:0/489236679 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff16803a510 con 0x5619f28c49d0 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.193+0000 7ff17b3e2740 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.193+0000 7ff17b3e2740 10 client.15135 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.193+0000 7ff17b3e2740 20 client.15135 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.193+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5619f28e0670 0x5619f28e1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.193+0000 7ff17b3e2740 4 client.15135 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.193+0000 7ff17b3e2740 1 -- 172.21.15.5:0/489236679 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619f2838600 con 0x5619f28e0670 2024-09-11T22:22:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.193+0000 7ff172d9d640 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5619f28e0670 0x5619f28e1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff156ff5640 10 client.15135 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.193+0000 7ff172d9d640 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5619f28e0670 0x5619f28e1ae0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5967fb870dc62c54 server_cookie=e79026833363ea87 in_seq=0 out_seq=0 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff156ff5640 1 -- 172.21.15.5:0/489236679 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619f2838600 con 0x5619f28e0670 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff156ff5640 10 client.15135 handle_command_reply: tid=0 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff17b3e2740 1 client.15135 shutdown 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff17b3e2740 1 -- 172.21.15.5:0/489236679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 msgr2=0x5619f28cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28cf470 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff168005c40 tx=0x7ff1680056d0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff17b3e2740 1 -- 172.21.15.5:0/489236679 shutdown_connections 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5619f28e0670 0x5619f28e1ae0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 0x5619f28cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/489236679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28cf470 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff17b3e2740 1 -- 172.21.15.5:0/489236679 >> 172.21.15.5:0/489236679 conn(0x5619f28c0f60 msgr2=0x5619f28c6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.194+0000 7ff156ff5640 0 client.15135 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:22:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.195+0000 7ff17b3e2740 1 -- 172.21.15.5:0/489236679 shutdown_connections 2024-09-11T22:22:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.195+0000 7ff17b3e2740 1 -- 172.21.15.5:0/489236679 wait complete. 2024-09-11T22:22:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.195+0000 7ff17b3e2740 20 client.15135 trim_cache size 0 max 16384 2024-09-11T22:22:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff17b3e2740 1 Processor -- start 2024-09-11T22:22:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff17b3e2740 1 -- start start 2024-09-11T22:22:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28c49d0 0x5619f28dffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28e1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28cfbb0 0x5619f28e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5619f28dfe20 con 0x5619f28c52f0 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5619f28e04f0 con 0x5619f28cfbb0 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619f28e0bc0 con 0x5619f28c49d0 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff172d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28e1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff172d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28e1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48286/0 (socket says 172.21.15.5:48286) 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff172d9d640 1 -- 172.21.15.5:0/3671805613 learned_addr learned my addr 172.21.15.5:0/3671805613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff17259c640 1 --2- 172.21.15.5:0/3671805613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28cfbb0 0x5619f28e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff172d9d640 1 -- 172.21.15.5:0/3671805613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28c49d0 msgr2=0x5619f28dffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff172d9d640 1 --2- 172.21.15.5:0/3671805613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28c49d0 0x5619f28dffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff172d9d640 1 -- 172.21.15.5:0/3671805613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28cfbb0 msgr2=0x5619f28e9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff172d9d640 1 --2- 172.21.15.5:0/3671805613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28cfbb0 0x5619f28e9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff172d9d640 1 -- 172.21.15.5:0/3671805613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619f28c5120 con 0x5619f28c52f0 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff17259c640 1 --2- 172.21.15.5:0/3671805613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28cfbb0 0x5619f28e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.213+0000 7ff172d9d640 1 --2- 172.21.15.5:0/3671805613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28e1410 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff164010370 tx=0x7ff164012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21423af0014e7eca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff158ff9640 1 -- 172.21.15.5:0/3671805613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164016040 con 0x5619f28c52f0 2024-09-11T22:22:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff158ff9640 1 -- 172.21.15.5:0/3671805613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff164003b70 con 0x5619f28c52f0 2024-09-11T22:22:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff158ff9640 1 -- 172.21.15.5:0/3671805613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff164012770 con 0x5619f28c52f0 2024-09-11T22:22:05.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff17b3e2740 1 -- 172.21.15.5:0/3671805613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 msgr2=0x5619f28e1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:05.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/3671805613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28e1410 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff164010370 tx=0x7ff164012f60 comp rx=0 tx=0).stop 2024-09-11T22:22:05.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff17b3e2740 1 -- 172.21.15.5:0/3671805613 shutdown_connections 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/3671805613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28cfbb0 0x5619f28e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/3671805613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c52f0 0x5619f28e1410 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/3671805613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28c49d0 0x5619f28dffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff17b3e2740 1 -- 172.21.15.5:0/3671805613 >> 172.21.15.5:0/3671805613 conn(0x5619f28c0f60 msgr2=0x5619f28cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff17b3e2740 1 -- 172.21.15.5:0/3671805613 shutdown_connections 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.214+0000 7ff17b3e2740 1 -- 172.21.15.5:0/3671805613 wait complete. 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17b3e2740 1 -- start start 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17b3e2740 1 Processor -- start 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 0x5619f28e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17b3e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5619f28e1960 con 0x5619f28c49d0 2024-09-11T22:22:05.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5619f28e1290 con 0x5619f28c52f0 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17b3e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5619f28e2030 con 0x5619f28cfbb0 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17b3e2740 10 client.0 _ll_register_callbacks cb 0x7ff168009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28e4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48290/0 (socket says 172.21.15.5:48290) 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17259c640 1 -- 172.21.15.5:0/4159113071 learned_addr learned my addr 172.21.15.5:0/4159113071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff172d9d640 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 0x5619f28e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17259c640 1 -- 172.21.15.5:0/4159113071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 msgr2=0x5619f28e57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17259c640 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17259c640 1 -- 172.21.15.5:0/4159113071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 msgr2=0x5619f28e52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17259c640 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 0x5619f28e52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.215+0000 7ff17259c640 1 -- 172.21.15.5:0/4159113071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619f2948d70 con 0x5619f28c49d0 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff17259c640 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28e4d60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff168006010 tx=0x7ff168031470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91eab2a14742a867 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:05.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff156ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff156ff5640 1 -- 172.21.15.5:0/4159113071 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168043040 con 0x5619f28c49d0 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff156ff5640 1 -- 172.21.15.5:0/4159113071 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff16803a680 con 0x5619f28c49d0 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff17b3e2740 1 -- 172.21.15.5:0/4159113071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff164003b70 con 0x5619f28c49d0 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff156ff5640 1 -- 172.21.15.5:0/4159113071 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff16803ab50 con 0x5619f28c49d0 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff17b3e2740 10 client.15141 fetch_fsmap learned FSMap version 27 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff17b3e2740 1 -- 172.21.15.5:0/4159113071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5619f2949fc0 con 0x5619f28c49d0 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff156ff5640 1 -- 172.21.15.5:0/4159113071 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff164003b70 con 0x5619f28c49d0 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff156ff5640 1 -- 172.21.15.5:0/4159113071 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff1680049c0 con 0x5619f28c49d0 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff17b3e2740 10 client.15141 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff17b3e2740 10 client.15141 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff17b3e2740 20 client.15141 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5619f294eab0 0x5619f294d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:05.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff17b3e2740 4 client.15141 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.216+0000 7ff17b3e2740 1 -- 172.21.15.5:0/4159113071 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "status"}) -- 0x5619f2832b10 con 0x5619f294eab0 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff172d9d640 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5619f294eab0 0x5619f294d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff156ff5640 10 client.15141 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff172d9d640 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5619f294eab0 0x5619f294d7c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc26481119f4e9cf server_cookie=da04564f3d70dd3 in_seq=0 out_seq=0 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff156ff5640 1 -- 172.21.15.5:0/4159113071 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x5619f2832b10 con 0x5619f294eab0 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff156ff5640 10 client.15141 handle_command_reply: tid=0 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff17b3e2740 1 client.15141 shutdown 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff17b3e2740 1 -- 172.21.15.5:0/4159113071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 msgr2=0x5619f28e4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28e4d60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff168006010 tx=0x7ff168031470 comp rx=0 tx=0).stop 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff17b3e2740 1 -- 172.21.15.5:0/4159113071 shutdown_connections 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5619f294eab0 0x5619f294d7c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5619f28cfbb0 0x5619f28e57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5619f28c52f0 0x5619f28e52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff156ff5640 0 client.15141 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:22:05.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff17b3e2740 1 --2- 172.21.15.5:0/4159113071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5619f28c49d0 0x5619f28e4d60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.217+0000 7ff17b3e2740 1 -- 172.21.15.5:0/4159113071 >> 172.21.15.5:0/4159113071 conn(0x5619f28c0f60 msgr2=0x5619f28cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:05.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff17b3e2740 1 -- 172.21.15.5:0/4159113071 shutdown_connections 2024-09-11T22:22:05.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff17b3e2740 1 -- 172.21.15.5:0/4159113071 wait complete. 2024-09-11T22:22:05.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff17b3e2740 20 client.15141 trim_cache size 0 max 16384 2024-09-11T22:22:05.224 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:05.224 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:22:05.224 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-11T22:22:05.224 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24454, 2024-09-11T22:22:05.225 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:22:05.225 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:22:05.225 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:22:05.225 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 97.97351784, 2024-09-11T22:22:05.225 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-11T22:22:05.225 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-09-11T22:22:05.225 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T22:22:05.225 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 125.865994366 2024-09-11T22:22:05.225 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:05.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 -- 172.21.15.5:0/3992523606 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff15c0a2b80 msgr2=0x7ff15c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:05.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff15c0a2b80 0x7ff15c0a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff16c06e280 tx=0x7ff16c0c5d80 comp rx=0 tx=0).stop 2024-09-11T22:22:05.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 -- 172.21.15.5:0/3992523606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff174105ad0 msgr2=0x7ff1740fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:05.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff174105ad0 0x7ff1740fd110 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00d720 tx=0x7ff16c00dbe0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 -- 172.21.15.5:0/3992523606 shutdown_connections 2024-09-11T22:22:05.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7ff15c0a2b80 0x7ff15c0a5040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff174105ad0 0x7ff1740fd110 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff174101ba0 0x7ff1740fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 --2- 172.21.15.5:0/3992523606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff174101200 0x7ff1740fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:05.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 -- 172.21.15.5:0/3992523606 >> 172.21.15.5:0/3992523606 conn(0x7ff174077210 msgr2=0x7ff1741005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:05.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.218+0000 7ff179009640 1 -- 172.21.15.5:0/3992523606 shutdown_connections 2024-09-11T22:22:05.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:05.219+0000 7ff179009640 1 -- 172.21.15.5:0/3992523606 wait complete. 2024-09-11T22:22:06.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:05 smithi005 ceph-mon[30943]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:22:06.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T22:22:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:05 smithi159 ceph-mon[35534]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:22:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:05 smithi172 ceph-mon[35461]: pgmap v336: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-11T22:22:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 -- 172.21.15.5:0/742588842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3010c2f0 msgr2=0x7fae3010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 --2- 172.21.15.5:0/742588842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3010c2f0 0x7fae3010e790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fae28009ca0 tx=0x7fae2802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 -- 172.21.15.5:0/742588842 shutdown_connections 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 --2- 172.21.15.5:0/742588842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3010c2f0 0x7fae3010e790 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 --2- 172.21.15.5:0/742588842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae301037a0 0x7fae3010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 --2- 172.21.15.5:0/742588842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae30102df0 0x7fae301031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 -- 172.21.15.5:0/742588842 >> 172.21.15.5:0/742588842 conn(0x7fae300fc7d0 msgr2=0x7fae300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 -- 172.21.15.5:0/742588842 shutdown_connections 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 -- 172.21.15.5:0/742588842 wait complete. 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 Processor -- start 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.788+0000 7fae355fd640 1 -- start start 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae355fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae30102df0 0x7fae30198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae355fd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae301037a0 0x7fae30198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae355fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae3010c2f0 0x7fae3019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae355fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae30112110 con 0x7fae30102df0 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae355fd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae30111f90 con 0x7fae301037a0 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae355fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae30112290 con 0x7fae3010c2f0 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae2effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae30102df0 0x7fae30198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae2effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae30102df0 0x7fae30198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48304/0 (socket says 172.21.15.5:48304) 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae2effd640 1 -- 172.21.15.5:0/1373041156 learned_addr learned my addr 172.21.15.5:0/1373041156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae3010c2f0 0x7fae3019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae267fc640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae301037a0 0x7fae30198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae2effd640 1 -- 172.21.15.5:0/1373041156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae3010c2f0 msgr2=0x7fae3019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae2effd640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae3010c2f0 0x7fae3019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae2effd640 1 -- 172.21.15.5:0/1373041156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae301037a0 msgr2=0x7fae30198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae2effd640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae301037a0 0x7fae30198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae2effd640 1 -- 172.21.15.5:0/1373041156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3019d5f0 con 0x7fae30102df0 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.789+0000 7fae2effd640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae30102df0 0x7fae30198640 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fae2000d770 tx=0x7fae2000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.790+0000 7fae2cff9640 1 -- 172.21.15.5:0/1373041156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20017070 con 0x7fae30102df0 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.790+0000 7fae355fd640 1 -- 172.21.15.5:0/1373041156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3019d8e0 con 0x7fae30102df0 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.790+0000 7fae2cff9640 1 -- 172.21.15.5:0/1373041156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae20005500 con 0x7fae30102df0 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.790+0000 7fae2cff9640 1 -- 172.21.15.5:0/1373041156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20004e00 con 0x7fae30102df0 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.790+0000 7fae355fd640 1 -- 172.21.15.5:0/1373041156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae301a51c0 con 0x7fae30102df0 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.792+0000 7fae2cff9640 1 -- 172.21.15.5:0/1373041156 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fae20004fa0 con 0x7fae30102df0 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.792+0000 7fae379d6740 1 Processor -- start 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.792+0000 7fae379d6740 1 -- start start 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.792+0000 7fae379d6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff1ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.792+0000 7fae379d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff28670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae379d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff2afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae379d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cdfe84090 con 0x562cdff28bb0 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae379d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562cdfe2ba70 con 0x562cdff1e2f0 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae379d6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562cdfe80050 con 0x562cdff1d9d0 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae267fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff2afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae267fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff2afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48316/0 (socket says 172.21.15.5:48316) 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae267fc640 1 -- 172.21.15.5:0/2406883452 learned_addr learned my addr 172.21.15.5:0/2406883452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/2406883452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff1ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae2effd640 1 --2- 172.21.15.5:0/2406883452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff28670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae267fc640 1 -- 172.21.15.5:0/2406883452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 msgr2=0x562cdff1ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae267fc640 1 --2- 172.21.15.5:0/2406883452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff1ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae267fc640 1 -- 172.21.15.5:0/2406883452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 msgr2=0x562cdff28670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae267fc640 1 --2- 172.21.15.5:0/2406883452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff28670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae267fc640 1 -- 172.21.15.5:0/2406883452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cdfe38130 con 0x562cdff28bb0 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae2effd640 1 --2- 172.21.15.5:0/2406883452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff28670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae2cff9640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fae1c0a2b80 0x7fae1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae267fc640 1 --2- 172.21.15.5:0/2406883452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff2afa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fae1800b990 tx=0x7fae1800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2f611a2e6300c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fae1c0a2b80 0x7fae1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae257fa640 1 -- 172.21.15.5:0/2406883452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1800ddd0 con 0x562cdff28bb0 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae257fa640 1 -- 172.21.15.5:0/2406883452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae18017900 con 0x562cdff28bb0 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae2cff9640 1 -- 172.21.15.5:0/1373041156 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fae200ceae0 con 0x7fae30102df0 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.793+0000 7fae257fa640 1 -- 172.21.15.5:0/2406883452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1800fa50 con 0x562cdff28bb0 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.794+0000 7fae379d6740 1 -- 172.21.15.5:0/2406883452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 msgr2=0x562cdff2afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.794+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fae1c0a2b80 0x7fae1c0a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae28005e30 tx=0x7fae28005fa0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.794+0000 7fae379d6740 1 --2- 172.21.15.5:0/2406883452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff2afa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fae1800b990 tx=0x7fae1800be50 comp rx=0 tx=0).stop 2024-09-11T22:22:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.794+0000 7fae379d6740 1 -- 172.21.15.5:0/2406883452 shutdown_connections 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.794+0000 7fae379d6740 1 --2- 172.21.15.5:0/2406883452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff2afa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.794+0000 7fae379d6740 1 --2- 172.21.15.5:0/2406883452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff28670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.794+0000 7fae379d6740 1 --2- 172.21.15.5:0/2406883452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff1ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.794+0000 7fae379d6740 1 -- 172.21.15.5:0/2406883452 >> 172.21.15.5:0/2406883452 conn(0x562cdff19f60 msgr2=0x562cdff1a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.794+0000 7fae379d6740 1 -- 172.21.15.5:0/2406883452 shutdown_connections 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.794+0000 7fae379d6740 1 -- 172.21.15.5:0/2406883452 wait complete. 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae379d6740 1 -- start start 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae379d6740 1 Processor -- start 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae379d6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff28380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae379d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff256d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae379d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff25c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae379d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cdff2fb90 con 0x562cdff28bb0 2024-09-11T22:22:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae379d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562cdfe3ef20 con 0x562cdff1e2f0 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae379d6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562cdff2fd10 con 0x562cdff1d9d0 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae379d6740 10 client.0 _ll_register_callbacks cb 0x562cdfe08030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae2f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff25c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae2f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff25c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48326/0 (socket says 172.21.15.5:48326) 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae2f7fe640 1 -- 172.21.15.5:0/3604123038 learned_addr learned my addr 172.21.15.5:0/3604123038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae2effd640 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff28380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae267fc640 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff256d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae2f7fe640 1 -- 172.21.15.5:0/3604123038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 msgr2=0x562cdff28380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff28380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae2f7fe640 1 -- 172.21.15.5:0/3604123038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 msgr2=0x562cdff256d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff256d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae2f7fe640 1 -- 172.21.15.5:0/3604123038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cdfbf52b0 con 0x562cdff28bb0 2024-09-11T22:22:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.795+0000 7fae267fc640 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff256d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff25c10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fae28035000 tx=0x7fae2802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f8c067385df541 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae167fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae167fc640 1 -- 172.21.15.5:0/3604123038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28031cd0 con 0x562cdff28bb0 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae167fc640 1 -- 172.21.15.5:0/3604123038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fae280045c0 con 0x562cdff28bb0 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae379d6740 1 -- 172.21.15.5:0/3604123038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae18017900 con 0x562cdff28bb0 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae167fc640 1 -- 172.21.15.5:0/3604123038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae28058070 con 0x562cdff28bb0 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae379d6740 10 client.15153 fetch_fsmap learned FSMap version 27 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae379d6740 1 -- 172.21.15.5:0/3604123038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562cdff3b380 con 0x562cdff28bb0 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae167fc640 1 -- 172.21.15.5:0/3604123038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae18017900 con 0x562cdff28bb0 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae167fc640 1 -- 172.21.15.5:0/3604123038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fae28030650 con 0x562cdff28bb0 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae379d6740 10 client.15153 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae379d6740 10 client.15153 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:22:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae379d6740 20 client.15153 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae379d6740 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562cdff3dab0 0x562cdff3ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae379d6740 4 client.15153 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.796+0000 7fae379d6740 1 -- 172.21.15.5:0/3604123038 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cdfe91600 con 0x562cdff3dab0 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.797+0000 7fae2effd640 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562cdff3dab0 0x562cdff3ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.797+0000 7fae167fc640 10 client.15153 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.797+0000 7fae2effd640 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562cdff3dab0 0x562cdff3ef20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac7949886bb2d910 server_cookie=1530be261164c05f in_seq=0 out_seq=0 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae167fc640 1 -- 172.21.15.5:0/3604123038 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cdfe91600 con 0x562cdff3dab0 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae167fc640 10 client.15153 handle_command_reply: tid=0 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 client.15153 shutdown 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 -- 172.21.15.5:0/3604123038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 msgr2=0x562cdff25c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff25c10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fae28035000 tx=0x7fae2802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 -- 172.21.15.5:0/3604123038 shutdown_connections 2024-09-11T22:22:06.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562cdff3dab0 0x562cdff3ef20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff25c10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff256d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 --2- 172.21.15.5:0/3604123038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff28380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae167fc640 0 client.15153 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:22:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 -- 172.21.15.5:0/3604123038 >> 172.21.15.5:0/3604123038 conn(0x562cdff19f60 msgr2=0x562cdff1fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 -- 172.21.15.5:0/3604123038 shutdown_connections 2024-09-11T22:22:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 1 -- 172.21.15.5:0/3604123038 wait complete. 2024-09-11T22:22:06.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.798+0000 7fae379d6740 20 client.15153 trim_cache size 0 max 16384 2024-09-11T22:22:06.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.816+0000 7fae379d6740 1 Processor -- start 2024-09-11T22:22:06.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.816+0000 7fae379d6740 1 -- start start 2024-09-11T22:22:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae379d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1d9d0 0x562cdff3ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae379d6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1e2f0 0x562cdff14ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae379d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff23fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae379d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cdff3d260 con 0x562cdff28bb0 2024-09-11T22:22:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae379d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562cdff3d930 con 0x562cdff1d9d0 2024-09-11T22:22:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae379d6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562cdff3e000 con 0x562cdff1e2f0 2024-09-11T22:22:06.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff23fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff23fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48338/0 (socket says 172.21.15.5:48338) 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2f7fe640 1 -- 172.21.15.5:0/100659950 learned_addr learned my addr 172.21.15.5:0/100659950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae267fc640 1 --2- 172.21.15.5:0/100659950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1e2f0 0x562cdff14ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2effd640 1 --2- 172.21.15.5:0/100659950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1d9d0 0x562cdff3ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2f7fe640 1 -- 172.21.15.5:0/100659950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1e2f0 msgr2=0x562cdff14ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/100659950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1e2f0 0x562cdff14ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2f7fe640 1 -- 172.21.15.5:0/100659950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1d9d0 msgr2=0x562cdff3ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/100659950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1d9d0 0x562cdff3ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2f7fe640 1 -- 172.21.15.5:0/100659950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cdff19d90 con 0x562cdff28bb0 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2effd640 1 --2- 172.21.15.5:0/100659950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1d9d0 0x562cdff3ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/100659950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff23fa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fae28009bf0 tx=0x7fae28031ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dec49882fee8bb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.817+0000 7fae24ff9640 1 -- 172.21.15.5:0/100659950 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28061030 con 0x562cdff28bb0 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae24ff9640 1 -- 172.21.15.5:0/100659950 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2804e070 con 0x562cdff28bb0 2024-09-11T22:22:06.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae24ff9640 1 -- 172.21.15.5:0/100659950 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2802f940 con 0x562cdff28bb0 2024-09-11T22:22:06.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae379d6740 1 -- 172.21.15.5:0/100659950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 msgr2=0x562cdff23fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae379d6740 1 --2- 172.21.15.5:0/100659950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff23fa0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fae28009bf0 tx=0x7fae28031ec0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae379d6740 1 -- 172.21.15.5:0/100659950 shutdown_connections 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae379d6740 1 --2- 172.21.15.5:0/100659950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff23fa0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae379d6740 1 --2- 172.21.15.5:0/100659950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1e2f0 0x562cdff14ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae379d6740 1 --2- 172.21.15.5:0/100659950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1d9d0 0x562cdff3ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae379d6740 1 -- 172.21.15.5:0/100659950 >> 172.21.15.5:0/100659950 conn(0x562cdff19f60 msgr2=0x562cdff25690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae379d6740 1 -- 172.21.15.5:0/100659950 shutdown_connections 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.818+0000 7fae379d6740 1 -- 172.21.15.5:0/100659950 wait complete. 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae379d6740 1 -- start start 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae379d6740 1 Processor -- start 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae379d6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff29350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae379d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff29890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae379d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff46170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae379d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cdff3eda0 con 0x562cdff28bb0 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae379d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562cdff3e6d0 con 0x562cdff1e2f0 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae379d6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562cdff3f470 con 0x562cdff1d9d0 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae379d6740 10 client.0 _ll_register_callbacks cb 0x7fae28003dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae2f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff46170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae2f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff46170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48348/0 (socket says 172.21.15.5:48348) 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae2f7fe640 1 -- 172.21.15.5:0/898193644 learned_addr learned my addr 172.21.15.5:0/898193644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae267fc640 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff29890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.819+0000 7fae2effd640 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff29350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae2f7fe640 1 -- 172.21.15.5:0/898193644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 msgr2=0x562cdff29350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff29350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae2f7fe640 1 -- 172.21.15.5:0/898193644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 msgr2=0x562cdff29890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff29890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae2f7fe640 1 -- 172.21.15.5:0/898193644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cdff466b0 con 0x562cdff28bb0 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae2f7fe640 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff46170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fae28043040 tx=0x7fae28003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167c42af160b2c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae167fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae167fc640 1 -- 172.21.15.5:0/898193644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28061070 con 0x562cdff28bb0 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae167fc640 1 -- 172.21.15.5:0/898193644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fae28004410 con 0x562cdff28bb0 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae379d6740 1 -- 172.21.15.5:0/898193644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae2804e070 con 0x562cdff28bb0 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae167fc640 1 -- 172.21.15.5:0/898193644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae28004610 con 0x562cdff28bb0 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae379d6740 10 client.15159 fetch_fsmap learned FSMap version 27 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.820+0000 7fae379d6740 1 -- 172.21.15.5:0/898193644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562cdff479b0 con 0x562cdff28bb0 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.821+0000 7fae167fc640 1 -- 172.21.15.5:0/898193644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2804e070 con 0x562cdff28bb0 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.821+0000 7fae167fc640 1 -- 172.21.15.5:0/898193644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fae2805da10 con 0x562cdff28bb0 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.821+0000 7fae379d6740 10 client.15159 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:22:06.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.821+0000 7fae379d6740 10 client.15159 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:22:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.821+0000 7fae379d6740 20 client.15159 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.821+0000 7fae379d6740 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562cdffac140 0x562cdffaae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.821+0000 7fae379d6740 4 client.15159 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:22:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.821+0000 7fae379d6740 1 -- 172.21.15.5:0/898193644 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "status"}) -- 0x562cdfe8bb10 con 0x562cdffac140 2024-09-11T22:22:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.822+0000 7fae2effd640 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562cdffac140 0x562cdffaae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.825+0000 7fae167fc640 10 client.15159 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:22:06.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.825+0000 7fae2effd640 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562cdffac140 0x562cdffaae50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5fbb389e7a55258 server_cookie=a39b8387df4510ad in_seq=0 out_seq=0 2024-09-11T22:22:06.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.826+0000 7fae167fc640 1 -- 172.21.15.5:0/898193644 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562cdfe8bb10 con 0x562cdffac140 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.826+0000 7fae167fc640 10 client.15159 handle_command_reply: tid=0 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.826+0000 7fae379d6740 1 client.15159 shutdown 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 1 -- 172.21.15.5:0/898193644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 msgr2=0x562cdff46170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff46170 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fae28043040 tx=0x7fae28003e70 comp rx=0 tx=0).stop 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 1 -- 172.21.15.5:0/898193644 shutdown_connections 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562cdffac140 0x562cdffaae50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cdff28bb0 0x562cdff46170 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cdff1e2f0 0x562cdff29890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 1 --2- 172.21.15.5:0/898193644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562cdff1d9d0 0x562cdff29350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 1 -- 172.21.15.5:0/898193644 >> 172.21.15.5:0/898193644 conn(0x562cdff19f60 msgr2=0x562cdff20530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae167fc640 0 client.15159 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 1 -- 172.21.15.5:0/898193644 shutdown_connections 2024-09-11T22:22:06.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 1 -- 172.21.15.5:0/898193644 wait complete. 2024-09-11T22:22:06.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.827+0000 7fae379d6740 20 client.15159 trim_cache size 0 max 16384 2024-09-11T22:22:06.829 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:22:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-11T22:22:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24473, 2024-09-11T22:22:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:22:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:22:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:22:06.829 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 100.596014586, 2024-09-11T22:22:06.830 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-11T22:22:06.830 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-09-11T22:22:06.830 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T22:22:06.830 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 130.07054286499999 2024-09-11T22:22:06.830 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 -- 172.21.15.5:0/1373041156 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fae1c0a2b80 msgr2=0x7fae1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fae1c0a2b80 0x7fae1c0a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fae28005e30 tx=0x7fae28005fa0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 -- 172.21.15.5:0/1373041156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae30102df0 msgr2=0x7fae30198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae30102df0 0x7fae30198640 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fae2000d770 tx=0x7fae2000dc30 comp rx=0 tx=0).stop 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 -- 172.21.15.5:0/1373041156 shutdown_connections 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fae1c0a2b80 0x7fae1c0a5040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae3010c2f0 0x7fae3019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae301037a0 0x7fae30198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 --2- 172.21.15.5:0/1373041156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae30102df0 0x7fae30198640 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 -- 172.21.15.5:0/1373041156 >> 172.21.15.5:0/1373041156 conn(0x7fae300fc7d0 msgr2=0x7fae300fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 -- 172.21.15.5:0/1373041156 shutdown_connections 2024-09-11T22:22:06.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:06.828+0000 7fae355fd640 1 -- 172.21.15.5:0/1373041156 wait complete. 2024-09-11T22:22:07.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T22:22:07.955 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:07 smithi005 ceph-mon[30943]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:22:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:07 smithi159 ceph-mon[35534]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:22:08.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:07 smithi172 ceph-mon[35461]: pgmap v337: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-11T22:22:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.422+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/3621962318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c102790 msgr2=0x7f7e9c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.422+0000 7f7ea1a6d640 1 --2- 172.21.15.5:0/3621962318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c102790 0x7f7e9c1067e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009ca0 tx=0x7f7e9002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.422+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/3621962318 shutdown_connections 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.422+0000 7f7ea1a6d640 1 --2- 172.21.15.5:0/3621962318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c106f10 0x7f7e9c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.422+0000 7f7ea1a6d640 1 --2- 172.21.15.5:0/3621962318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c102790 0x7f7e9c1067e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.422+0000 7f7ea1a6d640 1 --2- 172.21.15.5:0/3621962318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c101de0 0x7f7e9c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.422+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/3621962318 >> 172.21.15.5:0/3621962318 conn(0x7f7e9c0fd8c0 msgr2=0x7f7e9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.422+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/3621962318 shutdown_connections 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.422+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/3621962318 wait complete. 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7ea1a6d640 1 Processor -- start 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7ea1a6d640 1 -- start start 2024-09-11T22:22:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7ea1a6d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c101de0 0x7f7e9c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7ea1a6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c102790 0x7f7e9c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7ea1a6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c106f10 0x7f7e9c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7ea1a6d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c112010 con 0x7f7e9c106f10 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7ea1a6d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c111e90 con 0x7f7e9c101de0 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7ea1a6d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c112190 con 0x7f7e9c102790 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c106f10 0x7f7e9c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c106f10 0x7f7e9c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48362/0 (socket says 172.21.15.5:48362) 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7e9bfff640 1 -- 172.21.15.5:0/484937925 learned_addr learned my addr 172.21.15.5:0/484937925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.423+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c101de0 0x7f7e9c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e9affd640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c102790 0x7f7e9c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e9bfff640 1 -- 172.21.15.5:0/484937925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c102790 msgr2=0x7f7e9c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c102790 0x7f7e9c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e9bfff640 1 -- 172.21.15.5:0/484937925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c101de0 msgr2=0x7f7e9c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c101de0 0x7f7e9c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e9bfff640 1 -- 172.21.15.5:0/484937925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c19d560 con 0x7f7e9c106f10 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c101de0 0x7f7e9c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c106f10 0x7f7e9c19ce80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009fd0 tx=0x7f7e9400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e98ff9640 1 -- 172.21.15.5:0/484937925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94011830 con 0x7f7e9c106f10 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e98ff9640 1 -- 172.21.15.5:0/484937925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e94016070 con 0x7f7e9c106f10 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/484937925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c19d850 con 0x7f7e9c106f10 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7e98ff9640 1 -- 172.21.15.5:0/484937925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400f2e0 con 0x7f7e9c106f10 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.424+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/484937925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c1a5130 con 0x7f7e9c106f10 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.426+0000 7f7e98ff9640 1 -- 172.21.15.5:0/484937925 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7e940119d0 con 0x7f7e9c106f10 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7ea3e46740 1 Processor -- start 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7ea3e46740 1 -- start start 2024-09-11T22:22:08.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7e98ff9640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7e840a2c00 0x7f7e840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c68d0 0x5562551c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c71f0 0x5562551d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551d5fc0 0x5562551d83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55625512d090 con 0x5562551d5fc0 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5562550d4a70 con 0x5562551c68d0 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556255129050 con 0x5562551c71f0 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7e9affd640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7e840a2c00 0x7f7e840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7e98ff9640 1 -- 172.21.15.5:0/484937925 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f7e940ce700 con 0x7f7e9c106f10 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551d5fc0 0x5562551d83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551d5fc0 0x5562551d83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48364/0 (socket says 172.21.15.5:48364) 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7e9bfff640 1 -- 172.21.15.5:0/803511183 learned_addr learned my addr 172.21.15.5:0/803511183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:08.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/803511183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c68d0 0x5562551c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.427+0000 7f7e9affd640 1 --2- 172.21.15.5:0/803511183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c71f0 0x5562551d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.428+0000 7f7e9bfff640 1 -- 172.21.15.5:0/803511183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c71f0 msgr2=0x5562551d5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.428+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/803511183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c71f0 0x5562551d5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.428+0000 7f7e9bfff640 1 -- 172.21.15.5:0/803511183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c68d0 msgr2=0x5562551c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.428+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/803511183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c68d0 0x5562551c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.428+0000 7f7e9bfff640 1 -- 172.21.15.5:0/803511183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562550e1130 con 0x5562551d5fc0 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.428+0000 7f7e9affd640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7e840a2c00 0x7f7e840a50c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000f80 tx=0x7f7e90031d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.428+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/803511183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551d5fc0 0x5562551d83b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7e940cc180 tx=0x7f7e940047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fa28e81d404c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.428+0000 7f7e81ffb640 1 -- 172.21.15.5:0/803511183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9409aee0 con 0x5562551d5fc0 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.428+0000 7f7e81ffb640 1 -- 172.21.15.5:0/803511183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9409f290 con 0x5562551d5fc0 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.428+0000 7f7e81ffb640 1 -- 172.21.15.5:0/803511183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e940bc310 con 0x5562551d5fc0 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 -- 172.21.15.5:0/803511183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551d5fc0 msgr2=0x5562551d83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/803511183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551d5fc0 0x5562551d83b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7e940cc180 tx=0x7f7e940047d0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 -- 172.21.15.5:0/803511183 shutdown_connections 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/803511183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551d5fc0 0x5562551d83b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/803511183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c71f0 0x5562551d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/803511183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c68d0 0x5562551c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 -- 172.21.15.5:0/803511183 >> 172.21.15.5:0/803511183 conn(0x5562551c2f60 msgr2=0x5562551c3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 -- 172.21.15.5:0/803511183 shutdown_connections 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 -- 172.21.15.5:0/803511183 wait complete. 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 -- start start 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.429+0000 7f7ea3e46740 1 Processor -- start 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c68d0 0x5562551d5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c71f0 0x5562551d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551d5fc0 0x5562551d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5562551d8b90 con 0x5562551c71f0 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5562550e7f20 con 0x5562551d5fc0 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5562551d8d10 con 0x5562551c68d0 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7ea3e46740 10 client.0 _ll_register_callbacks cb 0x5562550b1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c71f0 0x5562551d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c71f0 0x5562551d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48372/0 (socket says 172.21.15.5:48372) 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9affd640 1 -- 172.21.15.5:0/3258976963 learned_addr learned my addr 172.21.15.5:0/3258976963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551d5fc0 0x5562551d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c68d0 0x5562551d5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9affd640 1 -- 172.21.15.5:0/3258976963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c68d0 msgr2=0x5562551d5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9affd640 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c68d0 0x5562551d5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9affd640 1 -- 172.21.15.5:0/3258976963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551d5fc0 msgr2=0x5562551d30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:08.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9affd640 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551d5fc0 0x5562551d30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9affd640 1 -- 172.21.15.5:0/3258976963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556254e9e2b0 con 0x5562551c71f0 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c68d0 0x5562551d5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551d5fc0 0x5562551d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e9affd640 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c71f0 0x5562551d2b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009ca0 tx=0x7f7e9002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171945d51c0a87ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e7affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e7affd640 1 -- 172.21.15.5:0/3258976963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90004690 con 0x5562551c71f0 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3258976963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e9409f290 con 0x5562551c71f0 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.430+0000 7f7e7affd640 1 -- 172.21.15.5:0/3258976963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f7e9002f610 con 0x5562551c71f0 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7e7affd640 1 -- 172.21.15.5:0/3258976963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e90004830 con 0x5562551c71f0 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7ea3e46740 10 client.15171 fetch_fsmap learned FSMap version 27 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3258976963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562551e44e0 con 0x5562551c71f0 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7e7affd640 1 -- 172.21.15.5:0/3258976963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e9409f290 con 0x5562551c71f0 2024-09-11T22:22:08.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7e7affd640 1 -- 172.21.15.5:0/3258976963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7e90030920 con 0x5562551c71f0 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7ea3e46740 10 client.15171 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7ea3e46740 10 client.15171 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7ea3e46740 20 client.15171 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5562551e6be0 0x5562551e8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7ea3e46740 4 client.15171 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3258976963 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55625513a600 con 0x5562551e6be0 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.431+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5562551e6be0 0x5562551e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.432+0000 7f7e7affd640 10 client.15171 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.432+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5562551e6be0 0x5562551e8050 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c67ed3e64770f6 server_cookie=95fd843f15f6dc9b in_seq=0 out_seq=0 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7e7affd640 1 -- 172.21.15.5:0/3258976963 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55625513a600 con 0x5562551e6be0 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7e7affd640 10 client.15171 handle_command_reply: tid=0 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 client.15171 shutdown 2024-09-11T22:22:08.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3258976963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c71f0 msgr2=0x5562551d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c71f0 0x5562551d2b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009ca0 tx=0x7f7e9002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3258976963 shutdown_connections 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5562551e6be0 0x5562551e8050 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551d5fc0 0x5562551d30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c71f0 0x5562551d2b90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7e7affd640 0 client.15171 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3258976963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551c68d0 0x5562551d5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3258976963 >> 172.21.15.5:0/3258976963 conn(0x5562551c2f60 msgr2=0x5562551c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3258976963 shutdown_connections 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3258976963 wait complete. 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.433+0000 7f7ea3e46740 20 client.15171 trim_cache size 0 max 16384 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7ea3e46740 1 Processor -- start 2024-09-11T22:22:08.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7ea3e46740 1 -- start start 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551bdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 0x5562551f02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 0x5562551f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5562551e6390 con 0x5562551c68d0 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5562551e6a60 con 0x5562551c71f0 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5562551e7130 con 0x5562551d5fc0 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551bdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551bdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48384/0 (socket says 172.21.15.5:48384) 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.452+0000 7f7e9b7fe640 1 -- 172.21.15.5:0/1856646986 learned_addr learned my addr 172.21.15.5:0/1856646986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/1856646986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 0x5562551f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e9affd640 1 --2- 172.21.15.5:0/1856646986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 0x5562551f02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e9b7fe640 1 -- 172.21.15.5:0/1856646986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 msgr2=0x5562551f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/1856646986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 0x5562551f07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e9b7fe640 1 -- 172.21.15.5:0/1856646986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 msgr2=0x5562551f02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/1856646986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 0x5562551f02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e9b7fe640 1 -- 172.21.15.5:0/1856646986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562551d3950 con 0x5562551c68d0 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/1856646986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 0x5562551f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/1856646986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551bdce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5562551d8d70 tx=0x7f7e8c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b74887811863f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e817fa640 1 -- 172.21.15.5:0/1856646986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c020070 con 0x5562551c68d0 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e817fa640 1 -- 172.21.15.5:0/1856646986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8c007c70 con 0x5562551c68d0 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.453+0000 7f7e817fa640 1 -- 172.21.15.5:0/1856646986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c014610 con 0x5562551c68d0 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 -- 172.21.15.5:0/1856646986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 msgr2=0x5562551bdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/1856646986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551bdce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x5562551d8d70 tx=0x7f7e8c008040 comp rx=0 tx=0).stop 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 -- 172.21.15.5:0/1856646986 shutdown_connections 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/1856646986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 0x5562551f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/1856646986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 0x5562551f02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/1856646986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551bdce0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 -- 172.21.15.5:0/1856646986 >> 172.21.15.5:0/1856646986 conn(0x5562551c2f60 msgr2=0x5562551d1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 -- 172.21.15.5:0/1856646986 shutdown_connections 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 -- 172.21.15.5:0/1856646986 wait complete. 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 -- start start 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 Processor -- start 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551e9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 0x5562551e9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.454+0000 7f7ea3e46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 0x55625524f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5562551e7ed0 con 0x5562551c68d0 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5562551e7800 con 0x5562551c71f0 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7ea3e46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5562551e85a0 con 0x5562551d5fc0 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7ea3e46740 10 client.0 _ll_register_callbacks cb 0x7f7e90009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551e9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551e9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48386/0 (socket says 172.21.15.5:48386) 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9b7fe640 1 -- 172.21.15.5:0/3323782221 learned_addr learned my addr 172.21.15.5:0/3323782221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9affd640 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 0x5562551e9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9bfff640 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 0x55625524f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9b7fe640 1 -- 172.21.15.5:0/3323782221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 msgr2=0x55625524f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 0x55625524f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9b7fe640 1 -- 172.21.15.5:0/3323782221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 msgr2=0x5562551e9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 0x5562551e9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9b7fe640 1 -- 172.21.15.5:0/3323782221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55625524f820 con 0x5562551c68d0 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9affd640 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 0x5562551e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e9b7fe640 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551e9900 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c01c4e0 tx=0x7f7e8c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a3a45b1e69c3ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e7affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e7affd640 1 -- 172.21.15.5:0/3323782221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c029030 con 0x5562551c68d0 2024-09-11T22:22:08.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e7affd640 1 -- 172.21.15.5:0/3323782221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f7e8c025450 con 0x5562551c68d0 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3323782221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8c007c70 con 0x5562551c68d0 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7e7affd640 1 -- 172.21.15.5:0/3323782221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e8c035050 con 0x5562551c68d0 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7ea3e46740 10 client.15177 fetch_fsmap learned FSMap version 27 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.455+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3323782221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556255250a10 con 0x5562551c68d0 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.456+0000 7f7e7affd640 1 -- 172.21.15.5:0/3323782221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e8c007c70 con 0x5562551c68d0 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.456+0000 7f7e7affd640 1 -- 172.21.15.5:0/3323782221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7e8c02c5a0 con 0x5562551c68d0 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.456+0000 7f7ea3e46740 10 client.15177 fetch_fsmap finished waiting for FSMap version 27 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.456+0000 7f7ea3e46740 10 client.15177 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.456+0000 7f7ea3e46740 20 client.15177 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.456+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556255255270 0x556255253f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.456+0000 7f7ea3e46740 4 client.15177 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.456+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3323782221 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "status"}) -- 0x556255134b10 con 0x556255255270 2024-09-11T22:22:08.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.456+0000 7f7e9affd640 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556255255270 0x556255253f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7e9affd640 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556255255270 0x556255253f80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2becfacd76d85842 server_cookie=e5a0a88816b20cfd in_seq=0 out_seq=0 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7e7affd640 10 client.15177 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7e7affd640 1 -- 172.21.15.5:0/3323782221 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x556255134b10 con 0x556255255270 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7e7affd640 10 client.15177 handle_command_reply: tid=0 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 client.15177 shutdown 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3323782221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 msgr2=0x5562551e9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551e9900 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c01c4e0 tx=0x7f7e8c01c510 comp rx=0 tx=0).stop 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3323782221 shutdown_connections 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556255255270 0x556255253f80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5562551d5fc0 0x55625524f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5562551c71f0 0x5562551e9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 --2- 172.21.15.5:0/3323782221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562551c68d0 0x5562551e9900 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7e7affd640 0 client.15177 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:08.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3323782221 >> 172.21.15.5:0/3323782221 conn(0x5562551c2f60 msgr2=0x5562551ce240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3323782221 shutdown_connections 2024-09-11T22:22:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 1 -- 172.21.15.5:0/3323782221 wait complete. 2024-09-11T22:22:08.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.457+0000 7f7ea3e46740 20 client.15177 trim_cache size 0 max 16384 2024-09-11T22:22:08.475 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:08.475 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:22:08.475 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-11T22:22:08.475 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24500, 2024-09-11T22:22:08.475 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:22:08.475 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:22:08.475 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:22:08.476 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 109.278133924, 2024-09-11T22:22:08.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 27, 2024-09-11T22:22:08.476 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 82, 2024-09-11T22:22:08.476 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 80, 2024-09-11T22:22:08.476 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 119.686448713 2024-09-11T22:22:08.476 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/484937925 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7e840a2c00 msgr2=0x7f7e840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7e840a2c00 0x7f7e840a50c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000f80 tx=0x7f7e90031d80 comp rx=0 tx=0).stop 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/484937925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c106f10 msgr2=0x7f7e9c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c106f10 0x7f7e9c19ce80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009fd0 tx=0x7f7e9400b8e0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/484937925 shutdown_connections 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f7e840a2c00 0x7f7e840a50c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e9c106f10 0x7f7e9c19ce80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e9c102790 0x7f7e9c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 --2- 172.21.15.5:0/484937925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e9c101de0 0x7f7e9c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/484937925 >> 172.21.15.5:0/484937925 conn(0x7f7e9c0fd8c0 msgr2=0x7f7e9c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/484937925 shutdown_connections 2024-09-11T22:22:08.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:08.458+0000 7f7ea1a6d640 1 -- 172.21.15.5:0/484937925 wait complete. 2024-09-11T22:22:09.398 INFO:teuthology.run_tasks:Running task sequential... 2024-09-11T22:22:09.408 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-11T22:22:09.408 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-11T22:22:09.408 INFO:teuthology.task.print:Enabling mgr modules 2024-09-11T22:22:09.408 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-11T22:22:09.409 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T22:22:09.409 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T22:22:09.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 -- 172.21.15.5:0/267645305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe774101b60 msgr2=0x7fe774101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 --2- 172.21.15.5:0/267645305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe774101b60 0x7fe774101f60 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe764009900 tx=0x7fe76402a410 comp rx=0 tx=0).stop 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 -- 172.21.15.5:0/267645305 shutdown_connections 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 --2- 172.21.15.5:0/267645305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe774103710 0x7fe774109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 --2- 172.21.15.5:0/267645305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe774102d50 0x7fe7741031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 --2- 172.21.15.5:0/267645305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe774101b60 0x7fe774101f60 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 -- 172.21.15.5:0/267645305 >> 172.21.15.5:0/267645305 conn(0x7fe7740fd2e0 msgr2=0x7fe7740ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 -- 172.21.15.5:0/267645305 shutdown_connections 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 -- 172.21.15.5:0/267645305 wait complete. 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 Processor -- start 2024-09-11T22:22:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.524+0000 7fe77a642640 1 -- start start 2024-09-11T22:22:09.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe77a642640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe774101b60 0x7fe77419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe77a642640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe774102d50 0x7fe77419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe77a642640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe774103710 0x7fe7741a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe77a642640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7740750d0 con 0x7fe774102d50 2024-09-11T22:22:09.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe77a642640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe774074f50 con 0x7fe774101b60 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe77a642640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe774075250 con 0x7fe774103710 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe774102d50 0x7fe77419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe774102d50 0x7fe77419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48398/0 (socket says 172.21.15.5:48398) 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7737fe640 1 -- 172.21.15.5:0/2877710622 learned_addr learned my addr 172.21.15.5:0/2877710622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7737fe640 1 -- 172.21.15.5:0/2877710622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe774103710 msgr2=0x7fe7741a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7737fe640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe774103710 0x7fe7741a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7737fe640 1 -- 172.21.15.5:0/2877710622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe774101b60 msgr2=0x7fe77419a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7737fe640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe774101b60 0x7fe77419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7737fe640 1 -- 172.21.15.5:0/2877710622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7741a1c30 con 0x7fe774102d50 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7737fe640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe774102d50 0x7fe77419a550 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe76800d770 tx=0x7fe76800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7717fa640 1 -- 172.21.15.5:0/2877710622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768017070 con 0x7fe774102d50 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7717fa640 1 -- 172.21.15.5:0/2877710622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe768005500 con 0x7fe774102d50 2024-09-11T22:22:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe7717fa640 1 -- 172.21.15.5:0/2877710622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768004e00 con 0x7fe774102d50 2024-09-11T22:22:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.525+0000 7fe77a642640 1 -- 172.21.15.5:0/2877710622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741a1f20 con 0x7fe774102d50 2024-09-11T22:22:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.526+0000 7fe77a642640 1 -- 172.21.15.5:0/2877710622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7741a2390 con 0x7fe774102d50 2024-09-11T22:22:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.526+0000 7fe77a642640 1 -- 172.21.15.5:0/2877710622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7741a25f0 con 0x7fe774102d50 2024-09-11T22:22:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.528+0000 7fe7717fa640 1 -- 172.21.15.5:0/2877710622 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fe768004fa0 con 0x7fe774102d50 2024-09-11T22:22:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.529+0000 7fe7717fa640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe75c0a2c30 0x7fe75c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.529+0000 7fe773fff640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe75c0a2c30 0x7fe75c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.529+0000 7fe7717fa640 1 -- 172.21.15.5:0/2877710622 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fe7680cefa0 con 0x7fe774102d50 2024-09-11T22:22:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.529+0000 7fe773fff640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe75c0a2c30 0x7fe75c0a50f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe7640096c0 tx=0x7fe764031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:09.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.532+0000 7fe7717fa640 1 -- 172.21.15.5:0/2877710622 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe76809be80 con 0x7fe774102d50 2024-09-11T22:22:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:09.759+0000 7fe77a642640 1 -- 172.21.15.5:0/2877710622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fe7741a27c0 con 0x7fe774102d50 2024-09-11T22:22:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:09 smithi005 ceph-mon[30943]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:09 smithi159 ceph-mon[35534]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:10.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:09 smithi172 ceph-mon[35461]: pgmap v338: 97 pgs: 97 active+clean; 642 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:10.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.766+0000 7fe7717fa640 1 -- 172.21.15.5:0/2877710622 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fe76809b780 con 0x7fe774102d50 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.769+0000 7fe77a642640 1 -- 172.21.15.5:0/2877710622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe75c0a2c30 msgr2=0x7fe75c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.769+0000 7fe77a642640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe75c0a2c30 0x7fe75c0a50f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe7640096c0 tx=0x7fe764031400 comp rx=0 tx=0).stop 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.769+0000 7fe77a642640 1 -- 172.21.15.5:0/2877710622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe774102d50 msgr2=0x7fe77419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.769+0000 7fe77a642640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe774102d50 0x7fe77419a550 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe76800d770 tx=0x7fe76800dc30 comp rx=0 tx=0).stop 2024-09-11T22:22:10.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.769+0000 7fe77a642640 1 -- 172.21.15.5:0/2877710622 shutdown_connections 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.769+0000 7fe77a642640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe75c0a2c30 0x7fe75c0a50f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.769+0000 7fe77a642640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe774103710 0x7fe7741a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.769+0000 7fe77a642640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe774102d50 0x7fe77419a550 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.770+0000 7fe77a642640 1 --2- 172.21.15.5:0/2877710622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe774101b60 0x7fe77419a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.770+0000 7fe77a642640 1 -- 172.21.15.5:0/2877710622 >> 172.21.15.5:0/2877710622 conn(0x7fe7740fd2e0 msgr2=0x7fe7740fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.770+0000 7fe77a642640 1 -- 172.21.15.5:0/2877710622 shutdown_connections 2024-09-11T22:22:10.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:10.770+0000 7fe77a642640 1 -- 172.21.15.5:0/2877710622 wait complete. 2024-09-11T22:22:10.788 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T22:22:10.819 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T22:22:10.819 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-11T22:22:10.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:10.819 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-11T22:22:10.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:10.837 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-11T22:22:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-09-11T22:22:10.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:22:10.908 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-11T22:22:10.908 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T22:22:10.909 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T22:22:10.909 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-11T22:22:11.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:10 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2877710622' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T22:22:11.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:10 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:11.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:10 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:11.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:10 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:10.928+0000 7f6e87851040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:22:11.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:10 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:10.989+0000 7f6e87851040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:22:11.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:10 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2877710622' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T22:22:11.079 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:10 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:11.079 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:10 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:11.079 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:10 smithi159 ceph-mgr[36903]: -- 172.21.15.159:0/2049454465 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4dded30e0 con 0x55a4ddeb1000 2024-09-11T22:22:11.079 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:10 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:10.921+0000 7eff7315e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:22:11.079 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:10 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:10.978+0000 7eff7315e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:22:11.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:10 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2877710622' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-11T22:22:11.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: ignoring --setuser ceph since I am not root 2024-09-11T22:22:11.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: ignoring --setgroup ceph since I am not root 2024-09-11T22:22:11.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:10 smithi172 ceph-mgr[37268]: -- 172.21.15.172:0/2756356021 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0d64cd0e0 con 0x55e0d64aa800 2024-09-11T22:22:11.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:10.916+0000 7fe7c9819040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T22:22:11.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:10.972+0000 7fe7c9819040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T22:22:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:11.472 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:11.120+0000 7fe7c9819040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:22:11.472 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:11.186+0000 7fe7c9819040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:22:11.473 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:11.332+0000 7fe7c9819040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:22:11.488 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:11 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:11.128+0000 7eff7315e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:22:11.488 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:11 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:11.196+0000 7eff7315e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:22:11.488 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:11 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:11.345+0000 7eff7315e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:22:11.488 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:11 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:11.487+0000 7eff7315e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:22:11.509 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:11.153+0000 7f6e87851040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-11T22:22:11.509 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:11.228+0000 7f6e87851040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T22:22:11.509 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:11.386+0000 7f6e87851040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:22:11.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.733+0000 7fe093087640 1 -- 172.21.15.5:0/1932237007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe08c1082e0 msgr2=0x7fe08c10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:11.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.733+0000 7fe093087640 1 --2- 172.21.15.5:0/1932237007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe08c1082e0 0x7fe08c10ec50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe084009930 tx=0x7fe08402a420 comp rx=0 tx=0).stop 2024-09-11T22:22:11.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.733+0000 7fe093087640 1 -- 172.21.15.5:0/1932237007 shutdown_connections 2024-09-11T22:22:11.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.733+0000 7fe093087640 1 --2- 172.21.15.5:0/1932237007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe08c1082e0 0x7fe08c10ec50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.733+0000 7fe093087640 1 --2- 172.21.15.5:0/1932237007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe08c103760 0x7fe08c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.733+0000 7fe093087640 1 --2- 172.21.15.5:0/1932237007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe08c102db0 0x7fe08c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.733+0000 7fe093087640 1 -- 172.21.15.5:0/1932237007 >> 172.21.15.5:0/1932237007 conn(0x7fe08c0fc7d0 msgr2=0x7fe08c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.733+0000 7fe093087640 1 -- 172.21.15.5:0/1932237007 shutdown_connections 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.733+0000 7fe093087640 1 -- 172.21.15.5:0/1932237007 wait complete. 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe093087640 1 Processor -- start 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe093087640 1 -- start start 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe093087640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe08c102db0 0x7fe08c100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe093087640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe08c103760 0x7fe08c100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe093087640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe08c1082e0 0x7fe08c101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe093087640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c112100 con 0x7fe08c103760 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe093087640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c111f80 con 0x7fe08c1082e0 2024-09-11T22:22:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe093087640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe08c112280 con 0x7fe08c102db0 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe08bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe08c103760 0x7fe08c100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe08bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe08c103760 0x7fe08c100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36028/0 (socket says 172.21.15.5:36028) 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe08bfff640 1 -- 172.21.15.5:0/825726092 learned_addr learned my addr 172.21.15.5:0/825726092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe091624640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe08c1082e0 0x7fe08c101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.734+0000 7fe090e23640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe08c102db0 0x7fe08c100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe08bfff640 1 -- 172.21.15.5:0/825726092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe08c102db0 msgr2=0x7fe08c100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe08bfff640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe08c102db0 0x7fe08c100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe08bfff640 1 -- 172.21.15.5:0/825726092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe08c1082e0 msgr2=0x7fe08c101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe08bfff640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe08c1082e0 0x7fe08c101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe08bfff640 1 -- 172.21.15.5:0/825726092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08c1a6a10 con 0x7fe08c103760 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe091624640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe08c1082e0 0x7fe08c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:11.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe08bfff640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe08c103760 0x7fe08c100f50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00ebc0 tx=0x7fe07c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe089ffb640 1 -- 172.21.15.5:0/825726092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c010630 con 0x7fe08c103760 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe089ffb640 1 -- 172.21.15.5:0/825726092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe07c010cb0 con 0x7fe08c103760 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe093087640 1 -- 172.21.15.5:0/825726092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08c1a6d00 con 0x7fe08c103760 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe089ffb640 1 -- 172.21.15.5:0/825726092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07c017c20 con 0x7fe08c103760 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.735+0000 7fe093087640 1 -- 172.21.15.5:0/825726092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe08c1a7230 con 0x7fe08c103760 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.736+0000 7fe093087640 1 -- 172.21.15.5:0/825726092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe08c103190 con 0x7fe08c103760 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.737+0000 7fe089ffb640 1 -- 172.21.15.5:0/825726092 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137849+0+0 (secure 0 0 0) 0x7fe07c0107d0 con 0x7fe08c103760 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.738+0000 7fe089ffb640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe0780a2c50 0x7fe0780a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.738+0000 7fe090e23640 1 -- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe0780a2c50 msgr2=0x7fe0780a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3591749770 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.738+0000 7fe089ffb640 1 -- 172.21.15.5:0/825726092 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fe07c0cf930 con 0x7fe08c103760 2024-09-11T22:22:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.738+0000 7fe090e23640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe0780a2c50 0x7fe0780a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:22:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.747+0000 7fe089ffb640 1 -- 172.21.15.5:0/825726092 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe07c09c870 con 0x7fe08c103760 2024-09-11T22:22:11.768 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:11.471+0000 7fe7c9819040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:22:11.768 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:11.537+0000 7f6e87851040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T22:22:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.939+0000 7fe090e23640 1 -- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe0780a2c50 msgr2=0x7fe0780a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3591749770 2024-09-11T22:22:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.939+0000 7fe090e23640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe0780a2c50 0x7fe0780a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:22:11.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.962+0000 7fe093087640 1 -- 172.21.15.5:0/825726092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe08c10a2c0 con 0x7fe08c103760 2024-09-11T22:22:11.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.963+0000 7fe089ffb640 1 -- 172.21.15.5:0/825726092 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7fe07c09c170 con 0x7fe08c103760 2024-09-11T22:22:11.964 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T22:22:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 -- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe0780a2c50 msgr2=0x7fe0780a5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:22:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe0780a2c50 0x7fe0780a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 -- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe08c103760 msgr2=0x7fe08c100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe08c103760 0x7fe08c100f50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00ebc0 tx=0x7fe07c00c6a0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 -- 172.21.15.5:0/825726092 shutdown_connections 2024-09-11T22:22:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fe0780a2c50 0x7fe0780a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe08c1082e0 0x7fe08c101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe08c103760 0x7fe08c100f50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 --2- 172.21.15.5:0/825726092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe08c102db0 0x7fe08c100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 -- 172.21.15.5:0/825726092 >> 172.21.15.5:0/825726092 conn(0x7fe08c0fc7d0 msgr2=0x7fe08c105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 -- 172.21.15.5:0/825726092 shutdown_connections 2024-09-11T22:22:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:11.965+0000 7fe093087640 1 -- 172.21.15.5:0/825726092 wait complete. 2024-09-11T22:22:12.024 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:11 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2877710622' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T22:22:12.024 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:11 smithi172 ceph-mon[35461]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T22:22:12.025 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:11.781+0000 7fe7c9819040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:22:12.025 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:11.841+0000 7fe7c9819040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:22:12.048 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:11 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2877710622' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T22:22:12.048 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:11 smithi159 ceph-mon[35534]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T22:22:12.048 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:11 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:11.801+0000 7eff7315e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:22:12.048 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:11 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:11.863+0000 7eff7315e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:22:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:11 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2877710622' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-11T22:22:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:11 smithi005 ceph-mon[30943]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-11T22:22:12.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:11.874+0000 7f6e87851040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T22:22:12.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:11.938+0000 7f6e87851040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T22:22:12.329 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:12 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:12.048+0000 7eff7315e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:22:12.359 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:12.023+0000 7fe7c9819040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:22:12.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:12 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:12.136+0000 7f6e87851040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T22:22:12.836 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T22:22:12.836 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T22:22:12.836 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-09-11T22:22:12.836 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T22:22:12.836 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T22:22:12.836 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T22:22:12.836 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T22:22:12.836 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T22:22:12.836 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T22:22:12.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:12.837 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-09-11T22:22:12.853 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T22:22:12.853 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T22:22:12.853 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T22:22:12.853 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:22:12.853 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42136sec preferred_lft 42136sec 2024-09-11T22:22:12.854 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-11T22:22:12.855 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-11T22:22:12.855 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-11T22:22:12.855 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-09-11T22:22:12.855 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-09-11T22:22:12.855 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T22:22:12.855 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:22:12.856 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-11T22:22:12.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-09-11T22:22:12.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-09-11T22:22:12.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-09-11T22:22:12.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T22:22:12.856 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-11T22:22:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:22:13.026 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:12 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/825726092' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:22:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:22:13.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:13.075 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T22:22:13.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:12 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/825726092' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:22:13.079 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:12 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:12.934+0000 7eff7315e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:22:13.079 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:13 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:13.001+0000 7eff7315e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:22:13.103 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-09-11T22:22:13.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:13.105 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-09-11T22:22:13.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:12 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/825726092' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-11T22:22:13.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:12.901+0000 7fe7c9819040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:22:13.110 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:12.967+0000 7fe7c9819040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:22:13.178 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-09-11T22:22:13.178 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T22:22:13.178 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-11T22:22:13.178 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-11T22:22:13.178 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T22:22:13.179 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:22:13.179 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-11T22:22:13.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-11T22:22:13.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-11T22:22:13.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-11T22:22:13.179 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-11T22:22:13.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:22:13.277 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:13 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:13.079+0000 7f6e87851040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T22:22:13.277 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:13 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:13.156+0000 7f6e87851040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T22:22:13.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:22:13.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:13.595 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-11T22:22:13.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:13.613 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-09-11T22:22:13.670 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:22:13.670 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-11T22:22:13.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:22:13.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T22:22:13.670 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-11T22:22:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:22:13.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:22:13.797 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T22:22:13.797 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:22:13.797 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-11T22:22:13.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T22:22:13.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T22:22:13.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T22:22:13.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T22:22:13.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T22:22:13.798 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-11T22:22:13.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:22:13.935 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:13.676+0000 7fe7c9819040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:22:13.936 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:13.791+0000 7fe7c9819040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:22:13.976 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:13 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:13.716+0000 7eff7315e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:22:13.976 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:13 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:13.833+0000 7eff7315e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:22:14.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:22:14.026 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T22:22:14.026 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-11T22:22:14.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-09-11T22:22:14.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T22:22:14.026 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-11T22:22:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:22:14.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:22:14.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T22:22:14.109 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T22:22:14.124 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T22:22:14.125 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T22:22:14.125 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T22:22:14.182 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-11T22:22:14.182 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-11T22:22:14.182 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 2024-09-11T22:22:14.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:14.183 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-11T22:22:14.214 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:13.935+0000 7fe7c9819040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:22:14.214 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:13.996+0000 7fe7c9819040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:22:14.214 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:14.145+0000 7fe7c9819040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:22:14.214 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:14.213+0000 7fe7c9819040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:22:14.228 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:13 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:13.940+0000 7f6e87851040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T22:22:14.228 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:14.071+0000 7f6e87851040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T22:22:14.260 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:13 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:13.975+0000 7eff7315e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:22:14.260 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:14.038+0000 7eff7315e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:22:14.260 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:14.190+0000 7eff7315e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:22:14.260 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:14.259+0000 7eff7315e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:22:14.288 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-11T22:22:14.288 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-11T22:22:14.288 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-09-11T22:22:14.421 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-11T22:22:14.421 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-11T22:22:14.421 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-09-11T22:22:14.421 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.159:3300,172.21.15.172:3300" 2024-09-11T22:22:14.421 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:3300,172.21.15.159:3300,172.21.15.172:3300:/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 2024-09-11T22:22:14.421 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=b99b16b2-708a-11ef-bcea-c7b262605968" will pass to kernel 2024-09-11T22:22:14.422 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-11T22:22:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-09-11T22:22:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-11T22:22:14.422 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-09-11T22:22:14.423 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-11T22:22:14.424 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T22:22:14.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:22:14.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T22:22:14.488 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:14.227+0000 7f6e87851040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T22:22:14.488 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:14.294+0000 7f6e87851040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T22:22:14.488 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:14.457+0000 7f6e87851040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T22:22:14.493 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T22:22:14.500 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:14.274+0000 7fe7c9819040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:22:14.500 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:14.434+0000 7fe7c9819040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:22:14.500 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:14.499+0000 7fe7c9819040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:22:14.548 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:14.321+0000 7eff7315e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:22:14.548 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:14.483+0000 7eff7315e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:22:14.549 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:14.548+0000 7eff7315e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:22:14.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T22:22:14.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:22:14.592 INFO:teuthology.orchestra.run.smithi005.stdout:client24841 2024-09-11T22:22:14.592 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:22:14.592 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:22:14.610 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T22:22:14.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T22:22:14.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:22:14.708 INFO:teuthology.orchestra.run.smithi005.stdout:client24841 2024-09-11T22:22:14.709 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T22:22:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T22:22:14.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:22:14.765 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:14.531+0000 7f6e87851040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T22:22:14.765 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:14.595+0000 7f6e87851040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T22:22:14.765 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:14.765+0000 7f6e87851040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T22:22:14.766 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:14.566+0000 7fe7c9819040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:22:14.766 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:14.648+0000 7fe7c9819040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:22:14.766 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:14.707+0000 7fe7c9819040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:22:14.766 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:14.765+0000 7fe7c9819040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:22:14.767 INFO:teuthology.orchestra.run.smithi005.stdout:client24841 2024-09-11T22:22:14.767 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b99b16b2-708a-11ef-bcea-c7b262605968.client24841/status') 2024-09-11T22:22:14.816 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:14.615+0000 7eff7315e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:22:14.816 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:14.697+0000 7eff7315e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:22:14.816 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:14.757+0000 7eff7315e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:22:14.816 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:14.816+0000 7eff7315e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:22:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T22:22:14.828 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.24841 (3)172.21.15.5:0/565534446 2024-09-11T22:22:14.828 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-09-11T22:22:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-09-11T22:22:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-09-11T22:22:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:69 bytes copied, 2.4589e-05 s, 2.8 MB/s 2024-09-11T22:22:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T22:22:14.835 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T22:22:14.845 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T22:22:14.845 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T22:22:14.845 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-11T22:22:14.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.951+0000 7fd46f518640 1 -- 172.21.15.5:0/3070791205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4681003b0 msgr2=0x7fd4681007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:14.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.951+0000 7fd46f518640 1 --2- 172.21.15.5:0/3070791205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4681003b0 0x7fd4681007b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd464009900 tx=0x7fd46402a410 comp rx=0 tx=0).stop 2024-09-11T22:22:14.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.951+0000 7fd46f518640 1 -- 172.21.15.5:0/3070791205 shutdown_connections 2024-09-11T22:22:14.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.951+0000 7fd46f518640 1 --2- 172.21.15.5:0/3070791205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd468101690 0x7fd468101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:14.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.951+0000 7fd46f518640 1 --2- 172.21.15.5:0/3070791205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd468100cf0 0x7fd468101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:14.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.951+0000 7fd46f518640 1 --2- 172.21.15.5:0/3070791205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4681003b0 0x7fd4681007b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:14.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.951+0000 7fd46f518640 1 -- 172.21.15.5:0/3070791205 >> 172.21.15.5:0/3070791205 conn(0x7fd4680fc1e0 msgr2=0x7fd4680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:14.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.951+0000 7fd46f518640 1 -- 172.21.15.5:0/3070791205 shutdown_connections 2024-09-11T22:22:14.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.951+0000 7fd46f518640 1 -- 172.21.15.5:0/3070791205 wait complete. 2024-09-11T22:22:14.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46f518640 1 Processor -- start 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46f518640 1 -- start start 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46f518640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4681003b0 0x7fd468199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46f518640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd468100cf0 0x7fd46819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46f518640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd468101690 0x7fd4681a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46f518640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd46810f9b0 con 0x7fd468100cf0 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46f518640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd46810f830 con 0x7fd4681003b0 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46f518640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd46810fb30 con 0x7fd468101690 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46cab3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd468100cf0 0x7fd46819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46cab3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd468100cf0 0x7fd46819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36034/0 (socket says 172.21.15.5:36034) 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46cab3640 1 -- 172.21.15.5:0/3775306701 learned_addr learned my addr 172.21.15.5:0/3775306701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:14.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46d2b4640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4681003b0 0x7fd468199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46dab5640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd468101690 0x7fd4681a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46cab3640 1 -- 172.21.15.5:0/3775306701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd468101690 msgr2=0x7fd4681a1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46cab3640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd468101690 0x7fd4681a1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46cab3640 1 -- 172.21.15.5:0/3775306701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4681003b0 msgr2=0x7fd468199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46cab3640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4681003b0 0x7fd468199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46cab3640 1 -- 172.21.15.5:0/3775306701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4681a1bd0 con 0x7fd468100cf0 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.952+0000 7fd46d2b4640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4681003b0 0x7fd468199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.953+0000 7fd46cab3640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd468100cf0 0x7fd46819a4f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd45800e9f0 tx=0x7fd45800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.953+0000 7fd45e7fc640 1 -- 172.21.15.5:0/3775306701 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45800cd80 con 0x7fd468100cf0 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.953+0000 7fd45e7fc640 1 -- 172.21.15.5:0/3775306701 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd458005500 con 0x7fd468100cf0 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.953+0000 7fd45e7fc640 1 -- 172.21.15.5:0/3775306701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458010690 con 0x7fd468100cf0 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.953+0000 7fd46f518640 1 -- 172.21.15.5:0/3775306701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4681a1ec0 con 0x7fd468100cf0 2024-09-11T22:22:14.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.953+0000 7fd46f518640 1 -- 172.21.15.5:0/3775306701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4681a2380 con 0x7fd468100cf0 2024-09-11T22:22:14.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.954+0000 7fd46f518640 1 -- 172.21.15.5:0/3775306701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46810b100 con 0x7fd468100cf0 2024-09-11T22:22:14.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.955+0000 7fd45e7fc640 1 -- 172.21.15.5:0/3775306701 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137849+0+0 (secure 0 0 0) 0x7fd4580046c0 con 0x7fd468100cf0 2024-09-11T22:22:14.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.956+0000 7fd45e7fc640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd4540a2d00 0x7fd4540a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:14.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.956+0000 7fd46d2b4640 1 -- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd4540a2d00 msgr2=0x7fd4540a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3591749770 2024-09-11T22:22:14.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.956+0000 7fd45e7fc640 1 -- 172.21.15.5:0/3775306701 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fd4580cf410 con 0x7fd468100cf0 2024-09-11T22:22:14.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.956+0000 7fd46d2b4640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd4540a2d00 0x7fd4540a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:22:14.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:14.960+0000 7fd45e7fc640 1 -- 172.21.15.5:0/3775306701 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd45809c350 con 0x7fd468100cf0 2024-09-11T22:22:15.062 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:14.835+0000 7f6e87851040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T22:22:15.062 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:14.908+0000 7f6e87851040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T22:22:15.062 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:14.998+0000 7f6e87851040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T22:22:15.062 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:15.061+0000 7f6e87851040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T22:22:15.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.157+0000 7fd46d2b4640 1 -- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd4540a2d00 msgr2=0x7fd4540a51c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3591749770 2024-09-11T22:22:15.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.157+0000 7fd46d2b4640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd4540a2d00 0x7fd4540a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:22:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.178+0000 7fd46f518640 1 -- 172.21.15.5:0/3775306701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fd468062e90 con 0x7fd468100cf0 2024-09-11T22:22:15.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.230+0000 7fd45e7fc640 1 -- 172.21.15.5:0/3775306701 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7fd45809bc50 con 0x7fd468100cf0 2024-09-11T22:22:15.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.232+0000 7fd46f518640 1 -- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd4540a2d00 msgr2=0x7fd4540a51c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:22:15.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.232+0000 7fd46f518640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd4540a2d00 0x7fd4540a51c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.232+0000 7fd46f518640 1 -- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd468100cf0 msgr2=0x7fd46819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:15.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.232+0000 7fd46f518640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd468100cf0 0x7fd46819a4f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd45800e9f0 tx=0x7fd45800edf0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.232+0000 7fd46f518640 1 -- 172.21.15.5:0/3775306701 shutdown_connections 2024-09-11T22:22:15.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.232+0000 7fd46f518640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7fd4540a2d00 0x7fd4540a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.232+0000 7fd46f518640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd468101690 0x7fd4681a1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.232+0000 7fd46f518640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd468100cf0 0x7fd46819a4f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.233+0000 7fd46f518640 1 --2- 172.21.15.5:0/3775306701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd4681003b0 0x7fd468199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.233+0000 7fd46f518640 1 -- 172.21.15.5:0/3775306701 >> 172.21.15.5:0/3775306701 conn(0x7fd4680fc1e0 msgr2=0x7fd4680fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:15.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.233+0000 7fd46f518640 1 -- 172.21.15.5:0/3775306701 shutdown_connections 2024-09-11T22:22:15.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.233+0000 7fd46f518640 1 -- 172.21.15.5:0/3775306701 wait complete. 2024-09-11T22:22:15.245 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-11T22:22:15.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:15 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3775306701' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T22:22:15.320 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:15.123+0000 7f6e87851040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T22:22:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.352+0000 7f24a9ed7640 1 -- 172.21.15.5:0/2067928142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a4107880 msgr2=0x7f24a4109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.352+0000 7f24a9ed7640 1 --2- 172.21.15.5:0/2067928142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a4107880 0x7f24a4109c90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f249c009ca0 tx=0x7f249c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.352+0000 7f24a9ed7640 1 -- 172.21.15.5:0/2067928142 shutdown_connections 2024-09-11T22:22:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.352+0000 7f24a9ed7640 1 --2- 172.21.15.5:0/2067928142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a4107880 0x7f24a4109c90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.352+0000 7f24a9ed7640 1 --2- 172.21.15.5:0/2067928142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4104f50 0x7f24a4107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.352+0000 7f24a9ed7640 1 --2- 172.21.15.5:0/2067928142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24a4102620 0x7f24a4104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.352+0000 7f24a9ed7640 1 -- 172.21.15.5:0/2067928142 >> 172.21.15.5:0/2067928142 conn(0x7f24a40fc1e0 msgr2=0x7f24a40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.352+0000 7f24a9ed7640 1 -- 172.21.15.5:0/2067928142 shutdown_connections 2024-09-11T22:22:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.352+0000 7f24a9ed7640 1 -- 172.21.15.5:0/2067928142 wait complete. 2024-09-11T22:22:15.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f24a9ed7640 1 Processor -- start 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f24a9ed7640 1 -- start start 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f24a9ed7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4102620 0x7f24a4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f24a9ed7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a4104f50 0x7f24a4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f24a9ed7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24a4107880 0x7f24a419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f24a9ed7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24a410d7e0 con 0x7f24a4104f50 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f24a9ed7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f24a410d660 con 0x7f24a4107880 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f24a9ed7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f24a410d960 con 0x7f24a4102620 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f249bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a4104f50 0x7f24a4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f249bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a4104f50 0x7f24a4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36042/0 (socket says 172.21.15.5:36042) 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f249bfff640 1 -- 172.21.15.5:0/3699790429 learned_addr learned my addr 172.21.15.5:0/3699790429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f24a37fe640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4102620 0x7f24a4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.353+0000 7f24a3fff640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24a4107880 0x7f24a419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:15.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f249bfff640 1 -- 172.21.15.5:0/3699790429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4102620 msgr2=0x7f24a4197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f249bfff640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4102620 0x7f24a4197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f249bfff640 1 -- 172.21.15.5:0/3699790429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24a4107880 msgr2=0x7f24a419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f249bfff640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24a4107880 0x7f24a419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f249bfff640 1 -- 172.21.15.5:0/3699790429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a419fa50 con 0x7f24a4104f50 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f24a37fe640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24a4102620 0x7f24a4197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f249bfff640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a4104f50 0x7f24a4198370 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f248c00e9c0 tx=0x7f248c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c00cd80 con 0x7f24a4104f50 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f248c005500 con 0x7f24a4104f50 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f24a9ed7640 1 -- 172.21.15.5:0/3699790429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a419fd40 con 0x7f24a4104f50 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248c010690 con 0x7f24a4104f50 2024-09-11T22:22:15.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.354+0000 7f24a9ed7640 1 -- 172.21.15.5:0/3699790429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a41a0230 con 0x7f24a4104f50 2024-09-11T22:22:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.355+0000 7f24a9ed7640 1 -- 172.21.15.5:0/3699790429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a40fdcd0 con 0x7f24a4104f50 2024-09-11T22:22:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.357+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137849+0+0 (secure 0 0 0) 0x7f248c0046c0 con 0x7f24a4104f50 2024-09-11T22:22:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.357+0000 7f24a17fa640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f24900a2c80 0x7f24900a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:15.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.357+0000 7f24a37fe640 1 -- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f24900a2c80 msgr2=0x7f24900a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3591749770 2024-09-11T22:22:15.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.357+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f248c0cf650 con 0x7f24a4104f50 2024-09-11T22:22:15.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.358+0000 7f24a37fe640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f24900a2c80 0x7f24900a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-11T22:22:15.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.361+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f248c0541b0 con 0x7f24a4104f50 2024-09-11T22:22:15.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.559+0000 7f24a37fe640 1 -- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f24900a2c80 msgr2=0x7f24900a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3591749770 2024-09-11T22:22:15.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.559+0000 7f24a37fe640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f24900a2c80 0x7f24900a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-11T22:22:15.563 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:15 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3775306701' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T22:22:15.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.566+0000 7f24a9ed7640 1 -- 172.21.15.5:0/3699790429 --> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f24a40626d0 con 0x7f24900a2c80 2024-09-11T22:22:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:15 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3775306701' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-11T22:22:15.860 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:22:15 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z[37264]: 2024-09-11T22:22:15.563+0000 7fe7c9819040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:22:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.960+0000 7f24a37fe640 1 -- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f24900a2c80 msgr2=0x7f24900a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3591749770 2024-09-11T22:22:15.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.960+0000 7f24a37fe640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f24900a2c80 0x7f24900a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-11T22:22:15.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.978+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f248c098500 con 0x7f24a4104f50 2024-09-11T22:22:15.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.979+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f24900a2c80 msgr2=0x7f24900a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:22:15.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:15.979+0000 7f24a17fa640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/3591749770,v1:172.21.15.5:6801/3591749770] conn(0x7f24900a2c80 0x7f24900a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:16.079 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:22:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y[36899]: 2024-09-11T22:22:15.616+0000 7eff7315e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:22:16.224 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:15.966+0000 7f6e87851040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T22:22:16.536 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:16.223+0000 7f6df21e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:16.536 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:16.223+0000 7f6df21e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:16.536 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:16.223+0000 7f6df21e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:16.536 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:16.223+0000 7f6df21e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:16.536 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:16.223+0000 7f6df21e9640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:16.536 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:16.226+0000 7f6def1a3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:16.536 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:16.226+0000 7f6def1a3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:16.536 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:16.226+0000 7f6def1a3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:16.536 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:16.226+0000 7f6def1a3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:16.536 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:16.226+0000 7f6def1a3640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:16.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3775306701' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-11T22:22:16.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:22:16.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: Standby manager daemon z restarted 2024-09-11T22:22:16.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: Standby manager daemon z started 2024-09-11T22:22:16.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:22:16.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:22:16.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:22:16.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:22:16.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: Standby manager daemon y restarted 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: Standby manager daemon y started 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: Active manager daemon x restarted 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: Activating manager daemon x 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: mgrmap e20: x(active, starting, since 0.0126963s), standbys: z, y 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:22:16.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:22:16.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:22:16.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:22:16.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:22:16.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:22:16.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:22:16.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:22:16.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: Manager daemon x is now available 2024-09-11T22:22:16.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:16.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:22:16.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:22:16.563 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3775306701' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-11T22:22:16.563 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:22:16.563 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: Standby manager daemon z restarted 2024-09-11T22:22:16.563 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: Standby manager daemon z started 2024-09-11T22:22:16.563 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:22:16.564 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:22:16.564 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:22:16.564 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:22:16.564 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: Standby manager daemon y restarted 2024-09-11T22:22:16.564 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: Standby manager daemon y started 2024-09-11T22:22:16.564 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:22:16.564 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:22:16.564 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: Active manager daemon x restarted 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: Activating manager daemon x 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: mgrmap e20: x(active, starting, since 0.0126963s), standbys: z, y 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:22:16.565 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:22:16.566 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:22:16.567 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:22:16.568 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:22:16.568 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: Manager daemon x is now available 2024-09-11T22:22:16.568 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:16.568 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:22:16.568 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:22:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3775306701' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-11T22:22:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:22:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: Standby manager daemon z restarted 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: Standby manager daemon z started 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.172:0/568354980' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: Standby manager daemon y restarted 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: Standby manager daemon y started 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.? 172.21.15.159:0/2230625424' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: Active manager daemon x restarted 2024-09-11T22:22:16.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: Activating manager daemon x 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: osdmap e83: 12 total, 12 up, 12 in 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: mgrmap e20: x(active, starting, since 0.0126963s), standbys: z, y 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:22:16.581 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:22:16.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:22:16.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:22:16.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:22:16.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:22:16.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:22:16.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T22:22:16.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-11T22:22:16.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T22:22:16.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T22:22:16.582 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T22:22:16.583 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: Manager daemon x is now available 2024-09-11T22:22:16.584 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:16.584 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-11T22:22:16.584 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-11T22:22:16.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:16.985+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f248c0cce20 con 0x7f24a4104f50 2024-09-11T22:22:16.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:16.986+0000 7f24a17fa640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2490051920 0x7f2490051d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:16.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:16.986+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f248c015e80 con 0x7f2490051920 2024-09-11T22:22:16.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:16.991+0000 7f24a37fe640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2490051920 0x7f2490051d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:16.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:16.993+0000 7f24a37fe640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2490051920 0x7f2490051d00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2494009ba0 tx=0x7f2494005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.226+0000 7f24a17fa640 1 -- 172.21.15.5:0/3699790429 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f248c015e80 con 0x7f2490051920 2024-09-11T22:22:17.228 INFO:teuthology.orchestra.run.smithi005.stdout:{} 2024-09-11T22:22:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.228+0000 7f24a9ed7640 1 -- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2490051920 msgr2=0x7f2490051d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.228+0000 7f24a9ed7640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2490051920 0x7f2490051d00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2494009ba0 tx=0x7f2494005f90 comp rx=0 tx=0).stop 2024-09-11T22:22:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.228+0000 7f24a9ed7640 1 -- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a4104f50 msgr2=0x7f24a4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.228+0000 7f24a9ed7640 1 --2- 172.21.15.5:0/3699790429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a4104f50 0x7f24a4198370 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f248c00e9c0 tx=0x7f248c00ee80 comp rx=0 tx=0).stop 2024-09-11T22:22:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.228+0000 7f24a3fff640 1 -- 172.21.15.5:0/3699790429 reap_dead start 2024-09-11T22:22:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.228+0000 7f24a9ed7640 1 -- 172.21.15.5:0/3699790429 shutdown_connections 2024-09-11T22:22:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.228+0000 7f24a9ed7640 1 -- 172.21.15.5:0/3699790429 >> 172.21.15.5:0/3699790429 conn(0x7f24a40fc1e0 msgr2=0x7f24a4102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.228+0000 7f24a9ed7640 1 -- 172.21.15.5:0/3699790429 shutdown_connections 2024-09-11T22:22:17.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.228+0000 7f24a9ed7640 1 -- 172.21.15.5:0/3699790429 wait complete. 2024-09-11T22:22:17.241 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-11T22:22:17.252 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-11T22:22:17.283 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T22:22:17.307 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:17.000+0000 7f6dec0dd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:17.307 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:17.000+0000 7f6dec0dd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:17.307 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:17.000+0000 7f6dec0dd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:17.307 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:17.000+0000 7f6dec0dd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:17.307 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:17.000+0000 7f6dec0dd640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:22:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.391+0000 7f361f2d7640 1 -- 172.21.15.5:0/1664825978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36181025c0 msgr2=0x7f3618108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.391+0000 7f361f2d7640 1 --2- 172.21.15.5:0/1664825978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36181025c0 0x7f3618108e50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3610009ca0 tx=0x7f361002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.391+0000 7f361f2d7640 1 -- 172.21.15.5:0/1664825978 shutdown_connections 2024-09-11T22:22:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.391+0000 7f361f2d7640 1 --2- 172.21.15.5:0/1664825978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36181025c0 0x7f3618108e50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.391+0000 7f361f2d7640 1 --2- 172.21.15.5:0/1664825978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3618101c00 0x7f3618102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.391+0000 7f361f2d7640 1 --2- 172.21.15.5:0/1664825978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3618100a10 0x7f3618100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.391+0000 7f361f2d7640 1 -- 172.21.15.5:0/1664825978 >> 172.21.15.5:0/1664825978 conn(0x7f36180fc170 msgr2=0x7f36180fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.391+0000 7f361f2d7640 1 -- 172.21.15.5:0/1664825978 shutdown_connections 2024-09-11T22:22:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.391+0000 7f361f2d7640 1 -- 172.21.15.5:0/1664825978 wait complete. 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.392+0000 7f361f2d7640 1 Processor -- start 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.392+0000 7f361f2d7640 1 -- start start 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.392+0000 7f361f2d7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3618100a10 0x7f3618069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.392+0000 7f361f2d7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3618101c00 0x7f361819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.392+0000 7f361f2d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36181025c0 0x7f361819f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.392+0000 7f361f2d7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f361810c690 con 0x7f36181025c0 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.392+0000 7f361f2d7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f361810c510 con 0x7f3618101c00 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.392+0000 7f361f2d7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f361810c810 con 0x7f3618100a10 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361d874640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36181025c0 0x7f361819f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361d073640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3618100a10 0x7f3618069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:17.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361d874640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36181025c0 0x7f361819f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36172/0 (socket says 172.21.15.5:36172) 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361d874640 1 -- 172.21.15.5:0/2038602107 learned_addr learned my addr 172.21.15.5:0/2038602107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361c872640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3618101c00 0x7f361819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361d874640 1 -- 172.21.15.5:0/2038602107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3618100a10 msgr2=0x7f3618069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361d874640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3618100a10 0x7f3618069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361d874640 1 -- 172.21.15.5:0/2038602107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3618101c00 msgr2=0x7f361819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361d874640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3618101c00 0x7f361819cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361d874640 1 -- 172.21.15.5:0/2038602107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361819fa90 con 0x7f36181025c0 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361c872640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3618101c00 0x7f361819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361d874640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36181025c0 0x7f361819f180 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3610031ae0 tx=0x7f3610031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f360e7fc640 1 -- 172.21.15.5:0/2038602107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610003c30 con 0x7f36181025c0 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f360e7fc640 1 -- 172.21.15.5:0/2038602107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3610003dd0 con 0x7f36181025c0 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361f2d7640 1 -- 172.21.15.5:0/2038602107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361819fcc0 con 0x7f36181025c0 2024-09-11T22:22:17.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f360e7fc640 1 -- 172.21.15.5:0/2038602107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361002fcb0 con 0x7f36181025c0 2024-09-11T22:22:17.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.393+0000 7f361f2d7640 1 -- 172.21.15.5:0/2038602107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36181a0200 con 0x7f36181025c0 2024-09-11T22:22:17.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.394+0000 7f361f2d7640 1 -- 172.21.15.5:0/2038602107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3618107e20 con 0x7f36181025c0 2024-09-11T22:22:17.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.396+0000 7f360e7fc640 1 -- 172.21.15.5:0/2038602107 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f3610004050 con 0x7f36181025c0 2024-09-11T22:22:17.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.396+0000 7f360e7fc640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f36040a2a10 0x7f36040a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:17.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.397+0000 7f361d073640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f36040a2a10 0x7f36040a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:17.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.397+0000 7f360e7fc640 1 -- 172.21.15.5:0/2038602107 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f36100f0200 con 0x7f36181025c0 2024-09-11T22:22:17.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.397+0000 7f361d073640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f36040a2a10 0x7f36040a4ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3614005fd0 tx=0x7f361400b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.404+0000 7f360e7fc640 1 -- 172.21.15.5:0/2038602107 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f36100bd3c0 con 0x7f36181025c0 2024-09-11T22:22:17.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.581+0000 7f361f2d7640 1 -- 172.21.15.5:0/2038602107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f361802da10 con 0x7f36181025c0 2024-09-11T22:22:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.583+0000 7f360e7fc640 1 -- 172.21.15.5:0/2038602107 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23068 (secure 0 0 0) 0x7f36100bccc0 con 0x7f36181025c0 2024-09-11T22:22:17.584 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:17.585 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:22:15.968337+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2380674497":"2024-09-12T22:22:15.968309+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/3591749770":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/1577283141":"2024-09-12T22:22:15.968309+0000","172.21.15.5:6801/3591749770":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1618823752":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/785116270":"2024-09-12T22:22:15.968309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:22:17.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 -- 172.21.15.5:0/2038602107 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f36040a2a10 msgr2=0x7f36040a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f36040a2a10 0x7f36040a4ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3614005fd0 tx=0x7f361400b390 comp rx=0 tx=0).stop 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 -- 172.21.15.5:0/2038602107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36181025c0 msgr2=0x7f361819f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36181025c0 0x7f361819f180 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3610031ae0 tx=0x7f3610031ee0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 -- 172.21.15.5:0/2038602107 shutdown_connections 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36181025c0 0x7f361819f180 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f36040a2a10 0x7f36040a4ed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3618101c00 0x7f361819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 --2- 172.21.15.5:0/2038602107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3618100a10 0x7f3618069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 -- 172.21.15.5:0/2038602107 >> 172.21.15.5:0/2038602107 conn(0x7f36180fc170 msgr2=0x7f3618104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 -- 172.21.15.5:0/2038602107 shutdown_connections 2024-09-11T22:22:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.585+0000 7f361f2d7640 1 -- 172.21.15.5:0/2038602107 wait complete. 2024-09-11T22:22:17.598 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T22:15:32.851002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T22:20:14.556626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T22:20:17.154834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T22:22:17.598 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T22:22:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.745+0000 7efe553fd640 1 -- 172.21.15.5:0/666719121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe50101b20 msgr2=0x7efe50101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.745+0000 7efe553fd640 1 --2- 172.21.15.5:0/666719121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe50101b20 0x7efe50101f20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7efe40009900 tx=0x7efe4002a410 comp rx=0 tx=0).stop 2024-09-11T22:22:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.746+0000 7efe553fd640 1 -- 172.21.15.5:0/666719121 shutdown_connections 2024-09-11T22:22:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.746+0000 7efe553fd640 1 --2- 172.21.15.5:0/666719121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe501036d0 0x7efe50109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.746+0000 7efe553fd640 1 --2- 172.21.15.5:0/666719121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe50102d10 0x7efe50103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.746+0000 7efe553fd640 1 --2- 172.21.15.5:0/666719121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe50101b20 0x7efe50101f20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.746+0000 7efe553fd640 1 -- 172.21.15.5:0/666719121 >> 172.21.15.5:0/666719121 conn(0x7efe500fd280 msgr2=0x7efe500ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.746+0000 7efe553fd640 1 -- 172.21.15.5:0/666719121 shutdown_connections 2024-09-11T22:22:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.746+0000 7efe553fd640 1 -- 172.21.15.5:0/666719121 wait complete. 2024-09-11T22:22:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.746+0000 7efe553fd640 1 Processor -- start 2024-09-11T22:22:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.746+0000 7efe553fd640 1 -- start start 2024-09-11T22:22:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe553fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe50101b20 0x7efe50197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe553fd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe50102d10 0x7efe501983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe553fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe501036d0 0x7efe5019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe553fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe5010d7f0 con 0x7efe501036d0 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe553fd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efe5010d670 con 0x7efe50102d10 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe553fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efe5010d970 con 0x7efe50101b20 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe501036d0 0x7efe5019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe501036d0 0x7efe5019f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36192/0 (socket says 172.21.15.5:36192) 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe50101b20 0x7efe50197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4f7fe640 1 -- 172.21.15.5:0/1907044606 learned_addr learned my addr 172.21.15.5:0/1907044606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4e7fc640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe50102d10 0x7efe501983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4f7fe640 1 -- 172.21.15.5:0/1907044606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe50101b20 msgr2=0x7efe50197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4f7fe640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe50101b20 0x7efe50197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4f7fe640 1 -- 172.21.15.5:0/1907044606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe50102d10 msgr2=0x7efe501983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4f7fe640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe50102d10 0x7efe501983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4f7fe640 1 -- 172.21.15.5:0/1907044606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5019fac0 con 0x7efe501036d0 2024-09-11T22:22:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe4f7fe640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe501036d0 0x7efe5019f450 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efe4800eb10 tx=0x7efe4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe37fff640 1 -- 172.21.15.5:0/1907044606 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48004b20 con 0x7efe501036d0 2024-09-11T22:22:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe553fd640 1 -- 172.21.15.5:0/1907044606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5019fdb0 con 0x7efe501036d0 2024-09-11T22:22:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.747+0000 7efe37fff640 1 -- 172.21.15.5:0/1907044606 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe48004cc0 con 0x7efe501036d0 2024-09-11T22:22:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.748+0000 7efe37fff640 1 -- 172.21.15.5:0/1907044606 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe48010760 con 0x7efe501036d0 2024-09-11T22:22:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.748+0000 7efe553fd640 1 -- 172.21.15.5:0/1907044606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe501a0220 con 0x7efe501036d0 2024-09-11T22:22:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.749+0000 7efe553fd640 1 -- 172.21.15.5:0/1907044606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe50108fa0 con 0x7efe501036d0 2024-09-11T22:22:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.750+0000 7efe37fff640 1 -- 172.21.15.5:0/1907044606 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7efe48005020 con 0x7efe501036d0 2024-09-11T22:22:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.751+0000 7efe37fff640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe380a2990 0x7efe380a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.751+0000 7efe37fff640 1 -- 172.21.15.5:0/1907044606 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7efe480cf350 con 0x7efe501036d0 2024-09-11T22:22:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.751+0000 7efe4effd640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe380a2990 0x7efe380a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.752+0000 7efe4effd640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe380a2990 0x7efe380a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe4002aa00 tx=0x7efe400315b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:17.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.755+0000 7efe37fff640 1 -- 172.21.15.5:0/1907044606 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe4809c440 con 0x7efe501036d0 2024-09-11T22:22:17.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.944+0000 7efe553fd640 1 -- 172.21.15.5:0/1907044606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe50062e90 con 0x7efe501036d0 2024-09-11T22:22:17.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.944+0000 7efe37fff640 1 -- 172.21.15.5:0/1907044606 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7efe4809bd40 con 0x7efe501036d0 2024-09-11T22:22:17.946 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-11T22:22:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 -- 172.21.15.5:0/1907044606 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe380a2990 msgr2=0x7efe380a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe380a2990 0x7efe380a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe4002aa00 tx=0x7efe400315b0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 -- 172.21.15.5:0/1907044606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe501036d0 msgr2=0x7efe5019f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe501036d0 0x7efe5019f450 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7efe4800eb10 tx=0x7efe4800c2d0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 -- 172.21.15.5:0/1907044606 shutdown_connections 2024-09-11T22:22:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe501036d0 0x7efe5019f450 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe380a2990 0x7efe380a4e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efe50102d10 0x7efe501983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 --2- 172.21.15.5:0/1907044606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe50101b20 0x7efe50197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:17.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 -- 172.21.15.5:0/1907044606 >> 172.21.15.5:0/1907044606 conn(0x7efe500fd280 msgr2=0x7efe50106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:17.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 -- 172.21.15.5:0/1907044606 shutdown_connections 2024-09-11T22:22:17.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:17.946+0000 7efe553fd640 1 -- 172.21.15.5:0/1907044606 wait complete. 2024-09-11T22:22:17.960 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T22:22:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.084+0000 7fb4efc23640 1 -- 172.21.15.5:0/1379179408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e8104f40 msgr2=0x7fb4e8107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:18.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.084+0000 7fb4efc23640 1 --2- 172.21.15.5:0/1379179408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e8104f40 0x7fb4e8107330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009900 tx=0x7fb4d802a410 comp rx=0 tx=0).stop 2024-09-11T22:22:18.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.084+0000 7fb4efc23640 1 -- 172.21.15.5:0/1379179408 shutdown_connections 2024-09-11T22:22:18.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.084+0000 7fb4efc23640 1 --2- 172.21.15.5:0/1379179408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e8107870 0x7fb4e8109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.084+0000 7fb4efc23640 1 --2- 172.21.15.5:0/1379179408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e8104f40 0x7fb4e8107330 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.084+0000 7fb4efc23640 1 --2- 172.21.15.5:0/1379179408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4e8102610 0x7fb4e8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.084+0000 7fb4efc23640 1 -- 172.21.15.5:0/1379179408 >> 172.21.15.5:0/1379179408 conn(0x7fb4e80fc1b0 msgr2=0x7fb4e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.085+0000 7fb4efc23640 1 -- 172.21.15.5:0/1379179408 shutdown_connections 2024-09-11T22:22:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.085+0000 7fb4efc23640 1 -- 172.21.15.5:0/1379179408 wait complete. 2024-09-11T22:22:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4efc23640 1 Processor -- start 2024-09-11T22:22:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4efc23640 1 -- start start 2024-09-11T22:22:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4efc23640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4e8102610 0x7fb4e8101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4efc23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e8104f40 0x7fb4e81005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4efc23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e8107870 0x7fb4e8100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4efc23640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810d7d0 con 0x7fb4e8107870 2024-09-11T22:22:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4efc23640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810d650 con 0x7fb4e8102610 2024-09-11T22:22:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4efc23640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e810d950 con 0x7fb4e8104f40 2024-09-11T22:22:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ee1c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e8107870 0x7fb4e8100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ee1c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e8107870 0x7fb4e8100b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36204/0 (socket says 172.21.15.5:36204) 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ee1c0640 1 -- 172.21.15.5:0/65233088 learned_addr learned my addr 172.21.15.5:0/65233088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ed9bf640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4e8102610 0x7fb4e8101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ed1be640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e8104f40 0x7fb4e81005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ed9bf640 1 -- 172.21.15.5:0/65233088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e8104f40 msgr2=0x7fb4e81005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ed9bf640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e8104f40 0x7fb4e81005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ed9bf640 1 -- 172.21.15.5:0/65233088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e8107870 msgr2=0x7fb4e8100b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ed9bf640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e8107870 0x7fb4e8100b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ed9bf640 1 -- 172.21.15.5:0/65233088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e8101390 con 0x7fb4e8102610 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ee1c0640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e8107870 0x7fb4e8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.086+0000 7fb4ed1be640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e8104f40 0x7fb4e81005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.087+0000 7fb4ed9bf640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4e8102610 0x7fb4e8101f80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e9f0 tx=0x7fb4e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.087+0000 7fb4deffd640 1 -- 172.21.15.5:0/65233088 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e400ccc0 con 0x7fb4e8102610 2024-09-11T22:22:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.087+0000 7fb4efc23640 1 -- 172.21.15.5:0/65233088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e81a2060 con 0x7fb4e8102610 2024-09-11T22:22:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.087+0000 7fb4deffd640 1 -- 172.21.15.5:0/65233088 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4e4010910 con 0x7fb4e8102610 2024-09-11T22:22:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.087+0000 7fb4deffd640 1 -- 172.21.15.5:0/65233088 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e4018960 con 0x7fb4e8102610 2024-09-11T22:22:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.088+0000 7fb4efc23640 1 -- 172.21.15.5:0/65233088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e81a2600 con 0x7fb4e8102610 2024-09-11T22:22:18.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.089+0000 7fb4efc23640 1 -- 172.21.15.5:0/65233088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4e80fdca0 con 0x7fb4e8102610 2024-09-11T22:22:18.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.090+0000 7fb4deffd640 1 -- 172.21.15.5:0/65233088 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fb4e4046b30 con 0x7fb4e8102610 2024-09-11T22:22:18.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.090+0000 7fb4deffd640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4d40a2990 0x7fb4d40a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:18.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.090+0000 7fb4deffd640 1 -- 172.21.15.5:0/65233088 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fb4e40cf320 con 0x7fb4e8102610 2024-09-11T22:22:18.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.090+0000 7fb4ed1be640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4d40a2990 0x7fb4d40a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.091+0000 7fb4ed1be640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4d40a2990 0x7fb4d40a4e50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8033040 tx=0x7fb4d8005fd0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:18.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.093+0000 7fb4deffd640 1 -- 172.21.15.5:0/65233088 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb4e409c410 con 0x7fb4e8102610 2024-09-11T22:22:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.285+0000 7fb4efc23640 1 -- 172.21.15.5:0/65233088 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb4e8062df0 con 0x7fb4e8102610 2024-09-11T22:22:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.286+0000 7fb4deffd640 1 -- 172.21.15.5:0/65233088 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7fb4e4046e20 con 0x7fb4e8102610 2024-09-11T22:22:18.287 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-11T22:22:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 -- 172.21.15.5:0/65233088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4d40a2990 msgr2=0x7fb4d40a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4d40a2990 0x7fb4d40a4e50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8033040 tx=0x7fb4d8005fd0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 -- 172.21.15.5:0/65233088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4e8102610 msgr2=0x7fb4e8101f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4e8102610 0x7fb4e8101f80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb4e400e9f0 tx=0x7fb4e400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 -- 172.21.15.5:0/65233088 shutdown_connections 2024-09-11T22:22:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4e8107870 0x7fb4e8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e8104f40 0x7fb4e81005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4d40a2990 0x7fb4d40a4e50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 --2- 172.21.15.5:0/65233088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4e8102610 0x7fb4e8101f80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 -- 172.21.15.5:0/65233088 >> 172.21.15.5:0/65233088 conn(0x7fb4e80fc1b0 msgr2=0x7fb4e8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:18.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 -- 172.21.15.5:0/65233088 shutdown_connections 2024-09-11T22:22:18.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.288+0000 7fb4efc23640 1 -- 172.21.15.5:0/65233088 wait complete. 2024-09-11T22:22:18.299 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-mon[30943]: mgrmap e21: x(active, since 1.02466s), standbys: z, y 2024-09-11T22:22:18.299 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-mon[30943]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:18.300 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-mon[30943]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:18.300 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2038602107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:18.300 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1907044606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:22:18.300 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:17 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:18.301 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T22:22:18.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:17 smithi159 ceph-mon[35534]: mgrmap e21: x(active, since 1.02466s), standbys: z, y 2024-09-11T22:22:18.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:17 smithi159 ceph-mon[35534]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:18.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:17 smithi159 ceph-mon[35534]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:18.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:17 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2038602107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:18.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:17 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1907044606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:22:18.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:17 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:18.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:17 smithi172 ceph-mon[35461]: mgrmap e21: x(active, since 1.02466s), standbys: z, y 2024-09-11T22:22:18.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:17 smithi172 ceph-mon[35461]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:18.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:17 smithi172 ceph-mon[35461]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:18.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:17 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2038602107' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:18.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:17 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1907044606' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:22:18.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:17 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.410+0000 7f82e1be4640 1 -- 172.21.15.5:0/2867759527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc1025e0 msgr2=0x7f82dc1049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.410+0000 7f82e1be4640 1 --2- 172.21.15.5:0/2867759527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc1025e0 0x7f82dc1049d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009900 tx=0x7f82cc02a410 comp rx=0 tx=0).stop 2024-09-11T22:22:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.410+0000 7f82e1be4640 1 -- 172.21.15.5:0/2867759527 shutdown_connections 2024-09-11T22:22:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.410+0000 7f82e1be4640 1 --2- 172.21.15.5:0/2867759527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82dc107840 0x7f82dc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.410+0000 7f82e1be4640 1 --2- 172.21.15.5:0/2867759527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82dc104f10 0x7f82dc107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.410+0000 7f82e1be4640 1 --2- 172.21.15.5:0/2867759527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc1025e0 0x7f82dc1049d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.410+0000 7f82e1be4640 1 -- 172.21.15.5:0/2867759527 >> 172.21.15.5:0/2867759527 conn(0x7f82dc0fc140 msgr2=0x7f82dc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.410+0000 7f82e1be4640 1 -- 172.21.15.5:0/2867759527 shutdown_connections 2024-09-11T22:22:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.410+0000 7f82e1be4640 1 -- 172.21.15.5:0/2867759527 wait complete. 2024-09-11T22:22:18.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.410+0000 7f82e1be4640 1 Processor -- start 2024-09-11T22:22:18.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82e1be4640 1 -- start start 2024-09-11T22:22:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82e1be4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc1025e0 0x7f82dc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82e1be4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82dc104f10 0x7f82dc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82e1be4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82dc107840 0x7f82dc19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82e1be4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc10d7f0 con 0x7f82dc1025e0 2024-09-11T22:22:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82e1be4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc10d670 con 0x7f82dc107840 2024-09-11T22:22:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82e1be4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f82dc10d970 con 0x7f82dc104f10 2024-09-11T22:22:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82db7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc1025e0 0x7f82dc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:18.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82daffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82dc104f10 0x7f82dc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82db7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc1025e0 0x7f82dc197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36236/0 (socket says 172.21.15.5:36236) 2024-09-11T22:22:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82db7fe640 1 -- 172.21.15.5:0/2763867159 learned_addr learned my addr 172.21.15.5:0/2763867159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82dbfff640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82dc107840 0x7f82dc19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82daffd640 1 -- 172.21.15.5:0/2763867159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82dc107840 msgr2=0x7f82dc19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82daffd640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82dc107840 0x7f82dc19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82daffd640 1 -- 172.21.15.5:0/2763867159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc1025e0 msgr2=0x7f82dc197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.411+0000 7f82daffd640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc1025e0 0x7f82dc197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.412+0000 7f82daffd640 1 -- 172.21.15.5:0/2763867159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82dc19faf0 con 0x7f82dc104f10 2024-09-11T22:22:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.412+0000 7f82db7fe640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc1025e0 0x7f82dc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:18.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.412+0000 7f82daffd640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82dc104f10 0x7f82dc198410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82dc069e80 tx=0x7f82d000dbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.412+0000 7f82d8ff9640 1 -- 172.21.15.5:0/2763867159 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0017070 con 0x7f82dc104f10 2024-09-11T22:22:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.412+0000 7f82d8ff9640 1 -- 172.21.15.5:0/2763867159 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d00099d0 con 0x7f82dc104f10 2024-09-11T22:22:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.412+0000 7f82d8ff9640 1 -- 172.21.15.5:0/2763867159 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00047a0 con 0x7f82dc104f10 2024-09-11T22:22:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.412+0000 7f82e1be4640 1 -- 172.21.15.5:0/2763867159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82dc19fde0 con 0x7f82dc104f10 2024-09-11T22:22:18.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.412+0000 7f82e1be4640 1 -- 172.21.15.5:0/2763867159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82dc1a02a0 con 0x7f82dc104f10 2024-09-11T22:22:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.414+0000 7f82e1be4640 1 -- 172.21.15.5:0/2763867159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82dc0fdc30 con 0x7f82dc104f10 2024-09-11T22:22:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.415+0000 7f82d8ff9640 1 -- 172.21.15.5:0/2763867159 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f82d0009b80 con 0x7f82dc104f10 2024-09-11T22:22:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.416+0000 7f82d8ff9640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f82c40a2a10 0x7f82c40a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.417+0000 7f82db7fe640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f82c40a2a10 0x7f82c40a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.417+0000 7f82d8ff9640 1 -- 172.21.15.5:0/2763867159 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f82d00ce920 con 0x7f82dc104f10 2024-09-11T22:22:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.417+0000 7f82db7fe640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f82c40a2a10 0x7f82c40a4ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82cc031440 tx=0x7f82cc0315b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:18.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.421+0000 7f82d8ff9640 1 -- 172.21.15.5:0/2763867159 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82d0020020 con 0x7f82dc104f10 2024-09-11T22:22:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.601+0000 7f82e1be4640 1 -- 172.21.15.5:0/2763867159 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f82dc062df0 con 0x7f82dc104f10 2024-09-11T22:22:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.601+0000 7f82d8ff9640 1 -- 172.21.15.5:0/2763867159 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f82d009ba10 con 0x7f82dc104f10 2024-09-11T22:22:18.603 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-09-11T22:22:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.603+0000 7f82e1be4640 1 -- 172.21.15.5:0/2763867159 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f82c40a2a10 msgr2=0x7f82c40a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.603+0000 7f82e1be4640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f82c40a2a10 0x7f82c40a4ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f82cc031440 tx=0x7f82cc0315b0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.603+0000 7f82e1be4640 1 -- 172.21.15.5:0/2763867159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82dc104f10 msgr2=0x7f82dc198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.603+0000 7f82e1be4640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82dc104f10 0x7f82dc198410 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82dc069e80 tx=0x7f82d000dbb0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.604+0000 7f82e1be4640 1 -- 172.21.15.5:0/2763867159 shutdown_connections 2024-09-11T22:22:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.604+0000 7f82e1be4640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f82c40a2a10 0x7f82c40a4ed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.604+0000 7f82e1be4640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82dc107840 0x7f82dc19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.604+0000 7f82e1be4640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82dc104f10 0x7f82dc198410 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.604+0000 7f82e1be4640 1 --2- 172.21.15.5:0/2763867159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82dc1025e0 0x7f82dc197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.604+0000 7f82e1be4640 1 -- 172.21.15.5:0/2763867159 >> 172.21.15.5:0/2763867159 conn(0x7f82dc0fc140 msgr2=0x7f82dc1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.604+0000 7f82e1be4640 1 -- 172.21.15.5:0/2763867159 shutdown_connections 2024-09-11T22:22:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:18.604+0000 7f82e1be4640 1 -- 172.21.15.5:0/2763867159 wait complete. 2024-09-11T22:22:18.616 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-11T22:22:18.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:22:18.895 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:19.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/65233088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2763867159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: Unable to set osd_memory_target on smithi172 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:19.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:19.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:18 smithi005 ceph-mon[30943]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-09-11T22:22:19.246 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:19.195+0000 7f6dd27eb640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:22:22:19] ENGINE Error in HTTPServer.serve 2024-09-11T22:22:19.246 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: Traceback (most recent call last): 2024-09-11T22:22:19.246 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:22:19.246 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self._connections.run(self.expiration_interval) 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self._run(expiration_interval) 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: s = self.context.wrap_socket( 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: return self.sslsocket_class._create( 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self.do_handshake() 2024-09-11T22:22:19.247 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:22:19.248 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: self._sslobj.do_handshake() 2024-09-11T22:22:19.248 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:22:19.248 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:22:19.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/65233088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2763867159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-11T22:22:19.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: Unable to set osd_memory_target on smithi172 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:19.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:19.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:18 smithi159 ceph-mon[35534]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-09-11T22:22:19.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/65233088' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2763867159' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: Unable to set osd_memory_target on smithi172 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:22:19.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:19.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:19.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:18 smithi172 ceph-mon[35461]: mgrmap e22: x(active, since 3s), standbys: z, y 2024-09-11T22:22:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.508+0000 7f51e881f640 1 -- 172.21.15.5:0/352144859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51e41082a0 msgr2=0x7f51e410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.508+0000 7f51e881f640 1 --2- 172.21.15.5:0/352144859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51e41082a0 0x7f51e410ec40 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009cd0 tx=0x7f51dc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:22:19.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.508+0000 7f51e881f640 1 -- 172.21.15.5:0/352144859 shutdown_connections 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.508+0000 7f51e881f640 1 --2- 172.21.15.5:0/352144859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51e41082a0 0x7f51e410ec40 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.508+0000 7f51e881f640 1 --2- 172.21.15.5:0/352144859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e41036e0 0x7f51e4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.508+0000 7f51e881f640 1 --2- 172.21.15.5:0/352144859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e4102d30 0x7f51e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.508+0000 7f51e881f640 1 -- 172.21.15.5:0/352144859 >> 172.21.15.5:0/352144859 conn(0x7f51e40fc7d0 msgr2=0x7f51e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.509+0000 7f51e881f640 1 -- 172.21.15.5:0/352144859 shutdown_connections 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.509+0000 7f51e881f640 1 -- 172.21.15.5:0/352144859 wait complete. 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.509+0000 7f51e881f640 1 Processor -- start 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e881f640 1 -- start start 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e881f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51e4102d30 0x7f51e41a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e881f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e41036e0 0x7f51e41a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e881f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e41082a0 0x7f51e419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e881f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51e41120a0 con 0x7f51e4102d30 2024-09-11T22:22:19.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e881f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51e4111f20 con 0x7f51e41082a0 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e881f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f51e4112220 con 0x7f51e41036e0 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51e4102d30 0x7f51e41a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51e4102d30 0x7f51e41a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36256/0 (socket says 172.21.15.5:36256) 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e259c640 1 -- 172.21.15.5:0/2943460234 learned_addr learned my addr 172.21.15.5:0/2943460234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e1d9b640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e41036e0 0x7f51e41a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e2d9d640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e41082a0 0x7f51e419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e259c640 1 -- 172.21.15.5:0/2943460234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e41036e0 msgr2=0x7f51e41a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e259c640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e41036e0 0x7f51e41a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e259c640 1 -- 172.21.15.5:0/2943460234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e41082a0 msgr2=0x7f51e419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.510+0000 7f51e259c640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e41082a0 0x7f51e419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.511+0000 7f51e259c640 1 -- 172.21.15.5:0/2943460234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e419acd0 con 0x7f51e4102d30 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.511+0000 7f51e1d9b640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e41036e0 0x7f51e41a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.511+0000 7f51e259c640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51e4102d30 0x7f51e41a03e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f51d400d950 tx=0x7f51d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.511+0000 7f51d37fe640 1 -- 172.21.15.5:0/2943460234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d4017070 con 0x7f51e4102d30 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.511+0000 7f51e881f640 1 -- 172.21.15.5:0/2943460234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e419afc0 con 0x7f51e4102d30 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.511+0000 7f51d37fe640 1 -- 172.21.15.5:0/2943460234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d4009c30 con 0x7f51e4102d30 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.511+0000 7f51d37fe640 1 -- 172.21.15.5:0/2943460234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d400f6b0 con 0x7f51e4102d30 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.511+0000 7f51e881f640 1 -- 172.21.15.5:0/2943460234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e41a72b0 con 0x7f51e4102d30 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.512+0000 7f51e881f640 1 -- 172.21.15.5:0/2943460234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51e4103110 con 0x7f51e4102d30 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.513+0000 7f51d37fe640 1 -- 172.21.15.5:0/2943460234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 138003+0+0 (secure 0 0 0) 0x7f51d40109d0 con 0x7f51e4102d30 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.514+0000 7f51d37fe640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f51cc0a2d80 0x7f51cc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.514+0000 7f51e1d9b640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f51cc0a2d80 0x7f51cc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.514+0000 7f51d37fe640 1 -- 172.21.15.5:0/2943460234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f51d40cf110 con 0x7f51e4102d30 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.515+0000 7f51e1d9b640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f51cc0a2d80 0x7f51cc0a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f51e4101fb0 tx=0x7f51d8005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:19.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.519+0000 7f51d37fe640 1 -- 172.21.15.5:0/2943460234 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51d4009de0 con 0x7f51e4102d30 2024-09-11T22:22:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.735+0000 7f51e881f640 1 -- 172.21.15.5:0/2943460234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51e410c2c0 con 0x7f51e4102d30 2024-09-11T22:22:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.736+0000 7f51d37fe640 1 -- 172.21.15.5:0/2943460234 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f51d409b610 con 0x7f51e4102d30 2024-09-11T22:22:19.738 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:19.738 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-11T22:22:15:230183+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:22:15.180031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:22:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-11T22:22:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 -- 172.21.15.5:0/2943460234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f51cc0a2d80 msgr2=0x7f51cc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f51cc0a2d80 0x7f51cc0a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f51e4101fb0 tx=0x7f51d8005f90 comp rx=0 tx=0).stop 2024-09-11T22:22:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 -- 172.21.15.5:0/2943460234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51e4102d30 msgr2=0x7f51e41a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51e4102d30 0x7f51e41a03e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f51d400d950 tx=0x7f51d400de10 comp rx=0 tx=0).stop 2024-09-11T22:22:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 -- 172.21.15.5:0/2943460234 shutdown_connections 2024-09-11T22:22:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f51cc0a2d80 0x7f51cc0a5240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51e41082a0 0x7f51e419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51e41036e0 0x7f51e41a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 --2- 172.21.15.5:0/2943460234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51e4102d30 0x7f51e41a03e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 -- 172.21.15.5:0/2943460234 >> 172.21.15.5:0/2943460234 conn(0x7f51e40fc7d0 msgr2=0x7f51e4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 -- 172.21.15.5:0/2943460234 shutdown_connections 2024-09-11T22:22:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:19.738+0000 7f51e881f640 1 -- 172.21.15.5:0/2943460234 wait complete. 2024-09-11T22:22:20.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-11T22:22:20.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T22:22:20.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-11T22:22:20.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-11T22:22:20.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T22:22:20.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: [11/Sep/2024:22:22:19] ENGINE Bus STARTING 2024-09-11T22:22:20.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: [11/Sep/2024:22:22:19] ENGINE Serving on https://172.21.15.5:7150 2024-09-11T22:22:20.114 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: [11/Sep/2024:22:22:19] ENGINE Error in HTTPServer.serve 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-11T22:22:20.115 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: [11/Sep/2024:22:22:19] ENGINE Serving on http://172.21.15.5:8765 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: [11/Sep/2024:22:22:19] ENGINE Bus STARTED 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:20.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:20.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:20.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:22:20.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:22:20.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2943460234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:20.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:22:20.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:19 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-11T22:22:20.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T22:22:20.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-11T22:22:20.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-11T22:22:20.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: [11/Sep/2024:22:22:19] ENGINE Bus STARTING 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: [11/Sep/2024:22:22:19] ENGINE Serving on https://172.21.15.5:7150 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: [11/Sep/2024:22:22:19] ENGINE Error in HTTPServer.serve 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: Traceback (most recent call last): 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: self._run(expiration_interval) 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: s = self.context.wrap_socket( 2024-09-11T22:22:20.330 INFO:journalctl@ceph.mon.b.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout: return self.sslsocket_class._create( 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout: self.do_handshake() 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout: self._sslobj.do_handshake() 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: [11/Sep/2024:22:22:19] ENGINE Serving on http://172.21.15.5:8765 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: [11/Sep/2024:22:22:19] ENGINE Bus STARTED 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:20.331 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:22:20.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:22:20.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2943460234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:20.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:22:20.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.332 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:19 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-11T22:22:20.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T22:22:20.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-11T22:22:20.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-11T22:22:20.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi172:/etc/ceph/ceph.conf 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: [11/Sep/2024:22:22:19] ENGINE Bus STARTING 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: [11/Sep/2024:22:22:19] ENGINE Serving on https://172.21.15.5:7150 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: [11/Sep/2024:22:22:19] ENGINE Error in HTTPServer.serve 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout: Traceback (most recent call last): 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout: self._connections.run(self.expiration_interval) 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout: self._run(expiration_interval) 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T22:22:20.361 INFO:journalctl@ceph.mon.c.smithi172.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout: s = self.context.wrap_socket( 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout: return self.sslsocket_class._create( 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout: self.do_handshake() 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout: self._sslobj.do_handshake() 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: [11/Sep/2024:22:22:19] ENGINE Serving on http://172.21.15.5:8765 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: [11/Sep/2024:22:22:19] ENGINE Bus STARTED 2024-09-11T22:22:20.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:20.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi172:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:20.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-11T22:22:20.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:22:20.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:22:20.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2943460234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:20.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: Updating smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T22:22:20.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:19 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:20.598 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-11T22:22:20.598 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-11T22:22:20.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T22:22:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.703+0000 7fddabe27640 1 -- 172.21.15.5:0/2186070825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdda4103610 msgr2=0x7fdda410baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.703+0000 7fddabe27640 1 --2- 172.21.15.5:0/2186070825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdda4103610 0x7fdda410baf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdd98009900 tx=0x7fdd9802a480 comp rx=0 tx=0).stop 2024-09-11T22:22:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.703+0000 7fddabe27640 1 -- 172.21.15.5:0/2186070825 shutdown_connections 2024-09-11T22:22:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.703+0000 7fddabe27640 1 --2- 172.21.15.5:0/2186070825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdda410c0c0 0x7fdda410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.703+0000 7fddabe27640 1 --2- 172.21.15.5:0/2186070825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdda4103610 0x7fdda410baf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.703+0000 7fddabe27640 1 --2- 172.21.15.5:0/2186070825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdda4102cf0 0x7fdda41030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.703+0000 7fddabe27640 1 -- 172.21.15.5:0/2186070825 >> 172.21.15.5:0/2186070825 conn(0x7fdda4077a10 msgr2=0x7fdda40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.703+0000 7fddabe27640 1 -- 172.21.15.5:0/2186070825 shutdown_connections 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.703+0000 7fddabe27640 1 -- 172.21.15.5:0/2186070825 wait complete. 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fddabe27640 1 Processor -- start 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fddabe27640 1 -- start start 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fddabe27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdda4102cf0 0x7fdda41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fddabe27640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdda4103610 0x7fdda4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fddabe27640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdda410c0c0 0x7fdda419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fddabe27640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4111ee0 con 0x7fdda4102cf0 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fddabe27640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4111d60 con 0x7fdda4103610 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fddabe27640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4112060 con 0x7fdda410c0c0 2024-09-11T22:22:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fdda9bc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdda4102cf0 0x7fdda41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fdda9bc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdda4102cf0 0x7fdda41983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36272/0 (socket says 172.21.15.5:36272) 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fdda9bc3640 1 -- 172.21.15.5:0/1340652397 learned_addr learned my addr 172.21.15.5:0/1340652397 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fdda93c2640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdda4103610 0x7fdda4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fddaa3c4640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdda410c0c0 0x7fdda419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fdda9bc3640 1 -- 172.21.15.5:0/1340652397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdda410c0c0 msgr2=0x7fdda419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fdda9bc3640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdda410c0c0 0x7fdda419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fdda9bc3640 1 -- 172.21.15.5:0/1340652397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdda4103610 msgr2=0x7fdda4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fdda9bc3640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdda4103610 0x7fdda4198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fdda9bc3640 1 -- 172.21.15.5:0/1340652397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda419d3c0 con 0x7fdda4102cf0 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.704+0000 7fdda9bc3640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdda4102cf0 0x7fdda41983d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00e9f0 tx=0x7fdd9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.705+0000 7fdda2ffd640 1 -- 172.21.15.5:0/1340652397 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00cd80 con 0x7fdda4102cf0 2024-09-11T22:22:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.705+0000 7fdda2ffd640 1 -- 172.21.15.5:0/1340652397 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd9c010910 con 0x7fdda4102cf0 2024-09-11T22:22:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.705+0000 7fddabe27640 1 -- 172.21.15.5:0/1340652397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda419d6b0 con 0x7fdda4102cf0 2024-09-11T22:22:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.705+0000 7fdda2ffd640 1 -- 172.21.15.5:0/1340652397 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c018990 con 0x7fdda4102cf0 2024-09-11T22:22:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.705+0000 7fddabe27640 1 -- 172.21.15.5:0/1340652397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda41a4f40 con 0x7fdda4102cf0 2024-09-11T22:22:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.706+0000 7fddabe27640 1 -- 172.21.15.5:0/1340652397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdda41030d0 con 0x7fdda4102cf0 2024-09-11T22:22:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.708+0000 7fdda2ffd640 1 -- 172.21.15.5:0/1340652397 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 138003+0+0 (secure 0 0 0) 0x7fdd9c010ac0 con 0x7fdda4102cf0 2024-09-11T22:22:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.708+0000 7fdda2ffd640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd8c0a2d90 0x7fdd8c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.708+0000 7fdda2ffd640 1 -- 172.21.15.5:0/1340652397 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fdd9c0cfd80 con 0x7fdda4102cf0 2024-09-11T22:22:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.708+0000 7fdda93c2640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd8c0a2d90 0x7fdd8c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.709+0000 7fdda93c2640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd8c0a2d90 0x7fdd8c0a5250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdd98000c00 tx=0x7fdd98005b50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:20.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.712+0000 7fdda2ffd640 1 -- 172.21.15.5:0/1340652397 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd9c09c830 con 0x7fdda4102cf0 2024-09-11T22:22:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.883+0000 7fddabe27640 1 -- 172.21.15.5:0/1340652397 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdda4062630 con 0x7fdd8c0a2d90 2024-09-11T22:22:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.890+0000 7fdda2ffd640 1 -- 172.21.15.5:0/1340652397 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258088 (secure 0 0 0) 0x7fdda4062630 con 0x7fdd8c0a2d90 2024-09-11T22:22:20.896 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 -- 172.21.15.5:0/1340652397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd8c0a2d90 msgr2=0x7fdd8c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd8c0a2d90 0x7fdd8c0a5250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdd98000c00 tx=0x7fdd98005b50 comp rx=0 tx=0).stop 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 -- 172.21.15.5:0/1340652397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdda4102cf0 msgr2=0x7fdda41983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdda4102cf0 0x7fdda41983d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00e9f0 tx=0x7fdd9c00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 -- 172.21.15.5:0/1340652397 shutdown_connections 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdda410c0c0 0x7fdda419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd8c0a2d90 0x7fdd8c0a5250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdda4103610 0x7fdda4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 --2- 172.21.15.5:0/1340652397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdda4102cf0 0x7fdda41983d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 -- 172.21.15.5:0/1340652397 >> 172.21.15.5:0/1340652397 conn(0x7fdda4077a10 msgr2=0x7fdda40fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 -- 172.21.15.5:0/1340652397 shutdown_connections 2024-09-11T22:22:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:20.893+0000 7fddabe27640 1 -- 172.21.15.5:0/1340652397 wait complete. 2024-09-11T22:22:20.906 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-11T22:22:19.987091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190734,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1294,"num_read_kb":836,"num_write":555,"num_write_kb":4403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181928,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385548,"kb_used_data":11292,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556276,"statfs":{"total":1151940427776,"available":1151545626624,"internally_reserved":0,"allocated":11563008,"data_stored":7937284,"data_compressed":241535,"data_compressed_allocated":319488,"data_compressed_original":2695168,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987365"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235071+0000","last_change":"2024-09-11T22:20:18.189744+0000","last_active":"2024-09-11T22:22:16.235071+0000","last_peered":"2024-09-11T22:22:16.235071+0000","last_clean":"2024-09-11T22:22:16.235071+0000","last_became_active":"2024-09-11T22:20:18.189398+0000","last_became_peered":"2024-09-11T22:20:18.189398+0000","last_unstale":"2024-09-11T22:22:16.235071+0000","last_undegraded":"2024-09-11T22:22:16.235071+0000","last_fullsized":"2024-09-11T22:22:16.235071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:24.152589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234137+0000","last_change":"2024-09-11T22:20:18.186727+0000","last_active":"2024-09-11T22:22:16.234137+0000","last_peered":"2024-09-11T22:22:16.234137+0000","last_clean":"2024-09-11T22:22:16.234137+0000","last_became_active":"2024-09-11T22:20:18.186414+0000","last_became_peered":"2024-09-11T22:20:18.186414+0000","last_unstale":"2024-09-11T22:22:16.234137+0000","last_undegraded":"2024-09-11T22:22:16.234137+0000","last_fullsized":"2024-09-11T22:22:16.234137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:53.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233845+0000","last_change":"2024-09-11T22:20:18.179984+0000","last_active":"2024-09-11T22:22:16.233845+0000","last_peered":"2024-09-11T22:22:16.233845+0000","last_clean":"2024-09-11T22:22:16.233845+0000","last_became_active":"2024-09-11T22:20:18.179329+0000","last_became_peered":"2024-09-11T22:20:18.179329+0000","last_unstale":"2024-09-11T22:22:16.233845+0000","last_undegraded":"2024-09-11T22:22:16.233845+0000","last_fullsized":"2024-09-11T22:22:16.233845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:19.149970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230208+0000","last_change":"2024-09-11T22:20:18.191768+0000","last_active":"2024-09-11T22:22:16.230208+0000","last_peered":"2024-09-11T22:22:16.230208+0000","last_clean":"2024-09-11T22:22:16.230208+0000","last_became_active":"2024-09-11T22:20:18.191336+0000","last_became_peered":"2024-09-11T22:20:18.191336+0000","last_unstale":"2024-09-11T22:22:16.230208+0000","last_undegraded":"2024-09-11T22:22:16.230208+0000","last_fullsized":"2024-09-11T22:22:16.230208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:44.768932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058858+0000","last_change":"2024-09-11T22:20:18.181469+0000","last_active":"2024-09-11T22:22:16.058858+0000","last_peered":"2024-09-11T22:22:16.058858+0000","last_clean":"2024-09-11T22:22:16.058858+0000","last_became_active":"2024-09-11T22:20:18.181100+0000","last_became_peered":"2024-09-11T22:20:18.181100+0000","last_unstale":"2024-09-11T22:22:16.058858+0000","last_undegraded":"2024-09-11T22:22:16.058858+0000","last_fullsized":"2024-09-11T22:22:16.058858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:14.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068675+0000","last_change":"2024-09-11T22:20:18.190855+0000","last_active":"2024-09-11T22:22:16.068675+0000","last_peered":"2024-09-11T22:22:16.068675+0000","last_clean":"2024-09-11T22:22:16.068675+0000","last_became_active":"2024-09-11T22:20:18.190523+0000","last_became_peered":"2024-09-11T22:20:18.190523+0000","last_unstale":"2024-09-11T22:22:16.068675+0000","last_undegraded":"2024-09-11T22:22:16.068675+0000","last_fullsized":"2024-09-11T22:22:16.068675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:18.574957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231733+0000","last_change":"2024-09-11T22:20:18.199213+0000","last_active":"2024-09-11T22:22:16.231733+0000","last_peered":"2024-09-11T22:22:16.231733+0000","last_clean":"2024-09-11T22:22:16.231733+0000","last_became_active":"2024-09-11T22:20:18.198913+0000","last_became_peered":"2024-09-11T22:20:18.198913+0000","last_unstale":"2024-09-11T22:22:16.231733+0000","last_undegraded":"2024-09-11T22:22:16.231733+0000","last_fullsized":"2024-09-11T22:22:16.231733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:22.682802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058633+0000","last_change":"2024-09-11T22:20:18.192458+0000","last_active":"2024-09-11T22:22:16.058633+0000","last_peered":"2024-09-11T22:22:16.058633+0000","last_clean":"2024-09-11T22:22:16.058633+0000","last_became_active":"2024-09-11T22:20:18.191967+0000","last_became_peered":"2024-09-11T22:20:18.191967+0000","last_unstale":"2024-09-11T22:22:16.058633+0000","last_undegraded":"2024-09-11T22:22:16.058633+0000","last_fullsized":"2024-09-11T22:22:16.058633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:52.686157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975713+0000","last_change":"2024-09-11T22:21:18.191732+0000","last_active":"2024-09-11T22:22:15.975713+0000","last_peered":"2024-09-11T22:22:15.975713+0000","last_clean":"2024-09-11T22:22:15.975713+0000","last_became_active":"2024-09-11T22:21:18.191375+0000","last_became_peered":"2024-09-11T22:21:18.191375+0000","last_unstale":"2024-09-11T22:22:15.975713+0000","last_undegraded":"2024-09-11T22:22:15.975713+0000","last_fullsized":"2024-09-11T22:22:15.975713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:13.645224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.069211+0000","last_change":"2024-09-11T22:20:18.193771+0000","last_active":"2024-09-11T22:22:16.069211+0000","last_peered":"2024-09-11T22:22:16.069211+0000","last_clean":"2024-09-11T22:22:16.069211+0000","last_became_active":"2024-09-11T22:20:18.193041+0000","last_became_peered":"2024-09-11T22:20:18.193041+0000","last_unstale":"2024-09-11T22:22:16.069211+0000","last_undegraded":"2024-09-11T22:22:16.069211+0000","last_fullsized":"2024-09-11T22:22:16.069211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:06.656169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231998+0000","last_change":"2024-09-11T22:20:18.197933+0000","last_active":"2024-09-11T22:22:16.231998+0000","last_peered":"2024-09-11T22:22:16.231998+0000","last_clean":"2024-09-11T22:22:16.231998+0000","last_became_active":"2024-09-11T22:20:18.197210+0000","last_became_peered":"2024-09-11T22:20:18.197210+0000","last_unstale":"2024-09-11T22:22:16.231998+0000","last_undegraded":"2024-09-11T22:22:16.231998+0000","last_fullsized":"2024-09-11T22:22:16.231998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:42.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230250+0000","last_change":"2024-09-11T22:20:18.190739+0000","last_active":"2024-09-11T22:22:16.230250+0000","last_peered":"2024-09-11T22:22:16.230250+0000","last_clean":"2024-09-11T22:22:16.230250+0000","last_became_active":"2024-09-11T22:20:18.190122+0000","last_became_peered":"2024-09-11T22:20:18.190122+0000","last_unstale":"2024-09-11T22:22:16.230250+0000","last_undegraded":"2024-09-11T22:22:16.230250+0000","last_fullsized":"2024-09-11T22:22:16.230250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.585035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230697+0000","last_change":"2024-09-11T22:20:18.190303+0000","last_active":"2024-09-11T22:22:16.230697+0000","last_peered":"2024-09-11T22:22:16.230697+0000","last_clean":"2024-09-11T22:22:16.230697+0000","last_became_active":"2024-09-11T22:20:18.189962+0000","last_became_peered":"2024-09-11T22:20:18.189962+0000","last_unstale":"2024-09-11T22:22:16.230697+0000","last_undegraded":"2024-09-11T22:22:16.230697+0000","last_fullsized":"2024-09-11T22:22:16.230697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:44.872813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233520+0000","last_change":"2024-09-11T22:20:18.185893+0000","last_active":"2024-09-11T22:22:16.233520+0000","last_peered":"2024-09-11T22:22:16.233520+0000","last_clean":"2024-09-11T22:22:16.233520+0000","last_became_active":"2024-09-11T22:20:18.185592+0000","last_became_peered":"2024-09-11T22:20:18.185592+0000","last_unstale":"2024-09-11T22:22:16.233520+0000","last_undegraded":"2024-09-11T22:22:16.233520+0000","last_fullsized":"2024-09-11T22:22:16.233520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:14.725301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233777+0000","last_change":"2024-09-11T22:20:18.193943+0000","last_active":"2024-09-11T22:22:16.233777+0000","last_peered":"2024-09-11T22:22:16.233777+0000","last_clean":"2024-09-11T22:22:16.233777+0000","last_became_active":"2024-09-11T22:20:18.193558+0000","last_became_peered":"2024-09-11T22:20:18.193558+0000","last_unstale":"2024-09-11T22:22:16.233777+0000","last_undegraded":"2024-09-11T22:22:16.233777+0000","last_fullsized":"2024-09-11T22:22:16.233777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:08.736161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975254+0000","last_change":"2024-09-11T22:20:18.191815+0000","last_active":"2024-09-11T22:22:15.975254+0000","last_peered":"2024-09-11T22:22:15.975254+0000","last_clean":"2024-09-11T22:22:15.975254+0000","last_became_active":"2024-09-11T22:20:18.191483+0000","last_became_peered":"2024-09-11T22:20:18.191483+0000","last_unstale":"2024-09-11T22:22:15.975254+0000","last_undegraded":"2024-09-11T22:22:15.975254+0000","last_fullsized":"2024-09-11T22:22:15.975254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:26.432117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235101+0000","last_change":"2024-09-11T22:20:18.190758+0000","last_active":"2024-09-11T22:22:16.235101+0000","last_peered":"2024-09-11T22:22:16.235101+0000","last_clean":"2024-09-11T22:22:16.235101+0000","last_became_active":"2024-09-11T22:20:18.190431+0000","last_became_peered":"2024-09-11T22:20:18.190431+0000","last_unstale":"2024-09-11T22:22:16.235101+0000","last_undegraded":"2024-09-11T22:22:16.235101+0000","last_fullsized":"2024-09-11T22:22:16.235101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:10.927051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973552+0000","last_change":"2024-09-11T22:20:18.185191+0000","last_active":"2024-09-11T22:22:15.973552+0000","last_peered":"2024-09-11T22:22:15.973552+0000","last_clean":"2024-09-11T22:22:15.973552+0000","last_became_active":"2024-09-11T22:20:18.184906+0000","last_became_peered":"2024-09-11T22:20:18.184906+0000","last_unstale":"2024-09-11T22:22:15.973552+0000","last_undegraded":"2024-09-11T22:22:15.973552+0000","last_fullsized":"2024-09-11T22:22:15.973552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:32.294780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977515+0000","last_change":"2024-09-11T22:20:18.187137+0000","last_active":"2024-09-11T22:22:15.977515+0000","last_peered":"2024-09-11T22:22:15.977515+0000","last_clean":"2024-09-11T22:22:15.977515+0000","last_became_active":"2024-09-11T22:20:18.186770+0000","last_became_peered":"2024-09-11T22:20:18.186770+0000","last_unstale":"2024-09-11T22:22:15.977515+0000","last_undegraded":"2024-09-11T22:22:15.977515+0000","last_fullsized":"2024-09-11T22:22:15.977515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:59.917695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234743+0000","last_change":"2024-09-11T22:20:18.197137+0000","last_active":"2024-09-11T22:22:16.234743+0000","last_peered":"2024-09-11T22:22:16.234743+0000","last_clean":"2024-09-11T22:22:16.234743+0000","last_became_active":"2024-09-11T22:20:18.192810+0000","last_became_peered":"2024-09-11T22:20:18.192810+0000","last_unstale":"2024-09-11T22:22:16.234743+0000","last_undegraded":"2024-09-11T22:22:16.234743+0000","last_fullsized":"2024-09-11T22:22:16.234743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:57.680726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230830+0000","last_change":"2024-09-11T22:20:18.187568+0000","last_active":"2024-09-11T22:22:16.230830+0000","last_peered":"2024-09-11T22:22:16.230830+0000","last_clean":"2024-09-11T22:22:16.230830+0000","last_became_active":"2024-09-11T22:20:18.187254+0000","last_became_peered":"2024-09-11T22:20:18.187254+0000","last_unstale":"2024-09-11T22:22:16.230830+0000","last_undegraded":"2024-09-11T22:22:16.230830+0000","last_fullsized":"2024-09-11T22:22:16.230830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:41.724926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975378+0000","last_change":"2024-09-11T22:20:18.190376+0000","last_active":"2024-09-11T22:22:15.975378+0000","last_peered":"2024-09-11T22:22:15.975378+0000","last_clean":"2024-09-11T22:22:15.975378+0000","last_became_active":"2024-09-11T22:20:18.190079+0000","last_became_peered":"2024-09-11T22:20:18.190079+0000","last_unstale":"2024-09-11T22:22:15.975378+0000","last_undegraded":"2024-09-11T22:22:15.975378+0000","last_fullsized":"2024-09-11T22:22:15.975378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:42.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058896+0000","last_change":"2024-09-11T22:20:18.188503+0000","last_active":"2024-09-11T22:22:16.058896+0000","last_peered":"2024-09-11T22:22:16.058896+0000","last_clean":"2024-09-11T22:22:16.058896+0000","last_became_active":"2024-09-11T22:20:18.187957+0000","last_became_peered":"2024-09-11T22:20:18.187957+0000","last_unstale":"2024-09-11T22:22:16.058896+0000","last_undegraded":"2024-09-11T22:22:16.058896+0000","last_fullsized":"2024-09-11T22:22:16.058896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:44:25.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065572+0000","last_change":"2024-09-11T22:20:18.196936+0000","last_active":"2024-09-11T22:22:16.065572+0000","last_peered":"2024-09-11T22:22:16.065572+0000","last_clean":"2024-09-11T22:22:16.065572+0000","last_became_active":"2024-09-11T22:20:18.196493+0000","last_became_peered":"2024-09-11T22:20:18.196493+0000","last_unstale":"2024-09-11T22:22:16.065572+0000","last_undegraded":"2024-09-11T22:22:16.065572+0000","last_fullsized":"2024-09-11T22:22:16.065572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:06.189806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059126+0000","last_change":"2024-09-11T22:20:18.195260+0000","last_active":"2024-09-11T22:22:16.059126+0000","last_peered":"2024-09-11T22:22:16.059126+0000","last_clean":"2024-09-11T22:22:16.059126+0000","last_became_active":"2024-09-11T22:20:18.194956+0000","last_became_peered":"2024-09-11T22:20:18.194956+0000","last_unstale":"2024-09-11T22:22:16.059126+0000","last_undegraded":"2024-09-11T22:22:16.059126+0000","last_fullsized":"2024-09-11T22:22:16.059126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:34.483291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977695+0000","last_change":"2024-09-11T22:20:18.190591+0000","last_active":"2024-09-11T22:22:15.977695+0000","last_peered":"2024-09-11T22:22:15.977695+0000","last_clean":"2024-09-11T22:22:15.977695+0000","last_became_active":"2024-09-11T22:20:18.190007+0000","last_became_peered":"2024-09-11T22:20:18.190007+0000","last_unstale":"2024-09-11T22:22:15.977695+0000","last_undegraded":"2024-09-11T22:22:15.977695+0000","last_fullsized":"2024-09-11T22:22:15.977695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:46.281705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234929+0000","last_change":"2024-09-11T22:20:18.193377+0000","last_active":"2024-09-11T22:22:16.234929+0000","last_peered":"2024-09-11T22:22:16.234929+0000","last_clean":"2024-09-11T22:22:16.234929+0000","last_became_active":"2024-09-11T22:20:18.193003+0000","last_became_peered":"2024-09-11T22:20:18.193003+0000","last_unstale":"2024-09-11T22:22:16.234929+0000","last_undegraded":"2024-09-11T22:22:16.234929+0000","last_fullsized":"2024-09-11T22:22:16.234929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:59.052966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230737+0000","last_change":"2024-09-11T22:20:18.189405+0000","last_active":"2024-09-11T22:22:16.230737+0000","last_peered":"2024-09-11T22:22:16.230737+0000","last_clean":"2024-09-11T22:22:16.230737+0000","last_became_active":"2024-09-11T22:20:18.189098+0000","last_became_peered":"2024-09-11T22:20:18.189098+0000","last_unstale":"2024-09-11T22:22:16.230737+0000","last_undegraded":"2024-09-11T22:22:16.230737+0000","last_fullsized":"2024-09-11T22:22:16.230737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:47.088192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233832+0000","last_change":"2024-09-11T22:20:18.191005+0000","last_active":"2024-09-11T22:22:16.233832+0000","last_peered":"2024-09-11T22:22:16.233832+0000","last_clean":"2024-09-11T22:22:16.233832+0000","last_became_active":"2024-09-11T22:20:18.190681+0000","last_became_peered":"2024-09-11T22:20:18.190681+0000","last_unstale":"2024-09-11T22:22:16.233832+0000","last_undegraded":"2024-09-11T22:22:16.233832+0000","last_fullsized":"2024-09-11T22:22:16.233832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:12.903929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977210+0000","last_change":"2024-09-11T22:20:18.187920+0000","last_active":"2024-09-11T22:22:15.977210+0000","last_peered":"2024-09-11T22:22:15.977210+0000","last_clean":"2024-09-11T22:22:15.977210+0000","last_became_active":"2024-09-11T22:20:18.187526+0000","last_became_peered":"2024-09-11T22:20:18.187526+0000","last_unstale":"2024-09-11T22:22:15.977210+0000","last_undegraded":"2024-09-11T22:22:15.977210+0000","last_fullsized":"2024-09-11T22:22:15.977210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.347857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977399+0000","last_change":"2024-09-11T22:20:18.193869+0000","last_active":"2024-09-11T22:22:15.977399+0000","last_peered":"2024-09-11T22:22:15.977399+0000","last_clean":"2024-09-11T22:22:15.977399+0000","last_became_active":"2024-09-11T22:20:18.193468+0000","last_became_peered":"2024-09-11T22:20:18.193468+0000","last_unstale":"2024-09-11T22:22:15.977399+0000","last_undegraded":"2024-09-11T22:22:15.977399+0000","last_fullsized":"2024-09-11T22:22:15.977399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:59.619611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234817+0000","last_change":"2024-09-11T22:20:18.193947+0000","last_active":"2024-09-11T22:22:16.234817+0000","last_peered":"2024-09-11T22:22:16.234817+0000","last_clean":"2024-09-11T22:22:16.234817+0000","last_became_active":"2024-09-11T22:20:18.192257+0000","last_became_peered":"2024-09-11T22:20:18.192257+0000","last_unstale":"2024-09-11T22:22:16.234817+0000","last_undegraded":"2024-09-11T22:22:16.234817+0000","last_fullsized":"2024-09-11T22:22:16.234817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.478730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058805+0000","last_change":"2024-09-11T22:20:18.184549+0000","last_active":"2024-09-11T22:22:16.058805+0000","last_peered":"2024-09-11T22:22:16.058805+0000","last_clean":"2024-09-11T22:22:16.058805+0000","last_became_active":"2024-09-11T22:20:18.184075+0000","last_became_peered":"2024-09-11T22:20:18.184075+0000","last_unstale":"2024-09-11T22:22:16.058805+0000","last_undegraded":"2024-09-11T22:22:16.058805+0000","last_fullsized":"2024-09-11T22:22:16.058805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:50.095643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.415258+0000","last_change":"2024-09-11T22:20:16.128524+0000","last_active":"2024-09-11T22:22:16.415258+0000","last_peered":"2024-09-11T22:22:16.415258+0000","last_clean":"2024-09-11T22:22:16.415258+0000","last_became_active":"2024-09-11T22:20:16.128185+0000","last_became_peered":"2024-09-11T22:20:16.128185+0000","last_unstale":"2024-09-11T22:22:16.415258+0000","last_undegraded":"2024-09-11T22:22:16.415258+0000","last_fullsized":"2024-09-11T22:22:16.415258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.022931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065720+0000","last_change":"2024-09-11T22:20:18.191578+0000","last_active":"2024-09-11T22:22:16.065720+0000","last_peered":"2024-09-11T22:22:16.065720+0000","last_clean":"2024-09-11T22:22:16.065720+0000","last_became_active":"2024-09-11T22:20:18.189568+0000","last_became_peered":"2024-09-11T22:20:18.189568+0000","last_unstale":"2024-09-11T22:22:16.065720+0000","last_undegraded":"2024-09-11T22:22:16.065720+0000","last_fullsized":"2024-09-11T22:22:16.065720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:41.737242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068742+0000","last_change":"2024-09-11T22:21:18.199693+0000","last_active":"2024-09-11T22:22:16.068742+0000","last_peered":"2024-09-11T22:22:16.068742+0000","last_clean":"2024-09-11T22:22:16.068742+0000","last_became_active":"2024-09-11T22:21:18.190954+0000","last_became_peered":"2024-09-11T22:21:18.190954+0000","last_unstale":"2024-09-11T22:22:16.068742+0000","last_undegraded":"2024-09-11T22:22:16.068742+0000","last_fullsized":"2024-09-11T22:22:16.068742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:11.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230688+0000","last_change":"2024-09-11T22:20:16.115602+0000","last_active":"2024-09-11T22:22:16.230688+0000","last_peered":"2024-09-11T22:22:16.230688+0000","last_clean":"2024-09-11T22:22:16.230688+0000","last_became_active":"2024-09-11T22:20:16.115280+0000","last_became_peered":"2024-09-11T22:20:16.115280+0000","last_unstale":"2024-09-11T22:22:16.230688+0000","last_undegraded":"2024-09-11T22:22:16.230688+0000","last_fullsized":"2024-09-11T22:22:16.230688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:41.942137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058816+0000","last_change":"2024-09-11T22:20:18.184125+0000","last_active":"2024-09-11T22:22:16.058816+0000","last_peered":"2024-09-11T22:22:16.058816+0000","last_clean":"2024-09-11T22:22:16.058816+0000","last_became_active":"2024-09-11T22:20:18.183785+0000","last_became_peered":"2024-09-11T22:20:18.183785+0000","last_unstale":"2024-09-11T22:22:16.058816+0000","last_undegraded":"2024-09-11T22:22:16.058816+0000","last_fullsized":"2024-09-11T22:22:16.058816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:33.230175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065780+0000","last_change":"2024-09-11T22:20:16.125852+0000","last_active":"2024-09-11T22:22:16.065780+0000","last_peered":"2024-09-11T22:22:16.065780+0000","last_clean":"2024-09-11T22:22:16.065780+0000","last_became_active":"2024-09-11T22:20:16.125250+0000","last_became_peered":"2024-09-11T22:20:16.125250+0000","last_unstale":"2024-09-11T22:22:16.065780+0000","last_undegraded":"2024-09-11T22:22:16.065780+0000","last_fullsized":"2024-09-11T22:22:16.065780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:34.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973235+0000","last_change":"2024-09-11T22:20:18.185147+0000","last_active":"2024-09-11T22:22:15.973235+0000","last_peered":"2024-09-11T22:22:15.973235+0000","last_clean":"2024-09-11T22:22:15.973235+0000","last_became_active":"2024-09-11T22:20:18.184840+0000","last_became_peered":"2024-09-11T22:20:18.184840+0000","last_unstale":"2024-09-11T22:22:15.973235+0000","last_undegraded":"2024-09-11T22:22:15.973235+0000","last_fullsized":"2024-09-11T22:22:15.973235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:59.703349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":243,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.277347+0000","last_change":"2024-09-11T22:20:16.122173+0000","last_active":"2024-09-11T22:22:16.277347+0000","last_peered":"2024-09-11T22:22:16.277347+0000","last_clean":"2024-09-11T22:22:16.277347+0000","last_became_active":"2024-09-11T22:20:16.121847+0000","last_became_peered":"2024-09-11T22:20:16.121847+0000","last_unstale":"2024-09-11T22:22:16.277347+0000","last_undegraded":"2024-09-11T22:22:16.277347+0000","last_fullsized":"2024-09-11T22:22:16.277347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:41.284557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":213,"num_read_kb":213,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975691+0000","last_change":"2024-09-11T22:20:18.188030+0000","last_active":"2024-09-11T22:22:15.975691+0000","last_peered":"2024-09-11T22:22:15.975691+0000","last_clean":"2024-09-11T22:22:15.975691+0000","last_became_active":"2024-09-11T22:20:18.187503+0000","last_became_peered":"2024-09-11T22:20:18.187503+0000","last_unstale":"2024-09-11T22:22:15.975691+0000","last_undegraded":"2024-09-11T22:22:15.975691+0000","last_fullsized":"2024-09-11T22:22:15.975691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:45.494149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231598+0000","last_change":"2024-09-11T22:20:16.128828+0000","last_active":"2024-09-11T22:22:16.231598+0000","last_peered":"2024-09-11T22:22:16.231598+0000","last_clean":"2024-09-11T22:22:16.231598+0000","last_became_active":"2024-09-11T22:20:16.128454+0000","last_became_peered":"2024-09-11T22:20:16.128454+0000","last_unstale":"2024-09-11T22:22:16.231598+0000","last_undegraded":"2024-09-11T22:22:16.231598+0000","last_fullsized":"2024-09-11T22:22:16.231598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:33.927564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234475+0000","last_change":"2024-09-11T22:20:18.193795+0000","last_active":"2024-09-11T22:22:16.234475+0000","last_peered":"2024-09-11T22:22:16.234475+0000","last_clean":"2024-09-11T22:22:16.234475+0000","last_became_active":"2024-09-11T22:20:18.193489+0000","last_became_peered":"2024-09-11T22:20:18.193489+0000","last_unstale":"2024-09-11T22:22:16.234475+0000","last_undegraded":"2024-09-11T22:22:16.234475+0000","last_fullsized":"2024-09-11T22:22:16.234475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:14.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065914+0000","last_change":"2024-09-11T22:20:16.187938+0000","last_active":"2024-09-11T22:22:16.065914+0000","last_peered":"2024-09-11T22:22:16.065914+0000","last_clean":"2024-09-11T22:22:16.065914+0000","last_became_active":"2024-09-11T22:20:16.187545+0000","last_became_peered":"2024-09-11T22:20:16.187545+0000","last_unstale":"2024-09-11T22:22:16.065914+0000","last_undegraded":"2024-09-11T22:22:16.065914+0000","last_fullsized":"2024-09-11T22:22:16.065914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:41.674356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975385+0000","last_change":"2024-09-11T22:21:18.191937+0000","last_active":"2024-09-11T22:22:15.975385+0000","last_peered":"2024-09-11T22:22:15.975385+0000","last_clean":"2024-09-11T22:22:15.975385+0000","last_became_active":"2024-09-11T22:21:18.191218+0000","last_became_peered":"2024-09-11T22:21:18.191218+0000","last_unstale":"2024-09-11T22:22:15.975385+0000","last_undegraded":"2024-09-11T22:22:15.975385+0000","last_fullsized":"2024-09-11T22:22:15.975385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:41.654630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'37","reported_seq":287,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.414971+0000","last_change":"2024-09-11T22:20:16.128082+0000","last_active":"2024-09-11T22:22:16.414971+0000","last_peered":"2024-09-11T22:22:16.414971+0000","last_clean":"2024-09-11T22:22:16.414971+0000","last_became_active":"2024-09-11T22:20:16.127731+0000","last_became_peered":"2024-09-11T22:20:16.127731+0000","last_unstale":"2024-09-11T22:22:16.414971+0000","last_undegraded":"2024-09-11T22:22:16.414971+0000","last_fullsized":"2024-09-11T22:22:16.414971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:33.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66480,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":226,"num_read_kb":178,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059064+0000","last_change":"2024-09-11T22:20:18.189219+0000","last_active":"2024-09-11T22:22:16.059064+0000","last_peered":"2024-09-11T22:22:16.059064+0000","last_clean":"2024-09-11T22:22:16.059064+0000","last_became_active":"2024-09-11T22:20:18.188687+0000","last_became_peered":"2024-09-11T22:20:18.188687+0000","last_unstale":"2024-09-11T22:22:16.059064+0000","last_undegraded":"2024-09-11T22:22:16.059064+0000","last_fullsized":"2024-09-11T22:22:16.059064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:24.171126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977378+0000","last_change":"2024-09-11T22:20:16.129724+0000","last_active":"2024-09-11T22:22:15.977378+0000","last_peered":"2024-09-11T22:22:15.977378+0000","last_clean":"2024-09-11T22:22:15.977378+0000","last_became_active":"2024-09-11T22:20:16.129351+0000","last_became_peered":"2024-09-11T22:20:16.129351+0000","last_unstale":"2024-09-11T22:22:15.977378+0000","last_undegraded":"2024-09-11T22:22:15.977378+0000","last_fullsized":"2024-09-11T22:22:15.977378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:09.642520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230435+0000","last_change":"2024-09-11T22:20:18.187031+0000","last_active":"2024-09-11T22:22:16.230435+0000","last_peered":"2024-09-11T22:22:16.230435+0000","last_clean":"2024-09-11T22:22:16.230435+0000","last_became_active":"2024-09-11T22:20:18.186698+0000","last_became_peered":"2024-09-11T22:20:18.186698+0000","last_unstale":"2024-09-11T22:22:16.230435+0000","last_undegraded":"2024-09-11T22:22:16.230435+0000","last_fullsized":"2024-09-11T22:22:16.230435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:04.810981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065725+0000","last_change":"2024-09-11T22:20:16.124925+0000","last_active":"2024-09-11T22:22:16.065725+0000","last_peered":"2024-09-11T22:22:16.065725+0000","last_clean":"2024-09-11T22:22:16.065725+0000","last_became_active":"2024-09-11T22:20:16.124580+0000","last_became_peered":"2024-09-11T22:20:16.124580+0000","last_unstale":"2024-09-11T22:22:16.065725+0000","last_undegraded":"2024-09-11T22:22:16.065725+0000","last_fullsized":"2024-09-11T22:22:16.065725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:56.417577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231838+0000","last_change":"2024-09-11T22:21:18.187783+0000","last_active":"2024-09-11T22:22:16.231838+0000","last_peered":"2024-09-11T22:22:16.231838+0000","last_clean":"2024-09-11T22:22:16.231838+0000","last_became_active":"2024-09-11T22:21:18.187358+0000","last_became_peered":"2024-09-11T22:21:18.187358+0000","last_unstale":"2024-09-11T22:22:16.231838+0000","last_undegraded":"2024-09-11T22:22:16.231838+0000","last_fullsized":"2024-09-11T22:22:16.231838+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:41.251531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.439108+0000","last_change":"2024-09-11T22:20:16.118934+0000","last_active":"2024-09-11T22:22:16.439108+0000","last_peered":"2024-09-11T22:22:16.439108+0000","last_clean":"2024-09-11T22:22:16.439108+0000","last_became_active":"2024-09-11T22:20:16.118614+0000","last_became_peered":"2024-09-11T22:20:16.118614+0000","last_unstale":"2024-09-11T22:22:16.439108+0000","last_undegraded":"2024-09-11T22:22:16.439108+0000","last_fullsized":"2024-09-11T22:22:16.439108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:05.791161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231721+0000","last_change":"2024-09-11T22:20:18.197692+0000","last_active":"2024-09-11T22:22:16.231721+0000","last_peered":"2024-09-11T22:22:16.231721+0000","last_clean":"2024-09-11T22:22:16.231721+0000","last_became_active":"2024-09-11T22:20:18.197336+0000","last_became_peered":"2024-09-11T22:20:18.197336+0000","last_unstale":"2024-09-11T22:22:16.231721+0000","last_undegraded":"2024-09-11T22:22:16.231721+0000","last_fullsized":"2024-09-11T22:22:16.231721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:55.172568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:16.186769+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:16.186367+0000","last_became_peered":"2024-09-11T22:20:16.186367+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.279387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065641+0000","last_change":"2024-09-11T22:20:18.195570+0000","last_active":"2024-09-11T22:22:16.065641+0000","last_peered":"2024-09-11T22:22:16.065641+0000","last_clean":"2024-09-11T22:22:16.065641+0000","last_became_active":"2024-09-11T22:20:18.195226+0000","last_became_peered":"2024-09-11T22:20:18.195226+0000","last_unstale":"2024-09-11T22:22:16.065641+0000","last_undegraded":"2024-09-11T22:22:16.065641+0000","last_fullsized":"2024-09-11T22:22:16.065641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:44.108090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235034+0000","last_change":"2024-09-11T22:21:18.194553+0000","last_active":"2024-09-11T22:22:16.235034+0000","last_peered":"2024-09-11T22:22:16.235034+0000","last_clean":"2024-09-11T22:22:16.235034+0000","last_became_active":"2024-09-11T22:21:18.188635+0000","last_became_peered":"2024-09-11T22:21:18.188635+0000","last_unstale":"2024-09-11T22:22:16.235034+0000","last_undegraded":"2024-09-11T22:22:16.235034+0000","last_fullsized":"2024-09-11T22:22:16.235034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:34.165195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234954+0000","last_change":"2024-09-11T22:20:18.194112+0000","last_active":"2024-09-11T22:22:16.234954+0000","last_peered":"2024-09-11T22:22:16.234954+0000","last_clean":"2024-09-11T22:22:16.234954+0000","last_became_active":"2024-09-11T22:20:18.193812+0000","last_became_peered":"2024-09-11T22:20:18.193812+0000","last_unstale":"2024-09-11T22:22:16.234954+0000","last_undegraded":"2024-09-11T22:22:16.234954+0000","last_fullsized":"2024-09-11T22:22:16.234954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:35.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231070+0000","last_change":"2024-09-11T22:21:18.198325+0000","last_active":"2024-09-11T22:22:16.231070+0000","last_peered":"2024-09-11T22:22:16.231070+0000","last_clean":"2024-09-11T22:22:16.231070+0000","last_became_active":"2024-09-11T22:21:18.193651+0000","last_became_peered":"2024-09-11T22:21:18.193651+0000","last_unstale":"2024-09-11T22:22:16.231070+0000","last_undegraded":"2024-09-11T22:22:16.231070+0000","last_fullsized":"2024-09-11T22:22:16.231070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:42.552492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232027+0000","last_change":"2024-09-11T22:20:18.191960+0000","last_active":"2024-09-11T22:22:16.232027+0000","last_peered":"2024-09-11T22:22:16.232027+0000","last_clean":"2024-09-11T22:22:16.232027+0000","last_became_active":"2024-09-11T22:20:18.191635+0000","last_became_peered":"2024-09-11T22:20:18.191635+0000","last_unstale":"2024-09-11T22:22:16.232027+0000","last_undegraded":"2024-09-11T22:22:16.232027+0000","last_fullsized":"2024-09-11T22:22:16.232027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:19:18.919247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.176627+0000","last_change":"2024-09-11T22:20:16.123608+0000","last_active":"2024-09-11T22:22:16.176627+0000","last_peered":"2024-09-11T22:22:16.176627+0000","last_clean":"2024-09-11T22:22:16.176627+0000","last_became_active":"2024-09-11T22:20:16.123193+0000","last_became_peered":"2024-09-11T22:20:16.123193+0000","last_unstale":"2024-09-11T22:22:16.176627+0000","last_undegraded":"2024-09-11T22:22:16.176627+0000","last_fullsized":"2024-09-11T22:22:16.176627+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:07.362115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231887+0000","last_change":"2024-09-11T22:20:18.199957+0000","last_active":"2024-09-11T22:22:16.231887+0000","last_peered":"2024-09-11T22:22:16.231887+0000","last_clean":"2024-09-11T22:22:16.231887+0000","last_became_active":"2024-09-11T22:20:18.199637+0000","last_became_peered":"2024-09-11T22:20:18.199637+0000","last_unstale":"2024-09-11T22:22:16.231887+0000","last_undegraded":"2024-09-11T22:22:16.231887+0000","last_fullsized":"2024-09-11T22:22:16.231887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:14.949002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.277747+0000","last_change":"2024-09-11T22:21:18.198896+0000","last_active":"2024-09-11T22:22:16.277747+0000","last_peered":"2024-09-11T22:22:16.277747+0000","last_clean":"2024-09-11T22:22:16.277747+0000","last_became_active":"2024-09-11T22:21:18.189489+0000","last_became_peered":"2024-09-11T22:21:18.189489+0000","last_unstale":"2024-09-11T22:22:16.277747+0000","last_undegraded":"2024-09-11T22:22:16.277747+0000","last_fullsized":"2024-09-11T22:22:16.277747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:19.908801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232111+0000","last_change":"2024-09-11T22:20:18.196691+0000","last_active":"2024-09-11T22:22:16.232111+0000","last_peered":"2024-09-11T22:22:16.232111+0000","last_clean":"2024-09-11T22:22:16.232111+0000","last_became_active":"2024-09-11T22:20:18.196364+0000","last_became_peered":"2024-09-11T22:20:18.196364+0000","last_unstale":"2024-09-11T22:22:16.232111+0000","last_undegraded":"2024-09-11T22:22:16.232111+0000","last_fullsized":"2024-09-11T22:22:16.232111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:50.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231876+0000","last_change":"2024-09-11T22:19:02.590079+0000","last_active":"2024-09-11T22:22:16.231876+0000","last_peered":"2024-09-11T22:22:16.231876+0000","last_clean":"2024-09-11T22:22:16.231876+0000","last_became_active":"2024-09-11T22:19:02.589644+0000","last_became_peered":"2024-09-11T22:19:02.589644+0000","last_unstale":"2024-09-11T22:22:16.231876+0000","last_undegraded":"2024-09-11T22:22:16.231876+0000","last_fullsized":"2024-09-11T22:22:16.231876+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_clean_scrub_stamp":"2024-09-11T22:15:33.244685+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:45.506190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.229755+0000","last_change":"2024-09-11T22:20:16.116661+0000","last_active":"2024-09-11T22:22:16.229755+0000","last_peered":"2024-09-11T22:22:16.229755+0000","last_clean":"2024-09-11T22:22:16.229755+0000","last_became_active":"2024-09-11T22:20:16.116196+0000","last_became_peered":"2024-09-11T22:20:16.116196+0000","last_unstale":"2024-09-11T22:22:16.229755+0000","last_undegraded":"2024-09-11T22:22:16.229755+0000","last_fullsized":"2024-09-11T22:22:16.229755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:08.875170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059167+0000","last_change":"2024-09-11T22:20:18.193176+0000","last_active":"2024-09-11T22:22:16.059167+0000","last_peered":"2024-09-11T22:22:16.059167+0000","last_clean":"2024-09-11T22:22:16.059167+0000","last_became_active":"2024-09-11T22:20:18.192312+0000","last_became_peered":"2024-09-11T22:20:18.192312+0000","last_unstale":"2024-09-11T22:22:16.059167+0000","last_undegraded":"2024-09-11T22:22:16.059167+0000","last_fullsized":"2024-09-11T22:22:16.059167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:08.738569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234694+0000","last_change":"2024-09-11T22:21:18.196985+0000","last_active":"2024-09-11T22:22:16.234694+0000","last_peered":"2024-09-11T22:22:16.234694+0000","last_clean":"2024-09-11T22:22:16.234694+0000","last_became_active":"2024-09-11T22:21:18.189487+0000","last_became_peered":"2024-09-11T22:21:18.189487+0000","last_unstale":"2024-09-11T22:22:16.234694+0000","last_undegraded":"2024-09-11T22:22:16.234694+0000","last_fullsized":"2024-09-11T22:22:16.234694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:20.715696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230470+0000","last_change":"2024-09-11T22:20:18.191118+0000","last_active":"2024-09-11T22:22:16.230470+0000","last_peered":"2024-09-11T22:22:16.230470+0000","last_clean":"2024-09-11T22:22:16.230470+0000","last_became_active":"2024-09-11T22:20:18.190760+0000","last_became_peered":"2024-09-11T22:20:18.190760+0000","last_unstale":"2024-09-11T22:22:16.230470+0000","last_undegraded":"2024-09-11T22:22:16.230470+0000","last_fullsized":"2024-09-11T22:22:16.230470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:59.220067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235387+0000","last_change":"2024-09-11T22:20:16.186407+0000","last_active":"2024-09-11T22:22:16.235387+0000","last_peered":"2024-09-11T22:22:16.235387+0000","last_clean":"2024-09-11T22:22:16.235387+0000","last_became_active":"2024-09-11T22:20:16.186105+0000","last_became_peered":"2024-09-11T22:20:16.186105+0000","last_unstale":"2024-09-11T22:22:16.235387+0000","last_undegraded":"2024-09-11T22:22:16.235387+0000","last_fullsized":"2024-09-11T22:22:16.235387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:43.942072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065626+0000","last_change":"2024-09-11T22:20:18.191970+0000","last_active":"2024-09-11T22:22:16.065626+0000","last_peered":"2024-09-11T22:22:16.065626+0000","last_clean":"2024-09-11T22:22:16.065626+0000","last_became_active":"2024-09-11T22:20:18.191623+0000","last_became_peered":"2024-09-11T22:20:18.191623+0000","last_unstale":"2024-09-11T22:22:16.065626+0000","last_undegraded":"2024-09-11T22:22:16.065626+0000","last_fullsized":"2024-09-11T22:22:16.065626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:33.213979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231514+0000","last_change":"2024-09-11T22:20:16.119936+0000","last_active":"2024-09-11T22:22:16.231514+0000","last_peered":"2024-09-11T22:22:16.231514+0000","last_clean":"2024-09-11T22:22:16.231514+0000","last_became_active":"2024-09-11T22:20:16.119453+0000","last_became_peered":"2024-09-11T22:20:16.119453+0000","last_unstale":"2024-09-11T22:22:16.231514+0000","last_undegraded":"2024-09-11T22:22:16.231514+0000","last_fullsized":"2024-09-11T22:22:16.231514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:20.717980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068790+0000","last_change":"2024-09-11T22:20:18.192581+0000","last_active":"2024-09-11T22:22:16.068790+0000","last_peered":"2024-09-11T22:22:16.068790+0000","last_clean":"2024-09-11T22:22:16.068790+0000","last_became_active":"2024-09-11T22:20:18.191702+0000","last_became_peered":"2024-09-11T22:20:18.191702+0000","last_unstale":"2024-09-11T22:22:16.068790+0000","last_undegraded":"2024-09-11T22:22:16.068790+0000","last_fullsized":"2024-09-11T22:22:16.068790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:48.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973402+0000","last_change":"2024-09-11T22:20:16.186485+0000","last_active":"2024-09-11T22:22:15.973402+0000","last_peered":"2024-09-11T22:22:15.973402+0000","last_clean":"2024-09-11T22:22:15.973402+0000","last_became_active":"2024-09-11T22:20:16.186171+0000","last_became_peered":"2024-09-11T22:20:16.186171+0000","last_unstale":"2024-09-11T22:22:15.973402+0000","last_undegraded":"2024-09-11T22:22:15.973402+0000","last_fullsized":"2024-09-11T22:22:15.973402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:26.373626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068876+0000","last_change":"2024-09-11T22:20:18.191361+0000","last_active":"2024-09-11T22:22:16.068876+0000","last_peered":"2024-09-11T22:22:16.068876+0000","last_clean":"2024-09-11T22:22:16.068876+0000","last_became_active":"2024-09-11T22:20:18.191036+0000","last_became_peered":"2024-09-11T22:20:18.191036+0000","last_unstale":"2024-09-11T22:22:16.068876+0000","last_undegraded":"2024-09-11T22:22:16.068876+0000","last_fullsized":"2024-09-11T22:22:16.068876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:29.124650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059181+0000","last_change":"2024-09-11T22:20:16.188156+0000","last_active":"2024-09-11T22:22:16.059181+0000","last_peered":"2024-09-11T22:22:16.059181+0000","last_clean":"2024-09-11T22:22:16.059181+0000","last_became_active":"2024-09-11T22:20:16.187809+0000","last_became_peered":"2024-09-11T22:20:16.187809+0000","last_unstale":"2024-09-11T22:22:16.059181+0000","last_undegraded":"2024-09-11T22:22:16.059181+0000","last_fullsized":"2024-09-11T22:22:16.059181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:04.219621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230896+0000","last_change":"2024-09-11T22:20:18.182380+0000","last_active":"2024-09-11T22:22:16.230896+0000","last_peered":"2024-09-11T22:22:16.230896+0000","last_clean":"2024-09-11T22:22:16.230896+0000","last_became_active":"2024-09-11T22:20:18.182062+0000","last_became_peered":"2024-09-11T22:20:18.182062+0000","last_unstale":"2024-09-11T22:22:16.230896+0000","last_undegraded":"2024-09-11T22:22:16.230896+0000","last_fullsized":"2024-09-11T22:22:16.230896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:15.575846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'50","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.179372+0000","last_change":"2024-09-11T22:20:16.117335+0000","last_active":"2024-09-11T22:22:16.179372+0000","last_peered":"2024-09-11T22:22:16.179372+0000","last_clean":"2024-09-11T22:22:16.179372+0000","last_became_active":"2024-09-11T22:20:16.116977+0000","last_became_peered":"2024-09-11T22:20:16.116977+0000","last_unstale":"2024-09-11T22:22:16.179372+0000","last_undegraded":"2024-09-11T22:22:16.179372+0000","last_fullsized":"2024-09-11T22:22:16.179372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:51.744202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059236+0000","last_change":"2024-09-11T22:20:18.194069+0000","last_active":"2024-09-11T22:22:16.059236+0000","last_peered":"2024-09-11T22:22:16.059236+0000","last_clean":"2024-09-11T22:22:16.059236+0000","last_became_active":"2024-09-11T22:20:18.193765+0000","last_became_peered":"2024-09-11T22:20:18.193765+0000","last_unstale":"2024-09-11T22:22:16.059236+0000","last_undegraded":"2024-09-11T22:22:16.059236+0000","last_fullsized":"2024-09-11T22:22:16.059236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:25.047405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230131+0000","last_change":"2024-09-11T22:20:16.460987+0000","last_active":"2024-09-11T22:22:16.230131+0000","last_peered":"2024-09-11T22:22:16.230131+0000","last_clean":"2024-09-11T22:22:16.230131+0000","last_became_active":"2024-09-11T22:20:16.460516+0000","last_became_peered":"2024-09-11T22:20:16.460516+0000","last_unstale":"2024-09-11T22:22:16.230131+0000","last_undegraded":"2024-09-11T22:22:16.230131+0000","last_fullsized":"2024-09-11T22:22:16.230131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:00.987981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234264+0000","last_change":"2024-09-11T22:21:18.188802+0000","last_active":"2024-09-11T22:22:16.234264+0000","last_peered":"2024-09-11T22:22:16.234264+0000","last_clean":"2024-09-11T22:22:16.234264+0000","last_became_active":"2024-09-11T22:21:18.188441+0000","last_became_peered":"2024-09-11T22:21:18.188441+0000","last_unstale":"2024-09-11T22:22:16.234264+0000","last_undegraded":"2024-09-11T22:22:16.234264+0000","last_fullsized":"2024-09-11T22:22:16.234264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:02.295441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975337+0000","last_change":"2024-09-11T22:20:16.186498+0000","last_active":"2024-09-11T22:22:15.975337+0000","last_peered":"2024-09-11T22:22:15.975337+0000","last_clean":"2024-09-11T22:22:15.975337+0000","last_became_active":"2024-09-11T22:20:16.186214+0000","last_became_peered":"2024-09-11T22:20:16.186214+0000","last_unstale":"2024-09-11T22:22:15.975337+0000","last_undegraded":"2024-09-11T22:22:15.975337+0000","last_fullsized":"2024-09-11T22:22:15.975337+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:09.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:18.187489+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:18.185649+0000","last_became_peered":"2024-09-11T22:20:18.185649+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:42.825075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059011+0000","last_change":"2024-09-11T22:20:16.117627+0000","last_active":"2024-09-11T22:22:16.059011+0000","last_peered":"2024-09-11T22:22:16.059011+0000","last_clean":"2024-09-11T22:22:16.059011+0000","last_became_active":"2024-09-11T22:20:16.117243+0000","last_became_peered":"2024-09-11T22:20:16.117243+0000","last_unstale":"2024-09-11T22:22:16.059011+0000","last_undegraded":"2024-09-11T22:22:16.059011+0000","last_fullsized":"2024-09-11T22:22:16.059011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:58.240535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234846+0000","last_change":"2024-09-11T22:20:18.184326+0000","last_active":"2024-09-11T22:22:16.234846+0000","last_peered":"2024-09-11T22:22:16.234846+0000","last_clean":"2024-09-11T22:22:16.234846+0000","last_became_active":"2024-09-11T22:20:18.183995+0000","last_became_peered":"2024-09-11T22:20:18.183995+0000","last_unstale":"2024-09-11T22:22:16.234846+0000","last_undegraded":"2024-09-11T22:22:16.234846+0000","last_fullsized":"2024-09-11T22:22:16.234846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:33.468000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977256+0000","last_change":"2024-09-11T22:20:16.127180+0000","last_active":"2024-09-11T22:22:15.977256+0000","last_peered":"2024-09-11T22:22:15.977256+0000","last_clean":"2024-09-11T22:22:15.977256+0000","last_became_active":"2024-09-11T22:20:16.126696+0000","last_became_peered":"2024-09-11T22:20:16.126696+0000","last_unstale":"2024-09-11T22:22:15.977256+0000","last_undegraded":"2024-09-11T22:22:15.977256+0000","last_fullsized":"2024-09-11T22:22:15.977256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:02.637225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973369+0000","last_change":"2024-09-11T22:21:18.188614+0000","last_active":"2024-09-11T22:22:15.973369+0000","last_peered":"2024-09-11T22:22:15.973369+0000","last_clean":"2024-09-11T22:22:15.973369+0000","last_became_active":"2024-09-11T22:21:18.188277+0000","last_became_peered":"2024-09-11T22:21:18.188277+0000","last_unstale":"2024-09-11T22:22:15.973369+0000","last_undegraded":"2024-09-11T22:22:15.973369+0000","last_fullsized":"2024-09-11T22:22:15.973369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:59.995647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058897+0000","last_change":"2024-09-11T22:20:16.128311+0000","last_active":"2024-09-11T22:22:16.058897+0000","last_peered":"2024-09-11T22:22:16.058897+0000","last_clean":"2024-09-11T22:22:16.058897+0000","last_became_active":"2024-09-11T22:20:16.127853+0000","last_became_peered":"2024-09-11T22:20:16.127853+0000","last_unstale":"2024-09-11T22:22:16.058897+0000","last_undegraded":"2024-09-11T22:22:16.058897+0000","last_fullsized":"2024-09-11T22:22:16.058897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:57.562470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231820+0000","last_change":"2024-09-11T22:20:18.195782+0000","last_active":"2024-09-11T22:22:16.231820+0000","last_peered":"2024-09-11T22:22:16.231820+0000","last_clean":"2024-09-11T22:22:16.231820+0000","last_became_active":"2024-09-11T22:20:18.195365+0000","last_became_peered":"2024-09-11T22:20:18.195365+0000","last_unstale":"2024-09-11T22:22:16.231820+0000","last_undegraded":"2024-09-11T22:22:16.231820+0000","last_fullsized":"2024-09-11T22:22:16.231820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:49.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065812+0000","last_change":"2024-09-11T22:20:18.199344+0000","last_active":"2024-09-11T22:22:16.065812+0000","last_peered":"2024-09-11T22:22:16.065812+0000","last_clean":"2024-09-11T22:22:16.065812+0000","last_became_active":"2024-09-11T22:20:18.199016+0000","last_became_peered":"2024-09-11T22:20:18.199016+0000","last_unstale":"2024-09-11T22:22:16.065812+0000","last_undegraded":"2024-09-11T22:22:16.065812+0000","last_fullsized":"2024-09-11T22:22:16.065812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:38.322831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059125+0000","last_change":"2024-09-11T22:20:16.122236+0000","last_active":"2024-09-11T22:22:16.059125+0000","last_peered":"2024-09-11T22:22:16.059125+0000","last_clean":"2024-09-11T22:22:16.059125+0000","last_became_active":"2024-09-11T22:20:16.121867+0000","last_became_peered":"2024-09-11T22:20:16.121867+0000","last_unstale":"2024-09-11T22:22:16.059125+0000","last_undegraded":"2024-09-11T22:22:16.059125+0000","last_fullsized":"2024-09-11T22:22:16.059125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:56.782445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975374+0000","last_change":"2024-09-11T22:20:18.187338+0000","last_active":"2024-09-11T22:22:15.975374+0000","last_peered":"2024-09-11T22:22:15.975374+0000","last_clean":"2024-09-11T22:22:15.975374+0000","last_became_active":"2024-09-11T22:20:18.186985+0000","last_became_peered":"2024-09-11T22:20:18.186985+0000","last_unstale":"2024-09-11T22:22:15.975374+0000","last_undegraded":"2024-09-11T22:22:15.975374+0000","last_fullsized":"2024-09-11T22:22:15.975374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:58.170680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059106+0000","last_change":"2024-09-11T22:20:16.128579+0000","last_active":"2024-09-11T22:22:16.059106+0000","last_peered":"2024-09-11T22:22:16.059106+0000","last_clean":"2024-09-11T22:22:16.059106+0000","last_became_active":"2024-09-11T22:20:16.127864+0000","last_became_peered":"2024-09-11T22:20:16.127864+0000","last_unstale":"2024-09-11T22:22:16.059106+0000","last_undegraded":"2024-09-11T22:22:16.059106+0000","last_fullsized":"2024-09-11T22:22:16.059106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:11:32.005487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234449+0000","last_change":"2024-09-11T22:20:18.185205+0000","last_active":"2024-09-11T22:22:16.234449+0000","last_peered":"2024-09-11T22:22:16.234449+0000","last_clean":"2024-09-11T22:22:16.234449+0000","last_became_active":"2024-09-11T22:20:18.184402+0000","last_became_peered":"2024-09-11T22:20:18.184402+0000","last_unstale":"2024-09-11T22:22:16.234449+0000","last_undegraded":"2024-09-11T22:22:16.234449+0000","last_fullsized":"2024-09-11T22:22:16.234449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:17:48.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.060208+0000","last_change":"2024-09-11T22:20:16.186372+0000","last_active":"2024-09-11T22:22:16.060208+0000","last_peered":"2024-09-11T22:22:16.060208+0000","last_clean":"2024-09-11T22:22:16.060208+0000","last_became_active":"2024-09-11T22:20:16.185990+0000","last_became_peered":"2024-09-11T22:20:16.185990+0000","last_unstale":"2024-09-11T22:22:16.060208+0000","last_undegraded":"2024-09-11T22:22:16.060208+0000","last_fullsized":"2024-09-11T22:22:16.060208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:23.296379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059971+0000","last_change":"2024-09-11T22:20:18.190670+0000","last_active":"2024-09-11T22:22:16.059971+0000","last_peered":"2024-09-11T22:22:16.059971+0000","last_clean":"2024-09-11T22:22:16.059971+0000","last_became_active":"2024-09-11T22:20:18.190339+0000","last_became_peered":"2024-09-11T22:20:18.190339+0000","last_unstale":"2024-09-11T22:22:16.059971+0000","last_undegraded":"2024-09-11T22:22:16.059971+0000","last_fullsized":"2024-09-11T22:22:16.059971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.085305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973670+0000","last_change":"2024-09-11T22:20:16.117646+0000","last_active":"2024-09-11T22:22:15.973670+0000","last_peered":"2024-09-11T22:22:15.973670+0000","last_clean":"2024-09-11T22:22:15.973670+0000","last_became_active":"2024-09-11T22:20:16.117111+0000","last_became_peered":"2024-09-11T22:20:16.117111+0000","last_unstale":"2024-09-11T22:22:15.973670+0000","last_undegraded":"2024-09-11T22:22:15.973670+0000","last_fullsized":"2024-09-11T22:22:15.973670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:36.470158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600132,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1158,"num_read_kb":717,"num_write":372,"num_write_kb":1463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1192,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1800396,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108858,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":443740,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1085440,"data_stored":1226275,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":443255,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201332,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":836827,"data_compressed":27823,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":983040,"data_stored":1037781,"data_compressed":44963,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443506,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":1167739,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":445182,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":433812,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":991232,"data_stored":503564,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":987136,"data_stored":510899,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":937984,"data_stored":444704,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135121,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200602,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:22:20.911 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T22:22:21.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.052+0000 7ff284142640 1 -- 172.21.15.5:0/196324524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff27c106990 msgr2=0x7ff27c106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.052+0000 7ff284142640 1 --2- 172.21.15.5:0/196324524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff27c106990 0x7ff27c106d70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ff278009930 tx=0x7ff27802a420 comp rx=0 tx=0).stop 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.052+0000 7ff284142640 1 -- 172.21.15.5:0/196324524 shutdown_connections 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.052+0000 7ff284142640 1 --2- 172.21.15.5:0/196324524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff27c101320 0x7ff27c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.052+0000 7ff284142640 1 --2- 172.21.15.5:0/196324524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff27c100980 0x7ff27c100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.052+0000 7ff284142640 1 --2- 172.21.15.5:0/196324524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff27c106990 0x7ff27c106d70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.052+0000 7ff284142640 1 -- 172.21.15.5:0/196324524 >> 172.21.15.5:0/196324524 conn(0x7ff27c077a10 msgr2=0x7ff27c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.052+0000 7ff284142640 1 -- 172.21.15.5:0/196324524 shutdown_connections 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.052+0000 7ff284142640 1 -- 172.21.15.5:0/196324524 wait complete. 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff284142640 1 Processor -- start 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff284142640 1 -- start start 2024-09-11T22:22:21.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff284142640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff27c100980 0x7ff27c197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff284142640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff27c101320 0x7ff27c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff284142640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff27c106990 0x7ff27c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff284142640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c110e20 con 0x7ff27c100980 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff284142640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c110ca0 con 0x7ff27c106990 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff284142640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c110fa0 con 0x7ff27c101320 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff281ede640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff27c100980 0x7ff27c197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff281ede640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff27c100980 0x7ff27c197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36292/0 (socket says 172.21.15.5:36292) 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff281ede640 1 -- 172.21.15.5:0/3079018952 learned_addr learned my addr 172.21.15.5:0/3079018952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff281ede640 1 -- 172.21.15.5:0/3079018952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff27c101320 msgr2=0x7ff27c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff281ede640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff27c101320 0x7ff27c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff281ede640 1 -- 172.21.15.5:0/3079018952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff27c106990 msgr2=0x7ff27c197b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:22:21.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff281ede640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff27c106990 0x7ff27c197b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff281ede640 1 -- 172.21.15.5:0/3079018952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27c1a68f0 con 0x7ff27c100980 2024-09-11T22:22:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.053+0000 7ff281ede640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff27c100980 0x7ff27c197080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff27802a940 tx=0x7ff278003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.054+0000 7ff272ffd640 1 -- 172.21.15.5:0/3079018952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27803f070 con 0x7ff27c100980 2024-09-11T22:22:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.054+0000 7ff284142640 1 -- 172.21.15.5:0/3079018952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27c1a6b80 con 0x7ff27c100980 2024-09-11T22:22:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.054+0000 7ff272ffd640 1 -- 172.21.15.5:0/3079018952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff278031c90 con 0x7ff27c100980 2024-09-11T22:22:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.054+0000 7ff272ffd640 1 -- 172.21.15.5:0/3079018952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278035030 con 0x7ff27c100980 2024-09-11T22:22:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.054+0000 7ff284142640 1 -- 172.21.15.5:0/3079018952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27c1a7070 con 0x7ff27c100980 2024-09-11T22:22:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.055+0000 7ff284142640 1 -- 172.21.15.5:0/3079018952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff27c100de0 con 0x7ff27c100980 2024-09-11T22:22:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.056+0000 7ff272ffd640 1 -- 172.21.15.5:0/3079018952 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7ff278042070 con 0x7ff27c100980 2024-09-11T22:22:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.057+0000 7ff272ffd640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff2680a2e20 0x7ff2680a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.057+0000 7ff2816dd640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff2680a2e20 0x7ff2680a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.057+0000 7ff272ffd640 1 -- 172.21.15.5:0/3079018952 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff2780f0590 con 0x7ff27c100980 2024-09-11T22:22:21.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.057+0000 7ff2816dd640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff2680a2e20 0x7ff2680a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff26c005fd0 tx=0x7ff26c005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:21.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.060+0000 7ff272ffd640 1 -- 172.21.15.5:0/3079018952 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2780bcfd0 con 0x7ff27c100980 2024-09-11T22:22:21.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.233+0000 7ff284142640 1 -- 172.21.15.5:0/3079018952 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff27c062630 con 0x7ff2680a2e20 2024-09-11T22:22:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.239+0000 7ff272ffd640 1 -- 172.21.15.5:0/3079018952 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258088 (secure 0 0 0) 0x7ff27c062630 con 0x7ff2680a2e20 2024-09-11T22:22:21.244 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-11T22:22:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 -- 172.21.15.5:0/3079018952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff2680a2e20 msgr2=0x7ff2680a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff2680a2e20 0x7ff2680a52e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff26c005fd0 tx=0x7ff26c005850 comp rx=0 tx=0).stop 2024-09-11T22:22:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 -- 172.21.15.5:0/3079018952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff27c100980 msgr2=0x7ff27c197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff27c100980 0x7ff27c197080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff27802a940 tx=0x7ff278003b90 comp rx=0 tx=0).stop 2024-09-11T22:22:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 -- 172.21.15.5:0/3079018952 shutdown_connections 2024-09-11T22:22:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff2680a2e20 0x7ff2680a52e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff27c106990 0x7ff27c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff27c101320 0x7ff27c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 --2- 172.21.15.5:0/3079018952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff27c100980 0x7ff27c197080 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 -- 172.21.15.5:0/3079018952 >> 172.21.15.5:0/3079018952 conn(0x7ff27c077a10 msgr2=0x7ff27c103930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 -- 172.21.15.5:0/3079018952 shutdown_connections 2024-09-11T22:22:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.243+0000 7ff284142640 1 -- 172.21.15.5:0/3079018952 wait complete. 2024-09-11T22:22:21.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:20 smithi005 ceph-mon[30943]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:21.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:20 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:20 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:20 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:20 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:21.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:20 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:21.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:21 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:21.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:21 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:21 smithi005 ceph-mon[30943]: mgrmap e23: x(active, since 5s), standbys: z, y 2024-09-11T22:22:21.257 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-11T22:22:19.987091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190734,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1294,"num_read_kb":836,"num_write":555,"num_write_kb":4403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181928,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385548,"kb_used_data":11292,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556276,"statfs":{"total":1151940427776,"available":1151545626624,"internally_reserved":0,"allocated":11563008,"data_stored":7937284,"data_compressed":241535,"data_compressed_allocated":319488,"data_compressed_original":2695168,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987365"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235071+0000","last_change":"2024-09-11T22:20:18.189744+0000","last_active":"2024-09-11T22:22:16.235071+0000","last_peered":"2024-09-11T22:22:16.235071+0000","last_clean":"2024-09-11T22:22:16.235071+0000","last_became_active":"2024-09-11T22:20:18.189398+0000","last_became_peered":"2024-09-11T22:20:18.189398+0000","last_unstale":"2024-09-11T22:22:16.235071+0000","last_undegraded":"2024-09-11T22:22:16.235071+0000","last_fullsized":"2024-09-11T22:22:16.235071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:24.152589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234137+0000","last_change":"2024-09-11T22:20:18.186727+0000","last_active":"2024-09-11T22:22:16.234137+0000","last_peered":"2024-09-11T22:22:16.234137+0000","last_clean":"2024-09-11T22:22:16.234137+0000","last_became_active":"2024-09-11T22:20:18.186414+0000","last_became_peered":"2024-09-11T22:20:18.186414+0000","last_unstale":"2024-09-11T22:22:16.234137+0000","last_undegraded":"2024-09-11T22:22:16.234137+0000","last_fullsized":"2024-09-11T22:22:16.234137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:53.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233845+0000","last_change":"2024-09-11T22:20:18.179984+0000","last_active":"2024-09-11T22:22:16.233845+0000","last_peered":"2024-09-11T22:22:16.233845+0000","last_clean":"2024-09-11T22:22:16.233845+0000","last_became_active":"2024-09-11T22:20:18.179329+0000","last_became_peered":"2024-09-11T22:20:18.179329+0000","last_unstale":"2024-09-11T22:22:16.233845+0000","last_undegraded":"2024-09-11T22:22:16.233845+0000","last_fullsized":"2024-09-11T22:22:16.233845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:19.149970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230208+0000","last_change":"2024-09-11T22:20:18.191768+0000","last_active":"2024-09-11T22:22:16.230208+0000","last_peered":"2024-09-11T22:22:16.230208+0000","last_clean":"2024-09-11T22:22:16.230208+0000","last_became_active":"2024-09-11T22:20:18.191336+0000","last_became_peered":"2024-09-11T22:20:18.191336+0000","last_unstale":"2024-09-11T22:22:16.230208+0000","last_undegraded":"2024-09-11T22:22:16.230208+0000","last_fullsized":"2024-09-11T22:22:16.230208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:44.768932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058858+0000","last_change":"2024-09-11T22:20:18.181469+0000","last_active":"2024-09-11T22:22:16.058858+0000","last_peered":"2024-09-11T22:22:16.058858+0000","last_clean":"2024-09-11T22:22:16.058858+0000","last_became_active":"2024-09-11T22:20:18.181100+0000","last_became_peered":"2024-09-11T22:20:18.181100+0000","last_unstale":"2024-09-11T22:22:16.058858+0000","last_undegraded":"2024-09-11T22:22:16.058858+0000","last_fullsized":"2024-09-11T22:22:16.058858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:14.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068675+0000","last_change":"2024-09-11T22:20:18.190855+0000","last_active":"2024-09-11T22:22:16.068675+0000","last_peered":"2024-09-11T22:22:16.068675+0000","last_clean":"2024-09-11T22:22:16.068675+0000","last_became_active":"2024-09-11T22:20:18.190523+0000","last_became_peered":"2024-09-11T22:20:18.190523+0000","last_unstale":"2024-09-11T22:22:16.068675+0000","last_undegraded":"2024-09-11T22:22:16.068675+0000","last_fullsized":"2024-09-11T22:22:16.068675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:18.574957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231733+0000","last_change":"2024-09-11T22:20:18.199213+0000","last_active":"2024-09-11T22:22:16.231733+0000","last_peered":"2024-09-11T22:22:16.231733+0000","last_clean":"2024-09-11T22:22:16.231733+0000","last_became_active":"2024-09-11T22:20:18.198913+0000","last_became_peered":"2024-09-11T22:20:18.198913+0000","last_unstale":"2024-09-11T22:22:16.231733+0000","last_undegraded":"2024-09-11T22:22:16.231733+0000","last_fullsized":"2024-09-11T22:22:16.231733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:22.682802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058633+0000","last_change":"2024-09-11T22:20:18.192458+0000","last_active":"2024-09-11T22:22:16.058633+0000","last_peered":"2024-09-11T22:22:16.058633+0000","last_clean":"2024-09-11T22:22:16.058633+0000","last_became_active":"2024-09-11T22:20:18.191967+0000","last_became_peered":"2024-09-11T22:20:18.191967+0000","last_unstale":"2024-09-11T22:22:16.058633+0000","last_undegraded":"2024-09-11T22:22:16.058633+0000","last_fullsized":"2024-09-11T22:22:16.058633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:52.686157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975713+0000","last_change":"2024-09-11T22:21:18.191732+0000","last_active":"2024-09-11T22:22:15.975713+0000","last_peered":"2024-09-11T22:22:15.975713+0000","last_clean":"2024-09-11T22:22:15.975713+0000","last_became_active":"2024-09-11T22:21:18.191375+0000","last_became_peered":"2024-09-11T22:21:18.191375+0000","last_unstale":"2024-09-11T22:22:15.975713+0000","last_undegraded":"2024-09-11T22:22:15.975713+0000","last_fullsized":"2024-09-11T22:22:15.975713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:13.645224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.069211+0000","last_change":"2024-09-11T22:20:18.193771+0000","last_active":"2024-09-11T22:22:16.069211+0000","last_peered":"2024-09-11T22:22:16.069211+0000","last_clean":"2024-09-11T22:22:16.069211+0000","last_became_active":"2024-09-11T22:20:18.193041+0000","last_became_peered":"2024-09-11T22:20:18.193041+0000","last_unstale":"2024-09-11T22:22:16.069211+0000","last_undegraded":"2024-09-11T22:22:16.069211+0000","last_fullsized":"2024-09-11T22:22:16.069211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:06.656169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231998+0000","last_change":"2024-09-11T22:20:18.197933+0000","last_active":"2024-09-11T22:22:16.231998+0000","last_peered":"2024-09-11T22:22:16.231998+0000","last_clean":"2024-09-11T22:22:16.231998+0000","last_became_active":"2024-09-11T22:20:18.197210+0000","last_became_peered":"2024-09-11T22:20:18.197210+0000","last_unstale":"2024-09-11T22:22:16.231998+0000","last_undegraded":"2024-09-11T22:22:16.231998+0000","last_fullsized":"2024-09-11T22:22:16.231998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:42.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230250+0000","last_change":"2024-09-11T22:20:18.190739+0000","last_active":"2024-09-11T22:22:16.230250+0000","last_peered":"2024-09-11T22:22:16.230250+0000","last_clean":"2024-09-11T22:22:16.230250+0000","last_became_active":"2024-09-11T22:20:18.190122+0000","last_became_peered":"2024-09-11T22:20:18.190122+0000","last_unstale":"2024-09-11T22:22:16.230250+0000","last_undegraded":"2024-09-11T22:22:16.230250+0000","last_fullsized":"2024-09-11T22:22:16.230250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.585035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230697+0000","last_change":"2024-09-11T22:20:18.190303+0000","last_active":"2024-09-11T22:22:16.230697+0000","last_peered":"2024-09-11T22:22:16.230697+0000","last_clean":"2024-09-11T22:22:16.230697+0000","last_became_active":"2024-09-11T22:20:18.189962+0000","last_became_peered":"2024-09-11T22:20:18.189962+0000","last_unstale":"2024-09-11T22:22:16.230697+0000","last_undegraded":"2024-09-11T22:22:16.230697+0000","last_fullsized":"2024-09-11T22:22:16.230697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:44.872813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233520+0000","last_change":"2024-09-11T22:20:18.185893+0000","last_active":"2024-09-11T22:22:16.233520+0000","last_peered":"2024-09-11T22:22:16.233520+0000","last_clean":"2024-09-11T22:22:16.233520+0000","last_became_active":"2024-09-11T22:20:18.185592+0000","last_became_peered":"2024-09-11T22:20:18.185592+0000","last_unstale":"2024-09-11T22:22:16.233520+0000","last_undegraded":"2024-09-11T22:22:16.233520+0000","last_fullsized":"2024-09-11T22:22:16.233520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:14.725301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233777+0000","last_change":"2024-09-11T22:20:18.193943+0000","last_active":"2024-09-11T22:22:16.233777+0000","last_peered":"2024-09-11T22:22:16.233777+0000","last_clean":"2024-09-11T22:22:16.233777+0000","last_became_active":"2024-09-11T22:20:18.193558+0000","last_became_peered":"2024-09-11T22:20:18.193558+0000","last_unstale":"2024-09-11T22:22:16.233777+0000","last_undegraded":"2024-09-11T22:22:16.233777+0000","last_fullsized":"2024-09-11T22:22:16.233777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:08.736161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975254+0000","last_change":"2024-09-11T22:20:18.191815+0000","last_active":"2024-09-11T22:22:15.975254+0000","last_peered":"2024-09-11T22:22:15.975254+0000","last_clean":"2024-09-11T22:22:15.975254+0000","last_became_active":"2024-09-11T22:20:18.191483+0000","last_became_peered":"2024-09-11T22:20:18.191483+0000","last_unstale":"2024-09-11T22:22:15.975254+0000","last_undegraded":"2024-09-11T22:22:15.975254+0000","last_fullsized":"2024-09-11T22:22:15.975254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:26.432117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235101+0000","last_change":"2024-09-11T22:20:18.190758+0000","last_active":"2024-09-11T22:22:16.235101+0000","last_peered":"2024-09-11T22:22:16.235101+0000","last_clean":"2024-09-11T22:22:16.235101+0000","last_became_active":"2024-09-11T22:20:18.190431+0000","last_became_peered":"2024-09-11T22:20:18.190431+0000","last_unstale":"2024-09-11T22:22:16.235101+0000","last_undegraded":"2024-09-11T22:22:16.235101+0000","last_fullsized":"2024-09-11T22:22:16.235101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:10.927051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973552+0000","last_change":"2024-09-11T22:20:18.185191+0000","last_active":"2024-09-11T22:22:15.973552+0000","last_peered":"2024-09-11T22:22:15.973552+0000","last_clean":"2024-09-11T22:22:15.973552+0000","last_became_active":"2024-09-11T22:20:18.184906+0000","last_became_peered":"2024-09-11T22:20:18.184906+0000","last_unstale":"2024-09-11T22:22:15.973552+0000","last_undegraded":"2024-09-11T22:22:15.973552+0000","last_fullsized":"2024-09-11T22:22:15.973552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:32.294780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977515+0000","last_change":"2024-09-11T22:20:18.187137+0000","last_active":"2024-09-11T22:22:15.977515+0000","last_peered":"2024-09-11T22:22:15.977515+0000","last_clean":"2024-09-11T22:22:15.977515+0000","last_became_active":"2024-09-11T22:20:18.186770+0000","last_became_peered":"2024-09-11T22:20:18.186770+0000","last_unstale":"2024-09-11T22:22:15.977515+0000","last_undegraded":"2024-09-11T22:22:15.977515+0000","last_fullsized":"2024-09-11T22:22:15.977515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:59.917695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234743+0000","last_change":"2024-09-11T22:20:18.197137+0000","last_active":"2024-09-11T22:22:16.234743+0000","last_peered":"2024-09-11T22:22:16.234743+0000","last_clean":"2024-09-11T22:22:16.234743+0000","last_became_active":"2024-09-11T22:20:18.192810+0000","last_became_peered":"2024-09-11T22:20:18.192810+0000","last_unstale":"2024-09-11T22:22:16.234743+0000","last_undegraded":"2024-09-11T22:22:16.234743+0000","last_fullsized":"2024-09-11T22:22:16.234743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:57.680726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230830+0000","last_change":"2024-09-11T22:20:18.187568+0000","last_active":"2024-09-11T22:22:16.230830+0000","last_peered":"2024-09-11T22:22:16.230830+0000","last_clean":"2024-09-11T22:22:16.230830+0000","last_became_active":"2024-09-11T22:20:18.187254+0000","last_became_peered":"2024-09-11T22:20:18.187254+0000","last_unstale":"2024-09-11T22:22:16.230830+0000","last_undegraded":"2024-09-11T22:22:16.230830+0000","last_fullsized":"2024-09-11T22:22:16.230830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:41.724926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975378+0000","last_change":"2024-09-11T22:20:18.190376+0000","last_active":"2024-09-11T22:22:15.975378+0000","last_peered":"2024-09-11T22:22:15.975378+0000","last_clean":"2024-09-11T22:22:15.975378+0000","last_became_active":"2024-09-11T22:20:18.190079+0000","last_became_peered":"2024-09-11T22:20:18.190079+0000","last_unstale":"2024-09-11T22:22:15.975378+0000","last_undegraded":"2024-09-11T22:22:15.975378+0000","last_fullsized":"2024-09-11T22:22:15.975378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:42.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058896+0000","last_change":"2024-09-11T22:20:18.188503+0000","last_active":"2024-09-11T22:22:16.058896+0000","last_peered":"2024-09-11T22:22:16.058896+0000","last_clean":"2024-09-11T22:22:16.058896+0000","last_became_active":"2024-09-11T22:20:18.187957+0000","last_became_peered":"2024-09-11T22:20:18.187957+0000","last_unstale":"2024-09-11T22:22:16.058896+0000","last_undegraded":"2024-09-11T22:22:16.058896+0000","last_fullsized":"2024-09-11T22:22:16.058896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:44:25.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065572+0000","last_change":"2024-09-11T22:20:18.196936+0000","last_active":"2024-09-11T22:22:16.065572+0000","last_peered":"2024-09-11T22:22:16.065572+0000","last_clean":"2024-09-11T22:22:16.065572+0000","last_became_active":"2024-09-11T22:20:18.196493+0000","last_became_peered":"2024-09-11T22:20:18.196493+0000","last_unstale":"2024-09-11T22:22:16.065572+0000","last_undegraded":"2024-09-11T22:22:16.065572+0000","last_fullsized":"2024-09-11T22:22:16.065572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:06.189806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059126+0000","last_change":"2024-09-11T22:20:18.195260+0000","last_active":"2024-09-11T22:22:16.059126+0000","last_peered":"2024-09-11T22:22:16.059126+0000","last_clean":"2024-09-11T22:22:16.059126+0000","last_became_active":"2024-09-11T22:20:18.194956+0000","last_became_peered":"2024-09-11T22:20:18.194956+0000","last_unstale":"2024-09-11T22:22:16.059126+0000","last_undegraded":"2024-09-11T22:22:16.059126+0000","last_fullsized":"2024-09-11T22:22:16.059126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:34.483291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977695+0000","last_change":"2024-09-11T22:20:18.190591+0000","last_active":"2024-09-11T22:22:15.977695+0000","last_peered":"2024-09-11T22:22:15.977695+0000","last_clean":"2024-09-11T22:22:15.977695+0000","last_became_active":"2024-09-11T22:20:18.190007+0000","last_became_peered":"2024-09-11T22:20:18.190007+0000","last_unstale":"2024-09-11T22:22:15.977695+0000","last_undegraded":"2024-09-11T22:22:15.977695+0000","last_fullsized":"2024-09-11T22:22:15.977695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:46.281705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234929+0000","last_change":"2024-09-11T22:20:18.193377+0000","last_active":"2024-09-11T22:22:16.234929+0000","last_peered":"2024-09-11T22:22:16.234929+0000","last_clean":"2024-09-11T22:22:16.234929+0000","last_became_active":"2024-09-11T22:20:18.193003+0000","last_became_peered":"2024-09-11T22:20:18.193003+0000","last_unstale":"2024-09-11T22:22:16.234929+0000","last_undegraded":"2024-09-11T22:22:16.234929+0000","last_fullsized":"2024-09-11T22:22:16.234929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:59.052966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230737+0000","last_change":"2024-09-11T22:20:18.189405+0000","last_active":"2024-09-11T22:22:16.230737+0000","last_peered":"2024-09-11T22:22:16.230737+0000","last_clean":"2024-09-11T22:22:16.230737+0000","last_became_active":"2024-09-11T22:20:18.189098+0000","last_became_peered":"2024-09-11T22:20:18.189098+0000","last_unstale":"2024-09-11T22:22:16.230737+0000","last_undegraded":"2024-09-11T22:22:16.230737+0000","last_fullsized":"2024-09-11T22:22:16.230737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:47.088192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233832+0000","last_change":"2024-09-11T22:20:18.191005+0000","last_active":"2024-09-11T22:22:16.233832+0000","last_peered":"2024-09-11T22:22:16.233832+0000","last_clean":"2024-09-11T22:22:16.233832+0000","last_became_active":"2024-09-11T22:20:18.190681+0000","last_became_peered":"2024-09-11T22:20:18.190681+0000","last_unstale":"2024-09-11T22:22:16.233832+0000","last_undegraded":"2024-09-11T22:22:16.233832+0000","last_fullsized":"2024-09-11T22:22:16.233832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:12.903929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977210+0000","last_change":"2024-09-11T22:20:18.187920+0000","last_active":"2024-09-11T22:22:15.977210+0000","last_peered":"2024-09-11T22:22:15.977210+0000","last_clean":"2024-09-11T22:22:15.977210+0000","last_became_active":"2024-09-11T22:20:18.187526+0000","last_became_peered":"2024-09-11T22:20:18.187526+0000","last_unstale":"2024-09-11T22:22:15.977210+0000","last_undegraded":"2024-09-11T22:22:15.977210+0000","last_fullsized":"2024-09-11T22:22:15.977210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.347857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977399+0000","last_change":"2024-09-11T22:20:18.193869+0000","last_active":"2024-09-11T22:22:15.977399+0000","last_peered":"2024-09-11T22:22:15.977399+0000","last_clean":"2024-09-11T22:22:15.977399+0000","last_became_active":"2024-09-11T22:20:18.193468+0000","last_became_peered":"2024-09-11T22:20:18.193468+0000","last_unstale":"2024-09-11T22:22:15.977399+0000","last_undegraded":"2024-09-11T22:22:15.977399+0000","last_fullsized":"2024-09-11T22:22:15.977399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:59.619611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234817+0000","last_change":"2024-09-11T22:20:18.193947+0000","last_active":"2024-09-11T22:22:16.234817+0000","last_peered":"2024-09-11T22:22:16.234817+0000","last_clean":"2024-09-11T22:22:16.234817+0000","last_became_active":"2024-09-11T22:20:18.192257+0000","last_became_peered":"2024-09-11T22:20:18.192257+0000","last_unstale":"2024-09-11T22:22:16.234817+0000","last_undegraded":"2024-09-11T22:22:16.234817+0000","last_fullsized":"2024-09-11T22:22:16.234817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.478730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058805+0000","last_change":"2024-09-11T22:20:18.184549+0000","last_active":"2024-09-11T22:22:16.058805+0000","last_peered":"2024-09-11T22:22:16.058805+0000","last_clean":"2024-09-11T22:22:16.058805+0000","last_became_active":"2024-09-11T22:20:18.184075+0000","last_became_peered":"2024-09-11T22:20:18.184075+0000","last_unstale":"2024-09-11T22:22:16.058805+0000","last_undegraded":"2024-09-11T22:22:16.058805+0000","last_fullsized":"2024-09-11T22:22:16.058805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:50.095643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.415258+0000","last_change":"2024-09-11T22:20:16.128524+0000","last_active":"2024-09-11T22:22:16.415258+0000","last_peered":"2024-09-11T22:22:16.415258+0000","last_clean":"2024-09-11T22:22:16.415258+0000","last_became_active":"2024-09-11T22:20:16.128185+0000","last_became_peered":"2024-09-11T22:20:16.128185+0000","last_unstale":"2024-09-11T22:22:16.415258+0000","last_undegraded":"2024-09-11T22:22:16.415258+0000","last_fullsized":"2024-09-11T22:22:16.415258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.022931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065720+0000","last_change":"2024-09-11T22:20:18.191578+0000","last_active":"2024-09-11T22:22:16.065720+0000","last_peered":"2024-09-11T22:22:16.065720+0000","last_clean":"2024-09-11T22:22:16.065720+0000","last_became_active":"2024-09-11T22:20:18.189568+0000","last_became_peered":"2024-09-11T22:20:18.189568+0000","last_unstale":"2024-09-11T22:22:16.065720+0000","last_undegraded":"2024-09-11T22:22:16.065720+0000","last_fullsized":"2024-09-11T22:22:16.065720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:41.737242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068742+0000","last_change":"2024-09-11T22:21:18.199693+0000","last_active":"2024-09-11T22:22:16.068742+0000","last_peered":"2024-09-11T22:22:16.068742+0000","last_clean":"2024-09-11T22:22:16.068742+0000","last_became_active":"2024-09-11T22:21:18.190954+0000","last_became_peered":"2024-09-11T22:21:18.190954+0000","last_unstale":"2024-09-11T22:22:16.068742+0000","last_undegraded":"2024-09-11T22:22:16.068742+0000","last_fullsized":"2024-09-11T22:22:16.068742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:11.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230688+0000","last_change":"2024-09-11T22:20:16.115602+0000","last_active":"2024-09-11T22:22:16.230688+0000","last_peered":"2024-09-11T22:22:16.230688+0000","last_clean":"2024-09-11T22:22:16.230688+0000","last_became_active":"2024-09-11T22:20:16.115280+0000","last_became_peered":"2024-09-11T22:20:16.115280+0000","last_unstale":"2024-09-11T22:22:16.230688+0000","last_undegraded":"2024-09-11T22:22:16.230688+0000","last_fullsized":"2024-09-11T22:22:16.230688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:41.942137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058816+0000","last_change":"2024-09-11T22:20:18.184125+0000","last_active":"2024-09-11T22:22:16.058816+0000","last_peered":"2024-09-11T22:22:16.058816+0000","last_clean":"2024-09-11T22:22:16.058816+0000","last_became_active":"2024-09-11T22:20:18.183785+0000","last_became_peered":"2024-09-11T22:20:18.183785+0000","last_unstale":"2024-09-11T22:22:16.058816+0000","last_undegraded":"2024-09-11T22:22:16.058816+0000","last_fullsized":"2024-09-11T22:22:16.058816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:33.230175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065780+0000","last_change":"2024-09-11T22:20:16.125852+0000","last_active":"2024-09-11T22:22:16.065780+0000","last_peered":"2024-09-11T22:22:16.065780+0000","last_clean":"2024-09-11T22:22:16.065780+0000","last_became_active":"2024-09-11T22:20:16.125250+0000","last_became_peered":"2024-09-11T22:20:16.125250+0000","last_unstale":"2024-09-11T22:22:16.065780+0000","last_undegraded":"2024-09-11T22:22:16.065780+0000","last_fullsized":"2024-09-11T22:22:16.065780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:34.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973235+0000","last_change":"2024-09-11T22:20:18.185147+0000","last_active":"2024-09-11T22:22:15.973235+0000","last_peered":"2024-09-11T22:22:15.973235+0000","last_clean":"2024-09-11T22:22:15.973235+0000","last_became_active":"2024-09-11T22:20:18.184840+0000","last_became_peered":"2024-09-11T22:20:18.184840+0000","last_unstale":"2024-09-11T22:22:15.973235+0000","last_undegraded":"2024-09-11T22:22:15.973235+0000","last_fullsized":"2024-09-11T22:22:15.973235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:59.703349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":243,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.277347+0000","last_change":"2024-09-11T22:20:16.122173+0000","last_active":"2024-09-11T22:22:16.277347+0000","last_peered":"2024-09-11T22:22:16.277347+0000","last_clean":"2024-09-11T22:22:16.277347+0000","last_became_active":"2024-09-11T22:20:16.121847+0000","last_became_peered":"2024-09-11T22:20:16.121847+0000","last_unstale":"2024-09-11T22:22:16.277347+0000","last_undegraded":"2024-09-11T22:22:16.277347+0000","last_fullsized":"2024-09-11T22:22:16.277347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:41.284557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":213,"num_read_kb":213,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975691+0000","last_change":"2024-09-11T22:20:18.188030+0000","last_active":"2024-09-11T22:22:15.975691+0000","last_peered":"2024-09-11T22:22:15.975691+0000","last_clean":"2024-09-11T22:22:15.975691+0000","last_became_active":"2024-09-11T22:20:18.187503+0000","last_became_peered":"2024-09-11T22:20:18.187503+0000","last_unstale":"2024-09-11T22:22:15.975691+0000","last_undegraded":"2024-09-11T22:22:15.975691+0000","last_fullsized":"2024-09-11T22:22:15.975691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:45.494149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231598+0000","last_change":"2024-09-11T22:20:16.128828+0000","last_active":"2024-09-11T22:22:16.231598+0000","last_peered":"2024-09-11T22:22:16.231598+0000","last_clean":"2024-09-11T22:22:16.231598+0000","last_became_active":"2024-09-11T22:20:16.128454+0000","last_became_peered":"2024-09-11T22:20:16.128454+0000","last_unstale":"2024-09-11T22:22:16.231598+0000","last_undegraded":"2024-09-11T22:22:16.231598+0000","last_fullsized":"2024-09-11T22:22:16.231598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:33.927564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234475+0000","last_change":"2024-09-11T22:20:18.193795+0000","last_active":"2024-09-11T22:22:16.234475+0000","last_peered":"2024-09-11T22:22:16.234475+0000","last_clean":"2024-09-11T22:22:16.234475+0000","last_became_active":"2024-09-11T22:20:18.193489+0000","last_became_peered":"2024-09-11T22:20:18.193489+0000","last_unstale":"2024-09-11T22:22:16.234475+0000","last_undegraded":"2024-09-11T22:22:16.234475+0000","last_fullsized":"2024-09-11T22:22:16.234475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:14.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065914+0000","last_change":"2024-09-11T22:20:16.187938+0000","last_active":"2024-09-11T22:22:16.065914+0000","last_peered":"2024-09-11T22:22:16.065914+0000","last_clean":"2024-09-11T22:22:16.065914+0000","last_became_active":"2024-09-11T22:20:16.187545+0000","last_became_peered":"2024-09-11T22:20:16.187545+0000","last_unstale":"2024-09-11T22:22:16.065914+0000","last_undegraded":"2024-09-11T22:22:16.065914+0000","last_fullsized":"2024-09-11T22:22:16.065914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:41.674356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975385+0000","last_change":"2024-09-11T22:21:18.191937+0000","last_active":"2024-09-11T22:22:15.975385+0000","last_peered":"2024-09-11T22:22:15.975385+0000","last_clean":"2024-09-11T22:22:15.975385+0000","last_became_active":"2024-09-11T22:21:18.191218+0000","last_became_peered":"2024-09-11T22:21:18.191218+0000","last_unstale":"2024-09-11T22:22:15.975385+0000","last_undegraded":"2024-09-11T22:22:15.975385+0000","last_fullsized":"2024-09-11T22:22:15.975385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:41.654630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'37","reported_seq":287,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.414971+0000","last_change":"2024-09-11T22:20:16.128082+0000","last_active":"2024-09-11T22:22:16.414971+0000","last_peered":"2024-09-11T22:22:16.414971+0000","last_clean":"2024-09-11T22:22:16.414971+0000","last_became_active":"2024-09-11T22:20:16.127731+0000","last_became_peered":"2024-09-11T22:20:16.127731+0000","last_unstale":"2024-09-11T22:22:16.414971+0000","last_undegraded":"2024-09-11T22:22:16.414971+0000","last_fullsized":"2024-09-11T22:22:16.414971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:33.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66480,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":226,"num_read_kb":178,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059064+0000","last_change":"2024-09-11T22:20:18.189219+0000","last_active":"2024-09-11T22:22:16.059064+0000","last_peered":"2024-09-11T22:22:16.059064+0000","last_clean":"2024-09-11T22:22:16.059064+0000","last_became_active":"2024-09-11T22:20:18.188687+0000","last_became_peered":"2024-09-11T22:20:18.188687+0000","last_unstale":"2024-09-11T22:22:16.059064+0000","last_undegraded":"2024-09-11T22:22:16.059064+0000","last_fullsized":"2024-09-11T22:22:16.059064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:24.171126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977378+0000","last_change":"2024-09-11T22:20:16.129724+0000","last_active":"2024-09-11T22:22:15.977378+0000","last_peered":"2024-09-11T22:22:15.977378+0000","last_clean":"2024-09-11T22:22:15.977378+0000","last_became_active":"2024-09-11T22:20:16.129351+0000","last_became_peered":"2024-09-11T22:20:16.129351+0000","last_unstale":"2024-09-11T22:22:15.977378+0000","last_undegraded":"2024-09-11T22:22:15.977378+0000","last_fullsized":"2024-09-11T22:22:15.977378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:09.642520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230435+0000","last_change":"2024-09-11T22:20:18.187031+0000","last_active":"2024-09-11T22:22:16.230435+0000","last_peered":"2024-09-11T22:22:16.230435+0000","last_clean":"2024-09-11T22:22:16.230435+0000","last_became_active":"2024-09-11T22:20:18.186698+0000","last_became_peered":"2024-09-11T22:20:18.186698+0000","last_unstale":"2024-09-11T22:22:16.230435+0000","last_undegraded":"2024-09-11T22:22:16.230435+0000","last_fullsized":"2024-09-11T22:22:16.230435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:04.810981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065725+0000","last_change":"2024-09-11T22:20:16.124925+0000","last_active":"2024-09-11T22:22:16.065725+0000","last_peered":"2024-09-11T22:22:16.065725+0000","last_clean":"2024-09-11T22:22:16.065725+0000","last_became_active":"2024-09-11T22:20:16.124580+0000","last_became_peered":"2024-09-11T22:20:16.124580+0000","last_unstale":"2024-09-11T22:22:16.065725+0000","last_undegraded":"2024-09-11T22:22:16.065725+0000","last_fullsized":"2024-09-11T22:22:16.065725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:56.417577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231838+0000","last_change":"2024-09-11T22:21:18.187783+0000","last_active":"2024-09-11T22:22:16.231838+0000","last_peered":"2024-09-11T22:22:16.231838+0000","last_clean":"2024-09-11T22:22:16.231838+0000","last_became_active":"2024-09-11T22:21:18.187358+0000","last_became_peered":"2024-09-11T22:21:18.187358+0000","last_unstale":"2024-09-11T22:22:16.231838+0000","last_undegraded":"2024-09-11T22:22:16.231838+0000","last_fullsized":"2024-09-11T22:22:16.231838+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:41.251531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.439108+0000","last_change":"2024-09-11T22:20:16.118934+0000","last_active":"2024-09-11T22:22:16.439108+0000","last_peered":"2024-09-11T22:22:16.439108+0000","last_clean":"2024-09-11T22:22:16.439108+0000","last_became_active":"2024-09-11T22:20:16.118614+0000","last_became_peered":"2024-09-11T22:20:16.118614+0000","last_unstale":"2024-09-11T22:22:16.439108+0000","last_undegraded":"2024-09-11T22:22:16.439108+0000","last_fullsized":"2024-09-11T22:22:16.439108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:05.791161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231721+0000","last_change":"2024-09-11T22:20:18.197692+0000","last_active":"2024-09-11T22:22:16.231721+0000","last_peered":"2024-09-11T22:22:16.231721+0000","last_clean":"2024-09-11T22:22:16.231721+0000","last_became_active":"2024-09-11T22:20:18.197336+0000","last_became_peered":"2024-09-11T22:20:18.197336+0000","last_unstale":"2024-09-11T22:22:16.231721+0000","last_undegraded":"2024-09-11T22:22:16.231721+0000","last_fullsized":"2024-09-11T22:22:16.231721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:55.172568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:16.186769+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:16.186367+0000","last_became_peered":"2024-09-11T22:20:16.186367+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.279387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065641+0000","last_change":"2024-09-11T22:20:18.195570+0000","last_active":"2024-09-11T22:22:16.065641+0000","last_peered":"2024-09-11T22:22:16.065641+0000","last_clean":"2024-09-11T22:22:16.065641+0000","last_became_active":"2024-09-11T22:20:18.195226+0000","last_became_peered":"2024-09-11T22:20:18.195226+0000","last_unstale":"2024-09-11T22:22:16.065641+0000","last_undegraded":"2024-09-11T22:22:16.065641+0000","last_fullsized":"2024-09-11T22:22:16.065641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:44.108090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235034+0000","last_change":"2024-09-11T22:21:18.194553+0000","last_active":"2024-09-11T22:22:16.235034+0000","last_peered":"2024-09-11T22:22:16.235034+0000","last_clean":"2024-09-11T22:22:16.235034+0000","last_became_active":"2024-09-11T22:21:18.188635+0000","last_became_peered":"2024-09-11T22:21:18.188635+0000","last_unstale":"2024-09-11T22:22:16.235034+0000","last_undegraded":"2024-09-11T22:22:16.235034+0000","last_fullsized":"2024-09-11T22:22:16.235034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:34.165195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234954+0000","last_change":"2024-09-11T22:20:18.194112+0000","last_active":"2024-09-11T22:22:16.234954+0000","last_peered":"2024-09-11T22:22:16.234954+0000","last_clean":"2024-09-11T22:22:16.234954+0000","last_became_active":"2024-09-11T22:20:18.193812+0000","last_became_peered":"2024-09-11T22:20:18.193812+0000","last_unstale":"2024-09-11T22:22:16.234954+0000","last_undegraded":"2024-09-11T22:22:16.234954+0000","last_fullsized":"2024-09-11T22:22:16.234954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:35.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231070+0000","last_change":"2024-09-11T22:21:18.198325+0000","last_active":"2024-09-11T22:22:16.231070+0000","last_peered":"2024-09-11T22:22:16.231070+0000","last_clean":"2024-09-11T22:22:16.231070+0000","last_became_active":"2024-09-11T22:21:18.193651+0000","last_became_peered":"2024-09-11T22:21:18.193651+0000","last_unstale":"2024-09-11T22:22:16.231070+0000","last_undegraded":"2024-09-11T22:22:16.231070+0000","last_fullsized":"2024-09-11T22:22:16.231070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:42.552492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232027+0000","last_change":"2024-09-11T22:20:18.191960+0000","last_active":"2024-09-11T22:22:16.232027+0000","last_peered":"2024-09-11T22:22:16.232027+0000","last_clean":"2024-09-11T22:22:16.232027+0000","last_became_active":"2024-09-11T22:20:18.191635+0000","last_became_peered":"2024-09-11T22:20:18.191635+0000","last_unstale":"2024-09-11T22:22:16.232027+0000","last_undegraded":"2024-09-11T22:22:16.232027+0000","last_fullsized":"2024-09-11T22:22:16.232027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:19:18.919247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.176627+0000","last_change":"2024-09-11T22:20:16.123608+0000","last_active":"2024-09-11T22:22:16.176627+0000","last_peered":"2024-09-11T22:22:16.176627+0000","last_clean":"2024-09-11T22:22:16.176627+0000","last_became_active":"2024-09-11T22:20:16.123193+0000","last_became_peered":"2024-09-11T22:20:16.123193+0000","last_unstale":"2024-09-11T22:22:16.176627+0000","last_undegraded":"2024-09-11T22:22:16.176627+0000","last_fullsized":"2024-09-11T22:22:16.176627+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:07.362115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231887+0000","last_change":"2024-09-11T22:20:18.199957+0000","last_active":"2024-09-11T22:22:16.231887+0000","last_peered":"2024-09-11T22:22:16.231887+0000","last_clean":"2024-09-11T22:22:16.231887+0000","last_became_active":"2024-09-11T22:20:18.199637+0000","last_became_peered":"2024-09-11T22:20:18.199637+0000","last_unstale":"2024-09-11T22:22:16.231887+0000","last_undegraded":"2024-09-11T22:22:16.231887+0000","last_fullsized":"2024-09-11T22:22:16.231887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:14.949002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.277747+0000","last_change":"2024-09-11T22:21:18.198896+0000","last_active":"2024-09-11T22:22:16.277747+0000","last_peered":"2024-09-11T22:22:16.277747+0000","last_clean":"2024-09-11T22:22:16.277747+0000","last_became_active":"2024-09-11T22:21:18.189489+0000","last_became_peered":"2024-09-11T22:21:18.189489+0000","last_unstale":"2024-09-11T22:22:16.277747+0000","last_undegraded":"2024-09-11T22:22:16.277747+0000","last_fullsized":"2024-09-11T22:22:16.277747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:19.908801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232111+0000","last_change":"2024-09-11T22:20:18.196691+0000","last_active":"2024-09-11T22:22:16.232111+0000","last_peered":"2024-09-11T22:22:16.232111+0000","last_clean":"2024-09-11T22:22:16.232111+0000","last_became_active":"2024-09-11T22:20:18.196364+0000","last_became_peered":"2024-09-11T22:20:18.196364+0000","last_unstale":"2024-09-11T22:22:16.232111+0000","last_undegraded":"2024-09-11T22:22:16.232111+0000","last_fullsized":"2024-09-11T22:22:16.232111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:50.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231876+0000","last_change":"2024-09-11T22:19:02.590079+0000","last_active":"2024-09-11T22:22:16.231876+0000","last_peered":"2024-09-11T22:22:16.231876+0000","last_clean":"2024-09-11T22:22:16.231876+0000","last_became_active":"2024-09-11T22:19:02.589644+0000","last_became_peered":"2024-09-11T22:19:02.589644+0000","last_unstale":"2024-09-11T22:22:16.231876+0000","last_undegraded":"2024-09-11T22:22:16.231876+0000","last_fullsized":"2024-09-11T22:22:16.231876+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_clean_scrub_stamp":"2024-09-11T22:15:33.244685+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:45.506190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.229755+0000","last_change":"2024-09-11T22:20:16.116661+0000","last_active":"2024-09-11T22:22:16.229755+0000","last_peered":"2024-09-11T22:22:16.229755+0000","last_clean":"2024-09-11T22:22:16.229755+0000","last_became_active":"2024-09-11T22:20:16.116196+0000","last_became_peered":"2024-09-11T22:20:16.116196+0000","last_unstale":"2024-09-11T22:22:16.229755+0000","last_undegraded":"2024-09-11T22:22:16.229755+0000","last_fullsized":"2024-09-11T22:22:16.229755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:08.875170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059167+0000","last_change":"2024-09-11T22:20:18.193176+0000","last_active":"2024-09-11T22:22:16.059167+0000","last_peered":"2024-09-11T22:22:16.059167+0000","last_clean":"2024-09-11T22:22:16.059167+0000","last_became_active":"2024-09-11T22:20:18.192312+0000","last_became_peered":"2024-09-11T22:20:18.192312+0000","last_unstale":"2024-09-11T22:22:16.059167+0000","last_undegraded":"2024-09-11T22:22:16.059167+0000","last_fullsized":"2024-09-11T22:22:16.059167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:08.738569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234694+0000","last_change":"2024-09-11T22:21:18.196985+0000","last_active":"2024-09-11T22:22:16.234694+0000","last_peered":"2024-09-11T22:22:16.234694+0000","last_clean":"2024-09-11T22:22:16.234694+0000","last_became_active":"2024-09-11T22:21:18.189487+0000","last_became_peered":"2024-09-11T22:21:18.189487+0000","last_unstale":"2024-09-11T22:22:16.234694+0000","last_undegraded":"2024-09-11T22:22:16.234694+0000","last_fullsized":"2024-09-11T22:22:16.234694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:20.715696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230470+0000","last_change":"2024-09-11T22:20:18.191118+0000","last_active":"2024-09-11T22:22:16.230470+0000","last_peered":"2024-09-11T22:22:16.230470+0000","last_clean":"2024-09-11T22:22:16.230470+0000","last_became_active":"2024-09-11T22:20:18.190760+0000","last_became_peered":"2024-09-11T22:20:18.190760+0000","last_unstale":"2024-09-11T22:22:16.230470+0000","last_undegraded":"2024-09-11T22:22:16.230470+0000","last_fullsized":"2024-09-11T22:22:16.230470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:59.220067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235387+0000","last_change":"2024-09-11T22:20:16.186407+0000","last_active":"2024-09-11T22:22:16.235387+0000","last_peered":"2024-09-11T22:22:16.235387+0000","last_clean":"2024-09-11T22:22:16.235387+0000","last_became_active":"2024-09-11T22:20:16.186105+0000","last_became_peered":"2024-09-11T22:20:16.186105+0000","last_unstale":"2024-09-11T22:22:16.235387+0000","last_undegraded":"2024-09-11T22:22:16.235387+0000","last_fullsized":"2024-09-11T22:22:16.235387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:43.942072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065626+0000","last_change":"2024-09-11T22:20:18.191970+0000","last_active":"2024-09-11T22:22:16.065626+0000","last_peered":"2024-09-11T22:22:16.065626+0000","last_clean":"2024-09-11T22:22:16.065626+0000","last_became_active":"2024-09-11T22:20:18.191623+0000","last_became_peered":"2024-09-11T22:20:18.191623+0000","last_unstale":"2024-09-11T22:22:16.065626+0000","last_undegraded":"2024-09-11T22:22:16.065626+0000","last_fullsized":"2024-09-11T22:22:16.065626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:33.213979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231514+0000","last_change":"2024-09-11T22:20:16.119936+0000","last_active":"2024-09-11T22:22:16.231514+0000","last_peered":"2024-09-11T22:22:16.231514+0000","last_clean":"2024-09-11T22:22:16.231514+0000","last_became_active":"2024-09-11T22:20:16.119453+0000","last_became_peered":"2024-09-11T22:20:16.119453+0000","last_unstale":"2024-09-11T22:22:16.231514+0000","last_undegraded":"2024-09-11T22:22:16.231514+0000","last_fullsized":"2024-09-11T22:22:16.231514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:20.717980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068790+0000","last_change":"2024-09-11T22:20:18.192581+0000","last_active":"2024-09-11T22:22:16.068790+0000","last_peered":"2024-09-11T22:22:16.068790+0000","last_clean":"2024-09-11T22:22:16.068790+0000","last_became_active":"2024-09-11T22:20:18.191702+0000","last_became_peered":"2024-09-11T22:20:18.191702+0000","last_unstale":"2024-09-11T22:22:16.068790+0000","last_undegraded":"2024-09-11T22:22:16.068790+0000","last_fullsized":"2024-09-11T22:22:16.068790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:48.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973402+0000","last_change":"2024-09-11T22:20:16.186485+0000","last_active":"2024-09-11T22:22:15.973402+0000","last_peered":"2024-09-11T22:22:15.973402+0000","last_clean":"2024-09-11T22:22:15.973402+0000","last_became_active":"2024-09-11T22:20:16.186171+0000","last_became_peered":"2024-09-11T22:20:16.186171+0000","last_unstale":"2024-09-11T22:22:15.973402+0000","last_undegraded":"2024-09-11T22:22:15.973402+0000","last_fullsized":"2024-09-11T22:22:15.973402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:26.373626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068876+0000","last_change":"2024-09-11T22:20:18.191361+0000","last_active":"2024-09-11T22:22:16.068876+0000","last_peered":"2024-09-11T22:22:16.068876+0000","last_clean":"2024-09-11T22:22:16.068876+0000","last_became_active":"2024-09-11T22:20:18.191036+0000","last_became_peered":"2024-09-11T22:20:18.191036+0000","last_unstale":"2024-09-11T22:22:16.068876+0000","last_undegraded":"2024-09-11T22:22:16.068876+0000","last_fullsized":"2024-09-11T22:22:16.068876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:29.124650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059181+0000","last_change":"2024-09-11T22:20:16.188156+0000","last_active":"2024-09-11T22:22:16.059181+0000","last_peered":"2024-09-11T22:22:16.059181+0000","last_clean":"2024-09-11T22:22:16.059181+0000","last_became_active":"2024-09-11T22:20:16.187809+0000","last_became_peered":"2024-09-11T22:20:16.187809+0000","last_unstale":"2024-09-11T22:22:16.059181+0000","last_undegraded":"2024-09-11T22:22:16.059181+0000","last_fullsized":"2024-09-11T22:22:16.059181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:04.219621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230896+0000","last_change":"2024-09-11T22:20:18.182380+0000","last_active":"2024-09-11T22:22:16.230896+0000","last_peered":"2024-09-11T22:22:16.230896+0000","last_clean":"2024-09-11T22:22:16.230896+0000","last_became_active":"2024-09-11T22:20:18.182062+0000","last_became_peered":"2024-09-11T22:20:18.182062+0000","last_unstale":"2024-09-11T22:22:16.230896+0000","last_undegraded":"2024-09-11T22:22:16.230896+0000","last_fullsized":"2024-09-11T22:22:16.230896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:15.575846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'50","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.179372+0000","last_change":"2024-09-11T22:20:16.117335+0000","last_active":"2024-09-11T22:22:16.179372+0000","last_peered":"2024-09-11T22:22:16.179372+0000","last_clean":"2024-09-11T22:22:16.179372+0000","last_became_active":"2024-09-11T22:20:16.116977+0000","last_became_peered":"2024-09-11T22:20:16.116977+0000","last_unstale":"2024-09-11T22:22:16.179372+0000","last_undegraded":"2024-09-11T22:22:16.179372+0000","last_fullsized":"2024-09-11T22:22:16.179372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:51.744202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059236+0000","last_change":"2024-09-11T22:20:18.194069+0000","last_active":"2024-09-11T22:22:16.059236+0000","last_peered":"2024-09-11T22:22:16.059236+0000","last_clean":"2024-09-11T22:22:16.059236+0000","last_became_active":"2024-09-11T22:20:18.193765+0000","last_became_peered":"2024-09-11T22:20:18.193765+0000","last_unstale":"2024-09-11T22:22:16.059236+0000","last_undegraded":"2024-09-11T22:22:16.059236+0000","last_fullsized":"2024-09-11T22:22:16.059236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:25.047405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230131+0000","last_change":"2024-09-11T22:20:16.460987+0000","last_active":"2024-09-11T22:22:16.230131+0000","last_peered":"2024-09-11T22:22:16.230131+0000","last_clean":"2024-09-11T22:22:16.230131+0000","last_became_active":"2024-09-11T22:20:16.460516+0000","last_became_peered":"2024-09-11T22:20:16.460516+0000","last_unstale":"2024-09-11T22:22:16.230131+0000","last_undegraded":"2024-09-11T22:22:16.230131+0000","last_fullsized":"2024-09-11T22:22:16.230131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:00.987981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234264+0000","last_change":"2024-09-11T22:21:18.188802+0000","last_active":"2024-09-11T22:22:16.234264+0000","last_peered":"2024-09-11T22:22:16.234264+0000","last_clean":"2024-09-11T22:22:16.234264+0000","last_became_active":"2024-09-11T22:21:18.188441+0000","last_became_peered":"2024-09-11T22:21:18.188441+0000","last_unstale":"2024-09-11T22:22:16.234264+0000","last_undegraded":"2024-09-11T22:22:16.234264+0000","last_fullsized":"2024-09-11T22:22:16.234264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:02.295441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975337+0000","last_change":"2024-09-11T22:20:16.186498+0000","last_active":"2024-09-11T22:22:15.975337+0000","last_peered":"2024-09-11T22:22:15.975337+0000","last_clean":"2024-09-11T22:22:15.975337+0000","last_became_active":"2024-09-11T22:20:16.186214+0000","last_became_peered":"2024-09-11T22:20:16.186214+0000","last_unstale":"2024-09-11T22:22:15.975337+0000","last_undegraded":"2024-09-11T22:22:15.975337+0000","last_fullsized":"2024-09-11T22:22:15.975337+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:09.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:18.187489+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:18.185649+0000","last_became_peered":"2024-09-11T22:20:18.185649+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:42.825075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059011+0000","last_change":"2024-09-11T22:20:16.117627+0000","last_active":"2024-09-11T22:22:16.059011+0000","last_peered":"2024-09-11T22:22:16.059011+0000","last_clean":"2024-09-11T22:22:16.059011+0000","last_became_active":"2024-09-11T22:20:16.117243+0000","last_became_peered":"2024-09-11T22:20:16.117243+0000","last_unstale":"2024-09-11T22:22:16.059011+0000","last_undegraded":"2024-09-11T22:22:16.059011+0000","last_fullsized":"2024-09-11T22:22:16.059011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:58.240535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234846+0000","last_change":"2024-09-11T22:20:18.184326+0000","last_active":"2024-09-11T22:22:16.234846+0000","last_peered":"2024-09-11T22:22:16.234846+0000","last_clean":"2024-09-11T22:22:16.234846+0000","last_became_active":"2024-09-11T22:20:18.183995+0000","last_became_peered":"2024-09-11T22:20:18.183995+0000","last_unstale":"2024-09-11T22:22:16.234846+0000","last_undegraded":"2024-09-11T22:22:16.234846+0000","last_fullsized":"2024-09-11T22:22:16.234846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:33.468000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977256+0000","last_change":"2024-09-11T22:20:16.127180+0000","last_active":"2024-09-11T22:22:15.977256+0000","last_peered":"2024-09-11T22:22:15.977256+0000","last_clean":"2024-09-11T22:22:15.977256+0000","last_became_active":"2024-09-11T22:20:16.126696+0000","last_became_peered":"2024-09-11T22:20:16.126696+0000","last_unstale":"2024-09-11T22:22:15.977256+0000","last_undegraded":"2024-09-11T22:22:15.977256+0000","last_fullsized":"2024-09-11T22:22:15.977256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:02.637225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973369+0000","last_change":"2024-09-11T22:21:18.188614+0000","last_active":"2024-09-11T22:22:15.973369+0000","last_peered":"2024-09-11T22:22:15.973369+0000","last_clean":"2024-09-11T22:22:15.973369+0000","last_became_active":"2024-09-11T22:21:18.188277+0000","last_became_peered":"2024-09-11T22:21:18.188277+0000","last_unstale":"2024-09-11T22:22:15.973369+0000","last_undegraded":"2024-09-11T22:22:15.973369+0000","last_fullsized":"2024-09-11T22:22:15.973369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:59.995647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058897+0000","last_change":"2024-09-11T22:20:16.128311+0000","last_active":"2024-09-11T22:22:16.058897+0000","last_peered":"2024-09-11T22:22:16.058897+0000","last_clean":"2024-09-11T22:22:16.058897+0000","last_became_active":"2024-09-11T22:20:16.127853+0000","last_became_peered":"2024-09-11T22:20:16.127853+0000","last_unstale":"2024-09-11T22:22:16.058897+0000","last_undegraded":"2024-09-11T22:22:16.058897+0000","last_fullsized":"2024-09-11T22:22:16.058897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:57.562470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231820+0000","last_change":"2024-09-11T22:20:18.195782+0000","last_active":"2024-09-11T22:22:16.231820+0000","last_peered":"2024-09-11T22:22:16.231820+0000","last_clean":"2024-09-11T22:22:16.231820+0000","last_became_active":"2024-09-11T22:20:18.195365+0000","last_became_peered":"2024-09-11T22:20:18.195365+0000","last_unstale":"2024-09-11T22:22:16.231820+0000","last_undegraded":"2024-09-11T22:22:16.231820+0000","last_fullsized":"2024-09-11T22:22:16.231820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:49.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065812+0000","last_change":"2024-09-11T22:20:18.199344+0000","last_active":"2024-09-11T22:22:16.065812+0000","last_peered":"2024-09-11T22:22:16.065812+0000","last_clean":"2024-09-11T22:22:16.065812+0000","last_became_active":"2024-09-11T22:20:18.199016+0000","last_became_peered":"2024-09-11T22:20:18.199016+0000","last_unstale":"2024-09-11T22:22:16.065812+0000","last_undegraded":"2024-09-11T22:22:16.065812+0000","last_fullsized":"2024-09-11T22:22:16.065812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:38.322831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059125+0000","last_change":"2024-09-11T22:20:16.122236+0000","last_active":"2024-09-11T22:22:16.059125+0000","last_peered":"2024-09-11T22:22:16.059125+0000","last_clean":"2024-09-11T22:22:16.059125+0000","last_became_active":"2024-09-11T22:20:16.121867+0000","last_became_peered":"2024-09-11T22:20:16.121867+0000","last_unstale":"2024-09-11T22:22:16.059125+0000","last_undegraded":"2024-09-11T22:22:16.059125+0000","last_fullsized":"2024-09-11T22:22:16.059125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:56.782445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975374+0000","last_change":"2024-09-11T22:20:18.187338+0000","last_active":"2024-09-11T22:22:15.975374+0000","last_peered":"2024-09-11T22:22:15.975374+0000","last_clean":"2024-09-11T22:22:15.975374+0000","last_became_active":"2024-09-11T22:20:18.186985+0000","last_became_peered":"2024-09-11T22:20:18.186985+0000","last_unstale":"2024-09-11T22:22:15.975374+0000","last_undegraded":"2024-09-11T22:22:15.975374+0000","last_fullsized":"2024-09-11T22:22:15.975374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:58.170680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059106+0000","last_change":"2024-09-11T22:20:16.128579+0000","last_active":"2024-09-11T22:22:16.059106+0000","last_peered":"2024-09-11T22:22:16.059106+0000","last_clean":"2024-09-11T22:22:16.059106+0000","last_became_active":"2024-09-11T22:20:16.127864+0000","last_became_peered":"2024-09-11T22:20:16.127864+0000","last_unstale":"2024-09-11T22:22:16.059106+0000","last_undegraded":"2024-09-11T22:22:16.059106+0000","last_fullsized":"2024-09-11T22:22:16.059106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:11:32.005487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234449+0000","last_change":"2024-09-11T22:20:18.185205+0000","last_active":"2024-09-11T22:22:16.234449+0000","last_peered":"2024-09-11T22:22:16.234449+0000","last_clean":"2024-09-11T22:22:16.234449+0000","last_became_active":"2024-09-11T22:20:18.184402+0000","last_became_peered":"2024-09-11T22:20:18.184402+0000","last_unstale":"2024-09-11T22:22:16.234449+0000","last_undegraded":"2024-09-11T22:22:16.234449+0000","last_fullsized":"2024-09-11T22:22:16.234449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:17:48.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.060208+0000","last_change":"2024-09-11T22:20:16.186372+0000","last_active":"2024-09-11T22:22:16.060208+0000","last_peered":"2024-09-11T22:22:16.060208+0000","last_clean":"2024-09-11T22:22:16.060208+0000","last_became_active":"2024-09-11T22:20:16.185990+0000","last_became_peered":"2024-09-11T22:20:16.185990+0000","last_unstale":"2024-09-11T22:22:16.060208+0000","last_undegraded":"2024-09-11T22:22:16.060208+0000","last_fullsized":"2024-09-11T22:22:16.060208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:23.296379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059971+0000","last_change":"2024-09-11T22:20:18.190670+0000","last_active":"2024-09-11T22:22:16.059971+0000","last_peered":"2024-09-11T22:22:16.059971+0000","last_clean":"2024-09-11T22:22:16.059971+0000","last_became_active":"2024-09-11T22:20:18.190339+0000","last_became_peered":"2024-09-11T22:20:18.190339+0000","last_unstale":"2024-09-11T22:22:16.059971+0000","last_undegraded":"2024-09-11T22:22:16.059971+0000","last_fullsized":"2024-09-11T22:22:16.059971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.085305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973670+0000","last_change":"2024-09-11T22:20:16.117646+0000","last_active":"2024-09-11T22:22:15.973670+0000","last_peered":"2024-09-11T22:22:15.973670+0000","last_clean":"2024-09-11T22:22:15.973670+0000","last_became_active":"2024-09-11T22:20:16.117111+0000","last_became_peered":"2024-09-11T22:20:16.117111+0000","last_unstale":"2024-09-11T22:22:15.973670+0000","last_undegraded":"2024-09-11T22:22:15.973670+0000","last_fullsized":"2024-09-11T22:22:15.973670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:36.470158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600132,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1158,"num_read_kb":717,"num_write":372,"num_write_kb":1463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1192,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1800396,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108858,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":443740,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1085440,"data_stored":1226275,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":443255,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201332,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":836827,"data_compressed":27823,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":983040,"data_stored":1037781,"data_compressed":44963,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443506,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":1167739,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":445182,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":433812,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":991232,"data_stored":503564,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":987136,"data_stored":510899,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":937984,"data_stored":444704,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135121,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200602,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:22:21.264 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-11T22:22:21.264 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T22:22:21.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:21 smithi159 ceph-mon[35534]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:21.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:21 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:21 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:21 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:21 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:21.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:21 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:21.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:21 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:21.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:21 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:21 smithi159 ceph-mon[35534]: mgrmap e23: x(active, since 5s), standbys: z, y 2024-09-11T22:22:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:21 smithi172 ceph-mon[35461]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:21 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:21 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:21 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:21 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:21 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:21 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:21 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:21 smithi172 ceph-mon[35461]: mgrmap e23: x(active, since 5s), standbys: z, y 2024-09-11T22:22:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.400+0000 7f4b99aa4640 1 -- 172.21.15.5:0/1554260333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9410c0e0 msgr2=0x7f4b9410e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.400+0000 7f4b99aa4640 1 --2- 172.21.15.5:0/1554260333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9410c0e0 0x7f4b9410e560 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009ca0 tx=0x7f4b8c02a950 comp rx=0 tx=0).stop 2024-09-11T22:22:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.400+0000 7f4b99aa4640 1 -- 172.21.15.5:0/1554260333 shutdown_connections 2024-09-11T22:22:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.400+0000 7f4b99aa4640 1 --2- 172.21.15.5:0/1554260333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9410c0e0 0x7f4b9410e560 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.400+0000 7f4b99aa4640 1 --2- 172.21.15.5:0/1554260333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b941035b0 0x7f4b9410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.400+0000 7f4b99aa4640 1 --2- 172.21.15.5:0/1554260333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b94102c90 0x7f4b94103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.400+0000 7f4b99aa4640 1 -- 172.21.15.5:0/1554260333 >> 172.21.15.5:0/1554260333 conn(0x7f4b940fc6a0 msgr2=0x7f4b940feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.400+0000 7f4b99aa4640 1 -- 172.21.15.5:0/1554260333 shutdown_connections 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.400+0000 7f4b99aa4640 1 -- 172.21.15.5:0/1554260333 wait complete. 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b99aa4640 1 Processor -- start 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b99aa4640 1 -- start start 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b99aa4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94102c90 0x7f4b941008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b99aa4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b941035b0 0x7f4b94100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b99aa4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9410c0e0 0x7f4b94101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b99aa4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94111ea0 con 0x7f4b9410c0e0 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b99aa4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94111d20 con 0x7f4b941035b0 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b99aa4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4b94112020 con 0x7f4b94102c90 2024-09-11T22:22:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9410c0e0 0x7f4b94101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9410c0e0 0x7f4b94101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57380/0 (socket says 172.21.15.5:57380) 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b93fff640 1 -- 172.21.15.5:0/3011174473 learned_addr learned my addr 172.21.15.5:0/3011174473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b92ffd640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b941035b0 0x7f4b94100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b937fe640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94102c90 0x7f4b941008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b93fff640 1 -- 172.21.15.5:0/3011174473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94102c90 msgr2=0x7f4b941008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b93fff640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94102c90 0x7f4b941008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b93fff640 1 -- 172.21.15.5:0/3011174473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b941035b0 msgr2=0x7f4b94100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b93fff640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b941035b0 0x7f4b94100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b93fff640 1 -- 172.21.15.5:0/3011174473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b941aad70 con 0x7f4b9410c0e0 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.401+0000 7f4b937fe640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94102c90 0x7f4b941008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.402+0000 7f4b93fff640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9410c0e0 0x7f4b94101360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c02ae70 tx=0x7f4b8c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.402+0000 7f4b90ff9640 1 -- 172.21.15.5:0/3011174473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c034040 con 0x7f4b9410c0e0 2024-09-11T22:22:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.402+0000 7f4b90ff9640 1 -- 172.21.15.5:0/3011174473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b8c030920 con 0x7f4b9410c0e0 2024-09-11T22:22:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.402+0000 7f4b99aa4640 1 -- 172.21.15.5:0/3011174473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b941aaf40 con 0x7f4b9410c0e0 2024-09-11T22:22:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.402+0000 7f4b90ff9640 1 -- 172.21.15.5:0/3011174473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c045480 con 0x7f4b9410c0e0 2024-09-11T22:22:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.402+0000 7f4b99aa4640 1 -- 172.21.15.5:0/3011174473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b941ab450 con 0x7f4b9410c0e0 2024-09-11T22:22:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.403+0000 7f4b99aa4640 1 -- 172.21.15.5:0/3011174473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b9406aca0 con 0x7f4b9410c0e0 2024-09-11T22:22:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.404+0000 7f4b90ff9640 1 -- 172.21.15.5:0/3011174473 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f4b8c037020 con 0x7f4b9410c0e0 2024-09-11T22:22:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.405+0000 7f4b90ff9640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4b7c0a2ea0 0x7f4b7c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.405+0000 7f4b90ff9640 1 -- 172.21.15.5:0/3011174473 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f4b8c0f0540 con 0x7f4b9410c0e0 2024-09-11T22:22:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.405+0000 7f4b937fe640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4b7c0a2ea0 0x7f4b7c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.405+0000 7f4b937fe640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4b7c0a2ea0 0x7f4b7c0a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009b40 tx=0x7f4b84005e50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.409+0000 7f4b90ff9640 1 -- 172.21.15.5:0/3011174473 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b8c0bcfc0 con 0x7f4b9410c0e0 2024-09-11T22:22:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.585+0000 7f4b99aa4640 1 -- 172.21.15.5:0/3011174473 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b94062630 con 0x7f4b7c0a2ea0 2024-09-11T22:22:21.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.591+0000 7f4b90ff9640 1 -- 172.21.15.5:0/3011174473 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258088 (secure 0 0 0) 0x7f4b94062630 con 0x7f4b7c0a2ea0 2024-09-11T22:22:21.595 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-11T22:22:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.594+0000 7f4b99aa4640 1 -- 172.21.15.5:0/3011174473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4b7c0a2ea0 msgr2=0x7f4b7c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.594+0000 7f4b99aa4640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4b7c0a2ea0 0x7f4b7c0a5360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009b40 tx=0x7f4b84005e50 comp rx=0 tx=0).stop 2024-09-11T22:22:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.594+0000 7f4b99aa4640 1 -- 172.21.15.5:0/3011174473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9410c0e0 msgr2=0x7f4b94101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.594+0000 7f4b99aa4640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9410c0e0 0x7f4b94101360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c02ae70 tx=0x7f4b8c02f5b0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.594+0000 7f4b99aa4640 1 -- 172.21.15.5:0/3011174473 shutdown_connections 2024-09-11T22:22:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.594+0000 7f4b99aa4640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b9410c0e0 0x7f4b94101360 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.594+0000 7f4b99aa4640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4b7c0a2ea0 0x7f4b7c0a5360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.594+0000 7f4b99aa4640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4b941035b0 0x7f4b94100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.594+0000 7f4b99aa4640 1 --2- 172.21.15.5:0/3011174473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b94102c90 0x7f4b941008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.594+0000 7f4b99aa4640 1 -- 172.21.15.5:0/3011174473 >> 172.21.15.5:0/3011174473 conn(0x7f4b940fc6a0 msgr2=0x7f4b940fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.595+0000 7f4b99aa4640 1 -- 172.21.15.5:0/3011174473 shutdown_connections 2024-09-11T22:22:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:21.595+0000 7f4b99aa4640 1 -- 172.21.15.5:0/3011174473 wait complete. 2024-09-11T22:22:21.607 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-11T22:22:19.987091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190734,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1294,"num_read_kb":836,"num_write":555,"num_write_kb":4403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181928,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":938,"ondisk_log_size":938,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385548,"kb_used_data":11292,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556276,"statfs":{"total":1151940427776,"available":1151545626624,"internally_reserved":0,"allocated":11563008,"data_stored":7937284,"data_compressed":241535,"data_compressed_allocated":319488,"data_compressed_original":2695168,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987365"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235071+0000","last_change":"2024-09-11T22:20:18.189744+0000","last_active":"2024-09-11T22:22:16.235071+0000","last_peered":"2024-09-11T22:22:16.235071+0000","last_clean":"2024-09-11T22:22:16.235071+0000","last_became_active":"2024-09-11T22:20:18.189398+0000","last_became_peered":"2024-09-11T22:20:18.189398+0000","last_unstale":"2024-09-11T22:22:16.235071+0000","last_undegraded":"2024-09-11T22:22:16.235071+0000","last_fullsized":"2024-09-11T22:22:16.235071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:24.152589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234137+0000","last_change":"2024-09-11T22:20:18.186727+0000","last_active":"2024-09-11T22:22:16.234137+0000","last_peered":"2024-09-11T22:22:16.234137+0000","last_clean":"2024-09-11T22:22:16.234137+0000","last_became_active":"2024-09-11T22:20:18.186414+0000","last_became_peered":"2024-09-11T22:20:18.186414+0000","last_unstale":"2024-09-11T22:22:16.234137+0000","last_undegraded":"2024-09-11T22:22:16.234137+0000","last_fullsized":"2024-09-11T22:22:16.234137+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:53.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233845+0000","last_change":"2024-09-11T22:20:18.179984+0000","last_active":"2024-09-11T22:22:16.233845+0000","last_peered":"2024-09-11T22:22:16.233845+0000","last_clean":"2024-09-11T22:22:16.233845+0000","last_became_active":"2024-09-11T22:20:18.179329+0000","last_became_peered":"2024-09-11T22:20:18.179329+0000","last_unstale":"2024-09-11T22:22:16.233845+0000","last_undegraded":"2024-09-11T22:22:16.233845+0000","last_fullsized":"2024-09-11T22:22:16.233845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:19.149970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230208+0000","last_change":"2024-09-11T22:20:18.191768+0000","last_active":"2024-09-11T22:22:16.230208+0000","last_peered":"2024-09-11T22:22:16.230208+0000","last_clean":"2024-09-11T22:22:16.230208+0000","last_became_active":"2024-09-11T22:20:18.191336+0000","last_became_peered":"2024-09-11T22:20:18.191336+0000","last_unstale":"2024-09-11T22:22:16.230208+0000","last_undegraded":"2024-09-11T22:22:16.230208+0000","last_fullsized":"2024-09-11T22:22:16.230208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:44.768932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058858+0000","last_change":"2024-09-11T22:20:18.181469+0000","last_active":"2024-09-11T22:22:16.058858+0000","last_peered":"2024-09-11T22:22:16.058858+0000","last_clean":"2024-09-11T22:22:16.058858+0000","last_became_active":"2024-09-11T22:20:18.181100+0000","last_became_peered":"2024-09-11T22:20:18.181100+0000","last_unstale":"2024-09-11T22:22:16.058858+0000","last_undegraded":"2024-09-11T22:22:16.058858+0000","last_fullsized":"2024-09-11T22:22:16.058858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:14.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068675+0000","last_change":"2024-09-11T22:20:18.190855+0000","last_active":"2024-09-11T22:22:16.068675+0000","last_peered":"2024-09-11T22:22:16.068675+0000","last_clean":"2024-09-11T22:22:16.068675+0000","last_became_active":"2024-09-11T22:20:18.190523+0000","last_became_peered":"2024-09-11T22:20:18.190523+0000","last_unstale":"2024-09-11T22:22:16.068675+0000","last_undegraded":"2024-09-11T22:22:16.068675+0000","last_fullsized":"2024-09-11T22:22:16.068675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:18.574957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231733+0000","last_change":"2024-09-11T22:20:18.199213+0000","last_active":"2024-09-11T22:22:16.231733+0000","last_peered":"2024-09-11T22:22:16.231733+0000","last_clean":"2024-09-11T22:22:16.231733+0000","last_became_active":"2024-09-11T22:20:18.198913+0000","last_became_peered":"2024-09-11T22:20:18.198913+0000","last_unstale":"2024-09-11T22:22:16.231733+0000","last_undegraded":"2024-09-11T22:22:16.231733+0000","last_fullsized":"2024-09-11T22:22:16.231733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:22.682802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058633+0000","last_change":"2024-09-11T22:20:18.192458+0000","last_active":"2024-09-11T22:22:16.058633+0000","last_peered":"2024-09-11T22:22:16.058633+0000","last_clean":"2024-09-11T22:22:16.058633+0000","last_became_active":"2024-09-11T22:20:18.191967+0000","last_became_peered":"2024-09-11T22:20:18.191967+0000","last_unstale":"2024-09-11T22:22:16.058633+0000","last_undegraded":"2024-09-11T22:22:16.058633+0000","last_fullsized":"2024-09-11T22:22:16.058633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:52.686157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975713+0000","last_change":"2024-09-11T22:21:18.191732+0000","last_active":"2024-09-11T22:22:15.975713+0000","last_peered":"2024-09-11T22:22:15.975713+0000","last_clean":"2024-09-11T22:22:15.975713+0000","last_became_active":"2024-09-11T22:21:18.191375+0000","last_became_peered":"2024-09-11T22:21:18.191375+0000","last_unstale":"2024-09-11T22:22:15.975713+0000","last_undegraded":"2024-09-11T22:22:15.975713+0000","last_fullsized":"2024-09-11T22:22:15.975713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:13.645224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.069211+0000","last_change":"2024-09-11T22:20:18.193771+0000","last_active":"2024-09-11T22:22:16.069211+0000","last_peered":"2024-09-11T22:22:16.069211+0000","last_clean":"2024-09-11T22:22:16.069211+0000","last_became_active":"2024-09-11T22:20:18.193041+0000","last_became_peered":"2024-09-11T22:20:18.193041+0000","last_unstale":"2024-09-11T22:22:16.069211+0000","last_undegraded":"2024-09-11T22:22:16.069211+0000","last_fullsized":"2024-09-11T22:22:16.069211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:06.656169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231998+0000","last_change":"2024-09-11T22:20:18.197933+0000","last_active":"2024-09-11T22:22:16.231998+0000","last_peered":"2024-09-11T22:22:16.231998+0000","last_clean":"2024-09-11T22:22:16.231998+0000","last_became_active":"2024-09-11T22:20:18.197210+0000","last_became_peered":"2024-09-11T22:20:18.197210+0000","last_unstale":"2024-09-11T22:22:16.231998+0000","last_undegraded":"2024-09-11T22:22:16.231998+0000","last_fullsized":"2024-09-11T22:22:16.231998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:42.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230250+0000","last_change":"2024-09-11T22:20:18.190739+0000","last_active":"2024-09-11T22:22:16.230250+0000","last_peered":"2024-09-11T22:22:16.230250+0000","last_clean":"2024-09-11T22:22:16.230250+0000","last_became_active":"2024-09-11T22:20:18.190122+0000","last_became_peered":"2024-09-11T22:20:18.190122+0000","last_unstale":"2024-09-11T22:22:16.230250+0000","last_undegraded":"2024-09-11T22:22:16.230250+0000","last_fullsized":"2024-09-11T22:22:16.230250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.585035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230697+0000","last_change":"2024-09-11T22:20:18.190303+0000","last_active":"2024-09-11T22:22:16.230697+0000","last_peered":"2024-09-11T22:22:16.230697+0000","last_clean":"2024-09-11T22:22:16.230697+0000","last_became_active":"2024-09-11T22:20:18.189962+0000","last_became_peered":"2024-09-11T22:20:18.189962+0000","last_unstale":"2024-09-11T22:22:16.230697+0000","last_undegraded":"2024-09-11T22:22:16.230697+0000","last_fullsized":"2024-09-11T22:22:16.230697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:44.872813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233520+0000","last_change":"2024-09-11T22:20:18.185893+0000","last_active":"2024-09-11T22:22:16.233520+0000","last_peered":"2024-09-11T22:22:16.233520+0000","last_clean":"2024-09-11T22:22:16.233520+0000","last_became_active":"2024-09-11T22:20:18.185592+0000","last_became_peered":"2024-09-11T22:20:18.185592+0000","last_unstale":"2024-09-11T22:22:16.233520+0000","last_undegraded":"2024-09-11T22:22:16.233520+0000","last_fullsized":"2024-09-11T22:22:16.233520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:14.725301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233777+0000","last_change":"2024-09-11T22:20:18.193943+0000","last_active":"2024-09-11T22:22:16.233777+0000","last_peered":"2024-09-11T22:22:16.233777+0000","last_clean":"2024-09-11T22:22:16.233777+0000","last_became_active":"2024-09-11T22:20:18.193558+0000","last_became_peered":"2024-09-11T22:20:18.193558+0000","last_unstale":"2024-09-11T22:22:16.233777+0000","last_undegraded":"2024-09-11T22:22:16.233777+0000","last_fullsized":"2024-09-11T22:22:16.233777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:08.736161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975254+0000","last_change":"2024-09-11T22:20:18.191815+0000","last_active":"2024-09-11T22:22:15.975254+0000","last_peered":"2024-09-11T22:22:15.975254+0000","last_clean":"2024-09-11T22:22:15.975254+0000","last_became_active":"2024-09-11T22:20:18.191483+0000","last_became_peered":"2024-09-11T22:20:18.191483+0000","last_unstale":"2024-09-11T22:22:15.975254+0000","last_undegraded":"2024-09-11T22:22:15.975254+0000","last_fullsized":"2024-09-11T22:22:15.975254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:26.432117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235101+0000","last_change":"2024-09-11T22:20:18.190758+0000","last_active":"2024-09-11T22:22:16.235101+0000","last_peered":"2024-09-11T22:22:16.235101+0000","last_clean":"2024-09-11T22:22:16.235101+0000","last_became_active":"2024-09-11T22:20:18.190431+0000","last_became_peered":"2024-09-11T22:20:18.190431+0000","last_unstale":"2024-09-11T22:22:16.235101+0000","last_undegraded":"2024-09-11T22:22:16.235101+0000","last_fullsized":"2024-09-11T22:22:16.235101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:10.927051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973552+0000","last_change":"2024-09-11T22:20:18.185191+0000","last_active":"2024-09-11T22:22:15.973552+0000","last_peered":"2024-09-11T22:22:15.973552+0000","last_clean":"2024-09-11T22:22:15.973552+0000","last_became_active":"2024-09-11T22:20:18.184906+0000","last_became_peered":"2024-09-11T22:20:18.184906+0000","last_unstale":"2024-09-11T22:22:15.973552+0000","last_undegraded":"2024-09-11T22:22:15.973552+0000","last_fullsized":"2024-09-11T22:22:15.973552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:32.294780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977515+0000","last_change":"2024-09-11T22:20:18.187137+0000","last_active":"2024-09-11T22:22:15.977515+0000","last_peered":"2024-09-11T22:22:15.977515+0000","last_clean":"2024-09-11T22:22:15.977515+0000","last_became_active":"2024-09-11T22:20:18.186770+0000","last_became_peered":"2024-09-11T22:20:18.186770+0000","last_unstale":"2024-09-11T22:22:15.977515+0000","last_undegraded":"2024-09-11T22:22:15.977515+0000","last_fullsized":"2024-09-11T22:22:15.977515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:59.917695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234743+0000","last_change":"2024-09-11T22:20:18.197137+0000","last_active":"2024-09-11T22:22:16.234743+0000","last_peered":"2024-09-11T22:22:16.234743+0000","last_clean":"2024-09-11T22:22:16.234743+0000","last_became_active":"2024-09-11T22:20:18.192810+0000","last_became_peered":"2024-09-11T22:20:18.192810+0000","last_unstale":"2024-09-11T22:22:16.234743+0000","last_undegraded":"2024-09-11T22:22:16.234743+0000","last_fullsized":"2024-09-11T22:22:16.234743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:57.680726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230830+0000","last_change":"2024-09-11T22:20:18.187568+0000","last_active":"2024-09-11T22:22:16.230830+0000","last_peered":"2024-09-11T22:22:16.230830+0000","last_clean":"2024-09-11T22:22:16.230830+0000","last_became_active":"2024-09-11T22:20:18.187254+0000","last_became_peered":"2024-09-11T22:20:18.187254+0000","last_unstale":"2024-09-11T22:22:16.230830+0000","last_undegraded":"2024-09-11T22:22:16.230830+0000","last_fullsized":"2024-09-11T22:22:16.230830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:41.724926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975378+0000","last_change":"2024-09-11T22:20:18.190376+0000","last_active":"2024-09-11T22:22:15.975378+0000","last_peered":"2024-09-11T22:22:15.975378+0000","last_clean":"2024-09-11T22:22:15.975378+0000","last_became_active":"2024-09-11T22:20:18.190079+0000","last_became_peered":"2024-09-11T22:20:18.190079+0000","last_unstale":"2024-09-11T22:22:15.975378+0000","last_undegraded":"2024-09-11T22:22:15.975378+0000","last_fullsized":"2024-09-11T22:22:15.975378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:42.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058896+0000","last_change":"2024-09-11T22:20:18.188503+0000","last_active":"2024-09-11T22:22:16.058896+0000","last_peered":"2024-09-11T22:22:16.058896+0000","last_clean":"2024-09-11T22:22:16.058896+0000","last_became_active":"2024-09-11T22:20:18.187957+0000","last_became_peered":"2024-09-11T22:20:18.187957+0000","last_unstale":"2024-09-11T22:22:16.058896+0000","last_undegraded":"2024-09-11T22:22:16.058896+0000","last_fullsized":"2024-09-11T22:22:16.058896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:44:25.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065572+0000","last_change":"2024-09-11T22:20:18.196936+0000","last_active":"2024-09-11T22:22:16.065572+0000","last_peered":"2024-09-11T22:22:16.065572+0000","last_clean":"2024-09-11T22:22:16.065572+0000","last_became_active":"2024-09-11T22:20:18.196493+0000","last_became_peered":"2024-09-11T22:20:18.196493+0000","last_unstale":"2024-09-11T22:22:16.065572+0000","last_undegraded":"2024-09-11T22:22:16.065572+0000","last_fullsized":"2024-09-11T22:22:16.065572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:06.189806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059126+0000","last_change":"2024-09-11T22:20:18.195260+0000","last_active":"2024-09-11T22:22:16.059126+0000","last_peered":"2024-09-11T22:22:16.059126+0000","last_clean":"2024-09-11T22:22:16.059126+0000","last_became_active":"2024-09-11T22:20:18.194956+0000","last_became_peered":"2024-09-11T22:20:18.194956+0000","last_unstale":"2024-09-11T22:22:16.059126+0000","last_undegraded":"2024-09-11T22:22:16.059126+0000","last_fullsized":"2024-09-11T22:22:16.059126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:34.483291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977695+0000","last_change":"2024-09-11T22:20:18.190591+0000","last_active":"2024-09-11T22:22:15.977695+0000","last_peered":"2024-09-11T22:22:15.977695+0000","last_clean":"2024-09-11T22:22:15.977695+0000","last_became_active":"2024-09-11T22:20:18.190007+0000","last_became_peered":"2024-09-11T22:20:18.190007+0000","last_unstale":"2024-09-11T22:22:15.977695+0000","last_undegraded":"2024-09-11T22:22:15.977695+0000","last_fullsized":"2024-09-11T22:22:15.977695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:46.281705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234929+0000","last_change":"2024-09-11T22:20:18.193377+0000","last_active":"2024-09-11T22:22:16.234929+0000","last_peered":"2024-09-11T22:22:16.234929+0000","last_clean":"2024-09-11T22:22:16.234929+0000","last_became_active":"2024-09-11T22:20:18.193003+0000","last_became_peered":"2024-09-11T22:20:18.193003+0000","last_unstale":"2024-09-11T22:22:16.234929+0000","last_undegraded":"2024-09-11T22:22:16.234929+0000","last_fullsized":"2024-09-11T22:22:16.234929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:59.052966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230737+0000","last_change":"2024-09-11T22:20:18.189405+0000","last_active":"2024-09-11T22:22:16.230737+0000","last_peered":"2024-09-11T22:22:16.230737+0000","last_clean":"2024-09-11T22:22:16.230737+0000","last_became_active":"2024-09-11T22:20:18.189098+0000","last_became_peered":"2024-09-11T22:20:18.189098+0000","last_unstale":"2024-09-11T22:22:16.230737+0000","last_undegraded":"2024-09-11T22:22:16.230737+0000","last_fullsized":"2024-09-11T22:22:16.230737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:47.088192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233832+0000","last_change":"2024-09-11T22:20:18.191005+0000","last_active":"2024-09-11T22:22:16.233832+0000","last_peered":"2024-09-11T22:22:16.233832+0000","last_clean":"2024-09-11T22:22:16.233832+0000","last_became_active":"2024-09-11T22:20:18.190681+0000","last_became_peered":"2024-09-11T22:20:18.190681+0000","last_unstale":"2024-09-11T22:22:16.233832+0000","last_undegraded":"2024-09-11T22:22:16.233832+0000","last_fullsized":"2024-09-11T22:22:16.233832+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:12.903929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977210+0000","last_change":"2024-09-11T22:20:18.187920+0000","last_active":"2024-09-11T22:22:15.977210+0000","last_peered":"2024-09-11T22:22:15.977210+0000","last_clean":"2024-09-11T22:22:15.977210+0000","last_became_active":"2024-09-11T22:20:18.187526+0000","last_became_peered":"2024-09-11T22:20:18.187526+0000","last_unstale":"2024-09-11T22:22:15.977210+0000","last_undegraded":"2024-09-11T22:22:15.977210+0000","last_fullsized":"2024-09-11T22:22:15.977210+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.347857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977399+0000","last_change":"2024-09-11T22:20:18.193869+0000","last_active":"2024-09-11T22:22:15.977399+0000","last_peered":"2024-09-11T22:22:15.977399+0000","last_clean":"2024-09-11T22:22:15.977399+0000","last_became_active":"2024-09-11T22:20:18.193468+0000","last_became_peered":"2024-09-11T22:20:18.193468+0000","last_unstale":"2024-09-11T22:22:15.977399+0000","last_undegraded":"2024-09-11T22:22:15.977399+0000","last_fullsized":"2024-09-11T22:22:15.977399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:59.619611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234817+0000","last_change":"2024-09-11T22:20:18.193947+0000","last_active":"2024-09-11T22:22:16.234817+0000","last_peered":"2024-09-11T22:22:16.234817+0000","last_clean":"2024-09-11T22:22:16.234817+0000","last_became_active":"2024-09-11T22:20:18.192257+0000","last_became_peered":"2024-09-11T22:20:18.192257+0000","last_unstale":"2024-09-11T22:22:16.234817+0000","last_undegraded":"2024-09-11T22:22:16.234817+0000","last_fullsized":"2024-09-11T22:22:16.234817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.478730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058805+0000","last_change":"2024-09-11T22:20:18.184549+0000","last_active":"2024-09-11T22:22:16.058805+0000","last_peered":"2024-09-11T22:22:16.058805+0000","last_clean":"2024-09-11T22:22:16.058805+0000","last_became_active":"2024-09-11T22:20:18.184075+0000","last_became_peered":"2024-09-11T22:20:18.184075+0000","last_unstale":"2024-09-11T22:22:16.058805+0000","last_undegraded":"2024-09-11T22:22:16.058805+0000","last_fullsized":"2024-09-11T22:22:16.058805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:50.095643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'7","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.415258+0000","last_change":"2024-09-11T22:20:16.128524+0000","last_active":"2024-09-11T22:22:16.415258+0000","last_peered":"2024-09-11T22:22:16.415258+0000","last_clean":"2024-09-11T22:22:16.415258+0000","last_became_active":"2024-09-11T22:20:16.128185+0000","last_became_peered":"2024-09-11T22:20:16.128185+0000","last_unstale":"2024-09-11T22:22:16.415258+0000","last_undegraded":"2024-09-11T22:22:16.415258+0000","last_fullsized":"2024-09-11T22:22:16.415258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.022931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1323,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065720+0000","last_change":"2024-09-11T22:20:18.191578+0000","last_active":"2024-09-11T22:22:16.065720+0000","last_peered":"2024-09-11T22:22:16.065720+0000","last_clean":"2024-09-11T22:22:16.065720+0000","last_became_active":"2024-09-11T22:20:18.189568+0000","last_became_peered":"2024-09-11T22:20:18.189568+0000","last_unstale":"2024-09-11T22:22:16.065720+0000","last_undegraded":"2024-09-11T22:22:16.065720+0000","last_fullsized":"2024-09-11T22:22:16.065720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:41.737242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068742+0000","last_change":"2024-09-11T22:21:18.199693+0000","last_active":"2024-09-11T22:22:16.068742+0000","last_peered":"2024-09-11T22:22:16.068742+0000","last_clean":"2024-09-11T22:22:16.068742+0000","last_became_active":"2024-09-11T22:21:18.190954+0000","last_became_peered":"2024-09-11T22:21:18.190954+0000","last_unstale":"2024-09-11T22:22:16.068742+0000","last_undegraded":"2024-09-11T22:22:16.068742+0000","last_fullsized":"2024-09-11T22:22:16.068742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:11.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230688+0000","last_change":"2024-09-11T22:20:16.115602+0000","last_active":"2024-09-11T22:22:16.230688+0000","last_peered":"2024-09-11T22:22:16.230688+0000","last_clean":"2024-09-11T22:22:16.230688+0000","last_became_active":"2024-09-11T22:20:16.115280+0000","last_became_peered":"2024-09-11T22:20:16.115280+0000","last_unstale":"2024-09-11T22:22:16.230688+0000","last_undegraded":"2024-09-11T22:22:16.230688+0000","last_fullsized":"2024-09-11T22:22:16.230688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:41.942137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058816+0000","last_change":"2024-09-11T22:20:18.184125+0000","last_active":"2024-09-11T22:22:16.058816+0000","last_peered":"2024-09-11T22:22:16.058816+0000","last_clean":"2024-09-11T22:22:16.058816+0000","last_became_active":"2024-09-11T22:20:18.183785+0000","last_became_peered":"2024-09-11T22:20:18.183785+0000","last_unstale":"2024-09-11T22:22:16.058816+0000","last_undegraded":"2024-09-11T22:22:16.058816+0000","last_fullsized":"2024-09-11T22:22:16.058816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:33.230175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065780+0000","last_change":"2024-09-11T22:20:16.125852+0000","last_active":"2024-09-11T22:22:16.065780+0000","last_peered":"2024-09-11T22:22:16.065780+0000","last_clean":"2024-09-11T22:22:16.065780+0000","last_became_active":"2024-09-11T22:20:16.125250+0000","last_became_peered":"2024-09-11T22:20:16.125250+0000","last_unstale":"2024-09-11T22:22:16.065780+0000","last_undegraded":"2024-09-11T22:22:16.065780+0000","last_fullsized":"2024-09-11T22:22:16.065780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:34.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973235+0000","last_change":"2024-09-11T22:20:18.185147+0000","last_active":"2024-09-11T22:22:15.973235+0000","last_peered":"2024-09-11T22:22:15.973235+0000","last_clean":"2024-09-11T22:22:15.973235+0000","last_became_active":"2024-09-11T22:20:18.184840+0000","last_became_peered":"2024-09-11T22:20:18.184840+0000","last_unstale":"2024-09-11T22:22:15.973235+0000","last_undegraded":"2024-09-11T22:22:15.973235+0000","last_fullsized":"2024-09-11T22:22:15.973235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:59.703349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":243,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.277347+0000","last_change":"2024-09-11T22:20:16.122173+0000","last_active":"2024-09-11T22:22:16.277347+0000","last_peered":"2024-09-11T22:22:16.277347+0000","last_clean":"2024-09-11T22:22:16.277347+0000","last_became_active":"2024-09-11T22:20:16.121847+0000","last_became_peered":"2024-09-11T22:20:16.121847+0000","last_unstale":"2024-09-11T22:22:16.277347+0000","last_undegraded":"2024-09-11T22:22:16.277347+0000","last_fullsized":"2024-09-11T22:22:16.277347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:41.284557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":213,"num_read_kb":213,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975691+0000","last_change":"2024-09-11T22:20:18.188030+0000","last_active":"2024-09-11T22:22:15.975691+0000","last_peered":"2024-09-11T22:22:15.975691+0000","last_clean":"2024-09-11T22:22:15.975691+0000","last_became_active":"2024-09-11T22:20:18.187503+0000","last_became_peered":"2024-09-11T22:20:18.187503+0000","last_unstale":"2024-09-11T22:22:15.975691+0000","last_undegraded":"2024-09-11T22:22:15.975691+0000","last_fullsized":"2024-09-11T22:22:15.975691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:45.494149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231598+0000","last_change":"2024-09-11T22:20:16.128828+0000","last_active":"2024-09-11T22:22:16.231598+0000","last_peered":"2024-09-11T22:22:16.231598+0000","last_clean":"2024-09-11T22:22:16.231598+0000","last_became_active":"2024-09-11T22:20:16.128454+0000","last_became_peered":"2024-09-11T22:20:16.128454+0000","last_unstale":"2024-09-11T22:22:16.231598+0000","last_undegraded":"2024-09-11T22:22:16.231598+0000","last_fullsized":"2024-09-11T22:22:16.231598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:33.927564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234475+0000","last_change":"2024-09-11T22:20:18.193795+0000","last_active":"2024-09-11T22:22:16.234475+0000","last_peered":"2024-09-11T22:22:16.234475+0000","last_clean":"2024-09-11T22:22:16.234475+0000","last_became_active":"2024-09-11T22:20:18.193489+0000","last_became_peered":"2024-09-11T22:20:18.193489+0000","last_unstale":"2024-09-11T22:22:16.234475+0000","last_undegraded":"2024-09-11T22:22:16.234475+0000","last_fullsized":"2024-09-11T22:22:16.234475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:14.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065914+0000","last_change":"2024-09-11T22:20:16.187938+0000","last_active":"2024-09-11T22:22:16.065914+0000","last_peered":"2024-09-11T22:22:16.065914+0000","last_clean":"2024-09-11T22:22:16.065914+0000","last_became_active":"2024-09-11T22:20:16.187545+0000","last_became_peered":"2024-09-11T22:20:16.187545+0000","last_unstale":"2024-09-11T22:22:16.065914+0000","last_undegraded":"2024-09-11T22:22:16.065914+0000","last_fullsized":"2024-09-11T22:22:16.065914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:41.674356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975385+0000","last_change":"2024-09-11T22:21:18.191937+0000","last_active":"2024-09-11T22:22:15.975385+0000","last_peered":"2024-09-11T22:22:15.975385+0000","last_clean":"2024-09-11T22:22:15.975385+0000","last_became_active":"2024-09-11T22:21:18.191218+0000","last_became_peered":"2024-09-11T22:21:18.191218+0000","last_unstale":"2024-09-11T22:22:15.975385+0000","last_undegraded":"2024-09-11T22:22:15.975385+0000","last_fullsized":"2024-09-11T22:22:15.975385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:41.654630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'37","reported_seq":287,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.414971+0000","last_change":"2024-09-11T22:20:16.128082+0000","last_active":"2024-09-11T22:22:16.414971+0000","last_peered":"2024-09-11T22:22:16.414971+0000","last_clean":"2024-09-11T22:22:16.414971+0000","last_became_active":"2024-09-11T22:20:16.127731+0000","last_became_peered":"2024-09-11T22:20:16.127731+0000","last_unstale":"2024-09-11T22:22:16.414971+0000","last_undegraded":"2024-09-11T22:22:16.414971+0000","last_fullsized":"2024-09-11T22:22:16.414971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:33.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66480,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":226,"num_read_kb":178,"num_write":39,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059064+0000","last_change":"2024-09-11T22:20:18.189219+0000","last_active":"2024-09-11T22:22:16.059064+0000","last_peered":"2024-09-11T22:22:16.059064+0000","last_clean":"2024-09-11T22:22:16.059064+0000","last_became_active":"2024-09-11T22:20:18.188687+0000","last_became_peered":"2024-09-11T22:20:18.188687+0000","last_unstale":"2024-09-11T22:22:16.059064+0000","last_undegraded":"2024-09-11T22:22:16.059064+0000","last_fullsized":"2024-09-11T22:22:16.059064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:24.171126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977378+0000","last_change":"2024-09-11T22:20:16.129724+0000","last_active":"2024-09-11T22:22:15.977378+0000","last_peered":"2024-09-11T22:22:15.977378+0000","last_clean":"2024-09-11T22:22:15.977378+0000","last_became_active":"2024-09-11T22:20:16.129351+0000","last_became_peered":"2024-09-11T22:20:16.129351+0000","last_unstale":"2024-09-11T22:22:15.977378+0000","last_undegraded":"2024-09-11T22:22:15.977378+0000","last_fullsized":"2024-09-11T22:22:15.977378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:09.642520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230435+0000","last_change":"2024-09-11T22:20:18.187031+0000","last_active":"2024-09-11T22:22:16.230435+0000","last_peered":"2024-09-11T22:22:16.230435+0000","last_clean":"2024-09-11T22:22:16.230435+0000","last_became_active":"2024-09-11T22:20:18.186698+0000","last_became_peered":"2024-09-11T22:20:18.186698+0000","last_unstale":"2024-09-11T22:22:16.230435+0000","last_undegraded":"2024-09-11T22:22:16.230435+0000","last_fullsized":"2024-09-11T22:22:16.230435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:04.810981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065725+0000","last_change":"2024-09-11T22:20:16.124925+0000","last_active":"2024-09-11T22:22:16.065725+0000","last_peered":"2024-09-11T22:22:16.065725+0000","last_clean":"2024-09-11T22:22:16.065725+0000","last_became_active":"2024-09-11T22:20:16.124580+0000","last_became_peered":"2024-09-11T22:20:16.124580+0000","last_unstale":"2024-09-11T22:22:16.065725+0000","last_undegraded":"2024-09-11T22:22:16.065725+0000","last_fullsized":"2024-09-11T22:22:16.065725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:56.417577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231838+0000","last_change":"2024-09-11T22:21:18.187783+0000","last_active":"2024-09-11T22:22:16.231838+0000","last_peered":"2024-09-11T22:22:16.231838+0000","last_clean":"2024-09-11T22:22:16.231838+0000","last_became_active":"2024-09-11T22:21:18.187358+0000","last_became_peered":"2024-09-11T22:21:18.187358+0000","last_unstale":"2024-09-11T22:22:16.231838+0000","last_undegraded":"2024-09-11T22:22:16.231838+0000","last_fullsized":"2024-09-11T22:22:16.231838+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:41.251531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.439108+0000","last_change":"2024-09-11T22:20:16.118934+0000","last_active":"2024-09-11T22:22:16.439108+0000","last_peered":"2024-09-11T22:22:16.439108+0000","last_clean":"2024-09-11T22:22:16.439108+0000","last_became_active":"2024-09-11T22:20:16.118614+0000","last_became_peered":"2024-09-11T22:20:16.118614+0000","last_unstale":"2024-09-11T22:22:16.439108+0000","last_undegraded":"2024-09-11T22:22:16.439108+0000","last_fullsized":"2024-09-11T22:22:16.439108+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:05.791161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231721+0000","last_change":"2024-09-11T22:20:18.197692+0000","last_active":"2024-09-11T22:22:16.231721+0000","last_peered":"2024-09-11T22:22:16.231721+0000","last_clean":"2024-09-11T22:22:16.231721+0000","last_became_active":"2024-09-11T22:20:18.197336+0000","last_became_peered":"2024-09-11T22:20:18.197336+0000","last_unstale":"2024-09-11T22:22:16.231721+0000","last_undegraded":"2024-09-11T22:22:16.231721+0000","last_fullsized":"2024-09-11T22:22:16.231721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:55.172568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:16.186769+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:16.186367+0000","last_became_peered":"2024-09-11T22:20:16.186367+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.279387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065641+0000","last_change":"2024-09-11T22:20:18.195570+0000","last_active":"2024-09-11T22:22:16.065641+0000","last_peered":"2024-09-11T22:22:16.065641+0000","last_clean":"2024-09-11T22:22:16.065641+0000","last_became_active":"2024-09-11T22:20:18.195226+0000","last_became_peered":"2024-09-11T22:20:18.195226+0000","last_unstale":"2024-09-11T22:22:16.065641+0000","last_undegraded":"2024-09-11T22:22:16.065641+0000","last_fullsized":"2024-09-11T22:22:16.065641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:44.108090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235034+0000","last_change":"2024-09-11T22:21:18.194553+0000","last_active":"2024-09-11T22:22:16.235034+0000","last_peered":"2024-09-11T22:22:16.235034+0000","last_clean":"2024-09-11T22:22:16.235034+0000","last_became_active":"2024-09-11T22:21:18.188635+0000","last_became_peered":"2024-09-11T22:21:18.188635+0000","last_unstale":"2024-09-11T22:22:16.235034+0000","last_undegraded":"2024-09-11T22:22:16.235034+0000","last_fullsized":"2024-09-11T22:22:16.235034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:34.165195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234954+0000","last_change":"2024-09-11T22:20:18.194112+0000","last_active":"2024-09-11T22:22:16.234954+0000","last_peered":"2024-09-11T22:22:16.234954+0000","last_clean":"2024-09-11T22:22:16.234954+0000","last_became_active":"2024-09-11T22:20:18.193812+0000","last_became_peered":"2024-09-11T22:20:18.193812+0000","last_unstale":"2024-09-11T22:22:16.234954+0000","last_undegraded":"2024-09-11T22:22:16.234954+0000","last_fullsized":"2024-09-11T22:22:16.234954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:35.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231070+0000","last_change":"2024-09-11T22:21:18.198325+0000","last_active":"2024-09-11T22:22:16.231070+0000","last_peered":"2024-09-11T22:22:16.231070+0000","last_clean":"2024-09-11T22:22:16.231070+0000","last_became_active":"2024-09-11T22:21:18.193651+0000","last_became_peered":"2024-09-11T22:21:18.193651+0000","last_unstale":"2024-09-11T22:22:16.231070+0000","last_undegraded":"2024-09-11T22:22:16.231070+0000","last_fullsized":"2024-09-11T22:22:16.231070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:42.552492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232027+0000","last_change":"2024-09-11T22:20:18.191960+0000","last_active":"2024-09-11T22:22:16.232027+0000","last_peered":"2024-09-11T22:22:16.232027+0000","last_clean":"2024-09-11T22:22:16.232027+0000","last_became_active":"2024-09-11T22:20:18.191635+0000","last_became_peered":"2024-09-11T22:20:18.191635+0000","last_unstale":"2024-09-11T22:22:16.232027+0000","last_undegraded":"2024-09-11T22:22:16.232027+0000","last_fullsized":"2024-09-11T22:22:16.232027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:19:18.919247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'41","reported_seq":73,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.176627+0000","last_change":"2024-09-11T22:20:16.123608+0000","last_active":"2024-09-11T22:22:16.176627+0000","last_peered":"2024-09-11T22:22:16.176627+0000","last_clean":"2024-09-11T22:22:16.176627+0000","last_became_active":"2024-09-11T22:20:16.123193+0000","last_became_peered":"2024-09-11T22:20:16.123193+0000","last_unstale":"2024-09-11T22:22:16.176627+0000","last_undegraded":"2024-09-11T22:22:16.176627+0000","last_fullsized":"2024-09-11T22:22:16.176627+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:07.362115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231887+0000","last_change":"2024-09-11T22:20:18.199957+0000","last_active":"2024-09-11T22:22:16.231887+0000","last_peered":"2024-09-11T22:22:16.231887+0000","last_clean":"2024-09-11T22:22:16.231887+0000","last_became_active":"2024-09-11T22:20:18.199637+0000","last_became_peered":"2024-09-11T22:20:18.199637+0000","last_unstale":"2024-09-11T22:22:16.231887+0000","last_undegraded":"2024-09-11T22:22:16.231887+0000","last_fullsized":"2024-09-11T22:22:16.231887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:14.949002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'6","reported_seq":82,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.277747+0000","last_change":"2024-09-11T22:21:18.198896+0000","last_active":"2024-09-11T22:22:16.277747+0000","last_peered":"2024-09-11T22:22:16.277747+0000","last_clean":"2024-09-11T22:22:16.277747+0000","last_became_active":"2024-09-11T22:21:18.189489+0000","last_became_peered":"2024-09-11T22:21:18.189489+0000","last_unstale":"2024-09-11T22:22:16.277747+0000","last_undegraded":"2024-09-11T22:22:16.277747+0000","last_fullsized":"2024-09-11T22:22:16.277747+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:19.908801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232111+0000","last_change":"2024-09-11T22:20:18.196691+0000","last_active":"2024-09-11T22:22:16.232111+0000","last_peered":"2024-09-11T22:22:16.232111+0000","last_clean":"2024-09-11T22:22:16.232111+0000","last_became_active":"2024-09-11T22:20:18.196364+0000","last_became_peered":"2024-09-11T22:20:18.196364+0000","last_unstale":"2024-09-11T22:22:16.232111+0000","last_undegraded":"2024-09-11T22:22:16.232111+0000","last_fullsized":"2024-09-11T22:22:16.232111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:50.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231876+0000","last_change":"2024-09-11T22:19:02.590079+0000","last_active":"2024-09-11T22:22:16.231876+0000","last_peered":"2024-09-11T22:22:16.231876+0000","last_clean":"2024-09-11T22:22:16.231876+0000","last_became_active":"2024-09-11T22:19:02.589644+0000","last_became_peered":"2024-09-11T22:19:02.589644+0000","last_unstale":"2024-09-11T22:22:16.231876+0000","last_undegraded":"2024-09-11T22:22:16.231876+0000","last_fullsized":"2024-09-11T22:22:16.231876+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_clean_scrub_stamp":"2024-09-11T22:15:33.244685+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:45.506190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.229755+0000","last_change":"2024-09-11T22:20:16.116661+0000","last_active":"2024-09-11T22:22:16.229755+0000","last_peered":"2024-09-11T22:22:16.229755+0000","last_clean":"2024-09-11T22:22:16.229755+0000","last_became_active":"2024-09-11T22:20:16.116196+0000","last_became_peered":"2024-09-11T22:20:16.116196+0000","last_unstale":"2024-09-11T22:22:16.229755+0000","last_undegraded":"2024-09-11T22:22:16.229755+0000","last_fullsized":"2024-09-11T22:22:16.229755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:08.875170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059167+0000","last_change":"2024-09-11T22:20:18.193176+0000","last_active":"2024-09-11T22:22:16.059167+0000","last_peered":"2024-09-11T22:22:16.059167+0000","last_clean":"2024-09-11T22:22:16.059167+0000","last_became_active":"2024-09-11T22:20:18.192312+0000","last_became_peered":"2024-09-11T22:20:18.192312+0000","last_unstale":"2024-09-11T22:22:16.059167+0000","last_undegraded":"2024-09-11T22:22:16.059167+0000","last_fullsized":"2024-09-11T22:22:16.059167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:08.738569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234694+0000","last_change":"2024-09-11T22:21:18.196985+0000","last_active":"2024-09-11T22:22:16.234694+0000","last_peered":"2024-09-11T22:22:16.234694+0000","last_clean":"2024-09-11T22:22:16.234694+0000","last_became_active":"2024-09-11T22:21:18.189487+0000","last_became_peered":"2024-09-11T22:21:18.189487+0000","last_unstale":"2024-09-11T22:22:16.234694+0000","last_undegraded":"2024-09-11T22:22:16.234694+0000","last_fullsized":"2024-09-11T22:22:16.234694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:20.715696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230470+0000","last_change":"2024-09-11T22:20:18.191118+0000","last_active":"2024-09-11T22:22:16.230470+0000","last_peered":"2024-09-11T22:22:16.230470+0000","last_clean":"2024-09-11T22:22:16.230470+0000","last_became_active":"2024-09-11T22:20:18.190760+0000","last_became_peered":"2024-09-11T22:20:18.190760+0000","last_unstale":"2024-09-11T22:22:16.230470+0000","last_undegraded":"2024-09-11T22:22:16.230470+0000","last_fullsized":"2024-09-11T22:22:16.230470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:59.220067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235387+0000","last_change":"2024-09-11T22:20:16.186407+0000","last_active":"2024-09-11T22:22:16.235387+0000","last_peered":"2024-09-11T22:22:16.235387+0000","last_clean":"2024-09-11T22:22:16.235387+0000","last_became_active":"2024-09-11T22:20:16.186105+0000","last_became_peered":"2024-09-11T22:20:16.186105+0000","last_unstale":"2024-09-11T22:22:16.235387+0000","last_undegraded":"2024-09-11T22:22:16.235387+0000","last_fullsized":"2024-09-11T22:22:16.235387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:43.942072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065626+0000","last_change":"2024-09-11T22:20:18.191970+0000","last_active":"2024-09-11T22:22:16.065626+0000","last_peered":"2024-09-11T22:22:16.065626+0000","last_clean":"2024-09-11T22:22:16.065626+0000","last_became_active":"2024-09-11T22:20:18.191623+0000","last_became_peered":"2024-09-11T22:20:18.191623+0000","last_unstale":"2024-09-11T22:22:16.065626+0000","last_undegraded":"2024-09-11T22:22:16.065626+0000","last_fullsized":"2024-09-11T22:22:16.065626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:33.213979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":141,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231514+0000","last_change":"2024-09-11T22:20:16.119936+0000","last_active":"2024-09-11T22:22:16.231514+0000","last_peered":"2024-09-11T22:22:16.231514+0000","last_clean":"2024-09-11T22:22:16.231514+0000","last_became_active":"2024-09-11T22:20:16.119453+0000","last_became_peered":"2024-09-11T22:20:16.119453+0000","last_unstale":"2024-09-11T22:22:16.231514+0000","last_undegraded":"2024-09-11T22:22:16.231514+0000","last_fullsized":"2024-09-11T22:22:16.231514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:20.717980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068790+0000","last_change":"2024-09-11T22:20:18.192581+0000","last_active":"2024-09-11T22:22:16.068790+0000","last_peered":"2024-09-11T22:22:16.068790+0000","last_clean":"2024-09-11T22:22:16.068790+0000","last_became_active":"2024-09-11T22:20:18.191702+0000","last_became_peered":"2024-09-11T22:20:18.191702+0000","last_unstale":"2024-09-11T22:22:16.068790+0000","last_undegraded":"2024-09-11T22:22:16.068790+0000","last_fullsized":"2024-09-11T22:22:16.068790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:48.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973402+0000","last_change":"2024-09-11T22:20:16.186485+0000","last_active":"2024-09-11T22:22:15.973402+0000","last_peered":"2024-09-11T22:22:15.973402+0000","last_clean":"2024-09-11T22:22:15.973402+0000","last_became_active":"2024-09-11T22:20:16.186171+0000","last_became_peered":"2024-09-11T22:20:16.186171+0000","last_unstale":"2024-09-11T22:22:15.973402+0000","last_undegraded":"2024-09-11T22:22:15.973402+0000","last_fullsized":"2024-09-11T22:22:15.973402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:26.373626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068876+0000","last_change":"2024-09-11T22:20:18.191361+0000","last_active":"2024-09-11T22:22:16.068876+0000","last_peered":"2024-09-11T22:22:16.068876+0000","last_clean":"2024-09-11T22:22:16.068876+0000","last_became_active":"2024-09-11T22:20:18.191036+0000","last_became_peered":"2024-09-11T22:20:18.191036+0000","last_unstale":"2024-09-11T22:22:16.068876+0000","last_undegraded":"2024-09-11T22:22:16.068876+0000","last_fullsized":"2024-09-11T22:22:16.068876+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:29.124650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059181+0000","last_change":"2024-09-11T22:20:16.188156+0000","last_active":"2024-09-11T22:22:16.059181+0000","last_peered":"2024-09-11T22:22:16.059181+0000","last_clean":"2024-09-11T22:22:16.059181+0000","last_became_active":"2024-09-11T22:20:16.187809+0000","last_became_peered":"2024-09-11T22:20:16.187809+0000","last_unstale":"2024-09-11T22:22:16.059181+0000","last_undegraded":"2024-09-11T22:22:16.059181+0000","last_fullsized":"2024-09-11T22:22:16.059181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:04.219621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230896+0000","last_change":"2024-09-11T22:20:18.182380+0000","last_active":"2024-09-11T22:22:16.230896+0000","last_peered":"2024-09-11T22:22:16.230896+0000","last_clean":"2024-09-11T22:22:16.230896+0000","last_became_active":"2024-09-11T22:20:18.182062+0000","last_became_peered":"2024-09-11T22:20:18.182062+0000","last_unstale":"2024-09-11T22:22:16.230896+0000","last_undegraded":"2024-09-11T22:22:16.230896+0000","last_fullsized":"2024-09-11T22:22:16.230896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:15.575846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'50","reported_seq":97,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.179372+0000","last_change":"2024-09-11T22:20:16.117335+0000","last_active":"2024-09-11T22:22:16.179372+0000","last_peered":"2024-09-11T22:22:16.179372+0000","last_clean":"2024-09-11T22:22:16.179372+0000","last_became_active":"2024-09-11T22:20:16.116977+0000","last_became_peered":"2024-09-11T22:20:16.116977+0000","last_unstale":"2024-09-11T22:22:16.179372+0000","last_undegraded":"2024-09-11T22:22:16.179372+0000","last_fullsized":"2024-09-11T22:22:16.179372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:51.744202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059236+0000","last_change":"2024-09-11T22:20:18.194069+0000","last_active":"2024-09-11T22:22:16.059236+0000","last_peered":"2024-09-11T22:22:16.059236+0000","last_clean":"2024-09-11T22:22:16.059236+0000","last_became_active":"2024-09-11T22:20:18.193765+0000","last_became_peered":"2024-09-11T22:20:18.193765+0000","last_unstale":"2024-09-11T22:22:16.059236+0000","last_undegraded":"2024-09-11T22:22:16.059236+0000","last_fullsized":"2024-09-11T22:22:16.059236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:25.047405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230131+0000","last_change":"2024-09-11T22:20:16.460987+0000","last_active":"2024-09-11T22:22:16.230131+0000","last_peered":"2024-09-11T22:22:16.230131+0000","last_clean":"2024-09-11T22:22:16.230131+0000","last_became_active":"2024-09-11T22:20:16.460516+0000","last_became_peered":"2024-09-11T22:20:16.460516+0000","last_unstale":"2024-09-11T22:22:16.230131+0000","last_undegraded":"2024-09-11T22:22:16.230131+0000","last_fullsized":"2024-09-11T22:22:16.230131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:00.987981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234264+0000","last_change":"2024-09-11T22:21:18.188802+0000","last_active":"2024-09-11T22:22:16.234264+0000","last_peered":"2024-09-11T22:22:16.234264+0000","last_clean":"2024-09-11T22:22:16.234264+0000","last_became_active":"2024-09-11T22:21:18.188441+0000","last_became_peered":"2024-09-11T22:21:18.188441+0000","last_unstale":"2024-09-11T22:22:16.234264+0000","last_undegraded":"2024-09-11T22:22:16.234264+0000","last_fullsized":"2024-09-11T22:22:16.234264+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:02.295441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975337+0000","last_change":"2024-09-11T22:20:16.186498+0000","last_active":"2024-09-11T22:22:15.975337+0000","last_peered":"2024-09-11T22:22:15.975337+0000","last_clean":"2024-09-11T22:22:15.975337+0000","last_became_active":"2024-09-11T22:20:16.186214+0000","last_became_peered":"2024-09-11T22:20:16.186214+0000","last_unstale":"2024-09-11T22:22:15.975337+0000","last_undegraded":"2024-09-11T22:22:15.975337+0000","last_fullsized":"2024-09-11T22:22:15.975337+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:09.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:18.187489+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:18.185649+0000","last_became_peered":"2024-09-11T22:20:18.185649+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:42.825075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059011+0000","last_change":"2024-09-11T22:20:16.117627+0000","last_active":"2024-09-11T22:22:16.059011+0000","last_peered":"2024-09-11T22:22:16.059011+0000","last_clean":"2024-09-11T22:22:16.059011+0000","last_became_active":"2024-09-11T22:20:16.117243+0000","last_became_peered":"2024-09-11T22:20:16.117243+0000","last_unstale":"2024-09-11T22:22:16.059011+0000","last_undegraded":"2024-09-11T22:22:16.059011+0000","last_fullsized":"2024-09-11T22:22:16.059011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:58.240535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234846+0000","last_change":"2024-09-11T22:20:18.184326+0000","last_active":"2024-09-11T22:22:16.234846+0000","last_peered":"2024-09-11T22:22:16.234846+0000","last_clean":"2024-09-11T22:22:16.234846+0000","last_became_active":"2024-09-11T22:20:18.183995+0000","last_became_peered":"2024-09-11T22:20:18.183995+0000","last_unstale":"2024-09-11T22:22:16.234846+0000","last_undegraded":"2024-09-11T22:22:16.234846+0000","last_fullsized":"2024-09-11T22:22:16.234846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:33.468000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977256+0000","last_change":"2024-09-11T22:20:16.127180+0000","last_active":"2024-09-11T22:22:15.977256+0000","last_peered":"2024-09-11T22:22:15.977256+0000","last_clean":"2024-09-11T22:22:15.977256+0000","last_became_active":"2024-09-11T22:20:16.126696+0000","last_became_peered":"2024-09-11T22:20:16.126696+0000","last_unstale":"2024-09-11T22:22:15.977256+0000","last_undegraded":"2024-09-11T22:22:15.977256+0000","last_fullsized":"2024-09-11T22:22:15.977256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:02.637225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973369+0000","last_change":"2024-09-11T22:21:18.188614+0000","last_active":"2024-09-11T22:22:15.973369+0000","last_peered":"2024-09-11T22:22:15.973369+0000","last_clean":"2024-09-11T22:22:15.973369+0000","last_became_active":"2024-09-11T22:21:18.188277+0000","last_became_peered":"2024-09-11T22:21:18.188277+0000","last_unstale":"2024-09-11T22:22:15.973369+0000","last_undegraded":"2024-09-11T22:22:15.973369+0000","last_fullsized":"2024-09-11T22:22:15.973369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:59.995647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058897+0000","last_change":"2024-09-11T22:20:16.128311+0000","last_active":"2024-09-11T22:22:16.058897+0000","last_peered":"2024-09-11T22:22:16.058897+0000","last_clean":"2024-09-11T22:22:16.058897+0000","last_became_active":"2024-09-11T22:20:16.127853+0000","last_became_peered":"2024-09-11T22:20:16.127853+0000","last_unstale":"2024-09-11T22:22:16.058897+0000","last_undegraded":"2024-09-11T22:22:16.058897+0000","last_fullsized":"2024-09-11T22:22:16.058897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:57.562470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231820+0000","last_change":"2024-09-11T22:20:18.195782+0000","last_active":"2024-09-11T22:22:16.231820+0000","last_peered":"2024-09-11T22:22:16.231820+0000","last_clean":"2024-09-11T22:22:16.231820+0000","last_became_active":"2024-09-11T22:20:18.195365+0000","last_became_peered":"2024-09-11T22:20:18.195365+0000","last_unstale":"2024-09-11T22:22:16.231820+0000","last_undegraded":"2024-09-11T22:22:16.231820+0000","last_fullsized":"2024-09-11T22:22:16.231820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:49.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065812+0000","last_change":"2024-09-11T22:20:18.199344+0000","last_active":"2024-09-11T22:22:16.065812+0000","last_peered":"2024-09-11T22:22:16.065812+0000","last_clean":"2024-09-11T22:22:16.065812+0000","last_became_active":"2024-09-11T22:20:18.199016+0000","last_became_peered":"2024-09-11T22:20:18.199016+0000","last_unstale":"2024-09-11T22:22:16.065812+0000","last_undegraded":"2024-09-11T22:22:16.065812+0000","last_fullsized":"2024-09-11T22:22:16.065812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:38.322831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059125+0000","last_change":"2024-09-11T22:20:16.122236+0000","last_active":"2024-09-11T22:22:16.059125+0000","last_peered":"2024-09-11T22:22:16.059125+0000","last_clean":"2024-09-11T22:22:16.059125+0000","last_became_active":"2024-09-11T22:20:16.121867+0000","last_became_peered":"2024-09-11T22:20:16.121867+0000","last_unstale":"2024-09-11T22:22:16.059125+0000","last_undegraded":"2024-09-11T22:22:16.059125+0000","last_fullsized":"2024-09-11T22:22:16.059125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:56.782445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975374+0000","last_change":"2024-09-11T22:20:18.187338+0000","last_active":"2024-09-11T22:22:15.975374+0000","last_peered":"2024-09-11T22:22:15.975374+0000","last_clean":"2024-09-11T22:22:15.975374+0000","last_became_active":"2024-09-11T22:20:18.186985+0000","last_became_peered":"2024-09-11T22:20:18.186985+0000","last_unstale":"2024-09-11T22:22:15.975374+0000","last_undegraded":"2024-09-11T22:22:15.975374+0000","last_fullsized":"2024-09-11T22:22:15.975374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:58.170680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059106+0000","last_change":"2024-09-11T22:20:16.128579+0000","last_active":"2024-09-11T22:22:16.059106+0000","last_peered":"2024-09-11T22:22:16.059106+0000","last_clean":"2024-09-11T22:22:16.059106+0000","last_became_active":"2024-09-11T22:20:16.127864+0000","last_became_peered":"2024-09-11T22:20:16.127864+0000","last_unstale":"2024-09-11T22:22:16.059106+0000","last_undegraded":"2024-09-11T22:22:16.059106+0000","last_fullsized":"2024-09-11T22:22:16.059106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:11:32.005487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234449+0000","last_change":"2024-09-11T22:20:18.185205+0000","last_active":"2024-09-11T22:22:16.234449+0000","last_peered":"2024-09-11T22:22:16.234449+0000","last_clean":"2024-09-11T22:22:16.234449+0000","last_became_active":"2024-09-11T22:20:18.184402+0000","last_became_peered":"2024-09-11T22:20:18.184402+0000","last_unstale":"2024-09-11T22:22:16.234449+0000","last_undegraded":"2024-09-11T22:22:16.234449+0000","last_fullsized":"2024-09-11T22:22:16.234449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:17:48.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.060208+0000","last_change":"2024-09-11T22:20:16.186372+0000","last_active":"2024-09-11T22:22:16.060208+0000","last_peered":"2024-09-11T22:22:16.060208+0000","last_clean":"2024-09-11T22:22:16.060208+0000","last_became_active":"2024-09-11T22:20:16.185990+0000","last_became_peered":"2024-09-11T22:20:16.185990+0000","last_unstale":"2024-09-11T22:22:16.060208+0000","last_undegraded":"2024-09-11T22:22:16.060208+0000","last_fullsized":"2024-09-11T22:22:16.060208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:23.296379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059971+0000","last_change":"2024-09-11T22:20:18.190670+0000","last_active":"2024-09-11T22:22:16.059971+0000","last_peered":"2024-09-11T22:22:16.059971+0000","last_clean":"2024-09-11T22:22:16.059971+0000","last_became_active":"2024-09-11T22:20:18.190339+0000","last_became_peered":"2024-09-11T22:20:18.190339+0000","last_unstale":"2024-09-11T22:22:16.059971+0000","last_undegraded":"2024-09-11T22:22:16.059971+0000","last_fullsized":"2024-09-11T22:22:16.059971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.085305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":146,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973670+0000","last_change":"2024-09-11T22:20:16.117646+0000","last_active":"2024-09-11T22:22:15.973670+0000","last_peered":"2024-09-11T22:22:15.973670+0000","last_clean":"2024-09-11T22:22:15.973670+0000","last_became_active":"2024-09-11T22:20:16.117111+0000","last_became_peered":"2024-09-11T22:20:16.117111+0000","last_unstale":"2024-09-11T22:22:15.973670+0000","last_undegraded":"2024-09-11T22:22:15.973670+0000","last_fullsized":"2024-09-11T22:22:15.973670+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:36.470158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":600132,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1158,"num_read_kb":717,"num_write":372,"num_write_kb":1463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1192,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1800396,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108858,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":443740,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1085440,"data_stored":1226275,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972399,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":443255,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201332,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":950272,"data_stored":836827,"data_compressed":27823,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430266,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":983040,"data_stored":1037781,"data_compressed":44963,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443506,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822724,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":1167739,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986249,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":445182,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":433812,"data_compressed":5772,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":991232,"data_stored":503564,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":987136,"data_stored":510899,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":937984,"data_stored":444704,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":67983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135121,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":200602,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:22:21.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:22:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:22.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:22 smithi005 ceph-mon[30943]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:22.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:22 smithi005 ceph-mon[30943]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:22.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:22 smithi005 ceph-mon[30943]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:22.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:22 smithi159 ceph-mon[35534]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:22.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:22 smithi159 ceph-mon[35534]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:22.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:22 smithi159 ceph-mon[35534]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:22.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:22 smithi172 ceph-mon[35461]: from='client.15303 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:22.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:22 smithi172 ceph-mon[35461]: from='client.15309 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:22.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:22 smithi172 ceph-mon[35461]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.435+0000 7fa2b5824640 1 -- 172.21.15.5:0/3656840542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b010e3c0 msgr2=0x7fa2b0110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.435+0000 7fa2b5824640 1 --2- 172.21.15.5:0/3656840542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b010e3c0 0x7fa2b0110810 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009ca0 tx=0x7fa2a002a950 comp rx=0 tx=0).stop 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.435+0000 7fa2b5824640 1 -- 172.21.15.5:0/3656840542 shutdown_connections 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.435+0000 7fa2b5824640 1 --2- 172.21.15.5:0/3656840542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b010e3c0 0x7fa2b0110810 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.435+0000 7fa2b5824640 1 --2- 172.21.15.5:0/3656840542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2b0074f40 0x7fa2b010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.435+0000 7fa2b5824640 1 --2- 172.21.15.5:0/3656840542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2b0074620 0x7fa2b0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.435+0000 7fa2b5824640 1 -- 172.21.15.5:0/3656840542 >> 172.21.15.5:0/3656840542 conn(0x7fa2b00fd8c0 msgr2=0x7fa2b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.435+0000 7fa2b5824640 1 -- 172.21.15.5:0/3656840542 shutdown_connections 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.435+0000 7fa2b5824640 1 -- 172.21.15.5:0/3656840542 wait complete. 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2b5824640 1 Processor -- start 2024-09-11T22:22:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2b5824640 1 -- start start 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2b5824640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2b0074620 0x7fa2b010dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2b5824640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b0074f40 0x7fa2b010ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2b5824640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2b010e3c0 0x7fa2b010b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2b5824640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0114150 con 0x7fa2b0074f40 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2b5824640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0113fd0 con 0x7fa2b010e3c0 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2b5824640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01142d0 con 0x7fa2b0074620 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b0074f40 0x7fa2b010ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b0074f40 0x7fa2b010ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57390/0 (socket says 172.21.15.5:57390) 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2ae7fc640 1 -- 172.21.15.5:0/1903757289 learned_addr learned my addr 172.21.15.5:0/1903757289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2ae7fc640 1 -- 172.21.15.5:0/1903757289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2b0074620 msgr2=0x7fa2b010dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2ae7fc640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2b0074620 0x7fa2b010dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:22.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2ae7fc640 1 -- 172.21.15.5:0/1903757289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2b010e3c0 msgr2=0x7fa2b010b270 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:22:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2ae7fc640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2b010e3c0 0x7fa2b010b270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.436+0000 7fa2ae7fc640 1 -- 172.21.15.5:0/1903757289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b010bb00 con 0x7fa2b0074f40 2024-09-11T22:22:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.437+0000 7fa2ae7fc640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b0074f40 0x7fa2b010ad30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa29800d950 tx=0x7fa29800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.437+0000 7fa2b4822640 1 -- 172.21.15.5:0/1903757289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298017070 con 0x7fa2b0074f40 2024-09-11T22:22:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.437+0000 7fa2b5824640 1 -- 172.21.15.5:0/1903757289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b01a06d0 con 0x7fa2b0074f40 2024-09-11T22:22:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.437+0000 7fa2b4822640 1 -- 172.21.15.5:0/1903757289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa298009c30 con 0x7fa2b0074f40 2024-09-11T22:22:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.437+0000 7fa2b4822640 1 -- 172.21.15.5:0/1903757289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29800f6b0 con 0x7fa2b0074f40 2024-09-11T22:22:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.437+0000 7fa2b5824640 1 -- 172.21.15.5:0/1903757289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b01a0c60 con 0x7fa2b0074f40 2024-09-11T22:22:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.438+0000 7fa2b5824640 1 -- 172.21.15.5:0/1903757289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b0075cd0 con 0x7fa2b0074f40 2024-09-11T22:22:22.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.439+0000 7fa2b4822640 1 -- 172.21.15.5:0/1903757289 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7fa2980109d0 con 0x7fa2b0074f40 2024-09-11T22:22:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.440+0000 7fa2b4822640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa29c0a2e10 0x7fa29c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.440+0000 7fa2aeffd640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa29c0a2e10 0x7fa29c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.440+0000 7fa2b4822640 1 -- 172.21.15.5:0/1903757289 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa2980cf7b0 con 0x7fa2b0074f40 2024-09-11T22:22:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.440+0000 7fa2aeffd640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa29c0a2e10 0x7fa29c0a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800a990 tx=0x7fa2a8005c70 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.443+0000 7fa2b4822640 1 -- 172.21.15.5:0/1903757289 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa298009de0 con 0x7fa2b0074f40 2024-09-11T22:22:22.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.653+0000 7fa2b5824640 1 -- 172.21.15.5:0/1903757289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2b01a0fb0 con 0x7fa2b0074f40 2024-09-11T22:22:22.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.654+0000 7fa2b4822640 1 -- 172.21.15.5:0/1903757289 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7fa29809c180 con 0x7fa2b0074f40 2024-09-11T22:22:22.656 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:22.656 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-11T22:22:15:230183+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:22:15.180031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:22:22.656 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-11T22:22:22.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.656+0000 7fa2b5824640 1 -- 172.21.15.5:0/1903757289 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa29c0a2e10 msgr2=0x7fa29c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:22.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.656+0000 7fa2b5824640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa29c0a2e10 0x7fa29c0a52d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800a990 tx=0x7fa2a8005c70 comp rx=0 tx=0).stop 2024-09-11T22:22:22.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.656+0000 7fa2b5824640 1 -- 172.21.15.5:0/1903757289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b0074f40 msgr2=0x7fa2b010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:22.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.656+0000 7fa2b5824640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b0074f40 0x7fa2b010ad30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa29800d950 tx=0x7fa29800de10 comp rx=0 tx=0).stop 2024-09-11T22:22:22.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.656+0000 7fa2b5824640 1 -- 172.21.15.5:0/1903757289 shutdown_connections 2024-09-11T22:22:22.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.656+0000 7fa2b5824640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa29c0a2e10 0x7fa29c0a52d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:22.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.656+0000 7fa2b5824640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2b010e3c0 0x7fa2b010b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:22.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.656+0000 7fa2b5824640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b0074f40 0x7fa2b010ad30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:22.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.656+0000 7fa2b5824640 1 --2- 172.21.15.5:0/1903757289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2b0074620 0x7fa2b010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:22.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.656+0000 7fa2b5824640 1 -- 172.21.15.5:0/1903757289 >> 172.21.15.5:0/1903757289 conn(0x7fa2b00fd8c0 msgr2=0x7fa2b00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:22.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.657+0000 7fa2b5824640 1 -- 172.21.15.5:0/1903757289 shutdown_connections 2024-09-11T22:22:22.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:22.657+0000 7fa2b5824640 1 -- 172.21.15.5:0/1903757289 wait complete. 2024-09-11T22:22:23.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:22:23.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:23 smithi172 ceph-mon[35461]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:23.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:23 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1903757289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:23.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:23 smithi005 ceph-mon[30943]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:23.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:23 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1903757289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:23 smithi159 ceph-mon[35534]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:22:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:23 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1903757289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.160+0000 7f8523c5c640 1 -- 172.21.15.5:0/3025155480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f851c101de0 msgr2=0x7f851c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.160+0000 7f8523c5c640 1 --2- 172.21.15.5:0/3025155480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f851c101de0 0x7f851c1021c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8518009900 tx=0x7f851802a410 comp rx=0 tx=0).stop 2024-09-11T22:22:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 -- 172.21.15.5:0/3025155480 shutdown_connections 2024-09-11T22:22:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 --2- 172.21.15.5:0/3025155480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f851c106f10 0x7f851c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 --2- 172.21.15.5:0/3025155480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f851c102790 0x7f851c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 --2- 172.21.15.5:0/3025155480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f851c101de0 0x7f851c1021c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 -- 172.21.15.5:0/3025155480 >> 172.21.15.5:0/3025155480 conn(0x7f851c0fd880 msgr2=0x7f851c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 -- 172.21.15.5:0/3025155480 shutdown_connections 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 -- 172.21.15.5:0/3025155480 wait complete. 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 Processor -- start 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 -- start start 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f851c101de0 0x7f851c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f851c102790 0x7f851c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f851c106f10 0x7f851c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.161+0000 7f8523c5c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f851c111f70 con 0x7f851c106f10 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f8523c5c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f851c111df0 con 0x7f851c102790 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f8523c5c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f851c1120f0 con 0x7f851c101de0 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85221f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f851c106f10 0x7f851c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85221f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f851c106f10 0x7f851c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57412/0 (socket says 172.21.15.5:57412) 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85219f8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f851c101de0 0x7f851c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85221f9640 1 -- 172.21.15.5:0/2433204431 learned_addr learned my addr 172.21.15.5:0/2433204431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85211f7640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f851c102790 0x7f851c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85221f9640 1 -- 172.21.15.5:0/2433204431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f851c101de0 msgr2=0x7f851c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85221f9640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f851c101de0 0x7f851c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85221f9640 1 -- 172.21.15.5:0/2433204431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f851c102790 msgr2=0x7f851c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85221f9640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f851c102790 0x7f851c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85221f9640 1 -- 172.21.15.5:0/2433204431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c1a6880 con 0x7f851c106f10 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f85221f9640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f851c106f10 0x7f851c074f70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f851400cce0 tx=0x7f851400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f8512ffd640 1 -- 172.21.15.5:0/2433204431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851400f450 con 0x7f851c106f10 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f8512ffd640 1 -- 172.21.15.5:0/2433204431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f851400fad0 con 0x7f851c106f10 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f8512ffd640 1 -- 172.21.15.5:0/2433204431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514018c10 con 0x7f851c106f10 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f8523c5c640 1 -- 172.21.15.5:0/2433204431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c1a6b70 con 0x7f851c106f10 2024-09-11T22:22:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.162+0000 7f8523c5c640 1 -- 172.21.15.5:0/2433204431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c1a70b0 con 0x7f851c106f10 2024-09-11T22:22:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.163+0000 7f8523c5c640 1 -- 172.21.15.5:0/2433204431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851c1021c0 con 0x7f851c106f10 2024-09-11T22:22:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.165+0000 7f8512ffd640 1 -- 172.21.15.5:0/2433204431 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f851400f5f0 con 0x7f851c106f10 2024-09-11T22:22:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.165+0000 7f8512ffd640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85080a2e90 0x7f85080a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.165+0000 7f85219f8640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85080a2e90 0x7f85080a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.165+0000 7f8512ffd640 1 -- 172.21.15.5:0/2433204431 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f85140cfcb0 con 0x7f851c106f10 2024-09-11T22:22:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.166+0000 7f85219f8640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85080a2e90 0x7f85080a5350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8518009230 tx=0x7f85180058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.169+0000 7f8512ffd640 1 -- 172.21.15.5:0/2433204431 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8514014030 con 0x7f851c106f10 2024-09-11T22:22:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.348+0000 7f8523c5c640 1 -- 172.21.15.5:0/2433204431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f851c109b60 con 0x7f851c106f10 2024-09-11T22:22:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.349+0000 7f8512ffd640 1 -- 172.21.15.5:0/2433204431 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23068 (secure 0 0 0) 0x7f851409c680 con 0x7f851c106f10 2024-09-11T22:22:24.350 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:24.350 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:22:15.968337+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2380674497":"2024-09-12T22:22:15.968309+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/3591749770":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/1577283141":"2024-09-12T22:22:15.968309+0000","172.21.15.5:6801/3591749770":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1618823752":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/785116270":"2024-09-12T22:22:15.968309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:22:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 -- 172.21.15.5:0/2433204431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85080a2e90 msgr2=0x7f85080a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85080a2e90 0x7f85080a5350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8518009230 tx=0x7f85180058d0 comp rx=0 tx=0).stop 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 -- 172.21.15.5:0/2433204431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f851c106f10 msgr2=0x7f851c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f851c106f10 0x7f851c074f70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f851400cce0 tx=0x7f851400a620 comp rx=0 tx=0).stop 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 -- 172.21.15.5:0/2433204431 shutdown_connections 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f851c106f10 0x7f851c074f70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85080a2e90 0x7f85080a5350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f851c102790 0x7f851c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 --2- 172.21.15.5:0/2433204431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f851c101de0 0x7f851c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 -- 172.21.15.5:0/2433204431 >> 172.21.15.5:0/2433204431 conn(0x7f851c0fd880 msgr2=0x7f851c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 -- 172.21.15.5:0/2433204431 shutdown_connections 2024-09-11T22:22:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:24.352+0000 7f8523c5c640 1 -- 172.21.15.5:0/2433204431 wait complete. 2024-09-11T22:22:25.092 DEBUG:tasks.fwd_scrub:Yielding 2024-09-11T22:22:25.092 INFO:teuthology.run_tasks:Running task exec... 2024-09-11T22:22:25.102 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T22:22:25.102 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T22:22:25.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-11T22:22:25.103 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-11T22:22:25.103 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:22:25.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.214+0000 7f73f0bb9640 1 -- 172.21.15.5:0/430259127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ec102d20 msgr2=0x7f73ec103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.214+0000 7f73f0bb9640 1 --2- 172.21.15.5:0/430259127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ec102d20 0x7f73ec103100 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009900 tx=0x7f73dc02a410 comp rx=0 tx=0).stop 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.214+0000 7f73f0bb9640 1 -- 172.21.15.5:0/430259127 shutdown_connections 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.214+0000 7f73f0bb9640 1 --2- 172.21.15.5:0/430259127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec10c0f0 0x7f73ec10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.214+0000 7f73f0bb9640 1 --2- 172.21.15.5:0/430259127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ec103640 0x7f73ec10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.214+0000 7f73f0bb9640 1 --2- 172.21.15.5:0/430259127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ec102d20 0x7f73ec103100 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.214+0000 7f73f0bb9640 1 -- 172.21.15.5:0/430259127 >> 172.21.15.5:0/430259127 conn(0x7f73ec0fc6b0 msgr2=0x7f73ec0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.214+0000 7f73f0bb9640 1 -- 172.21.15.5:0/430259127 shutdown_connections 2024-09-11T22:22:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.214+0000 7f73f0bb9640 1 -- 172.21.15.5:0/430259127 wait complete. 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73f0bb9640 1 Processor -- start 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73f0bb9640 1 -- start start 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73f0bb9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ec102d20 0x7f73ec198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73f0bb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ec103640 0x7f73ec1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73f0bb9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec10c0f0 0x7f73ec19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73f0bb9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec111f00 con 0x7f73ec103640 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73f0bb9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec111d80 con 0x7f73ec102d20 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73f0bb9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f73ec112080 con 0x7f73ec10c0f0 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73e1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ec103640 0x7f73ec1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73e1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ec103640 0x7f73ec1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57434/0 (socket says 172.21.15.5:57434) 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73e1d9b640 1 -- 172.21.15.5:0/2156976278 learned_addr learned my addr 172.21.15.5:0/2156976278 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73ea59c640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ec102d20 0x7f73ec198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.215+0000 7f73ead9d640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec10c0f0 0x7f73ec19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73e1d9b640 1 -- 172.21.15.5:0/2156976278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec10c0f0 msgr2=0x7f73ec19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73e1d9b640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec10c0f0 0x7f73ec19cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73e1d9b640 1 -- 172.21.15.5:0/2156976278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ec102d20 msgr2=0x7f73ec198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73e1d9b640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ec102d20 0x7f73ec198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73e1d9b640 1 -- 172.21.15.5:0/2156976278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73ec19d380 con 0x7f73ec103640 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73ead9d640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec10c0f0 0x7f73ec19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73e1d9b640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ec103640 0x7f73ec1988d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f73d400c890 tx=0x7f73d400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73e3fff640 1 -- 172.21.15.5:0/2156976278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4004220 con 0x7f73ec103640 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73f0bb9640 1 -- 172.21.15.5:0/2156976278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73ec19d670 con 0x7f73ec103640 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73e3fff640 1 -- 172.21.15.5:0/2156976278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d40043c0 con 0x7f73ec103640 2024-09-11T22:22:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73e3fff640 1 -- 172.21.15.5:0/2156976278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4004a00 con 0x7f73ec103640 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.216+0000 7f73f0bb9640 1 -- 172.21.15.5:0/2156976278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73ec1a4f00 con 0x7f73ec103640 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.217+0000 7f73f0bb9640 1 -- 172.21.15.5:0/2156976278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73ec103100 con 0x7f73ec103640 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.219+0000 7f73e3fff640 1 -- 172.21.15.5:0/2156976278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f73d40079b0 con 0x7f73ec103640 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.219+0000 7f73e3fff640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73d80a2ec0 0x7f73d80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.220+0000 7f73ea59c640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73d80a2ec0 0x7f73d80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.220+0000 7f73e3fff640 1 -- 172.21.15.5:0/2156976278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f73d40cf7d0 con 0x7f73ec103640 2024-09-11T22:22:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.220+0000 7f73ea59c640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73d80a2ec0 0x7f73d80a5380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f73dc000c00 tx=0x7f73dc0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.227+0000 7f73e3fff640 1 -- 172.21.15.5:0/2156976278 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73d4014030 con 0x7f73ec103640 2024-09-11T22:22:25.380 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:25.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.416+0000 7f73f0bb9640 1 -- 172.21.15.5:0/2156976278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f73ec1996e0 con 0x7f73ec103640 2024-09-11T22:22:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.420+0000 7f73e3fff640 1 -- 172.21.15.5:0/2156976278 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f73ec1996e0 con 0x7f73ec103640 2024-09-11T22:22:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.422+0000 7f73f0bb9640 1 -- 172.21.15.5:0/2156976278 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73d80a2ec0 msgr2=0x7f73d80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.422+0000 7f73f0bb9640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73d80a2ec0 0x7f73d80a5380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f73dc000c00 tx=0x7f73dc0058d0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.422+0000 7f73f0bb9640 1 -- 172.21.15.5:0/2156976278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ec103640 msgr2=0x7f73ec1988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.422+0000 7f73f0bb9640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ec103640 0x7f73ec1988d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f73d400c890 tx=0x7f73d400cd50 comp rx=0 tx=0).stop 2024-09-11T22:22:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.422+0000 7f73f0bb9640 1 -- 172.21.15.5:0/2156976278 shutdown_connections 2024-09-11T22:22:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.422+0000 7f73f0bb9640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73ec10c0f0 0x7f73ec19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.422+0000 7f73f0bb9640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73ec103640 0x7f73ec1988d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.422+0000 7f73f0bb9640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73d80a2ec0 0x7f73d80a5380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.422+0000 7f73f0bb9640 1 --2- 172.21.15.5:0/2156976278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73ec102d20 0x7f73ec198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.422+0000 7f73f0bb9640 1 -- 172.21.15.5:0/2156976278 >> 172.21.15.5:0/2156976278 conn(0x7f73ec0fc6b0 msgr2=0x7f73ec0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.423+0000 7f73f0bb9640 1 -- 172.21.15.5:0/2156976278 shutdown_connections 2024-09-11T22:22:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.423+0000 7f73f0bb9640 1 -- 172.21.15.5:0/2156976278 wait complete. 2024-09-11T22:22:25.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-11T22:22:25.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.542+0000 7f4277817640 1 -- 172.21.15.5:0/2272536456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270102cc0 msgr2=0x7f42701030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.542+0000 7f4277817640 1 --2- 172.21.15.5:0/2272536456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270102cc0 0x7f42701030a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f426c009900 tx=0x7f426c02a480 comp rx=0 tx=0).stop 2024-09-11T22:22:25.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.542+0000 7f4277817640 1 -- 172.21.15.5:0/2272536456 shutdown_connections 2024-09-11T22:22:25.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.542+0000 7f4277817640 1 --2- 172.21.15.5:0/2272536456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f427010c110 0x7f427010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.542+0000 7f4277817640 1 --2- 172.21.15.5:0/2272536456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42701035e0 0x7f427010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.542+0000 7f4277817640 1 --2- 172.21.15.5:0/2272536456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270102cc0 0x7f42701030a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.542+0000 7f4277817640 1 -- 172.21.15.5:0/2272536456 >> 172.21.15.5:0/2272536456 conn(0x7f42700fc6b0 msgr2=0x7f42700fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.543+0000 7f4277817640 1 -- 172.21.15.5:0/2272536456 shutdown_connections 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.543+0000 7f4277817640 1 -- 172.21.15.5:0/2272536456 wait complete. 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.543+0000 7f4277817640 1 Processor -- start 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.543+0000 7f4277817640 1 -- start start 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.543+0000 7f4277817640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270102cc0 0x7f427019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.543+0000 7f4277817640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42701035e0 0x7f427019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.543+0000 7f4277817640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f427010c110 0x7f427019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.543+0000 7f4277817640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4270111ef0 con 0x7f42701035e0 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.543+0000 7f4277817640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4270111d70 con 0x7f427010c110 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.543+0000 7f4277817640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4270112070 con 0x7f4270102cc0 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4274db2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42701035e0 0x7f427019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f42755b3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270102cc0 0x7f427019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4274db2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42701035e0 0x7f427019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57460/0 (socket says 172.21.15.5:57460) 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4274db2640 1 -- 172.21.15.5:0/1555165586 learned_addr learned my addr 172.21.15.5:0/1555165586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4275db4640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f427010c110 0x7f427019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4274db2640 1 -- 172.21.15.5:0/1555165586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270102cc0 msgr2=0x7f427019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4274db2640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270102cc0 0x7f427019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4274db2640 1 -- 172.21.15.5:0/1555165586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f427010c110 msgr2=0x7f427019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4274db2640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f427010c110 0x7f427019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4274db2640 1 -- 172.21.15.5:0/1555165586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427019f600 con 0x7f42701035e0 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4274db2640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42701035e0 0x7f427019ab50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f426000d740 tx=0x7f426000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f42667fc640 1 -- 172.21.15.5:0/1555165586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260017070 con 0x7f42701035e0 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f42667fc640 1 -- 172.21.15.5:0/1555165586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f426000de50 con 0x7f42701035e0 2024-09-11T22:22:25.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4277817640 1 -- 172.21.15.5:0/1555165586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427019f8f0 con 0x7f42701035e0 2024-09-11T22:22:25.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f42667fc640 1 -- 172.21.15.5:0/1555165586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426000f620 con 0x7f42701035e0 2024-09-11T22:22:25.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.544+0000 7f4277817640 1 -- 172.21.15.5:0/1555165586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42701a7180 con 0x7f42701035e0 2024-09-11T22:22:25.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.545+0000 7f4277817640 1 -- 172.21.15.5:0/1555165586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42701030a0 con 0x7f42701035e0 2024-09-11T22:22:25.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.547+0000 7f42667fc640 1 -- 172.21.15.5:0/1555165586 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f4260009c30 con 0x7f42701035e0 2024-09-11T22:22:25.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.547+0000 7f42667fc640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2ec0 0x7f425c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.547+0000 7f42755b3640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2ec0 0x7f425c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.548+0000 7f42667fc640 1 -- 172.21.15.5:0/1555165586 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f42600cf270 con 0x7f42701035e0 2024-09-11T22:22:25.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.548+0000 7f42755b3640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2ec0 0x7f425c0a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f426c009230 tx=0x7f426c0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.551+0000 7f42667fc640 1 -- 172.21.15.5:0/1555165586 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f426009b590 con 0x7f42701035e0 2024-09-11T22:22:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:25 smithi172 ceph-mon[35461]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-09-11T22:22:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:25 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2433204431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:25.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:25 smithi005 ceph-mon[30943]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-09-11T22:22:25.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:25 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2433204431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.739+0000 7f4277817640 1 -- 172.21.15.5:0/1555165586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f42701a73b0 con 0x7f42701035e0 2024-09-11T22:22:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.742+0000 7f42667fc640 1 -- 172.21.15.5:0/1555165586 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7f426009f890 con 0x7f42701035e0 2024-09-11T22:22:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 -- 172.21.15.5:0/1555165586 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2ec0 msgr2=0x7f425c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2ec0 0x7f425c0a5380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f426c009230 tx=0x7f426c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 -- 172.21.15.5:0/1555165586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42701035e0 msgr2=0x7f427019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42701035e0 0x7f427019ab50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f426000d740 tx=0x7f426000dc00 comp rx=0 tx=0).stop 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 -- 172.21.15.5:0/1555165586 shutdown_connections 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2ec0 0x7f425c0a5380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f427010c110 0x7f427019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42701035e0 0x7f427019ab50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 --2- 172.21.15.5:0/1555165586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270102cc0 0x7f427019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 -- 172.21.15.5:0/1555165586 >> 172.21.15.5:0/1555165586 conn(0x7f42700fc6b0 msgr2=0x7f427010d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 -- 172.21.15.5:0/1555165586 shutdown_connections 2024-09-11T22:22:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.745+0000 7f4277817640 1 -- 172.21.15.5:0/1555165586 wait complete. 2024-09-11T22:22:25.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-11T22:22:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:25 smithi159 ceph-mon[35534]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 1.1 KiB/s wr, 26 op/s 2024-09-11T22:22:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:25 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2433204431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:25.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.867+0000 7fda87a35640 1 -- 172.21.15.5:0/423635710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda80107b80 msgr2=0x7fda80107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.867+0000 7fda87a35640 1 --2- 172.21.15.5:0/423635710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda80107b80 0x7fda80107f60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009930 tx=0x7fda7c02a420 comp rx=0 tx=0).stop 2024-09-11T22:22:25.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.867+0000 7fda87a35640 1 -- 172.21.15.5:0/423635710 shutdown_connections 2024-09-11T22:22:25.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.867+0000 7fda87a35640 1 --2- 172.21.15.5:0/423635710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda80102480 0x7fda8010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.867+0000 7fda87a35640 1 --2- 172.21.15.5:0/423635710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda80101ae0 0x7fda80101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.867+0000 7fda87a35640 1 --2- 172.21.15.5:0/423635710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda80107b80 0x7fda80107f60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.867+0000 7fda87a35640 1 -- 172.21.15.5:0/423635710 >> 172.21.15.5:0/423635710 conn(0x7fda800fd7e0 msgr2=0x7fda800ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.867+0000 7fda87a35640 1 -- 172.21.15.5:0/423635710 shutdown_connections 2024-09-11T22:22:25.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.867+0000 7fda87a35640 1 -- 172.21.15.5:0/423635710 wait complete. 2024-09-11T22:22:25.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda87a35640 1 Processor -- start 2024-09-11T22:22:25.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda87a35640 1 -- start start 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda87a35640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda80101ae0 0x7fda80198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda87a35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda80102480 0x7fda80198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda87a35640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda80107b80 0x7fda8019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda87a35640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda80111f30 con 0x7fda80102480 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda87a35640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda80111db0 con 0x7fda80107b80 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda87a35640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda801120b0 con 0x7fda80101ae0 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda84fd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda80102480 0x7fda80198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda84fd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda80102480 0x7fda80198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57472/0 (socket says 172.21.15.5:57472) 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda84fd0640 1 -- 172.21.15.5:0/842385552 learned_addr learned my addr 172.21.15.5:0/842385552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda857d1640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda80101ae0 0x7fda80198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda85fd2640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda80107b80 0x7fda8019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda84fd0640 1 -- 172.21.15.5:0/842385552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda80101ae0 msgr2=0x7fda80198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda84fd0640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda80101ae0 0x7fda80198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda84fd0640 1 -- 172.21.15.5:0/842385552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda80107b80 msgr2=0x7fda8019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda84fd0640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda80107b80 0x7fda8019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda84fd0640 1 -- 172.21.15.5:0/842385552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8019d330 con 0x7fda80102480 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.868+0000 7fda85fd2640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda80107b80 0x7fda8019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.869+0000 7fda84fd0640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda80102480 0x7fda80198880 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fda7400c890 tx=0x7fda7400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.869+0000 7fda727fc640 1 -- 172.21.15.5:0/842385552 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74004220 con 0x7fda80102480 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.869+0000 7fda727fc640 1 -- 172.21.15.5:0/842385552 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda740043c0 con 0x7fda80102480 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.869+0000 7fda87a35640 1 -- 172.21.15.5:0/842385552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8019d620 con 0x7fda80102480 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.869+0000 7fda727fc640 1 -- 172.21.15.5:0/842385552 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740049e0 con 0x7fda80102480 2024-09-11T22:22:25.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.869+0000 7fda87a35640 1 -- 172.21.15.5:0/842385552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda801a4eb0 con 0x7fda80102480 2024-09-11T22:22:25.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.870+0000 7fda87a35640 1 -- 172.21.15.5:0/842385552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda80101f40 con 0x7fda80102480 2024-09-11T22:22:25.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.871+0000 7fda727fc640 1 -- 172.21.15.5:0/842385552 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7fda740079b0 con 0x7fda80102480 2024-09-11T22:22:25.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.872+0000 7fda727fc640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda6c0a2f40 0x7fda6c0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.872+0000 7fda727fc640 1 -- 172.21.15.5:0/842385552 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fda740cf6e0 con 0x7fda80102480 2024-09-11T22:22:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.872+0000 7fda857d1640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda6c0a2f40 0x7fda6c0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.872+0000 7fda857d1640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda6c0a2f40 0x7fda6c0a5400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009900 tx=0x7fda7c0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.876+0000 7fda727fc640 1 -- 172.21.15.5:0/842385552 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda74014030 con 0x7fda80102480 2024-09-11T22:22:25.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.890+0000 7f67c45da640 1 -- 172.21.15.5:0/4275516812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67bc102db0 msgr2=0x7f67bc103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.890+0000 7f67c45da640 1 --2- 172.21.15.5:0/4275516812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67bc102db0 0x7f67bc103190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009930 tx=0x7f67b802a420 comp rx=0 tx=0).stop 2024-09-11T22:22:25.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.890+0000 7f67c45da640 1 -- 172.21.15.5:0/4275516812 shutdown_connections 2024-09-11T22:22:25.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.890+0000 7f67c45da640 1 --2- 172.21.15.5:0/4275516812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67bc1082e0 0x7f67bc10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.890+0000 7f67c45da640 1 --2- 172.21.15.5:0/4275516812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67bc103760 0x7f67bc107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.890+0000 7f67c45da640 1 --2- 172.21.15.5:0/4275516812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67bc102db0 0x7f67bc103190 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.890+0000 7f67c45da640 1 -- 172.21.15.5:0/4275516812 >> 172.21.15.5:0/4275516812 conn(0x7f67bc0fc7d0 msgr2=0x7f67bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.890+0000 7f67c45da640 1 -- 172.21.15.5:0/4275516812 shutdown_connections 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.890+0000 7f67c45da640 1 -- 172.21.15.5:0/4275516812 wait complete. 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c45da640 1 Processor -- start 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c45da640 1 -- start start 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c45da640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67bc102db0 0x7f67bc100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c45da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67bc103760 0x7f67bc100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c45da640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67bc1082e0 0x7f67bc101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c45da640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc112090 con 0x7f67bc103760 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c45da640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc111f10 con 0x7f67bc1082e0 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c45da640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc112210 con 0x7f67bc102db0 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c1b75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67bc103760 0x7f67bc100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c1b75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67bc103760 0x7f67bc100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57490/0 (socket says 172.21.15.5:57490) 2024-09-11T22:22:25.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c1b75640 1 -- 172.21.15.5:0/4182141678 learned_addr learned my addr 172.21.15.5:0/4182141678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c2b77640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67bc1082e0 0x7f67bc101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.891+0000 7f67c2376640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67bc102db0 0x7f67bc100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67c1b75640 1 -- 172.21.15.5:0/4182141678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67bc102db0 msgr2=0x7f67bc100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67c1b75640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67bc102db0 0x7f67bc100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67c1b75640 1 -- 172.21.15.5:0/4182141678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67bc1082e0 msgr2=0x7f67bc101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67c1b75640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67bc1082e0 0x7f67bc101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67c1b75640 1 -- 172.21.15.5:0/4182141678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc1a6990 con 0x7f67bc103760 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67c2b77640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67bc1082e0 0x7f67bc101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67c1b75640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67bc103760 0x7f67bc100f50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00c8c0 tx=0x7f67ac00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67b37fe640 1 -- 172.21.15.5:0/4182141678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac007bb0 con 0x7f67bc103760 2024-09-11T22:22:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67c45da640 1 -- 172.21.15.5:0/4182141678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc1a6c80 con 0x7f67bc103760 2024-09-11T22:22:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67b37fe640 1 -- 172.21.15.5:0/4182141678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ac007d50 con 0x7f67bc103760 2024-09-11T22:22:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67b37fe640 1 -- 172.21.15.5:0/4182141678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac018830 con 0x7f67bc103760 2024-09-11T22:22:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.892+0000 7f67c45da640 1 -- 172.21.15.5:0/4182141678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a71c0 con 0x7f67bc103760 2024-09-11T22:22:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.895+0000 7f67b37fe640 1 -- 172.21.15.5:0/4182141678 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f67ac0189f0 con 0x7f67bc103760 2024-09-11T22:22:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.895+0000 7f67c69b3740 1 Processor -- start 2024-09-11T22:22:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c69b3740 1 -- start start 2024-09-11T22:22:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac33f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac34480 0x559e5ac3e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac41130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c69b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e5ab99280 con 0x559e5ac3ed40 2024-09-11T22:22:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c69b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e5ab40b10 con 0x559e5ac33b60 2024-09-11T22:22:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c69b3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559e5ab95240 con 0x559e5ac34480 2024-09-11T22:22:25.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c1b75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac41130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c1b75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac41130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57496/0 (socket says 172.21.15.5:57496) 2024-09-11T22:22:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c1b75640 1 -- 172.21.15.5:0/1057803006 learned_addr learned my addr 172.21.15.5:0/1057803006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c2b77640 1 --2- 172.21.15.5:0/1057803006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac33f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c2376640 1 --2- 172.21.15.5:0/1057803006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac34480 0x559e5ac3e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67b37fe640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67a80a2e10 0x7f67a80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.896+0000 7f67c2b77640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67a80a2e10 0x7f67a80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67c1b75640 1 -- 172.21.15.5:0/1057803006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac34480 msgr2=0x559e5ac3e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67c1b75640 1 --2- 172.21.15.5:0/1057803006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac34480 0x559e5ac3e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67c1b75640 1 -- 172.21.15.5:0/1057803006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 msgr2=0x559e5ac33f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67c1b75640 1 --2- 172.21.15.5:0/1057803006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac33f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67b37fe640 1 -- 172.21.15.5:0/4182141678 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f67ac0cf940 con 0x7f67bc103760 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67c1b75640 1 -- 172.21.15.5:0/1057803006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5ab4d1f0 con 0x559e5ac3ed40 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67c1b75640 1 --2- 172.21.15.5:0/1057803006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac41130 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f67ac098580 tx=0x7f67ac00fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c54b42a88ae327f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67c2b77640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67a80a2e10 0x7f67a80a52d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f67b4005fd0 tx=0x7f67b4004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67b0ff9640 1 -- 172.21.15.5:0/1057803006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac09b510 con 0x559e5ac3ed40 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67b0ff9640 1 -- 172.21.15.5:0/1057803006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ac017360 con 0x559e5ac3ed40 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.897+0000 7f67b0ff9640 1 -- 172.21.15.5:0/1057803006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00fac0 con 0x559e5ac3ed40 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 -- 172.21.15.5:0/1057803006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 msgr2=0x559e5ac41130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 --2- 172.21.15.5:0/1057803006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac41130 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f67ac098580 tx=0x7f67ac00fe90 comp rx=0 tx=0).stop 2024-09-11T22:22:25.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 -- 172.21.15.5:0/1057803006 shutdown_connections 2024-09-11T22:22:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 --2- 172.21.15.5:0/1057803006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac41130 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 --2- 172.21.15.5:0/1057803006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac34480 0x559e5ac3e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 --2- 172.21.15.5:0/1057803006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac33f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 -- 172.21.15.5:0/1057803006 >> 172.21.15.5:0/1057803006 conn(0x559e5ac300f0 msgr2=0x559e5ac304d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 -- 172.21.15.5:0/1057803006 shutdown_connections 2024-09-11T22:22:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 -- 172.21.15.5:0/1057803006 wait complete. 2024-09-11T22:22:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 -- start start 2024-09-11T22:22:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.898+0000 7f67c69b3740 1 Processor -- start 2024-09-11T22:22:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac3bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac34480 0x559e5ac3c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac3ed40 0x559e5ac3c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c69b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e5ac41910 con 0x559e5ac34480 2024-09-11T22:22:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c69b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e5ab53f10 con 0x559e5ac33b60 2024-09-11T22:22:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c69b3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559e5ac41a90 con 0x559e5ac3ed40 2024-09-11T22:22:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c69b3740 10 client.0 _ll_register_callbacks cb 0x559e5ab1ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c1b75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac34480 0x559e5ac3c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c1b75640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac34480 0x559e5ac3c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57504/0 (socket says 172.21.15.5:57504) 2024-09-11T22:22:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c1b75640 1 -- 172.21.15.5:0/3412416275 learned_addr learned my addr 172.21.15.5:0/3412416275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:25.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c2b77640 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac3ed40 0x559e5ac3c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c2376640 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac3bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c1b75640 1 -- 172.21.15.5:0/3412416275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac3ed40 msgr2=0x559e5ac3c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c1b75640 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac3ed40 0x559e5ac3c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c1b75640 1 -- 172.21.15.5:0/3412416275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 msgr2=0x559e5ac3bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c1b75640 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac3bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c1b75640 1 -- 172.21.15.5:0/3412416275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5a90a6a0 con 0x559e5ac34480 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67c1b75640 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac34480 0x559e5ac3c100 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f67ac09bb40 tx=0x7f67ac0a00b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cf6b5efbea75c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.899+0000 7f67a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67a5ffb640 1 -- 172.21.15.5:0/3412416275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac0a4820 con 0x559e5ac34480 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67a5ffb640 1 -- 172.21.15.5:0/3412416275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f67ac00fac0 con 0x559e5ac34480 2024-09-11T22:22:25.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67c69b3740 1 -- 172.21.15.5:0/3412416275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67ac017360 con 0x559e5ac34480 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67a5ffb640 1 -- 172.21.15.5:0/3412416275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67ac04f020 con 0x559e5ac34480 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67c69b3740 10 client.15357 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67c69b3740 1 -- 172.21.15.5:0/3412416275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e5ac4d4c0 con 0x559e5ac34480 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67a5ffb640 1 -- 172.21.15.5:0/3412416275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67ac017360 con 0x559e5ac34480 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67a5ffb640 1 -- 172.21.15.5:0/3412416275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f67ac04f1c0 con 0x559e5ac34480 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67c69b3740 10 client.15357 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67c69b3740 10 client.15357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67c69b3740 20 client.15357 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67c69b3740 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e5ac4e810 0x559e5ac50c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67c69b3740 4 client.15357 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.900+0000 7f67c69b3740 1 -- 172.21.15.5:0/3412416275 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e5aba67f0 con 0x559e5ac4e810 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.901+0000 7f67c2376640 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e5ac4e810 0x559e5ac50c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.901+0000 7f67a5ffb640 10 client.15357 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.901+0000 7f67c2376640 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e5ac4e810 0x559e5ac50c00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7590d4381877c106 server_cookie=279e5d8f7da825eb in_seq=0 out_seq=0 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67a5ffb640 1 -- 172.21.15.5:0/3412416275 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e5aba67f0 con 0x559e5ac4e810 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67a5ffb640 10 client.15357 handle_command_reply: tid=0 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 client.15357 shutdown 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 -- 172.21.15.5:0/3412416275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac34480 msgr2=0x559e5ac3c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac34480 0x559e5ac3c100 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f67ac09bb40 tx=0x7f67ac0a00b0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 -- 172.21.15.5:0/3412416275 shutdown_connections 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e5ac4e810 0x559e5ac50c00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac3ed40 0x559e5ac3c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac34480 0x559e5ac3c100 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 --2- 172.21.15.5:0/3412416275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac3bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 -- 172.21.15.5:0/3412416275 >> 172.21.15.5:0/3412416275 conn(0x559e5ac300f0 msgr2=0x559e5ac35e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67a5ffb640 0 client.15357 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 -- 172.21.15.5:0/3412416275 shutdown_connections 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 1 -- 172.21.15.5:0/3412416275 wait complete. 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.902+0000 7f67c69b3740 20 client.15357 trim_cache size 0 max 16384 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.919+0000 7f67c69b3740 1 Processor -- start 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.919+0000 7f67c69b3740 1 -- start start 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac35cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac34480 0x559e5ac53840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac53d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c69b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1ab760 con 0x559e5ac3ed40 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c69b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1ad2a0 con 0x559e5ac33b60 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c69b3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1abe30 con 0x559e5ac34480 2024-09-11T22:22:25.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c2b77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac53d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c2b77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac53d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57508/0 (socket says 172.21.15.5:57508) 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c2b77640 1 -- 172.21.15.5:0/909973580 learned_addr learned my addr 172.21.15.5:0/909973580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c2376640 1 --2- 172.21.15.5:0/909973580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac35cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c1b75640 1 --2- 172.21.15.5:0/909973580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac34480 0x559e5ac53840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c2376640 1 -- 172.21.15.5:0/909973580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac34480 msgr2=0x559e5ac53840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c2376640 1 --2- 172.21.15.5:0/909973580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac34480 0x559e5ac53840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c2376640 1 -- 172.21.15.5:0/909973580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 msgr2=0x559e5ac53d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c2376640 1 --2- 172.21.15.5:0/909973580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac53d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c2376640 1 -- 172.21.15.5:0/909973580 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e5ac2b830 con 0x559e5ac33b60 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.920+0000 7f67c2b77640 1 --2- 172.21.15.5:0/909973580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac53d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.921+0000 7f67c2376640 1 --2- 172.21.15.5:0/909973580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac35cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009900 tx=0x7f67b80058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6fc98d8073d6eea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.922+0000 7f67a7fff640 1 -- 172.21.15.5:0/909973580 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8035040 con 0x559e5ac33b60 2024-09-11T22:22:25.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.922+0000 7f67a7fff640 1 -- 172.21.15.5:0/909973580 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b804a070 con 0x559e5ac33b60 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.922+0000 7f67a7fff640 1 -- 172.21.15.5:0/909973580 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b8003850 con 0x559e5ac33b60 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 -- 172.21.15.5:0/909973580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 msgr2=0x559e5ac35cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 --2- 172.21.15.5:0/909973580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac35cc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67b8009900 tx=0x7f67b80058b0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 -- 172.21.15.5:0/909973580 shutdown_connections 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 --2- 172.21.15.5:0/909973580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac53d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 --2- 172.21.15.5:0/909973580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac34480 0x559e5ac53840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 --2- 172.21.15.5:0/909973580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac33b60 0x559e5ac35cc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 -- 172.21.15.5:0/909973580 >> 172.21.15.5:0/909973580 conn(0x559e5ac300f0 msgr2=0x559e5ac355b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 -- 172.21.15.5:0/909973580 shutdown_connections 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 -- 172.21.15.5:0/909973580 wait complete. 2024-09-11T22:22:25.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 -- start start 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.923+0000 7f67c69b3740 1 Processor -- start 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac33b60 0x559e5ac54010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac34480 0x559e5ac51360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c69b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c69b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1acbd0 con 0x559e5ac3ed40 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c69b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1ac500 con 0x559e5ac34480 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c69b3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1a9780 con 0x559e5ac33b60 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c69b3740 10 client.0 _ll_register_callbacks cb 0x7f67b40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c1b75640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac34480 0x559e5ac51360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c2b77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c1b75640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac34480 0x559e5ac51360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55262/0 (socket says 172.21.15.5:55262) 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c1b75640 1 -- 172.21.15.5:0/1178570427 learned_addr learned my addr 172.21.15.5:0/1178570427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:25.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c2376640 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac33b60 0x559e5ac54010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c2b77640 1 -- 172.21.15.5:0/1178570427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac33b60 msgr2=0x559e5ac54010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c2b77640 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac33b60 0x559e5ac54010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c2b77640 1 -- 172.21.15.5:0/1178570427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac34480 msgr2=0x559e5ac51360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c2b77640 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac34480 0x559e5ac51360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.924+0000 7f67c2b77640 1 -- 172.21.15.5:0/1178570427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e5ac520f0 con 0x559e5ac3ed40 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.925+0000 7f67c2b77640 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac518c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x559e5ac2fb40 tx=0x7f67b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954108ae6c403295 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.925+0000 7f67a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.925+0000 7f67a5ffb640 1 -- 172.21.15.5:0/1178570427 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b402e060 con 0x559e5ac3ed40 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.925+0000 7f67a5ffb640 1 -- 172.21.15.5:0/1178570427 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f67b4010a70 con 0x559e5ac3ed40 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.925+0000 7f67c69b3740 1 -- 172.21.15.5:0/1178570427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67b804a070 con 0x559e5ac3ed40 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.925+0000 7f67a5ffb640 1 -- 172.21.15.5:0/1178570427 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67b4010c70 con 0x559e5ac3ed40 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.925+0000 7f67c69b3740 10 client.15363 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:25.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.925+0000 7f67c69b3740 1 -- 172.21.15.5:0/1178570427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e5acb9280 con 0x559e5ac3ed40 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.926+0000 7f67a5ffb640 1 -- 172.21.15.5:0/1178570427 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67b804a070 con 0x559e5ac3ed40 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.926+0000 7f67a5ffb640 1 -- 172.21.15.5:0/1178570427 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f67b4028030 con 0x559e5ac3ed40 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.926+0000 7f67c69b3740 10 client.15363 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.926+0000 7f67c69b3740 10 client.15363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.926+0000 7f67c69b3740 20 client.15363 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.926+0000 7f67c69b3740 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e5acb99d0 0x559e5acbbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.926+0000 7f67c69b3740 4 client.15363 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.926+0000 7f67c69b3740 1 -- 172.21.15.5:0/1178570427 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559e5aba0d00 con 0x559e5acb99d0 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.926+0000 7f67c2376640 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e5acb99d0 0x559e5acbbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.927+0000 7f67c2376640 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e5acb99d0 0x559e5acbbdf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d93af68bf73857c8 server_cookie=ec55858b9412c161 in_seq=0 out_seq=0 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.927+0000 7f67a5ffb640 10 client.15363 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67a5ffb640 1 -- 172.21.15.5:0/1178570427 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559e5aba0d00 con 0x559e5acb99d0 2024-09-11T22:22:25.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67a5ffb640 10 client.15363 handle_command_reply: tid=0 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67c69b3740 1 client.15363 shutdown 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67c69b3740 1 -- 172.21.15.5:0/1178570427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 msgr2=0x559e5ac518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67c69b3740 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac518c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x559e5ac2fb40 tx=0x7f67b4004a40 comp rx=0 tx=0).stop 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67c69b3740 1 -- 172.21.15.5:0/1178570427 shutdown_connections 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67c69b3740 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e5acb99d0 0x559e5acbbdf0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67c69b3740 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e5ac3ed40 0x559e5ac518c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67c69b3740 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e5ac34480 0x559e5ac51360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67a5ffb640 0 client.15363 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67c69b3740 1 --2- 172.21.15.5:0/1178570427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e5ac33b60 0x559e5ac54010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67c69b3740 1 -- 172.21.15.5:0/1178570427 >> 172.21.15.5:0/1178570427 conn(0x559e5ac300f0 msgr2=0x559e5ac2f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.928+0000 7f67c69b3740 1 -- 172.21.15.5:0/1178570427 shutdown_connections 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.929+0000 7f67c69b3740 1 -- 172.21.15.5:0/1178570427 wait complete. 2024-09-11T22:22:25.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.929+0000 7f67c69b3740 20 client.15363 trim_cache size 0 max 16384 2024-09-11T22:22:25.944 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:25.944 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:22:25.944 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "c0c38f9b-957e-42ff-97d9-6f69752dbfcb", 2024-09-11T22:22:25.944 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:22:25.944 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.929+0000 7f67c45da640 1 -- 172.21.15.5:0/4182141678 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67a80a2e10 msgr2=0x7f67a80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.929+0000 7f67c45da640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67a80a2e10 0x7f67a80a52d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f67b4005fd0 tx=0x7f67b4004ab0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.929+0000 7f67c45da640 1 -- 172.21.15.5:0/4182141678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67bc103760 msgr2=0x7f67bc100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.929+0000 7f67c45da640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67bc103760 0x7f67bc100f50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00c8c0 tx=0x7f67ac00cd80 comp rx=0 tx=0).stop 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.930+0000 7f67c45da640 1 -- 172.21.15.5:0/4182141678 shutdown_connections 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.930+0000 7f67c45da640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67a80a2e10 0x7f67a80a52d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.930+0000 7f67c45da640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67bc1082e0 0x7f67bc101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.930+0000 7f67c45da640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67bc103760 0x7f67bc100f50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.930+0000 7f67c45da640 1 --2- 172.21.15.5:0/4182141678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67bc102db0 0x7f67bc100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.930+0000 7f67c45da640 1 -- 172.21.15.5:0/4182141678 >> 172.21.15.5:0/4182141678 conn(0x7f67bc0fc7d0 msgr2=0x7f67bc105cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.930+0000 7f67c45da640 1 -- 172.21.15.5:0/4182141678 shutdown_connections 2024-09-11T22:22:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:25.930+0000 7f67c45da640 1 -- 172.21.15.5:0/4182141678 wait complete. 2024-09-11T22:22:26.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.063+0000 7fda87a35640 1 -- 172.21.15.5:0/842385552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fda801a5210 con 0x7fda80102480 2024-09-11T22:22:26.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.067+0000 7fda727fc640 1 -- 172.21.15.5:0/842385552 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7fda7409c0b0 con 0x7fda80102480 2024-09-11T22:22:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 -- 172.21.15.5:0/842385552 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda6c0a2f40 msgr2=0x7fda6c0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda6c0a2f40 0x7fda6c0a5400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fda7c009900 tx=0x7fda7c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 -- 172.21.15.5:0/842385552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda80102480 msgr2=0x7fda80198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda80102480 0x7fda80198880 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fda7400c890 tx=0x7fda7400cd50 comp rx=0 tx=0).stop 2024-09-11T22:22:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 -- 172.21.15.5:0/842385552 shutdown_connections 2024-09-11T22:22:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda6c0a2f40 0x7fda6c0a5400 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda80107b80 0x7fda8019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda80102480 0x7fda80198880 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 --2- 172.21.15.5:0/842385552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda80101ae0 0x7fda80198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 -- 172.21.15.5:0/842385552 >> 172.21.15.5:0/842385552 conn(0x7fda800fd7e0 msgr2=0x7fda800fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.069+0000 7fda87a35640 1 -- 172.21.15.5:0/842385552 shutdown_connections 2024-09-11T22:22:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.070+0000 7fda87a35640 1 -- 172.21.15.5:0/842385552 wait complete. 2024-09-11T22:22:26.082 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-11T22:22:26.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.191+0000 7f99ac998640 1 -- 172.21.15.5:0/2089174368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a81035a0 msgr2=0x7f99a810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:26.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.191+0000 7f99ac998640 1 --2- 172.21.15.5:0/2089174368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a81035a0 0x7f99a810ba80 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f999c009900 tx=0x7f999c02a480 comp rx=0 tx=0).stop 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.191+0000 7f99ac998640 1 -- 172.21.15.5:0/2089174368 shutdown_connections 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.191+0000 7f99ac998640 1 --2- 172.21.15.5:0/2089174368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99a810c0d0 0x7f99a810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.191+0000 7f99ac998640 1 --2- 172.21.15.5:0/2089174368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a81035a0 0x7f99a810ba80 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.191+0000 7f99ac998640 1 --2- 172.21.15.5:0/2089174368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99a8102c80 0x7f99a8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.191+0000 7f99ac998640 1 -- 172.21.15.5:0/2089174368 >> 172.21.15.5:0/2089174368 conn(0x7f99a8077990 msgr2=0x7f99a80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.191+0000 7f99ac998640 1 -- 172.21.15.5:0/2089174368 shutdown_connections 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.191+0000 7f99ac998640 1 -- 172.21.15.5:0/2089174368 wait complete. 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.192+0000 7f99ac998640 1 Processor -- start 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.192+0000 7f99ac998640 1 -- start start 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.192+0000 7f99ac998640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99a8102c80 0x7f99a819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.192+0000 7f99ac998640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99a81035a0 0x7f99a819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.192+0000 7f99ac998640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a810c0d0 0x7f99a819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.192+0000 7f99ac998640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8111eb0 con 0x7f99a810c0d0 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.192+0000 7f99ac998640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8111d30 con 0x7f99a81035a0 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99ac998640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112030 con 0x7f99a8102c80 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99a8102c80 0x7f99a819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a810c0d0 0x7f99a819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a5d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99a81035a0 0x7f99a819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99a8102c80 0x7f99a819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:58948/0 (socket says 172.21.15.5:58948) 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a659c640 1 -- 172.21.15.5:0/2542625404 learned_addr learned my addr 172.21.15.5:0/2542625404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a6d9d640 1 -- 172.21.15.5:0/2542625404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99a8102c80 msgr2=0x7f99a819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a6d9d640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99a8102c80 0x7f99a819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a6d9d640 1 -- 172.21.15.5:0/2542625404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99a81035a0 msgr2=0x7f99a819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a6d9d640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99a81035a0 0x7f99a819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a6d9d640 1 -- 172.21.15.5:0/2542625404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a819f550 con 0x7f99a810c0d0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.193+0000 7f99a6d9d640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a810c0d0 0x7f99a819ee20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e940 tx=0x7f99a000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.194+0000 7f99977fe640 1 -- 172.21.15.5:0/2542625404 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0004d20 con 0x7f99a810c0d0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.194+0000 7f99ac998640 1 -- 172.21.15.5:0/2542625404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a819f840 con 0x7f99a810c0d0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.194+0000 7f99ac998640 1 -- 172.21.15.5:0/2542625404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81a70d0 con 0x7f99a810c0d0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.194+0000 7f99977fe640 1 -- 172.21.15.5:0/2542625404 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a0005020 con 0x7f99a810c0d0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.194+0000 7f99977fe640 1 -- 172.21.15.5:0/2542625404 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a00116b0 con 0x7f99a810c0d0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.195+0000 7f99ac998640 1 -- 172.21.15.5:0/2542625404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99880052c0 con 0x7f99a810c0d0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.197+0000 7f99977fe640 1 -- 172.21.15.5:0/2542625404 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f99a000cb20 con 0x7f99a810c0d0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.198+0000 7f99977fe640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f99900a2f40 0x7f99900a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.198+0000 7f99a659c640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f99900a2f40 0x7f99900a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.198+0000 7f99977fe640 1 -- 172.21.15.5:0/2542625404 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f99a00cf590 con 0x7f99a810c0d0 2024-09-11T22:22:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.198+0000 7f99a659c640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f99900a2f40 0x7f99900a5400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f999800bca0 tx=0x7f999800b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.202+0000 7f99977fe640 1 -- 172.21.15.5:0/2542625404 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99a0016030 con 0x7f99a810c0d0 2024-09-11T22:22:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.402+0000 7f99ac998640 1 -- 172.21.15.5:0/2542625404 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f9988000bf0 con 0x7f99900a2f40 2024-09-11T22:22:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.469+0000 7f99977fe640 1 -- 172.21.15.5:0/2542625404 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9988000bf0 con 0x7f99900a2f40 2024-09-11T22:22:26.611 INFO:teuthology.orchestra.run.smithi005.stdout:Schedule set for path / 2024-09-11T22:22:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 -- 172.21.15.5:0/2542625404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f99900a2f40 msgr2=0x7f99900a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f99900a2f40 0x7f99900a5400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f999800bca0 tx=0x7f999800b410 comp rx=0 tx=0).stop 2024-09-11T22:22:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 -- 172.21.15.5:0/2542625404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a810c0d0 msgr2=0x7f99a819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a810c0d0 0x7f99a819ee20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e940 tx=0x7f99a000ee00 comp rx=0 tx=0).stop 2024-09-11T22:22:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 -- 172.21.15.5:0/2542625404 shutdown_connections 2024-09-11T22:22:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99a810c0d0 0x7f99a819ee20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f99900a2f40 0x7f99900a5400 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f99a81035a0 0x7f99a819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 --2- 172.21.15.5:0/2542625404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99a8102c80 0x7f99a819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 -- 172.21.15.5:0/2542625404 >> 172.21.15.5:0/2542625404 conn(0x7f99a8077990 msgr2=0x7f99a8100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 -- 172.21.15.5:0/2542625404 shutdown_connections 2024-09-11T22:22:26.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.472+0000 7f99ac998640 1 -- 172.21.15.5:0/2542625404 wait complete. 2024-09-11T22:22:26.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-11T22:22:26.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2156976278' entity='client.admin' 2024-09-11T22:22:26.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:26.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:26.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:26.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:26.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1555165586' entity='client.admin' 2024-09-11T22:22:26.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:26.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:26.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:26.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:26.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/842385552' entity='client.admin' 2024-09-11T22:22:26.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:26.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:26.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:26.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:26 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:26.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:22:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.722+0000 7f80958e6640 1 -- 172.21.15.5:0/3454258087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80901069e0 msgr2=0x7f8090106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.722+0000 7f80958e6640 1 --2- 172.21.15.5:0/3454258087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80901069e0 0x7f8090106dc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8080009930 tx=0x7f808002a420 comp rx=0 tx=0).stop 2024-09-11T22:22:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.722+0000 7f80958e6640 1 -- 172.21.15.5:0/3454258087 shutdown_connections 2024-09-11T22:22:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.722+0000 7f80958e6640 1 --2- 172.21.15.5:0/3454258087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8090101370 0x7f809010d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.722+0000 7f80958e6640 1 --2- 172.21.15.5:0/3454258087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f80901009d0 0x7f8090100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.722+0000 7f80958e6640 1 --2- 172.21.15.5:0/3454258087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80901069e0 0x7f8090106dc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.722+0000 7f80958e6640 1 -- 172.21.15.5:0/3454258087 >> 172.21.15.5:0/3454258087 conn(0x7f80900fc6b0 msgr2=0x7f80900fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.722+0000 7f80958e6640 1 -- 172.21.15.5:0/3454258087 shutdown_connections 2024-09-11T22:22:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.722+0000 7f80958e6640 1 -- 172.21.15.5:0/3454258087 wait complete. 2024-09-11T22:22:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.723+0000 7f80958e6640 1 Processor -- start 2024-09-11T22:22:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.723+0000 7f80958e6640 1 -- start start 2024-09-11T22:22:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f80958e6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80901009d0 0x7f80901983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f80958e6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090101370 0x7f8090198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f80958e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80901069e0 0x7f809019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f80958e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8090110e70 con 0x7f80901069e0 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f80958e6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8090110cf0 con 0x7f80901009d0 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f80958e6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8090110ff0 con 0x7f8090101370 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80901069e0 0x7f809019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80901069e0 0x7f809019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57542/0 (socket says 172.21.15.5:57542) 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808f7fe640 1 -- 172.21.15.5:0/512823798 learned_addr learned my addr 172.21.15.5:0/512823798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808e7fc640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090101370 0x7f8090198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808effd640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80901009d0 0x7f80901983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808f7fe640 1 -- 172.21.15.5:0/512823798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090101370 msgr2=0x7f8090198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808f7fe640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090101370 0x7f8090198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808f7fe640 1 -- 172.21.15.5:0/512823798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80901009d0 msgr2=0x7f80901983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808f7fe640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80901009d0 0x7f80901983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808f7fe640 1 -- 172.21.15.5:0/512823798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f809019d3c0 con 0x7f80901069e0 2024-09-11T22:22:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808effd640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80901009d0 0x7f80901983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f808f7fe640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80901069e0 0x7f809019cc90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f808800eaa0 tx=0x7f808800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.724+0000 7f80948e4640 1 -- 172.21.15.5:0/512823798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808800cdb0 con 0x7f80901069e0 2024-09-11T22:22:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.725+0000 7f80948e4640 1 -- 172.21.15.5:0/512823798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8088005020 con 0x7f80901069e0 2024-09-11T22:22:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.725+0000 7f80948e4640 1 -- 172.21.15.5:0/512823798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80880116b0 con 0x7f80901069e0 2024-09-11T22:22:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.725+0000 7f80958e6640 1 -- 172.21.15.5:0/512823798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809019d6b0 con 0x7f80901069e0 2024-09-11T22:22:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.725+0000 7f80958e6640 1 -- 172.21.15.5:0/512823798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80901a4f40 con 0x7f80901069e0 2024-09-11T22:22:26.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.726+0000 7f80958e6640 1 -- 172.21.15.5:0/512823798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8090100e30 con 0x7f80901069e0 2024-09-11T22:22:26.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.728+0000 7f80948e4640 1 -- 172.21.15.5:0/512823798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f8088005560 con 0x7f80901069e0 2024-09-11T22:22:26.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.728+0000 7f80948e4640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80780a2ec0 0x7f80780a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:26.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.729+0000 7f80948e4640 1 -- 172.21.15.5:0/512823798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f80880cfa00 con 0x7f80901069e0 2024-09-11T22:22:26.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.729+0000 7f808effd640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80780a2ec0 0x7f80780a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:26.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.729+0000 7f808effd640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80780a2ec0 0x7f80780a5380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8080006010 tx=0x7f8080005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:26.730 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2156976278' entity='client.admin' 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1555165586' entity='client.admin' 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/842385552' entity='client.admin' 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:26.731 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:26.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:26.732 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:26 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:26.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.733+0000 7f80948e4640 1 -- 172.21.15.5:0/512823798 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8088016030 con 0x7f80901069e0 2024-09-11T22:22:26.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2156976278' entity='client.admin' 2024-09-11T22:22:26.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:26.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:26.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:26.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:26.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1555165586' entity='client.admin' 2024-09-11T22:22:26.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:26.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:26.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:26.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:26.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/842385552' entity='client.admin' 2024-09-11T22:22:26.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:22:26.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:22:26.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:22:26.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:26 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:22:26.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:26.952+0000 7f80958e6640 1 -- 172.21.15.5:0/512823798 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f8090062630 con 0x7f80780a2ec0 2024-09-11T22:22:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:27.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.018+0000 7f80948e4640 1 -- 172.21.15.5:0/512823798 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8090062630 con 0x7f80780a2ec0 2024-09-11T22:22:27.019 INFO:teuthology.orchestra.run.smithi005.stdout:Retention added to path / 2024-09-11T22:22:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 -- 172.21.15.5:0/512823798 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80780a2ec0 msgr2=0x7f80780a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80780a2ec0 0x7f80780a5380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8080006010 tx=0x7f8080005a20 comp rx=0 tx=0).stop 2024-09-11T22:22:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 -- 172.21.15.5:0/512823798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80901069e0 msgr2=0x7f809019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80901069e0 0x7f809019cc90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f808800eaa0 tx=0x7f808800ef60 comp rx=0 tx=0).stop 2024-09-11T22:22:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 -- 172.21.15.5:0/512823798 shutdown_connections 2024-09-11T22:22:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80901069e0 0x7f809019cc90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090101370 0x7f8090198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80780a2ec0 0x7f80780a5380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 --2- 172.21.15.5:0/512823798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80901009d0 0x7f80901983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 -- 172.21.15.5:0/512823798 >> 172.21.15.5:0/512823798 conn(0x7f80900fc6b0 msgr2=0x7f809006a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 -- 172.21.15.5:0/512823798 shutdown_connections 2024-09-11T22:22:27.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.020+0000 7f80958e6640 1 -- 172.21.15.5:0/512823798 wait complete. 2024-09-11T22:22:27.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T22:22:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.178+0000 7fdfdad18640 1 -- 172.21.15.5:0/131905745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd4101390 msgr2=0x7fdfd410d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.178+0000 7fdfdad18640 1 --2- 172.21.15.5:0/131905745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd4101390 0x7fdfd410d8c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009cd0 tx=0x7fdfcc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.178+0000 7fdfdad18640 1 -- 172.21.15.5:0/131905745 shutdown_connections 2024-09-11T22:22:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.178+0000 7fdfdad18640 1 --2- 172.21.15.5:0/131905745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd4101390 0x7fdfd410d8c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.178+0000 7fdfdad18640 1 --2- 172.21.15.5:0/131905745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd41009f0 0x7fdfd4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.178+0000 7fdfdad18640 1 --2- 172.21.15.5:0/131905745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd4106a00 0x7fdfd4106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.178+0000 7fdfdad18640 1 -- 172.21.15.5:0/131905745 >> 172.21.15.5:0/131905745 conn(0x7fdfd40fc6b0 msgr2=0x7fdfd40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.178+0000 7fdfdad18640 1 -- 172.21.15.5:0/131905745 shutdown_connections 2024-09-11T22:22:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.178+0000 7fdfdad18640 1 -- 172.21.15.5:0/131905745 wait complete. 2024-09-11T22:22:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfdad18640 1 Processor -- start 2024-09-11T22:22:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfdad18640 1 -- start start 2024-09-11T22:22:27.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfdad18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd41009f0 0x7fdfd4199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfdad18640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd4101390 0x7fdfd4199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfdad18640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd4106a00 0x7fdfd41a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfdad18640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4110e90 con 0x7fdfd41009f0 2024-09-11T22:22:27.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfdad18640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4110d10 con 0x7fdfd4106a00 2024-09-11T22:22:27.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfdad18640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd4111010 con 0x7fdfd4101390 2024-09-11T22:22:27.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfd8ab4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd41009f0 0x7fdfd4199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfd8ab4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd41009f0 0x7fdfd4199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57554/0 (socket says 172.21.15.5:57554) 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfd8ab4640 1 -- 172.21.15.5:0/846210943 learned_addr learned my addr 172.21.15.5:0/846210943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfd92b5640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd4106a00 0x7fdfd41a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.179+0000 7fdfcbfff640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd4101390 0x7fdfd4199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfd8ab4640 1 -- 172.21.15.5:0/846210943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd4101390 msgr2=0x7fdfd4199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfd8ab4640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd4101390 0x7fdfd4199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfd8ab4640 1 -- 172.21.15.5:0/846210943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd4106a00 msgr2=0x7fdfd41a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfd8ab4640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd4106a00 0x7fdfd41a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfd8ab4640 1 -- 172.21.15.5:0/846210943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd41a47c0 con 0x7fdfd41009f0 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfd8ab4640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd41009f0 0x7fdfd4199200 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000d950 tx=0x7fdfd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfc9ffb640 1 -- 172.21.15.5:0/846210943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0017070 con 0x7fdfd41009f0 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfc9ffb640 1 -- 172.21.15.5:0/846210943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd0009c30 con 0x7fdfd41009f0 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfdad18640 1 -- 172.21.15.5:0/846210943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd41a4990 con 0x7fdfd41009f0 2024-09-11T22:22:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfc9ffb640 1 -- 172.21.15.5:0/846210943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000f6b0 con 0x7fdfd41009f0 2024-09-11T22:22:27.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.180+0000 7fdfdad18640 1 -- 172.21.15.5:0/846210943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd41a4eb0 con 0x7fdfd41009f0 2024-09-11T22:22:27.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.181+0000 7fdfdad18640 1 -- 172.21.15.5:0/846210943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd4100e50 con 0x7fdfd41009f0 2024-09-11T22:22:27.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.182+0000 7fdfc9ffb640 1 -- 172.21.15.5:0/846210943 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7fdfd00109d0 con 0x7fdfd41009f0 2024-09-11T22:22:27.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.183+0000 7fdfc9ffb640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfbc0a2ec0 0x7fdfbc0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.183+0000 7fdfc9ffb640 1 -- 172.21.15.5:0/846210943 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fdfd00cf330 con 0x7fdfd41009f0 2024-09-11T22:22:27.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.183+0000 7fdfcbfff640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfbc0a2ec0 0x7fdfbc0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.184+0000 7fdfcbfff640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfbc0a2ec0 0x7fdfbc0a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdfd406ae50 tx=0x7fdfc4008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:27.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.187+0000 7fdfc9ffb640 1 -- 172.21.15.5:0/846210943 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdfd0009de0 con 0x7fdfd41009f0 2024-09-11T22:22:27.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.394+0000 7fdfdad18640 1 -- 172.21.15.5:0/846210943 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fdfd40fd350 con 0x7fdfbc0a2ec0 2024-09-11T22:22:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.401+0000 7fdfc9ffb640 1 -- 172.21.15.5:0/846210943 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fdfd40fd350 con 0x7fdfbc0a2ec0 2024-09-11T22:22:27.401 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T22:22:26", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-11T22:22:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 -- 172.21.15.5:0/846210943 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfbc0a2ec0 msgr2=0x7fdfbc0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfbc0a2ec0 0x7fdfbc0a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdfd406ae50 tx=0x7fdfc4008040 comp rx=0 tx=0).stop 2024-09-11T22:22:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 -- 172.21.15.5:0/846210943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd41009f0 msgr2=0x7fdfd4199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd41009f0 0x7fdfd4199200 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000d950 tx=0x7fdfd000de10 comp rx=0 tx=0).stop 2024-09-11T22:22:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 -- 172.21.15.5:0/846210943 shutdown_connections 2024-09-11T22:22:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfbc0a2ec0 0x7fdfbc0a5380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfd4106a00 0x7fdfd41a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfd4101390 0x7fdfd4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 --2- 172.21.15.5:0/846210943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfd41009f0 0x7fdfd4199200 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 -- 172.21.15.5:0/846210943 >> 172.21.15.5:0/846210943 conn(0x7fdfd40fc6b0 msgr2=0x7fdfd40fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 -- 172.21.15.5:0/846210943 shutdown_connections 2024-09-11T22:22:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.403+0000 7fdfdad18640 1 -- 172.21.15.5:0/846210943 wait complete. 2024-09-11T22:22:27.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T22:22:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.519+0000 7f0d81986640 1 -- 172.21.15.5:0/79184094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c106f10 msgr2=0x7f0d7c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.519+0000 7f0d81986640 1 --2- 172.21.15.5:0/79184094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c106f10 0x7f0d7c10ea90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009ca0 tx=0x7f0d7402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.519+0000 7f0d81986640 1 -- 172.21.15.5:0/79184094 shutdown_connections 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.519+0000 7f0d81986640 1 --2- 172.21.15.5:0/79184094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c106f10 0x7f0d7c10ea90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.519+0000 7f0d81986640 1 --2- 172.21.15.5:0/79184094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c102790 0x7f0d7c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.519+0000 7f0d81986640 1 --2- 172.21.15.5:0/79184094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c101de0 0x7f0d7c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.519+0000 7f0d81986640 1 -- 172.21.15.5:0/79184094 >> 172.21.15.5:0/79184094 conn(0x7f0d7c0fd8c0 msgr2=0x7f0d7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.519+0000 7f0d81986640 1 -- 172.21.15.5:0/79184094 shutdown_connections 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.519+0000 7f0d81986640 1 -- 172.21.15.5:0/79184094 wait complete. 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.520+0000 7f0d81986640 1 Processor -- start 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d81986640 1 -- start start 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d81986640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c101de0 0x7f0d7c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d81986640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c102790 0x7f0d7c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d81986640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c106f10 0x7f0d7c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d81986640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c111f70 con 0x7f0d7c102790 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d81986640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c111df0 con 0x7f0d7c101de0 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d81986640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1120f0 con 0x7f0d7c106f10 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d7a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c102790 0x7f0d7c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d7a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c102790 0x7f0d7c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57578/0 (socket says 172.21.15.5:57578) 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d7a7fc640 1 -- 172.21.15.5:0/2322643565 learned_addr learned my addr 172.21.15.5:0/2322643565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d7b7fe640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c106f10 0x7f0d7c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d7affd640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c101de0 0x7f0d7c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.521+0000 7f0d7a7fc640 1 -- 172.21.15.5:0/2322643565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c106f10 msgr2=0x7f0d7c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.522+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c106f10 0x7f0d7c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.522+0000 7f0d7a7fc640 1 -- 172.21.15.5:0/2322643565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c101de0 msgr2=0x7f0d7c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.522+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c101de0 0x7f0d7c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.522+0000 7f0d7a7fc640 1 -- 172.21.15.5:0/2322643565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c19d380 con 0x7f0d7c102790 2024-09-11T22:22:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.522+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c102790 0x7f0d7c198910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d950 tx=0x7f0d7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.522+0000 7f0d80984640 1 -- 172.21.15.5:0/2322643565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70017030 con 0x7f0d7c102790 2024-09-11T22:22:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.522+0000 7f0d80984640 1 -- 172.21.15.5:0/2322643565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d70004020 con 0x7f0d7c102790 2024-09-11T22:22:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.522+0000 7f0d81986640 1 -- 172.21.15.5:0/2322643565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c19d670 con 0x7f0d7c102790 2024-09-11T22:22:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.522+0000 7f0d80984640 1 -- 172.21.15.5:0/2322643565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700047a0 con 0x7f0d7c102790 2024-09-11T22:22:27.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.522+0000 7f0d81986640 1 -- 172.21.15.5:0/2322643565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c1a4f50 con 0x7f0d7c102790 2024-09-11T22:22:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d83d5f740 1 Processor -- start 2024-09-11T22:22:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d83d5f740 1 -- start start 2024-09-11T22:22:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d98f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d83d5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d9d984d2e0 con 0x55d9d98e73a0 2024-09-11T22:22:27.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d83d5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9d97f4ce0 con 0x55d9d98f6a90 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d83d5f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d9d98492a0 con 0x55d9d98e7cc0 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d7affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57608/0 (socket says 172.21.15.5:57608) 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d7b7fe640 1 -- 172.21.15.5:0/3990165491 learned_addr learned my addr 172.21.15.5:0/3990165491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.526+0000 7f0d80984640 1 -- 172.21.15.5:0/2322643565 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f0d700041d0 con 0x7f0d7c102790 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f0d7affd640 1 -- 172.21.15.5:0/3990165491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 msgr2=0x55d9d98f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f0d7affd640 1 --2- 172.21.15.5:0/3990165491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d98f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f0d7affd640 1 -- 172.21.15.5:0/3990165491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 msgr2=0x55d9d98e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f0d7affd640 1 --2- 172.21.15.5:0/3990165491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f0d7affd640 1 -- 172.21.15.5:0/3990165491 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d97d9440 con 0x55d9d98e7cc0 2024-09-11T22:22:27.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f0d7affd640 1 --2- 172.21.15.5:0/3990165491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f6550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009c00 tx=0x7f0d6c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2277080f5f88a7fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f0d61ffb640 1 -- 172.21.15.5:0/3990165491 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c004230 con 0x55d9d98e7cc0 2024-09-11T22:22:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f0d61ffb640 1 -- 172.21.15.5:0/3990165491 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6c0043d0 con 0x55d9d98e7cc0 2024-09-11T22:22:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f0d61ffb640 1 -- 172.21.15.5:0/3990165491 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c004dd0 con 0x55d9d98e7cc0 2024-09-11T22:22:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f0d80984640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0d640a2e90 0x7f0d640a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f6392ffd640 1 -- 172.21.15.5:0/2766091565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f638c105ee0 msgr2=0x7f638c106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.527+0000 7f6392ffd640 1 --2- 172.21.15.5:0/2766091565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f638c105ee0 0x7f638c106350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6380009900 tx=0x7f638002a480 comp rx=0 tx=0).stop 2024-09-11T22:22:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f6392ffd640 1 -- 172.21.15.5:0/2766091565 shutdown_connections 2024-09-11T22:22:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f6392ffd640 1 --2- 172.21.15.5:0/2766091565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f638c105ee0 0x7f638c106350 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f6392ffd640 1 --2- 172.21.15.5:0/2766091565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f638c1013d0 0x7f638c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f6392ffd640 1 --2- 172.21.15.5:0/2766091565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c100ab0 0x7f638c100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d7b7fe640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0d640a2e90 0x7f0d640a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d80984640 1 -- 172.21.15.5:0/2322643565 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f0d700cef10 con 0x7f0d7c102790 2024-09-11T22:22:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f6392ffd640 1 -- 172.21.15.5:0/2766091565 >> 172.21.15.5:0/2766091565 conn(0x7f638c0fc6b0 msgr2=0x7f638c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f6392ffd640 1 -- 172.21.15.5:0/2766091565 shutdown_connections 2024-09-11T22:22:27.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f6392ffd640 1 -- 172.21.15.5:0/2766091565 wait complete. 2024-09-11T22:22:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3990165491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 msgr2=0x55d9d98f6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/3990165491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f6550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c009c00 tx=0x7f0d6c011810 comp rx=0 tx=0).stop 2024-09-11T22:22:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3990165491 shutdown_connections 2024-09-11T22:22:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d7b7fe640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0d640a2e90 0x7f0d640a5350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0d74004af0 tx=0x7f0d74004a80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/3990165491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d98f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/3990165491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f6550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/3990165491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3990165491 >> 172.21.15.5:0/3990165491 conn(0x55d9d98e3a30 msgr2=0x55d9d98e3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f6392ffd640 1 Processor -- start 2024-09-11T22:22:27.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6392ffd640 1 -- start start 2024-09-11T22:22:27.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.528+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3990165491 shutdown_connections 2024-09-11T22:22:27.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3990165491 wait complete. 2024-09-11T22:22:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6392ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f638c100ab0 0x7f638c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6392ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f638c1013d0 0x7f638c1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6392ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c105ee0 0x7f638c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6392ffd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f638c1185a0 con 0x7f638c100ab0 2024-09-11T22:22:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6392ffd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f638c118420 con 0x7f638c105ee0 2024-09-11T22:22:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6392ffd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f638c118720 con 0x7f638c1013d0 2024-09-11T22:22:27.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6390d99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f638c100ab0 0x7f638c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f638bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f638c1013d0 0x7f638c1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6390d99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f638c100ab0 0x7f638c1a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57614/0 (socket says 172.21.15.5:57614) 2024-09-11T22:22:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6390d99640 1 -- 172.21.15.5:0/1415528368 learned_addr learned my addr 172.21.15.5:0/1415528368 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f0d83d5f740 1 -- start start 2024-09-11T22:22:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f639159a640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c105ee0 0x7f638c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6390d99640 1 -- 172.21.15.5:0/1415528368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f638c1013d0 msgr2=0x7f638c1a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6390d99640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f638c1013d0 0x7f638c1a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6390d99640 1 -- 172.21.15.5:0/1415528368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c105ee0 msgr2=0x7f638c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6390d99640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c105ee0 0x7f638c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f6390d99640 1 -- 172.21.15.5:0/1415528368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f638c19f000 con 0x7f638c100ab0 2024-09-11T22:22:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f638bfff640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f638c1013d0 0x7f638c1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f639159a640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c105ee0 0x7f638c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:27.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.529+0000 7f0d83d5f740 1 Processor -- start 2024-09-11T22:22:27.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f6390d99640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f638c100ab0 0x7f638c1a46d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f637c00e9f0 tx=0x7f637c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:27.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f6389ffb640 1 -- 172.21.15.5:0/1415528368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c00cd80 con 0x7f638c100ab0 2024-09-11T22:22:27.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f6392ffd640 1 -- 172.21.15.5:0/1415528368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f638c19f2f0 con 0x7f638c100ab0 2024-09-11T22:22:27.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98f6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f6392ffd640 1 -- 172.21.15.5:0/1415528368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f638c1ab590 con 0x7f638c100ab0 2024-09-11T22:22:27.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f6389ffb640 1 -- 172.21.15.5:0/1415528368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f637c005500 con 0x7f638c100ab0 2024-09-11T22:22:27.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f6389ffb640 1 -- 172.21.15.5:0/1415528368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f637c010690 con 0x7f638c100ab0 2024-09-11T22:22:27.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d98f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d7affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d7affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57624/0 (socket says 172.21.15.5:57624) 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d7affd640 1 -- 172.21.15.5:0/2371848715 learned_addr learned my addr 172.21.15.5:0/2371848715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d9d98f96c0 con 0x55d9d98e73a0 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9d9808050 con 0x55d9d98f6a90 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d7b7fe640 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d98f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d9d98f9840 con 0x55d9d98e7cc0 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.530+0000 7f0d83d5f740 10 client.0 _ll_register_callbacks cb 0x55d9d9848160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.531+0000 7f0d7affd640 1 -- 172.21.15.5:0/2371848715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 msgr2=0x55d9d98f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:27.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.531+0000 7f0d7affd640 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.531+0000 7f0d7affd640 1 -- 172.21.15.5:0/2371848715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 msgr2=0x55d9d98f3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:27.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.531+0000 7f0d7affd640 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d98f3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.531+0000 7f0d7affd640 1 -- 172.21.15.5:0/2371848715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d95be3c0 con 0x55d9d98e73a0 2024-09-11T22:22:27.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.531+0000 7f0d7affd640 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98f6480 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0049b0 tx=0x7f0d6c019b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1834ebc0162ce5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:27.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.531+0000 7f0d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:27.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.531+0000 7f6392ffd640 1 -- 172.21.15.5:0/1415528368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6378005290 con 0x7f638c100ab0 2024-09-11T22:22:27.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.531+0000 7f0d5affd640 1 -- 172.21.15.5:0/2371848715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c028020 con 0x55d9d98e73a0 2024-09-11T22:22:27.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d5affd640 1 -- 172.21.15.5:0/2371848715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f0d6c024620 con 0x55d9d98e73a0 2024-09-11T22:22:27.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6c0043d0 con 0x55d9d98e73a0 2024-09-11T22:22:27.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d5affd640 1 -- 172.21.15.5:0/2371848715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d6c033050 con 0x55d9d98e73a0 2024-09-11T22:22:27.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d83d5f740 10 client.15396 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9d9905140 con 0x55d9d98e73a0 2024-09-11T22:22:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d5affd640 1 -- 172.21.15.5:0/2371848715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6c0043d0 con 0x55d9d98e73a0 2024-09-11T22:22:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d5affd640 1 -- 172.21.15.5:0/2371848715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0d6c018070 con 0x55d9d98e73a0 2024-09-11T22:22:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d83d5f740 10 client.15396 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d83d5f740 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:22:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d83d5f740 20 client.15396 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9d9906980 0x55d9d9908d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d83d5f740 4 client.15396 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.532+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9d985a850 con 0x55d9d9906980 2024-09-11T22:22:27.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.533+0000 7f6389ffb640 1 -- 172.21.15.5:0/1415528368 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f637c0046c0 con 0x7f638c100ab0 2024-09-11T22:22:27.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.533+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9d9906980 0x55d9d9908d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.533+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9d9906980 0x55d9d9908d90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d03721a1e62904e5 server_cookie=d775e9791e91d579 in_seq=0 out_seq=0 2024-09-11T22:22:27.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.533+0000 7f0d5affd640 10 client.15396 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:27.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.534+0000 7f6389ffb640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63740a2fc0 0x7f63740a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.534+0000 7f0d5affd640 1 -- 172.21.15.5:0/2371848715 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9d985a850 con 0x55d9d9906980 2024-09-11T22:22:27.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.534+0000 7f0d5affd640 10 client.15396 handle_command_reply: tid=0 2024-09-11T22:22:27.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.534+0000 7f638bfff640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63740a2fc0 0x7f63740a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.534+0000 7f6389ffb640 1 -- 172.21.15.5:0/1415528368 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f637c0cfa20 con 0x7f638c100ab0 2024-09-11T22:22:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.534+0000 7f0d83d5f740 1 client.15396 shutdown 2024-09-11T22:22:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.534+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 msgr2=0x55d9d98f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.534+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98f6480 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0049b0 tx=0x7f0d6c019b90 comp rx=0 tx=0).stop 2024-09-11T22:22:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 shutdown_connections 2024-09-11T22:22:27.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9d9906980 0x55d9d9908d90 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d98f3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f0d5affd640 0 client.15396 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/2371848715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e73a0 0x55d9d98f6480 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 >> 172.21.15.5:0/2371848715 conn(0x55d9d98e3a30 msgr2=0x55d9d98e9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 shutdown_connections 2024-09-11T22:22:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f0d83d5f740 1 -- 172.21.15.5:0/2371848715 wait complete. 2024-09-11T22:22:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f638bfff640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63740a2fc0 0x7f63740a5480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f638c106550 tx=0x7f638400d040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.535+0000 7f0d83d5f740 20 client.15396 trim_cache size 0 max 16384 2024-09-11T22:22:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.539+0000 7f6389ffb640 1 -- 172.21.15.5:0/1415528368 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f637c09c390 con 0x7f638c100ab0 2024-09-11T22:22:27.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d83d5f740 1 Processor -- start 2024-09-11T22:22:27.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d83d5f740 1 -- start start 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e73a0 0x55d9d9902960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d990e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d83d5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1ab7c0 con 0x55d9d98e7cc0 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d83d5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1ad300 con 0x55d9d98f6a90 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d83d5f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1abe90 con 0x55d9d98e73a0 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d7affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e73a0 0x55d9d9902960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d7affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e73a0 0x55d9d9902960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59046/0 (socket says 172.21.15.5:59046) 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d7affd640 1 -- 172.21.15.5:0/4164824040 learned_addr learned my addr 172.21.15.5:0/4164824040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/4164824040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.553+0000 7f0d7b7fe640 1 --2- 172.21.15.5:0/4164824040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d990e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.554+0000 7f0d7affd640 1 -- 172.21.15.5:0/4164824040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 msgr2=0x55d9d990e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:27.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.554+0000 7f0d7affd640 1 --2- 172.21.15.5:0/4164824040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d990e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.554+0000 7f0d7affd640 1 -- 172.21.15.5:0/4164824040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e7cc0 msgr2=0x55d9d98f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:27.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.554+0000 7f0d7affd640 1 --2- 172.21.15.5:0/4164824040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.554+0000 7f0d7affd640 1 -- 172.21.15.5:0/4164824040 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9d98f26e0 con 0x55d9d98e73a0 2024-09-11T22:22:27.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.554+0000 7f0d7affd640 1 --2- 172.21.15.5:0/4164824040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e73a0 0x55d9d9902960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c012fd0 tx=0x7f0d6c009a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d03c2abafcc721 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:27.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.554+0000 7f0d617fa640 1 -- 172.21.15.5:0/4164824040 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c028040 con 0x55d9d98e73a0 2024-09-11T22:22:27.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.554+0000 7f0d617fa640 1 -- 172.21.15.5:0/4164824040 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6c01f040 con 0x55d9d98e73a0 2024-09-11T22:22:27.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.554+0000 7f0d617fa640 1 -- 172.21.15.5:0/4164824040 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c004570 con 0x55d9d98e73a0 2024-09-11T22:22:27.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.555+0000 7f0d83d5f740 1 -- 172.21.15.5:0/4164824040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e73a0 msgr2=0x55d9d9902960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.555+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/4164824040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e73a0 0x55d9d9902960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c012fd0 tx=0x7f0d6c009a50 comp rx=0 tx=0).stop 2024-09-11T22:22:27.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.555+0000 7f0d83d5f740 1 -- 172.21.15.5:0/4164824040 shutdown_connections 2024-09-11T22:22:27.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.555+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/4164824040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98f6a90 0x55d9d990e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.555+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/4164824040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98e7cc0 0x55d9d98f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.555+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/4164824040 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e73a0 0x55d9d9902960 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.555+0000 7f0d83d5f740 1 -- 172.21.15.5:0/4164824040 >> 172.21.15.5:0/4164824040 conn(0x55d9d98e3a30 msgr2=0x55d9d98eed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.555+0000 7f0d83d5f740 1 -- 172.21.15.5:0/4164824040 shutdown_connections 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.555+0000 7f0d83d5f740 1 -- 172.21.15.5:0/4164824040 wait complete. 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d83d5f740 1 -- start start 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d83d5f740 1 Processor -- start 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98e73a0 0x55d9d98ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98fa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d83d5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98f6a90 0x55d9d98fae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d7b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98f6a90 0x55d9d98fae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d83d5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1acc30 con 0x55d9d98f6a90 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d83d5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1ac560 con 0x55d9d98e73a0 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d83d5f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1a97a0 con 0x55d9d98e7cc0 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d83d5f740 10 client.0 _ll_register_callbacks cb 0x7f0d74009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:27.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.556+0000 7f0d7affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98e73a0 0x55d9d98ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d7affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98e73a0 0x55d9d98ff3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55394/0 (socket says 172.21.15.5:55394) 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d7affd640 1 -- 172.21.15.5:0/3307414308 learned_addr learned my addr 172.21.15.5:0/3307414308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98fa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d7a7fc640 1 -- 172.21.15.5:0/3307414308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98e73a0 msgr2=0x55d9d98ff3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98e73a0 0x55d9d98ff3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d7a7fc640 1 -- 172.21.15.5:0/3307414308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98f6a90 msgr2=0x55d9d98fae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98f6a90 0x55d9d98fae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d7a7fc640 1 -- 172.21.15.5:0/3307414308 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9d98e7af0 con 0x55d9d98e7cc0 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d7b7fe640 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98f6a90 0x55d9d98fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d7a7fc640 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98fa6e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0d700cc4e0 tx=0x7f0d700cf210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=369117dc052734dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:22:27.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.557+0000 7f0d5affd640 1 -- 172.21.15.5:0/3307414308 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7009b1e0 con 0x55d9d98e7cc0 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3307414308 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d6c01f040 con 0x55d9d98e7cc0 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d5affd640 1 -- 172.21.15.5:0/3307414308 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f0d70097850 con 0x55d9d98e7cc0 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d5affd640 1 -- 172.21.15.5:0/3307414308 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d7009f590 con 0x55d9d98e7cc0 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d83d5f740 10 client.25012 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3307414308 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9d98f4090 con 0x55d9d98e7cc0 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d5affd640 1 -- 172.21.15.5:0/3307414308 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d6c01f040 con 0x55d9d98e7cc0 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d5affd640 1 -- 172.21.15.5:0/3307414308 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0d700cc7c0 con 0x55d9d98e7cc0 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d83d5f740 10 client.25012 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d83d5f740 10 client.25012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d83d5f740 20 client.25012 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9d9906980 0x55d9d98ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:27.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d83d5f740 4 client.25012 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:22:27.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.558+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3307414308 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9d9854d60 con 0x55d9d9906980 2024-09-11T22:22:27.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.559+0000 7f0d7affd640 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9d9906980 0x55d9d98ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.559+0000 7f0d5affd640 10 client.25012 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.559+0000 7f0d7affd640 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9d9906980 0x55d9d98ec5f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6da3dd668e2494 server_cookie=8fb1f80c1b623668 in_seq=0 out_seq=0 2024-09-11T22:22:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.559+0000 7f0d5affd640 1 -- 172.21.15.5:0/3307414308 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d9d9854d60 con 0x55d9d9906980 2024-09-11T22:22:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.559+0000 7f0d5affd640 10 client.25012 handle_command_reply: tid=0 2024-09-11T22:22:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.559+0000 7f0d83d5f740 1 client.25012 shutdown 2024-09-11T22:22:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3307414308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 msgr2=0x55d9d98fa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98fa6e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0d700cc4e0 tx=0x7f0d700cf210 comp rx=0 tx=0).stop 2024-09-11T22:22:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3307414308 shutdown_connections 2024-09-11T22:22:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9d9906980 0x55d9d98ec5f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9d98f6a90 0x55d9d98fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d5affd640 0 client.25012 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:27.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9d98e7cc0 0x55d9d98fa6e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 1 --2- 172.21.15.5:0/3307414308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9d98e73a0 0x55d9d98ff3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3307414308 >> 172.21.15.5:0/3307414308 conn(0x55d9d98e3a30 msgr2=0x55d9d98eed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3307414308 shutdown_connections 2024-09-11T22:22:27.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 1 -- 172.21.15.5:0/3307414308 wait complete. 2024-09-11T22:22:27.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.560+0000 7f0d83d5f740 20 client.25012 trim_cache size 0 max 16384 2024-09-11T22:22:27.565 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:27.565 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:22:27.566 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:22:27.566 INFO:teuthology.orchestra.run.smithi005.stdout: "c0c38f9b-957e-42ff-97d9-6f69752dbfcb": { 2024-09-11T22:22:27.566 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:22:27.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "c0c38f9b-957e-42ff-97d9-6f69752dbfcb", 2024-09-11T22:22:27.566 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:22:27.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:22:27.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:22:27.566 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:27.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 -- 172.21.15.5:0/2322643565 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0d640a2e90 msgr2=0x7f0d640a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0d640a2e90 0x7f0d640a5350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0d74004af0 tx=0x7f0d74004a80 comp rx=0 tx=0).stop 2024-09-11T22:22:27.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 -- 172.21.15.5:0/2322643565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c102790 msgr2=0x7f0d7c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c102790 0x7f0d7c198910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d950 tx=0x7f0d7000de10 comp rx=0 tx=0).stop 2024-09-11T22:22:27.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 -- 172.21.15.5:0/2322643565 shutdown_connections 2024-09-11T22:22:27.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c106f10 0x7f0d7c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c102790 0x7f0d7c198910 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0d640a2e90 0x7f0d640a5350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 --2- 172.21.15.5:0/2322643565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c101de0 0x7f0d7c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 -- 172.21.15.5:0/2322643565 >> 172.21.15.5:0/2322643565 conn(0x7f0d7c0fd8c0 msgr2=0x7f0d7c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 -- 172.21.15.5:0/2322643565 shutdown_connections 2024-09-11T22:22:27.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.561+0000 7f0d81986640 1 -- 172.21.15.5:0/2322643565 wait complete. 2024-09-11T22:22:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.750+0000 7f6392ffd640 1 -- 172.21.15.5:0/1415528368 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f6378000bc0 con 0x7f63740a2fc0 2024-09-11T22:22:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.758+0000 7f6389ffb640 1 -- 172.21.15.5:0/1415528368 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f6378000bc0 con 0x7f63740a2fc0 2024-09-11T22:22:27.758 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-09-11T22:22:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 -- 172.21.15.5:0/1415528368 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63740a2fc0 msgr2=0x7f63740a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63740a2fc0 0x7f63740a5480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f638c106550 tx=0x7f638400d040 comp rx=0 tx=0).stop 2024-09-11T22:22:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 -- 172.21.15.5:0/1415528368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f638c100ab0 msgr2=0x7f638c1a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f638c100ab0 0x7f638c1a46d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f637c00e9f0 tx=0x7f637c00edf0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 -- 172.21.15.5:0/1415528368 shutdown_connections 2024-09-11T22:22:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63740a2fc0 0x7f63740a5480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f638c105ee0 0x7f638c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f638c1013d0 0x7f638c1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 --2- 172.21.15.5:0/1415528368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f638c100ab0 0x7f638c1a46d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 -- 172.21.15.5:0/1415528368 >> 172.21.15.5:0/1415528368 conn(0x7f638c0fc6b0 msgr2=0x7f638c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 -- 172.21.15.5:0/1415528368 shutdown_connections 2024-09-11T22:22:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:27.760+0000 7f6392ffd640 1 -- 172.21.15.5:0/1415528368 wait complete. 2024-09-11T22:22:27.773 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-11T22:22:27.800 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-11T22:22:27.819 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-11T22:22:27.841 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T22:22:27.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-11T22:22:27.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:27 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:22:27.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:27 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:22:27.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:27 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:22:27.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:27 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:22:27.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:27 smithi005 ceph-mon[30943]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-09-11T22:22:27.843 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:27 smithi005 ceph-mon[30943]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:27.844 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-11T22:22:27.876 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T22:22:27.877 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:22:28.004 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:27 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:22:28.004 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:27 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:22:28.004 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:27 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:22:28.004 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:27 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:22:28.004 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:27 smithi159 ceph-mon[35534]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-09-11T22:22:28.004 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:27 smithi159 ceph-mon[35534]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:27 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:22:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:27 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:22:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:27 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:22:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:27 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:22:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:27 smithi172 ceph-mon[35461]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 911 B/s wr, 20 op/s 2024-09-11T22:22:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:27 smithi172 ceph-mon[35461]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:28.308 INFO:tasks.cephfs.filesystem:scrub status for tag:c0c38f9b-957e-42ff-97d9-6f69752dbfcb - {'path': '/', 'tag': 'c0c38f9b-957e-42ff-97d9-6f69752dbfcb', 'options': 'recursive,force'} 2024-09-11T22:22:28.375 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:28.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:28 smithi005 ceph-mon[30943]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:28.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:28 smithi005 ceph-mon[30943]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:28.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:28 smithi005 ceph-mon[30943]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.891+0000 7fb8fa858640 1 -- 172.21.15.5:0/1777911817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8f41055b0 msgr2=0x7fb8f410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.891+0000 7fb8fa858640 1 --2- 172.21.15.5:0/1777911817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8f41055b0 0x7fb8f410d130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec009ca0 tx=0x7fb8ec02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.892+0000 7fb8fa858640 1 -- 172.21.15.5:0/1777911817 shutdown_connections 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.892+0000 7fb8fa858640 1 --2- 172.21.15.5:0/1777911817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8f41055b0 0x7fb8f410d130 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.892+0000 7fb8fa858640 1 --2- 172.21.15.5:0/1777911817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f4100e30 0x7fb8f4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.892+0000 7fb8fa858640 1 --2- 172.21.15.5:0/1777911817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f4100480 0x7fb8f4100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.892+0000 7fb8fa858640 1 -- 172.21.15.5:0/1777911817 >> 172.21.15.5:0/1777911817 conn(0x7fb8f4077210 msgr2=0x7fb8f40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.892+0000 7fb8fa858640 1 -- 172.21.15.5:0/1777911817 shutdown_connections 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.892+0000 7fb8fa858640 1 -- 172.21.15.5:0/1777911817 wait complete. 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.892+0000 7fb8fa858640 1 Processor -- start 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.892+0000 7fb8fa858640 1 -- start start 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8fa858640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f4100480 0x7fb8f4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8fa858640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8f4100e30 0x7fb8f4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8fa858640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f41055b0 0x7fb8f419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8fa858640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f410fd90 con 0x7fb8f4100e30 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8fa858640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f410fc10 con 0x7fb8f41055b0 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8fa858640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f410ff10 con 0x7fb8f4100480 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8f4100e30 0x7fb8f4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8f4100e30 0x7fb8f4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57648/0 (socket says 172.21.15.5:57648) 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f37fe640 1 -- 172.21.15.5:0/2114331055 learned_addr learned my addr 172.21.15.5:0/2114331055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f8df5640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f41055b0 0x7fb8f419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f3fff640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f4100480 0x7fb8f4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f37fe640 1 -- 172.21.15.5:0/2114331055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f4100480 msgr2=0x7fb8f4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f37fe640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f4100480 0x7fb8f4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f37fe640 1 -- 172.21.15.5:0/2114331055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f41055b0 msgr2=0x7fb8f419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f37fe640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f41055b0 0x7fb8f419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f37fe640 1 -- 172.21.15.5:0/2114331055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f419d3c0 con 0x7fb8f4100e30 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f37fe640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8f4100e30 0x7fb8f4198950 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800e9f0 tx=0x7fb8e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f17fa640 1 -- 172.21.15.5:0/2114331055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e800cd80 con 0x7fb8f4100e30 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f17fa640 1 -- 172.21.15.5:0/2114331055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8e8010910 con 0x7fb8f4100e30 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8f17fa640 1 -- 172.21.15.5:0/2114331055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8018990 con 0x7fb8f4100e30 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.893+0000 7fb8fa858640 1 -- 172.21.15.5:0/2114331055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f419d6b0 con 0x7fb8f4100e30 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.894+0000 7fb8fa858640 1 -- 172.21.15.5:0/2114331055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f41a4f90 con 0x7fb8f4100e30 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.894+0000 7fb8fa858640 1 -- 172.21.15.5:0/2114331055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f4100860 con 0x7fb8f4100e30 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.896+0000 7fb8f17fa640 1 -- 172.21.15.5:0/2114331055 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7fb8e8010ac0 con 0x7fb8f4100e30 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.896+0000 7fb8f17fa640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb8dc0a2e90 0x7fb8dc0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.896+0000 7fb8f3fff640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb8dc0a2e90 0x7fb8dc0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.896+0000 7fb8f17fa640 1 -- 172.21.15.5:0/2114331055 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fb8e80cf8d0 con 0x7fb8f4100e30 2024-09-11T22:22:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.897+0000 7fb8f3fff640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb8dc0a2e90 0x7fb8dc0a5350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400ac40 tx=0x7fb8e4010040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:28.901+0000 7fb8f17fa640 1 -- 172.21.15.5:0/2114331055 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8e809bc50 con 0x7fb8f4100e30 2024-09-11T22:22:29.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.074+0000 7fb8fa858640 1 -- 172.21.15.5:0/2114331055 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb8f4062630 con 0x7fb8dc0a2e90 2024-09-11T22:22:29.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:28 smithi159 ceph-mon[35534]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:29.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:28 smithi159 ceph-mon[35534]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:29.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:28 smithi159 ceph-mon[35534]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:29.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.081+0000 7fb8f17fa640 1 -- 172.21.15.5:0/2114331055 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258084 (secure 0 0 0) 0x7fb8e405c040 con 0x7fb8dc0a2e90 2024-09-11T22:22:29.084 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:29.085 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-11T22:22:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 -- 172.21.15.5:0/2114331055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb8dc0a2e90 msgr2=0x7fb8dc0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb8dc0a2e90 0x7fb8dc0a5350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400ac40 tx=0x7fb8e4010040 comp rx=0 tx=0).stop 2024-09-11T22:22:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 -- 172.21.15.5:0/2114331055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8f4100e30 msgr2=0x7fb8f4198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8f4100e30 0x7fb8f4198950 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800e9f0 tx=0x7fb8e800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:22:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 -- 172.21.15.5:0/2114331055 shutdown_connections 2024-09-11T22:22:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb8dc0a2e90 0x7fb8dc0a5350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb8f41055b0 0x7fb8f419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8f4100e30 0x7fb8f4198950 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 --2- 172.21.15.5:0/2114331055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8f4100480 0x7fb8f4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 -- 172.21.15.5:0/2114331055 >> 172.21.15.5:0/2114331055 conn(0x7fb8f4077210 msgr2=0x7fb8f4103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 -- 172.21.15.5:0/2114331055 shutdown_connections 2024-09-11T22:22:29.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:29.084+0000 7fb8fa858640 1 -- 172.21.15.5:0/2114331055 wait complete. 2024-09-11T22:22:29.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:28 smithi172 ceph-mon[35461]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:29.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:28 smithi172 ceph-mon[35461]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:29.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:28 smithi172 ceph-mon[35461]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:22:29.780 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-11T22:22:27.988468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391065,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1679,"num_read_kb":1266,"num_write":683,"num_write_kb":5325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181928,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1021,"ondisk_log_size":1021,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385620,"kb_used_data":11388,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556204,"statfs":{"total":1151940427776,"available":1151545552896,"internally_reserved":0,"allocated":11661312,"data_stored":8888895,"data_compressed":289017,"data_compressed_allocated":380928,"data_compressed_original":3612672,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200331,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":385,"num_read_kb":430,"num_write":128,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.988742"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235096+0000","last_change":"2024-09-11T22:20:18.189744+0000","last_active":"2024-09-11T22:22:16.235096+0000","last_peered":"2024-09-11T22:22:16.235096+0000","last_clean":"2024-09-11T22:22:16.235096+0000","last_became_active":"2024-09-11T22:20:18.189398+0000","last_became_peered":"2024-09-11T22:20:18.189398+0000","last_unstale":"2024-09-11T22:22:16.235096+0000","last_undegraded":"2024-09-11T22:22:16.235096+0000","last_fullsized":"2024-09-11T22:22:16.235096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:24.152589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234185+0000","last_change":"2024-09-11T22:20:18.186727+0000","last_active":"2024-09-11T22:22:16.234185+0000","last_peered":"2024-09-11T22:22:16.234185+0000","last_clean":"2024-09-11T22:22:16.234185+0000","last_became_active":"2024-09-11T22:20:18.186414+0000","last_became_peered":"2024-09-11T22:20:18.186414+0000","last_unstale":"2024-09-11T22:22:16.234185+0000","last_undegraded":"2024-09-11T22:22:16.234185+0000","last_fullsized":"2024-09-11T22:22:16.234185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:53.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233909+0000","last_change":"2024-09-11T22:20:18.179984+0000","last_active":"2024-09-11T22:22:16.233909+0000","last_peered":"2024-09-11T22:22:16.233909+0000","last_clean":"2024-09-11T22:22:16.233909+0000","last_became_active":"2024-09-11T22:20:18.179329+0000","last_became_peered":"2024-09-11T22:20:18.179329+0000","last_unstale":"2024-09-11T22:22:16.233909+0000","last_undegraded":"2024-09-11T22:22:16.233909+0000","last_fullsized":"2024-09-11T22:22:16.233909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:19.149970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230253+0000","last_change":"2024-09-11T22:20:18.191768+0000","last_active":"2024-09-11T22:22:16.230253+0000","last_peered":"2024-09-11T22:22:16.230253+0000","last_clean":"2024-09-11T22:22:16.230253+0000","last_became_active":"2024-09-11T22:20:18.191336+0000","last_became_peered":"2024-09-11T22:20:18.191336+0000","last_unstale":"2024-09-11T22:22:16.230253+0000","last_undegraded":"2024-09-11T22:22:16.230253+0000","last_fullsized":"2024-09-11T22:22:16.230253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:44.768932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058858+0000","last_change":"2024-09-11T22:20:18.181469+0000","last_active":"2024-09-11T22:22:16.058858+0000","last_peered":"2024-09-11T22:22:16.058858+0000","last_clean":"2024-09-11T22:22:16.058858+0000","last_became_active":"2024-09-11T22:20:18.181100+0000","last_became_peered":"2024-09-11T22:20:18.181100+0000","last_unstale":"2024-09-11T22:22:16.058858+0000","last_undegraded":"2024-09-11T22:22:16.058858+0000","last_fullsized":"2024-09-11T22:22:16.058858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:14.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068675+0000","last_change":"2024-09-11T22:20:18.190855+0000","last_active":"2024-09-11T22:22:16.068675+0000","last_peered":"2024-09-11T22:22:16.068675+0000","last_clean":"2024-09-11T22:22:16.068675+0000","last_became_active":"2024-09-11T22:20:18.190523+0000","last_became_peered":"2024-09-11T22:20:18.190523+0000","last_unstale":"2024-09-11T22:22:16.068675+0000","last_undegraded":"2024-09-11T22:22:16.068675+0000","last_fullsized":"2024-09-11T22:22:16.068675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:18.574957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231733+0000","last_change":"2024-09-11T22:20:18.199213+0000","last_active":"2024-09-11T22:22:16.231733+0000","last_peered":"2024-09-11T22:22:16.231733+0000","last_clean":"2024-09-11T22:22:16.231733+0000","last_became_active":"2024-09-11T22:20:18.198913+0000","last_became_peered":"2024-09-11T22:20:18.198913+0000","last_unstale":"2024-09-11T22:22:16.231733+0000","last_undegraded":"2024-09-11T22:22:16.231733+0000","last_fullsized":"2024-09-11T22:22:16.231733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:22.682802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058633+0000","last_change":"2024-09-11T22:20:18.192458+0000","last_active":"2024-09-11T22:22:16.058633+0000","last_peered":"2024-09-11T22:22:16.058633+0000","last_clean":"2024-09-11T22:22:16.058633+0000","last_became_active":"2024-09-11T22:20:18.191967+0000","last_became_peered":"2024-09-11T22:20:18.191967+0000","last_unstale":"2024-09-11T22:22:16.058633+0000","last_undegraded":"2024-09-11T22:22:16.058633+0000","last_fullsized":"2024-09-11T22:22:16.058633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:52.686157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975713+0000","last_change":"2024-09-11T22:21:18.191732+0000","last_active":"2024-09-11T22:22:15.975713+0000","last_peered":"2024-09-11T22:22:15.975713+0000","last_clean":"2024-09-11T22:22:15.975713+0000","last_became_active":"2024-09-11T22:21:18.191375+0000","last_became_peered":"2024-09-11T22:21:18.191375+0000","last_unstale":"2024-09-11T22:22:15.975713+0000","last_undegraded":"2024-09-11T22:22:15.975713+0000","last_fullsized":"2024-09-11T22:22:15.975713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:13.645224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.069211+0000","last_change":"2024-09-11T22:20:18.193771+0000","last_active":"2024-09-11T22:22:16.069211+0000","last_peered":"2024-09-11T22:22:16.069211+0000","last_clean":"2024-09-11T22:22:16.069211+0000","last_became_active":"2024-09-11T22:20:18.193041+0000","last_became_peered":"2024-09-11T22:20:18.193041+0000","last_unstale":"2024-09-11T22:22:16.069211+0000","last_undegraded":"2024-09-11T22:22:16.069211+0000","last_fullsized":"2024-09-11T22:22:16.069211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:06.656169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231998+0000","last_change":"2024-09-11T22:20:18.197933+0000","last_active":"2024-09-11T22:22:16.231998+0000","last_peered":"2024-09-11T22:22:16.231998+0000","last_clean":"2024-09-11T22:22:16.231998+0000","last_became_active":"2024-09-11T22:20:18.197210+0000","last_became_peered":"2024-09-11T22:20:18.197210+0000","last_unstale":"2024-09-11T22:22:16.231998+0000","last_undegraded":"2024-09-11T22:22:16.231998+0000","last_fullsized":"2024-09-11T22:22:16.231998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:42.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230250+0000","last_change":"2024-09-11T22:20:18.190739+0000","last_active":"2024-09-11T22:22:16.230250+0000","last_peered":"2024-09-11T22:22:16.230250+0000","last_clean":"2024-09-11T22:22:16.230250+0000","last_became_active":"2024-09-11T22:20:18.190122+0000","last_became_peered":"2024-09-11T22:20:18.190122+0000","last_unstale":"2024-09-11T22:22:16.230250+0000","last_undegraded":"2024-09-11T22:22:16.230250+0000","last_fullsized":"2024-09-11T22:22:16.230250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.585035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230697+0000","last_change":"2024-09-11T22:20:18.190303+0000","last_active":"2024-09-11T22:22:16.230697+0000","last_peered":"2024-09-11T22:22:16.230697+0000","last_clean":"2024-09-11T22:22:16.230697+0000","last_became_active":"2024-09-11T22:20:18.189962+0000","last_became_peered":"2024-09-11T22:20:18.189962+0000","last_unstale":"2024-09-11T22:22:16.230697+0000","last_undegraded":"2024-09-11T22:22:16.230697+0000","last_fullsized":"2024-09-11T22:22:16.230697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:44.872813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233520+0000","last_change":"2024-09-11T22:20:18.185893+0000","last_active":"2024-09-11T22:22:16.233520+0000","last_peered":"2024-09-11T22:22:16.233520+0000","last_clean":"2024-09-11T22:22:16.233520+0000","last_became_active":"2024-09-11T22:20:18.185592+0000","last_became_peered":"2024-09-11T22:20:18.185592+0000","last_unstale":"2024-09-11T22:22:16.233520+0000","last_undegraded":"2024-09-11T22:22:16.233520+0000","last_fullsized":"2024-09-11T22:22:16.233520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:14.725301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233777+0000","last_change":"2024-09-11T22:20:18.193943+0000","last_active":"2024-09-11T22:22:16.233777+0000","last_peered":"2024-09-11T22:22:16.233777+0000","last_clean":"2024-09-11T22:22:16.233777+0000","last_became_active":"2024-09-11T22:20:18.193558+0000","last_became_peered":"2024-09-11T22:20:18.193558+0000","last_unstale":"2024-09-11T22:22:16.233777+0000","last_undegraded":"2024-09-11T22:22:16.233777+0000","last_fullsized":"2024-09-11T22:22:16.233777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:08.736161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975254+0000","last_change":"2024-09-11T22:20:18.191815+0000","last_active":"2024-09-11T22:22:15.975254+0000","last_peered":"2024-09-11T22:22:15.975254+0000","last_clean":"2024-09-11T22:22:15.975254+0000","last_became_active":"2024-09-11T22:20:18.191483+0000","last_became_peered":"2024-09-11T22:20:18.191483+0000","last_unstale":"2024-09-11T22:22:15.975254+0000","last_undegraded":"2024-09-11T22:22:15.975254+0000","last_fullsized":"2024-09-11T22:22:15.975254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:26.432117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235101+0000","last_change":"2024-09-11T22:20:18.190758+0000","last_active":"2024-09-11T22:22:16.235101+0000","last_peered":"2024-09-11T22:22:16.235101+0000","last_clean":"2024-09-11T22:22:16.235101+0000","last_became_active":"2024-09-11T22:20:18.190431+0000","last_became_peered":"2024-09-11T22:20:18.190431+0000","last_unstale":"2024-09-11T22:22:16.235101+0000","last_undegraded":"2024-09-11T22:22:16.235101+0000","last_fullsized":"2024-09-11T22:22:16.235101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:10.927051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973552+0000","last_change":"2024-09-11T22:20:18.185191+0000","last_active":"2024-09-11T22:22:15.973552+0000","last_peered":"2024-09-11T22:22:15.973552+0000","last_clean":"2024-09-11T22:22:15.973552+0000","last_became_active":"2024-09-11T22:20:18.184906+0000","last_became_peered":"2024-09-11T22:20:18.184906+0000","last_unstale":"2024-09-11T22:22:15.973552+0000","last_undegraded":"2024-09-11T22:22:15.973552+0000","last_fullsized":"2024-09-11T22:22:15.973552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:32.294780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977515+0000","last_change":"2024-09-11T22:20:18.187137+0000","last_active":"2024-09-11T22:22:15.977515+0000","last_peered":"2024-09-11T22:22:15.977515+0000","last_clean":"2024-09-11T22:22:15.977515+0000","last_became_active":"2024-09-11T22:20:18.186770+0000","last_became_peered":"2024-09-11T22:20:18.186770+0000","last_unstale":"2024-09-11T22:22:15.977515+0000","last_undegraded":"2024-09-11T22:22:15.977515+0000","last_fullsized":"2024-09-11T22:22:15.977515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:59.917695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234743+0000","last_change":"2024-09-11T22:20:18.197137+0000","last_active":"2024-09-11T22:22:16.234743+0000","last_peered":"2024-09-11T22:22:16.234743+0000","last_clean":"2024-09-11T22:22:16.234743+0000","last_became_active":"2024-09-11T22:20:18.192810+0000","last_became_peered":"2024-09-11T22:20:18.192810+0000","last_unstale":"2024-09-11T22:22:16.234743+0000","last_undegraded":"2024-09-11T22:22:16.234743+0000","last_fullsized":"2024-09-11T22:22:16.234743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:57.680726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230830+0000","last_change":"2024-09-11T22:20:18.187568+0000","last_active":"2024-09-11T22:22:16.230830+0000","last_peered":"2024-09-11T22:22:16.230830+0000","last_clean":"2024-09-11T22:22:16.230830+0000","last_became_active":"2024-09-11T22:20:18.187254+0000","last_became_peered":"2024-09-11T22:20:18.187254+0000","last_unstale":"2024-09-11T22:22:16.230830+0000","last_undegraded":"2024-09-11T22:22:16.230830+0000","last_fullsized":"2024-09-11T22:22:16.230830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:41.724926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975378+0000","last_change":"2024-09-11T22:20:18.190376+0000","last_active":"2024-09-11T22:22:15.975378+0000","last_peered":"2024-09-11T22:22:15.975378+0000","last_clean":"2024-09-11T22:22:15.975378+0000","last_became_active":"2024-09-11T22:20:18.190079+0000","last_became_peered":"2024-09-11T22:20:18.190079+0000","last_unstale":"2024-09-11T22:22:15.975378+0000","last_undegraded":"2024-09-11T22:22:15.975378+0000","last_fullsized":"2024-09-11T22:22:15.975378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:42.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058896+0000","last_change":"2024-09-11T22:20:18.188503+0000","last_active":"2024-09-11T22:22:16.058896+0000","last_peered":"2024-09-11T22:22:16.058896+0000","last_clean":"2024-09-11T22:22:16.058896+0000","last_became_active":"2024-09-11T22:20:18.187957+0000","last_became_peered":"2024-09-11T22:20:18.187957+0000","last_unstale":"2024-09-11T22:22:16.058896+0000","last_undegraded":"2024-09-11T22:22:16.058896+0000","last_fullsized":"2024-09-11T22:22:16.058896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:44:25.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065572+0000","last_change":"2024-09-11T22:20:18.196936+0000","last_active":"2024-09-11T22:22:16.065572+0000","last_peered":"2024-09-11T22:22:16.065572+0000","last_clean":"2024-09-11T22:22:16.065572+0000","last_became_active":"2024-09-11T22:20:18.196493+0000","last_became_peered":"2024-09-11T22:20:18.196493+0000","last_unstale":"2024-09-11T22:22:16.065572+0000","last_undegraded":"2024-09-11T22:22:16.065572+0000","last_fullsized":"2024-09-11T22:22:16.065572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:06.189806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059171+0000","last_change":"2024-09-11T22:20:18.195260+0000","last_active":"2024-09-11T22:22:16.059171+0000","last_peered":"2024-09-11T22:22:16.059171+0000","last_clean":"2024-09-11T22:22:16.059171+0000","last_became_active":"2024-09-11T22:20:18.194956+0000","last_became_peered":"2024-09-11T22:20:18.194956+0000","last_unstale":"2024-09-11T22:22:16.059171+0000","last_undegraded":"2024-09-11T22:22:16.059171+0000","last_fullsized":"2024-09-11T22:22:16.059171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:34.483291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977741+0000","last_change":"2024-09-11T22:20:18.190591+0000","last_active":"2024-09-11T22:22:15.977741+0000","last_peered":"2024-09-11T22:22:15.977741+0000","last_clean":"2024-09-11T22:22:15.977741+0000","last_became_active":"2024-09-11T22:20:18.190007+0000","last_became_peered":"2024-09-11T22:20:18.190007+0000","last_unstale":"2024-09-11T22:22:15.977741+0000","last_undegraded":"2024-09-11T22:22:15.977741+0000","last_fullsized":"2024-09-11T22:22:15.977741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:46.281705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234929+0000","last_change":"2024-09-11T22:20:18.193377+0000","last_active":"2024-09-11T22:22:16.234929+0000","last_peered":"2024-09-11T22:22:16.234929+0000","last_clean":"2024-09-11T22:22:16.234929+0000","last_became_active":"2024-09-11T22:20:18.193003+0000","last_became_peered":"2024-09-11T22:20:18.193003+0000","last_unstale":"2024-09-11T22:22:16.234929+0000","last_undegraded":"2024-09-11T22:22:16.234929+0000","last_fullsized":"2024-09-11T22:22:16.234929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:59.052966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230737+0000","last_change":"2024-09-11T22:20:18.189405+0000","last_active":"2024-09-11T22:22:16.230737+0000","last_peered":"2024-09-11T22:22:16.230737+0000","last_clean":"2024-09-11T22:22:16.230737+0000","last_became_active":"2024-09-11T22:20:18.189098+0000","last_became_peered":"2024-09-11T22:20:18.189098+0000","last_unstale":"2024-09-11T22:22:16.230737+0000","last_undegraded":"2024-09-11T22:22:16.230737+0000","last_fullsized":"2024-09-11T22:22:16.230737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:47.088192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233900+0000","last_change":"2024-09-11T22:20:18.191005+0000","last_active":"2024-09-11T22:22:16.233900+0000","last_peered":"2024-09-11T22:22:16.233900+0000","last_clean":"2024-09-11T22:22:16.233900+0000","last_became_active":"2024-09-11T22:20:18.190681+0000","last_became_peered":"2024-09-11T22:20:18.190681+0000","last_unstale":"2024-09-11T22:22:16.233900+0000","last_undegraded":"2024-09-11T22:22:16.233900+0000","last_fullsized":"2024-09-11T22:22:16.233900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:12.903929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977265+0000","last_change":"2024-09-11T22:20:18.187920+0000","last_active":"2024-09-11T22:22:15.977265+0000","last_peered":"2024-09-11T22:22:15.977265+0000","last_clean":"2024-09-11T22:22:15.977265+0000","last_became_active":"2024-09-11T22:20:18.187526+0000","last_became_peered":"2024-09-11T22:20:18.187526+0000","last_unstale":"2024-09-11T22:22:15.977265+0000","last_undegraded":"2024-09-11T22:22:15.977265+0000","last_fullsized":"2024-09-11T22:22:15.977265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.347857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977444+0000","last_change":"2024-09-11T22:20:18.193869+0000","last_active":"2024-09-11T22:22:15.977444+0000","last_peered":"2024-09-11T22:22:15.977444+0000","last_clean":"2024-09-11T22:22:15.977444+0000","last_became_active":"2024-09-11T22:20:18.193468+0000","last_became_peered":"2024-09-11T22:20:18.193468+0000","last_unstale":"2024-09-11T22:22:15.977444+0000","last_undegraded":"2024-09-11T22:22:15.977444+0000","last_fullsized":"2024-09-11T22:22:15.977444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:59.619611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234869+0000","last_change":"2024-09-11T22:20:18.193947+0000","last_active":"2024-09-11T22:22:16.234869+0000","last_peered":"2024-09-11T22:22:16.234869+0000","last_clean":"2024-09-11T22:22:16.234869+0000","last_became_active":"2024-09-11T22:20:18.192257+0000","last_became_peered":"2024-09-11T22:20:18.192257+0000","last_unstale":"2024-09-11T22:22:16.234869+0000","last_undegraded":"2024-09-11T22:22:16.234869+0000","last_fullsized":"2024-09-11T22:22:16.234869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.478730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058805+0000","last_change":"2024-09-11T22:20:18.184549+0000","last_active":"2024-09-11T22:22:16.058805+0000","last_peered":"2024-09-11T22:22:16.058805+0000","last_clean":"2024-09-11T22:22:16.058805+0000","last_became_active":"2024-09-11T22:20:18.184075+0000","last_became_peered":"2024-09-11T22:20:18.184075+0000","last_unstale":"2024-09-11T22:22:16.058805+0000","last_undegraded":"2024-09-11T22:22:16.058805+0000","last_fullsized":"2024-09-11T22:22:16.058805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:50.095643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'8","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.428358+0000","last_change":"2024-09-11T22:20:16.128524+0000","last_active":"2024-09-11T22:22:25.428358+0000","last_peered":"2024-09-11T22:22:25.428358+0000","last_clean":"2024-09-11T22:22:25.428358+0000","last_became_active":"2024-09-11T22:20:16.128185+0000","last_became_peered":"2024-09-11T22:20:16.128185+0000","last_unstale":"2024-09-11T22:22:25.428358+0000","last_undegraded":"2024-09-11T22:22:25.428358+0000","last_fullsized":"2024-09-11T22:22:25.428358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.022931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1716,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065720+0000","last_change":"2024-09-11T22:20:18.191578+0000","last_active":"2024-09-11T22:22:16.065720+0000","last_peered":"2024-09-11T22:22:16.065720+0000","last_clean":"2024-09-11T22:22:16.065720+0000","last_became_active":"2024-09-11T22:20:18.189568+0000","last_became_peered":"2024-09-11T22:20:18.189568+0000","last_unstale":"2024-09-11T22:22:16.065720+0000","last_undegraded":"2024-09-11T22:22:16.065720+0000","last_fullsized":"2024-09-11T22:22:16.065720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:41.737242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068742+0000","last_change":"2024-09-11T22:21:18.199693+0000","last_active":"2024-09-11T22:22:16.068742+0000","last_peered":"2024-09-11T22:22:16.068742+0000","last_clean":"2024-09-11T22:22:16.068742+0000","last_became_active":"2024-09-11T22:21:18.190954+0000","last_became_peered":"2024-09-11T22:21:18.190954+0000","last_unstale":"2024-09-11T22:22:16.068742+0000","last_undegraded":"2024-09-11T22:22:16.068742+0000","last_fullsized":"2024-09-11T22:22:16.068742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:11.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230688+0000","last_change":"2024-09-11T22:20:16.115602+0000","last_active":"2024-09-11T22:22:16.230688+0000","last_peered":"2024-09-11T22:22:16.230688+0000","last_clean":"2024-09-11T22:22:16.230688+0000","last_became_active":"2024-09-11T22:20:16.115280+0000","last_became_peered":"2024-09-11T22:20:16.115280+0000","last_unstale":"2024-09-11T22:22:16.230688+0000","last_undegraded":"2024-09-11T22:22:16.230688+0000","last_fullsized":"2024-09-11T22:22:16.230688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:41.942137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058816+0000","last_change":"2024-09-11T22:20:18.184125+0000","last_active":"2024-09-11T22:22:16.058816+0000","last_peered":"2024-09-11T22:22:16.058816+0000","last_clean":"2024-09-11T22:22:16.058816+0000","last_became_active":"2024-09-11T22:20:18.183785+0000","last_became_peered":"2024-09-11T22:20:18.183785+0000","last_unstale":"2024-09-11T22:22:16.058816+0000","last_undegraded":"2024-09-11T22:22:16.058816+0000","last_fullsized":"2024-09-11T22:22:16.058816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:33.230175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065780+0000","last_change":"2024-09-11T22:20:16.125852+0000","last_active":"2024-09-11T22:22:16.065780+0000","last_peered":"2024-09-11T22:22:16.065780+0000","last_clean":"2024-09-11T22:22:16.065780+0000","last_became_active":"2024-09-11T22:20:16.125250+0000","last_became_peered":"2024-09-11T22:20:16.125250+0000","last_unstale":"2024-09-11T22:22:16.065780+0000","last_undegraded":"2024-09-11T22:22:16.065780+0000","last_fullsized":"2024-09-11T22:22:16.065780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:34.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973235+0000","last_change":"2024-09-11T22:20:18.185147+0000","last_active":"2024-09-11T22:22:15.973235+0000","last_peered":"2024-09-11T22:22:15.973235+0000","last_clean":"2024-09-11T22:22:15.973235+0000","last_became_active":"2024-09-11T22:20:18.184840+0000","last_became_peered":"2024-09-11T22:20:18.184840+0000","last_unstale":"2024-09-11T22:22:15.973235+0000","last_undegraded":"2024-09-11T22:22:15.973235+0000","last_fullsized":"2024-09-11T22:22:15.973235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:59.703349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":263,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.322621+0000","last_change":"2024-09-11T22:20:16.122173+0000","last_active":"2024-09-11T22:22:26.322621+0000","last_peered":"2024-09-11T22:22:26.322621+0000","last_clean":"2024-09-11T22:22:26.322621+0000","last_became_active":"2024-09-11T22:20:16.121847+0000","last_became_peered":"2024-09-11T22:20:16.121847+0000","last_unstale":"2024-09-11T22:22:26.322621+0000","last_undegraded":"2024-09-11T22:22:26.322621+0000","last_fullsized":"2024-09-11T22:22:26.322621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:41.284557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":233,"num_read_kb":233,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975691+0000","last_change":"2024-09-11T22:20:18.188030+0000","last_active":"2024-09-11T22:22:15.975691+0000","last_peered":"2024-09-11T22:22:15.975691+0000","last_clean":"2024-09-11T22:22:15.975691+0000","last_became_active":"2024-09-11T22:20:18.187503+0000","last_became_peered":"2024-09-11T22:20:18.187503+0000","last_unstale":"2024-09-11T22:22:15.975691+0000","last_undegraded":"2024-09-11T22:22:15.975691+0000","last_fullsized":"2024-09-11T22:22:15.975691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:45.494149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231598+0000","last_change":"2024-09-11T22:20:16.128828+0000","last_active":"2024-09-11T22:22:16.231598+0000","last_peered":"2024-09-11T22:22:16.231598+0000","last_clean":"2024-09-11T22:22:16.231598+0000","last_became_active":"2024-09-11T22:20:16.128454+0000","last_became_peered":"2024-09-11T22:20:16.128454+0000","last_unstale":"2024-09-11T22:22:16.231598+0000","last_undegraded":"2024-09-11T22:22:16.231598+0000","last_fullsized":"2024-09-11T22:22:16.231598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:33.927564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234475+0000","last_change":"2024-09-11T22:20:18.193795+0000","last_active":"2024-09-11T22:22:16.234475+0000","last_peered":"2024-09-11T22:22:16.234475+0000","last_clean":"2024-09-11T22:22:16.234475+0000","last_became_active":"2024-09-11T22:20:18.193489+0000","last_became_peered":"2024-09-11T22:20:18.193489+0000","last_unstale":"2024-09-11T22:22:16.234475+0000","last_undegraded":"2024-09-11T22:22:16.234475+0000","last_fullsized":"2024-09-11T22:22:16.234475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:14.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065914+0000","last_change":"2024-09-11T22:20:16.187938+0000","last_active":"2024-09-11T22:22:16.065914+0000","last_peered":"2024-09-11T22:22:16.065914+0000","last_clean":"2024-09-11T22:22:16.065914+0000","last_became_active":"2024-09-11T22:20:16.187545+0000","last_became_peered":"2024-09-11T22:20:16.187545+0000","last_unstale":"2024-09-11T22:22:16.065914+0000","last_undegraded":"2024-09-11T22:22:16.065914+0000","last_fullsized":"2024-09-11T22:22:16.065914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:41.674356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975385+0000","last_change":"2024-09-11T22:21:18.191937+0000","last_active":"2024-09-11T22:22:15.975385+0000","last_peered":"2024-09-11T22:22:15.975385+0000","last_clean":"2024-09-11T22:22:15.975385+0000","last_became_active":"2024-09-11T22:21:18.191218+0000","last_became_peered":"2024-09-11T22:21:18.191218+0000","last_unstale":"2024-09-11T22:22:15.975385+0000","last_undegraded":"2024-09-11T22:22:15.975385+0000","last_fullsized":"2024-09-11T22:22:15.975385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:41.654630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":312,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.428665+0000","last_change":"2024-09-11T22:20:16.128082+0000","last_active":"2024-09-11T22:22:26.428665+0000","last_peered":"2024-09-11T22:22:26.428665+0000","last_clean":"2024-09-11T22:22:26.428665+0000","last_became_active":"2024-09-11T22:20:16.127731+0000","last_became_peered":"2024-09-11T22:20:16.127731+0000","last_unstale":"2024-09-11T22:22:26.428665+0000","last_undegraded":"2024-09-11T22:22:26.428665+0000","last_fullsized":"2024-09-11T22:22:26.428665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:33.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69393,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":193,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059064+0000","last_change":"2024-09-11T22:20:18.189219+0000","last_active":"2024-09-11T22:22:16.059064+0000","last_peered":"2024-09-11T22:22:16.059064+0000","last_clean":"2024-09-11T22:22:16.059064+0000","last_became_active":"2024-09-11T22:20:18.188687+0000","last_became_peered":"2024-09-11T22:20:18.188687+0000","last_unstale":"2024-09-11T22:22:16.059064+0000","last_undegraded":"2024-09-11T22:22:16.059064+0000","last_fullsized":"2024-09-11T22:22:16.059064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:24.171126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977378+0000","last_change":"2024-09-11T22:20:16.129724+0000","last_active":"2024-09-11T22:22:15.977378+0000","last_peered":"2024-09-11T22:22:15.977378+0000","last_clean":"2024-09-11T22:22:15.977378+0000","last_became_active":"2024-09-11T22:20:16.129351+0000","last_became_peered":"2024-09-11T22:20:16.129351+0000","last_unstale":"2024-09-11T22:22:15.977378+0000","last_undegraded":"2024-09-11T22:22:15.977378+0000","last_fullsized":"2024-09-11T22:22:15.977378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:09.642520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230435+0000","last_change":"2024-09-11T22:20:18.187031+0000","last_active":"2024-09-11T22:22:16.230435+0000","last_peered":"2024-09-11T22:22:16.230435+0000","last_clean":"2024-09-11T22:22:16.230435+0000","last_became_active":"2024-09-11T22:20:18.186698+0000","last_became_peered":"2024-09-11T22:20:18.186698+0000","last_unstale":"2024-09-11T22:22:16.230435+0000","last_undegraded":"2024-09-11T22:22:16.230435+0000","last_fullsized":"2024-09-11T22:22:16.230435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:04.810981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065725+0000","last_change":"2024-09-11T22:20:16.124925+0000","last_active":"2024-09-11T22:22:16.065725+0000","last_peered":"2024-09-11T22:22:16.065725+0000","last_clean":"2024-09-11T22:22:16.065725+0000","last_became_active":"2024-09-11T22:20:16.124580+0000","last_became_peered":"2024-09-11T22:20:16.124580+0000","last_unstale":"2024-09-11T22:22:16.065725+0000","last_undegraded":"2024-09-11T22:22:16.065725+0000","last_fullsized":"2024-09-11T22:22:16.065725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:56.417577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231838+0000","last_change":"2024-09-11T22:21:18.187783+0000","last_active":"2024-09-11T22:22:16.231838+0000","last_peered":"2024-09-11T22:22:16.231838+0000","last_clean":"2024-09-11T22:22:16.231838+0000","last_became_active":"2024-09-11T22:21:18.187358+0000","last_became_peered":"2024-09-11T22:21:18.187358+0000","last_unstale":"2024-09-11T22:22:16.231838+0000","last_undegraded":"2024-09-11T22:22:16.231838+0000","last_fullsized":"2024-09-11T22:22:16.231838+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:41.251531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.452982+0000","last_change":"2024-09-11T22:20:16.118934+0000","last_active":"2024-09-11T22:22:25.452982+0000","last_peered":"2024-09-11T22:22:25.452982+0000","last_clean":"2024-09-11T22:22:25.452982+0000","last_became_active":"2024-09-11T22:20:16.118614+0000","last_became_peered":"2024-09-11T22:20:16.118614+0000","last_unstale":"2024-09-11T22:22:25.452982+0000","last_undegraded":"2024-09-11T22:22:25.452982+0000","last_fullsized":"2024-09-11T22:22:25.452982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:05.791161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231721+0000","last_change":"2024-09-11T22:20:18.197692+0000","last_active":"2024-09-11T22:22:16.231721+0000","last_peered":"2024-09-11T22:22:16.231721+0000","last_clean":"2024-09-11T22:22:16.231721+0000","last_became_active":"2024-09-11T22:20:18.197336+0000","last_became_peered":"2024-09-11T22:20:18.197336+0000","last_unstale":"2024-09-11T22:22:16.231721+0000","last_undegraded":"2024-09-11T22:22:16.231721+0000","last_fullsized":"2024-09-11T22:22:16.231721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:55.172568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:16.186769+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:16.186367+0000","last_became_peered":"2024-09-11T22:20:16.186367+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.279387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065641+0000","last_change":"2024-09-11T22:20:18.195570+0000","last_active":"2024-09-11T22:22:16.065641+0000","last_peered":"2024-09-11T22:22:16.065641+0000","last_clean":"2024-09-11T22:22:16.065641+0000","last_became_active":"2024-09-11T22:20:18.195226+0000","last_became_peered":"2024-09-11T22:20:18.195226+0000","last_unstale":"2024-09-11T22:22:16.065641+0000","last_undegraded":"2024-09-11T22:22:16.065641+0000","last_fullsized":"2024-09-11T22:22:16.065641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:44.108090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235034+0000","last_change":"2024-09-11T22:21:18.194553+0000","last_active":"2024-09-11T22:22:16.235034+0000","last_peered":"2024-09-11T22:22:16.235034+0000","last_clean":"2024-09-11T22:22:16.235034+0000","last_became_active":"2024-09-11T22:21:18.188635+0000","last_became_peered":"2024-09-11T22:21:18.188635+0000","last_unstale":"2024-09-11T22:22:16.235034+0000","last_undegraded":"2024-09-11T22:22:16.235034+0000","last_fullsized":"2024-09-11T22:22:16.235034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:34.165195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234954+0000","last_change":"2024-09-11T22:20:18.194112+0000","last_active":"2024-09-11T22:22:16.234954+0000","last_peered":"2024-09-11T22:22:16.234954+0000","last_clean":"2024-09-11T22:22:16.234954+0000","last_became_active":"2024-09-11T22:20:18.193812+0000","last_became_peered":"2024-09-11T22:20:18.193812+0000","last_unstale":"2024-09-11T22:22:16.234954+0000","last_undegraded":"2024-09-11T22:22:16.234954+0000","last_fullsized":"2024-09-11T22:22:16.234954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:35.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231070+0000","last_change":"2024-09-11T22:21:18.198325+0000","last_active":"2024-09-11T22:22:16.231070+0000","last_peered":"2024-09-11T22:22:16.231070+0000","last_clean":"2024-09-11T22:22:16.231070+0000","last_became_active":"2024-09-11T22:21:18.193651+0000","last_became_peered":"2024-09-11T22:21:18.193651+0000","last_unstale":"2024-09-11T22:22:16.231070+0000","last_undegraded":"2024-09-11T22:22:16.231070+0000","last_fullsized":"2024-09-11T22:22:16.231070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:42.552492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232027+0000","last_change":"2024-09-11T22:20:18.191960+0000","last_active":"2024-09-11T22:22:16.232027+0000","last_peered":"2024-09-11T22:22:16.232027+0000","last_clean":"2024-09-11T22:22:16.232027+0000","last_became_active":"2024-09-11T22:20:18.191635+0000","last_became_peered":"2024-09-11T22:20:18.191635+0000","last_unstale":"2024-09-11T22:22:16.232027+0000","last_undegraded":"2024-09-11T22:22:16.232027+0000","last_fullsized":"2024-09-11T22:22:16.232027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:19:18.919247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.988172+0000","last_change":"2024-09-11T22:20:16.123608+0000","last_active":"2024-09-11T22:22:26.988172+0000","last_peered":"2024-09-11T22:22:26.988172+0000","last_clean":"2024-09-11T22:22:26.988172+0000","last_became_active":"2024-09-11T22:20:16.123193+0000","last_became_peered":"2024-09-11T22:20:16.123193+0000","last_unstale":"2024-09-11T22:22:26.988172+0000","last_undegraded":"2024-09-11T22:22:26.988172+0000","last_fullsized":"2024-09-11T22:22:26.988172+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:07.362115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":26,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231887+0000","last_change":"2024-09-11T22:20:18.199957+0000","last_active":"2024-09-11T22:22:16.231887+0000","last_peered":"2024-09-11T22:22:16.231887+0000","last_clean":"2024-09-11T22:22:16.231887+0000","last_became_active":"2024-09-11T22:20:18.199637+0000","last_became_peered":"2024-09-11T22:20:18.199637+0000","last_unstale":"2024-09-11T22:22:16.231887+0000","last_undegraded":"2024-09-11T22:22:16.231887+0000","last_fullsized":"2024-09-11T22:22:16.231887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:14.949002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":94,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.321741+0000","last_change":"2024-09-11T22:21:18.198896+0000","last_active":"2024-09-11T22:22:25.321741+0000","last_peered":"2024-09-11T22:22:25.321741+0000","last_clean":"2024-09-11T22:22:25.321741+0000","last_became_active":"2024-09-11T22:21:18.189489+0000","last_became_peered":"2024-09-11T22:21:18.189489+0000","last_unstale":"2024-09-11T22:22:25.321741+0000","last_undegraded":"2024-09-11T22:22:25.321741+0000","last_fullsized":"2024-09-11T22:22:25.321741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:19.908801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1956,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232111+0000","last_change":"2024-09-11T22:20:18.196691+0000","last_active":"2024-09-11T22:22:16.232111+0000","last_peered":"2024-09-11T22:22:16.232111+0000","last_clean":"2024-09-11T22:22:16.232111+0000","last_became_active":"2024-09-11T22:20:18.196364+0000","last_became_peered":"2024-09-11T22:20:18.196364+0000","last_unstale":"2024-09-11T22:22:16.232111+0000","last_undegraded":"2024-09-11T22:22:16.232111+0000","last_fullsized":"2024-09-11T22:22:16.232111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:50.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"83'148","reported_seq":392,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:18.074584+0000","last_change":"2024-09-11T22:19:02.590079+0000","last_active":"2024-09-11T22:22:18.074584+0000","last_peered":"2024-09-11T22:22:18.074584+0000","last_clean":"2024-09-11T22:22:18.074584+0000","last_became_active":"2024-09-11T22:19:02.589644+0000","last_became_peered":"2024-09-11T22:19:02.589644+0000","last_unstale":"2024-09-11T22:22:18.074584+0000","last_undegraded":"2024-09-11T22:22:18.074584+0000","last_fullsized":"2024-09-11T22:22:18.074584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_clean_scrub_stamp":"2024-09-11T22:15:33.244685+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:45.506190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.229817+0000","last_change":"2024-09-11T22:20:16.116661+0000","last_active":"2024-09-11T22:22:16.229817+0000","last_peered":"2024-09-11T22:22:16.229817+0000","last_clean":"2024-09-11T22:22:16.229817+0000","last_became_active":"2024-09-11T22:20:16.116196+0000","last_became_peered":"2024-09-11T22:20:16.116196+0000","last_unstale":"2024-09-11T22:22:16.229817+0000","last_undegraded":"2024-09-11T22:22:16.229817+0000","last_fullsized":"2024-09-11T22:22:16.229817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:08.875170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059213+0000","last_change":"2024-09-11T22:20:18.193176+0000","last_active":"2024-09-11T22:22:16.059213+0000","last_peered":"2024-09-11T22:22:16.059213+0000","last_clean":"2024-09-11T22:22:16.059213+0000","last_became_active":"2024-09-11T22:20:18.192312+0000","last_became_peered":"2024-09-11T22:20:18.192312+0000","last_unstale":"2024-09-11T22:22:16.059213+0000","last_undegraded":"2024-09-11T22:22:16.059213+0000","last_fullsized":"2024-09-11T22:22:16.059213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:08.738569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234694+0000","last_change":"2024-09-11T22:21:18.196985+0000","last_active":"2024-09-11T22:22:16.234694+0000","last_peered":"2024-09-11T22:22:16.234694+0000","last_clean":"2024-09-11T22:22:16.234694+0000","last_became_active":"2024-09-11T22:21:18.189487+0000","last_became_peered":"2024-09-11T22:21:18.189487+0000","last_unstale":"2024-09-11T22:22:16.234694+0000","last_undegraded":"2024-09-11T22:22:16.234694+0000","last_fullsized":"2024-09-11T22:22:16.234694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:20.715696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230517+0000","last_change":"2024-09-11T22:20:18.191118+0000","last_active":"2024-09-11T22:22:16.230517+0000","last_peered":"2024-09-11T22:22:16.230517+0000","last_clean":"2024-09-11T22:22:16.230517+0000","last_became_active":"2024-09-11T22:20:18.190760+0000","last_became_peered":"2024-09-11T22:20:18.190760+0000","last_unstale":"2024-09-11T22:22:16.230517+0000","last_undegraded":"2024-09-11T22:22:16.230517+0000","last_fullsized":"2024-09-11T22:22:16.230517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:59.220067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.452301+0000","last_change":"2024-09-11T22:20:16.186407+0000","last_active":"2024-09-11T22:22:25.452301+0000","last_peered":"2024-09-11T22:22:25.452301+0000","last_clean":"2024-09-11T22:22:25.452301+0000","last_became_active":"2024-09-11T22:20:16.186105+0000","last_became_peered":"2024-09-11T22:20:16.186105+0000","last_unstale":"2024-09-11T22:22:25.452301+0000","last_undegraded":"2024-09-11T22:22:25.452301+0000","last_fullsized":"2024-09-11T22:22:25.452301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:43.942072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065681+0000","last_change":"2024-09-11T22:20:18.191970+0000","last_active":"2024-09-11T22:22:16.065681+0000","last_peered":"2024-09-11T22:22:16.065681+0000","last_clean":"2024-09-11T22:22:16.065681+0000","last_became_active":"2024-09-11T22:20:18.191623+0000","last_became_peered":"2024-09-11T22:20:18.191623+0000","last_unstale":"2024-09-11T22:22:16.065681+0000","last_undegraded":"2024-09-11T22:22:16.065681+0000","last_fullsized":"2024-09-11T22:22:16.065681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:33.213979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.441040+0000","last_change":"2024-09-11T22:20:16.119936+0000","last_active":"2024-09-11T22:22:25.441040+0000","last_peered":"2024-09-11T22:22:25.441040+0000","last_clean":"2024-09-11T22:22:25.441040+0000","last_became_active":"2024-09-11T22:20:16.119453+0000","last_became_peered":"2024-09-11T22:20:16.119453+0000","last_unstale":"2024-09-11T22:22:25.441040+0000","last_undegraded":"2024-09-11T22:22:25.441040+0000","last_fullsized":"2024-09-11T22:22:25.441040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:20.717980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068855+0000","last_change":"2024-09-11T22:20:18.192581+0000","last_active":"2024-09-11T22:22:16.068855+0000","last_peered":"2024-09-11T22:22:16.068855+0000","last_clean":"2024-09-11T22:22:16.068855+0000","last_became_active":"2024-09-11T22:20:18.191702+0000","last_became_peered":"2024-09-11T22:20:18.191702+0000","last_unstale":"2024-09-11T22:22:16.068855+0000","last_undegraded":"2024-09-11T22:22:16.068855+0000","last_fullsized":"2024-09-11T22:22:16.068855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:48.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973445+0000","last_change":"2024-09-11T22:20:16.186485+0000","last_active":"2024-09-11T22:22:15.973445+0000","last_peered":"2024-09-11T22:22:15.973445+0000","last_clean":"2024-09-11T22:22:15.973445+0000","last_became_active":"2024-09-11T22:20:16.186171+0000","last_became_peered":"2024-09-11T22:20:16.186171+0000","last_unstale":"2024-09-11T22:22:15.973445+0000","last_undegraded":"2024-09-11T22:22:15.973445+0000","last_fullsized":"2024-09-11T22:22:15.973445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:26.373626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068932+0000","last_change":"2024-09-11T22:20:18.191361+0000","last_active":"2024-09-11T22:22:16.068932+0000","last_peered":"2024-09-11T22:22:16.068932+0000","last_clean":"2024-09-11T22:22:16.068932+0000","last_became_active":"2024-09-11T22:20:18.191036+0000","last_became_peered":"2024-09-11T22:20:18.191036+0000","last_unstale":"2024-09-11T22:22:16.068932+0000","last_undegraded":"2024-09-11T22:22:16.068932+0000","last_fullsized":"2024-09-11T22:22:16.068932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:29.124650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059209+0000","last_change":"2024-09-11T22:20:16.188156+0000","last_active":"2024-09-11T22:22:16.059209+0000","last_peered":"2024-09-11T22:22:16.059209+0000","last_clean":"2024-09-11T22:22:16.059209+0000","last_became_active":"2024-09-11T22:20:16.187809+0000","last_became_peered":"2024-09-11T22:20:16.187809+0000","last_unstale":"2024-09-11T22:22:16.059209+0000","last_undegraded":"2024-09-11T22:22:16.059209+0000","last_fullsized":"2024-09-11T22:22:16.059209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:04.219621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230944+0000","last_change":"2024-09-11T22:20:18.182380+0000","last_active":"2024-09-11T22:22:16.230944+0000","last_peered":"2024-09-11T22:22:16.230944+0000","last_clean":"2024-09-11T22:22:16.230944+0000","last_became_active":"2024-09-11T22:20:18.182062+0000","last_became_peered":"2024-09-11T22:20:18.182062+0000","last_unstale":"2024-09-11T22:22:16.230944+0000","last_undegraded":"2024-09-11T22:22:16.230944+0000","last_fullsized":"2024-09-11T22:22:16.230944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:15.575846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'91","reported_seq":187,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.997407+0000","last_change":"2024-09-11T22:20:16.117335+0000","last_active":"2024-09-11T22:22:26.997407+0000","last_peered":"2024-09-11T22:22:26.997407+0000","last_clean":"2024-09-11T22:22:26.997407+0000","last_became_active":"2024-09-11T22:20:16.116977+0000","last_became_peered":"2024-09-11T22:20:16.116977+0000","last_unstale":"2024-09-11T22:22:26.997407+0000","last_undegraded":"2024-09-11T22:22:26.997407+0000","last_fullsized":"2024-09-11T22:22:26.997407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:51.744202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":244,"num_read_kb":215,"num_write":164,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059281+0000","last_change":"2024-09-11T22:20:18.194069+0000","last_active":"2024-09-11T22:22:16.059281+0000","last_peered":"2024-09-11T22:22:16.059281+0000","last_clean":"2024-09-11T22:22:16.059281+0000","last_became_active":"2024-09-11T22:20:18.193765+0000","last_became_peered":"2024-09-11T22:20:18.193765+0000","last_unstale":"2024-09-11T22:22:16.059281+0000","last_undegraded":"2024-09-11T22:22:16.059281+0000","last_fullsized":"2024-09-11T22:22:16.059281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:25.047405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230178+0000","last_change":"2024-09-11T22:20:16.460987+0000","last_active":"2024-09-11T22:22:16.230178+0000","last_peered":"2024-09-11T22:22:16.230178+0000","last_clean":"2024-09-11T22:22:16.230178+0000","last_became_active":"2024-09-11T22:20:16.460516+0000","last_became_peered":"2024-09-11T22:20:16.460516+0000","last_unstale":"2024-09-11T22:22:16.230178+0000","last_undegraded":"2024-09-11T22:22:16.230178+0000","last_fullsized":"2024-09-11T22:22:16.230178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:00.987981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234312+0000","last_change":"2024-09-11T22:21:18.188802+0000","last_active":"2024-09-11T22:22:16.234312+0000","last_peered":"2024-09-11T22:22:16.234312+0000","last_clean":"2024-09-11T22:22:16.234312+0000","last_became_active":"2024-09-11T22:21:18.188441+0000","last_became_peered":"2024-09-11T22:21:18.188441+0000","last_unstale":"2024-09-11T22:22:16.234312+0000","last_undegraded":"2024-09-11T22:22:16.234312+0000","last_fullsized":"2024-09-11T22:22:16.234312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:02.295441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975385+0000","last_change":"2024-09-11T22:20:16.186498+0000","last_active":"2024-09-11T22:22:15.975385+0000","last_peered":"2024-09-11T22:22:15.975385+0000","last_clean":"2024-09-11T22:22:15.975385+0000","last_became_active":"2024-09-11T22:20:16.186214+0000","last_became_peered":"2024-09-11T22:20:16.186214+0000","last_unstale":"2024-09-11T22:22:15.975385+0000","last_undegraded":"2024-09-11T22:22:15.975385+0000","last_fullsized":"2024-09-11T22:22:15.975385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:09.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:18.187489+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:18.185649+0000","last_became_peered":"2024-09-11T22:20:18.185649+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:42.825075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059064+0000","last_change":"2024-09-11T22:20:16.117627+0000","last_active":"2024-09-11T22:22:16.059064+0000","last_peered":"2024-09-11T22:22:16.059064+0000","last_clean":"2024-09-11T22:22:16.059064+0000","last_became_active":"2024-09-11T22:20:16.117243+0000","last_became_peered":"2024-09-11T22:20:16.117243+0000","last_unstale":"2024-09-11T22:22:16.059064+0000","last_undegraded":"2024-09-11T22:22:16.059064+0000","last_fullsized":"2024-09-11T22:22:16.059064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:58.240535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234846+0000","last_change":"2024-09-11T22:20:18.184326+0000","last_active":"2024-09-11T22:22:16.234846+0000","last_peered":"2024-09-11T22:22:16.234846+0000","last_clean":"2024-09-11T22:22:16.234846+0000","last_became_active":"2024-09-11T22:20:18.183995+0000","last_became_peered":"2024-09-11T22:20:18.183995+0000","last_unstale":"2024-09-11T22:22:16.234846+0000","last_undegraded":"2024-09-11T22:22:16.234846+0000","last_fullsized":"2024-09-11T22:22:16.234846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:33.468000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977314+0000","last_change":"2024-09-11T22:20:16.127180+0000","last_active":"2024-09-11T22:22:15.977314+0000","last_peered":"2024-09-11T22:22:15.977314+0000","last_clean":"2024-09-11T22:22:15.977314+0000","last_became_active":"2024-09-11T22:20:16.126696+0000","last_became_peered":"2024-09-11T22:20:16.126696+0000","last_unstale":"2024-09-11T22:22:15.977314+0000","last_undegraded":"2024-09-11T22:22:15.977314+0000","last_fullsized":"2024-09-11T22:22:15.977314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:02.637225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973369+0000","last_change":"2024-09-11T22:21:18.188614+0000","last_active":"2024-09-11T22:22:15.973369+0000","last_peered":"2024-09-11T22:22:15.973369+0000","last_clean":"2024-09-11T22:22:15.973369+0000","last_became_active":"2024-09-11T22:21:18.188277+0000","last_became_peered":"2024-09-11T22:21:18.188277+0000","last_unstale":"2024-09-11T22:22:15.973369+0000","last_undegraded":"2024-09-11T22:22:15.973369+0000","last_fullsized":"2024-09-11T22:22:15.973369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:59.995647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058897+0000","last_change":"2024-09-11T22:20:16.128311+0000","last_active":"2024-09-11T22:22:16.058897+0000","last_peered":"2024-09-11T22:22:16.058897+0000","last_clean":"2024-09-11T22:22:16.058897+0000","last_became_active":"2024-09-11T22:20:16.127853+0000","last_became_peered":"2024-09-11T22:20:16.127853+0000","last_unstale":"2024-09-11T22:22:16.058897+0000","last_undegraded":"2024-09-11T22:22:16.058897+0000","last_fullsized":"2024-09-11T22:22:16.058897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:57.562470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231820+0000","last_change":"2024-09-11T22:20:18.195782+0000","last_active":"2024-09-11T22:22:16.231820+0000","last_peered":"2024-09-11T22:22:16.231820+0000","last_clean":"2024-09-11T22:22:16.231820+0000","last_became_active":"2024-09-11T22:20:18.195365+0000","last_became_peered":"2024-09-11T22:20:18.195365+0000","last_unstale":"2024-09-11T22:22:16.231820+0000","last_undegraded":"2024-09-11T22:22:16.231820+0000","last_fullsized":"2024-09-11T22:22:16.231820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:49.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065812+0000","last_change":"2024-09-11T22:20:18.199344+0000","last_active":"2024-09-11T22:22:16.065812+0000","last_peered":"2024-09-11T22:22:16.065812+0000","last_clean":"2024-09-11T22:22:16.065812+0000","last_became_active":"2024-09-11T22:20:18.199016+0000","last_became_peered":"2024-09-11T22:20:18.199016+0000","last_unstale":"2024-09-11T22:22:16.065812+0000","last_undegraded":"2024-09-11T22:22:16.065812+0000","last_fullsized":"2024-09-11T22:22:16.065812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:38.322831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059125+0000","last_change":"2024-09-11T22:20:16.122236+0000","last_active":"2024-09-11T22:22:16.059125+0000","last_peered":"2024-09-11T22:22:16.059125+0000","last_clean":"2024-09-11T22:22:16.059125+0000","last_became_active":"2024-09-11T22:20:16.121867+0000","last_became_peered":"2024-09-11T22:20:16.121867+0000","last_unstale":"2024-09-11T22:22:16.059125+0000","last_undegraded":"2024-09-11T22:22:16.059125+0000","last_fullsized":"2024-09-11T22:22:16.059125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:56.782445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975374+0000","last_change":"2024-09-11T22:20:18.187338+0000","last_active":"2024-09-11T22:22:15.975374+0000","last_peered":"2024-09-11T22:22:15.975374+0000","last_clean":"2024-09-11T22:22:15.975374+0000","last_became_active":"2024-09-11T22:20:18.186985+0000","last_became_peered":"2024-09-11T22:20:18.186985+0000","last_unstale":"2024-09-11T22:22:15.975374+0000","last_undegraded":"2024-09-11T22:22:15.975374+0000","last_fullsized":"2024-09-11T22:22:15.975374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:58.170680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059106+0000","last_change":"2024-09-11T22:20:16.128579+0000","last_active":"2024-09-11T22:22:16.059106+0000","last_peered":"2024-09-11T22:22:16.059106+0000","last_clean":"2024-09-11T22:22:16.059106+0000","last_became_active":"2024-09-11T22:20:16.127864+0000","last_became_peered":"2024-09-11T22:20:16.127864+0000","last_unstale":"2024-09-11T22:22:16.059106+0000","last_undegraded":"2024-09-11T22:22:16.059106+0000","last_fullsized":"2024-09-11T22:22:16.059106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:11:32.005487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234497+0000","last_change":"2024-09-11T22:20:18.185205+0000","last_active":"2024-09-11T22:22:16.234497+0000","last_peered":"2024-09-11T22:22:16.234497+0000","last_clean":"2024-09-11T22:22:16.234497+0000","last_became_active":"2024-09-11T22:20:18.184402+0000","last_became_peered":"2024-09-11T22:20:18.184402+0000","last_unstale":"2024-09-11T22:22:16.234497+0000","last_undegraded":"2024-09-11T22:22:16.234497+0000","last_fullsized":"2024-09-11T22:22:16.234497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:17:48.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.060250+0000","last_change":"2024-09-11T22:20:16.186372+0000","last_active":"2024-09-11T22:22:16.060250+0000","last_peered":"2024-09-11T22:22:16.060250+0000","last_clean":"2024-09-11T22:22:16.060250+0000","last_became_active":"2024-09-11T22:20:16.185990+0000","last_became_peered":"2024-09-11T22:20:16.185990+0000","last_unstale":"2024-09-11T22:22:16.060250+0000","last_undegraded":"2024-09-11T22:22:16.060250+0000","last_fullsized":"2024-09-11T22:22:16.060250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:23.296379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.060015+0000","last_change":"2024-09-11T22:20:18.190670+0000","last_active":"2024-09-11T22:22:16.060015+0000","last_peered":"2024-09-11T22:22:16.060015+0000","last_clean":"2024-09-11T22:22:16.060015+0000","last_became_active":"2024-09-11T22:20:18.190339+0000","last_became_peered":"2024-09-11T22:20:18.190339+0000","last_unstale":"2024-09-11T22:22:16.060015+0000","last_undegraded":"2024-09-11T22:22:16.060015+0000","last_fullsized":"2024-09-11T22:22:16.060015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.085305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.928844+0000","last_change":"2024-09-11T22:20:16.117646+0000","last_active":"2024-09-11T22:22:25.928844+0000","last_peered":"2024-09-11T22:22:25.928844+0000","last_clean":"2024-09-11T22:22:25.928844+0000","last_became_active":"2024-09-11T22:20:16.117111+0000","last_became_peered":"2024-09-11T22:20:16.117111+0000","last_unstale":"2024-09-11T22:22:25.928844+0000","last_undegraded":"2024-09-11T22:22:25.928844+0000","last_fullsized":"2024-09-11T22:22:25.928844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:36.470158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800463,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1442,"num_read_kb":936,"num_write":480,"num_write_kb":2385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1192,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2401389,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108858,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579943,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":444133,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":1236133,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":836471,"data_compressed":27823,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201334,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":836827,"data_compressed":27952,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430268,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":1038174,"data_compressed":44963,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593792,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443506,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822726,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":1168132,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":445575,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215184,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":967613,"data_compressed":33979,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":513422,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":987136,"data_stored":513812,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":937984,"data_stored":445097,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332146,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":400147,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:22:29.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:22:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:30.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:29 smithi005 ceph-mon[30943]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 84 KiB/s wr, 46 op/s 2024-09-11T22:22:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:29 smithi159 ceph-mon[35534]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 84 KiB/s wr, 46 op/s 2024-09-11T22:22:30.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:29 smithi172 ceph-mon[35461]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 84 KiB/s wr, 46 op/s 2024-09-11T22:22:30.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.543+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3892667533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078102570 msgr2=0x7fe0781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.543+0000 7fe07cb2e640 1 --2- 172.21.15.5:0/3892667533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078102570 0x7fe0781029b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe060009900 tx=0x7fe06002a480 comp rx=0 tx=0).stop 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.543+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3892667533 shutdown_connections 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.543+0000 7fe07cb2e640 1 --2- 172.21.15.5:0/3892667533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe078102ef0 0x7fe07810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.543+0000 7fe07cb2e640 1 --2- 172.21.15.5:0/3892667533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078102570 0x7fe0781029b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.543+0000 7fe07cb2e640 1 --2- 172.21.15.5:0/3892667533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe078100e20 0x7fe078101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.543+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3892667533 >> 172.21.15.5:0/3892667533 conn(0x7fe0780fc7d0 msgr2=0x7fe0780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.543+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3892667533 shutdown_connections 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.543+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3892667533 wait complete. 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe07cb2e640 1 Processor -- start 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe07cb2e640 1 -- start start 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe07cb2e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe078100e20 0x7fe0781984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:30.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe07cb2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078102570 0x7fe078198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe07cb2e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe078102ef0 0x7fe07819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe07cb2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe078112060 con 0x7fe078102570 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe07cb2e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe078111ee0 con 0x7fe078100e20 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe07cb2e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781121e0 con 0x7fe078102ef0 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe075d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078102570 0x7fe078198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe075d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078102570 0x7fe078198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57666/0 (socket says 172.21.15.5:57666) 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe075d9b640 1 -- 172.21.15.5:0/3061812529 learned_addr learned my addr 172.21.15.5:0/3061812529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.544+0000 7fe07659c640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe078100e20 0x7fe0781984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe076d9d640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe078102ef0 0x7fe07819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe075d9b640 1 -- 172.21.15.5:0/3061812529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe078102ef0 msgr2=0x7fe07819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe075d9b640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe078102ef0 0x7fe07819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe075d9b640 1 -- 172.21.15.5:0/3061812529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe078100e20 msgr2=0x7fe0781984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe075d9b640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe078100e20 0x7fe0781984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe075d9b640 1 -- 172.21.15.5:0/3061812529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07819d470 con 0x7fe078102570 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe07659c640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe078100e20 0x7fe0781984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe075d9b640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078102570 0x7fe078198a00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe060006010 tx=0x7fe060003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe06f7fe640 1 -- 172.21.15.5:0/3061812529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06003f070 con 0x7fe078102570 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3061812529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07819d700 con 0x7fe078102570 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe06f7fe640 1 -- 172.21.15.5:0/3061812529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe060031c30 con 0x7fe078102570 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe06f7fe640 1 -- 172.21.15.5:0/3061812529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060035030 con 0x7fe078102570 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.545+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3061812529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0781a5040 con 0x7fe078102570 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.546+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3061812529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe078101200 con 0x7fe078102570 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.548+0000 7fe06f7fe640 1 -- 172.21.15.5:0/3061812529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7fe060042070 con 0x7fe078102570 2024-09-11T22:22:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.549+0000 7fe06f7fe640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0640a2e10 0x7fe0640a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.549+0000 7fe07659c640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0640a2e10 0x7fe0640a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.549+0000 7fe06f7fe640 1 -- 172.21.15.5:0/3061812529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fe0600eff50 con 0x7fe078102570 2024-09-11T22:22:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.549+0000 7fe07659c640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0640a2e10 0x7fe0640a52d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe068005fd0 tx=0x7fe068009600 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.553+0000 7fe06f7fe640 1 -- 172.21.15.5:0/3061812529 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe060031de0 con 0x7fe078102570 2024-09-11T22:22:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.726+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3061812529 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe078106fb0 con 0x7fe0640a2e10 2024-09-11T22:22:30.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.732+0000 7fe06f7fe640 1 -- 172.21.15.5:0/3061812529 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258085 (secure 0 0 0) 0x7fe078106fb0 con 0x7fe0640a2e10 2024-09-11T22:22:30.735 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:30.736 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-11T22:22:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3061812529 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0640a2e10 msgr2=0x7fe0640a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0640a2e10 0x7fe0640a52d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe068005fd0 tx=0x7fe068009600 comp rx=0 tx=0).stop 2024-09-11T22:22:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3061812529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078102570 msgr2=0x7fe078198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078102570 0x7fe078198a00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe060006010 tx=0x7fe060003c10 comp rx=0 tx=0).stop 2024-09-11T22:22:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3061812529 shutdown_connections 2024-09-11T22:22:30.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe078102ef0 0x7fe07819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078102570 0x7fe078198a00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0640a2e10 0x7fe0640a52d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 --2- 172.21.15.5:0/3061812529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe078100e20 0x7fe0781984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3061812529 >> 172.21.15.5:0/3061812529 conn(0x7fe0780fc7d0 msgr2=0x7fe0780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3061812529 shutdown_connections 2024-09-11T22:22:30.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:30.735+0000 7fe07cb2e640 1 -- 172.21.15.5:0/3061812529 wait complete. 2024-09-11T22:22:31.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:30 smithi005 ceph-mon[30943]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:30 smithi159 ceph-mon[35534]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:30 smithi172 ceph-mon[35461]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:31.473 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-11T22:22:29.989184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391065,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1679,"num_read_kb":1266,"num_write":683,"num_write_kb":5325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181928,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1021,"ondisk_log_size":1021,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385620,"kb_used_data":11388,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556204,"statfs":{"total":1151940427776,"available":1151545552896,"internally_reserved":0,"allocated":11661312,"data_stored":8888895,"data_compressed":289017,"data_compressed_allocated":380928,"data_compressed_original":3612672,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200331,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":385,"num_read_kb":430,"num_write":128,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002519"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235096+0000","last_change":"2024-09-11T22:20:18.189744+0000","last_active":"2024-09-11T22:22:16.235096+0000","last_peered":"2024-09-11T22:22:16.235096+0000","last_clean":"2024-09-11T22:22:16.235096+0000","last_became_active":"2024-09-11T22:20:18.189398+0000","last_became_peered":"2024-09-11T22:20:18.189398+0000","last_unstale":"2024-09-11T22:22:16.235096+0000","last_undegraded":"2024-09-11T22:22:16.235096+0000","last_fullsized":"2024-09-11T22:22:16.235096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:24.152589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234185+0000","last_change":"2024-09-11T22:20:18.186727+0000","last_active":"2024-09-11T22:22:16.234185+0000","last_peered":"2024-09-11T22:22:16.234185+0000","last_clean":"2024-09-11T22:22:16.234185+0000","last_became_active":"2024-09-11T22:20:18.186414+0000","last_became_peered":"2024-09-11T22:20:18.186414+0000","last_unstale":"2024-09-11T22:22:16.234185+0000","last_undegraded":"2024-09-11T22:22:16.234185+0000","last_fullsized":"2024-09-11T22:22:16.234185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:53.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233909+0000","last_change":"2024-09-11T22:20:18.179984+0000","last_active":"2024-09-11T22:22:16.233909+0000","last_peered":"2024-09-11T22:22:16.233909+0000","last_clean":"2024-09-11T22:22:16.233909+0000","last_became_active":"2024-09-11T22:20:18.179329+0000","last_became_peered":"2024-09-11T22:20:18.179329+0000","last_unstale":"2024-09-11T22:22:16.233909+0000","last_undegraded":"2024-09-11T22:22:16.233909+0000","last_fullsized":"2024-09-11T22:22:16.233909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:19.149970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230253+0000","last_change":"2024-09-11T22:20:18.191768+0000","last_active":"2024-09-11T22:22:16.230253+0000","last_peered":"2024-09-11T22:22:16.230253+0000","last_clean":"2024-09-11T22:22:16.230253+0000","last_became_active":"2024-09-11T22:20:18.191336+0000","last_became_peered":"2024-09-11T22:20:18.191336+0000","last_unstale":"2024-09-11T22:22:16.230253+0000","last_undegraded":"2024-09-11T22:22:16.230253+0000","last_fullsized":"2024-09-11T22:22:16.230253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:44.768932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058858+0000","last_change":"2024-09-11T22:20:18.181469+0000","last_active":"2024-09-11T22:22:16.058858+0000","last_peered":"2024-09-11T22:22:16.058858+0000","last_clean":"2024-09-11T22:22:16.058858+0000","last_became_active":"2024-09-11T22:20:18.181100+0000","last_became_peered":"2024-09-11T22:20:18.181100+0000","last_unstale":"2024-09-11T22:22:16.058858+0000","last_undegraded":"2024-09-11T22:22:16.058858+0000","last_fullsized":"2024-09-11T22:22:16.058858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:14.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068675+0000","last_change":"2024-09-11T22:20:18.190855+0000","last_active":"2024-09-11T22:22:16.068675+0000","last_peered":"2024-09-11T22:22:16.068675+0000","last_clean":"2024-09-11T22:22:16.068675+0000","last_became_active":"2024-09-11T22:20:18.190523+0000","last_became_peered":"2024-09-11T22:20:18.190523+0000","last_unstale":"2024-09-11T22:22:16.068675+0000","last_undegraded":"2024-09-11T22:22:16.068675+0000","last_fullsized":"2024-09-11T22:22:16.068675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:18.574957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231733+0000","last_change":"2024-09-11T22:20:18.199213+0000","last_active":"2024-09-11T22:22:16.231733+0000","last_peered":"2024-09-11T22:22:16.231733+0000","last_clean":"2024-09-11T22:22:16.231733+0000","last_became_active":"2024-09-11T22:20:18.198913+0000","last_became_peered":"2024-09-11T22:20:18.198913+0000","last_unstale":"2024-09-11T22:22:16.231733+0000","last_undegraded":"2024-09-11T22:22:16.231733+0000","last_fullsized":"2024-09-11T22:22:16.231733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:22.682802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058633+0000","last_change":"2024-09-11T22:20:18.192458+0000","last_active":"2024-09-11T22:22:16.058633+0000","last_peered":"2024-09-11T22:22:16.058633+0000","last_clean":"2024-09-11T22:22:16.058633+0000","last_became_active":"2024-09-11T22:20:18.191967+0000","last_became_peered":"2024-09-11T22:20:18.191967+0000","last_unstale":"2024-09-11T22:22:16.058633+0000","last_undegraded":"2024-09-11T22:22:16.058633+0000","last_fullsized":"2024-09-11T22:22:16.058633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:52.686157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975713+0000","last_change":"2024-09-11T22:21:18.191732+0000","last_active":"2024-09-11T22:22:15.975713+0000","last_peered":"2024-09-11T22:22:15.975713+0000","last_clean":"2024-09-11T22:22:15.975713+0000","last_became_active":"2024-09-11T22:21:18.191375+0000","last_became_peered":"2024-09-11T22:21:18.191375+0000","last_unstale":"2024-09-11T22:22:15.975713+0000","last_undegraded":"2024-09-11T22:22:15.975713+0000","last_fullsized":"2024-09-11T22:22:15.975713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:13.645224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.069211+0000","last_change":"2024-09-11T22:20:18.193771+0000","last_active":"2024-09-11T22:22:16.069211+0000","last_peered":"2024-09-11T22:22:16.069211+0000","last_clean":"2024-09-11T22:22:16.069211+0000","last_became_active":"2024-09-11T22:20:18.193041+0000","last_became_peered":"2024-09-11T22:20:18.193041+0000","last_unstale":"2024-09-11T22:22:16.069211+0000","last_undegraded":"2024-09-11T22:22:16.069211+0000","last_fullsized":"2024-09-11T22:22:16.069211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:06.656169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231998+0000","last_change":"2024-09-11T22:20:18.197933+0000","last_active":"2024-09-11T22:22:16.231998+0000","last_peered":"2024-09-11T22:22:16.231998+0000","last_clean":"2024-09-11T22:22:16.231998+0000","last_became_active":"2024-09-11T22:20:18.197210+0000","last_became_peered":"2024-09-11T22:20:18.197210+0000","last_unstale":"2024-09-11T22:22:16.231998+0000","last_undegraded":"2024-09-11T22:22:16.231998+0000","last_fullsized":"2024-09-11T22:22:16.231998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:42.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230250+0000","last_change":"2024-09-11T22:20:18.190739+0000","last_active":"2024-09-11T22:22:16.230250+0000","last_peered":"2024-09-11T22:22:16.230250+0000","last_clean":"2024-09-11T22:22:16.230250+0000","last_became_active":"2024-09-11T22:20:18.190122+0000","last_became_peered":"2024-09-11T22:20:18.190122+0000","last_unstale":"2024-09-11T22:22:16.230250+0000","last_undegraded":"2024-09-11T22:22:16.230250+0000","last_fullsized":"2024-09-11T22:22:16.230250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.585035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230697+0000","last_change":"2024-09-11T22:20:18.190303+0000","last_active":"2024-09-11T22:22:16.230697+0000","last_peered":"2024-09-11T22:22:16.230697+0000","last_clean":"2024-09-11T22:22:16.230697+0000","last_became_active":"2024-09-11T22:20:18.189962+0000","last_became_peered":"2024-09-11T22:20:18.189962+0000","last_unstale":"2024-09-11T22:22:16.230697+0000","last_undegraded":"2024-09-11T22:22:16.230697+0000","last_fullsized":"2024-09-11T22:22:16.230697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:44.872813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233520+0000","last_change":"2024-09-11T22:20:18.185893+0000","last_active":"2024-09-11T22:22:16.233520+0000","last_peered":"2024-09-11T22:22:16.233520+0000","last_clean":"2024-09-11T22:22:16.233520+0000","last_became_active":"2024-09-11T22:20:18.185592+0000","last_became_peered":"2024-09-11T22:20:18.185592+0000","last_unstale":"2024-09-11T22:22:16.233520+0000","last_undegraded":"2024-09-11T22:22:16.233520+0000","last_fullsized":"2024-09-11T22:22:16.233520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:14.725301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233777+0000","last_change":"2024-09-11T22:20:18.193943+0000","last_active":"2024-09-11T22:22:16.233777+0000","last_peered":"2024-09-11T22:22:16.233777+0000","last_clean":"2024-09-11T22:22:16.233777+0000","last_became_active":"2024-09-11T22:20:18.193558+0000","last_became_peered":"2024-09-11T22:20:18.193558+0000","last_unstale":"2024-09-11T22:22:16.233777+0000","last_undegraded":"2024-09-11T22:22:16.233777+0000","last_fullsized":"2024-09-11T22:22:16.233777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:08.736161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975254+0000","last_change":"2024-09-11T22:20:18.191815+0000","last_active":"2024-09-11T22:22:15.975254+0000","last_peered":"2024-09-11T22:22:15.975254+0000","last_clean":"2024-09-11T22:22:15.975254+0000","last_became_active":"2024-09-11T22:20:18.191483+0000","last_became_peered":"2024-09-11T22:20:18.191483+0000","last_unstale":"2024-09-11T22:22:15.975254+0000","last_undegraded":"2024-09-11T22:22:15.975254+0000","last_fullsized":"2024-09-11T22:22:15.975254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:26.432117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235101+0000","last_change":"2024-09-11T22:20:18.190758+0000","last_active":"2024-09-11T22:22:16.235101+0000","last_peered":"2024-09-11T22:22:16.235101+0000","last_clean":"2024-09-11T22:22:16.235101+0000","last_became_active":"2024-09-11T22:20:18.190431+0000","last_became_peered":"2024-09-11T22:20:18.190431+0000","last_unstale":"2024-09-11T22:22:16.235101+0000","last_undegraded":"2024-09-11T22:22:16.235101+0000","last_fullsized":"2024-09-11T22:22:16.235101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:10.927051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973552+0000","last_change":"2024-09-11T22:20:18.185191+0000","last_active":"2024-09-11T22:22:15.973552+0000","last_peered":"2024-09-11T22:22:15.973552+0000","last_clean":"2024-09-11T22:22:15.973552+0000","last_became_active":"2024-09-11T22:20:18.184906+0000","last_became_peered":"2024-09-11T22:20:18.184906+0000","last_unstale":"2024-09-11T22:22:15.973552+0000","last_undegraded":"2024-09-11T22:22:15.973552+0000","last_fullsized":"2024-09-11T22:22:15.973552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:32.294780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977515+0000","last_change":"2024-09-11T22:20:18.187137+0000","last_active":"2024-09-11T22:22:15.977515+0000","last_peered":"2024-09-11T22:22:15.977515+0000","last_clean":"2024-09-11T22:22:15.977515+0000","last_became_active":"2024-09-11T22:20:18.186770+0000","last_became_peered":"2024-09-11T22:20:18.186770+0000","last_unstale":"2024-09-11T22:22:15.977515+0000","last_undegraded":"2024-09-11T22:22:15.977515+0000","last_fullsized":"2024-09-11T22:22:15.977515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:59.917695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234743+0000","last_change":"2024-09-11T22:20:18.197137+0000","last_active":"2024-09-11T22:22:16.234743+0000","last_peered":"2024-09-11T22:22:16.234743+0000","last_clean":"2024-09-11T22:22:16.234743+0000","last_became_active":"2024-09-11T22:20:18.192810+0000","last_became_peered":"2024-09-11T22:20:18.192810+0000","last_unstale":"2024-09-11T22:22:16.234743+0000","last_undegraded":"2024-09-11T22:22:16.234743+0000","last_fullsized":"2024-09-11T22:22:16.234743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:57.680726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230830+0000","last_change":"2024-09-11T22:20:18.187568+0000","last_active":"2024-09-11T22:22:16.230830+0000","last_peered":"2024-09-11T22:22:16.230830+0000","last_clean":"2024-09-11T22:22:16.230830+0000","last_became_active":"2024-09-11T22:20:18.187254+0000","last_became_peered":"2024-09-11T22:20:18.187254+0000","last_unstale":"2024-09-11T22:22:16.230830+0000","last_undegraded":"2024-09-11T22:22:16.230830+0000","last_fullsized":"2024-09-11T22:22:16.230830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:41.724926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975378+0000","last_change":"2024-09-11T22:20:18.190376+0000","last_active":"2024-09-11T22:22:15.975378+0000","last_peered":"2024-09-11T22:22:15.975378+0000","last_clean":"2024-09-11T22:22:15.975378+0000","last_became_active":"2024-09-11T22:20:18.190079+0000","last_became_peered":"2024-09-11T22:20:18.190079+0000","last_unstale":"2024-09-11T22:22:15.975378+0000","last_undegraded":"2024-09-11T22:22:15.975378+0000","last_fullsized":"2024-09-11T22:22:15.975378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:42.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058896+0000","last_change":"2024-09-11T22:20:18.188503+0000","last_active":"2024-09-11T22:22:16.058896+0000","last_peered":"2024-09-11T22:22:16.058896+0000","last_clean":"2024-09-11T22:22:16.058896+0000","last_became_active":"2024-09-11T22:20:18.187957+0000","last_became_peered":"2024-09-11T22:20:18.187957+0000","last_unstale":"2024-09-11T22:22:16.058896+0000","last_undegraded":"2024-09-11T22:22:16.058896+0000","last_fullsized":"2024-09-11T22:22:16.058896+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:44:25.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065572+0000","last_change":"2024-09-11T22:20:18.196936+0000","last_active":"2024-09-11T22:22:16.065572+0000","last_peered":"2024-09-11T22:22:16.065572+0000","last_clean":"2024-09-11T22:22:16.065572+0000","last_became_active":"2024-09-11T22:20:18.196493+0000","last_became_peered":"2024-09-11T22:20:18.196493+0000","last_unstale":"2024-09-11T22:22:16.065572+0000","last_undegraded":"2024-09-11T22:22:16.065572+0000","last_fullsized":"2024-09-11T22:22:16.065572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:06.189806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059171+0000","last_change":"2024-09-11T22:20:18.195260+0000","last_active":"2024-09-11T22:22:16.059171+0000","last_peered":"2024-09-11T22:22:16.059171+0000","last_clean":"2024-09-11T22:22:16.059171+0000","last_became_active":"2024-09-11T22:20:18.194956+0000","last_became_peered":"2024-09-11T22:20:18.194956+0000","last_unstale":"2024-09-11T22:22:16.059171+0000","last_undegraded":"2024-09-11T22:22:16.059171+0000","last_fullsized":"2024-09-11T22:22:16.059171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:34.483291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977741+0000","last_change":"2024-09-11T22:20:18.190591+0000","last_active":"2024-09-11T22:22:15.977741+0000","last_peered":"2024-09-11T22:22:15.977741+0000","last_clean":"2024-09-11T22:22:15.977741+0000","last_became_active":"2024-09-11T22:20:18.190007+0000","last_became_peered":"2024-09-11T22:20:18.190007+0000","last_unstale":"2024-09-11T22:22:15.977741+0000","last_undegraded":"2024-09-11T22:22:15.977741+0000","last_fullsized":"2024-09-11T22:22:15.977741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:46.281705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234929+0000","last_change":"2024-09-11T22:20:18.193377+0000","last_active":"2024-09-11T22:22:16.234929+0000","last_peered":"2024-09-11T22:22:16.234929+0000","last_clean":"2024-09-11T22:22:16.234929+0000","last_became_active":"2024-09-11T22:20:18.193003+0000","last_became_peered":"2024-09-11T22:20:18.193003+0000","last_unstale":"2024-09-11T22:22:16.234929+0000","last_undegraded":"2024-09-11T22:22:16.234929+0000","last_fullsized":"2024-09-11T22:22:16.234929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:59.052966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230737+0000","last_change":"2024-09-11T22:20:18.189405+0000","last_active":"2024-09-11T22:22:16.230737+0000","last_peered":"2024-09-11T22:22:16.230737+0000","last_clean":"2024-09-11T22:22:16.230737+0000","last_became_active":"2024-09-11T22:20:18.189098+0000","last_became_peered":"2024-09-11T22:20:18.189098+0000","last_unstale":"2024-09-11T22:22:16.230737+0000","last_undegraded":"2024-09-11T22:22:16.230737+0000","last_fullsized":"2024-09-11T22:22:16.230737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:47.088192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233900+0000","last_change":"2024-09-11T22:20:18.191005+0000","last_active":"2024-09-11T22:22:16.233900+0000","last_peered":"2024-09-11T22:22:16.233900+0000","last_clean":"2024-09-11T22:22:16.233900+0000","last_became_active":"2024-09-11T22:20:18.190681+0000","last_became_peered":"2024-09-11T22:20:18.190681+0000","last_unstale":"2024-09-11T22:22:16.233900+0000","last_undegraded":"2024-09-11T22:22:16.233900+0000","last_fullsized":"2024-09-11T22:22:16.233900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:12.903929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977265+0000","last_change":"2024-09-11T22:20:18.187920+0000","last_active":"2024-09-11T22:22:15.977265+0000","last_peered":"2024-09-11T22:22:15.977265+0000","last_clean":"2024-09-11T22:22:15.977265+0000","last_became_active":"2024-09-11T22:20:18.187526+0000","last_became_peered":"2024-09-11T22:20:18.187526+0000","last_unstale":"2024-09-11T22:22:15.977265+0000","last_undegraded":"2024-09-11T22:22:15.977265+0000","last_fullsized":"2024-09-11T22:22:15.977265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.347857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977444+0000","last_change":"2024-09-11T22:20:18.193869+0000","last_active":"2024-09-11T22:22:15.977444+0000","last_peered":"2024-09-11T22:22:15.977444+0000","last_clean":"2024-09-11T22:22:15.977444+0000","last_became_active":"2024-09-11T22:20:18.193468+0000","last_became_peered":"2024-09-11T22:20:18.193468+0000","last_unstale":"2024-09-11T22:22:15.977444+0000","last_undegraded":"2024-09-11T22:22:15.977444+0000","last_fullsized":"2024-09-11T22:22:15.977444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:59.619611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234869+0000","last_change":"2024-09-11T22:20:18.193947+0000","last_active":"2024-09-11T22:22:16.234869+0000","last_peered":"2024-09-11T22:22:16.234869+0000","last_clean":"2024-09-11T22:22:16.234869+0000","last_became_active":"2024-09-11T22:20:18.192257+0000","last_became_peered":"2024-09-11T22:20:18.192257+0000","last_unstale":"2024-09-11T22:22:16.234869+0000","last_undegraded":"2024-09-11T22:22:16.234869+0000","last_fullsized":"2024-09-11T22:22:16.234869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.478730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058805+0000","last_change":"2024-09-11T22:20:18.184549+0000","last_active":"2024-09-11T22:22:16.058805+0000","last_peered":"2024-09-11T22:22:16.058805+0000","last_clean":"2024-09-11T22:22:16.058805+0000","last_became_active":"2024-09-11T22:20:18.184075+0000","last_became_peered":"2024-09-11T22:20:18.184075+0000","last_unstale":"2024-09-11T22:22:16.058805+0000","last_undegraded":"2024-09-11T22:22:16.058805+0000","last_fullsized":"2024-09-11T22:22:16.058805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:50.095643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'8","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.428358+0000","last_change":"2024-09-11T22:20:16.128524+0000","last_active":"2024-09-11T22:22:25.428358+0000","last_peered":"2024-09-11T22:22:25.428358+0000","last_clean":"2024-09-11T22:22:25.428358+0000","last_became_active":"2024-09-11T22:20:16.128185+0000","last_became_peered":"2024-09-11T22:20:16.128185+0000","last_unstale":"2024-09-11T22:22:25.428358+0000","last_undegraded":"2024-09-11T22:22:25.428358+0000","last_fullsized":"2024-09-11T22:22:25.428358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.022931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1716,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065720+0000","last_change":"2024-09-11T22:20:18.191578+0000","last_active":"2024-09-11T22:22:16.065720+0000","last_peered":"2024-09-11T22:22:16.065720+0000","last_clean":"2024-09-11T22:22:16.065720+0000","last_became_active":"2024-09-11T22:20:18.189568+0000","last_became_peered":"2024-09-11T22:20:18.189568+0000","last_unstale":"2024-09-11T22:22:16.065720+0000","last_undegraded":"2024-09-11T22:22:16.065720+0000","last_fullsized":"2024-09-11T22:22:16.065720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:41.737242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068742+0000","last_change":"2024-09-11T22:21:18.199693+0000","last_active":"2024-09-11T22:22:16.068742+0000","last_peered":"2024-09-11T22:22:16.068742+0000","last_clean":"2024-09-11T22:22:16.068742+0000","last_became_active":"2024-09-11T22:21:18.190954+0000","last_became_peered":"2024-09-11T22:21:18.190954+0000","last_unstale":"2024-09-11T22:22:16.068742+0000","last_undegraded":"2024-09-11T22:22:16.068742+0000","last_fullsized":"2024-09-11T22:22:16.068742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:11.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230688+0000","last_change":"2024-09-11T22:20:16.115602+0000","last_active":"2024-09-11T22:22:16.230688+0000","last_peered":"2024-09-11T22:22:16.230688+0000","last_clean":"2024-09-11T22:22:16.230688+0000","last_became_active":"2024-09-11T22:20:16.115280+0000","last_became_peered":"2024-09-11T22:20:16.115280+0000","last_unstale":"2024-09-11T22:22:16.230688+0000","last_undegraded":"2024-09-11T22:22:16.230688+0000","last_fullsized":"2024-09-11T22:22:16.230688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:41.942137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058816+0000","last_change":"2024-09-11T22:20:18.184125+0000","last_active":"2024-09-11T22:22:16.058816+0000","last_peered":"2024-09-11T22:22:16.058816+0000","last_clean":"2024-09-11T22:22:16.058816+0000","last_became_active":"2024-09-11T22:20:18.183785+0000","last_became_peered":"2024-09-11T22:20:18.183785+0000","last_unstale":"2024-09-11T22:22:16.058816+0000","last_undegraded":"2024-09-11T22:22:16.058816+0000","last_fullsized":"2024-09-11T22:22:16.058816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:33.230175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065780+0000","last_change":"2024-09-11T22:20:16.125852+0000","last_active":"2024-09-11T22:22:16.065780+0000","last_peered":"2024-09-11T22:22:16.065780+0000","last_clean":"2024-09-11T22:22:16.065780+0000","last_became_active":"2024-09-11T22:20:16.125250+0000","last_became_peered":"2024-09-11T22:20:16.125250+0000","last_unstale":"2024-09-11T22:22:16.065780+0000","last_undegraded":"2024-09-11T22:22:16.065780+0000","last_fullsized":"2024-09-11T22:22:16.065780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:34.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973235+0000","last_change":"2024-09-11T22:20:18.185147+0000","last_active":"2024-09-11T22:22:15.973235+0000","last_peered":"2024-09-11T22:22:15.973235+0000","last_clean":"2024-09-11T22:22:15.973235+0000","last_became_active":"2024-09-11T22:20:18.184840+0000","last_became_peered":"2024-09-11T22:20:18.184840+0000","last_unstale":"2024-09-11T22:22:15.973235+0000","last_undegraded":"2024-09-11T22:22:15.973235+0000","last_fullsized":"2024-09-11T22:22:15.973235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:59.703349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":263,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.322621+0000","last_change":"2024-09-11T22:20:16.122173+0000","last_active":"2024-09-11T22:22:26.322621+0000","last_peered":"2024-09-11T22:22:26.322621+0000","last_clean":"2024-09-11T22:22:26.322621+0000","last_became_active":"2024-09-11T22:20:16.121847+0000","last_became_peered":"2024-09-11T22:20:16.121847+0000","last_unstale":"2024-09-11T22:22:26.322621+0000","last_undegraded":"2024-09-11T22:22:26.322621+0000","last_fullsized":"2024-09-11T22:22:26.322621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:41.284557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":233,"num_read_kb":233,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975691+0000","last_change":"2024-09-11T22:20:18.188030+0000","last_active":"2024-09-11T22:22:15.975691+0000","last_peered":"2024-09-11T22:22:15.975691+0000","last_clean":"2024-09-11T22:22:15.975691+0000","last_became_active":"2024-09-11T22:20:18.187503+0000","last_became_peered":"2024-09-11T22:20:18.187503+0000","last_unstale":"2024-09-11T22:22:15.975691+0000","last_undegraded":"2024-09-11T22:22:15.975691+0000","last_fullsized":"2024-09-11T22:22:15.975691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:45.494149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231598+0000","last_change":"2024-09-11T22:20:16.128828+0000","last_active":"2024-09-11T22:22:16.231598+0000","last_peered":"2024-09-11T22:22:16.231598+0000","last_clean":"2024-09-11T22:22:16.231598+0000","last_became_active":"2024-09-11T22:20:16.128454+0000","last_became_peered":"2024-09-11T22:20:16.128454+0000","last_unstale":"2024-09-11T22:22:16.231598+0000","last_undegraded":"2024-09-11T22:22:16.231598+0000","last_fullsized":"2024-09-11T22:22:16.231598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:33.927564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234475+0000","last_change":"2024-09-11T22:20:18.193795+0000","last_active":"2024-09-11T22:22:16.234475+0000","last_peered":"2024-09-11T22:22:16.234475+0000","last_clean":"2024-09-11T22:22:16.234475+0000","last_became_active":"2024-09-11T22:20:18.193489+0000","last_became_peered":"2024-09-11T22:20:18.193489+0000","last_unstale":"2024-09-11T22:22:16.234475+0000","last_undegraded":"2024-09-11T22:22:16.234475+0000","last_fullsized":"2024-09-11T22:22:16.234475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:14.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065914+0000","last_change":"2024-09-11T22:20:16.187938+0000","last_active":"2024-09-11T22:22:16.065914+0000","last_peered":"2024-09-11T22:22:16.065914+0000","last_clean":"2024-09-11T22:22:16.065914+0000","last_became_active":"2024-09-11T22:20:16.187545+0000","last_became_peered":"2024-09-11T22:20:16.187545+0000","last_unstale":"2024-09-11T22:22:16.065914+0000","last_undegraded":"2024-09-11T22:22:16.065914+0000","last_fullsized":"2024-09-11T22:22:16.065914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:41.674356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975385+0000","last_change":"2024-09-11T22:21:18.191937+0000","last_active":"2024-09-11T22:22:15.975385+0000","last_peered":"2024-09-11T22:22:15.975385+0000","last_clean":"2024-09-11T22:22:15.975385+0000","last_became_active":"2024-09-11T22:21:18.191218+0000","last_became_peered":"2024-09-11T22:21:18.191218+0000","last_unstale":"2024-09-11T22:22:15.975385+0000","last_undegraded":"2024-09-11T22:22:15.975385+0000","last_fullsized":"2024-09-11T22:22:15.975385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:41.654630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":312,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.428665+0000","last_change":"2024-09-11T22:20:16.128082+0000","last_active":"2024-09-11T22:22:26.428665+0000","last_peered":"2024-09-11T22:22:26.428665+0000","last_clean":"2024-09-11T22:22:26.428665+0000","last_became_active":"2024-09-11T22:20:16.127731+0000","last_became_peered":"2024-09-11T22:20:16.127731+0000","last_unstale":"2024-09-11T22:22:26.428665+0000","last_undegraded":"2024-09-11T22:22:26.428665+0000","last_fullsized":"2024-09-11T22:22:26.428665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:33.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69393,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":193,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059064+0000","last_change":"2024-09-11T22:20:18.189219+0000","last_active":"2024-09-11T22:22:16.059064+0000","last_peered":"2024-09-11T22:22:16.059064+0000","last_clean":"2024-09-11T22:22:16.059064+0000","last_became_active":"2024-09-11T22:20:18.188687+0000","last_became_peered":"2024-09-11T22:20:18.188687+0000","last_unstale":"2024-09-11T22:22:16.059064+0000","last_undegraded":"2024-09-11T22:22:16.059064+0000","last_fullsized":"2024-09-11T22:22:16.059064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:24.171126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977378+0000","last_change":"2024-09-11T22:20:16.129724+0000","last_active":"2024-09-11T22:22:15.977378+0000","last_peered":"2024-09-11T22:22:15.977378+0000","last_clean":"2024-09-11T22:22:15.977378+0000","last_became_active":"2024-09-11T22:20:16.129351+0000","last_became_peered":"2024-09-11T22:20:16.129351+0000","last_unstale":"2024-09-11T22:22:15.977378+0000","last_undegraded":"2024-09-11T22:22:15.977378+0000","last_fullsized":"2024-09-11T22:22:15.977378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:09.642520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230435+0000","last_change":"2024-09-11T22:20:18.187031+0000","last_active":"2024-09-11T22:22:16.230435+0000","last_peered":"2024-09-11T22:22:16.230435+0000","last_clean":"2024-09-11T22:22:16.230435+0000","last_became_active":"2024-09-11T22:20:18.186698+0000","last_became_peered":"2024-09-11T22:20:18.186698+0000","last_unstale":"2024-09-11T22:22:16.230435+0000","last_undegraded":"2024-09-11T22:22:16.230435+0000","last_fullsized":"2024-09-11T22:22:16.230435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:04.810981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065725+0000","last_change":"2024-09-11T22:20:16.124925+0000","last_active":"2024-09-11T22:22:16.065725+0000","last_peered":"2024-09-11T22:22:16.065725+0000","last_clean":"2024-09-11T22:22:16.065725+0000","last_became_active":"2024-09-11T22:20:16.124580+0000","last_became_peered":"2024-09-11T22:20:16.124580+0000","last_unstale":"2024-09-11T22:22:16.065725+0000","last_undegraded":"2024-09-11T22:22:16.065725+0000","last_fullsized":"2024-09-11T22:22:16.065725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:56.417577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231838+0000","last_change":"2024-09-11T22:21:18.187783+0000","last_active":"2024-09-11T22:22:16.231838+0000","last_peered":"2024-09-11T22:22:16.231838+0000","last_clean":"2024-09-11T22:22:16.231838+0000","last_became_active":"2024-09-11T22:21:18.187358+0000","last_became_peered":"2024-09-11T22:21:18.187358+0000","last_unstale":"2024-09-11T22:22:16.231838+0000","last_undegraded":"2024-09-11T22:22:16.231838+0000","last_fullsized":"2024-09-11T22:22:16.231838+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:41.251531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.452982+0000","last_change":"2024-09-11T22:20:16.118934+0000","last_active":"2024-09-11T22:22:25.452982+0000","last_peered":"2024-09-11T22:22:25.452982+0000","last_clean":"2024-09-11T22:22:25.452982+0000","last_became_active":"2024-09-11T22:20:16.118614+0000","last_became_peered":"2024-09-11T22:20:16.118614+0000","last_unstale":"2024-09-11T22:22:25.452982+0000","last_undegraded":"2024-09-11T22:22:25.452982+0000","last_fullsized":"2024-09-11T22:22:25.452982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:05.791161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231721+0000","last_change":"2024-09-11T22:20:18.197692+0000","last_active":"2024-09-11T22:22:16.231721+0000","last_peered":"2024-09-11T22:22:16.231721+0000","last_clean":"2024-09-11T22:22:16.231721+0000","last_became_active":"2024-09-11T22:20:18.197336+0000","last_became_peered":"2024-09-11T22:20:18.197336+0000","last_unstale":"2024-09-11T22:22:16.231721+0000","last_undegraded":"2024-09-11T22:22:16.231721+0000","last_fullsized":"2024-09-11T22:22:16.231721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:55.172568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:16.186769+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:16.186367+0000","last_became_peered":"2024-09-11T22:20:16.186367+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.279387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065641+0000","last_change":"2024-09-11T22:20:18.195570+0000","last_active":"2024-09-11T22:22:16.065641+0000","last_peered":"2024-09-11T22:22:16.065641+0000","last_clean":"2024-09-11T22:22:16.065641+0000","last_became_active":"2024-09-11T22:20:18.195226+0000","last_became_peered":"2024-09-11T22:20:18.195226+0000","last_unstale":"2024-09-11T22:22:16.065641+0000","last_undegraded":"2024-09-11T22:22:16.065641+0000","last_fullsized":"2024-09-11T22:22:16.065641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:44.108090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235034+0000","last_change":"2024-09-11T22:21:18.194553+0000","last_active":"2024-09-11T22:22:16.235034+0000","last_peered":"2024-09-11T22:22:16.235034+0000","last_clean":"2024-09-11T22:22:16.235034+0000","last_became_active":"2024-09-11T22:21:18.188635+0000","last_became_peered":"2024-09-11T22:21:18.188635+0000","last_unstale":"2024-09-11T22:22:16.235034+0000","last_undegraded":"2024-09-11T22:22:16.235034+0000","last_fullsized":"2024-09-11T22:22:16.235034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:34.165195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234954+0000","last_change":"2024-09-11T22:20:18.194112+0000","last_active":"2024-09-11T22:22:16.234954+0000","last_peered":"2024-09-11T22:22:16.234954+0000","last_clean":"2024-09-11T22:22:16.234954+0000","last_became_active":"2024-09-11T22:20:18.193812+0000","last_became_peered":"2024-09-11T22:20:18.193812+0000","last_unstale":"2024-09-11T22:22:16.234954+0000","last_undegraded":"2024-09-11T22:22:16.234954+0000","last_fullsized":"2024-09-11T22:22:16.234954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:35.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231070+0000","last_change":"2024-09-11T22:21:18.198325+0000","last_active":"2024-09-11T22:22:16.231070+0000","last_peered":"2024-09-11T22:22:16.231070+0000","last_clean":"2024-09-11T22:22:16.231070+0000","last_became_active":"2024-09-11T22:21:18.193651+0000","last_became_peered":"2024-09-11T22:21:18.193651+0000","last_unstale":"2024-09-11T22:22:16.231070+0000","last_undegraded":"2024-09-11T22:22:16.231070+0000","last_fullsized":"2024-09-11T22:22:16.231070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:42.552492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232027+0000","last_change":"2024-09-11T22:20:18.191960+0000","last_active":"2024-09-11T22:22:16.232027+0000","last_peered":"2024-09-11T22:22:16.232027+0000","last_clean":"2024-09-11T22:22:16.232027+0000","last_became_active":"2024-09-11T22:20:18.191635+0000","last_became_peered":"2024-09-11T22:20:18.191635+0000","last_unstale":"2024-09-11T22:22:16.232027+0000","last_undegraded":"2024-09-11T22:22:16.232027+0000","last_fullsized":"2024-09-11T22:22:16.232027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:19:18.919247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.988172+0000","last_change":"2024-09-11T22:20:16.123608+0000","last_active":"2024-09-11T22:22:26.988172+0000","last_peered":"2024-09-11T22:22:26.988172+0000","last_clean":"2024-09-11T22:22:26.988172+0000","last_became_active":"2024-09-11T22:20:16.123193+0000","last_became_peered":"2024-09-11T22:20:16.123193+0000","last_unstale":"2024-09-11T22:22:26.988172+0000","last_undegraded":"2024-09-11T22:22:26.988172+0000","last_fullsized":"2024-09-11T22:22:26.988172+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:07.362115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":26,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231887+0000","last_change":"2024-09-11T22:20:18.199957+0000","last_active":"2024-09-11T22:22:16.231887+0000","last_peered":"2024-09-11T22:22:16.231887+0000","last_clean":"2024-09-11T22:22:16.231887+0000","last_became_active":"2024-09-11T22:20:18.199637+0000","last_became_peered":"2024-09-11T22:20:18.199637+0000","last_unstale":"2024-09-11T22:22:16.231887+0000","last_undegraded":"2024-09-11T22:22:16.231887+0000","last_fullsized":"2024-09-11T22:22:16.231887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:14.949002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":94,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.321741+0000","last_change":"2024-09-11T22:21:18.198896+0000","last_active":"2024-09-11T22:22:25.321741+0000","last_peered":"2024-09-11T22:22:25.321741+0000","last_clean":"2024-09-11T22:22:25.321741+0000","last_became_active":"2024-09-11T22:21:18.189489+0000","last_became_peered":"2024-09-11T22:21:18.189489+0000","last_unstale":"2024-09-11T22:22:25.321741+0000","last_undegraded":"2024-09-11T22:22:25.321741+0000","last_fullsized":"2024-09-11T22:22:25.321741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:19.908801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1956,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232111+0000","last_change":"2024-09-11T22:20:18.196691+0000","last_active":"2024-09-11T22:22:16.232111+0000","last_peered":"2024-09-11T22:22:16.232111+0000","last_clean":"2024-09-11T22:22:16.232111+0000","last_became_active":"2024-09-11T22:20:18.196364+0000","last_became_peered":"2024-09-11T22:20:18.196364+0000","last_unstale":"2024-09-11T22:22:16.232111+0000","last_undegraded":"2024-09-11T22:22:16.232111+0000","last_fullsized":"2024-09-11T22:22:16.232111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:50.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"83'148","reported_seq":392,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:18.074584+0000","last_change":"2024-09-11T22:19:02.590079+0000","last_active":"2024-09-11T22:22:18.074584+0000","last_peered":"2024-09-11T22:22:18.074584+0000","last_clean":"2024-09-11T22:22:18.074584+0000","last_became_active":"2024-09-11T22:19:02.589644+0000","last_became_peered":"2024-09-11T22:19:02.589644+0000","last_unstale":"2024-09-11T22:22:18.074584+0000","last_undegraded":"2024-09-11T22:22:18.074584+0000","last_fullsized":"2024-09-11T22:22:18.074584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_clean_scrub_stamp":"2024-09-11T22:15:33.244685+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:45.506190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.229817+0000","last_change":"2024-09-11T22:20:16.116661+0000","last_active":"2024-09-11T22:22:16.229817+0000","last_peered":"2024-09-11T22:22:16.229817+0000","last_clean":"2024-09-11T22:22:16.229817+0000","last_became_active":"2024-09-11T22:20:16.116196+0000","last_became_peered":"2024-09-11T22:20:16.116196+0000","last_unstale":"2024-09-11T22:22:16.229817+0000","last_undegraded":"2024-09-11T22:22:16.229817+0000","last_fullsized":"2024-09-11T22:22:16.229817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:08.875170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059213+0000","last_change":"2024-09-11T22:20:18.193176+0000","last_active":"2024-09-11T22:22:16.059213+0000","last_peered":"2024-09-11T22:22:16.059213+0000","last_clean":"2024-09-11T22:22:16.059213+0000","last_became_active":"2024-09-11T22:20:18.192312+0000","last_became_peered":"2024-09-11T22:20:18.192312+0000","last_unstale":"2024-09-11T22:22:16.059213+0000","last_undegraded":"2024-09-11T22:22:16.059213+0000","last_fullsized":"2024-09-11T22:22:16.059213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:08.738569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234694+0000","last_change":"2024-09-11T22:21:18.196985+0000","last_active":"2024-09-11T22:22:16.234694+0000","last_peered":"2024-09-11T22:22:16.234694+0000","last_clean":"2024-09-11T22:22:16.234694+0000","last_became_active":"2024-09-11T22:21:18.189487+0000","last_became_peered":"2024-09-11T22:21:18.189487+0000","last_unstale":"2024-09-11T22:22:16.234694+0000","last_undegraded":"2024-09-11T22:22:16.234694+0000","last_fullsized":"2024-09-11T22:22:16.234694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:20.715696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230517+0000","last_change":"2024-09-11T22:20:18.191118+0000","last_active":"2024-09-11T22:22:16.230517+0000","last_peered":"2024-09-11T22:22:16.230517+0000","last_clean":"2024-09-11T22:22:16.230517+0000","last_became_active":"2024-09-11T22:20:18.190760+0000","last_became_peered":"2024-09-11T22:20:18.190760+0000","last_unstale":"2024-09-11T22:22:16.230517+0000","last_undegraded":"2024-09-11T22:22:16.230517+0000","last_fullsized":"2024-09-11T22:22:16.230517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:59.220067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.452301+0000","last_change":"2024-09-11T22:20:16.186407+0000","last_active":"2024-09-11T22:22:25.452301+0000","last_peered":"2024-09-11T22:22:25.452301+0000","last_clean":"2024-09-11T22:22:25.452301+0000","last_became_active":"2024-09-11T22:20:16.186105+0000","last_became_peered":"2024-09-11T22:20:16.186105+0000","last_unstale":"2024-09-11T22:22:25.452301+0000","last_undegraded":"2024-09-11T22:22:25.452301+0000","last_fullsized":"2024-09-11T22:22:25.452301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:43.942072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065681+0000","last_change":"2024-09-11T22:20:18.191970+0000","last_active":"2024-09-11T22:22:16.065681+0000","last_peered":"2024-09-11T22:22:16.065681+0000","last_clean":"2024-09-11T22:22:16.065681+0000","last_became_active":"2024-09-11T22:20:18.191623+0000","last_became_peered":"2024-09-11T22:20:18.191623+0000","last_unstale":"2024-09-11T22:22:16.065681+0000","last_undegraded":"2024-09-11T22:22:16.065681+0000","last_fullsized":"2024-09-11T22:22:16.065681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:33.213979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.441040+0000","last_change":"2024-09-11T22:20:16.119936+0000","last_active":"2024-09-11T22:22:25.441040+0000","last_peered":"2024-09-11T22:22:25.441040+0000","last_clean":"2024-09-11T22:22:25.441040+0000","last_became_active":"2024-09-11T22:20:16.119453+0000","last_became_peered":"2024-09-11T22:20:16.119453+0000","last_unstale":"2024-09-11T22:22:25.441040+0000","last_undegraded":"2024-09-11T22:22:25.441040+0000","last_fullsized":"2024-09-11T22:22:25.441040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:20.717980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068855+0000","last_change":"2024-09-11T22:20:18.192581+0000","last_active":"2024-09-11T22:22:16.068855+0000","last_peered":"2024-09-11T22:22:16.068855+0000","last_clean":"2024-09-11T22:22:16.068855+0000","last_became_active":"2024-09-11T22:20:18.191702+0000","last_became_peered":"2024-09-11T22:20:18.191702+0000","last_unstale":"2024-09-11T22:22:16.068855+0000","last_undegraded":"2024-09-11T22:22:16.068855+0000","last_fullsized":"2024-09-11T22:22:16.068855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:48.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973445+0000","last_change":"2024-09-11T22:20:16.186485+0000","last_active":"2024-09-11T22:22:15.973445+0000","last_peered":"2024-09-11T22:22:15.973445+0000","last_clean":"2024-09-11T22:22:15.973445+0000","last_became_active":"2024-09-11T22:20:16.186171+0000","last_became_peered":"2024-09-11T22:20:16.186171+0000","last_unstale":"2024-09-11T22:22:15.973445+0000","last_undegraded":"2024-09-11T22:22:15.973445+0000","last_fullsized":"2024-09-11T22:22:15.973445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:26.373626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068932+0000","last_change":"2024-09-11T22:20:18.191361+0000","last_active":"2024-09-11T22:22:16.068932+0000","last_peered":"2024-09-11T22:22:16.068932+0000","last_clean":"2024-09-11T22:22:16.068932+0000","last_became_active":"2024-09-11T22:20:18.191036+0000","last_became_peered":"2024-09-11T22:20:18.191036+0000","last_unstale":"2024-09-11T22:22:16.068932+0000","last_undegraded":"2024-09-11T22:22:16.068932+0000","last_fullsized":"2024-09-11T22:22:16.068932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:29.124650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059209+0000","last_change":"2024-09-11T22:20:16.188156+0000","last_active":"2024-09-11T22:22:16.059209+0000","last_peered":"2024-09-11T22:22:16.059209+0000","last_clean":"2024-09-11T22:22:16.059209+0000","last_became_active":"2024-09-11T22:20:16.187809+0000","last_became_peered":"2024-09-11T22:20:16.187809+0000","last_unstale":"2024-09-11T22:22:16.059209+0000","last_undegraded":"2024-09-11T22:22:16.059209+0000","last_fullsized":"2024-09-11T22:22:16.059209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:04.219621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230944+0000","last_change":"2024-09-11T22:20:18.182380+0000","last_active":"2024-09-11T22:22:16.230944+0000","last_peered":"2024-09-11T22:22:16.230944+0000","last_clean":"2024-09-11T22:22:16.230944+0000","last_became_active":"2024-09-11T22:20:18.182062+0000","last_became_peered":"2024-09-11T22:20:18.182062+0000","last_unstale":"2024-09-11T22:22:16.230944+0000","last_undegraded":"2024-09-11T22:22:16.230944+0000","last_fullsized":"2024-09-11T22:22:16.230944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:15.575846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'91","reported_seq":187,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.997407+0000","last_change":"2024-09-11T22:20:16.117335+0000","last_active":"2024-09-11T22:22:26.997407+0000","last_peered":"2024-09-11T22:22:26.997407+0000","last_clean":"2024-09-11T22:22:26.997407+0000","last_became_active":"2024-09-11T22:20:16.116977+0000","last_became_peered":"2024-09-11T22:20:16.116977+0000","last_unstale":"2024-09-11T22:22:26.997407+0000","last_undegraded":"2024-09-11T22:22:26.997407+0000","last_fullsized":"2024-09-11T22:22:26.997407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:51.744202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":244,"num_read_kb":215,"num_write":164,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059281+0000","last_change":"2024-09-11T22:20:18.194069+0000","last_active":"2024-09-11T22:22:16.059281+0000","last_peered":"2024-09-11T22:22:16.059281+0000","last_clean":"2024-09-11T22:22:16.059281+0000","last_became_active":"2024-09-11T22:20:18.193765+0000","last_became_peered":"2024-09-11T22:20:18.193765+0000","last_unstale":"2024-09-11T22:22:16.059281+0000","last_undegraded":"2024-09-11T22:22:16.059281+0000","last_fullsized":"2024-09-11T22:22:16.059281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:25.047405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230178+0000","last_change":"2024-09-11T22:20:16.460987+0000","last_active":"2024-09-11T22:22:16.230178+0000","last_peered":"2024-09-11T22:22:16.230178+0000","last_clean":"2024-09-11T22:22:16.230178+0000","last_became_active":"2024-09-11T22:20:16.460516+0000","last_became_peered":"2024-09-11T22:20:16.460516+0000","last_unstale":"2024-09-11T22:22:16.230178+0000","last_undegraded":"2024-09-11T22:22:16.230178+0000","last_fullsized":"2024-09-11T22:22:16.230178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:00.987981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234312+0000","last_change":"2024-09-11T22:21:18.188802+0000","last_active":"2024-09-11T22:22:16.234312+0000","last_peered":"2024-09-11T22:22:16.234312+0000","last_clean":"2024-09-11T22:22:16.234312+0000","last_became_active":"2024-09-11T22:21:18.188441+0000","last_became_peered":"2024-09-11T22:21:18.188441+0000","last_unstale":"2024-09-11T22:22:16.234312+0000","last_undegraded":"2024-09-11T22:22:16.234312+0000","last_fullsized":"2024-09-11T22:22:16.234312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:02.295441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975385+0000","last_change":"2024-09-11T22:20:16.186498+0000","last_active":"2024-09-11T22:22:15.975385+0000","last_peered":"2024-09-11T22:22:15.975385+0000","last_clean":"2024-09-11T22:22:15.975385+0000","last_became_active":"2024-09-11T22:20:16.186214+0000","last_became_peered":"2024-09-11T22:20:16.186214+0000","last_unstale":"2024-09-11T22:22:15.975385+0000","last_undegraded":"2024-09-11T22:22:15.975385+0000","last_fullsized":"2024-09-11T22:22:15.975385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:09.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:18.187489+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:18.185649+0000","last_became_peered":"2024-09-11T22:20:18.185649+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:42.825075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059064+0000","last_change":"2024-09-11T22:20:16.117627+0000","last_active":"2024-09-11T22:22:16.059064+0000","last_peered":"2024-09-11T22:22:16.059064+0000","last_clean":"2024-09-11T22:22:16.059064+0000","last_became_active":"2024-09-11T22:20:16.117243+0000","last_became_peered":"2024-09-11T22:20:16.117243+0000","last_unstale":"2024-09-11T22:22:16.059064+0000","last_undegraded":"2024-09-11T22:22:16.059064+0000","last_fullsized":"2024-09-11T22:22:16.059064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:58.240535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234846+0000","last_change":"2024-09-11T22:20:18.184326+0000","last_active":"2024-09-11T22:22:16.234846+0000","last_peered":"2024-09-11T22:22:16.234846+0000","last_clean":"2024-09-11T22:22:16.234846+0000","last_became_active":"2024-09-11T22:20:18.183995+0000","last_became_peered":"2024-09-11T22:20:18.183995+0000","last_unstale":"2024-09-11T22:22:16.234846+0000","last_undegraded":"2024-09-11T22:22:16.234846+0000","last_fullsized":"2024-09-11T22:22:16.234846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:33.468000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977314+0000","last_change":"2024-09-11T22:20:16.127180+0000","last_active":"2024-09-11T22:22:15.977314+0000","last_peered":"2024-09-11T22:22:15.977314+0000","last_clean":"2024-09-11T22:22:15.977314+0000","last_became_active":"2024-09-11T22:20:16.126696+0000","last_became_peered":"2024-09-11T22:20:16.126696+0000","last_unstale":"2024-09-11T22:22:15.977314+0000","last_undegraded":"2024-09-11T22:22:15.977314+0000","last_fullsized":"2024-09-11T22:22:15.977314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:02.637225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973369+0000","last_change":"2024-09-11T22:21:18.188614+0000","last_active":"2024-09-11T22:22:15.973369+0000","last_peered":"2024-09-11T22:22:15.973369+0000","last_clean":"2024-09-11T22:22:15.973369+0000","last_became_active":"2024-09-11T22:21:18.188277+0000","last_became_peered":"2024-09-11T22:21:18.188277+0000","last_unstale":"2024-09-11T22:22:15.973369+0000","last_undegraded":"2024-09-11T22:22:15.973369+0000","last_fullsized":"2024-09-11T22:22:15.973369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:59.995647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058897+0000","last_change":"2024-09-11T22:20:16.128311+0000","last_active":"2024-09-11T22:22:16.058897+0000","last_peered":"2024-09-11T22:22:16.058897+0000","last_clean":"2024-09-11T22:22:16.058897+0000","last_became_active":"2024-09-11T22:20:16.127853+0000","last_became_peered":"2024-09-11T22:20:16.127853+0000","last_unstale":"2024-09-11T22:22:16.058897+0000","last_undegraded":"2024-09-11T22:22:16.058897+0000","last_fullsized":"2024-09-11T22:22:16.058897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:57.562470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231820+0000","last_change":"2024-09-11T22:20:18.195782+0000","last_active":"2024-09-11T22:22:16.231820+0000","last_peered":"2024-09-11T22:22:16.231820+0000","last_clean":"2024-09-11T22:22:16.231820+0000","last_became_active":"2024-09-11T22:20:18.195365+0000","last_became_peered":"2024-09-11T22:20:18.195365+0000","last_unstale":"2024-09-11T22:22:16.231820+0000","last_undegraded":"2024-09-11T22:22:16.231820+0000","last_fullsized":"2024-09-11T22:22:16.231820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:49.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065812+0000","last_change":"2024-09-11T22:20:18.199344+0000","last_active":"2024-09-11T22:22:16.065812+0000","last_peered":"2024-09-11T22:22:16.065812+0000","last_clean":"2024-09-11T22:22:16.065812+0000","last_became_active":"2024-09-11T22:20:18.199016+0000","last_became_peered":"2024-09-11T22:20:18.199016+0000","last_unstale":"2024-09-11T22:22:16.065812+0000","last_undegraded":"2024-09-11T22:22:16.065812+0000","last_fullsized":"2024-09-11T22:22:16.065812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:38.322831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059125+0000","last_change":"2024-09-11T22:20:16.122236+0000","last_active":"2024-09-11T22:22:16.059125+0000","last_peered":"2024-09-11T22:22:16.059125+0000","last_clean":"2024-09-11T22:22:16.059125+0000","last_became_active":"2024-09-11T22:20:16.121867+0000","last_became_peered":"2024-09-11T22:20:16.121867+0000","last_unstale":"2024-09-11T22:22:16.059125+0000","last_undegraded":"2024-09-11T22:22:16.059125+0000","last_fullsized":"2024-09-11T22:22:16.059125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:56.782445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975374+0000","last_change":"2024-09-11T22:20:18.187338+0000","last_active":"2024-09-11T22:22:15.975374+0000","last_peered":"2024-09-11T22:22:15.975374+0000","last_clean":"2024-09-11T22:22:15.975374+0000","last_became_active":"2024-09-11T22:20:18.186985+0000","last_became_peered":"2024-09-11T22:20:18.186985+0000","last_unstale":"2024-09-11T22:22:15.975374+0000","last_undegraded":"2024-09-11T22:22:15.975374+0000","last_fullsized":"2024-09-11T22:22:15.975374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:58.170680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059106+0000","last_change":"2024-09-11T22:20:16.128579+0000","last_active":"2024-09-11T22:22:16.059106+0000","last_peered":"2024-09-11T22:22:16.059106+0000","last_clean":"2024-09-11T22:22:16.059106+0000","last_became_active":"2024-09-11T22:20:16.127864+0000","last_became_peered":"2024-09-11T22:20:16.127864+0000","last_unstale":"2024-09-11T22:22:16.059106+0000","last_undegraded":"2024-09-11T22:22:16.059106+0000","last_fullsized":"2024-09-11T22:22:16.059106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:11:32.005487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234497+0000","last_change":"2024-09-11T22:20:18.185205+0000","last_active":"2024-09-11T22:22:16.234497+0000","last_peered":"2024-09-11T22:22:16.234497+0000","last_clean":"2024-09-11T22:22:16.234497+0000","last_became_active":"2024-09-11T22:20:18.184402+0000","last_became_peered":"2024-09-11T22:20:18.184402+0000","last_unstale":"2024-09-11T22:22:16.234497+0000","last_undegraded":"2024-09-11T22:22:16.234497+0000","last_fullsized":"2024-09-11T22:22:16.234497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:17:48.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.060250+0000","last_change":"2024-09-11T22:20:16.186372+0000","last_active":"2024-09-11T22:22:16.060250+0000","last_peered":"2024-09-11T22:22:16.060250+0000","last_clean":"2024-09-11T22:22:16.060250+0000","last_became_active":"2024-09-11T22:20:16.185990+0000","last_became_peered":"2024-09-11T22:20:16.185990+0000","last_unstale":"2024-09-11T22:22:16.060250+0000","last_undegraded":"2024-09-11T22:22:16.060250+0000","last_fullsized":"2024-09-11T22:22:16.060250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:23.296379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.060015+0000","last_change":"2024-09-11T22:20:18.190670+0000","last_active":"2024-09-11T22:22:16.060015+0000","last_peered":"2024-09-11T22:22:16.060015+0000","last_clean":"2024-09-11T22:22:16.060015+0000","last_became_active":"2024-09-11T22:20:18.190339+0000","last_became_peered":"2024-09-11T22:20:18.190339+0000","last_unstale":"2024-09-11T22:22:16.060015+0000","last_undegraded":"2024-09-11T22:22:16.060015+0000","last_fullsized":"2024-09-11T22:22:16.060015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.085305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.928844+0000","last_change":"2024-09-11T22:20:16.117646+0000","last_active":"2024-09-11T22:22:25.928844+0000","last_peered":"2024-09-11T22:22:25.928844+0000","last_clean":"2024-09-11T22:22:25.928844+0000","last_became_active":"2024-09-11T22:20:16.117111+0000","last_became_peered":"2024-09-11T22:20:16.117111+0000","last_unstale":"2024-09-11T22:22:25.928844+0000","last_undegraded":"2024-09-11T22:22:25.928844+0000","last_fullsized":"2024-09-11T22:22:25.928844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:36.470158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800463,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1442,"num_read_kb":936,"num_write":480,"num_write_kb":2385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1192,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2401389,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108858,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579943,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":444133,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":1236133,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":836471,"data_compressed":27823,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201334,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":836827,"data_compressed":27952,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430268,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":1038174,"data_compressed":44963,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593792,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443506,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822726,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":1168132,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":445575,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215184,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":978944,"data_stored":967613,"data_compressed":33979,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":513422,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":987136,"data_stored":513812,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":937984,"data_stored":445097,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332146,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":400147,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:22:31.477 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T22:22:31.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T22:22:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:32.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:31 smithi005 ceph-mon[30943]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 42 op/s 2024-09-11T22:22:32.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:31 smithi005 ceph-mon[30943]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:32.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:31 smithi159 ceph-mon[35534]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 42 op/s 2024-09-11T22:22:32.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:31 smithi159 ceph-mon[35534]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:31 smithi172 ceph-mon[35461]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 42 op/s 2024-09-11T22:22:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:31 smithi172 ceph-mon[35461]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:32.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.243+0000 7fa1bea3d640 1 -- 172.21.15.5:0/1264689865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b8102d40 msgr2=0x7fa1b8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:32.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.243+0000 7fa1bea3d640 1 --2- 172.21.15.5:0/1264689865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b8102d40 0x7fa1b8103120 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8009900 tx=0x7fa1a802a4a0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.244+0000 7fa1bea3d640 1 -- 172.21.15.5:0/1264689865 shutdown_connections 2024-09-11T22:22:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.244+0000 7fa1bea3d640 1 --2- 172.21.15.5:0/1264689865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b810c170 0x7fa1b810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.244+0000 7fa1bea3d640 1 --2- 172.21.15.5:0/1264689865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1b8103660 0x7fa1b810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.244+0000 7fa1bea3d640 1 --2- 172.21.15.5:0/1264689865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b8102d40 0x7fa1b8103120 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.244+0000 7fa1bea3d640 1 -- 172.21.15.5:0/1264689865 >> 172.21.15.5:0/1264689865 conn(0x7fa1b80fc7a0 msgr2=0x7fa1b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.244+0000 7fa1bea3d640 1 -- 172.21.15.5:0/1264689865 shutdown_connections 2024-09-11T22:22:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.244+0000 7fa1bea3d640 1 -- 172.21.15.5:0/1264689865 wait complete. 2024-09-11T22:22:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.244+0000 7fa1bea3d640 1 Processor -- start 2024-09-11T22:22:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.244+0000 7fa1bea3d640 1 -- start start 2024-09-11T22:22:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1bea3d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1b8102d40 0x7fa1b81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1bea3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b8103660 0x7fa1b8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1bea3d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b810c170 0x7fa1b8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1bea3d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8111fd0 con 0x7fa1b8103660 2024-09-11T22:22:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1bea3d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8111e50 con 0x7fa1b8102d40 2024-09-11T22:22:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1bea3d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8112150 con 0x7fa1b810c170 2024-09-11T22:22:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b8103660 0x7fa1b8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b8103660 0x7fa1b8100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49472/0 (socket says 172.21.15.5:49472) 2024-09-11T22:22:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b77fe640 1 -- 172.21.15.5:0/2452622754 learned_addr learned my addr 172.21.15.5:0/2452622754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1bcfda640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b810c170 0x7fa1b8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b7fff640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1b8102d40 0x7fa1b81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b77fe640 1 -- 172.21.15.5:0/2452622754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b810c170 msgr2=0x7fa1b8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b77fe640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b810c170 0x7fa1b8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b77fe640 1 -- 172.21.15.5:0/2452622754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1b8102d40 msgr2=0x7fa1b81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b77fe640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1b8102d40 0x7fa1b81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b77fe640 1 -- 172.21.15.5:0/2452622754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b81a6970 con 0x7fa1b8103660 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b7fff640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1b8102d40 0x7fa1b81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b77fe640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b8103660 0x7fa1b8100ee0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00b3c0 tx=0x7fa1ac00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b57fa640 1 -- 172.21.15.5:0/2452622754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac011830 con 0x7fa1b8103660 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1b57fa640 1 -- 172.21.15.5:0/2452622754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1ac016070 con 0x7fa1b8103660 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.245+0000 7fa1bea3d640 1 -- 172.21.15.5:0/2452622754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b81a6c60 con 0x7fa1b8103660 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.246+0000 7fa1b57fa640 1 -- 172.21.15.5:0/2452622754 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac00f2e0 con 0x7fa1b8103660 2024-09-11T22:22:32.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.246+0000 7fa1bea3d640 1 -- 172.21.15.5:0/2452622754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b81a71a0 con 0x7fa1b8103660 2024-09-11T22:22:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.246+0000 7fa1bea3d640 1 -- 172.21.15.5:0/2452622754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b8103120 con 0x7fa1b8103660 2024-09-11T22:22:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.248+0000 7fa1b57fa640 1 -- 172.21.15.5:0/2452622754 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7fa1ac0119d0 con 0x7fa1b8103660 2024-09-11T22:22:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.248+0000 7fa1b57fa640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa1a00a2e90 0x7fa1a00a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.248+0000 7fa1b7fff640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa1a00a2e90 0x7fa1a00a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.248+0000 7fa1b57fa640 1 -- 172.21.15.5:0/2452622754 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa1ac0ceb30 con 0x7fa1b8103660 2024-09-11T22:22:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.249+0000 7fa1b7fff640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa1a00a2e90 0x7fa1a00a5350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8005ab0 tx=0x7fa1a8005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:32.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.252+0000 7fa1b57fa640 1 -- 172.21.15.5:0/2452622754 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1ac09b610 con 0x7fa1b8103660 2024-09-11T22:22:32.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.426+0000 7fa1bea3d640 1 -- 172.21.15.5:0/2452622754 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa1b8062630 con 0x7fa1a00a2e90 2024-09-11T22:22:32.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.432+0000 7fa1b57fa640 1 -- 172.21.15.5:0/2452622754 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258087 (secure 0 0 0) 0x7fa1b8062630 con 0x7fa1a00a2e90 2024-09-11T22:22:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:32.436 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-11T22:22:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 -- 172.21.15.5:0/2452622754 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa1a00a2e90 msgr2=0x7fa1a00a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa1a00a2e90 0x7fa1a00a5350 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8005ab0 tx=0x7fa1a8005a20 comp rx=0 tx=0).stop 2024-09-11T22:22:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 -- 172.21.15.5:0/2452622754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b8103660 msgr2=0x7fa1b8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b8103660 0x7fa1b8100ee0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00b3c0 tx=0x7fa1ac00b880 comp rx=0 tx=0).stop 2024-09-11T22:22:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 -- 172.21.15.5:0/2452622754 shutdown_connections 2024-09-11T22:22:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1b810c170 0x7fa1b8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b8103660 0x7fa1b8100ee0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa1a00a2e90 0x7fa1a00a5350 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 --2- 172.21.15.5:0/2452622754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa1b8102d40 0x7fa1b81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 -- 172.21.15.5:0/2452622754 >> 172.21.15.5:0/2452622754 conn(0x7fa1b80fc7a0 msgr2=0x7fa1b80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 -- 172.21.15.5:0/2452622754 shutdown_connections 2024-09-11T22:22:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:32.435+0000 7fa1bea3d640 1 -- 172.21.15.5:0/2452622754 wait complete. 2024-09-11T22:22:33.112 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-11T22:22:31.989505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391065,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1679,"num_read_kb":1266,"num_write":683,"num_write_kb":5325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1181928,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1021,"ondisk_log_size":1021,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385652,"kb_used_data":11412,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556172,"statfs":{"total":1151940427776,"available":1151545520128,"internally_reserved":0,"allocated":11685888,"data_stored":9085527,"data_compressed":298206,"data_compressed_allocated":393216,"data_compressed_original":3796992,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200331,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":385,"num_read_kb":430,"num_write":128,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002414"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235096+0000","last_change":"2024-09-11T22:20:18.189744+0000","last_active":"2024-09-11T22:22:16.235096+0000","last_peered":"2024-09-11T22:22:16.235096+0000","last_clean":"2024-09-11T22:22:16.235096+0000","last_became_active":"2024-09-11T22:20:18.189398+0000","last_became_peered":"2024-09-11T22:20:18.189398+0000","last_unstale":"2024-09-11T22:22:16.235096+0000","last_undegraded":"2024-09-11T22:22:16.235096+0000","last_fullsized":"2024-09-11T22:22:16.235096+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:10:24.152589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234185+0000","last_change":"2024-09-11T22:20:18.186727+0000","last_active":"2024-09-11T22:22:16.234185+0000","last_peered":"2024-09-11T22:22:16.234185+0000","last_clean":"2024-09-11T22:22:16.234185+0000","last_became_active":"2024-09-11T22:20:18.186414+0000","last_became_peered":"2024-09-11T22:20:18.186414+0000","last_unstale":"2024-09-11T22:22:16.234185+0000","last_undegraded":"2024-09-11T22:22:16.234185+0000","last_fullsized":"2024-09-11T22:22:16.234185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:51:53.589444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233909+0000","last_change":"2024-09-11T22:20:18.179984+0000","last_active":"2024-09-11T22:22:16.233909+0000","last_peered":"2024-09-11T22:22:16.233909+0000","last_clean":"2024-09-11T22:22:16.233909+0000","last_became_active":"2024-09-11T22:20:18.179329+0000","last_became_peered":"2024-09-11T22:20:18.179329+0000","last_unstale":"2024-09-11T22:22:16.233909+0000","last_undegraded":"2024-09-11T22:22:16.233909+0000","last_fullsized":"2024-09-11T22:22:16.233909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:19.149970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230253+0000","last_change":"2024-09-11T22:20:18.191768+0000","last_active":"2024-09-11T22:22:16.230253+0000","last_peered":"2024-09-11T22:22:16.230253+0000","last_clean":"2024-09-11T22:22:16.230253+0000","last_became_active":"2024-09-11T22:20:18.191336+0000","last_became_peered":"2024-09-11T22:20:18.191336+0000","last_unstale":"2024-09-11T22:22:16.230253+0000","last_undegraded":"2024-09-11T22:22:16.230253+0000","last_fullsized":"2024-09-11T22:22:16.230253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:44.768932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058858+0000","last_change":"2024-09-11T22:20:18.181469+0000","last_active":"2024-09-11T22:22:16.058858+0000","last_peered":"2024-09-11T22:22:16.058858+0000","last_clean":"2024-09-11T22:22:16.058858+0000","last_became_active":"2024-09-11T22:20:18.181100+0000","last_became_peered":"2024-09-11T22:20:18.181100+0000","last_unstale":"2024-09-11T22:22:16.058858+0000","last_undegraded":"2024-09-11T22:22:16.058858+0000","last_fullsized":"2024-09-11T22:22:16.058858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:49:14.414233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068675+0000","last_change":"2024-09-11T22:20:18.190855+0000","last_active":"2024-09-11T22:22:16.068675+0000","last_peered":"2024-09-11T22:22:16.068675+0000","last_clean":"2024-09-11T22:22:16.068675+0000","last_became_active":"2024-09-11T22:20:18.190523+0000","last_became_peered":"2024-09-11T22:20:18.190523+0000","last_unstale":"2024-09-11T22:22:16.068675+0000","last_undegraded":"2024-09-11T22:22:16.068675+0000","last_fullsized":"2024-09-11T22:22:16.068675+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:12:18.574957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231733+0000","last_change":"2024-09-11T22:20:18.199213+0000","last_active":"2024-09-11T22:22:16.231733+0000","last_peered":"2024-09-11T22:22:16.231733+0000","last_clean":"2024-09-11T22:22:16.231733+0000","last_became_active":"2024-09-11T22:20:18.198913+0000","last_became_peered":"2024-09-11T22:20:18.198913+0000","last_unstale":"2024-09-11T22:22:16.231733+0000","last_undegraded":"2024-09-11T22:22:16.231733+0000","last_fullsized":"2024-09-11T22:22:16.231733+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:22.682802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058633+0000","last_change":"2024-09-11T22:20:18.192458+0000","last_active":"2024-09-11T22:22:16.058633+0000","last_peered":"2024-09-11T22:22:16.058633+0000","last_clean":"2024-09-11T22:22:16.058633+0000","last_became_active":"2024-09-11T22:20:18.191967+0000","last_became_peered":"2024-09-11T22:20:18.191967+0000","last_unstale":"2024-09-11T22:22:16.058633+0000","last_undegraded":"2024-09-11T22:22:16.058633+0000","last_fullsized":"2024-09-11T22:22:16.058633+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:59:52.686157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975713+0000","last_change":"2024-09-11T22:21:18.191732+0000","last_active":"2024-09-11T22:22:15.975713+0000","last_peered":"2024-09-11T22:22:15.975713+0000","last_clean":"2024-09-11T22:22:15.975713+0000","last_became_active":"2024-09-11T22:21:18.191375+0000","last_became_peered":"2024-09-11T22:21:18.191375+0000","last_unstale":"2024-09-11T22:22:15.975713+0000","last_undegraded":"2024-09-11T22:22:15.975713+0000","last_fullsized":"2024-09-11T22:22:15.975713+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:13.645224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.069211+0000","last_change":"2024-09-11T22:20:18.193771+0000","last_active":"2024-09-11T22:22:16.069211+0000","last_peered":"2024-09-11T22:22:16.069211+0000","last_clean":"2024-09-11T22:22:16.069211+0000","last_became_active":"2024-09-11T22:20:18.193041+0000","last_became_peered":"2024-09-11T22:20:18.193041+0000","last_unstale":"2024-09-11T22:22:16.069211+0000","last_undegraded":"2024-09-11T22:22:16.069211+0000","last_fullsized":"2024-09-11T22:22:16.069211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:34:06.656169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231998+0000","last_change":"2024-09-11T22:20:18.197933+0000","last_active":"2024-09-11T22:22:16.231998+0000","last_peered":"2024-09-11T22:22:16.231998+0000","last_clean":"2024-09-11T22:22:16.231998+0000","last_became_active":"2024-09-11T22:20:18.197210+0000","last_became_peered":"2024-09-11T22:20:18.197210+0000","last_unstale":"2024-09-11T22:22:16.231998+0000","last_undegraded":"2024-09-11T22:22:16.231998+0000","last_fullsized":"2024-09-11T22:22:16.231998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:42.635909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230250+0000","last_change":"2024-09-11T22:20:18.190739+0000","last_active":"2024-09-11T22:22:16.230250+0000","last_peered":"2024-09-11T22:22:16.230250+0000","last_clean":"2024-09-11T22:22:16.230250+0000","last_became_active":"2024-09-11T22:20:18.190122+0000","last_became_peered":"2024-09-11T22:20:18.190122+0000","last_unstale":"2024-09-11T22:22:16.230250+0000","last_undegraded":"2024-09-11T22:22:16.230250+0000","last_fullsized":"2024-09-11T22:22:16.230250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:38:25.585035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230697+0000","last_change":"2024-09-11T22:20:18.190303+0000","last_active":"2024-09-11T22:22:16.230697+0000","last_peered":"2024-09-11T22:22:16.230697+0000","last_clean":"2024-09-11T22:22:16.230697+0000","last_became_active":"2024-09-11T22:20:18.189962+0000","last_became_peered":"2024-09-11T22:20:18.189962+0000","last_unstale":"2024-09-11T22:22:16.230697+0000","last_undegraded":"2024-09-11T22:22:16.230697+0000","last_fullsized":"2024-09-11T22:22:16.230697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:44.872813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233520+0000","last_change":"2024-09-11T22:20:18.185893+0000","last_active":"2024-09-11T22:22:16.233520+0000","last_peered":"2024-09-11T22:22:16.233520+0000","last_clean":"2024-09-11T22:22:16.233520+0000","last_became_active":"2024-09-11T22:20:18.185592+0000","last_became_peered":"2024-09-11T22:20:18.185592+0000","last_unstale":"2024-09-11T22:22:16.233520+0000","last_undegraded":"2024-09-11T22:22:16.233520+0000","last_fullsized":"2024-09-11T22:22:16.233520+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:38:14.725301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233777+0000","last_change":"2024-09-11T22:20:18.193943+0000","last_active":"2024-09-11T22:22:16.233777+0000","last_peered":"2024-09-11T22:22:16.233777+0000","last_clean":"2024-09-11T22:22:16.233777+0000","last_became_active":"2024-09-11T22:20:18.193558+0000","last_became_peered":"2024-09-11T22:20:18.193558+0000","last_unstale":"2024-09-11T22:22:16.233777+0000","last_undegraded":"2024-09-11T22:22:16.233777+0000","last_fullsized":"2024-09-11T22:22:16.233777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:08.736161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975254+0000","last_change":"2024-09-11T22:20:18.191815+0000","last_active":"2024-09-11T22:22:15.975254+0000","last_peered":"2024-09-11T22:22:15.975254+0000","last_clean":"2024-09-11T22:22:15.975254+0000","last_became_active":"2024-09-11T22:20:18.191483+0000","last_became_peered":"2024-09-11T22:20:18.191483+0000","last_unstale":"2024-09-11T22:22:15.975254+0000","last_undegraded":"2024-09-11T22:22:15.975254+0000","last_fullsized":"2024-09-11T22:22:15.975254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:48:26.432117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235101+0000","last_change":"2024-09-11T22:20:18.190758+0000","last_active":"2024-09-11T22:22:16.235101+0000","last_peered":"2024-09-11T22:22:16.235101+0000","last_clean":"2024-09-11T22:22:16.235101+0000","last_became_active":"2024-09-11T22:20:18.190431+0000","last_became_peered":"2024-09-11T22:20:18.190431+0000","last_unstale":"2024-09-11T22:22:16.235101+0000","last_undegraded":"2024-09-11T22:22:16.235101+0000","last_fullsized":"2024-09-11T22:22:16.235101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:31:10.927051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973552+0000","last_change":"2024-09-11T22:20:18.185191+0000","last_active":"2024-09-11T22:22:15.973552+0000","last_peered":"2024-09-11T22:22:15.973552+0000","last_clean":"2024-09-11T22:22:15.973552+0000","last_became_active":"2024-09-11T22:20:18.184906+0000","last_became_peered":"2024-09-11T22:20:18.184906+0000","last_unstale":"2024-09-11T22:22:15.973552+0000","last_undegraded":"2024-09-11T22:22:15.973552+0000","last_fullsized":"2024-09-11T22:22:15.973552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:24:32.294780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977515+0000","last_change":"2024-09-11T22:20:18.187137+0000","last_active":"2024-09-11T22:22:15.977515+0000","last_peered":"2024-09-11T22:22:15.977515+0000","last_clean":"2024-09-11T22:22:15.977515+0000","last_became_active":"2024-09-11T22:20:18.186770+0000","last_became_peered":"2024-09-11T22:20:18.186770+0000","last_unstale":"2024-09-11T22:22:15.977515+0000","last_undegraded":"2024-09-11T22:22:15.977515+0000","last_fullsized":"2024-09-11T22:22:15.977515+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:01:59.917695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234743+0000","last_change":"2024-09-11T22:20:18.197137+0000","last_active":"2024-09-11T22:22:16.234743+0000","last_peered":"2024-09-11T22:22:16.234743+0000","last_clean":"2024-09-11T22:22:16.234743+0000","last_became_active":"2024-09-11T22:20:18.192810+0000","last_became_peered":"2024-09-11T22:20:18.192810+0000","last_unstale":"2024-09-11T22:22:16.234743+0000","last_undegraded":"2024-09-11T22:22:16.234743+0000","last_fullsized":"2024-09-11T22:22:16.234743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:44:57.680726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230830+0000","last_change":"2024-09-11T22:20:18.187568+0000","last_active":"2024-09-11T22:22:16.230830+0000","last_peered":"2024-09-11T22:22:16.230830+0000","last_clean":"2024-09-11T22:22:16.230830+0000","last_became_active":"2024-09-11T22:20:18.187254+0000","last_became_peered":"2024-09-11T22:20:18.187254+0000","last_unstale":"2024-09-11T22:22:16.230830+0000","last_undegraded":"2024-09-11T22:22:16.230830+0000","last_fullsized":"2024-09-11T22:22:16.230830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:41.724926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975426+0000","last_change":"2024-09-11T22:20:18.190376+0000","last_active":"2024-09-11T22:22:15.975426+0000","last_peered":"2024-09-11T22:22:15.975426+0000","last_clean":"2024-09-11T22:22:15.975426+0000","last_became_active":"2024-09-11T22:20:18.190079+0000","last_became_peered":"2024-09-11T22:20:18.190079+0000","last_unstale":"2024-09-11T22:22:15.975426+0000","last_undegraded":"2024-09-11T22:22:15.975426+0000","last_fullsized":"2024-09-11T22:22:15.975426+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:42.924093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058944+0000","last_change":"2024-09-11T22:20:18.188503+0000","last_active":"2024-09-11T22:22:16.058944+0000","last_peered":"2024-09-11T22:22:16.058944+0000","last_clean":"2024-09-11T22:22:16.058944+0000","last_became_active":"2024-09-11T22:20:18.187957+0000","last_became_peered":"2024-09-11T22:20:18.187957+0000","last_unstale":"2024-09-11T22:22:16.058944+0000","last_undegraded":"2024-09-11T22:22:16.058944+0000","last_fullsized":"2024-09-11T22:22:16.058944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:44:25.382338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065572+0000","last_change":"2024-09-11T22:20:18.196936+0000","last_active":"2024-09-11T22:22:16.065572+0000","last_peered":"2024-09-11T22:22:16.065572+0000","last_clean":"2024-09-11T22:22:16.065572+0000","last_became_active":"2024-09-11T22:20:18.196493+0000","last_became_peered":"2024-09-11T22:20:18.196493+0000","last_unstale":"2024-09-11T22:22:16.065572+0000","last_undegraded":"2024-09-11T22:22:16.065572+0000","last_fullsized":"2024-09-11T22:22:16.065572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:06.189806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059171+0000","last_change":"2024-09-11T22:20:18.195260+0000","last_active":"2024-09-11T22:22:16.059171+0000","last_peered":"2024-09-11T22:22:16.059171+0000","last_clean":"2024-09-11T22:22:16.059171+0000","last_became_active":"2024-09-11T22:20:18.194956+0000","last_became_peered":"2024-09-11T22:20:18.194956+0000","last_unstale":"2024-09-11T22:22:16.059171+0000","last_undegraded":"2024-09-11T22:22:16.059171+0000","last_fullsized":"2024-09-11T22:22:16.059171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:09:34.483291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977741+0000","last_change":"2024-09-11T22:20:18.190591+0000","last_active":"2024-09-11T22:22:15.977741+0000","last_peered":"2024-09-11T22:22:15.977741+0000","last_clean":"2024-09-11T22:22:15.977741+0000","last_became_active":"2024-09-11T22:20:18.190007+0000","last_became_peered":"2024-09-11T22:20:18.190007+0000","last_unstale":"2024-09-11T22:22:15.977741+0000","last_undegraded":"2024-09-11T22:22:15.977741+0000","last_fullsized":"2024-09-11T22:22:15.977741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:46.281705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234929+0000","last_change":"2024-09-11T22:20:18.193377+0000","last_active":"2024-09-11T22:22:16.234929+0000","last_peered":"2024-09-11T22:22:16.234929+0000","last_clean":"2024-09-11T22:22:16.234929+0000","last_became_active":"2024-09-11T22:20:18.193003+0000","last_became_peered":"2024-09-11T22:20:18.193003+0000","last_unstale":"2024-09-11T22:22:16.234929+0000","last_undegraded":"2024-09-11T22:22:16.234929+0000","last_fullsized":"2024-09-11T22:22:16.234929+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:59.052966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230737+0000","last_change":"2024-09-11T22:20:18.189405+0000","last_active":"2024-09-11T22:22:16.230737+0000","last_peered":"2024-09-11T22:22:16.230737+0000","last_clean":"2024-09-11T22:22:16.230737+0000","last_became_active":"2024-09-11T22:20:18.189098+0000","last_became_peered":"2024-09-11T22:20:18.189098+0000","last_unstale":"2024-09-11T22:22:16.230737+0000","last_undegraded":"2024-09-11T22:22:16.230737+0000","last_fullsized":"2024-09-11T22:22:16.230737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:47.088192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.233900+0000","last_change":"2024-09-11T22:20:18.191005+0000","last_active":"2024-09-11T22:22:16.233900+0000","last_peered":"2024-09-11T22:22:16.233900+0000","last_clean":"2024-09-11T22:22:16.233900+0000","last_became_active":"2024-09-11T22:20:18.190681+0000","last_became_peered":"2024-09-11T22:20:18.190681+0000","last_unstale":"2024-09-11T22:22:16.233900+0000","last_undegraded":"2024-09-11T22:22:16.233900+0000","last_fullsized":"2024-09-11T22:22:16.233900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:07:12.903929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977265+0000","last_change":"2024-09-11T22:20:18.187920+0000","last_active":"2024-09-11T22:22:15.977265+0000","last_peered":"2024-09-11T22:22:15.977265+0000","last_clean":"2024-09-11T22:22:15.977265+0000","last_became_active":"2024-09-11T22:20:18.187526+0000","last_became_peered":"2024-09-11T22:20:18.187526+0000","last_unstale":"2024-09-11T22:22:15.977265+0000","last_undegraded":"2024-09-11T22:22:15.977265+0000","last_fullsized":"2024-09-11T22:22:15.977265+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:03.347857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977444+0000","last_change":"2024-09-11T22:20:18.193869+0000","last_active":"2024-09-11T22:22:15.977444+0000","last_peered":"2024-09-11T22:22:15.977444+0000","last_clean":"2024-09-11T22:22:15.977444+0000","last_became_active":"2024-09-11T22:20:18.193468+0000","last_became_peered":"2024-09-11T22:20:18.193468+0000","last_unstale":"2024-09-11T22:22:15.977444+0000","last_undegraded":"2024-09-11T22:22:15.977444+0000","last_fullsized":"2024-09-11T22:22:15.977444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:28:59.619611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234869+0000","last_change":"2024-09-11T22:20:18.193947+0000","last_active":"2024-09-11T22:22:16.234869+0000","last_peered":"2024-09-11T22:22:16.234869+0000","last_clean":"2024-09-11T22:22:16.234869+0000","last_became_active":"2024-09-11T22:20:18.192257+0000","last_became_peered":"2024-09-11T22:20:18.192257+0000","last_unstale":"2024-09-11T22:22:16.234869+0000","last_undegraded":"2024-09-11T22:22:16.234869+0000","last_fullsized":"2024-09-11T22:22:16.234869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.478730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058805+0000","last_change":"2024-09-11T22:20:18.184549+0000","last_active":"2024-09-11T22:22:16.058805+0000","last_peered":"2024-09-11T22:22:16.058805+0000","last_clean":"2024-09-11T22:22:16.058805+0000","last_became_active":"2024-09-11T22:20:18.184075+0000","last_became_peered":"2024-09-11T22:20:18.184075+0000","last_unstale":"2024-09-11T22:22:16.058805+0000","last_undegraded":"2024-09-11T22:22:16.058805+0000","last_fullsized":"2024-09-11T22:22:16.058805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:15:50.095643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'8","reported_seq":154,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.428358+0000","last_change":"2024-09-11T22:20:16.128524+0000","last_active":"2024-09-11T22:22:25.428358+0000","last_peered":"2024-09-11T22:22:25.428358+0000","last_clean":"2024-09-11T22:22:25.428358+0000","last_became_active":"2024-09-11T22:20:16.128185+0000","last_became_peered":"2024-09-11T22:20:16.128185+0000","last_unstale":"2024-09-11T22:22:25.428358+0000","last_undegraded":"2024-09-11T22:22:25.428358+0000","last_fullsized":"2024-09-11T22:22:25.428358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:02:50.022931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1716,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065720+0000","last_change":"2024-09-11T22:20:18.191578+0000","last_active":"2024-09-11T22:22:16.065720+0000","last_peered":"2024-09-11T22:22:16.065720+0000","last_clean":"2024-09-11T22:22:16.065720+0000","last_became_active":"2024-09-11T22:20:18.189568+0000","last_became_peered":"2024-09-11T22:20:18.189568+0000","last_unstale":"2024-09-11T22:22:16.065720+0000","last_undegraded":"2024-09-11T22:22:16.065720+0000","last_fullsized":"2024-09-11T22:22:16.065720+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:41.737242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068742+0000","last_change":"2024-09-11T22:21:18.199693+0000","last_active":"2024-09-11T22:22:16.068742+0000","last_peered":"2024-09-11T22:22:16.068742+0000","last_clean":"2024-09-11T22:22:16.068742+0000","last_became_active":"2024-09-11T22:21:18.190954+0000","last_became_peered":"2024-09-11T22:21:18.190954+0000","last_unstale":"2024-09-11T22:22:16.068742+0000","last_undegraded":"2024-09-11T22:22:16.068742+0000","last_fullsized":"2024-09-11T22:22:16.068742+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:42:11.979102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230688+0000","last_change":"2024-09-11T22:20:16.115602+0000","last_active":"2024-09-11T22:22:16.230688+0000","last_peered":"2024-09-11T22:22:16.230688+0000","last_clean":"2024-09-11T22:22:16.230688+0000","last_became_active":"2024-09-11T22:20:16.115280+0000","last_became_peered":"2024-09-11T22:20:16.115280+0000","last_unstale":"2024-09-11T22:22:16.230688+0000","last_undegraded":"2024-09-11T22:22:16.230688+0000","last_fullsized":"2024-09-11T22:22:16.230688+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:41.942137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058816+0000","last_change":"2024-09-11T22:20:18.184125+0000","last_active":"2024-09-11T22:22:16.058816+0000","last_peered":"2024-09-11T22:22:16.058816+0000","last_clean":"2024-09-11T22:22:16.058816+0000","last_became_active":"2024-09-11T22:20:18.183785+0000","last_became_peered":"2024-09-11T22:20:18.183785+0000","last_unstale":"2024-09-11T22:22:16.058816+0000","last_undegraded":"2024-09-11T22:22:16.058816+0000","last_fullsized":"2024-09-11T22:22:16.058816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:33.230175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065780+0000","last_change":"2024-09-11T22:20:16.125852+0000","last_active":"2024-09-11T22:22:16.065780+0000","last_peered":"2024-09-11T22:22:16.065780+0000","last_clean":"2024-09-11T22:22:16.065780+0000","last_became_active":"2024-09-11T22:20:16.125250+0000","last_became_peered":"2024-09-11T22:20:16.125250+0000","last_unstale":"2024-09-11T22:22:16.065780+0000","last_undegraded":"2024-09-11T22:22:16.065780+0000","last_fullsized":"2024-09-11T22:22:16.065780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:34.421192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973235+0000","last_change":"2024-09-11T22:20:18.185147+0000","last_active":"2024-09-11T22:22:15.973235+0000","last_peered":"2024-09-11T22:22:15.973235+0000","last_clean":"2024-09-11T22:22:15.973235+0000","last_became_active":"2024-09-11T22:20:18.184840+0000","last_became_peered":"2024-09-11T22:20:18.184840+0000","last_unstale":"2024-09-11T22:22:15.973235+0000","last_undegraded":"2024-09-11T22:22:15.973235+0000","last_fullsized":"2024-09-11T22:22:15.973235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:25:59.703349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":263,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.322621+0000","last_change":"2024-09-11T22:20:16.122173+0000","last_active":"2024-09-11T22:22:26.322621+0000","last_peered":"2024-09-11T22:22:26.322621+0000","last_clean":"2024-09-11T22:22:26.322621+0000","last_became_active":"2024-09-11T22:20:16.121847+0000","last_became_peered":"2024-09-11T22:20:16.121847+0000","last_unstale":"2024-09-11T22:22:26.322621+0000","last_undegraded":"2024-09-11T22:22:26.322621+0000","last_fullsized":"2024-09-11T22:22:26.322621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:26:41.284557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":233,"num_read_kb":233,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975691+0000","last_change":"2024-09-11T22:20:18.188030+0000","last_active":"2024-09-11T22:22:15.975691+0000","last_peered":"2024-09-11T22:22:15.975691+0000","last_clean":"2024-09-11T22:22:15.975691+0000","last_became_active":"2024-09-11T22:20:18.187503+0000","last_became_peered":"2024-09-11T22:20:18.187503+0000","last_unstale":"2024-09-11T22:22:15.975691+0000","last_undegraded":"2024-09-11T22:22:15.975691+0000","last_fullsized":"2024-09-11T22:22:15.975691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:56:45.494149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231598+0000","last_change":"2024-09-11T22:20:16.128828+0000","last_active":"2024-09-11T22:22:16.231598+0000","last_peered":"2024-09-11T22:22:16.231598+0000","last_clean":"2024-09-11T22:22:16.231598+0000","last_became_active":"2024-09-11T22:20:16.128454+0000","last_became_peered":"2024-09-11T22:20:16.128454+0000","last_unstale":"2024-09-11T22:22:16.231598+0000","last_undegraded":"2024-09-11T22:22:16.231598+0000","last_fullsized":"2024-09-11T22:22:16.231598+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:10:33.927564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234475+0000","last_change":"2024-09-11T22:20:18.193795+0000","last_active":"2024-09-11T22:22:16.234475+0000","last_peered":"2024-09-11T22:22:16.234475+0000","last_clean":"2024-09-11T22:22:16.234475+0000","last_became_active":"2024-09-11T22:20:18.193489+0000","last_became_peered":"2024-09-11T22:20:18.193489+0000","last_unstale":"2024-09-11T22:22:16.234475+0000","last_undegraded":"2024-09-11T22:22:16.234475+0000","last_fullsized":"2024-09-11T22:22:16.234475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:14.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065914+0000","last_change":"2024-09-11T22:20:16.187938+0000","last_active":"2024-09-11T22:22:16.065914+0000","last_peered":"2024-09-11T22:22:16.065914+0000","last_clean":"2024-09-11T22:22:16.065914+0000","last_became_active":"2024-09-11T22:20:16.187545+0000","last_became_peered":"2024-09-11T22:20:16.187545+0000","last_unstale":"2024-09-11T22:22:16.065914+0000","last_undegraded":"2024-09-11T22:22:16.065914+0000","last_fullsized":"2024-09-11T22:22:16.065914+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:41.674356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975385+0000","last_change":"2024-09-11T22:21:18.191937+0000","last_active":"2024-09-11T22:22:15.975385+0000","last_peered":"2024-09-11T22:22:15.975385+0000","last_clean":"2024-09-11T22:22:15.975385+0000","last_became_active":"2024-09-11T22:21:18.191218+0000","last_became_peered":"2024-09-11T22:21:18.191218+0000","last_unstale":"2024-09-11T22:22:15.975385+0000","last_undegraded":"2024-09-11T22:22:15.975385+0000","last_fullsized":"2024-09-11T22:22:15.975385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:52:41.654630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'39","reported_seq":312,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.428665+0000","last_change":"2024-09-11T22:20:16.128082+0000","last_active":"2024-09-11T22:22:26.428665+0000","last_peered":"2024-09-11T22:22:26.428665+0000","last_clean":"2024-09-11T22:22:26.428665+0000","last_became_active":"2024-09-11T22:20:16.127731+0000","last_became_peered":"2024-09-11T22:20:16.127731+0000","last_unstale":"2024-09-11T22:22:26.428665+0000","last_undegraded":"2024-09-11T22:22:26.428665+0000","last_fullsized":"2024-09-11T22:22:26.428665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:59:33.478437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69393,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":249,"num_read_kb":193,"num_write":41,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059064+0000","last_change":"2024-09-11T22:20:18.189219+0000","last_active":"2024-09-11T22:22:16.059064+0000","last_peered":"2024-09-11T22:22:16.059064+0000","last_clean":"2024-09-11T22:22:16.059064+0000","last_became_active":"2024-09-11T22:20:18.188687+0000","last_became_peered":"2024-09-11T22:20:18.188687+0000","last_unstale":"2024-09-11T22:22:16.059064+0000","last_undegraded":"2024-09-11T22:22:16.059064+0000","last_fullsized":"2024-09-11T22:22:16.059064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:09:24.171126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977378+0000","last_change":"2024-09-11T22:20:16.129724+0000","last_active":"2024-09-11T22:22:15.977378+0000","last_peered":"2024-09-11T22:22:15.977378+0000","last_clean":"2024-09-11T22:22:15.977378+0000","last_became_active":"2024-09-11T22:20:16.129351+0000","last_became_peered":"2024-09-11T22:20:16.129351+0000","last_unstale":"2024-09-11T22:22:15.977378+0000","last_undegraded":"2024-09-11T22:22:15.977378+0000","last_fullsized":"2024-09-11T22:22:15.977378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:32:09.642520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230435+0000","last_change":"2024-09-11T22:20:18.187031+0000","last_active":"2024-09-11T22:22:16.230435+0000","last_peered":"2024-09-11T22:22:16.230435+0000","last_clean":"2024-09-11T22:22:16.230435+0000","last_became_active":"2024-09-11T22:20:18.186698+0000","last_became_peered":"2024-09-11T22:20:18.186698+0000","last_unstale":"2024-09-11T22:22:16.230435+0000","last_undegraded":"2024-09-11T22:22:16.230435+0000","last_fullsized":"2024-09-11T22:22:16.230435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:13:04.810981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065725+0000","last_change":"2024-09-11T22:20:16.124925+0000","last_active":"2024-09-11T22:22:16.065725+0000","last_peered":"2024-09-11T22:22:16.065725+0000","last_clean":"2024-09-11T22:22:16.065725+0000","last_became_active":"2024-09-11T22:20:16.124580+0000","last_became_peered":"2024-09-11T22:20:16.124580+0000","last_unstale":"2024-09-11T22:22:16.065725+0000","last_undegraded":"2024-09-11T22:22:16.065725+0000","last_fullsized":"2024-09-11T22:22:16.065725+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:35:56.417577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231838+0000","last_change":"2024-09-11T22:21:18.187783+0000","last_active":"2024-09-11T22:22:16.231838+0000","last_peered":"2024-09-11T22:22:16.231838+0000","last_clean":"2024-09-11T22:22:16.231838+0000","last_became_active":"2024-09-11T22:21:18.187358+0000","last_became_peered":"2024-09-11T22:21:18.187358+0000","last_unstale":"2024-09-11T22:22:16.231838+0000","last_undegraded":"2024-09-11T22:22:16.231838+0000","last_fullsized":"2024-09-11T22:22:16.231838+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:43:41.251531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":151,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.452982+0000","last_change":"2024-09-11T22:20:16.118934+0000","last_active":"2024-09-11T22:22:25.452982+0000","last_peered":"2024-09-11T22:22:25.452982+0000","last_clean":"2024-09-11T22:22:25.452982+0000","last_became_active":"2024-09-11T22:20:16.118614+0000","last_became_peered":"2024-09-11T22:20:16.118614+0000","last_unstale":"2024-09-11T22:22:25.452982+0000","last_undegraded":"2024-09-11T22:22:25.452982+0000","last_fullsized":"2024-09-11T22:22:25.452982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:05.791161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1374,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231721+0000","last_change":"2024-09-11T22:20:18.197692+0000","last_active":"2024-09-11T22:22:16.231721+0000","last_peered":"2024-09-11T22:22:16.231721+0000","last_clean":"2024-09-11T22:22:16.231721+0000","last_became_active":"2024-09-11T22:20:18.197336+0000","last_became_peered":"2024-09-11T22:20:18.197336+0000","last_unstale":"2024-09-11T22:22:16.231721+0000","last_undegraded":"2024-09-11T22:22:16.231721+0000","last_fullsized":"2024-09-11T22:22:16.231721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:25:55.172568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:16.186769+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:16.186367+0000","last_became_peered":"2024-09-11T22:20:16.186367+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:45.279387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065641+0000","last_change":"2024-09-11T22:20:18.195570+0000","last_active":"2024-09-11T22:22:16.065641+0000","last_peered":"2024-09-11T22:22:16.065641+0000","last_clean":"2024-09-11T22:22:16.065641+0000","last_became_active":"2024-09-11T22:20:18.195226+0000","last_became_peered":"2024-09-11T22:20:18.195226+0000","last_unstale":"2024-09-11T22:22:16.065641+0000","last_undegraded":"2024-09-11T22:22:16.065641+0000","last_fullsized":"2024-09-11T22:22:16.065641+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:44.108090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'3","reported_seq":20,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.235034+0000","last_change":"2024-09-11T22:21:18.194553+0000","last_active":"2024-09-11T22:22:16.235034+0000","last_peered":"2024-09-11T22:22:16.235034+0000","last_clean":"2024-09-11T22:22:16.235034+0000","last_became_active":"2024-09-11T22:21:18.188635+0000","last_became_peered":"2024-09-11T22:21:18.188635+0000","last_unstale":"2024-09-11T22:22:16.235034+0000","last_undegraded":"2024-09-11T22:22:16.235034+0000","last_fullsized":"2024-09-11T22:22:16.235034+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:11:34.165195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234954+0000","last_change":"2024-09-11T22:20:18.194112+0000","last_active":"2024-09-11T22:22:16.234954+0000","last_peered":"2024-09-11T22:22:16.234954+0000","last_clean":"2024-09-11T22:22:16.234954+0000","last_became_active":"2024-09-11T22:20:18.193812+0000","last_became_peered":"2024-09-11T22:20:18.193812+0000","last_unstale":"2024-09-11T22:22:16.234954+0000","last_undegraded":"2024-09-11T22:22:16.234954+0000","last_fullsized":"2024-09-11T22:22:16.234954+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:40:35.211611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231070+0000","last_change":"2024-09-11T22:21:18.198325+0000","last_active":"2024-09-11T22:22:16.231070+0000","last_peered":"2024-09-11T22:22:16.231070+0000","last_clean":"2024-09-11T22:22:16.231070+0000","last_became_active":"2024-09-11T22:21:18.193651+0000","last_became_peered":"2024-09-11T22:21:18.193651+0000","last_unstale":"2024-09-11T22:22:16.231070+0000","last_undegraded":"2024-09-11T22:22:16.231070+0000","last_fullsized":"2024-09-11T22:22:16.231070+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:21:42.552492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232027+0000","last_change":"2024-09-11T22:20:18.191960+0000","last_active":"2024-09-11T22:22:16.232027+0000","last_peered":"2024-09-11T22:22:16.232027+0000","last_clean":"2024-09-11T22:22:16.232027+0000","last_became_active":"2024-09-11T22:20:18.191635+0000","last_became_peered":"2024-09-11T22:20:18.191635+0000","last_unstale":"2024-09-11T22:22:16.232027+0000","last_undegraded":"2024-09-11T22:22:16.232027+0000","last_fullsized":"2024-09-11T22:22:16.232027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:19:18.919247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"83'67","reported_seq":115,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.988172+0000","last_change":"2024-09-11T22:20:16.123608+0000","last_active":"2024-09-11T22:22:26.988172+0000","last_peered":"2024-09-11T22:22:26.988172+0000","last_clean":"2024-09-11T22:22:26.988172+0000","last_became_active":"2024-09-11T22:20:16.123193+0000","last_became_peered":"2024-09-11T22:20:16.123193+0000","last_unstale":"2024-09-11T22:22:26.988172+0000","last_undegraded":"2024-09-11T22:22:26.988172+0000","last_fullsized":"2024-09-11T22:22:26.988172+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:07.362115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":26,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231887+0000","last_change":"2024-09-11T22:20:18.199957+0000","last_active":"2024-09-11T22:22:16.231887+0000","last_peered":"2024-09-11T22:22:16.231887+0000","last_clean":"2024-09-11T22:22:16.231887+0000","last_became_active":"2024-09-11T22:20:18.199637+0000","last_became_peered":"2024-09-11T22:20:18.199637+0000","last_unstale":"2024-09-11T22:22:16.231887+0000","last_undegraded":"2024-09-11T22:22:16.231887+0000","last_fullsized":"2024-09-11T22:22:16.231887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:12:14.949002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":94,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.321741+0000","last_change":"2024-09-11T22:21:18.198896+0000","last_active":"2024-09-11T22:22:25.321741+0000","last_peered":"2024-09-11T22:22:25.321741+0000","last_clean":"2024-09-11T22:22:25.321741+0000","last_became_active":"2024-09-11T22:21:18.189489+0000","last_became_peered":"2024-09-11T22:21:18.189489+0000","last_unstale":"2024-09-11T22:22:25.321741+0000","last_undegraded":"2024-09-11T22:22:25.321741+0000","last_fullsized":"2024-09-11T22:22:25.321741+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:45:19.908801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1956,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.232111+0000","last_change":"2024-09-11T22:20:18.196691+0000","last_active":"2024-09-11T22:22:16.232111+0000","last_peered":"2024-09-11T22:22:16.232111+0000","last_clean":"2024-09-11T22:22:16.232111+0000","last_became_active":"2024-09-11T22:20:18.196364+0000","last_became_peered":"2024-09-11T22:20:18.196364+0000","last_unstale":"2024-09-11T22:22:16.232111+0000","last_undegraded":"2024-09-11T22:22:16.232111+0000","last_fullsized":"2024-09-11T22:22:16.232111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:16:50.721807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"83'148","reported_seq":392,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:18.074584+0000","last_change":"2024-09-11T22:19:02.590079+0000","last_active":"2024-09-11T22:22:18.074584+0000","last_peered":"2024-09-11T22:22:18.074584+0000","last_clean":"2024-09-11T22:22:18.074584+0000","last_became_active":"2024-09-11T22:19:02.589644+0000","last_became_peered":"2024-09-11T22:19:02.589644+0000","last_unstale":"2024-09-11T22:22:18.074584+0000","last_undegraded":"2024-09-11T22:22:18.074584+0000","last_fullsized":"2024-09-11T22:22:18.074584+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:15:33.244685+0000","last_clean_scrub_stamp":"2024-09-11T22:15:33.244685+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:45.506190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.229817+0000","last_change":"2024-09-11T22:20:16.116661+0000","last_active":"2024-09-11T22:22:16.229817+0000","last_peered":"2024-09-11T22:22:16.229817+0000","last_clean":"2024-09-11T22:22:16.229817+0000","last_became_active":"2024-09-11T22:20:16.116196+0000","last_became_peered":"2024-09-11T22:20:16.116196+0000","last_unstale":"2024-09-11T22:22:16.229817+0000","last_undegraded":"2024-09-11T22:22:16.229817+0000","last_fullsized":"2024-09-11T22:22:16.229817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:08.875170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059213+0000","last_change":"2024-09-11T22:20:18.193176+0000","last_active":"2024-09-11T22:22:16.059213+0000","last_peered":"2024-09-11T22:22:16.059213+0000","last_clean":"2024-09-11T22:22:16.059213+0000","last_became_active":"2024-09-11T22:20:18.192312+0000","last_became_peered":"2024-09-11T22:20:18.192312+0000","last_unstale":"2024-09-11T22:22:16.059213+0000","last_undegraded":"2024-09-11T22:22:16.059213+0000","last_fullsized":"2024-09-11T22:22:16.059213+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:42:08.738569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234694+0000","last_change":"2024-09-11T22:21:18.196985+0000","last_active":"2024-09-11T22:22:16.234694+0000","last_peered":"2024-09-11T22:22:16.234694+0000","last_clean":"2024-09-11T22:22:16.234694+0000","last_became_active":"2024-09-11T22:21:18.189487+0000","last_became_peered":"2024-09-11T22:21:18.189487+0000","last_unstale":"2024-09-11T22:22:16.234694+0000","last_undegraded":"2024-09-11T22:22:16.234694+0000","last_fullsized":"2024-09-11T22:22:16.234694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:42:20.715696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230517+0000","last_change":"2024-09-11T22:20:18.191118+0000","last_active":"2024-09-11T22:22:16.230517+0000","last_peered":"2024-09-11T22:22:16.230517+0000","last_clean":"2024-09-11T22:22:16.230517+0000","last_became_active":"2024-09-11T22:20:18.190760+0000","last_became_peered":"2024-09-11T22:20:18.190760+0000","last_unstale":"2024-09-11T22:22:16.230517+0000","last_undegraded":"2024-09-11T22:22:16.230517+0000","last_fullsized":"2024-09-11T22:22:16.230517+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:59.220067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":149,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.452301+0000","last_change":"2024-09-11T22:20:16.186407+0000","last_active":"2024-09-11T22:22:25.452301+0000","last_peered":"2024-09-11T22:22:25.452301+0000","last_clean":"2024-09-11T22:22:25.452301+0000","last_became_active":"2024-09-11T22:20:16.186105+0000","last_became_peered":"2024-09-11T22:20:16.186105+0000","last_unstale":"2024-09-11T22:22:25.452301+0000","last_undegraded":"2024-09-11T22:22:25.452301+0000","last_fullsized":"2024-09-11T22:22:25.452301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:43.942072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065681+0000","last_change":"2024-09-11T22:20:18.191970+0000","last_active":"2024-09-11T22:22:16.065681+0000","last_peered":"2024-09-11T22:22:16.065681+0000","last_clean":"2024-09-11T22:22:16.065681+0000","last_became_active":"2024-09-11T22:20:18.191623+0000","last_became_peered":"2024-09-11T22:20:18.191623+0000","last_unstale":"2024-09-11T22:22:16.065681+0000","last_undegraded":"2024-09-11T22:22:16.065681+0000","last_fullsized":"2024-09-11T22:22:16.065681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:22:33.213979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'7","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.441040+0000","last_change":"2024-09-11T22:20:16.119936+0000","last_active":"2024-09-11T22:22:25.441040+0000","last_peered":"2024-09-11T22:22:25.441040+0000","last_clean":"2024-09-11T22:22:25.441040+0000","last_became_active":"2024-09-11T22:20:16.119453+0000","last_became_peered":"2024-09-11T22:20:16.119453+0000","last_unstale":"2024-09-11T22:22:25.441040+0000","last_undegraded":"2024-09-11T22:22:25.441040+0000","last_fullsized":"2024-09-11T22:22:25.441040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:26:20.717980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068855+0000","last_change":"2024-09-11T22:20:18.192581+0000","last_active":"2024-09-11T22:22:16.068855+0000","last_peered":"2024-09-11T22:22:16.068855+0000","last_clean":"2024-09-11T22:22:16.068855+0000","last_became_active":"2024-09-11T22:20:18.191702+0000","last_became_peered":"2024-09-11T22:20:18.191702+0000","last_unstale":"2024-09-11T22:22:16.068855+0000","last_undegraded":"2024-09-11T22:22:16.068855+0000","last_fullsized":"2024-09-11T22:22:16.068855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:42:48.194270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973445+0000","last_change":"2024-09-11T22:20:16.186485+0000","last_active":"2024-09-11T22:22:15.973445+0000","last_peered":"2024-09-11T22:22:15.973445+0000","last_clean":"2024-09-11T22:22:15.973445+0000","last_became_active":"2024-09-11T22:20:16.186171+0000","last_became_peered":"2024-09-11T22:20:16.186171+0000","last_unstale":"2024-09-11T22:22:15.973445+0000","last_undegraded":"2024-09-11T22:22:15.973445+0000","last_fullsized":"2024-09-11T22:22:15.973445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:26.373626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.068932+0000","last_change":"2024-09-11T22:20:18.191361+0000","last_active":"2024-09-11T22:22:16.068932+0000","last_peered":"2024-09-11T22:22:16.068932+0000","last_clean":"2024-09-11T22:22:16.068932+0000","last_became_active":"2024-09-11T22:20:18.191036+0000","last_became_peered":"2024-09-11T22:20:18.191036+0000","last_unstale":"2024-09-11T22:22:16.068932+0000","last_undegraded":"2024-09-11T22:22:16.068932+0000","last_fullsized":"2024-09-11T22:22:16.068932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:36:29.124650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059209+0000","last_change":"2024-09-11T22:20:16.188156+0000","last_active":"2024-09-11T22:22:16.059209+0000","last_peered":"2024-09-11T22:22:16.059209+0000","last_clean":"2024-09-11T22:22:16.059209+0000","last_became_active":"2024-09-11T22:20:16.187809+0000","last_became_peered":"2024-09-11T22:20:16.187809+0000","last_unstale":"2024-09-11T22:22:16.059209+0000","last_undegraded":"2024-09-11T22:22:16.059209+0000","last_fullsized":"2024-09-11T22:22:16.059209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:50:04.219621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230944+0000","last_change":"2024-09-11T22:20:18.182380+0000","last_active":"2024-09-11T22:22:16.230944+0000","last_peered":"2024-09-11T22:22:16.230944+0000","last_clean":"2024-09-11T22:22:16.230944+0000","last_became_active":"2024-09-11T22:20:18.182062+0000","last_became_peered":"2024-09-11T22:20:18.182062+0000","last_unstale":"2024-09-11T22:22:16.230944+0000","last_undegraded":"2024-09-11T22:22:16.230944+0000","last_fullsized":"2024-09-11T22:22:16.230944+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:57:15.575846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'91","reported_seq":187,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:26.997407+0000","last_change":"2024-09-11T22:20:16.117335+0000","last_active":"2024-09-11T22:22:26.997407+0000","last_peered":"2024-09-11T22:22:26.997407+0000","last_clean":"2024-09-11T22:22:26.997407+0000","last_became_active":"2024-09-11T22:20:16.116977+0000","last_became_peered":"2024-09-11T22:20:16.116977+0000","last_unstale":"2024-09-11T22:22:26.997407+0000","last_undegraded":"2024-09-11T22:22:26.997407+0000","last_fullsized":"2024-09-11T22:22:26.997407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:10:51.744202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":244,"num_read_kb":215,"num_write":164,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059281+0000","last_change":"2024-09-11T22:20:18.194069+0000","last_active":"2024-09-11T22:22:16.059281+0000","last_peered":"2024-09-11T22:22:16.059281+0000","last_clean":"2024-09-11T22:22:16.059281+0000","last_became_active":"2024-09-11T22:20:18.193765+0000","last_became_peered":"2024-09-11T22:20:18.193765+0000","last_unstale":"2024-09-11T22:22:16.059281+0000","last_undegraded":"2024-09-11T22:22:16.059281+0000","last_fullsized":"2024-09-11T22:22:16.059281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:58:25.047405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.230178+0000","last_change":"2024-09-11T22:20:16.460987+0000","last_active":"2024-09-11T22:22:16.230178+0000","last_peered":"2024-09-11T22:22:16.230178+0000","last_clean":"2024-09-11T22:22:16.230178+0000","last_became_active":"2024-09-11T22:20:16.460516+0000","last_became_peered":"2024-09-11T22:20:16.460516+0000","last_unstale":"2024-09-11T22:22:16.230178+0000","last_undegraded":"2024-09-11T22:22:16.230178+0000","last_fullsized":"2024-09-11T22:22:16.230178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:00.987981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234312+0000","last_change":"2024-09-11T22:21:18.188802+0000","last_active":"2024-09-11T22:22:16.234312+0000","last_peered":"2024-09-11T22:22:16.234312+0000","last_clean":"2024-09-11T22:22:16.234312+0000","last_became_active":"2024-09-11T22:21:18.188441+0000","last_became_peered":"2024-09-11T22:21:18.188441+0000","last_unstale":"2024-09-11T22:22:16.234312+0000","last_undegraded":"2024-09-11T22:22:16.234312+0000","last_fullsized":"2024-09-11T22:22:16.234312+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:02.295441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975385+0000","last_change":"2024-09-11T22:20:16.186498+0000","last_active":"2024-09-11T22:22:15.975385+0000","last_peered":"2024-09-11T22:22:15.975385+0000","last_clean":"2024-09-11T22:22:15.975385+0000","last_became_active":"2024-09-11T22:20:16.186214+0000","last_became_peered":"2024-09-11T22:20:16.186214+0000","last_unstale":"2024-09-11T22:22:15.975385+0000","last_undegraded":"2024-09-11T22:22:15.975385+0000","last_fullsized":"2024-09-11T22:22:15.975385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:09.222264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058771+0000","last_change":"2024-09-11T22:20:18.187489+0000","last_active":"2024-09-11T22:22:16.058771+0000","last_peered":"2024-09-11T22:22:16.058771+0000","last_clean":"2024-09-11T22:22:16.058771+0000","last_became_active":"2024-09-11T22:20:18.185649+0000","last_became_peered":"2024-09-11T22:20:18.185649+0000","last_unstale":"2024-09-11T22:22:16.058771+0000","last_undegraded":"2024-09-11T22:22:16.058771+0000","last_fullsized":"2024-09-11T22:22:16.058771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T01:17:42.825075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059064+0000","last_change":"2024-09-11T22:20:16.117627+0000","last_active":"2024-09-11T22:22:16.059064+0000","last_peered":"2024-09-11T22:22:16.059064+0000","last_clean":"2024-09-11T22:22:16.059064+0000","last_became_active":"2024-09-11T22:20:16.117243+0000","last_became_peered":"2024-09-11T22:20:16.117243+0000","last_unstale":"2024-09-11T22:22:16.059064+0000","last_undegraded":"2024-09-11T22:22:16.059064+0000","last_fullsized":"2024-09-11T22:22:16.059064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:02:58.240535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234846+0000","last_change":"2024-09-11T22:20:18.184326+0000","last_active":"2024-09-11T22:22:16.234846+0000","last_peered":"2024-09-11T22:22:16.234846+0000","last_clean":"2024-09-11T22:22:16.234846+0000","last_became_active":"2024-09-11T22:20:18.183995+0000","last_became_peered":"2024-09-11T22:20:18.183995+0000","last_unstale":"2024-09-11T22:22:16.234846+0000","last_undegraded":"2024-09-11T22:22:16.234846+0000","last_fullsized":"2024-09-11T22:22:16.234846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:33:33.468000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'4","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.977314+0000","last_change":"2024-09-11T22:20:16.127180+0000","last_active":"2024-09-11T22:22:15.977314+0000","last_peered":"2024-09-11T22:22:15.977314+0000","last_clean":"2024-09-11T22:22:15.977314+0000","last_became_active":"2024-09-11T22:20:16.126696+0000","last_became_peered":"2024-09-11T22:20:16.126696+0000","last_unstale":"2024-09-11T22:22:15.977314+0000","last_undegraded":"2024-09-11T22:22:15.977314+0000","last_fullsized":"2024-09-11T22:22:15.977314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:02.637225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.973369+0000","last_change":"2024-09-11T22:21:18.188614+0000","last_active":"2024-09-11T22:22:15.973369+0000","last_peered":"2024-09-11T22:22:15.973369+0000","last_clean":"2024-09-11T22:22:15.973369+0000","last_became_active":"2024-09-11T22:21:18.188277+0000","last_became_peered":"2024-09-11T22:21:18.188277+0000","last_unstale":"2024-09-11T22:22:15.973369+0000","last_undegraded":"2024-09-11T22:22:15.973369+0000","last_fullsized":"2024-09-11T22:22:15.973369+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:59.995647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.058897+0000","last_change":"2024-09-11T22:20:16.128311+0000","last_active":"2024-09-11T22:22:16.058897+0000","last_peered":"2024-09-11T22:22:16.058897+0000","last_clean":"2024-09-11T22:22:16.058897+0000","last_became_active":"2024-09-11T22:20:16.127853+0000","last_became_peered":"2024-09-11T22:20:16.127853+0000","last_unstale":"2024-09-11T22:22:16.058897+0000","last_undegraded":"2024-09-11T22:22:16.058897+0000","last_fullsized":"2024-09-11T22:22:16.058897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:58:57.562470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.231820+0000","last_change":"2024-09-11T22:20:18.195782+0000","last_active":"2024-09-11T22:22:16.231820+0000","last_peered":"2024-09-11T22:22:16.231820+0000","last_clean":"2024-09-11T22:22:16.231820+0000","last_became_active":"2024-09-11T22:20:18.195365+0000","last_became_peered":"2024-09-11T22:20:18.195365+0000","last_unstale":"2024-09-11T22:22:16.231820+0000","last_undegraded":"2024-09-11T22:22:16.231820+0000","last_fullsized":"2024-09-11T22:22:16.231820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:19:49.352544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.065812+0000","last_change":"2024-09-11T22:20:18.199344+0000","last_active":"2024-09-11T22:22:16.065812+0000","last_peered":"2024-09-11T22:22:16.065812+0000","last_clean":"2024-09-11T22:22:16.065812+0000","last_became_active":"2024-09-11T22:20:18.199016+0000","last_became_peered":"2024-09-11T22:20:18.199016+0000","last_unstale":"2024-09-11T22:22:16.065812+0000","last_undegraded":"2024-09-11T22:22:16.065812+0000","last_fullsized":"2024-09-11T22:22:16.065812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:38.322831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059125+0000","last_change":"2024-09-11T22:20:16.122236+0000","last_active":"2024-09-11T22:22:16.059125+0000","last_peered":"2024-09-11T22:22:16.059125+0000","last_clean":"2024-09-11T22:22:16.059125+0000","last_became_active":"2024-09-11T22:20:16.121867+0000","last_became_peered":"2024-09-11T22:20:16.121867+0000","last_unstale":"2024-09-11T22:22:16.059125+0000","last_undegraded":"2024-09-11T22:22:16.059125+0000","last_fullsized":"2024-09-11T22:22:16.059125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:35:56.782445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:15.975374+0000","last_change":"2024-09-11T22:20:18.187338+0000","last_active":"2024-09-11T22:22:15.975374+0000","last_peered":"2024-09-11T22:22:15.975374+0000","last_clean":"2024-09-11T22:22:15.975374+0000","last_became_active":"2024-09-11T22:20:18.186985+0000","last_became_peered":"2024-09-11T22:20:18.186985+0000","last_unstale":"2024-09-11T22:22:15.975374+0000","last_undegraded":"2024-09-11T22:22:15.975374+0000","last_fullsized":"2024-09-11T22:22:15.975374+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:58.170680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.059106+0000","last_change":"2024-09-11T22:20:16.128579+0000","last_active":"2024-09-11T22:22:16.059106+0000","last_peered":"2024-09-11T22:22:16.059106+0000","last_clean":"2024-09-11T22:22:16.059106+0000","last_became_active":"2024-09-11T22:20:16.127864+0000","last_became_peered":"2024-09-11T22:20:16.127864+0000","last_unstale":"2024-09-11T22:22:16.059106+0000","last_undegraded":"2024-09-11T22:22:16.059106+0000","last_fullsized":"2024-09-11T22:22:16.059106+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T02:11:32.005487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.234497+0000","last_change":"2024-09-11T22:20:18.185205+0000","last_active":"2024-09-11T22:22:16.234497+0000","last_peered":"2024-09-11T22:22:16.234497+0000","last_clean":"2024-09-11T22:22:16.234497+0000","last_became_active":"2024-09-11T22:20:18.184402+0000","last_became_peered":"2024-09-11T22:20:18.184402+0000","last_unstale":"2024-09-11T22:22:16.234497+0000","last_undegraded":"2024-09-11T22:22:16.234497+0000","last_fullsized":"2024-09-11T22:22:16.234497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:17:48.792719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.060250+0000","last_change":"2024-09-11T22:20:16.186372+0000","last_active":"2024-09-11T22:22:16.060250+0000","last_peered":"2024-09-11T22:22:16.060250+0000","last_clean":"2024-09-11T22:22:16.060250+0000","last_became_active":"2024-09-11T22:20:16.185990+0000","last_became_peered":"2024-09-11T22:20:16.185990+0000","last_unstale":"2024-09-11T22:22:16.060250+0000","last_undegraded":"2024-09-11T22:22:16.060250+0000","last_fullsized":"2024-09-11T22:22:16.060250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:19:23.296379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:16.060015+0000","last_change":"2024-09-11T22:20:18.190670+0000","last_active":"2024-09-11T22:22:16.060015+0000","last_peered":"2024-09-11T22:22:16.060015+0000","last_clean":"2024-09-11T22:22:16.060015+0000","last_became_active":"2024-09-11T22:20:18.190339+0000","last_became_peered":"2024-09-11T22:20:18.190339+0000","last_unstale":"2024-09-11T22:22:16.060015+0000","last_undegraded":"2024-09-11T22:22:16.060015+0000","last_fullsized":"2024-09-11T22:22:16.060015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:17.154911+0000","last_clean_scrub_stamp":"2024-09-11T22:20:17.154911+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:31:58.085305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":158,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-11T22:22:25.928844+0000","last_change":"2024-09-11T22:20:16.117646+0000","last_active":"2024-09-11T22:22:25.928844+0000","last_peered":"2024-09-11T22:22:25.928844+0000","last_clean":"2024-09-11T22:22:25.928844+0000","last_became_active":"2024-09-11T22:20:16.117111+0000","last_became_peered":"2024-09-11T22:20:16.117111+0000","last_unstale":"2024-09-11T22:22:25.928844+0000","last_undegraded":"2024-09-11T22:22:25.928844+0000","last_fullsized":"2024-09-11T22:22:25.928844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T22:20:15.101510+0000","last_clean_scrub_stamp":"2024-09-11T22:20:15.101510+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:34:36.470158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800463,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1442,"num_read_kb":936,"num_write":480,"num_write_kb":2385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1192,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798720,"data_stored":2401389,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108858,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579943,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":913408,"data_stored":444133,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":1236133,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":958464,"data_stored":836471,"data_compressed":27823,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201334,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":836827,"data_compressed":27952,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430268,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":1038174,"data_compressed":44963,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593792,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":917504,"data_stored":443506,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822726,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":1168132,"data_compressed":51119,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986251,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":445575,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215185,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1003520,"data_stored":1164245,"data_compressed":43168,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":513422,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":987136,"data_stored":513812,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40424,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704728,"statfs":{"total":95995035648,"available":95953641472,"internally_reserved":0,"allocated":937984,"data_stored":445097,"data_compressed":8677,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36286,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332146,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":167936,"data_stored":400147,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T22:22:33.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:22:33.426 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:33 smithi005 ceph-mon[30943]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 42 op/s 2024-09-11T22:22:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:33 smithi005 ceph-mon[30943]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:34.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.987+0000 7f0cc24ba640 1 -- 172.21.15.5:0/1208071083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc10c170 msgr2=0x7f0cbc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:34.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.987+0000 7f0cc24ba640 1 --2- 172.21.15.5:0/1208071083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc10c170 0x7f0cbc10e610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009ca0 tx=0x7f0cb402a950 comp rx=0 tx=0).stop 2024-09-11T22:22:34.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.987+0000 7f0cc24ba640 1 -- 172.21.15.5:0/1208071083 shutdown_connections 2024-09-11T22:22:34.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.987+0000 7f0cc24ba640 1 --2- 172.21.15.5:0/1208071083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc10c170 0x7f0cbc10e610 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:34.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.987+0000 7f0cc24ba640 1 --2- 172.21.15.5:0/1208071083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cbc103660 0x7f0cbc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:34.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.987+0000 7f0cc24ba640 1 --2- 172.21.15.5:0/1208071083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cbc102d40 0x7f0cbc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:34.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.987+0000 7f0cc24ba640 1 -- 172.21.15.5:0/1208071083 >> 172.21.15.5:0/1208071083 conn(0x7f0cbc0fc7a0 msgr2=0x7f0cbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:34.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.988+0000 7f0cc24ba640 1 -- 172.21.15.5:0/1208071083 shutdown_connections 2024-09-11T22:22:34.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.988+0000 7f0cc24ba640 1 -- 172.21.15.5:0/1208071083 wait complete. 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.988+0000 7f0cc24ba640 1 Processor -- start 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.988+0000 7f0cc24ba640 1 -- start start 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.988+0000 7f0cc24ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cbc102d40 0x7f0cbc198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.988+0000 7f0cc24ba640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cbc103660 0x7f0cbc198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.988+0000 7f0cc24ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc10c170 0x7f0cbc19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.988+0000 7f0cc24ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc111f70 con 0x7f0cbc10c170 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.988+0000 7f0cc24ba640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc111df0 con 0x7f0cbc103660 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.988+0000 7f0cc24ba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0cbc1120f0 con 0x7f0cbc102d40 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cc0a57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc10c170 0x7f0cbc19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cbbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cbc102d40 0x7f0cbc198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cc0a57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc10c170 0x7f0cbc19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49488/0 (socket says 172.21.15.5:49488) 2024-09-11T22:22:34.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cc0a57640 1 -- 172.21.15.5:0/845050041 learned_addr learned my addr 172.21.15.5:0/845050041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cb3fff640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cbc103660 0x7f0cbc198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cc0a57640 1 -- 172.21.15.5:0/845050041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cbc102d40 msgr2=0x7f0cbc198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cc0a57640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cbc102d40 0x7f0cbc198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cc0a57640 1 -- 172.21.15.5:0/845050041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cbc103660 msgr2=0x7f0cbc198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cc0a57640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cbc103660 0x7f0cbc198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cc0a57640 1 -- 172.21.15.5:0/845050041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cbc19d2e0 con 0x7f0cbc10c170 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.989+0000 7f0cc0a57640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc10c170 0x7f0cbc19cc00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0cb402ae70 tx=0x7f0cb402f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.990+0000 7f0cb9ffb640 1 -- 172.21.15.5:0/845050041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4034040 con 0x7f0cbc10c170 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.990+0000 7f0cc24ba640 1 -- 172.21.15.5:0/845050041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cbc19d570 con 0x7f0cbc10c170 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.990+0000 7f0cb9ffb640 1 -- 172.21.15.5:0/845050041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cb4030920 con 0x7f0cbc10c170 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.990+0000 7f0cb9ffb640 1 -- 172.21.15.5:0/845050041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4045480 con 0x7f0cbc10c170 2024-09-11T22:22:34.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.990+0000 7f0cc24ba640 1 -- 172.21.15.5:0/845050041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cbc1a4eb0 con 0x7f0cbc10c170 2024-09-11T22:22:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.991+0000 7f0cc24ba640 1 -- 172.21.15.5:0/845050041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cbc103120 con 0x7f0cbc10c170 2024-09-11T22:22:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.992+0000 7f0cb9ffb640 1 -- 172.21.15.5:0/845050041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f0cb4037020 con 0x7f0cbc10c170 2024-09-11T22:22:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.992+0000 7f0cb9ffb640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ca80a2e10 0x7f0ca80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.993+0000 7f0cb9ffb640 1 -- 172.21.15.5:0/845050041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f0cb40f04a0 con 0x7f0cbc10c170 2024-09-11T22:22:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.993+0000 7f0cbbfff640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ca80a2e10 0x7f0ca80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.993+0000 7f0cbbfff640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ca80a2e10 0x7f0ca80a52d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00bbe0 tx=0x7f0cac00b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:33.997+0000 7f0cb9ffb640 1 -- 172.21.15.5:0/845050041 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cb40bcf80 con 0x7f0cbc10c170 2024-09-11T22:22:34.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:33 smithi159 ceph-mon[35534]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 42 op/s 2024-09-11T22:22:34.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:33 smithi159 ceph-mon[35534]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:34.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:33 smithi172 ceph-mon[35461]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 77 KiB/s wr, 42 op/s 2024-09-11T22:22:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:33 smithi172 ceph-mon[35461]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T22:22:34.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.211+0000 7f0cc24ba640 1 -- 172.21.15.5:0/845050041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cbc1a50e0 con 0x7f0cbc10c170 2024-09-11T22:22:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.211+0000 7f0cb9ffb640 1 -- 172.21.15.5:0/845050041 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f0cb40bc880 con 0x7f0cbc10c170 2024-09-11T22:22:34.212 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:34.213 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-11T22:22:15:230183+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:22:15.180031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:22:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-11T22:22:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.213+0000 7f0cc24ba640 1 -- 172.21.15.5:0/845050041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ca80a2e10 msgr2=0x7f0ca80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.213+0000 7f0cc24ba640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ca80a2e10 0x7f0ca80a52d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00bbe0 tx=0x7f0cac00b390 comp rx=0 tx=0).stop 2024-09-11T22:22:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.214+0000 7f0cc24ba640 1 -- 172.21.15.5:0/845050041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc10c170 msgr2=0x7f0cbc19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.214+0000 7f0cc24ba640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc10c170 0x7f0cbc19cc00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0cb402ae70 tx=0x7f0cb402f5b0 comp rx=0 tx=0).stop 2024-09-11T22:22:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.214+0000 7f0cc24ba640 1 -- 172.21.15.5:0/845050041 shutdown_connections 2024-09-11T22:22:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.214+0000 7f0cc24ba640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cbc10c170 0x7f0cbc19cc00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.214+0000 7f0cc24ba640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ca80a2e10 0x7f0ca80a52d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.214+0000 7f0cc24ba640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0cbc103660 0x7f0cbc198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.214+0000 7f0cc24ba640 1 --2- 172.21.15.5:0/845050041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0cbc102d40 0x7f0cbc198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.214+0000 7f0cc24ba640 1 -- 172.21.15.5:0/845050041 >> 172.21.15.5:0/845050041 conn(0x7f0cbc0fc7a0 msgr2=0x7f0cbc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.214+0000 7f0cc24ba640 1 -- 172.21.15.5:0/845050041 shutdown_connections 2024-09-11T22:22:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:34.214+0000 7f0cc24ba640 1 -- 172.21.15.5:0/845050041 wait complete. 2024-09-11T22:22:34.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:34 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/845050041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:34.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:22:35.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:34 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/845050041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:35.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:34 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/845050041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:35.164 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:35.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.647+0000 7ff005329640 1 -- 172.21.15.5:0/1171217432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff000106f20 msgr2=0x7ff00010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:35.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.647+0000 7ff005329640 1 --2- 172.21.15.5:0/1171217432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff000106f20 0x7ff00010eaa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feff8009ca0 tx=0x7feff802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:22:35.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.647+0000 7ff005329640 1 -- 172.21.15.5:0/1171217432 shutdown_connections 2024-09-11T22:22:35.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.647+0000 7ff005329640 1 --2- 172.21.15.5:0/1171217432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff000106f20 0x7ff00010eaa0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:35.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.647+0000 7ff005329640 1 --2- 172.21.15.5:0/1171217432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0001027a0 0x7ff0001067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:35.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.647+0000 7ff005329640 1 --2- 172.21.15.5:0/1171217432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff000101df0 0x7ff0001021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:35.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.647+0000 7ff005329640 1 -- 172.21.15.5:0/1171217432 >> 172.21.15.5:0/1171217432 conn(0x7ff0000fd8b0 msgr2=0x7ff0000ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.647+0000 7ff005329640 1 -- 172.21.15.5:0/1171217432 shutdown_connections 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.647+0000 7ff005329640 1 -- 172.21.15.5:0/1171217432 wait complete. 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7ff005329640 1 Processor -- start 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7ff005329640 1 -- start start 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7ff005329640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff000101df0 0x7ff0001983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7ff005329640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0001027a0 0x7ff000198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7ff005329640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff000106f20 0x7ff00019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7ff005329640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff000111fa0 con 0x7ff000101df0 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7ff005329640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff000111e20 con 0x7ff000106f20 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7ff005329640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff000112120 con 0x7ff0001027a0 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7feffeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff000101df0 0x7ff0001983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7feffeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff000101df0 0x7ff0001983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49508/0 (socket says 172.21.15.5:49508) 2024-09-11T22:22:35.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7feffeffd640 1 -- 172.21.15.5:0/3244302764 learned_addr learned my addr 172.21.15.5:0/3244302764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7fefff7fe640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff000106f20 0x7ff00019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7feffeffd640 1 -- 172.21.15.5:0/3244302764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0001027a0 msgr2=0x7ff000198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7feffe7fc640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0001027a0 0x7ff000198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7feffeffd640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0001027a0 0x7ff000198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7feffeffd640 1 -- 172.21.15.5:0/3244302764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff000106f20 msgr2=0x7ff00019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7feffeffd640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff000106f20 0x7ff00019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7feffeffd640 1 -- 172.21.15.5:0/3244302764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00019d3a0 con 0x7ff000101df0 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.648+0000 7feffe7fc640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0001027a0 0x7ff000198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.649+0000 7feffeffd640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff000101df0 0x7ff0001983f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feff000d950 tx=0x7feff000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.649+0000 7fefe7fff640 1 -- 172.21.15.5:0/3244302764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0017070 con 0x7ff000101df0 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.649+0000 7ff005329640 1 -- 172.21.15.5:0/3244302764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00019d690 con 0x7ff000101df0 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.649+0000 7fefe7fff640 1 -- 172.21.15.5:0/3244302764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feff0009c30 con 0x7ff000101df0 2024-09-11T22:22:35.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.649+0000 7fefe7fff640 1 -- 172.21.15.5:0/3244302764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff000f6b0 con 0x7ff000101df0 2024-09-11T22:22:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.649+0000 7ff005329640 1 -- 172.21.15.5:0/3244302764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0001a4f70 con 0x7ff000101df0 2024-09-11T22:22:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.650+0000 7ff005329640 1 -- 172.21.15.5:0/3244302764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0001021d0 con 0x7ff000101df0 2024-09-11T22:22:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.651+0000 7fefe7fff640 1 -- 172.21.15.5:0/3244302764 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7feff00109d0 con 0x7ff000101df0 2024-09-11T22:22:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.652+0000 7fefe7fff640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fefe80a2e10 0x7fefe80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.652+0000 7feffe7fc640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fefe80a2e10 0x7fefe80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.652+0000 7fefe7fff640 1 -- 172.21.15.5:0/3244302764 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7feff00cf370 con 0x7ff000101df0 2024-09-11T22:22:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.652+0000 7feffe7fc640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fefe80a2e10 0x7fefe80a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feff4006fd0 tx=0x7feff4008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:35.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.656+0000 7fefe7fff640 1 -- 172.21.15.5:0/3244302764 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feff0009de0 con 0x7ff000101df0 2024-09-11T22:22:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:35 smithi005 ceph-mon[30943]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-11T22:22:35.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.869+0000 7ff005329640 1 -- 172.21.15.5:0/3244302764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff00010a190 con 0x7ff000101df0 2024-09-11T22:22:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.869+0000 7fefe7fff640 1 -- 172.21.15.5:0/3244302764 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7feff0097030 con 0x7ff000101df0 2024-09-11T22:22:35.870 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:35.870 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-11T22:22:15:230183+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:22:15.180031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:22:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-11T22:22:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.871+0000 7ff005329640 1 -- 172.21.15.5:0/3244302764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fefe80a2e10 msgr2=0x7fefe80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.871+0000 7ff005329640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fefe80a2e10 0x7fefe80a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7feff4006fd0 tx=0x7feff4008040 comp rx=0 tx=0).stop 2024-09-11T22:22:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.871+0000 7ff005329640 1 -- 172.21.15.5:0/3244302764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff000101df0 msgr2=0x7ff0001983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.871+0000 7ff005329640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff000101df0 0x7ff0001983f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feff000d950 tx=0x7feff000de10 comp rx=0 tx=0).stop 2024-09-11T22:22:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.872+0000 7ff005329640 1 -- 172.21.15.5:0/3244302764 shutdown_connections 2024-09-11T22:22:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.872+0000 7ff005329640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fefe80a2e10 0x7fefe80a52d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.872+0000 7ff005329640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff000106f20 0x7ff00019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.872+0000 7ff005329640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0001027a0 0x7ff000198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.872+0000 7ff005329640 1 --2- 172.21.15.5:0/3244302764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff000101df0 0x7ff0001983f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.872+0000 7ff005329640 1 -- 172.21.15.5:0/3244302764 >> 172.21.15.5:0/3244302764 conn(0x7ff0000fd8b0 msgr2=0x7ff0000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.872+0000 7ff005329640 1 -- 172.21.15.5:0/3244302764 shutdown_connections 2024-09-11T22:22:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:35.872+0000 7ff005329640 1 -- 172.21.15.5:0/3244302764 wait complete. 2024-09-11T22:22:36.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:35 smithi159 ceph-mon[35534]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-11T22:22:36.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:35 smithi172 ceph-mon[35461]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 55 op/s 2024-09-11T22:22:36.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:22:36.880 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:37.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:36 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3244302764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:37.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:36 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3244302764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:37.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:36 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3244302764' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:37.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.390+0000 7fab3929b640 1 -- 172.21.15.5:0/62002047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3410c180 msgr2=0x7fab3410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.390+0000 7fab3929b640 1 --2- 172.21.15.5:0/62002047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3410c180 0x7fab3410e600 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009ca0 tx=0x7fab2c02a950 comp rx=0 tx=0).stop 2024-09-11T22:22:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.390+0000 7fab3929b640 1 -- 172.21.15.5:0/62002047 shutdown_connections 2024-09-11T22:22:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.390+0000 7fab3929b640 1 --2- 172.21.15.5:0/62002047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab3410c180 0x7fab3410e600 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.390+0000 7fab3929b640 1 --2- 172.21.15.5:0/62002047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab341036c0 0x7fab3410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.390+0000 7fab3929b640 1 --2- 172.21.15.5:0/62002047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab34102da0 0x7fab34103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.390+0000 7fab3929b640 1 -- 172.21.15.5:0/62002047 >> 172.21.15.5:0/62002047 conn(0x7fab340fc7a0 msgr2=0x7fab340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.390+0000 7fab3929b640 1 -- 172.21.15.5:0/62002047 shutdown_connections 2024-09-11T22:22:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.390+0000 7fab3929b640 1 -- 172.21.15.5:0/62002047 wait complete. 2024-09-11T22:22:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab3929b640 1 Processor -- start 2024-09-11T22:22:37.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab3929b640 1 -- start start 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab3929b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab34102da0 0x7fab341983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab3929b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341036c0 0x7fab34198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab3929b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab3410c180 0x7fab3419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab3929b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111f40 con 0x7fab341036c0 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab3929b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111dc0 con 0x7fab34102da0 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab3929b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fab341120c0 con 0x7fab3410c180 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab327fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341036c0 0x7fab34198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab32ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab34102da0 0x7fab341983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab327fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341036c0 0x7fab34198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49532/0 (socket says 172.21.15.5:49532) 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab327fc640 1 -- 172.21.15.5:0/1071990216 learned_addr learned my addr 172.21.15.5:0/1071990216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.391+0000 7fab337fe640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab3410c180 0x7fab3419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab327fc640 1 -- 172.21.15.5:0/1071990216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab3410c180 msgr2=0x7fab3419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:37.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab327fc640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab3410c180 0x7fab3419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab327fc640 1 -- 172.21.15.5:0/1071990216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab34102da0 msgr2=0x7fab341983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab327fc640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab34102da0 0x7fab341983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab327fc640 1 -- 172.21.15.5:0/1071990216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab3419d380 con 0x7fab341036c0 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab32ffd640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab34102da0 0x7fab341983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab327fc640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341036c0 0x7fab34198910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fab2400d770 tx=0x7fab2400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab2bfff640 1 -- 172.21.15.5:0/1071990216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24017070 con 0x7fab341036c0 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab3929b640 1 -- 172.21.15.5:0/1071990216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab3419d670 con 0x7fab341036c0 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab2bfff640 1 -- 172.21.15.5:0/1071990216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab24005500 con 0x7fab341036c0 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab2bfff640 1 -- 172.21.15.5:0/1071990216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24004e00 con 0x7fab341036c0 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.392+0000 7fab3929b640 1 -- 172.21.15.5:0/1071990216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a4f50 con 0x7fab341036c0 2024-09-11T22:22:37.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.393+0000 7fab3929b640 1 -- 172.21.15.5:0/1071990216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab34103180 con 0x7fab341036c0 2024-09-11T22:22:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.394+0000 7fab2bfff640 1 -- 172.21.15.5:0/1071990216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7fab24004fa0 con 0x7fab341036c0 2024-09-11T22:22:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.395+0000 7fab2bfff640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fab200a2e10 0x7fab200a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.395+0000 7fab2bfff640 1 -- 172.21.15.5:0/1071990216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fab240cf0d0 con 0x7fab341036c0 2024-09-11T22:22:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.395+0000 7fab32ffd640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fab200a2e10 0x7fab200a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.396+0000 7fab32ffd640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fab200a2e10 0x7fab200a52d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00bfd0 tx=0x7fab1c00b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:37.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.399+0000 7fab2bfff640 1 -- 172.21.15.5:0/1071990216 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab2409b3f0 con 0x7fab341036c0 2024-09-11T22:22:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.577+0000 7fab3929b640 1 -- 172.21.15.5:0/1071990216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab34107220 con 0x7fab341036c0 2024-09-11T22:22:37.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.578+0000 7fab2bfff640 1 -- 172.21.15.5:0/1071990216 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+23068 (secure 0 0 0) 0x7fab2409f6f0 con 0x7fab341036c0 2024-09-11T22:22:37.579 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:37.579 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:22:15.968337+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2380674497":"2024-09-12T22:22:15.968309+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/3591749770":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/1577283141":"2024-09-12T22:22:15.968309+0000","172.21.15.5:6801/3591749770":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1618823752":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/785116270":"2024-09-12T22:22:15.968309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:22:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.580+0000 7fab3929b640 1 -- 172.21.15.5:0/1071990216 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fab200a2e10 msgr2=0x7fab200a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.580+0000 7fab3929b640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fab200a2e10 0x7fab200a52d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00bfd0 tx=0x7fab1c00b390 comp rx=0 tx=0).stop 2024-09-11T22:22:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.580+0000 7fab3929b640 1 -- 172.21.15.5:0/1071990216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341036c0 msgr2=0x7fab34198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.580+0000 7fab3929b640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341036c0 0x7fab34198910 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fab2400d770 tx=0x7fab2400dc30 comp rx=0 tx=0).stop 2024-09-11T22:22:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.580+0000 7fab3929b640 1 -- 172.21.15.5:0/1071990216 shutdown_connections 2024-09-11T22:22:37.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.580+0000 7fab3929b640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab3410c180 0x7fab3419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.580+0000 7fab3929b640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab341036c0 0x7fab34198910 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.580+0000 7fab3929b640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fab200a2e10 0x7fab200a52d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.580+0000 7fab3929b640 1 --2- 172.21.15.5:0/1071990216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab34102da0 0x7fab341983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.580+0000 7fab3929b640 1 -- 172.21.15.5:0/1071990216 >> 172.21.15.5:0/1071990216 conn(0x7fab340fc7a0 msgr2=0x7fab340fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.581+0000 7fab3929b640 1 -- 172.21.15.5:0/1071990216 shutdown_connections 2024-09-11T22:22:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:37.581+0000 7fab3929b640 1 -- 172.21.15.5:0/1071990216 wait complete. 2024-09-11T22:22:37.947 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:37 smithi005 ceph-mon[30943]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T22:22:37.947 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:37 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1071990216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:37 smithi159 ceph-mon[35534]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T22:22:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:37 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1071990216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:37 smithi172 ceph-mon[35461]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 40 op/s 2024-09-11T22:22:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:37 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1071990216' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:22:38.284 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-11T22:22:38.540 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:38.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:38 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:22:39.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:38 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:22:39.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:38 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:22:39.764 INFO:teuthology.orchestra.run.smithi005.stdout:{"run_dir":"/var/run/ceph"} 2024-09-11T22:22:39.764 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-11T22:22:39.764 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-11T22:22:39.764 DEBUG:tasks.quiescer:Yielding 2024-09-11T22:22:39.765 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T22:22:39.780 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T22:22:39.780 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T22:22:39.781 INFO:tasks.workunit:timeout=3h 2024-09-11T22:22:39.781 INFO:tasks.workunit:cleanup=True 2024-09-11T22:22:39.781 DEBUG:teuthology.orchestra.run.smithi005:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T22:22:39.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:22:39.799 INFO:teuthology.orchestra.run.smithi005.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-11T22:22:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-11T22:22:39.800 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 60h/96d Inode: 1099511628283 Links: 2 2024-09-11T22:22:39.800 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T22:22:39.800 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-11T22:22:39.800 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-11 22:20:37.187912806 +0000 2024-09-11T22:22:39.800 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-11 22:20:37.187912806 +0000 2024-09-11T22:22:39.800 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-11 22:22:14.490049363 +0000 2024-09-11T22:22:39.800 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-11 22:20:37.187912806 +0000 2024-09-11T22:22:39.800 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-11T22:22:39.800 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-11T22:22:39.878 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-11T22:22:39.934 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T22:22:40.050 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:40.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:39 smithi005 ceph-mon[30943]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-11T22:22:40.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:39 smithi159 ceph-mon[35534]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-11T22:22:40.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:39 smithi172 ceph-mon[35461]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 76 KiB/s wr, 44 op/s 2024-09-11T22:22:40.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.570+0000 7f6d499fd640 1 -- 172.21.15.5:0/3643864120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d440747f0 msgr2=0x7f6d44074bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:40.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.570+0000 7f6d499fd640 1 --2- 172.21.15.5:0/3643864120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d440747f0 0x7f6d44074bd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009900 tx=0x7f6d3402a410 comp rx=0 tx=0).stop 2024-09-11T22:22:40.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.570+0000 7f6d499fd640 1 -- 172.21.15.5:0/3643864120 shutdown_connections 2024-09-11T22:22:40.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.570+0000 7f6d499fd640 1 --2- 172.21.15.5:0/3643864120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d4410e480 0x7f6d441108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.570+0000 7f6d499fd640 1 --2- 172.21.15.5:0/3643864120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d44075110 0x7f6d4410deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.571+0000 7f6d499fd640 1 --2- 172.21.15.5:0/3643864120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d440747f0 0x7f6d44074bd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.571+0000 7f6d499fd640 1 -- 172.21.15.5:0/3643864120 >> 172.21.15.5:0/3643864120 conn(0x7f6d440fd8d0 msgr2=0x7f6d440ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:40.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.571+0000 7f6d499fd640 1 -- 172.21.15.5:0/3643864120 shutdown_connections 2024-09-11T22:22:40.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.571+0000 7f6d499fd640 1 -- 172.21.15.5:0/3643864120 wait complete. 2024-09-11T22:22:40.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.571+0000 7f6d499fd640 1 Processor -- start 2024-09-11T22:22:40.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.571+0000 7f6d499fd640 1 -- start start 2024-09-11T22:22:40.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d499fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d440747f0 0x7f6d4419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d499fd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d44075110 0x7f6d4419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d499fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4410e480 0x7f6d4419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d499fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44114260 con 0x7f6d4410e480 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d499fd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441140e0 con 0x7f6d44075110 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d499fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441143e0 con 0x7f6d440747f0 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4410e480 0x7f6d4419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4410e480 0x7f6d4419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49556/0 (socket says 172.21.15.5:49556) 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d437fe640 1 -- 172.21.15.5:0/2872981077 learned_addr learned my addr 172.21.15.5:0/2872981077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d42ffd640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d440747f0 0x7f6d4419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d427fc640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d44075110 0x7f6d4419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d437fe640 1 -- 172.21.15.5:0/2872981077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d440747f0 msgr2=0x7f6d4419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:40.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d437fe640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d440747f0 0x7f6d4419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d437fe640 1 -- 172.21.15.5:0/2872981077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d44075110 msgr2=0x7f6d4419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d437fe640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d44075110 0x7f6d4419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d437fe640 1 -- 172.21.15.5:0/2872981077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4419f590 con 0x7f6d4410e480 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.572+0000 7f6d427fc640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d44075110 0x7f6d4419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.573+0000 7f6d437fe640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4410e480 0x7f6d4419eeb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00ccb0 tx=0x7f6d3c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.573+0000 7f6d489fb640 1 -- 172.21.15.5:0/2872981077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c00f450 con 0x7f6d4410e480 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.573+0000 7f6d489fb640 1 -- 172.21.15.5:0/2872981077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d3c00fad0 con 0x7f6d4410e480 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.573+0000 7f6d499fd640 1 -- 172.21.15.5:0/2872981077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4419f880 con 0x7f6d4410e480 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.573+0000 7f6d489fb640 1 -- 172.21.15.5:0/2872981077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c018bf0 con 0x7f6d4410e480 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.573+0000 7f6d499fd640 1 -- 172.21.15.5:0/2872981077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d441a7160 con 0x7f6d4410e480 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.574+0000 7f6d499fd640 1 -- 172.21.15.5:0/2872981077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d44074bd0 con 0x7f6d4410e480 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.575+0000 7f6d489fb640 1 -- 172.21.15.5:0/2872981077 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f6d3c00f5f0 con 0x7f6d4410e480 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.576+0000 7f6d489fb640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d2c0a2e10 0x7f6d2c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.576+0000 7f6d42ffd640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d2c0a2e10 0x7f6d2c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.576+0000 7f6d489fb640 1 -- 172.21.15.5:0/2872981077 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f6d3c0cfea0 con 0x7f6d4410e480 2024-09-11T22:22:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.577+0000 7f6d42ffd640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d2c0a2e10 0x7f6d2c0a52d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d34000c00 tx=0x7f6d34005b50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:40.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.580+0000 7f6d489fb640 1 -- 172.21.15.5:0/2872981077 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d3c014030 con 0x7f6d4410e480 2024-09-11T22:22:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.791+0000 7f6d499fd640 1 -- 172.21.15.5:0/2872981077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d4419b940 con 0x7f6d4410e480 2024-09-11T22:22:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.791+0000 7f6d489fb640 1 -- 172.21.15.5:0/2872981077 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f6d4419b940 con 0x7f6d4410e480 2024-09-11T22:22:40.794 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:22:40.795 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-11T22:22:15:230183+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:22:15.180031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:22:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-11T22:22:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 -- 172.21.15.5:0/2872981077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d2c0a2e10 msgr2=0x7f6d2c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d2c0a2e10 0x7f6d2c0a52d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d34000c00 tx=0x7f6d34005b50 comp rx=0 tx=0).stop 2024-09-11T22:22:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 -- 172.21.15.5:0/2872981077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4410e480 msgr2=0x7f6d4419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4410e480 0x7f6d4419eeb0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c00ccb0 tx=0x7f6d3c00a620 comp rx=0 tx=0).stop 2024-09-11T22:22:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 -- 172.21.15.5:0/2872981077 shutdown_connections 2024-09-11T22:22:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d4410e480 0x7f6d4419eeb0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d2c0a2e10 0x7f6d2c0a52d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d44075110 0x7f6d4419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 --2- 172.21.15.5:0/2872981077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d440747f0 0x7f6d4419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 -- 172.21.15.5:0/2872981077 >> 172.21.15.5:0/2872981077 conn(0x7f6d440fd8d0 msgr2=0x7f6d440fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 -- 172.21.15.5:0/2872981077 shutdown_connections 2024-09-11T22:22:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:40.794+0000 7f6d499fd640 1 -- 172.21.15.5:0/2872981077 wait complete. 2024-09-11T22:22:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.977+0000 7f231a92c640 1 -- 172.21.15.5:0/3664442923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2314102480 msgr2=0x7f231410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.977+0000 7f231a92c640 1 --2- 172.21.15.5:0/3664442923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2314102480 0x7f231410e9b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f230c009cd0 tx=0x7f230c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.977+0000 7f231a92c640 1 -- 172.21.15.5:0/3664442923 shutdown_connections 2024-09-11T22:22:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.977+0000 7f231a92c640 1 --2- 172.21.15.5:0/3664442923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2314102480 0x7f231410e9b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.977+0000 7f231a92c640 1 --2- 172.21.15.5:0/3664442923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2314101ae0 0x7f2314101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.977+0000 7f231a92c640 1 --2- 172.21.15.5:0/3664442923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314107b80 0x7f2314107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.977+0000 7f231a92c640 1 -- 172.21.15.5:0/3664442923 >> 172.21.15.5:0/3664442923 conn(0x7f23140fd7e0 msgr2=0x7f23140ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.977+0000 7f231a92c640 1 -- 172.21.15.5:0/3664442923 shutdown_connections 2024-09-11T22:22:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.977+0000 7f231a92c640 1 -- 172.21.15.5:0/3664442923 wait complete. 2024-09-11T22:22:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f231a92c640 1 Processor -- start 2024-09-11T22:22:40.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f231a92c640 1 -- start start 2024-09-11T22:22:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f231a92c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314101ae0 0x7f2314198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f231a92c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2314102480 0x7f2314198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f231a92c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2314107b80 0x7f231419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f231a92c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2314111f30 con 0x7f2314107b80 2024-09-11T22:22:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f231a92c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2314111db0 con 0x7f2314102480 2024-09-11T22:22:40.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f231a92c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23141120b0 con 0x7f2314101ae0 2024-09-11T22:22:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f2318ec9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2314107b80 0x7f231419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f2318ec9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2314107b80 0x7f231419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49580/0 (socket says 172.21.15.5:49580) 2024-09-11T22:22:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.978+0000 7f2318ec9640 1 -- 172.21.15.5:0/2072520158 learned_addr learned my addr 172.21.15.5:0/2072520158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f23137fe640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2314102480 0x7f2314198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f2313fff640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314101ae0 0x7f2314198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f2318ec9640 1 -- 172.21.15.5:0/2072520158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314101ae0 msgr2=0x7f2314198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f2318ec9640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314101ae0 0x7f2314198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f2318ec9640 1 -- 172.21.15.5:0/2072520158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2314102480 msgr2=0x7f2314198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:40.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f2318ec9640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2314102480 0x7f2314198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f2318ec9640 1 -- 172.21.15.5:0/2072520158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231419d330 con 0x7f2314107b80 2024-09-11T22:22:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f2318ec9640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2314107b80 0x7f231419cc00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f230c005cc0 tx=0x7f230c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f23117fa640 1 -- 172.21.15.5:0/2072520158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c004040 con 0x7f2314107b80 2024-09-11T22:22:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f23117fa640 1 -- 172.21.15.5:0/2072520158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f230c0046c0 con 0x7f2314107b80 2024-09-11T22:22:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f231a92c640 1 -- 172.21.15.5:0/2072520158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231419d5c0 con 0x7f2314107b80 2024-09-11T22:22:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f23117fa640 1 -- 172.21.15.5:0/2072520158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c046480 con 0x7f2314107b80 2024-09-11T22:22:40.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.979+0000 7f231a92c640 1 -- 172.21.15.5:0/2072520158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23141a4eb0 con 0x7f2314107b80 2024-09-11T22:22:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f231d6c1740 1 Processor -- start 2024-09-11T22:22:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f23117fa640 1 -- 172.21.15.5:0/2072520158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f230c0305a0 con 0x7f2314107b80 2024-09-11T22:22:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f231d6c1740 1 -- start start 2024-09-11T22:22:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b8af0 0x5606b92b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 0x5606b92c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f231d6c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5606b921d2f0 con 0x5606b92b9410 2024-09-11T22:22:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f231d6c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5606b91c5270 con 0x5606b92c3cb0 2024-09-11T22:22:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f231d6c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5606b92192b0 con 0x5606b92b8af0 2024-09-11T22:22:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f2313fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f2313fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49588/0 (socket says 172.21.15.5:49588) 2024-09-11T22:22:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f2313fff640 1 -- 172.21.15.5:0/4186546527 learned_addr learned my addr 172.21.15.5:0/4186546527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f23137fe640 1 --2- 172.21.15.5:0/4186546527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b8af0 0x5606b92b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f23137fe640 1 --2- 172.21.15.5:0/4186546527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 0x5606b92c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f2313fff640 1 -- 172.21.15.5:0/4186546527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b8af0 msgr2=0x5606b92b8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f2313fff640 1 --2- 172.21.15.5:0/4186546527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b8af0 0x5606b92b8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f2313fff640 1 -- 172.21.15.5:0/4186546527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 msgr2=0x5606b92c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f23117fa640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f22fc0a2f40 0x7f22fc0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f2313fff640 1 --2- 172.21.15.5:0/4186546527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 0x5606b92c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.982+0000 7f2313fff640 1 -- 172.21.15.5:0/4186546527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606b91c4340 con 0x5606b92b9410 2024-09-11T22:22:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.983+0000 7f2318ec9640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f22fc0a2f40 0x7f22fc0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.983+0000 7f2313fff640 1 --2- 172.21.15.5:0/4186546527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c3770 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f230400b710 tx=0x7f2304004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79de0fdf5c51ba5b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.983+0000 7f23117fa640 1 -- 172.21.15.5:0/2072520158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f230c02e030 con 0x7f2314107b80 2024-09-11T22:22:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.983+0000 7f22fa7fc640 1 -- 172.21.15.5:0/4186546527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304003bd0 con 0x5606b92b9410 2024-09-11T22:22:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.983+0000 7f22fa7fc640 1 -- 172.21.15.5:0/4186546527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2304015070 con 0x5606b92b9410 2024-09-11T22:22:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.983+0000 7f22fa7fc640 1 -- 172.21.15.5:0/4186546527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304004e00 con 0x5606b92b9410 2024-09-11T22:22:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 -- 172.21.15.5:0/4186546527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 msgr2=0x5606b92c3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 --2- 172.21.15.5:0/4186546527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c3770 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f230400b710 tx=0x7f2304004610 comp rx=0 tx=0).stop 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f2318ec9640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f22fc0a2f40 0x7f22fc0a5400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f230c0ee420 tx=0x7f230c08f080 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 -- 172.21.15.5:0/4186546527 shutdown_connections 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 --2- 172.21.15.5:0/4186546527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 0x5606b92c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 --2- 172.21.15.5:0/4186546527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c3770 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 --2- 172.21.15.5:0/4186546527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b8af0 0x5606b92b8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 -- 172.21.15.5:0/4186546527 >> 172.21.15.5:0/4186546527 conn(0x5606b92b5060 msgr2=0x5606b92b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 -- 172.21.15.5:0/4186546527 shutdown_connections 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 -- 172.21.15.5:0/4186546527 wait complete. 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 -- start start 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 Processor -- start 2024-09-11T22:22:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b8af0 0x5606b92c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 0x5606b92c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:40.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5606b92c6880 con 0x5606b92b9410 2024-09-11T22:22:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5606b91a6290 con 0x5606b92c3cb0 2024-09-11T22:22:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.984+0000 7f231d6c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5606b92c6a00 con 0x5606b92b8af0 2024-09-11T22:22:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f231d6c1740 10 client.0 _ll_register_callbacks cb 0x5606b91d7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f2313fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f2313fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49598/0 (socket says 172.21.15.5:49598) 2024-09-11T22:22:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f2313fff640 1 -- 172.21.15.5:0/850212586 learned_addr learned my addr 172.21.15.5:0/850212586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f23137fe640 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b8af0 0x5606b92c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f2313fff640 1 -- 172.21.15.5:0/850212586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b8af0 msgr2=0x5606b92c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f2313fff640 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b8af0 0x5606b92c0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f2313fff640 1 -- 172.21.15.5:0/850212586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 msgr2=0x5606b92c15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f2313fff640 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 0x5606b92c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f2313fff640 1 -- 172.21.15.5:0/850212586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606b8f30210 con 0x5606b92b9410 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f2313fff640 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c10a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2304004430 tx=0x7f2304005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc7f7af458ee401 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f22f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f22f37fe640 1 -- 172.21.15.5:0/850212586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304028060 con 0x5606b92b9410 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f22f37fe640 1 -- 172.21.15.5:0/850212586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f2304011640 con 0x5606b92b9410 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f231d6c1740 1 -- 172.21.15.5:0/850212586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2304015070 con 0x5606b92b9410 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f22f37fe640 1 -- 172.21.15.5:0/850212586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2304011840 con 0x5606b92b9410 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f231d6c1740 10 client.15456 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f231d6c1740 1 -- 172.21.15.5:0/850212586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606b8ffc620 con 0x5606b92b9410 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.985+0000 7f22f37fe640 1 -- 172.21.15.5:0/850212586 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2304015070 con 0x5606b92b9410 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.986+0000 7f22f37fe640 1 -- 172.21.15.5:0/850212586 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2304022070 con 0x5606b92b9410 2024-09-11T22:22:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.986+0000 7f231d6c1740 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.986+0000 7f231d6c1740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.986+0000 7f231d6c1740 20 client.15456 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.986+0000 7f231d6c1740 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5606b92d3c20 0x5606b92d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.986+0000 7f231d6c1740 4 client.15456 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.986+0000 7f231d6c1740 1 -- 172.21.15.5:0/850212586 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606b922aa20 con 0x5606b92d3c20 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.986+0000 7f23137fe640 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5606b92d3c20 0x5606b92d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.987+0000 7f23137fe640 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5606b92d3c20 0x5606b92d6010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d5a8e2db32656c server_cookie=302ddba9e4aa31dc in_seq=0 out_seq=0 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.987+0000 7f22f37fe640 10 client.15456 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.987+0000 7f22f37fe640 1 -- 172.21.15.5:0/850212586 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606b922aa20 con 0x5606b92d3c20 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.987+0000 7f22f37fe640 10 client.15456 handle_command_reply: tid=0 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.987+0000 7f231d6c1740 1 client.15456 shutdown 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.987+0000 7f231d6c1740 1 -- 172.21.15.5:0/850212586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 msgr2=0x5606b92c10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f231d6c1740 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c10a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2304004430 tx=0x7f2304005fb0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f231d6c1740 1 -- 172.21.15.5:0/850212586 shutdown_connections 2024-09-11T22:22:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f231d6c1740 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5606b92d3c20 0x5606b92d6010 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f231d6c1740 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 0x5606b92c15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f231d6c1740 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b9410 0x5606b92c10a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f22f37fe640 0 client.15456 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f231d6c1740 1 --2- 172.21.15.5:0/850212586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b8af0 0x5606b92c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f231d6c1740 1 -- 172.21.15.5:0/850212586 >> 172.21.15.5:0/850212586 conn(0x5606b92b5060 msgr2=0x5606b92bad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f231d6c1740 1 -- 172.21.15.5:0/850212586 shutdown_connections 2024-09-11T22:22:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f231d6c1740 1 -- 172.21.15.5:0/850212586 wait complete. 2024-09-11T22:22:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:40.988+0000 7f231d6c1740 20 client.15456 trim_cache size 0 max 16384 2024-09-11T22:22:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.002+0000 7f231d6c1740 1 Processor -- start 2024-09-11T22:22:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.002+0000 7f231d6c1740 1 -- start start 2024-09-11T22:22:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b8af0 0x5606b92c1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 0x5606b92db5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 0x5606b92dbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f231d6c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23141ad7d0 con 0x5606b92b8af0 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f231d6c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23141af310 con 0x5606b92c3cb0 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f231d6c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23141adea0 con 0x5606b92b9410 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f23137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b8af0 0x5606b92c1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f23137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b8af0 0x5606b92c1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49608/0 (socket says 172.21.15.5:49608) 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f23137fe640 1 -- 172.21.15.5:0/1232382237 learned_addr learned my addr 172.21.15.5:0/1232382237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f2313fff640 1 --2- 172.21.15.5:0/1232382237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 0x5606b92db5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f23137fe640 1 -- 172.21.15.5:0/1232382237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 msgr2=0x5606b92db5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f23137fe640 1 --2- 172.21.15.5:0/1232382237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 0x5606b92db5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f23137fe640 1 -- 172.21.15.5:0/1232382237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 msgr2=0x5606b92dbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f23137fe640 1 --2- 172.21.15.5:0/1232382237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 0x5606b92dbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f23137fe640 1 -- 172.21.15.5:0/1232382237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606b92d6380 con 0x5606b92b8af0 2024-09-11T22:22:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f2313fff640 1 --2- 172.21.15.5:0/1232382237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 0x5606b92db5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f23137fe640 1 --2- 172.21.15.5:0/1232382237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b8af0 0x5606b92c1410 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f23080128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f22285e9aae2b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.003+0000 7f22f9ffb640 1 -- 172.21.15.5:0/1232382237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308004310 con 0x5606b92b8af0 2024-09-11T22:22:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.004+0000 7f22f9ffb640 1 -- 172.21.15.5:0/1232382237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23080044b0 con 0x5606b92b8af0 2024-09-11T22:22:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.004+0000 7f22f9ffb640 1 -- 172.21.15.5:0/1232382237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308003850 con 0x5606b92b8af0 2024-09-11T22:22:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.004+0000 7f231d6c1740 1 -- 172.21.15.5:0/1232382237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b8af0 msgr2=0x5606b92c1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.004+0000 7f231d6c1740 1 --2- 172.21.15.5:0/1232382237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b8af0 0x5606b92c1410 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2308000c00 tx=0x7f23080128e0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.004+0000 7f231d6c1740 1 -- 172.21.15.5:0/1232382237 shutdown_connections 2024-09-11T22:22:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.004+0000 7f231d6c1740 1 --2- 172.21.15.5:0/1232382237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92c3cb0 0x5606b92dbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.004+0000 7f231d6c1740 1 --2- 172.21.15.5:0/1232382237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 0x5606b92db5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.004+0000 7f231d6c1740 1 --2- 172.21.15.5:0/1232382237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92b8af0 0x5606b92c1410 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.004+0000 7f231d6c1740 1 -- 172.21.15.5:0/1232382237 >> 172.21.15.5:0/1232382237 conn(0x5606b92b5060 msgr2=0x5606b92bfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.005+0000 7f231d6c1740 1 -- 172.21.15.5:0/1232382237 shutdown_connections 2024-09-11T22:22:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.005+0000 7f231d6c1740 1 -- 172.21.15.5:0/1232382237 wait complete. 2024-09-11T22:22:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.005+0000 7f231d6c1740 1 -- start start 2024-09-11T22:22:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.005+0000 7f231d6c1740 1 Processor -- start 2024-09-11T22:22:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92b8af0 0x5606b92cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 0x5606b92c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f231d6c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92c3cb0 0x5606b92c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f231d6c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23141aec40 con 0x5606b92c3cb0 2024-09-11T22:22:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f231d6c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23141ae570 con 0x5606b92b8af0 2024-09-11T22:22:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f231d6c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23141ab7b0 con 0x5606b92b9410 2024-09-11T22:22:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f231d6c1740 10 client.0 _ll_register_callbacks cb 0x7f2304004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f23137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92c3cb0 0x5606b92c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:41.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f23137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92c3cb0 0x5606b92c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49610/0 (socket says 172.21.15.5:49610) 2024-09-11T22:22:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f23137fe640 1 -- 172.21.15.5:0/664757768 learned_addr learned my addr 172.21.15.5:0/664757768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f2313fff640 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 0x5606b92c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f23137fe640 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92b8af0 0x5606b92cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f23137fe640 1 -- 172.21.15.5:0/664757768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 msgr2=0x5606b92c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:41.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f23137fe640 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 0x5606b92c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f23137fe640 1 -- 172.21.15.5:0/664757768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92b8af0 msgr2=0x5606b92cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f23137fe640 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92b8af0 0x5606b92cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f23137fe640 1 -- 172.21.15.5:0/664757768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606b92c8250 con 0x5606b92c3cb0 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f23137fe640 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92c3cb0 0x5606b92c7b60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2308005bc0 tx=0x7f230800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155943fa59972d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f22f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f22f37fe640 1 -- 172.21.15.5:0/664757768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308004830 con 0x5606b92c3cb0 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f22f37fe640 1 -- 172.21.15.5:0/664757768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f23080049d0 con 0x5606b92c3cb0 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.006+0000 7f231d6c1740 1 -- 172.21.15.5:0/664757768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23080044b0 con 0x5606b92c3cb0 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f22f37fe640 1 -- 172.21.15.5:0/664757768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f230803e050 con 0x5606b92c3cb0 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f231d6c1740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f231d6c1740 1 -- 172.21.15.5:0/664757768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5606b933e3f0 con 0x5606b92c3cb0 2024-09-11T22:22:41.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f22f37fe640 1 -- 172.21.15.5:0/664757768 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23080044b0 con 0x5606b92c3cb0 2024-09-11T22:22:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f22f37fe640 1 -- 172.21.15.5:0/664757768 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2308034030 con 0x5606b92c3cb0 2024-09-11T22:22:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f231d6c1740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f231d6c1740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:22:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f231d6c1740 20 client.15462 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f231d6c1740 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5606b933ea70 0x5606b9340e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f231d6c1740 4 client.15462 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:22:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f231d6c1740 1 -- 172.21.15.5:0/664757768 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606b9224d70 con 0x5606b933ea70 2024-09-11T22:22:41.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.007+0000 7f2313fff640 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5606b933ea70 0x5606b9340e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.008+0000 7f22f37fe640 10 client.15462 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:41.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.008+0000 7f2313fff640 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5606b933ea70 0x5606b9340e90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ef0b8d4c08fa7 server_cookie=721a9f4f9b3978c4 in_seq=0 out_seq=0 2024-09-11T22:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f22f37fe640 1 -- 172.21.15.5:0/664757768 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606b9224d70 con 0x5606b933ea70 2024-09-11T22:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f22f37fe640 10 client.15462 handle_command_reply: tid=0 2024-09-11T22:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 client.15462 shutdown 2024-09-11T22:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 -- 172.21.15.5:0/664757768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92c3cb0 msgr2=0x5606b92c7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92c3cb0 0x5606b92c7b60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2308005bc0 tx=0x7f230800b6e0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 -- 172.21.15.5:0/664757768 shutdown_connections 2024-09-11T22:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5606b933ea70 0x5606b9340e90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606b92c3cb0 0x5606b92c7b60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606b92b9410 0x5606b92c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 --2- 172.21.15.5:0/664757768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606b92b8af0 0x5606b92cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f22f37fe640 0 client.15462 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 -- 172.21.15.5:0/664757768 >> 172.21.15.5:0/664757768 conn(0x5606b92b5060 msgr2=0x5606b92ba4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 -- 172.21.15.5:0/664757768 shutdown_connections 2024-09-11T22:22:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.019+0000 7f231d6c1740 1 -- 172.21.15.5:0/664757768 wait complete. 2024-09-11T22:22:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231d6c1740 20 client.15462 trim_cache size 0 max 16384 2024-09-11T22:22:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:22:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:22:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:22:41.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:22:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 -- 172.21.15.5:0/2072520158 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f22fc0a2f40 msgr2=0x7f22fc0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f22fc0a2f40 0x7f22fc0a5400 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f230c0ee420 tx=0x7f230c08f080 comp rx=0 tx=0).stop 2024-09-11T22:22:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 -- 172.21.15.5:0/2072520158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2314107b80 msgr2=0x7f231419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2314107b80 0x7f231419cc00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f230c005cc0 tx=0x7f230c003970 comp rx=0 tx=0).stop 2024-09-11T22:22:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 -- 172.21.15.5:0/2072520158 shutdown_connections 2024-09-11T22:22:41.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2314107b80 0x7f231419cc00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f22fc0a2f40 0x7f22fc0a5400 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2314102480 0x7f2314198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 --2- 172.21.15.5:0/2072520158 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314101ae0 0x7f2314198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 -- 172.21.15.5:0/2072520158 >> 172.21.15.5:0/2072520158 conn(0x7f23140fd7e0 msgr2=0x7f23140fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 -- 172.21.15.5:0/2072520158 shutdown_connections 2024-09-11T22:22:41.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:41.020+0000 7f231a92c640 1 -- 172.21.15.5:0/2072520158 wait complete. 2024-09-11T22:22:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:40 smithi005 ceph-mon[30943]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-11T22:22:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:40 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2872981077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:41.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:40 smithi159 ceph-mon[35534]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-11T22:22:41.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:40 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2872981077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:41.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:40 smithi172 ceph-mon[35461]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-11T22:22:41.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:40 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2872981077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:22:41.500 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-11T22:20:18.900781+0000', 'modified': '2024-09-11T22:22:15.180031+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24500, 'mds_1': 24473, 'mds_2': 24454, 'mds_3': 24452, 'mds_4': 24436}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14583': {'gid': 14583, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/2458034474', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 2458034474}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 2458034474}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14595': {'gid': 14595, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/2232645952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 2232645952}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 2232645952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24391': {'gid': 24391, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6833/1186865242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6832', 'nonce': 1186865242}, {'type': 'v1', 'addr': '172.21.15.172:6833', 'nonce': 1186865242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.172:6835/4149321843', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 4149321843}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 4149321843}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'k', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/2975110123', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 2975110123}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 2975110123}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.172:6837/990138139', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6836', 'nonce': 990138139}, {'type': 'v1', 'addr': '172.21.15.172:6837', 'nonce': 990138139}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.159:6835/1262598700', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 1262598700}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 1262598700}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24454': {'gid': 24454, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.172:6839/1937249261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6838', 'nonce': 1937249261}, {'type': 'v1', 'addr': '172.21.15.172:6839', 'nonce': 1937249261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.159:6837/2315339117', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 2315339117}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 2315339117}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24500': {'gid': 24500, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.159:6839/994288762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 994288762}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 994288762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24500, 'qdb_cluster': [24436, 24452, 24454, 24473, 24500]} 2024-09-11T22:22:41.500 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-11T22:22:41.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-11T22:22:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.296+0000 7efea6340640 1 -- 172.21.15.5:0/4216554625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea0102790 msgr2=0x7efea01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.296+0000 7efea6340640 1 --2- 172.21.15.5:0/4216554625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea0102790 0x7efea01067e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efe94009900 tx=0x7efe9402a410 comp rx=0 tx=0).stop 2024-09-11T22:22:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.296+0000 7efea6340640 1 -- 172.21.15.5:0/4216554625 shutdown_connections 2024-09-11T22:22:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.296+0000 7efea6340640 1 --2- 172.21.15.5:0/4216554625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efea0106f10 0x7efea010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.296+0000 7efea6340640 1 --2- 172.21.15.5:0/4216554625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea0102790 0x7efea01067e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.296+0000 7efea6340640 1 --2- 172.21.15.5:0/4216554625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efea0101de0 0x7efea01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.296+0000 7efea6340640 1 -- 172.21.15.5:0/4216554625 >> 172.21.15.5:0/4216554625 conn(0x7efea00fd8c0 msgr2=0x7efea00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.296+0000 7efea6340640 1 -- 172.21.15.5:0/4216554625 shutdown_connections 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.296+0000 7efea6340640 1 -- 172.21.15.5:0/4216554625 wait complete. 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efea6340640 1 Processor -- start 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efea6340640 1 -- start start 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efea6340640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea0101de0 0x7efea00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efea6340640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efea0102790 0x7efea0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efea6340640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efea0106f10 0x7efea0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efea6340640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efea0111f70 con 0x7efea0101de0 2024-09-11T22:22:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efea6340640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efea0111df0 con 0x7efea0106f10 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efea6340640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efea01120f0 con 0x7efea0102790 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea0101de0 0x7efea00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea0101de0 0x7efea00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56890/0 (socket says 172.21.15.5:56890) 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.297+0000 7efe9ffff640 1 -- 172.21.15.5:0/1214505749 learned_addr learned my addr 172.21.15.5:0/1214505749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efe9f7fe640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efea0102790 0x7efea0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efea48dd640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efea0106f10 0x7efea0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efe9ffff640 1 -- 172.21.15.5:0/1214505749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efea0102790 msgr2=0x7efea0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efe9ffff640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efea0102790 0x7efea0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efe9ffff640 1 -- 172.21.15.5:0/1214505749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efea0106f10 msgr2=0x7efea0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efe9ffff640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efea0106f10 0x7efea0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efe9ffff640 1 -- 172.21.15.5:0/1214505749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea01a6990 con 0x7efea0101de0 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efe9ffff640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea0101de0 0x7efea00744f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efe9000c970 tx=0x7efe9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efe9d7fa640 1 -- 172.21.15.5:0/1214505749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90007bb0 con 0x7efea0101de0 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efe9d7fa640 1 -- 172.21.15.5:0/1214505749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe90007d50 con 0x7efea0101de0 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efea6340640 1 -- 172.21.15.5:0/1214505749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea01a6c80 con 0x7efea0101de0 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efe9d7fa640 1 -- 172.21.15.5:0/1214505749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9000f450 con 0x7efea0101de0 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.298+0000 7efea6340640 1 -- 172.21.15.5:0/1214505749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea01a7170 con 0x7efea0101de0 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.300+0000 7efe9d7fa640 1 -- 172.21.15.5:0/1214505749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7efe90005020 con 0x7efea0101de0 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efea8719740 1 Processor -- start 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efea8719740 1 -- start start 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efea8719740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a619d0 0x55ac74a61db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efea8719740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a6c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efea8719740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a6efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efea8719740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ac749c8090 con 0x55ac74a622f0 2024-09-11T22:22:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efea8719740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ac7496fa70 con 0x55ac74a619d0 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efea8719740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac749c4050 con 0x55ac74a6cbb0 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a6c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a6c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56902/0 (socket says 172.21.15.5:56902) 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efe9ffff640 1 -- 172.21.15.5:0/2073070187 learned_addr learned my addr 172.21.15.5:0/2073070187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efea48dd640 1 --2- 172.21.15.5:0/2073070187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a619d0 0x55ac74a61db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.301+0000 7efe9d7fa640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe880a2e10 0x7efe880a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efe9f7fe640 1 --2- 172.21.15.5:0/2073070187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a6efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efea48dd640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe880a2e10 0x7efe880a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efe9d7fa640 1 -- 172.21.15.5:0/1214505749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7efe900cf600 con 0x7efea0101de0 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efea48dd640 1 -- 172.21.15.5:0/2073070187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 msgr2=0x55ac74a6efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efea48dd640 1 --2- 172.21.15.5:0/2073070187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a6efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efea48dd640 1 -- 172.21.15.5:0/2073070187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 msgr2=0x55ac74a6c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efea48dd640 1 --2- 172.21.15.5:0/2073070187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a6c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efea48dd640 1 -- 172.21.15.5:0/2073070187 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7497c130 con 0x55ac74a619d0 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efea48dd640 1 --2- 172.21.15.5:0/2073070187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a619d0 0x55ac74a61db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efe980127b0 tx=0x7efe98008f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9a2ae2d682a6d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efea48dd640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe880a2e10 0x7efe880a52d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe9801a000 tx=0x7efe9801c000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efe867fc640 1 -- 172.21.15.5:0/2073070187 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe98004d40 con 0x55ac74a619d0 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efe867fc640 1 -- 172.21.15.5:0/2073070187 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe98020030 con 0x55ac74a619d0 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.302+0000 7efe867fc640 1 -- 172.21.15.5:0/2073070187 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980053b0 con 0x55ac74a619d0 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.303+0000 7efea8719740 1 -- 172.21.15.5:0/2073070187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a619d0 msgr2=0x55ac74a61db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.303+0000 7efea8719740 1 --2- 172.21.15.5:0/2073070187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a619d0 0x55ac74a61db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efe980127b0 tx=0x7efe98008f90 comp rx=0 tx=0).stop 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.303+0000 7efea8719740 1 -- 172.21.15.5:0/2073070187 shutdown_connections 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.303+0000 7efea8719740 1 --2- 172.21.15.5:0/2073070187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a6efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.303+0000 7efea8719740 1 --2- 172.21.15.5:0/2073070187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a6c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.303+0000 7efea8719740 1 --2- 172.21.15.5:0/2073070187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a619d0 0x55ac74a61db0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.303+0000 7efea8719740 1 -- 172.21.15.5:0/2073070187 >> 172.21.15.5:0/2073070187 conn(0x55ac74a5df60 msgr2=0x55ac74a5e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.303+0000 7efea8719740 1 -- 172.21.15.5:0/2073070187 shutdown_connections 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.303+0000 7efea8719740 1 -- 172.21.15.5:0/2073070187 wait complete. 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efea8719740 1 -- start start 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efea8719740 1 Processor -- start 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efea8719740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a619d0 0x55ac74a6c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efea8719740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a69800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efea8719740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a6cbb0 0x55ac74a69d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efea8719740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ac74a6f780 con 0x55ac74a622f0 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efea8719740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ac74982f20 con 0x55ac74a6cbb0 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efea8719740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac74a6f900 con 0x55ac74a619d0 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efea8719740 10 client.0 _ll_register_callbacks cb 0x55ac7494c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:42.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efe9f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a69800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efe9f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a69800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56918/0 (socket says 172.21.15.5:56918) 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efe9f7fe640 1 -- 172.21.15.5:0/578519732 learned_addr learned my addr 172.21.15.5:0/578519732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.304+0000 7efea48dd640 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a6cbb0 0x55ac74a69d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe9ffff640 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a619d0 0x55ac74a6c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe9f7fe640 1 -- 172.21.15.5:0/578519732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a619d0 msgr2=0x55ac74a6c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe9f7fe640 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a619d0 0x55ac74a6c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe9f7fe640 1 -- 172.21.15.5:0/578519732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a6cbb0 msgr2=0x55ac74a69d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe9f7fe640 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a6cbb0 0x55ac74a69d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe9f7fe640 1 -- 172.21.15.5:0/578519732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac747392b0 con 0x55ac74a622f0 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe9f7fe640 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a69800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efe94031550 tx=0x7efe94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53f9fd6b332a461 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:42.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe7f7fe640 1 -- 172.21.15.5:0/578519732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94035040 con 0x55ac74a622f0 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe7f7fe640 1 -- 172.21.15.5:0/578519732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7efe94005020 con 0x55ac74a622f0 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efea8719740 1 -- 172.21.15.5:0/578519732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe98020030 con 0x55ac74a622f0 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe7f7fe640 1 -- 172.21.15.5:0/578519732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efe94005220 con 0x55ac74a622f0 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efea8719740 10 client.15474 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efea8719740 1 -- 172.21.15.5:0/578519732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac74a7b0f0 con 0x55ac74a622f0 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.305+0000 7efe7f7fe640 1 -- 172.21.15.5:0/578519732 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe98020030 con 0x55ac74a622f0 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.306+0000 7efe7f7fe640 1 -- 172.21.15.5:0/578519732 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efe94003680 con 0x55ac74a622f0 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.306+0000 7efea8719740 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.306+0000 7efea8719740 10 client.15474 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.306+0000 7efea8719740 20 client.15474 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.306+0000 7efea8719740 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ac74a7d700 0x55ac74a7eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.306+0000 7efea8719740 4 client.15474 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:42.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.306+0000 7efea8719740 1 -- 172.21.15.5:0/578519732 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac749d5600 con 0x55ac74a7d700 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.306+0000 7efe9ffff640 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ac74a7d700 0x55ac74a7eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.306+0000 7efe9ffff640 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ac74a7d700 0x55ac74a7eb70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=49ff5e1510f74392 server_cookie=760dc365aaf96ed3 in_seq=0 out_seq=0 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.306+0000 7efe7f7fe640 10 client.15474 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.307+0000 7efe7f7fe640 1 -- 172.21.15.5:0/578519732 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac749d5600 con 0x55ac74a7d700 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.307+0000 7efe7f7fe640 10 client.15474 handle_command_reply: tid=0 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.307+0000 7efea8719740 1 client.15474 shutdown 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 1 -- 172.21.15.5:0/578519732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 msgr2=0x55ac74a69800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a69800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7efe94031550 tx=0x7efe94004270 comp rx=0 tx=0).stop 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 1 -- 172.21.15.5:0/578519732 shutdown_connections 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ac74a7d700 0x55ac74a7eb70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a6cbb0 0x55ac74a69d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a622f0 0x55ac74a69800 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 1 --2- 172.21.15.5:0/578519732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a619d0 0x55ac74a6c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 1 -- 172.21.15.5:0/578519732 >> 172.21.15.5:0/578519732 conn(0x55ac74a5df60 msgr2=0x55ac74a63c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efe7f7fe640 0 client.15474 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:22:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 1 -- 172.21.15.5:0/578519732 shutdown_connections 2024-09-11T22:22:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 1 -- 172.21.15.5:0/578519732 wait complete. 2024-09-11T22:22:42.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.308+0000 7efea8719740 20 client.15474 trim_cache size 0 max 16384 2024-09-11T22:22:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.326+0000 7efea8719740 1 Processor -- start 2024-09-11T22:22:42.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efea8719740 1 -- start start 2024-09-11T22:22:42.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efea8719740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a64d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efea8719740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a622f0 0x55ac74a66a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efea8719740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a66fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efea8719740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ac74a7ceb0 con 0x55ac74a619d0 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efea8719740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ac74a7d580 con 0x55ac74a622f0 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efea8719740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac74a7dc50 con 0x55ac74a6cbb0 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a64d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a64d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56932/0 (socket says 172.21.15.5:56932) 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efe9ffff640 1 -- 172.21.15.5:0/2315838080 learned_addr learned my addr 172.21.15.5:0/2315838080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efea48dd640 1 --2- 172.21.15.5:0/2315838080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a66fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efe9f7fe640 1 --2- 172.21.15.5:0/2315838080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a622f0 0x55ac74a66a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efe9ffff640 1 -- 172.21.15.5:0/2315838080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 msgr2=0x55ac74a66fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efe9ffff640 1 --2- 172.21.15.5:0/2315838080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a66fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efe9ffff640 1 -- 172.21.15.5:0/2315838080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a622f0 msgr2=0x55ac74a66a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efe9ffff640 1 --2- 172.21.15.5:0/2315838080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a622f0 0x55ac74a66a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.327+0000 7efe9ffff640 1 -- 172.21.15.5:0/2315838080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac74a6a340 con 0x55ac74a619d0 2024-09-11T22:22:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.328+0000 7efe9ffff640 1 --2- 172.21.15.5:0/2315838080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a64d20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efe9009b570 tx=0x7efe90086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5eeafbb9b0fcc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.328+0000 7efe85ffb640 1 -- 172.21.15.5:0/2315838080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9001d040 con 0x55ac74a619d0 2024-09-11T22:22:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.328+0000 7efe85ffb640 1 -- 172.21.15.5:0/2315838080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe9009bf70 con 0x55ac74a619d0 2024-09-11T22:22:42.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.328+0000 7efe85ffb640 1 -- 172.21.15.5:0/2315838080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90005310 con 0x55ac74a619d0 2024-09-11T22:22:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.328+0000 7efea8719740 1 -- 172.21.15.5:0/2315838080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 msgr2=0x55ac74a64d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.328+0000 7efea8719740 1 --2- 172.21.15.5:0/2315838080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a64d20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efe9009b570 tx=0x7efe90086040 comp rx=0 tx=0).stop 2024-09-11T22:22:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 -- 172.21.15.5:0/2315838080 shutdown_connections 2024-09-11T22:22:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 --2- 172.21.15.5:0/2315838080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a66fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 --2- 172.21.15.5:0/2315838080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a622f0 0x55ac74a66a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 --2- 172.21.15.5:0/2315838080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a64d20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 -- 172.21.15.5:0/2315838080 >> 172.21.15.5:0/2315838080 conn(0x55ac74a5df60 msgr2=0x55ac74a64610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 -- 172.21.15.5:0/2315838080 shutdown_connections 2024-09-11T22:22:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 -- 172.21.15.5:0/2315838080 wait complete. 2024-09-11T22:22:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 -- start start 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 Processor -- start 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a86ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a622f0 0x55ac74a87220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a87760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ac74a7e9f0 con 0x55ac74a619d0 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ac74a7e320 con 0x55ac74a622f0 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ac74a7f0c0 con 0x55ac74a6cbb0 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.329+0000 7efea8719740 10 client.0 _ll_register_callbacks cb 0x7efe94009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a86ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe9ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a86ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56940/0 (socket says 172.21.15.5:56940) 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe9ffff640 1 -- 172.21.15.5:0/676591630 learned_addr learned my addr 172.21.15.5:0/676591630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe9f7fe640 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a622f0 0x55ac74a87220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efea48dd640 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a87760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe9ffff640 1 -- 172.21.15.5:0/676591630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 msgr2=0x55ac74a87760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe9ffff640 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a87760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe9ffff640 1 -- 172.21.15.5:0/676591630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a622f0 msgr2=0x55ac74a87220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe9ffff640 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a622f0 0x55ac74a87220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe9ffff640 1 -- 172.21.15.5:0/676591630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac74a81f50 con 0x55ac74a619d0 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efea48dd640 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a87760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe9ffff640 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a86ce0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efe900a3fb0 tx=0x7efe9009b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a75791982f7088 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe7f7fe640 1 -- 172.21.15.5:0/676591630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9001d070 con 0x55ac74a619d0 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe7f7fe640 1 -- 172.21.15.5:0/676591630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7efe90097ee0 con 0x55ac74a619d0 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efea8719740 1 -- 172.21.15.5:0/676591630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe9009bf70 con 0x55ac74a619d0 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efe7f7fe640 1 -- 172.21.15.5:0/676591630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efe9009b870 con 0x55ac74a619d0 2024-09-11T22:22:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efea8719740 10 client.15480 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.330+0000 7efea8719740 1 -- 172.21.15.5:0/676591630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ac74ae8360 con 0x55ac74a619d0 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.331+0000 7efe7f7fe640 1 -- 172.21.15.5:0/676591630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe9009bf70 con 0x55ac74a619d0 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.331+0000 7efe7f7fe640 1 -- 172.21.15.5:0/676591630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efe90005310 con 0x55ac74a619d0 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.331+0000 7efea8719740 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.331+0000 7efea8719740 10 client.15480 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.331+0000 7efea8719740 20 client.15480 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.331+0000 7efea8719740 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ac74aec9b0 0x55ac74aeb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.331+0000 7efea8719740 4 client.15480 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.331+0000 7efea8719740 1 -- 172.21.15.5:0/676591630 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "status"}) -- 0x55ac749cfb10 con 0x55ac74aec9b0 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.331+0000 7efe9f7fe640 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ac74aec9b0 0x55ac74aeb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efe9f7fe640 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ac74aec9b0 0x55ac74aeb6c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=34b01f81db095aec server_cookie=c9dc1334f7750bde in_seq=0 out_seq=0 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efe7f7fe640 10 client.15480 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:22:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efe7f7fe640 1 -- 172.21.15.5:0/676591630 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ac749cfb10 con 0x55ac74aec9b0 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efe7f7fe640 10 client.15480 handle_command_reply: tid=0 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 client.15480 shutdown 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 -- 172.21.15.5:0/676591630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 msgr2=0x55ac74a86ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a86ce0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efe900a3fb0 tx=0x7efe9009b6f0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 -- 172.21.15.5:0/676591630 shutdown_connections 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ac74aec9b0 0x55ac74aeb6c0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ac74a6cbb0 0x55ac74a87760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac74a622f0 0x55ac74a87220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efe7f7fe640 0 client.15480 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 --2- 172.21.15.5:0/676591630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac74a619d0 0x55ac74a86ce0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 -- 172.21.15.5:0/676591630 >> 172.21.15.5:0/676591630 conn(0x55ac74a5df60 msgr2=0x55ac74a64de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 -- 172.21.15.5:0/676591630 shutdown_connections 2024-09-11T22:22:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 1 -- 172.21.15.5:0/676591630 wait complete. 2024-09-11T22:22:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.332+0000 7efea8719740 20 client.15480 trim_cache size 0 max 16384 2024-09-11T22:22:42.339 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:22:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-11T22:22:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24436, 2024-09-11T22:22:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:22:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:22:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:22:42.339 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 133.07070202400001, 2024-09-11T22:22:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-09-11T22:22:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-11T22:22:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T22:22:42.340 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 168.446782423 2024-09-11T22:22:42.340 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:42.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 -- 172.21.15.5:0/1214505749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe880a2e10 msgr2=0x7efe880a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe880a2e10 0x7efe880a52d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efe9801a000 tx=0x7efe9801c000 comp rx=0 tx=0).stop 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 -- 172.21.15.5:0/1214505749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea0101de0 msgr2=0x7efea00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea0101de0 0x7efea00744f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efe9000c970 tx=0x7efe9000ce30 comp rx=0 tx=0).stop 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 -- 172.21.15.5:0/1214505749 shutdown_connections 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efe880a2e10 0x7efe880a52d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efea0106f10 0x7efea0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efea0102790 0x7efea0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 --2- 172.21.15.5:0/1214505749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea0101de0 0x7efea00744f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 -- 172.21.15.5:0/1214505749 >> 172.21.15.5:0/1214505749 conn(0x7efea00fd8c0 msgr2=0x7efea00fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 -- 172.21.15.5:0/1214505749 shutdown_connections 2024-09-11T22:22:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:42.333+0000 7efea6340640 1 -- 172.21.15.5:0/1214505749 wait complete. 2024-09-11T22:22:43.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-11T22:22:43.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:43 smithi005 ceph-mon[30943]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-11T22:22:43.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:43 smithi159 ceph-mon[35534]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-11T22:22:43.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:43 smithi172 ceph-mon[35461]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-11T22:22:43.373 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.888+0000 7f05edb00640 1 -- 172.21.15.5:0/1122487377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e8101de0 msgr2=0x7f05e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.888+0000 7f05edb00640 1 --2- 172.21.15.5:0/1122487377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e8101de0 0x7f05e81021c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009900 tx=0x7f05d802a3f0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.888+0000 7f05edb00640 1 -- 172.21.15.5:0/1122487377 shutdown_connections 2024-09-11T22:22:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.888+0000 7f05edb00640 1 --2- 172.21.15.5:0/1122487377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e8106f10 0x7f05e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.888+0000 7f05edb00640 1 --2- 172.21.15.5:0/1122487377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e8102790 0x7f05e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.888+0000 7f05edb00640 1 --2- 172.21.15.5:0/1122487377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e8101de0 0x7f05e81021c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.888+0000 7f05edb00640 1 -- 172.21.15.5:0/1122487377 >> 172.21.15.5:0/1122487377 conn(0x7f05e80fd8c0 msgr2=0x7f05e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.888+0000 7f05edb00640 1 -- 172.21.15.5:0/1122487377 shutdown_connections 2024-09-11T22:22:43.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.888+0000 7f05edb00640 1 -- 172.21.15.5:0/1122487377 wait complete. 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.889+0000 7f05edb00640 1 Processor -- start 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.889+0000 7f05edb00640 1 -- start start 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05edb00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e8101de0 0x7f05e819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05edb00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e8102790 0x7f05e819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05edb00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e8106f10 0x7f05e819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05edb00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05e8111f90 con 0x7f05e8101de0 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05edb00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f05e8111e10 con 0x7f05e8102790 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05edb00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f05e8112110 con 0x7f05e8106f10 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e8101de0 0x7f05e819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e8101de0 0x7f05e819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56962/0 (socket says 172.21.15.5:56962) 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e77fe640 1 -- 172.21.15.5:0/3536330227 learned_addr learned my addr 172.21.15.5:0/3536330227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e77fe640 1 -- 172.21.15.5:0/3536330227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e8106f10 msgr2=0x7f05e819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e7fff640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e8106f10 0x7f05e819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e77fe640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e8106f10 0x7f05e819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e77fe640 1 -- 172.21.15.5:0/3536330227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e8102790 msgr2=0x7f05e819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e77fe640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e8102790 0x7f05e819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e77fe640 1 -- 172.21.15.5:0/3536330227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e819f6a0 con 0x7f05e8101de0 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e7fff640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e8106f10 0x7f05e819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e77fe640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e8101de0 0x7f05e819a6f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f05d8005ab0 tx=0x7f05d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.890+0000 7f05e4ff9640 1 -- 172.21.15.5:0/3536330227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d803f030 con 0x7f05e8101de0 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.891+0000 7f05edb00640 1 -- 172.21.15.5:0/3536330227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e819f930 con 0x7f05e8101de0 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.891+0000 7f05e4ff9640 1 -- 172.21.15.5:0/3536330227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d8003ce0 con 0x7f05e8101de0 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.891+0000 7f05e4ff9640 1 -- 172.21.15.5:0/3536330227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8035070 con 0x7f05e8101de0 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.891+0000 7f05edb00640 1 -- 172.21.15.5:0/3536330227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e81a7270 con 0x7f05e8101de0 2024-09-11T22:22:43.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.893+0000 7f05e4ff9640 1 -- 172.21.15.5:0/3536330227 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f05d8042030 con 0x7f05e8101de0 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.893+0000 7f05efed9740 1 Processor -- start 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.893+0000 7f05efed9740 1 -- start start 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.893+0000 7f05efed9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a953edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.893+0000 7f05efed9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a9549670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.893+0000 7f05efed9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a954bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.893+0000 7f05efed9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b6a94a5090 con 0x55b6a9549bb0 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.893+0000 7f05efed9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b6a944ca70 con 0x55b6a953e9d0 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.893+0000 7f05efed9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b6a94a1050 con 0x55b6a953f2f0 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e4ff9640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05d00a2e10 0x7f05d00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a954bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a9549670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a954bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56968/0 (socket says 172.21.15.5:56968) 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e6ffd640 1 -- 172.21.15.5:0/2119286084 learned_addr learned my addr 172.21.15.5:0/2119286084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e7fff640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05d00a2e10 0x7f05d00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e7fff640 1 --2- 172.21.15.5:0/2119286084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a953edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e4ff9640 1 -- 172.21.15.5:0/3536330227 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f05d80effd0 con 0x7f05e8101de0 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e6ffd640 1 -- 172.21.15.5:0/2119286084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 msgr2=0x55b6a9549670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e6ffd640 1 --2- 172.21.15.5:0/2119286084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a9549670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e6ffd640 1 -- 172.21.15.5:0/2119286084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 msgr2=0x55b6a953edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e6ffd640 1 --2- 172.21.15.5:0/2119286084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a953edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e6ffd640 1 -- 172.21.15.5:0/2119286084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a9459130 con 0x55b6a9549bb0 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e6ffd640 1 --2- 172.21.15.5:0/2119286084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a954bfa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0097f0 tx=0x7f05dc009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f419e723d16956 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05e7fff640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05d00a2e10 0x7f05d00a52d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f05e0005fd0 tx=0x7f05e000a420 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:43.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05cdffb640 1 -- 172.21.15.5:0/2119286084 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc015020 con 0x55b6a9549bb0 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05cdffb640 1 -- 172.21.15.5:0/2119286084 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05dc011420 con 0x55b6a9549bb0 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.894+0000 7f05cdffb640 1 -- 172.21.15.5:0/2119286084 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc0105e0 con 0x55b6a9549bb0 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.895+0000 7f05efed9740 1 -- 172.21.15.5:0/2119286084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 msgr2=0x55b6a954bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.895+0000 7f05efed9740 1 --2- 172.21.15.5:0/2119286084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a954bfa0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0097f0 tx=0x7f05dc009cb0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.895+0000 7f05efed9740 1 -- 172.21.15.5:0/2119286084 shutdown_connections 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.895+0000 7f05efed9740 1 --2- 172.21.15.5:0/2119286084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a954bfa0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.895+0000 7f05efed9740 1 --2- 172.21.15.5:0/2119286084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a9549670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.895+0000 7f05efed9740 1 --2- 172.21.15.5:0/2119286084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a953edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.895+0000 7f05efed9740 1 -- 172.21.15.5:0/2119286084 >> 172.21.15.5:0/2119286084 conn(0x55b6a953af60 msgr2=0x55b6a953b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.895+0000 7f05efed9740 1 -- 172.21.15.5:0/2119286084 shutdown_connections 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.895+0000 7f05efed9740 1 -- 172.21.15.5:0/2119286084 wait complete. 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05efed9740 1 -- start start 2024-09-11T22:22:43.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05efed9740 1 Processor -- start 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05efed9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953e9d0 0x55b6a95493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05efed9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a9546710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05efed9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a9549bb0 0x55b6a9546c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05efed9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b6a954c780 con 0x55b6a953e9d0 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05efed9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b6a945ff20 con 0x55b6a9549bb0 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05efed9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b6a954c900 con 0x55b6a953f2f0 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05efed9740 10 client.0 _ll_register_callbacks cb 0x55b6a9429030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953e9d0 0x55b6a95493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953e9d0 0x55b6a95493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56972/0 (socket says 172.21.15.5:56972) 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.896+0000 7f05e77fe640 1 -- 172.21.15.5:0/1601584161 learned_addr learned my addr 172.21.15.5:0/1601584161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05e6ffd640 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a9546710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05e7fff640 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a9549bb0 0x55b6a9546c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05e77fe640 1 -- 172.21.15.5:0/1601584161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 msgr2=0x55b6a9546710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05e77fe640 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a9546710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05e77fe640 1 -- 172.21.15.5:0/1601584161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a9549bb0 msgr2=0x55b6a9546c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05e77fe640 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a9549bb0 0x55b6a9546c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05e77fe640 1 -- 172.21.15.5:0/1601584161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a92162b0 con 0x55b6a953e9d0 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05e77fe640 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953e9d0 0x55b6a95493c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f05d80bc070 tx=0x7f05d803b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bd699f63cf9707 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05c6ffd640 1 -- 172.21.15.5:0/1601584161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d8049070 con 0x55b6a953e9d0 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05c6ffd640 1 -- 172.21.15.5:0/1601584161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f05d803b460 con 0x55b6a953e9d0 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05efed9740 1 -- 172.21.15.5:0/1601584161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05dc011420 con 0x55b6a953e9d0 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05c6ffd640 1 -- 172.21.15.5:0/1601584161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05d80c0720 con 0x55b6a953e9d0 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05efed9740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05efed9740 1 -- 172.21.15.5:0/1601584161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6a9558010 con 0x55b6a953e9d0 2024-09-11T22:22:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.897+0000 7f05c6ffd640 1 -- 172.21.15.5:0/1601584161 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05dc011420 con 0x55b6a953e9d0 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.898+0000 7f05c6ffd640 1 -- 172.21.15.5:0/1601584161 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f05d80d51e0 con 0x55b6a953e9d0 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.898+0000 7f05efed9740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.898+0000 7f05efed9740 10 client.15492 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.898+0000 7f05efed9740 20 client.15492 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.898+0000 7f05efed9740 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b6a955a5f0 0x55b6a955ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.898+0000 7f05efed9740 4 client.15492 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.898+0000 7f05efed9740 1 -- 172.21.15.5:0/1601584161 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6a94b2600 con 0x55b6a955a5f0 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.898+0000 7f05e6ffd640 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b6a955a5f0 0x55b6a955ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.898+0000 7f05e6ffd640 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b6a955a5f0 0x55b6a955ba60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=623a0a45980a3597 server_cookie=b91a866b147666fd in_seq=0 out_seq=0 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.898+0000 7f05c6ffd640 10 client.15492 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05c6ffd640 1 -- 172.21.15.5:0/1601584161 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6a94b2600 con 0x55b6a955a5f0 2024-09-11T22:22:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05c6ffd640 10 client.15492 handle_command_reply: tid=0 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05efed9740 1 client.15492 shutdown 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05efed9740 1 -- 172.21.15.5:0/1601584161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953e9d0 msgr2=0x55b6a95493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05efed9740 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953e9d0 0x55b6a95493c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f05d80bc070 tx=0x7f05d803b800 comp rx=0 tx=0).stop 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05efed9740 1 -- 172.21.15.5:0/1601584161 shutdown_connections 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05efed9740 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b6a955a5f0 0x55b6a955ba60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05efed9740 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a9549bb0 0x55b6a9546c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05efed9740 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a9546710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05efed9740 1 --2- 172.21.15.5:0/1601584161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953e9d0 0x55b6a95493c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05c6ffd640 0 client.15492 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.899+0000 7f05efed9740 1 -- 172.21.15.5:0/1601584161 >> 172.21.15.5:0/1601584161 conn(0x55b6a953af60 msgr2=0x55b6a9540c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.900+0000 7f05efed9740 1 -- 172.21.15.5:0/1601584161 shutdown_connections 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.900+0000 7f05efed9740 1 -- 172.21.15.5:0/1601584161 wait complete. 2024-09-11T22:22:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.900+0000 7f05efed9740 20 client.15492 trim_cache size 0 max 16384 2024-09-11T22:22:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05efed9740 1 Processor -- start 2024-09-11T22:22:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05efed9740 1 -- start start 2024-09-11T22:22:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05efed9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a9559f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05efed9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a955acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05efed9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a95466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05efed9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b6a9559da0 con 0x55b6a9549bb0 2024-09-11T22:22:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05efed9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b6a955a470 con 0x55b6a953e9d0 2024-09-11T22:22:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05efed9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b6a955ab40 con 0x55b6a953f2f0 2024-09-11T22:22:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a95466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a95466d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56978/0 (socket says 172.21.15.5:56978) 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e7fff640 1 -- 172.21.15.5:0/1038644050 learned_addr learned my addr 172.21.15.5:0/1038644050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e6ffd640 1 --2- 172.21.15.5:0/1038644050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a955acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e77fe640 1 --2- 172.21.15.5:0/1038644050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a9559f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e7fff640 1 -- 172.21.15.5:0/1038644050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 msgr2=0x55b6a955acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e7fff640 1 --2- 172.21.15.5:0/1038644050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a955acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e7fff640 1 -- 172.21.15.5:0/1038644050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 msgr2=0x55b6a9559f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e7fff640 1 --2- 172.21.15.5:0/1038644050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a9559f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e7fff640 1 -- 172.21.15.5:0/1038644050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6a954c890 con 0x55b6a9549bb0 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.918+0000 7f05e77fe640 1 --2- 172.21.15.5:0/1038644050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a9559f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.919+0000 7f05e7fff640 1 --2- 172.21.15.5:0/1038644050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a95466d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f05e0012590 tx=0x7f05e0005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a889e7a3496ec31 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:43.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.919+0000 7f05cd7fa640 1 -- 172.21.15.5:0/1038644050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e002b070 con 0x55b6a9549bb0 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.919+0000 7f05cd7fa640 1 -- 172.21.15.5:0/1038644050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e001ba20 con 0x55b6a9549bb0 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.919+0000 7f05cd7fa640 1 -- 172.21.15.5:0/1038644050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0026bd0 con 0x55b6a9549bb0 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 -- 172.21.15.5:0/1038644050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 msgr2=0x55b6a95466d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 --2- 172.21.15.5:0/1038644050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a95466d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f05e0012590 tx=0x7f05e0005f40 comp rx=0 tx=0).stop 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 -- 172.21.15.5:0/1038644050 shutdown_connections 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 --2- 172.21.15.5:0/1038644050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a9549bb0 0x55b6a95466d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 --2- 172.21.15.5:0/1038644050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a953f2f0 0x55b6a955acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 --2- 172.21.15.5:0/1038644050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a9559f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 -- 172.21.15.5:0/1038644050 >> 172.21.15.5:0/1038644050 conn(0x55b6a953af60 msgr2=0x55b6a9545c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 -- 172.21.15.5:0/1038644050 shutdown_connections 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 -- 172.21.15.5:0/1038644050 wait complete. 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 -- start start 2024-09-11T22:22:43.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.920+0000 7f05efed9740 1 Processor -- start 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05efed9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a955ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05efed9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953f2f0 0x55b6a955f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05efed9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a9549bb0 0x55b6a955f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05efed9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b6a955b8e0 con 0x55b6a953f2f0 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05efed9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b6a955b210 con 0x55b6a953e9d0 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05efed9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b6a955bfb0 con 0x55b6a9549bb0 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953f2f0 0x55b6a955f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953f2f0 0x55b6a955f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56980/0 (socket says 172.21.15.5:56980) 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e6ffd640 1 -- 172.21.15.5:0/624056343 learned_addr learned my addr 172.21.15.5:0/624056343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05efed9740 10 client.0 _ll_register_callbacks cb 0x7f05d80bc1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e77fe640 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a955ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e7fff640 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a9549bb0 0x55b6a955f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e6ffd640 1 -- 172.21.15.5:0/624056343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a9549bb0 msgr2=0x55b6a955f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e6ffd640 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a9549bb0 0x55b6a955f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e6ffd640 1 -- 172.21.15.5:0/624056343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 msgr2=0x55b6a955ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e6ffd640 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a955ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e6ffd640 1 -- 172.21.15.5:0/624056343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6a95c2f10 con 0x55b6a953f2f0 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05e6ffd640 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953f2f0 0x55b6a955f280 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f05dc000c00 tx=0x7f05dc004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4b4d160039fdd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.921+0000 7f05c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05c6ffd640 1 -- 172.21.15.5:0/624056343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc015040 con 0x55b6a953f2f0 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05c6ffd640 1 -- 172.21.15.5:0/624056343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f05dc00bda0 con 0x55b6a953f2f0 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05efed9740 1 -- 172.21.15.5:0/624056343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e001ba20 con 0x55b6a953f2f0 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05c6ffd640 1 -- 172.21.15.5:0/624056343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05dc011420 con 0x55b6a953f2f0 2024-09-11T22:22:43.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05efed9740 10 client.15498 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05efed9740 1 -- 172.21.15.5:0/624056343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6a95c4140 con 0x55b6a953f2f0 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05c6ffd640 1 -- 172.21.15.5:0/624056343 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e001ba20 con 0x55b6a953f2f0 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05c6ffd640 1 -- 172.21.15.5:0/624056343 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f05dc0257a0 con 0x55b6a953f2f0 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05efed9740 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05efed9740 10 client.15498 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05efed9740 20 client.15498 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05efed9740 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b6a95c8b60 0x55b6a95c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05efed9740 4 client.15498 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.922+0000 7f05efed9740 1 -- 172.21.15.5:0/624056343 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "status"}) -- 0x55b6a94acb10 con 0x55b6a95c8b60 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.923+0000 7f05e77fe640 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b6a95c8b60 0x55b6a95c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.923+0000 7f05c6ffd640 10 client.15498 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:22:43.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.923+0000 7f05e77fe640 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b6a95c8b60 0x55b6a95c7870 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90f3600df0834e46 server_cookie=ff050ac54bfc0550 in_seq=0 out_seq=0 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.923+0000 7f05c6ffd640 1 -- 172.21.15.5:0/624056343 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b6a94acb10 con 0x55b6a95c8b60 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.923+0000 7f05c6ffd640 10 client.15498 handle_command_reply: tid=0 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.923+0000 7f05efed9740 1 client.15498 shutdown 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 1 -- 172.21.15.5:0/624056343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953f2f0 msgr2=0x55b6a955f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953f2f0 0x55b6a955f280 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f05dc000c00 tx=0x7f05dc004820 comp rx=0 tx=0).stop 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 1 -- 172.21.15.5:0/624056343 shutdown_connections 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b6a9549bb0 0x55b6a955f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6a953f2f0 0x55b6a955f280 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b6a95c8b60 0x55b6a95c7870 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05c6ffd640 0 client.15498 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 1 --2- 172.21.15.5:0/624056343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6a953e9d0 0x55b6a955ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 1 -- 172.21.15.5:0/624056343 >> 172.21.15.5:0/624056343 conn(0x55b6a953af60 msgr2=0x55b6a95466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:43.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 1 -- 172.21.15.5:0/624056343 shutdown_connections 2024-09-11T22:22:43.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 1 -- 172.21.15.5:0/624056343 wait complete. 2024-09-11T22:22:43.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05efed9740 20 client.15498 trim_cache size 0 max 16384 2024-09-11T22:22:43.938 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:43.938 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:22:43.938 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-11T22:22:43.938 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24452, 2024-09-11T22:22:43.939 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:22:43.939 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:22:43.939 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:22:43.939 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 135.67476479300001, 2024-09-11T22:22:43.939 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-09-11T22:22:43.939 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-11T22:22:43.939 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T22:22:43.939 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 175.27503330299999 2024-09-11T22:22:43.939 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:43.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05edb00640 1 -- 172.21.15.5:0/3536330227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05d00a2e10 msgr2=0x7f05d00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05edb00640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05d00a2e10 0x7f05d00a52d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f05e0005fd0 tx=0x7f05e000a420 comp rx=0 tx=0).stop 2024-09-11T22:22:43.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05edb00640 1 -- 172.21.15.5:0/3536330227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e8101de0 msgr2=0x7f05e819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:43.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.924+0000 7f05edb00640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e8101de0 0x7f05e819a6f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f05d8005ab0 tx=0x7f05d80042f0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.925+0000 7f05edb00640 1 -- 172.21.15.5:0/3536330227 shutdown_connections 2024-09-11T22:22:43.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.925+0000 7f05edb00640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e8106f10 0x7f05e819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.925+0000 7f05edb00640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05d00a2e10 0x7f05d00a52d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.925+0000 7f05edb00640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f05e8102790 0x7f05e819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.925+0000 7f05edb00640 1 --2- 172.21.15.5:0/3536330227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05e8101de0 0x7f05e819a6f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:43.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.925+0000 7f05edb00640 1 -- 172.21.15.5:0/3536330227 >> 172.21.15.5:0/3536330227 conn(0x7f05e80fd8c0 msgr2=0x7f05e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:43.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.925+0000 7f05edb00640 1 -- 172.21.15.5:0/3536330227 shutdown_connections 2024-09-11T22:22:43.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:43.925+0000 7f05edb00640 1 -- 172.21.15.5:0/3536330227 wait complete. 2024-09-11T22:22:44.662 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-11T22:22:44.927 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:45.234 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:45 smithi005 ceph-mon[30943]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.7 KiB/s wr, 26 op/s 2024-09-11T22:22:45.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:45 smithi159 ceph-mon[35534]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.7 KiB/s wr, 26 op/s 2024-09-11T22:22:45.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:45 smithi172 ceph-mon[35461]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 2.7 KiB/s wr, 26 op/s 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.476+0000 7f2d65af8640 1 -- 172.21.15.5:0/1479834070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d60102ef0 msgr2=0x7f2d6010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.476+0000 7f2d65af8640 1 --2- 172.21.15.5:0/1479834070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d60102ef0 0x7f2d6010da40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009ca0 tx=0x7f2d5802a950 comp rx=0 tx=0).stop 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.476+0000 7f2d65af8640 1 -- 172.21.15.5:0/1479834070 shutdown_connections 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.476+0000 7f2d65af8640 1 --2- 172.21.15.5:0/1479834070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d60102ef0 0x7f2d6010da40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.476+0000 7f2d65af8640 1 --2- 172.21.15.5:0/1479834070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d60102570 0x7f2d601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.476+0000 7f2d65af8640 1 --2- 172.21.15.5:0/1479834070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d60100e20 0x7f2d60101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.476+0000 7f2d65af8640 1 -- 172.21.15.5:0/1479834070 >> 172.21.15.5:0/1479834070 conn(0x7f2d600fc7d0 msgr2=0x7f2d600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.476+0000 7f2d65af8640 1 -- 172.21.15.5:0/1479834070 shutdown_connections 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.476+0000 7f2d65af8640 1 -- 172.21.15.5:0/1479834070 wait complete. 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d65af8640 1 Processor -- start 2024-09-11T22:22:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d65af8640 1 -- start start 2024-09-11T22:22:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d65af8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d60100e20 0x7f2d60196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d65af8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d60102570 0x7f2d601968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d65af8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d60102ef0 0x7f2d6019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d65af8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6010fe80 con 0x7f2d60102570 2024-09-11T22:22:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d65af8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6010fd00 con 0x7f2d60102ef0 2024-09-11T22:22:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d65af8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d60110000 con 0x7f2d60100e20 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d5effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d60102570 0x7f2d601968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d5effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d60102570 0x7f2d601968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57002/0 (socket says 172.21.15.5:57002) 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d5effd640 1 -- 172.21.15.5:0/3750208063 learned_addr learned my addr 172.21.15.5:0/3750208063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d60100e20 0x7f2d60196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.477+0000 7f2d5ffff640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d60102ef0 0x7f2d6019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d5effd640 1 -- 172.21.15.5:0/3750208063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d60100e20 msgr2=0x7f2d60196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d5effd640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d60100e20 0x7f2d60196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d5effd640 1 -- 172.21.15.5:0/3750208063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d60102ef0 msgr2=0x7f2d6019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d5effd640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d60102ef0 0x7f2d6019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d5effd640 1 -- 172.21.15.5:0/3750208063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6019b310 con 0x7f2d60102570 2024-09-11T22:22:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d5effd640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d60102570 0x7f2d601968a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400d950 tx=0x7f2d5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d5cff9640 1 -- 172.21.15.5:0/3750208063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54017070 con 0x7f2d60102570 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d65af8640 1 -- 172.21.15.5:0/3750208063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6019b600 con 0x7f2d60102570 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d5cff9640 1 -- 172.21.15.5:0/3750208063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d54009c30 con 0x7f2d60102570 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d5cff9640 1 -- 172.21.15.5:0/3750208063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5400f6b0 con 0x7f2d60102570 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.478+0000 7f2d65af8640 1 -- 172.21.15.5:0/3750208063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d601a2ee0 con 0x7f2d60102570 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d67ed1740 1 Processor -- start 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d5cff9640 1 -- 172.21.15.5:0/3750208063 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f2d540109d0 con 0x7f2d60102570 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d67ed1740 1 -- start start 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc237cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc246a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc246fc0 0x5604cc2493b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5604cc19e090 con 0x5604cc2378d0 2024-09-11T22:22:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5604cc145a70 con 0x5604cc2381f0 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5604cc19a050 con 0x5604cc246fc0 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc237cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d5f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc246a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc237cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57016/0 (socket says 172.21.15.5:57016) 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.481+0000 7f2d5ffff640 1 -- 172.21.15.5:0/1901282631 learned_addr learned my addr 172.21.15.5:0/1901282631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5effd640 1 --2- 172.21.15.5:0/1901282631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc246fc0 0x5604cc2493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/1901282631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc246fc0 msgr2=0x5604cc2493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/1901282631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc246fc0 0x5604cc2493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/1901282631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 msgr2=0x5604cc237cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/1901282631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc237cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/1901282631 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604cc152130 con 0x5604cc2381f0 2024-09-11T22:22:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5ffff640 1 --2- 172.21.15.5:0/1901282631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc237cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5cff9640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d480a2e10 0x7f2d480a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5ffff640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d480a2e10 0x7f2d480a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/1901282631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc246a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000ab40 tx=0x7f2d5000f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d42044c3d4ad077 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d5cff9640 1 -- 172.21.15.5:0/3750208063 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f2d540cf250 con 0x7f2d60102570 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d45ffb640 1 -- 172.21.15.5:0/1901282631 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50004020 con 0x5604cc2381f0 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d45ffb640 1 -- 172.21.15.5:0/1901282631 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d500041c0 con 0x5604cc2381f0 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.482+0000 7f2d45ffb640 1 -- 172.21.15.5:0/1901282631 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50018800 con 0x5604cc2381f0 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1901282631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 msgr2=0x5604cc246a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1901282631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc246a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000ab40 tx=0x7f2d5000f3c0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d5ffff640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d480a2e10 0x7f2d480a52d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d58005fe0 tx=0x7f2d58005c50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1901282631 shutdown_connections 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1901282631 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc246fc0 0x5604cc2493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1901282631 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc246a80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1901282631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc237cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1901282631 >> 172.21.15.5:0/1901282631 conn(0x5604cc233f60 msgr2=0x5604cc234340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1901282631 shutdown_connections 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1901282631 wait complete. 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.483+0000 7f2d67ed1740 1 -- start start 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d67ed1740 1 Processor -- start 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc246870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2381f0 0x5604cc243bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc246fc0 0x5604cc244100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5604cc249bf0 con 0x5604cc2378d0 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5604cc158f20 con 0x5604cc246fc0 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5604cc249d70 con 0x5604cc2381f0 2024-09-11T22:22:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d67ed1740 10 client.0 _ll_register_callbacks cb 0x5604cc122030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d5f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc246870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d5f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc246870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57026/0 (socket says 172.21.15.5:57026) 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/1532786866 learned_addr learned my addr 172.21.15.5:0/1532786866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d5effd640 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2381f0 0x5604cc243bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.484+0000 7f2d5ffff640 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc246fc0 0x5604cc244100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/1532786866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2381f0 msgr2=0x5604cc243bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2381f0 0x5604cc243bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/1532786866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc246fc0 msgr2=0x5604cc244100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc246fc0 0x5604cc244100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/1532786866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604cbf0f2b0 con 0x5604cc2378d0 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc246870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000a500 tx=0x7f2d50003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff8d6a753c687d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d3effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d3effd640 1 -- 172.21.15.5:0/1532786866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50027020 con 0x5604cc2378d0 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1532786866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d500041c0 con 0x5604cc2378d0 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d3effd640 1 -- 172.21.15.5:0/1532786866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f2d50004370 con 0x5604cc2378d0 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d3effd640 1 -- 172.21.15.5:0/1532786866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d50032050 con 0x5604cc2378d0 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d67ed1740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.485+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1532786866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604cc255550 con 0x5604cc2378d0 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.486+0000 7f2d3effd640 1 -- 172.21.15.5:0/1532786866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d500041c0 con 0x5604cc2378d0 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.486+0000 7f2d3effd640 1 -- 172.21.15.5:0/1532786866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d50004aa0 con 0x5604cc2378d0 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.486+0000 7f2d67ed1740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.486+0000 7f2d67ed1740 10 client.15510 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.486+0000 7f2d67ed1740 20 client.15510 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.486+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5604cc257820 0x5604cc258c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.486+0000 7f2d67ed1740 4 client.15510 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.486+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1532786866 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604cc1ab600 con 0x5604cc257820 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.486+0000 7f2d5effd640 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5604cc257820 0x5604cc258c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.487+0000 7f2d5effd640 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5604cc257820 0x5604cc258c90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f53198f341b4ac5 server_cookie=681640c179957cb in_seq=0 out_seq=0 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.487+0000 7f2d3effd640 10 client.15510 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.487+0000 7f2d3effd640 1 -- 172.21.15.5:0/1532786866 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604cc1ab600 con 0x5604cc257820 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.487+0000 7f2d3effd640 10 client.15510 handle_command_reply: tid=0 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.487+0000 7f2d67ed1740 1 client.15510 shutdown 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1532786866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 msgr2=0x5604cc246870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc246870 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2d5000a500 tx=0x7f2d50003dd0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1532786866 shutdown_connections 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5604cc257820 0x5604cc258c90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc246fc0 0x5604cc244100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2381f0 0x5604cc243bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1532786866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc246870 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1532786866 >> 172.21.15.5:0/1532786866 conn(0x5604cc233f60 msgr2=0x5604cc239c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d3effd640 0 client.15510 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:22:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1532786866 shutdown_connections 2024-09-11T22:22:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1532786866 wait complete. 2024-09-11T22:22:45.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.488+0000 7f2d67ed1740 20 client.15510 trim_cache size 0 max 16384 2024-09-11T22:22:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.506+0000 7f2d67ed1740 1 Processor -- start 2024-09-11T22:22:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.506+0000 7f2d67ed1740 1 -- start start 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.506+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc257820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.506+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc2585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.506+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc246fc0 0x5604cc257150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.506+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5604cc256fd0 con 0x5604cc2378d0 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.506+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5604cc2576a0 con 0x5604cc2381f0 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.506+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5604cc257d70 con 0x5604cc246fc0 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.506+0000 7f2d5f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc257820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.506+0000 7f2d5f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc257820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57034/0 (socket says 172.21.15.5:57034) 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/2026952663 learned_addr learned my addr 172.21.15.5:0/2026952663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d5effd640 1 --2- 172.21.15.5:0/2026952663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc2585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d5ffff640 1 --2- 172.21.15.5:0/2026952663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc246fc0 0x5604cc257150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/2026952663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc246fc0 msgr2=0x5604cc257150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/2026952663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc246fc0 0x5604cc257150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/2026952663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 msgr2=0x5604cc2585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/2026952663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc2585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d5f7fe640 1 -- 172.21.15.5:0/2026952663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604cc246490 con 0x5604cc2378d0 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d5effd640 1 --2- 172.21.15.5:0/2026952663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc2585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/2026952663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc257820 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2d500058c0 tx=0x7f2d5000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65280c97bc06da4c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d457fa640 1 -- 172.21.15.5:0/2026952663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50027070 con 0x5604cc2378d0 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d457fa640 1 -- 172.21.15.5:0/2026952663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d50003ad0 con 0x5604cc2378d0 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.507+0000 7f2d457fa640 1 -- 172.21.15.5:0/2026952663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d50003e40 con 0x5604cc2378d0 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.508+0000 7f2d67ed1740 1 -- 172.21.15.5:0/2026952663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 msgr2=0x5604cc257820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.508+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/2026952663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc257820 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2d500058c0 tx=0x7f2d5000a340 comp rx=0 tx=0).stop 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.508+0000 7f2d67ed1740 1 -- 172.21.15.5:0/2026952663 shutdown_connections 2024-09-11T22:22:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.508+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/2026952663 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc246fc0 0x5604cc257150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.508+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/2026952663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc2585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.508+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/2026952663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc2378d0 0x5604cc257820 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.508+0000 7f2d67ed1740 1 -- 172.21.15.5:0/2026952663 >> 172.21.15.5:0/2026952663 conn(0x5604cc233f60 msgr2=0x5604cc234340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.508+0000 7f2d67ed1740 1 -- 172.21.15.5:0/2026952663 shutdown_connections 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.508+0000 7f2d67ed1740 1 -- 172.21.15.5:0/2026952663 wait complete. 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.508+0000 7f2d67ed1740 1 -- start start 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d67ed1740 1 Processor -- start 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2378d0 0x5604cc259da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc25a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d67ed1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc246fc0 0x5604cc25a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5604cc258b10 con 0x5604cc246fc0 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5604cc258440 con 0x5604cc2381f0 2024-09-11T22:22:45.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d67ed1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5604cc2591e0 con 0x5604cc2378d0 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc246fc0 0x5604cc25a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc246fc0 0x5604cc25a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57046/0 (socket says 172.21.15.5:57046) 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d5ffff640 1 -- 172.21.15.5:0/1574920794 learned_addr learned my addr 172.21.15.5:0/1574920794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d67ed1740 10 client.0 _ll_register_callbacks cb 0x7f2d58005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d5effd640 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc25a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2378d0 0x5604cc259da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d5ffff640 1 -- 172.21.15.5:0/1574920794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2378d0 msgr2=0x5604cc259da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d5ffff640 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2378d0 0x5604cc259da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d5ffff640 1 -- 172.21.15.5:0/1574920794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 msgr2=0x5604cc25a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d5ffff640 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc25a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.509+0000 7f2d5ffff640 1 -- 172.21.15.5:0/1574920794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604cc2bfff0 con 0x5604cc246fc0 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.510+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2378d0 0x5604cc259da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.510+0000 7f2d5ffff640 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc246fc0 0x5604cc25a820 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009ca0 tx=0x7f2d580095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c1e5d2b8a8c588 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:45.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.510+0000 7f2d3effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.510+0000 7f2d3effd640 1 -- 172.21.15.5:0/1574920794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58057060 con 0x5604cc246fc0 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.510+0000 7f2d3effd640 1 -- 172.21.15.5:0/1574920794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f2d58004790 con 0x5604cc246fc0 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.510+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1574920794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d50003ad0 con 0x5604cc246fc0 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.510+0000 7f2d3effd640 1 -- 172.21.15.5:0/1574920794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d58034040 con 0x5604cc246fc0 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.510+0000 7f2d67ed1740 10 client.15516 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.510+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1574920794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604cc2c12d0 con 0x5604cc246fc0 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.510+0000 7f2d3effd640 1 -- 172.21.15.5:0/1574920794 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d50003ad0 con 0x5604cc246fc0 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.511+0000 7f2d3effd640 1 -- 172.21.15.5:0/1574920794 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d5803a430 con 0x5604cc246fc0 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.511+0000 7f2d67ed1740 10 client.15516 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.511+0000 7f2d67ed1740 10 client.15516 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.511+0000 7f2d67ed1740 20 client.15516 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.511+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5604cc2c5f70 0x5604cc2c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.511+0000 7f2d67ed1740 4 client.15516 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:22:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.511+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1574920794 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "status"}) -- 0x5604cc244d10 con 0x5604cc2c5f70 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.511+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5604cc2c5f70 0x5604cc2c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d3effd640 10 client.15516 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d5f7fe640 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5604cc2c5f70 0x5604cc2c4c80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35ef57d81123a70f server_cookie=acc01562ba7a545c in_seq=0 out_seq=0 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d3effd640 1 -- 172.21.15.5:0/1574920794 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5604cc244d10 con 0x5604cc2c5f70 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d3effd640 10 client.15516 handle_command_reply: tid=0 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 client.15516 shutdown 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1574920794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc246fc0 msgr2=0x5604cc25a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc246fc0 0x5604cc25a820 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009ca0 tx=0x7f2d580095c0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1574920794 shutdown_connections 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5604cc2c5f70 0x5604cc2c4c80 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604cc246fc0 0x5604cc25a820 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d3effd640 0 client.15516 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604cc2381f0 0x5604cc25a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 --2- 172.21.15.5:0/1574920794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604cc2378d0 0x5604cc259da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1574920794 >> 172.21.15.5:0/1574920794 conn(0x5604cc233f60 msgr2=0x5604cc23f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1574920794 shutdown_connections 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 1 -- 172.21.15.5:0/1574920794 wait complete. 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.512+0000 7f2d67ed1740 20 client.15516 trim_cache size 0 max 16384 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24454, 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:22:45.520 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 138.268080602, 2024-09-11T22:22:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-09-11T22:22:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-11T22:22:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T22:22:45.521 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 166.16055700800001 2024-09-11T22:22:45.521 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 -- 172.21.15.5:0/3750208063 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d480a2e10 msgr2=0x7f2d480a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d480a2e10 0x7f2d480a52d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2d58005fe0 tx=0x7f2d58005c50 comp rx=0 tx=0).stop 2024-09-11T22:22:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 -- 172.21.15.5:0/3750208063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d60102570 msgr2=0x7f2d601968a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d60102570 0x7f2d601968a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400d950 tx=0x7f2d5400de10 comp rx=0 tx=0).stop 2024-09-11T22:22:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 -- 172.21.15.5:0/3750208063 shutdown_connections 2024-09-11T22:22:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d480a2e10 0x7f2d480a52d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d60102ef0 0x7f2d6019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d60102570 0x7f2d601968a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 --2- 172.21.15.5:0/3750208063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d60100e20 0x7f2d60196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 -- 172.21.15.5:0/3750208063 >> 172.21.15.5:0/3750208063 conn(0x7f2d600fc7d0 msgr2=0x7f2d601068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 -- 172.21.15.5:0/3750208063 shutdown_connections 2024-09-11T22:22:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:45.513+0000 7f2d65af8640 1 -- 172.21.15.5:0/3750208063 wait complete. 2024-09-11T22:22:46.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-11T22:22:46.574 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:47.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.069+0000 7f5bcfe07640 1 -- 172.21.15.5:0/32185580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc8102d40 msgr2=0x7f5bc8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:47.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.069+0000 7f5bcfe07640 1 --2- 172.21.15.5:0/32185580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc8102d40 0x7f5bc8103120 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4009900 tx=0x7f5bc402a480 comp rx=0 tx=0).stop 2024-09-11T22:22:47.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.069+0000 7f5bcfe07640 1 -- 172.21.15.5:0/32185580 shutdown_connections 2024-09-11T22:22:47.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.069+0000 7f5bcfe07640 1 --2- 172.21.15.5:0/32185580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc810c170 0x7f5bc810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.069+0000 7f5bcfe07640 1 --2- 172.21.15.5:0/32185580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc8103660 0x7f5bc810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.069+0000 7f5bcfe07640 1 --2- 172.21.15.5:0/32185580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc8102d40 0x7f5bc8103120 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.069+0000 7f5bcfe07640 1 -- 172.21.15.5:0/32185580 >> 172.21.15.5:0/32185580 conn(0x7f5bc80fc7a0 msgr2=0x7f5bc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:47.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.069+0000 7f5bcfe07640 1 -- 172.21.15.5:0/32185580 shutdown_connections 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.069+0000 7f5bcfe07640 1 -- 172.21.15.5:0/32185580 wait complete. 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcfe07640 1 Processor -- start 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcfe07640 1 -- start start 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcfe07640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc8102d40 0x7f5bc81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcfe07640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc8103660 0x7f5bc8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcfe07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc810c170 0x7f5bc8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcfe07640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8112030 con 0x7f5bc810c170 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcfe07640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8111eb0 con 0x7f5bc8103660 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcfe07640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81121b0 con 0x7f5bc8102d40 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bce3a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc810c170 0x7f5bc8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bce3a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc810c170 0x7f5bc8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57064/0 (socket says 172.21.15.5:57064) 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bce3a4640 1 -- 172.21.15.5:0/3844701603 learned_addr learned my addr 172.21.15.5:0/3844701603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:47.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcdba3640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc8102d40 0x7f5bc81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc8103660 0x7f5bc8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bce3a4640 1 -- 172.21.15.5:0/3844701603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc8102d40 msgr2=0x7f5bc81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bce3a4640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc8102d40 0x7f5bc81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bce3a4640 1 -- 172.21.15.5:0/3844701603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc8103660 msgr2=0x7f5bc8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bce3a4640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc8103660 0x7f5bc8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bce3a4640 1 -- 172.21.15.5:0/3844701603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc81a6a60 con 0x7f5bc810c170 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.070+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc8103660 0x7f5bc8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.071+0000 7f5bce3a4640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc810c170 0x7f5bc8101420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000b970 tx=0x7f5bc000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.071+0000 7f5bbeffd640 1 -- 172.21.15.5:0/3844701603 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0019070 con 0x7f5bc810c170 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.071+0000 7f5bbeffd640 1 -- 172.21.15.5:0/3844701603 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc000cb40 con 0x7f5bc810c170 2024-09-11T22:22:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.071+0000 7f5bcfe07640 1 -- 172.21.15.5:0/3844701603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc81a6d50 con 0x7f5bc810c170 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.071+0000 7f5bbeffd640 1 -- 172.21.15.5:0/3844701603 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0007650 con 0x7f5bc810c170 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.071+0000 7f5bcfe07640 1 -- 172.21.15.5:0/3844701603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81a72b0 con 0x7f5bc810c170 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.073+0000 7f5bbeffd640 1 -- 172.21.15.5:0/3844701603 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f5bc001c070 con 0x7f5bc810c170 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.073+0000 7f5bd21e0740 1 Processor -- start 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bd21e0740 1 -- start start 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db78cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db78d2f0 0x55d2db797670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db797bb0 0x55d2db799fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2db6f3090 con 0x55d2db78c9d0 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2db69aa70 con 0x55d2db797bb0 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d2db6ef050 con 0x55d2db78d2f0 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bbeffd640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5bb40a2e90 0x7f5bb40a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bcd3a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db78cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bcd3a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db78cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57068/0 (socket says 172.21.15.5:57068) 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/4110251471 learned_addr learned my addr 172.21.15.5:0/4110251471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bce3a4640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5bb40a2e90 0x7f5bb40a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/4110251471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db797bb0 0x55d2db799fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bcdba3640 1 --2- 172.21.15.5:0/4110251471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db78d2f0 0x55d2db797670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bbeffd640 1 -- 172.21.15.5:0/3844701603 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f5bc00ceba0 con 0x7f5bc810c170 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/4110251471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db78d2f0 msgr2=0x55d2db797670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/4110251471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db78d2f0 0x55d2db797670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/4110251471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db797bb0 msgr2=0x55d2db799fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/4110251471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db797bb0 0x55d2db799fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.074+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/4110251471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2db6a7130 con 0x55d2db78c9d0 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.075+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/4110251471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db78cdb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800af20 tx=0x7f5bb800a3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0acd4e2c304cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.075+0000 7f5bce3a4640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5bb40a2e90 0x7f5bb40a5350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00cc280 tx=0x7f5bc000e040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.075+0000 7f5bb3fff640 1 -- 172.21.15.5:0/4110251471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb801b070 con 0x55d2db78c9d0 2024-09-11T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.075+0000 7f5bb3fff640 1 -- 172.21.15.5:0/4110251471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb80168d0 con 0x55d2db78c9d0 2024-09-11T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.075+0000 7f5bb3fff640 1 -- 172.21.15.5:0/4110251471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800f040 con 0x55d2db78c9d0 2024-09-11T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 -- 172.21.15.5:0/4110251471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 msgr2=0x55d2db78cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/4110251471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db78cdb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800af20 tx=0x7f5bb800a3a0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 -- 172.21.15.5:0/4110251471 shutdown_connections 2024-09-11T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/4110251471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db797bb0 0x55d2db799fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/4110251471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db78d2f0 0x55d2db797670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/4110251471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db78cdb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 -- 172.21.15.5:0/4110251471 >> 172.21.15.5:0/4110251471 conn(0x55d2db788f60 msgr2=0x55d2db789340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 -- 172.21.15.5:0/4110251471 shutdown_connections 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 -- 172.21.15.5:0/4110251471 wait complete. 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 -- start start 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.076+0000 7f5bd21e0740 1 Processor -- start 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db797480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78d2f0 0x55d2db7947d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db797bb0 0x55d2db794d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2db79a780 con 0x55d2db78c9d0 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2db6adf20 con 0x55d2db78d2f0 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d2db79a900 con 0x55d2db797bb0 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bd21e0740 10 client.0 _ll_register_callbacks cb 0x55d2db677030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:47.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bcd3a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db797480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bcd3a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db797480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57080/0 (socket says 172.21.15.5:57080) 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/2153328446 learned_addr learned my addr 172.21.15.5:0/2153328446 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bcdba3640 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78d2f0 0x55d2db7947d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/2153328446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db797bb0 msgr2=0x55d2db794d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db797bb0 0x55d2db794d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/2153328446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78d2f0 msgr2=0x55d2db7947d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78d2f0 0x55d2db7947d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/2153328446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2db4642b0 con 0x55d2db78c9d0 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db797480 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8015530 tx=0x7f5bb800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8e706c30b4722b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.077+0000 7f5bb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bb17fa640 1 -- 172.21.15.5:0/2153328446 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb801b040 con 0x55d2db78c9d0 2024-09-11T22:22:47.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bb17fa640 1 -- 172.21.15.5:0/2153328446 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f5bb8016a80 con 0x55d2db78c9d0 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2153328446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb80168d0 con 0x55d2db78c9d0 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bb17fa640 1 -- 172.21.15.5:0/2153328446 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5bb80155d0 con 0x55d2db78c9d0 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bd21e0740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2153328446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2db7a6130 con 0x55d2db78c9d0 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bb17fa640 1 -- 172.21.15.5:0/2153328446 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb80168d0 con 0x55d2db78c9d0 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bb17fa640 1 -- 172.21.15.5:0/2153328446 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5bb8015070 con 0x55d2db78c9d0 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bd21e0740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bd21e0740 10 client.15528 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bd21e0740 20 client.15528 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d2db7a8410 0x55d2db7a9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bd21e0740 4 client.15528 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.078+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2153328446 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2db700600 con 0x55d2db7a8410 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.079+0000 7f5bcdba3640 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d2db7a8410 0x55d2db7a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.079+0000 7f5bb17fa640 10 client.15528 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.079+0000 7f5bcdba3640 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d2db7a8410 0x55d2db7a9880 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f807935f92b663a server_cookie=867dc09139e7fe0 in_seq=0 out_seq=0 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bb17fa640 1 -- 172.21.15.5:0/2153328446 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2db700600 con 0x55d2db7a8410 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bb17fa640 10 client.15528 handle_command_reply: tid=0 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 client.15528 shutdown 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2153328446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 msgr2=0x55d2db797480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db797480 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8015530 tx=0x7f5bb800a250 comp rx=0 tx=0).stop 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2153328446 shutdown_connections 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d2db7a8410 0x55d2db7a9880 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db797bb0 0x55d2db794d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bb17fa640 0 client.15528 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78d2f0 0x55d2db7947d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/2153328446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db797480 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2153328446 >> 172.21.15.5:0/2153328446 conn(0x55d2db788f60 msgr2=0x55d2db78ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2153328446 shutdown_connections 2024-09-11T22:22:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2153328446 wait complete. 2024-09-11T22:22:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.080+0000 7f5bd21e0740 20 client.15528 trim_cache size 0 max 16384 2024-09-11T22:22:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bd21e0740 1 Processor -- start 2024-09-11T22:22:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bd21e0740 1 -- start start 2024-09-11T22:22:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db7a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78d2f0 0x55d2db794090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db797bb0 0x55d2db783ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2db7a7bc0 con 0x55d2db78c9d0 2024-09-11T22:22:47.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2db7a8290 con 0x55d2db78d2f0 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d2db7a8960 con 0x55d2db797bb0 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bcd3a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db7a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bcd3a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db7a7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57084/0 (socket says 172.21.15.5:57084) 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/2023623138 learned_addr learned my addr 172.21.15.5:0/2023623138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bcdba3640 1 --2- 172.21.15.5:0/2023623138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78d2f0 0x55d2db794090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/2023623138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db797bb0 msgr2=0x55d2db783ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.098+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/2023623138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db797bb0 0x55d2db783ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.099+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/2023623138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78d2f0 msgr2=0x55d2db794090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.099+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/2023623138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78d2f0 0x55d2db794090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.099+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/2023623138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2db78d120 con 0x55d2db78c9d0 2024-09-11T22:22:47.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.099+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/2023623138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db7a7d40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8004700 tx=0x7f5bb800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ac3ad0d9e50f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:47.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.099+0000 7f5bb37fe640 1 -- 172.21.15.5:0/2023623138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8037070 con 0x55d2db78c9d0 2024-09-11T22:22:47.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.099+0000 7f5bb37fe640 1 -- 172.21.15.5:0/2023623138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb800a370 con 0x55d2db78c9d0 2024-09-11T22:22:47.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.099+0000 7f5bb37fe640 1 -- 172.21.15.5:0/2023623138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb801b070 con 0x55d2db78c9d0 2024-09-11T22:22:47.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2023623138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 msgr2=0x55d2db7a7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/2023623138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db7a7d40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8004700 tx=0x7f5bb800ead0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2023623138 shutdown_connections 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/2023623138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db797bb0 0x55d2db783ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/2023623138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78d2f0 0x55d2db794090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/2023623138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db78c9d0 0x55d2db7a7d40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2023623138 >> 172.21.15.5:0/2023623138 conn(0x55d2db788f60 msgr2=0x55d2db789340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2023623138 shutdown_connections 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 -- 172.21.15.5:0/2023623138 wait complete. 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 -- start start 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.100+0000 7f5bd21e0740 1 Processor -- start 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78c9d0 0x55d2db7ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db78d2f0 0x55d2db7ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bd21e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db797bb0 0x55d2db810b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2db7a9700 con 0x55d2db797bb0 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2db7a9030 con 0x55d2db78c9d0 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bd21e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d2db7a9dd0 con 0x55d2db78d2f0 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bd21e0740 10 client.0 _ll_register_callbacks cb 0x7f5bb8019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcd3a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db797bb0 0x55d2db810b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcd3a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db797bb0 0x55d2db810b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57090/0 (socket says 172.21.15.5:57090) 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/1933974237 learned_addr learned my addr 172.21.15.5:0/1933974237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcdba3640 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db78d2f0 0x55d2db7ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78c9d0 0x55d2db7ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/1933974237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db78d2f0 msgr2=0x55d2db7ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db78d2f0 0x55d2db7ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/1933974237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78c9d0 msgr2=0x55d2db7ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:47.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78c9d0 0x55d2db7ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcd3a2640 1 -- 172.21.15.5:0/1933974237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2db811050 con 0x55d2db797bb0 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bcd3a2640 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db797bb0 0x55d2db810b10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8015490 tx=0x7f5bb802abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf612400a7840eda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.101+0000 7f5bb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bb17fa640 1 -- 172.21.15.5:0/1933974237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8037070 con 0x55d2db797bb0 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bd21e0740 1 -- 172.21.15.5:0/1933974237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb800a370 con 0x55d2db797bb0 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bb17fa640 1 -- 172.21.15.5:0/1933974237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f5bb802a600 con 0x55d2db797bb0 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bb17fa640 1 -- 172.21.15.5:0/1933974237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5bb801b040 con 0x55d2db797bb0 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bd21e0740 10 client.15534 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bd21e0740 1 -- 172.21.15.5:0/1933974237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d2db8121f0 con 0x55d2db797bb0 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bb17fa640 1 -- 172.21.15.5:0/1933974237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb800a370 con 0x55d2db797bb0 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bb17fa640 1 -- 172.21.15.5:0/1933974237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5bb8048510 con 0x55d2db797bb0 2024-09-11T22:22:47.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bd21e0740 10 client.15534 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bd21e0740 10 client.15534 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bd21e0740 20 client.15534 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d2db8167b0 0x55d2db8154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bd21e0740 4 client.15534 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.102+0000 7f5bd21e0740 1 -- 172.21.15.5:0/1933974237 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "status"}) -- 0x55d2db6fab10 con 0x55d2db8167b0 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bcdba3640 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d2db8167b0 0x55d2db8154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bcdba3640 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d2db8167b0 0x55d2db8154c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5bf7988bc8a32167 server_cookie=184e95183d02fd2b in_seq=0 out_seq=0 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bb17fa640 10 client.15534 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bb17fa640 1 -- 172.21.15.5:0/1933974237 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d2db6fab10 con 0x55d2db8167b0 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bb17fa640 10 client.15534 handle_command_reply: tid=0 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bd21e0740 1 client.15534 shutdown 2024-09-11T22:22:47.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bd21e0740 1 -- 172.21.15.5:0/1933974237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db797bb0 msgr2=0x55d2db810b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db797bb0 0x55d2db810b10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8015490 tx=0x7f5bb802abe0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bd21e0740 1 -- 172.21.15.5:0/1933974237 shutdown_connections 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d2db8167b0 0x55d2db8154c0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2db797bb0 0x55d2db810b10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2db78d2f0 0x55d2db7ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bd21e0740 1 --2- 172.21.15.5:0/1933974237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2db78c9d0 0x55d2db7ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.103+0000 7f5bb17fa640 0 client.15534 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bd21e0740 1 -- 172.21.15.5:0/1933974237 >> 172.21.15.5:0/1933974237 conn(0x55d2db788f60 msgr2=0x55d2db783ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bd21e0740 1 -- 172.21.15.5:0/1933974237 shutdown_connections 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bd21e0740 1 -- 172.21.15.5:0/1933974237 wait complete. 2024-09-11T22:22:47.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bd21e0740 20 client.15534 trim_cache size 0 max 16384 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24473, 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 140.87356312700001, 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T22:22:47.110 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 170.34808233199999 2024-09-11T22:22:47.111 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bcfe07640 1 -- 172.21.15.5:0/3844701603 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5bb40a2e90 msgr2=0x7f5bb40a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bcfe07640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5bb40a2e90 0x7f5bb40a5350 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00cc280 tx=0x7f5bc000e040 comp rx=0 tx=0).stop 2024-09-11T22:22:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bcfe07640 1 -- 172.21.15.5:0/3844701603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc810c170 msgr2=0x7f5bc8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bcfe07640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc810c170 0x7f5bc8101420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000b970 tx=0x7f5bc000be30 comp rx=0 tx=0).stop 2024-09-11T22:22:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bcfe07640 1 -- 172.21.15.5:0/3844701603 shutdown_connections 2024-09-11T22:22:47.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bcfe07640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc810c170 0x7f5bc8101420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bcfe07640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5bb40a2e90 0x7f5bb40a5350 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bcfe07640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5bc8103660 0x7f5bc8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bcfe07640 1 --2- 172.21.15.5:0/3844701603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc8102d40 0x7f5bc81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.104+0000 7f5bcfe07640 1 -- 172.21.15.5:0/3844701603 >> 172.21.15.5:0/3844701603 conn(0x7f5bc80fc7a0 msgr2=0x7f5bc80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.105+0000 7f5bcfe07640 1 -- 172.21.15.5:0/3844701603 shutdown_connections 2024-09-11T22:22:47.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:47.105+0000 7f5bcfe07640 1 -- 172.21.15.5:0/3844701603 wait complete. 2024-09-11T22:22:47.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:47 smithi005 ceph-mon[30943]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T22:22:47.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:47 smithi159 ceph-mon[35534]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T22:22:47.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:47 smithi172 ceph-mon[35461]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T22:22:47.794 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-11T22:22:48.050 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:48.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.542+0000 7f296c217640 1 -- 172.21.15.5:0/1243239084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2964102570 msgr2=0x7f29641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.542+0000 7f296c217640 1 --2- 172.21.15.5:0/1243239084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2964102570 0x7f29641029b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2960009900 tx=0x7f296002a480 comp rx=0 tx=0).stop 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.542+0000 7f296c217640 1 -- 172.21.15.5:0/1243239084 shutdown_connections 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.542+0000 7f296c217640 1 --2- 172.21.15.5:0/1243239084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964102ef0 0x7f296410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.542+0000 7f296c217640 1 --2- 172.21.15.5:0/1243239084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2964102570 0x7f29641029b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.542+0000 7f296c217640 1 --2- 172.21.15.5:0/1243239084 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2964100e20 0x7f2964101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.542+0000 7f296c217640 1 -- 172.21.15.5:0/1243239084 >> 172.21.15.5:0/1243239084 conn(0x7f29640fc7d0 msgr2=0x7f29640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.542+0000 7f296c217640 1 -- 172.21.15.5:0/1243239084 shutdown_connections 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.542+0000 7f296c217640 1 -- 172.21.15.5:0/1243239084 wait complete. 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f296c217640 1 Processor -- start 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f296c217640 1 -- start start 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f296c217640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2964100e20 0x7f2964199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f296c217640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2964102570 0x7f2964199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f296c217640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964102ef0 0x7f29641a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f296c217640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2964110f80 con 0x7f2964102570 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f296c217640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2964110e00 con 0x7f2964100e20 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f296c217640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2964111100 con 0x7f2964102ef0 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f29697b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2964102570 0x7f2964199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f29697b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2964102570 0x7f2964199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57108/0 (socket says 172.21.15.5:57108) 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f29697b2640 1 -- 172.21.15.5:0/2833861170 learned_addr learned my addr 172.21.15.5:0/2833861170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f296a7b4640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964102ef0 0x7f29641a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f2969fb3640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2964100e20 0x7f2964199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f29697b2640 1 -- 172.21.15.5:0/2833861170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964102ef0 msgr2=0x7f29641a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f29697b2640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964102ef0 0x7f29641a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f29697b2640 1 -- 172.21.15.5:0/2833861170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2964100e20 msgr2=0x7f2964199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f29697b2640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2964100e20 0x7f2964199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.543+0000 7f29697b2640 1 -- 172.21.15.5:0/2833861170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29641a48f0 con 0x7f2964102570 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.544+0000 7f296a7b4640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964102ef0 0x7f29641a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.544+0000 7f29697b2640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2964102570 0x7f2964199870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2960006010 tx=0x7f2960003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.544+0000 7f2956ffd640 1 -- 172.21.15.5:0/2833861170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296003f070 con 0x7f2964102570 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.544+0000 7f296c217640 1 -- 172.21.15.5:0/2833861170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29641a4ac0 con 0x7f2964102570 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.544+0000 7f2956ffd640 1 -- 172.21.15.5:0/2833861170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2960031c30 con 0x7f2964102570 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.544+0000 7f2956ffd640 1 -- 172.21.15.5:0/2833861170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960035030 con 0x7f2964102570 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.544+0000 7f296c217640 1 -- 172.21.15.5:0/2833861170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29641a4f20 con 0x7f2964102570 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.546+0000 7f2956ffd640 1 -- 172.21.15.5:0/2833861170 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f2960042070 con 0x7f2964102570 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.546+0000 7f296e5f0740 1 Processor -- start 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f296e5f0740 1 -- start start 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b78de0 0x562d13b791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b79700 0x562d13b83a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b83fc0 0x562d13b863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f296e5f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562d13adb090 con 0x562d13b83fc0 2024-09-11T22:22:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f296e5f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562d13a82a70 con 0x562d13b79700 2024-09-11T22:22:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f296e5f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562d13ad7050 con 0x562d13b78de0 2024-09-11T22:22:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f29697b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b83fc0 0x562d13b863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f2956ffd640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29500a2e10 0x7f29500a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f29697b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b83fc0 0x562d13b863b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57124/0 (socket says 172.21.15.5:57124) 2024-09-11T22:22:48.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f29697b2640 1 -- 172.21.15.5:0/2799225623 learned_addr learned my addr 172.21.15.5:0/2799225623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f296a7b4640 1 --2- 172.21.15.5:0/2799225623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b78de0 0x562d13b791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f2969fb3640 1 --2- 172.21.15.5:0/2799225623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b79700 0x562d13b83a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f2956ffd640 1 -- 172.21.15.5:0/2833861170 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f29600efe90 con 0x7f2964102570 2024-09-11T22:22:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f296a7b4640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29500a2e10 0x7f29500a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f29697b2640 1 -- 172.21.15.5:0/2799225623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b78de0 msgr2=0x562d13b791c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f29697b2640 1 --2- 172.21.15.5:0/2799225623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b78de0 0x562d13b791c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f29697b2640 1 -- 172.21.15.5:0/2799225623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b79700 msgr2=0x562d13b83a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.547+0000 7f29697b2640 1 --2- 172.21.15.5:0/2799225623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b79700 0x562d13b83a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.548+0000 7f29697b2640 1 -- 172.21.15.5:0/2799225623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d13a8f130 con 0x562d13b83fc0 2024-09-11T22:22:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.548+0000 7f29697b2640 1 --2- 172.21.15.5:0/2799225623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b83fc0 0x562d13b863b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f29600c0120 tx=0x7f296002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340ec6e29c0e6c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.548+0000 7f296a7b4640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29500a2e10 0x7f29500a52d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f295c007fd0 tx=0x7f295c0072a0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.548+0000 7f294ffff640 1 -- 172.21.15.5:0/2799225623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960049070 con 0x562d13b83fc0 2024-09-11T22:22:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.548+0000 7f294ffff640 1 -- 172.21.15.5:0/2799225623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29600bc930 con 0x562d13b83fc0 2024-09-11T22:22:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.548+0000 7f294ffff640 1 -- 172.21.15.5:0/2799225623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29600dc0d0 con 0x562d13b83fc0 2024-09-11T22:22:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 -- 172.21.15.5:0/2799225623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b83fc0 msgr2=0x562d13b863b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 --2- 172.21.15.5:0/2799225623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b83fc0 0x562d13b863b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f29600c0120 tx=0x7f296002a480 comp rx=0 tx=0).stop 2024-09-11T22:22:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 -- 172.21.15.5:0/2799225623 shutdown_connections 2024-09-11T22:22:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 --2- 172.21.15.5:0/2799225623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b83fc0 0x562d13b863b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 --2- 172.21.15.5:0/2799225623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b79700 0x562d13b83a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 --2- 172.21.15.5:0/2799225623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b78de0 0x562d13b791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 -- 172.21.15.5:0/2799225623 >> 172.21.15.5:0/2799225623 conn(0x562d13b75370 msgr2=0x562d13b75750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 -- 172.21.15.5:0/2799225623 shutdown_connections 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 -- 172.21.15.5:0/2799225623 wait complete. 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 -- start start 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.549+0000 7f296e5f0740 1 Processor -- start 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 0x562d13b83820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b80b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b83fc0 0x562d13b810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f296e5f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562d13b86950 con 0x562d13b79700 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f296e5f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562d13a95f20 con 0x562d13b78de0 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f296e5f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562d13b86ad0 con 0x562d13b83fc0 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f296e5f0740 10 client.0 _ll_register_callbacks cb 0x562d13a5f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f29697b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b80b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f29697b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b80b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57136/0 (socket says 172.21.15.5:57136) 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f29697b2640 1 -- 172.21.15.5:0/989488703 learned_addr learned my addr 172.21.15.5:0/989488703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f2969fb3640 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 0x562d13b83820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f296a7b4640 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b83fc0 0x562d13b810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f29697b2640 1 -- 172.21.15.5:0/989488703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b83fc0 msgr2=0x562d13b810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f29697b2640 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b83fc0 0x562d13b810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f29697b2640 1 -- 172.21.15.5:0/989488703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 msgr2=0x562d13b83820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f29697b2640 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 0x562d13b83820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f29697b2640 1 -- 172.21.15.5:0/989488703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d1384c2b0 con 0x562d13b79700 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f29697b2640 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b80b70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f29600b8690 tx=0x7f296003b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55c2f9f9908306b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f294d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.550+0000 7f294d7fa640 1 -- 172.21.15.5:0/989488703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960049070 con 0x562d13b79700 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f294d7fa640 1 -- 172.21.15.5:0/989488703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f29600bc6a0 con 0x562d13b79700 2024-09-11T22:22:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f296e5f0740 1 -- 172.21.15.5:0/989488703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29600bc930 con 0x562d13b79700 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f294d7fa640 1 -- 172.21.15.5:0/989488703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2960031de0 con 0x562d13b79700 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f296e5f0740 10 client.15546 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f296e5f0740 1 -- 172.21.15.5:0/989488703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562d13b86950 con 0x562d13b79700 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f294d7fa640 1 -- 172.21.15.5:0/989488703 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29600bc930 con 0x562d13b79700 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f294d7fa640 1 -- 172.21.15.5:0/989488703 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f29600ed740 con 0x562d13b79700 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f296e5f0740 10 client.15546 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f296e5f0740 10 client.15546 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f296e5f0740 20 client.15546 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f296e5f0740 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562d13b94610 0x562d13b95a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f296e5f0740 4 client.15546 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.551+0000 7f296e5f0740 1 -- 172.21.15.5:0/989488703 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d13ae8600 con 0x562d13b94610 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.552+0000 7f2969fb3640 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562d13b94610 0x562d13b95a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.553+0000 7f2969fb3640 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562d13b94610 0x562d13b95a80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b141d7d86e7ef118 server_cookie=5571d82274837fb7 in_seq=0 out_seq=0 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.553+0000 7f294d7fa640 10 client.15546 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.553+0000 7f294d7fa640 1 -- 172.21.15.5:0/989488703 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d13ae8600 con 0x562d13b94610 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.553+0000 7f294d7fa640 10 client.15546 handle_command_reply: tid=0 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.553+0000 7f296e5f0740 1 client.15546 shutdown 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 1 -- 172.21.15.5:0/989488703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 msgr2=0x562d13b80b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b80b70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f29600b8690 tx=0x7f296003b690 comp rx=0 tx=0).stop 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 1 -- 172.21.15.5:0/989488703 shutdown_connections 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562d13b94610 0x562d13b95a80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b83fc0 0x562d13b810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b80b70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f294d7fa640 0 client.15546 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 1 --2- 172.21.15.5:0/989488703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 0x562d13b83820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 1 -- 172.21.15.5:0/989488703 >> 172.21.15.5:0/989488703 conn(0x562d13b75370 msgr2=0x562d13b7b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 1 -- 172.21.15.5:0/989488703 shutdown_connections 2024-09-11T22:22:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 1 -- 172.21.15.5:0/989488703 wait complete. 2024-09-11T22:22:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.554+0000 7f296e5f0740 20 client.15546 trim_cache size 0 max 16384 2024-09-11T22:22:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f296e5f0740 1 Processor -- start 2024-09-11T22:22:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f296e5f0740 1 -- start start 2024-09-11T22:22:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 0x562d13b95a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b7f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b83fc0 0x562d13b7c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f296e5f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562d13b93dc0 con 0x562d13b79700 2024-09-11T22:22:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f296e5f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562d13b94490 con 0x562d13b78de0 2024-09-11T22:22:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f296e5f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562d13b94b60 con 0x562d13b83fc0 2024-09-11T22:22:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f29697b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b7f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f29697b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b7f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57152/0 (socket says 172.21.15.5:57152) 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f29697b2640 1 -- 172.21.15.5:0/2087020345 learned_addr learned my addr 172.21.15.5:0/2087020345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f2969fb3640 1 --2- 172.21.15.5:0/2087020345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 0x562d13b95a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f29697b2640 1 -- 172.21.15.5:0/2087020345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b83fc0 msgr2=0x562d13b7c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f29697b2640 1 --2- 172.21.15.5:0/2087020345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b83fc0 0x562d13b7c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f29697b2640 1 -- 172.21.15.5:0/2087020345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 msgr2=0x562d13b95a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f29697b2640 1 --2- 172.21.15.5:0/2087020345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 0x562d13b95a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f29697b2640 1 -- 172.21.15.5:0/2087020345 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d13b7fb20 con 0x562d13b79700 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.572+0000 7f2969fb3640 1 --2- 172.21.15.5:0/2087020345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 0x562d13b95a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.573+0000 7f29697b2640 1 --2- 172.21.15.5:0/2087020345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b7f440 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f29600ed570 tx=0x7f296003b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db830342fe8dac05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.573+0000 7f294f7fe640 1 -- 172.21.15.5:0/2087020345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2960049070 con 0x562d13b79700 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.573+0000 7f294f7fe640 1 -- 172.21.15.5:0/2087020345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29600bc930 con 0x562d13b79700 2024-09-11T22:22:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.573+0000 7f294f7fe640 1 -- 172.21.15.5:0/2087020345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29600ed740 con 0x562d13b79700 2024-09-11T22:22:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 -- 172.21.15.5:0/2087020345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 msgr2=0x562d13b7f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 --2- 172.21.15.5:0/2087020345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b7f440 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f29600ed570 tx=0x7f296003b870 comp rx=0 tx=0).stop 2024-09-11T22:22:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 -- 172.21.15.5:0/2087020345 shutdown_connections 2024-09-11T22:22:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 --2- 172.21.15.5:0/2087020345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b83fc0 0x562d13b7c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 --2- 172.21.15.5:0/2087020345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b79700 0x562d13b7f440 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 --2- 172.21.15.5:0/2087020345 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b78de0 0x562d13b95a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 -- 172.21.15.5:0/2087020345 >> 172.21.15.5:0/2087020345 conn(0x562d13b75370 msgr2=0x562d13b7b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 -- 172.21.15.5:0/2087020345 shutdown_connections 2024-09-11T22:22:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 -- 172.21.15.5:0/2087020345 wait complete. 2024-09-11T22:22:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 -- start start 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 Processor -- start 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b78de0 0x562d13b7d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b79700 0x562d13b7daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.574+0000 7f296e5f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b83fc0 0x562d13b9ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f296e5f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562d13b95900 con 0x562d13b78de0 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f296e5f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562d13b95230 con 0x562d13b83fc0 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f296e5f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562d13b95fd0 con 0x562d13b79700 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f296e5f0740 10 client.0 _ll_register_callbacks cb 0x7f29600c0670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f2969fb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b78de0 0x562d13b7d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f2969fb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b78de0 0x562d13b7d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57154/0 (socket says 172.21.15.5:57154) 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f2969fb3640 1 -- 172.21.15.5:0/1359013334 learned_addr learned my addr 172.21.15.5:0/1359013334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f29697b2640 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b79700 0x562d13b7daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f2969fb3640 1 -- 172.21.15.5:0/1359013334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b79700 msgr2=0x562d13b7daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f2969fb3640 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b79700 0x562d13b7daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f2969fb3640 1 -- 172.21.15.5:0/1359013334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b83fc0 msgr2=0x562d13b9ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f2969fb3640 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b83fc0 0x562d13b9ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f2969fb3640 1 -- 172.21.15.5:0/1359013334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d13b9d210 con 0x562d13b78de0 2024-09-11T22:22:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f2969fb3640 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b78de0 0x562d13b7d560 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f29580045f0 tx=0x7f295800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613d4e88faea38f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f294d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f294d7fa640 1 -- 172.21.15.5:0/1359013334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958003eb0 con 0x562d13b78de0 2024-09-11T22:22:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f296e5f0740 1 -- 172.21.15.5:0/1359013334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29600bc930 con 0x562d13b78de0 2024-09-11T22:22:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f294d7fa640 1 -- 172.21.15.5:0/1359013334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f295801d420 con 0x562d13b78de0 2024-09-11T22:22:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f294d7fa640 1 -- 172.21.15.5:0/1359013334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f295802d050 con 0x562d13b78de0 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.575+0000 7f296e5f0740 10 client.15552 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.576+0000 7f296e5f0740 1 -- 172.21.15.5:0/1359013334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562d13b9e570 con 0x562d13b78de0 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.576+0000 7f294d7fa640 1 -- 172.21.15.5:0/1359013334 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29600bc930 con 0x562d13b78de0 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.576+0000 7f294d7fa640 1 -- 172.21.15.5:0/1359013334 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2958011930 con 0x562d13b78de0 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.576+0000 7f296e5f0740 10 client.15552 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.576+0000 7f296e5f0740 10 client.15552 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.576+0000 7f296e5f0740 20 client.15552 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.576+0000 7f296e5f0740 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562d13c03320 0x562d13c02030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.576+0000 7f296e5f0740 4 client.15552 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.576+0000 7f296e5f0740 1 -- 172.21.15.5:0/1359013334 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "status"}) -- 0x562d13ae2b10 con 0x562d13c03320 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.576+0000 7f29697b2640 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562d13c03320 0x562d13c02030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f29697b2640 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562d13c03320 0x562d13c02030 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69768338219e5fe server_cookie=43d404ee99d0c49d in_seq=0 out_seq=0 2024-09-11T22:22:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f294d7fa640 10 client.15552 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f294d7fa640 1 -- 172.21.15.5:0/1359013334 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562d13ae2b10 con 0x562d13c03320 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f294d7fa640 10 client.15552 handle_command_reply: tid=0 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 client.15552 shutdown 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 -- 172.21.15.5:0/1359013334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b78de0 msgr2=0x562d13b7d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b78de0 0x562d13b7d560 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f29580045f0 tx=0x7f295800bd30 comp rx=0 tx=0).stop 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 -- 172.21.15.5:0/1359013334 shutdown_connections 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562d13c03320 0x562d13c02030 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d13b83fc0 0x562d13b9ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562d13b79700 0x562d13b7daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 --2- 172.21.15.5:0/1359013334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d13b78de0 0x562d13b7d560 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f294d7fa640 0 client.15552 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 -- 172.21.15.5:0/1359013334 >> 172.21.15.5:0/1359013334 conn(0x562d13b75370 msgr2=0x562d13b7b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 -- 172.21.15.5:0/1359013334 shutdown_connections 2024-09-11T22:22:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 1 -- 172.21.15.5:0/1359013334 wait complete. 2024-09-11T22:22:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.577+0000 7f296e5f0740 20 client.15552 trim_cache size 0 max 16384 2024-09-11T22:22:48.593 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "b99b16b2-708a-11ef-bcea-c7b262605968", 2024-09-11T22:22:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-11T22:22:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24500, 2024-09-11T22:22:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-11T22:22:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-11T22:22:48.593 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-11T22:22:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 149.398336847, 2024-09-11T22:22:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 28, 2024-09-11T22:22:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-11T22:22:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 82, 2024-09-11T22:22:48.594 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 159.80665231500001 2024-09-11T22:22:48.594 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 -- 172.21.15.5:0/2833861170 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29500a2e10 msgr2=0x7f29500a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29500a2e10 0x7f29500a52d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f295c007fd0 tx=0x7f295c0072a0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 -- 172.21.15.5:0/2833861170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2964102570 msgr2=0x7f2964199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2964102570 0x7f2964199870 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2960006010 tx=0x7f2960003c10 comp rx=0 tx=0).stop 2024-09-11T22:22:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 -- 172.21.15.5:0/2833861170 shutdown_connections 2024-09-11T22:22:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2964102ef0 0x7f29641a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2964102570 0x7f2964199870 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29500a2e10 0x7f29500a52d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 --2- 172.21.15.5:0/2833861170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2964100e20 0x7f2964199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 -- 172.21.15.5:0/2833861170 >> 172.21.15.5:0/2833861170 conn(0x7f29640fc7d0 msgr2=0x7f29640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 -- 172.21.15.5:0/2833861170 shutdown_connections 2024-09-11T22:22:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:48.578+0000 7f296c217640 1 -- 172.21.15.5:0/2833861170 wait complete. 2024-09-11T22:22:49.282 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:49 smithi005 ceph-mon[30943]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T22:22:49.315 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-11T22:22:49.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:49 smithi159 ceph-mon[35534]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T22:22:49.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:49 smithi172 ceph-mon[35461]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T22:22:51.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:51 smithi005 ceph-mon[30943]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T22:22:51.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:51 smithi159 ceph-mon[35534]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T22:22:51.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:51 smithi172 ceph-mon[35461]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 13 op/s 2024-09-11T22:22:53.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:53 smithi005 ceph-mon[30943]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-11T22:22:53.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:53 smithi159 ceph-mon[35534]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-11T22:22:53.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:53 smithi172 ceph-mon[35461]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.7 KiB/s wr, 14 op/s 2024-09-11T22:22:54.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.122+0000 7ff9ca591640 1 -- 172.21.15.5:0/3672695464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c41069c0 msgr2=0x7ff9c4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.122+0000 7ff9ca591640 1 --2- 172.21.15.5:0/3672695464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c41069c0 0x7ff9c4106da0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009930 tx=0x7ff9b402a420 comp rx=0 tx=0).stop 2024-09-11T22:22:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.122+0000 7ff9ca591640 1 -- 172.21.15.5:0/3672695464 shutdown_connections 2024-09-11T22:22:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.122+0000 7ff9ca591640 1 --2- 172.21.15.5:0/3672695464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c4101350 0x7ff9c410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.122+0000 7ff9ca591640 1 --2- 172.21.15.5:0/3672695464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c41009b0 0x7ff9c4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.122+0000 7ff9ca591640 1 --2- 172.21.15.5:0/3672695464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c41069c0 0x7ff9c4106da0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.122+0000 7ff9ca591640 1 -- 172.21.15.5:0/3672695464 >> 172.21.15.5:0/3672695464 conn(0x7ff9c4077990 msgr2=0x7ff9c40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:54.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.122+0000 7ff9ca591640 1 -- 172.21.15.5:0/3672695464 shutdown_connections 2024-09-11T22:22:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.122+0000 7ff9ca591640 1 -- 172.21.15.5:0/3672695464 wait complete. 2024-09-11T22:22:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.123+0000 7ff9ca591640 1 Processor -- start 2024-09-11T22:22:54.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.123+0000 7ff9ca591640 1 -- start start 2024-09-11T22:22:54.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.123+0000 7ff9ca591640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c41009b0 0x7ff9c41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.123+0000 7ff9ca591640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c4101350 0x7ff9c4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.123+0000 7ff9ca591640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c41069c0 0x7ff9c419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.123+0000 7ff9ca591640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4110e00 con 0x7ff9c4101350 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.123+0000 7ff9ca591640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4110c80 con 0x7ff9c41069c0 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.123+0000 7ff9ca591640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4110f80 con 0x7ff9c41009b0 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c4101350 0x7ff9c4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c4101350 0x7ff9c4198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60592/0 (socket says 172.21.15.5:60592) 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c37fe640 1 -- 172.21.15.5:0/3911603993 learned_addr learned my addr 172.21.15.5:0/3911603993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c41009b0 0x7ff9c41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c8b2e640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c41069c0 0x7ff9c419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c37fe640 1 -- 172.21.15.5:0/3911603993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c41009b0 msgr2=0x7ff9c41983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c41009b0 0x7ff9c41983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c37fe640 1 -- 172.21.15.5:0/3911603993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c41069c0 msgr2=0x7ff9c419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c41069c0 0x7ff9c419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c37fe640 1 -- 172.21.15.5:0/3911603993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c419d3b0 con 0x7ff9c4101350 2024-09-11T22:22:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c8b2e640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c41069c0 0x7ff9c419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c4101350 0x7ff9c4198900 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800d770 tx=0x7ff9b800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c17fa640 1 -- 172.21.15.5:0/3911603993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8017070 con 0x7ff9c4101350 2024-09-11T22:22:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c17fa640 1 -- 172.21.15.5:0/3911603993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b8005500 con 0x7ff9c4101350 2024-09-11T22:22:54.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9ca591640 1 -- 172.21.15.5:0/3911603993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c419d6a0 con 0x7ff9c4101350 2024-09-11T22:22:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9c17fa640 1 -- 172.21.15.5:0/3911603993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8004e00 con 0x7ff9c4101350 2024-09-11T22:22:54.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.124+0000 7ff9ca591640 1 -- 172.21.15.5:0/3911603993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41a4f30 con 0x7ff9c4101350 2024-09-11T22:22:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c17fa640 1 -- 172.21.15.5:0/3911603993 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7ff9b8004fa0 con 0x7ff9c4101350 2024-09-11T22:22:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9cd40b740 1 Processor -- start 2024-09-11T22:22:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9cd40b740 1 -- start start 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f3af0 0x562ceb0f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb1010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562ceb0582f0 con 0x562ceb0f4410 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ceb000270 con 0x562ceb0fecb0 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562ceb0542b0 con 0x562ceb0f3af0 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fe770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60600/0 (socket says 172.21.15.5:60600) 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c3fff640 1 -- 172.21.15.5:0/3744396183 learned_addr learned my addr 172.21.15.5:0/3744396183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/3744396183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb1010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c3fff640 1 -- 172.21.15.5:0/3744396183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f3af0 msgr2=0x562ceb0f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/3744396183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f3af0 0x562ceb0f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c3fff640 1 -- 172.21.15.5:0/3744396183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 msgr2=0x562ceb1010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/3744396183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb1010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c17fa640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9ac0a2ec0 0x7ff9ac0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c3fff640 1 -- 172.21.15.5:0/3744396183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ceafff340 con 0x562ceb0f4410 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.128+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/3744396183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.129+0000 7ff9c8b2e640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9ac0a2ec0 0x7ff9ac0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.129+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/3744396183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fe770 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff9b402a940 tx=0x7ff9b4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd74951eac3edce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.129+0000 7ff9c17fa640 1 -- 172.21.15.5:0/3911603993 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff9b80cf180 con 0x7ff9c4101350 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.129+0000 7ff9aa7fc640 1 -- 172.21.15.5:0/3744396183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4040070 con 0x562ceb0f4410 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.129+0000 7ff9aa7fc640 1 -- 172.21.15.5:0/3744396183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b40319f0 con 0x562ceb0f4410 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.129+0000 7ff9aa7fc640 1 -- 172.21.15.5:0/3744396183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4035040 con 0x562ceb0f4410 2024-09-11T22:22:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 -- 172.21.15.5:0/3744396183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 msgr2=0x562ceb0fe770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9c8b2e640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9ac0a2ec0 0x7ff9ac0a5380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0092f0 tx=0x7ff9bc0093c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/3744396183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fe770 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff9b402a940 tx=0x7ff9b4005c00 comp rx=0 tx=0).stop 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 -- 172.21.15.5:0/3744396183 shutdown_connections 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/3744396183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb1010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/3744396183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fe770 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/3744396183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f3af0 0x562ceb0f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 -- 172.21.15.5:0/3744396183 >> 172.21.15.5:0/3744396183 conn(0x562ceb0f0060 msgr2=0x562ceb0f0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 -- 172.21.15.5:0/3744396183 shutdown_connections 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 -- 172.21.15.5:0/3744396183 wait complete. 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 -- start start 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.130+0000 7ff9cd40b740 1 Processor -- start 2024-09-11T22:22:54.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f3af0 0x562ceb0fba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f4410 0x562ceb0fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb0fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562ceb101880 con 0x562ceb0f3af0 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562ceafe1290 con 0x562ceb0fecb0 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562ceb101a00 con 0x562ceb0f4410 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9cd40b740 10 client.0 _ll_register_callbacks cb 0x562ceb012140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f3af0 0x562ceb0fba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f3af0 0x562ceb0fba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60612/0 (socket says 172.21.15.5:60612) 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c3fff640 1 -- 172.21.15.5:0/4182198142 learned_addr learned my addr 172.21.15.5:0/4182198142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f4410 0x562ceb0fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c8b2e640 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb0fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c3fff640 1 -- 172.21.15.5:0/4182198142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f4410 msgr2=0x562ceb0fbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f4410 0x562ceb0fbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c3fff640 1 -- 172.21.15.5:0/4182198142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 msgr2=0x562ceb0fc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb0fc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c3fff640 1 -- 172.21.15.5:0/4182198142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cead6b210 con 0x562ceb0f3af0 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c8b2e640 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb0fc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f4410 0x562ceb0fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f3af0 0x562ceb0fba50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40052d0 tx=0x7ff9b402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeea934e19323f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9a37fe640 1 -- 172.21.15.5:0/4182198142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4040030 con 0x562ceb0f3af0 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9a37fe640 1 -- 172.21.15.5:0/4182198142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff9b4004260 con 0x562ceb0f3af0 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.131+0000 7ff9cd40b740 1 -- 172.21.15.5:0/4182198142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b40319f0 con 0x562ceb0f3af0 2024-09-11T22:22:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9a37fe640 1 -- 172.21.15.5:0/4182198142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9b4056050 con 0x562ceb0f3af0 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9cd40b740 10 client.15564 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9cd40b740 1 -- 172.21.15.5:0/4182198142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ceae37620 con 0x562ceb0f3af0 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9a37fe640 1 -- 172.21.15.5:0/4182198142 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b40319f0 con 0x562ceb0f3af0 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9a37fe640 1 -- 172.21.15.5:0/4182198142 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff9b40038f0 con 0x562ceb0f3af0 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9cd40b740 10 client.15564 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9cd40b740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9cd40b740 20 client.15564 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562ceb10e8d0 0x562ceb110cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9cd40b740 4 client.15564 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.132+0000 7ff9cd40b740 1 -- 172.21.15.5:0/4182198142 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ceb065a20 con 0x562ceb10e8d0 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.133+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562ceb10e8d0 0x562ceb110cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.133+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562ceb10e8d0 0x562ceb110cc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8153a7f732dc0e server_cookie=5d359e45617eb6cc in_seq=0 out_seq=0 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.133+0000 7ff9a37fe640 10 client.15564 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9a37fe640 1 -- 172.21.15.5:0/4182198142 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ceb065a20 con 0x562ceb10e8d0 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9a37fe640 10 client.15564 handle_command_reply: tid=0 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 client.15564 shutdown 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 -- 172.21.15.5:0/4182198142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f3af0 msgr2=0x562ceb0fba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f3af0 0x562ceb0fba50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40052d0 tx=0x7ff9b402a420 comp rx=0 tx=0).stop 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 -- 172.21.15.5:0/4182198142 shutdown_connections 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562ceb10e8d0 0x562ceb110cc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb0fc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f4410 0x562ceb0fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9a37fe640 0 client.15564 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/4182198142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f3af0 0x562ceb0fba50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 -- 172.21.15.5:0/4182198142 >> 172.21.15.5:0/4182198142 conn(0x562ceb0f0060 msgr2=0x562ceb0f5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 -- 172.21.15.5:0/4182198142 shutdown_connections 2024-09-11T22:22:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 1 -- 172.21.15.5:0/4182198142 wait complete. 2024-09-11T22:22:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.134+0000 7ff9cd40b740 20 client.15564 trim_cache size 0 max 16384 2024-09-11T22:22:54.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9cd40b740 1 Processor -- start 2024-09-11T22:22:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9cd40b740 1 -- start start 2024-09-11T22:22:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f3af0 0x562ceb0fa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb1000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41ad7a0 con 0x562ceb0f4410 2024-09-11T22:22:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41af2e0 con 0x562ceb0fecb0 2024-09-11T22:22:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41ade70 con 0x562ceb0f3af0 2024-09-11T22:22:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60616/0 (socket says 172.21.15.5:60616) 2024-09-11T22:22:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9c37fe640 1 -- 172.21.15.5:0/3075493887 learned_addr learned my addr 172.21.15.5:0/3075493887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9c8b2e640 1 --2- 172.21.15.5:0/3075493887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb1000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/3075493887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f3af0 0x562ceb0fa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9c37fe640 1 -- 172.21.15.5:0/3075493887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f3af0 msgr2=0x562ceb0fa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.149+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/3075493887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f3af0 0x562ceb0fa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.150+0000 7ff9c37fe640 1 -- 172.21.15.5:0/3075493887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 msgr2=0x562ceb1000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.150+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/3075493887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb1000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.150+0000 7ff9c37fe640 1 -- 172.21.15.5:0/3075493887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ceb0efe90 con 0x562ceb0f4410 2024-09-11T22:22:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.150+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/3075493887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f3af0 0x562ceb0fa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.150+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/3075493887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fc300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff9b809b3e0 tx=0x7ff9b8009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7b895652fe4b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.150+0000 7ff9a9ffb640 1 -- 172.21.15.5:0/3075493887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8020040 con 0x562ceb0f4410 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.150+0000 7ff9a9ffb640 1 -- 172.21.15.5:0/3075493887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b809f800 con 0x562ceb0f4410 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.150+0000 7ff9a9ffb640 1 -- 172.21.15.5:0/3075493887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8097ac0 con 0x562ceb0f4410 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 -- 172.21.15.5:0/3075493887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 msgr2=0x562ceb0fc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/3075493887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fc300 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff9b809b3e0 tx=0x7ff9b8009740 comp rx=0 tx=0).stop 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 -- 172.21.15.5:0/3075493887 shutdown_connections 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/3075493887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0fecb0 0x562ceb1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/3075493887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb0fc300 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/3075493887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0f3af0 0x562ceb0fa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 -- 172.21.15.5:0/3075493887 >> 172.21.15.5:0/3075493887 conn(0x562ceb0f0060 msgr2=0x562ceb0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 -- 172.21.15.5:0/3075493887 shutdown_connections 2024-09-11T22:22:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 -- 172.21.15.5:0/3075493887 wait complete. 2024-09-11T22:22:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 -- start start 2024-09-11T22:22:54.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.151+0000 7ff9cd40b740 1 Processor -- start 2024-09-11T22:22:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0f3af0 0x562ceb10fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb10ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9cd40b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0fecb0 0x562ceb110510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41aec10 con 0x562ceb0f4410 2024-09-11T22:22:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41ae540 con 0x562ceb0f3af0 2024-09-11T22:22:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9cd40b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41ab780 con 0x562ceb0fecb0 2024-09-11T22:22:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9cd40b740 10 client.0 _ll_register_callbacks cb 0x7ff9b40095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb10ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb10ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60628/0 (socket says 172.21.15.5:60628) 2024-09-11T22:22:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c37fe640 1 -- 172.21.15.5:0/2948642856 learned_addr learned my addr 172.21.15.5:0/2948642856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0f3af0 0x562ceb10fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c8b2e640 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0fecb0 0x562ceb110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c37fe640 1 -- 172.21.15.5:0/2948642856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0fecb0 msgr2=0x562ceb110510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0fecb0 0x562ceb110510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c37fe640 1 -- 172.21.15.5:0/2948642856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0f3af0 msgr2=0x562ceb10fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0f3af0 0x562ceb10fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c37fe640 1 -- 172.21.15.5:0/2948642856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ceb178250 con 0x562ceb0f4410 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0f3af0 0x562ceb10fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9c37fe640 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb10ffd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8097e20 tx=0x7ff9b80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4ac8ae7e5209cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9a37fe640 1 -- 172.21.15.5:0/2948642856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8020050 con 0x562ceb0f4410 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9a37fe640 1 -- 172.21.15.5:0/2948642856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7ff9b8097ac0 con 0x562ceb0f4410 2024-09-11T22:22:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.152+0000 7ff9cd40b740 1 -- 172.21.15.5:0/2948642856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9b809f800 con 0x562ceb0f4410 2024-09-11T22:22:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9a37fe640 1 -- 172.21.15.5:0/2948642856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff9b8099750 con 0x562ceb0f4410 2024-09-11T22:22:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9cd40b740 10 client.15570 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9cd40b740 1 -- 172.21.15.5:0/2948642856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562ceb1794b0 con 0x562ceb0f4410 2024-09-11T22:22:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9a37fe640 1 -- 172.21.15.5:0/2948642856 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b809f800 con 0x562ceb0f4410 2024-09-11T22:22:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9a37fe640 1 -- 172.21.15.5:0/2948642856 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff9b80b40c0 con 0x562ceb0f4410 2024-09-11T22:22:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9cd40b740 10 client.15570 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9cd40b740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:22:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9cd40b740 20 client.15570 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:54.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562ceb179c50 0x562ceb17c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9cd40b740 4 client.15570 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:22:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9cd40b740 1 -- 172.21.15.5:0/2948642856 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562ceb05fd70 con 0x562ceb179c50 2024-09-11T22:22:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.153+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562ceb179c50 0x562ceb17c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.154+0000 7ff9c3fff640 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562ceb179c50 0x562ceb17c070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baef9a3c253257ee server_cookie=64b7ad0a1aa5d272 in_seq=0 out_seq=0 2024-09-11T22:22:54.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.154+0000 7ff9a37fe640 10 client.15570 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.159+0000 7ff9a37fe640 1 -- 172.21.15.5:0/2948642856 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562ceb05fd70 con 0x562ceb179c50 2024-09-11T22:22:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.159+0000 7ff9a37fe640 10 client.15570 handle_command_reply: tid=0 2024-09-11T22:22:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.159+0000 7ff9cd40b740 1 client.15570 shutdown 2024-09-11T22:22:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 1 -- 172.21.15.5:0/2948642856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 msgr2=0x562ceb10ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb10ffd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8097e20 tx=0x7ff9b80057b0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 1 -- 172.21.15.5:0/2948642856 shutdown_connections 2024-09-11T22:22:54.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562ceb179c50 0x562ceb17c070 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562ceb0fecb0 0x562ceb110510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ceb0f4410 0x562ceb10ffd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 1 --2- 172.21.15.5:0/2948642856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562ceb0f3af0 0x562ceb10fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9a37fe640 0 client.15570 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 1 -- 172.21.15.5:0/2948642856 >> 172.21.15.5:0/2948642856 conn(0x562ceb0f0060 msgr2=0x562ceb0ee9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 1 -- 172.21.15.5:0/2948642856 shutdown_connections 2024-09-11T22:22:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 1 -- 172.21.15.5:0/2948642856 wait complete. 2024-09-11T22:22:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9cd40b740 20 client.15570 trim_cache size 0 max 16384 2024-09-11T22:22:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:22:54.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:22:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:22:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:22:54.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9ca591640 1 -- 172.21.15.5:0/3911603993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9ac0a2ec0 msgr2=0x7ff9ac0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9ca591640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9ac0a2ec0 0x7ff9ac0a5380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0092f0 tx=0x7ff9bc0093c0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9ca591640 1 -- 172.21.15.5:0/3911603993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c4101350 msgr2=0x7ff9c4198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9ca591640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c4101350 0x7ff9c4198900 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800d770 tx=0x7ff9b800dc30 comp rx=0 tx=0).stop 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9ca591640 1 -- 172.21.15.5:0/3911603993 shutdown_connections 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9ca591640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9ac0a2ec0 0x7ff9ac0a5380 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9ca591640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c41069c0 0x7ff9c419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9ca591640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c4101350 0x7ff9c4198900 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9ca591640 1 --2- 172.21.15.5:0/3911603993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c41009b0 0x7ff9c41983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.160+0000 7ff9ca591640 1 -- 172.21.15.5:0/3911603993 >> 172.21.15.5:0/3911603993 conn(0x7ff9c4077990 msgr2=0x7ff9c4069b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.161+0000 7ff9ca591640 1 -- 172.21.15.5:0/3911603993 shutdown_connections 2024-09-11T22:22:54.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:22:54.161+0000 7ff9ca591640 1 -- 172.21.15.5:0/3911603993 wait complete. 2024-09-11T22:22:55.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:55 smithi005 ceph-mon[30943]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T22:22:55.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:55 smithi159 ceph-mon[35534]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T22:22:55.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:55 smithi172 ceph-mon[35461]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 17 op/s 2024-09-11T22:22:57.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:57 smithi005 ceph-mon[30943]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:57.458 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:57 smithi172 ceph-mon[35461]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:57.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:57 smithi159 ceph-mon[35534]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:22:58.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:22:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:22:59.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.092+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2474800032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f8100100 msgr2=0x7f39f8100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:59.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.092+0000 7f39ff0e3640 1 --2- 172.21.15.5:0/2474800032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f8100100 0x7f39f8100560 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009900 tx=0x7f39e402a3f0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.092+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2474800032 shutdown_connections 2024-09-11T22:22:59.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.092+0000 7f39ff0e3640 1 --2- 172.21.15.5:0/2474800032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f8100aa0 0x7f39f810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.092+0000 7f39ff0e3640 1 --2- 172.21.15.5:0/2474800032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f8100100 0x7f39f8100560 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.092+0000 7f39ff0e3640 1 --2- 172.21.15.5:0/2474800032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39f8106b80 0x7f39f8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.092+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2474800032 >> 172.21.15.5:0/2474800032 conn(0x7f39f8077230 msgr2=0x7f39f80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.092+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2474800032 shutdown_connections 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.092+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2474800032 wait complete. 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39ff0e3640 1 Processor -- start 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39ff0e3640 1 -- start start 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39ff0e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f8100100 0x7f39f8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39ff0e3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f8100aa0 0x7f39f8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39ff0e3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39f8106b80 0x7f39f819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39ff0e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81120b0 con 0x7f39f8100100 2024-09-11T22:22:59.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39ff0e3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39f8111f30 con 0x7f39f8100aa0 2024-09-11T22:22:59.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39ff0e3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39f8112230 con 0x7f39f8106b80 2024-09-11T22:22:59.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39fce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f8100100 0x7f39f8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39fce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f8100100 0x7f39f8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60648/0 (socket says 172.21.15.5:60648) 2024-09-11T22:22:59.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39fce7f640 1 -- 172.21.15.5:0/2606097483 learned_addr learned my addr 172.21.15.5:0/2606097483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:59.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.093+0000 7f39fd680640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39f8106b80 0x7f39f819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39f7fff640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f8100aa0 0x7f39f8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39fce7f640 1 -- 172.21.15.5:0/2606097483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39f8106b80 msgr2=0x7f39f819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39fce7f640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39f8106b80 0x7f39f819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39fce7f640 1 -- 172.21.15.5:0/2606097483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f8100aa0 msgr2=0x7f39f8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39fce7f640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f8100aa0 0x7f39f8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39fce7f640 1 -- 172.21.15.5:0/2606097483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f819d500 con 0x7f39f8100100 2024-09-11T22:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39fce7f640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f8100100 0x7f39f8198550 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c970 tx=0x7f39ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39f5ffb640 1 -- 172.21.15.5:0/2606097483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec007bb0 con 0x7f39f8100100 2024-09-11T22:22:59.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39f5ffb640 1 -- 172.21.15.5:0/2606097483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ec007d50 con 0x7f39f8100100 2024-09-11T22:22:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2606097483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f819d7f0 con 0x7f39f8100100 2024-09-11T22:22:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.094+0000 7f39f5ffb640 1 -- 172.21.15.5:0/2606097483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00f450 con 0x7f39f8100100 2024-09-11T22:22:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.095+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2606097483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f81a50d0 con 0x7f39f8100100 2024-09-11T22:22:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.097+0000 7f39f5ffb640 1 -- 172.21.15.5:0/2606097483 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f39ec005020 con 0x7f39f8100100 2024-09-11T22:22:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f39f5ffb640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39e80a2e10 0x7f39e80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f3a014bc740 1 Processor -- start 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f39f7fff640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39e80a2e10 0x7f39e80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f3a014bc740 1 -- start start 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f39f5ffb640 1 -- 172.21.15.5:0/2606097483 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f39ec0cf540 con 0x7f39f8100100 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b58650 0x564184b58a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b59000 0x564184b53050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b53670 0x564184b53af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f3a014bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564184aaf2e0 con 0x564184b58650 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f3a014bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564184a56ce0 con 0x564184b53670 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f3a014bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564184aab2a0 con 0x564184b59000 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.099+0000 7f39fce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b58650 0x564184b58a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39fce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b58650 0x564184b58a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60656/0 (socket says 172.21.15.5:60656) 2024-09-11T22:22:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39fce7f640 1 -- 172.21.15.5:0/3197492579 learned_addr learned my addr 172.21.15.5:0/3197492579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39f7fff640 1 --2- 172.21.15.5:0/3197492579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b59000 0x564184b53050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39fd680640 1 --2- 172.21.15.5:0/3197492579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b53670 0x564184b53af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39f7fff640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39e80a2e10 0x7f39e80a52d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f39f80ff7f0 tx=0x7f39e4031490 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39fce7f640 1 -- 172.21.15.5:0/3197492579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b59000 msgr2=0x564184b53050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39fce7f640 1 --2- 172.21.15.5:0/3197492579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b59000 0x564184b53050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39fce7f640 1 -- 172.21.15.5:0/3197492579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b53670 msgr2=0x564184b53af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39fce7f640 1 --2- 172.21.15.5:0/3197492579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b53670 0x564184b53af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39fce7f640 1 -- 172.21.15.5:0/3197492579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564184a3b440 con 0x564184b58650 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39fce7f640 1 --2- 172.21.15.5:0/3197492579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b58650 0x564184b58a30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f39ec09b4b0 tx=0x7f39ec097b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e49e6b6126c74fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39e2ffd640 1 -- 172.21.15.5:0/3197492579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec01d040 con 0x564184b58650 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39e2ffd640 1 -- 172.21.15.5:0/3197492579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ec09fb60 con 0x564184b58650 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.100+0000 7f39e2ffd640 1 -- 172.21.15.5:0/3197492579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0ccd90 con 0x564184b58650 2024-09-11T22:22:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.101+0000 7f3a014bc740 1 -- 172.21.15.5:0/3197492579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b58650 msgr2=0x564184b58a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.101+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3197492579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b58650 0x564184b58a30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f39ec09b4b0 tx=0x7f39ec097b80 comp rx=0 tx=0).stop 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.101+0000 7f3a014bc740 1 -- 172.21.15.5:0/3197492579 shutdown_connections 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.101+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3197492579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b53670 0x564184b53af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.101+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3197492579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b59000 0x564184b53050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.101+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3197492579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b58650 0x564184b58a30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.101+0000 7f3a014bc740 1 -- 172.21.15.5:0/3197492579 >> 172.21.15.5:0/3197492579 conn(0x564184b454e0 msgr2=0x564184b458c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.101+0000 7f3a014bc740 1 -- 172.21.15.5:0/3197492579 shutdown_connections 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.101+0000 7f3a014bc740 1 -- 172.21.15.5:0/3197492579 wait complete. 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.102+0000 7f3a014bc740 1 -- start start 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.102+0000 7f3a014bc740 1 Processor -- start 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.102+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b5eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.102+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b58650 0x564184b5c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.102+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b59000 0x564184b5c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.102+0000 7f3a014bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564184b5f9b0 con 0x564184b53670 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.102+0000 7f3a014bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564184a6a050 con 0x564184b58650 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.102+0000 7f3a014bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564184b5fb30 con 0x564184b59000 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.102+0000 7f3a014bc740 10 client.0 _ll_register_callbacks cb 0x564184aaa160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.102+0000 7f39fce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b5eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39fce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b5eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60668/0 (socket says 172.21.15.5:60668) 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39fce7f640 1 -- 172.21.15.5:0/224856073 learned_addr learned my addr 172.21.15.5:0/224856073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39fd680640 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b59000 0x564184b5c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39f7fff640 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b58650 0x564184b5c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39fce7f640 1 -- 172.21.15.5:0/224856073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b59000 msgr2=0x564184b5c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39fce7f640 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b59000 0x564184b5c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39fce7f640 1 -- 172.21.15.5:0/224856073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b58650 msgr2=0x564184b5c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39fce7f640 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b58650 0x564184b5c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39fce7f640 1 -- 172.21.15.5:0/224856073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641848203c0 con 0x564184b53670 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39f7fff640 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b58650 0x564184b5c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39fce7f640 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b5eea0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c940 tx=0x7f39ec007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fdeab86286d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39d7fff640 1 -- 172.21.15.5:0/224856073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0b8070 con 0x564184b53670 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f39d7fff640 1 -- 172.21.15.5:0/224856073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f39ec005310 con 0x564184b53670 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.103+0000 7f3a014bc740 1 -- 172.21.15.5:0/224856073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39ec09fb60 con 0x564184b53670 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f39d7fff640 1 -- 172.21.15.5:0/224856073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f39ec01d070 con 0x564184b53670 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f3a014bc740 10 client.15582 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f3a014bc740 1 -- 172.21.15.5:0/224856073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564184b6b3c0 con 0x564184b53670 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f39d7fff640 1 -- 172.21.15.5:0/224856073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ec0cc9f0 con 0x564184b53670 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f39d7fff640 1 -- 172.21.15.5:0/224856073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f39ec0cf040 con 0x564184b53670 2024-09-11T22:22:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f3a014bc740 10 client.15582 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f3a014bc740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f3a014bc740 20 client.15582 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f3a014bc740 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564184b6c710 0x564184b6eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f3a014bc740 4 client.15582 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f3a014bc740 1 -- 172.21.15.5:0/224856073 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f39ec09fb60 con 0x564184b6c710 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.104+0000 7f39f7fff640 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564184b6c710 0x564184b6eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.105+0000 7f39f7fff640 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564184b6c710 0x564184b6eb00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7ed2a6587440249 server_cookie=f7ffddf61e8edba1 in_seq=0 out_seq=0 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.105+0000 7f39d7fff640 10 client.15582 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f39d7fff640 1 -- 172.21.15.5:0/224856073 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f39ec09fb60 con 0x564184b6c710 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f39d7fff640 10 client.15582 handle_command_reply: tid=0 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 client.15582 shutdown 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 -- 172.21.15.5:0/224856073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 msgr2=0x564184b5eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:59.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b5eea0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c940 tx=0x7f39ec007f80 comp rx=0 tx=0).stop 2024-09-11T22:22:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 -- 172.21.15.5:0/224856073 shutdown_connections 2024-09-11T22:22:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564184b6c710 0x564184b6eb00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f39d7fff640 0 client.15582 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b59000 0x564184b5c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b58650 0x564184b5c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 --2- 172.21.15.5:0/224856073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b5eea0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 -- 172.21.15.5:0/224856073 >> 172.21.15.5:0/224856073 conn(0x564184b454e0 msgr2=0x564184b44bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 -- 172.21.15.5:0/224856073 shutdown_connections 2024-09-11T22:22:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 1 -- 172.21.15.5:0/224856073 wait complete. 2024-09-11T22:22:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.106+0000 7f3a014bc740 20 client.15582 trim_cache size 0 max 16384 2024-09-11T22:22:59.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f3a014bc740 1 Processor -- start 2024-09-11T22:22:59.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f3a014bc740 1 -- start start 2024-09-11T22:22:59.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b53050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b58650 0x564184b5ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b59000 0x564184b71b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f3a014bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81ab7e0 con 0x564184b53670 2024-09-11T22:22:59.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f3a014bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81ad320 con 0x564184b59000 2024-09-11T22:22:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f3a014bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81abeb0 con 0x564184b58650 2024-09-11T22:22:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f39fce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b53050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f39fce7f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b53050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60672/0 (socket says 172.21.15.5:60672) 2024-09-11T22:22:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f39fce7f640 1 -- 172.21.15.5:0/3717443643 learned_addr learned my addr 172.21.15.5:0/3717443643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f39fd680640 1 --2- 172.21.15.5:0/3717443643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b59000 0x564184b71b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.138+0000 7f39f7fff640 1 --2- 172.21.15.5:0/3717443643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b58650 0x564184b5ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.139+0000 7f39fce7f640 1 -- 172.21.15.5:0/3717443643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b58650 msgr2=0x564184b5ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.139+0000 7f39fce7f640 1 --2- 172.21.15.5:0/3717443643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b58650 0x564184b5ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.139+0000 7f39fce7f640 1 -- 172.21.15.5:0/3717443643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b59000 msgr2=0x564184b71b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.139+0000 7f39fce7f640 1 --2- 172.21.15.5:0/3717443643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b59000 0x564184b71b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.139+0000 7f39fce7f640 1 -- 172.21.15.5:0/3717443643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564184b5a6a0 con 0x564184b53670 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.139+0000 7f39fce7f640 1 --2- 172.21.15.5:0/3717443643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b53050 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f39ec007590 tx=0x7f39ec005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efbff96e8a28815 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.139+0000 7f39e27fc640 1 -- 172.21.15.5:0/3717443643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec0b8070 con 0x564184b53670 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.139+0000 7f39e27fc640 1 -- 172.21.15.5:0/3717443643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ec09fb60 con 0x564184b53670 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.139+0000 7f39e27fc640 1 -- 172.21.15.5:0/3717443643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec005310 con 0x564184b53670 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.140+0000 7f3a014bc740 1 -- 172.21.15.5:0/3717443643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 msgr2=0x564184b53050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.140+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3717443643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b53050 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f39ec007590 tx=0x7f39ec005690 comp rx=0 tx=0).stop 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.140+0000 7f3a014bc740 1 -- 172.21.15.5:0/3717443643 shutdown_connections 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.140+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3717443643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b59000 0x564184b71b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.140+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3717443643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b58650 0x564184b5ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.140+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3717443643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b53670 0x564184b53050 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.140+0000 7f3a014bc740 1 -- 172.21.15.5:0/3717443643 >> 172.21.15.5:0/3717443643 conn(0x564184b454e0 msgr2=0x564184b458c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.140+0000 7f3a014bc740 1 -- 172.21.15.5:0/3717443643 shutdown_connections 2024-09-11T22:22:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.140+0000 7f3a014bc740 1 -- 172.21.15.5:0/3717443643 wait complete. 2024-09-11T22:22:59.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.141+0000 7f3a014bc740 1 -- start start 2024-09-11T22:22:59.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.141+0000 7f3a014bc740 1 Processor -- start 2024-09-11T22:22:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.141+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b53670 0x564184b6ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.141+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b58650 0x564184b704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.141+0000 7f3a014bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b59000 0x564184bd6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.141+0000 7f3a014bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81acc50 con 0x564184b59000 2024-09-11T22:22:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.141+0000 7f3a014bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81ac580 con 0x564184b53670 2024-09-11T22:22:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.141+0000 7f3a014bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81a97c0 con 0x564184b58650 2024-09-11T22:22:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.141+0000 7f3a014bc740 10 client.0 _ll_register_callbacks cb 0x7f39f0009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:22:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39fd680640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b59000 0x564184bd6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39fd680640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b59000 0x564184bd6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60686/0 (socket says 172.21.15.5:60686) 2024-09-11T22:22:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39fd680640 1 -- 172.21.15.5:0/3552801987 learned_addr learned my addr 172.21.15.5:0/3552801987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39fce7f640 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b53670 0x564184b6ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39fd680640 1 -- 172.21.15.5:0/3552801987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b58650 msgr2=0x564184b704d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39fd680640 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b58650 0x564184b704d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39fd680640 1 -- 172.21.15.5:0/3552801987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b53670 msgr2=0x564184b6ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39fd680640 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b53670 0x564184b6ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39fd680640 1 -- 172.21.15.5:0/3552801987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564184bd6590 con 0x564184b59000 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39fd680640 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b59000 0x564184bd6050 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f39f000ae70 tx=0x7f39f0017760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cedf2701fda1289 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.142+0000 7f39d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f39d7fff640 1 -- 172.21.15.5:0/3552801987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0025060 con 0x564184b59000 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f39d7fff640 1 -- 172.21.15.5:0/3552801987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f39f0020640 con 0x564184b59000 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f3a014bc740 1 -- 172.21.15.5:0/3552801987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39ec09fb60 con 0x564184b59000 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f39d7fff640 1 -- 172.21.15.5:0/3552801987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f39f0020840 con 0x564184b59000 2024-09-11T22:22:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f3a014bc740 10 client.15588 fetch_fsmap learned FSMap version 28 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f3a014bc740 1 -- 172.21.15.5:0/3552801987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564184bd78a0 con 0x564184b59000 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f39d7fff640 1 -- 172.21.15.5:0/3552801987 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39ec09fb60 con 0x564184b59000 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f39d7fff640 1 -- 172.21.15.5:0/3552801987 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f39f0028570 con 0x564184b59000 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f3a014bc740 10 client.15588 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f3a014bc740 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f3a014bc740 20 client.15588 populate_metadata read hostname 'smithi005' 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564184b6c710 0x564184bda080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f3a014bc740 4 client.15588 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.143+0000 7f3a014bc740 1 -- 172.21.15.5:0/3552801987 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564184abc850 con 0x564184b6c710 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f39fce7f640 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564184b6c710 0x564184bda080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f39fce7f640 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564184b6c710 0x564184bda080 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2dc7a05e62eca55 server_cookie=5b34695e2f1334f8 in_seq=0 out_seq=0 2024-09-11T22:22:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f39d7fff640 10 client.15588 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f39d7fff640 1 -- 172.21.15.5:0/3552801987 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564184abc850 con 0x564184b6c710 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f39d7fff640 10 client.15588 handle_command_reply: tid=0 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f3a014bc740 1 client.15588 shutdown 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f3a014bc740 1 -- 172.21.15.5:0/3552801987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b59000 msgr2=0x564184bd6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b59000 0x564184bd6050 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f39f000ae70 tx=0x7f39f0017760 comp rx=0 tx=0).stop 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f3a014bc740 1 -- 172.21.15.5:0/3552801987 shutdown_connections 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564184b6c710 0x564184bda080 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564184b59000 0x564184bd6050 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.144+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564184b58650 0x564184b704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.145+0000 7f3a014bc740 1 --2- 172.21.15.5:0/3552801987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564184b53670 0x564184b6ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.145+0000 7f3a014bc740 1 -- 172.21.15.5:0/3552801987 >> 172.21.15.5:0/3552801987 conn(0x564184b454e0 msgr2=0x564184b5aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.145+0000 7f39d7fff640 0 client.15588 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:22:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.145+0000 7f3a014bc740 1 -- 172.21.15.5:0/3552801987 shutdown_connections 2024-09-11T22:22:59.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.145+0000 7f3a014bc740 1 -- 172.21.15.5:0/3552801987 wait complete. 2024-09-11T22:22:59.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.145+0000 7f3a014bc740 20 client.15588 trim_cache size 0 max 16384 2024-09-11T22:22:59.150 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:22:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:22:59.150 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:22:59.150 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:22:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2606097483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39e80a2e10 msgr2=0x7f39e80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39e80a2e10 0x7f39e80a52d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f39f80ff7f0 tx=0x7f39e4031490 comp rx=0 tx=0).stop 2024-09-11T22:22:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2606097483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f8100100 msgr2=0x7f39f8198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:22:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f8100100 0x7f39f8198550 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00c970 tx=0x7f39ec00ce30 comp rx=0 tx=0).stop 2024-09-11T22:22:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2606097483 shutdown_connections 2024-09-11T22:22:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39f8106b80 0x7f39f819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39e80a2e10 0x7f39e80a52d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39f8100aa0 0x7f39f8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 --2- 172.21.15.5:0/2606097483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f8100100 0x7f39f8198550 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:22:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2606097483 >> 172.21.15.5:0/2606097483 conn(0x7f39f8077230 msgr2=0x7f39f8104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:22:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2606097483 shutdown_connections 2024-09-11T22:22:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:22:59.146+0000 7f39ff0e3640 1 -- 172.21.15.5:0/2606097483 wait complete. 2024-09-11T22:22:59.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:22:59 smithi005 ceph-mon[30943]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-09-11T22:22:59.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:22:59 smithi159 ceph-mon[35534]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-09-11T22:22:59.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:22:59 smithi172 ceph-mon[35461]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-09-11T22:22:59.838 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:22:59.838 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:23:00.136 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:00.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:23:00 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:23:00.029+0000 7f6dc6f14640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:23:00.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:23:00 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:23:00.029+0000 7f6dc6f14640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:23:00.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:23:00 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:23:00.029+0000 7f6dc6f14640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:23:00.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:23:00 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:23:00.030+0000 7f6dc6f14640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:23:00.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:23:00 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:23:00.030+0000 7f6dc6f14640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T22:23:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.673+0000 7f11cad77640 1 -- 172.21.15.5:0/4045830614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c4103740 msgr2=0x7f11c4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.673+0000 7f11cad77640 1 --2- 172.21.15.5:0/4045830614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c4103740 0x7f11c4107b80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009930 tx=0x7f11b002a420 comp rx=0 tx=0).stop 2024-09-11T22:23:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.673+0000 7f11cad77640 1 -- 172.21.15.5:0/4045830614 shutdown_connections 2024-09-11T22:23:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.673+0000 7f11cad77640 1 --2- 172.21.15.5:0/4045830614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11c4108290 0x7f11c410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.673+0000 7f11cad77640 1 --2- 172.21.15.5:0/4045830614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c4103740 0x7f11c4107b80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.673+0000 7f11cad77640 1 --2- 172.21.15.5:0/4045830614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c4102d90 0x7f11c4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.673+0000 7f11cad77640 1 -- 172.21.15.5:0/4045830614 >> 172.21.15.5:0/4045830614 conn(0x7f11c40fc7d0 msgr2=0x7f11c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.673+0000 7f11cad77640 1 -- 172.21.15.5:0/4045830614 shutdown_connections 2024-09-11T22:23:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.673+0000 7f11cad77640 1 -- 172.21.15.5:0/4045830614 wait complete. 2024-09-11T22:23:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.674+0000 7f11cad77640 1 Processor -- start 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.674+0000 7f11cad77640 1 -- start start 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.674+0000 7f11cad77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11c4102d90 0x7f11c4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.674+0000 7f11cad77640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c4103740 0x7f11c4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.674+0000 7f11cad77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c4108290 0x7f11c419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.674+0000 7f11cad77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4112090 con 0x7f11c4108290 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.674+0000 7f11cad77640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4111f10 con 0x7f11c4103740 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.674+0000 7f11cad77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f11c4112210 con 0x7f11c4102d90 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.674+0000 7f11c9314640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c4108290 0x7f11c419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c9314640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c4108290 0x7f11c419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60734/0 (socket says 172.21.15.5:60734) 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c9314640 1 -- 172.21.15.5:0/2800145330 learned_addr learned my addr 172.21.15.5:0/2800145330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c8b13640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11c4102d90 0x7f11c4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c3fff640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c4103740 0x7f11c4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:00.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c9314640 1 -- 172.21.15.5:0/2800145330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11c4102d90 msgr2=0x7f11c4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c9314640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11c4102d90 0x7f11c4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c9314640 1 -- 172.21.15.5:0/2800145330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c4103740 msgr2=0x7f11c4198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c9314640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c4103740 0x7f11c4198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c9314640 1 -- 172.21.15.5:0/2800145330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c419d4d0 con 0x7f11c4108290 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c9314640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c4108290 0x7f11c419cdf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00eaa0 tx=0x7f11bc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c1ffb640 1 -- 172.21.15.5:0/2800145330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc004d20 con 0x7f11c4108290 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11cad77640 1 -- 172.21.15.5:0/2800145330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c419d7c0 con 0x7f11c4108290 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c1ffb640 1 -- 172.21.15.5:0/2800145330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11bc005020 con 0x7f11c4108290 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.675+0000 7f11c1ffb640 1 -- 172.21.15.5:0/2800145330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc0116d0 con 0x7f11c4108290 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.676+0000 7f11cad77640 1 -- 172.21.15.5:0/2800145330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c41a50a0 con 0x7f11c4108290 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.677+0000 7f11cad77640 1 -- 172.21.15.5:0/2800145330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c4103170 con 0x7f11c4108290 2024-09-11T22:23:00.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.678+0000 7f11c1ffb640 1 -- 172.21.15.5:0/2800145330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138059+0+0 (secure 0 0 0) 0x7f11bc0051d0 con 0x7f11c4108290 2024-09-11T22:23:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.679+0000 7f11c1ffb640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f11b40a2e10 0x7f11b40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.679+0000 7f11c1ffb640 1 -- 172.21.15.5:0/2800145330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f11bc0cf850 con 0x7f11c4108290 2024-09-11T22:23:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.679+0000 7f11c8b13640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f11b40a2e10 0x7f11b40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.679+0000 7f11c8b13640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f11b40a2e10 0x7f11b40a52d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11b800abe0 tx=0x7f11b800a2d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:00.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.683+0000 7f11c1ffb640 1 -- 172.21.15.5:0/2800145330 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11bc016030 con 0x7f11c4108290 2024-09-11T22:23:00.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.901+0000 7f11cad77640 1 -- 172.21.15.5:0/2800145330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11c410a2a0 con 0x7f11c4108290 2024-09-11T22:23:00.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.902+0000 7f11c1ffb640 1 -- 172.21.15.5:0/2800145330 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7f11bc09c1c0 con 0x7f11c4108290 2024-09-11T22:23:00.903 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:23:00.903 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-11T22:22:15:230183+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:22:15.180031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:23:00.903 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-11T22:23:00.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 -- 172.21.15.5:0/2800145330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f11b40a2e10 msgr2=0x7f11b40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:00.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f11b40a2e10 0x7f11b40a52d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11b800abe0 tx=0x7f11b800a2d0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 -- 172.21.15.5:0/2800145330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c4108290 msgr2=0x7f11c419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:00.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c4108290 0x7f11c419cdf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00eaa0 tx=0x7f11bc00eea0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 -- 172.21.15.5:0/2800145330 shutdown_connections 2024-09-11T22:23:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11c4108290 0x7f11c419cdf0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f11b40a2e10 0x7f11b40a52d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11c4103740 0x7f11c4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 --2- 172.21.15.5:0/2800145330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11c4102d90 0x7f11c4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 -- 172.21.15.5:0/2800145330 >> 172.21.15.5:0/2800145330 conn(0x7f11c40fc7d0 msgr2=0x7f11c4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:00.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 -- 172.21.15.5:0/2800145330 shutdown_connections 2024-09-11T22:23:00.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:00.905+0000 7f11cad77640 1 -- 172.21.15.5:0/2800145330 wait complete. 2024-09-11T22:23:01.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:01 smithi005 ceph-mon[30943]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:23:01.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:01 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2800145330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:01 smithi159 ceph-mon[35534]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:23:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:01 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2800145330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:01.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:01 smithi172 ceph-mon[35461]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-11T22:23:01.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:01 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2800145330' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:01.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:23:01.861 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.387+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1329105728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd48100e20 msgr2=0x7efd48101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.387+0000 7efd4dbf1640 1 --2- 172.21.15.5:0/1329105728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd48100e20 0x7efd48101200 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efd38009900 tx=0x7efd3802a480 comp rx=0 tx=0).stop 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.387+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1329105728 shutdown_connections 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.387+0000 7efd4dbf1640 1 --2- 172.21.15.5:0/1329105728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd48102ef0 0x7efd4810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.387+0000 7efd4dbf1640 1 --2- 172.21.15.5:0/1329105728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd48102570 0x7efd481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.387+0000 7efd4dbf1640 1 --2- 172.21.15.5:0/1329105728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd48100e20 0x7efd48101200 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.387+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1329105728 >> 172.21.15.5:0/1329105728 conn(0x7efd480fc7d0 msgr2=0x7efd480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.387+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1329105728 shutdown_connections 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.387+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1329105728 wait complete. 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.388+0000 7efd4dbf1640 1 Processor -- start 2024-09-11T22:23:02.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.388+0000 7efd4dbf1640 1 -- start start 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.388+0000 7efd4dbf1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd48100e20 0x7efd480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.388+0000 7efd4dbf1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd48102570 0x7efd480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.388+0000 7efd4dbf1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd48102ef0 0x7efd480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.388+0000 7efd4dbf1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd48110f80 con 0x7efd48102ef0 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.388+0000 7efd4dbf1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd48110e00 con 0x7efd48100e20 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.388+0000 7efd4dbf1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd48111100 con 0x7efd48102570 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.388+0000 7efd47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd48102ef0 0x7efd480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd3effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd48102570 0x7efd480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd48102ef0 0x7efd480fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54040/0 (socket says 172.21.15.5:54040) 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd47fff640 1 -- 172.21.15.5:0/1586231473 learned_addr learned my addr 172.21.15.5:0/1586231473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd477fe640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd48100e20 0x7efd480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd47fff640 1 -- 172.21.15.5:0/1586231473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd48102570 msgr2=0x7efd480fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd47fff640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd48102570 0x7efd480fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd47fff640 1 -- 172.21.15.5:0/1586231473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd48100e20 msgr2=0x7efd480fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd47fff640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd48100e20 0x7efd480fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd47fff640 1 -- 172.21.15.5:0/1586231473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd481a69c0 con 0x7efd48102ef0 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd47fff640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd48102ef0 0x7efd480fd700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efd4000eaa0 tx=0x7efd4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd457fa640 1 -- 172.21.15.5:0/1586231473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4000cdb0 con 0x7efd48102ef0 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1586231473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd481a6cb0 con 0x7efd48102ef0 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd457fa640 1 -- 172.21.15.5:0/1586231473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd40005020 con 0x7efd48102ef0 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd457fa640 1 -- 172.21.15.5:0/1586231473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd400116d0 con 0x7efd48102ef0 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.389+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1586231473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd481a7240 con 0x7efd48102ef0 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.392+0000 7efd457fa640 1 -- 172.21.15.5:0/1586231473 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7efd400054d0 con 0x7efd48102ef0 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd4ffca740 1 Processor -- start 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd4ffca740 1 -- start start 2024-09-11T22:23:02.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d5710b40 0x5568d570f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d570fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d571a370 0x5568d571c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd4ffca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568d56752f0 con 0x5568d5710b40 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd4ffca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5568d561ccf0 con 0x5568d571a370 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd4ffca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5568d56712b0 con 0x5568d570f5d0 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd3effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d5710b40 0x5568d570f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd3effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d5710b40 0x5568d570f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54042/0 (socket says 172.21.15.5:54042) 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd3effd640 1 -- 172.21.15.5:0/817460731 learned_addr learned my addr 172.21.15.5:0/817460731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd477fe640 1 --2- 172.21.15.5:0/817460731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d570fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.393+0000 7efd3effd640 1 --2- 172.21.15.5:0/817460731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d571a370 0x5568d571c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd3effd640 1 -- 172.21.15.5:0/817460731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 msgr2=0x5568d570fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd3effd640 1 --2- 172.21.15.5:0/817460731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d570fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd3effd640 1 -- 172.21.15.5:0/817460731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d571a370 msgr2=0x5568d571c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd3effd640 1 --2- 172.21.15.5:0/817460731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d571a370 0x5568d571c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd3effd640 1 -- 172.21.15.5:0/817460731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d56013a0 con 0x5568d5710b40 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd477fe640 1 --2- 172.21.15.5:0/817460731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d570fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd457fa640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efd340a2ea0 0x7efd340a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd3effd640 1 --2- 172.21.15.5:0/817460731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d5710b40 0x5568d570f090 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efd300058a0 tx=0x7efd3000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cb4036cc9d2065 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd477fe640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efd340a2ea0 0x7efd340a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd457fa640 1 -- 172.21.15.5:0/1586231473 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7efd400cf860 con 0x7efd48102ef0 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd3dffb640 1 -- 172.21.15.5:0/817460731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30004030 con 0x5568d5710b40 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd3dffb640 1 -- 172.21.15.5:0/817460731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd300041d0 con 0x5568d5710b40 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.394+0000 7efd3dffb640 1 -- 172.21.15.5:0/817460731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300104c0 con 0x5568d5710b40 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd4ffca740 1 -- 172.21.15.5:0/817460731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d5710b40 msgr2=0x5568d570f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd4ffca740 1 --2- 172.21.15.5:0/817460731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d5710b40 0x5568d570f090 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efd300058a0 tx=0x7efd3000b840 comp rx=0 tx=0).stop 2024-09-11T22:23:02.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd477fe640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efd340a2ea0 0x7efd340a5360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd38009230 tx=0x7efd380058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd4ffca740 1 -- 172.21.15.5:0/817460731 shutdown_connections 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd4ffca740 1 --2- 172.21.15.5:0/817460731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d571a370 0x5568d571c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd4ffca740 1 --2- 172.21.15.5:0/817460731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d570fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd4ffca740 1 --2- 172.21.15.5:0/817460731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d5710b40 0x5568d570f090 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd4ffca740 1 -- 172.21.15.5:0/817460731 >> 172.21.15.5:0/817460731 conn(0x5568d56d8310 msgr2=0x5568d570bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd4ffca740 1 -- 172.21.15.5:0/817460731 shutdown_connections 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd4ffca740 1 -- 172.21.15.5:0/817460731 wait complete. 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.395+0000 7efd4ffca740 1 -- start start 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd4ffca740 1 Processor -- start 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d5719cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d5710b40 0x5568d5716f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d57174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd4ffca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5568d571d070 con 0x5568d571a370 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd4ffca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5568d5630060 con 0x5568d5710b40 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd4ffca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5568d571d1f0 con 0x5568d570f5d0 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd4ffca740 10 client.0 _ll_register_callbacks cb 0x5568d53838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d57174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d57174d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54046/0 (socket says 172.21.15.5:54046) 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd47fff640 1 -- 172.21.15.5:0/3115070798 learned_addr learned my addr 172.21.15.5:0/3115070798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd3effd640 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d5710b40 0x5568d5716f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd47fff640 1 -- 172.21.15.5:0/3115070798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 msgr2=0x5568d5719cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd47fff640 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d5719cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd47fff640 1 -- 172.21.15.5:0/3115070798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d5710b40 msgr2=0x5568d5716f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.396+0000 7efd47fff640 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d5710b40 0x5568d5716f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd47fff640 1 -- 172.21.15.5:0/3115070798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d53e63c0 con 0x5568d571a370 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd47fff640 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d57174d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efd40098050 tx=0x7efd4009bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d996a07af9acc43c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd26ffd640 1 -- 172.21.15.5:0/3115070798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4001f070 con 0x5568d571a370 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd26ffd640 1 -- 172.21.15.5:0/3115070798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7efd400a4520 con 0x5568d571a370 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd4ffca740 1 -- 172.21.15.5:0/3115070798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd300041d0 con 0x5568d571a370 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd26ffd640 1 -- 172.21.15.5:0/3115070798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd4009b040 con 0x5568d571a370 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd4ffca740 10 client.15612 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd4ffca740 1 -- 172.21.15.5:0/3115070798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568d5728ab0 con 0x5568d571a370 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd26ffd640 1 -- 172.21.15.5:0/3115070798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd300041d0 con 0x5568d571a370 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd26ffd640 1 -- 172.21.15.5:0/3115070798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efd4000f040 con 0x5568d571a370 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd4ffca740 10 client.15612 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.397+0000 7efd4ffca740 10 client.15612 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:23:02.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.398+0000 7efd4ffca740 20 client.15612 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.398+0000 7efd4ffca740 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5568d5729f10 0x5568d572c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.398+0000 7efd4ffca740 4 client.15612 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.398+0000 7efd4ffca740 1 -- 172.21.15.5:0/3115070798 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568d5682860 con 0x5568d5729f10 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.398+0000 7efd477fe640 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5568d5729f10 0x5568d572c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.398+0000 7efd26ffd640 10 client.15612 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.398+0000 7efd477fe640 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5568d5729f10 0x5568d572c390 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82ac3621e57d1bc3 server_cookie=56f86deac41066bd in_seq=0 out_seq=0 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd26ffd640 1 -- 172.21.15.5:0/3115070798 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5568d5682860 con 0x5568d5729f10 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd26ffd640 10 client.15612 handle_command_reply: tid=0 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 client.15612 shutdown 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 -- 172.21.15.5:0/3115070798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 msgr2=0x5568d57174d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d57174d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efd40098050 tx=0x7efd4009bd20 comp rx=0 tx=0).stop 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 -- 172.21.15.5:0/3115070798 shutdown_connections 2024-09-11T22:23:02.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5568d5729f10 0x5568d572c390 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d57174d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d5710b40 0x5568d5716f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd26ffd640 0 client.15612 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:23:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 --2- 172.21.15.5:0/3115070798 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d5719cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 -- 172.21.15.5:0/3115070798 >> 172.21.15.5:0/3115070798 conn(0x5568d56d8310 msgr2=0x5568d5711930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 -- 172.21.15.5:0/3115070798 shutdown_connections 2024-09-11T22:23:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 1 -- 172.21.15.5:0/3115070798 wait complete. 2024-09-11T22:23:02.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.399+0000 7efd4ffca740 20 client.15612 trim_cache size 0 max 16384 2024-09-11T22:23:02.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd4ffca740 1 Processor -- start 2024-09-11T22:23:02.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd4ffca740 1 -- start start 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d570f5d0 0x5568d5726340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d5710b40 0x5568d5712120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d5712660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd4ffca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd481afb70 con 0x5568d571a370 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd4ffca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b16b0 con 0x5568d570f5d0 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd4ffca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b0240 con 0x5568d5710b40 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d5712660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d5712660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54062/0 (socket says 172.21.15.5:54062) 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd47fff640 1 -- 172.21.15.5:0/423436550 learned_addr learned my addr 172.21.15.5:0/423436550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd3effd640 1 --2- 172.21.15.5:0/423436550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d5710b40 0x5568d5712120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd477fe640 1 --2- 172.21.15.5:0/423436550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d570f5d0 0x5568d5726340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd47fff640 1 -- 172.21.15.5:0/423436550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d5710b40 msgr2=0x5568d5712120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd47fff640 1 --2- 172.21.15.5:0/423436550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d5710b40 0x5568d5712120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd47fff640 1 -- 172.21.15.5:0/423436550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d570f5d0 msgr2=0x5568d5726340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd47fff640 1 --2- 172.21.15.5:0/423436550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d570f5d0 0x5568d5726340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.413+0000 7efd47fff640 1 -- 172.21.15.5:0/423436550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d5706a30 con 0x5568d571a370 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.414+0000 7efd47fff640 1 --2- 172.21.15.5:0/423436550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d5712660 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efd40098570 tx=0x7efd4009a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554e4b4bb92b0612 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.414+0000 7efd3d7fa640 1 -- 172.21.15.5:0/423436550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4001f070 con 0x5568d571a370 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.414+0000 7efd3d7fa640 1 -- 172.21.15.5:0/423436550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4009acc0 con 0x5568d571a370 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.414+0000 7efd3d7fa640 1 -- 172.21.15.5:0/423436550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd400a4560 con 0x5568d571a370 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 -- 172.21.15.5:0/423436550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 msgr2=0x5568d5712660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 --2- 172.21.15.5:0/423436550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d5712660 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efd40098570 tx=0x7efd4009a100 comp rx=0 tx=0).stop 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 -- 172.21.15.5:0/423436550 shutdown_connections 2024-09-11T22:23:02.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 --2- 172.21.15.5:0/423436550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d5712660 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 --2- 172.21.15.5:0/423436550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d5710b40 0x5568d5712120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 --2- 172.21.15.5:0/423436550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d570f5d0 0x5568d5726340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 -- 172.21.15.5:0/423436550 >> 172.21.15.5:0/423436550 conn(0x5568d56d8310 msgr2=0x5568d5716060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 -- 172.21.15.5:0/423436550 shutdown_connections 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 -- 172.21.15.5:0/423436550 wait complete. 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 -- start start 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.415+0000 7efd4ffca740 1 Processor -- start 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d571e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d5710b40 0x5568d571e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd4ffca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d571ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd4ffca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b0fe0 con 0x5568d571a370 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd4ffca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b0910 con 0x5568d5710b40 2024-09-11T22:23:02.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd4ffca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efd481adb50 con 0x5568d570f5d0 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd4ffca740 10 client.0 _ll_register_callbacks cb 0x7efd400057c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d571ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d571ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54074/0 (socket says 172.21.15.5:54074) 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd47fff640 1 -- 172.21.15.5:0/506484185 learned_addr learned my addr 172.21.15.5:0/506484185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd477fe640 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d571e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd3effd640 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d5710b40 0x5568d571e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd47fff640 1 -- 172.21.15.5:0/506484185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 msgr2=0x5568d571e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd47fff640 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d571e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd47fff640 1 -- 172.21.15.5:0/506484185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d5710b40 msgr2=0x5568d571e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd47fff640 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d5710b40 0x5568d571e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.416+0000 7efd47fff640 1 -- 172.21.15.5:0/506484185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d5793dd0 con 0x5568d571a370 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd47fff640 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d571ee00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efd4009bc30 tx=0x7efd400cceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fccf2e031888fac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:02.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd26ffd640 1 -- 172.21.15.5:0/506484185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4001f050 con 0x5568d571a370 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd26ffd640 1 -- 172.21.15.5:0/506484185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7efd400a4520 con 0x5568d571a370 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd4ffca740 1 -- 172.21.15.5:0/506484185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd4009acc0 con 0x5568d571a370 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd26ffd640 1 -- 172.21.15.5:0/506484185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd400a9050 con 0x5568d571a370 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd4ffca740 10 client.15618 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd4ffca740 1 -- 172.21.15.5:0/506484185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568d5795070 con 0x5568d571a370 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd26ffd640 1 -- 172.21.15.5:0/506484185 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd4009acc0 con 0x5568d571a370 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd26ffd640 1 -- 172.21.15.5:0/506484185 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efd400a11a0 con 0x5568d571a370 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd4ffca740 10 client.15618 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd4ffca740 10 client.15618 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd4ffca740 20 client.15618 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd4ffca740 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5568d5729f10 0x5568d57975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd4ffca740 4 client.15618 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.417+0000 7efd4ffca740 1 -- 172.21.15.5:0/506484185 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568d567cd70 con 0x5568d5729f10 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.418+0000 7efd477fe640 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5568d5729f10 0x5568d57975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.419+0000 7efd477fe640 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5568d5729f10 0x5568d57975a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea207a254be7fffc server_cookie=d89288ef167150b9 in_seq=0 out_seq=0 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.419+0000 7efd26ffd640 10 client.15618 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.420+0000 7efd26ffd640 1 -- 172.21.15.5:0/506484185 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568d567cd70 con 0x5568d5729f10 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.420+0000 7efd26ffd640 10 client.15618 handle_command_reply: tid=0 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.420+0000 7efd4ffca740 1 client.15618 shutdown 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.420+0000 7efd4ffca740 1 -- 172.21.15.5:0/506484185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 msgr2=0x5568d571ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.420+0000 7efd4ffca740 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d571ee00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efd4009bc30 tx=0x7efd400cceb0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.420+0000 7efd4ffca740 1 -- 172.21.15.5:0/506484185 shutdown_connections 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.420+0000 7efd4ffca740 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5568d5729f10 0x5568d57975a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4ffca740 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5568d571a370 0x5568d571ee00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd26ffd640 0 client.15618 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:23:02.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4ffca740 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5568d5710b40 0x5568d571e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4ffca740 1 --2- 172.21.15.5:0/506484185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5568d570f5d0 0x5568d571e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4ffca740 1 -- 172.21.15.5:0/506484185 >> 172.21.15.5:0/506484185 conn(0x5568d56d8310 msgr2=0x5568d570a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:02.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4ffca740 1 -- 172.21.15.5:0/506484185 shutdown_connections 2024-09-11T22:23:02.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4ffca740 1 -- 172.21.15.5:0/506484185 wait complete. 2024-09-11T22:23:02.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4ffca740 20 client.15618 trim_cache size 0 max 16384 2024-09-11T22:23:02.429 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:23:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1586231473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efd340a2ea0 msgr2=0x7efd340a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4dbf1640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efd340a2ea0 0x7efd340a5360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efd38009230 tx=0x7efd380058d0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1586231473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd48102ef0 msgr2=0x7efd480fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.421+0000 7efd4dbf1640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd48102ef0 0x7efd480fd700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efd4000eaa0 tx=0x7efd4000ef60 comp rx=0 tx=0).stop 2024-09-11T22:23:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.422+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1586231473 shutdown_connections 2024-09-11T22:23:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.422+0000 7efd4dbf1640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd48102ef0 0x7efd480fd700 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.422+0000 7efd4dbf1640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd48102570 0x7efd480fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.422+0000 7efd4dbf1640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efd340a2ea0 0x7efd340a5360 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.422+0000 7efd4dbf1640 1 --2- 172.21.15.5:0/1586231473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd48100e20 0x7efd480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.422+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1586231473 >> 172.21.15.5:0/1586231473 conn(0x7efd480fc7d0 msgr2=0x7efd4806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:02.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.422+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1586231473 shutdown_connections 2024-09-11T22:23:02.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:02.422+0000 7efd4dbf1640 1 -- 172.21.15.5:0/1586231473 wait complete. 2024-09-11T22:23:03.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:03 smithi005 ceph-mon[30943]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-11T22:23:03.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:03 smithi005 ceph-mon[30943]: mgrmap e24: x(active, since 46s), standbys: z, y 2024-09-11T22:23:03.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:03 smithi159 ceph-mon[35534]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-11T22:23:03.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:03 smithi159 ceph-mon[35534]: mgrmap e24: x(active, since 46s), standbys: z, y 2024-09-11T22:23:03.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:03 smithi172 ceph-mon[35461]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 511 B/s wr, 9 op/s 2024-09-11T22:23:03.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:03 smithi172 ceph-mon[35461]: mgrmap e24: x(active, since 46s), standbys: z, y 2024-09-11T22:23:03.552 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:23:03.822 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:04.316 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-11T22:23:04.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:23:05.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:05 smithi005 ceph-mon[30943]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T22:23:05.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:05 smithi172 ceph-mon[35461]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T22:23:05.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:05 smithi159 ceph-mon[35534]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-09-11T22:23:06.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.478+0000 7f1587dac640 1 -- 172.21.15.5:0/457692078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15801036e0 msgr2=0x7f1580107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:06.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.478+0000 7f1587dac640 1 --2- 172.21.15.5:0/457692078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15801036e0 0x7f1580107b90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1570009930 tx=0x7f157002a420 comp rx=0 tx=0).stop 2024-09-11T22:23:06.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.478+0000 7f1587dac640 1 -- 172.21.15.5:0/457692078 shutdown_connections 2024-09-11T22:23:06.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.478+0000 7f1587dac640 1 --2- 172.21.15.5:0/457692078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15801082a0 0x7f158010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.478+0000 7f1587dac640 1 --2- 172.21.15.5:0/457692078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15801036e0 0x7f1580107b90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.478+0000 7f1587dac640 1 --2- 172.21.15.5:0/457692078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1580102d30 0x7f1580103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.478+0000 7f1587dac640 1 -- 172.21.15.5:0/457692078 >> 172.21.15.5:0/457692078 conn(0x7f15800fc7d0 msgr2=0x7f15800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.478+0000 7f1587dac640 1 -- 172.21.15.5:0/457692078 shutdown_connections 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.478+0000 7f1587dac640 1 -- 172.21.15.5:0/457692078 wait complete. 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1587dac640 1 Processor -- start 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1587dac640 1 -- start start 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1587dac640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580102d30 0x7f15801985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1587dac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15801036e0 0x7f1580198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1587dac640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15801082a0 0x7f158019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1587dac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1580112060 con 0x7f15801036e0 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1587dac640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1580111ee0 con 0x7f1580102d30 2024-09-11T22:23:06.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1587dac640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15801121e0 con 0x7f15801082a0 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1585347640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15801036e0 0x7f1580198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1585347640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15801036e0 0x7f1580198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54086/0 (socket says 172.21.15.5:54086) 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1585347640 1 -- 172.21.15.5:0/564479786 learned_addr learned my addr 172.21.15.5:0/564479786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.479+0000 7f1585b48640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580102d30 0x7f15801985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1586349640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15801082a0 0x7f158019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1585347640 1 -- 172.21.15.5:0/564479786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15801082a0 msgr2=0x7f158019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1585347640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15801082a0 0x7f158019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1585347640 1 -- 172.21.15.5:0/564479786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580102d30 msgr2=0x7f15801985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1585347640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580102d30 0x7f15801985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1585347640 1 -- 172.21.15.5:0/564479786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158019d570 con 0x7f15801036e0 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1586349640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15801082a0 0x7f158019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:06.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1585347640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15801036e0 0x7f1580198b00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1570005940 tx=0x7f15700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:06.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1576ffd640 1 -- 172.21.15.5:0/564479786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157003f030 con 0x7f15801036e0 2024-09-11T22:23:06.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1587dac640 1 -- 172.21.15.5:0/564479786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158019d800 con 0x7f15801036e0 2024-09-11T22:23:06.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1576ffd640 1 -- 172.21.15.5:0/564479786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1570003ce0 con 0x7f15801036e0 2024-09-11T22:23:06.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1576ffd640 1 -- 172.21.15.5:0/564479786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570035070 con 0x7f15801036e0 2024-09-11T22:23:06.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.480+0000 7f1587dac640 1 -- 172.21.15.5:0/564479786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15801a5140 con 0x7f15801036e0 2024-09-11T22:23:06.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.483+0000 7f158a185740 1 Processor -- start 2024-09-11T22:23:06.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.483+0000 7f158a185740 1 -- start start 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.483+0000 7f1576ffd640 1 -- 172.21.15.5:0/564479786 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7f1570042030 con 0x7f15801036e0 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.483+0000 7f158a185740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb51b40 0x55b86bb50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.483+0000 7f158a185740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb505d0 0x55b86bb50a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.483+0000 7f158a185740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 0x55b86bb61b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.483+0000 7f158a185740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b86bab62f0 con 0x55b86bb51b40 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.483+0000 7f158a185740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b86ba5dcf0 con 0x55b86bb5f780 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.483+0000 7f158a185740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b86bab22b0 con 0x55b86bb505d0 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1586349640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb51b40 0x55b86bb50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1586349640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb51b40 0x55b86bb50090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54092/0 (socket says 172.21.15.5:54092) 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1586349640 1 -- 172.21.15.5:0/3905909124 learned_addr learned my addr 172.21.15.5:0/3905909124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1585b48640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb505d0 0x55b86bb50a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1585347640 1 --2- 172.21.15.5:0/3905909124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 0x55b86bb61b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1586349640 1 -- 172.21.15.5:0/3905909124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb505d0 msgr2=0x55b86bb50a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1586349640 1 --2- 172.21.15.5:0/3905909124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb505d0 0x55b86bb50a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1586349640 1 -- 172.21.15.5:0/3905909124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 msgr2=0x55b86bb61b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1586349640 1 --2- 172.21.15.5:0/3905909124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 0x55b86bb61b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1586349640 1 -- 172.21.15.5:0/3905909124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b86ba423a0 con 0x55b86bb51b40 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1585347640 1 --2- 172.21.15.5:0/3905909124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 0x55b86bb61b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1576ffd640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f156c0a2ea0 0x7f156c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1586349640 1 --2- 172.21.15.5:0/3905909124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb51b40 0x55b86bb50090 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1578009e40 tx=0x7f157800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c92ba79d7fa9b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1585b48640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f156c0a2ea0 0x7f156c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f1576ffd640 1 -- 172.21.15.5:0/564479786 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f15700efbb0 con 0x7f15801036e0 2024-09-11T22:23:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f156bfff640 1 -- 172.21.15.5:0/3905909124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578004d20 con 0x55b86bb51b40 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f156bfff640 1 -- 172.21.15.5:0/3905909124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1578005500 con 0x55b86bb51b40 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.484+0000 7f156bfff640 1 -- 172.21.15.5:0/3905909124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15780118e0 con 0x55b86bb51b40 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.485+0000 7f158a185740 1 -- 172.21.15.5:0/3905909124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb51b40 msgr2=0x55b86bb50090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.485+0000 7f1585b48640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f156c0a2ea0 0x7f156c0a5360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55b86bb4dac0 tx=0x7f157c00b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.485+0000 7f158a185740 1 --2- 172.21.15.5:0/3905909124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb51b40 0x55b86bb50090 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1578009e40 tx=0x7f157800f730 comp rx=0 tx=0).stop 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.485+0000 7f158a185740 1 -- 172.21.15.5:0/3905909124 shutdown_connections 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.485+0000 7f158a185740 1 --2- 172.21.15.5:0/3905909124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 0x55b86bb61b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.485+0000 7f158a185740 1 --2- 172.21.15.5:0/3905909124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb505d0 0x55b86bb50a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.485+0000 7f158a185740 1 --2- 172.21.15.5:0/3905909124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb51b40 0x55b86bb50090 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.485+0000 7f158a185740 1 -- 172.21.15.5:0/3905909124 >> 172.21.15.5:0/3905909124 conn(0x55b86bb19310 msgr2=0x55b86bb4cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.486+0000 7f158a185740 1 -- 172.21.15.5:0/3905909124 shutdown_connections 2024-09-11T22:23:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.486+0000 7f158a185740 1 -- 172.21.15.5:0/3905909124 wait complete. 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.486+0000 7f158a185740 1 -- start start 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.486+0000 7f158a185740 1 Processor -- start 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f158a185740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb5f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f158a185740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb51b40 0x55b86bb5c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f158a185740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 0x55b86bb5cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f158a185740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b86bb62480 con 0x55b86bb505d0 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f158a185740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b86ba71060 con 0x55b86bb5f780 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f158a185740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b86bb62600 con 0x55b86bb51b40 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f158a185740 10 client.0 _ll_register_callbacks cb 0x55b86b7c48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1585b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb5f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1585b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb5f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54106/0 (socket says 172.21.15.5:54106) 2024-09-11T22:23:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1585b48640 1 -- 172.21.15.5:0/3244700558 learned_addr learned my addr 172.21.15.5:0/3244700558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1585347640 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb51b40 0x55b86bb5c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1586349640 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 0x55b86bb5cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1585b48640 1 -- 172.21.15.5:0/3244700558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb51b40 msgr2=0x55b86bb5c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1585b48640 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb51b40 0x55b86bb5c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1585b48640 1 -- 172.21.15.5:0/3244700558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 msgr2=0x55b86bb5cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1585b48640 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 0x55b86bb5cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1585b48640 1 -- 172.21.15.5:0/3244700558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b86b8273c0 con 0x55b86bb505d0 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1586349640 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 0x55b86bb5cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f1585b48640 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb5f1e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f157c003e60 tx=0x7f157c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe00b5e7494de219 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.487+0000 7f15697fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f15697fa640 1 -- 172.21.15.5:0/3244700558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c02c060 con 0x55b86bb505d0 2024-09-11T22:23:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f15697fa640 1 -- 172.21.15.5:0/3244700558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f157c017030 con 0x55b86bb505d0 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f158a185740 1 -- 172.21.15.5:0/3244700558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1578005500 con 0x55b86bb505d0 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f15697fa640 1 -- 172.21.15.5:0/3244700558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f157c030410 con 0x55b86bb505d0 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f158a185740 10 client.15630 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f158a185740 1 -- 172.21.15.5:0/3244700558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b86bb6e000 con 0x55b86bb505d0 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f15697fa640 1 -- 172.21.15.5:0/3244700558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1578005500 con 0x55b86bb505d0 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f15697fa640 1 -- 172.21.15.5:0/3244700558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f157c011600 con 0x55b86bb505d0 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f158a185740 10 client.15630 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f158a185740 10 client.15630 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f158a185740 20 client.15630 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.488+0000 7f158a185740 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b86bb6f4c0 0x55b86bb718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.489+0000 7f158a185740 4 client.15630 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.489+0000 7f158a185740 1 -- 172.21.15.5:0/3244700558 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b86bac3860 con 0x55b86bb6f4c0 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.489+0000 7f1585347640 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b86bb6f4c0 0x55b86bb718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.490+0000 7f15697fa640 10 client.15630 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.490+0000 7f1585347640 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b86bb6f4c0 0x55b86bb718b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ecb314e80f81f7d server_cookie=1ae918b10f04136b in_seq=0 out_seq=0 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f15697fa640 1 -- 172.21.15.5:0/3244700558 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b86bac3860 con 0x55b86bb6f4c0 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f15697fa640 10 client.15630 handle_command_reply: tid=0 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 client.15630 shutdown 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 -- 172.21.15.5:0/3244700558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 msgr2=0x55b86bb5f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb5f1e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f157c003e60 tx=0x7f157c004730 comp rx=0 tx=0).stop 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 -- 172.21.15.5:0/3244700558 shutdown_connections 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b86bb6f4c0 0x55b86bb718b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb5f780 0x55b86bb5cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb51b40 0x55b86bb5c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 --2- 172.21.15.5:0/3244700558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb5f1e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f15697fa640 0 client.15630 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:23:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 -- 172.21.15.5:0/3244700558 >> 172.21.15.5:0/3244700558 conn(0x55b86bb19310 msgr2=0x55b86bb52960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 -- 172.21.15.5:0/3244700558 shutdown_connections 2024-09-11T22:23:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 1 -- 172.21.15.5:0/3244700558 wait complete. 2024-09-11T22:23:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.491+0000 7f158a185740 20 client.15630 trim_cache size 0 max 16384 2024-09-11T22:23:06.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f158a185740 1 Processor -- start 2024-09-11T22:23:06.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f158a185740 1 -- start start 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f158a185740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb505d0 0x55b86bb77160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f158a185740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb51b40 0x55b86bb776a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f158a185740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb5f780 0x55b86bb596f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f158a185740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15801ab810 con 0x55b86bb5f780 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f158a185740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15801ad350 con 0x55b86bb505d0 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f158a185740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15801abee0 con 0x55b86bb51b40 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f1586349640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb5f780 0x55b86bb596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f1586349640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb5f780 0x55b86bb596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54122/0 (socket says 172.21.15.5:54122) 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f1586349640 1 -- 172.21.15.5:0/472262296 learned_addr learned my addr 172.21.15.5:0/472262296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f1585b48640 1 --2- 172.21.15.5:0/472262296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb505d0 0x55b86bb77160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f1585347640 1 --2- 172.21.15.5:0/472262296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb51b40 0x55b86bb776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f1586349640 1 -- 172.21.15.5:0/472262296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb51b40 msgr2=0x55b86bb776a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f1586349640 1 --2- 172.21.15.5:0/472262296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb51b40 0x55b86bb776a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f1586349640 1 -- 172.21.15.5:0/472262296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb505d0 msgr2=0x55b86bb77160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f1586349640 1 --2- 172.21.15.5:0/472262296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb505d0 0x55b86bb77160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.505+0000 7f1586349640 1 -- 172.21.15.5:0/472262296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b86bb71c20 con 0x55b86bb5f780 2024-09-11T22:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.506+0000 7f1585b48640 1 --2- 172.21.15.5:0/472262296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb505d0 0x55b86bb77160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.506+0000 7f1586349640 1 --2- 172.21.15.5:0/472262296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb5f780 0x55b86bb596f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1578009710 tx=0x7f1578009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba6e73c06944203 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.506+0000 7f156b7fe640 1 -- 172.21.15.5:0/472262296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578027070 con 0x55b86bb5f780 2024-09-11T22:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.506+0000 7f156b7fe640 1 -- 172.21.15.5:0/472262296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1578005040 con 0x55b86bb5f780 2024-09-11T22:23:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.506+0000 7f156b7fe640 1 -- 172.21.15.5:0/472262296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1578011680 con 0x55b86bb5f780 2024-09-11T22:23:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 -- 172.21.15.5:0/472262296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb5f780 msgr2=0x55b86bb596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 --2- 172.21.15.5:0/472262296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb5f780 0x55b86bb596f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1578009710 tx=0x7f1578009b30 comp rx=0 tx=0).stop 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 -- 172.21.15.5:0/472262296 shutdown_connections 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 --2- 172.21.15.5:0/472262296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb5f780 0x55b86bb596f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 --2- 172.21.15.5:0/472262296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb51b40 0x55b86bb776a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 --2- 172.21.15.5:0/472262296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb505d0 0x55b86bb77160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 -- 172.21.15.5:0/472262296 >> 172.21.15.5:0/472262296 conn(0x55b86bb19310 msgr2=0x55b86bb5ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 -- 172.21.15.5:0/472262296 shutdown_connections 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 -- 172.21.15.5:0/472262296 wait complete. 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 -- start start 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.507+0000 7f158a185740 1 Processor -- start 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f158a185740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb67f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f158a185740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb51b40 0x55b86bb632f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f158a185740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb5f780 0x55b86bb63830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f158a185740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15801acc80 con 0x55b86bb505d0 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f158a185740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15801ac5b0 con 0x55b86bb51b40 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f158a185740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f15801a97f0 con 0x55b86bb5f780 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f158a185740 10 client.0 _ll_register_callbacks cb 0x7f157c00bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f1585b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb67f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f1585b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb67f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54134/0 (socket says 172.21.15.5:54134) 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f1585b48640 1 -- 172.21.15.5:0/3808761820 learned_addr learned my addr 172.21.15.5:0/3808761820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f1585347640 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb51b40 0x55b86bb632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.508+0000 7f1586349640 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb5f780 0x55b86bb63830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f1585b48640 1 -- 172.21.15.5:0/3808761820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb5f780 msgr2=0x55b86bb63830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f1585b48640 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb5f780 0x55b86bb63830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f1585b48640 1 -- 172.21.15.5:0/3808761820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb51b40 msgr2=0x55b86bb632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f1585b48640 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb51b40 0x55b86bb632f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f1585b48640 1 -- 172.21.15.5:0/3808761820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b86bb64060 con 0x55b86bb505d0 2024-09-11T22:23:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f1586349640 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb5f780 0x55b86bb63830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f1585347640 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb51b40 0x55b86bb632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f1585b48640 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb67f70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f157c005d00 tx=0x7f157c0116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a5b5216f97376f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f15697fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f15697fa640 1 -- 172.21.15.5:0/3808761820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c005450 con 0x55b86bb505d0 2024-09-11T22:23:06.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f15697fa640 1 -- 172.21.15.5:0/3808761820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f157c0055f0 con 0x55b86bb505d0 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f158a185740 1 -- 172.21.15.5:0/3808761820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1578005040 con 0x55b86bb505d0 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f15697fa640 1 -- 172.21.15.5:0/3808761820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f157c02c030 con 0x55b86bb505d0 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f158a185740 10 client.15636 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f158a185740 1 -- 172.21.15.5:0/3808761820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b86bbda120 con 0x55b86bb505d0 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.509+0000 7f15697fa640 1 -- 172.21.15.5:0/3808761820 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f157c004c80 con 0x55b86bb505d0 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.510+0000 7f15697fa640 1 -- 172.21.15.5:0/3808761820 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f157c030890 con 0x55b86bb505d0 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.510+0000 7f158a185740 10 client.15636 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.510+0000 7f158a185740 10 client.15636 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.510+0000 7f158a185740 20 client.15636 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.510+0000 7f158a185740 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b86bbda890 0x55b86bbdccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.510+0000 7f158a185740 4 client.15636 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:23:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.510+0000 7f158a185740 1 -- 172.21.15.5:0/3808761820 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1578005040 con 0x55b86bbda890 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.510+0000 7f1585347640 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b86bbda890 0x55b86bbdccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f15697fa640 10 client.15636 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f1585347640 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b86bbda890 0x55b86bbdccb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=628e410d486a2218 server_cookie=b765208d42e69ab6 in_seq=0 out_seq=0 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f15697fa640 1 -- 172.21.15.5:0/3808761820 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1578005040 con 0x55b86bbda890 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f15697fa640 10 client.15636 handle_command_reply: tid=0 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f158a185740 1 client.15636 shutdown 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f158a185740 1 -- 172.21.15.5:0/3808761820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 msgr2=0x55b86bb67f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f158a185740 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb67f70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f157c005d00 tx=0x7f157c0116f0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f158a185740 1 -- 172.21.15.5:0/3808761820 shutdown_connections 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f158a185740 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b86bb5f780 0x55b86bb63830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f158a185740 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b86bbda890 0x55b86bbdccb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f158a185740 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b86bb51b40 0x55b86bb632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f158a185740 1 --2- 172.21.15.5:0/3808761820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b86bb505d0 0x55b86bb67f70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f15697fa640 0 client.15636 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:23:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f158a185740 1 -- 172.21.15.5:0/3808761820 >> 172.21.15.5:0/3808761820 conn(0x55b86bb19310 msgr2=0x55b86bb4b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.511+0000 7f158a185740 1 -- 172.21.15.5:0/3808761820 shutdown_connections 2024-09-11T22:23:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f158a185740 1 -- 172.21.15.5:0/3808761820 wait complete. 2024-09-11T22:23:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f158a185740 20 client.15636 trim_cache size 0 max 16384 2024-09-11T22:23:06.519 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:23:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f1587dac640 1 -- 172.21.15.5:0/564479786 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f156c0a2ea0 msgr2=0x7f156c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f1587dac640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f156c0a2ea0 0x7f156c0a5360 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55b86bb4dac0 tx=0x7f157c00b390 comp rx=0 tx=0).stop 2024-09-11T22:23:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f1587dac640 1 -- 172.21.15.5:0/564479786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15801036e0 msgr2=0x7f1580198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f1587dac640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15801036e0 0x7f1580198b00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1570005940 tx=0x7f15700042f0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f1587dac640 1 -- 172.21.15.5:0/564479786 shutdown_connections 2024-09-11T22:23:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f1587dac640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f15801082a0 0x7f158019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f1587dac640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15801036e0 0x7f1580198b00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f1587dac640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f156c0a2ea0 0x7f156c0a5360 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f1587dac640 1 --2- 172.21.15.5:0/564479786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1580102d30 0x7f15801985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.512+0000 7f1587dac640 1 -- 172.21.15.5:0/564479786 >> 172.21.15.5:0/564479786 conn(0x7f15800fc7d0 msgr2=0x7f1580105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.513+0000 7f1587dac640 1 -- 172.21.15.5:0/564479786 shutdown_connections 2024-09-11T22:23:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:06.513+0000 7f1587dac640 1 -- 172.21.15.5:0/564479786 wait complete. 2024-09-11T22:23:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.251+0000 7f23d1391640 1 -- 172.21.15.5:0/2455085504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc102d20 msgr2=0x7f23cc103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.251+0000 7f23d1391640 1 --2- 172.21.15.5:0/2455085504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc102d20 0x7f23cc103100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009900 tx=0x7f23bc02a480 comp rx=0 tx=0).stop 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.251+0000 7f23d1391640 1 -- 172.21.15.5:0/2455085504 shutdown_connections 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.251+0000 7f23d1391640 1 --2- 172.21.15.5:0/2455085504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23cc10c0f0 0x7f23cc10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.251+0000 7f23d1391640 1 --2- 172.21.15.5:0/2455085504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23cc103640 0x7f23cc10bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.251+0000 7f23d1391640 1 --2- 172.21.15.5:0/2455085504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc102d20 0x7f23cc103100 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.251+0000 7f23d1391640 1 -- 172.21.15.5:0/2455085504 >> 172.21.15.5:0/2455085504 conn(0x7f23cc077990 msgr2=0x7f23cc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.251+0000 7f23d1391640 1 -- 172.21.15.5:0/2455085504 shutdown_connections 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.251+0000 7f23d1391640 1 -- 172.21.15.5:0/2455085504 wait complete. 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.251+0000 7f23d1391640 1 Processor -- start 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23d1391640 1 -- start start 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23d1391640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc102d20 0x7f23cc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23d1391640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23cc103640 0x7f23cc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23d1391640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23cc10c0f0 0x7f23cc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23d1391640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc111ec0 con 0x7f23cc102d20 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23d1391640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc111d40 con 0x7f23cc10c0f0 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23d1391640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc112040 con 0x7f23cc103640 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23caffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc102d20 0x7f23cc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23caffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc102d20 0x7f23cc198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54162/0 (socket says 172.21.15.5:54162) 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23caffd640 1 -- 172.21.15.5:0/3848156413 learned_addr learned my addr 172.21.15.5:0/3848156413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23cb7fe640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23cc10c0f0 0x7f23cc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23caffd640 1 -- 172.21.15.5:0/3848156413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23cc103640 msgr2=0x7f23cc198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23caffd640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23cc103640 0x7f23cc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23caffd640 1 -- 172.21.15.5:0/3848156413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23cc10c0f0 msgr2=0x7f23cc19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23caffd640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23cc10c0f0 0x7f23cc19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.252+0000 7f23caffd640 1 -- 172.21.15.5:0/3848156413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23cc19d340 con 0x7f23cc102d20 2024-09-11T22:23:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.253+0000 7f23caffd640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc102d20 0x7f23cc198350 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f23bc006010 tx=0x7f23bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.253+0000 7f23c8ff9640 1 -- 172.21.15.5:0/3848156413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc03f030 con 0x7f23cc102d20 2024-09-11T22:23:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.253+0000 7f23c8ff9640 1 -- 172.21.15.5:0/3848156413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23bc003ce0 con 0x7f23cc102d20 2024-09-11T22:23:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.253+0000 7f23d1391640 1 -- 172.21.15.5:0/3848156413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23cc19d5d0 con 0x7f23cc102d20 2024-09-11T22:23:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.253+0000 7f23c8ff9640 1 -- 172.21.15.5:0/3848156413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc035070 con 0x7f23cc102d20 2024-09-11T22:23:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.253+0000 7f23d1391640 1 -- 172.21.15.5:0/3848156413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23cc1a4ec0 con 0x7f23cc102d20 2024-09-11T22:23:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.255+0000 7f23c8ff9640 1 -- 172.21.15.5:0/3848156413 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7f23bc042030 con 0x7f23cc102d20 2024-09-11T22:23:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.256+0000 7f23c8ff9640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f23b80a2fd0 0x7f23b80a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.256+0000 7f23c3fff640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f23b80a2fd0 0x7f23b80a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.256+0000 7f23c8ff9640 1 -- 172.21.15.5:0/3848156413 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f23bc0efca0 con 0x7f23cc102d20 2024-09-11T22:23:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.256+0000 7f23d4209740 1 Processor -- start 2024-09-11T22:23:07.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.256+0000 7f23d4209740 1 -- start start 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.256+0000 7f23c3fff640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f23b80a2fd0 0x7f23b80a5490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005fd0 tx=0x7f23b4005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23d4209740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 0x55a83e816d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23d4209740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e825b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23d4209740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e8284b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23d4209740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a83e77b2f0 con 0x55a83e8172b0 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23d4209740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a83e723270 con 0x55a83e816990 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23d4209740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a83e7772b0 con 0x55a83e8260c0 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e825b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e825b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54164/0 (socket says 172.21.15.5:54164) 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23c3fff640 1 -- 172.21.15.5:0/3518114737 learned_addr learned my addr 172.21.15.5:0/3518114737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23cb7fe640 1 --2- 172.21.15.5:0/3518114737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e8284b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23caffd640 1 --2- 172.21.15.5:0/3518114737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 0x55a83e816d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23c3fff640 1 -- 172.21.15.5:0/3518114737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 msgr2=0x55a83e8284b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23c3fff640 1 --2- 172.21.15.5:0/3518114737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e8284b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23c3fff640 1 -- 172.21.15.5:0/3518114737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 msgr2=0x55a83e816d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23c3fff640 1 --2- 172.21.15.5:0/3518114737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 0x55a83e816d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.257+0000 7f23c3fff640 1 -- 172.21.15.5:0/3518114737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a83e722340 con 0x55a83e8172b0 2024-09-11T22:23:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.258+0000 7f23c3fff640 1 --2- 172.21.15.5:0/3518114737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e825b80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005fa0 tx=0x7f23b4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da4e018f059cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.258+0000 7f23c17fa640 1 -- 172.21.15.5:0/3518114737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4017070 con 0x55a83e8172b0 2024-09-11T22:23:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.258+0000 7f23c17fa640 1 -- 172.21.15.5:0/3518114737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b4023070 con 0x55a83e8172b0 2024-09-11T22:23:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.258+0000 7f23c17fa640 1 -- 172.21.15.5:0/3518114737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b400f880 con 0x55a83e8172b0 2024-09-11T22:23:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.258+0000 7f23d4209740 1 -- 172.21.15.5:0/3518114737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 msgr2=0x55a83e825b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.258+0000 7f23d4209740 1 --2- 172.21.15.5:0/3518114737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e825b80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005fa0 tx=0x7f23b4010000 comp rx=0 tx=0).stop 2024-09-11T22:23:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 -- 172.21.15.5:0/3518114737 shutdown_connections 2024-09-11T22:23:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 --2- 172.21.15.5:0/3518114737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e8284b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 --2- 172.21.15.5:0/3518114737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e825b80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 --2- 172.21.15.5:0/3518114737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 0x55a83e816d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 -- 172.21.15.5:0/3518114737 >> 172.21.15.5:0/3518114737 conn(0x55a83e813060 msgr2=0x55a83e813440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 -- 172.21.15.5:0/3518114737 shutdown_connections 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 -- 172.21.15.5:0/3518114737 wait complete. 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 -- start start 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 Processor -- start 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 0x55a83e823060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e8235a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e823ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a83e828c90 con 0x55a83e8172b0 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a83e704290 con 0x55a83e816990 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a83e828e10 con 0x55a83e8260c0 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.259+0000 7f23d4209740 10 client.0 _ll_register_callbacks cb 0x55a83e735140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e8235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e8235a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54172/0 (socket says 172.21.15.5:54172) 2024-09-11T22:23:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23c3fff640 1 -- 172.21.15.5:0/885867783 learned_addr learned my addr 172.21.15.5:0/885867783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23cb7fe640 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e823ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23caffd640 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 0x55a83e823060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23c3fff640 1 -- 172.21.15.5:0/885867783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 msgr2=0x55a83e823ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23c3fff640 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e823ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23c3fff640 1 -- 172.21.15.5:0/885867783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 msgr2=0x55a83e823060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23c3fff640 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 0x55a83e823060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23c3fff640 1 -- 172.21.15.5:0/885867783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a83e48e210 con 0x55a83e8172b0 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23caffd640 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 0x55a83e823060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23c3fff640 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e8235a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f23b4012040 tx=0x7f23b400db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272eedb2573376f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23b27fc640 1 -- 172.21.15.5:0/885867783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b4017040 con 0x55a83e8172b0 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23b27fc640 1 -- 172.21.15.5:0/885867783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f23b400fd10 con 0x55a83e8172b0 2024-09-11T22:23:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23d4209740 1 -- 172.21.15.5:0/885867783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23b4023070 con 0x55a83e8172b0 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23b27fc640 1 -- 172.21.15.5:0/885867783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23b401e470 con 0x55a83e8172b0 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23d4209740 10 client.15648 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.260+0000 7f23d4209740 1 -- 172.21.15.5:0/885867783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a83e55a620 con 0x55a83e8172b0 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.261+0000 7f23b27fc640 1 -- 172.21.15.5:0/885867783 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23b4023070 con 0x55a83e8172b0 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.261+0000 7f23b27fc640 1 -- 172.21.15.5:0/885867783 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f23b403a910 con 0x55a83e8172b0 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.261+0000 7f23d4209740 10 client.15648 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.261+0000 7f23d4209740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.261+0000 7f23d4209740 20 client.15648 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.261+0000 7f23d4209740 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a83e836040 0x55a83e838430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.261+0000 7f23d4209740 4 client.15648 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.261+0000 7f23d4209740 1 -- 172.21.15.5:0/885867783 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a83e788a20 con 0x55a83e836040 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.261+0000 7f23caffd640 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a83e836040 0x55a83e838430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.262+0000 7f23b27fc640 10 client.15648 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.262+0000 7f23caffd640 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a83e836040 0x55a83e838430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc714ff994b12cd1 server_cookie=e6505989310ebafb in_seq=0 out_seq=0 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.262+0000 7f23b27fc640 1 -- 172.21.15.5:0/885867783 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a83e788a20 con 0x55a83e836040 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.262+0000 7f23b27fc640 10 client.15648 handle_command_reply: tid=0 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.262+0000 7f23d4209740 1 client.15648 shutdown 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.262+0000 7f23d4209740 1 -- 172.21.15.5:0/885867783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 msgr2=0x55a83e8235a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.262+0000 7f23d4209740 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e8235a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f23b4012040 tx=0x7f23b400db80 comp rx=0 tx=0).stop 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.262+0000 7f23d4209740 1 -- 172.21.15.5:0/885867783 shutdown_connections 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.262+0000 7f23d4209740 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a83e836040 0x55a83e838430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.263+0000 7f23d4209740 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e823ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.263+0000 7f23d4209740 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8172b0 0x55a83e8235a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.263+0000 7f23d4209740 1 --2- 172.21.15.5:0/885867783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e816990 0x55a83e823060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.263+0000 7f23b27fc640 0 client.15648 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.263+0000 7f23d4209740 1 -- 172.21.15.5:0/885867783 >> 172.21.15.5:0/885867783 conn(0x55a83e813060 msgr2=0x55a83e818d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.263+0000 7f23d4209740 1 -- 172.21.15.5:0/885867783 shutdown_connections 2024-09-11T22:23:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.263+0000 7f23d4209740 1 -- 172.21.15.5:0/885867783 wait complete. 2024-09-11T22:23:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.263+0000 7f23d4209740 20 client.15648 trim_cache size 0 max 16384 2024-09-11T22:23:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.277+0000 7f23d4209740 1 Processor -- start 2024-09-11T22:23:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.277+0000 7f23d4209740 1 -- start start 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23d4209740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e816990 0x55a83e8317e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23d4209740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 0x55a83e8202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23d4209740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e8207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23d4209740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc1ab630 con 0x55a83e816990 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23d4209740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc1ad170 con 0x55a83e8172b0 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23d4209740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc1abd00 con 0x55a83e8260c0 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23caffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e816990 0x55a83e8317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23caffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e816990 0x55a83e8317e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54182/0 (socket says 172.21.15.5:54182) 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23caffd640 1 -- 172.21.15.5:0/85602046 learned_addr learned my addr 172.21.15.5:0/85602046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23cb7fe640 1 --2- 172.21.15.5:0/85602046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e8207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23caffd640 1 -- 172.21.15.5:0/85602046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 msgr2=0x55a83e8207e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23caffd640 1 --2- 172.21.15.5:0/85602046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e8207e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23c3fff640 1 --2- 172.21.15.5:0/85602046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 0x55a83e8202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23caffd640 1 -- 172.21.15.5:0/85602046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 msgr2=0x55a83e8202a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23caffd640 1 --2- 172.21.15.5:0/85602046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 0x55a83e8202a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23caffd640 1 -- 172.21.15.5:0/85602046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a83e821590 con 0x55a83e816990 2024-09-11T22:23:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23c3fff640 1 --2- 172.21.15.5:0/85602046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 0x55a83e8202a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.278+0000 7f23caffd640 1 --2- 172.21.15.5:0/85602046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e816990 0x55a83e8317e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0bbfd0 tx=0x7f23bc0bc100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95ff40d6fa2e3dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.279+0000 7f23c0ff9640 1 -- 172.21.15.5:0/85602046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc049040 con 0x55a83e816990 2024-09-11T22:23:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.279+0000 7f23c0ff9640 1 -- 172.21.15.5:0/85602046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23bc0d6020 con 0x55a83e816990 2024-09-11T22:23:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.279+0000 7f23c0ff9640 1 -- 172.21.15.5:0/85602046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0c00d0 con 0x55a83e816990 2024-09-11T22:23:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.279+0000 7f23d4209740 1 -- 172.21.15.5:0/85602046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e816990 msgr2=0x55a83e8317e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.279+0000 7f23d4209740 1 --2- 172.21.15.5:0/85602046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e816990 0x55a83e8317e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0bbfd0 tx=0x7f23bc0bc100 comp rx=0 tx=0).stop 2024-09-11T22:23:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.279+0000 7f23d4209740 1 -- 172.21.15.5:0/85602046 shutdown_connections 2024-09-11T22:23:07.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.279+0000 7f23d4209740 1 --2- 172.21.15.5:0/85602046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e8260c0 0x55a83e8207e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.279+0000 7f23d4209740 1 --2- 172.21.15.5:0/85602046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 0x55a83e8202a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.279+0000 7f23d4209740 1 --2- 172.21.15.5:0/85602046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e816990 0x55a83e8317e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.279+0000 7f23d4209740 1 -- 172.21.15.5:0/85602046 >> 172.21.15.5:0/85602046 conn(0x55a83e813060 msgr2=0x55a83e8310d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 1 -- 172.21.15.5:0/85602046 shutdown_connections 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 1 -- 172.21.15.5:0/85602046 wait complete. 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 1 -- start start 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 1 Processor -- start 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e816990 0x55a83e83b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 0x55a83e83ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8260c0 0x55a83e839a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc1acaa0 con 0x55a83e8260c0 2024-09-11T22:23:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc1ac3d0 con 0x55a83e8172b0 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc1a9610 con 0x55a83e816990 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23d4209740 10 client.0 _ll_register_callbacks cb 0x7f23b400dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8260c0 0x55a83e839a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8260c0 0x55a83e839a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54196/0 (socket says 172.21.15.5:54196) 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23cb7fe640 1 -- 172.21.15.5:0/4081106379 learned_addr learned my addr 172.21.15.5:0/4081106379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23c3fff640 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 0x55a83e83ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.280+0000 7f23caffd640 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e816990 0x55a83e83b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23cb7fe640 1 -- 172.21.15.5:0/4081106379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e816990 msgr2=0x55a83e83b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23cb7fe640 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e816990 0x55a83e83b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23cb7fe640 1 -- 172.21.15.5:0/4081106379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 msgr2=0x55a83e83ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23cb7fe640 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 0x55a83e83ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23cb7fe640 1 -- 172.21.15.5:0/4081106379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a83e83a130 con 0x55a83e8260c0 2024-09-11T22:23:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23cb7fe640 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8260c0 0x55a83e839a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f23c4012700 tx=0x7f23c4012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62fca7b7e717873 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23b27fc640 1 -- 172.21.15.5:0/4081106379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4021060 con 0x55a83e8260c0 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23b27fc640 1 -- 172.21.15.5:0/4081106379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f23c4003b70 con 0x55a83e8260c0 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23d4209740 1 -- 172.21.15.5:0/4081106379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23bc0d6020 con 0x55a83e8260c0 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23b27fc640 1 -- 172.21.15.5:0/4081106379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f23c4003d70 con 0x55a83e8260c0 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23d4209740 10 client.15654 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23d4209740 1 -- 172.21.15.5:0/4081106379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a83e8a14a0 con 0x55a83e8260c0 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.281+0000 7f23b27fc640 1 -- 172.21.15.5:0/4081106379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23bc0d6020 con 0x55a83e8260c0 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.282+0000 7f23b27fc640 1 -- 172.21.15.5:0/4081106379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f23c401b030 con 0x55a83e8260c0 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.282+0000 7f23d4209740 10 client.15654 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.282+0000 7f23d4209740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.282+0000 7f23d4209740 20 client.15654 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.282+0000 7f23d4209740 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a83e8a1c50 0x55a83e8a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.282+0000 7f23d4209740 4 client.15654 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:23:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.282+0000 7f23d4209740 1 -- 172.21.15.5:0/4081106379 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a83e782d70 con 0x55a83e8a1c50 2024-09-11T22:23:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.282+0000 7f23caffd640 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a83e8a1c50 0x55a83e8a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.282+0000 7f23caffd640 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a83e8a1c50 0x55a83e8a4070 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76d610df6485013f server_cookie=d92be8d23ed8739a in_seq=0 out_seq=0 2024-09-11T22:23:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.282+0000 7f23b27fc640 10 client.15654 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.289+0000 7f23b27fc640 1 -- 172.21.15.5:0/4081106379 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a83e782d70 con 0x55a83e8a1c50 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.289+0000 7f23b27fc640 10 client.15654 handle_command_reply: tid=0 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.289+0000 7f23d4209740 1 client.15654 shutdown 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 1 -- 172.21.15.5:0/4081106379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8260c0 msgr2=0x55a83e839a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8260c0 0x55a83e839a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f23c4012700 tx=0x7f23c4012bc0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 1 -- 172.21.15.5:0/4081106379 shutdown_connections 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a83e8a1c50 0x55a83e8a4070 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a83e8260c0 0x55a83e839a40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a83e8172b0 0x55a83e83ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 1 --2- 172.21.15.5:0/4081106379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a83e816990 0x55a83e83b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 1 -- 172.21.15.5:0/4081106379 >> 172.21.15.5:0/4081106379 conn(0x55a83e813060 msgr2=0x55a83e8318f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:07.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23b27fc640 0 client.15654 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 1 -- 172.21.15.5:0/4081106379 shutdown_connections 2024-09-11T22:23:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 1 -- 172.21.15.5:0/4081106379 wait complete. 2024-09-11T22:23:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d4209740 20 client.15654 trim_cache size 0 max 16384 2024-09-11T22:23:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:23:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:23:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:23:07.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:23:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d1391640 1 -- 172.21.15.5:0/3848156413 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f23b80a2fd0 msgr2=0x7f23b80a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d1391640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f23b80a2fd0 0x7f23b80a5490 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f23b4005fd0 tx=0x7f23b4005850 comp rx=0 tx=0).stop 2024-09-11T22:23:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d1391640 1 -- 172.21.15.5:0/3848156413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc102d20 msgr2=0x7f23cc198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.290+0000 7f23d1391640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc102d20 0x7f23cc198350 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f23bc006010 tx=0x7f23bc0042f0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.291+0000 7f23d1391640 1 -- 172.21.15.5:0/3848156413 shutdown_connections 2024-09-11T22:23:07.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.291+0000 7f23d1391640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f23b80a2fd0 0x7f23b80a5490 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.291+0000 7f23d1391640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23cc10c0f0 0x7f23cc19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.291+0000 7f23d1391640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23cc103640 0x7f23cc198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.291+0000 7f23d1391640 1 --2- 172.21.15.5:0/3848156413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc102d20 0x7f23cc198350 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.291+0000 7f23d1391640 1 -- 172.21.15.5:0/3848156413 >> 172.21.15.5:0/3848156413 conn(0x7f23cc077990 msgr2=0x7f23cc0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.291+0000 7f23d1391640 1 -- 172.21.15.5:0/3848156413 shutdown_connections 2024-09-11T22:23:07.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:07.291+0000 7f23d1391640 1 -- 172.21.15.5:0/3848156413 wait complete. 2024-09-11T22:23:07.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:07 smithi005 ceph-mon[30943]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:23:07.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:07 smithi172 ceph-mon[35461]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:23:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:07.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:23:07.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:07 smithi159 ceph-mon[35534]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:23:07.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.968+0000 7fc780453640 1 -- 172.21.15.5:0/3968606774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc778102d30 msgr2=0x7fc778103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.968+0000 7fc780453640 1 --2- 172.21.15.5:0/3968606774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc778102d30 0x7fc778103110 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc774009930 tx=0x7fc77402a460 comp rx=0 tx=0).stop 2024-09-11T22:23:07.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.968+0000 7fc780453640 1 -- 172.21.15.5:0/3968606774 shutdown_connections 2024-09-11T22:23:07.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.968+0000 7fc780453640 1 --2- 172.21.15.5:0/3968606774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7781082a0 0x7fc77810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.968+0000 7fc780453640 1 --2- 172.21.15.5:0/3968606774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7781036e0 0x7fc778107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.968+0000 7fc780453640 1 --2- 172.21.15.5:0/3968606774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc778102d30 0x7fc778103110 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.968+0000 7fc780453640 1 -- 172.21.15.5:0/3968606774 >> 172.21.15.5:0/3968606774 conn(0x7fc7780fc7d0 msgr2=0x7fc7780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:07.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.968+0000 7fc780453640 1 -- 172.21.15.5:0/3968606774 shutdown_connections 2024-09-11T22:23:07.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.968+0000 7fc780453640 1 -- 172.21.15.5:0/3968606774 wait complete. 2024-09-11T22:23:07.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.969+0000 7fc780453640 1 Processor -- start 2024-09-11T22:23:07.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.969+0000 7fc780453640 1 -- start start 2024-09-11T22:23:07.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.969+0000 7fc780453640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc778102d30 0x7fc7781a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.969+0000 7fc780453640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7781036e0 0x7fc7781a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.969+0000 7fc780453640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7781082a0 0x7fc77819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.969+0000 7fc780453640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc778112130 con 0x7fc7781082a0 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.969+0000 7fc780453640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc778111fb0 con 0x7fc7781036e0 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.969+0000 7fc780453640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7781122b0 con 0x7fc778102d30 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77e9f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7781082a0 0x7fc77819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77e9f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7781082a0 0x7fc77819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54212/0 (socket says 172.21.15.5:54212) 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77e9f0640 1 -- 172.21.15.5:0/94438299 learned_addr learned my addr 172.21.15.5:0/94438299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77e1ef640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc778102d30 0x7fc7781a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77d9ee640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7781036e0 0x7fc7781a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77e9f0640 1 -- 172.21.15.5:0/94438299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc778102d30 msgr2=0x7fc7781a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77e9f0640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc778102d30 0x7fc7781a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77e9f0640 1 -- 172.21.15.5:0/94438299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7781036e0 msgr2=0x7fc7781a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77e9f0640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7781036e0 0x7fc7781a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77e9f0640 1 -- 172.21.15.5:0/94438299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77819ad30 con 0x7fc7781082a0 2024-09-11T22:23:07.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77d9ee640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7781036e0 0x7fc7781a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc77e9f0640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7781082a0 0x7fc77819a600 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc77000cce0 tx=0x7fc77000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc76f7fe640 1 -- 172.21.15.5:0/94438299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77000f450 con 0x7fc7781082a0 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc76f7fe640 1 -- 172.21.15.5:0/94438299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77000fad0 con 0x7fc7781082a0 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc780453640 1 -- 172.21.15.5:0/94438299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77819b020 con 0x7fc7781082a0 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc76f7fe640 1 -- 172.21.15.5:0/94438299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770018c10 con 0x7fc7781082a0 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.970+0000 7fc780453640 1 -- 172.21.15.5:0/94438299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7781a7310 con 0x7fc7781082a0 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.971+0000 7fc780453640 1 -- 172.21.15.5:0/94438299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77806a850 con 0x7fc7781082a0 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.973+0000 7fc76f7fe640 1 -- 172.21.15.5:0/94438299 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7fc77000f5f0 con 0x7fc7781082a0 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.974+0000 7fc76f7fe640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7640a2f20 0x7fc7640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.974+0000 7fc77e1ef640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7640a2f20 0x7fc7640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.974+0000 7fc76f7fe640 1 -- 172.21.15.5:0/94438299 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fc7700cffa0 con 0x7fc7781082a0 2024-09-11T22:23:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.974+0000 7fc77e1ef640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7640a2f20 0x7fc7640a53e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc774006010 tx=0x7fc774000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:07.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:07.981+0000 7fc76f7fe640 1 -- 172.21.15.5:0/94438299 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc770014030 con 0x7fc7781082a0 2024-09-11T22:23:08.026 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.213+0000 7fc780453640 1 -- 172.21.15.5:0/94438299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc778103110 con 0x7fc7781082a0 2024-09-11T22:23:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.214+0000 7fc76f7fe640 1 -- 172.21.15.5:0/94438299 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7fc778103110 con 0x7fc7781082a0 2024-09-11T22:23:08.217 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:23:08.217 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-11T22:22:15:230183+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:22:15.180031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:23:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-11T22:23:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 -- 172.21.15.5:0/94438299 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7640a2f20 msgr2=0x7fc7640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7640a2f20 0x7fc7640a53e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc774006010 tx=0x7fc774000bc0 comp rx=0 tx=0).stop 2024-09-11T22:23:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 -- 172.21.15.5:0/94438299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7781082a0 msgr2=0x7fc77819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7781082a0 0x7fc77819a600 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc77000cce0 tx=0x7fc77000a620 comp rx=0 tx=0).stop 2024-09-11T22:23:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 -- 172.21.15.5:0/94438299 shutdown_connections 2024-09-11T22:23:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7781082a0 0x7fc77819a600 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7640a2f20 0x7fc7640a53e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7781036e0 0x7fc7781a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 --2- 172.21.15.5:0/94438299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc778102d30 0x7fc7781a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 -- 172.21.15.5:0/94438299 >> 172.21.15.5:0/94438299 conn(0x7fc7780fc7d0 msgr2=0x7fc7780fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 -- 172.21.15.5:0/94438299 shutdown_connections 2024-09-11T22:23:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:08.217+0000 7fc780453640 1 -- 172.21.15.5:0/94438299 wait complete. 2024-09-11T22:23:09.069 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-11T22:23:09.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-11T22:23:09.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:09 smithi005 ceph-mon[30943]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-11T22:23:09.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:09 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/94438299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.282+0000 7fa7031f7640 1 -- 172.21.15.5:0/2475018907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6fc102d30 msgr2=0x7fa6fc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.282+0000 7fa7031f7640 1 --2- 172.21.15.5:0/2475018907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6fc102d30 0x7fa6fc103110 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009cd0 tx=0x7fa6f402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.282+0000 7fa7031f7640 1 -- 172.21.15.5:0/2475018907 shutdown_connections 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.282+0000 7fa7031f7640 1 --2- 172.21.15.5:0/2475018907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc1082a0 0x7fa6fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.282+0000 7fa7031f7640 1 --2- 172.21.15.5:0/2475018907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6fc1036e0 0x7fa6fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.282+0000 7fa7031f7640 1 --2- 172.21.15.5:0/2475018907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6fc102d30 0x7fa6fc103110 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.282+0000 7fa7031f7640 1 -- 172.21.15.5:0/2475018907 >> 172.21.15.5:0/2475018907 conn(0x7fa6fc0fc7d0 msgr2=0x7fa6fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.282+0000 7fa7031f7640 1 -- 172.21.15.5:0/2475018907 shutdown_connections 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.282+0000 7fa7031f7640 1 -- 172.21.15.5:0/2475018907 wait complete. 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.283+0000 7fa7031f7640 1 Processor -- start 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.283+0000 7fa7031f7640 1 -- start start 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa7031f7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6fc102d30 0x7fa6fc1a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa7031f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6fc1036e0 0x7fa6fc1a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa7031f7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc1082a0 0x7fa6fc10de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa7031f7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1120a0 con 0x7fa6fc1036e0 2024-09-11T22:23:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa7031f7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc111f20 con 0x7fa6fc1082a0 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa7031f7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc112220 con 0x7fa6fc102d30 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6fc1036e0 0x7fa6fc1a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa701794640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc1082a0 0x7fa6fc10de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa700f93640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6fc102d30 0x7fa6fc1a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa701794640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc1082a0 0x7fa6fc10de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37144/0 (socket says 172.21.15.5:37144) 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa701794640 1 -- 172.21.15.5:0/2472448036 learned_addr learned my addr 172.21.15.5:0/2472448036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa6fbfff640 1 -- 172.21.15.5:0/2472448036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6fc102d30 msgr2=0x7fa6fc1a48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6fc102d30 0x7fa6fc1a48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa6fbfff640 1 -- 172.21.15.5:0/2472448036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc1082a0 msgr2=0x7fa6fc10de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc1082a0 0x7fa6fc10de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.284+0000 7fa6fbfff640 1 -- 172.21.15.5:0/2472448036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc10e5a0 con 0x7fa6fc1036e0 2024-09-11T22:23:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.285+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6fc1036e0 0x7fa6fc1a4e30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc06a370 tx=0x7fa6e400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.285+0000 7fa6f9ffb640 1 -- 172.21.15.5:0/2472448036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4017070 con 0x7fa6fc1036e0 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.285+0000 7fa6f9ffb640 1 -- 172.21.15.5:0/2472448036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e40099a0 con 0x7fa6fc1036e0 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.285+0000 7fa6f9ffb640 1 -- 172.21.15.5:0/2472448036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40047a0 con 0x7fa6fc1036e0 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.285+0000 7fa7031f7640 1 -- 172.21.15.5:0/2472448036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc10e890 con 0x7fa6fc1036e0 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.285+0000 7fa7031f7640 1 -- 172.21.15.5:0/2472448036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc1ab730 con 0x7fa6fc1036e0 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.288+0000 7fa7055d0740 1 Processor -- start 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.288+0000 7fa6f9ffb640 1 -- 172.21.15.5:0/2472448036 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7fa6e4009b50 con 0x7fa6fc1036e0 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.288+0000 7fa7055d0740 1 -- start start 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa109090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 0x556baa109a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 0x556baa116760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa7055d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556baa06f2f0 con 0x556baa10ab40 2024-09-11T22:23:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa7055d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556baa016cf0 con 0x556baa1095d0 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa7055d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556baa06b2b0 con 0x556baa114370 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa701794640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa109090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa701794640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa109090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54248/0 (socket says 172.21.15.5:54248) 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa701794640 1 -- 172.21.15.5:0/433351303 learned_addr learned my addr 172.21.15.5:0/433351303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa700f93640 1 --2- 172.21.15.5:0/433351303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 0x556baa109a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/433351303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 0x556baa116760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.289+0000 7fa701794640 1 -- 172.21.15.5:0/433351303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 msgr2=0x556baa116760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.290+0000 7fa701794640 1 --2- 172.21.15.5:0/433351303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 0x556baa116760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.290+0000 7fa701794640 1 -- 172.21.15.5:0/433351303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 msgr2=0x556baa109a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.290+0000 7fa701794640 1 --2- 172.21.15.5:0/433351303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 0x556baa109a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.290+0000 7fa701794640 1 -- 172.21.15.5:0/433351303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba9ffb3a0 con 0x556baa10ab40 2024-09-11T22:23:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.290+0000 7fa6f9ffb640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6e80a2ea0 0x7fa6e80a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.290+0000 7fa701794640 1 --2- 172.21.15.5:0/433351303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa109090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00bc10 tx=0x7fa6ec0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3544485156bf55ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.290+0000 7fa6f9ffb640 1 -- 172.21.15.5:0/2472448036 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa6e40cf310 con 0x7fa6fc1036e0 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.290+0000 7fa700f93640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6e80a2ea0 0x7fa6e80a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.290+0000 7fa6f2ffd640 1 -- 172.21.15.5:0/433351303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec01c480 con 0x556baa10ab40 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.290+0000 7fa6f2ffd640 1 -- 172.21.15.5:0/433351303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6ec01cb00 con 0x556baa10ab40 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa6f2ffd640 1 -- 172.21.15.5:0/433351303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec011b20 con 0x556baa10ab40 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa7055d0740 1 -- 172.21.15.5:0/433351303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 msgr2=0x556baa109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa700f93640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6e80a2ea0 0x7fa6e80a5360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4006010 tx=0x7fa6f4005ef0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa7055d0740 1 --2- 172.21.15.5:0/433351303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa109090 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00bc10 tx=0x7fa6ec0107a0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa7055d0740 1 -- 172.21.15.5:0/433351303 shutdown_connections 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa7055d0740 1 --2- 172.21.15.5:0/433351303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 0x556baa116760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa7055d0740 1 --2- 172.21.15.5:0/433351303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 0x556baa109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa7055d0740 1 --2- 172.21.15.5:0/433351303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa109090 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa7055d0740 1 -- 172.21.15.5:0/433351303 >> 172.21.15.5:0/433351303 conn(0x556baa0d2310 msgr2=0x556baa105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa7055d0740 1 -- 172.21.15.5:0/433351303 shutdown_connections 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.291+0000 7fa7055d0740 1 -- 172.21.15.5:0/433351303 wait complete. 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa7055d0740 1 -- start start 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa7055d0740 1 Processor -- start 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa1095d0 0x556baa113ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa110f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 0x556baa111490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa7055d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556baa117110 con 0x556baa10ab40 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa7055d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556baa02a060 con 0x556baa114370 2024-09-11T22:23:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa7055d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556baa117290 con 0x556baa1095d0 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa7055d0740 10 client.0 _ll_register_callbacks cb 0x556ba9d7d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa110f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa110f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54260/0 (socket says 172.21.15.5:54260) 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.292+0000 7fa6fbfff640 1 -- 172.21.15.5:0/2515448846 learned_addr learned my addr 172.21.15.5:0/2515448846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa701794640 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 0x556baa111490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa700f93640 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa1095d0 0x556baa113ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa700f93640 1 -- 172.21.15.5:0/2515448846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 msgr2=0x556baa111490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa700f93640 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 0x556baa111490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa700f93640 1 -- 172.21.15.5:0/2515448846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 msgr2=0x556baa110f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa700f93640 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa110f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa700f93640 1 -- 172.21.15.5:0/2515448846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba9de03c0 con 0x556baa1095d0 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa701794640 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 0x556baa111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa110f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa700f93640 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa1095d0 0x556baa113ca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005e30 tx=0x7fa6f402a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601331a417f4f033 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa6dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa6dbfff640 1 -- 172.21.15.5:0/2515448846 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4056060 con 0x556baa1095d0 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa6dbfff640 1 -- 172.21.15.5:0/2515448846 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa6f4041030 con 0x556baa1095d0 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.293+0000 7fa7055d0740 1 -- 172.21.15.5:0/2515448846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6ec01cb00 con 0x556baa1095d0 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa6dbfff640 1 -- 172.21.15.5:0/2515448846 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6f4003e00 con 0x556baa1095d0 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa7055d0740 10 client.25198 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa7055d0740 1 -- 172.21.15.5:0/2515448846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556baa122b50 con 0x556baa1095d0 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa6dbfff640 1 -- 172.21.15.5:0/2515448846 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f4036040 con 0x556baa1095d0 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa6dbfff640 1 -- 172.21.15.5:0/2515448846 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6f4030070 con 0x556baa1095d0 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa7055d0740 10 client.25198 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa7055d0740 10 client.25198 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa7055d0740 20 client.25198 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa7055d0740 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556baa124010 0x556baa126400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa7055d0740 4 client.25198 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.294+0000 7fa7055d0740 1 -- 172.21.15.5:0/2515448846 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa6ec01cb00 con 0x556baa124010 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.295+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556baa124010 0x556baa126400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.295+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556baa124010 0x556baa126400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7085d55882ddb0bf server_cookie=1dab7046ce03299c in_seq=0 out_seq=0 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.295+0000 7fa6dbfff640 10 client.25198 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa6dbfff640 1 -- 172.21.15.5:0/2515448846 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa6ec01cb00 con 0x556baa124010 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa6dbfff640 10 client.25198 handle_command_reply: tid=0 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 client.25198 shutdown 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 -- 172.21.15.5:0/2515448846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa1095d0 msgr2=0x556baa113ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa1095d0 0x556baa113ca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005e30 tx=0x7fa6f402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 -- 172.21.15.5:0/2515448846 shutdown_connections 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556baa124010 0x556baa126400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 0x556baa111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa110f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 --2- 172.21.15.5:0/2515448846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa1095d0 0x556baa113ca0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa6dbfff640 0 client.25198 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 -- 172.21.15.5:0/2515448846 >> 172.21.15.5:0/2515448846 conn(0x556baa0d2310 msgr2=0x556baa10b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 -- 172.21.15.5:0/2515448846 shutdown_connections 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 1 -- 172.21.15.5:0/2515448846 wait complete. 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.296+0000 7fa7055d0740 20 client.25198 trim_cache size 0 max 16384 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.309+0000 7fa7055d0740 1 Processor -- start 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.309+0000 7fa7055d0740 1 -- start start 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.309+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 0x556baa120400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa10c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 0x556baa10c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa7055d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1b3f90 con 0x556baa10ab40 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa7055d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1b5ad0 con 0x556baa1095d0 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa7055d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1b4660 con 0x556baa114370 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa10c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa10c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54264/0 (socket says 172.21.15.5:54264) 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6fbfff640 1 -- 172.21.15.5:0/1031089694 learned_addr learned my addr 172.21.15.5:0/1031089694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa700f93640 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 0x556baa120400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa701794640 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 0x556baa10c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6fbfff640 1 -- 172.21.15.5:0/1031089694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 msgr2=0x556baa10c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 0x556baa10c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6fbfff640 1 -- 172.21.15.5:0/1031089694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 msgr2=0x556baa120400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 0x556baa120400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6fbfff640 1 -- 172.21.15.5:0/1031089694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556baa114010 con 0x556baa10ab40 2024-09-11T22:23:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa701794640 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 0x556baa10c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa700f93640 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 0x556baa120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa10c120 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40cc960 tx=0x7fa6e4086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d037a5aca9f65b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6f27fc640 1 -- 172.21.15.5:0/1031089694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40b4070 con 0x556baa10ab40 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6f27fc640 1 -- 172.21.15.5:0/1031089694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e409bcf0 con 0x556baa10ab40 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.310+0000 7fa6f27fc640 1 -- 172.21.15.5:0/1031089694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4097c60 con 0x556baa10ab40 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.311+0000 7fa7055d0740 1 -- 172.21.15.5:0/1031089694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 msgr2=0x556baa10c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.311+0000 7fa7055d0740 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa10c120 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40cc960 tx=0x7fa6e4086040 comp rx=0 tx=0).stop 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.311+0000 7fa7055d0740 1 -- 172.21.15.5:0/1031089694 shutdown_connections 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.311+0000 7fa7055d0740 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa114370 0x556baa10c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.311+0000 7fa7055d0740 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa10c120 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.311+0000 7fa7055d0740 1 --2- 172.21.15.5:0/1031089694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa1095d0 0x556baa120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.311+0000 7fa7055d0740 1 -- 172.21.15.5:0/1031089694 >> 172.21.15.5:0/1031089694 conn(0x556baa0d2310 msgr2=0x556baa110f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.311+0000 7fa7055d0740 1 -- 172.21.15.5:0/1031089694 shutdown_connections 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.311+0000 7fa7055d0740 1 -- 172.21.15.5:0/1031089694 wait complete. 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.311+0000 7fa7055d0740 1 -- start start 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa7055d0740 1 Processor -- start 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa1095d0 0x556baa11c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa11cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa7055d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 0x556baa10d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa7055d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1b5400 con 0x556baa10ab40 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa7055d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1b4d30 con 0x556baa114370 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa7055d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc1b1f70 con 0x556baa1095d0 2024-09-11T22:23:09.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa7055d0740 10 client.0 _ll_register_callbacks cb 0x7fa6f4031700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa11cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa6fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa11cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54274/0 (socket says 172.21.15.5:54274) 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa6fbfff640 1 -- 172.21.15.5:0/1961117053 learned_addr learned my addr 172.21.15.5:0/1961117053 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa701794640 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 0x556baa10d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa6fbfff640 1 -- 172.21.15.5:0/1961117053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa1095d0 msgr2=0x556baa11c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa1095d0 0x556baa11c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa6fbfff640 1 -- 172.21.15.5:0/1961117053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 msgr2=0x556baa10d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 0x556baa10d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa6fbfff640 1 -- 172.21.15.5:0/1961117053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556baa10e090 con 0x556baa10ab40 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.312+0000 7fa701794640 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 0x556baa10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa6fbfff640 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa11cd40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa6e409b580 tx=0x7fa6e409af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f763e473a3e461 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:09.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa6dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa6dbfff640 1 -- 172.21.15.5:0/1961117053 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40b4040 con 0x556baa10ab40 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa6dbfff640 1 -- 172.21.15.5:0/1961117053 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa6e4084b30 con 0x556baa10ab40 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa7055d0740 1 -- 172.21.15.5:0/1961117053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6e409bcf0 con 0x556baa10ab40 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa6dbfff640 1 -- 172.21.15.5:0/1961117053 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6e409f9a0 con 0x556baa10ab40 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa7055d0740 10 client.15678 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa7055d0740 1 -- 172.21.15.5:0/1961117053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556baa18e730 con 0x556baa10ab40 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa6dbfff640 1 -- 172.21.15.5:0/1961117053 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6e409bcf0 con 0x556baa10ab40 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa6dbfff640 1 -- 172.21.15.5:0/1961117053 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa6e4097340 con 0x556baa10ab40 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa7055d0740 10 client.15678 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa7055d0740 10 client.15678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa7055d0740 20 client.15678 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa7055d0740 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556baa18ed70 0x556baa191160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:09.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa7055d0740 4 client.15678 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.313+0000 7fa7055d0740 1 -- 172.21.15.5:0/1961117053 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556baa07c860 con 0x556baa18ed70 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.314+0000 7fa700f93640 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556baa18ed70 0x556baa191160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.314+0000 7fa700f93640 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556baa18ed70 0x556baa191160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c3eb68d218b39f8 server_cookie=bf49ddf697f25fce in_seq=0 out_seq=0 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.314+0000 7fa6dbfff640 10 client.15678 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.314+0000 7fa6dbfff640 1 -- 172.21.15.5:0/1961117053 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556baa07c860 con 0x556baa18ed70 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.314+0000 7fa6dbfff640 10 client.15678 handle_command_reply: tid=0 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.314+0000 7fa7055d0740 1 client.15678 shutdown 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.314+0000 7fa7055d0740 1 -- 172.21.15.5:0/1961117053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 msgr2=0x556baa11cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.314+0000 7fa7055d0740 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa11cd40 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa6e409b580 tx=0x7fa6e409af50 comp rx=0 tx=0).stop 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.315+0000 7fa7055d0740 1 -- 172.21.15.5:0/1961117053 shutdown_connections 2024-09-11T22:23:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.315+0000 7fa7055d0740 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556baa18ed70 0x556baa191160 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.315+0000 7fa7055d0740 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556baa114370 0x556baa10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.315+0000 7fa7055d0740 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556baa10ab40 0x556baa11cd40 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.315+0000 7fa7055d0740 1 --2- 172.21.15.5:0/1961117053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556baa1095d0 0x556baa11c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.315+0000 7fa6dbfff640 0 client.15678 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:23:09.360 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:23:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.315+0000 7fa7055d0740 1 -- 172.21.15.5:0/1961117053 >> 172.21.15.5:0/1961117053 conn(0x556baa0d2310 msgr2=0x556baa1044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.315+0000 7fa7055d0740 1 -- 172.21.15.5:0/1961117053 shutdown_connections 2024-09-11T22:23:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.315+0000 7fa7055d0740 1 -- 172.21.15.5:0/1961117053 wait complete. 2024-09-11T22:23:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.315+0000 7fa7055d0740 20 client.15678 trim_cache size 0 max 16384 2024-09-11T22:23:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 -- 172.21.15.5:0/2472448036 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6e80a2ea0 msgr2=0x7fa6e80a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6e80a2ea0 0x7fa6e80a5360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4006010 tx=0x7fa6f4005ef0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 -- 172.21.15.5:0/2472448036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6fc1036e0 msgr2=0x7fa6fc1a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6fc1036e0 0x7fa6fc1a4e30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc06a370 tx=0x7fa6e400dbb0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 -- 172.21.15.5:0/2472448036 shutdown_connections 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6e80a2ea0 0x7fa6e80a5360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6fc1082a0 0x7fa6fc10de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6fc1036e0 0x7fa6fc1a4e30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 --2- 172.21.15.5:0/2472448036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6fc102d30 0x7fa6fc1a48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 -- 172.21.15.5:0/2472448036 >> 172.21.15.5:0/2472448036 conn(0x7fa6fc0fc7d0 msgr2=0x7fa6fc103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 -- 172.21.15.5:0/2472448036 shutdown_connections 2024-09-11T22:23:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:09.316+0000 7fa7031f7640 1 -- 172.21.15.5:0/2472448036 wait complete. 2024-09-11T22:23:09.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:09 smithi172 ceph-mon[35461]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-11T22:23:09.363 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:09 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/94438299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:09.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:09 smithi159 ceph-mon[35534]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 29 op/s 2024-09-11T22:23:09.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:09 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/94438299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:10.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:23:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:10.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.870+0000 7f2fd243f640 1 -- 172.21.15.5:0/3262949390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fcc102570 msgr2=0x7f2fcc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:10.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.870+0000 7f2fd243f640 1 --2- 172.21.15.5:0/3262949390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fcc102570 0x7f2fcc1029b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009900 tx=0x7f2fc002a480 comp rx=0 tx=0).stop 2024-09-11T22:23:10.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.870+0000 7f2fd243f640 1 -- 172.21.15.5:0/3262949390 shutdown_connections 2024-09-11T22:23:10.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.870+0000 7f2fd243f640 1 --2- 172.21.15.5:0/3262949390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.870+0000 7f2fd243f640 1 --2- 172.21.15.5:0/3262949390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fcc102570 0x7f2fcc1029b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.870+0000 7f2fd243f640 1 --2- 172.21.15.5:0/3262949390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc100e20 0x7f2fcc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.870+0000 7f2fd243f640 1 -- 172.21.15.5:0/3262949390 >> 172.21.15.5:0/3262949390 conn(0x7f2fcc0fc7d0 msgr2=0x7f2fcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.870+0000 7f2fd243f640 1 -- 172.21.15.5:0/3262949390 shutdown_connections 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.870+0000 7f2fd243f640 1 -- 172.21.15.5:0/3262949390 wait complete. 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.871+0000 7f2fd243f640 1 Processor -- start 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.871+0000 7f2fd243f640 1 -- start start 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fd243f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fcc100e20 0x7f2fcc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fd243f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc102570 0x7f2fcc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fd243f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fd243f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc06a300 con 0x7f2fcc100e20 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fd243f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc06a180 con 0x7f2fcc102570 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fd243f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc06a480 con 0x7f2fcc102ef0 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fcc100e20 0x7f2fcc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fcc100e20 0x7f2fcc198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54278/0 (socket says 172.21.15.5:54278) 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fcbfff640 1 -- 172.21.15.5:0/3395137099 learned_addr learned my addr 172.21.15.5:0/3395137099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fcb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc102570 0x7f2fcc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fd09dc640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fcbfff640 1 -- 172.21.15.5:0/3395137099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fcc102ef0 msgr2=0x7f2fcc19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fcbfff640 1 -- 172.21.15.5:0/3395137099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc102570 msgr2=0x7f2fcc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc102570 0x7f2fcc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fcbfff640 1 -- 172.21.15.5:0/3395137099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fcc19d420 con 0x7f2fcc100e20 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fd09dc640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.872+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc102570 0x7f2fcc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.873+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fcc100e20 0x7f2fcc198470 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00da90 tx=0x7f2fbc00df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.873+0000 7f2fc97fa640 1 -- 172.21.15.5:0/3395137099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc017070 con 0x7f2fcc100e20 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.873+0000 7f2fc97fa640 1 -- 172.21.15.5:0/3395137099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fbc009c00 con 0x7f2fcc100e20 2024-09-11T22:23:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.873+0000 7f2fc97fa640 1 -- 172.21.15.5:0/3395137099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc00f700 con 0x7f2fcc100e20 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.873+0000 7f2fd243f640 1 -- 172.21.15.5:0/3395137099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fcc19d710 con 0x7f2fcc100e20 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.873+0000 7f2fd243f640 1 -- 172.21.15.5:0/3395137099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fcc1a4ff0 con 0x7f2fcc100e20 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.876+0000 7f2fc97fa640 1 -- 172.21.15.5:0/3395137099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7f2fbc010460 con 0x7f2fcc100e20 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fc97fa640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2fb40a2f20 0x7f2fb40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fd4818740 1 Processor -- start 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fc97fa640 1 -- 172.21.15.5:0/3395137099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f2fbc0cf710 con 0x7f2fcc100e20 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2fb40a2f20 0x7f2fb40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fd4818740 1 -- start start 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c43b70 0x564144c43f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 0x564144c4e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ed50 0x564144c51140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fd4818740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564144ba52a0 con 0x564144c44490 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fd4818740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564144b4cb10 con 0x564144c43b70 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fd4818740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564144ba1260 con 0x564144c4ed50 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.877+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c43b70 0x564144c43f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c43b70 0x564144c43f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37180/0 (socket says 172.21.15.5:37180) 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fcbfff640 1 -- 172.21.15.5:0/1821051405 learned_addr learned my addr 172.21.15.5:0/1821051405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fd09dc640 1 --2- 172.21.15.5:0/1821051405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ed50 0x564144c51140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/1821051405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 0x564144c4e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2fb40a2f20 0x7f2fb40a53e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009230 tx=0x7f2fc00058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fcbfff640 1 -- 172.21.15.5:0/1821051405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ed50 msgr2=0x564144c51140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/1821051405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ed50 0x564144c51140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fcbfff640 1 -- 172.21.15.5:0/1821051405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 msgr2=0x564144c4e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/1821051405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 0x564144c4e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fcbfff640 1 -- 172.21.15.5:0/1821051405 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564144916b90 con 0x564144c43b70 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/1821051405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c43b70 0x564144c43f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0096d0 tx=0x7f2fbc09bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc78bba8e32986c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fb27fc640 1 -- 172.21.15.5:0/1821051405 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc099cf0 con 0x564144c43b70 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fb27fc640 1 -- 172.21.15.5:0/1821051405 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fbc010a60 con 0x564144c43b70 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.878+0000 7f2fb27fc640 1 -- 172.21.15.5:0/1821051405 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc09b0a0 con 0x564144c43b70 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.879+0000 7f2fd4818740 1 -- 172.21.15.5:0/1821051405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c43b70 msgr2=0x564144c43f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.879+0000 7f2fd4818740 1 --2- 172.21.15.5:0/1821051405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c43b70 0x564144c43f50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0096d0 tx=0x7f2fbc09bab0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.879+0000 7f2fd4818740 1 -- 172.21.15.5:0/1821051405 shutdown_connections 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.879+0000 7f2fd4818740 1 --2- 172.21.15.5:0/1821051405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ed50 0x564144c51140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.879+0000 7f2fd4818740 1 --2- 172.21.15.5:0/1821051405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 0x564144c4e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.879+0000 7f2fd4818740 1 --2- 172.21.15.5:0/1821051405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c43b70 0x564144c43f50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.879+0000 7f2fd4818740 1 -- 172.21.15.5:0/1821051405 >> 172.21.15.5:0/1821051405 conn(0x564144c40100 msgr2=0x564144c404e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.879+0000 7f2fd4818740 1 -- 172.21.15.5:0/1821051405 shutdown_connections 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.880+0000 7f2fd4818740 1 -- 172.21.15.5:0/1821051405 wait complete. 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.880+0000 7f2fd4818740 1 -- start start 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.880+0000 7f2fd4818740 1 Processor -- start 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c43b70 0x564144c4bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c44490 0x564144c4c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c43b70 0x564144c4bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c43b70 0x564144c4bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54294/0 (socket says 172.21.15.5:54294) 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fcbfff640 1 -- 172.21.15.5:0/589834539 learned_addr learned my addr 172.21.15.5:0/589834539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fd4818740 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ed50 0x564144c4c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564144c518c0 con 0x564144c43b70 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564144b2d6e0 con 0x564144c4ed50 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564144c51a40 con 0x564144c44490 2024-09-11T22:23:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fd4818740 10 client.0 _ll_register_callbacks cb 0x564144b51a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c44490 0x564144c4c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fcbfff640 1 -- 172.21.15.5:0/589834539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c44490 msgr2=0x564144c4c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c44490 0x564144c4c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fcbfff640 1 -- 172.21.15.5:0/589834539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ed50 msgr2=0x564144c4c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fd09dc640 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ed50 0x564144c4c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ed50 0x564144c4c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fcbfff640 1 -- 172.21.15.5:0/589834539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564144c5ade0 con 0x564144c43b70 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fd09dc640 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ed50 0x564144c4c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c43b70 0x564144c4bc50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0058c0 tx=0x7f2fbc0a47e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33f029dee900eaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.881+0000 7f2fab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.882+0000 7f2fab7fe640 1 -- 172.21.15.5:0/589834539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc0b3070 con 0x564144c43b70 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.882+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fbc010a60 con 0x564144c43b70 2024-09-11T22:23:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.882+0000 7f2fab7fe640 1 -- 172.21.15.5:0/589834539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f2fbc0ba310 con 0x564144c43b70 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.882+0000 7f2fab7fe640 1 -- 172.21.15.5:0/589834539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2fbc0860e0 con 0x564144c43b70 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.882+0000 7f2fd4818740 10 client.15690 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.882+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564144c5d730 con 0x564144c43b70 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.882+0000 7f2fab7fe640 1 -- 172.21.15.5:0/589834539 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fbc010a60 con 0x564144c43b70 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.882+0000 7f2fab7fe640 1 -- 172.21.15.5:0/589834539 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2fbc097740 con 0x564144c43b70 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.882+0000 7f2fd4818740 10 client.15690 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.882+0000 7f2fd4818740 10 client.15690 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.883+0000 7f2fd4818740 20 client.15690 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.883+0000 7f2fd4818740 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564144c5ed70 0x564144c611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.883+0000 7f2fd4818740 4 client.15690 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.883+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564144bb29d0 con 0x564144c5ed70 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.883+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564144c5ed70 0x564144c611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.885+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564144c5ed70 0x564144c611a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8fcac096542d66 server_cookie=bcce39529af4d9da in_seq=0 out_seq=0 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.885+0000 7f2fab7fe640 10 client.15690 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.886+0000 7f2fab7fe640 1 -- 172.21.15.5:0/589834539 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564144bb29d0 con 0x564144c5ed70 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.886+0000 7f2fab7fe640 10 client.15690 handle_command_reply: tid=0 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.886+0000 7f2fd4818740 1 client.15690 shutdown 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c43b70 msgr2=0x564144c4bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c43b70 0x564144c4bc50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0058c0 tx=0x7f2fbc0a47e0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 shutdown_connections 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564144c5ed70 0x564144c611a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ed50 0x564144c4c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c44490 0x564144c4c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 1 --2- 172.21.15.5:0/589834539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c43b70 0x564144c4bc50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 >> 172.21.15.5:0/589834539 conn(0x564144c40100 msgr2=0x564144c3ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fab7fe640 0 client.15690 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 shutdown_connections 2024-09-11T22:23:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 1 -- 172.21.15.5:0/589834539 wait complete. 2024-09-11T22:23:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.887+0000 7f2fd4818740 20 client.15690 trim_cache size 0 max 16384 2024-09-11T22:23:10.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.920+0000 7f2fd4818740 1 Processor -- start 2024-09-11T22:23:10.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.920+0000 7f2fd4818740 1 -- start start 2024-09-11T22:23:10.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 0x564144c66a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ed50 0x564144c66f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ac30 0x564144c48eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fd4818740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1ab700 con 0x564144c44490 2024-09-11T22:23:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fd4818740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1ad240 con 0x564144c4ac30 2024-09-11T22:23:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 0x564144c66a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fd4818740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1abdd0 con 0x564144c4ed50 2024-09-11T22:23:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 0x564144c66a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54310/0 (socket says 172.21.15.5:54310) 2024-09-11T22:23:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fcbfff640 1 -- 172.21.15.5:0/3470415032 learned_addr learned my addr 172.21.15.5:0/3470415032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/3470415032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ed50 0x564144c66f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fd09dc640 1 --2- 172.21.15.5:0/3470415032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ac30 0x564144c48eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fcbfff640 1 -- 172.21.15.5:0/3470415032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ed50 msgr2=0x564144c66f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/3470415032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ed50 0x564144c66f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fcbfff640 1 -- 172.21.15.5:0/3470415032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ac30 msgr2=0x564144c48eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/3470415032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ac30 0x564144c48eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.921+0000 7f2fcbfff640 1 -- 172.21.15.5:0/3470415032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564144c3ff30 con 0x564144c44490 2024-09-11T22:23:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.922+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/3470415032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 0x564144c66a40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc012040 tx=0x7f2fbc09bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceff1bb360ee4134 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.922+0000 7f2fb1ffb640 1 -- 172.21.15.5:0/3470415032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc0b3070 con 0x564144c44490 2024-09-11T22:23:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.922+0000 7f2fb1ffb640 1 -- 172.21.15.5:0/3470415032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fbc09a8b0 con 0x564144c44490 2024-09-11T22:23:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.922+0000 7f2fb1ffb640 1 -- 172.21.15.5:0/3470415032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc086280 con 0x564144c44490 2024-09-11T22:23:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.923+0000 7f2fd4818740 1 -- 172.21.15.5:0/3470415032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 msgr2=0x564144c66a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.923+0000 7f2fd4818740 1 --2- 172.21.15.5:0/3470415032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 0x564144c66a40 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc012040 tx=0x7f2fbc09bab0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.923+0000 7f2fd4818740 1 -- 172.21.15.5:0/3470415032 shutdown_connections 2024-09-11T22:23:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.923+0000 7f2fd4818740 1 --2- 172.21.15.5:0/3470415032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c4ac30 0x564144c48eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.923+0000 7f2fd4818740 1 --2- 172.21.15.5:0/3470415032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ed50 0x564144c66f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.923+0000 7f2fd4818740 1 --2- 172.21.15.5:0/3470415032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c44490 0x564144c66a40 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.923+0000 7f2fd4818740 1 -- 172.21.15.5:0/3470415032 >> 172.21.15.5:0/3470415032 conn(0x564144c40100 msgr2=0x564144c4a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.923+0000 7f2fd4818740 1 -- 172.21.15.5:0/3470415032 shutdown_connections 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.924+0000 7f2fd4818740 1 -- 172.21.15.5:0/3470415032 wait complete. 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.924+0000 7f2fd4818740 1 -- start start 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.924+0000 7f2fd4818740 1 Processor -- start 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c44490 0x564144c52e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ac30 0x564144c53370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd4818740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c4ed50 0x564144cc8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd4818740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1acb70 con 0x564144c4ed50 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd4818740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1ac4a0 con 0x564144c44490 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd4818740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2fcc1a96e0 con 0x564144c4ac30 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd4818740 10 client.0 _ll_register_callbacks cb 0x7f2fbc09c0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd09dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c4ed50 0x564144cc8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd09dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c4ed50 0x564144cc8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54322/0 (socket says 172.21.15.5:54322) 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd09dc640 1 -- 172.21.15.5:0/2750489073 learned_addr learned my addr 172.21.15.5:0/2750489073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ac30 0x564144c53370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c44490 0x564144c52e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fcb7fe640 1 -- 172.21.15.5:0/2750489073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c44490 msgr2=0x564144c52e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c44490 0x564144c52e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fcb7fe640 1 -- 172.21.15.5:0/2750489073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c4ed50 msgr2=0x564144cc8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c4ed50 0x564144cc8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fcb7fe640 1 -- 172.21.15.5:0/2750489073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564144c37260 con 0x564144c4ac30 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.925+0000 7f2fd09dc640 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c4ed50 0x564144cc8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.926+0000 7f2fcb7fe640 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ac30 0x564144c53370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0005940 tx=0x7f2fc003e070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8003b215653487ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.926+0000 7f2fab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.926+0000 7f2fab7fe640 1 -- 172.21.15.5:0/2750489073 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0035050 con 0x564144c4ac30 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.926+0000 7f2fab7fe640 1 -- 172.21.15.5:0/2750489073 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f2fc0031da0 con 0x564144c4ac30 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.926+0000 7f2fd4818740 1 -- 172.21.15.5:0/2750489073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fbc09a8b0 con 0x564144c4ac30 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.926+0000 7f2fab7fe640 1 -- 172.21.15.5:0/2750489073 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2fc0003e60 con 0x564144c4ac30 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.926+0000 7f2fd4818740 10 client.25216 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.926+0000 7f2fd4818740 1 -- 172.21.15.5:0/2750489073 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564144cc9f00 con 0x564144c4ac30 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.927+0000 7f2fab7fe640 1 -- 172.21.15.5:0/2750489073 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fbc09a8b0 con 0x564144c4ac30 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.927+0000 7f2fab7fe640 1 -- 172.21.15.5:0/2750489073 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2fc00522c0 con 0x564144c4ac30 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.927+0000 7f2fd4818740 10 client.25216 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.927+0000 7f2fd4818740 10 client.25216 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.927+0000 7f2fd4818740 20 client.25216 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:10.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.927+0000 7f2fd4818740 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564144cca6f0 0x564144cccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:10.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.927+0000 7f2fd4818740 4 client.25216 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-11T22:23:10.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.927+0000 7f2fd4818740 1 -- 172.21.15.5:0/2750489073 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x564144bacd20 con 0x564144cca6f0 2024-09-11T22:23:10.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.928+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564144cca6f0 0x564144cccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:10.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.928+0000 7f2fcbfff640 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564144cca6f0 0x564144cccb10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f564500947c0a1 server_cookie=c52fa7866283597c in_seq=0 out_seq=0 2024-09-11T22:23:10.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.928+0000 7f2fab7fe640 10 client.25216 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.940+0000 7f2fab7fe640 1 -- 172.21.15.5:0/2750489073 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564144bacd20 con 0x564144cca6f0 2024-09-11T22:23:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.940+0000 7f2fab7fe640 10 client.25216 handle_command_reply: tid=0 2024-09-11T22:23:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.940+0000 7f2fd4818740 1 client.25216 shutdown 2024-09-11T22:23:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 1 -- 172.21.15.5:0/2750489073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ac30 msgr2=0x564144c53370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ac30 0x564144c53370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0005940 tx=0x7f2fc003e070 comp rx=0 tx=0).stop 2024-09-11T22:23:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 1 -- 172.21.15.5:0/2750489073 shutdown_connections 2024-09-11T22:23:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564144cca6f0 0x564144cccb10 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564144c4ed50 0x564144cc8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564144c4ac30 0x564144c53370 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 1 --2- 172.21.15.5:0/2750489073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564144c44490 0x564144c52e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 1 -- 172.21.15.5:0/2750489073 >> 172.21.15.5:0/2750489073 conn(0x564144c40100 msgr2=0x564144c4bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fab7fe640 0 client.25216 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 1 -- 172.21.15.5:0/2750489073 shutdown_connections 2024-09-11T22:23:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 1 -- 172.21.15.5:0/2750489073 wait complete. 2024-09-11T22:23:10.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.941+0000 7f2fd4818740 20 client.25216 trim_cache size 0 max 16384 2024-09-11T22:23:10.943 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:23:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 28, 2024-09-11T22:23:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:23:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 2, 2024-09-11T22:23:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "7d67fffb": { 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 2, 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.5, 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:23:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:23:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:23:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:23:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:23:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:10.945 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:23:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 -- 172.21.15.5:0/3395137099 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2fb40a2f20 msgr2=0x7f2fb40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2fb40a2f20 0x7f2fb40a53e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0009230 tx=0x7f2fc00058d0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 -- 172.21.15.5:0/3395137099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fcc100e20 msgr2=0x7f2fcc198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fcc100e20 0x7f2fcc198470 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00da90 tx=0x7f2fbc00df50 comp rx=0 tx=0).stop 2024-09-11T22:23:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 -- 172.21.15.5:0/3395137099 shutdown_connections 2024-09-11T22:23:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fcc102ef0 0x7f2fcc19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2fb40a2f20 0x7f2fb40a53e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fcc102570 0x7f2fcc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 --2- 172.21.15.5:0/3395137099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fcc100e20 0x7f2fcc198470 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 -- 172.21.15.5:0/3395137099 >> 172.21.15.5:0/3395137099 conn(0x7f2fcc0fc7d0 msgr2=0x7f2fcc104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 -- 172.21.15.5:0/3395137099 shutdown_connections 2024-09-11T22:23:10.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:10.942+0000 7f2fd243f640 1 -- 172.21.15.5:0/3395137099 wait complete. 2024-09-11T22:23:11.350 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:11 smithi005 ceph-mon[30943]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:23:11.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:11 smithi172 ceph-mon[35461]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:23:11.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:11 smithi159 ceph-mon[35534]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-11T22:23:11.734 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7d67fffb', quiesce took 7.4 seconds. Will release after: 17.1 2024-09-11T22:23:11.823 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.349+0000 7f333e461640 1 -- 172.21.15.5:0/4181202952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3338100e20 msgr2=0x7f3338101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.349+0000 7f333e461640 1 --2- 172.21.15.5:0/4181202952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3338100e20 0x7f3338101200 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3328009900 tx=0x7f332802a480 comp rx=0 tx=0).stop 2024-09-11T22:23:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.349+0000 7f333e461640 1 -- 172.21.15.5:0/4181202952 shutdown_connections 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.349+0000 7f333e461640 1 --2- 172.21.15.5:0/4181202952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3338102ef0 0x7f333810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.349+0000 7f333e461640 1 --2- 172.21.15.5:0/4181202952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3338102570 0x7f33381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.349+0000 7f333e461640 1 --2- 172.21.15.5:0/4181202952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3338100e20 0x7f3338101200 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.349+0000 7f333e461640 1 -- 172.21.15.5:0/4181202952 >> 172.21.15.5:0/4181202952 conn(0x7f33380fc7d0 msgr2=0x7f33380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.349+0000 7f333e461640 1 -- 172.21.15.5:0/4181202952 shutdown_connections 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.349+0000 7f333e461640 1 -- 172.21.15.5:0/4181202952 wait complete. 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.350+0000 7f333e461640 1 Processor -- start 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.350+0000 7f333e461640 1 -- start start 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f333e461640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3338102570 0x7f3338198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f333e461640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3338102ef0 0x7f3338198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f333e461640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333819cfe0 0x7f333819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f333e461640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3338110f80 con 0x7f3338102ef0 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f333e461640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3338110e00 con 0x7f333819cfe0 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f333e461640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3338111100 con 0x7f3338102570 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f33377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3338102ef0 0x7f3338198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f33377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3338102ef0 0x7f3338198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59414/0 (socket says 172.21.15.5:59414) 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f33377fe640 1 -- 172.21.15.5:0/804671178 learned_addr learned my addr 172.21.15.5:0/804671178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f333c9fe640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333819cfe0 0x7f333819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f3337fff640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3338102570 0x7f3338198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f33377fe640 1 -- 172.21.15.5:0/804671178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3338102570 msgr2=0x7f3338198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f33377fe640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3338102570 0x7f3338198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f33377fe640 1 -- 172.21.15.5:0/804671178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333819cfe0 msgr2=0x7f333819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f33377fe640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333819cfe0 0x7f333819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f33377fe640 1 -- 172.21.15.5:0/804671178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333819daa0 con 0x7f3338102ef0 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.351+0000 7f333c9fe640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333819cfe0 0x7f333819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.352+0000 7f33377fe640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3338102ef0 0x7f3338198c50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f332c00d770 tx=0x7f332c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.352+0000 7f33357fa640 1 -- 172.21.15.5:0/804671178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c017070 con 0x7f3338102ef0 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.352+0000 7f33357fa640 1 -- 172.21.15.5:0/804671178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f332c005500 con 0x7f3338102ef0 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.352+0000 7f33357fa640 1 -- 172.21.15.5:0/804671178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c004e00 con 0x7f3338102ef0 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.352+0000 7f333e461640 1 -- 172.21.15.5:0/804671178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333819dd90 con 0x7f3338102ef0 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.352+0000 7f333e461640 1 -- 172.21.15.5:0/804671178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3338103910 con 0x7f3338102ef0 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f334083a740 1 Processor -- start 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f334083a740 1 -- start start 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f33357fa640 1 -- 172.21.15.5:0/804671178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7f332c004fa0 con 0x7f3338102ef0 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f334083a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a94b40 0x555590a93090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f334083a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a935d0 0x555590a93a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f334083a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590aa0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f334083a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5555909f92f0 con 0x555590a935d0 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f334083a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5555909a0cf0 con 0x555590a9e370 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f334083a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555909f52b0 con 0x555590a94b40 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f3337fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a935d0 0x555590a93a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f3337fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a935d0 0x555590a93a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59424/0 (socket says 172.21.15.5:59424) 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f3337fff640 1 -- 172.21.15.5:0/1413065579 learned_addr learned my addr 172.21.15.5:0/1413065579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.356+0000 7f333c9fe640 1 --2- 172.21.15.5:0/1413065579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a94b40 0x555590a93090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.357+0000 7f33377fe640 1 --2- 172.21.15.5:0/1413065579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590aa0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.357+0000 7f3337fff640 1 -- 172.21.15.5:0/1413065579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a94b40 msgr2=0x555590a93090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.357+0000 7f3337fff640 1 --2- 172.21.15.5:0/1413065579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a94b40 0x555590a93090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.357+0000 7f3337fff640 1 -- 172.21.15.5:0/1413065579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 msgr2=0x555590aa0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.357+0000 7f3337fff640 1 --2- 172.21.15.5:0/1413065579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590aa0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.357+0000 7f3337fff640 1 -- 172.21.15.5:0/1413065579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555909853a0 con 0x555590a935d0 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.357+0000 7f3337fff640 1 --2- 172.21.15.5:0/1413065579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a935d0 0x555590a93a10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f33280314e0 tx=0x7f33280040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ea56b6561d7fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.357+0000 7f33357fa640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33200a2f20 0x7f33200a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.357+0000 7f331e7fc640 1 -- 172.21.15.5:0/1413065579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328040070 con 0x555590a935d0 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.357+0000 7f333c9fe640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33200a2f20 0x7f33200a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f331e7fc640 1 -- 172.21.15.5:0/1413065579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33280044b0 con 0x555590a935d0 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f331e7fc640 1 -- 172.21.15.5:0/1413065579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328035040 con 0x555590a935d0 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f33357fa640 1 -- 172.21.15.5:0/804671178 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f332c0cf1b0 con 0x7f3338102ef0 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f334083a740 1 -- 172.21.15.5:0/1413065579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a935d0 msgr2=0x555590a93a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f333c9fe640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33200a2f20 0x7f33200a53e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3330005fd0 tx=0x7f3330011040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f334083a740 1 --2- 172.21.15.5:0/1413065579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a935d0 0x555590a93a10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f33280314e0 tx=0x7f33280040c0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f334083a740 1 -- 172.21.15.5:0/1413065579 shutdown_connections 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f334083a740 1 --2- 172.21.15.5:0/1413065579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590aa0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f334083a740 1 --2- 172.21.15.5:0/1413065579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a935d0 0x555590a93a10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f334083a740 1 --2- 172.21.15.5:0/1413065579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a94b40 0x555590a93090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.358+0000 7f334083a740 1 -- 172.21.15.5:0/1413065579 >> 172.21.15.5:0/1413065579 conn(0x555590a5c310 msgr2=0x555590a8fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.359+0000 7f334083a740 1 -- 172.21.15.5:0/1413065579 shutdown_connections 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.359+0000 7f334083a740 1 -- 172.21.15.5:0/1413065579 wait complete. 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.359+0000 7f334083a740 1 -- start start 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.359+0000 7f334083a740 1 Processor -- start 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.359+0000 7f334083a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a935d0 0x555590a9dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.359+0000 7f334083a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a94b40 0x555590a9afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.359+0000 7f334083a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a9e370 0x555590a9b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.359+0000 7f334083a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555590aa1010 con 0x555590a9e370 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.359+0000 7f334083a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5555909b4060 con 0x555590a935d0 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f334083a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555590aa1190 con 0x555590a94b40 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f334083a740 10 client.0 _ll_register_callbacks cb 0x5555907078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f333c9fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a9e370 0x555590a9b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f333c9fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a9e370 0x555590a9b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59434/0 (socket says 172.21.15.5:59434) 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f333c9fe640 1 -- 172.21.15.5:0/3370205530 learned_addr learned my addr 172.21.15.5:0/3370205530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f3337fff640 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a935d0 0x555590a9dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f33377fe640 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a94b40 0x555590a9afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f333c9fe640 1 -- 172.21.15.5:0/3370205530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a94b40 msgr2=0x555590a9afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f333c9fe640 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a94b40 0x555590a9afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f333c9fe640 1 -- 172.21.15.5:0/3370205530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a935d0 msgr2=0x555590a9dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:12.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f333c9fe640 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a935d0 0x555590a9dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f333c9fe640 1 -- 172.21.15.5:0/3370205530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55559076a3c0 con 0x555590a9e370 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f333c9fe640 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a9e370 0x555590a9b570 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f333001c8c0 tx=0x7f3330004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210b20673eccb7f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f33177fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.360+0000 7f33177fe640 1 -- 172.21.15.5:0/3370205530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333002c060 con 0x555590a9e370 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f334083a740 1 -- 172.21.15.5:0/3370205530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33280044b0 con 0x555590a9e370 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f33177fe640 1 -- 172.21.15.5:0/3370205530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f3330008bd0 con 0x555590a9e370 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f33177fe640 1 -- 172.21.15.5:0/3370205530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3330008dd0 con 0x555590a9e370 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f334083a740 10 client.15708 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f334083a740 1 -- 172.21.15.5:0/3370205530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555590aacac0 con 0x555590a9e370 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f33177fe640 1 -- 172.21.15.5:0/3370205530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33280044b0 con 0x555590a9e370 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f33177fe640 1 -- 172.21.15.5:0/3370205530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f333001a580 con 0x555590a9e370 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f334083a740 10 client.15708 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:12.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f334083a740 10 client.15708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f334083a740 20 client.15708 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f334083a740 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555590aadf80 0x555590ab03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f334083a740 4 client.15708 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.361+0000 7f334083a740 1 -- 172.21.15.5:0/3370205530 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555590a06860 con 0x555590aadf80 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.362+0000 7f3337fff640 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555590aadf80 0x555590ab03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.362+0000 7f3337fff640 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555590aadf80 0x555590ab03d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eeef456d01b49fbb server_cookie=ae05ff2a5f51417b in_seq=0 out_seq=0 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.362+0000 7f33177fe640 10 client.15708 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f33177fe640 1 -- 172.21.15.5:0/3370205530 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3328050e80 con 0x555590aadf80 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f33177fe640 10 client.15708 handle_command_reply: tid=0 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f334083a740 1 client.15708 shutdown 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f334083a740 1 -- 172.21.15.5:0/3370205530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a9e370 msgr2=0x555590a9b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f334083a740 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a9e370 0x555590a9b570 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f333001c8c0 tx=0x7f3330004a40 comp rx=0 tx=0).stop 2024-09-11T22:23:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f334083a740 1 -- 172.21.15.5:0/3370205530 shutdown_connections 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f334083a740 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555590aadf80 0x555590ab03d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f334083a740 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a9e370 0x555590a9b570 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f334083a740 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a94b40 0x555590a9afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f334083a740 1 --2- 172.21.15.5:0/3370205530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a935d0 0x555590a9dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f334083a740 1 -- 172.21.15.5:0/3370205530 >> 172.21.15.5:0/3370205530 conn(0x555590a5c310 msgr2=0x555590a95960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.363+0000 7f33177fe640 0 client.15708 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.364+0000 7f334083a740 1 -- 172.21.15.5:0/3370205530 shutdown_connections 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.364+0000 7f334083a740 1 -- 172.21.15.5:0/3370205530 wait complete. 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.364+0000 7f334083a740 20 client.15708 trim_cache size 0 max 16384 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f334083a740 1 Processor -- start 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f334083a740 1 -- start start 2024-09-11T22:23:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f334083a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a935d0 0x555590a8fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f334083a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aaa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f334083a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590a96120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f334083a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33381abbe0 con 0x555590a94b40 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f334083a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33381ad720 con 0x555590a9e370 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f334083a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33381ac2b0 con 0x555590a935d0 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f333c9fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590a96120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f333c9fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590a96120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38254/0 (socket says 172.21.15.5:38254) 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.377+0000 7f333c9fe640 1 -- 172.21.15.5:0/198248730 learned_addr learned my addr 172.21.15.5:0/198248730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.378+0000 7f33377fe640 1 --2- 172.21.15.5:0/198248730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aaa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.378+0000 7f333c9fe640 1 -- 172.21.15.5:0/198248730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a935d0 msgr2=0x555590a8fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.378+0000 7f333c9fe640 1 --2- 172.21.15.5:0/198248730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a935d0 0x555590a8fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.378+0000 7f333c9fe640 1 -- 172.21.15.5:0/198248730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 msgr2=0x555590aaa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.378+0000 7f333c9fe640 1 --2- 172.21.15.5:0/198248730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aaa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.378+0000 7f333c9fe640 1 -- 172.21.15.5:0/198248730 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555590a90730 con 0x555590a9e370 2024-09-11T22:23:12.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.378+0000 7f33377fe640 1 --2- 172.21.15.5:0/198248730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aaa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:12.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.378+0000 7f333c9fe640 1 --2- 172.21.15.5:0/198248730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590a96120 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f333001c5b0 tx=0x7f3330008f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9006f87f33efc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:12.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.379+0000 7f331dffb640 1 -- 172.21.15.5:0/198248730 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330004cb0 con 0x555590a9e370 2024-09-11T22:23:12.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.379+0000 7f331dffb640 1 -- 172.21.15.5:0/198248730 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3330004e50 con 0x555590a9e370 2024-09-11T22:23:12.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.379+0000 7f331dffb640 1 -- 172.21.15.5:0/198248730 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330030a40 con 0x555590a9e370 2024-09-11T22:23:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.380+0000 7f334083a740 1 -- 172.21.15.5:0/198248730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 msgr2=0x555590a96120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.380+0000 7f334083a740 1 --2- 172.21.15.5:0/198248730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590a96120 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f333001c5b0 tx=0x7f3330008f70 comp rx=0 tx=0).stop 2024-09-11T22:23:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.380+0000 7f334083a740 1 -- 172.21.15.5:0/198248730 shutdown_connections 2024-09-11T22:23:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.380+0000 7f334083a740 1 --2- 172.21.15.5:0/198248730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590a96120 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.380+0000 7f334083a740 1 --2- 172.21.15.5:0/198248730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aaa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.380+0000 7f334083a740 1 --2- 172.21.15.5:0/198248730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a935d0 0x555590a8fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.380+0000 7f334083a740 1 -- 172.21.15.5:0/198248730 >> 172.21.15.5:0/198248730 conn(0x555590a5c310 msgr2=0x555590a8f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.380+0000 7f334083a740 1 -- 172.21.15.5:0/198248730 shutdown_connections 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.380+0000 7f334083a740 1 -- 172.21.15.5:0/198248730 wait complete. 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.380+0000 7f334083a740 1 -- start start 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f334083a740 1 Processor -- start 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f334083a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a935d0 0x555590aa6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f334083a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aa1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f334083a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590aa20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f334083a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33381ad050 con 0x555590a94b40 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f334083a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33381ac980 con 0x555590a9e370 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f334083a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f33381a9bc0 con 0x555590a935d0 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f334083a740 10 client.0 _ll_register_callbacks cb 0x7f333000f710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f33377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aa1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f33377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aa1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59456/0 (socket says 172.21.15.5:59456) 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f33377fe640 1 -- 172.21.15.5:0/4188351033 learned_addr learned my addr 172.21.15.5:0/4188351033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.381+0000 7f3337fff640 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a935d0 0x555590aa6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f333c9fe640 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590aa20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f3337fff640 1 -- 172.21.15.5:0/4188351033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 msgr2=0x555590aa20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f3337fff640 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590aa20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f3337fff640 1 -- 172.21.15.5:0/4188351033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 msgr2=0x555590aa1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f3337fff640 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aa1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f3337fff640 1 -- 172.21.15.5:0/4188351033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555590aa2610 con 0x555590a935d0 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f333c9fe640 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590aa20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f33377fe640 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aa1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f3337fff640 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a935d0 0x555590aa6860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3328006010 tx=0x7f332802aa20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f378bb7728d7e146 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f33177fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.382+0000 7f33177fe640 1 -- 172.21.15.5:0/4188351033 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328035040 con 0x555590a935d0 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f334083a740 1 -- 172.21.15.5:0/4188351033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3330004e50 con 0x555590a935d0 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f33177fe640 1 -- 172.21.15.5:0/4188351033 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7f3328031d40 con 0x555590a935d0 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f33177fe640 1 -- 172.21.15.5:0/4188351033 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3328040040 con 0x555590a935d0 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f334083a740 10 client.25231 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f334083a740 1 -- 172.21.15.5:0/4188351033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555590b18960 con 0x555590a935d0 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f33177fe640 1 -- 172.21.15.5:0/4188351033 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3330004e50 con 0x555590a935d0 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f33177fe640 1 -- 172.21.15.5:0/4188351033 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3328045560 con 0x555590a935d0 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f334083a740 10 client.25231 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f334083a740 10 client.25231 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f334083a740 20 client.25231 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:12.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f334083a740 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555590b18c40 0x555590b1b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f334083a740 4 client.25231 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.383+0000 7f334083a740 1 -- 172.21.15.5:0/4188351033 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555590a00d70 con 0x555590b18c40 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.384+0000 7f33377fe640 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555590b18c40 0x555590b1b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.384+0000 7f33177fe640 10 client.25231 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.384+0000 7f33377fe640 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555590b18c40 0x555590b1b030 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab2f93d4546935b6 server_cookie=b7131a71b210c3ef in_seq=0 out_seq=0 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f33177fe640 1 -- 172.21.15.5:0/4188351033 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555590a00d70 con 0x555590b18c40 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f33177fe640 10 client.25231 handle_command_reply: tid=0 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 client.25231 shutdown 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 -- 172.21.15.5:0/4188351033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a935d0 msgr2=0x555590aa6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a935d0 0x555590aa6860 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3328006010 tx=0x7f332802aa20 comp rx=0 tx=0).stop 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 -- 172.21.15.5:0/4188351033 shutdown_connections 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555590b18c40 0x555590b1b030 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f33177fe640 0 client.25231 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:23:12.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555590a9e370 0x555590aa20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555590a94b40 0x555590aa1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 --2- 172.21.15.5:0/4188351033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555590a935d0 0x555590aa6860 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 -- 172.21.15.5:0/4188351033 >> 172.21.15.5:0/4188351033 conn(0x555590a5c310 msgr2=0x555590a8e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 -- 172.21.15.5:0/4188351033 shutdown_connections 2024-09-11T22:23:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 1 -- 172.21.15.5:0/4188351033 wait complete. 2024-09-11T22:23:12.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.385+0000 7f334083a740 20 client.25231 trim_cache size 0 max 16384 2024-09-11T22:23:12.400 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:23:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 -- 172.21.15.5:0/804671178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33200a2f20 msgr2=0x7f33200a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33200a2f20 0x7f33200a53e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3330005fd0 tx=0x7f3330011040 comp rx=0 tx=0).stop 2024-09-11T22:23:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 -- 172.21.15.5:0/804671178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3338102ef0 msgr2=0x7f3338198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3338102ef0 0x7f3338198c50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f332c00d770 tx=0x7f332c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:23:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 -- 172.21.15.5:0/804671178 shutdown_connections 2024-09-11T22:23:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33200a2f20 0x7f33200a53e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f333819cfe0 0x7f333819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3338102ef0 0x7f3338198c50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 --2- 172.21.15.5:0/804671178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3338102570 0x7f3338198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:12.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 -- 172.21.15.5:0/804671178 >> 172.21.15.5:0/804671178 conn(0x7f33380fc7d0 msgr2=0x7f333806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:12.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 -- 172.21.15.5:0/804671178 shutdown_connections 2024-09-11T22:23:12.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:12.386+0000 7f333e461640 1 -- 172.21.15.5:0/804671178 wait complete. 2024-09-11T22:23:13.151 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:23:13.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:13 smithi005 ceph-mon[30943]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-11T22:23:13.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:13 smithi172 ceph-mon[35461]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-11T22:23:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:13.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:13 smithi159 ceph-mon[35534]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-11T22:23:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.990+0000 7fa4f1040640 1 -- 172.21.15.5:0/505236818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102d30 msgr2=0x7fa4ec103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.990+0000 7fa4f1040640 1 --2- 172.21.15.5:0/505236818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102d30 0x7fa4ec103110 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009930 tx=0x7fa4dc02a420 comp rx=0 tx=0).stop 2024-09-11T22:23:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.991+0000 7fa4f1040640 1 -- 172.21.15.5:0/505236818 shutdown_connections 2024-09-11T22:23:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.991+0000 7fa4f1040640 1 --2- 172.21.15.5:0/505236818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec1082a0 0x7fa4ec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.991+0000 7fa4f1040640 1 --2- 172.21.15.5:0/505236818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec1036e0 0x7fa4ec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.991+0000 7fa4f1040640 1 --2- 172.21.15.5:0/505236818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102d30 0x7fa4ec103110 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.991+0000 7fa4f1040640 1 -- 172.21.15.5:0/505236818 >> 172.21.15.5:0/505236818 conn(0x7fa4ec0fc7d0 msgr2=0x7fa4ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.991+0000 7fa4f1040640 1 -- 172.21.15.5:0/505236818 shutdown_connections 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.991+0000 7fa4f1040640 1 -- 172.21.15.5:0/505236818 wait complete. 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4f1040640 1 Processor -- start 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4f1040640 1 -- start start 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4f1040640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec102d30 0x7fa4ec1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4f1040640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec1036e0 0x7fa4ec198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4f1040640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1082a0 0x7fa4ec19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4f1040640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1120a0 con 0x7fa4ec1082a0 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4f1040640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec111f20 con 0x7fa4ec102d30 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4f1040640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec112220 con 0x7fa4ec1036e0 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4eb59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1082a0 0x7fa4ec19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:13.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4eb59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1082a0 0x7fa4ec19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59478/0 (socket says 172.21.15.5:59478) 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4eb59e640 1 -- 172.21.15.5:0/2444851553 learned_addr learned my addr 172.21.15.5:0/2444851553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec102d30 0x7fa4ec1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.992+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec1036e0 0x7fa4ec198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4eb59e640 1 -- 172.21.15.5:0/2444851553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec1036e0 msgr2=0x7fa4ec198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4eb59e640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec1036e0 0x7fa4ec198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4eb59e640 1 -- 172.21.15.5:0/2444851553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec102d30 msgr2=0x7fa4ec1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4eb59e640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec102d30 0x7fa4ec1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4eb59e640 1 -- 172.21.15.5:0/2444851553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec19d590 con 0x7fa4ec1082a0 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec102d30 0x7fa4ec1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4eb59e640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1082a0 0x7fa4ec19ceb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009fd0 tx=0x7fa4e400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4d3fff640 1 -- 172.21.15.5:0/2444851553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4017070 con 0x7fa4ec1082a0 2024-09-11T22:23:13.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4f1040640 1 -- 172.21.15.5:0/2444851553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec19d880 con 0x7fa4ec1082a0 2024-09-11T22:23:13.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4d3fff640 1 -- 172.21.15.5:0/2444851553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e4003e50 con 0x7fa4ec1082a0 2024-09-11T22:23:13.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4f1040640 1 -- 172.21.15.5:0/2444851553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec1a5160 con 0x7fa4ec1082a0 2024-09-11T22:23:13.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.993+0000 7fa4d3fff640 1 -- 172.21.15.5:0/2444851553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4020060 con 0x7fa4ec1082a0 2024-09-11T22:23:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.997+0000 7fa4d3fff640 1 -- 172.21.15.5:0/2444851553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7fa4e4003970 con 0x7fa4ec1082a0 2024-09-11T22:23:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.997+0000 7fa4f3419740 1 Processor -- start 2024-09-11T22:23:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.997+0000 7fa4f3419740 1 -- start start 2024-09-11T22:23:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.997+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d17090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.997+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d17a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d17a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d17a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59494/0 (socket says 172.21.15.5:59494) 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d24760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d17090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4f3419740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3c1c7d2f0 con 0x55f3c1d175d0 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4f3419740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f3c1c24cf0 con 0x55f3c1d22370 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4f3419740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f3c1c792b0 con 0x55f3c1d18b40 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d17090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:51850/0 (socket says 172.21.15.5:51850) 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3985127598 learned_addr learned my addr 172.21.15.5:0/3985127598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3985127598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 msgr2=0x55f3c1d24760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/3985127598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d24760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.998+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3985127598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 msgr2=0x55f3c1d17a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.999+0000 7fa4d3fff640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d40a2ea0 0x7fa4d40a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.999+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/3985127598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d17a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.999+0000 7fa4ea59c640 1 -- 172.21.15.5:0/3985127598 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c1c093a0 con 0x55f3c1d18b40 2024-09-11T22:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.999+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/3985127598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d17a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.999+0000 7fa4d3fff640 1 -- 172.21.15.5:0/2444851553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa4e40cf3e0 con 0x7fa4ec1082a0 2024-09-11T22:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.999+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d40a2ea0 0x7fa4d40a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.999+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/3985127598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d17090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0005d00 tx=0x7fa4e000bb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=651e7a865383acb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.999+0000 7fa4d17fa640 1 -- 172.21.15.5:0/3985127598 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0022070 con 0x55f3c1d18b40 2024-09-11T22:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.999+0000 7fa4d17fa640 1 -- 172.21.15.5:0/3985127598 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e0015070 con 0x55f3c1d18b40 2024-09-11T22:23:14.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:13.999+0000 7fa4d17fa640 1 -- 172.21.15.5:0/3985127598 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e001c690 con 0x55f3c1d18b40 2024-09-11T22:23:14.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.000+0000 7fa4f3419740 1 -- 172.21.15.5:0/3985127598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 msgr2=0x55f3c1d17090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:14.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.000+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d40a2ea0 0x7fa4d40a5360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000c010 tx=0x7fa4e0020000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:14.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.000+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3985127598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d17090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0005d00 tx=0x7fa4e000bb80 comp rx=0 tx=0).stop 2024-09-11T22:23:14.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.000+0000 7fa4f3419740 1 -- 172.21.15.5:0/3985127598 shutdown_connections 2024-09-11T22:23:14.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.000+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3985127598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d24760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.000+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3985127598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d17a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.000+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3985127598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d17090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.000+0000 7fa4f3419740 1 -- 172.21.15.5:0/3985127598 >> 172.21.15.5:0/3985127598 conn(0x55f3c1ce0310 msgr2=0x55f3c1d13c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.000+0000 7fa4f3419740 1 -- 172.21.15.5:0/3985127598 shutdown_connections 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.001+0000 7fa4f3419740 1 -- 172.21.15.5:0/3985127598 wait complete. 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.001+0000 7fa4f3419740 1 -- start start 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.001+0000 7fa4f3419740 1 Processor -- start 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.001+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d175d0 0x55f3c1d21d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.001+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d1efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.001+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d22370 0x55f3c1d1f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.001+0000 7fa4f3419740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3c1d25070 con 0x55f3c1d22370 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.001+0000 7fa4f3419740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f3c1c38060 con 0x55f3c1d175d0 2024-09-11T22:23:14.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.001+0000 7fa4f3419740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f3c1d251f0 con 0x55f3c1d18b40 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4f3419740 10 client.0 _ll_register_callbacks cb 0x55f3c198b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4eb59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d22370 0x55f3c1d1f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d175d0 0x55f3c1d21d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d175d0 0x55f3c1d21d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38288/0 (socket says 172.21.15.5:38288) 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4eb59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d22370 0x55f3c1d1f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59496/0 (socket says 172.21.15.5:59496) 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4ead9d640 1 -- 172.21.15.5:0/3500970176 learned_addr learned my addr 172.21.15.5:0/3500970176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d1efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4eb59e640 1 -- 172.21.15.5:0/3500970176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 msgr2=0x55f3c1d1efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4eb59e640 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d1efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4eb59e640 1 -- 172.21.15.5:0/3500970176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d175d0 msgr2=0x55f3c1d21d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4eb59e640 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d175d0 0x55f3c1d21d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4eb59e640 1 -- 172.21.15.5:0/3500970176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c19ee3c0 con 0x55f3c1d22370 2024-09-11T22:23:14.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4eb59e640 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d22370 0x55f3c1d1f570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40cc8f0 tx=0x7fa4e409b2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9540bec1483f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.002+0000 7fa4ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4ceff5640 1 -- 172.21.15.5:0/3500970176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e409fa70 con 0x55f3c1d22370 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4ceff5640 1 -- 172.21.15.5:0/3500970176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa4e4097de0 con 0x55f3c1d22370 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4f3419740 1 -- 172.21.15.5:0/3500970176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4e0015070 con 0x55f3c1d22370 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4ceff5640 1 -- 172.21.15.5:0/3500970176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4e40a4330 con 0x55f3c1d22370 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4f3419740 10 client.15726 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4f3419740 1 -- 172.21.15.5:0/3500970176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f3c1d30ac0 con 0x55f3c1d22370 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4ceff5640 1 -- 172.21.15.5:0/3500970176 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e0015070 con 0x55f3c1d22370 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4ceff5640 1 -- 172.21.15.5:0/3500970176 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4e4004ae0 con 0x55f3c1d22370 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4f3419740 10 client.15726 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4f3419740 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4f3419740 20 client.15726 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:14.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.003+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3c1d32360 0x55f3c1d347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.004+0000 7fa4f3419740 4 client.15726 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.004+0000 7fa4f3419740 1 -- 172.21.15.5:0/3500970176 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3c1c8a860 con 0x55f3c1d32360 2024-09-11T22:23:14.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.004+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3c1d32360 0x55f3c1d347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.004+0000 7fa4ceff5640 10 client.15726 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.004+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3c1d32360 0x55f3c1d347b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c01c58118945bf1b server_cookie=23521ae3397e9625 in_seq=0 out_seq=0 2024-09-11T22:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4ceff5640 1 -- 172.21.15.5:0/3500970176 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3c1c8a860 con 0x55f3c1d32360 2024-09-11T22:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4ceff5640 10 client.15726 handle_command_reply: tid=0 2024-09-11T22:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4f3419740 1 client.15726 shutdown 2024-09-11T22:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4f3419740 1 -- 172.21.15.5:0/3500970176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d22370 msgr2=0x55f3c1d1f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d22370 0x55f3c1d1f570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40cc8f0 tx=0x7fa4e409b2b0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4f3419740 1 -- 172.21.15.5:0/3500970176 shutdown_connections 2024-09-11T22:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3c1d32360 0x55f3c1d347b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d22370 0x55f3c1d1f570 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d1efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3500970176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d175d0 0x55f3c1d21d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4f3419740 1 -- 172.21.15.5:0/3500970176 >> 172.21.15.5:0/3500970176 conn(0x55f3c1ce0310 msgr2=0x55f3c1d19960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.005+0000 7fa4ceff5640 0 client.15726 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.006+0000 7fa4f3419740 1 -- 172.21.15.5:0/3500970176 shutdown_connections 2024-09-11T22:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.006+0000 7fa4f3419740 1 -- 172.21.15.5:0/3500970176 wait complete. 2024-09-11T22:23:14.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.006+0000 7fa4f3419740 20 client.15726 trim_cache size 0 max 16384 2024-09-11T22:23:14.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.023+0000 7fa4f3419740 1 Processor -- start 2024-09-11T22:23:14.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4f3419740 1 -- start start 2024-09-11T22:23:14.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d2e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d34370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d1a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4f3419740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1ab670 con 0x55f3c1d175d0 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4f3419740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1ad1b0 con 0x55f3c1d22370 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4f3419740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1abd40 con 0x55f3c1d18b40 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d2e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d2e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59502/0 (socket says 172.21.15.5:59502) 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4ead9d640 1 -- 172.21.15.5:0/3264225913 learned_addr learned my addr 172.21.15.5:0/3264225913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4eb59e640 1 --2- 172.21.15.5:0/3264225913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d1a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4ead9d640 1 -- 172.21.15.5:0/3264225913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 msgr2=0x55f3c1d34370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/3264225913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d34370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4ead9d640 1 -- 172.21.15.5:0/3264225913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 msgr2=0x55f3c1d1a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/3264225913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d1a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4ead9d640 1 -- 172.21.15.5:0/3264225913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c1d22090 con 0x55f3c1d175d0 2024-09-11T22:23:14.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.024+0000 7fa4eb59e640 1 --2- 172.21.15.5:0/3264225913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d1a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.025+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/3264225913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d2e310 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005940 tx=0x7fa4dc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bf77868b841518 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.025+0000 7fa4d0ff9640 1 -- 172.21.15.5:0/3264225913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc035040 con 0x55f3c1d175d0 2024-09-11T22:23:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.025+0000 7fa4d0ff9640 1 -- 172.21.15.5:0/3264225913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4dc005500 con 0x55f3c1d175d0 2024-09-11T22:23:14.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.025+0000 7fa4d0ff9640 1 -- 172.21.15.5:0/3264225913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc0035f0 con 0x55f3c1d175d0 2024-09-11T22:23:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.026+0000 7fa4f3419740 1 -- 172.21.15.5:0/3264225913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 msgr2=0x55f3c1d2e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.026+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3264225913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d2e310 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc005940 tx=0x7fa4dc0058d0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.026+0000 7fa4f3419740 1 -- 172.21.15.5:0/3264225913 shutdown_connections 2024-09-11T22:23:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.026+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3264225913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d1a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.026+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3264225913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d18b40 0x55f3c1d34370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.026+0000 7fa4f3419740 1 --2- 172.21.15.5:0/3264225913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d175d0 0x55f3c1d2e310 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.026+0000 7fa4f3419740 1 -- 172.21.15.5:0/3264225913 >> 172.21.15.5:0/3264225913 conn(0x55f3c1ce0310 msgr2=0x55f3c1d21050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.026+0000 7fa4f3419740 1 -- 172.21.15.5:0/3264225913 shutdown_connections 2024-09-11T22:23:14.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.026+0000 7fa4f3419740 1 -- 172.21.15.5:0/3264225913 wait complete. 2024-09-11T22:23:14.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.027+0000 7fa4f3419740 1 -- start start 2024-09-11T22:23:14.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.027+0000 7fa4f3419740 1 Processor -- start 2024-09-11T22:23:14.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.028+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d175d0 0x55f3c1d2a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.028+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d18b40 0x55f3c1d2ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.028+0000 7fa4f3419740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d25dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.028+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d18b40 0x55f3c1d2ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.028+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d18b40 0x55f3c1d2ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59518/0 (socket says 172.21.15.5:59518) 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.028+0000 7fa4ea59c640 1 -- 172.21.15.5:0/1776305880 learned_addr learned my addr 172.21.15.5:0/1776305880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.028+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d175d0 0x55f3c1d2a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.028+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1acae0 con 0x55f3c1d18b40 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.028+0000 7fa4eb59e640 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d25dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.028+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1ac410 con 0x55f3c1d22370 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.029+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1a9650 con 0x55f3c1d175d0 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.029+0000 7fa4ea59c640 1 -- 172.21.15.5:0/1776305880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d175d0 msgr2=0x55f3c1d2a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.029+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d175d0 0x55f3c1d2a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.029+0000 7fa4ea59c640 1 -- 172.21.15.5:0/1776305880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 msgr2=0x55f3c1d25dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:14.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.029+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d25dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.029+0000 7fa4ea59c640 1 -- 172.21.15.5:0/1776305880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c1d1df80 con 0x55f3c1d18b40 2024-09-11T22:23:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.029+0000 7fa4f3419740 10 client.0 _ll_register_callbacks cb 0x7fa4e400bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.029+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d18b40 0x55f3c1d2ad40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0006010 tx=0x7fa4e0010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebb38baf028e7e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.029+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4dc005500 con 0x55f3c1d18b40 2024-09-11T22:23:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.029+0000 7fa4ceff5640 10 client.15732 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4ceff5640 1 -- 172.21.15.5:0/1776305880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e0022040 con 0x55f3c1d18b40 2024-09-11T22:23:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4ceff5640 1 -- 172.21.15.5:0/1776305880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8527+0+0 (secure 0 0 0) 0x7fa4e0011440 con 0x55f3c1d18b40 2024-09-11T22:23:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4ceff5640 1 -- 172.21.15.5:0/1776305880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa4e0011640 con 0x55f3c1d18b40 2024-09-11T22:23:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4f3419740 10 client.15732 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:14.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55f3c1d9c990 con 0x55f3c1d18b40 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4ceff5640 1 -- 172.21.15.5:0/1776305880 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa4e00117e0 con 0x55f3c1d18b40 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4f3419740 10 client.15732 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4f3419740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4f3419740 20 client.15732 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4f3419740 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3c1d9cec0 0x55f3c1d9f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4f3419740 4 client.15732 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.030+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3c1d34b40 con 0x55f3c1d9cec0 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.031+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3c1d9cec0 0x55f3c1d9f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.031+0000 7fa4ceff5640 10 client.15732 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.031+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3c1d9cec0 0x55f3c1d9f2e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3d6cf756857dec server_cookie=ae563e7e86eb8c15 in_seq=0 out_seq=0 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.031+0000 7fa4ceff5640 1 -- 172.21.15.5:0/1776305880 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3c1d34b40 con 0x55f3c1d9cec0 2024-09-11T22:23:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.031+0000 7fa4ceff5640 10 client.15732 handle_command_reply: tid=0 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.031+0000 7fa4f3419740 1 client.15732 shutdown 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d18b40 msgr2=0x55f3c1d2ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d18b40 0x55f3c1d2ad40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0006010 tx=0x7fa4e0010f50 comp rx=0 tx=0).stop 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 shutdown_connections 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3c1d9cec0 0x55f3c1d9f2e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4ceff5640 0 client.15732 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3c1d22370 0x55f3c1d25dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3c1d18b40 0x55f3c1d2ad40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 1 --2- 172.21.15.5:0/1776305880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3c1d175d0 0x55f3c1d2a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 >> 172.21.15.5:0/1776305880 conn(0x55f3c1ce0310 msgr2=0x55f3c1d12500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 shutdown_connections 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 1 -- 172.21.15.5:0/1776305880 wait complete. 2024-09-11T22:23:14.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.032+0000 7fa4f3419740 20 client.15732 trim_cache size 0 max 16384 2024-09-11T22:23:14.036 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:23:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 -- 172.21.15.5:0/2444851553 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d40a2ea0 msgr2=0x7fa4d40a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d40a2ea0 0x7fa4d40a5360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000c010 tx=0x7fa4e0020000 comp rx=0 tx=0).stop 2024-09-11T22:23:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 -- 172.21.15.5:0/2444851553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1082a0 msgr2=0x7fa4ec19ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:14.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1082a0 0x7fa4ec19ceb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009fd0 tx=0x7fa4e400b260 comp rx=0 tx=0).stop 2024-09-11T22:23:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 -- 172.21.15.5:0/2444851553 shutdown_connections 2024-09-11T22:23:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec1082a0 0x7fa4ec19ceb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec1036e0 0x7fa4ec198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d40a2ea0 0x7fa4d40a5360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 --2- 172.21.15.5:0/2444851553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec102d30 0x7fa4ec1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 -- 172.21.15.5:0/2444851553 >> 172.21.15.5:0/2444851553 conn(0x7fa4ec0fc7d0 msgr2=0x7fa4ec105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.033+0000 7fa4f1040640 1 -- 172.21.15.5:0/2444851553 shutdown_connections 2024-09-11T22:23:14.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:14.034+0000 7fa4f1040640 1 -- 172.21.15.5:0/2444851553 wait complete. 2024-09-11T22:23:15.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:15 smithi172 ceph-mon[35461]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-11T22:23:15.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:15 smithi005 ceph-mon[30943]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-11T22:23:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:15 smithi159 ceph-mon[35534]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-11T22:23:15.833 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:23:15.833 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:23:16.212 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:16.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-11T22:23:16.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:16 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-11T22:23:16.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-11T22:23:16.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:16 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-11T22:23:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-11T22:23:16.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:16 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-09-11T22:23:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.831+0000 7fe60b9b5640 1 -- 172.21.15.5:0/2350970620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe604102570 msgr2=0x7fe6041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.831+0000 7fe60b9b5640 1 --2- 172.21.15.5:0/2350970620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe604102570 0x7fe6041029b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009900 tx=0x7fe5f402a480 comp rx=0 tx=0).stop 2024-09-11T22:23:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.831+0000 7fe60b9b5640 1 -- 172.21.15.5:0/2350970620 shutdown_connections 2024-09-11T22:23:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.831+0000 7fe60b9b5640 1 --2- 172.21.15.5:0/2350970620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe604102ef0 0x7fe60410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.831+0000 7fe60b9b5640 1 --2- 172.21.15.5:0/2350970620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe604102570 0x7fe6041029b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.831+0000 7fe60b9b5640 1 --2- 172.21.15.5:0/2350970620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe604100e20 0x7fe604101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.831+0000 7fe60b9b5640 1 -- 172.21.15.5:0/2350970620 >> 172.21.15.5:0/2350970620 conn(0x7fe6040fc7d0 msgr2=0x7fe6040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:16.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.832+0000 7fe60b9b5640 1 -- 172.21.15.5:0/2350970620 shutdown_connections 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.832+0000 7fe60b9b5640 1 -- 172.21.15.5:0/2350970620 wait complete. 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.832+0000 7fe60b9b5640 1 Processor -- start 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.832+0000 7fe60b9b5640 1 -- start start 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe60b9b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe604100e20 0x7fe6041a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe60b9b5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe604102570 0x7fe6041a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe60b9b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe604102ef0 0x7fe60410c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe60b9b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041120d0 con 0x7fe604100e20 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe60b9b5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe604111f50 con 0x7fe604102570 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe60b9b5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe604112250 con 0x7fe604102ef0 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe609751640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe604100e20 0x7fe6041a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe609751640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe604100e20 0x7fe6041a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59546/0 (socket says 172.21.15.5:59546) 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe609751640 1 -- 172.21.15.5:0/992303227 learned_addr learned my addr 172.21.15.5:0/992303227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:16.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe608f50640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe604102570 0x7fe6041a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.833+0000 7fe609f52640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe604102ef0 0x7fe60410c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.834+0000 7fe609751640 1 -- 172.21.15.5:0/992303227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe604102ef0 msgr2=0x7fe60410c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:16.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.834+0000 7fe609751640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe604102ef0 0x7fe60410c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.834+0000 7fe609751640 1 -- 172.21.15.5:0/992303227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe604102570 msgr2=0x7fe6041a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.834+0000 7fe609751640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe604102570 0x7fe6041a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.834+0000 7fe609751640 1 -- 172.21.15.5:0/992303227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60410d1f0 con 0x7fe604100e20 2024-09-11T22:23:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.834+0000 7fe609751640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe604100e20 0x7fe6041a46a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe60000e9f0 tx=0x7fe60000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.834+0000 7fe5fa7fc640 1 -- 172.21.15.5:0/992303227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60000cd80 con 0x7fe604100e20 2024-09-11T22:23:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.834+0000 7fe5fa7fc640 1 -- 172.21.15.5:0/992303227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe600005500 con 0x7fe604100e20 2024-09-11T22:23:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.834+0000 7fe5fa7fc640 1 -- 172.21.15.5:0/992303227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600010690 con 0x7fe604100e20 2024-09-11T22:23:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.834+0000 7fe60b9b5640 1 -- 172.21.15.5:0/992303227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60410d4e0 con 0x7fe604100e20 2024-09-11T22:23:16.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.835+0000 7fe60b9b5640 1 -- 172.21.15.5:0/992303227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6041ab450 con 0x7fe604100e20 2024-09-11T22:23:16.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.838+0000 7fe5fa7fc640 1 -- 172.21.15.5:0/992303227 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7fe6000046c0 con 0x7fe604100e20 2024-09-11T22:23:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.838+0000 7fe60dd8e740 1 Processor -- start 2024-09-11T22:23:16.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.839+0000 7fe60dd8e740 1 -- start start 2024-09-11T22:23:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.839+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc652f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.839+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 0x5611fc65d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.839+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc660130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.839+0000 7fe60dd8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611fc5b8280 con 0x5611fc652b60 2024-09-11T22:23:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.839+0000 7fe609f52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc652f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.839+0000 7fe609f52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc652f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59562/0 (socket says 172.21.15.5:59562) 2024-09-11T22:23:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.839+0000 7fe609f52640 1 -- 172.21.15.5:0/3401327817 learned_addr learned my addr 172.21.15.5:0/3401327817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.839+0000 7fe609751640 1 --2- 172.21.15.5:0/3401327817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 0x5611fc65d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3401327817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5611fc55fb10 con 0x5611fc65dd40 2024-09-11T22:23:16.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3401327817 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5611fc5b4240 con 0x5611fc653480 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe5fa7fc640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5f00a2f20 0x7fe5f00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe608f50640 1 --2- 172.21.15.5:0/3401327817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc660130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe609f52640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5f00a2f20 0x7fe5f00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe609f52640 1 -- 172.21.15.5:0/3401327817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 msgr2=0x5611fc65d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe609f52640 1 --2- 172.21.15.5:0/3401327817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 0x5611fc65d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe609f52640 1 -- 172.21.15.5:0/3401327817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 msgr2=0x5611fc660130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe609f52640 1 --2- 172.21.15.5:0/3401327817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc660130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe609f52640 1 -- 172.21.15.5:0/3401327817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611fc56c1f0 con 0x5611fc652b60 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe5fa7fc640 1 -- 172.21.15.5:0/992303227 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fe6000cfb50 con 0x7fe604100e20 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe609751640 1 --2- 172.21.15.5:0/3401327817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 0x5611fc65d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.840+0000 7fe609f52640 1 --2- 172.21.15.5:0/3401327817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc652f40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc005f40 tx=0x7fe5fc014f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9de760d90badad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:16.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.841+0000 7fe5ef7fe640 1 -- 172.21.15.5:0/3401327817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc01ee50 con 0x5611fc652b60 2024-09-11T22:23:16.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.841+0000 7fe609f52640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5f00a2f20 0x7fe5f00a53e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc022000 tx=0x7fe5fc020000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:16.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.841+0000 7fe5ef7fe640 1 -- 172.21.15.5:0/3401327817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5fc023070 con 0x5611fc652b60 2024-09-11T22:23:16.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.841+0000 7fe5ef7fe640 1 -- 172.21.15.5:0/3401327817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc011b20 con 0x5611fc652b60 2024-09-11T22:23:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.842+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3401327817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 msgr2=0x5611fc652f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.842+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3401327817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc652f40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc005f40 tx=0x7fe5fc014f10 comp rx=0 tx=0).stop 2024-09-11T22:23:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.842+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3401327817 shutdown_connections 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.842+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3401327817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc660130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.842+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3401327817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 0x5611fc65d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.842+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3401327817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc652f40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.842+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3401327817 >> 172.21.15.5:0/3401327817 conn(0x5611fc64f0f0 msgr2=0x5611fc64f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.842+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3401327817 shutdown_connections 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.842+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3401327817 wait complete. 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.842+0000 7fe60dd8e740 1 -- start start 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe60dd8e740 1 Processor -- start 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc65abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 0x5611fc65b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc65b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe60dd8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611fc660960 con 0x5611fc652b60 2024-09-11T22:23:16.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe609f52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc65b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe609751640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc65abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe609f52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc65b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38338/0 (socket says 172.21.15.5:38338) 2024-09-11T22:23:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe609751640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc65abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59570/0 (socket says 172.21.15.5:59570) 2024-09-11T22:23:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe609f52640 1 -- 172.21.15.5:0/3744256872 learned_addr learned my addr 172.21.15.5:0/3744256872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3744256872 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5611fc572f10 con 0x5611fc65dd40 2024-09-11T22:23:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.843+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3744256872 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5611fc660ae0 con 0x5611fc653480 2024-09-11T22:23:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.844+0000 7fe608f50640 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 0x5611fc65b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.844+0000 7fe609751640 1 -- 172.21.15.5:0/3744256872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 msgr2=0x5611fc65b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.844+0000 7fe609751640 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 0x5611fc65b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.844+0000 7fe609751640 1 -- 172.21.15.5:0/3744256872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 msgr2=0x5611fc65b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.844+0000 7fe60dd8e740 10 client.0 _ll_register_callbacks cb 0x5611fc53ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.844+0000 7fe609751640 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc65b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.844+0000 7fe609751640 1 -- 172.21.15.5:0/3744256872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611fc3296a0 con 0x5611fc652b60 2024-09-11T22:23:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.844+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3744256872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5fc023070 con 0x5611fc652b60 2024-09-11T22:23:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.844+0000 7fe609751640 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc65abc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe6000cd020 tx=0x7fe600009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8903ebd51cd282a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.844+0000 7fe5ecff9640 10 client.15744 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.845+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3744256872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60009f060 con 0x5611fc652b60 2024-09-11T22:23:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.845+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3744256872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fe60009b440 con 0x5611fc652b60 2024-09-11T22:23:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.845+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3744256872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe60009b750 con 0x5611fc652b60 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.845+0000 7fe60dd8e740 10 client.15744 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.845+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3744256872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5611fc66c380 con 0x5611fc652b60 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.845+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3744256872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe600097b40 con 0x5611fc652b60 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.845+0000 7fe60dd8e740 10 client.15744 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.845+0000 7fe60dd8e740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.845+0000 7fe60dd8e740 20 client.15744 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.845+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611fc66d780 0x5611fc66fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.846+0000 7fe60dd8e740 4 client.15744 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.846+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3744256872 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611fc5c57f0 con 0x5611fc66d780 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.846+0000 7fe608f50640 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611fc66d780 0x5611fc66fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.846+0000 7fe608f50640 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611fc66d780 0x5611fc66fbb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25774ce660d69636 server_cookie=dc1f0433802103aa in_seq=0 out_seq=0 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.846+0000 7fe5ecff9640 10 client.15744 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.847+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3744256872 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611fc5c57f0 con 0x5611fc66d780 2024-09-11T22:23:16.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.847+0000 7fe5ecff9640 10 client.15744 handle_command_reply: tid=0 2024-09-11T22:23:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.847+0000 7fe60dd8e740 1 client.15744 shutdown 2024-09-11T22:23:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.847+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3744256872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 msgr2=0x5611fc65abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.847+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc65abc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe6000cd020 tx=0x7fe600009310 comp rx=0 tx=0).stop 2024-09-11T22:23:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.848+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3744256872 shutdown_connections 2024-09-11T22:23:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.848+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611fc66d780 0x5611fc66fbb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.848+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc65b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.848+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc653480 0x5611fc65b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.848+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3744256872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc652b60 0x5611fc65abc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.848+0000 7fe5ecff9640 0 client.15744 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.848+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3744256872 >> 172.21.15.5:0/3744256872 conn(0x5611fc64f0f0 msgr2=0x5611fc64e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.848+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3744256872 shutdown_connections 2024-09-11T22:23:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.848+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3744256872 wait complete. 2024-09-11T22:23:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.848+0000 7fe60dd8e740 20 client.15744 trim_cache size 0 max 16384 2024-09-11T22:23:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.881+0000 7fe60dd8e740 1 Processor -- start 2024-09-11T22:23:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.881+0000 7fe60dd8e740 1 -- start start 2024-09-11T22:23:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc652b60 0x5611fc6555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc653480 0x5611fc65b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc675430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe60dd8e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041b4000 con 0x5611fc653480 2024-09-11T22:23:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe60dd8e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041b5b40 con 0x5611fc65dd40 2024-09-11T22:23:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe60dd8e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041b46d0 con 0x5611fc652b60 2024-09-11T22:23:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe608f50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc653480 0x5611fc65b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe608f50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc653480 0x5611fc65b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59584/0 (socket says 172.21.15.5:59584) 2024-09-11T22:23:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe608f50640 1 -- 172.21.15.5:0/1797161950 learned_addr learned my addr 172.21.15.5:0/1797161950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe608f50640 1 -- 172.21.15.5:0/1797161950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc652b60 msgr2=0x5611fc6555c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:23:16.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe608f50640 1 --2- 172.21.15.5:0/1797161950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc652b60 0x5611fc6555c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe608f50640 1 -- 172.21.15.5:0/1797161950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 msgr2=0x5611fc675430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:23:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe608f50640 1 --2- 172.21.15.5:0/1797161950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc675430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe608f50640 1 -- 172.21.15.5:0/1797161950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611fc6701e0 con 0x5611fc653480 2024-09-11T22:23:16.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.882+0000 7fe608f50640 1 --2- 172.21.15.5:0/1797161950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc653480 0x5611fc65b290 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009900 tx=0x7fe5f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea16a90fc3138f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.883+0000 7fe5eeffd640 1 -- 172.21.15.5:0/1797161950 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4035040 con 0x5611fc653480 2024-09-11T22:23:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.883+0000 7fe5eeffd640 1 -- 172.21.15.5:0/1797161950 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f4031810 con 0x5611fc653480 2024-09-11T22:23:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.883+0000 7fe5eeffd640 1 -- 172.21.15.5:0/1797161950 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f4004a90 con 0x5611fc653480 2024-09-11T22:23:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.884+0000 7fe60dd8e740 1 -- 172.21.15.5:0/1797161950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc653480 msgr2=0x5611fc65b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.884+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/1797161950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc653480 0x5611fc65b290 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009900 tx=0x7fe5f4000bc0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.884+0000 7fe60dd8e740 1 -- 172.21.15.5:0/1797161950 shutdown_connections 2024-09-11T22:23:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.884+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/1797161950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc65dd40 0x5611fc675430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.884+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/1797161950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc653480 0x5611fc65b290 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.884+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/1797161950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc652b60 0x5611fc6555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.884+0000 7fe60dd8e740 1 -- 172.21.15.5:0/1797161950 >> 172.21.15.5:0/1797161950 conn(0x5611fc64f0f0 msgr2=0x5611fc65ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.884+0000 7fe60dd8e740 1 -- 172.21.15.5:0/1797161950 shutdown_connections 2024-09-11T22:23:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.884+0000 7fe60dd8e740 1 -- 172.21.15.5:0/1797161950 wait complete. 2024-09-11T22:23:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.884+0000 7fe60dd8e740 1 -- start start 2024-09-11T22:23:16.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.885+0000 7fe60dd8e740 1 Processor -- start 2024-09-11T22:23:16.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.885+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc652b60 0x5611fc666500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.885+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc653480 0x5611fc662d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.885+0000 7fe60dd8e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc65dd40 0x5611fc6632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.885+0000 7fe609f52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc65dd40 0x5611fc6632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.885+0000 7fe609f52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc65dd40 0x5611fc6632a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59600/0 (socket says 172.21.15.5:59600) 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe609f52640 1 -- 172.21.15.5:0/3624801406 learned_addr learned my addr 172.21.15.5:0/3624801406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041b5470 con 0x5611fc65dd40 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041b4da0 con 0x5611fc653480 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe6041b1fe0 con 0x5611fc652b60 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe609751640 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc652b60 0x5611fc666500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe60dd8e740 10 client.0 _ll_register_callbacks cb 0x7fe60009bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe608f50640 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc653480 0x5611fc662d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe609f52640 1 -- 172.21.15.5:0/3624801406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc652b60 msgr2=0x5611fc666500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe609f52640 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc652b60 0x5611fc666500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe609f52640 1 -- 172.21.15.5:0/3624801406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc653480 msgr2=0x5611fc662d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe609f52640 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc653480 0x5611fc662d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe609f52640 1 -- 172.21.15.5:0/3624801406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611fc663ad0 con 0x5611fc65dd40 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe609751640 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc652b60 0x5611fc666500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe608f50640 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc653480 0x5611fc662d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe609f52640 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc65dd40 0x5611fc6632a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009ee0 tx=0x7fe5fc0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7580782bb4a3017e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.886+0000 7fe5ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.887+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3624801406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc014510 con 0x5611fc65dd40 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.887+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3624801406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8585+0+0 (secure 0 0 0) 0x7fe5fc003970 con 0x5611fc65dd40 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.887+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f4031810 con 0x5611fc65dd40 2024-09-11T22:23:16.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.887+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3624801406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5fc0146b0 con 0x5611fc65dd40 2024-09-11T22:23:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.887+0000 7fe60dd8e740 10 client.15750 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.887+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611fc6d9500 con 0x5611fc65dd40 2024-09-11T22:23:16.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.887+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3624801406 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f4031810 con 0x5611fc65dd40 2024-09-11T22:23:16.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.887+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3624801406 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe5fc00f040 con 0x5611fc65dd40 2024-09-11T22:23:16.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.888+0000 7fe60dd8e740 10 client.15750 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.888+0000 7fe60dd8e740 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.888+0000 7fe60dd8e740 20 client.15750 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.888+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611fc6d99a0 0x5611fc6dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.888+0000 7fe60dd8e740 4 client.15750 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.888+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5611fc5bfd00 con 0x5611fc6d99a0 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.888+0000 7fe609751640 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611fc6d99a0 0x5611fc6dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.888+0000 7fe609751640 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611fc6d99a0 0x5611fc6dbdc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f9d01825b42f2b server_cookie=c7d72aa0df05d15e in_seq=0 out_seq=0 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.888+0000 7fe5ecff9640 10 client.15750 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe5ecff9640 1 -- 172.21.15.5:0/3624801406 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5611fc5bfd00 con 0x5611fc6d99a0 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe5ecff9640 10 client.15750 handle_command_reply: tid=0 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe60dd8e740 1 client.15750 shutdown 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc65dd40 msgr2=0x5611fc6632a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc65dd40 0x5611fc6632a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009ee0 tx=0x7fe5fc0093c0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 shutdown_connections 2024-09-11T22:23:16.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611fc6d99a0 0x5611fc6dbdc0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611fc65dd40 0x5611fc6632a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611fc653480 0x5611fc662d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe60dd8e740 1 --2- 172.21.15.5:0/3624801406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611fc652b60 0x5611fc666500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 >> 172.21.15.5:0/3624801406 conn(0x5611fc64f0f0 msgr2=0x5611fc6693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:16.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe5ecff9640 0 client.15750 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:16.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.890+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 shutdown_connections 2024-09-11T22:23:16.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.891+0000 7fe60dd8e740 1 -- 172.21.15.5:0/3624801406 wait complete. 2024-09-11T22:23:16.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.891+0000 7fe60dd8e740 20 client.15750 trim_cache size 0 max 16384 2024-09-11T22:23:16.893 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:23:16.894 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:23:16.894 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e22a2bf3-58e0-43a8-b12c-d35f425ff668", 2024-09-11T22:23:16.894 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:23:16.894 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.891+0000 7fe60b9b5640 1 -- 172.21.15.5:0/992303227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5f00a2f20 msgr2=0x7fe5f00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.891+0000 7fe60b9b5640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5f00a2f20 0x7fe5f00a53e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc022000 tx=0x7fe5fc020000 comp rx=0 tx=0).stop 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.891+0000 7fe60b9b5640 1 -- 172.21.15.5:0/992303227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe604100e20 msgr2=0x7fe6041a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.891+0000 7fe60b9b5640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe604100e20 0x7fe6041a46a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe60000e9f0 tx=0x7fe60000edf0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.892+0000 7fe60b9b5640 1 -- 172.21.15.5:0/992303227 shutdown_connections 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.892+0000 7fe60b9b5640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe604102ef0 0x7fe60410c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.892+0000 7fe60b9b5640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5f00a2f20 0x7fe5f00a53e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.892+0000 7fe60b9b5640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe604102570 0x7fe6041a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.892+0000 7fe60b9b5640 1 --2- 172.21.15.5:0/992303227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe604100e20 0x7fe6041a46a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.892+0000 7fe60b9b5640 1 -- 172.21.15.5:0/992303227 >> 172.21.15.5:0/992303227 conn(0x7fe6040fc7d0 msgr2=0x7fe604105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.892+0000 7fe60b9b5640 1 -- 172.21.15.5:0/992303227 shutdown_connections 2024-09-11T22:23:16.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:16.892+0000 7fe60b9b5640 1 -- 172.21.15.5:0/992303227 wait complete. 2024-09-11T22:23:17.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:17 smithi172 ceph-mon[35461]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-11T22:23:17.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:17 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-11T22:23:17.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:17 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-11T22:23:17.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:17 smithi172 ceph-mon[35461]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T22:23:17.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:17 smithi005 ceph-mon[30943]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-11T22:23:17.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:17 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-11T22:23:17.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:17 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-11T22:23:17.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:17 smithi005 ceph-mon[30943]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T22:23:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:17 smithi159 ceph-mon[35534]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.6 KiB/s wr, 10 op/s 2024-09-11T22:23:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:17 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-11T22:23:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:17 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-09-11T22:23:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:17 smithi159 ceph-mon[35534]: osdmap e84: 12 total, 12 up, 12 in 2024-09-11T22:23:17.752 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:23:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:18.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:18 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:23:18.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:18 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:23:18.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:18 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:23:18.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:18 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:23:18.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:18 smithi005 ceph-mon[30943]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T22:23:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:18 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:23:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:18 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:23:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:18 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:23:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:18 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:23:18.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:18 smithi159 ceph-mon[35534]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T22:23:18.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.579+0000 7f3b33ac9640 1 -- 172.21.15.5:0/2761021237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b2c106f40 msgr2=0x7f3b2c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:18.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.579+0000 7f3b33ac9640 1 --2- 172.21.15.5:0/2761021237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b2c106f40 0x7f3b2c10eac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009ca0 tx=0x7f3b2402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.579+0000 7f3b33ac9640 1 -- 172.21.15.5:0/2761021237 shutdown_connections 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.579+0000 7f3b33ac9640 1 --2- 172.21.15.5:0/2761021237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b2c106f40 0x7f3b2c10eac0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.579+0000 7f3b33ac9640 1 --2- 172.21.15.5:0/2761021237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2c1027c0 0x7f3b2c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.579+0000 7f3b33ac9640 1 --2- 172.21.15.5:0/2761021237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b2c101e10 0x7f3b2c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.579+0000 7f3b33ac9640 1 -- 172.21.15.5:0/2761021237 >> 172.21.15.5:0/2761021237 conn(0x7f3b2c0fd8d0 msgr2=0x7f3b2c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.579+0000 7f3b33ac9640 1 -- 172.21.15.5:0/2761021237 shutdown_connections 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.579+0000 7f3b33ac9640 1 -- 172.21.15.5:0/2761021237 wait complete. 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.580+0000 7f3b33ac9640 1 Processor -- start 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.580+0000 7f3b33ac9640 1 -- start start 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.580+0000 7f3b33ac9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b2c101e10 0x7f3b2c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b33ac9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2c1027c0 0x7f3b2c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b33ac9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b2c106f40 0x7f3b2c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b33ac9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c111fa0 con 0x7f3b2c106f40 2024-09-11T22:23:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b33ac9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c111e20 con 0x7f3b2c101e10 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b33ac9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c112120 con 0x7f3b2c1027c0 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b31865640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b2c101e10 0x7f3b2c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b32066640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b2c106f40 0x7f3b2c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b31865640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b2c101e10 0x7f3b2c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38372/0 (socket says 172.21.15.5:38372) 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b31865640 1 -- 172.21.15.5:0/3896954778 learned_addr learned my addr 172.21.15.5:0/3896954778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b31064640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2c1027c0 0x7f3b2c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b31865640 1 -- 172.21.15.5:0/3896954778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2c1027c0 msgr2=0x7f3b2c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b31865640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2c1027c0 0x7f3b2c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b31865640 1 -- 172.21.15.5:0/3896954778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b2c106f40 msgr2=0x7f3b2c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b31865640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b2c106f40 0x7f3b2c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b31865640 1 -- 172.21.15.5:0/3896954778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c1aad20 con 0x7f3b2c101e10 2024-09-11T22:23:18.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.581+0000 7f3b32066640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b2c106f40 0x7f3b2c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.582+0000 7f3b31865640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b2c101e10 0x7f3b2c0744f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c920 tx=0x7f3b2800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.582+0000 7f3b22ffd640 1 -- 172.21.15.5:0/3896954778 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28007c70 con 0x7f3b2c101e10 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.582+0000 7f3b33ac9640 1 -- 172.21.15.5:0/3896954778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c1aaf50 con 0x7f3b2c101e10 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.582+0000 7f3b22ffd640 1 -- 172.21.15.5:0/3896954778 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b28007e10 con 0x7f3b2c101e10 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.582+0000 7f3b33ac9640 1 -- 172.21.15.5:0/3896954778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c1ab3b0 con 0x7f3b2c101e10 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.582+0000 7f3b22ffd640 1 -- 172.21.15.5:0/3896954778 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28018890 con 0x7f3b2c101e10 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.585+0000 7f3b35ea2740 1 Processor -- start 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.585+0000 7f3b35ea2740 1 -- start start 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.585+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe944a0 0x55ffebe94880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.585+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebe9f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.585+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebea1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.585+0000 7f3b35ea2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffebdfa2e0 con 0x55ffebe9f680 2024-09-11T22:23:18.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.585+0000 7f3b35ea2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ffebda1ce0 con 0x55ffebe944a0 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.585+0000 7f3b35ea2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffebdf62a0 con 0x55ffebe94dc0 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b31064640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebea1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b31064640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebea1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59620/0 (socket says 172.21.15.5:59620) 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b31064640 1 -- 172.21.15.5:0/173497302 learned_addr learned my addr 172.21.15.5:0/173497302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b32066640 1 --2- 172.21.15.5:0/173497302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe944a0 0x55ffebe94880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b22ffd640 1 -- 172.21.15.5:0/3896954778 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7f3b28018b10 con 0x7f3b2c101e10 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b31064640 1 -- 172.21.15.5:0/173497302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 msgr2=0x55ffebe9f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b31064640 1 --2- 172.21.15.5:0/173497302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebe9f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b31064640 1 -- 172.21.15.5:0/173497302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe944a0 msgr2=0x55ffebe94880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b31064640 1 --2- 172.21.15.5:0/173497302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe944a0 0x55ffebe94880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b31064640 1 -- 172.21.15.5:0/173497302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffebd86440 con 0x55ffebe9f680 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.586+0000 7f3b31064640 1 --2- 172.21.15.5:0/173497302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebea1a70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0045e0 tx=0x7f3b1c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4f4f3c8e7b8b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.587+0000 7f3b17fff640 1 -- 172.21.15.5:0/173497302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c005220 con 0x55ffebe9f680 2024-09-11T22:23:18.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.587+0000 7f3b17fff640 1 -- 172.21.15.5:0/173497302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1c015070 con 0x55ffebe9f680 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.587+0000 7f3b17fff640 1 -- 172.21.15.5:0/173497302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c004d00 con 0x55ffebe9f680 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.588+0000 7f3b35ea2740 1 -- 172.21.15.5:0/173497302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 msgr2=0x55ffebea1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.588+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/173497302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebea1a70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0045e0 tx=0x7f3b1c004610 comp rx=0 tx=0).stop 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.588+0000 7f3b35ea2740 1 -- 172.21.15.5:0/173497302 shutdown_connections 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.588+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/173497302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebea1a70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.588+0000 7f3b22ffd640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3b180a2f20 0x7f3b180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.588+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/173497302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebe9f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.588+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/173497302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe944a0 0x55ffebe94880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.588+0000 7f3b35ea2740 1 -- 172.21.15.5:0/173497302 >> 172.21.15.5:0/173497302 conn(0x55ffebe90a30 msgr2=0x55ffebe90e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.588+0000 7f3b22ffd640 1 -- 172.21.15.5:0/3896954778 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f3b280cf7f0 con 0x7f3b2c101e10 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.588+0000 7f3b32066640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3b180a2f20 0x7f3b180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.589+0000 7f3b35ea2740 1 -- 172.21.15.5:0/173497302 shutdown_connections 2024-09-11T22:23:18.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.589+0000 7f3b32066640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3b180a2f20 0x7f3b180a53e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3b24031f80 tx=0x7f3b24005ef0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.589+0000 7f3b35ea2740 1 -- 172.21.15.5:0/173497302 wait complete. 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.589+0000 7f3b35ea2740 1 -- start start 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.589+0000 7f3b35ea2740 1 Processor -- start 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.590+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebe9efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.590+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebe9c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b31865640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebe9efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b31865640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebe9efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59622/0 (socket says 172.21.15.5:59622) 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b31865640 1 -- 172.21.15.5:0/4246071420 learned_addr learned my addr 172.21.15.5:0/4246071420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe9f680 0x55ffebe9c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffebea22a0 con 0x55ffebe944a0 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ffebdb5050 con 0x55ffebe9f680 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffebea2420 con 0x55ffebe94dc0 2024-09-11T22:23:18.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b35ea2740 10 client.0 _ll_register_callbacks cb 0x55ffebdf5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b31064640 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebe9c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b32066640 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe9f680 0x55ffebe9c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b31865640 1 -- 172.21.15.5:0/4246071420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 msgr2=0x55ffebe9c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b31865640 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebe9c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b31865640 1 -- 172.21.15.5:0/4246071420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe9f680 msgr2=0x55ffebe9c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b31865640 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe9f680 0x55ffebe9c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b31865640 1 -- 172.21.15.5:0/4246071420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffebb6b3c0 con 0x55ffebe944a0 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b31865640 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebe9efd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c8f0 tx=0x7f3b2809c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4cda0cedce1893 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.591+0000 7f3b157fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b157fa640 1 -- 172.21.15.5:0/4246071420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b280a4700 con 0x55ffebe944a0 2024-09-11T22:23:18.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b157fa640 1 -- 172.21.15.5:0/4246071420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f3b280a4270 con 0x55ffebe944a0 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b1c015070 con 0x55ffebe944a0 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b157fa640 1 -- 172.21.15.5:0/4246071420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3b280a44f0 con 0x55ffebe944a0 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b35ea2740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ffebeadcc0 con 0x55ffebe944a0 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b157fa640 1 -- 172.21.15.5:0/4246071420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1c015070 con 0x55ffebe944a0 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b157fa640 1 -- 172.21.15.5:0/4246071420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3b280977e0 con 0x55ffebe944a0 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b35ea2740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b35ea2740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b35ea2740 20 client.15762 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.592+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffebeaf180 0x55ffebeb1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.593+0000 7f3b35ea2740 4 client.15762 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.593+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffebe07850 con 0x55ffebeaf180 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.593+0000 7f3b31064640 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffebeaf180 0x55ffebeb1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.593+0000 7f3b157fa640 10 client.15762 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.593+0000 7f3b31064640 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffebeaf180 0x55ffebeb1570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58f72be7b2ebcaeb server_cookie=94e5640dec847a52 in_seq=0 out_seq=0 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.594+0000 7f3b157fa640 1 -- 172.21.15.5:0/4246071420 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffebe07850 con 0x55ffebeaf180 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.594+0000 7f3b157fa640 10 client.15762 handle_command_reply: tid=0 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.594+0000 7f3b35ea2740 1 client.15762 shutdown 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.594+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 msgr2=0x55ffebe9efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.594+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebe9efd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c8f0 tx=0x7f3b2809c250 comp rx=0 tx=0).stop 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.594+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 shutdown_connections 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.594+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffebeaf180 0x55ffebeb1570 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.595+0000 7f3b157fa640 0 client.15762 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:18.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.595+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe9f680 0x55ffebe9c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.595+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebe9c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.595+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/4246071420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebe9efd0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.595+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 >> 172.21.15.5:0/4246071420 conn(0x55ffebe90a30 msgr2=0x55ffebe96710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.595+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 shutdown_connections 2024-09-11T22:23:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.595+0000 7f3b35ea2740 1 -- 172.21.15.5:0/4246071420 wait complete. 2024-09-11T22:23:18.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.595+0000 7f3b35ea2740 20 client.15762 trim_cache size 0 max 16384 2024-09-11T22:23:18.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:18 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:23:18.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:18 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:23:18.611 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:18 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:23:18.612 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:18 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:23:18.612 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:18 smithi172 ceph-mon[35461]: osdmap e85: 12 total, 12 up, 12 in 2024-09-11T22:23:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.612+0000 7f3b35ea2740 1 Processor -- start 2024-09-11T22:23:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.612+0000 7f3b35ea2740 1 -- start start 2024-09-11T22:23:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe944a0 0x55ffebeab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe94dc0 0x55ffebe9ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebe96600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b35ea2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b3ef0 con 0x55ffebe9f680 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b35ea2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b5a30 con 0x55ffebe94dc0 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b31865640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe944a0 0x55ffebeab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b31865640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe944a0 0x55ffebeab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:51938/0 (socket says 172.21.15.5:51938) 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b31865640 1 -- 172.21.15.5:0/226358284 learned_addr learned my addr 172.21.15.5:0/226358284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b35ea2740 1 -- 172.21.15.5:0/226358284 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b45c0 con 0x55ffebe944a0 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b32066640 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebe96600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b31064640 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe94dc0 0x55ffebe9ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b31865640 1 -- 172.21.15.5:0/226358284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe94dc0 msgr2=0x55ffebe9ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b31865640 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe94dc0 0x55ffebe9ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b31865640 1 -- 172.21.15.5:0/226358284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 msgr2=0x55ffebe96600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b31865640 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebe96600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b31865640 1 -- 172.21.15.5:0/226358284 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffebe9b230 con 0x55ffebe944a0 2024-09-11T22:23:18.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.613+0000 7f3b31064640 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe94dc0 0x55ffebe9ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:18.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.614+0000 7f3b31865640 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe944a0 0x55ffebeab500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b28099d40 tx=0x7f3b2800a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cad103531521cf85 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:18.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.614+0000 7f3b32066640 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebe96600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:18.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.614+0000 7f3b177fe640 1 -- 172.21.15.5:0/226358284 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b280977e0 con 0x55ffebe944a0 2024-09-11T22:23:18.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.614+0000 7f3b177fe640 1 -- 172.21.15.5:0/226358284 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b280ba1e0 con 0x55ffebe944a0 2024-09-11T22:23:18.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.614+0000 7f3b177fe640 1 -- 172.21.15.5:0/226358284 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b280a44d0 con 0x55ffebe944a0 2024-09-11T22:23:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 -- 172.21.15.5:0/226358284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe944a0 msgr2=0x55ffebeab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe944a0 0x55ffebeab500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b28099d40 tx=0x7f3b2800a560 comp rx=0 tx=0).stop 2024-09-11T22:23:18.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 -- 172.21.15.5:0/226358284 shutdown_connections 2024-09-11T22:23:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe9f680 0x55ffebe96600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe94dc0 0x55ffebe9ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/226358284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe944a0 0x55ffebeab500 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 -- 172.21.15.5:0/226358284 >> 172.21.15.5:0/226358284 conn(0x55ffebe90a30 msgr2=0x55ffebe95ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 -- 172.21.15.5:0/226358284 shutdown_connections 2024-09-11T22:23:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 -- 172.21.15.5:0/226358284 wait complete. 2024-09-11T22:23:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 -- start start 2024-09-11T22:23:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.615+0000 7f3b35ea2740 1 Processor -- start 2024-09-11T22:23:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebea3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebea3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b35ea2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe9f680 0x55ffebf18cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b35ea2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b5360 con 0x55ffebe944a0 2024-09-11T22:23:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b35ea2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b4c90 con 0x55ffebe9f680 2024-09-11T22:23:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b35ea2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1b1ed0 con 0x55ffebe94dc0 2024-09-11T22:23:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b35ea2740 10 client.0 _ll_register_callbacks cb 0x7f3b2809b640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b31865640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebea3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b31865640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebea3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59636/0 (socket says 172.21.15.5:59636) 2024-09-11T22:23:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b31865640 1 -- 172.21.15.5:0/2801106498 learned_addr learned my addr 172.21.15.5:0/2801106498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b31064640 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebea3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b31865640 1 -- 172.21.15.5:0/2801106498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 msgr2=0x55ffebea3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b31865640 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebea3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b31865640 1 -- 172.21.15.5:0/2801106498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe9f680 msgr2=0x55ffebf18cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:23:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b31865640 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe9f680 0x55ffebf18cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.616+0000 7f3b31865640 1 -- 172.21.15.5:0/2801106498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffebf19210 con 0x55ffebe944a0 2024-09-11T22:23:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.617+0000 7f3b31865640 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebea3810 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3b28098460 tx=0x7f3b2809c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509c21ee327ff214 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.617+0000 7f3b157fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.617+0000 7f3b157fa640 1 -- 172.21.15.5:0/2801106498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2809aa30 con 0x55ffebe944a0 2024-09-11T22:23:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.617+0000 7f3b35ea2740 1 -- 172.21.15.5:0/2801106498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b280ba1e0 con 0x55ffebe944a0 2024-09-11T22:23:18.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.617+0000 7f3b157fa640 1 -- 172.21.15.5:0/2801106498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f3b2809abd0 con 0x55ffebe944a0 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.617+0000 7f3b157fa640 1 -- 172.21.15.5:0/2801106498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3b2809aee0 con 0x55ffebe944a0 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.617+0000 7f3b35ea2740 10 client.15768 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.617+0000 7f3b35ea2740 1 -- 172.21.15.5:0/2801106498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ffebf19210 con 0x55ffebe944a0 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.617+0000 7f3b157fa640 1 -- 172.21.15.5:0/2801106498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b280ba1e0 con 0x55ffebe944a0 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.618+0000 7f3b157fa640 1 -- 172.21.15.5:0/2801106498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3b280a0fd0 con 0x55ffebe944a0 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.618+0000 7f3b35ea2740 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.618+0000 7f3b35ea2740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.618+0000 7f3b35ea2740 20 client.15768 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.618+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffebeaf180 0x55ffebf1c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.618+0000 7f3b35ea2740 4 client.15768 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:23:18.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.618+0000 7f3b35ea2740 1 -- 172.21.15.5:0/2801106498 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ffebe01d60 con 0x55ffebeaf180 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.618+0000 7f3b31064640 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffebeaf180 0x55ffebf1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.618+0000 7f3b31064640 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffebeaf180 0x55ffebf1c680 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af76680bcf7af8a4 server_cookie=f3b285cccc16605e in_seq=0 out_seq=0 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b157fa640 10 client.15768 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b157fa640 1 -- 172.21.15.5:0/2801106498 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ffebe01d60 con 0x55ffebeaf180 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b157fa640 10 client.15768 handle_command_reply: tid=0 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b35ea2740 1 client.15768 shutdown 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b35ea2740 1 -- 172.21.15.5:0/2801106498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 msgr2=0x55ffebea3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebea3810 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3b28098460 tx=0x7f3b2809c250 comp rx=0 tx=0).stop 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b35ea2740 1 -- 172.21.15.5:0/2801106498 shutdown_connections 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffebeaf180 0x55ffebf1c680 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b157fa640 0 client.15768 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffebe9f680 0x55ffebf18cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffebe94dc0 0x55ffebea3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b35ea2740 1 --2- 172.21.15.5:0/2801106498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffebe944a0 0x55ffebea3810 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b35ea2740 1 -- 172.21.15.5:0/2801106498 >> 172.21.15.5:0/2801106498 conn(0x55ffebe90a30 msgr2=0x55ffebe9e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.619+0000 7f3b35ea2740 1 -- 172.21.15.5:0/2801106498 shutdown_connections 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.620+0000 7f3b35ea2740 1 -- 172.21.15.5:0/2801106498 wait complete. 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.620+0000 7f3b35ea2740 20 client.15768 trim_cache size 0 max 16384 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stdout: "e22a2bf3-58e0-43a8-b12c-d35f425ff668": { 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e22a2bf3-58e0-43a8-b12c-d35f425ff668", 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:23:18.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:18.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:18.625 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:23:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 -- 172.21.15.5:0/3896954778 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3b180a2f20 msgr2=0x7f3b180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3b180a2f20 0x7f3b180a53e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3b24031f80 tx=0x7f3b24005ef0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 -- 172.21.15.5:0/3896954778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b2c101e10 msgr2=0x7f3b2c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b2c101e10 0x7f3b2c0744f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800c920 tx=0x7f3b2800cde0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 -- 172.21.15.5:0/3896954778 shutdown_connections 2024-09-11T22:23:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b2c106f40 0x7f3b2c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b2c1027c0 0x7f3b2c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3b180a2f20 0x7f3b180a53e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 --2- 172.21.15.5:0/3896954778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3b2c101e10 0x7f3b2c0744f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 -- 172.21.15.5:0/3896954778 >> 172.21.15.5:0/3896954778 conn(0x7f3b2c0fd8d0 msgr2=0x7f3b2c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 -- 172.21.15.5:0/3896954778 shutdown_connections 2024-09-11T22:23:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:18.621+0000 7f3b33ac9640 1 -- 172.21.15.5:0/3896954778 wait complete. 2024-09-11T22:23:19.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:19 smithi005 ceph-mon[30943]: pgmap v36: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.9 KiB/s wr, 17 op/s 2024-09-11T22:23:19.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:19 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:23:19.512 INFO:tasks.cephfs.filesystem:scrub status for tag:e22a2bf3-58e0-43a8-b12c-d35f425ff668 - {'path': '/', 'tag': 'e22a2bf3-58e0-43a8-b12c-d35f425ff668', 'options': 'recursive,force'} 2024-09-11T22:23:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:19 smithi159 ceph-mon[35534]: pgmap v36: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.9 KiB/s wr, 17 op/s 2024-09-11T22:23:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:19 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:23:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:19 smithi172 ceph-mon[35461]: pgmap v36: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 2.9 KiB/s wr, 17 op/s 2024-09-11T22:23:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:19 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:23:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.426+0000 7f05333cf640 1 -- 172.21.15.5:0/2195552011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c101350 msgr2=0x7f052c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.426+0000 7f05333cf640 1 --2- 172.21.15.5:0/2195552011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c101350 0x7f052c10d880 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0524009cd0 tx=0x7f052402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.426+0000 7f05333cf640 1 -- 172.21.15.5:0/2195552011 shutdown_connections 2024-09-11T22:23:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.426+0000 7f05333cf640 1 --2- 172.21.15.5:0/2195552011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c101350 0x7f052c10d880 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.426+0000 7f05333cf640 1 --2- 172.21.15.5:0/2195552011 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052c1009b0 0x7f052c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.426+0000 7f05333cf640 1 --2- 172.21.15.5:0/2195552011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f052c1069c0 0x7f052c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.426+0000 7f05333cf640 1 -- 172.21.15.5:0/2195552011 >> 172.21.15.5:0/2195552011 conn(0x7f052c077990 msgr2=0x7f052c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.426+0000 7f05333cf640 1 -- 172.21.15.5:0/2195552011 shutdown_connections 2024-09-11T22:23:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.426+0000 7f05333cf640 1 -- 172.21.15.5:0/2195552011 wait complete. 2024-09-11T22:23:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.426+0000 7f05333cf640 1 Processor -- start 2024-09-11T22:23:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f05333cf640 1 -- start start 2024-09-11T22:23:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f05333cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052c1009b0 0x7f052c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f05333cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c101350 0x7f052c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f05333cf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f052c1069c0 0x7f052c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f05333cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f052c111ef0 con 0x7f052c101350 2024-09-11T22:23:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f05333cf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f052c111d70 con 0x7f052c1069c0 2024-09-11T22:23:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f05333cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f052c112070 con 0x7f052c1009b0 2024-09-11T22:23:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f053096a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c101350 0x7f052c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f053096a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c101350 0x7f052c198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59662/0 (socket says 172.21.15.5:59662) 2024-09-11T22:23:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f053096a640 1 -- 172.21.15.5:0/287711715 learned_addr learned my addr 172.21.15.5:0/287711715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f053196c640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f052c1069c0 0x7f052c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f053116b640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052c1009b0 0x7f052c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f053096a640 1 -- 172.21.15.5:0/287711715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052c1009b0 msgr2=0x7f052c1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f053096a640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052c1009b0 0x7f052c1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f053096a640 1 -- 172.21.15.5:0/287711715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f052c1069c0 msgr2=0x7f052c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f053096a640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f052c1069c0 0x7f052c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.427+0000 7f053096a640 1 -- 172.21.15.5:0/287711715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052c19d2d0 con 0x7f052c101350 2024-09-11T22:23:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.428+0000 7f053096a640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c101350 0x7f052c198820 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f051c00d950 tx=0x7f051c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.428+0000 7f05227fc640 1 -- 172.21.15.5:0/287711715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c017070 con 0x7f052c101350 2024-09-11T22:23:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.428+0000 7f05227fc640 1 -- 172.21.15.5:0/287711715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c009c30 con 0x7f052c101350 2024-09-11T22:23:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.428+0000 7f05227fc640 1 -- 172.21.15.5:0/287711715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c00f6b0 con 0x7f052c101350 2024-09-11T22:23:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.428+0000 7f05333cf640 1 -- 172.21.15.5:0/287711715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052c19d5c0 con 0x7f052c101350 2024-09-11T22:23:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.428+0000 7f05333cf640 1 -- 172.21.15.5:0/287711715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052c10e240 con 0x7f052c101350 2024-09-11T22:23:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.431+0000 7f05227fc640 1 -- 172.21.15.5:0/287711715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7f051c0109d0 con 0x7f052c101350 2024-09-11T22:23:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f0536244740 1 Processor -- start 2024-09-11T22:23:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f0536244740 1 -- start start 2024-09-11T22:23:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f0536244740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce958ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f0536244740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce959410 0x5555ce963770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f0536244740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce963cb0 0x5555ce9660a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f05227fc640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05180a2f50 0x7f05180a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f0536244740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5555ce8bd2f0 con 0x5555ce963cb0 2024-09-11T22:23:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f0536244740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5555ce865270 con 0x5555ce958af0 2024-09-11T22:23:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f0536244740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555ce8b92b0 con 0x5555ce959410 2024-09-11T22:23:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f053096a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce963cb0 0x5555ce9660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f053196c640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05180a2f50 0x7f05180a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f053096a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce963cb0 0x5555ce9660a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59678/0 (socket says 172.21.15.5:59678) 2024-09-11T22:23:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f053096a640 1 -- 172.21.15.5:0/2698471019 learned_addr learned my addr 172.21.15.5:0/2698471019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.432+0000 7f05227fc640 1 -- 172.21.15.5:0/287711715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f051c0cf3e0 con 0x7f052c101350 2024-09-11T22:23:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.433+0000 7f053096a640 1 -- 172.21.15.5:0/2698471019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce959410 msgr2=0x5555ce963770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.433+0000 7f053096a640 1 --2- 172.21.15.5:0/2698471019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce959410 0x5555ce963770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.433+0000 7f053096a640 1 -- 172.21.15.5:0/2698471019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 msgr2=0x5555ce958ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.433+0000 7f053096a640 1 --2- 172.21.15.5:0/2698471019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce958ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.433+0000 7f053096a640 1 -- 172.21.15.5:0/2698471019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ce864340 con 0x5555ce963cb0 2024-09-11T22:23:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.433+0000 7f053196c640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05180a2f50 0x7f05180a5410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0524004ab0 tx=0x7f0524009800 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.433+0000 7f053096a640 1 --2- 172.21.15.5:0/2698471019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce963cb0 0x5555ce9660a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f051c097ae0 tx=0x7f051c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee42b9bd84596fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.433+0000 7f05177fe640 1 -- 172.21.15.5:0/2698471019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c020070 con 0x5555ce963cb0 2024-09-11T22:23:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.433+0000 7f05177fe640 1 -- 172.21.15.5:0/2698471019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c09bd80 con 0x5555ce963cb0 2024-09-11T22:23:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.433+0000 7f05177fe640 1 -- 172.21.15.5:0/2698471019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c066320 con 0x5555ce963cb0 2024-09-11T22:23:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.434+0000 7f0536244740 1 -- 172.21.15.5:0/2698471019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce963cb0 msgr2=0x5555ce9660a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.434+0000 7f0536244740 1 --2- 172.21.15.5:0/2698471019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce963cb0 0x5555ce9660a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f051c097ae0 tx=0x7f051c00bf60 comp rx=0 tx=0).stop 2024-09-11T22:23:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.434+0000 7f0536244740 1 -- 172.21.15.5:0/2698471019 shutdown_connections 2024-09-11T22:23:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.434+0000 7f0536244740 1 --2- 172.21.15.5:0/2698471019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce963cb0 0x5555ce9660a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.434+0000 7f0536244740 1 --2- 172.21.15.5:0/2698471019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce959410 0x5555ce963770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.434+0000 7f0536244740 1 --2- 172.21.15.5:0/2698471019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce958ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.434+0000 7f0536244740 1 -- 172.21.15.5:0/2698471019 >> 172.21.15.5:0/2698471019 conn(0x5555ce955060 msgr2=0x5555ce955440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.434+0000 7f0536244740 1 -- 172.21.15.5:0/2698471019 shutdown_connections 2024-09-11T22:23:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.434+0000 7f0536244740 1 -- 172.21.15.5:0/2698471019 wait complete. 2024-09-11T22:23:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.434+0000 7f0536244740 1 -- start start 2024-09-11T22:23:20.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.435+0000 7f0536244740 1 Processor -- start 2024-09-11T22:23:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.435+0000 7f0536244740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce960de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.435+0000 7f0536244740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 0x5555ce961320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.435+0000 7f0536244740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce961860 0x5555ce96fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.435+0000 7f0536244740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5555ce966820 con 0x5555ce959410 2024-09-11T22:23:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.435+0000 7f0536244740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5555ce846290 con 0x5555ce958af0 2024-09-11T22:23:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.435+0000 7f0536244740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555ce9669a0 con 0x5555ce961860 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.435+0000 7f0536244740 10 client.0 _ll_register_callbacks cb 0x5555ce877140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.435+0000 7f053096a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 0x5555ce961320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053116b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce960de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053116b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce960de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38448/0 (socket says 172.21.15.5:38448) 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053116b640 1 -- 172.21.15.5:0/979719352 learned_addr learned my addr 172.21.15.5:0/979719352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053196c640 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce961860 0x5555ce96fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053116b640 1 -- 172.21.15.5:0/979719352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce961860 msgr2=0x5555ce96fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053116b640 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce961860 0x5555ce96fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053116b640 1 -- 172.21.15.5:0/979719352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 msgr2=0x5555ce961320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053116b640 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 0x5555ce961320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053116b640 1 -- 172.21.15.5:0/979719352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ce5d0210 con 0x5555ce958af0 2024-09-11T22:23:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053196c640 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce961860 0x5555ce96fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f053116b640 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce960de0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f052800dd30 tx=0x7f052800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c87a77009e504e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f0514ff9640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:23:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f0514ff9640 1 -- 172.21.15.5:0/979719352 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528011640 con 0x5555ce958af0 2024-09-11T22:23:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f0536244740 1 -- 172.21.15.5:0/979719352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f051c09bd80 con 0x5555ce958af0 2024-09-11T22:23:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.436+0000 7f0514ff9640 1 -- 172.21.15.5:0/979719352 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f05280117e0 con 0x5555ce958af0 2024-09-11T22:23:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0514ff9640 1 -- 172.21.15.5:0/979719352 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f05280119e0 con 0x5555ce958af0 2024-09-11T22:23:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0536244740 10 client.25325 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0536244740 1 -- 172.21.15.5:0/979719352 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555ce69c620 con 0x5555ce958af0 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0514ff9640 1 -- 172.21.15.5:0/979719352 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c09bd80 con 0x5555ce958af0 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0514ff9640 1 -- 172.21.15.5:0/979719352 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f05280103d0 con 0x5555ce958af0 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0536244740 10 client.25325 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0536244740 10 client.25325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0536244740 20 client.25325 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0536244740 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5555ce974640 0x5555ce976aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0536244740 4 client.25325 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f0536244740 1 -- 172.21.15.5:0/979719352 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555ce8caa20 con 0x5555ce974640 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.437+0000 7f053096a640 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5555ce974640 0x5555ce976aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.438+0000 7f0514ff9640 10 client.25325 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.438+0000 7f053096a640 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5555ce974640 0x5555ce976aa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=190e8958ae7c9a15 server_cookie=15d9f41d58ec0118 in_seq=0 out_seq=0 2024-09-11T22:23:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0514ff9640 1 -- 172.21.15.5:0/979719352 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555ce8caa20 con 0x5555ce974640 2024-09-11T22:23:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0514ff9640 10 client.25325 handle_command_reply: tid=0 2024-09-11T22:23:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 client.25325 shutdown 2024-09-11T22:23:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 -- 172.21.15.5:0/979719352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 msgr2=0x5555ce960de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce960de0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f052800dd30 tx=0x7f052800f730 comp rx=0 tx=0).stop 2024-09-11T22:23:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 -- 172.21.15.5:0/979719352 shutdown_connections 2024-09-11T22:23:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5555ce974640 0x5555ce976aa0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce961860 0x5555ce96fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 0x5555ce961320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0514ff9640 0 client.25325 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 --2- 172.21.15.5:0/979719352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce960de0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 -- 172.21.15.5:0/979719352 >> 172.21.15.5:0/979719352 conn(0x5555ce955060 msgr2=0x5555ce95adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 -- 172.21.15.5:0/979719352 shutdown_connections 2024-09-11T22:23:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 1 -- 172.21.15.5:0/979719352 wait complete. 2024-09-11T22:23:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.439+0000 7f0536244740 20 client.25325 trim_cache size 0 max 16384 2024-09-11T22:23:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f0536244740 1 Processor -- start 2024-09-11T22:23:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f0536244740 1 -- start start 2024-09-11T22:23:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f0536244740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce958af0 0x5555ce96fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f0536244740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 0x5555ce962d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f0536244740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce961860 0x5555ce97c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f0536244740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1afc10 con 0x5555ce959410 2024-09-11T22:23:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f0536244740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1b1750 con 0x5555ce961860 2024-09-11T22:23:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f0536244740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1b02e0 con 0x5555ce958af0 2024-09-11T22:23:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f053096a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 0x5555ce962d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f053096a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 0x5555ce962d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59702/0 (socket says 172.21.15.5:59702) 2024-09-11T22:23:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f053096a640 1 -- 172.21.15.5:0/2399723927 learned_addr learned my addr 172.21.15.5:0/2399723927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.468+0000 7f053116b640 1 --2- 172.21.15.5:0/2399723927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce958af0 0x5555ce96fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.469+0000 7f053196c640 1 --2- 172.21.15.5:0/2399723927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce961860 0x5555ce97c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.469+0000 7f053096a640 1 -- 172.21.15.5:0/2399723927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce958af0 msgr2=0x5555ce96fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.469+0000 7f053096a640 1 --2- 172.21.15.5:0/2399723927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce958af0 0x5555ce96fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.469+0000 7f053096a640 1 -- 172.21.15.5:0/2399723927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce961860 msgr2=0x5555ce97c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.469+0000 7f053096a640 1 --2- 172.21.15.5:0/2399723927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce961860 0x5555ce97c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.469+0000 7f053096a640 1 -- 172.21.15.5:0/2399723927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ce972e20 con 0x5555ce959410 2024-09-11T22:23:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.469+0000 7f053096a640 1 --2- 172.21.15.5:0/2399723927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 0x5555ce962d50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f051c09b610 tx=0x7f051c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f4951b6b00d7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.469+0000 7f0516ffd640 1 -- 172.21.15.5:0/2399723927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c0b9070 con 0x5555ce959410 2024-09-11T22:23:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.469+0000 7f0516ffd640 1 -- 172.21.15.5:0/2399723927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c09bd80 con 0x5555ce959410 2024-09-11T22:23:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.469+0000 7f0516ffd640 1 -- 172.21.15.5:0/2399723927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c020040 con 0x5555ce959410 2024-09-11T22:23:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.470+0000 7f0536244740 1 -- 172.21.15.5:0/2399723927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 msgr2=0x5555ce962d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.470+0000 7f0536244740 1 --2- 172.21.15.5:0/2399723927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 0x5555ce962d50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f051c09b610 tx=0x7f051c010d70 comp rx=0 tx=0).stop 2024-09-11T22:23:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.470+0000 7f0536244740 1 -- 172.21.15.5:0/2399723927 shutdown_connections 2024-09-11T22:23:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.470+0000 7f0536244740 1 --2- 172.21.15.5:0/2399723927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce961860 0x5555ce97c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.470+0000 7f0536244740 1 --2- 172.21.15.5:0/2399723927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce959410 0x5555ce962d50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.470+0000 7f0536244740 1 --2- 172.21.15.5:0/2399723927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce958af0 0x5555ce96fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.470+0000 7f0536244740 1 -- 172.21.15.5:0/2399723927 >> 172.21.15.5:0/2399723927 conn(0x5555ce955060 msgr2=0x5555ce962640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.470+0000 7f0536244740 1 -- 172.21.15.5:0/2399723927 shutdown_connections 2024-09-11T22:23:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.470+0000 7f0536244740 1 -- 172.21.15.5:0/2399723927 wait complete. 2024-09-11T22:23:20.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.471+0000 7f0536244740 1 -- start start 2024-09-11T22:23:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.471+0000 7f0536244740 1 Processor -- start 2024-09-11T22:23:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f0536244740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce9678a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f0536244740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce961860 0x5555ce967de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f0536244740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce968320 0x5555ce9de380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f0536244740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1b1080 con 0x5555ce961860 2024-09-11T22:23:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f053096a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce961860 0x5555ce967de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f053116b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce9678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f053096a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce961860 0x5555ce967de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59714/0 (socket says 172.21.15.5:59714) 2024-09-11T22:23:20.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f053096a640 1 -- 172.21.15.5:0/1515877710 learned_addr learned my addr 172.21.15.5:0/1515877710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f0536244740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1b09b0 con 0x5555ce958af0 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f0536244740 1 -- 172.21.15.5:0/1515877710 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f052c1adbf0 con 0x5555ce968320 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f0536244740 10 client.0 _ll_register_callbacks cb 0x7f052800d300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f053096a640 1 -- 172.21.15.5:0/1515877710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce968320 msgr2=0x5555ce9de380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f053096a640 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce968320 0x5555ce9de380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f053096a640 1 -- 172.21.15.5:0/1515877710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 msgr2=0x5555ce9678a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f053096a640 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce9678a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f053096a640 1 -- 172.21.15.5:0/1515877710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ce961690 con 0x5555ce961860 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.472+0000 7f053116b640 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce9678a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.473+0000 7f053096a640 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce961860 0x5555ce967de0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f051c097ae0 tx=0x7f051c0a4470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4159ebd1e9266c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:20.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.473+0000 7f0514ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.473+0000 7f0514ff9640 1 -- 172.21.15.5:0/1515877710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051c055060 con 0x5555ce961860 2024-09-11T22:23:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.473+0000 7f0514ff9640 1 -- 172.21.15.5:0/1515877710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f051c0a3e60 con 0x5555ce961860 2024-09-11T22:23:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.473+0000 7f0536244740 1 -- 172.21.15.5:0/1515877710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f051c09bd80 con 0x5555ce961860 2024-09-11T22:23:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.473+0000 7f0514ff9640 1 -- 172.21.15.5:0/1515877710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f051c0b9070 con 0x5555ce961860 2024-09-11T22:23:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.473+0000 7f0536244740 10 client.15786 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.473+0000 7f0536244740 1 -- 172.21.15.5:0/1515877710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5555ce966990 con 0x5555ce961860 2024-09-11T22:23:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.474+0000 7f0514ff9640 1 -- 172.21.15.5:0/1515877710 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f051c09bd80 con 0x5555ce961860 2024-09-11T22:23:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.474+0000 7f0514ff9640 1 -- 172.21.15.5:0/1515877710 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f051c0afba0 con 0x5555ce961860 2024-09-11T22:23:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.474+0000 7f0536244740 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.474+0000 7f0536244740 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.474+0000 7f0536244740 20 client.15786 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.474+0000 7f0536244740 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5555ce9df820 0x5555ce9e1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.474+0000 7f0536244740 4 client.15786 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:23:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.474+0000 7f0536244740 1 -- 172.21.15.5:0/1515877710 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555ce8c4d70 con 0x5555ce9df820 2024-09-11T22:23:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.474+0000 7f053116b640 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5555ce9df820 0x5555ce9e1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.475+0000 7f053116b640 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5555ce9df820 0x5555ce9e1c10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=728aec96333ae318 server_cookie=c7b8e79b59faa128 in_seq=0 out_seq=0 2024-09-11T22:23:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.475+0000 7f0514ff9640 10 client.15786 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0514ff9640 1 -- 172.21.15.5:0/1515877710 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555ce8c4d70 con 0x5555ce9df820 2024-09-11T22:23:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0514ff9640 10 client.15786 handle_command_reply: tid=0 2024-09-11T22:23:20.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 client.15786 shutdown 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 -- 172.21.15.5:0/1515877710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce961860 msgr2=0x5555ce967de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce961860 0x5555ce967de0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f051c097ae0 tx=0x7f051c0a4470 comp rx=0 tx=0).stop 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 -- 172.21.15.5:0/1515877710 shutdown_connections 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5555ce9df820 0x5555ce9e1c10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555ce968320 0x5555ce9de380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555ce961860 0x5555ce967de0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 --2- 172.21.15.5:0/1515877710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555ce958af0 0x5555ce9678a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 -- 172.21.15.5:0/1515877710 >> 172.21.15.5:0/1515877710 conn(0x5555ce955060 msgr2=0x5555ce9541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0514ff9640 0 client.15786 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 -- 172.21.15.5:0/1515877710 shutdown_connections 2024-09-11T22:23:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 1 -- 172.21.15.5:0/1515877710 wait complete. 2024-09-11T22:23:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.483+0000 7f0536244740 20 client.15786 trim_cache size 0 max 16384 2024-09-11T22:23:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:23:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:23:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:23:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:23:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.484+0000 7f05333cf640 1 -- 172.21.15.5:0/287711715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05180a2f50 msgr2=0x7f05180a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.484+0000 7f05333cf640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05180a2f50 0x7f05180a5410 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0524004ab0 tx=0x7f0524009800 comp rx=0 tx=0).stop 2024-09-11T22:23:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.484+0000 7f05333cf640 1 -- 172.21.15.5:0/287711715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c101350 msgr2=0x7f052c198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.484+0000 7f05333cf640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c101350 0x7f052c198820 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f051c00d950 tx=0x7f051c00de10 comp rx=0 tx=0).stop 2024-09-11T22:23:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.484+0000 7f05333cf640 1 -- 172.21.15.5:0/287711715 shutdown_connections 2024-09-11T22:23:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.484+0000 7f05333cf640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f05180a2f50 0x7f05180a5410 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.484+0000 7f05333cf640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f052c1069c0 0x7f052c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.484+0000 7f05333cf640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f052c101350 0x7f052c198820 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.484+0000 7f05333cf640 1 --2- 172.21.15.5:0/287711715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f052c1009b0 0x7f052c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.484+0000 7f05333cf640 1 -- 172.21.15.5:0/287711715 >> 172.21.15.5:0/287711715 conn(0x7f052c077990 msgr2=0x7f052c0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.485+0000 7f05333cf640 1 -- 172.21.15.5:0/287711715 shutdown_connections 2024-09-11T22:23:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:20.485+0000 7f05333cf640 1 -- 172.21.15.5:0/287711715 wait complete. 2024-09-11T22:23:21.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:21 smithi005 ceph-mon[30943]: pgmap v37: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T22:23:21.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:21 smithi159 ceph-mon[35534]: pgmap v37: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T22:23:21.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:21 smithi172 ceph-mon[35461]: pgmap v37: 97 pgs: 2 peering, 95 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-11T22:23:23.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:23 smithi005 ceph-mon[30943]: pgmap v38: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-11T22:23:23.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:23 smithi159 ceph-mon[35534]: pgmap v38: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-11T22:23:23.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:23 smithi172 ceph-mon[35461]: pgmap v38: 97 pgs: 1 peering, 96 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-11T22:23:25.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:25 smithi005 ceph-mon[30943]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 17 op/s 2024-09-11T22:23:25.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:25 smithi159 ceph-mon[35534]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 17 op/s 2024-09-11T22:23:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:25 smithi172 ceph-mon[35461]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.9 KiB/s wr, 17 op/s 2024-09-11T22:23:27.381 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:27 smithi159 ceph-mon[35534]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-11T22:23:27.381 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:27 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:23:27.381 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:27 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:23:27.381 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:27 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:23:27.381 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:27 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:23:27.381 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:27 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:23:27.382 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:27 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:23:27.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:27 smithi005 ceph-mon[30943]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-11T22:23:27.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:27 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:23:27.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:27 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:23:27.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:27 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:23:27.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:27 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:23:27.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:27 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:23:27.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:27 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:23:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:27 smithi172 ceph-mon[35461]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-09-11T22:23:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:27 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:23:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:27 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:23:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:27 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:23:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:27 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:23:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:27 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:23:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:27 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:23:28.836 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:23:29.122 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:29.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:29 smithi005 ceph-mon[30943]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-11T22:23:29.412 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:29 smithi172 ceph-mon[35461]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-11T22:23:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:29 smithi159 ceph-mon[35534]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-09-11T22:23:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.690+0000 7fb3c0c61640 1 -- 172.21.15.5:0/2007427818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3bc108290 msgr2=0x7fb3bc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.690+0000 7fb3c0c61640 1 --2- 172.21.15.5:0/2007427818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3bc108290 0x7fb3bc10ec30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009cd0 tx=0x7fb3b402ecf0 comp rx=0 tx=0).stop 2024-09-11T22:23:29.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.690+0000 7fb3c0c61640 1 -- 172.21.15.5:0/2007427818 shutdown_connections 2024-09-11T22:23:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.690+0000 7fb3c0c61640 1 --2- 172.21.15.5:0/2007427818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3bc108290 0x7fb3bc10ec30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.690+0000 7fb3c0c61640 1 --2- 172.21.15.5:0/2007427818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3bc103740 0x7fb3bc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.690+0000 7fb3c0c61640 1 --2- 172.21.15.5:0/2007427818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc102d90 0x7fb3bc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.690+0000 7fb3c0c61640 1 -- 172.21.15.5:0/2007427818 >> 172.21.15.5:0/2007427818 conn(0x7fb3bc0fc7d0 msgr2=0x7fb3bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.690+0000 7fb3c0c61640 1 -- 172.21.15.5:0/2007427818 shutdown_connections 2024-09-11T22:23:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.690+0000 7fb3c0c61640 1 -- 172.21.15.5:0/2007427818 wait complete. 2024-09-11T22:23:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.691+0000 7fb3c0c61640 1 Processor -- start 2024-09-11T22:23:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.691+0000 7fb3c0c61640 1 -- start start 2024-09-11T22:23:29.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3c0c61640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3bc102d90 0x7fb3bc0739e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:29.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3c0c61640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc103740 0x7fb3bc06ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:29.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3c0c61640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3bc108290 0x7fb3bc06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:29.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3c0c61640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc10fe80 con 0x7fb3bc108290 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3c0c61640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc10fd00 con 0x7fb3bc103740 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3c0c61640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc110000 con 0x7fb3bc102d90 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3b9d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc103740 0x7fb3bc06ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3b9d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc103740 0x7fb3bc06ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44240/0 (socket says 172.21.15.5:44240) 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3b9d9b640 1 -- 172.21.15.5:0/1914490224 learned_addr learned my addr 172.21.15.5:0/1914490224 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3ba59c640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3bc102d90 0x7fb3bc0739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.692+0000 7fb3bad9d640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3bc108290 0x7fb3bc06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3b9d9b640 1 -- 172.21.15.5:0/1914490224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3bc102d90 msgr2=0x7fb3bc0739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3b9d9b640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3bc102d90 0x7fb3bc0739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3b9d9b640 1 -- 172.21.15.5:0/1914490224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3bc108290 msgr2=0x7fb3bc06cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3b9d9b640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3bc108290 0x7fb3bc06cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:29.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3b9d9b640 1 -- 172.21.15.5:0/1914490224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc06d570 con 0x7fb3bc103740 2024-09-11T22:23:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3b9d9b640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc103740 0x7fb3bc06ca70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000d740 tx=0x7fb3b000dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3ab7fe640 1 -- 172.21.15.5:0/1914490224 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0017070 con 0x7fb3bc103740 2024-09-11T22:23:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3c0c61640 1 -- 172.21.15.5:0/1914490224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc06d860 con 0x7fb3bc103740 2024-09-11T22:23:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3ab7fe640 1 -- 172.21.15.5:0/1914490224 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b000de50 con 0x7fb3bc103740 2024-09-11T22:23:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3ab7fe640 1 -- 172.21.15.5:0/1914490224 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0004e00 con 0x7fb3bc103740 2024-09-11T22:23:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.693+0000 7fb3c0c61640 1 -- 172.21.15.5:0/1914490224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc071830 con 0x7fb3bc103740 2024-09-11T22:23:29.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.695+0000 7fb3c0c61640 1 -- 172.21.15.5:0/1914490224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3bc06a7f0 con 0x7fb3bc103740 2024-09-11T22:23:29.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.697+0000 7fb3ab7fe640 1 -- 172.21.15.5:0/1914490224 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7fb3b0004fa0 con 0x7fb3bc103740 2024-09-11T22:23:29.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.698+0000 7fb3ab7fe640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2f20 0x7fb3a40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:29.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.698+0000 7fb3ab7fe640 1 -- 172.21.15.5:0/1914490224 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fb3b00cf000 con 0x7fb3bc103740 2024-09-11T22:23:29.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.698+0000 7fb3ba59c640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2f20 0x7fb3a40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:29.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.699+0000 7fb3ba59c640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2f20 0x7fb3a40a53e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009bd0 tx=0x7fb3ac005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:29.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:29.700+0000 7fb3ab7fe640 1 -- 172.21.15.5:0/1914490224 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3b009baa0 con 0x7fb3bc103740 2024-09-11T22:23:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.129+0000 7fb3c0c61640 1 -- 172.21.15.5:0/1914490224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3bc072250 con 0x7fb3bc103740 2024-09-11T22:23:30.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.133+0000 7fb3ab7fe640 1 -- 172.21.15.5:0/1914490224 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11967 (secure 0 0 0) 0x7fb3b009b3a0 con 0x7fb3bc103740 2024-09-11T22:23:30.134 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:23:30.134 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-11T22:22:15:230183+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:22:15.180031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:23:30.134 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-11T22:23:30.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 -- 172.21.15.5:0/1914490224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2f20 msgr2=0x7fb3a40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:30.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2f20 0x7fb3a40a53e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009bd0 tx=0x7fb3ac005f90 comp rx=0 tx=0).stop 2024-09-11T22:23:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 -- 172.21.15.5:0/1914490224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc103740 msgr2=0x7fb3bc06ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc103740 0x7fb3bc06ca70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000d740 tx=0x7fb3b000dc00 comp rx=0 tx=0).stop 2024-09-11T22:23:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 -- 172.21.15.5:0/1914490224 shutdown_connections 2024-09-11T22:23:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3bc108290 0x7fb3bc06cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2f20 0x7fb3a40a53e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3bc103740 0x7fb3bc06ca70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 --2- 172.21.15.5:0/1914490224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3bc102d90 0x7fb3bc0739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 -- 172.21.15.5:0/1914490224 >> 172.21.15.5:0/1914490224 conn(0x7fb3bc0fc7d0 msgr2=0x7fb3bc105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 -- 172.21.15.5:0/1914490224 shutdown_connections 2024-09-11T22:23:30.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:30.136+0000 7fb3c0c61640 1 -- 172.21.15.5:0/1914490224 wait complete. 2024-09-11T22:23:30.974 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 7d67fffb --release --await' 2024-09-11T22:23:30.975 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 7d67fffb --release --await 2024-09-11T22:23:31.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:31 smithi005 ceph-mon[30943]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:23:31.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:31 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1914490224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:31.339 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:31.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:31 smithi159 ceph-mon[35534]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:23:31.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:31 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1914490224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:31.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:31 smithi172 ceph-mon[35461]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-11T22:23:31.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:31 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1914490224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:32.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.021+0000 7f60b43bc640 1 -- 172.21.15.5:0/1340540462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac106f20 msgr2=0x7f60ac10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:32.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.021+0000 7f60b43bc640 1 --2- 172.21.15.5:0/1340540462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac106f20 0x7f60ac10eaa0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009ca0 tx=0x7f60a402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.022+0000 7f60b43bc640 1 -- 172.21.15.5:0/1340540462 shutdown_connections 2024-09-11T22:23:32.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.022+0000 7f60b43bc640 1 --2- 172.21.15.5:0/1340540462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac106f20 0x7f60ac10eaa0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.022+0000 7f60b43bc640 1 --2- 172.21.15.5:0/1340540462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac1027a0 0x7f60ac1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.022+0000 7f60b43bc640 1 --2- 172.21.15.5:0/1340540462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac101df0 0x7f60ac1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.022+0000 7f60b43bc640 1 -- 172.21.15.5:0/1340540462 >> 172.21.15.5:0/1340540462 conn(0x7f60ac0fd8f0 msgr2=0x7f60ac0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:32.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.022+0000 7f60b43bc640 1 -- 172.21.15.5:0/1340540462 shutdown_connections 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.022+0000 7f60b43bc640 1 -- 172.21.15.5:0/1340540462 wait complete. 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.023+0000 7f60b43bc640 1 Processor -- start 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.023+0000 7f60b43bc640 1 -- start start 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b43bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac101df0 0x7f60ac1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b43bc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac1027a0 0x7f60ac198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b43bc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac106f20 0x7f60ac19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b43bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac111fd0 con 0x7f60ac101df0 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b43bc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac111e50 con 0x7f60ac106f20 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b43bc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac112150 con 0x7f60ac1027a0 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b2158640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac101df0 0x7f60ac1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b2158640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac101df0 0x7f60ac1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41364/0 (socket says 172.21.15.5:41364) 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b2158640 1 -- 172.21.15.5:0/539664023 learned_addr learned my addr 172.21.15.5:0/539664023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b1957640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac1027a0 0x7f60ac198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.024+0000 7f60b2959640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac106f20 0x7f60ac19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60b1957640 1 -- 172.21.15.5:0/539664023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac106f20 msgr2=0x7f60ac19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60b1957640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac106f20 0x7f60ac19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60b1957640 1 -- 172.21.15.5:0/539664023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac101df0 msgr2=0x7f60ac1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60b1957640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac101df0 0x7f60ac1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60b1957640 1 -- 172.21.15.5:0/539664023 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac19d380 con 0x7f60ac1027a0 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60b1957640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac1027a0 0x7f60ac198910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f609c00e9f0 tx=0x7f609c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60a37fe640 1 -- 172.21.15.5:0/539664023 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c00ccc0 con 0x7f60ac1027a0 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60b43bc640 1 -- 172.21.15.5:0/539664023 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac19d670 con 0x7f60ac1027a0 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60b43bc640 1 -- 172.21.15.5:0/539664023 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac1a4f50 con 0x7f60ac1027a0 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60a37fe640 1 -- 172.21.15.5:0/539664023 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609c010910 con 0x7f60ac1027a0 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.025+0000 7f60a37fe640 1 -- 172.21.15.5:0/539664023 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c018960 con 0x7f60ac1027a0 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.028+0000 7f60a37fe640 1 -- 172.21.15.5:0/539664023 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7f609c010ac0 con 0x7f60ac1027a0 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.029+0000 7f60a37fe640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2f20 0x7f60980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.029+0000 7f60a37fe640 1 -- 172.21.15.5:0/539664023 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f609c0cf970 con 0x7f60ac1027a0 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.029+0000 7f60b2158640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2f20 0x7f60980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.030+0000 7f60b2158640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2f20 0x7f60980a53e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f60a8005fd0 tx=0x7f60a800b2c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.030+0000 7f60b6795740 1 Processor -- start 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.031+0000 7f60b6795740 1 -- start start 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.031+0000 7f60b6795740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165ed0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.031+0000 7f60b6795740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ed0840 0x557165edf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.031+0000 7f60b6795740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ee1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.031+0000 7f60b2158640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165ed0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.031+0000 7f60b2158640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165ed0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:57092/0 (socket says 172.21.15.5:57092) 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.031+0000 7f60b1957640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ed0840 0x557165edf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.031+0000 7f60b1957640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ed0840 0x557165edf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59774/0 (socket says 172.21.15.5:59774) 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.032+0000 7f60b2959640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ee1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.032+0000 7f60b2158640 1 -- 172.21.15.5:0/1081326402 learned_addr learned my addr 172.21.15.5:0/1081326402 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.032+0000 7f60b6795740 1 -- 172.21.15.5:0/1081326402 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557165e35620 con 0x557165edf650 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.032+0000 7f60b6795740 1 -- 172.21.15.5:0/1081326402 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557165ddce50 con 0x557165ed0840 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.032+0000 7f60b6795740 1 -- 172.21.15.5:0/1081326402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557165e315e0 con 0x557165ecff20 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.033+0000 7f60b2158640 1 -- 172.21.15.5:0/1081326402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ed0840 msgr2=0x557165edf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.033+0000 7f60b2158640 1 --2- 172.21.15.5:0/1081326402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ed0840 0x557165edf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.033+0000 7f60b2158640 1 -- 172.21.15.5:0/1081326402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 msgr2=0x557165ee1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.033+0000 7f60b2158640 1 --2- 172.21.15.5:0/1081326402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ee1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.033+0000 7f60b2158640 1 -- 172.21.15.5:0/1081326402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557165dcdd40 con 0x557165ecff20 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.033+0000 7f60b1957640 1 --2- 172.21.15.5:0/1081326402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ed0840 0x557165edf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.033+0000 7f60b2959640 1 --2- 172.21.15.5:0/1081326402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ee1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.033+0000 7f60b2158640 1 --2- 172.21.15.5:0/1081326402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165ed0300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f60a800bf70 tx=0x7f60a8012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbccfedfafb2212c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:32.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.034+0000 7f60a0dea640 1 -- 172.21.15.5:0/1081326402 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8022070 con 0x557165ecff20 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.034+0000 7f60a0dea640 1 -- 172.21.15.5:0/1081326402 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a80047a0 con 0x557165ecff20 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.034+0000 7f60a0dea640 1 -- 172.21.15.5:0/1081326402 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a801e6d0 con 0x557165ecff20 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.034+0000 7f60b6795740 1 -- 172.21.15.5:0/1081326402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 msgr2=0x557165ed0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.034+0000 7f60b6795740 1 --2- 172.21.15.5:0/1081326402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165ed0300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f60a800bf70 tx=0x7f60a8012000 comp rx=0 tx=0).stop 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.035+0000 7f60b6795740 1 -- 172.21.15.5:0/1081326402 shutdown_connections 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.035+0000 7f60b6795740 1 --2- 172.21.15.5:0/1081326402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ee1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.035+0000 7f60b6795740 1 --2- 172.21.15.5:0/1081326402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ed0840 0x557165edf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.035+0000 7f60b6795740 1 --2- 172.21.15.5:0/1081326402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165ed0300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.035+0000 7f60b6795740 1 -- 172.21.15.5:0/1081326402 >> 172.21.15.5:0/1081326402 conn(0x557165ecc5f0 msgr2=0x557165ecc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.035+0000 7f60b6795740 1 -- 172.21.15.5:0/1081326402 shutdown_connections 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.035+0000 7f60b6795740 1 -- 172.21.15.5:0/1081326402 wait complete. 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.035+0000 7f60b6795740 1 -- start start 2024-09-11T22:23:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.035+0000 7f60b6795740 1 Processor -- start 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b6795740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165edf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b6795740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ed0840 0x557165edc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b6795740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165edf650 0x557165edcae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b6795740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557165ee2220 con 0x557165ed0840 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b6795740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557165dbdf10 con 0x557165edf650 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b6795740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557165ee23a0 con 0x557165ecff20 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b6795740 10 client.0 _ll_register_callbacks cb 0x557165de1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b2158640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165edf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b2158640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165edf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:57098/0 (socket says 172.21.15.5:57098) 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b2158640 1 -- 172.21.15.5:0/1723852799 learned_addr learned my addr 172.21.15.5:0/1723852799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.036+0000 7f60b1957640 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ed0840 0x557165edc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.037+0000 7f60b2959640 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165edf650 0x557165edcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.037+0000 7f60b2158640 1 -- 172.21.15.5:0/1723852799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165edf650 msgr2=0x557165edcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.037+0000 7f60b2158640 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165edf650 0x557165edcae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.037+0000 7f60b2158640 1 -- 172.21.15.5:0/1723852799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ed0840 msgr2=0x557165edc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.037+0000 7f60b2158640 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ed0840 0x557165edc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.037+0000 7f60b2158640 1 -- 172.21.15.5:0/1723852799 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557165b44580 con 0x557165ecff20 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.037+0000 7f60b1957640 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ed0840 0x557165edc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.037+0000 7f60b2158640 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165edf080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f60a800b490 tx=0x7f60a8004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3b9e60654753b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.037+0000 7f6095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.038+0000 7f6095ffb640 1 -- 172.21.15.5:0/1723852799 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8022040 con 0x557165ecff20 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.038+0000 7f6095ffb640 1 -- 172.21.15.5:0/1723852799 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f60a8005210 con 0x557165ecff20 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.038+0000 7f60b6795740 1 -- 172.21.15.5:0/1723852799 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a80047a0 con 0x557165ecff20 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.038+0000 7f6095ffb640 1 -- 172.21.15.5:0/1723852799 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60a8005410 con 0x557165ecff20 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.038+0000 7f60b6795740 10 client.25306 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.038+0000 7f60b6795740 1 -- 172.21.15.5:0/1723852799 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557165c0e960 con 0x557165ecff20 2024-09-11T22:23:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.038+0000 7f6095ffb640 1 -- 172.21.15.5:0/1723852799 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a80055b0 con 0x557165ecff20 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.039+0000 7f6095ffb640 1 -- 172.21.15.5:0/1723852799 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60a801e570 con 0x557165ecff20 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.039+0000 7f60b6795740 10 client.25306 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.039+0000 7f60b6795740 10 client.25306 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.039+0000 7f60b6795740 20 client.25306 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.039+0000 7f60b6795740 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557165ef36e0 0x557165ef5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.039+0000 7f60b6795740 4 client.25306 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.039+0000 7f60b6795740 1 -- 172.21.15.5:0/1723852799 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f60a80047a0 con 0x557165ef36e0 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.040+0000 7f60b1957640 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557165ef36e0 0x557165ef5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.040+0000 7f6095ffb640 10 client.25306 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.040+0000 7f60b1957640 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557165ef36e0 0x557165ef5ad0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=251d16609cdc82c3 server_cookie=b1a51c61d7431dbd in_seq=0 out_seq=0 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f6095ffb640 1 -- 172.21.15.5:0/1723852799 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f60a80047a0 con 0x557165ef36e0 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f6095ffb640 10 client.25306 handle_command_reply: tid=0 2024-09-11T22:23:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f60b6795740 1 client.25306 shutdown 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f60b6795740 1 -- 172.21.15.5:0/1723852799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 msgr2=0x557165edf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f60b6795740 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165edf080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f60a800b490 tx=0x7f60a8004e40 comp rx=0 tx=0).stop 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f60b6795740 1 -- 172.21.15.5:0/1723852799 shutdown_connections 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f60b6795740 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557165ef36e0 0x557165ef5ad0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f60b6795740 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165edf650 0x557165edcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f60b6795740 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ed0840 0x557165edc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f60b6795740 1 --2- 172.21.15.5:0/1723852799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ecff20 0x557165edf080 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f60b6795740 1 -- 172.21.15.5:0/1723852799 >> 172.21.15.5:0/1723852799 conn(0x557165ecc5f0 msgr2=0x557165ed2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.041+0000 7f6095ffb640 0 client.25306 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.042+0000 7f60b6795740 1 -- 172.21.15.5:0/1723852799 shutdown_connections 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.042+0000 7f60b6795740 1 -- 172.21.15.5:0/1723852799 wait complete. 2024-09-11T22:23:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.042+0000 7f60b6795740 20 client.25306 trim_cache size 0 max 16384 2024-09-11T22:23:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.075+0000 7f60b6795740 1 Processor -- start 2024-09-11T22:23:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.075+0000 7f60b6795740 1 -- start start 2024-09-11T22:23:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.075+0000 7f60b6795740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ecff20 0x557165ecc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.075+0000 7f60b6795740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 0x557165edee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.076+0000 7f60b6795740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ecb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.076+0000 7f60b2158640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ecff20 0x557165ecc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.076+0000 7f60b2158640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ecff20 0x557165ecc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59794/0 (socket says 172.21.15.5:59794) 2024-09-11T22:23:32.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.076+0000 7f60b1957640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 0x557165edee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.076+0000 7f60b1957640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 0x557165edee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:57110/0 (socket says 172.21.15.5:57110) 2024-09-11T22:23:32.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.076+0000 7f60b2959640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ecb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.076+0000 7f60b2959640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ecb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41392/0 (socket says 172.21.15.5:41392) 2024-09-11T22:23:32.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.076+0000 7f60b2158640 1 -- 172.21.15.5:0/2301185332 learned_addr learned my addr 172.21.15.5:0/2301185332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:32.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.077+0000 7f60b6795740 1 -- 172.21.15.5:0/2301185332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f609000e990 con 0x557165edf650 2024-09-11T22:23:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.077+0000 7f60b6795740 1 -- 172.21.15.5:0/2301185332 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60900104d0 con 0x557165ecff20 2024-09-11T22:23:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.077+0000 7f60b6795740 1 -- 172.21.15.5:0/2301185332 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f609000f060 con 0x557165ed0840 2024-09-11T22:23:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.077+0000 7f60b2959640 1 -- 172.21.15.5:0/2301185332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 msgr2=0x557165edee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.077+0000 7f60b2959640 1 --2- 172.21.15.5:0/2301185332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 0x557165edee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.077+0000 7f60b2959640 1 -- 172.21.15.5:0/2301185332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ecff20 msgr2=0x557165ecc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.077+0000 7f60b2959640 1 --2- 172.21.15.5:0/2301185332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ecff20 0x557165ecc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.077+0000 7f60b2959640 1 -- 172.21.15.5:0/2301185332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557165edcaa0 con 0x557165edf650 2024-09-11T22:23:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.078+0000 7f60b2959640 1 --2- 172.21.15.5:0/2301185332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ecb640 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f60a4005e30 tx=0x7f60a402f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca87328739f31041 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.078+0000 7f6097fff640 1 -- 172.21.15.5:0/2301185332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a404a070 con 0x557165edf650 2024-09-11T22:23:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.078+0000 7f6097fff640 1 -- 172.21.15.5:0/2301185332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a4036050 con 0x557165edf650 2024-09-11T22:23:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.078+0000 7f6097fff640 1 -- 172.21.15.5:0/2301185332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4004040 con 0x557165edf650 2024-09-11T22:23:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.079+0000 7f60b6795740 1 -- 172.21.15.5:0/2301185332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 msgr2=0x557165ecb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.079+0000 7f60b6795740 1 --2- 172.21.15.5:0/2301185332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ecb640 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f60a4005e30 tx=0x7f60a402f450 comp rx=0 tx=0).stop 2024-09-11T22:23:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.079+0000 7f60b6795740 1 -- 172.21.15.5:0/2301185332 shutdown_connections 2024-09-11T22:23:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.079+0000 7f60b6795740 1 --2- 172.21.15.5:0/2301185332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165edf650 0x557165ecb640 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.079+0000 7f60b6795740 1 --2- 172.21.15.5:0/2301185332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 0x557165edee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.079+0000 7f60b6795740 1 --2- 172.21.15.5:0/2301185332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165ecff20 0x557165ecc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.079+0000 7f60b6795740 1 -- 172.21.15.5:0/2301185332 >> 172.21.15.5:0/2301185332 conn(0x557165ecc5f0 msgr2=0x557165ecaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.080+0000 7f60b6795740 1 -- 172.21.15.5:0/2301185332 shutdown_connections 2024-09-11T22:23:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.080+0000 7f60b6795740 1 -- 172.21.15.5:0/2301185332 wait complete. 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.080+0000 7f60b6795740 1 -- start start 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.080+0000 7f60b6795740 1 Processor -- start 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b6795740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ecff20 0x557165eede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b6795740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 0x557165eee380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b6795740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165edf650 0x557165eee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b6795740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f609000fe00 con 0x557165ecff20 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b6795740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f609000f730 con 0x557165edf650 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b6795740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f609000c970 con 0x557165ed0840 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b6795740 10 client.0 _ll_register_callbacks cb 0x7f60a800ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b2158640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ecff20 0x557165eede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b2158640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ecff20 0x557165eede40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41394/0 (socket says 172.21.15.5:41394) 2024-09-11T22:23:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b2158640 1 -- 172.21.15.5:0/1942798323 learned_addr learned my addr 172.21.15.5:0/1942798323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.081+0000 7f60b1957640 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 0x557165eee380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.082+0000 7f60b2959640 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165edf650 0x557165eee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.082+0000 7f60b2158640 1 -- 172.21.15.5:0/1942798323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 msgr2=0x557165eee380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.082+0000 7f60b2158640 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 0x557165eee380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.082+0000 7f60b2158640 1 -- 172.21.15.5:0/1942798323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165edf650 msgr2=0x557165eee8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.082+0000 7f60b2158640 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165edf650 0x557165eee8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.082+0000 7f60b2158640 1 -- 172.21.15.5:0/1942798323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557165f5d050 con 0x557165ecff20 2024-09-11T22:23:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.083+0000 7f60b2158640 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ecff20 0x557165eede40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f60a800b490 tx=0x7f60a8004eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a0f92856091c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.083+0000 7f6095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.083+0000 7f6095ffb640 1 -- 172.21.15.5:0/1942798323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a8015040 con 0x557165ecff20 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.083+0000 7f6095ffb640 1 -- 172.21.15.5:0/1942798323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f60a802e070 con 0x557165ecff20 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.083+0000 7f60b6795740 1 -- 172.21.15.5:0/1942798323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60a4036050 con 0x557165ecff20 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.083+0000 7f6095ffb640 1 -- 172.21.15.5:0/1942798323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f60a8049050 con 0x557165ecff20 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.083+0000 7f60b6795740 10 client.15810 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.083+0000 7f60b6795740 1 -- 172.21.15.5:0/1942798323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557165f5e350 con 0x557165ecff20 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.083+0000 7f6095ffb640 1 -- 172.21.15.5:0/1942798323 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60a8041020 con 0x557165ecff20 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.084+0000 7f6095ffb640 1 -- 172.21.15.5:0/1942798323 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f60a803ccb0 con 0x557165ecff20 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.084+0000 7f60b6795740 10 client.15810 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.084+0000 7f60b6795740 10 client.15810 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:23:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.084+0000 7f60b6795740 20 client.15810 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.084+0000 7f60b6795740 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557165f5e980 0x557165f60da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.084+0000 7f60b6795740 4 client.15810 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7d67fffb", "release": true, "await": true}] 2024-09-11T22:23:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.084+0000 7f60b6795740 1 -- 172.21.15.5:0/1942798323 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7d67fffb", "release": true, "await": true}) -- 0x7f60a4036050 con 0x557165f5e980 2024-09-11T22:23:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.084+0000 7f60b1957640 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557165f5e980 0x557165f60da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.085+0000 7f60b1957640 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557165f5e980 0x557165f60da0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2abddac9821246b server_cookie=572e59b26b4da51f in_seq=0 out_seq=0 2024-09-11T22:23:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.085+0000 7f6095ffb640 10 client.15810 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f6095ffb640 1 -- 172.21.15.5:0/1942798323 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f60a4036050 con 0x557165f5e980 2024-09-11T22:23:32.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f6095ffb640 10 client.15810 handle_command_reply: tid=0 2024-09-11T22:23:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f60b6795740 1 client.15810 shutdown 2024-09-11T22:23:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f60b6795740 1 -- 172.21.15.5:0/1942798323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ecff20 msgr2=0x557165eede40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f60b6795740 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ecff20 0x557165eede40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f60a800b490 tx=0x7f60a8004eb0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f60b6795740 1 -- 172.21.15.5:0/1942798323 shutdown_connections 2024-09-11T22:23:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f60b6795740 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557165f5e980 0x557165f60da0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f60b6795740 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557165edf650 0x557165eee8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f60b6795740 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557165ed0840 0x557165eee380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f60b6795740 1 --2- 172.21.15.5:0/1942798323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557165ecff20 0x557165eede40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f60b6795740 1 -- 172.21.15.5:0/1942798323 >> 172.21.15.5:0/1942798323 conn(0x557165ecc5f0 msgr2=0x557165ecb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.087+0000 7f6095ffb640 0 client.15810 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.088+0000 7f60b6795740 1 -- 172.21.15.5:0/1942798323 shutdown_connections 2024-09-11T22:23:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.088+0000 7f60b6795740 1 -- 172.21.15.5:0/1942798323 wait complete. 2024-09-11T22:23:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.088+0000 7f60b6795740 20 client.15810 trim_cache size 0 max 16384 2024-09-11T22:23:32.089 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:23:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 28, 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 4, 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "7d67fffb": { 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4, 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.5, 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:23:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:23:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:23:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:23:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:23:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:23:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:23:32.091 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 -- 172.21.15.5:0/539664023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2f20 msgr2=0x7f60980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2f20 0x7f60980a53e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f60a8005fd0 tx=0x7f60a800b2c0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 -- 172.21.15.5:0/539664023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac1027a0 msgr2=0x7f60ac198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac1027a0 0x7f60ac198910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f609c00e9f0 tx=0x7f609c00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 -- 172.21.15.5:0/539664023 shutdown_connections 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2f20 0x7f60980a53e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac106f20 0x7f60ac19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac1027a0 0x7f60ac198910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 --2- 172.21.15.5:0/539664023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac101df0 0x7f60ac1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 -- 172.21.15.5:0/539664023 >> 172.21.15.5:0/539664023 conn(0x7f60ac0fd8f0 msgr2=0x7f60ac0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 -- 172.21.15.5:0/539664023 shutdown_connections 2024-09-11T22:23:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:32.089+0000 7f60b43bc640 1 -- 172.21.15.5:0/539664023 wait complete. 2024-09-11T22:23:33.000 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7d67fffb', total seconds elapsed: 28.7 2024-09-11T22:23:33.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:33 smithi005 ceph-mon[30943]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-11T22:23:33.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:33 smithi159 ceph-mon[35534]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-11T22:23:33.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:33 smithi172 ceph-mon[35461]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-11T22:23:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.610+0000 7f803e331640 1 -- 172.21.15.5:0/2156612397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8038103600 msgr2=0x7f803810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.610+0000 7f803e331640 1 --2- 172.21.15.5:0/2156612397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8038103600 0x7f803810bae0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f802c009900 tx=0x7f802c02a410 comp rx=0 tx=0).stop 2024-09-11T22:23:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.610+0000 7f803e331640 1 -- 172.21.15.5:0/2156612397 shutdown_connections 2024-09-11T22:23:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.610+0000 7f803e331640 1 --2- 172.21.15.5:0/2156612397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f803810c100 0x7f803810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.610+0000 7f803e331640 1 --2- 172.21.15.5:0/2156612397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8038103600 0x7f803810bae0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.610+0000 7f803e331640 1 --2- 172.21.15.5:0/2156612397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8038102ce0 0x7f80381030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.610+0000 7f803e331640 1 -- 172.21.15.5:0/2156612397 >> 172.21.15.5:0/2156612397 conn(0x7f80380fc6b0 msgr2=0x7f80380fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.610+0000 7f803e331640 1 -- 172.21.15.5:0/2156612397 shutdown_connections 2024-09-11T22:23:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.610+0000 7f803e331640 1 -- 172.21.15.5:0/2156612397 wait complete. 2024-09-11T22:23:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.611+0000 7f803e331640 1 Processor -- start 2024-09-11T22:23:33.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.611+0000 7f803e331640 1 -- start start 2024-09-11T22:23:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f803e331640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8038102ce0 0x7f8038198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f803e331640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8038103600 0x7f80381988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f803e331640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f803810c100 0x7f803819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f803e331640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8038111f20 con 0x7f8038102ce0 2024-09-11T22:23:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f803e331640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8038111da0 con 0x7f8038103600 2024-09-11T22:23:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f803e331640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f80381120a0 con 0x7f803810c100 2024-09-11T22:23:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f8037fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8038102ce0 0x7f8038198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f8037fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8038102ce0 0x7f8038198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41414/0 (socket says 172.21.15.5:41414) 2024-09-11T22:23:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f8037fff640 1 -- 172.21.15.5:0/1135878601 learned_addr learned my addr 172.21.15.5:0/1135878601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f80377fe640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8038103600 0x7f80381988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.612+0000 7f803c8ce640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f803810c100 0x7f803819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.613+0000 7f8037fff640 1 -- 172.21.15.5:0/1135878601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f803810c100 msgr2=0x7f803819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.613+0000 7f8037fff640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f803810c100 0x7f803819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.613+0000 7f8037fff640 1 -- 172.21.15.5:0/1135878601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8038103600 msgr2=0x7f80381988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.613+0000 7f8037fff640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8038103600 0x7f80381988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.613+0000 7f8037fff640 1 -- 172.21.15.5:0/1135878601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f803819d370 con 0x7f8038102ce0 2024-09-11T22:23:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.613+0000 7f803c8ce640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f803810c100 0x7f803819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.613+0000 7f80377fe640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8038103600 0x7f80381988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.613+0000 7f8037fff640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8038102ce0 0x7f8038198380 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f802800e9f0 tx=0x7f802800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.613+0000 7f80357fa640 1 -- 172.21.15.5:0/1135878601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802800cd80 con 0x7f8038102ce0 2024-09-11T22:23:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.613+0000 7f80357fa640 1 -- 172.21.15.5:0/1135878601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8028010910 con 0x7f8038102ce0 2024-09-11T22:23:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.614+0000 7f803e331640 1 -- 172.21.15.5:0/1135878601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f803819d660 con 0x7f8038102ce0 2024-09-11T22:23:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.614+0000 7f803e331640 1 -- 172.21.15.5:0/1135878601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80381a4ef0 con 0x7f8038102ce0 2024-09-11T22:23:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.614+0000 7f80357fa640 1 -- 172.21.15.5:0/1135878601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028018990 con 0x7f8038102ce0 2024-09-11T22:23:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.617+0000 7f80357fa640 1 -- 172.21.15.5:0/1135878601 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7f8028010ac0 con 0x7f8038102ce0 2024-09-11T22:23:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.617+0000 7f80410cd740 1 Processor -- start 2024-09-11T22:23:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.617+0000 7f80357fa640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80200a2f50 0x7f80200a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.618+0000 7f80357fa640 1 -- 172.21.15.5:0/1135878601 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f80280cfb50 con 0x7f8038102ce0 2024-09-11T22:23:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.618+0000 7f803c8ce640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80200a2f50 0x7f80200a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.618+0000 7f80410cd740 1 -- start start 2024-09-11T22:23:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.618+0000 7f803c8ce640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80200a2f50 0x7f80200a5410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8030009370 tx=0x7f8030009300 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f80410cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047daf0 0x560c7047ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f80410cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70488770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f80410cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c70488cb0 0x560c7048b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f80410cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c703e22f0 con 0x560c7047e410 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f80377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70488770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f80377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70488770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41428/0 (socket says 172.21.15.5:41428) 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f80377fe640 1 -- 172.21.15.5:0/1485647666 learned_addr learned my addr 172.21.15.5:0/1485647666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f8037fff640 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047daf0 0x560c7047ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f80410cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560c7038a270 con 0x560c7047daf0 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f80410cd740 1 -- 172.21.15.5:0/1485647666 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560c703de2b0 con 0x560c70488cb0 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.619+0000 7f803c8ce640 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c70488cb0 0x560c7048b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f80377fe640 1 -- 172.21.15.5:0/1485647666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c70488cb0 msgr2=0x560c7048b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f80377fe640 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c70488cb0 0x560c7048b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f80377fe640 1 -- 172.21.15.5:0/1485647666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047daf0 msgr2=0x560c7047ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f80377fe640 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047daf0 0x560c7047ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f80377fe640 1 -- 172.21.15.5:0/1485647666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c70389340 con 0x560c7047e410 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f8037fff640 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047daf0 0x560c7047ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f803c8ce640 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c70488cb0 0x560c7048b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f80377fe640 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70488770 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f80380697f0 tx=0x7f802c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a536c90acadacb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f801e7fc640 1 -- 172.21.15.5:0/1485647666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c035040 con 0x560c7047e410 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f801e7fc640 1 -- 172.21.15.5:0/1485647666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f802c004760 con 0x560c7047e410 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.620+0000 7f801e7fc640 1 -- 172.21.15.5:0/1485647666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c0039b0 con 0x560c7047e410 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.621+0000 7f80410cd740 1 -- 172.21.15.5:0/1485647666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 msgr2=0x560c70488770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.621+0000 7f80410cd740 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70488770 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f80380697f0 tx=0x7f802c000bc0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.621+0000 7f80410cd740 1 -- 172.21.15.5:0/1485647666 shutdown_connections 2024-09-11T22:23:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.622+0000 7f80410cd740 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c70488cb0 0x560c7048b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.622+0000 7f80410cd740 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70488770 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.622+0000 7f80410cd740 1 --2- 172.21.15.5:0/1485647666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047daf0 0x560c7047ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.622+0000 7f80410cd740 1 -- 172.21.15.5:0/1485647666 >> 172.21.15.5:0/1485647666 conn(0x560c7047a060 msgr2=0x560c7047a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.622+0000 7f80410cd740 1 -- 172.21.15.5:0/1485647666 shutdown_connections 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.622+0000 7f80410cd740 1 -- 172.21.15.5:0/1485647666 wait complete. 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.622+0000 7f80410cd740 1 -- start start 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.623+0000 7f80410cd740 1 Processor -- start 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.623+0000 7f80410cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 0x560c70485b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f80410cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70486050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f80410cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c70488cb0 0x560c70486590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f80410cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c7048b880 con 0x560c7047e410 2024-09-11T22:23:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f8037fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 0x560c70485b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f80377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70486050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f8037fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 0x560c70485b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:57172/0 (socket says 172.21.15.5:57172) 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f8037fff640 1 -- 172.21.15.5:0/149063594 learned_addr learned my addr 172.21.15.5:0/149063594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f80410cd740 1 -- 172.21.15.5:0/149063594 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560c7036b290 con 0x560c70488cb0 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f80410cd740 1 -- 172.21.15.5:0/149063594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560c7048ba00 con 0x560c7047daf0 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f803c8ce640 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c70488cb0 0x560c70486590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f80410cd740 10 client.0 _ll_register_callbacks cb 0x560c7039c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f8037fff640 1 -- 172.21.15.5:0/149063594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c70488cb0 msgr2=0x560c70486590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f8037fff640 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c70488cb0 0x560c70486590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f8037fff640 1 -- 172.21.15.5:0/149063594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 msgr2=0x560c70486050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f8037fff640 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70486050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f8037fff640 1 -- 172.21.15.5:0/149063594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c700f5210 con 0x560c7047daf0 2024-09-11T22:23:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.624+0000 7f80377fe640 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70486050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.625+0000 7f8037fff640 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 0x560c70485b10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f802809baf0 tx=0x7f802809c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=840cab0727305804 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.625+0000 7f80177fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:23:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.625+0000 7f80177fe640 1 -- 172.21.15.5:0/149063594 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028018d30 con 0x560c7047daf0 2024-09-11T22:23:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.625+0000 7f80410cd740 1 -- 172.21.15.5:0/149063594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f802c004760 con 0x560c7047daf0 2024-09-11T22:23:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.625+0000 7f80177fe640 1 -- 172.21.15.5:0/149063594 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f802809b240 con 0x560c7047daf0 2024-09-11T22:23:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.625+0000 7f80177fe640 1 -- 172.21.15.5:0/149063594 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f802809b440 con 0x560c7047daf0 2024-09-11T22:23:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.625+0000 7f80410cd740 10 client.25321 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.625+0000 7f80410cd740 1 -- 172.21.15.5:0/149063594 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c701c1620 con 0x560c7047daf0 2024-09-11T22:23:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.625+0000 7f80177fe640 1 -- 172.21.15.5:0/149063594 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f802c004760 con 0x560c7047daf0 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.625+0000 7f80177fe640 1 -- 172.21.15.5:0/149063594 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8028097c60 con 0x560c7047daf0 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.626+0000 7f80410cd740 10 client.25321 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.626+0000 7f80410cd740 10 client.25321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.626+0000 7f80410cd740 20 client.25321 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.626+0000 7f80410cd740 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560c70498b70 0x560c7049af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.626+0000 7f80410cd740 4 client.25321 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.626+0000 7f80410cd740 1 -- 172.21.15.5:0/149063594 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c703efa20 con 0x560c70498b70 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.626+0000 7f80377fe640 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560c70498b70 0x560c7049af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.627+0000 7f80177fe640 10 client.25321 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.627+0000 7f80377fe640 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560c70498b70 0x560c7049af60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e27fd3c6e4e2ee server_cookie=9c76ad41f4f9471d in_seq=0 out_seq=0 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.627+0000 7f80177fe640 1 -- 172.21.15.5:0/149063594 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c703efa20 con 0x560c70498b70 2024-09-11T22:23:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.627+0000 7f80177fe640 10 client.25321 handle_command_reply: tid=0 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.627+0000 7f80410cd740 1 client.25321 shutdown 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 1 -- 172.21.15.5:0/149063594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 msgr2=0x560c70485b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 0x560c70485b10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f802809baf0 tx=0x7f802809c660 comp rx=0 tx=0).stop 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 1 -- 172.21.15.5:0/149063594 shutdown_connections 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560c70498b70 0x560c7049af60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c70488cb0 0x560c70486590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047e410 0x560c70486050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 1 --2- 172.21.15.5:0/149063594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 0x560c70485b10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 1 -- 172.21.15.5:0/149063594 >> 172.21.15.5:0/149063594 conn(0x560c7047a060 msgr2=0x560c7047fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80177fe640 0 client.25321 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 1 -- 172.21.15.5:0/149063594 shutdown_connections 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 1 -- 172.21.15.5:0/149063594 wait complete. 2024-09-11T22:23:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.628+0000 7f80410cd740 20 client.25321 trim_cache size 0 max 16384 2024-09-11T22:23:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.642+0000 7f80410cd740 1 Processor -- start 2024-09-11T22:23:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.642+0000 7f80410cd740 1 -- start start 2024-09-11T22:23:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.642+0000 7f80410cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 0x560c704863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.642+0000 7f80410cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 0x560c704a0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80410cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c70488cb0 0x560c704a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80410cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f80381afc30 con 0x560c70488cb0 2024-09-11T22:23:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80410cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80381b1770 con 0x560c7047e410 2024-09-11T22:23:33.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80410cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f80381b0300 con 0x560c7047daf0 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80377fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 0x560c704a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80377fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 0x560c704a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59838/0 (socket says 172.21.15.5:59838) 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80377fe640 1 -- 172.21.15.5:0/989365002 learned_addr learned my addr 172.21.15.5:0/989365002 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f8037fff640 1 --2- 172.21.15.5:0/989365002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 0x560c704863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f803c8ce640 1 --2- 172.21.15.5:0/989365002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c70488cb0 0x560c704a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80377fe640 1 -- 172.21.15.5:0/989365002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 msgr2=0x560c704863c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80377fe640 1 --2- 172.21.15.5:0/989365002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 0x560c704863c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80377fe640 1 -- 172.21.15.5:0/989365002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c70488cb0 msgr2=0x560c704a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80377fe640 1 --2- 172.21.15.5:0/989365002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c70488cb0 0x560c704a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80377fe640 1 -- 172.21.15.5:0/989365002 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c70479e90 con 0x560c7047e410 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f803c8ce640 1 --2- 172.21.15.5:0/989365002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c70488cb0 0x560c704a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:33.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.643+0000 7f80377fe640 1 --2- 172.21.15.5:0/989365002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 0x560c704a0800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f802c005bc0 tx=0x7f802c031510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dec0b60880de6cef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.644+0000 7f801dffb640 1 -- 172.21.15.5:0/989365002 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c035050 con 0x560c7047e410 2024-09-11T22:23:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.644+0000 7f801dffb640 1 -- 172.21.15.5:0/989365002 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f802c004b50 con 0x560c7047e410 2024-09-11T22:23:33.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.644+0000 7f801dffb640 1 -- 172.21.15.5:0/989365002 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c0315b0 con 0x560c7047e410 2024-09-11T22:23:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.644+0000 7f80410cd740 1 -- 172.21.15.5:0/989365002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 msgr2=0x560c704a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.644+0000 7f80410cd740 1 --2- 172.21.15.5:0/989365002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 0x560c704a0800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f802c005bc0 tx=0x7f802c031510 comp rx=0 tx=0).stop 2024-09-11T22:23:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.644+0000 7f80410cd740 1 -- 172.21.15.5:0/989365002 shutdown_connections 2024-09-11T22:23:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.644+0000 7f80410cd740 1 --2- 172.21.15.5:0/989365002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c70488cb0 0x560c704a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.644+0000 7f80410cd740 1 --2- 172.21.15.5:0/989365002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 0x560c704a0800 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.644+0000 7f80410cd740 1 --2- 172.21.15.5:0/989365002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c7047daf0 0x560c704863c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.644+0000 7f80410cd740 1 -- 172.21.15.5:0/989365002 >> 172.21.15.5:0/989365002 conn(0x560c7047a060 msgr2=0x560c70487b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 1 -- 172.21.15.5:0/989365002 shutdown_connections 2024-09-11T22:23:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 1 -- 172.21.15.5:0/989365002 wait complete. 2024-09-11T22:23:33.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 1 -- start start 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 1 Processor -- start 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047daf0 0x560c704912b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 0x560c70482630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c70488cb0 0x560c70482b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f80381b10a0 con 0x560c7047daf0 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80381b09d0 con 0x560c7047e410 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f80381adc10 con 0x560c70488cb0 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f80410cd740 10 client.0 _ll_register_callbacks cb 0x7f80280980b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.645+0000 7f8037fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047daf0 0x560c704912b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f8037fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047daf0 0x560c704912b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41470/0 (socket says 172.21.15.5:41470) 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f8037fff640 1 -- 172.21.15.5:0/3898675908 learned_addr learned my addr 172.21.15.5:0/3898675908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f80377fe640 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 0x560c70482630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f8037fff640 1 -- 172.21.15.5:0/3898675908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c70488cb0 msgr2=0x560c70482b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f8037fff640 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c70488cb0 0x560c70482b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f8037fff640 1 -- 172.21.15.5:0/3898675908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 msgr2=0x560c70482630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f8037fff640 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 0x560c70482630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f8037fff640 1 -- 172.21.15.5:0/3898675908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c704752d0 con 0x560c7047daf0 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f8037fff640 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047daf0 0x560c704912b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f802809bfb0 tx=0x7f8028018d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b568e2b9204bde server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f80177fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f80177fe640 1 -- 172.21.15.5:0/3898675908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802809b750 con 0x560c7047daf0 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f80177fe640 1 -- 172.21.15.5:0/3898675908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f8028010640 con 0x560c7047daf0 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.646+0000 7f80410cd740 1 -- 172.21.15.5:0/3898675908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560c703e9d70 con 0x560c7047daf0 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80177fe640 1 -- 172.21.15.5:0/3898675908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f802809ae00 con 0x560c7047daf0 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80410cd740 10 client.15825 fetch_fsmap learned FSMap version 28 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80410cd740 1 -- 172.21.15.5:0/3898675908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560c70497760 con 0x560c7047daf0 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80177fe640 1 -- 172.21.15.5:0/3898675908 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560c703e9d70 con 0x560c7047daf0 2024-09-11T22:23:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80177fe640 1 -- 172.21.15.5:0/3898675908 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f80280cd330 con 0x560c7047daf0 2024-09-11T22:23:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80410cd740 10 client.15825 fetch_fsmap finished waiting for FSMap version 28 2024-09-11T22:23:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80410cd740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80410cd740 20 client.15825 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80410cd740 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560c705039b0 0x560c70505df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80410cd740 4 client.15825 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:23:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80410cd740 1 -- 172.21.15.5:0/3898675908 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c70366900 con 0x560c705039b0 2024-09-11T22:23:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.647+0000 7f80377fe640 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560c705039b0 0x560c70505df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.648+0000 7f80177fe640 10 client.15825 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.648+0000 7f80377fe640 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560c705039b0 0x560c70505df0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10accf233ff35d85 server_cookie=3dc9fd95b927d20a in_seq=0 out_seq=0 2024-09-11T22:23:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80177fe640 1 -- 172.21.15.5:0/3898675908 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c70366900 con 0x560c705039b0 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80177fe640 10 client.15825 handle_command_reply: tid=0 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 client.15825 shutdown 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 -- 172.21.15.5:0/3898675908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047daf0 msgr2=0x560c704912b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047daf0 0x560c704912b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f802809bfb0 tx=0x7f8028018d60 comp rx=0 tx=0).stop 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 -- 172.21.15.5:0/3898675908 shutdown_connections 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560c705039b0 0x560c70505df0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80177fe640 0 client.15825 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c70488cb0 0x560c70482b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c7047e410 0x560c70482630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 --2- 172.21.15.5:0/3898675908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c7047daf0 0x560c704912b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 -- 172.21.15.5:0/3898675908 >> 172.21.15.5:0/3898675908 conn(0x560c7047a060 msgr2=0x560c7047f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 -- 172.21.15.5:0/3898675908 shutdown_connections 2024-09-11T22:23:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 1 -- 172.21.15.5:0/3898675908 wait complete. 2024-09-11T22:23:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.652+0000 7f80410cd740 20 client.15825 trim_cache size 0 max 16384 2024-09-11T22:23:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:23:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:23:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:23:33.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:23:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 -- 172.21.15.5:0/1135878601 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80200a2f50 msgr2=0x7f80200a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80200a2f50 0x7f80200a5410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8030009370 tx=0x7f8030009300 comp rx=0 tx=0).stop 2024-09-11T22:23:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 -- 172.21.15.5:0/1135878601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8038102ce0 msgr2=0x7f8038198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8038102ce0 0x7f8038198380 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f802800e9f0 tx=0x7f802800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 -- 172.21.15.5:0/1135878601 shutdown_connections 2024-09-11T22:23:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f803810c100 0x7f803819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80200a2f50 0x7f80200a5410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8038103600 0x7f80381988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 --2- 172.21.15.5:0/1135878601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8038102ce0 0x7f8038198380 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 -- 172.21.15.5:0/1135878601 >> 172.21.15.5:0/1135878601 conn(0x7f80380fc6b0 msgr2=0x7f80380fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 -- 172.21.15.5:0/1135878601 shutdown_connections 2024-09-11T22:23:33.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:33.653+0000 7f803e331640 1 -- 172.21.15.5:0/1135878601 wait complete. 2024-09-11T22:23:35.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:35 smithi005 ceph-mon[30943]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T22:23:35.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:35 smithi159 ceph-mon[35534]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T22:23:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:35 smithi172 ceph-mon[35461]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 10 op/s 2024-09-11T22:23:37.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:37 smithi005 ceph-mon[30943]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:23:37.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:37 smithi159 ceph-mon[35534]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:23:37.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:37 smithi172 ceph-mon[35461]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-11T22:23:39.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:39 smithi005 ceph-mon[30943]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:23:39.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:39 smithi159 ceph-mon[35534]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:23:39.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:39 smithi172 ceph-mon[35461]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:23:40.371 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 94% (12139/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-11T22:23:41.057 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-11T22:23:41.057 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-11T22:23:41.057 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T22:23:41.057 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T22:23:41.057 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T22:23:41.057 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-11T22:23:41.057 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-11T22:23:41.058 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-11T22:23:41.063 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T22:23:41.228 INFO:tasks.workunit.client.0.smithi005.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T22:23:41.230 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T22:23:41.230 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T22:23:41.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:41 smithi005 ceph-mon[30943]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:23:41.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:41 smithi159 ceph-mon[35534]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:23:41.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:41 smithi172 ceph-mon[35461]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:23:41.936 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T22:23:42.021 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T22:23:42.062 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T22:23:42.063 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T22:23:42.063 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T22:23:42.103 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T22:23:42.106 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:23:42.107 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T22:23:42.123 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-11T22:23:42.124 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-11T22:23:42.124 DEBUG:teuthology.orchestra.run.smithi005:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-11T22:23:42.189 INFO:tasks.workunit.client.0.smithi005.stderr:+ dbench 1 2024-09-11T22:23:42.198 INFO:tasks.workunit.client.0.smithi005.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-11T22:23:42.198 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-11T22:23:42.198 INFO:tasks.workunit.client.0.smithi005.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-11T22:23:42.202 INFO:tasks.workunit.client.0.smithi005.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-11T22:23:42.303 INFO:tasks.workunit.client.0.smithi005.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-11T22:23:42.303 INFO:tasks.workunit.client.0.smithi005.stdout:releasing clients 2024-09-11T22:23:43.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 3627 38.60 MB/sec warmup 1 sec latency 6.579 ms 2024-09-11T22:23:43.470 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:43 smithi005 ceph-mon[30943]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:23:43.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:43 smithi159 ceph-mon[35534]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:23:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:43 smithi172 ceph-mon[35461]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-11T22:23:44.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 6457 25.97 MB/sec warmup 2 sec latency 12.951 ms 2024-09-11T22:23:45.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 9504 22.07 MB/sec warmup 3 sec latency 9.926 ms 2024-09-11T22:23:45.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:45 smithi005 ceph-mon[30943]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T22:23:45.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:45 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:23:45.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:45 smithi159 ceph-mon[35534]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T22:23:45.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:45 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:23:45.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:45 smithi172 ceph-mon[35461]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-11T22:23:45.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:45 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:23:46.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 12074 20.84 MB/sec warmup 4 sec latency 10.254 ms 2024-09-11T22:23:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.745+0000 7fdad5c53640 1 -- 172.21.15.5:0/1675863642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdad0102470 msgr2=0x7fdad010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.745+0000 7fdad5c53640 1 --2- 172.21.15.5:0/1675863642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdad0102470 0x7fdad010e9a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009cd0 tx=0x7fdac802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.745+0000 7fdad5c53640 1 -- 172.21.15.5:0/1675863642 shutdown_connections 2024-09-11T22:23:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.745+0000 7fdad5c53640 1 --2- 172.21.15.5:0/1675863642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdad0102470 0x7fdad010e9a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.745+0000 7fdad5c53640 1 --2- 172.21.15.5:0/1675863642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0101ad0 0x7fdad0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.745+0000 7fdad5c53640 1 --2- 172.21.15.5:0/1675863642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdad0107b70 0x7fdad0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.745+0000 7fdad5c53640 1 -- 172.21.15.5:0/1675863642 >> 172.21.15.5:0/1675863642 conn(0x7fdad00fd7d0 msgr2=0x7fdad00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.745+0000 7fdad5c53640 1 -- 172.21.15.5:0/1675863642 shutdown_connections 2024-09-11T22:23:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.745+0000 7fdad5c53640 1 -- 172.21.15.5:0/1675863642 wait complete. 2024-09-11T22:23:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.746+0000 7fdad5c53640 1 Processor -- start 2024-09-11T22:23:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.746+0000 7fdad5c53640 1 -- start start 2024-09-11T22:23:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.746+0000 7fdad5c53640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdad0101ad0 0x7fdad01983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.746+0000 7fdad5c53640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0102470 0x7fdad01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.746+0000 7fdad5c53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdad0107b70 0x7fdad019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.746+0000 7fdad5c53640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0111f20 con 0x7fdad0107b70 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.746+0000 7fdad5c53640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0111da0 con 0x7fdad0101ad0 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.746+0000 7fdad5c53640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdad01120a0 con 0x7fdad0102470 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.746+0000 7fdacffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdad0107b70 0x7fdad019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdacf7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdad0101ad0 0x7fdad01983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdaceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0102470 0x7fdad01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdacf7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdad0101ad0 0x7fdad01983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54578/0 (socket says 172.21.15.5:54578) 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdacf7fe640 1 -- 172.21.15.5:0/2937551307 learned_addr learned my addr 172.21.15.5:0/2937551307 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdacffff640 1 -- 172.21.15.5:0/2937551307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0102470 msgr2=0x7fdad01988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdacffff640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0102470 0x7fdad01988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdacffff640 1 -- 172.21.15.5:0/2937551307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdad0101ad0 msgr2=0x7fdad01983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdacffff640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdad0101ad0 0x7fdad01983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdacffff640 1 -- 172.21.15.5:0/2937551307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad019d3a0 con 0x7fdad0107b70 2024-09-11T22:23:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdad0101ad0 0x7fdad01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdacffff640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdad0107b70 0x7fdad019cc70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdac802ae00 tx=0x7fdac8031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdaccff9640 1 -- 172.21.15.5:0/2937551307 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8004040 con 0x7fdad0107b70 2024-09-11T22:23:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdaccff9640 1 -- 172.21.15.5:0/2937551307 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac80046c0 con 0x7fdad0107b70 2024-09-11T22:23:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdad5c53640 1 -- 172.21.15.5:0/2937551307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad019d630 con 0x7fdad0107b70 2024-09-11T22:23:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdaccff9640 1 -- 172.21.15.5:0/2937551307 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8030e40 con 0x7fdad0107b70 2024-09-11T22:23:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.747+0000 7fdad5c53640 1 -- 172.21.15.5:0/2937551307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad01a4f20 con 0x7fdad0107b70 2024-09-11T22:23:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.750+0000 7fdaccff9640 1 -- 172.21.15.5:0/2937551307 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138116+0+0 (secure 0 0 0) 0x7fdac80305a0 con 0x7fdad0107b70 2024-09-11T22:23:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.750+0000 7fdad89ea740 1 Processor -- start 2024-09-11T22:23:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.750+0000 7fdad89ea740 1 -- start start 2024-09-11T22:23:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 0x55de49628ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49633770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49633cb0 0x55de496360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdad89ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55de4958d2f0 con 0x55de49629410 2024-09-11T22:23:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdad89ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55de49535270 con 0x55de49628af0 2024-09-11T22:23:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdad89ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55de495892b0 con 0x55de49633cb0 2024-09-11T22:23:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdacf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49633770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdaceffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 0x55de49628ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdacf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49633770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37770/0 (socket says 172.21.15.5:37770) 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdacf7fe640 1 -- 172.21.15.5:0/752822521 learned_addr learned my addr 172.21.15.5:0/752822521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdaceffd640 1 --2- 172.21.15.5:0/752822521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49633cb0 0x55de496360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdaccff9640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab80a2f50 0x7fdab80a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdacffff640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab80a2f50 0x7fdab80a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdacf7fe640 1 -- 172.21.15.5:0/752822521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49633cb0 msgr2=0x55de496360a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/752822521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49633cb0 0x55de496360a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdacf7fe640 1 -- 172.21.15.5:0/752822521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 msgr2=0x55de49628ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/752822521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 0x55de49628ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdacf7fe640 1 -- 172.21.15.5:0/752822521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de49534340 con 0x55de49629410 2024-09-11T22:23:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdaceffd640 1 --2- 172.21.15.5:0/752822521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 0x55de49628ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdaccff9640 1 -- 172.21.15.5:0/2937551307 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fdac802e030 con 0x7fdad0107b70 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/752822521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49633770 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdac00115c0 tx=0x7fdac0011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bc88d9da89e085 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdab5ffb640 1 -- 172.21.15.5:0/752822521 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac000f660 con 0x55de49629410 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdab5ffb640 1 -- 172.21.15.5:0/752822521 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac000fce0 con 0x55de49629410 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.751+0000 7fdab5ffb640 1 -- 172.21.15.5:0/752822521 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0019de0 con 0x55de49629410 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.752+0000 7fdad89ea740 1 -- 172.21.15.5:0/752822521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 msgr2=0x55de49633770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.752+0000 7fdad89ea740 1 --2- 172.21.15.5:0/752822521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49633770 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdac00115c0 tx=0x7fdac0011a80 comp rx=0 tx=0).stop 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.752+0000 7fdacffff640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab80a2f50 0x7fdab80a5410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdac80ee2e0 tx=0x7fdac8088120 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.752+0000 7fdad89ea740 1 -- 172.21.15.5:0/752822521 shutdown_connections 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.752+0000 7fdad89ea740 1 --2- 172.21.15.5:0/752822521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49633cb0 0x55de496360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.752+0000 7fdad89ea740 1 --2- 172.21.15.5:0/752822521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49633770 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.752+0000 7fdad89ea740 1 --2- 172.21.15.5:0/752822521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 0x55de49628ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.752+0000 7fdad89ea740 1 -- 172.21.15.5:0/752822521 >> 172.21.15.5:0/752822521 conn(0x55de49625060 msgr2=0x55de49625440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.752+0000 7fdad89ea740 1 -- 172.21.15.5:0/752822521 shutdown_connections 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.752+0000 7fdad89ea740 1 -- 172.21.15.5:0/752822521 wait complete. 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdad89ea740 1 -- start start 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdad89ea740 1 Processor -- start 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49628af0 0x55de49630be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49629410 0x55de49631120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49633cb0 0x55de49631660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdad89ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55de49636880 con 0x55de49633cb0 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdad89ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55de49516290 con 0x55de49629410 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdad89ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55de49636a00 con 0x55de49628af0 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdaceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49628af0 0x55de49630be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdad89ea740 10 client.0 _ll_register_callbacks cb 0x55de49547140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdaceffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49628af0 0x55de49630be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45530/0 (socket says 172.21.15.5:45530) 2024-09-11T22:23:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdaceffd640 1 -- 172.21.15.5:0/2152315996 learned_addr learned my addr 172.21.15.5:0/2152315996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49629410 0x55de49631120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdaceffd640 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49633cb0 0x55de49631660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdaceffd640 1 -- 172.21.15.5:0/2152315996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49629410 msgr2=0x55de49631120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdaceffd640 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49629410 0x55de49631120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdaceffd640 1 -- 172.21.15.5:0/2152315996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49633cb0 msgr2=0x55de49631660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.753+0000 7fdaceffd640 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49633cb0 0x55de49631660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.754+0000 7fdaceffd640 1 -- 172.21.15.5:0/2152315996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de492a0210 con 0x55de49628af0 2024-09-11T22:23:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.754+0000 7fdaceffd640 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49628af0 0x55de49630be0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdac4012d50 tx=0x7fdac4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5d3b5e745740c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.754+0000 7fdaaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.754+0000 7fdaaeffd640 1 -- 172.21.15.5:0/2152315996 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4004250 con 0x55de49628af0 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.754+0000 7fdaaeffd640 1 -- 172.21.15.5:0/2152315996 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fdac40043f0 con 0x55de49628af0 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.754+0000 7fdad89ea740 1 -- 172.21.15.5:0/2152315996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac000fce0 con 0x55de49628af0 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.754+0000 7fdaaeffd640 1 -- 172.21.15.5:0/2152315996 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdac4029050 con 0x55de49628af0 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.754+0000 7fdad89ea740 10 client.25336 fetch_fsmap learned FSMap version 29 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.754+0000 7fdad89ea740 1 -- 172.21.15.5:0/2152315996 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55de4936c620 con 0x55de49628af0 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.754+0000 7fdaaeffd640 1 -- 172.21.15.5:0/2152315996 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac000fce0 con 0x55de49628af0 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.755+0000 7fdaaeffd640 1 -- 172.21.15.5:0/2152315996 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fdac4024030 con 0x55de49628af0 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.755+0000 7fdad89ea740 10 client.25336 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.755+0000 7fdad89ea740 10 client.25336 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.755+0000 7fdad89ea740 20 client.25336 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.755+0000 7fdad89ea740 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55de49643cd0 0x55de496460c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.755+0000 7fdad89ea740 4 client.25336 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.755+0000 7fdad89ea740 1 -- 172.21.15.5:0/2152315996 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de4959aa20 con 0x55de49643cd0 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.755+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55de49643cd0 0x55de496460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.755+0000 7fdaaeffd640 10 client.25336 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.755+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55de49643cd0 0x55de496460c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf6bbffb7876adf server_cookie=3ce9d9c8460b4f3d in_seq=0 out_seq=0 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdaaeffd640 1 -- 172.21.15.5:0/2152315996 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de4959aa20 con 0x55de49643cd0 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdaaeffd640 10 client.25336 handle_command_reply: tid=0 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 client.25336 shutdown 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 -- 172.21.15.5:0/2152315996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49628af0 msgr2=0x55de49630be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49628af0 0x55de49630be0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdac4012d50 tx=0x7fdac4004730 comp rx=0 tx=0).stop 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 -- 172.21.15.5:0/2152315996 shutdown_connections 2024-09-11T22:23:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55de49643cd0 0x55de496460c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49633cb0 0x55de49631660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdaaeffd640 0 client.25336 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49629410 0x55de49631120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 --2- 172.21.15.5:0/2152315996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49628af0 0x55de49630be0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 -- 172.21.15.5:0/2152315996 >> 172.21.15.5:0/2152315996 conn(0x55de49625060 msgr2=0x55de4962adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 -- 172.21.15.5:0/2152315996 shutdown_connections 2024-09-11T22:23:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 1 -- 172.21.15.5:0/2152315996 wait complete. 2024-09-11T22:23:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.756+0000 7fdad89ea740 20 client.25336 trim_cache size 0 max 16384 2024-09-11T22:23:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.770+0000 7fdad89ea740 1 Processor -- start 2024-09-11T22:23:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdad89ea740 1 -- start start 2024-09-11T22:23:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 0x55de4962b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49631490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49633cb0 0x55de4964b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdad89ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdad01ad8f0 con 0x55de49629410 2024-09-11T22:23:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdad89ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdad01af460 con 0x55de49628af0 2024-09-11T22:23:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdad89ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdad01adfc0 con 0x55de49633cb0 2024-09-11T22:23:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdacf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49631490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdacf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49631490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37782/0 (socket says 172.21.15.5:37782) 2024-09-11T22:23:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdacf7fe640 1 -- 172.21.15.5:0/716973051 learned_addr learned my addr 172.21.15.5:0/716973051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdaceffd640 1 --2- 172.21.15.5:0/716973051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 0x55de4962b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdaceffd640 1 --2- 172.21.15.5:0/716973051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49633cb0 0x55de4964b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdacf7fe640 1 -- 172.21.15.5:0/716973051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49633cb0 msgr2=0x55de4964b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/716973051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49633cb0 0x55de4964b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdacf7fe640 1 -- 172.21.15.5:0/716973051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 msgr2=0x55de4962b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/716973051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 0x55de4962b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdacf7fe640 1 -- 172.21.15.5:0/716973051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de49646430 con 0x55de49629410 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/716973051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49631490 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdac00115c0 tx=0x7fdac00195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed53caa17aecfcc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdab57fa640 1 -- 172.21.15.5:0/716973051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac0019ad0 con 0x55de49629410 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.771+0000 7fdab57fa640 1 -- 172.21.15.5:0/716973051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac0005510 con 0x55de49629410 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdab57fa640 1 -- 172.21.15.5:0/716973051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac000f9e0 con 0x55de49629410 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdad89ea740 1 -- 172.21.15.5:0/716973051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 msgr2=0x55de49631490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdad89ea740 1 --2- 172.21.15.5:0/716973051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49631490 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdac00115c0 tx=0x7fdac00195a0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdad89ea740 1 -- 172.21.15.5:0/716973051 shutdown_connections 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdad89ea740 1 --2- 172.21.15.5:0/716973051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49633cb0 0x55de4964b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdad89ea740 1 --2- 172.21.15.5:0/716973051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49629410 0x55de49631490 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdad89ea740 1 --2- 172.21.15.5:0/716973051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49628af0 0x55de4962b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdad89ea740 1 -- 172.21.15.5:0/716973051 >> 172.21.15.5:0/716973051 conn(0x55de49625060 msgr2=0x55de49632c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdad89ea740 1 -- 172.21.15.5:0/716973051 shutdown_connections 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdad89ea740 1 -- 172.21.15.5:0/716973051 wait complete. 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.772+0000 7fdad89ea740 1 -- start start 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdad89ea740 1 Processor -- start 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49628af0 0x55de4963beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49629410 0x55de4963c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdad89ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49633cb0 0x55de4962d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdad89ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdad01aed60 con 0x55de49628af0 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdad89ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdad01ae690 con 0x55de49633cb0 2024-09-11T22:23:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdad89ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdad01ab8d0 con 0x55de49629410 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdad89ea740 10 client.0 _ll_register_callbacks cb 0x7fdac4012990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49628af0 0x55de4963beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49628af0 0x55de4963beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37788/0 (socket says 172.21.15.5:37788) 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaceffd640 1 -- 172.21.15.5:0/1443923039 learned_addr learned my addr 172.21.15.5:0/1443923039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49629410 0x55de4963c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaceffd640 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49633cb0 0x55de4962d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaceffd640 1 -- 172.21.15.5:0/1443923039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49629410 msgr2=0x55de4963c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaceffd640 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49629410 0x55de4963c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaceffd640 1 -- 172.21.15.5:0/1443923039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49633cb0 msgr2=0x55de4962d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaceffd640 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49633cb0 0x55de4962d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaceffd640 1 -- 172.21.15.5:0/1443923039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de49629240 con 0x55de49628af0 2024-09-11T22:23:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaceffd640 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49628af0 0x55de4963beb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdac4014010 tx=0x7fdac4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ea824ff47c6c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.773+0000 7fdaaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdaaeffd640 1 -- 172.21.15.5:0/1443923039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4010900 con 0x55de49628af0 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdaaeffd640 1 -- 172.21.15.5:0/1443923039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fdac4012110 con 0x55de49628af0 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdad89ea740 1 -- 172.21.15.5:0/1443923039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdac0005510 con 0x55de49628af0 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdaaeffd640 1 -- 172.21.15.5:0/1443923039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdac4004030 con 0x55de49628af0 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdad89ea740 10 client.15843 fetch_fsmap learned FSMap version 29 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdad89ea740 1 -- 172.21.15.5:0/1443923039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55de496b2660 con 0x55de49628af0 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdaaeffd640 1 -- 172.21.15.5:0/1443923039 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac0005510 con 0x55de49628af0 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdaaeffd640 1 -- 172.21.15.5:0/1443923039 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fdac4024070 con 0x55de49628af0 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdad89ea740 10 client.15843 fetch_fsmap finished waiting for FSMap version 29 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdad89ea740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdad89ea740 20 client.15843 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdad89ea740 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55de496b2d00 0x55de496b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdad89ea740 4 client.15843 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:23:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdad89ea740 1 -- 172.21.15.5:0/1443923039 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de49594d70 con 0x55de496b2d00 2024-09-11T22:23:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.774+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55de496b2d00 0x55de496b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.775+0000 7fdaaeffd640 10 client.15843 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.775+0000 7fdacf7fe640 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55de496b2d00 0x55de496b5120 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7be9925c576b6b96 server_cookie=2d9e49baaf7c7180 in_seq=0 out_seq=0 2024-09-11T22:23:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.848+0000 7fdaaeffd640 1 -- 172.21.15.5:0/1443923039 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de49594d70 con 0x55de496b2d00 2024-09-11T22:23:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.848+0000 7fdaaeffd640 10 client.15843 handle_command_reply: tid=0 2024-09-11T22:23:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 client.15843 shutdown 2024-09-11T22:23:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 -- 172.21.15.5:0/1443923039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49628af0 msgr2=0x55de4963beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49628af0 0x55de4963beb0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdac4014010 tx=0x7fdac4014040 comp rx=0 tx=0).stop 2024-09-11T22:23:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 -- 172.21.15.5:0/1443923039 shutdown_connections 2024-09-11T22:23:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55de496b2d00 0x55de496b5120 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de49633cb0 0x55de4962d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55de49629410 0x55de4963c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 --2- 172.21.15.5:0/1443923039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de49628af0 0x55de4963beb0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdaaeffd640 0 client.15843 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 -- 172.21.15.5:0/1443923039 >> 172.21.15.5:0/1443923039 conn(0x55de49625060 msgr2=0x55de496239a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 -- 172.21.15.5:0/1443923039 shutdown_connections 2024-09-11T22:23:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 1 -- 172.21.15.5:0/1443923039 wait complete. 2024-09-11T22:23:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.849+0000 7fdad89ea740 20 client.15843 trim_cache size 0 max 16384 2024-09-11T22:23:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:23:46.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:23:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:23:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:23:46.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 -- 172.21.15.5:0/2937551307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab80a2f50 msgr2=0x7fdab80a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab80a2f50 0x7fdab80a5410 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdac80ee2e0 tx=0x7fdac8088120 comp rx=0 tx=0).stop 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 -- 172.21.15.5:0/2937551307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdad0107b70 msgr2=0x7fdad019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdad0107b70 0x7fdad019cc70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdac802ae00 tx=0x7fdac8031e60 comp rx=0 tx=0).stop 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 -- 172.21.15.5:0/2937551307 shutdown_connections 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdad0107b70 0x7fdad019cc70 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdad0102470 0x7fdad01988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab80a2f50 0x7fdab80a5410 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 --2- 172.21.15.5:0/2937551307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdad0101ad0 0x7fdad01983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 -- 172.21.15.5:0/2937551307 >> 172.21.15.5:0/2937551307 conn(0x7fdad00fd7d0 msgr2=0x7fdad00fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 -- 172.21.15.5:0/2937551307 shutdown_connections 2024-09-11T22:23:46.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:46.850+0000 7fdad5c53640 1 -- 172.21.15.5:0/2937551307 wait complete. 2024-09-11T22:23:47.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14789 19.49 MB/sec warmup 5 sec latency 25.924 ms 2024-09-11T22:23:47.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:47 smithi005 ceph-mon[30943]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:23:47.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:47 smithi159 ceph-mon[35534]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:23:47.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:47 smithi172 ceph-mon[35461]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-11T22:23:48.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 18042 19.37 MB/sec warmup 6 sec latency 34.041 ms 2024-09-11T22:23:49.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 21144 18.73 MB/sec warmup 7 sec latency 15.338 ms 2024-09-11T22:23:49.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:23:49.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:49 smithi005 ceph-mon[30943]: pgmap v51: 97 pgs: 97 active+clean; 8.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 725 KiB/s wr, 169 op/s 2024-09-11T22:23:49.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:49 smithi005 ceph-mon[30943]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-11T22:23:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:49 smithi159 ceph-mon[35534]: pgmap v51: 97 pgs: 97 active+clean; 8.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 725 KiB/s wr, 169 op/s 2024-09-11T22:23:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:49 smithi159 ceph-mon[35534]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-11T22:23:49.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:49 smithi172 ceph-mon[35461]: pgmap v51: 97 pgs: 97 active+clean; 8.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 725 KiB/s wr, 169 op/s 2024-09-11T22:23:49.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:49 smithi172 ceph-mon[35461]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-09-11T22:23:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:50.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 24038 18.34 MB/sec warmup 8 sec latency 11.187 ms 2024-09-11T22:23:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.303+0000 7f07c0718640 1 -- 172.21.15.5:0/2393529336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b8102ef0 msgr2=0x7f07b810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.303+0000 7f07c0718640 1 --2- 172.21.15.5:0/2393529336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b8102ef0 0x7f07b810da40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009ca0 tx=0x7f07a802a950 comp rx=0 tx=0).stop 2024-09-11T22:23:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.303+0000 7f07c0718640 1 -- 172.21.15.5:0/2393529336 shutdown_connections 2024-09-11T22:23:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.303+0000 7f07c0718640 1 --2- 172.21.15.5:0/2393529336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b8102ef0 0x7f07b810da40 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.303+0000 7f07c0718640 1 --2- 172.21.15.5:0/2393529336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102570 0x7f07b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.303+0000 7f07c0718640 1 --2- 172.21.15.5:0/2393529336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07b8100e20 0x7f07b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.303+0000 7f07c0718640 1 -- 172.21.15.5:0/2393529336 >> 172.21.15.5:0/2393529336 conn(0x7f07b80fc7d0 msgr2=0x7f07b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.303+0000 7f07c0718640 1 -- 172.21.15.5:0/2393529336 shutdown_connections 2024-09-11T22:23:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.303+0000 7f07c0718640 1 -- 172.21.15.5:0/2393529336 wait complete. 2024-09-11T22:23:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07c0718640 1 Processor -- start 2024-09-11T22:23:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07c0718640 1 -- start start 2024-09-11T22:23:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07c0718640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b8100e20 0x7f07b819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07c0718640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102570 0x7f07b819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07c0718640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07b8102ef0 0x7f07b819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07c0718640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8110f80 con 0x7f07b8100e20 2024-09-11T22:23:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07c0718640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8110e00 con 0x7f07b8102570 2024-09-11T22:23:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07c0718640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8111100 con 0x7f07b8102ef0 2024-09-11T22:23:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07be4b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b8100e20 0x7f07b819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07be4b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b8100e20 0x7f07b819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37812/0 (socket says 172.21.15.5:37812) 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07be4b4640 1 -- 172.21.15.5:0/1765890175 learned_addr learned my addr 172.21.15.5:0/1765890175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102570 0x7f07b819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07becb5640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07b8102ef0 0x7f07b819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07be4b4640 1 -- 172.21.15.5:0/1765890175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07b8102ef0 msgr2=0x7f07b819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07be4b4640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07b8102ef0 0x7f07b819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07be4b4640 1 -- 172.21.15.5:0/1765890175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102570 msgr2=0x7f07b819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07be4b4640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102570 0x7f07b819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07be4b4640 1 -- 172.21.15.5:0/1765890175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b819f690 con 0x7f07b8100e20 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.304+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102570 0x7f07b819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.305+0000 7f07be4b4640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b8100e20 0x7f07b819a6e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f07b000e9f0 tx=0x7f07b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.305+0000 7f07b77fe640 1 -- 172.21.15.5:0/1765890175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b000cd80 con 0x7f07b8100e20 2024-09-11T22:23:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.305+0000 7f07c0718640 1 -- 172.21.15.5:0/1765890175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b819f980 con 0x7f07b8100e20 2024-09-11T22:23:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.305+0000 7f07b77fe640 1 -- 172.21.15.5:0/1765890175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b0005500 con 0x7f07b8100e20 2024-09-11T22:23:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.305+0000 7f07b77fe640 1 -- 172.21.15.5:0/1765890175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0010690 con 0x7f07b8100e20 2024-09-11T22:23:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.305+0000 7f07c0718640 1 -- 172.21.15.5:0/1765890175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b81a7260 con 0x7f07b8100e20 2024-09-11T22:23:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.307+0000 7f07b77fe640 1 -- 172.21.15.5:0/1765890175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f07b00046c0 con 0x7f07b8100e20 2024-09-11T22:23:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.307+0000 7f07c2af1740 1 Processor -- start 2024-09-11T22:23:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07c2af1740 1 -- start start 2024-09-11T22:23:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 0x55ff11f41880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f4c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f4c680 0x55ff11f4ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07c2af1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff11ea72e0 con 0x55ff11f4c680 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07c2af1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff11e4ece0 con 0x55ff11f414a0 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07c2af1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff11ea32a0 con 0x55ff11f41dc0 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07b77fe640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f07a00a2cf0 0x7f07a00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07bdcb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f4c680 0x55ff11f4ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07becb5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 0x55ff11f41880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07bdcb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f4c680 0x55ff11f4ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37826/0 (socket says 172.21.15.5:37826) 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07becb5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 0x55ff11f41880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54644/0 (socket says 172.21.15.5:54644) 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07bdcb3640 1 -- 172.21.15.5:0/4192763879 learned_addr learned my addr 172.21.15.5:0/4192763879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07be4b4640 1 --2- 172.21.15.5:0/4192763879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f4c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07b77fe640 1 -- 172.21.15.5:0/1765890175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f07b00cf8d0 con 0x7f07b8100e20 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07becb5640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f07a00a2cf0 0x7f07a00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07bdcb3640 1 -- 172.21.15.5:0/4192763879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 msgr2=0x55ff11f4c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/4192763879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f4c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07bdcb3640 1 -- 172.21.15.5:0/4192763879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 msgr2=0x55ff11f41880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.308+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/4192763879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 0x55ff11f41880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.309+0000 7f07bdcb3640 1 -- 172.21.15.5:0/4192763879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff11e33440 con 0x55ff11f4c680 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.309+0000 7f07be4b4640 1 --2- 172.21.15.5:0/4192763879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f4c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.309+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/4192763879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f4c680 0x55ff11f4ea70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0114f0 tx=0x7f07ac0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857d98945b642b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.309+0000 7f07b4ff9640 1 -- 172.21.15.5:0/4192763879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac01e070 con 0x55ff11f4c680 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.309+0000 7f07becb5640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f07a00a2cf0 0x7f07a00a51b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07a8004ad0 tx=0x7f07a802a950 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.309+0000 7f07b4ff9640 1 -- 172.21.15.5:0/4192763879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ac0035f0 con 0x55ff11f4c680 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.309+0000 7f07b4ff9640 1 -- 172.21.15.5:0/4192763879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00f820 con 0x55ff11f4c680 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 -- 172.21.15.5:0/4192763879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f4c680 msgr2=0x55ff11f4ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 --2- 172.21.15.5:0/4192763879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f4c680 0x55ff11f4ea70 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0114f0 tx=0x7f07ac0119b0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 -- 172.21.15.5:0/4192763879 shutdown_connections 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 --2- 172.21.15.5:0/4192763879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f4c680 0x55ff11f4ea70 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 --2- 172.21.15.5:0/4192763879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f4c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 --2- 172.21.15.5:0/4192763879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 0x55ff11f41880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 -- 172.21.15.5:0/4192763879 >> 172.21.15.5:0/4192763879 conn(0x55ff11f3da30 msgr2=0x55ff11f3de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 -- 172.21.15.5:0/4192763879 shutdown_connections 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 -- 172.21.15.5:0/4192763879 wait complete. 2024-09-11T22:23:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 -- start start 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 Processor -- start 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.310+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 0x55ff11f4bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f49210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f4c680 0x55ff11f49750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07c2af1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff11f4f250 con 0x55ff11f414a0 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07c2af1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff11e62050 con 0x55ff11f4c680 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07c2af1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff11f4f3d0 con 0x55ff11f41dc0 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07c2af1740 10 client.0 _ll_register_callbacks cb 0x55ff11ea2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07bdcb3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f49210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07bdcb3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f49210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45608/0 (socket says 172.21.15.5:45608) 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07bdcb3640 1 -- 172.21.15.5:0/265083421 learned_addr learned my addr 172.21.15.5:0/265083421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07be4b4640 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 0x55ff11f4bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07becb5640 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f4c680 0x55ff11f49750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07bdcb3640 1 -- 172.21.15.5:0/265083421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f4c680 msgr2=0x55ff11f49750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f4c680 0x55ff11f49750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07bdcb3640 1 -- 172.21.15.5:0/265083421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 msgr2=0x55ff11f4bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 0x55ff11f4bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07bdcb3640 1 -- 172.21.15.5:0/265083421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff11c183c0 con 0x55ff11f41dc0 2024-09-11T22:23:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f49210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07ac011eb0 tx=0x7f07ac0039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=477d14d4abd212da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.311+0000 7f079dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:23:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f079dffb640 1 -- 172.21.15.5:0/265083421 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac01e040 con 0x55ff11f41dc0 2024-09-11T22:23:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f07c2af1740 1 -- 172.21.15.5:0/265083421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07ac0035f0 con 0x55ff11f41dc0 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f079dffb640 1 -- 172.21.15.5:0/265083421 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f07ac003b70 con 0x55ff11f41dc0 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f079dffb640 1 -- 172.21.15.5:0/265083421 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07ac032050 con 0x55ff11f41dc0 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f07c2af1740 10 client.25354 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f07c2af1740 1 -- 172.21.15.5:0/265083421 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff11f5abc0 con 0x55ff11f41dc0 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f079dffb640 1 -- 172.21.15.5:0/265083421 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ac0035f0 con 0x55ff11f41dc0 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f079dffb640 1 -- 172.21.15.5:0/265083421 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f07ac021030 con 0x55ff11f41dc0 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f07c2af1740 10 client.25354 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f07c2af1740 10 client.25354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f07c2af1740 20 client.25354 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f07c2af1740 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff11f5c020 0x55ff11f5e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f07c2af1740 4 client.25354 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f07c2af1740 1 -- 172.21.15.5:0/265083421 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff11eb4850 con 0x55ff11f5c020 2024-09-11T22:23:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.312+0000 7f07be4b4640 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff11f5c020 0x55ff11f5e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.313+0000 7f07be4b4640 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff11f5c020 0x55ff11f5e410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3777fe667005f59 server_cookie=ec37271c80f8a82c in_seq=0 out_seq=0 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.313+0000 7f079dffb640 10 client.25354 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.313+0000 7f079dffb640 1 -- 172.21.15.5:0/265083421 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff11eb4850 con 0x55ff11f5c020 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.313+0000 7f079dffb640 10 client.25354 handle_command_reply: tid=0 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 client.25354 shutdown 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 -- 172.21.15.5:0/265083421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 msgr2=0x55ff11f49210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f49210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07ac011eb0 tx=0x7f07ac0039d0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 -- 172.21.15.5:0/265083421 shutdown_connections 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff11f5c020 0x55ff11f5e410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f4c680 0x55ff11f49750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11f49210 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 --2- 172.21.15.5:0/265083421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 0x55ff11f4bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 -- 172.21.15.5:0/265083421 >> 172.21.15.5:0/265083421 conn(0x55ff11f3da30 msgr2=0x55ff11f43710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f079dffb640 0 client.25354 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 -- 172.21.15.5:0/265083421 shutdown_connections 2024-09-11T22:23:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 1 -- 172.21.15.5:0/265083421 wait complete. 2024-09-11T22:23:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.314+0000 7f07c2af1740 20 client.25354 trim_cache size 0 max 16384 2024-09-11T22:23:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.331+0000 7f07c2af1740 1 Processor -- start 2024-09-11T22:23:50.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.331+0000 7f07c2af1740 1 -- start start 2024-09-11T22:23:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f3da30 0x55ff11f584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 0x55ff11f491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f41dc0 0x55ff11f63c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07c2af1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81afc40 con 0x55ff11f41dc0 2024-09-11T22:23:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07c2af1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b1840 con 0x55ff11f414a0 2024-09-11T22:23:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07c2af1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b0310 con 0x55ff11f3da30 2024-09-11T22:23:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07be4b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f3da30 0x55ff11f584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07becb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f41dc0 0x55ff11f63c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07be4b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f3da30 0x55ff11f584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45612/0 (socket says 172.21.15.5:45612) 2024-09-11T22:23:50.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07be4b4640 1 -- 172.21.15.5:0/1528318633 learned_addr learned my addr 172.21.15.5:0/1528318633 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/1528318633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 0x55ff11f491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07be4b4640 1 -- 172.21.15.5:0/1528318633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 msgr2=0x55ff11f491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07be4b4640 1 --2- 172.21.15.5:0/1528318633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 0x55ff11f491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07be4b4640 1 -- 172.21.15.5:0/1528318633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f41dc0 msgr2=0x55ff11f63c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07be4b4640 1 --2- 172.21.15.5:0/1528318633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f41dc0 0x55ff11f63c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07be4b4640 1 -- 172.21.15.5:0/1528318633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff11f4c2d0 con 0x55ff11f3da30 2024-09-11T22:23:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07becb5640 1 --2- 172.21.15.5:0/1528318633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f41dc0 0x55ff11f63c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:50.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.332+0000 7f07be4b4640 1 --2- 172.21.15.5:0/1528318633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f3da30 0x55ff11f584b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f07b00ccfe0 tx=0x7f07b006ddd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f39017c523d479ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f079ffff640 1 -- 172.21.15.5:0/1528318633 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b009bd50 con 0x55ff11f3da30 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f079ffff640 1 -- 172.21.15.5:0/1528318633 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b00a0100 con 0x55ff11f3da30 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f079ffff640 1 -- 172.21.15.5:0/1528318633 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00a4530 con 0x55ff11f3da30 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f07c2af1740 1 -- 172.21.15.5:0/1528318633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f3da30 msgr2=0x55ff11f584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f07c2af1740 1 --2- 172.21.15.5:0/1528318633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f3da30 0x55ff11f584b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f07b00ccfe0 tx=0x7f07b006ddd0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f07c2af1740 1 -- 172.21.15.5:0/1528318633 shutdown_connections 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f07c2af1740 1 --2- 172.21.15.5:0/1528318633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f41dc0 0x55ff11f63c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f07c2af1740 1 --2- 172.21.15.5:0/1528318633 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f414a0 0x55ff11f491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f07c2af1740 1 --2- 172.21.15.5:0/1528318633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f3da30 0x55ff11f584b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f07c2af1740 1 -- 172.21.15.5:0/1528318633 >> 172.21.15.5:0/1528318633 conn(0x55ff11f498b0 msgr2=0x55ff11f47ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f07c2af1740 1 -- 172.21.15.5:0/1528318633 shutdown_connections 2024-09-11T22:23:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.333+0000 7f07c2af1740 1 -- 172.21.15.5:0/1528318633 wait complete. 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07c2af1740 1 -- start start 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07c2af1740 1 Processor -- start 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f3da30 0x55ff11f4f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 0x55ff11f4fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07c2af1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11fc4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07c2af1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b10b0 con 0x55ff11f414a0 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07c2af1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81b09e0 con 0x55ff11f3da30 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07c2af1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81adc20 con 0x55ff11f41dc0 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07c2af1740 10 client.0 _ll_register_callbacks cb 0x7f07ac01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07bdcb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 0x55ff11f4fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07bdcb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 0x55ff11f4fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37866/0 (socket says 172.21.15.5:37866) 2024-09-11T22:23:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07bdcb3640 1 -- 172.21.15.5:0/3373709293 learned_addr learned my addr 172.21.15.5:0/3373709293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07be4b4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f3da30 0x55ff11f4f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.334+0000 7f07becb5640 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11fc4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f07bdcb3640 1 -- 172.21.15.5:0/3373709293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 msgr2=0x55ff11fc4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11fc4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f07bdcb3640 1 -- 172.21.15.5:0/3373709293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f3da30 msgr2=0x55ff11f4f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f3da30 0x55ff11f4f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f07bdcb3640 1 -- 172.21.15.5:0/3373709293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff11fc5190 con 0x55ff11f414a0 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f07be4b4640 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f3da30 0x55ff11f4f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f07bdcb3640 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 0x55ff11f4fc60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f07ac004a30 tx=0x7f07ac0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afd3c0e02ef3571 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f079dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f079dffb640 1 -- 172.21.15.5:0/3373709293 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac01e040 con 0x55ff11f414a0 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f079dffb640 1 -- 172.21.15.5:0/3373709293 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f07ac003c50 con 0x55ff11f414a0 2024-09-11T22:23:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f07c2af1740 1 -- 172.21.15.5:0/3373709293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b00a0100 con 0x55ff11f414a0 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f079dffb640 1 -- 172.21.15.5:0/3373709293 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07ac0035f0 con 0x55ff11f414a0 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f07c2af1740 10 client.15861 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f07c2af1740 1 -- 172.21.15.5:0/3373709293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ff11fc6350 con 0x55ff11f414a0 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.335+0000 7f079dffb640 1 -- 172.21.15.5:0/3373709293 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b00a0100 con 0x55ff11f414a0 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f079dffb640 1 -- 172.21.15.5:0/3373709293 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f07ac03c030 con 0x55ff11f414a0 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f07c2af1740 10 client.15861 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f07c2af1740 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f07c2af1740 20 client.15861 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f07c2af1740 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff11f4c680 0x55ff11fc8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f07c2af1740 4 client.15861 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f07c2af1740 1 -- 172.21.15.5:0/3373709293 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ff11eaed60 con 0x55ff11f4c680 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f07be4b4640 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff11f4c680 0x55ff11fc8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f07be4b4640 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff11f4c680 0x55ff11fc8b60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e89776c4be16f6 server_cookie=74d556fdf130df82 in_seq=0 out_seq=0 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f079dffb640 10 client.15861 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f079dffb640 1 -- 172.21.15.5:0/3373709293 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ff11eaed60 con 0x55ff11f4c680 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f079dffb640 10 client.15861 handle_command_reply: tid=0 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.336+0000 7f07c2af1740 1 client.15861 shutdown 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 1 -- 172.21.15.5:0/3373709293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 msgr2=0x55ff11f4fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 0x55ff11f4fc60 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f07ac004a30 tx=0x7f07ac0048f0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 1 -- 172.21.15.5:0/3373709293 shutdown_connections 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff11f4c680 0x55ff11fc8b60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff11f41dc0 0x55ff11fc4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff11f414a0 0x55ff11f4fc60 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 1 --2- 172.21.15.5:0/3373709293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff11f3da30 0x55ff11f4f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 1 -- 172.21.15.5:0/3373709293 >> 172.21.15.5:0/3373709293 conn(0x55ff11f498b0 msgr2=0x55ff11f44870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f079dffb640 0 client.15861 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 1 -- 172.21.15.5:0/3373709293 shutdown_connections 2024-09-11T22:23:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 1 -- 172.21.15.5:0/3373709293 wait complete. 2024-09-11T22:23:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c2af1740 20 client.15861 trim_cache size 0 max 16384 2024-09-11T22:23:50.356 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:23:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:23:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:23:50.356 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:23:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c0718640 1 -- 172.21.15.5:0/1765890175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f07a00a2cf0 msgr2=0x7f07a00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c0718640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f07a00a2cf0 0x7f07a00a51b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07a8004ad0 tx=0x7f07a802a950 comp rx=0 tx=0).stop 2024-09-11T22:23:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c0718640 1 -- 172.21.15.5:0/1765890175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b8100e20 msgr2=0x7f07b819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.337+0000 7f07c0718640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b8100e20 0x7f07b819a6e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f07b000e9f0 tx=0x7f07b000edf0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.338+0000 7f07c0718640 1 -- 172.21.15.5:0/1765890175 shutdown_connections 2024-09-11T22:23:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.338+0000 7f07c0718640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07b8102ef0 0x7f07b819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.338+0000 7f07c0718640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f07a00a2cf0 0x7f07a00a51b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.338+0000 7f07c0718640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07b8102570 0x7f07b819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.338+0000 7f07c0718640 1 --2- 172.21.15.5:0/1765890175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b8100e20 0x7f07b819a6e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.338+0000 7f07c0718640 1 -- 172.21.15.5:0/1765890175 >> 172.21.15.5:0/1765890175 conn(0x7f07b80fc7d0 msgr2=0x7f07b8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.338+0000 7f07c0718640 1 -- 172.21.15.5:0/1765890175 shutdown_connections 2024-09-11T22:23:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:50.338+0000 7f07c0718640 1 -- 172.21.15.5:0/1765890175 wait complete. 2024-09-11T22:23:50.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:50 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:23:50.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:50 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:23:50.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:50 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:23:51.083 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:23:51.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:23:51.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 26854 17.82 MB/sec warmup 9 sec latency 14.112 ms 2024-09-11T22:23:51.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:51 smithi005 ceph-mon[30943]: pgmap v52: 97 pgs: 97 active+clean; 8.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 725 KiB/s wr, 165 op/s 2024-09-11T22:23:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:51.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:51 smithi159 ceph-mon[35534]: pgmap v52: 97 pgs: 97 active+clean; 8.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 725 KiB/s wr, 165 op/s 2024-09-11T22:23:51.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:51 smithi172 ceph-mon[35461]: pgmap v52: 97 pgs: 97 active+clean; 8.5 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 725 KiB/s wr, 165 op/s 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.930+0000 7f28668cd640 1 -- 172.21.15.5:0/2797170822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28601068d0 msgr2=0x7f2860106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.930+0000 7f28668cd640 1 --2- 172.21.15.5:0/2797170822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28601068d0 0x7f2860106cb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f285c009930 tx=0x7f285c02a420 comp rx=0 tx=0).stop 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.930+0000 7f28668cd640 1 -- 172.21.15.5:0/2797170822 shutdown_connections 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.930+0000 7f28668cd640 1 --2- 172.21.15.5:0/2797170822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2860101270 0x7f286010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.930+0000 7f28668cd640 1 --2- 172.21.15.5:0/2797170822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28601008d0 0x7f2860100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.930+0000 7f28668cd640 1 --2- 172.21.15.5:0/2797170822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28601068d0 0x7f2860106cb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.930+0000 7f28668cd640 1 -- 172.21.15.5:0/2797170822 >> 172.21.15.5:0/2797170822 conn(0x7f28600fc610 msgr2=0x7f28600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.930+0000 7f28668cd640 1 -- 172.21.15.5:0/2797170822 shutdown_connections 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.930+0000 7f28668cd640 1 -- 172.21.15.5:0/2797170822 wait complete. 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28668cd640 1 Processor -- start 2024-09-11T22:23:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28668cd640 1 -- start start 2024-09-11T22:23:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28668cd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28601008d0 0x7f2860199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28668cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2860101270 0x7f28601996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28668cd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28601068d0 0x7f28601a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28668cd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2860110cd0 con 0x7f2860101270 2024-09-11T22:23:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28668cd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2860110b50 con 0x7f28601008d0 2024-09-11T22:23:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28668cd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2860110e50 con 0x7f28601068d0 2024-09-11T22:23:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28650ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2860101270 0x7f28601996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28658cb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28601008d0 0x7f2860199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28650ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2860101270 0x7f28601996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44092/0 (socket says 172.21.15.5:44092) 2024-09-11T22:23:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28660cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28601068d0 0x7f28601a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:51.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28650ca640 1 -- 172.21.15.5:0/2415043083 learned_addr learned my addr 172.21.15.5:0/2415043083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:51.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28650ca640 1 -- 172.21.15.5:0/2415043083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28601068d0 msgr2=0x7f28601a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:51.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28650ca640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28601068d0 0x7f28601a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:51.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28650ca640 1 -- 172.21.15.5:0/2415043083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28601008d0 msgr2=0x7f2860199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:51.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28650ca640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28601008d0 0x7f2860199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.931+0000 7f28650ca640 1 -- 172.21.15.5:0/2415043083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28601a4720 con 0x7f2860101270 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.932+0000 7f28650ca640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2860101270 0x7f28601996a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f285000d770 tx=0x7f285000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.932+0000 7f2856ffd640 1 -- 172.21.15.5:0/2415043083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850017070 con 0x7f2860101270 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.932+0000 7f2856ffd640 1 -- 172.21.15.5:0/2415043083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2850005500 con 0x7f2860101270 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.932+0000 7f28668cd640 1 -- 172.21.15.5:0/2415043083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601a48f0 con 0x7f2860101270 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.932+0000 7f2856ffd640 1 -- 172.21.15.5:0/2415043083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850004e00 con 0x7f2860101270 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.932+0000 7f28668cd640 1 -- 172.21.15.5:0/2415043083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28601a4db0 con 0x7f2860101270 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.933+0000 7f28668cd640 1 -- 172.21.15.5:0/2415043083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2860100d30 con 0x7f2860101270 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.934+0000 7f2856ffd640 1 -- 172.21.15.5:0/2415043083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2850004fa0 con 0x7f2860101270 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.935+0000 7f2856ffd640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f284c0a2cf0 0x7f284c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:51.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.935+0000 7f28658cb640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f284c0a2cf0 0x7f284c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:51.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.935+0000 7f2856ffd640 1 -- 172.21.15.5:0/2415043083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f28500cee70 con 0x7f2860101270 2024-09-11T22:23:51.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.935+0000 7f28658cb640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f284c0a2cf0 0x7f284c0a51b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f285c009900 tx=0x7f285c0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:51.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:51.939+0000 7f2856ffd640 1 -- 172.21.15.5:0/2415043083 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f285009b3a0 con 0x7f2860101270 2024-09-11T22:23:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.160+0000 7f28668cd640 1 -- 172.21.15.5:0/2415043083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2860108e40 con 0x7f2860101270 2024-09-11T22:23:52.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.161+0000 7f2856ffd640 1 -- 172.21.15.5:0/2415043083 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11974 (secure 0 0 0) 0x7f285009f6a0 con 0x7f2860101270 2024-09-11T22:23:52.163 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:23:52.163 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-11T22:23:49:156575+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:23:48.779139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:23:52.163 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-11T22:23:52.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.163+0000 7f28668cd640 1 -- 172.21.15.5:0/2415043083 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f284c0a2cf0 msgr2=0x7f284c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:52.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.163+0000 7f28668cd640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f284c0a2cf0 0x7f284c0a51b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f285c009900 tx=0x7f285c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:23:52.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.163+0000 7f28668cd640 1 -- 172.21.15.5:0/2415043083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2860101270 msgr2=0x7f28601996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:52.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.163+0000 7f28668cd640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2860101270 0x7f28601996a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f285000d770 tx=0x7f285000dc30 comp rx=0 tx=0).stop 2024-09-11T22:23:52.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.163+0000 7f28668cd640 1 -- 172.21.15.5:0/2415043083 shutdown_connections 2024-09-11T22:23:52.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.163+0000 7f28668cd640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28601068d0 0x7f28601a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:52.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.163+0000 7f28668cd640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2860101270 0x7f28601996a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:52.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.163+0000 7f28668cd640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f284c0a2cf0 0x7f284c0a51b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:52.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.163+0000 7f28668cd640 1 --2- 172.21.15.5:0/2415043083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28601008d0 0x7f2860199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:52.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.163+0000 7f28668cd640 1 -- 172.21.15.5:0/2415043083 >> 172.21.15.5:0/2415043083 conn(0x7f28600fc610 msgr2=0x7f28600fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:52.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.164+0000 7f28668cd640 1 -- 172.21.15.5:0/2415043083 shutdown_connections 2024-09-11T22:23:52.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:52.164+0000 7f28668cd640 1 -- 172.21.15.5:0/2415043083 wait complete. 2024-09-11T22:23:52.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 29329 17.47 MB/sec warmup 10 sec latency 44.730 ms 2024-09-11T22:23:52.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:52 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2415043083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:52.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:52 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2415043083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:52.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:52 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2415043083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:23:52.867 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:23:53.131 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:53.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 32440 17.55 MB/sec warmup 11 sec latency 13.761 ms 2024-09-11T22:23:53.440 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:53 smithi005 ceph-mon[30943]: pgmap v53: 97 pgs: 97 active+clean; 15 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-09-11T22:23:53.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:53 smithi159 ceph-mon[35534]: pgmap v53: 97 pgs: 97 active+clean; 15 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-09-11T22:23:53.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:53 smithi172 ceph-mon[35461]: pgmap v53: 97 pgs: 97 active+clean; 15 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 1.4 MiB/s wr, 272 op/s 2024-09-11T22:23:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.702+0000 7f60a0dcb640 1 -- 172.21.15.5:0/3235749623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f609c106f40 msgr2=0x7f609c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.702+0000 7f60a0dcb640 1 --2- 172.21.15.5:0/3235749623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f609c106f40 0x7f609c10eac0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6094009ca0 tx=0x7f609402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.702+0000 7f60a0dcb640 1 -- 172.21.15.5:0/3235749623 shutdown_connections 2024-09-11T22:23:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.702+0000 7f60a0dcb640 1 --2- 172.21.15.5:0/3235749623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f609c106f40 0x7f609c10eac0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.702+0000 7f60a0dcb640 1 --2- 172.21.15.5:0/3235749623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f609c1027c0 0x7f609c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.702+0000 7f60a0dcb640 1 --2- 172.21.15.5:0/3235749623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f609c101e10 0x7f609c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.702+0000 7f60a0dcb640 1 -- 172.21.15.5:0/3235749623 >> 172.21.15.5:0/3235749623 conn(0x7f609c0fd8d0 msgr2=0x7f609c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.702+0000 7f60a0dcb640 1 -- 172.21.15.5:0/3235749623 shutdown_connections 2024-09-11T22:23:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.702+0000 7f60a0dcb640 1 -- 172.21.15.5:0/3235749623 wait complete. 2024-09-11T22:23:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f60a0dcb640 1 Processor -- start 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f60a0dcb640 1 -- start start 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f60a0dcb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f609c101e10 0x7f609c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f60a0dcb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f609c1027c0 0x7f609c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f60a0dcb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f609c106f40 0x7f609c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f60a0dcb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f609c111fa0 con 0x7f609c1027c0 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f60a0dcb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f609c111e20 con 0x7f609c101e10 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f60a0dcb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f609c112120 con 0x7f609c106f40 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f6093fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f609c1027c0 0x7f609c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f609a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f609c101e10 0x7f609c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f609a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f609c101e10 0x7f609c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48542/0 (socket says 172.21.15.5:48542) 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f6093fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f609c1027c0 0x7f609c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44118/0 (socket says 172.21.15.5:44118) 2024-09-11T22:23:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f609a59c640 1 -- 172.21.15.5:0/252788361 learned_addr learned my addr 172.21.15.5:0/252788361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f609ad9d640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f609c106f40 0x7f609c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f609a59c640 1 -- 172.21.15.5:0/252788361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f609c106f40 msgr2=0x7f609c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.703+0000 7f609a59c640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f609c106f40 0x7f609c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.704+0000 7f609a59c640 1 -- 172.21.15.5:0/252788361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f609c1027c0 msgr2=0x7f609c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.704+0000 7f609a59c640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f609c1027c0 0x7f609c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.704+0000 7f609a59c640 1 -- 172.21.15.5:0/252788361 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c19d390 con 0x7f609c101e10 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.704+0000 7f6093fff640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f609c1027c0 0x7f609c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.704+0000 7f609a59c640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f609c101e10 0x7f609c1983e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f608c00e9f0 tx=0x7f608c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.704+0000 7f60937fe640 1 -- 172.21.15.5:0/252788361 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c00cd80 con 0x7f609c101e10 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.704+0000 7f60a0dcb640 1 -- 172.21.15.5:0/252788361 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c19d680 con 0x7f609c101e10 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.704+0000 7f60937fe640 1 -- 172.21.15.5:0/252788361 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f608c010910 con 0x7f609c101e10 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.704+0000 7f60937fe640 1 -- 172.21.15.5:0/252788361 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c0188f0 con 0x7f609c101e10 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.704+0000 7f60a0dcb640 1 -- 172.21.15.5:0/252788361 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c1a4f60 con 0x7f609c101e10 2024-09-11T22:23:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.706+0000 7f60937fe640 1 -- 172.21.15.5:0/252788361 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f608c010ac0 con 0x7f609c101e10 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f60a31a4740 1 Processor -- start 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f60a31a4740 1 -- start start 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d831090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d831a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d83e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f60a31a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55845d7972f0 con 0x55845d8315d0 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f60a31a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55845d73ecf0 con 0x55845d83c370 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f60a31a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55845d7932b0 con 0x55845d832b40 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f609a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d831a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f609ad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d831090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f609a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d831a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44130/0 (socket says 172.21.15.5:44130) 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f609a59c640 1 -- 172.21.15.5:0/1813004076 learned_addr learned my addr 172.21.15.5:0/1813004076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f6093fff640 1 --2- 172.21.15.5:0/1813004076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d83e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.707+0000 7f60937fe640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60880a2cf0 0x7f60880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f609ad9d640 1 -- 172.21.15.5:0/1813004076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 msgr2=0x55845d83e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f609ad9d640 1 --2- 172.21.15.5:0/1813004076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d83e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f609ad9d640 1 -- 172.21.15.5:0/1813004076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 msgr2=0x55845d831a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f609ad9d640 1 --2- 172.21.15.5:0/1813004076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d831a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f609ad9d640 1 -- 172.21.15.5:0/1813004076 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55845d7233a0 con 0x55845d832b40 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f609a59c640 1 --2- 172.21.15.5:0/1813004076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d831a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f60937fe640 1 -- 172.21.15.5:0/252788361 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f608c0cf8b0 con 0x7f609c101e10 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f609ad9d640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60880a2cf0 0x7f60880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f609ad9d640 1 --2- 172.21.15.5:0/1813004076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d831090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6094006010 tx=0x7f6094009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22f16cfda48fe0eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f6090ff9640 1 -- 172.21.15.5:0/1813004076 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094042040 con 0x55845d832b40 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f609ad9d640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60880a2cf0 0x7f60880a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6094005e30 tx=0x7f609404e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f6090ff9640 1 -- 172.21.15.5:0/1813004076 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6094036040 con 0x55845d832b40 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.708+0000 7f6090ff9640 1 -- 172.21.15.5:0/1813004076 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609402f8f0 con 0x55845d832b40 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.709+0000 7f60a31a4740 1 -- 172.21.15.5:0/1813004076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 msgr2=0x55845d831090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.709+0000 7f60a31a4740 1 --2- 172.21.15.5:0/1813004076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d831090 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6094006010 tx=0x7f6094009800 comp rx=0 tx=0).stop 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.709+0000 7f60a31a4740 1 -- 172.21.15.5:0/1813004076 shutdown_connections 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.709+0000 7f60a31a4740 1 --2- 172.21.15.5:0/1813004076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d83e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.709+0000 7f60a31a4740 1 --2- 172.21.15.5:0/1813004076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d831a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.709+0000 7f60a31a4740 1 --2- 172.21.15.5:0/1813004076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d831090 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.709+0000 7f60a31a4740 1 -- 172.21.15.5:0/1813004076 >> 172.21.15.5:0/1813004076 conn(0x55845d7fa310 msgr2=0x55845d82dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.709+0000 7f60a31a4740 1 -- 172.21.15.5:0/1813004076 shutdown_connections 2024-09-11T22:23:53.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.709+0000 7f60a31a4740 1 -- 172.21.15.5:0/1813004076 wait complete. 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f60a31a4740 1 -- start start 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f60a31a4740 1 Processor -- start 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d83bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d839040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d839650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f60a31a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55845d83f070 con 0x55845d8315d0 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f60a31a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55845d752060 con 0x55845d83c370 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f60a31a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55845d83f1f0 con 0x55845d832b40 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f60a31a4740 10 client.0 _ll_register_callbacks cb 0x55845d4a58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f6093fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d839040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f609a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d83bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f609ad9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d839650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f6093fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d839040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56668/0 (socket says 172.21.15.5:56668) 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.710+0000 7f6093fff640 1 -- 172.21.15.5:0/1401895964 learned_addr learned my addr 172.21.15.5:0/1401895964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f6093fff640 1 -- 172.21.15.5:0/1401895964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 msgr2=0x55845d839650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f6093fff640 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d839650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f6093fff640 1 -- 172.21.15.5:0/1401895964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 msgr2=0x55845d83bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f6093fff640 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d83bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f6093fff640 1 -- 172.21.15.5:0/1401895964 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55845d5083c0 con 0x55845d832b40 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f609a59c640 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d83bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f609ad9d640 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d839650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f6093fff640 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d839040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6084012d20 tx=0x7f608400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63e53015a7dc956d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f6081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f6081ffb640 1 -- 172.21.15.5:0/1401895964 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60840120d0 con 0x55845d832b40 2024-09-11T22:23:53.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f6081ffb640 1 -- 172.21.15.5:0/1401895964 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f6084010430 con 0x55845d832b40 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f60a31a4740 1 -- 172.21.15.5:0/1401895964 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6094036040 con 0x55845d832b40 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f6081ffb640 1 -- 172.21.15.5:0/1401895964 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6084012270 con 0x55845d832b40 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f60a31a4740 10 client.25378 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.711+0000 7f60a31a4740 1 -- 172.21.15.5:0/1401895964 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55845d84aba0 con 0x55845d832b40 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f6081ffb640 1 -- 172.21.15.5:0/1401895964 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6094036040 con 0x55845d832b40 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f6081ffb640 1 -- 172.21.15.5:0/1401895964 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6084022030 con 0x55845d832b40 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f60a31a4740 10 client.25378 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f60a31a4740 10 client.25378 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f60a31a4740 20 client.25378 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f60a31a4740 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55845d84c100 0x55845d84e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f60a31a4740 4 client.25378 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f60a31a4740 1 -- 172.21.15.5:0/1401895964 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55845d7a4860 con 0x55845d84c100 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f609a59c640 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55845d84c100 0x55845d84e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f609a59c640 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55845d84c100 0x55845d84e520 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a5e58e8a6c6b41e server_cookie=b6089f6b08149dd7 in_seq=0 out_seq=0 2024-09-11T22:23:53.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.712+0000 7f6081ffb640 10 client.25378 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f6081ffb640 1 -- 172.21.15.5:0/1401895964 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55845d7a4860 con 0x55845d84c100 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f6081ffb640 10 client.25378 handle_command_reply: tid=0 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f60a31a4740 1 client.25378 shutdown 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f60a31a4740 1 -- 172.21.15.5:0/1401895964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 msgr2=0x55845d839040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f60a31a4740 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d839040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6084012d20 tx=0x7f608400f790 comp rx=0 tx=0).stop 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f60a31a4740 1 -- 172.21.15.5:0/1401895964 shutdown_connections 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f60a31a4740 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55845d84c100 0x55845d84e520 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f60a31a4740 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d839650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f60a31a4740 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d839040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f6081ffb640 0 client.25378 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f60a31a4740 1 --2- 172.21.15.5:0/1401895964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d8315d0 0x55845d83bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.713+0000 7f60a31a4740 1 -- 172.21.15.5:0/1401895964 >> 172.21.15.5:0/1401895964 conn(0x55845d7fa310 msgr2=0x55845d833960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.714+0000 7f60a31a4740 1 -- 172.21.15.5:0/1401895964 shutdown_connections 2024-09-11T22:23:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.714+0000 7f60a31a4740 1 -- 172.21.15.5:0/1401895964 wait complete. 2024-09-11T22:23:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.714+0000 7f60a31a4740 20 client.25378 trim_cache size 0 max 16384 2024-09-11T22:23:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.727+0000 7f60a31a4740 1 Processor -- start 2024-09-11T22:23:53.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.727+0000 7f60a31a4740 1 -- start start 2024-09-11T22:23:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.727+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d8315d0 0x55845d848310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d832b40 0x55845d838040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d853df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f60a31a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1afba0 con 0x55845d832b40 2024-09-11T22:23:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f60a31a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b17a0 con 0x55845d83c370 2024-09-11T22:23:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f60a31a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b0270 con 0x55845d8315d0 2024-09-11T22:23:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f609a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d8315d0 0x55845d848310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f6093fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d832b40 0x55845d838040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f609a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d8315d0 0x55845d848310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56682/0 (socket says 172.21.15.5:56682) 2024-09-11T22:23:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f609a59c640 1 -- 172.21.15.5:0/13064175 learned_addr learned my addr 172.21.15.5:0/13064175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f609ad9d640 1 --2- 172.21.15.5:0/13064175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d853df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f609a59c640 1 -- 172.21.15.5:0/13064175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 msgr2=0x55845d853df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f609a59c640 1 --2- 172.21.15.5:0/13064175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d853df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f609a59c640 1 -- 172.21.15.5:0/13064175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d832b40 msgr2=0x55845d838040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f609a59c640 1 --2- 172.21.15.5:0/13064175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d832b40 0x55845d838040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f609a59c640 1 -- 172.21.15.5:0/13064175 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55845d8399c0 con 0x55845d8315d0 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f6093fff640 1 --2- 172.21.15.5:0/13064175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d832b40 0x55845d838040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f609a59c640 1 --2- 172.21.15.5:0/13064175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d8315d0 0x55845d848310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f608c00cfd0 tx=0x7f608c0d1d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3be33fca998f83b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f6083fff640 1 -- 172.21.15.5:0/13064175 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c09bde0 con 0x55845d8315d0 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f6083fff640 1 -- 172.21.15.5:0/13064175 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f608c0a00e0 con 0x55845d8315d0 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.728+0000 7f6083fff640 1 -- 172.21.15.5:0/13064175 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c09b520 con 0x55845d8315d0 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.729+0000 7f60a31a4740 1 -- 172.21.15.5:0/13064175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d8315d0 msgr2=0x55845d848310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.729+0000 7f60a31a4740 1 --2- 172.21.15.5:0/13064175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d8315d0 0x55845d848310 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f608c00cfd0 tx=0x7f608c0d1d00 comp rx=0 tx=0).stop 2024-09-11T22:23:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.729+0000 7f60a31a4740 1 -- 172.21.15.5:0/13064175 shutdown_connections 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.729+0000 7f60a31a4740 1 --2- 172.21.15.5:0/13064175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d83c370 0x55845d853df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.729+0000 7f60a31a4740 1 --2- 172.21.15.5:0/13064175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d832b40 0x55845d838040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.729+0000 7f60a31a4740 1 --2- 172.21.15.5:0/13064175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d8315d0 0x55845d848310 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.729+0000 7f60a31a4740 1 -- 172.21.15.5:0/13064175 >> 172.21.15.5:0/13064175 conn(0x55845d7fa310 msgr2=0x55845d837930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.729+0000 7f60a31a4740 1 -- 172.21.15.5:0/13064175 shutdown_connections 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.729+0000 7f60a31a4740 1 -- 172.21.15.5:0/13064175 wait complete. 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f60a31a4740 1 -- start start 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f60a31a4740 1 Processor -- start 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d8315d0 0x55845d844660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d844ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f60a31a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d83c370 0x55845d835850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f60a31a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b1010 con 0x55845d83c370 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f60a31a4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b0940 con 0x55845d8315d0 2024-09-11T22:23:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f609a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d8315d0 0x55845d844660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f60a31a4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1adb80 con 0x55845d832b40 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f609a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d8315d0 0x55845d844660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48586/0 (socket says 172.21.15.5:48586) 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f609a59c640 1 -- 172.21.15.5:0/3016102087 learned_addr learned my addr 172.21.15.5:0/3016102087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f6093fff640 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d844ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f60a31a4740 10 client.0 _ll_register_callbacks cb 0x7f6084009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.730+0000 7f609ad9d640 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d83c370 0x55845d835850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f609a59c640 1 -- 172.21.15.5:0/3016102087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 msgr2=0x55845d844ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f609a59c640 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d844ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f609a59c640 1 -- 172.21.15.5:0/3016102087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d83c370 msgr2=0x55845d835850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f609a59c640 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d83c370 0x55845d835850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f609a59c640 1 -- 172.21.15.5:0/3016102087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55845d836080 con 0x55845d8315d0 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f609ad9d640 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d83c370 0x55845d835850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f609a59c640 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d8315d0 0x55845d844660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f608c0d1010 tx=0x7f608c0ccf80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32f0fafd8a18fd54 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:53.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f6081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f6081ffb640 1 -- 172.21.15.5:0/3016102087 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c09a0e0 con 0x55845d8315d0 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f6081ffb640 1 -- 172.21.15.5:0/3016102087 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f608c010430 con 0x55845d8315d0 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f60a31a4740 1 -- 172.21.15.5:0/3016102087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608c0a00e0 con 0x55845d8315d0 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f6081ffb640 1 -- 172.21.15.5:0/3016102087 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f608c09bde0 con 0x55845d8315d0 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f60a31a4740 10 client.25427 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.731+0000 7f60a31a4740 1 -- 172.21.15.5:0/3016102087 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55845d8b6860 con 0x55845d8315d0 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f6081ffb640 1 -- 172.21.15.5:0/3016102087 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f608c0a00e0 con 0x55845d8315d0 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f6081ffb640 1 -- 172.21.15.5:0/3016102087 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f608c0b2020 con 0x55845d8315d0 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f60a31a4740 10 client.25427 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f60a31a4740 10 client.25427 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f60a31a4740 20 client.25427 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f60a31a4740 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55845d8b6f80 0x55845d8b93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f60a31a4740 4 client.25427 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:23:53.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f60a31a4740 1 -- 172.21.15.5:0/3016102087 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55845d79ed70 con 0x55845d8b6f80 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f6093fff640 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55845d8b6f80 0x55845d8b93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f6093fff640 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55845d8b6f80 0x55845d8b93a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7a726bef72508793 server_cookie=ba5266ca066de0b3 in_seq=0 out_seq=0 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.732+0000 7f6081ffb640 10 client.25427 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f6081ffb640 1 -- 172.21.15.5:0/3016102087 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55845d79ed70 con 0x55845d8b6f80 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f6081ffb640 10 client.25427 handle_command_reply: tid=0 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 client.25427 shutdown 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 -- 172.21.15.5:0/3016102087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d8315d0 msgr2=0x55845d844660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d8315d0 0x55845d844660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f608c0d1010 tx=0x7f608c0ccf80 comp rx=0 tx=0).stop 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 -- 172.21.15.5:0/3016102087 shutdown_connections 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55845d8b6f80 0x55845d8b93a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55845d83c370 0x55845d835850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55845d832b40 0x55845d844ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 --2- 172.21.15.5:0/3016102087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55845d8315d0 0x55845d844660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 -- 172.21.15.5:0/3016102087 >> 172.21.15.5:0/3016102087 conn(0x55845d7fa310 msgr2=0x55845d838150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f6081ffb640 0 client.25427 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:23:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 -- 172.21.15.5:0/3016102087 shutdown_connections 2024-09-11T22:23:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 1 -- 172.21.15.5:0/3016102087 wait complete. 2024-09-11T22:23:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.733+0000 7f60a31a4740 20 client.25427 trim_cache size 0 max 16384 2024-09-11T22:23:53.749 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:23:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 -- 172.21.15.5:0/252788361 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60880a2cf0 msgr2=0x7f60880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60880a2cf0 0x7f60880a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6094005e30 tx=0x7f609404e000 comp rx=0 tx=0).stop 2024-09-11T22:23:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 -- 172.21.15.5:0/252788361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f609c101e10 msgr2=0x7f609c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f609c101e10 0x7f609c1983e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f608c00e9f0 tx=0x7f608c00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 -- 172.21.15.5:0/252788361 shutdown_connections 2024-09-11T22:23:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f609c106f40 0x7f609c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f609c1027c0 0x7f609c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60880a2cf0 0x7f60880a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 --2- 172.21.15.5:0/252788361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f609c101e10 0x7f609c1983e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 -- 172.21.15.5:0/252788361 >> 172.21.15.5:0/252788361 conn(0x7f609c0fd8d0 msgr2=0x7f609c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 -- 172.21.15.5:0/252788361 shutdown_connections 2024-09-11T22:23:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:53.734+0000 7f60a0dcb640 1 -- 172.21.15.5:0/252788361 wait complete. 2024-09-11T22:23:54.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 35708 17.66 MB/sec warmup 12 sec latency 38.108 ms 2024-09-11T22:23:54.469 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:23:54.732 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.269+0000 7fc5b75df640 1 -- 172.21.15.5:0/201999570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b0103ff0 msgr2=0x7fc5b01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.269+0000 7fc5b75df640 1 --2- 172.21.15.5:0/201999570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b0103ff0 0x7fc5b01043d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009ca0 tx=0x7fc5a002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.269+0000 7fc5b75df640 1 -- 172.21.15.5:0/201999570 shutdown_connections 2024-09-11T22:23:55.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.269+0000 7fc5b75df640 1 --2- 172.21.15.5:0/201999570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b0109120 0x7fc5b0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.269+0000 7fc5b75df640 1 --2- 172.21.15.5:0/201999570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5b01049a0 0x7fc5b01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.269+0000 7fc5b75df640 1 --2- 172.21.15.5:0/201999570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b0103ff0 0x7fc5b01043d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.269+0000 7fc5b75df640 1 -- 172.21.15.5:0/201999570 >> 172.21.15.5:0/201999570 conn(0x7fc5b00ffa90 msgr2=0x7fc5b0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:55.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.269+0000 7fc5b75df640 1 -- 172.21.15.5:0/201999570 shutdown_connections 2024-09-11T22:23:55.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.269+0000 7fc5b75df640 1 -- 172.21.15.5:0/201999570 wait complete. 2024-09-11T22:23:55.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b75df640 1 Processor -- start 2024-09-11T22:23:55.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b75df640 1 -- start start 2024-09-11T22:23:55.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b75df640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5b0103ff0 0x7fc5b019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b75df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b01049a0 0x7fc5b019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b75df640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b0109120 0x7fc5b019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b75df640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0114150 con 0x7fc5b01049a0 2024-09-11T22:23:55.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b75df640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0113fd0 con 0x7fc5b0103ff0 2024-09-11T22:23:55.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b75df640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01142d0 con 0x7fc5b0109120 2024-09-11T22:23:55.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b4b7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b01049a0 0x7fc5b019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b537b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5b0103ff0 0x7fc5b019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b4b7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b01049a0 0x7fc5b019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44192/0 (socket says 172.21.15.5:44192) 2024-09-11T22:23:55.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b537b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5b0103ff0 0x7fc5b019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48612/0 (socket says 172.21.15.5:48612) 2024-09-11T22:23:55.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b4b7a640 1 -- 172.21.15.5:0/2899616130 learned_addr learned my addr 172.21.15.5:0/2899616130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:55.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.270+0000 7fc5b5b7c640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b0109120 0x7fc5b019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5b4b7a640 1 -- 172.21.15.5:0/2899616130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b0109120 msgr2=0x7fc5b019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:55.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b0109120 0x7fc5b019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5b4b7a640 1 -- 172.21.15.5:0/2899616130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5b0103ff0 msgr2=0x7fc5b019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:55.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5b0103ff0 0x7fc5b019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5b4b7a640 1 -- 172.21.15.5:0/2899616130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b019f5e0 con 0x7fc5b01049a0 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5b537b640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5b0103ff0 0x7fc5b019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b01049a0 0x7fc5b019ab70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400e9f0 tx=0x7fc5a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5ae7fc640 1 -- 172.21.15.5:0/2899616130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400cd80 con 0x7fc5b01049a0 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5b75df640 1 -- 172.21.15.5:0/2899616130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b019f8d0 con 0x7fc5b01049a0 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5ae7fc640 1 -- 172.21.15.5:0/2899616130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a4010910 con 0x7fc5b01049a0 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5ae7fc640 1 -- 172.21.15.5:0/2899616130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40188f0 con 0x7fc5b01049a0 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.271+0000 7fc5b75df640 1 -- 172.21.15.5:0/2899616130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b01a71b0 con 0x7fc5b01049a0 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.273+0000 7fc5ae7fc640 1 -- 172.21.15.5:0/2899616130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc5a4010ac0 con 0x7fc5b01049a0 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b99b8740 1 Processor -- start 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b99b8740 1 -- start start 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a971273b40 0x55a971272090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a971272a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97127d370 0x55a97127f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a9711d82f0 con 0x55a97127d370 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a97117fcf0 con 0x55a971273b40 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a9711d42b0 con 0x55a9712725d0 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b5b7c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a971273b40 0x55a971272090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b537b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a971272a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b5b7c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a971273b40 0x55a971272090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48622/0 (socket says 172.21.15.5:48622) 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b5b7c640 1 -- 172.21.15.5:0/3388437164 learned_addr learned my addr 172.21.15.5:0/3388437164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b537b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a971272a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56708/0 (socket says 172.21.15.5:56708) 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.274+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/3388437164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97127d370 0x55a97127f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5ae7fc640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5980a2cf0 0x7fc5980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5b5b7c640 1 -- 172.21.15.5:0/3388437164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 msgr2=0x55a971272a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5b5b7c640 1 --2- 172.21.15.5:0/3388437164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a971272a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5b5b7c640 1 -- 172.21.15.5:0/3388437164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97127d370 msgr2=0x55a97127f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5b5b7c640 1 --2- 172.21.15.5:0/3388437164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97127d370 0x55a97127f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5b5b7c640 1 -- 172.21.15.5:0/3388437164 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9711643a0 con 0x55a971273b40 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5ae7fc640 1 -- 172.21.15.5:0/2899616130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fc5a40cf800 con 0x7fc5b01049a0 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5b5b7c640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5980a2cf0 0x7fc5980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5b5b7c640 1 --2- 172.21.15.5:0/3388437164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a971273b40 0x55a971272090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009da0 tx=0x7fc5a8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af70a22bd98e25de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5977fe640 1 -- 172.21.15.5:0/3388437164 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8017050 con 0x55a971273b40 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5977fe640 1 -- 172.21.15.5:0/3388437164 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a800f8a0 con 0x55a971273b40 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5977fe640 1 -- 172.21.15.5:0/3388437164 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a800e6a0 con 0x55a971273b40 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.275+0000 7fc5b5b7c640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5980a2cf0 0x7fc5980a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8023000 tx=0x7fc5a8024000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.276+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3388437164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a971273b40 msgr2=0x55a971272090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.276+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3388437164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a971273b40 0x55a971272090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009da0 tx=0x7fc5a8011810 comp rx=0 tx=0).stop 2024-09-11T22:23:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.276+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3388437164 shutdown_connections 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.276+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3388437164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97127d370 0x55a97127f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.276+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3388437164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a971272a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.276+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3388437164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a971273b40 0x55a971272090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.276+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3388437164 >> 172.21.15.5:0/3388437164 conn(0x55a97123b310 msgr2=0x55a97126ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.276+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3388437164 shutdown_connections 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.276+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3388437164 wait complete. 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.276+0000 7fc5b99b8740 1 -- start start 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b99b8740 1 Processor -- start 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a97127cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a97127a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a97127a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a971280070 con 0x55a971273b40 2024-09-11T22:23:55.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a971193060 con 0x55a97127d370 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a9712801f0 con 0x55a9712725d0 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b99b8740 10 client.0 _ll_register_callbacks cb 0x55a970ee68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b4b7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a97127a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b537b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a97127cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b4b7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a97127a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44210/0 (socket says 172.21.15.5:44210) 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b4b7a640 1 -- 172.21.15.5:0/4288529286 learned_addr learned my addr 172.21.15.5:0/4288529286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.277+0000 7fc5b5b7c640 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a97127a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc5b537b640 1 -- 172.21.15.5:0/4288529286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 msgr2=0x55a97127a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc5b537b640 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a97127a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc5b537b640 1 -- 172.21.15.5:0/4288529286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 msgr2=0x55a97127a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc5b537b640 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a97127a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc5b537b640 1 -- 172.21.15.5:0/4288529286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a970f493c0 con 0x55a9712725d0 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc5b537b640 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a97127cd90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0000f80 tx=0x7fc5a0009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a06613d019a7dbd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc594ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc594ff9640 1 -- 172.21.15.5:0/4288529286 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a002f810 con 0x55a9712725d0 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc594ff9640 1 -- 172.21.15.5:0/4288529286 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fc5a002f9b0 con 0x55a9712725d0 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc5b99b8740 1 -- 172.21.15.5:0/4288529286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5a800f8a0 con 0x55a9712725d0 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc594ff9640 1 -- 172.21.15.5:0/4288529286 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc5a0053050 con 0x55a9712725d0 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc5b99b8740 10 client.25396 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.278+0000 7fc5b99b8740 1 -- 172.21.15.5:0/4288529286 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a97128bb40 con 0x55a9712725d0 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc594ff9640 1 -- 172.21.15.5:0/4288529286 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a800f8a0 con 0x55a9712725d0 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc594ff9640 1 -- 172.21.15.5:0/4288529286 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fc5a004e030 con 0x55a9712725d0 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc5b99b8740 10 client.25396 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc5b99b8740 10 client.25396 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc5b99b8740 20 client.25396 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a97128d000 0x55a97128f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc5b99b8740 4 client.25396 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc5b99b8740 1 -- 172.21.15.5:0/4288529286 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9711e5860 con 0x55a97128d000 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a97128d000 0x55a97128f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc594ff9640 10 client.25396 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.279+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a97128d000 0x55a97128f430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d257b62faf9a2c1 server_cookie=e3d741a882d5f00d in_seq=0 out_seq=0 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.280+0000 7fc594ff9640 1 -- 172.21.15.5:0/4288529286 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9711e5860 con 0x55a97128d000 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.280+0000 7fc594ff9640 10 client.25396 handle_command_reply: tid=0 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.280+0000 7fc5b99b8740 1 client.25396 shutdown 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.280+0000 7fc5b99b8740 1 -- 172.21.15.5:0/4288529286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 msgr2=0x55a97127cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.280+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a97127cd90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0000f80 tx=0x7fc5a0009360 comp rx=0 tx=0).stop 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.281+0000 7fc5b99b8740 1 -- 172.21.15.5:0/4288529286 shutdown_connections 2024-09-11T22:23:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.281+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a97128d000 0x55a97128f430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.281+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a97127a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.281+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a97127a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.281+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/4288529286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a97127cd90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.281+0000 7fc5b99b8740 1 -- 172.21.15.5:0/4288529286 >> 172.21.15.5:0/4288529286 conn(0x55a97123b310 msgr2=0x55a971274960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.281+0000 7fc594ff9640 0 client.25396 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:23:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.281+0000 7fc5b99b8740 1 -- 172.21.15.5:0/4288529286 shutdown_connections 2024-09-11T22:23:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.281+0000 7fc5b99b8740 1 -- 172.21.15.5:0/4288529286 wait complete. 2024-09-11T22:23:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.281+0000 7fc5b99b8740 20 client.25396 trim_cache size 0 max 16384 2024-09-11T22:23:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.294+0000 7fc5b99b8740 1 Processor -- start 2024-09-11T22:23:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.294+0000 7fc5b99b8740 1 -- start start 2024-09-11T22:23:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.294+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9712725d0 0x55a971275120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.294+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a971273b40 0x55a971275660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a971294d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01afcd0 con 0x55a9712725d0 2024-09-11T22:23:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01b18d0 con 0x55a97127d370 2024-09-11T22:23:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01b03a0 con 0x55a971273b40 2024-09-11T22:23:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b537b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9712725d0 0x55a971275120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b537b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9712725d0 0x55a971275120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44212/0 (socket says 172.21.15.5:44212) 2024-09-11T22:23:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b537b640 1 -- 172.21.15.5:0/3439471109 learned_addr learned my addr 172.21.15.5:0/3439471109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b5b7c640 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a971294d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b537b640 1 -- 172.21.15.5:0/3439471109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a971273b40 msgr2=0x55a971275660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a971273b40 0x55a971275660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b537b640 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a971273b40 0x55a971275660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b537b640 1 -- 172.21.15.5:0/3439471109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 msgr2=0x55a971294d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b537b640 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a971294d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b537b640 1 -- 172.21.15.5:0/3439471109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a97127afc0 con 0x55a9712725d0 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a971273b40 0x55a971275660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b5b7c640 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a971294d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc5b537b640 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9712725d0 0x55a971275120 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc5a002c010 tx=0x7fc5a003cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b8f0e394bc61ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc596ffd640 1 -- 172.21.15.5:0/3439471109 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a005b020 con 0x55a9712725d0 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.295+0000 7fc596ffd640 1 -- 172.21.15.5:0/3439471109 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a0036040 con 0x55a9712725d0 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.296+0000 7fc596ffd640 1 -- 172.21.15.5:0/3439471109 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0004040 con 0x55a9712725d0 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.296+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3439471109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9712725d0 msgr2=0x55a971275120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.296+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9712725d0 0x55a971275120 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc5a002c010 tx=0x7fc5a003cbd0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.296+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3439471109 shutdown_connections 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.296+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a971294d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.296+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a971273b40 0x55a971275660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.296+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3439471109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9712725d0 0x55a971275120 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.296+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3439471109 >> 172.21.15.5:0/3439471109 conn(0x55a97123b310 msgr2=0x55a97126dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.296+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3439471109 shutdown_connections 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.296+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3439471109 wait complete. 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b99b8740 1 -- start start 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b99b8740 1 Processor -- start 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a971281430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a971281970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b99b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a971281eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01b1140 con 0x55a971273b40 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01b0a70 con 0x55a97127d370 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b99b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01adcb0 con 0x55a9712725d0 2024-09-11T22:23:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b99b8740 10 client.0 _ll_register_callbacks cb 0x7fc5a0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b4b7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a971281970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b537b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a971281430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b4b7a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a971281970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44226/0 (socket says 172.21.15.5:44226) 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b4b7a640 1 -- 172.21.15.5:0/3165459532 learned_addr learned my addr 172.21.15.5:0/3165459532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.297+0000 7fc5b5b7c640 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a971281eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc5b4b7a640 1 -- 172.21.15.5:0/3165459532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 msgr2=0x55a971281430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a971281430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc5b4b7a640 1 -- 172.21.15.5:0/3165459532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 msgr2=0x55a971281eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a971281eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc5b4b7a640 1 -- 172.21.15.5:0/3165459532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9712f6f40 con 0x55a971273b40 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc5b537b640 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a971281430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc5b4b7a640 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a971281970 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4098760 tx=0x7fc5a401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88136663430298bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc594ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc594ff9640 1 -- 172.21.15.5:0/3165459532 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40d2e60 con 0x55a971273b40 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc594ff9640 1 -- 172.21.15.5:0/3165459532 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fc5a4098400 con 0x55a971273b40 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3165459532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5a0036040 con 0x55a971273b40 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc594ff9640 1 -- 172.21.15.5:0/3165459532 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc5a40a4950 con 0x55a971273b40 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc5b99b8740 10 client.15903 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.298+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3165459532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a9712f81f0 con 0x55a971273b40 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc594ff9640 1 -- 172.21.15.5:0/3165459532 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a0036040 con 0x55a971273b40 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc594ff9640 1 -- 172.21.15.5:0/3165459532 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fc5a4018be0 con 0x55a971273b40 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc5b99b8740 10 client.15903 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc5b99b8740 10 client.15903 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc5b99b8740 20 client.15903 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a9712f8920 0x55a9712fad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc5b99b8740 4 client.15903 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3165459532 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9711dfd70 con 0x55a9712f8920 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc5b537b640 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a9712f8920 0x55a9712fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc5b537b640 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a9712f8920 0x55a9712fad40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=848a96173adf7b25 server_cookie=e121f726754d9c0c in_seq=0 out_seq=0 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.299+0000 7fc594ff9640 10 client.15903 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc594ff9640 1 -- 172.21.15.5:0/3165459532 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9711dfd70 con 0x55a9712f8920 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc594ff9640 10 client.15903 handle_command_reply: tid=0 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 client.15903 shutdown 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3165459532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 msgr2=0x55a971281970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a971281970 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4098760 tx=0x7fc5a401f000 comp rx=0 tx=0).stop 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3165459532 shutdown_connections 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a9712f8920 0x55a9712fad40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a97127d370 0x55a971281eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a971273b40 0x55a971281970 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 --2- 172.21.15.5:0/3165459532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9712725d0 0x55a971281430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3165459532 >> 172.21.15.5:0/3165459532 conn(0x55a97123b310 msgr2=0x55a97127b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc594ff9640 0 client.15903 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:23:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3165459532 shutdown_connections 2024-09-11T22:23:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 1 -- 172.21.15.5:0/3165459532 wait complete. 2024-09-11T22:23:55.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.300+0000 7fc5b99b8740 20 client.15903 trim_cache size 0 max 16384 2024-09-11T22:23:55.312 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:23:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 -- 172.21.15.5:0/2899616130 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5980a2cf0 msgr2=0x7fc5980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5980a2cf0 0x7fc5980a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8023000 tx=0x7fc5a8024000 comp rx=0 tx=0).stop 2024-09-11T22:23:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 -- 172.21.15.5:0/2899616130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b01049a0 msgr2=0x7fc5b019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b01049a0 0x7fc5b019ab70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400e9f0 tx=0x7fc5a400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 -- 172.21.15.5:0/2899616130 shutdown_connections 2024-09-11T22:23:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5b0109120 0x7fc5b019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b01049a0 0x7fc5b019ab70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5980a2cf0 0x7fc5980a51b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 --2- 172.21.15.5:0/2899616130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5b0103ff0 0x7fc5b019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 -- 172.21.15.5:0/2899616130 >> 172.21.15.5:0/2899616130 conn(0x7fc5b00ffa90 msgr2=0x7fc5b0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 -- 172.21.15.5:0/2899616130 shutdown_connections 2024-09-11T22:23:55.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:55.301+0000 7fc5b75df640 1 -- 172.21.15.5:0/2899616130 wait complete. 2024-09-11T22:23:55.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 38426 17.29 MB/sec warmup 13 sec latency 36.810 ms 2024-09-11T22:23:55.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:55 smithi005 ceph-mon[30943]: pgmap v54: 97 pgs: 97 active+clean; 15 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-11T22:23:55.578 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:55 smithi159 ceph-mon[35534]: pgmap v54: 97 pgs: 97 active+clean; 15 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-11T22:23:55.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:55 smithi172 ceph-mon[35461]: pgmap v54: 97 pgs: 97 active+clean; 15 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-11T22:23:56.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:23:56.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 41404 17.02 MB/sec warmup 14 sec latency 21.165 ms 2024-09-11T22:23:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:56.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.891+0000 7f97634aa640 1 -- 172.21.15.5:0/2924688346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c101de0 msgr2=0x7f975c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:56.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.891+0000 7f97634aa640 1 --2- 172.21.15.5:0/2924688346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c101de0 0x7f975c1021c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f9758009900 tx=0x7f975802a410 comp rx=0 tx=0).stop 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.891+0000 7f97634aa640 1 -- 172.21.15.5:0/2924688346 shutdown_connections 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.891+0000 7f97634aa640 1 --2- 172.21.15.5:0/2924688346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f975c106f10 0x7f975c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.891+0000 7f97634aa640 1 --2- 172.21.15.5:0/2924688346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f975c102790 0x7f975c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.891+0000 7f97634aa640 1 --2- 172.21.15.5:0/2924688346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c101de0 0x7f975c1021c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.891+0000 7f97634aa640 1 -- 172.21.15.5:0/2924688346 >> 172.21.15.5:0/2924688346 conn(0x7f975c0fd8c0 msgr2=0x7f975c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.892+0000 7f97634aa640 1 -- 172.21.15.5:0/2924688346 shutdown_connections 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.892+0000 7f97634aa640 1 -- 172.21.15.5:0/2924688346 wait complete. 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.892+0000 7f97634aa640 1 Processor -- start 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.892+0000 7f97634aa640 1 -- start start 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f97634aa640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f975c101de0 0x7f975c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f97634aa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f975c102790 0x7f975c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f97634aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c106f10 0x7f975c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f97634aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f975c111f70 con 0x7f975c106f10 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f97634aa640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f975c111df0 con 0x7f975c101de0 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f97634aa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1120f0 con 0x7f975c102790 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761a47640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c106f10 0x7f975c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761246640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f975c101de0 0x7f975c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9760a45640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f975c102790 0x7f975c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761246640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f975c101de0 0x7f975c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48646/0 (socket says 172.21.15.5:48646) 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761246640 1 -- 172.21.15.5:0/3576223817 learned_addr learned my addr 172.21.15.5:0/3576223817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761a47640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c106f10 0x7f975c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44244/0 (socket says 172.21.15.5:44244) 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761a47640 1 -- 172.21.15.5:0/3576223817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f975c102790 msgr2=0x7f975c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761a47640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f975c102790 0x7f975c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761a47640 1 -- 172.21.15.5:0/3576223817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f975c101de0 msgr2=0x7f975c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761a47640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f975c101de0 0x7f975c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761a47640 1 -- 172.21.15.5:0/3576223817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c19d3e0 con 0x7f975c106f10 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f9761a47640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c106f10 0x7f975c19cd00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9754009fd0 tx=0x7f975400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.893+0000 7f97527fc640 1 -- 172.21.15.5:0/3576223817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975400ce20 con 0x7f975c106f10 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.894+0000 7f97634aa640 1 -- 172.21.15.5:0/3576223817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c19d6d0 con 0x7f975c106f10 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.894+0000 7f97527fc640 1 -- 172.21.15.5:0/3576223817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9754005020 con 0x7f975c106f10 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.894+0000 7f97527fc640 1 -- 172.21.15.5:0/3576223817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754010720 con 0x7f975c106f10 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.894+0000 7f97634aa640 1 -- 172.21.15.5:0/3576223817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c1a4fb0 con 0x7f975c106f10 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.897+0000 7f97527fc640 1 -- 172.21.15.5:0/3576223817 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9754005540 con 0x7f975c106f10 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.897+0000 7f9765883740 1 Processor -- start 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.897+0000 7f9765883740 1 -- start start 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.897+0000 7f9765883740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c3b40 0x5559652c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.897+0000 7f9765883740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c25d0 0x5559652c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.897+0000 7f9765883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.897+0000 7f9765883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559652282f0 con 0x5559652cd370 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.897+0000 7f9765883740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5559651cfcf0 con 0x5559652c25d0 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.897+0000 7f9765883740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5559652242b0 con 0x5559652c3b40 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f97527fc640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97480a2cf0 0x7f97480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9760a45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761246640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c25d0 0x5559652c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761246640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c25d0 0x5559652c2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48658/0 (socket says 172.21.15.5:48658) 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9760a45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44260/0 (socket says 172.21.15.5:44260) 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761246640 1 -- 172.21.15.5:0/2325214335 learned_addr learned my addr 172.21.15.5:0/2325214335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9760a45640 1 --2- 172.21.15.5:0/2325214335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c3b40 0x5559652c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761a47640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97480a2cf0 0x7f97480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f97527fc640 1 -- 172.21.15.5:0/3576223817 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f97540cfa20 con 0x7f975c106f10 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761246640 1 -- 172.21.15.5:0/2325214335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c3b40 msgr2=0x5559652c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761246640 1 --2- 172.21.15.5:0/2325214335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c3b40 0x5559652c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761246640 1 -- 172.21.15.5:0/2325214335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 msgr2=0x5559652cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761246640 1 --2- 172.21.15.5:0/2325214335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761246640 1 -- 172.21.15.5:0/2325214335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559651b43a0 con 0x5559652c25d0 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761246640 1 --2- 172.21.15.5:0/2325214335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c25d0 0x5559652c2a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9758009a20 tx=0x7f9758005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=defaf5f8606cb88b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f9761a47640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97480a2cf0 0x7f97480a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97540cd1e0 tx=0x7f975406dec0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f97477fe640 1 -- 172.21.15.5:0/2325214335 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758035040 con 0x5559652c25d0 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f97477fe640 1 -- 172.21.15.5:0/2325214335 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758031870 con 0x5559652c25d0 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.898+0000 7f97477fe640 1 -- 172.21.15.5:0/2325214335 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580049a0 con 0x5559652c25d0 2024-09-11T22:23:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.899+0000 7f9765883740 1 -- 172.21.15.5:0/2325214335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c25d0 msgr2=0x5559652c2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.899+0000 7f9765883740 1 --2- 172.21.15.5:0/2325214335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c25d0 0x5559652c2a10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9758009a20 tx=0x7f9758005dd0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.899+0000 7f9765883740 1 -- 172.21.15.5:0/2325214335 shutdown_connections 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.899+0000 7f9765883740 1 --2- 172.21.15.5:0/2325214335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.899+0000 7f9765883740 1 --2- 172.21.15.5:0/2325214335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c25d0 0x5559652c2a10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.899+0000 7f9765883740 1 --2- 172.21.15.5:0/2325214335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c3b40 0x5559652c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.899+0000 7f9765883740 1 -- 172.21.15.5:0/2325214335 >> 172.21.15.5:0/2325214335 conn(0x55596528b310 msgr2=0x5559652bec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.899+0000 7f9765883740 1 -- 172.21.15.5:0/2325214335 shutdown_connections 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.899+0000 7f9765883740 1 -- 172.21.15.5:0/2325214335 wait complete. 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9765883740 1 -- start start 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9765883740 1 Processor -- start 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9765883740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 0x5559652cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9765883740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 0x5559652c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9765883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9765883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559652d0010 con 0x5559652cd370 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9765883740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5559651e3060 con 0x5559652c3b40 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9765883740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5559652d0190 con 0x5559652c25d0 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9765883740 10 client.0 _ll_register_callbacks cb 0x555964f368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9760a45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9760a45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44272/0 (socket says 172.21.15.5:44272) 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9760a45640 1 -- 172.21.15.5:0/2703626691 learned_addr learned my addr 172.21.15.5:0/2703626691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9761246640 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 0x5559652c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9760a45640 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 0x5559652cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.900+0000 7f9760a45640 1 -- 172.21.15.5:0/2703626691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 msgr2=0x5559652cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9760a45640 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 0x5559652cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9760a45640 1 -- 172.21.15.5:0/2703626691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 msgr2=0x5559652c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9760a45640 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 0x5559652c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9760a45640 1 -- 172.21.15.5:0/2703626691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555964f993c0 con 0x5559652cd370 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9760a45640 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652ca520 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f974c00f4f0 tx=0x7f974c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260dee8021131b55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9744ff9640 1 -- 172.21.15.5:0/2703626691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c004250 con 0x5559652cd370 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9744ff9640 1 -- 172.21.15.5:0/2703626691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f974c0043f0 con 0x5559652cd370 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9765883740 1 -- 172.21.15.5:0/2703626691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9758031870 con 0x5559652cd370 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9744ff9640 1 -- 172.21.15.5:0/2703626691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f974c005020 con 0x5559652cd370 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9765883740 10 client.15915 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9765883740 1 -- 172.21.15.5:0/2703626691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559652dba40 con 0x5559652cd370 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9744ff9640 1 -- 172.21.15.5:0/2703626691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758031870 con 0x5559652cd370 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9744ff9640 1 -- 172.21.15.5:0/2703626691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f974c02c070 con 0x5559652cd370 2024-09-11T22:23:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9765883740 10 client.15915 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9765883740 10 client.15915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:23:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.901+0000 7f9765883740 20 client.15915 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.902+0000 7f9765883740 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5559652dcf00 0x5559652df330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.902+0000 7f9765883740 4 client.15915 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.902+0000 7f9765883740 1 -- 172.21.15.5:0/2703626691 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555965235860 con 0x5559652dcf00 2024-09-11T22:23:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.902+0000 7f9761246640 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5559652dcf00 0x5559652df330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.902+0000 7f9761246640 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5559652dcf00 0x5559652df330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=184de3be620eab26 server_cookie=f04c81b01c7e557f in_seq=0 out_seq=0 2024-09-11T22:23:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.902+0000 7f9744ff9640 10 client.15915 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:23:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9744ff9640 1 -- 172.21.15.5:0/2703626691 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555965235860 con 0x5559652dcf00 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9744ff9640 10 client.15915 handle_command_reply: tid=0 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 client.15915 shutdown 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 -- 172.21.15.5:0/2703626691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 msgr2=0x5559652ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652ca520 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f974c00f4f0 tx=0x7f974c004730 comp rx=0 tx=0).stop 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 -- 172.21.15.5:0/2703626691 shutdown_connections 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5559652dcf00 0x5559652df330 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652cd370 0x5559652ca520 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 0x5559652c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9744ff9640 0 client.15915 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 --2- 172.21.15.5:0/2703626691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 0x5559652cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 -- 172.21.15.5:0/2703626691 >> 172.21.15.5:0/2703626691 conn(0x55596528b310 msgr2=0x5559652c4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 -- 172.21.15.5:0/2703626691 shutdown_connections 2024-09-11T22:23:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 1 -- 172.21.15.5:0/2703626691 wait complete. 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.903+0000 7f9765883740 20 client.15915 trim_cache size 0 max 16384 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.917+0000 7f9765883740 1 Processor -- start 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.917+0000 7f9765883740 1 -- start start 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.917+0000 7f9765883740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 0x5559652c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.917+0000 7f9765883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c3b40 0x5559652cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.917+0000 7f9765883740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652cd370 0x5559652c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.917+0000 7f9765883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ab7f0 con 0x5559652c3b40 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.917+0000 7f9765883740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ad3f0 con 0x5559652cd370 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.917+0000 7f9765883740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1abec0 con 0x5559652c25d0 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9761246640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c3b40 0x5559652cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9760a45640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 0x5559652c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9760a45640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 0x5559652c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56794/0 (socket says 172.21.15.5:56794) 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9761246640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c3b40 0x5559652cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44286/0 (socket says 172.21.15.5:44286) 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9760a45640 1 -- 172.21.15.5:0/2989497641 learned_addr learned my addr 172.21.15.5:0/2989497641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9760a45640 1 --2- 172.21.15.5:0/2989497641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652cd370 0x5559652c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9761246640 1 -- 172.21.15.5:0/2989497641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 msgr2=0x5559652c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9761246640 1 --2- 172.21.15.5:0/2989497641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 0x5559652c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9761246640 1 -- 172.21.15.5:0/2989497641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652cd370 msgr2=0x5559652c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9761246640 1 --2- 172.21.15.5:0/2989497641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652cd370 0x5559652c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9761246640 1 -- 172.21.15.5:0/2989497641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559652c8f70 con 0x5559652c3b40 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9760a45640 1 --2- 172.21.15.5:0/2989497641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 0x5559652c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9761246640 1 --2- 172.21.15.5:0/2989497641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c3b40 0x5559652cbf10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9758005e40 tx=0x7f97580053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9096ea36857bf71c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9746ffd640 1 -- 172.21.15.5:0/2989497641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758035040 con 0x5559652c3b40 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9746ffd640 1 -- 172.21.15.5:0/2989497641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758031670 con 0x5559652c3b40 2024-09-11T22:23:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.918+0000 7f9746ffd640 1 -- 172.21.15.5:0/2989497641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758050450 con 0x5559652c3b40 2024-09-11T22:23:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 -- 172.21.15.5:0/2989497641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c3b40 msgr2=0x5559652cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 --2- 172.21.15.5:0/2989497641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c3b40 0x5559652cbf10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9758005e40 tx=0x7f97580053d0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 -- 172.21.15.5:0/2989497641 shutdown_connections 2024-09-11T22:23:56.987 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:23:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 --2- 172.21.15.5:0/2989497641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652cd370 0x5559652c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 --2- 172.21.15.5:0/2989497641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c3b40 0x5559652cbf10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 --2- 172.21.15.5:0/2989497641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652c25d0 0x5559652c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 -- 172.21.15.5:0/2989497641 >> 172.21.15.5:0/2989497641 conn(0x55596528b310 msgr2=0x5559652cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 -- 172.21.15.5:0/2989497641 shutdown_connections 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 -- 172.21.15.5:0/2989497641 wait complete. 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 -- start start 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.919+0000 7f9765883740 1 Processor -- start 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9765883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c25d0 0x5559652d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9765883740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 0x5559652c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9765883740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652cd370 0x5559652c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9765883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1acc60 con 0x5559652c25d0 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9765883740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1ac590 con 0x5559652c3b40 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9765883740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f975c1a97d0 con 0x5559652cd370 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9765883740 10 client.0 _ll_register_callbacks cb 0x7f974c009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9760a45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c25d0 0x5559652d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9760a45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c25d0 0x5559652d58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44300/0 (socket says 172.21.15.5:44300) 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9761246640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 0x5559652c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9760a45640 1 -- 172.21.15.5:0/2591315879 learned_addr learned my addr 172.21.15.5:0/2591315879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9760a45640 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652cd370 0x5559652c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9760a45640 1 -- 172.21.15.5:0/2591315879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652cd370 msgr2=0x5559652c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9760a45640 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652cd370 0x5559652c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9760a45640 1 -- 172.21.15.5:0/2591315879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 msgr2=0x5559652c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9760a45640 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 0x5559652c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9760a45640 1 -- 172.21.15.5:0/2591315879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559652c7480 con 0x5559652c25d0 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9761246640 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 0x5559652c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9760a45640 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c25d0 0x5559652d58a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f974c004ee0 tx=0x7f974c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c68b3a66a53e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.920+0000 7f9744ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9744ff9640 1 -- 172.21.15.5:0/2591315879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c032020 con 0x5559652c25d0 2024-09-11T22:23:56.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9744ff9640 1 -- 172.21.15.5:0/2591315879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f974c0049b0 con 0x5559652c25d0 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9765883740 1 -- 172.21.15.5:0/2591315879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9758031670 con 0x5559652c25d0 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9744ff9640 1 -- 172.21.15.5:0/2591315879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f974c009ba0 con 0x5559652c25d0 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9765883740 10 client.15921 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9765883740 1 -- 172.21.15.5:0/2591315879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559653478e0 con 0x5559652c25d0 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9744ff9640 1 -- 172.21.15.5:0/2591315879 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9758031670 con 0x5559652c25d0 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9744ff9640 1 -- 172.21.15.5:0/2591315879 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f974c014840 con 0x5559652c25d0 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9765883740 10 client.15921 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9765883740 10 client.15921 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9765883740 20 client.15921 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9765883740 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555965348000 0x55596534a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9765883740 4 client.15921 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9765883740 1 -- 172.21.15.5:0/2591315879 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55596522fd70 con 0x555965348000 2024-09-11T22:23:56.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.921+0000 7f9761246640 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555965348000 0x55596534a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9744ff9640 10 client.15921 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9761246640 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555965348000 0x55596534a420 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e6122bf9cddc352 server_cookie=d65097fbb1154f2b in_seq=0 out_seq=0 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9744ff9640 1 -- 172.21.15.5:0/2591315879 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55596522fd70 con 0x555965348000 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9744ff9640 10 client.15921 handle_command_reply: tid=0 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 client.15921 shutdown 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 -- 172.21.15.5:0/2591315879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c25d0 msgr2=0x5559652d58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c25d0 0x5559652d58a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f974c004ee0 tx=0x7f974c004730 comp rx=0 tx=0).stop 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 -- 172.21.15.5:0/2591315879 shutdown_connections 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555965348000 0x55596534a420 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559652cd370 0x5559652c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559652c3b40 0x5559652c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 --2- 172.21.15.5:0/2591315879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559652c25d0 0x5559652d58a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 -- 172.21.15.5:0/2591315879 >> 172.21.15.5:0/2591315879 conn(0x55596528b310 msgr2=0x5559652cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9744ff9640 0 client.15921 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 -- 172.21.15.5:0/2591315879 shutdown_connections 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.922+0000 7f9765883740 1 -- 172.21.15.5:0/2591315879 wait complete. 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f9765883740 20 client.15921 trim_cache size 0 max 16384 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 -- 172.21.15.5:0/3576223817 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97480a2cf0 msgr2=0x7f97480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97480a2cf0 0x7f97480a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f97540cd1e0 tx=0x7f975406dec0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 -- 172.21.15.5:0/3576223817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c106f10 msgr2=0x7f975c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c106f10 0x7f975c19cd00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9754009fd0 tx=0x7f975400ef10 comp rx=0 tx=0).stop 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 -- 172.21.15.5:0/3576223817 shutdown_connections 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c106f10 0x7f975c19cd00 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f975c102790 0x7f975c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97480a2cf0 0x7f97480a51b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 --2- 172.21.15.5:0/3576223817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f975c101de0 0x7f975c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:56.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 -- 172.21.15.5:0/3576223817 >> 172.21.15.5:0/3576223817 conn(0x7f975c0fd8c0 msgr2=0x7f975c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:56.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 -- 172.21.15.5:0/3576223817 shutdown_connections 2024-09-11T22:23:56.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:56.923+0000 7f97634aa640 1 -- 172.21.15.5:0/3576223817 wait complete. 2024-09-11T22:23:57.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 43835 16.84 MB/sec warmup 15 sec latency 11.310 ms 2024-09-11T22:23:57.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:57 smithi005 ceph-mon[30943]: pgmap v55: 97 pgs: 97 active+clean; 15 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-11T22:23:57.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:57 smithi159 ceph-mon[35534]: pgmap v55: 97 pgs: 97 active+clean; 15 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-11T22:23:57.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:57 smithi172 ceph-mon[35461]: pgmap v55: 97 pgs: 97 active+clean; 15 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.4 MiB/s wr, 268 op/s 2024-09-11T22:23:57.776 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:23:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:58.303 INFO:tasks.workunit.client.0.smithi005.stdout: 1 46767 16.85 MB/sec warmup 16 sec latency 22.007 ms 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.589+0000 7f83bda8a640 1 -- 172.21.15.5:0/102475878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b8102d30 msgr2=0x7f83b8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.589+0000 7f83bda8a640 1 --2- 172.21.15.5:0/102475878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b8102d30 0x7f83b8103110 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009930 tx=0x7f83a802a420 comp rx=0 tx=0).stop 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.590+0000 7f83bda8a640 1 -- 172.21.15.5:0/102475878 shutdown_connections 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.590+0000 7f83bda8a640 1 --2- 172.21.15.5:0/102475878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83b81082a0 0x7f83b810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.590+0000 7f83bda8a640 1 --2- 172.21.15.5:0/102475878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b81036e0 0x7f83b8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.590+0000 7f83bda8a640 1 --2- 172.21.15.5:0/102475878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b8102d30 0x7f83b8103110 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.590+0000 7f83bda8a640 1 -- 172.21.15.5:0/102475878 >> 172.21.15.5:0/102475878 conn(0x7f83b80fc7d0 msgr2=0x7f83b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.590+0000 7f83bda8a640 1 -- 172.21.15.5:0/102475878 shutdown_connections 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.590+0000 7f83bda8a640 1 -- 172.21.15.5:0/102475878 wait complete. 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.590+0000 7f83bda8a640 1 Processor -- start 2024-09-11T22:23:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.590+0000 7f83bda8a640 1 -- start start 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83bda8a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83b8102d30 0x7f83b8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83bda8a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b81036e0 0x7f83b8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83bda8a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b81082a0 0x7f83b8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83bda8a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83b81120a0 con 0x7f83b81082a0 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83bda8a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8111f20 con 0x7f83b81036e0 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83bda8a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83b8112220 con 0x7f83b8102d30 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b81036e0 0x7f83b8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b81082a0 0x7f83b8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b81036e0 0x7f83b8100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48708/0 (socket says 172.21.15.5:48708) 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b6ffd640 1 -- 172.21.15.5:0/3875316620 learned_addr learned my addr 172.21.15.5:0/3875316620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b77fe640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83b8102d30 0x7f83b8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b7fff640 1 -- 172.21.15.5:0/3875316620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83b8102d30 msgr2=0x7f83b8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b7fff640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83b8102d30 0x7f83b8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b7fff640 1 -- 172.21.15.5:0/3875316620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b81036e0 msgr2=0x7f83b8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b7fff640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b81036e0 0x7f83b8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b7fff640 1 -- 172.21.15.5:0/3875316620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b81a6a80 con 0x7f83b81082a0 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b81036e0 0x7f83b8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.591+0000 7f83b7fff640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b81082a0 0x7f83b8101410 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009fd0 tx=0x7f83b000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.592+0000 7f83b4ff9640 1 -- 172.21.15.5:0/3875316620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b000ce20 con 0x7f83b81082a0 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.592+0000 7f83bda8a640 1 -- 172.21.15.5:0/3875316620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b81a6d70 con 0x7f83b81082a0 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.592+0000 7f83b4ff9640 1 -- 172.21.15.5:0/3875316620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b0005020 con 0x7f83b81082a0 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.592+0000 7f83b4ff9640 1 -- 172.21.15.5:0/3875316620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0010720 con 0x7f83b81082a0 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.592+0000 7f83bda8a640 1 -- 172.21.15.5:0/3875316620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b81a72d0 con 0x7f83b81082a0 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.594+0000 7f83b4ff9640 1 -- 172.21.15.5:0/3875316620 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f83b00108c0 con 0x7f83b81082a0 2024-09-11T22:23:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.594+0000 7f83bfe63740 1 Processor -- start 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.594+0000 7f83bfe63740 1 -- start start 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696eb40 0x556cf696d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf696da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf6978370 0x556cf697a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83bfe63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cf68d32f0 con 0x556cf696eb40 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83bfe63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556cf687acf0 con 0x556cf696d5d0 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83bfe63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556cf68cf2b0 con 0x556cf6978370 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696eb40 0x556cf696d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696eb40 0x556cf696d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44316/0 (socket says 172.21.15.5:44316) 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b6ffd640 1 -- 172.21.15.5:0/1866197203 learned_addr learned my addr 172.21.15.5:0/1866197203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b77fe640 1 --2- 172.21.15.5:0/1866197203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf696da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/1866197203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf6978370 0x556cf697a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b6ffd640 1 -- 172.21.15.5:0/1866197203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf6978370 msgr2=0x556cf697a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/1866197203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf6978370 0x556cf697a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b6ffd640 1 -- 172.21.15.5:0/1866197203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 msgr2=0x556cf696da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/1866197203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf696da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b6ffd640 1 -- 172.21.15.5:0/1866197203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf685f3a0 con 0x556cf696eb40 2024-09-11T22:23:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b4ff9640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83a40a2d70 0x7f83a40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/1866197203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696eb40 0x556cf696d090 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f83a00139d0 tx=0x7f83a0009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9dbd95339cabb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b77fe640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83a40a2d70 0x7f83a40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.595+0000 7f83b4ff9640 1 -- 172.21.15.5:0/3875316620 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f83b00cf830 con 0x7f83b81082a0 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.596+0000 7f83adffb640 1 -- 172.21.15.5:0/1866197203 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a000fa00 con 0x556cf696eb40 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.596+0000 7f83adffb640 1 -- 172.21.15.5:0/1866197203 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83a000fba0 con 0x556cf696eb40 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.596+0000 7f83adffb640 1 -- 172.21.15.5:0/1866197203 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0003c20 con 0x556cf696eb40 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 -- 172.21.15.5:0/1866197203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696eb40 msgr2=0x556cf696d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 --2- 172.21.15.5:0/1866197203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696eb40 0x556cf696d090 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f83a00139d0 tx=0x7f83a0009c60 comp rx=0 tx=0).stop 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83b77fe640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83a40a2d70 0x7f83a40a5230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009900 tx=0x7f83a80058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 -- 172.21.15.5:0/1866197203 shutdown_connections 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 --2- 172.21.15.5:0/1866197203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf6978370 0x556cf697a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 --2- 172.21.15.5:0/1866197203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf696da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 --2- 172.21.15.5:0/1866197203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696eb40 0x556cf696d090 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 -- 172.21.15.5:0/1866197203 >> 172.21.15.5:0/1866197203 conn(0x556cf6936310 msgr2=0x556cf6969c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 -- 172.21.15.5:0/1866197203 shutdown_connections 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 -- 172.21.15.5:0/1866197203 wait complete. 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 -- start start 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.597+0000 7f83bfe63740 1 Processor -- start 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf6977dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6975080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 0x556cf6975810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83bfe63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cf697b070 con 0x556cf6978370 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83bfe63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556cf688e060 con 0x556cf696d5d0 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83bfe63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556cf697b1f0 con 0x556cf696eb40 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83bfe63740 10 client.0 _ll_register_callbacks cb 0x556cf65e18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6975080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf6977dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6975080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56832/0 (socket says 172.21.15.5:56832) 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b6ffd640 1 -- 172.21.15.5:0/684301885 learned_addr learned my addr 172.21.15.5:0/684301885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b7fff640 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 0x556cf6975810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b77fe640 1 -- 172.21.15.5:0/684301885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 msgr2=0x556cf6975080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b77fe640 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6975080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b77fe640 1 -- 172.21.15.5:0/684301885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 msgr2=0x556cf6975810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b77fe640 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 0x556cf6975810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b77fe640 1 -- 172.21.15.5:0/684301885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf66443c0 con 0x556cf696d5d0 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6975080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f83b77fe640 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf6977dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83a8005940 tx=0x7f83a8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6626b36454fdb7ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.598+0000 7f8396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f8396ffd640 1 -- 172.21.15.5:0/684301885 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a804e070 con 0x556cf696d5d0 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f8396ffd640 1 -- 172.21.15.5:0/684301885 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f83a8003bc0 con 0x556cf696d5d0 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f83bfe63740 1 -- 172.21.15.5:0/684301885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83a000fba0 con 0x556cf696d5d0 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f8396ffd640 1 -- 172.21.15.5:0/684301885 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83a805a050 con 0x556cf696d5d0 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f83bfe63740 10 client.25475 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f83bfe63740 1 -- 172.21.15.5:0/684301885 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556cf6986bf0 con 0x556cf696d5d0 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f8396ffd640 1 -- 172.21.15.5:0/684301885 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83a000fba0 con 0x556cf696d5d0 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f8396ffd640 1 -- 172.21.15.5:0/684301885 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f83a803d450 con 0x556cf696d5d0 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f83bfe63740 10 client.25475 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f83bfe63740 10 client.25475 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:23:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f83bfe63740 20 client.25475 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f83bfe63740 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556cf69880c0 0x556cf698a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f83bfe63740 4 client.25475 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.599+0000 7f83bfe63740 1 -- 172.21.15.5:0/684301885 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cf68e0860 con 0x556cf69880c0 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.600+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556cf69880c0 0x556cf698a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.600+0000 7f8396ffd640 10 client.25475 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.600+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556cf69880c0 0x556cf698a560 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19c691b8101c50 server_cookie=7a4ffb1a489eeade in_seq=0 out_seq=0 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f8396ffd640 1 -- 172.21.15.5:0/684301885 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cf68e0860 con 0x556cf69880c0 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f8396ffd640 10 client.25475 handle_command_reply: tid=0 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 client.25475 shutdown 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 -- 172.21.15.5:0/684301885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 msgr2=0x556cf6977dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf6977dd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f83a8005940 tx=0x7f83a8009260 comp rx=0 tx=0).stop 2024-09-11T22:23:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 -- 172.21.15.5:0/684301885 shutdown_connections 2024-09-11T22:23:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556cf69880c0 0x556cf698a560 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f8396ffd640 0 client.25475 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:23:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 0x556cf6975810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6975080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 --2- 172.21.15.5:0/684301885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf6977dd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 -- 172.21.15.5:0/684301885 >> 172.21.15.5:0/684301885 conn(0x556cf6936310 msgr2=0x556cf696f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 -- 172.21.15.5:0/684301885 shutdown_connections 2024-09-11T22:23:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.601+0000 7f83bfe63740 1 -- 172.21.15.5:0/684301885 wait complete. 2024-09-11T22:23:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.602+0000 7f83bfe63740 20 client.25475 trim_cache size 0 max 16384 2024-09-11T22:23:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.615+0000 7f83bfe63740 1 Processor -- start 2024-09-11T22:23:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.615+0000 7f83bfe63740 1 -- start start 2024-09-11T22:23:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.615+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf698a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf698fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 0x556cf6990370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83bfe63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f839800e8d0 con 0x556cf6978370 2024-09-11T22:23:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83bfe63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83980104d0 con 0x556cf696d5d0 2024-09-11T22:23:58.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83bfe63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f839800efa0 con 0x556cf696eb40 2024-09-11T22:23:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 0x556cf6990370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf698fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 0x556cf6990370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44336/0 (socket says 172.21.15.5:44336) 2024-09-11T22:23:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf698fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56834/0 (socket says 172.21.15.5:56834) 2024-09-11T22:23:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b7fff640 1 -- 172.21.15.5:0/3498521241 learned_addr learned my addr 172.21.15.5:0/3498521241 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf698a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b7fff640 1 -- 172.21.15.5:0/3498521241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 msgr2=0x556cf698fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b7fff640 1 --2- 172.21.15.5:0/3498521241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf698fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b7fff640 1 -- 172.21.15.5:0/3498521241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 msgr2=0x556cf698a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:58.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b7fff640 1 --2- 172.21.15.5:0/3498521241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf698a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b7fff640 1 -- 172.21.15.5:0/3498521241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf6984370 con 0x556cf6978370 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b77fe640 1 --2- 172.21.15.5:0/3498521241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf698a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83b7fff640 1 --2- 172.21.15.5:0/3498521241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 0x556cf6990370 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005af0 tx=0x7f83b00cf6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18788dc6c39cca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.616+0000 7f83ad7fa640 1 -- 172.21.15.5:0/3498521241 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b00b8020 con 0x556cf6978370 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83ad7fa640 1 -- 172.21.15.5:0/3498521241 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b00a4980 con 0x556cf6978370 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83ad7fa640 1 -- 172.21.15.5:0/3498521241 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b0010bb0 con 0x556cf6978370 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83bfe63740 1 -- 172.21.15.5:0/3498521241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 msgr2=0x556cf6990370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83bfe63740 1 --2- 172.21.15.5:0/3498521241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 0x556cf6990370 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005af0 tx=0x7f83b00cf6b0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83bfe63740 1 -- 172.21.15.5:0/3498521241 shutdown_connections 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83bfe63740 1 --2- 172.21.15.5:0/3498521241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6978370 0x556cf6990370 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83bfe63740 1 --2- 172.21.15.5:0/3498521241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf698fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83bfe63740 1 --2- 172.21.15.5:0/3498521241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf696d5d0 0x556cf698a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83bfe63740 1 -- 172.21.15.5:0/3498521241 >> 172.21.15.5:0/3498521241 conn(0x556cf6936310 msgr2=0x556cf6964a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:58.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83bfe63740 1 -- 172.21.15.5:0/3498521241 shutdown_connections 2024-09-11T22:23:58.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.617+0000 7f83bfe63740 1 -- 172.21.15.5:0/3498521241 wait complete. 2024-09-11T22:23:58.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83bfe63740 1 -- start start 2024-09-11T22:23:58.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83bfe63740 1 Processor -- start 2024-09-11T22:23:58.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696d5d0 0x556cf69806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6980c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83bfe63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6978370 0x556cf69716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83bfe63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f839800fd40 con 0x556cf696d5d0 2024-09-11T22:23:58.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83bfe63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f839800f670 con 0x556cf6978370 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83bfe63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f839800c8b0 con 0x556cf696eb40 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83bfe63740 10 client.0 _ll_register_callbacks cb 0x7f83a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696d5d0 0x556cf69806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6980c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6978370 0x556cf69716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6980c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56844/0 (socket says 172.21.15.5:56844) 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83b7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6978370 0x556cf69716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48752/0 (socket says 172.21.15.5:48752) 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.618+0000 7f83b6ffd640 1 -- 172.21.15.5:0/1003153851 learned_addr learned my addr 172.21.15.5:0/1003153851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83b77fe640 1 -- 172.21.15.5:0/1003153851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 msgr2=0x556cf6980c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83b77fe640 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6980c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83b77fe640 1 -- 172.21.15.5:0/1003153851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6978370 msgr2=0x556cf69716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83b77fe640 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6978370 0x556cf69716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83b77fe640 1 -- 172.21.15.5:0/1003153851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf6971d50 con 0x556cf696d5d0 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6980c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83b7fff640 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6978370 0x556cf69716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83b77fe640 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696d5d0 0x556cf69806c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f83a803da10 tx=0x7f83a803d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c660ba1ad86fe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f8396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f8396ffd640 1 -- 172.21.15.5:0/1003153851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a804e040 con 0x556cf696d5d0 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f8396ffd640 1 -- 172.21.15.5:0/1003153851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f83a80039c0 con 0x556cf696d5d0 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83bfe63740 1 -- 172.21.15.5:0/1003153851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83b00a4980 con 0x556cf696d5d0 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f8396ffd640 1 -- 172.21.15.5:0/1003153851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83a8035040 con 0x556cf696d5d0 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83bfe63740 10 client.15939 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.619+0000 7f83bfe63740 1 -- 172.21.15.5:0/1003153851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556cf69f2780 con 0x556cf696d5d0 2024-09-11T22:23:58.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f8396ffd640 1 -- 172.21.15.5:0/1003153851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83b00a4980 con 0x556cf696d5d0 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f8396ffd640 1 -- 172.21.15.5:0/1003153851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f83a804a930 con 0x556cf696d5d0 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f83bfe63740 10 client.15939 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f83bfe63740 10 client.15939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f83bfe63740 20 client.15939 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f83bfe63740 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556cf69880c0 0x556cf69f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f83bfe63740 4 client.15939 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f83bfe63740 1 -- 172.21.15.5:0/1003153851 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cf68dad70 con 0x556cf69880c0 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556cf69880c0 0x556cf69f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f8396ffd640 10 client.15939 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.620+0000 7f83b6ffd640 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556cf69880c0 0x556cf69f4eb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cadb4891d98910c6 server_cookie=68a9d76c2a563cd2 in_seq=0 out_seq=0 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f8396ffd640 1 -- 172.21.15.5:0/1003153851 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cf68dad70 con 0x556cf69880c0 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f8396ffd640 10 client.15939 handle_command_reply: tid=0 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 client.15939 shutdown 2024-09-11T22:23:58.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 -- 172.21.15.5:0/1003153851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696d5d0 msgr2=0x556cf69806c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696d5d0 0x556cf69806c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f83a803da10 tx=0x7f83a803d680 comp rx=0 tx=0).stop 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 -- 172.21.15.5:0/1003153851 shutdown_connections 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556cf69880c0 0x556cf69f4eb0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6978370 0x556cf69716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556cf696eb40 0x556cf6980c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 --2- 172.21.15.5:0/1003153851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf696d5d0 0x556cf69806c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f8396ffd640 0 client.15939 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 -- 172.21.15.5:0/1003153851 >> 172.21.15.5:0/1003153851 conn(0x556cf6936310 msgr2=0x556cf6973950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 -- 172.21.15.5:0/1003153851 shutdown_connections 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 1 -- 172.21.15.5:0/1003153851 wait complete. 2024-09-11T22:23:58.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.621+0000 7f83bfe63740 20 client.15939 trim_cache size 0 max 16384 2024-09-11T22:23:58.635 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:23:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 -- 172.21.15.5:0/3875316620 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83a40a2d70 msgr2=0x7f83a40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83a40a2d70 0x7f83a40a5230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009900 tx=0x7f83a80058d0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 -- 172.21.15.5:0/3875316620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b81082a0 msgr2=0x7f83b8101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b81082a0 0x7f83b8101410 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009fd0 tx=0x7f83b000ef10 comp rx=0 tx=0).stop 2024-09-11T22:23:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 -- 172.21.15.5:0/3875316620 shutdown_connections 2024-09-11T22:23:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83b81082a0 0x7f83b8101410 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83a40a2d70 0x7f83a40a5230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83b81036e0 0x7f83b8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 --2- 172.21.15.5:0/3875316620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83b8102d30 0x7f83b8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 -- 172.21.15.5:0/3875316620 >> 172.21.15.5:0/3875316620 conn(0x7f83b80fc7d0 msgr2=0x7f83b8105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 -- 172.21.15.5:0/3875316620 shutdown_connections 2024-09-11T22:23:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:23:58.622+0000 7f83bda8a640 1 -- 172.21.15.5:0/3875316620 wait complete. 2024-09-11T22:23:59.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 49992 16.96 MB/sec warmup 17 sec latency 11.221 ms 2024-09-11T22:23:59.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:23:59.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:23:59 smithi005 ceph-mon[30943]: pgmap v56: 97 pgs: 97 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-09-11T22:23:59.578 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:23:59 smithi159 ceph-mon[35534]: pgmap v56: 97 pgs: 97 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-09-11T22:23:59.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:23:59 smithi172 ceph-mon[35461]: pgmap v56: 97 pgs: 97 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 2.1 MiB/s wr, 366 op/s 2024-09-11T22:23:59.652 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:23:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.947+0000 7fa2318c9640 1 -- 172.21.15.5:0/1279852528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c1035c0 msgr2=0x7fa22c10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.947+0000 7fa2318c9640 1 --2- 172.21.15.5:0/1279852528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c1035c0 0x7fa22c10baa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa220009900 tx=0x7fa22002a410 comp rx=0 tx=0).stop 2024-09-11T22:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.947+0000 7fa2318c9640 1 -- 172.21.15.5:0/1279852528 shutdown_connections 2024-09-11T22:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.947+0000 7fa2318c9640 1 --2- 172.21.15.5:0/1279852528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa22c10c0d0 0x7fa22c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.947+0000 7fa2318c9640 1 --2- 172.21.15.5:0/1279852528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c1035c0 0x7fa22c10baa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.947+0000 7fa2318c9640 1 --2- 172.21.15.5:0/1279852528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa22c102ca0 0x7fa22c103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.947+0000 7fa2318c9640 1 -- 172.21.15.5:0/1279852528 >> 172.21.15.5:0/1279852528 conn(0x7fa22c0fc6b0 msgr2=0x7fa22c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.947+0000 7fa2318c9640 1 -- 172.21.15.5:0/1279852528 shutdown_connections 2024-09-11T22:23:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.947+0000 7fa2318c9640 1 -- 172.21.15.5:0/1279852528 wait complete. 2024-09-11T22:23:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.947+0000 7fa2318c9640 1 Processor -- start 2024-09-11T22:23:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa2318c9640 1 -- start start 2024-09-11T22:23:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa2318c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c102ca0 0x7fa22c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa2318c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa22c10c0d0 0x7fa22c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa2318c9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa22c19ce50 0x7fa22c19d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa2318c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c111ed0 con 0x7fa22c102ca0 2024-09-11T22:23:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa2318c9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c111d50 con 0x7fa22c19ce50 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa2318c9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c112050 con 0x7fa22c10c0d0 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa22affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c102ca0 0x7fa22c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa22affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c102ca0 0x7fa22c198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44366/0 (socket says 172.21.15.5:44366) 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa22affd640 1 -- 172.21.15.5:0/2631772489 learned_addr learned my addr 172.21.15.5:0/2631772489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa22a7fc640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa22c10c0d0 0x7fa22c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa22c19ce50 0x7fa22c19d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa22affd640 1 -- 172.21.15.5:0/2631772489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa22c10c0d0 msgr2=0x7fa22c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa22affd640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa22c10c0d0 0x7fa22c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa22affd640 1 -- 172.21.15.5:0/2631772489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa22c19ce50 msgr2=0x7fa22c19d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa22affd640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa22c19ce50 0x7fa22c19d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.948+0000 7fa22affd640 1 -- 172.21.15.5:0/2631772489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c19d940 con 0x7fa22c102ca0 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.949+0000 7fa22affd640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c102ca0 0x7fa22c198590 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00d950 tx=0x7fa21c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.949+0000 7fa2308c7640 1 -- 172.21.15.5:0/2631772489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c017030 con 0x7fa22c102ca0 2024-09-11T22:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.949+0000 7fa2308c7640 1 -- 172.21.15.5:0/2631772489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa21c004020 con 0x7fa22c102ca0 2024-09-11T22:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.949+0000 7fa2318c9640 1 -- 172.21.15.5:0/2631772489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c19dc30 con 0x7fa22c102ca0 2024-09-11T22:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.949+0000 7fa2308c7640 1 -- 172.21.15.5:0/2631772489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21c0047a0 con 0x7fa22c102ca0 2024-09-11T22:23:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.949+0000 7fa2318c9640 1 -- 172.21.15.5:0/2631772489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c1a53f0 con 0x7fa22c102ca0 2024-09-11T22:23:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.951+0000 7fa2308c7640 1 -- 172.21.15.5:0/2631772489 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa21c0041d0 con 0x7fa22c102ca0 2024-09-11T22:23:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.951+0000 7fa234660740 1 Processor -- start 2024-09-11T22:23:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa234660740 1 -- start start 2024-09-11T22:23:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa234660740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe473ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa234660740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa234660740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 0x563cbe4810a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa234660740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563cbe3d82f0 con 0x563cbe474410 2024-09-11T22:23:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa234660740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563cbe380270 con 0x563cbe473af0 2024-09-11T22:23:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa234660740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563cbe3d42b0 con 0x563cbe47ecb0 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe473ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe473ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48778/0 (socket says 172.21.15.5:48778) 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22b7fe640 1 -- 172.21.15.5:0/1288605681 learned_addr learned my addr 172.21.15.5:0/1288605681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22a7fc640 1 --2- 172.21.15.5:0/1288605681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 0x563cbe4810a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa2308c7640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2140a2da0 0x7fa2140a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22b7fe640 1 -- 172.21.15.5:0/1288605681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 msgr2=0x563cbe4810a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/1288605681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 0x563cbe4810a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22b7fe640 1 -- 172.21.15.5:0/1288605681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 msgr2=0x563cbe47e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/1288605681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22b7fe640 1 -- 172.21.15.5:0/1288605681 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cbe37f340 con 0x563cbe473af0 2024-09-11T22:23:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.952+0000 7fa22affd640 1 --2- 172.21.15.5:0/1288605681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.953+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2140a2da0 0x7fa2140a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.953+0000 7fa2308c7640 1 -- 172.21.15.5:0/2631772489 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fa21c0cedc0 con 0x7fa22c102ca0 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.953+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/1288605681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe473ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa224009f50 tx=0x7fa2240118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d1e8542a9043e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.953+0000 7fa211ffb640 1 -- 172.21.15.5:0/1288605681 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224003bf0 con 0x563cbe473af0 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.953+0000 7fa211ffb640 1 -- 172.21.15.5:0/1288605681 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa224004630 con 0x563cbe473af0 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.953+0000 7fa211ffb640 1 -- 172.21.15.5:0/1288605681 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224019400 con 0x563cbe473af0 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa234660740 1 -- 172.21.15.5:0/1288605681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 msgr2=0x563cbe473ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2140a2da0 0x7fa2140a5260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa224012fd0 tx=0x7fa224019710 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa234660740 1 --2- 172.21.15.5:0/1288605681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe473ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa224009f50 tx=0x7fa2240118d0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa234660740 1 -- 172.21.15.5:0/1288605681 shutdown_connections 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa234660740 1 --2- 172.21.15.5:0/1288605681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 0x563cbe4810a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa234660740 1 --2- 172.21.15.5:0/1288605681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa234660740 1 --2- 172.21.15.5:0/1288605681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe473ed0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa234660740 1 -- 172.21.15.5:0/1288605681 >> 172.21.15.5:0/1288605681 conn(0x563cbe470060 msgr2=0x563cbe470440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa234660740 1 -- 172.21.15.5:0/1288605681 shutdown_connections 2024-09-11T22:23:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa234660740 1 -- 172.21.15.5:0/1288605681 wait complete. 2024-09-11T22:23:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.954+0000 7fa234660740 1 -- start start 2024-09-11T22:23:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa234660740 1 Processor -- start 2024-09-11T22:23:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa234660740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa234660740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe474410 0x563cbe47c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa234660740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe47ecb0 0x563cbe47c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa234660740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563cbe481880 con 0x563cbe47ecb0 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa234660740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563cbe361290 con 0x563cbe473af0 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa234660740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563cbe481a00 con 0x563cbe474410 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa234660740 10 client.0 _ll_register_callbacks cb 0x563cbe392140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe47ecb0 0x563cbe47c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe47ecb0 0x563cbe47c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44372/0 (socket says 172.21.15.5:44372) 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48784/0 (socket says 172.21.15.5:48784) 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22b7fe640 1 -- 172.21.15.5:0/530921871 learned_addr learned my addr 172.21.15.5:0/530921871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22a7fc640 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe474410 0x563cbe47c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22b7fe640 1 -- 172.21.15.5:0/530921871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe474410 msgr2=0x563cbe47c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe474410 0x563cbe47c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22b7fe640 1 -- 172.21.15.5:0/530921871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 msgr2=0x563cbe47bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.955+0000 7fa22b7fe640 1 -- 172.21.15.5:0/530921871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cbe0eb210 con 0x563cbe47ecb0 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.956+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe47ecb0 0x563cbe47c560 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa2240118d0 tx=0x7fa224017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9af6f3f12cdc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.956+0000 7fa20affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.956+0000 7fa20affd640 1 -- 172.21.15.5:0/530921871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224033020 con 0x563cbe47ecb0 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.956+0000 7fa20affd640 1 -- 172.21.15.5:0/530921871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fa22401f040 con 0x563cbe47ecb0 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.956+0000 7fa234660740 1 -- 172.21.15.5:0/530921871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa224004630 con 0x563cbe47ecb0 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.956+0000 7fa20affd640 1 -- 172.21.15.5:0/530921871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa22403e050 con 0x563cbe47ecb0 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.956+0000 7fa234660740 10 client.15951 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.956+0000 7fa234660740 1 -- 172.21.15.5:0/530921871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563cbe1b7620 con 0x563cbe47ecb0 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.956+0000 7fa20affd640 1 -- 172.21.15.5:0/530921871 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa224004630 con 0x563cbe47ecb0 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.956+0000 7fa20affd640 1 -- 172.21.15.5:0/530921871 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa22400fcc0 con 0x563cbe47ecb0 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.957+0000 7fa234660740 10 client.15951 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:59.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.957+0000 7fa234660740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.957+0000 7fa234660740 20 client.15951 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.957+0000 7fa234660740 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563cbe48eb40 0x563cbe490f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.957+0000 7fa234660740 4 client.15951 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.957+0000 7fa234660740 1 -- 172.21.15.5:0/530921871 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cbe3e5a20 con 0x563cbe48eb40 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.957+0000 7fa22affd640 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563cbe48eb40 0x563cbe490f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.957+0000 7fa22affd640 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563cbe48eb40 0x563cbe490f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3adc9957663cd978 server_cookie=98ee3d6f9e507aca in_seq=0 out_seq=0 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.957+0000 7fa20affd640 10 client.15951 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa20affd640 1 -- 172.21.15.5:0/530921871 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563cbe3e5a20 con 0x563cbe48eb40 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa20affd640 10 client.15951 handle_command_reply: tid=0 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 client.15951 shutdown 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 -- 172.21.15.5:0/530921871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe47ecb0 msgr2=0x563cbe47c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:59.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe47ecb0 0x563cbe47c560 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa2240118d0 tx=0x7fa224017000 comp rx=0 tx=0).stop 2024-09-11T22:23:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 -- 172.21.15.5:0/530921871 shutdown_connections 2024-09-11T22:23:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563cbe48eb40 0x563cbe490f30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe47ecb0 0x563cbe47c560 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa20affd640 0 client.15951 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:23:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe474410 0x563cbe47c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 --2- 172.21.15.5:0/530921871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 -- 172.21.15.5:0/530921871 >> 172.21.15.5:0/530921871 conn(0x563cbe470060 msgr2=0x563cbe475dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 -- 172.21.15.5:0/530921871 shutdown_connections 2024-09-11T22:23:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 1 -- 172.21.15.5:0/530921871 wait complete. 2024-09-11T22:23:59.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.958+0000 7fa234660740 20 client.15951 trim_cache size 0 max 16384 2024-09-11T22:23:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.974+0000 7fa234660740 1 Processor -- start 2024-09-11T22:23:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.974+0000 7fa234660740 1 -- start start 2024-09-11T22:23:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.974+0000 7fa234660740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.974+0000 7fa234660740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.974+0000 7fa234660740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 0x563cbe4967b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.974+0000 7fa234660740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1abe10 con 0x563cbe474410 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.974+0000 7fa234660740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1ada10 con 0x563cbe473af0 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.974+0000 7fa234660740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1ac4e0 con 0x563cbe47ecb0 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.974+0000 7fa22a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44384/0 (socket says 172.21.15.5:44384) 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22a7fc640 1 -- 172.21.15.5:0/2379642983 learned_addr learned my addr 172.21.15.5:0/2379642983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48796/0 (socket says 172.21.15.5:48796) 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/2379642983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 0x563cbe4967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22a7fc640 1 -- 172.21.15.5:0/2379642983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 msgr2=0x563cbe4967b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22a7fc640 1 --2- 172.21.15.5:0/2379642983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 0x563cbe4967b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22a7fc640 1 -- 172.21.15.5:0/2379642983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 msgr2=0x563cbe47a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22a7fc640 1 --2- 172.21.15.5:0/2379642983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22a7fc640 1 -- 172.21.15.5:0/2379642983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cbe4912a0 con 0x563cbe474410 2024-09-11T22:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22affd640 1 --2- 172.21.15.5:0/2379642983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/2379642983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 0x563cbe4967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa22a7fc640 1 --2- 172.21.15.5:0/2379642983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47c390 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa220005940 tx=0x7fa220032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9c5010ee2aa485 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.975+0000 7fa2117fa640 1 -- 172.21.15.5:0/2379642983 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220035040 con 0x563cbe474410 2024-09-11T22:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.976+0000 7fa2117fa640 1 -- 172.21.15.5:0/2379642983 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2200317b0 con 0x563cbe474410 2024-09-11T22:23:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.976+0000 7fa2117fa640 1 -- 172.21.15.5:0/2379642983 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220004770 con 0x563cbe474410 2024-09-11T22:23:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.976+0000 7fa234660740 1 -- 172.21.15.5:0/2379642983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 msgr2=0x563cbe47c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.976+0000 7fa234660740 1 --2- 172.21.15.5:0/2379642983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47c390 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa220005940 tx=0x7fa220032d30 comp rx=0 tx=0).stop 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.976+0000 7fa234660740 1 -- 172.21.15.5:0/2379642983 shutdown_connections 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.976+0000 7fa234660740 1 --2- 172.21.15.5:0/2379642983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe47ecb0 0x563cbe4967b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.976+0000 7fa234660740 1 --2- 172.21.15.5:0/2379642983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47c390 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.976+0000 7fa234660740 1 --2- 172.21.15.5:0/2379642983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe473af0 0x563cbe47a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.976+0000 7fa234660740 1 -- 172.21.15.5:0/2379642983 >> 172.21.15.5:0/2379642983 conn(0x563cbe470060 msgr2=0x563cbe47db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.976+0000 7fa234660740 1 -- 172.21.15.5:0/2379642983 shutdown_connections 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.977+0000 7fa234660740 1 -- 172.21.15.5:0/2379642983 wait complete. 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.977+0000 7fa234660740 1 -- start start 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.977+0000 7fa234660740 1 Processor -- start 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa234660740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe473af0 0x563cbe487320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa234660740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa234660740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe47ecb0 0x563cbe480010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa234660740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1ad280 con 0x563cbe474410 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa234660740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1acbb0 con 0x563cbe47ecb0 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa234660740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c1a9df0 con 0x563cbe473af0 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa234660740 10 client.0 _ll_register_callbacks cb 0x7fa224028070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44392/0 (socket says 172.21.15.5:44392) 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22a7fc640 1 -- 172.21.15.5:0/2888865976 learned_addr learned my addr 172.21.15.5:0/2888865976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe47ecb0 0x563cbe480010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22affd640 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe473af0 0x563cbe487320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22a7fc640 1 -- 172.21.15.5:0/2888865976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe473af0 msgr2=0x563cbe487320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22a7fc640 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe473af0 0x563cbe487320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22a7fc640 1 -- 172.21.15.5:0/2888865976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe47ecb0 msgr2=0x563cbe480010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22a7fc640 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe47ecb0 0x563cbe480010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22a7fc640 1 -- 172.21.15.5:0/2888865976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cbe474240 con 0x563cbe474410 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22b7fe640 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe47ecb0 0x563cbe480010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa22a7fc640 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47fad0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa220006010 tx=0x7fa220009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baaa2351054c2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa20affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.978+0000 7fa20affd640 1 -- 172.21.15.5:0/2888865976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220035050 con 0x563cbe474410 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa20affd640 1 -- 172.21.15.5:0/2888865976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fa220004030 con 0x563cbe474410 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa234660740 1 -- 172.21.15.5:0/2888865976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2200317b0 con 0x563cbe474410 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa20affd640 1 -- 172.21.15.5:0/2888865976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa220005630 con 0x563cbe474410 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa234660740 10 client.15957 fetch_fsmap learned FSMap version 30 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa234660740 1 -- 172.21.15.5:0/2888865976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563cbe485c00 con 0x563cbe474410 2024-09-11T22:23:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa20affd640 1 -- 172.21.15.5:0/2888865976 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2200317b0 con 0x563cbe474410 2024-09-11T22:23:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa20affd640 1 -- 172.21.15.5:0/2888865976 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa220056030 con 0x563cbe474410 2024-09-11T22:23:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa234660740 10 client.15957 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:23:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa234660740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:23:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa234660740 20 client.15957 populate_metadata read hostname 'smithi005' 2024-09-11T22:23:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa234660740 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563cbe48eb40 0x563cbe4fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:23:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa234660740 4 client.15957 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:23:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa234660740 1 -- 172.21.15.5:0/2888865976 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563cbe3dfd70 con 0x563cbe48eb40 2024-09-11T22:23:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.979+0000 7fa22affd640 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563cbe48eb40 0x563cbe4fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:23:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.980+0000 7fa22affd640 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563cbe48eb40 0x563cbe4fbcf0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69e3929f0fd173d2 server_cookie=a45a52193a213c00 in_seq=0 out_seq=0 2024-09-11T22:23:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:23:59.980+0000 7fa20affd640 10 client.15957 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa20affd640 1 -- 172.21.15.5:0/2888865976 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563cbe3dfd70 con 0x563cbe48eb40 2024-09-11T22:24:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa20affd640 10 client.15957 handle_command_reply: tid=0 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 client.15957 shutdown 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 -- 172.21.15.5:0/2888865976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 msgr2=0x563cbe47fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47fad0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa220006010 tx=0x7fa220009520 comp rx=0 tx=0).stop 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 -- 172.21.15.5:0/2888865976 shutdown_connections 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563cbe48eb40 0x563cbe4fbcf0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563cbe47ecb0 0x563cbe480010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563cbe474410 0x563cbe47fad0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 --2- 172.21.15.5:0/2888865976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563cbe473af0 0x563cbe487320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa20affd640 0 client.15957 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 -- 172.21.15.5:0/2888865976 >> 172.21.15.5:0/2888865976 conn(0x563cbe470060 msgr2=0x563cbe46e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 -- 172.21.15.5:0/2888865976 shutdown_connections 2024-09-11T22:24:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.115+0000 7fa234660740 1 -- 172.21.15.5:0/2888865976 wait complete. 2024-09-11T22:24:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa234660740 20 client.15957 trim_cache size 0 max 16384 2024-09-11T22:24:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:24:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:24:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:24:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:24:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 -- 172.21.15.5:0/2631772489 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2140a2da0 msgr2=0x7fa2140a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2140a2da0 0x7fa2140a5260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa224012fd0 tx=0x7fa224019710 comp rx=0 tx=0).stop 2024-09-11T22:24:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 -- 172.21.15.5:0/2631772489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c102ca0 msgr2=0x7fa22c198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c102ca0 0x7fa22c198590 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00d950 tx=0x7fa21c00de10 comp rx=0 tx=0).stop 2024-09-11T22:24:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 -- 172.21.15.5:0/2631772489 shutdown_connections 2024-09-11T22:24:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2140a2da0 0x7fa2140a5260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa22c19ce50 0x7fa22c19d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa22c10c0d0 0x7fa22c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 --2- 172.21.15.5:0/2631772489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa22c102ca0 0x7fa22c198590 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 -- 172.21.15.5:0/2631772489 >> 172.21.15.5:0/2631772489 conn(0x7fa22c0fc6b0 msgr2=0x7fa22c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 -- 172.21.15.5:0/2631772489 shutdown_connections 2024-09-11T22:24:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:00.116+0000 7fa2318c9640 1 -- 172.21.15.5:0/2631772489 wait complete. 2024-09-11T22:24:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.227+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/3405374354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e58105ab0 msgr2=0x7f4e58105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.227+0000 7f4e5d6e2640 1 --2- 172.21.15.5:0/3405374354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e58105ab0 0x7f4e58105e90 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009900 tx=0x7f4e4802a410 comp rx=0 tx=0).stop 2024-09-11T22:24:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.227+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/3405374354 shutdown_connections 2024-09-11T22:24:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.227+0000 7f4e5d6e2640 1 --2- 172.21.15.5:0/3405374354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e58101b80 0x7f4e5810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.227+0000 7f4e5d6e2640 1 --2- 172.21.15.5:0/3405374354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e581011e0 0x7f4e58101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.227+0000 7f4e5d6e2640 1 --2- 172.21.15.5:0/3405374354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e58105ab0 0x7f4e58105e90 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.227+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/3405374354 >> 172.21.15.5:0/3405374354 conn(0x7f4e580fbee0 msgr2=0x7f4e580fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.227+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/3405374354 shutdown_connections 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.227+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/3405374354 wait complete. 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.228+0000 7f4e5d6e2640 1 Processor -- start 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.228+0000 7f4e5d6e2640 1 -- start start 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.228+0000 7f4e5d6e2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e581011e0 0x7f4e5819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.228+0000 7f4e5d6e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e58101b80 0x7f4e5819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.228+0000 7f4e5d6e2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e58105ab0 0x7f4e5819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.228+0000 7f4e5d6e2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58110ed0 con 0x7f4e58101b80 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.228+0000 7f4e5d6e2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58110d50 con 0x7f4e58105ab0 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.228+0000 7f4e5d6e2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e58111050 con 0x7f4e581011e0 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e581011e0 0x7f4e5819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e567fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e58101b80 0x7f4e5819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e581011e0 0x7f4e5819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56918/0 (socket says 172.21.15.5:56918) 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e56ffd640 1 -- 172.21.15.5:0/1989540198 learned_addr learned my addr 172.21.15.5:0/1989540198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e577fe640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e58105ab0 0x7f4e5819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e56ffd640 1 -- 172.21.15.5:0/1989540198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e58105ab0 msgr2=0x7f4e5819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e58105ab0 0x7f4e5819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e56ffd640 1 -- 172.21.15.5:0/1989540198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e58101b80 msgr2=0x7f4e5819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e58101b80 0x7f4e5819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e56ffd640 1 -- 172.21.15.5:0/1989540198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5819f500 con 0x7f4e581011e0 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e577fe640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e58105ab0 0x7f4e5819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e567fc640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e58101b80 0x7f4e5819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e581011e0 0x7f4e5819a550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e48006010 tx=0x7f4e48003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:00.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e3ffff640 1 -- 172.21.15.5:0/1989540198 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4803f070 con 0x7f4e581011e0 2024-09-11T22:24:00.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e3ffff640 1 -- 172.21.15.5:0/1989540198 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e48031c30 con 0x7f4e581011e0 2024-09-11T22:24:00.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.229+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/1989540198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5819f790 con 0x7f4e581011e0 2024-09-11T22:24:00.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.230+0000 7f4e3ffff640 1 -- 172.21.15.5:0/1989540198 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48035030 con 0x7f4e581011e0 2024-09-11T22:24:00.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.230+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/1989540198 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581a70d0 con 0x7f4e581011e0 2024-09-11T22:24:00.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.232+0000 7f4e3ffff640 1 -- 172.21.15.5:0/1989540198 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4e48042070 con 0x7f4e581011e0 2024-09-11T22:24:00.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.232+0000 7f4e5fabb740 1 Processor -- start 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e5fabb740 1 -- start start 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e3ffff640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e400a2d70 0x7f4e400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdc0b40 0x5571ebdbf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdbf5d0 0x5571ebdbfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdca370 0x5571ebdcc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571ebd252f0 con 0x5571ebdca370 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5571ebccccf0 con 0x5571ebdc0b40 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5571ebd212b0 con 0x5571ebdbf5d0 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e577fe640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e400a2d70 0x7f4e400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdc0b40 0x5571ebdbf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdca370 0x5571ebdcc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdc0b40 0x5571ebdbf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48832/0 (socket says 172.21.15.5:48832) 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e56ffd640 1 -- 172.21.15.5:0/1256437682 learned_addr learned my addr 172.21.15.5:0/1256437682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e3ffff640 1 -- 172.21.15.5:0/1989540198 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f4e480efad0 con 0x7f4e581011e0 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e567fc640 1 --2- 172.21.15.5:0/1256437682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdbf5d0 0x5571ebdbfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e56ffd640 1 -- 172.21.15.5:0/1256437682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdbf5d0 msgr2=0x5571ebdbfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/1256437682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdbf5d0 0x5571ebdbfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e56ffd640 1 -- 172.21.15.5:0/1256437682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdca370 msgr2=0x5571ebdcc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/1256437682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdca370 0x5571ebdcc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e56ffd640 1 -- 172.21.15.5:0/1256437682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ebcb13a0 con 0x5571ebdc0b40 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.233+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/1256437682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdc0b40 0x5571ebdbf090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4e48047040 tx=0x7f4e48005b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=188c5c636bd9fa18 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.234+0000 7f4e577fe640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e400a2d70 0x7f4e400a5230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4e581066f0 tx=0x7f4e50011720 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.234+0000 7f4e3d7fa640 1 -- 172.21.15.5:0/1256437682 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48049070 con 0x5571ebdc0b40 2024-09-11T22:24:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.234+0000 7f4e3d7fa640 1 -- 172.21.15.5:0/1256437682 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e480c5a80 con 0x5571ebdc0b40 2024-09-11T22:24:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.234+0000 7f4e3d7fa640 1 -- 172.21.15.5:0/1256437682 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e480c4bc0 con 0x5571ebdc0b40 2024-09-11T22:24:00.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.234+0000 7f4e5fabb740 1 -- 172.21.15.5:0/1256437682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdc0b40 msgr2=0x5571ebdbf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.234+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/1256437682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdc0b40 0x5571ebdbf090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4e48047040 tx=0x7f4e48005b70 comp rx=0 tx=0).stop 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.234+0000 7f4e5fabb740 1 -- 172.21.15.5:0/1256437682 shutdown_connections 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.234+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/1256437682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdca370 0x5571ebdcc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.234+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/1256437682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdbf5d0 0x5571ebdbfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.234+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/1256437682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdc0b40 0x5571ebdbf090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 -- 172.21.15.5:0/1256437682 >> 172.21.15.5:0/1256437682 conn(0x5571ebd88310 msgr2=0x5571ebdbbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 -- 172.21.15.5:0/1256437682 shutdown_connections 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 -- 172.21.15.5:0/1256437682 wait complete. 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 -- start start 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 Processor -- start 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdc9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdc0b40 0x5571ebdc7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdca370 0x5571ebdc7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571ebdcce30 con 0x5571ebdc0b40 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5571ebce0060 con 0x5571ebdbf5d0 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5571ebdccfb0 con 0x5571ebdca370 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e5fabb740 10 client.0 _ll_register_callbacks cb 0x5571eba338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e567fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdc0b40 0x5571ebdc7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdc9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e567fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdc0b40 0x5571ebdc7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44424/0 (socket says 172.21.15.5:44424) 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e567fc640 1 -- 172.21.15.5:0/411266460 learned_addr learned my addr 172.21.15.5:0/411266460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdca370 0x5571ebdc7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.235+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdc9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48842/0 (socket says 172.21.15.5:48842) 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e56ffd640 1 -- 172.21.15.5:0/411266460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdca370 msgr2=0x5571ebdc7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdca370 0x5571ebdc7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e56ffd640 1 -- 172.21.15.5:0/411266460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdc0b40 msgr2=0x5571ebdc7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdc0b40 0x5571ebdc7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e56ffd640 1 -- 172.21.15.5:0/411266460 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571eba963c0 con 0x5571ebdbf5d0 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e577fe640 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdca370 0x5571ebdc7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e567fc640 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdc0b40 0x5571ebdc7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdc9d20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x5571ebdc1170 tx=0x7f4e480092a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540d5add215cb7b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e3aff5640 1 -- 172.21.15.5:0/411266460 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48049070 con 0x5571ebdbf5d0 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e3aff5640 1 -- 172.21.15.5:0/411266460 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f4e480c91b0 con 0x5571ebdbf5d0 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e5fabb740 1 -- 172.21.15.5:0/411266460 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e480c5a80 con 0x5571ebdbf5d0 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e3aff5640 1 -- 172.21.15.5:0/411266460 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e480c93b0 con 0x5571ebdbf5d0 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e5fabb740 10 client.25511 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.236+0000 7f4e5fabb740 1 -- 172.21.15.5:0/411266460 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5571ebdcce30 con 0x5571ebdbf5d0 2024-09-11T22:24:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.237+0000 7f4e3aff5640 1 -- 172.21.15.5:0/411266460 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e480c5a80 con 0x5571ebdbf5d0 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.237+0000 7f4e3aff5640 1 -- 172.21.15.5:0/411266460 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4e480bc2f0 con 0x5571ebdbf5d0 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.237+0000 7f4e5fabb740 10 client.25511 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.237+0000 7f4e5fabb740 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.237+0000 7f4e5fabb740 20 client.25511 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.237+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5571ebdd9980 0x5571ebddbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.237+0000 7f4e5fabb740 4 client.25511 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.237+0000 7f4e5fabb740 1 -- 172.21.15.5:0/411266460 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571ebd32860 con 0x5571ebdd9980 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.237+0000 7f4e567fc640 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5571ebdd9980 0x5571ebddbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.238+0000 7f4e567fc640 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5571ebdd9980 0x5571ebddbd70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fccadba5e10d456c server_cookie=89ad3f88f399d15e in_seq=0 out_seq=0 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.238+0000 7f4e3aff5640 10 client.25511 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.238+0000 7f4e3aff5640 1 -- 172.21.15.5:0/411266460 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571ebd32860 con 0x5571ebdd9980 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.238+0000 7f4e3aff5640 10 client.25511 handle_command_reply: tid=0 2024-09-11T22:24:00.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.238+0000 7f4e5fabb740 1 client.25511 shutdown 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 1 -- 172.21.15.5:0/411266460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 msgr2=0x5571ebdc9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdc9d20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x5571ebdc1170 tx=0x7f4e480092a0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 1 -- 172.21.15.5:0/411266460 shutdown_connections 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5571ebdd9980 0x5571ebddbd70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdca370 0x5571ebdc7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdc0b40 0x5571ebdc7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e3aff5640 0 client.25511 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/411266460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdc9d20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 1 -- 172.21.15.5:0/411266460 >> 172.21.15.5:0/411266460 conn(0x5571ebd88310 msgr2=0x5571ebdc19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 1 -- 172.21.15.5:0/411266460 shutdown_connections 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 1 -- 172.21.15.5:0/411266460 wait complete. 2024-09-11T22:24:00.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.239+0000 7f4e5fabb740 20 client.25511 trim_cache size 0 max 16384 2024-09-11T22:24:00.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.253+0000 7f4e5fabb740 1 Processor -- start 2024-09-11T22:24:00.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.253+0000 7f4e5fabb740 1 -- start start 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdc21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdca370 0x5571ebdc26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581afba0 con 0x5571ebdca370 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b17a0 con 0x5571ebdbf5d0 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b0270 con 0x5571ebdc0b40 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e567fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdc21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdca370 0x5571ebdc26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e567fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdc21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56950/0 (socket says 172.21.15.5:56950) 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e567fc640 1 -- 172.21.15.5:0/511711295 learned_addr learned my addr 172.21.15.5:0/511711295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:00.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48844/0 (socket says 172.21.15.5:48844) 2024-09-11T22:24:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e567fc640 1 -- 172.21.15.5:0/511711295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 msgr2=0x5571ebdd6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e567fc640 1 --2- 172.21.15.5:0/511711295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e567fc640 1 -- 172.21.15.5:0/511711295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdca370 msgr2=0x5571ebdc26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e567fc640 1 --2- 172.21.15.5:0/511711295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdca370 0x5571ebdc26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e567fc640 1 -- 172.21.15.5:0/511711295 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ebdc79a0 con 0x5571ebdc0b40 2024-09-11T22:24:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.254+0000 7f4e567fc640 1 --2- 172.21.15.5:0/511711295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdc21b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5571ebdc68b0 tx=0x7f4e4c00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cea929fe4d6a190 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e3cff9640 1 -- 172.21.15.5:0/511711295 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c005220 con 0x5571ebdc0b40 2024-09-11T22:24:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e3cff9640 1 -- 172.21.15.5:0/511711295 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4c010d40 con 0x5571ebdc0b40 2024-09-11T22:24:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e3cff9640 1 -- 172.21.15.5:0/511711295 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c004d70 con 0x5571ebdc0b40 2024-09-11T22:24:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e5fabb740 1 -- 172.21.15.5:0/511711295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 msgr2=0x5571ebdc21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/511711295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdc21b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5571ebdc68b0 tx=0x7f4e4c00b390 comp rx=0 tx=0).stop 2024-09-11T22:24:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e5fabb740 1 -- 172.21.15.5:0/511711295 shutdown_connections 2024-09-11T22:24:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/511711295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdca370 0x5571ebdc26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/511711295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdc21b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/511711295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e5fabb740 1 -- 172.21.15.5:0/511711295 >> 172.21.15.5:0/511711295 conn(0x5571ebd88310 msgr2=0x5571ebdc6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.255+0000 7f4e5fabb740 1 -- 172.21.15.5:0/511711295 shutdown_connections 2024-09-11T22:24:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e5fabb740 1 -- 172.21.15.5:0/511711295 wait complete. 2024-09-11T22:24:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e5fabb740 1 -- start start 2024-09-11T22:24:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e5fabb740 1 Processor -- start 2024-09-11T22:24:00.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdd2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e5fabb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdca370 0x5571ebdc3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b1010 con 0x5571ebdbf5d0 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581b0940 con 0x5571ebdca370 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e5fabb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e581adb80 con 0x5571ebdc0b40 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e56ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44438/0 (socket says 172.21.15.5:44438) 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e56ffd640 1 -- 172.21.15.5:0/753419265 learned_addr learned my addr 172.21.15.5:0/753419265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e5fabb740 10 client.0 _ll_register_callbacks cb 0x7f4e4c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e567fc640 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdd2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.256+0000 7f4e577fe640 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdca370 0x5571ebdc3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e56ffd640 1 -- 172.21.15.5:0/753419265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 msgr2=0x5571ebdd2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:00.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdd2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e56ffd640 1 -- 172.21.15.5:0/753419265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdca370 msgr2=0x5571ebdc3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:00.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdca370 0x5571ebdc3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e56ffd640 1 -- 172.21.15.5:0/753419265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571ebdc3ef0 con 0x5571ebdbf5d0 2024-09-11T22:24:00.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e567fc640 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdd2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:00.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e56ffd640 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd23e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e480ed6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f362cdf7ea53c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:00.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:00.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e3aff5640 1 -- 172.21.15.5:0/753419265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48049070 con 0x5571ebdbf5d0 2024-09-11T22:24:00.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e5fabb740 1 -- 172.21.15.5:0/753419265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e4c010d40 con 0x5571ebdbf5d0 2024-09-11T22:24:00.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e3aff5640 1 -- 172.21.15.5:0/753419265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f4e480c91b0 con 0x5571ebdbf5d0 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e3aff5640 1 -- 172.21.15.5:0/753419265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e480bbdd0 con 0x5571ebdbf5d0 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e5fabb740 10 client.15975 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.257+0000 7f4e5fabb740 1 -- 172.21.15.5:0/753419265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5571ebdc3ef0 con 0x5571ebdbf5d0 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.258+0000 7f4e3aff5640 1 -- 172.21.15.5:0/753419265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e4c010d40 con 0x5571ebdbf5d0 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.258+0000 7f4e3aff5640 1 -- 172.21.15.5:0/753419265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4e480bb450 con 0x5571ebdbf5d0 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.258+0000 7f4e5fabb740 10 client.15975 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.258+0000 7f4e5fabb740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.258+0000 7f4e5fabb740 20 client.15975 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.258+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5571ebe44310 0x5571ebe46730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.258+0000 7f4e5fabb740 4 client.15975 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.258+0000 7f4e5fabb740 1 -- 172.21.15.5:0/753419265 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571ebd2cd70 con 0x5571ebe44310 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.258+0000 7f4e567fc640 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5571ebe44310 0x5571ebe46730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.259+0000 7f4e567fc640 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5571ebe44310 0x5571ebe46730 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9818f82a3e37e358 server_cookie=6004435aefb47264 in_seq=0 out_seq=0 2024-09-11T22:24:00.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.259+0000 7f4e3aff5640 10 client.15975 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.259+0000 7f4e3aff5640 1 -- 172.21.15.5:0/753419265 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571ebd2cd70 con 0x5571ebe44310 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.259+0000 7f4e3aff5640 10 client.15975 handle_command_reply: tid=0 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.259+0000 7f4e5fabb740 1 client.15975 shutdown 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.259+0000 7f4e5fabb740 1 -- 172.21.15.5:0/753419265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdbf5d0 msgr2=0x5571ebdd23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.259+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd23e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f4e48000c00 tx=0x7f4e480ed6b0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5fabb740 1 -- 172.21.15.5:0/753419265 shutdown_connections 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5571ebe44310 0x5571ebe46730 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571ebdca370 0x5571ebdc3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5571ebdc0b40 0x5571ebdd2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5fabb740 1 --2- 172.21.15.5:0/753419265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571ebdbf5d0 0x5571ebdd23e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5fabb740 1 -- 172.21.15.5:0/753419265 >> 172.21.15.5:0/753419265 conn(0x5571ebd88310 msgr2=0x5571ebdba4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:00.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e3aff5640 0 client.15975 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:00.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5fabb740 1 -- 172.21.15.5:0/753419265 shutdown_connections 2024-09-11T22:24:00.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5fabb740 1 -- 172.21.15.5:0/753419265 wait complete. 2024-09-11T22:24:00.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5fabb740 20 client.15975 trim_cache size 0 max 16384 2024-09-11T22:24:00.266 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:24:00.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/1989540198 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e400a2d70 msgr2=0x7f4e400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5d6e2640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e400a2d70 0x7f4e400a5230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4e581066f0 tx=0x7f4e50011720 comp rx=0 tx=0).stop 2024-09-11T22:24:00.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/1989540198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e581011e0 msgr2=0x7f4e5819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:00.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.260+0000 7f4e5d6e2640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e581011e0 0x7f4e5819a550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e48006010 tx=0x7f4e48003c10 comp rx=0 tx=0).stop 2024-09-11T22:24:00.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.261+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/1989540198 shutdown_connections 2024-09-11T22:24:00.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.261+0000 7f4e5d6e2640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e400a2d70 0x7f4e400a5230 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.261+0000 7f4e5d6e2640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e58105ab0 0x7f4e5819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.261+0000 7f4e5d6e2640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e58101b80 0x7f4e5819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.261+0000 7f4e5d6e2640 1 --2- 172.21.15.5:0/1989540198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e581011e0 0x7f4e5819a550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:00.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.261+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/1989540198 >> 172.21.15.5:0/1989540198 conn(0x7f4e580fbee0 msgr2=0x7f4e5810c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:00.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.261+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/1989540198 shutdown_connections 2024-09-11T22:24:00.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:00.261+0000 7f4e5d6e2640 1 -- 172.21.15.5:0/1989540198 wait complete. 2024-09-11T22:24:00.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 52585 16.70 MB/sec warmup 18 sec latency 52.297 ms 2024-09-11T22:24:01.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 54149 16.24 MB/sec warmup 19 sec latency 29.920 ms 2024-09-11T22:24:01.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:01 smithi005 ceph-mon[30943]: pgmap v57: 97 pgs: 97 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-09-11T22:24:01.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:01 smithi159 ceph-mon[35534]: pgmap v57: 97 pgs: 97 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-09-11T22:24:01.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:01 smithi172 ceph-mon[35461]: pgmap v57: 97 pgs: 97 active+clean; 22 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-09-11T22:24:02.087 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:24:02.087 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:24:02.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 57200 16.35 MB/sec warmup 20 sec latency 8.631 ms 2024-09-11T22:24:02.360 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:03.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.989+0000 7f3061f50640 1 -- 172.21.15.5:0/2686386676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f305c106f40 msgr2=0x7f305c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:03.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.989+0000 7f3061f50640 1 --2- 172.21.15.5:0/2686386676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f305c106f40 0x7f305c10eac0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3054009ca0 tx=0x7f305402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.989+0000 7f3061f50640 1 -- 172.21.15.5:0/2686386676 shutdown_connections 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.989+0000 7f3061f50640 1 --2- 172.21.15.5:0/2686386676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f305c106f40 0x7f305c10eac0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.989+0000 7f3061f50640 1 --2- 172.21.15.5:0/2686386676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f305c1027c0 0x7f305c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.989+0000 7f3061f50640 1 --2- 172.21.15.5:0/2686386676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f305c101e10 0x7f305c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.989+0000 7f3061f50640 1 -- 172.21.15.5:0/2686386676 >> 172.21.15.5:0/2686386676 conn(0x7f305c0fd8d0 msgr2=0x7f305c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.990+0000 7f3061f50640 1 -- 172.21.15.5:0/2686386676 shutdown_connections 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.990+0000 7f3061f50640 1 -- 172.21.15.5:0/2686386676 wait complete. 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.990+0000 7f3061f50640 1 Processor -- start 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.990+0000 7f3061f50640 1 -- start start 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.990+0000 7f3061f50640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f305c101e10 0x7f305c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.990+0000 7f3061f50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f305c1027c0 0x7f305c19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f3061f50640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f305c106f40 0x7f305c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f3061f50640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f305c111fa0 con 0x7f305c1027c0 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f3061f50640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f305c111e20 con 0x7f305c106f40 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f3061f50640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f305c112120 con 0x7f305c101e10 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f305c1027c0 0x7f305c19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f305c1027c0 0x7f305c19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55070/0 (socket says 172.21.15.5:55070) 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305affd640 1 -- 172.21.15.5:0/4019135 learned_addr learned my addr 172.21.15.5:0/4019135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f305c106f40 0x7f305c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305b7fe640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f305c101e10 0x7f305c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305affd640 1 -- 172.21.15.5:0/4019135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f305c101e10 msgr2=0x7f305c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305affd640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f305c101e10 0x7f305c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305affd640 1 -- 172.21.15.5:0/4019135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f305c106f40 msgr2=0x7f305c19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305affd640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f305c106f40 0x7f305c19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305affd640 1 -- 172.21.15.5:0/4019135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305c19f610 con 0x7f305c1027c0 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f305affd640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f305c1027c0 0x7f305c19aba0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f305000e9f0 tx=0x7f305000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f3058ff9640 1 -- 172.21.15.5:0/4019135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305000cd80 con 0x7f305c1027c0 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f3061f50640 1 -- 172.21.15.5:0/4019135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305c19f900 con 0x7f305c1027c0 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f3058ff9640 1 -- 172.21.15.5:0/4019135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3050010910 con 0x7f305c1027c0 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f3058ff9640 1 -- 172.21.15.5:0/4019135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500188f0 con 0x7f305c1027c0 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.991+0000 7f3061f50640 1 -- 172.21.15.5:0/4019135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305c1a71e0 con 0x7f305c1027c0 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.994+0000 7f3058ff9640 1 -- 172.21.15.5:0/4019135 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3050010ac0 con 0x7f305c1027c0 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.994+0000 7f3064329740 1 Processor -- start 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.994+0000 7f3064329740 1 -- start start 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f3064329740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f3064329740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 0x56218b6bc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f3064329740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6bcd40 0x56218b6bf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f3064329740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56218b617280 con 0x56218b6b1b60 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f3064329740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56218b5beb10 con 0x56218b6bcd40 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f3064329740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56218b613240 con 0x56218b6b2480 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55076/0 (socket says 172.21.15.5:55076) 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305bfff640 1 -- 172.21.15.5:0/2775094820 learned_addr learned my addr 172.21.15.5:0/2775094820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305affd640 1 --2- 172.21.15.5:0/2775094820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6bcd40 0x56218b6bf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305b7fe640 1 --2- 172.21.15.5:0/2775094820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 0x56218b6bc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f3058ff9640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30440a2cf0 0x7f30440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305bfff640 1 -- 172.21.15.5:0/2775094820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 msgr2=0x56218b6bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305bfff640 1 --2- 172.21.15.5:0/2775094820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 0x56218b6bc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305bfff640 1 -- 172.21.15.5:0/2775094820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6bcd40 msgr2=0x56218b6bf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:03.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305bfff640 1 --2- 172.21.15.5:0/2775094820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6bcd40 0x56218b6bf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305bfff640 1 -- 172.21.15.5:0/2775094820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218b5cb1f0 con 0x56218b6b1b60 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305b7fe640 1 --2- 172.21.15.5:0/2775094820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 0x56218b6bc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f3058ff9640 1 -- 172.21.15.5:0/4019135 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f30500cf810 con 0x7f305c1027c0 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305bfff640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30440a2cf0 0x7f30440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.995+0000 7f305bfff640 1 --2- 172.21.15.5:0/2775094820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b1f40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3054009ca0 tx=0x7f3054009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a2de71c0ea057f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.996+0000 7f3041ffb640 1 -- 172.21.15.5:0/2775094820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054042040 con 0x56218b6b1b60 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.996+0000 7f3041ffb640 1 -- 172.21.15.5:0/2775094820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3054036040 con 0x56218b6b1b60 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.996+0000 7f305bfff640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30440a2cf0 0x7f30440a51b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f305404d000 tx=0x7f305404e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.996+0000 7f3041ffb640 1 -- 172.21.15.5:0/2775094820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305402f950 con 0x56218b6b1b60 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 -- 172.21.15.5:0/2775094820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 msgr2=0x56218b6b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 --2- 172.21.15.5:0/2775094820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b1f40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3054009ca0 tx=0x7f3054009800 comp rx=0 tx=0).stop 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 -- 172.21.15.5:0/2775094820 shutdown_connections 2024-09-11T22:24:03.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 --2- 172.21.15.5:0/2775094820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6bcd40 0x56218b6bf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 --2- 172.21.15.5:0/2775094820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 0x56218b6bc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 --2- 172.21.15.5:0/2775094820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b1f40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 -- 172.21.15.5:0/2775094820 >> 172.21.15.5:0/2775094820 conn(0x56218b6ae0f0 msgr2=0x56218b6ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 -- 172.21.15.5:0/2775094820 shutdown_connections 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 -- 172.21.15.5:0/2775094820 wait complete. 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 -- start start 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.997+0000 7f3064329740 1 Processor -- start 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f3064329740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f3064329740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 0x56218b6ba130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f3064329740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6bcd40 0x56218b6ba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f3064329740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56218b6bf910 con 0x56218b6b1b60 2024-09-11T22:24:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f3064329740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56218b5d1f10 con 0x56218b6bcd40 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f3064329740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56218b6bfa90 con 0x56218b6b2480 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f3064329740 10 client.0 _ll_register_callbacks cb 0x56218b59aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55080/0 (socket says 172.21.15.5:55080) 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305b7fe640 1 -- 172.21.15.5:0/164619862 learned_addr learned my addr 172.21.15.5:0/164619862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305bfff640 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6bcd40 0x56218b6ba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305affd640 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 0x56218b6ba130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305b7fe640 1 -- 172.21.15.5:0/164619862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 msgr2=0x56218b6ba130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305b7fe640 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 0x56218b6ba130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305b7fe640 1 -- 172.21.15.5:0/164619862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6bcd40 msgr2=0x56218b6ba670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305b7fe640 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6bcd40 0x56218b6ba670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305b7fe640 1 -- 172.21.15.5:0/164619862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218b3886a0 con 0x56218b6b1b60 2024-09-11T22:24:03.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305affd640 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 0x56218b6ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f305b7fe640 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b9bf0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f304c0126e0 tx=0x7f304c012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a67537de1de6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.998+0000 7f303affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f303affd640 1 -- 172.21.15.5:0/164619862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c0121d0 con 0x56218b6b1b60 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f303affd640 1 -- 172.21.15.5:0/164619862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f304c004050 con 0x56218b6b1b60 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f3064329740 1 -- 172.21.15.5:0/164619862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3054036040 con 0x56218b6b1b60 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f303affd640 1 -- 172.21.15.5:0/164619862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f304c004250 con 0x56218b6b1b60 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f3064329740 10 client.15987 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f3064329740 1 -- 172.21.15.5:0/164619862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56218b6cb590 con 0x56218b6b1b60 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f303affd640 1 -- 172.21.15.5:0/164619862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3054036040 con 0x56218b6b1b60 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f303affd640 1 -- 172.21.15.5:0/164619862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f304c003a60 con 0x56218b6b1b60 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f3064329740 10 client.15987 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:03.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f3064329740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f3064329740 20 client.15987 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f3064329740 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56218b6ccaf0 0x56218b6cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f3064329740 4 client.15987 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:02.999+0000 7f3064329740 1 -- 172.21.15.5:0/164619862 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56218b6247f0 con 0x56218b6ccaf0 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.000+0000 7f305affd640 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56218b6ccaf0 0x56218b6cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.000+0000 7f303affd640 10 client.15987 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.000+0000 7f305affd640 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56218b6ccaf0 0x56218b6cef90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe2d76fe3d3644a server_cookie=10e7901b61ae0d71 in_seq=0 out_seq=0 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.001+0000 7f303affd640 1 -- 172.21.15.5:0/164619862 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56218b6247f0 con 0x56218b6ccaf0 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.001+0000 7f303affd640 10 client.15987 handle_command_reply: tid=0 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 client.15987 shutdown 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 -- 172.21.15.5:0/164619862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 msgr2=0x56218b6b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b9bf0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f304c0126e0 tx=0x7f304c012ba0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 -- 172.21.15.5:0/164619862 shutdown_connections 2024-09-11T22:24:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56218b6ccaf0 0x56218b6cef90 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6bcd40 0x56218b6ba670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b2480 0x56218b6ba130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f303affd640 0 client.15987 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 --2- 172.21.15.5:0/164619862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6b9bf0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 -- 172.21.15.5:0/164619862 >> 172.21.15.5:0/164619862 conn(0x56218b6ae0f0 msgr2=0x56218b6ad270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 -- 172.21.15.5:0/164619862 shutdown_connections 2024-09-11T22:24:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 1 -- 172.21.15.5:0/164619862 wait complete. 2024-09-11T22:24:03.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.002+0000 7f3064329740 20 client.15987 trim_cache size 0 max 16384 2024-09-11T22:24:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f3064329740 1 Processor -- start 2024-09-11T22:24:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f3064329740 1 -- start start 2024-09-11T22:24:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f3064329740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6bba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f3064329740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 0x56218b6d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f3064329740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6bcd40 0x56218b6d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f3064329740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1afb50 con 0x56218b6b1b60 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f3064329740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1b1750 con 0x56218b6b2480 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f3064329740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1b0220 con 0x56218b6bcd40 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f305b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6bba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f305b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6bba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55092/0 (socket says 172.21.15.5:55092) 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f305bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6bcd40 0x56218b6d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f305b7fe640 1 -- 172.21.15.5:0/714210061 learned_addr learned my addr 172.21.15.5:0/714210061 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.022+0000 7f305affd640 1 --2- 172.21.15.5:0/714210061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 0x56218b6d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.023+0000 7f305b7fe640 1 -- 172.21.15.5:0/714210061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6bcd40 msgr2=0x56218b6d4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.023+0000 7f305b7fe640 1 --2- 172.21.15.5:0/714210061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6bcd40 0x56218b6d4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.023+0000 7f305b7fe640 1 -- 172.21.15.5:0/714210061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 msgr2=0x56218b6d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.023+0000 7f305b7fe640 1 --2- 172.21.15.5:0/714210061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 0x56218b6d4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.023+0000 7f305b7fe640 1 -- 172.21.15.5:0/714210061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218b6cf610 con 0x56218b6b1b60 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.023+0000 7f305bfff640 1 --2- 172.21.15.5:0/714210061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6bcd40 0x56218b6d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.023+0000 7f305b7fe640 1 --2- 172.21.15.5:0/714210061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6bba50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f304c009f30 tx=0x7f304c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f8bf9c51f5e2ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.023+0000 7f30417fa640 1 -- 172.21.15.5:0/714210061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c028040 con 0x56218b6b1b60 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.023+0000 7f30417fa640 1 -- 172.21.15.5:0/714210061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f304c00fe00 con 0x56218b6b1b60 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.023+0000 7f30417fa640 1 -- 172.21.15.5:0/714210061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c005290 con 0x56218b6b1b60 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.024+0000 7f3064329740 1 -- 172.21.15.5:0/714210061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 msgr2=0x56218b6bba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.024+0000 7f3064329740 1 --2- 172.21.15.5:0/714210061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6bba50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f304c009f30 tx=0x7f304c009f60 comp rx=0 tx=0).stop 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.024+0000 7f3064329740 1 -- 172.21.15.5:0/714210061 shutdown_connections 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.024+0000 7f3064329740 1 --2- 172.21.15.5:0/714210061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6bcd40 0x56218b6d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.024+0000 7f3064329740 1 --2- 172.21.15.5:0/714210061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 0x56218b6d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.024+0000 7f3064329740 1 --2- 172.21.15.5:0/714210061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6b1b60 0x56218b6bba50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.024+0000 7f3064329740 1 -- 172.21.15.5:0/714210061 >> 172.21.15.5:0/714210061 conn(0x56218b6ae0f0 msgr2=0x56218b6bb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.024+0000 7f3064329740 1 -- 172.21.15.5:0/714210061 shutdown_connections 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.024+0000 7f3064329740 1 -- 172.21.15.5:0/714210061 wait complete. 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.024+0000 7f3064329740 1 -- start start 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f3064329740 1 Processor -- start 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f3064329740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b1b60 0x56218b6b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f3064329740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 0x56218b6b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f3064329740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6bcd40 0x56218b6b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f3064329740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1b0fc0 con 0x56218b6bcd40 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f3064329740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1b08f0 con 0x56218b6b2480 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f3064329740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f305c1adb30 con 0x56218b6b1b60 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f3064329740 10 client.0 _ll_register_callbacks cb 0x7f304c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b1b60 0x56218b6b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b1b60 0x56218b6b6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:54936/0 (socket says 172.21.15.5:54936) 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305b7fe640 1 -- 172.21.15.5:0/3700559718 learned_addr learned my addr 172.21.15.5:0/3700559718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305affd640 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 0x56218b6b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305bfff640 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6bcd40 0x56218b6b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305b7fe640 1 -- 172.21.15.5:0/3700559718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 msgr2=0x56218b6b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305b7fe640 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 0x56218b6b7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305b7fe640 1 -- 172.21.15.5:0/3700559718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6bcd40 msgr2=0x56218b6b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305b7fe640 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6bcd40 0x56218b6b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305b7fe640 1 -- 172.21.15.5:0/3700559718 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218b736990 con 0x56218b6b1b60 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.025+0000 7f305affd640 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 0x56218b6b7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.026+0000 7f305b7fe640 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b1b60 0x56218b6b6d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f304c003dd0 tx=0x7f304c005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e75482511d2ceb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.026+0000 7f303affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.026+0000 7f303affd640 1 -- 172.21.15.5:0/3700559718 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c028040 con 0x56218b6b1b60 2024-09-11T22:24:03.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.026+0000 7f3064329740 1 -- 172.21.15.5:0/3700559718 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f304c00fe00 con 0x56218b6b1b60 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.026+0000 7f303affd640 1 -- 172.21.15.5:0/3700559718 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f304c00f920 con 0x56218b6b1b60 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.026+0000 7f303affd640 1 -- 172.21.15.5:0/3700559718 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f304c03a050 con 0x56218b6b1b60 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.026+0000 7f3064329740 10 client.25486 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.026+0000 7f3064329740 1 -- 172.21.15.5:0/3700559718 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56218b737c50 con 0x56218b6b1b60 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.026+0000 7f303affd640 1 -- 172.21.15.5:0/3700559718 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f304c00fe00 con 0x56218b6b1b60 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.027+0000 7f303affd640 1 -- 172.21.15.5:0/3700559718 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f304c010540 con 0x56218b6b1b60 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.027+0000 7f3064329740 10 client.25486 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.027+0000 7f3064329740 10 client.25486 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.027+0000 7f3064329740 20 client.25486 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.027+0000 7f3064329740 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56218b738300 0x56218b73a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.027+0000 7f3064329740 4 client.25486 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.027+0000 7f3064329740 1 -- 172.21.15.5:0/3700559718 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56218b61ed00 con 0x56218b738300 2024-09-11T22:24:03.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.027+0000 7f305affd640 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56218b738300 0x56218b73a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.027+0000 7f303affd640 10 client.25486 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.027+0000 7f305affd640 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56218b738300 0x56218b73a720 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea2ce4ceaa4e096 server_cookie=b9e5e5af60ffff1b in_seq=0 out_seq=0 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f303affd640 1 -- 172.21.15.5:0/3700559718 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56218b61ed00 con 0x56218b738300 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f303affd640 10 client.25486 handle_command_reply: tid=0 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f3064329740 1 client.25486 shutdown 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f3064329740 1 -- 172.21.15.5:0/3700559718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b1b60 msgr2=0x56218b6b6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f3064329740 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b1b60 0x56218b6b6d00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f304c003dd0 tx=0x7f304c005b90 comp rx=0 tx=0).stop 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f3064329740 1 -- 172.21.15.5:0/3700559718 shutdown_connections 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f3064329740 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56218b738300 0x56218b73a720 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f3064329740 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218b6bcd40 0x56218b6b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f3064329740 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56218b6b2480 0x56218b6b7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f3064329740 1 --2- 172.21.15.5:0/3700559718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56218b6b1b60 0x56218b6b6d00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f3064329740 1 -- 172.21.15.5:0/3700559718 >> 172.21.15.5:0/3700559718 conn(0x56218b6ae0f0 msgr2=0x56218b6ad270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:03.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.028+0000 7f303affd640 0 client.25486 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:03.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3064329740 1 -- 172.21.15.5:0/3700559718 shutdown_connections 2024-09-11T22:24:03.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3064329740 1 -- 172.21.15.5:0/3700559718 wait complete. 2024-09-11T22:24:03.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3064329740 20 client.25486 trim_cache size 0 max 16384 2024-09-11T22:24:03.041 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:24:03.041 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:24:03.041 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "08143139-142a-46b4-9f85-f1d1b2d12431", 2024-09-11T22:24:03.041 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:24:03.041 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:24:03.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3061f50640 1 -- 172.21.15.5:0/4019135 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30440a2cf0 msgr2=0x7f30440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:03.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3061f50640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30440a2cf0 0x7f30440a51b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f305404d000 tx=0x7f305404e000 comp rx=0 tx=0).stop 2024-09-11T22:24:03.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3061f50640 1 -- 172.21.15.5:0/4019135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f305c1027c0 msgr2=0x7f305c19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:03.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3061f50640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f305c1027c0 0x7f305c19aba0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f305000e9f0 tx=0x7f305000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3061f50640 1 -- 172.21.15.5:0/4019135 shutdown_connections 2024-09-11T22:24:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3061f50640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30440a2cf0 0x7f30440a51b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3061f50640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f305c106f40 0x7f305c19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3061f50640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f305c1027c0 0x7f305c19aba0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3061f50640 1 --2- 172.21.15.5:0/4019135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f305c101e10 0x7f305c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.029+0000 7f3061f50640 1 -- 172.21.15.5:0/4019135 >> 172.21.15.5:0/4019135 conn(0x7f305c0fd8d0 msgr2=0x7f305c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.030+0000 7f3061f50640 1 -- 172.21.15.5:0/4019135 shutdown_connections 2024-09-11T22:24:03.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:03.030+0000 7f3061f50640 1 -- 172.21.15.5:0/4019135 wait complete. 2024-09-11T22:24:03.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 60396 16.41 MB/sec warmup 21 sec latency 22.023 ms 2024-09-11T22:24:03.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:03 smithi005 ceph-mon[30943]: pgmap v58: 97 pgs: 97 active+clean; 17 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 457 op/s 2024-09-11T22:24:03.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:03 smithi159 ceph-mon[35534]: pgmap v58: 97 pgs: 97 active+clean; 17 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 457 op/s 2024-09-11T22:24:03.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:03 smithi172 ceph-mon[35461]: pgmap v58: 97 pgs: 97 active+clean; 17 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 457 op/s 2024-09-11T22:24:03.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:24:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:04.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 63257 16.30 MB/sec warmup 22 sec latency 11.584 ms 2024-09-11T22:24:04.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:04 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:24:04.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:04 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:04.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:04 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:24:04.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:04 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:24:04.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:04 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:04.478 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:04 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:04 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:24:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:04 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:04 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:24:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:04 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:24:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:04 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:04 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:04.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:04 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:24:04.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:04 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:04.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:04 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:24:04.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:04 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:24:04.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:04 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:04.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:04 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.563+0000 7f99114f5640 1 -- 172.21.15.5:0/2979339763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c101de0 msgr2=0x7f990c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.563+0000 7f99114f5640 1 --2- 172.21.15.5:0/2979339763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c101de0 0x7f990c1021c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009900 tx=0x7f98fc02a410 comp rx=0 tx=0).stop 2024-09-11T22:24:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.563+0000 7f99114f5640 1 -- 172.21.15.5:0/2979339763 shutdown_connections 2024-09-11T22:24:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.563+0000 7f99114f5640 1 --2- 172.21.15.5:0/2979339763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c106f10 0x7f990c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.563+0000 7f99114f5640 1 --2- 172.21.15.5:0/2979339763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c102790 0x7f990c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.563+0000 7f99114f5640 1 --2- 172.21.15.5:0/2979339763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c101de0 0x7f990c1021c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.563+0000 7f99114f5640 1 -- 172.21.15.5:0/2979339763 >> 172.21.15.5:0/2979339763 conn(0x7f990c0fd8c0 msgr2=0x7f990c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.563+0000 7f99114f5640 1 -- 172.21.15.5:0/2979339763 shutdown_connections 2024-09-11T22:24:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.563+0000 7f99114f5640 1 -- 172.21.15.5:0/2979339763 wait complete. 2024-09-11T22:24:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.563+0000 7f99114f5640 1 Processor -- start 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f99114f5640 1 -- start start 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f99114f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c101de0 0x7f990c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f99114f5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c102790 0x7f990c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f99114f5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c106f10 0x7f990c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f99114f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f990c111f70 con 0x7f990c101de0 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f99114f5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f990c111df0 con 0x7f990c106f10 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f99114f5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1120f0 con 0x7f990c102790 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f990affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c101de0 0x7f990c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f990affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c101de0 0x7f990c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55110/0 (socket says 172.21.15.5:55110) 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f990affd640 1 -- 172.21.15.5:0/608324076 learned_addr learned my addr 172.21.15.5:0/608324076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f990a7fc640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c102790 0x7f990c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f990b7fe640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c106f10 0x7f990c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f990affd640 1 -- 172.21.15.5:0/608324076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c102790 msgr2=0x7f990c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f990affd640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c102790 0x7f990c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f990affd640 1 -- 172.21.15.5:0/608324076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c106f10 msgr2=0x7f990c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f990affd640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c106f10 0x7f990c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.564+0000 7f990affd640 1 -- 172.21.15.5:0/608324076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c1a6970 con 0x7f990c101de0 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.565+0000 7f990affd640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c101de0 0x7f990c0744f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f98fc006010 tx=0x7f98fc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.565+0000 7f98f3fff640 1 -- 172.21.15.5:0/608324076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc03f030 con 0x7f990c101de0 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.565+0000 7f98f3fff640 1 -- 172.21.15.5:0/608324076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98fc004440 con 0x7f990c101de0 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.565+0000 7f99114f5640 1 -- 172.21.15.5:0/608324076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990c1a6c00 con 0x7f990c101de0 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.565+0000 7f98f3fff640 1 -- 172.21.15.5:0/608324076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc035070 con 0x7f990c101de0 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.565+0000 7f99114f5640 1 -- 172.21.15.5:0/608324076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990c1a70e0 con 0x7f990c101de0 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.567+0000 7f98f3fff640 1 -- 172.21.15.5:0/608324076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f98fc042030 con 0x7f990c101de0 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f99138ce740 1 Processor -- start 2024-09-11T22:24:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f99138ce740 1 -- start start 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f99138ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 0x5569dad03880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f99138ce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f99138ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad10a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f99138ce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5569dac692e0 con 0x5569dad03dc0 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f99138ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5569dac10ce0 con 0x5569dad034a0 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f99138ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5569dac652a0 con 0x5569dad0e680 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 0x5569dad03880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55116/0 (socket says 172.21.15.5:55116) 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990affd640 1 -- 172.21.15.5:0/1943564312 learned_addr learned my addr 172.21.15.5:0/1943564312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990a7fc640 1 --2- 172.21.15.5:0/1943564312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad10a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f98f3fff640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98f40a2cf0 0x7f98f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990b7fe640 1 -- 172.21.15.5:0/1943564312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 msgr2=0x5569dad10a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990b7fe640 1 --2- 172.21.15.5:0/1943564312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad10a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990b7fe640 1 -- 172.21.15.5:0/1943564312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 msgr2=0x5569dad0e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990b7fe640 1 --2- 172.21.15.5:0/1943564312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990b7fe640 1 -- 172.21.15.5:0/1943564312 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569dabf5440 con 0x5569dad034a0 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f98f3fff640 1 -- 172.21.15.5:0/608324076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f98fc0ef7d0 con 0x7f990c101de0 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.568+0000 7f990b7fe640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98f40a2cf0 0x7f98f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.569+0000 7f990b7fe640 1 --2- 172.21.15.5:0/1943564312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 0x5569dad03880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9904009fa0 tx=0x7f99040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9deedccde20bab17 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.569+0000 7f98f17fa640 1 -- 172.21.15.5:0/1943564312 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904017050 con 0x5569dad034a0 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.569+0000 7f990b7fe640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98f40a2cf0 0x7f98f40a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9904022000 tx=0x7f9904023000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.569+0000 7f98f17fa640 1 -- 172.21.15.5:0/1943564312 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9904003bb0 con 0x5569dad034a0 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.569+0000 7f98f17fa640 1 -- 172.21.15.5:0/1943564312 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990401b8d0 con 0x5569dad034a0 2024-09-11T22:24:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 -- 172.21.15.5:0/1943564312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 msgr2=0x5569dad03880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 --2- 172.21.15.5:0/1943564312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 0x5569dad03880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9904009fa0 tx=0x7f99040118d0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 -- 172.21.15.5:0/1943564312 shutdown_connections 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 --2- 172.21.15.5:0/1943564312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad10a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 --2- 172.21.15.5:0/1943564312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 --2- 172.21.15.5:0/1943564312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 0x5569dad03880 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 -- 172.21.15.5:0/1943564312 >> 172.21.15.5:0/1943564312 conn(0x5569dacffa30 msgr2=0x5569dacffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 -- 172.21.15.5:0/1943564312 shutdown_connections 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 -- 172.21.15.5:0/1943564312 wait complete. 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 -- start start 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.570+0000 7f99138ce740 1 Processor -- start 2024-09-11T22:24:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f99138ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 0x5569dad0dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f99138ce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f99138ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad0b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f99138ce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5569dad11250 con 0x5569dad03dc0 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f99138ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5569dac24050 con 0x5569dad034a0 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f99138ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5569dad113d0 con 0x5569dad0e680 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f99138ce740 10 client.0 _ll_register_callbacks cb 0x5569dac64160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 0x5569dad0dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55122/0 (socket says 172.21.15.5:55122) 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990a7fc640 1 -- 172.21.15.5:0/527453717 learned_addr learned my addr 172.21.15.5:0/527453717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990b7fe640 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad0b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990affd640 1 -- 172.21.15.5:0/527453717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 msgr2=0x5569dad0b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990affd640 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad0b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990affd640 1 -- 172.21.15.5:0/527453717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 msgr2=0x5569dad0b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990affd640 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990affd640 1 -- 172.21.15.5:0/527453717 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569da9da3c0 con 0x5569dad034a0 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.571+0000 7f990a7fc640 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f990affd640 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 0x5569dad0dfd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0bbcf0 tx=0x7f98fc0bbbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=515edbad0fd66112 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f98eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f98eeff5640 1 -- 172.21.15.5:0/527453717 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc049040 con 0x5569dad034a0 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f98eeff5640 1 -- 172.21.15.5:0/527453717 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f98fc0b83d0 con 0x5569dad034a0 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f99138ce740 1 -- 172.21.15.5:0/527453717 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5569dac76850 con 0x5569dad034a0 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f98eeff5640 1 -- 172.21.15.5:0/527453717 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f98fc0bb6c0 con 0x5569dad034a0 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f99138ce740 10 client.25547 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f99138ce740 1 -- 172.21.15.5:0/527453717 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5569dad1cc90 con 0x5569dad034a0 2024-09-11T22:24:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f98eeff5640 1 -- 172.21.15.5:0/527453717 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5569dac76850 con 0x5569dad034a0 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f98eeff5640 1 -- 172.21.15.5:0/527453717 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f98fc0bc050 con 0x5569dad034a0 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f99138ce740 10 client.25547 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f99138ce740 10 client.25547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f99138ce740 20 client.25547 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f99138ce740 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5569dad1e100 0x5569dad20560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.572+0000 7f99138ce740 4 client.25547 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.573+0000 7f99138ce740 1 -- 172.21.15.5:0/527453717 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569dac70d60 con 0x5569dad1e100 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.573+0000 7f990a7fc640 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5569dad1e100 0x5569dad20560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.573+0000 7f98eeff5640 10 client.25547 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.573+0000 7f990a7fc640 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5569dad1e100 0x5569dad20560 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee456247615cc5b6 server_cookie=d9fbfd1ef561780c in_seq=0 out_seq=0 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f98eeff5640 1 -- 172.21.15.5:0/527453717 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5569dac70d60 con 0x5569dad1e100 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f98eeff5640 10 client.25547 handle_command_reply: tid=0 2024-09-11T22:24:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 client.25547 shutdown 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 -- 172.21.15.5:0/527453717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 msgr2=0x5569dad0dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 0x5569dad0dfd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0bbcf0 tx=0x7f98fc0bbbe0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 -- 172.21.15.5:0/527453717 shutdown_connections 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5569dad1e100 0x5569dad20560 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad0b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad03dc0 0x5569dad0b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f98eeff5640 0 client.25547 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 --2- 172.21.15.5:0/527453717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad034a0 0x5569dad0dfd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 -- 172.21.15.5:0/527453717 >> 172.21.15.5:0/527453717 conn(0x5569dacffa30 msgr2=0x5569dacfeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 -- 172.21.15.5:0/527453717 shutdown_connections 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 1 -- 172.21.15.5:0/527453717 wait complete. 2024-09-11T22:24:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.574+0000 7f99138ce740 20 client.25547 trim_cache size 0 max 16384 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f99138ce740 1 Processor -- start 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f99138ce740 1 -- start start 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f99138ce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dacffe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f99138ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 0x5569dad1a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f99138ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad25e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f99138ce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1afb00 con 0x5569dad034a0 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f99138ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1b1700 con 0x5569dad03dc0 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f99138ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1b01d0 con 0x5569dad0e680 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f990affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dacffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f990affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dacffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55126/0 (socket says 172.21.15.5:55126) 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f990affd640 1 -- 172.21.15.5:0/425799550 learned_addr learned my addr 172.21.15.5:0/425799550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f990a7fc640 1 --2- 172.21.15.5:0/425799550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 0x5569dad1a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f990b7fe640 1 --2- 172.21.15.5:0/425799550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad25e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f990affd640 1 -- 172.21.15.5:0/425799550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 msgr2=0x5569dad25e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f990affd640 1 --2- 172.21.15.5:0/425799550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad25e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f990affd640 1 -- 172.21.15.5:0/425799550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 msgr2=0x5569dad1a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f990affd640 1 --2- 172.21.15.5:0/425799550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 0x5569dad1a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.593+0000 7f990affd640 1 -- 172.21.15.5:0/425799550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569dad0bbe0 con 0x5569dad034a0 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.594+0000 7f990affd640 1 --2- 172.21.15.5:0/425799550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dacffe10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0b81c0 tx=0x7f98fc0f5210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69f7444a29a15f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.594+0000 7f98f0ff9640 1 -- 172.21.15.5:0/425799550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc049040 con 0x5569dad034a0 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.594+0000 7f98f0ff9640 1 -- 172.21.15.5:0/425799550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98fc0bbd60 con 0x5569dad034a0 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.594+0000 7f98f0ff9640 1 -- 172.21.15.5:0/425799550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc0bbfc0 con 0x5569dad034a0 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 -- 172.21.15.5:0/425799550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 msgr2=0x5569dacffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 --2- 172.21.15.5:0/425799550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dacffe10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0b81c0 tx=0x7f98fc0f5210 comp rx=0 tx=0).stop 2024-09-11T22:24:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 -- 172.21.15.5:0/425799550 shutdown_connections 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 --2- 172.21.15.5:0/425799550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad25e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 --2- 172.21.15.5:0/425799550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 0x5569dad1a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 --2- 172.21.15.5:0/425799550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dacffe10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 -- 172.21.15.5:0/425799550 >> 172.21.15.5:0/425799550 conn(0x5569dacffa30 msgr2=0x5569dad068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 -- 172.21.15.5:0/425799550 shutdown_connections 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 -- 172.21.15.5:0/425799550 wait complete. 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 -- start start 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 Processor -- start 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dad16bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 0x5569dad11f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad12440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1b0f70 con 0x5569dad034a0 2024-09-11T22:24:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1b08a0 con 0x5569dad03dc0 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.595+0000 7f99138ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1adae0 con 0x5569dad0e680 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f99138ce740 10 client.0 _ll_register_callbacks cb 0x7f98fc03b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dad16bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990a7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 0x5569dad11f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dad16bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55142/0 (socket says 172.21.15.5:55142) 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990affd640 1 -- 172.21.15.5:0/1362832274 learned_addr learned my addr 172.21.15.5:0/1362832274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990b7fe640 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad12440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990affd640 1 -- 172.21.15.5:0/1362832274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 msgr2=0x5569dad12440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990affd640 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad12440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990affd640 1 -- 172.21.15.5:0/1362832274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 msgr2=0x5569dad11f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990affd640 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 0x5569dad11f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990affd640 1 -- 172.21.15.5:0/1362832274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569dad12a80 con 0x5569dad034a0 2024-09-11T22:24:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990a7fc640 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 0x5569dad11f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f990affd640 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dad16bd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f98fc005ab0 tx=0x7f98fc0b8680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bea293d64e1955 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f98eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f98eeff5640 1 -- 172.21.15.5:0/1362832274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98fc087060 con 0x5569dad034a0 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f98eeff5640 1 -- 172.21.15.5:0/1362832274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f98fc031bd0 con 0x5569dad034a0 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f99138ce740 1 -- 172.21.15.5:0/1362832274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98fc0bbd60 con 0x5569dad034a0 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f98eeff5640 1 -- 172.21.15.5:0/1362832274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f98fc049030 con 0x5569dad034a0 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f99138ce740 10 client.16011 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.596+0000 7f99138ce740 1 -- 172.21.15.5:0/1362832274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5569dad12a80 con 0x5569dad034a0 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.597+0000 7f98eeff5640 1 -- 172.21.15.5:0/1362832274 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98fc0bbd60 con 0x5569dad034a0 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.597+0000 7f98eeff5640 1 -- 172.21.15.5:0/1362832274 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f98fc0bb2c0 con 0x5569dad034a0 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.597+0000 7f99138ce740 10 client.16011 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.597+0000 7f99138ce740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.597+0000 7f99138ce740 20 client.16011 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.597+0000 7f99138ce740 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5569dad88f90 0x5569dad8b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.597+0000 7f99138ce740 4 client.16011 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.597+0000 7f99138ce740 1 -- 172.21.15.5:0/1362832274 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5569dabed210 con 0x5569dad88f90 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.597+0000 7f990a7fc640 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5569dad88f90 0x5569dad8b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.598+0000 7f990a7fc640 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5569dad88f90 0x5569dad8b3b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28272ae5a6468353 server_cookie=2e1f8dfefac93b52 in_seq=0 out_seq=0 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.598+0000 7f98eeff5640 10 client.16011 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f98eeff5640 1 -- 172.21.15.5:0/1362832274 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5569dabed210 con 0x5569dad88f90 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f98eeff5640 10 client.16011 handle_command_reply: tid=0 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f99138ce740 1 client.16011 shutdown 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f99138ce740 1 -- 172.21.15.5:0/1362832274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 msgr2=0x5569dad16bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f99138ce740 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dad16bd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f98fc005ab0 tx=0x7f98fc0b8680 comp rx=0 tx=0).stop 2024-09-11T22:24:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f99138ce740 1 -- 172.21.15.5:0/1362832274 shutdown_connections 2024-09-11T22:24:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f99138ce740 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5569dad88f90 0x5569dad8b3b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f99138ce740 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569dad0e680 0x5569dad12440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f99138ce740 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569dad03dc0 0x5569dad11f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f99138ce740 1 --2- 172.21.15.5:0/1362832274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569dad034a0 0x5569dad16bd0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f98eeff5640 0 client.16011 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.599+0000 7f99138ce740 1 -- 172.21.15.5:0/1362832274 >> 172.21.15.5:0/1362832274 conn(0x5569dacffa30 msgr2=0x5569dad05fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99138ce740 1 -- 172.21.15.5:0/1362832274 shutdown_connections 2024-09-11T22:24:04.865 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:24:04.865 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:24:04.866 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:24:04.866 INFO:teuthology.orchestra.run.smithi005.stdout: "08143139-142a-46b4-9f85-f1d1b2d12431": { 2024-09-11T22:24:04.866 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:24:04.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "08143139-142a-46b4-9f85-f1d1b2d12431", 2024-09-11T22:24:04.866 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:24:04.866 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:04.866 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:04.866 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:24:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99138ce740 1 -- 172.21.15.5:0/1362832274 wait complete. 2024-09-11T22:24:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99138ce740 20 client.16011 trim_cache size 0 max 16384 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 -- 172.21.15.5:0/608324076 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98f40a2cf0 msgr2=0x7f98f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98f40a2cf0 0x7f98f40a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9904022000 tx=0x7f9904023000 comp rx=0 tx=0).stop 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 -- 172.21.15.5:0/608324076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c101de0 msgr2=0x7f990c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c101de0 0x7f990c0744f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f98fc006010 tx=0x7f98fc004040 comp rx=0 tx=0).stop 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 -- 172.21.15.5:0/608324076 shutdown_connections 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98f40a2cf0 0x7f98f40a51b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c106f10 0x7f990c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c102790 0x7f990c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 --2- 172.21.15.5:0/608324076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c101de0 0x7f990c0744f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 -- 172.21.15.5:0/608324076 >> 172.21.15.5:0/608324076 conn(0x7f990c0fd8c0 msgr2=0x7f990c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 -- 172.21.15.5:0/608324076 shutdown_connections 2024-09-11T22:24:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:04.600+0000 7f99114f5640 1 -- 172.21.15.5:0/608324076 wait complete. 2024-09-11T22:24:05.305 INFO:tasks.workunit.client.0.smithi005.stdout: 1 66054 16.13 MB/sec warmup 23 sec latency 14.415 ms 2024-09-11T22:24:05.485 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:05 smithi005 ceph-mon[30943]: pgmap v59: 97 pgs: 97 active+clean; 17 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 350 op/s 2024-09-11T22:24:05.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:05 smithi159 ceph-mon[35534]: pgmap v59: 97 pgs: 97 active+clean; 17 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 350 op/s 2024-09-11T22:24:05.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:05 smithi172 ceph-mon[35461]: pgmap v59: 97 pgs: 97 active+clean; 17 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 350 op/s 2024-09-11T22:24:05.823 INFO:tasks.cephfs.filesystem:scrub status for tag:08143139-142a-46b4-9f85-f1d1b2d12431 - {'path': '/', 'tag': '08143139-142a-46b4-9f85-f1d1b2d12431', 'options': 'recursive,force'} 2024-09-11T22:24:06.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 68513 16.09 MB/sec warmup 24 sec latency 10.149 ms 2024-09-11T22:24:07.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 71506 16.14 MB/sec warmup 25 sec latency 9.215 ms 2024-09-11T22:24:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:07 smithi005 ceph-mon[30943]: pgmap v60: 97 pgs: 97 active+clean; 17 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 350 op/s 2024-09-11T22:24:07.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:07 smithi159 ceph-mon[35534]: pgmap v60: 97 pgs: 97 active+clean; 17 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 350 op/s 2024-09-11T22:24:07.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:07 smithi172 ceph-mon[35461]: pgmap v60: 97 pgs: 97 active+clean; 17 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 350 op/s 2024-09-11T22:24:08.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 74776 16.24 MB/sec warmup 26 sec latency 10.111 ms 2024-09-11T22:24:09.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 77524 16.12 MB/sec warmup 27 sec latency 12.178 ms 2024-09-11T22:24:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:09 smithi005 ceph-mon[30943]: pgmap v61: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 456 op/s 2024-09-11T22:24:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:09 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:24:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:09 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:24:09.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:09 smithi159 ceph-mon[35534]: pgmap v61: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 456 op/s 2024-09-11T22:24:09.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:09 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:24:09.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:09 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:24:09.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:09 smithi172 ceph-mon[35461]: pgmap v61: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 456 op/s 2024-09-11T22:24:09.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:09 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:24:09.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:09 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:24:10.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 80425 16.02 MB/sec warmup 28 sec latency 14.613 ms 2024-09-11T22:24:11.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 82790 15.96 MB/sec warmup 29 sec latency 14.832 ms 2024-09-11T22:24:11.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:11 smithi005 ceph-mon[30943]: pgmap v62: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.6 MiB/s wr, 359 op/s 2024-09-11T22:24:11.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:11 smithi159 ceph-mon[35534]: pgmap v62: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.6 MiB/s wr, 359 op/s 2024-09-11T22:24:11.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:11 smithi172 ceph-mon[35461]: pgmap v62: 97 pgs: 97 active+clean; 25 MiB data, 486 MiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.6 MiB/s wr, 359 op/s 2024-09-11T22:24:12.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 85735 16.00 MB/sec warmup 30 sec latency 20.578 ms 2024-09-11T22:24:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.208+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3109210786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d981069c0 msgr2=0x7f6d98106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.208+0000 7f6d9cb23640 1 --2- 172.21.15.5:0/3109210786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d981069c0 0x7f6d98106da0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009930 tx=0x7f6d8802a420 comp rx=0 tx=0).stop 2024-09-11T22:24:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.208+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3109210786 shutdown_connections 2024-09-11T22:24:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.208+0000 7f6d9cb23640 1 --2- 172.21.15.5:0/3109210786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d98101350 0x7f6d9810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.208+0000 7f6d9cb23640 1 --2- 172.21.15.5:0/3109210786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d981009b0 0x7f6d98100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.208+0000 7f6d9cb23640 1 --2- 172.21.15.5:0/3109210786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d981069c0 0x7f6d98106da0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.208+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3109210786 >> 172.21.15.5:0/3109210786 conn(0x7f6d98077990 msgr2=0x7f6d980fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.208+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3109210786 shutdown_connections 2024-09-11T22:24:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.208+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3109210786 wait complete. 2024-09-11T22:24:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9cb23640 1 Processor -- start 2024-09-11T22:24:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9cb23640 1 -- start start 2024-09-11T22:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9cb23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d981009b0 0x7f6d9819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9cb23640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d98101350 0x7f6d9819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9cb23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d981069c0 0x7f6d9819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9cb23640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9810fd40 con 0x7f6d981009b0 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9cb23640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9810fbc0 con 0x7f6d98101350 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9cb23640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9810fec0 con 0x7f6d981069c0 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d981009b0 0x7f6d9819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d981009b0 0x7f6d9819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57898/0 (socket says 172.21.15.5:57898) 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d9659c640 1 -- 172.21.15.5:0/3565979585 learned_addr learned my addr 172.21.15.5:0/3565979585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d95d9b640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d98101350 0x7f6d9819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.209+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d981069c0 0x7f6d9819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d9659c640 1 -- 172.21.15.5:0/3565979585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d981069c0 msgr2=0x7f6d9819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d9659c640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d981069c0 0x7f6d9819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d9659c640 1 -- 172.21.15.5:0/3565979585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d98101350 msgr2=0x7f6d9819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d9659c640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d98101350 0x7f6d9819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d9659c640 1 -- 172.21.15.5:0/3565979585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9819f510 con 0x7f6d981009b0 2024-09-11T22:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d981069c0 0x7f6d9819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d9659c640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d981009b0 0x7f6d9819a520 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6d8802a940 tx=0x7f6d88003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d877fe640 1 -- 172.21.15.5:0/3565979585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8803f070 con 0x7f6d981009b0 2024-09-11T22:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d877fe640 1 -- 172.21.15.5:0/3565979585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d88031c90 con 0x7f6d981009b0 2024-09-11T22:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d877fe640 1 -- 172.21.15.5:0/3565979585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88035030 con 0x7f6d981009b0 2024-09-11T22:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3565979585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9819f7a0 con 0x7f6d981009b0 2024-09-11T22:24:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.210+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3565979585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d981a7090 con 0x7f6d981009b0 2024-09-11T22:24:13.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.212+0000 7f6d877fe640 1 -- 172.21.15.5:0/3565979585 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6d88042070 con 0x7f6d981009b0 2024-09-11T22:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d9f8ba740 1 Processor -- start 2024-09-11T22:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d9f8ba740 1 -- start start 2024-09-11T22:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d347053ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d34705e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 0x55d3470610a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d346fb82f0 con 0x55d347053af0 2024-09-11T22:24:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d346f60270 con 0x55d34705ecb0 2024-09-11T22:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d346fb42b0 con 0x55d347054410 2024-09-11T22:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d96d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d347053ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d96d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d347053ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57914/0 (socket says 172.21.15.5:57914) 2024-09-11T22:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d96d9d640 1 -- 172.21.15.5:0/1204946537 learned_addr learned my addr 172.21.15.5:0/1204946537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d877fe640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d800a2e20 0x7f6d800a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d9659c640 1 --2- 172.21.15.5:0/1204946537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d34705e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d95d9b640 1 --2- 172.21.15.5:0/1204946537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 0x55d3470610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.213+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d800a2e20 0x7f6d800a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d96d9d640 1 -- 172.21.15.5:0/1204946537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 msgr2=0x55d34705e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/1204946537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d34705e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d96d9d640 1 -- 172.21.15.5:0/1204946537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 msgr2=0x55d3470610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/1204946537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 0x55d3470610a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d96d9d640 1 -- 172.21.15.5:0/1204946537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d346f5f340 con 0x55d347053af0 2024-09-11T22:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d95d9b640 1 --2- 172.21.15.5:0/1204946537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 0x55d3470610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d877fe640 1 -- 172.21.15.5:0/3565979585 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f6d880efbb0 con 0x7f6d981009b0 2024-09-11T22:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/1204946537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d347053ed0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6d90005c40 tx=0x7f6d90012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730ab870bd7ce1ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d84ff9640 1 -- 172.21.15.5:0/1204946537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90023030 con 0x55d347053af0 2024-09-11T22:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d84ff9640 1 -- 172.21.15.5:0/1204946537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d9001fc90 con 0x55d347053af0 2024-09-11T22:24:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d84ff9640 1 -- 172.21.15.5:0/1204946537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000fc00 con 0x55d347053af0 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.214+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d800a2e20 0x7f6d800a52e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d90023000 tx=0x7f6d90022000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1204946537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 msgr2=0x55d347053ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/1204946537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d347053ed0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6d90005c40 tx=0x7f6d90012f80 comp rx=0 tx=0).stop 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1204946537 shutdown_connections 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/1204946537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 0x55d3470610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/1204946537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d34705e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/1204946537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d347053ed0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1204946537 >> 172.21.15.5:0/1204946537 conn(0x55d347050060 msgr2=0x55d347050440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1204946537 shutdown_connections 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1204946537 wait complete. 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 -- start start 2024-09-11T22:24:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.215+0000 7f6d9f8ba740 1 Processor -- start 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347053af0 0x55d34705bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d347054410 0x55d34705c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d34705c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d347061880 con 0x55d34705ecb0 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d346f41290 con 0x55d347054410 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d347061a00 con 0x55d347053af0 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d9f8ba740 10 client.0 _ll_register_callbacks cb 0x55d346f72140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d9659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347053af0 0x55d34705bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d96d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d34705c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d95d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d347054410 0x55d34705c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d96d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d34705c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57930/0 (socket says 172.21.15.5:57930) 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d9659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347053af0 0x55d34705bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60808/0 (socket says 172.21.15.5:60808) 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d96d9d640 1 -- 172.21.15.5:0/1207034429 learned_addr learned my addr 172.21.15.5:0/1207034429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d96d9d640 1 -- 172.21.15.5:0/1207034429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347053af0 msgr2=0x55d34705bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347053af0 0x55d34705bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d96d9d640 1 -- 172.21.15.5:0/1207034429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d347054410 msgr2=0x55d34705c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d347054410 0x55d34705c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d96d9d640 1 -- 172.21.15.5:0/1207034429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d346ccb210 con 0x55d34705ecb0 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d34705c680 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6d90005c10 tx=0x7f6d900124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f57f6b572ad88f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d7dffb640 1 -- 172.21.15.5:0/1207034429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90004a00 con 0x55d34705ecb0 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d7dffb640 1 -- 172.21.15.5:0/1207034429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f6d90004ba0 con 0x55d34705ecb0 2024-09-11T22:24:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.216+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1207034429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d9001fc90 con 0x55d34705ecb0 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d7dffb640 1 -- 172.21.15.5:0/1207034429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d90004da0 con 0x55d34705ecb0 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d9f8ba740 10 client.16023 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1207034429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d346d97620 con 0x55d34705ecb0 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d7dffb640 1 -- 172.21.15.5:0/1207034429 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d9001fc90 con 0x55d34705ecb0 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d7dffb640 1 -- 172.21.15.5:0/1207034429 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6d9002f020 con 0x55d34705ecb0 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d9f8ba740 10 client.16023 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d9f8ba740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d9f8ba740 20 client.16023 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d34706ec60 0x55d3470710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d9f8ba740 4 client.16023 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1207034429 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d346fc5a20 con 0x55d34706ec60 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.217+0000 7f6d9659c640 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d34706ec60 0x55d3470710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.218+0000 7f6d9659c640 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d34706ec60 0x55d3470710a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cf8898c2e3b17e5 server_cookie=310137e449f4cb1e in_seq=0 out_seq=0 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.218+0000 7f6d7dffb640 10 client.16023 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.218+0000 7f6d7dffb640 1 -- 172.21.15.5:0/1207034429 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6d8809ae50 con 0x55d34706ec60 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.218+0000 7f6d7dffb640 10 client.16023 handle_command_reply: tid=0 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.218+0000 7f6d9f8ba740 1 client.16023 shutdown 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1207034429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 msgr2=0x55d34705c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d34705c680 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f6d90005c10 tx=0x7f6d900124f0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1207034429 shutdown_connections 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d34706ec60 0x55d3470710a0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d34705c680 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d347054410 0x55d34705c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d7dffb640 0 client.16023 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/1207034429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347053af0 0x55d34705bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1207034429 >> 172.21.15.5:0/1207034429 conn(0x55d347050060 msgr2=0x55d347055d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:13.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1207034429 shutdown_connections 2024-09-11T22:24:13.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/1207034429 wait complete. 2024-09-11T22:24:13.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.219+0000 7f6d9f8ba740 20 client.16023 trim_cache size 0 max 16384 2024-09-11T22:24:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.234+0000 7f6d9f8ba740 1 Processor -- start 2024-09-11T22:24:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.234+0000 7f6d9f8ba740 1 -- start start 2024-09-11T22:24:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d347053af0 0x55d34705c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d34705db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d3470769b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981afb30 con 0x55d34705ecb0 2024-09-11T22:24:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981b1730 con 0x55d347053af0 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981b0200 con 0x55d347054410 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d96d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d3470769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d96d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d3470769b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57934/0 (socket says 172.21.15.5:57934) 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d96d9d640 1 -- 172.21.15.5:0/2343787335 learned_addr learned my addr 172.21.15.5:0/2343787335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d9659c640 1 --2- 172.21.15.5:0/2343787335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d347053af0 0x55d34705c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d95d9b640 1 --2- 172.21.15.5:0/2343787335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d34705db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d96d9d640 1 -- 172.21.15.5:0/2343787335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 msgr2=0x55d34705db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/2343787335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d34705db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d96d9d640 1 -- 172.21.15.5:0/2343787335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d347053af0 msgr2=0x55d34705c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/2343787335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d347053af0 0x55d34705c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d96d9d640 1 -- 172.21.15.5:0/2343787335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d347051110 con 0x55d34705ecb0 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d95d9b640 1 --2- 172.21.15.5:0/2343787335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d34705db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:13.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/2343787335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d3470769b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6d90006010 tx=0x7f6d90009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507d800efcd00a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d7ffff640 1 -- 172.21.15.5:0/2343787335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9001fc90 con 0x55d34705ecb0 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d7ffff640 1 -- 172.21.15.5:0/2343787335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d9000f5d0 con 0x55d34705ecb0 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.235+0000 7f6d7ffff640 1 -- 172.21.15.5:0/2343787335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d900056f0 con 0x55d34705ecb0 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.236+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/2343787335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 msgr2=0x55d3470769b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.236+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/2343787335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d3470769b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6d90006010 tx=0x7f6d90009ea0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.236+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/2343787335 shutdown_connections 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.236+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/2343787335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d34705ecb0 0x55d3470769b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.236+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/2343787335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d34705db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.236+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/2343787335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d347053af0 0x55d34705c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.236+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/2343787335 >> 172.21.15.5:0/2343787335 conn(0x55d347050060 msgr2=0x55d34705d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.236+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/2343787335 shutdown_connections 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.236+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/2343787335 wait complete. 2024-09-11T22:24:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.236+0000 7f6d9f8ba740 1 -- start start 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9f8ba740 1 Processor -- start 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d3470d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d3470d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9f8ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 0x55d3470669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981b0fa0 con 0x55d347053af0 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981b08d0 con 0x55d34705ecb0 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9f8ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981adb10 con 0x55d347054410 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9f8ba740 10 client.0 _ll_register_callbacks cb 0x7f6d880bbe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d3470d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d3470d1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57946/0 (socket says 172.21.15.5:57946) 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9659c640 1 -- 172.21.15.5:0/279336132 learned_addr learned my addr 172.21.15.5:0/279336132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d95d9b640 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d3470d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d96d9d640 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 0x55d3470669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9659c640 1 -- 172.21.15.5:0/279336132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 msgr2=0x55d3470d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9659c640 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d3470d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9659c640 1 -- 172.21.15.5:0/279336132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 msgr2=0x55d3470669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9659c640 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 0x55d3470669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9659c640 1 -- 172.21.15.5:0/279336132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d347067040 con 0x55d347053af0 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d9659c640 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d3470d1b30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6d880b8a00 tx=0x7f6d88031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112ea81c57838104 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.237+0000 7f6d7dffb640 1 -- 172.21.15.5:0/279336132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88049040 con 0x55d347053af0 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d7dffb640 1 -- 172.21.15.5:0/279336132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f6d880c00d0 con 0x55d347053af0 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/279336132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d9000f5d0 con 0x55d347053af0 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d7dffb640 1 -- 172.21.15.5:0/279336132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d880c4ca0 con 0x55d347053af0 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d9f8ba740 10 client.16029 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/279336132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d3470d94b0 con 0x55d347053af0 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d7dffb640 1 -- 172.21.15.5:0/279336132 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d9000f5d0 con 0x55d347053af0 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d7dffb640 1 -- 172.21.15.5:0/279336132 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6d880ed4e0 con 0x55d347053af0 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d9f8ba740 10 client.16029 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d9f8ba740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d9f8ba740 20 client.16029 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3470d9bf0 0x55d3470dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d9f8ba740 4 client.16029 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/279336132 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d346fbfd70 con 0x55d3470d9bf0 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.238+0000 7f6d95d9b640 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3470d9bf0 0x55d3470dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.239+0000 7f6d7dffb640 10 client.16029 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.239+0000 7f6d95d9b640 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3470d9bf0 0x55d3470dc010 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d9a9b27cc3cb60 server_cookie=5ac4f60506d5938 in_seq=0 out_seq=0 2024-09-11T22:24:13.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 88946 16.09 MB/sec warmup 31 sec latency 35.877 ms 2024-09-11T22:24:13.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:13 smithi005 ceph-mon[30943]: pgmap v63: 97 pgs: 97 active+clean; 32 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 457 op/s 2024-09-11T22:24:13.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:13 smithi159 ceph-mon[35534]: pgmap v63: 97 pgs: 97 active+clean; 32 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 457 op/s 2024-09-11T22:24:13.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:13 smithi172 ceph-mon[35461]: pgmap v63: 97 pgs: 97 active+clean; 32 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 457 op/s 2024-09-11T22:24:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d7dffb640 1 -- 172.21.15.5:0/279336132 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d346fbfd70 con 0x55d3470d9bf0 2024-09-11T22:24:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d7dffb640 10 client.16029 handle_command_reply: tid=0 2024-09-11T22:24:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 client.16029 shutdown 2024-09-11T22:24:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/279336132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 msgr2=0x55d3470d1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:13.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d3470d1b30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6d880b8a00 tx=0x7f6d88031490 comp rx=0 tx=0).stop 2024-09-11T22:24:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/279336132 shutdown_connections 2024-09-11T22:24:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3470d9bf0 0x55d3470dc010 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d7dffb640 0 client.16029 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d34705ecb0 0x55d3470669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d347054410 0x55d3470d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 --2- 172.21.15.5:0/279336132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d347053af0 0x55d3470d1b30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/279336132 >> 172.21.15.5:0/279336132 conn(0x55d347050060 msgr2=0x55d34705d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/279336132 shutdown_connections 2024-09-11T22:24:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 1 -- 172.21.15.5:0/279336132 wait complete. 2024-09-11T22:24:13.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.640+0000 7f6d9f8ba740 20 client.16029 trim_cache size 0 max 16384 2024-09-11T22:24:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:24:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:24:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:24:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:24:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3565979585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d800a2e20 msgr2=0x7f6d800a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:13.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d800a2e20 0x7f6d800a52e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d90023000 tx=0x7f6d90022000 comp rx=0 tx=0).stop 2024-09-11T22:24:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3565979585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d981009b0 msgr2=0x7f6d9819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d981009b0 0x7f6d9819a520 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6d8802a940 tx=0x7f6d88003b90 comp rx=0 tx=0).stop 2024-09-11T22:24:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3565979585 shutdown_connections 2024-09-11T22:24:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d981069c0 0x7f6d9819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d800a2e20 0x7f6d800a52e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d98101350 0x7f6d9819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 --2- 172.21.15.5:0/3565979585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d981009b0 0x7f6d9819a520 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3565979585 >> 172.21.15.5:0/3565979585 conn(0x7f6d98077990 msgr2=0x7f6d980ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3565979585 shutdown_connections 2024-09-11T22:24:13.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:13.641+0000 7f6d9cb23640 1 -- 172.21.15.5:0/3565979585 wait complete. 2024-09-11T22:24:14.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 89171 15.60 MB/sec warmup 32 sec latency 427.414 ms 2024-09-11T22:24:15.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 91444 15.46 MB/sec warmup 33 sec latency 77.183 ms 2024-09-11T22:24:15.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:15 smithi005 ceph-mon[30943]: pgmap v64: 97 pgs: 97 active+clean; 32 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-11T22:24:15.578 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:15 smithi159 ceph-mon[35534]: pgmap v64: 97 pgs: 97 active+clean; 32 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-11T22:24:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:15 smithi172 ceph-mon[35461]: pgmap v64: 97 pgs: 97 active+clean; 32 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-11T22:24:16.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 93775 15.39 MB/sec warmup 34 sec latency 20.217 ms 2024-09-11T22:24:17.304 INFO:tasks.workunit.client.0.smithi005.stdout: 1 96405 15.35 MB/sec warmup 35 sec latency 14.444 ms 2024-09-11T22:24:17.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:17 smithi005 ceph-mon[30943]: pgmap v65: 97 pgs: 97 active+clean; 32 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-11T22:24:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:17 smithi159 ceph-mon[35534]: pgmap v65: 97 pgs: 97 active+clean; 32 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-11T22:24:17.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:17 smithi172 ceph-mon[35461]: pgmap v65: 97 pgs: 97 active+clean; 32 MiB data, 497 MiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-09-11T22:24:18.305 INFO:tasks.workunit.client.0.smithi005.stdout: 1 99565 15.45 MB/sec warmup 36 sec latency 9.936 ms 2024-09-11T22:24:18.501 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-11T22:24:18.501 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:24:18.770 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:19.305 INFO:tasks.workunit.client.0.smithi005.stdout: 1 102237 15.34 MB/sec warmup 37 sec latency 26.261 ms 2024-09-11T22:24:19.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.295+0000 7f2c2cb59640 1 -- 172.21.15.5:0/315132605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c28101420 msgr2=0x7f2c2810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:19.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.295+0000 7f2c2cb59640 1 --2- 172.21.15.5:0/315132605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c28101420 0x7f2c2810d950 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009930 tx=0x7f2c2002a420 comp rx=0 tx=0).stop 2024-09-11T22:24:19.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.295+0000 7f2c2cb59640 1 -- 172.21.15.5:0/315132605 shutdown_connections 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.295+0000 7f2c2cb59640 1 --2- 172.21.15.5:0/315132605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c28101420 0x7f2c2810d950 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.295+0000 7f2c2cb59640 1 --2- 172.21.15.5:0/315132605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c28100a80 0x7f2c28100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.295+0000 7f2c2cb59640 1 --2- 172.21.15.5:0/315132605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c28106a80 0x7f2c28106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.295+0000 7f2c2cb59640 1 -- 172.21.15.5:0/315132605 >> 172.21.15.5:0/315132605 conn(0x7f2c280fc7a0 msgr2=0x7f2c280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.295+0000 7f2c2cb59640 1 -- 172.21.15.5:0/315132605 shutdown_connections 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.295+0000 7f2c2cb59640 1 -- 172.21.15.5:0/315132605 wait complete. 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.295+0000 7f2c2cb59640 1 Processor -- start 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2cb59640 1 -- start start 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2cb59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c28100a80 0x7f2c28110830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2cb59640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c28101420 0x7f2c28110d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2cb59640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c28106a80 0x7f2c281112b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2cb59640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2806aeb0 con 0x7f2c28100a80 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2cb59640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2806ad30 con 0x7f2c28101420 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2cb59640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2806b030 con 0x7f2c28106a80 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c28100a80 0x7f2c28110830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c28100a80 0x7f2c28110830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57962/0 (socket says 172.21.15.5:57962) 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2659c640 1 -- 172.21.15.5:0/2102669787 learned_addr learned my addr 172.21.15.5:0/2102669787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c25d9b640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c28101420 0x7f2c28110d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c26d9d640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c28106a80 0x7f2c281112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2659c640 1 -- 172.21.15.5:0/2102669787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c28106a80 msgr2=0x7f2c281112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2659c640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c28106a80 0x7f2c281112b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2659c640 1 -- 172.21.15.5:0/2102669787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c28101420 msgr2=0x7f2c28110d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2659c640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c28101420 0x7f2c28110d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.296+0000 7f2c2659c640 1 -- 172.21.15.5:0/2102669787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c281a68b0 con 0x7f2c28100a80 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.297+0000 7f2c2659c640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c28100a80 0x7f2c28110830 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800db80 tx=0x7f2c1800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.297+0000 7f2c177fe640 1 -- 172.21.15.5:0/2102669787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18017070 con 0x7f2c28100a80 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.297+0000 7f2c177fe640 1 -- 172.21.15.5:0/2102669787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c18003b70 con 0x7f2c28100a80 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.297+0000 7f2c2cb59640 1 -- 172.21.15.5:0/2102669787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c281a6ba0 con 0x7f2c28100a80 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.297+0000 7f2c177fe640 1 -- 172.21.15.5:0/2102669787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c180106c0 con 0x7f2c28100a80 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.297+0000 7f2c2cb59640 1 -- 172.21.15.5:0/2102669787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c281a70e0 con 0x7f2c28100a80 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.298+0000 7f2c2cb59640 1 -- 172.21.15.5:0/2102669787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c28100ee0 con 0x7f2c28100a80 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.299+0000 7f2c177fe640 1 -- 172.21.15.5:0/2102669787 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2c18004020 con 0x7f2c28100a80 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.300+0000 7f2c177fe640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c100a2cf0 0x7f2c100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.300+0000 7f2c25d9b640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c100a2cf0 0x7f2c100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.300+0000 7f2c177fe640 1 -- 172.21.15.5:0/2102669787 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f2c180cf160 con 0x7f2c28100a80 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.301+0000 7f2c25d9b640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c100a2cf0 0x7f2c100a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c281122c0 tx=0x7f2c1c00b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.305+0000 7f2c177fe640 1 -- 172.21.15.5:0/2102669787 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c18020070 con 0x7f2c28100a80 2024-09-11T22:24:19.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.532+0000 7f2c2cb59640 1 -- 172.21.15.5:0/2102669787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c281a73c0 con 0x7f2c28100a80 2024-09-11T22:24:19.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:19 smithi005 ceph-mon[30943]: pgmap v66: 97 pgs: 97 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.7 MiB/s wr, 628 op/s 2024-09-11T22:24:19.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.532+0000 7f2c177fe640 1 -- 172.21.15.5:0/2102669787 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11974 (secure 0 0 0) 0x7f2c18097020 con 0x7f2c28100a80 2024-09-11T22:24:19.533 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:24:19.534 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-11T22:23:49:156575+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:23:48.779139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:24:19.534 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 -- 172.21.15.5:0/2102669787 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c100a2cf0 msgr2=0x7f2c100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c100a2cf0 0x7f2c100a51b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2c281122c0 tx=0x7f2c1c00b410 comp rx=0 tx=0).stop 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 -- 172.21.15.5:0/2102669787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c28100a80 msgr2=0x7f2c28110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c28100a80 0x7f2c28110830 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800db80 tx=0x7f2c1800b630 comp rx=0 tx=0).stop 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 -- 172.21.15.5:0/2102669787 shutdown_connections 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c28106a80 0x7f2c281112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c100a2cf0 0x7f2c100a51b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c28101420 0x7f2c28110d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 --2- 172.21.15.5:0/2102669787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c28100a80 0x7f2c28110830 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 -- 172.21.15.5:0/2102669787 >> 172.21.15.5:0/2102669787 conn(0x7f2c280fc7a0 msgr2=0x7f2c280fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:19.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 -- 172.21.15.5:0/2102669787 shutdown_connections 2024-09-11T22:24:19.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:19.535+0000 7f2c2cb59640 1 -- 172.21.15.5:0/2102669787 wait complete. 2024-09-11T22:24:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:19 smithi159 ceph-mon[35534]: pgmap v66: 97 pgs: 97 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.7 MiB/s wr, 628 op/s 2024-09-11T22:24:19.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:19 smithi172 ceph-mon[35461]: pgmap v66: 97 pgs: 97 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.7 MiB/s wr, 628 op/s 2024-09-11T22:24:20.238 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-11T22:24:20.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-11T22:24:20.305 INFO:tasks.workunit.client.0.smithi005.stdout: 1 104602 15.29 MB/sec warmup 38 sec latency 10.449 ms 2024-09-11T22:24:20.516 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:20.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:20 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2102669787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:24:20.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:20 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2102669787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:24:20.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:20 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2102669787' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:24:21.305 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 15.24 MB/sec warmup 39 sec latency 206.988 ms 2024-09-11T22:24:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.023+0000 7f50174b5640 1 -- 172.21.15.5:0/164669280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5010102d30 msgr2=0x7f5010103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.023+0000 7f50174b5640 1 --2- 172.21.15.5:0/164669280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5010102d30 0x7f5010103110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f500c009900 tx=0x7f500c02a410 comp rx=0 tx=0).stop 2024-09-11T22:24:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.023+0000 7f50174b5640 1 -- 172.21.15.5:0/164669280 shutdown_connections 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.023+0000 7f50174b5640 1 --2- 172.21.15.5:0/164669280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50101082a0 0x7f501010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.023+0000 7f50174b5640 1 --2- 172.21.15.5:0/164669280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50101036e0 0x7f5010107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.023+0000 7f50174b5640 1 --2- 172.21.15.5:0/164669280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5010102d30 0x7f5010103110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.023+0000 7f50174b5640 1 -- 172.21.15.5:0/164669280 >> 172.21.15.5:0/164669280 conn(0x7f50100fc7d0 msgr2=0x7f50100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.024+0000 7f50174b5640 1 -- 172.21.15.5:0/164669280 shutdown_connections 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.024+0000 7f50174b5640 1 -- 172.21.15.5:0/164669280 wait complete. 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.024+0000 7f50174b5640 1 Processor -- start 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.024+0000 7f50174b5640 1 -- start start 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f50174b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5010102d30 0x7f5010100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f50174b5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50101036e0 0x7f5010100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f50174b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50101082a0 0x7f5010101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f50174b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50101120a0 con 0x7f50101082a0 2024-09-11T22:24:21.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f50174b5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5010111f20 con 0x7f50101036e0 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f50174b5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5010112220 con 0x7f5010102d30 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5015251640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5010102d30 0x7f5010100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5015251640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5010102d30 0x7f5010100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60866/0 (socket says 172.21.15.5:60866) 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5014a50640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50101036e0 0x7f5010100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5015251640 1 -- 172.21.15.5:0/1844287951 learned_addr learned my addr 172.21.15.5:0/1844287951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5015a52640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50101082a0 0x7f5010101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5014a50640 1 -- 172.21.15.5:0/1844287951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5010102d30 msgr2=0x7f5010100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5014a50640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5010102d30 0x7f5010100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5014a50640 1 -- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50101082a0 msgr2=0x7f5010101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5014a50640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50101082a0 0x7f5010101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5014a50640 1 -- 172.21.15.5:0/1844287951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50101a6a60 con 0x7f50101036e0 2024-09-11T22:24:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5015a52640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50101082a0 0x7f5010101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f5014a50640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50101036e0 0x7f5010100ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f50100697d0 tx=0x7f500000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.025+0000 7f50067fc640 1 -- 172.21.15.5:0/1844287951 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500000cc40 con 0x7f50101036e0 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.026+0000 7f50174b5640 1 -- 172.21.15.5:0/1844287951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50101a6d50 con 0x7f50101036e0 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.026+0000 7f50067fc640 1 -- 172.21.15.5:0/1844287951 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500000cde0 con 0x7f50101036e0 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.026+0000 7f50067fc640 1 -- 172.21.15.5:0/1844287951 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000004a00 con 0x7f50101036e0 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.026+0000 7f50174b5640 1 -- 172.21.15.5:0/1844287951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50101a71e0 con 0x7f50101036e0 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.028+0000 7f50067fc640 1 -- 172.21.15.5:0/1844287951 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f5000004c60 con 0x7f50101036e0 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.028+0000 7f501988e740 1 Processor -- start 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.028+0000 7f501988e740 1 -- start start 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.028+0000 7f501988e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.028+0000 7f501988e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 0x56101fe95400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.028+0000 7f501988e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe95940 0x56101fe97d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f501988e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56101fdf02a0 con 0x56101fe8b080 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f501988e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56101fd97b10 con 0x56101fe8a760 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f501988e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56101fdec260 con 0x56101fe95940 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015251640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 0x56101fe95400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015a52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015a52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44516/0 (socket says 172.21.15.5:44516) 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5014a50640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe95940 0x56101fe97d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015a52640 1 -- 172.21.15.5:0/3674373991 learned_addr learned my addr 172.21.15.5:0/3674373991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015a52640 1 -- 172.21.15.5:0/3674373991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe95940 msgr2=0x56101fe97d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015a52640 1 --2- 172.21.15.5:0/3674373991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe95940 0x56101fe97d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015a52640 1 -- 172.21.15.5:0/3674373991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 msgr2=0x56101fe95400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015a52640 1 --2- 172.21.15.5:0/3674373991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 0x56101fe95400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015a52640 1 -- 172.21.15.5:0/3674373991 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56101fb61b90 con 0x56101fe8a760 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f50067fc640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ffc0a2cf0 0x7f4ffc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015251640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ffc0a2cf0 0x7f4ffc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f5015a52640 1 --2- 172.21.15.5:0/3674373991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8ab40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5008009fd0 tx=0x7f5008010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1f7e6972ce4d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f50067fc640 1 -- 172.21.15.5:0/1844287951 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f50000cf480 con 0x7f50101036e0 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.029+0000 7f4ffb7fe640 1 -- 172.21.15.5:0/3674373991 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5008011700 con 0x56101fe8a760 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f4ffb7fe640 1 -- 172.21.15.5:0/3674373991 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5008011d80 con 0x56101fe8a760 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f4ffb7fe640 1 -- 172.21.15.5:0/3674373991 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500800e3c0 con 0x56101fe8a760 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f501988e740 1 -- 172.21.15.5:0/3674373991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 msgr2=0x56101fe8ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f501988e740 1 --2- 172.21.15.5:0/3674373991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8ab40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5008009fd0 tx=0x7f5008010860 comp rx=0 tx=0).stop 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f5015251640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ffc0a2cf0 0x7f4ffc0a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x56101fe87b50 tx=0x7f500c005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f501988e740 1 -- 172.21.15.5:0/3674373991 shutdown_connections 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f501988e740 1 --2- 172.21.15.5:0/3674373991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe95940 0x56101fe97d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f501988e740 1 --2- 172.21.15.5:0/3674373991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 0x56101fe95400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f501988e740 1 --2- 172.21.15.5:0/3674373991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8ab40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f501988e740 1 -- 172.21.15.5:0/3674373991 >> 172.21.15.5:0/3674373991 conn(0x56101fe86cf0 msgr2=0x56101fe870d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f501988e740 1 -- 172.21.15.5:0/3674373991 shutdown_connections 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.030+0000 7f501988e740 1 -- 172.21.15.5:0/3674373991 wait complete. 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f501988e740 1 -- start start 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f501988e740 1 Processor -- start 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f501988e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe8a760 0x56101fe92800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f501988e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 0x56101fe92d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f501988e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe95940 0x56101fe93280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f501988e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56101fe98510 con 0x56101fe8b080 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f501988e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56101fd786e0 con 0x56101fe95940 2024-09-11T22:24:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f501988e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56101fe98690 con 0x56101fe8a760 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f501988e740 10 client.0 _ll_register_callbacks cb 0x56101fd9ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f5014a50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 0x56101fe92d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f5014a50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 0x56101fe92d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58006/0 (socket says 172.21.15.5:58006) 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f5014a50640 1 -- 172.21.15.5:0/2734021093 learned_addr learned my addr 172.21.15.5:0/2734021093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f5015a52640 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe95940 0x56101fe93280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.031+0000 7f5015251640 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe8a760 0x56101fe92800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f5014a50640 1 -- 172.21.15.5:0/2734021093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe8a760 msgr2=0x56101fe92800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f5014a50640 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe8a760 0x56101fe92800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f5014a50640 1 -- 172.21.15.5:0/2734021093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe95940 msgr2=0x56101fe93280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f5014a50640 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe95940 0x56101fe93280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f5014a50640 1 -- 172.21.15.5:0/2734021093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101fea19d0 con 0x56101fe8b080 2024-09-11T22:24:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f5015a52640 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe95940 0x56101fe93280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f5014a50640 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 0x56101fe92d40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f500000c6a0 tx=0x7f50000d2b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e9328198432db5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f4ff8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/2734021093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500001c070 con 0x56101fe8b080 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/2734021093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f50000860d0 con 0x56101fe8b080 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f501988e740 1 -- 172.21.15.5:0/2734021093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5008011d80 con 0x56101fe8b080 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/2734021093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f50000b4050 con 0x56101fe8b080 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f501988e740 10 client.16047 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f501988e740 1 -- 172.21.15.5:0/2734021093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56101fea43c0 con 0x56101fe8b080 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.032+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/2734021093 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5008011d80 con 0x56101fe8b080 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.033+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/2734021093 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f50000a4140 con 0x56101fe8b080 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.033+0000 7f501988e740 10 client.16047 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.033+0000 7f501988e740 10 client.16047 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:24:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.033+0000 7f501988e740 20 client.16047 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.033+0000 7f501988e740 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56101fea5940 0x56101fea7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.033+0000 7f501988e740 4 client.16047 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.033+0000 7f501988e740 1 -- 172.21.15.5:0/2734021093 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56101fdfd9d0 con 0x56101fea5940 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.033+0000 7f5015251640 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56101fea5940 0x56101fea7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.033+0000 7f5015251640 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56101fea5940 0x56101fea7d50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31645e99236c5930 server_cookie=6a8fe56e70052a2 in_seq=0 out_seq=0 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.033+0000 7f4ff8ff9640 10 client.16047 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/2734021093 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56101fdfd9d0 con 0x56101fea5940 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f4ff8ff9640 10 client.16047 handle_command_reply: tid=0 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 client.16047 shutdown 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 -- 172.21.15.5:0/2734021093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 msgr2=0x56101fe92d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 0x56101fe92d40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f500000c6a0 tx=0x7f50000d2b00 comp rx=0 tx=0).stop 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 -- 172.21.15.5:0/2734021093 shutdown_connections 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56101fea5940 0x56101fea7d50 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe95940 0x56101fe93280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8b080 0x56101fe92d40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f4ff8ff9640 0 client.16047 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 --2- 172.21.15.5:0/2734021093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe8a760 0x56101fe92800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 -- 172.21.15.5:0/2734021093 >> 172.21.15.5:0/2734021093 conn(0x56101fe86cf0 msgr2=0x56101fe85e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 -- 172.21.15.5:0/2734021093 shutdown_connections 2024-09-11T22:24:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 1 -- 172.21.15.5:0/2734021093 wait complete. 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.034+0000 7f501988e740 20 client.16047 trim_cache size 0 max 16384 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.053+0000 7f501988e740 1 Processor -- start 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.053+0000 7f501988e740 1 -- start start 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.053+0000 7f501988e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.053+0000 7f501988e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe8b080 0x56101fe94880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.053+0000 7f501988e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe95940 0x56101fead5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.053+0000 7f501988e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50101afca0 con 0x56101fe95940 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.053+0000 7f501988e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50101b18a0 con 0x56101fe8a760 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.053+0000 7f501988e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50101b0370 con 0x56101fe8b080 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015251640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015a52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe95940 0x56101fead5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5014a50640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe8b080 0x56101fe94880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015251640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44532/0 (socket says 172.21.15.5:44532) 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015251640 1 -- 172.21.15.5:0/758214756 learned_addr learned my addr 172.21.15.5:0/758214756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015251640 1 -- 172.21.15.5:0/758214756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe8b080 msgr2=0x56101fe94880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015251640 1 --2- 172.21.15.5:0/758214756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe8b080 0x56101fe94880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015251640 1 -- 172.21.15.5:0/758214756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe95940 msgr2=0x56101fead5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015251640 1 --2- 172.21.15.5:0/758214756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe95940 0x56101fead5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015251640 1 -- 172.21.15.5:0/758214756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56101fe86b20 con 0x56101fe8a760 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015a52640 1 --2- 172.21.15.5:0/758214756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe95940 0x56101fead5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f5015251640 1 --2- 172.21.15.5:0/758214756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8d210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f500c006010 tx=0x7f500c03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463317c56ff03e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f4ffaffd640 1 -- 172.21.15.5:0/758214756 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c057070 con 0x56101fe8a760 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f4ffaffd640 1 -- 172.21.15.5:0/758214756 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c031540 con 0x56101fe8a760 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.054+0000 7f4ffaffd640 1 -- 172.21.15.5:0/758214756 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c035040 con 0x56101fe8a760 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 -- 172.21.15.5:0/758214756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 msgr2=0x56101fe8d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 --2- 172.21.15.5:0/758214756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8d210 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f500c006010 tx=0x7f500c03f000 comp rx=0 tx=0).stop 2024-09-11T22:24:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 -- 172.21.15.5:0/758214756 shutdown_connections 2024-09-11T22:24:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 --2- 172.21.15.5:0/758214756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe95940 0x56101fead5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 --2- 172.21.15.5:0/758214756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe8b080 0x56101fe94880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 --2- 172.21.15.5:0/758214756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8a760 0x56101fe8d210 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 -- 172.21.15.5:0/758214756 >> 172.21.15.5:0/758214756 conn(0x56101fe86cf0 msgr2=0x56101fe91970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 -- 172.21.15.5:0/758214756 shutdown_connections 2024-09-11T22:24:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 -- 172.21.15.5:0/758214756 wait complete. 2024-09-11T22:24:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 -- start start 2024-09-11T22:24:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.055+0000 7f501988e740 1 Processor -- start 2024-09-11T22:24:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f501988e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8a760 0x56101fe9e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f501988e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8b080 0x56101fe99500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f501988e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe95940 0x56101fe99c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f501988e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50101b1110 con 0x56101fe8a760 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f501988e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50101b0a40 con 0x56101fe8b080 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f501988e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f50101adc80 con 0x56101fe95940 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f501988e740 10 client.0 _ll_register_callbacks cb 0x7f50000046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f5015251640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8a760 0x56101fe9e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f5015251640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8a760 0x56101fe9e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58020/0 (socket says 172.21.15.5:58020) 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f5015251640 1 -- 172.21.15.5:0/3198028054 learned_addr learned my addr 172.21.15.5:0/3198028054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f5014a50640 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8b080 0x56101fe99500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f5015a52640 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe95940 0x56101fe99c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f5015251640 1 -- 172.21.15.5:0/3198028054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe95940 msgr2=0x56101fe99c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f5015251640 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe95940 0x56101fe99c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f5015251640 1 -- 172.21.15.5:0/3198028054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8b080 msgr2=0x56101fe99500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f5015251640 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8b080 0x56101fe99500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.056+0000 7f5015251640 1 -- 172.21.15.5:0/3198028054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101fe82260 con 0x56101fe8a760 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f5015251640 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8a760 0x56101fe9e1d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f500c0096c0 tx=0x7f500c003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a36666d4a25bbd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f4ff8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/3198028054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c057070 con 0x56101fe8a760 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/3198028054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f500c003dc0 con 0x56101fe8a760 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f501988e740 1 -- 172.21.15.5:0/3198028054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f500c031540 con 0x56101fe8a760 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/3198028054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f500c0316f0 con 0x56101fe8a760 2024-09-11T22:24:21.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f501988e740 10 client.16053 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f501988e740 1 -- 172.21.15.5:0/3198028054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56101ff10550 con 0x56101fe8a760 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/3198028054 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c031540 con 0x56101fe8a760 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/3198028054 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f500c065030 con 0x56101fe8a760 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f501988e740 10 client.16053 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f501988e740 10 client.16053 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f501988e740 20 client.16053 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f501988e740 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56101ff10ce0 0x56101ff13100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f501988e740 4 client.16053 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.057+0000 7f501988e740 1 -- 172.21.15.5:0/3198028054 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x56101fdf7d20 con 0x56101ff10ce0 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.058+0000 7f5014a50640 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56101ff10ce0 0x56101ff13100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.058+0000 7f5014a50640 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56101ff10ce0 0x56101ff13100 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471f3c0d8ac393f4 server_cookie=95b72f828445e856 in_seq=0 out_seq=0 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.058+0000 7f4ff8ff9640 10 client.16053 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f4ff8ff9640 1 -- 172.21.15.5:0/3198028054 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x56101fdf7d20 con 0x56101ff10ce0 2024-09-11T22:24:21.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f4ff8ff9640 10 client.16053 handle_command_reply: tid=0 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 client.16053 shutdown 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 -- 172.21.15.5:0/3198028054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8a760 msgr2=0x56101fe9e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8a760 0x56101fe9e1d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f500c0096c0 tx=0x7f500c003710 comp rx=0 tx=0).stop 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 -- 172.21.15.5:0/3198028054 shutdown_connections 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56101ff10ce0 0x56101ff13100 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56101fe95940 0x56101fe99c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56101fe8b080 0x56101fe99500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f4ff8ff9640 0 client.16053 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 --2- 172.21.15.5:0/3198028054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56101fe8a760 0x56101fe9e1d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 -- 172.21.15.5:0/3198028054 >> 172.21.15.5:0/3198028054 conn(0x56101fe86cf0 msgr2=0x56101fe85e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 -- 172.21.15.5:0/3198028054 shutdown_connections 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 1 -- 172.21.15.5:0/3198028054 wait complete. 2024-09-11T22:24:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.162+0000 7f501988e740 20 client.16053 trim_cache size 0 max 16384 2024-09-11T22:24:21.412 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:24:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 30, 2024-09-11T22:24:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:24:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 6, 2024-09-11T22:24:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:24:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "76699de": { 2024-09-11T22:24:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 6, 2024-09-11T22:24:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:24:21.412 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:24:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:24:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:24:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:24:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:24:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 133.4, 2024-09-11T22:24:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:24:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:24:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:24:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:24:21.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:24:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:24:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:21.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:21.414 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:24:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 -- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ffc0a2cf0 msgr2=0x7f4ffc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ffc0a2cf0 0x7f4ffc0a51b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x56101fe87b50 tx=0x7f500c005a20 comp rx=0 tx=0).stop 2024-09-11T22:24:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 -- 172.21.15.5:0/1844287951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50101036e0 msgr2=0x7f5010100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50101036e0 0x7f5010100ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f50100697d0 tx=0x7f500000ee50 comp rx=0 tx=0).stop 2024-09-11T22:24:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 -- 172.21.15.5:0/1844287951 shutdown_connections 2024-09-11T22:24:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50101082a0 0x7f5010101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ffc0a2cf0 0x7f4ffc0a51b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50101036e0 0x7f5010100ed0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 --2- 172.21.15.5:0/1844287951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5010102d30 0x7f5010100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 -- 172.21.15.5:0/1844287951 >> 172.21.15.5:0/1844287951 conn(0x7f50100fc7d0 msgr2=0x7f5010105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 -- 172.21.15.5:0/1844287951 shutdown_connections 2024-09-11T22:24:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:21.163+0000 7f50174b5640 1 -- 172.21.15.5:0/1844287951 wait complete. 2024-09-11T22:24:21.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:21 smithi005 ceph-mon[30943]: pgmap v67: 97 pgs: 97 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.9 MiB/s wr, 522 op/s 2024-09-11T22:24:21.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:21 smithi159 ceph-mon[35534]: pgmap v67: 97 pgs: 97 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.9 MiB/s wr, 522 op/s 2024-09-11T22:24:21.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:21 smithi172 ceph-mon[35461]: pgmap v67: 97 pgs: 97 active+clean; 33 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.9 MiB/s wr, 522 op/s 2024-09-11T22:24:22.295 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '76699de', quiesce took 3.8 seconds. Will release after: 9.600000000000001 2024-09-11T22:24:22.305 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 14.86 MB/sec warmup 40 sec latency 1207.095 ms 2024-09-11T22:24:23.305 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 14.50 MB/sec warmup 41 sec latency 2207.188 ms 2024-09-11T22:24:23.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:23 smithi005 ceph-mon[30943]: pgmap v68: 97 pgs: 97 active+clean; 39 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.5 MiB/s wr, 609 op/s 2024-09-11T22:24:23.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:23 smithi159 ceph-mon[35534]: pgmap v68: 97 pgs: 97 active+clean; 39 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.5 MiB/s wr, 609 op/s 2024-09-11T22:24:23.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:23 smithi172 ceph-mon[35461]: pgmap v68: 97 pgs: 97 active+clean; 39 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 9.5 MiB/s wr, 609 op/s 2024-09-11T22:24:24.305 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 14.15 MB/sec warmup 42 sec latency 3207.296 ms 2024-09-11T22:24:25.305 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 13.82 MB/sec warmup 43 sec latency 4207.421 ms 2024-09-11T22:24:25.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:25 smithi005 ceph-mon[30943]: pgmap v69: 97 pgs: 97 active+clean; 39 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.8 MiB/s wr, 510 op/s 2024-09-11T22:24:25.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:25 smithi159 ceph-mon[35534]: pgmap v69: 97 pgs: 97 active+clean; 39 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.8 MiB/s wr, 510 op/s 2024-09-11T22:24:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:25 smithi172 ceph-mon[35461]: pgmap v69: 97 pgs: 97 active+clean; 39 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.8 MiB/s wr, 510 op/s 2024-09-11T22:24:26.306 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 13.51 MB/sec warmup 44 sec latency 5207.543 ms 2024-09-11T22:24:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.743+0000 7f983fa5a640 1 -- 172.21.15.5:0/2848801443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98381009b0 msgr2=0x7f9838100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.743+0000 7f983fa5a640 1 --2- 172.21.15.5:0/2848801443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98381009b0 0x7f9838100e10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9828009930 tx=0x7f982802a420 comp rx=0 tx=0).stop 2024-09-11T22:24:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.743+0000 7f983fa5a640 1 -- 172.21.15.5:0/2848801443 shutdown_connections 2024-09-11T22:24:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.743+0000 7f983fa5a640 1 --2- 172.21.15.5:0/2848801443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9838101350 0x7f983810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.743+0000 7f983fa5a640 1 --2- 172.21.15.5:0/2848801443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98381009b0 0x7f9838100e10 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.743+0000 7f983fa5a640 1 --2- 172.21.15.5:0/2848801443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98381069c0 0x7f9838106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.743+0000 7f983fa5a640 1 -- 172.21.15.5:0/2848801443 >> 172.21.15.5:0/2848801443 conn(0x7f9838077990 msgr2=0x7f98380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.743+0000 7f983fa5a640 1 -- 172.21.15.5:0/2848801443 shutdown_connections 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.743+0000 7f983fa5a640 1 -- 172.21.15.5:0/2848801443 wait complete. 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.744+0000 7f983fa5a640 1 Processor -- start 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.744+0000 7f983fa5a640 1 -- start start 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.744+0000 7f983fa5a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381009b0 0x7f98381982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.744+0000 7f983fa5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9838101350 0x7f9838198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.744+0000 7f983fa5a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98381069c0 0x7f983819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.744+0000 7f983fa5a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9838110e00 con 0x7f9838101350 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.744+0000 7f983fa5a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9838110c80 con 0x7f98381069c0 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.744+0000 7f983fa5a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9838110f80 con 0x7f98381009b0 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.744+0000 7f983cff5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9838101350 0x7f9838198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983cff5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9838101350 0x7f9838198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36184/0 (socket says 172.21.15.5:36184) 2024-09-11T22:24:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983cff5640 1 -- 172.21.15.5:0/3643162933 learned_addr learned my addr 172.21.15.5:0/3643162933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983dff7640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98381069c0 0x7f983819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983d7f6640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381009b0 0x7f98381982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983cff5640 1 -- 172.21.15.5:0/3643162933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381009b0 msgr2=0x7f98381982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983cff5640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381009b0 0x7f98381982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983cff5640 1 -- 172.21.15.5:0/3643162933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98381069c0 msgr2=0x7f983819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983cff5640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98381069c0 0x7f983819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983cff5640 1 -- 172.21.15.5:0/3643162933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983819d2d0 con 0x7f9838101350 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983d7f6640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381009b0 0x7f98381982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983cff5640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9838101350 0x7f9838198820 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9828009230 tx=0x7f9828004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f982e7fc640 1 -- 172.21.15.5:0/3643162933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982803f030 con 0x7f9838101350 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f982e7fc640 1 -- 172.21.15.5:0/3643162933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9828003ad0 con 0x7f9838101350 2024-09-11T22:24:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983fa5a640 1 -- 172.21.15.5:0/3643162933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983819d560 con 0x7f9838101350 2024-09-11T22:24:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f982e7fc640 1 -- 172.21.15.5:0/3643162933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828035070 con 0x7f9838101350 2024-09-11T22:24:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.745+0000 7f983fa5a640 1 -- 172.21.15.5:0/3643162933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98381a4e50 con 0x7f9838101350 2024-09-11T22:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.748+0000 7f982e7fc640 1 -- 172.21.15.5:0/3643162933 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9828042030 con 0x7f9838101350 2024-09-11T22:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.748+0000 7f98427f5740 1 Processor -- start 2024-09-11T22:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.748+0000 7f98427f5740 1 -- start start 2024-09-11T22:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.748+0000 7f98427f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x561696234db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.748+0000 7f98427f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x561696243b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.748+0000 7f98427f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x5616962464b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.748+0000 7f98427f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5616961992f0 con 0x5616962352f0 2024-09-11T22:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.748+0000 7f98427f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561696141270 con 0x5616962440c0 2024-09-11T22:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.748+0000 7f98427f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5616961952b0 con 0x5616962349d0 2024-09-11T22:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983d7f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x561696243b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983d7f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x561696243b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36190/0 (socket says 172.21.15.5:36190) 2024-09-11T22:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983d7f6640 1 -- 172.21.15.5:0/2518138107 learned_addr learned my addr 172.21.15.5:0/2518138107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f982e7fc640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98240a2da0 0x7f98240a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983dff7640 1 --2- 172.21.15.5:0/2518138107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x561696234db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983cff5640 1 --2- 172.21.15.5:0/2518138107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x5616962464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983dff7640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98240a2da0 0x7f98240a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983d7f6640 1 -- 172.21.15.5:0/2518138107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 msgr2=0x561696234db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983d7f6640 1 --2- 172.21.15.5:0/2518138107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x561696234db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983d7f6640 1 -- 172.21.15.5:0/2518138107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 msgr2=0x5616962464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983d7f6640 1 --2- 172.21.15.5:0/2518138107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x5616962464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983d7f6640 1 -- 172.21.15.5:0/2518138107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561696140340 con 0x5616962352f0 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983cff5640 1 --2- 172.21.15.5:0/2518138107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x5616962464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f982e7fc640 1 -- 172.21.15.5:0/3643162933 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f98280ef990 con 0x7f9838101350 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983d7f6640 1 --2- 172.21.15.5:0/2518138107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x561696243b80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f983400bcf0 tx=0x7f983400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8c0f5598563037 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f98237fe640 1 -- 172.21.15.5:0/2518138107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834015070 con 0x5616962352f0 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f983dff7640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98240a2da0 0x7f98240a5260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9830005c40 tx=0x7f9830007520 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f98237fe640 1 -- 172.21.15.5:0/2518138107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983401a940 con 0x5616962352f0 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.749+0000 7f98237fe640 1 -- 172.21.15.5:0/2518138107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834011aa0 con 0x5616962352f0 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.750+0000 7f98427f5740 1 -- 172.21.15.5:0/2518138107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 msgr2=0x561696243b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.750+0000 7f98427f5740 1 --2- 172.21.15.5:0/2518138107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x561696243b80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f983400bcf0 tx=0x7f983400f730 comp rx=0 tx=0).stop 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.750+0000 7f98427f5740 1 -- 172.21.15.5:0/2518138107 shutdown_connections 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.750+0000 7f98427f5740 1 --2- 172.21.15.5:0/2518138107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x5616962464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.750+0000 7f98427f5740 1 --2- 172.21.15.5:0/2518138107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x561696243b80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.750+0000 7f98427f5740 1 --2- 172.21.15.5:0/2518138107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x561696234db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.750+0000 7f98427f5740 1 -- 172.21.15.5:0/2518138107 >> 172.21.15.5:0/2518138107 conn(0x561696231060 msgr2=0x561696231440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.750+0000 7f98427f5740 1 -- 172.21.15.5:0/2518138107 shutdown_connections 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.750+0000 7f98427f5740 1 -- 172.21.15.5:0/2518138107 wait complete. 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f98427f5740 1 -- start start 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f98427f5740 1 Processor -- start 2024-09-11T22:24:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f98427f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x561696241080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f98427f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x5616962415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f98427f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x561696241b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f98427f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561696246cf0 con 0x5616962352f0 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f98427f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561696122290 con 0x5616962440c0 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f98427f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561696246e70 con 0x5616962349d0 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f98427f5740 10 client.0 _ll_register_callbacks cb 0x561696153140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f983cff5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x5616962415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f983cff5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x5616962415c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36206/0 (socket says 172.21.15.5:36206) 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f983cff5640 1 -- 172.21.15.5:0/2919863762 learned_addr learned my addr 172.21.15.5:0/2919863762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f983d7f6640 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x561696241080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.751+0000 7f983dff7640 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x561696241b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f983cff5640 1 -- 172.21.15.5:0/2919863762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 msgr2=0x561696241080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f983cff5640 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x561696241080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f983cff5640 1 -- 172.21.15.5:0/2919863762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 msgr2=0x561696241b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f983cff5640 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x561696241b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f983cff5640 1 -- 172.21.15.5:0/2919863762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561695eac210 con 0x5616962352f0 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f983cff5640 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x5616962415c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f98280ed6a0 tx=0x7f982808ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aaf6095f5470b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f9820ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f9820ff9640 1 -- 172.21.15.5:0/2919863762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828049050 con 0x5616962352f0 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f9820ff9640 1 -- 172.21.15.5:0/2919863762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f9828003c80 con 0x5616962352f0 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f98427f5740 1 -- 172.21.15.5:0/2919863762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983401a940 con 0x5616962352f0 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f9820ff9640 1 -- 172.21.15.5:0/2919863762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f98280bc570 con 0x5616962352f0 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f98427f5740 10 client.16065 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:26.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f98427f5740 1 -- 172.21.15.5:0/2919863762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561695f78620 con 0x5616962352f0 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.752+0000 7f9820ff9640 1 -- 172.21.15.5:0/2919863762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983401a940 con 0x5616962352f0 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.753+0000 7f9820ff9640 1 -- 172.21.15.5:0/2919863762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9828031bd0 con 0x5616962352f0 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.753+0000 7f98427f5740 10 client.16065 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.753+0000 7f98427f5740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.753+0000 7f98427f5740 20 client.16065 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.753+0000 7f98427f5740 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561696254140 0x561696256560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.753+0000 7f98427f5740 4 client.16065 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.753+0000 7f98427f5740 1 -- 172.21.15.5:0/2919863762 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616961a6a20 con 0x561696254140 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.753+0000 7f983d7f6640 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561696254140 0x561696256560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.753+0000 7f983d7f6640 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561696254140 0x561696256560 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f57d7f4a2e73e09 server_cookie=a61f8a4828e38867 in_seq=0 out_seq=0 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.753+0000 7f9820ff9640 10 client.16065 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f9820ff9640 1 -- 172.21.15.5:0/2919863762 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616961a6a20 con 0x561696254140 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f9820ff9640 10 client.16065 handle_command_reply: tid=0 2024-09-11T22:24:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f98427f5740 1 client.16065 shutdown 2024-09-11T22:24:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f98427f5740 1 -- 172.21.15.5:0/2919863762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 msgr2=0x5616962415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f98427f5740 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x5616962415c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f98280ed6a0 tx=0x7f982808ada0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f98427f5740 1 -- 172.21.15.5:0/2919863762 shutdown_connections 2024-09-11T22:24:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f98427f5740 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561696254140 0x561696256560 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f98427f5740 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x561696241b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f98427f5740 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x5616962415c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f9820ff9640 0 client.16065 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f98427f5740 1 --2- 172.21.15.5:0/2919863762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x561696241080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f98427f5740 1 -- 172.21.15.5:0/2919863762 >> 172.21.15.5:0/2919863762 conn(0x561696231060 msgr2=0x561696236dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.754+0000 7f98427f5740 1 -- 172.21.15.5:0/2919863762 shutdown_connections 2024-09-11T22:24:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.755+0000 7f98427f5740 1 -- 172.21.15.5:0/2919863762 wait complete. 2024-09-11T22:24:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.755+0000 7f98427f5740 20 client.16065 trim_cache size 0 max 16384 2024-09-11T22:24:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.769+0000 7f98427f5740 1 Processor -- start 2024-09-11T22:24:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.769+0000 7f98427f5740 1 -- start start 2024-09-11T22:24:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.769+0000 7f98427f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962349d0 0x56169625be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.769+0000 7f98427f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962352f0 0x56169625c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f98427f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x56169623e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f98427f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98381ab690 con 0x5616962349d0 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f98427f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f98381ad290 con 0x5616962440c0 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f98427f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98381abd60 con 0x5616962352f0 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f983d7f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962349d0 0x56169625be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f983d7f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962349d0 0x56169625be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36222/0 (socket says 172.21.15.5:36222) 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f983d7f6640 1 -- 172.21.15.5:0/3354211970 learned_addr learned my addr 172.21.15.5:0/3354211970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f983d7f6640 1 -- 172.21.15.5:0/3354211970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962352f0 msgr2=0x56169625c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f983d7f6640 1 --2- 172.21.15.5:0/3354211970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962352f0 0x56169625c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f983d7f6640 1 -- 172.21.15.5:0/3354211970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 msgr2=0x56169623e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f983d7f6640 1 --2- 172.21.15.5:0/3354211970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x56169623e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f983d7f6640 1 -- 172.21.15.5:0/3354211970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561696240010 con 0x5616962349d0 2024-09-11T22:24:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f983d7f6640 1 --2- 172.21.15.5:0/3354211970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962349d0 0x56169625be70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f983400bcf0 tx=0x7f983400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77f2301d9ad921d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f9822ffd640 1 -- 172.21.15.5:0/3354211970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834015050 con 0x5616962349d0 2024-09-11T22:24:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f9822ffd640 1 -- 172.21.15.5:0/3354211970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9834005520 con 0x5616962349d0 2024-09-11T22:24:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.770+0000 7f9822ffd640 1 -- 172.21.15.5:0/3354211970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834011bb0 con 0x5616962349d0 2024-09-11T22:24:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 -- 172.21.15.5:0/3354211970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962349d0 msgr2=0x56169625be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 --2- 172.21.15.5:0/3354211970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962349d0 0x56169625be70 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f983400bcf0 tx=0x7f983400b5e0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 -- 172.21.15.5:0/3354211970 shutdown_connections 2024-09-11T22:24:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 --2- 172.21.15.5:0/3354211970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x56169623e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 --2- 172.21.15.5:0/3354211970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962352f0 0x56169625c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 --2- 172.21.15.5:0/3354211970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962349d0 0x56169625be70 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 -- 172.21.15.5:0/3354211970 >> 172.21.15.5:0/3354211970 conn(0x561696231060 msgr2=0x561696243100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 -- 172.21.15.5:0/3354211970 shutdown_connections 2024-09-11T22:24:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 -- 172.21.15.5:0/3354211970 wait complete. 2024-09-11T22:24:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 -- start start 2024-09-11T22:24:26.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.771+0000 7f98427f5740 1 Processor -- start 2024-09-11T22:24:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f98427f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x56169624c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f98427f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x56169624ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f98427f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x561696248a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f98427f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98381acb00 con 0x5616962352f0 2024-09-11T22:24:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f98427f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f98381ac430 con 0x5616962440c0 2024-09-11T22:24:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f98427f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98381a9670 con 0x5616962349d0 2024-09-11T22:24:26.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f98427f5740 10 client.0 _ll_register_callbacks cb 0x7f9828009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983cff5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x56169624ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983cff5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x56169624ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36228/0 (socket says 172.21.15.5:36228) 2024-09-11T22:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983cff5640 1 -- 172.21.15.5:0/2624996326 learned_addr learned my addr 172.21.15.5:0/2624996326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983d7f6640 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x56169624c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983dff7640 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x561696248a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983cff5640 1 -- 172.21.15.5:0/2624996326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 msgr2=0x56169624c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983cff5640 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x56169624c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983cff5640 1 -- 172.21.15.5:0/2624996326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 msgr2=0x561696248a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983cff5640 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x561696248a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983cff5640 1 -- 172.21.15.5:0/2624996326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561696249260 con 0x5616962352f0 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983dff7640 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x561696248a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f983cff5640 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x56169624ca10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f98280b82d0 tx=0x7f98280b81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ee4020e1b3ade server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.772+0000 7f9820ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f9820ff9640 1 -- 172.21.15.5:0/2624996326 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828049070 con 0x5616962352f0 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f9820ff9640 1 -- 172.21.15.5:0/2624996326 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f9828031bd0 con 0x5616962352f0 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f98427f5740 1 -- 172.21.15.5:0/2624996326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9834005520 con 0x5616962352f0 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f9820ff9640 1 -- 172.21.15.5:0/2624996326 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f98280d8050 con 0x5616962352f0 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f98427f5740 10 client.16071 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f98427f5740 1 -- 172.21.15.5:0/2624996326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561696249260 con 0x5616962352f0 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f9820ff9640 1 -- 172.21.15.5:0/2624996326 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9834005520 con 0x5616962352f0 2024-09-11T22:24:26.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f9820ff9640 1 -- 172.21.15.5:0/2624996326 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f982803bcb0 con 0x5616962352f0 2024-09-11T22:24:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f98427f5740 10 client.16071 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f98427f5740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f98427f5740 20 client.16071 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f98427f5740 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5616962bffe0 0x5616962c2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f98427f5740 4 client.16071 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:24:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.773+0000 7f98427f5740 1 -- 172.21.15.5:0/2624996326 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5616961a0d70 con 0x5616962bffe0 2024-09-11T22:24:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.774+0000 7f983d7f6640 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5616962bffe0 0x5616962c2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.774+0000 7f9820ff9640 10 client.16071 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:26.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.774+0000 7f983d7f6640 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5616962bffe0 0x5616962c2400 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2322f9f88e0712ac server_cookie=28edd64c8969c4e5 in_seq=0 out_seq=0 2024-09-11T22:24:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.850+0000 7f9820ff9640 1 -- 172.21.15.5:0/2624996326 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5616961a0d70 con 0x5616962bffe0 2024-09-11T22:24:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.850+0000 7f9820ff9640 10 client.16071 handle_command_reply: tid=0 2024-09-11T22:24:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.850+0000 7f98427f5740 1 client.16071 shutdown 2024-09-11T22:24:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 1 -- 172.21.15.5:0/2624996326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 msgr2=0x56169624ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x56169624ca10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f98280b82d0 tx=0x7f98280b81e0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 1 -- 172.21.15.5:0/2624996326 shutdown_connections 2024-09-11T22:24:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5616962bffe0 0x5616962c2400 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5616962440c0 0x561696248a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f9820ff9640 0 client.16071 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616962352f0 0x56169624ca10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 1 --2- 172.21.15.5:0/2624996326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5616962349d0 0x56169624c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 1 -- 172.21.15.5:0/2624996326 >> 172.21.15.5:0/2624996326 conn(0x561696231060 msgr2=0x56169622f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 1 -- 172.21.15.5:0/2624996326 shutdown_connections 2024-09-11T22:24:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 1 -- 172.21.15.5:0/2624996326 wait complete. 2024-09-11T22:24:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f98427f5740 20 client.16071 trim_cache size 0 max 16384 2024-09-11T22:24:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:24:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:24:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:24:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:24:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f983fa5a640 1 -- 172.21.15.5:0/3643162933 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98240a2da0 msgr2=0x7f98240a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f983fa5a640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98240a2da0 0x7f98240a5260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9830005c40 tx=0x7f9830007520 comp rx=0 tx=0).stop 2024-09-11T22:24:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f983fa5a640 1 -- 172.21.15.5:0/3643162933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9838101350 msgr2=0x7f9838198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.851+0000 7f983fa5a640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9838101350 0x7f9838198820 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9828009230 tx=0x7f9828004270 comp rx=0 tx=0).stop 2024-09-11T22:24:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.852+0000 7f983fa5a640 1 -- 172.21.15.5:0/3643162933 shutdown_connections 2024-09-11T22:24:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.852+0000 7f983fa5a640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98240a2da0 0x7f98240a5260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.852+0000 7f983fa5a640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98381069c0 0x7f983819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.852+0000 7f983fa5a640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9838101350 0x7f9838198820 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.852+0000 7f983fa5a640 1 --2- 172.21.15.5:0/3643162933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98381009b0 0x7f98381982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.852+0000 7f983fa5a640 1 -- 172.21.15.5:0/3643162933 >> 172.21.15.5:0/3643162933 conn(0x7f9838077990 msgr2=0x7f98380699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.852+0000 7f983fa5a640 1 -- 172.21.15.5:0/3643162933 shutdown_connections 2024-09-11T22:24:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:26.852+0000 7f983fa5a640 1 -- 172.21.15.5:0/3643162933 wait complete. 2024-09-11T22:24:27.306 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 13.21 MB/sec warmup 45 sec latency 6207.610 ms 2024-09-11T22:24:27.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:27 smithi005 ceph-mon[30943]: pgmap v70: 97 pgs: 97 active+clean; 39 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.8 MiB/s wr, 510 op/s 2024-09-11T22:24:27.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:27 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:24:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:27 smithi159 ceph-mon[35534]: pgmap v70: 97 pgs: 97 active+clean; 39 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.8 MiB/s wr, 510 op/s 2024-09-11T22:24:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:27 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:24:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:27 smithi172 ceph-mon[35461]: pgmap v70: 97 pgs: 97 active+clean; 39 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.8 MiB/s wr, 510 op/s 2024-09-11T22:24:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:27 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:24:28.306 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 12.92 MB/sec warmup 46 sec latency 7207.718 ms 2024-09-11T22:24:28.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:24:28.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:24:28.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:24:28.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:24:28.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:28.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:24:28.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:24:28.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:24:29.306 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 12.65 MB/sec warmup 47 sec latency 8207.785 ms 2024-09-11T22:24:30.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:29 smithi172 ceph-mon[35461]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.8 MiB/s wr, 570 op/s 2024-09-11T22:24:30.306 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 12.38 MB/sec warmup 48 sec latency 9207.892 ms 2024-09-11T22:24:30.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:29 smithi005 ceph-mon[30943]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.8 MiB/s wr, 570 op/s 2024-09-11T22:24:30.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:29 smithi159 ceph-mon[35534]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 8.8 MiB/s wr, 570 op/s 2024-09-11T22:24:31.306 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 12.13 MB/sec warmup 49 sec latency 10208.005 ms 2024-09-11T22:24:31.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:24:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:31 smithi172 ceph-mon[35461]: pgmap v72: 97 pgs: 97 active+clean; 31 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 639 KiB/s wr, 146 op/s 2024-09-11T22:24:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:32.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:31 smithi005 ceph-mon[30943]: pgmap v72: 97 pgs: 97 active+clean; 31 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 639 KiB/s wr, 146 op/s 2024-09-11T22:24:32.306 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 11.89 MB/sec warmup 50 sec latency 11208.106 ms 2024-09-11T22:24:32.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:31 smithi159 ceph-mon[35534]: pgmap v72: 97 pgs: 97 active+clean; 31 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 639 KiB/s wr, 146 op/s 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.686+0000 7f77ee740640 1 -- 172.21.15.5:0/2999504868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8102570 msgr2=0x7f77e81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.686+0000 7f77ee740640 1 --2- 172.21.15.5:0/2999504868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8102570 0x7f77e81029b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009900 tx=0x7f77d002a480 comp rx=0 tx=0).stop 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.686+0000 7f77ee740640 1 -- 172.21.15.5:0/2999504868 shutdown_connections 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.686+0000 7f77ee740640 1 --2- 172.21.15.5:0/2999504868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e8102ef0 0x7f77e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.686+0000 7f77ee740640 1 --2- 172.21.15.5:0/2999504868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8102570 0x7f77e81029b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.686+0000 7f77ee740640 1 --2- 172.21.15.5:0/2999504868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e8100e20 0x7f77e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.686+0000 7f77ee740640 1 -- 172.21.15.5:0/2999504868 >> 172.21.15.5:0/2999504868 conn(0x7f77e80fc7d0 msgr2=0x7f77e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.686+0000 7f77ee740640 1 -- 172.21.15.5:0/2999504868 shutdown_connections 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.686+0000 7f77ee740640 1 -- 172.21.15.5:0/2999504868 wait complete. 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77ee740640 1 Processor -- start 2024-09-11T22:24:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77ee740640 1 -- start start 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77ee740640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e8100e20 0x7f77e8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77ee740640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8102570 0x7f77e8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77ee740640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e8102ef0 0x7f77e81a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77ee740640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8110f80 con 0x7f77e8102570 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77ee740640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8110e00 con 0x7f77e8100e20 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77ee740640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f77e8111100 con 0x7f77e8102ef0 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8102570 0x7f77e8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8102570 0x7f77e8199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58528/0 (socket says 172.21.15.5:58528) 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77dffff640 1 -- 172.21.15.5:0/4174481963 learned_addr learned my addr 172.21.15.5:0/4174481963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77e7fff640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e8100e20 0x7f77e8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77eccdd640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e8102ef0 0x7f77e81a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77dffff640 1 -- 172.21.15.5:0/4174481963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e8102ef0 msgr2=0x7f77e81a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77dffff640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e8102ef0 0x7f77e81a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77dffff640 1 -- 172.21.15.5:0/4174481963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e8100e20 msgr2=0x7f77e8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.687+0000 7f77dffff640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e8100e20 0x7f77e8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.688+0000 7f77dffff640 1 -- 172.21.15.5:0/4174481963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e81a4910 con 0x7f77e8102570 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.688+0000 7f77dffff640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8102570 0x7f77e8199890 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f77d0006010 tx=0x7f77d0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.688+0000 7f77e5ffb640 1 -- 172.21.15.5:0/4174481963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d003f070 con 0x7f77e8102570 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.688+0000 7f77e5ffb640 1 -- 172.21.15.5:0/4174481963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0031c30 con 0x7f77e8102570 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.688+0000 7f77ee740640 1 -- 172.21.15.5:0/4174481963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e81a4ae0 con 0x7f77e8102570 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.688+0000 7f77e5ffb640 1 -- 172.21.15.5:0/4174481963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0035030 con 0x7f77e8102570 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.688+0000 7f77ee740640 1 -- 172.21.15.5:0/4174481963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e81a4f90 con 0x7f77e8102570 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.689+0000 7f77ee740640 1 -- 172.21.15.5:0/4174481963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e8101200 con 0x7f77e8102570 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.690+0000 7f77e5ffb640 1 -- 172.21.15.5:0/4174481963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f77d0042070 con 0x7f77e8102570 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.691+0000 7f77e5ffb640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f77d40a2cf0 0x7f77d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.691+0000 7f77e7fff640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f77d40a2cf0 0x7f77d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:32.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.691+0000 7f77e5ffb640 1 -- 172.21.15.5:0/4174481963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f77d00efcf0 con 0x7f77e8102570 2024-09-11T22:24:32.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.691+0000 7f77e7fff640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f77d40a2cf0 0x7f77d40a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f77d8005fd0 tx=0x7f77d800b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:32.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.695+0000 7f77e5ffb640 1 -- 172.21.15.5:0/4174481963 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77d00f4070 con 0x7f77e8102570 2024-09-11T22:24:32.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.905+0000 7f77ee740640 1 -- 172.21.15.5:0/4174481963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77e8109090 con 0x7f77e8102570 2024-09-11T22:24:32.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.906+0000 7f77e5ffb640 1 -- 172.21.15.5:0/4174481963 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11974 (secure 0 0 0) 0x7f77d0031de0 con 0x7f77e8102570 2024-09-11T22:24:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:24:32.907 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-11T22:23:49:156575+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:23:48.779139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:24:32.907 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-11T22:24:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.908+0000 7f77ee740640 1 -- 172.21.15.5:0/4174481963 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f77d40a2cf0 msgr2=0x7f77d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.908+0000 7f77ee740640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f77d40a2cf0 0x7f77d40a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f77d8005fd0 tx=0x7f77d800b390 comp rx=0 tx=0).stop 2024-09-11T22:24:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.908+0000 7f77ee740640 1 -- 172.21.15.5:0/4174481963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8102570 msgr2=0x7f77e8199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.908+0000 7f77ee740640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8102570 0x7f77e8199890 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f77d0006010 tx=0x7f77d0003c10 comp rx=0 tx=0).stop 2024-09-11T22:24:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.908+0000 7f77ee740640 1 -- 172.21.15.5:0/4174481963 shutdown_connections 2024-09-11T22:24:32.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.908+0000 7f77ee740640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77e8102ef0 0x7f77e81a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:32.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.908+0000 7f77ee740640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77e8102570 0x7f77e8199890 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:32.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.908+0000 7f77ee740640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f77d40a2cf0 0x7f77d40a51b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:32.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.908+0000 7f77ee740640 1 --2- 172.21.15.5:0/4174481963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77e8100e20 0x7f77e8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:32.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.908+0000 7f77ee740640 1 -- 172.21.15.5:0/4174481963 >> 172.21.15.5:0/4174481963 conn(0x7f77e80fc7d0 msgr2=0x7f77e8105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:32.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.909+0000 7f77ee740640 1 -- 172.21.15.5:0/4174481963 shutdown_connections 2024-09-11T22:24:32.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:32.909+0000 7f77ee740640 1 -- 172.21.15.5:0/4174481963 wait complete. 2024-09-11T22:24:33.306 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 11.65 MB/sec warmup 51 sec latency 12208.157 ms 2024-09-11T22:24:33.724 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 76699de --release --await' 2024-09-11T22:24:33.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 76699de --release --await 2024-09-11T22:24:33.985 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:34.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:33 smithi005 ceph-mon[30943]: pgmap v73: 97 pgs: 97 active+clean; 31 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 639 KiB/s wr, 150 op/s 2024-09-11T22:24:34.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:33 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4174481963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:24:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:33 smithi172 ceph-mon[35461]: pgmap v73: 97 pgs: 97 active+clean; 31 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 639 KiB/s wr, 150 op/s 2024-09-11T22:24:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:33 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4174481963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:24:34.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106769 11.43 MB/sec warmup 52 sec latency 13208.420 ms 2024-09-11T22:24:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:33 smithi159 ceph-mon[35534]: pgmap v73: 97 pgs: 97 active+clean; 31 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 639 KiB/s wr, 150 op/s 2024-09-11T22:24:34.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:33 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4174481963' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:24:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.508+0000 7f8448157640 1 -- 172.21.15.5:0/2407417426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8440101410 msgr2=0x7f844010d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.508+0000 7f8448157640 1 --2- 172.21.15.5:0/2407417426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8440101410 0x7f844010d940 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8438009cd0 tx=0x7f843802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.508+0000 7f8448157640 1 -- 172.21.15.5:0/2407417426 shutdown_connections 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.508+0000 7f8448157640 1 --2- 172.21.15.5:0/2407417426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8440101410 0x7f844010d940 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.508+0000 7f8448157640 1 --2- 172.21.15.5:0/2407417426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8440100a70 0x7f8440100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.508+0000 7f8448157640 1 --2- 172.21.15.5:0/2407417426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8440106a70 0x7f8440106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.508+0000 7f8448157640 1 -- 172.21.15.5:0/2407417426 >> 172.21.15.5:0/2407417426 conn(0x7f84400fc790 msgr2=0x7f84400febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8448157640 1 -- 172.21.15.5:0/2407417426 shutdown_connections 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8448157640 1 -- 172.21.15.5:0/2407417426 wait complete. 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8448157640 1 Processor -- start 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8448157640 1 -- start start 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8448157640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8440100a70 0x7f84400fce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8448157640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8440101410 0x7f84400fd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8448157640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8440106a70 0x7f84400fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8448157640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8440110e90 con 0x7f8440100a70 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8448157640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8440110d10 con 0x7f8440101410 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8448157640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8440111010 con 0x7f8440106a70 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.509+0000 7f8445ef3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8440100a70 0x7f84400fce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8445ef3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8440100a70 0x7f84400fce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58554/0 (socket says 172.21.15.5:58554) 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8445ef3640 1 -- 172.21.15.5:0/3262047032 learned_addr learned my addr 172.21.15.5:0/3262047032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f84466f4640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8440106a70 0x7f84400fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f84456f2640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8440101410 0x7f84400fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8445ef3640 1 -- 172.21.15.5:0/3262047032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8440106a70 msgr2=0x7f84400fd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8445ef3640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8440106a70 0x7f84400fd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8445ef3640 1 -- 172.21.15.5:0/3262047032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8440101410 msgr2=0x7f84400fd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8445ef3640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8440101410 0x7f84400fd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8445ef3640 1 -- 172.21.15.5:0/3262047032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84401a6920 con 0x7f8440100a70 2024-09-11T22:24:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f84466f4640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8440106a70 0x7f84400fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f84456f2640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8440101410 0x7f84400fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8445ef3640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8440100a70 0x7f84400fce20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f843c00d950 tx=0x7f843c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8436ffd640 1 -- 172.21.15.5:0/3262047032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c017070 con 0x7f8440100a70 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8448157640 1 -- 172.21.15.5:0/3262047032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84401a6c10 con 0x7f8440100a70 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8436ffd640 1 -- 172.21.15.5:0/3262047032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c009c30 con 0x7f8440100a70 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8436ffd640 1 -- 172.21.15.5:0/3262047032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c00f6b0 con 0x7f8440100a70 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.510+0000 7f8448157640 1 -- 172.21.15.5:0/3262047032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84401a7150 con 0x7f8440100a70 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.512+0000 7f8436ffd640 1 -- 172.21.15.5:0/3262047032 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f843c0109d0 con 0x7f8440100a70 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f844a530740 1 Processor -- start 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f844a530740 1 -- start start 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f844a530740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26060 0x555f50c26440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f844a530740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 0x555f50c30d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f844a530740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c33630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f844a530740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f50b8b620 con 0x555f50c31240 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f844a530740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555f50b32e50 con 0x555f50c26980 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f844a530740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555f50b875e0 con 0x555f50c26060 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f8436ffd640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f842c0a2cf0 0x7f842c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f84456f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c33630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f84456f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c33630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58562/0 (socket says 172.21.15.5:58562) 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f84456f2640 1 -- 172.21.15.5:0/77981091 learned_addr learned my addr 172.21.15.5:0/77981091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.513+0000 7f8445ef3640 1 --2- 172.21.15.5:0/77981091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 0x555f50c30d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f84466f4640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f842c0a2cf0 0x7f842c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f84466f4640 1 --2- 172.21.15.5:0/77981091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26060 0x555f50c26440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f8436ffd640 1 -- 172.21.15.5:0/3262047032 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f843c0cf150 con 0x7f8440100a70 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f84456f2640 1 -- 172.21.15.5:0/77981091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26060 msgr2=0x555f50c26440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f84456f2640 1 --2- 172.21.15.5:0/77981091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26060 0x555f50c26440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f84456f2640 1 -- 172.21.15.5:0/77981091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 msgr2=0x555f50c30d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f84456f2640 1 --2- 172.21.15.5:0/77981091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 0x555f50c30d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f84456f2640 1 -- 172.21.15.5:0/77981091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f50b23d40 con 0x555f50c31240 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f84456f2640 1 --2- 172.21.15.5:0/77981091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c33630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f843000a9d0 tx=0x7f843000add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb059b168e3e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f84466f4640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f842c0a2cf0 0x7f842c0a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fe0 tx=0x7f8438005d80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f842bfff640 1 -- 172.21.15.5:0/77981091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430015070 con 0x555f50c31240 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f842bfff640 1 -- 172.21.15.5:0/77981091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843000cc00 con 0x555f50c31240 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.514+0000 7f842bfff640 1 -- 172.21.15.5:0/77981091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430018560 con 0x555f50c31240 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 -- 172.21.15.5:0/77981091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 msgr2=0x555f50c33630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 --2- 172.21.15.5:0/77981091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c33630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f843000a9d0 tx=0x7f843000add0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 -- 172.21.15.5:0/77981091 shutdown_connections 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 --2- 172.21.15.5:0/77981091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c33630 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 --2- 172.21.15.5:0/77981091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 0x555f50c30d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 --2- 172.21.15.5:0/77981091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26060 0x555f50c26440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 -- 172.21.15.5:0/77981091 >> 172.21.15.5:0/77981091 conn(0x555f50c225f0 msgr2=0x555f50c229d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 -- 172.21.15.5:0/77981091 shutdown_connections 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 -- 172.21.15.5:0/77981091 wait complete. 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 -- start start 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.515+0000 7f844a530740 1 Processor -- start 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f844a530740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f844a530740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 0x555f50c2e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f844a530740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c31240 0x555f50c2eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f844a530740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f50c33e10 con 0x555f50c26060 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f844a530740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555f50b13f10 con 0x555f50c31240 2024-09-11T22:24:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f844a530740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555f50c33f90 con 0x555f50c26980 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f844a530740 10 client.0 _ll_register_callbacks cb 0x555f50b37db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f8445ef3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f8445ef3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58574/0 (socket says 172.21.15.5:58574) 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f8445ef3640 1 -- 172.21.15.5:0/3365764056 learned_addr learned my addr 172.21.15.5:0/3365764056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f84466f4640 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c31240 0x555f50c2eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f84456f2640 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 0x555f50c2e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f8445ef3640 1 -- 172.21.15.5:0/3365764056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 msgr2=0x555f50c2e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f8445ef3640 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 0x555f50c2e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f8445ef3640 1 -- 172.21.15.5:0/3365764056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c31240 msgr2=0x555f50c2eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f8445ef3640 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c31240 0x555f50c2eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f8445ef3640 1 -- 172.21.15.5:0/3365764056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f5089a580 con 0x555f50c26060 2024-09-11T22:24:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f84456f2640 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 0x555f50c2e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f84466f4640 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c31240 0x555f50c2eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f8445ef3640 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2e070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f843c097310 tx=0x7f843c09b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3aba4e6252d268 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.516+0000 7f84297fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f84297fa640 1 -- 172.21.15.5:0/3365764056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c020040 con 0x555f50c26060 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f84297fa640 1 -- 172.21.15.5:0/3365764056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f843c084b70 con 0x555f50c26060 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f844a530740 1 -- 172.21.15.5:0/3365764056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843000cc00 con 0x555f50c26060 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f84297fa640 1 -- 172.21.15.5:0/3365764056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f843c09b630 con 0x555f50c26060 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f844a530740 10 client.16089 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f844a530740 1 -- 172.21.15.5:0/3365764056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f50964960 con 0x555f50c26060 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f84297fa640 1 -- 172.21.15.5:0/3365764056 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843000cc00 con 0x555f50c26060 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f84297fa640 1 -- 172.21.15.5:0/3365764056 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f843c060bc0 con 0x555f50c26060 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f844a530740 10 client.16089 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f844a530740 10 client.16089 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:24:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f844a530740 20 client.16089 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f844a530740 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f50c40e00 0x555f50c432a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f844a530740 4 client.16089 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f844a530740 1 -- 172.21.15.5:0/3365764056 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f50b98b90 con 0x555f50c40e00 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.517+0000 7f84456f2640 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f50c40e00 0x555f50c432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.518+0000 7f84456f2640 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f50c40e00 0x555f50c432a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47af547e70ec7832 server_cookie=ddea55b20f9a9138 in_seq=0 out_seq=0 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.518+0000 7f84297fa640 10 client.16089 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f84297fa640 1 -- 172.21.15.5:0/3365764056 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f50b98b90 con 0x555f50c40e00 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f84297fa640 10 client.16089 handle_command_reply: tid=0 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 client.16089 shutdown 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 -- 172.21.15.5:0/3365764056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 msgr2=0x555f50c2e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2e070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f843c097310 tx=0x7f843c09b050 comp rx=0 tx=0).stop 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 -- 172.21.15.5:0/3365764056 shutdown_connections 2024-09-11T22:24:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f50c40e00 0x555f50c432a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c31240 0x555f50c2eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 0x555f50c2e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 --2- 172.21.15.5:0/3365764056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2e070 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f84297fa640 0 client.16089 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 -- 172.21.15.5:0/3365764056 >> 172.21.15.5:0/3365764056 conn(0x555f50c225f0 msgr2=0x555f50c282d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 -- 172.21.15.5:0/3365764056 shutdown_connections 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 1 -- 172.21.15.5:0/3365764056 wait complete. 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.519+0000 7f844a530740 20 client.16089 trim_cache size 0 max 16384 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f844a530740 1 Processor -- start 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f844a530740 1 -- start start 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f844a530740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f844a530740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 0x555f50c48b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f844a530740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c31240 0x555f50c490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f844a530740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84401afbd0 con 0x555f50c26060 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f844a530740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b17d0 con 0x555f50c26980 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f844a530740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b02a0 con 0x555f50c31240 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f8445ef3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f8445ef3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58580/0 (socket says 172.21.15.5:58580) 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.536+0000 7f8445ef3640 1 -- 172.21.15.5:0/2980422435 learned_addr learned my addr 172.21.15.5:0/2980422435 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f8445ef3640 1 -- 172.21.15.5:0/2980422435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c31240 msgr2=0x555f50c490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f8445ef3640 1 --2- 172.21.15.5:0/2980422435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c31240 0x555f50c490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f84456f2640 1 --2- 172.21.15.5:0/2980422435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 0x555f50c48b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f8445ef3640 1 -- 172.21.15.5:0/2980422435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 msgr2=0x555f50c48b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f8445ef3640 1 --2- 172.21.15.5:0/2980422435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 0x555f50c48b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f8445ef3640 1 -- 172.21.15.5:0/2980422435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f50c22420 con 0x555f50c26060 2024-09-11T22:24:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f84456f2640 1 --2- 172.21.15.5:0/2980422435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 0x555f50c48b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:24:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f8445ef3640 1 --2- 172.21.15.5:0/2980422435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2cfd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f843c0058c0 tx=0x7f843c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41a3456b8611d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f842b7fe640 1 -- 172.21.15.5:0/2980422435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c020070 con 0x555f50c26060 2024-09-11T22:24:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f842b7fe640 1 -- 172.21.15.5:0/2980422435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c0a42a0 con 0x555f50c26060 2024-09-11T22:24:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.537+0000 7f842b7fe640 1 -- 172.21.15.5:0/2980422435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c084dd0 con 0x555f50c26060 2024-09-11T22:24:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 -- 172.21.15.5:0/2980422435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 msgr2=0x555f50c2cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 --2- 172.21.15.5:0/2980422435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2cfd0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f843c0058c0 tx=0x7f843c005f70 comp rx=0 tx=0).stop 2024-09-11T22:24:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 -- 172.21.15.5:0/2980422435 shutdown_connections 2024-09-11T22:24:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 --2- 172.21.15.5:0/2980422435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c31240 0x555f50c490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 --2- 172.21.15.5:0/2980422435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26980 0x555f50c48b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 --2- 172.21.15.5:0/2980422435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c26060 0x555f50c2cfd0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 -- 172.21.15.5:0/2980422435 >> 172.21.15.5:0/2980422435 conn(0x555f50c225f0 msgr2=0x555f50c300f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 -- 172.21.15.5:0/2980422435 shutdown_connections 2024-09-11T22:24:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 -- 172.21.15.5:0/2980422435 wait complete. 2024-09-11T22:24:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 -- start start 2024-09-11T22:24:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.538+0000 7f844a530740 1 Processor -- start 2024-09-11T22:24:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f844a530740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26060 0x555f50c391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f844a530740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 0x555f50c39720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f844a530740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f844a530740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b1040 con 0x555f50c31240 2024-09-11T22:24:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f844a530740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f84401b0970 con 0x555f50c26060 2024-09-11T22:24:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f844a530740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f84401adbb0 con 0x555f50c26980 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f844a530740 10 client.0 _ll_register_callbacks cb 0x7f843c0860d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84466f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84466f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c361b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58590/0 (socket says 172.21.15.5:58590) 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f8445ef3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26060 0x555f50c391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84466f4640 1 -- 172.21.15.5:0/18606894 learned_addr learned my addr 172.21.15.5:0/18606894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84456f2640 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 0x555f50c39720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84466f4640 1 -- 172.21.15.5:0/18606894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 msgr2=0x555f50c39720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84466f4640 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 0x555f50c39720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84466f4640 1 -- 172.21.15.5:0/18606894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26060 msgr2=0x555f50c391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84466f4640 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26060 0x555f50c391e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84466f4640 1 -- 172.21.15.5:0/18606894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f50c366f0 con 0x555f50c31240 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84466f4640 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c361b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8438009cd0 tx=0x7f8438009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb5dd3b5ccad895 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84297fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.539+0000 7f84297fa640 1 -- 172.21.15.5:0/18606894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438004590 con 0x555f50c31240 2024-09-11T22:24:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f844a530740 1 -- 172.21.15.5:0/18606894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843c0a42a0 con 0x555f50c31240 2024-09-11T22:24:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f84297fa640 1 -- 172.21.15.5:0/18606894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f8438004730 con 0x555f50c31240 2024-09-11T22:24:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f84297fa640 1 -- 172.21.15.5:0/18606894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f84380305a0 con 0x555f50c31240 2024-09-11T22:24:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f844a530740 10 client.16095 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f844a530740 1 -- 172.21.15.5:0/18606894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f50cac2a0 con 0x555f50c31240 2024-09-11T22:24:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f84297fa640 1 -- 172.21.15.5:0/18606894 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f843c0a42a0 con 0x555f50c31240 2024-09-11T22:24:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f84297fa640 1 -- 172.21.15.5:0/18606894 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f8438030070 con 0x555f50c31240 2024-09-11T22:24:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f844a530740 10 client.16095 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f844a530740 10 client.16095 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:24:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f844a530740 20 client.16095 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f844a530740 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f50cacad0 0x555f50caeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f844a530740 4 client.16095 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "76699de", "release": true, "await": true}] 2024-09-11T22:24:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f844a530740 1 -- 172.21.15.5:0/18606894 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "76699de", "release": true, "await": true}) -- 0x555f50b930a0 con 0x555f50cacad0 2024-09-11T22:24:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.540+0000 7f8445ef3640 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f50cacad0 0x555f50caeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.541+0000 7f8445ef3640 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f50cacad0 0x555f50caeef0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07b479759489ea2 server_cookie=fd4a3a9eb37824e9 in_seq=0 out_seq=0 2024-09-11T22:24:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.541+0000 7f84297fa640 10 client.16095 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:34.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f84297fa640 1 -- 172.21.15.5:0/18606894 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x555f50b930a0 con 0x555f50cacad0 2024-09-11T22:24:34.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f84297fa640 10 client.16095 handle_command_reply: tid=0 2024-09-11T22:24:34.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 client.16095 shutdown 2024-09-11T22:24:34.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 -- 172.21.15.5:0/18606894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 msgr2=0x555f50c361b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:34.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c361b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8438009cd0 tx=0x7f8438009960 comp rx=0 tx=0).stop 2024-09-11T22:24:34.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 -- 172.21.15.5:0/18606894 shutdown_connections 2024-09-11T22:24:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f50cacad0 0x555f50caeef0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f50c31240 0x555f50c361b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f50c26980 0x555f50c39720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f84297fa640 0 client.16095 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 --2- 172.21.15.5:0/18606894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f50c26060 0x555f50c391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 -- 172.21.15.5:0/18606894 >> 172.21.15.5:0/18606894 conn(0x555f50c225f0 msgr2=0x555f50c20f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 -- 172.21.15.5:0/18606894 shutdown_connections 2024-09-11T22:24:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 1 -- 172.21.15.5:0/18606894 wait complete. 2024-09-11T22:24:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.555+0000 7f844a530740 20 client.16095 trim_cache size 0 max 16384 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 30, 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 8, 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout: "76699de": { 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 8, 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:24:34.559 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 133.4, 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:34.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:34.561 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:24:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 -- 172.21.15.5:0/3262047032 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f842c0a2cf0 msgr2=0x7f842c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f842c0a2cf0 0x7f842c0a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fe0 tx=0x7f8438005d80 comp rx=0 tx=0).stop 2024-09-11T22:24:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 -- 172.21.15.5:0/3262047032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8440100a70 msgr2=0x7f84400fce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8440100a70 0x7f84400fce20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f843c00d950 tx=0x7f843c00de10 comp rx=0 tx=0).stop 2024-09-11T22:24:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 -- 172.21.15.5:0/3262047032 shutdown_connections 2024-09-11T22:24:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8440106a70 0x7f84400fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f842c0a2cf0 0x7f842c0a51b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8440101410 0x7f84400fd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 --2- 172.21.15.5:0/3262047032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8440100a70 0x7f84400fce20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 -- 172.21.15.5:0/3262047032 >> 172.21.15.5:0/3262047032 conn(0x7f84400fc790 msgr2=0x7f844006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 -- 172.21.15.5:0/3262047032 shutdown_connections 2024-09-11T22:24:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:34.556+0000 7f8448157640 1 -- 172.21.15.5:0/3262047032 wait complete. 2024-09-11T22:24:35.306 INFO:tasks.workunit.client.0.smithi005.stdout: 1 108929 11.37 MB/sec warmup 53 sec latency 13461.145 ms 2024-09-11T22:24:35.446 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '76699de', total seconds elapsed: 16.9 2024-09-11T22:24:35.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:24:36.085 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:36.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:35 smithi172 ceph-mon[35461]: pgmap v74: 97 pgs: 97 active+clean; 31 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 63 op/s 2024-09-11T22:24:36.208 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:35 smithi005 ceph-mon[30943]: pgmap v74: 97 pgs: 97 active+clean; 31 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 63 op/s 2024-09-11T22:24:36.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 111288 11.41 MB/sec warmup 54 sec latency 10.603 ms 2024-09-11T22:24:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:35 smithi159 ceph-mon[35534]: pgmap v74: 97 pgs: 97 active+clean; 31 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 63 op/s 2024-09-11T22:24:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.626+0000 7f4e79c07640 1 -- 172.21.15.5:0/3440522001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e741082a0 msgr2=0x7f4e7410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.626+0000 7f4e79c07640 1 --2- 172.21.15.5:0/3440522001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e741082a0 0x7f4e7410ec40 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009cd0 tx=0x7f4e6c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.626+0000 7f4e79c07640 1 -- 172.21.15.5:0/3440522001 shutdown_connections 2024-09-11T22:24:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.626+0000 7f4e79c07640 1 --2- 172.21.15.5:0/3440522001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e741082a0 0x7f4e7410ec40 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.626+0000 7f4e79c07640 1 --2- 172.21.15.5:0/3440522001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e741036e0 0x7f4e74107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.626+0000 7f4e79c07640 1 --2- 172.21.15.5:0/3440522001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e74102d30 0x7f4e74103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.626+0000 7f4e79c07640 1 -- 172.21.15.5:0/3440522001 >> 172.21.15.5:0/3440522001 conn(0x7f4e740fc7d0 msgr2=0x7f4e740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.626+0000 7f4e79c07640 1 -- 172.21.15.5:0/3440522001 shutdown_connections 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.626+0000 7f4e79c07640 1 -- 172.21.15.5:0/3440522001 wait complete. 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.626+0000 7f4e79c07640 1 Processor -- start 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e79c07640 1 -- start start 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e79c07640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e74102d30 0x7f4e74198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e79c07640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e741036e0 0x7f4e74198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e79c07640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e741082a0 0x7f4e7419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e79c07640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e741120a0 con 0x7f4e741036e0 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e79c07640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e74111f20 con 0x7f4e74102d30 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e79c07640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e74112220 con 0x7f4e741082a0 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e72ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e741036e0 0x7f4e74198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e737fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e74102d30 0x7f4e74198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e72ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e741036e0 0x7f4e74198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58594/0 (socket says 172.21.15.5:58594) 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e72ffd640 1 -- 172.21.15.5:0/4284273434 learned_addr learned my addr 172.21.15.5:0/4284273434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e73fff640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e741082a0 0x7f4e7419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e72ffd640 1 -- 172.21.15.5:0/4284273434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e741082a0 msgr2=0x7f4e7419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e741082a0 0x7f4e7419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e72ffd640 1 -- 172.21.15.5:0/4284273434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e74102d30 msgr2=0x7f4e74198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e74102d30 0x7f4e74198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e72ffd640 1 -- 172.21.15.5:0/4284273434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7419d510 con 0x7f4e741036e0 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.627+0000 7f4e737fe640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e74102d30 0x7f4e74198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.628+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e741036e0 0x7f4e74198aa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800e9c0 tx=0x7f4e6800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.628+0000 7f4e70ff9640 1 -- 172.21.15.5:0/4284273434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800ccc0 con 0x7f4e741036e0 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.628+0000 7f4e79c07640 1 -- 172.21.15.5:0/4284273434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7419d800 con 0x7f4e741036e0 2024-09-11T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.628+0000 7f4e70ff9640 1 -- 172.21.15.5:0/4284273434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e68005500 con 0x7f4e741036e0 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.628+0000 7f4e70ff9640 1 -- 172.21.15.5:0/4284273434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e68010690 con 0x7f4e741036e0 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.628+0000 7f4e79c07640 1 -- 172.21.15.5:0/4284273434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e741a50e0 con 0x7f4e741036e0 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.630+0000 7f4e70ff9640 1 -- 172.21.15.5:0/4284273434 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4e68004e80 con 0x7f4e741036e0 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e7bfe0740 1 Processor -- start 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e7bfe0740 1 -- start start 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e70ff9640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e5c0a2cf0 0x7f4e5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef573880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 0x5574ef57e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef580a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574ef4d92e0 con 0x5574ef5734a0 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5574ef480ce0 con 0x5574ef57e680 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5574ef4d52a0 con 0x5574ef573dc0 2024-09-11T22:24:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef573880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e73fff640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e5c0a2cf0 0x7f4e5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef573880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58606/0 (socket says 172.21.15.5:58606) 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e737fe640 1 -- 172.21.15.5:0/2345476203 learned_addr learned my addr 172.21.15.5:0/2345476203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/2345476203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 0x5574ef57e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e70ff9640 1 -- 172.21.15.5:0/4284273434 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f4e680cf7f0 con 0x7f4e741036e0 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e73fff640 1 --2- 172.21.15.5:0/2345476203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef580a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e72ffd640 1 -- 172.21.15.5:0/2345476203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 msgr2=0x5574ef580a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/2345476203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef580a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e72ffd640 1 -- 172.21.15.5:0/2345476203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 msgr2=0x5574ef573880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/2345476203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef573880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.631+0000 7f4e72ffd640 1 -- 172.21.15.5:0/2345476203 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ef465440 con 0x5574ef573dc0 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.632+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/2345476203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 0x5574ef57e140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4e680ccf00 tx=0x7f4e68004b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36f4e58f34475e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.632+0000 7f4e73fff640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e5c0a2cf0 0x7f4e5c0a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c004ab0 tx=0x7f4e6c009950 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:36.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.632+0000 7f4e59ffb640 1 -- 172.21.15.5:0/2345476203 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6809b630 con 0x5574ef573dc0 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.632+0000 7f4e59ffb640 1 -- 172.21.15.5:0/2345476203 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6809c3d0 con 0x5574ef573dc0 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.632+0000 7f4e59ffb640 1 -- 172.21.15.5:0/2345476203 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800ce60 con 0x5574ef573dc0 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2345476203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 msgr2=0x5574ef57e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/2345476203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 0x5574ef57e140 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4e680ccf00 tx=0x7f4e68004b00 comp rx=0 tx=0).stop 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2345476203 shutdown_connections 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/2345476203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef580a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/2345476203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 0x5574ef57e140 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/2345476203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef573880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2345476203 >> 172.21.15.5:0/2345476203 conn(0x5574ef56fa30 msgr2=0x5574ef56fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2345476203 shutdown_connections 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2345476203 wait complete. 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 -- start start 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.633+0000 7f4e7bfe0740 1 Processor -- start 2024-09-11T22:24:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef57dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 0x5574ef57b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef57b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574ef581250 con 0x5574ef5734a0 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5574ef494050 con 0x5574ef57e680 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5574ef5813d0 con 0x5574ef573dc0 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e7bfe0740 10 client.0 _ll_register_callbacks cb 0x5574ef4d4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef57dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e72ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 0x5574ef57b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef57dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58622/0 (socket says 172.21.15.5:58622) 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e737fe640 1 -- 172.21.15.5:0/2102199682 learned_addr learned my addr 172.21.15.5:0/2102199682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e73fff640 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef57b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e737fe640 1 -- 172.21.15.5:0/2102199682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 msgr2=0x5574ef57b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e737fe640 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 0x5574ef57b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e737fe640 1 -- 172.21.15.5:0/2102199682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 msgr2=0x5574ef57b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e737fe640 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef57b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e737fe640 1 -- 172.21.15.5:0/2102199682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ef24a3c0 con 0x5574ef5734a0 2024-09-11T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 0x5574ef57b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e737fe640 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef57dfd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4e64012cf0 tx=0x7f4e6400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c929b7d57f24c30c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e52ffd640 1 -- 172.21.15.5:0/2102199682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64004020 con 0x5574ef5734a0 2024-09-11T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e52ffd640 1 -- 172.21.15.5:0/2102199682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f4e640041c0 con 0x5574ef5734a0 2024-09-11T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.634+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2102199682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e6809c3d0 con 0x5574ef5734a0 2024-09-11T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e52ffd640 1 -- 172.21.15.5:0/2102199682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e640043c0 con 0x5574ef5734a0 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e7bfe0740 10 client.16107 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2102199682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574ef58cc80 con 0x5574ef5734a0 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e52ffd640 1 -- 172.21.15.5:0/2102199682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6809c3d0 con 0x5574ef5734a0 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e52ffd640 1 -- 172.21.15.5:0/2102199682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4e6401c470 con 0x5574ef5734a0 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e7bfe0740 10 client.16107 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e7bfe0740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e7bfe0740 20 client.16107 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5574ef58e190 0x5574ef5905f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e7bfe0740 4 client.16107 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2102199682 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574ef4e6850 con 0x5574ef58e190 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.635+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5574ef58e190 0x5574ef5905f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.636+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5574ef58e190 0x5574ef5905f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5886f5232675682b server_cookie=ee4cdd8f379bfee8 in_seq=0 out_seq=0 2024-09-11T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.636+0000 7f4e52ffd640 10 client.16107 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e52ffd640 1 -- 172.21.15.5:0/2102199682 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574ef4e6850 con 0x5574ef58e190 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e52ffd640 10 client.16107 handle_command_reply: tid=0 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 client.16107 shutdown 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2102199682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 msgr2=0x5574ef57dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef57dfd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4e64012cf0 tx=0x7f4e6400f790 comp rx=0 tx=0).stop 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2102199682 shutdown_connections 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5574ef58e190 0x5574ef5905f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef57b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef573dc0 0x5574ef57b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/2102199682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef57dfd0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2102199682 >> 172.21.15.5:0/2102199682 conn(0x5574ef56fa30 msgr2=0x5574ef575760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e52ffd640 0 client.16107 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2102199682 shutdown_connections 2024-09-11T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/2102199682 wait complete. 2024-09-11T22:24:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.637+0000 7f4e7bfe0740 20 client.16107 trim_cache size 0 max 16384 2024-09-11T22:24:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.655+0000 7f4e7bfe0740 1 Processor -- start 2024-09-11T22:24:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.655+0000 7f4e7bfe0740 1 -- start start 2024-09-11T22:24:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.655+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef56ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.655+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef573dc0 0x5574ef595ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.655+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef57e680 0x5574ef596400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.655+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e741ada10 con 0x5574ef5734a0 2024-09-11T22:24:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e741af610 con 0x5574ef573dc0 2024-09-11T22:24:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e741ae0e0 con 0x5574ef57e680 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef56ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef56ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58634/0 (socket says 172.21.15.5:58634) 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e737fe640 1 -- 172.21.15.5:0/3129811146 learned_addr learned my addr 172.21.15.5:0/3129811146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef573dc0 0x5574ef595ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e73fff640 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef57e680 0x5574ef596400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e737fe640 1 -- 172.21.15.5:0/3129811146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef57e680 msgr2=0x5574ef596400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e737fe640 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef57e680 0x5574ef596400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e737fe640 1 -- 172.21.15.5:0/3129811146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef573dc0 msgr2=0x5574ef595ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e737fe640 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef573dc0 0x5574ef595ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e737fe640 1 -- 172.21.15.5:0/3129811146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574ef57c600 con 0x5574ef5734a0 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e73fff640 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef57e680 0x5574ef596400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef573dc0 0x5574ef595ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e737fe640 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef56ea60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4e640059f0 tx=0x7f4e6401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905299b17c88a164 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e597fa640 1 -- 172.21.15.5:0/3129811146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64028030 con 0x5574ef5734a0 2024-09-11T22:24:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e597fa640 1 -- 172.21.15.5:0/3129811146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6401cad0 con 0x5574ef5734a0 2024-09-11T22:24:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.656+0000 7f4e597fa640 1 -- 172.21.15.5:0/3129811146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64005720 con 0x5574ef5734a0 2024-09-11T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.657+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/3129811146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 msgr2=0x5574ef56ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.657+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef56ea60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4e640059f0 tx=0x7f4e6401c6a0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.657+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/3129811146 shutdown_connections 2024-09-11T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.657+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef57e680 0x5574ef596400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.657+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef573dc0 0x5574ef595ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.657+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/3129811146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef5734a0 0x5574ef56ea60 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.657+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/3129811146 >> 172.21.15.5:0/3129811146 conn(0x5574ef56fa30 msgr2=0x5574ef56e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.657+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/3129811146 shutdown_connections 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.657+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/3129811146 wait complete. 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e7bfe0740 1 -- start start 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e7bfe0740 1 Processor -- start 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef5734a0 0x5574ef586b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef573dc0 0x5574ef583eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e7bfe0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef5843f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e741aee80 con 0x5574ef573dc0 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e741ae7b0 con 0x5574ef57e680 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e7bfe0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e741ab9f0 con 0x5574ef5734a0 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e7bfe0740 10 client.0 _ll_register_callbacks cb 0x7f4e64009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e72ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef573dc0 0x5574ef583eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef5734a0 0x5574ef586b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e72ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef573dc0 0x5574ef583eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58648/0 (socket says 172.21.15.5:58648) 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e737fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef5734a0 0x5574ef586b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:35896/0 (socket says 172.21.15.5:35896) 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e73fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef5843f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.658+0000 7f4e72ffd640 1 -- 172.21.15.5:0/1727568290 learned_addr learned my addr 172.21.15.5:0/1727568290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e737fe640 1 -- 172.21.15.5:0/1727568290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 msgr2=0x5574ef5843f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e737fe640 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef5843f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e737fe640 1 -- 172.21.15.5:0/1727568290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef573dc0 msgr2=0x5574ef583eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e737fe640 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef573dc0 0x5574ef583eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e737fe640 1 -- 172.21.15.5:0/1727568290 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574ef5849b0 con 0x5574ef5734a0 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef573dc0 0x5574ef583eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e737fe640 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef5734a0 0x5574ef586b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400f7d0 tx=0x7f4e64005470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1518c905c8088507 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e52ffd640 1 -- 172.21.15.5:0/1727568290 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64028040 con 0x5574ef5734a0 2024-09-11T22:24:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e52ffd640 1 -- 172.21.15.5:0/1727568290 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f4e64005660 con 0x5574ef5734a0 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/1727568290 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e6401cad0 con 0x5574ef5734a0 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e52ffd640 1 -- 172.21.15.5:0/1727568290 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4e6403b050 con 0x5574ef5734a0 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e7bfe0740 10 client.25591 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.659+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/1727568290 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5574ef5f8c40 con 0x5574ef5734a0 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e52ffd640 1 -- 172.21.15.5:0/1727568290 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e6401cad0 con 0x5574ef5734a0 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e52ffd640 1 -- 172.21.15.5:0/1727568290 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4e64010430 con 0x5574ef5734a0 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e7bfe0740 10 client.25591 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e7bfe0740 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e7bfe0740 20 client.25591 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5574ef5f9350 0x5574ef5fb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e7bfe0740 4 client.25591 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/1727568290 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574ef4e0d60 con 0x5574ef5f9350 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5574ef5f9350 0x5574ef5fb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e52ffd640 10 client.25591 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.660+0000 7f4e72ffd640 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5574ef5f9350 0x5574ef5fb770 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b19a907bee9ffb36 server_cookie=ad3eb2bc29989d3d in_seq=0 out_seq=0 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e52ffd640 1 -- 172.21.15.5:0/1727568290 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5574ef4e0d60 con 0x5574ef5f9350 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e52ffd640 10 client.25591 handle_command_reply: tid=0 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e7bfe0740 1 client.25591 shutdown 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/1727568290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef5734a0 msgr2=0x5574ef586b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef5734a0 0x5574ef586b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400f7d0 tx=0x7f4e64005470 comp rx=0 tx=0).stop 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/1727568290 shutdown_connections 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5574ef5f9350 0x5574ef5fb770 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5574ef57e680 0x5574ef5843f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574ef573dc0 0x5574ef583eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e7bfe0740 1 --2- 172.21.15.5:0/1727568290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5574ef5734a0 0x5574ef586b10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/1727568290 >> 172.21.15.5:0/1727568290 conn(0x5574ef56fa30 msgr2=0x5574ef56eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.661+0000 7f4e52ffd640 0 client.25591 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.662+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/1727568290 shutdown_connections 2024-09-11T22:24:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.662+0000 7f4e7bfe0740 1 -- 172.21.15.5:0/1727568290 wait complete. 2024-09-11T22:24:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.662+0000 7f4e7bfe0740 20 client.25591 trim_cache size 0 max 16384 2024-09-11T22:24:36.669 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:24:36.669 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:24:36.669 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:24:36.669 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:24:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.662+0000 7f4e79c07640 1 -- 172.21.15.5:0/4284273434 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e5c0a2cf0 msgr2=0x7f4e5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.662+0000 7f4e79c07640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e5c0a2cf0 0x7f4e5c0a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c004ab0 tx=0x7f4e6c009950 comp rx=0 tx=0).stop 2024-09-11T22:24:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.662+0000 7f4e79c07640 1 -- 172.21.15.5:0/4284273434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e741036e0 msgr2=0x7f4e74198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.662+0000 7f4e79c07640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e741036e0 0x7f4e74198aa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800e9c0 tx=0x7f4e6800ee80 comp rx=0 tx=0).stop 2024-09-11T22:24:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.663+0000 7f4e79c07640 1 -- 172.21.15.5:0/4284273434 shutdown_connections 2024-09-11T22:24:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.663+0000 7f4e79c07640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e741082a0 0x7f4e7419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.663+0000 7f4e79c07640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e741036e0 0x7f4e74198aa0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.663+0000 7f4e79c07640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e5c0a2cf0 0x7f4e5c0a51b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.663+0000 7f4e79c07640 1 --2- 172.21.15.5:0/4284273434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e74102d30 0x7f4e74198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.663+0000 7f4e79c07640 1 -- 172.21.15.5:0/4284273434 >> 172.21.15.5:0/4284273434 conn(0x7f4e740fc7d0 msgr2=0x7f4e74105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.663+0000 7f4e79c07640 1 -- 172.21.15.5:0/4284273434 shutdown_connections 2024-09-11T22:24:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:36.663+0000 7f4e79c07640 1 -- 172.21.15.5:0/4284273434 wait complete. 2024-09-11T22:24:37.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 114073 11.51 MB/sec warmup 55 sec latency 22.344 ms 2024-09-11T22:24:37.550 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:24:37.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:24:37.824 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:38.092 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:37 smithi005 ceph-mon[30943]: pgmap v75: 97 pgs: 97 active+clean; 31 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 63 op/s 2024-09-11T22:24:38.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:37 smithi172 ceph-mon[35461]: pgmap v75: 97 pgs: 97 active+clean; 31 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 63 op/s 2024-09-11T22:24:38.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 117071 11.61 MB/sec warmup 56 sec latency 30.360 ms 2024-09-11T22:24:38.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:37 smithi159 ceph-mon[35534]: pgmap v75: 97 pgs: 97 active+clean; 31 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 63 op/s 2024-09-11T22:24:38.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.333+0000 7f25520a5640 1 -- 172.21.15.5:0/4476359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254c100e20 msgr2=0x7f254c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:38.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.333+0000 7f25520a5640 1 --2- 172.21.15.5:0/4476359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254c100e20 0x7f254c101200 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f253c009900 tx=0x7f253c02a480 comp rx=0 tx=0).stop 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.333+0000 7f25520a5640 1 -- 172.21.15.5:0/4476359 shutdown_connections 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.333+0000 7f25520a5640 1 --2- 172.21.15.5:0/4476359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c102ef0 0x7f254c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.333+0000 7f25520a5640 1 --2- 172.21.15.5:0/4476359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f254c102570 0x7f254c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.333+0000 7f25520a5640 1 --2- 172.21.15.5:0/4476359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254c100e20 0x7f254c101200 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.333+0000 7f25520a5640 1 -- 172.21.15.5:0/4476359 >> 172.21.15.5:0/4476359 conn(0x7f254c0fc7d0 msgr2=0x7f254c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.333+0000 7f25520a5640 1 -- 172.21.15.5:0/4476359 shutdown_connections 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.333+0000 7f25520a5640 1 -- 172.21.15.5:0/4476359 wait complete. 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f25520a5640 1 Processor -- start 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f25520a5640 1 -- start start 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f25520a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c100e20 0x7f254c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f25520a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f254c102570 0x7f254c198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f25520a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254c102ef0 0x7f254c19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f25520a5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f254c110f80 con 0x7f254c102ef0 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f25520a5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f254c110e00 con 0x7f254c102570 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f25520a5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f254c111100 con 0x7f254c100e20 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f254bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254c102ef0 0x7f254c19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f254affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f254c102570 0x7f254c198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f254affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f254c102570 0x7f254c198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50908/0 (socket says 172.21.15.5:50908) 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f254bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254c102ef0 0x7f254c19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58680/0 (socket says 172.21.15.5:58680) 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f254affd640 1 -- 172.21.15.5:0/2471375095 learned_addr learned my addr 172.21.15.5:0/2471375095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.334+0000 7f254b7fe640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c100e20 0x7f254c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f254affd640 1 -- 172.21.15.5:0/2471375095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c100e20 msgr2=0x7f254c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f254affd640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c100e20 0x7f254c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f254affd640 1 -- 172.21.15.5:0/2471375095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254c102ef0 msgr2=0x7f254c19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f254affd640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254c102ef0 0x7f254c19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f254affd640 1 -- 172.21.15.5:0/2471375095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254c19d550 con 0x7f254c102570 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f254bfff640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254c102ef0 0x7f254c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f254affd640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f254c102570 0x7f254c198ae0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f254000e9f0 tx=0x7f254000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f2548ff9640 1 -- 172.21.15.5:0/2471375095 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254000cd80 con 0x7f254c102570 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f25520a5640 1 -- 172.21.15.5:0/2471375095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254c19d840 con 0x7f254c102570 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f2548ff9640 1 -- 172.21.15.5:0/2471375095 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2540010910 con 0x7f254c102570 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f2548ff9640 1 -- 172.21.15.5:0/2471375095 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25400188f0 con 0x7f254c102570 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.335+0000 7f25520a5640 1 -- 172.21.15.5:0/2471375095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254c1a5120 con 0x7f254c102570 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.336+0000 7f25520a5640 1 -- 172.21.15.5:0/2471375095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254c101200 con 0x7f254c102570 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.337+0000 7f2548ff9640 1 -- 172.21.15.5:0/2471375095 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2540010ac0 con 0x7f254c102570 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.338+0000 7f2548ff9640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2cf0 0x7f25340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.338+0000 7f2548ff9640 1 -- 172.21.15.5:0/2471375095 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f25400cfd60 con 0x7f254c102570 2024-09-11T22:24:38.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.338+0000 7f254b7fe640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2cf0 0x7f25340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:38.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.339+0000 7f254b7fe640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2cf0 0x7f25340a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f253c009230 tx=0x7f253c0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:38.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.342+0000 7f2548ff9640 1 -- 172.21.15.5:0/2471375095 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f254009c290 con 0x7f254c102570 2024-09-11T22:24:38.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.577+0000 7f25520a5640 1 -- 172.21.15.5:0/2471375095 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f254c199900 con 0x7f254c102570 2024-09-11T22:24:38.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.578+0000 7f2548ff9640 1 -- 172.21.15.5:0/2471375095 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11974 (secure 0 0 0) 0x7f254c199900 con 0x7f254c102570 2024-09-11T22:24:38.580 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:24:38.580 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-11T22:23:49:156575+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:23:48.779139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:24:38.580 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-11T22:24:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.580+0000 7f25520a5640 1 -- 172.21.15.5:0/2471375095 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2cf0 msgr2=0x7f25340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.580+0000 7f25520a5640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2cf0 0x7f25340a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f253c009230 tx=0x7f253c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.580+0000 7f25520a5640 1 -- 172.21.15.5:0/2471375095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f254c102570 msgr2=0x7f254c198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.580+0000 7f25520a5640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f254c102570 0x7f254c198ae0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f254000e9f0 tx=0x7f254000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.580+0000 7f25520a5640 1 -- 172.21.15.5:0/2471375095 shutdown_connections 2024-09-11T22:24:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.580+0000 7f25520a5640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f254c102ef0 0x7f254c19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.580+0000 7f25520a5640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2cf0 0x7f25340a51b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.580+0000 7f25520a5640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f254c102570 0x7f254c198ae0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.580+0000 7f25520a5640 1 --2- 172.21.15.5:0/2471375095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f254c100e20 0x7f254c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.580+0000 7f25520a5640 1 -- 172.21.15.5:0/2471375095 >> 172.21.15.5:0/2471375095 conn(0x7f254c0fc7d0 msgr2=0x7f254c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.581+0000 7f25520a5640 1 -- 172.21.15.5:0/2471375095 shutdown_connections 2024-09-11T22:24:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:38.581+0000 7f25520a5640 1 -- 172.21.15.5:0/2471375095 wait complete. 2024-09-11T22:24:39.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:38 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2471375095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:24:39.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 119686 11.58 MB/sec warmup 57 sec latency 13.738 ms 2024-09-11T22:24:39.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:38 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2471375095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:24:39.328 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:38 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2471375095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:24:39.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:24:39.668 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.943+0000 7f8adee77640 1 -- 172.21.15.5:0/4288802491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ad81069c0 msgr2=0x7f8ad8106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.943+0000 7f8adee77640 1 --2- 172.21.15.5:0/4288802491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ad81069c0 0x7f8ad8106da0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009930 tx=0x7f8acc02a420 comp rx=0 tx=0).stop 2024-09-11T22:24:39.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.943+0000 7f8adee77640 1 -- 172.21.15.5:0/4288802491 shutdown_connections 2024-09-11T22:24:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.943+0000 7f8adee77640 1 --2- 172.21.15.5:0/4288802491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ad8101350 0x7f8ad810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.943+0000 7f8adee77640 1 --2- 172.21.15.5:0/4288802491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad81009b0 0x7f8ad8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.943+0000 7f8adee77640 1 --2- 172.21.15.5:0/4288802491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ad81069c0 0x7f8ad8106da0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.943+0000 7f8adee77640 1 -- 172.21.15.5:0/4288802491 >> 172.21.15.5:0/4288802491 conn(0x7f8ad8077990 msgr2=0x7f8ad80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.943+0000 7f8adee77640 1 -- 172.21.15.5:0/4288802491 shutdown_connections 2024-09-11T22:24:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.943+0000 7f8adee77640 1 -- 172.21.15.5:0/4288802491 wait complete. 2024-09-11T22:24:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adee77640 1 Processor -- start 2024-09-11T22:24:39.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adee77640 1 -- start start 2024-09-11T22:24:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adee77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ad81009b0 0x7f8ad81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adee77640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad8101350 0x7f8ad8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adee77640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ad81069c0 0x7f8ad819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adee77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8110e00 con 0x7f8ad81009b0 2024-09-11T22:24:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adee77640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8110c80 con 0x7f8ad81069c0 2024-09-11T22:24:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adee77640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8110f80 con 0x7f8ad8101350 2024-09-11T22:24:39.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adcc13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ad81009b0 0x7f8ad81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adcc13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ad81009b0 0x7f8ad81983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58700/0 (socket says 172.21.15.5:58700) 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adcc13640 1 -- 172.21.15.5:0/2095197215 learned_addr learned my addr 172.21.15.5:0/2095197215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8add414640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ad81069c0 0x7f8ad819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad8101350 0x7f8ad8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adcc13640 1 -- 172.21.15.5:0/2095197215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad8101350 msgr2=0x7f8ad8198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adcc13640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad8101350 0x7f8ad8198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adcc13640 1 -- 172.21.15.5:0/2095197215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ad81069c0 msgr2=0x7f8ad819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adcc13640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ad81069c0 0x7f8ad819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.944+0000 7f8adcc13640 1 -- 172.21.15.5:0/2095197215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad819d3b0 con 0x7f8ad81009b0 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.945+0000 7f8adcc13640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ad81009b0 0x7f8ad81983c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009230 tx=0x7f8acc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:39.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.945+0000 7f8ad5ffb640 1 -- 172.21.15.5:0/2095197215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc03f030 con 0x7f8ad81009b0 2024-09-11T22:24:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.945+0000 7f8adee77640 1 -- 172.21.15.5:0/2095197215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad819d640 con 0x7f8ad81009b0 2024-09-11T22:24:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.945+0000 7f8ad5ffb640 1 -- 172.21.15.5:0/2095197215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8acc004440 con 0x7f8ad81009b0 2024-09-11T22:24:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.945+0000 7f8ad5ffb640 1 -- 172.21.15.5:0/2095197215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc035070 con 0x7f8ad81009b0 2024-09-11T22:24:39.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.945+0000 7f8adee77640 1 -- 172.21.15.5:0/2095197215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad81a4f30 con 0x7f8ad81009b0 2024-09-11T22:24:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.948+0000 7f8ad5ffb640 1 -- 172.21.15.5:0/2095197215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8acc042030 con 0x7f8ad81009b0 2024-09-11T22:24:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.948+0000 7f8ae1c11740 1 Processor -- start 2024-09-11T22:24:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.948+0000 7f8ae1c11740 1 -- start start 2024-09-11T22:24:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.948+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408629d0 0x558540862db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.948+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540871b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.948+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 0x5585408744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.948+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585407c72f0 con 0x5585408632f0 2024-09-11T22:24:39.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.948+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55854076f270 con 0x5585408629d0 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.948+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5585407c32b0 con 0x5585408720c0 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.948+0000 7f8adcc13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540871b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8adcc13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540871b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58712/0 (socket says 172.21.15.5:58712) 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8adcc13640 1 -- 172.21.15.5:0/3078150992 learned_addr learned my addr 172.21.15.5:0/3078150992 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8add414640 1 --2- 172.21.15.5:0/3078150992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408629d0 0x558540862db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/3078150992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 0x5585408744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8ad5ffb640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8ac80a2e20 0x7f8ac80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8ad7fff640 1 -- 172.21.15.5:0/3078150992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408629d0 msgr2=0x558540862db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/3078150992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408629d0 0x558540862db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8ad7fff640 1 -- 172.21.15.5:0/3078150992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 msgr2=0x558540871b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/3078150992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540871b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8ad7fff640 1 -- 172.21.15.5:0/3078150992 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55854076e340 con 0x5585408720c0 2024-09-11T22:24:39.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8add414640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8ac80a2e20 0x7f8ac80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8ad5ffb640 1 -- 172.21.15.5:0/2095197215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f8acc0ef470 con 0x7f8ad81009b0 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.949+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/3078150992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 0x5585408744b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009b60 tx=0x7f8ac4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e49692ab20b8347 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.950+0000 7f8add414640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8ac80a2e20 0x7f8ac80a52e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0005c40 tx=0x7f8ad0004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.950+0000 7f8ac2ffd640 1 -- 172.21.15.5:0/3078150992 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4009e50 con 0x5585408720c0 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.950+0000 7f8ac2ffd640 1 -- 172.21.15.5:0/3078150992 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac40035f0 con 0x5585408720c0 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.950+0000 7f8ac2ffd640 1 -- 172.21.15.5:0/3078150992 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40126b0 con 0x5585408720c0 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.950+0000 7f8ae1c11740 1 -- 172.21.15.5:0/3078150992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 msgr2=0x5585408744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.950+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/3078150992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 0x5585408744b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009b60 tx=0x7f8ac4010740 comp rx=0 tx=0).stop 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.951+0000 7f8ae1c11740 1 -- 172.21.15.5:0/3078150992 shutdown_connections 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.951+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/3078150992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 0x5585408744b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.951+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/3078150992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540871b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.951+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/3078150992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408629d0 0x558540862db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.951+0000 7f8ae1c11740 1 -- 172.21.15.5:0/3078150992 >> 172.21.15.5:0/3078150992 conn(0x55854085f060 msgr2=0x55854085f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.951+0000 7f8ae1c11740 1 -- 172.21.15.5:0/3078150992 shutdown_connections 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.951+0000 7f8ae1c11740 1 -- 172.21.15.5:0/3078150992 wait complete. 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.951+0000 7f8ae1c11740 1 -- start start 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.951+0000 7f8ae1c11740 1 Processor -- start 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408629d0 0x55854086f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x55854086f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 0x55854086faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558540874cf0 con 0x5585408632f0 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558540750290 con 0x5585408720c0 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558540874e70 con 0x5585408629d0 2024-09-11T22:24:39.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ae1c11740 10 client.0 _ll_register_callbacks cb 0x558540781140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x55854086f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8adcc13640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408629d0 0x55854086f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x55854086f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58726/0 (socket says 172.21.15.5:58726) 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8add414640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 0x55854086faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ad7fff640 1 -- 172.21.15.5:0/2466953574 learned_addr learned my addr 172.21.15.5:0/2466953574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ad7fff640 1 -- 172.21.15.5:0/2466953574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408629d0 msgr2=0x55854086f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408629d0 0x55854086f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ad7fff640 1 -- 172.21.15.5:0/2466953574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 msgr2=0x55854086faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 0x55854086faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ad7fff640 1 -- 172.21.15.5:0/2466953574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585404da210 con 0x5585408632f0 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8add414640 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 0x55854086faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8adcc13640 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408629d0 0x55854086f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:39.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x55854086f560 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40045e0 tx=0x7f8ac40056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec46e6ba1493a443 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ab7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.952+0000 7f8ab7fff640 1 -- 172.21.15.5:0/2466953574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac402a060 con 0x5585408632f0 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ae1c11740 1 -- 172.21.15.5:0/2466953574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac40035f0 con 0x5585408632f0 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ab7fff640 1 -- 172.21.15.5:0/2466953574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f8ac4003e40 con 0x5585408632f0 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ab7fff640 1 -- 172.21.15.5:0/2466953574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ac4004040 con 0x5585408632f0 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ae1c11740 10 client.16131 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ae1c11740 1 -- 172.21.15.5:0/2466953574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5585405a6620 con 0x5585408632f0 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ab7fff640 1 -- 172.21.15.5:0/2466953574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac40035f0 con 0x5585408632f0 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ab7fff640 1 -- 172.21.15.5:0/2466953574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f8ac400f040 con 0x5585408632f0 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ae1c11740 10 client.16131 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ae1c11740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ae1c11740 20 client.16131 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558540882010 0x558540884400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ae1c11740 4 client.16131 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8ae1c11740 1 -- 172.21.15.5:0/2466953574 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585407d4a20 con 0x558540882010 2024-09-11T22:24:39.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.953+0000 7f8adcc13640 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558540882010 0x558540884400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.954+0000 7f8ab7fff640 10 client.16131 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.954+0000 7f8adcc13640 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558540882010 0x558540884400 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b25e5a8bdb0159 server_cookie=446624640a26fdaa in_seq=0 out_seq=0 2024-09-11T22:24:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.955+0000 7f8ab7fff640 1 -- 172.21.15.5:0/2466953574 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585407d4a20 con 0x558540882010 2024-09-11T22:24:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.955+0000 7f8ab7fff640 10 client.16131 handle_command_reply: tid=0 2024-09-11T22:24:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 client.16131 shutdown 2024-09-11T22:24:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 -- 172.21.15.5:0/2466953574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 msgr2=0x55854086f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:39.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x55854086f560 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40045e0 tx=0x7f8ac40056c0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 -- 172.21.15.5:0/2466953574 shutdown_connections 2024-09-11T22:24:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558540882010 0x558540884400 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 0x55854086faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ab7fff640 0 client.16131 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x55854086f560 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/2466953574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408629d0 0x55854086f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 -- 172.21.15.5:0/2466953574 >> 172.21.15.5:0/2466953574 conn(0x55854085f060 msgr2=0x55854085e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 -- 172.21.15.5:0/2466953574 shutdown_connections 2024-09-11T22:24:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 1 -- 172.21.15.5:0/2466953574 wait complete. 2024-09-11T22:24:39.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.956+0000 7f8ae1c11740 20 client.16131 trim_cache size 0 max 16384 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.970+0000 7f8ae1c11740 1 Processor -- start 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.970+0000 7f8ae1c11740 1 -- start start 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408629d0 0x558540870f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540889af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 0x55854088a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81ab6e0 con 0x5585408632f0 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81ad2e0 con 0x5585408629d0 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81abdb0 con 0x5585408720c0 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540889af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ad7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540889af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58738/0 (socket says 172.21.15.5:58738) 2024-09-11T22:24:39.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ad7fff640 1 -- 172.21.15.5:0/3411979598 learned_addr learned my addr 172.21.15.5:0/3411979598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8adcc13640 1 --2- 172.21.15.5:0/3411979598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408629d0 0x558540870f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8add414640 1 --2- 172.21.15.5:0/3411979598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 0x55854088a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ad7fff640 1 -- 172.21.15.5:0/3411979598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 msgr2=0x55854088a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/3411979598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 0x55854088a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ad7fff640 1 -- 172.21.15.5:0/3411979598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408629d0 msgr2=0x558540870f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/3411979598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408629d0 0x558540870f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ad7fff640 1 -- 172.21.15.5:0/3411979598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558540871920 con 0x5585408632f0 2024-09-11T22:24:39.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/3411979598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540889af0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4001eb0 tx=0x7f8ac4003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f512e66f84d42e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.971+0000 7f8ac27fc640 1 -- 172.21.15.5:0/3411979598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac402a070 con 0x5585408632f0 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ac27fc640 1 -- 172.21.15.5:0/3411979598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac4003af0 con 0x5585408632f0 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ac27fc640 1 -- 172.21.15.5:0/3411979598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4012930 con 0x5585408632f0 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ae1c11740 1 -- 172.21.15.5:0/3411979598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 msgr2=0x558540889af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/3411979598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540889af0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4001eb0 tx=0x7f8ac4003780 comp rx=0 tx=0).stop 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ae1c11740 1 -- 172.21.15.5:0/3411979598 shutdown_connections 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/3411979598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408720c0 0x55854088a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/3411979598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408632f0 0x558540889af0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/3411979598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408629d0 0x558540870f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ae1c11740 1 -- 172.21.15.5:0/3411979598 >> 172.21.15.5:0/3411979598 conn(0x55854085f060 msgr2=0x558540870880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:39.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ae1c11740 1 -- 172.21.15.5:0/3411979598 shutdown_connections 2024-09-11T22:24:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.972+0000 7f8ae1c11740 1 -- 172.21.15.5:0/3411979598 wait complete. 2024-09-11T22:24:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.973+0000 7f8ae1c11740 1 -- start start 2024-09-11T22:24:39.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.973+0000 7f8ae1c11740 1 Processor -- start 2024-09-11T22:24:39.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.973+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408629d0 0x55854086c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.973+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408632f0 0x55854086c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.973+0000 7f8ae1c11740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 0x55854086caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.973+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81acb50 con 0x5585408629d0 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.973+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81ac480 con 0x5585408720c0 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.973+0000 7f8ae1c11740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81a96c0 con 0x5585408632f0 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8ae1c11740 10 client.0 _ll_register_callbacks cb 0x7f8acc0f2df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8adcc13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408629d0 0x55854086c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8adcc13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408629d0 0x55854086c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58742/0 (socket says 172.21.15.5:58742) 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8adcc13640 1 -- 172.21.15.5:0/4205759344 learned_addr learned my addr 172.21.15.5:0/4205759344 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8add414640 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 0x55854086caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8adcc13640 1 -- 172.21.15.5:0/4205759344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408632f0 msgr2=0x55854086c560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8adcc13640 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408632f0 0x55854086c560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8adcc13640 1 -- 172.21.15.5:0/4205759344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 msgr2=0x55854086caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8adcc13640 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 0x55854086caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8adcc13640 1 -- 172.21.15.5:0/4205759344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585408ebac0 con 0x5585408629d0 2024-09-11T22:24:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8add414640 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 0x55854086caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8adcc13640 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408629d0 0x55854086c020 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8acc03b5a0 tx=0x7f8acc0f18c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d2926943ccc6af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8ab7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8ab7fff640 1 -- 172.21.15.5:0/4205759344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8acc049040 con 0x5585408629d0 2024-09-11T22:24:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8ab7fff640 1 -- 172.21.15.5:0/4205759344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f8acc0b7f60 con 0x5585408629d0 2024-09-11T22:24:39.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8ae1c11740 1 -- 172.21.15.5:0/4205759344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac4003af0 con 0x5585408629d0 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8ab7fff640 1 -- 172.21.15.5:0/4205759344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8acc0bb8f0 con 0x5585408629d0 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8ae1c11740 10 client.16137 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8ae1c11740 1 -- 172.21.15.5:0/4205759344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5585408ecdb0 con 0x5585408629d0 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.974+0000 7f8ab7fff640 1 -- 172.21.15.5:0/4205759344 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ac4003af0 con 0x5585408629d0 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.975+0000 7f8ab7fff640 1 -- 172.21.15.5:0/4205759344 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f8acc04c060 con 0x5585408629d0 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.975+0000 7f8ae1c11740 10 client.16137 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.975+0000 7f8ae1c11740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.975+0000 7f8ae1c11740 20 client.16137 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.975+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5585408ed5a0 0x5585408ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.975+0000 7f8ae1c11740 4 client.16137 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.975+0000 7f8ae1c11740 1 -- 172.21.15.5:0/4205759344 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585407ced70 con 0x5585408ed5a0 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.975+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5585408ed5a0 0x5585408ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:39.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.975+0000 7f8ab7fff640 10 client.16137 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:39.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:39.975+0000 7f8ad7fff640 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5585408ed5a0 0x5585408ef9c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5e7ac8b6246aeba server_cookie=93a22423a55effbd in_seq=0 out_seq=0 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.061+0000 7f8ab7fff640 1 -- 172.21.15.5:0/4205759344 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585407ced70 con 0x5585408ed5a0 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.061+0000 7f8ab7fff640 10 client.16137 handle_command_reply: tid=0 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.061+0000 7f8ae1c11740 1 client.16137 shutdown 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.061+0000 7f8ae1c11740 1 -- 172.21.15.5:0/4205759344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408629d0 msgr2=0x55854086c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.061+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408629d0 0x55854086c020 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8acc03b5a0 tx=0x7f8acc0f18c0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8ae1c11740 1 -- 172.21.15.5:0/4205759344 shutdown_connections 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5585408ed5a0 0x5585408ef9c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585408720c0 0x55854086caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585408632f0 0x55854086c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8ae1c11740 1 --2- 172.21.15.5:0/4205759344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585408629d0 0x55854086c020 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8ab7fff640 0 client.16137 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8ae1c11740 1 -- 172.21.15.5:0/4205759344 >> 172.21.15.5:0/4205759344 conn(0x55854085f060 msgr2=0x558540864d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8ae1c11740 1 -- 172.21.15.5:0/4205759344 shutdown_connections 2024-09-11T22:24:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8ae1c11740 1 -- 172.21.15.5:0/4205759344 wait complete. 2024-09-11T22:24:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8ae1c11740 20 client.16137 trim_cache size 0 max 16384 2024-09-11T22:24:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:24:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:24:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:24:40.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:24:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8adee77640 1 -- 172.21.15.5:0/2095197215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8ac80a2e20 msgr2=0x7f8ac80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8adee77640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8ac80a2e20 0x7f8ac80a52e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0005c40 tx=0x7f8ad0004ab0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8adee77640 1 -- 172.21.15.5:0/2095197215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ad81009b0 msgr2=0x7f8ad81983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8adee77640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ad81009b0 0x7f8ad81983c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009230 tx=0x7f8acc004040 comp rx=0 tx=0).stop 2024-09-11T22:24:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8adee77640 1 -- 172.21.15.5:0/2095197215 shutdown_connections 2024-09-11T22:24:40.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8adee77640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8ac80a2e20 0x7f8ac80a52e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8adee77640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ad81069c0 0x7f8ad819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8adee77640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad8101350 0x7f8ad8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8adee77640 1 --2- 172.21.15.5:0/2095197215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ad81009b0 0x7f8ad81983c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.062+0000 7f8adee77640 1 -- 172.21.15.5:0/2095197215 >> 172.21.15.5:0/2095197215 conn(0x7f8ad8077990 msgr2=0x7f8ad8069b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.063+0000 7f8adee77640 1 -- 172.21.15.5:0/2095197215 shutdown_connections 2024-09-11T22:24:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:40.063+0000 7f8adee77640 1 -- 172.21.15.5:0/2095197215 wait complete. 2024-09-11T22:24:40.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:39 smithi005 ceph-mon[30943]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 400 KiB/s wr, 115 op/s 2024-09-11T22:24:40.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:39 smithi172 ceph-mon[35461]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 400 KiB/s wr, 115 op/s 2024-09-11T22:24:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 -- 172.21.15.5:0/3202883863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694102570 msgr2=0x7f56941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 --2- 172.21.15.5:0/3202883863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694102570 0x7f56941029b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f567c009900 tx=0x7f567c02a480 comp rx=0 tx=0).stop 2024-09-11T22:24:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 -- 172.21.15.5:0/3202883863 shutdown_connections 2024-09-11T22:24:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 --2- 172.21.15.5:0/3202883863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5694102ef0 0x7f569410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 --2- 172.21.15.5:0/3202883863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694102570 0x7f56941029b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 --2- 172.21.15.5:0/3202883863 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5694100e20 0x7f5694101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 -- 172.21.15.5:0/3202883863 >> 172.21.15.5:0/3202883863 conn(0x7f56940fc7d0 msgr2=0x7f56940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 -- 172.21.15.5:0/3202883863 shutdown_connections 2024-09-11T22:24:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 -- 172.21.15.5:0/3202883863 wait complete. 2024-09-11T22:24:40.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 Processor -- start 2024-09-11T22:24:40.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.190+0000 7f5699bcc640 1 -- start start 2024-09-11T22:24:40.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f5699bcc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5694100e20 0x7f569419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f5699bcc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694102570 0x7f569419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f5699bcc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5694102ef0 0x7f569419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f5699bcc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5694110f80 con 0x7f5694102570 2024-09-11T22:24:40.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f5699bcc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5694110e00 con 0x7f5694100e20 2024-09-11T22:24:40.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f5699bcc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5694111100 con 0x7f5694102ef0 2024-09-11T22:24:40.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f568affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694102570 0x7f569419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f56937fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5694100e20 0x7f569419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f568affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694102570 0x7f569419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58752/0 (socket says 172.21.15.5:58752) 2024-09-11T22:24:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f56937fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5694100e20 0x7f569419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50988/0 (socket says 172.21.15.5:50988) 2024-09-11T22:24:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f568affd640 1 -- 172.21.15.5:0/2026642854 learned_addr learned my addr 172.21.15.5:0/2026642854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f5693fff640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5694102ef0 0x7f569419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f568affd640 1 -- 172.21.15.5:0/2026642854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5694102ef0 msgr2=0x7f569419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f568affd640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5694102ef0 0x7f569419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f568affd640 1 -- 172.21.15.5:0/2026642854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5694100e20 msgr2=0x7f569419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f568affd640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5694100e20 0x7f569419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.191+0000 7f568affd640 1 -- 172.21.15.5:0/2026642854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569419f660 con 0x7f5694102570 2024-09-11T22:24:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.192+0000 7f568affd640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694102570 0x7f569419abf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f567c006010 tx=0x7f567c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.192+0000 7f56917fa640 1 -- 172.21.15.5:0/2026642854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c03f070 con 0x7f5694102570 2024-09-11T22:24:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.192+0000 7f56917fa640 1 -- 172.21.15.5:0/2026642854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c031c30 con 0x7f5694102570 2024-09-11T22:24:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.192+0000 7f5699bcc640 1 -- 172.21.15.5:0/2026642854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569419f8f0 con 0x7f5694102570 2024-09-11T22:24:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.192+0000 7f56917fa640 1 -- 172.21.15.5:0/2026642854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c035030 con 0x7f5694102570 2024-09-11T22:24:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.192+0000 7f5699bcc640 1 -- 172.21.15.5:0/2026642854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56941a7230 con 0x7f5694102570 2024-09-11T22:24:40.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f56917fa640 1 -- 172.21.15.5:0/2026642854 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f567c042070 con 0x7f5694102570 2024-09-11T22:24:40.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f569bfa5740 1 Processor -- start 2024-09-11T22:24:40.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f569bfa5740 1 -- start start 2024-09-11T22:24:40.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c10b40 0x565447c0f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c0f5d0 0x565447c0fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c1a370 0x565447c1c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f569bfa5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565447b752f0 con 0x565447c0f5d0 2024-09-11T22:24:40.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f569bfa5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565447b1ccf0 con 0x565447c10b40 2024-09-11T22:24:40.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f569bfa5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565447b712b0 con 0x565447c1a370 2024-09-11T22:24:40.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f56937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c0f5d0 0x565447c0fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f5693fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c10b40 0x565447c0f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f56937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c0f5d0 0x565447c0fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58760/0 (socket says 172.21.15.5:58760) 2024-09-11T22:24:40.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f5693fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c10b40 0x565447c0f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51002/0 (socket says 172.21.15.5:51002) 2024-09-11T22:24:40.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f56937fe640 1 -- 172.21.15.5:0/2892638817 learned_addr learned my addr 172.21.15.5:0/2892638817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:40.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.195+0000 7f568affd640 1 --2- 172.21.15.5:0/2892638817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c1a370 0x565447c1c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f56917fa640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56800a2e40 0x7f56800a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f5693fff640 1 -- 172.21.15.5:0/2892638817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c1a370 msgr2=0x565447c1c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:40.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f5693fff640 1 --2- 172.21.15.5:0/2892638817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c1a370 0x565447c1c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f5693fff640 1 -- 172.21.15.5:0/2892638817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c0f5d0 msgr2=0x565447c0fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:40.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f5693fff640 1 --2- 172.21.15.5:0/2892638817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c0f5d0 0x565447c0fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f5693fff640 1 -- 172.21.15.5:0/2892638817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565447b013a0 con 0x565447c10b40 2024-09-11T22:24:40.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f56917fa640 1 -- 172.21.15.5:0/2026642854 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f567c0ef670 con 0x7f5694102570 2024-09-11T22:24:40.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f5693fff640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56800a2e40 0x7f56800a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f5693fff640 1 --2- 172.21.15.5:0/2892638817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c10b40 0x565447c0f090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f568c009fd0 tx=0x7f568c011920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f1f0df3035ce8c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:40.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f5689ffb640 1 -- 172.21.15.5:0/2892638817 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c017070 con 0x565447c10b40 2024-09-11T22:24:40.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f5689ffb640 1 -- 172.21.15.5:0/2892638817 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568c00fc20 con 0x565447c10b40 2024-09-11T22:24:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.196+0000 7f5689ffb640 1 -- 172.21.15.5:0/2892638817 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c01ba20 con 0x565447c10b40 2024-09-11T22:24:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.197+0000 7f569bfa5740 1 -- 172.21.15.5:0/2892638817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c10b40 msgr2=0x565447c0f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.197+0000 7f5693fff640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56800a2e40 0x7f56800a5300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f568c022000 tx=0x7f568c023000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.197+0000 7f569bfa5740 1 --2- 172.21.15.5:0/2892638817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c10b40 0x565447c0f090 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f568c009fd0 tx=0x7f568c011920 comp rx=0 tx=0).stop 2024-09-11T22:24:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.197+0000 7f569bfa5740 1 -- 172.21.15.5:0/2892638817 shutdown_connections 2024-09-11T22:24:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.197+0000 7f569bfa5740 1 --2- 172.21.15.5:0/2892638817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c1a370 0x565447c1c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.197+0000 7f569bfa5740 1 --2- 172.21.15.5:0/2892638817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c0f5d0 0x565447c0fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.197+0000 7f569bfa5740 1 --2- 172.21.15.5:0/2892638817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c10b40 0x565447c0f090 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.197+0000 7f569bfa5740 1 -- 172.21.15.5:0/2892638817 >> 172.21.15.5:0/2892638817 conn(0x565447bd8310 msgr2=0x565447c0bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.197+0000 7f569bfa5740 1 -- 172.21.15.5:0/2892638817 shutdown_connections 2024-09-11T22:24:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.197+0000 7f569bfa5740 1 -- 172.21.15.5:0/2892638817 wait complete. 2024-09-11T22:24:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f569bfa5740 1 -- start start 2024-09-11T22:24:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f569bfa5740 1 Processor -- start 2024-09-11T22:24:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 0x565447c19d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 0x565447c16ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 0x565447c175e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f569bfa5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565447c1d070 con 0x565447c1a370 2024-09-11T22:24:40.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f569bfa5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565447b30060 con 0x565447c0f5d0 2024-09-11T22:24:40.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f569bfa5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565447c1d1f0 con 0x565447c10b40 2024-09-11T22:24:40.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f569bfa5740 10 client.0 _ll_register_callbacks cb 0x5654478838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:40.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 0x565447c175e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f568affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 0x565447c16ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f56937fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 0x565447c19d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f568affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 0x565447c16ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36002/0 (socket says 172.21.15.5:36002) 2024-09-11T22:24:40.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.198+0000 7f568affd640 1 -- 172.21.15.5:0/1577692355 learned_addr learned my addr 172.21.15.5:0/1577692355 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f56937fe640 1 -- 172.21.15.5:0/1577692355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 msgr2=0x565447c16ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f56937fe640 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 0x565447c16ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f56937fe640 1 -- 172.21.15.5:0/1577692355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 msgr2=0x565447c175e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f56937fe640 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 0x565447c175e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f56937fe640 1 -- 172.21.15.5:0/1577692355 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654478e63c0 con 0x565447c0f5d0 2024-09-11T22:24:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f56937fe640 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 0x565447c19d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56840126d0 tx=0x7f5684012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc40684b60009d35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f5672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:24:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f5672ffd640 1 -- 172.21.15.5:0/1577692355 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684004050 con 0x565447c0f5d0 2024-09-11T22:24:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f5672ffd640 1 -- 172.21.15.5:0/1577692355 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f56840041f0 con 0x565447c0f5d0 2024-09-11T22:24:40.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f569bfa5740 1 -- 172.21.15.5:0/1577692355 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f568c00fc20 con 0x565447c0f5d0 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f5672ffd640 1 -- 172.21.15.5:0/1577692355 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56840043f0 con 0x565447c0f5d0 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f569bfa5740 10 client.25673 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.199+0000 7f569bfa5740 1 -- 172.21.15.5:0/1577692355 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565447c28b40 con 0x565447c0f5d0 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.200+0000 7f5672ffd640 1 -- 172.21.15.5:0/1577692355 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568c00fc20 con 0x565447c0f5d0 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.200+0000 7f5672ffd640 1 -- 172.21.15.5:0/1577692355 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5684003c60 con 0x565447c0f5d0 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.200+0000 7f569bfa5740 10 client.25673 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.200+0000 7f569bfa5740 10 client.25673 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.200+0000 7f569bfa5740 20 client.25673 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.200+0000 7f569bfa5740 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x565447c2a000 0x565447c2c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.200+0000 7f569bfa5740 4 client.25673 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.200+0000 7f569bfa5740 1 -- 172.21.15.5:0/1577692355 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565447b82860 con 0x565447c2a000 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.200+0000 7f568affd640 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x565447c2a000 0x565447c2c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.201+0000 7f5672ffd640 10 client.25673 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.201+0000 7f568affd640 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x565447c2a000 0x565447c2c3f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2677a982da5571fa server_cookie=53dd5cd984817ffe in_seq=0 out_seq=0 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.201+0000 7f5672ffd640 1 -- 172.21.15.5:0/1577692355 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565447b82860 con 0x565447c2a000 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.201+0000 7f5672ffd640 10 client.25673 handle_command_reply: tid=0 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.201+0000 7f569bfa5740 1 client.25673 shutdown 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.201+0000 7f569bfa5740 1 -- 172.21.15.5:0/1577692355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 msgr2=0x565447c19d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.201+0000 7f569bfa5740 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 0x565447c19d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56840126d0 tx=0x7f5684012b90 comp rx=0 tx=0).stop 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.202+0000 7f569bfa5740 1 -- 172.21.15.5:0/1577692355 shutdown_connections 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.202+0000 7f569bfa5740 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x565447c2a000 0x565447c2c3f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.202+0000 7f569bfa5740 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 0x565447c175e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.202+0000 7f569bfa5740 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 0x565447c16ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.202+0000 7f5672ffd640 0 client.25673 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.202+0000 7f569bfa5740 1 --2- 172.21.15.5:0/1577692355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 0x565447c19d40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.202+0000 7f569bfa5740 1 -- 172.21.15.5:0/1577692355 >> 172.21.15.5:0/1577692355 conn(0x565447bd8310 msgr2=0x565447c11960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:40.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.202+0000 7f569bfa5740 1 -- 172.21.15.5:0/1577692355 shutdown_connections 2024-09-11T22:24:40.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.202+0000 7f569bfa5740 1 -- 172.21.15.5:0/1577692355 wait complete. 2024-09-11T22:24:40.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.202+0000 7f569bfa5740 20 client.25673 trim_cache size 0 max 16384 2024-09-11T22:24:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.215+0000 7f569bfa5740 1 Processor -- start 2024-09-11T22:24:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.215+0000 7f569bfa5740 1 -- start start 2024-09-11T22:24:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c0f5d0 0x565447c26310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c10b40 0x565447c0abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c1a370 0x565447c31c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f569bfa5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56941afb00 con 0x565447c10b40 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f569bfa5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56941b1700 con 0x565447c1a370 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f569bfa5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56941b01d0 con 0x565447c0f5d0 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f568affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c10b40 0x565447c0abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f568affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c10b40 0x565447c0abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58772/0 (socket says 172.21.15.5:58772) 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f568affd640 1 -- 172.21.15.5:0/3534631879 learned_addr learned my addr 172.21.15.5:0/3534631879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f56937fe640 1 --2- 172.21.15.5:0/3534631879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c0f5d0 0x565447c26310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f5693fff640 1 --2- 172.21.15.5:0/3534631879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c1a370 0x565447c31c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f568affd640 1 -- 172.21.15.5:0/3534631879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c0f5d0 msgr2=0x565447c26310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f568affd640 1 --2- 172.21.15.5:0/3534631879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c0f5d0 0x565447c26310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f568affd640 1 -- 172.21.15.5:0/3534631879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c1a370 msgr2=0x565447c31c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f568affd640 1 --2- 172.21.15.5:0/3534631879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c1a370 0x565447c31c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f568affd640 1 -- 172.21.15.5:0/3534631879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565447c1a0b0 con 0x565447c10b40 2024-09-11T22:24:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f568affd640 1 --2- 172.21.15.5:0/3534631879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c10b40 0x565447c0abf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f567c0b85d0 tx=0x7f567c0b7fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1757a62c195ee650 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f56897fa640 1 -- 172.21.15.5:0/3534631879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c049040 con 0x565447c10b40 2024-09-11T22:24:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f56897fa640 1 -- 172.21.15.5:0/3534631879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c031de0 con 0x565447c10b40 2024-09-11T22:24:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.216+0000 7f56897fa640 1 -- 172.21.15.5:0/3534631879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0ed000 con 0x565447c10b40 2024-09-11T22:24:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.217+0000 7f569bfa5740 1 -- 172.21.15.5:0/3534631879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c10b40 msgr2=0x565447c0abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.217+0000 7f569bfa5740 1 --2- 172.21.15.5:0/3534631879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c10b40 0x565447c0abf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f567c0b85d0 tx=0x7f567c0b7fd0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.217+0000 7f569bfa5740 1 -- 172.21.15.5:0/3534631879 shutdown_connections 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.217+0000 7f569bfa5740 1 --2- 172.21.15.5:0/3534631879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c1a370 0x565447c31c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.217+0000 7f569bfa5740 1 --2- 172.21.15.5:0/3534631879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c10b40 0x565447c0abf0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.217+0000 7f569bfa5740 1 --2- 172.21.15.5:0/3534631879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c0f5d0 0x565447c26310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.217+0000 7f569bfa5740 1 -- 172.21.15.5:0/3534631879 >> 172.21.15.5:0/3534631879 conn(0x565447bd8310 msgr2=0x565447c0a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.217+0000 7f569bfa5740 1 -- 172.21.15.5:0/3534631879 shutdown_connections 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.217+0000 7f569bfa5740 1 -- 172.21.15.5:0/3534631879 wait complete. 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.217+0000 7f569bfa5740 1 -- start start 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f569bfa5740 1 Processor -- start 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 0x565447c1eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 0x565447c1eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f569bfa5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 0x565447c8dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f569bfa5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56941b0f70 con 0x565447c1a370 2024-09-11T22:24:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f569bfa5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56941b08a0 con 0x565447c0f5d0 2024-09-11T22:24:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f569bfa5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56941adae0 con 0x565447c10b40 2024-09-11T22:24:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f569bfa5740 10 client.0 _ll_register_callbacks cb 0x7f5684009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 0x565447c8dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f5693fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 0x565447c8dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58786/0 (socket says 172.21.15.5:58786) 2024-09-11T22:24:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f5693fff640 1 -- 172.21.15.5:0/2708878957 learned_addr learned my addr 172.21.15.5:0/2708878957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f56937fe640 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 0x565447c1eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.218+0000 7f568affd640 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 0x565447c1eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f5693fff640 1 -- 172.21.15.5:0/2708878957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 msgr2=0x565447c1eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f5693fff640 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 0x565447c1eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f5693fff640 1 -- 172.21.15.5:0/2708878957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 msgr2=0x565447c1eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f5693fff640 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 0x565447c1eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f5693fff640 1 -- 172.21.15.5:0/2708878957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565447c8e2c0 con 0x565447c1a370 2024-09-11T22:24:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f568affd640 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 0x565447c1eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f5693fff640 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 0x565447c8dc80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f568c011e20 tx=0x7f568c01b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4baacf4547718d28 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:40.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f5672ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f5672ffd640 1 -- 172.21.15.5:0/2708878957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c037060 con 0x565447c1a370 2024-09-11T22:24:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f5672ffd640 1 -- 172.21.15.5:0/2708878957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f568c00f540 con 0x565447c1a370 2024-09-11T22:24:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f569bfa5740 1 -- 172.21.15.5:0/2708878957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f567c031de0 con 0x565447c1a370 2024-09-11T22:24:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f5672ffd640 1 -- 172.21.15.5:0/2708878957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f568c017070 con 0x565447c1a370 2024-09-11T22:24:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f569bfa5740 10 client.16155 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.219+0000 7f569bfa5740 1 -- 172.21.15.5:0/2708878957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565447c95610 con 0x565447c1a370 2024-09-11T22:24:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f5672ffd640 1 -- 172.21.15.5:0/2708878957 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c031de0 con 0x565447c1a370 2024-09-11T22:24:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f5672ffd640 1 -- 172.21.15.5:0/2708878957 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f568c032440 con 0x565447c1a370 2024-09-11T22:24:40.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f569bfa5740 10 client.16155 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:40.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f569bfa5740 10 client.16155 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:24:40.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f569bfa5740 20 client.16155 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:40.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f569bfa5740 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x565447c95b20 0x565447c97f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:40.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f569bfa5740 4 client.16155 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:24:40.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f569bfa5740 1 -- 172.21.15.5:0/2708878957 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565447b7cd70 con 0x565447c95b20 2024-09-11T22:24:40.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f56937fe640 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x565447c95b20 0x565447c97f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f56937fe640 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x565447c95b20 0x565447c97f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46ef60deffc56cd2 server_cookie=d47bbc3e3d1bd003 in_seq=0 out_seq=0 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.220+0000 7f5672ffd640 10 client.16155 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f5672ffd640 1 -- 172.21.15.5:0/2708878957 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565447b7cd70 con 0x565447c95b20 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f5672ffd640 10 client.16155 handle_command_reply: tid=0 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 client.16155 shutdown 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 -- 172.21.15.5:0/2708878957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 msgr2=0x565447c8dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 0x565447c8dc80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f568c011e20 tx=0x7f568c01b5a0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 -- 172.21.15.5:0/2708878957 shutdown_connections 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x565447c95b20 0x565447c97f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565447c1a370 0x565447c8dc80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565447c10b40 0x565447c1eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 --2- 172.21.15.5:0/2708878957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565447c0f5d0 0x565447c1eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 -- 172.21.15.5:0/2708878957 >> 172.21.15.5:0/2708878957 conn(0x565447bd8310 msgr2=0x565447c0ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f5672ffd640 0 client.16155 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:24:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 -- 172.21.15.5:0/2708878957 shutdown_connections 2024-09-11T22:24:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 1 -- 172.21.15.5:0/2708878957 wait complete. 2024-09-11T22:24:40.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.221+0000 7f569bfa5740 20 client.16155 trim_cache size 0 max 16384 2024-09-11T22:24:40.231 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:24:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 -- 172.21.15.5:0/2026642854 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56800a2e40 msgr2=0x7f56800a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56800a2e40 0x7f56800a5300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f568c022000 tx=0x7f568c023000 comp rx=0 tx=0).stop 2024-09-11T22:24:40.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 -- 172.21.15.5:0/2026642854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694102570 msgr2=0x7f569419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694102570 0x7f569419abf0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f567c006010 tx=0x7f567c003c10 comp rx=0 tx=0).stop 2024-09-11T22:24:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 -- 172.21.15.5:0/2026642854 shutdown_connections 2024-09-11T22:24:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5694102ef0 0x7f569419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5694102570 0x7f569419abf0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56800a2e40 0x7f56800a5300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 --2- 172.21.15.5:0/2026642854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5694100e20 0x7f569419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 -- 172.21.15.5:0/2026642854 >> 172.21.15.5:0/2026642854 conn(0x7f56940fc7d0 msgr2=0x7f569406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 -- 172.21.15.5:0/2026642854 shutdown_connections 2024-09-11T22:24:40.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:40.222+0000 7f5699bcc640 1 -- 172.21.15.5:0/2026642854 wait complete. 2024-09-11T22:24:40.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 121813 11.61 MB/sec warmup 58 sec latency 60.240 ms 2024-09-11T22:24:40.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:39 smithi159 ceph-mon[35534]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 400 KiB/s wr, 115 op/s 2024-09-11T22:24:41.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:24:41.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 124736 11.70 MB/sec warmup 59 sec latency 13.048 ms 2024-09-11T22:24:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.945+0000 7f0291f4b640 1 -- 172.21.15.5:0/1940528833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028c103740 msgr2=0x7f028c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.945+0000 7f0291f4b640 1 --2- 172.21.15.5:0/1940528833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028c103740 0x7f028c107b80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0274009930 tx=0x7f027402a420 comp rx=0 tx=0).stop 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.945+0000 7f0291f4b640 1 -- 172.21.15.5:0/1940528833 shutdown_connections 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.945+0000 7f0291f4b640 1 --2- 172.21.15.5:0/1940528833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f028c108290 0x7f028c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.945+0000 7f0291f4b640 1 --2- 172.21.15.5:0/1940528833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028c103740 0x7f028c107b80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.945+0000 7f0291f4b640 1 --2- 172.21.15.5:0/1940528833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c102d90 0x7f028c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.945+0000 7f0291f4b640 1 -- 172.21.15.5:0/1940528833 >> 172.21.15.5:0/1940528833 conn(0x7f028c0fc7d0 msgr2=0x7f028c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.945+0000 7f0291f4b640 1 -- 172.21.15.5:0/1940528833 shutdown_connections 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.945+0000 7f0291f4b640 1 -- 172.21.15.5:0/1940528833 wait complete. 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f0291f4b640 1 Processor -- start 2024-09-11T22:24:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f0291f4b640 1 -- start start 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f0291f4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028c102d90 0x7f028c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f0291f4b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f028c103740 0x7f028c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f0291f4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c108290 0x7f028c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f0291f4b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f028c112090 con 0x7f028c102d90 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f0291f4b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f028c111f10 con 0x7f028c103740 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f0291f4b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f028c112210 con 0x7f028c108290 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028c102d90 0x7f028c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028c102d90 0x7f028c198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33716/0 (socket says 172.21.15.5:33716) 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f028b7fe640 1 -- 172.21.15.5:0/3289222555 learned_addr learned my addr 172.21.15.5:0/3289222555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.946+0000 7f0283fff640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f028c103740 0x7f028c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f028bfff640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c108290 0x7f028c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f028b7fe640 1 -- 172.21.15.5:0/3289222555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c108290 msgr2=0x7f028c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f028b7fe640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c108290 0x7f028c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f028b7fe640 1 -- 172.21.15.5:0/3289222555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f028c103740 msgr2=0x7f028c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f028b7fe640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f028c103740 0x7f028c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f028b7fe640 1 -- 172.21.15.5:0/3289222555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c19d5c0 con 0x7f028c102d90 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f028b7fe640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028c102d90 0x7f028c198610 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9f0 tx=0x7f027c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f02897fa640 1 -- 172.21.15.5:0/3289222555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c00cd80 con 0x7f028c102d90 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f02897fa640 1 -- 172.21.15.5:0/3289222555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f027c005500 con 0x7f028c102d90 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f0291f4b640 1 -- 172.21.15.5:0/3289222555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c19d8b0 con 0x7f028c102d90 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f02897fa640 1 -- 172.21.15.5:0/3289222555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c010690 con 0x7f028c102d90 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.947+0000 7f0291f4b640 1 -- 172.21.15.5:0/3289222555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c1a5190 con 0x7f028c102d90 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f02897fa640 1 -- 172.21.15.5:0/3289222555 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f027c0046c0 con 0x7f028c102d90 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0294324740 1 Processor -- start 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0294324740 1 -- start start 2024-09-11T22:24:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0294324740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c56090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0294324740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c56a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0294324740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c63760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0294324740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f92bbc2f0 con 0x557f92c565d0 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0294324740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557f92b63cf0 con 0x557f92c57b40 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0294324740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557f92bb82b0 con 0x557f92c61370 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c56a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0283fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c63760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f028bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c56090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0283fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c63760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:34066/0 (socket says 172.21.15.5:34066) 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f0283fff640 1 -- 172.21.15.5:0/2078706937 learned_addr learned my addr 172.21.15.5:0/2078706937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.950+0000 7f02897fa640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f02780a2d70 0x7f02780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f0283fff640 1 -- 172.21.15.5:0/2078706937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 msgr2=0x557f92c56090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f0283fff640 1 --2- 172.21.15.5:0/2078706937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c56090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f028bfff640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f02780a2d70 0x7f02780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f0283fff640 1 -- 172.21.15.5:0/2078706937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 msgr2=0x557f92c56a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f0283fff640 1 --2- 172.21.15.5:0/2078706937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c56a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f0283fff640 1 -- 172.21.15.5:0/2078706937 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f92b483a0 con 0x557f92c61370 2024-09-11T22:24:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f02897fa640 1 -- 172.21.15.5:0/3289222555 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f027c0cf8c0 con 0x7f028c102d90 2024-09-11T22:24:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f0283fff640 1 --2- 172.21.15.5:0/2078706937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c63760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0274009900 tx=0x7f0274005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f243a73e2626dc8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f028bfff640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f02780a2d70 0x7f02780a5230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0284005fd0 tx=0x7f0284004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f0281ffb640 1 -- 172.21.15.5:0/2078706937 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274040070 con 0x557f92c61370 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f0281ffb640 1 -- 172.21.15.5:0/2078706937 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02740318f0 con 0x557f92c61370 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.951+0000 7f0281ffb640 1 -- 172.21.15.5:0/2078706937 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274035040 con 0x557f92c61370 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.952+0000 7f0294324740 1 -- 172.21.15.5:0/2078706937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 msgr2=0x557f92c63760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.952+0000 7f0294324740 1 --2- 172.21.15.5:0/2078706937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c63760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0274009900 tx=0x7f0274005c00 comp rx=0 tx=0).stop 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.952+0000 7f0294324740 1 -- 172.21.15.5:0/2078706937 shutdown_connections 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.952+0000 7f0294324740 1 --2- 172.21.15.5:0/2078706937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c63760 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.952+0000 7f0294324740 1 --2- 172.21.15.5:0/2078706937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c56a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.952+0000 7f0294324740 1 --2- 172.21.15.5:0/2078706937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c56090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.952+0000 7f0294324740 1 -- 172.21.15.5:0/2078706937 >> 172.21.15.5:0/2078706937 conn(0x557f92c1f310 msgr2=0x557f92c52c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.952+0000 7f0294324740 1 -- 172.21.15.5:0/2078706937 shutdown_connections 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.952+0000 7f0294324740 1 -- 172.21.15.5:0/2078706937 wait complete. 2024-09-11T22:24:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.952+0000 7f0294324740 1 -- start start 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f0294324740 1 Processor -- start 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f0294324740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c60c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f0294324740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c5df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f0294324740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c5e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f0294324740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f92c64010 con 0x557f92c565d0 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f0294324740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557f92b77060 con 0x557f92c57b40 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f0294324740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557f92c64190 con 0x557f92c61370 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f0294324740 10 client.0 _ll_register_callbacks cb 0x557f928ca8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c60c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c60c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33736/0 (socket says 172.21.15.5:33736) 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f028b7fe640 1 -- 172.21.15.5:0/745829024 learned_addr learned my addr 172.21.15.5:0/745829024 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f0283fff640 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c5df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.953+0000 7f028bfff640 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c5e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f028b7fe640 1 -- 172.21.15.5:0/745829024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 msgr2=0x557f92c5e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f028b7fe640 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c5e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f028b7fe640 1 -- 172.21.15.5:0/745829024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 msgr2=0x557f92c5df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f028b7fe640 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c5df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f028b7fe640 1 -- 172.21.15.5:0/745829024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f9292d3c0 con 0x557f92c565d0 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f0283fff640 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c5df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f028bfff640 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c5e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f028b7fe640 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c60c90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f027c098200 tx=0x7f027c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5cdb65931c8f7fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f026affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f026affd640 1 -- 172.21.15.5:0/745829024 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c09b700 con 0x557f92c565d0 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f026affd640 1 -- 172.21.15.5:0/745829024 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f027c097af0 con 0x557f92c565d0 2024-09-11T22:24:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f0294324740 1 -- 172.21.15.5:0/745829024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02740318f0 con 0x557f92c565d0 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f026affd640 1 -- 172.21.15.5:0/745829024 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f027c09c4a0 con 0x557f92c565d0 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f0294324740 10 client.16167 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f0294324740 1 -- 172.21.15.5:0/745829024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f92c6fa30 con 0x557f92c565d0 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.954+0000 7f026affd640 1 -- 172.21.15.5:0/745829024 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02740318f0 con 0x557f92c565d0 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.955+0000 7f026affd640 1 -- 172.21.15.5:0/745829024 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f027c0a4580 con 0x557f92c565d0 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.955+0000 7f0294324740 10 client.16167 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.955+0000 7f0294324740 10 client.16167 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.955+0000 7f0294324740 20 client.16167 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.955+0000 7f0294324740 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557f92c70f40 0x557f92c73390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.955+0000 7f0294324740 4 client.16167 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.955+0000 7f0294324740 1 -- 172.21.15.5:0/745829024 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f92bc9860 con 0x557f92c70f40 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.955+0000 7f0283fff640 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557f92c70f40 0x557f92c73390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.955+0000 7f0283fff640 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557f92c70f40 0x557f92c73390 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63eaed91997bcf4b server_cookie=5210c5d14e14bdd6 in_seq=0 out_seq=0 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.955+0000 7f026affd640 10 client.16167 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f026affd640 1 -- 172.21.15.5:0/745829024 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f92bc9860 con 0x557f92c70f40 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f026affd640 10 client.16167 handle_command_reply: tid=0 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 client.16167 shutdown 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 -- 172.21.15.5:0/745829024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 msgr2=0x557f92c60c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c60c90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f027c098200 tx=0x7f027c005730 comp rx=0 tx=0).stop 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 -- 172.21.15.5:0/745829024 shutdown_connections 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c5e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557f92c70f40 0x557f92c73390 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c5df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 --2- 172.21.15.5:0/745829024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c60c90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 -- 172.21.15.5:0/745829024 >> 172.21.15.5:0/745829024 conn(0x557f92c1f310 msgr2=0x557f92c58960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f026affd640 0 client.16167 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:24:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 -- 172.21.15.5:0/745829024 shutdown_connections 2024-09-11T22:24:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 1 -- 172.21.15.5:0/745829024 wait complete. 2024-09-11T22:24:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.956+0000 7f0294324740 20 client.16167 trim_cache size 0 max 16384 2024-09-11T22:24:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.971+0000 7f0294324740 1 Processor -- start 2024-09-11T22:24:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.971+0000 7f0294324740 1 -- start start 2024-09-11T22:24:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.971+0000 7f0294324740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c565d0 0x557f92c6d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.971+0000 7f0294324740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c72f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.971+0000 7f0294324740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c61370 0x557f92c62760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f0294324740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1ada70 con 0x557f92c61370 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f0294324740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1af670 con 0x557f92c57b40 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f0294324740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1ae140 con 0x557f92c565d0 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c565d0 0x557f92c6d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f0283fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c72f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f028bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c61370 0x557f92c62760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f0283fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c72f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48406/0 (socket says 172.21.15.5:48406) 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f0283fff640 1 -- 172.21.15.5:0/2633506123 learned_addr learned my addr 172.21.15.5:0/2633506123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c565d0 0x557f92c6d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:34094/0 (socket says 172.21.15.5:34094) 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f028b7fe640 1 -- 172.21.15.5:0/2633506123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 msgr2=0x557f92c72f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f028b7fe640 1 --2- 172.21.15.5:0/2633506123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c72f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f028b7fe640 1 -- 172.21.15.5:0/2633506123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c61370 msgr2=0x557f92c62760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f028b7fe640 1 --2- 172.21.15.5:0/2633506123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c61370 0x557f92c62760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f028b7fe640 1 -- 172.21.15.5:0/2633506123 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f92c63f50 con 0x557f92c565d0 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f028bfff640 1 --2- 172.21.15.5:0/2633506123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c61370 0x557f92c62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f028b7fe640 1 --2- 172.21.15.5:0/2633506123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c565d0 0x557f92c6d2b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f027c0049d0 tx=0x7f027c0cfc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d441ac6af63a023 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f02817fa640 1 -- 172.21.15.5:0/2633506123 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c09b700 con 0x557f92c565d0 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f02817fa640 1 -- 172.21.15.5:0/2633506123 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f027c09bda0 con 0x557f92c565d0 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.972+0000 7f02817fa640 1 -- 172.21.15.5:0/2633506123 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c098410 con 0x557f92c565d0 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.973+0000 7f0294324740 1 -- 172.21.15.5:0/2633506123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c565d0 msgr2=0x557f92c6d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.973+0000 7f0294324740 1 --2- 172.21.15.5:0/2633506123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c565d0 0x557f92c6d2b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f027c0049d0 tx=0x7f027c0cfc50 comp rx=0 tx=0).stop 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.973+0000 7f0294324740 1 -- 172.21.15.5:0/2633506123 shutdown_connections 2024-09-11T22:24:41.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.973+0000 7f0294324740 1 --2- 172.21.15.5:0/2633506123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c61370 0x557f92c62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.973+0000 7f0294324740 1 --2- 172.21.15.5:0/2633506123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c72f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.973+0000 7f0294324740 1 --2- 172.21.15.5:0/2633506123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c565d0 0x557f92c6d2b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.973+0000 7f0294324740 1 -- 172.21.15.5:0/2633506123 >> 172.21.15.5:0/2633506123 conn(0x557f92c1f310 msgr2=0x557f92c5df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.973+0000 7f0294324740 1 -- 172.21.15.5:0/2633506123 shutdown_connections 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.973+0000 7f0294324740 1 -- 172.21.15.5:0/2633506123 wait complete. 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f0294324740 1 -- start start 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f0294324740 1 Processor -- start 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f0294324740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c69730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f0294324740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c69c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f0294324740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c78300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f0294324740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1aeee0 con 0x557f92c565d0 2024-09-11T22:24:41.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f0294324740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1ae810 con 0x557f92c57b40 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f0294324740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1aba50 con 0x557f92c61370 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f0294324740 10 client.0 _ll_register_callbacks cb 0x7f027c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c69730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f028b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c69730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33764/0 (socket says 172.21.15.5:33764) 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f028b7fe640 1 -- 172.21.15.5:0/34262210 learned_addr learned my addr 172.21.15.5:0/34262210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f028bfff640 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c78300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.974+0000 7f0283fff640 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c69c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f028b7fe640 1 -- 172.21.15.5:0/34262210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 msgr2=0x557f92c78300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f028b7fe640 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c78300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f028b7fe640 1 -- 172.21.15.5:0/34262210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 msgr2=0x557f92c69c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f028b7fe640 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c69c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f028b7fe640 1 -- 172.21.15.5:0/34262210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f92c53960 con 0x557f92c565d0 2024-09-11T22:24:41.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f028bfff640 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c78300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f028b7fe640 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c69730 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f027c004c00 tx=0x7f027c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5880f49792ac553 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f026affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f026affd640 1 -- 172.21.15.5:0/34262210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c09b700 con 0x557f92c565d0 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f026affd640 1 -- 172.21.15.5:0/34262210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f027c09b3f0 con 0x557f92c565d0 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f0294324740 1 -- 172.21.15.5:0/34262210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f027c09bda0 con 0x557f92c565d0 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f026affd640 1 -- 172.21.15.5:0/34262210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f027c08b050 con 0x557f92c565d0 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f0294324740 10 client.16173 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f0294324740 1 -- 172.21.15.5:0/34262210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f92c68df0 con 0x557f92c565d0 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.975+0000 7f026affd640 1 -- 172.21.15.5:0/34262210 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f027c09bda0 con 0x557f92c565d0 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.976+0000 7f026affd640 1 -- 172.21.15.5:0/34262210 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f027c086a60 con 0x557f92c565d0 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.976+0000 7f0294324740 10 client.16173 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.976+0000 7f0294324740 10 client.16173 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:24:41.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.976+0000 7f0294324740 20 client.16173 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.976+0000 7f0294324740 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557f92cdbc00 0x557f92cddff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.976+0000 7f0294324740 4 client.16173 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.976+0000 7f0294324740 1 -- 172.21.15.5:0/34262210 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f92bc3d70 con 0x557f92cdbc00 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.976+0000 7f0283fff640 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557f92cdbc00 0x557f92cddff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.976+0000 7f026affd640 10 client.16173 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.976+0000 7f0283fff640 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557f92cdbc00 0x557f92cddff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c77606390d505490 server_cookie=cd54a16ca91491f8 in_seq=0 out_seq=0 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f026affd640 1 -- 172.21.15.5:0/34262210 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f92bc3d70 con 0x557f92cdbc00 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f026affd640 10 client.16173 handle_command_reply: tid=0 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 client.16173 shutdown 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 -- 172.21.15.5:0/34262210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 msgr2=0x557f92c69730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c69730 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f027c004c00 tx=0x7f027c00c7d0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 -- 172.21.15.5:0/34262210 shutdown_connections 2024-09-11T22:24:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f92c61370 0x557f92c78300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557f92cdbc00 0x557f92cddff0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f026affd640 0 client.16173 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:24:41.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f92c57b40 0x557f92c69c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 --2- 172.21.15.5:0/34262210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f92c565d0 0x557f92c69730 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 -- 172.21.15.5:0/34262210 >> 172.21.15.5:0/34262210 conn(0x557f92c1f310 msgr2=0x557f92c5c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:41.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 -- 172.21.15.5:0/34262210 shutdown_connections 2024-09-11T22:24:41.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 1 -- 172.21.15.5:0/34262210 wait complete. 2024-09-11T22:24:41.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.977+0000 7f0294324740 20 client.16173 trim_cache size 0 max 16384 2024-09-11T22:24:41.988 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:24:41.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 -- 172.21.15.5:0/3289222555 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f02780a2d70 msgr2=0x7f02780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:41.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f02780a2d70 0x7f02780a5230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0284005fd0 tx=0x7f0284004ab0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 -- 172.21.15.5:0/3289222555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028c102d90 msgr2=0x7f028c198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028c102d90 0x7f028c198610 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f027c00e9f0 tx=0x7f027c00edf0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 -- 172.21.15.5:0/3289222555 shutdown_connections 2024-09-11T22:24:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f028c108290 0x7f028c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f02780a2d70 0x7f02780a5230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f028c103740 0x7f028c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 --2- 172.21.15.5:0/3289222555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028c102d90 0x7f028c198610 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 -- 172.21.15.5:0/3289222555 >> 172.21.15.5:0/3289222555 conn(0x7f028c0fc7d0 msgr2=0x7f028c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 -- 172.21.15.5:0/3289222555 shutdown_connections 2024-09-11T22:24:41.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:41.978+0000 7f0291f4b640 1 -- 172.21.15.5:0/3289222555 wait complete. 2024-09-11T22:24:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:41 smithi172 ceph-mon[35461]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 371 KiB/s wr, 56 op/s 2024-09-11T22:24:42.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 127791 11.80 MB/sec warmup 60 sec latency 37.687 ms 2024-09-11T22:24:42.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:41 smithi005 ceph-mon[30943]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 371 KiB/s wr, 56 op/s 2024-09-11T22:24:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:41 smithi159 ceph-mon[35534]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 531 MiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 371 KiB/s wr, 56 op/s 2024-09-11T22:24:42.779 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:24:43.046 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:43.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 130497 11.81 MB/sec warmup 61 sec latency 10.061 ms 2024-09-11T22:24:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.556+0000 7f1de8157640 1 -- 172.21.15.5:0/1596025958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de0100e20 msgr2=0x7f1de0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.556+0000 7f1de8157640 1 --2- 172.21.15.5:0/1596025958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de0100e20 0x7f1de0101200 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009900 tx=0x7f1dd802a480 comp rx=0 tx=0).stop 2024-09-11T22:24:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.557+0000 7f1de8157640 1 -- 172.21.15.5:0/1596025958 shutdown_connections 2024-09-11T22:24:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.557+0000 7f1de8157640 1 --2- 172.21.15.5:0/1596025958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de0102ef0 0x7f1de010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.557+0000 7f1de8157640 1 --2- 172.21.15.5:0/1596025958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de0102570 0x7f1de01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.557+0000 7f1de8157640 1 --2- 172.21.15.5:0/1596025958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de0100e20 0x7f1de0101200 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.557+0000 7f1de8157640 1 -- 172.21.15.5:0/1596025958 >> 172.21.15.5:0/1596025958 conn(0x7f1de00fc7d0 msgr2=0x7f1de00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.557+0000 7f1de8157640 1 -- 172.21.15.5:0/1596025958 shutdown_connections 2024-09-11T22:24:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.557+0000 7f1de8157640 1 -- 172.21.15.5:0/1596025958 wait complete. 2024-09-11T22:24:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.557+0000 7f1de8157640 1 Processor -- start 2024-09-11T22:24:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de8157640 1 -- start start 2024-09-11T22:24:43.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de8157640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de0100e20 0x7f1de01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de8157640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de0102570 0x7f1de0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de8157640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de0102ef0 0x7f1de019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de8157640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1de0110f80 con 0x7f1de0100e20 2024-09-11T22:24:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de8157640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1de0110e00 con 0x7f1de0102570 2024-09-11T22:24:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de8157640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1de0111100 con 0x7f1de0102ef0 2024-09-11T22:24:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de5ef3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de0100e20 0x7f1de01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de56f2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de0102570 0x7f1de0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de5ef3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de0100e20 0x7f1de01984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33790/0 (socket says 172.21.15.5:33790) 2024-09-11T22:24:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de5ef3640 1 -- 172.21.15.5:0/4207802234 learned_addr learned my addr 172.21.15.5:0/4207802234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de66f4640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de0102ef0 0x7f1de019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de56f2640 1 -- 172.21.15.5:0/4207802234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de0102ef0 msgr2=0x7f1de019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de56f2640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de0102ef0 0x7f1de019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de56f2640 1 -- 172.21.15.5:0/4207802234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de0100e20 msgr2=0x7f1de01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de56f2640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de0100e20 0x7f1de01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.558+0000 7f1de56f2640 1 -- 172.21.15.5:0/4207802234 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de019d480 con 0x7f1de0102570 2024-09-11T22:24:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.559+0000 7f1de56f2640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de0102570 0x7f1de0198a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400e9f0 tx=0x7f1dd400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.559+0000 7f1ddeffd640 1 -- 172.21.15.5:0/4207802234 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd400ccc0 con 0x7f1de0102570 2024-09-11T22:24:43.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.559+0000 7f1ddeffd640 1 -- 172.21.15.5:0/4207802234 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd4010910 con 0x7f1de0102570 2024-09-11T22:24:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.559+0000 7f1de8157640 1 -- 172.21.15.5:0/4207802234 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de019d770 con 0x7f1de0102570 2024-09-11T22:24:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.559+0000 7f1ddeffd640 1 -- 172.21.15.5:0/4207802234 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4018960 con 0x7f1de0102570 2024-09-11T22:24:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.559+0000 7f1de8157640 1 -- 172.21.15.5:0/4207802234 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de01a5050 con 0x7f1de0102570 2024-09-11T22:24:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.561+0000 7f1ddeffd640 1 -- 172.21.15.5:0/4207802234 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1dd4010ac0 con 0x7f1de0102570 2024-09-11T22:24:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.561+0000 7f1dea530740 1 Processor -- start 2024-09-11T22:24:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1dea530740 1 -- start start 2024-09-11T22:24:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1dea530740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01624f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1dea530740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 0x55b01624fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1dea530740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b01625c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1dea530740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b0161b52f0 con 0x55b01625a370 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1dea530740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b01615ccf0 con 0x55b016250b40 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1dea530740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b0161b12b0 con 0x55b01624f5d0 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1de66f4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01624f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1de56f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b01625c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1de66f4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01624f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48428/0 (socket says 172.21.15.5:48428) 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1de66f4640 1 -- 172.21.15.5:0/2725961578 learned_addr learned my addr 172.21.15.5:0/2725961578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1de5ef3640 1 --2- 172.21.15.5:0/2725961578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 0x55b01624fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1ddeffd640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1dc80a2cf0 0x7f1dc80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.562+0000 7f1de66f4640 1 -- 172.21.15.5:0/2725961578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 msgr2=0x55b01624fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1de66f4640 1 --2- 172.21.15.5:0/2725961578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 0x55b01624fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1de66f4640 1 -- 172.21.15.5:0/2725961578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 msgr2=0x55b01625c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1de66f4640 1 --2- 172.21.15.5:0/2725961578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b01625c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1de66f4640 1 -- 172.21.15.5:0/2725961578 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0161413a0 con 0x55b016250b40 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1de56f2640 1 --2- 172.21.15.5:0/2725961578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b01625c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1ddeffd640 1 -- 172.21.15.5:0/4207802234 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f1dd40cf880 con 0x7f1de0102570 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1de66f4640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1dc80a2cf0 0x7f1dc80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1de66f4640 1 --2- 172.21.15.5:0/2725961578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01624f090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009fd0 tx=0x7f1dd00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73332cdefcc2d3df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1dc7fff640 1 -- 172.21.15.5:0/2725961578 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0004050 con 0x55b016250b40 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1dc7fff640 1 -- 172.21.15.5:0/2725961578 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd0004750 con 0x55b016250b40 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.563+0000 7f1dc7fff640 1 -- 172.21.15.5:0/2725961578 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0019400 con 0x55b016250b40 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 -- 172.21.15.5:0/2725961578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 msgr2=0x55b01624f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:43.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 --2- 172.21.15.5:0/2725961578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01624f090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009fd0 tx=0x7f1dd00118d0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1de66f4640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1dc80a2cf0 0x7f1dc80a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1dd001d000 tx=0x7f1dd0019640 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 -- 172.21.15.5:0/2725961578 shutdown_connections 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 --2- 172.21.15.5:0/2725961578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b01625c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 --2- 172.21.15.5:0/2725961578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 0x55b01624fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 --2- 172.21.15.5:0/2725961578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01624f090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 -- 172.21.15.5:0/2725961578 >> 172.21.15.5:0/2725961578 conn(0x55b016218310 msgr2=0x55b01624bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 -- 172.21.15.5:0/2725961578 shutdown_connections 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 -- 172.21.15.5:0/2725961578 wait complete. 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 -- start start 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.564+0000 7f1dea530740 1 Processor -- start 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1dea530740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b01624f5d0 0x55b016259ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1dea530740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b016250b40 0x55b016256f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1dea530740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b016257490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1dea530740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b01625d070 con 0x55b01625a370 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1dea530740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b016170060 con 0x55b01624f5d0 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1dea530740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b01625d1f0 con 0x55b016250b40 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1dea530740 10 client.0 _ll_register_callbacks cb 0x55b015ec38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1de5ef3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b01624f5d0 0x55b016259ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1de66f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b016257490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1de5ef3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b01624f5d0 0x55b016259ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48442/0 (socket says 172.21.15.5:48442) 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1de5ef3640 1 -- 172.21.15.5:0/363258111 learned_addr learned my addr 172.21.15.5:0/363258111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1de56f2640 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b016250b40 0x55b016256f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1de5ef3640 1 -- 172.21.15.5:0/363258111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b016250b40 msgr2=0x55b016256f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1de5ef3640 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b016250b40 0x55b016256f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1de5ef3640 1 -- 172.21.15.5:0/363258111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 msgr2=0x55b016257490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1de5ef3640 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b016257490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.565+0000 7f1de5ef3640 1 -- 172.21.15.5:0/363258111 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b015f263c0 con 0x55b01624f5d0 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1de5ef3640 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b01624f5d0 0x55b016259ca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8000c00 tx=0x7f1dd80043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89f33dff590c0367 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dc57fa640 1 -- 172.21.15.5:0/363258111 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8035040 con 0x55b01624f5d0 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dea530740 1 -- 172.21.15.5:0/363258111 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd0004750 con 0x55b01624f5d0 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dc57fa640 1 -- 172.21.15.5:0/363258111 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f1dd80037f0 con 0x55b01624f5d0 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dc57fa640 1 -- 172.21.15.5:0/363258111 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1dd80039f0 con 0x55b01624f5d0 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dea530740 10 client.25709 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dea530740 1 -- 172.21.15.5:0/363258111 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b016268ab0 con 0x55b01624f5d0 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dc57fa640 1 -- 172.21.15.5:0/363258111 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd0004750 con 0x55b01624f5d0 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dc57fa640 1 -- 172.21.15.5:0/363258111 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1dd8046470 con 0x55b01624f5d0 2024-09-11T22:24:43.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dea530740 10 client.25709 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dea530740 10 client.25709 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dea530740 20 client.25709 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dea530740 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b016269ec0 0x55b01626c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dea530740 4 client.25709 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.566+0000 7f1dea530740 1 -- 172.21.15.5:0/363258111 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0161c2860 con 0x55b016269ec0 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.567+0000 7f1de56f2640 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b016269ec0 0x55b01626c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.567+0000 7f1dc57fa640 10 client.25709 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.567+0000 7f1de56f2640 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b016269ec0 0x55b01626c2f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9a3c4bc7705de0 server_cookie=3b187f7a1e5148d4 in_seq=0 out_seq=0 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dc57fa640 1 -- 172.21.15.5:0/363258111 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0161c2860 con 0x55b016269ec0 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dc57fa640 10 client.25709 handle_command_reply: tid=0 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 client.25709 shutdown 2024-09-11T22:24:43.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 -- 172.21.15.5:0/363258111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b01624f5d0 msgr2=0x55b016259ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b01624f5d0 0x55b016259ca0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8000c00 tx=0x7f1dd80043b0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 -- 172.21.15.5:0/363258111 shutdown_connections 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b016269ec0 0x55b01626c2f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b016257490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b016250b40 0x55b016256f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 --2- 172.21.15.5:0/363258111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b01624f5d0 0x55b016259ca0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dc57fa640 0 client.25709 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 -- 172.21.15.5:0/363258111 >> 172.21.15.5:0/363258111 conn(0x55b016218310 msgr2=0x55b016251960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 -- 172.21.15.5:0/363258111 shutdown_connections 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 1 -- 172.21.15.5:0/363258111 wait complete. 2024-09-11T22:24:43.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.568+0000 7f1dea530740 20 client.25709 trim_cache size 0 max 16384 2024-09-11T22:24:43.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.582+0000 7f1dea530740 1 Processor -- start 2024-09-11T22:24:43.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.582+0000 7f1dea530740 1 -- start start 2024-09-11T22:24:43.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.582+0000 7f1dea530740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01624f5d0 0x55b016266310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.582+0000 7f1dea530740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b016271bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.582+0000 7f1dea530740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01625a370 0x55b016272100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.582+0000 7f1dea530740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01ab730 con 0x55b01624f5d0 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.582+0000 7f1dea530740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01ad330 con 0x55b016250b40 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.582+0000 7f1dea530740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01abe00 con 0x55b01625a370 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de66f4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01625a370 0x55b016272100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de5ef3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01624f5d0 0x55b016266310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de56f2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b016271bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de66f4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01625a370 0x55b016272100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:34160/0 (socket says 172.21.15.5:34160) 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de66f4640 1 -- 172.21.15.5:0/909759187 learned_addr learned my addr 172.21.15.5:0/909759187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de5ef3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01624f5d0 0x55b016266310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33822/0 (socket says 172.21.15.5:33822) 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de66f4640 1 -- 172.21.15.5:0/909759187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 msgr2=0x55b016271bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de66f4640 1 --2- 172.21.15.5:0/909759187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b016271bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de66f4640 1 -- 172.21.15.5:0/909759187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01624f5d0 msgr2=0x55b016266310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de66f4640 1 --2- 172.21.15.5:0/909759187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01624f5d0 0x55b016266310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de66f4640 1 -- 172.21.15.5:0/909759187 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b01624c960 con 0x55b01625a370 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de56f2640 1 --2- 172.21.15.5:0/909759187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b016271bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1de66f4640 1 --2- 172.21.15.5:0/909759187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01625a370 0x55b016272100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0011dd0 tx=0x7f1dd0028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa17772c4cc80b08 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.583+0000 7f1dc77fe640 1 -- 172.21.15.5:0/909759187 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0009b30 con 0x55b01625a370 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dc77fe640 1 -- 172.21.15.5:0/909759187 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd003d070 con 0x55b01625a370 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dc77fe640 1 -- 172.21.15.5:0/909759187 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd002d830 con 0x55b01625a370 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dea530740 1 -- 172.21.15.5:0/909759187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01625a370 msgr2=0x55b016272100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dea530740 1 --2- 172.21.15.5:0/909759187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01625a370 0x55b016272100 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0011dd0 tx=0x7f1dd0028000 comp rx=0 tx=0).stop 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dea530740 1 -- 172.21.15.5:0/909759187 shutdown_connections 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dea530740 1 --2- 172.21.15.5:0/909759187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01625a370 0x55b016272100 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dea530740 1 --2- 172.21.15.5:0/909759187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b016271bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dea530740 1 --2- 172.21.15.5:0/909759187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01624f5d0 0x55b016266310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dea530740 1 -- 172.21.15.5:0/909759187 >> 172.21.15.5:0/909759187 conn(0x55b016218310 msgr2=0x55b016256f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dea530740 1 -- 172.21.15.5:0/909759187 shutdown_connections 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.584+0000 7f1dea530740 1 -- 172.21.15.5:0/909759187 wait complete. 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.585+0000 7f1dea530740 1 -- start start 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.585+0000 7f1dea530740 1 Processor -- start 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.585+0000 7f1dea530740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 0x55b0162629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.585+0000 7f1dea530740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01625f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.585+0000 7f1dea530740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b01625f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.585+0000 7f1dea530740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01acba0 con 0x55b01625a370 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.585+0000 7f1dea530740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01ac4d0 con 0x55b016250b40 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.585+0000 7f1dea530740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01a9710 con 0x55b01624f5d0 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.585+0000 7f1dea530740 10 client.0 _ll_register_callbacks cb 0x7f1dd8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:43.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.585+0000 7f1de5ef3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 0x55b0162629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de56f2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01625f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de5ef3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 0x55b0162629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:34176/0 (socket says 172.21.15.5:34176) 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de5ef3640 1 -- 172.21.15.5:0/743481430 learned_addr learned my addr 172.21.15.5:0/743481430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de66f4640 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b01625f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de5ef3640 1 -- 172.21.15.5:0/743481430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 msgr2=0x55b01625f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de5ef3640 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01625f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de5ef3640 1 -- 172.21.15.5:0/743481430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 msgr2=0x55b01625f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de5ef3640 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b01625f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de5ef3640 1 -- 172.21.15.5:0/743481430 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b01625ff80 con 0x55b01624f5d0 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de56f2640 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01625f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1de5ef3640 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 0x55b0162629c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8000c00 tx=0x7f1dd8005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f9a4c2957940509 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1dc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:24:43.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1dc57fa640 1 -- 172.21.15.5:0/743481430 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8035030 con 0x55b01624f5d0 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1dc57fa640 1 -- 172.21.15.5:0/743481430 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f1dd80037f0 con 0x55b01624f5d0 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1dea530740 1 -- 172.21.15.5:0/743481430 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd003d070 con 0x55b01624f5d0 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.586+0000 7f1dc57fa640 1 -- 172.21.15.5:0/743481430 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1dd805e050 con 0x55b01624f5d0 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1dea530740 10 client.25660 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1dea530740 1 -- 172.21.15.5:0/743481430 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b0162d59a0 con 0x55b01624f5d0 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1dc57fa640 1 -- 172.21.15.5:0/743481430 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dd003d070 con 0x55b01624f5d0 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1dc57fa640 1 -- 172.21.15.5:0/743481430 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1dd80046c0 con 0x55b01624f5d0 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1dea530740 10 client.25660 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1dea530740 10 client.25660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1dea530740 20 client.25660 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1dea530740 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0162d60c0 0x55b0162d84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1dea530740 4 client.25660 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1dea530740 1 -- 172.21.15.5:0/743481430 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0161bcd70 con 0x55b0162d60c0 2024-09-11T22:24:43.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.587+0000 7f1de56f2640 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0162d60c0 0x55b0162d84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dc57fa640 10 client.25660 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1de56f2640 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0162d60c0 0x55b0162d84e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e675677731e58bed server_cookie=59b17d07c04a9eea in_seq=0 out_seq=0 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dc57fa640 1 -- 172.21.15.5:0/743481430 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0161bcd70 con 0x55b0162d60c0 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dc57fa640 10 client.25660 handle_command_reply: tid=0 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 client.25660 shutdown 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 -- 172.21.15.5:0/743481430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 msgr2=0x55b0162629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 0x55b0162629c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8000c00 tx=0x7f1dd8005b50 comp rx=0 tx=0).stop 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 -- 172.21.15.5:0/743481430 shutdown_connections 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0162d60c0 0x55b0162d84e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b01625a370 0x55b01625f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b016250b40 0x55b01625f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dc57fa640 0 client.25660 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 --2- 172.21.15.5:0/743481430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b01624f5d0 0x55b0162629c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 -- 172.21.15.5:0/743481430 >> 172.21.15.5:0/743481430 conn(0x55b016218310 msgr2=0x55b01624a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:43.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 -- 172.21.15.5:0/743481430 shutdown_connections 2024-09-11T22:24:43.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 1 -- 172.21.15.5:0/743481430 wait complete. 2024-09-11T22:24:43.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.588+0000 7f1dea530740 20 client.25660 trim_cache size 0 max 16384 2024-09-11T22:24:43.604 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:24:43.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 -- 172.21.15.5:0/4207802234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1dc80a2cf0 msgr2=0x7f1dc80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:43.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1dc80a2cf0 0x7f1dc80a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1dd001d000 tx=0x7f1dd0019640 comp rx=0 tx=0).stop 2024-09-11T22:24:43.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 -- 172.21.15.5:0/4207802234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de0102570 msgr2=0x7f1de0198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:43.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de0102570 0x7f1de0198a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400e9f0 tx=0x7f1dd400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 -- 172.21.15.5:0/4207802234 shutdown_connections 2024-09-11T22:24:43.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1de0102ef0 0x7f1de019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1dc80a2cf0 0x7f1dc80a51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1de0102570 0x7f1de0198a10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 --2- 172.21.15.5:0/4207802234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1de0100e20 0x7f1de01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:43.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 -- 172.21.15.5:0/4207802234 >> 172.21.15.5:0/4207802234 conn(0x7f1de00fc7d0 msgr2=0x7f1de006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:43.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 -- 172.21.15.5:0/4207802234 shutdown_connections 2024-09-11T22:24:43.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:43.589+0000 7f1de8157640 1 -- 172.21.15.5:0/4207802234 wait complete. 2024-09-11T22:24:44.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:43 smithi172 ceph-mon[35461]: pgmap v78: 97 pgs: 97 active+clean; 38 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 226 op/s 2024-09-11T22:24:44.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 132702 11.83 MB/sec warmup 62 sec latency 25.940 ms 2024-09-11T22:24:44.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:43 smithi005 ceph-mon[30943]: pgmap v78: 97 pgs: 97 active+clean; 38 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 226 op/s 2024-09-11T22:24:44.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:43 smithi159 ceph-mon[35534]: pgmap v78: 97 pgs: 97 active+clean; 38 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 226 op/s 2024-09-11T22:24:44.428 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:24:44.699 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.182+0000 7f881ec2b640 1 -- 172.21.15.5:0/2559602856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102ef0 msgr2=0x7f881810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.182+0000 7f881ec2b640 1 --2- 172.21.15.5:0/2559602856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102ef0 0x7f881810da40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8800009900 tx=0x7f880002a480 comp rx=0 tx=0).stop 2024-09-11T22:24:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.182+0000 7f881ec2b640 1 -- 172.21.15.5:0/2559602856 shutdown_connections 2024-09-11T22:24:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.182+0000 7f881ec2b640 1 --2- 172.21.15.5:0/2559602856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102ef0 0x7f881810da40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.182+0000 7f881ec2b640 1 --2- 172.21.15.5:0/2559602856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8818102570 0x7f88181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.182+0000 7f881ec2b640 1 --2- 172.21.15.5:0/2559602856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8818100e20 0x7f8818101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.182+0000 7f881ec2b640 1 -- 172.21.15.5:0/2559602856 >> 172.21.15.5:0/2559602856 conn(0x7f88180fc7d0 msgr2=0x7f88180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.182+0000 7f881ec2b640 1 -- 172.21.15.5:0/2559602856 shutdown_connections 2024-09-11T22:24:45.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.182+0000 7f881ec2b640 1 -- 172.21.15.5:0/2559602856 wait complete. 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881ec2b640 1 Processor -- start 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881ec2b640 1 -- start start 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881ec2b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8818100e20 0x7f881819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881ec2b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8818102570 0x7f881819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881ec2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102ef0 0x7f881819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881ec2b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8818110f80 con 0x7f8818102ef0 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881ec2b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8818110e00 con 0x7f8818100e20 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881ec2b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8818111100 con 0x7f8818102570 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881d1c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102ef0 0x7f881819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881c9c7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8818100e20 0x7f881819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881d1c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102ef0 0x7f881819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33842/0 (socket says 172.21.15.5:33842) 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f881d1c8640 1 -- 172.21.15.5:0/2551203258 learned_addr learned my addr 172.21.15.5:0/2551203258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.183+0000 7f8817fff640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8818102570 0x7f881819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f881d1c8640 1 -- 172.21.15.5:0/2551203258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8818102570 msgr2=0x7f881819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f881d1c8640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8818102570 0x7f881819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f881d1c8640 1 -- 172.21.15.5:0/2551203258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8818100e20 msgr2=0x7f881819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f881d1c8640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8818100e20 0x7f881819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f881d1c8640 1 -- 172.21.15.5:0/2551203258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881819f620 con 0x7f8818102ef0 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f881c9c7640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8818100e20 0x7f881819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f881d1c8640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102ef0 0x7f881819ef40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8800004770 tx=0x7f8800003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f8815ffb640 1 -- 172.21.15.5:0/2551203258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880003f070 con 0x7f8818102ef0 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f8815ffb640 1 -- 172.21.15.5:0/2551203258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8800031c30 con 0x7f8818102ef0 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f8815ffb640 1 -- 172.21.15.5:0/2551203258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800035030 con 0x7f8818102ef0 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f881ec2b640 1 -- 172.21.15.5:0/2551203258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881819f8b0 con 0x7f8818102ef0 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.184+0000 7f881ec2b640 1 -- 172.21.15.5:0/2551203258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88181a71f0 con 0x7f8818102ef0 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.186+0000 7f8815ffb640 1 -- 172.21.15.5:0/2551203258 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8800042070 con 0x7f8818102ef0 2024-09-11T22:24:45.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.187+0000 7f8821004740 1 Processor -- start 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.187+0000 7f8821004740 1 -- start start 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.187+0000 7f8821004740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 0x562f53460090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.187+0000 7f8821004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f53460a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.187+0000 7f8821004740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f5346d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.187+0000 7f8821004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562f533c62f0 con 0x562f534605d0 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8821004740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562f5336dcf0 con 0x562f5346b370 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.187+0000 7f8815ffb640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f88040a2cf0 0x7f88040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8821004740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562f533c22b0 con 0x562f53461b40 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f881c9c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f53460a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f881c9c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f53460a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33858/0 (socket says 172.21.15.5:33858) 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f881c9c7640 1 -- 172.21.15.5:0/2124391216 learned_addr learned my addr 172.21.15.5:0/2124391216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8817fff640 1 --2- 172.21.15.5:0/2124391216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f5346d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f881d1c8640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f88040a2cf0 0x7f88040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8817fff640 1 --2- 172.21.15.5:0/2124391216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 0x562f53460090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8815ffb640 1 -- 172.21.15.5:0/2551203258 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f88000eff30 con 0x7f8818102ef0 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8817fff640 1 -- 172.21.15.5:0/2124391216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 msgr2=0x562f53460090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8817fff640 1 --2- 172.21.15.5:0/2124391216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 0x562f53460090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8817fff640 1 -- 172.21.15.5:0/2124391216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 msgr2=0x562f53460a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8817fff640 1 --2- 172.21.15.5:0/2124391216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f53460a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8817fff640 1 -- 172.21.15.5:0/2124391216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f533523a0 con 0x562f5346b370 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f8817fff640 1 --2- 172.21.15.5:0/2124391216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f5346d760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f88180ff060 tx=0x7f8808009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bcc192739e04326 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f881d1c8640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f88040a2cf0 0x7f88040a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88000bc3f0 tx=0x7f88000a6f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f880effd640 1 -- 172.21.15.5:0/2124391216 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808011070 con 0x562f5346b370 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f880effd640 1 -- 172.21.15.5:0/2124391216 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8808003e50 con 0x562f5346b370 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.188+0000 7f880effd640 1 -- 172.21.15.5:0/2124391216 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808004250 con 0x562f5346b370 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.189+0000 7f8821004740 1 -- 172.21.15.5:0/2124391216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 msgr2=0x562f5346d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.189+0000 7f8821004740 1 --2- 172.21.15.5:0/2124391216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f5346d760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f88180ff060 tx=0x7f8808009990 comp rx=0 tx=0).stop 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.189+0000 7f8821004740 1 -- 172.21.15.5:0/2124391216 shutdown_connections 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.189+0000 7f8821004740 1 --2- 172.21.15.5:0/2124391216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f5346d760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.189+0000 7f8821004740 1 --2- 172.21.15.5:0/2124391216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f53460a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.189+0000 7f8821004740 1 --2- 172.21.15.5:0/2124391216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 0x562f53460090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.189+0000 7f8821004740 1 -- 172.21.15.5:0/2124391216 >> 172.21.15.5:0/2124391216 conn(0x562f53429310 msgr2=0x562f5345cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.189+0000 7f8821004740 1 -- 172.21.15.5:0/2124391216 shutdown_connections 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.189+0000 7f8821004740 1 -- 172.21.15.5:0/2124391216 wait complete. 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.189+0000 7f8821004740 1 -- start start 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8821004740 1 Processor -- start 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8821004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5346ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8821004740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 0x562f53468000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8821004740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f534685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8821004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562f5346e070 con 0x562f534605d0 2024-09-11T22:24:45.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8821004740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562f53381060 con 0x562f5346b370 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8821004740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562f5346e1f0 con 0x562f53461b40 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8821004740 10 client.0 _ll_register_callbacks cb 0x562f530d48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8817fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5346ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f881c9c7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 0x562f53468000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f881c9c7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 0x562f53468000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:34206/0 (socket says 172.21.15.5:34206) 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8817fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5346ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33872/0 (socket says 172.21.15.5:33872) 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f881c9c7640 1 -- 172.21.15.5:0/2076965582 learned_addr learned my addr 172.21.15.5:0/2076965582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f8817fff640 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f534685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f881c9c7640 1 -- 172.21.15.5:0/2076965582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 msgr2=0x562f534685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f881c9c7640 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f534685d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.190+0000 7f881c9c7640 1 -- 172.21.15.5:0/2076965582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 msgr2=0x562f5346ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f881c9c7640 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5346ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f881c9c7640 1 -- 172.21.15.5:0/2076965582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f531373c0 con 0x562f53461b40 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f8817fff640 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f534685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f881c9c7640 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 0x562f53468000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f88100044c0 tx=0x7f88100039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75b7f33f28a74e34 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f87f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f87f7fff640 1 -- 172.21.15.5:0/2076965582 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88100117d0 con 0x562f53461b40 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f87f7fff640 1 -- 172.21.15.5:0/2076965582 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f8810011970 con 0x562f53461b40 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f8821004740 1 -- 172.21.15.5:0/2076965582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8808003e50 con 0x562f53461b40 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f87f7fff640 1 -- 172.21.15.5:0/2076965582 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8810011b70 con 0x562f53461b40 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f8821004740 10 client.25672 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f8821004740 1 -- 172.21.15.5:0/2076965582 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f53482360 con 0x562f53461b40 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f87f7fff640 1 -- 172.21.15.5:0/2076965582 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8808003e50 con 0x562f53461b40 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.191+0000 7f87f7fff640 1 -- 172.21.15.5:0/2076965582 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f8810021030 con 0x562f53461b40 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.192+0000 7f8821004740 10 client.25672 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.192+0000 7f8821004740 10 client.25672 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:24:45.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.192+0000 7f8821004740 20 client.25672 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.192+0000 7f8821004740 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562f534838c0 0x562f53485ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.192+0000 7f8821004740 4 client.25672 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.192+0000 7f8821004740 1 -- 172.21.15.5:0/2076965582 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f533d3860 con 0x562f534838c0 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.192+0000 7f8817fff640 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562f534838c0 0x562f53485ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.192+0000 7f8817fff640 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562f534838c0 0x562f53485ce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f400175c93fee2a8 server_cookie=53c8ec63090c5a28 in_seq=0 out_seq=0 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.192+0000 7f87f7fff640 10 client.25672 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f87f7fff640 1 -- 172.21.15.5:0/2076965582 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f533d3860 con 0x562f534838c0 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f87f7fff640 10 client.25672 handle_command_reply: tid=0 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 client.25672 shutdown 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 -- 172.21.15.5:0/2076965582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 msgr2=0x562f53468000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 0x562f53468000 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f88100044c0 tx=0x7f88100039c0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 -- 172.21.15.5:0/2076965582 shutdown_connections 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562f534838c0 0x562f53485ce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f534685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f53461b40 0x562f53468000 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 --2- 172.21.15.5:0/2076965582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5346ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f87f7fff640 0 client.25672 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:24:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 -- 172.21.15.5:0/2076965582 >> 172.21.15.5:0/2076965582 conn(0x562f53429310 msgr2=0x562f53462960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 -- 172.21.15.5:0/2076965582 shutdown_connections 2024-09-11T22:24:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 1 -- 172.21.15.5:0/2076965582 wait complete. 2024-09-11T22:24:45.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.193+0000 7f8821004740 20 client.25672 trim_cache size 0 max 16384 2024-09-11T22:24:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.207+0000 7f8821004740 1 Processor -- start 2024-09-11T22:24:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.207+0000 7f8821004740 1 -- start start 2024-09-11T22:24:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.207+0000 7f8821004740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f534605d0 0x562f5347fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.207+0000 7f8821004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f53461b40 0x562f5348b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.207+0000 7f8821004740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f5348bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.207+0000 7f8821004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88181ab760 con 0x562f53461b40 2024-09-11T22:24:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f8821004740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f88181ad360 con 0x562f5346b370 2024-09-11T22:24:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f8821004740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f88181abe30 con 0x562f534605d0 2024-09-11T22:24:45.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f881c9c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f53461b40 0x562f5348b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f8817fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f534605d0 0x562f5347fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f881c9c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f53461b40 0x562f5348b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33886/0 (socket says 172.21.15.5:33886) 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f881c9c7640 1 -- 172.21.15.5:0/947093709 learned_addr learned my addr 172.21.15.5:0/947093709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f8817fff640 1 --2- 172.21.15.5:0/947093709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f5348bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f8817fff640 1 -- 172.21.15.5:0/947093709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 msgr2=0x562f5348bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f8817fff640 1 --2- 172.21.15.5:0/947093709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f5348bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f8817fff640 1 -- 172.21.15.5:0/947093709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f53461b40 msgr2=0x562f5348b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f8817fff640 1 --2- 172.21.15.5:0/947093709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f53461b40 0x562f5348b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f8817fff640 1 -- 172.21.15.5:0/947093709 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f53486340 con 0x562f534605d0 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f881c9c7640 1 --2- 172.21.15.5:0/947093709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f53461b40 0x562f5348b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f8817fff640 1 --2- 172.21.15.5:0/947093709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f534605d0 0x562f5347fba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8808009a00 tx=0x7f880800fe10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f0d2c5e17854b73 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f880e7fc640 1 -- 172.21.15.5:0/947093709 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880800fe80 con 0x562f534605d0 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f880e7fc640 1 -- 172.21.15.5:0/947093709 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8808003e50 con 0x562f534605d0 2024-09-11T22:24:45.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.208+0000 7f880e7fc640 1 -- 172.21.15.5:0/947093709 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880802d300 con 0x562f534605d0 2024-09-11T22:24:45.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.209+0000 7f8821004740 1 -- 172.21.15.5:0/947093709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f534605d0 msgr2=0x562f5347fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.209+0000 7f8821004740 1 --2- 172.21.15.5:0/947093709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f534605d0 0x562f5347fba0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8808009a00 tx=0x7f880800fe10 comp rx=0 tx=0).stop 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.209+0000 7f8821004740 1 -- 172.21.15.5:0/947093709 shutdown_connections 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.209+0000 7f8821004740 1 --2- 172.21.15.5:0/947093709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f5346b370 0x562f5348bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.209+0000 7f8821004740 1 --2- 172.21.15.5:0/947093709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f53461b40 0x562f5348b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.209+0000 7f8821004740 1 --2- 172.21.15.5:0/947093709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f534605d0 0x562f5347fba0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.209+0000 7f8821004740 1 -- 172.21.15.5:0/947093709 >> 172.21.15.5:0/947093709 conn(0x562f53429310 msgr2=0x562f534670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.209+0000 7f8821004740 1 -- 172.21.15.5:0/947093709 shutdown_connections 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.209+0000 7f8821004740 1 -- 172.21.15.5:0/947093709 wait complete. 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.209+0000 7f8821004740 1 -- start start 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8821004740 1 Processor -- start 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8821004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5347bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8821004740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f53461b40 0x562f5347c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8821004740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f5346b370 0x562f53478ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8821004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88181acbd0 con 0x562f534605d0 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8821004740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f88181ac500 con 0x562f53461b40 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8821004740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f88181a9760 con 0x562f5346b370 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8821004740 10 client.0 _ll_register_callbacks cb 0x7f881000bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8817fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5347bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8817fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5347bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33896/0 (socket says 172.21.15.5:33896) 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8817fff640 1 -- 172.21.15.5:0/3396848457 learned_addr learned my addr 172.21.15.5:0/3396848457 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f881c9c7640 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f53461b40 0x562f5347c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8817fff640 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f5346b370 0x562f53478ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8817fff640 1 -- 172.21.15.5:0/3396848457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f5346b370 msgr2=0x562f53478ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:45.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8817fff640 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f5346b370 0x562f53478ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8817fff640 1 -- 172.21.15.5:0/3396848457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f53461b40 msgr2=0x562f5347c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8817fff640 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f53461b40 0x562f5347c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f8817fff640 1 -- 172.21.15.5:0/3396848457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f53479220 con 0x562f534605d0 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.210+0000 7f881c9c7640 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f53461b40 0x562f5347c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f8817fff640 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5347bd10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8808002230 tx=0x7f88080053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db13e0127f6befc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f87f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f87f7fff640 1 -- 172.21.15.5:0/3396848457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808033040 con 0x562f534605d0 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f87f7fff640 1 -- 172.21.15.5:0/3396848457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f8808005110 con 0x562f534605d0 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f8821004740 1 -- 172.21.15.5:0/3396848457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8808003e50 con 0x562f534605d0 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f87f7fff640 1 -- 172.21.15.5:0/3396848457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8808005580 con 0x562f534605d0 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f8821004740 10 client.16209 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f8821004740 1 -- 172.21.15.5:0/3396848457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562f534eebe0 con 0x562f534605d0 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f87f7fff640 1 -- 172.21.15.5:0/3396848457 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8808003e50 con 0x562f534605d0 2024-09-11T22:24:45.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f87f7fff640 1 -- 172.21.15.5:0/3396848457 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f880801d020 con 0x562f534605d0 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f8821004740 10 client.16209 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f8821004740 10 client.16209 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.211+0000 7f8821004740 20 client.16209 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562f534ef1c0 0x562f534f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 4 client.16209 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 -- 172.21.15.5:0/3396848457 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562f533cdd70 con 0x562f534ef1c0 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f881c9c7640 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562f534ef1c0 0x562f534f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f881c9c7640 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562f534ef1c0 0x562f534f15b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef5110f3291f06db server_cookie=9fa6294a2d2b7e57 in_seq=0 out_seq=0 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f87f7fff640 10 client.16209 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f87f7fff640 1 -- 172.21.15.5:0/3396848457 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562f533cdd70 con 0x562f534ef1c0 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f87f7fff640 10 client.16209 handle_command_reply: tid=0 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 client.16209 shutdown 2024-09-11T22:24:45.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 -- 172.21.15.5:0/3396848457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 msgr2=0x562f5347bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5347bd10 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8808002230 tx=0x7f88080053e0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 -- 172.21.15.5:0/3396848457 shutdown_connections 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x562f534ef1c0 0x562f534f15b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562f5346b370 0x562f53478ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f53461b40 0x562f5347c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f87f7fff640 0 client.16209 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 --2- 172.21.15.5:0/3396848457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f534605d0 0x562f5347bd10 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.212+0000 7f8821004740 1 -- 172.21.15.5:0/3396848457 >> 172.21.15.5:0/3396848457 conn(0x562f53429310 msgr2=0x562f5345b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f8821004740 1 -- 172.21.15.5:0/3396848457 shutdown_connections 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f8821004740 1 -- 172.21.15.5:0/3396848457 wait complete. 2024-09-11T22:24:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f8821004740 20 client.16209 trim_cache size 0 max 16384 2024-09-11T22:24:45.222 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:24:45.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 -- 172.21.15.5:0/2551203258 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f88040a2cf0 msgr2=0x7f88040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f88040a2cf0 0x7f88040a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88000bc3f0 tx=0x7f88000a6f90 comp rx=0 tx=0).stop 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 -- 172.21.15.5:0/2551203258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102ef0 msgr2=0x7f881819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102ef0 0x7f881819ef40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8800004770 tx=0x7f8800003c10 comp rx=0 tx=0).stop 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 -- 172.21.15.5:0/2551203258 shutdown_connections 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102ef0 0x7f881819ef40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8818102570 0x7f881819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f88040a2cf0 0x7f88040a51b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 --2- 172.21.15.5:0/2551203258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8818100e20 0x7f881819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 -- 172.21.15.5:0/2551203258 >> 172.21.15.5:0/2551203258 conn(0x7f88180fc7d0 msgr2=0x7f881806a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 -- 172.21.15.5:0/2551203258 shutdown_connections 2024-09-11T22:24:45.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:45.213+0000 7f881ec2b640 1 -- 172.21.15.5:0/2551203258 wait complete. 2024-09-11T22:24:45.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 135290 11.87 MB/sec warmup 63 sec latency 29.451 ms 2024-09-11T22:24:45.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:24:46.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:45 smithi172 ceph-mon[35461]: pgmap v79: 97 pgs: 97 active+clean; 38 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-11T22:24:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:46.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 138377 11.96 MB/sec warmup 64 sec latency 10.883 ms 2024-09-11T22:24:46.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:45 smithi005 ceph-mon[30943]: pgmap v79: 97 pgs: 97 active+clean; 38 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-11T22:24:46.328 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:45 smithi159 ceph-mon[35534]: pgmap v79: 97 pgs: 97 active+clean; 38 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-11T22:24:46.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.786+0000 7f32f1bba640 1 -- 172.21.15.5:0/3988644013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ec101de0 msgr2=0x7f32ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:46.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.786+0000 7f32f1bba640 1 --2- 172.21.15.5:0/3988644013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ec101de0 0x7f32ec1021c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009900 tx=0x7f32dc02a410 comp rx=0 tx=0).stop 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.786+0000 7f32f1bba640 1 -- 172.21.15.5:0/3988644013 shutdown_connections 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.786+0000 7f32f1bba640 1 --2- 172.21.15.5:0/3988644013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec106f10 0x7f32ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.786+0000 7f32f1bba640 1 --2- 172.21.15.5:0/3988644013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ec102790 0x7f32ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.786+0000 7f32f1bba640 1 --2- 172.21.15.5:0/3988644013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ec101de0 0x7f32ec1021c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.786+0000 7f32f1bba640 1 -- 172.21.15.5:0/3988644013 >> 172.21.15.5:0/3988644013 conn(0x7f32ec0fd8c0 msgr2=0x7f32ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.786+0000 7f32f1bba640 1 -- 172.21.15.5:0/3988644013 shutdown_connections 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.786+0000 7f32f1bba640 1 -- 172.21.15.5:0/3988644013 wait complete. 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32f1bba640 1 Processor -- start 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32f1bba640 1 -- start start 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32f1bba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec101de0 0x7f32ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32f1bba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ec102790 0x7f32ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32f1bba640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ec106f10 0x7f32ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32f1bba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec111f70 con 0x7f32ec102790 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32f1bba640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec111df0 con 0x7f32ec106f10 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32f1bba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1120f0 con 0x7f32ec101de0 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ec102790 0x7f32ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ec102790 0x7f32ec074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33916/0 (socket says 172.21.15.5:33916) 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32eaffd640 1 -- 172.21.15.5:0/926501511 learned_addr learned my addr 172.21.15.5:0/926501511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.787+0000 7f32eb7fe640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec101de0 0x7f32ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32ebfff640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ec106f10 0x7f32ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32eaffd640 1 -- 172.21.15.5:0/926501511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec101de0 msgr2=0x7f32ec0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32eaffd640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec101de0 0x7f32ec0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32eaffd640 1 -- 172.21.15.5:0/926501511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ec106f10 msgr2=0x7f32ec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32eaffd640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ec106f10 0x7f32ec074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32eaffd640 1 -- 172.21.15.5:0/926501511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec1a68f0 con 0x7f32ec102790 2024-09-11T22:24:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32eaffd640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ec102790 0x7f32ec074a30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32e8ff9640 1 -- 172.21.15.5:0/926501511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e000cd80 con 0x7f32ec102790 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32e8ff9640 1 -- 172.21.15.5:0/926501511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e0010910 con 0x7f32ec102790 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32f1bba640 1 -- 172.21.15.5:0/926501511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ec1a6be0 con 0x7f32ec102790 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32e8ff9640 1 -- 172.21.15.5:0/926501511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0018990 con 0x7f32ec102790 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.788+0000 7f32f1bba640 1 -- 172.21.15.5:0/926501511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ec1a7120 con 0x7f32ec102790 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.790+0000 7f32e8ff9640 1 -- 172.21.15.5:0/926501511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f32e0010ac0 con 0x7f32ec102790 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32f3f93740 1 Processor -- start 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32e8ff9640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f32d40a2cf0 0x7f32d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32f3f93740 1 -- start start 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32ebfff640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f32d40a2cf0 0x7f32d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd764090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32e8ff9640 1 -- 172.21.15.5:0/926501511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f32e00cf7e0 con 0x7f32ec102790 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd764a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd76f370 0x55cabd771760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32f3f93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cabd6ca2f0 con 0x55cabd765b40 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32f3f93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cabd671cf0 con 0x55cabd76f370 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.791+0000 7f32f3f93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cabd6c62b0 con 0x55cabd7645d0 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd764090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd764090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33926/0 (socket says 172.21.15.5:33926) 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd764a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eb7fe640 1 -- 172.21.15.5:0/1066675347 learned_addr learned my addr 172.21.15.5:0/1066675347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32ebfff640 1 --2- 172.21.15.5:0/1066675347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd76f370 0x55cabd771760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eb7fe640 1 -- 172.21.15.5:0/1066675347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 msgr2=0x55cabd764a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eb7fe640 1 --2- 172.21.15.5:0/1066675347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd764a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eb7fe640 1 -- 172.21.15.5:0/1066675347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd76f370 msgr2=0x55cabd771760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:46.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eb7fe640 1 --2- 172.21.15.5:0/1066675347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd76f370 0x55cabd771760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eb7fe640 1 -- 172.21.15.5:0/1066675347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabd6563a0 con 0x55cabd765b40 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eaffd640 1 --2- 172.21.15.5:0/1066675347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd764a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32ebfff640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f32d40a2cf0 0x7f32d40a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005fd0 tx=0x7f32e4011b40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32eb7fe640 1 --2- 172.21.15.5:0/1066675347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd764090 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009900 tx=0x7f32dc005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e14f9afedd6a3fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32d1ffb640 1 -- 172.21.15.5:0/1066675347 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc040070 con 0x55cabd765b40 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32d1ffb640 1 -- 172.21.15.5:0/1066675347 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32dc031870 con 0x55cabd765b40 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.792+0000 7f32d1ffb640 1 -- 172.21.15.5:0/1066675347 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc035040 con 0x55cabd765b40 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.793+0000 7f32f3f93740 1 -- 172.21.15.5:0/1066675347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 msgr2=0x55cabd764090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.793+0000 7f32f3f93740 1 --2- 172.21.15.5:0/1066675347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd764090 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009900 tx=0x7f32dc005be0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.793+0000 7f32f3f93740 1 -- 172.21.15.5:0/1066675347 shutdown_connections 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.793+0000 7f32f3f93740 1 --2- 172.21.15.5:0/1066675347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd76f370 0x55cabd771760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.793+0000 7f32f3f93740 1 --2- 172.21.15.5:0/1066675347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd764a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.793+0000 7f32f3f93740 1 --2- 172.21.15.5:0/1066675347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd764090 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.793+0000 7f32f3f93740 1 -- 172.21.15.5:0/1066675347 >> 172.21.15.5:0/1066675347 conn(0x55cabd72d310 msgr2=0x55cabd760c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.793+0000 7f32f3f93740 1 -- 172.21.15.5:0/1066675347 shutdown_connections 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.793+0000 7f32f3f93740 1 -- 172.21.15.5:0/1066675347 wait complete. 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32f3f93740 1 -- start start 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32f3f93740 1 Processor -- start 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd7645d0 0x55cabd76ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 0x55cabd76bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd76f370 0x55cabd76c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32f3f93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cabd772010 con 0x55cabd7645d0 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32f3f93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cabd685060 con 0x55cabd765b40 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32f3f93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cabd772190 con 0x55cabd76f370 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32f3f93740 10 client.0 _ll_register_callbacks cb 0x55cabd3d88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd7645d0 0x55cabd76ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd7645d0 0x55cabd76ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33932/0 (socket says 172.21.15.5:33932) 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32eb7fe640 1 -- 172.21.15.5:0/3107378714 learned_addr learned my addr 172.21.15.5:0/3107378714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.794+0000 7f32ebfff640 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd76f370 0x55cabd76c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32eaffd640 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 0x55cabd76bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32eb7fe640 1 -- 172.21.15.5:0/3107378714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd76f370 msgr2=0x55cabd76c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32eb7fe640 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd76f370 0x55cabd76c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32eb7fe640 1 -- 172.21.15.5:0/3107378714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 msgr2=0x55cabd76bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32eb7fe640 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 0x55cabd76bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32eb7fe640 1 -- 172.21.15.5:0/3107378714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabd43b3c0 con 0x55cabd7645d0 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32eb7fe640 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd7645d0 0x55cabd76ec90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f32dc005940 tx=0x7f32dc005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2df0b647756739 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32caffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32caffd640 1 -- 172.21.15.5:0/3107378714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc040070 con 0x55cabd7645d0 2024-09-11T22:24:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32caffd640 1 -- 172.21.15.5:0/3107378714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f32dc004050 con 0x55cabd7645d0 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32f3f93740 1 -- 172.21.15.5:0/3107378714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32dc031870 con 0x55cabd7645d0 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32caffd640 1 -- 172.21.15.5:0/3107378714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f32dc056050 con 0x55cabd7645d0 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32f3f93740 10 client.16221 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32f3f93740 1 -- 172.21.15.5:0/3107378714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cabd77d9d0 con 0x55cabd7645d0 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.795+0000 7f32caffd640 1 -- 172.21.15.5:0/3107378714 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32dc031870 con 0x55cabd7645d0 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.796+0000 7f32caffd640 1 -- 172.21.15.5:0/3107378714 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f32dc0035f0 con 0x55cabd7645d0 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.796+0000 7f32f3f93740 10 client.16221 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.796+0000 7f32f3f93740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.796+0000 7f32f3f93740 20 client.16221 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.796+0000 7f32f3f93740 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cabd77ed20 0x55cabd781110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.796+0000 7f32f3f93740 4 client.16221 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.796+0000 7f32f3f93740 1 -- 172.21.15.5:0/3107378714 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cabd6d7860 con 0x55cabd77ed20 2024-09-11T22:24:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.796+0000 7f32eaffd640 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cabd77ed20 0x55cabd781110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.796+0000 7f32caffd640 10 client.16221 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.796+0000 7f32eaffd640 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cabd77ed20 0x55cabd781110 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4399cbd3bdc37193 server_cookie=e69125a8ec3e1c7e in_seq=0 out_seq=0 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32caffd640 1 -- 172.21.15.5:0/3107378714 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cabd6d7860 con 0x55cabd77ed20 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32caffd640 10 client.16221 handle_command_reply: tid=0 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 client.16221 shutdown 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 -- 172.21.15.5:0/3107378714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd7645d0 msgr2=0x55cabd76ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd7645d0 0x55cabd76ec90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f32dc005940 tx=0x7f32dc005360 comp rx=0 tx=0).stop 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 -- 172.21.15.5:0/3107378714 shutdown_connections 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cabd77ed20 0x55cabd781110 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd76f370 0x55cabd76c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 0x55cabd76bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 --2- 172.21.15.5:0/3107378714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd7645d0 0x55cabd76ec90 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32caffd640 0 client.16221 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 -- 172.21.15.5:0/3107378714 >> 172.21.15.5:0/3107378714 conn(0x55cabd72d310 msgr2=0x55cabd766960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 -- 172.21.15.5:0/3107378714 shutdown_connections 2024-09-11T22:24:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 1 -- 172.21.15.5:0/3107378714 wait complete. 2024-09-11T22:24:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.797+0000 7f32f3f93740 20 client.16221 trim_cache size 0 max 16384 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32f3f93740 1 Processor -- start 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32f3f93740 1 -- start start 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd77b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd7867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd76f370 0x55cabd786ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32f3f93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1afb70 con 0x55cabd765b40 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32f3f93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1b1770 con 0x55cabd76f370 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32f3f93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1b0240 con 0x55cabd7645d0 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd7867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32eb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd77b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd7867a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33940/0 (socket says 172.21.15.5:33940) 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32eaffd640 1 -- 172.21.15.5:0/2329420085 learned_addr learned my addr 172.21.15.5:0/2329420085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.811+0000 7f32ebfff640 1 --2- 172.21.15.5:0/2329420085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd76f370 0x55cabd786ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.812+0000 7f32eaffd640 1 -- 172.21.15.5:0/2329420085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 msgr2=0x55cabd77b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.812+0000 7f32eaffd640 1 --2- 172.21.15.5:0/2329420085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd77b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.812+0000 7f32eaffd640 1 -- 172.21.15.5:0/2329420085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd76f370 msgr2=0x55cabd786ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.812+0000 7f32eaffd640 1 --2- 172.21.15.5:0/2329420085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd76f370 0x55cabd786ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.812+0000 7f32eaffd640 1 -- 172.21.15.5:0/2329420085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cabd764d50 con 0x55cabd765b40 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.812+0000 7f32eaffd640 1 --2- 172.21.15.5:0/2329420085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd7867a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f32e00ccef0 tx=0x7f32e001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b789511e5236d7c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.812+0000 7f32d17fa640 1 -- 172.21.15.5:0/2329420085 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00b4e60 con 0x55cabd765b40 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.812+0000 7f32d17fa640 1 -- 172.21.15.5:0/2329420085 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e00b1070 con 0x55cabd765b40 2024-09-11T22:24:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.812+0000 7f32d17fa640 1 -- 172.21.15.5:0/2329420085 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e009b3c0 con 0x55cabd765b40 2024-09-11T22:24:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 -- 172.21.15.5:0/2329420085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 msgr2=0x55cabd7867a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 --2- 172.21.15.5:0/2329420085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd7867a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f32e00ccef0 tx=0x7f32e001a040 comp rx=0 tx=0).stop 2024-09-11T22:24:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 -- 172.21.15.5:0/2329420085 shutdown_connections 2024-09-11T22:24:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 --2- 172.21.15.5:0/2329420085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd76f370 0x55cabd786ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 --2- 172.21.15.5:0/2329420085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd765b40 0x55cabd7867a0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 --2- 172.21.15.5:0/2329420085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd77b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 -- 172.21.15.5:0/2329420085 >> 172.21.15.5:0/2329420085 conn(0x55cabd72d310 msgr2=0x55cabd76b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 -- 172.21.15.5:0/2329420085 shutdown_connections 2024-09-11T22:24:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 -- 172.21.15.5:0/2329420085 wait complete. 2024-09-11T22:24:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 -- start start 2024-09-11T22:24:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.813+0000 7f32f3f93740 1 Processor -- start 2024-09-11T22:24:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd7739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 0x55cabd773ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32f3f93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd76f370 0x55cabd7e8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32f3f93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1b0fe0 con 0x55cabd76f370 2024-09-11T22:24:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32f3f93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1b0910 con 0x55cabd765b40 2024-09-11T22:24:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32f3f93740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1adb50 con 0x55cabd7645d0 2024-09-11T22:24:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32f3f93740 10 client.0 _ll_register_callbacks cb 0x7f32e4011d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32ebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd76f370 0x55cabd7e8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32eb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd7739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32ebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd76f370 0x55cabd7e8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33954/0 (socket says 172.21.15.5:33954) 2024-09-11T22:24:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32eaffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 0x55cabd773ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32ebfff640 1 -- 172.21.15.5:0/2787333856 learned_addr learned my addr 172.21.15.5:0/2787333856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32eaffd640 1 -- 172.21.15.5:0/2787333856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 msgr2=0x55cabd7739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32eaffd640 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd7739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32eaffd640 1 -- 172.21.15.5:0/2787333856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd76f370 msgr2=0x55cabd7e8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32eaffd640 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd76f370 0x55cabd7e8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32eaffd640 1 -- 172.21.15.5:0/2787333856 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cabd7e8db0 con 0x55cabd765b40 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32ebfff640 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd76f370 0x55cabd7e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32eaffd640 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 0x55cabd773ee0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f32e00ccef0 tx=0x7f32e0005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b8c8e8143a1c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.814+0000 7f32caffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32caffd640 1 -- 172.21.15.5:0/2787333856 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e00b8070 con 0x55cabd765b40 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32f3f93740 1 -- 172.21.15.5:0/2787333856 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e00b1070 con 0x55cabd765b40 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32caffd640 1 -- 172.21.15.5:0/2787333856 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f32e009b160 con 0x55cabd765b40 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32caffd640 1 -- 172.21.15.5:0/2787333856 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f32e00b4e60 con 0x55cabd765b40 2024-09-11T22:24:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32f3f93740 10 client.25751 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32f3f93740 1 -- 172.21.15.5:0/2787333856 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cabd7ea090 con 0x55cabd765b40 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32caffd640 1 -- 172.21.15.5:0/2787333856 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32e00b1070 con 0x55cabd765b40 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32caffd640 1 -- 172.21.15.5:0/2787333856 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f32e00aa070 con 0x55cabd765b40 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32f3f93740 10 client.25751 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32f3f93740 10 client.25751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32f3f93740 20 client.25751 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32f3f93740 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cabd7ea730 0x55cabd7ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32f3f93740 4 client.25751 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32f3f93740 1 -- 172.21.15.5:0/2787333856 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cabd6d1d70 con 0x55cabd7ea730 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.815+0000 7f32eb7fe640 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cabd7ea730 0x55cabd7ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.816+0000 7f32eb7fe640 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cabd7ea730 0x55cabd7ecb50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94ef66228e7991c9 server_cookie=ab9fe6f46f6a9d69 in_seq=0 out_seq=0 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.816+0000 7f32caffd640 10 client.25751 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32caffd640 1 -- 172.21.15.5:0/2787333856 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cabd6d1d70 con 0x55cabd7ea730 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32caffd640 10 client.25751 handle_command_reply: tid=0 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 client.25751 shutdown 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 -- 172.21.15.5:0/2787333856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 msgr2=0x55cabd773ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 0x55cabd773ee0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f32e00ccef0 tx=0x7f32e0005cc0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 -- 172.21.15.5:0/2787333856 shutdown_connections 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cabd7ea730 0x55cabd7ecb50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cabd76f370 0x55cabd7e8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cabd765b40 0x55cabd773ee0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32caffd640 0 client.25751 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 --2- 172.21.15.5:0/2787333856 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cabd7645d0 0x55cabd7739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 -- 172.21.15.5:0/2787333856 >> 172.21.15.5:0/2787333856 conn(0x55cabd72d310 msgr2=0x55cabd75f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 -- 172.21.15.5:0/2787333856 shutdown_connections 2024-09-11T22:24:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 1 -- 172.21.15.5:0/2787333856 wait complete. 2024-09-11T22:24:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.817+0000 7f32f3f93740 20 client.25751 trim_cache size 0 max 16384 2024-09-11T22:24:46.832 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:24:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 -- 172.21.15.5:0/926501511 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f32d40a2cf0 msgr2=0x7f32d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f32d40a2cf0 0x7f32d40a51b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f32e4005fd0 tx=0x7f32e4011b40 comp rx=0 tx=0).stop 2024-09-11T22:24:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 -- 172.21.15.5:0/926501511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ec102790 msgr2=0x7f32ec074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ec102790 0x7f32ec074a30 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f32e000e9f0 tx=0x7f32e000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 -- 172.21.15.5:0/926501511 shutdown_connections 2024-09-11T22:24:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f32d40a2cf0 0x7f32d40a51b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ec106f10 0x7f32ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ec102790 0x7f32ec074a30 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 --2- 172.21.15.5:0/926501511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec101de0 0x7f32ec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 -- 172.21.15.5:0/926501511 >> 172.21.15.5:0/926501511 conn(0x7f32ec0fd8c0 msgr2=0x7f32ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:46.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 -- 172.21.15.5:0/926501511 shutdown_connections 2024-09-11T22:24:46.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:46.818+0000 7f32f1bba640 1 -- 172.21.15.5:0/926501511 wait complete. 2024-09-11T22:24:47.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 140835 11.91 MB/sec warmup 65 sec latency 33.767 ms 2024-09-11T22:24:48.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:47 smithi005 ceph-mon[30943]: pgmap v80: 97 pgs: 97 active+clean; 38 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-11T22:24:48.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:47 smithi172 ceph-mon[35461]: pgmap v80: 97 pgs: 97 active+clean; 38 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-11T22:24:48.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 142970 11.94 MB/sec warmup 66 sec latency 14.631 ms 2024-09-11T22:24:48.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:47 smithi159 ceph-mon[35534]: pgmap v80: 97 pgs: 97 active+clean; 38 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-11T22:24:48.652 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:24:48.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:24:48.914 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:49.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 145718 12.01 MB/sec warmup 67 sec latency 44.861 ms 2024-09-11T22:24:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.665+0000 7f961dceb640 1 -- 172.21.15.5:0/1991537308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9618100b30 msgr2=0x7f9618100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.665+0000 7f961dceb640 1 --2- 172.21.15.5:0/1991537308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9618100b30 0x7f9618100f10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9608009900 tx=0x7f960802a470 comp rx=0 tx=0).stop 2024-09-11T22:24:49.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 -- 172.21.15.5:0/1991537308 shutdown_connections 2024-09-11T22:24:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 --2- 172.21.15.5:0/1991537308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9618105f40 0x7f96181063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 --2- 172.21.15.5:0/1991537308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9618101450 0x7f9618105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 --2- 172.21.15.5:0/1991537308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9618100b30 0x7f9618100f10 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 -- 172.21.15.5:0/1991537308 >> 172.21.15.5:0/1991537308 conn(0x7f96180fc7a0 msgr2=0x7f96180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 -- 172.21.15.5:0/1991537308 shutdown_connections 2024-09-11T22:24:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 -- 172.21.15.5:0/1991537308 wait complete. 2024-09-11T22:24:49.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 Processor -- start 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 -- start start 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9618100b30 0x7f96181a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9618101450 0x7f96181a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9618105f40 0x7f961819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9618118570 con 0x7f9618105f40 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f96181183f0 con 0x7f9618101450 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.666+0000 7f961dceb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96181186f0 con 0x7f9618100b30 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f9617fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9618105f40 0x7f961819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f9617fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9618105f40 0x7f961819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33970/0 (socket says 172.21.15.5:33970) 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f960ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9618101450 0x7f96181a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f9617fff640 1 -- 172.21.15.5:0/2020114873 learned_addr learned my addr 172.21.15.5:0/2020114873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f96177fe640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9618100b30 0x7f96181a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f9617fff640 1 -- 172.21.15.5:0/2020114873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9618100b30 msgr2=0x7f96181a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f9617fff640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9618100b30 0x7f96181a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f9617fff640 1 -- 172.21.15.5:0/2020114873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9618101450 msgr2=0x7f96181a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f9617fff640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9618101450 0x7f96181a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f9617fff640 1 -- 172.21.15.5:0/2020114873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961819f050 con 0x7f9618105f40 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f9617fff640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9618105f40 0x7f961819e920 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f961000eb10 tx=0x7f961000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f96157fa640 1 -- 172.21.15.5:0/2020114873 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610004b20 con 0x7f9618105f40 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f961dceb640 1 -- 172.21.15.5:0/2020114873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961819f340 con 0x7f9618105f40 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f96157fa640 1 -- 172.21.15.5:0/2020114873 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9610004cc0 con 0x7f9618105f40 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f96157fa640 1 -- 172.21.15.5:0/2020114873 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9610010760 con 0x7f9618105f40 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.667+0000 7f961dceb640 1 -- 172.21.15.5:0/2020114873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96181ab630 con 0x7f9618105f40 2024-09-11T22:24:49.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.670+0000 7f96157fa640 1 -- 172.21.15.5:0/2020114873 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9610005020 con 0x7f9618105f40 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.670+0000 7f96200c4740 1 Processor -- start 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.670+0000 7f96200c4740 1 -- start start 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.670+0000 7f96200c4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x56187318df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.670+0000 7f96200c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x561873198800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.670+0000 7f96200c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561873198d40 0x56187319b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.670+0000 7f96200c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618730f3280 con 0x56187318e480 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.670+0000 7f96200c4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56187309ab10 con 0x56187318db60 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.670+0000 7f96200c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5618730ef240 con 0x561873198d40 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f96157fa640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f96040a2cf0 0x7f96040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f96177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x561873198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f960ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x56187318df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f96177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x561873198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33978/0 (socket says 172.21.15.5:33978) 2024-09-11T22:24:49.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f960ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x56187318df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48588/0 (socket says 172.21.15.5:48588) 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f96177fe640 1 -- 172.21.15.5:0/4190436573 learned_addr learned my addr 172.21.15.5:0/4190436573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f960ffff640 1 --2- 172.21.15.5:0/4190436573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561873198d40 0x56187319b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f9617fff640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f96040a2cf0 0x7f96040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f96157fa640 1 -- 172.21.15.5:0/2020114873 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f96100cf980 con 0x7f9618105f40 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f960ffff640 1 -- 172.21.15.5:0/4190436573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561873198d40 msgr2=0x56187319b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f960ffff640 1 --2- 172.21.15.5:0/4190436573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561873198d40 0x56187319b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f960ffff640 1 -- 172.21.15.5:0/4190436573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 msgr2=0x561873198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f960ffff640 1 --2- 172.21.15.5:0/4190436573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x561873198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f960ffff640 1 -- 172.21.15.5:0/4190436573 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618730a71f0 con 0x56187318db60 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f96177fe640 1 --2- 172.21.15.5:0/4190436573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x561873198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f960ffff640 1 --2- 172.21.15.5:0/4190436573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x56187318df40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f96180fff70 tx=0x7f9600004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a21d316fccc782a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:49.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f9617fff640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f96040a2cf0 0x7f96040a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f961007dfd0 tx=0x7f961007f040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.671+0000 7f960dffb640 1 -- 172.21.15.5:0/4190436573 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600004050 con 0x56187318db60 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.672+0000 7f960dffb640 1 -- 172.21.15.5:0/4190436573 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f960000fc70 con 0x56187318db60 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.672+0000 7f960dffb640 1 -- 172.21.15.5:0/4190436573 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600003b30 con 0x56187318db60 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.672+0000 7f96200c4740 1 -- 172.21.15.5:0/4190436573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 msgr2=0x56187318df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.672+0000 7f96200c4740 1 --2- 172.21.15.5:0/4190436573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x56187318df40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f96180fff70 tx=0x7f9600004730 comp rx=0 tx=0).stop 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.673+0000 7f96200c4740 1 -- 172.21.15.5:0/4190436573 shutdown_connections 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.673+0000 7f96200c4740 1 --2- 172.21.15.5:0/4190436573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561873198d40 0x56187319b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.673+0000 7f96200c4740 1 --2- 172.21.15.5:0/4190436573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x561873198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.673+0000 7f96200c4740 1 --2- 172.21.15.5:0/4190436573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x56187318df40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.673+0000 7f96200c4740 1 -- 172.21.15.5:0/4190436573 >> 172.21.15.5:0/4190436573 conn(0x56187318a0f0 msgr2=0x56187318a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.673+0000 7f96200c4740 1 -- 172.21.15.5:0/4190436573 shutdown_connections 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.673+0000 7f96200c4740 1 -- 172.21.15.5:0/4190436573 wait complete. 2024-09-11T22:24:49.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.673+0000 7f96200c4740 1 -- start start 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.673+0000 7f96200c4740 1 Processor -- start 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96200c4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x561873195c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96200c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x561873196190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96200c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561873198d40 0x5618731966d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96200c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56187319b910 con 0x561873198d40 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96200c4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5618730adf10 con 0x56187318db60 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96200c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56187319ba90 con 0x56187318e480 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96200c4740 10 client.0 _ll_register_callbacks cb 0x561873076a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x561873196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f960ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561873198d40 0x5618731966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x561873196190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:34304/0 (socket says 172.21.15.5:34304) 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f960ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x561873195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96177fe640 1 -- 172.21.15.5:0/1315763035 learned_addr learned my addr 172.21.15.5:0/1315763035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96177fe640 1 -- 172.21.15.5:0/1315763035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 msgr2=0x561873195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96177fe640 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x561873195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96177fe640 1 -- 172.21.15.5:0/1315763035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561873198d40 msgr2=0x5618731966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96177fe640 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561873198d40 0x5618731966d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96177fe640 1 -- 172.21.15.5:0/1315763035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561872e646a0 con 0x56187318e480 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f96177fe640 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x561873196190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9608005940 tx=0x7f96080317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=410e30d5111865e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f95f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.674+0000 7f95f6ffd640 1 -- 172.21.15.5:0/1315763035 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608035040 con 0x56187318e480 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f96200c4740 1 -- 172.21.15.5:0/1315763035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f960000fc70 con 0x56187318e480 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f95f6ffd640 1 -- 172.21.15.5:0/1315763035 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f9608004250 con 0x56187318e480 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f95f6ffd640 1 -- 172.21.15.5:0/1315763035 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9608004450 con 0x56187318e480 2024-09-11T22:24:49.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f96200c4740 10 client.25708 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f96200c4740 1 -- 172.21.15.5:0/1315763035 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618731a7590 con 0x56187318e480 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f95f6ffd640 1 -- 172.21.15.5:0/1315763035 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f960000fc70 con 0x56187318e480 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f95f6ffd640 1 -- 172.21.15.5:0/1315763035 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f96080035f0 con 0x56187318e480 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f96200c4740 10 client.25708 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f96200c4740 10 client.25708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f96200c4740 20 client.25708 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f96200c4740 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618731a8a20 0x5618731aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f96200c4740 4 client.25708 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f96200c4740 1 -- 172.21.15.5:0/1315763035 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618731007f0 con 0x5618731a8a20 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.675+0000 7f960ffff640 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618731a8a20 0x5618731aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.676+0000 7f95f6ffd640 10 client.25708 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.676+0000 7f960ffff640 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618731a8a20 0x5618731aae40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee11c696e9c672fc server_cookie=484f09a6d8ca4e65 in_seq=0 out_seq=0 2024-09-11T22:24:49.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f95f6ffd640 1 -- 172.21.15.5:0/1315763035 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618731007f0 con 0x5618731a8a20 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f95f6ffd640 10 client.25708 handle_command_reply: tid=0 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 client.25708 shutdown 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 -- 172.21.15.5:0/1315763035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 msgr2=0x561873196190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x561873196190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9608005940 tx=0x7f96080317e0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 -- 172.21.15.5:0/1315763035 shutdown_connections 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618731a8a20 0x5618731aae40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561873198d40 0x5618731966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x561873196190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 --2- 172.21.15.5:0/1315763035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x561873195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f95f6ffd640 0 client.25708 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 -- 172.21.15.5:0/1315763035 >> 172.21.15.5:0/1315763035 conn(0x56187318a0f0 msgr2=0x56187318fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 -- 172.21.15.5:0/1315763035 shutdown_connections 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 1 -- 172.21.15.5:0/1315763035 wait complete. 2024-09-11T22:24:49.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.680+0000 7f96200c4740 20 client.25708 trim_cache size 0 max 16384 2024-09-11T22:24:49.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96200c4740 1 Processor -- start 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96200c4740 1 -- start start 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96200c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318db60 0x561873196520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96200c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x5618731b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96200c4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561873198d40 0x5618731b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96200c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b4060 con 0x56187318db60 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96200c4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b5c60 con 0x561873198d40 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96200c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b4730 con 0x56187318e480 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x5618731b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f960ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318db60 0x561873196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96177fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x5618731b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:34306/0 (socket says 172.21.15.5:34306) 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96177fe640 1 -- 172.21.15.5:0/3708003790 learned_addr learned my addr 172.21.15.5:0/3708003790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:49.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f960ffff640 1 --2- 172.21.15.5:0/3708003790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561873198d40 0x5618731b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96177fe640 1 -- 172.21.15.5:0/3708003790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561873198d40 msgr2=0x5618731b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.701+0000 7f96177fe640 1 --2- 172.21.15.5:0/3708003790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561873198d40 0x5618731b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.702+0000 7f96177fe640 1 -- 172.21.15.5:0/3708003790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318db60 msgr2=0x561873196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.702+0000 7f96177fe640 1 --2- 172.21.15.5:0/3708003790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318db60 0x561873196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.702+0000 7f96177fe640 1 -- 172.21.15.5:0/3708003790 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618731ab1b0 con 0x56187318e480 2024-09-11T22:24:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.702+0000 7f960ffff640 1 --2- 172.21.15.5:0/3708003790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318db60 0x561873196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.702+0000 7f96177fe640 1 --2- 172.21.15.5:0/3708003790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x5618731b06f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9608003bb0 tx=0x7f9608009230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff5d48208ce97a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.702+0000 7f960d7fa640 1 -- 172.21.15.5:0/3708003790 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608035070 con 0x56187318e480 2024-09-11T22:24:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.702+0000 7f960d7fa640 1 -- 172.21.15.5:0/3708003790 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9608041070 con 0x56187318e480 2024-09-11T22:24:49.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.702+0000 7f960d7fa640 1 -- 172.21.15.5:0/3708003790 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608004da0 con 0x56187318e480 2024-09-11T22:24:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.703+0000 7f96200c4740 1 -- 172.21.15.5:0/3708003790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 msgr2=0x5618731b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.703+0000 7f96200c4740 1 --2- 172.21.15.5:0/3708003790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x5618731b06f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9608003bb0 tx=0x7f9608009230 comp rx=0 tx=0).stop 2024-09-11T22:24:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.703+0000 7f96200c4740 1 -- 172.21.15.5:0/3708003790 shutdown_connections 2024-09-11T22:24:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.703+0000 7f96200c4740 1 --2- 172.21.15.5:0/3708003790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561873198d40 0x5618731b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.703+0000 7f96200c4740 1 --2- 172.21.15.5:0/3708003790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56187318e480 0x5618731b06f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.703+0000 7f96200c4740 1 --2- 172.21.15.5:0/3708003790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318db60 0x561873196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.703+0000 7f96200c4740 1 -- 172.21.15.5:0/3708003790 >> 172.21.15.5:0/3708003790 conn(0x56187318a0f0 msgr2=0x561873197bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.703+0000 7f96200c4740 1 -- 172.21.15.5:0/3708003790 shutdown_connections 2024-09-11T22:24:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.703+0000 7f96200c4740 1 -- 172.21.15.5:0/3708003790 wait complete. 2024-09-11T22:24:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.703+0000 7f96200c4740 1 -- start start 2024-09-11T22:24:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96200c4740 1 Processor -- start 2024-09-11T22:24:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96200c4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x5618731a1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96200c4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x56187319c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96200c4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561873198d40 0x56187319cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96200c4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b54d0 con 0x56187318e480 2024-09-11T22:24:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96200c4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b4e00 con 0x56187318db60 2024-09-11T22:24:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96200c4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f96181b2040 con 0x561873198d40 2024-09-11T22:24:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96200c4740 10 client.0 _ll_register_callbacks cb 0x7f9608009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x56187319c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x56187319c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33988/0 (socket says 172.21.15.5:33988) 2024-09-11T22:24:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f96177fe640 1 -- 172.21.15.5:0/4052948272 learned_addr learned my addr 172.21.15.5:0/4052948272 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f960ffff640 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x5618731a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.704+0000 7f960ffff640 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561873198d40 0x56187319cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f96177fe640 1 -- 172.21.15.5:0/4052948272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561873198d40 msgr2=0x56187319cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:49.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f96177fe640 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561873198d40 0x56187319cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f96177fe640 1 -- 172.21.15.5:0/4052948272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 msgr2=0x5618731a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f96177fe640 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x5618731a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f96177fe640 1 -- 172.21.15.5:0/4052948272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56187319d600 con 0x56187318e480 2024-09-11T22:24:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f96177fe640 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x56187319c830 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9608003b80 tx=0x7f9608003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b42d00c71c7dce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f95f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f95f6ffd640 1 -- 172.21.15.5:0/4052948272 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608035040 con 0x56187318e480 2024-09-11T22:24:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f96200c4740 1 -- 172.21.15.5:0/4052948272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9608041070 con 0x56187318e480 2024-09-11T22:24:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f95f6ffd640 1 -- 172.21.15.5:0/4052948272 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f96080046c0 con 0x56187318e480 2024-09-11T22:24:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f95f6ffd640 1 -- 172.21.15.5:0/4052948272 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9608065050 con 0x56187318e480 2024-09-11T22:24:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f96200c4740 10 client.16245 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.705+0000 7f96200c4740 1 -- 172.21.15.5:0/4052948272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5618732137d0 con 0x56187318e480 2024-09-11T22:24:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f95f6ffd640 1 -- 172.21.15.5:0/4052948272 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9608041070 con 0x56187318e480 2024-09-11T22:24:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f95f6ffd640 1 -- 172.21.15.5:0/4052948272 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f960805d020 con 0x56187318e480 2024-09-11T22:24:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f96200c4740 10 client.16245 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f96200c4740 10 client.16245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f96200c4740 20 client.16245 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f96200c4740 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561873213ec0 0x5618732162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f96200c4740 4 client.16245 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:24:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f96200c4740 1 -- 172.21.15.5:0/4052948272 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5618730fad00 con 0x561873213ec0 2024-09-11T22:24:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f960ffff640 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561873213ec0 0x5618732162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f960ffff640 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561873213ec0 0x5618732162e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67f18e18a7c0231 server_cookie=b392efb29924667c in_seq=0 out_seq=0 2024-09-11T22:24:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.706+0000 7f95f6ffd640 10 client.16245 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:49.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.707+0000 7f95f6ffd640 1 -- 172.21.15.5:0/4052948272 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5618730fad00 con 0x561873213ec0 2024-09-11T22:24:49.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.707+0000 7f95f6ffd640 10 client.16245 handle_command_reply: tid=0 2024-09-11T22:24:49.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.707+0000 7f96200c4740 1 client.16245 shutdown 2024-09-11T22:24:49.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 1 -- 172.21.15.5:0/4052948272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 msgr2=0x56187319c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:49.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x56187319c830 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9608003b80 tx=0x7f9608003820 comp rx=0 tx=0).stop 2024-09-11T22:24:49.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 1 -- 172.21.15.5:0/4052948272 shutdown_connections 2024-09-11T22:24:49.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561873213ec0 0x5618732162e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561873198d40 0x56187319cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56187318e480 0x56187319c830 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 1 --2- 172.21.15.5:0/4052948272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56187318db60 0x5618731a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f95f6ffd640 0 client.16245 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 1 -- 172.21.15.5:0/4052948272 >> 172.21.15.5:0/4052948272 conn(0x56187318a0f0 msgr2=0x56187318f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 1 -- 172.21.15.5:0/4052948272 shutdown_connections 2024-09-11T22:24:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 1 -- 172.21.15.5:0/4052948272 wait complete. 2024-09-11T22:24:49.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f96200c4740 20 client.16245 trim_cache size 0 max 16384 2024-09-11T22:24:49.715 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:24:49.715 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:24:49.716 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "53ce156a-839f-4fbb-acef-8deee03510b5", 2024-09-11T22:24:49.716 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:24:49.716 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:24:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f961dceb640 1 -- 172.21.15.5:0/2020114873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f96040a2cf0 msgr2=0x7f96040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f961dceb640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f96040a2cf0 0x7f96040a51b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f961007dfd0 tx=0x7f961007f040 comp rx=0 tx=0).stop 2024-09-11T22:24:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f961dceb640 1 -- 172.21.15.5:0/2020114873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9618105f40 msgr2=0x7f961819e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:49.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.708+0000 7f961dceb640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9618105f40 0x7f961819e920 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f961000eb10 tx=0x7f961000c2d0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.709+0000 7f961dceb640 1 -- 172.21.15.5:0/2020114873 shutdown_connections 2024-09-11T22:24:49.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.709+0000 7f961dceb640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9618105f40 0x7f961819e920 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.709+0000 7f961dceb640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f96040a2cf0 0x7f96040a51b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.709+0000 7f961dceb640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9618101450 0x7f96181a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.709+0000 7f961dceb640 1 --2- 172.21.15.5:0/2020114873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9618100b30 0x7f96181a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:49.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.709+0000 7f961dceb640 1 -- 172.21.15.5:0/2020114873 >> 172.21.15.5:0/2020114873 conn(0x7f96180fc7a0 msgr2=0x7f96181038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:49.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.709+0000 7f961dceb640 1 -- 172.21.15.5:0/2020114873 shutdown_connections 2024-09-11T22:24:49.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:49.709+0000 7f961dceb640 1 -- 172.21.15.5:0/2020114873 wait complete. 2024-09-11T22:24:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:49 smithi005 ceph-mon[30943]: pgmap v81: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-09-11T22:24:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:49 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:24:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:49 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:49 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:24:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:49 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:24:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:49 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:49 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:49 smithi172 ceph-mon[35461]: pgmap v81: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-09-11T22:24:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:49 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:24:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:49 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:49 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:24:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:49 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:24:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:49 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:49 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:50.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 148336 12.00 MB/sec warmup 68 sec latency 42.899 ms 2024-09-11T22:24:50.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:49 smithi159 ceph-mon[35534]: pgmap v81: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-09-11T22:24:50.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:49 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:24:50.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:49 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:50.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:49 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:24:50.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:49 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:24:50.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:49 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:50.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:49 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:24:50.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:24:50.905 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:24:51.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 150530 12.02 MB/sec warmup 69 sec latency 21.500 ms 2024-09-11T22:24:51.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.426+0000 7f1705f12640 1 -- 172.21.15.5:0/1359967609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1700102ef0 msgr2=0x7f170010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:51.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.426+0000 7f1705f12640 1 --2- 172.21.15.5:0/1359967609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1700102ef0 0x7f170010da40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f16f8009ca0 tx=0x7f16f802a950 comp rx=0 tx=0).stop 2024-09-11T22:24:51.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.426+0000 7f1705f12640 1 -- 172.21.15.5:0/1359967609 shutdown_connections 2024-09-11T22:24:51.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.426+0000 7f1705f12640 1 --2- 172.21.15.5:0/1359967609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1700102ef0 0x7f170010da40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.426+0000 7f1705f12640 1 --2- 172.21.15.5:0/1359967609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1700102570 0x7f17001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.426+0000 7f1705f12640 1 --2- 172.21.15.5:0/1359967609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1700100e20 0x7f1700101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.426+0000 7f1705f12640 1 -- 172.21.15.5:0/1359967609 >> 172.21.15.5:0/1359967609 conn(0x7f17000fc7d0 msgr2=0x7f17000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.426+0000 7f1705f12640 1 -- 172.21.15.5:0/1359967609 shutdown_connections 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.426+0000 7f1705f12640 1 -- 172.21.15.5:0/1359967609 wait complete. 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.427+0000 7f1705f12640 1 Processor -- start 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.427+0000 7f1705f12640 1 -- start start 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.427+0000 7f1705f12640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1700100e20 0x7f1700198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.427+0000 7f1705f12640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1700102570 0x7f1700198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.427+0000 7f1705f12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1700102ef0 0x7f170019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f1705f12640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1700110f80 con 0x7f1700102ef0 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f1705f12640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1700110e00 con 0x7f1700100e20 2024-09-11T22:24:51.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f1705f12640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1700111100 con 0x7f1700102570 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1700102ef0 0x7f170019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16ff7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1700100e20 0x7f1700198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16feffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1700102570 0x7f1700198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1700102ef0 0x7f170019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37234/0 (socket says 172.21.15.5:37234) 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fffff640 1 -- 172.21.15.5:0/3377575327 learned_addr learned my addr 172.21.15.5:0/3377575327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fffff640 1 -- 172.21.15.5:0/3377575327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1700102570 msgr2=0x7f1700198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fffff640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1700102570 0x7f1700198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fffff640 1 -- 172.21.15.5:0/3377575327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1700100e20 msgr2=0x7f1700198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fffff640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1700100e20 0x7f1700198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fffff640 1 -- 172.21.15.5:0/3377575327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170019d4b0 con 0x7f1700102ef0 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1700100e20 0x7f1700198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fffff640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1700102ef0 0x7f170019cdd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f16f8004af0 tx=0x7f16f802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:51.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fcff9640 1 -- 172.21.15.5:0/3377575327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f8034040 con 0x7f1700102ef0 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f1705f12640 1 -- 172.21.15.5:0/3377575327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170019d740 con 0x7f1700102ef0 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fcff9640 1 -- 172.21.15.5:0/3377575327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f8030920 con 0x7f1700102ef0 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.428+0000 7f16fcff9640 1 -- 172.21.15.5:0/3377575327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f8004630 con 0x7f1700102ef0 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.429+0000 7f1705f12640 1 -- 172.21.15.5:0/3377575327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f170019dbf0 con 0x7f1700102ef0 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.431+0000 7f16fcff9640 1 -- 172.21.15.5:0/3377575327 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f16f8037020 con 0x7f1700102ef0 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.431+0000 7f17082eb740 1 Processor -- start 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.431+0000 7f17082eb740 1 -- start start 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.431+0000 7f17082eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5df880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.431+0000 7f17082eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5ea140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.431+0000 7f17082eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5ea680 0x5559df5eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.431+0000 7f17082eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559df5452e0 con 0x5559df5df4a0 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.431+0000 7f17082eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5559df4ecce0 con 0x5559df5dfdc0 2024-09-11T22:24:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.431+0000 7f17082eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5559df5412a0 con 0x5559df5ea680 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.431+0000 7f16fcff9640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f16e80a2cf0 0x7f16e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5df880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37246/0 (socket says 172.21.15.5:37246) 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16feffd640 1 -- 172.21.15.5:0/3667031610 learned_addr learned my addr 172.21.15.5:0/3667031610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/3667031610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16fffff640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f16e80a2cf0 0x7f16e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16feffd640 1 --2- 172.21.15.5:0/3667031610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5ea680 0x5559df5eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16fcff9640 1 -- 172.21.15.5:0/3377575327 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f16f80f0130 con 0x7f1700102ef0 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16ff7fe640 1 -- 172.21.15.5:0/3667031610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5ea680 msgr2=0x5559df5eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/3667031610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5ea680 0x5559df5eca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16ff7fe640 1 -- 172.21.15.5:0/3667031610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 msgr2=0x5559df5df880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/3667031610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5df880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16ff7fe640 1 -- 172.21.15.5:0/3667031610 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559df4d1440 con 0x5559df5dfdc0 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16feffd640 1 --2- 172.21.15.5:0/3667031610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16fffff640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f16e80a2cf0 0x7f16e80a51b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16f80b90f0 tx=0x7f16f808e6e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/3667031610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5ea140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009fd0 tx=0x7f16f00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d554ba9305fa92ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.432+0000 7f16e5ffb640 1 -- 172.21.15.5:0/3667031610 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0004030 con 0x5559df5dfdc0 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f16e5ffb640 1 -- 172.21.15.5:0/3667031610 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f0005500 con 0x5559df5dfdc0 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f16e5ffb640 1 -- 172.21.15.5:0/3667031610 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0004d40 con 0x5559df5dfdc0 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f17082eb740 1 -- 172.21.15.5:0/3667031610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 msgr2=0x5559df5ea140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f17082eb740 1 --2- 172.21.15.5:0/3667031610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5ea140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009fd0 tx=0x7f16f00118d0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f17082eb740 1 -- 172.21.15.5:0/3667031610 shutdown_connections 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f17082eb740 1 --2- 172.21.15.5:0/3667031610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5ea680 0x5559df5eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f17082eb740 1 --2- 172.21.15.5:0/3667031610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5ea140 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f17082eb740 1 --2- 172.21.15.5:0/3667031610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f17082eb740 1 -- 172.21.15.5:0/3667031610 >> 172.21.15.5:0/3667031610 conn(0x5559df5dba30 msgr2=0x5559df5dbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f17082eb740 1 -- 172.21.15.5:0/3667031610 shutdown_connections 2024-09-11T22:24:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.433+0000 7f17082eb740 1 -- 172.21.15.5:0/3667031610 wait complete. 2024-09-11T22:24:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f17082eb740 1 -- start start 2024-09-11T22:24:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f17082eb740 1 Processor -- start 2024-09-11T22:24:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f17082eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5e9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f17082eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f17082eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5ea680 0x5559df5e7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f17082eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559df5ed250 con 0x5559df5df4a0 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f17082eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5559df500050 con 0x5559df5dfdc0 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f17082eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5559df5ed3d0 con 0x5559df5ea680 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f17082eb740 10 client.0 _ll_register_callbacks cb 0x5559df540160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f16feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5e9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f16ff7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f16feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5e9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37250/0 (socket says 172.21.15.5:37250) 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.434+0000 7f16feffd640 1 -- 172.21.15.5:0/1325788135 learned_addr learned my addr 172.21.15.5:0/1325788135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16feffd640 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5ea680 0x5559df5e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16feffd640 1 -- 172.21.15.5:0/1325788135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5ea680 msgr2=0x5559df5e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16feffd640 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5ea680 0x5559df5e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16feffd640 1 -- 172.21.15.5:0/1325788135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 msgr2=0x5559df5e71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16feffd640 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5e71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16feffd640 1 -- 172.21.15.5:0/1325788135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559df2b63c0 con 0x5559df5df4a0 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5e71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16feffd640 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5e9f20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f16f4013410 tx=0x7f16f4009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9455f860e067de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16deffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16deffd640 1 -- 172.21.15.5:0/1325788135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4011d90 con 0x5559df5df4a0 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16deffd640 1 -- 172.21.15.5:0/1325788135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f16f4004050 con 0x5559df5df4a0 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f17082eb740 1 -- 172.21.15.5:0/1325788135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f0005500 con 0x5559df5df4a0 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16deffd640 1 -- 172.21.15.5:0/1325788135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16f4029050 con 0x5559df5df4a0 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f17082eb740 10 client.16257 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f17082eb740 1 -- 172.21.15.5:0/1325788135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559df5f8bd0 con 0x5559df5df4a0 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.435+0000 7f16deffd640 1 -- 172.21.15.5:0/1325788135 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f0005500 con 0x5559df5df4a0 2024-09-11T22:24:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.436+0000 7f16deffd640 1 -- 172.21.15.5:0/1325788135 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f16f400f3c0 con 0x5559df5df4a0 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.436+0000 7f17082eb740 10 client.16257 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.436+0000 7f17082eb740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.436+0000 7f17082eb740 20 client.16257 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.436+0000 7f17082eb740 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5559df5fa0e0 0x5559df5fc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.436+0000 7f17082eb740 4 client.16257 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.436+0000 7f17082eb740 1 -- 172.21.15.5:0/1325788135 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559df552850 con 0x5559df5fa0e0 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.436+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5559df5fa0e0 0x5559df5fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.436+0000 7f16deffd640 10 client.16257 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.436+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5559df5fa0e0 0x5559df5fc540 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff5d2c247e894820 server_cookie=35937f3776bce6cc in_seq=0 out_seq=0 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f16deffd640 1 -- 172.21.15.5:0/1325788135 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5559df552850 con 0x5559df5fa0e0 2024-09-11T22:24:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f16deffd640 10 client.16257 handle_command_reply: tid=0 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f17082eb740 1 client.16257 shutdown 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f17082eb740 1 -- 172.21.15.5:0/1325788135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 msgr2=0x5559df5e9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f17082eb740 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5e9f20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f16f4013410 tx=0x7f16f4009c10 comp rx=0 tx=0).stop 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f17082eb740 1 -- 172.21.15.5:0/1325788135 shutdown_connections 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f17082eb740 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5559df5fa0e0 0x5559df5fc540 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f17082eb740 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5ea680 0x5559df5e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f17082eb740 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5e71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f16deffd640 0 client.16257 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f17082eb740 1 --2- 172.21.15.5:0/1325788135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5e9f20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.438+0000 7f17082eb740 1 -- 172.21.15.5:0/1325788135 >> 172.21.15.5:0/1325788135 conn(0x5559df5dba30 msgr2=0x5559df5dab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.439+0000 7f17082eb740 1 -- 172.21.15.5:0/1325788135 shutdown_connections 2024-09-11T22:24:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.439+0000 7f17082eb740 1 -- 172.21.15.5:0/1325788135 wait complete. 2024-09-11T22:24:51.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.439+0000 7f17082eb740 20 client.16257 trim_cache size 0 max 16384 2024-09-11T22:24:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f17082eb740 1 Processor -- start 2024-09-11T22:24:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f17082eb740 1 -- start start 2024-09-11T22:24:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f17082eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5df4a0 0x5559df5f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f17082eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f17082eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5ea680 0x5559df5e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f17082eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17001afc10 con 0x5559df5ea680 2024-09-11T22:24:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f17082eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17001b1810 con 0x5559df5dfdc0 2024-09-11T22:24:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f17082eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17001b02e0 con 0x5559df5df4a0 2024-09-11T22:24:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f16feffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5df4a0 0x5559df5f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f16feffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5df4a0 0x5559df5f64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47820/0 (socket says 172.21.15.5:47820) 2024-09-11T22:24:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f16feffd640 1 -- 172.21.15.5:0/806155216 learned_addr learned my addr 172.21.15.5:0/806155216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.459+0000 7f16feffd640 1 --2- 172.21.15.5:0/806155216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5ea680 0x5559df5e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.460+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/806155216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.460+0000 7f16feffd640 1 -- 172.21.15.5:0/806155216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5df4a0 msgr2=0x5559df5f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.460+0000 7f16feffd640 1 --2- 172.21.15.5:0/806155216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5df4a0 0x5559df5f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.460+0000 7f16feffd640 1 -- 172.21.15.5:0/806155216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 msgr2=0x5559df5dbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.460+0000 7f16feffd640 1 --2- 172.21.15.5:0/806155216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5dbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.460+0000 7f16feffd640 1 -- 172.21.15.5:0/806155216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559df5e6180 con 0x5559df5ea680 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.460+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/806155216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5dbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.461+0000 7f16feffd640 1 --2- 172.21.15.5:0/806155216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5ea680 0x5559df5e1600 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009fd0 tx=0x7f16f40133a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4444903b51a12ead server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.461+0000 7f16e57fa640 1 -- 172.21.15.5:0/806155216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f400f5c0 con 0x5559df5ea680 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.461+0000 7f16e57fa640 1 -- 172.21.15.5:0/806155216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f400f9b0 con 0x5559df5ea680 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.461+0000 7f16e57fa640 1 -- 172.21.15.5:0/806155216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40046c0 con 0x5559df5ea680 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.462+0000 7f17082eb740 1 -- 172.21.15.5:0/806155216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5ea680 msgr2=0x5559df5e1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.462+0000 7f17082eb740 1 --2- 172.21.15.5:0/806155216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5ea680 0x5559df5e1600 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009fd0 tx=0x7f16f40133a0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.462+0000 7f17082eb740 1 -- 172.21.15.5:0/806155216 shutdown_connections 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.462+0000 7f17082eb740 1 --2- 172.21.15.5:0/806155216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5ea680 0x5559df5e1600 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.462+0000 7f17082eb740 1 --2- 172.21.15.5:0/806155216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5dfdc0 0x5559df5dbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.462+0000 7f17082eb740 1 --2- 172.21.15.5:0/806155216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5df4a0 0x5559df5f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.462+0000 7f17082eb740 1 -- 172.21.15.5:0/806155216 >> 172.21.15.5:0/806155216 conn(0x5559df5dba30 msgr2=0x5559df5e0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.463+0000 7f17082eb740 1 -- 172.21.15.5:0/806155216 shutdown_connections 2024-09-11T22:24:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.463+0000 7f17082eb740 1 -- 172.21.15.5:0/806155216 wait complete. 2024-09-11T22:24:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.463+0000 7f17082eb740 1 -- start start 2024-09-11T22:24:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.464+0000 7f17082eb740 1 Processor -- start 2024-09-11T22:24:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.464+0000 7f17082eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5ee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.464+0000 7f17082eb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5dfdc0 0x5559df5ee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.464+0000 7f17082eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5ea680 0x5559df5eedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.464+0000 7f17082eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17001b1080 con 0x5559df5df4a0 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.464+0000 7f17082eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17001b09b0 con 0x5559df5ea680 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.464+0000 7f17082eb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17001adbf0 con 0x5559df5dfdc0 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.464+0000 7f17082eb740 10 client.0 _ll_register_callbacks cb 0x7f16f4013770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.465+0000 7f16ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5dfdc0 0x5559df5ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.465+0000 7f16ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5dfdc0 0x5559df5ee880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47832/0 (socket says 172.21.15.5:47832) 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.465+0000 7f16ff7fe640 1 -- 172.21.15.5:0/4228999359 learned_addr learned my addr 172.21.15.5:0/4228999359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.465+0000 7f16ff7fe640 1 -- 172.21.15.5:0/4228999359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5ea680 msgr2=0x5559df5eedc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.465+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5ea680 0x5559df5eedc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.465+0000 7f16ff7fe640 1 -- 172.21.15.5:0/4228999359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 msgr2=0x5559df5ee340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.465+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5ee340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.465+0000 7f16ff7fe640 1 -- 172.21.15.5:0/4228999359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559df5ea290 con 0x5559df5dfdc0 2024-09-11T22:24:51.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.465+0000 7f16deffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:24:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.465+0000 7f16ff7fe640 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5dfdc0 0x5559df5ee880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009ef0 tx=0x7f16f0009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f14c5349ceb2bb0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.466+0000 7f16deffd640 1 -- 172.21.15.5:0/4228999359 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f0030060 con 0x5559df5dfdc0 2024-09-11T22:24:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.466+0000 7f17082eb740 1 -- 172.21.15.5:0/4228999359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f400f9b0 con 0x5559df5dfdc0 2024-09-11T22:24:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.466+0000 7f16deffd640 1 -- 172.21.15.5:0/4228999359 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f16f0016040 con 0x5559df5dfdc0 2024-09-11T22:24:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.466+0000 7f16deffd640 1 -- 172.21.15.5:0/4228999359 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f16f0030060 con 0x5559df5dfdc0 2024-09-11T22:24:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.466+0000 7f17082eb740 10 client.25732 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.466+0000 7f17082eb740 1 -- 172.21.15.5:0/4228999359 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559df66d690 con 0x5559df5dfdc0 2024-09-11T22:24:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.466+0000 7f16deffd640 1 -- 172.21.15.5:0/4228999359 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f400f9b0 con 0x5559df5dfdc0 2024-09-11T22:24:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.467+0000 7f16deffd640 1 -- 172.21.15.5:0/4228999359 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f16f00035f0 con 0x5559df5dfdc0 2024-09-11T22:24:51.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.467+0000 7f17082eb740 10 client.25732 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.467+0000 7f17082eb740 10 client.25732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.467+0000 7f17082eb740 20 client.25732 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.467+0000 7f17082eb740 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5559df66dca0 0x5559df6700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.467+0000 7f17082eb740 4 client.25732 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.467+0000 7f17082eb740 1 -- 172.21.15.5:0/4228999359 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5559df54cd60 con 0x5559df66dca0 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.467+0000 7f16feffd640 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5559df66dca0 0x5559df6700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.469+0000 7f16feffd640 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5559df66dca0 0x5559df6700c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f8cab621a76759 server_cookie=d5fcd8763de0f0a in_seq=0 out_seq=0 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.469+0000 7f16deffd640 10 client.25732 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f16deffd640 1 -- 172.21.15.5:0/4228999359 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5559df54cd60 con 0x5559df66dca0 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f16deffd640 10 client.25732 handle_command_reply: tid=0 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f17082eb740 1 client.25732 shutdown 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f17082eb740 1 -- 172.21.15.5:0/4228999359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5dfdc0 msgr2=0x5559df5ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:51.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f17082eb740 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5dfdc0 0x5559df5ee880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009ef0 tx=0x7f16f0009290 comp rx=0 tx=0).stop 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f17082eb740 1 -- 172.21.15.5:0/4228999359 shutdown_connections 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f17082eb740 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5559df66dca0 0x5559df6700c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f17082eb740 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5559df5ea680 0x5559df5eedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f17082eb740 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559df5dfdc0 0x5559df5ee880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f17082eb740 1 --2- 172.21.15.5:0/4228999359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559df5df4a0 0x5559df5ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f16deffd640 0 client.25732 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.470+0000 7f17082eb740 1 -- 172.21.15.5:0/4228999359 >> 172.21.15.5:0/4228999359 conn(0x5559df5dba30 msgr2=0x5559df5e1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.471+0000 7f17082eb740 1 -- 172.21.15.5:0/4228999359 shutdown_connections 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.471+0000 7f17082eb740 1 -- 172.21.15.5:0/4228999359 wait complete. 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.471+0000 7f17082eb740 20 client.25732 trim_cache size 0 max 16384 2024-09-11T22:24:51.480 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:24:51.481 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:24:51.481 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:24:51.481 INFO:teuthology.orchestra.run.smithi005.stdout: "53ce156a-839f-4fbb-acef-8deee03510b5": { 2024-09-11T22:24:51.481 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:24:51.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "53ce156a-839f-4fbb-acef-8deee03510b5", 2024-09-11T22:24:51.481 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:24:51.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:51.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:24:51.481 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:24:51.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 -- 172.21.15.5:0/3377575327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f16e80a2cf0 msgr2=0x7f16e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:51.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f16e80a2cf0 0x7f16e80a51b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f16f80b90f0 tx=0x7f16f808e6e0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 -- 172.21.15.5:0/3377575327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1700102ef0 msgr2=0x7f170019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:51.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1700102ef0 0x7f170019cdd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f16f8004af0 tx=0x7f16f802f3a0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 -- 172.21.15.5:0/3377575327 shutdown_connections 2024-09-11T22:24:51.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1700102ef0 0x7f170019cdd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1700102570 0x7f1700198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f16e80a2cf0 0x7f16e80a51b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 --2- 172.21.15.5:0/3377575327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1700100e20 0x7f1700198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:51.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 -- 172.21.15.5:0/3377575327 >> 172.21.15.5:0/3377575327 conn(0x7f17000fc7d0 msgr2=0x7f170006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:51.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 -- 172.21.15.5:0/3377575327 shutdown_connections 2024-09-11T22:24:51.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:24:51.473+0000 7f1705f12640 1 -- 172.21.15.5:0/3377575327 wait complete. 2024-09-11T22:24:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:51 smithi172 ceph-mon[35461]: pgmap v82: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 274 op/s 2024-09-11T22:24:52.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:51 smithi005 ceph-mon[30943]: pgmap v82: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 274 op/s 2024-09-11T22:24:52.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 153170 12.05 MB/sec warmup 70 sec latency 8.643 ms 2024-09-11T22:24:52.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:51 smithi159 ceph-mon[35534]: pgmap v82: 97 pgs: 97 active+clean; 46 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 1.6 MiB/s wr, 274 op/s 2024-09-11T22:24:52.479 INFO:tasks.cephfs.filesystem:scrub status for tag:53ce156a-839f-4fbb-acef-8deee03510b5 - {'path': '/', 'tag': '53ce156a-839f-4fbb-acef-8deee03510b5', 'options': 'recursive,force'} 2024-09-11T22:24:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.157+0000 7f620f023640 1 -- 172.21.15.5:0/2630314899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6208101ad0 msgr2=0x7f6208101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.157+0000 7f620f023640 1 --2- 172.21.15.5:0/2630314899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6208101ad0 0x7f6208101f30 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009930 tx=0x7f61f402a420 comp rx=0 tx=0).stop 2024-09-11T22:24:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.157+0000 7f620f023640 1 -- 172.21.15.5:0/2630314899 shutdown_connections 2024-09-11T22:24:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.157+0000 7f620f023640 1 --2- 172.21.15.5:0/2630314899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6208102470 0x7f620810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.157+0000 7f620f023640 1 --2- 172.21.15.5:0/2630314899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6208101ad0 0x7f6208101f30 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.157+0000 7f620f023640 1 --2- 172.21.15.5:0/2630314899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6208107b70 0x7f6208107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.157+0000 7f620f023640 1 -- 172.21.15.5:0/2630314899 >> 172.21.15.5:0/2630314899 conn(0x7f62080fd7b0 msgr2=0x7f62080ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.157+0000 7f620f023640 1 -- 172.21.15.5:0/2630314899 shutdown_connections 2024-09-11T22:24:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.157+0000 7f620f023640 1 -- 172.21.15.5:0/2630314899 wait complete. 2024-09-11T22:24:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620f023640 1 Processor -- start 2024-09-11T22:24:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620f023640 1 -- start start 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620f023640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6208101ad0 0x7f6208198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620f023640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6208102470 0x7f62081988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620f023640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6208107b70 0x7f620819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620f023640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6208111f20 con 0x7f6208101ad0 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620f023640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6208111da0 con 0x7f6208102470 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620f023640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62081120a0 con 0x7f6208107b70 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620cdbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6208101ad0 0x7f6208198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620cdbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6208101ad0 0x7f6208198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37274/0 (socket says 172.21.15.5:37274) 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620cdbf640 1 -- 172.21.15.5:0/905449410 learned_addr learned my addr 172.21.15.5:0/905449410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f620d5c0640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6208107b70 0x7f620819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.158+0000 7f6207fff640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6208102470 0x7f62081988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f620cdbf640 1 -- 172.21.15.5:0/905449410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6208107b70 msgr2=0x7f620819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f620cdbf640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6208107b70 0x7f620819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f620cdbf640 1 -- 172.21.15.5:0/905449410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6208102470 msgr2=0x7f62081988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f620cdbf640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6208102470 0x7f62081988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f620cdbf640 1 -- 172.21.15.5:0/905449410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620819d380 con 0x7f6208101ad0 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f620cdbf640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6208101ad0 0x7f6208198390 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00ebc0 tx=0x7f61fc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f6205ffb640 1 -- 172.21.15.5:0/905449410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc010630 con 0x7f6208101ad0 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f6205ffb640 1 -- 172.21.15.5:0/905449410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61fc010cb0 con 0x7f6208101ad0 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f620f023640 1 -- 172.21.15.5:0/905449410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620819d670 con 0x7f6208101ad0 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f6205ffb640 1 -- 172.21.15.5:0/905449410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc018bd0 con 0x7f6208101ad0 2024-09-11T22:24:53.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.159+0000 7f620f023640 1 -- 172.21.15.5:0/905449410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62081a4f00 con 0x7f6208101ad0 2024-09-11T22:24:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6205ffb640 1 -- 172.21.15.5:0/905449410 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f61fc0107d0 con 0x7f6208101ad0 2024-09-11T22:24:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6211dbd740 1 Processor -- start 2024-09-11T22:24:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6211dbd740 1 -- start start 2024-09-11T22:24:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a37ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 0x556e53a42770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a450a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6211dbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e5399c2f0 con 0x556e53a42cb0 2024-09-11T22:24:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6211dbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556e53944270 con 0x556e53a37af0 2024-09-11T22:24:53.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6211dbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556e539982b0 con 0x556e53a38410 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f620d5c0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a37ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6207fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f620d5c0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a37ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55022/0 (socket says 172.21.15.5:55022) 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f620d5c0640 1 -- 172.21.15.5:0/1605602031 learned_addr learned my addr 172.21.15.5:0/1605602031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6207fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37290/0 (socket says 172.21.15.5:37290) 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f620cdbf640 1 --2- 172.21.15.5:0/1605602031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 0x556e53a42770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.162+0000 7f6205ffb640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61f80a2da0 0x7f61f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f620d5c0640 1 -- 172.21.15.5:0/1605602031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 msgr2=0x556e53a42770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f620d5c0640 1 --2- 172.21.15.5:0/1605602031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 0x556e53a42770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f620d5c0640 1 -- 172.21.15.5:0/1605602031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 msgr2=0x556e53a450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f620d5c0640 1 --2- 172.21.15.5:0/1605602031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f620d5c0640 1 -- 172.21.15.5:0/1605602031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e53943340 con 0x556e53a37af0 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f6207fff640 1 --2- 172.21.15.5:0/1605602031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f6205ffb640 1 -- 172.21.15.5:0/905449410 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f61fc0cf9f0 con 0x7f6208101ad0 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f620d5c0640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61f80a2da0 0x7f61f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f620d5c0640 1 --2- 172.21.15.5:0/1605602031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a37ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6200009e20 tx=0x7f6200011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e455f379fc73cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f61f2ffd640 1 -- 172.21.15.5:0/1605602031 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200017050 con 0x556e53a37af0 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f620d5c0640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61f80a2da0 0x7f61f80a5260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6200023000 tx=0x7f6200024000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f61f2ffd640 1 -- 172.21.15.5:0/1605602031 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6200003c40 con 0x556e53a37af0 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.163+0000 7f61f2ffd640 1 -- 172.21.15.5:0/1605602031 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620000f620 con 0x556e53a37af0 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 -- 172.21.15.5:0/1605602031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 msgr2=0x556e53a37ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 --2- 172.21.15.5:0/1605602031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a37ed0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6200009e20 tx=0x7f6200011810 comp rx=0 tx=0).stop 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 -- 172.21.15.5:0/1605602031 shutdown_connections 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 --2- 172.21.15.5:0/1605602031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 --2- 172.21.15.5:0/1605602031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 0x556e53a42770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 --2- 172.21.15.5:0/1605602031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a37ed0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 -- 172.21.15.5:0/1605602031 >> 172.21.15.5:0/1605602031 conn(0x556e53a34060 msgr2=0x556e53a34440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 -- 172.21.15.5:0/1605602031 shutdown_connections 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 -- 172.21.15.5:0/1605602031 wait complete. 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 -- start start 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.164+0000 7f6211dbd740 1 Processor -- start 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 0x556e53a3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a40120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a42cb0 0x556e53a40660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6211dbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e53a45880 con 0x556e53a38410 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6211dbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556e53925290 con 0x556e53a42cb0 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6211dbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556e53a45a00 con 0x556e53a37af0 2024-09-11T22:24:53.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6211dbd740 10 client.0 _ll_register_callbacks cb 0x556e53956140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6207fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a40120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f620cdbf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 0x556e53a3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6207fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a40120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37296/0 (socket says 172.21.15.5:37296) 2024-09-11T22:24:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f620d5c0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a42cb0 0x556e53a40660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6207fff640 1 -- 172.21.15.5:0/2995098682 learned_addr learned my addr 172.21.15.5:0/2995098682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f620cdbf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 0x556e53a3fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47880/0 (socket says 172.21.15.5:47880) 2024-09-11T22:24:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6207fff640 1 -- 172.21.15.5:0/2995098682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 msgr2=0x556e53a3fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6207fff640 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 0x556e53a3fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6207fff640 1 -- 172.21.15.5:0/2995098682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a42cb0 msgr2=0x556e53a40660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6207fff640 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a42cb0 0x556e53a40660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.165+0000 7f6207fff640 1 -- 172.21.15.5:0/2995098682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e536af210 con 0x556e53a38410 2024-09-11T22:24:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f6207fff640 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a40120 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009930 tx=0x7f61f40317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595553877c69dde6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:53.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f61e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f61e7fff640 1 -- 172.21.15.5:0/2995098682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4035040 con 0x556e53a38410 2024-09-11T22:24:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f61e7fff640 1 -- 172.21.15.5:0/2995098682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f61f4004260 con 0x556e53a38410 2024-09-11T22:24:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f6211dbd740 1 -- 172.21.15.5:0/2995098682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6200003c40 con 0x556e53a38410 2024-09-11T22:24:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f61e7fff640 1 -- 172.21.15.5:0/2995098682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61f4004460 con 0x556e53a38410 2024-09-11T22:24:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f6211dbd740 10 client.16272 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f6211dbd740 1 -- 172.21.15.5:0/2995098682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e5377b620 con 0x556e53a38410 2024-09-11T22:24:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f61e7fff640 1 -- 172.21.15.5:0/2995098682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6200003c40 con 0x556e53a38410 2024-09-11T22:24:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f61e7fff640 1 -- 172.21.15.5:0/2995098682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f61f40035f0 con 0x556e53a38410 2024-09-11T22:24:53.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f6211dbd740 10 client.16272 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f6211dbd740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f6211dbd740 20 client.16272 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.166+0000 7f6211dbd740 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556e53a52cf0 0x556e53a55110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.167+0000 7f6211dbd740 4 client.16272 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.167+0000 7f6211dbd740 1 -- 172.21.15.5:0/2995098682 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e539a9a20 con 0x556e53a52cf0 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.167+0000 7f620cdbf640 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556e53a52cf0 0x556e53a55110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.167+0000 7f61e7fff640 10 client.16272 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.167+0000 7f620cdbf640 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556e53a52cf0 0x556e53a55110 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a019366b2e361a3 server_cookie=aa9fa37002ca4a95 in_seq=0 out_seq=0 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f61e7fff640 1 -- 172.21.15.5:0/2995098682 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e539a9a20 con 0x556e53a52cf0 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f61e7fff640 10 client.16272 handle_command_reply: tid=0 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 client.16272 shutdown 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 -- 172.21.15.5:0/2995098682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 msgr2=0x556e53a40120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:53.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a40120 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009930 tx=0x7f61f40317e0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 -- 172.21.15.5:0/2995098682 shutdown_connections 2024-09-11T22:24:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556e53a52cf0 0x556e53a55110 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a42cb0 0x556e53a40660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a40120 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 --2- 172.21.15.5:0/2995098682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 0x556e53a3fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f61e7fff640 0 client.16272 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 -- 172.21.15.5:0/2995098682 >> 172.21.15.5:0/2995098682 conn(0x556e53a34060 msgr2=0x556e53a39d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 -- 172.21.15.5:0/2995098682 shutdown_connections 2024-09-11T22:24:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.168+0000 7f6211dbd740 1 -- 172.21.15.5:0/2995098682 wait complete. 2024-09-11T22:24:53.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.169+0000 7f6211dbd740 20 client.16272 trim_cache size 0 max 16384 2024-09-11T22:24:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f6211dbd740 1 Processor -- start 2024-09-11T22:24:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f6211dbd740 1 -- start start 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a40490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 0x556e53a41b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a5a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f6211dbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62081ada30 con 0x556e53a42cb0 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f6211dbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62081af630 con 0x556e53a37af0 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f6211dbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62081ae100 con 0x556e53a38410 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f620d5c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a5a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f620d5c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a5a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37300/0 (socket says 172.21.15.5:37300) 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f620d5c0640 1 -- 172.21.15.5:0/182315849 learned_addr learned my addr 172.21.15.5:0/182315849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f620cdbf640 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a40490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f620d5c0640 1 -- 172.21.15.5:0/182315849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 msgr2=0x556e53a41b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f6207fff640 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 0x556e53a41b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f620d5c0640 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 0x556e53a41b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f620d5c0640 1 -- 172.21.15.5:0/182315849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 msgr2=0x556e53a40490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f620d5c0640 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a40490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f620d5c0640 1 -- 172.21.15.5:0/182315849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e53a42730 con 0x556e53a42cb0 2024-09-11T22:24:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f6207fff640 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 0x556e53a41b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:24:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.184+0000 7f620cdbf640 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a40490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:24:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f620d5c0640 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a5a9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6200014040 tx=0x7f620001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d5d5f0f44b3325 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:53.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f61f27fc640 1 -- 172.21.15.5:0/182315849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200009780 con 0x556e53a42cb0 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f61f27fc640 1 -- 172.21.15.5:0/182315849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f620000f3c0 con 0x556e53a42cb0 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f61f27fc640 1 -- 172.21.15.5:0/182315849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620000fe00 con 0x556e53a42cb0 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f6211dbd740 1 -- 172.21.15.5:0/182315849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 msgr2=0x556e53a5a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f6211dbd740 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a5a9c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f6200014040 tx=0x7f620001ccc0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f6211dbd740 1 -- 172.21.15.5:0/182315849 shutdown_connections 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f6211dbd740 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a42cb0 0x556e53a5a9c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f6211dbd740 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a38410 0x556e53a41b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f6211dbd740 1 --2- 172.21.15.5:0/182315849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a37af0 0x556e53a40490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.185+0000 7f6211dbd740 1 -- 172.21.15.5:0/182315849 >> 172.21.15.5:0/182315849 conn(0x556e53a34060 msgr2=0x556e53a41440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 1 -- 172.21.15.5:0/182315849 shutdown_connections 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 1 -- 172.21.15.5:0/182315849 wait complete. 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 1 -- start start 2024-09-11T22:24:53.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 1 Processor -- start 2024-09-11T22:24:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 0x556e53a4b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a3caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a42cb0 0x556e53a3d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62081aeea0 con 0x556e53a38410 2024-09-11T22:24:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62081ae7d0 con 0x556e53a42cb0 2024-09-11T22:24:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62081aba10 con 0x556e53a37af0 2024-09-11T22:24:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6211dbd740 10 client.0 _ll_register_callbacks cb 0x7f61f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:24:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6207fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a3caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6207fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a3caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37308/0 (socket says 172.21.15.5:37308) 2024-09-11T22:24:53.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f6207fff640 1 -- 172.21.15.5:0/148822143 learned_addr learned my addr 172.21.15.5:0/148822143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f620d5c0640 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a42cb0 0x556e53a3d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.186+0000 7f620cdbf640 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 0x556e53a4b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f6207fff640 1 -- 172.21.15.5:0/148822143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 msgr2=0x556e53a4b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f6207fff640 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 0x556e53a4b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f6207fff640 1 -- 172.21.15.5:0/148822143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a42cb0 msgr2=0x556e53a3d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f6207fff640 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a42cb0 0x556e53a3d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f6207fff640 1 -- 172.21.15.5:0/148822143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e53ac0da0 con 0x556e53a38410 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f6207fff640 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a3caf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009900 tx=0x7f61f4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4694cd608c1d0e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f61e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f61e7fff640 1 -- 172.21.15.5:0/148822143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4035070 con 0x556e53a38410 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f6211dbd740 1 -- 172.21.15.5:0/148822143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620000f3c0 con 0x556e53a38410 2024-09-11T22:24:53.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f61e7fff640 1 -- 172.21.15.5:0/148822143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f61f4005040 con 0x556e53a38410 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f61e7fff640 1 -- 172.21.15.5:0/148822143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61f4003800 con 0x556e53a38410 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f6211dbd740 10 client.16278 fetch_fsmap learned FSMap version 30 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f6211dbd740 1 -- 172.21.15.5:0/148822143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556e53ac1f00 con 0x556e53a38410 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.187+0000 7f61e7fff640 1 -- 172.21.15.5:0/148822143 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f620000f3c0 con 0x556e53a38410 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.188+0000 7f61e7fff640 1 -- 172.21.15.5:0/148822143 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f61f403d450 con 0x556e53a38410 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.188+0000 7f6211dbd740 10 client.16278 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.188+0000 7f6211dbd740 10 client.16278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.188+0000 7f6211dbd740 20 client.16278 populate_metadata read hostname 'smithi005' 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.188+0000 7f6211dbd740 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556e53ac2650 0x556e53ac4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.188+0000 7f6211dbd740 4 client.16278 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.188+0000 7f6211dbd740 1 -- 172.21.15.5:0/148822143 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556e539a3d70 con 0x556e53ac2650 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.188+0000 7f620cdbf640 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556e53ac2650 0x556e53ac4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:24:53.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.188+0000 7f61e7fff640 10 client.16278 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:53.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.188+0000 7f620cdbf640 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556e53ac2650 0x556e53ac4a70 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f0ed301906d32d server_cookie=3bfc0e8dd24f9fe0 in_seq=0 out_seq=0 2024-09-11T22:24:53.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 156074 12.12 MB/sec warmup 71 sec latency 25.624 ms 2024-09-11T22:24:53.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f61e7fff640 1 -- 172.21.15.5:0/148822143 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556e539a3d70 con 0x556e53ac2650 2024-09-11T22:24:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f61e7fff640 10 client.16278 handle_command_reply: tid=0 2024-09-11T22:24:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f6211dbd740 1 client.16278 shutdown 2024-09-11T22:24:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f6211dbd740 1 -- 172.21.15.5:0/148822143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 msgr2=0x556e53a3caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f6211dbd740 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a3caf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009900 tx=0x7f61f4005fd0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f6211dbd740 1 -- 172.21.15.5:0/148822143 shutdown_connections 2024-09-11T22:24:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f6211dbd740 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556e53ac2650 0x556e53ac4a70 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f6211dbd740 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556e53a42cb0 0x556e53a3d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f61e7fff640 0 client.16278 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:24:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f6211dbd740 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e53a38410 0x556e53a3caf0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f6211dbd740 1 --2- 172.21.15.5:0/148822143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556e53a37af0 0x556e53a4b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.307+0000 7f6211dbd740 1 -- 172.21.15.5:0/148822143 >> 172.21.15.5:0/148822143 conn(0x556e53a34060 msgr2=0x556e53a331c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.308+0000 7f6211dbd740 1 -- 172.21.15.5:0/148822143 shutdown_connections 2024-09-11T22:24:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.308+0000 7f6211dbd740 1 -- 172.21.15.5:0/148822143 wait complete. 2024-09-11T22:24:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.308+0000 7f6211dbd740 20 client.16278 trim_cache size 0 max 16384 2024-09-11T22:24:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:24:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:24:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:24:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:24:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.308+0000 7f620f023640 1 -- 172.21.15.5:0/905449410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61f80a2da0 msgr2=0x7f61f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.308+0000 7f620f023640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61f80a2da0 0x7f61f80a5260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6200023000 tx=0x7f6200024000 comp rx=0 tx=0).stop 2024-09-11T22:24:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.308+0000 7f620f023640 1 -- 172.21.15.5:0/905449410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6208101ad0 msgr2=0x7f6208198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:24:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.308+0000 7f620f023640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6208101ad0 0x7f6208198390 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f61fc00ebc0 tx=0x7f61fc00c6a0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.308+0000 7f620f023640 1 -- 172.21.15.5:0/905449410 shutdown_connections 2024-09-11T22:24:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.309+0000 7f620f023640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6208107b70 0x7f620819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.309+0000 7f620f023640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61f80a2da0 0x7f61f80a5260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.309+0000 7f620f023640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6208102470 0x7f62081988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.309+0000 7f620f023640 1 --2- 172.21.15.5:0/905449410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6208101ad0 0x7f6208198390 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:24:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.309+0000 7f620f023640 1 -- 172.21.15.5:0/905449410 >> 172.21.15.5:0/905449410 conn(0x7f62080fd7b0 msgr2=0x7f62080fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:24:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.309+0000 7f620f023640 1 -- 172.21.15.5:0/905449410 shutdown_connections 2024-09-11T22:24:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:24:53.309+0000 7f620f023640 1 -- 172.21.15.5:0/905449410 wait complete. 2024-09-11T22:24:54.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 158501 12.08 MB/sec warmup 72 sec latency 20.768 ms 2024-09-11T22:24:54.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:53 smithi005 ceph-mon[30943]: pgmap v83: 97 pgs: 97 active+clean; 51 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.0 MiB/s wr, 412 op/s 2024-09-11T22:24:54.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:53 smithi159 ceph-mon[35534]: pgmap v83: 97 pgs: 97 active+clean; 51 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.0 MiB/s wr, 412 op/s 2024-09-11T22:24:54.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:53 smithi172 ceph-mon[35461]: pgmap v83: 97 pgs: 97 active+clean; 51 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.0 MiB/s wr, 412 op/s 2024-09-11T22:24:55.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 160689 12.10 MB/sec warmup 73 sec latency 18.677 ms 2024-09-11T22:24:56.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 163547 12.17 MB/sec warmup 74 sec latency 67.454 ms 2024-09-11T22:24:56.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:55 smithi005 ceph-mon[30943]: pgmap v84: 97 pgs: 97 active+clean; 51 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-09-11T22:24:56.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:55 smithi159 ceph-mon[35534]: pgmap v84: 97 pgs: 97 active+clean; 51 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-09-11T22:24:56.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:55 smithi172 ceph-mon[35461]: pgmap v84: 97 pgs: 97 active+clean; 51 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-09-11T22:24:57.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 166388 12.19 MB/sec warmup 75 sec latency 20.369 ms 2024-09-11T22:24:58.131 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:57 smithi159 ceph-mon[35534]: pgmap v85: 97 pgs: 97 active+clean; 51 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-09-11T22:24:58.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 169150 12.20 MB/sec warmup 76 sec latency 9.213 ms 2024-09-11T22:24:58.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:57 smithi005 ceph-mon[30943]: pgmap v85: 97 pgs: 97 active+clean; 51 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-09-11T22:24:58.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:57 smithi172 ceph-mon[35461]: pgmap v85: 97 pgs: 97 active+clean; 51 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 241 op/s 2024-09-11T22:24:59.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 171427 12.22 MB/sec warmup 77 sec latency 15.244 ms 2024-09-11T22:24:59.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:58 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:24:59.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:58 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:24:59.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:58 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:24:59.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:58 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:24:59.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:58 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:24:59.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:58 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:25:00.260 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:24:59 smithi172 ceph-mon[35461]: pgmap v86: 97 pgs: 97 active+clean; 45 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 456 op/s 2024-09-11T22:25:00.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 174285 12.28 MB/sec warmup 78 sec latency 10.868 ms 2024-09-11T22:25:00.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:24:59 smithi005 ceph-mon[30943]: pgmap v86: 97 pgs: 97 active+clean; 45 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 456 op/s 2024-09-11T22:25:00.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:24:59 smithi159 ceph-mon[35534]: pgmap v86: 97 pgs: 97 active+clean; 45 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 456 op/s 2024-09-11T22:25:00.332 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-11T22:25:00.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:25:00.626 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:01.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:00 smithi005 ceph-mon[30943]: pgmap v87: 97 pgs: 97 active+clean; 45 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.6 MiB/s wr, 352 op/s 2024-09-11T22:25:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.168+0000 7fa832daf640 1 -- 172.21.15.5:0/2482149411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa82c101e10 msgr2=0x7fa82c1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.168+0000 7fa832daf640 1 --2- 172.21.15.5:0/2482149411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa82c101e10 0x7fa82c1021f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa820009900 tx=0x7fa82002a410 comp rx=0 tx=0).stop 2024-09-11T22:25:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 -- 172.21.15.5:0/2482149411 shutdown_connections 2024-09-11T22:25:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 --2- 172.21.15.5:0/2482149411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa82c106f40 0x7fa82c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 --2- 172.21.15.5:0/2482149411 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c1027c0 0x7fa82c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 --2- 172.21.15.5:0/2482149411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa82c101e10 0x7fa82c1021f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 -- 172.21.15.5:0/2482149411 >> 172.21.15.5:0/2482149411 conn(0x7fa82c0fd8d0 msgr2=0x7fa82c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:01.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 -- 172.21.15.5:0/2482149411 shutdown_connections 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 -- 172.21.15.5:0/2482149411 wait complete. 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 Processor -- start 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 -- start start 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa82c101e10 0x7fa82c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa82c1027c0 0x7fa82c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c106f40 0x7fa82c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c111fa0 con 0x7fa82c1027c0 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c111e20 con 0x7fa82c101e10 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.169+0000 7fa832daf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c112120 con 0x7fa82c106f40 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa82c1027c0 0x7fa82c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa83134c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c106f40 0x7fa82c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa82c1027c0 0x7fa82c19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37340/0 (socket says 172.21.15.5:37340) 2024-09-11T22:25:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa830b4b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa82c101e10 0x7fa82c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa82bfff640 1 -- 172.21.15.5:0/3033203976 learned_addr learned my addr 172.21.15.5:0/3033203976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa83134c640 1 -- 172.21.15.5:0/3033203976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa82c101e10 msgr2=0x7fa82c19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa83134c640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa82c101e10 0x7fa82c19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa83134c640 1 -- 172.21.15.5:0/3033203976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa82c1027c0 msgr2=0x7fa82c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa83134c640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa82c1027c0 0x7fa82c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa83134c640 1 -- 172.21.15.5:0/3033203976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c19f630 con 0x7fa82c106f40 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa82bfff640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa82c1027c0 0x7fa82c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa83134c640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c106f40 0x7fa82c19ef50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa82400da10 tx=0x7fa82400ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa829ffb640 1 -- 172.21.15.5:0/3033203976 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82400bd70 con 0x7fa82c106f40 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa832daf640 1 -- 172.21.15.5:0/3033203976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c19f920 con 0x7fa82c106f40 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa829ffb640 1 -- 172.21.15.5:0/3033203976 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa824003e50 con 0x7fa82c106f40 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.170+0000 7fa829ffb640 1 -- 172.21.15.5:0/3033203976 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8240106a0 con 0x7fa82c106f40 2024-09-11T22:25:01.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.171+0000 7fa832daf640 1 -- 172.21.15.5:0/3033203976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c1a7200 con 0x7fa82c106f40 2024-09-11T22:25:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.172+0000 7fa832daf640 1 -- 172.21.15.5:0/3033203976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82c1021f0 con 0x7fa82c106f40 2024-09-11T22:25:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.173+0000 7fa829ffb640 1 -- 172.21.15.5:0/3033203976 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa824003970 con 0x7fa82c106f40 2024-09-11T22:25:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.173+0000 7fa829ffb640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa81c0a2cf0 0x7fa81c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.174+0000 7fa830b4b640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa81c0a2cf0 0x7fa81c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.174+0000 7fa829ffb640 1 -- 172.21.15.5:0/3033203976 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fa8240cf7d0 con 0x7fa82c106f40 2024-09-11T22:25:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.174+0000 7fa830b4b640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa81c0a2cf0 0x7fa81c0a51b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa820009230 tx=0x7fa8200058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.178+0000 7fa829ffb640 1 -- 172.21.15.5:0/3033203976 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa82409c3b0 con 0x7fa82c106f40 2024-09-11T22:25:01.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 176381 12.23 MB/sec warmup 79 sec latency 21.785 ms 2024-09-11T22:25:01.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:00 smithi159 ceph-mon[35534]: pgmap v87: 97 pgs: 97 active+clean; 45 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.6 MiB/s wr, 352 op/s 2024-09-11T22:25:01.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:00 smithi172 ceph-mon[35461]: pgmap v87: 97 pgs: 97 active+clean; 45 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.6 MiB/s wr, 352 op/s 2024-09-11T22:25:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.397+0000 7fa832daf640 1 -- 172.21.15.5:0/3033203976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa82c1a74e0 con 0x7fa82c106f40 2024-09-11T22:25:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.398+0000 7fa829ffb640 1 -- 172.21.15.5:0/3033203976 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11974 (secure 0 0 0) 0x7fa82409bcb0 con 0x7fa82c106f40 2024-09-11T22:25:01.399 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:25:01.399 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-11T22:23:49:156575+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:23:48.779139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:25:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-11T22:25:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 -- 172.21.15.5:0/3033203976 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa81c0a2cf0 msgr2=0x7fa81c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa81c0a2cf0 0x7fa81c0a51b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa820009230 tx=0x7fa8200058d0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 -- 172.21.15.5:0/3033203976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c106f40 msgr2=0x7fa82c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c106f40 0x7fa82c19ef50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa82400da10 tx=0x7fa82400ded0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 -- 172.21.15.5:0/3033203976 shutdown_connections 2024-09-11T22:25:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa82c106f40 0x7fa82c19ef50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa82c1027c0 0x7fa82c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa81c0a2cf0 0x7fa81c0a51b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 --2- 172.21.15.5:0/3033203976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa82c101e10 0x7fa82c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 -- 172.21.15.5:0/3033203976 >> 172.21.15.5:0/3033203976 conn(0x7fa82c0fd8d0 msgr2=0x7fa82c0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 -- 172.21.15.5:0/3033203976 shutdown_connections 2024-09-11T22:25:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:01.401+0000 7fa832daf640 1 -- 172.21.15.5:0/3033203976 wait complete. 2024-09-11T22:25:02.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:01 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3033203976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:25:02.271 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-11T22:25:02.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-11T22:25:02.307 INFO:tasks.workunit.client.0.smithi005.stdout: 1 178586 12.24 MB/sec warmup 80 sec latency 15.132 ms 2024-09-11T22:25:02.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:01 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3033203976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:25:02.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:01 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3033203976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:25:02.547 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:03.012 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:02 smithi005 ceph-mon[30943]: pgmap v88: 97 pgs: 97 active+clean; 52 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 482 op/s 2024-09-11T22:25:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.107+0000 7efd01e3b640 1 -- 172.21.15.5:0/3038237536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc106f10 msgr2=0x7efcfc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.107+0000 7efd01e3b640 1 --2- 172.21.15.5:0/3038237536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc106f10 0x7efcfc10ea90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009ca0 tx=0x7efcf402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.107+0000 7efd01e3b640 1 -- 172.21.15.5:0/3038237536 shutdown_connections 2024-09-11T22:25:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.107+0000 7efd01e3b640 1 --2- 172.21.15.5:0/3038237536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc106f10 0x7efcfc10ea90 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.107+0000 7efd01e3b640 1 --2- 172.21.15.5:0/3038237536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc102790 0x7efcfc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.107+0000 7efd01e3b640 1 --2- 172.21.15.5:0/3038237536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc101de0 0x7efcfc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.107+0000 7efd01e3b640 1 -- 172.21.15.5:0/3038237536 >> 172.21.15.5:0/3038237536 conn(0x7efcfc0fd8c0 msgr2=0x7efcfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.107+0000 7efd01e3b640 1 -- 172.21.15.5:0/3038237536 shutdown_connections 2024-09-11T22:25:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.107+0000 7efd01e3b640 1 -- 172.21.15.5:0/3038237536 wait complete. 2024-09-11T22:25:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.108+0000 7efd01e3b640 1 Processor -- start 2024-09-11T22:25:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.108+0000 7efd01e3b640 1 -- start start 2024-09-11T22:25:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efd01e3b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc101de0 0x7efcfc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efd01e3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc102790 0x7efcfc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efd01e3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc106f10 0x7efcfc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efd01e3b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc111fc0 con 0x7efcfc106f10 2024-09-11T22:25:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efd01e3b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc111e40 con 0x7efcfc101de0 2024-09-11T22:25:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efd01e3b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc112140 con 0x7efcfc102790 2024-09-11T22:25:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc101de0 0x7efcfc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc106f10 0x7efcfc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc101de0 0x7efcfc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42468/0 (socket says 172.21.15.5:42468) 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfb7fe640 1 -- 172.21.15.5:0/2110257718 learned_addr learned my addr 172.21.15.5:0/2110257718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfaffd640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc102790 0x7efcfc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfbfff640 1 -- 172.21.15.5:0/2110257718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc102790 msgr2=0x7efcfc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfbfff640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc102790 0x7efcfc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfbfff640 1 -- 172.21.15.5:0/2110257718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc101de0 msgr2=0x7efcfc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfbfff640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc101de0 0x7efcfc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfbfff640 1 -- 172.21.15.5:0/2110257718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc1a68f0 con 0x7efcfc106f10 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc101de0 0x7efcfc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.109+0000 7efcfbfff640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc106f10 0x7efcfc074f70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efcf4005e30 tx=0x7efcf4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.110+0000 7efcf8ff9640 1 -- 172.21.15.5:0/2110257718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4031650 con 0x7efcfc106f10 2024-09-11T22:25:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.110+0000 7efcf8ff9640 1 -- 172.21.15.5:0/2110257718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf40317f0 con 0x7efcfc106f10 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.110+0000 7efd01e3b640 1 -- 172.21.15.5:0/2110257718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc1a6b80 con 0x7efcfc106f10 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.110+0000 7efcf8ff9640 1 -- 172.21.15.5:0/2110257718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf402fa30 con 0x7efcfc106f10 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.110+0000 7efd01e3b640 1 -- 172.21.15.5:0/2110257718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcfc1a7110 con 0x7efcfc106f10 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.112+0000 7efcf8ff9640 1 -- 172.21.15.5:0/2110257718 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7efcf402fc70 con 0x7efcfc106f10 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.112+0000 7efd04214740 1 Processor -- start 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.112+0000 7efd04214740 1 -- start start 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efd04214740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a24760 0x555f84a24b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efd04214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a25080 0x555f84a2f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efd04214740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a31d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efd04214740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f8498a2a0 con 0x555f84a2f940 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efd04214740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555f84931b10 con 0x555f84a24760 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efd04214740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555f84986260 con 0x555f84a25080 2024-09-11T22:25:03.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcf8ff9640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efce40a2d70 0x7efce40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfaffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a24760 0x555f84a24b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a25080 0x555f84a2f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a31d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a25080 0x555f84a2f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:51176/0 (socket says 172.21.15.5:51176) 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfb7fe640 1 -- 172.21.15.5:0/3554572328 learned_addr learned my addr 172.21.15.5:0/3554572328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a31d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45948/0 (socket says 172.21.15.5:45948) 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfbfff640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efce40a2d70 0x7efce40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcf8ff9640 1 -- 172.21.15.5:0/2110257718 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7efcf40f0560 con 0x7efcfc106f10 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfaffd640 1 -- 172.21.15.5:0/3554572328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a25080 msgr2=0x555f84a2f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfaffd640 1 --2- 172.21.15.5:0/3554572328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a25080 0x555f84a2f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfaffd640 1 -- 172.21.15.5:0/3554572328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a24760 msgr2=0x555f84a24b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfaffd640 1 --2- 172.21.15.5:0/3554572328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a24760 0x555f84a24b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.113+0000 7efcfaffd640 1 -- 172.21.15.5:0/3554572328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f846fbb90 con 0x555f84a2f940 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.114+0000 7efcfbfff640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efce40a2d70 0x7efce40a5230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efcf40ede30 tx=0x7efcf4039000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.114+0000 7efcfaffd640 1 --2- 172.21.15.5:0/3554572328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a31d30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efcf0012660 tx=0x7efcf000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863d1b02927e6b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.114+0000 7efce1ffb640 1 -- 172.21.15.5:0/3554572328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0004020 con 0x555f84a2f940 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.114+0000 7efce1ffb640 1 -- 172.21.15.5:0/3554572328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf0005500 con 0x555f84a2f940 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.114+0000 7efce1ffb640 1 -- 172.21.15.5:0/3554572328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0004c50 con 0x555f84a2f940 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.114+0000 7efd04214740 1 -- 172.21.15.5:0/3554572328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 msgr2=0x555f84a31d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.114+0000 7efd04214740 1 --2- 172.21.15.5:0/3554572328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a31d30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7efcf0012660 tx=0x7efcf000b390 comp rx=0 tx=0).stop 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.115+0000 7efd04214740 1 -- 172.21.15.5:0/3554572328 shutdown_connections 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.115+0000 7efd04214740 1 --2- 172.21.15.5:0/3554572328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a31d30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.115+0000 7efd04214740 1 --2- 172.21.15.5:0/3554572328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a25080 0x555f84a2f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.115+0000 7efd04214740 1 --2- 172.21.15.5:0/3554572328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a24760 0x555f84a24b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.115+0000 7efd04214740 1 -- 172.21.15.5:0/3554572328 >> 172.21.15.5:0/3554572328 conn(0x555f84a20cf0 msgr2=0x555f84a210d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.115+0000 7efd04214740 1 -- 172.21.15.5:0/3554572328 shutdown_connections 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.115+0000 7efd04214740 1 -- 172.21.15.5:0/3554572328 wait complete. 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.115+0000 7efd04214740 1 -- start start 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.115+0000 7efd04214740 1 Processor -- start 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efd04214740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a24760 0x555f84a2c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efd04214740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a25080 0x555f84a2cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efd04214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a2f940 0x555f84a2d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efd04214740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f84a32510 con 0x555f84a25080 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efd04214740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555f849126e0 con 0x555f84a24760 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efd04214740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555f84a32690 con 0x555f84a2f940 2024-09-11T22:25:03.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a25080 0x555f84a2cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a25080 0x555f84a2cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45954/0 (socket says 172.21.15.5:45954) 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfb7fe640 1 -- 172.21.15.5:0/2451498707 learned_addr learned my addr 172.21.15.5:0/2451498707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfaffd640 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a24760 0x555f84a2c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efd04214740 10 client.0 _ll_register_callbacks cb 0x555f84936a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfaffd640 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a2f940 0x555f84a2d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfb7fe640 1 -- 172.21.15.5:0/2451498707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a2f940 msgr2=0x555f84a2d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a2f940 0x555f84a2d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfb7fe640 1 -- 172.21.15.5:0/2451498707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a24760 msgr2=0x555f84a2c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a24760 0x555f84a2c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfb7fe640 1 -- 172.21.15.5:0/2451498707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f84a3bab0 con 0x555f84a25080 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfaffd640 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a2f940 0x555f84a2d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:25:03.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efd04214740 1 -- 172.21.15.5:0/2451498707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf0005500 con 0x555f84a25080 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a25080 0x555f84a2cdd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efcec012c80 tx=0x7efcec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3573e0183f2c0ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcdaffd640 10 client.16296 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcdaffd640 1 -- 172.21.15.5:0/2451498707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec0047a0 con 0x555f84a25080 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.116+0000 7efcdaffd640 1 -- 172.21.15.5:0/2451498707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7efcec004940 con 0x555f84a25080 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efcdaffd640 1 -- 172.21.15.5:0/2451498707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efcec004b80 con 0x555f84a25080 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efd04214740 10 client.16296 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efd04214740 1 -- 172.21.15.5:0/2451498707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=30}) -- 0x555f84a3e3b0 con 0x555f84a25080 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efcdaffd640 1 -- 172.21.15.5:0/2451498707 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7efcec004d20 con 0x555f84a25080 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efd04214740 10 client.16296 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efd04214740 10 client.16296 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efd04214740 20 client.16296 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efd04214740 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f84a3f7b0 0x555f84a41c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efd04214740 4 client.16296 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efd04214740 1 -- 172.21.15.5:0/2451498707 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f849979d0 con 0x555f84a3f7b0 2024-09-11T22:25:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.117+0000 7efcfaffd640 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f84a3f7b0 0x555f84a41c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.118+0000 7efcdaffd640 10 client.16296 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.118+0000 7efcfaffd640 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f84a3f7b0 0x555f84a41c40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9048c6e3f2d28973 server_cookie=125d061f69063818 in_seq=0 out_seq=0 2024-09-11T22:25:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.118+0000 7efcdaffd640 1 -- 172.21.15.5:0/2451498707 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f849979d0 con 0x555f84a3f7b0 2024-09-11T22:25:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.118+0000 7efcdaffd640 10 client.16296 handle_command_reply: tid=0 2024-09-11T22:25:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.118+0000 7efd04214740 1 client.16296 shutdown 2024-09-11T22:25:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 1 -- 172.21.15.5:0/2451498707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a25080 msgr2=0x555f84a2cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a25080 0x555f84a2cdd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7efcec012c80 tx=0x7efcec00f790 comp rx=0 tx=0).stop 2024-09-11T22:25:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 1 -- 172.21.15.5:0/2451498707 shutdown_connections 2024-09-11T22:25:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f84a3f7b0 0x555f84a41c40 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efcdaffd640 0 client.16296 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a2f940 0x555f84a2d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a25080 0x555f84a2cdd0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 1 --2- 172.21.15.5:0/2451498707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a24760 0x555f84a2c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 1 -- 172.21.15.5:0/2451498707 >> 172.21.15.5:0/2451498707 conn(0x555f84a20cf0 msgr2=0x555f84a269d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 1 -- 172.21.15.5:0/2451498707 shutdown_connections 2024-09-11T22:25:03.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 1 -- 172.21.15.5:0/2451498707 wait complete. 2024-09-11T22:25:03.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.119+0000 7efd04214740 20 client.16296 trim_cache size 0 max 16384 2024-09-11T22:25:03.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efd04214740 1 Processor -- start 2024-09-11T22:25:03.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efd04214740 1 -- start start 2024-09-11T22:25:03.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efd04214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 0x555f84a2e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efd04214740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a47550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efd04214740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a47a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efd04214740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1afba0 con 0x555f84a2f940 2024-09-11T22:25:03.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efd04214740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1b17a0 con 0x555f84a25080 2024-09-11T22:25:03.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efd04214740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1b0270 con 0x555f84a24760 2024-09-11T22:25:03.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efcfaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 0x555f84a2e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efcfaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 0x555f84a2e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:51194/0 (socket says 172.21.15.5:51194) 2024-09-11T22:25:03.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a47550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efcfaffd640 1 -- 172.21.15.5:0/1733854974 learned_addr learned my addr 172.21.15.5:0/1733854974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:03.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efcfaffd640 1 --2- 172.21.15.5:0/1733854974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a47a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efcfb7fe640 1 -- 172.21.15.5:0/1733854974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 msgr2=0x555f84a2e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/1733854974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 0x555f84a2e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efcfb7fe640 1 -- 172.21.15.5:0/1733854974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 msgr2=0x555f84a47a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/1733854974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a47a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.137+0000 7efcfb7fe640 1 -- 172.21.15.5:0/1733854974 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f84a1c260 con 0x555f84a25080 2024-09-11T22:25:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.138+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/1733854974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a47550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efcec000c00 tx=0x7efcec014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa00bca184f373 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.139+0000 7efce17fa640 1 -- 172.21.15.5:0/1733854974 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec004d20 con 0x555f84a25080 2024-09-11T22:25:03.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.139+0000 7efce17fa640 1 -- 172.21.15.5:0/1733854974 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf0005500 con 0x555f84a25080 2024-09-11T22:25:03.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.139+0000 7efce17fa640 1 -- 172.21.15.5:0/1733854974 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec01c950 con 0x555f84a25080 2024-09-11T22:25:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 -- 172.21.15.5:0/1733854974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 msgr2=0x555f84a47550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:03.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 --2- 172.21.15.5:0/1733854974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a47550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efcec000c00 tx=0x7efcec014000 comp rx=0 tx=0).stop 2024-09-11T22:25:03.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 -- 172.21.15.5:0/1733854974 shutdown_connections 2024-09-11T22:25:03.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 --2- 172.21.15.5:0/1733854974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a47a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 --2- 172.21.15.5:0/1733854974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a47550 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 --2- 172.21.15.5:0/1733854974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 0x555f84a2e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 -- 172.21.15.5:0/1733854974 >> 172.21.15.5:0/1733854974 conn(0x555f84a20cf0 msgr2=0x555f84a269d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:03.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 -- 172.21.15.5:0/1733854974 shutdown_connections 2024-09-11T22:25:03.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 -- 172.21.15.5:0/1733854974 wait complete. 2024-09-11T22:25:03.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 -- start start 2024-09-11T22:25:03.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.140+0000 7efd04214740 1 Processor -- start 2024-09-11T22:25:03.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efd04214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 0x555f84a348e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efd04214740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a34e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efd04214740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a35360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efd04214740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1b1010 con 0x555f84a2f940 2024-09-11T22:25:03.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efd04214740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1b0940 con 0x555f84a25080 2024-09-11T22:25:03.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efd04214740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efcfc1adb80 con 0x555f84a24760 2024-09-11T22:25:03.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efd04214740 10 client.0 _ll_register_callbacks cb 0x7efcec0128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:03.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a34e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a34e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42494/0 (socket says 172.21.15.5:42494) 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfaffd640 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a35360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfb7fe640 1 -- 172.21.15.5:0/1808090680 learned_addr learned my addr 172.21.15.5:0/1808090680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfaffd640 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 0x555f84a348e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfb7fe640 1 -- 172.21.15.5:0/1808090680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 msgr2=0x555f84a348e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 0x555f84a348e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfb7fe640 1 -- 172.21.15.5:0/1808090680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 msgr2=0x555f84a35360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a35360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfb7fe640 1 -- 172.21.15.5:0/1808090680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f84aaa530 con 0x555f84a25080 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a34e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efcec012c50 tx=0x7efcec004f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=231d03767f294b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.141+0000 7efcdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:25:03.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efcdaffd640 1 -- 172.21.15.5:0/1808090680 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcec005690 con 0x555f84a25080 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efcdaffd640 1 -- 172.21.15.5:0/1808090680 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7efcec02c030 con 0x555f84a25080 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efd04214740 1 -- 172.21.15.5:0/1808090680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf0005500 con 0x555f84a25080 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efcdaffd640 1 -- 172.21.15.5:0/1808090680 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efcec01c470 con 0x555f84a25080 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efd04214740 10 client.25826 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efd04214740 1 -- 172.21.15.5:0/1808090680 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555f84aab730 con 0x555f84a25080 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efcdaffd640 1 -- 172.21.15.5:0/1808090680 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcf0005500 con 0x555f84a25080 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efcdaffd640 1 -- 172.21.15.5:0/1808090680 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7efcec026460 con 0x555f84a25080 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efd04214740 10 client.25826 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efd04214740 10 client.25826 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efd04214740 20 client.25826 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efd04214740 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f84a3f7b0 0x555f84aadf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:03.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efd04214740 4 client.25826 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-11T22:25:03.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.142+0000 7efd04214740 1 -- 172.21.15.5:0/1808090680 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x555f84991d20 con 0x555f84a3f7b0 2024-09-11T22:25:03.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.143+0000 7efcfaffd640 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f84a3f7b0 0x555f84aadf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:03.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.143+0000 7efcfaffd640 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f84a3f7b0 0x555f84aadf20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dbddef92200078 server_cookie=3f51dddd32333ec7 in_seq=0 out_seq=0 2024-09-11T22:25:03.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.143+0000 7efcdaffd640 10 client.25826 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.247+0000 7efcdaffd640 1 -- 172.21.15.5:0/1808090680 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555f84991d20 con 0x555f84a3f7b0 2024-09-11T22:25:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.247+0000 7efcdaffd640 10 client.25826 handle_command_reply: tid=0 2024-09-11T22:25:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 client.25826 shutdown 2024-09-11T22:25:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 -- 172.21.15.5:0/1808090680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 msgr2=0x555f84a34e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a34e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efcec012c50 tx=0x7efcec004f30 comp rx=0 tx=0).stop 2024-09-11T22:25:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 -- 172.21.15.5:0/1808090680 shutdown_connections 2024-09-11T22:25:03.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f84a3f7b0 0x555f84aadf20 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f84a2f940 0x555f84a35360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f84a25080 0x555f84a34e20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 --2- 172.21.15.5:0/1808090680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f84a24760 0x555f84a348e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efcdaffd640 0 client.25826 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 -- 172.21.15.5:0/1808090680 >> 172.21.15.5:0/1808090680 conn(0x555f84a20cf0 msgr2=0x555f84a261b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 -- 172.21.15.5:0/1808090680 shutdown_connections 2024-09-11T22:25:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 1 -- 172.21.15.5:0/1808090680 wait complete. 2024-09-11T22:25:03.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.248+0000 7efd04214740 20 client.25826 trim_cache size 0 max 16384 2024-09-11T22:25:03.250 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 30, 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 10, 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "299bfba0": { 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 10, 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.7, 2024-09-11T22:25:03.251 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:25:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:25:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:25:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:25:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:25:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:25:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:03.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:03.253 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:25:03.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 -- 172.21.15.5:0/2110257718 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efce40a2d70 msgr2=0x7efce40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:03.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efce40a2d70 0x7efce40a5230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efcf40ede30 tx=0x7efcf4039000 comp rx=0 tx=0).stop 2024-09-11T22:25:03.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 -- 172.21.15.5:0/2110257718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc106f10 msgr2=0x7efcfc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:03.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc106f10 0x7efcfc074f70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7efcf4005e30 tx=0x7efcf4003f10 comp rx=0 tx=0).stop 2024-09-11T22:25:03.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 -- 172.21.15.5:0/2110257718 shutdown_connections 2024-09-11T22:25:03.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc106f10 0x7efcfc074f70 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efcfc102790 0x7efcfc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efce40a2d70 0x7efce40a5230 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 --2- 172.21.15.5:0/2110257718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efcfc101de0 0x7efcfc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:03.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 -- 172.21.15.5:0/2110257718 >> 172.21.15.5:0/2110257718 conn(0x7efcfc0fd8c0 msgr2=0x7efcfc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:03.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 -- 172.21.15.5:0/2110257718 shutdown_connections 2024-09-11T22:25:03.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:03.249+0000 7efd01e3b640 1 -- 172.21.15.5:0/2110257718 wait complete. 2024-09-11T22:25:03.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 12.29 MB/sec warmup 81 sec latency 142.282 ms 2024-09-11T22:25:03.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:02 smithi159 ceph-mon[35534]: pgmap v88: 97 pgs: 97 active+clean; 52 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 482 op/s 2024-09-11T22:25:03.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:02 smithi172 ceph-mon[35461]: pgmap v88: 97 pgs: 97 active+clean; 52 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.4 MiB/s wr, 482 op/s 2024-09-11T22:25:04.118 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '299bfba0', quiesce took 3.8 seconds. Will release after: 20.9 2024-09-11T22:25:04.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 12.15 MB/sec warmup 82 sec latency 1142.395 ms 2024-09-11T22:25:05.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 12.00 MB/sec warmup 83 sec latency 2142.504 ms 2024-09-11T22:25:05.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:05 smithi159 ceph-mon[35534]: pgmap v89: 97 pgs: 97 active+clean; 52 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-09-11T22:25:05.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:05 smithi172 ceph-mon[35461]: pgmap v89: 97 pgs: 97 active+clean; 52 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-09-11T22:25:05.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:05 smithi005 ceph-mon[30943]: pgmap v89: 97 pgs: 97 active+clean; 52 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-09-11T22:25:06.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 11.86 MB/sec warmup 84 sec latency 3142.624 ms 2024-09-11T22:25:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.411+0000 7f412cf3b640 1 -- 172.21.15.5:0/2786041070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4128101ad0 msgr2=0x7f4128101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.411+0000 7f412cf3b640 1 --2- 172.21.15.5:0/2786041070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4128101ad0 0x7f4128101f30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4110009930 tx=0x7f411002a420 comp rx=0 tx=0).stop 2024-09-11T22:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.411+0000 7f412cf3b640 1 -- 172.21.15.5:0/2786041070 shutdown_connections 2024-09-11T22:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.411+0000 7f412cf3b640 1 --2- 172.21.15.5:0/2786041070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4128102470 0x7f412810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.411+0000 7f412cf3b640 1 --2- 172.21.15.5:0/2786041070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4128101ad0 0x7f4128101f30 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.411+0000 7f412cf3b640 1 --2- 172.21.15.5:0/2786041070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4128107b70 0x7f4128107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.411+0000 7f412cf3b640 1 -- 172.21.15.5:0/2786041070 >> 172.21.15.5:0/2786041070 conn(0x7f41280fd7d0 msgr2=0x7f41280ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.411+0000 7f412cf3b640 1 -- 172.21.15.5:0/2786041070 shutdown_connections 2024-09-11T22:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.411+0000 7f412cf3b640 1 -- 172.21.15.5:0/2786041070 wait complete. 2024-09-11T22:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f412cf3b640 1 Processor -- start 2024-09-11T22:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f412cf3b640 1 -- start start 2024-09-11T22:25:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f412cf3b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4128101ad0 0x7f4128198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f412cf3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4128102470 0x7f4128198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f412cf3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4128107b70 0x7f412819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f412cf3b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4128111ee0 con 0x7f4128107b70 2024-09-11T22:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f412cf3b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4128111d60 con 0x7f4128101ad0 2024-09-11T22:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f412cf3b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4128112060 con 0x7f4128102470 2024-09-11T22:25:06.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f4126d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4128107b70 0x7f412819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f4126d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4128107b70 0x7f412819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45980/0 (socket says 172.21.15.5:45980) 2024-09-11T22:25:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f412659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4128101ad0 0x7f4128198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f4126d9d640 1 -- 172.21.15.5:0/121779220 learned_addr learned my addr 172.21.15.5:0/121779220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f411ffff640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4128102470 0x7f4128198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f4126d9d640 1 -- 172.21.15.5:0/121779220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4128102470 msgr2=0x7f4128198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f4126d9d640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4128102470 0x7f4128198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f4126d9d640 1 -- 172.21.15.5:0/121779220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4128101ad0 msgr2=0x7f4128198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f4126d9d640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4128101ad0 0x7f4128198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.412+0000 7f4126d9d640 1 -- 172.21.15.5:0/121779220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412819d310 con 0x7f4128107b70 2024-09-11T22:25:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.413+0000 7f412659c640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4128101ad0 0x7f4128198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.413+0000 7f4126d9d640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4128107b70 0x7f412819cbe0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f412000eaa0 tx=0x7f412000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.413+0000 7f411f7fe640 1 -- 172.21.15.5:0/121779220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412000cdb0 con 0x7f4128107b70 2024-09-11T22:25:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.413+0000 7f411f7fe640 1 -- 172.21.15.5:0/121779220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4120005020 con 0x7f4128107b70 2024-09-11T22:25:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.413+0000 7f412cf3b640 1 -- 172.21.15.5:0/121779220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412819d600 con 0x7f4128107b70 2024-09-11T22:25:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.413+0000 7f411f7fe640 1 -- 172.21.15.5:0/121779220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41200116b0 con 0x7f4128107b70 2024-09-11T22:25:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.413+0000 7f412cf3b640 1 -- 172.21.15.5:0/121779220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41281a4e90 con 0x7f4128107b70 2024-09-11T22:25:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.416+0000 7f411f7fe640 1 -- 172.21.15.5:0/121779220 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4120005560 con 0x7f4128107b70 2024-09-11T22:25:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.416+0000 7f412fcd6740 1 Processor -- start 2024-09-11T22:25:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f412fcd6740 1 -- start start 2024-09-11T22:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fd7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fd8410 0x563b12fe2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f412fcd6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563b12f3c2f0 con 0x563b12fe2cb0 2024-09-11T22:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f412fcd6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563b12ee4270 con 0x563b12fd7af0 2024-09-11T22:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f412fcd6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563b12f382b0 con 0x563b12fd8410 2024-09-11T22:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f411ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f411ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45984/0 (socket says 172.21.15.5:45984) 2024-09-11T22:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f411ffff640 1 -- 172.21.15.5:0/3496235566 learned_addr learned my addr 172.21.15.5:0/3496235566 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f412659c640 1 --2- 172.21.15.5:0/3496235566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fd8410 0x563b12fe2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f411ffff640 1 --2- 172.21.15.5:0/3496235566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fd7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f411f7fe640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f41140a2e20 0x7f41140a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f411ffff640 1 -- 172.21.15.5:0/3496235566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fd8410 msgr2=0x563b12fe2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f411ffff640 1 --2- 172.21.15.5:0/3496235566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fd8410 0x563b12fe2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f411ffff640 1 -- 172.21.15.5:0/3496235566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 msgr2=0x563b12fd7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f411ffff640 1 --2- 172.21.15.5:0/3496235566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fd7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f411ffff640 1 -- 172.21.15.5:0/3496235566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b12ee3340 con 0x563b12fe2cb0 2024-09-11T22:25:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.417+0000 7f4126d9d640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f41140a2e20 0x7f41140a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.418+0000 7f411f7fe640 1 -- 172.21.15.5:0/121779220 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f41200cf6a0 con 0x7f4128107b70 2024-09-11T22:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.418+0000 7f411ffff640 1 --2- 172.21.15.5:0/3496235566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe50a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4110031550 tx=0x7f4110005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b111eedf5e2fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.418+0000 7f411cff9640 1 -- 172.21.15.5:0/3496235566 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110035040 con 0x563b12fe2cb0 2024-09-11T22:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.418+0000 7f411cff9640 1 -- 172.21.15.5:0/3496235566 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4110004030 con 0x563b12fe2cb0 2024-09-11T22:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.418+0000 7f411cff9640 1 -- 172.21.15.5:0/3496235566 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41100039e0 con 0x563b12fe2cb0 2024-09-11T22:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 -- 172.21.15.5:0/3496235566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 msgr2=0x563b12fe50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:06.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f4126d9d640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f41140a2e20 0x7f41140a52e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4120097f80 tx=0x7f4120097e40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 --2- 172.21.15.5:0/3496235566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe50a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4110031550 tx=0x7f4110005b50 comp rx=0 tx=0).stop 2024-09-11T22:25:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 -- 172.21.15.5:0/3496235566 shutdown_connections 2024-09-11T22:25:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 --2- 172.21.15.5:0/3496235566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe50a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 --2- 172.21.15.5:0/3496235566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fd8410 0x563b12fe2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 --2- 172.21.15.5:0/3496235566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fd7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 -- 172.21.15.5:0/3496235566 >> 172.21.15.5:0/3496235566 conn(0x563b12fd4060 msgr2=0x563b12fd4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 -- 172.21.15.5:0/3496235566 shutdown_connections 2024-09-11T22:25:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 -- 172.21.15.5:0/3496235566 wait complete. 2024-09-11T22:25:06.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 -- start start 2024-09-11T22:25:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 Processor -- start 2024-09-11T22:25:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fd7af0 0x563b12fdfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd8410 0x563b12fe00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.419+0000 7f412fcd6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563b12fe5880 con 0x563b12fe2cb0 2024-09-11T22:25:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f412fcd6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563b12ec5290 con 0x563b12fd8410 2024-09-11T22:25:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f412fcd6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563b12fe5a00 con 0x563b12fd7af0 2024-09-11T22:25:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f412fcd6740 10 client.0 _ll_register_callbacks cb 0x563b12ef6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f411ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f411ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46000/0 (socket says 172.21.15.5:46000) 2024-09-11T22:25:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f411ffff640 1 -- 172.21.15.5:0/1586324373 learned_addr learned my addr 172.21.15.5:0/1586324373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f411ffff640 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fd7af0 0x563b12fdfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f412659c640 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd8410 0x563b12fe00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f411ffff640 1 -- 172.21.15.5:0/1586324373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fd7af0 msgr2=0x563b12fdfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f411ffff640 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fd7af0 0x563b12fdfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f411ffff640 1 -- 172.21.15.5:0/1586324373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd8410 msgr2=0x563b12fe00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f411ffff640 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd8410 0x563b12fe00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f411ffff640 1 -- 172.21.15.5:0/1586324373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b12c4f210 con 0x563b12fe2cb0 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f411ffff640 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe0630 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4110032d90 tx=0x7f411002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fd679065404326 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f410dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f410dffb640 1 -- 172.21.15.5:0/1586324373 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4110035070 con 0x563b12fe2cb0 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f410dffb640 1 -- 172.21.15.5:0/1586324373 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f4110051040 con 0x563b12fe2cb0 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.420+0000 7f412fcd6740 1 -- 172.21.15.5:0/1586324373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4110004030 con 0x563b12fe2cb0 2024-09-11T22:25:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f410dffb640 1 -- 172.21.15.5:0/1586324373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4110003760 con 0x563b12fe2cb0 2024-09-11T22:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f412fcd6740 10 client.16314 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f412fcd6740 1 -- 172.21.15.5:0/1586324373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b12d1b620 con 0x563b12fe2cb0 2024-09-11T22:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f410dffb640 1 -- 172.21.15.5:0/1586324373 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4110004030 con 0x563b12fe2cb0 2024-09-11T22:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f410dffb640 1 -- 172.21.15.5:0/1586324373 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4110054070 con 0x563b12fe2cb0 2024-09-11T22:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f412fcd6740 10 client.16314 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f412fcd6740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f412fcd6740 20 client.16314 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f412fcd6740 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563b12ff2ae0 0x563b12ff4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f412fcd6740 4 client.16314 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f412fcd6740 1 -- 172.21.15.5:0/1586324373 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b12f49a20 con 0x563b12ff2ae0 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.421+0000 7f412659c640 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563b12ff2ae0 0x563b12ff4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.422+0000 7f410dffb640 10 client.16314 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.422+0000 7f412659c640 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563b12ff2ae0 0x563b12ff4ed0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2032d6fb53053d server_cookie=675c528b1eeeaf72 in_seq=0 out_seq=0 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.422+0000 7f410dffb640 1 -- 172.21.15.5:0/1586324373 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b12f49a20 con 0x563b12ff2ae0 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.422+0000 7f410dffb640 10 client.16314 handle_command_reply: tid=0 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.422+0000 7f412fcd6740 1 client.16314 shutdown 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 1 -- 172.21.15.5:0/1586324373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 msgr2=0x563b12fe0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe0630 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4110032d90 tx=0x7f411002a420 comp rx=0 tx=0).stop 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 1 -- 172.21.15.5:0/1586324373 shutdown_connections 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563b12ff2ae0 0x563b12ff4ed0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fe2cb0 0x563b12fe0630 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd8410 0x563b12fe00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 1 --2- 172.21.15.5:0/1586324373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fd7af0 0x563b12fdfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f410dffb640 0 client.16314 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 1 -- 172.21.15.5:0/1586324373 >> 172.21.15.5:0/1586324373 conn(0x563b12fd4060 msgr2=0x563b12fd9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 1 -- 172.21.15.5:0/1586324373 shutdown_connections 2024-09-11T22:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 1 -- 172.21.15.5:0/1586324373 wait complete. 2024-09-11T22:25:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.423+0000 7f412fcd6740 20 client.16314 trim_cache size 0 max 16384 2024-09-11T22:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.437+0000 7f412fcd6740 1 Processor -- start 2024-09-11T22:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.437+0000 7f412fcd6740 1 -- start start 2024-09-11T22:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fda560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12fe0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fe2cb0 0x563b12feb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412fcd6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41281ad930 con 0x563b12fd8410 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412fcd6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41281af530 con 0x563b12fd7af0 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412fcd6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41281ae000 con 0x563b12fe2cb0 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12fe0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12fe0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46004/0 (socket says 172.21.15.5:46004) 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412659c640 1 -- 172.21.15.5:0/1454408682 learned_addr learned my addr 172.21.15.5:0/1454408682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412659c640 1 -- 172.21.15.5:0/1454408682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fe2cb0 msgr2=0x563b12feb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412659c640 1 --2- 172.21.15.5:0/1454408682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fe2cb0 0x563b12feb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412659c640 1 -- 172.21.15.5:0/1454408682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 msgr2=0x563b12fda560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412659c640 1 --2- 172.21.15.5:0/1454408682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fda560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412659c640 1 -- 172.21.15.5:0/1454408682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b12fde100 con 0x563b12fd8410 2024-09-11T22:25:06.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f412659c640 1 --2- 172.21.15.5:0/1454408682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12fe0460 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x563b12fdf420 tx=0x7f41180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea27ecb01dac3d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f410ffff640 1 -- 172.21.15.5:0/1454408682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118012310 con 0x563b12fd8410 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f410ffff640 1 -- 172.21.15.5:0/1454408682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41180124b0 con 0x563b12fd8410 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.438+0000 7f410ffff640 1 -- 172.21.15.5:0/1454408682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118010650 con 0x563b12fd8410 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 -- 172.21.15.5:0/1454408682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 msgr2=0x563b12fe0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 --2- 172.21.15.5:0/1454408682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12fe0460 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x563b12fdf420 tx=0x7f41180046c0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 -- 172.21.15.5:0/1454408682 shutdown_connections 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 --2- 172.21.15.5:0/1454408682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fe2cb0 0x563b12feb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 --2- 172.21.15.5:0/1454408682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12fe0460 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 --2- 172.21.15.5:0/1454408682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fda560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 -- 172.21.15.5:0/1454408682 >> 172.21.15.5:0/1454408682 conn(0x563b12fd4060 msgr2=0x563b12fe1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:06.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 -- 172.21.15.5:0/1454408682 shutdown_connections 2024-09-11T22:25:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 -- 172.21.15.5:0/1454408682 wait complete. 2024-09-11T22:25:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 -- start start 2024-09-11T22:25:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.439+0000 7f412fcd6740 1 Processor -- start 2024-09-11T22:25:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fdd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12ffa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412fcd6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fe2cb0 0x563b12ffa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412fcd6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41281aeda0 con 0x563b12fd8410 2024-09-11T22:25:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412fcd6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41281ae6d0 con 0x563b12fd7af0 2024-09-11T22:25:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412fcd6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f41281ab910 con 0x563b12fe2cb0 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412fcd6740 10 client.0 _ll_register_callbacks cb 0x7f4118009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12ffa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12ffa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46012/0 (socket says 172.21.15.5:46012) 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412659c640 1 -- 172.21.15.5:0/3074732277 learned_addr learned my addr 172.21.15.5:0/3074732277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f411ffff640 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fdd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f411ffff640 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fe2cb0 0x563b12ffa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412659c640 1 -- 172.21.15.5:0/3074732277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fe2cb0 msgr2=0x563b12ffa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412659c640 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fe2cb0 0x563b12ffa8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412659c640 1 -- 172.21.15.5:0/3074732277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 msgr2=0x563b12fdd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412659c640 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fdd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412659c640 1 -- 172.21.15.5:0/3074732277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b13060500 con 0x563b12fd8410 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f410dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412659c640 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12ffa130 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f41180045b0 tx=0x7f41180045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91408e50857eb54e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f410dffb640 1 -- 172.21.15.5:0/3074732277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118035060 con 0x563b12fd8410 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f410dffb640 1 -- 172.21.15.5:0/3074732277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f4118004b60 con 0x563b12fd8410 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f412fcd6740 1 -- 172.21.15.5:0/3074732277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563b12f43d70 con 0x563b12fd8410 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.440+0000 7f410dffb640 1 -- 172.21.15.5:0/3074732277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4118004d60 con 0x563b12fd8410 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f412fcd6740 10 client.16320 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f412fcd6740 1 -- 172.21.15.5:0/3074732277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b130616d0 con 0x563b12fd8410 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f410dffb640 1 -- 172.21.15.5:0/3074732277 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563b12f43d70 con 0x563b12fd8410 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f410dffb640 1 -- 172.21.15.5:0/3074732277 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4118010a90 con 0x563b12fd8410 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f412fcd6740 10 client.16320 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f412fcd6740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f412fcd6740 20 client.16320 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f412fcd6740 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563b13061eb0 0x563b130642d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f412fcd6740 4 client.16320 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:25:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f412fcd6740 1 -- 172.21.15.5:0/3074732277 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b12ec0900 con 0x563b13061eb0 2024-09-11T22:25:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.441+0000 7f411ffff640 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563b13061eb0 0x563b130642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.442+0000 7f410dffb640 10 client.16320 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.442+0000 7f411ffff640 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563b13061eb0 0x563b130642d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=422c017f257db62f server_cookie=85f91b69330afc51 in_seq=0 out_seq=0 2024-09-11T22:25:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f410dffb640 1 -- 172.21.15.5:0/3074732277 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b12ec0900 con 0x563b13061eb0 2024-09-11T22:25:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f410dffb640 10 client.16320 handle_command_reply: tid=0 2024-09-11T22:25:06.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f412fcd6740 1 client.16320 shutdown 2024-09-11T22:25:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f412fcd6740 1 -- 172.21.15.5:0/3074732277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 msgr2=0x563b12ffa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f412fcd6740 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12ffa130 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f41180045b0 tx=0x7f41180045e0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f412fcd6740 1 -- 172.21.15.5:0/3074732277 shutdown_connections 2024-09-11T22:25:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f412fcd6740 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563b13061eb0 0x563b130642d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f412fcd6740 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563b12fe2cb0 0x563b12ffa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f412fcd6740 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b12fd8410 0x563b12ffa130 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f410dffb640 0 client.16320 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:06.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f412fcd6740 1 --2- 172.21.15.5:0/3074732277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563b12fd7af0 0x563b12fdd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f412fcd6740 1 -- 172.21.15.5:0/3074732277 >> 172.21.15.5:0/3074732277 conn(0x563b12fd4060 msgr2=0x563b12fe1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.531+0000 7f412fcd6740 1 -- 172.21.15.5:0/3074732277 shutdown_connections 2024-09-11T22:25:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412fcd6740 1 -- 172.21.15.5:0/3074732277 wait complete. 2024-09-11T22:25:06.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412fcd6740 20 client.16320 trim_cache size 0 max 16384 2024-09-11T22:25:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:25:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:25:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:25:06.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:25:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 -- 172.21.15.5:0/121779220 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f41140a2e20 msgr2=0x7f41140a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f41140a2e20 0x7f41140a52e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4120097f80 tx=0x7f4120097e40 comp rx=0 tx=0).stop 2024-09-11T22:25:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 -- 172.21.15.5:0/121779220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4128107b70 msgr2=0x7f412819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4128107b70 0x7f412819cbe0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f412000eaa0 tx=0x7f412000ef60 comp rx=0 tx=0).stop 2024-09-11T22:25:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 -- 172.21.15.5:0/121779220 shutdown_connections 2024-09-11T22:25:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4128107b70 0x7f412819cbe0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4128102470 0x7f4128198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f41140a2e20 0x7f41140a52e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 --2- 172.21.15.5:0/121779220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4128101ad0 0x7f4128198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:06.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 -- 172.21.15.5:0/121779220 >> 172.21.15.5:0/121779220 conn(0x7f41280fd7d0 msgr2=0x7f41280fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 -- 172.21.15.5:0/121779220 shutdown_connections 2024-09-11T22:25:06.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:06.532+0000 7f412cf3b640 1 -- 172.21.15.5:0/121779220 wait complete. 2024-09-11T22:25:07.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 11.72 MB/sec warmup 85 sec latency 4142.687 ms 2024-09-11T22:25:07.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:07 smithi159 ceph-mon[35534]: pgmap v90: 97 pgs: 97 active+clean; 52 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-09-11T22:25:07.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:07 smithi172 ceph-mon[35461]: pgmap v90: 97 pgs: 97 active+clean; 52 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-09-11T22:25:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:07 smithi005 ceph-mon[30943]: pgmap v90: 97 pgs: 97 active+clean; 52 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 344 op/s 2024-09-11T22:25:08.309 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 11.58 MB/sec warmup 86 sec latency 5142.777 ms 2024-09-11T22:25:09.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 11.45 MB/sec warmup 87 sec latency 6142.852 ms 2024-09-11T22:25:09.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:09 smithi172 ceph-mon[35461]: pgmap v91: 97 pgs: 97 active+clean; 42 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 452 op/s 2024-09-11T22:25:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:09 smithi005 ceph-mon[30943]: pgmap v91: 97 pgs: 97 active+clean; 42 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 452 op/s 2024-09-11T22:25:09.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:09 smithi159 ceph-mon[35534]: pgmap v91: 97 pgs: 97 active+clean; 42 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 452 op/s 2024-09-11T22:25:10.308 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 11.32 MB/sec warmup 88 sec latency 7142.974 ms 2024-09-11T22:25:11.309 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 11.19 MB/sec warmup 89 sec latency 8143.078 ms 2024-09-11T22:25:11.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:11 smithi172 ceph-mon[35461]: pgmap v92: 97 pgs: 97 active+clean; 42 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.1 MiB/s wr, 237 op/s 2024-09-11T22:25:11.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:11 smithi005 ceph-mon[30943]: pgmap v92: 97 pgs: 97 active+clean; 42 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.1 MiB/s wr, 237 op/s 2024-09-11T22:25:11.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:11 smithi159 ceph-mon[35534]: pgmap v92: 97 pgs: 97 active+clean; 42 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.1 MiB/s wr, 237 op/s 2024-09-11T22:25:12.309 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 11.07 MB/sec warmup 90 sec latency 9143.137 ms 2024-09-11T22:25:13.309 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 10.94 MB/sec warmup 91 sec latency 10143.236 ms 2024-09-11T22:25:13.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:13 smithi172 ceph-mon[35461]: pgmap v93: 97 pgs: 97 active+clean; 54 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 260 op/s 2024-09-11T22:25:13.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:13 smithi005 ceph-mon[30943]: pgmap v93: 97 pgs: 97 active+clean; 54 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 260 op/s 2024-09-11T22:25:13.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:13 smithi159 ceph-mon[35534]: pgmap v93: 97 pgs: 97 active+clean; 54 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 260 op/s 2024-09-11T22:25:14.309 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 10.82 MB/sec warmup 92 sec latency 11143.350 ms 2024-09-11T22:25:15.309 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 10.71 MB/sec warmup 93 sec latency 12143.455 ms 2024-09-11T22:25:15.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:15 smithi172 ceph-mon[35461]: pgmap v94: 97 pgs: 97 active+clean; 54 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-11T22:25:15.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:15 smithi005 ceph-mon[30943]: pgmap v94: 97 pgs: 97 active+clean; 54 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-11T22:25:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:15 smithi159 ceph-mon[35534]: pgmap v94: 97 pgs: 97 active+clean; 54 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-11T22:25:16.309 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 10.59 MB/sec warmup 94 sec latency 13143.575 ms 2024-09-11T22:25:17.309 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 10.48 MB/sec warmup 95 sec latency 14143.718 ms 2024-09-11T22:25:17.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:17 smithi172 ceph-mon[35461]: pgmap v95: 97 pgs: 97 active+clean; 54 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-11T22:25:17.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:17 smithi005 ceph-mon[30943]: pgmap v95: 97 pgs: 97 active+clean; 54 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-11T22:25:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:17 smithi159 ceph-mon[35534]: pgmap v95: 97 pgs: 97 active+clean; 54 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-09-11T22:25:18.309 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 10.37 MB/sec warmup 96 sec latency 15143.854 ms 2024-09-11T22:25:19.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 10.27 MB/sec warmup 97 sec latency 16143.947 ms 2024-09-11T22:25:19.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:19 smithi172 ceph-mon[35461]: pgmap v96: 97 pgs: 97 active+clean; 54 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-11T22:25:19.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:19 smithi005 ceph-mon[30943]: pgmap v96: 97 pgs: 97 active+clean; 54 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-11T22:25:19.578 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:19 smithi159 ceph-mon[35534]: pgmap v96: 97 pgs: 97 active+clean; 54 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-11T22:25:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.640+0000 7f17d8d86640 1 -- 172.21.15.5:0/343088417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d4106a00 msgr2=0x7f17d4106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.640+0000 7f17d8d86640 1 --2- 172.21.15.5:0/343088417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d4106a00 0x7f17d4106de0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009930 tx=0x7f17c402a420 comp rx=0 tx=0).stop 2024-09-11T22:25:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.640+0000 7f17d8d86640 1 -- 172.21.15.5:0/343088417 shutdown_connections 2024-09-11T22:25:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.640+0000 7f17d8d86640 1 --2- 172.21.15.5:0/343088417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d4101390 0x7f17d410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.640+0000 7f17d8d86640 1 --2- 172.21.15.5:0/343088417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d41009f0 0x7f17d4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.640+0000 7f17d8d86640 1 --2- 172.21.15.5:0/343088417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d4106a00 0x7f17d4106de0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.640+0000 7f17d8d86640 1 -- 172.21.15.5:0/343088417 >> 172.21.15.5:0/343088417 conn(0x7f17d40fc6b0 msgr2=0x7f17d40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.640+0000 7f17d8d86640 1 -- 172.21.15.5:0/343088417 shutdown_connections 2024-09-11T22:25:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.640+0000 7f17d8d86640 1 -- 172.21.15.5:0/343088417 wait complete. 2024-09-11T22:25:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.641+0000 7f17d8d86640 1 Processor -- start 2024-09-11T22:25:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.641+0000 7f17d8d86640 1 -- start start 2024-09-11T22:25:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d8d86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41009f0 0x7f17d41991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d8d86640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d4101390 0x7f17d4199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d8d86640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d4106a00 0x7f17d41a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d8d86640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4110e90 con 0x7f17d41009f0 2024-09-11T22:25:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d8d86640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4110d10 con 0x7f17d4101390 2024-09-11T22:25:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d8d86640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4111010 con 0x7f17d4106a00 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41009f0 0x7f17d41991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41009f0 0x7f17d41991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43158/0 (socket says 172.21.15.5:43158) 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d259c640 1 -- 172.21.15.5:0/671337100 learned_addr learned my addr 172.21.15.5:0/671337100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d2d9d640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d4106a00 0x7f17d41a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d1d9b640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d4101390 0x7f17d4199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d259c640 1 -- 172.21.15.5:0/671337100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d4106a00 msgr2=0x7f17d41a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d259c640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d4106a00 0x7f17d41a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d259c640 1 -- 172.21.15.5:0/671337100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d4101390 msgr2=0x7f17d4199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d259c640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d4101390 0x7f17d4199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d259c640 1 -- 172.21.15.5:0/671337100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d41a47a0 con 0x7f17d41009f0 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17d259c640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41009f0 0x7f17d41991e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f17c402a940 tx=0x7f17c4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.642+0000 7f17c37fe640 1 -- 172.21.15.5:0/671337100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c403f070 con 0x7f17d41009f0 2024-09-11T22:25:19.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.643+0000 7f17c37fe640 1 -- 172.21.15.5:0/671337100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c4031c90 con 0x7f17d41009f0 2024-09-11T22:25:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.643+0000 7f17d8d86640 1 -- 172.21.15.5:0/671337100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d41a4970 con 0x7f17d41009f0 2024-09-11T22:25:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.643+0000 7f17c37fe640 1 -- 172.21.15.5:0/671337100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4035030 con 0x7f17d41009f0 2024-09-11T22:25:19.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.643+0000 7f17d8d86640 1 -- 172.21.15.5:0/671337100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d41a4dd0 con 0x7f17d41009f0 2024-09-11T22:25:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.645+0000 7f17c37fe640 1 -- 172.21.15.5:0/671337100 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f17c4042070 con 0x7f17d41009f0 2024-09-11T22:25:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.645+0000 7f17dbbfa740 1 Processor -- start 2024-09-11T22:25:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17dbbfa740 1 -- start start 2024-09-11T22:25:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb127add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1289b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb128c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55edb11df2f0 con 0x55edb127b310 2024-09-11T22:25:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55edb1187270 con 0x55edb127a9f0 2024-09-11T22:25:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55edb11db2b0 con 0x55edb128a0c0 2024-09-11T22:25:19.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1289b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1289b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43168/0 (socket says 172.21.15.5:43168) 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d259c640 1 -- 172.21.15.5:0/1313713882 learned_addr learned my addr 172.21.15.5:0/1313713882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d2d9d640 1 --2- 172.21.15.5:0/1313713882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb127add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d1d9b640 1 --2- 172.21.15.5:0/1313713882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb128c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17c37fe640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17bc0a2da0 0x7f17bc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d2d9d640 1 -- 172.21.15.5:0/1313713882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 msgr2=0x55edb128c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d2d9d640 1 --2- 172.21.15.5:0/1313713882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb128c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d2d9d640 1 -- 172.21.15.5:0/1313713882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 msgr2=0x55edb1289b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d2d9d640 1 --2- 172.21.15.5:0/1313713882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1289b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d2d9d640 1 -- 172.21.15.5:0/1313713882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb1186340 con 0x55edb127a9f0 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d259c640 1 --2- 172.21.15.5:0/1313713882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1289b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17d2d9d640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17bc0a2da0 0x7f17bc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.646+0000 7f17c37fe640 1 -- 172.21.15.5:0/671337100 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f17c40efc30 con 0x7f17d41009f0 2024-09-11T22:25:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.647+0000 7f17d2d9d640 1 --2- 172.21.15.5:0/1313713882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb127add0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f17cc007ec0 tx=0x7f17cc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42fa432a6cb73370 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.647+0000 7f17d2d9d640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17bc0a2da0 0x7f17bc0a5260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f17cc020000 tx=0x7f17cc01d000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.647+0000 7f17c0ff9640 1 -- 172.21.15.5:0/1313713882 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0122d0 con 0x55edb127a9f0 2024-09-11T22:25:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.647+0000 7f17c0ff9640 1 -- 172.21.15.5:0/1313713882 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17cc019a40 con 0x55edb127a9f0 2024-09-11T22:25:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.647+0000 7f17c0ff9640 1 -- 172.21.15.5:0/1313713882 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc010860 con 0x55edb127a9f0 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 -- 172.21.15.5:0/1313713882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 msgr2=0x55edb127add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/1313713882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb127add0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f17cc007ec0 tx=0x7f17cc00f790 comp rx=0 tx=0).stop 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 -- 172.21.15.5:0/1313713882 shutdown_connections 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/1313713882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb128c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/1313713882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1289b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/1313713882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb127add0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 -- 172.21.15.5:0/1313713882 >> 172.21.15.5:0/1313713882 conn(0x55edb1277060 msgr2=0x55edb1277440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 -- 172.21.15.5:0/1313713882 shutdown_connections 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 -- 172.21.15.5:0/1313713882 wait complete. 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 -- start start 2024-09-11T22:25:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 Processor -- start 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb1287070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb12875b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb1287af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55edb128cd20 con 0x55edb127b310 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55edb1168290 con 0x55edb127a9f0 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55edb128cea0 con 0x55edb128a0c0 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.648+0000 7f17dbbfa740 10 client.0 _ll_register_callbacks cb 0x55edb1199140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb12875b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb1287af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb12875b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43174/0 (socket says 172.21.15.5:43174) 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d1d9b640 1 -- 172.21.15.5:0/3226000092 learned_addr learned my addr 172.21.15.5:0/3226000092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d259c640 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb1287070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d1d9b640 1 -- 172.21.15.5:0/3226000092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 msgr2=0x55edb1287af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d1d9b640 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb1287af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d1d9b640 1 -- 172.21.15.5:0/3226000092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 msgr2=0x55edb1287070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d1d9b640 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb1287070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d1d9b640 1 -- 172.21.15.5:0/3226000092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb0ef2210 con 0x55edb127b310 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17d1d9b640 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb12875b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f17c800da10 tx=0x7f17c800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbeaa5f404f3ef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3226000092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8011640 con 0x55edb127b310 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3226000092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f17c80117e0 con 0x55edb127b310 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3226000092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17cc019a40 con 0x55edb127b310 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3226000092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17c80119e0 con 0x55edb127b310 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17dbbfa740 10 client.16332 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.649+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3226000092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55edb0fbe620 con 0x55edb127b310 2024-09-11T22:25:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.650+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3226000092 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17cc019a40 con 0x55edb127b310 2024-09-11T22:25:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.650+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3226000092 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f17c8019030 con 0x55edb127b310 2024-09-11T22:25:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.650+0000 7f17dbbfa740 10 client.16332 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.650+0000 7f17dbbfa740 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.650+0000 7f17dbbfa740 20 client.16332 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.650+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55edb129a2b0 0x55edb129c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.650+0000 7f17dbbfa740 4 client.16332 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.650+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3226000092 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edb11eca20 con 0x55edb129a2b0 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.650+0000 7f17d259c640 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55edb129a2b0 0x55edb129c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17d259c640 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55edb129a2b0 0x55edb129c750 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd349d5ebdc03bcb server_cookie=5e5fb896c322080d in_seq=0 out_seq=0 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17b9ffb640 10 client.16332 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3226000092 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edb11eca20 con 0x55edb129a2b0 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17b9ffb640 10 client.16332 handle_command_reply: tid=0 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17dbbfa740 1 client.16332 shutdown 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3226000092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 msgr2=0x55edb12875b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb12875b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f17c800da10 tx=0x7f17c800ded0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3226000092 shutdown_connections 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55edb129a2b0 0x55edb129c750 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb1287af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb12875b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3226000092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb1287070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17b9ffb640 0 client.16332 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.651+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3226000092 >> 172.21.15.5:0/3226000092 conn(0x55edb1277060 msgr2=0x55edb127cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.652+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3226000092 shutdown_connections 2024-09-11T22:25:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.652+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3226000092 wait complete. 2024-09-11T22:25:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.652+0000 7f17dbbfa740 20 client.16332 trim_cache size 0 max 16384 2024-09-11T22:25:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17dbbfa740 1 Processor -- start 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17dbbfa740 1 -- start start 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb127a9f0 0x55edb12a2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127b310 0x55edb12a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb128a0c0 0x55edb12842b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41ab6e0 con 0x55edb128a0c0 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41ad2e0 con 0x55edb127b310 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41abdb0 con 0x55edb127a9f0 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17d2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb128a0c0 0x55edb12842b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17d2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb128a0c0 0x55edb12842b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43182/0 (socket says 172.21.15.5:43182) 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.666+0000 7f17d2d9d640 1 -- 172.21.15.5:0/3447158558 learned_addr learned my addr 172.21.15.5:0/3447158558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17d259c640 1 --2- 172.21.15.5:0/3447158558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb127a9f0 0x55edb12a2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17d1d9b640 1 --2- 172.21.15.5:0/3447158558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127b310 0x55edb12a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17d2d9d640 1 -- 172.21.15.5:0/3447158558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb127a9f0 msgr2=0x55edb12a2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17d2d9d640 1 --2- 172.21.15.5:0/3447158558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb127a9f0 0x55edb12a2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17d2d9d640 1 -- 172.21.15.5:0/3447158558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127b310 msgr2=0x55edb12a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17d2d9d640 1 --2- 172.21.15.5:0/3447158558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127b310 0x55edb12a25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17d2d9d640 1 -- 172.21.15.5:0/3447158558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edb127b140 con 0x55edb128a0c0 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17d259c640 1 --2- 172.21.15.5:0/3447158558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb127a9f0 0x55edb12a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17d2d9d640 1 --2- 172.21.15.5:0/3447158558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb128a0c0 0x55edb12842b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17cc012740 tx=0x7f17cc01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32180d7ac9da8d3d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17bbfff640 1 -- 172.21.15.5:0/3447158558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc035070 con 0x55edb128a0c0 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17bbfff640 1 -- 172.21.15.5:0/3447158558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17cc012250 con 0x55edb128a0c0 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.667+0000 7f17bbfff640 1 -- 172.21.15.5:0/3447158558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc019560 con 0x55edb128a0c0 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447158558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb128a0c0 msgr2=0x55edb12842b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3447158558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb128a0c0 0x55edb12842b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17cc012740 tx=0x7f17cc01c040 comp rx=0 tx=0).stop 2024-09-11T22:25:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447158558 shutdown_connections 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3447158558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb128a0c0 0x55edb12842b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3447158558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127b310 0x55edb12a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3447158558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb127a9f0 0x55edb12a2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447158558 >> 172.21.15.5:0/3447158558 conn(0x55edb1277060 msgr2=0x55edb1286140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447158558 shutdown_connections 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447158558 wait complete. 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 -- start start 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 Processor -- start 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb1292510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1292a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb128da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41acb50 con 0x55edb127b310 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41ac480 con 0x55edb127a9f0 2024-09-11T22:25:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41a96c0 con 0x55edb128a0c0 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17dbbfa740 10 client.0 _ll_register_callbacks cb 0x7f17c8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.668+0000 7f17d1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1292a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17d1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1292a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43186/0 (socket says 172.21.15.5:43186) 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17d1d9b640 1 -- 172.21.15.5:0/3447224520 learned_addr learned my addr 172.21.15.5:0/3447224520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17d259c640 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb1292510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17d2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb128da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17d1d9b640 1 -- 172.21.15.5:0/3447224520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 msgr2=0x55edb128da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17d1d9b640 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb128da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17d1d9b640 1 -- 172.21.15.5:0/3447224520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 msgr2=0x55edb1292510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17d1d9b640 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb1292510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17d1d9b640 1 -- 172.21.15.5:0/3447224520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edb128e2b0 con 0x55edb127b310 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17d1d9b640 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1292a50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f17c8000c00 tx=0x7f17c80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6070c419951c6ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3447224520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c8026030 con 0x55edb127b310 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3447224520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f17c8005040 con 0x55edb127b310 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447224520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17cc012250 con 0x55edb127b310 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3447224520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f17c8038050 con 0x55edb127b310 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17dbbfa740 10 client.16338 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.669+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447224520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55edb1304d00 con 0x55edb127b310 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.670+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3447224520 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17cc012250 con 0x55edb127b310 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.670+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3447224520 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f17c8019040 con 0x55edb127b310 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.670+0000 7f17dbbfa740 10 client.16338 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.670+0000 7f17dbbfa740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.670+0000 7f17dbbfa740 20 client.16338 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.670+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55edb13055c0 0x55edb13079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.670+0000 7f17dbbfa740 4 client.16338 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:25:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.670+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447224520 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edb11e6d70 con 0x55edb13055c0 2024-09-11T22:25:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.670+0000 7f17d259c640 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55edb13055c0 0x55edb13079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.671+0000 7f17b9ffb640 10 client.16338 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.671+0000 7f17d259c640 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55edb13055c0 0x55edb13079e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=360cb21e1ed19169 server_cookie=45219eb2afee849b in_seq=0 out_seq=0 2024-09-11T22:25:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17b9ffb640 1 -- 172.21.15.5:0/3447224520 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55edb11e6d70 con 0x55edb13055c0 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17b9ffb640 10 client.16338 handle_command_reply: tid=0 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17dbbfa740 1 client.16338 shutdown 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447224520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 msgr2=0x55edb1292a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1292a50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f17c8000c00 tx=0x7f17c80039c0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447224520 shutdown_connections 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55edb13055c0 0x55edb13079e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55edb128a0c0 0x55edb128da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edb127b310 0x55edb1292a50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17b9ffb640 0 client.16338 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17dbbfa740 1 --2- 172.21.15.5:0/3447224520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edb127a9f0 0x55edb1292510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447224520 >> 172.21.15.5:0/3447224520 conn(0x55edb1277060 msgr2=0x55edb127c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.706+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447224520 shutdown_connections 2024-09-11T22:25:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17dbbfa740 1 -- 172.21.15.5:0/3447224520 wait complete. 2024-09-11T22:25:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17dbbfa740 20 client.16338 trim_cache size 0 max 16384 2024-09-11T22:25:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:25:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:25:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:25:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:25:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 -- 172.21.15.5:0/671337100 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17bc0a2da0 msgr2=0x7f17bc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17bc0a2da0 0x7f17bc0a5260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f17cc020000 tx=0x7f17cc01d000 comp rx=0 tx=0).stop 2024-09-11T22:25:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 -- 172.21.15.5:0/671337100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41009f0 msgr2=0x7f17d41991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41009f0 0x7f17d41991e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f17c402a940 tx=0x7f17c4003b90 comp rx=0 tx=0).stop 2024-09-11T22:25:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 -- 172.21.15.5:0/671337100 shutdown_connections 2024-09-11T22:25:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17d4106a00 0x7f17d41a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17bc0a2da0 0x7f17bc0a5260 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17d4101390 0x7f17d4199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 --2- 172.21.15.5:0/671337100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41009f0 0x7f17d41991e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 -- 172.21.15.5:0/671337100 >> 172.21.15.5:0/671337100 conn(0x7f17d40fc6b0 msgr2=0x7f17d40fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 -- 172.21.15.5:0/671337100 shutdown_connections 2024-09-11T22:25:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:19.707+0000 7f17d8d86640 1 -- 172.21.15.5:0/671337100 wait complete. 2024-09-11T22:25:20.309 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 10.16 MB/sec warmup 98 sec latency 17144.043 ms 2024-09-11T22:25:21.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 10.06 MB/sec warmup 99 sec latency 18144.131 ms 2024-09-11T22:25:21.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:21 smithi172 ceph-mon[35461]: pgmap v97: 97 pgs: 97 active+clean; 54 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.1 MiB/s wr, 65 op/s 2024-09-11T22:25:21.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:21 smithi005 ceph-mon[30943]: pgmap v97: 97 pgs: 97 active+clean; 54 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.1 MiB/s wr, 65 op/s 2024-09-11T22:25:21.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:21 smithi159 ceph-mon[35534]: pgmap v97: 97 pgs: 97 active+clean; 54 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.1 MiB/s wr, 65 op/s 2024-09-11T22:25:22.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 9.96 MB/sec warmup 100 sec latency 19144.245 ms 2024-09-11T22:25:22.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:25:22.787 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:23.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 9.86 MB/sec warmup 101 sec latency 20144.289 ms 2024-09-11T22:25:23.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:23 smithi005 ceph-mon[30943]: pgmap v98: 97 pgs: 97 active+clean; 52 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 3.0 MiB/s wr, 117 op/s 2024-09-11T22:25:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.300+0000 7f27d5194640 1 -- 172.21.15.5:0/3381380177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27d01036e0 msgr2=0x7f27d0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.300+0000 7f27d5194640 1 --2- 172.21.15.5:0/3381380177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27d01036e0 0x7f27d0107b90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009930 tx=0x7f27b802a420 comp rx=0 tx=0).stop 2024-09-11T22:25:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.300+0000 7f27d5194640 1 -- 172.21.15.5:0/3381380177 shutdown_connections 2024-09-11T22:25:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.300+0000 7f27d5194640 1 --2- 172.21.15.5:0/3381380177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27d01082a0 0x7f27d010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.300+0000 7f27d5194640 1 --2- 172.21.15.5:0/3381380177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27d01036e0 0x7f27d0107b90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.300+0000 7f27d5194640 1 --2- 172.21.15.5:0/3381380177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27d0102d30 0x7f27d0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.300+0000 7f27d5194640 1 -- 172.21.15.5:0/3381380177 >> 172.21.15.5:0/3381380177 conn(0x7f27d00fc7d0 msgr2=0x7f27d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.300+0000 7f27d5194640 1 -- 172.21.15.5:0/3381380177 shutdown_connections 2024-09-11T22:25:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.300+0000 7f27d5194640 1 -- 172.21.15.5:0/3381380177 wait complete. 2024-09-11T22:25:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.300+0000 7f27d5194640 1 Processor -- start 2024-09-11T22:25:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27d5194640 1 -- start start 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27d5194640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27d0102d30 0x7f27d0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27d5194640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27d01036e0 0x7f27d0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27d5194640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27d01082a0 0x7f27d0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27d5194640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01120a0 con 0x7f27d0102d30 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27d5194640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0111f20 con 0x7f27d01082a0 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27d5194640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0112220 con 0x7f27d01036e0 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27d0102d30 0x7f27d0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27d0102d30 0x7f27d0100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58684/0 (socket says 172.21.15.5:58684) 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27ced9d640 1 -- 172.21.15.5:0/2230633400 learned_addr learned my addr 172.21.15.5:0/2230633400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27cf59e640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27d01082a0 0x7f27d0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27ce59c640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27d01036e0 0x7f27d0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27ced9d640 1 -- 172.21.15.5:0/2230633400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27d01036e0 msgr2=0x7f27d0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27ced9d640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27d01036e0 0x7f27d0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27ced9d640 1 -- 172.21.15.5:0/2230633400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27d01082a0 msgr2=0x7f27d0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27ced9d640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27d01082a0 0x7f27d0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27ced9d640 1 -- 172.21.15.5:0/2230633400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d01a6a10 con 0x7f27d0102d30 2024-09-11T22:25:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.301+0000 7f27cf59e640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27d01082a0 0x7f27d0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:25:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.302+0000 7f27ced9d640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27d0102d30 0x7f27d0100990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f27c800c890 tx=0x7f27c800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.302+0000 7f27c7fff640 1 -- 172.21.15.5:0/2230633400 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8004220 con 0x7f27d0102d30 2024-09-11T22:25:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.302+0000 7f27c7fff640 1 -- 172.21.15.5:0/2230633400 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c80043c0 con 0x7f27d0102d30 2024-09-11T22:25:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.302+0000 7f27d5194640 1 -- 172.21.15.5:0/2230633400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d01a6d00 con 0x7f27d0102d30 2024-09-11T22:25:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.302+0000 7f27c7fff640 1 -- 172.21.15.5:0/2230633400 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c80049e0 con 0x7f27d0102d30 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.302+0000 7f27d5194640 1 -- 172.21.15.5:0/2230633400 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d01a71e0 con 0x7f27d0102d30 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.304+0000 7f27c7fff640 1 -- 172.21.15.5:0/2230633400 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f27c80079b0 con 0x7f27d0102d30 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.304+0000 7f27d756d740 1 Processor -- start 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.304+0000 7f27d756d740 1 -- start start 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27d756d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987ba4a0 0x5627987ba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27d756d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27d756d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987c5680 0x5627987c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27d756d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5627987202e0 con 0x5627987badc0 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27d756d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5627986c7ce0 con 0x5627987c5680 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27d756d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56279871c2a0 con 0x5627987ba4a0 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58692/0 (socket says 172.21.15.5:58692) 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27ced9d640 1 -- 172.21.15.5:0/1651475991 learned_addr learned my addr 172.21.15.5:0/1651475991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27cf59e640 1 --2- 172.21.15.5:0/1651475991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987ba4a0 0x5627987ba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27ce59c640 1 --2- 172.21.15.5:0/1651475991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987c5680 0x5627987c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27ced9d640 1 -- 172.21.15.5:0/1651475991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987ba4a0 msgr2=0x5627987ba880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27ced9d640 1 --2- 172.21.15.5:0/1651475991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987ba4a0 0x5627987ba880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27ced9d640 1 -- 172.21.15.5:0/1651475991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987c5680 msgr2=0x5627987c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27ced9d640 1 --2- 172.21.15.5:0/1651475991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987c5680 0x5627987c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27ced9d640 1 -- 172.21.15.5:0/1651475991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627986ac440 con 0x5627987badc0 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27c7fff640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27bc0a2cf0 0x7f27bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.305+0000 7f27ced9d640 1 --2- 172.21.15.5:0/1651475991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c5140 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f27c80ccbf0 tx=0x7f27c80d58e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35374b30ad76890d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.306+0000 7f27cf59e640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27bc0a2cf0 0x7f27bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.306+0000 7f27c7fff640 1 -- 172.21.15.5:0/2230633400 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f27c80cf600 con 0x7f27d0102d30 2024-09-11T22:25:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.306+0000 7f27c57fa640 1 -- 172.21.15.5:0/1651475991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8097d20 con 0x5627987badc0 2024-09-11T22:25:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.306+0000 7f27c57fa640 1 -- 172.21.15.5:0/1651475991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c809c180 con 0x5627987badc0 2024-09-11T22:25:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.306+0000 7f27c57fa640 1 -- 172.21.15.5:0/1651475991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c80a31b0 con 0x5627987badc0 2024-09-11T22:25:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.306+0000 7f27d756d740 1 -- 172.21.15.5:0/1651475991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 msgr2=0x5627987c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.306+0000 7f27cf59e640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27bc0a2cf0 0x7f27bc0a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005cc0 tx=0x7f27c00083f0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.306+0000 7f27d756d740 1 --2- 172.21.15.5:0/1651475991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c5140 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f27c80ccbf0 tx=0x7f27c80d58e0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 -- 172.21.15.5:0/1651475991 shutdown_connections 2024-09-11T22:25:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 --2- 172.21.15.5:0/1651475991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987c5680 0x5627987c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 --2- 172.21.15.5:0/1651475991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c5140 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 --2- 172.21.15.5:0/1651475991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987ba4a0 0x5627987ba880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 -- 172.21.15.5:0/1651475991 >> 172.21.15.5:0/1651475991 conn(0x5627987b6a30 msgr2=0x5627987b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 -- 172.21.15.5:0/1651475991 shutdown_connections 2024-09-11T22:25:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 -- 172.21.15.5:0/1651475991 wait complete. 2024-09-11T22:25:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 -- start start 2024-09-11T22:25:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 Processor -- start 2024-09-11T22:25:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627987c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987c5680 0x5627987c2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5627987c8250 con 0x5627987badc0 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5627986db050 con 0x5627987ba4a0 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5627987c83d0 con 0x5627987c5680 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.307+0000 7f27d756d740 10 client.0 _ll_register_callbacks cb 0x56279871b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27ce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27ce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58708/0 (socket says 172.21.15.5:58708) 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27ce59c640 1 -- 172.21.15.5:0/4268509313 learned_addr learned my addr 172.21.15.5:0/4268509313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27cf59e640 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987c5680 0x5627987c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27ced9d640 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627987c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27ce59c640 1 -- 172.21.15.5:0/4268509313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987c5680 msgr2=0x5627987c2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27ce59c640 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987c5680 0x5627987c2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27ce59c640 1 -- 172.21.15.5:0/4268509313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 msgr2=0x5627987c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27ce59c640 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627987c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27ce59c640 1 -- 172.21.15.5:0/4268509313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627984913c0 con 0x5627987badc0 2024-09-11T22:25:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27ce59c640 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c2250 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f27b802a940 tx=0x7f27b8031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d8192185561f7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27b67fc640 1 -- 172.21.15.5:0/4268509313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b803f070 con 0x5627987badc0 2024-09-11T22:25:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27b67fc640 1 -- 172.21.15.5:0/4268509313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f27b8004450 con 0x5627987badc0 2024-09-11T22:25:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27d756d740 1 -- 172.21.15.5:0/4268509313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27c809c180 con 0x5627987badc0 2024-09-11T22:25:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27b67fc640 1 -- 172.21.15.5:0/4268509313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f27b8035070 con 0x5627987badc0 2024-09-11T22:25:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27d756d740 10 client.16350 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.308+0000 7f27d756d740 1 -- 172.21.15.5:0/4268509313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5627987d3c50 con 0x5627987badc0 2024-09-11T22:25:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.309+0000 7f27b67fc640 1 -- 172.21.15.5:0/4268509313 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c809c180 con 0x5627987badc0 2024-09-11T22:25:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.309+0000 7f27b67fc640 1 -- 172.21.15.5:0/4268509313 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f27b8004c30 con 0x5627987badc0 2024-09-11T22:25:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.309+0000 7f27d756d740 10 client.16350 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.309+0000 7f27d756d740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.309+0000 7f27d756d740 20 client.16350 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.309+0000 7f27d756d740 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5627987d51b0 0x5627987d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.309+0000 7f27d756d740 4 client.16350 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.309+0000 7f27d756d740 1 -- 172.21.15.5:0/4268509313 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56279872d850 con 0x5627987d51b0 2024-09-11T22:25:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.309+0000 7f27ced9d640 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5627987d51b0 0x5627987d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.310+0000 7f27b67fc640 10 client.16350 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.310+0000 7f27ced9d640 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5627987d51b0 0x5627987d75c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=700e450cb177164c server_cookie=c764b76d2cfd2986 in_seq=0 out_seq=0 2024-09-11T22:25:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.310+0000 7f27b67fc640 1 -- 172.21.15.5:0/4268509313 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56279872d850 con 0x5627987d51b0 2024-09-11T22:25:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.310+0000 7f27b67fc640 10 client.16350 handle_command_reply: tid=0 2024-09-11T22:25:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.310+0000 7f27d756d740 1 client.16350 shutdown 2024-09-11T22:25:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.310+0000 7f27d756d740 1 -- 172.21.15.5:0/4268509313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 msgr2=0x5627987c2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.310+0000 7f27d756d740 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c2250 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f27b802a940 tx=0x7f27b8031860 comp rx=0 tx=0).stop 2024-09-11T22:25:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.311+0000 7f27d756d740 1 -- 172.21.15.5:0/4268509313 shutdown_connections 2024-09-11T22:25:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.311+0000 7f27d756d740 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5627987d51b0 0x5627987d75c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.311+0000 7f27d756d740 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987c5680 0x5627987c2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.311+0000 7f27d756d740 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987badc0 0x5627987c2250 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.311+0000 7f27d756d740 1 --2- 172.21.15.5:0/4268509313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627987c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.311+0000 7f27d756d740 1 -- 172.21.15.5:0/4268509313 >> 172.21.15.5:0/4268509313 conn(0x5627987b6a30 msgr2=0x5627987bc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.311+0000 7f27b67fc640 0 client.16350 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.311+0000 7f27d756d740 1 -- 172.21.15.5:0/4268509313 shutdown_connections 2024-09-11T22:25:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.311+0000 7f27d756d740 1 -- 172.21.15.5:0/4268509313 wait complete. 2024-09-11T22:25:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.311+0000 7f27d756d740 20 client.16350 trim_cache size 0 max 16384 2024-09-11T22:25:23.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.329+0000 7f27d756d740 1 Processor -- start 2024-09-11T22:25:23.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.329+0000 7f27d756d740 1 -- start start 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.329+0000 7f27d756d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627987d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.329+0000 7f27d756d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987badc0 0x5627987dce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.329+0000 7f27d756d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987dd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.329+0000 7f27d756d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01afcd0 con 0x5627987c5680 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.329+0000 7f27d756d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01b18d0 con 0x5627987ba4a0 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.329+0000 7f27d756d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01b03a0 con 0x5627987badc0 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.329+0000 7f27ced9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627987d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27cf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987dd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27ced9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627987d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48512/0 (socket says 172.21.15.5:48512) 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27ced9d640 1 -- 172.21.15.5:0/3644327772 learned_addr learned my addr 172.21.15.5:0/3644327772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27ce59c640 1 --2- 172.21.15.5:0/3644327772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987badc0 0x5627987dce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27ced9d640 1 -- 172.21.15.5:0/3644327772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987badc0 msgr2=0x5627987dce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27ced9d640 1 --2- 172.21.15.5:0/3644327772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987badc0 0x5627987dce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27ced9d640 1 -- 172.21.15.5:0/3644327772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 msgr2=0x5627987dd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27ced9d640 1 --2- 172.21.15.5:0/3644327772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987dd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27ced9d640 1 -- 172.21.15.5:0/3644327772 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627987b6860 con 0x5627987ba4a0 2024-09-11T22:25:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27cf59e640 1 --2- 172.21.15.5:0/3644327772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987dd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27ced9d640 1 --2- 172.21.15.5:0/3644327772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627987d1530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f27c8099d30 tx=0x7f27c80d5210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd7d905aa72daea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27c4ff9640 1 -- 172.21.15.5:0/3644327772 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c801d040 con 0x5627987ba4a0 2024-09-11T22:25:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27c4ff9640 1 -- 172.21.15.5:0/3644327772 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c809c180 con 0x5627987ba4a0 2024-09-11T22:25:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.330+0000 7f27c4ff9640 1 -- 172.21.15.5:0/3644327772 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c809b3e0 con 0x5627987ba4a0 2024-09-11T22:25:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.331+0000 7f27d756d740 1 -- 172.21.15.5:0/3644327772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 msgr2=0x5627987d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.331+0000 7f27d756d740 1 --2- 172.21.15.5:0/3644327772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627987d1530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f27c8099d30 tx=0x7f27c80d5210 comp rx=0 tx=0).stop 2024-09-11T22:25:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.331+0000 7f27d756d740 1 -- 172.21.15.5:0/3644327772 shutdown_connections 2024-09-11T22:25:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.331+0000 7f27d756d740 1 --2- 172.21.15.5:0/3644327772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987dd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.331+0000 7f27d756d740 1 --2- 172.21.15.5:0/3644327772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987badc0 0x5627987dce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.331+0000 7f27d756d740 1 --2- 172.21.15.5:0/3644327772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627987d1530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.331+0000 7f27d756d740 1 -- 172.21.15.5:0/3644327772 >> 172.21.15.5:0/3644327772 conn(0x5627987b6a30 msgr2=0x5627987c1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.331+0000 7f27d756d740 1 -- 172.21.15.5:0/3644327772 shutdown_connections 2024-09-11T22:25:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.331+0000 7f27d756d740 1 -- 172.21.15.5:0/3644327772 wait complete. 2024-09-11T22:25:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27d756d740 1 -- start start 2024-09-11T22:25:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27d756d740 1 Processor -- start 2024-09-11T22:25:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27d756d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627988371e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27d756d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987badc0 0x562798837720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27d756d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27d756d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01b1140 con 0x5627987c5680 2024-09-11T22:25:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27d756d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01b0a70 con 0x5627987ba4a0 2024-09-11T22:25:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27d756d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01adcb0 con 0x5627987badc0 2024-09-11T22:25:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27d756d740 10 client.0 _ll_register_callbacks cb 0x7f27b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27cf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27ce59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987badc0 0x562798837720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27cf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58734/0 (socket says 172.21.15.5:58734) 2024-09-11T22:25:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.332+0000 7f27cf59e640 1 -- 172.21.15.5:0/551221334 learned_addr learned my addr 172.21.15.5:0/551221334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27ced9d640 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627988371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27cf59e640 1 -- 172.21.15.5:0/551221334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987badc0 msgr2=0x562798837720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27cf59e640 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987badc0 0x562798837720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27cf59e640 1 -- 172.21.15.5:0/551221334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 msgr2=0x5627988371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27cf59e640 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627988371e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27cf59e640 1 -- 172.21.15.5:0/551221334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627987cb490 con 0x5627987c5680 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27cf59e640 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987cac60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f27c0010fd0 tx=0x7f27c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d0459719d1acc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27b67fc640 1 -- 172.21.15.5:0/551221334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0004b20 con 0x5627987c5680 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27b67fc640 1 -- 172.21.15.5:0/551221334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f27c0004cc0 con 0x5627987c5680 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.333+0000 7f27d756d740 1 -- 172.21.15.5:0/551221334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27c809c180 con 0x5627987c5680 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27b67fc640 1 -- 172.21.15.5:0/551221334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f27c001c5c0 con 0x5627987c5680 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27d756d740 10 client.16356 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27d756d740 1 -- 172.21.15.5:0/551221334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56279883f9b0 con 0x5627987c5680 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27b67fc640 1 -- 172.21.15.5:0/551221334 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27c809c180 con 0x5627987c5680 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27b67fc640 1 -- 172.21.15.5:0/551221334 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f27c00055e0 con 0x5627987c5680 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27d756d740 10 client.16356 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27d756d740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27d756d740 20 client.16356 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27d756d740 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562798840240 0x562798842660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27d756d740 4 client.16356 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27d756d740 1 -- 172.21.15.5:0/551221334 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562798727d60 con 0x562798840240 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.334+0000 7f27ced9d640 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562798840240 0x562798842660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27b67fc640 10 client.16356 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27ced9d640 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562798840240 0x562798842660 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3e08ae886007a1e server_cookie=a8c7c073013b348f in_seq=0 out_seq=0 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27b67fc640 1 -- 172.21.15.5:0/551221334 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562798727d60 con 0x562798840240 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27b67fc640 10 client.16356 handle_command_reply: tid=0 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27d756d740 1 client.16356 shutdown 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27d756d740 1 -- 172.21.15.5:0/551221334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 msgr2=0x5627987cac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27d756d740 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987cac60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f27c0010fd0 tx=0x7f27c0004a40 comp rx=0 tx=0).stop 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27d756d740 1 -- 172.21.15.5:0/551221334 shutdown_connections 2024-09-11T22:25:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27d756d740 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562798840240 0x562798842660 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27d756d740 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627987c5680 0x5627987cac60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27b67fc640 0 client.16356 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:23.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27d756d740 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627987badc0 0x562798837720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27d756d740 1 --2- 172.21.15.5:0/551221334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627987ba4a0 0x5627988371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.335+0000 7f27d756d740 1 -- 172.21.15.5:0/551221334 >> 172.21.15.5:0/551221334 conn(0x5627987b6a30 msgr2=0x5627987c3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:23.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.336+0000 7f27d756d740 1 -- 172.21.15.5:0/551221334 shutdown_connections 2024-09-11T22:25:23.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.336+0000 7f27d756d740 1 -- 172.21.15.5:0/551221334 wait complete. 2024-09-11T22:25:23.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.336+0000 7f27d756d740 20 client.16356 trim_cache size 0 max 16384 2024-09-11T22:25:23.349 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:25:23.350 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:25:23.350 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:25:23.350 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:25:23.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 -- 172.21.15.5:0/2230633400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27bc0a2cf0 msgr2=0x7f27bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:23.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27bc0a2cf0 0x7f27bc0a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005cc0 tx=0x7f27c00083f0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 -- 172.21.15.5:0/2230633400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27d0102d30 msgr2=0x7f27d0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27d0102d30 0x7f27d0100990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f27c800c890 tx=0x7f27c800cd50 comp rx=0 tx=0).stop 2024-09-11T22:25:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 -- 172.21.15.5:0/2230633400 shutdown_connections 2024-09-11T22:25:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27bc0a2cf0 0x7f27bc0a51b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27d01082a0 0x7f27d0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27d01036e0 0x7f27d0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 --2- 172.21.15.5:0/2230633400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27d0102d30 0x7f27d0100990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 -- 172.21.15.5:0/2230633400 >> 172.21.15.5:0/2230633400 conn(0x7f27d00fc7d0 msgr2=0x7f27d0105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 -- 172.21.15.5:0/2230633400 shutdown_connections 2024-09-11T22:25:23.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:23.337+0000 7f27d5194640 1 -- 172.21.15.5:0/2230633400 wait complete. 2024-09-11T22:25:23.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:23 smithi172 ceph-mon[35461]: pgmap v98: 97 pgs: 97 active+clean; 52 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 3.0 MiB/s wr, 117 op/s 2024-09-11T22:25:23.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:23 smithi159 ceph-mon[35534]: pgmap v98: 97 pgs: 97 active+clean; 52 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 3.0 MiB/s wr, 117 op/s 2024-09-11T22:25:24.226 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:25:24.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:25:24.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 9.76 MB/sec warmup 102 sec latency 21144.648 ms 2024-09-11T22:25:24.494 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:25.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:25:25.131 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:25 smithi005 ceph-mon[30943]: pgmap v99: 97 pgs: 97 active+clean; 52 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-11T22:25:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.129+0000 7fecf9cde640 1 -- 172.21.15.5:0/521881762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf4100e20 msgr2=0x7fecf4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:25.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.129+0000 7fecf9cde640 1 --2- 172.21.15.5:0/521881762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf4100e20 0x7fecf4101200 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fece4009900 tx=0x7fece402a480 comp rx=0 tx=0).stop 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.129+0000 7fecf9cde640 1 -- 172.21.15.5:0/521881762 shutdown_connections 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.129+0000 7fecf9cde640 1 --2- 172.21.15.5:0/521881762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf4102ef0 0x7fecf410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.129+0000 7fecf9cde640 1 --2- 172.21.15.5:0/521881762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecf4102570 0x7fecf41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.129+0000 7fecf9cde640 1 --2- 172.21.15.5:0/521881762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf4100e20 0x7fecf4101200 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.129+0000 7fecf9cde640 1 -- 172.21.15.5:0/521881762 >> 172.21.15.5:0/521881762 conn(0x7fecf40fc7d0 msgr2=0x7fecf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.129+0000 7fecf9cde640 1 -- 172.21.15.5:0/521881762 shutdown_connections 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.129+0000 7fecf9cde640 1 -- 172.21.15.5:0/521881762 wait complete. 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.130+0000 7fecf9cde640 1 Processor -- start 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.130+0000 7fecf9cde640 1 -- start start 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.130+0000 7fecf9cde640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecf4100e20 0x7fecf40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf9cde640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf4102570 0x7fecf40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf9cde640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf4102ef0 0x7fecf40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:25.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf9cde640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fecf4110f50 con 0x7fecf4102ef0 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf9cde640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fecf4110dd0 con 0x7fecf4100e20 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf9cde640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fecf41110d0 con 0x7fecf4102570 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf37fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecf4100e20 0x7fecf40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf4102ef0 0x7fecf40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf37fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecf4100e20 0x7fecf40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48540/0 (socket says 172.21.15.5:48540) 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf37fe640 1 -- 172.21.15.5:0/4123946196 learned_addr learned my addr 172.21.15.5:0/4123946196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7feceaffd640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf4102570 0x7fecf40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf3fff640 1 -- 172.21.15.5:0/4123946196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf4102570 msgr2=0x7fecf40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf3fff640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf4102570 0x7fecf40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf3fff640 1 -- 172.21.15.5:0/4123946196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecf4100e20 msgr2=0x7fecf40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf3fff640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecf4100e20 0x7fecf40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf3fff640 1 -- 172.21.15.5:0/4123946196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecf41a6960 con 0x7fecf4102ef0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf37fe640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecf4100e20 0x7fecf40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.131+0000 7fecf3fff640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf4102ef0 0x7fecf40fd700 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fecec00e940 tx=0x7fecec00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.132+0000 7fecf17fa640 1 -- 172.21.15.5:0/4123946196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec004d20 con 0x7fecf4102ef0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.132+0000 7fecf9cde640 1 -- 172.21.15.5:0/4123946196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecf41a6c50 con 0x7fecf4102ef0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.132+0000 7fecf17fa640 1 -- 172.21.15.5:0/4123946196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecec005020 con 0x7fecf4102ef0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.132+0000 7fecf17fa640 1 -- 172.21.15.5:0/4123946196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecec0116b0 con 0x7fecf4102ef0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.132+0000 7fecf9cde640 1 -- 172.21.15.5:0/4123946196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecf41a7190 con 0x7fecf4102ef0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.133+0000 7fecf9cde640 1 -- 172.21.15.5:0/4123946196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecf4101200 con 0x7fecf4102ef0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.134+0000 7fecf17fa640 1 -- 172.21.15.5:0/4123946196 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fecec0054d0 con 0x7fecf4102ef0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.135+0000 7fecf17fa640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fece00a2cf0 0x7fece00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.135+0000 7fecf37fe640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fece00a2cf0 0x7fece00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.135+0000 7fecf17fa640 1 -- 172.21.15.5:0/4123946196 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fecec0cf480 con 0x7fecf4102ef0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.135+0000 7fecf37fe640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fece00a2cf0 0x7fece00a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fece40096c0 tx=0x7fece4005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:25.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.139+0000 7fecf17fa640 1 -- 172.21.15.5:0/4123946196 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fecec016030 con 0x7fecf4102ef0 2024-09-11T22:25:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:25.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 9.67 MB/sec warmup 103 sec latency 22144.678 ms 2024-09-11T22:25:25.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:25 smithi172 ceph-mon[35461]: pgmap v99: 97 pgs: 97 active+clean; 52 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-11T22:25:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.360+0000 7fecf9cde640 1 -- 172.21.15.5:0/4123946196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecf410a0a0 con 0x7fecf4102ef0 2024-09-11T22:25:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.361+0000 7fecf17fa640 1 -- 172.21.15.5:0/4123946196 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11974 (secure 0 0 0) 0x7fecec09ba10 con 0x7fecf4102ef0 2024-09-11T22:25:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:25:25.363 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-11T22:23:49:156575+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:23:48.779139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:25:25.363 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-11T22:25:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.363+0000 7fecf9cde640 1 -- 172.21.15.5:0/4123946196 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fece00a2cf0 msgr2=0x7fece00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.363+0000 7fecf9cde640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fece00a2cf0 0x7fece00a51b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fece40096c0 tx=0x7fece4005a20 comp rx=0 tx=0).stop 2024-09-11T22:25:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.363+0000 7fecf9cde640 1 -- 172.21.15.5:0/4123946196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf4102ef0 msgr2=0x7fecf40fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.363+0000 7fecf9cde640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf4102ef0 0x7fecf40fd700 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fecec00e940 tx=0x7fecec00ee00 comp rx=0 tx=0).stop 2024-09-11T22:25:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.363+0000 7fecf9cde640 1 -- 172.21.15.5:0/4123946196 shutdown_connections 2024-09-11T22:25:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.363+0000 7fecf9cde640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecf4102ef0 0x7fecf40fd700 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.363+0000 7fecf9cde640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecf4102570 0x7fecf40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.363+0000 7fecf9cde640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fece00a2cf0 0x7fece00a51b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.363+0000 7fecf9cde640 1 --2- 172.21.15.5:0/4123946196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecf4100e20 0x7fecf40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.363+0000 7fecf9cde640 1 -- 172.21.15.5:0/4123946196 >> 172.21.15.5:0/4123946196 conn(0x7fecf40fc7d0 msgr2=0x7fecf406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:25.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.364+0000 7fecf9cde640 1 -- 172.21.15.5:0/4123946196 shutdown_connections 2024-09-11T22:25:25.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.364+0000 7fecf9cde640 1 -- 172.21.15.5:0/4123946196 wait complete. 2024-09-11T22:25:25.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:25 smithi159 ceph-mon[35534]: pgmap v99: 97 pgs: 97 active+clean; 52 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-11T22:25:25.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.838+0000 7f8056978640 1 -- 172.21.15.5:0/3882494128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8050106a70 msgr2=0x7f8050106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.838+0000 7f8056978640 1 --2- 172.21.15.5:0/3882494128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8050106a70 0x7f8050106e50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8040009930 tx=0x7f804002a420 comp rx=0 tx=0).stop 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.838+0000 7f8056978640 1 -- 172.21.15.5:0/3882494128 shutdown_connections 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.838+0000 7f8056978640 1 --2- 172.21.15.5:0/3882494128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8050101410 0x7f805010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.838+0000 7f8056978640 1 --2- 172.21.15.5:0/3882494128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8050100a70 0x7f8050100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.838+0000 7f8056978640 1 --2- 172.21.15.5:0/3882494128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8050106a70 0x7f8050106e50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.838+0000 7f8056978640 1 -- 172.21.15.5:0/3882494128 >> 172.21.15.5:0/3882494128 conn(0x7f80500fc790 msgr2=0x7f80500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.838+0000 7f8056978640 1 -- 172.21.15.5:0/3882494128 shutdown_connections 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.838+0000 7f8056978640 1 -- 172.21.15.5:0/3882494128 wait complete. 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.838+0000 7f8056978640 1 Processor -- start 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8056978640 1 -- start start 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8056978640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8050100a70 0x7f8050199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:25.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8056978640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8050101410 0x7f80501998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8056978640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8050106a70 0x7f80501a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8056978640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8050110e90 con 0x7f8050101410 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8056978640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8050110d10 con 0x7f8050100a70 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8056978640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8050111010 con 0x7f8050106a70 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8047fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8050101410 0x7f80501998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8047fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8050101410 0x7f80501998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58762/0 (socket says 172.21.15.5:58762) 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8047fff640 1 -- 172.21.15.5:0/2091933929 learned_addr learned my addr 172.21.15.5:0/2091933929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f804ffff640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8050100a70 0x7f8050199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8047fff640 1 -- 172.21.15.5:0/2091933929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8050106a70 msgr2=0x7f80501a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8054f15640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8050106a70 0x7f80501a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8047fff640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8050106a70 0x7f80501a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8047fff640 1 -- 172.21.15.5:0/2091933929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8050100a70 msgr2=0x7f8050199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8047fff640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8050100a70 0x7f8050199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.839+0000 7f8047fff640 1 -- 172.21.15.5:0/2091933929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80501a4940 con 0x7f8050101410 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.840+0000 7f8047fff640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8050101410 0x7f80501998c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f803800b4a0 tx=0x7f803800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.840+0000 7f804dffb640 1 -- 172.21.15.5:0/2091933929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038014070 con 0x7f8050101410 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.840+0000 7f8056978640 1 -- 172.21.15.5:0/2091933929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80501a4b10 con 0x7f8050101410 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.840+0000 7f804dffb640 1 -- 172.21.15.5:0/2091933929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8038010af0 con 0x7f8050101410 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.840+0000 7f804dffb640 1 -- 172.21.15.5:0/2091933929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038017d10 con 0x7f8050101410 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.840+0000 7f8056978640 1 -- 172.21.15.5:0/2091933929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80501a5050 con 0x7f8050101410 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.841+0000 7f8056978640 1 -- 172.21.15.5:0/2091933929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8050100ed0 con 0x7f8050101410 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.842+0000 7f804dffb640 1 -- 172.21.15.5:0/2091933929 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8038017070 con 0x7f8050101410 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.843+0000 7f804dffb640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f803c0a2cf0 0x7f803c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.843+0000 7f804ffff640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f803c0a2cf0 0x7f803c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:25.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.843+0000 7f804dffb640 1 -- 172.21.15.5:0/2091933929 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f80380ceb40 con 0x7f8050101410 2024-09-11T22:25:25.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.843+0000 7f804ffff640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f803c0a2cf0 0x7f803c0a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8040009900 tx=0x7f80400058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:25.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:25.846+0000 7f804dffb640 1 -- 172.21.15.5:0/2091933929 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f803809b7d0 con 0x7f8050101410 2024-09-11T22:25:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.058+0000 7f8056978640 1 -- 172.21.15.5:0/2091933929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8050109040 con 0x7f8050101410 2024-09-11T22:25:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.059+0000 7f804dffb640 1 -- 172.21.15.5:0/2091933929 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11974 (secure 0 0 0) 0x7f803809b0d0 con 0x7f8050101410 2024-09-11T22:25:26.061 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:25:26.061 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-11T22:23:49:156575+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:23:48.779139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2975110123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2975110123},{"type":"v1","addr":"172.21.15.159:6833","nonce":2975110123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:25:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-11T22:25:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 -- 172.21.15.5:0/2091933929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f803c0a2cf0 msgr2=0x7f803c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f803c0a2cf0 0x7f803c0a51b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8040009900 tx=0x7f80400058d0 comp rx=0 tx=0).stop 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 -- 172.21.15.5:0/2091933929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8050101410 msgr2=0x7f80501998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8050101410 0x7f80501998c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f803800b4a0 tx=0x7f803800b960 comp rx=0 tx=0).stop 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 -- 172.21.15.5:0/2091933929 shutdown_connections 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8050106a70 0x7f80501a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8050101410 0x7f80501998c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f803c0a2cf0 0x7f803c0a51b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 --2- 172.21.15.5:0/2091933929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8050100a70 0x7f8050199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 -- 172.21.15.5:0/2091933929 >> 172.21.15.5:0/2091933929 conn(0x7f80500fc790 msgr2=0x7f80500fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 -- 172.21.15.5:0/2091933929 shutdown_connections 2024-09-11T22:25:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:26.061+0000 7f8056978640 1 -- 172.21.15.5:0/2091933929 wait complete. 2024-09-11T22:25:26.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 9.58 MB/sec warmup 104 sec latency 23144.761 ms 2024-09-11T22:25:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:26 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4123946196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:25:26.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:26 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2091933929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:25:26.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:26 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4123946196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:25:26.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:26 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2091933929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:25:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:26 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4123946196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:25:26.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:26 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2091933929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:25:26.718 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:25:27.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 9.48 MB/sec warmup 105 sec latency 24144.833 ms 2024-09-11T22:25:27.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:27 smithi005 ceph-mon[30943]: pgmap v100: 97 pgs: 97 active+clean; 52 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-11T22:25:27.396 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 299bfba0 --release --await' 2024-09-11T22:25:27.396 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 299bfba0 --release --await 2024-09-11T22:25:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:27 smithi159 ceph-mon[35534]: pgmap v100: 97 pgs: 97 active+clean; 52 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-11T22:25:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:27 smithi172 ceph-mon[35461]: pgmap v100: 97 pgs: 97 active+clean; 52 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.0 MiB/s wr, 94 op/s 2024-09-11T22:25:28.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.075+0000 7f526cd30640 1 -- 172.21.15.5:0/3160187144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52681036e0 msgr2=0x7f5268107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.075+0000 7f526cd30640 1 --2- 172.21.15.5:0/3160187144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52681036e0 0x7f5268107b90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f525c009930 tx=0x7f525c02a420 comp rx=0 tx=0).stop 2024-09-11T22:25:28.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.076+0000 7f526cd30640 1 -- 172.21.15.5:0/3160187144 shutdown_connections 2024-09-11T22:25:28.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.076+0000 7f526cd30640 1 --2- 172.21.15.5:0/3160187144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52681082a0 0x7f526810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.076+0000 7f526cd30640 1 --2- 172.21.15.5:0/3160187144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52681036e0 0x7f5268107b90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.076+0000 7f526cd30640 1 --2- 172.21.15.5:0/3160187144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5268102d30 0x7f5268103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.076+0000 7f526cd30640 1 -- 172.21.15.5:0/3160187144 >> 172.21.15.5:0/3160187144 conn(0x7f52680fc7d0 msgr2=0x7f52680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.076+0000 7f526cd30640 1 -- 172.21.15.5:0/3160187144 shutdown_connections 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.076+0000 7f526cd30640 1 -- 172.21.15.5:0/3160187144 wait complete. 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.076+0000 7f526cd30640 1 Processor -- start 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.076+0000 7f526cd30640 1 -- start start 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f526cd30640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268102d30 0x7f5268100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f526cd30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52681036e0 0x7f5268100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f526cd30640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52681082a0 0x7f5268101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f526cd30640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5268112060 con 0x7f52681036e0 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f526cd30640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5268111ee0 con 0x7f52681082a0 2024-09-11T22:25:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f526cd30640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52681121e0 con 0x7f5268102d30 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52681036e0 0x7f5268100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52681036e0 0x7f5268100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58776/0 (socket says 172.21.15.5:58776) 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f5265d9b640 1 -- 172.21.15.5:0/3971737958 learned_addr learned my addr 172.21.15.5:0/3971737958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f5266d9d640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52681082a0 0x7f5268101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f526659c640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268102d30 0x7f5268100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f5265d9b640 1 -- 172.21.15.5:0/3971737958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268102d30 msgr2=0x7f5268100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f5265d9b640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268102d30 0x7f5268100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f5265d9b640 1 -- 172.21.15.5:0/3971737958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52681082a0 msgr2=0x7f5268101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f5265d9b640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52681082a0 0x7f5268101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f5265d9b640 1 -- 172.21.15.5:0/3971737958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52681a6a80 con 0x7f52681036e0 2024-09-11T22:25:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.077+0000 7f5265d9b640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52681036e0 0x7f5268100ed0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f525c005940 tx=0x7f525c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.078+0000 7f52577fe640 1 -- 172.21.15.5:0/3971737958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c03f030 con 0x7f52681036e0 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.078+0000 7f526cd30640 1 -- 172.21.15.5:0/3971737958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52681a6d10 con 0x7f52681036e0 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.078+0000 7f52577fe640 1 -- 172.21.15.5:0/3971737958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f525c003ce0 con 0x7f52681036e0 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.078+0000 7f52577fe640 1 -- 172.21.15.5:0/3971737958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c035070 con 0x7f52681036e0 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.078+0000 7f526cd30640 1 -- 172.21.15.5:0/3971737958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52681a71e0 con 0x7f52681036e0 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.080+0000 7f52577fe640 1 -- 172.21.15.5:0/3971737958 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f525c042030 con 0x7f52681036e0 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.080+0000 7f526f109740 1 Processor -- start 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526f109740 1 -- start start 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f52577fe640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52500a2cf0 0x7f52500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526f109740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526f109740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 0x5644aa6eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526f109740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6f5370 0x5644aa6f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526f109740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5644aa6502f0 con 0x5644aa6ebb40 2024-09-11T22:25:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526f109740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5644aa5f7cf0 con 0x5644aa6f5370 2024-09-11T22:25:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526f109740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5644aa64c2b0 con 0x5644aa6ea5d0 2024-09-11T22:25:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f52577fe640 1 -- 172.21.15.5:0/3971737958 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f525c0ef830 con 0x7f52681036e0 2024-09-11T22:25:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f5266d9d640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52500a2cf0 0x7f52500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6ea090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58778/0 (socket says 172.21.15.5:58778) 2024-09-11T22:25:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526659c640 1 -- 172.21.15.5:0/2165711088 learned_addr learned my addr 172.21.15.5:0/2165711088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f5265d9b640 1 --2- 172.21.15.5:0/2165711088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 0x5644aa6eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f5266d9d640 1 --2- 172.21.15.5:0/2165711088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6f5370 0x5644aa6f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526659c640 1 -- 172.21.15.5:0/2165711088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 msgr2=0x5644aa6eaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526659c640 1 --2- 172.21.15.5:0/2165711088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 0x5644aa6eaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526659c640 1 -- 172.21.15.5:0/2165711088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6f5370 msgr2=0x5644aa6f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526659c640 1 --2- 172.21.15.5:0/2165711088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6f5370 0x5644aa6f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.081+0000 7f526659c640 1 -- 172.21.15.5:0/2165711088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644aa5dc3a0 con 0x5644aa6ebb40 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.082+0000 7f5266d9d640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52500a2cf0 0x7f52500a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5260009fd0 tx=0x7f5260009da0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.082+0000 7f526659c640 1 --2- 172.21.15.5:0/2165711088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6ea090 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f525800bcd0 tx=0x7f52580107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea15913d3391f4e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.082+0000 7f5254ff9640 1 -- 172.21.15.5:0/2165711088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258005020 con 0x5644aa6ebb40 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.082+0000 7f5254ff9640 1 -- 172.21.15.5:0/2165711088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52580047a0 con 0x5644aa6ebb40 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.082+0000 7f5254ff9640 1 -- 172.21.15.5:0/2165711088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52580116b0 con 0x5644aa6ebb40 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.083+0000 7f526f109740 1 -- 172.21.15.5:0/2165711088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 msgr2=0x5644aa6ea090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.083+0000 7f526f109740 1 --2- 172.21.15.5:0/2165711088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6ea090 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f525800bcd0 tx=0x7f52580107a0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.083+0000 7f526f109740 1 -- 172.21.15.5:0/2165711088 shutdown_connections 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.083+0000 7f526f109740 1 --2- 172.21.15.5:0/2165711088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6f5370 0x5644aa6f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.083+0000 7f526f109740 1 --2- 172.21.15.5:0/2165711088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 0x5644aa6eaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.083+0000 7f526f109740 1 --2- 172.21.15.5:0/2165711088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6ea090 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.083+0000 7f526f109740 1 -- 172.21.15.5:0/2165711088 >> 172.21.15.5:0/2165711088 conn(0x5644aa6b3310 msgr2=0x5644aa6e6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.083+0000 7f526f109740 1 -- 172.21.15.5:0/2165711088 shutdown_connections 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.083+0000 7f526f109740 1 -- 172.21.15.5:0/2165711088 wait complete. 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.083+0000 7f526f109740 1 -- start start 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526f109740 1 Processor -- start 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526f109740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526f109740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ebb40 0x5644aa6f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526f109740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6f5370 0x5644aa6f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526f109740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5644aa6f8070 con 0x5644aa6f5370 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526f109740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5644aa60b060 con 0x5644aa6ebb40 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526f109740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5644aa6f81f0 con 0x5644aa6ea5d0 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526f109740 10 client.0 _ll_register_callbacks cb 0x5644aa35e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f5266d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6f5370 0x5644aa6f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ebb40 0x5644aa6f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:41690/0 (socket says 172.21.15.5:41690) 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.084+0000 7f526659c640 1 -- 172.21.15.5:0/1827769393 learned_addr learned my addr 172.21.15.5:0/1827769393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f5266d9d640 1 -- 172.21.15.5:0/1827769393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 msgr2=0x5644aa6f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f5266d9d640 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f5266d9d640 1 -- 172.21.15.5:0/1827769393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ebb40 msgr2=0x5644aa6f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f5266d9d640 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ebb40 0x5644aa6f1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f5266d9d640 1 -- 172.21.15.5:0/1827769393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644aa3c13c0 con 0x5644aa6f5370 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f526659c640 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f5265d9b640 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ebb40 0x5644aa6f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f5266d9d640 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6f5370 0x5644aa6f24d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x5644aa6eb070 tx=0x7f526001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393707ffb3010dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f524dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f524dffb640 1 -- 172.21.15.5:0/1827769393 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260004780 con 0x5644aa6f5370 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f524dffb640 1 -- 172.21.15.5:0/1827769393 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f5260029410 con 0x5644aa6f5370 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f526f109740 1 -- 172.21.15.5:0/1827769393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52580047a0 con 0x5644aa6f5370 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f524dffb640 1 -- 172.21.15.5:0/1827769393 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5260029610 con 0x5644aa6f5370 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f526f109740 10 client.16380 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f526f109740 1 -- 172.21.15.5:0/1827769393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644aa703ab0 con 0x5644aa6f5370 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.085+0000 7f524dffb640 1 -- 172.21.15.5:0/1827769393 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52580047a0 con 0x5644aa6f5370 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.086+0000 7f524dffb640 1 -- 172.21.15.5:0/1827769393 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f526000f040 con 0x5644aa6f5370 2024-09-11T22:25:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.086+0000 7f526f109740 10 client.16380 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.086+0000 7f526f109740 10 client.16380 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.086+0000 7f526f109740 20 client.16380 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.086+0000 7f526f109740 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5644aa704f70 0x5644aa707380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.086+0000 7f526f109740 4 client.16380 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.086+0000 7f526f109740 1 -- 172.21.15.5:0/1827769393 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644aa65d860 con 0x5644aa704f70 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.086+0000 7f526659c640 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5644aa704f70 0x5644aa707380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.086+0000 7f524dffb640 10 client.16380 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.086+0000 7f526659c640 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5644aa704f70 0x5644aa707380 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9e750e97b48a20f server_cookie=42e2440d1e3c215 in_seq=0 out_seq=0 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.087+0000 7f524dffb640 1 -- 172.21.15.5:0/1827769393 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644aa65d860 con 0x5644aa704f70 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.087+0000 7f524dffb640 10 client.16380 handle_command_reply: tid=0 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.087+0000 7f526f109740 1 client.16380 shutdown 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 1 -- 172.21.15.5:0/1827769393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6f5370 msgr2=0x5644aa6f24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6f5370 0x5644aa6f24d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x5644aa6eb070 tx=0x7f526001a000 comp rx=0 tx=0).stop 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 1 -- 172.21.15.5:0/1827769393 shutdown_connections 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5644aa704f70 0x5644aa707380 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6f5370 0x5644aa6f24d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ebb40 0x5644aa6f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 1 --2- 172.21.15.5:0/1827769393 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 1 -- 172.21.15.5:0/1827769393 >> 172.21.15.5:0/1827769393 conn(0x5644aa6b3310 msgr2=0x5644aa6ec930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f524dffb640 0 client.16380 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 1 -- 172.21.15.5:0/1827769393 shutdown_connections 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 1 -- 172.21.15.5:0/1827769393 wait complete. 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.088+0000 7f526f109740 20 client.16380 trim_cache size 0 max 16384 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526f109740 1 Processor -- start 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526f109740 1 -- start start 2024-09-11T22:25:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526f109740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ea5d0 0x5644aa701340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526f109740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ebb40 0x5644aa6f1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526f109740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6f5370 0x5644aa6ed120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526f109740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52681afc30 con 0x5644aa6ea5d0 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526f109740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f52681b1830 con 0x5644aa6f5370 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526f109740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52681b0300 con 0x5644aa6ebb40 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ea5d0 0x5644aa701340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ea5d0 0x5644aa701340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58782/0 (socket says 172.21.15.5:58782) 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f526659c640 1 -- 172.21.15.5:0/3403277689 learned_addr learned my addr 172.21.15.5:0/3403277689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f5266d9d640 1 --2- 172.21.15.5:0/3403277689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6f5370 0x5644aa6ed120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.116+0000 7f5265d9b640 1 --2- 172.21.15.5:0/3403277689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ebb40 0x5644aa6f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.117+0000 7f526659c640 1 -- 172.21.15.5:0/3403277689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ebb40 msgr2=0x5644aa6f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.117+0000 7f526659c640 1 --2- 172.21.15.5:0/3403277689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ebb40 0x5644aa6f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.117+0000 7f526659c640 1 -- 172.21.15.5:0/3403277689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6f5370 msgr2=0x5644aa6ed120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.117+0000 7f526659c640 1 --2- 172.21.15.5:0/3403277689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6f5370 0x5644aa6ed120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.117+0000 7f526659c640 1 -- 172.21.15.5:0/3403277689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644aa6eb790 con 0x5644aa6ea5d0 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.117+0000 7f526659c640 1 --2- 172.21.15.5:0/3403277689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ea5d0 0x5644aa701340 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5258010ca0 tx=0x7f525800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7473e10c75aa32cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f524ffff640 1 -- 172.21.15.5:0/3403277689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525802c020 con 0x5644aa6ea5d0 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f524ffff640 1 -- 172.21.15.5:0/3403277689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5258005510 con 0x5644aa6ea5d0 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f524ffff640 1 -- 172.21.15.5:0/3403277689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258004c90 con 0x5644aa6ea5d0 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f526f109740 1 -- 172.21.15.5:0/3403277689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ea5d0 msgr2=0x5644aa701340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f526f109740 1 --2- 172.21.15.5:0/3403277689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ea5d0 0x5644aa701340 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5258010ca0 tx=0x7f525800bf60 comp rx=0 tx=0).stop 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f526f109740 1 -- 172.21.15.5:0/3403277689 shutdown_connections 2024-09-11T22:25:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f526f109740 1 --2- 172.21.15.5:0/3403277689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6f5370 0x5644aa6ed120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f526f109740 1 --2- 172.21.15.5:0/3403277689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6ebb40 0x5644aa6f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f526f109740 1 --2- 172.21.15.5:0/3403277689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ea5d0 0x5644aa701340 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f526f109740 1 -- 172.21.15.5:0/3403277689 >> 172.21.15.5:0/3403277689 conn(0x5644aa6b3310 msgr2=0x5644aa6f0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f526f109740 1 -- 172.21.15.5:0/3403277689 shutdown_connections 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.118+0000 7f526f109740 1 -- 172.21.15.5:0/3403277689 wait complete. 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f526f109740 1 -- start start 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f526f109740 1 Processor -- start 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f526f109740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f526f109740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f526f109740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6f5370 0x5644aa6f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f526f109740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52681b10a0 con 0x5644aa6ebb40 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f526f109740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f52681b09d0 con 0x5644aa6ea5d0 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f526f109740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52681adc10 con 0x5644aa6f5370 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f526f109740 10 client.0 _ll_register_callbacks cb 0x7f52600124f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f5265d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58792/0 (socket says 172.21.15.5:58792) 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f5265d9b640 1 -- 172.21.15.5:0/3540662003 learned_addr learned my addr 172.21.15.5:0/3540662003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f526659c640 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.119+0000 7f5266d9d640 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6f5370 0x5644aa6f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f5265d9b640 1 -- 172.21.15.5:0/3540662003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6f5370 msgr2=0x5644aa6f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f5265d9b640 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6f5370 0x5644aa6f8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f5265d9b640 1 -- 172.21.15.5:0/3540662003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ea5d0 msgr2=0x5644aa6f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f5265d9b640 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f5265d9b640 1 -- 172.21.15.5:0/3540662003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644aa76dc10 con 0x5644aa6ebb40 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f526659c640 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f5265d9b640 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6f89b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f525c0bbac0 tx=0x7f525c0bb7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ddb63b2859a2b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f524dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f524dffb640 1 -- 172.21.15.5:0/3540662003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c049040 con 0x5644aa6ebb40 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f524dffb640 1 -- 172.21.15.5:0/3540662003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f525c077060 con 0x5644aa6ebb40 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f526f109740 1 -- 172.21.15.5:0/3540662003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5258005510 con 0x5644aa6ebb40 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f524dffb640 1 -- 172.21.15.5:0/3540662003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f525c0c4980 con 0x5644aa6ebb40 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f526f109740 10 client.16386 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f526f109740 1 -- 172.21.15.5:0/3540662003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644aa76ef20 con 0x5644aa6ebb40 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.120+0000 7f524dffb640 1 -- 172.21.15.5:0/3540662003 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5258005510 con 0x5644aa6ebb40 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.121+0000 7f524dffb640 1 -- 172.21.15.5:0/3540662003 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f525c0c44f0 con 0x5644aa6ebb40 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.121+0000 7f526f109740 10 client.16386 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.121+0000 7f526f109740 10 client.16386 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.121+0000 7f526f109740 20 client.16386 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.121+0000 7f526f109740 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5644aa76f650 0x5644aa771a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.121+0000 7f526f109740 4 client.16386 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.121+0000 7f526f109740 1 -- 172.21.15.5:0/3540662003 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644aa657d70 con 0x5644aa76f650 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.121+0000 7f526659c640 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5644aa76f650 0x5644aa771a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.121+0000 7f526659c640 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5644aa76f650 0x5644aa771a70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c082f6beaa2a9af server_cookie=191801e609e999e0 in_seq=0 out_seq=0 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.121+0000 7f524dffb640 10 client.16386 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f524dffb640 1 -- 172.21.15.5:0/3540662003 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644aa657d70 con 0x5644aa76f650 2024-09-11T22:25:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f524dffb640 10 client.16386 handle_command_reply: tid=0 2024-09-11T22:25:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 client.16386 shutdown 2024-09-11T22:25:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 -- 172.21.15.5:0/3540662003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 msgr2=0x5644aa6f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6f89b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f525c0bbac0 tx=0x7f525c0bb7e0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 -- 172.21.15.5:0/3540662003 shutdown_connections 2024-09-11T22:25:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5644aa76f650 0x5644aa771a70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f524dffb640 0 client.16386 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:25:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644aa6f5370 0x5644aa6f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644aa6ebb40 0x5644aa6f89b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 --2- 172.21.15.5:0/3540662003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644aa6ea5d0 0x5644aa6f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 -- 172.21.15.5:0/3540662003 >> 172.21.15.5:0/3540662003 conn(0x5644aa6b3310 msgr2=0x5644aa6f37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.143 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:25:28.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 -- 172.21.15.5:0/3540662003 shutdown_connections 2024-09-11T22:25:28.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 1 -- 172.21.15.5:0/3540662003 wait complete. 2024-09-11T22:25:28.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.122+0000 7f526f109740 20 client.16386 trim_cache size 0 max 16384 2024-09-11T22:25:28.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 -- 172.21.15.5:0/3971737958 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52500a2cf0 msgr2=0x7f52500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52500a2cf0 0x7f52500a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5260009fd0 tx=0x7f5260009da0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 -- 172.21.15.5:0/3971737958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52681036e0 msgr2=0x7f5268100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52681036e0 0x7f5268100ed0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f525c005940 tx=0x7f525c0042f0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 -- 172.21.15.5:0/3971737958 shutdown_connections 2024-09-11T22:25:28.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52500a2cf0 0x7f52500a51b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52681082a0 0x7f5268101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52681036e0 0x7f5268100ed0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 --2- 172.21.15.5:0/3971737958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5268102d30 0x7f5268100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 -- 172.21.15.5:0/3971737958 >> 172.21.15.5:0/3971737958 conn(0x7f52680fc7d0 msgr2=0x7f5268105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 -- 172.21.15.5:0/3971737958 shutdown_connections 2024-09-11T22:25:28.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.123+0000 7f526cd30640 1 -- 172.21.15.5:0/3971737958 wait complete. 2024-09-11T22:25:28.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181146 9.40 MB/sec warmup 106 sec latency 25144.897 ms 2024-09-11T22:25:28.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:25:28.393 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:28.480 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:25:28.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:25:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.934+0000 7f9c0c179640 1 -- 172.21.15.5:0/1481773211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04106f10 msgr2=0x7f9c0410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.934+0000 7f9c0c179640 1 --2- 172.21.15.5:0/1481773211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04106f10 0x7f9c0410ea90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009ca0 tx=0x7f9bfc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 -- 172.21.15.5:0/1481773211 shutdown_connections 2024-09-11T22:25:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 --2- 172.21.15.5:0/1481773211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04106f10 0x7f9c0410ea90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 --2- 172.21.15.5:0/1481773211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c04102790 0x7f9c041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 --2- 172.21.15.5:0/1481773211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c04101de0 0x7f9c041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 -- 172.21.15.5:0/1481773211 >> 172.21.15.5:0/1481773211 conn(0x7f9c040fd8c0 msgr2=0x7f9c040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 -- 172.21.15.5:0/1481773211 shutdown_connections 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 -- 172.21.15.5:0/1481773211 wait complete. 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 Processor -- start 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 -- start start 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04101de0 0x7f9c04198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c04102790 0x7f9c041989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c04106f10 0x7f9c0419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04111f70 con 0x7f9c04101de0 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c04111df0 con 0x7f9c04106f10 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041120f0 con 0x7f9c04102790 2024-09-11T22:25:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c09f15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04101de0 0x7f9c04198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c09f15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04101de0 0x7f9c04198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58808/0 (socket says 172.21.15.5:58808) 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c09f15640 1 -- 172.21.15.5:0/1649604481 learned_addr learned my addr 172.21.15.5:0/1649604481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0a716640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c04106f10 0x7f9c0419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c09714640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c04102790 0x7f9c041989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c09f15640 1 -- 172.21.15.5:0/1649604481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c04102790 msgr2=0x7f9c041989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c09f15640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c04102790 0x7f9c041989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c09f15640 1 -- 172.21.15.5:0/1649604481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c04106f10 msgr2=0x7f9c0419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c09f15640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c04106f10 0x7f9c0419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c09f15640 1 -- 172.21.15.5:0/1649604481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0419d440 con 0x7f9c04101de0 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c09f15640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04101de0 0x7f9c04198490 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000e9f0 tx=0x7f9c0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9bfaffd640 1 -- 172.21.15.5:0/1649604481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0000cd80 con 0x7f9c04101de0 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 -- 172.21.15.5:0/1649604481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0419d730 con 0x7f9c04101de0 2024-09-11T22:25:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9bfaffd640 1 -- 172.21.15.5:0/1649604481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c00010910 con 0x7f9c04101de0 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9bfaffd640 1 -- 172.21.15.5:0/1649604481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c00018990 con 0x7f9c04101de0 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.936+0000 7f9c0c179640 1 -- 172.21.15.5:0/1649604481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c041a5010 con 0x7f9c04101de0 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9bfaffd640 1 -- 172.21.15.5:0/1649604481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9c00010ac0 con 0x7f9c04101de0 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c0e552740 1 Processor -- start 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c0e552740 1 -- start start 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 0x556854d7f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7f860 0x556854d8e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d90a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c0e552740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556854ce4620 con 0x556854d8e650 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c0e552740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556854c8be50 con 0x556854d7ef40 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c0e552740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556854ce05e0 con 0x556854d7f860 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c09714640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d90a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c09714640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d90a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58810/0 (socket says 172.21.15.5:58810) 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c09714640 1 -- 172.21.15.5:0/3532378787 learned_addr learned my addr 172.21.15.5:0/3532378787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c0a716640 1 --2- 172.21.15.5:0/3532378787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 0x556854d7f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.939+0000 7f9c09f15640 1 --2- 172.21.15.5:0/3532378787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7f860 0x556854d8e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9c09714640 1 -- 172.21.15.5:0/3532378787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7f860 msgr2=0x556854d8e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9c09714640 1 --2- 172.21.15.5:0/3532378787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7f860 0x556854d8e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9c09714640 1 -- 172.21.15.5:0/3532378787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 msgr2=0x556854d7f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9c09714640 1 --2- 172.21.15.5:0/3532378787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 0x556854d7f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9bfaffd640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9bf00a2cf0 0x7f9bf00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9c09714640 1 -- 172.21.15.5:0/3532378787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556854c7cd40 con 0x556854d8e650 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9c0a716640 1 --2- 172.21.15.5:0/3532378787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 0x556854d7f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9c0a716640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9bf00a2cf0 0x7f9bf00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9bfaffd640 1 -- 172.21.15.5:0/1649604481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f9c000cf850 con 0x7f9c04101de0 2024-09-11T22:25:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9c09714640 1 --2- 172.21.15.5:0/3532378787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d90a40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400bc10 tx=0x7f9bf400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cd7b73c193c77c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9beffff640 1 -- 172.21.15.5:0/3532378787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4015070 con 0x556854d8e650 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9beffff640 1 -- 172.21.15.5:0/3532378787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf4011920 con 0x556854d8e650 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.940+0000 7f9beffff640 1 -- 172.21.15.5:0/3532378787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf40107e0 con 0x556854d8e650 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 -- 172.21.15.5:0/3532378787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 msgr2=0x556854d90a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0a716640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9bf00a2cf0 0x7f9bf00a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc004af0 tx=0x7f9bfc004a80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 --2- 172.21.15.5:0/3532378787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d90a40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400bc10 tx=0x7f9bf400f730 comp rx=0 tx=0).stop 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 -- 172.21.15.5:0/3532378787 shutdown_connections 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 --2- 172.21.15.5:0/3532378787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d90a40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 --2- 172.21.15.5:0/3532378787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7f860 0x556854d8e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 --2- 172.21.15.5:0/3532378787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 0x556854d7f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 -- 172.21.15.5:0/3532378787 >> 172.21.15.5:0/3532378787 conn(0x556854d7b5f0 msgr2=0x556854d7b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 -- 172.21.15.5:0/3532378787 shutdown_connections 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 -- 172.21.15.5:0/3532378787 wait complete. 2024-09-11T22:25:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 -- start start 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.941+0000 7f9c0e552740 1 Processor -- start 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 0x556854d8b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 0x556854d8bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0e552740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556854d91220 con 0x556854d8e650 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0e552740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556854c6cf10 con 0x556854d7f860 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0e552740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556854d913a0 con 0x556854d7ef40 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0e552740 10 client.0 _ll_register_callbacks cb 0x556854c90db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0a716640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0a716640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d8c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58812/0 (socket says 172.21.15.5:58812) 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0a716640 1 -- 172.21.15.5:0/2110761079 learned_addr learned my addr 172.21.15.5:0/2110761079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c09714640 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 0x556854d8bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c09f15640 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 0x556854d8b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0a716640 1 -- 172.21.15.5:0/2110761079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 msgr2=0x556854d8b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0a716640 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 0x556854d8b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0a716640 1 -- 172.21.15.5:0/2110761079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 msgr2=0x556854d8bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.942+0000 7f9c0a716640 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 0x556854d8bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9c0a716640 1 -- 172.21.15.5:0/2110761079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568549f3580 con 0x556854d8e650 2024-09-11T22:25:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9c09714640 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 0x556854d8bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9c0a716640 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d8c000 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc003ab0 tx=0x7f9bfc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557fe70faa3758ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9bed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2110761079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc004250 con 0x556854d8e650 2024-09-11T22:25:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9c0e552740 1 -- 172.21.15.5:0/2110761079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bf4011920 con 0x556854d8e650 2024-09-11T22:25:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2110761079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f9bfc0043f0 con 0x556854d8e650 2024-09-11T22:25:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2110761079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9bfc0045f0 con 0x556854d8e650 2024-09-11T22:25:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9c0e552740 10 client.16398 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9c0e552740 1 -- 172.21.15.5:0/2110761079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556854abd960 con 0x556854d8e650 2024-09-11T22:25:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2110761079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bf4011920 con 0x556854d8e650 2024-09-11T22:25:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.943+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2110761079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9bfc0305c0 con 0x556854d8e650 2024-09-11T22:25:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.944+0000 7f9c0e552740 10 client.16398 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.944+0000 7f9c0e552740 10 client.16398 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:25:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.944+0000 7f9c0e552740 20 client.16398 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.944+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556854d9e340 0x556854da0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.944+0000 7f9c0e552740 4 client.16398 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.944+0000 7f9c0e552740 1 -- 172.21.15.5:0/2110761079 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556854cf1b90 con 0x556854d9e340 2024-09-11T22:25:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.944+0000 7f9c09f15640 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556854d9e340 0x556854da0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.944+0000 7f9bed7fa640 10 client.16398 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.944+0000 7f9c09f15640 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556854d9e340 0x556854da0770 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88de47684ecd33ff server_cookie=7b3b1ab55dc25f9d in_seq=0 out_seq=0 2024-09-11T22:25:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2110761079 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556854cf1b90 con 0x556854d9e340 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9bed7fa640 10 client.16398 handle_command_reply: tid=0 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 client.16398 shutdown 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 -- 172.21.15.5:0/2110761079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 msgr2=0x556854d8c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d8c000 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc003ab0 tx=0x7f9bfc02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 -- 172.21.15.5:0/2110761079 shutdown_connections 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556854d9e340 0x556854da0770 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854d8c000 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9bed7fa640 0 client.16398 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 0x556854d8bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2110761079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 0x556854d8b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 -- 172.21.15.5:0/2110761079 >> 172.21.15.5:0/2110761079 conn(0x556854d7b5f0 msgr2=0x556854d81350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 -- 172.21.15.5:0/2110761079 shutdown_connections 2024-09-11T22:25:28.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 1 -- 172.21.15.5:0/2110761079 wait complete. 2024-09-11T22:25:28.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.945+0000 7f9c0e552740 20 client.16398 trim_cache size 0 max 16384 2024-09-11T22:25:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.962+0000 7f9c0e552740 1 Processor -- start 2024-09-11T22:25:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.962+0000 7f9c0e552740 1 -- start start 2024-09-11T22:25:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.962+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 0x556854d7aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.962+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d7f860 0x556854da6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d8e650 0x556854da65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c0e552740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041ab720 con 0x556854d7f860 2024-09-11T22:25:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c0e552740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041ad320 con 0x556854d7ef40 2024-09-11T22:25:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c0e552740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041abdf0 con 0x556854d8e650 2024-09-11T22:25:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09714640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d7f860 0x556854da6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09714640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d7f860 0x556854da6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58820/0 (socket says 172.21.15.5:58820) 2024-09-11T22:25:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09714640 1 -- 172.21.15.5:0/3523803891 learned_addr learned my addr 172.21.15.5:0/3523803891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09f15640 1 --2- 172.21.15.5:0/3523803891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 0x556854d7aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c0a716640 1 --2- 172.21.15.5:0/3523803891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d8e650 0x556854da65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09f15640 1 -- 172.21.15.5:0/3523803891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d8e650 msgr2=0x556854da65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09f15640 1 --2- 172.21.15.5:0/3523803891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d8e650 0x556854da65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09f15640 1 -- 172.21.15.5:0/3523803891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d7f860 msgr2=0x556854da6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09f15640 1 --2- 172.21.15.5:0/3523803891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d7f860 0x556854da6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09f15640 1 -- 172.21.15.5:0/3523803891 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556854d8a530 con 0x556854d7ef40 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09714640 1 --2- 172.21.15.5:0/3523803891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d7f860 0x556854da6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.963+0000 7f9c09f15640 1 --2- 172.21.15.5:0/3523803891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 0x556854d7aa10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c000ccf60 tx=0x7f9c00000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d45311c5e24349a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9bef7fe640 1 -- 172.21.15.5:0/3523803891 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c000a49a0 con 0x556854d7ef40 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9bef7fe640 1 -- 172.21.15.5:0/3523803891 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c0009a030 con 0x556854d7ef40 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9bef7fe640 1 -- 172.21.15.5:0/3523803891 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0009b470 con 0x556854d7ef40 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9c0e552740 1 -- 172.21.15.5:0/3523803891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 msgr2=0x556854d7aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9c0e552740 1 --2- 172.21.15.5:0/3523803891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 0x556854d7aa10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c000ccf60 tx=0x7f9c00000bc0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9c0e552740 1 -- 172.21.15.5:0/3523803891 shutdown_connections 2024-09-11T22:25:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9c0e552740 1 --2- 172.21.15.5:0/3523803891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d8e650 0x556854da65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9c0e552740 1 --2- 172.21.15.5:0/3523803891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d7f860 0x556854da6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9c0e552740 1 --2- 172.21.15.5:0/3523803891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7ef40 0x556854d7aa10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9c0e552740 1 -- 172.21.15.5:0/3523803891 >> 172.21.15.5:0/3523803891 conn(0x556854d7b5f0 msgr2=0x556854d8cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9c0e552740 1 -- 172.21.15.5:0/3523803891 shutdown_connections 2024-09-11T22:25:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.964+0000 7f9c0e552740 1 -- 172.21.15.5:0/3523803891 wait complete. 2024-09-11T22:25:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.965+0000 7f9c0e552740 1 -- start start 2024-09-11T22:25:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.965+0000 7f9c0e552740 1 Processor -- start 2024-09-11T22:25:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.965+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 0x556854d92cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.965+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 0x556854d93230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.965+0000 7f9c0e552740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854e02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.965+0000 7f9c0e552740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041acb90 con 0x556854d8e650 2024-09-11T22:25:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.965+0000 7f9c0e552740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041ac4c0 con 0x556854d7f860 2024-09-11T22:25:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.965+0000 7f9c0e552740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9c041a9700 con 0x556854d7ef40 2024-09-11T22:25:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c09f15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 0x556854d92cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c09f15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 0x556854d92cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:41756/0 (socket says 172.21.15.5:41756) 2024-09-11T22:25:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c0e552740 10 client.0 _ll_register_callbacks cb 0x7f9bfc003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c09f15640 1 -- 172.21.15.5:0/2313026537 learned_addr learned my addr 172.21.15.5:0/2313026537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c0a716640 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854e02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c09714640 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 0x556854d93230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c09f15640 1 -- 172.21.15.5:0/2313026537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 msgr2=0x556854d93230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c09f15640 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 0x556854d93230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c09f15640 1 -- 172.21.15.5:0/2313026537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 msgr2=0x556854e02090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c09f15640 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854e02090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c09f15640 1 -- 172.21.15.5:0/2313026537 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556854d7f690 con 0x556854d7ef40 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c0a716640 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854e02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9c09f15640 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 0x556854d92cf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c00098190 tx=0x7f9c0009beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53132fca5501cbd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9bed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.966+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2313026537 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0009c430 con 0x556854d7ef40 2024-09-11T22:25:28.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2313026537 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f9c0009b1f0 con 0x556854d7ef40 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9c0e552740 1 -- 172.21.15.5:0/2313026537 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c0009a030 con 0x556854d7ef40 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2313026537 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9c000a49a0 con 0x556854d7ef40 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9c0e552740 10 client.25852 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9c0e552740 1 -- 172.21.15.5:0/2313026537 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556854e09920 con 0x556854d7ef40 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2313026537 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c0009a030 con 0x556854d7ef40 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2313026537 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f9c000b2020 con 0x556854d7ef40 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9c0e552740 10 client.25852 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9c0e552740 10 client.25852 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9c0e552740 20 client.25852 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556854e0a050 0x556854e0c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9c0e552740 4 client.25852 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "299bfba0", "release": true, "await": true}] 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9c0e552740 1 -- 172.21.15.5:0/2313026537 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "299bfba0", "release": true, "await": true}) -- 0x556854cec0a0 con 0x556854e0a050 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.967+0000 7f9c09714640 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556854e0a050 0x556854e0c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.968+0000 7f9c09714640 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556854e0a050 0x556854e0c470 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f22734358f10427d server_cookie=21b26eea0c567381 in_seq=0 out_seq=0 2024-09-11T22:25:28.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.968+0000 7f9bed7fa640 10 client.25852 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9bed7fa640 1 -- 172.21.15.5:0/2313026537 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556854cec0a0 con 0x556854e0a050 2024-09-11T22:25:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9bed7fa640 10 client.25852 handle_command_reply: tid=0 2024-09-11T22:25:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 client.25852 shutdown 2024-09-11T22:25:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 -- 172.21.15.5:0/2313026537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 msgr2=0x556854d92cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 0x556854d92cf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c00098190 tx=0x7f9c0009beb0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 -- 172.21.15.5:0/2313026537 shutdown_connections 2024-09-11T22:25:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556854e0a050 0x556854e0c470 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556854d8e650 0x556854e02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556854d7f860 0x556854d93230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 --2- 172.21.15.5:0/2313026537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556854d7ef40 0x556854d92cf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9bed7fa640 0 client.25852 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 -- 172.21.15.5:0/2313026537 >> 172.21.15.5:0/2313026537 conn(0x556854d7b5f0 msgr2=0x556854d8b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 -- 172.21.15.5:0/2313026537 shutdown_connections 2024-09-11T22:25:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 1 -- 172.21.15.5:0/2313026537 wait complete. 2024-09-11T22:25:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.979+0000 7f9c0e552740 20 client.25852 trim_cache size 0 max 16384 2024-09-11T22:25:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:25:28.982 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 30, 2024-09-11T22:25:28.982 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 12, 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "299bfba0": { 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 12, 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.7, 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:25:28.983 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:25:28.984 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:25:28.984 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:25:28.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:25:28.984 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:25:28.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:28.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:28.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:28.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:28.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:28.984 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:25:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 -- 172.21.15.5:0/1649604481 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9bf00a2cf0 msgr2=0x7f9bf00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9bf00a2cf0 0x7f9bf00a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc004af0 tx=0x7f9bfc004a80 comp rx=0 tx=0).stop 2024-09-11T22:25:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 -- 172.21.15.5:0/1649604481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04101de0 msgr2=0x7f9c04198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04101de0 0x7f9c04198490 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000e9f0 tx=0x7f9c0000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 -- 172.21.15.5:0/1649604481 shutdown_connections 2024-09-11T22:25:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9bf00a2cf0 0x7f9bf00a51b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c04106f10 0x7f9c0419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c04102790 0x7f9c041989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 --2- 172.21.15.5:0/1649604481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c04101de0 0x7f9c04198490 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 -- 172.21.15.5:0/1649604481 >> 172.21.15.5:0/1649604481 conn(0x7f9c040fd8c0 msgr2=0x7f9c040fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 -- 172.21.15.5:0/1649604481 shutdown_connections 2024-09-11T22:25:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:28.980+0000 7f9c0c179640 1 -- 172.21.15.5:0/1649604481 wait complete. 2024-09-11T22:25:29.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 181650 9.31 MB/sec warmup 107 sec latency 25816.946 ms 2024-09-11T22:25:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:29 smithi005 ceph-mon[30943]: pgmap v101: 97 pgs: 97 active+clean; 52 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.9 MiB/s wr, 130 op/s 2024-09-11T22:25:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:29 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:25:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:29 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:25:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:29 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:25:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:29 smithi159 ceph-mon[35534]: pgmap v101: 97 pgs: 97 active+clean; 52 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.9 MiB/s wr, 130 op/s 2024-09-11T22:25:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:29 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:25:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:29 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:25:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:29 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:25:29.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:29 smithi172 ceph-mon[35461]: pgmap v101: 97 pgs: 97 active+clean; 52 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 2.9 MiB/s wr, 130 op/s 2024-09-11T22:25:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:29 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:25:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:29 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:25:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:29 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:25:29.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:25:29.921 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:30.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 184446 9.39 MB/sec warmup 108 sec latency 45.334 ms 2024-09-11T22:25:30.851 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '299bfba0', total seconds elapsed: 30.5 2024-09-11T22:25:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.103+0000 7fdcc69be640 1 -- 172.21.15.5:0/1187379645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc01037a0 msgr2=0x7fdcc010bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.103+0000 7fdcc69be640 1 --2- 172.21.15.5:0/1187379645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc01037a0 0x7fdcc010bc00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009900 tx=0x7fdcb402a410 comp rx=0 tx=0).stop 2024-09-11T22:25:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.103+0000 7fdcc69be640 1 -- 172.21.15.5:0/1187379645 shutdown_connections 2024-09-11T22:25:31.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.103+0000 7fdcc69be640 1 --2- 172.21.15.5:0/1187379645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc010c2f0 0x7fdcc010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.103+0000 7fdcc69be640 1 --2- 172.21.15.5:0/1187379645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc01037a0 0x7fdcc010bc00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.103+0000 7fdcc69be640 1 --2- 172.21.15.5:0/1187379645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc0102df0 0x7fdcc01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.103+0000 7fdcc69be640 1 -- 172.21.15.5:0/1187379645 >> 172.21.15.5:0/1187379645 conn(0x7fdcc00fc7d0 msgr2=0x7fdcc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.103+0000 7fdcc69be640 1 -- 172.21.15.5:0/1187379645 shutdown_connections 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.103+0000 7fdcc69be640 1 -- 172.21.15.5:0/1187379645 wait complete. 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcc69be640 1 Processor -- start 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcc69be640 1 -- start start 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcc69be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102df0 0x7fdcc0198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcc69be640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc01037a0 0x7fdcc0198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcc69be640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc010c2f0 0x7fdcc019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcc69be640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0112100 con 0x7fdcc0102df0 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcc69be640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0111f80 con 0x7fdcc010c2f0 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcc69be640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0112280 con 0x7fdcc01037a0 2024-09-11T22:25:31.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102df0 0x7fdcc0198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102df0 0x7fdcc0198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58850/0 (socket says 172.21.15.5:58850) 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcbffff640 1 -- 172.21.15.5:0/3162049108 learned_addr learned my addr 172.21.15.5:0/3162049108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcc4f5b640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc010c2f0 0x7fdcc019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.104+0000 7fdcbf7fe640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc01037a0 0x7fdcc0198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcbffff640 1 -- 172.21.15.5:0/3162049108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc01037a0 msgr2=0x7fdcc0198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcbffff640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc01037a0 0x7fdcc0198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcbffff640 1 -- 172.21.15.5:0/3162049108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc010c2f0 msgr2=0x7fdcc019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcbffff640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc010c2f0 0x7fdcc019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcbffff640 1 -- 172.21.15.5:0/3162049108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc019d5d0 con 0x7fdcc0102df0 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcbffff640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102df0 0x7fdcc0198620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000d950 tx=0x7fdcb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcbd7fa640 1 -- 172.21.15.5:0/3162049108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0017030 con 0x7fdcc0102df0 2024-09-11T22:25:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcc69be640 1 -- 172.21.15.5:0/3162049108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc019d8c0 con 0x7fdcc0102df0 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcbd7fa640 1 -- 172.21.15.5:0/3162049108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb0004020 con 0x7fdcc0102df0 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcbd7fa640 1 -- 172.21.15.5:0/3162049108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00047a0 con 0x7fdcc0102df0 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.105+0000 7fdcc69be640 1 -- 172.21.15.5:0/3162049108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01a51a0 con 0x7fdcc0102df0 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.107+0000 7fdcbd7fa640 1 -- 172.21.15.5:0/3162049108 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fdcb0004940 con 0x7fdcc0102df0 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc8d97740 1 Processor -- start 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc8d97740 1 -- start start 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 0x556826cc0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 0x556826ccd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556826c262f0 con 0x556826ccb370 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556826bcdcf0 con 0x556826cc1b40 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556826c222b0 con 0x556826cc05d0 2024-09-11T22:25:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc4f5b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc4f5b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48692/0 (socket says 172.21.15.5:48692) 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcbf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 0x556826ccd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcc4f5b640 1 -- 172.21.15.5:0/650316410 learned_addr learned my addr 172.21.15.5:0/650316410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcbffff640 1 --2- 172.21.15.5:0/650316410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 0x556826cc0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.108+0000 7fdcbd7fa640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdca80a2cf0 0x7fdca80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdcc4f5b640 1 -- 172.21.15.5:0/650316410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 msgr2=0x556826cc0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdcc4f5b640 1 --2- 172.21.15.5:0/650316410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 0x556826cc0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdcc4f5b640 1 -- 172.21.15.5:0/650316410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 msgr2=0x556826ccd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdcc4f5b640 1 --2- 172.21.15.5:0/650316410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 0x556826ccd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdcc4f5b640 1 -- 172.21.15.5:0/650316410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556826bb23a0 con 0x556826cc1b40 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdcbd7fa640 1 -- 172.21.15.5:0/3162049108 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fdcb00cee60 con 0x7fdcc0102df0 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdcbf7fe640 1 --2- 172.21.15.5:0/650316410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 0x556826ccd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdcc4f5b640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdca80a2cf0 0x7fdca80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdcc4f5b640 1 --2- 172.21.15.5:0/650316410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc0090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8011590 tx=0x7fdcb8011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cdfa3038911b1d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdca67fc640 1 -- 172.21.15.5:0/650316410 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8017050 con 0x556826cc1b40 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdcc4f5b640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdca80a2cf0 0x7fdca80a51b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8022000 tx=0x7fdcb8023000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdca67fc640 1 -- 172.21.15.5:0/650316410 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb800f7a0 con 0x556826cc1b40 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.109+0000 7fdca67fc640 1 -- 172.21.15.5:0/650316410 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb801a360 con 0x556826cc1b40 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.110+0000 7fdcc8d97740 1 -- 172.21.15.5:0/650316410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 msgr2=0x556826cc0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.110+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/650316410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc0090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8011590 tx=0x7fdcb8011a50 comp rx=0 tx=0).stop 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.110+0000 7fdcc8d97740 1 -- 172.21.15.5:0/650316410 shutdown_connections 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.110+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/650316410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 0x556826ccd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.110+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/650316410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 0x556826cc0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.110+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/650316410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc0090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.110+0000 7fdcc8d97740 1 -- 172.21.15.5:0/650316410 >> 172.21.15.5:0/650316410 conn(0x556826c89310 msgr2=0x556826cbcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.110+0000 7fdcc8d97740 1 -- 172.21.15.5:0/650316410 shutdown_connections 2024-09-11T22:25:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.110+0000 7fdcc8d97740 1 -- 172.21.15.5:0/650316410 wait complete. 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcc8d97740 1 -- start start 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcc8d97740 1 Processor -- start 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 0x556826ccadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 0x556826cc87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556826cce070 con 0x556826ccb370 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556826be1060 con 0x556826cc1b40 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556826cce1f0 con 0x556826cc05d0 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcc8d97740 10 client.0 _ll_register_callbacks cb 0x5568269348c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcc4f5b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 0x556826cc87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcbf7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcbf7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48694/0 (socket says 172.21.15.5:48694) 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcbf7fe640 1 -- 172.21.15.5:0/1009594165 learned_addr learned my addr 172.21.15.5:0/1009594165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.111+0000 7fdcbffff640 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 0x556826ccadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcbf7fe640 1 -- 172.21.15.5:0/1009594165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 msgr2=0x556826ccadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcbf7fe640 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 0x556826ccadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcbf7fe640 1 -- 172.21.15.5:0/1009594165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 msgr2=0x556826cc87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcbf7fe640 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 0x556826cc87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcbf7fe640 1 -- 172.21.15.5:0/1009594165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568269973c0 con 0x556826cc1b40 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcc4f5b640 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 0x556826cc87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcbffff640 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 0x556826ccadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcbf7fe640 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc8060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009900 tx=0x7fdcb40043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e796ad4a8c3ef6b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdc9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/1009594165 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4035040 con 0x556826cc1b40 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcc8d97740 1 -- 172.21.15.5:0/1009594165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556826c33860 con 0x556826cc1b40 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/1009594165 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fdcb4005450 con 0x556826cc1b40 2024-09-11T22:25:31.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/1009594165 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdcb4005650 con 0x556826cc1b40 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcc8d97740 10 client.25925 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.112+0000 7fdcc8d97740 1 -- 172.21.15.5:0/1009594165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556826cd9bb0 con 0x556826cc1b40 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/1009594165 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556826c33860 con 0x556826cc1b40 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/1009594165 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fdcb4003aa0 con 0x556826cc1b40 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdcc8d97740 10 client.25925 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdcc8d97740 10 client.25925 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdcc8d97740 20 client.25925 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x556826cdafc0 0x556826cdd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdcc8d97740 4 client.25925 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdcc8d97740 1 -- 172.21.15.5:0/1009594165 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556826c2dd70 con 0x556826cdafc0 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdcbffff640 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x556826cdafc0 0x556826cdd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdc9f7fe640 10 client.25925 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.113+0000 7fdcbffff640 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x556826cdafc0 0x556826cdd400 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=990b4c700563d0e7 server_cookie=bc7db88ad1c947b7 in_seq=0 out_seq=0 2024-09-11T22:25:31.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/1009594165 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556826c2dd70 con 0x556826cdafc0 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdc9f7fe640 10 client.25925 handle_command_reply: tid=0 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 client.25925 shutdown 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 -- 172.21.15.5:0/1009594165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 msgr2=0x556826cc8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc8060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009900 tx=0x7fdcb40043b0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 -- 172.21.15.5:0/1009594165 shutdown_connections 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826ccb370 0x556826cc87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x556826cdafc0 0x556826cdd400 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826cc8060 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdc9f7fe640 0 client.25925 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/1009594165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc05d0 0x556826ccadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 -- 172.21.15.5:0/1009594165 >> 172.21.15.5:0/1009594165 conn(0x556826c89310 msgr2=0x556826cc2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:31.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 -- 172.21.15.5:0/1009594165 shutdown_connections 2024-09-11T22:25:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 1 -- 172.21.15.5:0/1009594165 wait complete. 2024-09-11T22:25:31.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.114+0000 7fdcc8d97740 20 client.25925 trim_cache size 0 max 16384 2024-09-11T22:25:31.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.128+0000 7fdcc8d97740 1 Processor -- start 2024-09-11T22:25:31.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.128+0000 7fdcc8d97740 1 -- start start 2024-09-11T22:25:31.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.128+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826cc7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.128+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc1b40 0x556826ce2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.128+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826ccb370 0x556826ce3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.128+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01ada50 con 0x556826cc05d0 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.128+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01af650 con 0x556826ccb370 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.128+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01ae120 con 0x556826cc1b40 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826cc7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826cc7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58882/0 (socket says 172.21.15.5:58882) 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbffff640 1 -- 172.21.15.5:0/4290441095 learned_addr learned my addr 172.21.15.5:0/4290441095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbf7fe640 1 --2- 172.21.15.5:0/4290441095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc1b40 0x556826ce2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcc4f5b640 1 --2- 172.21.15.5:0/4290441095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826ccb370 0x556826ce3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbffff640 1 -- 172.21.15.5:0/4290441095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc1b40 msgr2=0x556826ce2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbffff640 1 --2- 172.21.15.5:0/4290441095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc1b40 0x556826ce2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbffff640 1 -- 172.21.15.5:0/4290441095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826ccb370 msgr2=0x556826ce3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:31.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbffff640 1 --2- 172.21.15.5:0/4290441095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826ccb370 0x556826ce3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbffff640 1 -- 172.21.15.5:0/4290441095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556826cbddd0 con 0x556826cc05d0 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbf7fe640 1 --2- 172.21.15.5:0/4290441095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc1b40 0x556826ce2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdcbffff640 1 --2- 172.21.15.5:0/4290441095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826cc7040 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0097740 tx=0x7fdcb00d4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe25c00bc8337e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdca5ffb640 1 -- 172.21.15.5:0/4290441095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0084030 con 0x556826cc05d0 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdca5ffb640 1 -- 172.21.15.5:0/4290441095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb0095e50 con 0x556826cc05d0 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.129+0000 7fdca5ffb640 1 -- 172.21.15.5:0/4290441095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00041d0 con 0x556826cc05d0 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.130+0000 7fdcc8d97740 1 -- 172.21.15.5:0/4290441095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 msgr2=0x556826cc7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.130+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/4290441095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826cc7040 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0097740 tx=0x7fdcb00d4040 comp rx=0 tx=0).stop 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.130+0000 7fdcc8d97740 1 -- 172.21.15.5:0/4290441095 shutdown_connections 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.130+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/4290441095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826ccb370 0x556826ce3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.130+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/4290441095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826cc1b40 0x556826ce2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.130+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/4290441095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826cc7040 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.130+0000 7fdcc8d97740 1 -- 172.21.15.5:0/4290441095 >> 172.21.15.5:0/4290441095 conn(0x556826c89310 msgr2=0x556826cc6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.130+0000 7fdcc8d97740 1 -- 172.21.15.5:0/4290441095 shutdown_connections 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.130+0000 7fdcc8d97740 1 -- 172.21.15.5:0/4290441095 wait complete. 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcc8d97740 1 -- start start 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcc8d97740 1 Processor -- start 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826ccf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826ccf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcc8d97740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826ccb370 0x556826ccfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01aeec0 con 0x556826cc05d0 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01ae7f0 con 0x556826cc1b40 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcc8d97740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc01aba30 con 0x556826ccb370 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcc8d97740 10 client.0 _ll_register_callbacks cb 0x7fdcb4009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:31.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826ccf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcbffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826ccf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58894/0 (socket says 172.21.15.5:58894) 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcbffff640 1 -- 172.21.15.5:0/706044704 learned_addr learned my addr 172.21.15.5:0/706044704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcbf7fe640 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826ccf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.131+0000 7fdcc4f5b640 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826ccb370 0x556826ccfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdcbffff640 1 -- 172.21.15.5:0/706044704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826ccb370 msgr2=0x556826ccfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdcbffff640 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826ccb370 0x556826ccfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdcbffff640 1 -- 172.21.15.5:0/706044704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 msgr2=0x556826ccf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdcbffff640 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826ccf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdcbffff640 1 -- 172.21.15.5:0/706044704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556826cc3120 con 0x556826cc05d0 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdcc4f5b640 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826ccb370 0x556826ccfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdcbffff640 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826ccf2a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009610 tx=0x7fdcb009f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a64cf47b7372622 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdc9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:31.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/706044704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb0084040 con 0x556826cc05d0 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/706044704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fdcb00041d0 con 0x556826cc05d0 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdcc8d97740 1 -- 172.21.15.5:0/706044704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb0095e50 con 0x556826cc05d0 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/706044704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdcb00cc790 con 0x556826cc05d0 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdcc8d97740 10 client.16422 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.132+0000 7fdcc8d97740 1 -- 172.21.15.5:0/706044704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556826d45e80 con 0x556826cc05d0 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/706044704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb0095e50 con 0x556826cc05d0 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/706044704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fdcb0004c30 con 0x556826cc05d0 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdcc8d97740 10 client.16422 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdcc8d97740 10 client.16422 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdcc8d97740 20 client.16422 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x556826d46540 0x556826d48930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdcc8d97740 4 client.16422 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:25:31.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdcc8d97740 1 -- 172.21.15.5:0/706044704 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556826baa170 con 0x556826d46540 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdcbf7fe640 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x556826d46540 0x556826d48930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdc9f7fe640 10 client.16422 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.133+0000 7fdcbf7fe640 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x556826d46540 0x556826d48930 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=195f67782786ee03 server_cookie=a90f20639bf5439 in_seq=0 out_seq=0 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdc9f7fe640 1 -- 172.21.15.5:0/706044704 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556826baa170 con 0x556826d46540 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdc9f7fe640 10 client.16422 handle_command_reply: tid=0 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 client.16422 shutdown 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 -- 172.21.15.5:0/706044704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 msgr2=0x556826ccf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826ccf2a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009610 tx=0x7fdcb009f690 comp rx=0 tx=0).stop 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 -- 172.21.15.5:0/706044704 shutdown_connections 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556826ccb370 0x556826ccfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x556826d46540 0x556826d48930 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556826cc1b40 0x556826ccf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 --2- 172.21.15.5:0/706044704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556826cc05d0 0x556826ccf2a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 -- 172.21.15.5:0/706044704 >> 172.21.15.5:0/706044704 conn(0x556826c89310 msgr2=0x556826cc7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:31.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdc9f7fe640 0 client.16422 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:25:31.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 -- 172.21.15.5:0/706044704 shutdown_connections 2024-09-11T22:25:31.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 1 -- 172.21.15.5:0/706044704 wait complete. 2024-09-11T22:25:31.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.134+0000 7fdcc8d97740 20 client.16422 trim_cache size 0 max 16384 2024-09-11T22:25:31.147 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 -- 172.21.15.5:0/3162049108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdca80a2cf0 msgr2=0x7fdca80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdca80a2cf0 0x7fdca80a51b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8022000 tx=0x7fdcb8023000 comp rx=0 tx=0).stop 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 -- 172.21.15.5:0/3162049108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102df0 msgr2=0x7fdcc0198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102df0 0x7fdcc0198620 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000d950 tx=0x7fdcb000de10 comp rx=0 tx=0).stop 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 -- 172.21.15.5:0/3162049108 shutdown_connections 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdca80a2cf0 0x7fdca80a51b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc010c2f0 0x7fdcc019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc01037a0 0x7fdcc0198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 --2- 172.21.15.5:0/3162049108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc0102df0 0x7fdcc0198620 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 -- 172.21.15.5:0/3162049108 >> 172.21.15.5:0/3162049108 conn(0x7fdcc00fc7d0 msgr2=0x7fdcc00fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 -- 172.21.15.5:0/3162049108 shutdown_connections 2024-09-11T22:25:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:31.135+0000 7fdcc69be640 1 -- 172.21.15.5:0/3162049108 wait complete. 2024-09-11T22:25:31.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 186791 9.38 MB/sec warmup 109 sec latency 33.436 ms 2024-09-11T22:25:31.411 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:31 smithi172 ceph-mon[35461]: pgmap v102: 97 pgs: 97 active+clean; 52 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-11T22:25:31.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:31 smithi005 ceph-mon[30943]: pgmap v102: 97 pgs: 97 active+clean; 52 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-11T22:25:31.578 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:31 smithi159 ceph-mon[35534]: pgmap v102: 97 pgs: 97 active+clean; 52 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.8 MiB/s wr, 87 op/s 2024-09-11T22:25:31.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:32.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 188920 9.41 MB/sec warmup 110 sec latency 10.890 ms 2024-09-11T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.622+0000 7f1823bdf640 1 -- 172.21.15.5:0/3596919150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f181c10c170 msgr2=0x7f181c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.622+0000 7f1823bdf640 1 --2- 172.21.15.5:0/3596919150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f181c10c170 0x7f181c10e610 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1814009ca0 tx=0x7f181402a950 comp rx=0 tx=0).stop 2024-09-11T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.622+0000 7f1823bdf640 1 -- 172.21.15.5:0/3596919150 shutdown_connections 2024-09-11T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.622+0000 7f1823bdf640 1 --2- 172.21.15.5:0/3596919150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f181c10c170 0x7f181c10e610 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.622+0000 7f1823bdf640 1 --2- 172.21.15.5:0/3596919150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f181c103660 0x7f181c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.622+0000 7f1823bdf640 1 --2- 172.21.15.5:0/3596919150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f181c102d40 0x7f181c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.622+0000 7f1823bdf640 1 -- 172.21.15.5:0/3596919150 >> 172.21.15.5:0/3596919150 conn(0x7f181c0fc7a0 msgr2=0x7f181c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.622+0000 7f1823bdf640 1 -- 172.21.15.5:0/3596919150 shutdown_connections 2024-09-11T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.622+0000 7f1823bdf640 1 -- 172.21.15.5:0/3596919150 wait complete. 2024-09-11T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f1823bdf640 1 Processor -- start 2024-09-11T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f1823bdf640 1 -- start start 2024-09-11T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f1823bdf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f181c102d40 0x7f181c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f1823bdf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f181c103660 0x7f181c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f1823bdf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f181c10c170 0x7f181c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f1823bdf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f181c111f80 con 0x7f181c10c170 2024-09-11T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f1823bdf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f181c111e00 con 0x7f181c102d40 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f1823bdf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f181c112100 con 0x7f181c103660 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f182197b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f181c102d40 0x7f181c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f182217c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f181c10c170 0x7f181c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f182197b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f181c102d40 0x7f181c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57136/0 (socket says 172.21.15.5:57136) 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f182197b640 1 -- 172.21.15.5:0/2481524509 learned_addr learned my addr 172.21.15.5:0/2481524509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f182117a640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f181c103660 0x7f181c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f182217c640 1 -- 172.21.15.5:0/2481524509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f181c103660 msgr2=0x7f181c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f182217c640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f181c103660 0x7f181c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f182217c640 1 -- 172.21.15.5:0/2481524509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f181c102d40 msgr2=0x7f181c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f182217c640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f181c102d40 0x7f181c1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.623+0000 7f182217c640 1 -- 172.21.15.5:0/2481524509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c1a6870 con 0x7f181c10c170 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.624+0000 7f182197b640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f181c102d40 0x7f181c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.624+0000 7f182217c640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f181c10c170 0x7f181c101420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1814005e30 tx=0x7f1814003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.624+0000 7f1812ffd640 1 -- 172.21.15.5:0/2481524509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814034040 con 0x7f181c10c170 2024-09-11T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.624+0000 7f1823bdf640 1 -- 172.21.15.5:0/2481524509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c1a6b00 con 0x7f181c10c170 2024-09-11T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.624+0000 7f1812ffd640 1 -- 172.21.15.5:0/2481524509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f181402fe20 con 0x7f181c10c170 2024-09-11T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.624+0000 7f1812ffd640 1 -- 172.21.15.5:0/2481524509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1814004680 con 0x7f181c10c170 2024-09-11T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.624+0000 7f1823bdf640 1 -- 172.21.15.5:0/2481524509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c1a7030 con 0x7f181c10c170 2024-09-11T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.626+0000 7f1812ffd640 1 -- 172.21.15.5:0/2481524509 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1814037020 con 0x7f181c10c170 2024-09-11T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f1825fb8740 1 Processor -- start 2024-09-11T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f1825fb8740 1 -- start start 2024-09-11T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18975090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d189755d0 0x558d18975a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18980370 0x558d18982760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f1825fb8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558d188db2f0 con 0x558d18976b40 2024-09-11T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f1825fb8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558d18882cf0 con 0x558d189755d0 2024-09-11T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f1825fb8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558d188d72b0 con 0x558d18980370 2024-09-11T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f1812ffd640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18080a2d70 0x7f18080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f182197b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d189755d0 0x558d18975a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.627+0000 7f182117a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18975090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182117a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18975090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36962/0 (socket says 172.21.15.5:36962) 2024-09-11T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182117a640 1 -- 172.21.15.5:0/2794015091 learned_addr learned my addr 172.21.15.5:0/2794015091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f1812ffd640 1 -- 172.21.15.5:0/2481524509 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f18140effa0 con 0x7f181c10c170 2024-09-11T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182217c640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18080a2d70 0x7f18080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182117a640 1 --2- 172.21.15.5:0/2794015091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18980370 0x558d18982760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182117a640 1 -- 172.21.15.5:0/2794015091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18980370 msgr2=0x558d18982760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182117a640 1 --2- 172.21.15.5:0/2794015091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18980370 0x558d18982760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182117a640 1 -- 172.21.15.5:0/2794015091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d189755d0 msgr2=0x558d18975a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182117a640 1 --2- 172.21.15.5:0/2794015091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d189755d0 0x558d18975a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182117a640 1 -- 172.21.15.5:0/2794015091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d188673a0 con 0x558d18976b40 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182217c640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18080a2d70 0x7f18080a5230 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18140bc290 tx=0x7f181402a950 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f182117a640 1 --2- 172.21.15.5:0/2794015091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18975090 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f180c0058a0 tx=0x7f180c00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9488c7fcee13a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f1807fff640 1 -- 172.21.15.5:0/2794015091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c0120d0 con 0x558d18976b40 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f1807fff640 1 -- 172.21.15.5:0/2794015091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c012c80 con 0x558d18976b40 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.628+0000 7f1807fff640 1 -- 172.21.15.5:0/2794015091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c0055d0 con 0x558d18976b40 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.629+0000 7f1825fb8740 1 -- 172.21.15.5:0/2794015091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 msgr2=0x558d18975090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.629+0000 7f1825fb8740 1 --2- 172.21.15.5:0/2794015091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18975090 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f180c0058a0 tx=0x7f180c00b6a0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.629+0000 7f1825fb8740 1 -- 172.21.15.5:0/2794015091 shutdown_connections 2024-09-11T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.629+0000 7f1825fb8740 1 --2- 172.21.15.5:0/2794015091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18980370 0x558d18982760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.629+0000 7f1825fb8740 1 --2- 172.21.15.5:0/2794015091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d189755d0 0x558d18975a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.629+0000 7f1825fb8740 1 --2- 172.21.15.5:0/2794015091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18975090 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.629+0000 7f1825fb8740 1 -- 172.21.15.5:0/2794015091 >> 172.21.15.5:0/2794015091 conn(0x558d1893e310 msgr2=0x558d18971c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.629+0000 7f1825fb8740 1 -- 172.21.15.5:0/2794015091 shutdown_connections 2024-09-11T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.629+0000 7f1825fb8740 1 -- 172.21.15.5:0/2794015091 wait complete. 2024-09-11T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f1825fb8740 1 -- start start 2024-09-11T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f1825fb8740 1 Processor -- start 2024-09-11T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d189755d0 0x558d1897fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18976b40 0x558d1897cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18980370 0x558d1897d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f1825fb8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558d18983010 con 0x558d18980370 2024-09-11T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f1825fb8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558d18896060 con 0x558d18976b40 2024-09-11T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f1825fb8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558d18983190 con 0x558d189755d0 2024-09-11T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f1825fb8740 10 client.0 _ll_register_callbacks cb 0x558d185e98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f182117a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18980370 0x558d1897d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.630+0000 7f182117a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18980370 0x558d1897d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36978/0 (socket says 172.21.15.5:36978) 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f182117a640 1 -- 172.21.15.5:0/3996980902 learned_addr learned my addr 172.21.15.5:0/3996980902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f182197b640 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18976b40 0x558d1897cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f182117a640 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d189755d0 0x558d1897fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f182117a640 1 -- 172.21.15.5:0/3996980902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d189755d0 msgr2=0x558d1897fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f182117a640 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d189755d0 0x558d1897fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f182117a640 1 -- 172.21.15.5:0/3996980902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18976b40 msgr2=0x558d1897cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f182117a640 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18976b40 0x558d1897cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f182117a640 1 -- 172.21.15.5:0/3996980902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d1864c3c0 con 0x558d18980370 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f182197b640 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18976b40 0x558d1897cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f182117a640 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18980370 0x558d1897d530 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f180c012f50 tx=0x7f180c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d49c2a2d5ec5ddf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f18057fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f18057fa640 1 -- 172.21.15.5:0/3996980902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c00e960 con 0x558d18980370 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f18057fa640 1 -- 172.21.15.5:0/3996980902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f180c00eb00 con 0x558d18980370 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f1825fb8740 1 -- 172.21.15.5:0/3996980902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180c012c80 con 0x558d18980370 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f18057fa640 1 -- 172.21.15.5:0/3996980902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f180c00ed00 con 0x558d18980370 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f1825fb8740 10 client.16434 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.631+0000 7f1825fb8740 1 -- 172.21.15.5:0/3996980902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d1898ea40 con 0x558d18980370 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.632+0000 7f18057fa640 1 -- 172.21.15.5:0/3996980902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c012c80 con 0x558d18980370 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.632+0000 7f18057fa640 1 -- 172.21.15.5:0/3996980902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f180c0104f0 con 0x558d18980370 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.632+0000 7f1825fb8740 10 client.16434 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.632+0000 7f1825fb8740 10 client.16434 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.632+0000 7f1825fb8740 20 client.16434 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.632+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x558d1898fde0 0x558d189921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.632+0000 7f1825fb8740 4 client.16434 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.632+0000 7f1825fb8740 1 -- 172.21.15.5:0/3996980902 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d188e8860 con 0x558d1898fde0 2024-09-11T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.632+0000 7f182197b640 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x558d1898fde0 0x558d189921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.633+0000 7f182197b640 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x558d1898fde0 0x558d189921d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42810029bc60934f server_cookie=6d61881e06da6b1d in_seq=0 out_seq=0 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.633+0000 7f18057fa640 10 client.16434 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.633+0000 7f18057fa640 1 -- 172.21.15.5:0/3996980902 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d188e8860 con 0x558d1898fde0 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.633+0000 7f18057fa640 10 client.16434 handle_command_reply: tid=0 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.633+0000 7f1825fb8740 1 client.16434 shutdown 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.633+0000 7f1825fb8740 1 -- 172.21.15.5:0/3996980902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18980370 msgr2=0x558d1897d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.633+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18980370 0x558d1897d530 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f180c012f50 tx=0x7f180c014040 comp rx=0 tx=0).stop 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.634+0000 7f1825fb8740 1 -- 172.21.15.5:0/3996980902 shutdown_connections 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.634+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x558d1898fde0 0x558d189921d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.634+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18980370 0x558d1897d530 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.634+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18976b40 0x558d1897cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.634+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3996980902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d189755d0 0x558d1897fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.634+0000 7f1825fb8740 1 -- 172.21.15.5:0/3996980902 >> 172.21.15.5:0/3996980902 conn(0x558d1893e310 msgr2=0x558d18977930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.634+0000 7f18057fa640 0 client.16434 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.634+0000 7f1825fb8740 1 -- 172.21.15.5:0/3996980902 shutdown_connections 2024-09-11T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.634+0000 7f1825fb8740 1 -- 172.21.15.5:0/3996980902 wait complete. 2024-09-11T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.634+0000 7f1825fb8740 20 client.16434 trim_cache size 0 max 16384 2024-09-11T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f1825fb8740 1 Processor -- start 2024-09-11T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f1825fb8740 1 -- start start 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d189755d0 0x558d1898c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18976b40 0x558d18978120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 0x558d18978660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f1825fb8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1afbf0 con 0x558d189755d0 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f1825fb8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1b17f0 con 0x558d18980370 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f1825fb8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1b02c0 con 0x558d18976b40 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182117a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d189755d0 0x558d1898c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182117a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d189755d0 0x558d1898c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36982/0 (socket says 172.21.15.5:36982) 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182117a640 1 -- 172.21.15.5:0/372415279 learned_addr learned my addr 172.21.15.5:0/372415279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182197b640 1 --2- 172.21.15.5:0/372415279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18976b40 0x558d18978120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182197b640 1 -- 172.21.15.5:0/372415279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 msgr2=0x558d18978660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182117a640 1 --2- 172.21.15.5:0/372415279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 0x558d18978660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182197b640 1 --2- 172.21.15.5:0/372415279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 0x558d18978660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182197b640 1 -- 172.21.15.5:0/372415279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d189755d0 msgr2=0x558d1898c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182197b640 1 --2- 172.21.15.5:0/372415279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d189755d0 0x558d1898c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182197b640 1 -- 172.21.15.5:0/372415279 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d18978ba0 con 0x558d18976b40 2024-09-11T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.648+0000 7f182117a640 1 --2- 172.21.15.5:0/372415279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 0x558d18978660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.649+0000 7f182197b640 1 --2- 172.21.15.5:0/372415279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18976b40 0x558d18978120 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x558d18992820 tx=0x7f1818005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1754e0c1ad8c97aa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.649+0000 7f18077fe640 1 -- 172.21.15.5:0/372415279 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18180120d0 con 0x558d18976b40 2024-09-11T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.649+0000 7f18077fe640 1 -- 172.21.15.5:0/372415279 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1818012270 con 0x558d18976b40 2024-09-11T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.649+0000 7f18077fe640 1 -- 172.21.15.5:0/372415279 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818010430 con 0x558d18976b40 2024-09-11T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 -- 172.21.15.5:0/372415279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18976b40 msgr2=0x558d18978120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 --2- 172.21.15.5:0/372415279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18976b40 0x558d18978120 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x558d18992820 tx=0x7f1818005f90 comp rx=0 tx=0).stop 2024-09-11T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 -- 172.21.15.5:0/372415279 shutdown_connections 2024-09-11T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 --2- 172.21.15.5:0/372415279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 0x558d18978660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 --2- 172.21.15.5:0/372415279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18976b40 0x558d18978120 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 --2- 172.21.15.5:0/372415279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d189755d0 0x558d1898c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 -- 172.21.15.5:0/372415279 >> 172.21.15.5:0/372415279 conn(0x558d1893e310 msgr2=0x558d1897b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 -- 172.21.15.5:0/372415279 shutdown_connections 2024-09-11T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 -- 172.21.15.5:0/372415279 wait complete. 2024-09-11T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 -- start start 2024-09-11T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.650+0000 7f1825fb8740 1 Processor -- start 2024-09-11T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d189755d0 0x558d18988950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18983c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f1825fb8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 0x558d189841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f1825fb8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1b1060 con 0x558d18976b40 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f1825fb8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1b0990 con 0x558d18980370 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f1825fb8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f181c1adbd0 con 0x558d189755d0 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f1825fb8740 10 client.0 _ll_register_callbacks cb 0x7f1818009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182197b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18983c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182197b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18983c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36992/0 (socket says 172.21.15.5:36992) 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182197b640 1 -- 172.21.15.5:0/3226535398 learned_addr learned my addr 172.21.15.5:0/3226535398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182117a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d189755d0 0x558d18988950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182117a640 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 0x558d189841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182197b640 1 -- 172.21.15.5:0/3226535398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d189755d0 msgr2=0x558d18988950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182197b640 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d189755d0 0x558d18988950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182197b640 1 -- 172.21.15.5:0/3226535398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 msgr2=0x558d189841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182197b640 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 0x558d189841c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182197b640 1 -- 172.21.15.5:0/3226535398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d189847b0 con 0x558d18976b40 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f182197b640 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18983c80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x558d18972a30 tx=0x7f1818009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f372f758374bf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.651+0000 7f18057fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f18057fa640 1 -- 172.21.15.5:0/3226535398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818004700 con 0x558d18976b40 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f18057fa640 1 -- 172.21.15.5:0/3226535398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7f18180048a0 con 0x558d18976b40 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f1825fb8740 1 -- 172.21.15.5:0/3226535398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1818012270 con 0x558d18976b40 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f18057fa640 1 -- 172.21.15.5:0/3226535398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1818004030 con 0x558d18976b40 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f1825fb8740 10 client.16440 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f1825fb8740 1 -- 172.21.15.5:0/3226535398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d189fa5a0 con 0x558d18976b40 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f18057fa640 1 -- 172.21.15.5:0/3226535398 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1818012270 con 0x558d18976b40 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f18057fa640 1 -- 172.21.15.5:0/3226535398 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1818038030 con 0x558d18976b40 2024-09-11T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f1825fb8740 10 client.16440 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f1825fb8740 10 client.16440 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f1825fb8740 20 client.16440 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x558d189faca0 0x558d189fd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f1825fb8740 4 client.16440 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f1825fb8740 1 -- 172.21.15.5:0/3226535398 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d188e2d70 con 0x558d189faca0 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.652+0000 7f182117a640 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x558d189faca0 0x558d189fd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f182117a640 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x558d189faca0 0x558d189fd0c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba120d98862fc3a2 server_cookie=6e552875f853cd87 in_seq=0 out_seq=0 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f18057fa640 10 client.16440 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f18057fa640 1 -- 172.21.15.5:0/3226535398 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f180c016020 con 0x558d189faca0 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f18057fa640 10 client.16440 handle_command_reply: tid=0 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 client.16440 shutdown 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 -- 172.21.15.5:0/3226535398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 msgr2=0x558d18983c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18983c80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x558d18972a30 tx=0x7f1818009310 comp rx=0 tx=0).stop 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 -- 172.21.15.5:0/3226535398 shutdown_connections 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x558d189faca0 0x558d189fd0c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18980370 0x558d189841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18976b40 0x558d18983c80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f18057fa640 0 client.16440 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 --2- 172.21.15.5:0/3226535398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d189755d0 0x558d18988950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 -- 172.21.15.5:0/3226535398 >> 172.21.15.5:0/3226535398 conn(0x558d1893e310 msgr2=0x558d1897e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 -- 172.21.15.5:0/3226535398 shutdown_connections 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.653+0000 7f1825fb8740 1 -- 172.21.15.5:0/3226535398 wait complete. 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1825fb8740 20 client.16440 trim_cache size 0 max 16384 2024-09-11T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 -- 172.21.15.5:0/2481524509 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18080a2d70 msgr2=0x7f18080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18080a2d70 0x7f18080a5230 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18140bc290 tx=0x7f181402a950 comp rx=0 tx=0).stop 2024-09-11T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 -- 172.21.15.5:0/2481524509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f181c10c170 msgr2=0x7f181c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f181c10c170 0x7f181c101420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1814005e30 tx=0x7f1814003c10 comp rx=0 tx=0).stop 2024-09-11T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 -- 172.21.15.5:0/2481524509 shutdown_connections 2024-09-11T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f181c10c170 0x7f181c101420 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f181c103660 0x7f181c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18080a2d70 0x7f18080a5230 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 --2- 172.21.15.5:0/2481524509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f181c102d40 0x7f181c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 -- 172.21.15.5:0/2481524509 >> 172.21.15.5:0/2481524509 conn(0x7f181c0fc7a0 msgr2=0x7f181c0fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 -- 172.21.15.5:0/2481524509 shutdown_connections 2024-09-11T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:32.654+0000 7f1823bdf640 1 -- 172.21.15.5:0/2481524509 wait complete. 2024-09-11T22:25:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 -- 172.21.15.5:0/2492982093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa800101ac0 msgr2=0x7fa800101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 --2- 172.21.15.5:0/2492982093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa800101ac0 0x7fa800101f20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009930 tx=0x7fa7e802a420 comp rx=0 tx=0).stop 2024-09-11T22:25:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 -- 172.21.15.5:0/2492982093 shutdown_connections 2024-09-11T22:25:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 --2- 172.21.15.5:0/2492982093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102460 0x7fa80010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 --2- 172.21.15.5:0/2492982093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa800101ac0 0x7fa800101f20 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 --2- 172.21.15.5:0/2492982093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa800107ad0 0x7fa800107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 -- 172.21.15.5:0/2492982093 >> 172.21.15.5:0/2492982093 conn(0x7fa8000fd7e0 msgr2=0x7fa8000ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 -- 172.21.15.5:0/2492982093 shutdown_connections 2024-09-11T22:25:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 -- 172.21.15.5:0/2492982093 wait complete. 2024-09-11T22:25:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 Processor -- start 2024-09-11T22:25:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.799+0000 7fa80527b640 1 -- start start 2024-09-11T22:25:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa80527b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa800101ac0 0x7fa8000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa80527b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102460 0x7fa800074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa80527b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa800107ad0 0x7fa800074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa80527b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa800111ed0 con 0x7fa800101ac0 2024-09-11T22:25:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa80527b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa800111d50 con 0x7fa800102460 2024-09-11T22:25:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa80527b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa800112050 con 0x7fa800107ad0 2024-09-11T22:25:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa800101ac0 0x7fa8000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa800101ac0 0x7fa8000744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37012/0 (socket says 172.21.15.5:37012) 2024-09-11T22:25:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7feffd640 1 -- 172.21.15.5:0/587623374 learned_addr learned my addr 172.21.15.5:0/587623374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102460 0x7fa800074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7ff7fe640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa800107ad0 0x7fa800074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7feffd640 1 -- 172.21.15.5:0/587623374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa800107ad0 msgr2=0x7fa800074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7feffd640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa800107ad0 0x7fa800074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7feffd640 1 -- 172.21.15.5:0/587623374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102460 msgr2=0x7fa800074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7feffd640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102460 0x7fa800074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7feffd640 1 -- 172.21.15.5:0/587623374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8001a6860 con 0x7fa800101ac0 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7feffd640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa800101ac0 0x7fa8000744f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000d950 tx=0x7fa7f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7fcff9640 1 -- 172.21.15.5:0/587623374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0017070 con 0x7fa800101ac0 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7fcff9640 1 -- 172.21.15.5:0/587623374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f0009c30 con 0x7fa800101ac0 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa7fcff9640 1 -- 172.21.15.5:0/587623374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000f6b0 con 0x7fa800101ac0 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa80527b640 1 -- 172.21.15.5:0/587623374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8001a6b50 con 0x7fa800101ac0 2024-09-11T22:25:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.800+0000 7fa80527b640 1 -- 172.21.15.5:0/587623374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8001a70c0 con 0x7fa800101ac0 2024-09-11T22:25:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.803+0000 7fa7fcff9640 1 -- 172.21.15.5:0/587623374 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa7f00109d0 con 0x7fa800101ac0 2024-09-11T22:25:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.803+0000 7fa808019740 1 Processor -- start 2024-09-11T22:25:32.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.803+0000 7fa808019740 1 -- start start 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa808019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868caf0 0x558d1868ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa808019740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868d410 0x558d18697770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa808019740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 0x558d1869a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa808019740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558d185f12f0 con 0x558d1868d410 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa808019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558d18599270 con 0x558d1868caf0 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa808019740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558d185ed2b0 con 0x558d18697cb0 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868d410 0x558d18697770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868caf0 0x558d1868ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868caf0 0x558d1868ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57220/0 (socket says 172.21.15.5:57220) 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868d410 0x558d18697770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37024/0 (socket says 172.21.15.5:37024) 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7ff7fe640 1 -- 172.21.15.5:0/402822078 learned_addr learned my addr 172.21.15.5:0/402822078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:32.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/402822078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 0x558d1869a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7fcff9640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa7ec0a2e20 0x7fa7ec0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7f7fff640 1 -- 172.21.15.5:0/402822078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868caf0 msgr2=0x558d1868ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/402822078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868caf0 0x558d1868ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7f7fff640 1 -- 172.21.15.5:0/402822078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868d410 msgr2=0x558d18697770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/402822078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868d410 0x558d18697770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7f7fff640 1 -- 172.21.15.5:0/402822078 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d18598340 con 0x558d18697cb0 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7feffd640 1 --2- 172.21.15.5:0/402822078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868d410 0x558d18697770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7ff7fe640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa7ec0a2e20 0x7fa7ec0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7fcff9640 1 -- 172.21.15.5:0/587623374 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fa7f00cefb0 con 0x7fa800101ac0 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.804+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/402822078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 0x558d1869a0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8005940 tx=0x7fa7e8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d218c0a965e88e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.805+0000 7fa7ff7fe640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa7ec0a2e20 0x7fa7ec0a52e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80136a0 tx=0x7fa7f8004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.805+0000 7fa7f57fa640 1 -- 172.21.15.5:0/402822078 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8040070 con 0x558d18697cb0 2024-09-11T22:25:32.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.805+0000 7fa7f57fa640 1 -- 172.21.15.5:0/402822078 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e80318f0 con 0x558d18697cb0 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.805+0000 7fa7f57fa640 1 -- 172.21.15.5:0/402822078 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8035040 con 0x558d18697cb0 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.805+0000 7fa808019740 1 -- 172.21.15.5:0/402822078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 msgr2=0x558d1869a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.805+0000 7fa808019740 1 --2- 172.21.15.5:0/402822078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 0x558d1869a0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8005940 tx=0x7fa7e8005c00 comp rx=0 tx=0).stop 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 -- 172.21.15.5:0/402822078 shutdown_connections 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 --2- 172.21.15.5:0/402822078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 0x558d1869a0a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 --2- 172.21.15.5:0/402822078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868d410 0x558d18697770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 --2- 172.21.15.5:0/402822078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868caf0 0x558d1868ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 -- 172.21.15.5:0/402822078 >> 172.21.15.5:0/402822078 conn(0x558d18689060 msgr2=0x558d18689440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 -- 172.21.15.5:0/402822078 shutdown_connections 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 -- 172.21.15.5:0/402822078 wait complete. 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 -- start start 2024-09-11T22:25:32.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 Processor -- start 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d18694af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 0x558d18695030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 0x558d18695570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558d1869a880 con 0x558d1868caf0 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558d1857a290 con 0x558d1868d410 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558d1869aa00 con 0x558d18697cb0 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa808019740 10 client.0 _ll_register_callbacks cb 0x558d185ab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d18694af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d18694af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37040/0 (socket says 172.21.15.5:37040) 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa7f7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 0x558d18695030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa7feffd640 1 -- 172.21.15.5:0/598841895 learned_addr learned my addr 172.21.15.5:0/598841895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.806+0000 7fa7ff7fe640 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 0x558d18695570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7feffd640 1 -- 172.21.15.5:0/598841895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 msgr2=0x558d18695570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7feffd640 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 0x558d18695570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7feffd640 1 -- 172.21.15.5:0/598841895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 msgr2=0x558d18695030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7feffd640 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 0x558d18695030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7feffd640 1 -- 172.21.15.5:0/598841895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d18304210 con 0x558d1868caf0 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7feffd640 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d18694af0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0010fd0 tx=0x7fa7f000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c98c56dd132b78f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7e67fc640 1 -- 172.21.15.5:0/598841895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0020050 con 0x558d1868caf0 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7e67fc640 1 -- 172.21.15.5:0/598841895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fa7f0097b00 con 0x558d1868caf0 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa808019740 1 -- 172.21.15.5:0/598841895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e80318f0 con 0x558d1868caf0 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7e67fc640 1 -- 172.21.15.5:0/598841895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7f009adf0 con 0x558d1868caf0 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa808019740 10 client.16452 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa808019740 1 -- 172.21.15.5:0/598841895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d183d0620 con 0x558d1868caf0 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7e67fc640 1 -- 172.21.15.5:0/598841895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e80318f0 con 0x558d1868caf0 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.807+0000 7fa7e67fc640 1 -- 172.21.15.5:0/598841895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa7f00d10d0 con 0x558d1868caf0 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.808+0000 7fa808019740 10 client.16452 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.808+0000 7fa808019740 10 client.16452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.808+0000 7fa808019740 20 client.16452 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.808+0000 7fa808019740 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558d186a7c40 0x558d186aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.808+0000 7fa808019740 4 client.16452 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.808+0000 7fa808019740 1 -- 172.21.15.5:0/598841895 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d185fea20 con 0x558d186a7c40 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.808+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558d186a7c40 0x558d186aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.808+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558d186a7c40 0x558d186aa0d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1438851bc474247d server_cookie=763916fb634518b6 in_seq=0 out_seq=0 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.808+0000 7fa7e67fc640 10 client.16452 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa7e67fc640 1 -- 172.21.15.5:0/598841895 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d185fea20 con 0x558d186a7c40 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa7e67fc640 10 client.16452 handle_command_reply: tid=0 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 client.16452 shutdown 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 -- 172.21.15.5:0/598841895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 msgr2=0x558d18694af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d18694af0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0010fd0 tx=0x7fa7f000bf60 comp rx=0 tx=0).stop 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 -- 172.21.15.5:0/598841895 shutdown_connections 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558d186a7c40 0x558d186aa0d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d18697cb0 0x558d18695570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 0x558d18695030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa7e67fc640 0 client.16452 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 --2- 172.21.15.5:0/598841895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d18694af0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 -- 172.21.15.5:0/598841895 >> 172.21.15.5:0/598841895 conn(0x558d18689060 msgr2=0x558d1868edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 -- 172.21.15.5:0/598841895 shutdown_connections 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 1 -- 172.21.15.5:0/598841895 wait complete. 2024-09-11T22:25:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.809+0000 7fa808019740 20 client.16452 trim_cache size 0 max 16384 2024-09-11T22:25:32.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.825+0000 7fa808019740 1 Processor -- start 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.825+0000 7fa808019740 1 -- start start 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.825+0000 7fa808019740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868caf0 0x558d18696a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.825+0000 7fa808019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 0x558d18695020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.825+0000 7fa808019740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18697cb0 0x558d18695560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.825+0000 7fa808019740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001afb00 con 0x558d18697cb0 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.825+0000 7fa808019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001b1700 con 0x558d1868d410 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.825+0000 7fa808019740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001b01d0 con 0x558d1868caf0 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868caf0 0x558d18696a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7ff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18697cb0 0x558d18695560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7f7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 0x558d18695020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868caf0 0x558d18696a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59958/0 (socket says 172.21.15.5:59958) 2024-09-11T22:25:32.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7feffd640 1 -- 172.21.15.5:0/461518637 learned_addr learned my addr 172.21.15.5:0/461518637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7f7fff640 1 -- 172.21.15.5:0/461518637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868caf0 msgr2=0x558d18696a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/461518637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868caf0 0x558d18696a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7f7fff640 1 -- 172.21.15.5:0/461518637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18697cb0 msgr2=0x558d18695560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/461518637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18697cb0 0x558d18695560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7f7fff640 1 -- 172.21.15.5:0/461518637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d18688e90 con 0x558d1868d410 2024-09-11T22:25:32.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/461518637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 0x558d18695020 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa7e802a940 tx=0x7fa7e802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e9ab59723a575c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7f4ff9640 1 -- 172.21.15.5:0/461518637 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8054020 con 0x558d1868d410 2024-09-11T22:25:32.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7f4ff9640 1 -- 172.21.15.5:0/461518637 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e80035f0 con 0x558d1868d410 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.826+0000 7fa7f4ff9640 1 -- 172.21.15.5:0/461518637 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8040040 con 0x558d1868d410 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.827+0000 7fa808019740 1 -- 172.21.15.5:0/461518637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 msgr2=0x558d18695020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.827+0000 7fa808019740 1 --2- 172.21.15.5:0/461518637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 0x558d18695020 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa7e802a940 tx=0x7fa7e802a420 comp rx=0 tx=0).stop 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.827+0000 7fa808019740 1 -- 172.21.15.5:0/461518637 shutdown_connections 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.827+0000 7fa808019740 1 --2- 172.21.15.5:0/461518637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d18697cb0 0x558d18695560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.827+0000 7fa808019740 1 --2- 172.21.15.5:0/461518637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d1868d410 0x558d18695020 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.827+0000 7fa808019740 1 --2- 172.21.15.5:0/461518637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868caf0 0x558d18696a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.827+0000 7fa808019740 1 -- 172.21.15.5:0/461518637 >> 172.21.15.5:0/461518637 conn(0x558d18689060 msgr2=0x558d18696350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.827+0000 7fa808019740 1 -- 172.21.15.5:0/461518637 shutdown_connections 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.827+0000 7fa808019740 1 -- 172.21.15.5:0/461518637 wait complete. 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa808019740 1 -- start start 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa808019740 1 Processor -- start 2024-09-11T22:25:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa808019740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d186a0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa808019740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868d410 0x558d18691600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa808019740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18697cb0 0x558d18691b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa808019740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001b0f70 con 0x558d1868caf0 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa808019740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001b08a0 con 0x558d18697cb0 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa808019740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa8001adae0 con 0x558d1868d410 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa808019740 10 client.0 _ll_register_callbacks cb 0x7fa7f00d27f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d186a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa7feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d186a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37054/0 (socket says 172.21.15.5:37054) 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa7feffd640 1 -- 172.21.15.5:0/2357376642 learned_addr learned my addr 172.21.15.5:0/2357376642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa7f7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868d410 0x558d18691600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa7ff7fe640 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18697cb0 0x558d18691b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa7feffd640 1 -- 172.21.15.5:0/2357376642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868d410 msgr2=0x558d18691600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa7feffd640 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868d410 0x558d18691600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa7feffd640 1 -- 172.21.15.5:0/2357376642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18697cb0 msgr2=0x558d18691b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa7feffd640 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18697cb0 0x558d18691b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.828+0000 7fa7feffd640 1 -- 172.21.15.5:0/2357376642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d18692180 con 0x558d1868caf0 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa7feffd640 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d186a0220 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0097e60 tx=0x7fa7f0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71cecd17b8f1633 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa7e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa7e67fc640 1 -- 172.21.15.5:0/2357376642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0020040 con 0x558d1868caf0 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa808019740 1 -- 172.21.15.5:0/2357376642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e80035f0 con 0x558d1868caf0 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa7e67fc640 1 -- 172.21.15.5:0/2357376642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8585+0+0 (secure 0 0 0) 0x7fa7f00cc940 con 0x558d1868caf0 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa7e67fc640 1 -- 172.21.15.5:0/2357376642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa7f00b9050 con 0x558d1868caf0 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa808019740 10 client.16458 fetch_fsmap learned FSMap version 30 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa808019740 1 -- 172.21.15.5:0/2357376642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d1869eb80 con 0x558d1868caf0 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa7e67fc640 1 -- 172.21.15.5:0/2357376642 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e80035f0 con 0x558d1868caf0 2024-09-11T22:25:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa7e67fc640 1 -- 172.21.15.5:0/2357376642 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa7f009a8e0 con 0x558d1868caf0 2024-09-11T22:25:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa808019740 10 client.16458 fetch_fsmap finished waiting for FSMap version 30 2024-09-11T22:25:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa808019740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa808019740 20 client.16458 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa808019740 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558d18712f80 0x558d187153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa808019740 4 client.16458 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:25:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa808019740 1 -- 172.21.15.5:0/2357376642 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d185f8d70 con 0x558d18712f80 2024-09-11T22:25:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.829+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558d18712f80 0x558d187153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.830+0000 7fa7f7fff640 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558d18712f80 0x558d187153a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ddb3e17926a70 server_cookie=9abd9bcc418b18e2 in_seq=0 out_seq=0 2024-09-11T22:25:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.830+0000 7fa7e67fc640 10 client.16458 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa7e67fc640 1 -- 172.21.15.5:0/2357376642 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d185f8d70 con 0x558d18712f80 2024-09-11T22:25:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa7e67fc640 10 client.16458 handle_command_reply: tid=0 2024-09-11T22:25:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 client.16458 shutdown 2024-09-11T22:25:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 -- 172.21.15.5:0/2357376642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 msgr2=0x558d186a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d186a0220 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa7f0097e60 tx=0x7fa7f0010d70 comp rx=0 tx=0).stop 2024-09-11T22:25:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 -- 172.21.15.5:0/2357376642 shutdown_connections 2024-09-11T22:25:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558d18712f80 0x558d187153a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558d18697cb0 0x558d18691b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558d1868d410 0x558d18691600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 --2- 172.21.15.5:0/2357376642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d1868caf0 0x558d186a0220 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 -- 172.21.15.5:0/2357376642 >> 172.21.15.5:0/2357376642 conn(0x558d18689060 msgr2=0x558d1868ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa7e67fc640 0 client.16458 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 -- 172.21.15.5:0/2357376642 shutdown_connections 2024-09-11T22:25:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.893+0000 7fa808019740 1 -- 172.21.15.5:0/2357376642 wait complete. 2024-09-11T22:25:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa808019740 20 client.16458 trim_cache size 0 max 16384 2024-09-11T22:25:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:25:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:25:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:25:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:25:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa80527b640 1 -- 172.21.15.5:0/587623374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa7ec0a2e20 msgr2=0x7fa7ec0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa80527b640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa7ec0a2e20 0x7fa7ec0a52e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80136a0 tx=0x7fa7f8004a40 comp rx=0 tx=0).stop 2024-09-11T22:25:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa80527b640 1 -- 172.21.15.5:0/587623374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa800101ac0 msgr2=0x7fa8000744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa80527b640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa800101ac0 0x7fa8000744f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000d950 tx=0x7fa7f000de10 comp rx=0 tx=0).stop 2024-09-11T22:25:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa80527b640 1 -- 172.21.15.5:0/587623374 shutdown_connections 2024-09-11T22:25:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa80527b640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa800107ad0 0x7fa800074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa80527b640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa7ec0a2e20 0x7fa7ec0a52e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa80527b640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa800102460 0x7fa800074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa80527b640 1 --2- 172.21.15.5:0/587623374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa800101ac0 0x7fa8000744f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.894+0000 7fa80527b640 1 -- 172.21.15.5:0/587623374 >> 172.21.15.5:0/587623374 conn(0x7fa8000fd7e0 msgr2=0x7fa8000fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.895+0000 7fa80527b640 1 -- 172.21.15.5:0/587623374 shutdown_connections 2024-09-11T22:25:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:32.895+0000 7fa80527b640 1 -- 172.21.15.5:0/587623374 wait complete. 2024-09-11T22:25:33.310 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191803 9.49 MB/sec warmup 111 sec latency 53.670 ms 2024-09-11T22:25:33.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:33 smithi005 ceph-mon[30943]: pgmap v103: 97 pgs: 97 active+clean; 42 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-09-11T22:25:33.454 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:25:33.575 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:33 smithi159 ceph-mon[35534]: pgmap v103: 97 pgs: 97 active+clean; 42 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-09-11T22:25:33.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:33 smithi172 ceph-mon[35461]: pgmap v103: 97 pgs: 97 active+clean; 42 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 195 op/s 2024-09-11T22:25:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:33.829 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:25:33 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setuser ceph since I am not root 2024-09-11T22:25:33.931 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:25:33 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setgroup ceph since I am not root 2024-09-11T22:25:33.931 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:25:33 smithi159 ceph-mds[65034]: -- 172.21.15.159:0/810391770 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d88158f860 con 0x55d88156d000 2024-09-11T22:25:33.931 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:25:33 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: starting mds.k at 2024-09-11T22:25:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.233+0000 7fbabb00e640 1 -- 172.21.15.5:0/1752746261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab4106a80 msgr2=0x7fbab4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.233+0000 7fbabb00e640 1 --2- 172.21.15.5:0/1752746261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab4106a80 0x7fbab4106e60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009930 tx=0x7fbab002a420 comp rx=0 tx=0).stop 2024-09-11T22:25:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.233+0000 7fbabb00e640 1 -- 172.21.15.5:0/1752746261 shutdown_connections 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.233+0000 7fbabb00e640 1 --2- 172.21.15.5:0/1752746261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbab4101420 0x7fbab410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.233+0000 7fbabb00e640 1 --2- 172.21.15.5:0/1752746261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab4100a80 0x7fbab4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.233+0000 7fbabb00e640 1 --2- 172.21.15.5:0/1752746261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab4106a80 0x7fbab4106e60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.233+0000 7fbabb00e640 1 -- 172.21.15.5:0/1752746261 >> 172.21.15.5:0/1752746261 conn(0x7fbab40fc7a0 msgr2=0x7fbab40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.233+0000 7fbabb00e640 1 -- 172.21.15.5:0/1752746261 shutdown_connections 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.233+0000 7fbabb00e640 1 -- 172.21.15.5:0/1752746261 wait complete. 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbabb00e640 1 Processor -- start 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbabb00e640 1 -- start start 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbabb00e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab4100a80 0x7fbab4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbabb00e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab4101420 0x7fbab41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbabb00e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbab4106a80 0x7fbab419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbabb00e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbab4110e40 con 0x7fbab4100a80 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbabb00e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbab4110cc0 con 0x7fbab4106a80 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbabb00e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbab4110fc0 con 0x7fbab4101420 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbab8daa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab4100a80 0x7fbab4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbab8daa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab4100a80 0x7fbab4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37076/0 (socket says 172.21.15.5:37076) 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbab8daa640 1 -- 172.21.15.5:0/4786397 learned_addr learned my addr 172.21.15.5:0/4786397 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab4101420 0x7fbab41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.234+0000 7fbab95ab640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbab4106a80 0x7fbab419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbab95ab640 1 -- 172.21.15.5:0/4786397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab4101420 msgr2=0x7fbab41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbab95ab640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab4101420 0x7fbab41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbab95ab640 1 -- 172.21.15.5:0/4786397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab4100a80 msgr2=0x7fbab4198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbab95ab640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab4100a80 0x7fbab4198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbab95ab640 1 -- 172.21.15.5:0/4786397 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab419d430 con 0x7fbab4106a80 2024-09-11T22:25:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbab8daa640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab4100a80 0x7fbab4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbaabfff640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab4101420 0x7fbab41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbab95ab640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbab4106a80 0x7fbab419cd50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009fd0 tx=0x7fbaac00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbaa9ffb640 1 -- 172.21.15.5:0/4786397 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac00ce20 con 0x7fbab4106a80 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbabb00e640 1 -- 172.21.15.5:0/4786397 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab419d720 con 0x7fbab4106a80 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbaa9ffb640 1 -- 172.21.15.5:0/4786397 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaac005020 con 0x7fbab4106a80 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.235+0000 7fbaa9ffb640 1 -- 172.21.15.5:0/4786397 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac010720 con 0x7fbab4106a80 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.236+0000 7fbabb00e640 1 -- 172.21.15.5:0/4786397 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab41a5000 con 0x7fbab4106a80 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.239+0000 7fbaa9ffb640 1 -- 172.21.15.5:0/4786397 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbaac005540 con 0x7fbab4106a80 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.239+0000 7fbabd3e7740 1 Processor -- start 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.239+0000 7fbaa9ffb640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fba9c0a2cf0 0x7fba9c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbabd3e7740 1 -- start start 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbaabfff640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fba9c0a2cf0 0x7fba9c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbaa9ffb640 1 -- 172.21.15.5:0/4786397 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fbaac0cfb10 con 0x7fbab4106a80 2024-09-11T22:25:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40fb40 0x563e5e40e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40e5d0 0x563e5e40ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e419370 0x563e5e41b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e5e3742f0 con 0x563e5e419370 2024-09-11T22:25:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563e5e31bcf0 con 0x563e5e40fb40 2024-09-11T22:25:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563e5e3702b0 con 0x563e5e40e5d0 2024-09-11T22:25:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbab95ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e419370 0x563e5e41b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbab8daa640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40fb40 0x563e5e40e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbab95ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e419370 0x563e5e41b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37078/0 (socket says 172.21.15.5:37078) 2024-09-11T22:25:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbab8daa640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40fb40 0x563e5e40e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57268/0 (socket says 172.21.15.5:57268) 2024-09-11T22:25:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbab95ab640 1 -- 172.21.15.5:0/2736942458 learned_addr learned my addr 172.21.15.5:0/2736942458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbaabfff640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fba9c0a2cf0 0x7fba9c0a51b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbab4105290 tx=0x7fbaa4005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.240+0000 7fbaabfff640 1 --2- 172.21.15.5:0/2736942458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40e5d0 0x563e5e40ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.241+0000 7fbab8daa640 1 -- 172.21.15.5:0/2736942458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40e5d0 msgr2=0x563e5e40ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.241+0000 7fbab8daa640 1 --2- 172.21.15.5:0/2736942458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40e5d0 0x563e5e40ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.241+0000 7fbab8daa640 1 -- 172.21.15.5:0/2736942458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e419370 msgr2=0x563e5e41b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.241+0000 7fbab8daa640 1 --2- 172.21.15.5:0/2736942458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e419370 0x563e5e41b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.241+0000 7fbab8daa640 1 -- 172.21.15.5:0/2736942458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5e3003a0 con 0x563e5e40fb40 2024-09-11T22:25:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.241+0000 7fbab95ab640 1 --2- 172.21.15.5:0/2736942458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e419370 0x563e5e41b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.241+0000 7fbab8daa640 1 --2- 172.21.15.5:0/2736942458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40fb40 0x563e5e40e090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009900 tx=0x7fbab0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5d94bc6a2993230 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.241+0000 7fba9affd640 1 -- 172.21.15.5:0/2736942458 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0035040 con 0x563e5e40fb40 2024-09-11T22:25:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.241+0000 7fba9affd640 1 -- 172.21.15.5:0/2736942458 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbab0031830 con 0x563e5e40fb40 2024-09-11T22:25:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.241+0000 7fba9affd640 1 -- 172.21.15.5:0/2736942458 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab00047a0 con 0x563e5e40fb40 2024-09-11T22:25:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2736942458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40fb40 msgr2=0x563e5e40e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2736942458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40fb40 0x563e5e40e090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009900 tx=0x7fbab0005dc0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2736942458 shutdown_connections 2024-09-11T22:25:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2736942458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e419370 0x563e5e41b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2736942458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40e5d0 0x563e5e40ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2736942458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40fb40 0x563e5e40e090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2736942458 >> 172.21.15.5:0/2736942458 conn(0x563e5e3d7310 msgr2=0x563e5e40ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2736942458 shutdown_connections 2024-09-11T22:25:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2736942458 wait complete. 2024-09-11T22:25:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 -- start start 2024-09-11T22:25:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.242+0000 7fbabd3e7740 1 Processor -- start 2024-09-11T22:25:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40e5d0 0x563e5e418ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e415f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e419370 0x563e5e416490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e5e41c010 con 0x563e5e40fb40 2024-09-11T22:25:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563e5e32f060 con 0x563e5e419370 2024-09-11T22:25:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563e5e41c190 con 0x563e5e40e5d0 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbabd3e7740 10 client.0 _ll_register_callbacks cb 0x563e5e0828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e415f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e415f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37094/0 (socket says 172.21.15.5:37094) 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbaabfff640 1 -- 172.21.15.5:0/3257988119 learned_addr learned my addr 172.21.15.5:0/3257988119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbab8daa640 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40e5d0 0x563e5e418ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbab95ab640 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e419370 0x563e5e416490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbaabfff640 1 -- 172.21.15.5:0/3257988119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40e5d0 msgr2=0x563e5e418ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbaabfff640 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40e5d0 0x563e5e418ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbaabfff640 1 -- 172.21.15.5:0/3257988119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e419370 msgr2=0x563e5e416490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbaabfff640 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e419370 0x563e5e416490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbaabfff640 1 -- 172.21.15.5:0/3257988119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5e0e53c0 con 0x563e5e40fb40 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.243+0000 7fbab95ab640 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e419370 0x563e5e416490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:25:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fbaabfff640 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e415f50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40037e0 tx=0x7fbaa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c6030a66cbeeaf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fba93fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fba93fff640 1 -- 172.21.15.5:0/3257988119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4003e20 con 0x563e5e40fb40 2024-09-11T22:25:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fbabd3e7740 1 -- 172.21.15.5:0/3257988119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbab0031830 con 0x563e5e40fb40 2024-09-11T22:25:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fba93fff640 1 -- 172.21.15.5:0/3257988119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fbaa400fb90 con 0x563e5e40fb40 2024-09-11T22:25:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fba93fff640 1 -- 172.21.15.5:0/3257988119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbaa400fd90 con 0x563e5e40fb40 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fbabd3e7740 10 client.16476 fetch_fsmap learned FSMap version 32 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fbabd3e7740 1 -- 172.21.15.5:0/3257988119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e5e427ab0 con 0x563e5e40fb40 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fba93fff640 1 -- 172.21.15.5:0/3257988119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbab0031830 con 0x563e5e40fb40 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fba93fff640 1 -- 172.21.15.5:0/3257988119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fbaa402d030 con 0x563e5e40fb40 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fbabd3e7740 10 client.16476 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fbabd3e7740 10 client.16476 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fbabd3e7740 20 client.16476 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x563e5e428f10 0x563e5e42b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.244+0000 7fbabd3e7740 4 client.16476 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.245+0000 7fbabd3e7740 1 -- 172.21.15.5:0/3257988119 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e5e381860 con 0x563e5e428f10 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.245+0000 7fbab8daa640 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x563e5e428f10 0x563e5e42b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.245+0000 7fbab8daa640 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x563e5e428f10 0x563e5e42b360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24a3feca82e4dd server_cookie=f6fd2136ed2e8e62 in_seq=0 out_seq=0 2024-09-11T22:25:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.245+0000 7fba93fff640 10 client.16476 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fba93fff640 1 -- 172.21.15.5:0/3257988119 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e5e381860 con 0x563e5e428f10 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fba93fff640 10 client.16476 handle_command_reply: tid=0 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 client.16476 shutdown 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 -- 172.21.15.5:0/3257988119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 msgr2=0x563e5e415f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e415f50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbaa40037e0 tx=0x7fbaa4004730 comp rx=0 tx=0).stop 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 -- 172.21.15.5:0/3257988119 shutdown_connections 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x563e5e428f10 0x563e5e42b360 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e419370 0x563e5e416490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e415f50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/3257988119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40e5d0 0x563e5e418ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fba93fff640 0 client.16476 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 -- 172.21.15.5:0/3257988119 >> 172.21.15.5:0/3257988119 conn(0x563e5e3d7310 msgr2=0x563e5e410960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 -- 172.21.15.5:0/3257988119 shutdown_connections 2024-09-11T22:25:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 1 -- 172.21.15.5:0/3257988119 wait complete. 2024-09-11T22:25:34.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.246+0000 7fbabd3e7740 20 client.16476 trim_cache size 0 max 16384 2024-09-11T22:25:34.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbabd3e7740 1 Processor -- start 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbabd3e7740 1 -- start start 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e3d7310 0x563e5e417ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40e5d0 0x563e5e411120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e411660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41ada60 con 0x563e5e40fb40 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41af660 con 0x563e5e40e5d0 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41ae130 con 0x563e5e3d7310 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbab95ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e411660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbab95ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e411660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37100/0 (socket says 172.21.15.5:37100) 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40e5d0 0x563e5e411120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbab95ab640 1 -- 172.21.15.5:0/2357441609 learned_addr learned my addr 172.21.15.5:0/2357441609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbab8daa640 1 --2- 172.21.15.5:0/2357441609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e3d7310 0x563e5e417ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbab95ab640 1 -- 172.21.15.5:0/2357441609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e3d7310 msgr2=0x563e5e417ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbab95ab640 1 --2- 172.21.15.5:0/2357441609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e3d7310 0x563e5e417ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbab95ab640 1 -- 172.21.15.5:0/2357441609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40e5d0 msgr2=0x563e5e411120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbab95ab640 1 --2- 172.21.15.5:0/2357441609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40e5d0 0x563e5e411120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.260+0000 7fbab95ab640 1 -- 172.21.15.5:0/2357441609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5e4144a0 con 0x563e5e40fb40 2024-09-11T22:25:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.261+0000 7fbab95ab640 1 --2- 172.21.15.5:0/2357441609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e411660 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0cd2d0 tx=0x7fbaac0bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea3640d376e97f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.261+0000 7fba9a7fc640 1 -- 172.21.15.5:0/2357441609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0b4030 con 0x563e5e40fb40 2024-09-11T22:25:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.261+0000 7fba9a7fc640 1 -- 172.21.15.5:0/2357441609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaac0051d0 con 0x563e5e40fb40 2024-09-11T22:25:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.261+0000 7fba9a7fc640 1 -- 172.21.15.5:0/2357441609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac09c4b0 con 0x563e5e40fb40 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.261+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2357441609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 msgr2=0x563e5e411660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.261+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2357441609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e411660 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0cd2d0 tx=0x7fbaac0bc040 comp rx=0 tx=0).stop 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.262+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2357441609 shutdown_connections 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.262+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2357441609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40fb40 0x563e5e411660 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.262+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2357441609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e40e5d0 0x563e5e411120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.262+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2357441609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e3d7310 0x563e5e417ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.262+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2357441609 >> 172.21.15.5:0/2357441609 conn(0x563e5e4165f0 msgr2=0x563e5e4177b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.262+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2357441609 shutdown_connections 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.262+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2357441609 wait complete. 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.262+0000 7fbabd3e7740 1 -- start start 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.262+0000 7fbabd3e7740 1 Processor -- start 2024-09-11T22:25:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.262+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e3d7310 0x563e5e421420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40e5d0 0x563e5e421960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbabd3e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40fb40 0x563e5e41e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41aef90 con 0x563e5e40e5d0 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41ae800 con 0x563e5e3d7310 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbabd3e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbab41aba40 con 0x563e5e40fb40 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbabd3e7740 10 client.0 _ll_register_callbacks cb 0x7fbaa4004110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbab8daa640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e3d7310 0x563e5e421420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40e5d0 0x563e5e421960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40e5d0 0x563e5e421960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37108/0 (socket says 172.21.15.5:37108) 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbab8daa640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e3d7310 0x563e5e421420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57286/0 (socket says 172.21.15.5:57286) 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbaabfff640 1 -- 172.21.15.5:0/2031407945 learned_addr learned my addr 172.21.15.5:0/2031407945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbab95ab640 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40fb40 0x563e5e41e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbaabfff640 1 -- 172.21.15.5:0/2031407945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40fb40 msgr2=0x563e5e41e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbaabfff640 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40fb40 0x563e5e41e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbaabfff640 1 -- 172.21.15.5:0/2031407945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e3d7310 msgr2=0x563e5e421420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbaabfff640 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e3d7310 0x563e5e421420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbaabfff640 1 -- 172.21.15.5:0/2031407945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5e41e930 con 0x563e5e40e5d0 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbaabfff640 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40e5d0 0x563e5e421960 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbaa401d660 tx=0x7fbaa401dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456453b965d92c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fba93fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fba93fff640 1 -- 172.21.15.5:0/2031407945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4011d70 con 0x563e5e40e5d0 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fba93fff640 1 -- 172.21.15.5:0/2031407945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fbaa4023040 con 0x563e5e40e5d0 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.263+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2031407945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaac0051d0 con 0x563e5e40e5d0 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fba93fff640 1 -- 172.21.15.5:0/2031407945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbaa403f050 con 0x563e5e40e5d0 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fbabd3e7740 10 client.16482 fetch_fsmap learned FSMap version 32 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2031407945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563e5e4942a0 con 0x563e5e40e5d0 2024-09-11T22:25:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fba93fff640 1 -- 172.21.15.5:0/2031407945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaac0051d0 con 0x563e5e40e5d0 2024-09-11T22:25:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fba93fff640 1 -- 172.21.15.5:0/2031407945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fbaa4036070 con 0x563e5e40e5d0 2024-09-11T22:25:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fbabd3e7740 10 client.16482 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T22:25:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fbabd3e7740 10 client.16482 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:25:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fbabd3e7740 20 client.16482 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x563e5e419370 0x563e5e496a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fbabd3e7740 4 client.16482 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:25:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.264+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2031407945 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e5e37bd70 con 0x563e5e419370 2024-09-11T22:25:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.265+0000 7fbab8daa640 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x563e5e419370 0x563e5e496a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.265+0000 7fbab8daa640 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x563e5e419370 0x563e5e496a90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76138f1a23e36ce0 server_cookie=5bc596a0a3335246 in_seq=0 out_seq=0 2024-09-11T22:25:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.265+0000 7fba93fff640 10 client.16482 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:25:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.265+0000 7fba93fff640 1 -- 172.21.15.5:0/2031407945 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e5e37bd70 con 0x563e5e419370 2024-09-11T22:25:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.265+0000 7fba93fff640 10 client.16482 handle_command_reply: tid=0 2024-09-11T22:25:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.265+0000 7fbabd3e7740 1 client.16482 shutdown 2024-09-11T22:25:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.265+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2031407945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40e5d0 msgr2=0x563e5e421960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.265+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40e5d0 0x563e5e421960 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbaa401d660 tx=0x7fbaa401dd60 comp rx=0 tx=0).stop 2024-09-11T22:25:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2031407945 shutdown_connections 2024-09-11T22:25:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x563e5e419370 0x563e5e496a90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fba93fff640 0 client.16482 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:25:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e5e40fb40 0x563e5e41e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5e40e5d0 0x563e5e421960 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabd3e7740 1 --2- 172.21.15.5:0/2031407945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5e3d7310 0x563e5e421420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2031407945 >> 172.21.15.5:0/2031407945 conn(0x563e5e4165f0 msgr2=0x563e5e417fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2031407945 shutdown_connections 2024-09-11T22:25:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabd3e7740 1 -- 172.21.15.5:0/2031407945 wait complete. 2024-09-11T22:25:34.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabd3e7740 20 client.16482 trim_cache size 0 max 16384 2024-09-11T22:25:34.273 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:25:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabb00e640 1 -- 172.21.15.5:0/4786397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fba9c0a2cf0 msgr2=0x7fba9c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabb00e640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fba9c0a2cf0 0x7fba9c0a51b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbab4105290 tx=0x7fbaa4005f90 comp rx=0 tx=0).stop 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabb00e640 1 -- 172.21.15.5:0/4786397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbab4106a80 msgr2=0x7fbab419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabb00e640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbab4106a80 0x7fbab419cd50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009fd0 tx=0x7fbaac00ef10 comp rx=0 tx=0).stop 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabb00e640 1 -- 172.21.15.5:0/4786397 shutdown_connections 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.266+0000 7fbabb00e640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fba9c0a2cf0 0x7fba9c0a51b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.267+0000 7fbabb00e640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbab4106a80 0x7fbab419cd50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.267+0000 7fbabb00e640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbab4101420 0x7fbab41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.267+0000 7fbabb00e640 1 --2- 172.21.15.5:0/4786397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbab4100a80 0x7fbab4198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.267+0000 7fbabb00e640 1 -- 172.21.15.5:0/4786397 >> 172.21.15.5:0/4786397 conn(0x7fbab40fc7a0 msgr2=0x7fbab406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.267+0000 7fbabb00e640 1 -- 172.21.15.5:0/4786397 shutdown_connections 2024-09-11T22:25:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:34.267+0000 7fbabb00e640 1 -- 172.21.15.5:0/4786397 wait complete. 2024-09-11T22:25:34.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 194321 9.51 MB/sec warmup 112 sec latency 18.931 ms 2024-09-11T22:25:34.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:34 smithi005 ceph-mon[30943]: daemon mds.k restarted 2024-09-11T22:25:34.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:34 smithi159 ceph-mon[35534]: daemon mds.k restarted 2024-09-11T22:25:34.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:34 smithi172 ceph-mon[35461]: daemon mds.k restarted 2024-09-11T22:25:35.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:25:35.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 196204 9.51 MB/sec warmup 113 sec latency 60.241 ms 2024-09-11T22:25:35.407 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:35.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:35 smithi005 ceph-mon[30943]: pgmap v104: 97 pgs: 97 active+clean; 42 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 144 op/s 2024-09-11T22:25:35.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:35 smithi005 ceph-mon[30943]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T22:25:35.531 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:35 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6832/4169170564,v1:172.21.15.159:6833/4169170564] up:boot 2024-09-11T22:25:35.531 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:35 smithi005 ceph-mon[30943]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:25:35.531 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:35 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:25:35.531 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:35 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:25:35.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:35 smithi159 ceph-mon[35534]: pgmap v104: 97 pgs: 97 active+clean; 42 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 144 op/s 2024-09-11T22:25:35.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:35 smithi159 ceph-mon[35534]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T22:25:35.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:35 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6832/4169170564,v1:172.21.15.159:6833/4169170564] up:boot 2024-09-11T22:25:35.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:35 smithi159 ceph-mon[35534]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:25:35.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:35 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:25:35.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:35 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:25:35.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:35 smithi172 ceph-mon[35461]: pgmap v104: 97 pgs: 97 active+clean; 42 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 144 op/s 2024-09-11T22:25:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:35 smithi172 ceph-mon[35461]: osdmap e86: 12 total, 12 up, 12 in 2024-09-11T22:25:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:35 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6832/4169170564,v1:172.21.15.159:6833/4169170564] up:boot 2024-09-11T22:25:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:35 smithi172 ceph-mon[35461]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:25:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:35 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:25:35.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:35 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:25:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 -- 172.21.15.5:0/2262562590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7581049a0 msgr2=0x7fc7581089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:36.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 --2- 172.21.15.5:0/2262562590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7581049a0 0x7fc7581089f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc748009900 tx=0x7fc74802a410 comp rx=0 tx=0).stop 2024-09-11T22:25:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 -- 172.21.15.5:0/2262562590 shutdown_connections 2024-09-11T22:25:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 --2- 172.21.15.5:0/2262562590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc758109120 0x7fc758110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 --2- 172.21.15.5:0/2262562590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7581049a0 0x7fc7581089f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 --2- 172.21.15.5:0/2262562590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc758103ff0 0x7fc7581043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 -- 172.21.15.5:0/2262562590 >> 172.21.15.5:0/2262562590 conn(0x7fc7580ffab0 msgr2=0x7fc758101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 -- 172.21.15.5:0/2262562590 shutdown_connections 2024-09-11T22:25:36.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 -- 172.21.15.5:0/2262562590 wait complete. 2024-09-11T22:25:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 Processor -- start 2024-09-11T22:25:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.261+0000 7fc75fc19640 1 -- start start 2024-09-11T22:25:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75fc19640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc758103ff0 0x7fc75819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75fc19640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7581049a0 0x7fc75819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75fc19640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc758109120 0x7fc75819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75fc19640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc758114170 con 0x7fc758103ff0 2024-09-11T22:25:36.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75fc19640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc758113ff0 con 0x7fc7581049a0 2024-09-11T22:25:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75fc19640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581142f0 con 0x7fc758109120 2024-09-11T22:25:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75d9b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc758103ff0 0x7fc75819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75d9b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc758103ff0 0x7fc75819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37132/0 (socket says 172.21.15.5:37132) 2024-09-11T22:25:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75d9b5640 1 -- 172.21.15.5:0/2630462078 learned_addr learned my addr 172.21.15.5:0/2630462078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75e1b6640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc758109120 0x7fc75819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7581049a0 0x7fc75819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75d9b5640 1 -- 172.21.15.5:0/2630462078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc758109120 msgr2=0x7fc75819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75d9b5640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc758109120 0x7fc75819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75d9b5640 1 -- 172.21.15.5:0/2630462078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7581049a0 msgr2=0x7fc75819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75d9b5640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7581049a0 0x7fc75819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75d9b5640 1 -- 172.21.15.5:0/2630462078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75819f5e0 con 0x7fc758103ff0 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.262+0000 7fc75d9b5640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc758103ff0 0x7fc75819a630 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d950 tx=0x7fc75400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.263+0000 7fc74effd640 1 -- 172.21.15.5:0/2630462078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754017070 con 0x7fc758103ff0 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.263+0000 7fc74effd640 1 -- 172.21.15.5:0/2630462078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc754009c30 con 0x7fc758103ff0 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.263+0000 7fc75fc19640 1 -- 172.21.15.5:0/2630462078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75819f8d0 con 0x7fc758103ff0 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.263+0000 7fc74effd640 1 -- 172.21.15.5:0/2630462078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75400f6b0 con 0x7fc758103ff0 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.263+0000 7fc75fc19640 1 -- 172.21.15.5:0/2630462078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7581a71b0 con 0x7fc758103ff0 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.265+0000 7fc74effd640 1 -- 172.21.15.5:0/2630462078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc7540109d0 con 0x7fc758103ff0 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.265+0000 7fc761ff2740 1 Processor -- start 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc761ff2740 1 -- start start 2024-09-11T22:25:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b914090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b9145d0 0x56418b914a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b91f370 0x56418b921760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc761ff2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56418b87a2f0 con 0x56418b9145d0 2024-09-11T22:25:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc761ff2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56418b821cf0 con 0x56418b915b40 2024-09-11T22:25:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc761ff2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56418b8762b0 con 0x56418b91f370 2024-09-11T22:25:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc75d9b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b9145d0 0x56418b914a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc75e1b6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b914090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc75d1b4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b91f370 0x56418b921760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc75e1b6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b914090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57292/0 (socket says 172.21.15.5:57292) 2024-09-11T22:25:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc75e1b6640 1 -- 172.21.15.5:0/1153286973 learned_addr learned my addr 172.21.15.5:0/1153286973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc75e1b6640 1 -- 172.21.15.5:0/1153286973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b91f370 msgr2=0x56418b921760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc75e1b6640 1 --2- 172.21.15.5:0/1153286973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b91f370 0x56418b921760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc75e1b6640 1 -- 172.21.15.5:0/1153286973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b9145d0 msgr2=0x56418b914a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc75e1b6640 1 --2- 172.21.15.5:0/1153286973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b9145d0 0x56418b914a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.266+0000 7fc74effd640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7440a2cf0 0x7fc7440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.267+0000 7fc75e1b6640 1 -- 172.21.15.5:0/1153286973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418b8063a0 con 0x56418b915b40 2024-09-11T22:25:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.267+0000 7fc75e1b6640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7440a2cf0 0x7fc7440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.267+0000 7fc75e1b6640 1 --2- 172.21.15.5:0/1153286973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b914090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc750009f70 tx=0x7fc750010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f310444995cf9c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.267+0000 7fc74effd640 1 -- 172.21.15.5:0/2630462078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fc7540cf070 con 0x7fc758103ff0 2024-09-11T22:25:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.267+0000 7fc743fff640 1 -- 172.21.15.5:0/1153286973 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750022070 con 0x56418b915b40 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.267+0000 7fc743fff640 1 -- 172.21.15.5:0/1153286973 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7500113e0 con 0x56418b915b40 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.267+0000 7fc743fff640 1 -- 172.21.15.5:0/1153286973 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750016070 con 0x56418b915b40 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.268+0000 7fc75e1b6640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7440a2cf0 0x7fc7440a51b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc75001f000 tx=0x7fc750022000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.268+0000 7fc761ff2740 1 -- 172.21.15.5:0/1153286973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 msgr2=0x56418b914090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.268+0000 7fc761ff2740 1 --2- 172.21.15.5:0/1153286973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b914090 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc750009f70 tx=0x7fc750010800 comp rx=0 tx=0).stop 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.268+0000 7fc761ff2740 1 -- 172.21.15.5:0/1153286973 shutdown_connections 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.268+0000 7fc761ff2740 1 --2- 172.21.15.5:0/1153286973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b91f370 0x56418b921760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.268+0000 7fc761ff2740 1 --2- 172.21.15.5:0/1153286973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b9145d0 0x56418b914a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.268+0000 7fc761ff2740 1 --2- 172.21.15.5:0/1153286973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b914090 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.268+0000 7fc761ff2740 1 -- 172.21.15.5:0/1153286973 >> 172.21.15.5:0/1153286973 conn(0x56418b8dd310 msgr2=0x56418b910c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.268+0000 7fc761ff2740 1 -- 172.21.15.5:0/1153286973 shutdown_connections 2024-09-11T22:25:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.268+0000 7fc761ff2740 1 -- 172.21.15.5:0/1153286973 wait complete. 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc761ff2740 1 -- start start 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc761ff2740 1 Processor -- start 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b91ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b915b40 0x56418b91bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b91f370 0x56418b91c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc761ff2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56418b922070 con 0x56418b915b40 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc761ff2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56418b835060 con 0x56418b91f370 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc761ff2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56418b9221f0 con 0x56418b9145d0 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc761ff2740 10 client.0 _ll_register_callbacks cb 0x56418b5888c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc75d1b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b915b40 0x56418b91bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc75d1b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b915b40 0x56418b91bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37158/0 (socket says 172.21.15.5:37158) 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc75d1b4640 1 -- 172.21.15.5:0/35544485 learned_addr learned my addr 172.21.15.5:0/35544485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc75d9b5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b91ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.269+0000 7fc75e1b6640 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b91f370 0x56418b91c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc75d1b4640 1 -- 172.21.15.5:0/35544485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 msgr2=0x56418b91ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b91ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc75d1b4640 1 -- 172.21.15.5:0/35544485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b91f370 msgr2=0x56418b91c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b91f370 0x56418b91c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc75d1b4640 1 -- 172.21.15.5:0/35544485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418b5eb3c0 con 0x56418b915b40 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc75d9b5640 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b91ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b915b40 0x56418b91bfe0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc748009900 tx=0x7fc748004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcbac99623634e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc7417fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc7417fa640 1 -- 172.21.15.5:0/35544485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748035040 con 0x56418b915b40 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc7417fa640 1 -- 172.21.15.5:0/35544485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fc7480054d0 con 0x56418b915b40 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc761ff2740 1 -- 172.21.15.5:0/35544485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7500113e0 con 0x56418b915b40 2024-09-11T22:25:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc7417fa640 1 -- 172.21.15.5:0/35544485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7480046c0 con 0x56418b915b40 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc761ff2740 10 client.16494 fetch_fsmap learned FSMap version 32 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.270+0000 7fc761ff2740 1 -- 172.21.15.5:0/35544485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56418b92db20 con 0x56418b915b40 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc7417fa640 1 -- 172.21.15.5:0/35544485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7500113e0 con 0x56418b915b40 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc7417fa640 1 -- 172.21.15.5:0/35544485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fc748003c60 con 0x56418b915b40 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc761ff2740 10 client.16494 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc761ff2740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc761ff2740 20 client.16494 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc761ff2740 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56418b92f030 0x56418b931440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc761ff2740 4 client.16494 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc761ff2740 1 -- 172.21.15.5:0/35544485 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56418b887860 con 0x56418b92f030 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc75d9b5640 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56418b92f030 0x56418b931440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc7417fa640 10 client.16494 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.271+0000 7fc75d9b5640 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56418b92f030 0x56418b931440 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a759042c6ce0b336 server_cookie=43eba4d2b776a89d in_seq=0 out_seq=0 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc7417fa640 1 -- 172.21.15.5:0/35544485 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56418b887860 con 0x56418b92f030 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc7417fa640 10 client.16494 handle_command_reply: tid=0 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 client.16494 shutdown 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 -- 172.21.15.5:0/35544485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b915b40 msgr2=0x56418b91bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b915b40 0x56418b91bfe0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc748009900 tx=0x7fc748004580 comp rx=0 tx=0).stop 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 -- 172.21.15.5:0/35544485 shutdown_connections 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56418b92f030 0x56418b931440 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b91f370 0x56418b91c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b915b40 0x56418b91bfe0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 --2- 172.21.15.5:0/35544485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b91ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 -- 172.21.15.5:0/35544485 >> 172.21.15.5:0/35544485 conn(0x56418b8dd310 msgr2=0x56418b916960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc7417fa640 0 client.16494 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 -- 172.21.15.5:0/35544485 shutdown_connections 2024-09-11T22:25:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 1 -- 172.21.15.5:0/35544485 wait complete. 2024-09-11T22:25:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.274+0000 7fc761ff2740 20 client.16494 trim_cache size 0 max 16384 2024-09-11T22:25:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc761ff2740 1 Processor -- start 2024-09-11T22:25:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc761ff2740 1 -- start start 2024-09-11T22:25:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b92b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b91b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 0x56418b936d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc761ff2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581afd10 con 0x56418b91f370 2024-09-11T22:25:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc761ff2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581b1910 con 0x56418b915b40 2024-09-11T22:25:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc761ff2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581b03e0 con 0x56418b9145d0 2024-09-11T22:25:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc75d1b4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b91b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc75e1b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 0x56418b936d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc75d1b4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b91b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57312/0 (socket says 172.21.15.5:57312) 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc75d1b4640 1 -- 172.21.15.5:0/515897796 learned_addr learned my addr 172.21.15.5:0/515897796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.289+0000 7fc75d9b5640 1 --2- 172.21.15.5:0/515897796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b92b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.290+0000 7fc75d1b4640 1 -- 172.21.15.5:0/515897796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 msgr2=0x56418b92b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.290+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/515897796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b92b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.290+0000 7fc75d1b4640 1 -- 172.21.15.5:0/515897796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 msgr2=0x56418b936d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.290+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/515897796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 0x56418b936d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.290+0000 7fc75d1b4640 1 -- 172.21.15.5:0/515897796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418b91a530 con 0x56418b915b40 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.290+0000 7fc75e1b6640 1 --2- 172.21.15.5:0/515897796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 0x56418b936d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.290+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/515897796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b91b8a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc74802aa00 tx=0x7fc748031970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3636e730442d606e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.290+0000 7fc7437fe640 1 -- 172.21.15.5:0/515897796 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748035030 con 0x56418b915b40 2024-09-11T22:25:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.290+0000 7fc7437fe640 1 -- 172.21.15.5:0/515897796 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc748041070 con 0x56418b915b40 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.290+0000 7fc7437fe640 1 -- 172.21.15.5:0/515897796 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748004be0 con 0x56418b915b40 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 -- 172.21.15.5:0/515897796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 msgr2=0x56418b91b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 --2- 172.21.15.5:0/515897796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b91b8a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc74802aa00 tx=0x7fc748031970 comp rx=0 tx=0).stop 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 -- 172.21.15.5:0/515897796 shutdown_connections 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 --2- 172.21.15.5:0/515897796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 0x56418b936d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 --2- 172.21.15.5:0/515897796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b91b8a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 --2- 172.21.15.5:0/515897796 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b92b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 -- 172.21.15.5:0/515897796 >> 172.21.15.5:0/515897796 conn(0x56418b8dd310 msgr2=0x56418b91a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 -- 172.21.15.5:0/515897796 shutdown_connections 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 -- 172.21.15.5:0/515897796 wait complete. 2024-09-11T22:25:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 -- start start 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.291+0000 7fc761ff2740 1 Processor -- start 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b927b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b922e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc761ff2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 0x56418b9233d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc761ff2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581b1240 con 0x56418b91f370 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc761ff2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581b0ab0 con 0x56418b915b40 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc761ff2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc7581adcf0 con 0x56418b9145d0 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc761ff2740 10 client.0 _ll_register_callbacks cb 0x7fc748009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75e1b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 0x56418b9233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75d1b4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b922e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75e1b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 0x56418b9233d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37180/0 (socket says 172.21.15.5:37180) 2024-09-11T22:25:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75e1b6640 1 -- 172.21.15.5:0/1166981998 learned_addr learned my addr 172.21.15.5:0/1166981998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b922e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57318/0 (socket says 172.21.15.5:57318) 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75d9b5640 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b927b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75d1b4640 1 -- 172.21.15.5:0/1166981998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 msgr2=0x56418b927b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b927b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75d1b4640 1 -- 172.21.15.5:0/1166981998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 msgr2=0x56418b9233d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 0x56418b9233d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.292+0000 7fc75d1b4640 1 -- 172.21.15.5:0/1166981998 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418b923c00 con 0x56418b915b40 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.293+0000 7fc75d1b4640 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b922e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56418b91cae0 tx=0x7fc748005580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cccaf7a50f72a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.293+0000 7fc7417fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.293+0000 7fc7417fa640 1 -- 172.21.15.5:0/1166981998 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748035030 con 0x56418b915b40 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.293+0000 7fc7417fa640 1 -- 172.21.15.5:0/1166981998 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fc748005770 con 0x56418b915b40 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.293+0000 7fc761ff2740 1 -- 172.21.15.5:0/1166981998 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc748041070 con 0x56418b915b40 2024-09-11T22:25:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.293+0000 7fc7417fa640 1 -- 172.21.15.5:0/1166981998 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc748067050 con 0x56418b915b40 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.293+0000 7fc761ff2740 10 client.26006 fetch_fsmap learned FSMap version 32 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.293+0000 7fc761ff2740 1 -- 172.21.15.5:0/1166981998 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56418b999cb0 con 0x56418b915b40 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.293+0000 7fc7417fa640 1 -- 172.21.15.5:0/1166981998 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc748031740 con 0x56418b915b40 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.293+0000 7fc7417fa640 1 -- 172.21.15.5:0/1166981998 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8378+0+0 (secure 0 0 0) 0x7fc748060060 con 0x56418b915b40 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc761ff2740 10 client.26006 fetch_fsmap finished waiting for FSMap version 32 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc761ff2740 10 client.26006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc761ff2740 20 client.26006 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc761ff2740 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56418b99a400 0x56418b99c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc761ff2740 4 client.26006 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc761ff2740 1 -- 172.21.15.5:0/1166981998 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc748041070 con 0x56418b99a400 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc75d9b5640 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56418b99a400 0x56418b99c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc7417fa640 10 client.26006 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc75d9b5640 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56418b99a400 0x56418b99c820 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac83fc436876d35 server_cookie=dcb730570ab3f109 in_seq=0 out_seq=0 2024-09-11T22:25:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc7417fa640 1 -- 172.21.15.5:0/1166981998 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc748041070 con 0x56418b99a400 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc7417fa640 10 client.26006 handle_command_reply: tid=0 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc761ff2740 1 client.26006 shutdown 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc761ff2740 1 -- 172.21.15.5:0/1166981998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 msgr2=0x56418b922e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.294+0000 7fc761ff2740 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b922e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56418b91cae0 tx=0x7fc748005580 comp rx=0 tx=0).stop 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc761ff2740 1 -- 172.21.15.5:0/1166981998 shutdown_connections 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc761ff2740 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56418b99a400 0x56418b99c820 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc761ff2740 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56418b91f370 0x56418b9233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc761ff2740 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56418b915b40 0x56418b922e90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc7417fa640 0 client.26006 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc761ff2740 1 --2- 172.21.15.5:0/1166981998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56418b9145d0 0x56418b927b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc761ff2740 1 -- 172.21.15.5:0/1166981998 >> 172.21.15.5:0/1166981998 conn(0x56418b8dd310 msgr2=0x56418b91a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc761ff2740 1 -- 172.21.15.5:0/1166981998 shutdown_connections 2024-09-11T22:25:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc761ff2740 1 -- 172.21.15.5:0/1166981998 wait complete. 2024-09-11T22:25:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc761ff2740 20 client.26006 trim_cache size 0 max 16384 2024-09-11T22:25:36.309 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 -- 172.21.15.5:0/2630462078 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7440a2cf0 msgr2=0x7fc7440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7440a2cf0 0x7fc7440a51b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc75001f000 tx=0x7fc750022000 comp rx=0 tx=0).stop 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 -- 172.21.15.5:0/2630462078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc758103ff0 msgr2=0x7fc75819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc758103ff0 0x7fc75819a630 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc75400d950 tx=0x7fc75400de10 comp rx=0 tx=0).stop 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 -- 172.21.15.5:0/2630462078 shutdown_connections 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc758109120 0x7fc75819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc7440a2cf0 0x7fc7440a51b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7581049a0 0x7fc75819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 --2- 172.21.15.5:0/2630462078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc758103ff0 0x7fc75819a630 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 -- 172.21.15.5:0/2630462078 >> 172.21.15.5:0/2630462078 conn(0x7fc7580ffab0 msgr2=0x7fc758100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 -- 172.21.15.5:0/2630462078 shutdown_connections 2024-09-11T22:25:36.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:36.295+0000 7fc75fc19640 1 -- 172.21.15.5:0/2630462078 wait complete. 2024-09-11T22:25:36.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 199008 9.58 MB/sec warmup 114 sec latency 24.491 ms 2024-09-11T22:25:37.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 201979 9.62 MB/sec warmup 115 sec latency 12.925 ms 2024-09-11T22:25:37.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:37 smithi005 ceph-mon[30943]: pgmap v106: 97 pgs: 97 active+clean; 42 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-09-11T22:25:37.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:37 smithi159 ceph-mon[35534]: pgmap v106: 97 pgs: 97 active+clean; 42 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-09-11T22:25:37.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:37 smithi172 ceph-mon[35461]: pgmap v106: 97 pgs: 97 active+clean; 42 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-09-11T22:25:38.190 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:25:38.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:25:38.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204643 9.64 MB/sec warmup 116 sec latency 10.710 ms 2024-09-11T22:25:38.470 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.120+0000 7faa04a05640 1 -- 172.21.15.5:0/773116038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa00102570 msgr2=0x7faa001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.120+0000 7faa04a05640 1 --2- 172.21.15.5:0/773116038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa00102570 0x7faa001029b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009900 tx=0x7fa9e802a480 comp rx=0 tx=0).stop 2024-09-11T22:25:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.121+0000 7faa04a05640 1 -- 172.21.15.5:0/773116038 shutdown_connections 2024-09-11T22:25:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.121+0000 7faa04a05640 1 --2- 172.21.15.5:0/773116038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa00102ef0 0x7faa0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.121+0000 7faa04a05640 1 --2- 172.21.15.5:0/773116038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa00102570 0x7faa001029b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.121+0000 7faa04a05640 1 --2- 172.21.15.5:0/773116038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa00100e20 0x7faa00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.121+0000 7faa04a05640 1 -- 172.21.15.5:0/773116038 >> 172.21.15.5:0/773116038 conn(0x7faa000fc7d0 msgr2=0x7faa000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.121+0000 7faa04a05640 1 -- 172.21.15.5:0/773116038 shutdown_connections 2024-09-11T22:25:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.121+0000 7faa04a05640 1 -- 172.21.15.5:0/773116038 wait complete. 2024-09-11T22:25:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.122+0000 7faa04a05640 1 Processor -- start 2024-09-11T22:25:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7faa04a05640 1 -- start start 2024-09-11T22:25:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7faa04a05640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa00100e20 0x7faa000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7faa04a05640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa00102570 0x7faa000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7faa04a05640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa00102ef0 0x7faa000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7faa04a05640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa00110f50 con 0x7faa00102ef0 2024-09-11T22:25:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7faa04a05640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa00110dd0 con 0x7faa00102570 2024-09-11T22:25:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7faa04a05640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faa001110d0 con 0x7faa00100e20 2024-09-11T22:25:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7fa9fdd9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa00102570 0x7faa000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7fa9fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa00102ef0 0x7faa000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7fa9fdd9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa00102570 0x7faa000fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57350/0 (socket says 172.21.15.5:57350) 2024-09-11T22:25:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7fa9fdd9b640 1 -- 172.21.15.5:0/2548396625 learned_addr learned my addr 172.21.15.5:0/2548396625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.123+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa00100e20 0x7faa000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7fa9fdd9b640 1 -- 172.21.15.5:0/2548396625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa00100e20 msgr2=0x7faa000fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa00100e20 0x7faa000fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7fa9fdd9b640 1 -- 172.21.15.5:0/2548396625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa00102ef0 msgr2=0x7faa000fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa00102ef0 0x7faa000fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7fa9fdd9b640 1 -- 172.21.15.5:0/2548396625 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa001aaeb0 con 0x7faa00102570 2024-09-11T22:25:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7fa9fed9d640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa00102ef0 0x7faa000fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa00102570 0x7faa000fd1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7fa9f77fe640 1 -- 172.21.15.5:0/2548396625 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e803f070 con 0x7faa00102570 2024-09-11T22:25:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7faa04a05640 1 -- 172.21.15.5:0/2548396625 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa001ab080 con 0x7faa00102570 2024-09-11T22:25:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7faa04a05640 1 -- 172.21.15.5:0/2548396625 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001ab610 con 0x7faa00102570 2024-09-11T22:25:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7fa9f77fe640 1 -- 172.21.15.5:0/2548396625 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e8031c30 con 0x7faa00102570 2024-09-11T22:25:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.124+0000 7fa9f77fe640 1 -- 172.21.15.5:0/2548396625 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8049060 con 0x7faa00102570 2024-09-11T22:25:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7faa06dde740 1 Processor -- start 2024-09-11T22:25:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7fa9f77fe640 1 -- 172.21.15.5:0/2548396625 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa9e8042070 con 0x7faa00102570 2024-09-11T22:25:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7faa06dde740 1 -- start start 2024-09-11T22:25:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7faa06dde740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca2b60 0x55d505ca2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7faa06dde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505cad800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7faa06dde740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505cb0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7faa06dde740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d505c08280 con 0x55d505cadd40 2024-09-11T22:25:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7faa06dde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d505bafb10 con 0x55d505ca3480 2024-09-11T22:25:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7faa06dde740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d505c04240 con 0x55d505ca2b60 2024-09-11T22:25:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505cad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505cad800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57360/0 (socket says 172.21.15.5:57360) 2024-09-11T22:25:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7fa9fe59c640 1 -- 172.21.15.5:0/1233146261 learned_addr learned my addr 172.21.15.5:0/1233146261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.127+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/1233146261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505cb0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9fed9d640 1 --2- 172.21.15.5:0/1233146261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca2b60 0x55d505ca2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9f77fe640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa9ec0a2cf0 0x7fa9ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9fe59c640 1 -- 172.21.15.5:0/1233146261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca2b60 msgr2=0x55d505ca2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/1233146261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca2b60 0x55d505ca2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9fe59c640 1 -- 172.21.15.5:0/1233146261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 msgr2=0x55d505cb0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/1233146261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505cb0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9fe59c640 1 -- 172.21.15.5:0/1233146261 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d505bbc1f0 con 0x55d505ca3480 2024-09-11T22:25:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9fed9d640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa9ec0a2cf0 0x7fa9ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9f77fe640 1 -- 172.21.15.5:0/2548396625 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fa9e80ef750 con 0x7faa00102570 2024-09-11T22:25:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/1233146261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505cad800 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009cf0 tx=0x7fa9f0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b735d86583529c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9f4ff9640 1 -- 172.21.15.5:0/1233146261 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0004230 con 0x55d505ca3480 2024-09-11T22:25:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9f4ff9640 1 -- 172.21.15.5:0/1233146261 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f00043d0 con 0x55d505ca3480 2024-09-11T22:25:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.128+0000 7fa9f4ff9640 1 -- 172.21.15.5:0/1233146261 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0019960 con 0x55d505ca3480 2024-09-11T22:25:39.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.129+0000 7faa06dde740 1 -- 172.21.15.5:0/1233146261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 msgr2=0x55d505cad800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:39.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.129+0000 7fa9fed9d640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa9ec0a2cf0 0x7fa9ec0a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8005fd0 tx=0x7fa9f8004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.129+0000 7faa06dde740 1 --2- 172.21.15.5:0/1233146261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505cad800 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009cf0 tx=0x7fa9f0011810 comp rx=0 tx=0).stop 2024-09-11T22:25:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.129+0000 7faa06dde740 1 -- 172.21.15.5:0/1233146261 shutdown_connections 2024-09-11T22:25:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.129+0000 7faa06dde740 1 --2- 172.21.15.5:0/1233146261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505cb0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.129+0000 7faa06dde740 1 --2- 172.21.15.5:0/1233146261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505cad800 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.129+0000 7faa06dde740 1 --2- 172.21.15.5:0/1233146261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca2b60 0x55d505ca2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.129+0000 7faa06dde740 1 -- 172.21.15.5:0/1233146261 >> 172.21.15.5:0/1233146261 conn(0x55d505c9f0f0 msgr2=0x55d505c9f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.129+0000 7faa06dde740 1 -- 172.21.15.5:0/1233146261 shutdown_connections 2024-09-11T22:25:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.129+0000 7faa06dde740 1 -- 172.21.15.5:0/1233146261 wait complete. 2024-09-11T22:25:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7faa06dde740 1 -- start start 2024-09-11T22:25:39.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7faa06dde740 1 Processor -- start 2024-09-11T22:25:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7faa06dde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7faa06dde740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca3480 0x55d505caab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7faa06dde740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505cab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7faa06dde740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d505cb0910 con 0x55d505cadd40 2024-09-11T22:25:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7faa06dde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d505bc2f10 con 0x55d505ca2b60 2024-09-11T22:25:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7faa06dde740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d505cb0a90 con 0x55d505ca3480 2024-09-11T22:25:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7faa06dde740 10 client.0 _ll_register_callbacks cb 0x55d505b8ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57374/0 (socket says 172.21.15.5:57374) 2024-09-11T22:25:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7fa9fe59c640 1 -- 172.21.15.5:0/1218199363 learned_addr learned my addr 172.21.15.5:0/1218199363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7fa9fed9d640 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505cab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.130+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca3480 0x55d505caab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.131+0000 7fa9fe59c640 1 -- 172.21.15.5:0/1218199363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca3480 msgr2=0x55d505caab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.131+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca3480 0x55d505caab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.131+0000 7fa9fe59c640 1 -- 172.21.15.5:0/1218199363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 msgr2=0x55d505cab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.131+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505cab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.131+0000 7fa9fe59c640 1 -- 172.21.15.5:0/1218199363 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5059796a0 con 0x55d505ca2b60 2024-09-11T22:25:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.131+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cad7f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0000c00 tx=0x7fa9f0003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6771e0e52d01e38c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:39.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.131+0000 7fa9e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:25:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.131+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/1218199363 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0027070 con 0x55d505ca2b60 2024-09-11T22:25:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.131+0000 7faa06dde740 1 -- 172.21.15.5:0/1218199363 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f00043d0 con 0x55d505ca2b60 2024-09-11T22:25:39.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.131+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/1218199363 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fa9f0005240 con 0x55d505ca2b60 2024-09-11T22:25:39.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/1218199363 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa9f0033050 con 0x55d505ca2b60 2024-09-11T22:25:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7faa06dde740 10 client.26018 fetch_fsmap learned FSMap version 33 2024-09-11T22:25:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7faa06dde740 1 -- 172.21.15.5:0/1218199363 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d505cbc4a0 con 0x55d505ca2b60 2024-09-11T22:25:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/1218199363 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f00043d0 con 0x55d505ca2b60 2024-09-11T22:25:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/1218199363 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7fa9f0004ba0 con 0x55d505ca2b60 2024-09-11T22:25:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7faa06dde740 10 client.26018 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:25:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7faa06dde740 10 client.26018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7faa06dde740 20 client.26018 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7faa06dde740 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d505cbd8b0 0x55d505cbfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7faa06dde740 4 client.26018 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.139+0000 7faa06dde740 1 -- 172.21.15.5:0/1218199363 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d505c157f0 con 0x55d505cbd8b0 2024-09-11T22:25:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.140+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d505cbd8b0 0x55d505cbfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.140+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d505cbd8b0 0x55d505cbfca0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f589d2028051ca38 server_cookie=644fea32e3f3393c in_seq=0 out_seq=0 2024-09-11T22:25:39.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.140+0000 7fa9e5ffb640 10 client.26018 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.143+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/1218199363 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d505c157f0 con 0x55d505cbd8b0 2024-09-11T22:25:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.143+0000 7fa9e5ffb640 10 client.26018 handle_command_reply: tid=0 2024-09-11T22:25:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.143+0000 7faa06dde740 1 client.26018 shutdown 2024-09-11T22:25:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 1 -- 172.21.15.5:0/1218199363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 msgr2=0x55d505cad7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cad7f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0000c00 tx=0x7fa9f0003820 comp rx=0 tx=0).stop 2024-09-11T22:25:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 1 -- 172.21.15.5:0/1218199363 shutdown_connections 2024-09-11T22:25:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d505cbd8b0 0x55d505cbfca0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505cab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca3480 0x55d505caab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 1 --2- 172.21.15.5:0/1218199363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cad7f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7fa9e5ffb640 0 client.26018 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 1 -- 172.21.15.5:0/1218199363 >> 172.21.15.5:0/1218199363 conn(0x55d505c9f0f0 msgr2=0x55d505ca4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 1 -- 172.21.15.5:0/1218199363 shutdown_connections 2024-09-11T22:25:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 1 -- 172.21.15.5:0/1218199363 wait complete. 2024-09-11T22:25:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.144+0000 7faa06dde740 20 client.26018 trim_cache size 0 max 16384 2024-09-11T22:25:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7faa06dde740 1 Processor -- start 2024-09-11T22:25:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7faa06dde740 1 -- start start 2024-09-11T22:25:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7faa06dde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cc54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7faa06dde740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca3480 0x55d505cc59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7faa06dde740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505ca7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7faa06dde740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e000e8d0 con 0x55d505cadd40 2024-09-11T22:25:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7faa06dde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0010440 con 0x55d505ca2b60 2024-09-11T22:25:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7faa06dde740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e000efa0 con 0x55d505ca3480 2024-09-11T22:25:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7fa9fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505ca7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cc54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7fa9fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505ca7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37224/0 (socket says 172.21.15.5:37224) 2024-09-11T22:25:39.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7fa9fed9d640 1 -- 172.21.15.5:0/857646189 learned_addr learned my addr 172.21.15.5:0/857646189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/857646189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca3480 0x55d505cc59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7fa9fe59c640 1 -- 172.21.15.5:0/857646189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca3480 msgr2=0x55d505cc59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/857646189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca3480 0x55d505cc59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7fa9fe59c640 1 -- 172.21.15.5:0/857646189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 msgr2=0x55d505ca7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.164+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/857646189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505ca7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.165+0000 7fa9fe59c640 1 -- 172.21.15.5:0/857646189 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d505cab640 con 0x55d505ca2b60 2024-09-11T22:25:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.165+0000 7fa9fed9d640 1 --2- 172.21.15.5:0/857646189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505ca7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.165+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/857646189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cc54a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0000c00 tx=0x7fa9f0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e630de678c05f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.165+0000 7fa9e7fff640 1 -- 172.21.15.5:0/857646189 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0030070 con 0x55d505ca2b60 2024-09-11T22:25:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.165+0000 7fa9e7fff640 1 -- 172.21.15.5:0/857646189 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f0009b70 con 0x55d505ca2b60 2024-09-11T22:25:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.165+0000 7fa9e7fff640 1 -- 172.21.15.5:0/857646189 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0027070 con 0x55d505ca2b60 2024-09-11T22:25:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 -- 172.21.15.5:0/857646189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 msgr2=0x55d505cc54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 --2- 172.21.15.5:0/857646189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cc54a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0000c00 tx=0x7fa9f0005f90 comp rx=0 tx=0).stop 2024-09-11T22:25:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 -- 172.21.15.5:0/857646189 shutdown_connections 2024-09-11T22:25:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 --2- 172.21.15.5:0/857646189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505cadd40 0x55d505ca7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 --2- 172.21.15.5:0/857646189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505ca3480 0x55d505cc59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 --2- 172.21.15.5:0/857646189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca2b60 0x55d505cc54a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 -- 172.21.15.5:0/857646189 >> 172.21.15.5:0/857646189 conn(0x55d505c9f0f0 msgr2=0x55d505ca9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:39.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 -- 172.21.15.5:0/857646189 shutdown_connections 2024-09-11T22:25:39.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 -- 172.21.15.5:0/857646189 wait complete. 2024-09-11T22:25:39.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 -- start start 2024-09-11T22:25:39.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.166+0000 7faa06dde740 1 Processor -- start 2024-09-11T22:25:39.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7faa06dde740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505ca2b60 0x55d505caf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7faa06dde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505caf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7faa06dde740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505cadd40 0x55d505cafad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7faa06dde740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e000fd70 con 0x55d505ca2b60 2024-09-11T22:25:39.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7faa06dde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e000f670 con 0x55d505ca3480 2024-09-11T22:25:39.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7faa06dde740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e000c8b0 con 0x55d505cadd40 2024-09-11T22:25:39.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7faa06dde740 10 client.0 _ll_register_callbacks cb 0x7fa9e80f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:39.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505ca2b60 0x55d505caf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505ca2b60 0x55d505caf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37236/0 (socket says 172.21.15.5:37236) 2024-09-11T22:25:39.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fe59c640 1 -- 172.21.15.5:0/2605073857 learned_addr learned my addr 172.21.15.5:0/2605073857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:39.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505caf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fed9d640 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505cadd40 0x55d505cafad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fe59c640 1 -- 172.21.15.5:0/2605073857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505cadd40 msgr2=0x55d505cafad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:39.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505cadd40 0x55d505cafad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fe59c640 1 -- 172.21.15.5:0/2605073857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 msgr2=0x55d505caf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505caf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fe59c640 1 -- 172.21.15.5:0/2605073857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d505ca32b0 con 0x55d505ca2b60 2024-09-11T22:25:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505caf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9fe59c640 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505ca2b60 0x55d505caf050 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00059f0 tx=0x7fa9f0019d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5893a26f299379 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.167+0000 7fa9e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/2605073857 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0030040 con 0x55d505ca2b60 2024-09-11T22:25:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/2605073857 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fa9f0004fa0 con 0x55d505ca2b60 2024-09-11T22:25:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7faa06dde740 1 -- 172.21.15.5:0/2605073857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f0009b70 con 0x55d505ca2b60 2024-09-11T22:25:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/2605073857 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa9f0027040 con 0x55d505ca2b60 2024-09-11T22:25:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7faa06dde740 10 client.16518 fetch_fsmap learned FSMap version 33 2024-09-11T22:25:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7faa06dde740 1 -- 172.21.15.5:0/2605073857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55d505d286e0 con 0x55d505ca2b60 2024-09-11T22:25:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/2605073857 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f0009b70 con 0x55d505ca2b60 2024-09-11T22:25:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/2605073857 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7fa9f0004b40 con 0x55d505ca2b60 2024-09-11T22:25:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7faa06dde740 10 client.16518 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:25:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7faa06dde740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7faa06dde740 20 client.16518 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7faa06dde740 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d505d28d10 0x55d505d2b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7faa06dde740 4 client.16518 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:25:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7faa06dde740 1 -- 172.21.15.5:0/2605073857 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d505c0fd00 con 0x55d505d28d10 2024-09-11T22:25:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.168+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d505d28d10 0x55d505d2b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.169+0000 7fa9fdd9b640 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d505d28d10 0x55d505d2b130 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c504877e44a7ff00 server_cookie=d0c804453403f197 in_seq=0 out_seq=0 2024-09-11T22:25:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.169+0000 7fa9e5ffb640 10 client.16518 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7fa9e5ffb640 1 -- 172.21.15.5:0/2605073857 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d505c0fd00 con 0x55d505d28d10 2024-09-11T22:25:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7fa9e5ffb640 10 client.16518 handle_command_reply: tid=0 2024-09-11T22:25:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 client.16518 shutdown 2024-09-11T22:25:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 -- 172.21.15.5:0/2605073857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505ca2b60 msgr2=0x55d505caf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505ca2b60 0x55d505caf050 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00059f0 tx=0x7fa9f0019d00 comp rx=0 tx=0).stop 2024-09-11T22:25:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 -- 172.21.15.5:0/2605073857 shutdown_connections 2024-09-11T22:25:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d505d28d10 0x55d505d2b130 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d505cadd40 0x55d505cafad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d505ca3480 0x55d505caf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7fa9e5ffb640 0 client.16518 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 --2- 172.21.15.5:0/2605073857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d505ca2b60 0x55d505caf050 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 -- 172.21.15.5:0/2605073857 >> 172.21.15.5:0/2605073857 conn(0x55d505c9f0f0 msgr2=0x55d505c9da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 -- 172.21.15.5:0/2605073857 shutdown_connections 2024-09-11T22:25:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 1 -- 172.21.15.5:0/2605073857 wait complete. 2024-09-11T22:25:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.170+0000 7faa06dde740 20 client.16518 trim_cache size 0 max 16384 2024-09-11T22:25:39.183 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:25:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:25:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e4fd361e-1140-41bd-a2fd-8a8941ec2127", 2024-09-11T22:25:39.183 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:25:39.184 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:25:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 -- 172.21.15.5:0/2548396625 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa9ec0a2cf0 msgr2=0x7fa9ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa9ec0a2cf0 0x7fa9ec0a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8005fd0 tx=0x7fa9f8004a40 comp rx=0 tx=0).stop 2024-09-11T22:25:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 -- 172.21.15.5:0/2548396625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa00102570 msgr2=0x7faa000fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa00102570 0x7faa000fd1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8006010 tx=0x7fa9e8003c10 comp rx=0 tx=0).stop 2024-09-11T22:25:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 -- 172.21.15.5:0/2548396625 shutdown_connections 2024-09-11T22:25:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa00102ef0 0x7faa000fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa9ec0a2cf0 0x7fa9ec0a51b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa00102570 0x7faa000fd1c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 --2- 172.21.15.5:0/2548396625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa00100e20 0x7faa000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 -- 172.21.15.5:0/2548396625 >> 172.21.15.5:0/2548396625 conn(0x7faa000fc7d0 msgr2=0x7faa0006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 -- 172.21.15.5:0/2548396625 shutdown_connections 2024-09-11T22:25:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:39.171+0000 7faa04a05640 1 -- 172.21.15.5:0/2548396625 wait complete. 2024-09-11T22:25:39.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 206709 9.67 MB/sec warmup 117 sec latency 39.204 ms 2024-09-11T22:25:39.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:39 smithi005 ceph-mon[30943]: pgmap v107: 97 pgs: 97 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 333 op/s 2024-09-11T22:25:39.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:39 smithi159 ceph-mon[35534]: pgmap v107: 97 pgs: 97 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 333 op/s 2024-09-11T22:25:39.612 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:39 smithi172 ceph-mon[35461]: pgmap v107: 97 pgs: 97 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 333 op/s 2024-09-11T22:25:39.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:25:40.261 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:25:40.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 209666 9.74 MB/sec warmup 118 sec latency 9.358 ms 2024-09-11T22:25:40.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:40 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:25:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:40 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:25:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:40 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:25:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:40 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:25:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:40 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:25:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:40 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:25:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:40 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:25:40.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:40 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:25:40.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:40 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:25:40.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:40 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:25:40.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:40 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:25:40.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:40 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:25:40.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:40 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:25:40.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:40 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:25:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:40 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:25:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:40 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:25:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:40 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:25:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:40 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:25:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:40 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:25:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:40 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:25:40.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:40 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:25:40.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.886+0000 7f14e4f95640 1 -- 172.21.15.5:0/316303218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14e0103660 msgr2=0x7f14e010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.886+0000 7f14e4f95640 1 --2- 172.21.15.5:0/316303218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14e0103660 0x7f14e010bb50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009900 tx=0x7f14d402a480 comp rx=0 tx=0).stop 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.886+0000 7f14e4f95640 1 -- 172.21.15.5:0/316303218 shutdown_connections 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.886+0000 7f14e4f95640 1 --2- 172.21.15.5:0/316303218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14e010c170 0x7f14e010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.886+0000 7f14e4f95640 1 --2- 172.21.15.5:0/316303218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14e0103660 0x7f14e010bb50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.886+0000 7f14e4f95640 1 --2- 172.21.15.5:0/316303218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e0102d40 0x7f14e0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.886+0000 7f14e4f95640 1 -- 172.21.15.5:0/316303218 >> 172.21.15.5:0/316303218 conn(0x7f14e00fc7a0 msgr2=0x7f14e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.886+0000 7f14e4f95640 1 -- 172.21.15.5:0/316303218 shutdown_connections 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.886+0000 7f14e4f95640 1 -- 172.21.15.5:0/316303218 wait complete. 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14e4f95640 1 Processor -- start 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14e4f95640 1 -- start start 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14e4f95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e0102d40 0x7f14e019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14e4f95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14e0103660 0x7f14e019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14e4f95640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14e010c170 0x7f14e019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14e4f95640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0111f70 con 0x7f14e0103660 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14e4f95640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14e0111df0 con 0x7f14e010c170 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14e4f95640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01120f0 con 0x7f14e0102d40 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14ddd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14e0103660 0x7f14e019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14ddd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14e0103660 0x7f14e019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37252/0 (socket says 172.21.15.5:37252) 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14ddd9b640 1 -- 172.21.15.5:0/3037795800 learned_addr learned my addr 172.21.15.5:0/3037795800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14de59c640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e0102d40 0x7f14e019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14ded9d640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14e010c170 0x7f14e019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14ddd9b640 1 -- 172.21.15.5:0/3037795800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e0102d40 msgr2=0x7f14e019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14ddd9b640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e0102d40 0x7f14e019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14ddd9b640 1 -- 172.21.15.5:0/3037795800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14e010c170 msgr2=0x7f14e019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14ddd9b640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14e010c170 0x7f14e019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.887+0000 7f14ddd9b640 1 -- 172.21.15.5:0/3037795800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e019f610 con 0x7f14e0103660 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.888+0000 7f14ddd9b640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14e0103660 0x7f14e019aba0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f14d4006010 tx=0x7f14d4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.888+0000 7f14cf7fe640 1 -- 172.21.15.5:0/3037795800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d403f030 con 0x7f14e0103660 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.888+0000 7f14cf7fe640 1 -- 172.21.15.5:0/3037795800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d4004440 con 0x7f14e0103660 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.888+0000 7f14e4f95640 1 -- 172.21.15.5:0/3037795800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e019f8a0 con 0x7f14e0103660 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.888+0000 7f14cf7fe640 1 -- 172.21.15.5:0/3037795800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4035070 con 0x7f14e0103660 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.888+0000 7f14e4f95640 1 -- 172.21.15.5:0/3037795800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e01a71e0 con 0x7f14e0103660 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.890+0000 7f14cf7fe640 1 -- 172.21.15.5:0/3037795800 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f14d4042030 con 0x7f14e0103660 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.890+0000 7f14e736e740 1 Processor -- start 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14e736e740 1 -- start start 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14e736e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f964a0 0x55cb71f96880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14e736e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fa1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14e736e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71fa1680 0x55cb71fa3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14e736e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cb71efc2e0 con 0x55cb71fa1680 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14e736e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cb71ea3ce0 con 0x55cb71f964a0 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14e736e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cb71ef82a0 con 0x55cb71f96dc0 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14ded9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f964a0 0x55cb71f96880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14de59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fa1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14ded9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f964a0 0x55cb71f96880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57406/0 (socket says 172.21.15.5:57406) 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14ded9d640 1 -- 172.21.15.5:0/2819080666 learned_addr learned my addr 172.21.15.5:0/2819080666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14ddd9b640 1 --2- 172.21.15.5:0/2819080666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71fa1680 0x55cb71fa3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14cf7fe640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f14c80a2cf0 0x7f14c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14ded9d640 1 -- 172.21.15.5:0/2819080666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 msgr2=0x55cb71fa1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.891+0000 7f14ded9d640 1 --2- 172.21.15.5:0/2819080666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fa1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.892+0000 7f14ded9d640 1 -- 172.21.15.5:0/2819080666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71fa1680 msgr2=0x55cb71fa3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.892+0000 7f14ded9d640 1 --2- 172.21.15.5:0/2819080666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71fa1680 0x55cb71fa3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.892+0000 7f14ded9d640 1 -- 172.21.15.5:0/2819080666 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb71e88440 con 0x55cb71f964a0 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.892+0000 7f14de59c640 1 --2- 172.21.15.5:0/2819080666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fa1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.892+0000 7f14cf7fe640 1 -- 172.21.15.5:0/3037795800 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f14d40ef840 con 0x7f14e0103660 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.892+0000 7f14ded9d640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f14c80a2cf0 0x7f14c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.892+0000 7f14ded9d640 1 --2- 172.21.15.5:0/2819080666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f964a0 0x55cb71f96880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009f50 tx=0x7f14d80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d237593316bedaf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.892+0000 7f14ccff9640 1 -- 172.21.15.5:0/2819080666 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8004050 con 0x55cb71f964a0 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.892+0000 7f14ccff9640 1 -- 172.21.15.5:0/2819080666 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d8004750 con 0x55cb71f964a0 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.892+0000 7f14ccff9640 1 -- 172.21.15.5:0/2819080666 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8019400 con 0x55cb71f964a0 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14e736e740 1 -- 172.21.15.5:0/2819080666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f964a0 msgr2=0x55cb71f96880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14ded9d640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f14c80a2cf0 0x7f14c80a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f14d8012fd0 tx=0x7f14d8019680 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14e736e740 1 --2- 172.21.15.5:0/2819080666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f964a0 0x55cb71f96880 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009f50 tx=0x7f14d80118d0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14e736e740 1 -- 172.21.15.5:0/2819080666 shutdown_connections 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14e736e740 1 --2- 172.21.15.5:0/2819080666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71fa1680 0x55cb71fa3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14e736e740 1 --2- 172.21.15.5:0/2819080666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fa1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14e736e740 1 --2- 172.21.15.5:0/2819080666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f964a0 0x55cb71f96880 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14e736e740 1 -- 172.21.15.5:0/2819080666 >> 172.21.15.5:0/2819080666 conn(0x55cb71f92a30 msgr2=0x55cb71f92e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14e736e740 1 -- 172.21.15.5:0/2819080666 shutdown_connections 2024-09-11T22:25:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14e736e740 1 -- 172.21.15.5:0/2819080666 wait complete. 2024-09-11T22:25:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.893+0000 7f14e736e740 1 -- start start 2024-09-11T22:25:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14e736e740 1 Processor -- start 2024-09-11T22:25:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14e736e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 0x55cb71fa0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14e736e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f96dc0 0x55cb71f9e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14e736e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71fa1680 0x55cb71f9e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14e736e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cb71fa4250 con 0x55cb71f964a0 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14e736e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cb71eb7050 con 0x55cb71f96dc0 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14e736e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cb71fa43d0 con 0x55cb71fa1680 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14e736e740 10 client.0 _ll_register_callbacks cb 0x55cb71ef7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14de59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 0x55cb71fa0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14de59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 0x55cb71fa0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37260/0 (socket says 172.21.15.5:37260) 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14de59c640 1 -- 172.21.15.5:0/4041298533 learned_addr learned my addr 172.21.15.5:0/4041298533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14ddd9b640 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f96dc0 0x55cb71f9e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14ded9d640 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71fa1680 0x55cb71f9e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14de59c640 1 -- 172.21.15.5:0/4041298533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71fa1680 msgr2=0x55cb71f9e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14de59c640 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71fa1680 0x55cb71f9e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14de59c640 1 -- 172.21.15.5:0/4041298533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f96dc0 msgr2=0x55cb71f9e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14de59c640 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f96dc0 0x55cb71f9e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14de59c640 1 -- 172.21.15.5:0/4041298533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb71c6d3c0 con 0x55cb71f964a0 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14ded9d640 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71fa1680 0x55cb71f9e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14de59c640 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 0x55cb71fa0f50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f14d000ba10 tx=0x7f14d000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf96f33da047ea0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.894+0000 7f14c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14c5ffb640 1 -- 172.21.15.5:0/4041298533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0003e80 con 0x55cb71f964a0 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14e736e740 1 -- 172.21.15.5:0/4041298533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d8004750 con 0x55cb71f964a0 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14c5ffb640 1 -- 172.21.15.5:0/4041298533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f14d0011440 con 0x55cb71f964a0 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14c5ffb640 1 -- 172.21.15.5:0/4041298533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f14d0011640 con 0x55cb71f964a0 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14e736e740 10 client.16530 fetch_fsmap learned FSMap version 33 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14e736e740 1 -- 172.21.15.5:0/4041298533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cb71fafc00 con 0x55cb71f964a0 2024-09-11T22:25:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14c5ffb640 1 -- 172.21.15.5:0/4041298533 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d8004750 con 0x55cb71f964a0 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14c5ffb640 1 -- 172.21.15.5:0/4041298533 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f14d0021030 con 0x55cb71f964a0 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14e736e740 10 client.16530 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14e736e740 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14e736e740 20 client.16530 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14e736e740 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cb71fb1110 0x55cb71fb3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14e736e740 4 client.16530 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14e736e740 1 -- 172.21.15.5:0/4041298533 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb71f09850 con 0x55cb71fb1110 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.895+0000 7f14ddd9b640 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cb71fb1110 0x55cb71fb3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.896+0000 7f14c5ffb640 10 client.16530 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.896+0000 7f14ddd9b640 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cb71fb1110 0x55cb71fb3580 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=420917babdc695be server_cookie=cb14f5d0eddc6237 in_seq=0 out_seq=0 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14c5ffb640 1 -- 172.21.15.5:0/4041298533 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb71f09850 con 0x55cb71fb1110 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14c5ffb640 10 client.16530 handle_command_reply: tid=0 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14e736e740 1 client.16530 shutdown 2024-09-11T22:25:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14e736e740 1 -- 172.21.15.5:0/4041298533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 msgr2=0x55cb71fa0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14e736e740 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 0x55cb71fa0f50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f14d000ba10 tx=0x7f14d000bed0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14e736e740 1 -- 172.21.15.5:0/4041298533 shutdown_connections 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14e736e740 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cb71fb1110 0x55cb71fb3580 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14e736e740 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71fa1680 0x55cb71f9e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14e736e740 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f96dc0 0x55cb71f9e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14c5ffb640 0 client.16530 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14e736e740 1 --2- 172.21.15.5:0/4041298533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 0x55cb71fa0f50 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.897+0000 7f14e736e740 1 -- 172.21.15.5:0/4041298533 >> 172.21.15.5:0/4041298533 conn(0x55cb71f92a30 msgr2=0x55cb71f91b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.898+0000 7f14e736e740 1 -- 172.21.15.5:0/4041298533 shutdown_connections 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.898+0000 7f14e736e740 1 -- 172.21.15.5:0/4041298533 wait complete. 2024-09-11T22:25:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.898+0000 7f14e736e740 20 client.16530 trim_cache size 0 max 16384 2024-09-11T22:25:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14e736e740 1 Processor -- start 2024-09-11T22:25:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14e736e740 1 -- start start 2024-09-11T22:25:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14e736e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f92a30 0x55cb71fad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14e736e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 0x55cb71f997d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14e736e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fb8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14e736e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01afc20 con 0x55cb71f964a0 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14e736e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01b1790 con 0x55cb71f92a30 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14e736e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01b02f0 con 0x55cb71f96dc0 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14ddd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 0x55cb71f997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14ded9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fb8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14ded9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fb8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60150/0 (socket says 172.21.15.5:60150) 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14ded9d640 1 -- 172.21.15.5:0/148042253 learned_addr learned my addr 172.21.15.5:0/148042253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14de59c640 1 --2- 172.21.15.5:0/148042253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f92a30 0x55cb71fad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14ded9d640 1 -- 172.21.15.5:0/148042253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f92a30 msgr2=0x55cb71fad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14ded9d640 1 --2- 172.21.15.5:0/148042253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f92a30 0x55cb71fad4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14ded9d640 1 -- 172.21.15.5:0/148042253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 msgr2=0x55cb71f997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.917+0000 7f14ded9d640 1 --2- 172.21.15.5:0/148042253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 0x55cb71f997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.918+0000 7f14ded9d640 1 -- 172.21.15.5:0/148042253 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb71f96bf0 con 0x55cb71f96dc0 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.918+0000 7f14ded9d640 1 --2- 172.21.15.5:0/148042253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fb8e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14d8006010 tx=0x7f14d8028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a2388c7fd45a757 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.918+0000 7f14c7fff640 1 -- 172.21.15.5:0/148042253 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d8009bf0 con 0x55cb71f96dc0 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.918+0000 7f14c7fff640 1 -- 172.21.15.5:0/148042253 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d8003d10 con 0x55cb71f96dc0 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.918+0000 7f14c7fff640 1 -- 172.21.15.5:0/148042253 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d800f8a0 con 0x55cb71f96dc0 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.919+0000 7f14e736e740 1 -- 172.21.15.5:0/148042253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 msgr2=0x55cb71fb8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.919+0000 7f14e736e740 1 --2- 172.21.15.5:0/148042253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fb8e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f14d8006010 tx=0x7f14d8028000 comp rx=0 tx=0).stop 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.919+0000 7f14e736e740 1 -- 172.21.15.5:0/148042253 shutdown_connections 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.919+0000 7f14e736e740 1 --2- 172.21.15.5:0/148042253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f96dc0 0x55cb71fb8e50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.919+0000 7f14e736e740 1 --2- 172.21.15.5:0/148042253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f964a0 0x55cb71f997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.919+0000 7f14e736e740 1 --2- 172.21.15.5:0/148042253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f92a30 0x55cb71fad4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.919+0000 7f14e736e740 1 -- 172.21.15.5:0/148042253 >> 172.21.15.5:0/148042253 conn(0x55cb71f9e8a0 msgr2=0x55cb71f990c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.919+0000 7f14e736e740 1 -- 172.21.15.5:0/148042253 shutdown_connections 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.919+0000 7f14e736e740 1 -- 172.21.15.5:0/148042253 wait complete. 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14e736e740 1 -- start start 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14e736e740 1 Processor -- start 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14e736e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f92a30 0x55cb71fa99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14e736e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f964a0 0x55cb71fa9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14e736e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f96dc0 0x55cb71f9ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14e736e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01b10c0 con 0x55cb71f92a30 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14e736e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01b09c0 con 0x55cb71f96dc0 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14e736e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f14e01adc00 con 0x55cb71f964a0 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14e736e740 10 client.0 _ll_register_callbacks cb 0x7f14d00042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14de59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f92a30 0x55cb71fa99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14de59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f92a30 0x55cb71fa99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37276/0 (socket says 172.21.15.5:37276) 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14de59c640 1 -- 172.21.15.5:0/3742874362 learned_addr learned my addr 172.21.15.5:0/3742874362 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14ded9d640 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f96dc0 0x55cb71f9ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.920+0000 7f14ddd9b640 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f964a0 0x55cb71fa9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.921+0000 7f14de59c640 1 -- 172.21.15.5:0/3742874362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f964a0 msgr2=0x55cb71fa9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.921+0000 7f14de59c640 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f964a0 0x55cb71fa9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.921+0000 7f14de59c640 1 -- 172.21.15.5:0/3742874362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f96dc0 msgr2=0x55cb71f9ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.921+0000 7f14de59c640 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f96dc0 0x55cb71f9ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.921+0000 7f14de59c640 1 -- 172.21.15.5:0/3742874362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb71f9b630 con 0x55cb71f92a30 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.921+0000 7f14de59c640 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f92a30 0x55cb71fa99b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f14d0000c00 tx=0x7f14d001ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88619698dc47e904 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.921+0000 7f14c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.921+0000 7f14c5ffb640 1 -- 172.21.15.5:0/3742874362 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0030060 con 0x55cb71f92a30 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.921+0000 7f14c5ffb640 1 -- 172.21.15.5:0/3742874362 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f14d0011440 con 0x55cb71f92a30 2024-09-11T22:25:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.921+0000 7f14e736e740 1 -- 172.21.15.5:0/3742874362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d8003d10 con 0x55cb71f92a30 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14c5ffb640 1 -- 172.21.15.5:0/3742874362 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f14d00119f0 con 0x55cb71f92a30 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14e736e740 10 client.16536 fetch_fsmap learned FSMap version 33 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14e736e740 1 -- 172.21.15.5:0/3742874362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55cb7201bb00 con 0x55cb71f92a30 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14c5ffb640 1 -- 172.21.15.5:0/3742874362 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d8003d10 con 0x55cb71f92a30 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14c5ffb640 1 -- 172.21.15.5:0/3742874362 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f14d0022ca0 con 0x55cb71f92a30 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14e736e740 10 client.16536 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14e736e740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14e736e740 20 client.16536 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14e736e740 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cb71fa1680 0x55cb7201e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14e736e740 4 client.16536 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14e736e740 1 -- 172.21.15.5:0/3742874362 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cb71f03d60 con 0x55cb71fa1680 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.922+0000 7f14ddd9b640 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cb71fa1680 0x55cb7201e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.923+0000 7f14c5ffb640 10 client.16536 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:40.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.923+0000 7f14ddd9b640 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cb71fa1680 0x55cb7201e210 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b85e369b814c197 server_cookie=cf5d4cf52f5d0303 in_seq=0 out_seq=0 2024-09-11T22:25:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14c5ffb640 1 -- 172.21.15.5:0/3742874362 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cb71f03d60 con 0x55cb71fa1680 2024-09-11T22:25:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14c5ffb640 10 client.16536 handle_command_reply: tid=0 2024-09-11T22:25:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 client.16536 shutdown 2024-09-11T22:25:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 -- 172.21.15.5:0/3742874362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f92a30 msgr2=0x55cb71fa99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f92a30 0x55cb71fa99b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f14d0000c00 tx=0x7f14d001ba50 comp rx=0 tx=0).stop 2024-09-11T22:25:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 -- 172.21.15.5:0/3742874362 shutdown_connections 2024-09-11T22:25:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cb71fa1680 0x55cb7201e210 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cb71f96dc0 0x55cb71f9ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cb71f964a0 0x55cb71fa9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 --2- 172.21.15.5:0/3742874362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb71f92a30 0x55cb71fa99b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 -- 172.21.15.5:0/3742874362 >> 172.21.15.5:0/3742874362 conn(0x55cb71f9e8a0 msgr2=0x55cb71f9c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:40.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14c5ffb640 0 client.16536 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:40.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 -- 172.21.15.5:0/3742874362 shutdown_connections 2024-09-11T22:25:40.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.925+0000 7f14e736e740 1 -- 172.21.15.5:0/3742874362 wait complete. 2024-09-11T22:25:40.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e736e740 20 client.16536 trim_cache size 0 max 16384 2024-09-11T22:25:40.931 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:25:40.931 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:25:40.931 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:25:40.931 INFO:teuthology.orchestra.run.smithi005.stdout: "e4fd361e-1140-41bd-a2fd-8a8941ec2127": { 2024-09-11T22:25:40.931 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:25:40.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e4fd361e-1140-41bd-a2fd-8a8941ec2127", 2024-09-11T22:25:40.932 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:25:40.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:40.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:25:40.932 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:25:40.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 -- 172.21.15.5:0/3037795800 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f14c80a2cf0 msgr2=0x7f14c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f14c80a2cf0 0x7f14c80a51b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f14d8012fd0 tx=0x7f14d8019680 comp rx=0 tx=0).stop 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 -- 172.21.15.5:0/3037795800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14e0103660 msgr2=0x7f14e019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14e0103660 0x7f14e019aba0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f14d4006010 tx=0x7f14d4004040 comp rx=0 tx=0).stop 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 -- 172.21.15.5:0/3037795800 shutdown_connections 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f14c80a2cf0 0x7f14c80a51b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14e010c170 0x7f14e019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14e0103660 0x7f14e019aba0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 --2- 172.21.15.5:0/3037795800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14e0102d40 0x7f14e019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 -- 172.21.15.5:0/3037795800 >> 172.21.15.5:0/3037795800 conn(0x7f14e00fc7a0 msgr2=0x7f14e010d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 -- 172.21.15.5:0/3037795800 shutdown_connections 2024-09-11T22:25:40.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:25:40.926+0000 7f14e4f95640 1 -- 172.21.15.5:0/3037795800 wait complete. 2024-09-11T22:25:41.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 212484 9.77 MB/sec warmup 119 sec latency 19.553 ms 2024-09-11T22:25:41.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:41 smithi005 ceph-mon[30943]: pgmap v108: 97 pgs: 97 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 333 op/s 2024-09-11T22:25:41.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:41 smithi159 ceph-mon[35534]: pgmap v108: 97 pgs: 97 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 333 op/s 2024-09-11T22:25:41.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:41 smithi172 ceph-mon[35461]: pgmap v108: 97 pgs: 97 active+clean; 41 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 333 op/s 2024-09-11T22:25:42.006 INFO:tasks.cephfs.filesystem:scrub status for tag:e4fd361e-1140-41bd-a2fd-8a8941ec2127 - {'path': '/', 'tag': 'e4fd361e-1140-41bd-a2fd-8a8941ec2127', 'options': 'recursive,force'} 2024-09-11T22:25:43.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 217577 14.15 MB/sec execute 1 sec latency 11.359 ms 2024-09-11T22:25:43.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:43 smithi005 ceph-mon[30943]: pgmap v109: 97 pgs: 97 active+clean; 49 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-09-11T22:25:43.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:43 smithi159 ceph-mon[35534]: pgmap v109: 97 pgs: 97 active+clean; 49 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-09-11T22:25:43.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:43 smithi172 ceph-mon[35461]: pgmap v109: 97 pgs: 97 active+clean; 49 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-09-11T22:25:44.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 220439 15.61 MB/sec execute 2 sec latency 11.091 ms 2024-09-11T22:25:45.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 223594 16.30 MB/sec execute 3 sec latency 10.603 ms 2024-09-11T22:25:45.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:45 smithi005 ceph-mon[30943]: pgmap v110: 97 pgs: 97 active+clean; 49 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-09-11T22:25:45.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:45 smithi159 ceph-mon[35534]: pgmap v110: 97 pgs: 97 active+clean; 49 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-09-11T22:25:45.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:45 smithi172 ceph-mon[35461]: pgmap v110: 97 pgs: 97 active+clean; 49 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 333 op/s 2024-09-11T22:25:45.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.986+0000 7f7dc8978640 1 -- 172.21.15.5:0/1770702508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc4101370 msgr2=0x7f7dc410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.986+0000 7f7dc8978640 1 --2- 172.21.15.5:0/1770702508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc4101370 0x7f7dc410d8a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009cd0 tx=0x7f7dbc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.986+0000 7f7dc8978640 1 -- 172.21.15.5:0/1770702508 shutdown_connections 2024-09-11T22:25:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.986+0000 7f7dc8978640 1 --2- 172.21.15.5:0/1770702508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc4101370 0x7f7dc410d8a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.986+0000 7f7dc8978640 1 --2- 172.21.15.5:0/1770702508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc41009d0 0x7f7dc4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.986+0000 7f7dc8978640 1 --2- 172.21.15.5:0/1770702508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7dc41069e0 0x7f7dc4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.986+0000 7f7dc8978640 1 -- 172.21.15.5:0/1770702508 >> 172.21.15.5:0/1770702508 conn(0x7f7dc40fc6f0 msgr2=0x7f7dc40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.986+0000 7f7dc8978640 1 -- 172.21.15.5:0/1770702508 shutdown_connections 2024-09-11T22:25:45.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.986+0000 7f7dc8978640 1 -- 172.21.15.5:0/1770702508 wait complete. 2024-09-11T22:25:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc8978640 1 Processor -- start 2024-09-11T22:25:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc8978640 1 -- start start 2024-09-11T22:25:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc8978640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc41009d0 0x7f7dc4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:45.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc8978640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7dc4101370 0x7f7dc4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc8978640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc41069e0 0x7f7dc419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc8978640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4110e20 con 0x7f7dc41009d0 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc8978640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4110ca0 con 0x7f7dc41069e0 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc8978640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4110fa0 con 0x7f7dc4101370 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc41009d0 0x7f7dc4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc41009d0 0x7f7dc4198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48674/0 (socket says 172.21.15.5:48674) 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc259c640 1 -- 172.21.15.5:0/3869931832 learned_addr learned my addr 172.21.15.5:0/3869931832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc1d9b640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7dc4101370 0x7f7dc4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc41069e0 0x7f7dc419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc259c640 1 -- 172.21.15.5:0/3869931832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7dc4101370 msgr2=0x7f7dc4198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc259c640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7dc4101370 0x7f7dc4198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc259c640 1 -- 172.21.15.5:0/3869931832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc41069e0 msgr2=0x7f7dc419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc259c640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc41069e0 0x7f7dc419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc259c640 1 -- 172.21.15.5:0/3869931832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc419d330 con 0x7f7dc41009d0 2024-09-11T22:25:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.987+0000 7f7dc259c640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc41009d0 0x7f7dc4198340 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7db400e9f0 tx=0x7f7db400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.988+0000 7f7db37fe640 1 -- 172.21.15.5:0/3869931832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db400cd80 con 0x7f7dc41009d0 2024-09-11T22:25:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.988+0000 7f7db37fe640 1 -- 172.21.15.5:0/3869931832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7db4005500 con 0x7f7dc41009d0 2024-09-11T22:25:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.988+0000 7f7dc8978640 1 -- 172.21.15.5:0/3869931832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc419d620 con 0x7f7dc41009d0 2024-09-11T22:25:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.988+0000 7f7db37fe640 1 -- 172.21.15.5:0/3869931832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4010690 con 0x7f7dc41009d0 2024-09-11T22:25:45.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.988+0000 7f7dc8978640 1 -- 172.21.15.5:0/3869931832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc41a4eb0 con 0x7f7dc41009d0 2024-09-11T22:25:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.990+0000 7f7db37fe640 1 -- 172.21.15.5:0/3869931832 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7db40046c0 con 0x7f7dc41009d0 2024-09-11T22:25:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.990+0000 7f7dcb7f1740 1 Processor -- start 2024-09-11T22:25:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dcb7f1740 1 -- start start 2024-09-11T22:25:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fff6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff7410 0x560370001770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x5603700040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56036ff5b2f0 con 0x560370001cb0 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56036ff03270 con 0x56036fff7410 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56036ff572b0 con 0x56036fff6af0 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dc2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fff6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dc2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fff6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56408/0 (socket says 172.21.15.5:56408) 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dc259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff7410 0x560370001770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2426497110 learned_addr learned my addr 172.21.15.5:0/2426497110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dc1d9b640 1 --2- 172.21.15.5:0/2426497110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x5603700040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2426497110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff7410 msgr2=0x560370001770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2426497110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff7410 0x560370001770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2426497110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 msgr2=0x5603700040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2426497110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x5603700040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.991+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2426497110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036ff02340 con 0x56036fff6af0 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.992+0000 7f7db37fe640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7dac0a2da0 0x7f7dac0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.992+0000 7f7dc259c640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7dac0a2da0 0x7f7dac0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.992+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2426497110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fff6ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009cd0 tx=0x7f7dbc009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d050717ee15831e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.992+0000 7f7db37fe640 1 -- 172.21.15.5:0/3869931832 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7db40cf840 con 0x7f7dc41009d0 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.992+0000 7f7db0ff9640 1 -- 172.21.15.5:0/2426497110 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc048070 con 0x56036fff6af0 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.992+0000 7f7db0ff9640 1 -- 172.21.15.5:0/2426497110 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dbc036070 con 0x56036fff6af0 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.992+0000 7f7db0ff9640 1 -- 172.21.15.5:0/2426497110 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0043a0 con 0x56036fff6af0 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2426497110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 msgr2=0x56036fff6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dc259c640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7dac0a2da0 0x7f7dac0a5260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7db40cd000 tx=0x7f7db40d5550 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2426497110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fff6ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009cd0 tx=0x7f7dbc009a20 comp rx=0 tx=0).stop 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2426497110 shutdown_connections 2024-09-11T22:25:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2426497110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x5603700040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2426497110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff7410 0x560370001770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2426497110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fff6ed0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2426497110 >> 172.21.15.5:0/2426497110 conn(0x56036fff3060 msgr2=0x56036fff3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2426497110 shutdown_connections 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2426497110 wait complete. 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 -- start start 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 Processor -- start 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff6af0 0x56036fffebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.993+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56036fff7410 0x56036ffff110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560370001cb0 0x56036ffff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:45.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560370008c90 con 0x56036fff7410 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56036fee4290 con 0x56036fff6af0 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560370008e10 con 0x560370001cb0 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dcb7f1740 10 client.0 _ll_register_callbacks cb 0x56036ff15140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56036fff7410 0x56036ffff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc2d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff6af0 0x56036fffebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56036fff7410 0x56036ffff110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48700/0 (socket says 172.21.15.5:48700) 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc1d9b640 1 -- 172.21.15.5:0/1915134373 learned_addr learned my addr 172.21.15.5:0/1915134373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560370001cb0 0x56036ffff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc1d9b640 1 -- 172.21.15.5:0/1915134373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560370001cb0 msgr2=0x56036ffff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc1d9b640 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560370001cb0 0x56036ffff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc1d9b640 1 -- 172.21.15.5:0/1915134373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff6af0 msgr2=0x56036fffebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:46.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc1d9b640 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff6af0 0x56036fffebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc1d9b640 1 -- 172.21.15.5:0/1915134373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56036fc6e210 con 0x56036fff7410 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dc1d9b640 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56036fff7410 0x56036ffff110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7db8012730 tx=0x7f7db8012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6735217c095432 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7da9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7da9ffb640 1 -- 172.21.15.5:0/1915134373 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db800f920 con 0x56036fff7410 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7da9ffb640 1 -- 172.21.15.5:0/1915134373 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7db800fac0 con 0x56036fff7410 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.994+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/1915134373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dbc036070 con 0x56036fff7410 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7da9ffb640 1 -- 172.21.15.5:0/1915134373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7db800fcc0 con 0x56036fff7410 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7dcb7f1740 10 client.16548 fetch_fsmap learned FSMap version 33 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/1915134373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56036fd3a620 con 0x56036fff7410 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7da9ffb640 1 -- 172.21.15.5:0/1915134373 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dbc036070 con 0x56036fff7410 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7da9ffb640 1 -- 172.21.15.5:0/1915134373 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f7db8022030 con 0x56036fff7410 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7dcb7f1740 10 client.16548 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:25:46.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7dcb7f1740 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7dcb7f1740 20 client.16548 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560370016100 0x560370018550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7dcb7f1740 4 client.16548 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/1915134373 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56036ff68a20 con 0x560370016100 2024-09-11T22:25:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.995+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560370016100 0x560370018550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.996+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560370016100 0x560370018550 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86557ff495336f9 server_cookie=d933c108c4c0d6e3 in_seq=0 out_seq=0 2024-09-11T22:25:46.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.996+0000 7f7da9ffb640 10 client.16548 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7da9ffb640 1 -- 172.21.15.5:0/1915134373 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56036ff68a20 con 0x560370016100 2024-09-11T22:25:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7da9ffb640 10 client.16548 handle_command_reply: tid=0 2024-09-11T22:25:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 client.16548 shutdown 2024-09-11T22:25:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/1915134373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56036fff7410 msgr2=0x56036ffff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:46.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56036fff7410 0x56036ffff110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7db8012730 tx=0x7f7db8012bf0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/1915134373 shutdown_connections 2024-09-11T22:25:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560370016100 0x560370018550 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560370001cb0 0x56036ffff650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56036fff7410 0x56036ffff110 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7da9ffb640 0 client.16548 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/1915134373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff6af0 0x56036fffebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/1915134373 >> 172.21.15.5:0/1915134373 conn(0x56036fff3060 msgr2=0x56036fff8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/1915134373 shutdown_connections 2024-09-11T22:25:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/1915134373 wait complete. 2024-09-11T22:25:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:45.997+0000 7f7dcb7f1740 20 client.16548 trim_cache size 0 max 16384 2024-09-11T22:25:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.011+0000 7f7dcb7f1740 1 Processor -- start 2024-09-11T22:25:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.011+0000 7f7dcb7f1740 1 -- start start 2024-09-11T22:25:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.011+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff6af0 0x56036fff9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.011+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff7410 0x56037001deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.011+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x56037001e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.011+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc41ab6f0 con 0x560370001cb0 2024-09-11T22:25:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.011+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc41ad260 con 0x56036fff6af0 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.011+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc41abdc0 con 0x56036fff7410 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x56037001e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff7410 0x56037001deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc1d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff7410 0x56037001deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56428/0 (socket says 172.21.15.5:56428) 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x56037001e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48710/0 (socket says 172.21.15.5:48710) 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc1d9b640 1 -- 172.21.15.5:0/2961322907 learned_addr learned my addr 172.21.15.5:0/2961322907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2961322907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff6af0 0x56036fff9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2961322907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff7410 msgr2=0x56037001deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2961322907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff7410 0x56037001deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2961322907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff6af0 msgr2=0x56036fff9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2961322907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff6af0 0x56036fff9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2961322907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56036fff4110 con 0x560370001cb0 2024-09-11T22:25:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc1d9b640 1 --2- 172.21.15.5:0/2961322907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff7410 0x56037001deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2961322907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x56037001e3f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc046040 tx=0x7f7dbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23193ad00e64176d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dabfff640 1 -- 172.21.15.5:0/2961322907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc048040 con 0x560370001cb0 2024-09-11T22:25:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dabfff640 1 -- 172.21.15.5:0/2961322907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dbc036070 con 0x560370001cb0 2024-09-11T22:25:46.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.012+0000 7f7dabfff640 1 -- 172.21.15.5:0/2961322907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc0047d0 con 0x560370001cb0 2024-09-11T22:25:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2961322907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 msgr2=0x56037001e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2961322907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x56037001e3f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc046040 tx=0x7f7dbc004a40 comp rx=0 tx=0).stop 2024-09-11T22:25:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2961322907 shutdown_connections 2024-09-11T22:25:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2961322907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x56037001e3f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2961322907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff7410 0x56037001deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2961322907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff6af0 0x56036fff9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2961322907 >> 172.21.15.5:0/2961322907 conn(0x56036fff3060 msgr2=0x560370000c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2961322907 shutdown_connections 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2961322907 wait complete. 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 -- start start 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.013+0000 7f7dcb7f1740 1 Processor -- start 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fffbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff7410 0x56036fffc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dcb7f1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x56036fffc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc41acb90 con 0x560370001cb0 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc41ac490 con 0x56036fff7410 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dcb7f1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc41a96d0 con 0x56036fff6af0 2024-09-11T22:25:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dcb7f1740 10 client.0 _ll_register_callbacks cb 0x7f7db8009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fffbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc2d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fffbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56444/0 (socket says 172.21.15.5:56444) 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2052154430 learned_addr learned my addr 172.21.15.5:0/2052154430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x56036fffc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc1d9b640 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff7410 0x56036fffc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2052154430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff7410 msgr2=0x56036fffc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff7410 0x56036fffc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2052154430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 msgr2=0x56036fffc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x56036fffc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc2d9d640 1 -- 172.21.15.5:0/2052154430 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037007fe80 con 0x56036fff6af0 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7dc2d9d640 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fffbc20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc005e30 tx=0x7f7dbc02b020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee52dec24bf5b252 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7da9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.014+0000 7f7da9ffb640 1 -- 172.21.15.5:0/2052154430 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc048040 con 0x56036fff6af0 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2052154430 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dbc036070 con 0x56036fff6af0 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7da9ffb640 1 -- 172.21.15.5:0/2052154430 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7dbc031ac0 con 0x56036fff6af0 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7da9ffb640 1 -- 172.21.15.5:0/2052154430 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7dbc004040 con 0x56036fff6af0 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7dcb7f1740 10 client.25996 fetch_fsmap learned FSMap version 33 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2052154430 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x560370081170 con 0x56036fff6af0 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7da9ffb640 1 -- 172.21.15.5:0/2052154430 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dbc036070 con 0x56036fff6af0 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7da9ffb640 1 -- 172.21.15.5:0/2052154430 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f7dbc04e040 con 0x56036fff6af0 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7dcb7f1740 10 client.25996 fetch_fsmap finished waiting for FSMap version 33 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7dcb7f1740 10 client.25996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7dcb7f1740 20 client.25996 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560370081390 0x560370083780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7dcb7f1740 4 client.25996 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:25:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2052154430 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56036ff62d70 con 0x560370081390 2024-09-11T22:25:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.015+0000 7f7dc1d9b640 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560370081390 0x560370083780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.016+0000 7f7dc1d9b640 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560370081390 0x560370083780 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=759a9af09bfb9be5 server_cookie=47b48af31d4bd364 in_seq=0 out_seq=0 2024-09-11T22:25:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.016+0000 7f7da9ffb640 10 client.25996 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7da9ffb640 1 -- 172.21.15.5:0/2052154430 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56036ff62d70 con 0x560370081390 2024-09-11T22:25:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7da9ffb640 10 client.25996 handle_command_reply: tid=0 2024-09-11T22:25:46.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 client.25996 shutdown 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2052154430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 msgr2=0x56036fffbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fffbc20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc005e30 tx=0x7f7dbc02b020 comp rx=0 tx=0).stop 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2052154430 shutdown_connections 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560370081390 0x560370083780 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560370001cb0 0x56036fffc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7da9ffb640 0 client.25996 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56036fff7410 0x56036fffc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 --2- 172.21.15.5:0/2052154430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56036fff6af0 0x56036fffbc20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2052154430 >> 172.21.15.5:0/2052154430 conn(0x56036fff3060 msgr2=0x56036fffeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2052154430 shutdown_connections 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 1 -- 172.21.15.5:0/2052154430 wait complete. 2024-09-11T22:25:46.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.131+0000 7f7dcb7f1740 20 client.25996 trim_cache size 0 max 16384 2024-09-11T22:25:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:25:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:25:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:25:46.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 -- 172.21.15.5:0/3869931832 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7dac0a2da0 msgr2=0x7f7dac0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7dac0a2da0 0x7f7dac0a5260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7db40cd000 tx=0x7f7db40d5550 comp rx=0 tx=0).stop 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 -- 172.21.15.5:0/3869931832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc41009d0 msgr2=0x7f7dc4198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc41009d0 0x7f7dc4198340 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7db400e9f0 tx=0x7f7db400edf0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 -- 172.21.15.5:0/3869931832 shutdown_connections 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7dac0a2da0 0x7f7dac0a5260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7dc41069e0 0x7f7dc419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7dc4101370 0x7f7dc4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 --2- 172.21.15.5:0/3869931832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc41009d0 0x7f7dc4198340 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 -- 172.21.15.5:0/3869931832 >> 172.21.15.5:0/3869931832 conn(0x7f7dc40fc6f0 msgr2=0x7f7dc40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:46.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 -- 172.21.15.5:0/3869931832 shutdown_connections 2024-09-11T22:25:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:46.132+0000 7f7dc8978640 1 -- 172.21.15.5:0/3869931832 wait complete. 2024-09-11T22:25:46.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 225954 14.37 MB/sec execute 4 sec latency 48.140 ms 2024-09-11T22:25:47.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 228114 14.28 MB/sec execute 5 sec latency 11.084 ms 2024-09-11T22:25:47.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:47 smithi005 ceph-mon[30943]: pgmap v111: 97 pgs: 97 active+clean; 49 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-09-11T22:25:47.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:47 smithi159 ceph-mon[35534]: pgmap v111: 97 pgs: 97 active+clean; 49 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-09-11T22:25:47.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:47 smithi172 ceph-mon[35461]: pgmap v111: 97 pgs: 97 active+clean; 49 MiB data, 762 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.1 MiB/s wr, 279 op/s 2024-09-11T22:25:48.311 INFO:tasks.workunit.client.0.smithi005.stdout: 1 230988 14.76 MB/sec execute 6 sec latency 46.538 ms 2024-09-11T22:25:49.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 233618 14.45 MB/sec execute 7 sec latency 12.797 ms 2024-09-11T22:25:49.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:49 smithi005 ceph-mon[30943]: pgmap v112: 97 pgs: 97 active+clean; 44 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.8 MiB/s wr, 470 op/s 2024-09-11T22:25:49.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:49 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:25:49.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:49 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:25:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:49 smithi159 ceph-mon[35534]: pgmap v112: 97 pgs: 97 active+clean; 44 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.8 MiB/s wr, 470 op/s 2024-09-11T22:25:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:49 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:25:49.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:49 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:25:49.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:49 smithi172 ceph-mon[35461]: pgmap v112: 97 pgs: 97 active+clean; 44 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.8 MiB/s wr, 470 op/s 2024-09-11T22:25:49.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:49 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:25:49.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:49 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:25:50.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 236366 14.19 MB/sec execute 8 sec latency 9.918 ms 2024-09-11T22:25:50.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:50 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:25:50.578 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:50 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:25:50.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:50 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:25:51.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 238599 14.17 MB/sec execute 9 sec latency 11.454 ms 2024-09-11T22:25:51.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:51 smithi005 ceph-mon[30943]: pgmap v113: 97 pgs: 97 active+clean; 44 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 300 op/s 2024-09-11T22:25:51.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:51 smithi159 ceph-mon[35534]: pgmap v113: 97 pgs: 97 active+clean; 44 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 300 op/s 2024-09-11T22:25:51.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:51 smithi172 ceph-mon[35461]: pgmap v113: 97 pgs: 97 active+clean; 44 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.6 MiB/s wr, 300 op/s 2024-09-11T22:25:52.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241561 14.56 MB/sec execute 10 sec latency 10.846 ms 2024-09-11T22:25:53.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244405 14.46 MB/sec execute 11 sec latency 15.460 ms 2024-09-11T22:25:53.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:53 smithi005 ceph-mon[30943]: pgmap v114: 97 pgs: 97 active+clean; 52 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 407 op/s 2024-09-11T22:25:53.578 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:53 smithi159 ceph-mon[35534]: pgmap v114: 97 pgs: 97 active+clean; 52 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 407 op/s 2024-09-11T22:25:53.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:53 smithi172 ceph-mon[35461]: pgmap v114: 97 pgs: 97 active+clean; 52 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 407 op/s 2024-09-11T22:25:54.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 246305 14.18 MB/sec execute 12 sec latency 11.315 ms 2024-09-11T22:25:55.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248949 14.17 MB/sec execute 13 sec latency 10.560 ms 2024-09-11T22:25:55.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:55 smithi005 ceph-mon[30943]: pgmap v115: 97 pgs: 97 active+clean; 52 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-09-11T22:25:55.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:55 smithi159 ceph-mon[35534]: pgmap v115: 97 pgs: 97 active+clean; 52 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-09-11T22:25:55.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:55 smithi172 ceph-mon[35461]: pgmap v115: 97 pgs: 97 active+clean; 52 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-09-11T22:25:56.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 251990 14.43 MB/sec execute 14 sec latency 47.177 ms 2024-09-11T22:25:57.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 254756 14.30 MB/sec execute 15 sec latency 12.581 ms 2024-09-11T22:25:57.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:57 smithi005 ceph-mon[30943]: pgmap v116: 97 pgs: 97 active+clean; 52 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-09-11T22:25:57.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:57 smithi159 ceph-mon[35534]: pgmap v116: 97 pgs: 97 active+clean; 52 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-09-11T22:25:57.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:57 smithi172 ceph-mon[35461]: pgmap v116: 97 pgs: 97 active+clean; 52 MiB data, 777 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 299 op/s 2024-09-11T22:25:58.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 257125 14.23 MB/sec execute 16 sec latency 10.125 ms 2024-09-11T22:25:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 -- 172.21.15.5:0/2655241984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a010c110 msgr2=0x7f32a010e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 --2- 172.21.15.5:0/2655241984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a010c110 0x7f32a010e590 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3298009ca0 tx=0x7f329802a950 comp rx=0 tx=0).stop 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 -- 172.21.15.5:0/2655241984 shutdown_connections 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 --2- 172.21.15.5:0/2655241984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a010c110 0x7f32a010e590 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 --2- 172.21.15.5:0/2655241984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32a01035e0 0x7f32a010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 --2- 172.21.15.5:0/2655241984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a0102cc0 0x7f32a01030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 -- 172.21.15.5:0/2655241984 >> 172.21.15.5:0/2655241984 conn(0x7f32a00fc6b0 msgr2=0x7f32a00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 -- 172.21.15.5:0/2655241984 shutdown_connections 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 -- 172.21.15.5:0/2655241984 wait complete. 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 Processor -- start 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.225+0000 7f32a8591640 1 -- start start 2024-09-11T22:25:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a8591640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a0102cc0 0x7f32a0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a8591640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a01035e0 0x7f32a0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a8591640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32a010c110 0x7f32a019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a8591640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0111ef0 con 0x7f32a0102cc0 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a8591640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0111d70 con 0x7f32a010c110 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a8591640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0112070 con 0x7f32a01035e0 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a632d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a0102cc0 0x7f32a0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a632d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a0102cc0 0x7f32a0198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34820/0 (socket says 172.21.15.5:34820) 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a632d640 1 -- 172.21.15.5:0/2823701812 learned_addr learned my addr 172.21.15.5:0/2823701812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32a010c110 0x7f32a019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a5b2c640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a01035e0 0x7f32a0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a632d640 1 -- 172.21.15.5:0/2823701812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a01035e0 msgr2=0x7f32a0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a632d640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a01035e0 0x7f32a0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a632d640 1 -- 172.21.15.5:0/2823701812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32a010c110 msgr2=0x7f32a019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a632d640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32a010c110 0x7f32a019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a632d640 1 -- 172.21.15.5:0/2823701812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a019d330 con 0x7f32a0102cc0 2024-09-11T22:25:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a5b2c640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a01035e0 0x7f32a0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:25:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a632d640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a0102cc0 0x7f32a0198340 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f329c00d770 tx=0x7f329c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32977fe640 1 -- 172.21.15.5:0/2823701812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c017070 con 0x7f32a0102cc0 2024-09-11T22:25:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32977fe640 1 -- 172.21.15.5:0/2823701812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f329c005500 con 0x7f32a0102cc0 2024-09-11T22:25:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a8591640 1 -- 172.21.15.5:0/2823701812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a019d620 con 0x7f32a0102cc0 2024-09-11T22:25:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32977fe640 1 -- 172.21.15.5:0/2823701812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c004e00 con 0x7f32a0102cc0 2024-09-11T22:25:59.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.226+0000 7f32a8591640 1 -- 172.21.15.5:0/2823701812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a01a4eb0 con 0x7f32a0102cc0 2024-09-11T22:25:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.229+0000 7f32977fe640 1 -- 172.21.15.5:0/2823701812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f329c004fa0 con 0x7f32a0102cc0 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.229+0000 7f32ab40a740 1 Processor -- start 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.229+0000 7f32ab40a740 1 -- start start 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 0x5594186d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186dd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186e00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32ab40a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5594186372f0 con 0x5594186ddcb0 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32ab40a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5594185df270 con 0x5594186d2af0 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32ab40a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594186332b0 con 0x5594186d3410 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32977fe640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f328c0a2e20 0x7f328c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a6b2e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 0x5594186d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a5b2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186e00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a6b2e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 0x5594186d2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35024/0 (socket says 172.21.15.5:35024) 2024-09-11T22:25:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a6b2e640 1 -- 172.21.15.5:0/474487210 learned_addr learned my addr 172.21.15.5:0/474487210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a5b2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186e00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34832/0 (socket says 172.21.15.5:34832) 2024-09-11T22:25:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f328c0a2e20 0x7f328c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32977fe640 1 -- 172.21.15.5:0/2823701812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f329c0cef00 con 0x7f32a0102cc0 2024-09-11T22:25:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a632d640 1 --2- 172.21.15.5:0/474487210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186dd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a5b2c640 1 -- 172.21.15.5:0/474487210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 msgr2=0x5594186dd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a5b2c640 1 --2- 172.21.15.5:0/474487210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186dd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a5b2c640 1 -- 172.21.15.5:0/474487210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 msgr2=0x5594186d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a5b2c640 1 --2- 172.21.15.5:0/474487210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 0x5594186d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a5b2c640 1 -- 172.21.15.5:0/474487210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594185de340 con 0x5594186ddcb0 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/474487210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 0x5594186d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a5b2c640 1 --2- 172.21.15.5:0/474487210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186e00a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3290009b70 tx=0x7f32900117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7651895bed3ba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.230+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f328c0a2e20 0x7f328c0a52e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3298005fe0 tx=0x7f3298005c50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f3294bfe640 1 -- 172.21.15.5:0/474487210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290009e60 con 0x5594186ddcb0 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f3294bfe640 1 -- 172.21.15.5:0/474487210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3290012930 con 0x5594186ddcb0 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f3294bfe640 1 -- 172.21.15.5:0/474487210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329000fa90 con 0x5594186ddcb0 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f32ab40a740 1 -- 172.21.15.5:0/474487210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 msgr2=0x5594186e00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f32ab40a740 1 --2- 172.21.15.5:0/474487210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186e00a0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3290009b70 tx=0x7f32900117b0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f32ab40a740 1 -- 172.21.15.5:0/474487210 shutdown_connections 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f32ab40a740 1 --2- 172.21.15.5:0/474487210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186e00a0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f32ab40a740 1 --2- 172.21.15.5:0/474487210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186dd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f32ab40a740 1 --2- 172.21.15.5:0/474487210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 0x5594186d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f32ab40a740 1 -- 172.21.15.5:0/474487210 >> 172.21.15.5:0/474487210 conn(0x5594186cf060 msgr2=0x5594186cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.231+0000 7f32ab40a740 1 -- 172.21.15.5:0/474487210 shutdown_connections 2024-09-11T22:25:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32ab40a740 1 -- 172.21.15.5:0/474487210 wait complete. 2024-09-11T22:25:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32ab40a740 1 -- start start 2024-09-11T22:25:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32ab40a740 1 Processor -- start 2024-09-11T22:25:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d2af0 0x5594186dab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d3410 0x5594186db0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186db610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32ab40a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5594186e0880 con 0x5594186ddcb0 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32ab40a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5594185c0290 con 0x5594186d3410 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32ab40a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594186e0a00 con 0x5594186d2af0 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32ab40a740 10 client.0 _ll_register_callbacks cb 0x5594185f1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32a6b2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186db610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32a6b2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186db610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34840/0 (socket says 172.21.15.5:34840) 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32a6b2e640 1 -- 172.21.15.5:0/3459591613 learned_addr learned my addr 172.21.15.5:0/3459591613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32a5b2c640 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d3410 0x5594186db0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.232+0000 7f32a632d640 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d2af0 0x5594186dab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f32a6b2e640 1 -- 172.21.15.5:0/3459591613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d2af0 msgr2=0x5594186dab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d2af0 0x5594186dab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f32a6b2e640 1 -- 172.21.15.5:0/3459591613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d3410 msgr2=0x5594186db0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d3410 0x5594186db0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f32a6b2e640 1 -- 172.21.15.5:0/3459591613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55941834a210 con 0x5594186ddcb0 2024-09-11T22:25:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f32a632d640 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d2af0 0x5594186dab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:25:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186db610 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f329802ae70 tx=0x7f329802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa61c3f2c78a778e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f3289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f3289ffb640 1 -- 172.21.15.5:0/3459591613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329804f030 con 0x5594186ddcb0 2024-09-11T22:25:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f32ab40a740 1 -- 172.21.15.5:0/3459591613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3290012930 con 0x5594186ddcb0 2024-09-11T22:25:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f3289ffb640 1 -- 172.21.15.5:0/3459591613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f329802fae0 con 0x5594186ddcb0 2024-09-11T22:25:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f3289ffb640 1 -- 172.21.15.5:0/3459591613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3298034040 con 0x5594186ddcb0 2024-09-11T22:25:59.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f32ab40a740 10 client.16566 fetch_fsmap learned FSMap version 34 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f32ab40a740 1 -- 172.21.15.5:0/3459591613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559418416620 con 0x5594186ddcb0 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f3289ffb640 1 -- 172.21.15.5:0/3459591613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3290012930 con 0x5594186ddcb0 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.233+0000 7f3289ffb640 1 -- 172.21.15.5:0/3459591613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f329805c590 con 0x5594186ddcb0 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.234+0000 7f32ab40a740 10 client.16566 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.234+0000 7f32ab40a740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.234+0000 7f32ab40a740 20 client.16566 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.234+0000 7f32ab40a740 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594186edba0 0x5594186f0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.234+0000 7f32ab40a740 4 client.16566 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.234+0000 7f32ab40a740 1 -- 172.21.15.5:0/3459591613 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559418644a20 con 0x5594186edba0 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.234+0000 7f32a632d640 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594186edba0 0x5594186f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.234+0000 7f3289ffb640 10 client.16566 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:59.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.234+0000 7f32a632d640 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594186edba0 0x5594186f0010 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c37a070072dea0a5 server_cookie=c0b7b2d1b59e3b3a in_seq=0 out_seq=0 2024-09-11T22:25:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f3289ffb640 1 -- 172.21.15.5:0/3459591613 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559418644a20 con 0x5594186edba0 2024-09-11T22:25:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f3289ffb640 10 client.16566 handle_command_reply: tid=0 2024-09-11T22:25:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 client.16566 shutdown 2024-09-11T22:25:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 -- 172.21.15.5:0/3459591613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 msgr2=0x5594186db610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186db610 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f329802ae70 tx=0x7f329802a950 comp rx=0 tx=0).stop 2024-09-11T22:25:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 -- 172.21.15.5:0/3459591613 shutdown_connections 2024-09-11T22:25:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594186edba0 0x5594186f0010 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186db610 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d3410 0x5594186db0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f3289ffb640 0 client.16566 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 --2- 172.21.15.5:0/3459591613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d2af0 0x5594186dab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 -- 172.21.15.5:0/3459591613 >> 172.21.15.5:0/3459591613 conn(0x5594186cf060 msgr2=0x5594186d4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 -- 172.21.15.5:0/3459591613 shutdown_connections 2024-09-11T22:25:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 1 -- 172.21.15.5:0/3459591613 wait complete. 2024-09-11T22:25:59.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.236+0000 7f32ab40a740 20 client.16566 trim_cache size 0 max 16384 2024-09-11T22:25:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.250+0000 7f32ab40a740 1 Processor -- start 2024-09-11T22:25:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32ab40a740 1 -- start start 2024-09-11T22:25:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 0x5594186db440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186f5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32ab40a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01ad8f0 con 0x5594186ddcb0 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32ab40a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01af430 con 0x5594186d2af0 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32ab40a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01adfc0 con 0x5594186d3410 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a6b2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a6b2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186f5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34856/0 (socket says 172.21.15.5:34856) 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a6b2e640 1 -- 172.21.15.5:0/771498807 learned_addr learned my addr 172.21.15.5:0/771498807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a632d640 1 --2- 172.21.15.5:0/771498807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 0x5594186db440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a5b2c640 1 --2- 172.21.15.5:0/771498807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a6b2e640 1 -- 172.21.15.5:0/771498807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 msgr2=0x5594186dcb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/771498807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186dcb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a6b2e640 1 -- 172.21.15.5:0/771498807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 msgr2=0x5594186db440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/771498807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 0x5594186db440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a6b2e640 1 -- 172.21.15.5:0/771498807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594186d3240 con 0x5594186ddcb0 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/771498807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186f5970 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f329805c980 tx=0x7f329802f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0f28ee74563aa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.251+0000 7f328bfff640 1 -- 172.21.15.5:0/771498807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298061070 con 0x5594186ddcb0 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f328bfff640 1 -- 172.21.15.5:0/771498807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3298004730 con 0x5594186ddcb0 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f328bfff640 1 -- 172.21.15.5:0/771498807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329804f020 con 0x5594186ddcb0 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f32ab40a740 1 -- 172.21.15.5:0/771498807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 msgr2=0x5594186f5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f32ab40a740 1 --2- 172.21.15.5:0/771498807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186f5970 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f329805c980 tx=0x7f329802f760 comp rx=0 tx=0).stop 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f32ab40a740 1 -- 172.21.15.5:0/771498807 shutdown_connections 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f32ab40a740 1 --2- 172.21.15.5:0/771498807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186ddcb0 0x5594186f5970 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f32ab40a740 1 --2- 172.21.15.5:0/771498807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186dcb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f32ab40a740 1 --2- 172.21.15.5:0/771498807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186d2af0 0x5594186db440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f32ab40a740 1 -- 172.21.15.5:0/771498807 >> 172.21.15.5:0/771498807 conn(0x5594186cf060 msgr2=0x5594186dc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f32ab40a740 1 -- 172.21.15.5:0/771498807 shutdown_connections 2024-09-11T22:25:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f32ab40a740 1 -- 172.21.15.5:0/771498807 wait complete. 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.252+0000 7f32ab40a740 1 -- start start 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32ab40a740 1 Processor -- start 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186d2af0 0x5594186e1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186e1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32ab40a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186ddcb0 0x5594186e23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32ab40a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01aed60 con 0x5594186d2af0 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32ab40a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01ae690 con 0x5594186ddcb0 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32ab40a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01ab8d0 con 0x5594186d3410 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32ab40a740 10 client.0 _ll_register_callbacks cb 0x7f329802f7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a632d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186d2af0 0x5594186e1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a632d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186d2af0 0x5594186e1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34868/0 (socket says 172.21.15.5:34868) 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a632d640 1 -- 172.21.15.5:0/876899475 learned_addr learned my addr 172.21.15.5:0/876899475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:25:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a5b2c640 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186e1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a6b2e640 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186ddcb0 0x5594186e23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a632d640 1 -- 172.21.15.5:0/876899475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 msgr2=0x5594186e1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a632d640 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186e1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a632d640 1 -- 172.21.15.5:0/876899475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186ddcb0 msgr2=0x5594186e23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a632d640 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186ddcb0 0x5594186e23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a632d640 1 -- 172.21.15.5:0/876899475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559418757940 con 0x5594186d2af0 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f32a632d640 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186d2af0 0x5594186e1940 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f329c09b080 tx=0x7f329c06a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d354af796c4a13e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f3289ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.253+0000 7f3289ffb640 1 -- 172.21.15.5:0/876899475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0b3070 con 0x5594186d2af0 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f3289ffb640 1 -- 172.21.15.5:0/876899475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f329c0b4260 con 0x5594186d2af0 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f32ab40a740 1 -- 172.21.15.5:0/876899475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3298004730 con 0x5594186d2af0 2024-09-11T22:25:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f3289ffb640 1 -- 172.21.15.5:0/876899475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f329c020040 con 0x5594186d2af0 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f32ab40a740 10 client.16572 fetch_fsmap learned FSMap version 34 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f32ab40a740 1 -- 172.21.15.5:0/876899475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559418758bf0 con 0x5594186d2af0 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f3289ffb640 1 -- 172.21.15.5:0/876899475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3298004730 con 0x5594186d2af0 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f3289ffb640 1 -- 172.21.15.5:0/876899475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f329c0cc830 con 0x5594186d2af0 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f32ab40a740 10 client.16572 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f32ab40a740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f32ab40a740 20 client.16572 populate_metadata read hostname 'smithi005' 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f32ab40a740 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594187593b0 0x55941875b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f32ab40a740 4 client.16572 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f32ab40a740 1 -- 172.21.15.5:0/876899475 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55941863ed70 con 0x5594187593b0 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.254+0000 7f32a5b2c640 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594187593b0 0x55941875b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.255+0000 7f3289ffb640 10 client.16572 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.255+0000 7f32a5b2c640 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594187593b0 0x55941875b7a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b6f2bce71af19c server_cookie=20d286d2b7d95550 in_seq=0 out_seq=0 2024-09-11T22:25:59.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259637 14.21 MB/sec execute 17 sec latency 15.581 ms 2024-09-11T22:25:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f3289ffb640 1 -- 172.21.15.5:0/876899475 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55941863ed70 con 0x5594187593b0 2024-09-11T22:25:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f3289ffb640 10 client.16572 handle_command_reply: tid=0 2024-09-11T22:25:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f32ab40a740 1 client.16572 shutdown 2024-09-11T22:25:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f32ab40a740 1 -- 172.21.15.5:0/876899475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186d2af0 msgr2=0x5594186e1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f32ab40a740 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186d2af0 0x5594186e1940 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f329c09b080 tx=0x7f329c06a2b0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f32ab40a740 1 -- 172.21.15.5:0/876899475 shutdown_connections 2024-09-11T22:25:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f32ab40a740 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594187593b0 0x55941875b7a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f32ab40a740 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594186ddcb0 0x5594186e23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f3289ffb640 0 client.16572 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:25:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f32ab40a740 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594186d3410 0x5594186e1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f32ab40a740 1 --2- 172.21.15.5:0/876899475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594186d2af0 0x5594186e1940 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.425+0000 7f32ab40a740 1 -- 172.21.15.5:0/876899475 >> 172.21.15.5:0/876899475 conn(0x5594186cf060 msgr2=0x5594186ce1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32ab40a740 1 -- 172.21.15.5:0/876899475 shutdown_connections 2024-09-11T22:25:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32ab40a740 1 -- 172.21.15.5:0/876899475 wait complete. 2024-09-11T22:25:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32ab40a740 20 client.16572 trim_cache size 0 max 16384 2024-09-11T22:25:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:25:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:25:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:25:59.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:25:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 -- 172.21.15.5:0/2823701812 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f328c0a2e20 msgr2=0x7f328c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f328c0a2e20 0x7f328c0a52e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3298005fe0 tx=0x7f3298005c50 comp rx=0 tx=0).stop 2024-09-11T22:25:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 -- 172.21.15.5:0/2823701812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a0102cc0 msgr2=0x7f32a0198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:25:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a0102cc0 0x7f32a0198340 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f329c00d770 tx=0x7f329c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:25:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 -- 172.21.15.5:0/2823701812 shutdown_connections 2024-09-11T22:25:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f328c0a2e20 0x7f328c0a52e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32a010c110 0x7f32a019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32a01035e0 0x7f32a0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 --2- 172.21.15.5:0/2823701812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32a0102cc0 0x7f32a0198340 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:25:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 -- 172.21.15.5:0/2823701812 >> 172.21.15.5:0/2823701812 conn(0x7f32a00fc6b0 msgr2=0x7f32a00fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:25:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 -- 172.21.15.5:0/2823701812 shutdown_connections 2024-09-11T22:25:59.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:25:59.426+0000 7f32a8591640 1 -- 172.21.15.5:0/2823701812 wait complete. 2024-09-11T22:25:59.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:25:59 smithi005 ceph-mon[30943]: pgmap v117: 97 pgs: 97 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-09-11T22:25:59.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:25:59 smithi159 ceph-mon[35534]: pgmap v117: 97 pgs: 97 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-09-11T22:25:59.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:25:59 smithi172 ceph-mon[35461]: pgmap v117: 97 pgs: 97 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.4 MiB/s wr, 402 op/s 2024-09-11T22:26:00.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 262531 14.39 MB/sec execute 18 sec latency 65.774 ms 2024-09-11T22:26:01.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 263695 13.83 MB/sec execute 19 sec latency 40.489 ms 2024-09-11T22:26:01.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:01 smithi005 ceph-mon[30943]: pgmap v118: 97 pgs: 97 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-09-11T22:26:01.578 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:01 smithi159 ceph-mon[35534]: pgmap v118: 97 pgs: 97 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-09-11T22:26:01.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:01 smithi172 ceph-mon[35461]: pgmap v118: 97 pgs: 97 active+clean; 60 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 209 op/s 2024-09-11T22:26:02.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266444 13.99 MB/sec execute 20 sec latency 12.934 ms 2024-09-11T22:26:03.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 269031 13.92 MB/sec execute 21 sec latency 89.820 ms 2024-09-11T22:26:03.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:03 smithi005 ceph-mon[30943]: pgmap v119: 97 pgs: 97 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.8 MiB/s wr, 458 op/s 2024-09-11T22:26:03.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:03 smithi159 ceph-mon[35534]: pgmap v119: 97 pgs: 97 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.8 MiB/s wr, 458 op/s 2024-09-11T22:26:03.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:03 smithi172 ceph-mon[35461]: pgmap v119: 97 pgs: 97 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.8 MiB/s wr, 458 op/s 2024-09-11T22:26:04.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 271211 13.82 MB/sec execute 22 sec latency 11.500 ms 2024-09-11T22:26:05.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 273763 13.83 MB/sec execute 23 sec latency 12.563 ms 2024-09-11T22:26:05.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:05 smithi005 ceph-mon[30943]: pgmap v120: 97 pgs: 97 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 351 op/s 2024-09-11T22:26:05.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:05 smithi159 ceph-mon[35534]: pgmap v120: 97 pgs: 97 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 351 op/s 2024-09-11T22:26:05.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:05 smithi172 ceph-mon[35461]: pgmap v120: 97 pgs: 97 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 351 op/s 2024-09-11T22:26:06.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 276526 13.85 MB/sec execute 24 sec latency 48.995 ms 2024-09-11T22:26:07.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 279095 13.77 MB/sec execute 25 sec latency 13.815 ms 2024-09-11T22:26:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:07 smithi005 ceph-mon[30943]: pgmap v121: 97 pgs: 97 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 351 op/s 2024-09-11T22:26:07.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:07 smithi159 ceph-mon[35534]: pgmap v121: 97 pgs: 97 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 351 op/s 2024-09-11T22:26:07.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:07 smithi172 ceph-mon[35461]: pgmap v121: 97 pgs: 97 active+clean; 56 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 351 op/s 2024-09-11T22:26:08.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 281311 13.78 MB/sec execute 26 sec latency 11.068 ms 2024-09-11T22:26:09.312 INFO:tasks.workunit.client.0.smithi005.stdout: 1 284249 13.90 MB/sec execute 27 sec latency 13.206 ms 2024-09-11T22:26:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:09 smithi005 ceph-mon[30943]: pgmap v122: 97 pgs: 97 active+clean; 62 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.7 MiB/s wr, 480 op/s 2024-09-11T22:26:09.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:09 smithi159 ceph-mon[35534]: pgmap v122: 97 pgs: 97 active+clean; 62 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.7 MiB/s wr, 480 op/s 2024-09-11T22:26:09.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:09 smithi172 ceph-mon[35461]: pgmap v122: 97 pgs: 97 active+clean; 62 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.7 MiB/s wr, 480 op/s 2024-09-11T22:26:10.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287415 14.04 MB/sec execute 28 sec latency 10.425 ms 2024-09-11T22:26:11.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 289735 13.85 MB/sec execute 29 sec latency 56.144 ms 2024-09-11T22:26:11.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:11 smithi005 ceph-mon[30943]: pgmap v123: 97 pgs: 97 active+clean; 62 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 377 op/s 2024-09-11T22:26:11.578 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:11 smithi159 ceph-mon[35534]: pgmap v123: 97 pgs: 97 active+clean; 62 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 377 op/s 2024-09-11T22:26:11.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:11 smithi172 ceph-mon[35461]: pgmap v123: 97 pgs: 97 active+clean; 62 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 377 op/s 2024-09-11T22:26:12.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:26:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:12.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 291854 13.82 MB/sec execute 30 sec latency 11.399 ms 2024-09-11T22:26:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.517+0000 7fbae8265640 1 -- 172.21.15.5:0/2201964594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae01009f0 msgr2=0x7fbae0100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.517+0000 7fbae8265640 1 --2- 172.21.15.5:0/2201964594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae01009f0 0x7fbae0100e50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009cd0 tx=0x7fbadc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.517+0000 7fbae8265640 1 -- 172.21.15.5:0/2201964594 shutdown_connections 2024-09-11T22:26:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.517+0000 7fbae8265640 1 --2- 172.21.15.5:0/2201964594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbae0101390 0x7fbae010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.517+0000 7fbae8265640 1 --2- 172.21.15.5:0/2201964594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae01009f0 0x7fbae0100e50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.517+0000 7fbae8265640 1 --2- 172.21.15.5:0/2201964594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbae0106a00 0x7fbae0106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.517+0000 7fbae8265640 1 -- 172.21.15.5:0/2201964594 >> 172.21.15.5:0/2201964594 conn(0x7fbae00fc6b0 msgr2=0x7fbae00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.517+0000 7fbae8265640 1 -- 172.21.15.5:0/2201964594 shutdown_connections 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.517+0000 7fbae8265640 1 -- 172.21.15.5:0/2201964594 wait complete. 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae8265640 1 Processor -- start 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae8265640 1 -- start start 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae8265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae01009f0 0x7fbae019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae8265640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbae0101390 0x7fbae019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae8265640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbae0106a00 0x7fbae019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae8265640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0111ef0 con 0x7fbae01009f0 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae8265640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0111d70 con 0x7fbae0101390 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae8265640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0112070 con 0x7fbae0106a00 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae6001640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae01009f0 0x7fbae019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae6001640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae01009f0 0x7fbae019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37670/0 (socket says 172.21.15.5:37670) 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae6001640 1 -- 172.21.15.5:0/428996339 learned_addr learned my addr 172.21.15.5:0/428996339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae5800640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbae0101390 0x7fbae019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.518+0000 7fbae6802640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbae0106a00 0x7fbae019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbae6001640 1 -- 172.21.15.5:0/428996339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbae0106a00 msgr2=0x7fbae019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbae6001640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbae0106a00 0x7fbae019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbae6001640 1 -- 172.21.15.5:0/428996339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbae0101390 msgr2=0x7fbae019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbae6001640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbae0101390 0x7fbae019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbae6001640 1 -- 172.21.15.5:0/428996339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae019f5a0 con 0x7fbae01009f0 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbae6001640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae01009f0 0x7fbae019a5b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbad400d950 tx=0x7fbad400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbad2ffd640 1 -- 172.21.15.5:0/428996339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad4017070 con 0x7fbae01009f0 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbad2ffd640 1 -- 172.21.15.5:0/428996339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad4009c30 con 0x7fbae01009f0 2024-09-11T22:26:12.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbae8265640 1 -- 172.21.15.5:0/428996339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae019f890 con 0x7fbae01009f0 2024-09-11T22:26:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbad2ffd640 1 -- 172.21.15.5:0/428996339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad400f6b0 con 0x7fbae01009f0 2024-09-11T22:26:12.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.519+0000 7fbae8265640 1 -- 172.21.15.5:0/428996339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae01a7120 con 0x7fbae01009f0 2024-09-11T22:26:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.521+0000 7fbad2ffd640 1 -- 172.21.15.5:0/428996339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbad40109d0 con 0x7fbae01009f0 2024-09-11T22:26:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.521+0000 7fbaeb004740 1 Processor -- start 2024-09-11T22:26:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbaeb004740 1 -- start start 2024-09-11T22:26:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a47ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a52770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbaeb004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5653099ac2f0 con 0x565309a52cb0 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbaeb004740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565309954270 con 0x565309a48410 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbaeb004740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5653099a82b0 con 0x565309a47af0 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbae6001640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a52770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbae6802640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a47ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbae6001640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a52770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36604/0 (socket says 172.21.15.5:36604) 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbae5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbae6001640 1 -- 172.21.15.5:0/3613128075 learned_addr learned my addr 172.21.15.5:0/3613128075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.522+0000 7fbae6802640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a47ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45044/0 (socket says 172.21.15.5:45044) 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbae5800640 1 -- 172.21.15.5:0/3613128075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 msgr2=0x565309a47ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbae5800640 1 --2- 172.21.15.5:0/3613128075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a47ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbae5800640 1 -- 172.21.15.5:0/3613128075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 msgr2=0x565309a52770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbae5800640 1 --2- 172.21.15.5:0/3613128075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a52770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbae5800640 1 -- 172.21.15.5:0/3613128075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565309953340 con 0x565309a52cb0 2024-09-11T22:26:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbad2ffd640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbacc0a2e20 0x7fbacc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbae6802640 1 --2- 172.21.15.5:0/3613128075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a47ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbae6001640 1 --2- 172.21.15.5:0/3613128075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a52770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbad2ffd640 1 -- 172.21.15.5:0/428996339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fbad40cef60 con 0x7fbae01009f0 2024-09-11T22:26:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbae6802640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbacc0a2e20 0x7fbacc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbae5800640 1 --2- 172.21.15.5:0/3613128075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a550a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbadc005e30 tx=0x7fbadc005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b59c4cab7a8dc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbacbfff640 1 -- 172.21.15.5:0/3613128075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc048070 con 0x565309a52cb0 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbacbfff640 1 -- 172.21.15.5:0/3613128075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbadc036070 con 0x565309a52cb0 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.523+0000 7fbacbfff640 1 -- 172.21.15.5:0/3613128075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004500 con 0x565309a52cb0 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 -- 172.21.15.5:0/3613128075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 msgr2=0x565309a550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 --2- 172.21.15.5:0/3613128075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a550a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbadc005e30 tx=0x7fbadc005fa0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbae6802640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbacc0a2e20 0x7fbacc0a52e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbad800b8f0 tx=0x7fbad800b2e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 -- 172.21.15.5:0/3613128075 shutdown_connections 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 --2- 172.21.15.5:0/3613128075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a550a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 --2- 172.21.15.5:0/3613128075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a52770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 --2- 172.21.15.5:0/3613128075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a47ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 -- 172.21.15.5:0/3613128075 >> 172.21.15.5:0/3613128075 conn(0x565309a44060 msgr2=0x565309a44440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:12.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 -- 172.21.15.5:0/3613128075 shutdown_connections 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 -- 172.21.15.5:0/3613128075 wait complete. 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 -- start start 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.524+0000 7fbaeb004740 1 Processor -- start 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a50600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbaeb004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565309a55880 con 0x565309a52cb0 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbaeb004740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565309935290 con 0x565309a48410 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbaeb004740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565309a55a00 con 0x565309a47af0 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbaeb004740 10 client.0 _ll_register_callbacks cb 0x565309966140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6802640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a50600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6802640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a50600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37692/0 (socket says 172.21.15.5:37692) 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6802640 1 -- 172.21.15.5:0/472175596 learned_addr learned my addr 172.21.15.5:0/472175596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6001640 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae5800640 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6802640 1 -- 172.21.15.5:0/472175596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 msgr2=0x565309a4fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6802640 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a4fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6802640 1 -- 172.21.15.5:0/472175596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 msgr2=0x565309a500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6802640 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6802640 1 -- 172.21.15.5:0/472175596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653096bf210 con 0x565309a52cb0 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6001640 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a4fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.525+0000 7fbae6802640 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a50600 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbad8003d10 tx=0x7fbad8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb08406ba95d64cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbac97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbac97fa640 1 -- 172.21.15.5:0/472175596 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8028440 con 0x565309a52cb0 2024-09-11T22:26:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbac97fa640 1 -- 172.21.15.5:0/472175596 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fbad8011440 con 0x565309a52cb0 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbaeb004740 1 -- 172.21.15.5:0/472175596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbadc036070 con 0x565309a52cb0 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbac97fa640 1 -- 172.21.15.5:0/472175596 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbad8011640 con 0x565309a52cb0 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbaeb004740 10 client.16584 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbaeb004740 1 -- 172.21.15.5:0/472175596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56530978b620 con 0x565309a52cb0 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbac97fa640 1 -- 172.21.15.5:0/472175596 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbadc036070 con 0x565309a52cb0 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbac97fa640 1 -- 172.21.15.5:0/472175596 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fbad802e030 con 0x565309a52cb0 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbaeb004740 10 client.16584 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbaeb004740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbaeb004740 20 client.16584 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbaeb004740 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565309a62ca0 0x565309a650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbaeb004740 4 client.16584 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.526+0000 7fbaeb004740 1 -- 172.21.15.5:0/472175596 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653099b9a20 con 0x565309a62ca0 2024-09-11T22:26:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.527+0000 7fbae6001640 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565309a62ca0 0x565309a650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.527+0000 7fbae6001640 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565309a62ca0 0x565309a650d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81b3000496c29f62 server_cookie=f07e1b4e0c72e64 in_seq=0 out_seq=0 2024-09-11T22:26:12.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.527+0000 7fbac97fa640 10 client.16584 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbac97fa640 1 -- 172.21.15.5:0/472175596 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653099b9a20 con 0x565309a62ca0 2024-09-11T22:26:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbac97fa640 10 client.16584 handle_command_reply: tid=0 2024-09-11T22:26:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 client.16584 shutdown 2024-09-11T22:26:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 -- 172.21.15.5:0/472175596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 msgr2=0x565309a50600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a50600 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fbad8003d10 tx=0x7fbad8004730 comp rx=0 tx=0).stop 2024-09-11T22:26:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 -- 172.21.15.5:0/472175596 shutdown_connections 2024-09-11T22:26:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565309a62ca0 0x565309a650d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a52cb0 0x565309a50600 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 --2- 172.21.15.5:0/472175596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a47af0 0x565309a4fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbac97fa640 0 client.16584 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 -- 172.21.15.5:0/472175596 >> 172.21.15.5:0/472175596 conn(0x565309a44060 msgr2=0x565309a49d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 -- 172.21.15.5:0/472175596 shutdown_connections 2024-09-11T22:26:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 1 -- 172.21.15.5:0/472175596 wait complete. 2024-09-11T22:26:12.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.529+0000 7fbaeb004740 20 client.16584 trim_cache size 0 max 16384 2024-09-11T22:26:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbaeb004740 1 Processor -- start 2024-09-11T22:26:12.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbaeb004740 1 -- start start 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a47af0 0x565309a502e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a6a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 0x565309a6af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbaeb004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01afb70 con 0x565309a47af0 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbaeb004740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01b16b0 con 0x565309a48410 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbaeb004740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01b0240 con 0x565309a52cb0 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbae6001640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a47af0 0x565309a502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbae6001640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a47af0 0x565309a502e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37702/0 (socket says 172.21.15.5:37702) 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbae6802640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 0x565309a6af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbae6001640 1 -- 172.21.15.5:0/1128164769 learned_addr learned my addr 172.21.15.5:0/1128164769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.546+0000 7fbae5800640 1 --2- 172.21.15.5:0/1128164769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a6a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.547+0000 7fbae6802640 1 -- 172.21.15.5:0/1128164769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 msgr2=0x565309a6a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.547+0000 7fbae6802640 1 --2- 172.21.15.5:0/1128164769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a6a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.547+0000 7fbae6802640 1 -- 172.21.15.5:0/1128164769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a47af0 msgr2=0x565309a502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.547+0000 7fbae6802640 1 --2- 172.21.15.5:0/1128164769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a47af0 0x565309a502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.547+0000 7fbae6802640 1 -- 172.21.15.5:0/1128164769 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565309a43e90 con 0x565309a52cb0 2024-09-11T22:26:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.547+0000 7fbae6001640 1 --2- 172.21.15.5:0/1128164769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a47af0 0x565309a502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.547+0000 7fbae6802640 1 --2- 172.21.15.5:0/1128164769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 0x565309a6af20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbad8028c00 tx=0x7fbad80058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=776ef679ce357005 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.547+0000 7fbacb7fe640 1 -- 172.21.15.5:0/1128164769 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8028440 con 0x565309a52cb0 2024-09-11T22:26:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.547+0000 7fbacb7fe640 1 -- 172.21.15.5:0/1128164769 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad8005040 con 0x565309a52cb0 2024-09-11T22:26:12.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.547+0000 7fbacb7fe640 1 -- 172.21.15.5:0/1128164769 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8004b20 con 0x565309a52cb0 2024-09-11T22:26:12.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 -- 172.21.15.5:0/1128164769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 msgr2=0x565309a6af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 --2- 172.21.15.5:0/1128164769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 0x565309a6af20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbad8028c00 tx=0x7fbad80058f0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 -- 172.21.15.5:0/1128164769 shutdown_connections 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 --2- 172.21.15.5:0/1128164769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 0x565309a6af20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 --2- 172.21.15.5:0/1128164769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a48410 0x565309a6a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 --2- 172.21.15.5:0/1128164769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a47af0 0x565309a502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 -- 172.21.15.5:0/1128164769 >> 172.21.15.5:0/1128164769 conn(0x565309a44060 msgr2=0x565309a51c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 -- 172.21.15.5:0/1128164769 shutdown_connections 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 -- 172.21.15.5:0/1128164769 wait complete. 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 -- start start 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.548+0000 7fbaeb004740 1 Processor -- start 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a47af0 0x565309a5b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a48410 0x565309a4c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbaeb004740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 0x565309a4d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbaeb004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01b0fe0 con 0x565309a48410 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbaeb004740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01b0910 con 0x565309a47af0 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbaeb004740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01adb50 con 0x565309a52cb0 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbaeb004740 10 client.0 _ll_register_callbacks cb 0x7fbad800b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a48410 0x565309a4c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae6001640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a47af0 0x565309a5b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a48410 0x565309a4c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37714/0 (socket says 172.21.15.5:37714) 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae5800640 1 -- 172.21.15.5:0/3130326123 learned_addr learned my addr 172.21.15.5:0/3130326123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae6802640 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 0x565309a4d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae5800640 1 -- 172.21.15.5:0/3130326123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 msgr2=0x565309a4d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae5800640 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 0x565309a4d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae5800640 1 -- 172.21.15.5:0/3130326123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a47af0 msgr2=0x565309a5b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae5800640 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a47af0 0x565309a5b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae5800640 1 -- 172.21.15.5:0/3130326123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565309a451f0 con 0x565309a48410 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbae5800640 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a48410 0x565309a4c9d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbadc006010 tx=0x7fbadc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1455492185139eb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.549+0000 7fbac97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbac97fa640 1 -- 172.21.15.5:0/3130326123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc052070 con 0x565309a48410 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbac97fa640 1 -- 172.21.15.5:0/3130326123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fbadc004440 con 0x565309a48410 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbaeb004740 1 -- 172.21.15.5:0/3130326123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbad8005040 con 0x565309a48410 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbac97fa640 1 -- 172.21.15.5:0/3130326123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbadc05e050 con 0x565309a48410 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbaeb004740 10 client.16590 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbaeb004740 1 -- 172.21.15.5:0/3130326123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565309a44cf0 con 0x565309a48410 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbac97fa640 1 -- 172.21.15.5:0/3130326123 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad8005040 con 0x565309a48410 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbac97fa640 1 -- 172.21.15.5:0/3130326123 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fbadc042030 con 0x565309a48410 2024-09-11T22:26:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbaeb004740 10 client.16590 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbaeb004740 10 client.16590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbaeb004740 20 client.16590 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbaeb004740 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565309acdac0 0x565309acfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbaeb004740 4 client.16590 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:26:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbaeb004740 1 -- 172.21.15.5:0/3130326123 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653099b3d70 con 0x565309acdac0 2024-09-11T22:26:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.550+0000 7fbae6001640 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565309acdac0 0x565309acfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.551+0000 7fbac97fa640 10 client.16590 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:12.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:12.551+0000 7fbae6001640 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565309acdac0 0x565309acfeb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29d6bbc6fa29cabd server_cookie=7618e320df6d4bec in_seq=0 out_seq=0 2024-09-11T22:26:12.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 -- 172.21.15.5:0/785887802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f225c102570 msgr2=0x7f225c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 --2- 172.21.15.5:0/785887802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f225c102570 0x7f225c1029b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2248009900 tx=0x7f224802a480 comp rx=0 tx=0).stop 2024-09-11T22:26:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 -- 172.21.15.5:0/785887802 shutdown_connections 2024-09-11T22:26:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 --2- 172.21.15.5:0/785887802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f225c102ef0 0x7f225c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 --2- 172.21.15.5:0/785887802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f225c102570 0x7f225c1029b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 --2- 172.21.15.5:0/785887802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f225c100e20 0x7f225c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 -- 172.21.15.5:0/785887802 >> 172.21.15.5:0/785887802 conn(0x7f225c0fc7d0 msgr2=0x7f225c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 -- 172.21.15.5:0/785887802 shutdown_connections 2024-09-11T22:26:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 -- 172.21.15.5:0/785887802 wait complete. 2024-09-11T22:26:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 Processor -- start 2024-09-11T22:26:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 -- start start 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f225c100e20 0x7f225c199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.801+0000 7f2262fed640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f225c102570 0x7f225c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2262fed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f225c102ef0 0x7f225c1a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2262fed640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f225c110f80 con 0x7f225c100e20 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2262fed640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f225c110e00 con 0x7f225c102570 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2262fed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f225c111100 con 0x7f225c102ef0 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2260d89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f225c100e20 0x7f225c199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2260d89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f225c100e20 0x7f225c199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37718/0 (socket says 172.21.15.5:37718) 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f225bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f225c102570 0x7f225c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2260d89640 1 -- 172.21.15.5:0/608389113 learned_addr learned my addr 172.21.15.5:0/608389113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f226158a640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f225c102ef0 0x7f225c1a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2260d89640 1 -- 172.21.15.5:0/608389113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f225c102ef0 msgr2=0x7f225c1a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2260d89640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f225c102ef0 0x7f225c1a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2260d89640 1 -- 172.21.15.5:0/608389113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f225c102570 msgr2=0x7f225c1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2260d89640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f225c102570 0x7f225c1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2260d89640 1 -- 172.21.15.5:0/608389113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c1a4950 con 0x7f225c100e20 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f225bfff640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f225c102570 0x7f225c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2260d89640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f225c100e20 0x7f225c199390 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f225000e9f0 tx=0x7f225000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2259ffb640 1 -- 172.21.15.5:0/608389113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225000cd80 con 0x7f225c100e20 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.802+0000 7f2259ffb640 1 -- 172.21.15.5:0/608389113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2250005500 con 0x7f225c100e20 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.803+0000 7f2262fed640 1 -- 172.21.15.5:0/608389113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c1a4b20 con 0x7f225c100e20 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.803+0000 7f2259ffb640 1 -- 172.21.15.5:0/608389113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250010690 con 0x7f225c100e20 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.803+0000 7f2262fed640 1 -- 172.21.15.5:0/608389113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c1a4fe0 con 0x7f225c100e20 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.805+0000 7f2259ffb640 1 -- 172.21.15.5:0/608389113 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f22500046c0 con 0x7f225c100e20 2024-09-11T22:26:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f2259ffb640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f224c0a2d70 0x7f224c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f2259ffb640 1 -- 172.21.15.5:0/608389113 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f22500cf870 con 0x7f225c100e20 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f225bfff640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f224c0a2d70 0x7f224c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f22653c6740 1 Processor -- start 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f22653c6740 1 -- start start 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f22653c6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229cf8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f22653c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d03140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f22653c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d05a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f22653c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565229c5e2e0 con 0x565229cf8dc0 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f22653c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565229c05ce0 con 0x565229d03680 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.806+0000 7f22653c6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565229c5a2a0 con 0x565229cf84a0 2024-09-11T22:26:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f226158a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d05a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f226158a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d05a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36656/0 (socket says 172.21.15.5:36656) 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f225bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d03140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f226158a640 1 -- 172.21.15.5:0/2891741686 learned_addr learned my addr 172.21.15.5:0/2891741686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f2260d89640 1 --2- 172.21.15.5:0/2891741686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229cf8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f225bfff640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f224c0a2d70 0x7f224c0a5230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2248009230 tx=0x7f22480058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f226158a640 1 -- 172.21.15.5:0/2891741686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 msgr2=0x565229cf8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f226158a640 1 --2- 172.21.15.5:0/2891741686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229cf8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f226158a640 1 -- 172.21.15.5:0/2891741686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 msgr2=0x565229d03140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f226158a640 1 --2- 172.21.15.5:0/2891741686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d03140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f226158a640 1 -- 172.21.15.5:0/2891741686 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565229bea440 con 0x565229d03680 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f225bfff640 1 --2- 172.21.15.5:0/2891741686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d03140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f226158a640 1 --2- 172.21.15.5:0/2891741686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d05a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2254009f50 tx=0x7f22540118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5816802bfa18f1ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f2246ffd640 1 -- 172.21.15.5:0/2891741686 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254004d20 con 0x565229d03680 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f2246ffd640 1 -- 172.21.15.5:0/2891741686 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2254004050 con 0x565229d03680 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.807+0000 7f2246ffd640 1 -- 172.21.15.5:0/2891741686 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254005280 con 0x565229d03680 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.808+0000 7f22653c6740 1 -- 172.21.15.5:0/2891741686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 msgr2=0x565229d05a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.808+0000 7f22653c6740 1 --2- 172.21.15.5:0/2891741686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d05a70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2254009f50 tx=0x7f22540118d0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.808+0000 7f22653c6740 1 -- 172.21.15.5:0/2891741686 shutdown_connections 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.808+0000 7f22653c6740 1 --2- 172.21.15.5:0/2891741686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d05a70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.808+0000 7f22653c6740 1 --2- 172.21.15.5:0/2891741686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d03140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.808+0000 7f22653c6740 1 --2- 172.21.15.5:0/2891741686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229cf8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.808+0000 7f22653c6740 1 -- 172.21.15.5:0/2891741686 >> 172.21.15.5:0/2891741686 conn(0x565229cf4a30 msgr2=0x565229cf4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.808+0000 7f22653c6740 1 -- 172.21.15.5:0/2891741686 shutdown_connections 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.808+0000 7f22653c6740 1 -- 172.21.15.5:0/2891741686 wait complete. 2024-09-11T22:26:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.808+0000 7f22653c6740 1 -- start start 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f22653c6740 1 Processor -- start 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f22653c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229cf84a0 0x565229d02ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f22653c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d00180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f22653c6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229d03680 0x565229d006c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f22653c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565229d06250 con 0x565229cf8dc0 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f22653c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565229c19050 con 0x565229cf84a0 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f22653c6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x565229d063d0 con 0x565229d03680 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f22653c6740 10 client.0 _ll_register_callbacks cb 0x565229c59160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f225bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d00180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f2260d89640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229cf84a0 0x565229d02ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f225bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d00180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37736/0 (socket says 172.21.15.5:37736) 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f225bfff640 1 -- 172.21.15.5:0/140877840 learned_addr learned my addr 172.21.15.5:0/140877840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f2260d89640 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229cf84a0 0x565229d02ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36668/0 (socket says 172.21.15.5:36668) 2024-09-11T22:26:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.809+0000 7f226158a640 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229d03680 0x565229d006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f2260d89640 1 -- 172.21.15.5:0/140877840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229d03680 msgr2=0x565229d006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f2260d89640 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229d03680 0x565229d006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f2260d89640 1 -- 172.21.15.5:0/140877840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 msgr2=0x565229d00180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f2260d89640 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d00180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f2260d89640 1 -- 172.21.15.5:0/140877840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652299cf3c0 con 0x565229cf84a0 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f225bfff640 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d00180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f2260d89640 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229cf84a0 0x565229d02ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22500ccf20 tx=0x7f225009b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4a9ed7c8da81195 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f223bfff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f223bfff640 1 -- 172.21.15.5:0/140877840 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500b9060 con 0x565229cf84a0 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f223bfff640 1 -- 172.21.15.5:0/140877840 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f22500a46d0 con 0x565229cf84a0 2024-09-11T22:26:12.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f22653c6740 1 -- 172.21.15.5:0/140877840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2254004050 con 0x565229cf84a0 2024-09-11T22:26:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f223bfff640 1 -- 172.21.15.5:0/140877840 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f22500a00a0 con 0x565229cf84a0 2024-09-11T22:26:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f22653c6740 10 client.26108 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.810+0000 7f22653c6740 1 -- 172.21.15.5:0/140877840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565229d11b20 con 0x565229cf84a0 2024-09-11T22:26:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f223bfff640 1 -- 172.21.15.5:0/140877840 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2254004050 con 0x565229cf84a0 2024-09-11T22:26:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f223bfff640 1 -- 172.21.15.5:0/140877840 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f225009adb0 con 0x565229cf84a0 2024-09-11T22:26:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f22653c6740 10 client.26108 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f22653c6740 10 client.26108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f22653c6740 20 client.26108 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f22653c6740 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565229d12e70 0x565229d15260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f22653c6740 4 client.26108 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f22653c6740 1 -- 172.21.15.5:0/140877840 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565229c6b850 con 0x565229d12e70 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f225bfff640 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565229d12e70 0x565229d15260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f225bfff640 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565229d12e70 0x565229d15260 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ceebcbfa7aec9c server_cookie=fb4acc8a6a4dd36f in_seq=0 out_seq=0 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.811+0000 7f223bfff640 10 client.26108 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.812+0000 7f223bfff640 1 -- 172.21.15.5:0/140877840 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565229c6b850 con 0x565229d12e70 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.812+0000 7f223bfff640 10 client.26108 handle_command_reply: tid=0 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.812+0000 7f22653c6740 1 client.26108 shutdown 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 1 -- 172.21.15.5:0/140877840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229cf84a0 msgr2=0x565229d02ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229cf84a0 0x565229d02ed0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f22500ccf20 tx=0x7f225009b710 comp rx=0 tx=0).stop 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 1 -- 172.21.15.5:0/140877840 shutdown_connections 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565229d12e70 0x565229d15260 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229d03680 0x565229d006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d00180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f223bfff640 0 client.26108 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 1 --2- 172.21.15.5:0/140877840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229cf84a0 0x565229d02ed0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 1 -- 172.21.15.5:0/140877840 >> 172.21.15.5:0/140877840 conn(0x565229cf4a30 msgr2=0x565229cfa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 1 -- 172.21.15.5:0/140877840 shutdown_connections 2024-09-11T22:26:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 1 -- 172.21.15.5:0/140877840 wait complete. 2024-09-11T22:26:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.813+0000 7f22653c6740 20 client.26108 trim_cache size 0 max 16384 2024-09-11T22:26:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.842+0000 7f22653c6740 1 Processor -- start 2024-09-11T22:26:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.842+0000 7f22653c6740 1 -- start start 2024-09-11T22:26:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f22653c6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229cff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f22653c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d1ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f22653c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d1b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f22653c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f225c1ad970 con 0x565229cf8dc0 2024-09-11T22:26:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f22653c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f225c1af4b0 con 0x565229d03680 2024-09-11T22:26:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f22653c6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f225c1ae040 con 0x565229cf84a0 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f225bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d1ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f2260d89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229cff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f225bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d1ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37744/0 (socket says 172.21.15.5:37744) 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f225bfff640 1 -- 172.21.15.5:0/225197656 learned_addr learned my addr 172.21.15.5:0/225197656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f226158a640 1 --2- 172.21.15.5:0/225197656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d1b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f2260d89640 1 -- 172.21.15.5:0/225197656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 msgr2=0x565229d1b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f2260d89640 1 --2- 172.21.15.5:0/225197656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d1b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f2260d89640 1 -- 172.21.15.5:0/225197656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 msgr2=0x565229d1ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f2260d89640 1 --2- 172.21.15.5:0/225197656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d1ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f2260d89640 1 -- 172.21.15.5:0/225197656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565229cf44d0 con 0x565229cf84a0 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.843+0000 7f225bfff640 1 --2- 172.21.15.5:0/225197656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d1ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.844+0000 7f2260d89640 1 --2- 172.21.15.5:0/225197656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229cff0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f22500a4430 tx=0x7f2250004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c11f6c14e76b6e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.844+0000 7f22467fc640 1 -- 172.21.15.5:0/225197656 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500b9040 con 0x565229cf84a0 2024-09-11T22:26:12.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.844+0000 7f22467fc640 1 -- 172.21.15.5:0/225197656 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22500a46d0 con 0x565229cf84a0 2024-09-11T22:26:12.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.844+0000 7f22467fc640 1 -- 172.21.15.5:0/225197656 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225009abb0 con 0x565229cf84a0 2024-09-11T22:26:12.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.844+0000 7f22653c6740 1 -- 172.21.15.5:0/225197656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 msgr2=0x565229cff0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.844+0000 7f22653c6740 1 --2- 172.21.15.5:0/225197656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229cff0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f22500a4430 tx=0x7f2250004a40 comp rx=0 tx=0).stop 2024-09-11T22:26:12.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.845+0000 7f22653c6740 1 -- 172.21.15.5:0/225197656 shutdown_connections 2024-09-11T22:26:12.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.845+0000 7f22653c6740 1 --2- 172.21.15.5:0/225197656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d1b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.845+0000 7f22653c6740 1 --2- 172.21.15.5:0/225197656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d1ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.845+0000 7f22653c6740 1 --2- 172.21.15.5:0/225197656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229cff0c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.845+0000 7f22653c6740 1 -- 172.21.15.5:0/225197656 >> 172.21.15.5:0/225197656 conn(0x565229cf4a30 msgr2=0x565229cf3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.845+0000 7f22653c6740 1 -- 172.21.15.5:0/225197656 shutdown_connections 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.845+0000 7f22653c6740 1 -- 172.21.15.5:0/225197656 wait complete. 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.845+0000 7f22653c6740 1 -- start start 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.845+0000 7f22653c6740 1 Processor -- start 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f22653c6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229d09380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f22653c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f22653c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d09e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f22653c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f225c1aede0 con 0x565229cf8dc0 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f22653c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f225c1ae710 con 0x565229d03680 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f22653c6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f225c1ab950 con 0x565229cf84a0 2024-09-11T22:26:12.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f22653c6740 10 client.0 _ll_register_callbacks cb 0x7f225009bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f2260d89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229d09380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f225bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f2260d89640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229d09380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45112/0 (socket says 172.21.15.5:45112) 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f2260d89640 1 -- 172.21.15.5:0/327771063 learned_addr learned my addr 172.21.15.5:0/327771063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f226158a640 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d09e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f2260d89640 1 -- 172.21.15.5:0/327771063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 msgr2=0x565229d09e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f2260d89640 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d09e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f2260d89640 1 -- 172.21.15.5:0/327771063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 msgr2=0x565229d098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f2260d89640 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f2260d89640 1 -- 172.21.15.5:0/327771063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565229d7caf0 con 0x565229cf84a0 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f2260d89640 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229d09380 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f225000c6a0 tx=0x7f225009b370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0db6a5606da152c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:12.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.846+0000 7f223bfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f223bfff640 1 -- 172.21.15.5:0/327771063 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22500a1380 con 0x565229cf84a0 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f223bfff640 1 -- 172.21.15.5:0/327771063 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f2250097a40 con 0x565229cf84a0 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f22653c6740 1 -- 172.21.15.5:0/327771063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22500a46d0 con 0x565229cf84a0 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f223bfff640 1 -- 172.21.15.5:0/327771063 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f22500a4960 con 0x565229cf84a0 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f22653c6740 10 client.26050 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f22653c6740 1 -- 172.21.15.5:0/327771063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565229d7caf0 con 0x565229cf84a0 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f223bfff640 1 -- 172.21.15.5:0/327771063 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22500a46d0 con 0x565229cf84a0 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f223bfff640 1 -- 172.21.15.5:0/327771063 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f22500ad070 con 0x565229cf84a0 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f22653c6740 10 client.26050 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f22653c6740 10 client.26050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f22653c6740 20 client.26050 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:12.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f22653c6740 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565229d7e010 0x565229d80400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f22653c6740 4 client.26050 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f22653c6740 1 -- 172.21.15.5:0/327771063 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565229c65d60 con 0x565229d7e010 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.847+0000 7f225bfff640 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565229d7e010 0x565229d80400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f225bfff640 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565229d7e010 0x565229d80400 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38f47c38923d8f8d server_cookie=bca8d7beef37e8eb in_seq=0 out_seq=0 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f223bfff640 10 client.26050 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f223bfff640 1 -- 172.21.15.5:0/327771063 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x565229c65d60 con 0x565229d7e010 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f223bfff640 10 client.26050 handle_command_reply: tid=0 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f22653c6740 1 client.26050 shutdown 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f22653c6740 1 -- 172.21.15.5:0/327771063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 msgr2=0x565229d09380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f22653c6740 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229d09380 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f225000c6a0 tx=0x7f225009b370 comp rx=0 tx=0).stop 2024-09-11T22:26:12.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f22653c6740 1 -- 172.21.15.5:0/327771063 shutdown_connections 2024-09-11T22:26:12.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f22653c6740 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565229d7e010 0x565229d80400 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f22653c6740 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565229d03680 0x565229d09e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f22653c6740 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565229cf8dc0 0x565229d098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f223bfff640 0 client.26050 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:12.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f22653c6740 1 --2- 172.21.15.5:0/327771063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565229cf84a0 0x565229d09380 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.848+0000 7f22653c6740 1 -- 172.21.15.5:0/327771063 >> 172.21.15.5:0/327771063 conn(0x565229cf4a30 msgr2=0x565229d019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:12.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f22653c6740 1 -- 172.21.15.5:0/327771063 shutdown_connections 2024-09-11T22:26:12.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f22653c6740 1 -- 172.21.15.5:0/327771063 wait complete. 2024-09-11T22:26:12.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f22653c6740 20 client.26050 trim_cache size 0 max 16384 2024-09-11T22:26:12.857 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:26:12.857 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:26:12.857 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:26:12.857 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:26:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f2262fed640 1 -- 172.21.15.5:0/608389113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f224c0a2d70 msgr2=0x7f224c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f2262fed640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f224c0a2d70 0x7f224c0a5230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2248009230 tx=0x7f22480058d0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f2262fed640 1 -- 172.21.15.5:0/608389113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f225c100e20 msgr2=0x7f225c199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f2262fed640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f225c100e20 0x7f225c199390 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f225000e9f0 tx=0x7f225000edf0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f2262fed640 1 -- 172.21.15.5:0/608389113 shutdown_connections 2024-09-11T22:26:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f2262fed640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f225c102ef0 0x7f225c1a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f2262fed640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f224c0a2d70 0x7f224c0a5230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f2262fed640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f225c102570 0x7f225c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f2262fed640 1 --2- 172.21.15.5:0/608389113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f225c100e20 0x7f225c199390 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.849+0000 7f2262fed640 1 -- 172.21.15.5:0/608389113 >> 172.21.15.5:0/608389113 conn(0x7f225c0fc7d0 msgr2=0x7f225c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.850+0000 7f2262fed640 1 -- 172.21.15.5:0/608389113 shutdown_connections 2024-09-11T22:26:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:12.850+0000 7f2262fed640 1 -- 172.21.15.5:0/608389113 wait complete. 2024-09-11T22:26:13.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294723 13.95 MB/sec execute 31 sec latency 46.615 ms 2024-09-11T22:26:13.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:13 smithi005 ceph-mon[30943]: pgmap v124: 97 pgs: 97 active+clean; 70 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-09-11T22:26:13.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:13 smithi159 ceph-mon[35534]: pgmap v124: 97 pgs: 97 active+clean; 70 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-09-11T22:26:13.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:13 smithi172 ceph-mon[35461]: pgmap v124: 97 pgs: 97 active+clean; 70 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-09-11T22:26:13.787 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:26:13.787 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:26:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.850+0000 7fbac97fa640 1 -- 172.21.15.5:0/3130326123 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653099b3d70 con 0x565309acdac0 2024-09-11T22:26:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.850+0000 7fbac97fa640 10 client.16590 handle_command_reply: tid=0 2024-09-11T22:26:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.850+0000 7fbaeb004740 1 client.16590 shutdown 2024-09-11T22:26:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.850+0000 7fbaeb004740 1 -- 172.21.15.5:0/3130326123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a48410 msgr2=0x565309a4c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.850+0000 7fbaeb004740 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a48410 0x565309a4c9d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbadc006010 tx=0x7fbadc009a20 comp rx=0 tx=0).stop 2024-09-11T22:26:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.850+0000 7fbaeb004740 1 -- 172.21.15.5:0/3130326123 shutdown_connections 2024-09-11T22:26:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.850+0000 7fbaeb004740 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x565309acdac0 0x565309acfeb0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.851+0000 7fbaeb004740 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565309a52cb0 0x565309a4d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.851+0000 7fbaeb004740 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565309a48410 0x565309a4c9d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.851+0000 7fbaeb004740 1 --2- 172.21.15.5:0/3130326123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565309a47af0 0x565309a5b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.851+0000 7fbaeb004740 1 -- 172.21.15.5:0/3130326123 >> 172.21.15.5:0/3130326123 conn(0x565309a44060 msgr2=0x565309a49550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.851+0000 7fbac97fa640 0 client.16590 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.851+0000 7fbaeb004740 1 -- 172.21.15.5:0/3130326123 shutdown_connections 2024-09-11T22:26:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.851+0000 7fbaeb004740 1 -- 172.21.15.5:0/3130326123 wait complete. 2024-09-11T22:26:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.851+0000 7fbaeb004740 20 client.16590 trim_cache size 0 max 16384 2024-09-11T22:26:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:26:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:26:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:26:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:26:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.852+0000 7fbae8265640 1 -- 172.21.15.5:0/428996339 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbacc0a2e20 msgr2=0x7fbacc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.852+0000 7fbae8265640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbacc0a2e20 0x7fbacc0a52e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbad800b8f0 tx=0x7fbad800b2e0 comp rx=0 tx=0).stop 2024-09-11T22:26:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.852+0000 7fbae8265640 1 -- 172.21.15.5:0/428996339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae01009f0 msgr2=0x7fbae019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.852+0000 7fbae8265640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae01009f0 0x7fbae019a5b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbad400d950 tx=0x7fbad400de10 comp rx=0 tx=0).stop 2024-09-11T22:26:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.853+0000 7fbae8265640 1 -- 172.21.15.5:0/428996339 shutdown_connections 2024-09-11T22:26:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.853+0000 7fbae8265640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbae0106a00 0x7fbae019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.853+0000 7fbae8265640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbacc0a2e20 0x7fbacc0a52e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.853+0000 7fbae8265640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbae0101390 0x7fbae019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.853+0000 7fbae8265640 1 --2- 172.21.15.5:0/428996339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbae01009f0 0x7fbae019a5b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.853+0000 7fbae8265640 1 -- 172.21.15.5:0/428996339 >> 172.21.15.5:0/428996339 conn(0x7fbae00fc6b0 msgr2=0x7fbae010c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.854+0000 7fbae8265640 1 -- 172.21.15.5:0/428996339 shutdown_connections 2024-09-11T22:26:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:13.854+0000 7fbae8265640 1 -- 172.21.15.5:0/428996339 wait complete. 2024-09-11T22:26:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:14.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 297360 13.91 MB/sec execute 32 sec latency 13.458 ms 2024-09-11T22:26:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.644+0000 7f088db46640 1 -- 172.21.15.5:0/2581649049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888103740 msgr2=0x7f0888107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.644+0000 7f088db46640 1 --2- 172.21.15.5:0/2581649049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888103740 0x7f0888107b80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f087c009930 tx=0x7f087c02a420 comp rx=0 tx=0).stop 2024-09-11T22:26:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.645+0000 7f088db46640 1 -- 172.21.15.5:0/2581649049 shutdown_connections 2024-09-11T22:26:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.645+0000 7f088db46640 1 --2- 172.21.15.5:0/2581649049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0888108290 0x7f088810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.645+0000 7f088db46640 1 --2- 172.21.15.5:0/2581649049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888103740 0x7f0888107b80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.645+0000 7f088db46640 1 --2- 172.21.15.5:0/2581649049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0888102d90 0x7f0888103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.645+0000 7f088db46640 1 -- 172.21.15.5:0/2581649049 >> 172.21.15.5:0/2581649049 conn(0x7f08880fc7d0 msgr2=0x7f08880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.645+0000 7f088db46640 1 -- 172.21.15.5:0/2581649049 shutdown_connections 2024-09-11T22:26:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.645+0000 7f088db46640 1 -- 172.21.15.5:0/2581649049 wait complete. 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.645+0000 7f088db46640 1 Processor -- start 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.645+0000 7f088db46640 1 -- start start 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f088db46640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0888102d90 0x7f08881a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f088db46640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888103740 0x7f08881a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f088db46640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0888108290 0x7f088819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f088db46640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0888112050 con 0x7f0888103740 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f088db46640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0888111ed0 con 0x7f0888108290 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f088db46640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f08881121d0 con 0x7f0888102d90 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f0886ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888103740 0x7f08881a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f0886ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888103740 0x7f08881a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37754/0 (socket says 172.21.15.5:37754) 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f0886ffd640 1 -- 172.21.15.5:0/1743703741 learned_addr learned my addr 172.21.15.5:0/1743703741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f08877fe640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0888102d90 0x7f08881a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f0887fff640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0888108290 0x7f088819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f0886ffd640 1 -- 172.21.15.5:0/1743703741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0888102d90 msgr2=0x7f08881a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f0886ffd640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0888102d90 0x7f08881a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f0886ffd640 1 -- 172.21.15.5:0/1743703741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0888108290 msgr2=0x7f088819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f0886ffd640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0888108290 0x7f088819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f0886ffd640 1 -- 172.21.15.5:0/1743703741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088819aeb0 con 0x7f0888103740 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.646+0000 7f0886ffd640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888103740 0x7f08881a0ab0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f087c005940 tx=0x7f087c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.647+0000 7f0884ff9640 1 -- 172.21.15.5:0/1743703741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c03f030 con 0x7f0888103740 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.647+0000 7f088db46640 1 -- 172.21.15.5:0/1743703741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088819b140 con 0x7f0888103740 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.647+0000 7f0884ff9640 1 -- 172.21.15.5:0/1743703741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f087c0044c0 con 0x7f0888103740 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.647+0000 7f0884ff9640 1 -- 172.21.15.5:0/1743703741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c035070 con 0x7f0888103740 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.647+0000 7f088db46640 1 -- 172.21.15.5:0/1743703741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08881a7320 con 0x7f0888103740 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.648+0000 7f088db46640 1 -- 172.21.15.5:0/1743703741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f088806a7f0 con 0x7f0888103740 2024-09-11T22:26:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.649+0000 7f0884ff9640 1 -- 172.21.15.5:0/1743703741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f087c042030 con 0x7f0888103740 2024-09-11T22:26:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.651+0000 7f0884ff9640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f08700a2cf0 0x7f08700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.651+0000 7f08877fe640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f08700a2cf0 0x7f08700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.651+0000 7f0884ff9640 1 -- 172.21.15.5:0/1743703741 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f087c0ef440 con 0x7f0888103740 2024-09-11T22:26:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.652+0000 7f08877fe640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f08700a2cf0 0x7f08700a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f08780095f0 tx=0x7f0878005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.656+0000 7f0884ff9640 1 -- 172.21.15.5:0/1743703741 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f087c0bc020 con 0x7f0888103740 2024-09-11T22:26:14.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.896+0000 7f088db46640 1 -- 172.21.15.5:0/1743703741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f088810c2b0 con 0x7f0888103740 2024-09-11T22:26:14.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.897+0000 7f0884ff9640 1 -- 172.21.15.5:0/1743703741 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11967 (secure 0 0 0) 0x7f087c0bb920 con 0x7f0888103740 2024-09-11T22:26:14.899 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:26:14.899 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-11T22:25:49:149525+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:25:48.784660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25970":{"gid":25970,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4169170564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4169170564},{"type":"v1","addr":"172.21.15.159:6833","nonce":4169170564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:26:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-11T22:26:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 -- 172.21.15.5:0/1743703741 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f08700a2cf0 msgr2=0x7f08700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f08700a2cf0 0x7f08700a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f08780095f0 tx=0x7f0878005f90 comp rx=0 tx=0).stop 2024-09-11T22:26:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 -- 172.21.15.5:0/1743703741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888103740 msgr2=0x7f08881a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888103740 0x7f08881a0ab0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f087c005940 tx=0x7f087c0040c0 comp rx=0 tx=0).stop 2024-09-11T22:26:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 -- 172.21.15.5:0/1743703741 shutdown_connections 2024-09-11T22:26:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f08700a2cf0 0x7f08700a51b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0888108290 0x7f088819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0888103740 0x7f08881a0ab0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 --2- 172.21.15.5:0/1743703741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0888102d90 0x7f08881a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 -- 172.21.15.5:0/1743703741 >> 172.21.15.5:0/1743703741 conn(0x7f08880fc7d0 msgr2=0x7f0888100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 -- 172.21.15.5:0/1743703741 shutdown_connections 2024-09-11T22:26:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:14.899+0000 7f088db46640 1 -- 172.21.15.5:0/1743703741 wait complete. 2024-09-11T22:26:15.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 299492 13.85 MB/sec execute 33 sec latency 39.969 ms 2024-09-11T22:26:15.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:15 smithi005 ceph-mon[30943]: pgmap v125: 97 pgs: 97 active+clean; 70 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 259 op/s 2024-09-11T22:26:15.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:15 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1743703741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:15 smithi159 ceph-mon[35534]: pgmap v125: 97 pgs: 97 active+clean; 70 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 259 op/s 2024-09-11T22:26:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:15 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1743703741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:15 smithi172 ceph-mon[35461]: pgmap v125: 97 pgs: 97 active+clean; 70 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 259 op/s 2024-09-11T22:26:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:15 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1743703741' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:15.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:26:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:16.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 301911 13.85 MB/sec execute 34 sec latency 52.705 ms 2024-09-11T22:26:16.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.446+0000 7fe153d67640 1 -- 172.21.15.5:0/80883012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe14c1027a0 msgr2=0x7fe14c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:16.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.446+0000 7fe153d67640 1 --2- 172.21.15.5:0/80883012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe14c1027a0 0x7fe14c1067f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009900 tx=0x7fe13c02a410 comp rx=0 tx=0).stop 2024-09-11T22:26:16.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.446+0000 7fe153d67640 1 -- 172.21.15.5:0/80883012 shutdown_connections 2024-09-11T22:26:16.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.446+0000 7fe153d67640 1 --2- 172.21.15.5:0/80883012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe14c106f20 0x7fe14c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.446+0000 7fe153d67640 1 --2- 172.21.15.5:0/80883012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe14c1027a0 0x7fe14c1067f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.446+0000 7fe153d67640 1 --2- 172.21.15.5:0/80883012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c101df0 0x7fe14c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.446+0000 7fe153d67640 1 -- 172.21.15.5:0/80883012 >> 172.21.15.5:0/80883012 conn(0x7fe14c0fd8b0 msgr2=0x7fe14c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.447+0000 7fe153d67640 1 -- 172.21.15.5:0/80883012 shutdown_connections 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.447+0000 7fe153d67640 1 -- 172.21.15.5:0/80883012 wait complete. 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.447+0000 7fe153d67640 1 Processor -- start 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.447+0000 7fe153d67640 1 -- start start 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.447+0000 7fe153d67640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c101df0 0x7fe14c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.447+0000 7fe153d67640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe14c1027a0 0x7fe14c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.447+0000 7fe153d67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe14c106f20 0x7fe14c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.447+0000 7fe153d67640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c111f80 con 0x7fe14c106f20 2024-09-11T22:26:16.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.447+0000 7fe153d67640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c111e00 con 0x7fe14c1027a0 2024-09-11T22:26:16.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.447+0000 7fe153d67640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c112100 con 0x7fe14c101df0 2024-09-11T22:26:16.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151b03640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c101df0 0x7fe14c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe152304640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe14c106f20 0x7fe14c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151302640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe14c1027a0 0x7fe14c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151b03640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c101df0 0x7fe14c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45142/0 (socket says 172.21.15.5:45142) 2024-09-11T22:26:16.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151b03640 1 -- 172.21.15.5:0/1260056259 learned_addr learned my addr 172.21.15.5:0/1260056259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:16.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151302640 1 -- 172.21.15.5:0/1260056259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c101df0 msgr2=0x7fe14c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:16.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151302640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c101df0 0x7fe14c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151302640 1 -- 172.21.15.5:0/1260056259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe14c106f20 msgr2=0x7fe14c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151302640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe14c106f20 0x7fe14c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151302640 1 -- 172.21.15.5:0/1260056259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c19d3d0 con 0x7fe14c1027a0 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151b03640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c101df0 0x7fe14c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe152304640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe14c106f20 0x7fe14c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe151302640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe14c1027a0 0x7fe14c198960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe13c005940 tx=0x7fe13c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe142ffd640 1 -- 172.21.15.5:0/1260056259 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c03f030 con 0x7fe14c1027a0 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe153d67640 1 -- 172.21.15.5:0/1260056259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c19d660 con 0x7fe14c1027a0 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe142ffd640 1 -- 172.21.15.5:0/1260056259 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c0044c0 con 0x7fe14c1027a0 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe142ffd640 1 -- 172.21.15.5:0/1260056259 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c035070 con 0x7fe14c1027a0 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.448+0000 7fe153d67640 1 -- 172.21.15.5:0/1260056259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c1a4fa0 con 0x7fe14c1027a0 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.451+0000 7fe142ffd640 1 -- 172.21.15.5:0/1260056259 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe13c042030 con 0x7fe14c1027a0 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.451+0000 7fe156140740 1 Processor -- start 2024-09-11T22:26:16.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.451+0000 7fe156140740 1 -- start start 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.451+0000 7fe142ffd640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1380a2d70 0x7fe1380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe156140740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fab8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe152304640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1380a2d70 0x7fe1380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe156140740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 0x56342fab8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe142ffd640 1 -- 172.21.15.5:0/1260056259 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe13c0ef6d0 con 0x7fe14c1027a0 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe156140740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe156140740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56342fa1e2f0 con 0x56342fac3370 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe156140740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56342f9c5cf0 con 0x56342fab9b40 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe156140740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56342fa1a2b0 con 0x56342fab85d0 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151b03640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fab8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151b03640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fab8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36728/0 (socket says 172.21.15.5:36728) 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151b03640 1 -- 172.21.15.5:0/1643217269 learned_addr learned my addr 172.21.15.5:0/1643217269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:16.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151302640 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 0x56342fab8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151302640 1 -- 172.21.15.5:0/1643217269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 msgr2=0x56342fab8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151302640 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fab8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151302640 1 -- 172.21.15.5:0/1643217269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 msgr2=0x56342fac5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe152304640 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151302640 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151302640 1 -- 172.21.15.5:0/1643217269 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342f9aa3a0 con 0x56342fab85d0 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151b03640 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fab8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe152304640 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe152304640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1380a2d70 0x7fe1380a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe144005fd0 tx=0x7fe144009e70 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.452+0000 7fe151302640 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 0x56342fab8a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0ecde0 tx=0x7fe13c003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d94e068b3deab637 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:16.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.453+0000 7fe137fff640 1 -- 172.21.15.5:0/1643217269 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c049040 con 0x56342fab85d0 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.453+0000 7fe137fff640 1 -- 172.21.15.5:0/1643217269 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c0c4820 con 0x56342fab85d0 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.453+0000 7fe137fff640 1 -- 172.21.15.5:0/1643217269 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c031bd0 con 0x56342fab85d0 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.453+0000 7fe156140740 1 -- 172.21.15.5:0/1643217269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 msgr2=0x56342fab8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.453+0000 7fe156140740 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 0x56342fab8a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0ecde0 tx=0x7fe13c003eb0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.454+0000 7fe156140740 1 -- 172.21.15.5:0/1643217269 shutdown_connections 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.454+0000 7fe156140740 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.454+0000 7fe156140740 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 0x56342fab8a10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.454+0000 7fe156140740 1 --2- 172.21.15.5:0/1643217269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fab8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.454+0000 7fe156140740 1 -- 172.21.15.5:0/1643217269 >> 172.21.15.5:0/1643217269 conn(0x56342fa81310 msgr2=0x56342fab4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.454+0000 7fe156140740 1 -- 172.21.15.5:0/1643217269 shutdown_connections 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.454+0000 7fe156140740 1 -- 172.21.15.5:0/1643217269 wait complete. 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.454+0000 7fe156140740 1 -- start start 2024-09-11T22:26:16.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.454+0000 7fe156140740 1 Processor -- start 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe156140740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 0x56342fac2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe156140740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fac0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe156140740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe156140740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56342fa1a2b0 con 0x56342fac3370 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe156140740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56342f9d9060 con 0x56342fab9b40 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe156140740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56342fac5e30 con 0x56342fab85d0 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe156140740 10 client.0 _ll_register_callbacks cb 0x56342f72c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe152304640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe152304640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37810/0 (socket says 172.21.15.5:37810) 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe152304640 1 -- 172.21.15.5:0/1433507712 learned_addr learned my addr 172.21.15.5:0/1433507712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe151302640 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fac0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe151b03640 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 0x56342fac2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe152304640 1 -- 172.21.15.5:0/1433507712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 msgr2=0x56342fac2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe152304640 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 0x56342fac2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe152304640 1 -- 172.21.15.5:0/1433507712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 msgr2=0x56342fac0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe152304640 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fac0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe152304640 1 -- 172.21.15.5:0/1433507712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342f78f3c0 con 0x56342fac3370 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe151b03640 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 0x56342fac2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe151302640 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fac0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe152304640 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac0800 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe144012340 tx=0x7fe144004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c73f4d68fcf29fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.455+0000 7fe1357fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe1357fa640 1 -- 172.21.15.5:0/1433507712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144023070 con 0x56342fac3370 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe1357fa640 1 -- 172.21.15.5:0/1433507712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe14401f800 con 0x56342fac3370 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe156140740 1 -- 172.21.15.5:0/1433507712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe13c0c4820 con 0x56342fac3370 2024-09-11T22:26:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe1357fa640 1 -- 172.21.15.5:0/1433507712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe144010e40 con 0x56342fac3370 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe156140740 10 client.16623 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe156140740 1 -- 172.21.15.5:0/1433507712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56342fad1790 con 0x56342fac3370 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe1357fa640 1 -- 172.21.15.5:0/1433507712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c0c4820 con 0x56342fac3370 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe1357fa640 1 -- 172.21.15.5:0/1433507712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe14400e3b0 con 0x56342fac3370 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe156140740 10 client.16623 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe156140740 10 client.16623 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe156140740 20 client.16623 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe156140740 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x56342fad2c50 0x56342fad5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe156140740 4 client.16623 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.456+0000 7fe156140740 1 -- 172.21.15.5:0/1433507712 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56342fa2b860 con 0x56342fad2c50 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.457+0000 7fe151b03640 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x56342fad2c50 0x56342fad5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.457+0000 7fe151b03640 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x56342fad2c50 0x56342fad5070 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d0be6a964799399d server_cookie=455bfe029f9401f7 in_seq=0 out_seq=0 2024-09-11T22:26:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.457+0000 7fe1357fa640 10 client.16623 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:26:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe1357fa640 1 -- 172.21.15.5:0/1433507712 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56342fa2b860 con 0x56342fad2c50 2024-09-11T22:26:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe1357fa640 10 client.16623 handle_command_reply: tid=0 2024-09-11T22:26:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 client.16623 shutdown 2024-09-11T22:26:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 -- 172.21.15.5:0/1433507712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 msgr2=0x56342fac0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac0800 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe144012340 tx=0x7fe144004ab0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 -- 172.21.15.5:0/1433507712 shutdown_connections 2024-09-11T22:26:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x56342fad2c50 0x56342fad5070 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fac3370 0x56342fac0800 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fab9b40 0x56342fac0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 --2- 172.21.15.5:0/1433507712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab85d0 0x56342fac2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe1357fa640 0 client.16623 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:26:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 -- 172.21.15.5:0/1433507712 >> 172.21.15.5:0/1433507712 conn(0x56342fa81310 msgr2=0x56342faba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 -- 172.21.15.5:0/1433507712 shutdown_connections 2024-09-11T22:26:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 1 -- 172.21.15.5:0/1433507712 wait complete. 2024-09-11T22:26:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.458+0000 7fe156140740 20 client.16623 trim_cache size 0 max 16384 2024-09-11T22:26:16.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe156140740 1 Processor -- start 2024-09-11T22:26:16.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe156140740 1 -- start start 2024-09-11T22:26:16.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe156140740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe156140740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab9b40 0x56342fabf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe156140740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 0x56342fabb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe156140740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1ada30 con 0x56342fab85d0 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe156140740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1af570 con 0x56342fac3370 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe156140740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1ae100 con 0x56342fab9b40 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe151b03640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe151b03640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37814/0 (socket says 172.21.15.5:37814) 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe151b03640 1 -- 172.21.15.5:0/3805245915 learned_addr learned my addr 172.21.15.5:0/3805245915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe152304640 1 --2- 172.21.15.5:0/3805245915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 0x56342fabb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.472+0000 7fe151302640 1 --2- 172.21.15.5:0/3805245915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab9b40 0x56342fabf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.473+0000 7fe151b03640 1 -- 172.21.15.5:0/3805245915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab9b40 msgr2=0x56342fabf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.473+0000 7fe151b03640 1 --2- 172.21.15.5:0/3805245915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab9b40 0x56342fabf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.473+0000 7fe151b03640 1 -- 172.21.15.5:0/3805245915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 msgr2=0x56342fabb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.473+0000 7fe151b03640 1 --2- 172.21.15.5:0/3805245915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 0x56342fabb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.473+0000 7fe151b03640 1 -- 172.21.15.5:0/3805245915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342fac3130 con 0x56342fab85d0 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.473+0000 7fe152304640 1 --2- 172.21.15.5:0/3805245915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 0x56342fabb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.473+0000 7fe151b03640 1 --2- 172.21.15.5:0/3805245915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac0780 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe148012b10 tx=0x7fe148005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae589845bb93d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.473+0000 7fe1377fe640 1 -- 172.21.15.5:0/3805245915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14800fca0 con 0x56342fab85d0 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.473+0000 7fe1377fe640 1 -- 172.21.15.5:0/3805245915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14800fe40 con 0x56342fab85d0 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.473+0000 7fe1377fe640 1 -- 172.21.15.5:0/3805245915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148010430 con 0x56342fab85d0 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 -- 172.21.15.5:0/3805245915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 msgr2=0x56342fac0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 --2- 172.21.15.5:0/3805245915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac0780 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe148012b10 tx=0x7fe148005f90 comp rx=0 tx=0).stop 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 -- 172.21.15.5:0/3805245915 shutdown_connections 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 --2- 172.21.15.5:0/3805245915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 0x56342fabb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 --2- 172.21.15.5:0/3805245915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab9b40 0x56342fabf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 --2- 172.21.15.5:0/3805245915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac0780 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 -- 172.21.15.5:0/3805245915 >> 172.21.15.5:0/3805245915 conn(0x56342fa81310 msgr2=0x56342faafa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 -- 172.21.15.5:0/3805245915 shutdown_connections 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 -- 172.21.15.5:0/3805245915 wait complete. 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 -- start start 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.474+0000 7fe156140740 1 Processor -- start 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe156140740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe156140740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab9b40 0x56342fac66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe156140740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 0x56342fb3b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe156140740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1aeea0 con 0x56342fab85d0 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe156140740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1ae7d0 con 0x56342fac3370 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe156140740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1aba10 con 0x56342fab9b40 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe156140740 10 client.0 _ll_register_callbacks cb 0x7fe144012110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:16.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe151b03640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe152304640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 0x56342fb3b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe151b03640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37822/0 (socket says 172.21.15.5:37822) 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe151b03640 1 -- 172.21.15.5:0/3289477335 learned_addr learned my addr 172.21.15.5:0/3289477335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe151302640 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab9b40 0x56342fac66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe152304640 1 -- 172.21.15.5:0/3289477335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab9b40 msgr2=0x56342fac66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe152304640 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab9b40 0x56342fac66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe152304640 1 -- 172.21.15.5:0/3289477335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 msgr2=0x56342fac61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe152304640 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe152304640 1 -- 172.21.15.5:0/3289477335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342fb3be40 con 0x56342fac3370 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.475+0000 7fe151b03640 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe152304640 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 0x56342fb3b900 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe14400e7d0 tx=0x7fe14400e5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b8f43f887339651 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:16.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe1357fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe1357fa640 1 -- 172.21.15.5:0/3289477335 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144023050 con 0x56342fac3370 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe1357fa640 1 -- 172.21.15.5:0/3289477335 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe14400edf0 con 0x56342fac3370 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe156140740 1 -- 172.21.15.5:0/3289477335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe14800fe40 con 0x56342fac3370 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe1357fa640 1 -- 172.21.15.5:0/3289477335 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe14401f810 con 0x56342fac3370 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe156140740 10 client.26138 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe156140740 1 -- 172.21.15.5:0/3289477335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56342fb3cfe0 con 0x56342fac3370 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe1357fa640 1 -- 172.21.15.5:0/3289477335 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe14800fe40 con 0x56342fac3370 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe1357fa640 1 -- 172.21.15.5:0/3289477335 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe14403f030 con 0x56342fac3370 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe156140740 10 client.26138 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe156140740 10 client.26138 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe156140740 20 client.26138 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:16.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.476+0000 7fe156140740 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x56342fb3d6f0 0x56342fb3fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.477+0000 7fe156140740 4 client.26138 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.477+0000 7fe156140740 1 -- 172.21.15.5:0/3289477335 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56342fa25d70 con 0x56342fb3d6f0 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.477+0000 7fe151b03640 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x56342fb3d6f0 0x56342fb3fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.477+0000 7fe1357fa640 10 client.26138 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.477+0000 7fe151b03640 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x56342fb3d6f0 0x56342fb3fae0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5c1e9e594be2f776 server_cookie=13455d8210e6e2ff in_seq=0 out_seq=0 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.477+0000 7fe1357fa640 1 -- 172.21.15.5:0/3289477335 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56342fa25d70 con 0x56342fb3d6f0 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.477+0000 7fe1357fa640 10 client.26138 handle_command_reply: tid=0 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.477+0000 7fe156140740 1 client.26138 shutdown 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 1 -- 172.21.15.5:0/3289477335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 msgr2=0x56342fb3b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 0x56342fb3b900 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe14400e7d0 tx=0x7fe14400e5e0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 1 -- 172.21.15.5:0/3289477335 shutdown_connections 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x56342fb3d6f0 0x56342fb3fae0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56342fac3370 0x56342fb3b900 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56342fab9b40 0x56342fac66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe1357fa640 0 client.26138 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:26:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 1 --2- 172.21.15.5:0/3289477335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56342fab85d0 0x56342fac61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 1 -- 172.21.15.5:0/3289477335 >> 172.21.15.5:0/3289477335 conn(0x56342fa81310 msgr2=0x56342fabf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 1 -- 172.21.15.5:0/3289477335 shutdown_connections 2024-09-11T22:26:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 1 -- 172.21.15.5:0/3289477335 wait complete. 2024-09-11T22:26:16.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe156140740 20 client.26138 trim_cache size 0 max 16384 2024-09-11T22:26:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:26:16.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe153d67640 1 -- 172.21.15.5:0/1260056259 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1380a2d70 msgr2=0x7fe1380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:16.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe153d67640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1380a2d70 0x7fe1380a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe144005fd0 tx=0x7fe144009e70 comp rx=0 tx=0).stop 2024-09-11T22:26:16.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe153d67640 1 -- 172.21.15.5:0/1260056259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe14c1027a0 msgr2=0x7fe14c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:16.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe153d67640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe14c1027a0 0x7fe14c198960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe13c005940 tx=0x7fe13c0040c0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe153d67640 1 -- 172.21.15.5:0/1260056259 shutdown_connections 2024-09-11T22:26:16.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe153d67640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe14c106f20 0x7fe14c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe153d67640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1380a2d70 0x7fe1380a5230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe153d67640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe14c1027a0 0x7fe14c198960 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe153d67640 1 --2- 172.21.15.5:0/1260056259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe14c101df0 0x7fe14c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:16.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.478+0000 7fe153d67640 1 -- 172.21.15.5:0/1260056259 >> 172.21.15.5:0/1260056259 conn(0x7fe14c0fd8b0 msgr2=0x7fe14c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:16.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.479+0000 7fe153d67640 1 -- 172.21.15.5:0/1260056259 shutdown_connections 2024-09-11T22:26:16.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:16.479+0000 7fe153d67640 1 -- 172.21.15.5:0/1260056259 wait complete. 2024-09-11T22:26:16.723 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T22:26:16.723 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T22:26:16.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:26:17.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 304696 13.83 MB/sec execute 35 sec latency 14.148 ms 2024-09-11T22:26:17.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:17 smithi005 ceph-mon[30943]: pgmap v126: 97 pgs: 97 active+clean; 70 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 259 op/s 2024-09-11T22:26:17.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:26:17.547 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:17 smithi159 ceph-mon[35534]: pgmap v126: 97 pgs: 97 active+clean; 70 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 259 op/s 2024-09-11T22:26:17.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:17 smithi172 ceph-mon[35461]: pgmap v126: 97 pgs: 97 active+clean; 70 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 259 op/s 2024-09-11T22:26:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.048+0000 7f94469be640 1 -- 172.21.15.5:0/2784152173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440101420 msgr2=0x7f944010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.048+0000 7f94469be640 1 --2- 172.21.15.5:0/2784152173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440101420 0x7f944010d950 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9438009cd0 tx=0x7f943802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.048+0000 7f94469be640 1 -- 172.21.15.5:0/2784152173 shutdown_connections 2024-09-11T22:26:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.048+0000 7f94469be640 1 --2- 172.21.15.5:0/2784152173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440101420 0x7f944010d950 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.048+0000 7f94469be640 1 --2- 172.21.15.5:0/2784152173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9440100a80 0x7f9440100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.048+0000 7f94469be640 1 --2- 172.21.15.5:0/2784152173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9440106a80 0x7f9440106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 -- 172.21.15.5:0/2784152173 >> 172.21.15.5:0/2784152173 conn(0x7f94400fc7a0 msgr2=0x7f94400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 -- 172.21.15.5:0/2784152173 shutdown_connections 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 -- 172.21.15.5:0/2784152173 wait complete. 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 Processor -- start 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 -- start start 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440100a80 0x7f944019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9440101420 0x7f944019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9440106a80 0x7f944019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9440110e80 con 0x7f9440100a80 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9440110d00 con 0x7f9440101420 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f94469be640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9440111000 con 0x7f9440106a80 2024-09-11T22:26:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f943ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440100a80 0x7f944019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.049+0000 7f943ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440100a80 0x7f944019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37846/0 (socket says 172.21.15.5:37846) 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f943ffff640 1 -- 172.21.15.5:0/2975525559 learned_addr learned my addr 172.21.15.5:0/2975525559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f9437fff640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9440101420 0x7f944019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f9444f5b640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9440106a80 0x7f944019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f943ffff640 1 -- 172.21.15.5:0/2975525559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9440106a80 msgr2=0x7f944019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f943ffff640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9440106a80 0x7f944019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f943ffff640 1 -- 172.21.15.5:0/2975525559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9440101420 msgr2=0x7f944019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f943ffff640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9440101420 0x7f944019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f943ffff640 1 -- 172.21.15.5:0/2975525559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944019f580 con 0x7f9440100a80 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f9444f5b640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9440106a80 0x7f944019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f943ffff640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440100a80 0x7f944019a5d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f943000e9f0 tx=0x7f943000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f943dffb640 1 -- 172.21.15.5:0/2975525559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943000cd80 con 0x7f9440100a80 2024-09-11T22:26:18.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f94469be640 1 -- 172.21.15.5:0/2975525559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944019f870 con 0x7f9440100a80 2024-09-11T22:26:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f943dffb640 1 -- 172.21.15.5:0/2975525559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9430005500 con 0x7f9440100a80 2024-09-11T22:26:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f943dffb640 1 -- 172.21.15.5:0/2975525559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430010690 con 0x7f9440100a80 2024-09-11T22:26:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.050+0000 7f94469be640 1 -- 172.21.15.5:0/2975525559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94401a7150 con 0x7f9440100a80 2024-09-11T22:26:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.051+0000 7f94469be640 1 -- 172.21.15.5:0/2975525559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9440100ee0 con 0x7f9440100a80 2024-09-11T22:26:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.052+0000 7f943dffb640 1 -- 172.21.15.5:0/2975525559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f94300046c0 con 0x7f9440100a80 2024-09-11T22:26:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.053+0000 7f943dffb640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f942c0a2cf0 0x7f942c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.053+0000 7f9437fff640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f942c0a2cf0 0x7f942c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.053+0000 7f943dffb640 1 -- 172.21.15.5:0/2975525559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f94300cfb20 con 0x7f9440100a80 2024-09-11T22:26:18.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.054+0000 7f9437fff640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f942c0a2cf0 0x7f942c0a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f944010d730 tx=0x7f942800b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:18.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.056+0000 7f943dffb640 1 -- 172.21.15.5:0/2975525559 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94300541b0 con 0x7f9440100a80 2024-09-11T22:26:18.100 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.292+0000 7f94469be640 1 -- 172.21.15.5:0/2975525559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f944019b930 con 0x7f9440100a80 2024-09-11T22:26:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.293+0000 7f943dffb640 1 -- 172.21.15.5:0/2975525559 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11967 (secure 0 0 0) 0x7f944019b930 con 0x7f9440100a80 2024-09-11T22:26:18.294 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:26:18.294 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-11T22:25:49:149525+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:25:48.784660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25970":{"gid":25970,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4169170564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4169170564},{"type":"v1","addr":"172.21.15.159:6833","nonce":4169170564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:26:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 -- 172.21.15.5:0/2975525559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f942c0a2cf0 msgr2=0x7f942c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f942c0a2cf0 0x7f942c0a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f944010d730 tx=0x7f942800b390 comp rx=0 tx=0).stop 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 -- 172.21.15.5:0/2975525559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440100a80 msgr2=0x7f944019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440100a80 0x7f944019a5d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f943000e9f0 tx=0x7f943000edf0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 -- 172.21.15.5:0/2975525559 shutdown_connections 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9440106a80 0x7f944019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f942c0a2cf0 0x7f942c0a51b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9440101420 0x7f944019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 --2- 172.21.15.5:0/2975525559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9440100a80 0x7f944019a5d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 -- 172.21.15.5:0/2975525559 >> 172.21.15.5:0/2975525559 conn(0x7f94400fc7a0 msgr2=0x7f944006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 -- 172.21.15.5:0/2975525559 shutdown_connections 2024-09-11T22:26:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:18.295+0000 7f94469be640 1 -- 172.21.15.5:0/2975525559 wait complete. 2024-09-11T22:26:18.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 306571 13.76 MB/sec execute 36 sec latency 134.241 ms 2024-09-11T22:26:19.114 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T22:26:19.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T22:26:19.313 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309081 13.76 MB/sec execute 37 sec latency 33.164 ms 2024-09-11T22:26:19.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:19 smithi005 ceph-mon[30943]: pgmap v127: 97 pgs: 97 active+clean; 67 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 494 op/s 2024-09-11T22:26:19.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:19 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2975525559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.395+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1951760919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc102570 msgr2=0x7f5fcc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.395+0000 7f5fd29e9640 1 --2- 172.21.15.5:0/1951760919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc102570 0x7f5fcc1029b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009900 tx=0x7f5fc002a480 comp rx=0 tx=0).stop 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.395+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1951760919 shutdown_connections 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.395+0000 7f5fd29e9640 1 --2- 172.21.15.5:0/1951760919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fcc102ef0 0x7f5fcc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.395+0000 7f5fd29e9640 1 --2- 172.21.15.5:0/1951760919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc102570 0x7f5fcc1029b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.395+0000 7f5fd29e9640 1 --2- 172.21.15.5:0/1951760919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fcc100e20 0x7f5fcc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.395+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1951760919 >> 172.21.15.5:0/1951760919 conn(0x7f5fcc0fc7d0 msgr2=0x7f5fcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.395+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1951760919 shutdown_connections 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.395+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1951760919 wait complete. 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fd29e9640 1 Processor -- start 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fd29e9640 1 -- start start 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fd29e9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fcc100e20 0x7f5fcc1a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fd29e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc102570 0x7f5fcc1a4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fd29e9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fcc102ef0 0x7f5fcc19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fd29e9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc110f80 con 0x7f5fcc102570 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fd29e9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc110e00 con 0x7f5fcc100e20 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fd29e9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc111100 con 0x7f5fcc102ef0 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fcb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc102570 0x7f5fcc1a4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fcc100e20 0x7f5fcc1a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fcc100e20 0x7f5fcc1a4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36796/0 (socket says 172.21.15.5:36796) 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fcb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc102570 0x7f5fcc1a4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37876/0 (socket says 172.21.15.5:37876) 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fcbfff640 1 -- 172.21.15.5:0/1520554049 learned_addr learned my addr 172.21.15.5:0/1520554049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.396+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fcc102ef0 0x7f5fcc19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fcb7fe640 1 -- 172.21.15.5:0/1520554049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fcc102ef0 msgr2=0x7f5fcc19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fcb7fe640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fcc102ef0 0x7f5fcc19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fcb7fe640 1 -- 172.21.15.5:0/1520554049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fcc100e20 msgr2=0x7f5fcc1a4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fcb7fe640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fcc100e20 0x7f5fcc1a4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fcb7fe640 1 -- 172.21.15.5:0/1520554049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc19f4e0 con 0x7f5fcc102570 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fcbfff640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fcc100e20 0x7f5fcc1a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fcb7fe640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc102570 0x7f5fcc1a4d60 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0005940 tx=0x7f5fc0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fc97fa640 1 -- 172.21.15.5:0/1520554049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc003f070 con 0x7f5fcc102570 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fc97fa640 1 -- 172.21.15.5:0/1520554049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc0031c30 con 0x7f5fcc102570 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fc97fa640 1 -- 172.21.15.5:0/1520554049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0035030 con 0x7f5fcc102570 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1520554049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fcc19f770 con 0x7f5fcc102570 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.397+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1520554049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fcc1ab5a0 con 0x7f5fcc102570 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fc97fa640 1 -- 172.21.15.5:0/1520554049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f5fc0042070 con 0x7f5fcc102570 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fd4dc2740 1 Processor -- start 2024-09-11T22:26:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fd4dc2740 1 -- start start 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca118a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca118aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca1197760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fca10f02f0 con 0x55fca1195370 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fca1097cf0 con 0x55fca118a5d0 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fca10ec2b0 con 0x55fca118bb40 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fc97fa640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fb40a2cf0 0x7f5fb40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fcb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca1197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca118aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.400+0000 7f5fd0f86640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca118a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fd0f86640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca118a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45222/0 (socket says 172.21.15.5:45222) 2024-09-11T22:26:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fd0f86640 1 -- 172.21.15.5:0/2443346908 learned_addr learned my addr 172.21.15.5:0/2443346908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fb40a2cf0 0x7f5fb40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fc97fa640 1 -- 172.21.15.5:0/1520554049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f5fc00efa90 con 0x7f5fcc102570 2024-09-11T22:26:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fd0f86640 1 -- 172.21.15.5:0/2443346908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 msgr2=0x55fca118aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/2443346908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca118aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fd0f86640 1 -- 172.21.15.5:0/2443346908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 msgr2=0x55fca1197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/2443346908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca1197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fd0f86640 1 -- 172.21.15.5:0/2443346908 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca107c3a0 con 0x55fca118bb40 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fcbfff640 1 --2- 172.21.15.5:0/2443346908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca118aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fb40a2cf0 0x7f5fb40a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4010c40 tx=0x7f5fc400c000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/2443346908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca118a090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4012d80 tx=0x7f5fc4010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ad07ca2d040d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.401+0000 7f5fb27fc640 1 -- 172.21.15.5:0/2443346908 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4024070 con 0x55fca118bb40 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fb27fc640 1 -- 172.21.15.5:0/2443346908 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc40125a0 con 0x55fca118bb40 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fb27fc640 1 -- 172.21.15.5:0/2443346908 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4005280 con 0x55fca118bb40 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/2443346908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 msgr2=0x55fca118a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/2443346908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca118a090 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4012d80 tx=0x7f5fc4010740 comp rx=0 tx=0).stop 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/2443346908 shutdown_connections 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/2443346908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca1197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/2443346908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca118aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/2443346908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca118a090 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/2443346908 >> 172.21.15.5:0/2443346908 conn(0x55fca1153310 msgr2=0x55fca1186c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/2443346908 shutdown_connections 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.402+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/2443346908 wait complete. 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.403+0000 7f5fd4dc2740 1 -- start start 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.403+0000 7f5fd4dc2740 1 Processor -- start 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.403+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca1194d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.403+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca1192000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.403+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.403+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fca1198010 con 0x55fca1195370 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.403+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fca10ab060 con 0x55fca118a5d0 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.403+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fca1198190 con 0x55fca118bb40 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fd4dc2740 10 client.0 _ll_register_callbacks cb 0x55fca0dfe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fd0f86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fcbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca1194d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fd0f86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11925d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37892/0 (socket says 172.21.15.5:37892) 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fd0f86640 1 -- 172.21.15.5:0/209433319 learned_addr learned my addr 172.21.15.5:0/209433319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fcb7fe640 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca1192000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fcbfff640 1 -- 172.21.15.5:0/209433319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 msgr2=0x55fca1192000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fcbfff640 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca1192000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fcbfff640 1 -- 172.21.15.5:0/209433319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 msgr2=0x55fca11925d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fcbfff640 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11925d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fcbfff640 1 -- 172.21.15.5:0/209433319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca0e613c0 con 0x55fca118a5d0 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fcbfff640 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca1194d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55fca1187ac0 tx=0x7f5fbc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de263c03c7c6d984 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fab7fe640 1 -- 172.21.15.5:0/209433319 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0120d0 con 0x55fca118a5d0 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fab7fe640 1 -- 172.21.15.5:0/209433319 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f5fbc004020 con 0x55fca118a5d0 2024-09-11T22:26:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.404+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/209433319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc40125a0 con 0x55fca118a5d0 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fab7fe640 1 -- 172.21.15.5:0/209433319 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5fbc012270 con 0x55fca118a5d0 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fd4dc2740 10 client.26156 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/209433319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fca11a3af0 con 0x55fca118a5d0 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fab7fe640 1 -- 172.21.15.5:0/209433319 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc40125a0 con 0x55fca118a5d0 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fab7fe640 1 -- 172.21.15.5:0/209433319 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f5fbc01c470 con 0x55fca118a5d0 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fd4dc2740 10 client.26156 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fd4dc2740 10 client.26156 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fd4dc2740 20 client.26156 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55fca11a4fb0 0x55fca11a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fd4dc2740 4 client.26156 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/209433319 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca10fd860 con 0x55fca11a4fb0 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.405+0000 7f5fcb7fe640 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55fca11a4fb0 0x55fca11a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.406+0000 7f5fcb7fe640 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55fca11a4fb0 0x55fca11a73c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dfa786dae0728235 server_cookie=121ae4857eb6b350 in_seq=0 out_seq=0 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.406+0000 7f5fab7fe640 10 client.26156 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.406+0000 7f5fab7fe640 1 -- 172.21.15.5:0/209433319 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fca10fd860 con 0x55fca11a4fb0 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.406+0000 7f5fab7fe640 10 client.26156 handle_command_reply: tid=0 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.406+0000 7f5fd4dc2740 1 client.26156 shutdown 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.406+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/209433319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 msgr2=0x55fca1194d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.406+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca1194d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55fca1187ac0 tx=0x7f5fbc00f790 comp rx=0 tx=0).stop 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.407+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/209433319 shutdown_connections 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.407+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11925d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.407+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca1192000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.407+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55fca11a4fb0 0x55fca11a73c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.407+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/209433319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca1194d50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.407+0000 7f5fab7fe640 0 client.26156 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.407+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/209433319 >> 172.21.15.5:0/209433319 conn(0x55fca1153310 msgr2=0x55fca118c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.407+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/209433319 shutdown_connections 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.407+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/209433319 wait complete. 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.407+0000 7f5fd4dc2740 20 client.26156 trim_cache size 0 max 16384 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.420+0000 7f5fd4dc2740 1 Processor -- start 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.420+0000 7f5fd4dc2740 1 -- start start 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.420+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca1186e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.420+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca11a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.420+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11918c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.420+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1b3fe0 con 0x55fca1195370 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.420+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1b5b20 con 0x55fca118a5d0 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.420+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1b46b0 con 0x55fca118bb40 2024-09-11T22:26:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fd0f86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fd0f86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11918c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37906/0 (socket says 172.21.15.5:37906) 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fd0f86640 1 -- 172.21.15.5:0/1510418811 learned_addr learned my addr 172.21.15.5:0/1510418811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fcbfff640 1 --2- 172.21.15.5:0/1510418811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca1186e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fcb7fe640 1 --2- 172.21.15.5:0/1510418811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca11a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fd0f86640 1 -- 172.21.15.5:0/1510418811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 msgr2=0x55fca11a12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/1510418811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca11a12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fd0f86640 1 -- 172.21.15.5:0/1510418811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 msgr2=0x55fca1186e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/1510418811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca1186e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fd0f86640 1 -- 172.21.15.5:0/1510418811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca1190550 con 0x55fca1195370 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/1510418811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11918c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400ce70 tx=0x7f5fc4012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98064f59ac706580 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.421+0000 7f5fb1ffb640 1 -- 172.21.15.5:0/1510418811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4024040 con 0x55fca1195370 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.422+0000 7f5fb1ffb640 1 -- 172.21.15.5:0/1510418811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc4012890 con 0x55fca1195370 2024-09-11T22:26:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.422+0000 7f5fb1ffb640 1 -- 172.21.15.5:0/1510418811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc401b040 con 0x55fca1195370 2024-09-11T22:26:19.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.422+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/1510418811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 msgr2=0x55fca11918c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:19.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.422+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/1510418811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11918c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400ce70 tx=0x7f5fc4012450 comp rx=0 tx=0).stop 2024-09-11T22:26:19.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.422+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/1510418811 shutdown_connections 2024-09-11T22:26:19.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.422+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/1510418811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca11918c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.422+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/1510418811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118bb40 0x55fca11a12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.422+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/1510418811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118a5d0 0x55fca1186e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.422+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/1510418811 >> 172.21.15.5:0/1510418811 conn(0x55fca1153310 msgr2=0x55fca1186700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.422+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/1510418811 shutdown_connections 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.423+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/1510418811 wait complete. 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.423+0000 7f5fd4dc2740 1 -- start start 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.423+0000 7f5fd4dc2740 1 Processor -- start 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.423+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118a5d0 0x55fca1198950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.423+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118bb40 0x55fca1198e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.423+0000 7f5fd4dc2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca120dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.423+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1b5450 con 0x55fca1195370 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.423+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1b4d80 con 0x55fca118bb40 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.423+0000 7f5fd4dc2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc1b1fc0 con 0x55fca118a5d0 2024-09-11T22:26:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fd4dc2740 10 client.0 _ll_register_callbacks cb 0x7f5fbc009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fcb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118bb40 0x55fca1198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fd0f86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca120dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fd0f86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca120dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37916/0 (socket says 172.21.15.5:37916) 2024-09-11T22:26:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fcb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118bb40 0x55fca1198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36826/0 (socket says 172.21.15.5:36826) 2024-09-11T22:26:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fd0f86640 1 -- 172.21.15.5:0/50327258 learned_addr learned my addr 172.21.15.5:0/50327258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fcbfff640 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118a5d0 0x55fca1198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fd0f86640 1 -- 172.21.15.5:0/50327258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118a5d0 msgr2=0x55fca1198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118a5d0 0x55fca1198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fd0f86640 1 -- 172.21.15.5:0/50327258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118bb40 msgr2=0x55fca1198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118bb40 0x55fca1198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fd0f86640 1 -- 172.21.15.5:0/50327258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca120e210 con 0x55fca1195370 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fcbfff640 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118a5d0 0x55fca1198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fcb7fe640 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118bb40 0x55fca1198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fd0f86640 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca120dc50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4012cd0 tx=0x7f5fc4012d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfe2589a4addcda server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.424+0000 7f5fab7fe640 1 -- 172.21.15.5:0/50327258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4038070 con 0x55fca1195370 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fab7fe640 1 -- 172.21.15.5:0/50327258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f5fc40051a0 con 0x55fca1195370 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/50327258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc4012890 con 0x55fca1195370 2024-09-11T22:26:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fab7fe640 1 -- 172.21.15.5:0/50327258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5fc4044050 con 0x55fca1195370 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fd4dc2740 10 client.16653 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/50327258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fca120f4d0 con 0x55fca1195370 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fab7fe640 1 -- 172.21.15.5:0/50327258 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc4012890 con 0x55fca1195370 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fab7fe640 1 -- 172.21.15.5:0/50327258 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f5fc400f040 con 0x55fca1195370 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fd4dc2740 10 client.16653 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fd4dc2740 10 client.16653 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fd4dc2740 20 client.16653 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55fca120fa20 0x55fca1211e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fd4dc2740 4 client.16653 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.425+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/50327258 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fca10f7d70 con 0x55fca120fa20 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.426+0000 7f5fcbfff640 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55fca120fa20 0x55fca1211e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.426+0000 7f5fcbfff640 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55fca120fa20 0x55fca1211e40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52e814013953e893 server_cookie=53cde3dd951f9316 in_seq=0 out_seq=0 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.426+0000 7f5fab7fe640 10 client.16653 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:26:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.426+0000 7f5fab7fe640 1 -- 172.21.15.5:0/50327258 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fca10f7d70 con 0x55fca120fa20 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.426+0000 7f5fab7fe640 10 client.16653 handle_command_reply: tid=0 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.426+0000 7f5fd4dc2740 1 client.16653 shutdown 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.426+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/50327258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 msgr2=0x55fca120dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.426+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca120dc50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4012cd0 tx=0x7f5fc4012d00 comp rx=0 tx=0).stop 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.426+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/50327258 shutdown_connections 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.426+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fca1195370 0x55fca120dc50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.427+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55fca120fa20 0x55fca1211e40 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.427+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fca118bb40 0x55fca1198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.427+0000 7f5fab7fe640 0 client.16653 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.427+0000 7f5fd4dc2740 1 --2- 172.21.15.5:0/50327258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca118a5d0 0x55fca1198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.427+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/50327258 >> 172.21.15.5:0/50327258 conn(0x55fca1153310 msgr2=0x55fca11854e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.427+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/50327258 shutdown_connections 2024-09-11T22:26:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.427+0000 7f5fd4dc2740 1 -- 172.21.15.5:0/50327258 wait complete. 2024-09-11T22:26:19.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.427+0000 7f5fd4dc2740 20 client.16653 trim_cache size 0 max 16384 2024-09-11T22:26:19.444 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.428+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1520554049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fb40a2cf0 msgr2=0x7f5fb40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.428+0000 7f5fd29e9640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fb40a2cf0 0x7f5fb40a51b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4010c40 tx=0x7f5fc400c000 comp rx=0 tx=0).stop 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.428+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1520554049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc102570 msgr2=0x7f5fcc1a4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.428+0000 7f5fd29e9640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc102570 0x7f5fcc1a4d60 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0005940 tx=0x7f5fc0003c10 comp rx=0 tx=0).stop 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.429+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1520554049 shutdown_connections 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.429+0000 7f5fd29e9640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fcc102ef0 0x7f5fcc19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.429+0000 7f5fd29e9640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc102570 0x7f5fcc1a4d60 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.429+0000 7f5fd29e9640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fb40a2cf0 0x7f5fb40a51b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.429+0000 7f5fd29e9640 1 --2- 172.21.15.5:0/1520554049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fcc100e20 0x7f5fcc1a4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.429+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1520554049 >> 172.21.15.5:0/1520554049 conn(0x7f5fcc0fc7d0 msgr2=0x7f5fcc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.429+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1520554049 shutdown_connections 2024-09-11T22:26:19.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:19.429+0000 7f5fd29e9640 1 -- 172.21.15.5:0/1520554049 wait complete. 2024-09-11T22:26:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:19 smithi159 ceph-mon[35534]: pgmap v127: 97 pgs: 97 active+clean; 67 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 494 op/s 2024-09-11T22:26:19.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:19 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2975525559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:19.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:19 smithi172 ceph-mon[35461]: pgmap v127: 97 pgs: 97 active+clean; 67 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.3 MiB/s wr, 494 op/s 2024-09-11T22:26:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:19 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2975525559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:20.197 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:26:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:20.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312110 13.86 MB/sec execute 38 sec latency 11.378 ms 2024-09-11T22:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.810+0000 7f56a771a640 1 -- 172.21.15.5:0/446134890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a0100e20 msgr2=0x7f56a0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.810+0000 7f56a771a640 1 --2- 172.21.15.5:0/446134890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a0100e20 0x7f56a0101200 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f569c009900 tx=0x7f569c02a480 comp rx=0 tx=0).stop 2024-09-11T22:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.810+0000 7f56a771a640 1 -- 172.21.15.5:0/446134890 shutdown_connections 2024-09-11T22:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.810+0000 7f56a771a640 1 --2- 172.21.15.5:0/446134890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a0102ef0 0x7f56a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.810+0000 7f56a771a640 1 --2- 172.21.15.5:0/446134890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0102570 0x7f56a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.810+0000 7f56a771a640 1 --2- 172.21.15.5:0/446134890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a0100e20 0x7f56a0101200 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.810+0000 7f56a771a640 1 -- 172.21.15.5:0/446134890 >> 172.21.15.5:0/446134890 conn(0x7f56a00fc7d0 msgr2=0x7f56a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.810+0000 7f56a771a640 1 -- 172.21.15.5:0/446134890 shutdown_connections 2024-09-11T22:26:20.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.810+0000 7f56a771a640 1 -- 172.21.15.5:0/446134890 wait complete. 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a771a640 1 Processor -- start 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a771a640 1 -- start start 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a771a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0100e20 0x7f56a01962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a771a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a0102570 0x7f56a0196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a771a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a0102ef0 0x7f56a019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a771a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010fe80 con 0x7f56a0102570 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a771a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56a010fd00 con 0x7f56a0100e20 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a771a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0110000 con 0x7f56a0102ef0 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a54b6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0100e20 0x7f56a01962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a4cb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a0102570 0x7f56a0196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a54b6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0100e20 0x7f56a01962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36838/0 (socket says 172.21.15.5:36838) 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a54b6640 1 -- 172.21.15.5:0/1122852845 learned_addr learned my addr 172.21.15.5:0/1122852845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.811+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a0102ef0 0x7f56a019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56a54b6640 1 -- 172.21.15.5:0/1122852845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a0102ef0 msgr2=0x7f56a019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56a54b6640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a0102ef0 0x7f56a019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56a54b6640 1 -- 172.21.15.5:0/1122852845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a0102570 msgr2=0x7f56a0196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56a54b6640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a0102570 0x7f56a0196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56a54b6640 1 -- 172.21.15.5:0/1122852845 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a019b270 con 0x7f56a0100e20 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a0102ef0 0x7f56a019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56a54b6640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0100e20 0x7f56a01962c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f569c006010 tx=0x7f569c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56967fc640 1 -- 172.21.15.5:0/1122852845 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c03f070 con 0x7f56a0100e20 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56a771a640 1 -- 172.21.15.5:0/1122852845 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a019b500 con 0x7f56a0100e20 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56967fc640 1 -- 172.21.15.5:0/1122852845 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f569c031c30 con 0x7f56a0100e20 2024-09-11T22:26:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56967fc640 1 -- 172.21.15.5:0/1122852845 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c035030 con 0x7f56a0100e20 2024-09-11T22:26:20.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.812+0000 7f56a771a640 1 -- 172.21.15.5:0/1122852845 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01a2e40 con 0x7f56a0100e20 2024-09-11T22:26:20.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.815+0000 7f56a9af3740 1 Processor -- start 2024-09-11T22:26:20.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.815+0000 7f56a9af3740 1 -- start start 2024-09-11T22:26:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.815+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d3750 0x555bbb1d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.815+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 0x555bbb1e2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1e5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a9af3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555bbb138160 con 0x555bbb1d4070 2024-09-11T22:26:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a9af3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555bbb0df720 con 0x555bbb1e2e40 2024-09-11T22:26:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a9af3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555bbb134120 con 0x555bbb1d3750 2024-09-11T22:26:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a4cb5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1e5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a5cb7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d3750 0x555bbb1d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a4cb5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1e5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36844/0 (socket says 172.21.15.5:36844) 2024-09-11T22:26:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a4cb5640 1 -- 172.21.15.5:0/1975350975 learned_addr learned my addr 172.21.15.5:0/1975350975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a5cb7640 1 -- 172.21.15.5:0/1975350975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 msgr2=0x555bbb1e5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/1975350975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1e5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56967fc640 1 -- 172.21.15.5:0/1122852845 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f569c042070 con 0x7f56a0100e20 2024-09-11T22:26:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a5cb7640 1 -- 172.21.15.5:0/1975350975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 msgr2=0x555bbb1e2900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:26:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/1975350975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 0x555bbb1e2900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a5cb7640 1 -- 172.21.15.5:0/1975350975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bbaea9ba0 con 0x555bbb1d3750 2024-09-11T22:26:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.816+0000 7f56a4cb5640 1 --2- 172.21.15.5:0/1975350975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:20.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.817+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/1975350975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d3750 0x555bbb1d3b30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f56980039c0 tx=0x7f569800b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10e11a651193a329 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:20.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.817+0000 7f568b7fe640 1 -- 172.21.15.5:0/1975350975 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56980104f0 con 0x555bbb1d3750 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.817+0000 7f568b7fe640 1 -- 172.21.15.5:0/1975350975 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f569801b070 con 0x555bbb1d3750 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.817+0000 7f568b7fe640 1 -- 172.21.15.5:0/1975350975 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5698016a30 con 0x555bbb1d3750 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.817+0000 7f56967fc640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f568c0a2cf0 0x7f568c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.817+0000 7f56a54b6640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f568c0a2cf0 0x7f568c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.817+0000 7f56967fc640 1 -- 172.21.15.5:0/1122852845 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f569c0efaf0 con 0x7f56a0100e20 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 -- 172.21.15.5:0/1975350975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d3750 msgr2=0x555bbb1d3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 --2- 172.21.15.5:0/1975350975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d3750 0x555bbb1d3b30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f56980039c0 tx=0x7f569800b710 comp rx=0 tx=0).stop 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a54b6640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f568c0a2cf0 0x7f568c0a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x555bbb1d6440 tx=0x7f569c040450 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 -- 172.21.15.5:0/1975350975 shutdown_connections 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 --2- 172.21.15.5:0/1975350975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1e5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 --2- 172.21.15.5:0/1975350975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 0x555bbb1e2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 --2- 172.21.15.5:0/1975350975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d3750 0x555bbb1d3b30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 -- 172.21.15.5:0/1975350975 >> 172.21.15.5:0/1975350975 conn(0x555bbb1cfde0 msgr2=0x555bbb1d01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 -- 172.21.15.5:0/1975350975 shutdown_connections 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 -- 172.21.15.5:0/1975350975 wait complete. 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 -- start start 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.818+0000 7f56a9af3740 1 Processor -- start 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1e02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1e0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a9af3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555bbb1e5a70 con 0x555bbb1d3750 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a9af3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555bbb0c0900 con 0x555bbb1e2e40 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a9af3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555bbb1e5bf0 con 0x555bbb1d4070 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a9af3740 10 client.0 _ll_register_callbacks cb 0x555bbb0be780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a5cb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a4cb5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1e02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a5cb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1dfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37948/0 (socket says 172.21.15.5:37948) 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a5cb7640 1 -- 172.21.15.5:0/2866189554 learned_addr learned my addr 172.21.15.5:0/2866189554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.819+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.820+0000 7f56a4cb5640 1 -- 172.21.15.5:0/2866189554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 msgr2=0x555bbb1e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.820+0000 7f56a4cb5640 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1e0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.820+0000 7f56a4cb5640 1 -- 172.21.15.5:0/2866189554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 msgr2=0x555bbb1dfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.820+0000 7f56a4cb5640 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1dfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.820+0000 7f56a4cb5640 1 -- 172.21.15.5:0/2866189554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bbb100c50 con 0x555bbb1d4070 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.820+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.820+0000 7f56a4cb5640 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1e02c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56900126d0 tx=0x7f5690012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9905010bcafdd9f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:20.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.820+0000 7f5688ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.821+0000 7f5688ff9640 1 -- 172.21.15.5:0/2866189554 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690004050 con 0x555bbb1d4070 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.821+0000 7f5688ff9640 1 -- 172.21.15.5:0/2866189554 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f56900041f0 con 0x555bbb1d4070 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.821+0000 7f56a9af3740 1 -- 172.21.15.5:0/2866189554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569801b070 con 0x555bbb1d4070 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.821+0000 7f5688ff9640 1 -- 172.21.15.5:0/2866189554 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f56900043f0 con 0x555bbb1d4070 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.821+0000 7f56a9af3740 10 client.26110 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.821+0000 7f56a9af3740 1 -- 172.21.15.5:0/2866189554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555bbb1f1810 con 0x555bbb1d4070 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.822+0000 7f5688ff9640 1 -- 172.21.15.5:0/2866189554 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f569801b070 con 0x555bbb1d4070 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.822+0000 7f5688ff9640 1 -- 172.21.15.5:0/2866189554 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f5690003ca0 con 0x555bbb1d4070 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.822+0000 7f56a9af3740 10 client.26110 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.822+0000 7f56a9af3740 10 client.26110 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:26:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.822+0000 7f56a9af3740 20 client.26110 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.822+0000 7f56a9af3740 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555bbb1f2cd0 0x555bbb1f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.822+0000 7f56a9af3740 4 client.26110 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.822+0000 7f56a9af3740 1 -- 172.21.15.5:0/2866189554 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bbb145890 con 0x555bbb1f2cd0 2024-09-11T22:26:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.822+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555bbb1f2cd0 0x555bbb1f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.823+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555bbb1f2cd0 0x555bbb1f50c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1925a59388d72a7 server_cookie=a9b85003603ba6cb in_seq=0 out_seq=0 2024-09-11T22:26:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.823+0000 7f5688ff9640 10 client.26110 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f5688ff9640 1 -- 172.21.15.5:0/2866189554 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555bbb145890 con 0x555bbb1f2cd0 2024-09-11T22:26:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f5688ff9640 10 client.26110 handle_command_reply: tid=0 2024-09-11T22:26:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f56a9af3740 1 client.26110 shutdown 2024-09-11T22:26:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f56a9af3740 1 -- 172.21.15.5:0/2866189554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 msgr2=0x555bbb1e02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f56a9af3740 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1e02c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f56900126d0 tx=0x7f5690012b90 comp rx=0 tx=0).stop 2024-09-11T22:26:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f56a9af3740 1 -- 172.21.15.5:0/2866189554 shutdown_connections 2024-09-11T22:26:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f56a9af3740 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555bbb1f2cd0 0x555bbb1f50c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f56a9af3740 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f56a9af3740 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1e02c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f56a9af3740 1 --2- 172.21.15.5:0/2866189554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1dfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f56a9af3740 1 -- 172.21.15.5:0/2866189554 >> 172.21.15.5:0/2866189554 conn(0x555bbb1cfde0 msgr2=0x555bbb1d5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.825+0000 7f5688ff9640 0 client.26110 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.826+0000 7f56a9af3740 1 -- 172.21.15.5:0/2866189554 shutdown_connections 2024-09-11T22:26:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.826+0000 7f56a9af3740 1 -- 172.21.15.5:0/2866189554 wait complete. 2024-09-11T22:26:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.826+0000 7f56a9af3740 20 client.26110 trim_cache size 0 max 16384 2024-09-11T22:26:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a9af3740 1 Processor -- start 2024-09-11T22:26:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a9af3740 1 -- start start 2024-09-11T22:26:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1de650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1fa970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a9af3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01ab790 con 0x555bbb1d3750 2024-09-11T22:26:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a9af3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01ad2d0 con 0x555bbb1e2e40 2024-09-11T22:26:20.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a9af3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01abe60 con 0x555bbb1d4070 2024-09-11T22:26:20.849 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a5cb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1de650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a4cb5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1fa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a4cb5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1fa970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45292/0 (socket says 172.21.15.5:45292) 2024-09-11T22:26:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a4cb5640 1 -- 172.21.15.5:0/285553761 learned_addr learned my addr 172.21.15.5:0/285553761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:20.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.847+0000 7f56a5cb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1de650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37960/0 (socket says 172.21.15.5:37960) 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.848+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/285553761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.848+0000 7f56a4cb5640 1 -- 172.21.15.5:0/285553761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 msgr2=0x555bbb1faeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.848+0000 7f56a4cb5640 1 --2- 172.21.15.5:0/285553761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1faeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.848+0000 7f56a4cb5640 1 -- 172.21.15.5:0/285553761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 msgr2=0x555bbb1de650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.848+0000 7f56a4cb5640 1 --2- 172.21.15.5:0/285553761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1de650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.848+0000 7f56a4cb5640 1 -- 172.21.15.5:0/285553761 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bbb1d08a0 con 0x555bbb1d4070 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.848+0000 7f56a4cb5640 1 --2- 172.21.15.5:0/285553761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1fa970 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5690009f20 tx=0x7f5690012660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8ae5117f5e679bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.848+0000 7f568affd640 1 -- 172.21.15.5:0/285553761 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690028040 con 0x555bbb1d4070 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.848+0000 7f568affd640 1 -- 172.21.15.5:0/285553761 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f569000fe00 con 0x555bbb1d4070 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.848+0000 7f568affd640 1 -- 172.21.15.5:0/285553761 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690005290 con 0x555bbb1d4070 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.849+0000 7f56a9af3740 1 -- 172.21.15.5:0/285553761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 msgr2=0x555bbb1fa970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.849+0000 7f56a9af3740 1 --2- 172.21.15.5:0/285553761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1fa970 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5690009f20 tx=0x7f5690012660 comp rx=0 tx=0).stop 2024-09-11T22:26:20.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.849+0000 7f56a9af3740 1 -- 172.21.15.5:0/285553761 shutdown_connections 2024-09-11T22:26:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.849+0000 7f56a9af3740 1 --2- 172.21.15.5:0/285553761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1e2e40 0x555bbb1faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.849+0000 7f56a9af3740 1 --2- 172.21.15.5:0/285553761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1d4070 0x555bbb1fa970 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.849+0000 7f56a9af3740 1 --2- 172.21.15.5:0/285553761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d3750 0x555bbb1de650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.849+0000 7f56a9af3740 1 -- 172.21.15.5:0/285553761 >> 172.21.15.5:0/285553761 conn(0x555bbb1cfde0 msgr2=0x555bbb1ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.849+0000 7f56a9af3740 1 -- 172.21.15.5:0/285553761 shutdown_connections 2024-09-11T22:26:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.849+0000 7f56a9af3740 1 -- 172.21.15.5:0/285553761 wait complete. 2024-09-11T22:26:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a9af3740 1 -- start start 2024-09-11T22:26:20.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a9af3740 1 Processor -- start 2024-09-11T22:26:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1d3750 0x555bbb1e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 0x555bbb1e7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a9af3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1e2e40 0x555bbb25c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a9af3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01acc00 con 0x555bbb1d4070 2024-09-11T22:26:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a9af3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01ac530 con 0x555bbb1d3750 2024-09-11T22:26:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a9af3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56a01a9770 con 0x555bbb1e2e40 2024-09-11T22:26:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a9af3740 10 client.0 _ll_register_callbacks cb 0x7f5690009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a4cb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 0x555bbb1e7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a5cb7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1d3750 0x555bbb1e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a4cb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 0x555bbb1e7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37966/0 (socket says 172.21.15.5:37966) 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.850+0000 7f56a4cb5640 1 -- 172.21.15.5:0/157191232 learned_addr learned my addr 172.21.15.5:0/157191232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1e2e40 0x555bbb25c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f56a4cb5640 1 -- 172.21.15.5:0/157191232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1e2e40 msgr2=0x555bbb25c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f56a4cb5640 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1e2e40 0x555bbb25c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f56a4cb5640 1 -- 172.21.15.5:0/157191232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1d3750 msgr2=0x555bbb1e6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f56a4cb5640 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1d3750 0x555bbb1e6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f56a4cb5640 1 -- 172.21.15.5:0/157191232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bbb1ded30 con 0x555bbb1d4070 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f56a4cb5640 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 0x555bbb1e7520 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5690000c00 tx=0x7f569000ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce615a67798d1b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f5688ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f5688ff9640 1 -- 172.21.15.5:0/157191232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690028040 con 0x555bbb1d4070 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f5688ff9640 1 -- 172.21.15.5:0/157191232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f5690005280 con 0x555bbb1d4070 2024-09-11T22:26:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f56a9af3740 1 -- 172.21.15.5:0/157191232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569000fe00 con 0x555bbb1d4070 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f5688ff9640 1 -- 172.21.15.5:0/157191232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f569003c050 con 0x555bbb1d4070 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f56a9af3740 10 client.16668 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.851+0000 7f56a9af3740 1 -- 172.21.15.5:0/157191232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555bbb1d3ea0 con 0x555bbb1d4070 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.852+0000 7f5688ff9640 1 -- 172.21.15.5:0/157191232 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f569000fe00 con 0x555bbb1d4070 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.852+0000 7f5688ff9640 1 -- 172.21.15.5:0/157191232 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f5690004b80 con 0x555bbb1d4070 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.852+0000 7f56a9af3740 10 client.16668 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.852+0000 7f56a9af3740 10 client.16668 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.852+0000 7f56a9af3740 20 client.16668 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.852+0000 7f56a9af3740 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555bbb25e3b0 0x555bbb2607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.852+0000 7f56a9af3740 4 client.16668 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.852+0000 7f56a9af3740 1 -- 172.21.15.5:0/157191232 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x555bbb13fbe0 con 0x555bbb25e3b0 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.852+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555bbb25e3b0 0x555bbb2607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.853+0000 7f5688ff9640 10 client.16668 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.853+0000 7f56a5cb7640 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555bbb25e3b0 0x555bbb2607d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2306159dbcd27f40 server_cookie=5aa956b15c514767 in_seq=0 out_seq=0 2024-09-11T22:26:20.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f5688ff9640 1 -- 172.21.15.5:0/157191232 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555bbb13fbe0 con 0x555bbb25e3b0 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f5688ff9640 10 client.16668 handle_command_reply: tid=0 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f56a9af3740 1 client.16668 shutdown 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f56a9af3740 1 -- 172.21.15.5:0/157191232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 msgr2=0x555bbb1e7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f56a9af3740 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 0x555bbb1e7520 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5690000c00 tx=0x7f569000ffb0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f56a9af3740 1 -- 172.21.15.5:0/157191232 shutdown_connections 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f56a9af3740 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555bbb25e3b0 0x555bbb2607d0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f56a9af3740 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555bbb1e2e40 0x555bbb25c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f56a9af3740 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555bbb1d4070 0x555bbb1e7520 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f5688ff9640 0 client.16668 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f56a9af3740 1 --2- 172.21.15.5:0/157191232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555bbb1d3750 0x555bbb1e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.938+0000 7f56a9af3740 1 -- 172.21.15.5:0/157191232 >> 172.21.15.5:0/157191232 conn(0x555bbb1cfde0 msgr2=0x555bbb1ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a9af3740 1 -- 172.21.15.5:0/157191232 shutdown_connections 2024-09-11T22:26:20.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a9af3740 1 -- 172.21.15.5:0/157191232 wait complete. 2024-09-11T22:26:20.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a9af3740 20 client.16668 trim_cache size 0 max 16384 2024-09-11T22:26:20.941 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:26:20.941 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 34, 2024-09-11T22:26:20.941 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:26:20.941 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 14, 2024-09-11T22:26:20.941 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:26:20.941 INFO:teuthology.orchestra.run.smithi005.stdout: "9d83d6f2": { 2024-09-11T22:26:20.941 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 14, 2024-09-11T22:26:20.941 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.2, 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:26:20.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:20.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:20.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:20.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:20.943 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:20.943 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:26:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a771a640 1 -- 172.21.15.5:0/1122852845 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f568c0a2cf0 msgr2=0x7f568c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:20.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a771a640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f568c0a2cf0 0x7f568c0a51b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x555bbb1d6440 tx=0x7f569c040450 comp rx=0 tx=0).stop 2024-09-11T22:26:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a771a640 1 -- 172.21.15.5:0/1122852845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0100e20 msgr2=0x7f56a01962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a771a640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0100e20 0x7f56a01962c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f569c006010 tx=0x7f569c003c10 comp rx=0 tx=0).stop 2024-09-11T22:26:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a771a640 1 -- 172.21.15.5:0/1122852845 shutdown_connections 2024-09-11T22:26:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a771a640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56a0102ef0 0x7f56a019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a771a640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56a0102570 0x7f56a0196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a771a640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f568c0a2cf0 0x7f568c0a51b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a771a640 1 --2- 172.21.15.5:0/1122852845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56a0100e20 0x7f56a01962c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.939+0000 7f56a771a640 1 -- 172.21.15.5:0/1122852845 >> 172.21.15.5:0/1122852845 conn(0x7f56a00fc7d0 msgr2=0x7f56a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.940+0000 7f56a771a640 1 -- 172.21.15.5:0/1122852845 shutdown_connections 2024-09-11T22:26:20.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:20.940+0000 7f56a771a640 1 -- 172.21.15.5:0/1122852845 wait complete. 2024-09-11T22:26:21.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 13.56 MB/sec execute 39 sec latency 420.301 ms 2024-09-11T22:26:21.319 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:21 smithi005 ceph-mon[30943]: pgmap v128: 97 pgs: 97 active+clean; 67 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 365 op/s 2024-09-11T22:26:21.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:21 smithi159 ceph-mon[35534]: pgmap v128: 97 pgs: 97 active+clean; 67 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 365 op/s 2024-09-11T22:26:21.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:21 smithi172 ceph-mon[35461]: pgmap v128: 97 pgs: 97 active+clean; 67 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 365 op/s 2024-09-11T22:26:21.986 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9d83d6f2', quiesce took 5.3 seconds. Will release after: 14.849999999999998 2024-09-11T22:26:22.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.211+0000 7fea3f2a6640 1 -- 172.21.15.5:0/1396201837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea381008d0 msgr2=0x7fea38100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:22.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.211+0000 7fea3f2a6640 1 --2- 172.21.15.5:0/1396201837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea381008d0 0x7fea38100cb0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fea28009ca0 tx=0x7fea2802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.211+0000 7fea3f2a6640 1 -- 172.21.15.5:0/1396201837 shutdown_connections 2024-09-11T22:26:22.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.211+0000 7fea3f2a6640 1 --2- 172.21.15.5:0/1396201837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea3810e480 0x7fea38110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.211+0000 7fea3f2a6640 1 --2- 172.21.15.5:0/1396201837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea38101280 0x7fea3810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.211+0000 7fea3f2a6640 1 --2- 172.21.15.5:0/1396201837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea381008d0 0x7fea38100cb0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.211+0000 7fea3f2a6640 1 -- 172.21.15.5:0/1396201837 >> 172.21.15.5:0/1396201837 conn(0x7fea380fc7a0 msgr2=0x7fea380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.212+0000 7fea3f2a6640 1 -- 172.21.15.5:0/1396201837 shutdown_connections 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.212+0000 7fea3f2a6640 1 -- 172.21.15.5:0/1396201837 wait complete. 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.212+0000 7fea3f2a6640 1 Processor -- start 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.212+0000 7fea3f2a6640 1 -- start start 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.212+0000 7fea3f2a6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea381008d0 0x7fea3819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.212+0000 7fea3f2a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea38101280 0x7fea3819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.212+0000 7fea3f2a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea3810e480 0x7fea3819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.212+0000 7fea3f2a6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea381141b0 con 0x7fea38101280 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.212+0000 7fea3f2a6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea38114030 con 0x7fea381008d0 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.212+0000 7fea3f2a6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea38114330 con 0x7fea3810e480 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3c841640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea38101280 0x7fea3819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3c841640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea38101280 0x7fea3819ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51816/0 (socket says 172.21.15.5:51816) 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3c841640 1 -- 172.21.15.5:0/777497175 learned_addr learned my addr 172.21.15.5:0/777497175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3d843640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea3810e480 0x7fea3819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3d042640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea381008d0 0x7fea3819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3c841640 1 -- 172.21.15.5:0/777497175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea3810e480 msgr2=0x7fea3819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3c841640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea3810e480 0x7fea3819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3c841640 1 -- 172.21.15.5:0/777497175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea381008d0 msgr2=0x7fea3819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3c841640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea381008d0 0x7fea3819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3c841640 1 -- 172.21.15.5:0/777497175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3819f680 con 0x7fea38101280 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3c841640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea38101280 0x7fea3819ac10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00d770 tx=0x7fea2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea367fc640 1 -- 172.21.15.5:0/777497175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c017070 con 0x7fea38101280 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea367fc640 1 -- 172.21.15.5:0/777497175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea2c005500 con 0x7fea38101280 2024-09-11T22:26:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3f2a6640 1 -- 172.21.15.5:0/777497175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3819f970 con 0x7fea38101280 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea367fc640 1 -- 172.21.15.5:0/777497175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c004e00 con 0x7fea38101280 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.213+0000 7fea3f2a6640 1 -- 172.21.15.5:0/777497175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea381a7250 con 0x7fea38101280 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.216+0000 7fea367fc640 1 -- 172.21.15.5:0/777497175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fea2c004fa0 con 0x7fea38101280 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.216+0000 7fea4167f740 1 Processor -- start 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.216+0000 7fea4167f740 1 -- start start 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.216+0000 7fea4167f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.216+0000 7fea4167f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea4167f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 0x55ff0e6db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea4167f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e6342f0 con 0x55ff0e6cfb40 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea4167f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e5dbcf0 con 0x55ff0e6d9370 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea4167f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e6302b0 con 0x55ff0e6ce5d0 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea367fc640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fea200a2d70 0x7fea200a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51824/0 (socket says 172.21.15.5:51824) 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 -- 172.21.15.5:0/4169419271 learned_addr learned my addr 172.21.15.5:0/4169419271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d042640 1 --2- 172.21.15.5:0/4169419271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fea200a2d70 0x7fea200a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3c841640 1 --2- 172.21.15.5:0/4169419271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 0x55ff0e6db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea367fc640 1 -- 172.21.15.5:0/777497175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fea2c0cedb0 con 0x7fea38101280 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 -- 172.21.15.5:0/4169419271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6ce5d0 msgr2=0x55ff0e6cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 --2- 172.21.15.5:0/4169419271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 -- 172.21.15.5:0/4169419271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 msgr2=0x55ff0e6db760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 --2- 172.21.15.5:0/4169419271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 0x55ff0e6db760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 -- 172.21.15.5:0/4169419271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff0e5c03a0 con 0x55ff0e6cfb40 2024-09-11T22:26:22.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 --2- 172.21.15.5:0/4169419271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6ce090 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fea300118d0 tx=0x7fea30011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b68a1b5d700348f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea3d843640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fea200a2d70 0x7fea200a5230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea3000f790 tx=0x7fea3001c000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.217+0000 7fea1f7fe640 1 -- 172.21.15.5:0/4169419271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3000fe30 con 0x55ff0e6cfb40 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea1f7fe640 1 -- 172.21.15.5:0/4169419271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea30011260 con 0x55ff0e6cfb40 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea1f7fe640 1 -- 172.21.15.5:0/4169419271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30025790 con 0x55ff0e6cfb40 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea4167f740 1 -- 172.21.15.5:0/4169419271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 msgr2=0x55ff0e6ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea4167f740 1 --2- 172.21.15.5:0/4169419271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6ce090 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fea300118d0 tx=0x7fea30011d90 comp rx=0 tx=0).stop 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea4167f740 1 -- 172.21.15.5:0/4169419271 shutdown_connections 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea4167f740 1 --2- 172.21.15.5:0/4169419271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 0x55ff0e6db760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea4167f740 1 --2- 172.21.15.5:0/4169419271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea4167f740 1 --2- 172.21.15.5:0/4169419271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6ce090 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea4167f740 1 -- 172.21.15.5:0/4169419271 >> 172.21.15.5:0/4169419271 conn(0x55ff0e697310 msgr2=0x55ff0e6cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea4167f740 1 -- 172.21.15.5:0/4169419271 shutdown_connections 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.218+0000 7fea4167f740 1 -- 172.21.15.5:0/4169419271 wait complete. 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea4167f740 1 -- start start 2024-09-11T22:26:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea4167f740 1 Processor -- start 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea4167f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6d8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea4167f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6d6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea4167f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea4167f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e6dc010 con 0x55ff0e6d9370 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea4167f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e5ef060 con 0x55ff0e6ce5d0 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea4167f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e6dc190 con 0x55ff0e6cfb40 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea4167f740 10 client.0 _ll_register_callbacks cb 0x55ff0e3428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea3d843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea3d843640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51840/0 (socket says 172.21.15.5:51840) 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.219+0000 7fea3d843640 1 -- 172.21.15.5:0/3452540109 learned_addr learned my addr 172.21.15.5:0/3452540109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea3d843640 1 -- 172.21.15.5:0/3452540109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6cfb40 msgr2=0x55ff0e6d6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea3d843640 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6d6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea3d843640 1 -- 172.21.15.5:0/3452540109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6ce5d0 msgr2=0x55ff0e6d8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea3d042640 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6d8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea3d843640 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6d8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea3d843640 1 -- 172.21.15.5:0/3452540109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff0e3a53c0 con 0x55ff0e6d9370 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea3d042640 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea3d843640 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d6640 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fea30006010 tx=0x7fea300046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e47f4e853218e7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea1cff9640 1 -- 172.21.15.5:0/3452540109 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3002b070 con 0x55ff0e6d9370 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea1cff9640 1 -- 172.21.15.5:0/3452540109 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fea3000db20 con 0x55ff0e6d9370 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea4167f740 1 -- 172.21.15.5:0/3452540109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea30011260 con 0x55ff0e6d9370 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea1cff9640 1 -- 172.21.15.5:0/3452540109 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea3000dd40 con 0x55ff0e6d9370 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea4167f740 10 client.16680 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea4167f740 1 -- 172.21.15.5:0/3452540109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ff0e6e7b30 con 0x55ff0e6d9370 2024-09-11T22:26:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.220+0000 7fea1cff9640 1 -- 172.21.15.5:0/3452540109 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea30011260 con 0x55ff0e6d9370 2024-09-11T22:26:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.221+0000 7fea1cff9640 1 -- 172.21.15.5:0/3452540109 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fea30030690 con 0x55ff0e6d9370 2024-09-11T22:26:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.221+0000 7fea4167f740 10 client.16680 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.221+0000 7fea4167f740 10 client.16680 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:26:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.221+0000 7fea4167f740 20 client.16680 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.221+0000 7fea4167f740 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ff0e6e8ff0 0x55ff0e6eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.221+0000 7fea4167f740 4 client.16680 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.221+0000 7fea4167f740 1 -- 172.21.15.5:0/3452540109 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff0e641860 con 0x55ff0e6e8ff0 2024-09-11T22:26:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.221+0000 7fea3d042640 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ff0e6e8ff0 0x55ff0e6eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.224+0000 7fea3d042640 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ff0e6e8ff0 0x55ff0e6eb3e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e538b79ec85eec93 server_cookie=6ee1f539adcf091a in_seq=0 out_seq=0 2024-09-11T22:26:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.224+0000 7fea1cff9640 10 client.16680 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:26:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.225+0000 7fea1cff9640 1 -- 172.21.15.5:0/3452540109 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff0e641860 con 0x55ff0e6e8ff0 2024-09-11T22:26:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.225+0000 7fea1cff9640 10 client.16680 handle_command_reply: tid=0 2024-09-11T22:26:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.225+0000 7fea4167f740 1 client.16680 shutdown 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 1 -- 172.21.15.5:0/3452540109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6d9370 msgr2=0x55ff0e6d6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d6640 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fea30006010 tx=0x7fea300046d0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 1 -- 172.21.15.5:0/3452540109 shutdown_connections 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ff0e6e8ff0 0x55ff0e6eb3e0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d6640 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6d6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 1 --2- 172.21.15.5:0/3452540109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 1 -- 172.21.15.5:0/3452540109 >> 172.21.15.5:0/3452540109 conn(0x55ff0e697310 msgr2=0x55ff0e6d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea1cff9640 0 client.16680 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 1 -- 172.21.15.5:0/3452540109 shutdown_connections 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 1 -- 172.21.15.5:0/3452540109 wait complete. 2024-09-11T22:26:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.226+0000 7fea4167f740 20 client.16680 trim_cache size 0 max 16384 2024-09-11T22:26:22.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea4167f740 1 Processor -- start 2024-09-11T22:26:22.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea4167f740 1 -- start start 2024-09-11T22:26:22.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea4167f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6e52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea4167f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea4167f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea4167f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea381afb50 con 0x55ff0e6ce5d0 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea4167f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea381b1690 con 0x55ff0e6cfb40 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea4167f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea381b0220 con 0x55ff0e6d9370 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea3d042640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6e52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea3d042640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6e52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51856/0 (socket says 172.21.15.5:51856) 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea3d042640 1 -- 172.21.15.5:0/350003089 learned_addr learned my addr 172.21.15.5:0/350003089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea3d042640 1 -- 172.21.15.5:0/350003089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6d9370 msgr2=0x55ff0e6d1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea3d042640 1 --2- 172.21.15.5:0/350003089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea3d042640 1 -- 172.21.15.5:0/350003089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6cfb40 msgr2=0x55ff0e6d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea3d042640 1 --2- 172.21.15.5:0/350003089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.239+0000 7fea3d042640 1 -- 172.21.15.5:0/350003089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff0e6c9310 con 0x55ff0e6ce5d0 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea3d042640 1 --2- 172.21.15.5:0/350003089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6e52b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fea28009ca0 tx=0x7fea28005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b9cebaea81d30d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea1effd640 1 -- 172.21.15.5:0/350003089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28031650 con 0x55ff0e6ce5d0 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea1effd640 1 -- 172.21.15.5:0/350003089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea28036040 con 0x55ff0e6ce5d0 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea1effd640 1 -- 172.21.15.5:0/350003089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28030820 con 0x55ff0e6ce5d0 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea4167f740 1 -- 172.21.15.5:0/350003089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6ce5d0 msgr2=0x55ff0e6e52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea4167f740 1 --2- 172.21.15.5:0/350003089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6e52b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fea28009ca0 tx=0x7fea28005fa0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea4167f740 1 -- 172.21.15.5:0/350003089 shutdown_connections 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea4167f740 1 --2- 172.21.15.5:0/350003089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea4167f740 1 --2- 172.21.15.5:0/350003089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea4167f740 1 --2- 172.21.15.5:0/350003089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6e52b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.240+0000 7fea4167f740 1 -- 172.21.15.5:0/350003089 >> 172.21.15.5:0/350003089 conn(0x55ff0e697310 msgr2=0x55ff0e6d5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 1 -- 172.21.15.5:0/350003089 shutdown_connections 2024-09-11T22:26:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 1 -- 172.21.15.5:0/350003089 wait complete. 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 1 -- start start 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 1 Processor -- start 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6e1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea381b0fc0 con 0x55ff0e6cfb40 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea381b08f0 con 0x55ff0e6d9370 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fea381adb30 con 0x55ff0e6ce5d0 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea4167f740 10 client.0 _ll_register_callbacks cb 0x7fea28005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.241+0000 7fea3c841640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6e1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3c841640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6e1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51872/0 (socket says 172.21.15.5:51872) 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3d042640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3c841640 1 -- 172.21.15.5:0/1071641716 learned_addr learned my addr 172.21.15.5:0/1071641716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3d843640 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3c841640 1 -- 172.21.15.5:0/1071641716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6ce5d0 msgr2=0x55ff0e6e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3c841640 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6e1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3c841640 1 -- 172.21.15.5:0/1071641716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 msgr2=0x55ff0e6d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3c841640 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3c841640 1 -- 172.21.15.5:0/1071641716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff0e6d2d60 con 0x55ff0e6cfb40 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3d843640 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea3c841640 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6e1b40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009610 tx=0x7fea2c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743d75750ce0924c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea1cff9640 1 -- 172.21.15.5:0/1071641716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0b4060 con 0x55ff0e6cfb40 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea1cff9640 1 -- 172.21.15.5:0/1071641716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fea2c009cc0 con 0x55ff0e6cfb40 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea4167f740 1 -- 172.21.15.5:0/1071641716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea28036040 con 0x55ff0e6cfb40 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea1cff9640 1 -- 172.21.15.5:0/1071641716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fea2c09ae80 con 0x55ff0e6cfb40 2024-09-11T22:26:22.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea4167f740 10 client.16686 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea4167f740 1 -- 172.21.15.5:0/1071641716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ff0e7533c0 con 0x55ff0e6cfb40 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.242+0000 7fea1cff9640 1 -- 172.21.15.5:0/1071641716 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea28036040 con 0x55ff0e6cfb40 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.243+0000 7fea1cff9640 1 -- 172.21.15.5:0/1071641716 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fea2c0cc740 con 0x55ff0e6cfb40 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.243+0000 7fea4167f740 10 client.16686 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.243+0000 7fea4167f740 10 client.16686 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.243+0000 7fea4167f740 20 client.16686 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.243+0000 7fea4167f740 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ff0e753ac0 0x55ff0e755ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.243+0000 7fea4167f740 4 client.16686 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.243+0000 7fea4167f740 1 -- 172.21.15.5:0/1071641716 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff0e63bd70 con 0x55ff0e753ac0 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.243+0000 7fea3d042640 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ff0e753ac0 0x55ff0e755ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.243+0000 7fea3d042640 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ff0e753ac0 0x55ff0e755ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=195f2321847aa7dc server_cookie=aff9c48cadc977cf in_seq=0 out_seq=0 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.243+0000 7fea1cff9640 10 client.16686 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea1cff9640 1 -- 172.21.15.5:0/1071641716 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff0e63bd70 con 0x55ff0e753ac0 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea1cff9640 10 client.16686 handle_command_reply: tid=0 2024-09-11T22:26:22.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 client.16686 shutdown 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 -- 172.21.15.5:0/1071641716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 msgr2=0x55ff0e6e1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6e1b40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009610 tx=0x7fea2c01e000 comp rx=0 tx=0).stop 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 -- 172.21.15.5:0/1071641716 shutdown_connections 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ff0e753ac0 0x55ff0e755ee0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff0e6d9370 0x55ff0e6d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff0e6cfb40 0x55ff0e6e1b40 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 --2- 172.21.15.5:0/1071641716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff0e6ce5d0 0x55ff0e6e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 -- 172.21.15.5:0/1071641716 >> 172.21.15.5:0/1071641716 conn(0x55ff0e697310 msgr2=0x55ff0e6d7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea1cff9640 0 client.16686 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 -- 172.21.15.5:0/1071641716 shutdown_connections 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 1 -- 172.21.15.5:0/1071641716 wait complete. 2024-09-11T22:26:22.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.244+0000 7fea4167f740 20 client.16686 trim_cache size 0 max 16384 2024-09-11T22:26:22.264 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:26:22.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 -- 172.21.15.5:0/777497175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fea200a2d70 msgr2=0x7fea200a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:22.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fea200a2d70 0x7fea200a5230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea3000f790 tx=0x7fea3001c000 comp rx=0 tx=0).stop 2024-09-11T22:26:22.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 -- 172.21.15.5:0/777497175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea38101280 msgr2=0x7fea3819ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:22.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea38101280 0x7fea3819ac10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00d770 tx=0x7fea2c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:26:22.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 -- 172.21.15.5:0/777497175 shutdown_connections 2024-09-11T22:26:22.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea3810e480 0x7fea3819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea38101280 0x7fea3819ac10 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fea200a2d70 0x7fea200a5230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 --2- 172.21.15.5:0/777497175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea381008d0 0x7fea3819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:22.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 -- 172.21.15.5:0/777497175 >> 172.21.15.5:0/777497175 conn(0x7fea380fc7a0 msgr2=0x7fea3810f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:22.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 -- 172.21.15.5:0/777497175 shutdown_connections 2024-09-11T22:26:22.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:22.245+0000 7fea3f2a6640 1 -- 172.21.15.5:0/777497175 wait complete. 2024-09-11T22:26:22.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 13.22 MB/sec execute 40 sec latency 1420.382 ms 2024-09-11T22:26:23.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 12.89 MB/sec execute 41 sec latency 2420.490 ms 2024-09-11T22:26:23.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:26:23.545 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:23 smithi005 ceph-mon[30943]: pgmap v129: 97 pgs: 97 active+clean; 78 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 461 op/s 2024-09-11T22:26:23.578 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:23 smithi159 ceph-mon[35534]: pgmap v129: 97 pgs: 97 active+clean; 78 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 461 op/s 2024-09-11T22:26:23.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:23 smithi172 ceph-mon[35461]: pgmap v129: 97 pgs: 97 active+clean; 78 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 5.1 MiB/s wr, 461 op/s 2024-09-11T22:26:23.644 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 -- 172.21.15.5:0/2260326784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad1c100a80 msgr2=0x7fad1c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 --2- 172.21.15.5:0/2260326784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad1c100a80 0x7fad1c100ee0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009930 tx=0x7fad0c02a420 comp rx=0 tx=0).stop 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 -- 172.21.15.5:0/2260326784 shutdown_connections 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 --2- 172.21.15.5:0/2260326784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c101420 0x7fad1c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 --2- 172.21.15.5:0/2260326784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad1c100a80 0x7fad1c100ee0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 --2- 172.21.15.5:0/2260326784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c106a80 0x7fad1c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 -- 172.21.15.5:0/2260326784 >> 172.21.15.5:0/2260326784 conn(0x7fad1c0fc7a0 msgr2=0x7fad1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 -- 172.21.15.5:0/2260326784 shutdown_connections 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 -- 172.21.15.5:0/2260326784 wait complete. 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 Processor -- start 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.139+0000 7fad23f38640 1 -- start start 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad23f38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c100a80 0x7fad1c0fce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad23f38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad1c101420 0x7fad1c0fd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad23f38640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c106a80 0x7fad1c0fd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad23f38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c110e80 con 0x7fad1c101420 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad23f38640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c110d00 con 0x7fad1c106a80 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad23f38640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c111000 con 0x7fad1c100a80 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad214d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad1c101420 0x7fad1c0fd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad214d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad1c101420 0x7fad1c0fd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51898/0 (socket says 172.21.15.5:51898) 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad214d3640 1 -- 172.21.15.5:0/2913601327 learned_addr learned my addr 172.21.15.5:0/2913601327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad21cd4640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c100a80 0x7fad1c0fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad214d3640 1 -- 172.21.15.5:0/2913601327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c100a80 msgr2=0x7fad1c0fce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad214d3640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c100a80 0x7fad1c0fce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad214d3640 1 -- 172.21.15.5:0/2913601327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c106a80 msgr2=0x7fad1c0fd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad214d3640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c106a80 0x7fad1c0fd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad214d3640 1 -- 172.21.15.5:0/2913601327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1c1a68e0 con 0x7fad1c101420 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad21cd4640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c100a80 0x7fad1c0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:24.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad214d3640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad1c101420 0x7fad1c0fd340 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009230 tx=0x7fad0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad12ffd640 1 -- 172.21.15.5:0/2913601327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c03f030 con 0x7fad1c101420 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad23f38640 1 -- 172.21.15.5:0/2913601327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1c1a6b70 con 0x7fad1c101420 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad12ffd640 1 -- 172.21.15.5:0/2913601327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad0c003ad0 con 0x7fad1c101420 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.140+0000 7fad12ffd640 1 -- 172.21.15.5:0/2913601327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0c035070 con 0x7fad1c101420 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.141+0000 7fad23f38640 1 -- 172.21.15.5:0/2913601327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1c1a7050 con 0x7fad1c101420 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.143+0000 7fad12ffd640 1 -- 172.21.15.5:0/2913601327 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fad0c042030 con 0x7fad1c101420 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.143+0000 7fad26311740 1 Processor -- start 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.143+0000 7fad26311740 1 -- start start 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad26311740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f129090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad26311740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f129a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad26311740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 0x55795f136760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad26311740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55795f08f2f0 con 0x55795f1295d0 2024-09-11T22:26:24.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad26311740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55795f036cf0 con 0x55795f12ab40 2024-09-11T22:26:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad26311740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55795f08b2b0 con 0x55795f134370 2024-09-11T22:26:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad12ffd640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fad080a2cf0 0x7fad080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad21cd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f129a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad21cd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f129a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51912/0 (socket says 172.21.15.5:51912) 2024-09-11T22:26:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad21cd4640 1 -- 172.21.15.5:0/740289354 learned_addr learned my addr 172.21.15.5:0/740289354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad224d5640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fad080a2cf0 0x7fad080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad12ffd640 1 -- 172.21.15.5:0/2913601327 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fad0c0efb80 con 0x7fad1c101420 2024-09-11T22:26:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad224d5640 1 --2- 172.21.15.5:0/740289354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f129090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad214d3640 1 --2- 172.21.15.5:0/740289354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 0x55795f136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad21cd4640 1 -- 172.21.15.5:0/740289354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 msgr2=0x55795f136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad21cd4640 1 --2- 172.21.15.5:0/740289354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 0x55795f136760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad21cd4640 1 -- 172.21.15.5:0/740289354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 msgr2=0x55795f129090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad21cd4640 1 --2- 172.21.15.5:0/740289354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f129090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad21cd4640 1 -- 172.21.15.5:0/740289354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55795f01b3a0 con 0x55795f1295d0 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad214d3640 1 --2- 172.21.15.5:0/740289354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 0x55795f136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad21cd4640 1 --2- 172.21.15.5:0/740289354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f129a10 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fad18009c80 tx=0x7fad18010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c277565847b15566 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.144+0000 7fad224d5640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fad080a2cf0 0x7fad080a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fad14005c40 tx=0x7fad140094a0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.145+0000 7fad07fff640 1 -- 172.21.15.5:0/740289354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18004030 con 0x55795f1295d0 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.145+0000 7fad07fff640 1 -- 172.21.15.5:0/740289354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad180041d0 con 0x55795f1295d0 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.145+0000 7fad07fff640 1 -- 172.21.15.5:0/740289354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18012650 con 0x55795f1295d0 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.145+0000 7fad26311740 1 -- 172.21.15.5:0/740289354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 msgr2=0x55795f129a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:24.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.145+0000 7fad26311740 1 --2- 172.21.15.5:0/740289354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f129a10 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fad18009c80 tx=0x7fad18010740 comp rx=0 tx=0).stop 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.145+0000 7fad26311740 1 -- 172.21.15.5:0/740289354 shutdown_connections 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.145+0000 7fad26311740 1 --2- 172.21.15.5:0/740289354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 0x55795f136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.145+0000 7fad26311740 1 --2- 172.21.15.5:0/740289354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f129a10 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.145+0000 7fad26311740 1 --2- 172.21.15.5:0/740289354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f129090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.145+0000 7fad26311740 1 -- 172.21.15.5:0/740289354 >> 172.21.15.5:0/740289354 conn(0x55795f0f2310 msgr2=0x55795f125c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.146+0000 7fad26311740 1 -- 172.21.15.5:0/740289354 shutdown_connections 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.146+0000 7fad26311740 1 -- 172.21.15.5:0/740289354 wait complete. 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.146+0000 7fad26311740 1 -- start start 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.146+0000 7fad26311740 1 Processor -- start 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad26311740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f1295d0 0x55795f133d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad26311740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f130fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad26311740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f1315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad26311740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55795f137070 con 0x55795f134370 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad26311740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55795f04a060 con 0x55795f12ab40 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad26311740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55795f1371f0 con 0x55795f1295d0 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad26311740 10 client.0 _ll_register_callbacks cb 0x55795ed9d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad224d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f1315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad224d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f1315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51920/0 (socket says 172.21.15.5:51920) 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad224d5640 1 -- 172.21.15.5:0/624208621 learned_addr learned my addr 172.21.15.5:0/624208621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad214d3640 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f130fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad21cd4640 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f1295d0 0x55795f133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad224d5640 1 -- 172.21.15.5:0/624208621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f1295d0 msgr2=0x55795f133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad224d5640 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f1295d0 0x55795f133d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad224d5640 1 -- 172.21.15.5:0/624208621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 msgr2=0x55795f130fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad224d5640 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f130fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad224d5640 1 -- 172.21.15.5:0/624208621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55795ee003c0 con 0x55795f134370 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad224d5640 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f1315d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fad14012480 tx=0x7fad14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcd49a2b6082de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad057fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad057fa640 1 -- 172.21.15.5:0/624208621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14004d20 con 0x55795f134370 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad057fa640 1 -- 172.21.15.5:0/624208621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fad14010b10 con 0x55795f134370 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.147+0000 7fad26311740 1 -- 172.21.15.5:0/624208621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad180041d0 con 0x55795f134370 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad057fa640 1 -- 172.21.15.5:0/624208621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fad14010d10 con 0x55795f134370 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad26311740 10 client.16698 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad26311740 1 -- 172.21.15.5:0/624208621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55795f142b20 con 0x55795f134370 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad057fa640 1 -- 172.21.15.5:0/624208621 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad180041d0 con 0x55795f134370 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad057fa640 1 -- 172.21.15.5:0/624208621 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fad140265c0 con 0x55795f134370 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad26311740 10 client.16698 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad26311740 10 client.16698 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:26:24.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad26311740 20 client.16698 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad26311740 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55795f143fe0 0x55795f1463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad26311740 4 client.16698 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad26311740 1 -- 172.21.15.5:0/624208621 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55795f09c860 con 0x55795f143fe0 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.148+0000 7fad21cd4640 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55795f143fe0 0x55795f1463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.149+0000 7fad21cd4640 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55795f143fe0 0x55795f1463d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed8f7e2b190e3f3e server_cookie=5f8f158d57650c05 in_seq=0 out_seq=0 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.149+0000 7fad057fa640 10 client.16698 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.149+0000 7fad057fa640 1 -- 172.21.15.5:0/624208621 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55795f09c860 con 0x55795f143fe0 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.149+0000 7fad057fa640 10 client.16698 handle_command_reply: tid=0 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.149+0000 7fad26311740 1 client.16698 shutdown 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 1 -- 172.21.15.5:0/624208621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 msgr2=0x55795f1315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f1315d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fad14012480 tx=0x7fad14004a40 comp rx=0 tx=0).stop 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 1 -- 172.21.15.5:0/624208621 shutdown_connections 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55795f143fe0 0x55795f1463d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f1315d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad057fa640 0 client.16698 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f130fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 1 --2- 172.21.15.5:0/624208621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f1295d0 0x55795f133d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 1 -- 172.21.15.5:0/624208621 >> 172.21.15.5:0/624208621 conn(0x55795f0f2310 msgr2=0x55795f12b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 1 -- 172.21.15.5:0/624208621 shutdown_connections 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 1 -- 172.21.15.5:0/624208621 wait complete. 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.150+0000 7fad26311740 20 client.16698 trim_cache size 0 max 16384 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad26311740 1 Processor -- start 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad26311740 1 -- start start 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad26311740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f125e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad26311740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad26311740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 0x55795f1308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad26311740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1afb80 con 0x55795f1295d0 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad26311740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1b16c0 con 0x55795f12ab40 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad26311740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1b0250 con 0x55795f134370 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad21cd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f125e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad21cd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f125e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51934/0 (socket says 172.21.15.5:51934) 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad21cd4640 1 -- 172.21.15.5:0/879600225 learned_addr learned my addr 172.21.15.5:0/879600225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.163+0000 7fad214d3640 1 --2- 172.21.15.5:0/879600225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.164+0000 7fad224d5640 1 --2- 172.21.15.5:0/879600225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 0x55795f1308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.164+0000 7fad21cd4640 1 -- 172.21.15.5:0/879600225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 msgr2=0x55795f1308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.164+0000 7fad21cd4640 1 --2- 172.21.15.5:0/879600225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 0x55795f1308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.164+0000 7fad21cd4640 1 -- 172.21.15.5:0/879600225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 msgr2=0x55795f140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.164+0000 7fad21cd4640 1 --2- 172.21.15.5:0/879600225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.164+0000 7fad21cd4640 1 -- 172.21.15.5:0/879600225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55795f12ff90 con 0x55795f1295d0 2024-09-11T22:26:24.192 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:26:24.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.164+0000 7fad21cd4640 1 --2- 172.21.15.5:0/879600225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f125e10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fad18011010 tx=0x7fad18005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd76e6a5bc68bf95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:24.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.164+0000 7fad077fe640 1 -- 172.21.15.5:0/879600225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18005850 con 0x55795f1295d0 2024-09-11T22:26:24.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.164+0000 7fad077fe640 1 -- 172.21.15.5:0/879600225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad180055f0 con 0x55795f1295d0 2024-09-11T22:26:24.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.164+0000 7fad077fe640 1 -- 172.21.15.5:0/879600225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad18012c00 con 0x55795f1295d0 2024-09-11T22:26:24.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 -- 172.21.15.5:0/879600225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 msgr2=0x55795f125e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 --2- 172.21.15.5:0/879600225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f125e10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fad18011010 tx=0x7fad18005f90 comp rx=0 tx=0).stop 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 -- 172.21.15.5:0/879600225 shutdown_connections 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 --2- 172.21.15.5:0/879600225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f134370 0x55795f1308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 --2- 172.21.15.5:0/879600225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f12ab40 0x55795f140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 --2- 172.21.15.5:0/879600225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f1295d0 0x55795f125e10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 -- 172.21.15.5:0/879600225 >> 172.21.15.5:0/879600225 conn(0x55795f0f2310 msgr2=0x55795f125700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 -- 172.21.15.5:0/879600225 shutdown_connections 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 -- 172.21.15.5:0/879600225 wait complete. 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 -- start start 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.165+0000 7fad26311740 1 Processor -- start 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad26311740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f1295d0 0x55795f138270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad26311740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f12ab40 0x55795f1387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad26311740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad26311740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1b0ff0 con 0x55795f134370 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad26311740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1b0920 con 0x55795f1295d0 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad26311740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1adb60 con 0x55795f12ab40 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad26311740 10 client.0 _ll_register_callbacks cb 0x7fad14012100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad224d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad224d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f138cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51948/0 (socket says 172.21.15.5:51948) 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad224d5640 1 -- 172.21.15.5:0/789426218 learned_addr learned my addr 172.21.15.5:0/789426218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad21cd4640 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f1295d0 0x55795f138270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad214d3640 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f12ab40 0x55795f1387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad224d5640 1 -- 172.21.15.5:0/789426218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f12ab40 msgr2=0x55795f1387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad224d5640 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f12ab40 0x55795f1387b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad224d5640 1 -- 172.21.15.5:0/789426218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f1295d0 msgr2=0x55795f138270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:24.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad224d5640 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f1295d0 0x55795f138270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad224d5640 1 -- 172.21.15.5:0/789426218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55795f1340a0 con 0x55795f134370 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad21cd4640 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f1295d0 0x55795f138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad224d5640 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f138cf0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fad14012450 tx=0x7fad14027ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9e6e6f6c526f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.166+0000 7fad057fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad057fa640 1 -- 172.21.15.5:0/789426218 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14032040 con 0x55795f134370 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad26311740 1 -- 172.21.15.5:0/789426218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55795f096d70 con 0x55795f134370 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad057fa640 1 -- 172.21.15.5:0/789426218 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fad14004b20 con 0x55795f134370 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad057fa640 1 -- 172.21.15.5:0/789426218 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fad14004d20 con 0x55795f134370 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad26311740 10 client.16704 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad26311740 1 -- 172.21.15.5:0/789426218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55795f1aed90 con 0x55795f134370 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad057fa640 1 -- 172.21.15.5:0/789426218 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55795f096d70 con 0x55795f134370 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad057fa640 1 -- 172.21.15.5:0/789426218 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fad1403d020 con 0x55795f134370 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad26311740 10 client.16704 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad26311740 10 client.16704 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad26311740 20 client.16704 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad26311740 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55795f1af500 0x55795f1b1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad26311740 4 client.16704 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.167+0000 7fad26311740 1 -- 172.21.15.5:0/789426218 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55795f013170 con 0x55795f1af500 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad21cd4640 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55795f1af500 0x55795f1b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad057fa640 10 client.16704 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad21cd4640 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55795f1af500 0x55795f1b1920 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fbe596f1cc9f4c79 server_cookie=e48ea0638d5103d7 in_seq=0 out_seq=0 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad057fa640 1 -- 172.21.15.5:0/789426218 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55795f013170 con 0x55795f1af500 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad057fa640 10 client.16704 handle_command_reply: tid=0 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad26311740 1 client.16704 shutdown 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad26311740 1 -- 172.21.15.5:0/789426218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 msgr2=0x55795f138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad26311740 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f138cf0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fad14012450 tx=0x7fad14027ee0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad26311740 1 -- 172.21.15.5:0/789426218 shutdown_connections 2024-09-11T22:26:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad26311740 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55795f1af500 0x55795f1b1920 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad26311740 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55795f134370 0x55795f138cf0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad057fa640 0 client.16704 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:26:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad26311740 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55795f12ab40 0x55795f1387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad26311740 1 --2- 172.21.15.5:0/789426218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55795f1295d0 0x55795f138270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.168+0000 7fad26311740 1 -- 172.21.15.5:0/789426218 >> 172.21.15.5:0/789426218 conn(0x55795f0f2310 msgr2=0x55795f12b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:24.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad26311740 1 -- 172.21.15.5:0/789426218 shutdown_connections 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad26311740 1 -- 172.21.15.5:0/789426218 wait complete. 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad26311740 20 client.16704 trim_cache size 0 max 16384 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 -- 172.21.15.5:0/2913601327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fad080a2cf0 msgr2=0x7fad080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fad080a2cf0 0x7fad080a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fad14005c40 tx=0x7fad140094a0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 -- 172.21.15.5:0/2913601327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad1c101420 msgr2=0x7fad1c0fd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad1c101420 0x7fad1c0fd340 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009230 tx=0x7fad0c004270 comp rx=0 tx=0).stop 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 -- 172.21.15.5:0/2913601327 shutdown_connections 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fad080a2cf0 0x7fad080a51b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad1c106a80 0x7fad1c0fd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad1c101420 0x7fad1c0fd340 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 --2- 172.21.15.5:0/2913601327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c100a80 0x7fad1c0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 -- 172.21.15.5:0/2913601327 >> 172.21.15.5:0/2913601327 conn(0x7fad1c0fc7a0 msgr2=0x7fad1c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 -- 172.21.15.5:0/2913601327 shutdown_connections 2024-09-11T22:26:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:24.169+0000 7fad23f38640 1 -- 172.21.15.5:0/2913601327 wait complete. 2024-09-11T22:26:24.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 12.59 MB/sec execute 42 sec latency 3420.567 ms 2024-09-11T22:26:25.159 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:26:25.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 12.30 MB/sec execute 43 sec latency 4420.617 ms 2024-09-11T22:26:25.432 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:25.553 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:25 smithi005 ceph-mon[30943]: pgmap v130: 97 pgs: 97 active+clean; 78 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 331 op/s 2024-09-11T22:26:25.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:25 smithi159 ceph-mon[35534]: pgmap v130: 97 pgs: 97 active+clean; 78 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 331 op/s 2024-09-11T22:26:25.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:25 smithi172 ceph-mon[35461]: pgmap v130: 97 pgs: 97 active+clean; 78 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 331 op/s 2024-09-11T22:26:25.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.957+0000 7f95b92f2640 1 -- 172.21.15.5:0/770434840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95b4106a80 msgr2=0x7f95b4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:25.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.957+0000 7f95b92f2640 1 --2- 172.21.15.5:0/770434840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95b4106a80 0x7f95b4106e60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009930 tx=0x7f95a402a420 comp rx=0 tx=0).stop 2024-09-11T22:26:25.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.957+0000 7f95b92f2640 1 -- 172.21.15.5:0/770434840 shutdown_connections 2024-09-11T22:26:25.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.957+0000 7f95b92f2640 1 --2- 172.21.15.5:0/770434840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95b4101420 0x7f95b410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.957+0000 7f95b92f2640 1 --2- 172.21.15.5:0/770434840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95b4100a80 0x7f95b4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.957+0000 7f95b92f2640 1 --2- 172.21.15.5:0/770434840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95b4106a80 0x7f95b4106e60 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.957+0000 7f95b92f2640 1 -- 172.21.15.5:0/770434840 >> 172.21.15.5:0/770434840 conn(0x7f95b40fc7a0 msgr2=0x7f95b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.957+0000 7f95b92f2640 1 -- 172.21.15.5:0/770434840 shutdown_connections 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.957+0000 7f95b92f2640 1 -- 172.21.15.5:0/770434840 wait complete. 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.958+0000 7f95b92f2640 1 Processor -- start 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.958+0000 7f95b92f2640 1 -- start start 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.958+0000 7f95b92f2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95b4100a80 0x7f95b41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.958+0000 7f95b92f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95b4101420 0x7f95b41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.958+0000 7f95b92f2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95b4106a80 0x7f95b41a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.958+0000 7f95b92f2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4110e80 con 0x7f95b4101420 2024-09-11T22:26:25.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.958+0000 7f95b92f2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4110d00 con 0x7f95b4100a80 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.958+0000 7f95b92f2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4111000 con 0x7f95b4106a80 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95b4101420 0x7f95b41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95b4101420 0x7f95b41997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51958/0 (socket says 172.21.15.5:51958) 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b27fc640 1 -- 172.21.15.5:0/1997169174 learned_addr learned my addr 172.21.15.5:0/1997169174 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95b4100a80 0x7f95b41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b27fc640 1 -- 172.21.15.5:0/1997169174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95b4106a80 msgr2=0x7f95b41a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b27fc640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95b4106a80 0x7f95b41a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b27fc640 1 -- 172.21.15.5:0/1997169174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95b4100a80 msgr2=0x7f95b41992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b27fc640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95b4100a80 0x7f95b41992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b27fc640 1 -- 172.21.15.5:0/1997169174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b41a4870 con 0x7f95b4101420 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95b4100a80 0x7f95b41992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:25.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b27fc640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95b4101420 0x7f95b41997f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f95a800d950 tx=0x7f95a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f959bfff640 1 -- 172.21.15.5:0/1997169174 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a8017030 con 0x7f95b4101420 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b92f2640 1 -- 172.21.15.5:0/1997169174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b41a4a40 con 0x7f95b4101420 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f959bfff640 1 -- 172.21.15.5:0/1997169174 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a8004020 con 0x7f95b4101420 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f959bfff640 1 -- 172.21.15.5:0/1997169174 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a80047a0 con 0x7f95b4101420 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.959+0000 7f95b92f2640 1 -- 172.21.15.5:0/1997169174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b41a4fd0 con 0x7f95b4101420 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.962+0000 7f959bfff640 1 -- 172.21.15.5:0/1997169174 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f95a80041d0 con 0x7f95b4101420 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.962+0000 7f95bb6cb740 1 Processor -- start 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.962+0000 7f95bb6cb740 1 -- start start 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.962+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2cff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.962+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2cff5d0 0x561dc2cffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.962+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2d0e780 0x561dc2d10b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.962+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561dc2c652f0 con 0x561dc2cff5d0 2024-09-11T22:26:25.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.962+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561dc2c0ccf0 con 0x561dc2d00b40 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.962+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561dc2c612b0 con 0x561dc2d0e780 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2cff5d0 0x561dc2cffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f959bfff640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f959c0a2cf0 0x7f959c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2cff5d0 0x561dc2cffa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51960/0 (socket says 172.21.15.5:51960) 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b2ffd640 1 -- 172.21.15.5:0/3183659200 learned_addr learned my addr 172.21.15.5:0/3183659200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b27fc640 1 --2- 172.21.15.5:0/3183659200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2d0e780 0x561dc2d10b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b37fe640 1 --2- 172.21.15.5:0/3183659200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2cff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f959bfff640 1 -- 172.21.15.5:0/1997169174 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f95a80cef30 con 0x7f95b4101420 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b37fe640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f959c0a2cf0 0x7f959c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b2ffd640 1 -- 172.21.15.5:0/3183659200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2d0e780 msgr2=0x561dc2d10b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/3183659200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2d0e780 0x561dc2d10b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b2ffd640 1 -- 172.21.15.5:0/3183659200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 msgr2=0x561dc2cff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:25.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/3183659200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2cff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b2ffd640 1 -- 172.21.15.5:0/3183659200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc2bf13a0 con 0x561dc2cff5d0 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/3183659200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2cff5d0 0x561dc2cffa10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009930 tx=0x7f95a4005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b13b26d08011096 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95b37fe640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f959c0a2cf0 0x7f959c0a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f95ac005fd0 tx=0x7f95ac004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95997fa640 1 -- 172.21.15.5:0/3183659200 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a403f070 con 0x561dc2cff5d0 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95997fa640 1 -- 172.21.15.5:0/3183659200 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a4004030 con 0x561dc2cff5d0 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.963+0000 7f95997fa640 1 -- 172.21.15.5:0/3183659200 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4035070 con 0x561dc2cff5d0 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.964+0000 7f95bb6cb740 1 -- 172.21.15.5:0/3183659200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2cff5d0 msgr2=0x561dc2cffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.964+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/3183659200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2cff5d0 0x561dc2cffa10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009930 tx=0x7f95a4005e60 comp rx=0 tx=0).stop 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.964+0000 7f95bb6cb740 1 -- 172.21.15.5:0/3183659200 shutdown_connections 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.964+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/3183659200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2d0e780 0x561dc2d10b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.964+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/3183659200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2cff5d0 0x561dc2cffa10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.964+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/3183659200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2cff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.964+0000 7f95bb6cb740 1 -- 172.21.15.5:0/3183659200 >> 172.21.15.5:0/3183659200 conn(0x561dc2cc8310 msgr2=0x561dc2cfbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.964+0000 7f95bb6cb740 1 -- 172.21.15.5:0/3183659200 shutdown_connections 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.964+0000 7f95bb6cb740 1 -- 172.21.15.5:0/3183659200 wait complete. 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.964+0000 7f95bb6cb740 1 -- start start 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95bb6cb740 1 Processor -- start 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 0x561dc2d0e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2d0b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561dc2d11420 con 0x561dc2d0e780 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561dc2c20060 con 0x561dc2d00b40 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561dc2d115a0 con 0x561dc2cff5d0 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95bb6cb740 10 client.0 _ll_register_callbacks cb 0x561dc29738c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:25.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51968/0 (socket says 172.21.15.5:51968) 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b37fe640 1 -- 172.21.15.5:0/1090745308 learned_addr learned my addr 172.21.15.5:0/1090745308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 0x561dc2d0e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b27fc640 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2d0b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b37fe640 1 -- 172.21.15.5:0/1090745308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 msgr2=0x561dc2d0e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b37fe640 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 0x561dc2d0e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b37fe640 1 -- 172.21.15.5:0/1090745308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 msgr2=0x561dc2d0b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b37fe640 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2d0b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b37fe640 1 -- 172.21.15.5:0/1090745308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc29d63c0 con 0x561dc2d0e780 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.965+0000 7f95b27fc640 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2d0b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f95b37fe640 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0b970 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f95ac004300 tx=0x7f95ac009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b7dd477d623728 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f9596ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:25.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f9596ff5640 1 -- 172.21.15.5:0/1090745308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00ec30 con 0x561dc2d0e780 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f9596ff5640 1 -- 172.21.15.5:0/1090745308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f95ac00edd0 con 0x561dc2d0e780 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1090745308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a4004030 con 0x561dc2d0e780 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f9596ff5640 1 -- 172.21.15.5:0/1090745308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f95ac02d050 con 0x561dc2d0e780 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f95bb6cb740 10 client.16716 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1090745308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561dc2d1ced0 con 0x561dc2d0e780 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f9596ff5640 1 -- 172.21.15.5:0/1090745308 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a4004030 con 0x561dc2d0e780 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f9596ff5640 1 -- 172.21.15.5:0/1090745308 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f95ac00f040 con 0x561dc2d0e780 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f95bb6cb740 10 client.16716 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f95bb6cb740 10 client.16716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f95bb6cb740 20 client.16716 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561dc2d1e390 0x561dc2d20830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f95bb6cb740 4 client.16716 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:25.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.966+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1090745308 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dc2c72860 con 0x561dc2d1e390 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.967+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561dc2d1e390 0x561dc2d20830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.967+0000 7f9596ff5640 10 client.16716 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.967+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561dc2d1e390 0x561dc2d20830 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3f4b98d9fd7e94 server_cookie=cda1047d55dae831 in_seq=0 out_seq=0 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f9596ff5640 1 -- 172.21.15.5:0/1090745308 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561dc2c72860 con 0x561dc2d1e390 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f9596ff5640 10 client.16716 handle_command_reply: tid=0 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 client.16716 shutdown 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1090745308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 msgr2=0x561dc2d0b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0b970 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f95ac004300 tx=0x7f95ac009500 comp rx=0 tx=0).stop 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1090745308 shutdown_connections 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561dc2d1e390 0x561dc2d20830 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0b970 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2d0b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1090745308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 0x561dc2d0e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f9596ff5640 0 client.16716 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1090745308 >> 172.21.15.5:0/1090745308 conn(0x561dc2cc8310 msgr2=0x561dc2d01930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1090745308 shutdown_connections 2024-09-11T22:26:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1090745308 wait complete. 2024-09-11T22:26:25.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.968+0000 7f95bb6cb740 20 client.16716 trim_cache size 0 max 16384 2024-09-11T22:26:25.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95bb6cb740 1 Processor -- start 2024-09-11T22:26:25.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95bb6cb740 1 -- start start 2024-09-11T22:26:25.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 0x561dc2cfbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2d1a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41ab8a0 con 0x561dc2d0e780 2024-09-11T22:26:25.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41ad3e0 con 0x561dc2d00b40 2024-09-11T22:26:25.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41abf70 con 0x561dc2cff5d0 2024-09-11T22:26:25.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51982/0 (socket says 172.21.15.5:51982) 2024-09-11T22:26:25.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b37fe640 1 -- 172.21.15.5:0/752540176 learned_addr learned my addr 172.21.15.5:0/752540176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:25.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 0x561dc2cfbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b27fc640 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2d1a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b37fe640 1 -- 172.21.15.5:0/752540176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 msgr2=0x561dc2cfbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b37fe640 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 0x561dc2cfbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b37fe640 1 -- 172.21.15.5:0/752540176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 msgr2=0x561dc2d1a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b37fe640 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2d1a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b37fe640 1 -- 172.21.15.5:0/752540176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dc2d0a3c0 con 0x561dc2d0e780 2024-09-11T22:26:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 0x561dc2cfbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.982+0000 7f95b27fc640 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2d1a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.983+0000 7f95b37fe640 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0ac90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0092f0 tx=0x7f95ac009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3009db20edfbef1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.983+0000 7f9598ff9640 1 -- 172.21.15.5:0/752540176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac01f650 con 0x561dc2d0e780 2024-09-11T22:26:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.983+0000 7f9598ff9640 1 -- 172.21.15.5:0/752540176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ac037070 con 0x561dc2d0e780 2024-09-11T22:26:25.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.983+0000 7f9598ff9640 1 -- 172.21.15.5:0/752540176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0297b0 con 0x561dc2d0e780 2024-09-11T22:26:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 -- 172.21.15.5:0/752540176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 msgr2=0x561dc2d0ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0ac90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0092f0 tx=0x7f95ac009960 comp rx=0 tx=0).stop 2024-09-11T22:26:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 -- 172.21.15.5:0/752540176 shutdown_connections 2024-09-11T22:26:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d0e780 0x561dc2d0ac90 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2d00b40 0x561dc2d1a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/752540176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2cff5d0 0x561dc2cfbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 -- 172.21.15.5:0/752540176 >> 172.21.15.5:0/752540176 conn(0x561dc2cc8310 msgr2=0x561dc2cfb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 -- 172.21.15.5:0/752540176 shutdown_connections 2024-09-11T22:26:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 -- 172.21.15.5:0/752540176 wait complete. 2024-09-11T22:26:25.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 -- start start 2024-09-11T22:26:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 Processor -- start 2024-09-11T22:26:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2cff5d0 0x561dc2d16830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d00b40 0x561dc2d16d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2d0e780 0x561dc2d13800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41acd10 con 0x561dc2d00b40 2024-09-11T22:26:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41ac640 con 0x561dc2cff5d0 2024-09-11T22:26:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95b41a9880 con 0x561dc2d0e780 2024-09-11T22:26:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95bb6cb740 10 client.0 _ll_register_callbacks cb 0x7f95ac003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.984+0000 7f95b27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d00b40 0x561dc2d16d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95b27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d00b40 0x561dc2d16d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51994/0 (socket says 172.21.15.5:51994) 2024-09-11T22:26:25.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95b27fc640 1 -- 172.21.15.5:0/1754724310 learned_addr learned my addr 172.21.15.5:0/1754724310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:25.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95b37fe640 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2d0e780 0x561dc2d13800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2cff5d0 0x561dc2d16830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95b27fc640 1 -- 172.21.15.5:0/1754724310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2d0e780 msgr2=0x561dc2d13800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:25.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95b27fc640 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2d0e780 0x561dc2d13800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95b27fc640 1 -- 172.21.15.5:0/1754724310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2cff5d0 msgr2=0x561dc2d16830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:25.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95b27fc640 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2cff5d0 0x561dc2d16830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95b27fc640 1 -- 172.21.15.5:0/1754724310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dc2d17430 con 0x561dc2d00b40 2024-09-11T22:26:25.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95b27fc640 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d00b40 0x561dc2d16d70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f95a80cc6a0 tx=0x7f95a806a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7e3291265b80d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f9596ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f9596ff5640 1 -- 172.21.15.5:0/1754724310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a809b410 con 0x561dc2d00b40 2024-09-11T22:26:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f9596ff5640 1 -- 172.21.15.5:0/1754724310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f95a80a3bf0 con 0x561dc2d00b40 2024-09-11T22:26:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1754724310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95ac037070 con 0x561dc2d00b40 2024-09-11T22:26:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f9596ff5640 1 -- 172.21.15.5:0/1754724310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f95a809f7c0 con 0x561dc2d00b40 2024-09-11T22:26:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95bb6cb740 10 client.16722 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1754724310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561dc2d89660 con 0x561dc2d00b40 2024-09-11T22:26:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.985+0000 7f9596ff5640 1 -- 172.21.15.5:0/1754724310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ac037070 con 0x561dc2d00b40 2024-09-11T22:26:25.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.986+0000 7f9596ff5640 1 -- 172.21.15.5:0/1754724310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f95a80b40f0 con 0x561dc2d00b40 2024-09-11T22:26:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.986+0000 7f95bb6cb740 10 client.16722 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.986+0000 7f95bb6cb740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.986+0000 7f95bb6cb740 20 client.16722 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.986+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561dc2d89d90 0x561dc2d8c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.986+0000 7f95bb6cb740 4 client.16722 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:26:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.986+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1754724310 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561dc2c6cd70 con 0x561dc2d89d90 2024-09-11T22:26:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.986+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561dc2d89d90 0x561dc2d8c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:25.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.986+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561dc2d89d90 0x561dc2d8c1b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29c69710b2b5ca6b server_cookie=412dd5e0cbb2fdd9 in_seq=0 out_seq=0 2024-09-11T22:26:25.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.986+0000 7f9596ff5640 10 client.16722 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:25.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f9596ff5640 1 -- 172.21.15.5:0/1754724310 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561dc2c6cd70 con 0x561dc2d89d90 2024-09-11T22:26:25.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f9596ff5640 10 client.16722 handle_command_reply: tid=0 2024-09-11T22:26:25.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 client.16722 shutdown 2024-09-11T22:26:25.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1754724310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d00b40 msgr2=0x561dc2d16d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:25.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d00b40 0x561dc2d16d70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f95a80cc6a0 tx=0x7f95a806a340 comp rx=0 tx=0).stop 2024-09-11T22:26:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1754724310 shutdown_connections 2024-09-11T22:26:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561dc2d89d90 0x561dc2d8c1b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561dc2d0e780 0x561dc2d13800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561dc2d00b40 0x561dc2d16d70 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 --2- 172.21.15.5:0/1754724310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561dc2cff5d0 0x561dc2d16830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f9596ff5640 0 client.16722 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1754724310 >> 172.21.15.5:0/1754724310 conn(0x561dc2cc8310 msgr2=0x561dc2d01930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1754724310 shutdown_connections 2024-09-11T22:26:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 1 -- 172.21.15.5:0/1754724310 wait complete. 2024-09-11T22:26:25.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95bb6cb740 20 client.16722 trim_cache size 0 max 16384 2024-09-11T22:26:26.000 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95b92f2640 1 -- 172.21.15.5:0/1997169174 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f959c0a2cf0 msgr2=0x7f959c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95b92f2640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f959c0a2cf0 0x7f959c0a51b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f95ac005fd0 tx=0x7f95ac004a40 comp rx=0 tx=0).stop 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95b92f2640 1 -- 172.21.15.5:0/1997169174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95b4101420 msgr2=0x7f95b41997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.987+0000 7f95b92f2640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95b4101420 0x7f95b41997f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f95a800d950 tx=0x7f95a800de10 comp rx=0 tx=0).stop 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.988+0000 7f95b92f2640 1 -- 172.21.15.5:0/1997169174 shutdown_connections 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.988+0000 7f95b92f2640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95b4106a80 0x7f95b41a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.988+0000 7f95b92f2640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95b4101420 0x7f95b41997f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.988+0000 7f95b92f2640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f959c0a2cf0 0x7f959c0a51b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.988+0000 7f95b92f2640 1 --2- 172.21.15.5:0/1997169174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95b4100a80 0x7f95b41992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.988+0000 7f95b92f2640 1 -- 172.21.15.5:0/1997169174 >> 172.21.15.5:0/1997169174 conn(0x7f95b40fc7a0 msgr2=0x7f95b40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:26.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.988+0000 7f95b92f2640 1 -- 172.21.15.5:0/1997169174 shutdown_connections 2024-09-11T22:26:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:25.988+0000 7f95b92f2640 1 -- 172.21.15.5:0/1997169174 wait complete. 2024-09-11T22:26:26.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 12.02 MB/sec execute 44 sec latency 5420.708 ms 2024-09-11T22:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.957+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/281238272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d041035c0 msgr2=0x7f7d0410baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.957+0000 7f7d0b3bd640 1 --2- 172.21.15.5:0/281238272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d041035c0 0x7f7d0410baa0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009900 tx=0x7f7cf402a480 comp rx=0 tx=0).stop 2024-09-11T22:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.957+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/281238272 shutdown_connections 2024-09-11T22:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.957+0000 7f7d0b3bd640 1 --2- 172.21.15.5:0/281238272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d0410c0c0 0x7f7d0410e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.957+0000 7f7d0b3bd640 1 --2- 172.21.15.5:0/281238272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d041035c0 0x7f7d0410baa0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.957+0000 7f7d0b3bd640 1 --2- 172.21.15.5:0/281238272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d04102ca0 0x7f7d04103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.957+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/281238272 >> 172.21.15.5:0/281238272 conn(0x7f7d040fc6f0 msgr2=0x7f7d040feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.957+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/281238272 shutdown_connections 2024-09-11T22:26:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.957+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/281238272 wait complete. 2024-09-11T22:26:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d0b3bd640 1 Processor -- start 2024-09-11T22:26:26.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d0b3bd640 1 -- start start 2024-09-11T22:26:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d0b3bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d04102ca0 0x7f7d04198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d0b3bd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d041035c0 0x7f7d04198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d0b3bd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d0410c0c0 0x7f7d0419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d0b3bd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04111f30 con 0x7f7d04102ca0 2024-09-11T22:26:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d0b3bd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04111db0 con 0x7f7d0410c0c0 2024-09-11T22:26:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d0b3bd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041120b0 con 0x7f7d041035c0 2024-09-11T22:26:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d09159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d04102ca0 0x7f7d04198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d09159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d04102ca0 0x7f7d04198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52016/0 (socket says 172.21.15.5:52016) 2024-09-11T22:26:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d09159640 1 -- 172.21.15.5:0/1324833668 learned_addr learned my addr 172.21.15.5:0/1324833668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d0995a640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d0410c0c0 0x7f7d0419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d09159640 1 -- 172.21.15.5:0/1324833668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d041035c0 msgr2=0x7f7d04198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d08958640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d041035c0 0x7f7d04198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.958+0000 7f7d09159640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d041035c0 0x7f7d04198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7d09159640 1 -- 172.21.15.5:0/1324833668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d0410c0c0 msgr2=0x7f7d0419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7d09159640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d0410c0c0 0x7f7d0419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7d09159640 1 -- 172.21.15.5:0/1324833668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0419d2f0 con 0x7f7d04102ca0 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7d0995a640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d0410c0c0 0x7f7d0419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7d08958640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d041035c0 0x7f7d04198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7d09159640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d04102ca0 0x7f7d04198300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000d770 tx=0x7f7d0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7cfa7fc640 1 -- 172.21.15.5:0/1324833668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00017070 con 0x7f7d04102ca0 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7cfa7fc640 1 -- 172.21.15.5:0/1324833668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d00005500 con 0x7f7d04102ca0 2024-09-11T22:26:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/1324833668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0419d5e0 con 0x7f7d04102ca0 2024-09-11T22:26:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7cfa7fc640 1 -- 172.21.15.5:0/1324833668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00004e00 con 0x7f7d04102ca0 2024-09-11T22:26:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.959+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/1324833668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041a4e70 con 0x7f7d04102ca0 2024-09-11T22:26:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.961+0000 7f7cfa7fc640 1 -- 172.21.15.5:0/1324833668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7d00004fa0 con 0x7f7d04102ca0 2024-09-11T22:26:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.962+0000 7f7d0e235740 1 Processor -- start 2024-09-11T22:26:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.962+0000 7f7d0e235740 1 -- start start 2024-09-11T22:26:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.962+0000 7f7cfa7fc640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7cf00a2da0 0x7f7cf00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.962+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e0ff8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.962+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e0ff92f0 0x55c6e1007b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.962+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e100a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.962+0000 7f7d0e235740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c6e0f5d2f0 con 0x55c6e10080c0 2024-09-11T22:26:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.962+0000 7f7d0e235740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c6e0f05270 con 0x55c6e0ff89d0 2024-09-11T22:26:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.962+0000 7f7d0e235740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c6e0f592b0 con 0x55c6e0ff92f0 2024-09-11T22:26:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.962+0000 7f7d0995a640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7cf00a2da0 0x7f7cf00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d0995a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e100a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7cfa7fc640 1 -- 172.21.15.5:0/1324833668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7d000cefe0 con 0x7f7d04102ca0 2024-09-11T22:26:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d09159640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e0ff8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d0995a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e100a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52030/0 (socket says 172.21.15.5:52030) 2024-09-11T22:26:26.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d0995a640 1 -- 172.21.15.5:0/1678925862 learned_addr learned my addr 172.21.15.5:0/1678925862 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d08958640 1 --2- 172.21.15.5:0/1678925862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e0ff92f0 0x55c6e1007b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d0995a640 1 -- 172.21.15.5:0/1678925862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e0ff92f0 msgr2=0x55c6e1007b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d0995a640 1 --2- 172.21.15.5:0/1678925862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e0ff92f0 0x55c6e1007b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d0995a640 1 -- 172.21.15.5:0/1678925862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 msgr2=0x55c6e0ff8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d0995a640 1 --2- 172.21.15.5:0/1678925862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e0ff8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d0995a640 1 -- 172.21.15.5:0/1678925862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e0f04340 con 0x55c6e10080c0 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d0995a640 1 --2- 172.21.15.5:0/1678925862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e100a4b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc013e90 tx=0x7f7cfc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114ff6536c2a420c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7d0995a640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7cf00a2da0 0x7f7cf00a5260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc010ca0 tx=0x7f7cfc015000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7cef7fe640 1 -- 172.21.15.5:0/1678925862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc021070 con 0x55c6e10080c0 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7cef7fe640 1 -- 172.21.15.5:0/1678925862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cfc007c00 con 0x55c6e10080c0 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.963+0000 7f7cef7fe640 1 -- 172.21.15.5:0/1678925862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc005900 con 0x55c6e10080c0 2024-09-11T22:26:26.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.964+0000 7f7d0e235740 1 -- 172.21.15.5:0/1678925862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 msgr2=0x55c6e100a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.964+0000 7f7d0e235740 1 --2- 172.21.15.5:0/1678925862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e100a4b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc013e90 tx=0x7f7cfc0107a0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.964+0000 7f7d0e235740 1 -- 172.21.15.5:0/1678925862 shutdown_connections 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.964+0000 7f7d0e235740 1 --2- 172.21.15.5:0/1678925862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e100a4b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.964+0000 7f7d0e235740 1 --2- 172.21.15.5:0/1678925862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e0ff92f0 0x55c6e1007b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.964+0000 7f7d0e235740 1 --2- 172.21.15.5:0/1678925862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e0ff8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.964+0000 7f7d0e235740 1 -- 172.21.15.5:0/1678925862 >> 172.21.15.5:0/1678925862 conn(0x55c6e0ff5060 msgr2=0x55c6e0ff5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.964+0000 7f7d0e235740 1 -- 172.21.15.5:0/1678925862 shutdown_connections 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.964+0000 7f7d0e235740 1 -- 172.21.15.5:0/1678925862 wait complete. 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.964+0000 7f7d0e235740 1 -- start start 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d0e235740 1 Processor -- start 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff89d0 0x55c6e10050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff92f0 0x55c6e1005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e10080c0 0x55c6e1005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d0e235740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c6e100ac90 con 0x55c6e0ff89d0 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d0e235740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c6e0ee6290 con 0x55c6e0ff92f0 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d0e235740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c6e100ae10 con 0x55c6e10080c0 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d0e235740 10 client.0 _ll_register_callbacks cb 0x55c6e0f17140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d09159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff89d0 0x55c6e10050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d09159640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff89d0 0x55c6e10050c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52032/0 (socket says 172.21.15.5:52032) 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d09159640 1 -- 172.21.15.5:0/2688757733 learned_addr learned my addr 172.21.15.5:0/2688757733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d08958640 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff92f0 0x55c6e1005600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d09159640 1 -- 172.21.15.5:0/2688757733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e10080c0 msgr2=0x55c6e1005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d09159640 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e10080c0 0x55c6e1005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d09159640 1 -- 172.21.15.5:0/2688757733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff92f0 msgr2=0x55c6e1005600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:26.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d09159640 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff92f0 0x55c6e1005600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d09159640 1 -- 172.21.15.5:0/2688757733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e0c70210 con 0x55c6e0ff89d0 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.965+0000 7f7d08958640 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff92f0 0x55c6e1005600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d09159640 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff89d0 0x55c6e10050c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7d00097900 tx=0x7f7d00004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5717d4186015a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7cecff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7cecff9640 1 -- 172.21.15.5:0/2688757733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00020040 con 0x55c6e0ff89d0 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7cecff9640 1 -- 172.21.15.5:0/2688757733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7d00009cc0 con 0x55c6e0ff89d0 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d0e235740 1 -- 172.21.15.5:0/2688757733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c6e0f6aa20 con 0x55c6e0ff89d0 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7cecff9640 1 -- 172.21.15.5:0/2688757733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7d0009f850 con 0x55c6e0ff89d0 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d0e235740 10 client.16734 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d0e235740 1 -- 172.21.15.5:0/2688757733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c6e0d3c620 con 0x55c6e0ff89d0 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7cecff9640 1 -- 172.21.15.5:0/2688757733 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c6e0f6aa20 con 0x55c6e0ff89d0 2024-09-11T22:26:26.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7cecff9640 1 -- 172.21.15.5:0/2688757733 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f7d000a3c80 con 0x55c6e0ff89d0 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d0e235740 10 client.16734 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d0e235740 10 client.16734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d0e235740 20 client.16734 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d0e235740 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c6e10180d0 0x55c6e101a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d0e235740 4 client.16734 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d0e235740 1 -- 172.21.15.5:0/2688757733 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6e0f64d70 con 0x55c6e10180d0 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.966+0000 7f7d08958640 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c6e10180d0 0x55c6e101a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.967+0000 7f7d08958640 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c6e10180d0 0x55c6e101a520 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25da6a37357d8306 server_cookie=2acd94a8927fe169 in_seq=0 out_seq=0 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.967+0000 7f7cecff9640 10 client.16734 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.967+0000 7f7cecff9640 1 -- 172.21.15.5:0/2688757733 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6e0f64d70 con 0x55c6e10180d0 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.967+0000 7f7cecff9640 10 client.16734 handle_command_reply: tid=0 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 client.16734 shutdown 2024-09-11T22:26:26.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 -- 172.21.15.5:0/2688757733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff89d0 msgr2=0x55c6e10050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff89d0 0x55c6e10050c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7d00097900 tx=0x7f7d00004780 comp rx=0 tx=0).stop 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 -- 172.21.15.5:0/2688757733 shutdown_connections 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c6e10180d0 0x55c6e101a520 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e10080c0 0x55c6e1005b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff92f0 0x55c6e1005600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 --2- 172.21.15.5:0/2688757733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff89d0 0x55c6e10050c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 -- 172.21.15.5:0/2688757733 >> 172.21.15.5:0/2688757733 conn(0x55c6e0ff5060 msgr2=0x55c6e0ffadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7cecff9640 0 client.16734 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 -- 172.21.15.5:0/2688757733 shutdown_connections 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 1 -- 172.21.15.5:0/2688757733 wait complete. 2024-09-11T22:26:26.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.968+0000 7f7d0e235740 20 client.16734 trim_cache size 0 max 16384 2024-09-11T22:26:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.982+0000 7f7d0e235740 1 Processor -- start 2024-09-11T22:26:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.982+0000 7f7d0e235740 1 -- start start 2024-09-11T22:26:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.982+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e1007930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.982+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e0ff92f0 0x55c6e0ffacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.982+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e101fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.982+0000 7f7d0e235740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041ab6c0 con 0x55c6e10080c0 2024-09-11T22:26:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.982+0000 7f7d0e235740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041ad200 con 0x55c6e0ff89d0 2024-09-11T22:26:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.982+0000 7f7d0e235740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041abd90 con 0x55c6e0ff92f0 2024-09-11T22:26:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7d0995a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e101fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7d0995a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e101fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52040/0 (socket says 172.21.15.5:52040) 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7d0995a640 1 -- 172.21.15.5:0/1270878469 learned_addr learned my addr 172.21.15.5:0/1270878469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7d09159640 1 --2- 172.21.15.5:0/1270878469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e1007930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7d0995a640 1 -- 172.21.15.5:0/1270878469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e0ff92f0 msgr2=0x55c6e0ffacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7d0995a640 1 --2- 172.21.15.5:0/1270878469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e0ff92f0 0x55c6e0ffacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7d0995a640 1 -- 172.21.15.5:0/1270878469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 msgr2=0x55c6e1007930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7d0995a640 1 --2- 172.21.15.5:0/1270878469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e1007930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7d0995a640 1 -- 172.21.15.5:0/1270878469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e0ff3650 con 0x55c6e10080c0 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7d0995a640 1 --2- 172.21.15.5:0/1270878469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e101fe50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc007500 tx=0x7f7cfc007530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fe64d29d244a64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7ceeffd640 1 -- 172.21.15.5:0/1270878469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc032070 con 0x55c6e10080c0 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7ceeffd640 1 -- 172.21.15.5:0/1270878469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cfc00de10 con 0x55c6e10080c0 2024-09-11T22:26:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.983+0000 7f7ceeffd640 1 -- 172.21.15.5:0/1270878469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc021070 con 0x55c6e10080c0 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 -- 172.21.15.5:0/1270878469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 msgr2=0x55c6e101fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 --2- 172.21.15.5:0/1270878469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e101fe50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc007500 tx=0x7f7cfc007530 comp rx=0 tx=0).stop 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 -- 172.21.15.5:0/1270878469 shutdown_connections 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 --2- 172.21.15.5:0/1270878469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e10080c0 0x55c6e101fe50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 --2- 172.21.15.5:0/1270878469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e0ff92f0 0x55c6e0ffacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 --2- 172.21.15.5:0/1270878469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e1007930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 -- 172.21.15.5:0/1270878469 >> 172.21.15.5:0/1270878469 conn(0x55c6e0ff5060 msgr2=0x55c6e0ffa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 -- 172.21.15.5:0/1270878469 shutdown_connections 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 -- 172.21.15.5:0/1270878469 wait complete. 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 -- start start 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.984+0000 7f7d0e235740 1 Processor -- start 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e1010590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff92f0 0x55c6e100b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d0e235740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e10080c0 0x55c6e100bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d0e235740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041acb30 con 0x55c6e0ff92f0 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d0e235740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041ac460 con 0x55c6e0ff89d0 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d0e235740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d041a96a0 con 0x55c6e10080c0 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d0e235740 10 client.0 _ll_register_callbacks cb 0x7f7d0009b270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d08958640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff92f0 0x55c6e100b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d08958640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff92f0 0x55c6e100b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52054/0 (socket says 172.21.15.5:52054) 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d08958640 1 -- 172.21.15.5:0/561457321 learned_addr learned my addr 172.21.15.5:0/561457321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d09159640 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e1010590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d08958640 1 -- 172.21.15.5:0/561457321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e10080c0 msgr2=0x55c6e100bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d08958640 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e10080c0 0x55c6e100bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d08958640 1 -- 172.21.15.5:0/561457321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 msgr2=0x55c6e1010590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:26.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d08958640 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e1010590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d08958640 1 -- 172.21.15.5:0/561457321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e100c3b0 con 0x55c6e0ff92f0 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7d08958640 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff92f0 0x55c6e100b8b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7d04100280 tx=0x7f7cf4005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b079e71d673a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7cecff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7cecff9640 1 -- 172.21.15.5:0/561457321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4035040 con 0x55c6e0ff92f0 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.985+0000 7f7cecff9640 1 -- 172.21.15.5:0/561457321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7cf4005340 con 0x55c6e0ff92f0 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7d0e235740 1 -- 172.21.15.5:0/561457321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cfc00de10 con 0x55c6e0ff92f0 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7cecff9640 1 -- 172.21.15.5:0/561457321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7cf4051050 con 0x55c6e0ff92f0 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7d0e235740 10 client.16740 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7d0e235740 1 -- 172.21.15.5:0/561457321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c6e1082c20 con 0x55c6e0ff92f0 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7cecff9640 1 -- 172.21.15.5:0/561457321 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cfc00de10 con 0x55c6e0ff92f0 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7cecff9640 1 -- 172.21.15.5:0/561457321 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f7cf4031810 con 0x55c6e0ff92f0 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7d0e235740 10 client.16740 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:26.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7d0e235740 10 client.16740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7d0e235740 20 client.16740 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7d0e235740 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c6e1083320 0x55c6e1085740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7d0e235740 4 client.16740 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:26:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7d0e235740 1 -- 172.21.15.5:0/561457321 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6e0ee1900 con 0x55c6e1083320 2024-09-11T22:26:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.986+0000 7f7d09159640 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c6e1083320 0x55c6e1085740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.987+0000 7f7d09159640 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c6e1083320 0x55c6e1085740 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22df15573c685737 server_cookie=1da700e387bb0c54 in_seq=0 out_seq=0 2024-09-11T22:26:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:26.987+0000 7f7cecff9640 10 client.16740 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7cecff9640 1 -- 172.21.15.5:0/561457321 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6e0ee1900 con 0x55c6e1083320 2024-09-11T22:26:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7cecff9640 10 client.16740 handle_command_reply: tid=0 2024-09-11T22:26:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 client.16740 shutdown 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 -- 172.21.15.5:0/561457321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff92f0 msgr2=0x55c6e100b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff92f0 0x55c6e100b8b0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7d04100280 tx=0x7f7cf4005040 comp rx=0 tx=0).stop 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 -- 172.21.15.5:0/561457321 shutdown_connections 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c6e1083320 0x55c6e1085740 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e10080c0 0x55c6e100bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7cecff9640 0 client.16740 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6e0ff92f0 0x55c6e100b8b0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 --2- 172.21.15.5:0/561457321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6e0ff89d0 0x55c6e1010590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 -- 172.21.15.5:0/561457321 >> 172.21.15.5:0/561457321 conn(0x55c6e0ff5060 msgr2=0x55c6e0ffadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 -- 172.21.15.5:0/561457321 shutdown_connections 2024-09-11T22:26:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 1 -- 172.21.15.5:0/561457321 wait complete. 2024-09-11T22:26:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.068+0000 7f7d0e235740 20 client.16740 trim_cache size 0 max 16384 2024-09-11T22:26:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:26:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:26:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:26:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:26:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.069+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/1324833668 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7cf00a2da0 msgr2=0x7f7cf00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.069+0000 7f7d0b3bd640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7cf00a2da0 0x7f7cf00a5260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc010ca0 tx=0x7f7cfc015000 comp rx=0 tx=0).stop 2024-09-11T22:26:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.069+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/1324833668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d04102ca0 msgr2=0x7f7d04198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.069+0000 7f7d0b3bd640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d04102ca0 0x7f7d04198300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000d770 tx=0x7f7d0000dc30 comp rx=0 tx=0).stop 2024-09-11T22:26:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.070+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/1324833668 shutdown_connections 2024-09-11T22:26:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.070+0000 7f7d0b3bd640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7cf00a2da0 0x7f7cf00a5260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.070+0000 7f7d0b3bd640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d0410c0c0 0x7f7d0419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.070+0000 7f7d0b3bd640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d041035c0 0x7f7d04198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.070+0000 7f7d0b3bd640 1 --2- 172.21.15.5:0/1324833668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d04102ca0 0x7f7d04198300 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.070+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/1324833668 >> 172.21.15.5:0/1324833668 conn(0x7f7d040fc6f0 msgr2=0x7f7d040fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.070+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/1324833668 shutdown_connections 2024-09-11T22:26:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:27.070+0000 7f7d0b3bd640 1 -- 172.21.15.5:0/1324833668 wait complete. 2024-09-11T22:26:27.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 11.75 MB/sec execute 45 sec latency 6420.749 ms 2024-09-11T22:26:27.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:27 smithi005 ceph-mon[30943]: pgmap v131: 97 pgs: 97 active+clean; 78 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 331 op/s 2024-09-11T22:26:27.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:27 smithi159 ceph-mon[35534]: pgmap v131: 97 pgs: 97 active+clean; 78 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 331 op/s 2024-09-11T22:26:27.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:27 smithi172 ceph-mon[35461]: pgmap v131: 97 pgs: 97 active+clean; 78 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 331 op/s 2024-09-11T22:26:27.966 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:26:27.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:26:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:28.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 11.49 MB/sec execute 46 sec latency 7420.797 ms 2024-09-11T22:26:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.749+0000 7f7c929f2640 1 -- 172.21.15.5:0/2087080576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8c106f40 msgr2=0x7f7c8c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.749+0000 7f7c929f2640 1 --2- 172.21.15.5:0/2087080576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8c106f40 0x7f7c8c10eac0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009ca0 tx=0x7f7c8402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.749+0000 7f7c929f2640 1 -- 172.21.15.5:0/2087080576 shutdown_connections 2024-09-11T22:26:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.749+0000 7f7c929f2640 1 --2- 172.21.15.5:0/2087080576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8c106f40 0x7f7c8c10eac0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.749+0000 7f7c929f2640 1 --2- 172.21.15.5:0/2087080576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c1027c0 0x7f7c8c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.749+0000 7f7c929f2640 1 --2- 172.21.15.5:0/2087080576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c8c101e10 0x7f7c8c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.749+0000 7f7c929f2640 1 -- 172.21.15.5:0/2087080576 >> 172.21.15.5:0/2087080576 conn(0x7f7c8c0fd8d0 msgr2=0x7f7c8c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.749+0000 7f7c929f2640 1 -- 172.21.15.5:0/2087080576 shutdown_connections 2024-09-11T22:26:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.749+0000 7f7c929f2640 1 -- 172.21.15.5:0/2087080576 wait complete. 2024-09-11T22:26:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.750+0000 7f7c929f2640 1 Processor -- start 2024-09-11T22:26:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.750+0000 7f7c929f2640 1 -- start start 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c929f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8c101e10 0x7f7c8c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c929f2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c8c1027c0 0x7f7c8c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c929f2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c106f40 0x7f7c8c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c929f2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c111fa0 con 0x7f7c8c101e10 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c929f2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c111e20 con 0x7f7c8c106f40 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c929f2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c112120 con 0x7f7c8c1027c0 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8c101e10 0x7f7c8c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8c101e10 0x7f7c8c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52062/0 (socket says 172.21.15.5:52062) 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c8bfff640 1 -- 172.21.15.5:0/3787768149 learned_addr learned my addr 172.21.15.5:0/3787768149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c90f8f640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c106f40 0x7f7c8c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.751+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c8c1027c0 0x7f7c8c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c8bfff640 1 -- 172.21.15.5:0/3787768149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c8c1027c0 msgr2=0x7f7c8c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c8c1027c0 0x7f7c8c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c8bfff640 1 -- 172.21.15.5:0/3787768149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c106f40 msgr2=0x7f7c8c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c106f40 0x7f7c8c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c8bfff640 1 -- 172.21.15.5:0/3787768149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8c19d390 con 0x7f7c8c101e10 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8c101e10 0x7f7c8c1983e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00e9f0 tx=0x7f7c7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c897fa640 1 -- 172.21.15.5:0/3787768149 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c00cd80 con 0x7f7c8c101e10 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c897fa640 1 -- 172.21.15.5:0/3787768149 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c005500 con 0x7f7c8c101e10 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c897fa640 1 -- 172.21.15.5:0/3787768149 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c010690 con 0x7f7c8c101e10 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c929f2640 1 -- 172.21.15.5:0/3787768149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8c19d680 con 0x7f7c8c101e10 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.752+0000 7f7c929f2640 1 -- 172.21.15.5:0/3787768149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8c1a4f60 con 0x7f7c8c101e10 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.754+0000 7f7c897fa640 1 -- 172.21.15.5:0/3787768149 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7c7c0046c0 con 0x7f7c8c101e10 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c94dcb740 1 Processor -- start 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c94dcb740 1 -- start start 2024-09-11T22:26:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c4485f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c4490800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c4493130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573c43eb280 con 0x5573c4490d40 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5573c4392b10 con 0x5573c4486480 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5573c43e7240 con 0x5573c4485b60 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c4493130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c4493130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52074/0 (socket says 172.21.15.5:52074) 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c8bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c4490800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c8b7fe640 1 -- 172.21.15.5:0/3460945351 learned_addr learned my addr 172.21.15.5:0/3460945351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c90f8f640 1 --2- 172.21.15.5:0/3460945351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c4485f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c8b7fe640 1 -- 172.21.15.5:0/3460945351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 msgr2=0x5573c4485f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/3460945351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c4485f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c8b7fe640 1 -- 172.21.15.5:0/3460945351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 msgr2=0x5573c4490800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/3460945351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c4490800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.755+0000 7f7c8b7fe640 1 -- 172.21.15.5:0/3460945351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c439f1f0 con 0x5573c4490d40 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.756+0000 7f7c897fa640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7c740a2d70 0x7f7c740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.756+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/3460945351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c4493130 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009c10 tx=0x7f7c80011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dacde9409e0c1b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.756+0000 7f7c90f8f640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7c740a2d70 0x7f7c740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.756+0000 7f7c727fc640 1 -- 172.21.15.5:0/3460945351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80004230 con 0x5573c4490d40 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.756+0000 7f7c897fa640 1 -- 172.21.15.5:0/3787768149 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7c7c0cf870 con 0x7f7c8c101e10 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.756+0000 7f7c727fc640 1 -- 172.21.15.5:0/3460945351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c800043d0 con 0x5573c4490d40 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.756+0000 7f7c727fc640 1 -- 172.21.15.5:0/3460945351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80004e60 con 0x5573c4490d40 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3460945351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 msgr2=0x5573c4493130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c90f8f640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7c740a2d70 0x7f7c740a5230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7c84004af0 tx=0x7f7c84004a80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3460945351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c4493130 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009c10 tx=0x7f7c80011810 comp rx=0 tx=0).stop 2024-09-11T22:26:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3460945351 shutdown_connections 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3460945351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c4493130 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3460945351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c4490800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3460945351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c4485f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3460945351 >> 172.21.15.5:0/3460945351 conn(0x5573c44820f0 msgr2=0x5573c44824d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3460945351 shutdown_connections 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3460945351 wait complete. 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 -- start start 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.757+0000 7f7c94dcb740 1 Processor -- start 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c448dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c448e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c448e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573c4493910 con 0x5573c4490d40 2024-09-11T22:26:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5573c43a5f10 con 0x5573c4486480 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5573c4493a90 con 0x5573c4485b60 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c94dcb740 10 client.0 _ll_register_callbacks cb 0x5573c436ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c90f8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c448e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c90f8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c448e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52082/0 (socket says 172.21.15.5:52082) 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c90f8f640 1 -- 172.21.15.5:0/3715358928 learned_addr learned my addr 172.21.15.5:0/3715358928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c448e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c448dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c90f8f640 1 -- 172.21.15.5:0/3715358928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 msgr2=0x5573c448dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c90f8f640 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c448dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c90f8f640 1 -- 172.21.15.5:0/3715358928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 msgr2=0x5573c448e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c90f8f640 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c448e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c90f8f640 1 -- 172.21.15.5:0/3715358928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c415c6a0 con 0x5573c4490d40 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.758+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c448e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c90f8f640 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c448e670 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7c84035000 tx=0x7f7c8402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2287fa4d106467fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/3715358928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c84004250 con 0x5573c4490d40 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/3715358928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7c840043f0 con 0x5573c4490d40 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3715358928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c800043d0 con 0x5573c4490d40 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/3715358928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7c840045f0 con 0x5573c4490d40 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c94dcb740 10 client.16752 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3715358928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5573c449f4e0 con 0x5573c4490d40 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/3715358928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c800043d0 con 0x5573c4490d40 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/3715358928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f7c840305c0 con 0x5573c4490d40 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c94dcb740 10 client.16752 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c94dcb740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c94dcb740 20 client.16752 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573c44a08f0 0x5573c44a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c94dcb740 4 client.16752 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.759+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3715358928 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573c43f87f0 con 0x5573c44a08f0 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.760+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573c44a08f0 0x5573c44a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.760+0000 7f7c6b7fe640 10 client.16752 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.760+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573c44a08f0 0x5573c44a2d00 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b3cfeae9610b7f server_cookie=108cbebc4e49ea9e in_seq=0 out_seq=0 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/3715358928 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573c43f87f0 con 0x5573c44a08f0 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c6b7fe640 10 client.16752 handle_command_reply: tid=0 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 client.16752 shutdown 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3715358928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 msgr2=0x5573c448e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c448e670 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f7c84035000 tx=0x7f7c8402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3715358928 shutdown_connections 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573c44a08f0 0x5573c44a2d00 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c448e670 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c448e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3715358928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c448dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3715358928 >> 172.21.15.5:0/3715358928 conn(0x5573c44820f0 msgr2=0x5573c4487dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c6b7fe640 0 client.16752 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3715358928 shutdown_connections 2024-09-11T22:26:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3715358928 wait complete. 2024-09-11T22:26:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.761+0000 7f7c94dcb740 20 client.16752 trim_cache size 0 max 16384 2024-09-11T22:26:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.799+0000 7f7c94dcb740 1 Processor -- start 2024-09-11T22:26:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.799+0000 7f7c94dcb740 1 -- start start 2024-09-11T22:26:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.799+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c448d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.799+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c44a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.799+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c44a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.799+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1afc00 con 0x5573c4490d40 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.799+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1b1740 con 0x5573c4486480 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.799+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1b02d0 con 0x5573c4485b60 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c90f8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c44a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c90f8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c44a8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52098/0 (socket says 172.21.15.5:52098) 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c90f8f640 1 -- 172.21.15.5:0/3376694714 learned_addr learned my addr 172.21.15.5:0/3376694714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/3376694714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c448d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/3376694714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c44a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c8bfff640 1 -- 172.21.15.5:0/3376694714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 msgr2=0x5573c44a85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/3376694714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c44a85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c8bfff640 1 -- 172.21.15.5:0/3376694714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 msgr2=0x5573c44a8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/3376694714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c44a8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c8bfff640 1 -- 172.21.15.5:0/3376694714 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573c4481f20 con 0x5573c4485b60 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c90f8f640 1 --2- 172.21.15.5:0/3376694714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c44a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/3376694714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c448d4d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0ccf20 tx=0x7f7c7c0d1cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=360784ad7712a26 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c71ffb640 1 -- 172.21.15.5:0/3376694714 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c09b650 con 0x5573c4485b60 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c71ffb640 1 -- 172.21.15.5:0/3376694714 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c0d2e50 con 0x5573c4485b60 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.800+0000 7f7c71ffb640 1 -- 172.21.15.5:0/3376694714 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c098360 con 0x5573c4485b60 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.801+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3376694714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 msgr2=0x5573c448d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.801+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3376694714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c448d4d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0ccf20 tx=0x7f7c7c0d1cc0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.801+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3376694714 shutdown_connections 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.801+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3376694714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4490d40 0x5573c44a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.801+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3376694714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c44a85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.801+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/3376694714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4485b60 0x5573c448d4d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.801+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3376694714 >> 172.21.15.5:0/3376694714 conn(0x5573c44820f0 msgr2=0x5573c4487dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.801+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3376694714 shutdown_connections 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.801+0000 7f7c94dcb740 1 -- 172.21.15.5:0/3376694714 wait complete. 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.802+0000 7f7c94dcb740 1 -- start start 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.802+0000 7f7c94dcb740 1 Processor -- start 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.802+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4485b60 0x5573c4494a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.802+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c4494fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.802+0000 7f7c94dcb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4490d40 0x5573c44954f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.802+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1b1070 con 0x5573c4485b60 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.802+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1b09a0 con 0x5573c4486480 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.802+0000 7f7c94dcb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8c1adbe0 con 0x5573c4490d40 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.802+0000 7f7c94dcb740 10 client.0 _ll_register_callbacks cb 0x7f7c84009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4485b60 0x5573c4494a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4485b60 0x5573c4494a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52112/0 (socket says 172.21.15.5:52112) 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8bfff640 1 -- 172.21.15.5:0/2703621168 learned_addr learned my addr 172.21.15.5:0/2703621168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c4494fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c90f8f640 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4490d40 0x5573c44954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8bfff640 1 -- 172.21.15.5:0/2703621168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4490d40 msgr2=0x5573c44954f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4490d40 0x5573c44954f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8bfff640 1 -- 172.21.15.5:0/2703621168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 msgr2=0x5573c4494fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c4494fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8bfff640 1 -- 172.21.15.5:0/2703621168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573c450a5f0 con 0x5573c4485b60 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c4494fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c8bfff640 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4485b60 0x5573c4494a70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0986c0 tx=0x7f7c7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904d212f2e490425 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2703621168 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c0b8020 con 0x5573c4485b60 2024-09-11T22:26:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2703621168 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7c7c0cd1a0 con 0x5573c4485b60 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.803+0000 7f7c94dcb740 1 -- 172.21.15.5:0/2703621168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c7c0d2e50 con 0x5573c4485b60 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2703621168 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7c7c09b650 con 0x5573c4485b60 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c94dcb740 10 client.16758 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c94dcb740 1 -- 172.21.15.5:0/2703621168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5573c450b8f0 con 0x5573c4485b60 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2703621168 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c7c0d2e50 con 0x5573c4485b60 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2703621168 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f7c7c0a44d0 con 0x5573c4485b60 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c94dcb740 10 client.16758 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c94dcb740 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c94dcb740 20 client.16758 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573c450bf10 0x5573c450e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c94dcb740 4 client.16758 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c94dcb740 1 -- 172.21.15.5:0/2703621168 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5573c43f2d00 con 0x5573c450bf10 2024-09-11T22:26:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.804+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573c450bf10 0x5573c450e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.805+0000 7f7c8b7fe640 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573c450bf10 0x5573c450e330 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa01ab989f4eb7cf server_cookie=2d029e9559e9ce3 in_seq=0 out_seq=0 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.805+0000 7f7c6b7fe640 10 client.16758 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2703621168 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5573c43f2d00 con 0x5573c450bf10 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c6b7fe640 10 client.16758 handle_command_reply: tid=0 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c94dcb740 1 client.16758 shutdown 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c94dcb740 1 -- 172.21.15.5:0/2703621168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4485b60 msgr2=0x5573c4494a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4485b60 0x5573c4494a70 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c0986c0 tx=0x7f7c7c004a40 comp rx=0 tx=0).stop 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c94dcb740 1 -- 172.21.15.5:0/2703621168 shutdown_connections 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573c450bf10 0x5573c450e330 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573c4490d40 0x5573c44954f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573c4486480 0x5573c4494fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c94dcb740 1 --2- 172.21.15.5:0/2703621168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573c4485b60 0x5573c4494a70 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c94dcb740 1 -- 172.21.15.5:0/2703621168 >> 172.21.15.5:0/2703621168 conn(0x5573c44820f0 msgr2=0x5573c4480a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:28.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.806+0000 7f7c6b7fe640 0 client.16758 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:28.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.807+0000 7f7c94dcb740 1 -- 172.21.15.5:0/2703621168 shutdown_connections 2024-09-11T22:26:28.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.807+0000 7f7c94dcb740 1 -- 172.21.15.5:0/2703621168 wait complete. 2024-09-11T22:26:28.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.807+0000 7f7c94dcb740 20 client.16758 trim_cache size 0 max 16384 2024-09-11T22:26:28.816 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:26:28.816 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:26:28.816 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e47d6bef-30d2-4481-aa2d-9fc5c5c2f2fa", 2024-09-11T22:26:28.816 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:26:28.816 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:26:28.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.807+0000 7f7c929f2640 1 -- 172.21.15.5:0/3787768149 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7c740a2d70 msgr2=0x7f7c740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:28.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.807+0000 7f7c929f2640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7c740a2d70 0x7f7c740a5230 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7c84004af0 tx=0x7f7c84004a80 comp rx=0 tx=0).stop 2024-09-11T22:26:28.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.807+0000 7f7c929f2640 1 -- 172.21.15.5:0/3787768149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8c101e10 msgr2=0x7f7c8c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:28.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.807+0000 7f7c929f2640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8c101e10 0x7f7c8c1983e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00e9f0 tx=0x7f7c7c00edf0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.807+0000 7f7c929f2640 1 -- 172.21.15.5:0/3787768149 shutdown_connections 2024-09-11T22:26:28.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.808+0000 7f7c929f2640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7c740a2d70 0x7f7c740a5230 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.808+0000 7f7c929f2640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7c8c106f40 0x7f7c8c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.808+0000 7f7c929f2640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7c8c1027c0 0x7f7c8c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.808+0000 7f7c929f2640 1 --2- 172.21.15.5:0/3787768149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c8c101e10 0x7f7c8c1983e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:28.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.808+0000 7f7c929f2640 1 -- 172.21.15.5:0/3787768149 >> 172.21.15.5:0/3787768149 conn(0x7f7c8c0fd8d0 msgr2=0x7f7c8c0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:28.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.808+0000 7f7c929f2640 1 -- 172.21.15.5:0/3787768149 shutdown_connections 2024-09-11T22:26:28.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:28.808+0000 7f7c929f2640 1 -- 172.21.15.5:0/3787768149 wait complete. 2024-09-11T22:26:29.079 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:26:28 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:26:28.818+0000 7f54cdab7640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...6,head] /volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{1=1,2=1,3=1,4=1} v64414 ap=3 RANDEPHEMERALPIN f(v1827 m2024-09-11T22:26:20.786762+0000 3=3+0) n(v17 rc2024-09-11T22:26:20.786762+0000 b780288 4=3+1) old_inodes=3 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:270 nref=3)) caps={24841=pAsLsXsFs/pAsLsXsFsx@3764} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x556c7fef9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v2606>,<0x10000000008/~dmtmp v158>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v14>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v30>,<0x10000000001/sv_1 v53>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v64414>,<0x10000000008/~dmtmp v3978>,<0x10000000007/client0 v296>,<0x10000000006/clients v270>,<0x10000000005/tmp v270>,<0x100000001fb/client.0 v273>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v279>,<0x10000000001/sv_1 v295>,<0x10000000000/qa v277>,<0x1/volumes v267>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T22:26:20.251773+0000 9=9+0)","ondisk_value.rstat":"n(v0 rc2024-09-11T22:26:20.786762+0000 b780288 4=3+1)","memory_value.dirstat":"f(v1827 m2024-09-11T22:26:20.786762+0000 3=3+0)","memory_value.rstat":"n(v17 rc2024-09-11T22:26:20.786762+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-11T22:26:29.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 11.25 MB/sec execute 47 sec latency 8420.826 ms 2024-09-11T22:26:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:29 smithi005 ceph-mon[30943]: pgmap v132: 97 pgs: 97 active+clean; 77 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 370 op/s 2024-09-11T22:26:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:29 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:26:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:29 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:26:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:29 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:26:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:29 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:26:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:29 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:26:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:29 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:26:29.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:29 smithi005 ceph-mon[30943]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:26:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:29 smithi159 ceph-mon[35534]: pgmap v132: 97 pgs: 97 active+clean; 77 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 370 op/s 2024-09-11T22:26:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:29 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:26:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:29 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:26:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:29 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:26:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:29 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:26:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:29 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:26:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:29 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:26:29.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:29 smithi159 ceph-mon[35534]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:26:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:29 smithi172 ceph-mon[35461]: pgmap v132: 97 pgs: 97 active+clean; 77 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 370 op/s 2024-09-11T22:26:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:29 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:26:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:29 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:26:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:29 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:26:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:29 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:26:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:29 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:26:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:29 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:26:29.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:29 smithi172 ceph-mon[35461]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:26:29.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:26:30.146 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:30.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 11.01 MB/sec execute 48 sec latency 9420.920 ms 2024-09-11T22:26:30.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:30 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:26:30.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:30 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:26:30.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:30 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:26:30.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:30 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:26:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:30 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:26:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:30 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:26:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:30 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:26:30.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:30 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:26:30.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:30 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:26:30.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:30 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:26:30.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:30 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:26:30.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:30 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:26:30.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.773+0000 7f0ff8316640 1 -- 172.21.15.5:0/122217951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff0102570 msgr2=0x7f0ff01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:30.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.773+0000 7f0ff8316640 1 --2- 172.21.15.5:0/122217951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff0102570 0x7f0ff01029b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009900 tx=0x7f0fe402a480 comp rx=0 tx=0).stop 2024-09-11T22:26:30.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.773+0000 7f0ff8316640 1 -- 172.21.15.5:0/122217951 shutdown_connections 2024-09-11T22:26:30.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.773+0000 7f0ff8316640 1 --2- 172.21.15.5:0/122217951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff0102ef0 0x7f0ff010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.773+0000 7f0ff8316640 1 --2- 172.21.15.5:0/122217951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff0102570 0x7f0ff01029b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.773+0000 7f0ff8316640 1 --2- 172.21.15.5:0/122217951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff0100e20 0x7f0ff0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.773+0000 7f0ff8316640 1 -- 172.21.15.5:0/122217951 >> 172.21.15.5:0/122217951 conn(0x7f0ff00fc7d0 msgr2=0x7f0ff00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.773+0000 7f0ff8316640 1 -- 172.21.15.5:0/122217951 shutdown_connections 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.773+0000 7f0ff8316640 1 -- 172.21.15.5:0/122217951 wait complete. 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff8316640 1 Processor -- start 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff8316640 1 -- start start 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff8316640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff0100e20 0x7f0ff0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff8316640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff0102570 0x7f0ff0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff8316640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff0102ef0 0x7f0ff019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff8316640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0110fd0 con 0x7f0ff0100e20 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff8316640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0110e50 con 0x7f0ff0102570 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff8316640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0111150 con 0x7f0ff0102ef0 2024-09-11T22:26:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff60b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff0100e20 0x7f0ff0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff60b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff0100e20 0x7f0ff0198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52130/0 (socket says 172.21.15.5:52130) 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff60b2640 1 -- 172.21.15.5:0/903022456 learned_addr learned my addr 172.21.15.5:0/903022456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff0102570 0x7f0ff0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff68b3640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff0102ef0 0x7f0ff019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff60b2640 1 -- 172.21.15.5:0/903022456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff0102ef0 msgr2=0x7f0ff019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff0102ef0 0x7f0ff019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff60b2640 1 -- 172.21.15.5:0/903022456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff0102570 msgr2=0x7f0ff0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff0102570 0x7f0ff0198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.774+0000 7f0ff60b2640 1 -- 172.21.15.5:0/903022456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff019d4f0 con 0x7f0ff0100e20 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.775+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff0100e20 0x7f0ff0198540 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c970 tx=0x7f0fec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.775+0000 7f0fe37fe640 1 -- 172.21.15.5:0/903022456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec007bb0 con 0x7f0ff0100e20 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.775+0000 7f0ff8316640 1 -- 172.21.15.5:0/903022456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff019d7e0 con 0x7f0ff0100e20 2024-09-11T22:26:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.775+0000 7f0fe37fe640 1 -- 172.21.15.5:0/903022456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fec007d50 con 0x7f0ff0100e20 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.775+0000 7f0fe37fe640 1 -- 172.21.15.5:0/903022456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec00f450 con 0x7f0ff0100e20 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.775+0000 7f0ff8316640 1 -- 172.21.15.5:0/903022456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff01a50c0 con 0x7f0ff0100e20 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.777+0000 7f0fe37fe640 1 -- 172.21.15.5:0/903022456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0fec005020 con 0x7f0ff0100e20 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.777+0000 7f0ffa6ef740 1 Processor -- start 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ffa6ef740 1 -- start start 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dca4a0 0x555f49dca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dd5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49dd7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f49d302e0 con 0x555f49dcadc0 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555f49cd7ce0 con 0x555f49dd5680 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555f49d2c2a0 con 0x555f49dca4a0 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff60b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dd5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff60b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dd5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52134/0 (socket says 172.21.15.5:52134) 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff60b2640 1 -- 172.21.15.5:0/4251288990 learned_addr learned my addr 172.21.15.5:0/4251288990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff68b3640 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dca4a0 0x555f49dca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0fe37fe640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fdc0a2cf0 0x7f0fdc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49dd7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff68b3640 1 -- 172.21.15.5:0/4251288990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 msgr2=0x555f49dd7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff68b3640 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49dd7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff68b3640 1 -- 172.21.15.5:0/4251288990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 msgr2=0x555f49dd5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff68b3640 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dd5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff68b3640 1 -- 172.21.15.5:0/4251288990 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f49cbc440 con 0x555f49dca4a0 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49dd7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.778+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dd5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.779+0000 7f0ff68b3640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fdc0a2cf0 0x7f0fdc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.779+0000 7f0fe37fe640 1 -- 172.21.15.5:0/903022456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f0fec0cf2d0 con 0x7f0ff0100e20 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.779+0000 7f0ff68b3640 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dca4a0 0x555f49dca880 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800f3e0 tx=0x7f0fe800f8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1a4c5c62ad9473c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.779+0000 7f0ff68b3640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fdc0a2cf0 0x7f0fdc0a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8020000 tx=0x7f0fe801d000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.779+0000 7f0fe0dea640 1 -- 172.21.15.5:0/4251288990 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8023070 con 0x555f49dca4a0 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.779+0000 7f0fe0dea640 1 -- 172.21.15.5:0/4251288990 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe8012b90 con 0x555f49dca4a0 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.779+0000 7f0fe0dea640 1 -- 172.21.15.5:0/4251288990 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe801db20 con 0x555f49dca4a0 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/4251288990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dca4a0 msgr2=0x555f49dca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dca4a0 0x555f49dca880 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800f3e0 tx=0x7f0fe800f8a0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/4251288990 shutdown_connections 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49dd7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dd5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/4251288990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dca4a0 0x555f49dca880 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/4251288990 >> 172.21.15.5:0/4251288990 conn(0x555f49dc6a30 msgr2=0x555f49dc6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/4251288990 shutdown_connections 2024-09-11T22:26:30.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/4251288990 wait complete. 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 -- start start 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.780+0000 7f0ffa6ef740 1 Processor -- start 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dd4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dcadc0 0x555f49dd2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49dd2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555f49dd8250 con 0x555f49dca4a0 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555f49ceb050 con 0x555f49dd5680 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555f49dd83d0 con 0x555f49dcadc0 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ffa6ef740 10 client.0 _ll_register_callbacks cb 0x555f49d2b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff60b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dd4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff60b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dd4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52150/0 (socket says 172.21.15.5:52150) 2024-09-11T22:26:30.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff60b2640 1 -- 172.21.15.5:0/2402594624 learned_addr learned my addr 172.21.15.5:0/2402594624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dcadc0 0x555f49dd2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff68b3640 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49dd2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff60b2640 1 -- 172.21.15.5:0/2402594624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dcadc0 msgr2=0x555f49dd2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dcadc0 0x555f49dd2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff60b2640 1 -- 172.21.15.5:0/2402594624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 msgr2=0x555f49dd2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49dd2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff60b2640 1 -- 172.21.15.5:0/2402594624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f49aa13c0 con 0x555f49dca4a0 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff68b3640 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49dd2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dd4f70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0fec097bb0 tx=0x7f0fec00a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af8197a20c3255b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0fd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.781+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/2402594624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0cf070 con 0x555f49dca4a0 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/2402594624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f0fec0d2d00 con 0x555f49dca4a0 2024-09-11T22:26:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/2402594624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe8012b90 con 0x555f49dca4a0 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/2402594624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0fec01d040 con 0x555f49dca4a0 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0ffa6ef740 10 client.16770 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/2402594624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555f49de3c20 con 0x555f49dca4a0 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/2402594624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe8012b90 con 0x555f49dca4a0 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/2402594624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f0fec005310 con 0x555f49dca4a0 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0ffa6ef740 10 client.16770 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0ffa6ef740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0ffa6ef740 20 client.16770 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f49de5030 0x555f49de7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0ffa6ef740 4 client.16770 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.782+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/2402594624 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f49d3d850 con 0x555f49de5030 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.783+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f49de5030 0x555f49de7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.783+0000 7f0fd9ffb640 10 client.16770 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:30.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.783+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f49de5030 0x555f49de7440 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64af1c8eb63cdc36 server_cookie=c6a711be6a62f8d3 in_seq=0 out_seq=0 2024-09-11T22:26:30.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/2402594624 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f49d3d850 con 0x555f49de5030 2024-09-11T22:26:30.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0fd9ffb640 10 client.16770 handle_command_reply: tid=0 2024-09-11T22:26:30.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 client.16770 shutdown 2024-09-11T22:26:30.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/2402594624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 msgr2=0x555f49dd4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:30.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dd4f70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0fec097bb0 tx=0x7f0fec00a4b0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/2402594624 shutdown_connections 2024-09-11T22:26:30.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f49de5030 0x555f49de7440 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49dd2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dcadc0 0x555f49dd2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0fd9ffb640 0 client.16770 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/2402594624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dd4f70 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/2402594624 >> 172.21.15.5:0/2402594624 conn(0x555f49dc6a30 msgr2=0x555f49dc5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/2402594624 shutdown_connections 2024-09-11T22:26:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/2402594624 wait complete. 2024-09-11T22:26:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.784+0000 7f0ffa6ef740 20 client.16770 trim_cache size 0 max 16384 2024-09-11T22:26:30.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ffa6ef740 1 Processor -- start 2024-09-11T22:26:30.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ffa6ef740 1 -- start start 2024-09-11T22:26:30.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dca4a0 0x555f49dd2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dc5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dd5680 0x555f49decce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01ada70 con 0x555f49dcadc0 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01af5b0 con 0x555f49dca4a0 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01ae140 con 0x555f49dd5680 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ff58b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dc5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ff58b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dc5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52156/0 (socket says 172.21.15.5:52156) 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ff58b1640 1 -- 172.21.15.5:0/3564642832 learned_addr learned my addr 172.21.15.5:0/3564642832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.802+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/3564642832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dca4a0 0x555f49dd2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.803+0000 7f0ff58b1640 1 -- 172.21.15.5:0/3564642832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dd5680 msgr2=0x555f49decce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.803+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/3564642832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dd5680 0x555f49decce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.803+0000 7f0ff58b1640 1 -- 172.21.15.5:0/3564642832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dca4a0 msgr2=0x555f49dd2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.803+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/3564642832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dca4a0 0x555f49dd2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.803+0000 7f0ff58b1640 1 -- 172.21.15.5:0/3564642832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f49dca2d0 con 0x555f49dcadc0 2024-09-11T22:26:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.803+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/3564642832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dc5a80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009230 tx=0x7f0fe4005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571dabe85d8d0aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.803+0000 7f0fdbfff640 1 -- 172.21.15.5:0/3564642832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4035040 con 0x555f49dcadc0 2024-09-11T22:26:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.803+0000 7f0fdbfff640 1 -- 172.21.15.5:0/3564642832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe4049070 con 0x555f49dcadc0 2024-09-11T22:26:30.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.803+0000 7f0fdbfff640 1 -- 172.21.15.5:0/3564642832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4003850 con 0x555f49dcadc0 2024-09-11T22:26:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.804+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/3564642832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 msgr2=0x555f49dc5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.804+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/3564642832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dc5a80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009230 tx=0x7f0fe4005d70 comp rx=0 tx=0).stop 2024-09-11T22:26:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.804+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/3564642832 shutdown_connections 2024-09-11T22:26:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.804+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/3564642832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dd5680 0x555f49decce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.804+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/3564642832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dcadc0 0x555f49dc5a80 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.804+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/3564642832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dca4a0 0x555f49dd2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.804+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/3564642832 >> 172.21.15.5:0/3564642832 conn(0x555f49dc6a30 msgr2=0x555f49dc5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.804+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/3564642832 shutdown_connections 2024-09-11T22:26:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.804+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/3564642832 wait complete. 2024-09-11T22:26:30.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ffa6ef740 1 -- start start 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ffa6ef740 1 Processor -- start 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dcadc0 0x555f49ddaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ffa6ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49ddb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01aeee0 con 0x555f49dca4a0 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01ae810 con 0x555f49dd5680 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ffa6ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff01aba50 con 0x555f49dcadc0 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ffa6ef740 10 client.0 _ll_register_callbacks cb 0x7f0fec0d2a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ff60b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ff60b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dddb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52170/0 (socket says 172.21.15.5:52170) 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ff60b2640 1 -- 172.21.15.5:0/1156920571 learned_addr learned my addr 172.21.15.5:0/1156920571 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:30.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ff68b3640 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49ddb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.805+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dcadc0 0x555f49ddaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0ff60b2640 1 -- 172.21.15.5:0/1156920571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dcadc0 msgr2=0x555f49ddaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dcadc0 0x555f49ddaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0ff60b2640 1 -- 172.21.15.5:0/1156920571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 msgr2=0x555f49ddb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49ddb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0ff60b2640 1 -- 172.21.15.5:0/1156920571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f49ddb9d0 con 0x555f49dca4a0 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0ff60b2640 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dddb50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0cccf0 tx=0x7f0fec0cf1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e523edc29f10fee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0fd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/1156920571 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0cf040 con 0x555f49dca4a0 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/1156920571 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f0fec09ada0 con 0x555f49dca4a0 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/1156920571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fe4049070 con 0x555f49dca4a0 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/1156920571 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0fec09fb00 con 0x555f49dca4a0 2024-09-11T22:26:30.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0ffa6ef740 10 client.16776 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/1156920571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555f49e4fa20 con 0x555f49dca4a0 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.806+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/1156920571 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe4049070 con 0x555f49dca4a0 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.807+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/1156920571 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f0fec0a0c90 con 0x555f49dca4a0 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.807+0000 7f0ffa6ef740 10 client.16776 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.807+0000 7f0ffa6ef740 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.807+0000 7f0ffa6ef740 20 client.16776 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.807+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f49e50140 0x555f49e52560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.807+0000 7f0ffa6ef740 4 client.16776 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.807+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/1156920571 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555f49dd2af0 con 0x555f49e50140 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.807+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f49e50140 0x555f49e52560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.807+0000 7f0ff58b1640 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f49e50140 0x555f49e52560 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1d0ac110e88e2a server_cookie=3c684526518771e4 in_seq=0 out_seq=0 2024-09-11T22:26:30.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.807+0000 7f0fd9ffb640 10 client.16776 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:30.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0fd9ffb640 1 -- 172.21.15.5:0/1156920571 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555f49dd2af0 con 0x555f49e50140 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0fd9ffb640 10 client.16776 handle_command_reply: tid=0 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 client.16776 shutdown 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/1156920571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 msgr2=0x555f49dddb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dddb50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0cccf0 tx=0x7f0fec0cf1e0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/1156920571 shutdown_connections 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555f49e50140 0x555f49e52560 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555f49dd5680 0x555f49ddb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555f49dcadc0 0x555f49ddaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 --2- 172.21.15.5:0/1156920571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555f49dca4a0 0x555f49dddb50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/1156920571 >> 172.21.15.5:0/1156920571 conn(0x555f49dc6a30 msgr2=0x555f49dc5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0fd9ffb640 0 client.16776 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/1156920571 shutdown_connections 2024-09-11T22:26:30.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 1 -- 172.21.15.5:0/1156920571 wait complete. 2024-09-11T22:26:30.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.808+0000 7f0ffa6ef740 20 client.16776 trim_cache size 0 max 16384 2024-09-11T22:26:30.814 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:26:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:26:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:26:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: "e47d6bef-30d2-4481-aa2d-9fc5c5c2f2fa": { 2024-09-11T22:26:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:26:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e47d6bef-30d2-4481-aa2d-9fc5c5c2f2fa", 2024-09-11T22:26:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:26:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:30.814 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:30.815 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:26:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 -- 172.21.15.5:0/903022456 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fdc0a2cf0 msgr2=0x7f0fdc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fdc0a2cf0 0x7f0fdc0a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8020000 tx=0x7f0fe801d000 comp rx=0 tx=0).stop 2024-09-11T22:26:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 -- 172.21.15.5:0/903022456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff0100e20 msgr2=0x7f0ff0198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff0100e20 0x7f0ff0198540 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c970 tx=0x7f0fec00ce30 comp rx=0 tx=0).stop 2024-09-11T22:26:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 -- 172.21.15.5:0/903022456 shutdown_connections 2024-09-11T22:26:30.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff0102ef0 0x7f0ff019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fdc0a2cf0 0x7f0fdc0a51b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff0102570 0x7f0ff0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 --2- 172.21.15.5:0/903022456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff0100e20 0x7f0ff0198540 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 -- 172.21.15.5:0/903022456 >> 172.21.15.5:0/903022456 conn(0x7f0ff00fc7d0 msgr2=0x7f0ff0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 -- 172.21.15.5:0/903022456 shutdown_connections 2024-09-11T22:26:30.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:30.809+0000 7f0ff8316640 1 -- 172.21.15.5:0/903022456 wait complete. 2024-09-11T22:26:31.314 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 10.79 MB/sec execute 49 sec latency 10421.016 ms 2024-09-11T22:26:31.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:31 smithi005 ceph-mon[30943]: pgmap v133: 97 pgs: 97 active+clean; 77 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 135 op/s 2024-09-11T22:26:31.669 INFO:tasks.cephfs.filesystem:scrub status for tag:e47d6bef-30d2-4481-aa2d-9fc5c5c2f2fa - {'path': '/', 'tag': 'e47d6bef-30d2-4481-aa2d-9fc5c5c2f2fa', 'options': 'recursive,force'} 2024-09-11T22:26:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:31 smithi159 ceph-mon[35534]: pgmap v133: 97 pgs: 97 active+clean; 77 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 135 op/s 2024-09-11T22:26:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:31 smithi172 ceph-mon[35461]: pgmap v133: 97 pgs: 97 active+clean; 77 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 135 op/s 2024-09-11T22:26:32.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 10.57 MB/sec execute 50 sec latency 11421.136 ms 2024-09-11T22:26:33.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 10.37 MB/sec execute 51 sec latency 12421.246 ms 2024-09-11T22:26:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:33 smithi005 ceph-mon[30943]: pgmap v134: 97 pgs: 97 active+clean; 59 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-09-11T22:26:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:33 smithi159 ceph-mon[35534]: pgmap v134: 97 pgs: 97 active+clean; 59 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-09-11T22:26:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:33 smithi172 ceph-mon[35461]: pgmap v134: 97 pgs: 97 active+clean; 59 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-09-11T22:26:34.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 10.17 MB/sec execute 52 sec latency 13421.356 ms 2024-09-11T22:26:35.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 9.98 MB/sec execute 53 sec latency 14421.462 ms 2024-09-11T22:26:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:35 smithi005 ceph-mon[30943]: pgmap v135: 97 pgs: 97 active+clean; 59 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T22:26:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:35 smithi159 ceph-mon[35534]: pgmap v135: 97 pgs: 97 active+clean; 59 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T22:26:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:35 smithi172 ceph-mon[35461]: pgmap v135: 97 pgs: 97 active+clean; 59 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T22:26:36.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 9.79 MB/sec execute 54 sec latency 15421.560 ms 2024-09-11T22:26:36.838 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:26:37.096 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:37.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 9.61 MB/sec execute 55 sec latency 16421.683 ms 2024-09-11T22:26:37.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:37 smithi005 ceph-mon[30943]: pgmap v136: 97 pgs: 97 active+clean; 59 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T22:26:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 -- 172.21.15.5:0/3022923312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8102ef0 msgr2=0x7ff7f810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 --2- 172.21.15.5:0/3022923312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8102ef0 0x7ff7f810da40 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009ca0 tx=0x7ff7f002a950 comp rx=0 tx=0).stop 2024-09-11T22:26:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 -- 172.21.15.5:0/3022923312 shutdown_connections 2024-09-11T22:26:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 --2- 172.21.15.5:0/3022923312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8102ef0 0x7ff7f810da40 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 --2- 172.21.15.5:0/3022923312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7f8102570 0x7ff7f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 --2- 172.21.15.5:0/3022923312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8100e20 0x7ff7f8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 -- 172.21.15.5:0/3022923312 >> 172.21.15.5:0/3022923312 conn(0x7ff7f80fc7d0 msgr2=0x7ff7f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 -- 172.21.15.5:0/3022923312 shutdown_connections 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 -- 172.21.15.5:0/3022923312 wait complete. 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 Processor -- start 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.624+0000 7ff800677640 1 -- start start 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff800677640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8100e20 0x7ff7f8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff800677640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8102570 0x7ff7f8199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff800677640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7f8102ef0 0x7ff7f81a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff800677640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8110fa0 con 0x7ff7f8100e20 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff800677640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8110e20 con 0x7ff7f8102570 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff800677640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8111120 con 0x7ff7f8102ef0 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7fe413640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8100e20 0x7ff7f8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7fe413640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8100e20 0x7ff7f8199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55886/0 (socket says 172.21.15.5:55886) 2024-09-11T22:26:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7fe413640 1 -- 172.21.15.5:0/2388464954 learned_addr learned my addr 172.21.15.5:0/2388464954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7fec14640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7f8102ef0 0x7ff7f81a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7fe413640 1 -- 172.21.15.5:0/2388464954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7f8102ef0 msgr2=0x7ff7f81a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7fe413640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7f8102ef0 0x7ff7f81a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7fe413640 1 -- 172.21.15.5:0/2388464954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8102570 msgr2=0x7ff7f8199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7fe413640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8102570 0x7ff7f8199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7fe413640 1 -- 172.21.15.5:0/2388464954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f81a4910 con 0x7ff7f8100e20 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7fe413640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8100e20 0x7ff7f8199350 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400d950 tx=0x7ff7f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7ef7fe640 1 -- 172.21.15.5:0/2388464954 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4017070 con 0x7ff7f8100e20 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff800677640 1 -- 172.21.15.5:0/2388464954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f81a4ae0 con 0x7ff7f8100e20 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7ef7fe640 1 -- 172.21.15.5:0/2388464954 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7f4009c30 con 0x7ff7f8100e20 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.625+0000 7ff7ef7fe640 1 -- 172.21.15.5:0/2388464954 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f400f6b0 con 0x7ff7f8100e20 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.626+0000 7ff800677640 1 -- 172.21.15.5:0/2388464954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f81a4ff0 con 0x7ff7f8100e20 2024-09-11T22:26:37.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.627+0000 7ff800677640 1 -- 172.21.15.5:0/2388464954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f8101200 con 0x7ff7f8100e20 2024-09-11T22:26:37.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.628+0000 7ff7ef7fe640 1 -- 172.21.15.5:0/2388464954 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff7f40109d0 con 0x7ff7f8100e20 2024-09-11T22:26:37.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.628+0000 7ff7ef7fe640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff7e40a2cf0 0x7ff7e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:37.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.628+0000 7ff7fdc12640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff7e40a2cf0 0x7ff7e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:37.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.628+0000 7ff7ef7fe640 1 -- 172.21.15.5:0/2388464954 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7ff7f40cf150 con 0x7ff7f8100e20 2024-09-11T22:26:37.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.629+0000 7ff7fdc12640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff7e40a2cf0 0x7ff7e40a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7f806aac0 tx=0x7ff7e8006c90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:37.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.633+0000 7ff7ef7fe640 1 -- 172.21.15.5:0/2388464954 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7f4009de0 con 0x7ff7f8100e20 2024-09-11T22:26:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:37 smithi159 ceph-mon[35534]: pgmap v136: 97 pgs: 97 active+clean; 59 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T22:26:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.845+0000 7ff800677640 1 -- 172.21.15.5:0/2388464954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7f81a5220 con 0x7ff7f8100e20 2024-09-11T22:26:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.846+0000 7ff7ef7fe640 1 -- 172.21.15.5:0/2388464954 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11967 (secure 0 0 0) 0x7ff7f409bd30 con 0x7ff7f8100e20 2024-09-11T22:26:37.846 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:26:37.847 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-11T22:25:49:149525+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:25:48.784660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25970":{"gid":25970,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4169170564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4169170564},{"type":"v1","addr":"172.21.15.159:6833","nonce":4169170564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:26:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-11T22:26:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 -- 172.21.15.5:0/2388464954 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff7e40a2cf0 msgr2=0x7ff7e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff7e40a2cf0 0x7ff7e40a51b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7f806aac0 tx=0x7ff7e8006c90 comp rx=0 tx=0).stop 2024-09-11T22:26:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 -- 172.21.15.5:0/2388464954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8100e20 msgr2=0x7ff7f8199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8100e20 0x7ff7f8199350 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400d950 tx=0x7ff7f400de10 comp rx=0 tx=0).stop 2024-09-11T22:26:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 -- 172.21.15.5:0/2388464954 shutdown_connections 2024-09-11T22:26:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7f8102ef0 0x7ff7f81a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff7e40a2cf0 0x7ff7e40a51b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8102570 0x7ff7f8199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 --2- 172.21.15.5:0/2388464954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8100e20 0x7ff7f8199350 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 -- 172.21.15.5:0/2388464954 >> 172.21.15.5:0/2388464954 conn(0x7ff7f80fc7d0 msgr2=0x7ff7f8106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 -- 172.21.15.5:0/2388464954 shutdown_connections 2024-09-11T22:26:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:37.848+0000 7ff800677640 1 -- 172.21.15.5:0/2388464954 wait complete. 2024-09-11T22:26:37.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:37 smithi172 ceph-mon[35461]: pgmap v136: 97 pgs: 97 active+clean; 59 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-11T22:26:38.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 9.44 MB/sec execute 56 sec latency 17421.753 ms 2024-09-11T22:26:38.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:38 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2388464954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:38.675 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 9d83d6f2 --release --await' 2024-09-11T22:26:38.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 9d83d6f2 --release --await 2024-09-11T22:26:38.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:38 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2388464954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:38.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:38 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2388464954' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:39.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 312898 9.28 MB/sec execute 57 sec latency 18421.813 ms 2024-09-11T22:26:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.493+0000 7f6c3a854640 1 -- 172.21.15.5:0/3114722212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c34106f10 msgr2=0x7f6c3410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.493+0000 7f6c3a854640 1 --2- 172.21.15.5:0/3114722212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c34106f10 0x7f6c3410ea90 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009ca0 tx=0x7f6c2c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.493+0000 7f6c3a854640 1 -- 172.21.15.5:0/3114722212 shutdown_connections 2024-09-11T22:26:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.493+0000 7f6c3a854640 1 --2- 172.21.15.5:0/3114722212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c34106f10 0x7f6c3410ea90 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.493+0000 7f6c3a854640 1 --2- 172.21.15.5:0/3114722212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c34102790 0x7f6c341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.493+0000 7f6c3a854640 1 --2- 172.21.15.5:0/3114722212 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c34101de0 0x7f6c341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.494+0000 7f6c3a854640 1 -- 172.21.15.5:0/3114722212 >> 172.21.15.5:0/3114722212 conn(0x7f6c340fd8c0 msgr2=0x7f6c340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.494+0000 7f6c3a854640 1 -- 172.21.15.5:0/3114722212 shutdown_connections 2024-09-11T22:26:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.494+0000 7f6c3a854640 1 -- 172.21.15.5:0/3114722212 wait complete. 2024-09-11T22:26:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.494+0000 7f6c3a854640 1 Processor -- start 2024-09-11T22:26:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.494+0000 7f6c3a854640 1 -- start start 2024-09-11T22:26:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c3a854640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c34101de0 0x7f6c34198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c3a854640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c34102790 0x7f6c34198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c3a854640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c34106f10 0x7f6c3419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c3a854640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c34112010 con 0x7f6c34101de0 2024-09-11T22:26:39.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c3a854640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c34111e90 con 0x7f6c34106f10 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c3a854640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c34112190 con 0x7f6c34102790 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c34101de0 0x7f6c34198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c34101de0 0x7f6c34198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55906/0 (socket says 172.21.15.5:55906) 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c33fff640 1 -- 172.21.15.5:0/75325023 learned_addr learned my addr 172.21.15.5:0/75325023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c38df1640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c34106f10 0x7f6c3419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c337fe640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c34102790 0x7f6c34198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c33fff640 1 -- 172.21.15.5:0/75325023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c34102790 msgr2=0x7f6c34198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c33fff640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c34102790 0x7f6c34198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c33fff640 1 -- 172.21.15.5:0/75325023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c34106f10 msgr2=0x7f6c3419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c33fff640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c34106f10 0x7f6c3419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c33fff640 1 -- 172.21.15.5:0/75325023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c3419d520 con 0x7f6c34101de0 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c33fff640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c34101de0 0x7f6c34198570 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400b4a0 tx=0x7f6c2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c317fa640 1 -- 172.21.15.5:0/75325023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24014070 con 0x7f6c34101de0 2024-09-11T22:26:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c3a854640 1 -- 172.21.15.5:0/75325023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c3419d810 con 0x7f6c34101de0 2024-09-11T22:26:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c317fa640 1 -- 172.21.15.5:0/75325023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c24010af0 con 0x7f6c34101de0 2024-09-11T22:26:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.495+0000 7f6c317fa640 1 -- 172.21.15.5:0/75325023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24017d10 con 0x7f6c34101de0 2024-09-11T22:26:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.496+0000 7f6c3a854640 1 -- 172.21.15.5:0/75325023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c341a50f0 con 0x7f6c34101de0 2024-09-11T22:26:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.498+0000 7f6c317fa640 1 -- 172.21.15.5:0/75325023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6c24017070 con 0x7f6c34101de0 2024-09-11T22:26:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.498+0000 7f6c3cc2d740 1 Processor -- start 2024-09-11T22:26:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.498+0000 7f6c3cc2d740 1 -- start start 2024-09-11T22:26:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.498+0000 7f6c317fa640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c1c0a2cf0 0x7f6c1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.498+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f43f40 0x55b5e2f44320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.498+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f44860 0x55b5e2f53110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.498+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2f55a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.498+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2ea9620 con 0x55b5e2f43f40 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.498+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2e50e50 con 0x55b5e2f53650 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.498+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2ea55e0 con 0x55b5e2f44860 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c38df1640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c1c0a2cf0 0x7f6c1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f43f40 0x55b5e2f44320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c317fa640 1 -- 172.21.15.5:0/75325023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f6c240ce800 con 0x7f6c34101de0 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f43f40 0x55b5e2f44320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55920/0 (socket says 172.21.15.5:55920) 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c33fff640 1 -- 172.21.15.5:0/260608481 learned_addr learned my addr 172.21.15.5:0/260608481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c337fe640 1 --2- 172.21.15.5:0/260608481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f44860 0x55b5e2f53110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c38df1640 1 --2- 172.21.15.5:0/260608481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2f55a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c33fff640 1 -- 172.21.15.5:0/260608481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f44860 msgr2=0x55b5e2f53110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c33fff640 1 --2- 172.21.15.5:0/260608481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f44860 0x55b5e2f53110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c33fff640 1 -- 172.21.15.5:0/260608481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 msgr2=0x55b5e2f55a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c33fff640 1 --2- 172.21.15.5:0/260608481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2f55a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c33fff640 1 -- 172.21.15.5:0/260608481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e2e41d40 con 0x55b5e2f43f40 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c38df1640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c1c0a2cf0 0x7f6c1c0a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c006010 tx=0x7f6c2c031d20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c33fff640 1 --2- 172.21.15.5:0/260608481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f43f40 0x55b5e2f44320 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6c2409ad80 tx=0x7f6c24017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163444812a2a6a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.499+0000 7f6c1a7fc640 1 -- 172.21.15.5:0/260608481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c240adb10 con 0x55b5e2f43f40 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c1a7fc640 1 -- 172.21.15.5:0/260608481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2409b4f0 con 0x55b5e2f43f40 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c1a7fc640 1 -- 172.21.15.5:0/260608481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c240af0b0 con 0x55b5e2f43f40 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/260608481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f43f40 msgr2=0x55b5e2f44320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/260608481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f43f40 0x55b5e2f44320 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6c2409ad80 tx=0x7f6c24017730 comp rx=0 tx=0).stop 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/260608481 shutdown_connections 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/260608481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2f55a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/260608481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f44860 0x55b5e2f53110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/260608481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f43f40 0x55b5e2f44320 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/260608481 >> 172.21.15.5:0/260608481 conn(0x55b5e2f405f0 msgr2=0x55b5e2f409d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:39.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/260608481 shutdown_connections 2024-09-11T22:26:39.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.500+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/260608481 wait complete. 2024-09-11T22:26:39.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c3cc2d740 1 -- start start 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c3cc2d740 1 Processor -- start 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f43f40 0x55b5e2f505d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f50b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f53650 0x55b5e2f51050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2f56220 con 0x55b5e2f44860 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2e31f10 con 0x55b5e2f43f40 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2f563a0 con 0x55b5e2f53650 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c3cc2d740 10 client.0 _ll_register_callbacks cb 0x55b5e2e55db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f50b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f50b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55926/0 (socket says 172.21.15.5:55926) 2024-09-11T22:26:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c337fe640 1 -- 172.21.15.5:0/3215122052 learned_addr learned my addr 172.21.15.5:0/3215122052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.501+0000 7f6c33fff640 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f43f40 0x55b5e2f505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c38df1640 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f53650 0x55b5e2f51050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c337fe640 1 -- 172.21.15.5:0/3215122052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f53650 msgr2=0x55b5e2f51050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c337fe640 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f53650 0x55b5e2f51050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c337fe640 1 -- 172.21.15.5:0/3215122052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f43f40 msgr2=0x55b5e2f505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c337fe640 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f43f40 0x55b5e2f505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c337fe640 1 -- 172.21.15.5:0/3215122052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e2bb8580 con 0x55b5e2f44860 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c337fe640 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f50b10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6c28010500 tx=0x7f6c280109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfe01cb925b5b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c137fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c137fe640 1 -- 172.21.15.5:0/3215122052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2801a020 con 0x55b5e2f44860 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c137fe640 1 -- 172.21.15.5:0/3215122052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f6c28016750 con 0x55b5e2f44860 2024-09-11T22:26:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3215122052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2409b4f0 con 0x55b5e2f44860 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c137fe640 1 -- 172.21.15.5:0/3215122052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6c28025050 con 0x55b5e2f44860 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c3cc2d740 10 client.16794 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.502+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3215122052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b5e2c82960 con 0x55b5e2f44860 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.503+0000 7f6c137fe640 1 -- 172.21.15.5:0/3215122052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2409b4f0 con 0x55b5e2f44860 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.503+0000 7f6c137fe640 1 -- 172.21.15.5:0/3215122052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f6c28014b90 con 0x55b5e2f44860 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.503+0000 7f6c3cc2d740 10 client.16794 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.503+0000 7f6c3cc2d740 10 client.16794 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.503+0000 7f6c3cc2d740 20 client.16794 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.503+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b5e2f632e0 0x55b5e2f65750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.503+0000 7f6c3cc2d740 4 client.16794 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.503+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3215122052 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5e2eb6b90 con 0x55b5e2f632e0 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.503+0000 7f6c33fff640 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b5e2f632e0 0x55b5e2f65750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.504+0000 7f6c33fff640 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b5e2f632e0 0x55b5e2f65750 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7314d8efef0b43aa server_cookie=2fba8afdc10c9a08 in_seq=0 out_seq=0 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.504+0000 7f6c137fe640 10 client.16794 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.504+0000 7f6c137fe640 1 -- 172.21.15.5:0/3215122052 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5e2eb6b90 con 0x55b5e2f632e0 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.504+0000 7f6c137fe640 10 client.16794 handle_command_reply: tid=0 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.504+0000 7f6c3cc2d740 1 client.16794 shutdown 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3215122052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 msgr2=0x55b5e2f50b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f50b10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6c28010500 tx=0x7f6c280109c0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3215122052 shutdown_connections 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b5e2f632e0 0x55b5e2f65750 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f53650 0x55b5e2f51050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f50b10 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3215122052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f43f40 0x55b5e2f505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c137fe640 0 client.16794 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3215122052 >> 172.21.15.5:0/3215122052 conn(0x55b5e2f405f0 msgr2=0x55b5e2f46350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:39.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3215122052 shutdown_connections 2024-09-11T22:26:39.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3215122052 wait complete. 2024-09-11T22:26:39.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.505+0000 7f6c3cc2d740 20 client.16794 trim_cache size 0 max 16384 2024-09-11T22:26:39.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c3cc2d740 1 Processor -- start 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c3cc2d740 1 -- start start 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f43f40 0x55b5e2f6b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f6b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2f4d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c341ab9a0 con 0x55b5e2f44860 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c341ad4e0 con 0x55b5e2f53650 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c341ac070 con 0x55b5e2f43f40 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f6b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f6b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55938/0 (socket says 172.21.15.5:55938) 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c337fe640 1 -- 172.21.15.5:0/1885355905 learned_addr learned my addr 172.21.15.5:0/1885355905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.522+0000 7f6c33fff640 1 --2- 172.21.15.5:0/1885355905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f43f40 0x55b5e2f6b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c337fe640 1 -- 172.21.15.5:0/1885355905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f43f40 msgr2=0x55b5e2f6b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c337fe640 1 --2- 172.21.15.5:0/1885355905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f43f40 0x55b5e2f6b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c337fe640 1 -- 172.21.15.5:0/1885355905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 msgr2=0x55b5e2f4d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c337fe640 1 --2- 172.21.15.5:0/1885355905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2f4d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c337fe640 1 -- 172.21.15.5:0/1885355905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e2f45550 con 0x55b5e2f44860 2024-09-11T22:26:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c337fe640 1 --2- 172.21.15.5:0/1885355905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f6b5a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6c28010f40 tx=0x7f6c2800be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830ebd966d48e155 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c19ffb640 1 -- 172.21.15.5:0/1885355905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2801a040 con 0x55b5e2f44860 2024-09-11T22:26:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c19ffb640 1 -- 172.21.15.5:0/1885355905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c28012e10 con 0x55b5e2f44860 2024-09-11T22:26:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c19ffb640 1 -- 172.21.15.5:0/1885355905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c28016bb0 con 0x55b5e2f44860 2024-09-11T22:26:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/1885355905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 msgr2=0x55b5e2f6b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.523+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/1885355905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f6b5a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6c28010f40 tx=0x7f6c2800be90 comp rx=0 tx=0).stop 2024-09-11T22:26:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/1885355905 shutdown_connections 2024-09-11T22:26:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/1885355905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2f4d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/1885355905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f6b5a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/1885355905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f43f40 0x55b5e2f6b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/1885355905 >> 172.21.15.5:0/1885355905 conn(0x55b5e2f405f0 msgr2=0x55b5e2f4eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/1885355905 shutdown_connections 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/1885355905 wait complete. 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 -- start start 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 Processor -- start 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f43f40 0x55b5e2f577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f57cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2fcd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c341ace10 con 0x55b5e2f44860 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c341ac740 con 0x55b5e2f53650 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c341a9980 con 0x55b5e2f43f40 2024-09-11T22:26:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.524+0000 7f6c3cc2d740 10 client.0 _ll_register_callbacks cb 0x7f6c2800bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f57cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f57cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55946/0 (socket says 172.21.15.5:55946) 2024-09-11T22:26:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c337fe640 1 -- 172.21.15.5:0/3829000905 learned_addr learned my addr 172.21.15.5:0/3829000905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c33fff640 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f43f40 0x55b5e2f577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c38df1640 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2fcd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c337fe640 1 -- 172.21.15.5:0/3829000905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f43f40 msgr2=0x55b5e2f577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c337fe640 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f43f40 0x55b5e2f577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c337fe640 1 -- 172.21.15.5:0/3829000905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 msgr2=0x55b5e2fcd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c337fe640 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2fcd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c337fe640 1 -- 172.21.15.5:0/3829000905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e2fcd570 con 0x55b5e2f44860 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c38df1640 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2fcd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c337fe640 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f57cf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6c28012760 tx=0x7f6c28012790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94415120479ac24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c137fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c137fe640 1 -- 172.21.15.5:0/3829000905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2801a040 con 0x55b5e2f44860 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c137fe640 1 -- 172.21.15.5:0/3829000905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f6c280169f0 con 0x55b5e2f44860 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3829000905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c28012e10 con 0x55b5e2f44860 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c137fe640 1 -- 172.21.15.5:0/3829000905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6c28016c10 con 0x55b5e2f44860 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.525+0000 7f6c3cc2d740 10 client.16800 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.526+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3829000905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b5e2fce730 con 0x55b5e2f44860 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.526+0000 7f6c137fe640 1 -- 172.21.15.5:0/3829000905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c28012e10 con 0x55b5e2f44860 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.526+0000 7f6c137fe640 1 -- 172.21.15.5:0/3829000905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f6c28029430 con 0x55b5e2f44860 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.526+0000 7f6c3cc2d740 10 client.16800 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.526+0000 7f6c3cc2d740 10 client.16800 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:26:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.526+0000 7f6c3cc2d740 20 client.16800 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.526+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b5e2fcee80 0x55b5e2fd12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.526+0000 7f6c3cc2d740 4 client.16800 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9d83d6f2", "release": true, "await": true}] 2024-09-11T22:26:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.526+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3829000905 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9d83d6f2", "release": true, "await": true}) -- 0x55b5e2eb10a0 con 0x55b5e2fcee80 2024-09-11T22:26:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.526+0000 7f6c33fff640 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b5e2fcee80 0x55b5e2fd12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.527+0000 7f6c33fff640 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b5e2fcee80 0x55b5e2fd12a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de9b795be8133b2 server_cookie=7a80b51c45e3b5a4 in_seq=0 out_seq=0 2024-09-11T22:26:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.527+0000 7f6c137fe640 10 client.16800 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c137fe640 1 -- 172.21.15.5:0/3829000905 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b5e2eb10a0 con 0x55b5e2fcee80 2024-09-11T22:26:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c137fe640 10 client.16800 handle_command_reply: tid=0 2024-09-11T22:26:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 client.16800 shutdown 2024-09-11T22:26:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3829000905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 msgr2=0x55b5e2f57cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f57cf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6c28012760 tx=0x7f6c28012790 comp rx=0 tx=0).stop 2024-09-11T22:26:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3829000905 shutdown_connections 2024-09-11T22:26:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b5e2fcee80 0x55b5e2fd12a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b5e2f53650 0x55b5e2fcd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c137fe640 0 client.16800 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5e2f44860 0x55b5e2f57cf0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 --2- 172.21.15.5:0/3829000905 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5e2f43f40 0x55b5e2f577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3829000905 >> 172.21.15.5:0/3829000905 conn(0x55b5e2f405f0 msgr2=0x55b5e2f3ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3829000905 shutdown_connections 2024-09-11T22:26:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 1 -- 172.21.15.5:0/3829000905 wait complete. 2024-09-11T22:26:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.538+0000 7f6c3cc2d740 20 client.16800 trim_cache size 0 max 16384 2024-09-11T22:26:39.540 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:26:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 34, 2024-09-11T22:26:39.540 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 16, 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "9d83d6f2": { 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 16, 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.2, 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:26:39.541 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:26:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:26:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:26:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:26:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:26:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:39.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:39.542 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 -- 172.21.15.5:0/75325023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c1c0a2cf0 msgr2=0x7f6c1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c1c0a2cf0 0x7f6c1c0a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c006010 tx=0x7f6c2c031d20 comp rx=0 tx=0).stop 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 -- 172.21.15.5:0/75325023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c34101de0 msgr2=0x7f6c34198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c34101de0 0x7f6c34198570 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400b4a0 tx=0x7f6c2400b960 comp rx=0 tx=0).stop 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 -- 172.21.15.5:0/75325023 shutdown_connections 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c1c0a2cf0 0x7f6c1c0a51b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c34106f10 0x7f6c3419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c34102790 0x7f6c34198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 --2- 172.21.15.5:0/75325023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c34101de0 0x7f6c34198570 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 -- 172.21.15.5:0/75325023 >> 172.21.15.5:0/75325023 conn(0x7f6c340fd8c0 msgr2=0x7f6c340fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 -- 172.21.15.5:0/75325023 shutdown_connections 2024-09-11T22:26:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:39.539+0000 7f6c3a854640 1 -- 172.21.15.5:0/75325023 wait complete. 2024-09-11T22:26:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:39 smithi005 ceph-mon[30943]: pgmap v137: 97 pgs: 97 active+clean; 62 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.6 MiB/s wr, 177 op/s 2024-09-11T22:26:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:26:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:26:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:39 smithi159 ceph-mon[35534]: pgmap v137: 97 pgs: 97 active+clean; 62 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.6 MiB/s wr, 177 op/s 2024-09-11T22:26:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:26:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:26:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:39 smithi172 ceph-mon[35461]: pgmap v137: 97 pgs: 97 active+clean; 62 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 2.6 MiB/s wr, 177 op/s 2024-09-11T22:26:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:26:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:26:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.163+0000 7fcabe05e640 1 -- 172.21.15.5:0/319474369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab8107b70 msgr2=0x7fcab8107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.163+0000 7fcabe05e640 1 --2- 172.21.15.5:0/319474369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab8107b70 0x7fcab8107f50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009930 tx=0x7fcaa802a420 comp rx=0 tx=0).stop 2024-09-11T22:26:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.163+0000 7fcabe05e640 1 -- 172.21.15.5:0/319474369 shutdown_connections 2024-09-11T22:26:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.163+0000 7fcabe05e640 1 --2- 172.21.15.5:0/319474369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcab8102470 0x7fcab810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.163+0000 7fcabe05e640 1 --2- 172.21.15.5:0/319474369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab8101ad0 0x7fcab8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.163+0000 7fcabe05e640 1 --2- 172.21.15.5:0/319474369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab8107b70 0x7fcab8107f50 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.163+0000 7fcabe05e640 1 -- 172.21.15.5:0/319474369 >> 172.21.15.5:0/319474369 conn(0x7fcab80fd7d0 msgr2=0x7fcab80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.164+0000 7fcabe05e640 1 -- 172.21.15.5:0/319474369 shutdown_connections 2024-09-11T22:26:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.164+0000 7fcabe05e640 1 -- 172.21.15.5:0/319474369 wait complete. 2024-09-11T22:26:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.164+0000 7fcabe05e640 1 Processor -- start 2024-09-11T22:26:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.164+0000 7fcabe05e640 1 -- start start 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.164+0000 7fcabe05e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcab8101ad0 0x7fcab81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.164+0000 7fcabe05e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab8102470 0x7fcab8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.164+0000 7fcabe05e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab8107b70 0x7fcab819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.164+0000 7fcabe05e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8111f20 con 0x7fcab8107b70 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.164+0000 7fcabe05e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8111da0 con 0x7fcab8101ad0 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.164+0000 7fcabe05e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81120a0 con 0x7fcab8102470 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab8107b70 0x7fcab819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcab8101ad0 0x7fcab81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab8107b70 0x7fcab819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55966/0 (socket says 172.21.15.5:55966) 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcaaeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab8102470 0x7fcab8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab7fff640 1 -- 172.21.15.5:0/21666592 learned_addr learned my addr 172.21.15.5:0/21666592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab7fff640 1 -- 172.21.15.5:0/21666592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab8102470 msgr2=0x7fcab8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab7fff640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab8102470 0x7fcab8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab7fff640 1 -- 172.21.15.5:0/21666592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcab8101ad0 msgr2=0x7fcab81983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab7fff640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcab8101ad0 0x7fcab81983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab7fff640 1 -- 172.21.15.5:0/21666592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab819d3e0 con 0x7fcab8107b70 2024-09-11T22:26:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab7fff640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab8107b70 0x7fcab819ccb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcab000eb10 tx=0x7fcab000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab57fa640 1 -- 172.21.15.5:0/21666592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0004b20 con 0x7fcab8107b70 2024-09-11T22:26:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcabe05e640 1 -- 172.21.15.5:0/21666592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab819d6d0 con 0x7fcab8107b70 2024-09-11T22:26:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab57fa640 1 -- 172.21.15.5:0/21666592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcab0004cc0 con 0x7fcab8107b70 2024-09-11T22:26:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcab57fa640 1 -- 172.21.15.5:0/21666592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0010760 con 0x7fcab8107b70 2024-09-11T22:26:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.165+0000 7fcabe05e640 1 -- 172.21.15.5:0/21666592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab81a4f60 con 0x7fcab8107b70 2024-09-11T22:26:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.168+0000 7fcab57fa640 1 -- 172.21.15.5:0/21666592 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fcab0005020 con 0x7fcab8107b70 2024-09-11T22:26:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.168+0000 7fcac0df9740 1 Processor -- start 2024-09-11T22:26:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.168+0000 7fcac0df9740 1 -- start start 2024-09-11T22:26:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c935ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c940770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c9430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcac0df9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56268c89a2f0 con 0x56268c936410 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcac0df9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56268c842270 con 0x56268c940cb0 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcac0df9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56268c8962b0 con 0x56268c935af0 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcaaeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c935ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcab77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c940770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcab77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c940770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55978/0 (socket says 172.21.15.5:55978) 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcaaeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c935ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:53934/0 (socket says 172.21.15.5:53934) 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcab77fe640 1 -- 172.21.15.5:0/2169212990 learned_addr learned my addr 172.21.15.5:0/2169212990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/2169212990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c9430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcab57fa640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcaa40a2da0 0x7fcaa40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcaaeffd640 1 -- 172.21.15.5:0/2169212990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 msgr2=0x56268c9430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/2169212990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c9430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcaaeffd640 1 -- 172.21.15.5:0/2169212990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 msgr2=0x56268c940770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/2169212990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c940770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcaaeffd640 1 -- 172.21.15.5:0/2169212990 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268c841340 con 0x56268c935af0 2024-09-11T22:26:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcab7fff640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcaa40a2da0 0x7fcaa40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcab57fa640 1 -- 172.21.15.5:0/21666592 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fcab00cfac0 con 0x7fcab8107b70 2024-09-11T22:26:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.169+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/2169212990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c935ed0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcab80ffd40 tx=0x7fcaa0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9813bfbe90d6501a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcaadffb640 1 -- 172.21.15.5:0/2169212990 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0024070 con 0x56268c935af0 2024-09-11T22:26:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcaadffb640 1 -- 172.21.15.5:0/2169212990 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa001f660 con 0x56268c935af0 2024-09-11T22:26:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcaadffb640 1 -- 172.21.15.5:0/2169212990 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00037f0 con 0x56268c935af0 2024-09-11T22:26:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcac0df9740 1 -- 172.21.15.5:0/2169212990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 msgr2=0x56268c935ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcac0df9740 1 --2- 172.21.15.5:0/2169212990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c935ed0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcab80ffd40 tx=0x7fcaa0004730 comp rx=0 tx=0).stop 2024-09-11T22:26:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcab7fff640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcaa40a2da0 0x7fcaa40a5260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcab009bf30 tx=0x7fcab006df60 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcac0df9740 1 -- 172.21.15.5:0/2169212990 shutdown_connections 2024-09-11T22:26:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcac0df9740 1 --2- 172.21.15.5:0/2169212990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c9430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcac0df9740 1 --2- 172.21.15.5:0/2169212990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c940770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcac0df9740 1 --2- 172.21.15.5:0/2169212990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c935ed0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcac0df9740 1 -- 172.21.15.5:0/2169212990 >> 172.21.15.5:0/2169212990 conn(0x56268c932060 msgr2=0x56268c932440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcac0df9740 1 -- 172.21.15.5:0/2169212990 shutdown_connections 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.170+0000 7fcac0df9740 1 -- 172.21.15.5:0/2169212990 wait complete. 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcac0df9740 1 -- start start 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcac0df9740 1 Processor -- start 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c93dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c93e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c93e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcac0df9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56268c943880 con 0x56268c936410 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcac0df9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56268c823290 con 0x56268c940cb0 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcac0df9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56268c943a00 con 0x56268c935af0 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcac0df9740 10 client.0 _ll_register_callbacks cb 0x56268c854140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcab77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c93e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcab77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c93e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55994/0 (socket says 172.21.15.5:55994) 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcab77fe640 1 -- 172.21.15.5:0/3666593709 learned_addr learned my addr 172.21.15.5:0/3666593709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c93dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.171+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c93e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fcab77fe640 1 -- 172.21.15.5:0/3666593709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 msgr2=0x56268c93dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fcab77fe640 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c93dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fcab77fe640 1 -- 172.21.15.5:0/3666593709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 msgr2=0x56268c93e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fcab77fe640 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c93e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fcab77fe640 1 -- 172.21.15.5:0/3666593709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268c5ad210 con 0x56268c936410 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fcab77fe640 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c93e170 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009900 tx=0x7fcaa8031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72399c977fca87b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fca96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fca96ffd640 1 -- 172.21.15.5:0/3666593709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8035040 con 0x56268c936410 2024-09-11T22:26:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fca96ffd640 1 -- 172.21.15.5:0/3666593709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fcaa8004460 con 0x56268c936410 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fcac0df9740 1 -- 172.21.15.5:0/3666593709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa001f660 con 0x56268c936410 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fca96ffd640 1 -- 172.21.15.5:0/3666593709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcaa8005040 con 0x56268c936410 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fcac0df9740 10 client.16812 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fcac0df9740 1 -- 172.21.15.5:0/3666593709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56268c679620 con 0x56268c936410 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.172+0000 7fca96ffd640 1 -- 172.21.15.5:0/3666593709 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa001f660 con 0x56268c936410 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.173+0000 7fca96ffd640 1 -- 172.21.15.5:0/3666593709 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fcaa80035f0 con 0x56268c936410 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.173+0000 7fcac0df9740 10 client.16812 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.173+0000 7fcac0df9740 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.173+0000 7fcac0df9740 20 client.16812 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.173+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56268c950cf0 0x56268c9530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.173+0000 7fcac0df9740 4 client.16812 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.173+0000 7fcac0df9740 1 -- 172.21.15.5:0/3666593709 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56268c8a7a20 con 0x56268c950cf0 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.173+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56268c950cf0 0x56268c9530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.173+0000 7fca96ffd640 10 client.16812 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.173+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56268c950cf0 0x56268c9530e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5d4003bab69df1c server_cookie=8683ac8a435cd369 in_seq=0 out_seq=0 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fca96ffd640 1 -- 172.21.15.5:0/3666593709 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56268c8a7a20 con 0x56268c950cf0 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fca96ffd640 10 client.16812 handle_command_reply: tid=0 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 client.16812 shutdown 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 -- 172.21.15.5:0/3666593709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 msgr2=0x56268c93e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c93e170 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009900 tx=0x7fcaa8031850 comp rx=0 tx=0).stop 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 -- 172.21.15.5:0/3666593709 shutdown_connections 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56268c950cf0 0x56268c9530e0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c93e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c93e170 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3666593709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c93dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fca96ffd640 0 client.16812 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 -- 172.21.15.5:0/3666593709 >> 172.21.15.5:0/3666593709 conn(0x56268c932060 msgr2=0x56268c937dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 -- 172.21.15.5:0/3666593709 shutdown_connections 2024-09-11T22:26:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.174+0000 7fcac0df9740 1 -- 172.21.15.5:0/3666593709 wait complete. 2024-09-11T22:26:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.175+0000 7fcac0df9740 20 client.16812 trim_cache size 0 max 16384 2024-09-11T22:26:40.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcac0df9740 1 Processor -- start 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcac0df9740 1 -- start start 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c935af0 0x56268c93e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c936410 0x56268c9589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c940cb0 0x56268c958ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcac0df9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81ab7f0 con 0x56268c940cb0 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcac0df9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81ad330 con 0x56268c935af0 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcac0df9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81abec0 con 0x56268c936410 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcaaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c940cb0 0x56268c958ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcaaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c940cb0 0x56268c958ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56006/0 (socket says 172.21.15.5:56006) 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcaaeffd640 1 -- 172.21.15.5:0/1771256580 learned_addr learned my addr 172.21.15.5:0/1771256580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/1771256580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c935af0 0x56268c93e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcab77fe640 1 --2- 172.21.15.5:0/1771256580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c936410 0x56268c9589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcaaeffd640 1 -- 172.21.15.5:0/1771256580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c936410 msgr2=0x56268c9589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/1771256580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c936410 0x56268c9589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcaaeffd640 1 -- 172.21.15.5:0/1771256580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c935af0 msgr2=0x56268c93e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/1771256580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c935af0 0x56268c93e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.191+0000 7fcaaeffd640 1 -- 172.21.15.5:0/1771256580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268c94d6c0 con 0x56268c940cb0 2024-09-11T22:26:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.192+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/1771256580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c940cb0 0x56268c958ef0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00097a0 tx=0x7fcaa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e123b20ad8708f9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.192+0000 7fcaad7fa640 1 -- 172.21.15.5:0/1771256580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0024040 con 0x56268c940cb0 2024-09-11T22:26:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.192+0000 7fcaad7fa640 1 -- 172.21.15.5:0/1771256580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa001f710 con 0x56268c940cb0 2024-09-11T22:26:40.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.192+0000 7fcaad7fa640 1 -- 172.21.15.5:0/1771256580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0004d60 con 0x56268c940cb0 2024-09-11T22:26:40.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.192+0000 7fcac0df9740 1 -- 172.21.15.5:0/1771256580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c940cb0 msgr2=0x56268c958ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.192+0000 7fcac0df9740 1 --2- 172.21.15.5:0/1771256580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c940cb0 0x56268c958ef0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00097a0 tx=0x7fcaa0004730 comp rx=0 tx=0).stop 2024-09-11T22:26:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 -- 172.21.15.5:0/1771256580 shutdown_connections 2024-09-11T22:26:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 --2- 172.21.15.5:0/1771256580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c940cb0 0x56268c958ef0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 --2- 172.21.15.5:0/1771256580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c936410 0x56268c9589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 --2- 172.21.15.5:0/1771256580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c935af0 0x56268c93e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 -- 172.21.15.5:0/1771256580 >> 172.21.15.5:0/1771256580 conn(0x56268c932060 msgr2=0x56268c93fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 -- 172.21.15.5:0/1771256580 shutdown_connections 2024-09-11T22:26:40.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 -- 172.21.15.5:0/1771256580 wait complete. 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 -- start start 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 Processor -- start 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c949550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c944870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c944e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81acc60 con 0x56268c936410 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81ac590 con 0x56268c940cb0 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.193+0000 7fcac0df9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcab81a97d0 con 0x56268c935af0 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcac0df9740 10 client.0 _ll_register_callbacks cb 0x7fcaa8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcab77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c944870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcaaeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c949550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcab77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c944870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56016/0 (socket says 172.21.15.5:56016) 2024-09-11T22:26:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcab77fe640 1 -- 172.21.15.5:0/3040805894 learned_addr learned my addr 172.21.15.5:0/3040805894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c944e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcab77fe640 1 -- 172.21.15.5:0/3040805894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 msgr2=0x56268c949550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcab77fe640 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c949550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcab77fe640 1 -- 172.21.15.5:0/3040805894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 msgr2=0x56268c944e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcab77fe640 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c944e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcab77fe640 1 -- 172.21.15.5:0/3040805894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268c945690 con 0x56268c936410 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c944e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcab77fe640 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c944870 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8005940 tx=0x7fcaa8003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623aa3736698c53e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fca96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fca96ffd640 1 -- 172.21.15.5:0/3040805894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8035070 con 0x56268c936410 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fca96ffd640 1 -- 172.21.15.5:0/3040805894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fcaa80051e0 con 0x56268c936410 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcac0df9740 1 -- 172.21.15.5:0/3040805894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa001f710 con 0x56268c936410 2024-09-11T22:26:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fca96ffd640 1 -- 172.21.15.5:0/3040805894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcaa8005d50 con 0x56268c936410 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcac0df9740 10 client.16818 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.194+0000 7fcac0df9740 1 -- 172.21.15.5:0/3040805894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56268c9bbaa0 con 0x56268c936410 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fca96ffd640 1 -- 172.21.15.5:0/3040805894 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa001f710 con 0x56268c936410 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fca96ffd640 1 -- 172.21.15.5:0/3040805894 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fcaa80053e0 con 0x56268c936410 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fcac0df9740 10 client.16818 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fcac0df9740 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fcac0df9740 20 client.16818 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56268c9bc220 0x56268c9be640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fcac0df9740 4 client.16818 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fcac0df9740 1 -- 172.21.15.5:0/3040805894 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56268c8a1d70 con 0x56268c9bc220 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56268c9bc220 0x56268c9be640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56268c9bc220 0x56268c9be640 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d412ece10039b1 server_cookie=8c569d3d638247c6 in_seq=0 out_seq=0 2024-09-11T22:26:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.195+0000 7fca96ffd640 10 client.16818 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.232+0000 7fca96ffd640 1 -- 172.21.15.5:0/3040805894 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56268c8a1d70 con 0x56268c9bc220 2024-09-11T22:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.232+0000 7fca96ffd640 10 client.16818 handle_command_reply: tid=0 2024-09-11T22:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.232+0000 7fcac0df9740 1 client.16818 shutdown 2024-09-11T22:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 1 -- 172.21.15.5:0/3040805894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 msgr2=0x56268c944870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c944870 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8005940 tx=0x7fcaa8003800 comp rx=0 tx=0).stop 2024-09-11T22:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 1 -- 172.21.15.5:0/3040805894 shutdown_connections 2024-09-11T22:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56268c9bc220 0x56268c9be640 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56268c940cb0 0x56268c944e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56268c936410 0x56268c944870 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fca96ffd640 0 client.16818 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 1 --2- 172.21.15.5:0/3040805894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56268c935af0 0x56268c949550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 1 -- 172.21.15.5:0/3040805894 >> 172.21.15.5:0/3040805894 conn(0x56268c932060 msgr2=0x56268c9375a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 1 -- 172.21.15.5:0/3040805894 shutdown_connections 2024-09-11T22:26:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 1 -- 172.21.15.5:0/3040805894 wait complete. 2024-09-11T22:26:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.233+0000 7fcac0df9740 20 client.16818 trim_cache size 0 max 16384 2024-09-11T22:26:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:26:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:26:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:26:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 -- 172.21.15.5:0/21666592 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcaa40a2da0 msgr2=0x7fcaa40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcaa40a2da0 0x7fcaa40a5260 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcab009bf30 tx=0x7fcab006df60 comp rx=0 tx=0).stop 2024-09-11T22:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 -- 172.21.15.5:0/21666592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab8107b70 msgr2=0x7fcab819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab8107b70 0x7fcab819ccb0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcab000eb10 tx=0x7fcab000c2d0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 -- 172.21.15.5:0/21666592 shutdown_connections 2024-09-11T22:26:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab8107b70 0x7fcab819ccb0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcab8102470 0x7fcab8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcaa40a2da0 0x7fcaa40a5260 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 --2- 172.21.15.5:0/21666592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcab8101ad0 0x7fcab81983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 -- 172.21.15.5:0/21666592 >> 172.21.15.5:0/21666592 conn(0x7fcab80fd7d0 msgr2=0x7fcab80fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 -- 172.21.15.5:0/21666592 shutdown_connections 2024-09-11T22:26:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:40.234+0000 7fcabe05e640 1 -- 172.21.15.5:0/21666592 wait complete. 2024-09-11T22:26:40.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 315271 9.33 MB/sec execute 58 sec latency 18644.012 ms 2024-09-11T22:26:40.624 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9d83d6f2', total seconds elapsed: 23.9 2024-09-11T22:26:40.624 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-11T22:26:40.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:26:40.895 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:41.315 INFO:tasks.workunit.client.0.smithi005.stdout: 1 317372 9.37 MB/sec execute 59 sec latency 43.346 ms 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.560+0000 7f2979004640 1 -- 172.21.15.5:0/2213947919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2974100e20 msgr2=0x7f2974101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.560+0000 7f2979004640 1 --2- 172.21.15.5:0/2213947919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2974100e20 0x7f2974101200 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f2964009ca0 tx=0x7f296402a950 comp rx=0 tx=0).stop 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.560+0000 7f2979004640 1 -- 172.21.15.5:0/2213947919 shutdown_connections 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.560+0000 7f2979004640 1 --2- 172.21.15.5:0/2213947919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2974102ef0 0x7f297410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.560+0000 7f2979004640 1 --2- 172.21.15.5:0/2213947919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2974102570 0x7f29741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.560+0000 7f2979004640 1 --2- 172.21.15.5:0/2213947919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2974100e20 0x7f2974101200 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.560+0000 7f2979004640 1 -- 172.21.15.5:0/2213947919 >> 172.21.15.5:0/2213947919 conn(0x7f29740fc7d0 msgr2=0x7f29740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.560+0000 7f2979004640 1 -- 172.21.15.5:0/2213947919 shutdown_connections 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.560+0000 7f2979004640 1 -- 172.21.15.5:0/2213947919 wait complete. 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.561+0000 7f2979004640 1 Processor -- start 2024-09-11T22:26:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.561+0000 7f2979004640 1 -- start start 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.561+0000 7f2979004640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2974100e20 0x7f29741984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f2979004640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2974102570 0x7f2974198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f2979004640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2974102ef0 0x7f297419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f2979004640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2974110f80 con 0x7f2974100e20 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f2979004640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2974110e00 con 0x7f2974102570 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f2979004640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2974111100 con 0x7f2974102ef0 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f2972d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2974100e20 0x7f29741984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f2972d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2974100e20 0x7f29741984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44482/0 (socket says 172.21.15.5:44482) 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f2972d9d640 1 -- 172.21.15.5:0/2941582464 learned_addr learned my addr 172.21.15.5:0/2941582464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f297359e640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2974102ef0 0x7f297419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f297359e640 1 -- 172.21.15.5:0/2941582464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2974102570 msgr2=0x7f2974198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f297359e640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2974102570 0x7f2974198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f297359e640 1 -- 172.21.15.5:0/2941582464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2974100e20 msgr2=0x7f29741984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f297359e640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2974100e20 0x7f29741984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f297359e640 1 -- 172.21.15.5:0/2941582464 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297419d470 con 0x7f2974102ef0 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f297359e640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2974102ef0 0x7f297419cd90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f296c00e970 tx=0x7f296c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.562+0000 7f296bfff640 1 -- 172.21.15.5:0/2941582464 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c00ccc0 con 0x7f2974102ef0 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.563+0000 7f296bfff640 1 -- 172.21.15.5:0/2941582464 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296c005020 con 0x7f2974102ef0 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.563+0000 7f296bfff640 1 -- 172.21.15.5:0/2941582464 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c004a00 con 0x7f2974102ef0 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.563+0000 7f2979004640 1 -- 172.21.15.5:0/2941582464 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297419d760 con 0x7f2974102ef0 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.563+0000 7f2979004640 1 -- 172.21.15.5:0/2941582464 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741a5040 con 0x7f2974102ef0 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.564+0000 7f2979004640 1 -- 172.21.15.5:0/2941582464 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2974101200 con 0x7f2974102ef0 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.565+0000 7f296bfff640 1 -- 172.21.15.5:0/2941582464 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f296c004ba0 con 0x7f2974102ef0 2024-09-11T22:26:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.566+0000 7f296bfff640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29600a2cf0 0x7f29600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.566+0000 7f296bfff640 1 -- 172.21.15.5:0/2941582464 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f296c0cf520 con 0x7f2974102ef0 2024-09-11T22:26:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.566+0000 7f2972d9d640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29600a2cf0 0x7f29600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.566+0000 7f2972d9d640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29600a2cf0 0x7f29600a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2964009a60 tx=0x7f2964005c50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.570+0000 7f296bfff640 1 -- 172.21.15.5:0/2941582464 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f296c09c0a0 con 0x7f2974102ef0 2024-09-11T22:26:41.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:41 smithi005 ceph-mon[30943]: pgmap v138: 97 pgs: 97 active+clean; 62 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.6 MiB/s wr, 137 op/s 2024-09-11T22:26:41.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:41 smithi159 ceph-mon[35534]: pgmap v138: 97 pgs: 97 active+clean; 62 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.6 MiB/s wr, 137 op/s 2024-09-11T22:26:41.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.831+0000 7f2979004640 1 -- 172.21.15.5:0/2941582464 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f297410a0a0 con 0x7f2974102ef0 2024-09-11T22:26:41.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.832+0000 7f296bfff640 1 -- 172.21.15.5:0/2941582464 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11967 (secure 0 0 0) 0x7f296c09b9a0 con 0x7f2974102ef0 2024-09-11T22:26:41.833 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:26:41.833 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-11T22:25:49:149525+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:25:48.784660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25970":{"gid":25970,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4169170564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4169170564},{"type":"v1","addr":"172.21.15.159:6833","nonce":4169170564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:26:41.833 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-11T22:26:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 -- 172.21.15.5:0/2941582464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29600a2cf0 msgr2=0x7f29600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29600a2cf0 0x7f29600a51b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2964009a60 tx=0x7f2964005c50 comp rx=0 tx=0).stop 2024-09-11T22:26:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 -- 172.21.15.5:0/2941582464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2974102ef0 msgr2=0x7f297419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2974102ef0 0x7f297419cd90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f296c00e970 tx=0x7f296c00ee30 comp rx=0 tx=0).stop 2024-09-11T22:26:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 -- 172.21.15.5:0/2941582464 shutdown_connections 2024-09-11T22:26:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2974102ef0 0x7f297419cd90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f29600a2cf0 0x7f29600a51b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2974102570 0x7f2974198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 --2- 172.21.15.5:0/2941582464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2974100e20 0x7f29741984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 -- 172.21.15.5:0/2941582464 >> 172.21.15.5:0/2941582464 conn(0x7f29740fc7d0 msgr2=0x7f297406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 -- 172.21.15.5:0/2941582464 shutdown_connections 2024-09-11T22:26:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:41.835+0000 7f2979004640 1 -- 172.21.15.5:0/2941582464 wait complete. 2024-09-11T22:26:41.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:41 smithi172 ceph-mon[35461]: pgmap v138: 97 pgs: 97 active+clean; 62 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.6 MiB/s wr, 137 op/s 2024-09-11T22:26:42.316 INFO:tasks.workunit.client.0.smithi005.stdout: 1 319727 9.44 MB/sec execute 60 sec latency 68.735 ms 2024-09-11T22:26:42.687 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-11T22:26:42.687 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-11T22:26:42.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:42 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2941582464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:42.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:42 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2941582464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:42.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:42 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2941582464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:26:42.994 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:26:43.316 INFO:tasks.workunit.client.0.smithi005.stdout: 1 322784 9.57 MB/sec execute 61 sec latency 13.169 ms 2024-09-11T22:26:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.627+0000 7f8177ed2640 1 -- 172.21.15.5:0/2962535112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170102d90 msgr2=0x7f8170103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.627+0000 7f8177ed2640 1 --2- 172.21.15.5:0/2962535112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170102d90 0x7f8170103170 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f816c009930 tx=0x7f816c02a420 comp rx=0 tx=0).stop 2024-09-11T22:26:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.627+0000 7f8177ed2640 1 -- 172.21.15.5:0/2962535112 shutdown_connections 2024-09-11T22:26:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.627+0000 7f8177ed2640 1 --2- 172.21.15.5:0/2962535112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8170108290 0x7f817010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.627+0000 7f8177ed2640 1 --2- 172.21.15.5:0/2962535112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8170103740 0x7f8170107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.627+0000 7f8177ed2640 1 --2- 172.21.15.5:0/2962535112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170102d90 0x7f8170103170 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.627+0000 7f8177ed2640 1 -- 172.21.15.5:0/2962535112 >> 172.21.15.5:0/2962535112 conn(0x7f81700fc7d0 msgr2=0x7f81700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.627+0000 7f8177ed2640 1 -- 172.21.15.5:0/2962535112 shutdown_connections 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.627+0000 7f8177ed2640 1 -- 172.21.15.5:0/2962535112 wait complete. 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8177ed2640 1 Processor -- start 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8177ed2640 1 -- start start 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8177ed2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170102d90 0x7f81701a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8177ed2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8170103740 0x7f81701a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8177ed2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8170108290 0x7f817019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8177ed2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8170112090 con 0x7f8170102d90 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8177ed2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8170111f10 con 0x7f8170103740 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8177ed2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8170112210 con 0x7f8170108290 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8175c6e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170102d90 0x7f81701a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8175c6e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170102d90 0x7f81701a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44506/0 (socket says 172.21.15.5:44506) 2024-09-11T22:26:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f8175c6e640 1 -- 172.21.15.5:0/1181208094 learned_addr learned my addr 172.21.15.5:0/1181208094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f817546d640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8170103740 0x7f81701a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.628+0000 7f817646f640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8170108290 0x7f817019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8175c6e640 1 -- 172.21.15.5:0/1181208094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8170108290 msgr2=0x7f817019a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8175c6e640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8170108290 0x7f817019a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8175c6e640 1 -- 172.21.15.5:0/1181208094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8170103740 msgr2=0x7f81701a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8175c6e640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8170103740 0x7f81701a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8175c6e640 1 -- 172.21.15.5:0/1181208094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817019ad50 con 0x7f8170102d90 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f817646f640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8170108290 0x7f817019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8175c6e640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170102d90 0x7f81701a0460 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f816c009230 tx=0x7f816c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8166ffd640 1 -- 172.21.15.5:0/1181208094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c03f030 con 0x7f8170102d90 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8177ed2640 1 -- 172.21.15.5:0/1181208094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817019afe0 con 0x7f8170102d90 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8166ffd640 1 -- 172.21.15.5:0/1181208094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816c003ad0 con 0x7f8170102d90 2024-09-11T22:26:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8166ffd640 1 -- 172.21.15.5:0/1181208094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c035070 con 0x7f8170102d90 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.629+0000 7f8177ed2640 1 -- 172.21.15.5:0/1181208094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81701a7330 con 0x7f8170102d90 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.631+0000 7f8166ffd640 1 -- 172.21.15.5:0/1181208094 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f816c042030 con 0x7f8170102d90 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817a2ab740 1 Processor -- start 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817a2ab740 1 -- start start 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c1170 0x5589c36cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 0x5589c36cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817a2ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5589c3625160 con 0x5589c36cba30 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817a2ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5589c35cc720 con 0x5589c36c0850 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817a2ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5589c3621120 con 0x5589c36c1170 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817646f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817646f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52432/0 (socket says 172.21.15.5:52432) 2024-09-11T22:26:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817646f640 1 -- 172.21.15.5:0/223392454 learned_addr learned my addr 172.21.15.5:0/223392454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f8175c6e640 1 --2- 172.21.15.5:0/223392454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c1170 0x5589c36cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f817546d640 1 --2- 172.21.15.5:0/223392454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 0x5589c36cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.632+0000 7f8166ffd640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f815c0a2d70 0x7f815c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f817646f640 1 -- 172.21.15.5:0/223392454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c1170 msgr2=0x5589c36cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f817646f640 1 --2- 172.21.15.5:0/223392454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c1170 0x5589c36cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f817646f640 1 -- 172.21.15.5:0/223392454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 msgr2=0x5589c36cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f817646f640 1 --2- 172.21.15.5:0/223392454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 0x5589c36cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f817646f640 1 -- 172.21.15.5:0/223392454 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589c3396ba0 con 0x5589c36c0850 2024-09-11T22:26:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f8166ffd640 1 -- 172.21.15.5:0/1181208094 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f816c0efa70 con 0x7f8170102d90 2024-09-11T22:26:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f817646f640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f815c0a2d70 0x7f815c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f817646f640 1 --2- 172.21.15.5:0/223392454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c0c30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8168010420 tx=0x7f81680108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bb32c1c915fb094 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f815bfff640 1 -- 172.21.15.5:0/223392454 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168023070 con 0x5589c36c0850 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f815bfff640 1 -- 172.21.15.5:0/223392454 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816801c470 con 0x5589c36c0850 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f817646f640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f815c0a2d70 0x7f815c0a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8168020000 tx=0x7f816801a000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.633+0000 7f815bfff640 1 -- 172.21.15.5:0/223392454 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816801d680 con 0x5589c36c0850 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 -- 172.21.15.5:0/223392454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 msgr2=0x5589c36c0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 --2- 172.21.15.5:0/223392454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c0c30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8168010420 tx=0x7f81680108e0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 -- 172.21.15.5:0/223392454 shutdown_connections 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 --2- 172.21.15.5:0/223392454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 0x5589c36cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 --2- 172.21.15.5:0/223392454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c1170 0x5589c36cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 --2- 172.21.15.5:0/223392454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c0c30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 -- 172.21.15.5:0/223392454 >> 172.21.15.5:0/223392454 conn(0x5589c36bcde0 msgr2=0x5589c36bd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 -- 172.21.15.5:0/223392454 shutdown_connections 2024-09-11T22:26:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 -- 172.21.15.5:0/223392454 wait complete. 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 -- start start 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.634+0000 7f817a2ab740 1 Processor -- start 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c0850 0x5589c36c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c1170 0x5589c36c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36cba30 0x5589c36c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f817a2ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5589c36ce600 con 0x5589c36c0850 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f817a2ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5589c35ad900 con 0x5589c36c1170 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f817a2ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5589c36ce780 con 0x5589c36cba30 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f817a2ab740 10 client.0 _ll_register_callbacks cb 0x5589c35ab780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f8175c6e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c0850 0x5589c36c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f8175c6e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c0850 0x5589c36c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44536/0 (socket says 172.21.15.5:44536) 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f8175c6e640 1 -- 172.21.15.5:0/1389372975 learned_addr learned my addr 172.21.15.5:0/1389372975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f817646f640 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36cba30 0x5589c36c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f817546d640 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c1170 0x5589c36c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f8175c6e640 1 -- 172.21.15.5:0/1389372975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36cba30 msgr2=0x5589c36c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f8175c6e640 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36cba30 0x5589c36c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f8175c6e640 1 -- 172.21.15.5:0/1389372975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c1170 msgr2=0x5589c36c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f8175c6e640 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c1170 0x5589c36c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.635+0000 7f8175c6e640 1 -- 172.21.15.5:0/1389372975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589c35edc50 con 0x5589c36c0850 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f8175c6e640 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c0850 0x5589c36c8950 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f816c0ed1e0 tx=0x7f816c0dc130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62123ab5ad5185ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f81597fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f81597fa640 1 -- 172.21.15.5:0/1389372975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c049070 con 0x5589c36c0850 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f81597fa640 1 -- 172.21.15.5:0/1389372975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f816c003c80 con 0x5589c36c0850 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f817a2ab740 1 -- 172.21.15.5:0/1389372975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f816801c470 con 0x5589c36c0850 2024-09-11T22:26:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f81597fa640 1 -- 172.21.15.5:0/1389372975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f816c0bb8b0 con 0x5589c36c0850 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f817a2ab740 10 client.16836 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f817a2ab740 1 -- 172.21.15.5:0/1389372975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5589c36da2e0 con 0x5589c36c0850 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f81597fa640 1 -- 172.21.15.5:0/1389372975 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816801c470 con 0x5589c36c0850 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f81597fa640 1 -- 172.21.15.5:0/1389372975 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f816c0bf050 con 0x5589c36c0850 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f817a2ab740 10 client.16836 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f817a2ab740 10 client.16836 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f817a2ab740 20 client.16836 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5589c36db6d0 0x5589c36ddae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f817a2ab740 4 client.16836 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.636+0000 7f817a2ab740 1 -- 172.21.15.5:0/1389372975 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589c3632890 con 0x5589c36db6d0 2024-09-11T22:26:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.637+0000 7f817546d640 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5589c36db6d0 0x5589c36ddae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.637+0000 7f817546d640 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5589c36db6d0 0x5589c36ddae0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49eb4469c205156a server_cookie=5a944f5f137df1f8 in_seq=0 out_seq=0 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.637+0000 7f81597fa640 10 client.16836 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f81597fa640 1 -- 172.21.15.5:0/1389372975 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589c3632890 con 0x5589c36db6d0 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f81597fa640 10 client.16836 handle_command_reply: tid=0 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 client.16836 shutdown 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 -- 172.21.15.5:0/1389372975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c0850 msgr2=0x5589c36c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c0850 0x5589c36c8950 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f816c0ed1e0 tx=0x7f816c0dc130 comp rx=0 tx=0).stop 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 -- 172.21.15.5:0/1389372975 shutdown_connections 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5589c36db6d0 0x5589c36ddae0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36cba30 0x5589c36c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c1170 0x5589c36c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f81597fa640 0 client.16836 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1389372975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c0850 0x5589c36c8950 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 -- 172.21.15.5:0/1389372975 >> 172.21.15.5:0/1389372975 conn(0x5589c36bcde0 msgr2=0x5589c36c2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 -- 172.21.15.5:0/1389372975 shutdown_connections 2024-09-11T22:26:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.638+0000 7f817a2ab740 1 -- 172.21.15.5:0/1389372975 wait complete. 2024-09-11T22:26:43.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.639+0000 7f817a2ab740 20 client.16836 trim_cache size 0 max 16384 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817a2ab740 1 Processor -- start 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817a2ab740 1 -- start start 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c1170 0x5589c36c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 0x5589c36e33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817a2ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81701afbd0 con 0x5589c36cba30 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817a2ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81701b1710 con 0x5589c36c0850 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817a2ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81701b02a0 con 0x5589c36c1170 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817646f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 0x5589c36e33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817646f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 0x5589c36e33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44552/0 (socket says 172.21.15.5:44552) 2024-09-11T22:26:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817646f640 1 -- 172.21.15.5:0/3534586793 learned_addr learned my addr 172.21.15.5:0/3534586793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f8175c6e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.656+0000 7f817546d640 1 --2- 172.21.15.5:0/3534586793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c1170 0x5589c36c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.657+0000 7f817646f640 1 -- 172.21.15.5:0/3534586793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c1170 msgr2=0x5589c36c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.657+0000 7f817646f640 1 --2- 172.21.15.5:0/3534586793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c1170 0x5589c36c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.657+0000 7f817646f640 1 -- 172.21.15.5:0/3534586793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 msgr2=0x5589c36c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.657+0000 7f817646f640 1 --2- 172.21.15.5:0/3534586793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.657+0000 7f817646f640 1 -- 172.21.15.5:0/3534586793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589c36bcc10 con 0x5589c36cba30 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.657+0000 7f8175c6e640 1 --2- 172.21.15.5:0/3534586793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.657+0000 7f817646f640 1 --2- 172.21.15.5:0/3534586793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 0x5589c36e33f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f816801ce80 tx=0x7f81680096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acc56760f501fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.657+0000 7f815b7fe640 1 -- 172.21.15.5:0/3534586793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8168023050 con 0x5589c36cba30 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.657+0000 7f815b7fe640 1 -- 172.21.15.5:0/3534586793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81680136c0 con 0x5589c36cba30 2024-09-11T22:26:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.657+0000 7f815b7fe640 1 -- 172.21.15.5:0/3534586793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816800c7b0 con 0x5589c36cba30 2024-09-11T22:26:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 -- 172.21.15.5:0/3534586793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 msgr2=0x5589c36e33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 --2- 172.21.15.5:0/3534586793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 0x5589c36e33f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f816801ce80 tx=0x7f81680096c0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 -- 172.21.15.5:0/3534586793 shutdown_connections 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 --2- 172.21.15.5:0/3534586793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36cba30 0x5589c36e33f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 --2- 172.21.15.5:0/3534586793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c1170 0x5589c36c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 --2- 172.21.15.5:0/3534586793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36c0850 0x5589c36c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 -- 172.21.15.5:0/3534586793 >> 172.21.15.5:0/3534586793 conn(0x5589c36bcde0 msgr2=0x5589c36c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 -- 172.21.15.5:0/3534586793 shutdown_connections 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 -- 172.21.15.5:0/3534586793 wait complete. 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 -- start start 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.658+0000 7f817a2ab740 1 Processor -- start 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c0850 0x5589c36d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c1170 0x5589c36d3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817a2ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36cba30 0x5589c36d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817a2ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81701b1040 con 0x5589c36c1170 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817a2ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81701b0970 con 0x5589c36cba30 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817a2ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81701adbb0 con 0x5589c36c0850 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817a2ab740 10 client.0 _ll_register_callbacks cb 0x7f816c0bbe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817546d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c1170 0x5589c36d3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817546d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c1170 0x5589c36d3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44566/0 (socket says 172.21.15.5:44566) 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817546d640 1 -- 172.21.15.5:0/1870906276 learned_addr learned my addr 172.21.15.5:0/1870906276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f8175c6e640 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c0850 0x5589c36d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817646f640 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36cba30 0x5589c36d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817546d640 1 -- 172.21.15.5:0/1870906276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c0850 msgr2=0x5589c36d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817546d640 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c0850 0x5589c36d39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817546d640 1 -- 172.21.15.5:0/1870906276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36cba30 msgr2=0x5589c36d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817546d640 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36cba30 0x5589c36d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817546d640 1 -- 172.21.15.5:0/1870906276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589c36bda70 con 0x5589c36c1170 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f8175c6e640 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c0850 0x5589c36d39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.659+0000 7f817646f640 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36cba30 0x5589c36d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f817546d640 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c1170 0x5589c36d3f10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f817019b760 tx=0x7f8160010a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b63a685c906cd9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f81597fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f81597fa640 1 -- 172.21.15.5:0/1870906276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160022070 con 0x5589c36c1170 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f817a2ab740 1 -- 172.21.15.5:0/1870906276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589c362cbe0 con 0x5589c36c1170 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f81597fa640 1 -- 172.21.15.5:0/1870906276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f816001f9f0 con 0x5589c36c1170 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f81597fa640 1 -- 172.21.15.5:0/1870906276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f816001fbf0 con 0x5589c36c1170 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f817a2ab740 10 client.16842 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f817a2ab740 1 -- 172.21.15.5:0/1870906276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5589c36b83f0 con 0x5589c36c1170 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f81597fa640 1 -- 172.21.15.5:0/1870906276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5589c362cbe0 con 0x5589c36c1170 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f81597fa640 1 -- 172.21.15.5:0/1870906276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f81600265a0 con 0x5589c36c1170 2024-09-11T22:26:43.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f817a2ab740 10 client.16842 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f817a2ab740 10 client.16842 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:26:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f817a2ab740 20 client.16842 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5589c374f6b0 0x5589c3751ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f817a2ab740 4 client.16842 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-11T22:26:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.660+0000 7f817a2ab740 1 -- 172.21.15.5:0/1870906276 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x5589c35a8f40 con 0x5589c374f6b0 2024-09-11T22:26:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.661+0000 7f8175c6e640 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5589c374f6b0 0x5589c3751ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.661+0000 7f8175c6e640 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5589c374f6b0 0x5589c3751ad0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d3b86cc8936b79 server_cookie=4fd4ab19e7091c00 in_seq=0 out_seq=0 2024-09-11T22:26:43.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.661+0000 7f81597fa640 10 client.16842 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.776+0000 7f81597fa640 1 -- 172.21.15.5:0/1870906276 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5589c35a8f40 con 0x5589c374f6b0 2024-09-11T22:26:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.776+0000 7f81597fa640 10 client.16842 handle_command_reply: tid=0 2024-09-11T22:26:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.776+0000 7f817a2ab740 1 client.16842 shutdown 2024-09-11T22:26:43.778 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:43 smithi005 ceph-mon[30943]: pgmap v139: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 243 op/s 2024-09-11T22:26:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.776+0000 7f817a2ab740 1 -- 172.21.15.5:0/1870906276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c1170 msgr2=0x5589c36d3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.776+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c1170 0x5589c36d3f10 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f817019b760 tx=0x7f8160010a20 comp rx=0 tx=0).stop 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f817a2ab740 1 -- 172.21.15.5:0/1870906276 shutdown_connections 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5589c374f6b0 0x5589c3751ad0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589c36cba30 0x5589c36d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589c36c1170 0x5589c36d3f10 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f81597fa640 0 client.16842 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f817a2ab740 1 --2- 172.21.15.5:0/1870906276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589c36c0850 0x5589c36d39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f817a2ab740 1 -- 172.21.15.5:0/1870906276 >> 172.21.15.5:0/1870906276 conn(0x5589c36bcde0 msgr2=0x5589c36c8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f817a2ab740 1 -- 172.21.15.5:0/1870906276 shutdown_connections 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f817a2ab740 1 -- 172.21.15.5:0/1870906276 wait complete. 2024-09-11T22:26:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f817a2ab740 20 client.16842 trim_cache size 0 max 16384 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 34, 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 18, 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout: "5e46c553": { 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 18, 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:26:43.780 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.2, 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:43.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:26:43.782 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:26:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f8177ed2640 1 -- 172.21.15.5:0/1181208094 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f815c0a2d70 msgr2=0x7f815c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.777+0000 7f8177ed2640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f815c0a2d70 0x7f815c0a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8168020000 tx=0x7f816801a000 comp rx=0 tx=0).stop 2024-09-11T22:26:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.778+0000 7f8177ed2640 1 -- 172.21.15.5:0/1181208094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170102d90 msgr2=0x7f81701a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.778+0000 7f8177ed2640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170102d90 0x7f81701a0460 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f816c009230 tx=0x7f816c004270 comp rx=0 tx=0).stop 2024-09-11T22:26:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.778+0000 7f8177ed2640 1 -- 172.21.15.5:0/1181208094 shutdown_connections 2024-09-11T22:26:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.778+0000 7f8177ed2640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8170108290 0x7f817019a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.778+0000 7f8177ed2640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f815c0a2d70 0x7f815c0a5230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.778+0000 7f8177ed2640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8170103740 0x7f81701a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.778+0000 7f8177ed2640 1 --2- 172.21.15.5:0/1181208094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170102d90 0x7f81701a0460 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.778+0000 7f8177ed2640 1 -- 172.21.15.5:0/1181208094 >> 172.21.15.5:0/1181208094 conn(0x7f81700fc7d0 msgr2=0x7f8170100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.778+0000 7f8177ed2640 1 -- 172.21.15.5:0/1181208094 shutdown_connections 2024-09-11T22:26:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:26:43.778+0000 7f8177ed2640 1 -- 172.21.15.5:0/1181208094 wait complete. 2024-09-11T22:26:43.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:43 smithi159 ceph-mon[35534]: pgmap v139: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 243 op/s 2024-09-11T22:26:43.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:43 smithi172 ceph-mon[35461]: pgmap v139: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 243 op/s 2024-09-11T22:26:44.316 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 9.45 MB/sec execute 62 sec latency 545.993 ms 2024-09-11T22:26:44.743 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5e46c553', quiesce took 4.1 seconds. Will release after: 16.049999999999997 2024-09-11T22:26:45.316 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 9.30 MB/sec execute 63 sec latency 1546.131 ms 2024-09-11T22:26:45.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:45 smithi005 ceph-mon[30943]: pgmap v140: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-11T22:26:45.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:45 smithi159 ceph-mon[35534]: pgmap v140: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-11T22:26:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:45 smithi172 ceph-mon[35461]: pgmap v140: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-11T22:26:46.316 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 9.15 MB/sec execute 64 sec latency 2546.233 ms 2024-09-11T22:26:47.316 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 9.01 MB/sec execute 65 sec latency 3546.337 ms 2024-09-11T22:26:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:47 smithi005 ceph-mon[30943]: pgmap v141: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-11T22:26:47.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:47 smithi159 ceph-mon[35534]: pgmap v141: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-11T22:26:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:47 smithi172 ceph-mon[35461]: pgmap v141: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 142 op/s 2024-09-11T22:26:48.316 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 8.87 MB/sec execute 66 sec latency 4546.444 ms 2024-09-11T22:26:49.316 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 8.74 MB/sec execute 67 sec latency 5546.542 ms 2024-09-11T22:26:49.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:49 smithi005 ceph-mon[30943]: pgmap v142: 97 pgs: 97 active+clean; 59 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-09-11T22:26:49.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:49 smithi159 ceph-mon[35534]: pgmap v142: 97 pgs: 97 active+clean; 59 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-09-11T22:26:49.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:49 smithi172 ceph-mon[35461]: pgmap v142: 97 pgs: 97 active+clean; 59 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-09-11T22:26:50.316 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 8.61 MB/sec execute 68 sec latency 6546.666 ms 2024-09-11T22:26:51.317 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 8.49 MB/sec execute 69 sec latency 7546.739 ms 2024-09-11T22:26:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:51 smithi005 ceph-mon[30943]: pgmap v143: 97 pgs: 97 active+clean; 59 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-11T22:26:51.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:51 smithi159 ceph-mon[35534]: pgmap v143: 97 pgs: 97 active+clean; 59 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-11T22:26:51.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:51 smithi172 ceph-mon[35461]: pgmap v143: 97 pgs: 97 active+clean; 59 MiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-11T22:26:52.317 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 8.37 MB/sec execute 70 sec latency 8546.830 ms 2024-09-11T22:26:53.317 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 8.25 MB/sec execute 71 sec latency 9546.897 ms 2024-09-11T22:26:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.336+0000 7f90d9768640 1 -- 172.21.15.5:0/1349722716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d4101350 msgr2=0x7f90d410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.336+0000 7f90d9768640 1 --2- 172.21.15.5:0/1349722716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d4101350 0x7f90d410d880 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009930 tx=0x7f90cc02a420 comp rx=0 tx=0).stop 2024-09-11T22:26:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.336+0000 7f90d9768640 1 -- 172.21.15.5:0/1349722716 shutdown_connections 2024-09-11T22:26:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.336+0000 7f90d9768640 1 --2- 172.21.15.5:0/1349722716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d4101350 0x7f90d410d880 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.336+0000 7f90d9768640 1 --2- 172.21.15.5:0/1349722716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d41009b0 0x7f90d4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.336+0000 7f90d9768640 1 --2- 172.21.15.5:0/1349722716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d41069c0 0x7f90d4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.336+0000 7f90d9768640 1 -- 172.21.15.5:0/1349722716 >> 172.21.15.5:0/1349722716 conn(0x7f90d4077990 msgr2=0x7f90d40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.336+0000 7f90d9768640 1 -- 172.21.15.5:0/1349722716 shutdown_connections 2024-09-11T22:26:53.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.336+0000 7f90d9768640 1 -- 172.21.15.5:0/1349722716 wait complete. 2024-09-11T22:26:53.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.336+0000 7f90d9768640 1 Processor -- start 2024-09-11T22:26:53.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d9768640 1 -- start start 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d9768640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d41009b0 0x7f90d4102580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d9768640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d4101350 0x7f90d4102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d9768640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d41069c0 0x7f90d4103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d9768640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4110e00 con 0x7f90d41009b0 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d9768640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4110c80 con 0x7f90d4101350 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d9768640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90d4110f80 con 0x7f90d41069c0 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d41009b0 0x7f90d4102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d41009b0 0x7f90d4102580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43700/0 (socket says 172.21.15.5:43700) 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2402305825 learned_addr learned my addr 172.21.15.5:0/2402305825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d27fc640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d4101350 0x7f90d4102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.337+0000 7f90d37fe640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d41069c0 0x7f90d4103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2402305825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d41069c0 msgr2=0x7f90d4103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d41069c0 0x7f90d4103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2402305825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d4101350 msgr2=0x7f90d4102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d4101350 0x7f90d4102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2402305825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d41a67e0 con 0x7f90d41009b0 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d41009b0 0x7f90d4102580 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f90c400eab0 tx=0x7f90c400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90bbfff640 1 -- 172.21.15.5:0/2402305825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c400ce40 con 0x7f90d41009b0 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90bbfff640 1 -- 172.21.15.5:0/2402305825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c4011950 con 0x7f90d41009b0 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90d9768640 1 -- 172.21.15.5:0/2402305825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d41a6ad0 con 0x7f90d41009b0 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90bbfff640 1 -- 172.21.15.5:0/2402305825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4010ae0 con 0x7f90d41009b0 2024-09-11T22:26:53.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.338+0000 7f90d9768640 1 -- 172.21.15.5:0/2402305825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d41a7020 con 0x7f90d41009b0 2024-09-11T22:26:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.340+0000 7f90bbfff640 1 -- 172.21.15.5:0/2402305825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f90c4011b00 con 0x7f90d41009b0 2024-09-11T22:26:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90dc5da740 1 Processor -- start 2024-09-11T22:26:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90dc5da740 1 -- start start 2024-09-11T22:26:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 0x55a85bb65ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb70cb0 0x55a85bb730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90dc5da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a85baca2f0 con 0x55a85bb66410 2024-09-11T22:26:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90dc5da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a85ba72270 con 0x55a85bb70cb0 2024-09-11T22:26:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90dc5da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a85bac62b0 con 0x55a85bb65af0 2024-09-11T22:26:53.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90bbfff640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90bc0a2da0 0x7f90bc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb70770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43716/0 (socket says 172.21.15.5:43716) 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2041974576 learned_addr learned my addr 172.21.15.5:0/2041974576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90d27fc640 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb70cb0 0x55a85bb730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90d37fe640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90bc0a2da0 0x7f90bc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90d37fe640 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 0x55a85bb65ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.341+0000 7f90bbfff640 1 -- 172.21.15.5:0/2402305825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f90c40cfb10 con 0x7f90d41009b0 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2041974576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 msgr2=0x55a85bb65ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 0x55a85bb65ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2041974576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb70cb0 msgr2=0x55a85bb730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb70cb0 0x55a85bb730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2041974576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a85ba71340 con 0x55a85bb66410 2024-09-11T22:26:53.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90d37fe640 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 0x55a85bb65ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90d27fc640 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb70cb0 0x55a85bb730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb70770 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f90c405cc70 tx=0x7f90c4010540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f5eabdd2387dce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90d37fe640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90bc0a2da0 0x7f90bc0a5260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90cc005940 tx=0x7f90cc0058b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90b97fa640 1 -- 172.21.15.5:0/2041974576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c409c6f0 con 0x55a85bb66410 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90b97fa640 1 -- 172.21.15.5:0/2041974576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c409bff0 con 0x55a85bb66410 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.342+0000 7f90b97fa640 1 -- 172.21.15.5:0/2041974576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40b00a0 con 0x55a85bb66410 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 -- 172.21.15.5:0/2041974576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 msgr2=0x55a85bb70770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb70770 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f90c405cc70 tx=0x7f90c4010540 comp rx=0 tx=0).stop 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 -- 172.21.15.5:0/2041974576 shutdown_connections 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb70cb0 0x55a85bb730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb70770 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 --2- 172.21.15.5:0/2041974576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 0x55a85bb65ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 -- 172.21.15.5:0/2041974576 >> 172.21.15.5:0/2041974576 conn(0x55a85bb62060 msgr2=0x55a85bb62440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 -- 172.21.15.5:0/2041974576 shutdown_connections 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 -- 172.21.15.5:0/2041974576 wait complete. 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 -- start start 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.343+0000 7f90dc5da740 1 Processor -- start 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb65af0 0x55a85bb6dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb6e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb70cb0 0x55a85bb6e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90dc5da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a85bb73880 con 0x55a85bb66410 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90dc5da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a85ba53290 con 0x55a85bb65af0 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90dc5da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a85bb73a00 con 0x55a85bb70cb0 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90dc5da740 10 client.0 _ll_register_callbacks cb 0x55a85ba84140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb6e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb6e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43722/0 (socket says 172.21.15.5:43722) 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d27fc640 1 -- 172.21.15.5:0/4034938017 learned_addr learned my addr 172.21.15.5:0/4034938017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d37fe640 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb70cb0 0x55a85bb6e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb65af0 0x55a85bb6dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d27fc640 1 -- 172.21.15.5:0/4034938017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb70cb0 msgr2=0x55a85bb6e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d27fc640 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb70cb0 0x55a85bb6e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d27fc640 1 -- 172.21.15.5:0/4034938017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb65af0 msgr2=0x55a85bb6dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d27fc640 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb65af0 0x55a85bb6dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d27fc640 1 -- 172.21.15.5:0/4034938017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a85b7dd210 con 0x55a85bb66410 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90d27fc640 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb6e100 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f90c800bad0 tx=0x7f90c800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c48d928d448f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.344+0000 7f90b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90b6ff5640 1 -- 172.21.15.5:0/4034938017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8003e20 con 0x55a85bb66410 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90b6ff5640 1 -- 172.21.15.5:0/4034938017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f90c8011440 con 0x55a85bb66410 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90dc5da740 1 -- 172.21.15.5:0/4034938017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c409bff0 con 0x55a85bb66410 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90b6ff5640 1 -- 172.21.15.5:0/4034938017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90c8011640 con 0x55a85bb66410 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90dc5da740 10 client.16854 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90dc5da740 1 -- 172.21.15.5:0/4034938017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a85b8a9620 con 0x55a85bb66410 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90b6ff5640 1 -- 172.21.15.5:0/4034938017 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c409bff0 con 0x55a85bb66410 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90b6ff5640 1 -- 172.21.15.5:0/4034938017 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f90c8021030 con 0x55a85bb66410 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90dc5da740 10 client.16854 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90dc5da740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90dc5da740 20 client.16854 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90dc5da740 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a85bb80cb0 0x55a85bb83100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90dc5da740 4 client.16854 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.345+0000 7f90dc5da740 1 -- 172.21.15.5:0/4034938017 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a85bad7a20 con 0x55a85bb80cb0 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.346+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a85bb80cb0 0x55a85bb83100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.346+0000 7f90b6ff5640 10 client.16854 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.346+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a85bb80cb0 0x55a85bb83100 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ccc8dcabfb1a5f0 server_cookie=db91e8e4225db144 in_seq=0 out_seq=0 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90b6ff5640 1 -- 172.21.15.5:0/4034938017 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a85bad7a20 con 0x55a85bb80cb0 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90b6ff5640 10 client.16854 handle_command_reply: tid=0 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 client.16854 shutdown 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 -- 172.21.15.5:0/4034938017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 msgr2=0x55a85bb6e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb6e100 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f90c800bad0 tx=0x7f90c800bf90 comp rx=0 tx=0).stop 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 -- 172.21.15.5:0/4034938017 shutdown_connections 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a85bb80cb0 0x55a85bb83100 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb70cb0 0x55a85bb6e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90b6ff5640 0 client.16854 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb66410 0x55a85bb6e100 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 --2- 172.21.15.5:0/4034938017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb65af0 0x55a85bb6dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 -- 172.21.15.5:0/4034938017 >> 172.21.15.5:0/4034938017 conn(0x55a85bb62060 msgr2=0x55a85bb67d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 -- 172.21.15.5:0/4034938017 shutdown_connections 2024-09-11T22:26:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 1 -- 172.21.15.5:0/4034938017 wait complete. 2024-09-11T22:26:53.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.347+0000 7f90dc5da740 20 client.16854 trim_cache size 0 max 16384 2024-09-11T22:26:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.361+0000 7f90dc5da740 1 Processor -- start 2024-09-11T22:26:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90dc5da740 1 -- start start 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb65af0 0x55a85bb6e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb66410 0x55a85bb88a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb70cb0 0x55a85bb88f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90dc5da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41afb30 con 0x55a85bb65af0 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90dc5da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41b1670 con 0x55a85bb70cb0 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90dc5da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41b0200 con 0x55a85bb66410 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb65af0 0x55a85bb6e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb65af0 0x55a85bb6e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43732/0 (socket says 172.21.15.5:43732) 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2898785468 learned_addr learned my addr 172.21.15.5:0/2898785468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d27fc640 1 --2- 172.21.15.5:0/2898785468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb66410 0x55a85bb88a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2898785468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb66410 msgr2=0x55a85bb88a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/2898785468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb66410 0x55a85bb88a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2898785468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb70cb0 msgr2=0x55a85bb88f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/2898785468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb70cb0 0x55a85bb88f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d2ffd640 1 -- 172.21.15.5:0/2898785468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a85bb65920 con 0x55a85bb65af0 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d27fc640 1 --2- 172.21.15.5:0/2898785468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb66410 0x55a85bb88a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/2898785468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb65af0 0x55a85bb6e470 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f90c40cd820 tx=0x7f90c4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b435745b7372c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.362+0000 7f90b8ff9640 1 -- 172.21.15.5:0/2898785468 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c409c6f0 con 0x55a85bb65af0 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90b8ff9640 1 -- 172.21.15.5:0/2898785468 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c40b4070 con 0x55a85bb65af0 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90b8ff9640 1 -- 172.21.15.5:0/2898785468 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c409c490 con 0x55a85bb65af0 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 -- 172.21.15.5:0/2898785468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb65af0 msgr2=0x55a85bb6e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 --2- 172.21.15.5:0/2898785468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb65af0 0x55a85bb6e470 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f90c40cd820 tx=0x7f90c4000f40 comp rx=0 tx=0).stop 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 -- 172.21.15.5:0/2898785468 shutdown_connections 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 --2- 172.21.15.5:0/2898785468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb70cb0 0x55a85bb88f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 --2- 172.21.15.5:0/2898785468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb66410 0x55a85bb88a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 --2- 172.21.15.5:0/2898785468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb65af0 0x55a85bb6e470 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 -- 172.21.15.5:0/2898785468 >> 172.21.15.5:0/2898785468 conn(0x55a85bb62060 msgr2=0x55a85bb6fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 -- 172.21.15.5:0/2898785468 shutdown_connections 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 -- 172.21.15.5:0/2898785468 wait complete. 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 -- start start 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.363+0000 7f90dc5da740 1 Processor -- start 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 0x55a85bb74940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb66410 0x55a85bb74e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90dc5da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb70cb0 0x55a85bb753c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90dc5da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41b0fa0 con 0x55a85bb70cb0 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90dc5da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41b08d0 con 0x55a85bb66410 2024-09-11T22:26:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90dc5da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90d41adb10 con 0x55a85bb65af0 2024-09-11T22:26:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90dc5da740 10 client.0 _ll_register_callbacks cb 0x7f90c8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:26:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb70cb0 0x55a85bb753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb70cb0 0x55a85bb753c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43734/0 (socket says 172.21.15.5:43734) 2024-09-11T22:26:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d37fe640 1 -- 172.21.15.5:0/452607174 learned_addr learned my addr 172.21.15.5:0/452607174 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:26:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d27fc640 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb66410 0x55a85bb74e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 0x55a85bb74940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d37fe640 1 -- 172.21.15.5:0/452607174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 msgr2=0x55a85bb74940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d37fe640 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 0x55a85bb74940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d37fe640 1 -- 172.21.15.5:0/452607174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb66410 msgr2=0x55a85bb74e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:26:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d37fe640 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb66410 0x55a85bb74e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d37fe640 1 -- 172.21.15.5:0/452607174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a85bbea9e0 con 0x55a85bb70cb0 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 0x55a85bb74940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90d37fe640 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb70cb0 0x55a85bb753c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f90cc033000 tx=0x7f90cc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf9506d943d0d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.364+0000 7f90b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90b6ff5640 1 -- 172.21.15.5:0/452607174 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc035040 con 0x55a85bb70cb0 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90b6ff5640 1 -- 172.21.15.5:0/452607174 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f90cc005240 con 0x55a85bb70cb0 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90dc5da740 1 -- 172.21.15.5:0/452607174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c40b4070 con 0x55a85bb70cb0 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90b6ff5640 1 -- 172.21.15.5:0/452607174 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90cc005440 con 0x55a85bb70cb0 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90dc5da740 10 client.16860 fetch_fsmap learned FSMap version 34 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90dc5da740 1 -- 172.21.15.5:0/452607174 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a85bbebb80 con 0x55a85bb70cb0 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90b6ff5640 1 -- 172.21.15.5:0/452607174 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c40b4070 con 0x55a85bb70cb0 2024-09-11T22:26:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90b6ff5640 1 -- 172.21.15.5:0/452607174 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f90cc059070 con 0x55a85bb70cb0 2024-09-11T22:26:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90dc5da740 10 client.16860 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:26:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90dc5da740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:26:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90dc5da740 20 client.16860 populate_metadata read hostname 'smithi005' 2024-09-11T22:26:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90dc5da740 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a85bbec350 0x55a85bbee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:26:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90dc5da740 4 client.16860 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:26:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.365+0000 7f90dc5da740 1 -- 172.21.15.5:0/452607174 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a85bad1d70 con 0x55a85bbec350 2024-09-11T22:26:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.366+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a85bbec350 0x55a85bbee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:26:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.366+0000 7f90b6ff5640 10 client.16860 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:53.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.366+0000 7f90d2ffd640 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a85bbec350 0x55a85bbee770 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ab8c3a0e5981c0 server_cookie=dbfe51b23b55673f in_seq=0 out_seq=0 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90b6ff5640 1 -- 172.21.15.5:0/452607174 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a85bad1d70 con 0x55a85bbec350 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90b6ff5640 10 client.16860 handle_command_reply: tid=0 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 client.16860 shutdown 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 -- 172.21.15.5:0/452607174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb70cb0 msgr2=0x55a85bb753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb70cb0 0x55a85bb753c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f90cc033000 tx=0x7f90cc005d50 comp rx=0 tx=0).stop 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 -- 172.21.15.5:0/452607174 shutdown_connections 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a85bbec350 0x55a85bbee770 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a85bb70cb0 0x55a85bb753c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90b6ff5640 0 client.16860 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a85bb66410 0x55a85bb74e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 --2- 172.21.15.5:0/452607174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a85bb65af0 0x55a85bb74940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 -- 172.21.15.5:0/452607174 >> 172.21.15.5:0/452607174 conn(0x55a85bb62060 msgr2=0x55a85bb609a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 -- 172.21.15.5:0/452607174 shutdown_connections 2024-09-11T22:26:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 1 -- 172.21.15.5:0/452607174 wait complete. 2024-09-11T22:26:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90dc5da740 20 client.16860 trim_cache size 0 max 16384 2024-09-11T22:26:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:26:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:26:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:26:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:26:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90d9768640 1 -- 172.21.15.5:0/2402305825 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90bc0a2da0 msgr2=0x7f90bc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90d9768640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90bc0a2da0 0x7f90bc0a5260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90cc005940 tx=0x7f90cc0058b0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90d9768640 1 -- 172.21.15.5:0/2402305825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d41009b0 msgr2=0x7f90d4102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:26:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90d9768640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d41009b0 0x7f90d4102580 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f90c400eab0 tx=0x7f90c400ef70 comp rx=0 tx=0).stop 2024-09-11T22:26:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90d9768640 1 -- 172.21.15.5:0/2402305825 shutdown_connections 2024-09-11T22:26:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90d9768640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90d41069c0 0x7f90d4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90d9768640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90bc0a2da0 0x7f90bc0a5260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90d9768640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90d4101350 0x7f90d4102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90d9768640 1 --2- 172.21.15.5:0/2402305825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90d41009b0 0x7f90d4102580 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:26:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.444+0000 7f90d9768640 1 -- 172.21.15.5:0/2402305825 >> 172.21.15.5:0/2402305825 conn(0x7f90d4077990 msgr2=0x7f90d40ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:26:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.445+0000 7f90d9768640 1 -- 172.21.15.5:0/2402305825 shutdown_connections 2024-09-11T22:26:53.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:26:53.445+0000 7f90d9768640 1 -- 172.21.15.5:0/2402305825 wait complete. 2024-09-11T22:26:53.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:53 smithi005 ceph-mon[30943]: pgmap v144: 97 pgs: 97 active+clean; 60 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2024-09-11T22:26:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:53 smithi159 ceph-mon[35534]: pgmap v144: 97 pgs: 97 active+clean; 60 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2024-09-11T22:26:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:53 smithi172 ceph-mon[35461]: pgmap v144: 97 pgs: 97 active+clean; 60 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 206 op/s 2024-09-11T22:26:54.317 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 8.13 MB/sec execute 72 sec latency 10547.005 ms 2024-09-11T22:26:55.317 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 8.02 MB/sec execute 73 sec latency 11547.068 ms 2024-09-11T22:26:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:55 smithi005 ceph-mon[30943]: pgmap v145: 97 pgs: 97 active+clean; 60 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-09-11T22:26:55.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:55 smithi159 ceph-mon[35534]: pgmap v145: 97 pgs: 97 active+clean; 60 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-09-11T22:26:55.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:55 smithi172 ceph-mon[35461]: pgmap v145: 97 pgs: 97 active+clean; 60 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-09-11T22:26:56.317 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 7.91 MB/sec execute 74 sec latency 12547.156 ms 2024-09-11T22:26:57.317 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 7.81 MB/sec execute 75 sec latency 13547.253 ms 2024-09-11T22:26:57.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:57 smithi005 ceph-mon[30943]: pgmap v146: 97 pgs: 97 active+clean; 60 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-09-11T22:26:57.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:57 smithi159 ceph-mon[35534]: pgmap v146: 97 pgs: 97 active+clean; 60 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-09-11T22:26:57.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:57 smithi172 ceph-mon[35461]: pgmap v146: 97 pgs: 97 active+clean; 60 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 1.6 MiB/s wr, 99 op/s 2024-09-11T22:26:58.317 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 7.71 MB/sec execute 76 sec latency 14547.380 ms 2024-09-11T22:26:59.317 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 7.61 MB/sec execute 77 sec latency 15547.505 ms 2024-09-11T22:26:59.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:26:59 smithi005 ceph-mon[30943]: pgmap v147: 97 pgs: 97 active+clean; 49 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-11T22:26:59.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:26:59 smithi159 ceph-mon[35534]: pgmap v147: 97 pgs: 97 active+clean; 49 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-11T22:26:59.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:26:59 smithi172 ceph-mon[35461]: pgmap v147: 97 pgs: 97 active+clean; 49 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-11T22:27:00.317 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 7.51 MB/sec execute 78 sec latency 16547.629 ms 2024-09-11T22:27:00.794 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:27:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:01.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 7.41 MB/sec execute 79 sec latency 17547.717 ms 2024-09-11T22:27:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.583+0000 7f7e84922640 1 -- 172.21.15.5:0/3852851202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e80102ac0 msgr2=0x7f7e801000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.583+0000 7f7e84922640 1 --2- 172.21.15.5:0/3852851202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e80102ac0 0x7f7e801000b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009900 tx=0x7f7e7002a480 comp rx=0 tx=0).stop 2024-09-11T22:27:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.583+0000 7f7e84922640 1 -- 172.21.15.5:0/3852851202 shutdown_connections 2024-09-11T22:27:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.583+0000 7f7e84922640 1 --2- 172.21.15.5:0/3852851202 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80100c10 0x7f7e80110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.583+0000 7f7e84922640 1 --2- 172.21.15.5:0/3852851202 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e80102ea0 0x7f7e801005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.583+0000 7f7e84922640 1 --2- 172.21.15.5:0/3852851202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e80102ac0 0x7f7e801000b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.583+0000 7f7e84922640 1 -- 172.21.15.5:0/3852851202 >> 172.21.15.5:0/3852851202 conn(0x7f7e80077210 msgr2=0x7f7e800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.583+0000 7f7e84922640 1 -- 172.21.15.5:0/3852851202 shutdown_connections 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.583+0000 7f7e84922640 1 -- 172.21.15.5:0/3852851202 wait complete. 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e84922640 1 Processor -- start 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e84922640 1 -- start start 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e84922640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e80100c10 0x7f7e8019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e84922640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e80102ac0 0x7f7e8019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e84922640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80102ea0 0x7f7e8019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e84922640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80114170 con 0x7f7e80102ac0 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e84922640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e80113ff0 con 0x7f7e80102ea0 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e84922640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801142f0 con 0x7f7e80100c10 2024-09-11T22:27:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e7dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e80102ac0 0x7f7e8019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e7dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e80102ac0 0x7f7e8019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60280/0 (socket says 172.21.15.5:60280) 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e7dd9b640 1 -- 172.21.15.5:0/3838750836 learned_addr learned my addr 172.21.15.5:0/3838750836 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e7ed9d640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80102ea0 0x7f7e8019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.584+0000 7f7e7e59c640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e80100c10 0x7f7e8019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e7dd9b640 1 -- 172.21.15.5:0/3838750836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e80100c10 msgr2=0x7f7e8019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e7dd9b640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e80100c10 0x7f7e8019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e7dd9b640 1 -- 172.21.15.5:0/3838750836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80102ea0 msgr2=0x7f7e8019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e7dd9b640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80102ea0 0x7f7e8019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e7dd9b640 1 -- 172.21.15.5:0/3838750836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8019f660 con 0x7f7e80102ac0 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e7e59c640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e80100c10 0x7f7e8019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e7dd9b640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e80102ac0 0x7f7e8019abf0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400d770 tx=0x7f7e7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e6f7fe640 1 -- 172.21.15.5:0/3838750836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74017070 con 0x7f7e80102ac0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e6f7fe640 1 -- 172.21.15.5:0/3838750836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e74005500 con 0x7f7e80102ac0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e84922640 1 -- 172.21.15.5:0/3838750836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8019f950 con 0x7f7e80102ac0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e6f7fe640 1 -- 172.21.15.5:0/3838750836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74004e00 con 0x7f7e80102ac0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.585+0000 7f7e84922640 1 -- 172.21.15.5:0/3838750836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e801a7230 con 0x7f7e80102ac0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.586+0000 7f7e84922640 1 -- 172.21.15.5:0/3838750836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e801000b0 con 0x7f7e80102ac0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.587+0000 7f7e6f7fe640 1 -- 172.21.15.5:0/3838750836 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7e74004fa0 con 0x7f7e80102ac0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.588+0000 7f7e6f7fe640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e680a2cf0 0x7f7e680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.588+0000 7f7e6f7fe640 1 -- 172.21.15.5:0/3838750836 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7e740ceeb0 con 0x7f7e80102ac0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.588+0000 7f7e7e59c640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e680a2cf0 0x7f7e680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.588+0000 7f7e7e59c640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e680a2cf0 0x7f7e680a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7e70000c00 tx=0x7f7e70032c70 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.592+0000 7f7e6f7fe640 1 -- 172.21.15.5:0/3838750836 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e7409ba90 con 0x7f7e80102ac0 2024-09-11T22:27:01.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:27:01.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.806+0000 7f7e84922640 1 -- 172.21.15.5:0/3838750836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e8010bf10 con 0x7f7e80102ac0 2024-09-11T22:27:01.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:01 smithi005 ceph-mon[30943]: pgmap v148: 97 pgs: 97 active+clean; 49 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 809 KiB/s wr, 82 op/s 2024-09-11T22:27:01.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.807+0000 7f7e6f7fe640 1 -- 172.21.15.5:0/3838750836 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11967 (secure 0 0 0) 0x7f7e7409b390 con 0x7f7e80102ac0 2024-09-11T22:27:01.809 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:27:01.810 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-11T22:25:49:149525+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:25:48.784660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25970":{"gid":25970,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4169170564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4169170564},{"type":"v1","addr":"172.21.15.159:6833","nonce":4169170564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:27:01.810 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-11T22:27:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 -- 172.21.15.5:0/3838750836 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e680a2cf0 msgr2=0x7f7e680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e680a2cf0 0x7f7e680a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7e70000c00 tx=0x7f7e70032c70 comp rx=0 tx=0).stop 2024-09-11T22:27:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 -- 172.21.15.5:0/3838750836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e80102ac0 msgr2=0x7f7e8019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e80102ac0 0x7f7e8019abf0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7e7400d770 tx=0x7f7e7400dc30 comp rx=0 tx=0).stop 2024-09-11T22:27:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 -- 172.21.15.5:0/3838750836 shutdown_connections 2024-09-11T22:27:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e680a2cf0 0x7f7e680a51b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e80102ea0 0x7f7e8019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e80102ac0 0x7f7e8019abf0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 --2- 172.21.15.5:0/3838750836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e80100c10 0x7f7e8019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 -- 172.21.15.5:0/3838750836 >> 172.21.15.5:0/3838750836 conn(0x7f7e80077210 msgr2=0x7f7e800fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 -- 172.21.15.5:0/3838750836 shutdown_connections 2024-09-11T22:27:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:01.810+0000 7f7e84922640 1 -- 172.21.15.5:0/3838750836 wait complete. 2024-09-11T22:27:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:01 smithi159 ceph-mon[35534]: pgmap v148: 97 pgs: 97 active+clean; 49 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 809 KiB/s wr, 82 op/s 2024-09-11T22:27:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:01 smithi172 ceph-mon[35461]: pgmap v148: 97 pgs: 97 active+clean; 49 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 809 KiB/s wr, 82 op/s 2024-09-11T22:27:02.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 7.32 MB/sec execute 80 sec latency 18547.819 ms 2024-09-11T22:27:02.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:02 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3838750836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:27:02.754 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 5e46c553 --release --await' 2024-09-11T22:27:02.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 5e46c553 --release --await 2024-09-11T22:27:02.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:02 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3838750836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:27:02.850 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:02.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:02 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3838750836' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:27:03.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 7.23 MB/sec execute 81 sec latency 19547.851 ms 2024-09-11T22:27:03.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.351+0000 7f19a4189640 1 -- 172.21.15.5:0/2276883466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f199c100e20 msgr2=0x7f199c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.351+0000 7f19a4189640 1 --2- 172.21.15.5:0/2276883466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f199c100e20 0x7f199c101200 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f1998009ca0 tx=0x7f199802a950 comp rx=0 tx=0).stop 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.351+0000 7f19a4189640 1 -- 172.21.15.5:0/2276883466 shutdown_connections 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.351+0000 7f19a4189640 1 --2- 172.21.15.5:0/2276883466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c102ef0 0x7f199c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.351+0000 7f19a4189640 1 --2- 172.21.15.5:0/2276883466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c102570 0x7f199c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.351+0000 7f19a4189640 1 --2- 172.21.15.5:0/2276883466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f199c100e20 0x7f199c101200 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.351+0000 7f19a4189640 1 -- 172.21.15.5:0/2276883466 >> 172.21.15.5:0/2276883466 conn(0x7f199c0fc7d0 msgr2=0x7f199c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.351+0000 7f19a4189640 1 -- 172.21.15.5:0/2276883466 shutdown_connections 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.351+0000 7f19a4189640 1 -- 172.21.15.5:0/2276883466 wait complete. 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a4189640 1 Processor -- start 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a4189640 1 -- start start 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a4189640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c100e20 0x7f199c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a4189640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f199c102570 0x7f199c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a4189640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c102ef0 0x7f199c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a4189640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f199c110f80 con 0x7f199c102570 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a4189640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f199c110e00 con 0x7f199c100e20 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a4189640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f199c111100 con 0x7f199c102ef0 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a1724640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f199c102570 0x7f199c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a1724640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f199c102570 0x7f199c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60296/0 (socket says 172.21.15.5:60296) 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a1724640 1 -- 172.21.15.5:0/2399619121 learned_addr learned my addr 172.21.15.5:0/2399619121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a1f25640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c100e20 0x7f199c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.352+0000 7f19a2726640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c102ef0 0x7f199c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f19a1724640 1 -- 172.21.15.5:0/2399619121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c102ef0 msgr2=0x7f199c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f19a1724640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c102ef0 0x7f199c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f19a1724640 1 -- 172.21.15.5:0/2399619121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c100e20 msgr2=0x7f199c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:03.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f19a1724640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c100e20 0x7f199c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f19a1724640 1 -- 172.21.15.5:0/2399619121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c19d4e0 con 0x7f199c102570 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f19a1724640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f199c102570 0x7f199c198a70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f198c00d770 tx=0x7f198c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f1992ffd640 1 -- 172.21.15.5:0/2399619121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c017070 con 0x7f199c102570 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f1992ffd640 1 -- 172.21.15.5:0/2399619121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f198c005500 con 0x7f199c102570 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f19a4189640 1 -- 172.21.15.5:0/2399619121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199c19d7d0 con 0x7f199c102570 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f1992ffd640 1 -- 172.21.15.5:0/2399619121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c004e00 con 0x7f199c102570 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.353+0000 7f19a4189640 1 -- 172.21.15.5:0/2399619121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199c1a50b0 con 0x7f199c102570 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.355+0000 7f1992ffd640 1 -- 172.21.15.5:0/2399619121 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f198c004fa0 con 0x7f199c102570 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a6562740 1 Processor -- start 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a6562740 1 -- start start 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a6562740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 0x56074ddf0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a6562740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddfb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a6562740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddfb680 0x56074ddfda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a6562740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56074dd562e0 con 0x56074ddf0dc0 2024-09-11T22:27:03.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a6562740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56074dcfdce0 con 0x56074ddf04a0 2024-09-11T22:27:03.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a6562740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56074dd522a0 con 0x56074ddfb680 2024-09-11T22:27:03.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a1f25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddfb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a1f25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddfb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60300/0 (socket says 172.21.15.5:60300) 2024-09-11T22:27:03.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a1f25640 1 -- 172.21.15.5:0/2146186416 learned_addr learned my addr 172.21.15.5:0/2146186416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:03.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a2726640 1 --2- 172.21.15.5:0/2146186416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 0x56074ddf0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.356+0000 7f19a1724640 1 --2- 172.21.15.5:0/2146186416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddfb680 0x56074ddfda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f19a1f25640 1 -- 172.21.15.5:0/2146186416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddfb680 msgr2=0x56074ddfda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:03.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f19a1f25640 1 --2- 172.21.15.5:0/2146186416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddfb680 0x56074ddfda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f19a1f25640 1 -- 172.21.15.5:0/2146186416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 msgr2=0x56074ddf0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f1992ffd640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f19880a2d70 0x7f19880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f19a1f25640 1 --2- 172.21.15.5:0/2146186416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 0x56074ddf0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f19a1f25640 1 -- 172.21.15.5:0/2146186416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56074dce2440 con 0x56074ddf0dc0 2024-09-11T22:27:03.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f19a2726640 1 --2- 172.21.15.5:0/2146186416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 0x56074ddf0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:03.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f19a2726640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f19880a2d70 0x7f19880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f1992ffd640 1 -- 172.21.15.5:0/2399619121 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f198c0cee10 con 0x7f199c102570 2024-09-11T22:27:03.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f19a1f25640 1 --2- 172.21.15.5:0/2146186416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddfb140 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1998004ad0 tx=0x7f19980098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf5f39c06c49fc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:03.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f1987fff640 1 -- 172.21.15.5:0/2146186416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998048070 con 0x56074ddf0dc0 2024-09-11T22:27:03.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f1987fff640 1 -- 172.21.15.5:0/2146186416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f199802f930 con 0x56074ddf0dc0 2024-09-11T22:27:03.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f19a2726640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f19880a2d70 0x7f19880a5230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1994004650 tx=0x7f19940046c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.357+0000 7f1987fff640 1 -- 172.21.15.5:0/2146186416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998034040 con 0x56074ddf0dc0 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.358+0000 7f19a6562740 1 -- 172.21.15.5:0/2146186416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 msgr2=0x56074ddfb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.358+0000 7f19a6562740 1 --2- 172.21.15.5:0/2146186416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddfb140 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1998004ad0 tx=0x7f19980098d0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.358+0000 7f19a6562740 1 -- 172.21.15.5:0/2146186416 shutdown_connections 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.358+0000 7f19a6562740 1 --2- 172.21.15.5:0/2146186416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddfb680 0x56074ddfda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.358+0000 7f19a6562740 1 --2- 172.21.15.5:0/2146186416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddfb140 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.358+0000 7f19a6562740 1 --2- 172.21.15.5:0/2146186416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 0x56074ddf0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.358+0000 7f19a6562740 1 -- 172.21.15.5:0/2146186416 >> 172.21.15.5:0/2146186416 conn(0x56074ddeca30 msgr2=0x56074ddece10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.358+0000 7f19a6562740 1 -- 172.21.15.5:0/2146186416 shutdown_connections 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.358+0000 7f19a6562740 1 -- 172.21.15.5:0/2146186416 wait complete. 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a6562740 1 -- start start 2024-09-11T22:27:03.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a6562740 1 Processor -- start 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a6562740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf04a0 0x56074ddfaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a6562740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddf8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a6562740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddfb680 0x56074ddf8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a6562740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56074ddfe250 con 0x56074ddf0dc0 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a6562740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56074dd11050 con 0x56074ddfb680 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a6562740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56074ddfe3d0 con 0x56074ddf04a0 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a6562740 10 client.0 _ll_register_callbacks cb 0x56074dd51160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a1724640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddf8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a1724640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddf8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60304/0 (socket says 172.21.15.5:60304) 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a1724640 1 -- 172.21.15.5:0/2931536921 learned_addr learned my addr 172.21.15.5:0/2931536921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a1f25640 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf04a0 0x56074ddfaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.359+0000 7f19a2726640 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddfb680 0x56074ddf8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a1724640 1 -- 172.21.15.5:0/2931536921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf04a0 msgr2=0x56074ddfaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a1724640 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf04a0 0x56074ddfaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a1724640 1 -- 172.21.15.5:0/2931536921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddfb680 msgr2=0x56074ddf8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a1724640 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddfb680 0x56074ddf8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a1724640 1 -- 172.21.15.5:0/2931536921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56074dac73c0 con 0x56074ddf0dc0 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a2726640 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddfb680 0x56074ddf8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a1724640 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddf8220 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f198c0cc520 tx=0x7f198c00df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ceb1a2a8a7d43e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19857fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19857fa640 1 -- 172.21.15.5:0/2931536921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c020070 con 0x56074ddf0dc0 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19857fa640 1 -- 172.21.15.5:0/2931536921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f198c0979b0 con 0x56074ddf0dc0 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a6562740 1 -- 172.21.15.5:0/2931536921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f199802f930 con 0x56074ddf0dc0 2024-09-11T22:27:03.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19857fa640 1 -- 172.21.15.5:0/2931536921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f198c09aca0 con 0x56074ddf0dc0 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a6562740 10 client.16878 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a6562740 1 -- 172.21.15.5:0/2931536921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56074de09c10 con 0x56074ddf0dc0 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19857fa640 1 -- 172.21.15.5:0/2931536921 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f199802f930 con 0x56074ddf0dc0 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19857fa640 1 -- 172.21.15.5:0/2931536921 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f198c0b40f0 con 0x56074ddf0dc0 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a6562740 10 client.16878 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a6562740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a6562740 20 client.16878 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a6562740 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56074de0b020 0x56074de0d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a6562740 4 client.16878 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.360+0000 7f19a6562740 1 -- 172.21.15.5:0/2931536921 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56074dd63850 con 0x56074de0b020 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.361+0000 7f19a1f25640 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56074de0b020 0x56074de0d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.361+0000 7f19a1f25640 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56074de0b020 0x56074de0d430 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba185d984325ebc server_cookie=b9e17a7c1cdde959 in_seq=0 out_seq=0 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.361+0000 7f19857fa640 10 client.16878 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19857fa640 1 -- 172.21.15.5:0/2931536921 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56074dd63850 con 0x56074de0b020 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19857fa640 10 client.16878 handle_command_reply: tid=0 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19a6562740 1 client.16878 shutdown 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19a6562740 1 -- 172.21.15.5:0/2931536921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 msgr2=0x56074ddf8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19a6562740 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddf8220 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f198c0cc520 tx=0x7f198c00df40 comp rx=0 tx=0).stop 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19a6562740 1 -- 172.21.15.5:0/2931536921 shutdown_connections 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19a6562740 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56074de0b020 0x56074de0d430 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19a6562740 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddfb680 0x56074ddf8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19a6562740 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074ddf8220 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19857fa640 0 client.16878 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:03.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19a6562740 1 --2- 172.21.15.5:0/2931536921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf04a0 0x56074ddfaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.362+0000 7f19a6562740 1 -- 172.21.15.5:0/2931536921 >> 172.21.15.5:0/2931536921 conn(0x56074ddeca30 msgr2=0x56074ddebb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.363+0000 7f19a6562740 1 -- 172.21.15.5:0/2931536921 shutdown_connections 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.363+0000 7f19a6562740 1 -- 172.21.15.5:0/2931536921 wait complete. 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.363+0000 7f19a6562740 20 client.16878 trim_cache size 0 max 16384 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a6562740 1 Processor -- start 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a6562740 1 -- start start 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a6562740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddeca30 0x56074ddfa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a6562740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf04a0 0x56074de12d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a6562740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074de13240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a6562740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1ad880 con 0x56074ddf0dc0 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a6562740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1af3c0 con 0x56074ddeca30 2024-09-11T22:27:03.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a6562740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1adf50 con 0x56074ddf04a0 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a2726640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074de13240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a2726640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074de13240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60306/0 (socket says 172.21.15.5:60306) 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a2726640 1 -- 172.21.15.5:0/2448917678 learned_addr learned my addr 172.21.15.5:0/2448917678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a1724640 1 --2- 172.21.15.5:0/2448917678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf04a0 0x56074de12d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a1f25640 1 --2- 172.21.15.5:0/2448917678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddeca30 0x56074ddfa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a2726640 1 -- 172.21.15.5:0/2448917678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf04a0 msgr2=0x56074de12d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a2726640 1 --2- 172.21.15.5:0/2448917678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf04a0 0x56074de12d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a2726640 1 -- 172.21.15.5:0/2448917678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddeca30 msgr2=0x56074ddfa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a2726640 1 --2- 172.21.15.5:0/2448917678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddeca30 0x56074ddfa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a2726640 1 -- 172.21.15.5:0/2448917678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56074ddec4d0 con 0x56074ddf0dc0 2024-09-11T22:27:03.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.380+0000 7f19a1f25640 1 --2- 172.21.15.5:0/2448917678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddeca30 0x56074ddfa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19a2726640 1 --2- 172.21.15.5:0/2448917678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074de13240 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1994005f00 tx=0x7f1994016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c187a7c45c1321a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19877fe640 1 -- 172.21.15.5:0/2448917678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1994011bc0 con 0x56074ddf0dc0 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19877fe640 1 -- 172.21.15.5:0/2448917678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1994011d60 con 0x56074ddf0dc0 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19877fe640 1 -- 172.21.15.5:0/2448917678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199401c7a0 con 0x56074ddf0dc0 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19a6562740 1 -- 172.21.15.5:0/2448917678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 msgr2=0x56074de13240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19a6562740 1 --2- 172.21.15.5:0/2448917678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074de13240 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1994005f00 tx=0x7f1994016000 comp rx=0 tx=0).stop 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19a6562740 1 -- 172.21.15.5:0/2448917678 shutdown_connections 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19a6562740 1 --2- 172.21.15.5:0/2448917678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddf0dc0 0x56074de13240 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19a6562740 1 --2- 172.21.15.5:0/2448917678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf04a0 0x56074de12d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19a6562740 1 --2- 172.21.15.5:0/2448917678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddeca30 0x56074ddfa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19a6562740 1 -- 172.21.15.5:0/2448917678 >> 172.21.15.5:0/2448917678 conn(0x56074ddf88c0 msgr2=0x56074ddf9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19a6562740 1 -- 172.21.15.5:0/2448917678 shutdown_connections 2024-09-11T22:27:03.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.381+0000 7f19a6562740 1 -- 172.21.15.5:0/2448917678 wait complete. 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a6562740 1 -- start start 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a6562740 1 Processor -- start 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a6562740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddeca30 0x56074de01510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a6562740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 0x56074de01a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a6562740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf0dc0 0x56074de01f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a6562740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1aecf0 con 0x56074ddeca30 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a6562740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1ae620 con 0x56074ddf04a0 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a6562740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f199c1ab860 con 0x56074ddf0dc0 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a6562740 10 client.0 _ll_register_callbacks cb 0x7f198c00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a1f25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddeca30 0x56074de01510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a1f25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddeca30 0x56074de01510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60310/0 (socket says 172.21.15.5:60310) 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a1f25640 1 -- 172.21.15.5:0/1819963766 learned_addr learned my addr 172.21.15.5:0/1819963766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a2726640 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf0dc0 0x56074de01f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.382+0000 7f19a1724640 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 0x56074de01a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19a1f25640 1 -- 172.21.15.5:0/1819963766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf0dc0 msgr2=0x56074de01f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19a1f25640 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf0dc0 0x56074de01f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19a1f25640 1 -- 172.21.15.5:0/1819963766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 msgr2=0x56074de01a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19a1f25640 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 0x56074de01a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19a1f25640 1 -- 172.21.15.5:0/1819963766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56074de74e80 con 0x56074ddeca30 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19a1724640 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 0x56074de01a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19a1f25640 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddeca30 0x56074de01510 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f199802ae70 tx=0x7f1998003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d0a1417cd68b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19857fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:03.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19857fa640 1 -- 172.21.15.5:0/1819963766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998054070 con 0x56074ddeca30 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19857fa640 1 -- 172.21.15.5:0/1819963766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f199802f930 con 0x56074ddeca30 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19a6562740 1 -- 172.21.15.5:0/1819963766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1994011d60 con 0x56074ddeca30 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19857fa640 1 -- 172.21.15.5:0/1819963766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1998048040 con 0x56074ddeca30 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19a6562740 10 client.16884 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19a6562740 1 -- 172.21.15.5:0/1819963766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56074de76130 con 0x56074ddeca30 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.383+0000 7f19857fa640 1 -- 172.21.15.5:0/1819963766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1994011d60 con 0x56074ddeca30 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.384+0000 7f19857fa640 1 -- 172.21.15.5:0/1819963766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f1998004240 con 0x56074ddeca30 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.384+0000 7f19a6562740 10 client.16884 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.384+0000 7f19a6562740 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.384+0000 7f19a6562740 20 client.16884 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.384+0000 7f19a6562740 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56074ddfb680 0x56074de784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:03.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.384+0000 7f19a6562740 4 client.16884 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:27:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.384+0000 7f19a6562740 1 -- 172.21.15.5:0/1819963766 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56074dd5dd60 con 0x56074ddfb680 2024-09-11T22:27:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.384+0000 7f19a1724640 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56074ddfb680 0x56074de784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.384+0000 7f19857fa640 10 client.16884 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.384+0000 7f19a1724640 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56074ddfb680 0x56074de784d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73ba7938dff0f46e server_cookie=9cbe84284cb55988 in_seq=0 out_seq=0 2024-09-11T22:27:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19857fa640 1 -- 172.21.15.5:0/1819963766 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56074dd5dd60 con 0x56074ddfb680 2024-09-11T22:27:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19857fa640 10 client.16884 handle_command_reply: tid=0 2024-09-11T22:27:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 client.16884 shutdown 2024-09-11T22:27:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 -- 172.21.15.5:0/1819963766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddeca30 msgr2=0x56074de01510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddeca30 0x56074de01510 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f199802ae70 tx=0x7f1998003dc0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 -- 172.21.15.5:0/1819963766 shutdown_connections 2024-09-11T22:27:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56074ddfb680 0x56074de784d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56074ddf0dc0 0x56074de01f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56074ddf04a0 0x56074de01a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 --2- 172.21.15.5:0/1819963766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56074ddeca30 0x56074de01510 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19857fa640 0 client.16884 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:03.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 -- 172.21.15.5:0/1819963766 >> 172.21.15.5:0/1819963766 conn(0x56074ddf88c0 msgr2=0x56074ddfa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 -- 172.21.15.5:0/1819963766 shutdown_connections 2024-09-11T22:27:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 1 -- 172.21.15.5:0/1819963766 wait complete. 2024-09-11T22:27:03.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.385+0000 7f19a6562740 20 client.16884 trim_cache size 0 max 16384 2024-09-11T22:27:03.407 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:27:03.407 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:27:03.407 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:27:03.407 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:27:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 -- 172.21.15.5:0/2399619121 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f19880a2d70 msgr2=0x7f19880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f19880a2d70 0x7f19880a5230 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1994004650 tx=0x7f19940046c0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 -- 172.21.15.5:0/2399619121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f199c102570 msgr2=0x7f199c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f199c102570 0x7f199c198a70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f198c00d770 tx=0x7f198c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:27:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 -- 172.21.15.5:0/2399619121 shutdown_connections 2024-09-11T22:27:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f199c102ef0 0x7f199c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f199c102570 0x7f199c198a70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f19880a2d70 0x7f19880a5230 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 --2- 172.21.15.5:0/2399619121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f199c100e20 0x7f199c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 -- 172.21.15.5:0/2399619121 >> 172.21.15.5:0/2399619121 conn(0x7f199c0fc7d0 msgr2=0x7f199c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 -- 172.21.15.5:0/2399619121 shutdown_connections 2024-09-11T22:27:03.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:03.386+0000 7f19a4189640 1 -- 172.21.15.5:0/2399619121 wait complete. 2024-09-11T22:27:03.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:03 smithi005 ceph-mon[30943]: pgmap v149: 97 pgs: 97 active+clean; 44 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 839 KiB/s wr, 103 op/s 2024-09-11T22:27:03.657 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:03.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:03 smithi159 ceph-mon[35534]: pgmap v149: 97 pgs: 97 active+clean; 44 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 839 KiB/s wr, 103 op/s 2024-09-11T22:27:03.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:03 smithi172 ceph-mon[35461]: pgmap v149: 97 pgs: 97 active+clean; 44 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 839 KiB/s wr, 103 op/s 2024-09-11T22:27:04.234 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:27:04.234 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:27:04.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323222 7.14 MB/sec execute 82 sec latency 20547.900 ms 2024-09-11T22:27:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.825+0000 7fde5cb9d640 1 -- 172.21.15.5:0/1909688170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde581036c0 msgr2=0x7fde5810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.825+0000 7fde5cb9d640 1 --2- 172.21.15.5:0/1909688170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde581036c0 0x7fde5810bbb0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fde40009900 tx=0x7fde4002a480 comp rx=0 tx=0).stop 2024-09-11T22:27:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.825+0000 7fde5cb9d640 1 -- 172.21.15.5:0/1909688170 shutdown_connections 2024-09-11T22:27:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.825+0000 7fde5cb9d640 1 --2- 172.21.15.5:0/1909688170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde5810c180 0x7fde5810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.825+0000 7fde5cb9d640 1 --2- 172.21.15.5:0/1909688170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde581036c0 0x7fde5810bbb0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.825+0000 7fde5cb9d640 1 --2- 172.21.15.5:0/1909688170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde58102da0 0x7fde58103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.825+0000 7fde5cb9d640 1 -- 172.21.15.5:0/1909688170 >> 172.21.15.5:0/1909688170 conn(0x7fde580fc7a0 msgr2=0x7fde580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.825+0000 7fde5cb9d640 1 -- 172.21.15.5:0/1909688170 shutdown_connections 2024-09-11T22:27:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.825+0000 7fde5cb9d640 1 -- 172.21.15.5:0/1909688170 wait complete. 2024-09-11T22:27:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde5cb9d640 1 Processor -- start 2024-09-11T22:27:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde5cb9d640 1 -- start start 2024-09-11T22:27:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde5cb9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde58102da0 0x7fde58198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde5cb9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde581036c0 0x7fde58198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde5cb9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde5810c180 0x7fde5819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde5cb9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde58111fa0 con 0x7fde5810c180 2024-09-11T22:27:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde5cb9d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde58111e20 con 0x7fde581036c0 2024-09-11T22:27:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde5cb9d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde58112120 con 0x7fde58102da0 2024-09-11T22:27:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde56d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde5810c180 0x7fde5819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde56d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde5810c180 0x7fde5819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60326/0 (socket says 172.21.15.5:60326) 2024-09-11T22:27:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde56d9d640 1 -- 172.21.15.5:0/872777899 learned_addr learned my addr 172.21.15.5:0/872777899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.826+0000 7fde4ffff640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde581036c0 0x7fde58198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde5659c640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde58102da0 0x7fde58198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde56d9d640 1 -- 172.21.15.5:0/872777899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde58102da0 msgr2=0x7fde58198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde56d9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde58102da0 0x7fde58198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde56d9d640 1 -- 172.21.15.5:0/872777899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde581036c0 msgr2=0x7fde58198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde56d9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde581036c0 0x7fde58198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde56d9d640 1 -- 172.21.15.5:0/872777899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5819d3f0 con 0x7fde5810c180 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde56d9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde5810c180 0x7fde5819cd10 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fde5000cce0 tx=0x7fde5000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde4f7fe640 1 -- 172.21.15.5:0/872777899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5000f450 con 0x7fde5810c180 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde5cb9d640 1 -- 172.21.15.5:0/872777899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5819d6e0 con 0x7fde5810c180 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde4f7fe640 1 -- 172.21.15.5:0/872777899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5000fad0 con 0x7fde5810c180 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde4f7fe640 1 -- 172.21.15.5:0/872777899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50018c10 con 0x7fde5810c180 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.827+0000 7fde5cb9d640 1 -- 172.21.15.5:0/872777899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde581a4fc0 con 0x7fde5810c180 2024-09-11T22:27:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde4f7fe640 1 -- 172.21.15.5:0/872777899 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fde5000f5f0 con 0x7fde5810c180 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde5ef76740 1 Processor -- start 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde5ef76740 1 -- start start 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e0f60 0x5636886e1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e1880 0x5636886f0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x5636886f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde5ef76740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563688646620 con 0x5636886e0f60 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde5ef76740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636885ede50 con 0x5636886e1880 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde5ef76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5636886425e0 con 0x5636886f0650 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e0f60 0x5636886e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e0f60 0x5636886e1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60334/0 (socket says 172.21.15.5:60334) 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.830+0000 7fde4ffff640 1 -- 172.21.15.5:0/4064788949 learned_addr learned my addr 172.21.15.5:0/4064788949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4f7fe640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fde440a2cf0 0x7fde440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde5659c640 1 --2- 172.21.15.5:0/4064788949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e1880 0x5636886f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4ffff640 1 --2- 172.21.15.5:0/4064788949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x5636886f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde56d9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fde440a2cf0 0x7fde440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4ffff640 1 -- 172.21.15.5:0/4064788949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 msgr2=0x5636886f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4ffff640 1 --2- 172.21.15.5:0/4064788949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x5636886f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4ffff640 1 -- 172.21.15.5:0/4064788949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e1880 msgr2=0x5636886f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4ffff640 1 --2- 172.21.15.5:0/4064788949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e1880 0x5636886f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4ffff640 1 -- 172.21.15.5:0/4064788949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636885ded40 con 0x5636886e0f60 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4f7fe640 1 -- 172.21.15.5:0/872777899 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fde500cfb80 con 0x7fde5810c180 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4ffff640 1 --2- 172.21.15.5:0/4064788949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e0f60 0x5636886e1340 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fde4002a9a0 tx=0x7fde40005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c98ed40c59d993f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4cff9640 1 -- 172.21.15.5:0/4064788949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40035040 con 0x5636886e0f60 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde56d9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fde440a2cf0 0x7fde440a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde5009bff0 tx=0x7fde500914f0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4cff9640 1 -- 172.21.15.5:0/4064788949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde400035f0 con 0x5636886e0f60 2024-09-11T22:27:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.831+0000 7fde4cff9640 1 -- 172.21.15.5:0/4064788949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4003a650 con 0x5636886e0f60 2024-09-11T22:27:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.832+0000 7fde5ef76740 1 -- 172.21.15.5:0/4064788949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e0f60 msgr2=0x5636886e1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.832+0000 7fde5ef76740 1 --2- 172.21.15.5:0/4064788949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e0f60 0x5636886e1340 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fde4002a9a0 tx=0x7fde40005d90 comp rx=0 tx=0).stop 2024-09-11T22:27:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.832+0000 7fde5ef76740 1 -- 172.21.15.5:0/4064788949 shutdown_connections 2024-09-11T22:27:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.832+0000 7fde5ef76740 1 --2- 172.21.15.5:0/4064788949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x5636886f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.832+0000 7fde5ef76740 1 --2- 172.21.15.5:0/4064788949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e1880 0x5636886f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.832+0000 7fde5ef76740 1 --2- 172.21.15.5:0/4064788949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e0f60 0x5636886e1340 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.832+0000 7fde5ef76740 1 -- 172.21.15.5:0/4064788949 >> 172.21.15.5:0/4064788949 conn(0x5636886dd5f0 msgr2=0x5636886dd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.832+0000 7fde5ef76740 1 -- 172.21.15.5:0/4064788949 shutdown_connections 2024-09-11T22:27:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.832+0000 7fde5ef76740 1 -- 172.21.15.5:0/4064788949 wait complete. 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.832+0000 7fde5ef76740 1 -- start start 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde5ef76740 1 Processor -- start 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636886ed580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886e1880 0x5636886edac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886f0650 0x5636886ee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde5ef76740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5636886f3220 con 0x5636886f0650 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde5ef76740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636885cef10 con 0x5636886e0f60 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde5ef76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5636886f33a0 con 0x5636886e1880 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde5ef76740 10 client.0 _ll_register_callbacks cb 0x5636885f2db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886f0650 0x5636886ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886f0650 0x5636886ee000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60336/0 (socket says 172.21.15.5:60336) 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde4ffff640 1 -- 172.21.15.5:0/3779800811 learned_addr learned my addr 172.21.15.5:0/3779800811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde5659c640 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886e1880 0x5636886edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde4ffff640 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636886ed580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde4ffff640 1 -- 172.21.15.5:0/3779800811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886e1880 msgr2=0x5636886edac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde4ffff640 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886e1880 0x5636886edac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde4ffff640 1 -- 172.21.15.5:0/3779800811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 msgr2=0x5636886ed580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde4ffff640 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636886ed580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde4ffff640 1 -- 172.21.15.5:0/3779800811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563688355580 con 0x5636886f0650 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde4ffff640 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886f0650 0x5636886ee000 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fde40004350 tx=0x7fde40003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e654422db23c280c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.833+0000 7fde3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde3dffb640 1 -- 172.21.15.5:0/3779800811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40035040 con 0x5636886f0650 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde3dffb640 1 -- 172.21.15.5:0/3779800811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fde40045040 con 0x5636886f0650 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde5ef76740 1 -- 172.21.15.5:0/3779800811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde400035f0 con 0x5636886f0650 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde3dffb640 1 -- 172.21.15.5:0/3779800811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fde4005d050 con 0x5636886f0650 2024-09-11T22:27:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde5ef76740 10 client.16896 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde5ef76740 1 -- 172.21.15.5:0/3779800811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56368841f960 con 0x5636886f0650 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde3dffb640 1 -- 172.21.15.5:0/3779800811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde400035f0 con 0x5636886f0650 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde3dffb640 1 -- 172.21.15.5:0/3779800811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fde400499a0 con 0x5636886f0650 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde5ef76740 10 client.16896 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde5ef76740 10 client.16896 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde5ef76740 20 client.16896 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde5ef76740 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5636887000e0 0x5636887024d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde5ef76740 4 client.16896 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.834+0000 7fde5ef76740 1 -- 172.21.15.5:0/3779800811 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563688653b90 con 0x5636887000e0 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.835+0000 7fde5659c640 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5636887000e0 0x5636887024d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.835+0000 7fde5659c640 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5636887000e0 0x5636887024d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa9a94808ab081d6 server_cookie=102f6877f029ecf1 in_seq=0 out_seq=0 2024-09-11T22:27:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.835+0000 7fde3dffb640 10 client.16896 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde3dffb640 1 -- 172.21.15.5:0/3779800811 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563688653b90 con 0x5636887000e0 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde3dffb640 10 client.16896 handle_command_reply: tid=0 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 client.16896 shutdown 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 -- 172.21.15.5:0/3779800811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886f0650 msgr2=0x5636886ee000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886f0650 0x5636886ee000 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fde40004350 tx=0x7fde40003f30 comp rx=0 tx=0).stop 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 -- 172.21.15.5:0/3779800811 shutdown_connections 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5636887000e0 0x5636887024d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde3dffb640 0 client.16896 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886f0650 0x5636886ee000 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886e1880 0x5636886edac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 --2- 172.21.15.5:0/3779800811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636886ed580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 -- 172.21.15.5:0/3779800811 >> 172.21.15.5:0/3779800811 conn(0x5636886dd5f0 msgr2=0x5636886e3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 -- 172.21.15.5:0/3779800811 shutdown_connections 2024-09-11T22:27:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 1 -- 172.21.15.5:0/3779800811 wait complete. 2024-09-11T22:27:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.836+0000 7fde5ef76740 20 client.16896 trim_cache size 0 max 16384 2024-09-11T22:27:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.856+0000 7fde5ef76740 1 Processor -- start 2024-09-11T22:27:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.856+0000 7fde5ef76740 1 -- start start 2024-09-11T22:27:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.856+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636887054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.856+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688705a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.856+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x5636886ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.856+0000 7fde5ef76740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde581ab630 con 0x5636886e1880 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.856+0000 7fde5ef76740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde581ad170 con 0x5636886e0f60 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.856+0000 7fde5ef76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde581abd00 con 0x5636886f0650 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.856+0000 7fde5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688705a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688705a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60344/0 (socket says 172.21.15.5:60344) 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde5659c640 1 -- 172.21.15.5:0/1205681133 learned_addr learned my addr 172.21.15.5:0/1205681133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde4ffff640 1 --2- 172.21.15.5:0/1205681133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x5636886ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde4ffff640 1 --2- 172.21.15.5:0/1205681133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636887054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde5659c640 1 -- 172.21.15.5:0/1205681133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 msgr2=0x5636886ea7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde5659c640 1 --2- 172.21.15.5:0/1205681133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x5636886ea7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde5659c640 1 -- 172.21.15.5:0/1205681133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 msgr2=0x5636887054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde5659c640 1 --2- 172.21.15.5:0/1205681133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636887054f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde5659c640 1 -- 172.21.15.5:0/1205681133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636886e16b0 con 0x5636886e1880 2024-09-11T22:27:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde5659c640 1 --2- 172.21.15.5:0/1205681133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688705a30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x5636886ece10 tx=0x7fde48008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454160c0f96a086d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde3ffff640 1 -- 172.21.15.5:0/1205681133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48016a60 con 0x5636886e1880 2024-09-11T22:27:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde3ffff640 1 -- 172.21.15.5:0/1205681133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde48020030 con 0x5636886e1880 2024-09-11T22:27:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.857+0000 7fde3ffff640 1 -- 172.21.15.5:0/1205681133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4801c250 con 0x5636886e1880 2024-09-11T22:27:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.858+0000 7fde5ef76740 1 -- 172.21.15.5:0/1205681133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 msgr2=0x563688705a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.858+0000 7fde5ef76740 1 --2- 172.21.15.5:0/1205681133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688705a30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x5636886ece10 tx=0x7fde48008040 comp rx=0 tx=0).stop 2024-09-11T22:27:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.858+0000 7fde5ef76740 1 -- 172.21.15.5:0/1205681133 shutdown_connections 2024-09-11T22:27:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.858+0000 7fde5ef76740 1 --2- 172.21.15.5:0/1205681133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x5636886ea7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.858+0000 7fde5ef76740 1 --2- 172.21.15.5:0/1205681133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688705a30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.858+0000 7fde5ef76740 1 --2- 172.21.15.5:0/1205681133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636887054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.858+0000 7fde5ef76740 1 -- 172.21.15.5:0/1205681133 >> 172.21.15.5:0/1205681133 conn(0x5636886dd5f0 msgr2=0x5636886ebe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.858+0000 7fde5ef76740 1 -- 172.21.15.5:0/1205681133 shutdown_connections 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.858+0000 7fde5ef76740 1 -- 172.21.15.5:0/1205681133 wait complete. 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5ef76740 1 -- start start 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5ef76740 1 Processor -- start 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636887035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688703ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5ef76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x563688769a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5ef76740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde581acaa0 con 0x5636886e1880 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5ef76740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde581ac3d0 con 0x5636886e0f60 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5ef76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fde581a9610 con 0x5636886f0650 2024-09-11T22:27:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5ef76740 10 client.0 _ll_register_callbacks cb 0x7fde40031780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688703ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688703ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60358/0 (socket says 172.21.15.5:60358) 2024-09-11T22:27:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde5659c640 1 -- 172.21.15.5:0/926222105 learned_addr learned my addr 172.21.15.5:0/926222105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.859+0000 7fde4ffff640 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636887035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde4ffff640 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x563688769a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde5659c640 1 -- 172.21.15.5:0/926222105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 msgr2=0x563688769a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde5659c640 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x563688769a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde5659c640 1 -- 172.21.15.5:0/926222105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 msgr2=0x5636887035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde5659c640 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636887035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde5659c640 1 -- 172.21.15.5:0/926222105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563688704370 con 0x5636886e1880 2024-09-11T22:27:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde5659c640 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688703ae0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fde48008010 tx=0x7fde48024e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af35a8ed9af71e10 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde3dffb640 1 -- 172.21.15.5:0/926222105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48029070 con 0x5636886e1880 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde3dffb640 1 -- 172.21.15.5:0/926222105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fde480243d0 con 0x5636886e1880 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde5ef76740 1 -- 172.21.15.5:0/926222105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde48020030 con 0x5636886e1880 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde3dffb640 1 -- 172.21.15.5:0/926222105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fde480245d0 con 0x5636886e1880 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde5ef76740 10 client.16902 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.860+0000 7fde5ef76740 1 -- 172.21.15.5:0/926222105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56368876b070 con 0x5636886e1880 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.861+0000 7fde3dffb640 1 -- 172.21.15.5:0/926222105 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde48020030 con 0x5636886e1880 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.861+0000 7fde3dffb640 1 -- 172.21.15.5:0/926222105 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fde480133d0 con 0x5636886e1880 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.861+0000 7fde5ef76740 10 client.16902 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.861+0000 7fde5ef76740 10 client.16902 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.861+0000 7fde5ef76740 20 client.16902 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.861+0000 7fde5ef76740 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5636887000e0 0x56368876d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.861+0000 7fde5ef76740 4 client.16902 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5e46c553", "release": true, "await": true}] 2024-09-11T22:27:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.861+0000 7fde5ef76740 1 -- 172.21.15.5:0/926222105 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5e46c553", "release": true, "await": true}) -- 0x56368864e0a0 con 0x5636887000e0 2024-09-11T22:27:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.861+0000 7fde4ffff640 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5636887000e0 0x56368876d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.862+0000 7fde4ffff640 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5636887000e0 0x56368876d7a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c7e96d923c956aa server_cookie=7cbecc1db19c22e0 in_seq=0 out_seq=0 2024-09-11T22:27:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.862+0000 7fde3dffb640 10 client.16902 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.873+0000 7fde3dffb640 1 -- 172.21.15.5:0/926222105 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56368864e0a0 con 0x5636887000e0 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.873+0000 7fde3dffb640 10 client.16902 handle_command_reply: tid=0 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.873+0000 7fde5ef76740 1 client.16902 shutdown 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 1 -- 172.21.15.5:0/926222105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 msgr2=0x563688703ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688703ae0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fde48008010 tx=0x7fde48024e90 comp rx=0 tx=0).stop 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 1 -- 172.21.15.5:0/926222105 shutdown_connections 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5636887000e0 0x56368876d7a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5636886f0650 0x563688769a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636886e1880 0x563688703ae0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 1 --2- 172.21.15.5:0/926222105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636886e0f60 0x5636887035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 1 -- 172.21.15.5:0/926222105 >> 172.21.15.5:0/926222105 conn(0x5636886dd5f0 msgr2=0x5636886e2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde3dffb640 0 client.16902 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 1 -- 172.21.15.5:0/926222105 shutdown_connections 2024-09-11T22:27:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 1 -- 172.21.15.5:0/926222105 wait complete. 2024-09-11T22:27:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5ef76740 20 client.16902 trim_cache size 0 max 16384 2024-09-11T22:27:04.876 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:27:04.876 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 34, 2024-09-11T22:27:04.876 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:27:04.876 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 20, 2024-09-11T22:27:04.876 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "5e46c553": { 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 20, 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.2, 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:27:04.877 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:27:04.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:27:04.878 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:27:04.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:27:04.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:27:04.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:27:04.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:27:04.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:27:04.878 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5cb9d640 1 -- 172.21.15.5:0/872777899 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fde440a2cf0 msgr2=0x7fde440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.874+0000 7fde5cb9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fde440a2cf0 0x7fde440a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fde5009bff0 tx=0x7fde500914f0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.875+0000 7fde5cb9d640 1 -- 172.21.15.5:0/872777899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde5810c180 msgr2=0x7fde5819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.875+0000 7fde5cb9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde5810c180 0x7fde5819cd10 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fde5000cce0 tx=0x7fde5000a620 comp rx=0 tx=0).stop 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.875+0000 7fde5cb9d640 1 -- 172.21.15.5:0/872777899 shutdown_connections 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.875+0000 7fde5cb9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde5810c180 0x7fde5819cd10 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.875+0000 7fde5cb9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fde440a2cf0 0x7fde440a51b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.875+0000 7fde5cb9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde581036c0 0x7fde58198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.875+0000 7fde5cb9d640 1 --2- 172.21.15.5:0/872777899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde58102da0 0x7fde58198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.875+0000 7fde5cb9d640 1 -- 172.21.15.5:0/872777899 >> 172.21.15.5:0/872777899 conn(0x7fde580fc7a0 msgr2=0x7fde580fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.875+0000 7fde5cb9d640 1 -- 172.21.15.5:0/872777899 shutdown_connections 2024-09-11T22:27:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:04.875+0000 7fde5cb9d640 1 -- 172.21.15.5:0/872777899 wait complete. 2024-09-11T22:27:04.889 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:05.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323822 7.07 MB/sec execute 83 sec latency 21107.660 ms 2024-09-11T22:27:05.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:05 smithi005 ceph-mon[30943]: pgmap v150: 97 pgs: 97 active+clean; 44 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 269 KiB/s wr, 75 op/s 2024-09-11T22:27:05.735 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5e46c553', total seconds elapsed: 25.1 2024-09-11T22:27:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:05 smithi159 ceph-mon[35534]: pgmap v150: 97 pgs: 97 active+clean; 44 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 269 KiB/s wr, 75 op/s 2024-09-11T22:27:05.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:05 smithi172 ceph-mon[35461]: pgmap v150: 97 pgs: 97 active+clean; 44 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 269 KiB/s wr, 75 op/s 2024-09-11T22:27:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.280+0000 7efc27820640 1 -- 172.21.15.5:0/237444903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc201082a0 msgr2=0x7efc2010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.280+0000 7efc27820640 1 --2- 172.21.15.5:0/237444903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc201082a0 0x7efc2010ec40 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7efc18009c10 tx=0x7efc1802a6e0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.281+0000 7efc27820640 1 -- 172.21.15.5:0/237444903 shutdown_connections 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.281+0000 7efc27820640 1 --2- 172.21.15.5:0/237444903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc201082a0 0x7efc2010ec40 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.281+0000 7efc27820640 1 --2- 172.21.15.5:0/237444903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc201036e0 0x7efc20107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.281+0000 7efc27820640 1 --2- 172.21.15.5:0/237444903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc20102d30 0x7efc20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.281+0000 7efc27820640 1 -- 172.21.15.5:0/237444903 >> 172.21.15.5:0/237444903 conn(0x7efc200fc7d0 msgr2=0x7efc200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.281+0000 7efc27820640 1 -- 172.21.15.5:0/237444903 shutdown_connections 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.281+0000 7efc27820640 1 -- 172.21.15.5:0/237444903 wait complete. 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.281+0000 7efc27820640 1 Processor -- start 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc27820640 1 -- start start 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc27820640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc20102d30 0x7efc20100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc27820640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc201036e0 0x7efc20100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc27820640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc201082a0 0x7efc20101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc27820640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc20112050 con 0x7efc201036e0 2024-09-11T22:27:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc27820640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efc20111ed0 con 0x7efc20102d30 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc27820640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efc201121d0 con 0x7efc201082a0 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc24dbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc201036e0 0x7efc20100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc255bc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc20102d30 0x7efc20100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc25dbd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc201082a0 0x7efc20101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc255bc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc20102d30 0x7efc20100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58284/0 (socket says 172.21.15.5:58284) 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc24dbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc201036e0 0x7efc20100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60372/0 (socket says 172.21.15.5:60372) 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc255bc640 1 -- 172.21.15.5:0/4034757115 learned_addr learned my addr 172.21.15.5:0/4034757115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc255bc640 1 -- 172.21.15.5:0/4034757115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc201082a0 msgr2=0x7efc20101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc255bc640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc201082a0 0x7efc20101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc255bc640 1 -- 172.21.15.5:0/4034757115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc201036e0 msgr2=0x7efc20100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc255bc640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc201036e0 0x7efc20100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc255bc640 1 -- 172.21.15.5:0/4034757115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc201a6a20 con 0x7efc20102d30 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.282+0000 7efc24dbb640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc201036e0 0x7efc20100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.283+0000 7efc255bc640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc20102d30 0x7efc20100990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e9f0 tx=0x7efc1c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.283+0000 7efc167fc640 1 -- 172.21.15.5:0/4034757115 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00cd80 con 0x7efc20102d30 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.283+0000 7efc27820640 1 -- 172.21.15.5:0/4034757115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc201a6d10 con 0x7efc20102d30 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.283+0000 7efc167fc640 1 -- 172.21.15.5:0/4034757115 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c005500 con 0x7efc20102d30 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.283+0000 7efc167fc640 1 -- 172.21.15.5:0/4034757115 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c010690 con 0x7efc20102d30 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.283+0000 7efc27820640 1 -- 172.21.15.5:0/4034757115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc201a71f0 con 0x7efc20102d30 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.284+0000 7efc27820640 1 -- 172.21.15.5:0/4034757115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc20103110 con 0x7efc20102d30 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.285+0000 7efc167fc640 1 -- 172.21.15.5:0/4034757115 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7efc1c0046c0 con 0x7efc20102d30 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.286+0000 7efc167fc640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efc0c0a2cf0 0x7efc0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.286+0000 7efc167fc640 1 -- 172.21.15.5:0/4034757115 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7efc1c0cfa40 con 0x7efc20102d30 2024-09-11T22:27:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.286+0000 7efc24dbb640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efc0c0a2cf0 0x7efc0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.287+0000 7efc24dbb640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efc0c0a2cf0 0x7efc0c0a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc100096c0 tx=0x7efc10009260 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.289+0000 7efc167fc640 1 -- 172.21.15.5:0/4034757115 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc1c0541b0 con 0x7efc20102d30 2024-09-11T22:27:06.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 326432 7.18 MB/sec execute 84 sec latency 52.879 ms 2024-09-11T22:27:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.518+0000 7efc27820640 1 -- 172.21.15.5:0/4034757115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc20101ec0 con 0x7efc20102d30 2024-09-11T22:27:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.518+0000 7efc167fc640 1 -- 172.21.15.5:0/4034757115 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11967 (secure 0 0 0) 0x7efc20101ec0 con 0x7efc20102d30 2024-09-11T22:27:06.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:27:06.520 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-11T22:25:49:149525+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:25:48.784660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25970":{"gid":25970,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4169170564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4169170564},{"type":"v1","addr":"172.21.15.159:6833","nonce":4169170564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:27:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-11T22:27:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 -- 172.21.15.5:0/4034757115 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efc0c0a2cf0 msgr2=0x7efc0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efc0c0a2cf0 0x7efc0c0a51b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc100096c0 tx=0x7efc10009260 comp rx=0 tx=0).stop 2024-09-11T22:27:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 -- 172.21.15.5:0/4034757115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc20102d30 msgr2=0x7efc20100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc20102d30 0x7efc20100990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e9f0 tx=0x7efc1c00edf0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 -- 172.21.15.5:0/4034757115 shutdown_connections 2024-09-11T22:27:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc201082a0 0x7efc20101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc201036e0 0x7efc20100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efc0c0a2cf0 0x7efc0c0a51b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 --2- 172.21.15.5:0/4034757115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc20102d30 0x7efc20100990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 -- 172.21.15.5:0/4034757115 >> 172.21.15.5:0/4034757115 conn(0x7efc200fc7d0 msgr2=0x7efc20105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 -- 172.21.15.5:0/4034757115 shutdown_connections 2024-09-11T22:27:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:06.521+0000 7efc27820640 1 -- 172.21.15.5:0/4034757115 wait complete. 2024-09-11T22:27:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.543+0000 7f6340a2e640 1 -- 172.21.15.5:0/2998370685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f633c101390 msgr2=0x7f633c10d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.543+0000 7f6340a2e640 1 --2- 172.21.15.5:0/2998370685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f633c101390 0x7f633c10d8c0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f6334009cd0 tx=0x7f633402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.543+0000 7f6340a2e640 1 -- 172.21.15.5:0/2998370685 shutdown_connections 2024-09-11T22:27:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.543+0000 7f6340a2e640 1 --2- 172.21.15.5:0/2998370685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f633c101390 0x7f633c10d8c0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.543+0000 7f6340a2e640 1 --2- 172.21.15.5:0/2998370685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f633c1009f0 0x7f633c100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.543+0000 7f6340a2e640 1 --2- 172.21.15.5:0/2998370685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633c106a00 0x7f633c106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.543+0000 7f6340a2e640 1 -- 172.21.15.5:0/2998370685 >> 172.21.15.5:0/2998370685 conn(0x7f633c0fc6b0 msgr2=0x7f633c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.543+0000 7f6340a2e640 1 -- 172.21.15.5:0/2998370685 shutdown_connections 2024-09-11T22:27:06.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.543+0000 7f6340a2e640 1 -- 172.21.15.5:0/2998370685 wait complete. 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f6340a2e640 1 Processor -- start 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f6340a2e640 1 -- start start 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f6340a2e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633c1009f0 0x7f633c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f6340a2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f633c101390 0x7f633c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f6340a2e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f633c106a00 0x7f633c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f6340a2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f633c110e40 con 0x7f633c101390 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f6340a2e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f633c110cc0 con 0x7f633c106a00 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f6340a2e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f633c110fc0 con 0x7f633c1009f0 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f633a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633c1009f0 0x7f633c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f6339d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f633c101390 0x7f633c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f633ad9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f633c106a00 0x7f633c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f633a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633c1009f0 0x7f633c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:40278/0 (socket says 172.21.15.5:40278) 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.544+0000 7f633a59c640 1 -- 172.21.15.5:0/976514307 learned_addr learned my addr 172.21.15.5:0/976514307 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f633a59c640 1 -- 172.21.15.5:0/976514307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f633c106a00 msgr2=0x7f633c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f633a59c640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f633c106a00 0x7f633c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f633a59c640 1 -- 172.21.15.5:0/976514307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f633c101390 msgr2=0x7f633c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f633a59c640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f633c101390 0x7f633c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f633a59c640 1 -- 172.21.15.5:0/976514307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633c19d350 con 0x7f633c1009f0 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f6339d9b640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f633c101390 0x7f633c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f633a59c640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633c1009f0 0x7f633c198360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e9f0 tx=0x7f632c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f63337fe640 1 -- 172.21.15.5:0/976514307 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c00cd80 con 0x7f633c1009f0 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f63337fe640 1 -- 172.21.15.5:0/976514307 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f632c005500 con 0x7f633c1009f0 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f6340a2e640 1 -- 172.21.15.5:0/976514307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633c19d640 con 0x7f633c1009f0 2024-09-11T22:27:06.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.545+0000 7f63337fe640 1 -- 172.21.15.5:0/976514307 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c010690 con 0x7f633c1009f0 2024-09-11T22:27:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.546+0000 7f6340a2e640 1 -- 172.21.15.5:0/976514307 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633c1a4ed0 con 0x7f633c1009f0 2024-09-11T22:27:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.549+0000 7f63337fe640 1 -- 172.21.15.5:0/976514307 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f632c0046c0 con 0x7f633c1009f0 2024-09-11T22:27:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.549+0000 7f63437cb740 1 Processor -- start 2024-09-11T22:27:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.549+0000 7f63437cb740 1 -- start start 2024-09-11T22:27:06.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.549+0000 7f63437cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702748af0 0x559702748ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.549+0000 7f63437cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702749410 0x559702753770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f63437cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x5597027560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f63437cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5597026ad2f0 con 0x559702749410 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f63437cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559702655270 con 0x559702748af0 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f63437cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5597026a92b0 con 0x559702753cb0 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f6339d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x5597027560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f6339d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x5597027560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:40292/0 (socket says 172.21.15.5:40292) 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f6339d9b640 1 -- 172.21.15.5:0/1776942877 learned_addr learned my addr 172.21.15.5:0/1776942877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f633a59c640 1 --2- 172.21.15.5:0/1776942877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702749410 0x559702753770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f633ad9d640 1 --2- 172.21.15.5:0/1776942877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702748af0 0x559702748ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f6339d9b640 1 -- 172.21.15.5:0/1776942877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702748af0 msgr2=0x559702748ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f6339d9b640 1 --2- 172.21.15.5:0/1776942877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702748af0 0x559702748ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f6339d9b640 1 -- 172.21.15.5:0/1776942877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702749410 msgr2=0x559702753770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:06.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f6339d9b640 1 --2- 172.21.15.5:0/1776942877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702749410 0x559702753770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f6339d9b640 1 -- 172.21.15.5:0/1776942877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559702654340 con 0x559702753cb0 2024-09-11T22:27:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f63337fe640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63280a2e20 0x7f63280a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.550+0000 7f6339d9b640 1 --2- 172.21.15.5:0/1776942877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x5597027560a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6324009c80 tx=0x7f632400e780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3613240aecfac6f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.551+0000 7f633ad9d640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63280a2e20 0x7f63280a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.551+0000 7f6330ff9640 1 -- 172.21.15.5:0/1776942877 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324009d70 con 0x559702753cb0 2024-09-11T22:27:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.551+0000 7f63337fe640 1 -- 172.21.15.5:0/976514307 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f632c0cf7f0 con 0x7f633c1009f0 2024-09-11T22:27:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.551+0000 7f6330ff9640 1 -- 172.21.15.5:0/1776942877 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63240035f0 con 0x559702753cb0 2024-09-11T22:27:06.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.551+0000 7f6330ff9640 1 -- 172.21.15.5:0/1776942877 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324020660 con 0x559702753cb0 2024-09-11T22:27:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.551+0000 7f63437cb740 1 -- 172.21.15.5:0/1776942877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 msgr2=0x5597027560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.551+0000 7f63437cb740 1 --2- 172.21.15.5:0/1776942877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x5597027560a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6324009c80 tx=0x7f632400e780 comp rx=0 tx=0).stop 2024-09-11T22:27:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.551+0000 7f633ad9d640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63280a2e20 0x7f63280a52e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6334004ab0 tx=0x7f6334004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.552+0000 7f63437cb740 1 -- 172.21.15.5:0/1776942877 shutdown_connections 2024-09-11T22:27:06.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.552+0000 7f63437cb740 1 --2- 172.21.15.5:0/1776942877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x5597027560a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.552+0000 7f63437cb740 1 --2- 172.21.15.5:0/1776942877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702749410 0x559702753770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.552+0000 7f63437cb740 1 --2- 172.21.15.5:0/1776942877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702748af0 0x559702748ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.552+0000 7f63437cb740 1 -- 172.21.15.5:0/1776942877 >> 172.21.15.5:0/1776942877 conn(0x559702745060 msgr2=0x559702745440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.552+0000 7f63437cb740 1 -- 172.21.15.5:0/1776942877 shutdown_connections 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.552+0000 7f63437cb740 1 -- 172.21.15.5:0/1776942877 wait complete. 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.552+0000 7f63437cb740 1 -- start start 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.552+0000 7f63437cb740 1 Processor -- start 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f63437cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702748af0 0x55970274cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f63437cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702749410 0x55970274d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f63437cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702753cb0 0x559702751660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f63437cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559702756820 con 0x559702753cb0 2024-09-11T22:27:06.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f63437cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559702636290 con 0x559702748af0 2024-09-11T22:27:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f63437cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5597027569a0 con 0x559702749410 2024-09-11T22:27:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f6339d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702749410 0x55970274d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f6339d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702749410 0x55970274d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:40304/0 (socket says 172.21.15.5:40304) 2024-09-11T22:27:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f6339d9b640 1 -- 172.21.15.5:0/906063268 learned_addr learned my addr 172.21.15.5:0/906063268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f633a59c640 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702748af0 0x55970274cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f633ad9d640 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702753cb0 0x559702751660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f63437cb740 10 client.0 _ll_register_callbacks cb 0x559702667140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f6339d9b640 1 -- 172.21.15.5:0/906063268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702748af0 msgr2=0x55970274cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:06.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f6339d9b640 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702748af0 0x55970274cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f6339d9b640 1 -- 172.21.15.5:0/906063268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702753cb0 msgr2=0x559702751660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f6339d9b640 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702753cb0 0x559702751660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f6339d9b640 1 -- 172.21.15.5:0/906063268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597023c0210 con 0x559702749410 2024-09-11T22:27:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f6339d9b640 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702749410 0x55970274d310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6324000c00 tx=0x7f6324003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fc57691fac2c782 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.553+0000 7f6321ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:27:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f6321ffb640 1 -- 172.21.15.5:0/906063268 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324009d70 con 0x559702749410 2024-09-11T22:27:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f63437cb740 1 -- 172.21.15.5:0/906063268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63240035f0 con 0x559702749410 2024-09-11T22:27:06.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f6321ffb640 1 -- 172.21.15.5:0/906063268 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f6324005050 con 0x559702749410 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f6321ffb640 1 -- 172.21.15.5:0/906063268 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6324038050 con 0x559702749410 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f63437cb740 10 client.26311 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f63437cb740 1 -- 172.21.15.5:0/906063268 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55970248c620 con 0x559702749410 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f6321ffb640 1 -- 172.21.15.5:0/906063268 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63240035f0 con 0x559702749410 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f6321ffb640 1 -- 172.21.15.5:0/906063268 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f6324004ad0 con 0x559702749410 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f63437cb740 10 client.26311 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f63437cb740 10 client.26311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f63437cb740 20 client.26311 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f63437cb740 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559702763fb0 0x559702766430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f63437cb740 4 client.26311 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f63437cb740 1 -- 172.21.15.5:0/906063268 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597026baa20 con 0x559702763fb0 2024-09-11T22:27:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.554+0000 7f633a59c640 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559702763fb0 0x559702766430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.555+0000 7f633a59c640 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559702763fb0 0x559702766430 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cae02bf95b5cd0b server_cookie=a85e77dc02a6f37d in_seq=0 out_seq=0 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.555+0000 7f6321ffb640 10 client.26311 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f6321ffb640 1 -- 172.21.15.5:0/906063268 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597026baa20 con 0x559702763fb0 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f6321ffb640 10 client.26311 handle_command_reply: tid=0 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 client.26311 shutdown 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 -- 172.21.15.5:0/906063268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702749410 msgr2=0x55970274d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702749410 0x55970274d310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6324000c00 tx=0x7f6324003c60 comp rx=0 tx=0).stop 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 -- 172.21.15.5:0/906063268 shutdown_connections 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559702763fb0 0x559702766430 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702753cb0 0x559702751660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702749410 0x55970274d310 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 --2- 172.21.15.5:0/906063268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702748af0 0x55970274cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 -- 172.21.15.5:0/906063268 >> 172.21.15.5:0/906063268 conn(0x559702745060 msgr2=0x55970274adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f6321ffb640 0 client.26311 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 -- 172.21.15.5:0/906063268 shutdown_connections 2024-09-11T22:27:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 1 -- 172.21.15.5:0/906063268 wait complete. 2024-09-11T22:27:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.556+0000 7f63437cb740 20 client.26311 trim_cache size 0 max 16384 2024-09-11T22:27:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.572+0000 7f63437cb740 1 Processor -- start 2024-09-11T22:27:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.572+0000 7f63437cb740 1 -- start start 2024-09-11T22:27:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f63437cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702751650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f63437cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 0x5597027440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f63437cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x55970276bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f63437cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1ab830 con 0x559702748af0 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f63437cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1ad370 con 0x559702749410 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f63437cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1abf00 con 0x559702753cb0 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702751650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702751650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60410/0 (socket says 172.21.15.5:60410) 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633a59c640 1 -- 172.21.15.5:0/2174526502 learned_addr learned my addr 172.21.15.5:0/2174526502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f6339d9b640 1 --2- 172.21.15.5:0/2174526502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 0x5597027440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633ad9d640 1 --2- 172.21.15.5:0/2174526502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x55970276bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633a59c640 1 -- 172.21.15.5:0/2174526502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 msgr2=0x55970276bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633a59c640 1 --2- 172.21.15.5:0/2174526502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x55970276bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633a59c640 1 -- 172.21.15.5:0/2174526502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 msgr2=0x5597027440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633a59c640 1 --2- 172.21.15.5:0/2174526502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 0x5597027440b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633a59c640 1 -- 172.21.15.5:0/2174526502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559702749240 con 0x559702748af0 2024-09-11T22:27:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633ad9d640 1 --2- 172.21.15.5:0/2174526502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x55970276bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f633a59c640 1 --2- 172.21.15.5:0/2174526502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702751650 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f632c09b970 tx=0x7f632c0d1c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9019d13cdd8b11fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.573+0000 7f6323fff640 1 -- 172.21.15.5:0/2174526502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c09b7b0 con 0x559702748af0 2024-09-11T22:27:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f6323fff640 1 -- 172.21.15.5:0/2174526502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f632c09c370 con 0x559702748af0 2024-09-11T22:27:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f6323fff640 1 -- 172.21.15.5:0/2174526502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c09b5d0 con 0x559702748af0 2024-09-11T22:27:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f63437cb740 1 -- 172.21.15.5:0/2174526502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 msgr2=0x559702751650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f63437cb740 1 --2- 172.21.15.5:0/2174526502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702751650 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f632c09b970 tx=0x7f632c0d1c40 comp rx=0 tx=0).stop 2024-09-11T22:27:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f63437cb740 1 -- 172.21.15.5:0/2174526502 shutdown_connections 2024-09-11T22:27:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f63437cb740 1 --2- 172.21.15.5:0/2174526502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x55970276bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f63437cb740 1 --2- 172.21.15.5:0/2174526502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 0x5597027440b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f63437cb740 1 --2- 172.21.15.5:0/2174526502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702751650 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f63437cb740 1 -- 172.21.15.5:0/2174526502 >> 172.21.15.5:0/2174526502 conn(0x559702745060 msgr2=0x5597027439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f63437cb740 1 -- 172.21.15.5:0/2174526502 shutdown_connections 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.574+0000 7f63437cb740 1 -- 172.21.15.5:0/2174526502 wait complete. 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f63437cb740 1 -- start start 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f63437cb740 1 Processor -- start 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f63437cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702760760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f63437cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 0x55970275ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f63437cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x55970275b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f63437cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1acca0 con 0x559702748af0 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f63437cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1ac5d0 con 0x559702749410 2024-09-11T22:27:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f63437cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f633c1a9810 con 0x559702753cb0 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f63437cb740 10 client.0 _ll_register_callbacks cb 0x7f632c0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f633a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702760760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f633a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702760760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60418/0 (socket says 172.21.15.5:60418) 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f6339d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 0x55970275ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f633a59c640 1 -- 172.21.15.5:0/3455935443 learned_addr learned my addr 172.21.15.5:0/3455935443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f6339d9b640 1 -- 172.21.15.5:0/3455935443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 msgr2=0x55970275b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f633ad9d640 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x55970275b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f6339d9b640 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x55970275b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f6339d9b640 1 -- 172.21.15.5:0/3455935443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 msgr2=0x559702760760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f6339d9b640 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702760760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f6339d9b640 1 -- 172.21.15.5:0/3455935443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55970275bae0 con 0x559702749410 2024-09-11T22:27:06.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f633a59c640 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702760760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.575+0000 7f633ad9d640 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x55970275b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.576+0000 7f6339d9b640 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 0x55970275ad10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6324004f20 tx=0x7f6324005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd72385db5e03b48 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.576+0000 7f6321ffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.576+0000 7f6321ffb640 1 -- 172.21.15.5:0/3455935443 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63240041c0 con 0x559702749410 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.576+0000 7f6321ffb640 1 -- 172.21.15.5:0/3455935443 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f632402aa20 con 0x559702749410 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.576+0000 7f63437cb740 1 -- 172.21.15.5:0/3455935443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f632c09c370 con 0x559702749410 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.576+0000 7f6321ffb640 1 -- 172.21.15.5:0/3455935443 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6324049050 con 0x559702749410 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.576+0000 7f63437cb740 10 client.26384 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.576+0000 7f63437cb740 1 -- 172.21.15.5:0/3455935443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5597027cdd10 con 0x559702749410 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.576+0000 7f6321ffb640 1 -- 172.21.15.5:0/3455935443 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f632c09c370 con 0x559702749410 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.577+0000 7f6321ffb640 1 -- 172.21.15.5:0/3455935443 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f6324005480 con 0x559702749410 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.577+0000 7f63437cb740 10 client.26384 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:06.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.577+0000 7f63437cb740 10 client.26384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.577+0000 7f63437cb740 20 client.26384 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.577+0000 7f63437cb740 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5597027ce400 0x5597027d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.577+0000 7f63437cb740 4 client.26384 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:27:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.577+0000 7f63437cb740 1 -- 172.21.15.5:0/3455935443 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597026b4d70 con 0x5597027ce400 2024-09-11T22:27:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.577+0000 7f633a59c640 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5597027ce400 0x5597027d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.577+0000 7f633a59c640 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5597027ce400 0x5597027d0820 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a56cc4a7fa158770 server_cookie=933d94f71309d6f9 in_seq=0 out_seq=0 2024-09-11T22:27:06.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.577+0000 7f6321ffb640 10 client.26384 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f6321ffb640 1 -- 172.21.15.5:0/3455935443 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5597026b4d70 con 0x5597027ce400 2024-09-11T22:27:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f6321ffb640 10 client.26384 handle_command_reply: tid=0 2024-09-11T22:27:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 client.26384 shutdown 2024-09-11T22:27:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 -- 172.21.15.5:0/3455935443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 msgr2=0x55970275ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 0x55970275ad10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6324004f20 tx=0x7f6324005da0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 -- 172.21.15.5:0/3455935443 shutdown_connections 2024-09-11T22:27:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5597027ce400 0x5597027d0820 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559702753cb0 0x55970275b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559702749410 0x55970275ad10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f6321ffb640 0 client.26384 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 --2- 172.21.15.5:0/3455935443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559702748af0 0x559702760760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 -- 172.21.15.5:0/3455935443 >> 172.21.15.5:0/3455935443 conn(0x559702745060 msgr2=0x55970274c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 -- 172.21.15.5:0/3455935443 shutdown_connections 2024-09-11T22:27:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 1 -- 172.21.15.5:0/3455935443 wait complete. 2024-09-11T22:27:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.634+0000 7f63437cb740 20 client.26384 trim_cache size 0 max 16384 2024-09-11T22:27:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:27:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:27:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:27:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:27:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 -- 172.21.15.5:0/976514307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63280a2e20 msgr2=0x7f63280a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63280a2e20 0x7f63280a52e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6334004ab0 tx=0x7f6334004a40 comp rx=0 tx=0).stop 2024-09-11T22:27:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 -- 172.21.15.5:0/976514307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633c1009f0 msgr2=0x7f633c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633c1009f0 0x7f633c198360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f632c00e9f0 tx=0x7f632c00edf0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 -- 172.21.15.5:0/976514307 shutdown_connections 2024-09-11T22:27:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f63280a2e20 0x7f63280a52e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f633c106a00 0x7f633c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f633c101390 0x7f633c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 --2- 172.21.15.5:0/976514307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f633c1009f0 0x7f633c198360 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 -- 172.21.15.5:0/976514307 >> 172.21.15.5:0/976514307 conn(0x7f633c0fc6b0 msgr2=0x7f633c104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 -- 172.21.15.5:0/976514307 shutdown_connections 2024-09-11T22:27:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:06.635+0000 7f6340a2e640 1 -- 172.21.15.5:0/976514307 wait complete. 2024-09-11T22:27:07.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 328909 7.21 MB/sec execute 85 sec latency 15.637 ms 2024-09-11T22:27:07.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:27:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:07.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:07 smithi005 ceph-mon[30943]: pgmap v151: 97 pgs: 97 active+clean; 44 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 269 KiB/s wr, 75 op/s 2024-09-11T22:27:07.755 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:07 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4034757115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:27:07.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:07 smithi159 ceph-mon[35534]: pgmap v151: 97 pgs: 97 active+clean; 44 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 269 KiB/s wr, 75 op/s 2024-09-11T22:27:07.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:07 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4034757115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:27:07.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:07 smithi172 ceph-mon[35461]: pgmap v151: 97 pgs: 97 active+clean; 44 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 269 KiB/s wr, 75 op/s 2024-09-11T22:27:07.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:07 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4034757115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:27:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.279+0000 7f812fed1640 1 -- 172.21.15.5:0/2250888586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8128101de0 msgr2=0x7f81281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.279+0000 7f812fed1640 1 --2- 172.21.15.5:0/2250888586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8128101de0 0x7f81281021c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8124009900 tx=0x7f812402a410 comp rx=0 tx=0).stop 2024-09-11T22:27:08.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.279+0000 7f812fed1640 1 -- 172.21.15.5:0/2250888586 shutdown_connections 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.279+0000 7f812fed1640 1 --2- 172.21.15.5:0/2250888586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8128106f10 0x7f812810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.279+0000 7f812fed1640 1 --2- 172.21.15.5:0/2250888586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8128102790 0x7f81281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.279+0000 7f812fed1640 1 --2- 172.21.15.5:0/2250888586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8128101de0 0x7f81281021c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.279+0000 7f812fed1640 1 -- 172.21.15.5:0/2250888586 >> 172.21.15.5:0/2250888586 conn(0x7f81280fd8c0 msgr2=0x7f81280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.279+0000 7f812fed1640 1 -- 172.21.15.5:0/2250888586 shutdown_connections 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.279+0000 7f812fed1640 1 -- 172.21.15.5:0/2250888586 wait complete. 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812fed1640 1 Processor -- start 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812fed1640 1 -- start start 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812fed1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8128101de0 0x7f81281983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812fed1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8128102790 0x7f8128198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812fed1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8128106f10 0x7f812819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812fed1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8128111f70 con 0x7f8128101de0 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812fed1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8128111df0 con 0x7f8128106f10 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812fed1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81281120f0 con 0x7f8128102790 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812dc6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8128101de0 0x7f81281983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812dc6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8128101de0 0x7f81281983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60430/0 (socket says 172.21.15.5:60430) 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.280+0000 7f812dc6d640 1 -- 172.21.15.5:0/4113661532 learned_addr learned my addr 172.21.15.5:0/4113661532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f812dc6d640 1 -- 172.21.15.5:0/4113661532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8128102790 msgr2=0x7f8128198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f812dc6d640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8128102790 0x7f8128198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f812dc6d640 1 -- 172.21.15.5:0/4113661532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8128106f10 msgr2=0x7f812819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f812dc6d640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8128106f10 0x7f812819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f812dc6d640 1 -- 172.21.15.5:0/4113661532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812819d380 con 0x7f8128101de0 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f812dc6d640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8128101de0 0x7f81281983d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8124006010 tx=0x7f81240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f811effd640 1 -- 172.21.15.5:0/4113661532 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812403f030 con 0x7f8128101de0 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f811effd640 1 -- 172.21.15.5:0/4113661532 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8124003ce0 con 0x7f8128101de0 2024-09-11T22:27:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f811effd640 1 -- 172.21.15.5:0/4113661532 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124035070 con 0x7f8128101de0 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f812fed1640 1 -- 172.21.15.5:0/4113661532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812819d610 con 0x7f8128101de0 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.281+0000 7f812fed1640 1 -- 172.21.15.5:0/4113661532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81281a4f50 con 0x7f8128101de0 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.283+0000 7f811effd640 1 -- 172.21.15.5:0/4113661532 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8124042030 con 0x7f8128101de0 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f81322aa740 1 Processor -- start 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f81322aa740 1 -- start start 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f81322aa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a4ab40 0x557392a49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f81322aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a49a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f81322aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a54370 0x557392a56760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f81322aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573929af2f0 con 0x557392a495d0 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f81322aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557392956cf0 con 0x557392a54370 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f81322aa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5573929ab2b0 con 0x557392a4ab40 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f811effd640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81140a2d70 0x7f81140a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f812dc6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a49a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f812dc6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a49a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60446/0 (socket says 172.21.15.5:60446) 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f812dc6d640 1 -- 172.21.15.5:0/2431947544 learned_addr learned my addr 172.21.15.5:0/2431947544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.284+0000 7f812e46e640 1 --2- 172.21.15.5:0/2431947544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a4ab40 0x557392a49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f812d46c640 1 --2- 172.21.15.5:0/2431947544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a54370 0x557392a56760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f812e46e640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81140a2d70 0x7f81140a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f811effd640 1 -- 172.21.15.5:0/4113661532 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f81240ef6f0 con 0x7f8128101de0 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f812dc6d640 1 -- 172.21.15.5:0/2431947544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a4ab40 msgr2=0x557392a49090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f812dc6d640 1 --2- 172.21.15.5:0/2431947544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a4ab40 0x557392a49090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f812dc6d640 1 -- 172.21.15.5:0/2431947544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a54370 msgr2=0x557392a56760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f812dc6d640 1 --2- 172.21.15.5:0/2431947544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a54370 0x557392a56760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f812dc6d640 1 -- 172.21.15.5:0/2431947544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55739293b3a0 con 0x557392a495d0 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f812d46c640 1 --2- 172.21.15.5:0/2431947544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a54370 0x557392a56760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:08.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f812dc6d640 1 --2- 172.21.15.5:0/2431947544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a49a10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c00 tx=0x7f812404b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74b32c156b82794 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f812e46e640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81140a2d70 0x7f81140a5230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8120013a40 tx=0x7f8120005e90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f8113fff640 1 -- 172.21.15.5:0/2431947544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124049070 con 0x557392a495d0 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f8113fff640 1 -- 172.21.15.5:0/2431947544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812403b580 con 0x557392a495d0 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.285+0000 7f8113fff640 1 -- 172.21.15.5:0/2431947544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240a6030 con 0x557392a495d0 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.286+0000 7f81322aa740 1 -- 172.21.15.5:0/2431947544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 msgr2=0x557392a49a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.286+0000 7f81322aa740 1 --2- 172.21.15.5:0/2431947544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a49a10 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c00 tx=0x7f812404b000 comp rx=0 tx=0).stop 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.286+0000 7f81322aa740 1 -- 172.21.15.5:0/2431947544 shutdown_connections 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.286+0000 7f81322aa740 1 --2- 172.21.15.5:0/2431947544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a54370 0x557392a56760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.286+0000 7f81322aa740 1 --2- 172.21.15.5:0/2431947544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a49a10 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.286+0000 7f81322aa740 1 --2- 172.21.15.5:0/2431947544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a4ab40 0x557392a49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.286+0000 7f81322aa740 1 -- 172.21.15.5:0/2431947544 >> 172.21.15.5:0/2431947544 conn(0x557392a12310 msgr2=0x557392a45c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.286+0000 7f81322aa740 1 -- 172.21.15.5:0/2431947544 shutdown_connections 2024-09-11T22:27:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.286+0000 7f81322aa740 1 -- 172.21.15.5:0/2431947544 wait complete. 2024-09-11T22:27:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.286+0000 7f81322aa740 1 -- start start 2024-09-11T22:27:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.287+0000 7f81322aa740 1 Processor -- start 2024-09-11T22:27:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f81322aa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a495d0 0x557392a53de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f81322aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a51090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f81322aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a54370 0x557392a517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f81322aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573929af2f0 con 0x557392a54370 2024-09-11T22:27:08.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f81322aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55739296a060 con 0x557392a4ab40 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f81322aa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557392a56e30 con 0x557392a495d0 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f812dc6d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a495d0 0x557392a53de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f812d46c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a51090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f812d46c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a51090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58380/0 (socket says 172.21.15.5:58380) 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f812d46c640 1 -- 172.21.15.5:0/3183978183 learned_addr learned my addr 172.21.15.5:0/3183978183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f81322aa740 10 client.0 _ll_register_callbacks cb 0x5573926bd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f812e46e640 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a54370 0x557392a517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f812d46c640 1 -- 172.21.15.5:0/3183978183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a495d0 msgr2=0x557392a53de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f812d46c640 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a495d0 0x557392a53de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.288+0000 7f812d46c640 1 -- 172.21.15.5:0/3183978183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a54370 msgr2=0x557392a517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f812d46c640 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a54370 0x557392a517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f812d46c640 1 -- 172.21.15.5:0/3183978183 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573927203c0 con 0x557392a4ab40 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f812e46e640 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a54370 0x557392a517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f812d46c640 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a51090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8118009760 tx=0x7f8118009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=820fbbc3a5a4255 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f81117fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f81117fa640 1 -- 172.21.15.5:0/3183978183 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118008ba0 con 0x557392a4ab40 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f81117fa640 1 -- 172.21.15.5:0/3183978183 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f8118008d40 con 0x557392a4ab40 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f81322aa740 1 -- 172.21.15.5:0/3183978183 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5573929bc860 con 0x557392a4ab40 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f81117fa640 1 -- 172.21.15.5:0/3183978183 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8118005020 con 0x557392a4ab40 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f81322aa740 10 client.26390 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.289+0000 7f81322aa740 1 -- 172.21.15.5:0/3183978183 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557392a62820 con 0x557392a4ab40 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.290+0000 7f81117fa640 1 -- 172.21.15.5:0/3183978183 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5573929bc860 con 0x557392a4ab40 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.290+0000 7f81117fa640 1 -- 172.21.15.5:0/3183978183 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f8118004cc0 con 0x557392a4ab40 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.290+0000 7f81322aa740 10 client.26390 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.290+0000 7f81322aa740 10 client.26390 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:27:08.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.290+0000 7f81322aa740 20 client.26390 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.290+0000 7f81322aa740 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x557392a63cd0 0x557392a66120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.290+0000 7f81322aa740 4 client.26390 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.290+0000 7f81322aa740 1 -- 172.21.15.5:0/3183978183 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573929b6d70 con 0x557392a63cd0 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.290+0000 7f812dc6d640 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x557392a63cd0 0x557392a66120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.291+0000 7f81117fa640 10 client.26390 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.291+0000 7f812dc6d640 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x557392a63cd0 0x557392a66120 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d3ad4760dbc7a94 server_cookie=c7791eeba3d8cfac in_seq=0 out_seq=0 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81117fa640 1 -- 172.21.15.5:0/3183978183 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573929b6d70 con 0x557392a63cd0 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81117fa640 10 client.26390 handle_command_reply: tid=0 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 client.26390 shutdown 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 -- 172.21.15.5:0/3183978183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 msgr2=0x557392a51090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a51090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8118009760 tx=0x7f8118009b60 comp rx=0 tx=0).stop 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 -- 172.21.15.5:0/3183978183 shutdown_connections 2024-09-11T22:27:08.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x557392a63cd0 0x557392a66120 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a54370 0x557392a517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a51090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81117fa640 0 client.26390 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:27:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 --2- 172.21.15.5:0/3183978183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a495d0 0x557392a53de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 -- 172.21.15.5:0/3183978183 >> 172.21.15.5:0/3183978183 conn(0x557392a12310 msgr2=0x557392a4b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 -- 172.21.15.5:0/3183978183 shutdown_connections 2024-09-11T22:27:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 1 -- 172.21.15.5:0/3183978183 wait complete. 2024-09-11T22:27:08.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.292+0000 7f81322aa740 20 client.26390 trim_cache size 0 max 16384 2024-09-11T22:27:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.307+0000 7f81322aa740 1 Processor -- start 2024-09-11T22:27:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.307+0000 7f81322aa740 1 -- start start 2024-09-11T22:27:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.307+0000 7f81322aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.307+0000 7f81322aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a4c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.307+0000 7f81322aa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 0x557392a4c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.307+0000 7f81322aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81281ab800 con 0x557392a495d0 2024-09-11T22:27:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.307+0000 7f81322aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81281ad340 con 0x557392a4ab40 2024-09-11T22:27:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.307+0000 7f81322aa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81281abed0 con 0x557392a54370 2024-09-11T22:27:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812dc6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812d46c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a4c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812dc6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60474/0 (socket says 172.21.15.5:60474) 2024-09-11T22:27:08.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812d46c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a4c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58390/0 (socket says 172.21.15.5:58390) 2024-09-11T22:27:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812dc6d640 1 -- 172.21.15.5:0/1735505063 learned_addr learned my addr 172.21.15.5:0/1735505063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812e46e640 1 --2- 172.21.15.5:0/1735505063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 0x557392a4c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812dc6d640 1 -- 172.21.15.5:0/1735505063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 msgr2=0x557392a4c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812dc6d640 1 --2- 172.21.15.5:0/1735505063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 0x557392a4c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812dc6d640 1 -- 172.21.15.5:0/1735505063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 msgr2=0x557392a4c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812dc6d640 1 --2- 172.21.15.5:0/1735505063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a4c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812dc6d640 1 -- 172.21.15.5:0/1735505063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557392a46960 con 0x557392a495d0 2024-09-11T22:27:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812d46c640 1 --2- 172.21.15.5:0/1735505063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a4c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f812dc6d640 1 --2- 172.21.15.5:0/1735505063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5ff50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c00 tx=0x7f81240bb870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642f2d4799d05c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:08.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f81137fe640 1 -- 172.21.15.5:0/1735505063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124049040 con 0x557392a495d0 2024-09-11T22:27:08.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f81137fe640 1 -- 172.21.15.5:0/1735505063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81240bc330 con 0x557392a495d0 2024-09-11T22:27:08.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.308+0000 7f81137fe640 1 -- 172.21.15.5:0/1735505063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81240a6040 con 0x557392a495d0 2024-09-11T22:27:08.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.309+0000 7f81322aa740 1 -- 172.21.15.5:0/1735505063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 msgr2=0x557392a5ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:08.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.309+0000 7f81322aa740 1 --2- 172.21.15.5:0/1735505063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5ff50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8124000c00 tx=0x7f81240bb870 comp rx=0 tx=0).stop 2024-09-11T22:27:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.309+0000 7f81322aa740 1 -- 172.21.15.5:0/1735505063 shutdown_connections 2024-09-11T22:27:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.309+0000 7f81322aa740 1 --2- 172.21.15.5:0/1735505063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 0x557392a4c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.309+0000 7f81322aa740 1 --2- 172.21.15.5:0/1735505063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a4c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.309+0000 7f81322aa740 1 --2- 172.21.15.5:0/1735505063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5ff50 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.309+0000 7f81322aa740 1 -- 172.21.15.5:0/1735505063 >> 172.21.15.5:0/1735505063 conn(0x557392a12310 msgr2=0x557392a50180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.309+0000 7f81322aa740 1 -- 172.21.15.5:0/1735505063 shutdown_connections 2024-09-11T22:27:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.309+0000 7f81322aa740 1 -- 172.21.15.5:0/1735505063 wait complete. 2024-09-11T22:27:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f81322aa740 1 -- start start 2024-09-11T22:27:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f81322aa740 1 Processor -- start 2024-09-11T22:27:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f81322aa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f81322aa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a57cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f81322aa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 0x557392a582d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f812dc6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f81322aa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81281acc70 con 0x557392a495d0 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f81322aa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81281ac5a0 con 0x557392a4ab40 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f81322aa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81281a97e0 con 0x557392a54370 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f812dc6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60488/0 (socket says 172.21.15.5:60488) 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f812dc6d640 1 -- 172.21.15.5:0/4220120204 learned_addr learned my addr 172.21.15.5:0/4220120204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f81322aa740 10 client.0 _ll_register_callbacks cb 0x7f811800fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f812d46c640 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a57cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.310+0000 7f812e46e640 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 0x557392a582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f812dc6d640 1 -- 172.21.15.5:0/4220120204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 msgr2=0x557392a582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f812dc6d640 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 0x557392a582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f812dc6d640 1 -- 172.21.15.5:0/4220120204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 msgr2=0x557392a57cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f812dc6d640 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a57cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f812dc6d640 1 -- 172.21.15.5:0/4220120204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557392a58b00 con 0x557392a495d0 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f812d46c640 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a57cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f812e46e640 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 0x557392a582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f812dc6d640 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5c940 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f81240b8090 tx=0x7f812402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af181525efcb4fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f81117fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f81117fa640 1 -- 172.21.15.5:0/4220120204 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124049070 con 0x557392a495d0 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f81117fa640 1 -- 172.21.15.5:0/4220120204 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f81240ed020 con 0x557392a495d0 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f81322aa740 1 -- 172.21.15.5:0/4220120204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81240bc330 con 0x557392a495d0 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f81117fa640 1 -- 172.21.15.5:0/4220120204 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f81240a6070 con 0x557392a495d0 2024-09-11T22:27:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f81322aa740 10 client.16944 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f81322aa740 1 -- 172.21.15.5:0/4220120204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557392aceb30 con 0x557392a495d0 2024-09-11T22:27:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.311+0000 7f81117fa640 1 -- 172.21.15.5:0/4220120204 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81240bc330 con 0x557392a495d0 2024-09-11T22:27:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.312+0000 7f81117fa640 1 -- 172.21.15.5:0/4220120204 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f81240c4410 con 0x557392a495d0 2024-09-11T22:27:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.312+0000 7f81322aa740 10 client.16944 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.312+0000 7f81322aa740 10 client.16944 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:27:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.312+0000 7f81322aa740 20 client.16944 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.312+0000 7f81322aa740 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x557392aceec0 0x557392ad12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.312+0000 7f81322aa740 4 client.16944 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:27:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.312+0000 7f81322aa740 1 -- 172.21.15.5:0/4220120204 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557392933170 con 0x557392aceec0 2024-09-11T22:27:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.312+0000 7f812d46c640 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x557392aceec0 0x557392ad12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.312+0000 7f812d46c640 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x557392aceec0 0x557392ad12b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ba7aa926ac06a6bf server_cookie=98edf46478da7af6 in_seq=0 out_seq=0 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.312+0000 7f81117fa640 10 client.16944 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81117fa640 1 -- 172.21.15.5:0/4220120204 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557392933170 con 0x557392aceec0 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81117fa640 10 client.16944 handle_command_reply: tid=0 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 client.16944 shutdown 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 -- 172.21.15.5:0/4220120204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 msgr2=0x557392a5c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5c940 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f81240b8090 tx=0x7f812402a410 comp rx=0 tx=0).stop 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 -- 172.21.15.5:0/4220120204 shutdown_connections 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x557392aceec0 0x557392ad12b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557392a54370 0x557392a582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557392a4ab40 0x557392a57cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81117fa640 0 client.16944 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:27:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 --2- 172.21.15.5:0/4220120204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557392a495d0 0x557392a5c940 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 -- 172.21.15.5:0/4220120204 >> 172.21.15.5:0/4220120204 conn(0x557392a12310 msgr2=0x557392a528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 -- 172.21.15.5:0/4220120204 shutdown_connections 2024-09-11T22:27:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 1 -- 172.21.15.5:0/4220120204 wait complete. 2024-09-11T22:27:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.313+0000 7f81322aa740 20 client.16944 trim_cache size 0 max 16384 2024-09-11T22:27:08.321 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:27:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 -- 172.21.15.5:0/4113661532 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81140a2d70 msgr2=0x7f81140a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81140a2d70 0x7f81140a5230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8120013a40 tx=0x7f8120005e90 comp rx=0 tx=0).stop 2024-09-11T22:27:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 -- 172.21.15.5:0/4113661532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8128101de0 msgr2=0x7f81281983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8128101de0 0x7f81281983d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8124006010 tx=0x7f81240042f0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 -- 172.21.15.5:0/4113661532 shutdown_connections 2024-09-11T22:27:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81140a2d70 0x7f81140a5230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8128106f10 0x7f812819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8128102790 0x7f8128198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 --2- 172.21.15.5:0/4113661532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8128101de0 0x7f81281983d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:08.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 -- 172.21.15.5:0/4113661532 >> 172.21.15.5:0/4113661532 conn(0x7f81280fd8c0 msgr2=0x7f81280febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:08.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 -- 172.21.15.5:0/4113661532 shutdown_connections 2024-09-11T22:27:08.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:08.314+0000 7f812fed1640 1 -- 172.21.15.5:0/4113661532 wait complete. 2024-09-11T22:27:08.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 330769 7.26 MB/sec execute 86 sec latency 90.798 ms 2024-09-11T22:27:09.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:27:09.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 333749 7.39 MB/sec execute 87 sec latency 11.105 ms 2024-09-11T22:27:09.405 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:09.740 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:09 smithi005 ceph-mon[30943]: pgmap v152: 97 pgs: 97 active+clean; 48 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 745 KiB/s wr, 167 op/s 2024-09-11T22:27:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:09 smithi159 ceph-mon[35534]: pgmap v152: 97 pgs: 97 active+clean; 48 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 745 KiB/s wr, 167 op/s 2024-09-11T22:27:09.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:09 smithi172 ceph-mon[35461]: pgmap v152: 97 pgs: 97 active+clean; 48 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 745 KiB/s wr, 167 op/s 2024-09-11T22:27:10.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 -- 172.21.15.5:0/616227866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97401027c0 msgr2=0x7f9740106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:10.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 --2- 172.21.15.5:0/616227866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97401027c0 0x7f9740106810 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f972c009900 tx=0x7f972c02a410 comp rx=0 tx=0).stop 2024-09-11T22:27:10.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 -- 172.21.15.5:0/616227866 shutdown_connections 2024-09-11T22:27:10.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 --2- 172.21.15.5:0/616227866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9740106f40 0x7f974010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 --2- 172.21.15.5:0/616227866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97401027c0 0x7f9740106810 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 --2- 172.21.15.5:0/616227866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9740101e10 0x7f97401021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 -- 172.21.15.5:0/616227866 >> 172.21.15.5:0/616227866 conn(0x7f97400fd8d0 msgr2=0x7f97400ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:10.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 -- 172.21.15.5:0/616227866 shutdown_connections 2024-09-11T22:27:10.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 -- 172.21.15.5:0/616227866 wait complete. 2024-09-11T22:27:10.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 Processor -- start 2024-09-11T22:27:10.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.260+0000 7f9746d87640 1 -- start start 2024-09-11T22:27:10.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9746d87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9740101e10 0x7f97400744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9746d87640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97401027c0 0x7f9740074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9746d87640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9740106f40 0x7f9740074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9746d87640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9740111fa0 con 0x7f9740101e10 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9746d87640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9740111e20 con 0x7f9740106f40 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9746d87640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9740112120 con 0x7f97401027c0 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9744b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9740101e10 0x7f97400744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9744b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9740101e10 0x7f97400744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60508/0 (socket says 172.21.15.5:60508) 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9744b23640 1 -- 172.21.15.5:0/3652560089 learned_addr learned my addr 172.21.15.5:0/3652560089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f973ffff640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97401027c0 0x7f9740074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9745324640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9740106f40 0x7f9740074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9744b23640 1 -- 172.21.15.5:0/3652560089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97401027c0 msgr2=0x7f9740074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9744b23640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97401027c0 0x7f9740074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9744b23640 1 -- 172.21.15.5:0/3652560089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9740106f40 msgr2=0x7f9740074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9744b23640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9740106f40 0x7f9740074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9744b23640 1 -- 172.21.15.5:0/3652560089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97401a68e0 con 0x7f9740101e10 2024-09-11T22:27:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f973ffff640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97401027c0 0x7f9740074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f9744b23640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9740101e10 0x7f97400744f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f973400d950 tx=0x7f973400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.261+0000 7f973dffb640 1 -- 172.21.15.5:0/3652560089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734017070 con 0x7f9740101e10 2024-09-11T22:27:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.262+0000 7f973dffb640 1 -- 172.21.15.5:0/3652560089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9734009c30 con 0x7f9740101e10 2024-09-11T22:27:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.262+0000 7f9746d87640 1 -- 172.21.15.5:0/3652560089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97401a6bd0 con 0x7f9740101e10 2024-09-11T22:27:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.262+0000 7f973dffb640 1 -- 172.21.15.5:0/3652560089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973400f6b0 con 0x7f9740101e10 2024-09-11T22:27:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.262+0000 7f9746d87640 1 -- 172.21.15.5:0/3652560089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97401a70e0 con 0x7f9740101e10 2024-09-11T22:27:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.264+0000 7f973dffb640 1 -- 172.21.15.5:0/3652560089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f97340109d0 con 0x7f9740101e10 2024-09-11T22:27:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.264+0000 7f9749160740 1 Processor -- start 2024-09-11T22:27:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9749160740 1 -- start start 2024-09-11T22:27:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9749160740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 0x5602db462090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9749160740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db462a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9749160740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db477f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9749160740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5602db3c82f0 con 0x5602db4625d0 2024-09-11T22:27:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9749160740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5602db36fcf0 con 0x5602db475b80 2024-09-11T22:27:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9749160740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5602db3c42b0 con 0x5602db463b40 2024-09-11T22:27:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9744b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db462a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9745324640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 0x5602db462090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9744b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db462a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60518/0 (socket says 172.21.15.5:60518) 2024-09-11T22:27:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f9744b23640 1 -- 172.21.15.5:0/3577364647 learned_addr learned my addr 172.21.15.5:0/3577364647 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f973dffb640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97300a2cf0 0x7f97300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.265+0000 7f973ffff640 1 --2- 172.21.15.5:0/3577364647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db477f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f9745324640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97300a2cf0 0x7f97300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f973dffb640 1 -- 172.21.15.5:0/3652560089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f97340cf0e0 con 0x7f9740101e10 2024-09-11T22:27:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f9744b23640 1 -- 172.21.15.5:0/3577364647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 msgr2=0x5602db462090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f9744b23640 1 --2- 172.21.15.5:0/3577364647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 0x5602db462090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f9744b23640 1 -- 172.21.15.5:0/3577364647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 msgr2=0x5602db477f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f9744b23640 1 --2- 172.21.15.5:0/3577364647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db477f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f9744b23640 1 -- 172.21.15.5:0/3577364647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602db3543a0 con 0x5602db4625d0 2024-09-11T22:27:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f9744b23640 1 --2- 172.21.15.5:0/3577364647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db462a10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f97340cc790 tx=0x7f9734010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf908d18f82d1bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f9745324640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97300a2cf0 0x7f97300a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97380094d0 tx=0x7f9738004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f972affd640 1 -- 172.21.15.5:0/3577364647 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734020030 con 0x5602db4625d0 2024-09-11T22:27:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f972affd640 1 -- 172.21.15.5:0/3577364647 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9734009de0 con 0x5602db4625d0 2024-09-11T22:27:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.266+0000 7f972affd640 1 -- 172.21.15.5:0/3577364647 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973406eb30 con 0x5602db4625d0 2024-09-11T22:27:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 -- 172.21.15.5:0/3577364647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 msgr2=0x5602db462a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 --2- 172.21.15.5:0/3577364647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db462a10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f97340cc790 tx=0x7f9734010d70 comp rx=0 tx=0).stop 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 -- 172.21.15.5:0/3577364647 shutdown_connections 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 --2- 172.21.15.5:0/3577364647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db477f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 --2- 172.21.15.5:0/3577364647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db462a10 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 --2- 172.21.15.5:0/3577364647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 0x5602db462090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 -- 172.21.15.5:0/3577364647 >> 172.21.15.5:0/3577364647 conn(0x5602db42b310 msgr2=0x5602db45ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 -- 172.21.15.5:0/3577364647 shutdown_connections 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 -- 172.21.15.5:0/3577364647 wait complete. 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 -- start start 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.267+0000 7f9749160740 1 Processor -- start 2024-09-11T22:27:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9749160740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db4625d0 0x5602db475580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9749160740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db463b40 0x5602db472830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9749160740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db475b80 0x5602db472e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9749160740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5602db478820 con 0x5602db475b80 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9749160740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5602db383060 con 0x5602db463b40 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9749160740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5602db4789a0 con 0x5602db4625d0 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9749160740 10 client.0 _ll_register_callbacks cb 0x5602db0d68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9745324640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db475b80 0x5602db472e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9745324640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db475b80 0x5602db472e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60526/0 (socket says 172.21.15.5:60526) 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9745324640 1 -- 172.21.15.5:0/3518206234 learned_addr learned my addr 172.21.15.5:0/3518206234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f973ffff640 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db463b40 0x5602db472830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9744b23640 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db4625d0 0x5602db475580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9745324640 1 -- 172.21.15.5:0/3518206234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db4625d0 msgr2=0x5602db475580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9745324640 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db4625d0 0x5602db475580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9745324640 1 -- 172.21.15.5:0/3518206234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db463b40 msgr2=0x5602db472830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9745324640 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db463b40 0x5602db472830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.268+0000 7f9745324640 1 -- 172.21.15.5:0/3518206234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602db1393c0 con 0x5602db475b80 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f9745324640 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db475b80 0x5602db472e20 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f97380092f0 tx=0x7f9738004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b17dea6ebf14d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f971ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f971ffff640 1 -- 172.21.15.5:0/3518206234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738025070 con 0x5602db475b80 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f971ffff640 1 -- 172.21.15.5:0/3518206234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f97380047e0 con 0x5602db475b80 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f9749160740 1 -- 172.21.15.5:0/3518206234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9734009de0 con 0x5602db475b80 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f971ffff640 1 -- 172.21.15.5:0/3518206234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9738030050 con 0x5602db475b80 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f9749160740 10 client.16956 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f9749160740 1 -- 172.21.15.5:0/3518206234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5602db484370 con 0x5602db475b80 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f971ffff640 1 -- 172.21.15.5:0/3518206234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9734009de0 con 0x5602db475b80 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f971ffff640 1 -- 172.21.15.5:0/3518206234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f9738005580 con 0x5602db475b80 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f9749160740 10 client.16956 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f9749160740 10 client.16956 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f9749160740 20 client.16956 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f9749160740 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5602db485780 0x5602db487c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f9749160740 4 client.16956 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.269+0000 7f9749160740 1 -- 172.21.15.5:0/3518206234 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602db3d5860 con 0x5602db485780 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.270+0000 7f9744b23640 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5602db485780 0x5602db487c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.270+0000 7f9744b23640 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5602db485780 0x5602db487c00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0cc373a7b4e5d7c server_cookie=12b77ee14ef86689 in_seq=0 out_seq=0 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.270+0000 7f971ffff640 10 client.16956 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f971ffff640 1 -- 172.21.15.5:0/3518206234 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602db3d5860 con 0x5602db485780 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f971ffff640 10 client.16956 handle_command_reply: tid=0 2024-09-11T22:27:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 client.16956 shutdown 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 -- 172.21.15.5:0/3518206234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db475b80 msgr2=0x5602db472e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db475b80 0x5602db472e20 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f97380092f0 tx=0x7f9738004a40 comp rx=0 tx=0).stop 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 -- 172.21.15.5:0/3518206234 shutdown_connections 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db475b80 0x5602db472e20 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5602db485780 0x5602db487c00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f971ffff640 0 client.16956 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db463b40 0x5602db472830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 --2- 172.21.15.5:0/3518206234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db4625d0 0x5602db475580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 -- 172.21.15.5:0/3518206234 >> 172.21.15.5:0/3518206234 conn(0x5602db42b310 msgr2=0x5602db464990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 -- 172.21.15.5:0/3518206234 shutdown_connections 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 1 -- 172.21.15.5:0/3518206234 wait complete. 2024-09-11T22:27:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.271+0000 7f9749160740 20 client.16956 trim_cache size 0 max 16384 2024-09-11T22:27:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9749160740 1 Processor -- start 2024-09-11T22:27:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9749160740 1 -- start start 2024-09-11T22:27:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9749160740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db4625d0 0x5602db45ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9749160740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db463b40 0x5602db481ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9749160740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db48d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9749160740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97401ab790 con 0x5602db463b40 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9749160740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97401ad2d0 con 0x5602db475b80 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9749160740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f97401abe60 con 0x5602db4625d0 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f973ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db463b40 0x5602db481ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f973ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db463b40 0x5602db481ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60530/0 (socket says 172.21.15.5:60530) 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9744b23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db4625d0 0x5602db45ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9745324640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db48d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f973ffff640 1 -- 172.21.15.5:0/1457037420 learned_addr learned my addr 172.21.15.5:0/1457037420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9745324640 1 -- 172.21.15.5:0/1457037420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db4625d0 msgr2=0x5602db45ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9745324640 1 --2- 172.21.15.5:0/1457037420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db4625d0 0x5602db45ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9745324640 1 -- 172.21.15.5:0/1457037420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db463b40 msgr2=0x5602db481ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9745324640 1 --2- 172.21.15.5:0/1457037420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db463b40 0x5602db481ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f9745324640 1 -- 172.21.15.5:0/1457037420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602db463760 con 0x5602db475b80 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.286+0000 7f973ffff640 1 --2- 172.21.15.5:0/1457037420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db463b40 0x5602db481ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.287+0000 7f9745324640 1 --2- 172.21.15.5:0/1457037420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db48d520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9738006010 tx=0x7f9738009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5806b214df121d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.287+0000 7f972a7fc640 1 -- 172.21.15.5:0/1457037420 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738025070 con 0x5602db475b80 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.287+0000 7f972a7fc640 1 -- 172.21.15.5:0/1457037420 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f973800ce50 con 0x5602db475b80 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.287+0000 7f972a7fc640 1 -- 172.21.15.5:0/1457037420 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738010460 con 0x5602db475b80 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 -- 172.21.15.5:0/1457037420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 msgr2=0x5602db48d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 --2- 172.21.15.5:0/1457037420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db48d520 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9738006010 tx=0x7f9738009dd0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 -- 172.21.15.5:0/1457037420 shutdown_connections 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 --2- 172.21.15.5:0/1457037420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db48d520 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 --2- 172.21.15.5:0/1457037420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db463b40 0x5602db481ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 --2- 172.21.15.5:0/1457037420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db4625d0 0x5602db45ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 -- 172.21.15.5:0/1457037420 >> 172.21.15.5:0/1457037420 conn(0x5602db42b310 msgr2=0x5602db45e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 -- 172.21.15.5:0/1457037420 shutdown_connections 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 -- 172.21.15.5:0/1457037420 wait complete. 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 -- start start 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.288+0000 7f9749160740 1 Processor -- start 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9749160740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db47dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9749160740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 0x5602db47e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9749160740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db47ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9749160740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97401acc00 con 0x5602db4625d0 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9749160740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97401ac530 con 0x5602db475b80 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9749160740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f97401a9790 con 0x5602db463b40 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9749160740 10 client.0 _ll_register_callbacks cb 0x7f9738005bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9744b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db47dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9744b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db47dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60534/0 (socket says 172.21.15.5:60534) 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9744b23640 1 -- 172.21.15.5:0/1981062221 learned_addr learned my addr 172.21.15.5:0/1981062221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f973ffff640 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 0x5602db47e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9745324640 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db47ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9744b23640 1 -- 172.21.15.5:0/1981062221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 msgr2=0x5602db47e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9744b23640 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 0x5602db47e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9744b23640 1 -- 172.21.15.5:0/1981062221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 msgr2=0x5602db47ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9744b23640 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db47ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.289+0000 7f9744b23640 1 -- 172.21.15.5:0/1981062221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602db47b490 con 0x5602db4625d0 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.290+0000 7f973ffff640 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 0x5602db47e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.290+0000 7f9744b23640 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db47dec0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f97340ccdf0 tx=0x7f973400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7291e9ec48e361a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.290+0000 7f971ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.290+0000 7f971ffff640 1 -- 172.21.15.5:0/1981062221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734020040 con 0x5602db4625d0 2024-09-11T22:27:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.290+0000 7f971ffff640 1 -- 172.21.15.5:0/1981062221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f9734097c30 con 0x5602db4625d0 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.290+0000 7f9749160740 1 -- 172.21.15.5:0/1981062221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f973800ce50 con 0x5602db4625d0 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.290+0000 7f971ffff640 1 -- 172.21.15.5:0/1981062221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9734009de0 con 0x5602db4625d0 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.290+0000 7f9749160740 10 client.16962 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.290+0000 7f9749160740 1 -- 172.21.15.5:0/1981062221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5602db4f1080 con 0x5602db4625d0 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.290+0000 7f971ffff640 1 -- 172.21.15.5:0/1981062221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f973800ce50 con 0x5602db4625d0 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.291+0000 7f971ffff640 1 -- 172.21.15.5:0/1981062221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f973409aa10 con 0x5602db4625d0 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.291+0000 7f9749160740 10 client.16962 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.291+0000 7f9749160740 10 client.16962 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.291+0000 7f9749160740 20 client.16962 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.291+0000 7f9749160740 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5602db4f1310 0x5602db4f3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.291+0000 7f9749160740 4 client.16962 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.291+0000 7f9749160740 1 -- 172.21.15.5:0/1981062221 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602db3cfd70 con 0x5602db4f1310 2024-09-11T22:27:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.291+0000 7f973ffff640 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5602db4f1310 0x5602db4f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.291+0000 7f971ffff640 10 client.16962 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.291+0000 7f973ffff640 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5602db4f1310 0x5602db4f3700 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=493472b5ffdd9c4c server_cookie=a87ce0541ae68e4c in_seq=0 out_seq=0 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f971ffff640 1 -- 172.21.15.5:0/1981062221 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602db3cfd70 con 0x5602db4f1310 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f971ffff640 10 client.16962 handle_command_reply: tid=0 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f9749160740 1 client.16962 shutdown 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f9749160740 1 -- 172.21.15.5:0/1981062221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 msgr2=0x5602db47dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f9749160740 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db47dec0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f97340ccdf0 tx=0x7f973400bf60 comp rx=0 tx=0).stop 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f9749160740 1 -- 172.21.15.5:0/1981062221 shutdown_connections 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f9749160740 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5602db4f1310 0x5602db4f3700 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f9749160740 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5602db475b80 0x5602db47ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f9749160740 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5602db463b40 0x5602db47e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f9749160740 1 --2- 172.21.15.5:0/1981062221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5602db4625d0 0x5602db47dec0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f9749160740 1 -- 172.21.15.5:0/1981062221 >> 172.21.15.5:0/1981062221 conn(0x5602db42b310 msgr2=0x5602db474090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f971ffff640 0 client.16962 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:27:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.292+0000 7f9749160740 1 -- 172.21.15.5:0/1981062221 shutdown_connections 2024-09-11T22:27:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.293+0000 7f9749160740 1 -- 172.21.15.5:0/1981062221 wait complete. 2024-09-11T22:27:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.293+0000 7f9749160740 20 client.16962 trim_cache size 0 max 16384 2024-09-11T22:27:10.299 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:27:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.293+0000 7f9746d87640 1 -- 172.21.15.5:0/3652560089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97300a2cf0 msgr2=0x7f97300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.293+0000 7f9746d87640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97300a2cf0 0x7f97300a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f97380094d0 tx=0x7f9738004ab0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.293+0000 7f9746d87640 1 -- 172.21.15.5:0/3652560089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9740101e10 msgr2=0x7f97400744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.293+0000 7f9746d87640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9740101e10 0x7f97400744f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f973400d950 tx=0x7f973400de10 comp rx=0 tx=0).stop 2024-09-11T22:27:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.294+0000 7f9746d87640 1 -- 172.21.15.5:0/3652560089 shutdown_connections 2024-09-11T22:27:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.294+0000 7f9746d87640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f97300a2cf0 0x7f97300a51b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.294+0000 7f9746d87640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9740106f40 0x7f9740074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.294+0000 7f9746d87640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97401027c0 0x7f9740074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.294+0000 7f9746d87640 1 --2- 172.21.15.5:0/3652560089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9740101e10 0x7f97400744f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.294+0000 7f9746d87640 1 -- 172.21.15.5:0/3652560089 >> 172.21.15.5:0/3652560089 conn(0x7f97400fd8d0 msgr2=0x7f97400fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.294+0000 7f9746d87640 1 -- 172.21.15.5:0/3652560089 shutdown_connections 2024-09-11T22:27:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:10.294+0000 7f9746d87640 1 -- 172.21.15.5:0/3652560089 wait complete. 2024-09-11T22:27:10.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 336438 7.45 MB/sec execute 88 sec latency 25.933 ms 2024-09-11T22:27:11.126 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:27:11.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337570 7.44 MB/sec execute 89 sec latency 442.814 ms 2024-09-11T22:27:11.382 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:11.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:11 smithi005 ceph-mon[30943]: pgmap v153: 97 pgs: 97 active+clean; 48 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 506 KiB/s wr, 113 op/s 2024-09-11T22:27:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:11 smithi159 ceph-mon[35534]: pgmap v153: 97 pgs: 97 active+clean; 48 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 506 KiB/s wr, 113 op/s 2024-09-11T22:27:11.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:11 smithi172 ceph-mon[35461]: pgmap v153: 97 pgs: 97 active+clean; 48 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 506 KiB/s wr, 113 op/s 2024-09-11T22:27:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.019+0000 7fd0a5574640 1 -- 172.21.15.5:0/3690313643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0102570 msgr2=0x7fd0a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.019+0000 7fd0a5574640 1 --2- 172.21.15.5:0/3690313643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0102570 0x7fd0a01029b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd094009900 tx=0x7fd09402a480 comp rx=0 tx=0).stop 2024-09-11T22:27:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.019+0000 7fd0a5574640 1 -- 172.21.15.5:0/3690313643 shutdown_connections 2024-09-11T22:27:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.019+0000 7fd0a5574640 1 --2- 172.21.15.5:0/3690313643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0a0102ef0 0x7fd0a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.019+0000 7fd0a5574640 1 --2- 172.21.15.5:0/3690313643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0102570 0x7fd0a01029b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.019+0000 7fd0a5574640 1 --2- 172.21.15.5:0/3690313643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0a0100e20 0x7fd0a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.019+0000 7fd0a5574640 1 -- 172.21.15.5:0/3690313643 >> 172.21.15.5:0/3690313643 conn(0x7fd0a00fc7d0 msgr2=0x7fd0a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.019+0000 7fd0a5574640 1 -- 172.21.15.5:0/3690313643 shutdown_connections 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.019+0000 7fd0a5574640 1 -- 172.21.15.5:0/3690313643 wait complete. 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.019+0000 7fd0a5574640 1 Processor -- start 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd0a5574640 1 -- start start 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd0a5574640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0100e20 0x7fd0a01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd0a5574640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0a0102570 0x7fd0a0199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd0a5574640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0a0102ef0 0x7fd0a01a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd0a5574640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a0110ff0 con 0x7fd0a0100e20 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd0a5574640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a0110e70 con 0x7fd0a0102ef0 2024-09-11T22:27:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd0a5574640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a0111170 con 0x7fd0a0102570 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd09effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0100e20 0x7fd0a01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd09effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0100e20 0x7fd0a01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38358/0 (socket says 172.21.15.5:38358) 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd09effd640 1 -- 172.21.15.5:0/1901944660 learned_addr learned my addr 172.21.15.5:0/1901944660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd09effd640 1 -- 172.21.15.5:0/1901944660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0a0102570 msgr2=0x7fd0a0199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd09effd640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0a0102570 0x7fd0a0199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd09effd640 1 -- 172.21.15.5:0/1901944660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0a0102ef0 msgr2=0x7fd0a01a4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0a0102ef0 0x7fd0a01a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd09effd640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0a0102ef0 0x7fd0a01a4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd09effd640 1 -- 172.21.15.5:0/1901944660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0a01a49a0 con 0x7fd0a0100e20 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.020+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0a0102ef0 0x7fd0a01a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.021+0000 7fd09effd640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0100e20 0x7fd0a01993e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd09000b4a0 tx=0x7fd09000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.021+0000 7fd087fff640 1 -- 172.21.15.5:0/1901944660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090014070 con 0x7fd0a0100e20 2024-09-11T22:27:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.021+0000 7fd0a5574640 1 -- 172.21.15.5:0/1901944660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0a01a4b70 con 0x7fd0a0100e20 2024-09-11T22:27:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.021+0000 7fd087fff640 1 -- 172.21.15.5:0/1901944660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd090010af0 con 0x7fd0a0100e20 2024-09-11T22:27:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.021+0000 7fd087fff640 1 -- 172.21.15.5:0/1901944660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090017d10 con 0x7fd0a0100e20 2024-09-11T22:27:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.021+0000 7fd0a5574640 1 -- 172.21.15.5:0/1901944660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0a01a5030 con 0x7fd0a0100e20 2024-09-11T22:27:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.024+0000 7fd087fff640 1 -- 172.21.15.5:0/1901944660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd090017070 con 0x7fd0a0100e20 2024-09-11T22:27:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.024+0000 7fd0a794d740 1 Processor -- start 2024-09-11T22:27:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.024+0000 7fd0a794d740 1 -- start start 2024-09-11T22:27:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.024+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c692090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.024+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c692a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.024+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 0x55805c69f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.024+0000 7fd0a794d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55805c5f82f0 con 0x55805c6925d0 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.024+0000 7fd0a794d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55805c59fcf0 con 0x55805c693b40 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.024+0000 7fd0a794d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55805c5f42b0 con 0x55805c69d370 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c692a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c692a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38360/0 (socket says 172.21.15.5:38360) 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09effd640 1 -- 172.21.15.5:0/1380101464 learned_addr learned my addr 172.21.15.5:0/1380101464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09e7fc640 1 --2- 172.21.15.5:0/1380101464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 0x55805c69f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/1380101464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c692090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd087fff640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0880a2cf0 0x7fd0880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0880a2cf0 0x7fd0880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09effd640 1 -- 172.21.15.5:0/1380101464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 msgr2=0x55805c69f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09effd640 1 --2- 172.21.15.5:0/1380101464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 0x55805c69f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09effd640 1 -- 172.21.15.5:0/1380101464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 msgr2=0x55805c692090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09effd640 1 --2- 172.21.15.5:0/1380101464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c692090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09effd640 1 -- 172.21.15.5:0/1380101464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55805c5843a0 con 0x55805c6925d0 2024-09-11T22:27:12.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd087fff640 1 -- 172.21.15.5:0/1901944660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fd0900ce860 con 0x7fd0a0100e20 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09effd640 1 --2- 172.21.15.5:0/1380101464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c692a10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd0900cbf60 tx=0x7fd0900047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1058a5e416bc1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0880a2cf0 0x7fd0880a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd098009fd0 tx=0x7fd098004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd0857fa640 1 -- 172.21.15.5:0/1380101464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09001e020 con 0x55805c6925d0 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd0857fa640 1 -- 172.21.15.5:0/1380101464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09009b4f0 con 0x55805c6925d0 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.025+0000 7fd0857fa640 1 -- 172.21.15.5:0/1380101464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0900a3a60 con 0x55805c6925d0 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.026+0000 7fd0a794d740 1 -- 172.21.15.5:0/1380101464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 msgr2=0x55805c692a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.026+0000 7fd0a794d740 1 --2- 172.21.15.5:0/1380101464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c692a10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd0900cbf60 tx=0x7fd0900047d0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.026+0000 7fd0a794d740 1 -- 172.21.15.5:0/1380101464 shutdown_connections 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.026+0000 7fd0a794d740 1 --2- 172.21.15.5:0/1380101464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 0x55805c69f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.026+0000 7fd0a794d740 1 --2- 172.21.15.5:0/1380101464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c692a10 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.026+0000 7fd0a794d740 1 --2- 172.21.15.5:0/1380101464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c692090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.026+0000 7fd0a794d740 1 -- 172.21.15.5:0/1380101464 >> 172.21.15.5:0/1380101464 conn(0x55805c65b310 msgr2=0x55805c68ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.026+0000 7fd0a794d740 1 -- 172.21.15.5:0/1380101464 shutdown_connections 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.026+0000 7fd0a794d740 1 -- 172.21.15.5:0/1380101464 wait complete. 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd0a794d740 1 -- start start 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd0a794d740 1 Processor -- start 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c6925d0 0x55805c69cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c69a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c69d370 0x55805c69a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd0a794d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55805c6a0010 con 0x55805c69d370 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd0a794d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55805c5b3060 con 0x55805c693b40 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd0a794d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55805c6a0190 con 0x55805c6925d0 2024-09-11T22:27:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd0a794d740 10 client.0 _ll_register_callbacks cb 0x55805c3068c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd09f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c69d370 0x55805c69a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd09f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c69d370 0x55805c69a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38362/0 (socket says 172.21.15.5:38362) 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd09f7fe640 1 -- 172.21.15.5:0/3890174852 learned_addr learned my addr 172.21.15.5:0/3890174852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd09effd640 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c6925d0 0x55805c69cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd09e7fc640 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c69a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd09f7fe640 1 -- 172.21.15.5:0/3890174852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c6925d0 msgr2=0x55805c69cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c6925d0 0x55805c69cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd09f7fe640 1 -- 172.21.15.5:0/3890174852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 msgr2=0x55805c69a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c69a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.027+0000 7fd09f7fe640 1 -- 172.21.15.5:0/3890174852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55805c3693c0 con 0x55805c69d370 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c69d370 0x55805c69a650 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd098003bb0 tx=0x7fd098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e7a0950a22d91 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd082ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd082ff5640 1 -- 172.21.15.5:0/3890174852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09801c050 con 0x55805c69d370 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd082ff5640 1 -- 172.21.15.5:0/3890174852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fd098015020 con 0x55805c69d370 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd0a794d740 1 -- 172.21.15.5:0/3890174852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09009b4f0 con 0x55805c69d370 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd082ff5640 1 -- 172.21.15.5:0/3890174852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd098015220 con 0x55805c69d370 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd0a794d740 10 client.16974 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd0a794d740 1 -- 172.21.15.5:0/3890174852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55805c6abb70 con 0x55805c69d370 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd082ff5640 1 -- 172.21.15.5:0/3890174852 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09009b4f0 con 0x55805c69d370 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd082ff5640 1 -- 172.21.15.5:0/3890174852 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fd0980219a0 con 0x55805c69d370 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd0a794d740 10 client.16974 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd0a794d740 10 client.16974 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd0a794d740 20 client.16974 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd0a794d740 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55805c6ad030 0x55805c6af480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd0a794d740 4 client.16974 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.028+0000 7fd0a794d740 1 -- 172.21.15.5:0/3890174852 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55805c605860 con 0x55805c6ad030 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.029+0000 7fd09effd640 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55805c6ad030 0x55805c6af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.029+0000 7fd082ff5640 10 client.16974 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.029+0000 7fd09effd640 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55805c6ad030 0x55805c6af480 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bdc0509a430c35 server_cookie=f30c0c9b6bf302e4 in_seq=0 out_seq=0 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd082ff5640 1 -- 172.21.15.5:0/3890174852 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55805c605860 con 0x55805c6ad030 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd082ff5640 10 client.16974 handle_command_reply: tid=0 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 client.16974 shutdown 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 -- 172.21.15.5:0/3890174852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c69d370 msgr2=0x55805c69a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c69d370 0x55805c69a650 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd098003bb0 tx=0x7fd098004a40 comp rx=0 tx=0).stop 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 -- 172.21.15.5:0/3890174852 shutdown_connections 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55805c6ad030 0x55805c6af480 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c69d370 0x55805c69a650 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd082ff5640 0 client.16974 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:27:12.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c69a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 --2- 172.21.15.5:0/3890174852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c6925d0 0x55805c69cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 -- 172.21.15.5:0/3890174852 >> 172.21.15.5:0/3890174852 conn(0x55805c65b310 msgr2=0x55805c694990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 -- 172.21.15.5:0/3890174852 shutdown_connections 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 1 -- 172.21.15.5:0/3890174852 wait complete. 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.030+0000 7fd0a794d740 20 client.16974 trim_cache size 0 max 16384 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.043+0000 7fd0a794d740 1 Processor -- start 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.043+0000 7fd0a794d740 1 -- start start 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.043+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c6925d0 0x55805c68dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.043+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c693b40 0x55805c695180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c69d370 0x55805c6956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd0a794d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01adb50 con 0x55805c693b40 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd0a794d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01af690 con 0x55805c69d370 2024-09-11T22:27:12.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd0a794d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01ae220 con 0x55805c6925d0 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c693b40 0x55805c695180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c693b40 0x55805c695180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38370/0 (socket says 172.21.15.5:38370) 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09e7fc640 1 -- 172.21.15.5:0/897765025 learned_addr learned my addr 172.21.15.5:0/897765025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09effd640 1 --2- 172.21.15.5:0/897765025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c6925d0 0x55805c68dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/897765025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c69d370 0x55805c6956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09e7fc640 1 -- 172.21.15.5:0/897765025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c6925d0 msgr2=0x55805c68dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09e7fc640 1 --2- 172.21.15.5:0/897765025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c6925d0 0x55805c68dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09e7fc640 1 -- 172.21.15.5:0/897765025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c69d370 msgr2=0x55805c6956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09e7fc640 1 --2- 172.21.15.5:0/897765025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c69d370 0x55805c6956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09e7fc640 1 -- 172.21.15.5:0/897765025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55805c698ff0 con 0x55805c693b40 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/897765025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c69d370 0x55805c6956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd09e7fc640 1 --2- 172.21.15.5:0/897765025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c693b40 0x55805c695180 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd0a006aac0 tx=0x7fd094004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd089cf00ccf3e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:12.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.044+0000 7fd084ff9640 1 -- 172.21.15.5:0/897765025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094035040 con 0x55805c693b40 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd084ff9640 1 -- 172.21.15.5:0/897765025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd094040070 con 0x55805c693b40 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd084ff9640 1 -- 172.21.15.5:0/897765025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0940035f0 con 0x55805c693b40 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd0a794d740 1 -- 172.21.15.5:0/897765025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c693b40 msgr2=0x55805c695180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd0a794d740 1 --2- 172.21.15.5:0/897765025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c693b40 0x55805c695180 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd0a006aac0 tx=0x7fd094004310 comp rx=0 tx=0).stop 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd0a794d740 1 -- 172.21.15.5:0/897765025 shutdown_connections 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd0a794d740 1 --2- 172.21.15.5:0/897765025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c69d370 0x55805c6956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd0a794d740 1 --2- 172.21.15.5:0/897765025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c693b40 0x55805c695180 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd0a794d740 1 --2- 172.21.15.5:0/897765025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c6925d0 0x55805c68dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd0a794d740 1 -- 172.21.15.5:0/897765025 >> 172.21.15.5:0/897765025 conn(0x55805c65b310 msgr2=0x55805c68d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd0a794d740 1 -- 172.21.15.5:0/897765025 shutdown_connections 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.045+0000 7fd0a794d740 1 -- 172.21.15.5:0/897765025 wait complete. 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd0a794d740 1 -- start start 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd0a794d740 1 Processor -- start 2024-09-11T22:27:12.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c6a1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c6a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd0a794d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 0x55805c6a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd0a794d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01aefc0 con 0x55805c6925d0 2024-09-11T22:27:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd0a794d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01ae8f0 con 0x55805c693b40 2024-09-11T22:27:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd0a794d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0a01abb30 con 0x55805c69d370 2024-09-11T22:27:12.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd0a794d740 10 client.0 _ll_register_callbacks cb 0x7fd098009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c6a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c6a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38374/0 (socket says 172.21.15.5:38374) 2024-09-11T22:27:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09effd640 1 -- 172.21.15.5:0/2989474517 learned_addr learned my addr 172.21.15.5:0/2989474517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09e7fc640 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c6a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 0x55805c6a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09effd640 1 -- 172.21.15.5:0/2989474517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 msgr2=0x55805c6a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09effd640 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 0x55805c6a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09effd640 1 -- 172.21.15.5:0/2989474517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 msgr2=0x55805c6a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09effd640 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c6a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09effd640 1 -- 172.21.15.5:0/2989474517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55805c716df0 con 0x55805c6925d0 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.046+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 0x55805c6a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.047+0000 7fd09effd640 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c6a1220 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd09009aaf0 tx=0x7fd090017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ab442c0766efd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.047+0000 7fd082ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.047+0000 7fd082ff5640 1 -- 172.21.15.5:0/2989474517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09001e040 con 0x55805c6925d0 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.047+0000 7fd082ff5640 1 -- 172.21.15.5:0/2989474517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fd090096bc0 con 0x55805c6925d0 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.047+0000 7fd0a794d740 1 -- 172.21.15.5:0/2989474517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd094040070 con 0x55805c6925d0 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.047+0000 7fd082ff5640 1 -- 172.21.15.5:0/2989474517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd09009a930 con 0x55805c6925d0 2024-09-11T22:27:12.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.047+0000 7fd0a794d740 10 client.16980 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.047+0000 7fd0a794d740 1 -- 172.21.15.5:0/2989474517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55805c7180d0 con 0x55805c6925d0 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.047+0000 7fd082ff5640 1 -- 172.21.15.5:0/2989474517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd094040070 con 0x55805c6925d0 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.047+0000 7fd082ff5640 1 -- 172.21.15.5:0/2989474517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fd090061020 con 0x55805c6925d0 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.048+0000 7fd0a794d740 10 client.16980 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.048+0000 7fd0a794d740 10 client.16980 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.048+0000 7fd0a794d740 20 client.16980 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.048+0000 7fd0a794d740 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55805c6ad030 0x55805c71a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.048+0000 7fd0a794d740 4 client.16980 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.048+0000 7fd0a794d740 1 -- 172.21.15.5:0/2989474517 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55805c5ffd70 con 0x55805c6ad030 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.048+0000 7fd09e7fc640 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55805c6ad030 0x55805c71a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.048+0000 7fd09e7fc640 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55805c6ad030 0x55805c71a310 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8814fb5592afbe58 server_cookie=3c6fad5f9e00ff0b in_seq=0 out_seq=0 2024-09-11T22:27:12.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.048+0000 7fd082ff5640 10 client.16980 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:27:12.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd082ff5640 1 -- 172.21.15.5:0/2989474517 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55805c5ffd70 con 0x55805c6ad030 2024-09-11T22:27:12.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd082ff5640 10 client.16980 handle_command_reply: tid=0 2024-09-11T22:27:12.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 client.16980 shutdown 2024-09-11T22:27:12.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 -- 172.21.15.5:0/2989474517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 msgr2=0x55805c6a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:12.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c6a1220 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd09009aaf0 tx=0x7fd090017730 comp rx=0 tx=0).stop 2024-09-11T22:27:12.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 -- 172.21.15.5:0/2989474517 shutdown_connections 2024-09-11T22:27:12.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55805c6ad030 0x55805c71a310 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55805c69d370 0x55805c6a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd082ff5640 0 client.16980 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:27:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55805c693b40 0x55805c6a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 --2- 172.21.15.5:0/2989474517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55805c6925d0 0x55805c6a1220 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 -- 172.21.15.5:0/2989474517 >> 172.21.15.5:0/2989474517 conn(0x55805c65b310 msgr2=0x55805c69c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 -- 172.21.15.5:0/2989474517 shutdown_connections 2024-09-11T22:27:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 1 -- 172.21.15.5:0/2989474517 wait complete. 2024-09-11T22:27:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.049+0000 7fd0a794d740 20 client.16980 trim_cache size 0 max 16384 2024-09-11T22:27:12.071 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:27:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 -- 172.21.15.5:0/1901944660 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0880a2cf0 msgr2=0x7fd0880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0880a2cf0 0x7fd0880a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd098009fd0 tx=0x7fd098004ab0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 -- 172.21.15.5:0/1901944660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0100e20 msgr2=0x7fd0a01993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0100e20 0x7fd0a01993e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd09000b4a0 tx=0x7fd09000b960 comp rx=0 tx=0).stop 2024-09-11T22:27:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 -- 172.21.15.5:0/1901944660 shutdown_connections 2024-09-11T22:27:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0880a2cf0 0x7fd0880a51b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0a0102ef0 0x7fd0a01a4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0a0102570 0x7fd0a0199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 --2- 172.21.15.5:0/1901944660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0a0100e20 0x7fd0a01993e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 -- 172.21.15.5:0/1901944660 >> 172.21.15.5:0/1901944660 conn(0x7fd0a00fc7d0 msgr2=0x7fd0a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 -- 172.21.15.5:0/1901944660 shutdown_connections 2024-09-11T22:27:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:12.050+0000 7fd0a5574640 1 -- 172.21.15.5:0/1901944660 wait complete. 2024-09-11T22:27:12.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337570 7.36 MB/sec execute 90 sec latency 1442.859 ms 2024-09-11T22:27:12.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:27:13.241 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:13.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337570 7.28 MB/sec execute 91 sec latency 2442.908 ms 2024-09-11T22:27:13.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:13 smithi005 ceph-mon[30943]: pgmap v154: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 193 op/s 2024-09-11T22:27:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:13 smithi159 ceph-mon[35534]: pgmap v154: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 193 op/s 2024-09-11T22:27:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:13 smithi172 ceph-mon[35461]: pgmap v154: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 193 op/s 2024-09-11T22:27:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.042+0000 7fda4fcb8640 1 -- 172.21.15.5:0/219020330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda48102ef0 msgr2=0x7fda4810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.042+0000 7fda4fcb8640 1 --2- 172.21.15.5:0/219020330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda48102ef0 0x7fda4810da40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fda40009ca0 tx=0x7fda4002a950 comp rx=0 tx=0).stop 2024-09-11T22:27:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.042+0000 7fda4fcb8640 1 -- 172.21.15.5:0/219020330 shutdown_connections 2024-09-11T22:27:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.042+0000 7fda4fcb8640 1 --2- 172.21.15.5:0/219020330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda48102ef0 0x7fda4810da40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.042+0000 7fda4fcb8640 1 --2- 172.21.15.5:0/219020330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda48102570 0x7fda481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.042+0000 7fda4fcb8640 1 --2- 172.21.15.5:0/219020330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda48100e20 0x7fda48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.042+0000 7fda4fcb8640 1 -- 172.21.15.5:0/219020330 >> 172.21.15.5:0/219020330 conn(0x7fda480fc7d0 msgr2=0x7fda480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.043+0000 7fda4fcb8640 1 -- 172.21.15.5:0/219020330 shutdown_connections 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.043+0000 7fda4fcb8640 1 -- 172.21.15.5:0/219020330 wait complete. 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.043+0000 7fda4fcb8640 1 Processor -- start 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.043+0000 7fda4fcb8640 1 -- start start 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4fcb8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda48100e20 0x7fda48197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4fcb8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda48102570 0x7fda481979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4fcb8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda48102ef0 0x7fda4819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4fcb8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda48110f80 con 0x7fda48100e20 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4fcb8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda48110e00 con 0x7fda48102570 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4fcb8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda48111100 con 0x7fda48102ef0 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4da54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda48100e20 0x7fda48197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4da54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda48100e20 0x7fda48197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38394/0 (socket says 172.21.15.5:38394) 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4da54640 1 -- 172.21.15.5:0/1514521348 learned_addr learned my addr 172.21.15.5:0/1514521348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4da54640 1 -- 172.21.15.5:0/1514521348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda48102ef0 msgr2=0x7fda4819bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4da54640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda48102ef0 0x7fda4819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4da54640 1 -- 172.21.15.5:0/1514521348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda48102570 msgr2=0x7fda481979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4d253640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda48102570 0x7fda481979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4da54640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda48102570 0x7fda481979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4da54640 1 -- 172.21.15.5:0/1514521348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4819c410 con 0x7fda48100e20 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4d253640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda48102570 0x7fda481979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda4da54640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda48100e20 0x7fda48197460 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fda4400d770 tx=0x7fda4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.044+0000 7fda3effd640 1 -- 172.21.15.5:0/1514521348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44017070 con 0x7fda48100e20 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.045+0000 7fda4fcb8640 1 -- 172.21.15.5:0/1514521348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4819c700 con 0x7fda48100e20 2024-09-11T22:27:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.045+0000 7fda3effd640 1 -- 172.21.15.5:0/1514521348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44005500 con 0x7fda48100e20 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.045+0000 7fda3effd640 1 -- 172.21.15.5:0/1514521348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44004e00 con 0x7fda48100e20 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.045+0000 7fda4fcb8640 1 -- 172.21.15.5:0/1514521348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda481a3fe0 con 0x7fda48100e20 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.047+0000 7fda3effd640 1 -- 172.21.15.5:0/1514521348 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fda44004fa0 con 0x7fda48100e20 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.047+0000 7fda52091740 1 Processor -- start 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.047+0000 7fda52091740 1 -- start start 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda52091740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03212450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda52091740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212de0 0x55ff03213220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda52091740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03213760 0x55ff03219f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda52091740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff0316e2f0 con 0x55ff03213760 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda52091740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff03115cf0 con 0x55ff03212de0 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda52091740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff0316a2b0 con 0x55ff03212070 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda3effd640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda340a2cf0 0x7fda340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4d253640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03213760 0x55ff03219f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4d253640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03213760 0x55ff03219f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38398/0 (socket says 172.21.15.5:38398) 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4d253640 1 -- 172.21.15.5:0/1118168926 learned_addr learned my addr 172.21.15.5:0/1118168926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4e255640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda340a2cf0 0x7fda340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4e255640 1 --2- 172.21.15.5:0/1118168926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03212450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda3effd640 1 -- 172.21.15.5:0/1514521348 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fda440cec30 con 0x7fda48100e20 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4da54640 1 --2- 172.21.15.5:0/1118168926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212de0 0x55ff03213220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4d253640 1 -- 172.21.15.5:0/1118168926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 msgr2=0x55ff03212450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4d253640 1 --2- 172.21.15.5:0/1118168926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03212450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4d253640 1 -- 172.21.15.5:0/1118168926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212de0 msgr2=0x55ff03213220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4d253640 1 --2- 172.21.15.5:0/1118168926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212de0 0x55ff03213220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4d253640 1 -- 172.21.15.5:0/1118168926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff030fa3a0 con 0x55ff03213760 2024-09-11T22:27:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4da54640 1 --2- 172.21.15.5:0/1118168926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212de0 0x55ff03213220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.048+0000 7fda4d253640 1 --2- 172.21.15.5:0/1118168926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03213760 0x55ff03219f60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fda38009ba0 tx=0x7fda3800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622cba8dc4660259 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda4e255640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda340a2cf0 0x7fda340a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fda40030040 tx=0x7fda400095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda33fff640 1 -- 172.21.15.5:0/1118168926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38016070 con 0x55ff03213760 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda33fff640 1 -- 172.21.15.5:0/1118168926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda38011950 con 0x55ff03213760 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda33fff640 1 -- 172.21.15.5:0/1118168926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38010870 con 0x55ff03213760 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda52091740 1 -- 172.21.15.5:0/1118168926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03213760 msgr2=0x55ff03219f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda52091740 1 --2- 172.21.15.5:0/1118168926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03213760 0x55ff03219f60 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fda38009ba0 tx=0x7fda3800e6c0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda52091740 1 -- 172.21.15.5:0/1118168926 shutdown_connections 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda52091740 1 --2- 172.21.15.5:0/1118168926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03213760 0x55ff03219f60 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda52091740 1 --2- 172.21.15.5:0/1118168926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212de0 0x55ff03213220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda52091740 1 --2- 172.21.15.5:0/1118168926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03212450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.049+0000 7fda52091740 1 -- 172.21.15.5:0/1118168926 >> 172.21.15.5:0/1118168926 conn(0x55ff031d1310 msgr2=0x55ff03204720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.050+0000 7fda52091740 1 -- 172.21.15.5:0/1118168926 shutdown_connections 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.050+0000 7fda52091740 1 -- 172.21.15.5:0/1118168926 wait complete. 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.050+0000 7fda52091740 1 -- start start 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.050+0000 7fda52091740 1 Processor -- start 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.050+0000 7fda52091740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03219d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.050+0000 7fda52091740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff03217010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.050+0000 7fda52091740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 0x55ff03217550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda52091740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff0321a4a0 con 0x55ff03212de0 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda52091740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff03129060 con 0x55ff03213760 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda52091740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff0321a620 con 0x55ff03212070 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda52091740 10 client.0 _ll_register_callbacks cb 0x55ff02e7c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4d253640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff03217010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4d253640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff03217010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38414/0 (socket says 172.21.15.5:38414) 2024-09-11T22:27:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4d253640 1 -- 172.21.15.5:0/2239076069 learned_addr learned my addr 172.21.15.5:0/2239076069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4e255640 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 0x55ff03217550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4da54640 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03219d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4d253640 1 -- 172.21.15.5:0/2239076069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 msgr2=0x55ff03219d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4d253640 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03219d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4d253640 1 -- 172.21.15.5:0/2239076069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 msgr2=0x55ff03217550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4d253640 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 0x55ff03217550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4d253640 1 -- 172.21.15.5:0/2239076069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff02edf3c0 con 0x55ff03212de0 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda4d253640 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff03217010 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fda38010410 tx=0x7fda38011680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c065e3e2c1141cd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda317fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda317fa640 1 -- 172.21.15.5:0/2239076069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38027020 con 0x55ff03212de0 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda317fa640 1 -- 172.21.15.5:0/2239076069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fda38004020 con 0x55ff03212de0 2024-09-11T22:27:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.051+0000 7fda52091740 1 -- 172.21.15.5:0/2239076069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda38011950 con 0x55ff03212de0 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda317fa640 1 -- 172.21.15.5:0/2239076069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fda3802f050 con 0x55ff03212de0 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda52091740 10 client.16992 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda52091740 1 -- 172.21.15.5:0/2239076069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ff03225da0 con 0x55ff03212de0 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda317fa640 1 -- 172.21.15.5:0/2239076069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda38011950 con 0x55ff03212de0 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda317fa640 1 -- 172.21.15.5:0/2239076069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fda3800f040 con 0x55ff03212de0 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda52091740 10 client.16992 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda52091740 10 client.16992 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda52091740 20 client.16992 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda52091740 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ff03227200 0x55ff032295f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda52091740 4 client.16992 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.052+0000 7fda52091740 1 -- 172.21.15.5:0/2239076069 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff0317b860 con 0x55ff03227200 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.053+0000 7fda4da54640 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ff03227200 0x55ff032295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.053+0000 7fda4da54640 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ff03227200 0x55ff032295f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1114a79a81bcd555 server_cookie=86e75dcf63e732f in_seq=0 out_seq=0 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.053+0000 7fda317fa640 10 client.16992 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda317fa640 1 -- 172.21.15.5:0/2239076069 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff0317b860 con 0x55ff03227200 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda317fa640 10 client.16992 handle_command_reply: tid=0 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 client.16992 shutdown 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 -- 172.21.15.5:0/2239076069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 msgr2=0x55ff03217010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff03217010 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fda38010410 tx=0x7fda38011680 comp rx=0 tx=0).stop 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 -- 172.21.15.5:0/2239076069 shutdown_connections 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ff03227200 0x55ff032295f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 0x55ff03217550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff03217010 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 --2- 172.21.15.5:0/2239076069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03219d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda317fa640 0 client.16992 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:27:14.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 -- 172.21.15.5:0/2239076069 >> 172.21.15.5:0/2239076069 conn(0x55ff031d1310 msgr2=0x55ff03203d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 -- 172.21.15.5:0/2239076069 shutdown_connections 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 1 -- 172.21.15.5:0/2239076069 wait complete. 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.054+0000 7fda52091740 20 client.16992 trim_cache size 0 max 16384 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.067+0000 7fda52091740 1 Processor -- start 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.067+0000 7fda52091740 1 -- start start 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.067+0000 7fda52091740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03211750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda52091740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0322ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda52091740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 0x55ff0322f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda52091740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda481ab770 con 0x55ff03212de0 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda52091740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda481ad2b0 con 0x55ff03213760 2024-09-11T22:27:14.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda52091740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda481abe40 con 0x55ff03212070 2024-09-11T22:27:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4d253640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0322ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4d253640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0322ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38424/0 (socket says 172.21.15.5:38424) 2024-09-11T22:27:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4d253640 1 -- 172.21.15.5:0/2200575482 learned_addr learned my addr 172.21.15.5:0/2200575482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4da54640 1 --2- 172.21.15.5:0/2200575482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03211750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4e255640 1 --2- 172.21.15.5:0/2200575482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 0x55ff0322f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4d253640 1 -- 172.21.15.5:0/2200575482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 msgr2=0x55ff03211750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:14.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4d253640 1 --2- 172.21.15.5:0/2200575482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03211750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4d253640 1 -- 172.21.15.5:0/2200575482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 msgr2=0x55ff0322f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4d253640 1 --2- 172.21.15.5:0/2200575482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 0x55ff0322f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4d253640 1 -- 172.21.15.5:0/2200575482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff03205b80 con 0x55ff03212de0 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4e255640 1 --2- 172.21.15.5:0/2200575482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 0x55ff0322f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda4d253640 1 --2- 172.21.15.5:0/2200575482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0322ec80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fda38004950 tx=0x7fda38009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673ac7546aff8e0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda337fe640 1 -- 172.21.15.5:0/2200575482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38038070 con 0x55ff03212de0 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda337fe640 1 -- 172.21.15.5:0/2200575482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda380097a0 con 0x55ff03212de0 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.068+0000 7fda337fe640 1 -- 172.21.15.5:0/2200575482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38027070 con 0x55ff03212de0 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.069+0000 7fda52091740 1 -- 172.21.15.5:0/2200575482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 msgr2=0x55ff0322ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.069+0000 7fda52091740 1 --2- 172.21.15.5:0/2200575482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0322ec80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fda38004950 tx=0x7fda38009aa0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.069+0000 7fda52091740 1 -- 172.21.15.5:0/2200575482 shutdown_connections 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.069+0000 7fda52091740 1 --2- 172.21.15.5:0/2200575482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03213760 0x55ff0322f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.069+0000 7fda52091740 1 --2- 172.21.15.5:0/2200575482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0322ec80 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.069+0000 7fda52091740 1 --2- 172.21.15.5:0/2200575482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03212070 0x55ff03211750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.069+0000 7fda52091740 1 -- 172.21.15.5:0/2200575482 >> 172.21.15.5:0/2200575482 conn(0x55ff031d1310 msgr2=0x55ff03219090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.069+0000 7fda52091740 1 -- 172.21.15.5:0/2200575482 shutdown_connections 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.069+0000 7fda52091740 1 -- 172.21.15.5:0/2200575482 wait complete. 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda52091740 1 -- start start 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda52091740 1 Processor -- start 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda52091740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212070 0x55ff0320df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda52091740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0320e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda52091740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03213760 0x55ff03290c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda52091740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda481acbe0 con 0x55ff03212de0 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda52091740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda481ac510 con 0x55ff03212070 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda52091740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fda481a9750 con 0x55ff03213760 2024-09-11T22:27:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda52091740 10 client.0 _ll_register_callbacks cb 0x7fda40009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda4d253640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0320e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda4d253640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0320e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38434/0 (socket says 172.21.15.5:38434) 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda4d253640 1 -- 172.21.15.5:0/3773173244 learned_addr learned my addr 172.21.15.5:0/3773173244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.070+0000 7fda4da54640 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212070 0x55ff0320df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda4d253640 1 -- 172.21.15.5:0/3773173244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03213760 msgr2=0x55ff03290c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda4e255640 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03213760 0x55ff03290c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda4d253640 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03213760 0x55ff03290c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda4d253640 1 -- 172.21.15.5:0/3773173244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212070 msgr2=0x55ff0320df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda4d253640 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212070 0x55ff0320df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda4d253640 1 -- 172.21.15.5:0/3773173244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff032911a0 con 0x55ff03212de0 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda4e255640 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03213760 0x55ff03290c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda4da54640 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212070 0x55ff0320df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda4d253640 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0320e4d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fda380093e0 tx=0x7fda38005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5034be66308d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda317fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda317fa640 1 -- 172.21.15.5:0/3773173244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda38027050 con 0x55ff03212de0 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda317fa640 1 -- 172.21.15.5:0/3773173244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fda38004d80 con 0x55ff03212de0 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda52091740 1 -- 172.21.15.5:0/3773173244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda380097a0 con 0x55ff03212de0 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda317fa640 1 -- 172.21.15.5:0/3773173244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fda38038040 con 0x55ff03212de0 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda52091740 10 client.16998 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda52091740 1 -- 172.21.15.5:0/3773173244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ff032922d0 con 0x55ff03212de0 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.071+0000 7fda317fa640 1 -- 172.21.15.5:0/3773173244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda380097a0 con 0x55ff03212de0 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.072+0000 7fda317fa640 1 -- 172.21.15.5:0/3773173244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fda380448e0 con 0x55ff03212de0 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.072+0000 7fda52091740 10 client.16998 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.072+0000 7fda52091740 10 client.16998 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:27:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.072+0000 7fda52091740 20 client.16998 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.072+0000 7fda52091740 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ff032929c0 0x55ff03294de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.072+0000 7fda52091740 4 client.16998 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.072+0000 7fda52091740 1 -- 172.21.15.5:0/3773173244 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff03175d70 con 0x55ff032929c0 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.072+0000 7fda4da54640 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ff032929c0 0x55ff03294de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.072+0000 7fda317fa640 10 client.16998 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.072+0000 7fda4da54640 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ff032929c0 0x55ff03294de0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57d336acf809b5b1 server_cookie=74882c735e1db3aa in_seq=0 out_seq=0 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda317fa640 1 -- 172.21.15.5:0/3773173244 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff03175d70 con 0x55ff032929c0 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda317fa640 10 client.16998 handle_command_reply: tid=0 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 client.16998 shutdown 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 -- 172.21.15.5:0/3773173244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 msgr2=0x55ff0320e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0320e4d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fda380093e0 tx=0x7fda38005fc0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 -- 172.21.15.5:0/3773173244 shutdown_connections 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ff032929c0 0x55ff03294de0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff03213760 0x55ff03290c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda317fa640 0 client.16998 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff03212de0 0x55ff0320e4d0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 --2- 172.21.15.5:0/3773173244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff03212070 0x55ff0320df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 -- 172.21.15.5:0/3773173244 >> 172.21.15.5:0/3773173244 conn(0x55ff031d1310 msgr2=0x55ff03219090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 -- 172.21.15.5:0/3773173244 shutdown_connections 2024-09-11T22:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 1 -- 172.21.15.5:0/3773173244 wait complete. 2024-09-11T22:27:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda52091740 20 client.16998 trim_cache size 0 max 16384 2024-09-11T22:27:14.091 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda4fcb8640 1 -- 172.21.15.5:0/1514521348 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda340a2cf0 msgr2=0x7fda340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda4fcb8640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda340a2cf0 0x7fda340a51b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fda40030040 tx=0x7fda400095c0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda4fcb8640 1 -- 172.21.15.5:0/1514521348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda48100e20 msgr2=0x7fda48197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.073+0000 7fda4fcb8640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda48100e20 0x7fda48197460 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fda4400d770 tx=0x7fda4400dc30 comp rx=0 tx=0).stop 2024-09-11T22:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.074+0000 7fda4fcb8640 1 -- 172.21.15.5:0/1514521348 shutdown_connections 2024-09-11T22:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.074+0000 7fda4fcb8640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda48102ef0 0x7fda4819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.074+0000 7fda4fcb8640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fda340a2cf0 0x7fda340a51b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.074+0000 7fda4fcb8640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda48102570 0x7fda481979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.074+0000 7fda4fcb8640 1 --2- 172.21.15.5:0/1514521348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda48100e20 0x7fda48197460 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.074+0000 7fda4fcb8640 1 -- 172.21.15.5:0/1514521348 >> 172.21.15.5:0/1514521348 conn(0x7fda480fc7d0 msgr2=0x7fda481068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.074+0000 7fda4fcb8640 1 -- 172.21.15.5:0/1514521348 shutdown_connections 2024-09-11T22:27:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:14.074+0000 7fda4fcb8640 1 -- 172.21.15.5:0/1514521348 wait complete. 2024-09-11T22:27:14.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337570 7.20 MB/sec execute 92 sec latency 3442.993 ms 2024-09-11T22:27:14.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:27:15.115 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:15.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337570 7.12 MB/sec execute 93 sec latency 4443.076 ms 2024-09-11T22:27:15.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.627+0000 7f896c47b640 1 -- 172.21.15.5:0/3636413741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8964102790 msgr2=0x7f89641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:15.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.627+0000 7f896c47b640 1 --2- 172.21.15.5:0/3636413741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8964102790 0x7f89641067e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8954009900 tx=0x7f895402a410 comp rx=0 tx=0).stop 2024-09-11T22:27:15.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.627+0000 7f896c47b640 1 -- 172.21.15.5:0/3636413741 shutdown_connections 2024-09-11T22:27:15.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.627+0000 7f896c47b640 1 --2- 172.21.15.5:0/3636413741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8964106f10 0x7f896410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.627+0000 7f896c47b640 1 --2- 172.21.15.5:0/3636413741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8964102790 0x7f89641067e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.627+0000 7f896c47b640 1 --2- 172.21.15.5:0/3636413741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8964101de0 0x7f89641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.627+0000 7f896c47b640 1 -- 172.21.15.5:0/3636413741 >> 172.21.15.5:0/3636413741 conn(0x7f89640fd8c0 msgr2=0x7f89640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.627+0000 7f896c47b640 1 -- 172.21.15.5:0/3636413741 shutdown_connections 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.627+0000 7f896c47b640 1 -- 172.21.15.5:0/3636413741 wait complete. 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f896c47b640 1 Processor -- start 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f896c47b640 1 -- start start 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f896c47b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8964101de0 0x7f89641a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f896c47b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8964102790 0x7f89641a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f896c47b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8964106f10 0x7f896419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f896c47b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8964112010 con 0x7f8964102790 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f896c47b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8964111e90 con 0x7f8964101de0 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f896c47b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8964112190 con 0x7f8964106f10 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f8969a16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8964102790 0x7f89641a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f8969a16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8964102790 0x7f89641a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38450/0 (socket says 172.21.15.5:38450) 2024-09-11T22:27:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f8969a16640 1 -- 172.21.15.5:0/2391252802 learned_addr learned my addr 172.21.15.5:0/2391252802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f896aa18640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8964106f10 0x7f896419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f896a217640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8964101de0 0x7f89641a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f8969a16640 1 -- 172.21.15.5:0/2391252802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8964106f10 msgr2=0x7f896419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f8969a16640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8964106f10 0x7f896419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f8969a16640 1 -- 172.21.15.5:0/2391252802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8964101de0 msgr2=0x7f89641a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f8969a16640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8964101de0 0x7f89641a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.628+0000 7f8969a16640 1 -- 172.21.15.5:0/2391252802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896419ad00 con 0x7f8964102790 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.629+0000 7f8969a16640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8964102790 0x7f89641a0950 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8954005ab0 tx=0x7f89540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.629+0000 7f895b7fe640 1 -- 172.21.15.5:0/2391252802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895403f030 con 0x7f8964102790 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.629+0000 7f895b7fe640 1 -- 172.21.15.5:0/2391252802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8954003ce0 con 0x7f8964102790 2024-09-11T22:27:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.629+0000 7f895b7fe640 1 -- 172.21.15.5:0/2391252802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954035070 con 0x7f8964102790 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.629+0000 7f896c47b640 1 -- 172.21.15.5:0/2391252802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896419af90 con 0x7f8964102790 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.629+0000 7f896c47b640 1 -- 172.21.15.5:0/2391252802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89641a72e0 con 0x7f8964102790 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.631+0000 7f895b7fe640 1 -- 172.21.15.5:0/2391252802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8954042030 con 0x7f8964102790 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.631+0000 7f896e854740 1 Processor -- start 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896e854740 1 -- start start 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896e854740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93db40 0x55777b93c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896e854740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93c5d0 0x55777b93ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896e854740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b949760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896e854740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55777b8a22f0 con 0x55777b93db40 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896e854740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55777b849cf0 con 0x55777b947370 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896e854740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55777b89e2b0 con 0x55777b93c5d0 2024-09-11T22:27:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896aa18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93db40 0x55777b93c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f895b7fe640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89500a2cf0 0x7f89500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896aa18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93db40 0x55777b93c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38452/0 (socket says 172.21.15.5:38452) 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896aa18640 1 -- 172.21.15.5:0/2953832141 learned_addr learned my addr 172.21.15.5:0/2953832141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896a217640 1 --2- 172.21.15.5:0/2953832141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93c5d0 0x55777b93ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f8969a16640 1 --2- 172.21.15.5:0/2953832141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b949760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896aa18640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89500a2cf0 0x7f89500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f895b7fe640 1 -- 172.21.15.5:0/2391252802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f89540efd80 con 0x7f8964102790 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896aa18640 1 -- 172.21.15.5:0/2953832141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93c5d0 msgr2=0x55777b93ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896aa18640 1 --2- 172.21.15.5:0/2953832141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93c5d0 0x55777b93ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896aa18640 1 -- 172.21.15.5:0/2953832141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 msgr2=0x55777b949760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896aa18640 1 --2- 172.21.15.5:0/2953832141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b949760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f896aa18640 1 -- 172.21.15.5:0/2953832141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777b82e3a0 con 0x55777b93db40 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.632+0000 7f8969a16640 1 --2- 172.21.15.5:0/2953832141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b949760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.633+0000 7f896aa18640 1 --2- 172.21.15.5:0/2953832141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93db40 0x55777b93c090 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f895c00a990 tx=0x7f895c00ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13888510018dfc50 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.633+0000 7f8958ff9640 1 -- 172.21.15.5:0/2953832141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c01d030 con 0x55777b93db40 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.633+0000 7f8958ff9640 1 -- 172.21.15.5:0/2953832141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c015c60 con 0x55777b93db40 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.633+0000 7f8958ff9640 1 -- 172.21.15.5:0/2953832141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895c0213c0 con 0x55777b93db40 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 -- 172.21.15.5:0/2953832141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93db40 msgr2=0x55777b93c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896aa18640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89500a2cf0 0x7f89500a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f895c020000 tx=0x7f895c01e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 --2- 172.21.15.5:0/2953832141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93db40 0x55777b93c090 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f895c00a990 tx=0x7f895c00ae50 comp rx=0 tx=0).stop 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 -- 172.21.15.5:0/2953832141 shutdown_connections 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 --2- 172.21.15.5:0/2953832141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b949760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 --2- 172.21.15.5:0/2953832141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93c5d0 0x55777b93ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 --2- 172.21.15.5:0/2953832141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93db40 0x55777b93c090 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 -- 172.21.15.5:0/2953832141 >> 172.21.15.5:0/2953832141 conn(0x55777b905310 msgr2=0x55777b938c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 -- 172.21.15.5:0/2953832141 shutdown_connections 2024-09-11T22:27:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 -- 172.21.15.5:0/2953832141 wait complete. 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 -- start start 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 Processor -- start 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b946d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93db40 0x55777b943ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b9445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55777b94a070 con 0x55777b93c5d0 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55777b85d060 con 0x55777b947370 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.634+0000 7f896e854740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55777b94a1f0 con 0x55777b93db40 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896e854740 10 client.0 _ll_register_callbacks cb 0x55777b5b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896a217640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b946d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896a217640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b946d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38464/0 (socket says 172.21.15.5:38464) 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896a217640 1 -- 172.21.15.5:0/2788918278 learned_addr learned my addr 172.21.15.5:0/2788918278 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f8969a16640 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93db40 0x55777b943ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896aa18640 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b9445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896a217640 1 -- 172.21.15.5:0/2788918278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93db40 msgr2=0x55777b943ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896a217640 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93db40 0x55777b943ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896a217640 1 -- 172.21.15.5:0/2788918278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 msgr2=0x55777b9445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896a217640 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b9445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896a217640 1 -- 172.21.15.5:0/2788918278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777b6133c0 con 0x55777b93c5d0 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896a217640 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b946d40 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f89600104d0 tx=0x7f8960010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3090e8b6b0f95d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f894dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f894dffb640 1 -- 172.21.15.5:0/2788918278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896001a020 con 0x55777b93c5d0 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896e854740 1 -- 172.21.15.5:0/2788918278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f895c015c60 con 0x55777b93c5d0 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f894dffb640 1 -- 172.21.15.5:0/2788918278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f8960016690 con 0x55777b93c5d0 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f894dffb640 1 -- 172.21.15.5:0/2788918278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8960025050 con 0x55777b93c5d0 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896e854740 10 client.17010 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.635+0000 7f896e854740 1 -- 172.21.15.5:0/2788918278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55777b955b40 con 0x55777b93c5d0 2024-09-11T22:27:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.636+0000 7f894dffb640 1 -- 172.21.15.5:0/2788918278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f895c015c60 con 0x55777b93c5d0 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.636+0000 7f894dffb640 1 -- 172.21.15.5:0/2788918278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f8960014a90 con 0x55777b93c5d0 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.636+0000 7f896e854740 10 client.17010 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.636+0000 7f896e854740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.636+0000 7f896e854740 20 client.17010 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.636+0000 7f896e854740 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777b957000 0x55777b9593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.636+0000 7f896e854740 4 client.17010 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.636+0000 7f896e854740 1 -- 172.21.15.5:0/2788918278 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55777b8af860 con 0x55777b957000 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.636+0000 7f8969a16640 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777b957000 0x55777b9593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.637+0000 7f894dffb640 10 client.17010 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.637+0000 7f8969a16640 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777b957000 0x55777b9593f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b760a4b17645c9e server_cookie=4d9e8ef012fb7797 in_seq=0 out_seq=0 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.637+0000 7f894dffb640 1 -- 172.21.15.5:0/2788918278 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55777b8af860 con 0x55777b957000 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.637+0000 7f894dffb640 10 client.17010 handle_command_reply: tid=0 2024-09-11T22:27:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.637+0000 7f896e854740 1 client.17010 shutdown 2024-09-11T22:27:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 1 -- 172.21.15.5:0/2788918278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 msgr2=0x55777b946d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b946d40 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f89600104d0 tx=0x7f8960010990 comp rx=0 tx=0).stop 2024-09-11T22:27:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 1 -- 172.21.15.5:0/2788918278 shutdown_connections 2024-09-11T22:27:15.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777b957000 0x55777b9593f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b9445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93db40 0x55777b943ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f894dffb640 0 client.17010 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 1 --2- 172.21.15.5:0/2788918278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b946d40 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 1 -- 172.21.15.5:0/2788918278 >> 172.21.15.5:0/2788918278 conn(0x55777b905310 msgr2=0x55777b93e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 1 -- 172.21.15.5:0/2788918278 shutdown_connections 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 1 -- 172.21.15.5:0/2788918278 wait complete. 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.638+0000 7f896e854740 20 client.17010 trim_cache size 0 max 16384 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.651+0000 7f896e854740 1 Processor -- start 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.651+0000 7f896e854740 1 -- start start 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.651+0000 7f896e854740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b9438b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.651+0000 7f896e854740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93db40 0x55777b93f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.651+0000 7f896e854740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b93f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.651+0000 7f896e854740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89641afc20 con 0x55777b93c5d0 2024-09-11T22:27:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896e854740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b1760 con 0x55777b947370 2024-09-11T22:27:15.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896e854740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b02f0 con 0x55777b93db40 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896a217640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b9438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896a217640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b9438b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38474/0 (socket says 172.21.15.5:38474) 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896a217640 1 -- 172.21.15.5:0/1577477053 learned_addr learned my addr 172.21.15.5:0/1577477053 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f8969a16640 1 --2- 172.21.15.5:0/1577477053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93db40 0x55777b93f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896aa18640 1 --2- 172.21.15.5:0/1577477053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b93f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896a217640 1 -- 172.21.15.5:0/1577477053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93db40 msgr2=0x55777b93f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896a217640 1 --2- 172.21.15.5:0/1577477053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93db40 0x55777b93f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896a217640 1 -- 172.21.15.5:0/1577477053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 msgr2=0x55777b93f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896a217640 1 --2- 172.21.15.5:0/1577477053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b93f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896a217640 1 -- 172.21.15.5:0/1577477053 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777b942fa0 con 0x55777b93c5d0 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f896a217640 1 --2- 172.21.15.5:0/1577477053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b9438b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f896000bad0 tx=0x7f896000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8515f0ccb2b98f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:15.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f894ffff640 1 -- 172.21.15.5:0/1577477053 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896001a050 con 0x55777b93c5d0 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f894ffff640 1 -- 172.21.15.5:0/1577477053 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8960010e90 con 0x55777b93c5d0 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.652+0000 7f894ffff640 1 -- 172.21.15.5:0/1577477053 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89600168f0 con 0x55777b93c5d0 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 -- 172.21.15.5:0/1577477053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 msgr2=0x55777b9438b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 --2- 172.21.15.5:0/1577477053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b9438b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f896000bad0 tx=0x7f896000b400 comp rx=0 tx=0).stop 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 -- 172.21.15.5:0/1577477053 shutdown_connections 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 --2- 172.21.15.5:0/1577477053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b947370 0x55777b93f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 --2- 172.21.15.5:0/1577477053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b93db40 0x55777b93f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 --2- 172.21.15.5:0/1577477053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b9438b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 -- 172.21.15.5:0/1577477053 >> 172.21.15.5:0/1577477053 conn(0x55777b905310 msgr2=0x55777b937d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 -- 172.21.15.5:0/1577477053 shutdown_connections 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 -- 172.21.15.5:0/1577477053 wait complete. 2024-09-11T22:27:15.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 -- start start 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.653+0000 7f896e854740 1 Processor -- start 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896e854740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b94b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896e854740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b93db40 0x55777b94b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896e854740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b947370 0x55777b94bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896e854740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b1090 con 0x55777b93c5d0 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896e854740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f89641b09c0 con 0x55777b93db40 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896e854740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f89641adc00 con 0x55777b947370 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896e854740 10 client.0 _ll_register_callbacks cb 0x7f896000b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896a217640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b94b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896a217640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b94b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38486/0 (socket says 172.21.15.5:38486) 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896a217640 1 -- 172.21.15.5:0/759574802 learned_addr learned my addr 172.21.15.5:0/759574802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896a217640 1 -- 172.21.15.5:0/759574802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b947370 msgr2=0x55777b94bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:15.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896a217640 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b947370 0x55777b94bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f8969a16640 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b93db40 0x55777b94b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896a217640 1 -- 172.21.15.5:0/759574802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b93db40 msgr2=0x55777b94b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896a217640 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b93db40 0x55777b94b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896a217640 1 -- 172.21.15.5:0/759574802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777b9c0ee0 con 0x55777b93c5d0 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896a217640 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b94b4a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8960014f30 tx=0x7f896000bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41a416fdf16b4d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f894dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f894dffb640 1 -- 172.21.15.5:0/759574802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896001a040 con 0x55777b93c5d0 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f894dffb640 1 -- 172.21.15.5:0/759574802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f8960016730 con 0x55777b93c5d0 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.654+0000 7f896e854740 1 -- 172.21.15.5:0/759574802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8960010e90 con 0x55777b93c5d0 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f894dffb640 1 -- 172.21.15.5:0/759574802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f89600059c0 con 0x55777b93c5d0 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f896e854740 10 client.17016 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f896e854740 1 -- 172.21.15.5:0/759574802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55777b9c21e0 con 0x55777b93c5d0 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f894dffb640 1 -- 172.21.15.5:0/759574802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8960010e90 con 0x55777b93c5d0 2024-09-11T22:27:15.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f894dffb640 1 -- 172.21.15.5:0/759574802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f89600061b0 con 0x55777b93c5d0 2024-09-11T22:27:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f896e854740 10 client.17016 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f896e854740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f896e854740 20 client.17016 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f896e854740 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777b9c2830 0x55777b9c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f896e854740 4 client.17016 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:27:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f896e854740 1 -- 172.21.15.5:0/759574802 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55777b8a9d70 con 0x55777b9c2830 2024-09-11T22:27:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.655+0000 7f8969a16640 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777b9c2830 0x55777b9c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f8969a16640 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777b9c2830 0x55777b9c4c80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af977b9ac60ccd1c server_cookie=fd160fd1727c9ccc in_seq=0 out_seq=0 2024-09-11T22:27:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f894dffb640 10 client.17016 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:15.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f894dffb640 1 -- 172.21.15.5:0/759574802 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55777b8a9d70 con 0x55777b9c2830 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f894dffb640 10 client.17016 handle_command_reply: tid=0 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 client.17016 shutdown 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 -- 172.21.15.5:0/759574802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 msgr2=0x55777b94b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b94b4a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8960014f30 tx=0x7f896000bb00 comp rx=0 tx=0).stop 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 -- 172.21.15.5:0/759574802 shutdown_connections 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777b9c2830 0x55777b9c4c80 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777b947370 0x55777b94bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777b93db40 0x55777b94b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 --2- 172.21.15.5:0/759574802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777b93c5d0 0x55777b94b4a0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 -- 172.21.15.5:0/759574802 >> 172.21.15.5:0/759574802 conn(0x55777b905310 msgr2=0x55777b945850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f894dffb640 0 client.17016 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 -- 172.21.15.5:0/759574802 shutdown_connections 2024-09-11T22:27:15.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 1 -- 172.21.15.5:0/759574802 wait complete. 2024-09-11T22:27:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.656+0000 7f896e854740 20 client.17016 trim_cache size 0 max 16384 2024-09-11T22:27:15.675 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:27:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 -- 172.21.15.5:0/2391252802 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89500a2cf0 msgr2=0x7f89500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:15.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89500a2cf0 0x7f89500a51b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f895c020000 tx=0x7f895c01e000 comp rx=0 tx=0).stop 2024-09-11T22:27:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 -- 172.21.15.5:0/2391252802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8964102790 msgr2=0x7f89641a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8964102790 0x7f89641a0950 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8954005ab0 tx=0x7f89540042f0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 -- 172.21.15.5:0/2391252802 shutdown_connections 2024-09-11T22:27:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8964106f10 0x7f896419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8964102790 0x7f89641a0950 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89500a2cf0 0x7f89500a51b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 --2- 172.21.15.5:0/2391252802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8964101de0 0x7f89641a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 -- 172.21.15.5:0/2391252802 >> 172.21.15.5:0/2391252802 conn(0x7f89640fd8c0 msgr2=0x7f89640fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 -- 172.21.15.5:0/2391252802 shutdown_connections 2024-09-11T22:27:15.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:15.657+0000 7f896c47b640 1 -- 172.21.15.5:0/2391252802 wait complete. 2024-09-11T22:27:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:15 smithi005 ceph-mon[30943]: pgmap v155: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-11T22:27:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:15 smithi159 ceph-mon[35534]: pgmap v155: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-11T22:27:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:15 smithi172 ceph-mon[35461]: pgmap v155: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-11T22:27:16.318 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337570 7.04 MB/sec execute 94 sec latency 5443.206 ms 2024-09-11T22:27:17.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337570 6.97 MB/sec execute 95 sec latency 6443.311 ms 2024-09-11T22:27:17.370 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:27:17.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:27:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:17.773 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:17 smithi005 ceph-mon[30943]: pgmap v156: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-11T22:27:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:17 smithi159 ceph-mon[35534]: pgmap v156: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-11T22:27:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:17 smithi172 ceph-mon[35461]: pgmap v156: 97 pgs: 97 active+clean; 55 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-11T22:27:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.131+0000 7f17b413f640 1 -- 172.21.15.5:0/1238793038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17ac101280 msgr2=0x7f17ac10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.131+0000 7f17b413f640 1 --2- 172.21.15.5:0/1238793038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17ac101280 0x7f17ac10df40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f179c009900 tx=0x7f179c02a410 comp rx=0 tx=0).stop 2024-09-11T22:27:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.131+0000 7f17b413f640 1 -- 172.21.15.5:0/1238793038 shutdown_connections 2024-09-11T22:27:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.131+0000 7f17b413f640 1 --2- 172.21.15.5:0/1238793038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17ac10e480 0x7f17ac110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.131+0000 7f17b413f640 1 --2- 172.21.15.5:0/1238793038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17ac101280 0x7f17ac10df40 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.131+0000 7f17b413f640 1 --2- 172.21.15.5:0/1238793038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17ac1008d0 0x7f17ac100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.131+0000 7f17b413f640 1 -- 172.21.15.5:0/1238793038 >> 172.21.15.5:0/1238793038 conn(0x7f17ac0fc7a0 msgr2=0x7f17ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.132+0000 7f17b413f640 1 -- 172.21.15.5:0/1238793038 shutdown_connections 2024-09-11T22:27:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.132+0000 7f17b413f640 1 -- 172.21.15.5:0/1238793038 wait complete. 2024-09-11T22:27:18.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.132+0000 7f17b413f640 1 Processor -- start 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.132+0000 7f17b413f640 1 -- start start 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.132+0000 7f17b413f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17ac1008d0 0x7f17ac19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.132+0000 7f17b413f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17ac101280 0x7f17ac19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.132+0000 7f17b413f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17ac10e480 0x7f17ac19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.132+0000 7f17b413f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1141e0 con 0x7f17ac101280 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.132+0000 7f17b413f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac114060 con 0x7f17ac1008d0 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.132+0000 7f17b413f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac114360 con 0x7f17ac10e480 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b16da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17ac101280 0x7f17ac19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b16da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17ac101280 0x7f17ac19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38512/0 (socket says 172.21.15.5:38512) 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b16da640 1 -- 172.21.15.5:0/1692399341 learned_addr learned my addr 172.21.15.5:0/1692399341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b26dc640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17ac10e480 0x7f17ac19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b1edb640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17ac1008d0 0x7f17ac19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b16da640 1 -- 172.21.15.5:0/1692399341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17ac10e480 msgr2=0x7f17ac19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b16da640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17ac10e480 0x7f17ac19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b16da640 1 -- 172.21.15.5:0/1692399341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17ac1008d0 msgr2=0x7f17ac19a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b16da640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17ac1008d0 0x7f17ac19a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b16da640 1 -- 172.21.15.5:0/1692399341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ac19f6b0 con 0x7f17ac101280 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b1edb640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17ac1008d0 0x7f17ac19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b16da640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17ac101280 0x7f17ac19ac40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f179c031550 tx=0x7f179c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17a2ffd640 1 -- 172.21.15.5:0/1692399341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c03f030 con 0x7f17ac101280 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17a2ffd640 1 -- 172.21.15.5:0/1692399341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c003ce0 con 0x7f17ac101280 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b413f640 1 -- 172.21.15.5:0/1692399341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ac19f940 con 0x7f17ac101280 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17a2ffd640 1 -- 172.21.15.5:0/1692399341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c035070 con 0x7f17ac101280 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.133+0000 7f17b413f640 1 -- 172.21.15.5:0/1692399341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ac1a7280 con 0x7f17ac101280 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.136+0000 7f17a2ffd640 1 -- 172.21.15.5:0/1692399341 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f179c042030 con 0x7f17ac101280 2024-09-11T22:27:18.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.136+0000 7f17b6518740 1 Processor -- start 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.136+0000 7f17b6518740 1 -- start start 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.136+0000 7f17b6518740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b92b60 0x556a90b92f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.136+0000 7f17b6518740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90b9d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.136+0000 7f17b6518740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b9dd40 0x556a90ba0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.136+0000 7f17b6518740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556a90af8280 con 0x556a90b93480 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.136+0000 7f17b6518740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a90a9fb10 con 0x556a90b9dd40 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.136+0000 7f17b6518740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556a90af4240 con 0x556a90b92b60 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b1edb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90b9d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17a2ffd640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17980a2d70 0x7f17980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b1edb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90b9d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38518/0 (socket says 172.21.15.5:38518) 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b1edb640 1 -- 172.21.15.5:0/2727483504 learned_addr learned my addr 172.21.15.5:0/2727483504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b26dc640 1 --2- 172.21.15.5:0/2727483504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b92b60 0x556a90b92f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b26dc640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17980a2d70 0x7f17980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17a2ffd640 1 -- 172.21.15.5:0/1692399341 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f179c0efe40 con 0x7f17ac101280 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b16da640 1 --2- 172.21.15.5:0/2727483504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b9dd40 0x556a90ba0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b1edb640 1 -- 172.21.15.5:0/2727483504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b92b60 msgr2=0x556a90b92f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b1edb640 1 --2- 172.21.15.5:0/2727483504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b92b60 0x556a90b92f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b1edb640 1 -- 172.21.15.5:0/2727483504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b9dd40 msgr2=0x556a90ba0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b1edb640 1 --2- 172.21.15.5:0/2727483504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b9dd40 0x556a90ba0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b1edb640 1 -- 172.21.15.5:0/2727483504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a90aac1f0 con 0x556a90b93480 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b1edb640 1 --2- 172.21.15.5:0/2727483504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90b9d800 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f17a800bb70 tx=0x7f17a800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbe40da282f0149 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f17b26dc640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17980a2d70 0x7f17980a5230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f17ac06a0f0 tx=0x7f17a4005df0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f1797fff640 1 -- 172.21.15.5:0/2727483504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a8017620 con 0x556a90b93480 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f1797fff640 1 -- 172.21.15.5:0/2727483504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a8017ca0 con 0x556a90b93480 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.137+0000 7f1797fff640 1 -- 172.21.15.5:0/2727483504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a800fc90 con 0x556a90b93480 2024-09-11T22:27:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.138+0000 7f17b6518740 1 -- 172.21.15.5:0/2727483504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 msgr2=0x556a90b9d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.138+0000 7f17b6518740 1 --2- 172.21.15.5:0/2727483504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90b9d800 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f17a800bb70 tx=0x7f17a800d710 comp rx=0 tx=0).stop 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.138+0000 7f17b6518740 1 -- 172.21.15.5:0/2727483504 shutdown_connections 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.138+0000 7f17b6518740 1 --2- 172.21.15.5:0/2727483504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b9dd40 0x556a90ba0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.138+0000 7f17b6518740 1 --2- 172.21.15.5:0/2727483504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90b9d800 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.138+0000 7f17b6518740 1 --2- 172.21.15.5:0/2727483504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b92b60 0x556a90b92f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.138+0000 7f17b6518740 1 -- 172.21.15.5:0/2727483504 >> 172.21.15.5:0/2727483504 conn(0x556a90b8f0f0 msgr2=0x556a90b8f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.138+0000 7f17b6518740 1 -- 172.21.15.5:0/2727483504 shutdown_connections 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.138+0000 7f17b6518740 1 -- 172.21.15.5:0/2727483504 wait complete. 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b6518740 1 -- start start 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b6518740 1 Processor -- start 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b6518740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b92b60 0x556a90b9ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b6518740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 0x556a90b9b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b6518740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b9dd40 0x556a90b9b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b6518740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556a90ba0910 con 0x556a90b9dd40 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b6518740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a90ab2f10 con 0x556a90b92b60 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b6518740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556a90ba0a90 con 0x556a90b93480 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b6518740 10 client.0 _ll_register_callbacks cb 0x556a90a7ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b26dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b9dd40 0x556a90b9b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b26dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b9dd40 0x556a90b9b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38524/0 (socket says 172.21.15.5:38524) 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b26dc640 1 -- 172.21.15.5:0/3338783529 learned_addr learned my addr 172.21.15.5:0/3338783529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.139+0000 7f17b1edb640 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b92b60 0x556a90b9ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17b16da640 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 0x556a90b9b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17b26dc640 1 -- 172.21.15.5:0/3338783529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 msgr2=0x556a90b9b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17b26dc640 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 0x556a90b9b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17b26dc640 1 -- 172.21.15.5:0/3338783529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b92b60 msgr2=0x556a90b9ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17b26dc640 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b92b60 0x556a90b9ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17b26dc640 1 -- 172.21.15.5:0/3338783529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a908696a0 con 0x556a90b9dd40 2024-09-11T22:27:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17b26dc640 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b9dd40 0x556a90b9b5e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f17a4016ef0 tx=0x7f17a4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b742ae9c18c3e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17957fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17957fa640 1 -- 172.21.15.5:0/3338783529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4028060 con 0x556a90b9dd40 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17957fa640 1 -- 172.21.15.5:0/3338783529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f17a401a0d0 con 0x556a90b9dd40 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17b6518740 1 -- 172.21.15.5:0/3338783529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a8017ca0 con 0x556a90b9dd40 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17957fa640 1 -- 172.21.15.5:0/3338783529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f17a4006720 con 0x556a90b9dd40 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17b6518740 10 client.17028 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.140+0000 7f17b6518740 1 -- 172.21.15.5:0/3338783529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a90bac460 con 0x556a90b9dd40 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.141+0000 7f17957fa640 1 -- 172.21.15.5:0/3338783529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a8017ca0 con 0x556a90b9dd40 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.141+0000 7f17957fa640 1 -- 172.21.15.5:0/3338783529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f17a4023410 con 0x556a90b9dd40 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.141+0000 7f17b6518740 10 client.17028 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.141+0000 7f17b6518740 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.141+0000 7f17b6518740 20 client.17028 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.141+0000 7f17b6518740 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556a90bad920 0x556a90bafd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.141+0000 7f17b6518740 4 client.17028 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.141+0000 7f17b6518740 1 -- 172.21.15.5:0/3338783529 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a90b057f0 con 0x556a90bad920 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.141+0000 7f17b1edb640 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556a90bad920 0x556a90bafd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.142+0000 7f17b1edb640 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556a90bad920 0x556a90bafd70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db28fb81d28faadf server_cookie=4c44986ca5d4e9b1 in_seq=0 out_seq=0 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.142+0000 7f17957fa640 10 client.17028 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.142+0000 7f17957fa640 1 -- 172.21.15.5:0/3338783529 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a90b057f0 con 0x556a90bad920 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.142+0000 7f17957fa640 10 client.17028 handle_command_reply: tid=0 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.142+0000 7f17b6518740 1 client.17028 shutdown 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 1 -- 172.21.15.5:0/3338783529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b9dd40 msgr2=0x556a90b9b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b9dd40 0x556a90b9b5e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f17a4016ef0 tx=0x7f17a4004b50 comp rx=0 tx=0).stop 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 1 -- 172.21.15.5:0/3338783529 shutdown_connections 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556a90bad920 0x556a90bafd70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b9dd40 0x556a90b9b5e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 0x556a90b9b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17957fa640 0 client.17028 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 1 --2- 172.21.15.5:0/3338783529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b92b60 0x556a90b9ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 1 -- 172.21.15.5:0/3338783529 >> 172.21.15.5:0/3338783529 conn(0x556a90b8f0f0 msgr2=0x556a90b94dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 1 -- 172.21.15.5:0/3338783529 shutdown_connections 2024-09-11T22:27:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 1 -- 172.21.15.5:0/3338783529 wait complete. 2024-09-11T22:27:18.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.143+0000 7f17b6518740 20 client.17028 trim_cache size 0 max 16384 2024-09-11T22:27:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.160+0000 7f17b6518740 1 Processor -- start 2024-09-11T22:27:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.160+0000 7f17b6518740 1 -- start start 2024-09-11T22:27:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b6518740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b92b60 0x556a90b94cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b6518740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90bb5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b6518740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b9dd40 0x556a90bb5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b6518740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1afcf0 con 0x556a90b93480 2024-09-11T22:27:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b6518740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1b1830 con 0x556a90b92b60 2024-09-11T22:27:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b6518740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1b03c0 con 0x556a90b9dd40 2024-09-11T22:27:18.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b16da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90bb5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b16da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90bb5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38528/0 (socket says 172.21.15.5:38528) 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b16da640 1 -- 172.21.15.5:0/2367066425 learned_addr learned my addr 172.21.15.5:0/2367066425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b26dc640 1 --2- 172.21.15.5:0/2367066425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b9dd40 0x556a90bb5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b1edb640 1 --2- 172.21.15.5:0/2367066425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b92b60 0x556a90b94cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b16da640 1 -- 172.21.15.5:0/2367066425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b9dd40 msgr2=0x556a90bb5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b16da640 1 --2- 172.21.15.5:0/2367066425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b9dd40 0x556a90bb5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b16da640 1 -- 172.21.15.5:0/2367066425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b92b60 msgr2=0x556a90b94cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b16da640 1 --2- 172.21.15.5:0/2367066425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b92b60 0x556a90b94cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b16da640 1 -- 172.21.15.5:0/2367066425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a90b8ef20 con 0x556a90b93480 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.161+0000 7f17b16da640 1 --2- 172.21.15.5:0/2367066425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90bb5640 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f179c0ed5b0 tx=0x7f179c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d6623a103dc16b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17977fe640 1 -- 172.21.15.5:0/2367066425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c0d5070 con 0x556a90b93480 2024-09-11T22:27:18.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17977fe640 1 -- 172.21.15.5:0/2367066425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c0bc320 con 0x556a90b93480 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17977fe640 1 -- 172.21.15.5:0/2367066425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c049030 con 0x556a90b93480 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17b6518740 1 -- 172.21.15.5:0/2367066425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 msgr2=0x556a90bb5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17b6518740 1 --2- 172.21.15.5:0/2367066425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90bb5640 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f179c0ed5b0 tx=0x7f179c02a410 comp rx=0 tx=0).stop 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17b6518740 1 -- 172.21.15.5:0/2367066425 shutdown_connections 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17b6518740 1 --2- 172.21.15.5:0/2367066425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b9dd40 0x556a90bb5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17b6518740 1 --2- 172.21.15.5:0/2367066425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b93480 0x556a90bb5640 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17b6518740 1 --2- 172.21.15.5:0/2367066425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b92b60 0x556a90b94cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17b6518740 1 -- 172.21.15.5:0/2367066425 >> 172.21.15.5:0/2367066425 conn(0x556a90b8f0f0 msgr2=0x556a90b945b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17b6518740 1 -- 172.21.15.5:0/2367066425 shutdown_connections 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.162+0000 7f17b6518740 1 -- 172.21.15.5:0/2367066425 wait complete. 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b6518740 1 -- start start 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b6518740 1 Processor -- start 2024-09-11T22:27:18.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b6518740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b92b60 0x556a90ba5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b6518740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 0x556a90ba6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b6518740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b9dd40 0x556a90ba0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b6518740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1b1160 con 0x556a90b92b60 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b6518740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1b0a90 con 0x556a90b9dd40 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b6518740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f17ac1adcd0 con 0x556a90b93480 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b6518740 10 client.0 _ll_register_callbacks cb 0x7f17a4008070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b1edb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b92b60 0x556a90ba5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b1edb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b92b60 0x556a90ba5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38532/0 (socket says 172.21.15.5:38532) 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.163+0000 7f17b1edb640 1 -- 172.21.15.5:0/2594073941 learned_addr learned my addr 172.21.15.5:0/2594073941 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b26dc640 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b9dd40 0x556a90ba0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b16da640 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 0x556a90ba6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b1edb640 1 -- 172.21.15.5:0/2594073941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 msgr2=0x556a90ba6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b1edb640 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 0x556a90ba6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b1edb640 1 -- 172.21.15.5:0/2594073941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b9dd40 msgr2=0x556a90ba0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b1edb640 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b9dd40 0x556a90ba0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b1edb640 1 -- 172.21.15.5:0/2594073941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a90ba11e0 con 0x556a90b92b60 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b16da640 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 0x556a90ba6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b1edb640 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b92b60 0x556a90ba5cf0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f17a800ba90 tx=0x7f17a800f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375c2b002aa66bbb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17957fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17957fa640 1 -- 172.21.15.5:0/2594073941 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a800f620 con 0x556a90b92b60 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17957fa640 1 -- 172.21.15.5:0/2594073941 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f17a800f9e0 con 0x556a90b92b60 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b6518740 1 -- 172.21.15.5:0/2594073941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f179c0bc320 con 0x556a90b92b60 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17957fa640 1 -- 172.21.15.5:0/2594073941 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f17a8039050 con 0x556a90b92b60 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b6518740 10 client.17034 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17b6518740 1 -- 172.21.15.5:0/2594073941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a90c18d20 con 0x556a90b92b60 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.164+0000 7f17957fa640 1 -- 172.21.15.5:0/2594073941 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c0bc320 con 0x556a90b92b60 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.165+0000 7f17957fa640 1 -- 172.21.15.5:0/2594073941 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f17a8017900 con 0x556a90b92b60 2024-09-11T22:27:18.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.165+0000 7f17b6518740 10 client.17034 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:18.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.165+0000 7f17b6518740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:18.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.165+0000 7f17b6518740 20 client.17034 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:18.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.165+0000 7f17b6518740 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556a90c191c0 0x556a90c1b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:18.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.165+0000 7f17b6518740 4 client.17034 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:27:18.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.165+0000 7f17b6518740 1 -- 172.21.15.5:0/2594073941 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556a90affd00 con 0x556a90c191c0 2024-09-11T22:27:18.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.165+0000 7f17b16da640 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556a90c191c0 0x556a90c1b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:18.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.165+0000 7f17b16da640 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556a90c191c0 0x556a90c1b5b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f6978470d09bd0b server_cookie=f394f62aa30ab3c6 in_seq=0 out_seq=0 2024-09-11T22:27:18.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.165+0000 7f17957fa640 10 client.17034 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:18.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17957fa640 1 -- 172.21.15.5:0/2594073941 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556a90affd00 con 0x556a90c191c0 2024-09-11T22:27:18.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17957fa640 10 client.17034 handle_command_reply: tid=0 2024-09-11T22:27:18.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 client.17034 shutdown 2024-09-11T22:27:18.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 -- 172.21.15.5:0/2594073941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b92b60 msgr2=0x556a90ba5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:18.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b92b60 0x556a90ba5cf0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f17a800ba90 tx=0x7f17a800f970 comp rx=0 tx=0).stop 2024-09-11T22:27:18.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 -- 172.21.15.5:0/2594073941 shutdown_connections 2024-09-11T22:27:18.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556a90c191c0 0x556a90c1b5b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a90b9dd40 0x556a90ba0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556a90b93480 0x556a90ba6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17957fa640 0 client.17034 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 --2- 172.21.15.5:0/2594073941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a90b92b60 0x556a90ba5cf0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 -- 172.21.15.5:0/2594073941 >> 172.21.15.5:0/2594073941 conn(0x556a90b8f0f0 msgr2=0x556a90b9b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 -- 172.21.15.5:0/2594073941 shutdown_connections 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 1 -- 172.21.15.5:0/2594073941 wait complete. 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.167+0000 7f17b6518740 20 client.17034 trim_cache size 0 max 16384 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "572cd3fb-377b-4c27-af61-0d2a171a942f", 2024-09-11T22:27:18.174 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:27:18.175 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:27:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 -- 172.21.15.5:0/1692399341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17980a2d70 msgr2=0x7f17980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17980a2d70 0x7f17980a5230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f17ac06a0f0 tx=0x7f17a4005df0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 -- 172.21.15.5:0/1692399341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17ac101280 msgr2=0x7f17ac19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17ac101280 0x7f17ac19ac40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f179c031550 tx=0x7f179c004360 comp rx=0 tx=0).stop 2024-09-11T22:27:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 -- 172.21.15.5:0/1692399341 shutdown_connections 2024-09-11T22:27:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17ac10e480 0x7f17ac19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17ac101280 0x7f17ac19ac40 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f17980a2d70 0x7f17980a5230 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 --2- 172.21.15.5:0/1692399341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17ac1008d0 0x7f17ac19a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 -- 172.21.15.5:0/1692399341 >> 172.21.15.5:0/1692399341 conn(0x7f17ac0fc7a0 msgr2=0x7f17ac0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 -- 172.21.15.5:0/1692399341 shutdown_connections 2024-09-11T22:27:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:18.168+0000 7f17b413f640 1 -- 172.21.15.5:0/1692399341 wait complete. 2024-09-11T22:27:18.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337570 6.90 MB/sec execute 96 sec latency 7443.430 ms 2024-09-11T22:27:18.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:27:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:19.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337570 6.83 MB/sec execute 97 sec latency 8443.498 ms 2024-09-11T22:27:19.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:19 smithi005 ceph-mon[30943]: pgmap v157: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-11T22:27:19.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:19 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:27:19.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:19 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:27:19.630 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:19 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:27:19.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:19 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:27:19.631 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:19 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:27:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.729+0000 7fe561391640 1 -- 172.21.15.5:0/201628181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1029b0 msgr2=0x7fe55c102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.729+0000 7fe561391640 1 --2- 172.21.15.5:0/201628181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1029b0 0x7fe55c102d90 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009900 tx=0x7fe54c02a410 comp rx=0 tx=0).stop 2024-09-11T22:27:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.729+0000 7fe561391640 1 -- 172.21.15.5:0/201628181 shutdown_connections 2024-09-11T22:27:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.729+0000 7fe561391640 1 --2- 172.21.15.5:0/201628181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c101a30 0x7fe55c10d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.729+0000 7fe561391640 1 --2- 172.21.15.5:0/201628181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c101070 0x7fe55c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.729+0000 7fe561391640 1 --2- 172.21.15.5:0/201628181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1029b0 0x7fe55c102d90 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.729+0000 7fe561391640 1 -- 172.21.15.5:0/201628181 >> 172.21.15.5:0/201628181 conn(0x7fe55c0770f0 msgr2=0x7fe55c0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.729+0000 7fe561391640 1 -- 172.21.15.5:0/201628181 shutdown_connections 2024-09-11T22:27:19.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.729+0000 7fe561391640 1 -- 172.21.15.5:0/201628181 wait complete. 2024-09-11T22:27:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.715+0000 7fae84b52640 1 -- 172.21.15.5:0/3309680563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae80101e10 msgr2=0x7fae801021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.715+0000 7fae84b52640 1 --2- 172.21.15.5:0/3309680563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae80101e10 0x7fae801021f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fae70009900 tx=0x7fae7002a3f0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.715+0000 7fae84b52640 1 -- 172.21.15.5:0/3309680563 shutdown_connections 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.715+0000 7fae84b52640 1 --2- 172.21.15.5:0/3309680563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae80106f40 0x7fae8010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.715+0000 7fae84b52640 1 --2- 172.21.15.5:0/3309680563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae801027c0 0x7fae80106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.715+0000 7fae84b52640 1 --2- 172.21.15.5:0/3309680563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae80101e10 0x7fae801021f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.715+0000 7fae84b52640 1 -- 172.21.15.5:0/3309680563 >> 172.21.15.5:0/3309680563 conn(0x7fae800fd8d0 msgr2=0x7fae800ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.715+0000 7fae84b52640 1 -- 172.21.15.5:0/3309680563 shutdown_connections 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.715+0000 7fae84b52640 1 -- 172.21.15.5:0/3309680563 wait complete. 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.716+0000 7fae84b52640 1 Processor -- start 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.716+0000 7fae84b52640 1 -- start start 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.716+0000 7fae84b52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae80101e10 0x7fae800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.716+0000 7fae84b52640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae801027c0 0x7fae80074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.716+0000 7fae84b52640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae80106f40 0x7fae80074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.716+0000 7fae84b52640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae80111fa0 con 0x7fae80101e10 2024-09-11T22:27:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.716+0000 7fae84b52640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae80111e20 con 0x7fae801027c0 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.716+0000 7fae84b52640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae80112120 con 0x7fae80106f40 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.716+0000 7fae7e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae80101e10 0x7fae800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.716+0000 7fae7e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae80101e10 0x7fae800744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38564/0 (socket says 172.21.15.5:38564) 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae7e59c640 1 -- 172.21.15.5:0/4035157676 learned_addr learned my addr 172.21.15.5:0/4035157676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae7ed9d640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae80106f40 0x7fae80074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae7dd9b640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae801027c0 0x7fae80074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae7e59c640 1 -- 172.21.15.5:0/4035157676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae80106f40 msgr2=0x7fae80074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae7e59c640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae80106f40 0x7fae80074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae7e59c640 1 -- 172.21.15.5:0/4035157676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae801027c0 msgr2=0x7fae80074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae7e59c640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae801027c0 0x7fae80074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae7e59c640 1 -- 172.21.15.5:0/4035157676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae801a6800 con 0x7fae80101e10 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae7e59c640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae80101e10 0x7fae800744f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fae70005ab0 tx=0x7fae700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae6f7fe640 1 -- 172.21.15.5:0/4035157676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7003f030 con 0x7fae80101e10 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae84b52640 1 -- 172.21.15.5:0/4035157676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae801a6a90 con 0x7fae80101e10 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae6f7fe640 1 -- 172.21.15.5:0/4035157676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae70003ce0 con 0x7fae80101e10 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae6f7fe640 1 -- 172.21.15.5:0/4035157676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70035070 con 0x7fae80101e10 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.717+0000 7fae84b52640 1 -- 172.21.15.5:0/4035157676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae801a6f70 con 0x7fae80101e10 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.720+0000 7fae6f7fe640 1 -- 172.21.15.5:0/4035157676 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fae70042030 con 0x7fae80101e10 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.720+0000 7fae86f2b740 1 Processor -- start 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.720+0000 7fae86f2b740 1 -- start start 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.720+0000 7fae6f7fe640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fae680a2cf0 0x7fae680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.720+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e21e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e229140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e229680 0x56142e22ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae86f2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56142e1842e0 con 0x56142e21e4a0 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae86f2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56142e12bce0 con 0x56142e229680 2024-09-11T22:27:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae86f2b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56142e1802a0 con 0x56142e21edc0 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae6f7fe640 1 -- 172.21.15.5:0/4035157676 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fae700efe30 con 0x7fae80101e10 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e21e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e21e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38574/0 (socket says 172.21.15.5:38574) 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7e59c640 1 -- 172.21.15.5:0/3513678363 learned_addr learned my addr 172.21.15.5:0/3513678363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7ed9d640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fae680a2cf0 0x7fae680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7dd9b640 1 --2- 172.21.15.5:0/3513678363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e229140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7ed9d640 1 --2- 172.21.15.5:0/3513678363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e229680 0x56142e22ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7e59c640 1 -- 172.21.15.5:0/3513678363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 msgr2=0x56142e229140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7e59c640 1 --2- 172.21.15.5:0/3513678363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e229140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7e59c640 1 -- 172.21.15.5:0/3513678363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e229680 msgr2=0x56142e22ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7e59c640 1 --2- 172.21.15.5:0/3513678363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e229680 0x56142e22ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7e59c640 1 -- 172.21.15.5:0/3513678363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142e110440 con 0x56142e21e4a0 2024-09-11T22:27:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7e59c640 1 --2- 172.21.15.5:0/3513678363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e21e880 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fae700bc010 tx=0x7fae7002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c194b70e8eacacbf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae7ed9d640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fae680a2cf0 0x7fae680a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x56142e21bdb0 tx=0x7fae7800a680 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.721+0000 7fae6cff9640 1 -- 172.21.15.5:0/3513678363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae70049040 con 0x56142e21e4a0 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae6cff9640 1 -- 172.21.15.5:0/3513678363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae700bca10 con 0x56142e21e4a0 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae6cff9640 1 -- 172.21.15.5:0/3513678363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7003bd70 con 0x56142e21e4a0 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae86f2b740 1 -- 172.21.15.5:0/3513678363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 msgr2=0x56142e21e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae86f2b740 1 --2- 172.21.15.5:0/3513678363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e21e880 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fae700bc010 tx=0x7fae7002a3f0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae86f2b740 1 -- 172.21.15.5:0/3513678363 shutdown_connections 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae86f2b740 1 --2- 172.21.15.5:0/3513678363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e229680 0x56142e22ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae86f2b740 1 --2- 172.21.15.5:0/3513678363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e229140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae86f2b740 1 --2- 172.21.15.5:0/3513678363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e21e880 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae86f2b740 1 -- 172.21.15.5:0/3513678363 >> 172.21.15.5:0/3513678363 conn(0x56142e21aa30 msgr2=0x56142e21ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae86f2b740 1 -- 172.21.15.5:0/3513678363 shutdown_connections 2024-09-11T22:27:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.722+0000 7fae86f2b740 1 -- 172.21.15.5:0/3513678363 wait complete. 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae86f2b740 1 -- start start 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae86f2b740 1 Processor -- start 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e228fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e226280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e229680 0x56142e2267e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae86f2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56142e1842e0 con 0x56142e21e4a0 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae86f2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56142e13f050 con 0x56142e229680 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae86f2b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56142e22c010 con 0x56142e21edc0 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae86f2b740 10 client.0 _ll_register_callbacks cb 0x56142e17f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae7e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e228fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae7e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e228fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38586/0 (socket says 172.21.15.5:38586) 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae7e59c640 1 -- 172.21.15.5:0/660695323 learned_addr learned my addr 172.21.15.5:0/660695323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.723+0000 7fae7ed9d640 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e229680 0x56142e2267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae7dd9b640 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e226280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae7e59c640 1 -- 172.21.15.5:0/660695323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 msgr2=0x56142e226280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae7e59c640 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e226280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae7e59c640 1 -- 172.21.15.5:0/660695323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e229680 msgr2=0x56142e2267e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae7e59c640 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e229680 0x56142e2267e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae7e59c640 1 -- 172.21.15.5:0/660695323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142def53c0 con 0x56142e21e4a0 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae7dd9b640 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e226280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae7e59c640 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e228fd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fae700820d0 tx=0x7fae700c0730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d923021bb0d6c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae65ffb640 1 -- 172.21.15.5:0/660695323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700d5060 con 0x56142e21e4a0 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae65ffb640 1 -- 172.21.15.5:0/660695323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fae700ed7c0 con 0x56142e21e4a0 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae86f2b740 1 -- 172.21.15.5:0/660695323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae700bca10 con 0x56142e21e4a0 2024-09-11T22:27:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae65ffb640 1 -- 172.21.15.5:0/660695323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae70049050 con 0x56142e21e4a0 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae86f2b740 10 client.17046 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae86f2b740 1 -- 172.21.15.5:0/660695323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56142e2377d0 con 0x56142e21e4a0 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae65ffb640 1 -- 172.21.15.5:0/660695323 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae700bca10 con 0x56142e21e4a0 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.724+0000 7fae65ffb640 1 -- 172.21.15.5:0/660695323 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fae7003b460 con 0x56142e21e4a0 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.725+0000 7fae86f2b740 10 client.17046 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.725+0000 7fae86f2b740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.725+0000 7fae86f2b740 20 client.17046 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.725+0000 7fae86f2b740 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56142e2389c0 0x56142e23adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.725+0000 7fae86f2b740 4 client.17046 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.725+0000 7fae86f2b740 1 -- 172.21.15.5:0/660695323 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56142e191850 con 0x56142e2389c0 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.725+0000 7fae7dd9b640 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56142e2389c0 0x56142e23adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.726+0000 7fae7dd9b640 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56142e2389c0 0x56142e23adf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a7c1662c7538ed9 server_cookie=4e97136486995fc3 in_seq=0 out_seq=0 2024-09-11T22:27:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.726+0000 7fae65ffb640 10 client.17046 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.726+0000 7fae65ffb640 1 -- 172.21.15.5:0/660695323 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56142e191850 con 0x56142e2389c0 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.726+0000 7fae65ffb640 10 client.17046 handle_command_reply: tid=0 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.726+0000 7fae86f2b740 1 client.17046 shutdown 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 1 -- 172.21.15.5:0/660695323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 msgr2=0x56142e228fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e228fd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fae700820d0 tx=0x7fae700c0730 comp rx=0 tx=0).stop 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 1 -- 172.21.15.5:0/660695323 shutdown_connections 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56142e2389c0 0x56142e23adf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e229680 0x56142e2267e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e226280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae65ffb640 0 client.17046 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 1 --2- 172.21.15.5:0/660695323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21e4a0 0x56142e228fd0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 1 -- 172.21.15.5:0/660695323 >> 172.21.15.5:0/660695323 conn(0x56142e21aa30 msgr2=0x56142e220760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 1 -- 172.21.15.5:0/660695323 shutdown_connections 2024-09-11T22:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 1 -- 172.21.15.5:0/660695323 wait complete. 2024-09-11T22:27:19.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.727+0000 7fae86f2b740 20 client.17046 trim_cache size 0 max 16384 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe561391640 1 Processor -- start 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe561391640 1 -- start start 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe561391640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c101070 0x7fe55c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe561391640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c101a30 0x7fe55c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe561391640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1029b0 0x7fe55c19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe561391640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c110ed0 con 0x7fe55c1029b0 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe561391640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c110d50 con 0x7fe55c101070 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe561391640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c111050 con 0x7fe55c101a30 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1029b0 0x7fe55c19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1029b0 0x7fe55c19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38608/0 (socket says 172.21.15.5:38608) 2024-09-11T22:27:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe55b7fe640 1 -- 172.21.15.5:0/3962610475 learned_addr learned my addr 172.21.15.5:0/3962610475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe55affd640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c101070 0x7fe55c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.730+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c101a30 0x7fe55c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe55b7fe640 1 -- 172.21.15.5:0/3962610475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c101a30 msgr2=0x7fe55c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe55b7fe640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c101a30 0x7fe55c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe55b7fe640 1 -- 172.21.15.5:0/3962610475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c101070 msgr2=0x7fe55c198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe55b7fe640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c101070 0x7fe55c198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe55b7fe640 1 -- 172.21.15.5:0/3962610475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c19d380 con 0x7fe55c1029b0 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c101a30 0x7fe55c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe55b7fe640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1029b0 0x7fe55c19cc50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe55400eb10 tx=0x7fe55400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe543fff640 1 -- 172.21.15.5:0/3962610475 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554004b20 con 0x7fe55c1029b0 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe543fff640 1 -- 172.21.15.5:0/3962610475 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe554004cc0 con 0x7fe55c1029b0 2024-09-11T22:27:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe561391640 1 -- 172.21.15.5:0/3962610475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c19d670 con 0x7fe55c1029b0 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe543fff640 1 -- 172.21.15.5:0/3962610475 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554010760 con 0x7fe55c1029b0 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.731+0000 7fe561391640 1 -- 172.21.15.5:0/3962610475 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c1a4f00 con 0x7fe55c1029b0 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe564209740 1 Processor -- start 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe543fff640 1 -- 172.21.15.5:0/3962610475 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe554005020 con 0x7fe55c1029b0 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe564209740 1 -- start start 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe564209740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b8080 0x561a0c1b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe564209740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1b6ed0 0x561a0c1b7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe564209740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1ca350 0x561a0c1cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe564209740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a0c11b2f0 con 0x561a0c1ca350 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe564209740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561a0c0c3270 con 0x561a0c1b6ed0 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe564209740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561a0c1172b0 con 0x561a0c1b8080 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe55a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1ca350 0x561a0c1cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe55a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1ca350 0x561a0c1cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38620/0 (socket says 172.21.15.5:38620) 2024-09-11T22:27:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.734+0000 7fe55a7fc640 1 -- 172.21.15.5:0/2051882559 learned_addr learned my addr 172.21.15.5:0/2051882559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe55affd640 1 --2- 172.21.15.5:0/2051882559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1b6ed0 0x561a0c1b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/2051882559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b8080 0x561a0c1b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe543fff640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5440a2e20 0x7fe5440a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe55a7fc640 1 -- 172.21.15.5:0/2051882559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b8080 msgr2=0x561a0c1b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/2051882559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b8080 0x561a0c1b6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe55a7fc640 1 -- 172.21.15.5:0/2051882559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1b6ed0 msgr2=0x561a0c1b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/2051882559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1b6ed0 0x561a0c1b7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe55a7fc640 1 -- 172.21.15.5:0/2051882559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0c0c2340 con 0x561a0c1ca350 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe55b7fe640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5440a2e20 0x7fe5440a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe543fff640 1 -- 172.21.15.5:0/3962610475 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe5540cfa70 con 0x7fe55c1029b0 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/2051882559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1ca350 0x561a0c1cc740 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe55000bfa0 tx=0x7fe55000b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c07c89e95eabff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe5417fa640 1 -- 172.21.15.5:0/2051882559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550018030 con 0x561a0c1ca350 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe5417fa640 1 -- 172.21.15.5:0/2051882559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe550005020 con 0x561a0c1ca350 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.735+0000 7fe5417fa640 1 -- 172.21.15.5:0/2051882559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5500049e0 con 0x561a0c1ca350 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe564209740 1 -- 172.21.15.5:0/2051882559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1ca350 msgr2=0x561a0c1cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe564209740 1 --2- 172.21.15.5:0/2051882559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1ca350 0x561a0c1cc740 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe55000bfa0 tx=0x7fe55000b660 comp rx=0 tx=0).stop 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe55b7fe640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5440a2e20 0x7fe5440a52e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5540049d0 tx=0x7fe5540058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe564209740 1 -- 172.21.15.5:0/2051882559 shutdown_connections 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe564209740 1 --2- 172.21.15.5:0/2051882559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1ca350 0x561a0c1cc740 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe564209740 1 --2- 172.21.15.5:0/2051882559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1b6ed0 0x561a0c1b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe564209740 1 --2- 172.21.15.5:0/2051882559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b8080 0x561a0c1b6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe564209740 1 -- 172.21.15.5:0/2051882559 >> 172.21.15.5:0/2051882559 conn(0x561a0c1b3060 msgr2=0x561a0c1b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe564209740 1 -- 172.21.15.5:0/2051882559 shutdown_connections 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe564209740 1 -- 172.21.15.5:0/2051882559 wait complete. 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.736+0000 7fe564209740 1 -- start start 2024-09-11T22:27:19.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe564209740 1 Processor -- start 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe564209740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1b6ed0 0x561a0c1ba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe564209740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe564209740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1ca350 0x561a0c1bb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe564209740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a0c1cd0b0 con 0x561a0c1b8080 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe564209740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561a0c0a4290 con 0x561a0c1b6ed0 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe564209740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561a0c1cd230 con 0x561a0c1ca350 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe564209740 10 client.0 _ll_register_callbacks cb 0x561a0c0d5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38628/0 (socket says 172.21.15.5:38628) 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55affd640 1 -- 172.21.15.5:0/3410963463 learned_addr learned my addr 172.21.15.5:0/3410963463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1b6ed0 0x561a0c1ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1ca350 0x561a0c1bb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55affd640 1 -- 172.21.15.5:0/3410963463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1ca350 msgr2=0x561a0c1bb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55affd640 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1ca350 0x561a0c1bb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55affd640 1 -- 172.21.15.5:0/3410963463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1b6ed0 msgr2=0x561a0c1ba740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55affd640 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1b6ed0 0x561a0c1ba740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55affd640 1 -- 172.21.15.5:0/3410963463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0be2e210 con 0x561a0c1b8080 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1ca350 0x561a0c1bb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe55affd640 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bac80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0314e0 tx=0x7fe54c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940931d47f2a8851 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.737+0000 7fe53eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe53eff5640 1 -- 172.21.15.5:0/3410963463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c035070 con 0x561a0c1b8080 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe53eff5640 1 -- 172.21.15.5:0/3410963463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe54c003a00 con 0x561a0c1b8080 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe564209740 1 -- 172.21.15.5:0/3410963463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe550005020 con 0x561a0c1b8080 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe53eff5640 1 -- 172.21.15.5:0/3410963463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe54c003c00 con 0x561a0c1b8080 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe564209740 10 client.17058 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:19.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe564209740 1 -- 172.21.15.5:0/3410963463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a0befa620 con 0x561a0c1b8080 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe53eff5640 1 -- 172.21.15.5:0/3410963463 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe550005020 con 0x561a0c1b8080 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe53eff5640 1 -- 172.21.15.5:0/3410963463 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe54c03b430 con 0x561a0c1b8080 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe564209740 10 client.17058 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe564209740 10 client.17058 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe564209740 20 client.17058 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe564209740 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561a0c1da4d0 0x561a0c1dc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe564209740 4 client.17058 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.738+0000 7fe564209740 1 -- 172.21.15.5:0/3410963463 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a0c128a20 con 0x561a0c1da4d0 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.739+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561a0c1da4d0 0x561a0c1dc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.739+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561a0c1da4d0 0x561a0c1dc8e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a30603e533cad7 server_cookie=a8bfe488796ca33e in_seq=0 out_seq=0 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.739+0000 7fe53eff5640 10 client.17058 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe53eff5640 1 -- 172.21.15.5:0/3410963463 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a0c128a20 con 0x561a0c1da4d0 2024-09-11T22:27:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe53eff5640 10 client.17058 handle_command_reply: tid=0 2024-09-11T22:27:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 client.17058 shutdown 2024-09-11T22:27:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 -- 172.21.15.5:0/3410963463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 msgr2=0x561a0c1bac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bac80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe54c0314e0 tx=0x7fe54c0043b0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 -- 172.21.15.5:0/3410963463 shutdown_connections 2024-09-11T22:27:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561a0c1da4d0 0x561a0c1dc8e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1ca350 0x561a0c1bb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bac80 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe53eff5640 0 client.17058 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 --2- 172.21.15.5:0/3410963463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1b6ed0 0x561a0c1ba740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 -- 172.21.15.5:0/3410963463 >> 172.21.15.5:0/3410963463 conn(0x561a0c1b3060 msgr2=0x561a0c1b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 -- 172.21.15.5:0/3410963463 shutdown_connections 2024-09-11T22:27:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 1 -- 172.21.15.5:0/3410963463 wait complete. 2024-09-11T22:27:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.740+0000 7fe564209740 20 client.17058 trim_cache size 0 max 16384 2024-09-11T22:27:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.746+0000 7fae86f2b740 1 Processor -- start 2024-09-11T22:27:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.746+0000 7fae86f2b740 1 -- start start 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.746+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e21e4a0 0x56142e224b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.746+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e240710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.746+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e229680 0x56142e240c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.746+0000 7fae86f2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae801afa10 con 0x56142e229680 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.746+0000 7fae86f2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae801b1550 con 0x56142e21e4a0 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.746+0000 7fae86f2b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae801b00e0 con 0x56142e21edc0 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e229680 0x56142e240c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e229680 0x56142e240c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38638/0 (socket says 172.21.15.5:38638) 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7ed9d640 1 -- 172.21.15.5:0/2319017649 learned_addr learned my addr 172.21.15.5:0/2319017649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7dd9b640 1 --2- 172.21.15.5:0/2319017649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e240710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7e59c640 1 --2- 172.21.15.5:0/2319017649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e21e4a0 0x56142e224b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7ed9d640 1 -- 172.21.15.5:0/2319017649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 msgr2=0x56142e240710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7ed9d640 1 --2- 172.21.15.5:0/2319017649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e240710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7ed9d640 1 -- 172.21.15.5:0/2319017649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e21e4a0 msgr2=0x56142e224b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7ed9d640 1 --2- 172.21.15.5:0/2319017649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e21e4a0 0x56142e224b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7ed9d640 1 -- 172.21.15.5:0/2319017649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142e225230 con 0x56142e229680 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae7ed9d640 1 --2- 172.21.15.5:0/2319017649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e229680 0x56142e240c50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fae78013180 tx=0x7fae78004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b5526e8f98b75c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae67fff640 1 -- 172.21.15.5:0/2319017649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78028480 con 0x56142e229680 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.747+0000 7fae67fff640 1 -- 172.21.15.5:0/2319017649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78028b00 con 0x56142e229680 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.748+0000 7fae67fff640 1 -- 172.21.15.5:0/2319017649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78012b80 con 0x56142e229680 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.748+0000 7fae86f2b740 1 -- 172.21.15.5:0/2319017649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e229680 msgr2=0x56142e240c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.748+0000 7fae86f2b740 1 --2- 172.21.15.5:0/2319017649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e229680 0x56142e240c50 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fae78013180 tx=0x7fae78004ab0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.748+0000 7fae86f2b740 1 -- 172.21.15.5:0/2319017649 shutdown_connections 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.748+0000 7fae86f2b740 1 --2- 172.21.15.5:0/2319017649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e229680 0x56142e240c50 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.748+0000 7fae86f2b740 1 --2- 172.21.15.5:0/2319017649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e21edc0 0x56142e240710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.748+0000 7fae86f2b740 1 --2- 172.21.15.5:0/2319017649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e21e4a0 0x56142e224b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.748+0000 7fae86f2b740 1 -- 172.21.15.5:0/2319017649 >> 172.21.15.5:0/2319017649 conn(0x56142e21aa30 msgr2=0x56142e219b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.748+0000 7fae86f2b740 1 -- 172.21.15.5:0/2319017649 shutdown_connections 2024-09-11T22:27:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.748+0000 7fae86f2b740 1 -- 172.21.15.5:0/2319017649 wait complete. 2024-09-11T22:27:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae86f2b740 1 -- start start 2024-09-11T22:27:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae86f2b740 1 Processor -- start 2024-09-11T22:27:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e21e4a0 0x56142e22eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21edc0 0x56142e22f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae86f2b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e229680 0x56142e22f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae86f2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae801b0e80 con 0x56142e21edc0 2024-09-11T22:27:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae86f2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae801b07b0 con 0x56142e21e4a0 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae86f2b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fae801ad9f0 con 0x56142e229680 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae7dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21edc0 0x56142e22f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae86f2b740 10 client.0 _ll_register_callbacks cb 0x7fae700daf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae7dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21edc0 0x56142e22f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38640/0 (socket says 172.21.15.5:38640) 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae7dd9b640 1 -- 172.21.15.5:0/3348734945 learned_addr learned my addr 172.21.15.5:0/3348734945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae7ed9d640 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e229680 0x56142e22f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae7dd9b640 1 -- 172.21.15.5:0/3348734945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e229680 msgr2=0x56142e22f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.749+0000 7fae7dd9b640 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e229680 0x56142e22f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae7dd9b640 1 -- 172.21.15.5:0/3348734945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e21e4a0 msgr2=0x56142e22eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae7dd9b640 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e21e4a0 0x56142e22eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae7dd9b640 1 -- 172.21.15.5:0/3348734945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142e2a26f0 con 0x56142e21edc0 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae7dd9b640 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21edc0 0x56142e22f410 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x56142e225b10 tx=0x7fae74012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e4b6d4abad52f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae65ffb640 1 -- 172.21.15.5:0/3348734945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74004460 con 0x56142e21edc0 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae65ffb640 1 -- 172.21.15.5:0/3348734945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fae74011910 con 0x56142e21edc0 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae86f2b740 1 -- 172.21.15.5:0/3348734945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae78028b00 con 0x56142e21edc0 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae65ffb640 1 -- 172.21.15.5:0/3348734945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae7402b050 con 0x56142e21edc0 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae86f2b740 10 client.17064 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae86f2b740 1 -- 172.21.15.5:0/3348734945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56142e2a3a10 con 0x56142e21edc0 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.750+0000 7fae65ffb640 1 -- 172.21.15.5:0/3348734945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78028b00 con 0x56142e21edc0 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.751+0000 7fae65ffb640 1 -- 172.21.15.5:0/3348734945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fae7401b300 con 0x56142e21edc0 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.751+0000 7fae86f2b740 10 client.17064 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.751+0000 7fae86f2b740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.751+0000 7fae86f2b740 20 client.17064 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.751+0000 7fae86f2b740 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56142e2a41c0 0x56142e2a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.751+0000 7fae86f2b740 4 client.17064 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.751+0000 7fae86f2b740 1 -- 172.21.15.5:0/3348734945 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56142e18bd60 con 0x56142e2a41c0 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.751+0000 7fae7e59c640 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56142e2a41c0 0x56142e2a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.751+0000 7fae7e59c640 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56142e2a41c0 0x56142e2a65e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed29572cb59bf92 server_cookie=828b2bb2f39a513 in_seq=0 out_seq=0 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.751+0000 7fae65ffb640 10 client.17064 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae65ffb640 1 -- 172.21.15.5:0/3348734945 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x56142e18bd60 con 0x56142e2a41c0 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae65ffb640 10 client.17064 handle_command_reply: tid=0 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 client.17064 shutdown 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 -- 172.21.15.5:0/3348734945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21edc0 msgr2=0x56142e22f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21edc0 0x56142e22f410 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x56142e225b10 tx=0x7fae74012b80 comp rx=0 tx=0).stop 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 -- 172.21.15.5:0/3348734945 shutdown_connections 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56142e2a41c0 0x56142e2a65e0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56142e229680 0x56142e22f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56142e21edc0 0x56142e22f410 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 --2- 172.21.15.5:0/3348734945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56142e21e4a0 0x56142e22eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 -- 172.21.15.5:0/3348734945 >> 172.21.15.5:0/3348734945 conn(0x56142e21aa30 msgr2=0x56142e219b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae65ffb640 0 client.17064 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 -- 172.21.15.5:0/3348734945 shutdown_connections 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 1 -- 172.21.15.5:0/3348734945 wait complete. 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae86f2b740 20 client.17064 trim_cache size 0 max 16384 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stdout: "572cd3fb-377b-4c27-af61-0d2a171a942f": { 2024-09-11T22:27:19.769 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:27:19.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "572cd3fb-377b-4c27-af61-0d2a171a942f", 2024-09-11T22:27:19.770 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:27:19.770 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:27:19.770 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:27:19.770 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:27:19.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae84b52640 1 -- 172.21.15.5:0/4035157676 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fae680a2cf0 msgr2=0x7fae680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae84b52640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fae680a2cf0 0x7fae680a51b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x56142e21bdb0 tx=0x7fae7800a680 comp rx=0 tx=0).stop 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae84b52640 1 -- 172.21.15.5:0/4035157676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae80101e10 msgr2=0x7fae800744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.752+0000 7fae84b52640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae80101e10 0x7fae800744f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fae70005ab0 tx=0x7fae700042f0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.753+0000 7fae84b52640 1 -- 172.21.15.5:0/4035157676 shutdown_connections 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.753+0000 7fae84b52640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae80106f40 0x7fae80074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.753+0000 7fae84b52640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fae680a2cf0 0x7fae680a51b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.753+0000 7fae84b52640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae801027c0 0x7fae80074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.753+0000 7fae84b52640 1 --2- 172.21.15.5:0/4035157676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae80101e10 0x7fae800744f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.753+0000 7fae84b52640 1 -- 172.21.15.5:0/4035157676 >> 172.21.15.5:0/4035157676 conn(0x7fae800fd8d0 msgr2=0x7fae800ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.753+0000 7fae84b52640 1 -- 172.21.15.5:0/4035157676 shutdown_connections 2024-09-11T22:27:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:19.753+0000 7fae84b52640 1 -- 172.21.15.5:0/4035157676 wait complete. 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.758+0000 7fe564209740 1 Processor -- start 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.758+0000 7fe564209740 1 -- start start 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.758+0000 7fe564209740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b6ed0 0x561a0c1b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.758+0000 7fe564209740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.758+0000 7fe564209740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1ca350 0x561a0c1e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.758+0000 7fe564209740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ab6b0 con 0x561a0c1b8080 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.758+0000 7fe564209740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ad1f0 con 0x561a0c1ca350 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.758+0000 7fe564209740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1abd80 con 0x561a0c1b6ed0 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.758+0000 7fe55affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe55affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38654/0 (socket says 172.21.15.5:38654) 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe55affd640 1 -- 172.21.15.5:0/2372641572 learned_addr learned my addr 172.21.15.5:0/2372641572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/2372641572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1ca350 0x561a0c1e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/2372641572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b6ed0 0x561a0c1b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe55affd640 1 -- 172.21.15.5:0/2372641572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b6ed0 msgr2=0x561a0c1b9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe55affd640 1 --2- 172.21.15.5:0/2372641572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b6ed0 0x561a0c1b9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe55affd640 1 -- 172.21.15.5:0/2372641572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1ca350 msgr2=0x561a0c1e21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe55affd640 1 --2- 172.21.15.5:0/2372641572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1ca350 0x561a0c1e21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe55affd640 1 -- 172.21.15.5:0/2372641572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0c1b9a40 con 0x561a0c1b8080 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe55affd640 1 --2- 172.21.15.5:0/2372641572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bc6b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009850 tx=0x7fe54c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e971dcbebefcb10 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe540ff9640 1 -- 172.21.15.5:0/2372641572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c051070 con 0x561a0c1b8080 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe540ff9640 1 -- 172.21.15.5:0/2372641572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c005510 con 0x561a0c1b8080 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.759+0000 7fe540ff9640 1 -- 172.21.15.5:0/2372641572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c035070 con 0x561a0c1b8080 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.760+0000 7fe564209740 1 -- 172.21.15.5:0/2372641572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 msgr2=0x561a0c1bc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.760+0000 7fe564209740 1 --2- 172.21.15.5:0/2372641572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bc6b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009850 tx=0x7fe54c02a410 comp rx=0 tx=0).stop 2024-09-11T22:27:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.760+0000 7fe564209740 1 -- 172.21.15.5:0/2372641572 shutdown_connections 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.760+0000 7fe564209740 1 --2- 172.21.15.5:0/2372641572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1ca350 0x561a0c1e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.760+0000 7fe564209740 1 --2- 172.21.15.5:0/2372641572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b8080 0x561a0c1bc6b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.760+0000 7fe564209740 1 --2- 172.21.15.5:0/2372641572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b6ed0 0x561a0c1b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.760+0000 7fe564209740 1 -- 172.21.15.5:0/2372641572 >> 172.21.15.5:0/2372641572 conn(0x561a0c1b3060 msgr2=0x561a0c1bbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.760+0000 7fe564209740 1 -- 172.21.15.5:0/2372641572 shutdown_connections 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.760+0000 7fe564209740 1 -- 172.21.15.5:0/2372641572 wait complete. 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.760+0000 7fe564209740 1 -- start start 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe564209740 1 Processor -- start 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe564209740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b6ed0 0x561a0c1d24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe564209740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b8080 0x561a0c1d2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe564209740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1ca350 0x561a0c1e1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe564209740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1acb20 con 0x561a0c1b6ed0 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe564209740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ac450 con 0x561a0c1ca350 2024-09-11T22:27:19.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe564209740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1a9690 con 0x561a0c1b8080 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe564209740 10 client.0 _ll_register_callbacks cb 0x7fe54c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b6ed0 0x561a0c1d24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b6ed0 0x561a0c1d24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38658/0 (socket says 172.21.15.5:38658) 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55a7fc640 1 -- 172.21.15.5:0/804672804 learned_addr learned my addr 172.21.15.5:0/804672804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55affd640 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b8080 0x561a0c1d2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1ca350 0x561a0c1e1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55a7fc640 1 -- 172.21.15.5:0/804672804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b8080 msgr2=0x561a0c1d2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b8080 0x561a0c1d2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55a7fc640 1 -- 172.21.15.5:0/804672804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1ca350 msgr2=0x561a0c1e1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1ca350 0x561a0c1e1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55a7fc640 1 -- 172.21.15.5:0/804672804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0c1d3070 con 0x561a0c1b6ed0 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe55a7fc640 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b6ed0 0x561a0c1d24f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe550012540 tx=0x7fe54c005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aadf32b59b942bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:19.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.761+0000 7fe53eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe53eff5640 1 -- 172.21.15.5:0/804672804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe550018040 con 0x561a0c1b6ed0 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe564209740 1 -- 172.21.15.5:0/804672804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe54c005510 con 0x561a0c1b6ed0 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe53eff5640 1 -- 172.21.15.5:0/804672804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe550010720 con 0x561a0c1b6ed0 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe53eff5640 1 -- 172.21.15.5:0/804672804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe550010920 con 0x561a0c1b6ed0 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe564209740 10 client.17070 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe564209740 1 -- 172.21.15.5:0/804672804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a0c2456d0 con 0x561a0c1b6ed0 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe53eff5640 1 -- 172.21.15.5:0/804672804 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c005510 con 0x561a0c1b6ed0 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe53eff5640 1 -- 172.21.15.5:0/804672804 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe55002b3b0 con 0x561a0c1b6ed0 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe564209740 10 client.17070 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe564209740 10 client.17070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe564209740 20 client.17070 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe564209740 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561a0c245c40 0x561a0c248060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:19.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe564209740 4 client.17070 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:27:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.762+0000 7fe564209740 1 -- 172.21.15.5:0/804672804 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a0c122d70 con 0x561a0c245c40 2024-09-11T22:27:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.763+0000 7fe55affd640 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561a0c245c40 0x561a0c248060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.763+0000 7fe55affd640 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561a0c245c40 0x561a0c248060 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9fff848400daa70 server_cookie=1cc8362f459a5768 in_seq=0 out_seq=0 2024-09-11T22:27:19.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.763+0000 7fe53eff5640 10 client.17070 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:19 smithi159 ceph-mon[35534]: pgmap v157: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-11T22:27:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:19 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:27:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:19 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:27:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:19 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:27:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:19 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:27:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:19 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:27:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:19 smithi172 ceph-mon[35461]: pgmap v157: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-11T22:27:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:19 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:27:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:19 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:27:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:19 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:27:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:19 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:27:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:19 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:27:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe53eff5640 1 -- 172.21.15.5:0/804672804 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a0c122d70 con 0x561a0c245c40 2024-09-11T22:27:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe53eff5640 10 client.17070 handle_command_reply: tid=0 2024-09-11T22:27:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 client.17070 shutdown 2024-09-11T22:27:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 -- 172.21.15.5:0/804672804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b6ed0 msgr2=0x561a0c1d24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b6ed0 0x561a0c1d24f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe550012540 tx=0x7fe54c005de0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 -- 172.21.15.5:0/804672804 shutdown_connections 2024-09-11T22:27:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561a0c245c40 0x561a0c248060 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe53eff5640 0 client.17070 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561a0c1ca350 0x561a0c1e1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561a0c1b8080 0x561a0c1d2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 --2- 172.21.15.5:0/804672804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0c1b6ed0 0x561a0c1d24f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 -- 172.21.15.5:0/804672804 >> 172.21.15.5:0/804672804 conn(0x561a0c1b3060 msgr2=0x561a0c1baf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 -- 172.21.15.5:0/804672804 shutdown_connections 2024-09-11T22:27:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 1 -- 172.21.15.5:0/804672804 wait complete. 2024-09-11T22:27:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.878+0000 7fe564209740 20 client.17070 trim_cache size 0 max 16384 2024-09-11T22:27:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:27:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:27:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:27:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:27:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 -- 172.21.15.5:0/3962610475 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5440a2e20 msgr2=0x7fe5440a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5440a2e20 0x7fe5440a52e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe5540049d0 tx=0x7fe5540058d0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 -- 172.21.15.5:0/3962610475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1029b0 msgr2=0x7fe55c19cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1029b0 0x7fe55c19cc50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe55400eb10 tx=0x7fe55400c2d0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 -- 172.21.15.5:0/3962610475 shutdown_connections 2024-09-11T22:27:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1029b0 0x7fe55c19cc50 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe55c101a30 0x7fe55c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5440a2e20 0x7fe5440a52e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 --2- 172.21.15.5:0/3962610475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c101070 0x7fe55c198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 -- 172.21.15.5:0/3962610475 >> 172.21.15.5:0/3962610475 conn(0x7fe55c0770f0 msgr2=0x7fe55c1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 -- 172.21.15.5:0/3962610475 shutdown_connections 2024-09-11T22:27:19.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:19.879+0000 7fe561391640 1 -- 172.21.15.5:0/3962610475 wait complete. 2024-09-11T22:27:20.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 338637 6.81 MB/sec execute 98 sec latency 8971.459 ms 2024-09-11T22:27:20.634 INFO:tasks.cephfs.filesystem:scrub status for tag:572cd3fb-377b-4c27-af61-0d2a171a942f - {'path': '/', 'tag': '572cd3fb-377b-4c27-af61-0d2a171a942f', 'options': 'recursive,force'} 2024-09-11T22:27:21.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 341064 6.88 MB/sec execute 99 sec latency 12.734 ms 2024-09-11T22:27:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:21 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:27:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:21 smithi005 ceph-mon[30943]: pgmap v158: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-09-11T22:27:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:21 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:27:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:21 smithi159 ceph-mon[35534]: pgmap v158: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-09-11T22:27:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:21 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:27:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:21 smithi172 ceph-mon[35461]: pgmap v158: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.4 MiB/s wr, 106 op/s 2024-09-11T22:27:22.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 343506 6.94 MB/sec execute 100 sec latency 14.924 ms 2024-09-11T22:27:23.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 345629 6.99 MB/sec execute 101 sec latency 14.326 ms 2024-09-11T22:27:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:23 smithi005 ceph-mon[30943]: pgmap v159: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T22:27:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:23 smithi159 ceph-mon[35534]: pgmap v159: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T22:27:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:23 smithi172 ceph-mon[35461]: pgmap v159: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 202 op/s 2024-09-11T22:27:24.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 348148 7.05 MB/sec execute 102 sec latency 12.362 ms 2024-09-11T22:27:24.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:24 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:27:24.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:24 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:27:24.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:24 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:27:24.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:24 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:27:24.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:24 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:27:24.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:24 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:27:25.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 350539 7.11 MB/sec execute 103 sec latency 20.757 ms 2024-09-11T22:27:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:25 smithi005 ceph-mon[30943]: pgmap v160: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-11T22:27:25.828 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:25 smithi159 ceph-mon[35534]: pgmap v160: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-11T22:27:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:25 smithi172 ceph-mon[35461]: pgmap v160: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-11T22:27:26.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 352568 7.15 MB/sec execute 104 sec latency 41.891 ms 2024-09-11T22:27:27.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 354980 7.21 MB/sec execute 105 sec latency 14.696 ms 2024-09-11T22:27:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:27 smithi005 ceph-mon[30943]: pgmap v161: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-11T22:27:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:27 smithi159 ceph-mon[35534]: pgmap v161: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-11T22:27:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:27 smithi172 ceph-mon[35461]: pgmap v161: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-11T22:27:28.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 357108 7.22 MB/sec execute 106 sec latency 18.235 ms 2024-09-11T22:27:29.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359229 7.27 MB/sec execute 107 sec latency 14.429 ms 2024-09-11T22:27:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:29 smithi005 ceph-mon[30943]: pgmap v162: 97 pgs: 97 active+clean; 62 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-09-11T22:27:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:29 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:27:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:29 smithi159 ceph-mon[35534]: pgmap v162: 97 pgs: 97 active+clean; 62 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-09-11T22:27:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:29 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:27:29.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:29 smithi172 ceph-mon[35461]: pgmap v162: 97 pgs: 97 active+clean; 62 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-09-11T22:27:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:29 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:27:30.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 361805 7.35 MB/sec execute 108 sec latency 22.586 ms 2024-09-11T22:27:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:30 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:27:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:30 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:27:30.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:30 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:27:30.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:30 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:27:30.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:30 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:27:30.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:30 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:27:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:30 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:27:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:30 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:27:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:30 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:27:31.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 363652 7.35 MB/sec execute 109 sec latency 17.053 ms 2024-09-11T22:27:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:31 smithi005 ceph-mon[30943]: pgmap v163: 97 pgs: 97 active+clean; 62 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-09-11T22:27:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:31 smithi159 ceph-mon[35534]: pgmap v163: 97 pgs: 97 active+clean; 62 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-09-11T22:27:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:31 smithi172 ceph-mon[35461]: pgmap v163: 97 pgs: 97 active+clean; 62 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-09-11T22:27:32.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 366006 7.41 MB/sec execute 110 sec latency 15.572 ms 2024-09-11T22:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.971+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1884176596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c810e430 msgr2=0x7fe4c8110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.971+0000 7fe4ce23c640 1 --2- 172.21.15.5:0/1884176596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c810e430 0x7fe4c8110820 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009ca0 tx=0x7fe4c002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.971+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1884176596 shutdown_connections 2024-09-11T22:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.971+0000 7fe4ce23c640 1 --2- 172.21.15.5:0/1884176596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c810e430 0x7fe4c8110820 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.971+0000 7fe4ce23c640 1 --2- 172.21.15.5:0/1884176596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c8101230 0x7fe4c810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.971+0000 7fe4ce23c640 1 --2- 172.21.15.5:0/1884176596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c8100880 0x7fe4c8100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.971+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1884176596 >> 172.21.15.5:0/1884176596 conn(0x7fe4c8077990 msgr2=0x7fe4c80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.971+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1884176596 shutdown_connections 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.971+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1884176596 wait complete. 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4ce23c640 1 Processor -- start 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4ce23c640 1 -- start start 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4ce23c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c8100880 0x7fe4c819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4ce23c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c8101230 0x7fe4c819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4ce23c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c810e430 0x7fe4c819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4ce23c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8114100 con 0x7fe4c810e430 2024-09-11T22:27:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4ce23c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8113f80 con 0x7fe4c8100880 2024-09-11T22:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4ce23c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8114280 con 0x7fe4c8101230 2024-09-11T22:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c810e430 0x7fe4c819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c8100880 0x7fe4c819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c810e430 0x7fe4c819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46760/0 (socket says 172.21.15.5:46760) 2024-09-11T22:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4c7fff640 1 -- 172.21.15.5:0/1296400877 learned_addr learned my addr 172.21.15.5:0/1296400877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.972+0000 7fe4c6ffd640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c8101230 0x7fe4c819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4c77fe640 1 -- 172.21.15.5:0/1296400877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c8101230 msgr2=0x7fe4c819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c8101230 0x7fe4c819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4c77fe640 1 -- 172.21.15.5:0/1296400877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c810e430 msgr2=0x7fe4c819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c810e430 0x7fe4c819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4c77fe640 1 -- 172.21.15.5:0/1296400877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c819f540 con 0x7fe4c8100880 2024-09-11T22:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c8100880 0x7fe4c819a550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800e9f0 tx=0x7fe4b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4c4ff9640 1 -- 172.21.15.5:0/1296400877 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b800ccc0 con 0x7fe4c8100880 2024-09-11T22:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4c4ff9640 1 -- 172.21.15.5:0/1296400877 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b8005020 con 0x7fe4c8100880 2024-09-11T22:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1296400877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c819f830 con 0x7fe4c8100880 2024-09-11T22:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4c4ff9640 1 -- 172.21.15.5:0/1296400877 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80049e0 con 0x7fe4c8100880 2024-09-11T22:27:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.973+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1296400877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c81a70c0 con 0x7fe4c8100880 2024-09-11T22:27:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4c4ff9640 1 -- 172.21.15.5:0/1296400877 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe4b8005560 con 0x7fe4c8100880 2024-09-11T22:27:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4d0fd5740 1 Processor -- start 2024-09-11T22:27:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4d0fd5740 1 -- start start 2024-09-11T22:27:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657992af0 0x557657992ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657993410 0x55765799d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55765799dcb0 0x5576579a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576578f72f0 con 0x557657993410 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55765789f270 con 0x55765799dcb0 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5576578f32b0 con 0x557657992af0 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657993410 0x55765799d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657993410 0x55765799d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46768/0 (socket says 172.21.15.5:46768) 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4c77fe640 1 -- 172.21.15.5:0/1587437864 learned_addr learned my addr 172.21.15.5:0/1587437864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4c6ffd640 1 --2- 172.21.15.5:0/1587437864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55765799dcb0 0x5576579a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.976+0000 7fe4c7fff640 1 --2- 172.21.15.5:0/1587437864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657992af0 0x557657992ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c4ff9640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4b00a2da0 0x7fe4b00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c77fe640 1 -- 172.21.15.5:0/1587437864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657992af0 msgr2=0x557657992ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/1587437864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657992af0 0x557657992ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c77fe640 1 -- 172.21.15.5:0/1587437864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55765799dcb0 msgr2=0x5576579a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/1587437864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55765799dcb0 0x5576579a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c77fe640 1 -- 172.21.15.5:0/1587437864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55765789e340 con 0x557657993410 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c7fff640 1 --2- 172.21.15.5:0/1587437864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657992af0 0x557657992ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c7fff640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4b00a2da0 0x7fe4b00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c4ff9640 1 -- 172.21.15.5:0/1296400877 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe4b80cf3e0 con 0x7fe4c8100880 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/1587437864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657993410 0x55765799d770 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80cca60 tx=0x7fe4b8013b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea87296075421a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4adffb640 1 -- 172.21.15.5:0/1587437864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b809b400 con 0x557657993410 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4adffb640 1 -- 172.21.15.5:0/1587437864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b80051d0 con 0x557657993410 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4adffb640 1 -- 172.21.15.5:0/1587437864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b809b220 con 0x557657993410 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.977+0000 7fe4c7fff640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4b00a2da0 0x7fe4b00a5260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009a60 tx=0x7fe4c003f040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.978+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/1587437864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657993410 msgr2=0x55765799d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.978+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/1587437864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657993410 0x55765799d770 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80cca60 tx=0x7fe4b8013b60 comp rx=0 tx=0).stop 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.978+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/1587437864 shutdown_connections 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.978+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/1587437864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55765799dcb0 0x5576579a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.978+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/1587437864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657993410 0x55765799d770 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.978+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/1587437864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657992af0 0x557657992ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.978+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/1587437864 >> 172.21.15.5:0/1587437864 conn(0x55765798f060 msgr2=0x55765798f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.978+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/1587437864 shutdown_connections 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.978+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/1587437864 wait complete. 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.978+0000 7fe4d0fd5740 1 -- start start 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4d0fd5740 1 Processor -- start 2024-09-11T22:27:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657992af0 0x55765799abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 0x55765799b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55765799dcb0 0x55765799b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576579a0820 con 0x55765799dcb0 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557657880290 con 0x557657992af0 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5576579a09a0 con 0x557657993410 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4d0fd5740 10 client.0 _ll_register_callbacks cb 0x5576578b1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55765799dcb0 0x55765799b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55765799dcb0 0x55765799b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46776/0 (socket says 172.21.15.5:46776) 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c7fff640 1 -- 172.21.15.5:0/3347353302 learned_addr learned my addr 172.21.15.5:0/3347353302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c6ffd640 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 0x55765799b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657992af0 0x55765799abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c7fff640 1 -- 172.21.15.5:0/3347353302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 msgr2=0x55765799b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c7fff640 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 0x55765799b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c7fff640 1 -- 172.21.15.5:0/3347353302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657992af0 msgr2=0x55765799abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c7fff640 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657992af0 0x55765799abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c7fff640 1 -- 172.21.15.5:0/3347353302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55765760a210 con 0x55765799dcb0 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c6ffd640 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 0x55765799b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.979+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657992af0 0x55765799abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.980+0000 7fe4c7fff640 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55765799dcb0 0x55765799b660 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0031710 tx=0x7fe4c0005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54ca40400182be7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.980+0000 7fe4a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.980+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/3347353302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c002fda0 con 0x55765799dcb0 2024-09-11T22:27:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.980+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/3347353302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe4c0003970 con 0x55765799dcb0 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.980+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3347353302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b80051d0 con 0x55765799dcb0 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.980+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/3347353302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe4c0003b70 con 0x55765799dcb0 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.980+0000 7fe4d0fd5740 10 client.17082 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.980+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3347353302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5576576d6620 con 0x55765799dcb0 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.980+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/3347353302 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b80051d0 con 0x55765799dcb0 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.980+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/3347353302 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe4c004f030 con 0x55765799dcb0 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.981+0000 7fe4d0fd5740 10 client.17082 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.981+0000 7fe4d0fd5740 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.981+0000 7fe4d0fd5740 20 client.17082 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.981+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5576579adcb0 0x5576579b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.981+0000 7fe4d0fd5740 4 client.17082 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.981+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3347353302 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557657904a20 con 0x5576579adcb0 2024-09-11T22:27:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.981+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5576579adcb0 0x5576579b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.981+0000 7fe4a6ffd640 10 client.17082 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.981+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5576579adcb0 0x5576579b0130 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e042121ca2760dd server_cookie=3c2da46dc05692d3 in_seq=0 out_seq=0 2024-09-11T22:27:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.983+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/3347353302 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557657904a20 con 0x5576579adcb0 2024-09-11T22:27:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.983+0000 7fe4a6ffd640 10 client.17082 handle_command_reply: tid=0 2024-09-11T22:27:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.983+0000 7fe4d0fd5740 1 client.17082 shutdown 2024-09-11T22:27:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3347353302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55765799dcb0 msgr2=0x55765799b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55765799dcb0 0x55765799b660 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0031710 tx=0x7fe4c0005d80 comp rx=0 tx=0).stop 2024-09-11T22:27:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3347353302 shutdown_connections 2024-09-11T22:27:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5576579adcb0 0x5576579b0130 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55765799dcb0 0x55765799b660 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 0x55765799b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/3347353302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657992af0 0x55765799abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3347353302 >> 172.21.15.5:0/3347353302 conn(0x55765798f060 msgr2=0x557657994dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4a6ffd640 0 client.17082 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3347353302 shutdown_connections 2024-09-11T22:27:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3347353302 wait complete. 2024-09-11T22:27:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:32.984+0000 7fe4d0fd5740 20 client.17082 trim_cache size 0 max 16384 2024-09-11T22:27:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4d0fd5740 1 Processor -- start 2024-09-11T22:27:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4d0fd5740 1 -- start start 2024-09-11T22:27:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x55765799b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 0x55765799cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55765799dcb0 0x5576579b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81afb60 con 0x557657992af0 2024-09-11T22:27:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81b16a0 con 0x55765799dcb0 2024-09-11T22:27:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81b0230 con 0x557657993410 2024-09-11T22:27:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x55765799b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x55765799b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46780/0 (socket says 172.21.15.5:46780) 2024-09-11T22:27:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 0x55765799cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c77fe640 1 -- 172.21.15.5:0/3196750536 learned_addr learned my addr 172.21.15.5:0/3196750536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c7fff640 1 --2- 172.21.15.5:0/3196750536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55765799dcb0 0x5576579b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c77fe640 1 -- 172.21.15.5:0/3196750536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 msgr2=0x55765799cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/3196750536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 0x55765799cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c77fe640 1 -- 172.21.15.5:0/3196750536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55765799dcb0 msgr2=0x5576579b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/3196750536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55765799dcb0 0x5576579b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c77fe640 1 -- 172.21.15.5:0/3196750536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557657993240 con 0x557657992af0 2024-09-11T22:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.000+0000 7fe4c6ffd640 1 --2- 172.21.15.5:0/3196750536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 0x55765799cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.001+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/3196750536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x55765799b490 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80cd0f0 tx=0x7fe4b807a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e07c43347a78f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.001+0000 7fe4ad7fa640 1 -- 172.21.15.5:0/3196750536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b801c040 con 0x557657992af0 2024-09-11T22:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.001+0000 7fe4ad7fa640 1 -- 172.21.15.5:0/3196750536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b80051d0 con 0x557657992af0 2024-09-11T22:27:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.001+0000 7fe4ad7fa640 1 -- 172.21.15.5:0/3196750536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80a4530 con 0x557657992af0 2024-09-11T22:27:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.001+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3196750536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 msgr2=0x55765799b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.001+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/3196750536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x55765799b490 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80cd0f0 tx=0x7fe4b807a040 comp rx=0 tx=0).stop 2024-09-11T22:27:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3196750536 shutdown_connections 2024-09-11T22:27:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/3196750536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55765799dcb0 0x5576579b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/3196750536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557657993410 0x55765799cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/3196750536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x55765799b490 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3196750536 >> 172.21.15.5:0/3196750536 conn(0x55765798f060 msgr2=0x55765799c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3196750536 shutdown_connections 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/3196750536 wait complete. 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 -- start start 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 Processor -- start 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x5576579a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.002+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657993410 0x5576579a6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4d0fd5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55765799dcb0 0x557657997200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81b0fd0 con 0x557657992af0 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81b0900 con 0x557657993410 2024-09-11T22:27:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4d0fd5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81adb40 con 0x55765799dcb0 2024-09-11T22:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4d0fd5740 10 client.0 _ll_register_callbacks cb 0x7fe4b8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x5576579a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x5576579a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46790/0 (socket says 172.21.15.5:46790) 2024-09-11T22:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c77fe640 1 -- 172.21.15.5:0/328459032 learned_addr learned my addr 172.21.15.5:0/328459032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c6ffd640 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657993410 0x5576579a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c7fff640 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55765799dcb0 0x557657997200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c77fe640 1 -- 172.21.15.5:0/328459032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55765799dcb0 msgr2=0x557657997200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55765799dcb0 0x557657997200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c77fe640 1 -- 172.21.15.5:0/328459032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657993410 msgr2=0x5576579a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657993410 0x5576579a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c77fe640 1 -- 172.21.15.5:0/328459032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557657997940 con 0x557657992af0 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4c77fe640 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x5576579a5e20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80a4000 tx=0x7fe4b80cf2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f568d25413f19625 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/328459032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b801c040 con 0x557657992af0 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/328459032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe4b8099bc0 con 0x557657992af0 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.003+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/328459032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b80051d0 con 0x557657992af0 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/328459032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe4b809b400 con 0x557657992af0 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4d0fd5740 10 client.17088 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/328459032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557657a18380 con 0x557657992af0 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/328459032 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b80051d0 con 0x557657992af0 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/328459032 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe4b8004bc0 con 0x557657992af0 2024-09-11T22:27:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4d0fd5740 10 client.17088 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4d0fd5740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4d0fd5740 20 client.17088 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557657a18600 0x557657a1a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4d0fd5740 4 client.17088 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/328459032 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576578fed70 con 0x557657a18600 2024-09-11T22:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.004+0000 7fe4c6ffd640 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557657a18600 0x557657a1a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.005+0000 7fe4c6ffd640 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557657a18600 0x557657a1a9f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519c0b47f0e2751f server_cookie=c5fc8be8769bc648 in_seq=0 out_seq=0 2024-09-11T22:27:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.005+0000 7fe4a6ffd640 10 client.17088 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4a6ffd640 1 -- 172.21.15.5:0/328459032 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576578fed70 con 0x557657a18600 2024-09-11T22:27:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4a6ffd640 10 client.17088 handle_command_reply: tid=0 2024-09-11T22:27:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4d0fd5740 1 client.17088 shutdown 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/328459032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 msgr2=0x5576579a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x5576579a5e20 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80a4000 tx=0x7fe4b80cf2c0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/328459032 shutdown_connections 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557657a18600 0x557657a1a9f0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55765799dcb0 0x557657997200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4a6ffd640 0 client.17088 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557657993410 0x5576579a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4d0fd5740 1 --2- 172.21.15.5:0/328459032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557657992af0 0x5576579a5e20 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.132+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/328459032 >> 172.21.15.5:0/328459032 conn(0x55765798f060 msgr2=0x55765798e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.133+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/328459032 shutdown_connections 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.133+0000 7fe4d0fd5740 1 -- 172.21.15.5:0/328459032 wait complete. 2024-09-11T22:27:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.133+0000 7fe4d0fd5740 20 client.17088 trim_cache size 0 max 16384 2024-09-11T22:27:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:27:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:27:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:27:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:27:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.133+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1296400877 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4b00a2da0 msgr2=0x7fe4b00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4b00a2da0 0x7fe4b00a5260 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009a60 tx=0x7fe4c003f040 comp rx=0 tx=0).stop 2024-09-11T22:27:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1296400877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c8100880 msgr2=0x7fe4c819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c8100880 0x7fe4c819a550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800e9f0 tx=0x7fe4b800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1296400877 shutdown_connections 2024-09-11T22:27:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4c810e430 0x7fe4c819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c8101230 0x7fe4c819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4b00a2da0 0x7fe4b00a5260 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 --2- 172.21.15.5:0/1296400877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4c8100880 0x7fe4c819a550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1296400877 >> 172.21.15.5:0/1296400877 conn(0x7fe4c8077990 msgr2=0x7fe4c80ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1296400877 shutdown_connections 2024-09-11T22:27:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:33.134+0000 7fe4ce23c640 1 -- 172.21.15.5:0/1296400877 wait complete. 2024-09-11T22:27:33.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 368321 7.46 MB/sec execute 111 sec latency 58.569 ms 2024-09-11T22:27:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:33 smithi005 ceph-mon[30943]: pgmap v164: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-09-11T22:27:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:33 smithi159 ceph-mon[35534]: pgmap v164: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-09-11T22:27:33.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:33 smithi172 ceph-mon[35461]: pgmap v164: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 269 op/s 2024-09-11T22:27:34.319 INFO:tasks.workunit.client.0.smithi005.stdout: 1 370403 7.49 MB/sec execute 112 sec latency 13.811 ms 2024-09-11T22:27:35.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 372820 7.55 MB/sec execute 113 sec latency 14.339 ms 2024-09-11T22:27:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:35 smithi005 ceph-mon[30943]: pgmap v165: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-11T22:27:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:35 smithi159 ceph-mon[35534]: pgmap v165: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-11T22:27:35.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:35 smithi172 ceph-mon[35461]: pgmap v165: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-11T22:27:36.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 375226 7.58 MB/sec execute 114 sec latency 15.061 ms 2024-09-11T22:27:37.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 377384 7.63 MB/sec execute 115 sec latency 10.536 ms 2024-09-11T22:27:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:37 smithi005 ceph-mon[30943]: pgmap v166: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-11T22:27:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:37 smithi159 ceph-mon[35534]: pgmap v166: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-11T22:27:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:37 smithi172 ceph-mon[35461]: pgmap v166: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-11T22:27:38.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 379786 7.68 MB/sec execute 116 sec latency 36.558 ms 2024-09-11T22:27:39.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 381764 7.67 MB/sec execute 117 sec latency 17.104 ms 2024-09-11T22:27:39.843 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:39 smithi005 ceph-mon[30943]: pgmap v167: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.4 MiB/s wr, 352 op/s 2024-09-11T22:27:39.847 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:39 smithi159 ceph-mon[35534]: pgmap v167: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.4 MiB/s wr, 352 op/s 2024-09-11T22:27:39.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:39 smithi172 ceph-mon[35461]: pgmap v167: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.4 MiB/s wr, 352 op/s 2024-09-11T22:27:40.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 383864 7.72 MB/sec execute 118 sec latency 47.505 ms 2024-09-11T22:27:41.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 386503 7.78 MB/sec execute 119 sec latency 16.350 ms 2024-09-11T22:27:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:41 smithi005 ceph-mon[30943]: pgmap v168: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-11T22:27:41.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:41 smithi159 ceph-mon[35534]: pgmap v168: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-11T22:27:41.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:41 smithi172 ceph-mon[35461]: pgmap v168: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-09-11T22:27:42.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 388380 7.79 MB/sec execute 120 sec latency 15.697 ms 2024-09-11T22:27:43.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 390675 7.84 MB/sec execute 121 sec latency 55.726 ms 2024-09-11T22:27:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:43 smithi005 ceph-mon[30943]: pgmap v169: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 376 op/s 2024-09-11T22:27:43.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:43 smithi159 ceph-mon[35534]: pgmap v169: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 376 op/s 2024-09-11T22:27:43.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:43 smithi172 ceph-mon[35461]: pgmap v169: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 376 op/s 2024-09-11T22:27:44.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 392844 7.85 MB/sec execute 122 sec latency 32.789 ms 2024-09-11T22:27:45.320 INFO:tasks.workunit.client.0.smithi005.stdout: 1 394951 7.89 MB/sec execute 123 sec latency 11.631 ms 2024-09-11T22:27:45.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:45 smithi005 ceph-mon[30943]: pgmap v170: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-09-11T22:27:45.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:45 smithi159 ceph-mon[35534]: pgmap v170: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-09-11T22:27:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:45 smithi172 ceph-mon[35461]: pgmap v170: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-09-11T22:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.230+0000 7f3e252ba640 1 -- 172.21.15.5:0/688126009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e201069c0 msgr2=0x7f3e20106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.230+0000 7f3e252ba640 1 --2- 172.21.15.5:0/688126009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e201069c0 0x7f3e20106da0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009cd0 tx=0x7f3e1002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.230+0000 7f3e252ba640 1 -- 172.21.15.5:0/688126009 shutdown_connections 2024-09-11T22:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.230+0000 7f3e252ba640 1 --2- 172.21.15.5:0/688126009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e20101350 0x7f3e2010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.230+0000 7f3e252ba640 1 --2- 172.21.15.5:0/688126009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e201009b0 0x7f3e20100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.230+0000 7f3e252ba640 1 --2- 172.21.15.5:0/688126009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e201069c0 0x7f3e20106da0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.230+0000 7f3e252ba640 1 -- 172.21.15.5:0/688126009 >> 172.21.15.5:0/688126009 conn(0x7f3e20077990 msgr2=0x7f3e200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.230+0000 7f3e252ba640 1 -- 172.21.15.5:0/688126009 shutdown_connections 2024-09-11T22:27:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.230+0000 7f3e252ba640 1 -- 172.21.15.5:0/688126009 wait complete. 2024-09-11T22:27:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.230+0000 7f3e252ba640 1 Processor -- start 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e252ba640 1 -- start start 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e252ba640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e201009b0 0x7f3e201991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e252ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e20101350 0x7f3e20199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e252ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e201069c0 0x7f3e201a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e252ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20110e00 con 0x7f3e201069c0 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e252ba640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20110c80 con 0x7f3e201009b0 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e252ba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20110f80 con 0x7f3e20101350 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e201069c0 0x7f3e201a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e1e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e20101350 0x7f3e20199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e201009b0 0x7f3e201991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e1e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e20101350 0x7f3e20199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:55214/0 (socket says 172.21.15.5:55214) 2024-09-11T22:27:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e1e7fc640 1 -- 172.21.15.5:0/493957090 learned_addr learned my addr 172.21.15.5:0/493957090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e1effd640 1 -- 172.21.15.5:0/493957090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e20101350 msgr2=0x7f3e20199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e1effd640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e20101350 0x7f3e20199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e1effd640 1 -- 172.21.15.5:0/493957090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e201069c0 msgr2=0x7f3e201a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e1effd640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e201069c0 0x7f3e201a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.231+0000 7f3e1effd640 1 -- 172.21.15.5:0/493957090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e201a4790 con 0x7f3e201009b0 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.232+0000 7f3e1f7fe640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e201069c0 0x7f3e201a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.232+0000 7f3e1e7fc640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e20101350 0x7f3e20199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.232+0000 7f3e1effd640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e201009b0 0x7f3e201991d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3e10000f80 tx=0x7f3e10031e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.232+0000 7f3e07fff640 1 -- 172.21.15.5:0/493957090 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10003e40 con 0x7f3e201009b0 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.232+0000 7f3e252ba640 1 -- 172.21.15.5:0/493957090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e20196fc0 con 0x7f3e201009b0 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.232+0000 7f3e07fff640 1 -- 172.21.15.5:0/493957090 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e10004520 con 0x7f3e201009b0 2024-09-11T22:27:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.232+0000 7f3e07fff640 1 -- 172.21.15.5:0/493957090 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10030d10 con 0x7f3e201009b0 2024-09-11T22:27:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.232+0000 7f3e252ba640 1 -- 172.21.15.5:0/493957090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e201974b0 con 0x7f3e201009b0 2024-09-11T22:27:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.234+0000 7f3e2805b740 1 Processor -- start 2024-09-11T22:27:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.234+0000 7f3e2805b740 1 -- start start 2024-09-11T22:27:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a041336ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041337410 0x55a041341770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a0413440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e2805b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a04129b2f0 con 0x55a041341cb0 2024-09-11T22:27:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e2805b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a041243270 con 0x55a041337410 2024-09-11T22:27:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e2805b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0412972b0 con 0x55a041336af0 2024-09-11T22:27:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a041336ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a0413440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a041336ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:55222/0 (socket says 172.21.15.5:55222) 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a0413440a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48382/0 (socket says 172.21.15.5:48382) 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1f7fe640 1 -- 172.21.15.5:0/893749441 learned_addr learned my addr 172.21.15.5:0/893749441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e07fff640 1 -- 172.21.15.5:0/493957090 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3e100046d0 con 0x7f3e201009b0 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1e7fc640 1 -- 172.21.15.5:0/893749441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 msgr2=0x55a041336ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1e7fc640 1 --2- 172.21.15.5:0/893749441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a041336ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1e7fc640 1 -- 172.21.15.5:0/893749441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041337410 msgr2=0x55a041341770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1e7fc640 1 --2- 172.21.15.5:0/893749441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041337410 0x55a041341770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1e7fc640 1 -- 172.21.15.5:0/893749441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a041242340 con 0x55a041341cb0 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1f7fe640 1 --2- 172.21.15.5:0/893749441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a041336ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.235+0000 7f3e1e7fc640 1 --2- 172.21.15.5:0/893749441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a0413440a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3e140114c0 tx=0x7f3e14011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee5c1bdc0900555 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e057fa640 1 -- 172.21.15.5:0/893749441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14004030 con 0x55a041341cb0 2024-09-11T22:27:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e057fa640 1 -- 172.21.15.5:0/893749441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e140041d0 con 0x55a041341cb0 2024-09-11T22:27:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e057fa640 1 -- 172.21.15.5:0/893749441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1400f960 con 0x55a041341cb0 2024-09-11T22:27:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e07fff640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e080a2da0 0x7f3e080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e1f7fe640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e080a2da0 0x7f3e080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e07fff640 1 -- 172.21.15.5:0/493957090 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f3e1002e030 con 0x7f3e201009b0 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e2805b740 1 -- 172.21.15.5:0/893749441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 msgr2=0x55a0413440a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e2805b740 1 --2- 172.21.15.5:0/893749441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a0413440a0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3e140114c0 tx=0x7f3e14011980 comp rx=0 tx=0).stop 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e2805b740 1 -- 172.21.15.5:0/893749441 shutdown_connections 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e2805b740 1 --2- 172.21.15.5:0/893749441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a0413440a0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e2805b740 1 --2- 172.21.15.5:0/893749441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041337410 0x55a041341770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e2805b740 1 --2- 172.21.15.5:0/893749441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a041336ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e2805b740 1 -- 172.21.15.5:0/893749441 >> 172.21.15.5:0/893749441 conn(0x55a041333060 msgr2=0x55a041333440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e2805b740 1 -- 172.21.15.5:0/893749441 shutdown_connections 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e1f7fe640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e080a2da0 0x7f3e080a5260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3e18005f60 tx=0x7f3e18012040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.236+0000 7f3e2805b740 1 -- 172.21.15.5:0/893749441 wait complete. 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e2805b740 1 -- start start 2024-09-11T22:27:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e2805b740 1 Processor -- start 2024-09-11T22:27:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a04133eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041337410 0x55a04133f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a04133f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e2805b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a041344880 con 0x55a041341cb0 2024-09-11T22:27:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e2805b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a041224290 con 0x55a041337410 2024-09-11T22:27:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e2805b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a041344a00 con 0x55a041336af0 2024-09-11T22:27:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e2805b740 10 client.0 _ll_register_callbacks cb 0x55a041255140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a04133eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a04133f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e1e7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041337410 0x55a04133f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a04133f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48388/0 (socket says 172.21.15.5:48388) 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a04133eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:55226/0 (socket says 172.21.15.5:55226) 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e1f7fe640 1 -- 172.21.15.5:0/1610529703 learned_addr learned my addr 172.21.15.5:0/1610529703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.237+0000 7f3e1f7fe640 1 -- 172.21.15.5:0/1610529703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 msgr2=0x55a04133eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e1f7fe640 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a04133eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e1f7fe640 1 -- 172.21.15.5:0/1610529703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041337410 msgr2=0x55a04133f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e1f7fe640 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041337410 0x55a04133f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e1f7fe640 1 -- 172.21.15.5:0/1610529703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a040fae210 con 0x55a041341cb0 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e1f7fe640 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a04133f620 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3e1801e6f0 tx=0x7f3e18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f860a56440960d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e02ff5640 1 -- 172.21.15.5:0/1610529703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1802c020 con 0x55a041341cb0 2024-09-11T22:27:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e02ff5640 1 -- 172.21.15.5:0/1610529703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f3e1801c6d0 con 0x55a041341cb0 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e2805b740 1 -- 172.21.15.5:0/1610529703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e140041d0 con 0x55a041341cb0 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e02ff5640 1 -- 172.21.15.5:0/1610529703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3e18037050 con 0x55a041341cb0 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e2805b740 10 client.17100 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e2805b740 1 -- 172.21.15.5:0/1610529703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a04107a620 con 0x55a041341cb0 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.238+0000 7f3e02ff5640 1 -- 172.21.15.5:0/1610529703 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e140041d0 con 0x55a041341cb0 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.239+0000 7f3e02ff5640 1 -- 172.21.15.5:0/1610529703 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f3e18027030 con 0x55a041341cb0 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.239+0000 7f3e2805b740 10 client.17100 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.239+0000 7f3e2805b740 10 client.17100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.239+0000 7f3e2805b740 20 client.17100 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.239+0000 7f3e2805b740 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a041351c40 0x55a041354050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.239+0000 7f3e2805b740 4 client.17100 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.239+0000 7f3e2805b740 1 -- 172.21.15.5:0/1610529703 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0412a8a20 con 0x55a041351c40 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.239+0000 7f3e1effd640 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a041351c40 0x55a041354050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.239+0000 7f3e02ff5640 10 client.17100 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.239+0000 7f3e1effd640 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a041351c40 0x55a041354050 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b601355f3aae62e server_cookie=2d7631ae9f9759df in_seq=0 out_seq=0 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e02ff5640 1 -- 172.21.15.5:0/1610529703 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0412a8a20 con 0x55a041351c40 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e02ff5640 10 client.17100 handle_command_reply: tid=0 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 client.17100 shutdown 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 -- 172.21.15.5:0/1610529703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 msgr2=0x55a04133f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a04133f620 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3e1801e6f0 tx=0x7f3e18004730 comp rx=0 tx=0).stop 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 -- 172.21.15.5:0/1610529703 shutdown_connections 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a041351c40 0x55a041354050 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041341cb0 0x55a04133f620 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041337410 0x55a04133f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 --2- 172.21.15.5:0/1610529703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041336af0 0x55a04133eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e02ff5640 0 client.17100 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 -- 172.21.15.5:0/1610529703 >> 172.21.15.5:0/1610529703 conn(0x55a041333060 msgr2=0x55a041338d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 -- 172.21.15.5:0/1610529703 shutdown_connections 2024-09-11T22:27:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 1 -- 172.21.15.5:0/1610529703 wait complete. 2024-09-11T22:27:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.240+0000 7f3e2805b740 20 client.17100 trim_cache size 0 max 16384 2024-09-11T22:27:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e2805b740 1 Processor -- start 2024-09-11T22:27:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e2805b740 1 -- start start 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041337410 0x55a0413599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041341cb0 0x55a041359ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e2805b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3e201ab650 con 0x55a041336af0 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e2805b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3e201ad190 con 0x55a041341cb0 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e2805b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e201abd20 con 0x55a041337410 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48390/0 (socket says 172.21.15.5:48390) 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e1effd640 1 -- 172.21.15.5:0/2828487192 learned_addr learned my addr 172.21.15.5:0/2828487192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e1f7fe640 1 --2- 172.21.15.5:0/2828487192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041341cb0 0x55a041359ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e1e7fc640 1 --2- 172.21.15.5:0/2828487192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041337410 0x55a0413599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e1effd640 1 -- 172.21.15.5:0/2828487192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041337410 msgr2=0x55a0413599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e1effd640 1 --2- 172.21.15.5:0/2828487192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041337410 0x55a0413599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e1effd640 1 -- 172.21.15.5:0/2828487192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041341cb0 msgr2=0x55a041359ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e1effd640 1 --2- 172.21.15.5:0/2828487192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041341cb0 0x55a041359ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.255+0000 7f3e1effd640 1 -- 172.21.15.5:0/2828487192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a041339940 con 0x55a041336af0 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.256+0000 7f3e1effd640 1 --2- 172.21.15.5:0/2828487192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133f450 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3e100b98e0 tx=0x7f3e1003d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00b64ba17da03b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.256+0000 7f3e04ff9640 1 -- 172.21.15.5:0/2828487192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100c12c0 con 0x55a041336af0 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.256+0000 7f3e04ff9640 1 -- 172.21.15.5:0/2828487192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e10036040 con 0x55a041336af0 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.256+0000 7f3e04ff9640 1 -- 172.21.15.5:0/2828487192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100ee220 con 0x55a041336af0 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.256+0000 7f3e2805b740 1 -- 172.21.15.5:0/2828487192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 msgr2=0x55a04133f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.256+0000 7f3e2805b740 1 --2- 172.21.15.5:0/2828487192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133f450 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3e100b98e0 tx=0x7f3e1003d040 comp rx=0 tx=0).stop 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 -- 172.21.15.5:0/2828487192 shutdown_connections 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 --2- 172.21.15.5:0/2828487192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041341cb0 0x55a041359ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 --2- 172.21.15.5:0/2828487192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041337410 0x55a0413599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 --2- 172.21.15.5:0/2828487192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133f450 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 -- 172.21.15.5:0/2828487192 >> 172.21.15.5:0/2828487192 conn(0x55a041333060 msgr2=0x55a041340c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 -- 172.21.15.5:0/2828487192 shutdown_connections 2024-09-11T22:27:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 -- 172.21.15.5:0/2828487192 wait complete. 2024-09-11T22:27:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 -- start start 2024-09-11T22:27:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 Processor -- start 2024-09-11T22:27:46.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041337410 0x55a04133c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041341cb0 0x55a04133c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3e201acac0 con 0x55a041336af0 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3e201ac3f0 con 0x55a041341cb0 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e201a9630 con 0x55a041337410 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e2805b740 10 client.0 _ll_register_callbacks cb 0x7f3e18009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48406/0 (socket says 172.21.15.5:48406) 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e1effd640 1 -- 172.21.15.5:0/3177484643 learned_addr learned my addr 172.21.15.5:0/3177484643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e1f7fe640 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041341cb0 0x55a04133c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.257+0000 7f3e1e7fc640 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041337410 0x55a04133c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e1effd640 1 -- 172.21.15.5:0/3177484643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041337410 msgr2=0x55a04133c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e1effd640 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041337410 0x55a04133c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e1effd640 1 -- 172.21.15.5:0/3177484643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041341cb0 msgr2=0x55a04133c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e1effd640 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041341cb0 0x55a04133c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e1effd640 1 -- 172.21.15.5:0/3177484643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0413bb980 con 0x55a041336af0 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e1effd640 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133bae0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3e100c55c0 tx=0x7f3e1002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3d461ec498e30d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e02ff5640 1 -- 172.21.15.5:0/3177484643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100c5af0 con 0x55a041336af0 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e02ff5640 1 -- 172.21.15.5:0/3177484643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f3e100b9580 con 0x55a041336af0 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e2805b740 1 -- 172.21.15.5:0/3177484643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e10036040 con 0x55a041336af0 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e02ff5640 1 -- 172.21.15.5:0/3177484643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3e100ee220 con 0x55a041336af0 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e2805b740 10 client.17106 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e2805b740 1 -- 172.21.15.5:0/3177484643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a0413bcb90 con 0x55a041336af0 2024-09-11T22:27:46.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.258+0000 7f3e02ff5640 1 -- 172.21.15.5:0/3177484643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e10036040 con 0x55a041336af0 2024-09-11T22:27:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.259+0000 7f3e02ff5640 1 -- 172.21.15.5:0/3177484643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f3e1004e070 con 0x55a041336af0 2024-09-11T22:27:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.259+0000 7f3e2805b740 10 client.17106 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.259+0000 7f3e2805b740 10 client.17106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.259+0000 7f3e2805b740 20 client.17106 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.259+0000 7f3e2805b740 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0413bd150 0x55a0413bf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.259+0000 7f3e2805b740 4 client.17106 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:27:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.259+0000 7f3e2805b740 1 -- 172.21.15.5:0/3177484643 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0412a2d70 con 0x55a0413bd150 2024-09-11T22:27:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.259+0000 7f3e1e7fc640 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0413bd150 0x55a0413bf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.260+0000 7f3e02ff5640 10 client.17106 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:46.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.260+0000 7f3e1e7fc640 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0413bd150 0x55a0413bf570 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fa10215689cf2fe server_cookie=a96069614bcdd3f7 in_seq=0 out_seq=0 2024-09-11T22:27:46.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 397450 7.95 MB/sec execute 124 sec latency 22.738 ms 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.385+0000 7f3e02ff5640 1 -- 172.21.15.5:0/3177484643 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0412a2d70 con 0x55a0413bd150 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.385+0000 7f3e02ff5640 10 client.17106 handle_command_reply: tid=0 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.385+0000 7f3e2805b740 1 client.17106 shutdown 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.385+0000 7f3e2805b740 1 -- 172.21.15.5:0/3177484643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 msgr2=0x55a04133bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.385+0000 7f3e2805b740 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133bae0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3e100c55c0 tx=0x7f3e1002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.386+0000 7f3e2805b740 1 -- 172.21.15.5:0/3177484643 shutdown_connections 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.386+0000 7f3e2805b740 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0413bd150 0x55a0413bf570 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.386+0000 7f3e2805b740 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a041341cb0 0x55a04133c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.386+0000 7f3e2805b740 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a041337410 0x55a04133c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.386+0000 7f3e2805b740 1 --2- 172.21.15.5:0/3177484643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a041336af0 0x55a04133bae0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.386+0000 7f3e2805b740 1 -- 172.21.15.5:0/3177484643 >> 172.21.15.5:0/3177484643 conn(0x55a041333060 msgr2=0x55a041340c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.386+0000 7f3e02ff5640 0 client.17106 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.386+0000 7f3e2805b740 1 -- 172.21.15.5:0/3177484643 shutdown_connections 2024-09-11T22:27:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.386+0000 7f3e2805b740 1 -- 172.21.15.5:0/3177484643 wait complete. 2024-09-11T22:27:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.386+0000 7f3e2805b740 20 client.17106 trim_cache size 0 max 16384 2024-09-11T22:27:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:27:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:27:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:27:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:27:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.388+0000 7f3e252ba640 1 -- 172.21.15.5:0/493957090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e080a2da0 msgr2=0x7f3e080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.388+0000 7f3e252ba640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e080a2da0 0x7f3e080a5260 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3e18005f60 tx=0x7f3e18012040 comp rx=0 tx=0).stop 2024-09-11T22:27:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.388+0000 7f3e252ba640 1 -- 172.21.15.5:0/493957090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e201009b0 msgr2=0x7f3e201991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.388+0000 7f3e252ba640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e201009b0 0x7f3e201991d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3e10000f80 tx=0x7f3e10031e40 comp rx=0 tx=0).stop 2024-09-11T22:27:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.388+0000 7f3e252ba640 1 -- 172.21.15.5:0/493957090 shutdown_connections 2024-09-11T22:27:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.389+0000 7f3e252ba640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e201069c0 0x7f3e201a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.389+0000 7f3e252ba640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e20101350 0x7f3e20199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.389+0000 7f3e252ba640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e080a2da0 0x7f3e080a5260 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.389+0000 7f3e252ba640 1 --2- 172.21.15.5:0/493957090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e201009b0 0x7f3e201991d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.389+0000 7f3e252ba640 1 -- 172.21.15.5:0/493957090 >> 172.21.15.5:0/493957090 conn(0x7f3e20077990 msgr2=0x7f3e200ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.389+0000 7f3e252ba640 1 -- 172.21.15.5:0/493957090 shutdown_connections 2024-09-11T22:27:46.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:46.389+0000 7f3e252ba640 1 -- 172.21.15.5:0/493957090 wait complete. 2024-09-11T22:27:47.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 399276 7.94 MB/sec execute 125 sec latency 56.278 ms 2024-09-11T22:27:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:47 smithi005 ceph-mon[30943]: pgmap v171: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-09-11T22:27:47.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:47 smithi159 ceph-mon[35534]: pgmap v171: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-09-11T22:27:47.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:47 smithi172 ceph-mon[35461]: pgmap v171: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 288 op/s 2024-09-11T22:27:48.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 401618 7.99 MB/sec execute 126 sec latency 11.425 ms 2024-09-11T22:27:49.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 404400 8.06 MB/sec execute 127 sec latency 19.694 ms 2024-09-11T22:27:49.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:49 smithi005 ceph-mon[30943]: pgmap v172: 97 pgs: 97 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 488 op/s 2024-09-11T22:27:49.828 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:49 smithi159 ceph-mon[35534]: pgmap v172: 97 pgs: 97 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 488 op/s 2024-09-11T22:27:49.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:49 smithi172 ceph-mon[35461]: pgmap v172: 97 pgs: 97 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 488 op/s 2024-09-11T22:27:50.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 406226 8.05 MB/sec execute 128 sec latency 14.327 ms 2024-09-11T22:27:50.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:27:50.916 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:51.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 408519 8.10 MB/sec execute 129 sec latency 27.424 ms 2024-09-11T22:27:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.484+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3534221690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85c8102ef0 msgr2=0x7f85c810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.484+0000 7f85cdfb4640 1 --2- 172.21.15.5:0/3534221690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85c8102ef0 0x7f85c810da40 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009900 tx=0x7f85c002a480 comp rx=0 tx=0).stop 2024-09-11T22:27:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.484+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3534221690 shutdown_connections 2024-09-11T22:27:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.484+0000 7f85cdfb4640 1 --2- 172.21.15.5:0/3534221690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85c8102ef0 0x7f85c810da40 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.484+0000 7f85cdfb4640 1 --2- 172.21.15.5:0/3534221690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85c8102570 0x7f85c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 --2- 172.21.15.5:0/3534221690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c8100e20 0x7f85c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3534221690 >> 172.21.15.5:0/3534221690 conn(0x7f85c80fc7d0 msgr2=0x7f85c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3534221690 shutdown_connections 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3534221690 wait complete. 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 Processor -- start 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 -- start start 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85c8100e20 0x7f85c8199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85c8102570 0x7f85c81998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c8102ef0 0x7f85c81a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8110f80 con 0x7f85c8100e20 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8110e00 con 0x7f85c8102570 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.485+0000 7f85cdfb4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8111100 con 0x7f85c8102ef0 2024-09-11T22:27:51.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85c8100e20 0x7f85c8199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85c8100e20 0x7f85c8199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48746/0 (socket says 172.21.15.5:48746) 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85bffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85c8102570 0x7f85c81998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c77fe640 1 -- 172.21.15.5:0/3107505500 learned_addr learned my addr 172.21.15.5:0/3107505500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c7fff640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c8102ef0 0x7f85c81a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c77fe640 1 -- 172.21.15.5:0/3107505500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c8102ef0 msgr2=0x7f85c81a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c77fe640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c8102ef0 0x7f85c81a4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c77fe640 1 -- 172.21.15.5:0/3107505500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85c8102570 msgr2=0x7f85c81998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c77fe640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85c8102570 0x7f85c81998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c77fe640 1 -- 172.21.15.5:0/3107505500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c81a4950 con 0x7f85c8100e20 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85bffff640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85c8102570 0x7f85c81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c77fe640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85c8100e20 0x7f85c8199390 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f85b800eb90 tx=0x7f85b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:51.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c57fa640 1 -- 172.21.15.5:0/3107505500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8004d20 con 0x7f85c8100e20 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c57fa640 1 -- 172.21.15.5:0/3107505500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85b800ee90 con 0x7f85c8100e20 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3107505500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c81a4b20 con 0x7f85c8100e20 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85c57fa640 1 -- 172.21.15.5:0/3107505500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b8010830 con 0x7f85c8100e20 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.486+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3107505500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c81a4fb0 con 0x7f85c8100e20 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.489+0000 7f85c57fa640 1 -- 172.21.15.5:0/3107505500 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f85b8005020 con 0x7f85c8100e20 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.489+0000 7f85d038d740 1 Processor -- start 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.489+0000 7f85d038d740 1 -- start start 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85d038d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196e7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85d038d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e7dc0 0x5585196f2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85d038d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 0x5585196f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85d038d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55851964d2e0 con 0x5585196f2680 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85d038d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5585195f4ce0 con 0x5585196e7dc0 2024-09-11T22:27:51.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85d038d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5585196492a0 con 0x5585196e74a0 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c57fa640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85b40a2cf0 0x7f85b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 0x5585196f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196e7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:37924/0 (socket says 172.21.15.5:37924) 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c7fff640 1 -- 172.21.15.5:0/3902827568 learned_addr learned my addr 172.21.15.5:0/3902827568 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c57fa640 1 -- 172.21.15.5:0/3107505500 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f85b80cfa70 con 0x7f85c8100e20 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c77fe640 1 --2- 172.21.15.5:0/3902827568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e7dc0 0x5585196f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c7fff640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85b40a2cf0 0x7f85b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c7fff640 1 -- 172.21.15.5:0/3902827568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e7dc0 msgr2=0x5585196f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c7fff640 1 --2- 172.21.15.5:0/3902827568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e7dc0 0x5585196f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c7fff640 1 -- 172.21.15.5:0/3902827568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 msgr2=0x5585196f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c7fff640 1 --2- 172.21.15.5:0/3902827568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 0x5585196f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85c7fff640 1 -- 172.21.15.5:0/3902827568 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585195d9440 con 0x5585196e74a0 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.490+0000 7f85bffff640 1 --2- 172.21.15.5:0/3902827568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 0x5585196f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.491+0000 7f85c7fff640 1 --2- 172.21.15.5:0/3902827568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196e7880 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f85c002a9a0 tx=0x7f85c0004460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ffe4fa3e24453e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.491+0000 7f85bdffb640 1 -- 172.21.15.5:0/3902827568 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0035040 con 0x5585196e74a0 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.491+0000 7f85bdffb640 1 -- 172.21.15.5:0/3902827568 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c004d070 con 0x5585196e74a0 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.491+0000 7f85c7fff640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85b40a2cf0 0x7f85b40a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85c004c000 tx=0x7f85c004a000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.491+0000 7f85bdffb640 1 -- 172.21.15.5:0/3902827568 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c003d460 con 0x5585196e74a0 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 -- 172.21.15.5:0/3902827568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 msgr2=0x5585196e7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 --2- 172.21.15.5:0/3902827568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196e7880 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f85c002a9a0 tx=0x7f85c0004460 comp rx=0 tx=0).stop 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 -- 172.21.15.5:0/3902827568 shutdown_connections 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 --2- 172.21.15.5:0/3902827568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 0x5585196f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 --2- 172.21.15.5:0/3902827568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e7dc0 0x5585196f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 --2- 172.21.15.5:0/3902827568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196e7880 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 -- 172.21.15.5:0/3902827568 >> 172.21.15.5:0/3902827568 conn(0x5585196e3a30 msgr2=0x5585196e3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 -- 172.21.15.5:0/3902827568 shutdown_connections 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 -- 172.21.15.5:0/3902827568 wait complete. 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 -- start start 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.492+0000 7f85d038d740 1 Processor -- start 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85d038d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85d038d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e7dc0 0x5585196ef250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85d038d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 0x5585196ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85d038d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585196f9660 con 0x5585196e7dc0 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85d038d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558519608050 con 0x5585196f2680 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85d038d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5585196f97e0 con 0x5585196e74a0 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85d038d740 10 client.0 _ll_register_callbacks cb 0x558519648160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e7dc0 0x5585196ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85bffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e7dc0 0x5585196ef250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48756/0 (socket says 172.21.15.5:48756) 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85bffff640 1 -- 172.21.15.5:0/2030349966 learned_addr learned my addr 172.21.15.5:0/2030349966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196f1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:37938/0 (socket says 172.21.15.5:37938) 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85c7fff640 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 0x5585196ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85c77fe640 1 -- 172.21.15.5:0/2030349966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 msgr2=0x5585196ef790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85c77fe640 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 0x5585196ef790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85c77fe640 1 -- 172.21.15.5:0/2030349966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e7dc0 msgr2=0x5585196ef250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85c77fe640 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e7dc0 0x5585196ef250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85c77fe640 1 -- 172.21.15.5:0/2030349966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585193be3c0 con 0x5585196e74a0 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85bffff640 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e7dc0 0x5585196ef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85c77fe640 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196f1fa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f85b80cd180 tx=0x7f85b809b970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdfa6e4190d27524 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.493+0000 7f85a6ffd640 1 -- 172.21.15.5:0/2030349966 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b80a0300 con 0x5585196e74a0 2024-09-11T22:27:51.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85a6ffd640 1 -- 172.21.15.5:0/2030349966 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f85b80985c0 con 0x5585196e74a0 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85d038d740 1 -- 172.21.15.5:0/2030349966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c004d070 con 0x5585196e74a0 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85a6ffd640 1 -- 172.21.15.5:0/2030349966 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f85b80a4bc0 con 0x5585196e74a0 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85d038d740 10 client.26458 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85d038d740 1 -- 172.21.15.5:0/2030349966 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558519705040 con 0x5585196e74a0 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85a6ffd640 1 -- 172.21.15.5:0/2030349966 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c004d070 con 0x5585196e74a0 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85a6ffd640 1 -- 172.21.15.5:0/2030349966 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f85b809f070 con 0x5585196e74a0 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85d038d740 10 client.26458 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85d038d740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85d038d740 20 client.26458 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85d038d740 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558519706500 0x5585197088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85d038d740 4 client.26458 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85d038d740 1 -- 172.21.15.5:0/2030349966 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55851965a850 con 0x558519706500 2024-09-11T22:27:51.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.494+0000 7f85bffff640 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558519706500 0x5585197088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.495+0000 7f85a6ffd640 10 client.26458 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.495+0000 7f85bffff640 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558519706500 0x5585197088f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e7a902c8752928 server_cookie=10b78e3842d658a2 in_seq=0 out_seq=0 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.495+0000 7f85a6ffd640 1 -- 172.21.15.5:0/2030349966 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55851965a850 con 0x558519706500 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.495+0000 7f85a6ffd640 10 client.26458 handle_command_reply: tid=0 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 client.26458 shutdown 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 -- 172.21.15.5:0/2030349966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 msgr2=0x5585196f1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196f1fa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f85b80cd180 tx=0x7f85b809b970 comp rx=0 tx=0).stop 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 -- 172.21.15.5:0/2030349966 shutdown_connections 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558519706500 0x5585197088f0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 0x5585196ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e7dc0 0x5585196ef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 --2- 172.21.15.5:0/2030349966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e74a0 0x5585196f1fa0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85a6ffd640 0 client.26458 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:51.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 -- 172.21.15.5:0/2030349966 >> 172.21.15.5:0/2030349966 conn(0x5585196e3a30 msgr2=0x5585196e9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:51.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 -- 172.21.15.5:0/2030349966 shutdown_connections 2024-09-11T22:27:51.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 1 -- 172.21.15.5:0/2030349966 wait complete. 2024-09-11T22:27:51.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.496+0000 7f85d038d740 20 client.26458 trim_cache size 0 max 16384 2024-09-11T22:27:51.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.515+0000 7f85d038d740 1 Processor -- start 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.515+0000 7f85d038d740 1 -- start start 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.515+0000 7f85d038d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e74a0 0x55851970e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.515+0000 7f85d038d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e7dc0 0x55851970e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.515+0000 7f85d038d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 0x5585196ec4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.515+0000 7f85d038d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81ab820 con 0x5585196e74a0 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.515+0000 7f85d038d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81ad360 con 0x5585196f2680 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.515+0000 7f85d038d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81abef0 con 0x5585196e7dc0 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e74a0 0x55851970e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85bffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e7dc0 0x55851970e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e74a0 0x55851970e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48762/0 (socket says 172.21.15.5:48762) 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c77fe640 1 -- 172.21.15.5:0/4115536908 learned_addr learned my addr 172.21.15.5:0/4115536908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c7fff640 1 --2- 172.21.15.5:0/4115536908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 0x5585196ec4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c77fe640 1 -- 172.21.15.5:0/4115536908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e7dc0 msgr2=0x55851970e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c77fe640 1 --2- 172.21.15.5:0/4115536908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e7dc0 0x55851970e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c77fe640 1 -- 172.21.15.5:0/4115536908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 msgr2=0x5585196ec4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c77fe640 1 --2- 172.21.15.5:0/4115536908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 0x5585196ec4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c77fe640 1 -- 172.21.15.5:0/4115536908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585196ed7a0 con 0x5585196e74a0 2024-09-11T22:27:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c7fff640 1 --2- 172.21.15.5:0/4115536908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 0x5585196ec4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:51.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85c77fe640 1 --2- 172.21.15.5:0/4115536908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e74a0 0x55851970e0d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f85b800cba0 tx=0x7f85b809a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f0126870d18655 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85bd7fa640 1 -- 172.21.15.5:0/4115536908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b80a0300 con 0x5585196e74a0 2024-09-11T22:27:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85bd7fa640 1 -- 172.21.15.5:0/4115536908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85b80985c0 con 0x5585196e74a0 2024-09-11T22:27:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.516+0000 7f85bd7fa640 1 -- 172.21.15.5:0/4115536908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b809ba90 con 0x5585196e74a0 2024-09-11T22:27:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.517+0000 7f85d038d740 1 -- 172.21.15.5:0/4115536908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e74a0 msgr2=0x55851970e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.517+0000 7f85d038d740 1 --2- 172.21.15.5:0/4115536908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e74a0 0x55851970e0d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f85b800cba0 tx=0x7f85b809a460 comp rx=0 tx=0).stop 2024-09-11T22:27:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.517+0000 7f85d038d740 1 -- 172.21.15.5:0/4115536908 shutdown_connections 2024-09-11T22:27:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.517+0000 7f85d038d740 1 --2- 172.21.15.5:0/4115536908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196f2680 0x5585196ec4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.517+0000 7f85d038d740 1 --2- 172.21.15.5:0/4115536908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e7dc0 0x55851970e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.517+0000 7f85d038d740 1 --2- 172.21.15.5:0/4115536908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196e74a0 0x55851970e0d0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.517+0000 7f85d038d740 1 -- 172.21.15.5:0/4115536908 >> 172.21.15.5:0/4115536908 conn(0x5585196e3a30 msgr2=0x5585196e2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.517+0000 7f85d038d740 1 -- 172.21.15.5:0/4115536908 shutdown_connections 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.517+0000 7f85d038d740 1 -- 172.21.15.5:0/4115536908 wait complete. 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.517+0000 7f85d038d740 1 -- start start 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85d038d740 1 Processor -- start 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85d038d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e74a0 0x5585196fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85d038d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e7dc0 0x5585196fd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85d038d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 0x5585197700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85d038d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81acc90 con 0x5585196f2680 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85d038d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81ac5c0 con 0x5585196e74a0 2024-09-11T22:27:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85d038d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81a9800 con 0x5585196e7dc0 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85d038d740 10 client.0 _ll_register_callbacks cb 0x7f85b8004ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e74a0 0x5585196fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85c77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e74a0 0x5585196fcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52524/0 (socket says 172.21.15.5:52524) 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85c77fe640 1 -- 172.21.15.5:0/232088991 learned_addr learned my addr 172.21.15.5:0/232088991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85bffff640 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e7dc0 0x5585196fd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.518+0000 7f85c7fff640 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 0x5585197700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85c77fe640 1 -- 172.21.15.5:0/232088991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e7dc0 msgr2=0x5585196fd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85c77fe640 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e7dc0 0x5585196fd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85c77fe640 1 -- 172.21.15.5:0/232088991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 msgr2=0x5585197700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85c77fe640 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 0x5585197700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85c77fe640 1 -- 172.21.15.5:0/232088991 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585197705f0 con 0x5585196e74a0 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85c77fe640 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e74a0 0x5585196fcba0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f85b809a200 tx=0x7f85b809b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d784a0de2aabc960 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85a6ffd640 1 -- 172.21.15.5:0/232088991 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b809ba90 con 0x5585196e74a0 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85d038d740 1 -- 172.21.15.5:0/232088991 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85b80985c0 con 0x5585196e74a0 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85a6ffd640 1 -- 172.21.15.5:0/232088991 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f85b8097ca0 con 0x5585196e74a0 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85a6ffd640 1 -- 172.21.15.5:0/232088991 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f85b80a0300 con 0x5585196e74a0 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85d038d740 10 client.26540 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85d038d740 1 -- 172.21.15.5:0/232088991 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5585197705f0 con 0x5585196e74a0 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.519+0000 7f85a6ffd640 1 -- 172.21.15.5:0/232088991 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85b80985c0 con 0x5585196e74a0 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.520+0000 7f85a6ffd640 1 -- 172.21.15.5:0/232088991 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f85b809f030 con 0x5585196e74a0 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.520+0000 7f85d038d740 10 client.26540 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.520+0000 7f85d038d740 10 client.26540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.520+0000 7f85d038d740 20 client.26540 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.520+0000 7f85d038d740 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558519771a40 0x558519773e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.520+0000 7f85d038d740 4 client.26540 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.520+0000 7f85d038d740 1 -- 172.21.15.5:0/232088991 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558519654d60 con 0x558519771a40 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.520+0000 7f85bffff640 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558519771a40 0x558519773e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.520+0000 7f85bffff640 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558519771a40 0x558519773e30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70a0d319da850f4a server_cookie=168681fb57659df5 in_seq=0 out_seq=0 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.520+0000 7f85a6ffd640 10 client.26540 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85a6ffd640 1 -- 172.21.15.5:0/232088991 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558519654d60 con 0x558519771a40 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85a6ffd640 10 client.26540 handle_command_reply: tid=0 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 client.26540 shutdown 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 -- 172.21.15.5:0/232088991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e74a0 msgr2=0x5585196fcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e74a0 0x5585196fcba0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f85b809a200 tx=0x7f85b809b970 comp rx=0 tx=0).stop 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 -- 172.21.15.5:0/232088991 shutdown_connections 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558519771a40 0x558519773e30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585196f2680 0x5585197700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5585196e7dc0 0x5585196fd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85a6ffd640 0 client.26540 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 --2- 172.21.15.5:0/232088991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585196e74a0 0x5585196fcba0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 -- 172.21.15.5:0/232088991 >> 172.21.15.5:0/232088991 conn(0x5585196e3a30 msgr2=0x5585196f0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 -- 172.21.15.5:0/232088991 shutdown_connections 2024-09-11T22:27:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 1 -- 172.21.15.5:0/232088991 wait complete. 2024-09-11T22:27:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.521+0000 7f85d038d740 20 client.26540 trim_cache size 0 max 16384 2024-09-11T22:27:51.527 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:27:51.527 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:27:51.527 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:27:51.527 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:27:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3107505500 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85b40a2cf0 msgr2=0x7f85b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85b40a2cf0 0x7f85b40a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85c004c000 tx=0x7f85c004a000 comp rx=0 tx=0).stop 2024-09-11T22:27:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3107505500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85c8100e20 msgr2=0x7f85c8199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85c8100e20 0x7f85c8199390 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f85b800eb90 tx=0x7f85b800c6a0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3107505500 shutdown_connections 2024-09-11T22:27:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85c8102ef0 0x7f85c81a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85b40a2cf0 0x7f85b40a51b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85c8102570 0x7f85c81998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 --2- 172.21.15.5:0/3107505500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85c8100e20 0x7f85c8199390 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3107505500 >> 172.21.15.5:0/3107505500 conn(0x7f85c80fc7d0 msgr2=0x7f85c81068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3107505500 shutdown_connections 2024-09-11T22:27:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:51.522+0000 7f85cdfb4640 1 -- 172.21.15.5:0/3107505500 wait complete. 2024-09-11T22:27:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:51 smithi005 ceph-mon[30943]: pgmap v173: 97 pgs: 97 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 309 op/s 2024-09-11T22:27:51.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:51 smithi159 ceph-mon[35534]: pgmap v173: 97 pgs: 97 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 309 op/s 2024-09-11T22:27:51.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:51 smithi172 ceph-mon[35461]: pgmap v173: 97 pgs: 97 active+clean; 59 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.6 MiB/s wr, 309 op/s 2024-09-11T22:27:52.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 410930 8.13 MB/sec execute 130 sec latency 17.122 ms 2024-09-11T22:27:52.695 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:27:52.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:27:52.970 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:53.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 413010 8.16 MB/sec execute 131 sec latency 19.482 ms 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/228456476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fcc10c170 msgr2=0x7f7fcc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 --2- 172.21.15.5:0/228456476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fcc10c170 0x7f7fcc10e610 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009ca0 tx=0x7f7fc402a950 comp rx=0 tx=0).stop 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/228456476 shutdown_connections 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 --2- 172.21.15.5:0/228456476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fcc10c170 0x7f7fcc10e610 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 --2- 172.21.15.5:0/228456476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc103660 0x7f7fcc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 --2- 172.21.15.5:0/228456476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fcc102d40 0x7f7fcc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/228456476 >> 172.21.15.5:0/228456476 conn(0x7f7fcc0fc7a0 msgr2=0x7f7fcc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/228456476 shutdown_connections 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/228456476 wait complete. 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 Processor -- start 2024-09-11T22:27:53.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.515+0000 7f7fd3d7b640 1 -- start start 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd3d7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc102d40 0x7f7fcc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd3d7b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fcc103660 0x7f7fcc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd3d7b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fcc10c170 0x7f7fcc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd3d7b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc111f70 con 0x7f7fcc103660 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd3d7b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc111df0 con 0x7f7fcc10c170 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd3d7b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7fcc1120f0 con 0x7f7fcc102d40 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1316640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fcc103660 0x7f7fcc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1316640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fcc103660 0x7f7fcc100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48790/0 (socket says 172.21.15.5:48790) 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1b17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc102d40 0x7f7fcc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1316640 1 -- 172.21.15.5:0/2861849860 learned_addr learned my addr 172.21.15.5:0/2861849860 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd2318640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fcc10c170 0x7f7fcc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1316640 1 -- 172.21.15.5:0/2861849860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc102d40 msgr2=0x7f7fcc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:53.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1316640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc102d40 0x7f7fcc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1316640 1 -- 172.21.15.5:0/2861849860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fcc10c170 msgr2=0x7f7fcc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1316640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fcc10c170 0x7f7fcc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1316640 1 -- 172.21.15.5:0/2861849860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fcc1a6910 con 0x7f7fcc103660 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1b17640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc102d40 0x7f7fcc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.516+0000 7f7fd1316640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fcc103660 0x7f7fcc100ee0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000d740 tx=0x7f7fc000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.517+0000 7f7fbeffd640 1 -- 172.21.15.5:0/2861849860 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0017070 con 0x7f7fcc103660 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.517+0000 7f7fbeffd640 1 -- 172.21.15.5:0/2861849860 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc000de50 con 0x7f7fcc103660 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.517+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/2861849860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fcc1a6c00 con 0x7f7fcc103660 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.517+0000 7f7fbeffd640 1 -- 172.21.15.5:0/2861849860 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000f620 con 0x7f7fcc103660 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.517+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/2861849860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fcc1a7140 con 0x7f7fcc103660 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.518+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/2861849860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fcc103120 con 0x7f7fcc103660 2024-09-11T22:27:53.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.519+0000 7f7fbeffd640 1 -- 172.21.15.5:0/2861849860 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7fc0009c30 con 0x7f7fcc103660 2024-09-11T22:27:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.519+0000 7f7fbeffd640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7fb80a2cf0 0x7f7fb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.519+0000 7f7fbeffd640 1 -- 172.21.15.5:0/2861849860 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f7fc00cef30 con 0x7f7fcc103660 2024-09-11T22:27:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.520+0000 7f7fd1b17640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7fb80a2cf0 0x7f7fb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.520+0000 7f7fd1b17640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7fb80a2cf0 0x7f7fb80a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009fc0 tx=0x7f7fc8005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:53.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.524+0000 7f7fbeffd640 1 -- 172.21.15.5:0/2861849860 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7fc009b4c0 con 0x7f7fcc103660 2024-09-11T22:27:53.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.740+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/2861849860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fcc1071e0 con 0x7f7fcc103660 2024-09-11T22:27:53.742 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:53 smithi005 ceph-mon[30943]: pgmap v174: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 397 op/s 2024-09-11T22:27:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.741+0000 7f7fbeffd640 1 -- 172.21.15.5:0/2861849860 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11967 (secure 0 0 0) 0x7f7fc009f7c0 con 0x7f7fcc103660 2024-09-11T22:27:53.742 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:27:53.742 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-11T22:25:49:149525+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:25:48.784660+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25970":{"gid":25970,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4169170564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4169170564},{"type":"v1","addr":"172.21.15.159:6833","nonce":4169170564}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:27:53.743 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-11T22:27:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.743+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/2861849860 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7fb80a2cf0 msgr2=0x7f7fb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.743+0000 7f7fd3d7b640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7fb80a2cf0 0x7f7fb80a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009fc0 tx=0x7f7fc8005f90 comp rx=0 tx=0).stop 2024-09-11T22:27:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.743+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/2861849860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fcc103660 msgr2=0x7f7fcc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.743+0000 7f7fd3d7b640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fcc103660 0x7f7fcc100ee0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000d740 tx=0x7f7fc000dc00 comp rx=0 tx=0).stop 2024-09-11T22:27:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.743+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/2861849860 shutdown_connections 2024-09-11T22:27:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.743+0000 7f7fd3d7b640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7fb80a2cf0 0x7f7fb80a51b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:53.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.743+0000 7f7fd3d7b640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fcc10c170 0x7f7fcc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.743+0000 7f7fd3d7b640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fcc103660 0x7f7fcc100ee0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.743+0000 7f7fd3d7b640 1 --2- 172.21.15.5:0/2861849860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fcc102d40 0x7f7fcc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.743+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/2861849860 >> 172.21.15.5:0/2861849860 conn(0x7f7fcc0fc7a0 msgr2=0x7f7fcc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.744+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/2861849860 shutdown_connections 2024-09-11T22:27:53.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:53.744+0000 7f7fd3d7b640 1 -- 172.21.15.5:0/2861849860 wait complete. 2024-09-11T22:27:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:53 smithi159 ceph-mon[35534]: pgmap v174: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 397 op/s 2024-09-11T22:27:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:53 smithi172 ceph-mon[35461]: pgmap v174: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.3 MiB/s wr, 397 op/s 2024-09-11T22:27:54.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 415293 8.20 MB/sec execute 132 sec latency 88.857 ms 2024-09-11T22:27:54.456 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:27:54.726 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:54.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:54 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2861849860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:27:54.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:54 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2861849860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:27:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:54 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2861849860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:27:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.277+0000 7fdac6c51640 1 -- 172.21.15.5:0/227450939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0106f10 msgr2=0x7fdac010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.277+0000 7fdac6c51640 1 --2- 172.21.15.5:0/227450939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0106f10 0x7fdac010ea90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009ca0 tx=0x7fdab802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.277+0000 7fdac6c51640 1 -- 172.21.15.5:0/227450939 shutdown_connections 2024-09-11T22:27:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.277+0000 7fdac6c51640 1 --2- 172.21.15.5:0/227450939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0106f10 0x7fdac010ea90 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.277+0000 7fdac6c51640 1 --2- 172.21.15.5:0/227450939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac0102790 0x7fdac01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.277+0000 7fdac6c51640 1 --2- 172.21.15.5:0/227450939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdac0101de0 0x7fdac01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.277+0000 7fdac6c51640 1 -- 172.21.15.5:0/227450939 >> 172.21.15.5:0/227450939 conn(0x7fdac00fd8c0 msgr2=0x7fdac00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.277+0000 7fdac6c51640 1 -- 172.21.15.5:0/227450939 shutdown_connections 2024-09-11T22:27:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.277+0000 7fdac6c51640 1 -- 172.21.15.5:0/227450939 wait complete. 2024-09-11T22:27:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac6c51640 1 Processor -- start 2024-09-11T22:27:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac6c51640 1 -- start start 2024-09-11T22:27:55.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac6c51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101de0 0x7fdac01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac6c51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdac0102790 0x7fdac0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac6c51640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac0106f10 0x7fdac019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac6c51640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0111f70 con 0x7fdac0101de0 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac6c51640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0111df0 con 0x7fdac0106f10 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac6c51640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01120f0 con 0x7fdac0102790 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac49ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101de0 0x7fdac01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac49ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101de0 0x7fdac01983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48802/0 (socket says 172.21.15.5:48802) 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac49ed640 1 -- 172.21.15.5:0/2622572317 learned_addr learned my addr 172.21.15.5:0/2622572317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac51ee640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac0106f10 0x7fdac019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdabffff640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdac0102790 0x7fdac0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac49ed640 1 -- 172.21.15.5:0/2622572317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdac0102790 msgr2=0x7fdac0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:55.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac49ed640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdac0102790 0x7fdac0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac49ed640 1 -- 172.21.15.5:0/2622572317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac0106f10 msgr2=0x7fdac019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac49ed640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac0106f10 0x7fdac019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.278+0000 7fdac49ed640 1 -- 172.21.15.5:0/2622572317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac019d3a0 con 0x7fdac0101de0 2024-09-11T22:27:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.279+0000 7fdac49ed640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101de0 0x7fdac01983f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fdab400e9f0 tx=0x7fdab400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.279+0000 7fdabdffb640 1 -- 172.21.15.5:0/2622572317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab400cd80 con 0x7fdac0101de0 2024-09-11T22:27:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.279+0000 7fdabdffb640 1 -- 172.21.15.5:0/2622572317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab4010910 con 0x7fdac0101de0 2024-09-11T22:27:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.279+0000 7fdac6c51640 1 -- 172.21.15.5:0/2622572317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac019d690 con 0x7fdac0101de0 2024-09-11T22:27:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.279+0000 7fdabdffb640 1 -- 172.21.15.5:0/2622572317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab4018990 con 0x7fdac0101de0 2024-09-11T22:27:55.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.279+0000 7fdac6c51640 1 -- 172.21.15.5:0/2622572317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac01a4f70 con 0x7fdac0101de0 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.281+0000 7fdabdffb640 1 -- 172.21.15.5:0/2622572317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fdab4010ac0 con 0x7fdac0101de0 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac902a740 1 Processor -- start 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac902a740 1 -- start start 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac902a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 0x55bfc098f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac902a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc098fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac902a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc099a780 0x55bfc099cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdabdffb640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab00a2cf0 0x7fdab00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac902a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bfc08f12f0 con 0x55bfc0990f50 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac902a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bfc0898cf0 con 0x55bfc099a780 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac902a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bfc08ed2b0 con 0x55bfc098f9e0 2024-09-11T22:27:55.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdabffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc099a780 0x55bfc099cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac49ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc098fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac51ee640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab00a2cf0 0x7fdab00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdabffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc099a780 0x55bfc099cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52572/0 (socket says 172.21.15.5:52572) 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdabffff640 1 -- 172.21.15.5:0/1716138907 learned_addr learned my addr 172.21.15.5:0/1716138907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdac51ee640 1 --2- 172.21.15.5:0/1716138907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 0x55bfc098f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.282+0000 7fdabdffb640 1 -- 172.21.15.5:0/2622572317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fdab40cf7a0 con 0x7fdac0101de0 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdabffff640 1 -- 172.21.15.5:0/1716138907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 msgr2=0x55bfc098fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdabffff640 1 --2- 172.21.15.5:0/1716138907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc098fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdabffff640 1 -- 172.21.15.5:0/1716138907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 msgr2=0x55bfc098f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdabffff640 1 --2- 172.21.15.5:0/1716138907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 0x55bfc098f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdabffff640 1 -- 172.21.15.5:0/1716138907 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfc087d3a0 con 0x55bfc099a780 2024-09-11T22:27:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdac49ed640 1 --2- 172.21.15.5:0/1716138907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc098fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdabffff640 1 --2- 172.21.15.5:0/1716138907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc099a780 0x55bfc099cb70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009c10 tx=0x7fdaac011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55624037754454b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdac51ee640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab00a2cf0 0x7fdab00a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdab8004af0 tx=0x7fdab80095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdaaaffd640 1 -- 172.21.15.5:0/1716138907 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac004230 con 0x55bfc099a780 2024-09-11T22:27:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdaaaffd640 1 -- 172.21.15.5:0/1716138907 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaac0043d0 con 0x55bfc099a780 2024-09-11T22:27:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.283+0000 7fdaaaffd640 1 -- 172.21.15.5:0/1716138907 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac004e10 con 0x55bfc099a780 2024-09-11T22:27:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 -- 172.21.15.5:0/1716138907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc099a780 msgr2=0x55bfc099cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 --2- 172.21.15.5:0/1716138907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc099a780 0x55bfc099cb70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009c10 tx=0x7fdaac011810 comp rx=0 tx=0).stop 2024-09-11T22:27:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 -- 172.21.15.5:0/1716138907 shutdown_connections 2024-09-11T22:27:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 --2- 172.21.15.5:0/1716138907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc099a780 0x55bfc099cb70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 --2- 172.21.15.5:0/1716138907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc098fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 --2- 172.21.15.5:0/1716138907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 0x55bfc098f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 -- 172.21.15.5:0/1716138907 >> 172.21.15.5:0/1716138907 conn(0x55bfc0954310 msgr2=0x55bfc098c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 -- 172.21.15.5:0/1716138907 shutdown_connections 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 -- 172.21.15.5:0/1716138907 wait complete. 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 -- start start 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.284+0000 7fdac902a740 1 Processor -- start 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac902a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc098f9e0 0x55bfc099a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac902a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 0x55bfc0997330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac902a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc099a780 0x55bfc0997870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac902a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bfc099d480 con 0x55bfc0990f50 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac902a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bfc08ac060 con 0x55bfc098f9e0 2024-09-11T22:27:55.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac902a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bfc099d600 con 0x55bfc099a780 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac902a740 10 client.0 _ll_register_callbacks cb 0x55bfc05ff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac49ed640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc098f9e0 0x55bfc099a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdabffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 0x55bfc0997330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac49ed640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc098f9e0 0x55bfc099a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52582/0 (socket says 172.21.15.5:52582) 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac51ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc099a780 0x55bfc0997870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac49ed640 1 -- 172.21.15.5:0/3148060508 learned_addr learned my addr 172.21.15.5:0/3148060508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdabffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 0x55bfc0997330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48812/0 (socket says 172.21.15.5:48812) 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdabffff640 1 -- 172.21.15.5:0/3148060508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc099a780 msgr2=0x55bfc0997870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdabffff640 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc099a780 0x55bfc0997870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdabffff640 1 -- 172.21.15.5:0/3148060508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc098f9e0 msgr2=0x55bfc099a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdabffff640 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc098f9e0 0x55bfc099a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdabffff640 1 -- 172.21.15.5:0/3148060508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfc06623c0 con 0x55bfc0990f50 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.285+0000 7fdac49ed640 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc098f9e0 0x55bfc099a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fdabffff640 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 0x55bfc0997330 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fdaac000c00 tx=0x7fdaac005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe0559a9aaae44e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fda9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fda9ffff640 1 -- 172.21.15.5:0/3148060508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac024620 con 0x55bfc0990f50 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fda9ffff640 1 -- 172.21.15.5:0/3148060508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fdaac0247c0 con 0x55bfc0990f50 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fdac902a740 1 -- 172.21.15.5:0/3148060508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaac0043d0 con 0x55bfc0990f50 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fda9ffff640 1 -- 172.21.15.5:0/3148060508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdaac032050 con 0x55bfc0990f50 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fdac902a740 10 client.17142 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fdac902a740 1 -- 172.21.15.5:0/3148060508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bfc09a8e30 con 0x55bfc0990f50 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fda9ffff640 1 -- 172.21.15.5:0/3148060508 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaac0043d0 con 0x55bfc0990f50 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fda9ffff640 1 -- 172.21.15.5:0/3148060508 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fdaac018070 con 0x55bfc0990f50 2024-09-11T22:27:55.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fdac902a740 10 client.17142 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fdac902a740 10 client.17142 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fdac902a740 20 client.17142 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fdac902a740 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55bfc09aa330 0x55bfc09ac720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fdac902a740 4 client.17142 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.286+0000 7fdac902a740 1 -- 172.21.15.5:0/3148060508 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfc08fe860 con 0x55bfc09aa330 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.287+0000 7fdac49ed640 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55bfc09aa330 0x55bfc09ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.287+0000 7fdac49ed640 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55bfc09aa330 0x55bfc09ac720 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c7ce2cd2428bc824 server_cookie=2bcff62c15165cf5 in_seq=0 out_seq=0 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.287+0000 7fda9ffff640 10 client.17142 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fda9ffff640 1 -- 172.21.15.5:0/3148060508 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bfc08fe860 con 0x55bfc09aa330 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fda9ffff640 10 client.17142 handle_command_reply: tid=0 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 client.17142 shutdown 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 -- 172.21.15.5:0/3148060508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 msgr2=0x55bfc0997330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:55.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 0x55bfc0997330 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fdaac000c00 tx=0x7fdaac005130 comp rx=0 tx=0).stop 2024-09-11T22:27:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 -- 172.21.15.5:0/3148060508 shutdown_connections 2024-09-11T22:27:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55bfc09aa330 0x55bfc09ac720 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc099a780 0x55bfc0997870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc0990f50 0x55bfc0997330 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 --2- 172.21.15.5:0/3148060508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc098f9e0 0x55bfc099a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fda9ffff640 0 client.17142 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:27:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 -- 172.21.15.5:0/3148060508 >> 172.21.15.5:0/3148060508 conn(0x55bfc0954310 msgr2=0x55bfc0991d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 -- 172.21.15.5:0/3148060508 shutdown_connections 2024-09-11T22:27:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 1 -- 172.21.15.5:0/3148060508 wait complete. 2024-09-11T22:27:55.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.288+0000 7fdac902a740 20 client.17142 trim_cache size 0 max 16384 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.301+0000 7fdac902a740 1 Processor -- start 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.301+0000 7fdac902a740 1 -- start start 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac902a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc098f9e0 0x55bfc09a6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac902a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 0x55bfc0995c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac902a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc099a780 0x55bfc0996140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac902a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01ab820 con 0x55bfc098f9e0 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac902a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01ad360 con 0x55bfc0990f50 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac902a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01abef0 con 0x55bfc099a780 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac49ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc098f9e0 0x55bfc09a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac49ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc098f9e0 0x55bfc09a6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48816/0 (socket says 172.21.15.5:48816) 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac49ed640 1 -- 172.21.15.5:0/2512521535 learned_addr learned my addr 172.21.15.5:0/2512521535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:55.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdabffff640 1 --2- 172.21.15.5:0/2512521535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 0x55bfc0995c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac51ee640 1 --2- 172.21.15.5:0/2512521535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc099a780 0x55bfc0996140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac49ed640 1 -- 172.21.15.5:0/2512521535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc099a780 msgr2=0x55bfc0996140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac49ed640 1 --2- 172.21.15.5:0/2512521535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc099a780 0x55bfc0996140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac49ed640 1 -- 172.21.15.5:0/2512521535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 msgr2=0x55bfc0995c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac49ed640 1 --2- 172.21.15.5:0/2512521535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 0x55bfc0995c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac49ed640 1 -- 172.21.15.5:0/2512521535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfc0992530 con 0x55bfc098f9e0 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdac49ed640 1 --2- 172.21.15.5:0/2512521535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc098f9e0 0x55bfc09a6720 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdab40986b0 tx=0x7fdab40cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1b18e1fd944997 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.302+0000 7fdaaa7fc640 1 -- 172.21.15.5:0/2512521535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab40cd130 con 0x55bfc098f9e0 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdaaa7fc640 1 -- 172.21.15.5:0/2512521535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab40cd2d0 con 0x55bfc098f9e0 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdaaa7fc640 1 -- 172.21.15.5:0/2512521535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab409b3e0 con 0x55bfc098f9e0 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdac902a740 1 -- 172.21.15.5:0/2512521535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc098f9e0 msgr2=0x55bfc09a6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:55.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdac902a740 1 --2- 172.21.15.5:0/2512521535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc098f9e0 0x55bfc09a6720 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdab40986b0 tx=0x7fdab40cf040 comp rx=0 tx=0).stop 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdac902a740 1 -- 172.21.15.5:0/2512521535 shutdown_connections 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdac902a740 1 --2- 172.21.15.5:0/2512521535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc099a780 0x55bfc0996140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdac902a740 1 --2- 172.21.15.5:0/2512521535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 0x55bfc0995c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdac902a740 1 --2- 172.21.15.5:0/2512521535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc098f9e0 0x55bfc09a6720 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdac902a740 1 -- 172.21.15.5:0/2512521535 >> 172.21.15.5:0/2512521535 conn(0x55bfc0954310 msgr2=0x55bfc09972f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdac902a740 1 -- 172.21.15.5:0/2512521535 shutdown_connections 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.303+0000 7fdac902a740 1 -- 172.21.15.5:0/2512521535 wait complete. 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac902a740 1 -- start start 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac902a740 1 Processor -- start 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac902a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc09a2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac902a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 0x55bfc0993de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac902a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc099a780 0x55bfc0994320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac902a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01acc90 con 0x55bfc099a780 2024-09-11T22:27:55.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac902a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01ac5c0 con 0x55bfc0990f50 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac902a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01a9800 con 0x55bfc098f9e0 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac902a740 10 client.0 _ll_register_callbacks cb 0x7fdab4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac49ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc09a2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac51ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc099a780 0x55bfc0994320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdabffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 0x55bfc0993de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac49ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc09a2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:38026/0 (socket says 172.21.15.5:38026) 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac49ed640 1 -- 172.21.15.5:0/1474026760 learned_addr learned my addr 172.21.15.5:0/1474026760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.304+0000 7fdac51ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc099a780 0x55bfc0994320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48818/0 (socket says 172.21.15.5:48818) 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdac49ed640 1 -- 172.21.15.5:0/1474026760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 msgr2=0x55bfc0993de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdac49ed640 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 0x55bfc0993de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdac49ed640 1 -- 172.21.15.5:0/1474026760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc099a780 msgr2=0x55bfc0994320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdac49ed640 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc099a780 0x55bfc0994320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdac49ed640 1 -- 172.21.15.5:0/1474026760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfc09948e0 con 0x55bfc098f9e0 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdabffff640 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 0x55bfc0993de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdac51ee640 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc099a780 0x55bfc0994320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdac49ed640 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc09a2d70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000c00 tx=0x7fdab400cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c34154ba5f410489 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fda9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fda9ffff640 1 -- 172.21.15.5:0/1474026760 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab40abaa0 con 0x55bfc098f9e0 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fda9ffff640 1 -- 172.21.15.5:0/1474026760 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fdab40abc40 con 0x55bfc098f9e0 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdac902a740 1 -- 172.21.15.5:0/1474026760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab40cd2d0 con 0x55bfc098f9e0 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fda9ffff640 1 -- 172.21.15.5:0/1474026760 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdab40a4b40 con 0x55bfc098f9e0 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdac902a740 10 client.26488 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.305+0000 7fdac902a740 1 -- 172.21.15.5:0/1474026760 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bfc0a14bc0 con 0x55bfc098f9e0 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fda9ffff640 1 -- 172.21.15.5:0/1474026760 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdab40cd2d0 con 0x55bfc098f9e0 2024-09-11T22:27:55.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fda9ffff640 1 -- 172.21.15.5:0/1474026760 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fdab4097a80 con 0x55bfc098f9e0 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fdac902a740 10 client.26488 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fdac902a740 10 client.26488 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fdac902a740 20 client.26488 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fdac902a740 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55bfc0a15280 0x55bfc0a176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fdac902a740 4 client.26488 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fdac902a740 1 -- 172.21.15.5:0/1474026760 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfc08f8d70 con 0x55bfc0a15280 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fdabffff640 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55bfc0a15280 0x55bfc0a176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fda9ffff640 10 client.26488 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.306+0000 7fdabffff640 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55bfc0a15280 0x55bfc0a176a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4ed7ae2d28a7565 server_cookie=1289ad0d4f5a4312 in_seq=0 out_seq=0 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fda9ffff640 1 -- 172.21.15.5:0/1474026760 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfc08f8d70 con 0x55bfc0a15280 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fda9ffff640 10 client.26488 handle_command_reply: tid=0 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 client.26488 shutdown 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 -- 172.21.15.5:0/1474026760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 msgr2=0x55bfc09a2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:55.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc09a2d70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000c00 tx=0x7fdab400cf90 comp rx=0 tx=0).stop 2024-09-11T22:27:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 -- 172.21.15.5:0/1474026760 shutdown_connections 2024-09-11T22:27:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55bfc0a15280 0x55bfc0a176a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc099a780 0x55bfc0994320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bfc0990f50 0x55bfc0993de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 --2- 172.21.15.5:0/1474026760 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfc098f9e0 0x55bfc09a2d70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fda9ffff640 0 client.26488 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:27:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 -- 172.21.15.5:0/1474026760 >> 172.21.15.5:0/1474026760 conn(0x55bfc0954310 msgr2=0x55bfc0995b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 -- 172.21.15.5:0/1474026760 shutdown_connections 2024-09-11T22:27:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 1 -- 172.21.15.5:0/1474026760 wait complete. 2024-09-11T22:27:55.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.307+0000 7fdac902a740 20 client.26488 trim_cache size 0 max 16384 2024-09-11T22:27:55.322 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:27:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 -- 172.21.15.5:0/2622572317 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab00a2cf0 msgr2=0x7fdab00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab00a2cf0 0x7fdab00a51b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdab8004af0 tx=0x7fdab80095c0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 -- 172.21.15.5:0/2622572317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101de0 msgr2=0x7fdac01983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101de0 0x7fdac01983f0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fdab400e9f0 tx=0x7fdab400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 -- 172.21.15.5:0/2622572317 shutdown_connections 2024-09-11T22:27:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdab00a2cf0 0x7fdab00a51b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdac0106f10 0x7fdac019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdac0102790 0x7fdac0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 --2- 172.21.15.5:0/2622572317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdac0101de0 0x7fdac01983f0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 -- 172.21.15.5:0/2622572317 >> 172.21.15.5:0/2622572317 conn(0x7fdac00fd8c0 msgr2=0x7fdac00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 -- 172.21.15.5:0/2622572317 shutdown_connections 2024-09-11T22:27:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:55.308+0000 7fdac6c51640 1 -- 172.21.15.5:0/2622572317 wait complete. 2024-09-11T22:27:55.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 417375 8.19 MB/sec execute 133 sec latency 29.227 ms 2024-09-11T22:27:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:55 smithi005 ceph-mon[30943]: pgmap v175: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-09-11T22:27:55.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:55 smithi159 ceph-mon[35534]: pgmap v175: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-09-11T22:27:55.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:55 smithi172 ceph-mon[35461]: pgmap v175: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-09-11T22:27:56.043 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:27:56.320 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:56.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 419375 8.23 MB/sec execute 134 sec latency 22.675 ms 2024-09-11T22:27:56.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.812+0000 7fe8ede4b640 1 -- 172.21.15.5:0/1262284089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8101de0 msgr2=0x7fe8e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:56.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.812+0000 7fe8ede4b640 1 --2- 172.21.15.5:0/1262284089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8101de0 0x7fe8e81021c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8009900 tx=0x7fe8d802a410 comp rx=0 tx=0).stop 2024-09-11T22:27:56.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.812+0000 7fe8ede4b640 1 -- 172.21.15.5:0/1262284089 shutdown_connections 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.812+0000 7fe8ede4b640 1 --2- 172.21.15.5:0/1262284089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e8106f10 0x7fe8e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.812+0000 7fe8ede4b640 1 --2- 172.21.15.5:0/1262284089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8102790 0x7fe8e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.812+0000 7fe8ede4b640 1 --2- 172.21.15.5:0/1262284089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8101de0 0x7fe8e81021c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.812+0000 7fe8ede4b640 1 -- 172.21.15.5:0/1262284089 >> 172.21.15.5:0/1262284089 conn(0x7fe8e80fd8c0 msgr2=0x7fe8e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.813+0000 7fe8ede4b640 1 -- 172.21.15.5:0/1262284089 shutdown_connections 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.813+0000 7fe8ede4b640 1 -- 172.21.15.5:0/1262284089 wait complete. 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.813+0000 7fe8ede4b640 1 Processor -- start 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.813+0000 7fe8ede4b640 1 -- start start 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8ede4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e8101de0 0x7fe8e8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8ede4b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8102790 0x7fe8e81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8ede4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8106f10 0x7fe8e819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8ede4b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8111f80 con 0x7fe8e8106f10 2024-09-11T22:27:56.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8ede4b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8111e00 con 0x7fe8e8102790 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8ede4b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8112100 con 0x7fe8e8101de0 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8106f10 0x7fe8e819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8106f10 0x7fe8e819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48832/0 (socket says 172.21.15.5:48832) 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8e7fff640 1 -- 172.21.15.5:0/2428721559 learned_addr learned my addr 172.21.15.5:0/2428721559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e8101de0 0x7fe8e8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8102790 0x7fe8e81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8e7fff640 1 -- 172.21.15.5:0/2428721559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e8101de0 msgr2=0x7fe8e8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8e7fff640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e8101de0 0x7fe8e8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8e7fff640 1 -- 172.21.15.5:0/2428721559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8102790 msgr2=0x7fe8e81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8e7fff640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8102790 0x7fe8e81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.814+0000 7fe8e7fff640 1 -- 172.21.15.5:0/2428721559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e819d410 con 0x7fe8e8106f10 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.815+0000 7fe8e7fff640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8106f10 0x7fe8e819cd30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000eaa0 tx=0x7fe8e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.815+0000 7fe8e4ff9640 1 -- 172.21.15.5:0/2428721559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e000cdb0 con 0x7fe8e8106f10 2024-09-11T22:27:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.815+0000 7fe8ede4b640 1 -- 172.21.15.5:0/2428721559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e819d700 con 0x7fe8e8106f10 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.815+0000 7fe8e4ff9640 1 -- 172.21.15.5:0/2428721559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e0005020 con 0x7fe8e8106f10 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.815+0000 7fe8e4ff9640 1 -- 172.21.15.5:0/2428721559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00116b0 con 0x7fe8e8106f10 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.815+0000 7fe8ede4b640 1 -- 172.21.15.5:0/2428721559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e81a4fe0 con 0x7fe8e8106f10 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.817+0000 7fe8e4ff9640 1 -- 172.21.15.5:0/2428721559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe8e0005560 con 0x7fe8e8106f10 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8f0224740 1 Processor -- start 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8f0224740 1 -- start start 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14440 0x55a7e0f14820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14df0 0x55a7e0f19470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f19ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e4ff9640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe8d00a2cf0 0x7fe8d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8f0224740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a7e0e702f0 con 0x55a7e0f19a40 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8f0224740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a7e0e17cf0 con 0x55a7e0f14df0 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8f0224740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7e0e6c2b0 con 0x55a7e0f14440 2024-09-11T22:27:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14440 0x55a7e0f14820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14df0 0x55a7e0f19470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14440 0x55a7e0f14820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:38060/0 (socket says 172.21.15.5:38060) 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/2167823455 learned_addr learned my addr 172.21.15.5:0/2167823455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e7fff640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe8d00a2cf0 0x7fe8d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/2167823455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f19ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e4ff9640 1 -- 172.21.15.5:0/2428721559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe8e00cf5f0 con 0x7fe8e8106f10 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e77fe640 1 -- 172.21.15.5:0/2167823455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14440 msgr2=0x55a7e0f14820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/2167823455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14440 0x55a7e0f14820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e77fe640 1 -- 172.21.15.5:0/2167823455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 msgr2=0x55a7e0f19ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/2167823455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f19ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e77fe640 1 -- 172.21.15.5:0/2167823455 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e0dfc3a0 con 0x55a7e0f14df0 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.818+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/2167823455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f19ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.819+0000 7fe8e7fff640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe8d00a2cf0 0x7fe8d00a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0097860 tx=0x7fe8e00866b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.819+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/2167823455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14df0 0x55a7e0f19470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe8d802aa00 tx=0x7fe8d80040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df45ddf2cac8f6c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.819+0000 7fe8cdffb640 1 -- 172.21.15.5:0/2167823455 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8035040 con 0x55a7e0f14df0 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.819+0000 7fe8cdffb640 1 -- 172.21.15.5:0/2167823455 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d8005520 con 0x55a7e0f14df0 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.819+0000 7fe8cdffb640 1 -- 172.21.15.5:0/2167823455 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8003850 con 0x55a7e0f14df0 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 -- 172.21.15.5:0/2167823455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14df0 msgr2=0x55a7e0f19470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 --2- 172.21.15.5:0/2167823455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14df0 0x55a7e0f19470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe8d802aa00 tx=0x7fe8d80040f0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 -- 172.21.15.5:0/2167823455 shutdown_connections 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 --2- 172.21.15.5:0/2167823455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f19ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 --2- 172.21.15.5:0/2167823455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14df0 0x55a7e0f19470 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 --2- 172.21.15.5:0/2167823455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14440 0x55a7e0f14820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 -- 172.21.15.5:0/2167823455 >> 172.21.15.5:0/2167823455 conn(0x55a7e0ed3310 msgr2=0x55a7e0f066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 -- 172.21.15.5:0/2167823455 shutdown_connections 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 -- 172.21.15.5:0/2167823455 wait complete. 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 -- start start 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.820+0000 7fe8f0224740 1 Processor -- start 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 0x55a7e0f1fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f1d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f1d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8f0224740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a7e0f20870 con 0x55a7e0f19a40 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8f0224740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a7e0e2b060 con 0x55a7e0f14440 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8f0224740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7e0f209f0 con 0x55a7e0f14df0 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8f0224740 10 client.0 _ll_register_callbacks cb 0x55a7e0b7e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 0x55a7e0f1fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f1d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 0x55a7e0f1fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52654/0 (socket says 172.21.15.5:52654) 2024-09-11T22:27:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/2125969228 learned_addr learned my addr 172.21.15.5:0/2125969228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f1d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/2125969228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 msgr2=0x55a7e0f1d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f1d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/2125969228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 msgr2=0x55a7e0f1d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f1d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/2125969228 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e0be13c0 con 0x55a7e0f14440 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f1d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 0x55a7e0f1fe30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc013d10 tx=0x7fe8dc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e78c567fbed7ff00 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.821+0000 7fe8c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/2125969228 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc010e70 con 0x55a7e0f14440 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8f0224740 1 -- 172.21.15.5:0/2125969228 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d8005520 con 0x55a7e0f14440 2024-09-11T22:27:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/2125969228 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe8dc021420 con 0x55a7e0f14440 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/2125969228 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe8dc021620 con 0x55a7e0f14440 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8f0224740 10 client.26576 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8f0224740 1 -- 172.21.15.5:0/2125969228 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a7e0f2c230 con 0x55a7e0f14440 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/2125969228 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d8005520 con 0x55a7e0f14440 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/2125969228 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe8dc020730 con 0x55a7e0f14440 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8f0224740 10 client.26576 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8f0224740 10 client.26576 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8f0224740 20 client.26576 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8f0224740 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a7e0f2d6f0 0x55a7e0f2fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8f0224740 4 client.26576 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.822+0000 7fe8f0224740 1 -- 172.21.15.5:0/2125969228 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7e0e7d860 con 0x55a7e0f2d6f0 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.823+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a7e0f2d6f0 0x55a7e0f2fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.823+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a7e0f2d6f0 0x55a7e0f2fb00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f24dd72d5fa04655 server_cookie=16ccb8e859ccc5bf in_seq=0 out_seq=0 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.823+0000 7fe8c6ffd640 10 client.26576 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/2125969228 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7e0e7d860 con 0x55a7e0f2d6f0 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8c6ffd640 10 client.26576 handle_command_reply: tid=0 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 client.26576 shutdown 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 -- 172.21.15.5:0/2125969228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 msgr2=0x55a7e0f1fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 0x55a7e0f1fe30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc013d10 tx=0x7fe8dc004730 comp rx=0 tx=0).stop 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 -- 172.21.15.5:0/2125969228 shutdown_connections 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f1d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f1d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a7e0f2d6f0 0x55a7e0f2fb00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 --2- 172.21.15.5:0/2125969228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 0x55a7e0f1fe30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 -- 172.21.15.5:0/2125969228 >> 172.21.15.5:0/2125969228 conn(0x55a7e0ed3310 msgr2=0x55a7e0f13a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8c6ffd640 0 client.26576 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:27:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 -- 172.21.15.5:0/2125969228 shutdown_connections 2024-09-11T22:27:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 1 -- 172.21.15.5:0/2125969228 wait complete. 2024-09-11T22:27:56.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.824+0000 7fe8f0224740 20 client.26576 trim_cache size 0 max 16384 2024-09-11T22:27:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.839+0000 7fe8f0224740 1 Processor -- start 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.839+0000 7fe8f0224740 1 -- start start 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.839+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f14440 0x55a7e0f29b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.839+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f05500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.839+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f19a40 0x55a7e0f05a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.839+0000 7fe8f0224740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81ab7f0 con 0x55a7e0f14440 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.839+0000 7fe8f0224740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81ad330 con 0x55a7e0f19a40 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.839+0000 7fe8f0224740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81abec0 con 0x55a7e0f14df0 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f14440 0x55a7e0f29b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f05500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f14440 0x55a7e0f29b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48872/0 (socket says 172.21.15.5:48872) 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f05500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:38076/0 (socket says 172.21.15.5:38076) 2024-09-11T22:27:56.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/3640997233 learned_addr learned my addr 172.21.15.5:0/3640997233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/3640997233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f19a40 0x55a7e0f05a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/3640997233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 msgr2=0x55a7e0f05500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/3640997233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f05500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/3640997233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f19a40 msgr2=0x55a7e0f05a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/3640997233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f19a40 0x55a7e0f05a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/3640997233 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e0f06c20 con 0x55a7e0f14440 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/3640997233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f05500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/3640997233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f14440 0x55a7e0f29b10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc013aa0 tx=0x7fe8dc005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d21f32ceb41039a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.840+0000 7fe8cd7fa640 1 -- 172.21.15.5:0/3640997233 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc037070 con 0x55a7e0f14440 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8cd7fa640 1 -- 172.21.15.5:0/3640997233 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8dc021420 con 0x55a7e0f14440 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8cd7fa640 1 -- 172.21.15.5:0/3640997233 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc0217c0 con 0x55a7e0f14440 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8f0224740 1 -- 172.21.15.5:0/3640997233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f14440 msgr2=0x55a7e0f29b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8f0224740 1 --2- 172.21.15.5:0/3640997233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f14440 0x55a7e0f29b10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc013aa0 tx=0x7fe8dc005fb0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8f0224740 1 -- 172.21.15.5:0/3640997233 shutdown_connections 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8f0224740 1 --2- 172.21.15.5:0/3640997233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f19a40 0x55a7e0f05a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8f0224740 1 --2- 172.21.15.5:0/3640997233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f05500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8f0224740 1 --2- 172.21.15.5:0/3640997233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f14440 0x55a7e0f29b10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8f0224740 1 -- 172.21.15.5:0/3640997233 >> 172.21.15.5:0/3640997233 conn(0x55a7e0ed3310 msgr2=0x55a7e0f18df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8f0224740 1 -- 172.21.15.5:0/3640997233 shutdown_connections 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.841+0000 7fe8f0224740 1 -- 172.21.15.5:0/3640997233 wait complete. 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8f0224740 1 -- start start 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8f0224740 1 Processor -- start 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 0x55a7e0f25c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f261c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8f0224740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f343c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8f0224740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81acc60 con 0x55a7e0f19a40 2024-09-11T22:27:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8f0224740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81ac590 con 0x55a7e0f14440 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8f0224740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81a97d0 con 0x55a7e0f14df0 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8f0224740 10 client.0 _ll_register_callbacks cb 0x7fe8dc013570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 0x55a7e0f25c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.842+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f343c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8e6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f343c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48882/0 (socket says 172.21.15.5:48882) 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/611873069 learned_addr learned my addr 172.21.15.5:0/611873069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f261c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/611873069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 msgr2=0x55a7e0f261c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f261c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/611873069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 msgr2=0x55a7e0f25c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 0x55a7e0f25c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8e6ffd640 1 -- 172.21.15.5:0/611873069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e0f26800 con 0x55a7e0f19a40 2024-09-11T22:27:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8e6ffd640 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f343c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x55a7e0f18030 tx=0x7fe8dc00b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72daa58795e9edb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/611873069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc037070 con 0x55a7e0f19a40 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/611873069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fe8dc004020 con 0x55a7e0f19a40 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8f0224740 1 -- 172.21.15.5:0/611873069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8dc021420 con 0x55a7e0f19a40 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/611873069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe8dc004280 con 0x55a7e0f19a40 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8f0224740 10 client.17166 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.843+0000 7fe8f0224740 1 -- 172.21.15.5:0/611873069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a7e0f978c0 con 0x55a7e0f19a40 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.844+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/611873069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8dc021420 con 0x55a7e0f19a40 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.844+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/611873069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe8dc03d060 con 0x55a7e0f19a40 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.844+0000 7fe8f0224740 10 client.17166 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.844+0000 7fe8f0224740 10 client.17166 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.844+0000 7fe8f0224740 20 client.17166 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:56.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.844+0000 7fe8f0224740 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a7e0f97f10 0x55a7e0f9a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.844+0000 7fe8f0224740 4 client.17166 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.844+0000 7fe8f0224740 1 -- 172.21.15.5:0/611873069 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7e0e77d70 con 0x55a7e0f97f10 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.844+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a7e0f97f10 0x55a7e0f9a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a7e0f97f10 0x55a7e0f9a300 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0e35528164d6be8 server_cookie=c0e31296742977b3 in_seq=0 out_seq=0 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8c6ffd640 10 client.17166 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8c6ffd640 1 -- 172.21.15.5:0/611873069 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7e0e77d70 con 0x55a7e0f97f10 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8c6ffd640 10 client.17166 handle_command_reply: tid=0 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8f0224740 1 client.17166 shutdown 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8f0224740 1 -- 172.21.15.5:0/611873069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 msgr2=0x55a7e0f343c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8f0224740 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f343c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x55a7e0f18030 tx=0x7fe8dc00b710 comp rx=0 tx=0).stop 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8f0224740 1 -- 172.21.15.5:0/611873069 shutdown_connections 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8f0224740 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7e0f19a40 0x55a7e0f343c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8f0224740 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e0f14df0 0x55a7e0f261c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8f0224740 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a7e0f97f10 0x55a7e0f9a300 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8f0224740 1 --2- 172.21.15.5:0/611873069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7e0f14440 0x55a7e0f25c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.845+0000 7fe8f0224740 1 -- 172.21.15.5:0/611873069 >> 172.21.15.5:0/611873069 conn(0x55a7e0ed3310 msgr2=0x55a7e0f13250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.846+0000 7fe8c6ffd640 0 client.17166 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:27:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.846+0000 7fe8f0224740 1 -- 172.21.15.5:0/611873069 shutdown_connections 2024-09-11T22:27:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.846+0000 7fe8f0224740 1 -- 172.21.15.5:0/611873069 wait complete. 2024-09-11T22:27:56.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.846+0000 7fe8f0224740 20 client.17166 trim_cache size 0 max 16384 2024-09-11T22:27:56.857 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:27:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.846+0000 7fe8ede4b640 1 -- 172.21.15.5:0/2428721559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe8d00a2cf0 msgr2=0x7fe8d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.846+0000 7fe8ede4b640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe8d00a2cf0 0x7fe8d00a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0097860 tx=0x7fe8e00866b0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.846+0000 7fe8ede4b640 1 -- 172.21.15.5:0/2428721559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8106f10 msgr2=0x7fe8e819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.846+0000 7fe8ede4b640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8106f10 0x7fe8e819cd30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000eaa0 tx=0x7fe8e000ef60 comp rx=0 tx=0).stop 2024-09-11T22:27:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.847+0000 7fe8ede4b640 1 -- 172.21.15.5:0/2428721559 shutdown_connections 2024-09-11T22:27:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.847+0000 7fe8ede4b640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8106f10 0x7fe8e819cd30 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.847+0000 7fe8ede4b640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe8d00a2cf0 0x7fe8d00a51b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.847+0000 7fe8ede4b640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8102790 0x7fe8e81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.847+0000 7fe8ede4b640 1 --2- 172.21.15.5:0/2428721559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e8101de0 0x7fe8e8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.847+0000 7fe8ede4b640 1 -- 172.21.15.5:0/2428721559 >> 172.21.15.5:0/2428721559 conn(0x7fe8e80fd8c0 msgr2=0x7fe8e80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.847+0000 7fe8ede4b640 1 -- 172.21.15.5:0/2428721559 shutdown_connections 2024-09-11T22:27:56.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:56.847+0000 7fe8ede4b640 1 -- 172.21.15.5:0/2428721559 wait complete. 2024-09-11T22:27:57.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 421957 8.27 MB/sec execute 135 sec latency 19.673 ms 2024-09-11T22:27:57.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:27:57.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:57 smithi005 ceph-mon[30943]: pgmap v176: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-09-11T22:27:57.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:57 smithi159 ceph-mon[35534]: pgmap v176: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-09-11T22:27:57.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:57 smithi172 ceph-mon[35461]: pgmap v176: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 287 op/s 2024-09-11T22:27:57.926 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:27:58.321 INFO:tasks.workunit.client.0.smithi005.stdout: 1 423715 8.28 MB/sec execute 136 sec latency 46.749 ms 2024-09-11T22:27:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.448+0000 7fa1043f4640 1 -- 172.21.15.5:0/991259356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0fc100e20 msgr2=0x7fa0fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.448+0000 7fa1043f4640 1 --2- 172.21.15.5:0/991259356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0fc100e20 0x7fa0fc101200 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009900 tx=0x7fa0f802a480 comp rx=0 tx=0).stop 2024-09-11T22:27:58.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.449+0000 7fa1043f4640 1 -- 172.21.15.5:0/991259356 shutdown_connections 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.449+0000 7fa1043f4640 1 --2- 172.21.15.5:0/991259356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0fc102ef0 0x7fa0fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.449+0000 7fa1043f4640 1 --2- 172.21.15.5:0/991259356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0fc102570 0x7fa0fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.449+0000 7fa1043f4640 1 --2- 172.21.15.5:0/991259356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0fc100e20 0x7fa0fc101200 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.449+0000 7fa1043f4640 1 -- 172.21.15.5:0/991259356 >> 172.21.15.5:0/991259356 conn(0x7fa0fc0fc7d0 msgr2=0x7fa0fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.449+0000 7fa1043f4640 1 -- 172.21.15.5:0/991259356 shutdown_connections 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.449+0000 7fa1043f4640 1 -- 172.21.15.5:0/991259356 wait complete. 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.449+0000 7fa1043f4640 1 Processor -- start 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa1043f4640 1 -- start start 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa1043f4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0fc100e20 0x7fa0fc199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa1043f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0fc102570 0x7fa0fc1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa1043f4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0fc102ef0 0x7fa0fc1a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa1043f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc110f80 con 0x7fa0fc102570 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa1043f4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc110e00 con 0x7fa0fc100e20 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa1043f4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc111100 con 0x7fa0fc102ef0 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa10198f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0fc102570 0x7fa0fc1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa10198f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0fc102570 0x7fa0fc1998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48898/0 (socket says 172.21.15.5:48898) 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa10198f640 1 -- 172.21.15.5:0/577186156 learned_addr learned my addr 172.21.15.5:0/577186156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa102991640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0fc102ef0 0x7fa0fc1a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa102190640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0fc100e20 0x7fa0fc199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa10198f640 1 -- 172.21.15.5:0/577186156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0fc102ef0 msgr2=0x7fa0fc1a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa10198f640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0fc102ef0 0x7fa0fc1a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa10198f640 1 -- 172.21.15.5:0/577186156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0fc100e20 msgr2=0x7fa0fc199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa10198f640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0fc100e20 0x7fa0fc199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa10198f640 1 -- 172.21.15.5:0/577186156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc1a4940 con 0x7fa0fc102570 2024-09-11T22:27:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.450+0000 7fa10198f640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0fc102570 0x7fa0fc1998c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00e9f0 tx=0x7fa0ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.451+0000 7fa0f37fe640 1 -- 172.21.15.5:0/577186156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec00cd80 con 0x7fa0fc102570 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.451+0000 7fa1043f4640 1 -- 172.21.15.5:0/577186156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc1a4b10 con 0x7fa0fc102570 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.451+0000 7fa0f37fe640 1 -- 172.21.15.5:0/577186156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ec005500 con 0x7fa0fc102570 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.451+0000 7fa0f37fe640 1 -- 172.21.15.5:0/577186156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec010690 con 0x7fa0fc102570 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.451+0000 7fa1043f4640 1 -- 172.21.15.5:0/577186156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc1a4fd0 con 0x7fa0fc102570 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.453+0000 7fa0f37fe640 1 -- 172.21.15.5:0/577186156 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa0ec0046c0 con 0x7fa0fc102570 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.453+0000 7fa1067cd740 1 Processor -- start 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.453+0000 7fa1067cd740 1 -- start start 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 0x55f4dd7b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa1067cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd70c2f0 con 0x55f4dd7a7b40 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa1067cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd6b3cf0 con 0x55f4dd7b5780 2024-09-11T22:27:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa1067cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd7082b0 con 0x55f4dd7a65d0 2024-09-11T22:27:58.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa102991640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa102991640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7a6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48914/0 (socket says 172.21.15.5:48914) 2024-09-11T22:27:58.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa102991640 1 -- 172.21.15.5:0/2863008608 learned_addr learned my addr 172.21.15.5:0/2863008608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:58.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa10198f640 1 --2- 172.21.15.5:0/2863008608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 0x55f4dd7b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa102190640 1 --2- 172.21.15.5:0/2863008608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa102991640 1 -- 172.21.15.5:0/2863008608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 msgr2=0x55f4dd7a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:58.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa102991640 1 --2- 172.21.15.5:0/2863008608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa102991640 1 -- 172.21.15.5:0/2863008608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 msgr2=0x55f4dd7b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa102991640 1 --2- 172.21.15.5:0/2863008608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 0x55f4dd7b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa102991640 1 -- 172.21.15.5:0/2863008608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dd6983a0 con 0x55f4dd7a7b40 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa0f37fe640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0e80a2cf0 0x7fa0e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.454+0000 7fa0f37fe640 1 -- 172.21.15.5:0/577186156 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fa0ec0cf870 con 0x7fa0fc102570 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.455+0000 7fa102991640 1 --2- 172.21.15.5:0/2863008608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7a6090 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009f50 tx=0x7fa0f40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25159f15c8a8a60d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.455+0000 7fa102190640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0e80a2cf0 0x7fa0e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.455+0000 7fa0f0dea640 1 -- 172.21.15.5:0/2863008608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4004b20 con 0x55f4dd7a7b40 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.455+0000 7fa0f0dea640 1 -- 172.21.15.5:0/2863008608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0f4004cc0 con 0x55f4dd7a7b40 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.455+0000 7fa0f0dea640 1 -- 172.21.15.5:0/2863008608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4005020 con 0x55f4dd7a7b40 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa1067cd740 1 -- 172.21.15.5:0/2863008608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 msgr2=0x55f4dd7a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa102190640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0e80a2cf0 0x7fa0e80a51b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009230 tx=0x7fa0f80058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:58.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa1067cd740 1 --2- 172.21.15.5:0/2863008608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7a6090 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009f50 tx=0x7fa0f40118d0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa1067cd740 1 -- 172.21.15.5:0/2863008608 shutdown_connections 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa1067cd740 1 --2- 172.21.15.5:0/2863008608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 0x55f4dd7b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa1067cd740 1 --2- 172.21.15.5:0/2863008608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa1067cd740 1 --2- 172.21.15.5:0/2863008608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7a6090 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa1067cd740 1 -- 172.21.15.5:0/2863008608 >> 172.21.15.5:0/2863008608 conn(0x55f4dd76f310 msgr2=0x55f4dd7a2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa1067cd740 1 -- 172.21.15.5:0/2863008608 shutdown_connections 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa1067cd740 1 -- 172.21.15.5:0/2863008608 wait complete. 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.456+0000 7fa1067cd740 1 -- start start 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa1067cd740 1 Processor -- start 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7b5780 0x55f4dd7b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa1067cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd7b8480 con 0x55f4dd7a7b40 2024-09-11T22:27:58.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa1067cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd6c7060 con 0x55f4dd7a65d0 2024-09-11T22:27:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa1067cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4dd7b8600 con 0x55f4dd7b5780 2024-09-11T22:27:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa1067cd740 10 client.0 _ll_register_callbacks cb 0x55f4dd41a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa10198f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa10198f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48928/0 (socket says 172.21.15.5:48928) 2024-09-11T22:27:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa10198f640 1 -- 172.21.15.5:0/394670872 learned_addr learned my addr 172.21.15.5:0/394670872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa102190640 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa102991640 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7b5780 0x55f4dd7b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa10198f640 1 -- 172.21.15.5:0/394670872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7b5780 msgr2=0x55f4dd7b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa10198f640 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7b5780 0x55f4dd7b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa10198f640 1 -- 172.21.15.5:0/394670872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 msgr2=0x55f4dd7b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa10198f640 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa10198f640 1 -- 172.21.15.5:0/394670872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dd47d3c0 con 0x55f4dd7a7b40 2024-09-11T22:27:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa10198f640 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b23b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c6a0 tx=0x7fa0ec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d819adb19c80d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.457+0000 7fa0e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/394670872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec09b890 con 0x55f4dd7a7b40 2024-09-11T22:27:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa1067cd740 1 -- 172.21.15.5:0/394670872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0f4004cc0 con 0x55f4dd7a7b40 2024-09-11T22:27:58.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/394670872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fa0ec0a4730 con 0x55f4dd7a7b40 2024-09-11T22:27:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/394670872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa0ec0d1e60 con 0x55f4dd7a7b40 2024-09-11T22:27:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa1067cd740 10 client.17178 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa1067cd740 1 -- 172.21.15.5:0/394670872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f4dd7c3f10 con 0x55f4dd7a7b40 2024-09-11T22:27:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/394670872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0f4004cc0 con 0x55f4dd7a7b40 2024-09-11T22:27:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/394670872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fa0ec09b230 con 0x55f4dd7a7b40 2024-09-11T22:27:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa1067cd740 10 client.17178 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:58.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa1067cd740 10 client.17178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:27:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa1067cd740 20 client.17178 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa1067cd740 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55f4dd7c52b0 0x55f4dd7c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa1067cd740 4 client.17178 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa1067cd740 1 -- 172.21.15.5:0/394670872 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4dd719860 con 0x55f4dd7c52b0 2024-09-11T22:27:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.458+0000 7fa102190640 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55f4dd7c52b0 0x55f4dd7c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.459+0000 7fa102190640 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55f4dd7c52b0 0x55f4dd7c76a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d70bdc273b2240b server_cookie=596a2fc02f22b176 in_seq=0 out_seq=0 2024-09-11T22:27:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.459+0000 7fa0e5ffb640 10 client.17178 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:27:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.459+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/394670872 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4dd719860 con 0x55f4dd7c52b0 2024-09-11T22:27:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.459+0000 7fa0e5ffb640 10 client.17178 handle_command_reply: tid=0 2024-09-11T22:27:58.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 client.17178 shutdown 2024-09-11T22:27:58.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 -- 172.21.15.5:0/394670872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 msgr2=0x55f4dd7b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:58.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b23b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00c6a0 tx=0x7fa0ec009310 comp rx=0 tx=0).stop 2024-09-11T22:27:58.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 -- 172.21.15.5:0/394670872 shutdown_connections 2024-09-11T22:27:58.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55f4dd7c52b0 0x55f4dd7c76a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7b5780 0x55f4dd7b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b23b0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa0e5ffb640 0 client.17178 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:27:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 --2- 172.21.15.5:0/394670872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 -- 172.21.15.5:0/394670872 >> 172.21.15.5:0/394670872 conn(0x55f4dd76f310 msgr2=0x55f4dd7a8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 -- 172.21.15.5:0/394670872 shutdown_connections 2024-09-11T22:27:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 1 -- 172.21.15.5:0/394670872 wait complete. 2024-09-11T22:27:58.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.460+0000 7fa1067cd740 20 client.17178 trim_cache size 0 max 16384 2024-09-11T22:27:58.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.475+0000 7fa1067cd740 1 Processor -- start 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.475+0000 7fa1067cd740 1 -- start start 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.475+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7c1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 0x55f4dd7ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa1067cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1ab860 con 0x55f4dd7a7b40 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa1067cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1ad3a0 con 0x55f4dd7b5780 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa1067cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1abf30 con 0x55f4dd7a65d0 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa102190640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7c1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa10198f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa102190640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7c1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:38122/0 (socket says 172.21.15.5:38122) 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa10198f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48930/0 (socket says 172.21.15.5:48930) 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa102190640 1 -- 172.21.15.5:0/1761108837 learned_addr learned my addr 172.21.15.5:0/1761108837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:58.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa102991640 1 --2- 172.21.15.5:0/1761108837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 0x55f4dd7ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa10198f640 1 -- 172.21.15.5:0/1761108837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 msgr2=0x55f4dd7c1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa10198f640 1 --2- 172.21.15.5:0/1761108837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7c1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa10198f640 1 -- 172.21.15.5:0/1761108837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 msgr2=0x55f4dd7ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa10198f640 1 --2- 172.21.15.5:0/1761108837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 0x55f4dd7ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa10198f640 1 -- 172.21.15.5:0/1761108837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dd7a9120 con 0x55f4dd7a7b40 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa102190640 1 --2- 172.21.15.5:0/1761108837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7c1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa102991640 1 --2- 172.21.15.5:0/1761108837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 0x55f4dd7ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.476+0000 7fa10198f640 1 --2- 172.21.15.5:0/1761108837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b1390 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec09ba50 tx=0x7fa0ec09ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ab6bccf3b981a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.477+0000 7fa0e7fff640 1 -- 172.21.15.5:0/1761108837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec0a9070 con 0x55f4dd7a7b40 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.477+0000 7fa0e7fff640 1 -- 172.21.15.5:0/1761108837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ec09c450 con 0x55f4dd7a7b40 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.477+0000 7fa0e7fff640 1 -- 172.21.15.5:0/1761108837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec09b6b0 con 0x55f4dd7a7b40 2024-09-11T22:27:58.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.477+0000 7fa1067cd740 1 -- 172.21.15.5:0/1761108837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 msgr2=0x55f4dd7b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.477+0000 7fa1067cd740 1 --2- 172.21.15.5:0/1761108837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b1390 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec09ba50 tx=0x7fa0ec09ac10 comp rx=0 tx=0).stop 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.477+0000 7fa1067cd740 1 -- 172.21.15.5:0/1761108837 shutdown_connections 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.477+0000 7fa1067cd740 1 --2- 172.21.15.5:0/1761108837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7b5780 0x55f4dd7ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.477+0000 7fa1067cd740 1 --2- 172.21.15.5:0/1761108837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7b1390 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.477+0000 7fa1067cd740 1 --2- 172.21.15.5:0/1761108837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7c1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.477+0000 7fa1067cd740 1 -- 172.21.15.5:0/1761108837 >> 172.21.15.5:0/1761108837 conn(0x55f4dd76f310 msgr2=0x55f4dd7b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 1 -- 172.21.15.5:0/1761108837 shutdown_connections 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 1 -- 172.21.15.5:0/1761108837 wait complete. 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 1 -- start start 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 1 Processor -- start 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7b5780 0x55f4dd82ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1accd0 con 0x55f4dd7b5780 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1ac600 con 0x55f4dd7a65d0 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1a9840 con 0x55f4dd7a7b40 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa1067cd740 10 client.0 _ll_register_callbacks cb 0x7fa0ec0d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa102991640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7b5780 0x55f4dd82ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa102190640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa102190640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52732/0 (socket says 172.21.15.5:52732) 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa102190640 1 -- 172.21.15.5:0/4155572603 learned_addr learned my addr 172.21.15.5:0/4155572603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.478+0000 7fa10198f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa102190640 1 -- 172.21.15.5:0/4155572603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a7b40 msgr2=0x55f4dd7ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa102190640 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa102190640 1 -- 172.21.15.5:0/4155572603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7b5780 msgr2=0x55f4dd82ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa102190640 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7b5780 0x55f4dd82ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa102190640 1 -- 172.21.15.5:0/4155572603 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dd82f450 con 0x55f4dd7a65d0 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa10198f640 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa102991640 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7b5780 0x55f4dd82ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa102190640 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7b9c60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8006010 tx=0x7fa0f8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ea788e0577935f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa0e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/4155572603 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8035040 con 0x55f4dd7a65d0 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/4155572603 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7fa0f8005220 con 0x55f4dd7a65d0 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.479+0000 7fa1067cd740 1 -- 172.21.15.5:0/4155572603 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0ec09c450 con 0x55f4dd7a65d0 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/4155572603 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa0f8005420 con 0x55f4dd7a65d0 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa1067cd740 10 client.26600 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa1067cd740 1 -- 172.21.15.5:0/4155572603 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f4dd8306f0 con 0x55f4dd7a65d0 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/4155572603 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0ec09c450 con 0x55f4dd7a65d0 2024-09-11T22:27:58.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/4155572603 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fa0f8044070 con 0x55f4dd7a65d0 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa1067cd740 10 client.26600 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa1067cd740 10 client.26600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa1067cd740 20 client.26600 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa1067cd740 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55f4dd830e10 0x55f4dd833200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa1067cd740 4 client.26600 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa1067cd740 1 -- 172.21.15.5:0/4155572603 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4dd713d70 con 0x55f4dd830e10 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.480+0000 7fa10198f640 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55f4dd830e10 0x55f4dd833200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa10198f640 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55f4dd830e10 0x55f4dd833200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1922431ccc3f98d1 server_cookie=bab29c39dd95a376 in_seq=0 out_seq=0 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa0e5ffb640 10 client.26600 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa0e5ffb640 1 -- 172.21.15.5:0/4155572603 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4dd713d70 con 0x55f4dd830e10 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa0e5ffb640 10 client.26600 handle_command_reply: tid=0 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 client.26600 shutdown 2024-09-11T22:27:58.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 -- 172.21.15.5:0/4155572603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 msgr2=0x55f4dd7b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7b9c60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8006010 tx=0x7fa0f8009260 comp rx=0 tx=0).stop 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 -- 172.21.15.5:0/4155572603 shutdown_connections 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55f4dd830e10 0x55f4dd833200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4dd7b5780 0x55f4dd82ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4dd7a7b40 0x55f4dd7ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa0e5ffb640 0 client.26600 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 --2- 172.21.15.5:0/4155572603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4dd7a65d0 0x55f4dd7b9c60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 -- 172.21.15.5:0/4155572603 >> 172.21.15.5:0/4155572603 conn(0x55f4dd76f310 msgr2=0x55f4dd7b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 -- 172.21.15.5:0/4155572603 shutdown_connections 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 1 -- 172.21.15.5:0/4155572603 wait complete. 2024-09-11T22:27:58.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.481+0000 7fa1067cd740 20 client.26600 trim_cache size 0 max 16384 2024-09-11T22:27:58.497 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 -- 172.21.15.5:0/577186156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0e80a2cf0 msgr2=0x7fa0e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0e80a2cf0 0x7fa0e80a51b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009230 tx=0x7fa0f80058d0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 -- 172.21.15.5:0/577186156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0fc102570 msgr2=0x7fa0fc1998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0fc102570 0x7fa0fc1998c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00e9f0 tx=0x7fa0ec00edf0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 -- 172.21.15.5:0/577186156 shutdown_connections 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0fc102ef0 0x7fa0fc1a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0fc102570 0x7fa0fc1998c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0e80a2cf0 0x7fa0e80a51b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 --2- 172.21.15.5:0/577186156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa0fc100e20 0x7fa0fc199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 -- 172.21.15.5:0/577186156 >> 172.21.15.5:0/577186156 conn(0x7fa0fc0fc7d0 msgr2=0x7fa0fc0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:58.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 -- 172.21.15.5:0/577186156 shutdown_connections 2024-09-11T22:27:58.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:27:58.482+0000 7fa1043f4640 1 -- 172.21.15.5:0/577186156 wait complete. 2024-09-11T22:27:59.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 425941 8.32 MB/sec execute 137 sec latency 86.160 ms 2024-09-11T22:27:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 -- 172.21.15.5:0/1185551595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62001009b0 msgr2=0x7f6200100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 --2- 172.21.15.5:0/1185551595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62001009b0 0x7f6200100e10 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f61f4009930 tx=0x7f61f402a420 comp rx=0 tx=0).stop 2024-09-11T22:27:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 -- 172.21.15.5:0/1185551595 shutdown_connections 2024-09-11T22:27:59.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 --2- 172.21.15.5:0/1185551595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6200101350 0x7f620010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 --2- 172.21.15.5:0/1185551595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62001009b0 0x7f6200100e10 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 --2- 172.21.15.5:0/1185551595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62001069c0 0x7f6200106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 -- 172.21.15.5:0/1185551595 >> 172.21.15.5:0/1185551595 conn(0x7f6200077990 msgr2=0x7f62000fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 -- 172.21.15.5:0/1185551595 shutdown_connections 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 -- 172.21.15.5:0/1185551595 wait complete. 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 Processor -- start 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.484+0000 7f6207742640 1 -- start start 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f6207742640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62001009b0 0x7f6200197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f6207742640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6200101350 0x7f62001977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f6207742640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62001069c0 0x7f620019bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f6207742640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6200110e00 con 0x7f62001009b0 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f6207742640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6200110c80 con 0x7f62001069c0 2024-09-11T22:27:59.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f6207742640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6200110f80 con 0x7f6200101350 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f62054de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62001009b0 0x7f6200197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f62054de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62001009b0 0x7f6200197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48958/0 (socket says 172.21.15.5:48958) 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f62054de640 1 -- 172.21.15.5:0/1400147694 learned_addr learned my addr 172.21.15.5:0/1400147694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f6205cdf640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62001069c0 0x7f620019bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f6204cdd640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6200101350 0x7f62001977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f62054de640 1 -- 172.21.15.5:0/1400147694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6200101350 msgr2=0x7f62001977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f62054de640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6200101350 0x7f62001977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f62054de640 1 -- 172.21.15.5:0/1400147694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62001069c0 msgr2=0x7f620019bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f62054de640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62001069c0 0x7f620019bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f62054de640 1 -- 172.21.15.5:0/1400147694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620019c270 con 0x7f62001009b0 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f62054de640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62001009b0 0x7f6200197280 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f61f000e9f0 tx=0x7f61f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f61fe7fc640 1 -- 172.21.15.5:0/1400147694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f000cd80 con 0x7f62001009b0 2024-09-11T22:27:59.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f61fe7fc640 1 -- 172.21.15.5:0/1400147694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f0005500 con 0x7f62001009b0 2024-09-11T22:27:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f61fe7fc640 1 -- 172.21.15.5:0/1400147694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0010690 con 0x7f62001009b0 2024-09-11T22:27:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f6207742640 1 -- 172.21.15.5:0/1400147694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620019c560 con 0x7f62001009b0 2024-09-11T22:27:59.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.485+0000 7f6207742640 1 -- 172.21.15.5:0/1400147694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62001a3df0 con 0x7f62001009b0 2024-09-11T22:27:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f61fe7fc640 1 -- 172.21.15.5:0/1400147694 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f61f00046c0 con 0x7f62001009b0 2024-09-11T22:27:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f620a4da740 1 Processor -- start 2024-09-11T22:27:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f620a4da740 1 -- start start 2024-09-11T22:27:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f620a4da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a335c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f620a4da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c6410 0x558a335d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f620a4da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335d0cb0 0x558a335d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f620a4da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558a3352a2f0 con 0x558a335c5af0 2024-09-11T22:27:59.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f620a4da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558a334d2270 con 0x558a335c6410 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f620a4da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558a335262b0 con 0x558a335d0cb0 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f62054de640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c6410 0x558a335d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f62054de640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c6410 0x558a335d0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52776/0 (socket says 172.21.15.5:52776) 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f62054de640 1 -- 172.21.15.5:0/1467286773 learned_addr learned my addr 172.21.15.5:0/1467286773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.489+0000 7f6205cdf640 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a335c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f6204cdd640 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335d0cb0 0x558a335d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f62054de640 1 -- 172.21.15.5:0/1467286773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335d0cb0 msgr2=0x558a335d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f62054de640 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335d0cb0 0x558a335d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f62054de640 1 -- 172.21.15.5:0/1467286773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 msgr2=0x558a335c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f62054de640 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a335c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f62054de640 1 -- 172.21.15.5:0/1467286773 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a334d1340 con 0x558a335c6410 2024-09-11T22:27:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f6205cdf640 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a335c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f6204cdd640 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335d0cb0 0x558a335d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f62054de640 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c6410 0x558a335d0770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f61f00cceb0 tx=0x7f61f00d5b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b76658e8631a483 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f61fe7fc640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61e80a2e20 0x7f61e80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f6205cdf640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61e80a2e20 0x7f61e80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f61e77fe640 1 -- 172.21.15.5:0/1467286773 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f00de070 con 0x558a335c6410 2024-09-11T22:27:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f61fe7fc640 1 -- 172.21.15.5:0/1400147694 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f61f00cf830 con 0x7f62001009b0 2024-09-11T22:27:59.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f61e77fe640 1 -- 172.21.15.5:0/1467286773 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f000df60 con 0x558a335c6410 2024-09-11T22:27:59.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.490+0000 7f61e77fe640 1 -- 172.21.15.5:0/1467286773 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f009b850 con 0x558a335c6410 2024-09-11T22:27:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.491+0000 7f620a4da740 1 -- 172.21.15.5:0/1467286773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c6410 msgr2=0x558a335d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.491+0000 7f6205cdf640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61e80a2e20 0x7f61e80a52e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61f8005fd0 tx=0x7f61f8009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.491+0000 7f620a4da740 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c6410 0x558a335d0770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f61f00cceb0 tx=0x7f61f00d5b40 comp rx=0 tx=0).stop 2024-09-11T22:27:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.491+0000 7f620a4da740 1 -- 172.21.15.5:0/1467286773 shutdown_connections 2024-09-11T22:27:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.491+0000 7f620a4da740 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335d0cb0 0x558a335d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.491+0000 7f620a4da740 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c6410 0x558a335d0770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.491+0000 7f620a4da740 1 --2- 172.21.15.5:0/1467286773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a335c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.491+0000 7f620a4da740 1 -- 172.21.15.5:0/1467286773 >> 172.21.15.5:0/1467286773 conn(0x558a335c2060 msgr2=0x558a335c2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.491+0000 7f620a4da740 1 -- 172.21.15.5:0/1467286773 shutdown_connections 2024-09-11T22:27:59.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.491+0000 7f620a4da740 1 -- 172.21.15.5:0/1467286773 wait complete. 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.492+0000 7f620a4da740 1 -- start start 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.492+0000 7f620a4da740 1 Processor -- start 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.492+0000 7f620a4da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c5af0 0x558a335cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.492+0000 7f620a4da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 0x558a335ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.492+0000 7f620a4da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335d0cb0 0x558a335ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.492+0000 7f620a4da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558a335d5a90 con 0x558a335d0cb0 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.492+0000 7f620a4da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558a334b3290 con 0x558a335c5af0 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.492+0000 7f620a4da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558a335d5c10 con 0x558a335c6410 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.492+0000 7f620a4da740 10 client.0 _ll_register_callbacks cb 0x558a334e4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.492+0000 7f6205cdf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335d0cb0 0x558a335ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f62054de640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c5af0 0x558a335cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f62054de640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c5af0 0x558a335cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52778/0 (socket says 172.21.15.5:52778) 2024-09-11T22:27:59.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f62054de640 1 -- 172.21.15.5:0/2907617575 learned_addr learned my addr 172.21.15.5:0/2907617575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f6204cdd640 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 0x558a335ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f62054de640 1 -- 172.21.15.5:0/2907617575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 msgr2=0x558a335ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f62054de640 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 0x558a335ce0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f62054de640 1 -- 172.21.15.5:0/2907617575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335d0cb0 msgr2=0x558a335ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f62054de640 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335d0cb0 0x558a335ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f62054de640 1 -- 172.21.15.5:0/2907617575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a3323d210 con 0x558a335c5af0 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f62054de640 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c5af0 0x558a335cdb60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f61f00d51b0 tx=0x7f61f0009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52587c6a153f8fb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f61e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f61e4ff9640 1 -- 172.21.15.5:0/2907617575 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f00de030 con 0x558a335c5af0 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f61e4ff9640 1 -- 172.21.15.5:0/2907617575 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f61f0098380 con 0x558a335c5af0 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.493+0000 7f620a4da740 1 -- 172.21.15.5:0/2907617575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61f000df60 con 0x558a335c5af0 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.494+0000 7f61e4ff9640 1 -- 172.21.15.5:0/2907617575 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f61f0022020 con 0x558a335c5af0 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.494+0000 7f620a4da740 10 client.26612 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:59.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.494+0000 7f620a4da740 1 -- 172.21.15.5:0/2907617575 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558a33309620 con 0x558a335c5af0 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.494+0000 7f61e4ff9640 1 -- 172.21.15.5:0/2907617575 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f009c410 con 0x558a335c5af0 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.494+0000 7f61e4ff9640 1 -- 172.21.15.5:0/2907617575 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f61f00cfce0 con 0x558a335c5af0 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.494+0000 7f620a4da740 10 client.26612 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.495+0000 7f620a4da740 10 client.26612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.495+0000 7f620a4da740 20 client.26612 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.495+0000 7f620a4da740 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558a335e2c60 0x558a335e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.495+0000 7f620a4da740 4 client.26612 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.495+0000 7f620a4da740 1 -- 172.21.15.5:0/2907617575 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f61f000df60 con 0x558a335e2c60 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.495+0000 7f6204cdd640 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558a335e2c60 0x558a335e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.496+0000 7f6204cdd640 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558a335e2c60 0x558a335e5050 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7727985bb46ae110 server_cookie=b382b2cc454919f0 in_seq=0 out_seq=0 2024-09-11T22:27:59.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.496+0000 7f61e4ff9640 10 client.26612 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f61e4ff9640 1 -- 172.21.15.5:0/2907617575 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f61f000df60 con 0x558a335e2c60 2024-09-11T22:27:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f61e4ff9640 10 client.26612 handle_command_reply: tid=0 2024-09-11T22:27:59.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 client.26612 shutdown 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 -- 172.21.15.5:0/2907617575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c5af0 msgr2=0x558a335cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c5af0 0x558a335cdb60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f61f00d51b0 tx=0x7f61f0009720 comp rx=0 tx=0).stop 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 -- 172.21.15.5:0/2907617575 shutdown_connections 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558a335e2c60 0x558a335e5050 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335d0cb0 0x558a335ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 0x558a335ce0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 --2- 172.21.15.5:0/2907617575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335c5af0 0x558a335cdb60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 -- 172.21.15.5:0/2907617575 >> 172.21.15.5:0/2907617575 conn(0x558a335c2060 msgr2=0x558a335c7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f61e4ff9640 0 client.26612 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 -- 172.21.15.5:0/2907617575 shutdown_connections 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 1 -- 172.21.15.5:0/2907617575 wait complete. 2024-09-11T22:27:59.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.497+0000 7f620a4da740 20 client.26612 trim_cache size 0 max 16384 2024-09-11T22:27:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.516+0000 7f620a4da740 1 Processor -- start 2024-09-11T22:27:59.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f620a4da740 1 -- start start 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f620a4da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c5af0 0x558a335c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f620a4da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c6410 0x558a335cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f620a4da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 0x558a335e8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f620a4da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62001ab740 con 0x558a335c6410 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f620a4da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62001ad280 con 0x558a335d0cb0 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f620a4da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62001abe10 con 0x558a335c5af0 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6204cdd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c6410 0x558a335cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6204cdd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c6410 0x558a335cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48996/0 (socket says 172.21.15.5:48996) 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6204cdd640 1 -- 172.21.15.5:0/1048253423 learned_addr learned my addr 172.21.15.5:0/1048253423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6205cdf640 1 --2- 172.21.15.5:0/1048253423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 0x558a335e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f62054de640 1 --2- 172.21.15.5:0/1048253423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c5af0 0x558a335c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6204cdd640 1 -- 172.21.15.5:0/1048253423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c5af0 msgr2=0x558a335c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:59.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6204cdd640 1 --2- 172.21.15.5:0/1048253423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c5af0 0x558a335c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6204cdd640 1 -- 172.21.15.5:0/1048253423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 msgr2=0x558a335e8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6204cdd640 1 --2- 172.21.15.5:0/1048253423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 0x558a335e8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6204cdd640 1 -- 172.21.15.5:0/1048253423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a335c1e90 con 0x558a335c6410 2024-09-11T22:27:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6205cdf640 1 --2- 172.21.15.5:0/1048253423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 0x558a335e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:27:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.517+0000 7f6204cdd640 1 --2- 172.21.15.5:0/1048253423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c6410 0x558a335cfad0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f61f4005940 tx=0x7f61f4032f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb3cccc01c8575c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f61e6ffd640 1 -- 172.21.15.5:0/1048253423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4035040 con 0x558a335c6410 2024-09-11T22:27:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f61e6ffd640 1 -- 172.21.15.5:0/1048253423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f4048070 con 0x558a335c6410 2024-09-11T22:27:59.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f61e6ffd640 1 -- 172.21.15.5:0/1048253423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f4004920 con 0x558a335c6410 2024-09-11T22:27:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f620a4da740 1 -- 172.21.15.5:0/1048253423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c6410 msgr2=0x558a335cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f620a4da740 1 --2- 172.21.15.5:0/1048253423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c6410 0x558a335cfad0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f61f4005940 tx=0x7f61f4032f50 comp rx=0 tx=0).stop 2024-09-11T22:27:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f620a4da740 1 -- 172.21.15.5:0/1048253423 shutdown_connections 2024-09-11T22:27:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f620a4da740 1 --2- 172.21.15.5:0/1048253423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 0x558a335e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f620a4da740 1 --2- 172.21.15.5:0/1048253423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c6410 0x558a335cfad0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f620a4da740 1 --2- 172.21.15.5:0/1048253423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c5af0 0x558a335c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f620a4da740 1 -- 172.21.15.5:0/1048253423 >> 172.21.15.5:0/1048253423 conn(0x558a335c2060 msgr2=0x558a335cf3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f620a4da740 1 -- 172.21.15.5:0/1048253423 shutdown_connections 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.518+0000 7f620a4da740 1 -- 172.21.15.5:0/1048253423 wait complete. 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f620a4da740 1 -- start start 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f620a4da740 1 Processor -- start 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f620a4da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a33647d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f620a4da740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 0x558a33648260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f620a4da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 0x558a335d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f620a4da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62001acbb0 con 0x558a335c5af0 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f620a4da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62001ac4e0 con 0x558a335d0cb0 2024-09-11T22:27:59.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f620a4da740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62001a9720 con 0x558a335c6410 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f620a4da740 10 client.0 _ll_register_callbacks cb 0x7f61f00d5900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f62054de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a33647d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f62054de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a33647d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49000/0 (socket says 172.21.15.5:49000) 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f62054de640 1 -- 172.21.15.5:0/4019892325 learned_addr learned my addr 172.21.15.5:0/4019892325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f6204cdd640 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 0x558a33648260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.519+0000 7f6205cdf640 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 0x558a335d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f62054de640 1 -- 172.21.15.5:0/4019892325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 msgr2=0x558a33648260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f62054de640 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 0x558a33648260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f62054de640 1 -- 172.21.15.5:0/4019892325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 msgr2=0x558a335d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f62054de640 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 0x558a335d4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f62054de640 1 -- 172.21.15.5:0/4019892325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a335d4c30 con 0x558a335c5af0 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f6204cdd640 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 0x558a33648260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:59.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f6205cdf640 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 0x558a335d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f62054de640 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a33647d20 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f61f000e9c0 tx=0x7f61f00d5550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cae0b7c5a30668f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f61e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f61e4ff9640 1 -- 172.21.15.5:0/4019892325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f00de070 con 0x558a335c5af0 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f61e4ff9640 1 -- 172.21.15.5:0/4019892325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f61f00a44f0 con 0x558a335c5af0 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f620a4da740 1 -- 172.21.15.5:0/4019892325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61f4048070 con 0x558a335c5af0 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.520+0000 7f61e4ff9640 1 -- 172.21.15.5:0/4019892325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f61f00035f0 con 0x558a335c5af0 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f620a4da740 10 client.17202 fetch_fsmap learned FSMap version 34 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f620a4da740 1 -- 172.21.15.5:0/4019892325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558a3364f590 con 0x558a335c5af0 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f61e4ff9640 1 -- 172.21.15.5:0/4019892325 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61f4048070 con 0x558a335c5af0 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f61e4ff9640 1 -- 172.21.15.5:0/4019892325 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f61f00a4bd0 con 0x558a335c5af0 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f620a4da740 10 client.17202 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f620a4da740 10 client.17202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:27:59.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f620a4da740 20 client.17202 populate_metadata read hostname 'smithi005' 2024-09-11T22:27:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f620a4da740 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558a3364faa0 0x558a33651ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:27:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f620a4da740 4 client.17202 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:27:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f620a4da740 1 -- 172.21.15.5:0/4019892325 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a33537a20 con 0x558a3364faa0 2024-09-11T22:27:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.521+0000 7f6204cdd640 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558a3364faa0 0x558a33651ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:27:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.522+0000 7f6204cdd640 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558a3364faa0 0x558a33651ec0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c01d253e7291cd8 server_cookie=51bd81c19a2ee0e0 in_seq=0 out_seq=0 2024-09-11T22:27:59.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.522+0000 7f61e4ff9640 10 client.17202 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:59.548 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:27:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.728+0000 7f61e4ff9640 1 -- 172.21.15.5:0/4019892325 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a33537a20 con 0x558a3364faa0 2024-09-11T22:27:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.728+0000 7f61e4ff9640 10 client.17202 handle_command_reply: tid=0 2024-09-11T22:27:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.728+0000 7f620a4da740 1 client.17202 shutdown 2024-09-11T22:27:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.728+0000 7f620a4da740 1 -- 172.21.15.5:0/4019892325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 msgr2=0x558a33647d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.728+0000 7f620a4da740 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a33647d20 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f61f000e9c0 tx=0x7f61f00d5550 comp rx=0 tx=0).stop 2024-09-11T22:27:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.728+0000 7f620a4da740 1 -- 172.21.15.5:0/4019892325 shutdown_connections 2024-09-11T22:27:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.728+0000 7f620a4da740 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558a3364faa0 0x558a33651ec0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.728+0000 7f620a4da740 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a335d0cb0 0x558a335d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.729+0000 7f620a4da740 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558a335c6410 0x558a33648260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.729+0000 7f61e4ff9640 0 client.17202 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:27:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.729+0000 7f620a4da740 1 --2- 172.21.15.5:0/4019892325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a335c5af0 0x558a33647d20 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.729+0000 7f620a4da740 1 -- 172.21.15.5:0/4019892325 >> 172.21.15.5:0/4019892325 conn(0x558a335c2060 msgr2=0x558a335cbfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.729+0000 7f620a4da740 1 -- 172.21.15.5:0/4019892325 shutdown_connections 2024-09-11T22:27:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.729+0000 7f620a4da740 1 -- 172.21.15.5:0/4019892325 wait complete. 2024-09-11T22:27:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.729+0000 7f620a4da740 20 client.17202 trim_cache size 0 max 16384 2024-09-11T22:27:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:27:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:27:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:27:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:27:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.731+0000 7f6207742640 1 -- 172.21.15.5:0/1400147694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61e80a2e20 msgr2=0x7f61e80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.731+0000 7f6207742640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61e80a2e20 0x7f61e80a52e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f61f8005fd0 tx=0x7f61f8009500 comp rx=0 tx=0).stop 2024-09-11T22:27:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.731+0000 7f6207742640 1 -- 172.21.15.5:0/1400147694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62001009b0 msgr2=0x7f6200197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:27:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.731+0000 7f6207742640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62001009b0 0x7f6200197280 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f61f000e9f0 tx=0x7f61f000edf0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.731+0000 7f6207742640 1 -- 172.21.15.5:0/1400147694 shutdown_connections 2024-09-11T22:27:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.731+0000 7f6207742640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61e80a2e20 0x7f61e80a52e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.731+0000 7f6207742640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62001069c0 0x7f620019bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.731+0000 7f6207742640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6200101350 0x7f62001977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.731+0000 7f6207742640 1 --2- 172.21.15.5:0/1400147694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62001009b0 0x7f6200197280 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:27:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.731+0000 7f6207742640 1 -- 172.21.15.5:0/1400147694 >> 172.21.15.5:0/1400147694 conn(0x7f6200077990 msgr2=0x7f6200103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:27:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.732+0000 7f6207742640 1 -- 172.21.15.5:0/1400147694 shutdown_connections 2024-09-11T22:27:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:27:59.732+0000 7f6207742640 1 -- 172.21.15.5:0/1400147694 wait complete. 2024-09-11T22:27:59.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:27:59 smithi005 ceph-mon[30943]: pgmap v177: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-09-11T22:27:59.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:27:59 smithi159 ceph-mon[35534]: pgmap v177: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-09-11T22:27:59.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:27:59 smithi172 ceph-mon[35461]: pgmap v177: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 396 op/s 2024-09-11T22:27:59.905 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:00.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 427534 8.30 MB/sec execute 138 sec latency 54.058 ms 2024-09-11T22:28:00.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.444+0000 7f1b981a3640 1 -- 172.21.15.5:0/3649659925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b90103660 msgr2=0x7f1b9010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:00.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.444+0000 7f1b981a3640 1 --2- 172.21.15.5:0/3649659925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b90103660 0x7f1b9010bb50 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1b80009900 tx=0x7f1b8002a480 comp rx=0 tx=0).stop 2024-09-11T22:28:00.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.444+0000 7f1b981a3640 1 -- 172.21.15.5:0/3649659925 shutdown_connections 2024-09-11T22:28:00.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.444+0000 7f1b981a3640 1 --2- 172.21.15.5:0/3649659925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9010c170 0x7f1b9010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.444+0000 7f1b981a3640 1 --2- 172.21.15.5:0/3649659925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b90103660 0x7f1b9010bb50 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.444+0000 7f1b981a3640 1 --2- 172.21.15.5:0/3649659925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b90102d40 0x7f1b90103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.445+0000 7f1b981a3640 1 -- 172.21.15.5:0/3649659925 >> 172.21.15.5:0/3649659925 conn(0x7f1b900fc7a0 msgr2=0x7f1b900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.445+0000 7f1b981a3640 1 -- 172.21.15.5:0/3649659925 shutdown_connections 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.445+0000 7f1b981a3640 1 -- 172.21.15.5:0/3649659925 wait complete. 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.445+0000 7f1b981a3640 1 Processor -- start 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b981a3640 1 -- start start 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b981a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b90102d40 0x7f1b9019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b981a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b90103660 0x7f1b9019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b981a3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b9010c170 0x7f1b9019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b981a3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90111f70 con 0x7f1b90103660 2024-09-11T22:28:00.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b981a3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90111df0 con 0x7f1b9010c170 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b981a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901120f0 con 0x7f1b90102d40 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b9573e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b90103660 0x7f1b9019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b9573e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b90103660 0x7f1b9019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49020/0 (socket says 172.21.15.5:49020) 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b95f3f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b90102d40 0x7f1b9019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b9573e640 1 -- 172.21.15.5:0/3097188214 learned_addr learned my addr 172.21.15.5:0/3097188214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.446+0000 7f1b95f3f640 1 -- 172.21.15.5:0/3097188214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b9010c170 msgr2=0x7f1b9019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.447+0000 7f1b96740640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b9010c170 0x7f1b9019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.447+0000 7f1b95f3f640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b9010c170 0x7f1b9019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.447+0000 7f1b95f3f640 1 -- 172.21.15.5:0/3097188214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b90103660 msgr2=0x7f1b9019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.447+0000 7f1b95f3f640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b90103660 0x7f1b9019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.447+0000 7f1b95f3f640 1 -- 172.21.15.5:0/3097188214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b9019f620 con 0x7f1b90102d40 2024-09-11T22:28:00.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.447+0000 7f1b9573e640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b90103660 0x7f1b9019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:00.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.447+0000 7f1b96740640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b9010c170 0x7f1b9019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:28:00.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.447+0000 7f1b95f3f640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b90102d40 0x7f1b9019a670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c380 tx=0x7f1b8c00c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:00.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.447+0000 7f1b86ffd640 1 -- 172.21.15.5:0/3097188214 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c023070 con 0x7f1b90102d40 2024-09-11T22:28:00.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.448+0000 7f1b86ffd640 1 -- 172.21.15.5:0/3097188214 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b8c015e20 con 0x7f1b90102d40 2024-09-11T22:28:00.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.448+0000 7f1b981a3640 1 -- 172.21.15.5:0/3097188214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b9019f910 con 0x7f1b90102d40 2024-09-11T22:28:00.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.448+0000 7f1b86ffd640 1 -- 172.21.15.5:0/3097188214 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c016830 con 0x7f1b90102d40 2024-09-11T22:28:00.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.448+0000 7f1b981a3640 1 -- 172.21.15.5:0/3097188214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b90103ac0 con 0x7f1b90102d40 2024-09-11T22:28:00.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.452+0000 7f1b9a57c740 1 Processor -- start 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.452+0000 7f1b86ffd640 1 -- 172.21.15.5:0/3097188214 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1b8c026070 con 0x7f1b90102d40 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.452+0000 7f1b9a57c740 1 -- start start 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.452+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 0x5560ffada090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.452+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 0x5560ffadaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.452+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffae7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.452+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5560ffa402f0 con 0x5560ffae5370 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.452+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5560ff9e7cf0 con 0x5560ffada5d0 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.452+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5560ffa3c2b0 con 0x5560ffadbb40 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.452+0000 7f1b96740640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 0x5560ffada090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.453+0000 7f1b96740640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 0x5560ffada090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:38186/0 (socket says 172.21.15.5:38186) 2024-09-11T22:28:00.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.453+0000 7f1b96740640 1 -- 172.21.15.5:0/1573588572 learned_addr learned my addr 172.21.15.5:0/1573588572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.453+0000 7f1b9573e640 1 --2- 172.21.15.5:0/1573588572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffae7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.453+0000 7f1b96740640 1 -- 172.21.15.5:0/1573588572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 msgr2=0x5560ffadaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.453+0000 7f1b96740640 1 --2- 172.21.15.5:0/1573588572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 0x5560ffadaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.453+0000 7f1b96740640 1 -- 172.21.15.5:0/1573588572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 msgr2=0x5560ffae7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.453+0000 7f1b96740640 1 --2- 172.21.15.5:0/1573588572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffae7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.453+0000 7f1b96740640 1 -- 172.21.15.5:0/1573588572 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560ff9cc3a0 con 0x5560ffadbb40 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.453+0000 7f1b9573e640 1 --2- 172.21.15.5:0/1573588572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffae7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.454+0000 7f1b86ffd640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b7c0a2d70 0x7f1b7c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.454+0000 7f1b95f3f640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b7c0a2d70 0x7f1b7c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.454+0000 7f1b96740640 1 --2- 172.21.15.5:0/1573588572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 0x5560ffada090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009ee0 tx=0x7f1b8800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=441f29dba36eb86c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.454+0000 7f1b86ffd640 1 -- 172.21.15.5:0/3097188214 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f1b8c0d7540 con 0x7f1b90102d40 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.454+0000 7f1b7bfff640 1 -- 172.21.15.5:0/1573588572 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88011640 con 0x5560ffadbb40 2024-09-11T22:28:00.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.454+0000 7f1b7bfff640 1 -- 172.21.15.5:0/1573588572 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b88011cc0 con 0x5560ffadbb40 2024-09-11T22:28:00.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.454+0000 7f1b7bfff640 1 -- 172.21.15.5:0/1573588572 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8801be40 con 0x5560ffadbb40 2024-09-11T22:28:00.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.455+0000 7f1b95f3f640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b7c0a2d70 0x7f1b7c0a5230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x5560ffad7ac0 tx=0x7f1b8c075a90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:00.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.455+0000 7f1b9a57c740 1 -- 172.21.15.5:0/1573588572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 msgr2=0x5560ffada090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:00.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.455+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/1573588572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 0x5560ffada090 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009ee0 tx=0x7f1b8800f730 comp rx=0 tx=0).stop 2024-09-11T22:28:00.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.456+0000 7f1b9a57c740 1 -- 172.21.15.5:0/1573588572 shutdown_connections 2024-09-11T22:28:00.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.456+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/1573588572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffae7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.456+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/1573588572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 0x5560ffadaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.456+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/1573588572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 0x5560ffada090 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.456+0000 7f1b9a57c740 1 -- 172.21.15.5:0/1573588572 >> 172.21.15.5:0/1573588572 conn(0x5560ffaa3310 msgr2=0x5560ffad6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:00.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.456+0000 7f1b9a57c740 1 -- 172.21.15.5:0/1573588572 shutdown_connections 2024-09-11T22:28:00.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.456+0000 7f1b9a57c740 1 -- 172.21.15.5:0/1573588572 wait complete. 2024-09-11T22:28:00.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.456+0000 7f1b9a57c740 1 -- start start 2024-09-11T22:28:00.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.457+0000 7f1b9a57c740 1 Processor -- start 2024-09-11T22:28:00.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.457+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffada5d0 0x5560ffae4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffadbb40 0x5560ffae1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffae5370 0x5560ffae2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5560ffae8110 con 0x5560ffadbb40 2024-09-11T22:28:00.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5560ff9fb060 con 0x5560ffae5370 2024-09-11T22:28:00.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5560ffae8290 con 0x5560ffada5d0 2024-09-11T22:28:00.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9a57c740 10 client.0 _ll_register_callbacks cb 0x5560ff74e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:00.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9573e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffadbb40 0x5560ffae1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9573e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffadbb40 0x5560ffae1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49040/0 (socket says 172.21.15.5:49040) 2024-09-11T22:28:00.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9573e640 1 -- 172.21.15.5:0/2429226886 learned_addr learned my addr 172.21.15.5:0/2429226886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:00.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b96740640 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffada5d0 0x5560ffae4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9573e640 1 -- 172.21.15.5:0/2429226886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffada5d0 msgr2=0x5560ffae4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:00.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9573e640 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffada5d0 0x5560ffae4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9573e640 1 -- 172.21.15.5:0/2429226886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffae5370 msgr2=0x5560ffae2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9573e640 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffae5370 0x5560ffae2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9573e640 1 -- 172.21.15.5:0/2429226886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560ff7b13c0 con 0x5560ffadbb40 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b9573e640 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffadbb40 0x5560ffae1fd0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000c00 tx=0x7f1b80031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c8fb439aadec0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.458+0000 7f1b797fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.459+0000 7f1b797fa640 1 -- 172.21.15.5:0/2429226886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80035040 con 0x5560ffadbb40 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.459+0000 7f1b797fa640 1 -- 172.21.15.5:0/2429226886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f1b80004240 con 0x5560ffadbb40 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.459+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2429226886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b88011cc0 con 0x5560ffadbb40 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.459+0000 7f1b797fa640 1 -- 172.21.15.5:0/2429226886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1b80005d50 con 0x5560ffadbb40 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.459+0000 7f1b9a57c740 10 client.17220 fetch_fsmap learned FSMap version 34 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.459+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2429226886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5560ffaf3b60 con 0x5560ffadbb40 2024-09-11T22:28:00.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.460+0000 7f1b797fa640 1 -- 172.21.15.5:0/2429226886 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b80004440 con 0x5560ffadbb40 2024-09-11T22:28:00.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.460+0000 7f1b797fa640 1 -- 172.21.15.5:0/2429226886 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f1b800035f0 con 0x5560ffadbb40 2024-09-11T22:28:00.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.460+0000 7f1b9a57c740 10 client.17220 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:28:00.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.460+0000 7f1b9a57c740 10 client.17220 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:28:00.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.460+0000 7f1b9a57c740 20 client.17220 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:00.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.460+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5560ffaf4fc0 0x5560ffaf73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.460+0000 7f1b9a57c740 4 client.17220 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:00.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.460+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2429226886 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1b88011cc0 con 0x5560ffaf4fc0 2024-09-11T22:28:00.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.460+0000 7f1b96740640 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5560ffaf4fc0 0x5560ffaf73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.461+0000 7f1b96740640 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5560ffaf4fc0 0x5560ffaf73e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f5aeeb0186eb8ee server_cookie=e5b451642bc970a9 in_seq=0 out_seq=0 2024-09-11T22:28:00.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.461+0000 7f1b797fa640 10 client.17220 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:28:00.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b797fa640 1 -- 172.21.15.5:0/2429226886 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1b88011cc0 con 0x5560ffaf4fc0 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b797fa640 10 client.17220 handle_command_reply: tid=0 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b9a57c740 1 client.17220 shutdown 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2429226886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffadbb40 msgr2=0x5560ffae1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffadbb40 0x5560ffae1fd0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000c00 tx=0x7f1b80031860 comp rx=0 tx=0).stop 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2429226886 shutdown_connections 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5560ffaf4fc0 0x5560ffaf73e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b797fa640 0 client.17220 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffae5370 0x5560ffae2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffadbb40 0x5560ffae1fd0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/2429226886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffada5d0 0x5560ffae4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2429226886 >> 172.21.15.5:0/2429226886 conn(0x5560ffaa3310 msgr2=0x5560ffadc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.462+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2429226886 shutdown_connections 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.463+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2429226886 wait complete. 2024-09-11T22:28:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.463+0000 7f1b9a57c740 20 client.17220 trim_cache size 0 max 16384 2024-09-11T22:28:00.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.483+0000 7f1b9a57c740 1 Processor -- start 2024-09-11T22:28:00.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.483+0000 7f1b9a57c740 1 -- start start 2024-09-11T22:28:00.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.483+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffada5d0 0x5560ffaf13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.483+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffadbb40 0x5560ffadd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.483+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffadd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.483+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901ab970 con 0x5560ffae5370 2024-09-11T22:28:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.483+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901ad4b0 con 0x5560ffadbb40 2024-09-11T22:28:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.483+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901ac040 con 0x5560ffada5d0 2024-09-11T22:28:00.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b9573e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffadbb40 0x5560ffadd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b96740640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffadd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b9573e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffadbb40 0x5560ffadd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52868/0 (socket says 172.21.15.5:52868) 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b9573e640 1 -- 172.21.15.5:0/2845016478 learned_addr learned my addr 172.21.15.5:0/2845016478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b96740640 1 --2- 172.21.15.5:0/2845016478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffada5d0 0x5560ffaf13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b96740640 1 -- 172.21.15.5:0/2845016478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffada5d0 msgr2=0x5560ffaf13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b96740640 1 --2- 172.21.15.5:0/2845016478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffada5d0 0x5560ffaf13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b96740640 1 -- 172.21.15.5:0/2845016478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffadbb40 msgr2=0x5560ffadd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b96740640 1 --2- 172.21.15.5:0/2845016478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffadbb40 0x5560ffadd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b96740640 1 -- 172.21.15.5:0/2845016478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560ffae5090 con 0x5560ffae5370 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b9573e640 1 --2- 172.21.15.5:0/2845016478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffadbb40 0x5560ffadd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.484+0000 7f1b96740640 1 --2- 172.21.15.5:0/2845016478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffadd660 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009d20 tx=0x7f1b8801b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e205aa763bfa420 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:00.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.485+0000 7f1b7b7fe640 1 -- 172.21.15.5:0/2845016478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8801bc50 con 0x5560ffae5370 2024-09-11T22:28:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.485+0000 7f1b7b7fe640 1 -- 172.21.15.5:0/2845016478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b880092a0 con 0x5560ffae5370 2024-09-11T22:28:00.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.485+0000 7f1b7b7fe640 1 -- 172.21.15.5:0/2845016478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88005040 con 0x5560ffae5370 2024-09-11T22:28:00.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.485+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2845016478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 msgr2=0x5560ffadd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:00.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.485+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/2845016478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffadd660 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009d20 tx=0x7f1b8801b940 comp rx=0 tx=0).stop 2024-09-11T22:28:00.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.486+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2845016478 shutdown_connections 2024-09-11T22:28:00.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.486+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/2845016478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffadd660 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.486+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/2845016478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffadbb40 0x5560ffadd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.486+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/2845016478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffada5d0 0x5560ffaf13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.486+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2845016478 >> 172.21.15.5:0/2845016478 conn(0x5560ffaa3310 msgr2=0x5560ffae10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.486+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2845016478 shutdown_connections 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.486+0000 7f1b9a57c740 1 -- 172.21.15.5:0/2845016478 wait complete. 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.486+0000 7f1b9a57c740 1 -- start start 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.486+0000 7f1b9a57c740 1 Processor -- start 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 0x5560ffae9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 0x5560ffae99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b9a57c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffae9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b96740640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 0x5560ffae9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901acde0 con 0x5560ffae5370 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b9a57c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901ac710 con 0x5560ffada5d0 2024-09-11T22:28:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b96740640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 0x5560ffae9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52878/0 (socket says 172.21.15.5:52878) 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b96740640 1 -- 172.21.15.5:0/826031625 learned_addr learned my addr 172.21.15.5:0/826031625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b9a57c740 1 -- 172.21.15.5:0/826031625 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901a9990 con 0x5560ffadbb40 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b96740640 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffae9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b9573e640 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 0x5560ffae99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b9a57c740 10 client.0 _ll_register_callbacks cb 0x7f1b8c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b96740640 1 -- 172.21.15.5:0/826031625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 msgr2=0x5560ffae99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b96740640 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 0x5560ffae99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b96740640 1 -- 172.21.15.5:0/826031625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 msgr2=0x5560ffae9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b96740640 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffae9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.487+0000 7f1b96740640 1 -- 172.21.15.5:0/826031625 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560ffb5ee40 con 0x5560ffada5d0 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.488+0000 7f1b9a57c740 1 -- 172.21.15.5:0/826031625 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b880092a0 con 0x5560ffada5d0 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.488+0000 7f1b96740640 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 0x5560ffae9460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009900 tx=0x7f1b880299f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e298795b88a6f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.488+0000 7f1b797fa640 10 client.26633 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:28:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.488+0000 7f1b797fa640 1 -- 172.21.15.5:0/826031625 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8801b9b0 con 0x5560ffada5d0 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.488+0000 7f1b797fa640 1 -- 172.21.15.5:0/826031625 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8585+0+0 (secure 0 0 0) 0x7f1b8801bb50 con 0x5560ffada5d0 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.488+0000 7f1b797fa640 1 -- 172.21.15.5:0/826031625 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1b8801bd90 con 0x5560ffada5d0 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.488+0000 7f1b9a57c740 10 client.26633 fetch_fsmap learned FSMap version 34 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.488+0000 7f1b9a57c740 1 -- 172.21.15.5:0/826031625 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=34}) -- 0x5560ffb5ffe0 con 0x5560ffada5d0 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.489+0000 7f1b797fa640 1 -- 172.21.15.5:0/826031625 <== mon.2 v2:172.21.15.159:3300/0 4 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7f1b88005040 con 0x5560ffada5d0 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.489+0000 7f1b9a57c740 10 client.26633 fetch_fsmap finished waiting for FSMap version 34 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.489+0000 7f1b9a57c740 10 client.26633 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.489+0000 7f1b9a57c740 20 client.26633 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.489+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5560ffaf4fc0 0x5560ffb62370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.489+0000 7f1b9a57c740 4 client.26633 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.489+0000 7f1b9a57c740 1 -- 172.21.15.5:0/826031625 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560ffa4d860 con 0x5560ffaf4fc0 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.489+0000 7f1b9573e640 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5560ffaf4fc0 0x5560ffb62370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.489+0000 7f1b9573e640 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5560ffaf4fc0 0x5560ffb62370 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4924c6f47671f1cd server_cookie=8c873f755493d428 in_seq=0 out_seq=0 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.489+0000 7f1b797fa640 10 client.26633 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b797fa640 1 -- 172.21.15.5:0/826031625 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560ffa4d860 con 0x5560ffaf4fc0 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b797fa640 10 client.26633 handle_command_reply: tid=0 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 client.26633 shutdown 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 -- 172.21.15.5:0/826031625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 msgr2=0x5560ffae9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 0x5560ffae9460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009900 tx=0x7f1b880299f0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 -- 172.21.15.5:0/826031625 shutdown_connections 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5560ffaf4fc0 0x5560ffb62370 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560ffae5370 0x5560ffae9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5560ffadbb40 0x5560ffae99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 --2- 172.21.15.5:0/826031625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560ffada5d0 0x5560ffae9460 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 -- 172.21.15.5:0/826031625 >> 172.21.15.5:0/826031625 conn(0x5560ffaa3310 msgr2=0x5560ffad54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b797fa640 0 client.26633 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:28:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 -- 172.21.15.5:0/826031625 shutdown_connections 2024-09-11T22:28:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 1 -- 172.21.15.5:0/826031625 wait complete. 2024-09-11T22:28:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.490+0000 7f1b9a57c740 20 client.26633 trim_cache size 0 max 16384 2024-09-11T22:28:00.501 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:28:00.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 -- 172.21.15.5:0/3097188214 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b7c0a2d70 msgr2=0x7f1b7c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b7c0a2d70 0x7f1b7c0a5230 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x5560ffad7ac0 tx=0x7f1b8c075a90 comp rx=0 tx=0).stop 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 -- 172.21.15.5:0/3097188214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b90102d40 msgr2=0x7f1b9019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b90102d40 0x7f1b9019a670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00c380 tx=0x7f1b8c00c840 comp rx=0 tx=0).stop 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 -- 172.21.15.5:0/3097188214 shutdown_connections 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b7c0a2d70 0x7f1b7c0a5230 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b9010c170 0x7f1b9019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b90103660 0x7f1b9019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 --2- 172.21.15.5:0/3097188214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b90102d40 0x7f1b9019a670 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 -- 172.21.15.5:0/3097188214 >> 172.21.15.5:0/3097188214 conn(0x7f1b900fc7a0 msgr2=0x7f1b90101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 -- 172.21.15.5:0/3097188214 shutdown_connections 2024-09-11T22:28:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:00.491+0000 7f1b981a3640 1 -- 172.21.15.5:0/3097188214 wait complete. 2024-09-11T22:28:00.550 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:28:00 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setuser ceph since I am not root 2024-09-11T22:28:00.551 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:28:00 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setgroup ceph since I am not root 2024-09-11T22:28:00.551 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:28:00 smithi159 ceph-mds[65034]: -- 172.21.15.159:0/2196238815 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fbfa64f860 con 0x55fbfa62d000 2024-09-11T22:28:00.551 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:28:00 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: starting mds.k at 2024-09-11T22:28:00.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:00 smithi005 ceph-mon[30943]: daemon mds.k restarted 2024-09-11T22:28:00.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:00 smithi159 ceph-mon[35534]: daemon mds.k restarted 2024-09-11T22:28:00.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:00 smithi172 ceph-mon[35461]: daemon mds.k restarted 2024-09-11T22:28:01.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 429444 8.34 MB/sec execute 139 sec latency 24.995 ms 2024-09-11T22:28:01.381 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:28:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:01.775 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:01 smithi005 ceph-mon[30943]: pgmap v178: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-09-11T22:28:01.775 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:01 smithi005 ceph-mon[30943]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T22:28:01.776 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:01 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6832/1394880869,v1:172.21.15.159:6833/1394880869] up:boot 2024-09-11T22:28:01.776 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:01 smithi005 ceph-mon[30943]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:28:01.776 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:01 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:28:01.776 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:01 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:28:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:01 smithi159 ceph-mon[35534]: pgmap v178: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-09-11T22:28:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:01 smithi159 ceph-mon[35534]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T22:28:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:01 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6832/1394880869,v1:172.21.15.159:6833/1394880869] up:boot 2024-09-11T22:28:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:01 smithi159 ceph-mon[35534]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:28:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:01 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:28:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:01 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:28:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:01 smithi172 ceph-mon[35461]: pgmap v178: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-09-11T22:28:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:01 smithi172 ceph-mon[35461]: osdmap e87: 12 total, 12 up, 12 in 2024-09-11T22:28:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:01 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6832/1394880869,v1:172.21.15.159:6833/1394880869] up:boot 2024-09-11T22:28:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:01 smithi172 ceph-mon[35461]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:28:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:01 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:28:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:01 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:28:02.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.191+0000 7f4d260db640 1 -- 172.21.15.5:0/3135831985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d20102570 msgr2=0x7f4d201029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:02.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.191+0000 7f4d260db640 1 --2- 172.21.15.5:0/3135831985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d20102570 0x7f4d201029b0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009900 tx=0x7f4d0802a480 comp rx=0 tx=0).stop 2024-09-11T22:28:02.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.191+0000 7f4d260db640 1 -- 172.21.15.5:0/3135831985 shutdown_connections 2024-09-11T22:28:02.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.191+0000 7f4d260db640 1 --2- 172.21.15.5:0/3135831985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d20102ef0 0x7f4d2010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.191+0000 7f4d260db640 1 --2- 172.21.15.5:0/3135831985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d20102570 0x7f4d201029b0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.191+0000 7f4d260db640 1 --2- 172.21.15.5:0/3135831985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d20100e20 0x7f4d20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.191+0000 7f4d260db640 1 -- 172.21.15.5:0/3135831985 >> 172.21.15.5:0/3135831985 conn(0x7f4d200fc7d0 msgr2=0x7f4d200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.191+0000 7f4d260db640 1 -- 172.21.15.5:0/3135831985 shutdown_connections 2024-09-11T22:28:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.191+0000 7f4d260db640 1 -- 172.21.15.5:0/3135831985 wait complete. 2024-09-11T22:28:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.191+0000 7f4d260db640 1 Processor -- start 2024-09-11T22:28:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d260db640 1 -- start start 2024-09-11T22:28:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d260db640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d20100e20 0x7f4d20197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d260db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d20102570 0x7f4d20197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d260db640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d20102ef0 0x7f4d2019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d260db640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d20110f50 con 0x7f4d20102570 2024-09-11T22:28:02.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d260db640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d20110dd0 con 0x7f4d20102ef0 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d260db640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201110d0 con 0x7f4d20100e20 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d1effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d20102570 0x7f4d20197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d1effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d20102570 0x7f4d20197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57198/0 (socket says 172.21.15.5:57198) 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d1effd640 1 -- 172.21.15.5:0/88767889 learned_addr learned my addr 172.21.15.5:0/88767889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d20102ef0 0x7f4d2019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d20100e20 0x7f4d20197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d1effd640 1 -- 172.21.15.5:0/88767889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d20100e20 msgr2=0x7f4d20197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d1effd640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d20100e20 0x7f4d20197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d1effd640 1 -- 172.21.15.5:0/88767889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d20102ef0 msgr2=0x7f4d2019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d1effd640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d20102ef0 0x7f4d2019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.192+0000 7f4d1effd640 1 -- 172.21.15.5:0/88767889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2019c3e0 con 0x7f4d20102570 2024-09-11T22:28:02.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.193+0000 7f4d1effd640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d20102570 0x7f4d20197970 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4d08006010 tx=0x7f4d08003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.193+0000 7f4d1cff9640 1 -- 172.21.15.5:0/88767889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0803f070 con 0x7f4d20102570 2024-09-11T22:28:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.193+0000 7f4d260db640 1 -- 172.21.15.5:0/88767889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2019c670 con 0x7f4d20102570 2024-09-11T22:28:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.193+0000 7f4d1cff9640 1 -- 172.21.15.5:0/88767889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d08031c30 con 0x7f4d20102570 2024-09-11T22:28:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.193+0000 7f4d1cff9640 1 -- 172.21.15.5:0/88767889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d08035030 con 0x7f4d20102570 2024-09-11T22:28:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.193+0000 7f4d260db640 1 -- 172.21.15.5:0/88767889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d201a3fb0 con 0x7f4d20102570 2024-09-11T22:28:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.196+0000 7f4d1cff9640 1 -- 172.21.15.5:0/88767889 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4d08042070 con 0x7f4d20102570 2024-09-11T22:28:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.196+0000 7f4d284b4740 1 Processor -- start 2024-09-11T22:28:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d284b4740 1 -- start start 2024-09-11T22:28:02.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cdb40 0x55558a1cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cc5d0 0x55558a1cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1d9580 0x55558a1db970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d284b4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55558a1322f0 con 0x55558a1cdb40 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d284b4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55558a0d9cf0 con 0x55558a1d9580 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d284b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55558a12e2b0 con 0x55558a1cc5d0 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cc5d0 0x55558a1cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d1f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cc5d0 0x55558a1cca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:49038/0 (socket says 172.21.15.5:49038) 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cdb40 0x55558a1cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d1f7fe640 1 -- 172.21.15.5:0/2948669386 learned_addr learned my addr 172.21.15.5:0/2948669386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d1effd640 1 --2- 172.21.15.5:0/2948669386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1d9580 0x55558a1db970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d1f7fe640 1 -- 172.21.15.5:0/2948669386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1d9580 msgr2=0x55558a1db970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:02.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/2948669386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1d9580 0x55558a1db970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.197+0000 7f4d1f7fe640 1 -- 172.21.15.5:0/2948669386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cdb40 msgr2=0x55558a1cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.198+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/2948669386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cdb40 0x55558a1cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.198+0000 7f4d1f7fe640 1 -- 172.21.15.5:0/2948669386 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558a0be3a0 con 0x55558a1cc5d0 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.198+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/2948669386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cdb40 0x55558a1cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.198+0000 7f4d1cff9640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4d0c0a2cf0 0x7f4d0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.198+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/2948669386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cc5d0 0x55558a1cca10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4d100039f0 tx=0x7f4d10003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58cf665bcd5712f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.198+0000 7f4d15ffb640 1 -- 172.21.15.5:0/2948669386 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10004b20 con 0x55558a1cc5d0 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.198+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4d0c0a2cf0 0x7f4d0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.198+0000 7f4d15ffb640 1 -- 172.21.15.5:0/2948669386 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d10004cc0 con 0x55558a1cc5d0 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.198+0000 7f4d15ffb640 1 -- 172.21.15.5:0/2948669386 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100127d0 con 0x55558a1cc5d0 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.198+0000 7f4d1cff9640 1 -- 172.21.15.5:0/88767889 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7f4d080efae0 con 0x7f4d20102570 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.199+0000 7f4d284b4740 1 -- 172.21.15.5:0/2948669386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cc5d0 msgr2=0x55558a1cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:02.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.199+0000 7f4d284b4740 1 --2- 172.21.15.5:0/2948669386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cc5d0 0x55558a1cca10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4d100039f0 tx=0x7f4d10003eb0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.199+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4d0c0a2cf0 0x7f4d0c0a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009f70 tx=0x7f4d18005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.199+0000 7f4d284b4740 1 -- 172.21.15.5:0/2948669386 shutdown_connections 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.199+0000 7f4d284b4740 1 --2- 172.21.15.5:0/2948669386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1d9580 0x55558a1db970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.199+0000 7f4d284b4740 1 --2- 172.21.15.5:0/2948669386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cc5d0 0x55558a1cca10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.199+0000 7f4d284b4740 1 --2- 172.21.15.5:0/2948669386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cdb40 0x55558a1cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.199+0000 7f4d284b4740 1 -- 172.21.15.5:0/2948669386 >> 172.21.15.5:0/2948669386 conn(0x55558a195310 msgr2=0x55558a1c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.199+0000 7f4d284b4740 1 -- 172.21.15.5:0/2948669386 shutdown_connections 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.199+0000 7f4d284b4740 1 -- 172.21.15.5:0/2948669386 wait complete. 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.200+0000 7f4d284b4740 1 -- start start 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.200+0000 7f4d284b4740 1 Processor -- start 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 0x55558a1d8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cdb40 0x55558a1d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1d6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d284b4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55558a1dc280 con 0x55558a1d9580 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d284b4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55558a0ed060 con 0x55558a1cc5d0 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d284b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55558a1dc400 con 0x55558a1cdb40 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d284b4740 10 client.0 _ll_register_callbacks cb 0x555589e408c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 0x55558a1d8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1d6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 0x55558a1d8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39108/0 (socket says 172.21.15.5:39108) 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1f7fe640 1 -- 172.21.15.5:0/2455652395 learned_addr learned my addr 172.21.15.5:0/2455652395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1effd640 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cdb40 0x55558a1d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1ffff640 1 -- 172.21.15.5:0/2455652395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cdb40 msgr2=0x55558a1d61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cdb40 0x55558a1d61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1ffff640 1 -- 172.21.15.5:0/2455652395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 msgr2=0x55558a1d8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 0x55558a1d8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1ffff640 1 -- 172.21.15.5:0/2455652395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555589ea33c0 con 0x55558a1d9580 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1d6770 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4d18012cb0 tx=0x7f4d18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef13e7ff4b7c95ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.201+0000 7f4cfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.202+0000 7f4cfeffd640 1 -- 172.21.15.5:0/2455652395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1802e060 con 0x55558a1d9580 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.202+0000 7f4cfeffd640 1 -- 172.21.15.5:0/2455652395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7f4d18018030 con 0x55558a1d9580 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.202+0000 7f4d284b4740 1 -- 172.21.15.5:0/2455652395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d10004cc0 con 0x55558a1d9580 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.202+0000 7f4cfeffd640 1 -- 172.21.15.5:0/2455652395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4d18009bc0 con 0x55558a1d9580 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.202+0000 7f4d284b4740 10 client.17238 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.202+0000 7f4d284b4740 1 -- 172.21.15.5:0/2455652395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55558a1e7d30 con 0x55558a1d9580 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.202+0000 7f4cfeffd640 1 -- 172.21.15.5:0/2455652395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d10004cc0 con 0x55558a1d9580 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.202+0000 7f4cfeffd640 1 -- 172.21.15.5:0/2455652395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4d180316c0 con 0x55558a1d9580 2024-09-11T22:28:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.203+0000 7f4d284b4740 10 client.17238 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.203+0000 7f4d284b4740 10 client.17238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.203+0000 7f4d284b4740 20 client.17238 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.203+0000 7f4d284b4740 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55558a1e91f0 0x55558a1eb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.203+0000 7f4d284b4740 4 client.17238 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.203+0000 7f4d284b4740 1 -- 172.21.15.5:0/2455652395 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55558a13f860 con 0x55558a1e91f0 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.203+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55558a1e91f0 0x55558a1eb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.203+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55558a1e91f0 0x55558a1eb640 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8e7da9e980aff0 server_cookie=ad564556406dcbd9 in_seq=0 out_seq=0 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.203+0000 7f4cfeffd640 10 client.17238 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4cfeffd640 1 -- 172.21.15.5:0/2455652395 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55558a13f860 con 0x55558a1e91f0 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4cfeffd640 10 client.17238 handle_command_reply: tid=0 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 client.17238 shutdown 2024-09-11T22:28:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 -- 172.21.15.5:0/2455652395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 msgr2=0x55558a1d6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1d6770 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4d18012cb0 tx=0x7f4d18004730 comp rx=0 tx=0).stop 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 -- 172.21.15.5:0/2455652395 shutdown_connections 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55558a1e91f0 0x55558a1eb640 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1d6770 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cdb40 0x55558a1d61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 --2- 172.21.15.5:0/2455652395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 0x55558a1d8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4cfeffd640 0 client.17238 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 -- 172.21.15.5:0/2455652395 >> 172.21.15.5:0/2455652395 conn(0x55558a195310 msgr2=0x55558a1ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 -- 172.21.15.5:0/2455652395 shutdown_connections 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 1 -- 172.21.15.5:0/2455652395 wait complete. 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.205+0000 7f4d284b4740 20 client.17238 trim_cache size 0 max 16384 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.218+0000 7f4d284b4740 1 Processor -- start 2024-09-11T22:28:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.218+0000 7f4d284b4740 1 -- start start 2024-09-11T22:28:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cc5d0 0x55558a1c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cdb40 0x55558a1e5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1d9580 0x55558a1f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d284b4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201ab7d0 con 0x55558a1cc5d0 2024-09-11T22:28:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d284b4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201ad310 con 0x55558a1cdb40 2024-09-11T22:28:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d284b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201abea0 con 0x55558a1d9580 2024-09-11T22:28:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cc5d0 0x55558a1c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cc5d0 0x55558a1c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57226/0 (socket says 172.21.15.5:57226) 2024-09-11T22:28:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1f7fe640 1 -- 172.21.15.5:0/853964064 learned_addr learned my addr 172.21.15.5:0/853964064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/853964064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1d9580 0x55558a1f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1effd640 1 --2- 172.21.15.5:0/853964064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cdb40 0x55558a1e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1f7fe640 1 -- 172.21.15.5:0/853964064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1d9580 msgr2=0x55558a1f0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/853964064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1d9580 0x55558a1f0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1f7fe640 1 -- 172.21.15.5:0/853964064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cdb40 msgr2=0x55558a1e5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/853964064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cdb40 0x55558a1e5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1f7fe640 1 -- 172.21.15.5:0/853964064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558a1d1dd0 con 0x55558a1cc5d0 2024-09-11T22:28:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/853964064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cc5d0 0x55558a1c8e10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x55558a1ebeb0 tx=0x7f4d10000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d7ca10c4ca39cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.219+0000 7f4d157fa640 1 -- 172.21.15.5:0/853964064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100124e0 con 0x55558a1cc5d0 2024-09-11T22:28:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d157fa640 1 -- 172.21.15.5:0/853964064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d10012b60 con 0x55558a1cc5d0 2024-09-11T22:28:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d157fa640 1 -- 172.21.15.5:0/853964064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10028630 con 0x55558a1cc5d0 2024-09-11T22:28:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d284b4740 1 -- 172.21.15.5:0/853964064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cc5d0 msgr2=0x55558a1c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d284b4740 1 --2- 172.21.15.5:0/853964064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cc5d0 0x55558a1c8e10 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x55558a1ebeb0 tx=0x7f4d10000f40 comp rx=0 tx=0).stop 2024-09-11T22:28:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d284b4740 1 -- 172.21.15.5:0/853964064 shutdown_connections 2024-09-11T22:28:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d284b4740 1 --2- 172.21.15.5:0/853964064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1d9580 0x55558a1f0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d284b4740 1 --2- 172.21.15.5:0/853964064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cdb40 0x55558a1e5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d284b4740 1 --2- 172.21.15.5:0/853964064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1cc5d0 0x55558a1c8e10 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d284b4740 1 -- 172.21.15.5:0/853964064 >> 172.21.15.5:0/853964064 conn(0x55558a195310 msgr2=0x55558a1c8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d284b4740 1 -- 172.21.15.5:0/853964064 shutdown_connections 2024-09-11T22:28:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.220+0000 7f4d284b4740 1 -- 172.21.15.5:0/853964064 wait complete. 2024-09-11T22:28:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d284b4740 1 -- start start 2024-09-11T22:28:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d284b4740 1 Processor -- start 2024-09-11T22:28:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 0x55558a1e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cdb40 0x55558a1e1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d284b4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1de660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d284b4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201acc40 con 0x55558a1d9580 2024-09-11T22:28:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d284b4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201ac570 con 0x55558a1cc5d0 2024-09-11T22:28:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d284b4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201a97b0 con 0x55558a1cdb40 2024-09-11T22:28:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d284b4740 10 client.0 _ll_register_callbacks cb 0x7f4d18012f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:02.247 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:28:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1de660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d1f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 0x55558a1e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1de660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57230/0 (socket says 172.21.15.5:57230) 2024-09-11T22:28:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d1ffff640 1 -- 172.21.15.5:0/307574352 learned_addr learned my addr 172.21.15.5:0/307574352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.221+0000 7f4d1effd640 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cdb40 0x55558a1e1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4d1ffff640 1 -- 172.21.15.5:0/307574352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cdb40 msgr2=0x55558a1e1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cdb40 0x55558a1e1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4d1ffff640 1 -- 172.21.15.5:0/307574352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 msgr2=0x55558a1e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 0x55558a1e1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4d1ffff640 1 -- 172.21.15.5:0/307574352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55558a1e22c0 con 0x55558a1d9580 2024-09-11T22:28:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4d1ffff640 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1de660 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4d18031cd0 tx=0x7f4d180093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0a6f1325dc8d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4cfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4cfeffd640 1 -- 172.21.15.5:0/307574352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1801e7c0 con 0x55558a1d9580 2024-09-11T22:28:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4cfeffd640 1 -- 172.21.15.5:0/307574352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7f4d18004020 con 0x55558a1d9580 2024-09-11T22:28:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4d284b4740 1 -- 172.21.15.5:0/307574352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d10012b60 con 0x55558a1d9580 2024-09-11T22:28:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4cfeffd640 1 -- 172.21.15.5:0/307574352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4d18004220 con 0x55558a1d9580 2024-09-11T22:28:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4d284b4740 10 client.17244 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4d284b4740 1 -- 172.21.15.5:0/307574352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55558a2544c0 con 0x55558a1d9580 2024-09-11T22:28:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4cfeffd640 1 -- 172.21.15.5:0/307574352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d10012b60 con 0x55558a1d9580 2024-09-11T22:28:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.222+0000 7f4cfeffd640 1 -- 172.21.15.5:0/307574352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4d18004a10 con 0x55558a1d9580 2024-09-11T22:28:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.223+0000 7f4d284b4740 10 client.17244 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.223+0000 7f4d284b4740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.223+0000 7f4d284b4740 20 client.17244 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.223+0000 7f4d284b4740 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55558a254c90 0x55558a2570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.223+0000 7f4d284b4740 4 client.17244 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:28:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.223+0000 7f4d284b4740 1 -- 172.21.15.5:0/307574352 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55558a139d70 con 0x55558a254c90 2024-09-11T22:28:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.223+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55558a254c90 0x55558a2570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.223+0000 7f4cfeffd640 10 client.17244 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.223+0000 7f4d1f7fe640 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55558a254c90 0x55558a2570b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=755f331a24acd427 server_cookie=fb08b0b002f0f682 in_seq=0 out_seq=0 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4cfeffd640 1 -- 172.21.15.5:0/307574352 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55558a139d70 con 0x55558a254c90 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4cfeffd640 10 client.17244 handle_command_reply: tid=0 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 client.17244 shutdown 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 -- 172.21.15.5:0/307574352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 msgr2=0x55558a1de660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1de660 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f4d18031cd0 tx=0x7f4d180093b0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 -- 172.21.15.5:0/307574352 shutdown_connections 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55558a254c90 0x55558a2570b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55558a1d9580 0x55558a1de660 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55558a1cdb40 0x55558a1e1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4cfeffd640 0 client.17244 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 --2- 172.21.15.5:0/307574352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55558a1cc5d0 0x55558a1e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 -- 172.21.15.5:0/307574352 >> 172.21.15.5:0/307574352 conn(0x55558a195310 msgr2=0x55558a1ce930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 -- 172.21.15.5:0/307574352 shutdown_connections 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 1 -- 172.21.15.5:0/307574352 wait complete. 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.224+0000 7f4d284b4740 20 client.17244 trim_cache size 0 max 16384 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 -- 172.21.15.5:0/88767889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4d0c0a2cf0 msgr2=0x7f4d0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4d0c0a2cf0 0x7f4d0c0a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009f70 tx=0x7f4d18005f90 comp rx=0 tx=0).stop 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 -- 172.21.15.5:0/88767889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d20102570 msgr2=0x7f4d20197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d20102570 0x7f4d20197970 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4d08006010 tx=0x7f4d08003c10 comp rx=0 tx=0).stop 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 -- 172.21.15.5:0/88767889 shutdown_connections 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4d0c0a2cf0 0x7f4d0c0a51b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4d20102ef0 0x7f4d2019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d20102570 0x7f4d20197970 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 --2- 172.21.15.5:0/88767889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d20100e20 0x7f4d20197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:02.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 -- 172.21.15.5:0/88767889 >> 172.21.15.5:0/88767889 conn(0x7f4d200fc7d0 msgr2=0x7f4d200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:02.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 -- 172.21.15.5:0/88767889 shutdown_connections 2024-09-11T22:28:02.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:02.225+0000 7f4d260db640 1 -- 172.21.15.5:0/88767889 wait complete. 2024-09-11T22:28:02.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 431265 8.32 MB/sec execute 140 sec latency 24.858 ms 2024-09-11T22:28:03.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 433489 8.36 MB/sec execute 141 sec latency 11.453 ms 2024-09-11T22:28:03.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:03 smithi005 ceph-mon[30943]: pgmap v180: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-11T22:28:03.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:03 smithi159 ceph-mon[35534]: pgmap v180: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-11T22:28:03.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:03 smithi172 ceph-mon[35461]: pgmap v180: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-11T22:28:04.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 435679 8.39 MB/sec execute 142 sec latency 118.565 ms 2024-09-11T22:28:04.507 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:28:04.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:28:04.775 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:05.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 437738 8.41 MB/sec execute 143 sec latency 36.799 ms 2024-09-11T22:28:05.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:05 smithi005 ceph-mon[30943]: pgmap v181: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-11T22:28:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.670+0000 7fb12fb3b640 1 -- 172.21.15.5:0/812771017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128102ef0 msgr2=0x7fb12810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.670+0000 7fb12fb3b640 1 --2- 172.21.15.5:0/812771017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128102ef0 0x7fb12810da40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb120009ca0 tx=0x7fb12002a950 comp rx=0 tx=0).stop 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.670+0000 7fb12fb3b640 1 -- 172.21.15.5:0/812771017 shutdown_connections 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.670+0000 7fb12fb3b640 1 --2- 172.21.15.5:0/812771017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128102ef0 0x7fb12810da40 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.670+0000 7fb12fb3b640 1 --2- 172.21.15.5:0/812771017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb128102570 0x7fb1281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.670+0000 7fb12fb3b640 1 --2- 172.21.15.5:0/812771017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb128100e20 0x7fb128101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.670+0000 7fb12fb3b640 1 -- 172.21.15.5:0/812771017 >> 172.21.15.5:0/812771017 conn(0x7fb1280fc7d0 msgr2=0x7fb1280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.670+0000 7fb12fb3b640 1 -- 172.21.15.5:0/812771017 shutdown_connections 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.670+0000 7fb12fb3b640 1 -- 172.21.15.5:0/812771017 wait complete. 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12fb3b640 1 Processor -- start 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12fb3b640 1 -- start start 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12fb3b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb128100e20 0x7fb128198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12fb3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128102570 0x7fb128198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12fb3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb128102ef0 0x7fb12819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12fb3b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb128110f80 con 0x7fb128102570 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12fb3b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb128110e00 con 0x7fb128100e20 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12fb3b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb128111100 con 0x7fb128102ef0 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12d0d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128102570 0x7fb128198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12d8d7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb128100e20 0x7fb128198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12d0d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128102570 0x7fb128198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57250/0 (socket says 172.21.15.5:57250) 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12d0d6640 1 -- 172.21.15.5:0/1810332987 learned_addr learned my addr 172.21.15.5:0/1810332987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12e0d8640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb128102ef0 0x7fb12819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.671+0000 7fb12d0d6640 1 -- 172.21.15.5:0/1810332987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb128102ef0 msgr2=0x7fb12819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.672+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb128102ef0 0x7fb12819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.672+0000 7fb12d0d6640 1 -- 172.21.15.5:0/1810332987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb128100e20 msgr2=0x7fb128198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.672+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb128100e20 0x7fb128198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.672+0000 7fb12d0d6640 1 -- 172.21.15.5:0/1810332987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12819d400 con 0x7fb128102570 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.672+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128102570 0x7fb128198990 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9f0 tx=0x7fb11800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.672+0000 7fb11effd640 1 -- 172.21.15.5:0/1810332987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11800cd80 con 0x7fb128102570 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.672+0000 7fb12fb3b640 1 -- 172.21.15.5:0/1810332987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12819d6f0 con 0x7fb128102570 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.672+0000 7fb11effd640 1 -- 172.21.15.5:0/1810332987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb118010910 con 0x7fb128102570 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.672+0000 7fb11effd640 1 -- 172.21.15.5:0/1810332987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1180188f0 con 0x7fb128102570 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.672+0000 7fb12fb3b640 1 -- 172.21.15.5:0/1810332987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1281a4fd0 con 0x7fb128102570 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.675+0000 7fb11effd640 1 -- 172.21.15.5:0/1810332987 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb118010ac0 con 0x7fb128102570 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.675+0000 7fb131f14740 1 Processor -- start 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.675+0000 7fb131f14740 1 -- start start 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb131f14740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 0x559694447e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb131f14740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694456c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb131f14740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x559694459540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb131f14740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596943ad280 con 0x559694448380 2024-09-11T22:28:05.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb131f14740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559694354b10 con 0x559694457150 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb131f14740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5596943a9240 con 0x559694447a60 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12d8d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694456c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12e0d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 0x559694447e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12d0d6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x559694459540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12d8d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694456c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57254/0 (socket says 172.21.15.5:57254) 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12d8d7640 1 -- 172.21.15.5:0/4013382638 learned_addr learned my addr 172.21.15.5:0/4013382638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12e0d8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 0x559694447e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:49096/0 (socket says 172.21.15.5:49096) 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb11effd640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1140a2cf0 0x7fb1140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12e0d8640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1140a2cf0 0x7fb1140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12d8d7640 1 -- 172.21.15.5:0/4013382638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 msgr2=0x559694447e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/4013382638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 0x559694447e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12d8d7640 1 -- 172.21.15.5:0/4013382638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 msgr2=0x559694459540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/4013382638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x559694459540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12d8d7640 1 -- 172.21.15.5:0/4013382638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596943611f0 con 0x559694448380 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb11effd640 1 -- 172.21.15.5:0/1810332987 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fb1180cf820 con 0x7fb128102570 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.676+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/4013382638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694456c10 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb124009d20 tx=0x7fb124011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3897d3120ce5f68d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.677+0000 7fb113fff640 1 -- 172.21.15.5:0/4013382638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12401e070 con 0x559694448380 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.677+0000 7fb12e0d8640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1140a2cf0 0x7fb1140a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb120004ad0 tx=0x7fb12002a950 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.677+0000 7fb113fff640 1 -- 172.21.15.5:0/4013382638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124019b10 con 0x559694448380 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.677+0000 7fb113fff640 1 -- 172.21.15.5:0/4013382638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12400eb30 con 0x559694448380 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 -- 172.21.15.5:0/4013382638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 msgr2=0x559694456c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 --2- 172.21.15.5:0/4013382638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694456c10 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fb124009d20 tx=0x7fb124011810 comp rx=0 tx=0).stop 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 -- 172.21.15.5:0/4013382638 shutdown_connections 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 --2- 172.21.15.5:0/4013382638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x559694459540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 --2- 172.21.15.5:0/4013382638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694456c10 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 --2- 172.21.15.5:0/4013382638 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 0x559694447e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 -- 172.21.15.5:0/4013382638 >> 172.21.15.5:0/4013382638 conn(0x5596944440f0 msgr2=0x5596944444d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 -- 172.21.15.5:0/4013382638 shutdown_connections 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 -- 172.21.15.5:0/4013382638 wait complete. 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 -- start start 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.678+0000 7fb131f14740 1 Processor -- start 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb131f14740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694447a60 0x559694454090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb131f14740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x5596944545d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb131f14740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694457150 0x559694454b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb131f14740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559694459d80 con 0x559694448380 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb131f14740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559694367f10 con 0x559694447a60 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb131f14740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559694459f00 con 0x559694457150 2024-09-11T22:28:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb131f14740 10 client.0 _ll_register_callbacks cb 0x559694330a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d0d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x5596944545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d8d7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694447a60 0x559694454090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d0d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x5596944545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57262/0 (socket says 172.21.15.5:57262) 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d0d6640 1 -- 172.21.15.5:0/4270711801 learned_addr learned my addr 172.21.15.5:0/4270711801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12e0d8640 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694457150 0x559694454b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d0d6640 1 -- 172.21.15.5:0/4270711801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694457150 msgr2=0x559694454b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694457150 0x559694454b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d0d6640 1 -- 172.21.15.5:0/4270711801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694447a60 msgr2=0x559694454090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694447a60 0x559694454090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d0d6640 1 -- 172.21.15.5:0/4270711801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55969411e6a0 con 0x559694448380 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694447a60 0x559694454090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x5596944545d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb118098780 tx=0x7fb118009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c69c58a76aab0fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb1117fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb1117fa640 1 -- 172.21.15.5:0/4270711801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11809b400 con 0x559694448380 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb1117fa640 1 -- 172.21.15.5:0/4270711801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fb118010430 con 0x559694448380 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.679+0000 7fb131f14740 1 -- 172.21.15.5:0/4270711801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb124019b10 con 0x559694448380 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb1117fa640 1 -- 172.21.15.5:0/4270711801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb1180a00b0 con 0x559694448380 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb131f14740 10 client.17256 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb131f14740 1 -- 172.21.15.5:0/4270711801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559694465a20 con 0x559694448380 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb1117fa640 1 -- 172.21.15.5:0/4270711801 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124019b10 con 0x559694448380 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb1117fa640 1 -- 172.21.15.5:0/4270711801 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fb118086030 con 0x559694448380 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb131f14740 10 client.17256 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb131f14740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb131f14740 20 client.17256 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb131f14740 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559694466ee0 0x5596944692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb131f14740 4 client.17256 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb131f14740 1 -- 172.21.15.5:0/4270711801 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596943ba7f0 con 0x559694466ee0 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.680+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559694466ee0 0x5596944692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.681+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559694466ee0 0x5596944692d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=533152da49a91e98 server_cookie=bf3719bff81d82d4 in_seq=0 out_seq=0 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.681+0000 7fb1117fa640 10 client.17256 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.681+0000 7fb1117fa640 1 -- 172.21.15.5:0/4270711801 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596943ba7f0 con 0x559694466ee0 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.681+0000 7fb1117fa640 10 client.17256 handle_command_reply: tid=0 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.681+0000 7fb131f14740 1 client.17256 shutdown 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 1 -- 172.21.15.5:0/4270711801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 msgr2=0x5596944545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x5596944545d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb118098780 tx=0x7fb118009700 comp rx=0 tx=0).stop 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 1 -- 172.21.15.5:0/4270711801 shutdown_connections 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559694466ee0 0x5596944692d0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694457150 0x559694454b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x5596944545d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 1 --2- 172.21.15.5:0/4270711801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694447a60 0x559694454090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 1 -- 172.21.15.5:0/4270711801 >> 172.21.15.5:0/4270711801 conn(0x5596944440f0 msgr2=0x559694449db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb1117fa640 0 client.17256 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 1 -- 172.21.15.5:0/4270711801 shutdown_connections 2024-09-11T22:28:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 1 -- 172.21.15.5:0/4270711801 wait complete. 2024-09-11T22:28:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.682+0000 7fb131f14740 20 client.17256 trim_cache size 0 max 16384 2024-09-11T22:28:05.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb131f14740 1 Processor -- start 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb131f14740 1 -- start start 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb131f14740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694447a60 0x559694455ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb131f14740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694448380 0x55969446eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb131f14740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x55969446f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb131f14740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281ab700 con 0x559694447a60 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb131f14740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281ad240 con 0x559694457150 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb131f14740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281abdd0 con 0x559694448380 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb12d8d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694447a60 0x559694455ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb12d8d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694447a60 0x559694455ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57268/0 (socket says 172.21.15.5:57268) 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb12d8d7640 1 -- 172.21.15.5:0/1026591670 learned_addr learned my addr 172.21.15.5:0/1026591670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/1026591670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694448380 0x55969446eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.700+0000 7fb12e0d8640 1 --2- 172.21.15.5:0/1026591670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x55969446f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.701+0000 7fb12d8d7640 1 -- 172.21.15.5:0/1026591670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694448380 msgr2=0x55969446eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.701+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/1026591670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694448380 0x55969446eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.701+0000 7fb12d8d7640 1 -- 172.21.15.5:0/1026591670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 msgr2=0x55969446f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.701+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/1026591670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x55969446f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.701+0000 7fb12d8d7640 1 -- 172.21.15.5:0/1026591670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559694445280 con 0x559694447a60 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.701+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/1026591670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694448380 0x55969446eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.701+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/1026591670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694447a60 0x559694455ef0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb1240058c0 tx=0x7fb124005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07abfe133ae4bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.701+0000 7fb1137fe640 1 -- 172.21.15.5:0/1026591670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124033070 con 0x559694447a60 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.701+0000 7fb1137fe640 1 -- 172.21.15.5:0/1026591670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12401f040 con 0x559694447a60 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.701+0000 7fb1137fe640 1 -- 172.21.15.5:0/1026591670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12401e040 con 0x559694447a60 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 -- 172.21.15.5:0/1026591670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694447a60 msgr2=0x559694455ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 --2- 172.21.15.5:0/1026591670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694447a60 0x559694455ef0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb1240058c0 tx=0x7fb124005f70 comp rx=0 tx=0).stop 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 -- 172.21.15.5:0/1026591670 shutdown_connections 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 --2- 172.21.15.5:0/1026591670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x55969446f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 --2- 172.21.15.5:0/1026591670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694448380 0x55969446eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 --2- 172.21.15.5:0/1026591670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694447a60 0x559694455ef0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 -- 172.21.15.5:0/1026591670 >> 172.21.15.5:0/1026591670 conn(0x5596944440f0 msgr2=0x5596944557e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 -- 172.21.15.5:0/1026591670 shutdown_connections 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 -- 172.21.15.5:0/1026591670 wait complete. 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 -- start start 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.702+0000 7fb131f14740 1 Processor -- start 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb131f14740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 0x55969445f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb131f14740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694450de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb131f14740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x559694451320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb131f14740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281acb70 con 0x559694448380 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb131f14740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281ac4a0 con 0x559694457150 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb131f14740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281a96e0 con 0x559694447a60 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb131f14740 10 client.0 _ll_register_callbacks cb 0x7fb11809bc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d0d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694450de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d0d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694450de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57270/0 (socket says 172.21.15.5:57270) 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d0d6640 1 -- 172.21.15.5:0/1611080982 learned_addr learned my addr 172.21.15.5:0/1611080982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 0x55969445f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12e0d8640 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x559694451320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d0d6640 1 -- 172.21.15.5:0/1611080982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 msgr2=0x55969445f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 0x55969445f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d0d6640 1 -- 172.21.15.5:0/1611080982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 msgr2=0x559694451320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x559694451320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d0d6640 1 -- 172.21.15.5:0/1611080982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559694451b50 con 0x559694448380 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 0x55969445f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb12d0d6640 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694450de0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb11800c6a0 tx=0x7fb118009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76fa06f25613ead server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb1117fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.703+0000 7fb1117fa640 1 -- 172.21.15.5:0/1611080982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11809b710 con 0x559694448380 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb1117fa640 1 -- 172.21.15.5:0/1611080982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fb1180a4dc0 con 0x559694448380 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb131f14740 1 -- 172.21.15.5:0/1611080982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12401f040 con 0x559694448380 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb1117fa640 1 -- 172.21.15.5:0/1611080982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb1180a4970 con 0x559694448380 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb131f14740 10 client.17262 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb131f14740 1 -- 172.21.15.5:0/1611080982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5596944d1b30 con 0x559694448380 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb1117fa640 1 -- 172.21.15.5:0/1611080982 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb12401f040 con 0x559694448380 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb1117fa640 1 -- 172.21.15.5:0/1611080982 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fb118086040 con 0x559694448380 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb131f14740 10 client.17262 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb131f14740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb131f14740 20 client.17262 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb131f14740 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5596944d2250 0x5596944d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb131f14740 4 client.17262 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb131f14740 1 -- 172.21.15.5:0/1611080982 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5596943b4d00 con 0x5596944d2250 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.704+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5596944d2250 0x5596944d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.705+0000 7fb1117fa640 10 client.17262 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.705+0000 7fb12d8d7640 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5596944d2250 0x5596944d4670 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95d05eea6ecfd480 server_cookie=969c3d707c9cba0d in_seq=0 out_seq=0 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb1117fa640 1 -- 172.21.15.5:0/1611080982 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5596943b4d00 con 0x5596944d2250 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb1117fa640 10 client.17262 handle_command_reply: tid=0 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb131f14740 1 client.17262 shutdown 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb131f14740 1 -- 172.21.15.5:0/1611080982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 msgr2=0x559694450de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb131f14740 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694450de0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb11800c6a0 tx=0x7fb118009310 comp rx=0 tx=0).stop 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb131f14740 1 -- 172.21.15.5:0/1611080982 shutdown_connections 2024-09-11T22:28:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb131f14740 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5596944d2250 0x5596944d4670 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb131f14740 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559694457150 0x559694451320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb131f14740 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559694448380 0x559694450de0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb131f14740 1 --2- 172.21.15.5:0/1611080982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559694447a60 0x55969445f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb1117fa640 0 client.17262 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.706+0000 7fb131f14740 1 -- 172.21.15.5:0/1611080982 >> 172.21.15.5:0/1611080982 conn(0x5596944440f0 msgr2=0x559694442a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb131f14740 1 -- 172.21.15.5:0/1611080982 shutdown_connections 2024-09-11T22:28:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb131f14740 1 -- 172.21.15.5:0/1611080982 wait complete. 2024-09-11T22:28:05.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb131f14740 20 client.17262 trim_cache size 0 max 16384 2024-09-11T22:28:05.714 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:28:05.714 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:28:05.714 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "7d38d584-59d3-4c01-a5d2-99f9a8830d5a", 2024-09-11T22:28:05.714 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:28:05.714 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:28:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 -- 172.21.15.5:0/1810332987 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1140a2cf0 msgr2=0x7fb1140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1140a2cf0 0x7fb1140a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb120004ad0 tx=0x7fb12002a950 comp rx=0 tx=0).stop 2024-09-11T22:28:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 -- 172.21.15.5:0/1810332987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128102570 msgr2=0x7fb128198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128102570 0x7fb128198990 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fb11800e9f0 tx=0x7fb11800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 -- 172.21.15.5:0/1810332987 shutdown_connections 2024-09-11T22:28:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb128102ef0 0x7fb12819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128102570 0x7fb128198990 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1140a2cf0 0x7fb1140a51b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 --2- 172.21.15.5:0/1810332987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb128100e20 0x7fb128198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 -- 172.21.15.5:0/1810332987 >> 172.21.15.5:0/1810332987 conn(0x7fb1280fc7d0 msgr2=0x7fb128069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 -- 172.21.15.5:0/1810332987 shutdown_connections 2024-09-11T22:28:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:05.707+0000 7fb12fb3b640 1 -- 172.21.15.5:0/1810332987 wait complete. 2024-09-11T22:28:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:05 smithi159 ceph-mon[35534]: pgmap v181: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-11T22:28:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:05 smithi172 ceph-mon[35461]: pgmap v181: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-11T22:28:06.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 440104 8.45 MB/sec execute 144 sec latency 41.119 ms 2024-09-11T22:28:06.601 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:06 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:28:06.601 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:06 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:06.601 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:06 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:28:06.601 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:06 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:28:06.601 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:06 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:06.601 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:06 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:06.603 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:28:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:06 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:28:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:06 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:06 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:28:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:06 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:28:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:06 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:06 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:06 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:28:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:06 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:06 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:28:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:06 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:28:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:06 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:06 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:07.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 441464 8.43 MB/sec execute 145 sec latency 17.160 ms 2024-09-11T22:28:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.415+0000 7f52bfdf7640 1 -- 172.21.15.5:0/3400849124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b8106f20 msgr2=0x7f52b810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.415+0000 7f52bfdf7640 1 --2- 172.21.15.5:0/3400849124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b8106f20 0x7f52b810eaa0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009ca0 tx=0x7f52b002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.415+0000 7f52bfdf7640 1 -- 172.21.15.5:0/3400849124 shutdown_connections 2024-09-11T22:28:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.415+0000 7f52bfdf7640 1 --2- 172.21.15.5:0/3400849124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b8106f20 0x7f52b810eaa0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.415+0000 7f52bfdf7640 1 --2- 172.21.15.5:0/3400849124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b81027a0 0x7f52b81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.415+0000 7f52bfdf7640 1 --2- 172.21.15.5:0/3400849124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b8101df0 0x7f52b81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.415+0000 7f52bfdf7640 1 -- 172.21.15.5:0/3400849124 >> 172.21.15.5:0/3400849124 conn(0x7f52b80fd8b0 msgr2=0x7f52b80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.415+0000 7f52bfdf7640 1 -- 172.21.15.5:0/3400849124 shutdown_connections 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.415+0000 7f52bfdf7640 1 -- 172.21.15.5:0/3400849124 wait complete. 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52bfdf7640 1 Processor -- start 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52bfdf7640 1 -- start start 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52bfdf7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b8101df0 0x7f52b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52bfdf7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b81027a0 0x7f52b8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52bfdf7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8106f20 0x7f52b8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52bfdf7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8111f80 con 0x7f52b8106f20 2024-09-11T22:28:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52bfdf7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8111e00 con 0x7f52b81027a0 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52bfdf7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8112100 con 0x7f52b8101df0 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52be394640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8106f20 0x7f52b8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52be394640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8106f20 0x7f52b8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57280/0 (socket says 172.21.15.5:57280) 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52be394640 1 -- 172.21.15.5:0/2556223659 learned_addr learned my addr 172.21.15.5:0/2556223659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52bd392640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b81027a0 0x7f52b8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52bdb93640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b8101df0 0x7f52b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52be394640 1 -- 172.21.15.5:0/2556223659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b8101df0 msgr2=0x7f52b80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52be394640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b8101df0 0x7f52b80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.416+0000 7f52be394640 1 -- 172.21.15.5:0/2556223659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b81027a0 msgr2=0x7f52b8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.417+0000 7f52be394640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b81027a0 0x7f52b8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.417+0000 7f52be394640 1 -- 172.21.15.5:0/2556223659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b81a68c0 con 0x7f52b8106f20 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.417+0000 7f52bd392640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b81027a0 0x7f52b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.417+0000 7f52be394640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8106f20 0x7f52b8074f70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f52b0031a20 tx=0x7f52b0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.417+0000 7f52aeffd640 1 -- 172.21.15.5:0/2556223659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0003b70 con 0x7f52b8106f20 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.417+0000 7f52bfdf7640 1 -- 172.21.15.5:0/2556223659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b81a6b50 con 0x7f52b8106f20 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.417+0000 7f52aeffd640 1 -- 172.21.15.5:0/2556223659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b0003d10 con 0x7f52b8106f20 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.417+0000 7f52aeffd640 1 -- 172.21.15.5:0/2556223659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b002fc40 con 0x7f52b8106f20 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.417+0000 7f52bfdf7640 1 -- 172.21.15.5:0/2556223659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81a70e0 con 0x7f52b8106f20 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.419+0000 7f52aeffd640 1 -- 172.21.15.5:0/2556223659 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f52b0004050 con 0x7f52b8106f20 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52c21d0740 1 Processor -- start 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52c21d0740 1 -- start start 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f354a0 0x55ffa8f35880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f40140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f42a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52c21d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffa8e9b2e0 con 0x55ffa8f354a0 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52c21d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ffa8e42ce0 con 0x55ffa8f40680 2024-09-11T22:28:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52c21d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffa8e972a0 con 0x55ffa8f35dc0 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52aeffd640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52a40a2cf0 0x7f52a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52bd392640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f354a0 0x55ffa8f35880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52bdb93640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f40140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52bd392640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f354a0 0x55ffa8f35880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57282/0 (socket says 172.21.15.5:57282) 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52bd392640 1 -- 172.21.15.5:0/1204504827 learned_addr learned my addr 172.21.15.5:0/1204504827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52aeffd640 1 -- 172.21.15.5:0/2556223659 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7f52b00f04b0 con 0x7f52b8106f20 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.420+0000 7f52be394640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52a40a2cf0 0x7f52a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52bdb93640 1 -- 172.21.15.5:0/1204504827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 msgr2=0x55ffa8f42a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52bdb93640 1 --2- 172.21.15.5:0/1204504827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f42a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52bdb93640 1 -- 172.21.15.5:0/1204504827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f354a0 msgr2=0x55ffa8f35880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52bdb93640 1 --2- 172.21.15.5:0/1204504827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f354a0 0x55ffa8f35880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52bdb93640 1 -- 172.21.15.5:0/1204504827 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa8e27440 con 0x55ffa8f35dc0 2024-09-11T22:28:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52bd392640 1 --2- 172.21.15.5:0/1204504827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f354a0 0x55ffa8f35880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52bdb93640 1 --2- 172.21.15.5:0/1204504827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f40140 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009c20 tx=0x7f52b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe20f46ed29f4be5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52be394640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52a40a2cf0 0x7f52a40a51b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f52b00c52a0 tx=0x7f52b0033040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52a3fff640 1 -- 172.21.15.5:0/1204504827 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4009dd0 con 0x55ffa8f35dc0 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52a3fff640 1 -- 172.21.15.5:0/1204504827 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b4005020 con 0x55ffa8f35dc0 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.421+0000 7f52a3fff640 1 -- 172.21.15.5:0/1204504827 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4004be0 con 0x55ffa8f35dc0 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.422+0000 7f52c21d0740 1 -- 172.21.15.5:0/1204504827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f35dc0 msgr2=0x55ffa8f40140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.422+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1204504827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f40140 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009c20 tx=0x7f52b4011810 comp rx=0 tx=0).stop 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.422+0000 7f52c21d0740 1 -- 172.21.15.5:0/1204504827 shutdown_connections 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.422+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1204504827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f42a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.422+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1204504827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f40140 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.422+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1204504827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f354a0 0x55ffa8f35880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.422+0000 7f52c21d0740 1 -- 172.21.15.5:0/1204504827 >> 172.21.15.5:0/1204504827 conn(0x55ffa8f31a30 msgr2=0x55ffa8f31e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.422+0000 7f52c21d0740 1 -- 172.21.15.5:0/1204504827 shutdown_connections 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.422+0000 7f52c21d0740 1 -- 172.21.15.5:0/1204504827 wait complete. 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.422+0000 7f52c21d0740 1 -- start start 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52c21d0740 1 Processor -- start 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 0x55ffa8f3ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f3d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52c21d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ffa8f43250 con 0x55ffa8f35dc0 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52c21d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ffa8e56050 con 0x55ffa8f40680 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52c21d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ffa8f433d0 con 0x55ffa8f354a0 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52c21d0740 10 client.0 _ll_register_callbacks cb 0x55ffa8e96160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bd392640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 0x55ffa8f3ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bdb93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bd392640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 0x55ffa8f3ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:49138/0 (socket says 172.21.15.5:49138) 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bd392640 1 -- 172.21.15.5:0/1408934780 learned_addr learned my addr 172.21.15.5:0/1408934780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bd392640 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f3d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bdb93640 1 -- 172.21.15.5:0/1408934780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 msgr2=0x55ffa8f3ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bdb93640 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 0x55ffa8f3ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bdb93640 1 -- 172.21.15.5:0/1408934780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 msgr2=0x55ffa8f3d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bdb93640 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f3d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bdb93640 1 -- 172.21.15.5:0/1408934780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa8c0c3c0 con 0x55ffa8f35dc0 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.423+0000 7f52bd392640 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 0x55ffa8f3ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52bdb93640 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f3d1e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000c00 tx=0x7f52b4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90603222f5798bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52a17fa640 1 -- 172.21.15.5:0/1408934780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4029020 con 0x55ffa8f35dc0 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52c21d0740 1 -- 172.21.15.5:0/1408934780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b4005020 con 0x55ffa8f35dc0 2024-09-11T22:28:07.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52a17fa640 1 -- 172.21.15.5:0/1408934780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7f52b4004c50 con 0x55ffa8f35dc0 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52a17fa640 1 -- 172.21.15.5:0/1408934780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f52b4034050 con 0x55ffa8f35dc0 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52c21d0740 10 client.17274 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52c21d0740 1 -- 172.21.15.5:0/1408934780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ffa8f4ebd0 con 0x55ffa8f35dc0 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52a17fa640 1 -- 172.21.15.5:0/1408934780 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b4005020 con 0x55ffa8f35dc0 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52a17fa640 1 -- 172.21.15.5:0/1408934780 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f52b4023420 con 0x55ffa8f35dc0 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52c21d0740 10 client.17274 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.424+0000 7f52c21d0740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.425+0000 7f52c21d0740 20 client.17274 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.425+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffa8f50030 0x55ffa8f52420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.425+0000 7f52c21d0740 4 client.17274 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.425+0000 7f52c21d0740 1 -- 172.21.15.5:0/1408934780 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffa8ea8850 con 0x55ffa8f50030 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.425+0000 7f52bd392640 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffa8f50030 0x55ffa8f52420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.425+0000 7f52bd392640 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffa8f50030 0x55ffa8f52420 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e50f2e7efc7f49 server_cookie=8373542883695f96 in_seq=0 out_seq=0 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.425+0000 7f52a17fa640 10 client.17274 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52a17fa640 1 -- 172.21.15.5:0/1408934780 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffa8ea8850 con 0x55ffa8f50030 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52a17fa640 10 client.17274 handle_command_reply: tid=0 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 client.17274 shutdown 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 -- 172.21.15.5:0/1408934780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 msgr2=0x55ffa8f3d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f3d1e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f52b4000c00 tx=0x7f52b4004740 comp rx=0 tx=0).stop 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 -- 172.21.15.5:0/1408934780 shutdown_connections 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffa8f50030 0x55ffa8f52420 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f3d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52a17fa640 0 client.17274 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f3d1e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1408934780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 0x55ffa8f3ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 -- 172.21.15.5:0/1408934780 >> 172.21.15.5:0/1408934780 conn(0x55ffa8f31a30 msgr2=0x55ffa8f37760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 -- 172.21.15.5:0/1408934780 shutdown_connections 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 1 -- 172.21.15.5:0/1408934780 wait complete. 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.427+0000 7f52c21d0740 20 client.17274 trim_cache size 0 max 16384 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.444+0000 7f52c21d0740 1 Processor -- start 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.444+0000 7f52c21d0740 1 -- start start 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.444+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f354a0 0x55ffa8f4c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f3c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f40680 0x55ffa8f38900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52c21d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81afb20 con 0x55ffa8f35dc0 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52c21d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b1660 con 0x55ffa8f354a0 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52c21d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b01f0 con 0x55ffa8f40680 2024-09-11T22:28:07.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bd392640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f354a0 0x55ffa8f4c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bd392640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f354a0 0x55ffa8f4c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39228/0 (socket says 172.21.15.5:39228) 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bd392640 1 -- 172.21.15.5:0/3946291203 learned_addr learned my addr 172.21.15.5:0/3946291203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bd392640 1 --2- 172.21.15.5:0/3946291203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f40680 0x55ffa8f38900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bdb93640 1 --2- 172.21.15.5:0/3946291203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f3c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bd392640 1 -- 172.21.15.5:0/3946291203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f40680 msgr2=0x55ffa8f38900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bd392640 1 --2- 172.21.15.5:0/3946291203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f40680 0x55ffa8f38900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bd392640 1 -- 172.21.15.5:0/3946291203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 msgr2=0x55ffa8f3c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bd392640 1 --2- 172.21.15.5:0/3946291203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f3c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bd392640 1 -- 172.21.15.5:0/3946291203 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa8f31860 con 0x55ffa8f354a0 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.445+0000 7f52bd392640 1 --2- 172.21.15.5:0/3946291203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f354a0 0x55ffa8f4c510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55ffa8f52ba0 tx=0x7f52a8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c41fce88fba858 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52a37fe640 1 -- 172.21.15.5:0/3946291203 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8019040 con 0x55ffa8f354a0 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52a37fe640 1 -- 172.21.15.5:0/3946291203 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a8005020 con 0x55ffa8f354a0 2024-09-11T22:28:07.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52a37fe640 1 -- 172.21.15.5:0/3946291203 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8004a90 con 0x55ffa8f354a0 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52c21d0740 1 -- 172.21.15.5:0/3946291203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f354a0 msgr2=0x55ffa8f4c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52c21d0740 1 --2- 172.21.15.5:0/3946291203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f354a0 0x55ffa8f4c510 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55ffa8f52ba0 tx=0x7f52a8005f90 comp rx=0 tx=0).stop 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52c21d0740 1 -- 172.21.15.5:0/3946291203 shutdown_connections 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52c21d0740 1 --2- 172.21.15.5:0/3946291203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f40680 0x55ffa8f38900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52c21d0740 1 --2- 172.21.15.5:0/3946291203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f3c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52c21d0740 1 --2- 172.21.15.5:0/3946291203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f354a0 0x55ffa8f4c510 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52c21d0740 1 -- 172.21.15.5:0/3946291203 >> 172.21.15.5:0/3946291203 conn(0x55ffa8f31a30 msgr2=0x55ffa8f30370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52c21d0740 1 -- 172.21.15.5:0/3946291203 shutdown_connections 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.446+0000 7f52c21d0740 1 -- 172.21.15.5:0/3946291203 wait complete. 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52c21d0740 1 -- start start 2024-09-11T22:28:07.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52c21d0740 1 Processor -- start 2024-09-11T22:28:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 0x55ffa8f48680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f48bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52c21d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f39510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52c21d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b0f90 con 0x55ffa8f35dc0 2024-09-11T22:28:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52c21d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81b08c0 con 0x55ffa8f40680 2024-09-11T22:28:07.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52c21d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81adb00 con 0x55ffa8f354a0 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52c21d0740 10 client.0 _ll_register_callbacks cb 0x7f52a8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52bdb93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f48bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52bd392640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 0x55ffa8f48680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52bdb93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f48bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57310/0 (socket says 172.21.15.5:57310) 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52bdb93640 1 -- 172.21.15.5:0/1779308584 learned_addr learned my addr 172.21.15.5:0/1779308584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.447+0000 7f52bd392640 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f39510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52bdb93640 1 -- 172.21.15.5:0/1779308584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 msgr2=0x55ffa8f48680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52bdb93640 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 0x55ffa8f48680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52bdb93640 1 -- 172.21.15.5:0/1779308584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 msgr2=0x55ffa8f39510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52bdb93640 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f39510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52bdb93640 1 -- 172.21.15.5:0/1779308584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa8f39a50 con 0x55ffa8f35dc0 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52bd392640 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f39510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:28:07.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52bdb93640 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f48bc0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f52b4011850 tx=0x7f52b4009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c626a0518b3767cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52a17fa640 1 -- 172.21.15.5:0/1779308584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4029040 con 0x55ffa8f35dc0 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52c21d0740 1 -- 172.21.15.5:0/1779308584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a8005020 con 0x55ffa8f35dc0 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52a17fa640 1 -- 172.21.15.5:0/1779308584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7f52b4023ab0 con 0x55ffa8f35dc0 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52a17fa640 1 -- 172.21.15.5:0/1779308584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f52b4040050 con 0x55ffa8f35dc0 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52c21d0740 10 client.17280 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52c21d0740 1 -- 172.21.15.5:0/1779308584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ffa8fba210 con 0x55ffa8f35dc0 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.448+0000 7f52a17fa640 1 -- 172.21.15.5:0/1779308584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a8005020 con 0x55ffa8f35dc0 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.449+0000 7f52a17fa640 1 -- 172.21.15.5:0/1779308584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f52b4004420 con 0x55ffa8f35dc0 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.449+0000 7f52c21d0740 10 client.17280 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.449+0000 7f52c21d0740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.449+0000 7f52c21d0740 20 client.17280 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:07.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.449+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffa8fba930 0x55ffa8fbcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.449+0000 7f52c21d0740 4 client.17280 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:28:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.449+0000 7f52c21d0740 1 -- 172.21.15.5:0/1779308584 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ffa8f3dab0 con 0x55ffa8fba930 2024-09-11T22:28:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.449+0000 7f52bd392640 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffa8fba930 0x55ffa8fbcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.449+0000 7f52bd392640 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffa8fba930 0x55ffa8fbcd50 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8364bc8687c3b31 server_cookie=781708ce802bfa9e in_seq=0 out_seq=0 2024-09-11T22:28:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.449+0000 7f52a17fa640 10 client.17280 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52a17fa640 1 -- 172.21.15.5:0/1779308584 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ffa8f3dab0 con 0x55ffa8fba930 2024-09-11T22:28:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52a17fa640 10 client.17280 handle_command_reply: tid=0 2024-09-11T22:28:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 client.17280 shutdown 2024-09-11T22:28:07.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 -- 172.21.15.5:0/1779308584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 msgr2=0x55ffa8f48bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f48bc0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f52b4011850 tx=0x7f52b4009a70 comp rx=0 tx=0).stop 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 -- 172.21.15.5:0/1779308584 shutdown_connections 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ffa8fba930 0x55ffa8fbcd50 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ffa8f40680 0x55ffa8f39510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ffa8f35dc0 0x55ffa8f48bc0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52a17fa640 0 client.17280 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 --2- 172.21.15.5:0/1779308584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffa8f354a0 0x55ffa8f48680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 -- 172.21.15.5:0/1779308584 >> 172.21.15.5:0/1779308584 conn(0x55ffa8f31a30 msgr2=0x55ffa8f37710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 -- 172.21.15.5:0/1779308584 shutdown_connections 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 1 -- 172.21.15.5:0/1779308584 wait complete. 2024-09-11T22:28:07.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.450+0000 7f52c21d0740 20 client.17280 trim_cache size 0 max 16384 2024-09-11T22:28:07.467 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:28:07.467 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:28:07.467 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:28:07.467 INFO:teuthology.orchestra.run.smithi005.stdout: "7d38d584-59d3-4c01-a5d2-99f9a8830d5a": { 2024-09-11T22:28:07.467 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:28:07.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "7d38d584-59d3-4c01-a5d2-99f9a8830d5a", 2024-09-11T22:28:07.467 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:28:07.467 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:07.467 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:07.467 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:28:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 -- 172.21.15.5:0/2556223659 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52a40a2cf0 msgr2=0x7f52a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52a40a2cf0 0x7f52a40a51b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f52b00c52a0 tx=0x7f52b0033040 comp rx=0 tx=0).stop 2024-09-11T22:28:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 -- 172.21.15.5:0/2556223659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8106f20 msgr2=0x7f52b8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8106f20 0x7f52b8074f70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f52b0031a20 tx=0x7f52b0031a50 comp rx=0 tx=0).stop 2024-09-11T22:28:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 -- 172.21.15.5:0/2556223659 shutdown_connections 2024-09-11T22:28:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52b8106f20 0x7f52b8074f70 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f52a40a2cf0 0x7f52a40a51b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52b81027a0 0x7f52b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 --2- 172.21.15.5:0/2556223659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52b8101df0 0x7f52b80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 -- 172.21.15.5:0/2556223659 >> 172.21.15.5:0/2556223659 conn(0x7f52b80fd8b0 msgr2=0x7f52b80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 -- 172.21.15.5:0/2556223659 shutdown_connections 2024-09-11T22:28:07.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:07.451+0000 7f52bfdf7640 1 -- 172.21.15.5:0/2556223659 wait complete. 2024-09-11T22:28:07.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:07 smithi005 ceph-mon[30943]: pgmap v182: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-11T22:28:07.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:07 smithi159 ceph-mon[35534]: pgmap v182: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-11T22:28:07.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:07 smithi172 ceph-mon[35461]: pgmap v182: 97 pgs: 97 active+clean; 63 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.7 MiB/s wr, 412 op/s 2024-09-11T22:28:08.273 INFO:tasks.cephfs.filesystem:scrub status for tag:7d38d584-59d3-4c01-a5d2-99f9a8830d5a - {'path': '/', 'tag': '7d38d584-59d3-4c01-a5d2-99f9a8830d5a', 'options': 'recursive,force'} 2024-09-11T22:28:08.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 443684 8.46 MB/sec execute 146 sec latency 13.725 ms 2024-09-11T22:28:09.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 445384 8.44 MB/sec execute 147 sec latency 62.841 ms 2024-09-11T22:28:09.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:09 smithi005 ceph-mon[30943]: pgmap v183: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 420 op/s 2024-09-11T22:28:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:09 smithi159 ceph-mon[35534]: pgmap v183: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 420 op/s 2024-09-11T22:28:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:09 smithi172 ceph-mon[35461]: pgmap v183: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 420 op/s 2024-09-11T22:28:10.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 447412 8.48 MB/sec execute 148 sec latency 125.685 ms 2024-09-11T22:28:11.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 449824 8.50 MB/sec execute 149 sec latency 38.128 ms 2024-09-11T22:28:11.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:11 smithi005 ceph-mon[30943]: pgmap v184: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 420 op/s 2024-09-11T22:28:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:11 smithi159 ceph-mon[35534]: pgmap v184: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 420 op/s 2024-09-11T22:28:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:11 smithi172 ceph-mon[35461]: pgmap v184: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 420 op/s 2024-09-11T22:28:12.322 INFO:tasks.workunit.client.0.smithi005.stdout: 1 451930 8.53 MB/sec execute 150 sec latency 15.509 ms 2024-09-11T22:28:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.833+0000 7fa5e34c9640 1 -- 172.21.15.5:0/264931522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc10e410 msgr2=0x7fa5dc110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.833+0000 7fa5e34c9640 1 --2- 172.21.15.5:0/264931522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc10e410 0x7fa5dc110800 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009ca0 tx=0x7fa5d402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.833+0000 7fa5e34c9640 1 -- 172.21.15.5:0/264931522 shutdown_connections 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.833+0000 7fa5e34c9640 1 --2- 172.21.15.5:0/264931522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc10e410 0x7fa5dc110800 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.833+0000 7fa5e34c9640 1 --2- 172.21.15.5:0/264931522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc101210 0x7fa5dc10ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.833+0000 7fa5e34c9640 1 --2- 172.21.15.5:0/264931522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5dc100860 0x7fa5dc100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.833+0000 7fa5e34c9640 1 -- 172.21.15.5:0/264931522 >> 172.21.15.5:0/264931522 conn(0x7fa5dc077990 msgr2=0x7fa5dc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.833+0000 7fa5e34c9640 1 -- 172.21.15.5:0/264931522 shutdown_connections 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.833+0000 7fa5e34c9640 1 -- 172.21.15.5:0/264931522 wait complete. 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e34c9640 1 Processor -- start 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e34c9640 1 -- start start 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e34c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc100860 0x7fa5dc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e34c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc101210 0x7fa5dc19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e34c9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5dc10e410 0x7fa5dc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e34c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc114080 con 0x7fa5dc100860 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e34c9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc113f00 con 0x7fa5dc10e410 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e34c9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc114200 con 0x7fa5dc101210 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e1265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc100860 0x7fa5dc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e1265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc100860 0x7fa5dc19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58350/0 (socket says 172.21.15.5:58350) 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e1265640 1 -- 172.21.15.5:0/2759141141 learned_addr learned my addr 172.21.15.5:0/2759141141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.834+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5dc10e410 0x7fa5dc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5e0a64640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc101210 0x7fa5dc19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5e1265640 1 -- 172.21.15.5:0/2759141141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc101210 msgr2=0x7fa5dc19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5e1265640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc101210 0x7fa5dc19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5e1265640 1 -- 172.21.15.5:0/2759141141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5dc10e410 msgr2=0x7fa5dc19edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5e1265640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5dc10e410 0x7fa5dc19edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5e1265640 1 -- 172.21.15.5:0/2759141141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5dc19f4e0 con 0x7fa5dc100860 2024-09-11T22:28:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5e1265640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc100860 0x7fa5dc19a4f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e9f0 tx=0x7fa5d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5d27fc640 1 -- 172.21.15.5:0/2759141141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d800cd00 con 0x7fa5dc100860 2024-09-11T22:28:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5d27fc640 1 -- 172.21.15.5:0/2759141141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d8010910 con 0x7fa5dc100860 2024-09-11T22:28:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5e34c9640 1 -- 172.21.15.5:0/2759141141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5dc19f7d0 con 0x7fa5dc100860 2024-09-11T22:28:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5d27fc640 1 -- 172.21.15.5:0/2759141141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d80189c0 con 0x7fa5dc100860 2024-09-11T22:28:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.835+0000 7fa5e34c9640 1 -- 172.21.15.5:0/2759141141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5dc1a7060 con 0x7fa5dc100860 2024-09-11T22:28:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e625f740 1 Processor -- start 2024-09-11T22:28:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e625f740 1 -- start start 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5d27fc640 1 -- 172.21.15.5:0/2759141141 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa5d8010ac0 con 0x7fa5dc100860 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc8f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc8fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc9010a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e625f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5563fc8582f0 con 0x5563fc8f4410 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e625f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5563fc800270 con 0x5563fc8fecb0 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e625f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5563fc8542b0 con 0x5563fc8f3af0 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e1265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc8fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e1a66640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc8f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e1a66640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc8f3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45228/0 (socket says 172.21.15.5:45228) 2024-09-11T22:28:12.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.838+0000 7fa5e1a66640 1 -- 172.21.15.5:0/3801771476 learned_addr learned my addr 172.21.15.5:0/3801771476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5e0a64640 1 --2- 172.21.15.5:0/3801771476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc9010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5e1a66640 1 -- 172.21.15.5:0/3801771476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 msgr2=0x5563fc9010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/3801771476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc9010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5e1a66640 1 -- 172.21.15.5:0/3801771476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 msgr2=0x5563fc8fe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/3801771476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc8fe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5e1a66640 1 -- 172.21.15.5:0/3801771476 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563fc7ff340 con 0x5563fc8f3af0 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/3801771476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc8f3ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009ca0 tx=0x7fa5d4009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=841d38532447140d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5d27fc640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa5c80a2df0 0x7fa5c80a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5c77fe640 1 -- 172.21.15.5:0/3801771476 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4031db0 con 0x5563fc8f3af0 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5e1265640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa5c80a2df0 0x7fa5c80a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5c77fe640 1 -- 172.21.15.5:0/3801771476 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d4036030 con 0x5563fc8f3af0 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5c77fe640 1 -- 172.21.15.5:0/3801771476 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4004330 con 0x5563fc8f3af0 2024-09-11T22:28:12.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.839+0000 7fa5d27fc640 1 -- 172.21.15.5:0/2759141141 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fa5d80cf760 con 0x7fa5dc100860 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.840+0000 7fa5e625f740 1 -- 172.21.15.5:0/3801771476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 msgr2=0x5563fc8f3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.840+0000 7fa5e1265640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa5c80a2df0 0x7fa5c80a52b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5d80ccf20 tx=0x7fa5d80d5550 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.840+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3801771476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc8f3ed0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009ca0 tx=0x7fa5d4009800 comp rx=0 tx=0).stop 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.840+0000 7fa5e625f740 1 -- 172.21.15.5:0/3801771476 shutdown_connections 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.840+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3801771476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc9010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.840+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3801771476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc8fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.840+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3801771476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc8f3ed0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.840+0000 7fa5e625f740 1 -- 172.21.15.5:0/3801771476 >> 172.21.15.5:0/3801771476 conn(0x5563fc8f0060 msgr2=0x5563fc8f0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.841+0000 7fa5e625f740 1 -- 172.21.15.5:0/3801771476 shutdown_connections 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.841+0000 7fa5e625f740 1 -- 172.21.15.5:0/3801771476 wait complete. 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.841+0000 7fa5e625f740 1 -- start start 2024-09-11T22:28:12.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.841+0000 7fa5e625f740 1 Processor -- start 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.841+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f3af0 0x5563fc8fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.841+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f4410 0x5563fc8fc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.841+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc8fc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.841+0000 7fa5e625f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5563fc901700 con 0x5563fc8f3af0 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.841+0000 7fa5e625f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5563fc7e1290 con 0x5563fc8fecb0 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.841+0000 7fa5e625f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5563fc901880 con 0x5563fc8f4410 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e625f740 10 client.0 _ll_register_callbacks cb 0x5563fc812140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e1a66640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f3af0 0x5563fc8fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e0a64640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f4410 0x5563fc8fc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e1a66640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f3af0 0x5563fc8fbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58356/0 (socket says 172.21.15.5:58356) 2024-09-11T22:28:12.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e1a66640 1 -- 172.21.15.5:0/3327303519 learned_addr learned my addr 172.21.15.5:0/3327303519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc8fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e0a64640 1 -- 172.21.15.5:0/3327303519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 msgr2=0x5563fc8fc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e0a64640 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc8fc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e0a64640 1 -- 172.21.15.5:0/3327303519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f3af0 msgr2=0x5563fc8fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e0a64640 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f3af0 0x5563fc8fbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e0a64640 1 -- 172.21.15.5:0/3327303519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563fc56b210 con 0x5563fc8f4410 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e0a64640 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f4410 0x5563fc8fc0e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc102610 tx=0x7fa5cc012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2731c178ca6341d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/3327303519 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc004050 con 0x5563fc8f4410 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5e625f740 1 -- 172.21.15.5:0/3327303519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5d4036030 con 0x5563fc8f4410 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/3327303519 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fa5cc0041f0 con 0x5563fc8f4410 2024-09-11T22:28:12.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.842+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/3327303519 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa5cc0043f0 con 0x5563fc8f4410 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5e625f740 10 client.26626 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5e625f740 1 -- 172.21.15.5:0/3327303519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5563fc637620 con 0x5563fc8f4410 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/3327303519 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5d4036030 con 0x5563fc8f4410 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/3327303519 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fa5cc003be0 con 0x5563fc8f4410 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5e625f740 10 client.26626 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5e625f740 10 client.26626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5e625f740 20 client.26626 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5563fc90eae0 0x5563fc910ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5e625f740 4 client.26626 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5e625f740 1 -- 172.21.15.5:0/3327303519 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563fc865a20 con 0x5563fc90eae0 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5563fc90eae0 0x5563fc910ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5563fc90eae0 0x5563fc910ef0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dc224143295ae13 server_cookie=d977edef3b7190e in_seq=0 out_seq=0 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.843+0000 7fa5c4ff9640 10 client.26626 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/3327303519 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563fc865a20 con 0x5563fc90eae0 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5c4ff9640 10 client.26626 handle_command_reply: tid=0 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5e625f740 1 client.26626 shutdown 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5e625f740 1 -- 172.21.15.5:0/3327303519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f4410 msgr2=0x5563fc8fc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f4410 0x5563fc8fc0e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc102610 tx=0x7fa5cc012b60 comp rx=0 tx=0).stop 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5e625f740 1 -- 172.21.15.5:0/3327303519 shutdown_connections 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5563fc90eae0 0x5563fc910ef0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc8fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5c4ff9640 0 client.26626 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f4410 0x5563fc8fc0e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3327303519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f3af0 0x5563fc8fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.844+0000 7fa5e625f740 1 -- 172.21.15.5:0/3327303519 >> 172.21.15.5:0/3327303519 conn(0x5563fc8f0060 msgr2=0x5563fc8f5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.845+0000 7fa5e625f740 1 -- 172.21.15.5:0/3327303519 shutdown_connections 2024-09-11T22:28:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.845+0000 7fa5e625f740 1 -- 172.21.15.5:0/3327303519 wait complete. 2024-09-11T22:28:12.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.845+0000 7fa5e625f740 20 client.26626 trim_cache size 0 max 16384 2024-09-11T22:28:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e625f740 1 Processor -- start 2024-09-11T22:28:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e625f740 1 -- start start 2024-09-11T22:28:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8f3af0 0x5563fc8fc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc916800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8fecb0 0x5563fc916d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e625f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc1afb20 con 0x5563fc8f4410 2024-09-11T22:28:12.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e625f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc1b1660 con 0x5563fc8f3af0 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e625f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc1b01f0 con 0x5563fc8fecb0 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e0a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc916800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e1a66640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8fecb0 0x5563fc916d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e0a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc916800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58372/0 (socket says 172.21.15.5:58372) 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e0a64640 1 -- 172.21.15.5:0/3659542395 learned_addr learned my addr 172.21.15.5:0/3659542395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/3659542395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8f3af0 0x5563fc8fc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e0a64640 1 -- 172.21.15.5:0/3659542395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8fecb0 msgr2=0x5563fc916d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e0a64640 1 --2- 172.21.15.5:0/3659542395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8fecb0 0x5563fc916d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e0a64640 1 -- 172.21.15.5:0/3659542395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8f3af0 msgr2=0x5563fc8fc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e0a64640 1 --2- 172.21.15.5:0/3659542395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8f3af0 0x5563fc8fc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e0a64640 1 -- 172.21.15.5:0/3659542395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563fc911570 con 0x5563fc8f4410 2024-09-11T22:28:12.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.861+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/3659542395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8fecb0 0x5563fc916d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.862+0000 7fa5e0a64640 1 --2- 172.21.15.5:0/3659542395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc916800 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc000c00 tx=0x7fa5cc0109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990e998cb6cdf24d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.862+0000 7fa5c6ffd640 1 -- 172.21.15.5:0/3659542395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc031070 con 0x5563fc8f4410 2024-09-11T22:28:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.862+0000 7fa5c6ffd640 1 -- 172.21.15.5:0/3659542395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5cc004000 con 0x5563fc8f4410 2024-09-11T22:28:12.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.862+0000 7fa5c6ffd640 1 -- 172.21.15.5:0/3659542395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc028040 con 0x5563fc8f4410 2024-09-11T22:28:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 -- 172.21.15.5:0/3659542395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 msgr2=0x5563fc916800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3659542395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc916800 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc000c00 tx=0x7fa5cc0109e0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 -- 172.21.15.5:0/3659542395 shutdown_connections 2024-09-11T22:28:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3659542395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8fecb0 0x5563fc916d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3659542395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc916800 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 --2- 172.21.15.5:0/3659542395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8f3af0 0x5563fc8fc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 -- 172.21.15.5:0/3659542395 >> 172.21.15.5:0/3659542395 conn(0x5563fc8f0060 msgr2=0x5563fc8fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 -- 172.21.15.5:0/3659542395 shutdown_connections 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 -- 172.21.15.5:0/3659542395 wait complete. 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 -- start start 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.863+0000 7fa5e625f740 1 Processor -- start 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc906d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc907240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e625f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc903ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e625f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc1b0f90 con 0x5563fc8f4410 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e625f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc1b08c0 con 0x5563fc8fecb0 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e625f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5dc1adb00 con 0x5563fc8f3af0 2024-09-11T22:28:12.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e625f740 10 client.0 _ll_register_callbacks cb 0x7fa5cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e1a66640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc906d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e0a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc907240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e1a66640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc906d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45238/0 (socket says 172.21.15.5:45238) 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e1a66640 1 -- 172.21.15.5:0/2943498715 learned_addr learned my addr 172.21.15.5:0/2943498715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc903ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e1a66640 1 -- 172.21.15.5:0/2943498715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 msgr2=0x5563fc903ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc903ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e1a66640 1 -- 172.21.15.5:0/2943498715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 msgr2=0x5563fc907240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc907240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.864+0000 7fa5e1a66640 1 -- 172.21.15.5:0/2943498715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563fc8fe730 con 0x5563fc8f3af0 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5e1a66640 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc906d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4006010 tx=0x7fa5d4047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef88283d69f6e21d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:28:12.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/2943498715 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4064060 con 0x5563fc8f3af0 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/2943498715 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fa5d402f610 con 0x5563fc8f3af0 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5e625f740 1 -- 172.21.15.5:0/2943498715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5cc004000 con 0x5563fc8f3af0 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/2943498715 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa5d4032ec0 con 0x5563fc8f3af0 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5e625f740 10 client.26632 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5e625f740 1 -- 172.21.15.5:0/2943498715 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5563fc8f0cf0 con 0x5563fc8f3af0 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/2943498715 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5cc004000 con 0x5563fc8f3af0 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/2943498715 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fa5d4050040 con 0x5563fc8f3af0 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5e625f740 10 client.26632 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5e625f740 10 client.26632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5e625f740 20 client.26632 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5e625f740 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5563fc982990 0x5563fc984d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:12.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5e625f740 4 client.26632 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:28:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.865+0000 7fa5e625f740 1 -- 172.21.15.5:0/2943498715 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563fc85fd70 con 0x5563fc982990 2024-09-11T22:28:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.866+0000 7fa5e0a64640 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5563fc982990 0x5563fc984d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.866+0000 7fa5c4ff9640 10 client.26632 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:12.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:12.866+0000 7fa5e0a64640 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5563fc982990 0x5563fc984d80 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1369c6a2bd73e4 server_cookie=dab255867953fd32 in_seq=0 out_seq=0 2024-09-11T22:28:13.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 454272 8.56 MB/sec execute 151 sec latency 53.570 ms 2024-09-11T22:28:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:13 smithi005 ceph-mon[30943]: pgmap v185: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 475 op/s 2024-09-11T22:28:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:13 smithi159 ceph-mon[35534]: pgmap v185: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 475 op/s 2024-09-11T22:28:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5c4ff9640 1 -- 172.21.15.5:0/2943498715 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563fc85fd70 con 0x5563fc982990 2024-09-11T22:28:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5c4ff9640 10 client.26632 handle_command_reply: tid=0 2024-09-11T22:28:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 client.26632 shutdown 2024-09-11T22:28:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 -- 172.21.15.5:0/2943498715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 msgr2=0x5563fc906d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc906d00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4006010 tx=0x7fa5d4047040 comp rx=0 tx=0).stop 2024-09-11T22:28:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 -- 172.21.15.5:0/2943498715 shutdown_connections 2024-09-11T22:28:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5563fc982990 0x5563fc984d80 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5563fc8fecb0 0x5563fc903ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5c4ff9640 0 client.26632 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563fc8f4410 0x5563fc907240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 --2- 172.21.15.5:0/2943498715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5563fc8f3af0 0x5563fc906d00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 -- 172.21.15.5:0/2943498715 >> 172.21.15.5:0/2943498715 conn(0x5563fc8f0060 msgr2=0x5563fc8f5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 -- 172.21.15.5:0/2943498715 shutdown_connections 2024-09-11T22:28:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 1 -- 172.21.15.5:0/2943498715 wait complete. 2024-09-11T22:28:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.846+0000 7fa5e625f740 20 client.26632 trim_cache size 0 max 16384 2024-09-11T22:28:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:28:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:28:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:28:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 -- 172.21.15.5:0/2759141141 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa5c80a2df0 msgr2=0x7fa5c80a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa5c80a2df0 0x7fa5c80a52b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa5d80ccf20 tx=0x7fa5d80d5550 comp rx=0 tx=0).stop 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 -- 172.21.15.5:0/2759141141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc100860 msgr2=0x7fa5dc19a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc100860 0x7fa5dc19a4f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e9f0 tx=0x7fa5d800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 -- 172.21.15.5:0/2759141141 shutdown_connections 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa5c80a2df0 0x7fa5c80a52b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5dc10e410 0x7fa5dc19edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5dc101210 0x7fa5dc19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 --2- 172.21.15.5:0/2759141141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc100860 0x7fa5dc19a4f0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 -- 172.21.15.5:0/2759141141 >> 172.21.15.5:0/2759141141 conn(0x7fa5dc077990 msgr2=0x7fa5dc0ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 -- 172.21.15.5:0/2759141141 shutdown_connections 2024-09-11T22:28:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:13.847+0000 7fa5e34c9640 1 -- 172.21.15.5:0/2759141141 wait complete. 2024-09-11T22:28:13.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:13 smithi172 ceph-mon[35461]: pgmap v185: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 475 op/s 2024-09-11T22:28:14.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 455742 8.54 MB/sec execute 152 sec latency 61.716 ms 2024-09-11T22:28:14.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:14 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:28:14.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:14 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:28:14.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:14 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:28:14.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:14 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:28:14.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:14 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:28:14.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:14 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:28:15.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 458133 8.58 MB/sec execute 153 sec latency 13.990 ms 2024-09-11T22:28:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:15 smithi005 ceph-mon[30943]: pgmap v186: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2024-09-11T22:28:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:15 smithi159 ceph-mon[35534]: pgmap v186: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2024-09-11T22:28:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:15 smithi172 ceph-mon[35461]: pgmap v186: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2024-09-11T22:28:16.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 459772 8.69 MB/sec execute 154 sec latency 57.425 ms 2024-09-11T22:28:17.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 462296 8.73 MB/sec execute 155 sec latency 13.624 ms 2024-09-11T22:28:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:17 smithi159 ceph-mon[35534]: pgmap v187: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2024-09-11T22:28:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:17 smithi172 ceph-mon[35461]: pgmap v187: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2024-09-11T22:28:18.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:17 smithi005 ceph-mon[30943]: pgmap v187: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 221 op/s 2024-09-11T22:28:18.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 464498 8.72 MB/sec execute 156 sec latency 20.847 ms 2024-09-11T22:28:19.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 466581 8.75 MB/sec execute 157 sec latency 47.992 ms 2024-09-11T22:28:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:19 smithi159 ceph-mon[35534]: pgmap v188: 97 pgs: 97 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 424 op/s 2024-09-11T22:28:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:19 smithi172 ceph-mon[35461]: pgmap v188: 97 pgs: 97 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 424 op/s 2024-09-11T22:28:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:19 smithi005 ceph-mon[30943]: pgmap v188: 97 pgs: 97 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 424 op/s 2024-09-11T22:28:20.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 469201 8.80 MB/sec execute 158 sec latency 32.237 ms 2024-09-11T22:28:20.579 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-11T22:28:20.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:28:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:21.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 471177 8.78 MB/sec execute 159 sec latency 51.443 ms 2024-09-11T22:28:21.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.406+0000 7ff637c62640 1 -- 172.21.15.5:0/737963910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff630112640 msgr2=0x7ff630112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:21.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.406+0000 7ff637c62640 1 --2- 172.21.15.5:0/737963910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff630112640 0x7ff630112a20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009890 tx=0x7ff62c02a380 comp rx=0 tx=0).stop 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.406+0000 7ff637c62640 1 -- 172.21.15.5:0/737963910 shutdown_connections 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.406+0000 7ff637c62640 1 --2- 172.21.15.5:0/737963910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63010cf30 0x7ff630119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.406+0000 7ff637c62640 1 --2- 172.21.15.5:0/737963910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff63010c590 0x7ff63010c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.406+0000 7ff637c62640 1 --2- 172.21.15.5:0/737963910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff630112640 0x7ff630112a20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.406+0000 7ff637c62640 1 -- 172.21.15.5:0/737963910 >> 172.21.15.5:0/737963910 conn(0x7ff63006cdc0 msgr2=0x7ff63006d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.406+0000 7ff637c62640 1 -- 172.21.15.5:0/737963910 shutdown_connections 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.406+0000 7ff637c62640 1 -- 172.21.15.5:0/737963910 wait complete. 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff637c62640 1 Processor -- start 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff637c62640 1 -- start start 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff637c62640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63010c590 0x7ff6301a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:21.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff637c62640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63010cf30 0x7ff6301a3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff637c62640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff630112640 0x7ff6301a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff637c62640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff63011c9a0 con 0x7ff63010c590 2024-09-11T22:28:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff637c62640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff63011c820 con 0x7ff630112640 2024-09-11T22:28:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff637c62640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff63011cb20 con 0x7ff63010cf30 2024-09-11T22:28:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff6359fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63010c590 0x7ff6301a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff6359fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63010c590 0x7ff6301a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51744/0 (socket says 172.21.15.5:51744) 2024-09-11T22:28:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff6359fe640 1 -- 172.21.15.5:0/2589499988 learned_addr learned my addr 172.21.15.5:0/2589499988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff6361ff640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff630112640 0x7ff6301a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:21.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff6359fe640 1 -- 172.21.15.5:0/2589499988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63010cf30 msgr2=0x7ff6301a3360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:28:21.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff6359fe640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63010cf30 0x7ff6301a3360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:21.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff6359fe640 1 -- 172.21.15.5:0/2589499988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff630112640 msgr2=0x7ff6301a76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:21.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff6359fe640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff630112640 0x7ff6301a76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:21.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.407+0000 7ff6359fe640 1 -- 172.21.15.5:0/2589499988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6301a7dd0 con 0x7ff63010c590 2024-09-11T22:28:21.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.408+0000 7ff6359fe640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63010c590 0x7ff6301a2e20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009860 tx=0x7ff62c033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:21.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.408+0000 7ff626ffd640 1 -- 172.21.15.5:0/2589499988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c03f020 con 0x7ff63010c590 2024-09-11T22:28:21.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.408+0000 7ff637c62640 1 -- 172.21.15.5:0/2589499988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6301a8060 con 0x7ff63010c590 2024-09-11T22:28:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.408+0000 7ff637c62640 1 -- 172.21.15.5:0/2589499988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6301af9a0 con 0x7ff63010c590 2024-09-11T22:28:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.408+0000 7ff626ffd640 1 -- 172.21.15.5:0/2589499988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff62c032770 con 0x7ff63010c590 2024-09-11T22:28:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.408+0000 7ff626ffd640 1 -- 172.21.15.5:0/2589499988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c037040 con 0x7ff63010c590 2024-09-11T22:28:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.409+0000 7ff637c62640 1 -- 172.21.15.5:0/2589499988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff63010dcb0 con 0x7ff63010c590 2024-09-11T22:28:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.410+0000 7ff626ffd640 1 -- 172.21.15.5:0/2589499988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff62c042020 con 0x7ff63010c590 2024-09-11T22:28:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.411+0000 7ff626ffd640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff61c0a2cf0 0x7ff61c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:21.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.411+0000 7ff626ffd640 1 -- 172.21.15.5:0/2589499988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7ff62c0eedc0 con 0x7ff63010c590 2024-09-11T22:28:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.411+0000 7ff6351fd640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff61c0a2cf0 0x7ff61c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.412+0000 7ff6351fd640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff61c0a2cf0 0x7ff61c0a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff63006d330 tx=0x7ff620005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:21.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.416+0000 7ff626ffd640 1 -- 172.21.15.5:0/2589499988 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff62c049030 con 0x7ff63010c590 2024-09-11T22:28:21.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.646+0000 7ff637c62640 1 -- 172.21.15.5:0/2589499988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff630114a40 con 0x7ff63010c590 2024-09-11T22:28:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.647+0000 7ff626ffd640 1 -- 172.21.15.5:0/2589499988 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11967 (secure 0 0 0) 0x7ff62c0bbab0 con 0x7ff63010c590 2024-09-11T22:28:21.649 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:28:21.649 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-11T22:28:00:557115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:28:00.557102+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26542":{"gid":26542,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1394880869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1394880869},{"type":"v1","addr":"172.21.15.159:6833","nonce":1394880869}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:28:21.650 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-11T22:28:21.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 -- 172.21.15.5:0/2589499988 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff61c0a2cf0 msgr2=0x7ff61c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:21.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff61c0a2cf0 0x7ff61c0a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff63006d330 tx=0x7ff620005f90 comp rx=0 tx=0).stop 2024-09-11T22:28:21.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 -- 172.21.15.5:0/2589499988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63010c590 msgr2=0x7ff6301a2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63010c590 0x7ff6301a2e20 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff62c009860 tx=0x7ff62c033d10 comp rx=0 tx=0).stop 2024-09-11T22:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 -- 172.21.15.5:0/2589499988 shutdown_connections 2024-09-11T22:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff61c0a2cf0 0x7ff61c0a51b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff630112640 0x7ff6301a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff63010cf30 0x7ff6301a3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 --2- 172.21.15.5:0/2589499988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63010c590 0x7ff6301a2e20 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 -- 172.21.15.5:0/2589499988 >> 172.21.15.5:0/2589499988 conn(0x7ff63006cdc0 msgr2=0x7ff63010aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 -- 172.21.15.5:0/2589499988 shutdown_connections 2024-09-11T22:28:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:21.649+0000 7ff637c62640 1 -- 172.21.15.5:0/2589499988 wait complete. 2024-09-11T22:28:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:21 smithi159 ceph-mon[35534]: pgmap v189: 97 pgs: 97 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-09-11T22:28:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:21 smithi172 ceph-mon[35461]: pgmap v189: 97 pgs: 97 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-09-11T22:28:22.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:21 smithi005 ceph-mon[30943]: pgmap v189: 97 pgs: 97 active+clean; 68 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-09-11T22:28:22.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 473442 8.82 MB/sec execute 160 sec latency 11.468 ms 2024-09-11T22:28:22.509 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-11T22:28:22.509 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-11T22:28:22.774 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:22.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:22 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2589499988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:28:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:22 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2589499988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:28:23.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:22 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2589499988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:28:23.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476036 8.87 MB/sec execute 161 sec latency 14.671 ms 2024-09-11T22:28:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.316+0000 7efddb939640 1 -- 172.21.15.5:0/754906876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd4102db0 msgr2=0x7efdd4103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.316+0000 7efddb939640 1 --2- 172.21.15.5:0/754906876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd4102db0 0x7efdd4103190 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009930 tx=0x7efdc802a420 comp rx=0 tx=0).stop 2024-09-11T22:28:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 -- 172.21.15.5:0/754906876 shutdown_connections 2024-09-11T22:28:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 --2- 172.21.15.5:0/754906876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdd41082e0 0x7efdd410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 --2- 172.21.15.5:0/754906876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd4103760 0x7efdd4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 --2- 172.21.15.5:0/754906876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd4102db0 0x7efdd4103190 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 -- 172.21.15.5:0/754906876 >> 172.21.15.5:0/754906876 conn(0x7efdd40fc7d0 msgr2=0x7efdd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 -- 172.21.15.5:0/754906876 shutdown_connections 2024-09-11T22:28:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 -- 172.21.15.5:0/754906876 wait complete. 2024-09-11T22:28:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 Processor -- start 2024-09-11T22:28:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 -- start start 2024-09-11T22:28:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdd4102db0 0x7efdd4100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.317+0000 7efddb939640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd4103760 0x7efdd4100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efddb939640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd41082e0 0x7efdd4101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efddb939640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdd4112100 con 0x7efdd4103760 2024-09-11T22:28:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efddb939640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efdd4111f80 con 0x7efdd4102db0 2024-09-11T22:28:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efddb939640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efdd4112280 con 0x7efdd41082e0 2024-09-11T22:28:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd8ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd4103760 0x7efdd4100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd96d5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdd4102db0 0x7efdd4100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd8ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd4103760 0x7efdd4100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51760/0 (socket says 172.21.15.5:51760) 2024-09-11T22:28:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd96d5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdd4102db0 0x7efdd4100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42300/0 (socket says 172.21.15.5:42300) 2024-09-11T22:28:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd8ed4640 1 -- 172.21.15.5:0/3198031334 learned_addr learned my addr 172.21.15.5:0/3198031334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd41082e0 0x7efdd4101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd8ed4640 1 -- 172.21.15.5:0/3198031334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd41082e0 msgr2=0x7efdd4101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd41082e0 0x7efdd4101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd8ed4640 1 -- 172.21.15.5:0/3198031334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdd4102db0 msgr2=0x7efdd4100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdd4102db0 0x7efdd4100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd8ed4640 1 -- 172.21.15.5:0/3198031334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd41a6ab0 con 0x7efdd4103760 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd41082e0 0x7efdd4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd96d5640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdd4102db0 0x7efdd4100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd4103760 0x7efdd4100f50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7efdd000d750 tx=0x7efdd000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.318+0000 7efdc67fc640 1 -- 172.21.15.5:0/3198031334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0017070 con 0x7efdd4103760 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.319+0000 7efddb939640 1 -- 172.21.15.5:0/3198031334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd41a6da0 con 0x7efdd4103760 2024-09-11T22:28:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.319+0000 7efdc67fc640 1 -- 172.21.15.5:0/3198031334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd0005500 con 0x7efdd4103760 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.319+0000 7efdc67fc640 1 -- 172.21.15.5:0/3198031334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0004e00 con 0x7efdd4103760 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.319+0000 7efddb939640 1 -- 172.21.15.5:0/3198031334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd41a7250 con 0x7efdd4103760 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.321+0000 7efdc67fc640 1 -- 172.21.15.5:0/3198031334 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7efdd0004fa0 con 0x7efdd4103760 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdddd12740 1 Processor -- start 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdddd12740 1 -- start start 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdddd12740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fc9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdddd12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fca080 0x55f795fd4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdddd12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fd4940 0x55f795fd6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdddd12740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f795f2f2a0 con 0x55f795fc9760 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdddd12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f795ed6b10 con 0x55f795fca080 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdddd12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f795f2b260 con 0x55f795fd4940 2024-09-11T22:28:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd9ed6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fc9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd9ed6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fc9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51762/0 (socket says 172.21.15.5:51762) 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd9ed6640 1 -- 172.21.15.5:0/3481080162 learned_addr learned my addr 172.21.15.5:0/3481080162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd96d5640 1 --2- 172.21.15.5:0/3481080162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fca080 0x55f795fd4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/3481080162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fd4940 0x55f795fd6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdc67fc640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdc00a2cf0 0x7efdc00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd9ed6640 1 -- 172.21.15.5:0/3481080162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fd4940 msgr2=0x55f795fd6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/3481080162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fd4940 0x55f795fd6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd9ed6640 1 -- 172.21.15.5:0/3481080162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fca080 msgr2=0x55f795fd4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/3481080162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fca080 0x55f795fd4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd9ed6640 1 -- 172.21.15.5:0/3481080162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f795ca0b90 con 0x55f795fc9760 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.322+0000 7efdd96d5640 1 --2- 172.21.15.5:0/3481080162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fca080 0x55f795fd4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.323+0000 7efdc67fc640 1 -- 172.21.15.5:0/3198031334 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7efdd00cede0 con 0x7efdd4103760 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.323+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdc00a2cf0 0x7efdc00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.323+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/3481080162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fc9b40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7efdcc011790 tx=0x7efdcc011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a3d07d730bde4c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.323+0000 7efdbf7fe640 1 -- 172.21.15.5:0/3481080162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc00f9a0 con 0x55f795fc9760 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.323+0000 7efdbf7fe640 1 -- 172.21.15.5:0/3481080162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdcc00fc80 con 0x55f795fc9760 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.323+0000 7efdbf7fe640 1 -- 172.21.15.5:0/3481080162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc01cbc0 con 0x55f795fc9760 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 -- 172.21.15.5:0/3481080162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 msgr2=0x55f795fc9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdc00a2cf0 0x7efdc00a51b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efdcc020000 tx=0x7efdcc021000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 --2- 172.21.15.5:0/3481080162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fc9b40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7efdcc011790 tx=0x7efdcc011c50 comp rx=0 tx=0).stop 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 -- 172.21.15.5:0/3481080162 shutdown_connections 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 --2- 172.21.15.5:0/3481080162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fd4940 0x55f795fd6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 --2- 172.21.15.5:0/3481080162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fca080 0x55f795fd4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 --2- 172.21.15.5:0/3481080162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fc9b40 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 -- 172.21.15.5:0/3481080162 >> 172.21.15.5:0/3481080162 conn(0x55f795fc5cf0 msgr2=0x55f795fc60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 -- 172.21.15.5:0/3481080162 shutdown_connections 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 -- 172.21.15.5:0/3481080162 wait complete. 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 -- start start 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.324+0000 7efdddd12740 1 Processor -- start 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdddd12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fc9760 0x55f795fd1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdddd12740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fd1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdddd12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 0x55f795fd2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdddd12740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f795fd7510 con 0x55f795fca080 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdddd12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f795eb76e0 con 0x55f795fd4940 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdddd12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f795fd7690 con 0x55f795fc9760 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdddd12740 10 client.0 _ll_register_callbacks cb 0x55f795edba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd8ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fd1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd9ed6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 0x55f795fd2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd8ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fd1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51768/0 (socket says 172.21.15.5:51768) 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd8ed4640 1 -- 172.21.15.5:0/178434242 learned_addr learned my addr 172.21.15.5:0/178434242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd96d5640 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fc9760 0x55f795fd1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd9ed6640 1 -- 172.21.15.5:0/178434242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fc9760 msgr2=0x55f795fd1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fc9760 0x55f795fd1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd9ed6640 1 -- 172.21.15.5:0/178434242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 msgr2=0x55f795fd1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fd1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd9ed6640 1 -- 172.21.15.5:0/178434242 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f795fe0980 con 0x55f795fd4940 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.325+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fd1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 0x55f795fd2200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efdcc011790 tx=0x7efdcc004910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ed0b64575b29c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdbcff9640 1 -- 172.21.15.5:0/178434242 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc036060 con 0x55f795fd4940 2024-09-11T22:28:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdddd12740 1 -- 172.21.15.5:0/178434242 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdcc00fc80 con 0x55f795fd4940 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdbcff9640 1 -- 172.21.15.5:0/178434242 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7efdcc02e070 con 0x55f795fd4940 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdbcff9640 1 -- 172.21.15.5:0/178434242 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efdcc00fe30 con 0x55f795fd4940 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdddd12740 10 client.26720 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdddd12740 1 -- 172.21.15.5:0/178434242 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f795fe32e0 con 0x55f795fd4940 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdbcff9640 1 -- 172.21.15.5:0/178434242 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdcc00fc80 con 0x55f795fd4940 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdbcff9640 1 -- 172.21.15.5:0/178434242 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7efdcc005050 con 0x55f795fd4940 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdddd12740 10 client.26720 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdddd12740 10 client.26720 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.326+0000 7efdddd12740 20 client.26720 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.327+0000 7efdddd12740 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f795fe4630 0x55f795fe6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.327+0000 7efdddd12740 4 client.26720 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.327+0000 7efdddd12740 1 -- 172.21.15.5:0/178434242 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f795f3c9d0 con 0x55f795fe4630 2024-09-11T22:28:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.328+0000 7efdd96d5640 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f795fe4630 0x55f795fe6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.332+0000 7efdbcff9640 10 client.26720 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.333+0000 7efdd96d5640 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f795fe4630 0x55f795fe6a20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b28be1f531127427 server_cookie=8e9c4fadb5f72e7a in_seq=0 out_seq=0 2024-09-11T22:28:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdbcff9640 1 -- 172.21.15.5:0/178434242 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f795f3c9d0 con 0x55f795fe4630 2024-09-11T22:28:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdbcff9640 10 client.26720 handle_command_reply: tid=0 2024-09-11T22:28:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 client.26720 shutdown 2024-09-11T22:28:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 -- 172.21.15.5:0/178434242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 msgr2=0x55f795fd2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 0x55f795fd2200 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efdcc011790 tx=0x7efdcc004910 comp rx=0 tx=0).stop 2024-09-11T22:28:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 -- 172.21.15.5:0/178434242 shutdown_connections 2024-09-11T22:28:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f795fe4630 0x55f795fe6a20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 0x55f795fd2200 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fd1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 --2- 172.21.15.5:0/178434242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fc9760 0x55f795fd1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdbcff9640 0 client.26720 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 -- 172.21.15.5:0/178434242 >> 172.21.15.5:0/178434242 conn(0x55f795fc5cf0 msgr2=0x55f795fc4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 -- 172.21.15.5:0/178434242 shutdown_connections 2024-09-11T22:28:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.335+0000 7efdddd12740 1 -- 172.21.15.5:0/178434242 wait complete. 2024-09-11T22:28:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.336+0000 7efdddd12740 20 client.26720 trim_cache size 0 max 16384 2024-09-11T22:28:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdddd12740 1 Processor -- start 2024-09-11T22:28:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdddd12740 1 -- start start 2024-09-11T22:28:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdddd12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fc9760 0x55f795fd3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdddd12740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdddd12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fd4940 0x55f795fec720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdddd12740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41afcf0 con 0x55f795fca080 2024-09-11T22:28:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdddd12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41b1830 con 0x55f795fc9760 2024-09-11T22:28:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdddd12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41b03c0 con 0x55f795fd4940 2024-09-11T22:28:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdd8ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdd8ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fec1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51776/0 (socket says 172.21.15.5:51776) 2024-09-11T22:28:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdd8ed4640 1 -- 172.21.15.5:0/2371981554 learned_addr learned my addr 172.21.15.5:0/2371981554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/2371981554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fd4940 0x55f795fec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.354+0000 7efdd96d5640 1 --2- 172.21.15.5:0/2371981554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fc9760 0x55f795fd3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.355+0000 7efdd8ed4640 1 -- 172.21.15.5:0/2371981554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fd4940 msgr2=0x55f795fec720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.355+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/2371981554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fd4940 0x55f795fec720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.355+0000 7efdd8ed4640 1 -- 172.21.15.5:0/2371981554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fc9760 msgr2=0x55f795fd3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.355+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/2371981554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fc9760 0x55f795fd3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.355+0000 7efdd8ed4640 1 -- 172.21.15.5:0/2371981554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f795fe6fb0 con 0x55f795fca080 2024-09-11T22:28:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.355+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/2371981554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fec1e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7efdd0097ce0 tx=0x7efdd0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55dbeeec5cff341 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.355+0000 7efdbeffd640 1 -- 172.21.15.5:0/2371981554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0020040 con 0x55f795fca080 2024-09-11T22:28:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.355+0000 7efdbeffd640 1 -- 172.21.15.5:0/2371981554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd009f610 con 0x55f795fca080 2024-09-11T22:28:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.355+0000 7efdbeffd640 1 -- 172.21.15.5:0/2371981554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0097980 con 0x55f795fca080 2024-09-11T22:28:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 -- 172.21.15.5:0/2371981554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 msgr2=0x55f795fec1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 --2- 172.21.15.5:0/2371981554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fec1e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7efdd0097ce0 tx=0x7efdd0009c30 comp rx=0 tx=0).stop 2024-09-11T22:28:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 -- 172.21.15.5:0/2371981554 shutdown_connections 2024-09-11T22:28:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 --2- 172.21.15.5:0/2371981554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fd4940 0x55f795fec720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 --2- 172.21.15.5:0/2371981554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fca080 0x55f795fec1e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 --2- 172.21.15.5:0/2371981554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fc9760 0x55f795fd3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 -- 172.21.15.5:0/2371981554 >> 172.21.15.5:0/2371981554 conn(0x55f795fc5cf0 msgr2=0x55f795fc4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 -- 172.21.15.5:0/2371981554 shutdown_connections 2024-09-11T22:28:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 -- 172.21.15.5:0/2371981554 wait complete. 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 -- start start 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.356+0000 7efdddd12740 1 Processor -- start 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdddd12740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fdcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdddd12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fca080 0x55f795fdd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdddd12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 0x55f795fd7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdddd12740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41b1160 con 0x55f795fc9760 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdddd12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41b0a90 con 0x55f795fd4940 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdddd12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efdd41adcd0 con 0x55f795fca080 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdddd12740 10 client.0 _ll_register_callbacks cb 0x7efdcc011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd96d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fdcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd96d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fdcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51792/0 (socket says 172.21.15.5:51792) 2024-09-11T22:28:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd96d5640 1 -- 172.21.15.5:0/3902436652 learned_addr learned my addr 172.21.15.5:0/3902436652 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fca080 0x55f795fdd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd9ed6640 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 0x55f795fd7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd96d5640 1 -- 172.21.15.5:0/3902436652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fca080 msgr2=0x55f795fdd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd96d5640 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fca080 0x55f795fdd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd96d5640 1 -- 172.21.15.5:0/3902436652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 msgr2=0x55f795fd7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd96d5640 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 0x55f795fd7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd96d5640 1 -- 172.21.15.5:0/3902436652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f795fc6980 con 0x55f795fc9760 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdd96d5640 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fdcca0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efdc8004650 tx=0x7efdc8005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bd0e787b0f2884 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.357+0000 7efdbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdbcff9640 1 -- 172.21.15.5:0/3902436652 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8035040 con 0x55f795fc9760 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdbcff9640 1 -- 172.21.15.5:0/3902436652 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7efdc8004b70 con 0x55f795fc9760 2024-09-11T22:28:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdddd12740 1 -- 172.21.15.5:0/3902436652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd009f610 con 0x55f795fc9760 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdbcff9640 1 -- 172.21.15.5:0/3902436652 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efdc8004d70 con 0x55f795fc9760 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdddd12740 10 client.17322 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdddd12740 1 -- 172.21.15.5:0/3902436652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f795fc1260 con 0x55f795fc9760 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdbcff9640 1 -- 172.21.15.5:0/3902436652 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd009f610 con 0x55f795fc9760 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdbcff9640 1 -- 172.21.15.5:0/3902436652 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7efdc8031640 con 0x55f795fc9760 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdddd12740 10 client.17322 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdddd12740 10 client.17322 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdddd12740 20 client.17322 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdddd12740 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f796050250 0x55f796052670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdddd12740 4 client.17322 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdddd12740 1 -- 172.21.15.5:0/3902436652 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x55f795f36d20 con 0x55f796050250 2024-09-11T22:28:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.358+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f796050250 0x55f796052670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.359+0000 7efdd8ed4640 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f796050250 0x55f796052670 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29bc0b434a1f3879 server_cookie=eb5358d52c907017 in_seq=0 out_seq=0 2024-09-11T22:28:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.359+0000 7efdbcff9640 10 client.17322 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.519+0000 7efdbcff9640 1 -- 172.21.15.5:0/3902436652 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f795f36d20 con 0x55f796050250 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.519+0000 7efdbcff9640 10 client.17322 handle_command_reply: tid=0 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.519+0000 7efdddd12740 1 client.17322 shutdown 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.519+0000 7efdddd12740 1 -- 172.21.15.5:0/3902436652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 msgr2=0x55f795fdcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.519+0000 7efdddd12740 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fdcca0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efdc8004650 tx=0x7efdc8005500 comp rx=0 tx=0).stop 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efdddd12740 1 -- 172.21.15.5:0/3902436652 shutdown_connections 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efdddd12740 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f796050250 0x55f796052670 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efdddd12740 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f795fd4940 0x55f795fd7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efdddd12740 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f795fca080 0x55f795fdd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efdddd12740 1 --2- 172.21.15.5:0/3902436652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f795fc9760 0x55f795fdcca0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efdddd12740 1 -- 172.21.15.5:0/3902436652 >> 172.21.15.5:0/3902436652 conn(0x55f795fc5cf0 msgr2=0x55f795fd1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:23.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efdbcff9640 0 client.17322 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:23.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efdddd12740 1 -- 172.21.15.5:0/3902436652 shutdown_connections 2024-09-11T22:28:23.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efdddd12740 1 -- 172.21.15.5:0/3902436652 wait complete. 2024-09-11T22:28:23.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efdddd12740 20 client.17322 trim_cache size 0 max 16384 2024-09-11T22:28:23.522 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:28:23.522 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-11T22:28:23.522 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:28:23.522 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 22, 2024-09-11T22:28:23.522 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "e5aba961": { 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 22, 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.1, 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:28:23.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:28:23.524 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:28:23.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:23.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:23.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:23.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:23.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:23.524 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:28:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efddb939640 1 -- 172.21.15.5:0/3198031334 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdc00a2cf0 msgr2=0x7efdc00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efddb939640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdc00a2cf0 0x7efdc00a51b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efdcc020000 tx=0x7efdcc021000 comp rx=0 tx=0).stop 2024-09-11T22:28:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efddb939640 1 -- 172.21.15.5:0/3198031334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd4103760 msgr2=0x7efdd4100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.520+0000 7efddb939640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd4103760 0x7efdd4100f50 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7efdd000d750 tx=0x7efdd000dc10 comp rx=0 tx=0).stop 2024-09-11T22:28:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.521+0000 7efddb939640 1 -- 172.21.15.5:0/3198031334 shutdown_connections 2024-09-11T22:28:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.521+0000 7efddb939640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd41082e0 0x7efdd4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.521+0000 7efddb939640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd4103760 0x7efdd4100f50 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.521+0000 7efddb939640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdc00a2cf0 0x7efdc00a51b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.521+0000 7efddb939640 1 --2- 172.21.15.5:0/3198031334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efdd4102db0 0x7efdd4100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.521+0000 7efddb939640 1 -- 172.21.15.5:0/3198031334 >> 172.21.15.5:0/3198031334 conn(0x7efdd40fc7d0 msgr2=0x7efdd4106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.521+0000 7efddb939640 1 -- 172.21.15.5:0/3198031334 shutdown_connections 2024-09-11T22:28:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:23.521+0000 7efddb939640 1 -- 172.21.15.5:0/3198031334 wait complete. 2024-09-11T22:28:23.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:23 smithi005 ceph-mon[30943]: pgmap v190: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 421 op/s 2024-09-11T22:28:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:23 smithi172 ceph-mon[35461]: pgmap v190: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 421 op/s 2024-09-11T22:28:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:23 smithi159 ceph-mon[35534]: pgmap v190: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.6 MiB/s wr, 421 op/s 2024-09-11T22:28:24.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.81 MB/sec execute 162 sec latency 808.981 ms 2024-09-11T22:28:24.565 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e5aba961', quiesce took 4.0 seconds. Will release after: 16.1 2024-09-11T22:28:25.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.76 MB/sec execute 163 sec latency 1809.099 ms 2024-09-11T22:28:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:25 smithi172 ceph-mon[35461]: pgmap v191: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 316 op/s 2024-09-11T22:28:26.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:25 smithi005 ceph-mon[30943]: pgmap v191: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 316 op/s 2024-09-11T22:28:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:25 smithi159 ceph-mon[35534]: pgmap v191: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 316 op/s 2024-09-11T22:28:26.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.71 MB/sec execute 164 sec latency 2809.184 ms 2024-09-11T22:28:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.947+0000 7fbd79731640 1 -- 172.21.15.5:0/1861047522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd74102cc0 msgr2=0x7fbd741030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.947+0000 7fbd79731640 1 --2- 172.21.15.5:0/1861047522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd74102cc0 0x7fbd741030a0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009900 tx=0x7fbd6402a480 comp rx=0 tx=0).stop 2024-09-11T22:28:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.948+0000 7fbd79731640 1 -- 172.21.15.5:0/1861047522 shutdown_connections 2024-09-11T22:28:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.948+0000 7fbd79731640 1 --2- 172.21.15.5:0/1861047522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd7410c110 0x7fbd7410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.948+0000 7fbd79731640 1 --2- 172.21.15.5:0/1861047522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd741035e0 0x7fbd7410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.948+0000 7fbd79731640 1 --2- 172.21.15.5:0/1861047522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd74102cc0 0x7fbd741030a0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.948+0000 7fbd79731640 1 -- 172.21.15.5:0/1861047522 >> 172.21.15.5:0/1861047522 conn(0x7fbd740fc6b0 msgr2=0x7fbd740fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.948+0000 7fbd79731640 1 -- 172.21.15.5:0/1861047522 shutdown_connections 2024-09-11T22:28:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.948+0000 7fbd79731640 1 -- 172.21.15.5:0/1861047522 wait complete. 2024-09-11T22:28:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.948+0000 7fbd79731640 1 Processor -- start 2024-09-11T22:28:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.948+0000 7fbd79731640 1 -- start start 2024-09-11T22:28:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd79731640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd74102cc0 0x7fbd74198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd79731640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd741035e0 0x7fbd741988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd79731640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd7410c110 0x7fbd7419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd79731640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74111ef0 con 0x7fbd741035e0 2024-09-11T22:28:26.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd79731640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74111d70 con 0x7fbd7410c110 2024-09-11T22:28:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd79731640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74112070 con 0x7fbd74102cc0 2024-09-11T22:28:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd741035e0 0x7fbd741988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd741035e0 0x7fbd741988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51804/0 (socket says 172.21.15.5:51804) 2024-09-11T22:28:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd727fc640 1 -- 172.21.15.5:0/2034625341 learned_addr learned my addr 172.21.15.5:0/2034625341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd72ffd640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd74102cc0 0x7fbd74198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd737fe640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd7410c110 0x7fbd7419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd727fc640 1 -- 172.21.15.5:0/2034625341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd74102cc0 msgr2=0x7fbd74198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd727fc640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd74102cc0 0x7fbd74198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd727fc640 1 -- 172.21.15.5:0/2034625341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd7410c110 msgr2=0x7fbd7419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd727fc640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd7410c110 0x7fbd7419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd727fc640 1 -- 172.21.15.5:0/2034625341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7419d360 con 0x7fbd741035e0 2024-09-11T22:28:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd737fe640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd7410c110 0x7fbd7419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:28:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.949+0000 7fbd727fc640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd741035e0 0x7fbd741988b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800d740 tx=0x7fbd6800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.950+0000 7fbd5bfff640 1 -- 172.21.15.5:0/2034625341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68017070 con 0x7fbd741035e0 2024-09-11T22:28:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.950+0000 7fbd5bfff640 1 -- 172.21.15.5:0/2034625341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6800de50 con 0x7fbd741035e0 2024-09-11T22:28:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.950+0000 7fbd79731640 1 -- 172.21.15.5:0/2034625341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7419d650 con 0x7fbd741035e0 2024-09-11T22:28:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.950+0000 7fbd5bfff640 1 -- 172.21.15.5:0/2034625341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6800f620 con 0x7fbd741035e0 2024-09-11T22:28:26.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.950+0000 7fbd79731640 1 -- 172.21.15.5:0/2034625341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd741a4ee0 con 0x7fbd741035e0 2024-09-11T22:28:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.952+0000 7fbd5bfff640 1 -- 172.21.15.5:0/2034625341 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbd68009c30 con 0x7fbd741035e0 2024-09-11T22:28:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.952+0000 7fbd7c4cc740 1 Processor -- start 2024-09-11T22:28:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.952+0000 7fbd7c4cc740 1 -- start start 2024-09-11T22:28:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 0x5578c74cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cc410 0x5578c74d6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5578c74302f0 con 0x5578c74d6cb0 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5578c73d8270 con 0x5578c74cbaf0 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578c742c2b0 con 0x5578c74cc410 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51806/0 (socket says 172.21.15.5:51806) 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd727fc640 1 -- 172.21.15.5:0/413446693 learned_addr learned my addr 172.21.15.5:0/413446693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd72ffd640 1 --2- 172.21.15.5:0/413446693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cc410 0x5578c74d6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd5bfff640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd5c0a2da0 0x7fbd5c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd737fe640 1 --2- 172.21.15.5:0/413446693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 0x5578c74cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd737fe640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd5c0a2da0 0x7fbd5c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd727fc640 1 -- 172.21.15.5:0/413446693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cc410 msgr2=0x5578c74d6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd727fc640 1 --2- 172.21.15.5:0/413446693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cc410 0x5578c74d6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd727fc640 1 -- 172.21.15.5:0/413446693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 msgr2=0x5578c74cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd727fc640 1 --2- 172.21.15.5:0/413446693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 0x5578c74cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.953+0000 7fbd5bfff640 1 -- 172.21.15.5:0/2034625341 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fbd680cf050 con 0x7fbd741035e0 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.954+0000 7fbd727fc640 1 -- 172.21.15.5:0/413446693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578c73d7340 con 0x5578c74d6cb0 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.954+0000 7fbd727fc640 1 --2- 172.21.15.5:0/413446693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d90a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbd6809b4c0 tx=0x7fbd68000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d9b377cfc4ff1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.954+0000 7fbd597fa640 1 -- 172.21.15.5:0/413446693 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68020020 con 0x5578c74d6cb0 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.954+0000 7fbd737fe640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd5c0a2da0 0x7fbd5c0a5260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c005c40 tx=0x7fbd6c009280 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.954+0000 7fbd597fa640 1 -- 172.21.15.5:0/413446693 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6809b530 con 0x5578c74d6cb0 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.954+0000 7fbd597fa640 1 -- 172.21.15.5:0/413446693 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680bb070 con 0x5578c74d6cb0 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/413446693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 msgr2=0x5578c74d90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/413446693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d90a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbd6809b4c0 tx=0x7fbd68000bc0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/413446693 shutdown_connections 2024-09-11T22:28:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/413446693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d90a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/413446693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cc410 0x5578c74d6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/413446693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 0x5578c74cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/413446693 >> 172.21.15.5:0/413446693 conn(0x5578c74c8060 msgr2=0x5578c74c8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/413446693 shutdown_connections 2024-09-11T22:28:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/413446693 wait complete. 2024-09-11T22:28:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 -- start start 2024-09-11T22:28:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 Processor -- start 2024-09-11T22:28:26.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cbaf0 0x5578c74d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74d6cb0 0x5578c74d45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5578c74d9820 con 0x5578c74cc410 2024-09-11T22:28:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5578c73b9290 con 0x5578c74d6cb0 2024-09-11T22:28:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5578c74d99a0 con 0x5578c74cbaf0 2024-09-11T22:28:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.955+0000 7fbd7c4cc740 10 client.0 _ll_register_callbacks cb 0x5578c73ea140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51814/0 (socket says 172.21.15.5:51814) 2024-09-11T22:28:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd727fc640 1 -- 172.21.15.5:0/1925668739 learned_addr learned my addr 172.21.15.5:0/1925668739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:26.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd72ffd640 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cbaf0 0x5578c74d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd737fe640 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74d6cb0 0x5578c74d45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd727fc640 1 -- 172.21.15.5:0/1925668739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cbaf0 msgr2=0x5578c74d3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd727fc640 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cbaf0 0x5578c74d3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd727fc640 1 -- 172.21.15.5:0/1925668739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74d6cb0 msgr2=0x5578c74d45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd727fc640 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74d6cb0 0x5578c74d45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd727fc640 1 -- 172.21.15.5:0/1925668739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578c7143210 con 0x5578c74cc410 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd727fc640 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d40a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbd68097990 tx=0x7fbd680a41a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a98b33133da87c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd56ff5640 1 -- 172.21.15.5:0/1925668739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68020050 con 0x5578c74cc410 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd56ff5640 1 -- 172.21.15.5:0/1925668739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fbd680cc9e0 con 0x5578c74cc410 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/1925668739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6809b530 con 0x5578c74cc410 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd56ff5640 1 -- 172.21.15.5:0/1925668739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbd680bb040 con 0x5578c74cc410 2024-09-11T22:28:26.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd7c4cc740 10 client.17334 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.956+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/1925668739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5578c720f620 con 0x5578c74cc410 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.957+0000 7fbd56ff5640 1 -- 172.21.15.5:0/1925668739 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6809b530 con 0x5578c74cc410 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.957+0000 7fbd56ff5640 1 -- 172.21.15.5:0/1925668739 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbd6809a980 con 0x5578c74cc410 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.957+0000 7fbd7c4cc740 10 client.17334 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.957+0000 7fbd7c4cc740 10 client.17334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.957+0000 7fbd7c4cc740 20 client.17334 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.957+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5578c74e69f0 0x5578c74e8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.957+0000 7fbd7c4cc740 4 client.17334 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.957+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/1925668739 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578c743da20 con 0x5578c74e69f0 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.957+0000 7fbd72ffd640 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5578c74e69f0 0x5578c74e8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.958+0000 7fbd56ff5640 10 client.17334 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:26.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.958+0000 7fbd72ffd640 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5578c74e69f0 0x5578c74e8de0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b10a46e7731cb4 server_cookie=1e097ef10ab4a6c6 in_seq=0 out_seq=0 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.958+0000 7fbd56ff5640 1 -- 172.21.15.5:0/1925668739 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578c743da20 con 0x5578c74e69f0 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.958+0000 7fbd56ff5640 10 client.17334 handle_command_reply: tid=0 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.958+0000 7fbd7c4cc740 1 client.17334 shutdown 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.958+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/1925668739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 msgr2=0x5578c74d40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.958+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d40a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbd68097990 tx=0x7fbd680a41a0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.959+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/1925668739 shutdown_connections 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.959+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5578c74e69f0 0x5578c74e8de0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.959+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74d6cb0 0x5578c74d45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.959+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d40a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.959+0000 7fbd56ff5640 0 client.17334 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.959+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/1925668739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cbaf0 0x5578c74d3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.959+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/1925668739 >> 172.21.15.5:0/1925668739 conn(0x5578c74c8060 msgr2=0x5578c74cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:26.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.959+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/1925668739 shutdown_connections 2024-09-11T22:28:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.959+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/1925668739 wait complete. 2024-09-11T22:28:26.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.959+0000 7fbd7c4cc740 20 client.17334 trim_cache size 0 max 16384 2024-09-11T22:28:26.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd7c4cc740 1 Processor -- start 2024-09-11T22:28:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd7c4cc740 1 -- start start 2024-09-11T22:28:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 0x5578c74d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74d6cb0 0x5578c74ee3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741ad970 con 0x5578c74cc410 2024-09-11T22:28:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741af4b0 con 0x5578c74cbaf0 2024-09-11T22:28:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741ae040 con 0x5578c74d6cb0 2024-09-11T22:28:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51820/0 (socket says 172.21.15.5:51820) 2024-09-11T22:28:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd727fc640 1 -- 172.21.15.5:0/2994717139 learned_addr learned my addr 172.21.15.5:0/2994717139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd72ffd640 1 --2- 172.21.15.5:0/2994717139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 0x5578c74d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd727fc640 1 -- 172.21.15.5:0/2994717139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74d6cb0 msgr2=0x5578c74ee3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd737fe640 1 --2- 172.21.15.5:0/2994717139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74d6cb0 0x5578c74ee3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd727fc640 1 --2- 172.21.15.5:0/2994717139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74d6cb0 0x5578c74ee3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd727fc640 1 -- 172.21.15.5:0/2994717139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 msgr2=0x5578c74d5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd727fc640 1 --2- 172.21.15.5:0/2994717139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 0x5578c74d5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd727fc640 1 -- 172.21.15.5:0/2994717139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578c74d9990 con 0x5578c74cc410 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.974+0000 7fbd72ffd640 1 --2- 172.21.15.5:0/2994717139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 0x5578c74d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd727fc640 1 --2- 172.21.15.5:0/2994717139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d4410 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbd68097130 tx=0x7fbd68099a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9961bd8b607f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd58ff9640 1 -- 172.21.15.5:0/2994717139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68020070 con 0x5578c74cc410 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd58ff9640 1 -- 172.21.15.5:0/2994717139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6809b530 con 0x5578c74cc410 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd58ff9640 1 -- 172.21.15.5:0/2994717139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680a3d10 con 0x5578c74cc410 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2994717139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 msgr2=0x5578c74d4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:26.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/2994717139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d4410 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fbd68097130 tx=0x7fbd68099a40 comp rx=0 tx=0).stop 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2994717139 shutdown_connections 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/2994717139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74d6cb0 0x5578c74ee3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/2994717139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74cc410 0x5578c74d4410 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/2994717139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cbaf0 0x5578c74d5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2994717139 >> 172.21.15.5:0/2994717139 conn(0x5578c74c8060 msgr2=0x5578c74d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.975+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2994717139 shutdown_connections 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2994717139 wait complete. 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd7c4cc740 1 -- start start 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd7c4cc740 1 Processor -- start 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cbaf0 0x5578c74df0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cc410 0x5578c74d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd7c4cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741aede0 con 0x5578c74d6cb0 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741ae710 con 0x5578c74cc410 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd7c4cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741ab950 con 0x5578c74cbaf0 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd7c4cc740 10 client.0 _ll_register_callbacks cb 0x7fbd64005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51832/0 (socket says 172.21.15.5:51832) 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd737fe640 1 -- 172.21.15.5:0/2498492379 learned_addr learned my addr 172.21.15.5:0/2498492379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd72ffd640 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cbaf0 0x5578c74df0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.976+0000 7fbd727fc640 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cc410 0x5578c74d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd737fe640 1 -- 172.21.15.5:0/2498492379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cbaf0 msgr2=0x5578c74df0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd737fe640 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cbaf0 0x5578c74df0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd737fe640 1 -- 172.21.15.5:0/2498492379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cc410 msgr2=0x5578c74d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:26.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd737fe640 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cc410 0x5578c74d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd737fe640 1 -- 172.21.15.5:0/2498492379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578c74c7e90 con 0x5578c74d6cb0 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd737fe640 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d0af0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c010970 tx=0x7fbd6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5561f5e3cf261e16 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd56ff5640 1 -- 172.21.15.5:0/2498492379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c021990 con 0x5578c74d6cb0 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd56ff5640 1 -- 172.21.15.5:0/2498492379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fbd6c021b30 con 0x5578c74d6cb0 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2498492379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6809b530 con 0x5578c74d6cb0 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd56ff5640 1 -- 172.21.15.5:0/2498492379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbd6c021d30 con 0x5578c74d6cb0 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd7c4cc740 10 client.17340 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2498492379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5578c7550eb0 con 0x5578c74d6cb0 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.977+0000 7fbd56ff5640 1 -- 172.21.15.5:0/2498492379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6809b530 con 0x5578c74d6cb0 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.978+0000 7fbd56ff5640 1 -- 172.21.15.5:0/2498492379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbd6c030030 con 0x5578c74d6cb0 2024-09-11T22:28:26.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.978+0000 7fbd7c4cc740 10 client.17340 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.978+0000 7fbd7c4cc740 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.978+0000 7fbd7c4cc740 20 client.17340 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.978+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5578c7551650 0x5578c7553a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.978+0000 7fbd7c4cc740 4 client.17340 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:28:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.978+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2498492379 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578c7437d70 con 0x5578c7551650 2024-09-11T22:28:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.978+0000 7fbd72ffd640 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5578c7551650 0x5578c7553a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.978+0000 7fbd72ffd640 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5578c7551650 0x5578c7553a70 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ee420f40ad0c0a5 server_cookie=ed076c7350a83928 in_seq=0 out_seq=0 2024-09-11T22:28:26.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:26.979+0000 7fbd56ff5640 10 client.17340 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd56ff5640 1 -- 172.21.15.5:0/2498492379 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578c7437d70 con 0x5578c7551650 2024-09-11T22:28:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd56ff5640 10 client.17340 handle_command_reply: tid=0 2024-09-11T22:28:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 client.17340 shutdown 2024-09-11T22:28:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2498492379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 msgr2=0x5578c74d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d0af0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c010970 tx=0x7fbd6c004a40 comp rx=0 tx=0).stop 2024-09-11T22:28:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2498492379 shutdown_connections 2024-09-11T22:28:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5578c7551650 0x5578c7553a70 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578c74d6cb0 0x5578c74d0af0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd56ff5640 0 client.17340 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5578c74cc410 0x5578c74d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 --2- 172.21.15.5:0/2498492379 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5578c74cbaf0 0x5578c74df0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2498492379 >> 172.21.15.5:0/2498492379 conn(0x5578c74c8060 msgr2=0x5578c74d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2498492379 shutdown_connections 2024-09-11T22:28:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.116+0000 7fbd7c4cc740 1 -- 172.21.15.5:0/2498492379 wait complete. 2024-09-11T22:28:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd7c4cc740 20 client.17340 trim_cache size 0 max 16384 2024-09-11T22:28:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:28:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:28:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:28:27.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:28:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd79731640 1 -- 172.21.15.5:0/2034625341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd5c0a2da0 msgr2=0x7fbd5c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd79731640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd5c0a2da0 0x7fbd5c0a5260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c005c40 tx=0x7fbd6c009280 comp rx=0 tx=0).stop 2024-09-11T22:28:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd79731640 1 -- 172.21.15.5:0/2034625341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd741035e0 msgr2=0x7fbd741988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd79731640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd741035e0 0x7fbd741988b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800d740 tx=0x7fbd6800dc00 comp rx=0 tx=0).stop 2024-09-11T22:28:27.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd79731640 1 -- 172.21.15.5:0/2034625341 shutdown_connections 2024-09-11T22:28:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd79731640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd5c0a2da0 0x7fbd5c0a5260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd79731640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd7410c110 0x7fbd7419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd79731640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd741035e0 0x7fbd741988b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd79731640 1 --2- 172.21.15.5:0/2034625341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd74102cc0 0x7fbd74198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.117+0000 7fbd79731640 1 -- 172.21.15.5:0/2034625341 >> 172.21.15.5:0/2034625341 conn(0x7fbd740fc6b0 msgr2=0x7fbd740fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.118+0000 7fbd79731640 1 -- 172.21.15.5:0/2034625341 shutdown_connections 2024-09-11T22:28:27.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:27.118+0000 7fbd79731640 1 -- 172.21.15.5:0/2034625341 wait complete. 2024-09-11T22:28:27.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.65 MB/sec execute 165 sec latency 3809.230 ms 2024-09-11T22:28:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:27 smithi172 ceph-mon[35461]: pgmap v192: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 316 op/s 2024-09-11T22:28:28.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:27 smithi005 ceph-mon[30943]: pgmap v192: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 316 op/s 2024-09-11T22:28:28.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:27 smithi159 ceph-mon[35534]: pgmap v192: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 316 op/s 2024-09-11T22:28:28.323 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.60 MB/sec execute 166 sec latency 4809.301 ms 2024-09-11T22:28:29.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.55 MB/sec execute 167 sec latency 5809.336 ms 2024-09-11T22:28:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:29 smithi172 ceph-mon[35461]: pgmap v193: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 366 op/s 2024-09-11T22:28:29.881 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:29 smithi159 ceph-mon[35534]: pgmap v193: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 366 op/s 2024-09-11T22:28:30.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:29 smithi005 ceph-mon[30943]: pgmap v193: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 366 op/s 2024-09-11T22:28:30.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.50 MB/sec execute 168 sec latency 6809.370 ms 2024-09-11T22:28:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:30 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:28:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:30 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:28:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:30 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:28:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:30 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:28:31.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:30 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:28:31.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:30 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:28:31.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:30 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:28:31.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:30 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:28:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:30 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:28:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:30 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:28:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:30 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:28:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:30 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:28:31.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.45 MB/sec execute 169 sec latency 7809.406 ms 2024-09-11T22:28:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:31 smithi005 ceph-mon[30943]: pgmap v194: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 3.2 MiB/s wr, 163 op/s 2024-09-11T22:28:31.872 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:31 smithi172 ceph-mon[35461]: pgmap v194: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 3.2 MiB/s wr, 163 op/s 2024-09-11T22:28:32.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:31 smithi159 ceph-mon[35534]: pgmap v194: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 3.2 MiB/s wr, 163 op/s 2024-09-11T22:28:32.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.40 MB/sec execute 170 sec latency 8809.507 ms 2024-09-11T22:28:33.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.35 MB/sec execute 171 sec latency 9809.643 ms 2024-09-11T22:28:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:33 smithi172 ceph-mon[35461]: pgmap v195: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 4.0 MiB/s wr, 291 op/s 2024-09-11T22:28:34.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:33 smithi005 ceph-mon[30943]: pgmap v195: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 4.0 MiB/s wr, 291 op/s 2024-09-11T22:28:34.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:33 smithi159 ceph-mon[35534]: pgmap v195: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 4.0 MiB/s wr, 291 op/s 2024-09-11T22:28:34.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.30 MB/sec execute 172 sec latency 10809.785 ms 2024-09-11T22:28:35.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.25 MB/sec execute 173 sec latency 11809.910 ms 2024-09-11T22:28:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:35 smithi172 ceph-mon[35461]: pgmap v196: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-11T22:28:36.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:35 smithi005 ceph-mon[30943]: pgmap v196: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-11T22:28:36.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:35 smithi159 ceph-mon[35534]: pgmap v196: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-11T22:28:36.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.20 MB/sec execute 174 sec latency 12810.051 ms 2024-09-11T22:28:37.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.16 MB/sec execute 175 sec latency 13810.169 ms 2024-09-11T22:28:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:37 smithi172 ceph-mon[35461]: pgmap v197: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-11T22:28:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:37 smithi005 ceph-mon[30943]: pgmap v197: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-11T22:28:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:37 smithi159 ceph-mon[35534]: pgmap v197: 97 pgs: 97 active+clean; 72 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-11T22:28:38.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:28:38.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.11 MB/sec execute 176 sec latency 14810.231 ms 2024-09-11T22:28:38.544 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:39.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.045+0000 7fd596933640 1 -- 172.21.15.5:0/920891710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd590102790 msgr2=0x7fd5901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.045+0000 7fd596933640 1 --2- 172.21.15.5:0/920891710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd590102790 0x7fd5901067e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fd578009900 tx=0x7fd57802a410 comp rx=0 tx=0).stop 2024-09-11T22:28:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 -- 172.21.15.5:0/920891710 shutdown_connections 2024-09-11T22:28:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 --2- 172.21.15.5:0/920891710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd590106f10 0x7fd59010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 --2- 172.21.15.5:0/920891710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd590102790 0x7fd5901067e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 --2- 172.21.15.5:0/920891710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd590101de0 0x7fd5901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 -- 172.21.15.5:0/920891710 >> 172.21.15.5:0/920891710 conn(0x7fd5900fd8c0 msgr2=0x7fd5900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 -- 172.21.15.5:0/920891710 shutdown_connections 2024-09-11T22:28:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 -- 172.21.15.5:0/920891710 wait complete. 2024-09-11T22:28:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 Processor -- start 2024-09-11T22:28:39.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 -- start start 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd590101de0 0x7fd590198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd590102790 0x7fd590198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd590106f10 0x7fd59019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd590111f90 con 0x7fd590106f10 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd590111e10 con 0x7fd590102790 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.046+0000 7fd596933640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd590112110 con 0x7fd590101de0 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd594ed0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd590106f10 0x7fd59019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd594ed0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd590106f10 0x7fd59019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40496/0 (socket says 172.21.15.5:40496) 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd594ed0640 1 -- 172.21.15.5:0/946690688 learned_addr learned my addr 172.21.15.5:0/946690688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd58ffff640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd590101de0 0x7fd590198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd587fff640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd590102790 0x7fd590198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd594ed0640 1 -- 172.21.15.5:0/946690688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd590101de0 msgr2=0x7fd590198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd594ed0640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd590101de0 0x7fd590198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd594ed0640 1 -- 172.21.15.5:0/946690688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd590102790 msgr2=0x7fd590198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd594ed0640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd590102790 0x7fd590198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd594ed0640 1 -- 172.21.15.5:0/946690688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59019d3c0 con 0x7fd590106f10 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd594ed0640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd590106f10 0x7fd59019cce0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd58800cce0 tx=0x7fd58800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd58dffb640 1 -- 172.21.15.5:0/946690688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58800f450 con 0x7fd590106f10 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd58dffb640 1 -- 172.21.15.5:0/946690688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58800fad0 con 0x7fd590106f10 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd596933640 1 -- 172.21.15.5:0/946690688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59019d6b0 con 0x7fd590106f10 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd58dffb640 1 -- 172.21.15.5:0/946690688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588018b60 con 0x7fd590106f10 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.047+0000 7fd596933640 1 -- 172.21.15.5:0/946690688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5901a4f90 con 0x7fd590106f10 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd58dffb640 1 -- 172.21.15.5:0/946690688 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd58800fc80 con 0x7fd590106f10 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd598d0c740 1 Processor -- start 2024-09-11T22:28:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd598d0c740 1 -- start start 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5d8d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d9c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d9ca90 0x55adc5d9ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd598d0c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55adc5cf32e0 con 0x55adc5d8dcc0 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd598d0c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55adc5c9ace0 con 0x55adc5d8d3a0 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd598d0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55adc5cef2a0 con 0x55adc5d9ca90 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd58ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d9c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd58dffb640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd57c0a2d70 0x7fd57c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd58ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d9c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40512/0 (socket says 172.21.15.5:40512) 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.050+0000 7fd58ffff640 1 -- 172.21.15.5:0/977046312 learned_addr learned my addr 172.21.15.5:0/977046312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd594ed0640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd57c0a2d70 0x7fd57c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd587fff640 1 --2- 172.21.15.5:0/977046312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5d8d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd58ffff640 1 -- 172.21.15.5:0/977046312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d9ca90 msgr2=0x55adc5d9ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd58ffff640 1 --2- 172.21.15.5:0/977046312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d9ca90 0x55adc5d9ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd58dffb640 1 -- 172.21.15.5:0/946690688 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fd5880cf940 con 0x7fd590106f10 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd58ffff640 1 -- 172.21.15.5:0/977046312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 msgr2=0x55adc5d8d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd58ffff640 1 --2- 172.21.15.5:0/977046312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5d8d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd58ffff640 1 -- 172.21.15.5:0/977046312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc5c7f440 con 0x55adc5d8dcc0 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd58ffff640 1 --2- 172.21.15.5:0/977046312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d9c550 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd580009a70 tx=0x7fd580009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c233223c4d55955 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd5867fc640 1 -- 172.21.15.5:0/977046312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580017070 con 0x55adc5d8dcc0 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd594ed0640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd57c0a2d70 0x7fd57c0a5230 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd5880cd650 tx=0x7fd58800cf10 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd5867fc640 1 -- 172.21.15.5:0/977046312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd580013470 con 0x55adc5d8dcc0 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.051+0000 7fd5867fc640 1 -- 172.21.15.5:0/977046312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580012630 con 0x55adc5d8dcc0 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 -- 172.21.15.5:0/977046312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 msgr2=0x55adc5d9c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 --2- 172.21.15.5:0/977046312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d9c550 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd580009a70 tx=0x7fd580009f30 comp rx=0 tx=0).stop 2024-09-11T22:28:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 -- 172.21.15.5:0/977046312 shutdown_connections 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 --2- 172.21.15.5:0/977046312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d9ca90 0x55adc5d9ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 --2- 172.21.15.5:0/977046312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d9c550 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 --2- 172.21.15.5:0/977046312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5d8d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 -- 172.21.15.5:0/977046312 >> 172.21.15.5:0/977046312 conn(0x55adc5d89a30 msgr2=0x55adc5d89e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 -- 172.21.15.5:0/977046312 shutdown_connections 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 -- 172.21.15.5:0/977046312 wait complete. 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 -- start start 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.052+0000 7fd598d0c740 1 Processor -- start 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5d9c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d99700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d9ca90 0x55adc5d99cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd598d0c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55adc5d9f6c0 con 0x55adc5d8dcc0 2024-09-11T22:28:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd598d0c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55adc5cae050 con 0x55adc5d8d3a0 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd598d0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55adc5d9f840 con 0x55adc5d9ca90 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd598d0c740 10 client.0 _ll_register_callbacks cb 0x55adc5cee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd58ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d99700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd58ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d99700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40522/0 (socket says 172.21.15.5:40522) 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd58ffff640 1 -- 172.21.15.5:0/94809540 learned_addr learned my addr 172.21.15.5:0/94809540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd587fff640 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d9ca90 0x55adc5d99cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd587fff640 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5d9c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd58ffff640 1 -- 172.21.15.5:0/94809540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d9ca90 msgr2=0x55adc5d99cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd58ffff640 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d9ca90 0x55adc5d99cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd58ffff640 1 -- 172.21.15.5:0/94809540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 msgr2=0x55adc5d9c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd58ffff640 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5d9c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd58ffff640 1 -- 172.21.15.5:0/94809540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc5a643c0 con 0x55adc5d8dcc0 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd587fff640 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5d9c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd58ffff640 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d99700 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd580004810 tx=0x7fd580013cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834c28cd6ce7047a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.053+0000 7fd56f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd56f7fe640 1 -- 172.21.15.5:0/94809540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580017050 con 0x55adc5d8dcc0 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd56f7fe640 1 -- 172.21.15.5:0/94809540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fd58001c450 con 0x55adc5d8dcc0 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd598d0c740 1 -- 172.21.15.5:0/94809540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd580013470 con 0x55adc5d8dcc0 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd56f7fe640 1 -- 172.21.15.5:0/94809540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd58001c650 con 0x55adc5d8dcc0 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd598d0c740 10 client.17352 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd598d0c740 1 -- 172.21.15.5:0/94809540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55adc5dab120 con 0x55adc5d8dcc0 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd56f7fe640 1 -- 172.21.15.5:0/94809540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd580013470 con 0x55adc5d8dcc0 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd56f7fe640 1 -- 172.21.15.5:0/94809540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd580026940 con 0x55adc5d8dcc0 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd598d0c740 10 client.17352 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd598d0c740 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd598d0c740 20 client.17352 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd598d0c740 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55adc5dac520 0x55adc5dae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd598d0c740 4 client.17352 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.054+0000 7fd598d0c740 1 -- 172.21.15.5:0/94809540 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adc5d00850 con 0x55adc5dac520 2024-09-11T22:28:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.055+0000 7fd587fff640 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55adc5dac520 0x55adc5dae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.055+0000 7fd56f7fe640 10 client.17352 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.055+0000 7fd587fff640 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55adc5dac520 0x55adc5dae910 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=198e858d518a5f45 server_cookie=c5c4f097720e6da5 in_seq=0 out_seq=0 2024-09-11T22:28:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd56f7fe640 1 -- 172.21.15.5:0/94809540 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55adc5d00850 con 0x55adc5dac520 2024-09-11T22:28:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd56f7fe640 10 client.17352 handle_command_reply: tid=0 2024-09-11T22:28:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 client.17352 shutdown 2024-09-11T22:28:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 -- 172.21.15.5:0/94809540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 msgr2=0x55adc5d99700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d99700 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fd580004810 tx=0x7fd580013cb0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 -- 172.21.15.5:0/94809540 shutdown_connections 2024-09-11T22:28:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55adc5dac520 0x55adc5dae910 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d9ca90 0x55adc5d99cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d8dcc0 0x55adc5d99700 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 --2- 172.21.15.5:0/94809540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5d9c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 -- 172.21.15.5:0/94809540 >> 172.21.15.5:0/94809540 conn(0x55adc5d89a30 msgr2=0x55adc5d8f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd56f7fe640 0 client.17352 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 -- 172.21.15.5:0/94809540 shutdown_connections 2024-09-11T22:28:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 1 -- 172.21.15.5:0/94809540 wait complete. 2024-09-11T22:28:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.056+0000 7fd598d0c740 20 client.17352 trim_cache size 0 max 16384 2024-09-11T22:28:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.073+0000 7fd598d0c740 1 Processor -- start 2024-09-11T22:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.073+0000 7fd598d0c740 1 -- start start 2024-09-11T22:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.073+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8d3a0 0x55adc5da8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.073+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8dcc0 0x55adc5d98fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5db3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd598d0c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901ab7a0 con 0x55adc5d9ca90 2024-09-11T22:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd598d0c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901ad2e0 con 0x55adc5d8dcc0 2024-09-11T22:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd598d0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901abe70 con 0x55adc5d8d3a0 2024-09-11T22:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd587fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5db3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd587fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5db3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40538/0 (socket says 172.21.15.5:40538) 2024-09-11T22:28:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd587fff640 1 -- 172.21.15.5:0/2568376932 learned_addr learned my addr 172.21.15.5:0/2568376932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd587fff640 1 --2- 172.21.15.5:0/2568376932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8d3a0 0x55adc5da8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd58ffff640 1 --2- 172.21.15.5:0/2568376932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8dcc0 0x55adc5d98fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd587fff640 1 -- 172.21.15.5:0/2568376932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8d3a0 msgr2=0x55adc5da8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd587fff640 1 --2- 172.21.15.5:0/2568376932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8d3a0 0x55adc5da8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd587fff640 1 -- 172.21.15.5:0/2568376932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8dcc0 msgr2=0x55adc5d98fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd587fff640 1 --2- 172.21.15.5:0/2568376932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8dcc0 0x55adc5d98fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd587fff640 1 -- 172.21.15.5:0/2568376932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc5d9c7c0 con 0x55adc5d9ca90 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd587fff640 1 --2- 172.21.15.5:0/2568376932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5db3f60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd57802a930 tx=0x7fd57802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f855e18dbdef1e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd585ffb640 1 -- 172.21.15.5:0/2568376932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578035040 con 0x55adc5d9ca90 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd585ffb640 1 -- 172.21.15.5:0/2568376932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd578004310 con 0x55adc5d9ca90 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.074+0000 7fd585ffb640 1 -- 172.21.15.5:0/2568376932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578005420 con 0x55adc5d9ca90 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.075+0000 7fd598d0c740 1 -- 172.21.15.5:0/2568376932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 msgr2=0x55adc5db3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.075+0000 7fd598d0c740 1 --2- 172.21.15.5:0/2568376932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5db3f60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fd57802a930 tx=0x7fd57802a410 comp rx=0 tx=0).stop 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.075+0000 7fd598d0c740 1 -- 172.21.15.5:0/2568376932 shutdown_connections 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.075+0000 7fd598d0c740 1 --2- 172.21.15.5:0/2568376932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5db3f60 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.075+0000 7fd598d0c740 1 --2- 172.21.15.5:0/2568376932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8dcc0 0x55adc5d98fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.075+0000 7fd598d0c740 1 --2- 172.21.15.5:0/2568376932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8d3a0 0x55adc5da8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.075+0000 7fd598d0c740 1 -- 172.21.15.5:0/2568376932 >> 172.21.15.5:0/2568376932 conn(0x55adc5d89a30 msgr2=0x55adc5d89e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.075+0000 7fd598d0c740 1 -- 172.21.15.5:0/2568376932 shutdown_connections 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.075+0000 7fd598d0c740 1 -- 172.21.15.5:0/2568376932 wait complete. 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.075+0000 7fd598d0c740 1 -- start start 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd598d0c740 1 Processor -- start 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5da4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8dcc0 0x55adc5da51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd598d0c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5da0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd598d0c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901acc10 con 0x55adc5d9ca90 2024-09-11T22:28:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd598d0c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901ac540 con 0x55adc5d8d3a0 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd598d0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd5901a9780 con 0x55adc5d8dcc0 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd598d0c740 10 client.0 _ll_register_callbacks cb 0x7fd5800098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd587fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5da0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd587fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5da0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40552/0 (socket says 172.21.15.5:40552) 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd587fff640 1 -- 172.21.15.5:0/4054601854 learned_addr learned my addr 172.21.15.5:0/4054601854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd58ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8dcc0 0x55adc5da51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.076+0000 7fd587fff640 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5da4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd58ffff640 1 -- 172.21.15.5:0/4054601854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 msgr2=0x55adc5da4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd58ffff640 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5da4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd58ffff640 1 -- 172.21.15.5:0/4054601854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 msgr2=0x55adc5da0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd58ffff640 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5da0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd58ffff640 1 -- 172.21.15.5:0/4054601854 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc5da0890 con 0x55adc5d8dcc0 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd587fff640 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5da0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd58ffff640 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8dcc0 0x55adc5da51e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd580010010 tx=0x7fd58000e6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de420cc40bb3576 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd56f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd56f7fe640 1 -- 172.21.15.5:0/4054601854 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580017070 con 0x55adc5d8dcc0 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd56f7fe640 1 -- 172.21.15.5:0/4054601854 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fd58001c7f0 con 0x55adc5d8dcc0 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd598d0c740 1 -- 172.21.15.5:0/4054601854 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd578004310 con 0x55adc5d8dcc0 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd56f7fe640 1 -- 172.21.15.5:0/4054601854 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd58001c9f0 con 0x55adc5d8dcc0 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd598d0c740 10 client.26680 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd598d0c740 1 -- 172.21.15.5:0/4054601854 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55adc5e16a50 con 0x55adc5d8dcc0 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.077+0000 7fd56f7fe640 1 -- 172.21.15.5:0/4054601854 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd578004310 con 0x55adc5d8dcc0 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.078+0000 7fd56f7fe640 1 -- 172.21.15.5:0/4054601854 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd58002e900 con 0x55adc5d8dcc0 2024-09-11T22:28:39.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.078+0000 7fd598d0c740 10 client.26680 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.078+0000 7fd598d0c740 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.078+0000 7fd598d0c740 20 client.26680 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.078+0000 7fd598d0c740 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55adc5e17140 0x55adc5e19560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.078+0000 7fd598d0c740 4 client.26680 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.078+0000 7fd598d0c740 1 -- 172.21.15.5:0/4054601854 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55adc5cfad60 con 0x55adc5e17140 2024-09-11T22:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.078+0000 7fd587fff640 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55adc5e17140 0x55adc5e19560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.078+0000 7fd587fff640 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55adc5e17140 0x55adc5e19560 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d52a1aca2a05dde server_cookie=7ae819bb4c442904 in_seq=0 out_seq=0 2024-09-11T22:28:39.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.078+0000 7fd56f7fe640 10 client.26680 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd56f7fe640 1 -- 172.21.15.5:0/4054601854 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55adc5cfad60 con 0x55adc5e17140 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd56f7fe640 10 client.26680 handle_command_reply: tid=0 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 client.26680 shutdown 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 -- 172.21.15.5:0/4054601854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8dcc0 msgr2=0x55adc5da51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8dcc0 0x55adc5da51e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd580010010 tx=0x7fd58000e6e0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 -- 172.21.15.5:0/4054601854 shutdown_connections 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55adc5e17140 0x55adc5e19560 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55adc5d9ca90 0x55adc5da0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55adc5d8dcc0 0x55adc5da51e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd56f7fe640 0 client.26680 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 --2- 172.21.15.5:0/4054601854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55adc5d8d3a0 0x55adc5da4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 -- 172.21.15.5:0/4054601854 >> 172.21.15.5:0/4054601854 conn(0x55adc5d89a30 msgr2=0x55adc5d944f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:39.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 -- 172.21.15.5:0/4054601854 shutdown_connections 2024-09-11T22:28:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 1 -- 172.21.15.5:0/4054601854 wait complete. 2024-09-11T22:28:39.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd598d0c740 20 client.26680 trim_cache size 0 max 16384 2024-09-11T22:28:39.088 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:28:39.088 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:28:39.088 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:28:39.088 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd596933640 1 -- 172.21.15.5:0/946690688 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd57c0a2d70 msgr2=0x7fd57c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd596933640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd57c0a2d70 0x7fd57c0a5230 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd5880cd650 tx=0x7fd58800cf10 comp rx=0 tx=0).stop 2024-09-11T22:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd596933640 1 -- 172.21.15.5:0/946690688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd590106f10 msgr2=0x7fd59019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.079+0000 7fd596933640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd590106f10 0x7fd59019cce0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd58800cce0 tx=0x7fd58800a620 comp rx=0 tx=0).stop 2024-09-11T22:28:39.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.080+0000 7fd596933640 1 -- 172.21.15.5:0/946690688 shutdown_connections 2024-09-11T22:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.080+0000 7fd596933640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd590106f10 0x7fd59019cce0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.080+0000 7fd596933640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd57c0a2d70 0x7fd57c0a5230 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.080+0000 7fd596933640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd590102790 0x7fd590198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.080+0000 7fd596933640 1 --2- 172.21.15.5:0/946690688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd590101de0 0x7fd590198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.080+0000 7fd596933640 1 -- 172.21.15.5:0/946690688 >> 172.21.15.5:0/946690688 conn(0x7fd5900fd8c0 msgr2=0x7fd5900fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.080+0000 7fd596933640 1 -- 172.21.15.5:0/946690688 shutdown_connections 2024-09-11T22:28:39.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:39.080+0000 7fd596933640 1 -- 172.21.15.5:0/946690688 wait complete. 2024-09-11T22:28:39.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.07 MB/sec execute 177 sec latency 15810.264 ms 2024-09-11T22:28:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:39 smithi005 ceph-mon[30943]: pgmap v198: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-09-11T22:28:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:39 smithi172 ceph-mon[35461]: pgmap v198: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-09-11T22:28:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:39 smithi159 ceph-mon[35534]: pgmap v198: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-09-11T22:28:40.187 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:28:40.188 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:28:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.218+0000 7fe304f54640 1 -- 172.21.15.5:0/1059453180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe300101380 msgr2=0x7fe30010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.218+0000 7fe304f54640 1 --2- 172.21.15.5:0/1059453180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe300101380 0x7fe30010d8b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009cd0 tx=0x7fe2f802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.218+0000 7fe304f54640 1 -- 172.21.15.5:0/1059453180 shutdown_connections 2024-09-11T22:28:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.218+0000 7fe304f54640 1 --2- 172.21.15.5:0/1059453180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe300101380 0x7fe30010d8b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.218+0000 7fe304f54640 1 --2- 172.21.15.5:0/1059453180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3001009e0 0x7fe300100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.218+0000 7fe304f54640 1 --2- 172.21.15.5:0/1059453180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3001069f0 0x7fe300106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.218+0000 7fe304f54640 1 -- 172.21.15.5:0/1059453180 >> 172.21.15.5:0/1059453180 conn(0x7fe3000fc6a0 msgr2=0x7fe3000feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.218+0000 7fe304f54640 1 -- 172.21.15.5:0/1059453180 shutdown_connections 2024-09-11T22:28:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.218+0000 7fe304f54640 1 -- 172.21.15.5:0/1059453180 wait complete. 2024-09-11T22:28:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.219+0000 7fe304f54640 1 Processor -- start 2024-09-11T22:28:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.219+0000 7fe304f54640 1 -- start start 2024-09-11T22:28:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe304f54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3001009e0 0x7fe300199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe304f54640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe300101380 0x7fe3001996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe304f54640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3001069f0 0x7fe3001a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe304f54640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe300110e30 con 0x7fe3001009e0 2024-09-11T22:28:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe304f54640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe300110cb0 con 0x7fe3001069f0 2024-09-11T22:28:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe304f54640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe300110fb0 con 0x7fe300101380 2024-09-11T22:28:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3001009e0 0x7fe300199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3001009e0 0x7fe300199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40570/0 (socket says 172.21.15.5:40570) 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fe59c640 1 -- 172.21.15.5:0/3043835446 learned_addr learned my addr 172.21.15.5:0/3043835446 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3001069f0 0x7fe3001a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe300101380 0x7fe3001996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fe59c640 1 -- 172.21.15.5:0/3043835446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe300101380 msgr2=0x7fe3001996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fe59c640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe300101380 0x7fe3001996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fe59c640 1 -- 172.21.15.5:0/3043835446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3001069f0 msgr2=0x7fe3001a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fe59c640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3001069f0 0x7fe3001a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fe59c640 1 -- 172.21.15.5:0/3043835446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3001a4720 con 0x7fe3001009e0 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.220+0000 7fe2fe59c640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3001009e0 0x7fe300199160 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000d950 tx=0x7fe2f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.221+0000 7fe2ef7fe640 1 -- 172.21.15.5:0/3043835446 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0017030 con 0x7fe3001009e0 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.221+0000 7fe2ef7fe640 1 -- 172.21.15.5:0/3043835446 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f0004020 con 0x7fe3001009e0 2024-09-11T22:28:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.221+0000 7fe304f54640 1 -- 172.21.15.5:0/3043835446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3001a48f0 con 0x7fe3001009e0 2024-09-11T22:28:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.221+0000 7fe2ef7fe640 1 -- 172.21.15.5:0/3043835446 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00047a0 con 0x7fe3001009e0 2024-09-11T22:28:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.221+0000 7fe304f54640 1 -- 172.21.15.5:0/3043835446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3001a4e60 con 0x7fe3001009e0 2024-09-11T22:28:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.223+0000 7fe2ef7fe640 1 -- 172.21.15.5:0/3043835446 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe2f00041d0 con 0x7fe3001009e0 2024-09-11T22:28:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.223+0000 7fe307dc8740 1 Processor -- start 2024-09-11T22:28:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe307dc8740 1 -- start start 2024-09-11T22:28:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x56347537fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x56347538a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x56347538d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe307dc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5634752e42f0 con 0x56347538acb0 2024-09-11T22:28:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe307dc8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56347528c270 con 0x56347537faf0 2024-09-11T22:28:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe307dc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5634752e02b0 con 0x563475380410 2024-09-11T22:28:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x56347538d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x56347537fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x56347538d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40582/0 (socket says 172.21.15.5:40582) 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fdd9b640 1 -- 172.21.15.5:0/1895240678 learned_addr learned my addr 172.21.15.5:0/1895240678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fe59c640 1 --2- 172.21.15.5:0/1895240678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x56347538a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fed9d640 1 -- 172.21.15.5:0/1895240678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 msgr2=0x56347538a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/1895240678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x56347538a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fed9d640 1 -- 172.21.15.5:0/1895240678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 msgr2=0x56347538d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/1895240678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x56347538d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fed9d640 1 -- 172.21.15.5:0/1895240678 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347528b340 con 0x56347537faf0 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2ef7fe640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe2e80a2e20 0x7fe2e80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.224+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/1895240678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x56347538d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.225+0000 7fe2fe59c640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe2e80a2e20 0x7fe2e80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.225+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/1895240678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x56347537fed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009ca0 tx=0x7fe2f8009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e49df270e8e25d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.225+0000 7fe2ef7fe640 1 -- 172.21.15.5:0/3043835446 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fe2f00cedc0 con 0x7fe3001009e0 2024-09-11T22:28:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.225+0000 7fe2ecff9640 1 -- 172.21.15.5:0/1895240678 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8048070 con 0x56347537faf0 2024-09-11T22:28:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.225+0000 7fe2ecff9640 1 -- 172.21.15.5:0/1895240678 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f8036070 con 0x56347537faf0 2024-09-11T22:28:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.225+0000 7fe2ecff9640 1 -- 172.21.15.5:0/1895240678 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8004580 con 0x56347537faf0 2024-09-11T22:28:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 -- 172.21.15.5:0/1895240678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 msgr2=0x56347537fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe2fe59c640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe2e80a2e20 0x7fe2e80a52e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x56347537cec0 tx=0x7fe2f00d25a0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:40.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 --2- 172.21.15.5:0/1895240678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x56347537fed0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009ca0 tx=0x7fe2f8009a20 comp rx=0 tx=0).stop 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 -- 172.21.15.5:0/1895240678 shutdown_connections 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 --2- 172.21.15.5:0/1895240678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x56347538d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 --2- 172.21.15.5:0/1895240678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x56347538a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 --2- 172.21.15.5:0/1895240678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x56347537fed0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 -- 172.21.15.5:0/1895240678 >> 172.21.15.5:0/1895240678 conn(0x56347537c060 msgr2=0x56347537c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 -- 172.21.15.5:0/1895240678 shutdown_connections 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 -- 172.21.15.5:0/1895240678 wait complete. 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 -- start start 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.226+0000 7fe307dc8740 1 Processor -- start 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x563475387c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x563475388170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe307dc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56347538d880 con 0x56347538acb0 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe307dc8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56347526d290 con 0x56347537faf0 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe307dc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56347538da00 con 0x563475380410 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe307dc8740 10 client.0 _ll_register_callbacks cb 0x56347529e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753886b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40590/0 (socket says 172.21.15.5:40590) 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fed9d640 1 -- 172.21.15.5:0/113007327 learned_addr learned my addr 172.21.15.5:0/113007327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x563475388170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x563475387c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fed9d640 1 -- 172.21.15.5:0/113007327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 msgr2=0x563475388170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x563475388170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fed9d640 1 -- 172.21.15.5:0/113007327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 msgr2=0x563475387c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x563475387c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fed9d640 1 -- 172.21.15.5:0/113007327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563474ff7210 con 0x56347538acb0 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x563475388170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753886b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80041e0 tx=0x7fe2f80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a5a9dcd5d1f345 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.227+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/113007327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8048070 con 0x56347538acb0 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/113007327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fe2f8031ac0 con 0x56347538acb0 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe307dc8740 1 -- 172.21.15.5:0/113007327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2f8036070 con 0x56347538acb0 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/113007327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe2f8004810 con 0x56347538acb0 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe307dc8740 10 client.17370 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe307dc8740 1 -- 172.21.15.5:0/113007327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5634750c3620 con 0x56347538acb0 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/113007327 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f8036070 con 0x56347538acb0 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/113007327 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fe2f8042030 con 0x56347538acb0 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe307dc8740 10 client.17370 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe307dc8740 10 client.17370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe307dc8740 20 client.17370 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe307dc8740 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56347539ab20 0x56347539cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe307dc8740 4 client.17370 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe307dc8740 1 -- 172.21.15.5:0/113007327 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634752f1a20 con 0x56347539ab20 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.228+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56347539ab20 0x56347539cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.229+0000 7fe2e5ffb640 10 client.17370 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.229+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56347539ab20 0x56347539cf10 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=898e765729bf12ca server_cookie=3b1cef3664c6c930 in_seq=0 out_seq=0 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.229+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/113007327 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634752f1a20 con 0x56347539ab20 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.229+0000 7fe2e5ffb640 10 client.17370 handle_command_reply: tid=0 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.229+0000 7fe307dc8740 1 client.17370 shutdown 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 1 -- 172.21.15.5:0/113007327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 msgr2=0x5634753886b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753886b0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80041e0 tx=0x7fe2f80040e0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 1 -- 172.21.15.5:0/113007327 shutdown_connections 2024-09-11T22:28:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56347539ab20 0x56347539cf10 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753886b0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x563475388170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe2e5ffb640 0 client.17370 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 1 --2- 172.21.15.5:0/113007327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x563475387c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 1 -- 172.21.15.5:0/113007327 >> 172.21.15.5:0/113007327 conn(0x56347537c060 msgr2=0x563475381d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 1 -- 172.21.15.5:0/113007327 shutdown_connections 2024-09-11T22:28:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 1 -- 172.21.15.5:0/113007327 wait complete. 2024-09-11T22:28:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.230+0000 7fe307dc8740 20 client.17370 trim_cache size 0 max 16384 2024-09-11T22:28:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.244+0000 7fe307dc8740 1 Processor -- start 2024-09-11T22:28:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.244+0000 7fe307dc8740 1 -- start start 2024-09-11T22:28:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.244+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347537faf0 0x5634753884e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.244+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563475380410 0x563475389ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.244+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347538acb0 0x56347539ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.244+0000 7fe307dc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001ad8f0 con 0x56347537faf0 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.244+0000 7fe307dc8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001af430 con 0x563475380410 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.244+0000 7fe307dc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001adfc0 con 0x56347538acb0 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347537faf0 0x5634753884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fdd9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563475380410 0x563475389ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fdd9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563475380410 0x563475389ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45156/0 (socket says 172.21.15.5:45156) 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fdd9b640 1 -- 172.21.15.5:0/417905788 learned_addr learned my addr 172.21.15.5:0/417905788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/417905788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347538acb0 0x56347539ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fdd9b640 1 -- 172.21.15.5:0/417905788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347538acb0 msgr2=0x56347539ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:40.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/417905788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347538acb0 0x56347539ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fdd9b640 1 -- 172.21.15.5:0/417905788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347537faf0 msgr2=0x5634753884e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/417905788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347537faf0 0x5634753884e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fdd9b640 1 -- 172.21.15.5:0/417905788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563475397660 con 0x563475380410 2024-09-11T22:28:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/417905788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563475380410 0x563475389ba0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4005fd0 tx=0x7fe2f4013c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2530f5c4ce22e78 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2e7fff640 1 -- 172.21.15.5:0/417905788 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4013110 con 0x563475380410 2024-09-11T22:28:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2e7fff640 1 -- 172.21.15.5:0/417905788 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f40132b0 con 0x563475380410 2024-09-11T22:28:40.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.245+0000 7fe2e7fff640 1 -- 172.21.15.5:0/417905788 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f401dbc0 con 0x563475380410 2024-09-11T22:28:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 -- 172.21.15.5:0/417905788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563475380410 msgr2=0x563475389ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 --2- 172.21.15.5:0/417905788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563475380410 0x563475389ba0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4005fd0 tx=0x7fe2f4013c10 comp rx=0 tx=0).stop 2024-09-11T22:28:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 -- 172.21.15.5:0/417905788 shutdown_connections 2024-09-11T22:28:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 --2- 172.21.15.5:0/417905788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56347538acb0 0x56347539ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 --2- 172.21.15.5:0/417905788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563475380410 0x563475389ba0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 --2- 172.21.15.5:0/417905788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347537faf0 0x5634753884e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 -- 172.21.15.5:0/417905788 >> 172.21.15.5:0/417905788 conn(0x56347537c060 msgr2=0x563475389490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 -- 172.21.15.5:0/417905788 shutdown_connections 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 -- 172.21.15.5:0/417905788 wait complete. 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 -- start start 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.246+0000 7fe307dc8740 1 Processor -- start 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x56347539d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x56347539dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe307dc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753fd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe307dc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001aed60 con 0x56347538acb0 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe307dc8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001ae690 con 0x56347537faf0 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe307dc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3001ab8d0 con 0x563475380410 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe307dc8740 10 client.0 _ll_register_callbacks cb 0x7fe2f8046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753fd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40608/0 (socket says 172.21.15.5:40608) 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fdd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x56347539dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fed9d640 1 -- 172.21.15.5:0/118707144 learned_addr learned my addr 172.21.15.5:0/118707144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x56347539d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fed9d640 1 -- 172.21.15.5:0/118707144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 msgr2=0x56347539dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x56347539dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fed9d640 1 -- 172.21.15.5:0/118707144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 msgr2=0x56347539d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x56347539d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fed9d640 1 -- 172.21.15.5:0/118707144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634753fda60 con 0x56347538acb0 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2fed9d640 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753fd420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80042b0 tx=0x7fe2f8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3060e7c596307d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.247+0000 7fe2e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/118707144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8048070 con 0x56347538acb0 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe307dc8740 1 -- 172.21.15.5:0/118707144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2f40132b0 con 0x56347538acb0 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/118707144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8585+0+0 (secure 0 0 0) 0x7fe2f80318c0 con 0x56347538acb0 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/118707144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe2f8004630 con 0x56347538acb0 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe307dc8740 10 client.17376 fetch_fsmap learned FSMap version 36 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe307dc8740 1 -- 172.21.15.5:0/118707144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563475404e00 con 0x56347538acb0 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/118707144 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f40132b0 con 0x56347538acb0 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/118707144 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fe2f8042040 con 0x56347538acb0 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe307dc8740 10 client.17376 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe307dc8740 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe307dc8740 20 client.17376 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe307dc8740 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56347539ab20 0x5634754074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe307dc8740 4 client.17376 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:28:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe307dc8740 1 -- 172.21.15.5:0/118707144 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634752ebd70 con 0x56347539ab20 2024-09-11T22:28:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.248+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56347539ab20 0x5634754074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.249+0000 7fe2e5ffb640 10 client.17376 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.249+0000 7fe2fdd9b640 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56347539ab20 0x5634754074b0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf357913a2a4713 server_cookie=c99716a01643c985 in_seq=0 out_seq=0 2024-09-11T22:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.289+0000 7fe2e5ffb640 1 -- 172.21.15.5:0/118707144 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634752ebd70 con 0x56347539ab20 2024-09-11T22:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.289+0000 7fe2e5ffb640 10 client.17376 handle_command_reply: tid=0 2024-09-11T22:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 client.17376 shutdown 2024-09-11T22:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 -- 172.21.15.5:0/118707144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 msgr2=0x5634753fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753fd420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80042b0 tx=0x7fe2f8009920 comp rx=0 tx=0).stop 2024-09-11T22:28:40.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 -- 172.21.15.5:0/118707144 shutdown_connections 2024-09-11T22:28:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56347539ab20 0x5634754074b0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56347538acb0 0x5634753fd420 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563475380410 0x56347539dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe2e5ffb640 0 client.17376 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 --2- 172.21.15.5:0/118707144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56347537faf0 0x56347539d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 -- 172.21.15.5:0/118707144 >> 172.21.15.5:0/118707144 conn(0x56347537c060 msgr2=0x56347537b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 -- 172.21.15.5:0/118707144 shutdown_connections 2024-09-11T22:28:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 1 -- 172.21.15.5:0/118707144 wait complete. 2024-09-11T22:28:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe307dc8740 20 client.17376 trim_cache size 0 max 16384 2024-09-11T22:28:40.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:28:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:28:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:28:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:28:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe304f54640 1 -- 172.21.15.5:0/3043835446 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe2e80a2e20 msgr2=0x7fe2e80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:40.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe304f54640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe2e80a2e20 0x7fe2e80a52e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x56347537cec0 tx=0x7fe2f00d25a0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe304f54640 1 -- 172.21.15.5:0/3043835446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3001009e0 msgr2=0x7fe300199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.290+0000 7fe304f54640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3001009e0 0x7fe300199160 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000d950 tx=0x7fe2f000de10 comp rx=0 tx=0).stop 2024-09-11T22:28:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.291+0000 7fe304f54640 1 -- 172.21.15.5:0/3043835446 shutdown_connections 2024-09-11T22:28:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.291+0000 7fe304f54640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe2e80a2e20 0x7fe2e80a52e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.291+0000 7fe304f54640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3001069f0 0x7fe3001a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.291+0000 7fe304f54640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe300101380 0x7fe3001996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.291+0000 7fe304f54640 1 --2- 172.21.15.5:0/3043835446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3001009e0 0x7fe300199160 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.291+0000 7fe304f54640 1 -- 172.21.15.5:0/3043835446 >> 172.21.15.5:0/3043835446 conn(0x7fe3000fc6a0 msgr2=0x7fe3000fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.291+0000 7fe304f54640 1 -- 172.21.15.5:0/3043835446 shutdown_connections 2024-09-11T22:28:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:40.291+0000 7fe304f54640 1 -- 172.21.15.5:0/3043835446 wait complete. 2024-09-11T22:28:40.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 8.02 MB/sec execute 178 sec latency 16810.305 ms 2024-09-11T22:28:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:40.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:28:40.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:40 smithi005 ceph-mon[30943]: daemon mds.k restarted 2024-09-11T22:28:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.998+0000 7f4920d8e640 1 -- 172.21.15.5:0/2391751954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c103660 msgr2=0x7f491c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.998+0000 7f4920d8e640 1 --2- 172.21.15.5:0/2391751954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c103660 0x7f491c10bb50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f4904009900 tx=0x7f490402a480 comp rx=0 tx=0).stop 2024-09-11T22:28:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.998+0000 7f4920d8e640 1 -- 172.21.15.5:0/2391751954 shutdown_connections 2024-09-11T22:28:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.998+0000 7f4920d8e640 1 --2- 172.21.15.5:0/2391751954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c10c170 0x7f491c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.998+0000 7f4920d8e640 1 --2- 172.21.15.5:0/2391751954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c103660 0x7f491c10bb50 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.998+0000 7f4920d8e640 1 --2- 172.21.15.5:0/2391751954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f491c102d40 0x7f491c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.998+0000 7f4920d8e640 1 -- 172.21.15.5:0/2391751954 >> 172.21.15.5:0/2391751954 conn(0x7f491c0fc7a0 msgr2=0x7f491c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.998+0000 7f4920d8e640 1 -- 172.21.15.5:0/2391751954 shutdown_connections 2024-09-11T22:28:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.998+0000 7f4920d8e640 1 -- 172.21.15.5:0/2391751954 wait complete. 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4920d8e640 1 Processor -- start 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4920d8e640 1 -- start start 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4920d8e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f491c102d40 0x7f491c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4920d8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c103660 0x7f491c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4920d8e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c10c170 0x7f491c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4920d8e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f491c111f70 con 0x7f491c103660 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4920d8e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f491c111df0 con 0x7f491c102d40 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4920d8e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f491c1120f0 con 0x7f491c10c170 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4913fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c103660 0x7f491c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4913fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c103660 0x7f491c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40620/0 (socket says 172.21.15.5:40620) 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f4913fff640 1 -- 172.21.15.5:0/3175463587 learned_addr learned my addr 172.21.15.5:0/3175463587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f491a59c640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f491c102d40 0x7f491c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f491ad9d640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c10c170 0x7f491c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f491a59c640 1 -- 172.21.15.5:0/3175463587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c10c170 msgr2=0x7f491c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f491a59c640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c10c170 0x7f491c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f491a59c640 1 -- 172.21.15.5:0/3175463587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c103660 msgr2=0x7f491c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f491a59c640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c103660 0x7f491c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:40.999+0000 7f491a59c640 1 -- 172.21.15.5:0/3175463587 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c1a6930 con 0x7f491c102d40 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.000+0000 7f4913fff640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c103660 0x7f491c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.000+0000 7f491a59c640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f491c102d40 0x7f491c1009a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f490c00d6d0 tx=0x7f490c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.000+0000 7f49137fe640 1 -- 172.21.15.5:0/3175463587 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c017070 con 0x7f491c102d40 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.000+0000 7f4920d8e640 1 -- 172.21.15.5:0/3175463587 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491c1a6c20 con 0x7f491c102d40 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.000+0000 7f4920d8e640 1 -- 172.21.15.5:0/3175463587 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491c1a71b0 con 0x7f491c102d40 2024-09-11T22:28:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.000+0000 7f49137fe640 1 -- 172.21.15.5:0/3175463587 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c009820 con 0x7f491c102d40 2024-09-11T22:28:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.000+0000 7f49137fe640 1 -- 172.21.15.5:0/3175463587 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c020060 con 0x7f491c102d40 2024-09-11T22:28:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.001+0000 7f4920d8e640 1 -- 172.21.15.5:0/3175463587 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48fc005290 con 0x7f491c102d40 2024-09-11T22:28:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.002+0000 7f49137fe640 1 -- 172.21.15.5:0/3175463587 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f490c005220 con 0x7f491c102d40 2024-09-11T22:28:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.003+0000 7f49137fe640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49080a2cf0 0x7f49080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.003+0000 7f49137fe640 1 -- 172.21.15.5:0/3175463587 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f490c0cf170 con 0x7f491c102d40 2024-09-11T22:28:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.003+0000 7f4913fff640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49080a2cf0 0x7f49080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.004+0000 7f4913fff640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49080a2cf0 0x7f49080a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4904031550 tx=0x7f4904005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.006+0000 7f49137fe640 1 -- 172.21.15.5:0/3175463587 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f490c09bcf0 con 0x7f491c102d40 2024-09-11T22:28:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:40 smithi159 ceph-mon[35534]: daemon mds.k restarted 2024-09-11T22:28:41.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:28:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setuser ceph since I am not root 2024-09-11T22:28:41.080 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:28:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setgroup ceph since I am not root 2024-09-11T22:28:41.080 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:28:40 smithi159 ceph-mds[65034]: -- 172.21.15.159:0/174028483 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f24895860 con 0x560f24873000 2024-09-11T22:28:41.080 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:28:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: starting mds.k at 2024-09-11T22:28:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:40 smithi172 ceph-mon[35461]: daemon mds.k restarted 2024-09-11T22:28:41.116 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:41.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.241+0000 7f4920d8e640 1 -- 172.21.15.5:0/3175463587 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48fc005580 con 0x7f491c102d40 2024-09-11T22:28:41.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.242+0000 7f49137fe640 1 -- 172.21.15.5:0/3175463587 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11967 (secure 0 0 0) 0x7f490c09b5f0 con 0x7f491c102d40 2024-09-11T22:28:41.243 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:28:41.243 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":38,"btime":"2024-09-11T22:28:40:621476+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:28:40.621460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26777":{"gid":26777,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/3129135413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3129135413},{"type":"v1","addr":"172.21.15.159:6833","nonce":3129135413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:28:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 38 2024-09-11T22:28:41.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 -- 172.21.15.5:0/3175463587 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49080a2cf0 msgr2=0x7f49080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:41.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49080a2cf0 0x7f49080a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4904031550 tx=0x7f4904005a20 comp rx=0 tx=0).stop 2024-09-11T22:28:41.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 -- 172.21.15.5:0/3175463587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f491c102d40 msgr2=0x7f491c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:41.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f491c102d40 0x7f491c1009a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f490c00d6d0 tx=0x7f490c00db90 comp rx=0 tx=0).stop 2024-09-11T22:28:41.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 -- 172.21.15.5:0/3175463587 shutdown_connections 2024-09-11T22:28:41.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f491c10c170 0x7f491c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f491c103660 0x7f491c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49080a2cf0 0x7f49080a51b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 --2- 172.21.15.5:0/3175463587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f491c102d40 0x7f491c1009a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 -- 172.21.15.5:0/3175463587 >> 172.21.15.5:0/3175463587 conn(0x7f491c0fc7a0 msgr2=0x7f491c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:41.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 -- 172.21.15.5:0/3175463587 shutdown_connections 2024-09-11T22:28:41.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.244+0000 7f4920d8e640 1 -- 172.21.15.5:0/3175463587 wait complete. 2024-09-11T22:28:41.324 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 7.98 MB/sec execute 179 sec latency 17810.352 ms 2024-09-11T22:28:41.778 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:41 smithi005 ceph-mon[30943]: pgmap v199: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-11T22:28:41.778 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:41 smithi005 ceph-mon[30943]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T22:28:41.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:41 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6832/3129135413,v1:172.21.15.159:6833/3129135413] up:boot 2024-09-11T22:28:41.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:41 smithi005 ceph-mon[30943]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:28:41.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:41 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:28:41.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:41 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:28:41.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:41 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3175463587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:28:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.776+0000 7f32054a6640 1 -- 172.21.15.5:0/744135501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3200102de0 msgr2=0x7f32001031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.776+0000 7f32054a6640 1 --2- 172.21.15.5:0/744135501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3200102de0 0x7f32001031c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f31f8009be0 tx=0x7f31f802a750 comp rx=0 tx=0).stop 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.776+0000 7f32054a6640 1 -- 172.21.15.5:0/744135501 shutdown_connections 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.776+0000 7f32054a6640 1 --2- 172.21.15.5:0/744135501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320010c180 0x7f320010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.776+0000 7f32054a6640 1 --2- 172.21.15.5:0/744135501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3200103700 0x7f320010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.776+0000 7f32054a6640 1 --2- 172.21.15.5:0/744135501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3200102de0 0x7f32001031c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.776+0000 7f32054a6640 1 -- 172.21.15.5:0/744135501 >> 172.21.15.5:0/744135501 conn(0x7f32000fc7a0 msgr2=0x7f32000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.776+0000 7f32054a6640 1 -- 172.21.15.5:0/744135501 shutdown_connections 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.776+0000 7f32054a6640 1 -- 172.21.15.5:0/744135501 wait complete. 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.776+0000 7f32054a6640 1 Processor -- start 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f32054a6640 1 -- start start 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f32054a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3200102de0 0x7f3200100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f32054a6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3200103700 0x7f3200100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f32054a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320010c180 0x7f32001014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f32054a6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3200111f20 con 0x7f320010c180 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f32054a6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3200111da0 con 0x7f3200103700 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f32054a6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f32001120a0 con 0x7f3200102de0 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f31ff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320010c180 0x7f32001014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f31ff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320010c180 0x7f32001014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48452/0 (socket says 172.21.15.5:48452) 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f31ff7fe640 1 -- 172.21.15.5:0/358626088 learned_addr learned my addr 172.21.15.5:0/358626088 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f31fe7fc640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3200103700 0x7f3200100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.777+0000 7f31feffd640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3200102de0 0x7f3200100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f31ff7fe640 1 -- 172.21.15.5:0/358626088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3200102de0 msgr2=0x7f3200100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f31ff7fe640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3200102de0 0x7f3200100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f31ff7fe640 1 -- 172.21.15.5:0/358626088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3200103700 msgr2=0x7f3200100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f31ff7fe640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3200103700 0x7f3200100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f31ff7fe640 1 -- 172.21.15.5:0/358626088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32001a6960 con 0x7f320010c180 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f31ff7fe640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320010c180 0x7f32001014c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f31f000d950 tx=0x7f31f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f31f7fff640 1 -- 172.21.15.5:0/358626088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0017030 con 0x7f320010c180 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f32054a6640 1 -- 172.21.15.5:0/358626088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32001a6c50 con 0x7f320010c180 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f31f7fff640 1 -- 172.21.15.5:0/358626088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31f0004020 con 0x7f320010c180 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f31f7fff640 1 -- 172.21.15.5:0/358626088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0003c00 con 0x7f320010c180 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.778+0000 7f32054a6640 1 -- 172.21.15.5:0/358626088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32001a7190 con 0x7f320010c180 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.779+0000 7f32054a6640 1 -- 172.21.15.5:0/358626088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32001031c0 con 0x7f320010c180 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.781+0000 7f31f7fff640 1 -- 172.21.15.5:0/358626088 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f31f001a070 con 0x7f320010c180 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.781+0000 7f31f7fff640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f31ec0a2cf0 0x7f31ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.781+0000 7f31feffd640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f31ec0a2cf0 0x7f31ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.781+0000 7f31f7fff640 1 -- 172.21.15.5:0/358626088 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f31f00cea60 con 0x7f320010c180 2024-09-11T22:28:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.782+0000 7f31feffd640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f31ec0a2cf0 0x7f31ec0a51b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31f8009330 tx=0x7f31f80092c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:41.789+0000 7f31f7fff640 1 -- 172.21.15.5:0/358626088 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31f009b640 con 0x7f320010c180 2024-09-11T22:28:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.007+0000 7f32054a6640 1 -- 172.21.15.5:0/358626088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f320002d7c0 con 0x7f320010c180 2024-09-11T22:28:42.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.007+0000 7f31f7fff640 1 -- 172.21.15.5:0/358626088 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11967 (secure 0 0 0) 0x7f31f009af40 con 0x7f320010c180 2024-09-11T22:28:42.008 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:28:42.008 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":38,"btime":"2024-09-11T22:28:40:621476+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:28:40.621460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26777":{"gid":26777,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/3129135413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":3129135413},{"type":"v1","addr":"172.21.15.159:6833","nonce":3129135413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:28:42.009 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 38 2024-09-11T22:28:42.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 -- 172.21.15.5:0/358626088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f31ec0a2cf0 msgr2=0x7f31ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:42.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f31ec0a2cf0 0x7f31ec0a51b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f31f8009330 tx=0x7f31f80092c0 comp rx=0 tx=0).stop 2024-09-11T22:28:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 -- 172.21.15.5:0/358626088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320010c180 msgr2=0x7f32001014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320010c180 0x7f32001014c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f31f000d950 tx=0x7f31f000de10 comp rx=0 tx=0).stop 2024-09-11T22:28:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 -- 172.21.15.5:0/358626088 shutdown_connections 2024-09-11T22:28:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f320010c180 0x7f32001014c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f31ec0a2cf0 0x7f31ec0a51b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3200103700 0x7f3200100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 --2- 172.21.15.5:0/358626088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3200102de0 0x7f3200100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 -- 172.21.15.5:0/358626088 >> 172.21.15.5:0/358626088 conn(0x7f32000fc7a0 msgr2=0x7f32000fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 -- 172.21.15.5:0/358626088 shutdown_connections 2024-09-11T22:28:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:42.010+0000 7f32054a6640 1 -- 172.21.15.5:0/358626088 wait complete. 2024-09-11T22:28:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:41 smithi159 ceph-mon[35534]: pgmap v199: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-11T22:28:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:41 smithi159 ceph-mon[35534]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T22:28:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:41 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6832/3129135413,v1:172.21.15.159:6833/3129135413] up:boot 2024-09-11T22:28:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:41 smithi159 ceph-mon[35534]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:28:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:41 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:28:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:41 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:28:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:41 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3175463587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:28:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:41 smithi172 ceph-mon[35461]: pgmap v199: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-11T22:28:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:41 smithi172 ceph-mon[35461]: osdmap e88: 12 total, 12 up, 12 in 2024-09-11T22:28:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:41 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6832/3129135413,v1:172.21.15.159:6833/3129135413] up:boot 2024-09-11T22:28:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:41 smithi172 ceph-mon[35461]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:28:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:41 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:28:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:41 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:28:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:41 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3175463587' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:28:42.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:28:42.383 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 7.93 MB/sec execute 180 sec latency 18810.399 ms 2024-09-11T22:28:42.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:42 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/358626088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:28:42.979 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id e5aba961 --release --await' 2024-09-11T22:28:42.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id e5aba961 --release --await 2024-09-11T22:28:43.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:42 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/358626088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:28:43.083 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:43.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:42 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/358626088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:28:43.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 7.89 MB/sec execute 181 sec latency 19810.455 ms 2024-09-11T22:28:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 -- 172.21.15.5:0/1740150070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9498102ef0 msgr2=0x7f949810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 --2- 172.21.15.5:0/1740150070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9498102ef0 0x7f949810da40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9488009ca0 tx=0x7f948802a950 comp rx=0 tx=0).stop 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 -- 172.21.15.5:0/1740150070 shutdown_connections 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 --2- 172.21.15.5:0/1740150070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9498102ef0 0x7f949810da40 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 --2- 172.21.15.5:0/1740150070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9498102570 0x7f94981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 --2- 172.21.15.5:0/1740150070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9498100e20 0x7f9498101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 -- 172.21.15.5:0/1740150070 >> 172.21.15.5:0/1740150070 conn(0x7f94980fc7d0 msgr2=0x7f94980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 -- 172.21.15.5:0/1740150070 shutdown_connections 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 -- 172.21.15.5:0/1740150070 wait complete. 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 Processor -- start 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.632+0000 7f949fdd5640 1 -- start start 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949fdd5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9498100e20 0x7f94981a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949fdd5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9498102570 0x7f94981a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949fdd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9498102ef0 0x7f949819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949fdd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9498111020 con 0x7f9498102ef0 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949fdd5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9498110ea0 con 0x7f9498102570 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949fdd5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94981111a0 con 0x7f9498100e20 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949e372640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9498102ef0 0x7f949819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949e372640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9498102ef0 0x7f949819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48472/0 (socket says 172.21.15.5:48472) 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949e372640 1 -- 172.21.15.5:0/3766700077 learned_addr learned my addr 172.21.15.5:0/3766700077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949db71640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9498100e20 0x7f94981a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949e372640 1 -- 172.21.15.5:0/3766700077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9498100e20 msgr2=0x7f94981a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949e372640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9498100e20 0x7f94981a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949e372640 1 -- 172.21.15.5:0/3766700077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9498102570 msgr2=0x7f94981a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949d370640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9498102570 0x7f94981a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949e372640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9498102570 0x7f94981a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949e372640 1 -- 172.21.15.5:0/3766700077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949819ad50 con 0x7f9498102ef0 2024-09-11T22:28:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949db71640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9498100e20 0x7f94981a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f949e372640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9498102ef0 0x7f949819a620 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9488006010 tx=0x7f948802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.633+0000 7f9496ffd640 1 -- 172.21.15.5:0/3766700077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488034040 con 0x7f9498102ef0 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.634+0000 7f9496ffd640 1 -- 172.21.15.5:0/3766700077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9488003e00 con 0x7f9498102ef0 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.634+0000 7f9496ffd640 1 -- 172.21.15.5:0/3766700077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488045480 con 0x7f9498102ef0 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.634+0000 7f949fdd5640 1 -- 172.21.15.5:0/3766700077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949819afe0 con 0x7f9498102ef0 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.634+0000 7f949fdd5640 1 -- 172.21.15.5:0/3766700077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94981a7330 con 0x7f9498102ef0 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.636+0000 7f9496ffd640 1 -- 172.21.15.5:0/3766700077 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9488037020 con 0x7f9498102ef0 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.636+0000 7f94a21ae740 1 Processor -- start 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.636+0000 7f94a21ae740 1 -- start start 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e02b40 0x55a994e01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e01a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e0c370 0x55a994e0e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f94a21ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a994d672f0 con 0x55a994e015d0 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f94a21ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a994d0ecf0 con 0x55a994e0c370 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f94a21ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a994d632b0 con 0x55a994e02b40 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949db71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e01a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949db71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e01a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48478/0 (socket says 172.21.15.5:48478) 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949db71640 1 -- 172.21.15.5:0/2996054251 learned_addr learned my addr 172.21.15.5:0/2996054251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949d370640 1 --2- 172.21.15.5:0/2996054251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e02b40 0x55a994e01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949d370640 1 --2- 172.21.15.5:0/2996054251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e0c370 0x55a994e0e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f9496ffd640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94800a2cf0 0x7f94800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949db71640 1 -- 172.21.15.5:0/2996054251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e02b40 msgr2=0x55a994e01090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949db71640 1 --2- 172.21.15.5:0/2996054251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e02b40 0x55a994e01090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949db71640 1 -- 172.21.15.5:0/2996054251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e0c370 msgr2=0x55a994e0e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949db71640 1 --2- 172.21.15.5:0/2996054251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e0c370 0x55a994e0e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949db71640 1 -- 172.21.15.5:0/2996054251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a994cf33a0 con 0x55a994e015d0 2024-09-11T22:28:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f949e372640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94800a2cf0 0x7f94800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.637+0000 7f9496ffd640 1 -- 172.21.15.5:0/3766700077 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f94880f05a0 con 0x7f9498102ef0 2024-09-11T22:28:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.638+0000 7f949db71640 1 --2- 172.21.15.5:0/2996054251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e01a10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9490007850 tx=0x7f9490007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ccc31f2d09fa57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.638+0000 7f947ffff640 1 -- 172.21.15.5:0/2996054251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949000d950 con 0x55a994e015d0 2024-09-11T22:28:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.638+0000 7f947ffff640 1 -- 172.21.15.5:0/2996054251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9490018070 con 0x55a994e015d0 2024-09-11T22:28:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.638+0000 7f947ffff640 1 -- 172.21.15.5:0/2996054251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9490013e40 con 0x55a994e015d0 2024-09-11T22:28:43.694 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:28:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.638+0000 7f94a21ae740 1 -- 172.21.15.5:0/2996054251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 msgr2=0x55a994e01a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.638+0000 7f949e372640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94800a2cf0 0x7f94800a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f94880ede70 tx=0x7f948802a950 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.638+0000 7f94a21ae740 1 --2- 172.21.15.5:0/2996054251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e01a10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9490007850 tx=0x7f9490007d10 comp rx=0 tx=0).stop 2024-09-11T22:28:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.639+0000 7f94a21ae740 1 -- 172.21.15.5:0/2996054251 shutdown_connections 2024-09-11T22:28:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.639+0000 7f94a21ae740 1 --2- 172.21.15.5:0/2996054251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e0c370 0x55a994e0e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.639+0000 7f94a21ae740 1 --2- 172.21.15.5:0/2996054251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e01a10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.639+0000 7f94a21ae740 1 --2- 172.21.15.5:0/2996054251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e02b40 0x55a994e01090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.639+0000 7f94a21ae740 1 -- 172.21.15.5:0/2996054251 >> 172.21.15.5:0/2996054251 conn(0x55a994dca310 msgr2=0x55a994dfdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.639+0000 7f94a21ae740 1 -- 172.21.15.5:0/2996054251 shutdown_connections 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.639+0000 7f94a21ae740 1 -- 172.21.15.5:0/2996054251 wait complete. 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.639+0000 7f94a21ae740 1 -- start start 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.639+0000 7f94a21ae740 1 Processor -- start 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 0x55a994e0bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e08fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e0c370 0x55a994e09530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f94a21ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a994e0f070 con 0x55a994e02b40 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f94a21ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a994d22060 con 0x55a994e015d0 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f94a21ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a994e0f1f0 con 0x55a994e0c370 2024-09-11T22:28:43.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f94a21ae740 10 client.0 _ll_register_callbacks cb 0x55a994a758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949db71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e08fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949db71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e08fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48488/0 (socket says 172.21.15.5:48488) 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949db71640 1 -- 172.21.15.5:0/802810970 learned_addr learned my addr 172.21.15.5:0/802810970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949d370640 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 0x55a994e0bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949d370640 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e0c370 0x55a994e09530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949db71640 1 -- 172.21.15.5:0/802810970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e0c370 msgr2=0x55a994e09530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949db71640 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e0c370 0x55a994e09530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949db71640 1 -- 172.21.15.5:0/802810970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 msgr2=0x55a994e0bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949db71640 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 0x55a994e0bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949db71640 1 -- 172.21.15.5:0/802810970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a994ad83c0 con 0x55a994e02b40 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f949db71640 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e08fb0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f94900139c0 tx=0x7f9490013730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c92661c6cbb1e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:43.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f947d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f947d7fa640 1 -- 172.21.15.5:0/802810970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949000d950 con 0x55a994e02b40 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.640+0000 7f947d7fa640 1 -- 172.21.15.5:0/802810970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f949000daf0 con 0x55a994e02b40 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f94a21ae740 1 -- 172.21.15.5:0/802810970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9490018070 con 0x55a994e02b40 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f947d7fa640 1 -- 172.21.15.5:0/802810970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f949000dd10 con 0x55a994e02b40 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f94a21ae740 10 client.17403 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f94a21ae740 1 -- 172.21.15.5:0/802810970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a994e1aad0 con 0x55a994e02b40 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f947d7fa640 1 -- 172.21.15.5:0/802810970 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9490018070 con 0x55a994e02b40 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f947d7fa640 1 -- 172.21.15.5:0/802810970 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f949001c3f0 con 0x55a994e02b40 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f94a21ae740 10 client.17403 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f94a21ae740 10 client.17403 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f94a21ae740 20 client.17403 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f94a21ae740 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55a994e1bf30 0x55a994e1e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f94a21ae740 4 client.17403 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f94a21ae740 1 -- 172.21.15.5:0/802810970 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a994d74860 con 0x55a994e1bf30 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.641+0000 7f949d370640 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55a994e1bf30 0x55a994e1e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.642+0000 7f949d370640 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55a994e1bf30 0x55a994e1e320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=77a772d7462d3bc6 server_cookie=f9fcf47eb43f7c46 in_seq=0 out_seq=0 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.642+0000 7f947d7fa640 10 client.17403 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f947d7fa640 1 -- 172.21.15.5:0/802810970 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a994d74860 con 0x55a994e1bf30 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f947d7fa640 10 client.17403 handle_command_reply: tid=0 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 client.17403 shutdown 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 -- 172.21.15.5:0/802810970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 msgr2=0x55a994e08fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e08fb0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f94900139c0 tx=0x7f9490013730 comp rx=0 tx=0).stop 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 -- 172.21.15.5:0/802810970 shutdown_connections 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55a994e1bf30 0x55a994e1e320 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e0c370 0x55a994e09530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f947d7fa640 0 client.17403 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e08fb0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 --2- 172.21.15.5:0/802810970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 0x55a994e0bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 -- 172.21.15.5:0/802810970 >> 172.21.15.5:0/802810970 conn(0x55a994dca310 msgr2=0x55a994e03930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 -- 172.21.15.5:0/802810970 shutdown_connections 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 1 -- 172.21.15.5:0/802810970 wait complete. 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.643+0000 7f94a21ae740 20 client.17403 trim_cache size 0 max 16384 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f94a21ae740 1 Processor -- start 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f94a21ae740 1 -- start start 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 0x55a994e18340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e0af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e0c370 0x55a994e0d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f94a21ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94981afc90 con 0x55a994e02b40 2024-09-11T22:28:43.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f94a21ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94981b17d0 con 0x55a994e015d0 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f94a21ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94981b0360 con 0x55a994e0c370 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f949db71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e0af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f949db71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e0af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48504/0 (socket says 172.21.15.5:48504) 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.656+0000 7f949db71640 1 -- 172.21.15.5:0/723484261 learned_addr learned my addr 172.21.15.5:0/723484261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f949d370640 1 --2- 172.21.15.5:0/723484261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 0x55a994e18340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f949db71640 1 -- 172.21.15.5:0/723484261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e0c370 msgr2=0x55a994e0d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f949db71640 1 --2- 172.21.15.5:0/723484261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e0c370 0x55a994e0d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f949db71640 1 -- 172.21.15.5:0/723484261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 msgr2=0x55a994e18340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f949db71640 1 --2- 172.21.15.5:0/723484261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 0x55a994e18340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f949db71640 1 -- 172.21.15.5:0/723484261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a994dfded0 con 0x55a994e02b40 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f949d370640 1 --2- 172.21.15.5:0/723484261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 0x55a994e18340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f949db71640 1 --2- 172.21.15.5:0/723484261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e0af20 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f949001c960 tx=0x7f9490000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a416340dcb10a9e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:43.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f947f7fe640 1 -- 172.21.15.5:0/723484261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949001ce60 con 0x55a994e02b40 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f947f7fe640 1 -- 172.21.15.5:0/723484261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9490018040 con 0x55a994e02b40 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.657+0000 7f947f7fe640 1 -- 172.21.15.5:0/723484261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949001c9e0 con 0x55a994e02b40 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 -- 172.21.15.5:0/723484261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 msgr2=0x55a994e0af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 --2- 172.21.15.5:0/723484261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e0af20 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f949001c960 tx=0x7f9490000bc0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 -- 172.21.15.5:0/723484261 shutdown_connections 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 --2- 172.21.15.5:0/723484261 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e0c370 0x55a994e0d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 --2- 172.21.15.5:0/723484261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e02b40 0x55a994e0af20 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 --2- 172.21.15.5:0/723484261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e015d0 0x55a994e18340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 -- 172.21.15.5:0/723484261 >> 172.21.15.5:0/723484261 conn(0x55a994dca310 msgr2=0x55a994e0a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 -- 172.21.15.5:0/723484261 shutdown_connections 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 -- 172.21.15.5:0/723484261 wait complete. 2024-09-11T22:28:43.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 -- start start 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.658+0000 7f94a21ae740 1 Processor -- start 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e02b40 0x55a994e14de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f94a21ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e0c370 0x55a994e23170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f94a21ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94981b1100 con 0x55a994e015d0 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f94a21ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94981b0a30 con 0x55a994e0c370 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f94a21ae740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94981adc70 con 0x55a994e02b40 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f94a21ae740 10 client.0 _ll_register_callbacks cb 0x7f948c00cb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f949d370640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f949d370640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e148a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48520/0 (socket says 172.21.15.5:48520) 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f949d370640 1 -- 172.21.15.5:0/768614255 learned_addr learned my addr 172.21.15.5:0/768614255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:43.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f949db71640 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e02b40 0x55a994e14de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f949d370640 1 -- 172.21.15.5:0/768614255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e02b40 msgr2=0x55a994e14de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f949d370640 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e02b40 0x55a994e14de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f949d370640 1 -- 172.21.15.5:0/768614255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e0c370 msgr2=0x55a994e23170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f949d370640 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e0c370 0x55a994e23170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f949d370640 1 -- 172.21.15.5:0/768614255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a994e239a0 con 0x55a994e015d0 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f949d370640 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e148a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f948c006fd0 tx=0x7f948c018a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ee67c816081f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f947d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.659+0000 7f947d7fa640 1 -- 172.21.15.5:0/768614255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c00f040 con 0x55a994e015d0 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f947d7fa640 1 -- 172.21.15.5:0/768614255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f948c021550 con 0x55a994e015d0 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f94a21ae740 1 -- 172.21.15.5:0/768614255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9490018040 con 0x55a994e015d0 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f947d7fa640 1 -- 172.21.15.5:0/768614255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f948c0059c0 con 0x55a994e015d0 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f94a21ae740 10 client.17409 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f94a21ae740 1 -- 172.21.15.5:0/768614255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a994e13140 con 0x55a994e015d0 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f947d7fa640 1 -- 172.21.15.5:0/768614255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9490018040 con 0x55a994e015d0 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f947d7fa640 1 -- 172.21.15.5:0/768614255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f948c015040 con 0x55a994e015d0 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f94a21ae740 10 client.17409 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f94a21ae740 10 client.17409 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f94a21ae740 20 client.17409 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f94a21ae740 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55a994e86d30 0x55a994e89120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f94a21ae740 4 client.17409 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f94a21ae740 1 -- 172.21.15.5:0/768614255 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a994d6ed70 con 0x55a994e86d30 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.660+0000 7f949db71640 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55a994e86d30 0x55a994e89120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f949db71640 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55a994e86d30 0x55a994e89120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a92d5a2b1c434f8 server_cookie=aa690518fffa0037 in_seq=0 out_seq=0 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f947d7fa640 10 client.17409 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f947d7fa640 1 -- 172.21.15.5:0/768614255 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a994d6ed70 con 0x55a994e86d30 2024-09-11T22:28:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f947d7fa640 10 client.17409 handle_command_reply: tid=0 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 client.17409 shutdown 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 -- 172.21.15.5:0/768614255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 msgr2=0x55a994e148a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e148a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f948c006fd0 tx=0x7f948c018a30 comp rx=0 tx=0).stop 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 -- 172.21.15.5:0/768614255 shutdown_connections 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55a994e86d30 0x55a994e89120 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a994e0c370 0x55a994e23170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a994e02b40 0x55a994e14de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f947d7fa640 0 client.17409 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 --2- 172.21.15.5:0/768614255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a994e015d0 0x55a994e148a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 -- 172.21.15.5:0/768614255 >> 172.21.15.5:0/768614255 conn(0x55a994dca310 msgr2=0x55a994e0b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 -- 172.21.15.5:0/768614255 shutdown_connections 2024-09-11T22:28:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 1 -- 172.21.15.5:0/768614255 wait complete. 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.661+0000 7f94a21ae740 20 client.17409 trim_cache size 0 max 16384 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 -- 172.21.15.5:0/3766700077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94800a2cf0 msgr2=0x7f94800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94800a2cf0 0x7f94800a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f94880ede70 tx=0x7f948802a950 comp rx=0 tx=0).stop 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 -- 172.21.15.5:0/3766700077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9498102ef0 msgr2=0x7f949819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9498102ef0 0x7f949819a620 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9488006010 tx=0x7f948802f6f0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 -- 172.21.15.5:0/3766700077 shutdown_connections 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9498102ef0 0x7f949819a620 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94800a2cf0 0x7f94800a51b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9498102570 0x7f94981a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 --2- 172.21.15.5:0/3766700077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9498100e20 0x7f94981a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 -- 172.21.15.5:0/3766700077 >> 172.21.15.5:0/3766700077 conn(0x7f94980fc7d0 msgr2=0x7f949806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 -- 172.21.15.5:0/3766700077 shutdown_connections 2024-09-11T22:28:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:43.662+0000 7f949fdd5640 1 -- 172.21.15.5:0/3766700077 wait complete. 2024-09-11T22:28:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:43 smithi005 ceph-mon[30943]: pgmap v201: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-11T22:28:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:44.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:43 smithi159 ceph-mon[35534]: pgmap v201: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-11T22:28:44.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:43 smithi172 ceph-mon[35461]: pgmap v201: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-11T22:28:44.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476167 7.84 MB/sec execute 182 sec latency 20810.525 ms 2024-09-11T22:28:44.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:28:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.987+0000 7f6c77a80640 1 -- 172.21.15.5:0/3167573358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c70100e20 msgr2=0x7f6c70101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.987+0000 7f6c77a80640 1 --2- 172.21.15.5:0/3167573358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c70100e20 0x7f6c70101200 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6c60009900 tx=0x7f6c6002a480 comp rx=0 tx=0).stop 2024-09-11T22:28:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.987+0000 7f6c77a80640 1 -- 172.21.15.5:0/3167573358 shutdown_connections 2024-09-11T22:28:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.987+0000 7f6c77a80640 1 --2- 172.21.15.5:0/3167573358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70102ef0 0x7f6c7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.987+0000 7f6c77a80640 1 --2- 172.21.15.5:0/3167573358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70102570 0x7f6c701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.987+0000 7f6c77a80640 1 --2- 172.21.15.5:0/3167573358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c70100e20 0x7f6c70101200 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.987+0000 7f6c77a80640 1 -- 172.21.15.5:0/3167573358 >> 172.21.15.5:0/3167573358 conn(0x7f6c700fc7d0 msgr2=0x7f6c700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.987+0000 7f6c77a80640 1 -- 172.21.15.5:0/3167573358 shutdown_connections 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.987+0000 7f6c77a80640 1 -- 172.21.15.5:0/3167573358 wait complete. 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c77a80640 1 Processor -- start 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c77a80640 1 -- start start 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c77a80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c70100e20 0x7f6c70198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c77a80640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70102570 0x7f6c70198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c77a80640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70102ef0 0x7f6c7019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c77a80640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c70110f50 con 0x7f6c70100e20 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c77a80640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c70110dd0 con 0x7f6c70102570 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c77a80640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c701110d0 con 0x7f6c70102ef0 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c7581c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c70100e20 0x7f6c70198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c7581c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c70100e20 0x7f6c70198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48530/0 (socket says 172.21.15.5:48530) 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c7581c640 1 -- 172.21.15.5:0/3180804179 learned_addr learned my addr 172.21.15.5:0/3180804179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c7601d640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70102ef0 0x7f6c7019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c7501b640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70102570 0x7f6c70198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c7581c640 1 -- 172.21.15.5:0/3180804179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70102ef0 msgr2=0x7f6c7019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c7581c640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70102ef0 0x7f6c7019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.988+0000 7f6c7581c640 1 -- 172.21.15.5:0/3180804179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70102570 msgr2=0x7f6c70198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.989+0000 7f6c7581c640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70102570 0x7f6c70198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.989+0000 7f6c7581c640 1 -- 172.21.15.5:0/3180804179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7019d4d0 con 0x7f6c70100e20 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.989+0000 7f6c7581c640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c70100e20 0x7f6c70198520 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6c60006010 tx=0x7f6c60003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.989+0000 7f6c6effd640 1 -- 172.21.15.5:0/3180804179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6003f070 con 0x7f6c70100e20 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.989+0000 7f6c6effd640 1 -- 172.21.15.5:0/3180804179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c60031c30 con 0x7f6c70100e20 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.989+0000 7f6c77a80640 1 -- 172.21.15.5:0/3180804179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7019d760 con 0x7f6c70100e20 2024-09-11T22:28:45.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.989+0000 7f6c6effd640 1 -- 172.21.15.5:0/3180804179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60035030 con 0x7f6c70100e20 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.989+0000 7f6c77a80640 1 -- 172.21.15.5:0/3180804179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c701a50a0 con 0x7f6c70100e20 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.991+0000 7f6c6effd640 1 -- 172.21.15.5:0/3180804179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6c60042070 con 0x7f6c70100e20 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c79e59740 1 Processor -- start 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c79e59740 1 -- start start 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x5591686ee440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686f9240 0x5591686fb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c79e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559168653620 con 0x5591686ee980 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c79e59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5591685fae50 con 0x5591686ee060 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c79e59740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55916864f5e0 con 0x5591686f9240 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c7581c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c7601d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x5591686ee440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c6effd640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c580a2d70 0x7f6c580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c7581c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48534/0 (socket says 172.21.15.5:48534) 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c7601d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x5591686ee440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56844/0 (socket says 172.21.15.5:56844) 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c7581c640 1 -- 172.21.15.5:0/874296933 learned_addr learned my addr 172.21.15.5:0/874296933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.992+0000 7f6c7501b640 1 --2- 172.21.15.5:0/874296933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686f9240 0x5591686fb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c7601d640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c580a2d70 0x7f6c580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c6effd640 1 -- 172.21.15.5:0/3180804179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f6c600efaa0 con 0x7f6c70100e20 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c7581c640 1 -- 172.21.15.5:0/874296933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686f9240 msgr2=0x5591686fb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c7581c640 1 --2- 172.21.15.5:0/874296933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686f9240 0x5591686fb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c7581c640 1 -- 172.21.15.5:0/874296933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 msgr2=0x5591686ee440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c7581c640 1 --2- 172.21.15.5:0/874296933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x5591686ee440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c7581c640 1 -- 172.21.15.5:0/874296933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591685ebd40 con 0x5591686ee980 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c7581c640 1 --2- 172.21.15.5:0/874296933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686f8d00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6c60046000 tx=0x7f6c6003b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12214e1e489a8491 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c7601d640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c580a2d70 0x7f6c580a5230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009fd0 tx=0x7f6c68004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c57fff640 1 -- 172.21.15.5:0/874296933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60049070 con 0x5591686ee980 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c57fff640 1 -- 172.21.15.5:0/874296933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c600b8640 con 0x5591686ee980 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.993+0000 7f6c57fff640 1 -- 172.21.15.5:0/874296933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6003be50 con 0x5591686ee980 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.994+0000 7f6c79e59740 1 -- 172.21.15.5:0/874296933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 msgr2=0x5591686f8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.994+0000 7f6c79e59740 1 --2- 172.21.15.5:0/874296933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686f8d00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6c60046000 tx=0x7f6c6003b790 comp rx=0 tx=0).stop 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.994+0000 7f6c79e59740 1 -- 172.21.15.5:0/874296933 shutdown_connections 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.994+0000 7f6c79e59740 1 --2- 172.21.15.5:0/874296933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686f9240 0x5591686fb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.994+0000 7f6c79e59740 1 --2- 172.21.15.5:0/874296933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686f8d00 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.994+0000 7f6c79e59740 1 --2- 172.21.15.5:0/874296933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x5591686ee440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.994+0000 7f6c79e59740 1 -- 172.21.15.5:0/874296933 >> 172.21.15.5:0/874296933 conn(0x5591686ea5f0 msgr2=0x5591686ea9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.994+0000 7f6c79e59740 1 -- 172.21.15.5:0/874296933 shutdown_connections 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.994+0000 7f6c79e59740 1 -- 172.21.15.5:0/874296933 wait complete. 2024-09-11T22:28:45.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.994+0000 7f6c79e59740 1 -- start start 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c79e59740 1 Processor -- start 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686ee060 0x5591686f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee980 0x5591686f6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591686f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c79e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5591686fbbd0 con 0x5591686f9240 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c79e59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5591685dbf10 con 0x5591686ee980 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c79e59740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5591686fbd50 con 0x5591686ee060 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c79e59740 10 client.0 _ll_register_callbacks cb 0x5591685ffdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c7601d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591686f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c7601d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591686f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48548/0 (socket says 172.21.15.5:48548) 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c7601d640 1 -- 172.21.15.5:0/2619849415 learned_addr learned my addr 172.21.15.5:0/2619849415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c7501b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee980 0x5591686f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c7581c640 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686ee060 0x5591686f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c7601d640 1 -- 172.21.15.5:0/2619849415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686ee060 msgr2=0x5591686f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c7601d640 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686ee060 0x5591686f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.995+0000 7f6c7601d640 1 -- 172.21.15.5:0/2619849415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee980 msgr2=0x5591686f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c7601d640 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee980 0x5591686f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c7601d640 1 -- 172.21.15.5:0/2619849415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559168362580 con 0x5591686f9240 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c7601d640 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591686f6c60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6c680133f0 tx=0x7f6c68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60fadc70678cea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c557fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c557fa640 1 -- 172.21.15.5:0/2619849415 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68013cb0 con 0x5591686f9240 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c557fa640 1 -- 172.21.15.5:0/2619849415 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f6c68021440 con 0x5591686f9240 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c79e59740 1 -- 172.21.15.5:0/2619849415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559168660b90 con 0x5591686f9240 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c557fa640 1 -- 172.21.15.5:0/2619849415 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6c68013e50 con 0x5591686f9240 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c79e59740 10 client.17421 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c79e59740 1 -- 172.21.15.5:0/2619849415 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55916842c960 con 0x5591686f9240 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c557fa640 1 -- 172.21.15.5:0/2619849415 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559168660b90 con 0x5591686f9240 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c557fa640 1 -- 172.21.15.5:0/2619849415 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6c6802b6b0 con 0x5591686f9240 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c79e59740 10 client.17421 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.996+0000 7f6c79e59740 10 client.17421 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.997+0000 7f6c79e59740 20 client.17421 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.997+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559168708bd0 0x55916870b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.997+0000 7f6c79e59740 4 client.17421 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.997+0000 7f6c79e59740 1 -- 172.21.15.5:0/2619849415 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55916865b0a0 con 0x559168708bd0 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.997+0000 7f6c7581c640 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559168708bd0 0x55916870b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.997+0000 7f6c557fa640 10 client.17421 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.997+0000 7f6c7581c640 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559168708bd0 0x55916870b010 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46b3ef0483e11 server_cookie=8c8ea5109fb583fd in_seq=0 out_seq=0 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c557fa640 1 -- 172.21.15.5:0/2619849415 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55916865b0a0 con 0x559168708bd0 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c557fa640 10 client.17421 handle_command_reply: tid=0 2024-09-11T22:28:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c79e59740 1 client.17421 shutdown 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c79e59740 1 -- 172.21.15.5:0/2619849415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 msgr2=0x5591686f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591686f6c60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6c680133f0 tx=0x7f6c68004a40 comp rx=0 tx=0).stop 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c79e59740 1 -- 172.21.15.5:0/2619849415 shutdown_connections 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559168708bd0 0x55916870b010 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591686f6c60 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee980 0x5591686f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2619849415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686ee060 0x5591686f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c557fa640 0 client.17421 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.998+0000 7f6c79e59740 1 -- 172.21.15.5:0/2619849415 >> 172.21.15.5:0/2619849415 conn(0x5591686ea5f0 msgr2=0x5591686f0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.999+0000 7f6c79e59740 1 -- 172.21.15.5:0/2619849415 shutdown_connections 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.999+0000 7f6c79e59740 1 -- 172.21.15.5:0/2619849415 wait complete. 2024-09-11T22:28:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:44.999+0000 7f6c79e59740 20 client.17421 trim_cache size 0 max 16384 2024-09-11T22:28:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.016+0000 7f6c79e59740 1 Processor -- start 2024-09-11T22:28:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.016+0000 7f6c79e59740 1 -- start start 2024-09-11T22:28:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x5591686f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686ee980 0x5591686f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591687108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c79e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c701ada20 con 0x5591686f9240 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c79e59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c701af560 con 0x5591686ee060 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c79e59740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c701ae0f0 con 0x5591686ee980 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c7601d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591687108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c7601d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591687108e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48554/0 (socket says 172.21.15.5:48554) 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c7601d640 1 -- 172.21.15.5:0/3747508881 learned_addr learned my addr 172.21.15.5:0/3747508881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c7501b640 1 --2- 172.21.15.5:0/3747508881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686ee980 0x5591686f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c7581c640 1 --2- 172.21.15.5:0/3747508881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x5591686f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c7601d640 1 -- 172.21.15.5:0/3747508881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686ee980 msgr2=0x5591686f0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c7601d640 1 --2- 172.21.15.5:0/3747508881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686ee980 0x5591686f0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c7601d640 1 -- 172.21.15.5:0/3747508881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 msgr2=0x5591686f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c7601d640 1 --2- 172.21.15.5:0/3747508881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x5591686f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.017+0000 7f6c7601d640 1 -- 172.21.15.5:0/3747508881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591686eb280 con 0x5591686f9240 2024-09-11T22:28:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.018+0000 7f6c7601d640 1 --2- 172.21.15.5:0/3747508881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591687108e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009c50 tx=0x7f6c68009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96aacb9af3be726 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.018+0000 7f6c577fe640 1 -- 172.21.15.5:0/3747508881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68037070 con 0x5591686f9240 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.018+0000 7f6c577fe640 1 -- 172.21.15.5:0/3747508881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c68013e50 con 0x5591686f9240 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.018+0000 7f6c577fe640 1 -- 172.21.15.5:0/3747508881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68005020 con 0x5591686f9240 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.018+0000 7f6c79e59740 1 -- 172.21.15.5:0/3747508881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 msgr2=0x5591687108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.019+0000 7f6c79e59740 1 --2- 172.21.15.5:0/3747508881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591687108e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009c50 tx=0x7f6c68009cf0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.019+0000 7f6c79e59740 1 -- 172.21.15.5:0/3747508881 shutdown_connections 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.019+0000 7f6c79e59740 1 --2- 172.21.15.5:0/3747508881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686f9240 0x5591687108e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.019+0000 7f6c79e59740 1 --2- 172.21.15.5:0/3747508881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686ee980 0x5591686f0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.019+0000 7f6c79e59740 1 --2- 172.21.15.5:0/3747508881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x5591686f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.019+0000 7f6c79e59740 1 -- 172.21.15.5:0/3747508881 >> 172.21.15.5:0/3747508881 conn(0x5591686ea5f0 msgr2=0x5591686e9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.019+0000 7f6c79e59740 1 -- 172.21.15.5:0/3747508881 shutdown_connections 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.019+0000 7f6c79e59740 1 -- 172.21.15.5:0/3747508881 wait complete. 2024-09-11T22:28:45.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.019+0000 7f6c79e59740 1 -- start start 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.019+0000 7f6c79e59740 1 Processor -- start 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x559168701410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686fd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c79e59740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686f9240 0x5591686fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c79e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c701aee90 con 0x5591686ee980 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c79e59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c701ae7c0 con 0x5591686ee060 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c79e59740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6c701aba00 con 0x5591686f9240 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c79e59740 10 client.0 _ll_register_callbacks cb 0x7f6c6800fd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7501b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7501b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686fd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48562/0 (socket says 172.21.15.5:48562) 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7501b640 1 -- 172.21.15.5:0/2459072551 learned_addr learned my addr 172.21.15.5:0/2459072551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7601d640 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686f9240 0x5591686fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7581c640 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x559168701410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7501b640 1 -- 172.21.15.5:0/2459072551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686f9240 msgr2=0x5591686fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7501b640 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686f9240 0x5591686fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7501b640 1 -- 172.21.15.5:0/2459072551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 msgr2=0x559168701410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7501b640 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x559168701410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7501b640 1 -- 172.21.15.5:0/2459072551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591686fe330 con 0x5591686ee980 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c7501b640 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686fd750 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005d00 tx=0x7f6c64012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc553d0eaeedea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.020+0000 7f6c557fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c557fa640 1 -- 172.21.15.5:0/2459072551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c64021030 con 0x5591686ee980 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c557fa640 1 -- 172.21.15.5:0/2459072551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f6c64010c30 con 0x5591686ee980 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c79e59740 1 -- 172.21.15.5:0/2459072551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c68013e50 con 0x5591686ee980 2024-09-11T22:28:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c557fa640 1 -- 172.21.15.5:0/2459072551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6c64010e30 con 0x5591686ee980 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c79e59740 10 client.17427 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c79e59740 1 -- 172.21.15.5:0/2459072551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5591687747f0 con 0x5591686ee980 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c557fa640 1 -- 172.21.15.5:0/2459072551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c68013e50 con 0x5591686ee980 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c557fa640 1 -- 172.21.15.5:0/2459072551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f6c6401d440 con 0x5591686ee980 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c79e59740 10 client.17427 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c79e59740 10 client.17427 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c79e59740 20 client.17427 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559168774fc0 0x5591687773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.021+0000 7f6c79e59740 4 client.17427 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e5aba961", "release": true, "await": true}] 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.022+0000 7f6c79e59740 1 -- 172.21.15.5:0/2459072551 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e5aba961", "release": true, "await": true}) -- 0x5591686e5950 con 0x559168774fc0 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.022+0000 7f6c7581c640 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559168774fc0 0x5591687773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.022+0000 7f6c7581c640 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559168774fc0 0x5591687773e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64888bf26f173af7 server_cookie=60554f238029b5d2 in_seq=0 out_seq=0 2024-09-11T22:28:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.022+0000 7f6c557fa640 10 client.17427 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:45.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c557fa640 1 -- 172.21.15.5:0/2459072551 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5591686e5950 con 0x559168774fc0 2024-09-11T22:28:45.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c557fa640 10 client.17427 handle_command_reply: tid=0 2024-09-11T22:28:45.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c79e59740 1 client.17427 shutdown 2024-09-11T22:28:45.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c79e59740 1 -- 172.21.15.5:0/2459072551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 msgr2=0x5591686fd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686fd750 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005d00 tx=0x7f6c64012b60 comp rx=0 tx=0).stop 2024-09-11T22:28:45.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c79e59740 1 -- 172.21.15.5:0/2459072551 shutdown_connections 2024-09-11T22:28:45.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559168774fc0 0x5591687773e0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591686f9240 0x5591686fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591686ee980 0x5591686fd750 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c79e59740 1 --2- 172.21.15.5:0/2459072551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591686ee060 0x559168701410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.036+0000 7f6c79e59740 1 -- 172.21.15.5:0/2459072551 >> 172.21.15.5:0/2459072551 conn(0x5591686ea5f0 msgr2=0x5591686e8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.037+0000 7f6c557fa640 0 client.17427 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.037+0000 7f6c79e59740 1 -- 172.21.15.5:0/2459072551 shutdown_connections 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.037+0000 7f6c79e59740 1 -- 172.21.15.5:0/2459072551 wait complete. 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.037+0000 7f6c79e59740 20 client.17427 trim_cache size 0 max 16384 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 38, 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 24, 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stdout: "e5aba961": { 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 24, 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:28:45.039 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.1, 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:45.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:45.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:45.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:45.041 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:28:45.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 -- 172.21.15.5:0/3180804179 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c580a2d70 msgr2=0x7f6c580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c580a2d70 0x7f6c580a5230 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009fd0 tx=0x7f6c68004ab0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 -- 172.21.15.5:0/3180804179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c70100e20 msgr2=0x7f6c70198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c70100e20 0x7f6c70198520 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6c60006010 tx=0x7f6c60003c10 comp rx=0 tx=0).stop 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 -- 172.21.15.5:0/3180804179 shutdown_connections 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c70102ef0 0x7f6c7019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6c580a2d70 0x7f6c580a5230 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c70102570 0x7f6c70198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 --2- 172.21.15.5:0/3180804179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c70100e20 0x7f6c70198520 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 -- 172.21.15.5:0/3180804179 >> 172.21.15.5:0/3180804179 conn(0x7f6c700fc7d0 msgr2=0x7f6c7006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 -- 172.21.15.5:0/3180804179 shutdown_connections 2024-09-11T22:28:45.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.038+0000 7f6c77a80640 1 -- 172.21.15.5:0/3180804179 wait complete. 2024-09-11T22:28:45.262 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:45.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476529 7.81 MB/sec execute 183 sec latency 21530.999 ms 2024-09-11T22:28:45.645 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:45 smithi005 ceph-mon[30943]: pgmap v202: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-11T22:28:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.899+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1653127305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4102ef0 msgr2=0x7ffbf410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.899+0000 7ffbf9cbd640 1 --2- 172.21.15.5:0/1653127305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4102ef0 0x7ffbf410da40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ffbec009ca0 tx=0x7ffbec02a950 comp rx=0 tx=0).stop 2024-09-11T22:28:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.899+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1653127305 shutdown_connections 2024-09-11T22:28:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.899+0000 7ffbf9cbd640 1 --2- 172.21.15.5:0/1653127305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4102ef0 0x7ffbf410da40 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.899+0000 7ffbf9cbd640 1 --2- 172.21.15.5:0/1653127305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4102570 0x7ffbf41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.899+0000 7ffbf9cbd640 1 --2- 172.21.15.5:0/1653127305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4100e20 0x7ffbf4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.899+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1653127305 >> 172.21.15.5:0/1653127305 conn(0x7ffbf40fc7d0 msgr2=0x7ffbf40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.899+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1653127305 shutdown_connections 2024-09-11T22:28:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.899+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1653127305 wait complete. 2024-09-11T22:28:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.899+0000 7ffbf9cbd640 1 Processor -- start 2024-09-11T22:28:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf9cbd640 1 -- start start 2024-09-11T22:28:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf9cbd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4100e20 0x7ffbf40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf9cbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4102570 0x7ffbf40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf9cbd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4102ef0 0x7ffbf40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf9cbd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf4110f80 con 0x7ffbf4102570 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf9cbd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf4110e00 con 0x7ffbf4100e20 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf9cbd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf4111100 con 0x7ffbf4102ef0 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4100e20 0x7ffbf40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbeaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4102570 0x7ffbf40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4102ef0 0x7ffbf40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4100e20 0x7ffbf40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56890/0 (socket says 172.21.15.5:56890) 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbeaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4102570 0x7ffbf40fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48588/0 (socket says 172.21.15.5:48588) 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4102ef0 0x7ffbf40fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:53566/0 (socket says 172.21.15.5:53566) 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf37fe640 1 -- 172.21.15.5:0/1362996508 learned_addr learned my addr 172.21.15.5:0/1362996508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf37fe640 1 -- 172.21.15.5:0/1362996508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4102ef0 msgr2=0x7ffbf40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf37fe640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4102ef0 0x7ffbf40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf37fe640 1 -- 172.21.15.5:0/1362996508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4102570 msgr2=0x7ffbf40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf37fe640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4102570 0x7ffbf40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbf37fe640 1 -- 172.21.15.5:0/1362996508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf41a69c0 con 0x7ffbf4100e20 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.900+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4102570 0x7ffbf40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.901+0000 7ffbf37fe640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4100e20 0x7ffbf40fcc80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e9f0 tx=0x7ffbe400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.901+0000 7ffbf17fa640 1 -- 172.21.15.5:0/1362996508 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe400cd80 con 0x7ffbf4100e20 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.901+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1362996508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf41a6cb0 con 0x7ffbf4100e20 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.901+0000 7ffbf17fa640 1 -- 172.21.15.5:0/1362996508 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbe4010910 con 0x7ffbf4100e20 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.901+0000 7ffbf17fa640 1 -- 172.21.15.5:0/1362996508 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe40188f0 con 0x7ffbf4100e20 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.901+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1362996508 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf41a71e0 con 0x7ffbf4100e20 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.903+0000 7ffbf17fa640 1 -- 172.21.15.5:0/1362996508 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ffbe4010ac0 con 0x7ffbf4100e20 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbfc096740 1 Processor -- start 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbfc096740 1 -- start start 2024-09-11T22:28:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9db40 0x555db4a9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4a9ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbf17fa640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2cf0 0x7ffbe00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4aa7370 0x555db4aa9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbfc096740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555db4a022f0 con 0x555db4aa7370 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbfc096740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555db49a9cf0 con 0x555db4a9db40 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbfc096740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555db49fe2b0 con 0x555db4a9c5d0 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbf37fe640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2cf0 0x7ffbe00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9db40 0x555db4a9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbf17fa640 1 -- 172.21.15.5:0/1362996508 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7ffbe40cf8a0 con 0x7ffbf4100e20 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4aa7370 0x555db4aa9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4aa7370 0x555db4aa9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48592/0 (socket says 172.21.15.5:48592) 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbeaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4a9ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.904+0000 7ffbf3fff640 1 -- 172.21.15.5:0/1508082362 learned_addr learned my addr 172.21.15.5:0/1508082362 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.905+0000 7ffbeaffd640 1 -- 172.21.15.5:0/1508082362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9db40 msgr2=0x555db4a9c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.905+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/1508082362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9db40 0x555db4a9c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.905+0000 7ffbeaffd640 1 -- 172.21.15.5:0/1508082362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4aa7370 msgr2=0x555db4aa9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.905+0000 7ffbf37fe640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2cf0 0x7ffbe00a51b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e9c0 tx=0x7ffbe400cf20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.905+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/1508082362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4aa7370 0x555db4aa9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.905+0000 7ffbeaffd640 1 -- 172.21.15.5:0/1508082362 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db498e3a0 con 0x555db4a9c5d0 2024-09-11T22:28:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.905+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/1508082362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4a9ca10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009f80 tx=0x7ffbdc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7003347504c6c3f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.905+0000 7ffbe9ffb640 1 -- 172.21.15.5:0/1508082362 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc004310 con 0x555db4a9c5d0 2024-09-11T22:28:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.905+0000 7ffbe9ffb640 1 -- 172.21.15.5:0/1508082362 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc0044b0 con 0x555db4a9c5d0 2024-09-11T22:28:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.905+0000 7ffbe9ffb640 1 -- 172.21.15.5:0/1508082362 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc0035f0 con 0x555db4a9c5d0 2024-09-11T22:28:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.906+0000 7ffbfc096740 1 -- 172.21.15.5:0/1508082362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 msgr2=0x555db4a9ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.906+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1508082362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4a9ca10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009f80 tx=0x7ffbdc0118d0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.906+0000 7ffbfc096740 1 -- 172.21.15.5:0/1508082362 shutdown_connections 2024-09-11T22:28:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.906+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1508082362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4aa7370 0x555db4aa9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.906+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1508082362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4a9ca10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.906+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1508082362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9db40 0x555db4a9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.906+0000 7ffbfc096740 1 -- 172.21.15.5:0/1508082362 >> 172.21.15.5:0/1508082362 conn(0x555db4a65310 msgr2=0x555db4a98c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.906+0000 7ffbfc096740 1 -- 172.21.15.5:0/1508082362 shutdown_connections 2024-09-11T22:28:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.906+0000 7ffbfc096740 1 -- 172.21.15.5:0/1508082362 wait complete. 2024-09-11T22:28:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.906+0000 7ffbfc096740 1 -- start start 2024-09-11T22:28:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbfc096740 1 Processor -- start 2024-09-11T22:28:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9c5d0 0x555db4aa6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 0x555db4aa3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4aa7370 0x555db4aa4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbfc096740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555db4aaa070 con 0x555db4a9db40 2024-09-11T22:28:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbfc096740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555db49bd060 con 0x555db4a9c5d0 2024-09-11T22:28:45.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbfc096740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555db4aaa1f0 con 0x555db4aa7370 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbfc096740 10 client.0 _ll_register_callbacks cb 0x555db47108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbeaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 0x555db4aa3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbeaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 0x555db4aa3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48602/0 (socket says 172.21.15.5:48602) 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbeaffd640 1 -- 172.21.15.5:0/981791350 learned_addr learned my addr 172.21.15.5:0/981791350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbf3fff640 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9c5d0 0x555db4aa6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbf3fff640 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4aa7370 0x555db4aa4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbeaffd640 1 -- 172.21.15.5:0/981791350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4aa7370 msgr2=0x555db4aa4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4aa7370 0x555db4aa4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbeaffd640 1 -- 172.21.15.5:0/981791350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9c5d0 msgr2=0x555db4aa6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9c5d0 0x555db4aa6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.907+0000 7ffbeaffd640 1 -- 172.21.15.5:0/981791350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db47733c0 con 0x555db4a9db40 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbf3fff640 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4aa7370 0x555db4aa4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:28:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 0x555db4aa3f20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00a010 tx=0x7ffbdc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e282736a3ac8551e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/981791350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc029070 con 0x555db4a9db40 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/981791350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7ffbdc003a00 con 0x555db4a9db40 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbfc096740 1 -- 172.21.15.5:0/981791350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbdc0044b0 con 0x555db4a9db40 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/981791350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ffbdc035050 con 0x555db4a9db40 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbfc096740 10 client.17439 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbfc096740 1 -- 172.21.15.5:0/981791350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555db4ab5a80 con 0x555db4a9db40 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/981791350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc0044b0 con 0x555db4a9db40 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.908+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/981791350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7ffbdc00f480 con 0x555db4a9db40 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.909+0000 7ffbfc096740 10 client.17439 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.909+0000 7ffbfc096740 10 client.17439 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.909+0000 7ffbfc096740 20 client.17439 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.909+0000 7ffbfc096740 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555db4ab6dd0 0x555db4ab91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.909+0000 7ffbfc096740 4 client.17439 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.909+0000 7ffbfc096740 1 -- 172.21.15.5:0/981791350 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555db4a0f860 con 0x555db4ab6dd0 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.909+0000 7ffbf3fff640 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555db4ab6dd0 0x555db4ab91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.909+0000 7ffbf3fff640 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555db4ab6dd0 0x555db4ab91c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=91c9d6a1ed2f51ea server_cookie=8f870070924210c9 in_seq=0 out_seq=0 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.909+0000 7ffbd2ffd640 10 client.17439 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/981791350 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555db4a0f860 con 0x555db4ab6dd0 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbd2ffd640 10 client.17439 handle_command_reply: tid=0 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbfc096740 1 client.17439 shutdown 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbfc096740 1 -- 172.21.15.5:0/981791350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 msgr2=0x555db4aa3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbfc096740 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 0x555db4aa3f20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00a010 tx=0x7ffbdc0043c0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbfc096740 1 -- 172.21.15.5:0/981791350 shutdown_connections 2024-09-11T22:28:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbfc096740 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4aa7370 0x555db4aa4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbfc096740 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 0x555db4aa3f20 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbfc096740 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555db4ab6dd0 0x555db4ab91c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbfc096740 1 --2- 172.21.15.5:0/981791350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9c5d0 0x555db4aa6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbfc096740 1 -- 172.21.15.5:0/981791350 >> 172.21.15.5:0/981791350 conn(0x555db4a65310 msgr2=0x555db4a9e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.910+0000 7ffbd2ffd640 0 client.17439 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.911+0000 7ffbfc096740 1 -- 172.21.15.5:0/981791350 shutdown_connections 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.911+0000 7ffbfc096740 1 -- 172.21.15.5:0/981791350 wait complete. 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.911+0000 7ffbfc096740 20 client.17439 trim_cache size 0 max 16384 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.925+0000 7ffbfc096740 1 Processor -- start 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.925+0000 7ffbfc096740 1 -- start start 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a65310 0x555db4ab3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4aa5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 0x555db4abea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbfc096740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41afc50 con 0x555db4a9db40 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbfc096740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41b1790 con 0x555db4a65310 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbfc096740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41b0320 con 0x555db4a9c5d0 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a65310 0x555db4ab3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbeaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4aa5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbeaffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4aa5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:53592/0 (socket says 172.21.15.5:53592) 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbeaffd640 1 -- 172.21.15.5:0/1790649157 learned_addr learned my addr 172.21.15.5:0/1790649157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a65310 0x555db4ab3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56928/0 (socket says 172.21.15.5:56928) 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbf3fff640 1 --2- 172.21.15.5:0/1790649157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 0x555db4abea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbeaffd640 1 -- 172.21.15.5:0/1790649157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a65310 msgr2=0x555db4ab3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/1790649157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a65310 0x555db4ab3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbeaffd640 1 -- 172.21.15.5:0/1790649157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 msgr2=0x555db4abea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/1790649157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 0x555db4abea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbeaffd640 1 -- 172.21.15.5:0/1790649157 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db4aa3480 con 0x555db4a9c5d0 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.926+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/1790649157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4aa5e90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00f8d0 tx=0x7ffbdc009e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4ffe8bd2614b412 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.927+0000 7ffbe97fa640 1 -- 172.21.15.5:0/1790649157 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc029070 con 0x555db4a9c5d0 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.927+0000 7ffbe97fa640 1 -- 172.21.15.5:0/1790649157 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc004430 con 0x555db4a9c5d0 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.927+0000 7ffbe97fa640 1 -- 172.21.15.5:0/1790649157 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc02e440 con 0x555db4a9c5d0 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.927+0000 7ffbfc096740 1 -- 172.21.15.5:0/1790649157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 msgr2=0x555db4aa5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.927+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1790649157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4aa5e90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc00f8d0 tx=0x7ffbdc009e00 comp rx=0 tx=0).stop 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.927+0000 7ffbfc096740 1 -- 172.21.15.5:0/1790649157 shutdown_connections 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.927+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1790649157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a9db40 0x555db4abea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.927+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1790649157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4aa5e90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.927+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1790649157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a65310 0x555db4ab3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.927+0000 7ffbfc096740 1 -- 172.21.15.5:0/1790649157 >> 172.21.15.5:0/1790649157 conn(0x555db4aa45c0 msgr2=0x555db4aa5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 1 -- 172.21.15.5:0/1790649157 shutdown_connections 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 1 -- 172.21.15.5:0/1790649157 wait complete. 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 1 -- start start 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 1 Processor -- start 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a65310 0x555db4aaf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4aafd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9db40 0x555db4aa0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41b10c0 con 0x555db4a65310 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41b09f0 con 0x555db4a9db40 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41adc30 con 0x555db4a9c5d0 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbfc096740 10 client.0 _ll_register_callbacks cb 0x7ffbdc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.928+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a65310 0x555db4aaf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a65310 0x555db4aaf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48624/0 (socket says 172.21.15.5:48624) 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbf3fff640 1 -- 172.21.15.5:0/1507617273 learned_addr learned my addr 172.21.15.5:0/1507617273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4aafd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbf3fff640 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9db40 0x555db4aa0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbf3fff640 1 -- 172.21.15.5:0/1507617273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 msgr2=0x555db4aafd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbf3fff640 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4aafd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbf3fff640 1 -- 172.21.15.5:0/1507617273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9db40 msgr2=0x555db4aa0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbf3fff640 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9db40 0x555db4aa0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbf3fff640 1 -- 172.21.15.5:0/1507617273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db4aa1400 con 0x555db4a65310 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbf3fff640 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a65310 0x555db4aaf7f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ffbec02a990 tx=0x7ffbec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1657d72368b62b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/1507617273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec05c060 con 0x555db4a65310 2024-09-11T22:28:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/1507617273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7ffbec030700 con 0x555db4a65310 2024-09-11T22:28:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbfc096740 1 -- 172.21.15.5:0/1507617273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbdc004430 con 0x555db4a65310 2024-09-11T22:28:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/1507617273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ffbec044040 con 0x555db4a65310 2024-09-11T22:28:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbfc096740 10 client.17445 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.929+0000 7ffbfc096740 1 -- 172.21.15.5:0/1507617273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555db4b21670 con 0x555db4a65310 2024-09-11T22:28:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/1507617273 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc004430 con 0x555db4a65310 2024-09-11T22:28:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/1507617273 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7ffbec054030 con 0x555db4a65310 2024-09-11T22:28:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbfc096740 10 client.17445 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbfc096740 10 client.17445 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:28:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbfc096740 20 client.17445 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555db4aa7370 0x555db4b23de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbfc096740 4 client.17445 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:28:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbfc096740 1 -- 172.21.15.5:0/1507617273 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555db4a09d70 con 0x555db4aa7370 2024-09-11T22:28:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555db4aa7370 0x555db4b23de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbeaffd640 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555db4aa7370 0x555db4b23de0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88c901b265fb2cf3 server_cookie=dd1c8778839545a0 in_seq=0 out_seq=0 2024-09-11T22:28:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.930+0000 7ffbd2ffd640 10 client.17445 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:28:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/1507617273 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555db4a09d70 con 0x555db4aa7370 2024-09-11T22:28:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbd2ffd640 10 client.17445 handle_command_reply: tid=0 2024-09-11T22:28:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 client.17445 shutdown 2024-09-11T22:28:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 -- 172.21.15.5:0/1507617273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a65310 msgr2=0x555db4aaf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a65310 0x555db4aaf7f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7ffbec02a990 tx=0x7ffbec0095c0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 -- 172.21.15.5:0/1507617273 shutdown_connections 2024-09-11T22:28:45.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555db4aa7370 0x555db4b23de0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555db4a9db40 0x555db4aa0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555db4a9c5d0 0x555db4aafd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 --2- 172.21.15.5:0/1507617273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db4a65310 0x555db4aaf7f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.954 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 -- 172.21.15.5:0/1507617273 >> 172.21.15.5:0/1507617273 conn(0x555db4aa45c0 msgr2=0x555db4aa2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbd2ffd640 0 client.17445 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 -- 172.21.15.5:0/1507617273 shutdown_connections 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 1 -- 172.21.15.5:0/1507617273 wait complete. 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.931+0000 7ffbfc096740 20 client.17445 trim_cache size 0 max 16384 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1362996508 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2cf0 msgr2=0x7ffbe00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2cf0 0x7ffbe00a51b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e9c0 tx=0x7ffbe400cf20 comp rx=0 tx=0).stop 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1362996508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4100e20 msgr2=0x7ffbf40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4100e20 0x7ffbf40fcc80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffbe400e9f0 tx=0x7ffbe400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1362996508 shutdown_connections 2024-09-11T22:28:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf4102ef0 0x7ffbf40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4102570 0x7ffbf40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2cf0 0x7ffbe00a51b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 --2- 172.21.15.5:0/1362996508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4100e20 0x7ffbf40fcc80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:45.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1362996508 >> 172.21.15.5:0/1362996508 conn(0x7ffbf40fc7d0 msgr2=0x7ffbf406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:45.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1362996508 shutdown_connections 2024-09-11T22:28:45.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:45.932+0000 7ffbf9cbd640 1 -- 172.21.15.5:0/1362996508 wait complete. 2024-09-11T22:28:46.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:45 smithi159 ceph-mon[35534]: pgmap v202: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-11T22:28:46.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:45 smithi172 ceph-mon[35461]: pgmap v202: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-11T22:28:46.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 478793 7.82 MB/sec execute 184 sec latency 31.999 ms 2024-09-11T22:28:46.666 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e5aba961', total seconds elapsed: 26.1 2024-09-11T22:28:47.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:28:47.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 481103 7.86 MB/sec execute 185 sec latency 14.871 ms 2024-09-11T22:28:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:47.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:47 smithi005 ceph-mon[30943]: pgmap v203: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-11T22:28:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.030+0000 7f823619d640 1 -- 172.21.15.5:0/1049867481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82301068d0 msgr2=0x7f8230106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.030+0000 7f823619d640 1 --2- 172.21.15.5:0/1049867481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82301068d0 0x7f8230106cb0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f822c009930 tx=0x7f822c02a420 comp rx=0 tx=0).stop 2024-09-11T22:28:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.030+0000 7f823619d640 1 -- 172.21.15.5:0/1049867481 shutdown_connections 2024-09-11T22:28:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.030+0000 7f823619d640 1 --2- 172.21.15.5:0/1049867481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8230101270 0x7f823010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.030+0000 7f823619d640 1 --2- 172.21.15.5:0/1049867481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82301008d0 0x7f8230100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.030+0000 7f823619d640 1 --2- 172.21.15.5:0/1049867481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82301068d0 0x7f8230106cb0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.030+0000 7f823619d640 1 -- 172.21.15.5:0/1049867481 >> 172.21.15.5:0/1049867481 conn(0x7f82300fc610 msgr2=0x7f82300fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.030+0000 7f823619d640 1 -- 172.21.15.5:0/1049867481 shutdown_connections 2024-09-11T22:28:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.030+0000 7f823619d640 1 -- 172.21.15.5:0/1049867481 wait complete. 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823619d640 1 Processor -- start 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823619d640 1 -- start start 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823619d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82301008d0 0x7f82300fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823619d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8230101270 0x7f82300fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823619d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82301068d0 0x7f82300fd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823619d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8230110cd0 con 0x7f82301068d0 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823619d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8230110b50 con 0x7f8230101270 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823619d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8230110e50 con 0x7f82301008d0 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823599c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82301068d0 0x7f82300fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823599c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82301068d0 0x7f82300fd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48646/0 (socket says 172.21.15.5:48646) 2024-09-11T22:28:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823599c640 1 -- 172.21.15.5:0/1235380715 learned_addr learned my addr 172.21.15.5:0/1235380715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.031+0000 7f823499a640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8230101270 0x7f82300fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f823519b640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82301008d0 0x7f82300fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f823599c640 1 -- 172.21.15.5:0/1235380715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82301008d0 msgr2=0x7f82300fcca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f823599c640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82301008d0 0x7f82300fcca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f823599c640 1 -- 172.21.15.5:0/1235380715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8230101270 msgr2=0x7f82300fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f823599c640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8230101270 0x7f82300fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f823599c640 1 -- 172.21.15.5:0/1235380715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82301a6740 con 0x7f82301068d0 2024-09-11T22:28:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f823499a640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8230101270 0x7f82300fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f823599c640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82301068d0 0x7f82300fd720 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f822800eb10 tx=0x7f822800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:48.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f82267fc640 1 -- 172.21.15.5:0/1235380715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228004b20 con 0x7f82301068d0 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f82267fc640 1 -- 172.21.15.5:0/1235380715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8228004cc0 con 0x7f82301068d0 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f823619d640 1 -- 172.21.15.5:0/1235380715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82301a6a30 con 0x7f82301068d0 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f82267fc640 1 -- 172.21.15.5:0/1235380715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228010760 con 0x7f82301068d0 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.032+0000 7f823619d640 1 -- 172.21.15.5:0/1235380715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82301a6f20 con 0x7f82301068d0 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.035+0000 7f82267fc640 1 -- 172.21.15.5:0/1235380715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8228005020 con 0x7f82301068d0 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.035+0000 7f82397d8740 1 Processor -- start 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f82397d8740 1 -- start start 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f82397d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa5353270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f82397d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa5353bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f82397d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa5360940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f82397d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555fa52b92f0 con 0x555fa535e550 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f82397d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555fa5260cf0 con 0x555fa5354d20 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f82397d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555fa52b52b0 con 0x555fa53537b0 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f82267fc640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f821c0a2cf0 0x7f821c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823499a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa5360940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823499a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa5360940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48652/0 (socket says 172.21.15.5:48652) 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823499a640 1 -- 172.21.15.5:0/1871313988 learned_addr learned my addr 172.21.15.5:0/1871313988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823499a640 1 --2- 172.21.15.5:0/1871313988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa5353270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823519b640 1 --2- 172.21.15.5:0/1871313988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa5353bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823599c640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f821c0a2cf0 0x7f821c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f82267fc640 1 -- 172.21.15.5:0/1235380715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f82280cfa30 con 0x7f82301068d0 2024-09-11T22:28:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823499a640 1 -- 172.21.15.5:0/1871313988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 msgr2=0x555fa5353bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823499a640 1 --2- 172.21.15.5:0/1871313988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa5353bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823499a640 1 -- 172.21.15.5:0/1871313988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 msgr2=0x555fa5353270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823499a640 1 --2- 172.21.15.5:0/1871313988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa5353270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.036+0000 7f823499a640 1 -- 172.21.15.5:0/1871313988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa52453a0 con 0x555fa535e550 2024-09-11T22:28:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.037+0000 7f823499a640 1 --2- 172.21.15.5:0/1871313988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa5360940 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8220004650 tx=0x7f8220009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d23e09de8507946 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.037+0000 7f823599c640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f821c0a2cf0 0x7f821c0a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f822809bea0 tx=0x7f822800efd0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.037+0000 7f821b7fe640 1 -- 172.21.15.5:0/1871313988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822000fc50 con 0x555fa535e550 2024-09-11T22:28:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.037+0000 7f821b7fe640 1 -- 172.21.15.5:0/1871313988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822000fdf0 con 0x555fa535e550 2024-09-11T22:28:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.037+0000 7f821b7fe640 1 -- 172.21.15.5:0/1871313988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822000e5b0 con 0x555fa535e550 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 -- 172.21.15.5:0/1871313988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 msgr2=0x555fa5360940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 --2- 172.21.15.5:0/1871313988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa5360940 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8220004650 tx=0x7f8220009b80 comp rx=0 tx=0).stop 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 -- 172.21.15.5:0/1871313988 shutdown_connections 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 --2- 172.21.15.5:0/1871313988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa5360940 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 --2- 172.21.15.5:0/1871313988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa5353bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 --2- 172.21.15.5:0/1871313988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa5353270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 -- 172.21.15.5:0/1871313988 >> 172.21.15.5:0/1871313988 conn(0x555fa531c310 msgr2=0x555fa534fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 -- 172.21.15.5:0/1871313988 shutdown_connections 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 -- 172.21.15.5:0/1871313988 wait complete. 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 -- start start 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.038+0000 7f82397d8740 1 Processor -- start 2024-09-11T22:28:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f82397d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa535df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f82397d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa535b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f82397d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa535b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f82397d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555fa5361250 con 0x555fa535e550 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f82397d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555fa5274060 con 0x555fa5354d20 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f82397d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555fa53613d0 con 0x555fa53537b0 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f82397d8740 10 client.0 _ll_register_callbacks cb 0x555fa4fc78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823519b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa535b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823499a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa535b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823519b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa535b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56986/0 (socket says 172.21.15.5:56986) 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823499a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa535b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48668/0 (socket says 172.21.15.5:48668) 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823519b640 1 -- 172.21.15.5:0/214730757 learned_addr learned my addr 172.21.15.5:0/214730757 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823499a640 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa535df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823499a640 1 -- 172.21.15.5:0/214730757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 msgr2=0x555fa535df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823499a640 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa535df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823499a640 1 -- 172.21.15.5:0/214730757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 msgr2=0x555fa535b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823499a640 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa535b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823499a640 1 -- 172.21.15.5:0/214730757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa502a3c0 con 0x555fa535e550 2024-09-11T22:28:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823519b640 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa535b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f823499a640 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa535b890 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8220001eb0 tx=0x7f822001ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2637d84e1fd51b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.039+0000 7f8218ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f8218ff9640 1 -- 172.21.15.5:0/214730757 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220005610 con 0x555fa535e550 2024-09-11T22:28:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f82397d8740 1 -- 172.21.15.5:0/214730757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f822000fdf0 con 0x555fa535e550 2024-09-11T22:28:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f8218ff9640 1 -- 172.21.15.5:0/214730757 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f8220028040 con 0x555fa535e550 2024-09-11T22:28:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f8218ff9640 1 -- 172.21.15.5:0/214730757 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f82200046c0 con 0x555fa535e550 2024-09-11T22:28:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f82397d8740 10 client.17457 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f82397d8740 1 -- 172.21.15.5:0/214730757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555fa536cd20 con 0x555fa535e550 2024-09-11T22:28:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f8218ff9640 1 -- 172.21.15.5:0/214730757 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822000fdf0 con 0x555fa535e550 2024-09-11T22:28:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f8218ff9640 1 -- 172.21.15.5:0/214730757 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f8220033030 con 0x555fa535e550 2024-09-11T22:28:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f82397d8740 10 client.17457 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f82397d8740 10 client.17457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:28:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f82397d8740 20 client.17457 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f82397d8740 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555fa536e070 0x555fa5370460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f82397d8740 4 client.17457 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.040+0000 7f82397d8740 1 -- 172.21.15.5:0/214730757 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fa52c6860 con 0x555fa536e070 2024-09-11T22:28:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.041+0000 7f823519b640 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555fa536e070 0x555fa5370460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.041+0000 7f823519b640 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555fa536e070 0x555fa5370460 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d151dfde673cf904 server_cookie=7fc3f531b51f41c1 in_seq=0 out_seq=0 2024-09-11T22:28:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.041+0000 7f8218ff9640 10 client.17457 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:28:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f8218ff9640 1 -- 172.21.15.5:0/214730757 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fa52c6860 con 0x555fa536e070 2024-09-11T22:28:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f8218ff9640 10 client.17457 handle_command_reply: tid=0 2024-09-11T22:28:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 client.17457 shutdown 2024-09-11T22:28:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 -- 172.21.15.5:0/214730757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 msgr2=0x555fa535b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa535b890 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8220001eb0 tx=0x7f822001ff00 comp rx=0 tx=0).stop 2024-09-11T22:28:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 -- 172.21.15.5:0/214730757 shutdown_connections 2024-09-11T22:28:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555fa536e070 0x555fa5370460 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa535b890 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa535b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 --2- 172.21.15.5:0/214730757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa535df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 -- 172.21.15.5:0/214730757 >> 172.21.15.5:0/214730757 conn(0x555fa531c310 msgr2=0x555fa5355b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f8218ff9640 0 client.17457 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:28:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 -- 172.21.15.5:0/214730757 shutdown_connections 2024-09-11T22:28:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 1 -- 172.21.15.5:0/214730757 wait complete. 2024-09-11T22:28:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.042+0000 7f82397d8740 20 client.17457 trim_cache size 0 max 16384 2024-09-11T22:28:48.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.057+0000 7f82397d8740 1 Processor -- start 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.057+0000 7f82397d8740 1 -- start start 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.057+0000 7f82397d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa53537b0 0x555fa536a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.057+0000 7f82397d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa5354d20 0x555fa5356300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.057+0000 7f82397d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa535e550 0x555fa5356840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.057+0000 7f82397d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82301afb20 con 0x555fa5354d20 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.057+0000 7f82397d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82301b1660 con 0x555fa53537b0 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.057+0000 7f82397d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f82301b01f0 con 0x555fa535e550 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.057+0000 7f823519b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa5354d20 0x555fa5356300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.057+0000 7f823499a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa53537b0 0x555fa536a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823519b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa5354d20 0x555fa5356300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48674/0 (socket says 172.21.15.5:48674) 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823519b640 1 -- 172.21.15.5:0/1139192043 learned_addr learned my addr 172.21.15.5:0/1139192043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823499a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa53537b0 0x555fa536a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56988/0 (socket says 172.21.15.5:56988) 2024-09-11T22:28:48.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823499a640 1 --2- 172.21.15.5:0/1139192043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa535e550 0x555fa5356840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823499a640 1 -- 172.21.15.5:0/1139192043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa535e550 msgr2=0x555fa5356840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823499a640 1 --2- 172.21.15.5:0/1139192043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa535e550 0x555fa5356840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823499a640 1 -- 172.21.15.5:0/1139192043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa5354d20 msgr2=0x555fa5356300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823499a640 1 --2- 172.21.15.5:0/1139192043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa5354d20 0x555fa5356300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823499a640 1 -- 172.21.15.5:0/1139192043 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fa53534b0 con 0x555fa53537b0 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823519b640 1 --2- 172.21.15.5:0/1139192043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa5354d20 0x555fa5356300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f823499a640 1 --2- 172.21.15.5:0/1139192043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa53537b0 0x555fa536a4f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f822001fd20 tx=0x7f8220000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38473f21b9e762db server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f821affd640 1 -- 172.21.15.5:0/1139192043 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220009c10 con 0x555fa53537b0 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f821affd640 1 -- 172.21.15.5:0/1139192043 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8220005470 con 0x555fa53537b0 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.058+0000 7f821affd640 1 -- 172.21.15.5:0/1139192043 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220004bc0 con 0x555fa53537b0 2024-09-11T22:28:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.059+0000 7f82397d8740 1 -- 172.21.15.5:0/1139192043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa53537b0 msgr2=0x555fa536a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.059+0000 7f82397d8740 1 --2- 172.21.15.5:0/1139192043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa53537b0 0x555fa536a4f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f822001fd20 tx=0x7f8220000bc0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.059+0000 7f82397d8740 1 -- 172.21.15.5:0/1139192043 shutdown_connections 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.059+0000 7f82397d8740 1 --2- 172.21.15.5:0/1139192043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa535e550 0x555fa5356840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.059+0000 7f82397d8740 1 --2- 172.21.15.5:0/1139192043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa5354d20 0x555fa5356300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.059+0000 7f82397d8740 1 --2- 172.21.15.5:0/1139192043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa53537b0 0x555fa536a4f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.059+0000 7f82397d8740 1 -- 172.21.15.5:0/1139192043 >> 172.21.15.5:0/1139192043 conn(0x555fa531c310 msgr2=0x555fa535a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.059+0000 7f82397d8740 1 -- 172.21.15.5:0/1139192043 shutdown_connections 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.059+0000 7f82397d8740 1 -- 172.21.15.5:0/1139192043 wait complete. 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f82397d8740 1 -- start start 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f82397d8740 1 Processor -- start 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f82397d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa5366660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f82397d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa5366ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f82397d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa53574a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f82397d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82301b0f90 con 0x555fa535e550 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f82397d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82301b08c0 con 0x555fa5354d20 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f82397d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f82301adb00 con 0x555fa53537b0 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f82397d8740 10 client.0 _ll_register_callbacks cb 0x7f82200116d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f823499a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa53574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f823519b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa5366ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f823499a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa53574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48682/0 (socket says 172.21.15.5:48682) 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f823499a640 1 -- 172.21.15.5:0/342311512 learned_addr learned my addr 172.21.15.5:0/342311512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f823519b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa5366ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57000/0 (socket says 172.21.15.5:57000) 2024-09-11T22:28:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.060+0000 7f823499a640 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa5366660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f823499a640 1 -- 172.21.15.5:0/342311512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 msgr2=0x555fa5366660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f823499a640 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa5366660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f823499a640 1 -- 172.21.15.5:0/342311512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 msgr2=0x555fa5366ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f823499a640 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa5366ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f823499a640 1 -- 172.21.15.5:0/342311512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fa5359090 con 0x555fa535e550 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f823499a640 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa53574a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8220005710 tx=0x7f822000ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e1e9538db5e3fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f8218ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f8218ff9640 1 -- 172.21.15.5:0/342311512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822001f840 con 0x555fa535e550 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f8218ff9640 1 -- 172.21.15.5:0/342311512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f8220009ba0 con 0x555fa535e550 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f82397d8740 1 -- 172.21.15.5:0/342311512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8220005470 con 0x555fa535e550 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f8218ff9640 1 -- 172.21.15.5:0/342311512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f822000fd00 con 0x555fa535e550 2024-09-11T22:28:48.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f82397d8740 10 client.17463 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f82397d8740 1 -- 172.21.15.5:0/342311512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555fa53d8070 con 0x555fa535e550 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.061+0000 7f8218ff9640 1 -- 172.21.15.5:0/342311512 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8220005470 con 0x555fa535e550 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.062+0000 7f8218ff9640 1 -- 172.21.15.5:0/342311512 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f822001d020 con 0x555fa535e550 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.062+0000 7f82397d8740 10 client.17463 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.062+0000 7f82397d8740 10 client.17463 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.062+0000 7f82397d8740 20 client.17463 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.062+0000 7f82397d8740 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555fa53d8600 0x555fa53daa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.062+0000 7f82397d8740 4 client.17463 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.062+0000 7f82397d8740 1 -- 172.21.15.5:0/342311512 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fa52c0d70 con 0x555fa53d8600 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.062+0000 7f823519b640 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555fa53d8600 0x555fa53daa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.062+0000 7f8218ff9640 10 client.17463 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.062+0000 7f823519b640 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555fa53d8600 0x555fa53daa20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50ab8e2e9b5a3e49 server_cookie=9ae946982690dc07 in_seq=0 out_seq=0 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f8218ff9640 1 -- 172.21.15.5:0/342311512 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fa52c0d70 con 0x555fa53d8600 2024-09-11T22:28:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f8218ff9640 10 client.17463 handle_command_reply: tid=0 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 client.17463 shutdown 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 -- 172.21.15.5:0/342311512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 msgr2=0x555fa53574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa53574a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8220005710 tx=0x7f822000ff90 comp rx=0 tx=0).stop 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 -- 172.21.15.5:0/342311512 shutdown_connections 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555fa53d8600 0x555fa53daa20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f8218ff9640 0 client.17463 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa535e550 0x555fa53574a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555fa5354d20 0x555fa5366ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 --2- 172.21.15.5:0/342311512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555fa53537b0 0x555fa5366660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 -- 172.21.15.5:0/342311512 >> 172.21.15.5:0/342311512 conn(0x555fa531c310 msgr2=0x555fa534e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 -- 172.21.15.5:0/342311512 shutdown_connections 2024-09-11T22:28:48.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 1 -- 172.21.15.5:0/342311512 wait complete. 2024-09-11T22:28:48.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.063+0000 7f82397d8740 20 client.17463 trim_cache size 0 max 16384 2024-09-11T22:28:48.081 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 -- 172.21.15.5:0/1235380715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f821c0a2cf0 msgr2=0x7f821c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f821c0a2cf0 0x7f821c0a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f822809bea0 tx=0x7f822800efd0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 -- 172.21.15.5:0/1235380715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82301068d0 msgr2=0x7f82300fd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82301068d0 0x7f82300fd720 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f822800eb10 tx=0x7f822800c2d0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 -- 172.21.15.5:0/1235380715 shutdown_connections 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82301068d0 0x7f82300fd720 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f821c0a2cf0 0x7f821c0a51b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8230101270 0x7f82300fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 --2- 172.21.15.5:0/1235380715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82301008d0 0x7f82300fcca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 -- 172.21.15.5:0/1235380715 >> 172.21.15.5:0/1235380715 conn(0x7f82300fc610 msgr2=0x7f823006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.064+0000 7f823619d640 1 -- 172.21.15.5:0/1235380715 shutdown_connections 2024-09-11T22:28:48.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:48.065+0000 7f823619d640 1 -- 172.21.15.5:0/1235380715 wait complete. 2024-09-11T22:28:48.083 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:47 smithi159 ceph-mon[35534]: pgmap v203: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-11T22:28:48.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:47 smithi172 ceph-mon[35461]: pgmap v203: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-11T22:28:48.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 483511 7.89 MB/sec execute 186 sec latency 25.362 ms 2024-09-11T22:28:48.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:28:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:49.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 485627 7.88 MB/sec execute 187 sec latency 30.393 ms 2024-09-11T22:28:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.623+0000 7fb3bd84e640 1 -- 172.21.15.5:0/2095755093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3b81008c0 msgr2=0x7fb3b8100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.623+0000 7fb3bd84e640 1 --2- 172.21.15.5:0/2095755093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3b81008c0 0x7fb3b8100ca0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009ca0 tx=0x7fb3a802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.623+0000 7fb3bd84e640 1 -- 172.21.15.5:0/2095755093 shutdown_connections 2024-09-11T22:28:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.623+0000 7fb3bd84e640 1 --2- 172.21.15.5:0/2095755093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3b810e470 0x7fb3b8110860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.623+0000 7fb3bd84e640 1 --2- 172.21.15.5:0/2095755093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b8101270 0x7fb3b810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.623+0000 7fb3bd84e640 1 --2- 172.21.15.5:0/2095755093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3b81008c0 0x7fb3b8100ca0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.623+0000 7fb3bd84e640 1 -- 172.21.15.5:0/2095755093 >> 172.21.15.5:0/2095755093 conn(0x7fb3b80fc790 msgr2=0x7fb3b80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.623+0000 7fb3bd84e640 1 -- 172.21.15.5:0/2095755093 shutdown_connections 2024-09-11T22:28:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.623+0000 7fb3bd84e640 1 -- 172.21.15.5:0/2095755093 wait complete. 2024-09-11T22:28:49.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.624+0000 7fb3bd84e640 1 Processor -- start 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.624+0000 7fb3bd84e640 1 -- start start 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.624+0000 7fb3bd84e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b81008c0 0x7fb3b81a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.624+0000 7fb3bd84e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3b8101270 0x7fb3b81a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.624+0000 7fb3bd84e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3b810e470 0x7fb3b819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.624+0000 7fb3bd84e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b8114240 con 0x7fb3b81008c0 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.624+0000 7fb3bd84e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81140c0 con 0x7fb3b8101270 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.624+0000 7fb3bd84e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81143c0 con 0x7fb3b810e470 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.624+0000 7fb3b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b81008c0 0x7fb3b81a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b67fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3b8101270 0x7fb3b81a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b81008c0 0x7fb3b81a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48700/0 (socket says 172.21.15.5:48700) 2024-09-11T22:28:49.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b6ffd640 1 -- 172.21.15.5:0/3815941401 learned_addr learned my addr 172.21.15.5:0/3815941401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b77fe640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3b810e470 0x7fb3b819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b77fe640 1 -- 172.21.15.5:0/3815941401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3b8101270 msgr2=0x7fb3b81a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b77fe640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3b8101270 0x7fb3b81a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b77fe640 1 -- 172.21.15.5:0/3815941401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b81008c0 msgr2=0x7fb3b81a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b77fe640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b81008c0 0x7fb3b81a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b77fe640 1 -- 172.21.15.5:0/3815941401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b819acd0 con 0x7fb3b810e470 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b6ffd640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b81008c0 0x7fb3b81a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3b77fe640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3b810e470 0x7fb3b819a5a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8102aa0 tx=0x7fb3b000ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3bc84c640 1 -- 172.21.15.5:0/3815941401 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000ccc0 con 0x7fb3b810e470 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3bc84c640 1 -- 172.21.15.5:0/3815941401 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b00035f0 con 0x7fb3b810e470 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3bc84c640 1 -- 172.21.15.5:0/3815941401 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000ce60 con 0x7fb3b810e470 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.625+0000 7fb3bd84e640 1 -- 172.21.15.5:0/3815941401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b819afc0 con 0x7fb3b810e470 2024-09-11T22:28:49.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.626+0000 7fb3bd84e640 1 -- 172.21.15.5:0/3815941401 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b81a72b0 con 0x7fb3b810e470 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.627+0000 7fb3bc84c640 1 -- 172.21.15.5:0/3815941401 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb3b0004020 con 0x7fb3b810e470 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.628+0000 7fb3bfc27740 1 Processor -- start 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.628+0000 7fb3bc84c640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2d70 0x7fb3a40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.628+0000 7fb3bfc27740 1 -- start start 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.628+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2d70 0x7fb3a40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.628+0000 7fb3bc84c640 1 -- 172.21.15.5:0/3815941401 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7fb3b00cf650 con 0x7fb3b810e470 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.628+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d7c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 0x556342d7ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d87370 0x556342d89760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3bfc27740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556342ce22f0 con 0x556342d87370 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3bfc27740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556342c89cf0 con 0x556342d7c5d0 2024-09-11T22:28:49.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3bfc27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556342cde2b0 con 0x556342d7db40 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d7c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d7c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:53660/0 (socket says 172.21.15.5:53660) 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b6ffd640 1 -- 172.21.15.5:0/1350817427 learned_addr learned my addr 172.21.15.5:0/1350817427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b77fe640 1 --2- 172.21.15.5:0/1350817427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d87370 0x556342d89760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/1350817427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 0x556342d7ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b6ffd640 1 -- 172.21.15.5:0/1350817427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 msgr2=0x556342d7ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b6ffd640 1 --2- 172.21.15.5:0/1350817427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 0x556342d7ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b6ffd640 1 -- 172.21.15.5:0/1350817427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d87370 msgr2=0x556342d89760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b6ffd640 1 --2- 172.21.15.5:0/1350817427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d87370 0x556342d89760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2d70 0x7fb3a40a5230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb3b810e890 tx=0x7fb3a0009db0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b6ffd640 1 -- 172.21.15.5:0/1350817427 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556342c6e3a0 con 0x556342d7db40 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3b6ffd640 1 --2- 172.21.15.5:0/1350817427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d7c090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80097b0 tx=0x7fb3a802f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6d3e7a248ccebcb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:49.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3adffb640 1 -- 172.21.15.5:0/1350817427 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8003bf0 con 0x556342d7db40 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3adffb640 1 -- 172.21.15.5:0/1350817427 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a8036040 con 0x556342d7db40 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.629+0000 7fb3adffb640 1 -- 172.21.15.5:0/1350817427 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80046a0 con 0x556342d7db40 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.630+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1350817427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 msgr2=0x556342d7c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.630+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/1350817427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d7c090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80097b0 tx=0x7fb3a802f440 comp rx=0 tx=0).stop 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.630+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1350817427 shutdown_connections 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.630+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/1350817427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d87370 0x556342d89760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.630+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/1350817427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 0x556342d7ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.630+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/1350817427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d7c090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.630+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1350817427 >> 172.21.15.5:0/1350817427 conn(0x556342d45310 msgr2=0x556342d78c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.630+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1350817427 shutdown_connections 2024-09-11T22:28:49.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.630+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1350817427 wait complete. 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.631+0000 7fb3bfc27740 1 -- start start 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.631+0000 7fb3bfc27740 1 Processor -- start 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.631+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7c5d0 0x556342d86ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.631+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d83f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.631+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d87370 0x556342d84490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.631+0000 7fb3bfc27740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556342d89e90 con 0x556342d7c5d0 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.631+0000 7fb3bfc27740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556342c9d060 con 0x556342d87370 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.631+0000 7fb3bfc27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556342d8a010 con 0x556342d7db40 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.631+0000 7fb3bfc27740 10 client.0 _ll_register_callbacks cb 0x5563429f08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7c5d0 0x556342d86ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d83f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7c5d0 0x556342d86ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48726/0 (socket says 172.21.15.5:48726) 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b6ffd640 1 -- 172.21.15.5:0/1643834510 learned_addr learned my addr 172.21.15.5:0/1643834510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b77fe640 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d87370 0x556342d84490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b67fc640 1 -- 172.21.15.5:0/1643834510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d87370 msgr2=0x556342d84490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d87370 0x556342d84490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b67fc640 1 -- 172.21.15.5:0/1643834510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7c5d0 msgr2=0x556342d86ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7c5d0 0x556342d86ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b67fc640 1 -- 172.21.15.5:0/1643834510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556342a533c0 con 0x556342d7db40 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d83f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0018000 tx=0x7fb3a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=491f1a850d910b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb396ffd640 1 -- 172.21.15.5:0/1643834510 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0024030 con 0x556342d7db40 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb396ffd640 1 -- 172.21.15.5:0/1643834510 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7fb3a000f040 con 0x556342d7db40 2024-09-11T22:28:49.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.632+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1643834510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a8036040 con 0x556342d7db40 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb396ffd640 1 -- 172.21.15.5:0/1643834510 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb3a001a070 con 0x556342d7db40 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb3bfc27740 10 client.26794 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1643834510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556342d95880 con 0x556342d7db40 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb396ffd640 1 -- 172.21.15.5:0/1643834510 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a8036040 con 0x556342d7db40 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb396ffd640 1 -- 172.21.15.5:0/1643834510 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fb3a001d050 con 0x556342d7db40 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb3bfc27740 10 client.26794 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb3bfc27740 10 client.26794 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb3bfc27740 20 client.26794 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556342d96d90 0x556342d991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb3bfc27740 4 client.26794 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:49.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1643834510 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556342cef860 con 0x556342d96d90 2024-09-11T22:28:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.633+0000 7fb3b6ffd640 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556342d96d90 0x556342d991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.634+0000 7fb3b6ffd640 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556342d96d90 0x556342d991e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ccd9dbbeb2821b9 server_cookie=a6f8304844758545 in_seq=0 out_seq=0 2024-09-11T22:28:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.634+0000 7fb396ffd640 10 client.26794 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:28:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.634+0000 7fb396ffd640 1 -- 172.21.15.5:0/1643834510 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556342cef860 con 0x556342d96d90 2024-09-11T22:28:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.634+0000 7fb396ffd640 10 client.26794 handle_command_reply: tid=0 2024-09-11T22:28:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.634+0000 7fb3bfc27740 1 client.26794 shutdown 2024-09-11T22:28:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.634+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1643834510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 msgr2=0x556342d83f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:49.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.634+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d83f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0018000 tx=0x7fb3a00046c0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.635+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1643834510 shutdown_connections 2024-09-11T22:28:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.635+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556342d96d90 0x556342d991e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.635+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d87370 0x556342d84490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.635+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7db40 0x556342d83f50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.635+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/1643834510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7c5d0 0x556342d86ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.635+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1643834510 >> 172.21.15.5:0/1643834510 conn(0x556342d45310 msgr2=0x556342d7e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.635+0000 7fb396ffd640 0 client.26794 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:28:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.635+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1643834510 shutdown_connections 2024-09-11T22:28:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.635+0000 7fb3bfc27740 1 -- 172.21.15.5:0/1643834510 wait complete. 2024-09-11T22:28:49.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.635+0000 7fb3bfc27740 20 client.26794 trim_cache size 0 max 16384 2024-09-11T22:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.649+0000 7fb3bfc27740 1 Processor -- start 2024-09-11T22:28:49.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.649+0000 7fb3bfc27740 1 -- start start 2024-09-11T22:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d45310 0x556342d85ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 0x556342d7f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 0x556342d7f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b67fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 0x556342d7f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b67fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 0x556342d7f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57050/0 (socket says 172.21.15.5:57050) 2024-09-11T22:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d45310 0x556342d85ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b67fc640 1 -- 172.21.15.5:0/3786908464 learned_addr learned my addr 172.21.15.5:0/3786908464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3bfc27740 1 -- 172.21.15.5:0/3786908464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81afc80 con 0x556342d7db40 2024-09-11T22:28:49.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3bfc27740 1 -- 172.21.15.5:0/3786908464 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81b17c0 con 0x556342d7c5d0 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3bfc27740 1 -- 172.21.15.5:0/3786908464 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81b0350 con 0x556342d45310 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b77fe640 1 --2- 172.21.15.5:0/3786908464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 0x556342d7f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b67fc640 1 -- 172.21.15.5:0/3786908464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d45310 msgr2=0x556342d85ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/3786908464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d45310 0x556342d85ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b67fc640 1 -- 172.21.15.5:0/3786908464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 msgr2=0x556342d7f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/3786908464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 0x556342d7f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b67fc640 1 -- 172.21.15.5:0/3786908464 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556342d79960 con 0x556342d7c5d0 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.650+0000 7fb3b77fe640 1 --2- 172.21.15.5:0/3786908464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 0x556342d7f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.651+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/3786908464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 0x556342d7f120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0004810 tx=0x7fb3a000ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab31ce1a78dfdff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.651+0000 7fb3ad7fa640 1 -- 172.21.15.5:0/3786908464 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0035070 con 0x556342d7c5d0 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.651+0000 7fb3ad7fa640 1 -- 172.21.15.5:0/3786908464 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a0016d60 con 0x556342d7c5d0 2024-09-11T22:28:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.651+0000 7fb3ad7fa640 1 -- 172.21.15.5:0/3786908464 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0024070 con 0x556342d7c5d0 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 -- 172.21.15.5:0/3786908464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 msgr2=0x556342d7f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/3786908464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 0x556342d7f120 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0004810 tx=0x7fb3a000ee10 comp rx=0 tx=0).stop 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 -- 172.21.15.5:0/3786908464 shutdown_connections 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/3786908464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 0x556342d7f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/3786908464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d7c5d0 0x556342d7f120 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/3786908464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d45310 0x556342d85ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 -- 172.21.15.5:0/3786908464 >> 172.21.15.5:0/3786908464 conn(0x556342d845f0 msgr2=0x556342d857b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 -- 172.21.15.5:0/3786908464 shutdown_connections 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 -- 172.21.15.5:0/3786908464 wait complete. 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 -- start start 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.652+0000 7fb3bfc27740 1 Processor -- start 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d45310 0x556342d8f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7c5d0 0x556342d8f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3bfc27740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 0x556342d8c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3bfc27740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81b10f0 con 0x556342d7db40 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3bfc27740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81b0a20 con 0x556342d45310 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3bfc27740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b81adc60 con 0x556342d7c5d0 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3bfc27740 10 client.0 _ll_register_callbacks cb 0x7fb3a000e8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d45310 0x556342d8f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 0x556342d8c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d45310 0x556342d8f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57066/0 (socket says 172.21.15.5:57066) 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b6ffd640 1 -- 172.21.15.5:0/2361672307 learned_addr learned my addr 172.21.15.5:0/2361672307 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7c5d0 0x556342d8f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b6ffd640 1 -- 172.21.15.5:0/2361672307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7c5d0 msgr2=0x556342d8f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b6ffd640 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7c5d0 0x556342d8f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b6ffd640 1 -- 172.21.15.5:0/2361672307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 msgr2=0x556342d8c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b6ffd640 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 0x556342d8c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b6ffd640 1 -- 172.21.15.5:0/2361672307 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556342d8c7b0 con 0x556342d45310 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb3b6ffd640 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d45310 0x556342d8f2a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8004230 tx=0x7fb3a80043a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d4c3b687a710c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.653+0000 7fb396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb396ffd640 1 -- 172.21.15.5:0/2361672307 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8060060 con 0x556342d45310 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb3bfc27740 1 -- 172.21.15.5:0/2361672307 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a0016d60 con 0x556342d45310 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb396ffd640 1 -- 172.21.15.5:0/2361672307 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7fb3a8031450 con 0x556342d45310 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb396ffd640 1 -- 172.21.15.5:0/2361672307 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb3a8031650 con 0x556342d45310 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb3bfc27740 10 client.26867 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb3bfc27740 1 -- 172.21.15.5:0/2361672307 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556342e02090 con 0x556342d45310 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb396ffd640 1 -- 172.21.15.5:0/2361672307 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a0016d60 con 0x556342d45310 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb396ffd640 1 -- 172.21.15.5:0/2361672307 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fb3a8044070 con 0x556342d45310 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb3bfc27740 10 client.26867 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:49.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb3bfc27740 10 client.26867 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb3bfc27740 20 client.26867 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556342d87370 0x556342e04440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb3bfc27740 4 client.26867 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.654+0000 7fb3bfc27740 1 -- 172.21.15.5:0/2361672307 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556342ce9d70 con 0x556342d87370 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556342d87370 0x556342e04440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3b67fc640 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556342d87370 0x556342e04440 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7029dfb4d996049e server_cookie=8d51bcef2d514696 in_seq=0 out_seq=0 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb396ffd640 10 client.26867 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb396ffd640 1 -- 172.21.15.5:0/2361672307 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556342ce9d70 con 0x556342d87370 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb396ffd640 10 client.26867 handle_command_reply: tid=0 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3bfc27740 1 client.26867 shutdown 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3bfc27740 1 -- 172.21.15.5:0/2361672307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d45310 msgr2=0x556342d8f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d45310 0x556342d8f2a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8004230 tx=0x7fb3a80043a0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3bfc27740 1 -- 172.21.15.5:0/2361672307 shutdown_connections 2024-09-11T22:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556342d87370 0x556342e04440 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556342d7db40 0x556342d8c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556342d7c5d0 0x556342d8f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3bfc27740 1 --2- 172.21.15.5:0/2361672307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556342d45310 0x556342d8f2a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb396ffd640 0 client.26867 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.655+0000 7fb3bfc27740 1 -- 172.21.15.5:0/2361672307 >> 172.21.15.5:0/2361672307 conn(0x556342d845f0 msgr2=0x556342d77d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bfc27740 1 -- 172.21.15.5:0/2361672307 shutdown_connections 2024-09-11T22:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bfc27740 1 -- 172.21.15.5:0/2361672307 wait complete. 2024-09-11T22:28:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bfc27740 20 client.26867 trim_cache size 0 max 16384 2024-09-11T22:28:49.665 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 -- 172.21.15.5:0/3815941401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2d70 msgr2=0x7fb3a40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2d70 0x7fb3a40a5230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb3b810e890 tx=0x7fb3a0009db0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 -- 172.21.15.5:0/3815941401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3b810e470 msgr2=0x7fb3b819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3b810e470 0x7fb3b819a5a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8102aa0 tx=0x7fb3b000ed10 comp rx=0 tx=0).stop 2024-09-11T22:28:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 -- 172.21.15.5:0/3815941401 shutdown_connections 2024-09-11T22:28:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3b810e470 0x7fb3b819a5a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3a40a2d70 0x7fb3a40a5230 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3b8101270 0x7fb3b81a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 --2- 172.21.15.5:0/3815941401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b81008c0 0x7fb3b81a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 -- 172.21.15.5:0/3815941401 >> 172.21.15.5:0/3815941401 conn(0x7fb3b80fc790 msgr2=0x7fb3b80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 -- 172.21.15.5:0/3815941401 shutdown_connections 2024-09-11T22:28:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:49.656+0000 7fb3bd84e640 1 -- 172.21.15.5:0/3815941401 wait complete. 2024-09-11T22:28:49.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:49 smithi005 ceph-mon[30943]: pgmap v204: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.0 MiB/s wr, 136 op/s 2024-09-11T22:28:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:49 smithi159 ceph-mon[35534]: pgmap v204: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.0 MiB/s wr, 136 op/s 2024-09-11T22:28:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:49 smithi172 ceph-mon[35461]: pgmap v204: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.0 MiB/s wr, 136 op/s 2024-09-11T22:28:50.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 487380 7.91 MB/sec execute 188 sec latency 50.557 ms 2024-09-11T22:28:50.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:28:50.746 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:51.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.258+0000 7f4293998640 1 -- 172.21.15.5:0/716543103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f428c074f40 msgr2=0x7f428c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:51.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.258+0000 7f4293998640 1 --2- 172.21.15.5:0/716543103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f428c074f40 0x7f428c10de80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4280009900 tx=0x7f428002a480 comp rx=0 tx=0).stop 2024-09-11T22:28:51.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.258+0000 7f4293998640 1 -- 172.21.15.5:0/716543103 shutdown_connections 2024-09-11T22:28:51.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.258+0000 7f4293998640 1 --2- 172.21.15.5:0/716543103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c10e3c0 0x7f428c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.258+0000 7f4293998640 1 --2- 172.21.15.5:0/716543103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f428c074f40 0x7f428c10de80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.258+0000 7f4293998640 1 --2- 172.21.15.5:0/716543103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f428c074620 0x7f428c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.258+0000 7f4293998640 1 -- 172.21.15.5:0/716543103 >> 172.21.15.5:0/716543103 conn(0x7f428c0fd8c0 msgr2=0x7f428c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.258+0000 7f4293998640 1 -- 172.21.15.5:0/716543103 shutdown_connections 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.258+0000 7f4293998640 1 -- 172.21.15.5:0/716543103 wait complete. 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.258+0000 7f4293998640 1 Processor -- start 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4293998640 1 -- start start 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4293998640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c074620 0x7f428c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4293998640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f428c074f40 0x7f428c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4293998640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f428c10e3c0 0x7f428c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4293998640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f428c114140 con 0x7f428c074620 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4293998640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f428c113fc0 con 0x7f428c074f40 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4293998640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1142c0 con 0x7f428c10e3c0 2024-09-11T22:28:51.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4291734640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c074620 0x7f428c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4291734640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c074620 0x7f428c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48756/0 (socket says 172.21.15.5:48756) 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4291734640 1 -- 172.21.15.5:0/1402072026 learned_addr learned my addr 172.21.15.5:0/1402072026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4290f33640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f428c074f40 0x7f428c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4291f35640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f428c10e3c0 0x7f428c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4291734640 1 -- 172.21.15.5:0/1402072026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f428c10e3c0 msgr2=0x7f428c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4291734640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f428c10e3c0 0x7f428c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4291734640 1 -- 172.21.15.5:0/1402072026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f428c074f40 msgr2=0x7f428c19abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4291734640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f428c074f40 0x7f428c19abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.259+0000 7f4291734640 1 -- 172.21.15.5:0/1402072026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c19f650 con 0x7f428c074620 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.260+0000 7f4291734640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c074620 0x7f428c19a6a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f427c00ebc0 tx=0x7f427c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.260+0000 7f428a7fc640 1 -- 172.21.15.5:0/1402072026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c010630 con 0x7f428c074620 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.260+0000 7f428a7fc640 1 -- 172.21.15.5:0/1402072026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f427c010cb0 con 0x7f428c074620 2024-09-11T22:28:51.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.260+0000 7f428a7fc640 1 -- 172.21.15.5:0/1402072026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c018bd0 con 0x7f428c074620 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.260+0000 7f4293998640 1 -- 172.21.15.5:0/1402072026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c19f940 con 0x7f428c074620 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.260+0000 7f4293998640 1 -- 172.21.15.5:0/1402072026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c1a7220 con 0x7f428c074620 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.262+0000 7f428a7fc640 1 -- 172.21.15.5:0/1402072026 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f427c0107d0 con 0x7f428c074620 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.263+0000 7f4295d71740 1 Processor -- start 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.263+0000 7f4295d71740 1 -- start start 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.263+0000 7f4295d71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 0x55ff65ca6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.263+0000 7f428a7fc640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f42740a2d70 0x7f42740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.263+0000 7f4295d71740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65ca6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.263+0000 7f4295d71740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65cb1370 0x55ff65cb3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.263+0000 7f4295d71740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff65c0c2f0 con 0x55ff65cb1370 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.263+0000 7f4295d71740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff65bb3cf0 con 0x55ff65ca65d0 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.263+0000 7f4295d71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff65c082b0 con 0x55ff65ca7b40 2024-09-11T22:28:51.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4290f33640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65ca6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4291734640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f42740a2d70 0x7f42740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4291f35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65cb1370 0x55ff65cb3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4290f33640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65ca6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57090/0 (socket says 172.21.15.5:57090) 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4290f33640 1 -- 172.21.15.5:0/1298935556 learned_addr learned my addr 172.21.15.5:0/1298935556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f428a7fc640 1 -- 172.21.15.5:0/1402072026 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f427c0cfb40 con 0x7f428c074620 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4291f35640 1 --2- 172.21.15.5:0/1298935556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 0x55ff65ca6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4290f33640 1 -- 172.21.15.5:0/1298935556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 msgr2=0x55ff65ca6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4290f33640 1 --2- 172.21.15.5:0/1298935556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 0x55ff65ca6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4290f33640 1 -- 172.21.15.5:0/1298935556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65cb1370 msgr2=0x55ff65cb3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4290f33640 1 --2- 172.21.15.5:0/1298935556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65cb1370 0x55ff65cb3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4290f33640 1 -- 172.21.15.5:0/1298935556 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff65b983a0 con 0x55ff65ca65d0 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4291734640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f42740a2d70 0x7f42740a5230 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f427c09bcc0 tx=0x7f427c013b60 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:51.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f4290f33640 1 --2- 172.21.15.5:0/1298935556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65ca6a10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4280009900 tx=0x7f4280005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c27aaf14ff3599e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.264+0000 7f42737fe640 1 -- 172.21.15.5:0/1298935556 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280040070 con 0x55ff65ca65d0 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f42737fe640 1 -- 172.21.15.5:0/1298935556 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4280031870 con 0x55ff65ca65d0 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f42737fe640 1 -- 172.21.15.5:0/1298935556 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280035040 con 0x55ff65ca65d0 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f4295d71740 1 -- 172.21.15.5:0/1298935556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 msgr2=0x55ff65ca6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f4295d71740 1 --2- 172.21.15.5:0/1298935556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65ca6a10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4280009900 tx=0x7f4280005be0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f4295d71740 1 -- 172.21.15.5:0/1298935556 shutdown_connections 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f4295d71740 1 --2- 172.21.15.5:0/1298935556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65cb1370 0x55ff65cb3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f4295d71740 1 --2- 172.21.15.5:0/1298935556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65ca6a10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f4295d71740 1 --2- 172.21.15.5:0/1298935556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 0x55ff65ca6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f4295d71740 1 -- 172.21.15.5:0/1298935556 >> 172.21.15.5:0/1298935556 conn(0x55ff65c6f310 msgr2=0x55ff65ca2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f4295d71740 1 -- 172.21.15.5:0/1298935556 shutdown_connections 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.265+0000 7f4295d71740 1 -- 172.21.15.5:0/1298935556 wait complete. 2024-09-11T22:28:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4295d71740 1 -- start start 2024-09-11T22:28:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4295d71740 1 Processor -- start 2024-09-11T22:28:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4295d71740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4295d71740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65cadf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4295d71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65cb1370 0x55ff65cae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4295d71740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff65cb4010 con 0x55ff65ca7b40 2024-09-11T22:28:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4295d71740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff65bc7060 con 0x55ff65ca65d0 2024-09-11T22:28:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4295d71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff65cb4190 con 0x55ff65cb1370 2024-09-11T22:28:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4295d71740 10 client.0 _ll_register_callbacks cb 0x55ff6591a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4290f33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65cadf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4291f35640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4290f33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65cadf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48784/0 (socket says 172.21.15.5:48784) 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4291f35640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57096/0 (socket says 172.21.15.5:57096) 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4290f33640 1 -- 172.21.15.5:0/3320781330 learned_addr learned my addr 172.21.15.5:0/3320781330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.266+0000 7f4291f35640 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65cb1370 0x55ff65cae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.267+0000 7f4290f33640 1 -- 172.21.15.5:0/3320781330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65cb1370 msgr2=0x55ff65cae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.267+0000 7f4290f33640 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65cb1370 0x55ff65cae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.267+0000 7f4290f33640 1 -- 172.21.15.5:0/3320781330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 msgr2=0x55ff65cb0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.267+0000 7f4290f33640 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.267+0000 7f4290f33640 1 -- 172.21.15.5:0/3320781330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff6597d3c0 con 0x55ff65ca7b40 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.267+0000 7f4290f33640 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65cadf70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4280006010 tx=0x7f428002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18eb822db7eed08 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:51.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.267+0000 7f4270ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.267+0000 7f4270ff9640 1 -- 172.21.15.5:0/3320781330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280040030 con 0x55ff65ca7b40 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.267+0000 7f4270ff9640 1 -- 172.21.15.5:0/3320781330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f4280004050 con 0x55ff65ca7b40 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.267+0000 7f4295d71740 1 -- 172.21.15.5:0/3320781330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4280031870 con 0x55ff65ca7b40 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.268+0000 7f4270ff9640 1 -- 172.21.15.5:0/3320781330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4280056050 con 0x55ff65ca7b40 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.268+0000 7f4295d71740 10 client.17490 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.268+0000 7f4295d71740 1 -- 172.21.15.5:0/3320781330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ff65cbfa20 con 0x55ff65ca7b40 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.268+0000 7f4270ff9640 1 -- 172.21.15.5:0/3320781330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4280031870 con 0x55ff65ca7b40 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.268+0000 7f4270ff9640 1 -- 172.21.15.5:0/3320781330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f42800035f0 con 0x55ff65ca7b40 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.269+0000 7f4295d71740 10 client.17490 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.269+0000 7f4295d71740 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.269+0000 7f4295d71740 20 client.17490 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.269+0000 7f4295d71740 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff65cc0d70 0x55ff65cc3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.269+0000 7f4295d71740 4 client.17490 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:51.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.269+0000 7f4295d71740 1 -- 172.21.15.5:0/3320781330 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff65c19860 con 0x55ff65cc0d70 2024-09-11T22:28:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.269+0000 7f4291f35640 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff65cc0d70 0x55ff65cc3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.269+0000 7f4270ff9640 10 client.17490 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.269+0000 7f4291f35640 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff65cc0d70 0x55ff65cc3160 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e556279d6143c37 server_cookie=9671d69b3dbc0150 in_seq=0 out_seq=0 2024-09-11T22:28:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4270ff9640 1 -- 172.21.15.5:0/3320781330 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff65c19860 con 0x55ff65cc0d70 2024-09-11T22:28:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4270ff9640 10 client.17490 handle_command_reply: tid=0 2024-09-11T22:28:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4295d71740 1 client.17490 shutdown 2024-09-11T22:28:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4295d71740 1 -- 172.21.15.5:0/3320781330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 msgr2=0x55ff65cadf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4295d71740 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65cadf70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4280006010 tx=0x7f428002a480 comp rx=0 tx=0).stop 2024-09-11T22:28:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4295d71740 1 -- 172.21.15.5:0/3320781330 shutdown_connections 2024-09-11T22:28:51.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4295d71740 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff65cc0d70 0x55ff65cc3160 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4295d71740 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65cb1370 0x55ff65cae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4295d71740 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65cadf70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4295d71740 1 --2- 172.21.15.5:0/3320781330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4270ff9640 0 client.17490 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:51.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.270+0000 7f4295d71740 1 -- 172.21.15.5:0/3320781330 >> 172.21.15.5:0/3320781330 conn(0x55ff65c6f310 msgr2=0x55ff65ca8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:51.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.271+0000 7f4295d71740 1 -- 172.21.15.5:0/3320781330 shutdown_connections 2024-09-11T22:28:51.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.271+0000 7f4295d71740 1 -- 172.21.15.5:0/3320781330 wait complete. 2024-09-11T22:28:51.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.271+0000 7f4295d71740 20 client.17490 trim_cache size 0 max 16384 2024-09-11T22:28:51.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.285+0000 7f4295d71740 1 Processor -- start 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.285+0000 7f4295d71740 1 -- start start 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4295d71740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65ca9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4295d71740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65ca9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4295d71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65cb1370 0x55ff65cc8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4295d71740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1afc70 con 0x55ff65ca7b40 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4295d71740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1b17b0 con 0x55ff65ca65d0 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4295d71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1b0340 con 0x55ff65cb1370 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4290f33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65ca9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4290f33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65ca9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50166/0 (socket says 172.21.15.5:50166) 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4290f33640 1 -- 172.21.15.5:0/2610108433 learned_addr learned my addr 172.21.15.5:0/2610108433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4291f35640 1 --2- 172.21.15.5:0/2610108433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65ca9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4291f35640 1 --2- 172.21.15.5:0/2610108433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65cb1370 0x55ff65cc8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4290f33640 1 -- 172.21.15.5:0/2610108433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65cb1370 msgr2=0x55ff65cc8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:51.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4290f33640 1 --2- 172.21.15.5:0/2610108433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65cb1370 0x55ff65cc8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4290f33640 1 -- 172.21.15.5:0/2610108433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 msgr2=0x55ff65ca9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:51.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4290f33640 1 --2- 172.21.15.5:0/2610108433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65ca9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4290f33640 1 -- 172.21.15.5:0/2610108433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff65caeab0 con 0x55ff65ca7b40 2024-09-11T22:28:51.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4290f33640 1 --2- 172.21.15.5:0/2610108433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65ca9660 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f42800057c0 tx=0x7f4280009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4298476d2ccfbfc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:51.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4272ffd640 1 -- 172.21.15.5:0/2610108433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428005f070 con 0x55ff65ca7b40 2024-09-11T22:28:51.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.286+0000 7f4272ffd640 1 -- 172.21.15.5:0/2610108433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f428004d070 con 0x55ff65ca7b40 2024-09-11T22:28:51.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.287+0000 7f4272ffd640 1 -- 172.21.15.5:0/2610108433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4280040070 con 0x55ff65ca7b40 2024-09-11T22:28:51.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.287+0000 7f4295d71740 1 -- 172.21.15.5:0/2610108433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 msgr2=0x55ff65ca9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.287+0000 7f4295d71740 1 --2- 172.21.15.5:0/2610108433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65ca9660 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f42800057c0 tx=0x7f4280009680 comp rx=0 tx=0).stop 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.287+0000 7f4295d71740 1 -- 172.21.15.5:0/2610108433 shutdown_connections 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.287+0000 7f4295d71740 1 --2- 172.21.15.5:0/2610108433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65cb1370 0x55ff65cc8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.287+0000 7f4295d71740 1 --2- 172.21.15.5:0/2610108433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65ca7b40 0x55ff65ca9660 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.287+0000 7f4295d71740 1 --2- 172.21.15.5:0/2610108433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65ca9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.287+0000 7f4295d71740 1 -- 172.21.15.5:0/2610108433 >> 172.21.15.5:0/2610108433 conn(0x55ff65c6f310 msgr2=0x55ff65cafff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.287+0000 7f4295d71740 1 -- 172.21.15.5:0/2610108433 shutdown_connections 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.287+0000 7f4295d71740 1 -- 172.21.15.5:0/2610108433 wait complete. 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4295d71740 1 -- start start 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4295d71740 1 Processor -- start 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4295d71740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4295d71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 0x55ff65caaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4295d71740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65cb1370 0x55ff65cab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4295d71740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1b10e0 con 0x55ff65cb1370 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4295d71740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1b0a10 con 0x55ff65ca65d0 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4295d71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f428c1adc50 con 0x55ff65ca7b40 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4295d71740 10 client.0 _ll_register_callbacks cb 0x7f4284009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4291f35640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4291f35640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43152/0 (socket says 172.21.15.5:43152) 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4291f35640 1 -- 172.21.15.5:0/1986175580 learned_addr learned my addr 172.21.15.5:0/1986175580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4291f35640 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65cb1370 0x55ff65cab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.288+0000 7f4290f33640 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 0x55ff65caaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4291f35640 1 -- 172.21.15.5:0/1986175580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 msgr2=0x55ff65caaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4291f35640 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 0x55ff65caaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4291f35640 1 -- 172.21.15.5:0/1986175580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65cb1370 msgr2=0x55ff65cab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4291f35640 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65cb1370 0x55ff65cab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4291f35640 1 -- 172.21.15.5:0/1986175580 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff65cab7d0 con 0x55ff65ca65d0 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4290f33640 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 0x55ff65caaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4291f35640 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb9a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55ff65ca3920 tx=0x7f42840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75f3b5afce6d561c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4270ff9640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4270ff9640 1 -- 172.21.15.5:0/1986175580 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840260d0 con 0x55ff65ca65d0 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4270ff9640 1 -- 172.21.15.5:0/1986175580 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f4284026270 con 0x55ff65ca65d0 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4295d71740 1 -- 172.21.15.5:0/1986175580 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f428004d070 con 0x55ff65ca65d0 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4270ff9640 1 -- 172.21.15.5:0/1986175580 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4284013b80 con 0x55ff65ca65d0 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4295d71740 10 client.26885 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.289+0000 7f4295d71740 1 -- 172.21.15.5:0/1986175580 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ff65d2b5f0 con 0x55ff65ca65d0 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.290+0000 7f4270ff9640 1 -- 172.21.15.5:0/1986175580 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f428004d070 con 0x55ff65ca65d0 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.290+0000 7f4270ff9640 1 -- 172.21.15.5:0/1986175580 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f4284035070 con 0x55ff65ca65d0 2024-09-11T22:28:51.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.290+0000 7f4295d71740 10 client.26885 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.290+0000 7f4295d71740 10 client.26885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.290+0000 7f4295d71740 20 client.26885 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.290+0000 7f4295d71740 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff65d2bd50 0x55ff65d2e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.290+0000 7f4295d71740 4 client.26885 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:28:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.290+0000 7f4295d71740 1 -- 172.21.15.5:0/1986175580 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff65c13d70 con 0x55ff65d2bd50 2024-09-11T22:28:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.290+0000 7f4290f33640 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff65d2bd50 0x55ff65d2e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4270ff9640 10 client.26885 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:51.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4290f33640 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff65d2bd50 0x55ff65d2e170 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b1f3f0bd48a8ed server_cookie=400803a59c30069e in_seq=0 out_seq=0 2024-09-11T22:28:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4270ff9640 1 -- 172.21.15.5:0/1986175580 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff65c13d70 con 0x55ff65d2bd50 2024-09-11T22:28:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4270ff9640 10 client.26885 handle_command_reply: tid=0 2024-09-11T22:28:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4295d71740 1 client.26885 shutdown 2024-09-11T22:28:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4295d71740 1 -- 172.21.15.5:0/1986175580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 msgr2=0x55ff65cb9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:51.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4295d71740 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb9a40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55ff65ca3920 tx=0x7f42840046c0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4295d71740 1 -- 172.21.15.5:0/1986175580 shutdown_connections 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4295d71740 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ff65d2bd50 0x55ff65d2e170 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4295d71740 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff65cb1370 0x55ff65cab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4295d71740 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff65ca7b40 0x55ff65caaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4295d71740 1 --2- 172.21.15.5:0/1986175580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff65ca65d0 0x55ff65cb9a40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4295d71740 1 -- 172.21.15.5:0/1986175580 >> 172.21.15.5:0/1986175580 conn(0x55ff65c6f310 msgr2=0x55ff65ca14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.291+0000 7f4270ff9640 0 client.26885 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4295d71740 1 -- 172.21.15.5:0/1986175580 shutdown_connections 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4295d71740 1 -- 172.21.15.5:0/1986175580 wait complete. 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4295d71740 20 client.26885 trim_cache size 0 max 16384 2024-09-11T22:28:51.297 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:28:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 -- 172.21.15.5:0/1402072026 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f42740a2d70 msgr2=0x7f42740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f42740a2d70 0x7f42740a5230 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f427c09bcc0 tx=0x7f427c013b60 comp rx=0 tx=0).stop 2024-09-11T22:28:51.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 -- 172.21.15.5:0/1402072026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c074620 msgr2=0x7f428c19a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c074620 0x7f428c19a6a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f427c00ebc0 tx=0x7f427c00c6a0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 -- 172.21.15.5:0/1402072026 shutdown_connections 2024-09-11T22:28:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f428c10e3c0 0x7f428c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f42740a2d70 0x7f42740a5230 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f428c074f40 0x7f428c19abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 --2- 172.21.15.5:0/1402072026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c074620 0x7f428c19a6a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 -- 172.21.15.5:0/1402072026 >> 172.21.15.5:0/1402072026 conn(0x7f428c0fd8c0 msgr2=0x7f428c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 -- 172.21.15.5:0/1402072026 shutdown_connections 2024-09-11T22:28:51.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:51.292+0000 7f4293998640 1 -- 172.21.15.5:0/1402072026 wait complete. 2024-09-11T22:28:51.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 489676 7.93 MB/sec execute 189 sec latency 39.414 ms 2024-09-11T22:28:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:51 smithi005 ceph-mon[30943]: pgmap v205: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.0 MiB/s wr, 136 op/s 2024-09-11T22:28:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:51 smithi159 ceph-mon[35534]: pgmap v205: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.0 MiB/s wr, 136 op/s 2024-09-11T22:28:52.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:51 smithi172 ceph-mon[35461]: pgmap v205: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.0 MiB/s wr, 136 op/s 2024-09-11T22:28:52.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 491734 7.95 MB/sec execute 190 sec latency 16.206 ms 2024-09-11T22:28:53.104 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:28:53.104 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:28:53.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 494155 7.98 MB/sec execute 191 sec latency 14.609 ms 2024-09-11T22:28:53.381 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.398+0000 7f79a000a640 1 -- 172.21.15.5:0/569196324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79981069c0 msgr2=0x7f7998106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.398+0000 7f79a000a640 1 --2- 172.21.15.5:0/569196324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79981069c0 0x7f7998106da0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7994009930 tx=0x7f799402a420 comp rx=0 tx=0).stop 2024-09-11T22:28:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.398+0000 7f79a000a640 1 -- 172.21.15.5:0/569196324 shutdown_connections 2024-09-11T22:28:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.398+0000 7f79a000a640 1 --2- 172.21.15.5:0/569196324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7998101350 0x7f799810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.398+0000 7f79a000a640 1 --2- 172.21.15.5:0/569196324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79981009b0 0x7f7998100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.398+0000 7f79a000a640 1 --2- 172.21.15.5:0/569196324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79981069c0 0x7f7998106da0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.398+0000 7f79a000a640 1 -- 172.21.15.5:0/569196324 >> 172.21.15.5:0/569196324 conn(0x7f7998077990 msgr2=0x7f79980fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.398+0000 7f79a000a640 1 -- 172.21.15.5:0/569196324 shutdown_connections 2024-09-11T22:28:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.398+0000 7f79a000a640 1 -- 172.21.15.5:0/569196324 wait complete. 2024-09-11T22:28:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f79a000a640 1 Processor -- start 2024-09-11T22:28:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f79a000a640 1 -- start start 2024-09-11T22:28:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f79a000a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79981009b0 0x7f7998102880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f79a000a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7998101350 0x7f7998102dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f79a000a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79981069c0 0x7f7998103300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f79a000a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7998110e50 con 0x7f79981009b0 2024-09-11T22:28:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f79a000a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7998110cd0 con 0x7f79981069c0 2024-09-11T22:28:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f79a000a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7998110fd0 con 0x7f7998101350 2024-09-11T22:28:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799dda6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79981009b0 0x7f7998102880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799dda6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79981009b0 0x7f7998102880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50198/0 (socket says 172.21.15.5:50198) 2024-09-11T22:28:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799dda6640 1 -- 172.21.15.5:0/3768768562 learned_addr learned my addr 172.21.15.5:0/3768768562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799e5a7640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79981069c0 0x7f7998103300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799d5a5640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7998101350 0x7f7998102dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799e5a7640 1 -- 172.21.15.5:0/3768768562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7998101350 msgr2=0x7f7998102dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799e5a7640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7998101350 0x7f7998102dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799e5a7640 1 -- 172.21.15.5:0/3768768562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79981009b0 msgr2=0x7f7998102880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799e5a7640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79981009b0 0x7f7998102880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799e5a7640 1 -- 172.21.15.5:0/3768768562 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79981a68d0 con 0x7f79981069c0 2024-09-11T22:28:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.399+0000 7f799dda6640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79981009b0 0x7f7998102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.400+0000 7f799e5a7640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79981069c0 0x7f7998103300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f799000e940 tx=0x7f799000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.400+0000 7f798effd640 1 -- 172.21.15.5:0/3768768562 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7990004d20 con 0x7f79981069c0 2024-09-11T22:28:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.400+0000 7f79a000a640 1 -- 172.21.15.5:0/3768768562 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79981a6bc0 con 0x7f79981069c0 2024-09-11T22:28:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.400+0000 7f798effd640 1 -- 172.21.15.5:0/3768768562 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7990005020 con 0x7f79981069c0 2024-09-11T22:28:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.400+0000 7f798effd640 1 -- 172.21.15.5:0/3768768562 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79900116d0 con 0x7f79981069c0 2024-09-11T22:28:53.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.400+0000 7f79a000a640 1 -- 172.21.15.5:0/3768768562 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79981a7160 con 0x7f79981069c0 2024-09-11T22:28:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.403+0000 7f798effd640 1 -- 172.21.15.5:0/3768768562 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f79900054d0 con 0x7f79981069c0 2024-09-11T22:28:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.403+0000 7f798effd640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79840a2da0 0x7f79840a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.403+0000 7f798effd640 1 -- 172.21.15.5:0/3768768562 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f79900cf8a0 con 0x7f79981069c0 2024-09-11T22:28:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.404+0000 7f799dda6640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79840a2da0 0x7f79840a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.404+0000 7f79a2e76740 1 Processor -- start 2024-09-11T22:28:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.404+0000 7f79a2e76740 1 -- start start 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.404+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.404+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f3410 0x557a3c6fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.404+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 0x557a3c7000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.404+0000 7f79a2e76740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557a3c6572f0 con 0x557a3c6f2af0 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.404+0000 7f79a2e76740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557a3c5ff270 con 0x557a3c6fdcb0 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.404+0000 7f79a2e76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557a3c6532b0 con 0x557a3c6f3410 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.404+0000 7f799dda6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799dda6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50212/0 (socket says 172.21.15.5:50212) 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799dda6640 1 -- 172.21.15.5:0/2387436167 learned_addr learned my addr 172.21.15.5:0/2387436167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799e5a7640 1 --2- 172.21.15.5:0/2387436167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 0x557a3c7000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799dda6640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79840a2da0 0x7f79840a5260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7994000c00 tx=0x7f7994031510 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799d5a5640 1 --2- 172.21.15.5:0/2387436167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f3410 0x557a3c6fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799e5a7640 1 -- 172.21.15.5:0/2387436167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f3410 msgr2=0x557a3c6fd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799e5a7640 1 --2- 172.21.15.5:0/2387436167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f3410 0x557a3c6fd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799e5a7640 1 -- 172.21.15.5:0/2387436167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 msgr2=0x557a3c6f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799e5a7640 1 --2- 172.21.15.5:0/2387436167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799e5a7640 1 -- 172.21.15.5:0/2387436167 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3c5fe340 con 0x557a3c6fdcb0 2024-09-11T22:28:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799d5a5640 1 --2- 172.21.15.5:0/2387436167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f3410 0x557a3c6fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:28:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f799e5a7640 1 --2- 172.21.15.5:0/2387436167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 0x557a3c7000a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f79900ccf50 tx=0x7f799009b740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ac44c9c900a1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f7983fff640 1 -- 172.21.15.5:0/2387436167 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799001f070 con 0x557a3c6fdcb0 2024-09-11T22:28:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f7983fff640 1 -- 172.21.15.5:0/2387436167 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79900a00d0 con 0x557a3c6fdcb0 2024-09-11T22:28:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.405+0000 7f7983fff640 1 -- 172.21.15.5:0/2387436167 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79900a4500 con 0x557a3c6fdcb0 2024-09-11T22:28:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.406+0000 7f79a2e76740 1 -- 172.21.15.5:0/2387436167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 msgr2=0x557a3c7000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.406+0000 7f79a2e76740 1 --2- 172.21.15.5:0/2387436167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 0x557a3c7000a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f79900ccf50 tx=0x7f799009b740 comp rx=0 tx=0).stop 2024-09-11T22:28:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.406+0000 7f79a2e76740 1 -- 172.21.15.5:0/2387436167 shutdown_connections 2024-09-11T22:28:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.406+0000 7f79a2e76740 1 --2- 172.21.15.5:0/2387436167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 0x557a3c7000a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.406+0000 7f79a2e76740 1 --2- 172.21.15.5:0/2387436167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f3410 0x557a3c6fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.406+0000 7f79a2e76740 1 --2- 172.21.15.5:0/2387436167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.406+0000 7f79a2e76740 1 -- 172.21.15.5:0/2387436167 >> 172.21.15.5:0/2387436167 conn(0x557a3c6ef060 msgr2=0x557a3c6ef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.406+0000 7f79a2e76740 1 -- 172.21.15.5:0/2387436167 shutdown_connections 2024-09-11T22:28:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.406+0000 7f79a2e76740 1 -- 172.21.15.5:0/2387436167 wait complete. 2024-09-11T22:28:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f79a2e76740 1 -- start start 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f79a2e76740 1 Processor -- start 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f2af0 0x557a3c6fab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c6fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6fdcb0 0x557a3c6fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f79a2e76740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557a3c700880 con 0x557a3c6f3410 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f79a2e76740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557a3c5e0290 con 0x557a3c6f2af0 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f79a2e76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557a3c700a00 con 0x557a3c6fdcb0 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f79a2e76740 10 client.0 _ll_register_callbacks cb 0x557a3c611140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f799dda6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f2af0 0x557a3c6fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f799dda6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f2af0 0x557a3c6fab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43174/0 (socket says 172.21.15.5:43174) 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f799dda6640 1 -- 172.21.15.5:0/1974169237 learned_addr learned my addr 172.21.15.5:0/1974169237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f799e5a7640 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6fdcb0 0x557a3c6fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.407+0000 7f799d5a5640 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c6fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f799dda6640 1 -- 172.21.15.5:0/1974169237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6fdcb0 msgr2=0x557a3c6fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f799dda6640 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6fdcb0 0x557a3c6fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f799dda6640 1 -- 172.21.15.5:0/1974169237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 msgr2=0x557a3c6fb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f799dda6640 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c6fb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f799dda6640 1 -- 172.21.15.5:0/1974169237 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3c36a210 con 0x557a3c6f2af0 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f799d5a5640 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c6fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f799dda6640 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f2af0 0x557a3c6fab60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7994005470 tx=0x7f7994009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63cc4c16e3ff309d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f79817fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f79817fa640 1 -- 172.21.15.5:0/1974169237 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994035040 con 0x557a3c6f2af0 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f79a2e76740 1 -- 172.21.15.5:0/1974169237 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79900a00d0 con 0x557a3c6f2af0 2024-09-11T22:28:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f79817fa640 1 -- 172.21.15.5:0/1974169237 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f79940037f0 con 0x557a3c6f2af0 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f79817fa640 1 -- 172.21.15.5:0/1974169237 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f79940039f0 con 0x557a3c6f2af0 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f79a2e76740 10 client.26897 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.408+0000 7f79a2e76740 1 -- 172.21.15.5:0/1974169237 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557a3c436620 con 0x557a3c6f2af0 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f79817fa640 1 -- 172.21.15.5:0/1974169237 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79900a00d0 con 0x557a3c6f2af0 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f79817fa640 1 -- 172.21.15.5:0/1974169237 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f799403c440 con 0x557a3c6f2af0 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f79a2e76740 10 client.26897 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f79a2e76740 10 client.26897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f79a2e76740 20 client.26897 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f79a2e76740 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a3c70dbf0 0x557a3c710000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f79a2e76740 4 client.26897 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f79a2e76740 1 -- 172.21.15.5:0/1974169237 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a3c664a20 con 0x557a3c70dbf0 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f799d5a5640 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a3c70dbf0 0x557a3c710000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f79817fa640 10 client.26897 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.409+0000 7f799d5a5640 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a3c70dbf0 0x557a3c710000 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c314132ba99fe41 server_cookie=c65e079f75182e92 in_seq=0 out_seq=0 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.410+0000 7f79817fa640 1 -- 172.21.15.5:0/1974169237 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a3c664a20 con 0x557a3c70dbf0 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.410+0000 7f79817fa640 10 client.26897 handle_command_reply: tid=0 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.410+0000 7f79a2e76740 1 client.26897 shutdown 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 1 -- 172.21.15.5:0/1974169237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f2af0 msgr2=0x557a3c6fab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f2af0 0x557a3c6fab60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7994005470 tx=0x7f7994009260 comp rx=0 tx=0).stop 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 1 -- 172.21.15.5:0/1974169237 shutdown_connections 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a3c70dbf0 0x557a3c710000 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6fdcb0 0x557a3c6fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c6fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 1 --2- 172.21.15.5:0/1974169237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f2af0 0x557a3c6fab60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 1 -- 172.21.15.5:0/1974169237 >> 172.21.15.5:0/1974169237 conn(0x557a3c6ef060 msgr2=0x557a3c6f4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79817fa640 0 client.26897 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 1 -- 172.21.15.5:0/1974169237 shutdown_connections 2024-09-11T22:28:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 1 -- 172.21.15.5:0/1974169237 wait complete. 2024-09-11T22:28:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.411+0000 7f79a2e76740 20 client.26897 trim_cache size 0 max 16384 2024-09-11T22:28:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.426+0000 7f79a2e76740 1 Processor -- start 2024-09-11T22:28:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.426+0000 7f79a2e76740 1 -- start start 2024-09-11T22:28:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.426+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6fb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.426+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f3410 0x557a3c6fcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.426+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6fdcb0 0x557a3c715910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.426+0000 7f79a2e76740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79981afbf0 con 0x557a3c6f2af0 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.426+0000 7f79a2e76740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f79981b1730 con 0x557a3c6f3410 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.426+0000 7f79a2e76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f79981b02c0 con 0x557a3c6fdcb0 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799dda6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6fb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799d5a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f3410 0x557a3c6fcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799dda6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6fb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50240/0 (socket says 172.21.15.5:50240) 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799dda6640 1 -- 172.21.15.5:0/3253535416 learned_addr learned my addr 172.21.15.5:0/3253535416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799e5a7640 1 --2- 172.21.15.5:0/3253535416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6fdcb0 0x557a3c715910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799dda6640 1 -- 172.21.15.5:0/3253535416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6fdcb0 msgr2=0x557a3c715910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799dda6640 1 --2- 172.21.15.5:0/3253535416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6fdcb0 0x557a3c715910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799dda6640 1 -- 172.21.15.5:0/3253535416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f3410 msgr2=0x557a3c6fcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799dda6640 1 --2- 172.21.15.5:0/3253535416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f3410 0x557a3c6fcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799dda6640 1 -- 172.21.15.5:0/3253535416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a3c6f2870 con 0x557a3c6f2af0 2024-09-11T22:28:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f799dda6640 1 --2- 172.21.15.5:0/3253535416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6fb410 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7994005d00 tx=0x7f79940098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7d38344df7dde0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f79837fe640 1 -- 172.21.15.5:0/3253535416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994035070 con 0x557a3c6f2af0 2024-09-11T22:28:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f79837fe640 1 -- 172.21.15.5:0/3253535416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7994004b30 con 0x557a3c6f2af0 2024-09-11T22:28:53.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.427+0000 7f79837fe640 1 -- 172.21.15.5:0/3253535416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799405e040 con 0x557a3c6f2af0 2024-09-11T22:28:53.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.428+0000 7f79a2e76740 1 -- 172.21.15.5:0/3253535416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 msgr2=0x557a3c6fb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.428+0000 7f79a2e76740 1 --2- 172.21.15.5:0/3253535416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6fb410 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7994005d00 tx=0x7f79940098b0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.428+0000 7f79a2e76740 1 -- 172.21.15.5:0/3253535416 shutdown_connections 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.428+0000 7f79a2e76740 1 --2- 172.21.15.5:0/3253535416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6fdcb0 0x557a3c715910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.428+0000 7f79a2e76740 1 --2- 172.21.15.5:0/3253535416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6f3410 0x557a3c6fcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.428+0000 7f79a2e76740 1 --2- 172.21.15.5:0/3253535416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f2af0 0x557a3c6fb410 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.428+0000 7f79a2e76740 1 -- 172.21.15.5:0/3253535416 >> 172.21.15.5:0/3253535416 conn(0x557a3c6ef060 msgr2=0x557a3c6fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.428+0000 7f79a2e76740 1 -- 172.21.15.5:0/3253535416 shutdown_connections 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.428+0000 7f79a2e76740 1 -- 172.21.15.5:0/3253535416 wait complete. 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.428+0000 7f79a2e76740 1 -- start start 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f79a2e76740 1 Processor -- start 2024-09-11T22:28:53.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f2af0 0x557a3c700e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c701360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f79a2e76740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 0x557a3c7768d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f79a2e76740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79981b1060 con 0x557a3c6f3410 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f79a2e76740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f79981b0990 con 0x557a3c6fdcb0 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f79a2e76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f79981adbd0 con 0x557a3c6f2af0 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f79a2e76740 10 client.0 _ll_register_callbacks cb 0x7f7994009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799d5a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c701360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799e5a7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 0x557a3c7768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799d5a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c701360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50252/0 (socket says 172.21.15.5:50252) 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799d5a5640 1 -- 172.21.15.5:0/728938143 learned_addr learned my addr 172.21.15.5:0/728938143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:53.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799dda6640 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f2af0 0x557a3c700e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799e5a7640 1 -- 172.21.15.5:0/728938143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f2af0 msgr2=0x557a3c700e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799e5a7640 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f2af0 0x557a3c700e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799e5a7640 1 -- 172.21.15.5:0/728938143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 msgr2=0x557a3c701360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799e5a7640 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c701360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799e5a7640 1 -- 172.21.15.5:0/728938143 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a3c776e10 con 0x557a3c6fdcb0 2024-09-11T22:28:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.429+0000 7f799d5a5640 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c701360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.430+0000 7f799e5a7640 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 0x557a3c7768d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x557a3c6f3a30 tx=0x7f7990005840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0100be6fabd1580 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.430+0000 7f79817fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.430+0000 7f79817fa640 1 -- 172.21.15.5:0/728938143 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799001f040 con 0x557a3c6fdcb0 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.430+0000 7f79817fa640 1 -- 172.21.15.5:0/728938143 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f79900a4500 con 0x557a3c6fdcb0 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.430+0000 7f79a2e76740 1 -- 172.21.15.5:0/728938143 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7994004b30 con 0x557a3c6fdcb0 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.430+0000 7f79817fa640 1 -- 172.21.15.5:0/728938143 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f79900a4990 con 0x557a3c6fdcb0 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.430+0000 7f79a2e76740 10 client.26903 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.430+0000 7f79a2e76740 1 -- 172.21.15.5:0/728938143 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557a3c778000 con 0x557a3c6fdcb0 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.430+0000 7f79817fa640 1 -- 172.21.15.5:0/728938143 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7994004b30 con 0x557a3c6fdcb0 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.431+0000 7f79817fa640 1 -- 172.21.15.5:0/728938143 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f799000f040 con 0x557a3c6fdcb0 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.431+0000 7f79a2e76740 10 client.26903 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.431+0000 7f79a2e76740 10 client.26903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.431+0000 7f79a2e76740 20 client.26903 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.431+0000 7f79a2e76740 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a3c7783e0 0x557a3c77a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.431+0000 7f79a2e76740 4 client.26903 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:28:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.431+0000 7f79a2e76740 1 -- 172.21.15.5:0/728938143 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a3c65ed70 con 0x557a3c7783e0 2024-09-11T22:28:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.431+0000 7f799dda6640 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a3c7783e0 0x557a3c77a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.431+0000 7f79817fa640 10 client.26903 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.431+0000 7f799dda6640 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a3c7783e0 0x557a3c77a7d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2173d341f0963b0b server_cookie=4f51ac7838de9aa5 in_seq=0 out_seq=0 2024-09-11T22:28:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.528+0000 7f79817fa640 1 -- 172.21.15.5:0/728938143 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a3c65ed70 con 0x557a3c7783e0 2024-09-11T22:28:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79817fa640 10 client.26903 handle_command_reply: tid=0 2024-09-11T22:28:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79a2e76740 1 client.26903 shutdown 2024-09-11T22:28:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79a2e76740 1 -- 172.21.15.5:0/728938143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 msgr2=0x557a3c7768d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79a2e76740 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 0x557a3c7768d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x557a3c6f3a30 tx=0x7f7990005840 comp rx=0 tx=0).stop 2024-09-11T22:28:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79a2e76740 1 -- 172.21.15.5:0/728938143 shutdown_connections 2024-09-11T22:28:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79a2e76740 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a3c7783e0 0x557a3c77a7d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79a2e76740 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a3c6fdcb0 0x557a3c7768d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79a2e76740 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a3c6f3410 0x557a3c701360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79817fa640 0 client.26903 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79a2e76740 1 --2- 172.21.15.5:0/728938143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a3c6f2af0 0x557a3c700e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.529+0000 7f79a2e76740 1 -- 172.21.15.5:0/728938143 >> 172.21.15.5:0/728938143 conn(0x557a3c6ef060 msgr2=0x557a3c6ed9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.530+0000 7f79a2e76740 1 -- 172.21.15.5:0/728938143 shutdown_connections 2024-09-11T22:28:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.530+0000 7f79a2e76740 1 -- 172.21.15.5:0/728938143 wait complete. 2024-09-11T22:28:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.530+0000 7f79a2e76740 20 client.26903 trim_cache size 0 max 16384 2024-09-11T22:28:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:28:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:28:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:28:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:28:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 -- 172.21.15.5:0/3768768562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79840a2da0 msgr2=0x7f79840a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79840a2da0 0x7f79840a5260 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7994000c00 tx=0x7f7994031510 comp rx=0 tx=0).stop 2024-09-11T22:28:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 -- 172.21.15.5:0/3768768562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79981069c0 msgr2=0x7f7998103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79981069c0 0x7f7998103300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f799000e940 tx=0x7f799000ee00 comp rx=0 tx=0).stop 2024-09-11T22:28:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 -- 172.21.15.5:0/3768768562 shutdown_connections 2024-09-11T22:28:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79840a2da0 0x7f79840a5260 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f79981069c0 0x7f7998103300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7998101350 0x7f7998102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 --2- 172.21.15.5:0/3768768562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79981009b0 0x7f7998102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 -- 172.21.15.5:0/3768768562 >> 172.21.15.5:0/3768768562 conn(0x7f7998077990 msgr2=0x7f7998104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 -- 172.21.15.5:0/3768768562 shutdown_connections 2024-09-11T22:28:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:28:53.531+0000 7f79a000a640 1 -- 172.21.15.5:0/3768768562 wait complete. 2024-09-11T22:28:53.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:53 smithi005 ceph-mon[30943]: pgmap v206: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 228 op/s 2024-09-11T22:28:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.895+0000 7f8505a7b640 1 -- 172.21.15.5:0/4033438664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85001036e0 msgr2=0x7f8500107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.895+0000 7f8505a7b640 1 --2- 172.21.15.5:0/4033438664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85001036e0 0x7f8500107b90 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009930 tx=0x7f84f402a420 comp rx=0 tx=0).stop 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.896+0000 7f8505a7b640 1 -- 172.21.15.5:0/4033438664 shutdown_connections 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.896+0000 7f8505a7b640 1 --2- 172.21.15.5:0/4033438664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85001082a0 0x7f850010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.896+0000 7f8505a7b640 1 --2- 172.21.15.5:0/4033438664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85001036e0 0x7f8500107b90 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.896+0000 7f8505a7b640 1 --2- 172.21.15.5:0/4033438664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8500102d30 0x7f8500103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.896+0000 7f8505a7b640 1 -- 172.21.15.5:0/4033438664 >> 172.21.15.5:0/4033438664 conn(0x7f85000fc7d0 msgr2=0x7f85000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.896+0000 7f8505a7b640 1 -- 172.21.15.5:0/4033438664 shutdown_connections 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.896+0000 7f8505a7b640 1 -- 172.21.15.5:0/4033438664 wait complete. 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.896+0000 7f8505a7b640 1 Processor -- start 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f8505a7b640 1 -- start start 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f8505a7b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8500102d30 0x7f8500198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f8505a7b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85001036e0 0x7f8500198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f8505a7b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85001082a0 0x7f850019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f8505a7b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85001120a0 con 0x7f8500102d30 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f8505a7b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8500111f20 con 0x7f85001082a0 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f8505a7b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8500112220 con 0x7f85001036e0 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f84ff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8500102d30 0x7f8500198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f84ff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8500102d30 0x7f8500198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50270/0 (socket says 172.21.15.5:50270) 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f84ff7fe640 1 -- 172.21.15.5:0/2790015230 learned_addr learned my addr 172.21.15.5:0/2790015230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f84feffd640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85001036e0 0x7f8500198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.897+0000 7f84fffff640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85001082a0 0x7f850019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f84ff7fe640 1 -- 172.21.15.5:0/2790015230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85001036e0 msgr2=0x7f8500198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f84ff7fe640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85001036e0 0x7f8500198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f84ff7fe640 1 -- 172.21.15.5:0/2790015230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85001082a0 msgr2=0x7f850019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f84ff7fe640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85001082a0 0x7f850019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f84ff7fe640 1 -- 172.21.15.5:0/2790015230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850019d5f0 con 0x7f8500102d30 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f84ff7fe640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8500102d30 0x7f8500198640 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f84f000e9f0 tx=0x7f84f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f84fcff9640 1 -- 172.21.15.5:0/2790015230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f000cd80 con 0x7f8500102d30 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f8505a7b640 1 -- 172.21.15.5:0/2790015230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850019d8e0 con 0x7f8500102d30 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f84fcff9640 1 -- 172.21.15.5:0/2790015230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f0010910 con 0x7f8500102d30 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f84fcff9640 1 -- 172.21.15.5:0/2790015230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f00188f0 con 0x7f8500102d30 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.898+0000 7f8505a7b640 1 -- 172.21.15.5:0/2790015230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85001a51c0 con 0x7f8500102d30 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.901+0000 7f84fcff9640 1 -- 172.21.15.5:0/2790015230 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f84f0010ac0 con 0x7f8500102d30 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.901+0000 7f8507e54740 1 Processor -- start 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.901+0000 7f8507e54740 1 -- start start 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.901+0000 7f8507e54740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eb98f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.901+0000 7f8507e54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eb99480 0x55b37eba3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.901+0000 7f8507e54740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eba3d40 0x55b37eba6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.901+0000 7f8507e54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b37eafe280 con 0x55b37eb99480 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.901+0000 7f8507e54740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b37eaa5b10 con 0x55b37eba3d40 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.901+0000 7f84fcff9640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84e80a2cf0 0x7f84e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.901+0000 7f8507e54740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b37eafa240 con 0x55b37eb98b60 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84feffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eba3d40 0x55b37eba6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84ff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eb99480 0x55b37eba3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84fffff640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84e80a2cf0 0x7f84e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84feffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eba3d40 0x55b37eba6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43206/0 (socket says 172.21.15.5:43206) 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84feffd640 1 -- 172.21.15.5:0/3143113984 learned_addr learned my addr 172.21.15.5:0/3143113984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84fffff640 1 --2- 172.21.15.5:0/3143113984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eb98f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84fcff9640 1 -- 172.21.15.5:0/2790015230 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f84f00cf820 con 0x7f8500102d30 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84feffd640 1 -- 172.21.15.5:0/3143113984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 msgr2=0x55b37eb98f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84feffd640 1 --2- 172.21.15.5:0/3143113984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eb98f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84feffd640 1 -- 172.21.15.5:0/3143113984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eb99480 msgr2=0x55b37eba3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84feffd640 1 --2- 172.21.15.5:0/3143113984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eb99480 0x55b37eba3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84feffd640 1 -- 172.21.15.5:0/3143113984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37eab21f0 con 0x55b37eba3d40 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84fffff640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84e80a2cf0 0x7f84e80a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009fd0 tx=0x7f84f8009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84feffd640 1 --2- 172.21.15.5:0/3143113984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eba3d40 0x55b37eba6130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f84f4000c00 tx=0x7f84f4005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa539f6026be464e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84e5ffb640 1 -- 172.21.15.5:0/3143113984 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4040070 con 0x55b37eba3d40 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84e5ffb640 1 -- 172.21.15.5:0/3143113984 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f4004030 con 0x55b37eba3d40 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.902+0000 7f84e5ffb640 1 -- 172.21.15.5:0/3143113984 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4035040 con 0x55b37eba3d40 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.903+0000 7f8507e54740 1 -- 172.21.15.5:0/3143113984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eba3d40 msgr2=0x55b37eba6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.903+0000 7f8507e54740 1 --2- 172.21.15.5:0/3143113984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eba3d40 0x55b37eba6130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f84f4000c00 tx=0x7f84f4005d50 comp rx=0 tx=0).stop 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.903+0000 7f8507e54740 1 -- 172.21.15.5:0/3143113984 shutdown_connections 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.903+0000 7f8507e54740 1 --2- 172.21.15.5:0/3143113984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eba3d40 0x55b37eba6130 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.903+0000 7f8507e54740 1 --2- 172.21.15.5:0/3143113984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eb99480 0x55b37eba3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.903+0000 7f8507e54740 1 --2- 172.21.15.5:0/3143113984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eb98f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.903+0000 7f8507e54740 1 -- 172.21.15.5:0/3143113984 >> 172.21.15.5:0/3143113984 conn(0x55b37eb950f0 msgr2=0x55b37eb954d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.903+0000 7f8507e54740 1 -- 172.21.15.5:0/3143113984 shutdown_connections 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.903+0000 7f8507e54740 1 -- 172.21.15.5:0/3143113984 wait complete. 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f8507e54740 1 -- start start 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f8507e54740 1 Processor -- start 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f8507e54740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f8507e54740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f8507e54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f8507e54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b37eba6910 con 0x55b37eba3d40 2024-09-11T22:28:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f8507e54740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b37eab8f10 con 0x55b37eb99480 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f8507e54740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b37eba6a90 con 0x55b37eb98b60 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f8507e54740 10 client.0 _ll_register_callbacks cb 0x55b37ea81a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f84fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f84feffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f84ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f84feffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43222/0 (socket says 172.21.15.5:43222) 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f84feffd640 1 -- 172.21.15.5:0/2811525418 learned_addr learned my addr 172.21.15.5:0/2811525418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f84fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50292/0 (socket says 172.21.15.5:50292) 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.904+0000 7f84ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:51150/0 (socket says 172.21.15.5:51150) 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84feffd640 1 -- 172.21.15.5:0/2811525418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 msgr2=0x55b37eba0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84feffd640 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84feffd640 1 -- 172.21.15.5:0/2811525418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 msgr2=0x55b37eba1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84feffd640 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84feffd640 1 -- 172.21.15.5:0/2811525418 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b37e86f6a0 con 0x55b37eb99480 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84ff7fe640 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84fffff640 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84feffd640 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba1130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f84f4032f60 tx=0x7f84f4032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22451656c2675ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84deffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84deffd640 1 -- 172.21.15.5:0/2811525418 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4040070 con 0x55b37eb99480 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84deffd640 1 -- 172.21.15.5:0/2811525418 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f84f40054f0 con 0x55b37eb99480 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f8507e54740 1 -- 172.21.15.5:0/2811525418 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84f4004030 con 0x55b37eb99480 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f84deffd640 1 -- 172.21.15.5:0/2811525418 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f84f4035040 con 0x55b37eb99480 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f8507e54740 10 client.26915 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.905+0000 7f8507e54740 1 -- 172.21.15.5:0/2811525418 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b37ebb24e0 con 0x55b37eb99480 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f84deffd640 1 -- 172.21.15.5:0/2811525418 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f4004030 con 0x55b37eb99480 2024-09-11T22:28:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f84deffd640 1 -- 172.21.15.5:0/2811525418 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f84f4054030 con 0x55b37eb99480 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f8507e54740 10 client.26915 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f8507e54740 10 client.26915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f8507e54740 20 client.26915 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f8507e54740 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b37ebb3830 0x55b37ebb5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f8507e54740 4 client.26915 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f8507e54740 1 -- 172.21.15.5:0/2811525418 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b37eb0b7f0 con 0x55b37ebb3830 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f84ff7fe640 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b37ebb3830 0x55b37ebb5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f84ff7fe640 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b37ebb3830 0x55b37ebb5c20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d21f3586b5752ad server_cookie=23423ea5973573ce in_seq=0 out_seq=0 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.906+0000 7f84deffd640 10 client.26915 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.907+0000 7f84deffd640 1 -- 172.21.15.5:0/2811525418 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b37eb0b7f0 con 0x55b37ebb3830 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.907+0000 7f84deffd640 10 client.26915 handle_command_reply: tid=0 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.907+0000 7f8507e54740 1 client.26915 shutdown 2024-09-11T22:28:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.907+0000 7f8507e54740 1 -- 172.21.15.5:0/2811525418 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 msgr2=0x55b37eba1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.907+0000 7f8507e54740 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba1130 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f84f4032f60 tx=0x7f84f4032f90 comp rx=0 tx=0).stop 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.908+0000 7f8507e54740 1 -- 172.21.15.5:0/2811525418 shutdown_connections 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.908+0000 7f8507e54740 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b37ebb3830 0x55b37ebb5c20 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.908+0000 7f8507e54740 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.908+0000 7f8507e54740 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba1130 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.908+0000 7f8507e54740 1 --2- 172.21.15.5:0/2811525418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.908+0000 7f84deffd640 0 client.26915 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.908+0000 7f8507e54740 1 -- 172.21.15.5:0/2811525418 >> 172.21.15.5:0/2811525418 conn(0x55b37eb950f0 msgr2=0x55b37eb9ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.908+0000 7f8507e54740 1 -- 172.21.15.5:0/2811525418 shutdown_connections 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.908+0000 7f8507e54740 1 -- 172.21.15.5:0/2811525418 wait complete. 2024-09-11T22:28:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.908+0000 7f8507e54740 20 client.26915 trim_cache size 0 max 16384 2024-09-11T22:28:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f8507e54740 1 Processor -- start 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f8507e54740 1 -- start start 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f8507e54740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb98b60 0x55b37eba2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f8507e54740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb99480 0x55b37ebbb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f8507e54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37ebbb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f8507e54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85001ad9e0 con 0x55b37eba3d40 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f8507e54740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85001af520 con 0x55b37eb98b60 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f8507e54740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85001ae0b0 con 0x55b37eb99480 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f84fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37ebbb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f84fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37ebbb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50296/0 (socket says 172.21.15.5:50296) 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f84fffff640 1 -- 172.21.15.5:0/3900270170 learned_addr learned my addr 172.21.15.5:0/3900270170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.926+0000 7f84feffd640 1 --2- 172.21.15.5:0/3900270170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb99480 0x55b37ebbb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84ff7fe640 1 --2- 172.21.15.5:0/3900270170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb98b60 0x55b37eba2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84fffff640 1 -- 172.21.15.5:0/3900270170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb99480 msgr2=0x55b37ebbb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84fffff640 1 --2- 172.21.15.5:0/3900270170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb99480 0x55b37ebbb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84fffff640 1 -- 172.21.15.5:0/3900270170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb98b60 msgr2=0x55b37eba2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84fffff640 1 --2- 172.21.15.5:0/3900270170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb98b60 0x55b37eba2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84fffff640 1 -- 172.21.15.5:0/3900270170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37eb992b0 con 0x55b37eba3d40 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84feffd640 1 --2- 172.21.15.5:0/3900270170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb99480 0x55b37ebbb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84fffff640 1 --2- 172.21.15.5:0/3900270170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37ebbb760 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f84f8003f40 tx=0x7f84f8008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638d73942909d324 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84e57fa640 1 -- 172.21.15.5:0/3900270170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80045c0 con 0x55b37eba3d40 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84e57fa640 1 -- 172.21.15.5:0/3900270170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f8004760 con 0x55b37eba3d40 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.927+0000 7f84e57fa640 1 -- 172.21.15.5:0/3900270170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8010670 con 0x55b37eba3d40 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.928+0000 7f8507e54740 1 -- 172.21.15.5:0/3900270170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 msgr2=0x55b37ebbb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.928+0000 7f8507e54740 1 --2- 172.21.15.5:0/3900270170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37ebbb760 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f84f8003f40 tx=0x7f84f8008000 comp rx=0 tx=0).stop 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.928+0000 7f8507e54740 1 -- 172.21.15.5:0/3900270170 shutdown_connections 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.928+0000 7f8507e54740 1 --2- 172.21.15.5:0/3900270170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37ebbb760 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.928+0000 7f8507e54740 1 --2- 172.21.15.5:0/3900270170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb99480 0x55b37ebbb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.928+0000 7f8507e54740 1 --2- 172.21.15.5:0/3900270170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb98b60 0x55b37eba2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.928+0000 7f8507e54740 1 -- 172.21.15.5:0/3900270170 >> 172.21.15.5:0/3900270170 conn(0x55b37eb950f0 msgr2=0x55b37eba2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.928+0000 7f8507e54740 1 -- 172.21.15.5:0/3900270170 shutdown_connections 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.928+0000 7f8507e54740 1 -- 172.21.15.5:0/3900270170 wait complete. 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f8507e54740 1 -- start start 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f8507e54740 1 Processor -- start 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f8507e54740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f8507e54740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f8507e54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f8507e54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85001aee50 con 0x55b37eba3d40 2024-09-11T22:28:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f8507e54740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85001ae780 con 0x55b37eb99480 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f8507e54740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85001ab9c0 con 0x55b37eb98b60 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f8507e54740 10 client.0 _ll_register_callbacks cb 0x7f84f00980c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f84fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f84feffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f84ff7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f84feffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43232/0 (socket says 172.21.15.5:43232) 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f84fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50312/0 (socket says 172.21.15.5:50312) 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.929+0000 7f84feffd640 1 -- 172.21.15.5:0/3066833714 learned_addr learned my addr 172.21.15.5:0/3066833714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84feffd640 1 -- 172.21.15.5:0/3066833714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 msgr2=0x55b37eba7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84feffd640 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84feffd640 1 -- 172.21.15.5:0/3066833714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 msgr2=0x55b37eba8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84feffd640 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84feffd640 1 -- 172.21.15.5:0/3066833714 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b37ec1d4e0 con 0x55b37eb99480 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84ff7fe640 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84fffff640 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84feffd640 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba8200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f84f4001eb0 tx=0x7f84f4031b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d884f0329d5ab4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84deffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84deffd640 1 -- 172.21.15.5:0/3066833714 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4040040 con 0x55b37eb99480 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84deffd640 1 -- 172.21.15.5:0/3066833714 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f84f40055d0 con 0x55b37eb99480 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f8507e54740 1 -- 172.21.15.5:0/3066833714 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84f8004760 con 0x55b37eb99480 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84deffd640 1 -- 172.21.15.5:0/3066833714 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f84f4066050 con 0x55b37eb99480 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f8507e54740 10 client.26921 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f8507e54740 1 -- 172.21.15.5:0/3066833714 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b37ec1e7a0 con 0x55b37eb99480 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.930+0000 7f84deffd640 1 -- 172.21.15.5:0/3066833714 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f8004760 con 0x55b37eb99480 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.931+0000 7f84deffd640 1 -- 172.21.15.5:0/3066833714 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f84f405d070 con 0x55b37eb99480 2024-09-11T22:28:53.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.931+0000 7f8507e54740 10 client.26921 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.931+0000 7f8507e54740 10 client.26921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.931+0000 7f8507e54740 20 client.26921 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.931+0000 7f8507e54740 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b37ec1ee30 0x55b37ec21250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.931+0000 7f8507e54740 4 client.26921 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.931+0000 7f8507e54740 1 -- 172.21.15.5:0/3066833714 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b37eb05d00 con 0x55b37ec1ee30 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.931+0000 7f84ff7fe640 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b37ec1ee30 0x55b37ec21250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.931+0000 7f84ff7fe640 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b37ec1ee30 0x55b37ec21250 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=112597efb19dccf8 server_cookie=e08e5a544162d0c1 in_seq=0 out_seq=0 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.931+0000 7f84deffd640 10 client.26921 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.932+0000 7f84deffd640 1 -- 172.21.15.5:0/3066833714 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b37eb05d00 con 0x55b37ec1ee30 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.932+0000 7f84deffd640 10 client.26921 handle_command_reply: tid=0 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.932+0000 7f8507e54740 1 client.26921 shutdown 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 1 -- 172.21.15.5:0/3066833714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 msgr2=0x55b37eba8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba8200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f84f4001eb0 tx=0x7f84f4031b60 comp rx=0 tx=0).stop 2024-09-11T22:28:53.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 1 -- 172.21.15.5:0/3066833714 shutdown_connections 2024-09-11T22:28:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b37ec1ee30 0x55b37ec21250 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37eba3d40 0x55b37eba8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37eb99480 0x55b37eba8200 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 1 --2- 172.21.15.5:0/3066833714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b37eb98b60 0x55b37eba7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 1 -- 172.21.15.5:0/3066833714 >> 172.21.15.5:0/3066833714 conn(0x55b37eb950f0 msgr2=0x55b37eba2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f84deffd640 0 client.26921 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 1 -- 172.21.15.5:0/3066833714 shutdown_connections 2024-09-11T22:28:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 1 -- 172.21.15.5:0/3066833714 wait complete. 2024-09-11T22:28:53.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.933+0000 7f8507e54740 20 client.26921 trim_cache size 0 max 16384 2024-09-11T22:28:53.942 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:28:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:28:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "39242eb7-00cf-4228-ae5b-a26f3b048852", 2024-09-11T22:28:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:28:53.943 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:28:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 -- 172.21.15.5:0/2790015230 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84e80a2cf0 msgr2=0x7f84e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84e80a2cf0 0x7f84e80a51b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009fd0 tx=0x7f84f8009500 comp rx=0 tx=0).stop 2024-09-11T22:28:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 -- 172.21.15.5:0/2790015230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8500102d30 msgr2=0x7f8500198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8500102d30 0x7f8500198640 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f84f000e9f0 tx=0x7f84f000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 -- 172.21.15.5:0/2790015230 shutdown_connections 2024-09-11T22:28:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84e80a2cf0 0x7f84e80a51b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85001082a0 0x7f850019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85001036e0 0x7f8500198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 --2- 172.21.15.5:0/2790015230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8500102d30 0x7f8500198640 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 -- 172.21.15.5:0/2790015230 >> 172.21.15.5:0/2790015230 conn(0x7f85000fc7d0 msgr2=0x7f8500105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 -- 172.21.15.5:0/2790015230 shutdown_connections 2024-09-11T22:28:53.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:53.934+0000 7f8505a7b640 1 -- 172.21.15.5:0/2790015230 wait complete. 2024-09-11T22:28:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:53 smithi159 ceph-mon[35534]: pgmap v206: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 228 op/s 2024-09-11T22:28:54.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:53 smithi172 ceph-mon[35461]: pgmap v206: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 228 op/s 2024-09-11T22:28:54.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 495713 7.97 MB/sec execute 192 sec latency 44.638 ms 2024-09-11T22:28:54.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:28:55.052 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:28:55.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 497865 8.00 MB/sec execute 193 sec latency 34.391 ms 2024-09-11T22:28:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.623+0000 7f90cabfe640 1 -- 172.21.15.5:0/292194844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4100e20 msgr2=0x7f90c4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.623+0000 7f90cabfe640 1 --2- 172.21.15.5:0/292194844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4100e20 0x7f90c4101200 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009900 tx=0x7f90b802a480 comp rx=0 tx=0).stop 2024-09-11T22:28:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.623+0000 7f90cabfe640 1 -- 172.21.15.5:0/292194844 shutdown_connections 2024-09-11T22:28:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.623+0000 7f90cabfe640 1 --2- 172.21.15.5:0/292194844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90c4102ef0 0x7f90c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.623+0000 7f90cabfe640 1 --2- 172.21.15.5:0/292194844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90c4102570 0x7f90c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.623+0000 7f90cabfe640 1 --2- 172.21.15.5:0/292194844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4100e20 0x7f90c4101200 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.623+0000 7f90cabfe640 1 -- 172.21.15.5:0/292194844 >> 172.21.15.5:0/292194844 conn(0x7f90c40fc7d0 msgr2=0x7f90c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.623+0000 7f90cabfe640 1 -- 172.21.15.5:0/292194844 shutdown_connections 2024-09-11T22:28:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.623+0000 7f90cabfe640 1 -- 172.21.15.5:0/292194844 wait complete. 2024-09-11T22:28:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90cabfe640 1 Processor -- start 2024-09-11T22:28:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90cabfe640 1 -- start start 2024-09-11T22:28:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90cabfe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90c4100e20 0x7f90c41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90cabfe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90c4102570 0x7f90c4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90cabfe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4102ef0 0x7f90c419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90cabfe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4110f80 con 0x7f90c4102ef0 2024-09-11T22:28:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90cabfe640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4110e00 con 0x7f90c4102570 2024-09-11T22:28:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90cabfe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4111100 con 0x7f90c4100e20 2024-09-11T22:28:55.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c899a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90c4100e20 0x7f90c41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c919b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4102ef0 0x7f90c419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90c4102570 0x7f90c4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c899a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90c4100e20 0x7f90c41984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:51176/0 (socket says 172.21.15.5:51176) 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c899a640 1 -- 172.21.15.5:0/1478100647 learned_addr learned my addr 172.21.15.5:0/1478100647 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c899a640 1 -- 172.21.15.5:0/1478100647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90c4102570 msgr2=0x7f90c4198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c899a640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90c4102570 0x7f90c4198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c899a640 1 -- 172.21.15.5:0/1478100647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4102ef0 msgr2=0x7f90c419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c899a640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4102ef0 0x7f90c419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c899a640 1 -- 172.21.15.5:0/1478100647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c419d470 con 0x7f90c4100e20 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.624+0000 7f90c919b640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4102ef0 0x7f90c419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.625+0000 7f90c899a640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90c4100e20 0x7f90c41984c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f90b8006010 tx=0x7f90b8003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:55.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.625+0000 7f90c1ffb640 1 -- 172.21.15.5:0/1478100647 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b803f070 con 0x7f90c4100e20 2024-09-11T22:28:55.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.625+0000 7f90c1ffb640 1 -- 172.21.15.5:0/1478100647 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b8031c30 con 0x7f90c4100e20 2024-09-11T22:28:55.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.625+0000 7f90cabfe640 1 -- 172.21.15.5:0/1478100647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c419d700 con 0x7f90c4100e20 2024-09-11T22:28:55.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.625+0000 7f90c1ffb640 1 -- 172.21.15.5:0/1478100647 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8035030 con 0x7f90c4100e20 2024-09-11T22:28:55.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.625+0000 7f90cabfe640 1 -- 172.21.15.5:0/1478100647 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c41a5040 con 0x7f90c4100e20 2024-09-11T22:28:55.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90c1ffb640 1 -- 172.21.15.5:0/1478100647 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f90b8042070 con 0x7f90c4100e20 2024-09-11T22:28:55.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90ccfd7740 1 Processor -- start 2024-09-11T22:28:55.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90ccfd7740 1 -- start start 2024-09-11T22:28:55.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 0x56320bf35880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf35dc0 0x56320bf40140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf40680 0x56320bf42a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56320be9b2e0 con 0x56320bf40680 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56320be42ce0 con 0x56320bf35dc0 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56320be972a0 con 0x56320bf354a0 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90c919b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 0x56320bf35880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90c919b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 0x56320bf35880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:51178/0 (socket says 172.21.15.5:51178) 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90c919b640 1 -- 172.21.15.5:0/1170931488 learned_addr learned my addr 172.21.15.5:0/1170931488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90c3fff640 1 --2- 172.21.15.5:0/1170931488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf40680 0x56320bf42a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90c899a640 1 --2- 172.21.15.5:0/1170931488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf35dc0 0x56320bf40140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90c919b640 1 -- 172.21.15.5:0/1170931488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf35dc0 msgr2=0x56320bf40140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90c919b640 1 --2- 172.21.15.5:0/1170931488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf35dc0 0x56320bf40140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90c919b640 1 -- 172.21.15.5:0/1170931488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf40680 msgr2=0x56320bf42a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.628+0000 7f90c919b640 1 --2- 172.21.15.5:0/1170931488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf40680 0x56320bf42a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.629+0000 7f90c919b640 1 -- 172.21.15.5:0/1170931488 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320be27440 con 0x56320bf354a0 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.629+0000 7f90c899a640 1 --2- 172.21.15.5:0/1170931488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf35dc0 0x56320bf40140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.629+0000 7f90c1ffb640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90b40a2cf0 0x7f90b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.629+0000 7f90c919b640 1 --2- 172.21.15.5:0/1170931488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 0x56320bf35880 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0117e0 tx=0x7f90bc011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fad340a49b3d346 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.629+0000 7f90c899a640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90b40a2cf0 0x7f90b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.629+0000 7f90c1ffb640 1 -- 172.21.15.5:0/1478100647 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f90b80ef7c0 con 0x7f90c4100e20 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.629+0000 7f90c899a640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90b40a2cf0 0x7f90b40a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x56320bf32890 tx=0x7f90b802a480 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.631+0000 7f90aeffd640 1 -- 172.21.15.5:0/1170931488 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc011310 con 0x56320bf354a0 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.631+0000 7f90aeffd640 1 -- 172.21.15.5:0/1170931488 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90bc003e50 con 0x56320bf354a0 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.631+0000 7f90aeffd640 1 -- 172.21.15.5:0/1170931488 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc004620 con 0x56320bf354a0 2024-09-11T22:28:55.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1170931488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 msgr2=0x56320bf35880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1170931488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 0x56320bf35880 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0117e0 tx=0x7f90bc011ca0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1170931488 shutdown_connections 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1170931488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf40680 0x56320bf42a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1170931488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf35dc0 0x56320bf40140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1170931488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 0x56320bf35880 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1170931488 >> 172.21.15.5:0/1170931488 conn(0x56320bf31a30 msgr2=0x56320bf31e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1170931488 shutdown_connections 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1170931488 wait complete. 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 -- start start 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.632+0000 7f90ccfd7740 1 Processor -- start 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf354a0 0x56320bf40050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf35dc0 0x56320bf3d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 0x56320bf3d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56320bf43250 con 0x56320bf354a0 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56320be56050 con 0x56320bf40680 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56320bf433d0 con 0x56320bf35dc0 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90ccfd7740 10 client.0 _ll_register_callbacks cb 0x56320be96160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c919b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf354a0 0x56320bf40050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf35dc0 0x56320bf3d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf35dc0 0x56320bf3d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:51188/0 (socket says 172.21.15.5:51188) 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c919b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf354a0 0x56320bf40050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50354/0 (socket says 172.21.15.5:50354) 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c3fff640 1 -- 172.21.15.5:0/1031915444 learned_addr learned my addr 172.21.15.5:0/1031915444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c919b640 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 0x56320bf3d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c3fff640 1 -- 172.21.15.5:0/1031915444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 msgr2=0x56320bf3d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c3fff640 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 0x56320bf3d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c3fff640 1 -- 172.21.15.5:0/1031915444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf354a0 msgr2=0x56320bf40050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c3fff640 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf354a0 0x56320bf40050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.633+0000 7f90c3fff640 1 -- 172.21.15.5:0/1031915444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320bc0c3c0 con 0x56320bf35dc0 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.634+0000 7f90c3fff640 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf35dc0 0x56320bf3d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f90b0012700 tx=0x7f90b0012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55e9a04d62ce245 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.634+0000 7f90a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.634+0000 7f90a3fff640 1 -- 172.21.15.5:0/1031915444 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b000f8c0 con 0x56320bf35dc0 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.634+0000 7f90a3fff640 1 -- 172.21.15.5:0/1031915444 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f90b000fa60 con 0x56320bf35dc0 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.634+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1031915444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90bc003e50 con 0x56320bf35dc0 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.634+0000 7f90a3fff640 1 -- 172.21.15.5:0/1031915444 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f90b000fca0 con 0x56320bf35dc0 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.634+0000 7f90ccfd7740 10 client.26866 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.634+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1031915444 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56320bf4ece0 con 0x56320bf35dc0 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.634+0000 7f90a3fff640 1 -- 172.21.15.5:0/1031915444 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90bc003e50 con 0x56320bf35dc0 2024-09-11T22:28:55.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.634+0000 7f90a3fff640 1 -- 172.21.15.5:0/1031915444 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f90b0010430 con 0x56320bf35dc0 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.635+0000 7f90ccfd7740 10 client.26866 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.635+0000 7f90ccfd7740 10 client.26866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.635+0000 7f90ccfd7740 20 client.26866 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.635+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56320bf501f0 0x56320bf52630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.635+0000 7f90ccfd7740 4 client.26866 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.635+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1031915444 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56320bea8850 con 0x56320bf501f0 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.635+0000 7f90c919b640 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56320bf501f0 0x56320bf52630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.635+0000 7f90a3fff640 10 client.26866 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.635+0000 7f90c919b640 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56320bf501f0 0x56320bf52630 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e151c89dfdf7e0 server_cookie=8d06365e71a06f04 in_seq=0 out_seq=0 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.637+0000 7f90a3fff640 1 -- 172.21.15.5:0/1031915444 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56320bea8850 con 0x56320bf501f0 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90a3fff640 10 client.26866 handle_command_reply: tid=0 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 client.26866 shutdown 2024-09-11T22:28:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1031915444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf35dc0 msgr2=0x56320bf3d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:55.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf35dc0 0x56320bf3d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f90b0012700 tx=0x7f90b0012bc0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1031915444 shutdown_connections 2024-09-11T22:28:55.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56320bf501f0 0x56320bf52630 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 0x56320bf3d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf35dc0 0x56320bf3d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90a3fff640 0 client.26866 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:55.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1031915444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf354a0 0x56320bf40050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1031915444 >> 172.21.15.5:0/1031915444 conn(0x56320bf31a30 msgr2=0x56320bf30b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:55.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1031915444 shutdown_connections 2024-09-11T22:28:55.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1031915444 wait complete. 2024-09-11T22:28:55.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.638+0000 7f90ccfd7740 20 client.26866 trim_cache size 0 max 16384 2024-09-11T22:28:55.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.656+0000 7f90ccfd7740 1 Processor -- start 2024-09-11T22:28:55.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90ccfd7740 1 -- start start 2024-09-11T22:28:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf354a0 0x56320bf4c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf3bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf40680 0x56320bf57f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41ad910 con 0x56320bf35dc0 2024-09-11T22:28:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41af450 con 0x56320bf354a0 2024-09-11T22:28:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41adfe0 con 0x56320bf40680 2024-09-11T22:28:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf3bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c919b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf40680 0x56320bf57f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf3bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50368/0 (socket says 172.21.15.5:50368) 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c3fff640 1 -- 172.21.15.5:0/1639870398 learned_addr learned my addr 172.21.15.5:0/1639870398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c919b640 1 --2- 172.21.15.5:0/1639870398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf354a0 0x56320bf4c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c3fff640 1 -- 172.21.15.5:0/1639870398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf40680 msgr2=0x56320bf57f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c3fff640 1 --2- 172.21.15.5:0/1639870398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf40680 0x56320bf57f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c3fff640 1 -- 172.21.15.5:0/1639870398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf354a0 msgr2=0x56320bf4c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c3fff640 1 --2- 172.21.15.5:0/1639870398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf354a0 0x56320bf4c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c3fff640 1 -- 172.21.15.5:0/1639870398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320bf3c2b0 con 0x56320bf35dc0 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.657+0000 7f90c3fff640 1 --2- 172.21.15.5:0/1639870398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf3bbd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009e00 tx=0x7f90b00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403e52d07665a89e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ae7fc640 1 -- 172.21.15.5:0/1639870398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0010ac0 con 0x56320bf35dc0 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ae7fc640 1 -- 172.21.15.5:0/1639870398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b0010c60 con 0x56320bf35dc0 2024-09-11T22:28:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ae7fc640 1 -- 172.21.15.5:0/1639870398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b00035f0 con 0x56320bf35dc0 2024-09-11T22:28:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1639870398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 msgr2=0x56320bf3bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1639870398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf3bbd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009e00 tx=0x7f90b00096c0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1639870398 shutdown_connections 2024-09-11T22:28:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1639870398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf40680 0x56320bf57f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1639870398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf3bbd0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1639870398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf354a0 0x56320bf4c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1639870398 >> 172.21.15.5:0/1639870398 conn(0x56320bf31a30 msgr2=0x56320bf30b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.658+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1639870398 shutdown_connections 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.659+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1639870398 wait complete. 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.659+0000 7f90ccfd7740 1 -- start start 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.659+0000 7f90ccfd7740 1 Processor -- start 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.659+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 0x56320bf48920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.659+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf48e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.659+0000 7f90ccfd7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 0x56320bf456c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.659+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41aed80 con 0x56320bf35dc0 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.659+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41ae6b0 con 0x56320bf40680 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.659+0000 7f90ccfd7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41ab8f0 con 0x56320bf354a0 2024-09-11T22:28:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90ccfd7740 10 client.0 _ll_register_callbacks cb 0x7f90b0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf48e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c919b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 0x56320bf48920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf48e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50380/0 (socket says 172.21.15.5:50380) 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c3fff640 1 -- 172.21.15.5:0/1895871906 learned_addr learned my addr 172.21.15.5:0/1895871906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c3fff640 1 -- 172.21.15.5:0/1895871906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 msgr2=0x56320bf48920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c3fff640 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 0x56320bf48920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c3fff640 1 -- 172.21.15.5:0/1895871906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 msgr2=0x56320bf456c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c919b640 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 0x56320bf456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c3fff640 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 0x56320bf456c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c3fff640 1 -- 172.21.15.5:0/1895871906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320bf3dc60 con 0x56320bf35dc0 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c919b640 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 0x56320bf456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:28:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90c3fff640 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf48e60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009730 tx=0x7f90b0005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f45b52635d9725c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.660+0000 7f90a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90a3fff640 1 -- 172.21.15.5:0/1895871906 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0033060 con 0x56320bf35dc0 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1895871906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b0010c60 con 0x56320bf35dc0 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90a3fff640 1 -- 172.21.15.5:0/1895871906 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8585+0+0 (secure 0 0 0) 0x7f90b00055d0 con 0x56320bf35dc0 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90a3fff640 1 -- 172.21.15.5:0/1895871906 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f90b000f790 con 0x56320bf35dc0 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90ccfd7740 10 client.17550 fetch_fsmap learned FSMap version 38 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1895871906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56320bf403c0 con 0x56320bf35dc0 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90a3fff640 1 -- 172.21.15.5:0/1895871906 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b0010c60 con 0x56320bf35dc0 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90a3fff640 1 -- 172.21.15.5:0/1895871906 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f90b001c470 con 0x56320bf35dc0 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90ccfd7740 10 client.17550 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90ccfd7740 10 client.17550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:28:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90ccfd7740 20 client.17550 populate_metadata read hostname 'smithi005' 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56320bfbbda0 0x56320bfbe1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90ccfd7740 4 client.17550 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.661+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1895871906 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56320bea2d60 con 0x56320bfbbda0 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90c919b640 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56320bfbbda0 0x56320bfbe1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90c919b640 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56320bfbbda0 0x56320bfbe1c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9200d2cba01d72a1 server_cookie=416af72b2567bd21 in_seq=0 out_seq=0 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90a3fff640 10 client.17550 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90a3fff640 1 -- 172.21.15.5:0/1895871906 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56320bea2d60 con 0x56320bfbbda0 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90a3fff640 10 client.17550 handle_command_reply: tid=0 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90ccfd7740 1 client.17550 shutdown 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1895871906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 msgr2=0x56320bf48e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf48e60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009730 tx=0x7f90b0005b90 comp rx=0 tx=0).stop 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1895871906 shutdown_connections 2024-09-11T22:28:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56320bfbbda0 0x56320bfbe1c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56320bf40680 0x56320bf456c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320bf35dc0 0x56320bf48e60 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90ccfd7740 1 --2- 172.21.15.5:0/1895871906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56320bf354a0 0x56320bf48920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1895871906 >> 172.21.15.5:0/1895871906 conn(0x56320bf31a30 msgr2=0x56320bf38050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.662+0000 7f90a3fff640 0 client.17550 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:28:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1895871906 shutdown_connections 2024-09-11T22:28:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90ccfd7740 1 -- 172.21.15.5:0/1895871906 wait complete. 2024-09-11T22:28:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90ccfd7740 20 client.17550 trim_cache size 0 max 16384 2024-09-11T22:28:55.673 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:28:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:28:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:28:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "39242eb7-00cf-4228-ae5b-a26f3b048852": { 2024-09-11T22:28:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:28:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "39242eb7-00cf-4228-ae5b-a26f3b048852", 2024-09-11T22:28:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:28:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:28:55.674 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:28:55.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90cabfe640 1 -- 172.21.15.5:0/1478100647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90b40a2cf0 msgr2=0x7f90b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:55.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90cabfe640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90b40a2cf0 0x7f90b40a51b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x56320bf32890 tx=0x7f90b802a480 comp rx=0 tx=0).stop 2024-09-11T22:28:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90cabfe640 1 -- 172.21.15.5:0/1478100647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90c4100e20 msgr2=0x7f90c41984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:28:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90cabfe640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90c4100e20 0x7f90c41984c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f90b8006010 tx=0x7f90b8003c10 comp rx=0 tx=0).stop 2024-09-11T22:28:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90cabfe640 1 -- 172.21.15.5:0/1478100647 shutdown_connections 2024-09-11T22:28:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90cabfe640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90c4102ef0 0x7f90c419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90cabfe640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90b40a2cf0 0x7f90b40a51b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90cabfe640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90c4102570 0x7f90c4198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90cabfe640 1 --2- 172.21.15.5:0/1478100647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90c4100e20 0x7f90c41984c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:28:55.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.663+0000 7f90cabfe640 1 -- 172.21.15.5:0/1478100647 >> 172.21.15.5:0/1478100647 conn(0x7f90c40fc7d0 msgr2=0x7f90c406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:28:55.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.664+0000 7f90cabfe640 1 -- 172.21.15.5:0/1478100647 shutdown_connections 2024-09-11T22:28:55.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:28:55.664+0000 7f90cabfe640 1 -- 172.21.15.5:0/1478100647 wait complete. 2024-09-11T22:28:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:55 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:28:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:55 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:55 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:28:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:55 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:28:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:55 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:55 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:55 smithi005 ceph-mon[30943]: pgmap v207: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 3.7 MiB/s wr, 162 op/s 2024-09-11T22:28:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:55 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:28:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:55 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:55 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:28:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:55 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:28:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:55 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:56.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:55 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:56.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:55 smithi159 ceph-mon[35534]: pgmap v207: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 3.7 MiB/s wr, 162 op/s 2024-09-11T22:28:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:55 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:28:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:55 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:55 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:28:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:55 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:28:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:55 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:55 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:28:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:55 smithi172 ceph-mon[35461]: pgmap v207: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 3.7 MiB/s wr, 162 op/s 2024-09-11T22:28:56.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500005 8.00 MB/sec execute 194 sec latency 35.474 ms 2024-09-11T22:28:56.517 INFO:tasks.cephfs.filesystem:scrub status for tag:39242eb7-00cf-4228-ae5b-a26f3b048852 - {'path': '/', 'tag': '39242eb7-00cf-4228-ae5b-a26f3b048852', 'options': 'recursive,force'} 2024-09-11T22:28:57.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 502119 8.03 MB/sec execute 195 sec latency 14.288 ms 2024-09-11T22:28:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:57 smithi005 ceph-mon[30943]: pgmap v208: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 3.7 MiB/s wr, 162 op/s 2024-09-11T22:28:58.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:57 smithi159 ceph-mon[35534]: pgmap v208: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 3.7 MiB/s wr, 162 op/s 2024-09-11T22:28:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:57 smithi172 ceph-mon[35461]: pgmap v208: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 3.7 MiB/s wr, 162 op/s 2024-09-11T22:28:58.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 504628 8.07 MB/sec execute 196 sec latency 15.220 ms 2024-09-11T22:28:59.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 506473 8.06 MB/sec execute 197 sec latency 19.920 ms 2024-09-11T22:29:00.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:28:59 smithi005 ceph-mon[30943]: pgmap v209: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 348 op/s 2024-09-11T22:29:00.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:28:59 smithi159 ceph-mon[35534]: pgmap v209: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 348 op/s 2024-09-11T22:29:00.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:28:59 smithi172 ceph-mon[35461]: pgmap v209: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 348 op/s 2024-09-11T22:29:00.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 508599 8.09 MB/sec execute 198 sec latency 123.166 ms 2024-09-11T22:29:01.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 510126 8.07 MB/sec execute 199 sec latency 22.378 ms 2024-09-11T22:29:01.911 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:01 smithi172 ceph-mon[35461]: pgmap v210: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2024-09-11T22:29:02.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:01 smithi005 ceph-mon[30943]: pgmap v210: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2024-09-11T22:29:02.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:01 smithi159 ceph-mon[35534]: pgmap v210: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 290 op/s 2024-09-11T22:29:02.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 512291 8.10 MB/sec execute 200 sec latency 19.977 ms 2024-09-11T22:29:03.325 INFO:tasks.workunit.client.0.smithi005.stdout: 1 514776 8.13 MB/sec execute 201 sec latency 15.289 ms 2024-09-11T22:29:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:03 smithi005 ceph-mon[30943]: pgmap v211: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 409 op/s 2024-09-11T22:29:04.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:03 smithi159 ceph-mon[35534]: pgmap v211: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 409 op/s 2024-09-11T22:29:04.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:03 smithi172 ceph-mon[35461]: pgmap v211: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.0 MiB/s wr, 409 op/s 2024-09-11T22:29:04.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 516773 8.14 MB/sec execute 202 sec latency 19.444 ms 2024-09-11T22:29:05.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:04 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:29:05.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:04 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:29:05.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:04 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:29:05.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:04 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:29:05.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:04 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:29:05.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:04 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:29:05.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 518289 8.14 MB/sec execute 203 sec latency 33.929 ms 2024-09-11T22:29:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:05 smithi005 ceph-mon[30943]: pgmap v212: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 305 op/s 2024-09-11T22:29:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:05 smithi005 ceph-mon[30943]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T22:29:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:05 smithi159 ceph-mon[35534]: pgmap v212: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 305 op/s 2024-09-11T22:29:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:05 smithi159 ceph-mon[35534]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T22:29:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:05 smithi172 ceph-mon[35461]: pgmap v212: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.3 MiB/s wr, 305 op/s 2024-09-11T22:29:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:05 smithi172 ceph-mon[35461]: osdmap e89: 12 total, 12 up, 12 in 2024-09-11T22:29:06.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 519836 8.14 MB/sec execute 204 sec latency 46.373 ms 2024-09-11T22:29:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.623+0000 7ff774c5e640 1 -- 172.21.15.5:0/2129723834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff770101350 msgr2=0x7ff77010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.623+0000 7ff774c5e640 1 --2- 172.21.15.5:0/2129723834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff770101350 0x7ff77010d880 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7ff768009cd0 tx=0x7ff76802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.623+0000 7ff774c5e640 1 -- 172.21.15.5:0/2129723834 shutdown_connections 2024-09-11T22:29:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.623+0000 7ff774c5e640 1 --2- 172.21.15.5:0/2129723834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff770101350 0x7ff77010d880 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.623+0000 7ff774c5e640 1 --2- 172.21.15.5:0/2129723834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7701009b0 0x7ff770100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.623+0000 7ff774c5e640 1 --2- 172.21.15.5:0/2129723834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7701069c0 0x7ff770106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.623+0000 7ff774c5e640 1 -- 172.21.15.5:0/2129723834 >> 172.21.15.5:0/2129723834 conn(0x7ff770077990 msgr2=0x7ff7700fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.623+0000 7ff774c5e640 1 -- 172.21.15.5:0/2129723834 shutdown_connections 2024-09-11T22:29:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.623+0000 7ff774c5e640 1 -- 172.21.15.5:0/2129723834 wait complete. 2024-09-11T22:29:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff774c5e640 1 Processor -- start 2024-09-11T22:29:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff774c5e640 1 -- start start 2024-09-11T22:29:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff774c5e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7701009b0 0x7ff770199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff774c5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff770101350 0x7ff7701996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff774c5e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7701069c0 0x7ff7701a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff774c5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff770110e50 con 0x7ff770101350 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff774c5e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff770110cd0 con 0x7ff7701009b0 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff774c5e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff770110fd0 con 0x7ff7701069c0 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff767fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff770101350 0x7ff7701996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff76e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7701009b0 0x7ff770199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff767fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff770101350 0x7ff7701996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42408/0 (socket says 172.21.15.5:42408) 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff767fff640 1 -- 172.21.15.5:0/949040953 learned_addr learned my addr 172.21.15.5:0/949040953 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.624+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7701069c0 0x7ff7701a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff767fff640 1 -- 172.21.15.5:0/949040953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7701069c0 msgr2=0x7ff7701a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff767fff640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7701069c0 0x7ff7701a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff767fff640 1 -- 172.21.15.5:0/949040953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7701009b0 msgr2=0x7ff770199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff767fff640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7701009b0 0x7ff770199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff767fff640 1 -- 172.21.15.5:0/949040953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7701a4740 con 0x7ff770101350 2024-09-11T22:29:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff767fff640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff770101350 0x7ff7701996c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff75800e9f0 tx=0x7ff75800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff7677fe640 1 -- 172.21.15.5:0/949040953 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75800cd80 con 0x7ff770101350 2024-09-11T22:29:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff7677fe640 1 -- 172.21.15.5:0/949040953 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff758010910 con 0x7ff770101350 2024-09-11T22:29:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff774c5e640 1 -- 172.21.15.5:0/949040953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7701a4910 con 0x7ff770101350 2024-09-11T22:29:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff7677fe640 1 -- 172.21.15.5:0/949040953 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758018990 con 0x7ff770101350 2024-09-11T22:29:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.625+0000 7ff774c5e640 1 -- 172.21.15.5:0/949040953 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7701a4e80 con 0x7ff770101350 2024-09-11T22:29:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.628+0000 7ff7677fe640 1 -- 172.21.15.5:0/949040953 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff758010ac0 con 0x7ff770101350 2024-09-11T22:29:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.628+0000 7ff7779f7740 1 Processor -- start 2024-09-11T22:29:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.628+0000 7ff7779f7740 1 -- start start 2024-09-11T22:29:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 0x5630b2d8ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8d410 0x5630b2d97770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d97cb0 0x5630b2d9a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff7779f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630b2cf12f0 con 0x5630b2d8d410 2024-09-11T22:29:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff7779f7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5630b2c99270 con 0x5630b2d8caf0 2024-09-11T22:29:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff7779f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5630b2ced2b0 con 0x5630b2d97cb0 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76ed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 0x5630b2d8ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8d410 0x5630b2d97770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76ed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 0x5630b2d8ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44200/0 (socket says 172.21.15.5:44200) 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76ed9d640 1 -- 172.21.15.5:0/3855520155 learned_addr learned my addr 172.21.15.5:0/3855520155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff7677fe640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff75c0a2e20 0x7ff75c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff767fff640 1 --2- 172.21.15.5:0/3855520155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d97cb0 0x5630b2d9a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff75c0a2e20 0x7ff75c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76e59c640 1 -- 172.21.15.5:0/3855520155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d97cb0 msgr2=0x5630b2d9a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76e59c640 1 --2- 172.21.15.5:0/3855520155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d97cb0 0x5630b2d9a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76e59c640 1 -- 172.21.15.5:0/3855520155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 msgr2=0x5630b2d8ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76e59c640 1 --2- 172.21.15.5:0/3855520155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 0x5630b2d8ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76e59c640 1 -- 172.21.15.5:0/3855520155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b2c98340 con 0x5630b2d8d410 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff7677fe640 1 -- 172.21.15.5:0/949040953 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8673+0+0 (secure 0 0 0) 0x7ff7580cf7c0 con 0x7ff770101350 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76e59c640 1 --2- 172.21.15.5:0/3855520155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8d410 0x5630b2d97770 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff7600045b0 tx=0x7ff76000bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821871819d2864ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff75c0a2e20 0x7ff75c0a52e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff768009a60 tx=0x7ff76802a8e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.629+0000 7ff764ff9640 1 -- 172.21.15.5:0/3855520155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760003bd0 con 0x5630b2d8d410 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff764ff9640 1 -- 172.21.15.5:0/3855520155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff760015070 con 0x5630b2d8d410 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff764ff9640 1 -- 172.21.15.5:0/3855520155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760004d70 con 0x5630b2d8d410 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff7779f7740 1 -- 172.21.15.5:0/3855520155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8d410 msgr2=0x5630b2d97770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff7779f7740 1 --2- 172.21.15.5:0/3855520155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8d410 0x5630b2d97770 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff7600045b0 tx=0x7ff76000bb20 comp rx=0 tx=0).stop 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff7779f7740 1 -- 172.21.15.5:0/3855520155 shutdown_connections 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff7779f7740 1 --2- 172.21.15.5:0/3855520155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d97cb0 0x5630b2d9a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff7779f7740 1 --2- 172.21.15.5:0/3855520155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8d410 0x5630b2d97770 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff7779f7740 1 --2- 172.21.15.5:0/3855520155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 0x5630b2d8ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff7779f7740 1 -- 172.21.15.5:0/3855520155 >> 172.21.15.5:0/3855520155 conn(0x5630b2d89060 msgr2=0x5630b2d89440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff7779f7740 1 -- 172.21.15.5:0/3855520155 shutdown_connections 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.630+0000 7ff7779f7740 1 -- 172.21.15.5:0/3855520155 wait complete. 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff7779f7740 1 -- start start 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff7779f7740 1 Processor -- start 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 0x5630b2d94b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8d410 0x5630b2d95090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2d955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff7779f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630b2d9a880 con 0x5630b2d97cb0 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff7779f7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5630b2c7a290 con 0x5630b2d8caf0 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff7779f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5630b2d9aa00 con 0x5630b2d8d410 2024-09-11T22:29:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff7779f7740 10 client.0 _ll_register_callbacks cb 0x5630b2cab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff76ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2d955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff767fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8d410 0x5630b2d95090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff76e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 0x5630b2d94b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff76e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 0x5630b2d94b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44212/0 (socket says 172.21.15.5:44212) 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff76e59c640 1 -- 172.21.15.5:0/1367004134 learned_addr learned my addr 172.21.15.5:0/1367004134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff76ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2d955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42424/0 (socket says 172.21.15.5:42424) 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff767fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8d410 0x5630b2d95090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47846/0 (socket says 172.21.15.5:47846) 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff76ed9d640 1 -- 172.21.15.5:0/1367004134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8d410 msgr2=0x5630b2d95090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8d410 0x5630b2d95090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff76ed9d640 1 -- 172.21.15.5:0/1367004134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 msgr2=0x5630b2d94b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 0x5630b2d94b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.631+0000 7ff76ed9d640 1 -- 172.21.15.5:0/1367004134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b2a04210 con 0x5630b2d97cb0 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2d955d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff76802ae00 tx=0x7ff7680095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d06d781447de0b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff755ffb640 1 -- 172.21.15.5:0/1367004134 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76804e030 con 0x5630b2d97cb0 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff755ffb640 1 -- 172.21.15.5:0/1367004134 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8673+0+0 (secure 0 0 0) 0x7ff7680043d0 con 0x5630b2d97cb0 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff7779f7740 1 -- 172.21.15.5:0/1367004134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff760015070 con 0x5630b2d97cb0 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff755ffb640 1 -- 172.21.15.5:0/1367004134 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff7680045d0 con 0x5630b2d97cb0 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff7779f7740 10 client.17562 fetch_fsmap learned FSMap version 38 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff7779f7740 1 -- 172.21.15.5:0/1367004134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5630b2ad0620 con 0x5630b2d97cb0 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff755ffb640 1 -- 172.21.15.5:0/1367004134 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff760015070 con 0x5630b2d97cb0 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff755ffb640 1 -- 172.21.15.5:0/1367004134 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7ff768030070 con 0x5630b2d97cb0 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff7779f7740 10 client.17562 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff7779f7740 10 client.17562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff7779f7740 20 client.17562 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff7779f7740 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5630b2da7bc0 0x5630b2da9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff7779f7740 4 client.17562 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.632+0000 7ff7779f7740 1 -- 172.21.15.5:0/1367004134 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630b2cfea20 con 0x5630b2da7bc0 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.633+0000 7ff76e59c640 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5630b2da7bc0 0x5630b2da9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.633+0000 7ff755ffb640 10 client.17562 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.633+0000 7ff76e59c640 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5630b2da7bc0 0x5630b2da9ff0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=855e423b4925aa66 server_cookie=1456d9ca528dea8d in_seq=0 out_seq=0 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff755ffb640 1 -- 172.21.15.5:0/1367004134 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630b2cfea20 con 0x5630b2da7bc0 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff755ffb640 10 client.17562 handle_command_reply: tid=0 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 client.17562 shutdown 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 -- 172.21.15.5:0/1367004134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 msgr2=0x5630b2d955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2d955d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff76802ae00 tx=0x7ff7680095c0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 -- 172.21.15.5:0/1367004134 shutdown_connections 2024-09-11T22:29:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5630b2da7bc0 0x5630b2da9ff0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2d955d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8d410 0x5630b2d95090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 --2- 172.21.15.5:0/1367004134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8caf0 0x5630b2d94b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff755ffb640 0 client.17562 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 -- 172.21.15.5:0/1367004134 >> 172.21.15.5:0/1367004134 conn(0x5630b2d89060 msgr2=0x5630b2d8edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 -- 172.21.15.5:0/1367004134 shutdown_connections 2024-09-11T22:29:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 1 -- 172.21.15.5:0/1367004134 wait complete. 2024-09-11T22:29:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.634+0000 7ff7779f7740 20 client.17562 trim_cache size 0 max 16384 2024-09-11T22:29:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff7779f7740 1 Processor -- start 2024-09-11T22:29:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff7779f7740 1 -- start start 2024-09-11T22:29:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8caf0 0x5630b2d8f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8d410 0x5630b2d95400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2daf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff7779f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7701ad9d0 con 0x5630b2d97cb0 2024-09-11T22:29:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff7779f7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7701af510 con 0x5630b2d8d410 2024-09-11T22:29:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff7779f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7701ae0a0 con 0x5630b2d8caf0 2024-09-11T22:29:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff76ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2daf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff76ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2daf950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42428/0 (socket says 172.21.15.5:42428) 2024-09-11T22:29:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff76ed9d640 1 -- 172.21.15.5:0/3439843026 learned_addr learned my addr 172.21.15.5:0/3439843026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff76e59c640 1 --2- 172.21.15.5:0/3439843026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8caf0 0x5630b2d8f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.649+0000 7ff767fff640 1 --2- 172.21.15.5:0/3439843026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8d410 0x5630b2d95400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.650+0000 7ff76ed9d640 1 -- 172.21.15.5:0/3439843026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8caf0 msgr2=0x5630b2d8f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.650+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/3439843026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8caf0 0x5630b2d8f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.650+0000 7ff76ed9d640 1 -- 172.21.15.5:0/3439843026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8d410 msgr2=0x5630b2d95400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.650+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/3439843026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8d410 0x5630b2d95400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.650+0000 7ff76ed9d640 1 -- 172.21.15.5:0/3439843026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630b2daa670 con 0x5630b2d97cb0 2024-09-11T22:29:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.650+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/3439843026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2daf950 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff768005120 tx=0x7ff768009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d2cddf6e4caf65 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.650+0000 7ff757fff640 1 -- 172.21.15.5:0/3439843026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76804e070 con 0x5630b2d97cb0 2024-09-11T22:29:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.650+0000 7ff757fff640 1 -- 172.21.15.5:0/3439843026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff768036040 con 0x5630b2d97cb0 2024-09-11T22:29:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.650+0000 7ff757fff640 1 -- 172.21.15.5:0/3439843026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768030cc0 con 0x5630b2d97cb0 2024-09-11T22:29:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 -- 172.21.15.5:0/3439843026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 msgr2=0x5630b2daf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 --2- 172.21.15.5:0/3439843026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2daf950 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff768005120 tx=0x7ff768009890 comp rx=0 tx=0).stop 2024-09-11T22:29:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 -- 172.21.15.5:0/3439843026 shutdown_connections 2024-09-11T22:29:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 --2- 172.21.15.5:0/3439843026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d97cb0 0x5630b2daf950 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 --2- 172.21.15.5:0/3439843026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8d410 0x5630b2d95400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 --2- 172.21.15.5:0/3439843026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d8caf0 0x5630b2d8f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 -- 172.21.15.5:0/3439843026 >> 172.21.15.5:0/3439843026 conn(0x5630b2d89060 msgr2=0x5630b2d96bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 -- 172.21.15.5:0/3439843026 shutdown_connections 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 -- 172.21.15.5:0/3439843026 wait complete. 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 -- start start 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.651+0000 7ff7779f7740 1 Processor -- start 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8caf0 0x5630b2d9ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8d410 0x5630b2da04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff7779f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d97cb0 0x5630b2d913e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff7779f7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7701aee40 con 0x5630b2d8caf0 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff7779f7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7701ae770 con 0x5630b2d8d410 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff7779f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff7701ab9d0 con 0x5630b2d97cb0 2024-09-11T22:29:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff7779f7740 10 client.0 _ll_register_callbacks cb 0x7ff768009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff76e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8caf0 0x5630b2d9ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff76e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8caf0 0x5630b2d9ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42432/0 (socket says 172.21.15.5:42432) 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff76e59c640 1 -- 172.21.15.5:0/966061454 learned_addr learned my addr 172.21.15.5:0/966061454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff767fff640 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8d410 0x5630b2da04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff76ed9d640 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d97cb0 0x5630b2d913e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff76e59c640 1 -- 172.21.15.5:0/966061454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d97cb0 msgr2=0x5630b2d913e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff76e59c640 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d97cb0 0x5630b2d913e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff76e59c640 1 -- 172.21.15.5:0/966061454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8d410 msgr2=0x5630b2da04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff76e59c640 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8d410 0x5630b2da04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff76e59c640 1 -- 172.21.15.5:0/966061454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630b2d842d0 con 0x5630b2d8caf0 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff76e59c640 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8caf0 0x5630b2d9ff60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff7600045b0 tx=0x7ff76000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425111a7a7791b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff755ffb640 1 -- 172.21.15.5:0/966061454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff760011440 con 0x5630b2d8caf0 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff755ffb640 1 -- 172.21.15.5:0/966061454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8673+0+0 (secure 0 0 0) 0x7ff7600115e0 con 0x5630b2d8caf0 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.652+0000 7ff7779f7740 1 -- 172.21.15.5:0/966061454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff768036040 con 0x5630b2d8caf0 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff755ffb640 1 -- 172.21.15.5:0/966061454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff7600117e0 con 0x5630b2d8caf0 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff7779f7740 10 client.17568 fetch_fsmap learned FSMap version 38 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff7779f7740 1 -- 172.21.15.5:0/966061454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5630b2e16720 con 0x5630b2d8caf0 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff755ffb640 1 -- 172.21.15.5:0/966061454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff768036040 con 0x5630b2d8caf0 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff755ffb640 1 -- 172.21.15.5:0/966061454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7ff760019980 con 0x5630b2d8caf0 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff7779f7740 10 client.17568 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff7779f7740 10 client.17568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff7779f7740 20 client.17568 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff7779f7740 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5630b2e16cb0 0x5630b2e190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff7779f7740 4 client.17568 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:29:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff7779f7740 1 -- 172.21.15.5:0/966061454 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630b2cf8d70 con 0x5630b2e16cb0 2024-09-11T22:29:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.653+0000 7ff767fff640 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5630b2e16cb0 0x5630b2e190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.654+0000 7ff755ffb640 10 client.17568 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.654+0000 7ff767fff640 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5630b2e16cb0 0x5630b2e190a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6213dc168394a91 server_cookie=776e540a7ae76d49 in_seq=0 out_seq=0 2024-09-11T22:29:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.780+0000 7ff755ffb640 1 -- 172.21.15.5:0/966061454 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630b2cf8d70 con 0x5630b2e16cb0 2024-09-11T22:29:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.780+0000 7ff755ffb640 10 client.17568 handle_command_reply: tid=0 2024-09-11T22:29:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.780+0000 7ff7779f7740 1 client.17568 shutdown 2024-09-11T22:29:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.780+0000 7ff7779f7740 1 -- 172.21.15.5:0/966061454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8caf0 msgr2=0x5630b2d9ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.780+0000 7ff7779f7740 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8caf0 0x5630b2d9ff60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7ff7600045b0 tx=0x7ff76000bf90 comp rx=0 tx=0).stop 2024-09-11T22:29:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.780+0000 7ff7779f7740 1 -- 172.21.15.5:0/966061454 shutdown_connections 2024-09-11T22:29:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.780+0000 7ff7779f7740 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5630b2e16cb0 0x5630b2e190a0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.780+0000 7ff755ffb640 0 client.17568 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.780+0000 7ff7779f7740 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5630b2d97cb0 0x5630b2d913e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff7779f7740 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5630b2d8d410 0x5630b2da04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff7779f7740 1 --2- 172.21.15.5:0/966061454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630b2d8caf0 0x5630b2d9ff60 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff7779f7740 1 -- 172.21.15.5:0/966061454 >> 172.21.15.5:0/966061454 conn(0x5630b2d89060 msgr2=0x5630b2d879a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff7779f7740 1 -- 172.21.15.5:0/966061454 shutdown_connections 2024-09-11T22:29:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff7779f7740 1 -- 172.21.15.5:0/966061454 wait complete. 2024-09-11T22:29:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff7779f7740 20 client.17568 trim_cache size 0 max 16384 2024-09-11T22:29:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:29:06.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:29:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:29:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:29:06.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff774c5e640 1 -- 172.21.15.5:0/949040953 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff75c0a2e20 msgr2=0x7ff75c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff774c5e640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff75c0a2e20 0x7ff75c0a52e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff768009a60 tx=0x7ff76802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff774c5e640 1 -- 172.21.15.5:0/949040953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff770101350 msgr2=0x7ff7701996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff774c5e640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff770101350 0x7ff7701996c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff75800e9f0 tx=0x7ff75800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff774c5e640 1 -- 172.21.15.5:0/949040953 shutdown_connections 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.781+0000 7ff774c5e640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7701069c0 0x7ff7701a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.782+0000 7ff774c5e640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff770101350 0x7ff7701996c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.782+0000 7ff774c5e640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff75c0a2e20 0x7ff75c0a52e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.782+0000 7ff774c5e640 1 --2- 172.21.15.5:0/949040953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7701009b0 0x7ff770199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.782+0000 7ff774c5e640 1 -- 172.21.15.5:0/949040953 >> 172.21.15.5:0/949040953 conn(0x7ff770077990 msgr2=0x7ff7700fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.782+0000 7ff774c5e640 1 -- 172.21.15.5:0/949040953 shutdown_connections 2024-09-11T22:29:06.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:06.782+0000 7ff774c5e640 1 -- 172.21.15.5:0/949040953 wait complete. 2024-09-11T22:29:07.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 522046 8.16 MB/sec execute 205 sec latency 87.853 ms 2024-09-11T22:29:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:07 smithi005 ceph-mon[30943]: pgmap v214: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 366 op/s 2024-09-11T22:29:08.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:07 smithi159 ceph-mon[35534]: pgmap v214: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 366 op/s 2024-09-11T22:29:08.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:07 smithi172 ceph-mon[35461]: pgmap v214: 97 pgs: 97 active+clean; 91 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.8 MiB/s wr, 366 op/s 2024-09-11T22:29:08.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 523863 8.17 MB/sec execute 206 sec latency 13.951 ms 2024-09-11T22:29:09.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 526156 8.20 MB/sec execute 207 sec latency 17.355 ms 2024-09-11T22:29:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:09 smithi005 ceph-mon[30943]: pgmap v215: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T22:29:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:09 smithi159 ceph-mon[35534]: pgmap v215: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T22:29:10.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:09 smithi172 ceph-mon[35461]: pgmap v215: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T22:29:10.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 528262 8.20 MB/sec execute 208 sec latency 19.596 ms 2024-09-11T22:29:11.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 530240 8.21 MB/sec execute 209 sec latency 14.448 ms 2024-09-11T22:29:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:11 smithi005 ceph-mon[30943]: pgmap v216: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T22:29:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:11 smithi159 ceph-mon[35534]: pgmap v216: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T22:29:12.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:11 smithi172 ceph-mon[35461]: pgmap v216: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T22:29:12.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 532876 8.25 MB/sec execute 210 sec latency 15.523 ms 2024-09-11T22:29:13.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 534312 8.25 MB/sec execute 211 sec latency 214.776 ms 2024-09-11T22:29:13.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:13 smithi005 ceph-mon[30943]: pgmap v217: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-09-11T22:29:13.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:13 smithi005 ceph-mon[30943]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T22:29:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:13 smithi159 ceph-mon[35534]: pgmap v217: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-09-11T22:29:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:13 smithi159 ceph-mon[35534]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T22:29:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:13 smithi172 ceph-mon[35461]: pgmap v217: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-09-11T22:29:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:13 smithi172 ceph-mon[35461]: osdmap e90: 12 total, 12 up, 12 in 2024-09-11T22:29:14.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 536671 8.27 MB/sec execute 212 sec latency 15.530 ms 2024-09-11T22:29:15.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 538413 8.26 MB/sec execute 213 sec latency 58.927 ms 2024-09-11T22:29:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:15 smithi005 ceph-mon[30943]: pgmap v219: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 307 op/s 2024-09-11T22:29:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:15 smithi159 ceph-mon[35534]: pgmap v219: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 307 op/s 2024-09-11T22:29:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:15 smithi172 ceph-mon[35461]: pgmap v219: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 307 op/s 2024-09-11T22:29:16.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 540599 8.29 MB/sec execute 214 sec latency 14.886 ms 2024-09-11T22:29:17.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 543153 8.31 MB/sec execute 215 sec latency 15.894 ms 2024-09-11T22:29:17.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:17 smithi005 ceph-mon[30943]: pgmap v220: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-09-11T22:29:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:17 smithi159 ceph-mon[35534]: pgmap v220: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-09-11T22:29:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:17 smithi172 ceph-mon[35461]: pgmap v220: 97 pgs: 97 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 290 op/s 2024-09-11T22:29:18.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 545079 8.32 MB/sec execute 216 sec latency 15.733 ms 2024-09-11T22:29:19.326 INFO:tasks.workunit.client.0.smithi005.stdout: 1 547465 8.35 MB/sec execute 217 sec latency 13.995 ms 2024-09-11T22:29:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:19 smithi005 ceph-mon[30943]: pgmap v221: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-09-11T22:29:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:19 smithi159 ceph-mon[35534]: pgmap v221: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-09-11T22:29:19.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:19 smithi172 ceph-mon[35461]: pgmap v221: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-09-11T22:29:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.873+0000 7f4c424a6640 1 -- 172.21.15.5:0/2007358805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c3c1035e0 msgr2=0x7f4c3c10bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.873+0000 7f4c424a6640 1 --2- 172.21.15.5:0/2007358805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c3c1035e0 0x7f4c3c10bac0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009900 tx=0x7f4c2402a480 comp rx=0 tx=0).stop 2024-09-11T22:29:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.873+0000 7f4c424a6640 1 -- 172.21.15.5:0/2007358805 shutdown_connections 2024-09-11T22:29:19.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.873+0000 7f4c424a6640 1 --2- 172.21.15.5:0/2007358805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c3c10c110 0x7f4c3c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.873+0000 7f4c424a6640 1 --2- 172.21.15.5:0/2007358805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c3c1035e0 0x7f4c3c10bac0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.873+0000 7f4c424a6640 1 --2- 172.21.15.5:0/2007358805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3c102cc0 0x7f4c3c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.873+0000 7f4c424a6640 1 -- 172.21.15.5:0/2007358805 >> 172.21.15.5:0/2007358805 conn(0x7f4c3c0fc6b0 msgr2=0x7f4c3c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.873+0000 7f4c424a6640 1 -- 172.21.15.5:0/2007358805 shutdown_connections 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c424a6640 1 -- 172.21.15.5:0/2007358805 wait complete. 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c424a6640 1 Processor -- start 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c424a6640 1 -- start start 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c424a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c3c102cc0 0x7f4c3c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c424a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c3c1035e0 0x7f4c3c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c424a6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3c10c110 0x7f4c3c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c424a6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c111ef0 con 0x7f4c3c1035e0 2024-09-11T22:29:19.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c424a6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c111d70 con 0x7f4c3c10c110 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c424a6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c112070 con 0x7f4c3c102cc0 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c3c1035e0 0x7f4c3c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c3c1035e0 0x7f4c3c1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38880/0 (socket says 172.21.15.5:38880) 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.874+0000 7f4c33fff640 1 -- 172.21.15.5:0/1696925881 learned_addr learned my addr 172.21.15.5:0/1696925881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c3bfff640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c3c102cc0 0x7f4c3c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c40a43640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3c10c110 0x7f4c3c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c33fff640 1 -- 172.21.15.5:0/1696925881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c3c102cc0 msgr2=0x7f4c3c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c33fff640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c3c102cc0 0x7f4c3c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c33fff640 1 -- 172.21.15.5:0/1696925881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3c10c110 msgr2=0x7f4c3c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c33fff640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3c10c110 0x7f4c3c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c33fff640 1 -- 172.21.15.5:0/1696925881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3c19d370 con 0x7f4c3c1035e0 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c33fff640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c3c1035e0 0x7f4c3c1988c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006010 tx=0x7f4c24004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:19.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c39ffb640 1 -- 172.21.15.5:0/1696925881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2403f030 con 0x7f4c3c1035e0 2024-09-11T22:29:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c39ffb640 1 -- 172.21.15.5:0/1696925881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c24004440 con 0x7f4c3c1035e0 2024-09-11T22:29:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c424a6640 1 -- 172.21.15.5:0/1696925881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3c19d600 con 0x7f4c3c1035e0 2024-09-11T22:29:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c39ffb640 1 -- 172.21.15.5:0/1696925881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24035070 con 0x7f4c3c1035e0 2024-09-11T22:29:19.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.875+0000 7f4c424a6640 1 -- 172.21.15.5:0/1696925881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3c1a4ef0 con 0x7f4c3c1035e0 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c39ffb640 1 -- 172.21.15.5:0/1696925881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4c24042030 con 0x7f4c3c1035e0 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c4523a740 1 Processor -- start 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c4523a740 1 -- start start 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c4523a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593d550e2f0 con 0x5593d55b4cb0 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c4523a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5593d54b6270 con 0x5593d55aa410 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c4523a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5593d550a2b0 con 0x5593d55a9af0 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38882/0 (socket says 172.21.15.5:38882) 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c33fff640 1 -- 172.21.15.5:0/164317334 learned_addr learned my addr 172.21.15.5:0/164317334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c3bfff640 1 --2- 172.21.15.5:0/164317334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c40a43640 1 --2- 172.21.15.5:0/164317334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c39ffb640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4c280a2e20 0x7f4c280a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c33fff640 1 -- 172.21.15.5:0/164317334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 msgr2=0x5593d55a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c33fff640 1 --2- 172.21.15.5:0/164317334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55a9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c33fff640 1 -- 172.21.15.5:0/164317334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 msgr2=0x5593d55b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c33fff640 1 --2- 172.21.15.5:0/164317334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c33fff640 1 -- 172.21.15.5:0/164317334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d54b5340 con 0x5593d55b4cb0 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.879+0000 7f4c40a43640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4c280a2e20 0x7f4c280a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.880+0000 7f4c39ffb640 1 -- 172.21.15.5:0/1696925881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8629+0+0 (secure 0 0 0) 0x7f4c240ef780 con 0x7f4c3c1035e0 2024-09-11T22:29:19.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.880+0000 7f4c33fff640 1 --2- 172.21.15.5:0/164317334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b70a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4c240ecdd0 tx=0x7f4c2408acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1635fd300ed1e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.880+0000 7f4c327fc640 1 -- 172.21.15.5:0/164317334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24049070 con 0x5593d55b4cb0 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.880+0000 7f4c40a43640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4c280a2e20 0x7f4c280a52e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4c34005fd0 tx=0x7f4c34014040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.880+0000 7f4c327fc640 1 -- 172.21.15.5:0/164317334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c240bbce0 con 0x5593d55b4cb0 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.880+0000 7f4c327fc640 1 -- 172.21.15.5:0/164317334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c240c1170 con 0x5593d55b4cb0 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 -- 172.21.15.5:0/164317334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 msgr2=0x5593d55b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 --2- 172.21.15.5:0/164317334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b70a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4c240ecdd0 tx=0x7f4c2408acd0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 -- 172.21.15.5:0/164317334 shutdown_connections 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 --2- 172.21.15.5:0/164317334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b70a0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 --2- 172.21.15.5:0/164317334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 --2- 172.21.15.5:0/164317334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 -- 172.21.15.5:0/164317334 >> 172.21.15.5:0/164317334 conn(0x5593d55a6060 msgr2=0x5593d55a6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 -- 172.21.15.5:0/164317334 shutdown_connections 2024-09-11T22:29:19.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 -- 172.21.15.5:0/164317334 wait complete. 2024-09-11T22:29:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 -- start start 2024-09-11T22:29:19.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.881+0000 7f4c4523a740 1 Processor -- start 2024-09-11T22:29:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c4523a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593d55b7640 con 0x5593d55b4cb0 2024-09-11T22:29:19.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c4523a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5593d5497290 con 0x5593d55aa410 2024-09-11T22:29:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c4523a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5593d55b77c0 con 0x5593d55a9af0 2024-09-11T22:29:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c4523a740 10 client.0 _ll_register_callbacks cb 0x5593d54c8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c40a43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c3bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c40a43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38886/0 (socket says 172.21.15.5:38886) 2024-09-11T22:29:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c40a43640 1 -- 172.21.15.5:0/1118461847 learned_addr learned my addr 172.21.15.5:0/1118461847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c33fff640 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c40a43640 1 -- 172.21.15.5:0/1118461847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 msgr2=0x5593d55b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c40a43640 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c40a43640 1 -- 172.21.15.5:0/1118461847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 msgr2=0x5593d55b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c40a43640 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c40a43640 1 -- 172.21.15.5:0/1118461847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d5221210 con 0x5593d55b4cb0 2024-09-11T22:29:19.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c3bfff640 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c40a43640 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b2520 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4c34010830 tx=0x7f4c34009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4324b5c1b19fd9e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/1118461847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3401b830 con 0x5593d55b4cb0 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/1118461847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8629+0+0 (secure 0 0 0) 0x7f4c3401b9d0 con 0x5593d55b4cb0 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c4523a740 1 -- 172.21.15.5:0/1118461847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c240bbce0 con 0x5593d55b4cb0 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.882+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/1118461847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4c34010e20 con 0x5593d55b4cb0 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c4523a740 10 client.17580 fetch_fsmap learned FSMap version 38 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c4523a740 1 -- 172.21.15.5:0/1118461847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5593d52ed620 con 0x5593d55b4cb0 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/1118461847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c240bbce0 con 0x5593d55b4cb0 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/1118461847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f4c340273f0 con 0x5593d55b4cb0 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c4523a740 10 client.17580 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c4523a740 10 client.17580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c4523a740 20 client.17580 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c4523a740 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593d55c48f0 0x5593d55c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c4523a740 4 client.17580 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c4523a740 1 -- 172.21.15.5:0/1118461847 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593d551ba20 con 0x5593d55c48f0 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.883+0000 7f4c3bfff640 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593d55c48f0 0x5593d55c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.884+0000 7f4c3bfff640 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593d55c48f0 0x5593d55c6d40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4d762cafdd91c5 server_cookie=6c871b0f01e836e7 in_seq=0 out_seq=0 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.884+0000 7f4c1b7fe640 10 client.17580 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/1118461847 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593d551ba20 con 0x5593d55c48f0 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c1b7fe640 10 client.17580 handle_command_reply: tid=0 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c4523a740 1 client.17580 shutdown 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c4523a740 1 -- 172.21.15.5:0/1118461847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 msgr2=0x5593d55b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c4523a740 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b2520 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4c34010830 tx=0x7f4c34009500 comp rx=0 tx=0).stop 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c4523a740 1 -- 172.21.15.5:0/1118461847 shutdown_connections 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c4523a740 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593d55c48f0 0x5593d55c6d40 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c4523a740 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55b2520 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c4523a740 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c1b7fe640 0 client.17580 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c4523a740 1 --2- 172.21.15.5:0/1118461847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.885+0000 7f4c4523a740 1 -- 172.21.15.5:0/1118461847 >> 172.21.15.5:0/1118461847 conn(0x5593d55a6060 msgr2=0x5593d55abdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.886+0000 7f4c4523a740 1 -- 172.21.15.5:0/1118461847 shutdown_connections 2024-09-11T22:29:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.886+0000 7f4c4523a740 1 -- 172.21.15.5:0/1118461847 wait complete. 2024-09-11T22:29:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.886+0000 7f4c4523a740 20 client.17580 trim_cache size 0 max 16384 2024-09-11T22:29:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.900+0000 7f4c4523a740 1 Processor -- start 2024-09-11T22:29:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c4523a740 1 -- start start 2024-09-11T22:29:19.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c4523a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1ab830 con 0x5593d55b4cb0 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c4523a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1ad370 con 0x5593d55aa410 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c4523a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1abf00 con 0x5593d55a9af0 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c40a43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c3bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c33fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c40a43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38896/0 (socket says 172.21.15.5:38896) 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c3bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:55180/0 (socket says 172.21.15.5:55180) 2024-09-11T22:29:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c40a43640 1 -- 172.21.15.5:0/1468955437 learned_addr learned my addr 172.21.15.5:0/1468955437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c40a43640 1 -- 172.21.15.5:0/1468955437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 msgr2=0x5593d55b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c40a43640 1 --2- 172.21.15.5:0/1468955437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c40a43640 1 -- 172.21.15.5:0/1468955437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 msgr2=0x5593d55b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c40a43640 1 --2- 172.21.15.5:0/1468955437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c40a43640 1 -- 172.21.15.5:0/1468955437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d55c73c0 con 0x5593d55b4cb0 2024-09-11T22:29:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c3bfff640 1 --2- 172.21.15.5:0/1468955437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.901+0000 7f4c40a43640 1 --2- 172.21.15.5:0/1468955437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55cc6a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4c34000f80 tx=0x7f4c34027ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db46867cf938488b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c31ffb640 1 -- 172.21.15.5:0/1468955437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34032070 con 0x5593d55b4cb0 2024-09-11T22:29:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c31ffb640 1 -- 172.21.15.5:0/1468955437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c34027cd0 con 0x5593d55b4cb0 2024-09-11T22:29:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c31ffb640 1 -- 172.21.15.5:0/1468955437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34037460 con 0x5593d55b4cb0 2024-09-11T22:29:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c4523a740 1 -- 172.21.15.5:0/1468955437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 msgr2=0x5593d55cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c4523a740 1 --2- 172.21.15.5:0/1468955437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55cc6a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4c34000f80 tx=0x7f4c34027ea0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c4523a740 1 -- 172.21.15.5:0/1468955437 shutdown_connections 2024-09-11T22:29:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c4523a740 1 --2- 172.21.15.5:0/1468955437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55b4cb0 0x5593d55cc6a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c4523a740 1 --2- 172.21.15.5:0/1468955437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55aa410 0x5593d55b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c4523a740 1 --2- 172.21.15.5:0/1468955437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c4523a740 1 -- 172.21.15.5:0/1468955437 >> 172.21.15.5:0/1468955437 conn(0x5593d55a6060 msgr2=0x5593d55b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c4523a740 1 -- 172.21.15.5:0/1468955437 shutdown_connections 2024-09-11T22:29:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.902+0000 7f4c4523a740 1 -- 172.21.15.5:0/1468955437 wait complete. 2024-09-11T22:29:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c4523a740 1 -- start start 2024-09-11T22:29:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c4523a740 1 Processor -- start 2024-09-11T22:29:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55bd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55aa410 0x5593d55b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c4523a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55b4cb0 0x5593d55b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c4523a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1acca0 con 0x5593d55aa410 2024-09-11T22:29:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c4523a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1ac5d0 con 0x5593d55b4cb0 2024-09-11T22:29:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c4523a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3c1a9830 con 0x5593d55a9af0 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c4523a740 10 client.0 _ll_register_callbacks cb 0x7f4c34009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55aa410 0x5593d55b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55aa410 0x5593d55b8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38906/0 (socket says 172.21.15.5:38906) 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c33fff640 1 -- 172.21.15.5:0/3620778392 learned_addr learned my addr 172.21.15.5:0/3620778392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c3bfff640 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55bd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.903+0000 7f4c40a43640 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55b4cb0 0x5593d55b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c33fff640 1 -- 172.21.15.5:0/3620778392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 msgr2=0x5593d55bd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c33fff640 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55bd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c33fff640 1 -- 172.21.15.5:0/3620778392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55b4cb0 msgr2=0x5593d55b8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c33fff640 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55b4cb0 0x5593d55b8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c33fff640 1 -- 172.21.15.5:0/3620778392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d55a5e90 con 0x5593d55aa410 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c33fff640 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55aa410 0x5593d55b8590 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4c24000c00 tx=0x7f4c2402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823f475887cf3a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/3620778392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24049040 con 0x5593d55aa410 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/3620778392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8629+0+0 (secure 0 0 0) 0x7f4c24003cd0 con 0x5593d55aa410 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c4523a740 1 -- 172.21.15.5:0/3620778392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c34027cd0 con 0x5593d55aa410 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/3620778392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4c240bffe0 con 0x5593d55aa410 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c4523a740 10 client.17586 fetch_fsmap learned FSMap version 38 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c4523a740 1 -- 172.21.15.5:0/3620778392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5593d562f5c0 con 0x5593d55aa410 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/3620778392 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c34027cd0 con 0x5593d55aa410 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.904+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/3620778392 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f4c240ed0e0 con 0x5593d55aa410 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.905+0000 7f4c4523a740 10 client.17586 fetch_fsmap finished waiting for FSMap version 38 2024-09-11T22:29:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.905+0000 7f4c4523a740 10 client.17586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.905+0000 7f4c4523a740 20 client.17586 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.905+0000 7f4c4523a740 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593d562f9a0 0x5593d5631dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.905+0000 7f4c4523a740 4 client.17586 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:29:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.905+0000 7f4c4523a740 1 -- 172.21.15.5:0/3620778392 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593d5515d70 con 0x5593d562f9a0 2024-09-11T22:29:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.905+0000 7f4c3bfff640 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593d562f9a0 0x5593d5631dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.905+0000 7f4c3bfff640 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593d562f9a0 0x5593d5631dc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56c4c66dc69920f server_cookie=403943d75e45e99f in_seq=0 out_seq=0 2024-09-11T22:29:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:19.905+0000 7f4c1b7fe640 10 client.17586 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:20.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.047+0000 7f4c1b7fe640 1 -- 172.21.15.5:0/3620778392 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593d5515d70 con 0x5593d562f9a0 2024-09-11T22:29:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.047+0000 7f4c1b7fe640 10 client.17586 handle_command_reply: tid=0 2024-09-11T22:29:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.047+0000 7f4c4523a740 1 client.17586 shutdown 2024-09-11T22:29:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.047+0000 7f4c4523a740 1 -- 172.21.15.5:0/3620778392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55aa410 msgr2=0x5593d55b8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.047+0000 7f4c4523a740 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55aa410 0x5593d55b8590 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4c24000c00 tx=0x7f4c2402a480 comp rx=0 tx=0).stop 2024-09-11T22:29:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.047+0000 7f4c4523a740 1 -- 172.21.15.5:0/3620778392 shutdown_connections 2024-09-11T22:29:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.047+0000 7f4c4523a740 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593d562f9a0 0x5593d5631dc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.047+0000 7f4c4523a740 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593d55b4cb0 0x5593d55b8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.047+0000 7f4c4523a740 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d55aa410 0x5593d55b8590 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.047+0000 7f4c4523a740 1 --2- 172.21.15.5:0/3620778392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593d55a9af0 0x5593d55bd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:20.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.048+0000 7f4c4523a740 1 -- 172.21.15.5:0/3620778392 >> 172.21.15.5:0/3620778392 conn(0x5593d55a6060 msgr2=0x5593d55ab550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.048+0000 7f4c1b7fe640 0 client.17586 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.048+0000 7f4c4523a740 1 -- 172.21.15.5:0/3620778392 shutdown_connections 2024-09-11T22:29:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.048+0000 7f4c4523a740 1 -- 172.21.15.5:0/3620778392 wait complete. 2024-09-11T22:29:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.048+0000 7f4c4523a740 20 client.17586 trim_cache size 0 max 16384 2024-09-11T22:29:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:29:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:29:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:29:20.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:29:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 -- 172.21.15.5:0/1696925881 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4c280a2e20 msgr2=0x7f4c280a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4c280a2e20 0x7f4c280a52e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4c34005fd0 tx=0x7f4c34014040 comp rx=0 tx=0).stop 2024-09-11T22:29:20.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 -- 172.21.15.5:0/1696925881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c3c1035e0 msgr2=0x7f4c3c1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c3c1035e0 0x7f4c3c1988c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006010 tx=0x7f4c24004040 comp rx=0 tx=0).stop 2024-09-11T22:29:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 -- 172.21.15.5:0/1696925881 shutdown_connections 2024-09-11T22:29:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4c280a2e20 0x7f4c280a52e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4c3c10c110 0x7f4c3c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c3c1035e0 0x7f4c3c1988c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 --2- 172.21.15.5:0/1696925881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c3c102cc0 0x7f4c3c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 -- 172.21.15.5:0/1696925881 >> 172.21.15.5:0/1696925881 conn(0x7f4c3c0fc6b0 msgr2=0x7f4c3c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 -- 172.21.15.5:0/1696925881 shutdown_connections 2024-09-11T22:29:20.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:20.049+0000 7f4c424a6640 1 -- 172.21.15.5:0/1696925881 wait complete. 2024-09-11T22:29:20.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 549466 8.34 MB/sec execute 218 sec latency 104.149 ms 2024-09-11T22:29:21.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 551206 8.36 MB/sec execute 219 sec latency 46.167 ms 2024-09-11T22:29:21.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:29:20 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setuser ceph since I am not root 2024-09-11T22:29:21.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:29:20 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setgroup ceph since I am not root 2024-09-11T22:29:21.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:29:20 smithi159 ceph-mds[65034]: -- 172.21.15.159:0/108927576 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e5398bf860 con 0x55e53989d000 2024-09-11T22:29:21.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:29:20 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: starting mds.k at 2024-09-11T22:29:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:21 smithi005 ceph-mon[30943]: pgmap v222: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-09-11T22:29:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:21 smithi005 ceph-mon[30943]: daemon mds.k restarted 2024-09-11T22:29:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:21 smithi159 ceph-mon[35534]: pgmap v222: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-09-11T22:29:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:21 smithi159 ceph-mon[35534]: daemon mds.k restarted 2024-09-11T22:29:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:21 smithi172 ceph-mon[35461]: pgmap v222: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-09-11T22:29:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:21 smithi172 ceph-mon[35461]: daemon mds.k restarted 2024-09-11T22:29:22.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 553600 8.38 MB/sec execute 220 sec latency 14.857 ms 2024-09-11T22:29:22.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:22 smithi005 ceph-mon[30943]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T22:29:22.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:22 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6832/1755675305,v1:172.21.15.159:6833/1755675305] up:boot 2024-09-11T22:29:22.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:22 smithi005 ceph-mon[30943]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:29:22.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:22 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:29:22.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:22 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:29:22.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:22 smithi159 ceph-mon[35534]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T22:29:22.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:22 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6832/1755675305,v1:172.21.15.159:6833/1755675305] up:boot 2024-09-11T22:29:22.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:22 smithi159 ceph-mon[35534]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:29:22.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:22 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:29:22.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:22 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:29:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:22 smithi172 ceph-mon[35461]: osdmap e91: 12 total, 12 up, 12 in 2024-09-11T22:29:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:22 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6832/1755675305,v1:172.21.15.159:6833/1755675305] up:boot 2024-09-11T22:29:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:22 smithi172 ceph-mon[35461]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:29:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:22 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:29:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:22 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:29:23.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555666 8.39 MB/sec execute 221 sec latency 15.975 ms 2024-09-11T22:29:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:23 smithi005 ceph-mon[30943]: pgmap v224: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 360 op/s 2024-09-11T22:29:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:23 smithi159 ceph-mon[35534]: pgmap v224: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 360 op/s 2024-09-11T22:29:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:23 smithi172 ceph-mon[35461]: pgmap v224: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 360 op/s 2024-09-11T22:29:23.995 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-11T22:29:23.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:29:24.267 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:24.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 558046 8.42 MB/sec execute 222 sec latency 15.628 ms 2024-09-11T22:29:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.790+0000 7f066ff96640 1 -- 172.21.15.5:0/2403300591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668103780 msgr2=0x7f0668107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.790+0000 7f066ff96640 1 --2- 172.21.15.5:0/2403300591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668103780 0x7f0668107bc0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0658009930 tx=0x7f065802a420 comp rx=0 tx=0).stop 2024-09-11T22:29:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.790+0000 7f066ff96640 1 -- 172.21.15.5:0/2403300591 shutdown_connections 2024-09-11T22:29:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.790+0000 7f066ff96640 1 --2- 172.21.15.5:0/2403300591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0668108300 0x7f066810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.790+0000 7f066ff96640 1 --2- 172.21.15.5:0/2403300591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668103780 0x7f0668107bc0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.790+0000 7f066ff96640 1 --2- 172.21.15.5:0/2403300591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0668102dd0 0x7f06681031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.790+0000 7f066ff96640 1 -- 172.21.15.5:0/2403300591 >> 172.21.15.5:0/2403300591 conn(0x7f06680fc7d0 msgr2=0x7f06680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.790+0000 7f066ff96640 1 -- 172.21.15.5:0/2403300591 shutdown_connections 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.790+0000 7f066ff96640 1 -- 172.21.15.5:0/2403300591 wait complete. 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066ff96640 1 Processor -- start 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066ff96640 1 -- start start 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066ff96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668102dd0 0x7f06681985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066ff96640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0668103780 0x7f0668198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066ff96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0668108300 0x7f066819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066ff96640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0668112050 con 0x7f0668102dd0 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066ff96640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0668111ed0 con 0x7f0668103780 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066ff96640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f06681121d0 con 0x7f0668108300 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066dd32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668102dd0 0x7f06681985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066dd32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668102dd0 0x7f06681985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49718/0 (socket says 172.21.15.5:49718) 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066dd32640 1 -- 172.21.15.5:0/1751475350 learned_addr learned my addr 172.21.15.5:0/1751475350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066d531640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0668103780 0x7f0668198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.791+0000 7f066e533640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0668108300 0x7f066819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f066dd32640 1 -- 172.21.15.5:0/1751475350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0668108300 msgr2=0x7f066819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f066dd32640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0668108300 0x7f066819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f066dd32640 1 -- 172.21.15.5:0/1751475350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0668103780 msgr2=0x7f0668198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f066dd32640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0668103780 0x7f0668198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f066dd32640 1 -- 172.21.15.5:0/1751475350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066819d580 con 0x7f0668102dd0 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f066e533640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0668108300 0x7f066819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f066dd32640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668102dd0 0x7f06681985d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f066400e9f0 tx=0x7f066400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f065effd640 1 -- 172.21.15.5:0/1751475350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066400cd80 con 0x7f0668102dd0 2024-09-11T22:29:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f066ff96640 1 -- 172.21.15.5:0/1751475350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066819d870 con 0x7f0668102dd0 2024-09-11T22:29:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f065effd640 1 -- 172.21.15.5:0/1751475350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0664010910 con 0x7f0668102dd0 2024-09-11T22:29:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f065effd640 1 -- 172.21.15.5:0/1751475350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06640188f0 con 0x7f0668102dd0 2024-09-11T22:29:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.792+0000 7f066ff96640 1 -- 172.21.15.5:0/1751475350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06681a5150 con 0x7f0668102dd0 2024-09-11T22:29:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.794+0000 7f066ff96640 1 -- 172.21.15.5:0/1751475350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06681031b0 con 0x7f0668102dd0 2024-09-11T22:29:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.795+0000 7f065effd640 1 -- 172.21.15.5:0/1751475350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0664010ac0 con 0x7f0668102dd0 2024-09-11T22:29:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.795+0000 7f065effd640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f06540a2cf0 0x7f06540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.795+0000 7f066d531640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f06540a2cf0 0x7f06540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.796+0000 7f065effd640 1 -- 172.21.15.5:0/1751475350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f06640cf940 con 0x7f0668102dd0 2024-09-11T22:29:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.796+0000 7f066d531640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f06540a2cf0 0x7f06540a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0658005940 tx=0x7f06580058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:24.800+0000 7f065effd640 1 -- 172.21.15.5:0/1751475350 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f066409c520 con 0x7f0668102dd0 2024-09-11T22:29:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.029+0000 7f066ff96640 1 -- 172.21.15.5:0/1751475350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0668199930 con 0x7f0668102dd0 2024-09-11T22:29:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.030+0000 7f065effd640 1 -- 172.21.15.5:0/1751475350 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11967 (secure 0 0 0) 0x7f066409be20 con 0x7f0668102dd0 2024-09-11T22:29:25.031 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:29:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"btime":"2024-09-11T22:29:21:436884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:29:21.436856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26914":{"gid":26914,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1755675305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1755675305},{"type":"v1","addr":"172.21.15.159:6833","nonce":1755675305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:29:25.031 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 40 2024-09-11T22:29:25.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.033+0000 7f066ff96640 1 -- 172.21.15.5:0/1751475350 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f06540a2cf0 msgr2=0x7f06540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:25.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.033+0000 7f066ff96640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f06540a2cf0 0x7f06540a51b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0658005940 tx=0x7f06580058d0 comp rx=0 tx=0).stop 2024-09-11T22:29:25.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.033+0000 7f066ff96640 1 -- 172.21.15.5:0/1751475350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668102dd0 msgr2=0x7f06681985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:25.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.033+0000 7f066ff96640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668102dd0 0x7f06681985d0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f066400e9f0 tx=0x7f066400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:29:25.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.033+0000 7f066ff96640 1 -- 172.21.15.5:0/1751475350 shutdown_connections 2024-09-11T22:29:25.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.034+0000 7f066ff96640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0668108300 0x7f066819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:25.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.034+0000 7f066ff96640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f06540a2cf0 0x7f06540a51b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:25.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.034+0000 7f066ff96640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0668103780 0x7f0668198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:25.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.034+0000 7f066ff96640 1 --2- 172.21.15.5:0/1751475350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0668102dd0 0x7f06681985d0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:25.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.034+0000 7f066ff96640 1 -- 172.21.15.5:0/1751475350 >> 172.21.15.5:0/1751475350 conn(0x7f06680fc7d0 msgr2=0x7f0668105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:25.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.034+0000 7f066ff96640 1 -- 172.21.15.5:0/1751475350 shutdown_connections 2024-09-11T22:29:25.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:25.034+0000 7f066ff96640 1 -- 172.21.15.5:0/1751475350 wait complete. 2024-09-11T22:29:25.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 560145 8.41 MB/sec execute 223 sec latency 23.775 ms 2024-09-11T22:29:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:25 smithi005 ceph-mon[30943]: pgmap v225: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 347 op/s 2024-09-11T22:29:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:25 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1751475350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:29:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:25 smithi159 ceph-mon[35534]: pgmap v225: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 347 op/s 2024-09-11T22:29:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:25 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1751475350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:29:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:25 smithi172 ceph-mon[35461]: pgmap v225: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 347 op/s 2024-09-11T22:29:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:25 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1751475350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:29:25.907 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-11T22:29:25.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-11T22:29:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:26.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 561858 8.43 MB/sec execute 224 sec latency 81.226 ms 2024-09-11T22:29:26.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:29:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.869+0000 7f7240ba7640 1 -- 172.21.15.5:0/3932832386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f723c102d30 msgr2=0x7f723c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.869+0000 7f7240ba7640 1 --2- 172.21.15.5:0/3932832386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f723c102d30 0x7f723c103110 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f722c009930 tx=0x7f722c02a420 comp rx=0 tx=0).stop 2024-09-11T22:29:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.869+0000 7f7240ba7640 1 -- 172.21.15.5:0/3932832386 shutdown_connections 2024-09-11T22:29:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.869+0000 7f7240ba7640 1 --2- 172.21.15.5:0/3932832386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c1082a0 0x7f723c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.869+0000 7f7240ba7640 1 --2- 172.21.15.5:0/3932832386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f723c1036e0 0x7f723c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.869+0000 7f7240ba7640 1 --2- 172.21.15.5:0/3932832386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f723c102d30 0x7f723c103110 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.869+0000 7f7240ba7640 1 -- 172.21.15.5:0/3932832386 >> 172.21.15.5:0/3932832386 conn(0x7f723c0fc7d0 msgr2=0x7f723c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.869+0000 7f7240ba7640 1 -- 172.21.15.5:0/3932832386 shutdown_connections 2024-09-11T22:29:27.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.869+0000 7f7240ba7640 1 -- 172.21.15.5:0/3932832386 wait complete. 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f7240ba7640 1 Processor -- start 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f7240ba7640 1 -- start start 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f7240ba7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c102d30 0x7f723c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f7240ba7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f723c1036e0 0x7f723c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f7240ba7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f723c1082a0 0x7f723c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f7240ba7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f723c1120f0 con 0x7f723c1082a0 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f7240ba7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f723c111f70 con 0x7f723c1036e0 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f7240ba7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f723c112270 con 0x7f723c102d30 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f723a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c102d30 0x7f723c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f723ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f723c1082a0 0x7f723c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f7239d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f723c1036e0 0x7f723c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f723a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c102d30 0x7f723c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45990/0 (socket says 172.21.15.5:45990) 2024-09-11T22:29:27.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.870+0000 7f723a59c640 1 -- 172.21.15.5:0/1254381233 learned_addr learned my addr 172.21.15.5:0/1254381233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f723a59c640 1 -- 172.21.15.5:0/1254381233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f723c1036e0 msgr2=0x7f723c198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f723a59c640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f723c1036e0 0x7f723c198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f723a59c640 1 -- 172.21.15.5:0/1254381233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f723c1082a0 msgr2=0x7f723c19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f723a59c640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f723c1082a0 0x7f723c19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f723a59c640 1 -- 172.21.15.5:0/1254381233 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c19d620 con 0x7f723c102d30 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f723ad9d640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f723c1082a0 0x7f723c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f7239d9b640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f723c1036e0 0x7f723c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f723a59c640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c102d30 0x7f723c198670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f722c02a940 tx=0x7f722c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f722b7fe640 1 -- 172.21.15.5:0/1254381233 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c03f070 con 0x7f723c102d30 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f7240ba7640 1 -- 172.21.15.5:0/1254381233 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723c19d8b0 con 0x7f723c102d30 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f722b7fe640 1 -- 172.21.15.5:0/1254381233 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f722c031c90 con 0x7f723c102d30 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f722b7fe640 1 -- 172.21.15.5:0/1254381233 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c035030 con 0x7f723c102d30 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.871+0000 7f7240ba7640 1 -- 172.21.15.5:0/1254381233 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723c1a51f0 con 0x7f723c102d30 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.874+0000 7f722b7fe640 1 -- 172.21.15.5:0/1254381233 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f722c042070 con 0x7f723c102d30 2024-09-11T22:29:27.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.874+0000 7f7242f80740 1 Processor -- start 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f7242f80740 1 -- start start 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f7242f80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 0x561616967a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f7242f80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967f60 0x561616976810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f7242f80740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616976d50 0x561616979140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f7242f80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5616168cd2a0 con 0x561616967f60 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f7242f80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561616874b10 con 0x561616967640 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f7242f80740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5616168c9260 con 0x561616976d50 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f723a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967f60 0x561616976810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f723ad9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 0x561616967a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f7239d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616976d50 0x561616979140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f723ad9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 0x561616967a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49824/0 (socket says 172.21.15.5:49824) 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f723ad9d640 1 -- 172.21.15.5:0/2094334250 learned_addr learned my addr 172.21.15.5:0/2094334250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f723a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967f60 0x561616976810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49734/0 (socket says 172.21.15.5:49734) 2024-09-11T22:29:27.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f723ad9d640 1 -- 172.21.15.5:0/2094334250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616976d50 msgr2=0x561616979140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f723ad9d640 1 --2- 172.21.15.5:0/2094334250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616976d50 0x561616979140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f723ad9d640 1 -- 172.21.15.5:0/2094334250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967f60 msgr2=0x561616976810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f723ad9d640 1 --2- 172.21.15.5:0/2094334250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967f60 0x561616976810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f722b7fe640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72240a2cf0 0x7f72240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f723ad9d640 1 -- 172.21.15.5:0/2094334250 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56161663eb90 con 0x561616967640 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.875+0000 7f7239d9b640 1 --2- 172.21.15.5:0/2094334250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616976d50 0x561616979140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.876+0000 7f7239d9b640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72240a2cf0 0x7f72240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.876+0000 7f722b7fe640 1 -- 172.21.15.5:0/1254381233 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f722c0efb60 con 0x7f723c102d30 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.876+0000 7f723ad9d640 1 --2- 172.21.15.5:0/2094334250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 0x561616967a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7234010710 tx=0x7f7234010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9af15e6e4e75fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.876+0000 7f7228ff9640 1 -- 172.21.15.5:0/2094334250 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723400e8d0 con 0x561616967640 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.876+0000 7f7228ff9640 1 -- 172.21.15.5:0/2094334250 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7234012450 con 0x561616967640 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.876+0000 7f7228ff9640 1 -- 172.21.15.5:0/2094334250 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723401c670 con 0x561616967640 2024-09-11T22:29:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 -- 172.21.15.5:0/2094334250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 msgr2=0x561616967a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7239d9b640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72240a2cf0 0x7f72240a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7230009fd0 tx=0x7f7230005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 --2- 172.21.15.5:0/2094334250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 0x561616967a20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7234010710 tx=0x7f7234010bd0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 -- 172.21.15.5:0/2094334250 shutdown_connections 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 --2- 172.21.15.5:0/2094334250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616976d50 0x561616979140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 --2- 172.21.15.5:0/2094334250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967f60 0x561616976810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 --2- 172.21.15.5:0/2094334250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 0x561616967a20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 -- 172.21.15.5:0/2094334250 >> 172.21.15.5:0/2094334250 conn(0x561616963cf0 msgr2=0x5616169640d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 -- 172.21.15.5:0/2094334250 shutdown_connections 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 -- 172.21.15.5:0/2094334250 wait complete. 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 -- start start 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.877+0000 7f7242f80740 1 Processor -- start 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7242f80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967640 0x561616973c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7242f80740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 0x5616169741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7242f80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616976d50 0x5616169746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7242f80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561616979920 con 0x561616967640 2024-09-11T22:29:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7242f80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5616168556e0 con 0x561616976d50 2024-09-11T22:29:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7242f80740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561616979aa0 con 0x561616967f60 2024-09-11T22:29:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7242f80740 10 client.0 _ll_register_callbacks cb 0x561616879a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f723a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967640 0x561616973c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f723a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967640 0x561616973c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49748/0 (socket says 172.21.15.5:49748) 2024-09-11T22:29:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f723a59c640 1 -- 172.21.15.5:0/3148438689 learned_addr learned my addr 172.21.15.5:0/3148438689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7239d9b640 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 0x5616169741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f723ad9d640 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616976d50 0x5616169746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f723a59c640 1 -- 172.21.15.5:0/3148438689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 msgr2=0x5616169741b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f723a59c640 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 0x5616169741b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f723a59c640 1 -- 172.21.15.5:0/3148438689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616976d50 msgr2=0x5616169746f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f723a59c640 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616976d50 0x5616169746f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f723a59c640 1 -- 172.21.15.5:0/3148438689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561616982e80 con 0x561616967640 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f723a59c640 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967640 0x561616973c70 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f722c006010 tx=0x7f722c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4baca3a8b7a7dc6d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7221ffb640 1 -- 172.21.15.5:0/3148438689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c049030 con 0x561616967640 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7221ffb640 1 -- 172.21.15.5:0/3148438689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f722c0ed4f0 con 0x561616967640 2024-09-11T22:29:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7242f80740 1 -- 172.21.15.5:0/3148438689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7234012450 con 0x561616967640 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7221ffb640 1 -- 172.21.15.5:0/3148438689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f722c0bbc30 con 0x561616967640 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7242f80740 10 client.17610 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.878+0000 7f7242f80740 1 -- 172.21.15.5:0/3148438689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561616985810 con 0x561616967640 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.879+0000 7f7221ffb640 1 -- 172.21.15.5:0/3148438689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7234012450 con 0x561616967640 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.879+0000 7f7221ffb640 1 -- 172.21.15.5:0/3148438689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f722c0b8760 con 0x561616967640 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.879+0000 7f7242f80740 10 client.17610 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.879+0000 7f7242f80740 10 client.17610 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.879+0000 7f7242f80740 20 client.17610 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.879+0000 7f7242f80740 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561616986c70 0x5616169890e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.879+0000 7f7242f80740 4 client.17610 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.879+0000 7f7242f80740 1 -- 172.21.15.5:0/3148438689 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616168da9d0 con 0x561616986c70 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.879+0000 7f7239d9b640 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561616986c70 0x5616169890e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7221ffb640 10 client.17610 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7239d9b640 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561616986c70 0x5616169890e0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7bdb15b0e9f748 server_cookie=6036162bfcfc5601 in_seq=0 out_seq=0 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7221ffb640 1 -- 172.21.15.5:0/3148438689 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616168da9d0 con 0x561616986c70 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7221ffb640 10 client.17610 handle_command_reply: tid=0 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 client.17610 shutdown 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 -- 172.21.15.5:0/3148438689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967640 msgr2=0x561616973c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967640 0x561616973c70 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f722c006010 tx=0x7f722c009620 comp rx=0 tx=0).stop 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 -- 172.21.15.5:0/3148438689 shutdown_connections 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561616986c70 0x5616169890e0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616976d50 0x5616169746f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 0x5616169741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 --2- 172.21.15.5:0/3148438689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616967640 0x561616973c70 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7221ffb640 0 client.17610 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 -- 172.21.15.5:0/3148438689 >> 172.21.15.5:0/3148438689 conn(0x561616963cf0 msgr2=0x561616969a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 -- 172.21.15.5:0/3148438689 shutdown_connections 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.880+0000 7f7242f80740 1 -- 172.21.15.5:0/3148438689 wait complete. 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.881+0000 7f7242f80740 20 client.17610 trim_cache size 0 max 16384 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.897+0000 7f7242f80740 1 Processor -- start 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.897+0000 7f7242f80740 1 -- start start 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f7242f80740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967640 0x561616962d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f7242f80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967f60 0x56161698e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f7242f80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161698ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f7242f80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f723c1ab850 con 0x561616976d50 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f7242f80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f723c1ad390 con 0x561616967f60 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f7242f80740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f723c1abf20 con 0x561616967640 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f723ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161698ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f723a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967640 0x561616962d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f723ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161698ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49758/0 (socket says 172.21.15.5:49758) 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f723ad9d640 1 -- 172.21.15.5:0/3750230260 learned_addr learned my addr 172.21.15.5:0/3750230260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f7239d9b640 1 --2- 172.21.15.5:0/3750230260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967f60 0x56161698e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f723ad9d640 1 -- 172.21.15.5:0/3750230260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967640 msgr2=0x561616962d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f723ad9d640 1 --2- 172.21.15.5:0/3750230260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967640 0x561616962d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f723ad9d640 1 -- 172.21.15.5:0/3750230260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967f60 msgr2=0x56161698e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f723ad9d640 1 --2- 172.21.15.5:0/3750230260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967f60 0x56161698e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f723ad9d640 1 -- 172.21.15.5:0/3750230260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561616963790 con 0x561616976d50 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.898+0000 7f723ad9d640 1 --2- 172.21.15.5:0/3750230260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161698ef00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f72340092a0 tx=0x7f7234012e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5e9f567ce3b455 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.899+0000 7f7223fff640 1 -- 172.21.15.5:0/3750230260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723400e8d0 con 0x561616976d50 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.899+0000 7f7223fff640 1 -- 172.21.15.5:0/3750230260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f723402b070 con 0x561616976d50 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.899+0000 7f7223fff640 1 -- 172.21.15.5:0/3750230260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723401c410 con 0x561616976d50 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.899+0000 7f7242f80740 1 -- 172.21.15.5:0/3750230260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 msgr2=0x56161698ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.899+0000 7f7242f80740 1 --2- 172.21.15.5:0/3750230260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161698ef00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f72340092a0 tx=0x7f7234012e70 comp rx=0 tx=0).stop 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.899+0000 7f7242f80740 1 -- 172.21.15.5:0/3750230260 shutdown_connections 2024-09-11T22:29:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.899+0000 7f7242f80740 1 --2- 172.21.15.5:0/3750230260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161698ef00 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.899+0000 7f7242f80740 1 --2- 172.21.15.5:0/3750230260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967f60 0x56161698e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.899+0000 7f7242f80740 1 --2- 172.21.15.5:0/3750230260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967640 0x561616962d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.899+0000 7f7242f80740 1 -- 172.21.15.5:0/3750230260 >> 172.21.15.5:0/3750230260 conn(0x561616963cf0 msgr2=0x561616962650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 1 -- 172.21.15.5:0/3750230260 shutdown_connections 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 1 -- 172.21.15.5:0/3750230260 wait complete. 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 1 -- start start 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 1 Processor -- start 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 0x56161697f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 0x56161697b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161697bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f723c1accc0 con 0x561616976d50 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f723c1ac5f0 con 0x561616967640 2024-09-11T22:29:27.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f723c1a9830 con 0x561616967f60 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7242f80740 10 client.0 _ll_register_callbacks cb 0x7f722c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f723ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161697bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f723a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 0x56161697f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7239d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 0x56161697b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f723ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161697bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49774/0 (socket says 172.21.15.5:49774) 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f7239d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 0x56161697b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46020/0 (socket says 172.21.15.5:46020) 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.900+0000 7f723ad9d640 1 -- 172.21.15.5:0/1912209522 learned_addr learned my addr 172.21.15.5:0/1912209522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f723ad9d640 1 -- 172.21.15.5:0/1912209522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 msgr2=0x56161697b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f723ad9d640 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 0x56161697b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f723ad9d640 1 -- 172.21.15.5:0/1912209522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 msgr2=0x56161697f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f723ad9d640 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 0x56161697f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f723ad9d640 1 -- 172.21.15.5:0/1912209522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56161695f260 con 0x561616976d50 2024-09-11T22:29:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f723ad9d640 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161697bbf0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f723400e6c0 tx=0x7f7234004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9adc18196641ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f7221ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f7221ffb640 1 -- 172.21.15.5:0/1912209522 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7234033070 con 0x561616976d50 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f7221ffb640 1 -- 172.21.15.5:0/1912209522 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f7234012800 con 0x561616976d50 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f7242f80740 1 -- 172.21.15.5:0/1912209522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f723402b070 con 0x561616976d50 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f7221ffb640 1 -- 172.21.15.5:0/1912209522 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7234012a20 con 0x561616976d50 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f7242f80740 10 client.17616 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f7242f80740 1 -- 172.21.15.5:0/1912209522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5616169f2690 con 0x561616976d50 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.901+0000 7f7221ffb640 1 -- 172.21.15.5:0/1912209522 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f723402b070 con 0x561616976d50 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.902+0000 7f7221ffb640 1 -- 172.21.15.5:0/1912209522 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f723400f040 con 0x561616976d50 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.902+0000 7f7242f80740 10 client.17616 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.902+0000 7f7242f80740 10 client.17616 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.902+0000 7f7242f80740 20 client.17616 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.902+0000 7f7242f80740 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5616169f2b80 0x5616169f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.902+0000 7f7242f80740 4 client.17616 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.902+0000 7f7242f80740 1 -- 172.21.15.5:0/1912209522 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x5616168d4d20 con 0x5616169f2b80 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.902+0000 7f723a59c640 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5616169f2b80 0x5616169f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.902+0000 7f723a59c640 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5616169f2b80 0x5616169f4f70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=157a66c557f91d46 server_cookie=85454e00fcf72f2 in_seq=0 out_seq=0 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.902+0000 7f7221ffb640 10 client.17616 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.984+0000 7f7221ffb640 1 -- 172.21.15.5:0/1912209522 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5616168d4d20 con 0x5616169f2b80 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.984+0000 7f7221ffb640 10 client.17616 handle_command_reply: tid=0 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.984+0000 7f7242f80740 1 client.17616 shutdown 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 1 -- 172.21.15.5:0/1912209522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 msgr2=0x56161697bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161697bbf0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f723400e6c0 tx=0x7f7234004bd0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 1 -- 172.21.15.5:0/1912209522 shutdown_connections 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5616169f2b80 0x5616169f4f70 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561616976d50 0x56161697bbf0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7221ffb640 0 client.17616 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:27.048 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:29:27.048 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 40, 2024-09-11T22:29:27.048 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:29:27.048 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 26, 2024-09-11T22:29:27.048 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:29:27.048 INFO:teuthology.orchestra.run.smithi005.stdout: "75bd8d2f": { 2024-09-11T22:29:27.048 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 26, 2024-09-11T22:29:27.048 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 136.6, 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:27.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561616967f60 0x56161697b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 1 --2- 172.21.15.5:0/1912209522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561616967640 0x56161697f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 1 -- 172.21.15.5:0/1912209522 >> 172.21.15.5:0/1912209522 conn(0x561616963cf0 msgr2=0x561616962e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 1 -- 172.21.15.5:0/1912209522 shutdown_connections 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 1 -- 172.21.15.5:0/1912209522 wait complete. 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.985+0000 7f7242f80740 20 client.17616 trim_cache size 0 max 16384 2024-09-11T22:29:27.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 -- 172.21.15.5:0/1254381233 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72240a2cf0 msgr2=0x7f72240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72240a2cf0 0x7f72240a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7230009fd0 tx=0x7f7230005f90 comp rx=0 tx=0).stop 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 -- 172.21.15.5:0/1254381233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c102d30 msgr2=0x7f723c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c102d30 0x7f723c198670 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f722c02a940 tx=0x7f722c003b90 comp rx=0 tx=0).stop 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 -- 172.21.15.5:0/1254381233 shutdown_connections 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f723c1082a0 0x7f723c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72240a2cf0 0x7f72240a51b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f723c1036e0 0x7f723c198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 --2- 172.21.15.5:0/1254381233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f723c102d30 0x7f723c198670 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 -- 172.21.15.5:0/1254381233 >> 172.21.15.5:0/1254381233 conn(0x7f723c0fc7d0 msgr2=0x7f723c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 -- 172.21.15.5:0/1254381233 shutdown_connections 2024-09-11T22:29:27.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:26.986+0000 7f7240ba7640 1 -- 172.21.15.5:0/1254381233 wait complete. 2024-09-11T22:29:27.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.42 MB/sec execute 225 sec latency 328.579 ms 2024-09-11T22:29:27.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:27 smithi005 ceph-mon[30943]: pgmap v226: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 347 op/s 2024-09-11T22:29:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:27 smithi159 ceph-mon[35534]: pgmap v226: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 347 op/s 2024-09-11T22:29:27.831 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '75bd8d2f', quiesce took 3.8 seconds. Will release after: 12.8 2024-09-11T22:29:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:27 smithi172 ceph-mon[35461]: pgmap v226: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 347 op/s 2024-09-11T22:29:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:28.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.38 MB/sec execute 226 sec latency 1328.619 ms 2024-09-11T22:29:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 -- 172.21.15.5:0/2008655170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f552c101df0 msgr2=0x7f552c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 --2- 172.21.15.5:0/2008655170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f552c101df0 0x7f552c1021d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f551c009900 tx=0x7f551c02a410 comp rx=0 tx=0).stop 2024-09-11T22:29:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 -- 172.21.15.5:0/2008655170 shutdown_connections 2024-09-11T22:29:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 --2- 172.21.15.5:0/2008655170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f552c106f20 0x7f552c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 --2- 172.21.15.5:0/2008655170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c1027a0 0x7f552c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 --2- 172.21.15.5:0/2008655170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f552c101df0 0x7f552c1021d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 -- 172.21.15.5:0/2008655170 >> 172.21.15.5:0/2008655170 conn(0x7f552c0fd8b0 msgr2=0x7f552c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 -- 172.21.15.5:0/2008655170 shutdown_connections 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 -- 172.21.15.5:0/2008655170 wait complete. 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 Processor -- start 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.911+0000 7f5532e60640 1 -- start start 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f5532e60640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c101df0 0x7f552c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f5532e60640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f552c1027a0 0x7f552c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f5532e60640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f552c106f20 0x7f552c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f5532e60640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f552c111f80 con 0x7f552c106f20 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f5532e60640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f552c111e00 con 0x7f552c1027a0 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f5532e60640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f552c112100 con 0x7f552c101df0 2024-09-11T22:29:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f55313fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f552c106f20 0x7f552c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f55313fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f552c106f20 0x7f552c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49804/0 (socket says 172.21.15.5:49804) 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f55313fd640 1 -- 172.21.15.5:0/1229411647 learned_addr learned my addr 172.21.15.5:0/1229411647 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f5530bfc640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c101df0 0x7f552c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f552bfff640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f552c1027a0 0x7f552c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f55313fd640 1 -- 172.21.15.5:0/1229411647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c101df0 msgr2=0x7f552c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f55313fd640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c101df0 0x7f552c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f55313fd640 1 -- 172.21.15.5:0/1229411647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f552c1027a0 msgr2=0x7f552c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f55313fd640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f552c1027a0 0x7f552c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f55313fd640 1 -- 172.21.15.5:0/1229411647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c19f6a0 con 0x7f552c106f20 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f552bfff640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f552c1027a0 0x7f552c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f55313fd640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f552c106f20 0x7f552c19efc0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f552400cad0 tx=0x7f552400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.912+0000 7f5529ffb640 1 -- 172.21.15.5:0/1229411647 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552400ae20 con 0x7f552c106f20 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.913+0000 7f5529ffb640 1 -- 172.21.15.5:0/1229411647 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5524005020 con 0x7f552c106f20 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.913+0000 7f5532e60640 1 -- 172.21.15.5:0/1229411647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c19f990 con 0x7f552c106f20 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.913+0000 7f5529ffb640 1 -- 172.21.15.5:0/1229411647 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552400f6f0 con 0x7f552c106f20 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.913+0000 7f5532e60640 1 -- 172.21.15.5:0/1229411647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c1a7270 con 0x7f552c106f20 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.915+0000 7f5529ffb640 1 -- 172.21.15.5:0/1229411647 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f55240051d0 con 0x7f552c106f20 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.916+0000 7f5535239740 1 Processor -- start 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.916+0000 7f5535239740 1 -- start start 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.916+0000 7f5529ffb640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55140a2cf0 0x7f55140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.916+0000 7f5535239740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e44a0 0x559ad64e4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.916+0000 7f5535239740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 0x559ad64ef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.916+0000 7f5535239740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64ef680 0x559ad64f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.916+0000 7f5535239740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ad644a2e0 con 0x559ad64e44a0 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.916+0000 7f5535239740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559ad63f1ce0 con 0x559ad64ef680 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.916+0000 7f5535239740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559ad64462a0 con 0x559ad64e4dc0 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f552bfff640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55140a2cf0 0x7f55140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f5530bfc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e44a0 0x559ad64e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f5529ffb640 1 -- 172.21.15.5:0/1229411647 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f55240cf5d0 con 0x7f552c106f20 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f5530bfc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e44a0 0x559ad64e4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49808/0 (socket says 172.21.15.5:49808) 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f5530bfc640 1 -- 172.21.15.5:0/3544689031 learned_addr learned my addr 172.21.15.5:0/3544689031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f552bfff640 1 --2- 172.21.15.5:0/3544689031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 0x559ad64ef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f55313fd640 1 --2- 172.21.15.5:0/3544689031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64ef680 0x559ad64f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f5530bfc640 1 -- 172.21.15.5:0/3544689031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 msgr2=0x559ad64ef140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f5530bfc640 1 --2- 172.21.15.5:0/3544689031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 0x559ad64ef140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f5530bfc640 1 -- 172.21.15.5:0/3544689031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64ef680 msgr2=0x559ad64f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f5530bfc640 1 --2- 172.21.15.5:0/3544689031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64ef680 0x559ad64f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f5530bfc640 1 -- 172.21.15.5:0/3544689031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ad63d6440 con 0x559ad64e44a0 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f552bfff640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55140a2cf0 0x7f55140a51b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55200058c0 tx=0x7f5520009ad0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.917+0000 7f5530bfc640 1 --2- 172.21.15.5:0/3544689031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e44a0 0x559ad64e4880 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f551c033000 tx=0x7f551c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdae07616229c9e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5512ffd640 1 -- 172.21.15.5:0/3544689031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c03f070 con 0x559ad64e44a0 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5512ffd640 1 -- 172.21.15.5:0/3544689031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f551c0317b0 con 0x559ad64e44a0 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5512ffd640 1 -- 172.21.15.5:0/3544689031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c035070 con 0x559ad64e44a0 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5535239740 1 -- 172.21.15.5:0/3544689031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e44a0 msgr2=0x559ad64e4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5535239740 1 --2- 172.21.15.5:0/3544689031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e44a0 0x559ad64e4880 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f551c033000 tx=0x7f551c032ce0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5535239740 1 -- 172.21.15.5:0/3544689031 shutdown_connections 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5535239740 1 --2- 172.21.15.5:0/3544689031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64ef680 0x559ad64f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5535239740 1 --2- 172.21.15.5:0/3544689031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 0x559ad64ef140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5535239740 1 --2- 172.21.15.5:0/3544689031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e44a0 0x559ad64e4880 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5535239740 1 -- 172.21.15.5:0/3544689031 >> 172.21.15.5:0/3544689031 conn(0x559ad64e0a30 msgr2=0x559ad64e0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5535239740 1 -- 172.21.15.5:0/3544689031 shutdown_connections 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.918+0000 7f5535239740 1 -- 172.21.15.5:0/3544689031 wait complete. 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f5535239740 1 -- start start 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f5535239740 1 Processor -- start 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f5535239740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 0x559ad64eef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f5535239740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 0x559ad64ec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f5535239740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64ef680 0x559ad64ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f5535239740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ad64f2250 con 0x559ad64ef680 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f5535239740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559ad6405050 con 0x559ad64e44a0 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f5535239740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559ad64f23d0 con 0x559ad64e4dc0 2024-09-11T22:29:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f5535239740 10 client.0 _ll_register_callbacks cb 0x559ad6445160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f55313fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64ef680 0x559ad64ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.919+0000 7f55313fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64ef680 0x559ad64ec760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49824/0 (socket says 172.21.15.5:49824) 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f55313fd640 1 -- 172.21.15.5:0/2229724927 learned_addr learned my addr 172.21.15.5:0/2229724927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f5530bfc640 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 0x559ad64eef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f552bfff640 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 0x559ad64ec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f55313fd640 1 -- 172.21.15.5:0/2229724927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 msgr2=0x559ad64ec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f55313fd640 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 0x559ad64ec220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f55313fd640 1 -- 172.21.15.5:0/2229724927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 msgr2=0x559ad64eef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f55313fd640 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 0x559ad64eef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f55313fd640 1 -- 172.21.15.5:0/2229724927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ad61bb3c0 con 0x559ad64ef680 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f552bfff640 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 0x559ad64ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f5530bfc640 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 0x559ad64eef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f55313fd640 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64ef680 0x559ad64ec760 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f552409b9e0 tx=0x7f552401c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6df229f0b3a27bb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f550bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f550bfff640 1 -- 172.21.15.5:0/2229724927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55240a46c0 con 0x559ad64ef680 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f550bfff640 1 -- 172.21.15.5:0/2229724927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f5524012110 con 0x559ad64ef680 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f5535239740 1 -- 172.21.15.5:0/2229724927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f551c0317b0 con 0x559ad64ef680 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f550bfff640 1 -- 172.21.15.5:0/2229724927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5524011cd0 con 0x559ad64ef680 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f5535239740 10 client.17628 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.920+0000 7f5535239740 1 -- 172.21.15.5:0/2229724927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559ad64fdbc0 con 0x559ad64ef680 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.921+0000 7f550bfff640 1 -- 172.21.15.5:0/2229724927 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f551c0317b0 con 0x559ad64ef680 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.921+0000 7f550bfff640 1 -- 172.21.15.5:0/2229724927 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f552409aea0 con 0x559ad64ef680 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.921+0000 7f5535239740 10 client.17628 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.921+0000 7f5535239740 10 client.17628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.921+0000 7f5535239740 20 client.17628 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.921+0000 7f5535239740 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559ad64ff020 0x559ad6501490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.921+0000 7f5535239740 4 client.17628 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.921+0000 7f5535239740 1 -- 172.21.15.5:0/2229724927 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ad6457850 con 0x559ad64ff020 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.921+0000 7f5530bfc640 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559ad64ff020 0x559ad6501490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.922+0000 7f5530bfc640 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559ad64ff020 0x559ad6501490 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a71b59914cc8879 server_cookie=db497bbefd93f3a2 in_seq=0 out_seq=0 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.922+0000 7f550bfff640 10 client.17628 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.922+0000 7f550bfff640 1 -- 172.21.15.5:0/2229724927 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ad6457850 con 0x559ad64ff020 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.922+0000 7f550bfff640 10 client.17628 handle_command_reply: tid=0 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.922+0000 7f5535239740 1 client.17628 shutdown 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 1 -- 172.21.15.5:0/2229724927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64ef680 msgr2=0x559ad64ec760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64ef680 0x559ad64ec760 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f552409b9e0 tx=0x7f552401c2b0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 1 -- 172.21.15.5:0/2229724927 shutdown_connections 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559ad64ff020 0x559ad6501490 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64ef680 0x559ad64ec760 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e4dc0 0x559ad64ec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f550bfff640 0 client.17628 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 1 --2- 172.21.15.5:0/2229724927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 0x559ad64eef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 1 -- 172.21.15.5:0/2229724927 >> 172.21.15.5:0/2229724927 conn(0x559ad64e0a30 msgr2=0x559ad64dfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 1 -- 172.21.15.5:0/2229724927 shutdown_connections 2024-09-11T22:29:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 1 -- 172.21.15.5:0/2229724927 wait complete. 2024-09-11T22:29:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.923+0000 7f5535239740 20 client.17628 trim_cache size 0 max 16384 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.941+0000 7f5535239740 1 Processor -- start 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.941+0000 7f5535239740 1 -- start start 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.941+0000 7f5535239740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e0a30 0x559ad64ee190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.941+0000 7f5535239740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 0x559ad6506d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.941+0000 7f5535239740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e4dc0 0x559ad65072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.941+0000 7f5535239740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1afbd0 con 0x559ad64e4dc0 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.941+0000 7f5535239740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1b1710 con 0x559ad64e44a0 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.941+0000 7f5535239740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1b02a0 con 0x559ad64e0a30 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.941+0000 7f55313fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e4dc0 0x559ad65072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.941+0000 7f55313fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e4dc0 0x559ad65072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49828/0 (socket says 172.21.15.5:49828) 2024-09-11T22:29:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f55313fd640 1 -- 172.21.15.5:0/1299906160 learned_addr learned my addr 172.21.15.5:0/1299906160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f5530bfc640 1 --2- 172.21.15.5:0/1299906160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e0a30 0x559ad64ee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f552bfff640 1 --2- 172.21.15.5:0/1299906160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 0x559ad6506d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f55313fd640 1 -- 172.21.15.5:0/1299906160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e0a30 msgr2=0x559ad64ee190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f55313fd640 1 --2- 172.21.15.5:0/1299906160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e0a30 0x559ad64ee190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f55313fd640 1 -- 172.21.15.5:0/1299906160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 msgr2=0x559ad6506d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f55313fd640 1 --2- 172.21.15.5:0/1299906160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 0x559ad6506d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f55313fd640 1 -- 172.21.15.5:0/1299906160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ad64e42d0 con 0x559ad64e4dc0 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f552bfff640 1 --2- 172.21.15.5:0/1299906160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 0x559ad6506d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f55313fd640 1 --2- 172.21.15.5:0/1299906160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e4dc0 0x559ad65072a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5524097eb0 tx=0x7f552409c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edae77ebfbd03428 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f55127fc640 1 -- 172.21.15.5:0/1299906160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55240a46c0 con 0x559ad64e4dc0 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f55127fc640 1 -- 172.21.15.5:0/1299906160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f552409ba50 con 0x559ad64e4dc0 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.942+0000 7f55127fc640 1 -- 172.21.15.5:0/1299906160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552409b590 con 0x559ad64e4dc0 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 -- 172.21.15.5:0/1299906160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e4dc0 msgr2=0x559ad65072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 --2- 172.21.15.5:0/1299906160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e4dc0 0x559ad65072a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f5524097eb0 tx=0x7f552409c150 comp rx=0 tx=0).stop 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 -- 172.21.15.5:0/1299906160 shutdown_connections 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 --2- 172.21.15.5:0/1299906160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e4dc0 0x559ad65072a0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 --2- 172.21.15.5:0/1299906160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e44a0 0x559ad6506d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 --2- 172.21.15.5:0/1299906160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e0a30 0x559ad64ee190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 -- 172.21.15.5:0/1299906160 >> 172.21.15.5:0/1299906160 conn(0x559ad64ec8c0 msgr2=0x559ad64eda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 -- 172.21.15.5:0/1299906160 shutdown_connections 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 -- 172.21.15.5:0/1299906160 wait complete. 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 -- start start 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.943+0000 7f5535239740 1 Processor -- start 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5535239740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e0a30 0x559ad64f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5535239740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e44a0 0x559ad64f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5535239740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e4dc0 0x559ad64f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5535239740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1b1040 con 0x559ad64e0a30 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5535239740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1b0970 con 0x559ad64e4dc0 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5535239740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1adbb0 con 0x559ad64e44a0 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5535239740 10 client.0 _ll_register_callbacks cb 0x7f552409b3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5530bfc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e0a30 0x559ad64f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5530bfc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e0a30 0x559ad64f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49832/0 (socket says 172.21.15.5:49832) 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5530bfc640 1 -- 172.21.15.5:0/2631728316 learned_addr learned my addr 172.21.15.5:0/2631728316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f55313fd640 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e4dc0 0x559ad64f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f552bfff640 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e44a0 0x559ad64f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5530bfc640 1 -- 172.21.15.5:0/2631728316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e44a0 msgr2=0x559ad64f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5530bfc640 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e44a0 0x559ad64f50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5530bfc640 1 -- 172.21.15.5:0/2631728316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e4dc0 msgr2=0x559ad64f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5530bfc640 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e4dc0 0x559ad64f55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f5530bfc640 1 -- 172.21.15.5:0/2631728316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ad64f5dd0 con 0x559ad64e0a30 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f55313fd640 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e4dc0 0x559ad64f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.944+0000 7f552bfff640 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e44a0 0x559ad64f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.945+0000 7f5530bfc640 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e0a30 0x559ad64f7d10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f551c02a450 tx=0x7f551c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a392cc17682e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.945+0000 7f550bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.945+0000 7f550bfff640 1 -- 172.21.15.5:0/2631728316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c03f040 con 0x559ad64e0a30 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.945+0000 7f550bfff640 1 -- 172.21.15.5:0/2631728316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f551c0053e0 con 0x559ad64e0a30 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.945+0000 7f5535239740 1 -- 172.21.15.5:0/2631728316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f552409ba50 con 0x559ad64e0a30 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.945+0000 7f550bfff640 1 -- 172.21.15.5:0/2631728316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f551c035040 con 0x559ad64e0a30 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.945+0000 7f5535239740 10 client.17634 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.945+0000 7f5535239740 1 -- 172.21.15.5:0/2631728316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559ad6569c70 con 0x559ad64e0a30 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.945+0000 7f550bfff640 1 -- 172.21.15.5:0/2631728316 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f552409ba50 con 0x559ad64e0a30 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.945+0000 7f550bfff640 1 -- 172.21.15.5:0/2631728316 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f551c003c90 con 0x559ad64e0a30 2024-09-11T22:29:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.946+0000 7f5535239740 10 client.17634 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.946+0000 7f5535239740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.946+0000 7f5535239740 20 client.17634 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.946+0000 7f5535239740 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559ad64ef680 0x559ad656bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.946+0000 7f5535239740 4 client.17634 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.946+0000 7f5535239740 1 -- 172.21.15.5:0/2631728316 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ad6451d60 con 0x559ad64ef680 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.946+0000 7f552bfff640 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559ad64ef680 0x559ad656bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.946+0000 7f552bfff640 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559ad64ef680 0x559ad656bfd0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4662e6da74b9d975 server_cookie=df3f611ef7086572 in_seq=0 out_seq=0 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.946+0000 7f550bfff640 10 client.17634 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f550bfff640 1 -- 172.21.15.5:0/2631728316 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559ad6451d60 con 0x559ad64ef680 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f550bfff640 10 client.17634 handle_command_reply: tid=0 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 client.17634 shutdown 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 -- 172.21.15.5:0/2631728316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e0a30 msgr2=0x559ad64f7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e0a30 0x559ad64f7d10 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f551c02a450 tx=0x7f551c005a20 comp rx=0 tx=0).stop 2024-09-11T22:29:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 -- 172.21.15.5:0/2631728316 shutdown_connections 2024-09-11T22:29:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559ad64ef680 0x559ad656bfd0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ad64e4dc0 0x559ad64f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ad64e44a0 0x559ad64f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 --2- 172.21.15.5:0/2631728316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ad64e0a30 0x559ad64f7d10 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 -- 172.21.15.5:0/2631728316 >> 172.21.15.5:0/2631728316 conn(0x559ad64ec8c0 msgr2=0x559ad64df350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f550bfff640 0 client.17634 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 -- 172.21.15.5:0/2631728316 shutdown_connections 2024-09-11T22:29:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 1 -- 172.21.15.5:0/2631728316 wait complete. 2024-09-11T22:29:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.947+0000 7f5535239740 20 client.17634 trim_cache size 0 max 16384 2024-09-11T22:29:28.955 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:29:28.956 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:29:28.956 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:29:28.956 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:29:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 -- 172.21.15.5:0/1229411647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55140a2cf0 msgr2=0x7f55140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55140a2cf0 0x7f55140a51b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f55200058c0 tx=0x7f5520009ad0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 -- 172.21.15.5:0/1229411647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f552c106f20 msgr2=0x7f552c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f552c106f20 0x7f552c19efc0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f552400cad0 tx=0x7f552400cf90 comp rx=0 tx=0).stop 2024-09-11T22:29:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 -- 172.21.15.5:0/1229411647 shutdown_connections 2024-09-11T22:29:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f552c106f20 0x7f552c19efc0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55140a2cf0 0x7f55140a51b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f552c1027a0 0x7f552c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 --2- 172.21.15.5:0/1229411647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f552c101df0 0x7f552c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 -- 172.21.15.5:0/1229411647 >> 172.21.15.5:0/1229411647 conn(0x7f552c0fd8b0 msgr2=0x7f552c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 -- 172.21.15.5:0/1229411647 shutdown_connections 2024-09-11T22:29:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:28.948+0000 7f5532e60640 1 -- 172.21.15.5:0/1229411647 wait complete. 2024-09-11T22:29:29.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.34 MB/sec execute 227 sec latency 2328.656 ms 2024-09-11T22:29:29.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:29 smithi005 ceph-mon[30943]: pgmap v227: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 378 op/s 2024-09-11T22:29:29.786 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:29:29.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:29:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:29 smithi159 ceph-mon[35534]: pgmap v227: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 378 op/s 2024-09-11T22:29:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:29 smithi172 ceph-mon[35461]: pgmap v227: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 378 op/s 2024-09-11T22:29:30.045 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:30.327 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.30 MB/sec execute 228 sec latency 3328.739 ms 2024-09-11T22:29:30.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 -- 172.21.15.5:0/80157812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa678106a80 msgr2=0x7fa678106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 --2- 172.21.15.5:0/80157812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa678106a80 0x7fa678106e60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fa668009930 tx=0x7fa66802a420 comp rx=0 tx=0).stop 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 -- 172.21.15.5:0/80157812 shutdown_connections 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 --2- 172.21.15.5:0/80157812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678101420 0x7fa67810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 --2- 172.21.15.5:0/80157812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa678100a80 0x7fa678100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 --2- 172.21.15.5:0/80157812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa678106a80 0x7fa678106e60 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 -- 172.21.15.5:0/80157812 >> 172.21.15.5:0/80157812 conn(0x7fa6780fc7a0 msgr2=0x7fa6780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 -- 172.21.15.5:0/80157812 shutdown_connections 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 -- 172.21.15.5:0/80157812 wait complete. 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 Processor -- start 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.611+0000 7fa67da4e640 1 -- start start 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa67da4e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa678100a80 0x7fa6780fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa67da4e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa678101420 0x7fa6780fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:30.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa67da4e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678106a80 0x7fa6780fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa67da4e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa678110e80 con 0x7fa678101420 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa67da4e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa678110d00 con 0x7fa678106a80 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa67da4e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa678111000 con 0x7fa678100a80 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa66ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa678101420 0x7fa6780fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa66ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa678101420 0x7fa6780fd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49858/0 (socket says 172.21.15.5:49858) 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa66ffff640 1 -- 172.21.15.5:0/3831614178 learned_addr learned my addr 172.21.15.5:0/3831614178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa676ffd640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa678100a80 0x7fa6780fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa6777fe640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678106a80 0x7fa6780fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa66ffff640 1 -- 172.21.15.5:0/3831614178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa678100a80 msgr2=0x7fa6780fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa66ffff640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa678100a80 0x7fa6780fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa66ffff640 1 -- 172.21.15.5:0/3831614178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678106a80 msgr2=0x7fa6780fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa66ffff640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678106a80 0x7fa6780fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:30.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.612+0000 7fa66ffff640 1 -- 172.21.15.5:0/3831614178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6781a69c0 con 0x7fa678101420 2024-09-11T22:29:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.613+0000 7fa66ffff640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa678101420 0x7fa6780fd330 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa66000c970 tx=0x7fa66000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.613+0000 7fa67ca4c640 1 -- 172.21.15.5:0/3831614178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660007bb0 con 0x7fa678101420 2024-09-11T22:29:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.613+0000 7fa67ca4c640 1 -- 172.21.15.5:0/3831614178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa660007d50 con 0x7fa678101420 2024-09-11T22:29:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.613+0000 7fa67da4e640 1 -- 172.21.15.5:0/3831614178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6781a6cb0 con 0x7fa678101420 2024-09-11T22:29:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.613+0000 7fa67ca4c640 1 -- 172.21.15.5:0/3831614178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66000f450 con 0x7fa678101420 2024-09-11T22:29:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.613+0000 7fa67da4e640 1 -- 172.21.15.5:0/3831614178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6781a71a0 con 0x7fa678101420 2024-09-11T22:29:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.614+0000 7fa67da4e640 1 -- 172.21.15.5:0/3831614178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa678100ee0 con 0x7fa678101420 2024-09-11T22:29:30.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.616+0000 7fa67ca4c640 1 -- 172.21.15.5:0/3831614178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa660005020 con 0x7fa678101420 2024-09-11T22:29:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.616+0000 7fa67ca4c640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6640a2d70 0x7fa6640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.617+0000 7fa676ffd640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6640a2d70 0x7fa6640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.617+0000 7fa67ca4c640 1 -- 172.21.15.5:0/3831614178 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fa6600cf150 con 0x7fa678101420 2024-09-11T22:29:30.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.617+0000 7fa676ffd640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6640a2d70 0x7fa6640a5230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa668009900 tx=0x7fa6680058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:30.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.621+0000 7fa67ca4c640 1 -- 172.21.15.5:0/3831614178 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa660014030 con 0x7fa678101420 2024-09-11T22:29:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.887+0000 7fa67da4e640 1 -- 172.21.15.5:0/3831614178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa67810a000 con 0x7fa678101420 2024-09-11T22:29:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.888+0000 7fa67ca4c640 1 -- 172.21.15.5:0/3831614178 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11967 (secure 0 0 0) 0x7fa66009b6e0 con 0x7fa678101420 2024-09-11T22:29:30.889 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:29:30.889 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"btime":"2024-09-11T22:29:21:436884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:29:21.436856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26914":{"gid":26914,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1755675305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1755675305},{"type":"v1","addr":"172.21.15.159:6833","nonce":1755675305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:29:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 40 2024-09-11T22:29:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 -- 172.21.15.5:0/3831614178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6640a2d70 msgr2=0x7fa6640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6640a2d70 0x7fa6640a5230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa668009900 tx=0x7fa6680058d0 comp rx=0 tx=0).stop 2024-09-11T22:29:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 -- 172.21.15.5:0/3831614178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa678101420 msgr2=0x7fa6780fd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa678101420 0x7fa6780fd330 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa66000c970 tx=0x7fa66000ce30 comp rx=0 tx=0).stop 2024-09-11T22:29:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 -- 172.21.15.5:0/3831614178 shutdown_connections 2024-09-11T22:29:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6640a2d70 0x7fa6640a5230 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa678106a80 0x7fa6780fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa678101420 0x7fa6780fd330 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 --2- 172.21.15.5:0/3831614178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa678100a80 0x7fa6780fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 -- 172.21.15.5:0/3831614178 >> 172.21.15.5:0/3831614178 conn(0x7fa6780fc7a0 msgr2=0x7fa67806aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 -- 172.21.15.5:0/3831614178 shutdown_connections 2024-09-11T22:29:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:30.890+0000 7fa67da4e640 1 -- 172.21.15.5:0/3831614178 wait complete. 2024-09-11T22:29:31.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.27 MB/sec execute 229 sec latency 4328.781 ms 2024-09-11T22:29:31.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:31 smithi005 ceph-mon[30943]: pgmap v228: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 378 op/s 2024-09-11T22:29:31.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:31 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:29:31.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:31 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3831614178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:29:31.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:31 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:29:31.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:31 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:29:31.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:31 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:29:31.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:29:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:31 smithi159 ceph-mon[35534]: pgmap v228: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 378 op/s 2024-09-11T22:29:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:31 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:29:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:31 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3831614178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:29:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:31 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:29:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:31 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:29:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:31 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:29:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:31 smithi172 ceph-mon[35461]: pgmap v228: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 378 op/s 2024-09-11T22:29:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:31 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:29:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:31 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3831614178' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:29:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:31 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:29:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:31 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:29:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:31 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:29:31.898 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:32.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.23 MB/sec execute 230 sec latency 5328.841 ms 2024-09-11T22:29:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.422+0000 7f3846535640 1 -- 172.21.15.5:0/4226848914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3840101de0 msgr2=0x7f38401021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.422+0000 7f3846535640 1 --2- 172.21.15.5:0/4226848914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3840101de0 0x7f38401021c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3830009900 tx=0x7f383002a410 comp rx=0 tx=0).stop 2024-09-11T22:29:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.422+0000 7f3846535640 1 -- 172.21.15.5:0/4226848914 shutdown_connections 2024-09-11T22:29:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.422+0000 7f3846535640 1 --2- 172.21.15.5:0/4226848914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3840106f10 0x7f384010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.422+0000 7f3846535640 1 --2- 172.21.15.5:0/4226848914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3840102790 0x7f38401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.422+0000 7f3846535640 1 --2- 172.21.15.5:0/4226848914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3840101de0 0x7f38401021c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.422+0000 7f3846535640 1 -- 172.21.15.5:0/4226848914 >> 172.21.15.5:0/4226848914 conn(0x7f38400fd8c0 msgr2=0x7f38400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.423+0000 7f3846535640 1 -- 172.21.15.5:0/4226848914 shutdown_connections 2024-09-11T22:29:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.423+0000 7f3846535640 1 -- 172.21.15.5:0/4226848914 wait complete. 2024-09-11T22:29:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.423+0000 7f3846535640 1 Processor -- start 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.423+0000 7f3846535640 1 -- start start 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f3846535640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3840101de0 0x7f3840198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f3846535640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3840102790 0x7f3840198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f3846535640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3840106f10 0x7f384019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f3846535640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3840111f70 con 0x7f3840101de0 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f3846535640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3840111df0 con 0x7f3840106f10 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f3846535640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38401120f0 con 0x7f3840102790 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f383ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3840101de0 0x7f3840198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f383ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3840101de0 0x7f3840198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58966/0 (socket says 172.21.15.5:58966) 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f383ffff640 1 -- 172.21.15.5:0/570973665 learned_addr learned my addr 172.21.15.5:0/570973665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f383f7fe640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3840102790 0x7f3840198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f3844ad2640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3840106f10 0x7f384019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f383ffff640 1 -- 172.21.15.5:0/570973665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3840102790 msgr2=0x7f3840198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f383ffff640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3840102790 0x7f3840198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f383ffff640 1 -- 172.21.15.5:0/570973665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3840106f10 msgr2=0x7f384019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f383ffff640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3840106f10 0x7f384019cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f383ffff640 1 -- 172.21.15.5:0/570973665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f384019d400 con 0x7f3840101de0 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.424+0000 7f383ffff640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3840101de0 0x7f3840198450 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3830006010 tx=0x7f38300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.425+0000 7f383d7fa640 1 -- 172.21.15.5:0/570973665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383003f030 con 0x7f3840101de0 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.425+0000 7f3846535640 1 -- 172.21.15.5:0/570973665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f384019d690 con 0x7f3840101de0 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.425+0000 7f383d7fa640 1 -- 172.21.15.5:0/570973665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3830003ce0 con 0x7f3840101de0 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.425+0000 7f383d7fa640 1 -- 172.21.15.5:0/570973665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830035070 con 0x7f3840101de0 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.425+0000 7f3846535640 1 -- 172.21.15.5:0/570973665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38401a4fd0 con 0x7f3840101de0 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.427+0000 7f383d7fa640 1 -- 172.21.15.5:0/570973665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3830042030 con 0x7f3840101de0 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.427+0000 7f384890e740 1 Processor -- start 2024-09-11T22:29:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f384890e740 1 -- start start 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f384890e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 0x55f0f4305090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f384890e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f4305a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f384890e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4310370 0x55f0f4312760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f384890e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0f426b2f0 con 0x55f0f43055d0 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f384890e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f0f4212cf0 con 0x55f0f4306b40 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f384890e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f0f42672b0 con 0x55f0f4310370 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f4305a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f3844ad2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 0x55f0f4305090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f4305a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58978/0 (socket says 172.21.15.5:58978) 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f3844ad2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 0x55f0f4305090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34238/0 (socket says 172.21.15.5:34238) 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383ffff640 1 -- 172.21.15.5:0/217947708 learned_addr learned my addr 172.21.15.5:0/217947708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383d7fa640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38280a2cf0 0x7f38280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383f7fe640 1 --2- 172.21.15.5:0/217947708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4310370 0x55f0f4312760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f3844ad2640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38280a2cf0 0x7f38280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383ffff640 1 -- 172.21.15.5:0/217947708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4310370 msgr2=0x55f0f4312760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383ffff640 1 --2- 172.21.15.5:0/217947708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4310370 0x55f0f4312760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383ffff640 1 -- 172.21.15.5:0/217947708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 msgr2=0x55f0f4305090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383ffff640 1 --2- 172.21.15.5:0/217947708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 0x55f0f4305090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383d7fa640 1 -- 172.21.15.5:0/570973665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f38300efb50 con 0x7f3840101de0 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.428+0000 7f383ffff640 1 -- 172.21.15.5:0/217947708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0f41f73a0 con 0x55f0f43055d0 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.429+0000 7f383ffff640 1 --2- 172.21.15.5:0/217947708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f4305a10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f38300bbdf0 tx=0x7f383002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3799654311294ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.429+0000 7f3844ad2640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38280a2cf0 0x7f38280a51b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3838005fd0 tx=0x7f3838004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.429+0000 7f38267fc640 1 -- 172.21.15.5:0/217947708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830049070 con 0x55f0f43055d0 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.429+0000 7f38267fc640 1 -- 172.21.15.5:0/217947708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f383003b580 con 0x55f0f43055d0 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.429+0000 7f38267fc640 1 -- 172.21.15.5:0/217947708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300bc0f0 con 0x55f0f43055d0 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 -- 172.21.15.5:0/217947708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 msgr2=0x55f0f4305a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 --2- 172.21.15.5:0/217947708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f4305a10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f38300bbdf0 tx=0x7f383002a410 comp rx=0 tx=0).stop 2024-09-11T22:29:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 -- 172.21.15.5:0/217947708 shutdown_connections 2024-09-11T22:29:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 --2- 172.21.15.5:0/217947708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4310370 0x55f0f4312760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 --2- 172.21.15.5:0/217947708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f4305a10 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 --2- 172.21.15.5:0/217947708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 0x55f0f4305090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 -- 172.21.15.5:0/217947708 >> 172.21.15.5:0/217947708 conn(0x55f0f42ce310 msgr2=0x55f0f4301c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 -- 172.21.15.5:0/217947708 shutdown_connections 2024-09-11T22:29:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 -- 172.21.15.5:0/217947708 wait complete. 2024-09-11T22:29:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 -- start start 2024-09-11T22:29:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 Processor -- start 2024-09-11T22:29:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.430+0000 7f384890e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f43055d0 0x55f0f430fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f384890e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4306b40 0x55f0f430d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f384890e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4310370 0x55f0f430d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f384890e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0f4312e30 con 0x55f0f4306b40 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f384890e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f0f4226060 con 0x55f0f4310370 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f384890e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f0f4312fb0 con 0x55f0f43055d0 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f384890e740 10 client.0 _ll_register_callbacks cb 0x55f0f3f798c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f383f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4306b40 0x55f0f430d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f383f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4306b40 0x55f0f430d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58988/0 (socket says 172.21.15.5:58988) 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f383f7fe640 1 -- 172.21.15.5:0/3776794312 learned_addr learned my addr 172.21.15.5:0/3776794312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f383ffff640 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f43055d0 0x55f0f430fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f3844ad2640 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4310370 0x55f0f430d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f383f7fe640 1 -- 172.21.15.5:0/3776794312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f43055d0 msgr2=0x55f0f430fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f383f7fe640 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f43055d0 0x55f0f430fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f383f7fe640 1 -- 172.21.15.5:0/3776794312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4310370 msgr2=0x55f0f430d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f383f7fe640 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4310370 0x55f0f430d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f383f7fe640 1 -- 172.21.15.5:0/3776794312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0f3fdc3c0 con 0x55f0f4306b40 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f3844ad2640 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4310370 0x55f0f430d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f383f7fe640 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4306b40 0x55f0f430d050 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f383400b9f0 tx=0x7f383400beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa93ba74e7565b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f381f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f381f7fe640 1 -- 172.21.15.5:0/3776794312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f383401b640 con 0x55f0f4306b40 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f381f7fe640 1 -- 172.21.15.5:0/3776794312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f383401b7e0 con 0x55f0f4306b40 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.431+0000 7f384890e740 1 -- 172.21.15.5:0/3776794312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f0f4278860 con 0x55f0f4306b40 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f381f7fe640 1 -- 172.21.15.5:0/3776794312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f383401b9e0 con 0x55f0f4306b40 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f384890e740 10 client.17652 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f384890e740 1 -- 172.21.15.5:0/3776794312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f0f431e860 con 0x55f0f4306b40 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f381f7fe640 1 -- 172.21.15.5:0/3776794312 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f0f4278860 con 0x55f0f4306b40 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f381f7fe640 1 -- 172.21.15.5:0/3776794312 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f3834019020 con 0x55f0f4306b40 2024-09-11T22:29:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f384890e740 10 client.17652 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f384890e740 10 client.17652 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f384890e740 20 client.17652 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f384890e740 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f0f431fd70 0x55f0f43221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f384890e740 4 client.17652 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f384890e740 1 -- 172.21.15.5:0/3776794312 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0f4272d70 con 0x55f0f431fd70 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.432+0000 7f383ffff640 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f0f431fd70 0x55f0f43221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.433+0000 7f383ffff640 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f0f431fd70 0x55f0f43221b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3eaf5af8da3e7a93 server_cookie=8af31daa92772cdb in_seq=0 out_seq=0 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.433+0000 7f381f7fe640 10 client.17652 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.433+0000 7f381f7fe640 1 -- 172.21.15.5:0/3776794312 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0f4272d70 con 0x55f0f431fd70 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.433+0000 7f381f7fe640 10 client.17652 handle_command_reply: tid=0 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.433+0000 7f384890e740 1 client.17652 shutdown 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 1 -- 172.21.15.5:0/3776794312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4306b40 msgr2=0x55f0f430d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4306b40 0x55f0f430d050 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f383400b9f0 tx=0x7f383400beb0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 1 -- 172.21.15.5:0/3776794312 shutdown_connections 2024-09-11T22:29:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f0f431fd70 0x55f0f43221b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4310370 0x55f0f430d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4306b40 0x55f0f430d050 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 1 --2- 172.21.15.5:0/3776794312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f43055d0 0x55f0f430fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f381f7fe640 0 client.17652 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:29:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 1 -- 172.21.15.5:0/3776794312 >> 172.21.15.5:0/3776794312 conn(0x55f0f42ce310 msgr2=0x55f0f4307990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 1 -- 172.21.15.5:0/3776794312 shutdown_connections 2024-09-11T22:29:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 1 -- 172.21.15.5:0/3776794312 wait complete. 2024-09-11T22:29:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.434+0000 7f384890e740 20 client.17652 trim_cache size 0 max 16384 2024-09-11T22:29:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f384890e740 1 Processor -- start 2024-09-11T22:29:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f384890e740 1 -- start start 2024-09-11T22:29:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f384890e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f430c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f384890e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 0x55f0f4327a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f384890e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4310370 0x55f0f4327fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f384890e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38401ada90 con 0x55f0f43055d0 2024-09-11T22:29:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f384890e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38401af5d0 con 0x55f0f4306b40 2024-09-11T22:29:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f384890e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38401ae160 con 0x55f0f4310370 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f383ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f430c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f383ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f430c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58996/0 (socket says 172.21.15.5:58996) 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f383ffff640 1 -- 172.21.15.5:0/868988579 learned_addr learned my addr 172.21.15.5:0/868988579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f383f7fe640 1 --2- 172.21.15.5:0/868988579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 0x55f0f4327a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.447+0000 7f3844ad2640 1 --2- 172.21.15.5:0/868988579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4310370 0x55f0f4327fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f383ffff640 1 -- 172.21.15.5:0/868988579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4310370 msgr2=0x55f0f4327fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f383ffff640 1 --2- 172.21.15.5:0/868988579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4310370 0x55f0f4327fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f383ffff640 1 -- 172.21.15.5:0/868988579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 msgr2=0x55f0f4327a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f383ffff640 1 --2- 172.21.15.5:0/868988579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 0x55f0f4327a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f383ffff640 1 -- 172.21.15.5:0/868988579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0f430b5a0 con 0x55f0f43055d0 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f383ffff640 1 --2- 172.21.15.5:0/868988579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f430c030 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f38300bc080 tx=0x7f38300bc960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940dae8ecc4a05bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f3825ffb640 1 -- 172.21.15.5:0/868988579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3830049040 con 0x55f0f43055d0 2024-09-11T22:29:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f3825ffb640 1 -- 172.21.15.5:0/868988579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38300c04a0 con 0x55f0f43055d0 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f3825ffb640 1 -- 172.21.15.5:0/868988579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38300c0160 con 0x55f0f43055d0 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f384890e740 1 -- 172.21.15.5:0/868988579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 msgr2=0x55f0f430c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.448+0000 7f384890e740 1 --2- 172.21.15.5:0/868988579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f430c030 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f38300bc080 tx=0x7f38300bc960 comp rx=0 tx=0).stop 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.449+0000 7f384890e740 1 -- 172.21.15.5:0/868988579 shutdown_connections 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.449+0000 7f384890e740 1 --2- 172.21.15.5:0/868988579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4310370 0x55f0f4327fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.449+0000 7f384890e740 1 --2- 172.21.15.5:0/868988579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f4306b40 0x55f0f4327a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.449+0000 7f384890e740 1 --2- 172.21.15.5:0/868988579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f43055d0 0x55f0f430c030 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.449+0000 7f384890e740 1 -- 172.21.15.5:0/868988579 >> 172.21.15.5:0/868988579 conn(0x55f0f42ce310 msgr2=0x55f0f430b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.449+0000 7f384890e740 1 -- 172.21.15.5:0/868988579 shutdown_connections 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.449+0000 7f384890e740 1 -- 172.21.15.5:0/868988579 wait complete. 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.449+0000 7f384890e740 1 -- start start 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.449+0000 7f384890e740 1 Processor -- start 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f384890e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f43055d0 0x55f0f4318700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f384890e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4306b40 0x55f0f4313a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f384890e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4310370 0x55f0f4313f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f384890e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38401aef00 con 0x55f0f4310370 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f384890e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38401ae830 con 0x55f0f43055d0 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f384890e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38401aba70 con 0x55f0f4306b40 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f384890e740 10 client.0 _ll_register_callbacks cb 0x7f3834004140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f3844ad2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4310370 0x55f0f4313f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f3844ad2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4310370 0x55f0f4313f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59008/0 (socket says 172.21.15.5:59008) 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f3844ad2640 1 -- 172.21.15.5:0/899046583 learned_addr learned my addr 172.21.15.5:0/899046583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f383ffff640 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f43055d0 0x55f0f4318700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f383f7fe640 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4306b40 0x55f0f4313a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f3844ad2640 1 -- 172.21.15.5:0/899046583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4306b40 msgr2=0x55f0f4313a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f3844ad2640 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4306b40 0x55f0f4313a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f3844ad2640 1 -- 172.21.15.5:0/899046583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f43055d0 msgr2=0x55f0f4318700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f3844ad2640 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f43055d0 0x55f0f4318700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f3844ad2640 1 -- 172.21.15.5:0/899046583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0f43147a0 con 0x55f0f4310370 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f3844ad2640 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4310370 0x55f0f4313f70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f3838003bb0 tx=0x7f3838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e016e17280243 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f381f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.450+0000 7f381f7fe640 1 -- 172.21.15.5:0/899046583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838019040 con 0x55f0f4310370 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f384890e740 1 -- 172.21.15.5:0/899046583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38300c04a0 con 0x55f0f4310370 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f381f7fe640 1 -- 172.21.15.5:0/899046583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f3838013050 con 0x55f0f4310370 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f381f7fe640 1 -- 172.21.15.5:0/899046583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3838013250 con 0x55f0f4310370 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f384890e740 10 client.17658 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f384890e740 1 -- 172.21.15.5:0/899046583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f0f438aa30 con 0x55f0f4310370 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f381f7fe640 1 -- 172.21.15.5:0/899046583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38300c04a0 con 0x55f0f4310370 2024-09-11T22:29:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f381f7fe640 1 -- 172.21.15.5:0/899046583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f3838021670 con 0x55f0f4310370 2024-09-11T22:29:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f384890e740 10 client.17658 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f384890e740 10 client.17658 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:29:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f384890e740 20 client.17658 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f384890e740 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f0f438b150 0x55f0f438d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f384890e740 4 client.17658 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:29:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.451+0000 7f384890e740 1 -- 172.21.15.5:0/899046583 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0f41ef170 con 0x55f0f438b150 2024-09-11T22:29:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f383ffff640 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f0f438b150 0x55f0f438d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f383ffff640 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f0f438b150 0x55f0f438d540 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f330f52859c03c7 server_cookie=8064c9f3b07c4a21 in_seq=0 out_seq=0 2024-09-11T22:29:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f381f7fe640 10 client.17658 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:29:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f381f7fe640 1 -- 172.21.15.5:0/899046583 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0f41ef170 con 0x55f0f438b150 2024-09-11T22:29:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f381f7fe640 10 client.17658 handle_command_reply: tid=0 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f384890e740 1 client.17658 shutdown 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f384890e740 1 -- 172.21.15.5:0/899046583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4310370 msgr2=0x55f0f4313f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f384890e740 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4310370 0x55f0f4313f70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f3838003bb0 tx=0x7f3838004a40 comp rx=0 tx=0).stop 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f384890e740 1 -- 172.21.15.5:0/899046583 shutdown_connections 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f384890e740 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55f0f438b150 0x55f0f438d540 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f384890e740 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0f4310370 0x55f0f4313f70 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f384890e740 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f0f4306b40 0x55f0f4313a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f384890e740 1 --2- 172.21.15.5:0/899046583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0f43055d0 0x55f0f4318700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f384890e740 1 -- 172.21.15.5:0/899046583 >> 172.21.15.5:0/899046583 conn(0x55f0f42ce310 msgr2=0x55f0f430c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.452+0000 7f381f7fe640 0 client.17658 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f384890e740 1 -- 172.21.15.5:0/899046583 shutdown_connections 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f384890e740 1 -- 172.21.15.5:0/899046583 wait complete. 2024-09-11T22:29:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f384890e740 20 client.17658 trim_cache size 0 max 16384 2024-09-11T22:29:32.460 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:29:32.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 -- 172.21.15.5:0/570973665 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38280a2cf0 msgr2=0x7f38280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38280a2cf0 0x7f38280a51b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3838005fd0 tx=0x7f3838004ab0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 -- 172.21.15.5:0/570973665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3840101de0 msgr2=0x7f3840198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3840101de0 0x7f3840198450 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3830006010 tx=0x7f38300042f0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 -- 172.21.15.5:0/570973665 shutdown_connections 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38280a2cf0 0x7f38280a51b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3840106f10 0x7f384019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3840102790 0x7f3840198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 --2- 172.21.15.5:0/570973665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3840101de0 0x7f3840198450 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 -- 172.21.15.5:0/570973665 >> 172.21.15.5:0/570973665 conn(0x7f38400fd8c0 msgr2=0x7f38400fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 -- 172.21.15.5:0/570973665 shutdown_connections 2024-09-11T22:29:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:32.453+0000 7f3846535640 1 -- 172.21.15.5:0/570973665 wait complete. 2024-09-11T22:29:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.145+0000 7f6be1c1e640 1 -- 172.21.15.5:0/873479427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc107b70 msgr2=0x7f6bdc107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.145+0000 7f6be1c1e640 1 --2- 172.21.15.5:0/873479427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc107b70 0x7f6bdc107f50 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009930 tx=0x7f6bcc02a420 comp rx=0 tx=0).stop 2024-09-11T22:29:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.145+0000 7f6be1c1e640 1 -- 172.21.15.5:0/873479427 shutdown_connections 2024-09-11T22:29:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.145+0000 7f6be1c1e640 1 --2- 172.21.15.5:0/873479427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bdc102470 0x7f6bdc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.145+0000 7f6be1c1e640 1 --2- 172.21.15.5:0/873479427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bdc101ad0 0x7f6bdc101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.145+0000 7f6be1c1e640 1 --2- 172.21.15.5:0/873479427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc107b70 0x7f6bdc107f50 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.145+0000 7f6be1c1e640 1 -- 172.21.15.5:0/873479427 >> 172.21.15.5:0/873479427 conn(0x7f6bdc0fd7d0 msgr2=0x7f6bdc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.145+0000 7f6be1c1e640 1 -- 172.21.15.5:0/873479427 shutdown_connections 2024-09-11T22:29:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.145+0000 7f6be1c1e640 1 -- 172.21.15.5:0/873479427 wait complete. 2024-09-11T22:29:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.145+0000 7f6be1c1e640 1 Processor -- start 2024-09-11T22:29:33.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6be1c1e640 1 -- start start 2024-09-11T22:29:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6be1c1e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc101ad0 0x7f6bdc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6be1c1e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bdc102470 0x7f6bdc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6be1c1e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bdc107b70 0x7f6bdc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6be1c1e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc111f70 con 0x7f6bdc101ad0 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6be1c1e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc111df0 con 0x7f6bdc102470 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6be1c1e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1120f0 con 0x7f6bdc107b70 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc101ad0 0x7f6bdc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc101ad0 0x7f6bdc198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59026/0 (socket says 172.21.15.5:59026) 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1148878358 learned_addr learned my addr 172.21.15.5:0/1148878358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6bdaffd640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bdc102470 0x7f6bdc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1148878358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bdc107b70 msgr2=0x7f6bdc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bdc107b70 0x7f6bdc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1148878358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bdc102470 msgr2=0x7f6bdc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bdc102470 0x7f6bdc198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1148878358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bdc19d3f0 con 0x7f6bdc101ad0 2024-09-11T22:29:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.146+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc101ad0 0x7f6bdc198400 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc02a940 tx=0x7f6bcc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.147+0000 7f6bd8ff9640 1 -- 172.21.15.5:0/1148878358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc03f070 con 0x7f6bdc101ad0 2024-09-11T22:29:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.147+0000 7f6bd8ff9640 1 -- 172.21.15.5:0/1148878358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bcc031c90 con 0x7f6bdc101ad0 2024-09-11T22:29:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.147+0000 7f6bd8ff9640 1 -- 172.21.15.5:0/1148878358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc035030 con 0x7f6bdc101ad0 2024-09-11T22:29:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.147+0000 7f6be1c1e640 1 -- 172.21.15.5:0/1148878358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bdc19d680 con 0x7f6bdc101ad0 2024-09-11T22:29:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.147+0000 7f6be1c1e640 1 -- 172.21.15.5:0/1148878358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bdc1a4f70 con 0x7f6bdc101ad0 2024-09-11T22:29:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.149+0000 7f6bd8ff9640 1 -- 172.21.15.5:0/1148878358 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6bcc042070 con 0x7f6bdc101ad0 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.149+0000 7f6be4a8d740 1 Processor -- start 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.149+0000 7f6be4a8d740 1 -- start start 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756bd5af0 0x564756bd5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756be0cb0 0x564756be30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564756b3a2f0 con 0x564756bd6410 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564756ae2270 con 0x564756be0cb0 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564756b362b0 con 0x564756bd5af0 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59028/0 (socket says 172.21.15.5:59028) 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/3011103709 learned_addr learned my addr 172.21.15.5:0/3011103709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/3011103709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756bd5af0 0x564756bd5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdaffd640 1 --2- 172.21.15.5:0/3011103709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756be0cb0 0x564756be30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bd8ff9640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6bc40a2e20 0x7f6bc40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/3011103709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756bd5af0 msgr2=0x564756bd5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/3011103709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756bd5af0 0x564756bd5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/3011103709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756be0cb0 msgr2=0x564756be30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/3011103709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756be0cb0 0x564756be30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/3011103709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564756ae1340 con 0x564756bd6410 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6bc40a2e20 0x7f6bc40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bd8ff9640 1 -- 172.21.15.5:0/1148878358 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6bcc0efcc0 con 0x7f6bdc101ad0 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.150+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/3011103709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be0770 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0ed2b0 tx=0x7f6bcc0b83e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39ae13b2c091179 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6bc1ffb640 1 -- 172.21.15.5:0/3011103709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc049030 con 0x564756bd6410 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6bc1ffb640 1 -- 172.21.15.5:0/3011103709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bcc0cb3c0 con 0x564756bd6410 2024-09-11T22:29:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6bc1ffb640 1 -- 172.21.15.5:0/3011103709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0bbf10 con 0x564756bd6410 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6be4a8d740 1 -- 172.21.15.5:0/3011103709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 msgr2=0x564756be0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6bc40a2e20 0x7f6bc40a52e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400ade0 tx=0x7f6bd4004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/3011103709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be0770 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0ed2b0 tx=0x7f6bcc0b83e0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6be4a8d740 1 -- 172.21.15.5:0/3011103709 shutdown_connections 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/3011103709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756be0cb0 0x564756be30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/3011103709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be0770 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/3011103709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756bd5af0 0x564756bd5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6be4a8d740 1 -- 172.21.15.5:0/3011103709 >> 172.21.15.5:0/3011103709 conn(0x564756bd2060 msgr2=0x564756bd2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6be4a8d740 1 -- 172.21.15.5:0/3011103709 shutdown_connections 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.151+0000 7f6be4a8d740 1 -- 172.21.15.5:0/3011103709 wait complete. 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6be4a8d740 1 -- start start 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6be4a8d740 1 Processor -- start 2024-09-11T22:29:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd5af0 0x564756bddb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756bd6410 0x564756bde060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756be0cb0 0x564756bde5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564756be3640 con 0x564756be0cb0 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564756ac3290 con 0x564756bd5af0 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564756be37c0 con 0x564756bd6410 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6be4a8d740 10 client.0 _ll_register_callbacks cb 0x564756af4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6bdbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756be0cb0 0x564756bde5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6bdbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756be0cb0 0x564756bde5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59032/0 (socket says 172.21.15.5:59032) 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6bdbfff640 1 -- 172.21.15.5:0/794391421 learned_addr learned my addr 172.21.15.5:0/794391421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd5af0 0x564756bddb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.152+0000 7f6bdaffd640 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756bd6410 0x564756bde060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bdbfff640 1 -- 172.21.15.5:0/794391421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756bd6410 msgr2=0x564756bde060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756bd6410 0x564756bde060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bdbfff640 1 -- 172.21.15.5:0/794391421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd5af0 msgr2=0x564756bddb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd5af0 0x564756bddb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bdbfff640 1 -- 172.21.15.5:0/794391421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56475684d210 con 0x564756be0cb0 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756be0cb0 0x564756bde5a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4012a30 tx=0x7f6bd400a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae260b3fced6371 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bbaffd640 1 -- 172.21.15.5:0/794391421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd401dd00 con 0x564756be0cb0 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bbaffd640 1 -- 172.21.15.5:0/794391421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6bd401b450 con 0x564756be0cb0 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6be4a8d740 1 -- 172.21.15.5:0/794391421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564756b47a20 con 0x564756be0cb0 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bbaffd640 1 -- 172.21.15.5:0/794391421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6bd401b650 con 0x564756be0cb0 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6be4a8d740 10 client.17670 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6be4a8d740 1 -- 172.21.15.5:0/794391421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564756919620 con 0x564756be0cb0 2024-09-11T22:29:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bbaffd640 1 -- 172.21.15.5:0/794391421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564756b47a20 con 0x564756be0cb0 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6bbaffd640 1 -- 172.21.15.5:0/794391421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f6bd4024020 con 0x564756be0cb0 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6be4a8d740 10 client.17670 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.153+0000 7f6be4a8d740 10 client.17670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.154+0000 7f6be4a8d740 20 client.17670 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.154+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564756bf0920 0x564756bf2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.154+0000 7f6be4a8d740 4 client.17670 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.154+0000 7f6be4a8d740 1 -- 172.21.15.5:0/794391421 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564756b41d70 con 0x564756bf0920 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.154+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564756bf0920 0x564756bf2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.154+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564756bf0920 0x564756bf2d50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86df23f12be70c8d server_cookie=6b696a717df28fbb in_seq=0 out_seq=0 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.154+0000 7f6bbaffd640 10 client.17670 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6bbaffd640 1 -- 172.21.15.5:0/794391421 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564756b41d70 con 0x564756bf0920 2024-09-11T22:29:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6bbaffd640 10 client.17670 handle_command_reply: tid=0 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 client.17670 shutdown 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 -- 172.21.15.5:0/794391421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756be0cb0 msgr2=0x564756bde5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756be0cb0 0x564756bde5a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4012a30 tx=0x7f6bd400a490 comp rx=0 tx=0).stop 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 -- 172.21.15.5:0/794391421 shutdown_connections 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564756bf0920 0x564756bf2d50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756be0cb0 0x564756bde5a0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756bd6410 0x564756bde060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6bbaffd640 0 client.17670 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/794391421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd5af0 0x564756bddb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 -- 172.21.15.5:0/794391421 >> 172.21.15.5:0/794391421 conn(0x564756bd2060 msgr2=0x564756bd7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 -- 172.21.15.5:0/794391421 shutdown_connections 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 1 -- 172.21.15.5:0/794391421 wait complete. 2024-09-11T22:29:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.155+0000 7f6be4a8d740 20 client.17670 trim_cache size 0 max 16384 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6be4a8d740 1 Processor -- start 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6be4a8d740 1 -- start start 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd5af0 0x564756bde3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd6410 0x564756bdfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 0x564756bf8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1ada40 con 0x564756bd5af0 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1af580 con 0x564756bd6410 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1ae110 con 0x564756be0cb0 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd5af0 0x564756bde3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd5af0 0x564756bde3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59040/0 (socket says 172.21.15.5:59040) 2024-09-11T22:29:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1746392517 learned_addr learned my addr 172.21.15.5:0/1746392517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/1746392517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 0x564756bf8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1746392517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 msgr2=0x564756bf8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/1746392517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 0x564756bf8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1746392517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd6410 msgr2=0x564756bdfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/1746392517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd6410 0x564756bdfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdb7fe640 1 -- 172.21.15.5:0/1746392517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564756bd1e90 con 0x564756bd5af0 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/1746392517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 0x564756bf8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/1746392517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd5af0 0x564756bde3d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b85a0 tx=0x7f6bcc0ef040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963554d935d838fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bc17fa640 1 -- 172.21.15.5:0/1746392517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc049040 con 0x564756bd5af0 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bc17fa640 1 -- 172.21.15.5:0/1746392517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bcc031e40 con 0x564756bd5af0 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.170+0000 7f6bc17fa640 1 -- 172.21.15.5:0/1746392517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0cafc0 con 0x564756bd5af0 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.171+0000 7f6be4a8d740 1 -- 172.21.15.5:0/1746392517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd5af0 msgr2=0x564756bde3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.171+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/1746392517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd5af0 0x564756bde3d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc0b85a0 tx=0x7f6bcc0ef040 comp rx=0 tx=0).stop 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.171+0000 7f6be4a8d740 1 -- 172.21.15.5:0/1746392517 shutdown_connections 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.171+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/1746392517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 0x564756bf8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.171+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/1746392517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd6410 0x564756bdfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.171+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/1746392517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd5af0 0x564756bde3d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.171+0000 7f6be4a8d740 1 -- 172.21.15.5:0/1746392517 >> 172.21.15.5:0/1746392517 conn(0x564756bd2060 msgr2=0x564756bdf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.171+0000 7f6be4a8d740 1 -- 172.21.15.5:0/1746392517 shutdown_connections 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.171+0000 7f6be4a8d740 1 -- 172.21.15.5:0/1746392517 wait complete. 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.171+0000 7f6be4a8d740 1 -- start start 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6be4a8d740 1 Processor -- start 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd5af0 0x564756be3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6be4a8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 0x564756c5da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1aeeb0 con 0x564756bd6410 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1ae7e0 con 0x564756bd5af0 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6be4a8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6bdc1aba20 con 0x564756be0cb0 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6be4a8d740 10 client.0 _ll_register_callbacks cb 0x7f6bd400aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59054/0 (socket says 172.21.15.5:59054) 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdaffd640 1 -- 172.21.15.5:0/2571703554 learned_addr learned my addr 172.21.15.5:0/2571703554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 0x564756c5da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd5af0 0x564756be3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdaffd640 1 -- 172.21.15.5:0/2571703554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 msgr2=0x564756c5da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdaffd640 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 0x564756c5da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdaffd640 1 -- 172.21.15.5:0/2571703554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd5af0 msgr2=0x564756be3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdaffd640 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd5af0 0x564756be3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdaffd640 1 -- 172.21.15.5:0/2571703554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564756c5dfc0 con 0x564756bd6410 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdbfff640 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 0x564756c5da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bdaffd640 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be43d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0ffb40 tx=0x7f6bd00123f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51884ba65059b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.172+0000 7f6bbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6bbaffd640 1 -- 172.21.15.5:0/2571703554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0005020 con 0x564756bd6410 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6bbaffd640 1 -- 172.21.15.5:0/2571703554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6bd00051c0 con 0x564756bd6410 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6be4a8d740 1 -- 172.21.15.5:0/2571703554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564756abe900 con 0x564756bd6410 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6bbaffd640 1 -- 172.21.15.5:0/2571703554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6bd00053c0 con 0x564756bd6410 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6be4a8d740 10 client.17676 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6be4a8d740 1 -- 172.21.15.5:0/2571703554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564756c5f2a0 con 0x564756bd6410 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6bbaffd640 1 -- 172.21.15.5:0/2571703554 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564756abe900 con 0x564756bd6410 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6bbaffd640 1 -- 172.21.15.5:0/2571703554 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f6bd0019030 con 0x564756bd6410 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6be4a8d740 10 client.17676 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6be4a8d740 10 client.17676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6be4a8d740 20 client.17676 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564756c5fa40 0x564756c61e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6be4a8d740 4 client.17676 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:29:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6be4a8d740 1 -- 172.21.15.5:0/2571703554 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564756b0b820 con 0x564756c5fa40 2024-09-11T22:29:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.173+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564756c5fa40 0x564756c61e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.174+0000 7f6bbaffd640 10 client.17676 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.174+0000 7f6bdb7fe640 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564756c5fa40 0x564756c61e60 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63b3f60e68eff88f server_cookie=9f65decf347b961 in_seq=0 out_seq=0 2024-09-11T22:29:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.262+0000 7f6bbaffd640 1 -- 172.21.15.5:0/2571703554 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564756b0b820 con 0x564756c5fa40 2024-09-11T22:29:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.262+0000 7f6bbaffd640 10 client.17676 handle_command_reply: tid=0 2024-09-11T22:29:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.262+0000 7f6be4a8d740 1 client.17676 shutdown 2024-09-11T22:29:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 1 -- 172.21.15.5:0/2571703554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 msgr2=0x564756be43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be43d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc0ffb40 tx=0x7f6bd00123f0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 1 -- 172.21.15.5:0/2571703554 shutdown_connections 2024-09-11T22:29:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564756c5fa40 0x564756c61e60 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564756be0cb0 0x564756c5da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6bbaffd640 0 client.17676 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564756bd6410 0x564756be43d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 1 --2- 172.21.15.5:0/2571703554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564756bd5af0 0x564756be3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 1 -- 172.21.15.5:0/2571703554 >> 172.21.15.5:0/2571703554 conn(0x564756bd2060 msgr2=0x564756bd11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 1 -- 172.21.15.5:0/2571703554 shutdown_connections 2024-09-11T22:29:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 1 -- 172.21.15.5:0/2571703554 wait complete. 2024-09-11T22:29:33.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.263+0000 7f6be4a8d740 20 client.17676 trim_cache size 0 max 16384 2024-09-11T22:29:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:29:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:29:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:29:33.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 -- 172.21.15.5:0/1148878358 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6bc40a2e20 msgr2=0x7f6bc40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6bc40a2e20 0x7f6bc40a52e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400ade0 tx=0x7f6bd4004a40 comp rx=0 tx=0).stop 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 -- 172.21.15.5:0/1148878358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc101ad0 msgr2=0x7f6bdc198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc101ad0 0x7f6bdc198400 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc02a940 tx=0x7f6bcc003b90 comp rx=0 tx=0).stop 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 -- 172.21.15.5:0/1148878358 shutdown_connections 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bdc107b70 0x7f6bdc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6bc40a2e20 0x7f6bc40a52e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bdc102470 0x7f6bdc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 --2- 172.21.15.5:0/1148878358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bdc101ad0 0x7f6bdc198400 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 -- 172.21.15.5:0/1148878358 >> 172.21.15.5:0/1148878358 conn(0x7f6bdc0fd7d0 msgr2=0x7f6bdc0fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 -- 172.21.15.5:0/1148878358 shutdown_connections 2024-09-11T22:29:33.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:33.264+0000 7f6be1c1e640 1 -- 172.21.15.5:0/1148878358 wait complete. 2024-09-11T22:29:33.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.20 MB/sec execute 231 sec latency 6328.925 ms 2024-09-11T22:29:33.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:29:33.695 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:33.817 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:33 smithi005 ceph-mon[30943]: pgmap v229: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 379 op/s 2024-09-11T22:29:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:33 smithi159 ceph-mon[35534]: pgmap v229: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 379 op/s 2024-09-11T22:29:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:33 smithi172 ceph-mon[35461]: pgmap v229: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 379 op/s 2024-09-11T22:29:34.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.200+0000 7fb51f9ee640 1 -- 172.21.15.5:0/223069802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5181082a0 msgr2=0x7fb51810ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.200+0000 7fb51f9ee640 1 --2- 172.21.15.5:0/223069802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5181082a0 0x7fb51810ec10 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb510009cd0 tx=0x7fb51002a910 comp rx=0 tx=0).stop 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.200+0000 7fb51f9ee640 1 -- 172.21.15.5:0/223069802 shutdown_connections 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.200+0000 7fb51f9ee640 1 --2- 172.21.15.5:0/223069802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5181082a0 0x7fb51810ec10 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.200+0000 7fb51f9ee640 1 --2- 172.21.15.5:0/223069802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb518103720 0x7fb518107b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.200+0000 7fb51f9ee640 1 --2- 172.21.15.5:0/223069802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb518102d70 0x7fb518103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.200+0000 7fb51f9ee640 1 -- 172.21.15.5:0/223069802 >> 172.21.15.5:0/223069802 conn(0x7fb5180fc7d0 msgr2=0x7fb5180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.200+0000 7fb51f9ee640 1 -- 172.21.15.5:0/223069802 shutdown_connections 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.200+0000 7fb51f9ee640 1 -- 172.21.15.5:0/223069802 wait complete. 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.200+0000 7fb51f9ee640 1 Processor -- start 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51f9ee640 1 -- start start 2024-09-11T22:29:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51f9ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb518102d70 0x7fb5181985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51f9ee640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb518103720 0x7fb518198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51f9ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5181082a0 0x7fb51819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51f9ee640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb518112120 con 0x7fb518102d70 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51f9ee640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb518111fa0 con 0x7fb518103720 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51f9ee640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181122a0 con 0x7fb5181082a0 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51d78a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb518102d70 0x7fb5181985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51d78a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb518102d70 0x7fb5181985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59084/0 (socket says 172.21.15.5:59084) 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51d78a640 1 -- 172.21.15.5:0/1743207764 learned_addr learned my addr 172.21.15.5:0/1743207764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51d78a640 1 -- 172.21.15.5:0/1743207764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5181082a0 msgr2=0x7fb51819cea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51d78a640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5181082a0 0x7fb51819cea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51d78a640 1 -- 172.21.15.5:0/1743207764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb518103720 msgr2=0x7fb518198b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:29:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51d78a640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb518103720 0x7fb518198b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51d78a640 1 -- 172.21.15.5:0/1743207764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51819d580 con 0x7fb518102d70 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.201+0000 7fb51d78a640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb518102d70 0x7fb5181985d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb51400b450 tx=0x7fb51400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.202+0000 7fb50e7fc640 1 -- 172.21.15.5:0/1743207764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514015020 con 0x7fb518102d70 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.202+0000 7fb51f9ee640 1 -- 172.21.15.5:0/1743207764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51819d870 con 0x7fb518102d70 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.202+0000 7fb50e7fc640 1 -- 172.21.15.5:0/1743207764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb514011d70 con 0x7fb518102d70 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.202+0000 7fb50e7fc640 1 -- 172.21.15.5:0/1743207764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51400f340 con 0x7fb518102d70 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.202+0000 7fb51f9ee640 1 -- 172.21.15.5:0/1743207764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181a5150 con 0x7fb518102d70 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.204+0000 7fb50e7fc640 1 -- 172.21.15.5:0/1743207764 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb514018020 con 0x7fb518102d70 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb521dc7740 1 Processor -- start 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb521dc7740 1 -- start start 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a6892090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a6892a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 0x55b8a689f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb521dc7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b8a67f82f0 con 0x55b8a6893b40 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb521dc7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b8a679fcf0 con 0x55b8a689d370 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb521dc7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b8a67f42b0 con 0x55b8a68925d0 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb50e7fc640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb5040a2d70 0x7fb5040a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb51df8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a6892090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb51df8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a6892090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59086/0 (socket says 172.21.15.5:59086) 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb51df8b640 1 -- 172.21.15.5:0/3128490965 learned_addr learned my addr 172.21.15.5:0/3128490965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb51d78a640 1 --2- 172.21.15.5:0/3128490965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a6892a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.205+0000 7fb51df8b640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb5040a2d70 0x7fb5040a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb50e7fc640 1 -- 172.21.15.5:0/1743207764 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fb5140ceeb0 con 0x7fb518102d70 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb51cf89640 1 --2- 172.21.15.5:0/3128490965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 0x55b8a689f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb51df8b640 1 -- 172.21.15.5:0/3128490965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 msgr2=0x55b8a6892a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb51df8b640 1 --2- 172.21.15.5:0/3128490965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a6892a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb51df8b640 1 -- 172.21.15.5:0/3128490965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 msgr2=0x55b8a689f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb51df8b640 1 --2- 172.21.15.5:0/3128490965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 0x55b8a689f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb51df8b640 1 -- 172.21.15.5:0/3128490965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a67843a0 con 0x55b8a6893b40 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb51df8b640 1 --2- 172.21.15.5:0/3128490965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a6892090 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb510005e30 tx=0x7fb510032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfc5da05adff618 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb51df8b640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb5040a2d70 0x7fb5040a5230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb51002ae30 tx=0x7fb510047000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb5037fe640 1 -- 172.21.15.5:0/3128490965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51004e070 con 0x55b8a6893b40 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb5037fe640 1 -- 172.21.15.5:0/3128490965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb510036040 con 0x55b8a6893b40 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.206+0000 7fb5037fe640 1 -- 172.21.15.5:0/3128490965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510030a30 con 0x55b8a6893b40 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.207+0000 7fb521dc7740 1 -- 172.21.15.5:0/3128490965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 msgr2=0x55b8a6892090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.207+0000 7fb521dc7740 1 --2- 172.21.15.5:0/3128490965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a6892090 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb510005e30 tx=0x7fb510032ed0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.207+0000 7fb521dc7740 1 -- 172.21.15.5:0/3128490965 shutdown_connections 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.207+0000 7fb521dc7740 1 --2- 172.21.15.5:0/3128490965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 0x55b8a689f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.207+0000 7fb521dc7740 1 --2- 172.21.15.5:0/3128490965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a6892a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.207+0000 7fb521dc7740 1 --2- 172.21.15.5:0/3128490965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a6892090 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.207+0000 7fb521dc7740 1 -- 172.21.15.5:0/3128490965 >> 172.21.15.5:0/3128490965 conn(0x55b8a685b310 msgr2=0x55b8a688ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.207+0000 7fb521dc7740 1 -- 172.21.15.5:0/3128490965 shutdown_connections 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.207+0000 7fb521dc7740 1 -- 172.21.15.5:0/3128490965 wait complete. 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.207+0000 7fb521dc7740 1 -- start start 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb521dc7740 1 Processor -- start 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a68925d0 0x55b8a689cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 0x55b8a689a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a689d370 0x55b8a689a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb521dc7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b8a68a0010 con 0x55b8a68925d0 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb521dc7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b8a67b3060 con 0x55b8a6893b40 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb521dc7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b8a68a0190 con 0x55b8a689d370 2024-09-11T22:29:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb521dc7740 10 client.0 _ll_register_callbacks cb 0x55b8a65068c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb51d78a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a68925d0 0x55b8a689cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb51d78a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a68925d0 0x55b8a689cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59096/0 (socket says 172.21.15.5:59096) 2024-09-11T22:29:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb51d78a640 1 -- 172.21.15.5:0/1081022375 learned_addr learned my addr 172.21.15.5:0/1081022375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:34.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb51df8b640 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a689d370 0x55b8a689a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.242 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:29:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb51cf89640 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 0x55b8a689a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb51d78a640 1 -- 172.21.15.5:0/1081022375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a689d370 msgr2=0x55b8a689a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb51d78a640 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a689d370 0x55b8a689a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb51d78a640 1 -- 172.21.15.5:0/1081022375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 msgr2=0x55b8a689a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb51d78a640 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 0x55b8a689a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.208+0000 7fb51d78a640 1 -- 172.21.15.5:0/1081022375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a65693c0 con 0x55b8a68925d0 2024-09-11T22:29:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb51d78a640 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a68925d0 0x55b8a689cd80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb51400be10 tx=0x7fb514020f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8105437dfd114c4f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb500ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb500ff9640 1 -- 172.21.15.5:0/1081022375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5140b4060 con 0x55b8a68925d0 2024-09-11T22:29:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb500ff9640 1 -- 172.21.15.5:0/1081022375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fb51401ebf0 con 0x55b8a68925d0 2024-09-11T22:29:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb521dc7740 1 -- 172.21.15.5:0/1081022375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb510036040 con 0x55b8a68925d0 2024-09-11T22:29:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb500ff9640 1 -- 172.21.15.5:0/1081022375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb51409bba0 con 0x55b8a68925d0 2024-09-11T22:29:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb521dc7740 10 client.17688 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb521dc7740 1 -- 172.21.15.5:0/1081022375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b8a68abb40 con 0x55b8a68925d0 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb500ff9640 1 -- 172.21.15.5:0/1081022375 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb510036040 con 0x55b8a68925d0 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb500ff9640 1 -- 172.21.15.5:0/1081022375 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fb5140b4200 con 0x55b8a68925d0 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb521dc7740 10 client.17688 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb521dc7740 10 client.17688 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb521dc7740 20 client.17688 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb521dc7740 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b8a68ad000 0x55b8a68af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb521dc7740 4 client.17688 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.209+0000 7fb521dc7740 1 -- 172.21.15.5:0/1081022375 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8a6805860 con 0x55b8a68ad000 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.210+0000 7fb51cf89640 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b8a68ad000 0x55b8a68af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.210+0000 7fb500ff9640 10 client.17688 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.210+0000 7fb51cf89640 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b8a68ad000 0x55b8a68af410 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d334133cb2ee50c server_cookie=6a005a5cd2592936 in_seq=0 out_seq=0 2024-09-11T22:29:34.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb500ff9640 1 -- 172.21.15.5:0/1081022375 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8a6805860 con 0x55b8a68ad000 2024-09-11T22:29:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb500ff9640 10 client.17688 handle_command_reply: tid=0 2024-09-11T22:29:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 client.17688 shutdown 2024-09-11T22:29:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 -- 172.21.15.5:0/1081022375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a68925d0 msgr2=0x55b8a689cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a68925d0 0x55b8a689cd80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb51400be10 tx=0x7fb514020f10 comp rx=0 tx=0).stop 2024-09-11T22:29:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 -- 172.21.15.5:0/1081022375 shutdown_connections 2024-09-11T22:29:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a689d370 0x55b8a689a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b8a68ad000 0x55b8a68af410 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 0x55b8a689a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 --2- 172.21.15.5:0/1081022375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a68925d0 0x55b8a689cd80 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb500ff9640 0 client.17688 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 -- 172.21.15.5:0/1081022375 >> 172.21.15.5:0/1081022375 conn(0x55b8a685b310 msgr2=0x55b8a6894960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 -- 172.21.15.5:0/1081022375 shutdown_connections 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 1 -- 172.21.15.5:0/1081022375 wait complete. 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.211+0000 7fb521dc7740 20 client.17688 trim_cache size 0 max 16384 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.224+0000 7fb521dc7740 1 Processor -- start 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.224+0000 7fb521dc7740 1 -- start start 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.224+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a6895120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.224+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 0x55b8a6895660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.224+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a689d370 0x55b8a68b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.224+0000 7fb521dc7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181adba0 con 0x55b8a689d370 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.224+0000 7fb521dc7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181af6e0 con 0x55b8a6893b40 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.224+0000 7fb521dc7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181ae270 con 0x55b8a68925d0 2024-09-11T22:29:34.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.224+0000 7fb51df8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a689d370 0x55b8a68b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.224+0000 7fb51df8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a689d370 0x55b8a68b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59106/0 (socket says 172.21.15.5:59106) 2024-09-11T22:29:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb51df8b640 1 -- 172.21.15.5:0/2245147044 learned_addr learned my addr 172.21.15.5:0/2245147044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb51cf89640 1 --2- 172.21.15.5:0/2245147044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 0x55b8a6895660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb51d78a640 1 --2- 172.21.15.5:0/2245147044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a6895120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb51df8b640 1 -- 172.21.15.5:0/2245147044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 msgr2=0x55b8a6895120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb51df8b640 1 --2- 172.21.15.5:0/2245147044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a6895120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb51df8b640 1 -- 172.21.15.5:0/2245147044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 msgr2=0x55b8a6895660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb51df8b640 1 --2- 172.21.15.5:0/2245147044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 0x55b8a6895660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb51df8b640 1 -- 172.21.15.5:0/2245147044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8a689a9b0 con 0x55b8a689d370 2024-09-11T22:29:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb51cf89640 1 --2- 172.21.15.5:0/2245147044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 0x55b8a6895660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb51df8b640 1 --2- 172.21.15.5:0/2245147044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a689d370 0x55b8a68b4ce0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb510030660 tx=0x7fb510031f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2366f173e989bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb502ffd640 1 -- 172.21.15.5:0/2245147044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510059070 con 0x55b8a689d370 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb502ffd640 1 -- 172.21.15.5:0/2245147044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb510036040 con 0x55b8a689d370 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.225+0000 7fb502ffd640 1 -- 172.21.15.5:0/2245147044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51004e040 con 0x55b8a689d370 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 -- 172.21.15.5:0/2245147044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a689d370 msgr2=0x55b8a68b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 --2- 172.21.15.5:0/2245147044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a689d370 0x55b8a68b4ce0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb510030660 tx=0x7fb510031f20 comp rx=0 tx=0).stop 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 -- 172.21.15.5:0/2245147044 shutdown_connections 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 --2- 172.21.15.5:0/2245147044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a689d370 0x55b8a68b4ce0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 --2- 172.21.15.5:0/2245147044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a6893b40 0x55b8a6895660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 --2- 172.21.15.5:0/2245147044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a6895120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 -- 172.21.15.5:0/2245147044 >> 172.21.15.5:0/2245147044 conn(0x55b8a685b310 msgr2=0x55b8a688dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:34.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 -- 172.21.15.5:0/2245147044 shutdown_connections 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 -- 172.21.15.5:0/2245147044 wait complete. 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 -- start start 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 Processor -- start 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.226+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a68a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a68a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb521dc7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 0x55b8a68a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb521dc7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181af010 con 0x55b8a6893b40 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb521dc7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181ae940 con 0x55b8a689d370 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb521dc7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb5181abb80 con 0x55b8a68925d0 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb521dc7740 10 client.0 _ll_register_callbacks cb 0x7fb5140092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51cf89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a68a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51cf89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a68a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59118/0 (socket says 172.21.15.5:59118) 2024-09-11T22:29:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51cf89640 1 -- 172.21.15.5:0/526537013 learned_addr learned my addr 172.21.15.5:0/526537013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51d78a640 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a68a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51df8b640 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 0x55b8a68a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51cf89640 1 -- 172.21.15.5:0/526537013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 msgr2=0x55b8a68a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51cf89640 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a68a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51cf89640 1 -- 172.21.15.5:0/526537013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 msgr2=0x55b8a68a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51cf89640 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 0x55b8a68a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51cf89640 1 -- 172.21.15.5:0/526537013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8a68a2930 con 0x55b8a6893b40 2024-09-11T22:29:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51df8b640 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 0x55b8a68a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:34.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb51cf89640 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a68a5960 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb508005d80 tx=0x7fb508012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcb5b9cd8ab999c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb500ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb500ff9640 1 -- 172.21.15.5:0/526537013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508004460 con 0x55b8a6893b40 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb500ff9640 1 -- 172.21.15.5:0/526537013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fb508005020 con 0x55b8a6893b40 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb521dc7740 1 -- 172.21.15.5:0/526537013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb510036040 con 0x55b8a6893b40 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb500ff9640 1 -- 172.21.15.5:0/526537013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fb508029050 con 0x55b8a6893b40 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.227+0000 7fb521dc7740 10 client.17694 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.228+0000 7fb521dc7740 1 -- 172.21.15.5:0/526537013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b8a69184b0 con 0x55b8a6893b40 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.228+0000 7fb500ff9640 1 -- 172.21.15.5:0/526537013 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb510036040 con 0x55b8a6893b40 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.228+0000 7fb500ff9640 1 -- 172.21.15.5:0/526537013 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fb5080035f0 con 0x55b8a6893b40 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.228+0000 7fb521dc7740 10 client.17694 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.228+0000 7fb521dc7740 10 client.17694 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:29:34.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.228+0000 7fb521dc7740 20 client.17694 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.228+0000 7fb521dc7740 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b8a6918cb0 0x55b8a691b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.228+0000 7fb521dc7740 4 client.17694 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.228+0000 7fb521dc7740 1 -- 172.21.15.5:0/526537013 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8a67ffd70 con 0x55b8a6918cb0 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.228+0000 7fb51d78a640 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b8a6918cb0 0x55b8a691b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb500ff9640 10 client.17694 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb51d78a640 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b8a6918cb0 0x55b8a691b0d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cdb18bc44fdb57c8 server_cookie=e105cded04118676 in_seq=0 out_seq=0 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb500ff9640 1 -- 172.21.15.5:0/526537013 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8a67ffd70 con 0x55b8a6918cb0 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb500ff9640 10 client.17694 handle_command_reply: tid=0 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 client.17694 shutdown 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 -- 172.21.15.5:0/526537013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 msgr2=0x55b8a68a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a68a5960 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fb508005d80 tx=0x7fb508012820 comp rx=0 tx=0).stop 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 -- 172.21.15.5:0/526537013 shutdown_connections 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55b8a6918cb0 0x55b8a691b0d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b8a689d370 0x55b8a68a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8a6893b40 0x55b8a68a5960 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb500ff9640 0 client.17694 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 --2- 172.21.15.5:0/526537013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b8a68925d0 0x55b8a68a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 -- 172.21.15.5:0/526537013 >> 172.21.15.5:0/526537013 conn(0x55b8a685b310 msgr2=0x55b8a688d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 -- 172.21.15.5:0/526537013 shutdown_connections 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 1 -- 172.21.15.5:0/526537013 wait complete. 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.229+0000 7fb521dc7740 20 client.17694 trim_cache size 0 max 16384 2024-09-11T22:29:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 -- 172.21.15.5:0/1743207764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb5040a2d70 msgr2=0x7fb5040a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb5040a2d70 0x7fb5040a5230 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb51002ae30 tx=0x7fb510047000 comp rx=0 tx=0).stop 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 -- 172.21.15.5:0/1743207764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb518102d70 msgr2=0x7fb5181985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb518102d70 0x7fb5181985d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb51400b450 tx=0x7fb51400b910 comp rx=0 tx=0).stop 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 -- 172.21.15.5:0/1743207764 shutdown_connections 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5181082a0 0x7fb51819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb5040a2d70 0x7fb5040a5230 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb518103720 0x7fb518198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 --2- 172.21.15.5:0/1743207764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb518102d70 0x7fb5181985d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 -- 172.21.15.5:0/1743207764 >> 172.21.15.5:0/1743207764 conn(0x7fb5180fc7d0 msgr2=0x7fb518106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 -- 172.21.15.5:0/1743207764 shutdown_connections 2024-09-11T22:29:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:34.230+0000 7fb51f9ee640 1 -- 172.21.15.5:0/1743207764 wait complete. 2024-09-11T22:29:34.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.16 MB/sec execute 232 sec latency 7329.021 ms 2024-09-11T22:29:35.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:29:35.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.13 MB/sec execute 233 sec latency 8329.063 ms 2024-09-11T22:29:35.377 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:35.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:35 smithi005 ceph-mon[30943]: pgmap v230: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 127 op/s 2024-09-11T22:29:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:35 smithi159 ceph-mon[35534]: pgmap v230: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 127 op/s 2024-09-11T22:29:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:35 smithi172 ceph-mon[35461]: pgmap v230: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 127 op/s 2024-09-11T22:29:36.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.016+0000 7f16364bb640 1 -- 172.21.15.5:0/3849774681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1630102790 msgr2=0x7f16301067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.016+0000 7f16364bb640 1 --2- 172.21.15.5:0/3849774681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1630102790 0x7f16301067e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f1618009900 tx=0x7f161802a410 comp rx=0 tx=0).stop 2024-09-11T22:29:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.016+0000 7f16364bb640 1 -- 172.21.15.5:0/3849774681 shutdown_connections 2024-09-11T22:29:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.016+0000 7f16364bb640 1 --2- 172.21.15.5:0/3849774681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630106f10 0x7f163010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.016+0000 7f16364bb640 1 --2- 172.21.15.5:0/3849774681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1630102790 0x7f16301067e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.016+0000 7f16364bb640 1 --2- 172.21.15.5:0/3849774681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630101de0 0x7f16301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.016+0000 7f16364bb640 1 -- 172.21.15.5:0/3849774681 >> 172.21.15.5:0/3849774681 conn(0x7f16300fd8c0 msgr2=0x7f16300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.016+0000 7f16364bb640 1 -- 172.21.15.5:0/3849774681 shutdown_connections 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.016+0000 7f16364bb640 1 -- 172.21.15.5:0/3849774681 wait complete. 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f16364bb640 1 Processor -- start 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f16364bb640 1 -- start start 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f16364bb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630101de0 0x7f1630198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f16364bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1630102790 0x7f16301989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f16364bb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630106f10 0x7f163019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f16364bb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1630111f70 con 0x7f1630102790 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f16364bb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1630111df0 con 0x7f1630106f10 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f16364bb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16301120f0 con 0x7f1630101de0 2024-09-11T22:29:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f1627fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1630102790 0x7f16301989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f1627fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1630102790 0x7f16301989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59142/0 (socket says 172.21.15.5:59142) 2024-09-11T22:29:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f1627fff640 1 -- 172.21.15.5:0/3425763453 learned_addr learned my addr 172.21.15.5:0/3425763453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f1634a58640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630106f10 0x7f163019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f162ffff640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630101de0 0x7f1630198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f1627fff640 1 -- 172.21.15.5:0/3425763453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630101de0 msgr2=0x7f1630198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f1627fff640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630101de0 0x7f1630198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f1627fff640 1 -- 172.21.15.5:0/3425763453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630106f10 msgr2=0x7f163019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f1627fff640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630106f10 0x7f163019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f1627fff640 1 -- 172.21.15.5:0/3425763453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163019d440 con 0x7f1630102790 2024-09-11T22:29:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.017+0000 7f162ffff640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630101de0 0x7f1630198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.018+0000 7f1627fff640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1630102790 0x7f16301989d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1618006010 tx=0x7f16180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.018+0000 7f162dffb640 1 -- 172.21.15.5:0/3425763453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161803f030 con 0x7f1630102790 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.018+0000 7f162dffb640 1 -- 172.21.15.5:0/3425763453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1618003ce0 con 0x7f1630102790 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.018+0000 7f16364bb640 1 -- 172.21.15.5:0/3425763453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163019d6d0 con 0x7f1630102790 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.018+0000 7f162dffb640 1 -- 172.21.15.5:0/3425763453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618035070 con 0x7f1630102790 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.018+0000 7f16364bb640 1 -- 172.21.15.5:0/3425763453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16301a5010 con 0x7f1630102790 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.020+0000 7f162dffb640 1 -- 172.21.15.5:0/3425763453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1618042030 con 0x7f1630102790 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.020+0000 7f1638894740 1 Processor -- start 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1638894740 1 -- start start 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1638894740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68cab40 0x55d8b68c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1638894740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68c95d0 0x55d8b68c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1638894740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1638894740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d8b682f2f0 con 0x55d8b68d4370 2024-09-11T22:29:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1638894740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d8b67d6cf0 con 0x55d8b68cab40 2024-09-11T22:29:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1638894740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d8b682b2b0 con 0x55d8b68c95d0 2024-09-11T22:29:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f162dffb640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f161c0a2d70 0x7f161c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1627fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1634a58640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f161c0a2d70 0x7f161c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1627fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68d6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59158/0 (socket says 172.21.15.5:59158) 2024-09-11T22:29:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1627fff640 1 -- 172.21.15.5:0/556038913 learned_addr learned my addr 172.21.15.5:0/556038913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f162dffb640 1 -- 172.21.15.5:0/3425763453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f16180efb00 con 0x7f1630102790 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f162ffff640 1 --2- 172.21.15.5:0/556038913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68c95d0 0x55d8b68c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1634a58640 1 --2- 172.21.15.5:0/556038913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68cab40 0x55d8b68c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1627fff640 1 -- 172.21.15.5:0/556038913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68c95d0 msgr2=0x55d8b68c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1627fff640 1 --2- 172.21.15.5:0/556038913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68c95d0 0x55d8b68c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1627fff640 1 -- 172.21.15.5:0/556038913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68cab40 msgr2=0x55d8b68c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.021+0000 7f1627fff640 1 --2- 172.21.15.5:0/556038913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68cab40 0x55d8b68c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.022+0000 7f1627fff640 1 -- 172.21.15.5:0/556038913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8b67bb3a0 con 0x55d8b68d4370 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.022+0000 7f1634a58640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f161c0a2d70 0x7f161c0a5230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1628005c40 tx=0x7f1628009c90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.022+0000 7f1627fff640 1 --2- 172.21.15.5:0/556038913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68d6760 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f16180b84a0 tx=0x7f161808f170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d513bc80b3936 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.022+0000 7f16267fc640 1 -- 172.21.15.5:0/556038913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618049070 con 0x55d8b68d4370 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.022+0000 7f16267fc640 1 -- 172.21.15.5:0/556038913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1618087190 con 0x55d8b68d4370 2024-09-11T22:29:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.022+0000 7f16267fc640 1 -- 172.21.15.5:0/556038913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16180bc040 con 0x55d8b68d4370 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 -- 172.21.15.5:0/556038913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 msgr2=0x55d8b68d6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 --2- 172.21.15.5:0/556038913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68d6760 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f16180b84a0 tx=0x7f161808f170 comp rx=0 tx=0).stop 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 -- 172.21.15.5:0/556038913 shutdown_connections 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 --2- 172.21.15.5:0/556038913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68d6760 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 --2- 172.21.15.5:0/556038913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68c95d0 0x55d8b68c9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 --2- 172.21.15.5:0/556038913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68cab40 0x55d8b68c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 -- 172.21.15.5:0/556038913 >> 172.21.15.5:0/556038913 conn(0x55d8b6892310 msgr2=0x55d8b68c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 -- 172.21.15.5:0/556038913 shutdown_connections 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 -- 172.21.15.5:0/556038913 wait complete. 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 -- start start 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.023+0000 7f1638894740 1 Processor -- start 2024-09-11T22:29:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1638894740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68c95d0 0x55d8b68d3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1638894740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68cab40 0x55d8b68d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1638894740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68d4370 0x55d8b68d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1638894740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d8b682f2f0 con 0x55d8b68cab40 2024-09-11T22:29:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1638894740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d8b67ea060 con 0x55d8b68d4370 2024-09-11T22:29:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1638894740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d8b68d6e30 con 0x55d8b68c95d0 2024-09-11T22:29:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1638894740 10 client.0 _ll_register_callbacks cb 0x55d8b653d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1627fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68cab40 0x55d8b68d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1627fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68cab40 0x55d8b68d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59168/0 (socket says 172.21.15.5:59168) 2024-09-11T22:29:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1627fff640 1 -- 172.21.15.5:0/3171208686 learned_addr learned my addr 172.21.15.5:0/3171208686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1634a58640 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68d4370 0x55d8b68d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f162ffff640 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68c95d0 0x55d8b68d3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1627fff640 1 -- 172.21.15.5:0/3171208686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68c95d0 msgr2=0x55d8b68d3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1627fff640 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68c95d0 0x55d8b68d3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1627fff640 1 -- 172.21.15.5:0/3171208686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68d4370 msgr2=0x55d8b68d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1627fff640 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68d4370 0x55d8b68d17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1627fff640 1 -- 172.21.15.5:0/3171208686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8b65a03c0 con 0x55d8b68cab40 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1634a58640 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68d4370 0x55d8b68d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1627fff640 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68cab40 0x55d8b68d10a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f161803d040 tx=0x7f1618031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73977c706e7948d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f160f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f160f7fe640 1 -- 172.21.15.5:0/3171208686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618049070 con 0x55d8b68cab40 2024-09-11T22:29:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f160f7fe640 1 -- 172.21.15.5:0/3171208686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f16180bc2a0 con 0x55d8b68cab40 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.024+0000 7f1638894740 1 -- 172.21.15.5:0/3171208686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1618087190 con 0x55d8b68cab40 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f160f7fe640 1 -- 172.21.15.5:0/3171208686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f16180bb9a0 con 0x55d8b68cab40 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f1638894740 10 client.17706 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f1638894740 1 -- 172.21.15.5:0/3171208686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d8b68e27e0 con 0x55d8b68cab40 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f160f7fe640 1 -- 172.21.15.5:0/3171208686 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1618087190 con 0x55d8b68cab40 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f160f7fe640 1 -- 172.21.15.5:0/3171208686 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f16180d9030 con 0x55d8b68cab40 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f1638894740 10 client.17706 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f1638894740 10 client.17706 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f1638894740 20 client.17706 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f1638894740 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55d8b68e3b30 0x55d8b68e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f1638894740 4 client.17706 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.025+0000 7f1638894740 1 -- 172.21.15.5:0/3171208686 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8b683c860 con 0x55d8b68e3b30 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.026+0000 7f162ffff640 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55d8b68e3b30 0x55d8b68e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.027+0000 7f162ffff640 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55d8b68e3b30 0x55d8b68e5f20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c1c975a21b92947 server_cookie=39bee9e362dfc4cb in_seq=0 out_seq=0 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.027+0000 7f160f7fe640 10 client.17706 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f160f7fe640 1 -- 172.21.15.5:0/3171208686 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8b683c860 con 0x55d8b68e3b30 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f160f7fe640 10 client.17706 handle_command_reply: tid=0 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 client.17706 shutdown 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 -- 172.21.15.5:0/3171208686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68cab40 msgr2=0x55d8b68d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68cab40 0x55d8b68d10a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f161803d040 tx=0x7f1618031a60 comp rx=0 tx=0).stop 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 -- 172.21.15.5:0/3171208686 shutdown_connections 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55d8b68e3b30 0x55d8b68e5f20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68d4370 0x55d8b68d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68cab40 0x55d8b68d10a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 --2- 172.21.15.5:0/3171208686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68c95d0 0x55d8b68d3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 -- 172.21.15.5:0/3171208686 >> 172.21.15.5:0/3171208686 conn(0x55d8b6892310 msgr2=0x55d8b68cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f160f7fe640 0 client.17706 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 -- 172.21.15.5:0/3171208686 shutdown_connections 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 1 -- 172.21.15.5:0/3171208686 wait complete. 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.028+0000 7f1638894740 20 client.17706 trim_cache size 0 max 16384 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1638894740 1 Processor -- start 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1638894740 1 -- start start 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1638894740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 0x55d8b68d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1638894740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 0x55d8b68dff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1638894740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1638894740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16301ada20 con 0x55d8b68d4370 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1638894740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16301af560 con 0x55d8b68c95d0 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1638894740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16301ae0f0 con 0x55d8b68cab40 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1634a58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1634a58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68dc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59178/0 (socket says 172.21.15.5:59178) 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1634a58640 1 -- 172.21.15.5:0/3529347940 learned_addr learned my addr 172.21.15.5:0/3529347940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1627fff640 1 --2- 172.21.15.5:0/3529347940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 0x55d8b68dff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f162ffff640 1 --2- 172.21.15.5:0/3529347940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 0x55d8b68d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1634a58640 1 -- 172.21.15.5:0/3529347940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 msgr2=0x55d8b68dff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1634a58640 1 --2- 172.21.15.5:0/3529347940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 0x55d8b68dff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1634a58640 1 -- 172.21.15.5:0/3529347940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 msgr2=0x55d8b68d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1634a58640 1 --2- 172.21.15.5:0/3529347940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 0x55d8b68d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f1634a58640 1 -- 172.21.15.5:0/3529347940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8b68c6960 con 0x55d8b68d4370 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.042+0000 7f162ffff640 1 --2- 172.21.15.5:0/3529347940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 0x55d8b68d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1634a58640 1 --2- 172.21.15.5:0/3529347940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68dc4a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1628009790 tx=0x7f1628004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188db5b59cf2c1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1625ffb640 1 -- 172.21.15.5:0/3529347940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162801e970 con 0x55d8b68d4370 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1625ffb640 1 -- 172.21.15.5:0/3529347940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1628020400 con 0x55d8b68d4370 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1625ffb640 1 -- 172.21.15.5:0/3529347940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628011640 con 0x55d8b68d4370 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1638894740 1 -- 172.21.15.5:0/3529347940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 msgr2=0x55d8b68dc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1638894740 1 --2- 172.21.15.5:0/3529347940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68dc4a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1628009790 tx=0x7f1628004ab0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1638894740 1 -- 172.21.15.5:0/3529347940 shutdown_connections 2024-09-11T22:29:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1638894740 1 --2- 172.21.15.5:0/3529347940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b68dc4a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1638894740 1 --2- 172.21.15.5:0/3529347940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 0x55d8b68dff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1638894740 1 --2- 172.21.15.5:0/3529347940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 0x55d8b68d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1638894740 1 -- 172.21.15.5:0/3529347940 >> 172.21.15.5:0/3529347940 conn(0x55d8b6892310 msgr2=0x55d8b68d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1638894740 1 -- 172.21.15.5:0/3529347940 shutdown_connections 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.043+0000 7f1638894740 1 -- 172.21.15.5:0/3529347940 wait complete. 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.044+0000 7f1638894740 1 -- start start 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.044+0000 7f1638894740 1 Processor -- start 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.044+0000 7f1638894740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 0x55d8b68eb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.044+0000 7f1638894740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 0x55d8b68eb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.044+0000 7f1638894740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b69497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.044+0000 7f1638894740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16301aee90 con 0x55d8b68d4370 2024-09-11T22:29:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.044+0000 7f1638894740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f16301ae7c0 con 0x55d8b68c95d0 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.044+0000 7f1638894740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f16301aba00 con 0x55d8b68cab40 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.044+0000 7f1638894740 10 client.0 _ll_register_callbacks cb 0x7f16280125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f1634a58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b69497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f1634a58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b69497b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59190/0 (socket says 172.21.15.5:59190) 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f1634a58640 1 -- 172.21.15.5:0/65322705 learned_addr learned my addr 172.21.15.5:0/65322705 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f1627fff640 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 0x55d8b68eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f162ffff640 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 0x55d8b68eb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f1634a58640 1 -- 172.21.15.5:0/65322705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 msgr2=0x55d8b68eb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f1634a58640 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 0x55d8b68eb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f1634a58640 1 -- 172.21.15.5:0/65322705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 msgr2=0x55d8b68eb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f1634a58640 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 0x55d8b68eb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f1634a58640 1 -- 172.21.15.5:0/65322705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8b68cc120 con 0x55d8b68d4370 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.045+0000 7f1627fff640 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 0x55d8b68eb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f1634a58640 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b69497b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1628005c10 tx=0x7f1628020ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1742859bc6ebe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f160f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f160f7fe640 1 -- 172.21.15.5:0/65322705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162801ee20 con 0x55d8b68d4370 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f160f7fe640 1 -- 172.21.15.5:0/65322705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f1628004b20 con 0x55d8b68d4370 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f1638894740 1 -- 172.21.15.5:0/65322705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1628020400 con 0x55d8b68d4370 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f160f7fe640 1 -- 172.21.15.5:0/65322705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1628004d20 con 0x55d8b68d4370 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f1638894740 10 client.17712 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f1638894740 1 -- 172.21.15.5:0/65322705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d8b6952240 con 0x55d8b68d4370 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f160f7fe640 1 -- 172.21.15.5:0/65322705 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1628020400 con 0x55d8b68d4370 2024-09-11T22:29:36.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f160f7fe640 1 -- 172.21.15.5:0/65322705 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f162803b030 con 0x55d8b68d4370 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f1638894740 10 client.17712 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f1638894740 10 client.17712 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f1638894740 20 client.17712 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f1638894740 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55d8b6952980 0x55d8b6954da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f1638894740 4 client.17712 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.046+0000 7f1638894740 1 -- 172.21.15.5:0/65322705 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8b6836d70 con 0x55d8b6952980 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.047+0000 7f162ffff640 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55d8b6952980 0x55d8b6954da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.047+0000 7f162ffff640 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55d8b6952980 0x55d8b6954da0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f41aa1f3b096790f server_cookie=c5f35502bcda35c2 in_seq=0 out_seq=0 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.047+0000 7f160f7fe640 10 client.17712 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.047+0000 7f160f7fe640 1 -- 172.21.15.5:0/65322705 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8b6836d70 con 0x55d8b6952980 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.047+0000 7f160f7fe640 10 client.17712 handle_command_reply: tid=0 2024-09-11T22:29:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.047+0000 7f1638894740 1 client.17712 shutdown 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 1 -- 172.21.15.5:0/65322705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 msgr2=0x55d8b69497b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b69497b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f1628005c10 tx=0x7f1628020ef0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 1 -- 172.21.15.5:0/65322705 shutdown_connections 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55d8b6952980 0x55d8b6954da0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8b68d4370 0x55d8b69497b0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d8b68cab40 0x55d8b68eb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 1 --2- 172.21.15.5:0/65322705 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d8b68c95d0 0x55d8b68eb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 1 -- 172.21.15.5:0/65322705 >> 172.21.15.5:0/65322705 conn(0x55d8b6892310 msgr2=0x55d8b68cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f160f7fe640 0 client.17712 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 1 -- 172.21.15.5:0/65322705 shutdown_connections 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 1 -- 172.21.15.5:0/65322705 wait complete. 2024-09-11T22:29:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f1638894740 20 client.17712 trim_cache size 0 max 16384 2024-09-11T22:29:36.068 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:29:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f16364bb640 1 -- 172.21.15.5:0/3425763453 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f161c0a2d70 msgr2=0x7f161c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f16364bb640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f161c0a2d70 0x7f161c0a5230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1628005c40 tx=0x7f1628009c90 comp rx=0 tx=0).stop 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f16364bb640 1 -- 172.21.15.5:0/3425763453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1630102790 msgr2=0x7f16301989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f16364bb640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1630102790 0x7f16301989d0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f1618006010 tx=0x7f16180042f0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f16364bb640 1 -- 172.21.15.5:0/3425763453 shutdown_connections 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f16364bb640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f161c0a2d70 0x7f161c0a5230 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f16364bb640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1630106f10 0x7f163019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f16364bb640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1630102790 0x7f16301989d0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f16364bb640 1 --2- 172.21.15.5:0/3425763453 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1630101de0 0x7f1630198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.048+0000 7f16364bb640 1 -- 172.21.15.5:0/3425763453 >> 172.21.15.5:0/3425763453 conn(0x7f16300fd8c0 msgr2=0x7f16300fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.049+0000 7f16364bb640 1 -- 172.21.15.5:0/3425763453 shutdown_connections 2024-09-11T22:29:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:36.049+0000 7f16364bb640 1 -- 172.21.15.5:0/3425763453 wait complete. 2024-09-11T22:29:36.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.09 MB/sec execute 234 sec latency 9329.133 ms 2024-09-11T22:29:36.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:29:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:37.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.06 MB/sec execute 235 sec latency 10329.199 ms 2024-09-11T22:29:37.721 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:37 smithi005 ceph-mon[30943]: pgmap v231: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 127 op/s 2024-09-11T22:29:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:37 smithi159 ceph-mon[35534]: pgmap v231: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 127 op/s 2024-09-11T22:29:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:37 smithi172 ceph-mon[35461]: pgmap v231: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 127 op/s 2024-09-11T22:29:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.940+0000 7fe17de70640 1 -- 172.21.15.5:0/18666571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe17810c170 msgr2=0x7fe17810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.940+0000 7fe17de70640 1 --2- 172.21.15.5:0/18666571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe17810c170 0x7fe17810e610 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe170009ca0 tx=0x7fe17002a950 comp rx=0 tx=0).stop 2024-09-11T22:29:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.940+0000 7fe17de70640 1 -- 172.21.15.5:0/18666571 shutdown_connections 2024-09-11T22:29:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.940+0000 7fe17de70640 1 --2- 172.21.15.5:0/18666571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe17810c170 0x7fe17810e610 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.940+0000 7fe17de70640 1 --2- 172.21.15.5:0/18666571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe178103660 0x7fe17810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.940+0000 7fe17de70640 1 --2- 172.21.15.5:0/18666571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe178102d40 0x7fe178103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.940+0000 7fe17de70640 1 -- 172.21.15.5:0/18666571 >> 172.21.15.5:0/18666571 conn(0x7fe1780fc7a0 msgr2=0x7fe1780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.940+0000 7fe17de70640 1 -- 172.21.15.5:0/18666571 shutdown_connections 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.940+0000 7fe17de70640 1 -- 172.21.15.5:0/18666571 wait complete. 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.941+0000 7fe17de70640 1 Processor -- start 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.941+0000 7fe17de70640 1 -- start start 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.941+0000 7fe17de70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe178102d40 0x7fe1781983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.941+0000 7fe17de70640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe178103660 0x7fe178198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.941+0000 7fe17de70640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe17810c170 0x7fe17819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.941+0000 7fe17de70640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe178111f70 con 0x7fe178102d40 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.941+0000 7fe17de70640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe178111df0 con 0x7fe17810c170 2024-09-11T22:29:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.941+0000 7fe17de70640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781120f0 con 0x7fe178103660 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe178102d40 0x7fe1781983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe178102d40 0x7fe1781983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59206/0 (socket says 172.21.15.5:59206) 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1777fe640 1 -- 172.21.15.5:0/425823031 learned_addr learned my addr 172.21.15.5:0/425823031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe177fff640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe17810c170 0x7fe17819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe16effd640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe178103660 0x7fe178198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1777fe640 1 -- 172.21.15.5:0/425823031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe178103660 msgr2=0x7fe178198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1777fe640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe178103660 0x7fe178198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1777fe640 1 -- 172.21.15.5:0/425823031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe17810c170 msgr2=0x7fe17819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1777fe640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe17810c170 0x7fe17819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1777fe640 1 -- 172.21.15.5:0/425823031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17819d380 con 0x7fe178102d40 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe16effd640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe178103660 0x7fe178198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1777fe640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe178102d40 0x7fe1781983d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe16800d950 tx=0x7fe16800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1757fa640 1 -- 172.21.15.5:0/425823031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168017070 con 0x7fe178102d40 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe17de70640 1 -- 172.21.15.5:0/425823031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17819d670 con 0x7fe178102d40 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1757fa640 1 -- 172.21.15.5:0/425823031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe168009c30 con 0x7fe178102d40 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe1757fa640 1 -- 172.21.15.5:0/425823031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16800f6b0 con 0x7fe178102d40 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.942+0000 7fe17de70640 1 -- 172.21.15.5:0/425823031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1781a4f50 con 0x7fe178102d40 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe1757fa640 1 -- 172.21.15.5:0/425823031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe1680109d0 con 0x7fe178102d40 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe180249740 1 Processor -- start 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe180249740 1 -- start start 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe180249740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe180249740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd1f75d0 0x5606bd1f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe180249740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd202370 0x5606bd204760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe180249740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5606bd15d2f0 con 0x5606bd1f8b40 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe180249740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5606bd104cf0 con 0x5606bd202370 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe180249740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5606bd1592b0 con 0x5606bd1f75d0 2024-09-11T22:29:37.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe177fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe177fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59212/0 (socket says 172.21.15.5:59212) 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe177fff640 1 -- 172.21.15.5:0/3251662916 learned_addr learned my addr 172.21.15.5:0/3251662916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.945+0000 7fe1777fe640 1 --2- 172.21.15.5:0/3251662916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd1f75d0 0x5606bd1f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe1757fa640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1640a2cf0 0x7fe1640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe16effd640 1 --2- 172.21.15.5:0/3251662916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd202370 0x5606bd204760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe177fff640 1 -- 172.21.15.5:0/3251662916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd1f75d0 msgr2=0x5606bd1f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe177fff640 1 --2- 172.21.15.5:0/3251662916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd1f75d0 0x5606bd1f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe177fff640 1 -- 172.21.15.5:0/3251662916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd202370 msgr2=0x5606bd204760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe177fff640 1 --2- 172.21.15.5:0/3251662916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd202370 0x5606bd204760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe177fff640 1 -- 172.21.15.5:0/3251662916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606bd0e93a0 con 0x5606bd1f8b40 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe177fff640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1640a2cf0 0x7fe1640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe1757fa640 1 -- 172.21.15.5:0/425823031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fe1680cf110 con 0x7fe178102d40 2024-09-11T22:29:37.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe177fff640 1 --2- 172.21.15.5:0/3251662916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1f7090 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe17002f9c0 tx=0x7fe17002f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa499d0ce4a7c89f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe16dffb640 1 -- 172.21.15.5:0/3251662916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170041040 con 0x5606bd1f8b40 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe16dffb640 1 -- 172.21.15.5:0/3251662916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe17002fc10 con 0x5606bd1f8b40 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.946+0000 7fe16dffb640 1 -- 172.21.15.5:0/3251662916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe170034040 con 0x5606bd1f8b40 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe180249740 1 -- 172.21.15.5:0/3251662916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 msgr2=0x5606bd1f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe180249740 1 --2- 172.21.15.5:0/3251662916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1f7090 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fe17002f9c0 tx=0x7fe17002f9f0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe177fff640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1640a2cf0 0x7fe1640a51b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe17004c000 tx=0x7fe17004d000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe180249740 1 -- 172.21.15.5:0/3251662916 shutdown_connections 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe180249740 1 --2- 172.21.15.5:0/3251662916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd202370 0x5606bd204760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe180249740 1 --2- 172.21.15.5:0/3251662916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd1f75d0 0x5606bd1f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe180249740 1 --2- 172.21.15.5:0/3251662916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1f7090 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe180249740 1 -- 172.21.15.5:0/3251662916 >> 172.21.15.5:0/3251662916 conn(0x5606bd1c0310 msgr2=0x5606bd1f3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe180249740 1 -- 172.21.15.5:0/3251662916 shutdown_connections 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe180249740 1 -- 172.21.15.5:0/3251662916 wait complete. 2024-09-11T22:29:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.947+0000 7fe180249740 1 -- start start 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe180249740 1 Processor -- start 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe180249740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd1f75d0 0x5606bd201d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe180249740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1ff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe180249740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd202370 0x5606bd1ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe180249740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5606bd205070 con 0x5606bd1f8b40 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe180249740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5606bd118060 con 0x5606bd202370 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe180249740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5606bd2051f0 con 0x5606bd1f75d0 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe180249740 10 client.0 _ll_register_callbacks cb 0x5606bce6b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe16effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1ff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe16effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1ff000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59218/0 (socket says 172.21.15.5:59218) 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe16effd640 1 -- 172.21.15.5:0/2103978716 learned_addr learned my addr 172.21.15.5:0/2103978716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe177fff640 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd202370 0x5606bd1ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe1777fe640 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd1f75d0 0x5606bd201d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe16effd640 1 -- 172.21.15.5:0/2103978716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd1f75d0 msgr2=0x5606bd201d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe16effd640 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd1f75d0 0x5606bd201d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe16effd640 1 -- 172.21.15.5:0/2103978716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd202370 msgr2=0x5606bd1ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe16effd640 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd202370 0x5606bd1ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.948+0000 7fe16effd640 1 -- 172.21.15.5:0/2103978716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606bcece3c0 con 0x5606bd1f8b40 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe16effd640 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1ff000 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe16000f670 tx=0x7fe16000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979dcd19939546e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe156ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe156ffd640 1 -- 172.21.15.5:0/2103978716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160016070 con 0x5606bd1f8b40 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe156ffd640 1 -- 172.21.15.5:0/2103978716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fe16000bb00 con 0x5606bd1f8b40 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe180249740 1 -- 172.21.15.5:0/2103978716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe17002fc10 con 0x5606bd1f8b40 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe156ffd640 1 -- 172.21.15.5:0/2103978716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe16000bd00 con 0x5606bd1f8b40 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe180249740 10 client.17724 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe180249740 1 -- 172.21.15.5:0/2103978716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5606bd210ba0 con 0x5606bd1f8b40 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe156ffd640 1 -- 172.21.15.5:0/2103978716 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe17002fc10 con 0x5606bd1f8b40 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe156ffd640 1 -- 172.21.15.5:0/2103978716 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe16001a520 con 0x5606bd1f8b40 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe180249740 10 client.17724 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe180249740 10 client.17724 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.949+0000 7fe180249740 20 client.17724 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.950+0000 7fe180249740 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5606bd2120b0 0x5606bd214510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.950+0000 7fe180249740 4 client.17724 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.950+0000 7fe180249740 1 -- 172.21.15.5:0/2103978716 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606bd16a860 con 0x5606bd2120b0 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.950+0000 7fe1777fe640 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5606bd2120b0 0x5606bd214510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.950+0000 7fe1777fe640 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5606bd2120b0 0x5606bd214510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70d843f0c949b2a9 server_cookie=51481ce865a4aeae in_seq=0 out_seq=0 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.950+0000 7fe156ffd640 10 client.17724 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe156ffd640 1 -- 172.21.15.5:0/2103978716 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606bd16a860 con 0x5606bd2120b0 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe156ffd640 10 client.17724 handle_command_reply: tid=0 2024-09-11T22:29:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 client.17724 shutdown 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 -- 172.21.15.5:0/2103978716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 msgr2=0x5606bd1ff000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1ff000 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fe16000f670 tx=0x7fe16000fb30 comp rx=0 tx=0).stop 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 -- 172.21.15.5:0/2103978716 shutdown_connections 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5606bd2120b0 0x5606bd214510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd202370 0x5606bd1ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd1ff000 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe156ffd640 0 client.17724 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 --2- 172.21.15.5:0/2103978716 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd1f75d0 0x5606bd201d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 -- 172.21.15.5:0/2103978716 >> 172.21.15.5:0/2103978716 conn(0x5606bd1c0310 msgr2=0x5606bd1f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 -- 172.21.15.5:0/2103978716 shutdown_connections 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 1 -- 172.21.15.5:0/2103978716 wait complete. 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.951+0000 7fe180249740 20 client.17724 trim_cache size 0 max 16384 2024-09-11T22:29:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.964+0000 7fe180249740 1 Processor -- start 2024-09-11T22:29:37.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.964+0000 7fe180249740 1 -- start start 2024-09-11T22:29:37.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe180249740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f75d0 0x5606bd20e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe180249740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f8b40 0x5606bd1fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe180249740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 0x5606bd1fa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe180249740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781ad9e0 con 0x5606bd1f75d0 2024-09-11T22:29:37.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe180249740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781af520 con 0x5606bd1f8b40 2024-09-11T22:29:37.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe180249740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781ae0b0 con 0x5606bd202370 2024-09-11T22:29:37.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f75d0 0x5606bd20e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe1777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f75d0 0x5606bd20e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59220/0 (socket says 172.21.15.5:59220) 2024-09-11T22:29:37.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe1777fe640 1 -- 172.21.15.5:0/3518827283 learned_addr learned my addr 172.21.15.5:0/3518827283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe177fff640 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 0x5606bd1fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe1777fe640 1 -- 172.21.15.5:0/3518827283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 msgr2=0x5606bd1fa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe1777fe640 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 0x5606bd1fa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe16effd640 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f8b40 0x5606bd1fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe1777fe640 1 -- 172.21.15.5:0/3518827283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f8b40 msgr2=0x5606bd1fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe1777fe640 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f8b40 0x5606bd1fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe1777fe640 1 -- 172.21.15.5:0/3518827283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606bd1f8760 con 0x5606bd1f75d0 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe177fff640 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 0x5606bd1fa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe16effd640 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f8b40 0x5606bd1fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe1777fe640 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f75d0 0x5606bd20e310 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe16809b620 tx=0x7fe168086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653ccb1a5f82b39a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.965+0000 7fe16d7fa640 1 -- 172.21.15.5:0/3518827283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168020040 con 0x5606bd1f75d0 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe16d7fa640 1 -- 172.21.15.5:0/3518827283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe16809b690 con 0x5606bd1f75d0 2024-09-11T22:29:37.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe16d7fa640 1 -- 172.21.15.5:0/3518827283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168097d00 con 0x5606bd1f75d0 2024-09-11T22:29:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe180249740 1 -- 172.21.15.5:0/3518827283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f75d0 msgr2=0x5606bd20e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe180249740 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f75d0 0x5606bd20e310 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fe16809b620 tx=0x7fe168086040 comp rx=0 tx=0).stop 2024-09-11T22:29:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe180249740 1 -- 172.21.15.5:0/3518827283 shutdown_connections 2024-09-11T22:29:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe180249740 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 0x5606bd1fa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe180249740 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f8b40 0x5606bd1fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe180249740 1 --2- 172.21.15.5:0/3518827283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f75d0 0x5606bd20e310 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe180249740 1 -- 172.21.15.5:0/3518827283 >> 172.21.15.5:0/3518827283 conn(0x5606bd1c0310 msgr2=0x5606bd1fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe180249740 1 -- 172.21.15.5:0/3518827283 shutdown_connections 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.966+0000 7fe180249740 1 -- 172.21.15.5:0/3518827283 wait complete. 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe180249740 1 -- start start 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe180249740 1 Processor -- start 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe180249740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f75d0 0x5606bd20ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe180249740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd205e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe180249740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 0x5606bd2063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe180249740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781aee50 con 0x5606bd1f8b40 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe180249740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781ae780 con 0x5606bd1f75d0 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe180249740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe1781ab9c0 con 0x5606bd202370 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe180249740 10 client.0 _ll_register_callbacks cb 0x7fe17806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe16effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd205e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe16effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd205e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59224/0 (socket says 172.21.15.5:59224) 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.967+0000 7fe16effd640 1 -- 172.21.15.5:0/3930331472 learned_addr learned my addr 172.21.15.5:0/3930331472 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe177fff640 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 0x5606bd2063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe1777fe640 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f75d0 0x5606bd20ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe16effd640 1 -- 172.21.15.5:0/3930331472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 msgr2=0x5606bd2063a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe16effd640 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 0x5606bd2063a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe16effd640 1 -- 172.21.15.5:0/3930331472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f75d0 msgr2=0x5606bd20ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe16effd640 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f75d0 0x5606bd20ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe16effd640 1 -- 172.21.15.5:0/3930331472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606bd206ba0 con 0x5606bd1f8b40 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe16effd640 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd205e60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe16001ab50 tx=0x7fe1600046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f54f78e8953670 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe156ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe156ffd640 1 -- 172.21.15.5:0/3930331472 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160016040 con 0x5606bd1f8b40 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe156ffd640 1 -- 172.21.15.5:0/3930331472 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fe16001a600 con 0x5606bd1f8b40 2024-09-11T22:29:37.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe180249740 1 -- 172.21.15.5:0/3930331472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe16809b690 con 0x5606bd1f8b40 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe156ffd640 1 -- 172.21.15.5:0/3930331472 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe16003a050 con 0x5606bd1f8b40 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe180249740 10 client.17730 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.968+0000 7fe180249740 1 -- 172.21.15.5:0/3930331472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5606bd27ccc0 con 0x5606bd1f8b40 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.969+0000 7fe156ffd640 1 -- 172.21.15.5:0/3930331472 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe16809b690 con 0x5606bd1f8b40 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.969+0000 7fe156ffd640 1 -- 172.21.15.5:0/3930331472 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe160035030 con 0x5606bd1f8b40 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.969+0000 7fe180249740 10 client.17730 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.969+0000 7fe180249740 10 client.17730 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.969+0000 7fe180249740 20 client.17730 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.969+0000 7fe180249740 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5606bd27d4c0 0x5606bd27f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.969+0000 7fe180249740 4 client.17730 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.969+0000 7fe180249740 1 -- 172.21.15.5:0/3930331472 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606bd164d70 con 0x5606bd27d4c0 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.969+0000 7fe1777fe640 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5606bd27d4c0 0x5606bd27f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe1777fe640 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5606bd27d4c0 0x5606bd27f8e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b46ede90e40542a server_cookie=d17ab20dd6980969 in_seq=0 out_seq=0 2024-09-11T22:29:37.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe156ffd640 10 client.17730 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe156ffd640 1 -- 172.21.15.5:0/3930331472 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606bd164d70 con 0x5606bd27d4c0 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe156ffd640 10 client.17730 handle_command_reply: tid=0 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe180249740 1 client.17730 shutdown 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe180249740 1 -- 172.21.15.5:0/3930331472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 msgr2=0x5606bd205e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe180249740 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd205e60 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fe16001ab50 tx=0x7fe1600046e0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe180249740 1 -- 172.21.15.5:0/3930331472 shutdown_connections 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe180249740 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5606bd27d4c0 0x5606bd27f8e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe180249740 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5606bd202370 0x5606bd2063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe180249740 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606bd1f8b40 0x5606bd205e60 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe180249740 1 --2- 172.21.15.5:0/3930331472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5606bd1f75d0 0x5606bd20ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.970+0000 7fe180249740 1 -- 172.21.15.5:0/3930331472 >> 172.21.15.5:0/3930331472 conn(0x5606bd1c0310 msgr2=0x5606bd200860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.971+0000 7fe156ffd640 0 client.17730 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.971+0000 7fe180249740 1 -- 172.21.15.5:0/3930331472 shutdown_connections 2024-09-11T22:29:37.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.971+0000 7fe180249740 1 -- 172.21.15.5:0/3930331472 wait complete. 2024-09-11T22:29:37.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.971+0000 7fe180249740 20 client.17730 trim_cache size 0 max 16384 2024-09-11T22:29:37.986 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:29:37.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.971+0000 7fe17de70640 1 -- 172.21.15.5:0/425823031 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1640a2cf0 msgr2=0x7fe1640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:37.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.971+0000 7fe17de70640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1640a2cf0 0x7fe1640a51b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe17004c000 tx=0x7fe17004d000 comp rx=0 tx=0).stop 2024-09-11T22:29:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.971+0000 7fe17de70640 1 -- 172.21.15.5:0/425823031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe178102d40 msgr2=0x7fe1781983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.971+0000 7fe17de70640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe178102d40 0x7fe1781983d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe16800d950 tx=0x7fe16800de10 comp rx=0 tx=0).stop 2024-09-11T22:29:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.972+0000 7fe17de70640 1 -- 172.21.15.5:0/425823031 shutdown_connections 2024-09-11T22:29:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.972+0000 7fe17de70640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1640a2cf0 0x7fe1640a51b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.972+0000 7fe17de70640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe17810c170 0x7fe17819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.972+0000 7fe17de70640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe178103660 0x7fe178198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.972+0000 7fe17de70640 1 --2- 172.21.15.5:0/425823031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe178102d40 0x7fe1781983d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.972+0000 7fe17de70640 1 -- 172.21.15.5:0/425823031 >> 172.21.15.5:0/425823031 conn(0x7fe1780fc7a0 msgr2=0x7fe1780fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.972+0000 7fe17de70640 1 -- 172.21.15.5:0/425823031 shutdown_connections 2024-09-11T22:29:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:37.972+0000 7fe17de70640 1 -- 172.21.15.5:0/425823031 wait complete. 2024-09-11T22:29:38.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 8.02 MB/sec execute 236 sec latency 11329.280 ms 2024-09-11T22:29:38.777 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:29:39.036 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:39.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 7.99 MB/sec execute 237 sec latency 12329.340 ms 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.549+0000 7fc6df059640 1 -- 172.21.15.5:0/1760328398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6d8102790 msgr2=0x7fc6d81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.549+0000 7fc6df059640 1 --2- 172.21.15.5:0/1760328398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6d8102790 0x7fc6d81067e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009900 tx=0x7fc6c802a410 comp rx=0 tx=0).stop 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.549+0000 7fc6df059640 1 -- 172.21.15.5:0/1760328398 shutdown_connections 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.549+0000 7fc6df059640 1 --2- 172.21.15.5:0/1760328398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d8106f10 0x7fc6d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.549+0000 7fc6df059640 1 --2- 172.21.15.5:0/1760328398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6d8102790 0x7fc6d81067e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.549+0000 7fc6df059640 1 --2- 172.21.15.5:0/1760328398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d8101de0 0x7fc6d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.549+0000 7fc6df059640 1 -- 172.21.15.5:0/1760328398 >> 172.21.15.5:0/1760328398 conn(0x7fc6d80fd8c0 msgr2=0x7fc6d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.549+0000 7fc6df059640 1 -- 172.21.15.5:0/1760328398 shutdown_connections 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.549+0000 7fc6df059640 1 -- 172.21.15.5:0/1760328398 wait complete. 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6df059640 1 Processor -- start 2024-09-11T22:29:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6df059640 1 -- start start 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6df059640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6d8101de0 0x7fc6d80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6df059640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d8102790 0x7fc6d8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6df059640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d8106f10 0x7fc6d8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6df059640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d8111f90 con 0x7fc6d8101de0 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6df059640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d8111e10 con 0x7fc6d8102790 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6df059640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d8112110 con 0x7fc6d8106f10 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6dcdf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6d8101de0 0x7fc6d80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6dcdf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6d8101de0 0x7fc6d80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59236/0 (socket says 172.21.15.5:59236) 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/3926987479 learned_addr learned my addr 172.21.15.5:0/3926987479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6d7fff640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d8102790 0x7fc6d8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6dd5f6640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d8106f10 0x7fc6d8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/3926987479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d8106f10 msgr2=0x7fc6d8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.550+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d8106f10 0x7fc6d8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/3926987479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d8102790 msgr2=0x7fc6d8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d8102790 0x7fc6d8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/3926987479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d81a6950 con 0x7fc6d8101de0 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6dd5f6640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d8106f10 0x7fc6d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6d7fff640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d8102790 0x7fc6d8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6d8101de0 0x7fc6d80744f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000b4a0 tx=0x7fc6c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6d5ffb640 1 -- 172.21.15.5:0/3926987479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0014070 con 0x7fc6d8101de0 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6df059640 1 -- 172.21.15.5:0/3926987479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d81a6c40 con 0x7fc6d8101de0 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6d5ffb640 1 -- 172.21.15.5:0/3926987479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6c0010af0 con 0x7fc6d8101de0 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6d5ffb640 1 -- 172.21.15.5:0/3926987479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0017d10 con 0x7fc6d8101de0 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.551+0000 7fc6df059640 1 -- 172.21.15.5:0/3926987479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d81a7130 con 0x7fc6d8101de0 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.553+0000 7fc6d5ffb640 1 -- 172.21.15.5:0/3926987479 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc6c0017070 con 0x7fc6d8101de0 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6e1432740 1 Processor -- start 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6e1432740 1 -- start start 2024-09-11T22:29:39.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b23ab40 0x55a98b239090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b239a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b244370 0x55a98b246760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6e1432740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a98b19f2f0 con 0x55a98b2395d0 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6e1432740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a98b146cf0 con 0x55a98b244370 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6e1432740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a98b19b2b0 con 0x55a98b23ab40 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6d5ffb640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6c40a2cf0 0x7fc6c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6dcdf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b239a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6dcdf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b239a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59252/0 (socket says 172.21.15.5:59252) 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/967000416 learned_addr learned my addr 172.21.15.5:0/967000416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.554+0000 7fc6d7fff640 1 --2- 172.21.15.5:0/967000416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b244370 0x55a98b246760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6dd5f6640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6c40a2cf0 0x7fc6c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6dd5f6640 1 --2- 172.21.15.5:0/967000416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b23ab40 0x55a98b239090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6d5ffb640 1 -- 172.21.15.5:0/3926987479 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc6c00ce860 con 0x7fc6d8101de0 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/967000416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b23ab40 msgr2=0x55a98b239090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/967000416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b23ab40 0x55a98b239090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/967000416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b244370 msgr2=0x55a98b246760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/967000416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b244370 0x55a98b246760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/967000416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98b12b3a0 con 0x55a98b2395d0 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/967000416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b239a10 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000be60 tx=0x7fc6c0009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07456c42a0ab68e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6dd5f6640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6c40a2cf0 0x7fc6c40a51b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009270 tx=0x7fc6d0009770 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6ceffd640 1 -- 172.21.15.5:0/967000416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c001e070 con 0x55a98b2395d0 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6ceffd640 1 -- 172.21.15.5:0/967000416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6c009b4f0 con 0x55a98b2395d0 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.555+0000 7fc6ceffd640 1 -- 172.21.15.5:0/967000416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00cc190 con 0x55a98b2395d0 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 -- 172.21.15.5:0/967000416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 msgr2=0x55a98b239a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 --2- 172.21.15.5:0/967000416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b239a10 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000be60 tx=0x7fc6c0009390 comp rx=0 tx=0).stop 2024-09-11T22:29:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 -- 172.21.15.5:0/967000416 shutdown_connections 2024-09-11T22:29:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 --2- 172.21.15.5:0/967000416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b244370 0x55a98b246760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 --2- 172.21.15.5:0/967000416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b239a10 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 --2- 172.21.15.5:0/967000416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b23ab40 0x55a98b239090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 -- 172.21.15.5:0/967000416 >> 172.21.15.5:0/967000416 conn(0x55a98b202310 msgr2=0x55a98b235c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 -- 172.21.15.5:0/967000416 shutdown_connections 2024-09-11T22:29:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 -- 172.21.15.5:0/967000416 wait complete. 2024-09-11T22:29:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 -- start start 2024-09-11T22:29:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.556+0000 7fc6e1432740 1 Processor -- start 2024-09-11T22:29:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b243df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b23ab40 0x55a98b2410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b244370 0x55a98b2417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6e1432740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a98b246fb0 con 0x55a98b2395d0 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6e1432740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a98b15a060 con 0x55a98b244370 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6e1432740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a98b247130 con 0x55a98b23ab40 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6e1432740 10 client.0 _ll_register_callbacks cb 0x55a98aead8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6dcdf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b243df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6dcdf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b243df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59254/0 (socket says 172.21.15.5:59254) 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/2874101805 learned_addr learned my addr 172.21.15.5:0/2874101805 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/2874101805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b23ab40 msgr2=0x55a98b2410a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b23ab40 0x55a98b2410a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/2874101805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b244370 msgr2=0x55a98b2417e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b244370 0x55a98b2417e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6dcdf5640 1 -- 172.21.15.5:0/2874101805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98af103c0 con 0x55a98b2395d0 2024-09-11T22:29:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b243df0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc6c009ad80 tx=0x7fc6c0010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4eeb5e1f416001c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.557+0000 7fc6b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6b7fff640 1 -- 172.21.15.5:0/2874101805 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c001e050 con 0x55a98b2395d0 2024-09-11T22:29:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6b7fff640 1 -- 172.21.15.5:0/2874101805 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc6c009b050 con 0x55a98b2395d0 2024-09-11T22:29:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6e1432740 1 -- 172.21.15.5:0/2874101805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6c009b4f0 con 0x55a98b2395d0 2024-09-11T22:29:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6b7fff640 1 -- 172.21.15.5:0/2874101805 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc6c00cc3a0 con 0x55a98b2395d0 2024-09-11T22:29:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6e1432740 10 client.17742 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6e1432740 1 -- 172.21.15.5:0/2874101805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a98b252ae0 con 0x55a98b2395d0 2024-09-11T22:29:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6b7fff640 1 -- 172.21.15.5:0/2874101805 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6c009b4f0 con 0x55a98b2395d0 2024-09-11T22:29:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6b7fff640 1 -- 172.21.15.5:0/2874101805 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc6c0061050 con 0x55a98b2395d0 2024-09-11T22:29:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6e1432740 10 client.17742 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6e1432740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6e1432740 20 client.17742 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6e1432740 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a98b253e30 0x55a98b256220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6e1432740 4 client.17742 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6e1432740 1 -- 172.21.15.5:0/2874101805 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a98b1ac860 con 0x55a98b253e30 2024-09-11T22:29:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.558+0000 7fc6d7fff640 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a98b253e30 0x55a98b256220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.559+0000 7fc6b7fff640 10 client.17742 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.559+0000 7fc6d7fff640 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a98b253e30 0x55a98b256220 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35b4ac1efa85bc7 server_cookie=753ec26e81a47ff3 in_seq=0 out_seq=0 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6b7fff640 1 -- 172.21.15.5:0/2874101805 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a98b1ac860 con 0x55a98b253e30 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6b7fff640 10 client.17742 handle_command_reply: tid=0 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 client.17742 shutdown 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 -- 172.21.15.5:0/2874101805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 msgr2=0x55a98b243df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b243df0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fc6c009ad80 tx=0x7fc6c0010f30 comp rx=0 tx=0).stop 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 -- 172.21.15.5:0/2874101805 shutdown_connections 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a98b253e30 0x55a98b256220 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b244370 0x55a98b2417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b23ab40 0x55a98b2410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6b7fff640 0 client.17742 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 --2- 172.21.15.5:0/2874101805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b2395d0 0x55a98b243df0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 -- 172.21.15.5:0/2874101805 >> 172.21.15.5:0/2874101805 conn(0x55a98b202310 msgr2=0x55a98b23b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:39.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 -- 172.21.15.5:0/2874101805 shutdown_connections 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 1 -- 172.21.15.5:0/2874101805 wait complete. 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.560+0000 7fc6e1432740 20 client.17742 trim_cache size 0 max 16384 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6e1432740 1 Processor -- start 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6e1432740 1 -- start start 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b2395d0 0x55a98b2503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b23ab40 0x55a98b240980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b244370 0x55a98b259240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6e1432740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81afcc0 con 0x55a98b244370 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6e1432740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81b1800 con 0x55a98b23ab40 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6e1432740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81b0390 con 0x55a98b2395d0 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6dd5f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b244370 0x55a98b259240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6dd5f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b244370 0x55a98b259240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59256/0 (socket says 172.21.15.5:59256) 2024-09-11T22:29:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6dd5f6640 1 -- 172.21.15.5:0/3931483090 learned_addr learned my addr 172.21.15.5:0/3931483090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.573+0000 7fc6d7fff640 1 --2- 172.21.15.5:0/3931483090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b23ab40 0x55a98b240980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/3931483090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b2395d0 0x55a98b2503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6dd5f6640 1 -- 172.21.15.5:0/3931483090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b2395d0 msgr2=0x55a98b2503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6dd5f6640 1 --2- 172.21.15.5:0/3931483090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b2395d0 0x55a98b2503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6dd5f6640 1 -- 172.21.15.5:0/3931483090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b23ab40 msgr2=0x55a98b240980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6dd5f6640 1 --2- 172.21.15.5:0/3931483090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b23ab40 0x55a98b240980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6dd5f6640 1 -- 172.21.15.5:0/3931483090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98b23c040 con 0x55a98b244370 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/3931483090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b2395d0 0x55a98b2503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6dd5f6640 1 --2- 172.21.15.5:0/3931483090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b244370 0x55a98b259240 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000da40 tx=0x7fc6d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7996464c0b6afaee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6ce7fc640 1 -- 172.21.15.5:0/3931483090 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0023030 con 0x55a98b244370 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6ce7fc640 1 -- 172.21.15.5:0/3931483090 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d000de40 con 0x55a98b244370 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.574+0000 7fc6ce7fc640 1 -- 172.21.15.5:0/3931483090 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0019040 con 0x55a98b244370 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 -- 172.21.15.5:0/3931483090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b244370 msgr2=0x55a98b259240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:39.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 --2- 172.21.15.5:0/3931483090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b244370 0x55a98b259240 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000da40 tx=0x7fc6d0004ab0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 -- 172.21.15.5:0/3931483090 shutdown_connections 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 --2- 172.21.15.5:0/3931483090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b244370 0x55a98b259240 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 --2- 172.21.15.5:0/3931483090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b23ab40 0x55a98b240980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 --2- 172.21.15.5:0/3931483090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b2395d0 0x55a98b2503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 -- 172.21.15.5:0/3931483090 >> 172.21.15.5:0/3931483090 conn(0x55a98b202310 msgr2=0x55a98b23b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 -- 172.21.15.5:0/3931483090 shutdown_connections 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 -- 172.21.15.5:0/3931483090 wait complete. 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 -- start start 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 Processor -- start 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.575+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b2395d0 0x55a98b259b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b23ab40 0x55a98b256ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6e1432740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b244370 0x55a98b257630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6e1432740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81b1130 con 0x55a98b23ab40 2024-09-11T22:29:39.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6e1432740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81b0a60 con 0x55a98b2395d0 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6e1432740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d81adca0 con 0x55a98b244370 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6e1432740 10 client.0 _ll_register_callbacks cb 0x7fc6c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b23ab40 0x55a98b256ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b23ab40 0x55a98b256ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59270/0 (socket says 172.21.15.5:59270) 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6d7fff640 1 -- 172.21.15.5:0/1703984702 learned_addr learned my addr 172.21.15.5:0/1703984702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6dd5f6640 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b244370 0x55a98b257630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b2395d0 0x55a98b259b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6d7fff640 1 -- 172.21.15.5:0/1703984702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b244370 msgr2=0x55a98b257630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6d7fff640 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b244370 0x55a98b257630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6d7fff640 1 -- 172.21.15.5:0/1703984702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b2395d0 msgr2=0x55a98b259b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6d7fff640 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b2395d0 0x55a98b259b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6d7fff640 1 -- 172.21.15.5:0/1703984702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98b2bd770 con 0x55a98b23ab40 2024-09-11T22:29:39.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6d7fff640 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b23ab40 0x55a98b256ea0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc6c802a930 tx=0x7fc6c8004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f56490cd6809ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6b7fff640 1 -- 172.21.15.5:0/1703984702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8035040 con 0x55a98b23ab40 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.576+0000 7fc6b7fff640 1 -- 172.21.15.5:0/1703984702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc6c8005040 con 0x55a98b23ab40 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6e1432740 1 -- 172.21.15.5:0/1703984702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6d000de40 con 0x55a98b23ab40 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6b7fff640 1 -- 172.21.15.5:0/1703984702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc6c8053050 con 0x55a98b23ab40 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6e1432740 10 client.17748 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6e1432740 1 -- 172.21.15.5:0/1703984702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a98b2be900 con 0x55a98b23ab40 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6b7fff640 1 -- 172.21.15.5:0/1703984702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d000de40 con 0x55a98b23ab40 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6b7fff640 1 -- 172.21.15.5:0/1703984702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc6c80035f0 con 0x55a98b23ab40 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6e1432740 10 client.17748 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6e1432740 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:39.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6e1432740 20 client.17748 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6e1432740 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a98b2bf000 0x55a98b2c1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6e1432740 4 client.17748 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:29:39.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6e1432740 1 -- 172.21.15.5:0/1703984702 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a98b1a6d70 con 0x55a98b2bf000 2024-09-11T22:29:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.577+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a98b2bf000 0x55a98b2c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6b7fff640 10 client.17748 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6dcdf5640 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a98b2bf000 0x55a98b2c1420 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7465dfe8c708b3fe server_cookie=4350dab5d54ba65 in_seq=0 out_seq=0 2024-09-11T22:29:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6b7fff640 1 -- 172.21.15.5:0/1703984702 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a98b1a6d70 con 0x55a98b2bf000 2024-09-11T22:29:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6b7fff640 10 client.17748 handle_command_reply: tid=0 2024-09-11T22:29:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 client.17748 shutdown 2024-09-11T22:29:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 -- 172.21.15.5:0/1703984702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b23ab40 msgr2=0x55a98b256ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b23ab40 0x55a98b256ea0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc6c802a930 tx=0x7fc6c8004130 comp rx=0 tx=0).stop 2024-09-11T22:29:39.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 -- 172.21.15.5:0/1703984702 shutdown_connections 2024-09-11T22:29:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a98b2bf000 0x55a98b2c1420 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a98b244370 0x55a98b257630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a98b23ab40 0x55a98b256ea0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 --2- 172.21.15.5:0/1703984702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a98b2395d0 0x55a98b259b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 -- 172.21.15.5:0/1703984702 >> 172.21.15.5:0/1703984702 conn(0x55a98b202310 msgr2=0x55a98b2344e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6b7fff640 0 client.17748 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 -- 172.21.15.5:0/1703984702 shutdown_connections 2024-09-11T22:29:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 1 -- 172.21.15.5:0/1703984702 wait complete. 2024-09-11T22:29:39.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.578+0000 7fc6e1432740 20 client.17748 trim_cache size 0 max 16384 2024-09-11T22:29:39.595 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:29:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 -- 172.21.15.5:0/3926987479 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6c40a2cf0 msgr2=0x7fc6c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6c40a2cf0 0x7fc6c40a51b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009270 tx=0x7fc6d0009770 comp rx=0 tx=0).stop 2024-09-11T22:29:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 -- 172.21.15.5:0/3926987479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6d8101de0 msgr2=0x7fc6d80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6d8101de0 0x7fc6d80744f0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000b4a0 tx=0x7fc6c000b960 comp rx=0 tx=0).stop 2024-09-11T22:29:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 -- 172.21.15.5:0/3926987479 shutdown_connections 2024-09-11T22:29:39.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6d8106f10 0x7fc6d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6c40a2cf0 0x7fc6c40a51b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6d8102790 0x7fc6d8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 --2- 172.21.15.5:0/3926987479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6d8101de0 0x7fc6d80744f0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 -- 172.21.15.5:0/3926987479 >> 172.21.15.5:0/3926987479 conn(0x7fc6d80fd8c0 msgr2=0x7fc6d80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 -- 172.21.15.5:0/3926987479 shutdown_connections 2024-09-11T22:29:39.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:39.579+0000 7fc6df059640 1 -- 172.21.15.5:0/3926987479 wait complete. 2024-09-11T22:29:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:39 smithi005 ceph-mon[30943]: pgmap v232: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 209 op/s 2024-09-11T22:29:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:39 smithi159 ceph-mon[35534]: pgmap v232: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 209 op/s 2024-09-11T22:29:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:39 smithi172 ceph-mon[35461]: pgmap v232: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 209 op/s 2024-09-11T22:29:40.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 7.96 MB/sec execute 238 sec latency 13329.426 ms 2024-09-11T22:29:40.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:29:40.902 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:41.328 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 7.92 MB/sec execute 239 sec latency 14329.477 ms 2024-09-11T22:29:41.426 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:29:41.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:29:41.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.454+0000 7f8e4b14c640 1 -- 172.21.15.5:0/3235389422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44106a70 msgr2=0x7f8e44106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:41.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.454+0000 7f8e4b14c640 1 --2- 172.21.15.5:0/3235389422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44106a70 0x7f8e44106e50 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009930 tx=0x7f8e3802a420 comp rx=0 tx=0).stop 2024-09-11T22:29:41.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.454+0000 7f8e4b14c640 1 -- 172.21.15.5:0/3235389422 shutdown_connections 2024-09-11T22:29:41.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.454+0000 7f8e4b14c640 1 --2- 172.21.15.5:0/3235389422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44101410 0x7f8e4410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.454+0000 7f8e4b14c640 1 --2- 172.21.15.5:0/3235389422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e44100a70 0x7f8e44100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.454+0000 7f8e4b14c640 1 --2- 172.21.15.5:0/3235389422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44106a70 0x7f8e44106e50 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.454+0000 7f8e4b14c640 1 -- 172.21.15.5:0/3235389422 >> 172.21.15.5:0/3235389422 conn(0x7f8e440fc790 msgr2=0x7f8e440febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:41.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.454+0000 7f8e4b14c640 1 -- 172.21.15.5:0/3235389422 shutdown_connections 2024-09-11T22:29:41.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.454+0000 7f8e4b14c640 1 -- 172.21.15.5:0/3235389422 wait complete. 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e4b14c640 1 Processor -- start 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e4b14c640 1 -- start start 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e4b14c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e44100a70 0x7f8e441983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e4b14c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44101410 0x7f8e441988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e4b14c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44106a70 0x7f8e4419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e4b14c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44110e80 con 0x7f8e44101410 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e4b14c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44110d00 con 0x7f8e44106a70 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e4b14c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e44111000 con 0x7f8e44100a70 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44101410 0x7f8e441988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44101410 0x7f8e441988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54078/0 (socket says 172.21.15.5:54078) 2024-09-11T22:29:41.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.455+0000 7f8e43fff640 1 -- 172.21.15.5:0/2099490827 learned_addr learned my addr 172.21.15.5:0/2099490827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e48ee8640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e44100a70 0x7f8e441983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e496e9640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44106a70 0x7f8e4419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e43fff640 1 -- 172.21.15.5:0/2099490827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e44100a70 msgr2=0x7f8e441983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e43fff640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e44100a70 0x7f8e441983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e43fff640 1 -- 172.21.15.5:0/2099490827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44106a70 msgr2=0x7f8e4419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e43fff640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44106a70 0x7f8e4419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e43fff640 1 -- 172.21.15.5:0/2099490827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4419d360 con 0x7f8e44101410 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e496e9640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44106a70 0x7f8e4419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e43fff640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44101410 0x7f8e441988f0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c920 tx=0x7f8e3000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e41ffb640 1 -- 172.21.15.5:0/2099490827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30007c70 con 0x7f8e44101410 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e4b14c640 1 -- 172.21.15.5:0/2099490827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4419d650 con 0x7f8e44101410 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e41ffb640 1 -- 172.21.15.5:0/2099490827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e30007e10 con 0x7f8e44101410 2024-09-11T22:29:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.456+0000 7f8e41ffb640 1 -- 172.21.15.5:0/2099490827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30018890 con 0x7f8e44101410 2024-09-11T22:29:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.457+0000 7f8e4b14c640 1 -- 172.21.15.5:0/2099490827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e441a4f30 con 0x7f8e44101410 2024-09-11T22:29:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.458+0000 7f8e4b14c640 1 -- 172.21.15.5:0/2099490827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e44100ed0 con 0x7f8e44101410 2024-09-11T22:29:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.459+0000 7f8e41ffb640 1 -- 172.21.15.5:0/2099490827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8e30018a70 con 0x7f8e44101410 2024-09-11T22:29:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.460+0000 7f8e41ffb640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e340a2cf0 0x7f8e340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.460+0000 7f8e48ee8640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e340a2cf0 0x7f8e340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.460+0000 7f8e41ffb640 1 -- 172.21.15.5:0/2099490827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f8e300cf5e0 con 0x7f8e44101410 2024-09-11T22:29:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.461+0000 7f8e48ee8640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e340a2cf0 0x7f8e340a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8e38006010 tx=0x7f8e38000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.465+0000 7f8e41ffb640 1 -- 172.21.15.5:0/2099490827 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e30014030 con 0x7f8e44101410 2024-09-11T22:29:41.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:41 smithi005 ceph-mon[30943]: pgmap v233: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-09-11T22:29:41.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.720+0000 7f8e4b14c640 1 -- 172.21.15.5:0/2099490827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e44109ff0 con 0x7f8e44101410 2024-09-11T22:29:41.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.721+0000 7f8e41ffb640 1 -- 172.21.15.5:0/2099490827 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11967 (secure 0 0 0) 0x7f8e3009c1c0 con 0x7f8e44101410 2024-09-11T22:29:41.722 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:29:41.722 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"btime":"2024-09-11T22:29:21:436884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:29:21.436856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26914":{"gid":26914,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1755675305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1755675305},{"type":"v1","addr":"172.21.15.159:6833","nonce":1755675305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:29:41.722 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 40 2024-09-11T22:29:41.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.724+0000 7f8e4b14c640 1 -- 172.21.15.5:0/2099490827 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e340a2cf0 msgr2=0x7f8e340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.724+0000 7f8e4b14c640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e340a2cf0 0x7f8e340a51b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8e38006010 tx=0x7f8e38000bc0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.724+0000 7f8e4b14c640 1 -- 172.21.15.5:0/2099490827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44101410 msgr2=0x7f8e441988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.724+0000 7f8e4b14c640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44101410 0x7f8e441988f0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000c920 tx=0x7f8e3000cde0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.725+0000 7f8e4b14c640 1 -- 172.21.15.5:0/2099490827 shutdown_connections 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.725+0000 7f8e4b14c640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e340a2cf0 0x7f8e340a51b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.725+0000 7f8e4b14c640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e44106a70 0x7f8e4419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.725+0000 7f8e4b14c640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e44101410 0x7f8e441988f0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.725+0000 7f8e4b14c640 1 --2- 172.21.15.5:0/2099490827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e44100a70 0x7f8e441983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.725+0000 7f8e4b14c640 1 -- 172.21.15.5:0/2099490827 >> 172.21.15.5:0/2099490827 conn(0x7f8e440fc790 msgr2=0x7f8e4406a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.725+0000 7f8e4b14c640 1 -- 172.21.15.5:0/2099490827 shutdown_connections 2024-09-11T22:29:41.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:41.725+0000 7f8e4b14c640 1 -- 172.21.15.5:0/2099490827 wait complete. 2024-09-11T22:29:41.735 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:41.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:41 smithi159 ceph-mon[35534]: pgmap v233: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-09-11T22:29:41.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:41 smithi172 ceph-mon[35461]: pgmap v233: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-09-11T22:29:42.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 7.89 MB/sec execute 240 sec latency 15329.777 ms 2024-09-11T22:29:42.576 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 75bd8d2f --release --await' 2024-09-11T22:29:42.577 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 75bd8d2f --release --await 2024-09-11T22:29:42.601 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:42 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2099490827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:29:42.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:42 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2099490827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:29:42.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:42 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2099490827' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:29:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.122+0000 7fc0e5828640 1 -- 172.21.15.5:0/2650900833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e0106a80 msgr2=0x7fc0e0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.122+0000 7fc0e5828640 1 --2- 172.21.15.5:0/2650900833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e0106a80 0x7fc0e0106e60 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0009930 tx=0x7fc0d002a420 comp rx=0 tx=0).stop 2024-09-11T22:29:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.122+0000 7fc0e5828640 1 -- 172.21.15.5:0/2650900833 shutdown_connections 2024-09-11T22:29:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.122+0000 7fc0e5828640 1 --2- 172.21.15.5:0/2650900833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0e0101420 0x7fc0e010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.122+0000 7fc0e5828640 1 --2- 172.21.15.5:0/2650900833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0e0100a80 0x7fc0e0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.122+0000 7fc0e5828640 1 --2- 172.21.15.5:0/2650900833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e0106a80 0x7fc0e0106e60 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.122+0000 7fc0e5828640 1 -- 172.21.15.5:0/2650900833 >> 172.21.15.5:0/2650900833 conn(0x7fc0e00fc7a0 msgr2=0x7fc0e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.122+0000 7fc0e5828640 1 -- 172.21.15.5:0/2650900833 shutdown_connections 2024-09-11T22:29:43.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.122+0000 7fc0e5828640 1 -- 172.21.15.5:0/2650900833 wait complete. 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0e5828640 1 Processor -- start 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0e5828640 1 -- start start 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0e5828640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0e0100a80 0x7fc0e0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0e5828640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0e0101420 0x7fc0e0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0e5828640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e0106a80 0x7fc0e019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0e5828640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0110ea0 con 0x7fc0e0106a80 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0e5828640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0110d20 con 0x7fc0e0100a80 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0e5828640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e0111020 con 0x7fc0e0101420 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e0106a80 0x7fc0e019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e0106a80 0x7fc0e019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54090/0 (socket says 172.21.15.5:54090) 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.123+0000 7fc0df7fe640 1 -- 172.21.15.5:0/2058971043 learned_addr learned my addr 172.21.15.5:0/2058971043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:43.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0d7fff640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0e0101420 0x7fc0e0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0deffd640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0e0100a80 0x7fc0e0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0df7fe640 1 -- 172.21.15.5:0/2058971043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0e0101420 msgr2=0x7fc0e0198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0e0101420 0x7fc0e0198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0df7fe640 1 -- 172.21.15.5:0/2058971043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0e0100a80 msgr2=0x7fc0e0198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0e0100a80 0x7fc0e0198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0df7fe640 1 -- 172.21.15.5:0/2058971043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e019d4d0 con 0x7fc0e0106a80 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e0106a80 0x7fc0e019cdf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800cce0 tx=0x7fc0d800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0e4826640 1 -- 172.21.15.5:0/2058971043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d800f450 con 0x7fc0e0106a80 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0e4826640 1 -- 172.21.15.5:0/2058971043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d800fad0 con 0x7fc0e0106a80 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0e5828640 1 -- 172.21.15.5:0/2058971043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e019d7c0 con 0x7fc0e0106a80 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0e4826640 1 -- 172.21.15.5:0/2058971043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8018c10 con 0x7fc0e0106a80 2024-09-11T22:29:43.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.124+0000 7fc0e5828640 1 -- 172.21.15.5:0/2058971043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e01a50a0 con 0x7fc0e0106a80 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.126+0000 7fc0e4826640 1 -- 172.21.15.5:0/2058971043 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc0d800f5f0 con 0x7fc0e0106a80 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0e7c01740 1 Processor -- start 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0e7c01740 1 -- start start 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2378f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0e4826640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc0cc0a2d70 0x7fc0cc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 0x5626a2383800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2386130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626a22de280 con 0x5626a2383d40 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5626a2285b10 con 0x5626a2379480 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5626a22da240 con 0x5626a2378b60 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2386130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0deffd640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc0cc0a2d70 0x7fc0cc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2386130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54106/0 (socket says 172.21.15.5:54106) 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0df7fe640 1 -- 172.21.15.5:0/526556583 learned_addr learned my addr 172.21.15.5:0/526556583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0e4826640 1 -- 172.21.15.5:0/2058971043 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc0d80cfbe0 con 0x7fc0e0106a80 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.127+0000 7fc0d7fff640 1 --2- 172.21.15.5:0/526556583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2378f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0d7fff640 1 --2- 172.21.15.5:0/526556583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 0x5626a2383800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0df7fe640 1 -- 172.21.15.5:0/526556583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 msgr2=0x5626a2378f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/526556583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2378f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0df7fe640 1 -- 172.21.15.5:0/526556583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 msgr2=0x5626a2383800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/526556583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 0x5626a2383800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0df7fe640 1 -- 172.21.15.5:0/526556583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626a22921f0 con 0x5626a2383d40 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0d7fff640 1 --2- 172.21.15.5:0/526556583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 0x5626a2383800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0deffd640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc0cc0a2d70 0x7fc0cc0a5230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0000c00 tx=0x7fc0d0031490 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/526556583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2386130 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800ab20 tx=0x7fc0d801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed55663d4e0e8a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0d5ffb640 1 -- 172.21.15.5:0/526556583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d809b9c0 con 0x5626a2383d40 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0d5ffb640 1 -- 172.21.15.5:0/526556583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d809c760 con 0x5626a2383d40 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.128+0000 7fc0d5ffb640 1 -- 172.21.15.5:0/526556583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80b5260 con 0x5626a2383d40 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 -- 172.21.15.5:0/526556583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 msgr2=0x5626a2386130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/526556583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2386130 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800ab20 tx=0x7fc0d801baf0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 -- 172.21.15.5:0/526556583 shutdown_connections 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/526556583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2386130 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/526556583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 0x5626a2383800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/526556583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2378f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 -- 172.21.15.5:0/526556583 >> 172.21.15.5:0/526556583 conn(0x5626a23750f0 msgr2=0x5626a23754d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 -- 172.21.15.5:0/526556583 shutdown_connections 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 -- 172.21.15.5:0/526556583 wait complete. 2024-09-11T22:29:43.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 -- start start 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.129+0000 7fc0e7c01740 1 Processor -- start 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2378b60 0x5626a2380c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2379480 0x5626a2381190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a23816d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626a2386910 con 0x5626a2383d40 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5626a2298f10 con 0x5626a2378b60 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5626a2386a90 con 0x5626a2379480 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0e7c01740 10 client.0 _ll_register_callbacks cb 0x5626a2261a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a23816d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a23816d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54116/0 (socket says 172.21.15.5:54116) 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0df7fe640 1 -- 172.21.15.5:0/4285470135 learned_addr learned my addr 172.21.15.5:0/4285470135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0deffd640 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2378b60 0x5626a2380c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0d7fff640 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2379480 0x5626a2381190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0df7fe640 1 -- 172.21.15.5:0/4285470135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2379480 msgr2=0x5626a2381190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2379480 0x5626a2381190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0df7fe640 1 -- 172.21.15.5:0/4285470135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2378b60 msgr2=0x5626a2380c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2378b60 0x5626a2380c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0df7fe640 1 -- 172.21.15.5:0/4285470135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626a204f6a0 con 0x5626a2383d40 2024-09-11T22:29:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0deffd640 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2378b60 0x5626a2380c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a23816d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8098a30 tx=0x7fc0d809b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25d892ccc3f26b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0beffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0beffd640 1 -- 172.21.15.5:0/4285470135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d80b5440 con 0x5626a2383d40 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0beffd640 1 -- 172.21.15.5:0/4285470135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc0d80b0580 con 0x5626a2383d40 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.130+0000 7fc0e7c01740 1 -- 172.21.15.5:0/4285470135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d809c760 con 0x5626a2383d40 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0beffd640 1 -- 172.21.15.5:0/4285470135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc0d80b5020 con 0x5626a2383d40 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0e7c01740 10 client.17766 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0e7c01740 1 -- 172.21.15.5:0/4285470135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5626a2392540 con 0x5626a2383d40 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0beffd640 1 -- 172.21.15.5:0/4285470135 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d809c760 con 0x5626a2383d40 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0beffd640 1 -- 172.21.15.5:0/4285470135 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc0d80cd510 con 0x5626a2383d40 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0e7c01740 10 client.17766 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0e7c01740 10 client.17766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0e7c01740 20 client.17766 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5626a2393890 0x5626a2395c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0e7c01740 4 client.17766 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0e7c01740 1 -- 172.21.15.5:0/4285470135 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626a22eb7f0 con 0x5626a2393890 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.131+0000 7fc0deffd640 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5626a2393890 0x5626a2395c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.132+0000 7fc0beffd640 10 client.17766 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.132+0000 7fc0deffd640 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5626a2393890 0x5626a2395c80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf84af98e2397294 server_cookie=8eee2423ad56faea in_seq=0 out_seq=0 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.132+0000 7fc0beffd640 1 -- 172.21.15.5:0/4285470135 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5626a22eb7f0 con 0x5626a2393890 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.132+0000 7fc0beffd640 10 client.17766 handle_command_reply: tid=0 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 client.17766 shutdown 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 -- 172.21.15.5:0/4285470135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 msgr2=0x5626a23816d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a23816d0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc0d8098a30 tx=0x7fc0d809b9c0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 -- 172.21.15.5:0/4285470135 shutdown_connections 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5626a2393890 0x5626a2395c80 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a23816d0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2379480 0x5626a2381190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/4285470135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2378b60 0x5626a2380c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0beffd640 0 client.17766 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 -- 172.21.15.5:0/4285470135 >> 172.21.15.5:0/4285470135 conn(0x5626a23750f0 msgr2=0x5626a237ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 -- 172.21.15.5:0/4285470135 shutdown_connections 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 1 -- 172.21.15.5:0/4285470135 wait complete. 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.133+0000 7fc0e7c01740 20 client.17766 trim_cache size 0 max 16384 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.150+0000 7fc0e7c01740 1 Processor -- start 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.150+0000 7fc0e7c01740 1 -- start start 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2381520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2379480 0x5626a2398ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2383d40 0x5626a23991e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01ada40 con 0x5626a2379480 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01af580 con 0x5626a2383d40 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01ae110 con 0x5626a2378b60 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2379480 0x5626a2398ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2379480 0x5626a2398ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54122/0 (socket says 172.21.15.5:54122) 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0d7fff640 1 -- 172.21.15.5:0/3961672090 learned_addr learned my addr 172.21.15.5:0/3961672090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0deffd640 1 --2- 172.21.15.5:0/3961672090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2381520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/3961672090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2383d40 0x5626a23991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0d7fff640 1 -- 172.21.15.5:0/3961672090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 msgr2=0x5626a2381520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0d7fff640 1 --2- 172.21.15.5:0/3961672090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2381520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0d7fff640 1 -- 172.21.15.5:0/3961672090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2383d40 msgr2=0x5626a23991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0d7fff640 1 --2- 172.21.15.5:0/3961672090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2383d40 0x5626a23991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.151+0000 7fc0d7fff640 1 -- 172.21.15.5:0/3961672090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626a23761d0 con 0x5626a2379480 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.152+0000 7fc0d7fff640 1 --2- 172.21.15.5:0/3961672090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2379480 0x5626a2398ca0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8006e10 tx=0x7fc0c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584fd0d45ef07c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.152+0000 7fc0d57fa640 1 -- 172.21.15.5:0/3961672090 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c801b020 con 0x5626a2379480 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.152+0000 7fc0d57fa640 1 -- 172.21.15.5:0/3961672090 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c8014e00 con 0x5626a2379480 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.152+0000 7fc0d57fa640 1 -- 172.21.15.5:0/3961672090 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80163d0 con 0x5626a2379480 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.152+0000 7fc0e7c01740 1 -- 172.21.15.5:0/3961672090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2379480 msgr2=0x5626a2398ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.152+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/3961672090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2379480 0x5626a2398ca0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8006e10 tx=0x7fc0c80046c0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.152+0000 7fc0e7c01740 1 -- 172.21.15.5:0/3961672090 shutdown_connections 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.152+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/3961672090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2383d40 0x5626a23991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.152+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/3961672090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2379480 0x5626a2398ca0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.152+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/3961672090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2381520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.153+0000 7fc0e7c01740 1 -- 172.21.15.5:0/3961672090 >> 172.21.15.5:0/3961672090 conn(0x5626a23750f0 msgr2=0x5626a2382bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.153+0000 7fc0e7c01740 1 -- 172.21.15.5:0/3961672090 shutdown_connections 2024-09-11T22:29:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.153+0000 7fc0e7c01740 1 -- 172.21.15.5:0/3961672090 wait complete. 2024-09-11T22:29:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.153+0000 7fc0e7c01740 1 -- start start 2024-09-11T22:29:43.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.153+0000 7fc0e7c01740 1 Processor -- start 2024-09-11T22:29:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2398cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 0x5626a2399200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0e7c01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2393c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01aeeb0 con 0x5626a2383d40 2024-09-11T22:29:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01ae7e0 con 0x5626a2379480 2024-09-11T22:29:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0e7c01740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e01aba20 con 0x5626a2378b60 2024-09-11T22:29:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0e7c01740 10 client.0 _ll_register_callbacks cb 0x7fc0c800a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2393c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2393c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54136/0 (socket says 172.21.15.5:54136) 2024-09-11T22:29:43.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0df7fe640 1 -- 172.21.15.5:0/1998633114 learned_addr learned my addr 172.21.15.5:0/1998633114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0d7fff640 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 0x5626a2399200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0deffd640 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2398cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0df7fe640 1 -- 172.21.15.5:0/1998633114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 msgr2=0x5626a2398cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2398cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0df7fe640 1 -- 172.21.15.5:0/1998633114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 msgr2=0x5626a2399200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 0x5626a2399200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0df7fe640 1 -- 172.21.15.5:0/1998633114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626a2394230 con 0x5626a2383d40 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0df7fe640 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2393c70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80984c0 tx=0x7fc0d800f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206c6cd8db0e3a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0beffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.154+0000 7fc0beffd640 1 -- 172.21.15.5:0/1998633114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d809be30 con 0x5626a2383d40 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0beffd640 1 -- 172.21.15.5:0/1998633114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fc0d80851a0 con 0x5626a2383d40 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0e7c01740 1 -- 172.21.15.5:0/1998633114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0c8014e00 con 0x5626a2383d40 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0beffd640 1 -- 172.21.15.5:0/1998633114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc0d807f050 con 0x5626a2383d40 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0e7c01740 10 client.17772 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0e7c01740 1 -- 172.21.15.5:0/1998633114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5626a2402ed0 con 0x5626a2383d40 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0beffd640 1 -- 172.21.15.5:0/1998633114 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c8014e00 con 0x5626a2383d40 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0beffd640 1 -- 172.21.15.5:0/1998633114 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc0d809b4b0 con 0x5626a2383d40 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0e7c01740 10 client.17772 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0e7c01740 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0e7c01740 20 client.17772 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5626a2403400 0x5626a24057f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0e7c01740 4 client.17772 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.155+0000 7fc0e7c01740 1 -- 172.21.15.5:0/1998633114 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5626a22e5d00 con 0x5626a2403400 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.156+0000 7fc0deffd640 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5626a2403400 0x5626a24057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.156+0000 7fc0deffd640 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5626a2403400 0x5626a24057f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=890946545bada863 server_cookie=24634415d0895756 in_seq=0 out_seq=0 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.156+0000 7fc0beffd640 10 client.17772 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.157+0000 7fc0beffd640 1 -- 172.21.15.5:0/1998633114 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5626a22e5d00 con 0x5626a2403400 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.157+0000 7fc0beffd640 10 client.17772 handle_command_reply: tid=0 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.157+0000 7fc0e7c01740 1 client.17772 shutdown 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.157+0000 7fc0e7c01740 1 -- 172.21.15.5:0/1998633114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 msgr2=0x5626a2393c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:43.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.157+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2393c70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80984c0 tx=0x7fc0d800f970 comp rx=0 tx=0).stop 2024-09-11T22:29:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.157+0000 7fc0e7c01740 1 -- 172.21.15.5:0/1998633114 shutdown_connections 2024-09-11T22:29:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.157+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5626a2403400 0x5626a24057f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.157+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626a2383d40 0x5626a2393c70 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.158+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626a2379480 0x5626a2399200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.158+0000 7fc0e7c01740 1 --2- 172.21.15.5:0/1998633114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5626a2378b60 0x5626a2398cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.158+0000 7fc0e7c01740 1 -- 172.21.15.5:0/1998633114 >> 172.21.15.5:0/1998633114 conn(0x5626a23750f0 msgr2=0x5626a238ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.158+0000 7fc0beffd640 0 client.17772 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.158+0000 7fc0e7c01740 1 -- 172.21.15.5:0/1998633114 shutdown_connections 2024-09-11T22:29:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.158+0000 7fc0e7c01740 1 -- 172.21.15.5:0/1998633114 wait complete. 2024-09-11T22:29:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.158+0000 7fc0e7c01740 20 client.17772 trim_cache size 0 max 16384 2024-09-11T22:29:43.176 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:29:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:29:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "4e525aac-35f7-4e3a-a8e7-9502ace711c5", 2024-09-11T22:29:43.176 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:29:43.176 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 -- 172.21.15.5:0/2058971043 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc0cc0a2d70 msgr2=0x7fc0cc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc0cc0a2d70 0x7fc0cc0a5230 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0000c00 tx=0x7fc0d0031490 comp rx=0 tx=0).stop 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 -- 172.21.15.5:0/2058971043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e0106a80 msgr2=0x7fc0e019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e0106a80 0x7fc0e019cdf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800cce0 tx=0x7fc0d800a620 comp rx=0 tx=0).stop 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 -- 172.21.15.5:0/2058971043 shutdown_connections 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e0106a80 0x7fc0e019cdf0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0e0101420 0x7fc0e0198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc0cc0a2d70 0x7fc0cc0a5230 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 --2- 172.21.15.5:0/2058971043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0e0100a80 0x7fc0e0198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 -- 172.21.15.5:0/2058971043 >> 172.21.15.5:0/2058971043 conn(0x7fc0e00fc7a0 msgr2=0x7fc0e006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 -- 172.21.15.5:0/2058971043 shutdown_connections 2024-09-11T22:29:43.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:43.159+0000 7fc0e5828640 1 -- 172.21.15.5:0/2058971043 wait complete. 2024-09-11T22:29:43.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 7.86 MB/sec execute 241 sec latency 16329.849 ms 2024-09-11T22:29:43.424 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:43.475 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:29:43 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:29:43.170+0000 7f54cdab7640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...a,head] /volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{1=1,2=1,3=1,4=1} v115619 ap=3 RANDEPHEMERALPIN f(v3301 m2024-09-11T22:29:26.297811+0000 3=3+0) n(v36 rc2024-09-11T22:29:26.297811+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:580 nref=3)) caps={24841=pAsLsXsFs/pAsLsXsFsx@6720} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x556c7fef9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v2606>,<0x10000000008/~dmtmp v158>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v14>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v30>,<0x10000000001/sv_1 v53>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v115619>,<0x10000000008/~dmtmp v7235>,<0x10000000007/client0 v603>,<0x10000000006/clients v530>,<0x10000000005/tmp v529>,<0x100000001fb/client.0 v534>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v540>,<0x10000000001/sv_1 v556>,<0x10000000000/qa v538>,<0x1/volumes v518>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T22:29:25.757823+0000 9=9+0)","ondisk_value.rstat":"n(v0 rc2024-09-11T22:29:26.297811+0000 b780288 4=3+1)","memory_value.dirstat":"f(v3301 m2024-09-11T22:29:26.297811+0000 3=3+0)","memory_value.rstat":"n(v36 rc2024-09-11T22:29:26.297811+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-11T22:29:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:43 smithi005 ceph-mon[30943]: pgmap v234: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-09-11T22:29:43.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:43 smithi159 ceph-mon[35534]: pgmap v234: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-09-11T22:29:43.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:43 smithi172 ceph-mon[35461]: pgmap v234: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 2.3 MiB/s wr, 134 op/s 2024-09-11T22:29:44.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:29:44.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563065 7.82 MB/sec execute 242 sec latency 17329.883 ms 2024-09-11T22:29:44.576 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:44 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:29:44.576 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:44 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:29:44.577 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:44 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:29:44.577 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:44 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:29:44.577 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:44 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:29:44.577 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:44 smithi005 ceph-mon[30943]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:29:44.577 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:44 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:29:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 -- 172.21.15.5:0/2524357553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd20074f40 msgr2=0x7fdd2010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 --2- 172.21.15.5:0/2524357553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd20074f40 0x7fdd2010de80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009900 tx=0x7fdd1002a480 comp rx=0 tx=0).stop 2024-09-11T22:29:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 -- 172.21.15.5:0/2524357553 shutdown_connections 2024-09-11T22:29:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 --2- 172.21.15.5:0/2524357553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd2010e3c0 0x7fdd20110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 --2- 172.21.15.5:0/2524357553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd20074f40 0x7fdd2010de80 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 --2- 172.21.15.5:0/2524357553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd20074620 0x7fdd20074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 -- 172.21.15.5:0/2524357553 >> 172.21.15.5:0/2524357553 conn(0x7fdd200fd8c0 msgr2=0x7fdd200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 -- 172.21.15.5:0/2524357553 shutdown_connections 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 -- 172.21.15.5:0/2524357553 wait complete. 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 Processor -- start 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.590+0000 7fdd27dad640 1 -- start start 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd27dad640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd20074620 0x7fdd201a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd27dad640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd20074f40 0x7fdd201a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd27dad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd2010e3c0 0x7fdd2019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd27dad640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20114280 con 0x7fdd2010e3c0 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd27dad640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20114100 con 0x7fdd20074f40 2024-09-11T22:29:44.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd27dad640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd20114400 con 0x7fdd20074620 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd2634a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd2010e3c0 0x7fdd2019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd2634a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd2010e3c0 0x7fdd2019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54156/0 (socket says 172.21.15.5:54156) 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd2634a640 1 -- 172.21.15.5:0/3622416383 learned_addr learned my addr 172.21.15.5:0/3622416383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd25348640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd20074f40 0x7fdd201a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd2634a640 1 -- 172.21.15.5:0/3622416383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd20074620 msgr2=0x7fdd201a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd2634a640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd20074620 0x7fdd201a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd2634a640 1 -- 172.21.15.5:0/3622416383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd20074f40 msgr2=0x7fdd201a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd2634a640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd20074f40 0x7fdd201a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd2634a640 1 -- 172.21.15.5:0/3622416383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd2019acf0 con 0x7fdd2010e3c0 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd2634a640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd2010e3c0 0x7fdd2019a5c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800b970 tx=0x7fdd1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd16ffd640 1 -- 172.21.15.5:0/3622416383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18019070 con 0x7fdd2010e3c0 2024-09-11T22:29:44.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.591+0000 7fdd16ffd640 1 -- 172.21.15.5:0/3622416383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd1800cb40 con 0x7fdd2010e3c0 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.592+0000 7fdd16ffd640 1 -- 172.21.15.5:0/3622416383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18007650 con 0x7fdd2010e3c0 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.592+0000 7fdd27dad640 1 -- 172.21.15.5:0/3622416383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2019afe0 con 0x7fdd2010e3c0 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.592+0000 7fdd27dad640 1 -- 172.21.15.5:0/3622416383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd201a72d0 con 0x7fdd2010e3c0 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.594+0000 7fdd16ffd640 1 -- 172.21.15.5:0/3622416383 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fdd1801c070 con 0x7fdd2010e3c0 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.594+0000 7fdd2a186740 1 Processor -- start 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.594+0000 7fdd2a186740 1 -- start start 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.594+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b82640e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826418d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b82641b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd2a186740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b826373620 con 0x55b82640e060 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd2a186740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b82631ae50 con 0x55b826419240 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd2a186740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b82636f5e0 con 0x55b82640e980 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd16ffd640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd0c0a2cf0 0x7fdd0c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd25348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b82640e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd25348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b82640e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54166/0 (socket says 172.21.15.5:54166) 2024-09-11T22:29:44.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd25348640 1 -- 172.21.15.5:0/1885968216 learned_addr learned my addr 172.21.15.5:0/1885968216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:44.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd2634a640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd0c0a2cf0 0x7fdd0c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd25b49640 1 --2- 172.21.15.5:0/1885968216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826418d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd16ffd640 1 -- 172.21.15.5:0/3622416383 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fdd180cea20 con 0x7fdd2010e3c0 2024-09-11T22:29:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd25348640 1 -- 172.21.15.5:0/1885968216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 msgr2=0x55b826418d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd25348640 1 --2- 172.21.15.5:0/1885968216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826418d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd25348640 1 -- 172.21.15.5:0/1885968216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 msgr2=0x55b82641b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd25348640 1 --2- 172.21.15.5:0/1885968216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b82641b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd25348640 1 -- 172.21.15.5:0/1885968216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82630bd40 con 0x55b82640e060 2024-09-11T22:29:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd2634a640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd0c0a2cf0 0x7fdd0c0a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd180cc2f0 tx=0x7fdd18069f40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.595+0000 7fdd25348640 1 --2- 172.21.15.5:0/1885968216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b82640e440 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009230 tx=0x7fdd10004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d1c4b4f44c30b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:44.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.596+0000 7fdd0bfff640 1 -- 172.21.15.5:0/1885968216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10041070 con 0x55b82640e060 2024-09-11T22:29:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.596+0000 7fdd0bfff640 1 -- 172.21.15.5:0/1885968216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd10005500 con 0x55b82640e060 2024-09-11T22:29:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.596+0000 7fdd0bfff640 1 -- 172.21.15.5:0/1885968216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10035050 con 0x55b82640e060 2024-09-11T22:29:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.596+0000 7fdd2a186740 1 -- 172.21.15.5:0/1885968216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 msgr2=0x55b82640e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.596+0000 7fdd2a186740 1 --2- 172.21.15.5:0/1885968216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b82640e440 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009230 tx=0x7fdd10004180 comp rx=0 tx=0).stop 2024-09-11T22:29:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.596+0000 7fdd2a186740 1 -- 172.21.15.5:0/1885968216 shutdown_connections 2024-09-11T22:29:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.596+0000 7fdd2a186740 1 --2- 172.21.15.5:0/1885968216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b82641b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.596+0000 7fdd2a186740 1 --2- 172.21.15.5:0/1885968216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826418d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.596+0000 7fdd2a186740 1 --2- 172.21.15.5:0/1885968216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b82640e440 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.596+0000 7fdd2a186740 1 -- 172.21.15.5:0/1885968216 >> 172.21.15.5:0/1885968216 conn(0x55b82640a5f0 msgr2=0x55b82640a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.597+0000 7fdd2a186740 1 -- 172.21.15.5:0/1885968216 shutdown_connections 2024-09-11T22:29:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.597+0000 7fdd2a186740 1 -- 172.21.15.5:0/1885968216 wait complete. 2024-09-11T22:29:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.597+0000 7fdd2a186740 1 -- start start 2024-09-11T22:29:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.597+0000 7fdd2a186740 1 Processor -- start 2024-09-11T22:29:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.597+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b826416130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.597+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826416670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.597+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b826416bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd2a186740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b82641be10 con 0x55b82640e060 2024-09-11T22:29:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd2a186740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b8262fbf10 con 0x55b826419240 2024-09-11T22:29:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd2a186740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b82641bf90 con 0x55b82640e980 2024-09-11T22:29:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd2a186740 10 client.0 _ll_register_callbacks cb 0x55b82631fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b826416130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b826416130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54168/0 (socket says 172.21.15.5:54168) 2024-09-11T22:29:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25348640 1 -- 172.21.15.5:0/4204353945 learned_addr learned my addr 172.21.15.5:0/4204353945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25b49640 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826416670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25348640 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b826416bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25348640 1 -- 172.21.15.5:0/4204353945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 msgr2=0x55b826416670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25348640 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826416670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25348640 1 -- 172.21.15.5:0/4204353945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 msgr2=0x55b826416bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25348640 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b826416bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25348640 1 -- 172.21.15.5:0/4204353945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b826082580 con 0x55b82640e060 2024-09-11T22:29:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd25348640 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b826416130 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fdd10000c00 tx=0x7fdd10004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54592a7c028518f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd097fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd097fa640 1 -- 172.21.15.5:0/4204353945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10035040 con 0x55b82640e060 2024-09-11T22:29:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd097fa640 1 -- 172.21.15.5:0/4204353945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fdd100317b0 con 0x55b82640e060 2024-09-11T22:29:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.598+0000 7fdd2a186740 1 -- 172.21.15.5:0/4204353945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd10005500 con 0x55b82640e060 2024-09-11T22:29:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd097fa640 1 -- 172.21.15.5:0/4204353945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdd10041040 con 0x55b82640e060 2024-09-11T22:29:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd2a186740 10 client.17784 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd2a186740 1 -- 172.21.15.5:0/4204353945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b82614c960 con 0x55b82640e060 2024-09-11T22:29:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd097fa640 1 -- 172.21.15.5:0/4204353945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd10005500 con 0x55b82640e060 2024-09-11T22:29:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd097fa640 1 -- 172.21.15.5:0/4204353945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fdd10003cf0 con 0x55b82640e060 2024-09-11T22:29:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd2a186740 10 client.17784 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd2a186740 10 client.17784 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:29:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd2a186740 20 client.17784 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd2a186740 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b826428c70 0x55b82642b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd2a186740 4 client.17784 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd2a186740 1 -- 172.21.15.5:0/4204353945 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b826380b90 con 0x55b826428c70 2024-09-11T22:29:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.599+0000 7fdd25b49640 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b826428c70 0x55b82642b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.600+0000 7fdd25b49640 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b826428c70 0x55b82642b060 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=112897fb3b81f474 server_cookie=4177ff2fd4465bfb in_seq=0 out_seq=0 2024-09-11T22:29:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.600+0000 7fdd097fa640 10 client.17784 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.600+0000 7fdd097fa640 1 -- 172.21.15.5:0/4204353945 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b826380b90 con 0x55b826428c70 2024-09-11T22:29:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.600+0000 7fdd097fa640 10 client.17784 handle_command_reply: tid=0 2024-09-11T22:29:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 client.17784 shutdown 2024-09-11T22:29:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 -- 172.21.15.5:0/4204353945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 msgr2=0x55b826416130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b826416130 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fdd10000c00 tx=0x7fdd10004d70 comp rx=0 tx=0).stop 2024-09-11T22:29:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 -- 172.21.15.5:0/4204353945 shutdown_connections 2024-09-11T22:29:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b826428c70 0x55b82642b060 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b826416bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826416670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd097fa640 0 client.17784 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 --2- 172.21.15.5:0/4204353945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b826416130 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 -- 172.21.15.5:0/4204353945 >> 172.21.15.5:0/4204353945 conn(0x55b82640a5f0 msgr2=0x55b826410320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 -- 172.21.15.5:0/4204353945 shutdown_connections 2024-09-11T22:29:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 1 -- 172.21.15.5:0/4204353945 wait complete. 2024-09-11T22:29:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.601+0000 7fdd2a186740 20 client.17784 trim_cache size 0 max 16384 2024-09-11T22:29:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd2a186740 1 Processor -- start 2024-09-11T22:29:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd2a186740 1 -- start start 2024-09-11T22:29:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e060 0x55b826409640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b82640e980 0x55b826430630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b826419240 0x55b826430b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd2a186740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201afb50 con 0x55b826419240 2024-09-11T22:29:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd2a186740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201b1690 con 0x55b82640e980 2024-09-11T22:29:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd2a186740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201b0220 con 0x55b82640e060 2024-09-11T22:29:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd25348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b826419240 0x55b826430b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd25348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b826419240 0x55b826430b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54174/0 (socket says 172.21.15.5:54174) 2024-09-11T22:29:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd25348640 1 -- 172.21.15.5:0/405533889 learned_addr learned my addr 172.21.15.5:0/405533889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd25348640 1 --2- 172.21.15.5:0/405533889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e060 0x55b826409640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.619+0000 7fdd25b49640 1 --2- 172.21.15.5:0/405533889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b82640e980 0x55b826430630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.620+0000 7fdd25348640 1 -- 172.21.15.5:0/405533889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e060 msgr2=0x55b826409640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.620+0000 7fdd25348640 1 --2- 172.21.15.5:0/405533889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e060 0x55b826409640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.620+0000 7fdd25348640 1 -- 172.21.15.5:0/405533889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b82640e980 msgr2=0x55b826430630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:44.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.620+0000 7fdd25348640 1 --2- 172.21.15.5:0/405533889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b82640e980 0x55b826430630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.620+0000 7fdd25348640 1 -- 172.21.15.5:0/405533889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b82640e7b0 con 0x55b826419240 2024-09-11T22:29:44.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.620+0000 7fdd25348640 1 --2- 172.21.15.5:0/405533889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b826419240 0x55b826430b70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fdd1003d5d0 tx=0x7fdd10005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950969ede5efcd37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:44.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.620+0000 7fdd0b7fe640 1 -- 172.21.15.5:0/405533889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1004a070 con 0x55b826419240 2024-09-11T22:29:44.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.620+0000 7fdd0b7fe640 1 -- 172.21.15.5:0/405533889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd100040d0 con 0x55b826419240 2024-09-11T22:29:44.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.620+0000 7fdd0b7fe640 1 -- 172.21.15.5:0/405533889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10035040 con 0x55b826419240 2024-09-11T22:29:44.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.621+0000 7fdd2a186740 1 -- 172.21.15.5:0/405533889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b826419240 msgr2=0x55b826430b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:44.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.621+0000 7fdd2a186740 1 --2- 172.21.15.5:0/405533889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b826419240 0x55b826430b70 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fdd1003d5d0 tx=0x7fdd10005330 comp rx=0 tx=0).stop 2024-09-11T22:29:44.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.621+0000 7fdd2a186740 1 -- 172.21.15.5:0/405533889 shutdown_connections 2024-09-11T22:29:44.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.621+0000 7fdd2a186740 1 --2- 172.21.15.5:0/405533889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b826419240 0x55b826430b70 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.621+0000 7fdd2a186740 1 --2- 172.21.15.5:0/405533889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b82640e980 0x55b826430630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.621+0000 7fdd2a186740 1 --2- 172.21.15.5:0/405533889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e060 0x55b826409640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.621+0000 7fdd2a186740 1 -- 172.21.15.5:0/405533889 >> 172.21.15.5:0/405533889 conn(0x55b82640a5f0 msgr2=0x55b826408f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:44.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.621+0000 7fdd2a186740 1 -- 172.21.15.5:0/405533889 shutdown_connections 2024-09-11T22:29:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.621+0000 7fdd2a186740 1 -- 172.21.15.5:0/405533889 wait complete. 2024-09-11T22:29:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.621+0000 7fdd2a186740 1 -- start start 2024-09-11T22:29:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd2a186740 1 Processor -- start 2024-09-11T22:29:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b8264213f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826421930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd2a186740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b82641e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd2a186740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201b0fc0 con 0x55b82640e060 2024-09-11T22:29:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd2a186740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201b08f0 con 0x55b826419240 2024-09-11T22:29:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd2a186740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd201adb30 con 0x55b82640e980 2024-09-11T22:29:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd2a186740 10 client.0 _ll_register_callbacks cb 0x7fdd10004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd25348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b8264213f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd25348640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b8264213f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54182/0 (socket says 172.21.15.5:54182) 2024-09-11T22:29:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd25348640 1 -- 172.21.15.5:0/2321050214 learned_addr learned my addr 172.21.15.5:0/2321050214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.622+0000 7fdd25348640 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b82641e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd25b49640 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826421930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd25348640 1 -- 172.21.15.5:0/2321050214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 msgr2=0x55b826421930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd25348640 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826421930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd25348640 1 -- 172.21.15.5:0/2321050214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 msgr2=0x55b82641e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd25348640 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b82641e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd25348640 1 -- 172.21.15.5:0/2321050214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b82641e8f0 con 0x55b82640e060 2024-09-11T22:29:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd25b49640 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826421930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd25348640 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b8264213f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fdd1002a9a0 tx=0x7fdd100316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dd76018d69acc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd097fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd097fa640 1 -- 172.21.15.5:0/2321050214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10066060 con 0x55b82640e060 2024-09-11T22:29:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd097fa640 1 -- 172.21.15.5:0/2321050214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fdd10031790 con 0x55b82640e060 2024-09-11T22:29:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd2a186740 1 -- 172.21.15.5:0/2321050214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd100040d0 con 0x55b82640e060 2024-09-11T22:29:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd097fa640 1 -- 172.21.15.5:0/2321050214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdd1004a030 con 0x55b82640e060 2024-09-11T22:29:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd2a186740 10 client.17790 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd2a186740 1 -- 172.21.15.5:0/2321050214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b826494010 con 0x55b82640e060 2024-09-11T22:29:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.623+0000 7fdd097fa640 1 -- 172.21.15.5:0/2321050214 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd100040d0 con 0x55b82640e060 2024-09-11T22:29:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.624+0000 7fdd097fa640 1 -- 172.21.15.5:0/2321050214 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fdd100615f0 con 0x55b82640e060 2024-09-11T22:29:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.624+0000 7fdd2a186740 10 client.17790 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.624+0000 7fdd2a186740 10 client.17790 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:29:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.624+0000 7fdd2a186740 20 client.17790 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.624+0000 7fdd2a186740 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b8264946d0 0x55b826496ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.624+0000 7fdd2a186740 4 client.17790 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "75bd8d2f", "release": true, "await": true}] 2024-09-11T22:29:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.624+0000 7fdd2a186740 1 -- 172.21.15.5:0/2321050214 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "75bd8d2f", "release": true, "await": true}) -- 0x55b82637b0a0 con 0x55b8264946d0 2024-09-11T22:29:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.624+0000 7fdd25b49640 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b8264946d0 0x55b826496ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.625+0000 7fdd25b49640 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b8264946d0 0x55b826496ac0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3a135c78b21fb53 server_cookie=c070451913d420c in_seq=0 out_seq=0 2024-09-11T22:29:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.625+0000 7fdd097fa640 10 client.17790 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd097fa640 1 -- 172.21.15.5:0/2321050214 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b82637b0a0 con 0x55b8264946d0 2024-09-11T22:29:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd097fa640 10 client.17790 handle_command_reply: tid=0 2024-09-11T22:29:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd2a186740 1 client.17790 shutdown 2024-09-11T22:29:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd2a186740 1 -- 172.21.15.5:0/2321050214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 msgr2=0x55b8264213f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd2a186740 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b8264213f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fdd1002a9a0 tx=0x7fdd100316f0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd2a186740 1 -- 172.21.15.5:0/2321050214 shutdown_connections 2024-09-11T22:29:44.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd2a186740 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b8264946d0 0x55b826496ac0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd2a186740 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b826419240 0x55b82641e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd2a186740 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b82640e980 0x55b826421930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd2a186740 1 --2- 172.21.15.5:0/2321050214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b82640e060 0x55b8264213f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd097fa640 0 client.17790 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:44.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.636+0000 7fdd2a186740 1 -- 172.21.15.5:0/2321050214 >> 172.21.15.5:0/2321050214 conn(0x55b82640a5f0 msgr2=0x55b826409750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:44.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.637+0000 7fdd2a186740 1 -- 172.21.15.5:0/2321050214 shutdown_connections 2024-09-11T22:29:44.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.637+0000 7fdd2a186740 1 -- 172.21.15.5:0/2321050214 wait complete. 2024-09-11T22:29:44.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.637+0000 7fdd2a186740 20 client.17790 trim_cache size 0 max 16384 2024-09-11T22:29:44.654 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:29:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 40, 2024-09-11T22:29:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:29:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 28, 2024-09-11T22:29:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:29:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "75bd8d2f": { 2024-09-11T22:29:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 28, 2024-09-11T22:29:44.655 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:29:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:29:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:29:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:29:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:29:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:29:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 136.6, 2024-09-11T22:29:44.656 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:29:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:29:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:29:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:29:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:29:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:29:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:44.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:44.658 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:29:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 -- 172.21.15.5:0/3622416383 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd0c0a2cf0 msgr2=0x7fdd0c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd0c0a2cf0 0x7fdd0c0a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd180cc2f0 tx=0x7fdd18069f40 comp rx=0 tx=0).stop 2024-09-11T22:29:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 -- 172.21.15.5:0/3622416383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd2010e3c0 msgr2=0x7fdd2019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd2010e3c0 0x7fdd2019a5c0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800b970 tx=0x7fdd1800be30 comp rx=0 tx=0).stop 2024-09-11T22:29:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 -- 172.21.15.5:0/3622416383 shutdown_connections 2024-09-11T22:29:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd2010e3c0 0x7fdd2019a5c0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd0c0a2cf0 0x7fdd0c0a51b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd20074f40 0x7fdd201a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 --2- 172.21.15.5:0/3622416383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd20074620 0x7fdd201a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 -- 172.21.15.5:0/3622416383 >> 172.21.15.5:0/3622416383 conn(0x7fdd200fd8c0 msgr2=0x7fdd200ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 -- 172.21.15.5:0/3622416383 shutdown_connections 2024-09-11T22:29:44.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:44.638+0000 7fdd27dad640 1 -- 172.21.15.5:0/3622416383 wait complete. 2024-09-11T22:29:44.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:44 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:29:44.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:44 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:29:44.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:44 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:29:44.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:44 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:29:44.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:44 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:29:44.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:44 smithi159 ceph-mon[35534]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:29:44.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:44 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:29:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:44 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:29:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:44 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:29:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:44 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:29:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:44 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:29:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:44 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:29:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:44 smithi172 ceph-mon[35461]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:29:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:44 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:29:44.886 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:29:45.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 564903 7.84 MB/sec execute 243 sec latency 17641.739 ms 2024-09-11T22:29:45.556 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '75bd8d2f', total seconds elapsed: 21.6 2024-09-11T22:29:45.784 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:45 smithi005 ceph-mon[30943]: pgmap v235: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-09-11T22:29:45.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:45 smithi159 ceph-mon[35534]: pgmap v235: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-09-11T22:29:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:45 smithi172 ceph-mon[35461]: pgmap v235: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-09-11T22:29:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.031+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1126313954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b2c100e20 msgr2=0x7f6b2c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.031+0000 7f6b31b0e640 1 --2- 172.21.15.5:0/1126313954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b2c100e20 0x7f6b2c101200 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009900 tx=0x7f6b1c02a480 comp rx=0 tx=0).stop 2024-09-11T22:29:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.031+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1126313954 shutdown_connections 2024-09-11T22:29:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.031+0000 7f6b31b0e640 1 --2- 172.21.15.5:0/1126313954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c102ef0 0x7f6b2c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.031+0000 7f6b31b0e640 1 --2- 172.21.15.5:0/1126313954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b2c102570 0x7f6b2c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.031+0000 7f6b31b0e640 1 --2- 172.21.15.5:0/1126313954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b2c100e20 0x7f6b2c101200 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.031+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1126313954 >> 172.21.15.5:0/1126313954 conn(0x7f6b2c0fc7d0 msgr2=0x7f6b2c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.031+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1126313954 shutdown_connections 2024-09-11T22:29:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.031+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1126313954 wait complete. 2024-09-11T22:29:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b31b0e640 1 Processor -- start 2024-09-11T22:29:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b31b0e640 1 -- start start 2024-09-11T22:29:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b31b0e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c100e20 0x7f6b2c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b31b0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b2c102570 0x7f6b2c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b31b0e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b2c102ef0 0x7f6b2c19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b31b0e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c112080 con 0x7f6b2c102570 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b31b0e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c111f00 con 0x7f6b2c102ef0 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b31b0e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c112200 con 0x7f6b2c100e20 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b2c102570 0x7f6b2c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b2c102570 0x7f6b2c198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54196/0 (socket says 172.21.15.5:54196) 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b23fff640 1 -- 172.21.15.5:0/1799840129 learned_addr learned my addr 172.21.15.5:0/1799840129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b2c102ef0 0x7f6b2c19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c100e20 0x7f6b2c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b23fff640 1 -- 172.21.15.5:0/1799840129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c100e20 msgr2=0x7f6b2c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b23fff640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c100e20 0x7f6b2c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b23fff640 1 -- 172.21.15.5:0/1799840129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b2c102ef0 msgr2=0x7f6b2c19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b23fff640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b2c102ef0 0x7f6b2c19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b23fff640 1 -- 172.21.15.5:0/1799840129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2c19d470 con 0x7f6b2c102570 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.032+0000 7f6b23fff640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b2c102570 0x7f6b2c198a00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e9f0 tx=0x7f6b1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.033+0000 7f6b297fa640 1 -- 172.21.15.5:0/1799840129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400cd80 con 0x7f6b2c102570 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.033+0000 7f6b297fa640 1 -- 172.21.15.5:0/1799840129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b14010910 con 0x7f6b2c102570 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.033+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1799840129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2c19d760 con 0x7f6b2c102570 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.033+0000 7f6b297fa640 1 -- 172.21.15.5:0/1799840129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14018990 con 0x7f6b2c102570 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.033+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1799840129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b2c1a5040 con 0x7f6b2c102570 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.035+0000 7f6b297fa640 1 -- 172.21.15.5:0/1799840129 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6b14010ac0 con 0x7f6b2c102570 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.035+0000 7f6b33ee7740 1 Processor -- start 2024-09-11T22:29:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b33ee7740 1 -- start start 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 0x5647fc1e0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 0x5647fc1e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647fc1372e0 con 0x5647fc1d58b0 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5647fc0dece0 con 0x5647fc1e0a90 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5647fc1332a0 con 0x5647fc1d61d0 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b2b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 0x5647fc1e0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1d5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54200/0 (socket says 172.21.15.5:54200) 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b2bfff640 1 -- 172.21.15.5:0/3115757178 learned_addr learned my addr 172.21.15.5:0/3115757178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b23fff640 1 --2- 172.21.15.5:0/3115757178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 0x5647fc1e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b2bfff640 1 -- 172.21.15.5:0/3115757178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 msgr2=0x5647fc1e0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/3115757178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 0x5647fc1e0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b2bfff640 1 -- 172.21.15.5:0/3115757178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 msgr2=0x5647fc1e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/3115757178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 0x5647fc1e2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b2bfff640 1 -- 172.21.15.5:0/3115757178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647fc0c3440 con 0x5647fc1d58b0 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/3115757178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 0x5647fc1e0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.036+0000 7f6b23fff640 1 --2- 172.21.15.5:0/3115757178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 0x5647fc1e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.037+0000 7f6b297fa640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6b180a2d70 0x7f6b180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.037+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/3115757178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1d5c90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009fd0 tx=0x7f6b240118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d86e60770b6b162 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.037+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6b180a2d70 0x7f6b180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.037+0000 7f6b297fa640 1 -- 172.21.15.5:0/1799840129 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6b140cf700 con 0x7f6b2c102570 2024-09-11T22:29:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.037+0000 7f6b21ffb640 1 -- 172.21.15.5:0/3115757178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24004d20 con 0x5647fc1d58b0 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.037+0000 7f6b21ffb640 1 -- 172.21.15.5:0/3115757178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b24004050 con 0x5647fc1d58b0 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.037+0000 7f6b21ffb640 1 -- 172.21.15.5:0/3115757178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24005280 con 0x5647fc1d58b0 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6b180a2d70 0x7f6b180a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009230 tx=0x7f6b1c0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b33ee7740 1 -- 172.21.15.5:0/3115757178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 msgr2=0x5647fc1d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/3115757178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1d5c90 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6b24009fd0 tx=0x7f6b240118d0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b33ee7740 1 -- 172.21.15.5:0/3115757178 shutdown_connections 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/3115757178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 0x5647fc1e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/3115757178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 0x5647fc1e0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/3115757178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1d5c90 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b33ee7740 1 -- 172.21.15.5:0/3115757178 >> 172.21.15.5:0/3115757178 conn(0x5647fc1d1e40 msgr2=0x5647fc1d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b33ee7740 1 -- 172.21.15.5:0/3115757178 shutdown_connections 2024-09-11T22:29:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b33ee7740 1 -- 172.21.15.5:0/3115757178 wait complete. 2024-09-11T22:29:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.038+0000 7f6b33ee7740 1 -- start start 2024-09-11T22:29:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b33ee7740 1 Processor -- start 2024-09-11T22:29:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d58b0 0x5647fc1e0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 0x5647fc1dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1e0a90 0x5647fc1ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647fc1e3660 con 0x5647fc1e0a90 2024-09-11T22:29:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5647fc0f2050 con 0x5647fc1d58b0 2024-09-11T22:29:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5647fc1e37e0 con 0x5647fc1d61d0 2024-09-11T22:29:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b33ee7740 10 client.0 _ll_register_callbacks cb 0x5647fc132160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b2b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d58b0 0x5647fc1e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b2b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d58b0 0x5647fc1e0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41860/0 (socket says 172.21.15.5:41860) 2024-09-11T22:29:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b2b7fe640 1 -- 172.21.15.5:0/383729211 learned_addr learned my addr 172.21.15.5:0/383729211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b23fff640 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 0x5647fc1dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.039+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1e0a90 0x5647fc1ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b23fff640 1 -- 172.21.15.5:0/383729211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d58b0 msgr2=0x5647fc1e0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b23fff640 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d58b0 0x5647fc1e0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b23fff640 1 -- 172.21.15.5:0/383729211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1e0a90 msgr2=0x5647fc1ddc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b23fff640 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1e0a90 0x5647fc1ddc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b23fff640 1 -- 172.21.15.5:0/383729211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647fbea83c0 con 0x5647fc1d61d0 2024-09-11T22:29:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1e0a90 0x5647fc1ddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d58b0 0x5647fc1e0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b23fff640 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 0x5647fc1dd6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6b140cce10 tx=0x7f6b1409bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff9f0059c128005b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b0affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:29:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b0affd640 1 -- 172.21.15.5:0/383729211 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1409bc90 con 0x5647fc1d61d0 2024-09-11T22:29:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b0affd640 1 -- 172.21.15.5:0/383729211 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6b140cfbb0 con 0x5647fc1d61d0 2024-09-11T22:29:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b33ee7740 1 -- 172.21.15.5:0/383729211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b24004050 con 0x5647fc1d61d0 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b0affd640 1 -- 172.21.15.5:0/383729211 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6b1409ff90 con 0x5647fc1d61d0 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b33ee7740 10 client.27070 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.040+0000 7f6b33ee7740 1 -- 172.21.15.5:0/383729211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5647fc1ef080 con 0x5647fc1d61d0 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b0affd640 1 -- 172.21.15.5:0/383729211 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b24004050 con 0x5647fc1d61d0 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b0affd640 1 -- 172.21.15.5:0/383729211 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f6b1409b0e0 con 0x5647fc1d61d0 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b33ee7740 10 client.27070 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b33ee7740 10 client.27070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b33ee7740 20 client.27070 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647fc1f0540 0x5647fc1f2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b33ee7740 4 client.27070 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:46.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b33ee7740 1 -- 172.21.15.5:0/383729211 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647fc144850 con 0x5647fc1f0540 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647fc1f0540 0x5647fc1f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b0affd640 10 client.27070 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.041+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647fc1f0540 0x5647fc1f2930 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83e2aee21f068627 server_cookie=c88e46ff477860eb in_seq=0 out_seq=0 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b0affd640 1 -- 172.21.15.5:0/383729211 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647fc144850 con 0x5647fc1f0540 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b0affd640 10 client.27070 handle_command_reply: tid=0 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 client.27070 shutdown 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 -- 172.21.15.5:0/383729211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 msgr2=0x5647fc1dd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 0x5647fc1dd6b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6b140cce10 tx=0x7f6b1409bb60 comp rx=0 tx=0).stop 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 -- 172.21.15.5:0/383729211 shutdown_connections 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647fc1f0540 0x5647fc1f2930 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1e0a90 0x5647fc1ddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d61d0 0x5647fc1dd6b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/383729211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d58b0 0x5647fc1e0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b0affd640 0 client.27070 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:46.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 -- 172.21.15.5:0/383729211 >> 172.21.15.5:0/383729211 conn(0x5647fc1d1e40 msgr2=0x5647fc1ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 -- 172.21.15.5:0/383729211 shutdown_connections 2024-09-11T22:29:46.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.042+0000 7f6b33ee7740 1 -- 172.21.15.5:0/383729211 wait complete. 2024-09-11T22:29:46.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.043+0000 7f6b33ee7740 20 client.27070 trim_cache size 0 max 16384 2024-09-11T22:29:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.062+0000 7f6b33ee7740 1 Processor -- start 2024-09-11T22:29:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.062+0000 7f6b33ee7740 1 -- start start 2024-09-11T22:29:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d58b0 0x5647fc1ec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d61d0 0x5647fc1f8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 0x5647fc1f8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c1ab710 con 0x5647fc1d61d0 2024-09-11T22:29:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c1ad250 con 0x5647fc1e0a90 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c1abde0 con 0x5647fc1d58b0 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d61d0 0x5647fc1f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d61d0 0x5647fc1f8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54212/0 (socket says 172.21.15.5:54212) 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b23fff640 1 -- 172.21.15.5:0/2771180975 learned_addr learned my addr 172.21.15.5:0/2771180975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/2771180975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d58b0 0x5647fc1ec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/2771180975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 0x5647fc1f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b23fff640 1 -- 172.21.15.5:0/2771180975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d58b0 msgr2=0x5647fc1ec8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b23fff640 1 --2- 172.21.15.5:0/2771180975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d58b0 0x5647fc1ec8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b23fff640 1 -- 172.21.15.5:0/2771180975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 msgr2=0x5647fc1f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b23fff640 1 --2- 172.21.15.5:0/2771180975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 0x5647fc1f8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b23fff640 1 -- 172.21.15.5:0/2771180975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647fc1d6000 con 0x5647fc1d61d0 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/2771180975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 0x5647fc1f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:29:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.063+0000 7f6b23fff640 1 --2- 172.21.15.5:0/2771180975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d61d0 0x5647fc1f8200 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6b140ba480 tx=0x7f6b140d3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bf16093b8655f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.064+0000 7f6b217fa640 1 -- 172.21.15.5:0/2771180975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1409bc90 con 0x5647fc1d61d0 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.064+0000 7f6b217fa640 1 -- 172.21.15.5:0/2771180975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b140a4850 con 0x5647fc1d61d0 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.064+0000 7f6b217fa640 1 -- 172.21.15.5:0/2771180975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b140979e0 con 0x5647fc1d61d0 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.064+0000 7f6b33ee7740 1 -- 172.21.15.5:0/2771180975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d61d0 msgr2=0x5647fc1f8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.064+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/2771180975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d61d0 0x5647fc1f8200 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6b140ba480 tx=0x7f6b140d3000 comp rx=0 tx=0).stop 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.064+0000 7f6b33ee7740 1 -- 172.21.15.5:0/2771180975 shutdown_connections 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.064+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/2771180975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1e0a90 0x5647fc1f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.064+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/2771180975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d61d0 0x5647fc1f8200 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.064+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/2771180975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1d58b0 0x5647fc1ec8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.064+0000 7f6b33ee7740 1 -- 172.21.15.5:0/2771180975 >> 172.21.15.5:0/2771180975 conn(0x5647fc1d1e40 msgr2=0x5647fc1d8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.065+0000 7f6b33ee7740 1 -- 172.21.15.5:0/2771180975 shutdown_connections 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.065+0000 7f6b33ee7740 1 -- 172.21.15.5:0/2771180975 wait complete. 2024-09-11T22:29:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.065+0000 7f6b33ee7740 1 -- start start 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.065+0000 7f6b33ee7740 1 Processor -- start 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d61d0 0x5647fc1cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b33ee7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1e0a90 0x5647fc1ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c1acb80 con 0x5647fc1d58b0 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c1ac4b0 con 0x5647fc1d61d0 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b33ee7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2c1a96f0 con 0x5647fc1e0a90 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b33ee7740 10 client.0 _ll_register_callbacks cb 0x7f6b1400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b2b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b2bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1e0a90 0x5647fc1ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b2b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54214/0 (socket says 172.21.15.5:54214) 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b2b7fe640 1 -- 172.21.15.5:0/4218539803 learned_addr learned my addr 172.21.15.5:0/4218539803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b23fff640 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d61d0 0x5647fc1cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b2bfff640 1 -- 172.21.15.5:0/4218539803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d61d0 msgr2=0x5647fc1cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d61d0 0x5647fc1cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b2bfff640 1 -- 172.21.15.5:0/4218539803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 msgr2=0x5647fc1cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b2bfff640 1 -- 172.21.15.5:0/4218539803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647fc1e7b20 con 0x5647fc1e0a90 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.066+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.067+0000 7f6b2bfff640 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1e0a90 0x5647fc1ce300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b2401b040 tx=0x7f6b24005820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1ebec9e6854662 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.067+0000 7f6b0affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.067+0000 7f6b0affd640 1 -- 172.21.15.5:0/4218539803 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b24005990 con 0x5647fc1e0a90 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.067+0000 7f6b33ee7740 1 -- 172.21.15.5:0/4218539803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b140a4850 con 0x5647fc1e0a90 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.067+0000 7f6b0affd640 1 -- 172.21.15.5:0/4218539803 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f6b2400f4e0 con 0x5647fc1e0a90 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.067+0000 7f6b0affd640 1 -- 172.21.15.5:0/4218539803 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6b2400f6e0 con 0x5647fc1e0a90 2024-09-11T22:29:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.067+0000 7f6b33ee7740 10 client.27076 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.067+0000 7f6b33ee7740 1 -- 172.21.15.5:0/4218539803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5647fc1e8d90 con 0x5647fc1e0a90 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b0affd640 1 -- 172.21.15.5:0/4218539803 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b140a4850 con 0x5647fc1e0a90 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b0affd640 1 -- 172.21.15.5:0/4218539803 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f6b24023820 con 0x5647fc1e0a90 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b33ee7740 10 client.27076 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b33ee7740 10 client.27076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b33ee7740 20 client.27076 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647fc1e9310 0x5647fc2619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b33ee7740 4 client.27076 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b33ee7740 1 -- 172.21.15.5:0/4218539803 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5647fc13ed60 con 0x5647fc1e9310 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647fc1e9310 0x5647fc2619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b0affd640 10 client.27076 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.068+0000 7f6b2b7fe640 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647fc1e9310 0x5647fc2619d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e3648cce30fc01 server_cookie=33b761e34a1a6e9c in_seq=0 out_seq=0 2024-09-11T22:29:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.078+0000 7f6b0affd640 1 -- 172.21.15.5:0/4218539803 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5647fc13ed60 con 0x5647fc1e9310 2024-09-11T22:29:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.078+0000 7f6b0affd640 10 client.27076 handle_command_reply: tid=0 2024-09-11T22:29:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.079+0000 7f6b33ee7740 1 client.27076 shutdown 2024-09-11T22:29:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.086+0000 7f6b33ee7740 1 -- 172.21.15.5:0/4218539803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1e0a90 msgr2=0x5647fc1ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.086+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1e0a90 0x5647fc1ce300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6b2401b040 tx=0x7f6b24005820 comp rx=0 tx=0).stop 2024-09-11T22:29:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.088+0000 7f6b33ee7740 1 -- 172.21.15.5:0/4218539803 shutdown_connections 2024-09-11T22:29:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.088+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647fc1e9310 0x5647fc2619d0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.088+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647fc1e0a90 0x5647fc1ce300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.088+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647fc1d61d0 0x5647fc1cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.088+0000 7f6b33ee7740 1 --2- 172.21.15.5:0/4218539803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647fc1d58b0 0x5647fc1cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.088+0000 7f6b0affd640 0 client.27076 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.088+0000 7f6b33ee7740 1 -- 172.21.15.5:0/4218539803 >> 172.21.15.5:0/4218539803 conn(0x5647fc1d1e40 msgr2=0x5647fc1d7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.089+0000 7f6b33ee7740 1 -- 172.21.15.5:0/4218539803 shutdown_connections 2024-09-11T22:29:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.089+0000 7f6b33ee7740 1 -- 172.21.15.5:0/4218539803 wait complete. 2024-09-11T22:29:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.089+0000 7f6b33ee7740 20 client.27076 trim_cache size 0 max 16384 2024-09-11T22:29:46.091 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:29:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:29:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:29:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "4e525aac-35f7-4e3a-a8e7-9502ace711c5": { 2024-09-11T22:29:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:29:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4e525aac-35f7-4e3a-a8e7-9502ace711c5", 2024-09-11T22:29:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:29:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:46.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:29:46.091 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:29:46.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.091+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1799840129 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6b180a2d70 msgr2=0x7f6b180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.091+0000 7f6b31b0e640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6b180a2d70 0x7f6b180a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009230 tx=0x7f6b1c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.091+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1799840129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b2c102570 msgr2=0x7f6b2c198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.091+0000 7f6b31b0e640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b2c102570 0x7f6b2c198a00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400e9f0 tx=0x7f6b1400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.091+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1799840129 shutdown_connections 2024-09-11T22:29:46.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.091+0000 7f6b31b0e640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6b180a2d70 0x7f6b180a5230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.091+0000 7f6b31b0e640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b2c102ef0 0x7f6b2c19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.091+0000 7f6b31b0e640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b2c102570 0x7f6b2c198a00 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.091+0000 7f6b31b0e640 1 --2- 172.21.15.5:0/1799840129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b2c100e20 0x7f6b2c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.091+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1799840129 >> 172.21.15.5:0/1799840129 conn(0x7f6b2c0fc7d0 msgr2=0x7f6b2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.092+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1799840129 shutdown_connections 2024-09-11T22:29:46.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:29:46.092+0000 7f6b31b0e640 1 -- 172.21.15.5:0/1799840129 wait complete. 2024-09-11T22:29:46.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 566324 7.84 MB/sec execute 244 sec latency 45.630 ms 2024-09-11T22:29:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.356+0000 7fd4a3b42640 1 -- 172.21.15.5:0/1003305537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd49c100fd0 msgr2=0x7fd49c10d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.356+0000 7fd4a3b42640 1 --2- 172.21.15.5:0/1003305537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd49c100fd0 0x7fd49c10d030 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd494009ca0 tx=0x7fd49402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.356+0000 7fd4a3b42640 1 -- 172.21.15.5:0/1003305537 shutdown_connections 2024-09-11T22:29:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.356+0000 7fd4a3b42640 1 --2- 172.21.15.5:0/1003305537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd49c100fd0 0x7fd49c10d030 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.356+0000 7fd4a3b42640 1 --2- 172.21.15.5:0/1003305537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd49c100630 0x7fd49c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.356+0000 7fd4a3b42640 1 --2- 172.21.15.5:0/1003305537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd49c106220 0x7fd49c106600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.356+0000 7fd4a3b42640 1 -- 172.21.15.5:0/1003305537 >> 172.21.15.5:0/1003305537 conn(0x7fd49c075ef0 msgr2=0x7fd49c076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.356+0000 7fd4a3b42640 1 -- 172.21.15.5:0/1003305537 shutdown_connections 2024-09-11T22:29:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.356+0000 7fd4a3b42640 1 -- 172.21.15.5:0/1003305537 wait complete. 2024-09-11T22:29:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.357+0000 7fd4a3b42640 1 Processor -- start 2024-09-11T22:29:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.357+0000 7fd4a3b42640 1 -- start start 2024-09-11T22:29:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a3b42640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd49c100630 0x7fd49c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a3b42640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd49c100fd0 0x7fd49c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a3b42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd49c106220 0x7fd49c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a3b42640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c111f40 con 0x7fd49c106220 2024-09-11T22:29:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a3b42640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c111dc0 con 0x7fd49c100fd0 2024-09-11T22:29:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a3b42640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c1120c0 con 0x7fd49c100630 2024-09-11T22:29:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a20df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd49c106220 0x7fd49c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a10dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd49c100fd0 0x7fd49c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a20df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd49c106220 0x7fd49c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54224/0 (socket says 172.21.15.5:54224) 2024-09-11T22:29:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a20df640 1 -- 172.21.15.5:0/500980646 learned_addr learned my addr 172.21.15.5:0/500980646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a18de640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd49c100630 0x7fd49c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a20df640 1 -- 172.21.15.5:0/500980646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd49c100630 msgr2=0x7fd49c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a20df640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd49c100630 0x7fd49c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a20df640 1 -- 172.21.15.5:0/500980646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd49c100fd0 msgr2=0x7fd49c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a20df640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd49c100fd0 0x7fd49c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.358+0000 7fd4a20df640 1 -- 172.21.15.5:0/500980646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c19d350 con 0x7fd49c106220 2024-09-11T22:29:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.359+0000 7fd4a20df640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd49c106220 0x7fd49c19cc20 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd494031a20 tx=0x7fd494031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.359+0000 7fd492ffd640 1 -- 172.21.15.5:0/500980646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494003b70 con 0x7fd49c106220 2024-09-11T22:29:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.359+0000 7fd4a3b42640 1 -- 172.21.15.5:0/500980646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49c19d5e0 con 0x7fd49c106220 2024-09-11T22:29:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.359+0000 7fd4a3b42640 1 -- 172.21.15.5:0/500980646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49c1a4ed0 con 0x7fd49c106220 2024-09-11T22:29:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.359+0000 7fd492ffd640 1 -- 172.21.15.5:0/500980646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd494003d10 con 0x7fd49c106220 2024-09-11T22:29:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.359+0000 7fd492ffd640 1 -- 172.21.15.5:0/500980646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49402fc40 con 0x7fd49c106220 2024-09-11T22:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.362+0000 7fd4a68db740 1 Processor -- start 2024-09-11T22:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.363+0000 7fd492ffd640 1 -- 172.21.15.5:0/500980646 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd494004050 con 0x7fd49c106220 2024-09-11T22:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.363+0000 7fd4a68db740 1 -- start start 2024-09-11T22:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.363+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e835ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.363+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 0x5591e836db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.363+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e836e0c0 0x5591e83704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.363+0000 7fd4a68db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5591e82c32f0 con 0x5591e835e9b0 2024-09-11T22:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.363+0000 7fd4a68db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5591e826b270 con 0x5591e835f2d0 2024-09-11T22:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.363+0000 7fd4a68db740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5591e82bf2b0 con 0x5591e836e0c0 2024-09-11T22:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a18de640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 0x5591e836db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a18de640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 0x5591e836db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41916/0 (socket says 172.21.15.5:41916) 2024-09-11T22:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a18de640 1 -- 172.21.15.5:0/4018126197 learned_addr learned my addr 172.21.15.5:0/4018126197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/4018126197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e836e0c0 0x5591e83704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/4018126197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e835ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a18de640 1 -- 172.21.15.5:0/4018126197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e836e0c0 msgr2=0x5591e83704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a18de640 1 --2- 172.21.15.5:0/4018126197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e836e0c0 0x5591e83704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a18de640 1 -- 172.21.15.5:0/4018126197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 msgr2=0x5591e835ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a18de640 1 --2- 172.21.15.5:0/4018126197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e835ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a18de640 1 -- 172.21.15.5:0/4018126197 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e826a340 con 0x5591e835f2d0 2024-09-11T22:29:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/4018126197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e835ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:29:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.364+0000 7fd492ffd640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd4880a2e20 0x7fd4880a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.365+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd4880a2e20 0x7fd4880a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.365+0000 7fd492ffd640 1 -- 172.21.15.5:0/500980646 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fd4940f02a0 con 0x7fd49c106220 2024-09-11T22:29:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.365+0000 7fd4a18de640 1 --2- 172.21.15.5:0/4018126197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 0x5591e836db80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd498009ca0 tx=0x7fd498011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc08146f7ccdc4cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.365+0000 7fd487fff640 1 -- 172.21.15.5:0/4018126197 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49801e070 con 0x5591e835f2d0 2024-09-11T22:29:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.365+0000 7fd487fff640 1 -- 172.21.15.5:0/4018126197 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd498019b10 con 0x5591e835f2d0 2024-09-11T22:29:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.365+0000 7fd487fff640 1 -- 172.21.15.5:0/4018126197 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49800eae0 con 0x5591e835f2d0 2024-09-11T22:29:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.366+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd4880a2e20 0x7fd4880a52e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009fd0 tx=0x7fd48c013040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.366+0000 7fd4a68db740 1 -- 172.21.15.5:0/4018126197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 msgr2=0x5591e836db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.366+0000 7fd4a68db740 1 --2- 172.21.15.5:0/4018126197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 0x5591e836db80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd498009ca0 tx=0x7fd498011810 comp rx=0 tx=0).stop 2024-09-11T22:29:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.367+0000 7fd4a68db740 1 -- 172.21.15.5:0/4018126197 shutdown_connections 2024-09-11T22:29:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.367+0000 7fd4a68db740 1 --2- 172.21.15.5:0/4018126197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e836e0c0 0x5591e83704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.367+0000 7fd4a68db740 1 --2- 172.21.15.5:0/4018126197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 0x5591e836db80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.367+0000 7fd4a68db740 1 --2- 172.21.15.5:0/4018126197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e835ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.367+0000 7fd4a68db740 1 -- 172.21.15.5:0/4018126197 >> 172.21.15.5:0/4018126197 conn(0x5591e835b060 msgr2=0x5591e835b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.367+0000 7fd4a68db740 1 -- 172.21.15.5:0/4018126197 shutdown_connections 2024-09-11T22:29:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.367+0000 7fd4a68db740 1 -- 172.21.15.5:0/4018126197 wait complete. 2024-09-11T22:29:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.367+0000 7fd4a68db740 1 -- start start 2024-09-11T22:29:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.368+0000 7fd4a68db740 1 Processor -- start 2024-09-11T22:29:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.368+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e836b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.368+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e836b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.368+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e836e0c0 0x5591e836bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.368+0000 7fd4a68db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5591e8370c90 con 0x5591e835e9b0 2024-09-11T22:29:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.368+0000 7fd4a68db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5591e824c290 con 0x5591e836e0c0 2024-09-11T22:29:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.368+0000 7fd4a68db740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5591e8370e10 con 0x5591e835f2d0 2024-09-11T22:29:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.368+0000 7fd4a68db740 10 client.0 _ll_register_callbacks cb 0x5591e827d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a10dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e836b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a18de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e836b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a10dd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e836b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:52468/0 (socket says 172.21.15.5:52468) 2024-09-11T22:29:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a10dd640 1 -- 172.21.15.5:0/1398139010 learned_addr learned my addr 172.21.15.5:0/1398139010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a20df640 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e836e0c0 0x5591e836bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a10dd640 1 -- 172.21.15.5:0/1398139010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e836e0c0 msgr2=0x5591e836bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e836e0c0 0x5591e836bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a10dd640 1 -- 172.21.15.5:0/1398139010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 msgr2=0x5591e836b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e836b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a10dd640 1 -- 172.21.15.5:0/1398139010 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e7fd6210 con 0x5591e835f2d0 2024-09-11T22:29:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.369+0000 7fd4a18de640 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e836b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.370+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e836b5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0038a0 tx=0x7fd48c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c291cb96d95fb21 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.370+0000 7fd4857fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:29:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.370+0000 7fd4857fa640 1 -- 172.21.15.5:0/1398139010 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48c003ca0 con 0x5591e835f2d0 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.371+0000 7fd4a68db740 1 -- 172.21.15.5:0/1398139010 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd498019b10 con 0x5591e835f2d0 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.371+0000 7fd4857fa640 1 -- 172.21.15.5:0/1398139010 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fd48c00fa80 con 0x5591e835f2d0 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.371+0000 7fd4857fa640 1 -- 172.21.15.5:0/1398139010 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd48c003e40 con 0x5591e835f2d0 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.371+0000 7fd4a68db740 10 client.27088 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.371+0000 7fd4a68db740 1 -- 172.21.15.5:0/1398139010 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5591e80a2620 con 0x5591e835f2d0 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.371+0000 7fd4857fa640 1 -- 172.21.15.5:0/1398139010 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd498019b10 con 0x5591e835f2d0 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.372+0000 7fd4857fa640 1 -- 172.21.15.5:0/1398139010 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fd48c02f030 con 0x5591e835f2d0 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.372+0000 7fd4a68db740 10 client.27088 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.372+0000 7fd4a68db740 10 client.27088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.372+0000 7fd4a68db740 20 client.27088 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.372+0000 7fd4a68db740 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5591e837e260 0x5591e83806b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.372+0000 7fd4a68db740 4 client.27088 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.372+0000 7fd4a68db740 1 -- 172.21.15.5:0/1398139010 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591e82d0a20 con 0x5591e837e260 2024-09-11T22:29:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.372+0000 7fd4a18de640 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5591e837e260 0x5591e83806b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.372+0000 7fd4857fa640 10 client.27088 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.372+0000 7fd4a18de640 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5591e837e260 0x5591e83806b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b3bcf88da87e96 server_cookie=9da2c01b595c07b0 in_seq=0 out_seq=0 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.373+0000 7fd4857fa640 1 -- 172.21.15.5:0/1398139010 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591e82d0a20 con 0x5591e837e260 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.373+0000 7fd4857fa640 10 client.27088 handle_command_reply: tid=0 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.373+0000 7fd4a68db740 1 client.27088 shutdown 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 1 -- 172.21.15.5:0/1398139010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 msgr2=0x5591e836b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e836b5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0038a0 tx=0x7fd48c004730 comp rx=0 tx=0).stop 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 1 -- 172.21.15.5:0/1398139010 shutdown_connections 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5591e837e260 0x5591e83806b0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e836e0c0 0x5591e836bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e836b5c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 1 --2- 172.21.15.5:0/1398139010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e836b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 1 -- 172.21.15.5:0/1398139010 >> 172.21.15.5:0/1398139010 conn(0x5591e835b060 msgr2=0x5591e8360cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4857fa640 0 client.27088 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 1 -- 172.21.15.5:0/1398139010 shutdown_connections 2024-09-11T22:29:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 1 -- 172.21.15.5:0/1398139010 wait complete. 2024-09-11T22:29:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.374+0000 7fd4a68db740 20 client.27088 trim_cache size 0 max 16384 2024-09-11T22:29:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a68db740 1 Processor -- start 2024-09-11T22:29:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a68db740 1 -- start start 2024-09-11T22:29:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8386010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e8386550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e836e0c0 0x5591e83682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a68db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd48000e8d0 con 0x5591e835e9b0 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a68db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd480010410 con 0x5591e836e0c0 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a68db740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd48000efa0 con 0x5591e835f2d0 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a18de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8386010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a18de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8386010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54244/0 (socket says 172.21.15.5:54244) 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/1458746597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e8386550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a18de640 1 -- 172.21.15.5:0/1458746597 learned_addr learned my addr 172.21.15.5:0/1458746597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.391+0000 7fd4a20df640 1 --2- 172.21.15.5:0/1458746597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e836e0c0 0x5591e83682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.392+0000 7fd4a18de640 1 -- 172.21.15.5:0/1458746597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 msgr2=0x5591e8386550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.392+0000 7fd4a18de640 1 --2- 172.21.15.5:0/1458746597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e8386550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.392+0000 7fd4a18de640 1 -- 172.21.15.5:0/1458746597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e836e0c0 msgr2=0x5591e83682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.392+0000 7fd4a18de640 1 --2- 172.21.15.5:0/1458746597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e836e0c0 0x5591e83682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.392+0000 7fd4a18de640 1 -- 172.21.15.5:0/1458746597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591e8370820 con 0x5591e835e9b0 2024-09-11T22:29:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.392+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/1458746597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e8386550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.392+0000 7fd4a18de640 1 --2- 172.21.15.5:0/1458746597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8386010 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e610 tx=0x7fd49800e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddb06ecb5dba7c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.392+0000 7fd4877fe640 1 -- 172.21.15.5:0/1458746597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49801e040 con 0x5591e835e9b0 2024-09-11T22:29:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.392+0000 7fd4877fe640 1 -- 172.21.15.5:0/1458746597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49801f040 con 0x5591e835e9b0 2024-09-11T22:29:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.392+0000 7fd4877fe640 1 -- 172.21.15.5:0/1458746597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4980035f0 con 0x5591e835e9b0 2024-09-11T22:29:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 -- 172.21.15.5:0/1458746597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 msgr2=0x5591e8386010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 --2- 172.21.15.5:0/1458746597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8386010 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e610 tx=0x7fd49800e5a0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 -- 172.21.15.5:0/1458746597 shutdown_connections 2024-09-11T22:29:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 --2- 172.21.15.5:0/1458746597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e836e0c0 0x5591e83682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 --2- 172.21.15.5:0/1458746597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e835f2d0 0x5591e8386550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 --2- 172.21.15.5:0/1458746597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8386010 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 -- 172.21.15.5:0/1458746597 >> 172.21.15.5:0/1458746597 conn(0x5591e835b060 msgr2=0x5591e836b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 -- 172.21.15.5:0/1458746597 shutdown_connections 2024-09-11T22:29:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 -- 172.21.15.5:0/1458746597 wait complete. 2024-09-11T22:29:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 -- start start 2024-09-11T22:29:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.393+0000 7fd4a68db740 1 Processor -- start 2024-09-11T22:29:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8376990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 0x5591e8371cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a68db740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e836e0c0 0x5591e83722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a68db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd48000fd40 con 0x5591e835e9b0 2024-09-11T22:29:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a68db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd48000f670 con 0x5591e835f2d0 2024-09-11T22:29:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a68db740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd48000c8b0 con 0x5591e836e0c0 2024-09-11T22:29:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a68db740 10 client.0 _ll_register_callbacks cb 0x7fd48c004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a18de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8376990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a18de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8376990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54258/0 (socket says 172.21.15.5:54258) 2024-09-11T22:29:46.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a18de640 1 -- 172.21.15.5:0/2347228540 learned_addr learned my addr 172.21.15.5:0/2347228540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a10dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 0x5591e8371cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a20df640 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e836e0c0 0x5591e83722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a18de640 1 -- 172.21.15.5:0/2347228540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e836e0c0 msgr2=0x5591e83722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a18de640 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e836e0c0 0x5591e83722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a18de640 1 -- 172.21.15.5:0/2347228540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 msgr2=0x5591e8371cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a18de640 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 0x5591e8371cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a18de640 1 -- 172.21.15.5:0/2347228540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591e8372ad0 con 0x5591e835e9b0 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4a18de640 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8376990 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd498009ca0 tx=0x7fd498005480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2474df6e5846066 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.394+0000 7fd4857fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4857fa640 1 -- 172.21.15.5:0/2347228540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49801e070 con 0x5591e835e9b0 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4857fa640 1 -- 172.21.15.5:0/2347228540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7fd49800e8c0 con 0x5591e835e9b0 2024-09-11T22:29:46.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4a68db740 1 -- 172.21.15.5:0/2347228540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd49801f040 con 0x5591e835e9b0 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4857fa640 1 -- 172.21.15.5:0/2347228540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd49800e3b0 con 0x5591e835e9b0 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4a68db740 10 client.17826 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4a68db740 1 -- 172.21.15.5:0/2347228540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5591e83e9100 con 0x5591e835e9b0 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4857fa640 1 -- 172.21.15.5:0/2347228540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49801f040 con 0x5591e835e9b0 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4857fa640 1 -- 172.21.15.5:0/2347228540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fd49802f070 con 0x5591e835e9b0 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4a68db740 10 client.17826 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4a68db740 10 client.17826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4a68db740 20 client.17826 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4a68db740 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5591e83e95c0 0x5591e83eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4a68db740 4 client.17826 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:29:46.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4a68db740 1 -- 172.21.15.5:0/2347228540 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591e82cad70 con 0x5591e83e95c0 2024-09-11T22:29:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.395+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5591e83e95c0 0x5591e83eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.396+0000 7fd4a10dd640 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5591e83e95c0 0x5591e83eb9b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fee2ddb1c53038b server_cookie=a54ef1e6ae9d96e2 in_seq=0 out_seq=0 2024-09-11T22:29:46.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.396+0000 7fd4857fa640 10 client.17826 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4857fa640 1 -- 172.21.15.5:0/2347228540 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591e82cad70 con 0x5591e83e95c0 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4857fa640 10 client.17826 handle_command_reply: tid=0 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 client.17826 shutdown 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 -- 172.21.15.5:0/2347228540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 msgr2=0x5591e8376990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8376990 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd498009ca0 tx=0x7fd498005480 comp rx=0 tx=0).stop 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 -- 172.21.15.5:0/2347228540 shutdown_connections 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5591e83e95c0 0x5591e83eb9b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5591e836e0c0 0x5591e83722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5591e835f2d0 0x5591e8371cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4857fa640 0 client.17826 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 --2- 172.21.15.5:0/2347228540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591e835e9b0 0x5591e8376990 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 -- 172.21.15.5:0/2347228540 >> 172.21.15.5:0/2347228540 conn(0x5591e835b060 msgr2=0x5591e836b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 -- 172.21.15.5:0/2347228540 shutdown_connections 2024-09-11T22:29:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.445+0000 7fd4a68db740 1 -- 172.21.15.5:0/2347228540 wait complete. 2024-09-11T22:29:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a68db740 20 client.17826 trim_cache size 0 max 16384 2024-09-11T22:29:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:29:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:29:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:29:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:29:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 -- 172.21.15.5:0/500980646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd4880a2e20 msgr2=0x7fd4880a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd4880a2e20 0x7fd4880a52e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd48c009fd0 tx=0x7fd48c013040 comp rx=0 tx=0).stop 2024-09-11T22:29:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 -- 172.21.15.5:0/500980646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd49c106220 msgr2=0x7fd49c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd49c106220 0x7fd49c19cc20 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd494031a20 tx=0x7fd494031a50 comp rx=0 tx=0).stop 2024-09-11T22:29:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 -- 172.21.15.5:0/500980646 shutdown_connections 2024-09-11T22:29:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd49c106220 0x7fd49c19cc20 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd4880a2e20 0x7fd4880a52e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd49c100fd0 0x7fd49c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 --2- 172.21.15.5:0/500980646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd49c100630 0x7fd49c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 -- 172.21.15.5:0/500980646 >> 172.21.15.5:0/500980646 conn(0x7fd49c075ef0 msgr2=0x7fd49c0ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 -- 172.21.15.5:0/500980646 shutdown_connections 2024-09-11T22:29:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:46.446+0000 7fd4a3b42640 1 -- 172.21.15.5:0/500980646 wait complete. 2024-09-11T22:29:46.933 INFO:tasks.cephfs.filesystem:scrub status for tag:4e525aac-35f7-4e3a-a8e7-9502ace711c5 - {'path': '/', 'tag': '4e525aac-35f7-4e3a-a8e7-9502ace711c5', 'options': 'recursive,force'} 2024-09-11T22:29:47.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 568613 7.86 MB/sec execute 245 sec latency 33.108 ms 2024-09-11T22:29:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:47 smithi005 ceph-mon[30943]: pgmap v236: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-09-11T22:29:47.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:47 smithi159 ceph-mon[35534]: pgmap v236: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-09-11T22:29:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:47 smithi172 ceph-mon[35461]: pgmap v236: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-09-11T22:29:48.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 570761 7.86 MB/sec execute 246 sec latency 14.678 ms 2024-09-11T22:29:49.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 572801 7.88 MB/sec execute 247 sec latency 15.557 ms 2024-09-11T22:29:49.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:49 smithi005 ceph-mon[30943]: pgmap v237: 97 pgs: 97 active+clean; 92 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-11T22:29:49.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:49 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:29:49.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:49 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:29:49.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:49 smithi159 ceph-mon[35534]: pgmap v237: 97 pgs: 97 active+clean; 92 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-11T22:29:49.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:49 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:29:49.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:49 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:29:49.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:49 smithi172 ceph-mon[35461]: pgmap v237: 97 pgs: 97 active+clean; 92 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 2.7 MiB/s wr, 214 op/s 2024-09-11T22:29:49.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:49 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:29:49.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:49 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:29:50.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 575440 7.92 MB/sec execute 248 sec latency 14.560 ms 2024-09-11T22:29:51.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 577005 7.91 MB/sec execute 249 sec latency 98.439 ms 2024-09-11T22:29:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:51 smithi005 ceph-mon[30943]: pgmap v238: 97 pgs: 97 active+clean; 92 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-11T22:29:51.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:51 smithi159 ceph-mon[35534]: pgmap v238: 97 pgs: 97 active+clean; 92 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-11T22:29:51.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:51 smithi172 ceph-mon[35461]: pgmap v238: 97 pgs: 97 active+clean; 92 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-11T22:29:52.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 579347 7.93 MB/sec execute 250 sec latency 16.853 ms 2024-09-11T22:29:53.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 581567 7.94 MB/sec execute 251 sec latency 16.566 ms 2024-09-11T22:29:53.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:53 smithi005 ceph-mon[30943]: pgmap v239: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-09-11T22:29:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:53 smithi159 ceph-mon[35534]: pgmap v239: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-09-11T22:29:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:53 smithi172 ceph-mon[35461]: pgmap v239: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-09-11T22:29:54.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 583652 7.96 MB/sec execute 252 sec latency 17.916 ms 2024-09-11T22:29:55.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 586161 7.99 MB/sec execute 253 sec latency 14.165 ms 2024-09-11T22:29:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:55 smithi005 ceph-mon[30943]: pgmap v240: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-11T22:29:55.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:55 smithi159 ceph-mon[35534]: pgmap v240: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-11T22:29:55.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:55 smithi172 ceph-mon[35461]: pgmap v240: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-11T22:29:56.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 587766 7.98 MB/sec execute 254 sec latency 21.121 ms 2024-09-11T22:29:57.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 590009 8.00 MB/sec execute 255 sec latency 92.506 ms 2024-09-11T22:29:57.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:57 smithi005 ceph-mon[30943]: pgmap v241: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-11T22:29:57.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:57 smithi159 ceph-mon[35534]: pgmap v241: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-11T22:29:57.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:57 smithi172 ceph-mon[35461]: pgmap v241: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 211 op/s 2024-09-11T22:29:58.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 592212 8.00 MB/sec execute 256 sec latency 30.795 ms 2024-09-11T22:29:59.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 594268 8.02 MB/sec execute 257 sec latency 12.656 ms 2024-09-11T22:29:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.536+0000 7f1386303640 1 -- 172.21.15.5:0/3449142061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1380101350 msgr2=0x7f138010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.536+0000 7f1386303640 1 --2- 172.21.15.5:0/3449142061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1380101350 0x7f138010d880 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f1378009cd0 tx=0x7f137802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.537+0000 7f1386303640 1 -- 172.21.15.5:0/3449142061 shutdown_connections 2024-09-11T22:29:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.537+0000 7f1386303640 1 --2- 172.21.15.5:0/3449142061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1380101350 0x7f138010d880 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.537+0000 7f1386303640 1 --2- 172.21.15.5:0/3449142061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13801009b0 0x7f1380100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.537+0000 7f1386303640 1 --2- 172.21.15.5:0/3449142061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13801069c0 0x7f1380106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.537+0000 7f1386303640 1 -- 172.21.15.5:0/3449142061 >> 172.21.15.5:0/3449142061 conn(0x7f1380077990 msgr2=0x7f13800fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.537+0000 7f1386303640 1 -- 172.21.15.5:0/3449142061 shutdown_connections 2024-09-11T22:29:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.537+0000 7f1386303640 1 -- 172.21.15.5:0/3449142061 wait complete. 2024-09-11T22:29:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.537+0000 7f1386303640 1 Processor -- start 2024-09-11T22:29:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.537+0000 7f1386303640 1 -- start start 2024-09-11T22:29:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f1386303640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13801009b0 0x7f1380197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f1386303640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1380101350 0x7f1380197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f1386303640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13801069c0 0x7f138019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f1386303640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1380110e00 con 0x7f1380101350 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f1386303640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1380110c80 con 0x7f13801009b0 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f1386303640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1380110f80 con 0x7f13801069c0 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1380101350 0x7f1380197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1380101350 0x7f1380197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52172/0 (socket says 172.21.15.5:52172) 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13801009b0 0x7f1380197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137f7fe640 1 -- 172.21.15.5:0/67710136 learned_addr learned my addr 172.21.15.5:0/67710136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f13848a0640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13801069c0 0x7f138019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137f7fe640 1 -- 172.21.15.5:0/67710136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13801069c0 msgr2=0x7f138019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137f7fe640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13801069c0 0x7f138019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137f7fe640 1 -- 172.21.15.5:0/67710136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13801009b0 msgr2=0x7f1380197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137f7fe640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13801009b0 0x7f1380197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137f7fe640 1 -- 172.21.15.5:0/67710136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138019c1f0 con 0x7f1380101350 2024-09-11T22:29:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137f7fe640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1380101350 0x7f1380197740 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f137400e9f0 tx=0x7f137400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137d7fa640 1 -- 172.21.15.5:0/67710136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137400cd80 con 0x7f1380101350 2024-09-11T22:29:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137d7fa640 1 -- 172.21.15.5:0/67710136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1374005500 con 0x7f1380101350 2024-09-11T22:29:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f1386303640 1 -- 172.21.15.5:0/67710136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138019c4e0 con 0x7f1380101350 2024-09-11T22:29:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f137d7fa640 1 -- 172.21.15.5:0/67710136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374010690 con 0x7f1380101350 2024-09-11T22:29:59.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.538+0000 7f1386303640 1 -- 172.21.15.5:0/67710136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13801a3d70 con 0x7f1380101350 2024-09-11T22:29:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.541+0000 7f138909a740 1 Processor -- start 2024-09-11T22:29:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.541+0000 7f138909a740 1 -- start start 2024-09-11T22:29:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.541+0000 7f137d7fa640 1 -- 172.21.15.5:0/67710136 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f13740046c0 con 0x7f1380101350 2024-09-11T22:29:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.541+0000 7f138909a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be70990 0x55e80be70d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.541+0000 7f138909a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be712b0 0x55e80be7d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.541+0000 7f138909a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 0x55e80be802b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.541+0000 7f138909a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e80bdd52f0 con 0x55e80be7dec0 2024-09-11T22:29:59.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.541+0000 7f138909a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e80bd7d270 con 0x55e80be712b0 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.541+0000 7f138909a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e80bdd12b0 con 0x55e80be70990 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13848a0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be70990 0x55e80be70d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f137f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 0x55e80be802b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f137ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be712b0 0x55e80be7d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13848a0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be70990 0x55e80be70d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:40994/0 (socket says 172.21.15.5:40994) 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13848a0640 1 -- 172.21.15.5:0/2753610950 learned_addr learned my addr 172.21.15.5:0/2753610950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13848a0640 1 -- 172.21.15.5:0/2753610950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be712b0 msgr2=0x55e80be7d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13848a0640 1 --2- 172.21.15.5:0/2753610950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be712b0 0x55e80be7d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13848a0640 1 -- 172.21.15.5:0/2753610950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 msgr2=0x55e80be802b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13848a0640 1 --2- 172.21.15.5:0/2753610950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 0x55e80be802b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13848a0640 1 -- 172.21.15.5:0/2753610950 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80bd7c340 con 0x55e80be70990 2024-09-11T22:29:59.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f137d7fa640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13680a2da0 0x7f13680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13848a0640 1 --2- 172.21.15.5:0/2753610950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be70990 0x55e80be70d70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1378009cd0 tx=0x7f1378009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dde8cdfa46bc974 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f137ffff640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13680a2da0 0x7f13680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f137d7fa640 1 -- 172.21.15.5:0/67710136 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f13740cf8b0 con 0x7f1380101350 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13667fc640 1 -- 172.21.15.5:0/2753610950 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378048070 con 0x55e80be70990 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13667fc640 1 -- 172.21.15.5:0/2753610950 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1378036070 con 0x55e80be70990 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.542+0000 7f13667fc640 1 -- 172.21.15.5:0/2753610950 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13780043a0 con 0x55e80be70990 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f138909a740 1 -- 172.21.15.5:0/2753610950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be70990 msgr2=0x55e80be70d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f137ffff640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13680a2da0 0x7f13680a5260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55e80be6dec0 tx=0x7f1370005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f138909a740 1 --2- 172.21.15.5:0/2753610950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be70990 0x55e80be70d70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1378009cd0 tx=0x7f1378009a20 comp rx=0 tx=0).stop 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f138909a740 1 -- 172.21.15.5:0/2753610950 shutdown_connections 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f138909a740 1 --2- 172.21.15.5:0/2753610950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 0x55e80be802b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f138909a740 1 --2- 172.21.15.5:0/2753610950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be712b0 0x55e80be7d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f138909a740 1 --2- 172.21.15.5:0/2753610950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be70990 0x55e80be70d70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f138909a740 1 -- 172.21.15.5:0/2753610950 >> 172.21.15.5:0/2753610950 conn(0x55e80be6d060 msgr2=0x55e80be6d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f138909a740 1 -- 172.21.15.5:0/2753610950 shutdown_connections 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f138909a740 1 -- 172.21.15.5:0/2753610950 wait complete. 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.543+0000 7f138909a740 1 -- start start 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f138909a740 1 Processor -- start 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f138909a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 0x55e80be7aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f138909a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be712b0 0x55e80be7b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f138909a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 0x55e80be7b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f138909a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e80be80a90 con 0x55e80be7dec0 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f138909a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e80bd5e290 con 0x55e80be70990 2024-09-11T22:29:59.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f138909a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e80be80c10 con 0x55e80be712b0 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f138909a740 10 client.0 _ll_register_callbacks cb 0x55e80bd8f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f137ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 0x55e80be7aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f137f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be712b0 0x55e80be7b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f13848a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 0x55e80be7b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f137ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 0x55e80be7aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53854/0 (socket says 172.21.15.5:53854) 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f137f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be712b0 0x55e80be7b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:41010/0 (socket says 172.21.15.5:41010) 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.544+0000 7f137ffff640 1 -- 172.21.15.5:0/1686385144 learned_addr learned my addr 172.21.15.5:0/1686385144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f13848a0640 1 -- 172.21.15.5:0/1686385144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be712b0 msgr2=0x55e80be7b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f13848a0640 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be712b0 0x55e80be7b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f13848a0640 1 -- 172.21.15.5:0/1686385144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 msgr2=0x55e80be7aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f13848a0640 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 0x55e80be7aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f13848a0640 1 -- 172.21.15.5:0/1686385144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80bae8210 con 0x55e80be7dec0 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f137ffff640 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 0x55e80be7aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f13848a0640 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 0x55e80be7b960 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f137802ae00 tx=0x7f1378009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f236006da5f21ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f135f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f135f7fe640 1 -- 172.21.15.5:0/1686385144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378048040 con 0x55e80be7dec0 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f138909a740 1 -- 172.21.15.5:0/1686385144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1378036070 con 0x55e80be7dec0 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f135f7fe640 1 -- 172.21.15.5:0/1686385144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f1378031ac0 con 0x55e80be7dec0 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f135f7fe640 1 -- 172.21.15.5:0/1686385144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1378031cc0 con 0x55e80be7dec0 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f138909a740 10 client.17838 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f138909a740 1 -- 172.21.15.5:0/1686385144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e80bbb4620 con 0x55e80be7dec0 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.545+0000 7f135f7fe640 1 -- 172.21.15.5:0/1686385144 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1378036070 con 0x55e80be7dec0 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.546+0000 7f135f7fe640 1 -- 172.21.15.5:0/1686385144 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f1378042030 con 0x55e80be7dec0 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.546+0000 7f138909a740 10 client.17838 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:59.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.546+0000 7f138909a740 10 client.17838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.546+0000 7f138909a740 20 client.17838 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.546+0000 7f138909a740 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e80be74870 0x55e80be74c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.546+0000 7f138909a740 4 client.17838 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:29:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.546+0000 7f138909a740 1 -- 172.21.15.5:0/1686385144 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e80bde2a20 con 0x55e80be74870 2024-09-11T22:29:59.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.546+0000 7f137ffff640 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e80be74870 0x55e80be74c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.547+0000 7f135f7fe640 10 client.17838 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.547+0000 7f137ffff640 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e80be74870 0x55e80be74c70 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbe045b0d2812a31 server_cookie=10fa2050e6d71d7b in_seq=0 out_seq=0 2024-09-11T22:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.548+0000 7f135f7fe640 1 -- 172.21.15.5:0/1686385144 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e80bde2a20 con 0x55e80be74870 2024-09-11T22:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.548+0000 7f135f7fe640 10 client.17838 handle_command_reply: tid=0 2024-09-11T22:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.548+0000 7f138909a740 1 client.17838 shutdown 2024-09-11T22:29:59.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 1 -- 172.21.15.5:0/1686385144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 msgr2=0x55e80be7b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 0x55e80be7b960 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f137802ae00 tx=0x7f1378009640 comp rx=0 tx=0).stop 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 1 -- 172.21.15.5:0/1686385144 shutdown_connections 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e80be74870 0x55e80be74c70 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be7dec0 0x55e80be7b960 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be712b0 0x55e80be7b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f135f7fe640 0 client.17838 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 1 --2- 172.21.15.5:0/1686385144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 0x55e80be7aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 1 -- 172.21.15.5:0/1686385144 >> 172.21.15.5:0/1686385144 conn(0x55e80be6d060 msgr2=0x55e80be72dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 1 -- 172.21.15.5:0/1686385144 shutdown_connections 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 1 -- 172.21.15.5:0/1686385144 wait complete. 2024-09-11T22:29:59.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.549+0000 7f138909a740 20 client.17838 trim_cache size 0 max 16384 2024-09-11T22:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f138909a740 1 Processor -- start 2024-09-11T22:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f138909a740 1 -- start start 2024-09-11T22:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f138909a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 0x55e80be78120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f138909a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be712b0 0x55e80be78660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f138909a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be7dec0 0x55e80be79660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f138909a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13801ab650 con 0x55e80be712b0 2024-09-11T22:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f138909a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13801ad190 con 0x55e80be70990 2024-09-11T22:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f138909a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13801abd20 con 0x55e80be7dec0 2024-09-11T22:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f137f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be712b0 0x55e80be78660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f137f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be712b0 0x55e80be78660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52206/0 (socket says 172.21.15.5:52206) 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f137f7fe640 1 -- 172.21.15.5:0/1166954580 learned_addr learned my addr 172.21.15.5:0/1166954580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f13848a0640 1 --2- 172.21.15.5:0/1166954580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be7dec0 0x55e80be79660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.565+0000 7f137ffff640 1 --2- 172.21.15.5:0/1166954580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 0x55e80be78120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.566+0000 7f137f7fe640 1 -- 172.21.15.5:0/1166954580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be7dec0 msgr2=0x55e80be79660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.566+0000 7f137f7fe640 1 --2- 172.21.15.5:0/1166954580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be7dec0 0x55e80be79660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.566+0000 7f137f7fe640 1 -- 172.21.15.5:0/1166954580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 msgr2=0x55e80be78120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.566+0000 7f137f7fe640 1 --2- 172.21.15.5:0/1166954580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 0x55e80be78120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.566+0000 7f137f7fe640 1 -- 172.21.15.5:0/1166954580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80be75d10 con 0x55e80be712b0 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.566+0000 7f137f7fe640 1 --2- 172.21.15.5:0/1166954580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be712b0 0x55e80be78660 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1374098760 tx=0x7f1374009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff2e1cf45c43ff8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.566+0000 7f1365ffb640 1 -- 172.21.15.5:0/1166954580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137409bb60 con 0x55e80be712b0 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.566+0000 7f1365ffb640 1 -- 172.21.15.5:0/1166954580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137409c490 con 0x55e80be712b0 2024-09-11T22:29:59.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.566+0000 7f1365ffb640 1 -- 172.21.15.5:0/1166954580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137409b6f0 con 0x55e80be712b0 2024-09-11T22:29:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 -- 172.21.15.5:0/1166954580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be712b0 msgr2=0x55e80be78660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 --2- 172.21.15.5:0/1166954580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be712b0 0x55e80be78660 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f1374098760 tx=0x7f1374009310 comp rx=0 tx=0).stop 2024-09-11T22:29:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 -- 172.21.15.5:0/1166954580 shutdown_connections 2024-09-11T22:29:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 --2- 172.21.15.5:0/1166954580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be7dec0 0x55e80be79660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 --2- 172.21.15.5:0/1166954580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be712b0 0x55e80be78660 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 --2- 172.21.15.5:0/1166954580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be70990 0x55e80be78120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 -- 172.21.15.5:0/1166954580 >> 172.21.15.5:0/1166954580 conn(0x55e80be6d060 msgr2=0x55e80be7cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 -- 172.21.15.5:0/1166954580 shutdown_connections 2024-09-11T22:29:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 -- 172.21.15.5:0/1166954580 wait complete. 2024-09-11T22:29:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 -- start start 2024-09-11T22:29:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.567+0000 7f138909a740 1 Processor -- start 2024-09-11T22:29:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f138909a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be70990 0x55e80be8e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f138909a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be712b0 0x55e80be8eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f138909a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be7dec0 0x55e80be8f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f138909a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13801acac0 con 0x55e80be70990 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f138909a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13801ac3f0 con 0x55e80be712b0 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f138909a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13801a9630 con 0x55e80be7dec0 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f138909a740 10 client.0 _ll_register_callbacks cb 0x7f1378046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f137ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be70990 0x55e80be8e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f137ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be70990 0x55e80be8e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52216/0 (socket says 172.21.15.5:52216) 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f137ffff640 1 -- 172.21.15.5:0/1400727044 learned_addr learned my addr 172.21.15.5:0/1400727044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f137f7fe640 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be712b0 0x55e80be8eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f13848a0640 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be7dec0 0x55e80be8f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f137ffff640 1 -- 172.21.15.5:0/1400727044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be7dec0 msgr2=0x55e80be8f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f137ffff640 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be7dec0 0x55e80be8f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f137ffff640 1 -- 172.21.15.5:0/1400727044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be712b0 msgr2=0x55e80be8eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f137ffff640 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be712b0 0x55e80be8eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f137ffff640 1 -- 172.21.15.5:0/1400727044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80bef4c90 con 0x55e80be70990 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f137ffff640 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be70990 0x55e80be8e970 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1370005b90 tx=0x7f13700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9d146870dbccec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f135f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.568+0000 7f135f7fe640 1 -- 172.21.15.5:0/1400727044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137002b500 con 0x55e80be70990 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f135f7fe640 1 -- 172.21.15.5:0/1400727044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8585+0+0 (secure 0 0 0) 0x7f137002b6a0 con 0x55e80be70990 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f138909a740 1 -- 172.21.15.5:0/1400727044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f137409c490 con 0x55e80be70990 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f135f7fe640 1 -- 172.21.15.5:0/1400727044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f137002b8a0 con 0x55e80be70990 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f138909a740 10 client.17844 fetch_fsmap learned FSMap version 40 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f138909a740 1 -- 172.21.15.5:0/1400727044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e80bef5fc0 con 0x55e80be70990 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f135f7fe640 1 -- 172.21.15.5:0/1400727044 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f137409c490 con 0x55e80be70990 2024-09-11T22:29:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f135f7fe640 1 -- 172.21.15.5:0/1400727044 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f13700333f0 con 0x55e80be70990 2024-09-11T22:29:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f138909a740 10 client.17844 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:29:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f138909a740 10 client.17844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:29:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f138909a740 20 client.17844 populate_metadata read hostname 'smithi005' 2024-09-11T22:29:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f138909a740 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e80bef6760 0x55e80bef8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:29:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f138909a740 4 client.17844 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:29:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f138909a740 1 -- 172.21.15.5:0/1400727044 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e80bddcd70 con 0x55e80bef6760 2024-09-11T22:29:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.569+0000 7f137f7fe640 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e80bef6760 0x55e80bef8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:29:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.570+0000 7f137f7fe640 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e80bef6760 0x55e80bef8b80 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8fd3b84eeaafdfa server_cookie=28d29bd892a18341 in_seq=0 out_seq=0 2024-09-11T22:29:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.570+0000 7f135f7fe640 10 client.17844 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.699+0000 7f135f7fe640 1 -- 172.21.15.5:0/1400727044 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e80bddcd70 con 0x55e80bef6760 2024-09-11T22:29:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.699+0000 7f135f7fe640 10 client.17844 handle_command_reply: tid=0 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 client.17844 shutdown 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 -- 172.21.15.5:0/1400727044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be70990 msgr2=0x55e80be8e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be70990 0x55e80be8e970 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f1370005b90 tx=0x7f13700046c0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 -- 172.21.15.5:0/1400727044 shutdown_connections 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e80bef6760 0x55e80bef8b80 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e80be7dec0 0x55e80be8f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f135f7fe640 0 client.17844 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e80be712b0 0x55e80be8eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 --2- 172.21.15.5:0/1400727044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e80be70990 0x55e80be8e970 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 -- 172.21.15.5:0/1400727044 >> 172.21.15.5:0/1400727044 conn(0x55e80be6d060 msgr2=0x55e80be7ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 -- 172.21.15.5:0/1400727044 shutdown_connections 2024-09-11T22:29:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.700+0000 7f138909a740 1 -- 172.21.15.5:0/1400727044 wait complete. 2024-09-11T22:29:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.701+0000 7f138909a740 20 client.17844 trim_cache size 0 max 16384 2024-09-11T22:29:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:29:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:29:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:29:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:29:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.701+0000 7f1386303640 1 -- 172.21.15.5:0/67710136 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13680a2da0 msgr2=0x7f13680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.701+0000 7f1386303640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13680a2da0 0x7f13680a5260 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55e80be6dec0 tx=0x7f1370005f90 comp rx=0 tx=0).stop 2024-09-11T22:29:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.701+0000 7f1386303640 1 -- 172.21.15.5:0/67710136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1380101350 msgr2=0x7f1380197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:29:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.701+0000 7f1386303640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1380101350 0x7f1380197740 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f137400e9f0 tx=0x7f137400edf0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.702+0000 7f1386303640 1 -- 172.21.15.5:0/67710136 shutdown_connections 2024-09-11T22:29:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.702+0000 7f1386303640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13801069c0 0x7f138019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.702+0000 7f1386303640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1380101350 0x7f1380197740 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.702+0000 7f1386303640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13680a2da0 0x7f13680a5260 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.702+0000 7f1386303640 1 --2- 172.21.15.5:0/67710136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13801009b0 0x7f1380197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:29:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.702+0000 7f1386303640 1 -- 172.21.15.5:0/67710136 >> 172.21.15.5:0/67710136 conn(0x7f1380077990 msgr2=0x7f1380103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:29:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.702+0000 7f1386303640 1 -- 172.21.15.5:0/67710136 shutdown_connections 2024-09-11T22:29:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:29:59.702+0000 7f1386303640 1 -- 172.21.15.5:0/67710136 wait complete. 2024-09-11T22:29:59.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:29:59 smithi005 ceph-mon[30943]: pgmap v242: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 323 op/s 2024-09-11T22:29:59.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:29:59 smithi159 ceph-mon[35534]: pgmap v242: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 323 op/s 2024-09-11T22:29:59.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:29:59 smithi172 ceph-mon[35461]: pgmap v242: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 323 op/s 2024-09-11T22:30:00.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 596373 8.04 MB/sec execute 258 sec latency 106.211 ms 2024-09-11T22:30:00.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:00 smithi005 ceph-mon[30943]: overall HEALTH_OK 2024-09-11T22:30:00.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:00 smithi159 ceph-mon[35534]: overall HEALTH_OK 2024-09-11T22:30:00.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:00 smithi172 ceph-mon[35461]: overall HEALTH_OK 2024-09-11T22:30:01.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 597744 8.03 MB/sec execute 259 sec latency 21.799 ms 2024-09-11T22:30:01.760 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:01 smithi172 ceph-mon[35461]: pgmap v243: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-09-11T22:30:01.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:01 smithi005 ceph-mon[30943]: pgmap v243: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-09-11T22:30:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:01 smithi159 ceph-mon[35534]: pgmap v243: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-09-11T22:30:02.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 599879 8.05 MB/sec execute 260 sec latency 27.797 ms 2024-09-11T22:30:03.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 601748 8.06 MB/sec execute 261 sec latency 15.969 ms 2024-09-11T22:30:03.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:03 smithi005 ceph-mon[30943]: pgmap v244: 97 pgs: 97 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.2 MiB/s wr, 429 op/s 2024-09-11T22:30:03.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:03 smithi159 ceph-mon[35534]: pgmap v244: 97 pgs: 97 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.2 MiB/s wr, 429 op/s 2024-09-11T22:30:03.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:03 smithi172 ceph-mon[35461]: pgmap v244: 97 pgs: 97 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.2 MiB/s wr, 429 op/s 2024-09-11T22:30:04.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 604086 8.08 MB/sec execute 262 sec latency 13.819 ms 2024-09-11T22:30:05.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 604872 8.06 MB/sec execute 263 sec latency 32.097 ms 2024-09-11T22:30:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:05 smithi005 ceph-mon[30943]: pgmap v245: 97 pgs: 97 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 318 op/s 2024-09-11T22:30:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:05 smithi005 ceph-mon[30943]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T22:30:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:05 smithi159 ceph-mon[35534]: pgmap v245: 97 pgs: 97 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 318 op/s 2024-09-11T22:30:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:05 smithi159 ceph-mon[35534]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T22:30:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:05 smithi172 ceph-mon[35461]: pgmap v245: 97 pgs: 97 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 318 op/s 2024-09-11T22:30:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:05 smithi172 ceph-mon[35461]: osdmap e92: 12 total, 12 up, 12 in 2024-09-11T22:30:06.329 INFO:tasks.workunit.client.0.smithi005.stdout: 1 606978 8.07 MB/sec execute 264 sec latency 27.261 ms 2024-09-11T22:30:07.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 608850 8.08 MB/sec execute 265 sec latency 16.453 ms 2024-09-11T22:30:07.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:07 smithi005 ceph-mon[30943]: pgmap v247: 97 pgs: 97 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.0 MiB/s wr, 382 op/s 2024-09-11T22:30:07.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:07 smithi159 ceph-mon[35534]: pgmap v247: 97 pgs: 97 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.0 MiB/s wr, 382 op/s 2024-09-11T22:30:07.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:07 smithi172 ceph-mon[35461]: pgmap v247: 97 pgs: 97 active+clean; 90 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.0 MiB/s wr, 382 op/s 2024-09-11T22:30:08.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 611172 8.10 MB/sec execute 266 sec latency 15.465 ms 2024-09-11T22:30:09.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 613228 8.10 MB/sec execute 267 sec latency 21.897 ms 2024-09-11T22:30:09.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:09 smithi005 ceph-mon[30943]: pgmap v248: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 336 op/s 2024-09-11T22:30:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:09 smithi159 ceph-mon[35534]: pgmap v248: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 336 op/s 2024-09-11T22:30:09.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:09 smithi172 ceph-mon[35461]: pgmap v248: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 336 op/s 2024-09-11T22:30:10.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 615191 8.12 MB/sec execute 268 sec latency 11.504 ms 2024-09-11T22:30:11.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 617449 8.13 MB/sec execute 269 sec latency 74.284 ms 2024-09-11T22:30:11.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:11 smithi005 ceph-mon[30943]: pgmap v249: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 336 op/s 2024-09-11T22:30:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:11 smithi159 ceph-mon[35534]: pgmap v249: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 336 op/s 2024-09-11T22:30:11.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:11 smithi172 ceph-mon[35461]: pgmap v249: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 336 op/s 2024-09-11T22:30:12.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 619513 8.15 MB/sec execute 270 sec latency 13.911 ms 2024-09-11T22:30:12.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.802+0000 7f0f0c00a640 1 -- 172.21.15.5:0/2927284398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f041009d0 msgr2=0x7f0f04100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.802+0000 7f0f0c00a640 1 --2- 172.21.15.5:0/2927284398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f041009d0 0x7f0f04100e30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009cd0 tx=0x7f0ef802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.802+0000 7f0f0c00a640 1 -- 172.21.15.5:0/2927284398 shutdown_connections 2024-09-11T22:30:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.802+0000 7f0f0c00a640 1 --2- 172.21.15.5:0/2927284398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f04101370 0x7f0f0410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.802+0000 7f0f0c00a640 1 --2- 172.21.15.5:0/2927284398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f041009d0 0x7f0f04100e30 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.802+0000 7f0f0c00a640 1 --2- 172.21.15.5:0/2927284398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f041069e0 0x7f0f04106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.802+0000 7f0f0c00a640 1 -- 172.21.15.5:0/2927284398 >> 172.21.15.5:0/2927284398 conn(0x7f0f040fc6b0 msgr2=0x7f0f040fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.802+0000 7f0f0c00a640 1 -- 172.21.15.5:0/2927284398 shutdown_connections 2024-09-11T22:30:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.802+0000 7f0f0c00a640 1 -- 172.21.15.5:0/2927284398 wait complete. 2024-09-11T22:30:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f0c00a640 1 Processor -- start 2024-09-11T22:30:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f0c00a640 1 -- start start 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f0c00a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f041009d0 0x7f0f041991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f0c00a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f04101370 0x7f0f04199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f0c00a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f041069e0 0x7f0f041a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f0c00a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f04110e20 con 0x7f0f041009d0 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f0c00a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f04110ca0 con 0x7f0f04101370 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f0c00a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0f04110fa0 con 0x7f0f041069e0 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f09da6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f041009d0 0x7f0f041991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f095a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f04101370 0x7f0f04199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f095a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f04101370 0x7f0f04199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38796/0 (socket says 172.21.15.5:38796) 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f09da6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f041009d0 0x7f0f041991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52790/0 (socket says 172.21.15.5:52790) 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f095a5640 1 -- 172.21.15.5:0/3521916161 learned_addr learned my addr 172.21.15.5:0/3521916161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f041069e0 0x7f0f041a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f09da6640 1 -- 172.21.15.5:0/3521916161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f041069e0 msgr2=0x7f0f041a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f09da6640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f041069e0 0x7f0f041a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f09da6640 1 -- 172.21.15.5:0/3521916161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f04101370 msgr2=0x7f0f04199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f09da6640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f04101370 0x7f0f04199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f09da6640 1 -- 172.21.15.5:0/3521916161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f041a4780 con 0x7f0f041009d0 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.803+0000 7f0f095a5640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f04101370 0x7f0f04199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.804+0000 7f0f09da6640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f041009d0 0x7f0f041991c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e9f0 tx=0x7f0f0000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.804+0000 7f0ef6ffd640 1 -- 172.21.15.5:0/3521916161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000ccc0 con 0x7f0f041009d0 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.804+0000 7f0f0c00a640 1 -- 172.21.15.5:0/3521916161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f041a4950 con 0x7f0f041009d0 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.804+0000 7f0ef6ffd640 1 -- 172.21.15.5:0/3521916161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f00010910 con 0x7f0f041009d0 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.804+0000 7f0ef6ffd640 1 -- 172.21.15.5:0/3521916161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f00018960 con 0x7f0f041009d0 2024-09-11T22:30:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.804+0000 7f0f0c00a640 1 -- 172.21.15.5:0/3521916161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f041a4ec0 con 0x7f0f041009d0 2024-09-11T22:30:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.806+0000 7f0ef6ffd640 1 -- 172.21.15.5:0/3521916161 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0f00010ac0 con 0x7f0f041009d0 2024-09-11T22:30:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0ee77740 1 Processor -- start 2024-09-11T22:30:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0ee77740 1 -- start start 2024-09-11T22:30:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d92908ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908d410 0x55d929097770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d929097cb0 0x55d92909a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d928ff12f0 con 0x55d92908caf0 2024-09-11T22:30:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d928f99270 con 0x55d929097cb0 2024-09-11T22:30:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d928fed2b0 con 0x55d92908d410 2024-09-11T22:30:12.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0a5a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d92908ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f09da6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908d410 0x55d929097770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f095a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d929097cb0 0x55d92909a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0a5a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d92908ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52796/0 (socket says 172.21.15.5:52796) 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.807+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/3357922892 learned_addr learned my addr 172.21.15.5:0/3357922892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0ef6ffd640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ef00a2da0 0x7f0ef00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/3357922892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908d410 msgr2=0x55d929097770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/3357922892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908d410 0x55d929097770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/3357922892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d929097cb0 msgr2=0x55d92909a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/3357922892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d929097cb0 0x55d92909a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/3357922892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d928f98340 con 0x55d92908caf0 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0ef6ffd640 1 -- 172.21.15.5:0/3521916161 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f0f000cf830 con 0x7f0f041009d0 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ef00a2da0 0x7f0ef00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/3357922892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d92908ced0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b880 tx=0x7f0efc00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fffdc58edf2bb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0eeffff640 1 -- 172.21.15.5:0/3357922892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc024070 con 0x55d92908caf0 2024-09-11T22:30:12.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0eeffff640 1 -- 172.21.15.5:0/3357922892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0efc010b20 con 0x55d92908caf0 2024-09-11T22:30:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ef00a2da0 0x7f0ef00a5260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0efc021000 tx=0x7f0efc01e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.808+0000 7f0eeffff640 1 -- 172.21.15.5:0/3357922892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc011990 con 0x55d92908caf0 2024-09-11T22:30:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.809+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3357922892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 msgr2=0x55d92908ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.809+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/3357922892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d92908ced0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00b880 tx=0x7f0efc00bd40 comp rx=0 tx=0).stop 2024-09-11T22:30:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.809+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3357922892 shutdown_connections 2024-09-11T22:30:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.809+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/3357922892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d929097cb0 0x55d92909a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.809+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/3357922892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908d410 0x55d929097770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.809+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/3357922892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d92908ced0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.809+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3357922892 >> 172.21.15.5:0/3357922892 conn(0x55d929089060 msgr2=0x55d929089440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:12.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.809+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3357922892 shutdown_connections 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.810+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3357922892 wait complete. 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.810+0000 7f0f0ee77740 1 -- start start 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.810+0000 7f0f0ee77740 1 Processor -- start 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d929094b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d92908d410 0x55d9290950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d929097cb0 0x55d9290955e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d92909a880 con 0x55d92908caf0 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d928f7a290 con 0x55d92908d410 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d92909aa00 con 0x55d929097cb0 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f09da6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d929094b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0ee77740 10 client.0 _ll_register_callbacks cb 0x55d928fab140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f09da6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d929094b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52806/0 (socket says 172.21.15.5:52806) 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f09da6640 1 -- 172.21.15.5:0/915902873 learned_addr learned my addr 172.21.15.5:0/915902873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d929097cb0 0x55d9290955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f095a5640 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d92908d410 0x55d9290950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/915902873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d92908d410 msgr2=0x55d9290950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d92908d410 0x55d9290950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/915902873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 msgr2=0x55d929094b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d929094b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/915902873 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d928d04210 con 0x55d929097cb0 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.811+0000 7f0f09da6640 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d929094b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.812+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d929097cb0 0x55d9290955e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0efc01ff50 tx=0x7f0efc004210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17ab90a6229d5d90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.812+0000 7f0eed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.812+0000 7f0eed7fa640 1 -- 172.21.15.5:0/915902873 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc024040 con 0x55d929097cb0 2024-09-11T22:30:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.812+0000 7f0eed7fa640 1 -- 172.21.15.5:0/915902873 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f0efc010640 con 0x55d929097cb0 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.812+0000 7f0f0ee77740 1 -- 172.21.15.5:0/915902873 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0efc010b20 con 0x55d929097cb0 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.812+0000 7f0eed7fa640 1 -- 172.21.15.5:0/915902873 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0efc010840 con 0x55d929097cb0 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.812+0000 7f0f0ee77740 10 client.27124 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.812+0000 7f0f0ee77740 1 -- 172.21.15.5:0/915902873 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d928dd0620 con 0x55d929097cb0 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.812+0000 7f0eed7fa640 1 -- 172.21.15.5:0/915902873 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0efc010b20 con 0x55d929097cb0 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.813+0000 7f0eed7fa640 1 -- 172.21.15.5:0/915902873 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f0efc030020 con 0x55d929097cb0 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.813+0000 7f0f0ee77740 10 client.27124 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.813+0000 7f0f0ee77740 10 client.27124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.813+0000 7f0f0ee77740 20 client.27124 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.813+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9290abf10 0x55d9290ae300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.813+0000 7f0f0ee77740 4 client.27124 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.813+0000 7f0f0ee77740 1 -- 172.21.15.5:0/915902873 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d928ffea20 con 0x55d9290abf10 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.813+0000 7f0f09da6640 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9290abf10 0x55d9290ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.813+0000 7f0eed7fa640 10 client.27124 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:12.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.813+0000 7f0f09da6640 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9290abf10 0x55d9290ae300 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e2aeb152eb55be8 server_cookie=328f467a6e7bc612 in_seq=0 out_seq=0 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0eed7fa640 1 -- 172.21.15.5:0/915902873 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d928ffea20 con 0x55d9290abf10 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0eed7fa640 10 client.27124 handle_command_reply: tid=0 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0f0ee77740 1 client.27124 shutdown 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0f0ee77740 1 -- 172.21.15.5:0/915902873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d929097cb0 msgr2=0x55d9290955e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d929097cb0 0x55d9290955e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0efc01ff50 tx=0x7f0efc004210 comp rx=0 tx=0).stop 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0f0ee77740 1 -- 172.21.15.5:0/915902873 shutdown_connections 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9290abf10 0x55d9290ae300 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d929097cb0 0x55d9290955e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d92908d410 0x55d9290950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0eed7fa640 0 client.27124 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/915902873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908caf0 0x55d929094b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.814+0000 7f0f0ee77740 1 -- 172.21.15.5:0/915902873 >> 172.21.15.5:0/915902873 conn(0x55d929089060 msgr2=0x55d92908ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.815+0000 7f0f0ee77740 1 -- 172.21.15.5:0/915902873 shutdown_connections 2024-09-11T22:30:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.815+0000 7f0f0ee77740 1 -- 172.21.15.5:0/915902873 wait complete. 2024-09-11T22:30:12.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.815+0000 7f0f0ee77740 20 client.27124 trim_cache size 0 max 16384 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f0ee77740 1 Processor -- start 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f0ee77740 1 -- start start 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 0x55d92908f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908d410 0x55d9290953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d929097cb0 0x55d9290880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041ab8e0 con 0x55d92908d410 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041ad420 con 0x55d929097cb0 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041abfb0 con 0x55d92908caf0 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f095a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908d410 0x55d9290953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f095a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908d410 0x55d9290953e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52814/0 (socket says 172.21.15.5:52814) 2024-09-11T22:30:12.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f095a5640 1 -- 172.21.15.5:0/1400317617 learned_addr learned my addr 172.21.15.5:0/1400317617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/1400317617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d929097cb0 0x55d9290880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f09da6640 1 --2- 172.21.15.5:0/1400317617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 0x55d92908f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f095a5640 1 -- 172.21.15.5:0/1400317617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 msgr2=0x55d92908f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f095a5640 1 --2- 172.21.15.5:0/1400317617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 0x55d92908f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f095a5640 1 -- 172.21.15.5:0/1400317617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d929097cb0 msgr2=0x55d9290880b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f095a5640 1 --2- 172.21.15.5:0/1400317617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d929097cb0 0x55d9290880b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.830+0000 7f0f095a5640 1 -- 172.21.15.5:0/1400317617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d929090700 con 0x55d92908d410 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.831+0000 7f0f095a5640 1 --2- 172.21.15.5:0/1400317617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908d410 0x55d9290953e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0ef802ae00 tx=0x7f0ef8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447085bc5cf72924 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.831+0000 7f0eef7fe640 1 -- 172.21.15.5:0/1400317617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef80098d0 con 0x55d92908d410 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.831+0000 7f0eef7fe640 1 -- 172.21.15.5:0/1400317617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef8036040 con 0x55d92908d410 2024-09-11T22:30:12.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.831+0000 7f0eef7fe640 1 -- 172.21.15.5:0/1400317617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef8030800 con 0x55d92908d410 2024-09-11T22:30:12.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 -- 172.21.15.5:0/1400317617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908d410 msgr2=0x55d9290953e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/1400317617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908d410 0x55d9290953e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0ef802ae00 tx=0x7f0ef8004a40 comp rx=0 tx=0).stop 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 -- 172.21.15.5:0/1400317617 shutdown_connections 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/1400317617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d929097cb0 0x55d9290880b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/1400317617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d92908d410 0x55d9290953e0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/1400317617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 0x55d92908f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 -- 172.21.15.5:0/1400317617 >> 172.21.15.5:0/1400317617 conn(0x55d929089060 msgr2=0x55d9290879a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 -- 172.21.15.5:0/1400317617 shutdown_connections 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 -- 172.21.15.5:0/1400317617 wait complete. 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 -- start start 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.832+0000 7f0f0ee77740 1 Processor -- start 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 0x55d92909adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d92908d410 0x55d92909b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0ee77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d929097cb0 0x55d9291149b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041acd50 con 0x55d929097cb0 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041ac680 con 0x55d92908d410 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0ee77740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0f041a98e0 con 0x55d92908caf0 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0ee77740 10 client.0 _ll_register_callbacks cb 0x7f0efc022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0a5a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d929097cb0 0x55d9291149b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0a5a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d929097cb0 0x55d9291149b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52826/0 (socket says 172.21.15.5:52826) 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f09da6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 0x55d92909adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/3987893444 learned_addr learned my addr 172.21.15.5:0/3987893444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f095a5640 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d92908d410 0x55d92909b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/3987893444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 msgr2=0x55d92909adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:12.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 0x55d92909adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/3987893444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d92908d410 msgr2=0x55d92909b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d92908d410 0x55d92909b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0a5a7640 1 -- 172.21.15.5:0/3987893444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d929114ef0 con 0x55d929097cb0 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f09da6640 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 0x55d92909adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0f0a5a7640 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d929097cb0 0x55d9291149b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00bfd0 tx=0x7f0efc010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0ec25435b6dd50 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.833+0000 7f0eed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0eed7fa640 1 -- 172.21.15.5:0/3987893444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0efc024040 con 0x55d929097cb0 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0eed7fa640 1 -- 172.21.15.5:0/3987893444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f0efc01f230 con 0x55d929097cb0 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3987893444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ef8036040 con 0x55d929097cb0 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0eed7fa640 1 -- 172.21.15.5:0/3987893444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0efc047050 con 0x55d929097cb0 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0f0ee77740 10 client.17862 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3987893444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d929116060 con 0x55d929097cb0 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0eed7fa640 1 -- 172.21.15.5:0/3987893444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ef8036040 con 0x55d929097cb0 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0eed7fa640 1 -- 172.21.15.5:0/3987893444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f0efc030040 con 0x55d929097cb0 2024-09-11T22:30:12.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0f0ee77740 10 client.17862 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0f0ee77740 10 client.17862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0f0ee77740 20 client.17862 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9291166b0 0x55d929118aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0f0ee77740 4 client.17862 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:30:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.834+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3987893444 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d928ff8d70 con 0x55d9291166b0 2024-09-11T22:30:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.835+0000 7f0f09da6640 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9291166b0 0x55d929118aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.835+0000 7f0f09da6640 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9291166b0 0x55d929118aa0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ab15d571a2f234 server_cookie=feb753f670f9402a in_seq=0 out_seq=0 2024-09-11T22:30:12.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.835+0000 7f0eed7fa640 10 client.17862 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0eed7fa640 1 -- 172.21.15.5:0/3987893444 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d928ff8d70 con 0x55d9291166b0 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0eed7fa640 10 client.17862 handle_command_reply: tid=0 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0f0ee77740 1 client.17862 shutdown 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3987893444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d929097cb0 msgr2=0x55d9291149b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d929097cb0 0x55d9291149b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00bfd0 tx=0x7f0efc010f60 comp rx=0 tx=0).stop 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3987893444 shutdown_connections 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d9291166b0 0x55d929118aa0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0eed7fa640 0 client.17862 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d929097cb0 0x55d9291149b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d92908d410 0x55d92909b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0f0ee77740 1 --2- 172.21.15.5:0/3987893444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d92908caf0 0x55d92909adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3987893444 >> 172.21.15.5:0/3987893444 conn(0x55d929089060 msgr2=0x55d9290881c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.955+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3987893444 shutdown_connections 2024-09-11T22:30:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.956+0000 7f0f0ee77740 1 -- 172.21.15.5:0/3987893444 wait complete. 2024-09-11T22:30:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.956+0000 7f0f0ee77740 20 client.17862 trim_cache size 0 max 16384 2024-09-11T22:30:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:30:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:30:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:30:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:30:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.956+0000 7f0f0c00a640 1 -- 172.21.15.5:0/3521916161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ef00a2da0 msgr2=0x7f0ef00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.956+0000 7f0f0c00a640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ef00a2da0 0x7f0ef00a5260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0efc021000 tx=0x7f0efc01e000 comp rx=0 tx=0).stop 2024-09-11T22:30:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.956+0000 7f0f0c00a640 1 -- 172.21.15.5:0/3521916161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f041009d0 msgr2=0x7f0f041991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.956+0000 7f0f0c00a640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f041009d0 0x7f0f041991c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000e9f0 tx=0x7f0f0000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.957+0000 7f0f0c00a640 1 -- 172.21.15.5:0/3521916161 shutdown_connections 2024-09-11T22:30:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.957+0000 7f0f0c00a640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f041069e0 0x7f0f041a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.957+0000 7f0f0c00a640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ef00a2da0 0x7f0ef00a5260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.957+0000 7f0f0c00a640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f04101370 0x7f0f04199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.957+0000 7f0f0c00a640 1 --2- 172.21.15.5:0/3521916161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f041009d0 0x7f0f041991c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.957+0000 7f0f0c00a640 1 -- 172.21.15.5:0/3521916161 >> 172.21.15.5:0/3521916161 conn(0x7f0f040fc6b0 msgr2=0x7f0f040fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.957+0000 7f0f0c00a640 1 -- 172.21.15.5:0/3521916161 shutdown_connections 2024-09-11T22:30:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:12.957+0000 7f0f0c00a640 1 -- 172.21.15.5:0/3521916161 wait complete. 2024-09-11T22:30:13.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 621744 8.17 MB/sec execute 271 sec latency 61.014 ms 2024-09-11T22:30:13.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:13 smithi159 ceph-mon[35534]: pgmap v250: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-09-11T22:30:13.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:13 smithi159 ceph-mon[35534]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T22:30:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:13 smithi005 ceph-mon[30943]: pgmap v250: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-09-11T22:30:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:13 smithi005 ceph-mon[30943]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T22:30:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:13 smithi172 ceph-mon[35461]: pgmap v250: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-09-11T22:30:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:13 smithi172 ceph-mon[35461]: osdmap e93: 12 total, 12 up, 12 in 2024-09-11T22:30:14.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 623727 8.16 MB/sec execute 272 sec latency 15.202 ms 2024-09-11T22:30:15.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 625887 8.18 MB/sec execute 273 sec latency 16.122 ms 2024-09-11T22:30:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:15 smithi005 ceph-mon[30943]: pgmap v252: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-09-11T22:30:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:15 smithi159 ceph-mon[35534]: pgmap v252: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-09-11T22:30:15.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:15 smithi172 ceph-mon[35461]: pgmap v252: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-09-11T22:30:16.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 628593 8.21 MB/sec execute 274 sec latency 15.651 ms 2024-09-11T22:30:16.386 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T22:30:16.386 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-11T22:30:16.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:30:16.656 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:16.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:30:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.204+0000 7f27ae739640 1 -- 172.21.15.5:0/262436363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a8102ef0 msgr2=0x7f27a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.204+0000 7f27ae739640 1 --2- 172.21.15.5:0/262436363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a8102ef0 0x7f27a810da40 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009ca0 tx=0x7f27a002a950 comp rx=0 tx=0).stop 2024-09-11T22:30:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.204+0000 7f27ae739640 1 -- 172.21.15.5:0/262436363 shutdown_connections 2024-09-11T22:30:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.204+0000 7f27ae739640 1 --2- 172.21.15.5:0/262436363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a8102ef0 0x7f27a810da40 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.204+0000 7f27ae739640 1 --2- 172.21.15.5:0/262436363 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a8102570 0x7f27a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.204+0000 7f27ae739640 1 --2- 172.21.15.5:0/262436363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a8100e20 0x7f27a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.204+0000 7f27ae739640 1 -- 172.21.15.5:0/262436363 >> 172.21.15.5:0/262436363 conn(0x7f27a80fc7d0 msgr2=0x7f27a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.204+0000 7f27ae739640 1 -- 172.21.15.5:0/262436363 shutdown_connections 2024-09-11T22:30:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.204+0000 7f27ae739640 1 -- 172.21.15.5:0/262436363 wait complete. 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.205+0000 7f27ae739640 1 Processor -- start 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.205+0000 7f27ae739640 1 -- start start 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27ae739640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a8100e20 0x7f27a8107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27ae739640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a8102570 0x7f27a8108020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27ae739640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a8102ef0 0x7f27a8108560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27ae739640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27a8110f80 con 0x7f27a8102ef0 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27ae739640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27a8110e00 con 0x7f27a8100e20 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27ae739640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27a8111100 con 0x7f27a8102570 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27accd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a8102ef0 0x7f27a8108560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27a77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a8102570 0x7f27a8108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27a77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a8102570 0x7f27a8108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:39042/0 (socket says 172.21.15.5:39042) 2024-09-11T22:30:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27a77fe640 1 -- 172.21.15.5:0/4004417867 learned_addr learned my addr 172.21.15.5:0/4004417867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27accd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a8102ef0 0x7f27a8108560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52846/0 (socket says 172.21.15.5:52846) 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.206+0000 7f27a7fff640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a8100e20 0x7f27a8107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.207+0000 7f27accd6640 1 -- 172.21.15.5:0/4004417867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a8102570 msgr2=0x7f27a8108020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.207+0000 7f27accd6640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a8102570 0x7f27a8108020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.207+0000 7f27accd6640 1 -- 172.21.15.5:0/4004417867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a8100e20 msgr2=0x7f27a8107ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.207+0000 7f27accd6640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a8100e20 0x7f27a8107ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.207+0000 7f27accd6640 1 -- 172.21.15.5:0/4004417867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a81a6950 con 0x7f27a8102ef0 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.207+0000 7f27accd6640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a8102ef0 0x7f27a8108560 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f27a0005cc0 tx=0x7f27a002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.207+0000 7f27a57fa640 1 -- 172.21.15.5:0/4004417867 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0034040 con 0x7f27a8102ef0 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.207+0000 7f27a57fa640 1 -- 172.21.15.5:0/4004417867 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27a0030920 con 0x7f27a8102ef0 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.207+0000 7f27a57fa640 1 -- 172.21.15.5:0/4004417867 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0004600 con 0x7f27a8102ef0 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.208+0000 7f27ae739640 1 -- 172.21.15.5:0/4004417867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a81a6be0 con 0x7f27a8102ef0 2024-09-11T22:30:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.208+0000 7f27ae739640 1 -- 172.21.15.5:0/4004417867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a81a7070 con 0x7f27a8102ef0 2024-09-11T22:30:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.210+0000 7f27ae739640 1 -- 172.21.15.5:0/4004417867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a8101200 con 0x7f27a8102ef0 2024-09-11T22:30:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.211+0000 7f27a57fa640 1 -- 172.21.15.5:0/4004417867 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f27a0037020 con 0x7f27a8102ef0 2024-09-11T22:30:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.212+0000 7f27a57fa640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27900a2e40 0x7f27900a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.212+0000 7f27a57fa640 1 -- 172.21.15.5:0/4004417867 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f27a00f0120 con 0x7f27a8102ef0 2024-09-11T22:30:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.214+0000 7f27a7fff640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27900a2e40 0x7f27900a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.214+0000 7f27a7fff640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27900a2e40 0x7f27900a5300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2798009be0 tx=0x7f2798005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.219+0000 7f27a57fa640 1 -- 172.21.15.5:0/4004417867 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27a00bcde0 con 0x7f27a8102ef0 2024-09-11T22:30:17.299 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:17.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 630176 8.21 MB/sec execute 275 sec latency 30.369 ms 2024-09-11T22:30:17.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.455+0000 7f27ae739640 1 -- 172.21.15.5:0/4004417867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27a81a72a0 con 0x7f27a8102ef0 2024-09-11T22:30:17.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.456+0000 7f27a57fa640 1 -- 172.21.15.5:0/4004417867 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11967 (secure 0 0 0) 0x7f27a00098b0 con 0x7f27a8102ef0 2024-09-11T22:30:17.457 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:30:17.457 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"btime":"2024-09-11T22:29:21:436884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:29:21.436856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26914":{"gid":26914,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1755675305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1755675305},{"type":"v1","addr":"172.21.15.159:6833","nonce":1755675305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:30:17.457 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 40 2024-09-11T22:30:17.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 -- 172.21.15.5:0/4004417867 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27900a2e40 msgr2=0x7f27900a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:17.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27900a2e40 0x7f27900a5300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2798009be0 tx=0x7f2798005f90 comp rx=0 tx=0).stop 2024-09-11T22:30:17.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 -- 172.21.15.5:0/4004417867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a8102ef0 msgr2=0x7f27a8108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:17.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a8102ef0 0x7f27a8108560 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f27a0005cc0 tx=0x7f27a002f3a0 comp rx=0 tx=0).stop 2024-09-11T22:30:17.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 -- 172.21.15.5:0/4004417867 shutdown_connections 2024-09-11T22:30:17.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27a8102ef0 0x7f27a8108560 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:17.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a8102570 0x7f27a8108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:17.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27900a2e40 0x7f27900a5300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:17.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 --2- 172.21.15.5:0/4004417867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27a8100e20 0x7f27a8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:17.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 -- 172.21.15.5:0/4004417867 >> 172.21.15.5:0/4004417867 conn(0x7f27a80fc7d0 msgr2=0x7f27a8103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:17.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 -- 172.21.15.5:0/4004417867 shutdown_connections 2024-09-11T22:30:17.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:17.458+0000 7f27ae739640 1 -- 172.21.15.5:0/4004417867 wait complete. 2024-09-11T22:30:17.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:17 smithi005 ceph-mon[30943]: pgmap v253: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-09-11T22:30:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:17 smithi159 ceph-mon[35534]: pgmap v253: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-09-11T22:30:17.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:17 smithi172 ceph-mon[35461]: pgmap v253: 97 pgs: 97 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.2 MiB/s wr, 191 op/s 2024-09-11T22:30:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.151+0000 7fc5fab0c640 1 -- 172.21.15.5:0/895258806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f4103780 msgr2=0x7fc5f4107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.151+0000 7fc5fab0c640 1 --2- 172.21.15.5:0/895258806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f4103780 0x7fc5f4107bc0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009cd0 tx=0x7fc5e402a8d0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.151+0000 7fc5fab0c640 1 -- 172.21.15.5:0/895258806 shutdown_connections 2024-09-11T22:30:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.151+0000 7fc5fab0c640 1 --2- 172.21.15.5:0/895258806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f4108300 0x7fc5f410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.151+0000 7fc5fab0c640 1 --2- 172.21.15.5:0/895258806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f4103780 0x7fc5f4107bc0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.151+0000 7fc5fab0c640 1 --2- 172.21.15.5:0/895258806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f4102dd0 0x7fc5f41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.151+0000 7fc5fab0c640 1 -- 172.21.15.5:0/895258806 >> 172.21.15.5:0/895258806 conn(0x7fc5f40fc7d0 msgr2=0x7fc5f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.151+0000 7fc5fab0c640 1 -- 172.21.15.5:0/895258806 shutdown_connections 2024-09-11T22:30:18.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.151+0000 7fc5fab0c640 1 -- 172.21.15.5:0/895258806 wait complete. 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5fab0c640 1 Processor -- start 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5fab0c640 1 -- start start 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5fab0c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f4102dd0 0x7fc5f41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5fab0c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f4103780 0x7fc5f4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5fab0c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f4108300 0x7fc5f419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5fab0c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f4112050 con 0x7fc5f4108300 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5fab0c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f4111ed0 con 0x7fc5f4102dd0 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5fab0c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41121d0 con 0x7fc5f4103780 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5f90a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f4108300 0x7fc5f419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5f88a8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f4102dd0 0x7fc5f41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5f90a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f4108300 0x7fc5f419cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52866/0 (socket says 172.21.15.5:52866) 2024-09-11T22:30:18.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f4103780 0x7fc5f4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5f90a9640 1 -- 172.21.15.5:0/95262594 learned_addr learned my addr 172.21.15.5:0/95262594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5f90a9640 1 -- 172.21.15.5:0/95262594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f4103780 msgr2=0x7fc5f4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5f90a9640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f4103780 0x7fc5f4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5f90a9640 1 -- 172.21.15.5:0/95262594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f4102dd0 msgr2=0x7fc5f41985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5f90a9640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f4102dd0 0x7fc5f41985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.152+0000 7fc5f90a9640 1 -- 172.21.15.5:0/95262594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f419d580 con 0x7fc5f4108300 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.153+0000 7fc5f90a9640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f4108300 0x7fc5f419cea0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00e970 tx=0x7fc5dc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.153+0000 7fc5f1ffb640 1 -- 172.21.15.5:0/95262594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00ccc0 con 0x7fc5f4108300 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.153+0000 7fc5fab0c640 1 -- 172.21.15.5:0/95262594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f419d870 con 0x7fc5f4108300 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.153+0000 7fc5f1ffb640 1 -- 172.21.15.5:0/95262594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5dc00ce60 con 0x7fc5f4108300 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.153+0000 7fc5f1ffb640 1 -- 172.21.15.5:0/95262594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc010490 con 0x7fc5f4108300 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.153+0000 7fc5fab0c640 1 -- 172.21.15.5:0/95262594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f41a5150 con 0x7fc5f4108300 2024-09-11T22:30:18.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.155+0000 7fc5f1ffb640 1 -- 172.21.15.5:0/95262594 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc5dc005020 con 0x7fc5f4108300 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5fcee5740 1 Processor -- start 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5fcee5740 1 -- start start 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5f1ffb640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5e00a2cf0 0x7fc5e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 0x560945814760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945823550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945825e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56094577a2e0 con 0x560945823a90 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560945721ce0 con 0x560945814380 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5609457762a0 con 0x560945814ca0 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5f3fff640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5e00a2cf0 0x7fc5e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5f88a8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 0x560945814760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5f88a8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 0x560945814760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38866/0 (socket says 172.21.15.5:38866) 2024-09-11T22:30:18.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5f88a8640 1 -- 172.21.15.5:0/1260966147 learned_addr learned my addr 172.21.15.5:0/1260966147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5f3fff640 1 --2- 172.21.15.5:0/1260966147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945823550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5f1ffb640 1 -- 172.21.15.5:0/95262594 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fc5dc0cf960 con 0x7fc5f4108300 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.156+0000 7fc5f90a9640 1 --2- 172.21.15.5:0/1260966147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945825e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5f88a8640 1 -- 172.21.15.5:0/1260966147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 msgr2=0x560945823550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/1260966147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945823550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5f88a8640 1 -- 172.21.15.5:0/1260966147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 msgr2=0x560945825e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/1260966147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945825e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5f88a8640 1 -- 172.21.15.5:0/1260966147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560945706440 con 0x560945814380 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5f90a9640 1 --2- 172.21.15.5:0/1260966147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945825e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/1260966147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 0x560945814760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009cf0 tx=0x7fc5ec011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fc986522ffe2224 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5f3fff640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5e00a2cf0 0x7fc5e00a51b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4006010 tx=0x7fc5e4031000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5eaffd640 1 -- 172.21.15.5:0/1260966147 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec004230 con 0x560945814380 2024-09-11T22:30:18.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5eaffd640 1 -- 172.21.15.5:0/1260966147 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec0043d0 con 0x560945814380 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.157+0000 7fc5eaffd640 1 -- 172.21.15.5:0/1260966147 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0198c0 con 0x560945814380 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 -- 172.21.15.5:0/1260966147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 msgr2=0x560945814760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/1260966147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 0x560945814760 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009cf0 tx=0x7fc5ec011810 comp rx=0 tx=0).stop 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 -- 172.21.15.5:0/1260966147 shutdown_connections 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/1260966147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945825e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/1260966147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945823550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/1260966147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 0x560945814760 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 -- 172.21.15.5:0/1260966147 >> 172.21.15.5:0/1260966147 conn(0x560945810a30 msgr2=0x560945810e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 -- 172.21.15.5:0/1260966147 shutdown_connections 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 -- 172.21.15.5:0/1260966147 wait complete. 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 -- start start 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.158+0000 7fc5fcee5740 1 Processor -- start 2024-09-11T22:30:18.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 0x5609458233b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945820660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945820ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560945826660 con 0x560945823a90 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560945735050 con 0x560945814380 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5609458267e0 con 0x560945814ca0 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5fcee5740 10 client.0 _ll_register_callbacks cb 0x560945775160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f90a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945820ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f90a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945820ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52878/0 (socket says 172.21.15.5:52878) 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f88a8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 0x5609458233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f90a9640 1 -- 172.21.15.5:0/2582327876 learned_addr learned my addr 172.21.15.5:0/2582327876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f3fff640 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945820660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f90a9640 1 -- 172.21.15.5:0/2582327876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 msgr2=0x560945820660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:18.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f90a9640 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945820660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f90a9640 1 -- 172.21.15.5:0/2582327876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 msgr2=0x5609458233b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f90a9640 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 0x5609458233b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f90a9640 1 -- 172.21.15.5:0/2582327876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609454eb3c0 con 0x560945823a90 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5f90a9640 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945820ba0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc098780 tx=0x7fc5dc09c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4570b234bf5058 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5d3fff640 1 -- 172.21.15.5:0/2582327876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc0a0160 con 0x560945823a90 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.159+0000 7fc5d3fff640 1 -- 172.21.15.5:0/2582327876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fc5dc005310 con 0x560945823a90 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5fcee5740 1 -- 172.21.15.5:0/2582327876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ec0043d0 con 0x560945823a90 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5d3fff640 1 -- 172.21.15.5:0/2582327876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc5dc09f020 con 0x560945823a90 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5fcee5740 10 client.17880 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5fcee5740 1 -- 172.21.15.5:0/2582327876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560945832020 con 0x560945823a90 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5d3fff640 1 -- 172.21.15.5:0/2582327876 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec0043d0 con 0x560945823a90 2024-09-11T22:30:18.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5d3fff640 1 -- 172.21.15.5:0/2582327876 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc5dc056ba0 con 0x560945823a90 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5fcee5740 10 client.17880 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5fcee5740 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5fcee5740 20 client.17880 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560945833640 0x560945835a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5fcee5740 4 client.17880 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5fcee5740 1 -- 172.21.15.5:0/2582327876 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560945787850 con 0x560945833640 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.160+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560945833640 0x560945835a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.161+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560945833640 0x560945835a30 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1944d4d406ce5cda server_cookie=df6e81a3a24bdb15 in_seq=0 out_seq=0 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.161+0000 7fc5d3fff640 10 client.17880 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5d3fff640 1 -- 172.21.15.5:0/2582327876 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560945787850 con 0x560945833640 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5d3fff640 10 client.17880 handle_command_reply: tid=0 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5fcee5740 1 client.17880 shutdown 2024-09-11T22:30:18.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5fcee5740 1 -- 172.21.15.5:0/2582327876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 msgr2=0x560945820ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945820ba0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc098780 tx=0x7fc5dc09c4b0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5fcee5740 1 -- 172.21.15.5:0/2582327876 shutdown_connections 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560945833640 0x560945835a30 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5d3fff640 0 client.17880 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945823a90 0x560945820ba0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945820660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/2582327876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945814380 0x5609458233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.162+0000 7fc5fcee5740 1 -- 172.21.15.5:0/2582327876 >> 172.21.15.5:0/2582327876 conn(0x560945810a30 msgr2=0x56094580fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.163+0000 7fc5fcee5740 1 -- 172.21.15.5:0/2582327876 shutdown_connections 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.163+0000 7fc5fcee5740 1 -- 172.21.15.5:0/2582327876 wait complete. 2024-09-11T22:30:18.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.163+0000 7fc5fcee5740 20 client.17880 trim_cache size 0 max 16384 2024-09-11T22:30:18.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.182+0000 7fc5fcee5740 1 Processor -- start 2024-09-11T22:30:18.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.182+0000 7fc5fcee5740 1 -- start start 2024-09-11T22:30:18.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x56094582f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x56094581ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945823a90 0x56094583b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41ab8e0 con 0x560945814380 2024-09-11T22:30:18.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41ad420 con 0x560945823a90 2024-09-11T22:30:18.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41abfb0 con 0x560945814ca0 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f88a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x56094582f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f88a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x56094582f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52892/0 (socket says 172.21.15.5:52892) 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f88a8640 1 -- 172.21.15.5:0/3154564480 learned_addr learned my addr 172.21.15.5:0/3154564480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x56094581ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f90a9640 1 --2- 172.21.15.5:0/3154564480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945823a90 0x56094583b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f88a8640 1 -- 172.21.15.5:0/3154564480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 msgr2=0x56094581ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/3154564480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x56094581ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f88a8640 1 -- 172.21.15.5:0/3154564480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945823a90 msgr2=0x56094583b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/3154564480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945823a90 0x56094583b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f88a8640 1 -- 172.21.15.5:0/3154564480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094581f610 con 0x560945814380 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f3fff640 1 --2- 172.21.15.5:0/3154564480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x56094581ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.183+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/3154564480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x56094582f900 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec004140 tx=0x7fc5ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b86a3fc5ad5fa60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:18.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.184+0000 7fc5ea7fc640 1 -- 172.21.15.5:0/3154564480 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec031070 con 0x560945814380 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.184+0000 7fc5ea7fc640 1 -- 172.21.15.5:0/3154564480 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec003ad0 con 0x560945814380 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.184+0000 7fc5ea7fc640 1 -- 172.21.15.5:0/3154564480 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec019a80 con 0x560945814380 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.184+0000 7fc5fcee5740 1 -- 172.21.15.5:0/3154564480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 msgr2=0x56094582f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.184+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/3154564480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x56094582f900 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec004140 tx=0x7fc5ec011810 comp rx=0 tx=0).stop 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.184+0000 7fc5fcee5740 1 -- 172.21.15.5:0/3154564480 shutdown_connections 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.184+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/3154564480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945823a90 0x56094583b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.184+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/3154564480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x56094581ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.184+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/3154564480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x56094582f900 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.184+0000 7fc5fcee5740 1 -- 172.21.15.5:0/3154564480 >> 172.21.15.5:0/3154564480 conn(0x560945810a30 msgr2=0x56094581bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 1 -- 172.21.15.5:0/3154564480 shutdown_connections 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 1 -- 172.21.15.5:0/3154564480 wait complete. 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 1 -- start start 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 1 Processor -- start 2024-09-11T22:30:18.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x560945827af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945828030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945823a90 0x56094589d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41acd50 con 0x560945814380 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41ac680 con 0x560945823a90 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f41a98e0 con 0x560945814ca0 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5fcee5740 10 client.0 _ll_register_callbacks cb 0x7fc5dc009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.185+0000 7fc5f88a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x560945827af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5f88a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x560945827af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52906/0 (socket says 172.21.15.5:52906) 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5f88a8640 1 -- 172.21.15.5:0/309916577 learned_addr learned my addr 172.21.15.5:0/309916577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945828030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5f90a9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945823a90 0x56094589d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5f88a8640 1 -- 172.21.15.5:0/309916577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 msgr2=0x560945828030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:18.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945828030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5f88a8640 1 -- 172.21.15.5:0/309916577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945823a90 msgr2=0x56094589d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945823a90 0x56094589d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5f88a8640 1 -- 172.21.15.5:0/309916577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609458288c0 con 0x560945814380 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5f88a8640 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x560945827af0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec005890 tx=0x7fc5ec009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ca7cddcc7076c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5d3fff640 1 -- 172.21.15.5:0/309916577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec031040 con 0x560945814380 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5d3fff640 1 -- 172.21.15.5:0/309916577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fc5ec003c80 con 0x560945814380 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5fcee5740 1 -- 172.21.15.5:0/309916577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ec003ad0 con 0x560945814380 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5d3fff640 1 -- 172.21.15.5:0/309916577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc5ec0035f0 con 0x560945814380 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5fcee5740 10 client.17886 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.186+0000 7fc5fcee5740 1 -- 172.21.15.5:0/309916577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56094589e680 con 0x560945814380 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.187+0000 7fc5d3fff640 1 -- 172.21.15.5:0/309916577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec003ad0 con 0x560945814380 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.187+0000 7fc5d3fff640 1 -- 172.21.15.5:0/309916577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc5ec03e030 con 0x560945814380 2024-09-11T22:30:18.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.187+0000 7fc5fcee5740 10 client.17886 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:18.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.187+0000 7fc5fcee5740 10 client.17886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:18.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.187+0000 7fc5fcee5740 20 client.17886 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:18.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.187+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560945833640 0x5609458a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:18.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.187+0000 7fc5fcee5740 4 client.17886 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:30:18.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.187+0000 7fc5fcee5740 1 -- 172.21.15.5:0/309916577 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560945781d60 con 0x560945833640 2024-09-11T22:30:18.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.187+0000 7fc5f3fff640 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560945833640 0x5609458a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:18.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5f3fff640 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560945833640 0x5609458a0b00 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78d4f073e4c2614 server_cookie=dc6bf32ef4a8cb2c in_seq=0 out_seq=0 2024-09-11T22:30:18.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5d3fff640 10 client.17886 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:18.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5d3fff640 1 -- 172.21.15.5:0/309916577 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560945781d60 con 0x560945833640 2024-09-11T22:30:18.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5d3fff640 10 client.17886 handle_command_reply: tid=0 2024-09-11T22:30:18.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5fcee5740 1 client.17886 shutdown 2024-09-11T22:30:18.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5fcee5740 1 -- 172.21.15.5:0/309916577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 msgr2=0x560945827af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:18.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x560945827af0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec005890 tx=0x7fc5ec009ea0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5fcee5740 1 -- 172.21.15.5:0/309916577 shutdown_connections 2024-09-11T22:30:18.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560945833640 0x5609458a0b00 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560945823a90 0x56094589d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5d3fff640 0 client.17886 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560945814ca0 0x560945828030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5fcee5740 1 --2- 172.21.15.5:0/309916577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560945814380 0x560945827af0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5fcee5740 1 -- 172.21.15.5:0/309916577 >> 172.21.15.5:0/309916577 conn(0x560945810a30 msgr2=0x56094581bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.188+0000 7fc5fcee5740 1 -- 172.21.15.5:0/309916577 shutdown_connections 2024-09-11T22:30:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fcee5740 1 -- 172.21.15.5:0/309916577 wait complete. 2024-09-11T22:30:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fcee5740 20 client.17886 trim_cache size 0 max 16384 2024-09-11T22:30:18.195 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:30:18.195 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:30:18.195 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:30:18.196 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:30:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 -- 172.21.15.5:0/95262594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5e00a2cf0 msgr2=0x7fc5e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5e00a2cf0 0x7fc5e00a51b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4006010 tx=0x7fc5e4031000 comp rx=0 tx=0).stop 2024-09-11T22:30:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 -- 172.21.15.5:0/95262594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f4108300 msgr2=0x7fc5f419cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f4108300 0x7fc5f419cea0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00e970 tx=0x7fc5dc00ed70 comp rx=0 tx=0).stop 2024-09-11T22:30:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 -- 172.21.15.5:0/95262594 shutdown_connections 2024-09-11T22:30:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5f4108300 0x7fc5f419cea0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f4103780 0x7fc5f4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5e00a2cf0 0x7fc5e00a51b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 --2- 172.21.15.5:0/95262594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5f4102dd0 0x7fc5f41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 -- 172.21.15.5:0/95262594 >> 172.21.15.5:0/95262594 conn(0x7fc5f40fc7d0 msgr2=0x7fc5f4105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 -- 172.21.15.5:0/95262594 shutdown_connections 2024-09-11T22:30:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:18.189+0000 7fc5fab0c640 1 -- 172.21.15.5:0/95262594 wait complete. 2024-09-11T22:30:18.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 632468 8.23 MB/sec execute 276 sec latency 54.473 ms 2024-09-11T22:30:18.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:18 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4004417867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:18.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:18 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4004417867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:18.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:18 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4004417867' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:19.076 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-11T22:30:19.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-11T22:30:19.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 634257 8.22 MB/sec execute 277 sec latency 23.469 ms 2024-09-11T22:30:19.554 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:30:19.554 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:30:19.683 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:19.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:19 smithi005 ceph-mon[30943]: pgmap v254: 97 pgs: 97 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-09-11T22:30:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:19 smithi159 ceph-mon[35534]: pgmap v254: 97 pgs: 97 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-09-11T22:30:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:19 smithi172 ceph-mon[35461]: pgmap v254: 97 pgs: 97 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-09-11T22:30:20.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.221+0000 7fd80301b640 1 -- 172.21.15.5:0/544511855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc100e20 msgr2=0x7fd7fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.221+0000 7fd80301b640 1 --2- 172.21.15.5:0/544511855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc100e20 0x7fd7fc101200 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009ca0 tx=0x7fd7f402a950 comp rx=0 tx=0).stop 2024-09-11T22:30:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.221+0000 7fd80301b640 1 -- 172.21.15.5:0/544511855 shutdown_connections 2024-09-11T22:30:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.221+0000 7fd80301b640 1 --2- 172.21.15.5:0/544511855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7fc102ef0 0x7fd7fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.221+0000 7fd80301b640 1 --2- 172.21.15.5:0/544511855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7fc102570 0x7fd7fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.221+0000 7fd80301b640 1 --2- 172.21.15.5:0/544511855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc100e20 0x7fd7fc101200 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.221+0000 7fd80301b640 1 -- 172.21.15.5:0/544511855 >> 172.21.15.5:0/544511855 conn(0x7fd7fc0fc7d0 msgr2=0x7fd7fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.221+0000 7fd80301b640 1 -- 172.21.15.5:0/544511855 shutdown_connections 2024-09-11T22:30:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.221+0000 7fd80301b640 1 -- 172.21.15.5:0/544511855 wait complete. 2024-09-11T22:30:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd80301b640 1 Processor -- start 2024-09-11T22:30:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd80301b640 1 -- start start 2024-09-11T22:30:20.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd80301b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7fc100e20 0x7fd7fc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd80301b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7fc102570 0x7fd7fc198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd80301b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc102ef0 0x7fd7fc19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd80301b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc110fa0 con 0x7fd7fc102ef0 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd80301b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc110e20 con 0x7fd7fc100e20 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd80301b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc111120 con 0x7fd7fc102570 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd7fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7fc102570 0x7fd7fc198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd7fbfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7fc102570 0x7fd7fc198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:39124/0 (socket says 172.21.15.5:39124) 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.222+0000 7fd7fbfff640 1 -- 172.21.15.5:0/3566951419 learned_addr learned my addr 172.21.15.5:0/3566951419 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd8015b8640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc102ef0 0x7fd7fc19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd800db7640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7fc100e20 0x7fd7fc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd8015b8640 1 -- 172.21.15.5:0/3566951419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7fc102570 msgr2=0x7fd7fc198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd8015b8640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7fc102570 0x7fd7fc198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd8015b8640 1 -- 172.21.15.5:0/3566951419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7fc100e20 msgr2=0x7fd7fc1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd8015b8640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7fc100e20 0x7fd7fc1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd8015b8640 1 -- 172.21.15.5:0/3566951419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc19d4a0 con 0x7fd7fc102ef0 2024-09-11T22:30:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd8015b8640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc102ef0 0x7fd7fc19cdc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00e970 tx=0x7fd7ec00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd7f9ffb640 1 -- 172.21.15.5:0/3566951419 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec00ccc0 con 0x7fd7fc102ef0 2024-09-11T22:30:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd80301b640 1 -- 172.21.15.5:0/3566951419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7fc19d790 con 0x7fd7fc102ef0 2024-09-11T22:30:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd7f9ffb640 1 -- 172.21.15.5:0/3566951419 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec005020 con 0x7fd7fc102ef0 2024-09-11T22:30:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.223+0000 7fd7f9ffb640 1 -- 172.21.15.5:0/3566951419 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec004a00 con 0x7fd7fc102ef0 2024-09-11T22:30:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.224+0000 7fd80301b640 1 -- 172.21.15.5:0/3566951419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7fc1a5070 con 0x7fd7fc102ef0 2024-09-11T22:30:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.226+0000 7fd7f9ffb640 1 -- 172.21.15.5:0/3566951419 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd7ec0051d0 con 0x7fd7fc102ef0 2024-09-11T22:30:20.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.226+0000 7fd8053f4740 1 Processor -- start 2024-09-11T22:30:20.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd8053f4740 1 -- start start 2024-09-11T22:30:20.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd7f9ffb640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd7e40a2cf0 0x7fd7e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8640 0x55b877fb8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8f60 0x55b877fc7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 0x55b877fca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd8053f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b877f1e2a0 con 0x55b877fb8f60 2024-09-11T22:30:20.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd8053f4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b877ec5b10 con 0x55b877fb8640 2024-09-11T22:30:20.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd8053f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b877f1a260 con 0x55b877fc7d50 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd7fbfff640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd7e40a2cf0 0x7fd7e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd7f9ffb640 1 -- 172.21.15.5:0/3566951419 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fd7ec0cf490 con 0x7fd7fc102ef0 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd800db7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8640 0x55b877fb8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd800db7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8640 0x55b877fb8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38920/0 (socket says 172.21.15.5:38920) 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd800db7640 1 -- 172.21.15.5:0/1453313909 learned_addr learned my addr 172.21.15.5:0/1453313909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd8015b8640 1 --2- 172.21.15.5:0/1453313909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 0x55b877fca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd7fbfff640 1 --2- 172.21.15.5:0/1453313909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8f60 0x55b877fc7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd800db7640 1 -- 172.21.15.5:0/1453313909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 msgr2=0x55b877fca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd800db7640 1 --2- 172.21.15.5:0/1453313909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 0x55b877fca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd800db7640 1 -- 172.21.15.5:0/1453313909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8f60 msgr2=0x55b877fc7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd800db7640 1 --2- 172.21.15.5:0/1453313909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8f60 0x55b877fc7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.227+0000 7fd800db7640 1 -- 172.21.15.5:0/1453313909 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b877c8fb90 con 0x55b877fb8640 2024-09-11T22:30:20.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.228+0000 7fd8015b8640 1 --2- 172.21.15.5:0/1453313909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 0x55b877fca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:20.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.228+0000 7fd7fbfff640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd7e40a2cf0 0x7fd7e40a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00097f0 tx=0x7fd7f00114f0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:20.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.228+0000 7fd800db7640 1 --2- 172.21.15.5:0/1453313909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8640 0x55b877fb8a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd7f402ae70 tx=0x7fd7f4005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=742ba1e4e876ba9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:20.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.228+0000 7fd7e2ffd640 1 -- 172.21.15.5:0/1453313909 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4048070 con 0x55b877fb8640 2024-09-11T22:30:20.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.228+0000 7fd7e2ffd640 1 -- 172.21.15.5:0/1453313909 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7f4003e50 con 0x55b877fb8640 2024-09-11T22:30:20.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.228+0000 7fd7e2ffd640 1 -- 172.21.15.5:0/1453313909 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4034040 con 0x55b877fb8640 2024-09-11T22:30:20.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 -- 172.21.15.5:0/1453313909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8640 msgr2=0x55b877fb8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 --2- 172.21.15.5:0/1453313909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8640 0x55b877fb8a20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd7f402ae70 tx=0x7fd7f4005f80 comp rx=0 tx=0).stop 2024-09-11T22:30:20.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 -- 172.21.15.5:0/1453313909 shutdown_connections 2024-09-11T22:30:20.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 --2- 172.21.15.5:0/1453313909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 0x55b877fca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 --2- 172.21.15.5:0/1453313909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8f60 0x55b877fc7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 --2- 172.21.15.5:0/1453313909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8640 0x55b877fb8a20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 -- 172.21.15.5:0/1453313909 >> 172.21.15.5:0/1453313909 conn(0x55b877fb4cf0 msgr2=0x55b877fb50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 -- 172.21.15.5:0/1453313909 shutdown_connections 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 -- 172.21.15.5:0/1453313909 wait complete. 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 -- start start 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.229+0000 7fd8053f4740 1 Processor -- start 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fc4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 0x55b877fc51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fc7d50 0x55b877fc5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8053f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b877fca920 con 0x55b877fc7d50 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8053f4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b877ea66e0 con 0x55b877fb8f60 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8053f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b877fcaaa0 con 0x55b877fb8640 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8053f4740 10 client.0 _ll_register_callbacks cb 0x55b877ecaa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:20.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8015b8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fc7d50 0x55b877fc5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd7fbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 0x55b877fc51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd800db7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fc4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8015b8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fc7d50 0x55b877fc5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52936/0 (socket says 172.21.15.5:52936) 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8015b8640 1 -- 172.21.15.5:0/319566312 learned_addr learned my addr 172.21.15.5:0/319566312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8015b8640 1 -- 172.21.15.5:0/319566312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 msgr2=0x55b877fc4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8015b8640 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fc4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8015b8640 1 -- 172.21.15.5:0/319566312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 msgr2=0x55b877fc51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8015b8640 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 0x55b877fc51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd8015b8640 1 -- 172.21.15.5:0/319566312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b877fd3ed0 con 0x55b877fc7d50 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd800db7640 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fc4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.230+0000 7fd7fbfff640 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 0x55b877fc51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd8015b8640 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fc7d50 0x55b877fc5730 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0ccb70 tx=0x7fd7ec09b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914208464754d86c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:20.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd7dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd7dbfff640 1 -- 172.21.15.5:0/319566312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec01c040 con 0x55b877fc7d50 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd8053f4740 1 -- 172.21.15.5:0/319566312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7f4003e50 con 0x55b877fc7d50 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd7dbfff640 1 -- 172.21.15.5:0/319566312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fd7ec055ba0 con 0x55b877fc7d50 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd7dbfff640 1 -- 172.21.15.5:0/319566312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd7ec0a4550 con 0x55b877fc7d50 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd8053f4740 10 client.17898 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd8053f4740 1 -- 172.21.15.5:0/319566312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b877fd68c0 con 0x55b877fc7d50 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd7dbfff640 1 -- 172.21.15.5:0/319566312 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7f4003e50 con 0x55b877fc7d50 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd7dbfff640 1 -- 172.21.15.5:0/319566312 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fd7ec004ba0 con 0x55b877fc7d50 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd8053f4740 10 client.17898 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd8053f4740 10 client.17898 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd8053f4740 20 client.17898 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:20.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd8053f4740 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b877fd7d20 0x55b877fda190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd8053f4740 4 client.17898 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.231+0000 7fd8053f4740 1 -- 172.21.15.5:0/319566312 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b877f2b9d0 con 0x55b877fd7d20 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.232+0000 7fd800db7640 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b877fd7d20 0x55b877fda190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.232+0000 7fd7dbfff640 10 client.17898 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.232+0000 7fd800db7640 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b877fd7d20 0x55b877fda190 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d263db25eda70d server_cookie=9fa15973b56ca6d1 in_seq=0 out_seq=0 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd7dbfff640 1 -- 172.21.15.5:0/319566312 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b877f2b9d0 con 0x55b877fd7d20 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd7dbfff640 10 client.17898 handle_command_reply: tid=0 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 client.17898 shutdown 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 -- 172.21.15.5:0/319566312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fc7d50 msgr2=0x55b877fc5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fc7d50 0x55b877fc5730 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0ccb70 tx=0x7fd7ec09b8e0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 -- 172.21.15.5:0/319566312 shutdown_connections 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b877fd7d20 0x55b877fda190 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fc7d50 0x55b877fc5730 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 0x55b877fc51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd7dbfff640 0 client.17898 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:20.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 --2- 172.21.15.5:0/319566312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fc4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 -- 172.21.15.5:0/319566312 >> 172.21.15.5:0/319566312 conn(0x55b877fb4cf0 msgr2=0x55b877fb3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:20.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 -- 172.21.15.5:0/319566312 shutdown_connections 2024-09-11T22:30:20.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.233+0000 7fd8053f4740 1 -- 172.21.15.5:0/319566312 wait complete. 2024-09-11T22:30:20.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.234+0000 7fd8053f4740 20 client.17898 trim_cache size 0 max 16384 2024-09-11T22:30:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8053f4740 1 Processor -- start 2024-09-11T22:30:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8053f4740 1 -- start start 2024-09-11T22:30:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8640 0x55b877fb3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 0x55b877fdfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 0x55b877fdffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8053f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1ab7e0 con 0x55b877fb8640 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8053f4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1ad320 con 0x55b877fb8f60 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8053f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1abeb0 con 0x55b877fc7d50 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd800db7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8640 0x55b877fb3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd800db7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8640 0x55b877fb3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52944/0 (socket says 172.21.15.5:52944) 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd800db7640 1 -- 172.21.15.5:0/4051363133 learned_addr learned my addr 172.21.15.5:0/4051363133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8015b8640 1 --2- 172.21.15.5:0/4051363133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 0x55b877fdffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd7fbfff640 1 --2- 172.21.15.5:0/4051363133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 0x55b877fdfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8015b8640 1 -- 172.21.15.5:0/4051363133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 msgr2=0x55b877fdfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8015b8640 1 --2- 172.21.15.5:0/4051363133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 0x55b877fdfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8015b8640 1 -- 172.21.15.5:0/4051363133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8640 msgr2=0x55b877fb3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8015b8640 1 --2- 172.21.15.5:0/4051363133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8640 0x55b877fb3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd8015b8640 1 -- 172.21.15.5:0/4051363133 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b877fb8d90 con 0x55b877fc7d50 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.252+0000 7fd800db7640 1 --2- 172.21.15.5:0/4051363133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8640 0x55b877fb3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd8015b8640 1 --2- 172.21.15.5:0/4051363133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 0x55b877fdffa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0a4020 tx=0x7fd7ec099df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=202fb6ff38700706 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd7e27fc640 1 -- 172.21.15.5:0/4051363133 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec01c040 con 0x55b877fc7d50 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd7e27fc640 1 -- 172.21.15.5:0/4051363133 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec0860e0 con 0x55b877fc7d50 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd7e27fc640 1 -- 172.21.15.5:0/4051363133 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec0a40c0 con 0x55b877fc7d50 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd8053f4740 1 -- 172.21.15.5:0/4051363133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 msgr2=0x55b877fdffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd8053f4740 1 --2- 172.21.15.5:0/4051363133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 0x55b877fdffa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec0a4020 tx=0x7fd7ec099df0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd8053f4740 1 -- 172.21.15.5:0/4051363133 shutdown_connections 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd8053f4740 1 --2- 172.21.15.5:0/4051363133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fc7d50 0x55b877fdffa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd8053f4740 1 --2- 172.21.15.5:0/4051363133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fb8f60 0x55b877fdfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd8053f4740 1 --2- 172.21.15.5:0/4051363133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8640 0x55b877fb3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd8053f4740 1 -- 172.21.15.5:0/4051363133 >> 172.21.15.5:0/4051363133 conn(0x55b877fb4cf0 msgr2=0x55b877fb3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:20.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd8053f4740 1 -- 172.21.15.5:0/4051363133 shutdown_connections 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.253+0000 7fd8053f4740 1 -- 172.21.15.5:0/4051363133 wait complete. 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd8053f4740 1 -- start start 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd8053f4740 1 Processor -- start 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fcc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8f60 0x55b877fcc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd8053f4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fc7d50 0x55b878041a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd8053f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1acc50 con 0x55b877fb8f60 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd8053f4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1ac580 con 0x55b877fc7d50 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd8053f4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1a97e0 con 0x55b877fb8640 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd8053f4740 10 client.0 _ll_register_callbacks cb 0x7fd7ec09bfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd7fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8f60 0x55b877fcc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.254+0000 7fd800db7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fcc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd8015b8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fc7d50 0x55b878041a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd800db7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fcc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:39162/0 (socket says 172.21.15.5:39162) 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd800db7640 1 -- 172.21.15.5:0/274632477 learned_addr learned my addr 172.21.15.5:0/274632477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd800db7640 1 -- 172.21.15.5:0/274632477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fc7d50 msgr2=0x55b878041a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd800db7640 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fc7d50 0x55b878041a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd800db7640 1 -- 172.21.15.5:0/274632477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8f60 msgr2=0x55b877fcc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd800db7640 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8f60 0x55b877fcc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd800db7640 1 -- 172.21.15.5:0/274632477 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b878041fa0 con 0x55b877fb8640 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd8015b8640 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fc7d50 0x55b878041a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd800db7640 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fcc240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd7f402af40 tx=0x7fd7f402fc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c3bd3b426a1a2d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd7dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd7dbfff640 1 -- 172.21.15.5:0/274632477 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4048040 con 0x55b877fb8640 2024-09-11T22:30:20.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd7dbfff640 1 -- 172.21.15.5:0/274632477 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fd7f4003970 con 0x55b877fb8640 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.255+0000 7fd8053f4740 1 -- 172.21.15.5:0/274632477 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7ec0860e0 con 0x55b877fb8640 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd7dbfff640 1 -- 172.21.15.5:0/274632477 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd7f4034040 con 0x55b877fb8640 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd8053f4740 10 client.27172 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd8053f4740 1 -- 172.21.15.5:0/274632477 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b878043260 con 0x55b877fb8640 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd7dbfff640 1 -- 172.21.15.5:0/274632477 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7ec0860e0 con 0x55b877fb8640 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd7dbfff640 1 -- 172.21.15.5:0/274632477 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fd7f4050460 con 0x55b877fb8640 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd8053f4740 10 client.27172 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd8053f4740 10 client.27172 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd8053f4740 20 client.27172 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd8053f4740 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b877fd7d20 0x55b8780455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd8053f4740 4 client.27172 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd8053f4740 1 -- 172.21.15.5:0/274632477 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x55b877f25d20 con 0x55b877fd7d20 2024-09-11T22:30:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.256+0000 7fd7fbfff640 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b877fd7d20 0x55b8780455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.257+0000 7fd7fbfff640 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b877fd7d20 0x55b8780455b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca99566307a1930 server_cookie=f4ebb39be90e8c75 in_seq=0 out_seq=0 2024-09-11T22:30:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.257+0000 7fd7dbfff640 10 client.27172 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:20.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636261 8.24 MB/sec execute 278 sec latency 55.245 ms 2024-09-11T22:30:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.361+0000 7fd7dbfff640 1 -- 172.21.15.5:0/274632477 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b877f25d20 con 0x55b877fd7d20 2024-09-11T22:30:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.361+0000 7fd7dbfff640 10 client.27172 handle_command_reply: tid=0 2024-09-11T22:30:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.361+0000 7fd8053f4740 1 client.27172 shutdown 2024-09-11T22:30:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 1 -- 172.21.15.5:0/274632477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 msgr2=0x55b877fcc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fcc240 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd7f402af40 tx=0x7fd7f402fc50 comp rx=0 tx=0).stop 2024-09-11T22:30:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 1 -- 172.21.15.5:0/274632477 shutdown_connections 2024-09-11T22:30:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b877fd7d20 0x55b8780455b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b877fc7d50 0x55b878041a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b877fb8f60 0x55b877fcc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 1 --2- 172.21.15.5:0/274632477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b877fb8640 0x55b877fcc240 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd7dbfff640 0 client.27172 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 1 -- 172.21.15.5:0/274632477 >> 172.21.15.5:0/274632477 conn(0x55b877fb4cf0 msgr2=0x55b877fc5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 1 -- 172.21.15.5:0/274632477 shutdown_connections 2024-09-11T22:30:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 1 -- 172.21.15.5:0/274632477 wait complete. 2024-09-11T22:30:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.362+0000 7fd8053f4740 20 client.27172 trim_cache size 0 max 16384 2024-09-11T22:30:20.364 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:30:20.364 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 40, 2024-09-11T22:30:20.364 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:30:20.364 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 30, 2024-09-11T22:30:20.364 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "b21ec0ac": { 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 30, 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.4, 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:30:20.365 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:30:20.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:30:20.366 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:30:20.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:20.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:20.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:20.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:20.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:20.366 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:30:20.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 -- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd7e40a2cf0 msgr2=0x7fd7e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd7e40a2cf0 0x7fd7e40a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00097f0 tx=0x7fd7f00114f0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 -- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc102ef0 msgr2=0x7fd7fc19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc102ef0 0x7fd7fc19cdc0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00e970 tx=0x7fd7ec00ee30 comp rx=0 tx=0).stop 2024-09-11T22:30:20.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 -- 172.21.15.5:0/3566951419 shutdown_connections 2024-09-11T22:30:20.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7fc102ef0 0x7fd7fc19cdc0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7fc102570 0x7fd7fc198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd7e40a2cf0 0x7fd7e40a51b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 --2- 172.21.15.5:0/3566951419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7fc100e20 0x7fd7fc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 -- 172.21.15.5:0/3566951419 >> 172.21.15.5:0/3566951419 conn(0x7fd7fc0fc7d0 msgr2=0x7fd7fc06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:20.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 -- 172.21.15.5:0/3566951419 shutdown_connections 2024-09-11T22:30:20.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.363+0000 7fd80301b640 1 -- 172.21.15.5:0/3566951419 wait complete. 2024-09-11T22:30:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.851+0000 7fd17e2cf640 1 -- 172.21.15.5:0/917472342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd178102ef0 msgr2=0x7fd17810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.851+0000 7fd17e2cf640 1 --2- 172.21.15.5:0/917472342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd178102ef0 0x7fd17810da40 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd170009ca0 tx=0x7fd17002a950 comp rx=0 tx=0).stop 2024-09-11T22:30:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.851+0000 7fd17e2cf640 1 -- 172.21.15.5:0/917472342 shutdown_connections 2024-09-11T22:30:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.851+0000 7fd17e2cf640 1 --2- 172.21.15.5:0/917472342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd178102ef0 0x7fd17810da40 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.851+0000 7fd17e2cf640 1 --2- 172.21.15.5:0/917472342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd178102570 0x7fd1781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.851+0000 7fd17e2cf640 1 --2- 172.21.15.5:0/917472342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd178100e20 0x7fd178101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.851+0000 7fd17e2cf640 1 -- 172.21.15.5:0/917472342 >> 172.21.15.5:0/917472342 conn(0x7fd1780fc7d0 msgr2=0x7fd1780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.851+0000 7fd17e2cf640 1 -- 172.21.15.5:0/917472342 shutdown_connections 2024-09-11T22:30:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.851+0000 7fd17e2cf640 1 -- 172.21.15.5:0/917472342 wait complete. 2024-09-11T22:30:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17e2cf640 1 Processor -- start 2024-09-11T22:30:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17e2cf640 1 -- start start 2024-09-11T22:30:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17e2cf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd178100e20 0x7fd17819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17e2cf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd178102570 0x7fd17819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17e2cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd178102ef0 0x7fd17819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17e2cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd178110fa0 con 0x7fd178102ef0 2024-09-11T22:30:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17e2cf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd178110e20 con 0x7fd178102570 2024-09-11T22:30:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17e2cf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd178111120 con 0x7fd178100e20 2024-09-11T22:30:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17c86c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd178102ef0 0x7fd17819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17c86c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd178102ef0 0x7fd17819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52974/0 (socket says 172.21.15.5:52974) 2024-09-11T22:30:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17c86c640 1 -- 172.21.15.5:0/2776061703 learned_addr learned my addr 172.21.15.5:0/2776061703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd16ffff640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd178102570 0x7fd17819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd177fff640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd178100e20 0x7fd17819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17c86c640 1 -- 172.21.15.5:0/2776061703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd178100e20 msgr2=0x7fd17819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17c86c640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd178100e20 0x7fd17819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17c86c640 1 -- 172.21.15.5:0/2776061703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd178102570 msgr2=0x7fd17819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17c86c640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd178102570 0x7fd17819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17c86c640 1 -- 172.21.15.5:0/2776061703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17819f690 con 0x7fd178102ef0 2024-09-11T22:30:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.852+0000 7fd17c86c640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd178102ef0 0x7fd17819efb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd17002ae70 tx=0x7fd17002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.853+0000 7fd175ffb640 1 -- 172.21.15.5:0/2776061703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170034040 con 0x7fd178102ef0 2024-09-11T22:30:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.853+0000 7fd175ffb640 1 -- 172.21.15.5:0/2776061703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd170030920 con 0x7fd178102ef0 2024-09-11T22:30:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.853+0000 7fd17e2cf640 1 -- 172.21.15.5:0/2776061703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17819f920 con 0x7fd178102ef0 2024-09-11T22:30:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.853+0000 7fd175ffb640 1 -- 172.21.15.5:0/2776061703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1700047f0 con 0x7fd178102ef0 2024-09-11T22:30:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.853+0000 7fd17e2cf640 1 -- 172.21.15.5:0/2776061703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1781a7260 con 0x7fd178102ef0 2024-09-11T22:30:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.854+0000 7fd17e2cf640 1 -- 172.21.15.5:0/2776061703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd178101200 con 0x7fd178102ef0 2024-09-11T22:30:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.855+0000 7fd175ffb640 1 -- 172.21.15.5:0/2776061703 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd170037020 con 0x7fd178102ef0 2024-09-11T22:30:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.856+0000 7fd175ffb640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1640a2cf0 0x7fd1640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.856+0000 7fd177fff640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1640a2cf0 0x7fd1640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.856+0000 7fd175ffb640 1 -- 172.21.15.5:0/2776061703 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fd1700f02c0 con 0x7fd178102ef0 2024-09-11T22:30:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.856+0000 7fd177fff640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1640a2cf0 0x7fd1640a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd16800bbe0 tx=0x7fd16800b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:20.859+0000 7fd175ffb640 1 -- 172.21.15.5:0/2776061703 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1700bcf80 con 0x7fd178102ef0 2024-09-11T22:30:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.071+0000 7fd17e2cf640 1 -- 172.21.15.5:0/2776061703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1781a7430 con 0x7fd178102ef0 2024-09-11T22:30:21.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.072+0000 7fd175ffb640 1 -- 172.21.15.5:0/2776061703 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11967 (secure 0 0 0) 0x7fd1700bc880 con 0x7fd178102ef0 2024-09-11T22:30:21.074 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:30:21.075 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"btime":"2024-09-11T22:29:21:436884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:29:21.436856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26914":{"gid":26914,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1755675305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1755675305},{"type":"v1","addr":"172.21.15.159:6833","nonce":1755675305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:30:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 40 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 -- 172.21.15.5:0/2776061703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1640a2cf0 msgr2=0x7fd1640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1640a2cf0 0x7fd1640a51b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd16800bbe0 tx=0x7fd16800b390 comp rx=0 tx=0).stop 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 -- 172.21.15.5:0/2776061703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd178102ef0 msgr2=0x7fd17819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd178102ef0 0x7fd17819efb0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd17002ae70 tx=0x7fd17002f5d0 comp rx=0 tx=0).stop 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 -- 172.21.15.5:0/2776061703 shutdown_connections 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd178102ef0 0x7fd17819efb0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1640a2cf0 0x7fd1640a51b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd178102570 0x7fd17819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 --2- 172.21.15.5:0/2776061703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd178100e20 0x7fd17819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 -- 172.21.15.5:0/2776061703 >> 172.21.15.5:0/2776061703 conn(0x7fd1780fc7d0 msgr2=0x7fd178069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.074+0000 7fd17e2cf640 1 -- 172.21.15.5:0/2776061703 shutdown_connections 2024-09-11T22:30:21.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:21.075+0000 7fd17e2cf640 1 -- 172.21.15.5:0/2776061703 wait complete. 2024-09-11T22:30:21.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 8.21 MB/sec execute 279 sec latency 971.634 ms 2024-09-11T22:30:21.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:21 smithi005 ceph-mon[30943]: pgmap v255: 97 pgs: 97 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-09-11T22:30:21.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:21 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2776061703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:21.605 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b21ec0ac', quiesce took 5.2 seconds. Will release after: 14.2 2024-09-11T22:30:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:21 smithi159 ceph-mon[35534]: pgmap v255: 97 pgs: 97 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-09-11T22:30:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:21 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2776061703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:21 smithi172 ceph-mon[35461]: pgmap v255: 97 pgs: 97 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.8 MiB/s wr, 322 op/s 2024-09-11T22:30:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:21 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2776061703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:22.330 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 8.18 MB/sec execute 280 sec latency 1971.663 ms 2024-09-11T22:30:22.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:30:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.161+0000 7f13fc386640 1 -- 172.21.15.5:0/3061623122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4102790 msgr2=0x7f13f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.161+0000 7f13fc386640 1 --2- 172.21.15.5:0/3061623122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4102790 0x7f13f41067e0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009900 tx=0x7f13e402a410 comp rx=0 tx=0).stop 2024-09-11T22:30:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.161+0000 7f13fc386640 1 -- 172.21.15.5:0/3061623122 shutdown_connections 2024-09-11T22:30:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.161+0000 7f13fc386640 1 --2- 172.21.15.5:0/3061623122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13f4106f10 0x7f13f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.161+0000 7f13fc386640 1 --2- 172.21.15.5:0/3061623122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4102790 0x7f13f41067e0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.161+0000 7f13fc386640 1 --2- 172.21.15.5:0/3061623122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f4101de0 0x7f13f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.161+0000 7f13fc386640 1 -- 172.21.15.5:0/3061623122 >> 172.21.15.5:0/3061623122 conn(0x7f13f40fd8c0 msgr2=0x7f13f40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.161+0000 7f13fc386640 1 -- 172.21.15.5:0/3061623122 shutdown_connections 2024-09-11T22:30:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.161+0000 7f13fc386640 1 -- 172.21.15.5:0/3061623122 wait complete. 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fc386640 1 Processor -- start 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fc386640 1 -- start start 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fc386640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4101de0 0x7f13f4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fc386640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f4102790 0x7f13f41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fc386640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13f4106f10 0x7f13f419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fc386640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4111f30 con 0x7f13f4101de0 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fc386640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4111db0 con 0x7f13f4106f10 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fc386640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41120b0 con 0x7f13f4102790 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa122640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4101de0 0x7f13f4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa122640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4101de0 0x7f13f4198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43118/0 (socket says 172.21.15.5:43118) 2024-09-11T22:30:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa122640 1 -- 172.21.15.5:0/3207762790 learned_addr learned my addr 172.21.15.5:0/3207762790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa923640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13f4106f10 0x7f13f419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13f9921640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f4102790 0x7f13f41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa122640 1 -- 172.21.15.5:0/3207762790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f4102790 msgr2=0x7f13f41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa122640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f4102790 0x7f13f41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa122640 1 -- 172.21.15.5:0/3207762790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13f4106f10 msgr2=0x7f13f419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa122640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13f4106f10 0x7f13f419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa122640 1 -- 172.21.15.5:0/3207762790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f419d330 con 0x7f13f4101de0 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa923640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13f4106f10 0x7f13f419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13f9921640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f4102790 0x7f13f41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.162+0000 7f13fa122640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4101de0 0x7f13f4198380 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f13f000c920 tx=0x7f13f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.163+0000 7f13eb7fe640 1 -- 172.21.15.5:0/3207762790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0007c70 con 0x7f13f4101de0 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.163+0000 7f13fc386640 1 -- 172.21.15.5:0/3207762790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f419d620 con 0x7f13f4101de0 2024-09-11T22:30:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.163+0000 7f13eb7fe640 1 -- 172.21.15.5:0/3207762790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f0007e10 con 0x7f13f4101de0 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.163+0000 7f13eb7fe640 1 -- 172.21.15.5:0/3207762790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0018890 con 0x7f13f4101de0 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.163+0000 7f13fc386640 1 -- 172.21.15.5:0/3207762790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f41a4f00 con 0x7f13f4101de0 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.165+0000 7f13eb7fe640 1 -- 172.21.15.5:0/3207762790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f13f0018a70 con 0x7f13f4101de0 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.165+0000 7f13fe75f740 1 Processor -- start 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fe75f740 1 -- start start 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a107090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a107a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a116780 0x560c0a118b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fe75f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c0a06d2f0 con 0x560c0a108b40 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fe75f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560c0a014cf0 con 0x560c0a1075d0 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fe75f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560c0a0692b0 con 0x560c0a116780 2024-09-11T22:30:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fa923640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fa923640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a107090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43126/0 (socket says 172.21.15.5:43126) 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fa923640 1 -- 172.21.15.5:0/2708686780 learned_addr learned my addr 172.21.15.5:0/2708686780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fa122640 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a107a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13eb7fe640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13e00a2cf0 0x7f13e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fa923640 1 -- 172.21.15.5:0/2708686780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a116780 msgr2=0x560c0a118b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13f9921640 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a116780 0x560c0a118b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13eb7fe640 1 -- 172.21.15.5:0/3207762790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f13f00cf6c0 con 0x7f13f4101de0 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fa923640 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a116780 0x560c0a118b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fa923640 1 -- 172.21.15.5:0/2708686780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 msgr2=0x560c0a107a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fa923640 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a107a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fa923640 1 -- 172.21.15.5:0/2708686780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c09ff93a0 con 0x560c0a108b40 2024-09-11T22:30:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13fa122640 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.166+0000 7f13f9921640 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a116780 0x560c0a118b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.167+0000 7f13fa923640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13e00a2cf0 0x7f13e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.167+0000 7f13fa923640 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a107090 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f13ec010420 tx=0x7f13ec0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3896e869948de857 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.167+0000 7f13e8dea640 1 -- 172.21.15.5:0/2708686780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec023070 con 0x560c0a108b40 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.167+0000 7f13e8dea640 1 -- 172.21.15.5:0/2708686780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ec01c470 con 0x560c0a108b40 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.167+0000 7f13e8dea640 1 -- 172.21.15.5:0/2708686780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec01d680 con 0x560c0a108b40 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.167+0000 7f13fa923640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13e00a2cf0 0x7f13e00a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13ec020000 tx=0x7f13ec01a000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 -- 172.21.15.5:0/2708686780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 msgr2=0x560c0a107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a107090 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f13ec010420 tx=0x7f13ec0108e0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 -- 172.21.15.5:0/2708686780 shutdown_connections 2024-09-11T22:30:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a116780 0x560c0a118b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2708686780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a107090 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 -- 172.21.15.5:0/2708686780 >> 172.21.15.5:0/2708686780 conn(0x560c0a0d0310 msgr2=0x560c0a103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 -- 172.21.15.5:0/2708686780 shutdown_connections 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 -- 172.21.15.5:0/2708686780 wait complete. 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 -- start start 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 Processor -- start 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a116140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a108b40 0x560c0a1133f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a116780 0x560c0a1139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560c0a119480 con 0x560c0a116780 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560c0a028060 con 0x560c0a1075d0 2024-09-11T22:30:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.168+0000 7f13fe75f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560c0a119600 con 0x560c0a108b40 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fe75f740 10 client.0 _ll_register_callbacks cb 0x560c09d7b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa923640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a116780 0x560c0a1139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa923640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a116780 0x560c0a1139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43134/0 (socket says 172.21.15.5:43134) 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa923640 1 -- 172.21.15.5:0/1751634861 learned_addr learned my addr 172.21.15.5:0/1751634861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13f9921640 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a108b40 0x560c0a1133f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa122640 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa923640 1 -- 172.21.15.5:0/1751634861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a108b40 msgr2=0x560c0a1133f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa923640 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a108b40 0x560c0a1133f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa923640 1 -- 172.21.15.5:0/1751634861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 msgr2=0x560c0a116140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa923640 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a116140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa923640 1 -- 172.21.15.5:0/1751634861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c09dde3c0 con 0x560c0a116780 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa122640 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fa923640 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a116780 0x560c0a1139e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f13ec01d720 tx=0x7f13ec01d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb34ed70d2d2e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13ddffb640 1 -- 172.21.15.5:0/1751634861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec023030 con 0x560c0a116780 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fe75f740 1 -- 172.21.15.5:0/1751634861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ec01c470 con 0x560c0a116780 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13ddffb640 1 -- 172.21.15.5:0/1751634861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f13ec01d920 con 0x560c0a116780 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13ddffb640 1 -- 172.21.15.5:0/1751634861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f13ec01ca30 con 0x560c0a116780 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.169+0000 7f13fe75f740 10 client.17922 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.170+0000 7f13fe75f740 1 -- 172.21.15.5:0/1751634861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560c0a124ee0 con 0x560c0a116780 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.170+0000 7f13ddffb640 1 -- 172.21.15.5:0/1751634861 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ec01c470 con 0x560c0a116780 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.170+0000 7f13ddffb640 1 -- 172.21.15.5:0/1751634861 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f13ec02f020 con 0x560c0a116780 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.170+0000 7f13fe75f740 10 client.17922 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.170+0000 7f13fe75f740 10 client.17922 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.170+0000 7f13fe75f740 20 client.17922 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.170+0000 7f13fe75f740 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x560c0a126230 0x560c0a128620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.170+0000 7f13fe75f740 4 client.17922 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.170+0000 7f13fe75f740 1 -- 172.21.15.5:0/1751634861 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c0a07a860 con 0x560c0a126230 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.170+0000 7f13fa122640 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x560c0a126230 0x560c0a128620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.171+0000 7f13fa122640 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x560c0a126230 0x560c0a128620 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ad63bc6e284380d7 server_cookie=2db5fb5156360c3f in_seq=0 out_seq=0 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.171+0000 7f13ddffb640 10 client.17922 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.171+0000 7f13ddffb640 1 -- 172.21.15.5:0/1751634861 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c0a07a860 con 0x560c0a126230 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.171+0000 7f13ddffb640 10 client.17922 handle_command_reply: tid=0 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.171+0000 7f13fe75f740 1 client.17922 shutdown 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.171+0000 7f13fe75f740 1 -- 172.21.15.5:0/1751634861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a116780 msgr2=0x560c0a1139e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.171+0000 7f13fe75f740 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a116780 0x560c0a1139e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f13ec01d720 tx=0x7f13ec01d750 comp rx=0 tx=0).stop 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.171+0000 7f13fe75f740 1 -- 172.21.15.5:0/1751634861 shutdown_connections 2024-09-11T22:30:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.171+0000 7f13fe75f740 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x560c0a126230 0x560c0a128620 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.172+0000 7f13ddffb640 0 client.17922 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:30:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.172+0000 7f13fe75f740 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a116780 0x560c0a1139e0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.172+0000 7f13fe75f740 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a108b40 0x560c0a1133f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.172+0000 7f13fe75f740 1 --2- 172.21.15.5:0/1751634861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.172+0000 7f13fe75f740 1 -- 172.21.15.5:0/1751634861 >> 172.21.15.5:0/1751634861 conn(0x560c0a0d0310 msgr2=0x560c0a109960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.172+0000 7f13fe75f740 1 -- 172.21.15.5:0/1751634861 shutdown_connections 2024-09-11T22:30:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.172+0000 7f13fe75f740 1 -- 172.21.15.5:0/1751634861 wait complete. 2024-09-11T22:30:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.172+0000 7f13fe75f740 20 client.17922 trim_cache size 0 max 16384 2024-09-11T22:30:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.184+0000 7f13fe75f740 1 Processor -- start 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13fe75f740 1 -- start start 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a122720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a12de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a116780 0x560c0a12e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13fe75f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41ab840 con 0x560c0a108b40 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13fe75f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41ad380 con 0x560c0a1075d0 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13fe75f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41abf10 con 0x560c0a116780 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13f9921640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a12de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13f9921640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a12de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43142/0 (socket says 172.21.15.5:43142) 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13f9921640 1 -- 172.21.15.5:0/2057083229 learned_addr learned my addr 172.21.15.5:0/2057083229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13fa122640 1 --2- 172.21.15.5:0/2057083229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a122720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13f9921640 1 -- 172.21.15.5:0/2057083229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a116780 msgr2=0x560c0a12e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13f9921640 1 --2- 172.21.15.5:0/2057083229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a116780 0x560c0a12e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13f9921640 1 -- 172.21.15.5:0/2057083229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 msgr2=0x560c0a122720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13f9921640 1 --2- 172.21.15.5:0/2057083229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a122720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13f9921640 1 -- 172.21.15.5:0/2057083229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c0a1164b0 con 0x560c0a108b40 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13f9921640 1 --2- 172.21.15.5:0/2057083229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a12de50 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009230 tx=0x7f13e4032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3152951986ca73 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.185+0000 7f13dffff640 1 -- 172.21.15.5:0/2057083229 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e4035040 con 0x560c0a108b40 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13dffff640 1 -- 172.21.15.5:0/2057083229 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e40317b0 con 0x560c0a108b40 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13dffff640 1 -- 172.21.15.5:0/2057083229 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e40046c0 con 0x560c0a108b40 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 -- 172.21.15.5:0/2057083229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 msgr2=0x560c0a12de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2057083229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a12de50 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f13e4009230 tx=0x7f13e4032d70 comp rx=0 tx=0).stop 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 -- 172.21.15.5:0/2057083229 shutdown_connections 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2057083229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a116780 0x560c0a12e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2057083229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a108b40 0x560c0a12de50 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2057083229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a1075d0 0x560c0a122720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 -- 172.21.15.5:0/2057083229 >> 172.21.15.5:0/2057083229 conn(0x560c0a0d0310 msgr2=0x560c0a115470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 -- 172.21.15.5:0/2057083229 shutdown_connections 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 -- 172.21.15.5:0/2057083229 wait complete. 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 -- start start 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.186+0000 7f13fe75f740 1 Processor -- start 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a1075d0 0x560c0a11ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a108b40 0x560c0a11b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fe75f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a116780 0x560c0a189df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fe75f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41accb0 con 0x560c0a1075d0 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fe75f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41ac5e0 con 0x560c0a116780 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fe75f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f13f41a9840 con 0x560c0a108b40 2024-09-11T22:30:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fe75f740 10 client.0 _ll_register_callbacks cb 0x7f13ec013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fa122640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a1075d0 0x560c0a11ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fa122640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a1075d0 0x560c0a11ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43144/0 (socket says 172.21.15.5:43144) 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fa122640 1 -- 172.21.15.5:0/2535738069 learned_addr learned my addr 172.21.15.5:0/2535738069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fa923640 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a116780 0x560c0a189df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fa122640 1 -- 172.21.15.5:0/2535738069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a108b40 msgr2=0x560c0a11b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fa122640 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a108b40 0x560c0a11b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fa122640 1 -- 172.21.15.5:0/2535738069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a116780 msgr2=0x560c0a189df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fa122640 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a116780 0x560c0a189df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fa122640 1 -- 172.21.15.5:0/2535738069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c0a18a430 con 0x560c0a1075d0 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fa122640 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a1075d0 0x560c0a11ae60 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f13f0097fd0 tx=0x7f13f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b1adcbd34e02fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13ddffb640 1 -- 172.21.15.5:0/2535738069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f009ff20 con 0x560c0a1075d0 2024-09-11T22:30:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13ddffb640 1 -- 172.21.15.5:0/2535738069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f13f000fb30 con 0x560c0a1075d0 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.187+0000 7f13fe75f740 1 -- 172.21.15.5:0/2535738069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e40317b0 con 0x560c0a1075d0 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13ddffb640 1 -- 172.21.15.5:0/2535738069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f13f00a47e0 con 0x560c0a1075d0 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13fe75f740 10 client.17928 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13fe75f740 1 -- 172.21.15.5:0/2535738069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560c0a191800 con 0x560c0a1075d0 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13ddffb640 1 -- 172.21.15.5:0/2535738069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e40317b0 con 0x560c0a1075d0 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13ddffb640 1 -- 172.21.15.5:0/2535738069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f13f00a4350 con 0x560c0a1075d0 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13fe75f740 10 client.17928 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13fe75f740 10 client.17928 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13fe75f740 20 client.17928 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x560c0a126230 0x560c0a193f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13fe75f740 4 client.17928 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13fe75f740 1 -- 172.21.15.5:0/2535738069 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c0a074d70 con 0x560c0a126230 2024-09-11T22:30:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.188+0000 7f13f9921640 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x560c0a126230 0x560c0a193f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13ddffb640 10 client.17928 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13f9921640 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x560c0a126230 0x560c0a193f90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=970ecbeebca4a694 server_cookie=3ea10b52835940e3 in_seq=0 out_seq=0 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13ddffb640 1 -- 172.21.15.5:0/2535738069 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c0a074d70 con 0x560c0a126230 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13ddffb640 10 client.17928 handle_command_reply: tid=0 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 client.17928 shutdown 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 -- 172.21.15.5:0/2535738069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a1075d0 msgr2=0x560c0a11ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a1075d0 0x560c0a11ae60 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f13f0097fd0 tx=0x7f13f0000bc0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 -- 172.21.15.5:0/2535738069 shutdown_connections 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x560c0a126230 0x560c0a193f90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560c0a116780 0x560c0a189df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560c0a108b40 0x560c0a11b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13ddffb640 0 client.17928 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:30:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 --2- 172.21.15.5:0/2535738069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c0a1075d0 0x560c0a11ae60 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 -- 172.21.15.5:0/2535738069 >> 172.21.15.5:0/2535738069 conn(0x560c0a0d0310 msgr2=0x560c0a111b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 -- 172.21.15.5:0/2535738069 shutdown_connections 2024-09-11T22:30:23.213 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:30:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 1 -- 172.21.15.5:0/2535738069 wait complete. 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.189+0000 7f13fe75f740 20 client.17928 trim_cache size 0 max 16384 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 -- 172.21.15.5:0/3207762790 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13e00a2cf0 msgr2=0x7f13e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13e00a2cf0 0x7f13e00a51b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f13ec020000 tx=0x7f13ec01a000 comp rx=0 tx=0).stop 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 -- 172.21.15.5:0/3207762790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4101de0 msgr2=0x7f13f4198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4101de0 0x7f13f4198380 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f13f000c920 tx=0x7f13f000cde0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 -- 172.21.15.5:0/3207762790 shutdown_connections 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13e00a2cf0 0x7f13e00a51b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13f4106f10 0x7f13f419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f4102790 0x7f13f41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 --2- 172.21.15.5:0/3207762790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13f4101de0 0x7f13f4198380 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 -- 172.21.15.5:0/3207762790 >> 172.21.15.5:0/3207762790 conn(0x7f13f40fd8c0 msgr2=0x7f13f40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 -- 172.21.15.5:0/3207762790 shutdown_connections 2024-09-11T22:30:23.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:23.190+0000 7f13fc386640 1 -- 172.21.15.5:0/3207762790 wait complete. 2024-09-11T22:30:23.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 8.15 MB/sec execute 281 sec latency 2972.005 ms 2024-09-11T22:30:23.751 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:23 smithi005 ceph-mon[30943]: pgmap v256: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 315 op/s 2024-09-11T22:30:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:23 smithi159 ceph-mon[35534]: pgmap v256: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 315 op/s 2024-09-11T22:30:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:23 smithi172 ceph-mon[35461]: pgmap v256: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 315 op/s 2024-09-11T22:30:24.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:30:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:24.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 8.12 MB/sec execute 282 sec latency 3972.045 ms 2024-09-11T22:30:24.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.788+0000 7ff0da03b640 1 -- 172.21.15.5:0/469522467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d41036e0 msgr2=0x7ff0d4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:24.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.788+0000 7ff0da03b640 1 --2- 172.21.15.5:0/469522467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d41036e0 0x7ff0d4107b90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009930 tx=0x7ff0c802a420 comp rx=0 tx=0).stop 2024-09-11T22:30:24.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.788+0000 7ff0da03b640 1 -- 172.21.15.5:0/469522467 shutdown_connections 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.788+0000 7ff0da03b640 1 --2- 172.21.15.5:0/469522467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d41082a0 0x7ff0d410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.788+0000 7ff0da03b640 1 --2- 172.21.15.5:0/469522467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d41036e0 0x7ff0d4107b90 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.788+0000 7ff0da03b640 1 --2- 172.21.15.5:0/469522467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d4102d30 0x7ff0d4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.788+0000 7ff0da03b640 1 -- 172.21.15.5:0/469522467 >> 172.21.15.5:0/469522467 conn(0x7ff0d40fc7d0 msgr2=0x7ff0d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.788+0000 7ff0da03b640 1 -- 172.21.15.5:0/469522467 shutdown_connections 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.788+0000 7ff0da03b640 1 -- 172.21.15.5:0/469522467 wait complete. 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.788+0000 7ff0da03b640 1 Processor -- start 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0da03b640 1 -- start start 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0da03b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d4102d30 0x7ff0d41a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0da03b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d41036e0 0x7ff0d41a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0da03b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d41082a0 0x7ff0d419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0da03b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41120a0 con 0x7ff0d4102d30 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0da03b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d4111f20 con 0x7ff0d41082a0 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0da03b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d4112220 con 0x7ff0d41036e0 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d4102d30 0x7ff0d41a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d4102d30 0x7ff0d41a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43162/0 (socket says 172.21.15.5:43162) 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0d37fe640 1 -- 172.21.15.5:0/2956800110 learned_addr learned my addr 172.21.15.5:0/2956800110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0d3fff640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d41082a0 0x7ff0d419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d41036e0 0x7ff0d41a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0d37fe640 1 -- 172.21.15.5:0/2956800110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d41036e0 msgr2=0x7ff0d41a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d41036e0 0x7ff0d41a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0d37fe640 1 -- 172.21.15.5:0/2956800110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d41082a0 msgr2=0x7ff0d419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d41082a0 0x7ff0d419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.789+0000 7ff0d37fe640 1 -- 172.21.15.5:0/2956800110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d419acd0 con 0x7ff0d4102d30 2024-09-11T22:30:24.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.790+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d4102d30 0x7ff0d41a03e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400c890 tx=0x7ff0c400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.790+0000 7ff0d0ff9640 1 -- 172.21.15.5:0/2956800110 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4004220 con 0x7ff0d4102d30 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.790+0000 7ff0d0ff9640 1 -- 172.21.15.5:0/2956800110 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c40043c0 con 0x7ff0d4102d30 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.790+0000 7ff0d0ff9640 1 -- 172.21.15.5:0/2956800110 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40049e0 con 0x7ff0d4102d30 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.790+0000 7ff0da03b640 1 -- 172.21.15.5:0/2956800110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d419afc0 con 0x7ff0d4102d30 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.790+0000 7ff0da03b640 1 -- 172.21.15.5:0/2956800110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d41a72b0 con 0x7ff0d4102d30 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.792+0000 7ff0d0ff9640 1 -- 172.21.15.5:0/2956800110 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff0c40079b0 con 0x7ff0d4102d30 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.792+0000 7ff0dc414740 1 Processor -- start 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.792+0000 7ff0dc414740 1 -- start start 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2bb40 0x557e27d2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 0x557e27d2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0dc414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557e27c902f0 con 0x557e27d35370 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0dc414740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557e27c37cf0 con 0x557e27d2a5d0 2024-09-11T22:30:24.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0dc414740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557e27c8c2b0 con 0x557e27d2bb40 2024-09-11T22:30:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d0ff9640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff0bc0a2cf0 0x7ff0bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d37760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43172/0 (socket says 172.21.15.5:43172) 2024-09-11T22:30:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d2ffd640 1 -- 172.21.15.5:0/356012989 learned_addr learned my addr 172.21.15.5:0/356012989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d3fff640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff0bc0a2cf0 0x7ff0bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d3fff640 1 --2- 172.21.15.5:0/356012989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2bb40 0x557e27d2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d0ff9640 1 -- 172.21.15.5:0/2956800110 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7ff0c40cf580 con 0x7ff0d4102d30 2024-09-11T22:30:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/356012989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 0x557e27d2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d2ffd640 1 -- 172.21.15.5:0/356012989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2bb40 msgr2=0x557e27d2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/356012989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2bb40 0x557e27d2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d2ffd640 1 -- 172.21.15.5:0/356012989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 msgr2=0x557e27d2aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/356012989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 0x557e27d2aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d2ffd640 1 -- 172.21.15.5:0/356012989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e27c1c3a0 con 0x557e27d35370 2024-09-11T22:30:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/356012989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d37760 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8006010 tx=0x7ff0c8032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8486a243369107 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.793+0000 7ff0d3fff640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff0bc0a2cf0 0x7ff0bc0a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0077f0 tx=0x7ff0cc0072a0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:24.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0b9ffb640 1 -- 172.21.15.5:0/356012989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c803f070 con 0x557e27d35370 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0b9ffb640 1 -- 172.21.15.5:0/356012989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c80317b0 con 0x557e27d35370 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0b9ffb640 1 -- 172.21.15.5:0/356012989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8035070 con 0x557e27d35370 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0dc414740 1 -- 172.21.15.5:0/356012989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 msgr2=0x557e27d37760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0dc414740 1 --2- 172.21.15.5:0/356012989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d37760 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8006010 tx=0x7ff0c8032cb0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0dc414740 1 -- 172.21.15.5:0/356012989 shutdown_connections 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0dc414740 1 --2- 172.21.15.5:0/356012989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d37760 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0dc414740 1 --2- 172.21.15.5:0/356012989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 0x557e27d2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0dc414740 1 --2- 172.21.15.5:0/356012989 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2bb40 0x557e27d2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0dc414740 1 -- 172.21.15.5:0/356012989 >> 172.21.15.5:0/356012989 conn(0x557e27cf3310 msgr2=0x557e27d26c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0dc414740 1 -- 172.21.15.5:0/356012989 shutdown_connections 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.794+0000 7ff0dc414740 1 -- 172.21.15.5:0/356012989 wait complete. 2024-09-11T22:30:24.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0dc414740 1 -- start start 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0dc414740 1 Processor -- start 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2a5d0 0x557e27d34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d2bb40 0x557e27d31fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d35370 0x557e27d32570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0dc414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557e27d38010 con 0x557e27d2bb40 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0dc414740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557e27c4b060 con 0x557e27d35370 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0dc414740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557e27d38190 con 0x557e27d2a5d0 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0dc414740 10 client.0 _ll_register_callbacks cb 0x557e2799e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d2bb40 0x557e27d31fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d2bb40 0x557e27d31fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43176/0 (socket says 172.21.15.5:43176) 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.795+0000 7ff0d2ffd640 1 -- 172.21.15.5:0/3364772975 learned_addr learned my addr 172.21.15.5:0/3364772975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:24.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0d3fff640 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d35370 0x557e27d32570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2a5d0 0x557e27d34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0d2ffd640 1 -- 172.21.15.5:0/3364772975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2a5d0 msgr2=0x557e27d34d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2a5d0 0x557e27d34d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0d2ffd640 1 -- 172.21.15.5:0/3364772975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d35370 msgr2=0x557e27d32570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d35370 0x557e27d32570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0d2ffd640 1 -- 172.21.15.5:0/3364772975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e27a013c0 con 0x557e27d2bb40 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d2bb40 0x557e27d31fd0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8031550 tx=0x7ff0c8005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684ce3a1feb7660f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/3364772975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c803f050 con 0x557e27d2bb40 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/3364772975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7ff0c803b870 con 0x557e27d2bb40 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0dc414740 1 -- 172.21.15.5:0/3364772975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c80317b0 con 0x557e27d2bb40 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/3364772975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff0c8035040 con 0x557e27d2bb40 2024-09-11T22:30:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0dc414740 10 client.17940 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.796+0000 7ff0dc414740 1 -- 172.21.15.5:0/3364772975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557e27d43a60 con 0x557e27d2bb40 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.797+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/3364772975 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c80317b0 con 0x557e27d2bb40 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.797+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/3364772975 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff0c804cb80 con 0x557e27d2bb40 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.797+0000 7ff0dc414740 10 client.17940 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.797+0000 7ff0dc414740 10 client.17940 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.797+0000 7ff0dc414740 20 client.17940 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.797+0000 7ff0dc414740 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557e27d44db0 0x557e27d471a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.797+0000 7ff0dc414740 4 client.17940 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.797+0000 7ff0dc414740 1 -- 172.21.15.5:0/3364772975 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e27c9d860 con 0x557e27d44db0 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.797+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557e27d44db0 0x557e27d471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557e27d44db0 0x557e27d471a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d537713980fa9251 server_cookie=76ab412befdc1cb0 in_seq=0 out_seq=0 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0b2ffd640 10 client.17940 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/3364772975 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557e27c9d860 con 0x557e27d44db0 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0b2ffd640 10 client.17940 handle_command_reply: tid=0 2024-09-11T22:30:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0dc414740 1 client.17940 shutdown 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0dc414740 1 -- 172.21.15.5:0/3364772975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d2bb40 msgr2=0x557e27d31fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0dc414740 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d2bb40 0x557e27d31fd0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8031550 tx=0x7ff0c8005470 comp rx=0 tx=0).stop 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0dc414740 1 -- 172.21.15.5:0/3364772975 shutdown_connections 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0dc414740 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557e27d44db0 0x557e27d471a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0dc414740 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d35370 0x557e27d32570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0dc414740 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d2bb40 0x557e27d31fd0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0dc414740 1 --2- 172.21.15.5:0/3364772975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2a5d0 0x557e27d34d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.798+0000 7ff0b2ffd640 0 client.17940 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.799+0000 7ff0dc414740 1 -- 172.21.15.5:0/3364772975 >> 172.21.15.5:0/3364772975 conn(0x557e27cf3310 msgr2=0x557e27d2c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.799+0000 7ff0dc414740 1 -- 172.21.15.5:0/3364772975 shutdown_connections 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.799+0000 7ff0dc414740 1 -- 172.21.15.5:0/3364772975 wait complete. 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.799+0000 7ff0dc414740 20 client.17940 trim_cache size 0 max 16384 2024-09-11T22:30:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.811+0000 7ff0dc414740 1 Processor -- start 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0dc414740 1 -- start start 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2a5d0 0x557e27d412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2bb40 0x557e27d31890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d36760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0dc414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41afcc0 con 0x557e27d35370 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0dc414740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41b1800 con 0x557e27d2bb40 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0dc414740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41b0390 con 0x557e27d2a5d0 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d36760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d36760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43188/0 (socket says 172.21.15.5:43188) 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d3fff640 1 -- 172.21.15.5:0/3890326010 learned_addr learned my addr 172.21.15.5:0/3890326010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/3890326010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2a5d0 0x557e27d412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/3890326010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2bb40 0x557e27d31890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d3fff640 1 -- 172.21.15.5:0/3890326010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2a5d0 msgr2=0x557e27d412b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d3fff640 1 --2- 172.21.15.5:0/3890326010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2a5d0 0x557e27d412b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d3fff640 1 -- 172.21.15.5:0/3890326010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2bb40 msgr2=0x557e27d31890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d3fff640 1 --2- 172.21.15.5:0/3890326010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2bb40 0x557e27d31890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d3fff640 1 -- 172.21.15.5:0/3890326010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e27d2b760 con 0x557e27d35370 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.812+0000 7ff0d3fff640 1 --2- 172.21.15.5:0/3890326010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d36760 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc003c30 tx=0x7ff0cc007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8219f914e631328 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0b97fa640 1 -- 172.21.15.5:0/3890326010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc003e60 con 0x557e27d35370 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0b97fa640 1 -- 172.21.15.5:0/3890326010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0cc00ce10 con 0x557e27d35370 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0b97fa640 1 -- 172.21.15.5:0/3890326010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc005610 con 0x557e27d35370 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0dc414740 1 -- 172.21.15.5:0/3890326010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 msgr2=0x557e27d36760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0dc414740 1 --2- 172.21.15.5:0/3890326010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d36760 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc003c30 tx=0x7ff0cc007520 comp rx=0 tx=0).stop 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0dc414740 1 -- 172.21.15.5:0/3890326010 shutdown_connections 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0dc414740 1 --2- 172.21.15.5:0/3890326010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d36760 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0dc414740 1 --2- 172.21.15.5:0/3890326010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2bb40 0x557e27d31890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0dc414740 1 --2- 172.21.15.5:0/3890326010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2a5d0 0x557e27d412b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0dc414740 1 -- 172.21.15.5:0/3890326010 >> 172.21.15.5:0/3890326010 conn(0x557e27cf3310 msgr2=0x557e27d34050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0dc414740 1 -- 172.21.15.5:0/3890326010 shutdown_connections 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.813+0000 7ff0dc414740 1 -- 172.21.15.5:0/3890326010 wait complete. 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0dc414740 1 -- start start 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0dc414740 1 Processor -- start 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 0x557e27d4c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2bb40 0x557e27d4c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0dc414740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d4ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0dc414740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41b1130 con 0x557e27d35370 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0dc414740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41b0a60 con 0x557e27d2a5d0 2024-09-11T22:30:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0dc414740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41adcc0 con 0x557e27d2bb40 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0dc414740 10 client.0 _ll_register_callbacks cb 0x7ff0cc007e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d4ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.814+0000 7ff0d37fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 0x557e27d4c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d4ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43198/0 (socket says 172.21.15.5:43198) 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0d3fff640 1 -- 172.21.15.5:0/820564879 learned_addr learned my addr 172.21.15.5:0/820564879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2bb40 0x557e27d4c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0d37fe640 1 -- 172.21.15.5:0/820564879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2bb40 msgr2=0x557e27d4c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2bb40 0x557e27d4c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0d37fe640 1 -- 172.21.15.5:0/820564879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 msgr2=0x557e27d4ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d4ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0d37fe640 1 -- 172.21.15.5:0/820564879 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e27d3bb00 con 0x557e27d2a5d0 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0d3fff640 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d4ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0d37fe640 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 0x557e27d4c3c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40ccbd0 tx=0x7ff0c4005300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87cc3628c0c0b80b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/820564879 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c401d040 con 0x557e27d2a5d0 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0dc414740 1 -- 172.21.15.5:0/820564879 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0cc00ce10 con 0x557e27d2a5d0 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/820564879 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7ff0c40ba380 con 0x557e27d2a5d0 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/820564879 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff0c409fd80 con 0x557e27d2a5d0 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.815+0000 7ff0dc414740 10 client.27269 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.816+0000 7ff0dc414740 1 -- 172.21.15.5:0/820564879 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557e27dafb00 con 0x557e27d2a5d0 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.816+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/820564879 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0cc00ce10 con 0x557e27d2a5d0 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.816+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/820564879 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff0c40a41b0 con 0x557e27d2a5d0 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.816+0000 7ff0dc414740 10 client.27269 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.816+0000 7ff0dc414740 10 client.27269 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:30:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.816+0000 7ff0dc414740 20 client.27269 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.816+0000 7ff0dc414740 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557e27db01d0 0x557e27db25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.816+0000 7ff0dc414740 4 client.27269 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.816+0000 7ff0dc414740 1 -- 172.21.15.5:0/820564879 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e27c97d70 con 0x557e27db01d0 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.816+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557e27db01d0 0x557e27db25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0d2ffd640 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557e27db01d0 0x557e27db25c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10e2c228eaeac35 server_cookie=41aee96bfe030731 in_seq=0 out_seq=0 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0b2ffd640 10 client.27269 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0b2ffd640 1 -- 172.21.15.5:0/820564879 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e27c97d70 con 0x557e27db01d0 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0b2ffd640 10 client.27269 handle_command_reply: tid=0 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 client.27269 shutdown 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 -- 172.21.15.5:0/820564879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 msgr2=0x557e27d4c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 0x557e27d4c3c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40ccbd0 tx=0x7ff0c4005300 comp rx=0 tx=0).stop 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 -- 172.21.15.5:0/820564879 shutdown_connections 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557e27d35370 0x557e27d4ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557e27d2bb40 0x557e27d4c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557e27db01d0 0x557e27db25c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 --2- 172.21.15.5:0/820564879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557e27d2a5d0 0x557e27d4c3c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0b2ffd640 0 client.27269 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:30:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 -- 172.21.15.5:0/820564879 >> 172.21.15.5:0/820564879 conn(0x557e27cf3310 msgr2=0x557e27d254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 -- 172.21.15.5:0/820564879 shutdown_connections 2024-09-11T22:30:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 1 -- 172.21.15.5:0/820564879 wait complete. 2024-09-11T22:30:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.817+0000 7ff0dc414740 20 client.27269 trim_cache size 0 max 16384 2024-09-11T22:30:24.835 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:30:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 -- 172.21.15.5:0/2956800110 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff0bc0a2cf0 msgr2=0x7ff0bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff0bc0a2cf0 0x7ff0bc0a51b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0077f0 tx=0x7ff0cc0072a0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 -- 172.21.15.5:0/2956800110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d4102d30 msgr2=0x7ff0d41a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d4102d30 0x7ff0d41a03e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400c890 tx=0x7ff0c400cd50 comp rx=0 tx=0).stop 2024-09-11T22:30:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 -- 172.21.15.5:0/2956800110 shutdown_connections 2024-09-11T22:30:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff0bc0a2cf0 0x7ff0bc0a51b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d41082a0 0x7ff0d419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0d41036e0 0x7ff0d41a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 --2- 172.21.15.5:0/2956800110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d4102d30 0x7ff0d41a03e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 -- 172.21.15.5:0/2956800110 >> 172.21.15.5:0/2956800110 conn(0x7ff0d40fc7d0 msgr2=0x7ff0d4100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 -- 172.21.15.5:0/2956800110 shutdown_connections 2024-09-11T22:30:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:24.818+0000 7ff0da03b640 1 -- 172.21.15.5:0/2956800110 wait complete. 2024-09-11T22:30:25.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 8.09 MB/sec execute 283 sec latency 4972.130 ms 2024-09-11T22:30:25.708 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:25 smithi005 ceph-mon[30943]: pgmap v257: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 270 op/s 2024-09-11T22:30:25.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:30:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:25 smithi159 ceph-mon[35534]: pgmap v257: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 270 op/s 2024-09-11T22:30:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:25 smithi172 ceph-mon[35461]: pgmap v257: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.8 MiB/s wr, 270 op/s 2024-09-11T22:30:25.969 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.053+0000 7f9da155b640 1 -- 172.21.15.5:0/2182618005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d9c1009d0 msgr2=0x7f9d9c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.053+0000 7f9da155b640 1 --2- 172.21.15.5:0/2182618005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d9c1009d0 0x7f9d9c100e30 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009930 tx=0x7f9d8402a420 comp rx=0 tx=0).stop 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.053+0000 7f9da155b640 1 -- 172.21.15.5:0/2182618005 shutdown_connections 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.053+0000 7f9da155b640 1 --2- 172.21.15.5:0/2182618005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d9c101370 0x7f9d9c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.053+0000 7f9da155b640 1 --2- 172.21.15.5:0/2182618005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d9c1009d0 0x7f9d9c100e30 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.053+0000 7f9da155b640 1 --2- 172.21.15.5:0/2182618005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d9c1069e0 0x7f9d9c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.053+0000 7f9da155b640 1 -- 172.21.15.5:0/2182618005 >> 172.21.15.5:0/2182618005 conn(0x7f9d9c0fc6f0 msgr2=0x7f9d9c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.053+0000 7f9da155b640 1 -- 172.21.15.5:0/2182618005 shutdown_connections 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9da155b640 1 -- 172.21.15.5:0/2182618005 wait complete. 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9da155b640 1 Processor -- start 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9da155b640 1 -- start start 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9da155b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d9c1009d0 0x7f9d9c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9da155b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d9c101370 0x7f9d9c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9da155b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d9c1069e0 0x7f9d9c1a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9da155b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c110e70 con 0x7f9d9c1009d0 2024-09-11T22:30:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9da155b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c110cf0 con 0x7f9d9c101370 2024-09-11T22:30:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9da155b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c110ff0 con 0x7f9d9c1069e0 2024-09-11T22:30:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9d9affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d9c1009d0 0x7f9d9c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9d9affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d9c1009d0 0x7f9d9c1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43220/0 (socket says 172.21.15.5:43220) 2024-09-11T22:30:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.054+0000 7f9d9affd640 1 -- 172.21.15.5:0/683396163 learned_addr learned my addr 172.21.15.5:0/683396163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d9c1069e0 0x7f9d9c1a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d93fff640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d9c101370 0x7f9d9c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d9affd640 1 -- 172.21.15.5:0/683396163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d9c1069e0 msgr2=0x7f9d9c1a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d9affd640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d9c1069e0 0x7f9d9c1a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d9affd640 1 -- 172.21.15.5:0/683396163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d9c101370 msgr2=0x7f9d9c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d9affd640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d9c101370 0x7f9d9c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d9affd640 1 -- 172.21.15.5:0/683396163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c1a47b0 con 0x7f9d9c1009d0 2024-09-11T22:30:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d9affd640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d9c1009d0 0x7f9d9c1991f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e9f0 tx=0x7f9d8c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d98ff9640 1 -- 172.21.15.5:0/683396163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00cd80 con 0x7f9d9c1009d0 2024-09-11T22:30:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9da155b640 1 -- 172.21.15.5:0/683396163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c1a4980 con 0x7f9d9c1009d0 2024-09-11T22:30:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d98ff9640 1 -- 172.21.15.5:0/683396163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c010910 con 0x7f9d9c1009d0 2024-09-11T22:30:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9d98ff9640 1 -- 172.21.15.5:0/683396163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c018990 con 0x7f9d9c1009d0 2024-09-11T22:30:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.055+0000 7f9da155b640 1 -- 172.21.15.5:0/683396163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c1a4ea0 con 0x7f9d9c1009d0 2024-09-11T22:30:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.057+0000 7f9d98ff9640 1 -- 172.21.15.5:0/683396163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9d8c010ac0 con 0x7f9d9c1009d0 2024-09-11T22:30:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9da43ce740 1 Processor -- start 2024-09-11T22:30:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9da43ce740 1 -- start start 2024-09-11T22:30:26.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b8af0 0x5642501b8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 0x5642501c3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9da43ce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56425011d2f0 con 0x5642501b8af0 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9da43ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5642500c5270 con 0x5642501b9410 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9da43ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5642501192b0 con 0x5642501c3cb0 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9d98ff9640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d880a2da0 0x7f9d880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9d9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b8af0 0x5642501b8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9d9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b8af0 0x5642501b8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43222/0 (socket says 172.21.15.5:43222) 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9d9b7fe640 1 -- 172.21.15.5:0/149362629 learned_addr learned my addr 172.21.15.5:0/149362629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d880a2da0 0x7f9d880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.058+0000 7f9d93fff640 1 --2- 172.21.15.5:0/149362629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d98ff9640 1 -- 172.21.15.5:0/683396163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f9d8c0cf910 con 0x7f9d9c1009d0 2024-09-11T22:30:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d9affd640 1 --2- 172.21.15.5:0/149362629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 0x5642501c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d9b7fe640 1 -- 172.21.15.5:0/149362629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 msgr2=0x5642501c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/149362629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d9b7fe640 1 -- 172.21.15.5:0/149362629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 msgr2=0x5642501c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/149362629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 0x5642501c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d9b7fe640 1 -- 172.21.15.5:0/149362629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642500c4340 con 0x5642501b8af0 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/149362629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b8af0 0x5642501b8ed0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9d94014f40 tx=0x7f9d94011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a93880c4d442670 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d880a2da0 0x7f9d880a5260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009370 tx=0x7f9d94016000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d917fa640 1 -- 172.21.15.5:0/149362629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400fd40 con 0x5642501b8af0 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d917fa640 1 -- 172.21.15.5:0/149362629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d94003e50 con 0x5642501b8af0 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.059+0000 7f9d917fa640 1 -- 172.21.15.5:0/149362629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94004710 con 0x5642501b8af0 2024-09-11T22:30:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 -- 172.21.15.5:0/149362629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b8af0 msgr2=0x5642501b8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 --2- 172.21.15.5:0/149362629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b8af0 0x5642501b8ed0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9d94014f40 tx=0x7f9d94011870 comp rx=0 tx=0).stop 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 -- 172.21.15.5:0/149362629 shutdown_connections 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 --2- 172.21.15.5:0/149362629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 --2- 172.21.15.5:0/149362629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 0x5642501c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 --2- 172.21.15.5:0/149362629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b8af0 0x5642501b8ed0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 -- 172.21.15.5:0/149362629 >> 172.21.15.5:0/149362629 conn(0x5642501b5060 msgr2=0x5642501b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 -- 172.21.15.5:0/149362629 shutdown_connections 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 -- 172.21.15.5:0/149362629 wait complete. 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 -- start start 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.060+0000 7f9da43ce740 1 Processor -- start 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 0x5642501c0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9da43ce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5642501c6880 con 0x5642501b9410 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9da43ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5642500a6290 con 0x5642501b8af0 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9da43ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5642501c6a00 con 0x5642501c3cb0 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9da43ce740 10 client.0 _ll_register_callbacks cb 0x5642500d7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43224/0 (socket says 172.21.15.5:43224) 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d93fff640 1 -- 172.21.15.5:0/1404104042 learned_addr learned my addr 172.21.15.5:0/1404104042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d9affd640 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 0x5642501c0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d93fff640 1 -- 172.21.15.5:0/1404104042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 msgr2=0x5642501c1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d93fff640 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d93fff640 1 -- 172.21.15.5:0/1404104042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 msgr2=0x5642501c0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d93fff640 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 0x5642501c0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d93fff640 1 -- 172.21.15.5:0/1404104042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424fe30210 con 0x5642501b9410 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d93fff640 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c10e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9d8402aa10 tx=0x7f9d84005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142565bef5e07c87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.061+0000 7f9d827fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9d827fc640 1 -- 172.21.15.5:0/1404104042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84035040 con 0x5642501b9410 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9da43ce740 1 -- 172.21.15.5:0/1404104042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d94003e50 con 0x5642501b9410 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9d827fc640 1 -- 172.21.15.5:0/1404104042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f9d840048c0 con 0x5642501b9410 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9d827fc640 1 -- 172.21.15.5:0/1404104042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9d84005d50 con 0x5642501b9410 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9da43ce740 10 client.17958 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9da43ce740 1 -- 172.21.15.5:0/1404104042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56424fefc620 con 0x5642501b9410 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9d827fc640 1 -- 172.21.15.5:0/1404104042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d94003e50 con 0x5642501b9410 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9d827fc640 1 -- 172.21.15.5:0/1404104042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9d84003ca0 con 0x5642501b9410 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9da43ce740 10 client.17958 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9da43ce740 10 client.17958 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9da43ce740 20 client.17958 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9da43ce740 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5642501d3c00 0x5642501d6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9da43ce740 4 client.17958 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9da43ce740 1 -- 172.21.15.5:0/1404104042 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425012aa20 con 0x5642501d3c00 2024-09-11T22:30:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.062+0000 7f9d9affd640 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5642501d3c00 0x5642501d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.063+0000 7f9d827fc640 10 client.17958 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.063+0000 7f9d9affd640 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5642501d3c00 0x5642501d6090 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5181cbfefae9ceea server_cookie=a9fe80515f3d879f in_seq=0 out_seq=0 2024-09-11T22:30:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9d827fc640 1 -- 172.21.15.5:0/1404104042 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56425012aa20 con 0x5642501d3c00 2024-09-11T22:30:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9d827fc640 10 client.17958 handle_command_reply: tid=0 2024-09-11T22:30:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 client.17958 shutdown 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 -- 172.21.15.5:0/1404104042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 msgr2=0x5642501c10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c10e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9d8402aa10 tx=0x7f9d84005270 comp rx=0 tx=0).stop 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 -- 172.21.15.5:0/1404104042 shutdown_connections 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5642501d3c00 0x5642501d6090 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c10e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9d827fc640 0 client.17958 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 --2- 172.21.15.5:0/1404104042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 0x5642501c0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 -- 172.21.15.5:0/1404104042 >> 172.21.15.5:0/1404104042 conn(0x5642501b5060 msgr2=0x5642501badc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 -- 172.21.15.5:0/1404104042 shutdown_connections 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 1 -- 172.21.15.5:0/1404104042 wait complete. 2024-09-11T22:30:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.064+0000 7f9da43ce740 20 client.17958 trim_cache size 0 max 16384 2024-09-11T22:30:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9da43ce740 1 Processor -- start 2024-09-11T22:30:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9da43ce740 1 -- start start 2024-09-11T22:30:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501b8af0 0x5642501c1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 0x5642501c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501c3cb0 0x5642501db9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9da43ce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1ad9c0 con 0x5642501c3cb0 2024-09-11T22:30:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9da43ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1af500 con 0x5642501b9410 2024-09-11T22:30:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9da43ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1ae090 con 0x5642501b8af0 2024-09-11T22:30:26.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501c3cb0 0x5642501db9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501c3cb0 0x5642501db9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43236/0 (socket says 172.21.15.5:43236) 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d9b7fe640 1 -- 172.21.15.5:0/3209855081 learned_addr learned my addr 172.21.15.5:0/3209855081 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d9affd640 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501b8af0 0x5642501c1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d93fff640 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 0x5642501c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d9b7fe640 1 -- 172.21.15.5:0/3209855081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501b8af0 msgr2=0x5642501c1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501b8af0 0x5642501c1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d9b7fe640 1 -- 172.21.15.5:0/3209855081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 msgr2=0x5642501c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 0x5642501c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d9b7fe640 1 -- 172.21.15.5:0/3209855081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642501b5cf0 con 0x5642501c3cb0 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d93fff640 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 0x5642501c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.079+0000 7f9d9affd640 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501b8af0 0x5642501c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:26.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.080+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501c3cb0 0x5642501db9a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009b90 tx=0x7f9d94009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabde994c96fe2e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.080+0000 7f9d90ff9640 1 -- 172.21.15.5:0/3209855081 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94035070 con 0x5642501c3cb0 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.080+0000 7f9d90ff9640 1 -- 172.21.15.5:0/3209855081 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d94014890 con 0x5642501c3cb0 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.080+0000 7f9d90ff9640 1 -- 172.21.15.5:0/3209855081 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94005630 con 0x5642501c3cb0 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 -- 172.21.15.5:0/3209855081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501c3cb0 msgr2=0x5642501db9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501c3cb0 0x5642501db9a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009b90 tx=0x7f9d94009520 comp rx=0 tx=0).stop 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 -- 172.21.15.5:0/3209855081 shutdown_connections 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501c3cb0 0x5642501db9a0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b9410 0x5642501c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 --2- 172.21.15.5:0/3209855081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501b8af0 0x5642501c1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 -- 172.21.15.5:0/3209855081 >> 172.21.15.5:0/3209855081 conn(0x5642501b5060 msgr2=0x5642501c2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 -- 172.21.15.5:0/3209855081 shutdown_connections 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 -- 172.21.15.5:0/3209855081 wait complete. 2024-09-11T22:30:26.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 -- start start 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 Processor -- start 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 0x5642501cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1aee30 con 0x5642501b9410 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1ae760 con 0x5642501b8af0 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1ab9c0 con 0x5642501c3cb0 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9da43ce740 10 client.0 _ll_register_callbacks cb 0x7f9d84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.081+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43240/0 (socket says 172.21.15.5:43240) 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d93fff640 1 -- 172.21.15.5:0/787505734 learned_addr learned my addr 172.21.15.5:0/787505734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d9affd640 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 0x5642501cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d93fff640 1 -- 172.21.15.5:0/787505734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 msgr2=0x5642501c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d93fff640 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d93fff640 1 -- 172.21.15.5:0/787505734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 msgr2=0x5642501cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d93fff640 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 0x5642501cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d93fff640 1 -- 172.21.15.5:0/787505734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642501c7eb0 con 0x5642501b9410 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d9b7fe640 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d9affd640 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 0x5642501cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d93fff640 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c7430 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f9d840317f0 tx=0x7f9d84031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad92dc405354318 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d827fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:26.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d827fc640 1 -- 172.21.15.5:0/787505734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84035070 con 0x5642501b9410 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d827fc640 1 -- 172.21.15.5:0/787505734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f9d84004ad0 con 0x5642501b9410 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9da43ce740 1 -- 172.21.15.5:0/787505734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d94014890 con 0x5642501b9410 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9d827fc640 1 -- 172.21.15.5:0/787505734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9d8405b050 con 0x5642501b9410 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9da43ce740 10 client.17964 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.082+0000 7f9da43ce740 1 -- 172.21.15.5:0/787505734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56425023e650 con 0x5642501b9410 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9d827fc640 1 -- 172.21.15.5:0/787505734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d94014890 con 0x5642501b9410 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9d827fc640 1 -- 172.21.15.5:0/787505734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9d84003d10 con 0x5642501b9410 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9da43ce740 10 client.17964 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9da43ce740 10 client.17964 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9da43ce740 20 client.17964 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9da43ce740 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56425023edd0 0x5642502411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9da43ce740 4 client.17964 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:30:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9da43ce740 1 -- 172.21.15.5:0/787505734 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564250124d70 con 0x56425023edd0 2024-09-11T22:30:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9d9affd640 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56425023edd0 0x5642502411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9d9affd640 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56425023edd0 0x5642502411f0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e63e77aa8559dda server_cookie=e4858a1d644a7fec in_seq=0 out_seq=0 2024-09-11T22:30:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.083+0000 7f9d827fc640 10 client.17964 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.178+0000 7f9d827fc640 1 -- 172.21.15.5:0/787505734 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564250124d70 con 0x56425023edd0 2024-09-11T22:30:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.178+0000 7f9d827fc640 10 client.17964 handle_command_reply: tid=0 2024-09-11T22:30:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.178+0000 7f9da43ce740 1 client.17964 shutdown 2024-09-11T22:30:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 1 -- 172.21.15.5:0/787505734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 msgr2=0x5642501c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c7430 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f9d840317f0 tx=0x7f9d84031b20 comp rx=0 tx=0).stop 2024-09-11T22:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 1 -- 172.21.15.5:0/787505734 shutdown_connections 2024-09-11T22:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56425023edd0 0x5642502411f0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5642501c3cb0 0x5642501c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9d827fc640 0 client.17964 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642501b9410 0x5642501c7430 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 1 --2- 172.21.15.5:0/787505734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642501b8af0 0x5642501cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 1 -- 172.21.15.5:0/787505734 >> 172.21.15.5:0/787505734 conn(0x5642501b5060 msgr2=0x5642501b41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 1 -- 172.21.15.5:0/787505734 shutdown_connections 2024-09-11T22:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 1 -- 172.21.15.5:0/787505734 wait complete. 2024-09-11T22:30:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.179+0000 7f9da43ce740 20 client.17964 trim_cache size 0 max 16384 2024-09-11T22:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:30:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.180+0000 7f9da155b640 1 -- 172.21.15.5:0/683396163 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d880a2da0 msgr2=0x7f9d880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.180+0000 7f9da155b640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d880a2da0 0x7f9d880a5260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009370 tx=0x7f9d94016000 comp rx=0 tx=0).stop 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.180+0000 7f9da155b640 1 -- 172.21.15.5:0/683396163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d9c1009d0 msgr2=0x7f9d9c1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.180+0000 7f9da155b640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d9c1009d0 0x7f9d9c1991f0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00e9f0 tx=0x7f9d8c00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.180+0000 7f9da155b640 1 -- 172.21.15.5:0/683396163 shutdown_connections 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.180+0000 7f9da155b640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d9c1069e0 0x7f9d9c1a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.180+0000 7f9da155b640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d880a2da0 0x7f9d880a5260 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.180+0000 7f9da155b640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d9c101370 0x7f9d9c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.180+0000 7f9da155b640 1 --2- 172.21.15.5:0/683396163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d9c1009d0 0x7f9d9c1991f0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.180+0000 7f9da155b640 1 -- 172.21.15.5:0/683396163 >> 172.21.15.5:0/683396163 conn(0x7f9d9c0fc6f0 msgr2=0x7f9d9c0fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.181+0000 7f9da155b640 1 -- 172.21.15.5:0/683396163 shutdown_connections 2024-09-11T22:30:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:26.181+0000 7f9da155b640 1 -- 172.21.15.5:0/683396163 wait complete. 2024-09-11T22:30:26.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 8.06 MB/sec execute 284 sec latency 5972.222 ms 2024-09-11T22:30:26.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.459+0000 7f0043ec1640 1 -- 172.21.15.5:0/1675476619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f003c103740 msgr2=0x7f003c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.459+0000 7f0043ec1640 1 --2- 172.21.15.5:0/1675476619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f003c103740 0x7f003c107b80 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f002c009930 tx=0x7f002c02a420 comp rx=0 tx=0).stop 2024-09-11T22:30:26.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.459+0000 7f0043ec1640 1 -- 172.21.15.5:0/1675476619 shutdown_connections 2024-09-11T22:30:26.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.459+0000 7f0043ec1640 1 --2- 172.21.15.5:0/1675476619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f003c108290 0x7f003c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.459+0000 7f0043ec1640 1 --2- 172.21.15.5:0/1675476619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f003c103740 0x7f003c107b80 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.459+0000 7f0043ec1640 1 --2- 172.21.15.5:0/1675476619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f003c102d90 0x7f003c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.459+0000 7f0043ec1640 1 -- 172.21.15.5:0/1675476619 >> 172.21.15.5:0/1675476619 conn(0x7f003c0fc7d0 msgr2=0x7f003c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.459+0000 7f0043ec1640 1 -- 172.21.15.5:0/1675476619 shutdown_connections 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.459+0000 7f0043ec1640 1 -- 172.21.15.5:0/1675476619 wait complete. 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.460+0000 7f0043ec1640 1 Processor -- start 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.460+0000 7f0043ec1640 1 -- start start 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.460+0000 7f0043ec1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f003c102d90 0x7f003c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.460+0000 7f0043ec1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f003c103740 0x7f003c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.460+0000 7f0043ec1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f003c108290 0x7f003c19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.460+0000 7f0043ec1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f003c112080 con 0x7f003c103740 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.460+0000 7f0043ec1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f003c111f00 con 0x7f003c102d90 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.460+0000 7f0043ec1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f003c112200 con 0x7f003c108290 2024-09-11T22:30:26.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f004145c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f003c103740 0x7f003c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f004145c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f003c103740 0x7f003c198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43260/0 (socket says 172.21.15.5:43260) 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f004145c640 1 -- 172.21.15.5:0/420176864 learned_addr learned my addr 172.21.15.5:0/420176864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f004245e640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f003c108290 0x7f003c19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f0041c5d640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f003c102d90 0x7f003c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f004145c640 1 -- 172.21.15.5:0/420176864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f003c108290 msgr2=0x7f003c19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f004145c640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f003c108290 0x7f003c19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f004145c640 1 -- 172.21.15.5:0/420176864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f003c102d90 msgr2=0x7f003c198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f004145c640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f003c102d90 0x7f003c198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f004145c640 1 -- 172.21.15.5:0/420176864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003c19d5b0 con 0x7f003c103740 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f004145c640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f003c103740 0x7f003c198b40 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f002c02a940 tx=0x7f002c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f0032ffd640 1 -- 172.21.15.5:0/420176864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c03f070 con 0x7f003c103740 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f0032ffd640 1 -- 172.21.15.5:0/420176864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f002c031c90 con 0x7f003c103740 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f0043ec1640 1 -- 172.21.15.5:0/420176864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003c19d840 con 0x7f003c103740 2024-09-11T22:30:26.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f0032ffd640 1 -- 172.21.15.5:0/420176864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c035030 con 0x7f003c103740 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.461+0000 7f0043ec1640 1 -- 172.21.15.5:0/420176864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f003c1a5180 con 0x7f003c103740 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.463+0000 7f0032ffd640 1 -- 172.21.15.5:0/420176864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f002c042070 con 0x7f003c103740 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f004629a740 1 Processor -- start 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f004629a740 1 -- start start 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f004629a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847fea4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f004629a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847fea45d0 0x55847fea4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f004629a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847feaf370 0x55847feb1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f004629a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55847fe0a2f0 con 0x55847feaf370 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f004629a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55847fdb1cf0 con 0x55847fea5b40 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f004629a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55847fe062b0 con 0x55847fea45d0 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f0032ffd640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00280a2d70 0x7f00280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f004145c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847feaf370 0x55847feb1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.464+0000 7f004245e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847fea4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f004245e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847fea4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35038/0 (socket says 172.21.15.5:35038) 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f004245e640 1 -- 172.21.15.5:0/1288477200 learned_addr learned my addr 172.21.15.5:0/1288477200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f004245e640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00280a2d70 0x7f00280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f0041c5d640 1 --2- 172.21.15.5:0/1288477200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847fea45d0 0x55847fea4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f0032ffd640 1 -- 172.21.15.5:0/420176864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f002c0efb90 con 0x7f003c103740 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f004245e640 1 -- 172.21.15.5:0/1288477200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847fea45d0 msgr2=0x55847fea4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f004245e640 1 --2- 172.21.15.5:0/1288477200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847fea45d0 0x55847fea4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f004245e640 1 -- 172.21.15.5:0/1288477200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847feaf370 msgr2=0x55847feb1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f004245e640 1 --2- 172.21.15.5:0/1288477200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847feaf370 0x55847feb1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f004245e640 1 -- 172.21.15.5:0/1288477200 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847fd963a0 con 0x55847fea5b40 2024-09-11T22:30:26.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f0041c5d640 1 --2- 172.21.15.5:0/1288477200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847fea45d0 0x55847fea4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f004245e640 1 --2- 172.21.15.5:0/1288477200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847fea4090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0034012d00 tx=0x7f0034010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be764e3d98423c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f004245e640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00280a2d70 0x7f00280a5230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0034010c40 tx=0x7f003400c000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f0027fff640 1 -- 172.21.15.5:0/1288477200 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034024070 con 0x55847fea5b40 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f0027fff640 1 -- 172.21.15.5:0/1288477200 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0034012520 con 0x55847fea5b40 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.465+0000 7f0027fff640 1 -- 172.21.15.5:0/1288477200 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034005280 con 0x55847fea5b40 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.466+0000 7f004629a740 1 -- 172.21.15.5:0/1288477200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 msgr2=0x55847fea4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.466+0000 7f004629a740 1 --2- 172.21.15.5:0/1288477200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847fea4090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0034012d00 tx=0x7f0034010740 comp rx=0 tx=0).stop 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.466+0000 7f004629a740 1 -- 172.21.15.5:0/1288477200 shutdown_connections 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.466+0000 7f004629a740 1 --2- 172.21.15.5:0/1288477200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847feaf370 0x55847feb1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.466+0000 7f004629a740 1 --2- 172.21.15.5:0/1288477200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847fea45d0 0x55847fea4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.466+0000 7f004629a740 1 --2- 172.21.15.5:0/1288477200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847fea4090 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.466+0000 7f004629a740 1 -- 172.21.15.5:0/1288477200 >> 172.21.15.5:0/1288477200 conn(0x55847fe6d310 msgr2=0x55847fea0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.466+0000 7f004629a740 1 -- 172.21.15.5:0/1288477200 shutdown_connections 2024-09-11T22:30:26.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.466+0000 7f004629a740 1 -- 172.21.15.5:0/1288477200 wait complete. 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004629a740 1 -- start start 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004629a740 1 Processor -- start 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004629a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea45d0 0x55847feaeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004629a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea5b40 0x55847feac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004629a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847feac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004629a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55847feb2010 con 0x55847fea5b40 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004629a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55847fdc5060 con 0x55847fea45d0 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004629a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55847feb2190 con 0x55847feaf370 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004629a740 10 client.0 _ll_register_callbacks cb 0x55847fb188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004145c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea5b40 0x55847feac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004145c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea5b40 0x55847feac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43264/0 (socket says 172.21.15.5:43264) 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004145c640 1 -- 172.21.15.5:0/2199205960 learned_addr learned my addr 172.21.15.5:0/2199205960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:26.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f0041c5d640 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea45d0 0x55847feaeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.467+0000 7f004245e640 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847feac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f004145c640 1 -- 172.21.15.5:0/2199205960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 msgr2=0x55847feac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f004145c640 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847feac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f004145c640 1 -- 172.21.15.5:0/2199205960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea45d0 msgr2=0x55847feaeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f004145c640 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea45d0 0x55847feaeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f004145c640 1 -- 172.21.15.5:0/2199205960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847fb7b3c0 con 0x55847fea5b40 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f004245e640 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847feac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f004145c640 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea5b40 0x55847feac050 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f002c0bbdf0 tx=0x7f002c08bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7151751d707baf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f00257fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f00257fa640 1 -- 172.21.15.5:0/2199205960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c049050 con 0x55847fea5b40 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f00257fa640 1 -- 172.21.15.5:0/2199205960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f002c0ed4f0 con 0x55847fea5b40 2024-09-11T22:30:26.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f004629a740 1 -- 172.21.15.5:0/2199205960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0034012520 con 0x55847fea5b40 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f00257fa640 1 -- 172.21.15.5:0/2199205960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f002c0c03f0 con 0x55847fea5b40 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f004629a740 10 client.17976 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f004629a740 1 -- 172.21.15.5:0/2199205960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55847febdaf0 con 0x55847fea5b40 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f00257fa640 1 -- 172.21.15.5:0/2199205960 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0034012520 con 0x55847fea5b40 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.468+0000 7f00257fa640 1 -- 172.21.15.5:0/2199205960 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f002c0b8760 con 0x55847fea5b40 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.469+0000 7f004629a740 10 client.17976 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.469+0000 7f004629a740 10 client.17976 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.469+0000 7f004629a740 20 client.17976 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.469+0000 7f004629a740 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55847febefa0 0x55847fec13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.469+0000 7f004629a740 4 client.17976 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.469+0000 7f004629a740 1 -- 172.21.15.5:0/2199205960 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55847fe17860 con 0x55847febefa0 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.469+0000 7f0041c5d640 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55847febefa0 0x55847fec13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.469+0000 7f0041c5d640 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55847febefa0 0x55847fec13b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c252c2a8858bd606 server_cookie=b6532a4ec2b3700c in_seq=0 out_seq=0 2024-09-11T22:30:26.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.469+0000 7f00257fa640 10 client.17976 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f00257fa640 1 -- 172.21.15.5:0/2199205960 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55847fe17860 con 0x55847febefa0 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f00257fa640 10 client.17976 handle_command_reply: tid=0 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 client.17976 shutdown 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 -- 172.21.15.5:0/2199205960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea5b40 msgr2=0x55847feac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea5b40 0x55847feac050 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f002c0bbdf0 tx=0x7f002c08bf50 comp rx=0 tx=0).stop 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 -- 172.21.15.5:0/2199205960 shutdown_connections 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55847febefa0 0x55847fec13b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847feac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea5b40 0x55847feac050 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f00257fa640 0 client.17976 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 --2- 172.21.15.5:0/2199205960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea45d0 0x55847feaeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 -- 172.21.15.5:0/2199205960 >> 172.21.15.5:0/2199205960 conn(0x55847fe6d310 msgr2=0x55847fea6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 -- 172.21.15.5:0/2199205960 shutdown_connections 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.470+0000 7f004629a740 1 -- 172.21.15.5:0/2199205960 wait complete. 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.471+0000 7f004629a740 20 client.17976 trim_cache size 0 max 16384 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.483+0000 7f004629a740 1 Processor -- start 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.483+0000 7f004629a740 1 -- start start 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f004629a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feadfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f004629a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847fec6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f004629a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847fec71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f004629a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f003c1ada50 con 0x55847fea45d0 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f004629a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f003c1af590 con 0x55847fea5b40 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f004629a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f003c1ae120 con 0x55847feaf370 2024-09-11T22:30:26.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f0041c5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feadfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f0041c5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feadfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43270/0 (socket says 172.21.15.5:43270) 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f0041c5d640 1 -- 172.21.15.5:0/1004650222 learned_addr learned my addr 172.21.15.5:0/1004650222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f004145c640 1 --2- 172.21.15.5:0/1004650222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847fec6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f0041c5d640 1 -- 172.21.15.5:0/1004650222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 msgr2=0x55847fec71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f004245e640 1 --2- 172.21.15.5:0/1004650222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847fec71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f0041c5d640 1 --2- 172.21.15.5:0/1004650222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847fec71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f0041c5d640 1 -- 172.21.15.5:0/1004650222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 msgr2=0x55847fec6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f0041c5d640 1 --2- 172.21.15.5:0/1004650222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847fec6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f0041c5d640 1 -- 172.21.15.5:0/1004650222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847fea0ed0 con 0x55847fea45d0 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.484+0000 7f0041c5d640 1 --2- 172.21.15.5:0/1004650222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feadfc0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0038000c00 tx=0x7f003800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0d59240e7b3c9f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f00277fe640 1 -- 172.21.15.5:0/1004650222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038003ea0 con 0x55847fea45d0 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f00277fe640 1 -- 172.21.15.5:0/1004650222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0038011920 con 0x55847fea45d0 2024-09-11T22:30:26.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f00277fe640 1 -- 172.21.15.5:0/1004650222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003801ca80 con 0x55847fea45d0 2024-09-11T22:30:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f004629a740 1 -- 172.21.15.5:0/1004650222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 msgr2=0x55847feadfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f004629a740 1 --2- 172.21.15.5:0/1004650222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feadfc0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f0038000c00 tx=0x7f003800bf90 comp rx=0 tx=0).stop 2024-09-11T22:30:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f004629a740 1 -- 172.21.15.5:0/1004650222 shutdown_connections 2024-09-11T22:30:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f004629a740 1 --2- 172.21.15.5:0/1004650222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847fec71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f004629a740 1 --2- 172.21.15.5:0/1004650222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847fec6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f004629a740 1 --2- 172.21.15.5:0/1004650222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feadfc0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f004629a740 1 -- 172.21.15.5:0/1004650222 >> 172.21.15.5:0/1004650222 conn(0x55847fe6d310 msgr2=0x55847fead8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f004629a740 1 -- 172.21.15.5:0/1004650222 shutdown_connections 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.485+0000 7f004629a740 1 -- 172.21.15.5:0/1004650222 wait complete. 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f004629a740 1 -- start start 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f004629a740 1 Processor -- start 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f004629a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feb34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f004629a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847feb3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f004629a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847feb3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f004629a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f003c1aeec0 con 0x55847fea45d0 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f004629a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f003c1ae7f0 con 0x55847fea5b40 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f004629a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f003c1aba50 con 0x55847feaf370 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f004629a740 10 client.0 _ll_register_callbacks cb 0x7f002c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:26.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f0041c5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feb34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f0041c5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feb34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43286/0 (socket says 172.21.15.5:43286) 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f0041c5d640 1 -- 172.21.15.5:0/282541069 learned_addr learned my addr 172.21.15.5:0/282541069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.486+0000 7f004145c640 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847feb3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f004245e640 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847feb3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f0041c5d640 1 -- 172.21.15.5:0/282541069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 msgr2=0x55847feb3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f0041c5d640 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847feb3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f0041c5d640 1 -- 172.21.15.5:0/282541069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 msgr2=0x55847feb3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f0041c5d640 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847feb3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f0041c5d640 1 -- 172.21.15.5:0/282541069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847ff28f40 con 0x55847fea45d0 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f004145c640 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847feb3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f0041c5d640 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feb34d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f003800b9e0 tx=0x7f00380056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37842363aa4f2327 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f00257fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:26.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f00257fa640 1 -- 172.21.15.5:0/282541069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038011640 con 0x55847fea45d0 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f00257fa640 1 -- 172.21.15.5:0/282541069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f0038011ad0 con 0x55847fea45d0 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f004629a740 1 -- 172.21.15.5:0/282541069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0038011920 con 0x55847fea45d0 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f00257fa640 1 -- 172.21.15.5:0/282541069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0038011cd0 con 0x55847fea45d0 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f004629a740 10 client.17982 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.487+0000 7f004629a740 1 -- 172.21.15.5:0/282541069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55847ff2a240 con 0x55847fea45d0 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.488+0000 7f00257fa640 1 -- 172.21.15.5:0/282541069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0038011920 con 0x55847fea45d0 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.488+0000 7f00257fa640 1 -- 172.21.15.5:0/282541069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f0038038520 con 0x55847fea45d0 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.488+0000 7f004629a740 10 client.17982 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.488+0000 7f004629a740 10 client.17982 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.488+0000 7f004629a740 20 client.17982 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.488+0000 7f004629a740 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55847ff2a850 0x55847ff2cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.488+0000 7f004629a740 4 client.17982 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:30:26.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.488+0000 7f004629a740 1 -- 172.21.15.5:0/282541069 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55847fe11d70 con 0x55847ff2a850 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.488+0000 7f004145c640 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55847ff2a850 0x55847ff2cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004145c640 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55847ff2a850 0x55847ff2cc40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed72c8c15fecd222 server_cookie=9807a530dbc03aa7 in_seq=0 out_seq=0 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f00257fa640 10 client.17982 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f00257fa640 1 -- 172.21.15.5:0/282541069 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55847fe11d70 con 0x55847ff2a850 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f00257fa640 10 client.17982 handle_command_reply: tid=0 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 client.17982 shutdown 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 -- 172.21.15.5:0/282541069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 msgr2=0x55847feb34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feb34d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f003800b9e0 tx=0x7f00380056d0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 -- 172.21.15.5:0/282541069 shutdown_connections 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55847ff2a850 0x55847ff2cc40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55847feaf370 0x55847feb3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55847fea5b40 0x55847feb3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f00257fa640 0 client.17982 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:30:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 --2- 172.21.15.5:0/282541069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55847fea45d0 0x55847feb34d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 -- 172.21.15.5:0/282541069 >> 172.21.15.5:0/282541069 conn(0x55847fe6d310 msgr2=0x55847feae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 -- 172.21.15.5:0/282541069 shutdown_connections 2024-09-11T22:30:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 1 -- 172.21.15.5:0/282541069 wait complete. 2024-09-11T22:30:26.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.489+0000 7f004629a740 20 client.17982 trim_cache size 0 max 16384 2024-09-11T22:30:26.505 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:30:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 -- 172.21.15.5:0/420176864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00280a2d70 msgr2=0x7f00280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00280a2d70 0x7f00280a5230 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0034010c40 tx=0x7f003400c000 comp rx=0 tx=0).stop 2024-09-11T22:30:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 -- 172.21.15.5:0/420176864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f003c103740 msgr2=0x7f003c198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f003c103740 0x7f003c198b40 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f002c02a940 tx=0x7f002c003b90 comp rx=0 tx=0).stop 2024-09-11T22:30:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 -- 172.21.15.5:0/420176864 shutdown_connections 2024-09-11T22:30:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f003c108290 0x7f003c19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f003c103740 0x7f003c198b40 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00280a2d70 0x7f00280a5230 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 --2- 172.21.15.5:0/420176864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f003c102d90 0x7f003c198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:26.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 -- 172.21.15.5:0/420176864 >> 172.21.15.5:0/420176864 conn(0x7f003c0fc7d0 msgr2=0x7f003c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:26.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 -- 172.21.15.5:0/420176864 shutdown_connections 2024-09-11T22:30:26.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:26.490+0000 7f0043ec1640 1 -- 172.21.15.5:0/420176864 wait complete. 2024-09-11T22:30:27.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 8.03 MB/sec execute 285 sec latency 6972.332 ms 2024-09-11T22:30:27.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:30:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:27 smithi005 ceph-mon[30943]: pgmap v258: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 262 op/s 2024-09-11T22:30:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:27 smithi159 ceph-mon[35534]: pgmap v258: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 262 op/s 2024-09-11T22:30:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:27 smithi172 ceph-mon[35461]: pgmap v258: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.7 MiB/s wr, 262 op/s 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 -- 172.21.15.5:0/94677571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397c103780 msgr2=0x7f397c107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 --2- 172.21.15.5:0/94677571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397c103780 0x7f397c107bc0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f3970009930 tx=0x7f397002a420 comp rx=0 tx=0).stop 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 -- 172.21.15.5:0/94677571 shutdown_connections 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 --2- 172.21.15.5:0/94677571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c108300 0x7f397c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 --2- 172.21.15.5:0/94677571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397c103780 0x7f397c107bc0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 --2- 172.21.15.5:0/94677571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f397c102dd0 0x7f397c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 -- 172.21.15.5:0/94677571 >> 172.21.15.5:0/94677571 conn(0x7f397c0fc7d0 msgr2=0x7f397c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 -- 172.21.15.5:0/94677571 shutdown_connections 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 -- 172.21.15.5:0/94677571 wait complete. 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 Processor -- start 2024-09-11T22:30:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.229+0000 7f3982c27640 1 -- start start 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f3982c27640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f397c102dd0 0x7f397c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f3982c27640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c103780 0x7f397c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f3982c27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397c108300 0x7f397c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f3982c27640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f397c112050 con 0x7f397c108300 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f3982c27640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f397c111ed0 con 0x7f397c103780 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f3982c27640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1121d0 con 0x7f397c102dd0 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f39811c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397c108300 0x7f397c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f39811c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397c108300 0x7f397c19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43304/0 (socket says 172.21.15.5:43304) 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f39811c4640 1 -- 172.21.15.5:0/3195075972 learned_addr learned my addr 172.21.15.5:0/3195075972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f397bfff640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c103780 0x7f397c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f39809c3640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f397c102dd0 0x7f397c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f39811c4640 1 -- 172.21.15.5:0/3195075972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f397c102dd0 msgr2=0x7f397c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f39811c4640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f397c102dd0 0x7f397c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f39811c4640 1 -- 172.21.15.5:0/3195075972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c103780 msgr2=0x7f397c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f39811c4640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c103780 0x7f397c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f39811c4640 1 -- 172.21.15.5:0/3195075972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397c19d540 con 0x7f397c108300 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f39811c4640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397c108300 0x7f397c19ce60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f397400cad0 tx=0x7f397400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f3979ffb640 1 -- 172.21.15.5:0/3195075972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397400ae20 con 0x7f397c108300 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f3982c27640 1 -- 172.21.15.5:0/3195075972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397c19d830 con 0x7f397c108300 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f3979ffb640 1 -- 172.21.15.5:0/3195075972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3974005020 con 0x7f397c108300 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.230+0000 7f3979ffb640 1 -- 172.21.15.5:0/3195075972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397400f6f0 con 0x7f397c108300 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.231+0000 7f3982c27640 1 -- 172.21.15.5:0/3195075972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f397c1a5110 con 0x7f397c108300 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f3979ffb640 1 -- 172.21.15.5:0/3195075972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f39740051d0 con 0x7f397c108300 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f3985000740 1 Processor -- start 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f3985000740 1 -- start start 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f3985000740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5ecb40 0x558bcf5eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f3985000740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5eb5d0 0x558bcf5eba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f3985000740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f3985000740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558bcf5512f0 con 0x558bcf5f6370 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f3985000740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558bcf4f8cf0 con 0x558bcf5eb5d0 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f3985000740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558bcf54d2b0 con 0x558bcf5ecb40 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f397bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f397bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43318/0 (socket says 172.21.15.5:43318) 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f397bfff640 1 -- 172.21.15.5:0/1025037777 learned_addr learned my addr 172.21.15.5:0/1025037777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f39809c3640 1 --2- 172.21.15.5:0/1025037777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5eb5d0 0x558bcf5eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f397bfff640 1 --2- 172.21.15.5:0/1025037777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5ecb40 0x558bcf5eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.234+0000 7f3979ffb640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39640a2cf0 0x7f39640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f39809c3640 1 -- 172.21.15.5:0/1025037777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5ecb40 msgr2=0x558bcf5eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f39809c3640 1 --2- 172.21.15.5:0/1025037777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5ecb40 0x558bcf5eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f39809c3640 1 -- 172.21.15.5:0/1025037777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 msgr2=0x558bcf5f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f39809c3640 1 --2- 172.21.15.5:0/1025037777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f39809c3640 1 -- 172.21.15.5:0/1025037777 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bcf4dd3a0 con 0x558bcf5eb5d0 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f397bfff640 1 --2- 172.21.15.5:0/1025037777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5ecb40 0x558bcf5eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f3979ffb640 1 -- 172.21.15.5:0/3195075972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f39740cf4f0 con 0x7f397c108300 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f39811c4640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39640a2cf0 0x7f39640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f39809c3640 1 --2- 172.21.15.5:0/1025037777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5eb5d0 0x558bcf5eba10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f396c00cba0 tx=0x7f396c00e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb3aefc5e7c7ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f3962ffd640 1 -- 172.21.15.5:0/1025037777 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c004220 con 0x558bcf5eb5d0 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f3962ffd640 1 -- 172.21.15.5:0/1025037777 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f396c0043c0 con 0x558bcf5eb5d0 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.235+0000 7f3962ffd640 1 -- 172.21.15.5:0/1025037777 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c0108f0 con 0x558bcf5eb5d0 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 -- 172.21.15.5:0/1025037777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5eb5d0 msgr2=0x558bcf5eba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 --2- 172.21.15.5:0/1025037777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5eb5d0 0x558bcf5eba10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f396c00cba0 tx=0x7f396c00e720 comp rx=0 tx=0).stop 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f39811c4640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39640a2cf0 0x7f39640a51b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f397400a6e0 tx=0x7f3974005b40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 -- 172.21.15.5:0/1025037777 shutdown_connections 2024-09-11T22:30:28.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 --2- 172.21.15.5:0/1025037777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 --2- 172.21.15.5:0/1025037777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5eb5d0 0x558bcf5eba10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 --2- 172.21.15.5:0/1025037777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5ecb40 0x558bcf5eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 -- 172.21.15.5:0/1025037777 >> 172.21.15.5:0/1025037777 conn(0x558bcf5b4310 msgr2=0x558bcf5e7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 -- 172.21.15.5:0/1025037777 shutdown_connections 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 -- 172.21.15.5:0/1025037777 wait complete. 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 -- start start 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.236+0000 7f3985000740 1 Processor -- start 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f3985000740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f3985000740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 0x558bcf5f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f3985000740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f3985000740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558bcf5f9070 con 0x558bcf5f6370 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f3985000740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558bcf50c060 con 0x558bcf5ecb40 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f3985000740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558bcf5f91f0 con 0x558bcf5eb5d0 2024-09-11T22:30:28.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f3985000740 10 client.0 _ll_register_callbacks cb 0x558bcf25f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f397bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f397bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43320/0 (socket says 172.21.15.5:43320) 2024-09-11T22:30:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f397bfff640 1 -- 172.21.15.5:0/3378860835 learned_addr learned my addr 172.21.15.5:0/3378860835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f39809c3640 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 0x558bcf5f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f397bfff640 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f397bfff640 1 -- 172.21.15.5:0/3378860835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5eb5d0 msgr2=0x558bcf5f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f397bfff640 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f397bfff640 1 -- 172.21.15.5:0/3378860835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 msgr2=0x558bcf5f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:28.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f397bfff640 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 0x558bcf5f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f397bfff640 1 -- 172.21.15.5:0/3378860835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bcf2c23c0 con 0x558bcf5f6370 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f39809c3640 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 0x558bcf5f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f397bfff640 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f3530 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3970009230 tx=0x7f3970041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec46497dc3f03b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.237+0000 7f395bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f395bfff640 1 -- 172.21.15.5:0/3378860835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970035030 con 0x558bcf5f6370 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f395bfff640 1 -- 172.21.15.5:0/3378860835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f3970004200 con 0x558bcf5f6370 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f3985000740 1 -- 172.21.15.5:0/3378860835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f396c0043c0 con 0x558bcf5f6370 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f395bfff640 1 -- 172.21.15.5:0/3378860835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3970004400 con 0x558bcf5f6370 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f3985000740 10 client.17994 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f3985000740 1 -- 172.21.15.5:0/3378860835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558bcf604aa0 con 0x558bcf5f6370 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f395bfff640 1 -- 172.21.15.5:0/3378860835 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f396c0043c0 con 0x558bcf5f6370 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f395bfff640 1 -- 172.21.15.5:0/3378860835 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f397003d6b0 con 0x558bcf5f6370 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f3985000740 10 client.17994 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:28.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f3985000740 10 client.17994 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f3985000740 20 client.17994 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f3985000740 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x558bcf605f60 0x558bcf608350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f3985000740 4 client.17994 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.238+0000 7f3985000740 1 -- 172.21.15.5:0/3378860835 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bcf55e860 con 0x558bcf605f60 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.239+0000 7f39809c3640 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x558bcf605f60 0x558bcf608350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.239+0000 7f39809c3640 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x558bcf605f60 0x558bcf608350 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6df0f5a61593cf7 server_cookie=ca8353eebdecaea8 in_seq=0 out_seq=0 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.239+0000 7f395bfff640 10 client.17994 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f395bfff640 1 -- 172.21.15.5:0/3378860835 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558bcf55e860 con 0x558bcf605f60 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f395bfff640 10 client.17994 handle_command_reply: tid=0 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 client.17994 shutdown 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 -- 172.21.15.5:0/3378860835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 msgr2=0x558bcf5f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f3530 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f3970009230 tx=0x7f3970041040 comp rx=0 tx=0).stop 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 -- 172.21.15.5:0/3378860835 shutdown_connections 2024-09-11T22:30:28.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x558bcf605f60 0x558bcf608350 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5f6370 0x558bcf5f3530 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 0x558bcf5f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 --2- 172.21.15.5:0/3378860835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f395bfff640 0 client.17994 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:30:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 -- 172.21.15.5:0/3378860835 >> 172.21.15.5:0/3378860835 conn(0x558bcf5b4310 msgr2=0x558bcf5ed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 -- 172.21.15.5:0/3378860835 shutdown_connections 2024-09-11T22:30:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 1 -- 172.21.15.5:0/3378860835 wait complete. 2024-09-11T22:30:28.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.240+0000 7f3985000740 20 client.17994 trim_cache size 0 max 16384 2024-09-11T22:30:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f3985000740 1 Processor -- start 2024-09-11T22:30:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f3985000740 1 -- start start 2024-09-11T22:30:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f3985000740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f3985000740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5ecb40 0x558bcf5ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f3985000740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5f6370 0x558bcf5ee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f3985000740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1ada50 con 0x558bcf5ecb40 2024-09-11T22:30:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f3985000740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1af590 con 0x558bcf5f6370 2024-09-11T22:30:28.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f3985000740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1ae120 con 0x558bcf5eb5d0 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f39809c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5ecb40 0x558bcf5ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f39809c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5ecb40 0x558bcf5ee120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43322/0 (socket says 172.21.15.5:43322) 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f39809c3640 1 -- 172.21.15.5:0/1225559172 learned_addr learned my addr 172.21.15.5:0/1225559172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f397bfff640 1 --2- 172.21.15.5:0/1225559172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5f6370 0x558bcf5ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.253+0000 7f397bfff640 1 --2- 172.21.15.5:0/1225559172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.254+0000 7f397bfff640 1 -- 172.21.15.5:0/1225559172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5eb5d0 msgr2=0x558bcf5f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.254+0000 7f397bfff640 1 --2- 172.21.15.5:0/1225559172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.254+0000 7f397bfff640 1 -- 172.21.15.5:0/1225559172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5ecb40 msgr2=0x558bcf5ee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.254+0000 7f397bfff640 1 --2- 172.21.15.5:0/1225559172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5ecb40 0x558bcf5ee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.254+0000 7f397bfff640 1 -- 172.21.15.5:0/1225559172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bcf5e8730 con 0x558bcf5f6370 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.254+0000 7f39809c3640 1 --2- 172.21.15.5:0/1225559172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5ecb40 0x558bcf5ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.254+0000 7f397bfff640 1 --2- 172.21.15.5:0/1225559172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5f6370 0x558bcf5ee660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f397002a940 tx=0x7f3970009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7251038e003e6e8d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:28.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.254+0000 7f39627fc640 1 -- 172.21.15.5:0/1225559172 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970058040 con 0x558bcf5f6370 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.254+0000 7f39627fc640 1 -- 172.21.15.5:0/1225559172 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3970059040 con 0x558bcf5f6370 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.254+0000 7f39627fc640 1 -- 172.21.15.5:0/1225559172 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970035040 con 0x558bcf5f6370 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 -- 172.21.15.5:0/1225559172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5f6370 msgr2=0x558bcf5ee660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 --2- 172.21.15.5:0/1225559172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5f6370 0x558bcf5ee660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f397002a940 tx=0x7f3970009260 comp rx=0 tx=0).stop 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 -- 172.21.15.5:0/1225559172 shutdown_connections 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 --2- 172.21.15.5:0/1225559172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5f6370 0x558bcf5ee660 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 --2- 172.21.15.5:0/1225559172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5ecb40 0x558bcf5ee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 --2- 172.21.15.5:0/1225559172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 -- 172.21.15.5:0/1225559172 >> 172.21.15.5:0/1225559172 conn(0x558bcf5b4310 msgr2=0x558bcf5f4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 -- 172.21.15.5:0/1225559172 shutdown_connections 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 -- 172.21.15.5:0/1225559172 wait complete. 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 -- start start 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.255+0000 7f3985000740 1 Processor -- start 2024-09-11T22:30:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f3985000740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f3985000740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 0x558bcf5f0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f3985000740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5f6370 0x558bcf5f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f3985000740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1aeec0 con 0x558bcf5eb5d0 2024-09-11T22:30:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f3985000740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1ae7f0 con 0x558bcf5ecb40 2024-09-11T22:30:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f3985000740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f397c1aba50 con 0x558bcf5f6370 2024-09-11T22:30:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f3985000740 10 client.0 _ll_register_callbacks cb 0x7f3970032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f397bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f397bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43328/0 (socket says 172.21.15.5:43328) 2024-09-11T22:30:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f397bfff640 1 -- 172.21.15.5:0/1586777590 learned_addr learned my addr 172.21.15.5:0/1586777590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f39809c3640 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 0x558bcf5f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f397bfff640 1 -- 172.21.15.5:0/1586777590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5f6370 msgr2=0x558bcf5f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f397bfff640 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5f6370 0x558bcf5f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f397bfff640 1 -- 172.21.15.5:0/1586777590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 msgr2=0x558bcf5f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f397bfff640 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 0x558bcf5f0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f397bfff640 1 -- 172.21.15.5:0/1586777590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bcf66fbf0 con 0x558bcf5eb5d0 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f397bfff640 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f0030 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f3970003c60 tx=0x7f397002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8beaddc1a162037 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f395bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f395bfff640 1 -- 172.21.15.5:0/1586777590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970058030 con 0x558bcf5eb5d0 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f395bfff640 1 -- 172.21.15.5:0/1586777590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f39700035f0 con 0x558bcf5eb5d0 2024-09-11T22:30:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.256+0000 7f3985000740 1 -- 172.21.15.5:0/1586777590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3970059040 con 0x558bcf5eb5d0 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f395bfff640 1 -- 172.21.15.5:0/1586777590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3970041040 con 0x558bcf5eb5d0 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f3985000740 10 client.18000 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f3985000740 1 -- 172.21.15.5:0/1586777590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558bcf670ed0 con 0x558bcf5eb5d0 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f395bfff640 1 -- 172.21.15.5:0/1586777590 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3970059040 con 0x558bcf5eb5d0 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f395bfff640 1 -- 172.21.15.5:0/1586777590 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f397005e9f0 con 0x558bcf5eb5d0 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f3985000740 10 client.18000 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f3985000740 10 client.18000 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f3985000740 20 client.18000 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f3985000740 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x558bcf671530 0x558bcf673950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f3985000740 4 client.18000 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f3985000740 1 -- 172.21.15.5:0/1586777590 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bcf558d70 con 0x558bcf671530 2024-09-11T22:30:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.257+0000 7f39809c3640 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x558bcf671530 0x558bcf673950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f39809c3640 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x558bcf671530 0x558bcf673950 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37d0fa27926b4278 server_cookie=d1cf5b9241cfd78d in_seq=0 out_seq=0 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f395bfff640 10 client.18000 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f395bfff640 1 -- 172.21.15.5:0/1586777590 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bcf558d70 con 0x558bcf671530 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f395bfff640 10 client.18000 handle_command_reply: tid=0 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 client.18000 shutdown 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 -- 172.21.15.5:0/1586777590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5eb5d0 msgr2=0x558bcf5f0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f0030 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f3970003c60 tx=0x7f397002a420 comp rx=0 tx=0).stop 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 -- 172.21.15.5:0/1586777590 shutdown_connections 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x558bcf671530 0x558bcf673950 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558bcf5f6370 0x558bcf5f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558bcf5ecb40 0x558bcf5f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f395bfff640 0 client.18000 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:30:28.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 --2- 172.21.15.5:0/1586777590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558bcf5eb5d0 0x558bcf5f0030 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 -- 172.21.15.5:0/1586777590 >> 172.21.15.5:0/1586777590 conn(0x558bcf5b4310 msgr2=0x558bcf5f5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 -- 172.21.15.5:0/1586777590 shutdown_connections 2024-09-11T22:30:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 1 -- 172.21.15.5:0/1586777590 wait complete. 2024-09-11T22:30:28.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.258+0000 7f3985000740 20 client.18000 trim_cache size 0 max 16384 2024-09-11T22:30:28.268 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:30:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 -- 172.21.15.5:0/3195075972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39640a2cf0 msgr2=0x7f39640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39640a2cf0 0x7f39640a51b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f397400a6e0 tx=0x7f3974005b40 comp rx=0 tx=0).stop 2024-09-11T22:30:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 -- 172.21.15.5:0/3195075972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397c108300 msgr2=0x7f397c19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397c108300 0x7f397c19ce60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f397400cad0 tx=0x7f397400cf90 comp rx=0 tx=0).stop 2024-09-11T22:30:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 -- 172.21.15.5:0/3195075972 shutdown_connections 2024-09-11T22:30:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f397c108300 0x7f397c19ce60 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39640a2cf0 0x7f39640a51b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f397c103780 0x7f397c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 --2- 172.21.15.5:0/3195075972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f397c102dd0 0x7f397c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 -- 172.21.15.5:0/3195075972 >> 172.21.15.5:0/3195075972 conn(0x7f397c0fc7d0 msgr2=0x7f397c105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 -- 172.21.15.5:0/3195075972 shutdown_connections 2024-09-11T22:30:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:28.259+0000 7f3982c27640 1 -- 172.21.15.5:0/3195075972 wait complete. 2024-09-11T22:30:28.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 8.01 MB/sec execute 286 sec latency 7972.431 ms 2024-09-11T22:30:29.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:30:29.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 7.98 MB/sec execute 287 sec latency 8972.493 ms 2024-09-11T22:30:29.446 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:29 smithi005 ceph-mon[30943]: pgmap v259: 97 pgs: 97 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-09-11T22:30:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:29 smithi159 ceph-mon[35534]: pgmap v259: 97 pgs: 97 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-09-11T22:30:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:29 smithi172 ceph-mon[35461]: pgmap v259: 97 pgs: 97 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-09-11T22:30:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.180+0000 7f268a2d3640 1 -- 172.21.15.5:0/3005269000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2684103660 msgr2=0x7f268410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:30.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.180+0000 7f268a2d3640 1 --2- 172.21.15.5:0/3005269000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2684103660 0x7f268410bb50 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f266c009900 tx=0x7f266c02a480 comp rx=0 tx=0).stop 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.180+0000 7f268a2d3640 1 -- 172.21.15.5:0/3005269000 shutdown_connections 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.180+0000 7f268a2d3640 1 --2- 172.21.15.5:0/3005269000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f268410c170 0x7f268410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.180+0000 7f268a2d3640 1 --2- 172.21.15.5:0/3005269000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2684103660 0x7f268410bb50 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.180+0000 7f268a2d3640 1 --2- 172.21.15.5:0/3005269000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2684102d40 0x7f2684103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.180+0000 7f268a2d3640 1 -- 172.21.15.5:0/3005269000 >> 172.21.15.5:0/3005269000 conn(0x7f26840fc7a0 msgr2=0x7f26840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.180+0000 7f268a2d3640 1 -- 172.21.15.5:0/3005269000 shutdown_connections 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.180+0000 7f268a2d3640 1 -- 172.21.15.5:0/3005269000 wait complete. 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f268a2d3640 1 Processor -- start 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f268a2d3640 1 -- start start 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f268a2d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2684102d40 0x7f26841009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f268a2d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2684103660 0x7f2684100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f268a2d3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f268410c170 0x7f2684101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f268a2d3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2684111f70 con 0x7f2684102d40 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f268a2d3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2684111df0 con 0x7f268410c170 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f268a2d3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26841120f0 con 0x7f2684103660 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f2683fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2684102d40 0x7f26841009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f2683fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2684102d40 0x7f26841009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43344/0 (socket says 172.21.15.5:43344) 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.181+0000 7f2683fff640 1 -- 172.21.15.5:0/1012951062 learned_addr learned my addr 172.21.15.5:0/1012951062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f267bfff640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2684103660 0x7f2684100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2688870640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f268410c170 0x7f2684101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2683fff640 1 -- 172.21.15.5:0/1012951062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2684103660 msgr2=0x7f2684100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2683fff640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2684103660 0x7f2684100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2683fff640 1 -- 172.21.15.5:0/1012951062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f268410c170 msgr2=0x7f2684101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2683fff640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f268410c170 0x7f2684101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2683fff640 1 -- 172.21.15.5:0/1012951062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26841a6930 con 0x7f2684102d40 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2688870640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f268410c170 0x7f2684101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f267bfff640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2684103660 0x7f2684100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2683fff640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2684102d40 0x7f26841009a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f267400d950 tx=0x7f267400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2681ffb640 1 -- 172.21.15.5:0/1012951062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674017070 con 0x7f2684102d40 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f268a2d3640 1 -- 172.21.15.5:0/1012951062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26841a6c20 con 0x7f2684102d40 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2681ffb640 1 -- 172.21.15.5:0/1012951062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2674009c30 con 0x7f2684102d40 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f2681ffb640 1 -- 172.21.15.5:0/1012951062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267400f6b0 con 0x7f2684102d40 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.182+0000 7f268a2d3640 1 -- 172.21.15.5:0/1012951062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26841a7180 con 0x7f2684102d40 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f2681ffb640 1 -- 172.21.15.5:0/1012951062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f26740109d0 con 0x7f2684102d40 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f268c6ac740 1 Processor -- start 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f268c6ac740 1 -- start start 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550effb40 0x558550efe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550efe5d0 0x558550efea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 0x558550f0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f268c6ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558550e642f0 con 0x558550efe5d0 2024-09-11T22:30:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f268c6ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558550e0bcf0 con 0x558550effb40 2024-09-11T22:30:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f268c6ac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558550e602b0 con 0x558550f09370 2024-09-11T22:30:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f2683fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550efe5d0 0x558550efea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f2683fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550efe5d0 0x558550efea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43356/0 (socket says 172.21.15.5:43356) 2024-09-11T22:30:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f2683fff640 1 -- 172.21.15.5:0/1842427700 learned_addr learned my addr 172.21.15.5:0/1842427700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f2688870640 1 --2- 172.21.15.5:0/1842427700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550effb40 0x558550efe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.185+0000 7f2681ffb640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f26700a2d70 0x7f26700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f267bfff640 1 --2- 172.21.15.5:0/1842427700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 0x558550f0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f2688870640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f26700a2d70 0x7f26700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f2681ffb640 1 -- 172.21.15.5:0/1012951062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f26740cf050 con 0x7f2684102d40 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f2683fff640 1 -- 172.21.15.5:0/1842427700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 msgr2=0x558550f0b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f2683fff640 1 --2- 172.21.15.5:0/1842427700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 0x558550f0b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f2683fff640 1 -- 172.21.15.5:0/1842427700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550effb40 msgr2=0x558550efe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f2683fff640 1 --2- 172.21.15.5:0/1842427700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550effb40 0x558550efe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f2683fff640 1 -- 172.21.15.5:0/1842427700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558550df03a0 con 0x558550efe5d0 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f2683fff640 1 --2- 172.21.15.5:0/1842427700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550efe5d0 0x558550efea10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f267409b200 tx=0x7f2674000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3d3843d17fdffc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f2688870640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f26700a2d70 0x7f26700a5230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f267c005fd0 tx=0x7f267c004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f267a7fc640 1 -- 172.21.15.5:0/1842427700 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674020070 con 0x558550efe5d0 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f267a7fc640 1 -- 172.21.15.5:0/1842427700 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267409b500 con 0x558550efe5d0 2024-09-11T22:30:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.186+0000 7f267a7fc640 1 -- 172.21.15.5:0/1842427700 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674097b70 con 0x558550efe5d0 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.187+0000 7f268c6ac740 1 -- 172.21.15.5:0/1842427700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550efe5d0 msgr2=0x558550efea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.187+0000 7f268c6ac740 1 --2- 172.21.15.5:0/1842427700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550efe5d0 0x558550efea10 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f267409b200 tx=0x7f2674000bc0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.187+0000 7f268c6ac740 1 -- 172.21.15.5:0/1842427700 shutdown_connections 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.187+0000 7f268c6ac740 1 --2- 172.21.15.5:0/1842427700 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 0x558550f0b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.187+0000 7f268c6ac740 1 --2- 172.21.15.5:0/1842427700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550efe5d0 0x558550efea10 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.187+0000 7f268c6ac740 1 --2- 172.21.15.5:0/1842427700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550effb40 0x558550efe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.187+0000 7f268c6ac740 1 -- 172.21.15.5:0/1842427700 >> 172.21.15.5:0/1842427700 conn(0x558550ec7310 msgr2=0x558550efac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.187+0000 7f268c6ac740 1 -- 172.21.15.5:0/1842427700 shutdown_connections 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.187+0000 7f268c6ac740 1 -- 172.21.15.5:0/1842427700 wait complete. 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f268c6ac740 1 -- start start 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f268c6ac740 1 Processor -- start 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 0x558550f08d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f06040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 0x558550f06650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f268c6ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558550f0bfb0 con 0x558550effb40 2024-09-11T22:30:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f268c6ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558550e1f060 con 0x558550f09370 2024-09-11T22:30:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f268c6ac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558550f0c130 con 0x558550efe5d0 2024-09-11T22:30:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f268c6ac740 10 client.0 _ll_register_callbacks cb 0x558550b728c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f267bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f06040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f267bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f06040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43372/0 (socket says 172.21.15.5:43372) 2024-09-11T22:30:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.188+0000 7f267bfff640 1 -- 172.21.15.5:0/3318880547 learned_addr learned my addr 172.21.15.5:0/3318880547 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f2688870640 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 0x558550f06650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f2683fff640 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 0x558550f08d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f267bfff640 1 -- 172.21.15.5:0/3318880547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 msgr2=0x558550f08d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f267bfff640 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 0x558550f08d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f267bfff640 1 -- 172.21.15.5:0/3318880547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 msgr2=0x558550f06650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f267bfff640 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 0x558550f06650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f267bfff640 1 -- 172.21.15.5:0/3318880547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558550bd53c0 con 0x558550effb40 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f2688870640 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 0x558550f06650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f267bfff640 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f06040 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f266c009900 tx=0x7f266c031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d08a7405218c6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f26637fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f26637fe640 1 -- 172.21.15.5:0/3318880547 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c035040 con 0x558550effb40 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f26637fe640 1 -- 172.21.15.5:0/3318880547 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f266c004450 con 0x558550effb40 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f268c6ac740 1 -- 172.21.15.5:0/3318880547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267409b500 con 0x558550effb40 2024-09-11T22:30:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f26637fe640 1 -- 172.21.15.5:0/3318880547 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f266c005020 con 0x558550effb40 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f268c6ac740 10 client.18012 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f268c6ac740 1 -- 172.21.15.5:0/3318880547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558550f17b00 con 0x558550effb40 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.189+0000 7f26637fe640 1 -- 172.21.15.5:0/3318880547 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267409b500 con 0x558550effb40 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.190+0000 7f26637fe640 1 -- 172.21.15.5:0/3318880547 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f266c0035f0 con 0x558550effb40 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.190+0000 7f268c6ac740 10 client.18012 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.190+0000 7f268c6ac740 10 client.18012 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.190+0000 7f268c6ac740 20 client.18012 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.190+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558550f19010 0x558550f1b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.190+0000 7f268c6ac740 4 client.18012 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.190+0000 7f268c6ac740 1 -- 172.21.15.5:0/3318880547 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558550e71860 con 0x558550f19010 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.190+0000 7f2683fff640 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558550f19010 0x558550f1b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.190+0000 7f2683fff640 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558550f19010 0x558550f1b450 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeebfba4d1feed50 server_cookie=8956d857a3d1a836 in_seq=0 out_seq=0 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.190+0000 7f26637fe640 10 client.18012 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:30.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f26637fe640 1 -- 172.21.15.5:0/3318880547 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558550e71860 con 0x558550f19010 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f26637fe640 10 client.18012 handle_command_reply: tid=0 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 client.18012 shutdown 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 -- 172.21.15.5:0/3318880547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 msgr2=0x558550f06040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f06040 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f266c009900 tx=0x7f266c031850 comp rx=0 tx=0).stop 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 -- 172.21.15.5:0/3318880547 shutdown_connections 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558550f19010 0x558550f1b450 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 0x558550f06650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f06040 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3318880547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 0x558550f08d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 -- 172.21.15.5:0/3318880547 >> 172.21.15.5:0/3318880547 conn(0x558550ec7310 msgr2=0x558550f00990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f26637fe640 0 client.18012 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 -- 172.21.15.5:0/3318880547 shutdown_connections 2024-09-11T22:30:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.191+0000 7f268c6ac740 1 -- 172.21.15.5:0/3318880547 wait complete. 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.192+0000 7f268c6ac740 20 client.18012 trim_cache size 0 max 16384 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.205+0000 7f268c6ac740 1 Processor -- start 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.205+0000 7f268c6ac740 1 -- start start 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.205+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 0x558550f15280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.205+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f01180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.205+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 0x558550f016c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.205+0000 7f268c6ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26841afbf0 con 0x558550effb40 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.205+0000 7f268c6ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b1730 con 0x558550f09370 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.205+0000 7f268c6ac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b02c0 con 0x558550efe5d0 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f267bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f01180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2683fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 0x558550f15280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2688870640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 0x558550f016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f267bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f01180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43386/0 (socket says 172.21.15.5:43386) 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f267bfff640 1 -- 172.21.15.5:0/1666333821 learned_addr learned my addr 172.21.15.5:0/1666333821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2683fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 0x558550f15280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:53604/0 (socket says 172.21.15.5:53604) 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2688870640 1 -- 172.21.15.5:0/1666333821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 msgr2=0x558550f15280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2688870640 1 --2- 172.21.15.5:0/1666333821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 0x558550f15280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2688870640 1 -- 172.21.15.5:0/1666333821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 msgr2=0x558550f01180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2688870640 1 --2- 172.21.15.5:0/1666333821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f01180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2688870640 1 -- 172.21.15.5:0/1666333821 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558550eff760 con 0x558550f09370 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f267bfff640 1 --2- 172.21.15.5:0/1666333821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f01180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2683fff640 1 --2- 172.21.15.5:0/1666333821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 0x558550f15280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2688870640 1 --2- 172.21.15.5:0/1666333821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 0x558550f016c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f267c012f90 tx=0x7f267c0099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97be9253e1f9af7a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.206+0000 7f2679ffb640 1 -- 172.21.15.5:0/1666333821 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c01d740 con 0x558550f09370 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.207+0000 7f2679ffb640 1 -- 172.21.15.5:0/1666333821 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267c01ddc0 con 0x558550f09370 2024-09-11T22:30:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.207+0000 7f2679ffb640 1 -- 172.21.15.5:0/1666333821 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c01e3f0 con 0x558550f09370 2024-09-11T22:30:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.207+0000 7f268c6ac740 1 -- 172.21.15.5:0/1666333821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 msgr2=0x558550f016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.207+0000 7f268c6ac740 1 --2- 172.21.15.5:0/1666333821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 0x558550f016c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f267c012f90 tx=0x7f267c0099c0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.207+0000 7f268c6ac740 1 -- 172.21.15.5:0/1666333821 shutdown_connections 2024-09-11T22:30:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.207+0000 7f268c6ac740 1 --2- 172.21.15.5:0/1666333821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550f09370 0x558550f016c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.207+0000 7f268c6ac740 1 --2- 172.21.15.5:0/1666333821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f01180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.207+0000 7f268c6ac740 1 --2- 172.21.15.5:0/1666333821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550efe5d0 0x558550f15280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.207+0000 7f268c6ac740 1 -- 172.21.15.5:0/1666333821 >> 172.21.15.5:0/1666333821 conn(0x558550ec7310 msgr2=0x558550f05130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.207+0000 7f268c6ac740 1 -- 172.21.15.5:0/1666333821 shutdown_connections 2024-09-11T22:30:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.208+0000 7f268c6ac740 1 -- 172.21.15.5:0/1666333821 wait complete. 2024-09-11T22:30:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.208+0000 7f268c6ac740 1 -- start start 2024-09-11T22:30:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.208+0000 7f268c6ac740 1 Processor -- start 2024-09-11T22:30:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.208+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550efe5d0 0x558550f11750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.208+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f11c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.208+0000 7f268c6ac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 0x558550f0cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.208+0000 7f268c6ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b1060 con 0x558550effb40 2024-09-11T22:30:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f268c6ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26841b0990 con 0x558550efe5d0 2024-09-11T22:30:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f268c6ac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f26841adbf0 con 0x558550f09370 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f268c6ac740 10 client.0 _ll_register_callbacks cb 0x7f266c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f267bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f11c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f267bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f11c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43394/0 (socket says 172.21.15.5:43394) 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f267bfff640 1 -- 172.21.15.5:0/3080285825 learned_addr learned my addr 172.21.15.5:0/3080285825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f2683fff640 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550efe5d0 0x558550f11750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f2688870640 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 0x558550f0cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f267bfff640 1 -- 172.21.15.5:0/3080285825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 msgr2=0x558550f0cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f267bfff640 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 0x558550f0cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f267bfff640 1 -- 172.21.15.5:0/3080285825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550efe5d0 msgr2=0x558550f11750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f267bfff640 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550efe5d0 0x558550f11750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f267bfff640 1 -- 172.21.15.5:0/3080285825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558550f0d550 con 0x558550effb40 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f2688870640 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 0x558550f0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f267bfff640 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f11c90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f266c009900 tx=0x7f266c003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aaf9abeb2f2e5a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.209+0000 7f26637fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f26637fe640 1 -- 172.21.15.5:0/3080285825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266c035070 con 0x558550effb40 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f26637fe640 1 -- 172.21.15.5:0/3080285825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f266c004250 con 0x558550effb40 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f268c6ac740 1 -- 172.21.15.5:0/3080285825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267c01ddc0 con 0x558550effb40 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f26637fe640 1 -- 172.21.15.5:0/3080285825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f266c065050 con 0x558550effb40 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f268c6ac740 10 client.18018 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f268c6ac740 1 -- 172.21.15.5:0/3080285825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558550f83910 con 0x558550effb40 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f26637fe640 1 -- 172.21.15.5:0/3080285825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267c01ddc0 con 0x558550effb40 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f26637fe640 1 -- 172.21.15.5:0/3080285825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f266c060030 con 0x558550effb40 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f268c6ac740 10 client.18018 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f268c6ac740 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f268c6ac740 20 client.18018 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558550f84080 0x558550f864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f268c6ac740 4 client.18018 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.210+0000 7f268c6ac740 1 -- 172.21.15.5:0/3080285825 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558550e6bd70 con 0x558550f84080 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.211+0000 7f2683fff640 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558550f84080 0x558550f864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.211+0000 7f2683fff640 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558550f84080 0x558550f864a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95b1edfe523a6c29 server_cookie=74cdbf038c39c191 in_seq=0 out_seq=0 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.211+0000 7f26637fe640 10 client.18018 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.211+0000 7f26637fe640 1 -- 172.21.15.5:0/3080285825 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558550e6bd70 con 0x558550f84080 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.211+0000 7f26637fe640 10 client.18018 handle_command_reply: tid=0 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.211+0000 7f268c6ac740 1 client.18018 shutdown 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.211+0000 7f268c6ac740 1 -- 172.21.15.5:0/3080285825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 msgr2=0x558550f11c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.211+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f11c90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f266c009900 tx=0x7f266c003e30 comp rx=0 tx=0).stop 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.211+0000 7f268c6ac740 1 -- 172.21.15.5:0/3080285825 shutdown_connections 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.211+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558550f84080 0x558550f864a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558550f09370 0x558550f0cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558550effb40 0x558550f11c90 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268c6ac740 1 --2- 172.21.15.5:0/3080285825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558550efe5d0 0x558550f11750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268c6ac740 1 -- 172.21.15.5:0/3080285825 >> 172.21.15.5:0/3080285825 conn(0x558550ec7310 msgr2=0x558550ef9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f26637fe640 0 client.18018 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268c6ac740 1 -- 172.21.15.5:0/3080285825 shutdown_connections 2024-09-11T22:30:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268c6ac740 1 -- 172.21.15.5:0/3080285825 wait complete. 2024-09-11T22:30:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268c6ac740 20 client.18018 trim_cache size 0 max 16384 2024-09-11T22:30:30.228 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:30:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268a2d3640 1 -- 172.21.15.5:0/1012951062 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f26700a2d70 msgr2=0x7f26700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268a2d3640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f26700a2d70 0x7f26700a5230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f267c005fd0 tx=0x7f267c004ab0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268a2d3640 1 -- 172.21.15.5:0/1012951062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2684102d40 msgr2=0x7f26841009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.212+0000 7f268a2d3640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2684102d40 0x7f26841009a0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f267400d950 tx=0x7f267400de10 comp rx=0 tx=0).stop 2024-09-11T22:30:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.213+0000 7f268a2d3640 1 -- 172.21.15.5:0/1012951062 shutdown_connections 2024-09-11T22:30:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.213+0000 7f268a2d3640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f26700a2d70 0x7f26700a5230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.213+0000 7f268a2d3640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f268410c170 0x7f2684101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.213+0000 7f268a2d3640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2684103660 0x7f2684100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.213+0000 7f268a2d3640 1 --2- 172.21.15.5:0/1012951062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2684102d40 0x7f26841009a0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.213+0000 7f268a2d3640 1 -- 172.21.15.5:0/1012951062 >> 172.21.15.5:0/1012951062 conn(0x7f26840fc7a0 msgr2=0x7f26840fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.213+0000 7f268a2d3640 1 -- 172.21.15.5:0/1012951062 shutdown_connections 2024-09-11T22:30:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:30.213+0000 7f268a2d3640 1 -- 172.21.15.5:0/1012951062 wait complete. 2024-09-11T22:30:30.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 7.95 MB/sec execute 288 sec latency 9972.584 ms 2024-09-11T22:30:31.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 7.92 MB/sec execute 289 sec latency 10972.646 ms 2024-09-11T22:30:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:31 smithi005 ceph-mon[30943]: pgmap v260: 97 pgs: 97 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-11T22:30:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:31 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:30:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:31 smithi159 ceph-mon[35534]: pgmap v260: 97 pgs: 97 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-11T22:30:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:31 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:30:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:31 smithi172 ceph-mon[35461]: pgmap v260: 97 pgs: 97 active+clean; 100 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 153 op/s 2024-09-11T22:30:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:31 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:30:32.142 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:30:32.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:30:32.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 7.90 MB/sec execute 290 sec latency 11972.681 ms 2024-09-11T22:30:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:32.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:32 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:30:32.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:32 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:30:32.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:32 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:30:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:32 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:30:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:32 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:30:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:32 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:30:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:32 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:30:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:32 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:30:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:32 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:30:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.936+0000 7f7d8612a640 1 -- 172.21.15.5:0/1614655532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d801082a0 msgr2=0x7f7d8010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.936+0000 7f7d8612a640 1 --2- 172.21.15.5:0/1614655532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d801082a0 0x7f7d8010ec40 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009930 tx=0x7f7d7802a420 comp rx=0 tx=0).stop 2024-09-11T22:30:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.936+0000 7f7d8612a640 1 -- 172.21.15.5:0/1614655532 shutdown_connections 2024-09-11T22:30:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.936+0000 7f7d8612a640 1 --2- 172.21.15.5:0/1614655532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d801082a0 0x7f7d8010ec40 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.936+0000 7f7d8612a640 1 --2- 172.21.15.5:0/1614655532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d801036e0 0x7f7d80107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.936+0000 7f7d8612a640 1 --2- 172.21.15.5:0/1614655532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d80102d30 0x7f7d80103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.936+0000 7f7d8612a640 1 -- 172.21.15.5:0/1614655532 >> 172.21.15.5:0/1614655532 conn(0x7f7d800fc7d0 msgr2=0x7f7d800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.936+0000 7f7d8612a640 1 -- 172.21.15.5:0/1614655532 shutdown_connections 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.936+0000 7f7d8612a640 1 -- 172.21.15.5:0/1614655532 wait complete. 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.936+0000 7f7d8612a640 1 Processor -- start 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d8612a640 1 -- start start 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d8612a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d80102d30 0x7f7d80100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d8612a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d801036e0 0x7f7d80100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d8612a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d801082a0 0x7f7d80101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d8612a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801120a0 con 0x7f7d801036e0 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d8612a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80111f20 con 0x7f7d801082a0 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d8612a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80112220 con 0x7f7d80102d30 2024-09-11T22:30:32.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d801036e0 0x7f7d80100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d801036e0 0x7f7d80100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35204/0 (socket says 172.21.15.5:35204) 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d7effd640 1 -- 172.21.15.5:0/2532300356 learned_addr learned my addr 172.21.15.5:0/2532300356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d80102d30 0x7f7d80100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d801082a0 0x7f7d80101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d7effd640 1 -- 172.21.15.5:0/2532300356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d80102d30 msgr2=0x7f7d80100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d7effd640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d80102d30 0x7f7d80100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d7effd640 1 -- 172.21.15.5:0/2532300356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d801082a0 msgr2=0x7f7d80101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d7effd640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d801082a0 0x7f7d80101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.937+0000 7f7d7effd640 1 -- 172.21.15.5:0/2532300356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d801a6980 con 0x7f7d801036e0 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.938+0000 7f7d7effd640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d801036e0 0x7f7d80100ed0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c970 tx=0x7f7d6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.938+0000 7f7d7cff9640 1 -- 172.21.15.5:0/2532300356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68007bb0 con 0x7f7d801036e0 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.938+0000 7f7d8612a640 1 -- 172.21.15.5:0/2532300356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d801a6c70 con 0x7f7d801036e0 2024-09-11T22:30:32.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.938+0000 7f7d7cff9640 1 -- 172.21.15.5:0/2532300356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d68007d50 con 0x7f7d801036e0 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.938+0000 7f7d7cff9640 1 -- 172.21.15.5:0/2532300356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6800f450 con 0x7f7d801036e0 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.938+0000 7f7d8612a640 1 -- 172.21.15.5:0/2532300356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d801a71b0 con 0x7f7d801036e0 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.940+0000 7f7d7cff9640 1 -- 172.21.15.5:0/2532300356 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7d68004f40 con 0x7f7d801036e0 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.940+0000 7f7d88503740 1 Processor -- start 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d88503740 1 -- start start 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d88503740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 0x556f3bbb4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d88503740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb5480 0x556f3bbbf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d88503740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbc2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d88503740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556f3bb1a280 con 0x556f3bbbfd40 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d88503740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556f3bac1b10 con 0x556f3bbb4b60 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d88503740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556f3bb16240 con 0x556f3bbb5480 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbc2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbc2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35210/0 (socket says 172.21.15.5:35210) 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7effd640 1 -- 172.21.15.5:0/333113706 learned_addr learned my addr 172.21.15.5:0/333113706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/333113706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb5480 0x556f3bbbf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/333113706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 0x556f3bbb4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7effd640 1 -- 172.21.15.5:0/333113706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb5480 msgr2=0x556f3bbbf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7effd640 1 --2- 172.21.15.5:0/333113706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb5480 0x556f3bbbf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7effd640 1 -- 172.21.15.5:0/333113706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 msgr2=0x556f3bbb4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7effd640 1 --2- 172.21.15.5:0/333113706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 0x556f3bbb4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7effd640 1 -- 172.21.15.5:0/333113706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3bace1f0 con 0x556f3bbbfd40 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/333113706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 0x556f3bbb4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.941+0000 7f7d7cff9640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7d6c0a2cf0 0x7f7d6c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.942+0000 7f7d7effd640 1 --2- 172.21.15.5:0/333113706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbc2130 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7d680cc6d0 tx=0x7f7d680d7780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2053c618382ded server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.942+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7d6c0a2cf0 0x7f7d6c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.942+0000 7f7d7cff9640 1 -- 172.21.15.5:0/2532300356 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f7d680cf080 con 0x7f7d801036e0 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.942+0000 7f7d75ffb640 1 -- 172.21.15.5:0/333113706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d680bb070 con 0x556f3bbbfd40 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.942+0000 7f7d75ffb640 1 -- 172.21.15.5:0/333113706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6809b5e0 con 0x556f3bbbfd40 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.942+0000 7f7d75ffb640 1 -- 172.21.15.5:0/333113706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68097880 con 0x556f3bbbfd40 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 -- 172.21.15.5:0/333113706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 msgr2=0x556f3bbc2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7d6c0a2cf0 0x7f7d6c0a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d78031550 tx=0x7f7d78005d50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 --2- 172.21.15.5:0/333113706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbc2130 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f7d680cc6d0 tx=0x7f7d680d7780 comp rx=0 tx=0).stop 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 -- 172.21.15.5:0/333113706 shutdown_connections 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 --2- 172.21.15.5:0/333113706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbc2130 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 --2- 172.21.15.5:0/333113706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb5480 0x556f3bbbf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 --2- 172.21.15.5:0/333113706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 0x556f3bbb4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 -- 172.21.15.5:0/333113706 >> 172.21.15.5:0/333113706 conn(0x556f3bbb10f0 msgr2=0x556f3bbb14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 -- 172.21.15.5:0/333113706 shutdown_connections 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 -- 172.21.15.5:0/333113706 wait complete. 2024-09-11T22:30:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 -- start start 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.943+0000 7f7d88503740 1 Processor -- start 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d88503740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 0x556f3bbbcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d88503740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb5480 0x556f3bbbd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d88503740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbbd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d88503740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556f3bbc2790 con 0x556f3bbbfd40 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d88503740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556f3bad4f10 con 0x556f3bbb4b60 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d88503740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556f3bbc2910 con 0x556f3bbb5480 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d88503740 10 client.0 _ll_register_callbacks cb 0x556f3ba9da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbbd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbbd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35224/0 (socket says 172.21.15.5:35224) 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7ffff640 1 -- 172.21.15.5:0/701888686 learned_addr learned my addr 172.21.15.5:0/701888686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7effd640 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb5480 0x556f3bbbd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 0x556f3bbbcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7ffff640 1 -- 172.21.15.5:0/701888686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb5480 msgr2=0x556f3bbbd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb5480 0x556f3bbbd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7ffff640 1 -- 172.21.15.5:0/701888686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 msgr2=0x556f3bbbcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 0x556f3bbbcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7ffff640 1 -- 172.21.15.5:0/701888686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3b88b6a0 con 0x556f3bbbfd40 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbbd6d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7d780042c0 tx=0x7f7d78009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c309dc566cb0057 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d5effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d5effd640 1 -- 172.21.15.5:0/701888686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7804d070 con 0x556f3bbbfd40 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d88503740 1 -- 172.21.15.5:0/701888686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d6809b5e0 con 0x556f3bbbfd40 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.944+0000 7f7d5effd640 1 -- 172.21.15.5:0/701888686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f7d78003d70 con 0x556f3bbbfd40 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d5effd640 1 -- 172.21.15.5:0/701888686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7d78059050 con 0x556f3bbbfd40 2024-09-11T22:30:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d88503740 10 client.18030 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d88503740 1 -- 172.21.15.5:0/701888686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556f3bbce3f0 con 0x556f3bbbfd40 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d5effd640 1 -- 172.21.15.5:0/701888686 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6809b5e0 con 0x556f3bbbfd40 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d5effd640 1 -- 172.21.15.5:0/701888686 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f7d7803c560 con 0x556f3bbbfd40 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d88503740 10 client.18030 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d88503740 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d88503740 20 client.18030 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d88503740 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556f3bbcf800 0x556f3bbd1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d88503740 4 client.18030 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d88503740 1 -- 172.21.15.5:0/701888686 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f3bb277f0 con 0x556f3bbcf800 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.945+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556f3bbcf800 0x556f3bbd1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.946+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556f3bbcf800 0x556f3bbd1c80 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2cc6d14aaebb99b server_cookie=e44adf35cde0a245 in_seq=0 out_seq=0 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.946+0000 7f7d5effd640 10 client.18030 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.946+0000 7f7d5effd640 1 -- 172.21.15.5:0/701888686 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f3bb277f0 con 0x556f3bbcf800 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.946+0000 7f7d5effd640 10 client.18030 handle_command_reply: tid=0 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.946+0000 7f7d88503740 1 client.18030 shutdown 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 1 -- 172.21.15.5:0/701888686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 msgr2=0x556f3bbbd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbbd6d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7d780042c0 tx=0x7f7d78009260 comp rx=0 tx=0).stop 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 1 -- 172.21.15.5:0/701888686 shutdown_connections 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556f3bbcf800 0x556f3bbd1c80 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbbfd40 0x556f3bbbd6d0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb5480 0x556f3bbbd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 1 --2- 172.21.15.5:0/701888686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbb4b60 0x556f3bbbcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 1 -- 172.21.15.5:0/701888686 >> 172.21.15.5:0/701888686 conn(0x556f3bbb10f0 msgr2=0x556f3bbb6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d5effd640 0 client.18030 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 1 -- 172.21.15.5:0/701888686 shutdown_connections 2024-09-11T22:30:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 1 -- 172.21.15.5:0/701888686 wait complete. 2024-09-11T22:30:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.947+0000 7f7d88503740 20 client.18030 trim_cache size 0 max 16384 2024-09-11T22:30:32.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d88503740 1 Processor -- start 2024-09-11T22:30:32.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d88503740 1 -- start start 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d88503740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 0x556f3bbbeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d88503740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbd75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d88503740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 0x556f3bbd7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d88503740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801afcc0 con 0x556f3bbb5480 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d88503740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801b1800 con 0x556f3bbbfd40 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d88503740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801b0390 con 0x556f3bbb4b60 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbd75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbd75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35238/0 (socket says 172.21.15.5:35238) 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7effd640 1 -- 172.21.15.5:0/373716464 learned_addr learned my addr 172.21.15.5:0/373716464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 0x556f3bbbeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 0x556f3bbd7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7effd640 1 -- 172.21.15.5:0/373716464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 msgr2=0x556f3bbbeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7effd640 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 0x556f3bbbeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7effd640 1 -- 172.21.15.5:0/373716464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 msgr2=0x556f3bbd7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7effd640 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 0x556f3bbd7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7effd640 1 -- 172.21.15.5:0/373716464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f3bbbbc00 con 0x556f3bbb5480 2024-09-11T22:30:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 0x556f3bbbeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.965+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 0x556f3bbd7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d7effd640 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbd75a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7d68007590 tx=0x7f7d6800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65622fe78ce6ffa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d757fa640 1 -- 172.21.15.5:0/373716464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6801d040 con 0x556f3bbb5480 2024-09-11T22:30:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d757fa640 1 -- 172.21.15.5:0/373716464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d680d29a0 con 0x556f3bbb5480 2024-09-11T22:30:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d757fa640 1 -- 172.21.15.5:0/373716464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d680bb040 con 0x556f3bbb5480 2024-09-11T22:30:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d88503740 1 -- 172.21.15.5:0/373716464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 msgr2=0x556f3bbd75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d88503740 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbd75a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f7d68007590 tx=0x7f7d6800cf60 comp rx=0 tx=0).stop 2024-09-11T22:30:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d88503740 1 -- 172.21.15.5:0/373716464 shutdown_connections 2024-09-11T22:30:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d88503740 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 0x556f3bbd7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d88503740 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbd75a0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d88503740 1 --2- 172.21.15.5:0/373716464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 0x556f3bbbeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d88503740 1 -- 172.21.15.5:0/373716464 >> 172.21.15.5:0/373716464 conn(0x556f3bbb10f0 msgr2=0x556f3bbbe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d88503740 1 -- 172.21.15.5:0/373716464 shutdown_connections 2024-09-11T22:30:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.966+0000 7f7d88503740 1 -- 172.21.15.5:0/373716464 wait complete. 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d88503740 1 -- start start 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d88503740 1 Processor -- start 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d88503740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 0x556f3bbc3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d88503740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbc3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d88503740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 0x556f3bbc44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d88503740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801b1130 con 0x556f3bbb5480 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d88503740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801b0a60 con 0x556f3bbbfd40 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d88503740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7d801adcc0 con 0x556f3bbb4b60 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d88503740 10 client.0 _ll_register_callbacks cb 0x7f7d78009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbc3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d7effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbc3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35254/0 (socket says 172.21.15.5:35254) 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.967+0000 7f7d7effd640 1 -- 172.21.15.5:0/2984702995 learned_addr learned my addr 172.21.15.5:0/2984702995 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:32.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 0x556f3bbc3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 0x556f3bbc44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d7effd640 1 -- 172.21.15.5:0/2984702995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 msgr2=0x556f3bbc3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d7effd640 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 0x556f3bbc3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d7effd640 1 -- 172.21.15.5:0/2984702995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 msgr2=0x556f3bbc44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d7effd640 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 0x556f3bbc44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d7effd640 1 -- 172.21.15.5:0/2984702995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f3bc39710 con 0x556f3bbb5480 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d7ffff640 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 0x556f3bbc44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d7effd640 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbc3fb0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c940 tx=0x7f7d680d7780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c84025eb35f594d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d5effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d5effd640 1 -- 172.21.15.5:0/2984702995 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6801d040 con 0x556f3bbb5480 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d5effd640 1 -- 172.21.15.5:0/2984702995 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f7d6809ba40 con 0x556f3bbb5480 2024-09-11T22:30:32.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d88503740 1 -- 172.21.15.5:0/2984702995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d680d29a0 con 0x556f3bbb5480 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d5effd640 1 -- 172.21.15.5:0/2984702995 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7d680bb040 con 0x556f3bbb5480 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d88503740 10 client.18036 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.968+0000 7f7d88503740 1 -- 172.21.15.5:0/2984702995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556f3bc3a980 con 0x556f3bbb5480 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.969+0000 7f7d5effd640 1 -- 172.21.15.5:0/2984702995 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d680d29a0 con 0x556f3bbb5480 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.969+0000 7f7d5effd640 1 -- 172.21.15.5:0/2984702995 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f7d680b4350 con 0x556f3bbb5480 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.969+0000 7f7d88503740 10 client.18036 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.969+0000 7f7d88503740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.969+0000 7f7d88503740 20 client.18036 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.969+0000 7f7d88503740 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556f3bc3ac20 0x556f3bc3d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.969+0000 7f7d88503740 4 client.18036 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.969+0000 7f7d88503740 1 -- 172.21.15.5:0/2984702995 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556f3bb21d00 con 0x556f3bc3ac20 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.969+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556f3bc3ac20 0x556f3bc3d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.970+0000 7f7d7f7fe640 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556f3bc3ac20 0x556f3bc3d010 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24d487d866f37583 server_cookie=311f9bc1dfd801c4 in_seq=0 out_seq=0 2024-09-11T22:30:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.970+0000 7f7d5effd640 10 client.18036 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d5effd640 1 -- 172.21.15.5:0/2984702995 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556f3bb21d00 con 0x556f3bc3ac20 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d5effd640 10 client.18036 handle_command_reply: tid=0 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 client.18036 shutdown 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 -- 172.21.15.5:0/2984702995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 msgr2=0x556f3bbc3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbc3fb0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c940 tx=0x7f7d680d7780 comp rx=0 tx=0).stop 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 -- 172.21.15.5:0/2984702995 shutdown_connections 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556f3bc3ac20 0x556f3bc3d010 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556f3bbbfd40 0x556f3bbc44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d5effd640 0 client.18036 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f3bbb5480 0x556f3bbc3fb0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 --2- 172.21.15.5:0/2984702995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556f3bbb4b60 0x556f3bbc3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 -- 172.21.15.5:0/2984702995 >> 172.21.15.5:0/2984702995 conn(0x556f3bbb10f0 msgr2=0x556f3bbbd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 -- 172.21.15.5:0/2984702995 shutdown_connections 2024-09-11T22:30:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 1 -- 172.21.15.5:0/2984702995 wait complete. 2024-09-11T22:30:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.971+0000 7f7d88503740 20 client.18036 trim_cache size 0 max 16384 2024-09-11T22:30:32.977 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:30:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:30:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "5657e91e-8c9a-459b-82e0-05e073f401b7", 2024-09-11T22:30:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:30:32.977 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:30:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 -- 172.21.15.5:0/2532300356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7d6c0a2cf0 msgr2=0x7f7d6c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7d6c0a2cf0 0x7f7d6c0a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7d78031550 tx=0x7f7d78005d50 comp rx=0 tx=0).stop 2024-09-11T22:30:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 -- 172.21.15.5:0/2532300356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d801036e0 msgr2=0x7f7d80100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d801036e0 0x7f7d80100ed0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c970 tx=0x7f7d6800ce30 comp rx=0 tx=0).stop 2024-09-11T22:30:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 -- 172.21.15.5:0/2532300356 shutdown_connections 2024-09-11T22:30:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7d6c0a2cf0 0x7f7d6c0a51b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d801082a0 0x7f7d80101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d801036e0 0x7f7d80100ed0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 --2- 172.21.15.5:0/2532300356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d80102d30 0x7f7d80100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 -- 172.21.15.5:0/2532300356 >> 172.21.15.5:0/2532300356 conn(0x7f7d800fc7d0 msgr2=0x7f7d80105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 -- 172.21.15.5:0/2532300356 shutdown_connections 2024-09-11T22:30:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:32.972+0000 7f7d8612a640 1 -- 172.21.15.5:0/2532300356 wait complete. 2024-09-11T22:30:33.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 7.87 MB/sec execute 291 sec latency 12972.725 ms 2024-09-11T22:30:33.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:33 smithi005 ceph-mon[30943]: pgmap v261: 97 pgs: 97 active+clean; 94 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 180 op/s 2024-09-11T22:30:33.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:30:34.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:33 smithi159 ceph-mon[35534]: pgmap v261: 97 pgs: 97 active+clean; 94 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 180 op/s 2024-09-11T22:30:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:33 smithi172 ceph-mon[35461]: pgmap v261: 97 pgs: 97 active+clean; 94 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 180 op/s 2024-09-11T22:30:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:34.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 7.84 MB/sec execute 292 sec latency 13972.831 ms 2024-09-11T22:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.687+0000 7f4ed8795640 1 -- 172.21.15.5:0/3397863257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed0101400 msgr2=0x7f4ed010d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.687+0000 7f4ed8795640 1 --2- 172.21.15.5:0/3397863257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed0101400 0x7f4ed010d930 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009cd0 tx=0x7f4ec802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.687+0000 7f4ed8795640 1 -- 172.21.15.5:0/3397863257 shutdown_connections 2024-09-11T22:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.687+0000 7f4ed8795640 1 --2- 172.21.15.5:0/3397863257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed0101400 0x7f4ed010d930 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.687+0000 7f4ed8795640 1 --2- 172.21.15.5:0/3397863257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ed0100a60 0x7f4ed0100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.687+0000 7f4ed8795640 1 --2- 172.21.15.5:0/3397863257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ed0106a60 0x7f4ed0106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.687+0000 7f4ed8795640 1 -- 172.21.15.5:0/3397863257 >> 172.21.15.5:0/3397863257 conn(0x7f4ed00fc780 msgr2=0x7f4ed00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.687+0000 7f4ed8795640 1 -- 172.21.15.5:0/3397863257 shutdown_connections 2024-09-11T22:30:34.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.687+0000 7f4ed8795640 1 -- 172.21.15.5:0/3397863257 wait complete. 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed8795640 1 Processor -- start 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed8795640 1 -- start start 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed8795640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ed0100a60 0x7f4ed00fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed8795640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ed0101400 0x7f4ed00fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed8795640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed0106a60 0x7f4ed00fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed8795640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed0110e80 con 0x7f4ed0106a60 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed8795640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed0110d00 con 0x7f4ed0101400 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed8795640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed0111000 con 0x7f4ed0100a60 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed6d32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed0106a60 0x7f4ed00fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed6d32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed0106a60 0x7f4ed00fd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35266/0 (socket says 172.21.15.5:35266) 2024-09-11T22:30:34.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed6d32640 1 -- 172.21.15.5:0/1602173477 learned_addr learned my addr 172.21.15.5:0/1602173477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ed0101400 0x7f4ed00fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed6531640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ed0100a60 0x7f4ed00fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed6d32640 1 -- 172.21.15.5:0/1602173477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ed0100a60 msgr2=0x7f4ed00fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed6d32640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ed0100a60 0x7f4ed00fce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed6d32640 1 -- 172.21.15.5:0/1602173477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ed0101400 msgr2=0x7f4ed00fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed6d32640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ed0101400 0x7f4ed00fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed6d32640 1 -- 172.21.15.5:0/1602173477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed01a6910 con 0x7f4ed0106a60 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.688+0000 7f4ed6531640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ed0100a60 0x7f4ed00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.689+0000 7f4ed6d32640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed0106a60 0x7f4ed00fd890 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8006010 tx=0x7f4ec8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.689+0000 7f4ec77fe640 1 -- 172.21.15.5:0/1602173477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8004450 con 0x7f4ed0106a60 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.689+0000 7f4ec77fe640 1 -- 172.21.15.5:0/1602173477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec8003e10 con 0x7f4ed0106a60 2024-09-11T22:30:34.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.689+0000 7f4ed8795640 1 -- 172.21.15.5:0/1602173477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed01a6ba0 con 0x7f4ed0106a60 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.689+0000 7f4ec77fe640 1 -- 172.21.15.5:0/1602173477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8046480 con 0x7f4ed0106a60 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.689+0000 7f4ed8795640 1 -- 172.21.15.5:0/1602173477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed01a7130 con 0x7f4ed0106a60 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.691+0000 7f4ec77fe640 1 -- 172.21.15.5:0/1602173477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4ec80045f0 con 0x7f4ed0106a60 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.691+0000 7f4edab6e740 1 Processor -- start 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4edab6e740 1 -- start start 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 0x55eb383f8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f8dc0 0x55eb38403140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb38403680 0x55eb38405a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4edab6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eb3835e2e0 con 0x55eb383f8dc0 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4edab6e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eb38305ce0 con 0x55eb383f84a0 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4edab6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55eb3835a2a0 con 0x55eb38403680 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4ed6531640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f8dc0 0x55eb38403140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4ed5d30640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 0x55eb383f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4ed6531640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f8dc0 0x55eb38403140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35270/0 (socket says 172.21.15.5:35270) 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4ed6531640 1 -- 172.21.15.5:0/1550635959 learned_addr learned my addr 172.21.15.5:0/1550635959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.692+0000 7f4ec77fe640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ebc0a2d70 0x7f4ebc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/1550635959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb38403680 0x55eb38405a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ed6531640 1 -- 172.21.15.5:0/1550635959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb38403680 msgr2=0x55eb38405a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ed6531640 1 --2- 172.21.15.5:0/1550635959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb38403680 0x55eb38405a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ed6531640 1 -- 172.21.15.5:0/1550635959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 msgr2=0x55eb383f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ed6531640 1 --2- 172.21.15.5:0/1550635959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 0x55eb383f8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ed6531640 1 -- 172.21.15.5:0/1550635959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb382ea440 con 0x55eb383f8dc0 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ed6d32640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ebc0a2d70 0x7f4ebc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/1550635959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 0x55eb383f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ec77fe640 1 -- 172.21.15.5:0/1602173477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f4ec802e030 con 0x7f4ed0106a60 2024-09-11T22:30:34.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ed6531640 1 --2- 172.21.15.5:0/1550635959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f8dc0 0x55eb38403140 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00bb70 tx=0x7f4ecc00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34cceff6223fd96 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ec4ff9640 1 -- 172.21.15.5:0/1550635959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc017620 con 0x55eb383f8dc0 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ec4ff9640 1 -- 172.21.15.5:0/1550635959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ecc017ca0 con 0x55eb383f8dc0 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.693+0000 7f4ec4ff9640 1 -- 172.21.15.5:0/1550635959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00fd10 con 0x55eb383f8dc0 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 -- 172.21.15.5:0/1550635959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f8dc0 msgr2=0x55eb38403140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 --2- 172.21.15.5:0/1550635959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f8dc0 0x55eb38403140 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00bb70 tx=0x7f4ecc00d710 comp rx=0 tx=0).stop 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4ed6d32640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ebc0a2d70 0x7f4ebc0a5230 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80bd230 tx=0x7f4ec802a8e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 -- 172.21.15.5:0/1550635959 shutdown_connections 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 --2- 172.21.15.5:0/1550635959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb38403680 0x55eb38405a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 --2- 172.21.15.5:0/1550635959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f8dc0 0x55eb38403140 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 --2- 172.21.15.5:0/1550635959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 0x55eb383f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 -- 172.21.15.5:0/1550635959 >> 172.21.15.5:0/1550635959 conn(0x55eb383f4a30 msgr2=0x55eb383f4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:34.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 -- 172.21.15.5:0/1550635959 shutdown_connections 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 -- 172.21.15.5:0/1550635959 wait complete. 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 -- start start 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.694+0000 7f4edab6e740 1 Processor -- start 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f84a0 0x55eb38402fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f8dc0 0x55eb38400250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb38403680 0x55eb38400790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4edab6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eb38406250 con 0x55eb38403680 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4edab6e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eb38319050 con 0x55eb383f8dc0 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4edab6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55eb384063d0 con 0x55eb383f84a0 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4edab6e740 10 client.0 _ll_register_callbacks cb 0x55eb38359160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed5d30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb38403680 0x55eb38400790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed5d30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb38403680 0x55eb38400790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35276/0 (socket says 172.21.15.5:35276) 2024-09-11T22:30:34.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed5d30640 1 -- 172.21.15.5:0/3542614388 learned_addr learned my addr 172.21.15.5:0/3542614388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f84a0 0x55eb38402fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed6531640 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f8dc0 0x55eb38400250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed5d30640 1 -- 172.21.15.5:0/3542614388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f84a0 msgr2=0x55eb38402fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f84a0 0x55eb38402fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed5d30640 1 -- 172.21.15.5:0/3542614388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f8dc0 msgr2=0x55eb38400250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f8dc0 0x55eb38400250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed5d30640 1 -- 172.21.15.5:0/3542614388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb380cf3c0 con 0x55eb38403680 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb38403680 0x55eb38400790 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4ec001a7e0 tx=0x7f4ec0009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db246e707d0eef4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.695+0000 7f4eb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/3542614388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0013a60 con 0x55eb38403680 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/3542614388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f4ec0013c00 con 0x55eb38403680 2024-09-11T22:30:34.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4edab6e740 1 -- 172.21.15.5:0/3542614388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ecc017ca0 con 0x55eb38403680 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/3542614388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4ec0029050 con 0x55eb38403680 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4edab6e740 10 client.18048 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4edab6e740 1 -- 172.21.15.5:0/3542614388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55eb38411cb0 con 0x55eb38403680 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/3542614388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ecc017ca0 con 0x55eb38403680 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/3542614388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f4ec0018c30 con 0x55eb38403680 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4edab6e740 10 client.18048 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4edab6e740 10 client.18048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4edab6e740 20 client.18048 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4edab6e740 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55eb38413170 0x55eb38415580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4edab6e740 4 client.18048 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4edab6e740 1 -- 172.21.15.5:0/3542614388 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb3836b850 con 0x55eb38413170 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.696+0000 7f4ed6531640 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55eb38413170 0x55eb38415580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.697+0000 7f4ed6531640 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55eb38413170 0x55eb38415580 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=312d13a04816d92 server_cookie=7df53ea067f20557 in_seq=0 out_seq=0 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.697+0000 7f4eb9ffb640 10 client.18048 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/3542614388 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb3836b850 con 0x55eb38413170 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4eb9ffb640 10 client.18048 handle_command_reply: tid=0 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 client.18048 shutdown 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 -- 172.21.15.5:0/3542614388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb38403680 msgr2=0x55eb38400790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb38403680 0x55eb38400790 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f4ec001a7e0 tx=0x7f4ec0009110 comp rx=0 tx=0).stop 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 -- 172.21.15.5:0/3542614388 shutdown_connections 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55eb38413170 0x55eb38415580 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb38403680 0x55eb38400790 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f8dc0 0x55eb38400250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4eb9ffb640 0 client.18048 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 --2- 172.21.15.5:0/3542614388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f84a0 0x55eb38402fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 -- 172.21.15.5:0/3542614388 >> 172.21.15.5:0/3542614388 conn(0x55eb383f4a30 msgr2=0x55eb383f3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 -- 172.21.15.5:0/3542614388 shutdown_connections 2024-09-11T22:30:34.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 1 -- 172.21.15.5:0/3542614388 wait complete. 2024-09-11T22:30:34.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.698+0000 7f4edab6e740 20 client.18048 trim_cache size 0 max 16384 2024-09-11T22:30:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.715+0000 7f4edab6e740 1 Processor -- start 2024-09-11T22:30:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.715+0000 7f4edab6e740 1 -- start start 2024-09-11T22:30:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.715+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f4a30 0x55eb3840f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.715+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 0x55eb383fa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.715+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f8dc0 0x55eb3841ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.715+0000 7f4edab6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed01afbb0 con 0x55eb383f4a30 2024-09-11T22:30:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.715+0000 7f4edab6e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed01b16f0 con 0x55eb383f84a0 2024-09-11T22:30:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.715+0000 7f4edab6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed01b0280 con 0x55eb383f8dc0 2024-09-11T22:30:34.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ed5d30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f4a30 0x55eb3840f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ed5d30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f4a30 0x55eb3840f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35292/0 (socket says 172.21.15.5:35292) 2024-09-11T22:30:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ed5d30640 1 -- 172.21.15.5:0/1649832667 learned_addr learned my addr 172.21.15.5:0/1649832667 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ed5d30640 1 -- 172.21.15.5:0/1649832667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f8dc0 msgr2=0x55eb3841ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/1649832667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f8dc0 0x55eb3841ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ed6531640 1 --2- 172.21.15.5:0/1649832667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 0x55eb383fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ed5d30640 1 -- 172.21.15.5:0/1649832667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 msgr2=0x55eb383fa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/1649832667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 0x55eb383fa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ed5d30640 1 -- 172.21.15.5:0/1649832667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb38415c00 con 0x55eb383f4a30 2024-09-11T22:30:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/1649832667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f4a30 0x55eb3840f4e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0017740 tx=0x7f4ec0006e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b430f36b22df5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ebbfff640 1 -- 172.21.15.5:0/1649832667 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0031020 con 0x55eb383f4a30 2024-09-11T22:30:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ebbfff640 1 -- 172.21.15.5:0/1649832667 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec0020070 con 0x55eb383f4a30 2024-09-11T22:30:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.716+0000 7f4ebbfff640 1 -- 172.21.15.5:0/1649832667 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0013a30 con 0x55eb383f4a30 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 -- 172.21.15.5:0/1649832667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f4a30 msgr2=0x55eb3840f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 --2- 172.21.15.5:0/1649832667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f4a30 0x55eb3840f4e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0017740 tx=0x7f4ec0006e40 comp rx=0 tx=0).stop 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 -- 172.21.15.5:0/1649832667 shutdown_connections 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 --2- 172.21.15.5:0/1649832667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f8dc0 0x55eb3841ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 --2- 172.21.15.5:0/1649832667 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f84a0 0x55eb383fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 --2- 172.21.15.5:0/1649832667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f4a30 0x55eb3840f4e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 -- 172.21.15.5:0/1649832667 >> 172.21.15.5:0/1649832667 conn(0x55eb384008f0 msgr2=0x55eb383f9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 -- 172.21.15.5:0/1649832667 shutdown_connections 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 -- 172.21.15.5:0/1649832667 wait complete. 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 -- start start 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.717+0000 7f4edab6e740 1 Processor -- start 2024-09-11T22:30:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f4a30 0x55eb38407480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f84a0 0x55eb384079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4edab6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f8dc0 0x55eb38407f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4edab6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed01b1020 con 0x55eb383f84a0 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4edab6e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed01b0950 con 0x55eb383f4a30 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4edab6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed01adbb0 con 0x55eb383f8dc0 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4edab6e740 10 client.0 _ll_register_callbacks cb 0x7f4ec001a410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed6531640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f84a0 0x55eb384079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed6531640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f84a0 0x55eb384079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35306/0 (socket says 172.21.15.5:35306) 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed6531640 1 -- 172.21.15.5:0/2873228872 learned_addr learned my addr 172.21.15.5:0/2873228872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f8dc0 0x55eb38407f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f4a30 0x55eb38407480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed6531640 1 -- 172.21.15.5:0/2873228872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f8dc0 msgr2=0x55eb38407f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed6531640 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f8dc0 0x55eb38407f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed6531640 1 -- 172.21.15.5:0/2873228872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f4a30 msgr2=0x55eb38407480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed6531640 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f4a30 0x55eb38407480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed6531640 1 -- 172.21.15.5:0/2873228872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb3847cea0 con 0x55eb383f84a0 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f4a30 0x55eb38407480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4ed6531640 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f84a0 0x55eb384079c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x55eb383f5890 tx=0x7f4ecc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c74752051a4e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4eb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.718+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/2873228872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc00f930 con 0x55eb383f84a0 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/2873228872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f4ecc00fad0 con 0x55eb383f84a0 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4edab6e740 1 -- 172.21.15.5:0/2873228872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ec0020070 con 0x55eb383f84a0 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/2873228872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4ecc038050 con 0x55eb383f84a0 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4edab6e740 10 client.18054 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:34.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4edab6e740 1 -- 172.21.15.5:0/2873228872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55eb3847e190 con 0x55eb383f84a0 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/2873228872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec0020070 con 0x55eb383f84a0 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/2873228872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f4ecc0279c0 con 0x55eb383f84a0 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4edab6e740 10 client.18054 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4edab6e740 10 client.18054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4edab6e740 20 client.18054 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4edab6e740 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55eb38403680 0x55eb384806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4edab6e740 4 client.18054 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4edab6e740 1 -- 172.21.15.5:0/2873228872 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb38365d60 con 0x55eb38403680 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.719+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55eb38403680 0x55eb384806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4ed5d30640 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55eb38403680 0x55eb384806c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab9b28c200c93b7 server_cookie=10a949dbf1bb4056 in_seq=0 out_seq=0 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4eb9ffb640 10 client.18054 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4eb9ffb640 1 -- 172.21.15.5:0/2873228872 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eb38365d60 con 0x55eb38403680 2024-09-11T22:30:34.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4eb9ffb640 10 client.18054 handle_command_reply: tid=0 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4edab6e740 1 client.18054 shutdown 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4edab6e740 1 -- 172.21.15.5:0/2873228872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f84a0 msgr2=0x55eb384079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4edab6e740 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f84a0 0x55eb384079c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x55eb383f5890 tx=0x7f4ecc005c50 comp rx=0 tx=0).stop 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4edab6e740 1 -- 172.21.15.5:0/2873228872 shutdown_connections 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4edab6e740 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55eb38403680 0x55eb384806c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4edab6e740 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb383f8dc0 0x55eb38407f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4edab6e740 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eb383f84a0 0x55eb384079c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4eb9ffb640 0 client.18054 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4edab6e740 1 --2- 172.21.15.5:0/2873228872 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eb383f4a30 0x55eb38407480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.720+0000 7f4edab6e740 1 -- 172.21.15.5:0/2873228872 >> 172.21.15.5:0/2873228872 conn(0x55eb384008f0 msgr2=0x55eb38401ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4edab6e740 1 -- 172.21.15.5:0/2873228872 shutdown_connections 2024-09-11T22:30:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4edab6e740 1 -- 172.21.15.5:0/2873228872 wait complete. 2024-09-11T22:30:34.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4edab6e740 20 client.18054 trim_cache size 0 max 16384 2024-09-11T22:30:34.728 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:30:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:30:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:30:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "5657e91e-8c9a-459b-82e0-05e073f401b7": { 2024-09-11T22:30:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:30:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5657e91e-8c9a-459b-82e0-05e073f401b7", 2024-09-11T22:30:34.728 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:30:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:34.729 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:34.729 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:30:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 -- 172.21.15.5:0/1602173477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ebc0a2d70 msgr2=0x7f4ebc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ebc0a2d70 0x7f4ebc0a5230 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80bd230 tx=0x7f4ec802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 -- 172.21.15.5:0/1602173477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed0106a60 msgr2=0x7f4ed00fd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:34.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed0106a60 0x7f4ed00fd890 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8006010 tx=0x7f4ec8003c20 comp rx=0 tx=0).stop 2024-09-11T22:30:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 -- 172.21.15.5:0/1602173477 shutdown_connections 2024-09-11T22:30:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed0106a60 0x7f4ed00fd890 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4ebc0a2d70 0x7f4ebc0a5230 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4ed0101400 0x7f4ed00fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 --2- 172.21.15.5:0/1602173477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ed0100a60 0x7f4ed00fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 -- 172.21.15.5:0/1602173477 >> 172.21.15.5:0/1602173477 conn(0x7f4ed00fc780 msgr2=0x7f4ed006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 -- 172.21.15.5:0/1602173477 shutdown_connections 2024-09-11T22:30:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:34.721+0000 7f4ed8795640 1 -- 172.21.15.5:0/1602173477 wait complete. 2024-09-11T22:30:35.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:34 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:30:35.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:34 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:30:35.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:34 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:30:35.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:34 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:30:35.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:34 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:30:35.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:34 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:30:35.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:34 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:30:35.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:34 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:30:35.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:34 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:30:35.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:34 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:30:35.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:34 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:30:35.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:34 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:30:35.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:34 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:30:35.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:34 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:30:35.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:34 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:30:35.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:34 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:30:35.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:34 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:30:35.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:34 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:30:35.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 7.81 MB/sec execute 293 sec latency 14972.890 ms 2024-09-11T22:30:35.805 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:30:35.864 INFO:tasks.cephfs.filesystem:scrub status for tag:5657e91e-8c9a-459b-82e0-05e073f401b7 - {'path': '/', 'tag': '5657e91e-8c9a-459b-82e0-05e073f401b7', 'options': 'recursive,force'} 2024-09-11T22:30:36.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:35 smithi005 ceph-mon[30943]: pgmap v262: 97 pgs: 97 active+clean; 94 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 827 KiB/s wr, 101 op/s 2024-09-11T22:30:36.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:35 smithi159 ceph-mon[35534]: pgmap v262: 97 pgs: 97 active+clean; 94 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 827 KiB/s wr, 101 op/s 2024-09-11T22:30:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:36.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:35 smithi172 ceph-mon[35461]: pgmap v262: 97 pgs: 97 active+clean; 94 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 827 KiB/s wr, 101 op/s 2024-09-11T22:30:36.331 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 7.79 MB/sec execute 294 sec latency 15972.921 ms 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.580+0000 7fef836f0640 1 -- 172.21.15.5:0/2011095136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c101df0 msgr2=0x7fef7c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.580+0000 7fef836f0640 1 --2- 172.21.15.5:0/2011095136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c101df0 0x7fef7c1021d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fef78009900 tx=0x7fef7802a410 comp rx=0 tx=0).stop 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.580+0000 7fef836f0640 1 -- 172.21.15.5:0/2011095136 shutdown_connections 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.580+0000 7fef836f0640 1 --2- 172.21.15.5:0/2011095136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c106f20 0x7fef7c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.580+0000 7fef836f0640 1 --2- 172.21.15.5:0/2011095136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef7c1027a0 0x7fef7c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.580+0000 7fef836f0640 1 --2- 172.21.15.5:0/2011095136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c101df0 0x7fef7c1021d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.580+0000 7fef836f0640 1 -- 172.21.15.5:0/2011095136 >> 172.21.15.5:0/2011095136 conn(0x7fef7c0fd8b0 msgr2=0x7fef7c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.580+0000 7fef836f0640 1 -- 172.21.15.5:0/2011095136 shutdown_connections 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.580+0000 7fef836f0640 1 -- 172.21.15.5:0/2011095136 wait complete. 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef836f0640 1 Processor -- start 2024-09-11T22:30:36.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef836f0640 1 -- start start 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef836f0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef7c101df0 0x7fef7c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef836f0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c1027a0 0x7fef7c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef836f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c106f20 0x7fef7c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef836f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c111f40 con 0x7fef7c106f20 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef836f0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c111dc0 con 0x7fef7c1027a0 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef836f0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1120c0 con 0x7fef7c101df0 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef81c8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c106f20 0x7fef7c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef81c8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c106f20 0x7fef7c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35330/0 (socket says 172.21.15.5:35330) 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef81c8d640 1 -- 172.21.15.5:0/2008602029 learned_addr learned my addr 172.21.15.5:0/2008602029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.581+0000 7fef80c8b640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c1027a0 0x7fef7c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef8148c640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef7c101df0 0x7fef7c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef81c8d640 1 -- 172.21.15.5:0/2008602029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef7c101df0 msgr2=0x7fef7c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:36.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef81c8d640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef7c101df0 0x7fef7c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef81c8d640 1 -- 172.21.15.5:0/2008602029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c1027a0 msgr2=0x7fef7c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef81c8d640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c1027a0 0x7fef7c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef81c8d640 1 -- 172.21.15.5:0/2008602029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c19d310 con 0x7fef7c106f20 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef8148c640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef7c101df0 0x7fef7c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef81c8d640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c106f20 0x7fef7c19cc30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fef7400eaa0 tx=0x7fef7400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef727fc640 1 -- 172.21.15.5:0/2008602029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74004d20 con 0x7fef7c106f20 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef836f0640 1 -- 172.21.15.5:0/2008602029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c19d600 con 0x7fef7c106f20 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef727fc640 1 -- 172.21.15.5:0/2008602029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef74005020 con 0x7fef7c106f20 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef727fc640 1 -- 172.21.15.5:0/2008602029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef740116d0 con 0x7fef7c106f20 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.582+0000 7fef836f0640 1 -- 172.21.15.5:0/2008602029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c1a4ee0 con 0x7fef7c106f20 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.583+0000 7fef836f0640 1 -- 172.21.15.5:0/2008602029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7c1021d0 con 0x7fef7c106f20 2024-09-11T22:30:36.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.584+0000 7fef727fc640 1 -- 172.21.15.5:0/2008602029 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fef740051d0 con 0x7fef7c106f20 2024-09-11T22:30:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.585+0000 7fef727fc640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fef680a2cf0 0x7fef680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.585+0000 7fef8148c640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fef680a2cf0 0x7fef680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.585+0000 7fef727fc640 1 -- 172.21.15.5:0/2008602029 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fef740cfa90 con 0x7fef7c106f20 2024-09-11T22:30:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.586+0000 7fef8148c640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fef680a2cf0 0x7fef680a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef78005ab0 tx=0x7fef78005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.589+0000 7fef727fc640 1 -- 172.21.15.5:0/2008602029 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef74016030 con 0x7fef7c106f20 2024-09-11T22:30:36.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.801+0000 7fef836f0640 1 -- 172.21.15.5:0/2008602029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef7c02d7c0 con 0x7fef7c106f20 2024-09-11T22:30:36.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.802+0000 7fef727fc640 1 -- 172.21.15.5:0/2008602029 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11967 (secure 0 0 0) 0x7fef7409c5e0 con 0x7fef7c106f20 2024-09-11T22:30:36.803 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:30:36.803 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"btime":"2024-09-11T22:29:21:436884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:29:21.436856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26914":{"gid":26914,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1755675305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1755675305},{"type":"v1","addr":"172.21.15.159:6833","nonce":1755675305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:30:36.803 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 40 2024-09-11T22:30:36.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 -- 172.21.15.5:0/2008602029 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fef680a2cf0 msgr2=0x7fef680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:36.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fef680a2cf0 0x7fef680a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef78005ab0 tx=0x7fef78005a20 comp rx=0 tx=0).stop 2024-09-11T22:30:36.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 -- 172.21.15.5:0/2008602029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c106f20 msgr2=0x7fef7c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:36.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c106f20 0x7fef7c19cc30 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fef7400eaa0 tx=0x7fef7400eea0 comp rx=0 tx=0).stop 2024-09-11T22:30:36.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 -- 172.21.15.5:0/2008602029 shutdown_connections 2024-09-11T22:30:36.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c106f20 0x7fef7c19cc30 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:36.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fef680a2cf0 0x7fef680a51b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:36.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef7c1027a0 0x7fef7c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:36.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 --2- 172.21.15.5:0/2008602029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fef7c101df0 0x7fef7c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:36.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 -- 172.21.15.5:0/2008602029 >> 172.21.15.5:0/2008602029 conn(0x7fef7c0fd8b0 msgr2=0x7fef7c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:36.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 -- 172.21.15.5:0/2008602029 shutdown_connections 2024-09-11T22:30:36.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:36.804+0000 7fef836f0640 1 -- 172.21.15.5:0/2008602029 wait complete. 2024-09-11T22:30:37.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 7.76 MB/sec execute 295 sec latency 16972.965 ms 2024-09-11T22:30:37.850 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id b21ec0ac --release --await' 2024-09-11T22:30:37.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id b21ec0ac --release --await 2024-09-11T22:30:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:37 smithi005 ceph-mon[30943]: pgmap v263: 97 pgs: 97 active+clean; 94 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 827 KiB/s wr, 101 op/s 2024-09-11T22:30:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:37 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2008602029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:37 smithi159 ceph-mon[35534]: pgmap v263: 97 pgs: 97 active+clean; 94 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 827 KiB/s wr, 101 op/s 2024-09-11T22:30:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:37 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2008602029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:37 smithi172 ceph-mon[35461]: pgmap v263: 97 pgs: 97 active+clean; 94 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 827 KiB/s wr, 101 op/s 2024-09-11T22:30:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:37 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2008602029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:38.114 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:38.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 636263 7.74 MB/sec execute 296 sec latency 17973.062 ms 2024-09-11T22:30:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.573+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/870082484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8101420 msgr2=0x7fe6d810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:38.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.573+0000 7fe6dfc1f640 1 --2- 172.21.15.5:0/870082484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8101420 0x7fe6d810d950 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009cd0 tx=0x7fe6d002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.573+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/870082484 shutdown_connections 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.573+0000 7fe6dfc1f640 1 --2- 172.21.15.5:0/870082484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8101420 0x7fe6d810d950 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.573+0000 7fe6dfc1f640 1 --2- 172.21.15.5:0/870082484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8100a80 0x7fe6d8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.573+0000 7fe6dfc1f640 1 --2- 172.21.15.5:0/870082484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d8106a80 0x7fe6d8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.573+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/870082484 >> 172.21.15.5:0/870082484 conn(0x7fe6d80fc7a0 msgr2=0x7fe6d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.573+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/870082484 shutdown_connections 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.573+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/870082484 wait complete. 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6dfc1f640 1 Processor -- start 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6dfc1f640 1 -- start start 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6dfc1f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d8100a80 0x7fe6d8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6dfc1f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8101420 0x7fe6d8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6dfc1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8106a80 0x7fe6d819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6dfc1f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8111fa0 con 0x7fe6d8106a80 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6dfc1f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8111e20 con 0x7fe6d8101420 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6dfc1f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8112120 con 0x7fe6d8100a80 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6de1bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8106a80 0x7fe6d819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6de1bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8106a80 0x7fe6d819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35358/0 (socket says 172.21.15.5:35358) 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6de1bc640 1 -- 172.21.15.5:0/2989853184 learned_addr learned my addr 172.21.15.5:0/2989853184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8101420 0x7fe6d8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.574+0000 7fe6dd9bb640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d8100a80 0x7fe6d8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6de1bc640 1 -- 172.21.15.5:0/2989853184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d8100a80 msgr2=0x7fe6d8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6de1bc640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d8100a80 0x7fe6d8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6de1bc640 1 -- 172.21.15.5:0/2989853184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8101420 msgr2=0x7fe6d8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6de1bc640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8101420 0x7fe6d8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6de1bc640 1 -- 172.21.15.5:0/2989853184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d819d3d0 con 0x7fe6d8106a80 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6de1bc640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8106a80 0x7fe6d819ccf0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0006010 tx=0x7fe6d0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6ceffd640 1 -- 172.21.15.5:0/2989853184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0004450 con 0x7fe6d8106a80 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/2989853184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d819d660 con 0x7fe6d8106a80 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6ceffd640 1 -- 172.21.15.5:0/2989853184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d0003e10 con 0x7fe6d8106a80 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6ceffd640 1 -- 172.21.15.5:0/2989853184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0046480 con 0x7fe6d8106a80 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.575+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/2989853184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81a4fa0 con 0x7fe6d8106a80 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.577+0000 7fe6ceffd640 1 -- 172.21.15.5:0/2989853184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe6d00045f0 con 0x7fe6d8106a80 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6e1ff8740 1 Processor -- start 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6e1ff8740 1 -- start start 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 0x55a419d07110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419d07650 0x55a419d09a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a419c5d620 con 0x55a419cf7f20 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a419c04e50 con 0x55a419cf8840 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a419c595e0 con 0x55a419d07650 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6dd1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6ceffd640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe6c40a2cf0 0x7fe6c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6dd1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35360/0 (socket says 172.21.15.5:35360) 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/3328873681 learned_addr learned my addr 172.21.15.5:0/3328873681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.578+0000 7fe6dd9bb640 1 --2- 172.21.15.5:0/3328873681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 0x55a419d07110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/3328873681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419d07650 0x55a419d09a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6de1bc640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe6c40a2cf0 0x7fe6c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/3328873681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419d07650 msgr2=0x55a419d09a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/3328873681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419d07650 0x55a419d09a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6ceffd640 1 -- 172.21.15.5:0/2989853184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fe6d002e030 con 0x7fe6d8106a80 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/3328873681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 msgr2=0x55a419d07110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/3328873681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 0x55a419d07110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/3328873681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a419bf5d40 con 0x55a419cf7f20 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6dd9bb640 1 --2- 172.21.15.5:0/3328873681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 0x55a419d07110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/3328873681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf8300 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8013590 tx=0x7fe6c800a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e955d2bb5da89e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6c3fff640 1 -- 172.21.15.5:0/3328873681 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8019030 con 0x55a419cf7f20 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6de1bc640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe6c40a2cf0 0x7fe6c40a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00b9cf0 tx=0x7fe6d00095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6c3fff640 1 -- 172.21.15.5:0/3328873681 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c80110d0 con 0x55a419cf7f20 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.579+0000 7fe6c3fff640 1 -- 172.21.15.5:0/3328873681 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800f8c0 con 0x55a419cf7f20 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/3328873681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 msgr2=0x55a419cf8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/3328873681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf8300 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8013590 tx=0x7fe6c800a800 comp rx=0 tx=0).stop 2024-09-11T22:30:38.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/3328873681 shutdown_connections 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/3328873681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419d07650 0x55a419d09a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/3328873681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 0x55a419d07110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/3328873681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf8300 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/3328873681 >> 172.21.15.5:0/3328873681 conn(0x55a419cf45f0 msgr2=0x55a419cf49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/3328873681 shutdown_connections 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/3328873681 wait complete. 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 -- start start 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.580+0000 7fe6e1ff8740 1 Processor -- start 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf7f20 0x55a419d04590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 0x55a419d04ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d05010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a419d0a220 con 0x55a419d07650 2024-09-11T22:30:38.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a419be5f10 con 0x55a419cf8840 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a419d0a3a0 con 0x55a419cf7f20 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6e1ff8740 10 client.0 _ll_register_callbacks cb 0x55a419c09db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d05010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d05010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35376/0 (socket says 172.21.15.5:35376) 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/1551595670 learned_addr learned my addr 172.21.15.5:0/1551595670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd9bb640 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 0x55a419d04ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf7f20 0x55a419d04590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/1551595670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf7f20 msgr2=0x55a419d04590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf7f20 0x55a419d04590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/1551595670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 msgr2=0x55a419d04ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 0x55a419d04ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/1551595670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a41996c580 con 0x55a419d07650 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d05010 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8011d10 tx=0x7fe6c8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad82fc862f2cd3d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.581+0000 7fe6c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1551595670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8019030 con 0x55a419d07650 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1551595670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fe6c801a070 con 0x55a419d07650 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1551595670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c80110d0 con 0x55a419d07650 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1551595670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe6c8032050 con 0x55a419d07650 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6e1ff8740 10 client.18072 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1551595670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a419a36960 con 0x55a419d07650 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1551595670 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c80110d0 con 0x55a419d07650 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1551595670 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe6c8004c60 con 0x55a419d07650 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6e1ff8740 10 client.18072 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6e1ff8740 10 client.18072 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:30:38.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6e1ff8740 20 client.18072 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a419d170e0 0x55a419d194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6e1ff8740 4 client.18072 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.582+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1551595670 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a419c6ab90 con 0x55a419d170e0 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.583+0000 7fe6dd9bb640 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a419d170e0 0x55a419d194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.583+0000 7fe6dd9bb640 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a419d170e0 0x55a419d194d0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fad626f917dcc40e server_cookie=665621d22835edeb in_seq=0 out_seq=0 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.583+0000 7fe6c17fa640 10 client.18072 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1551595670 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a419c6ab90 con 0x55a419d170e0 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6c17fa640 10 client.18072 handle_command_reply: tid=0 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 client.18072 shutdown 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1551595670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 msgr2=0x55a419d05010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d05010 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8011d10 tx=0x7fe6c8015040 comp rx=0 tx=0).stop 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1551595670 shutdown_connections 2024-09-11T22:30:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a419d170e0 0x55a419d194d0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d05010 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf8840 0x55a419d04ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6c17fa640 0 client.18072 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1551595670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf7f20 0x55a419d04590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1551595670 >> 172.21.15.5:0/1551595670 conn(0x55a419cf45f0 msgr2=0x55a419cfa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1551595670 shutdown_connections 2024-09-11T22:30:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1551595670 wait complete. 2024-09-11T22:30:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.584+0000 7fe6e1ff8740 20 client.18072 trim_cache size 0 max 16384 2024-09-11T22:30:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.601+0000 7fe6e1ff8740 1 Processor -- start 2024-09-11T22:30:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.601+0000 7fe6e1ff8740 1 -- start start 2024-09-11T22:30:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.601+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf8840 0x55a419d1ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419d07650 0x55a419d1f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81ab7f0 con 0x55a419cf7f20 2024-09-11T22:30:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81ad330 con 0x55a419d07650 2024-09-11T22:30:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81abec0 con 0x55a419cf8840 2024-09-11T22:30:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6dd1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6dd1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35382/0 (socket says 172.21.15.5:35382) 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/2477802146 learned_addr learned my addr 172.21.15.5:0/2477802146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6dd9bb640 1 --2- 172.21.15.5:0/2477802146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf8840 0x55a419d1ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/2477802146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf8840 msgr2=0x55a419d1ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/2477802146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf8840 0x55a419d1ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/2477802146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419d07650 msgr2=0x55a419d1f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/2477802146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419d07650 0x55a419d1f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/2477802146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a419cf2bb0 con 0x55a419cf7f20 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/2477802146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf3640 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80058c0 tx=0x7fe6c800a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3aef0f5d5eb799 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6c37fe640 1 -- 172.21.15.5:0/2477802146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8019070 con 0x55a419cf7f20 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6c37fe640 1 -- 172.21.15.5:0/2477802146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c800f980 con 0x55a419cf7f20 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.602+0000 7fe6c37fe640 1 -- 172.21.15.5:0/2477802146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8004660 con 0x55a419cf7f20 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/2477802146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 msgr2=0x55a419cf3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/2477802146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf3640 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80058c0 tx=0x7fe6c800a2f0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/2477802146 shutdown_connections 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/2477802146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419d07650 0x55a419d1f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/2477802146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf8840 0x55a419d1ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/2477802146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419cf7f20 0x55a419cf3640 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/2477802146 >> 172.21.15.5:0/2477802146 conn(0x55a419cf45f0 msgr2=0x55a419cf2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/2477802146 shutdown_connections 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/2477802146 wait complete. 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 -- start start 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.603+0000 7fe6e1ff8740 1 Processor -- start 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf7f20 0x55a419d0bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf8840 0x55a419d0c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6e1ff8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d0f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81acc60 con 0x55a419d07650 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81ac590 con 0x55a419cf7f20 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6e1ff8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81a9810 con 0x55a419cf8840 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6e1ff8740 10 client.0 _ll_register_callbacks cb 0x7fe6c8011bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6dd1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d0f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6dd1ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d0f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35384/0 (socket says 172.21.15.5:35384) 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/1137725945 learned_addr learned my addr 172.21.15.5:0/1137725945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6dd9bb640 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf8840 0x55a419d0c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf7f20 0x55a419d0bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/1137725945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf8840 msgr2=0x55a419d0c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf8840 0x55a419d0c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/1137725945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf7f20 msgr2=0x55a419d0bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf7f20 0x55a419d0bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.604+0000 7fe6dd1ba640 1 -- 172.21.15.5:0/1137725945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a419d0f990 con 0x55a419d07650 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6dd1ba640 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d0f2f0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8005bd0 tx=0x7fe6c800a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550cc1c71cda365c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1137725945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8019070 con 0x55a419d07650 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1137725945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fe6c8005460 con 0x55a419d07650 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1137725945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c800f980 con 0x55a419d07650 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1137725945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe6c8049050 con 0x55a419d07650 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6e1ff8740 10 client.18078 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1137725945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a419d826d0 con 0x55a419d07650 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1137725945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c800f980 con 0x55a419d07650 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1137725945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe6c802c480 con 0x55a419d07650 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6e1ff8740 10 client.18078 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6e1ff8740 10 client.18078 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:30:38.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6e1ff8740 20 client.18078 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:38.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.605+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a419d82d20 0x55a419d85110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:38.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.606+0000 7fe6e1ff8740 4 client.18078 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b21ec0ac", "release": true, "await": true}] 2024-09-11T22:30:38.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.606+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1137725945 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b21ec0ac", "release": true, "await": true}) -- 0x55a419c650a0 con 0x55a419d82d20 2024-09-11T22:30:38.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.606+0000 7fe6dd9bb640 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a419d82d20 0x55a419d85110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:38.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.606+0000 7fe6dd9bb640 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a419d82d20 0x55a419d85110 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d969e480088cfc80 server_cookie=6c1e170f26810e7b in_seq=0 out_seq=0 2024-09-11T22:30:38.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.606+0000 7fe6c17fa640 10 client.18078 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:38.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6c17fa640 1 -- 172.21.15.5:0/1137725945 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a419c650a0 con 0x55a419d82d20 2024-09-11T22:30:38.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6c17fa640 10 client.18078 handle_command_reply: tid=0 2024-09-11T22:30:38.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 client.18078 shutdown 2024-09-11T22:30:38.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1137725945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 msgr2=0x55a419d0f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:38.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d0f2f0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8005bd0 tx=0x7fe6c800a670 comp rx=0 tx=0).stop 2024-09-11T22:30:38.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1137725945 shutdown_connections 2024-09-11T22:30:38.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a419d82d20 0x55a419d85110 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a419d07650 0x55a419d0f2f0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a419cf8840 0x55a419d0c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 --2- 172.21.15.5:0/1137725945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a419cf7f20 0x55a419d0bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6c17fa640 0 client.18078 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:38.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1137725945 >> 172.21.15.5:0/1137725945 conn(0x55a419cf45f0 msgr2=0x55a419d04d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:38.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1137725945 shutdown_connections 2024-09-11T22:30:38.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 1 -- 172.21.15.5:0/1137725945 wait complete. 2024-09-11T22:30:38.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.618+0000 7fe6e1ff8740 20 client.18078 trim_cache size 0 max 16384 2024-09-11T22:30:38.620 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 40, 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 32, 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "b21ec0ac": { 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 32, 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:30:38.621 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.4, 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:38.622 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:38.623 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:30:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/2989853184 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe6c40a2cf0 msgr2=0x7fe6c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe6c40a2cf0 0x7fe6c40a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00b9cf0 tx=0x7fe6d00095c0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/2989853184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8106a80 msgr2=0x7fe6d819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8106a80 0x7fe6d819ccf0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0006010 tx=0x7fe6d0003c20 comp rx=0 tx=0).stop 2024-09-11T22:30:38.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/2989853184 shutdown_connections 2024-09-11T22:30:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8106a80 0x7fe6d819ccf0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe6c40a2cf0 0x7fe6c40a51b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8101420 0x7fe6d8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 --2- 172.21.15.5:0/2989853184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6d8100a80 0x7fe6d8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/2989853184 >> 172.21.15.5:0/2989853184 conn(0x7fe6d80fc7a0 msgr2=0x7fe6d80fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/2989853184 shutdown_connections 2024-09-11T22:30:38.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:38.619+0000 7fe6dfc1f640 1 -- 172.21.15.5:0/2989853184 wait complete. 2024-09-11T22:30:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.285+0000 7fbf2bd98640 1 -- 172.21.15.5:0/2233198057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24101250 msgr2=0x7fbf24107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:39.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.285+0000 7fbf2bd98640 1 --2- 172.21.15.5:0/2233198057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24101250 0x7fbf24107990 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009900 tx=0x7fbf1402a410 comp rx=0 tx=0).stop 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.285+0000 7fbf2bd98640 1 -- 172.21.15.5:0/2233198057 shutdown_connections 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.285+0000 7fbf2bd98640 1 --2- 172.21.15.5:0/2233198057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf241080f0 0x7fbf2410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.285+0000 7fbf2bd98640 1 --2- 172.21.15.5:0/2233198057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24101250 0x7fbf24107990 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.285+0000 7fbf2bd98640 1 --2- 172.21.15.5:0/2233198057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf241008a0 0x7fbf24100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.285+0000 7fbf2bd98640 1 -- 172.21.15.5:0/2233198057 >> 172.21.15.5:0/2233198057 conn(0x7fbf24075ef0 msgr2=0x7fbf24076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.286+0000 7fbf2bd98640 1 -- 172.21.15.5:0/2233198057 shutdown_connections 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.286+0000 7fbf2bd98640 1 -- 172.21.15.5:0/2233198057 wait complete. 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.286+0000 7fbf2bd98640 1 Processor -- start 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.286+0000 7fbf2bd98640 1 -- start start 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.286+0000 7fbf2bd98640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf241008a0 0x7fbf24198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.286+0000 7fbf2bd98640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf241080f0 0x7fbf24198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.286+0000 7fbf2bd98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2419cbf0 0x7fbf2419d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.286+0000 7fbf2bd98640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf24111f20 con 0x7fbf2419cbf0 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.286+0000 7fbf2bd98640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf24111da0 con 0x7fbf241080f0 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.286+0000 7fbf2bd98640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241120a0 con 0x7fbf241008a0 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2a335640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2419cbf0 0x7fbf2419d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2a335640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2419cbf0 0x7fbf2419d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35390/0 (socket says 172.21.15.5:35390) 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2a335640 1 -- 172.21.15.5:0/108936146 learned_addr learned my addr 172.21.15.5:0/108936146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf29b34640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf241008a0 0x7fbf24198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf29333640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf241080f0 0x7fbf24198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2a335640 1 -- 172.21.15.5:0/108936146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf241008a0 msgr2=0x7fbf24198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2a335640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf241008a0 0x7fbf24198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2a335640 1 -- 172.21.15.5:0/108936146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf241080f0 msgr2=0x7fbf24198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2a335640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf241080f0 0x7fbf24198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2a335640 1 -- 172.21.15.5:0/108936146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2419d6e0 con 0x7fbf2419cbf0 2024-09-11T22:30:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf29333640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf241080f0 0x7fbf24198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2a335640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2419cbf0 0x7fbf2419d0a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00eb10 tx=0x7fbf1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf1affd640 1 -- 172.21.15.5:0/108936146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c004b20 con 0x7fbf2419cbf0 2024-09-11T22:30:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf1affd640 1 -- 172.21.15.5:0/108936146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf1c004cc0 con 0x7fbf2419cbf0 2024-09-11T22:30:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2bd98640 1 -- 172.21.15.5:0/108936146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2419d9d0 con 0x7fbf2419cbf0 2024-09-11T22:30:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf1affd640 1 -- 172.21.15.5:0/108936146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c010760 con 0x7fbf2419cbf0 2024-09-11T22:30:39.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.287+0000 7fbf2bd98640 1 -- 172.21.15.5:0/108936146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf241a5140 con 0x7fbf2419cbf0 2024-09-11T22:30:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.290+0000 7fbf1affd640 1 -- 172.21.15.5:0/108936146 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbf1c005020 con 0x7fbf2419cbf0 2024-09-11T22:30:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.290+0000 7fbf2ec13740 1 Processor -- start 2024-09-11T22:30:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.290+0000 7fbf2ec13740 1 -- start start 2024-09-11T22:30:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.290+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e556eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.290+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5579770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.290+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e557c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.290+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2e54d32f0 con 0x55a2e556f410 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2e547b270 con 0x55a2e5579cb0 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a2e54cf2b0 con 0x55a2e556eaf0 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29b34640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5579770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29333640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e557c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29b34640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5579770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35402/0 (socket says 172.21.15.5:35402) 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29b34640 1 -- 172.21.15.5:0/2549750448 learned_addr learned my addr 172.21.15.5:0/2549750448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29333640 1 --2- 172.21.15.5:0/2549750448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e556eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29b34640 1 -- 172.21.15.5:0/2549750448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 msgr2=0x55a2e556eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29b34640 1 --2- 172.21.15.5:0/2549750448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e556eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29b34640 1 -- 172.21.15.5:0/2549750448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 msgr2=0x55a2e557c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29b34640 1 --2- 172.21.15.5:0/2549750448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e557c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf1affd640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbf100a2da0 0x7fbf100a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29b34640 1 -- 172.21.15.5:0/2549750448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e547a340 con 0x55a2e556f410 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf1affd640 1 -- 172.21.15.5:0/108936146 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fbf1c0cfb80 con 0x7fbf2419cbf0 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29333640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbf100a2da0 0x7fbf100a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf29b34640 1 --2- 172.21.15.5:0/2549750448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5579770 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009d20 tx=0x7fbf20011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2287eca1899be82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.291+0000 7fbf0ffff640 1 -- 172.21.15.5:0/2549750448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2001e070 con 0x55a2e556f410 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf0ffff640 1 -- 172.21.15.5:0/2549750448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf20009e10 con 0x55a2e556f410 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf0ffff640 1 -- 172.21.15.5:0/2549750448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20018910 con 0x55a2e556f410 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf2ec13740 1 -- 172.21.15.5:0/2549750448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 msgr2=0x55a2e5579770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/2549750448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5579770 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009d20 tx=0x7fbf20011810 comp rx=0 tx=0).stop 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf29333640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbf100a2da0 0x7fbf100a5260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbf14000c00 tx=0x7fbf14040040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf2ec13740 1 -- 172.21.15.5:0/2549750448 shutdown_connections 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/2549750448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e557c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/2549750448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5579770 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/2549750448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e556eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf2ec13740 1 -- 172.21.15.5:0/2549750448 >> 172.21.15.5:0/2549750448 conn(0x55a2e556b060 msgr2=0x55a2e556b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.292+0000 7fbf2ec13740 1 -- 172.21.15.5:0/2549750448 shutdown_connections 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2ec13740 1 -- 172.21.15.5:0/2549750448 wait complete. 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2ec13740 1 -- start start 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2ec13740 1 Processor -- start 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5576b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e556f410 0x55a2e5577070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e5579cb0 0x55a2e55775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2e557c880 con 0x55a2e5579cb0 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2e545c290 con 0x55a2e556f410 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a2e557ca00 con 0x55a2e556eaf0 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2ec13740 10 client.0 _ll_register_callbacks cb 0x55a2e548d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:39.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.293+0000 7fbf2a335640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e5579cb0 0x55a2e55775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf29b34640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5576b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf29b34640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5576b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:44806/0 (socket says 172.21.15.5:44806) 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf2a335640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e5579cb0 0x55a2e55775b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35414/0 (socket says 172.21.15.5:35414) 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf29b34640 1 -- 172.21.15.5:0/572862035 learned_addr learned my addr 172.21.15.5:0/572862035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf29333640 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e556f410 0x55a2e5577070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf2a335640 1 -- 172.21.15.5:0/572862035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 msgr2=0x55a2e5576b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf2a335640 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5576b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf2a335640 1 -- 172.21.15.5:0/572862035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e556f410 msgr2=0x55a2e5577070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf2a335640 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e556f410 0x55a2e5577070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf2a335640 1 -- 172.21.15.5:0/572862035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e51e6210 con 0x55a2e5579cb0 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf29b34640 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5576b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf2a335640 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e5579cb0 0x55a2e55775b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0cd200 tx=0x7fbf1c09b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884d83e390e7e19e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:39.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/572862035 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0a0380 con 0x55a2e5579cb0 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/572862035 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fbf1c0a47b0 con 0x55a2e5579cb0 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf2ec13740 1 -- 172.21.15.5:0/572862035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf20009e10 con 0x55a2e5579cb0 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/572862035 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbf1c0a4c40 con 0x55a2e5579cb0 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf2ec13740 10 client.18090 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf2ec13740 1 -- 172.21.15.5:0/572862035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a2e52b2620 con 0x55a2e5579cb0 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.294+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/572862035 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf20009e10 con 0x55a2e5579cb0 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.295+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/572862035 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fbf1c096bb0 con 0x55a2e5579cb0 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.295+0000 7fbf2ec13740 10 client.18090 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.295+0000 7fbf2ec13740 10 client.18090 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.295+0000 7fbf2ec13740 20 client.18090 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.295+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a2e5589b30 0x55a2e558bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.295+0000 7fbf2ec13740 4 client.18090 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.295+0000 7fbf2ec13740 1 -- 172.21.15.5:0/572862035 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2e54e0a20 con 0x55a2e5589b30 2024-09-11T22:30:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.295+0000 7fbf29b34640 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a2e5589b30 0x55a2e558bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.295+0000 7fbf29b34640 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a2e5589b30 0x55a2e558bfa0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c801bd47b45faad0 server_cookie=3c3586264b1f3bb0 in_seq=0 out_seq=0 2024-09-11T22:30:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.295+0000 7fbf0d7fa640 10 client.18090 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/572862035 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2e54e0a20 con 0x55a2e5589b30 2024-09-11T22:30:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf0d7fa640 10 client.18090 handle_command_reply: tid=0 2024-09-11T22:30:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 client.18090 shutdown 2024-09-11T22:30:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 -- 172.21.15.5:0/572862035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e5579cb0 msgr2=0x55a2e55775b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e5579cb0 0x55a2e55775b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0cd200 tx=0x7fbf1c09b9f0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 -- 172.21.15.5:0/572862035 shutdown_connections 2024-09-11T22:30:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a2e5589b30 0x55a2e558bfa0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e5579cb0 0x55a2e55775b0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e556f410 0x55a2e5577070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/572862035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5576b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf0d7fa640 0 client.18090 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 -- 172.21.15.5:0/572862035 >> 172.21.15.5:0/572862035 conn(0x55a2e556b060 msgr2=0x55a2e5570d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 -- 172.21.15.5:0/572862035 shutdown_connections 2024-09-11T22:30:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 1 -- 172.21.15.5:0/572862035 wait complete. 2024-09-11T22:30:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.297+0000 7fbf2ec13740 20 client.18090 trim_cache size 0 max 16384 2024-09-11T22:30:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf2ec13740 1 Processor -- start 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf2ec13740 1 -- start start 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e55773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e556a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e5591900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241ad870 con 0x55a2e556f410 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241af3b0 con 0x55a2e5579cb0 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241adf40 con 0x55a2e556eaf0 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf29333640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e556a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf29b34640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e55773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf29b34640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e55773e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:44822/0 (socket says 172.21.15.5:44822) 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf29b34640 1 -- 172.21.15.5:0/2138561861 learned_addr learned my addr 172.21.15.5:0/2138561861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf2a335640 1 --2- 172.21.15.5:0/2138561861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e5591900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf29b34640 1 -- 172.21.15.5:0/2138561861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 msgr2=0x55a2e5591900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf29b34640 1 --2- 172.21.15.5:0/2138561861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e5591900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf29b34640 1 -- 172.21.15.5:0/2138561861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 msgr2=0x55a2e556a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf29b34640 1 --2- 172.21.15.5:0/2138561861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e556a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf29b34640 1 -- 172.21.15.5:0/2138561861 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2e556f240 con 0x55a2e556eaf0 2024-09-11T22:30:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.312+0000 7fbf29333640 1 --2- 172.21.15.5:0/2138561861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e556a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf29b34640 1 --2- 172.21.15.5:0/2138561861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e55773e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009d20 tx=0x7fbf20009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3625201c262a56ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf0f7fe640 1 -- 172.21.15.5:0/2138561861 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20028040 con 0x55a2e556eaf0 2024-09-11T22:30:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf0f7fe640 1 -- 172.21.15.5:0/2138561861 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf200098f0 con 0x55a2e556eaf0 2024-09-11T22:30:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf0f7fe640 1 -- 172.21.15.5:0/2138561861 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2001e040 con 0x55a2e556eaf0 2024-09-11T22:30:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf2ec13740 1 -- 172.21.15.5:0/2138561861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 msgr2=0x55a2e55773e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/2138561861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e55773e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009d20 tx=0x7fbf20009780 comp rx=0 tx=0).stop 2024-09-11T22:30:39.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf2ec13740 1 -- 172.21.15.5:0/2138561861 shutdown_connections 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/2138561861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e5591900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/2138561861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e556a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/2138561861 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e55773e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf2ec13740 1 -- 172.21.15.5:0/2138561861 >> 172.21.15.5:0/2138561861 conn(0x55a2e556b060 msgr2=0x55a2e55699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf2ec13740 1 -- 172.21.15.5:0/2138561861 shutdown_connections 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.313+0000 7fbf2ec13740 1 -- 172.21.15.5:0/2138561861 wait complete. 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf2ec13740 1 -- start start 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf2ec13740 1 Processor -- start 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5581ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5582220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf2ec13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e557d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241aece0 con 0x55a2e556f410 2024-09-11T22:30:39.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241ae610 con 0x55a2e5579cb0 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf2ec13740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbf241ab870 con 0x55a2e556eaf0 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf2ec13740 10 client.0 _ll_register_callbacks cb 0x7fbf1c09be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf29333640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5582220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf29b34640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5581ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf29333640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5582220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35428/0 (socket says 172.21.15.5:35428) 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf2a335640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e557d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.314+0000 7fbf29333640 1 -- 172.21.15.5:0/3754013805 learned_addr learned my addr 172.21.15.5:0/3754013805 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf29b34640 1 -- 172.21.15.5:0/3754013805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 msgr2=0x55a2e557d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf29b34640 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e557d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf29b34640 1 -- 172.21.15.5:0/3754013805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 msgr2=0x55a2e5582220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf29b34640 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5582220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf29b34640 1 -- 172.21.15.5:0/3754013805 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2e557da60 con 0x55a2e556eaf0 2024-09-11T22:30:39.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf29333640 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5582220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf29b34640 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5581ce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009d20 tx=0x7fbf20011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a9b5fc27303d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/3754013805 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20031070 con 0x55a2e556eaf0 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/3754013805 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fbf20004630 con 0x55a2e556eaf0 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf2ec13740 1 -- 172.21.15.5:0/3754013805 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf200098f0 con 0x55a2e556eaf0 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/3754013805 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbf2003d050 con 0x55a2e556eaf0 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf2ec13740 10 client.27304 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf2ec13740 1 -- 172.21.15.5:0/3754013805 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a2e55f86f0 con 0x55a2e556eaf0 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.315+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/3754013805 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf200098f0 con 0x55a2e556eaf0 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.316+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/3754013805 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fbf200185e0 con 0x55a2e556eaf0 2024-09-11T22:30:39.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.316+0000 7fbf2ec13740 10 client.27304 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.316+0000 7fbf2ec13740 10 client.27304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.316+0000 7fbf2ec13740 20 client.27304 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.316+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a2e55f8c10 0x55a2e55fb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.316+0000 7fbf2ec13740 4 client.27304 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:30:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.316+0000 7fbf2ec13740 1 -- 172.21.15.5:0/3754013805 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2e54dad70 con 0x55a2e55f8c10 2024-09-11T22:30:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.316+0000 7fbf29333640 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a2e55f8c10 0x55a2e55fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.316+0000 7fbf0d7fa640 10 client.27304 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.316+0000 7fbf29333640 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a2e55f8c10 0x55a2e55fb050 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=193f504f32664108 server_cookie=21b7aea23d7606b6 in_seq=0 out_seq=0 2024-09-11T22:30:39.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 637521 7.73 MB/sec execute 297 sec latency 18266.620 ms 2024-09-11T22:30:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf0d7fa640 1 -- 172.21.15.5:0/3754013805 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2e54dad70 con 0x55a2e55f8c10 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf0d7fa640 10 client.27304 handle_command_reply: tid=0 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 client.27304 shutdown 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 -- 172.21.15.5:0/3754013805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 msgr2=0x55a2e5581ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5581ce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009d20 tx=0x7fbf20011810 comp rx=0 tx=0).stop 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 -- 172.21.15.5:0/3754013805 shutdown_connections 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a2e55f8c10 0x55a2e55fb050 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2e5579cb0 0x55a2e557d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf0d7fa640 0 client.27304 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e556f410 0x55a2e5582220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 --2- 172.21.15.5:0/3754013805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2e556eaf0 0x55a2e5581ce0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 -- 172.21.15.5:0/3754013805 >> 172.21.15.5:0/3754013805 conn(0x55a2e556b060 msgr2=0x55a2e5578bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 -- 172.21.15.5:0/3754013805 shutdown_connections 2024-09-11T22:30:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.366+0000 7fbf2ec13740 1 -- 172.21.15.5:0/3754013805 wait complete. 2024-09-11T22:30:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2ec13740 20 client.27304 trim_cache size 0 max 16384 2024-09-11T22:30:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:30:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:30:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:30:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:30:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 -- 172.21.15.5:0/108936146 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbf100a2da0 msgr2=0x7fbf100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbf100a2da0 0x7fbf100a5260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbf14000c00 tx=0x7fbf14040040 comp rx=0 tx=0).stop 2024-09-11T22:30:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 -- 172.21.15.5:0/108936146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2419cbf0 msgr2=0x7fbf2419d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2419cbf0 0x7fbf2419d0a0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00eb10 tx=0x7fbf1c00c2d0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 -- 172.21.15.5:0/108936146 shutdown_connections 2024-09-11T22:30:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf2419cbf0 0x7fbf2419d0a0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbf100a2da0 0x7fbf100a5260 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf241080f0 0x7fbf24198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 --2- 172.21.15.5:0/108936146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf241008a0 0x7fbf24198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 -- 172.21.15.5:0/108936146 >> 172.21.15.5:0/108936146 conn(0x7fbf24075ef0 msgr2=0x7fbf240ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 -- 172.21.15.5:0/108936146 shutdown_connections 2024-09-11T22:30:39.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:39.367+0000 7fbf2bd98640 1 -- 172.21.15.5:0/108936146 wait complete. 2024-09-11T22:30:39.478 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b21ec0ac', total seconds elapsed: 23.1 2024-09-11T22:30:39.479 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-11T22:30:39.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:30:39.750 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:40.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:39 smithi005 ceph-mon[30943]: pgmap v264: 97 pgs: 97 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-11T22:30:40.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:30:40.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:30:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:39 smithi159 ceph-mon[35534]: pgmap v264: 97 pgs: 97 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-11T22:30:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:30:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:30:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:39 smithi172 ceph-mon[35461]: pgmap v264: 97 pgs: 97 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-09-11T22:30:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:30:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:30:40.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 639690 7.75 MB/sec execute 298 sec latency 39.231 ms 2024-09-11T22:30:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.426+0000 7f393b4ab640 1 -- 172.21.15.5:0/4047183274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f393410c2f0 msgr2=0x7f393410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.426+0000 7f393b4ab640 1 --2- 172.21.15.5:0/4047183274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f393410c2f0 0x7f393410e790 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f392c009ca0 tx=0x7f392c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:30:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.426+0000 7f393b4ab640 1 -- 172.21.15.5:0/4047183274 shutdown_connections 2024-09-11T22:30:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.426+0000 7f393b4ab640 1 --2- 172.21.15.5:0/4047183274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f393410c2f0 0x7f393410e790 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.426+0000 7f393b4ab640 1 --2- 172.21.15.5:0/4047183274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39341037a0 0x7f393410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.426+0000 7f393b4ab640 1 --2- 172.21.15.5:0/4047183274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3934102df0 0x7f39341031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.426+0000 7f393b4ab640 1 -- 172.21.15.5:0/4047183274 >> 172.21.15.5:0/4047183274 conn(0x7f39340fc7d0 msgr2=0x7f39340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.426+0000 7f393b4ab640 1 -- 172.21.15.5:0/4047183274 shutdown_connections 2024-09-11T22:30:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.426+0000 7f393b4ab640 1 -- 172.21.15.5:0/4047183274 wait complete. 2024-09-11T22:30:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f393b4ab640 1 Processor -- start 2024-09-11T22:30:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f393b4ab640 1 -- start start 2024-09-11T22:30:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f393b4ab640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3934102df0 0x7f3934100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f393b4ab640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39341037a0 0x7f3934100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f393b4ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f393410c2f0 0x7f39341014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f393b4ab640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39341120f0 con 0x7f393410c2f0 2024-09-11T22:30:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f393b4ab640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3934111f70 con 0x7f39341037a0 2024-09-11T22:30:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f393b4ab640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3934112270 con 0x7f3934102df0 2024-09-11T22:30:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f3939a48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f393410c2f0 0x7f39341014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f3938a46640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39341037a0 0x7f3934100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f3939a48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f393410c2f0 0x7f39341014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35436/0 (socket says 172.21.15.5:35436) 2024-09-11T22:30:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f3939a48640 1 -- 172.21.15.5:0/3052564038 learned_addr learned my addr 172.21.15.5:0/3052564038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f3939247640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3934102df0 0x7f3934100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f3939a48640 1 -- 172.21.15.5:0/3052564038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3934102df0 msgr2=0x7f3934100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f3939a48640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3934102df0 0x7f3934100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f3939a48640 1 -- 172.21.15.5:0/3052564038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39341037a0 msgr2=0x7f3934100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.427+0000 7f3939a48640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39341037a0 0x7f3934100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.428+0000 7f3939a48640 1 -- 172.21.15.5:0/3052564038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39341a6a50 con 0x7f393410c2f0 2024-09-11T22:30:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.428+0000 7f3938a46640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39341037a0 0x7f3934100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.428+0000 7f3939a48640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f393410c2f0 0x7f39341014d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f392c031a20 tx=0x7f392c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.428+0000 7f392a7fc640 1 -- 172.21.15.5:0/3052564038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c003c30 con 0x7f393410c2f0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.428+0000 7f392a7fc640 1 -- 172.21.15.5:0/3052564038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c003dd0 con 0x7f393410c2f0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.428+0000 7f393b4ab640 1 -- 172.21.15.5:0/3052564038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39341a6ce0 con 0x7f393410c2f0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.428+0000 7f392a7fc640 1 -- 172.21.15.5:0/3052564038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c02fc40 con 0x7f393410c2f0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.428+0000 7f393b4ab640 1 -- 172.21.15.5:0/3052564038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39341a7220 con 0x7f393410c2f0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.429+0000 7f393b4ab640 1 -- 172.21.15.5:0/3052564038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39341031d0 con 0x7f393410c2f0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.430+0000 7f392a7fc640 1 -- 172.21.15.5:0/3052564038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f392c004b40 con 0x7f393410c2f0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.431+0000 7f392a7fc640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39200a2cf0 0x7f39200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.431+0000 7f3939247640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39200a2cf0 0x7f39200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.431+0000 7f392a7fc640 1 -- 172.21.15.5:0/3052564038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f392c0f0680 con 0x7f393410c2f0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.431+0000 7f3939247640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39200a2cf0 0x7f39200a51b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3930005d00 tx=0x7f393000b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.434+0000 7f392a7fc640 1 -- 172.21.15.5:0/3052564038 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f392c0bd3d0 con 0x7f393410c2f0 2024-09-11T22:30:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.667+0000 7f393b4ab640 1 -- 172.21.15.5:0/3052564038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39341072a0 con 0x7f393410c2f0 2024-09-11T22:30:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.668+0000 7f392a7fc640 1 -- 172.21.15.5:0/3052564038 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11967 (secure 0 0 0) 0x7f392c0bccd0 con 0x7f393410c2f0 2024-09-11T22:30:40.669 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:30:40.669 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"btime":"2024-09-11T22:29:21:436884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:29:21.436856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26914":{"gid":26914,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1755675305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1755675305},{"type":"v1","addr":"172.21.15.159:6833","nonce":1755675305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:30:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 40 2024-09-11T22:30:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.670+0000 7f393b4ab640 1 -- 172.21.15.5:0/3052564038 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39200a2cf0 msgr2=0x7f39200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.670+0000 7f393b4ab640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39200a2cf0 0x7f39200a51b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3930005d00 tx=0x7f393000b410 comp rx=0 tx=0).stop 2024-09-11T22:30:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.670+0000 7f393b4ab640 1 -- 172.21.15.5:0/3052564038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f393410c2f0 msgr2=0x7f39341014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.670+0000 7f393b4ab640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f393410c2f0 0x7f39341014d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f392c031a20 tx=0x7f392c031a50 comp rx=0 tx=0).stop 2024-09-11T22:30:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.670+0000 7f393b4ab640 1 -- 172.21.15.5:0/3052564038 shutdown_connections 2024-09-11T22:30:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.670+0000 7f393b4ab640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f393410c2f0 0x7f39341014d0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.670+0000 7f393b4ab640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39200a2cf0 0x7f39200a51b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.670+0000 7f393b4ab640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39341037a0 0x7f3934100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.670+0000 7f393b4ab640 1 --2- 172.21.15.5:0/3052564038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3934102df0 0x7f3934100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.670+0000 7f393b4ab640 1 -- 172.21.15.5:0/3052564038 >> 172.21.15.5:0/3052564038 conn(0x7f39340fc7d0 msgr2=0x7f39340fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.671+0000 7f393b4ab640 1 -- 172.21.15.5:0/3052564038 shutdown_connections 2024-09-11T22:30:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:40.671+0000 7f393b4ab640 1 -- 172.21.15.5:0/3052564038 wait complete. 2024-09-11T22:30:41.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 642131 7.76 MB/sec execute 299 sec latency 14.889 ms 2024-09-11T22:30:41.624 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-11T22:30:41.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-11T22:30:41.895 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:42.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:41 smithi005 ceph-mon[30943]: pgmap v265: 97 pgs: 97 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 889 KiB/s wr, 63 op/s 2024-09-11T22:30:42.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:41 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3052564038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:41 smithi159 ceph-mon[35534]: pgmap v265: 97 pgs: 97 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 889 KiB/s wr, 63 op/s 2024-09-11T22:30:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:41 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3052564038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:41 smithi172 ceph-mon[35461]: pgmap v265: 97 pgs: 97 active+clean; 90 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 889 KiB/s wr, 63 op/s 2024-09-11T22:30:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:41 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3052564038' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:30:42.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 643907 7.77 MB/sec execute 300 sec latency 39.119 ms 2024-09-11T22:30:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.506+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3647608930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4dc101410 msgr2=0x7fe4dc10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.506+0000 7fe4e0b6e640 1 --2- 172.21.15.5:0/3647608930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4dc101410 0x7fe4dc10d940 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009cd0 tx=0x7fe4d402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.506+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3647608930 shutdown_connections 2024-09-11T22:30:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.506+0000 7fe4e0b6e640 1 --2- 172.21.15.5:0/3647608930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4dc101410 0x7fe4dc10d940 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.506+0000 7fe4e0b6e640 1 --2- 172.21.15.5:0/3647608930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc100a70 0x7fe4dc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.506+0000 7fe4e0b6e640 1 --2- 172.21.15.5:0/3647608930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc106a70 0x7fe4dc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.506+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3647608930 >> 172.21.15.5:0/3647608930 conn(0x7fe4dc0fc790 msgr2=0x7fe4dc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.506+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3647608930 shutdown_connections 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.506+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3647608930 wait complete. 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.507+0000 7fe4e0b6e640 1 Processor -- start 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.507+0000 7fe4e0b6e640 1 -- start start 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.507+0000 7fe4e0b6e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4dc100a70 0x7fe4dc199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.507+0000 7fe4e0b6e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc101410 0x7fe4dc1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4e0b6e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc106a70 0x7fe4dc1a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4e0b6e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc110e70 con 0x7fe4dc100a70 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4e0b6e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc110cf0 con 0x7fe4dc106a70 2024-09-11T22:30:42.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4e0b6e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc110ff0 con 0x7fe4dc101410 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4dc100a70 0x7fe4dc199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4dc100a70 0x7fe4dc199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60564/0 (socket says 172.21.15.5:60564) 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc106a70 0x7fe4dc1a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4da59c640 1 -- 172.21.15.5:0/3012601071 learned_addr learned my addr 172.21.15.5:0/3012601071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4d9d9b640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc101410 0x7fe4dc1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4dad9d640 1 -- 172.21.15.5:0/3012601071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc101410 msgr2=0x7fe4dc1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc101410 0x7fe4dc1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4dad9d640 1 -- 172.21.15.5:0/3012601071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4dc100a70 msgr2=0x7fe4dc199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4dc100a70 0x7fe4dc199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4dad9d640 1 -- 172.21.15.5:0/3012601071 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc1a4840 con 0x7fe4dc106a70 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.508+0000 7fe4da59c640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4dc100a70 0x7fe4dc199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.509+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc106a70 0x7fe4dc1a4300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe4d402ae00 tx=0x7fe4d4031df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:42.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.509+0000 7fe4cb7fe640 1 -- 172.21.15.5:0/3012601071 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4003d80 con 0x7fe4dc106a70 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.509+0000 7fe4cb7fe640 1 -- 172.21.15.5:0/3012601071 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4004520 con 0x7fe4dc106a70 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.509+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3012601071 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc1a4a10 con 0x7fe4dc106a70 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.509+0000 7fe4cb7fe640 1 -- 172.21.15.5:0/3012601071 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4030d00 con 0x7fe4dc106a70 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.509+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3012601071 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc1a4ec0 con 0x7fe4dc106a70 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.511+0000 7fe4cb7fe640 1 -- 172.21.15.5:0/3012601071 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe4d40046d0 con 0x7fe4dc106a70 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4e2f47740 1 Processor -- start 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4e2f47740 1 -- start start 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4cb7fe640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4c40a2cf0 0x7fe4c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f3760 0x558b9e6f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f4080 0x558b9e6fe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4d9d9b640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4c40a2cf0 0x7fe4c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 0x558b9e700d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4cb7fe640 1 -- 172.21.15.5:0/3012601071 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fe4d402e030 con 0x7fe4dc106a70 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558b9e6592a0 con 0x558b9e6f4080 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558b9e600b10 con 0x558b9e6fe940 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558b9e655260 con 0x558b9e6f3760 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f4080 0x558b9e6fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f4080 0x558b9e6fe400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60578/0 (socket says 172.21.15.5:60578) 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4d9d9b640 1 -- 172.21.15.5:0/468416484 learned_addr learned my addr 172.21.15.5:0/468416484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.512+0000 7fe4dad9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 0x558b9e700d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4da59c640 1 --2- 172.21.15.5:0/468416484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f3760 0x558b9e6f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4d9d9b640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4c40a2cf0 0x7fe4c40a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0013570 tx=0x7fe4d0009f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4dad9d640 1 -- 172.21.15.5:0/468416484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f3760 msgr2=0x558b9e6f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/468416484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f3760 0x558b9e6f3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4dad9d640 1 -- 172.21.15.5:0/468416484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f4080 msgr2=0x558b9e6fe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:42.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/468416484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f4080 0x558b9e6fe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4dad9d640 1 -- 172.21.15.5:0/468416484 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b9e3cab90 con 0x558b9e6fe940 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/468416484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 0x558b9e700d30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40ee140 tx=0x7fe4d40bcda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b9b58fb39712ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4c8ff9640 1 -- 172.21.15.5:0/468416484 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40d8070 con 0x558b9e6fe940 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4c8ff9640 1 -- 172.21.15.5:0/468416484 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4036040 con 0x558b9e6fe940 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.513+0000 7fe4c8ff9640 1 -- 172.21.15.5:0/468416484 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40ee2b0 con 0x558b9e6fe940 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.514+0000 7fe4e2f47740 1 -- 172.21.15.5:0/468416484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 msgr2=0x558b9e700d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.514+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/468416484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 0x558b9e700d30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40ee140 tx=0x7fe4d40bcda0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.514+0000 7fe4e2f47740 1 -- 172.21.15.5:0/468416484 shutdown_connections 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.514+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/468416484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 0x558b9e700d30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.514+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/468416484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f4080 0x558b9e6fe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.514+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/468416484 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f3760 0x558b9e6f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.514+0000 7fe4e2f47740 1 -- 172.21.15.5:0/468416484 >> 172.21.15.5:0/468416484 conn(0x558b9e6efcf0 msgr2=0x558b9e6f00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:42.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.514+0000 7fe4e2f47740 1 -- 172.21.15.5:0/468416484 shutdown_connections 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.514+0000 7fe4e2f47740 1 -- 172.21.15.5:0/468416484 wait complete. 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4e2f47740 1 -- start start 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4e2f47740 1 Processor -- start 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e6fb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f4080 0x558b9e6fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 0x558b9e6fc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558b9e7012d0 con 0x558b9e6f3760 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558b9e5e16e0 con 0x558b9e6fe940 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558b9e701450 con 0x558b9e6f4080 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4e2f47740 10 client.0 _ll_register_callbacks cb 0x558b9e605a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e6fb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e6fb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60590/0 (socket says 172.21.15.5:60590) 2024-09-11T22:30:42.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4da59c640 1 -- 172.21.15.5:0/2023784599 learned_addr learned my addr 172.21.15.5:0/2023784599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 0x558b9e6fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4da59c640 1 -- 172.21.15.5:0/2023784599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f4080 msgr2=0x558b9e6fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4da59c640 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f4080 0x558b9e6fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4da59c640 1 -- 172.21.15.5:0/2023784599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 msgr2=0x558b9e6fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.515+0000 7fe4da59c640 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 0x558b9e6fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4da59c640 1 -- 172.21.15.5:0/2023784599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b9e70a730 con 0x558b9e6f3760 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 0x558b9e6fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4da59c640 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e6fb810 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc012d50 tx=0x7fe4cc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23585c99eeedf86c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/2023784599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00fe60 con 0x558b9e6f3760 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4e2f47740 1 -- 172.21.15.5:0/2023784599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d4036040 con 0x558b9e6f3760 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/2023784599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fe4cc012110 con 0x558b9e6f3760 2024-09-11T22:30:42.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/2023784599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe4cc004050 con 0x558b9e6f3760 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4e2f47740 10 client.18114 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4e2f47740 1 -- 172.21.15.5:0/2023784599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558b9e70d150 con 0x558b9e6f3760 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.516+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/2023784599 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4036040 con 0x558b9e6f3760 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.517+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/2023784599 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe4cc010430 con 0x558b9e6f3760 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.517+0000 7fe4e2f47740 10 client.18114 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.517+0000 7fe4e2f47740 10 client.18114 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.517+0000 7fe4e2f47740 20 client.18114 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.517+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558b9e70e660 0x558b9e710ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.517+0000 7fe4e2f47740 4 client.18114 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.517+0000 7fe4e2f47740 1 -- 172.21.15.5:0/2023784599 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b9e6669d0 con 0x558b9e70e660 2024-09-11T22:30:42.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.517+0000 7fe4d9d9b640 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558b9e70e660 0x558b9e710ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.517+0000 7fe4d9d9b640 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558b9e70e660 0x558b9e710ae0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12cbc95736f04eba server_cookie=e07b1487df1a880 in_seq=0 out_seq=0 2024-09-11T22:30:42.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.517+0000 7fe4c1ffb640 10 client.18114 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:42.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/2023784599 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b9e6669d0 con 0x558b9e70e660 2024-09-11T22:30:42.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4c1ffb640 10 client.18114 handle_command_reply: tid=0 2024-09-11T22:30:42.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 client.18114 shutdown 2024-09-11T22:30:42.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 -- 172.21.15.5:0/2023784599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 msgr2=0x558b9e6fb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:42.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e6fb810 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc012d50 tx=0x7fe4cc00f790 comp rx=0 tx=0).stop 2024-09-11T22:30:42.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 -- 172.21.15.5:0/2023784599 shutdown_connections 2024-09-11T22:30:42.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558b9e70e660 0x558b9e710ae0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6fe940 0x558b9e6fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f4080 0x558b9e6fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4c1ffb640 0 client.18114 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:42.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/2023784599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e6fb810 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 -- 172.21.15.5:0/2023784599 >> 172.21.15.5:0/2023784599 conn(0x558b9e6efcf0 msgr2=0x558b9e6f5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:42.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 -- 172.21.15.5:0/2023784599 shutdown_connections 2024-09-11T22:30:42.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 1 -- 172.21.15.5:0/2023784599 wait complete. 2024-09-11T22:30:42.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.518+0000 7fe4e2f47740 20 client.18114 trim_cache size 0 max 16384 2024-09-11T22:30:42.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.546+0000 7fe4e2f47740 1 Processor -- start 2024-09-11T22:30:42.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.546+0000 7fe4e2f47740 1 -- start start 2024-09-11T22:30:42.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e7163b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6f4080 0x558b9e7168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6fe940 0x558b9e6f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ab7a0 con 0x558b9e6f3760 2024-09-11T22:30:42.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ad2e0 con 0x558b9e6f4080 2024-09-11T22:30:42.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1abe70 con 0x558b9e6fe940 2024-09-11T22:30:42.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e7163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e7163b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60598/0 (socket says 172.21.15.5:60598) 2024-09-11T22:30:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4da59c640 1 -- 172.21.15.5:0/1533142601 learned_addr learned my addr 172.21.15.5:0/1533142601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/1533142601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6fe940 0x558b9e6f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4d9d9b640 1 --2- 172.21.15.5:0/1533142601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6f4080 0x558b9e7168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4dad9d640 1 -- 172.21.15.5:0/1533142601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6f4080 msgr2=0x558b9e7168f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/1533142601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6f4080 0x558b9e7168f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4dad9d640 1 -- 172.21.15.5:0/1533142601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 msgr2=0x558b9e7163b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/1533142601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e7163b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4dad9d640 1 -- 172.21.15.5:0/1533142601 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b9e6eb260 con 0x558b9e6fe940 2024-09-11T22:30:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.547+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/1533142601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6fe940 0x558b9e6f8a70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40c1480 tx=0x7fe4d40bce10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d229c4461f85f64c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4c3fff640 1 -- 172.21.15.5:0/1533142601 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40a2030 con 0x558b9e6fe940 2024-09-11T22:30:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4c3fff640 1 -- 172.21.15.5:0/1533142601 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4036040 con 0x558b9e6fe940 2024-09-11T22:30:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4c3fff640 1 -- 172.21.15.5:0/1533142601 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40d8040 con 0x558b9e6fe940 2024-09-11T22:30:42.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4e2f47740 1 -- 172.21.15.5:0/1533142601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6fe940 msgr2=0x558b9e6f8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/1533142601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6fe940 0x558b9e6f8a70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40c1480 tx=0x7fe4d40bce10 comp rx=0 tx=0).stop 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4e2f47740 1 -- 172.21.15.5:0/1533142601 shutdown_connections 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/1533142601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6fe940 0x558b9e6f8a70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/1533142601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6f4080 0x558b9e7168f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/1533142601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6f3760 0x558b9e7163b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4e2f47740 1 -- 172.21.15.5:0/1533142601 >> 172.21.15.5:0/1533142601 conn(0x558b9e6efcf0 msgr2=0x558b9e6fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4e2f47740 1 -- 172.21.15.5:0/1533142601 shutdown_connections 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.548+0000 7fe4e2f47740 1 -- 172.21.15.5:0/1533142601 wait complete. 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4e2f47740 1 -- start start 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4e2f47740 1 Processor -- start 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6f3760 0x558b9e702590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f4080 0x558b9e702ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4e2f47740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6fe940 0x558b9e778380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1acc10 con 0x558b9e6fe940 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ac540 con 0x558b9e6f3760 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4e2f47740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1a97a0 con 0x558b9e6f4080 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4e2f47740 10 client.0 _ll_register_callbacks cb 0x7fe4cc012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4dad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6fe940 0x558b9e778380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4dad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6fe940 0x558b9e778380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60610/0 (socket says 172.21.15.5:60610) 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4dad9d640 1 -- 172.21.15.5:0/952372611 learned_addr learned my addr 172.21.15.5:0/952372611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4d9d9b640 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f4080 0x558b9e702ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.549+0000 7fe4da59c640 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6f3760 0x558b9e702590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4dad9d640 1 -- 172.21.15.5:0/952372611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f4080 msgr2=0x558b9e702ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f4080 0x558b9e702ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4dad9d640 1 -- 172.21.15.5:0/952372611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6f3760 msgr2=0x558b9e702590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6f3760 0x558b9e702590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4dad9d640 1 -- 172.21.15.5:0/952372611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b9e703380 con 0x558b9e6fe940 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4d9d9b640 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f4080 0x558b9e702ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4dad9d640 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6fe940 0x558b9e778380 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40c5650 tx=0x7fe4d40307d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8f9cb3c8b42a7a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/952372611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40a2070 con 0x558b9e6fe940 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4e2f47740 1 -- 172.21.15.5:0/952372611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d4036040 con 0x558b9e6fe940 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/952372611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7fe4d40dbb50 con 0x558b9e6fe940 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/952372611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe4d40d8070 con 0x558b9e6fe940 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4e2f47740 10 client.18120 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.550+0000 7fe4e2f47740 1 -- 172.21.15.5:0/952372611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558b9e779980 con 0x558b9e6fe940 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.551+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/952372611 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d4036040 con 0x558b9e6fe940 2024-09-11T22:30:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.551+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/952372611 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe4d404e070 con 0x558b9e6fe940 2024-09-11T22:30:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.551+0000 7fe4e2f47740 10 client.18120 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.551+0000 7fe4e2f47740 10 client.18120 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:30:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.551+0000 7fe4e2f47740 20 client.18120 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.551+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558b9e779ce0 0x558b9e77c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.551+0000 7fe4e2f47740 4 client.18120 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-11T22:30:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.551+0000 7fe4e2f47740 1 -- 172.21.15.5:0/952372611 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x558b9e660d20 con 0x558b9e779ce0 2024-09-11T22:30:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.551+0000 7fe4da59c640 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558b9e779ce0 0x558b9e77c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.552+0000 7fe4c1ffb640 10 client.18120 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.552+0000 7fe4da59c640 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558b9e779ce0 0x558b9e77c0d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7c881c55a8497 server_cookie=7a15bf4c6b2f3716 in_seq=0 out_seq=0 2024-09-11T22:30:42.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4c1ffb640 1 -- 172.21.15.5:0/952372611 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558b9e660d20 con 0x558b9e779ce0 2024-09-11T22:30:42.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4c1ffb640 10 client.18120 handle_command_reply: tid=0 2024-09-11T22:30:42.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 client.18120 shutdown 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 -- 172.21.15.5:0/952372611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6fe940 msgr2=0x558b9e778380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6fe940 0x558b9e778380 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40c5650 tx=0x7fe4d40307d0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 -- 172.21.15.5:0/952372611 shutdown_connections 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558b9e779ce0 0x558b9e77c0d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558b9e6fe940 0x558b9e778380 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558b9e6f4080 0x558b9e702ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 --2- 172.21.15.5:0/952372611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558b9e6f3760 0x558b9e702590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 -- 172.21.15.5:0/952372611 >> 172.21.15.5:0/952372611 conn(0x558b9e6efcf0 msgr2=0x558b9e6f51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4c1ffb640 0 client.18120 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 -- 172.21.15.5:0/952372611 shutdown_connections 2024-09-11T22:30:42.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 1 -- 172.21.15.5:0/952372611 wait complete. 2024-09-11T22:30:42.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.642+0000 7fe4e2f47740 20 client.18120 trim_cache size 0 max 16384 2024-09-11T22:30:42.645 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:30:42.645 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 40, 2024-09-11T22:30:42.645 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:30:42.645 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 34, 2024-09-11T22:30:42.645 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:30:42.645 INFO:teuthology.orchestra.run.smithi005.stdout: "620b531d": { 2024-09-11T22:30:42.645 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 34, 2024-09-11T22:30:42.645 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:30:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:30:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:30:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:30:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:30:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:30:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.4, 2024-09-11T22:30:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:30:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:30:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:30:42.646 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:30:42.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:30:42.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:30:42.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:42.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:42.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:42.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:42.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:30:42.647 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:30:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3012601071 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4c40a2cf0 msgr2=0x7fe4c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4c40a2cf0 0x7fe4c40a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0013570 tx=0x7fe4d0009f90 comp rx=0 tx=0).stop 2024-09-11T22:30:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3012601071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc106a70 msgr2=0x7fe4dc1a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc106a70 0x7fe4dc1a4300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe4d402ae00 tx=0x7fe4d4031df0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3012601071 shutdown_connections 2024-09-11T22:30:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe4c40a2cf0 0x7fe4c40a51b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4dc106a70 0x7fe4dc1a4300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4dc101410 0x7fe4dc1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 --2- 172.21.15.5:0/3012601071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4dc100a70 0x7fe4dc199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:42.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3012601071 >> 172.21.15.5:0/3012601071 conn(0x7fe4dc0fc790 msgr2=0x7fe4dc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:42.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3012601071 shutdown_connections 2024-09-11T22:30:42.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:42.643+0000 7fe4e0b6e640 1 -- 172.21.15.5:0/3012601071 wait complete. 2024-09-11T22:30:43.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.75 MB/sec execute 301 sec latency 683.679 ms 2024-09-11T22:30:43.375 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '620b531d', quiesce took 3.9 seconds. Will release after: 15.499999999999998 2024-09-11T22:30:44.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:43 smithi005 ceph-mon[30943]: pgmap v266: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-11T22:30:44.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:43 smithi159 ceph-mon[35534]: pgmap v266: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-11T22:30:44.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:43 smithi172 ceph-mon[35461]: pgmap v266: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-11T22:30:44.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.72 MB/sec execute 302 sec latency 1683.707 ms 2024-09-11T22:30:45.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.70 MB/sec execute 303 sec latency 2683.798 ms 2024-09-11T22:30:46.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:45 smithi005 ceph-mon[30943]: pgmap v267: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-11T22:30:46.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:45 smithi159 ceph-mon[35534]: pgmap v267: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-11T22:30:46.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:45 smithi172 ceph-mon[35461]: pgmap v267: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-11T22:30:46.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.67 MB/sec execute 304 sec latency 3683.913 ms 2024-09-11T22:30:47.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.65 MB/sec execute 305 sec latency 4683.970 ms 2024-09-11T22:30:48.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:47 smithi005 ceph-mon[30943]: pgmap v268: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-11T22:30:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:47 smithi159 ceph-mon[35534]: pgmap v268: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-11T22:30:48.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:47 smithi172 ceph-mon[35461]: pgmap v268: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-11T22:30:48.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.62 MB/sec execute 306 sec latency 5684.008 ms 2024-09-11T22:30:49.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.60 MB/sec execute 307 sec latency 6684.050 ms 2024-09-11T22:30:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:49 smithi005 ceph-mon[30943]: pgmap v269: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-09-11T22:30:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:49 smithi159 ceph-mon[35534]: pgmap v269: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-09-11T22:30:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:49 smithi172 ceph-mon[35461]: pgmap v269: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-09-11T22:30:50.332 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.57 MB/sec execute 308 sec latency 7684.145 ms 2024-09-11T22:30:51.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.55 MB/sec execute 309 sec latency 8684.231 ms 2024-09-11T22:30:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:51 smithi005 ceph-mon[30943]: pgmap v270: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 145 op/s 2024-09-11T22:30:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:51 smithi159 ceph-mon[35534]: pgmap v270: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 145 op/s 2024-09-11T22:30:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:51 smithi172 ceph-mon[35461]: pgmap v270: 97 pgs: 97 active+clean; 94 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 145 op/s 2024-09-11T22:30:52.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.53 MB/sec execute 310 sec latency 9684.325 ms 2024-09-11T22:30:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 -- 172.21.15.5:0/1920220445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4950106a00 msgr2=0x7f4950106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 --2- 172.21.15.5:0/1920220445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4950106a00 0x7f4950106de0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4938009930 tx=0x7f493802a420 comp rx=0 tx=0).stop 2024-09-11T22:30:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 -- 172.21.15.5:0/1920220445 shutdown_connections 2024-09-11T22:30:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 --2- 172.21.15.5:0/1920220445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4950101390 0x7f495010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 --2- 172.21.15.5:0/1920220445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49501009f0 0x7f4950100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 --2- 172.21.15.5:0/1920220445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4950106a00 0x7f4950106de0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 -- 172.21.15.5:0/1920220445 >> 172.21.15.5:0/1920220445 conn(0x7f49500fc6b0 msgr2=0x7f49500fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 -- 172.21.15.5:0/1920220445 shutdown_connections 2024-09-11T22:30:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 -- 172.21.15.5:0/1920220445 wait complete. 2024-09-11T22:30:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 Processor -- start 2024-09-11T22:30:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.468+0000 7f4956bc0640 1 -- start start 2024-09-11T22:30:52.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f4956bc0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49501009f0 0x7f49501991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f4956bc0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4950101390 0x7f4950199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f4956bc0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4950106a00 0x7f49501a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f4956bc0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4950110e40 con 0x7f49501009f0 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f4956bc0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4950110cc0 con 0x7f4950101390 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f4956bc0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4950110fc0 con 0x7f4950106a00 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f495495c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49501009f0 0x7f49501991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f495495c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49501009f0 0x7f49501991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36866/0 (socket says 172.21.15.5:36866) 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f495495c640 1 -- 172.21.15.5:0/1638313374 learned_addr learned my addr 172.21.15.5:0/1638313374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f494ffff640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4950101390 0x7f4950199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f495495c640 1 -- 172.21.15.5:0/1638313374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4950106a00 msgr2=0x7f49501a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f495515d640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4950106a00 0x7f49501a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f495495c640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4950106a00 0x7f49501a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f495495c640 1 -- 172.21.15.5:0/1638313374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4950101390 msgr2=0x7f4950199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f495495c640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4950101390 0x7f4950199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f495495c640 1 -- 172.21.15.5:0/1638313374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49501a47b0 con 0x7f49501009f0 2024-09-11T22:30:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f494ffff640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4950101390 0x7f4950199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.469+0000 7f495495c640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49501009f0 0x7f49501991f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4938009230 tx=0x7f4938004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.470+0000 7f494dffb640 1 -- 172.21.15.5:0/1638313374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493803f030 con 0x7f49501009f0 2024-09-11T22:30:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.470+0000 7f4956bc0640 1 -- 172.21.15.5:0/1638313374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49501a4980 con 0x7f49501009f0 2024-09-11T22:30:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.470+0000 7f494dffb640 1 -- 172.21.15.5:0/1638313374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4938003ad0 con 0x7f49501009f0 2024-09-11T22:30:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.470+0000 7f494dffb640 1 -- 172.21.15.5:0/1638313374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938035070 con 0x7f49501009f0 2024-09-11T22:30:52.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.470+0000 7f4956bc0640 1 -- 172.21.15.5:0/1638313374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49501a4de0 con 0x7f49501009f0 2024-09-11T22:30:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.472+0000 7f494dffb640 1 -- 172.21.15.5:0/1638313374 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4938042030 con 0x7f49501009f0 2024-09-11T22:30:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.472+0000 7f4959a2f740 1 Processor -- start 2024-09-11T22:30:52.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.472+0000 7f4959a2f740 1 -- start start 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be86f9b0 0x5586be86fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be8702d0 0x5586be87eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be87f0c0 0x5586be8814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f4959a2f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5586be7d42f0 con 0x5586be86f9b0 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f4959a2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586be77c270 con 0x5586be8702d0 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f4959a2f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5586be7d02b0 con 0x5586be87f0c0 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f495515d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be86f9b0 0x5586be86fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f495515d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be86f9b0 0x5586be86fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36872/0 (socket says 172.21.15.5:36872) 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f495515d640 1 -- 172.21.15.5:0/1301138087 learned_addr learned my addr 172.21.15.5:0/1301138087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f494dffb640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f493c0a2da0 0x7f493c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f495495c640 1 --2- 172.21.15.5:0/1301138087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be8702d0 0x5586be87eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f494ffff640 1 --2- 172.21.15.5:0/1301138087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be87f0c0 0x5586be8814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f495515d640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f493c0a2da0 0x7f493c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f494dffb640 1 -- 172.21.15.5:0/1638313374 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f49380efa70 con 0x7f49501009f0 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f495515d640 1 -- 172.21.15.5:0/1301138087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be87f0c0 msgr2=0x5586be8814b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f495515d640 1 --2- 172.21.15.5:0/1301138087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be87f0c0 0x5586be8814b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f495515d640 1 -- 172.21.15.5:0/1301138087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be8702d0 msgr2=0x5586be87eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f495515d640 1 --2- 172.21.15.5:0/1301138087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be8702d0 0x5586be87eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.473+0000 7f495515d640 1 -- 172.21.15.5:0/1301138087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586be77b340 con 0x5586be86f9b0 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.474+0000 7f495515d640 1 --2- 172.21.15.5:0/1301138087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be86f9b0 0x5586be86fd90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4948013840 tx=0x7f4948013d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a533899120700a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.474+0000 7f495515d640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f493c0a2da0 0x7f493c0a5260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4948004000 tx=0x7f494801c000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.474+0000 7f4946ffd640 1 -- 172.21.15.5:0/1301138087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948022070 con 0x5586be86f9b0 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.474+0000 7f4946ffd640 1 -- 172.21.15.5:0/1301138087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4948003d40 con 0x5586be86f9b0 2024-09-11T22:30:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.474+0000 7f4946ffd640 1 -- 172.21.15.5:0/1301138087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494801d410 con 0x5586be86f9b0 2024-09-11T22:30:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 -- 172.21.15.5:0/1301138087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be86f9b0 msgr2=0x5586be86fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 --2- 172.21.15.5:0/1301138087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be86f9b0 0x5586be86fd90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f4948013840 tx=0x7f4948013d00 comp rx=0 tx=0).stop 2024-09-11T22:30:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 -- 172.21.15.5:0/1301138087 shutdown_connections 2024-09-11T22:30:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 --2- 172.21.15.5:0/1301138087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be87f0c0 0x5586be8814b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 --2- 172.21.15.5:0/1301138087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be8702d0 0x5586be87eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 --2- 172.21.15.5:0/1301138087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be86f9b0 0x5586be86fd90 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 -- 172.21.15.5:0/1301138087 >> 172.21.15.5:0/1301138087 conn(0x5586be86c060 msgr2=0x5586be86c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 -- 172.21.15.5:0/1301138087 shutdown_connections 2024-09-11T22:30:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 -- 172.21.15.5:0/1301138087 wait complete. 2024-09-11T22:30:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 -- start start 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.475+0000 7f4959a2f740 1 Processor -- start 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 0x5586be87c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be87c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be87f0c0 0x5586be87cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f4959a2f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5586be881c30 con 0x5586be8702d0 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f4959a2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586be75d290 con 0x5586be87f0c0 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f4959a2f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5586be881db0 con 0x5586be86f9b0 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f4959a2f740 10 client.0 _ll_register_callbacks cb 0x5586be78e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f494ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be87c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f494ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be87c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36880/0 (socket says 172.21.15.5:36880) 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f494ffff640 1 -- 172.21.15.5:0/238877783 learned_addr learned my addr 172.21.15.5:0/238877783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:52.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f495515d640 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be87f0c0 0x5586be87cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f495495c640 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 0x5586be87c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f494ffff640 1 -- 172.21.15.5:0/238877783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 msgr2=0x5586be87c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f494ffff640 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 0x5586be87c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f494ffff640 1 -- 172.21.15.5:0/238877783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be87f0c0 msgr2=0x5586be87cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f494ffff640 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be87f0c0 0x5586be87cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f494ffff640 1 -- 172.21.15.5:0/238877783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586be4e7210 con 0x5586be8702d0 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f495495c640 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 0x5586be87c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f494ffff640 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be87c580 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4940010670 tx=0x7f4940010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b19715b3a5da8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f492ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.476+0000 7f492ffff640 1 -- 172.21.15.5:0/238877783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494000ead0 con 0x5586be8702d0 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f492ffff640 1 -- 172.21.15.5:0/238877783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f494000ec70 con 0x5586be8702d0 2024-09-11T22:30:52.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f4959a2f740 1 -- 172.21.15.5:0/238877783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4948003d40 con 0x5586be8702d0 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f492ffff640 1 -- 172.21.15.5:0/238877783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4940010e70 con 0x5586be8702d0 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f4959a2f740 10 client.18132 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f4959a2f740 1 -- 172.21.15.5:0/238877783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5586be5b3620 con 0x5586be8702d0 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f492ffff640 1 -- 172.21.15.5:0/238877783 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4948003d40 con 0x5586be8702d0 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f492ffff640 1 -- 172.21.15.5:0/238877783 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f494000f040 con 0x5586be8702d0 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f4959a2f740 10 client.18132 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f4959a2f740 10 client.18132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f4959a2f740 20 client.18132 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f4959a2f740 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5586be88f120 0x5586be891530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f4959a2f740 4 client.18132 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f4959a2f740 1 -- 172.21.15.5:0/238877783 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586be7e1a20 con 0x5586be88f120 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.477+0000 7f495495c640 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5586be88f120 0x5586be891530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.478+0000 7f495495c640 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5586be88f120 0x5586be891530 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb000152eedd781 server_cookie=16b0bd6600024a69 in_seq=0 out_seq=0 2024-09-11T22:30:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.478+0000 7f492ffff640 10 client.18132 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f492ffff640 1 -- 172.21.15.5:0/238877783 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586be7e1a20 con 0x5586be88f120 2024-09-11T22:30:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f492ffff640 10 client.18132 handle_command_reply: tid=0 2024-09-11T22:30:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 client.18132 shutdown 2024-09-11T22:30:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 -- 172.21.15.5:0/238877783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 msgr2=0x5586be87c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be87c580 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f4940010670 tx=0x7f4940010b30 comp rx=0 tx=0).stop 2024-09-11T22:30:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 -- 172.21.15.5:0/238877783 shutdown_connections 2024-09-11T22:30:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5586be88f120 0x5586be891530 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be87f0c0 0x5586be87cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be87c580 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 --2- 172.21.15.5:0/238877783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 0x5586be87c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f492ffff640 0 client.18132 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 -- 172.21.15.5:0/238877783 >> 172.21.15.5:0/238877783 conn(0x5586be86c060 msgr2=0x5586be871cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 -- 172.21.15.5:0/238877783 shutdown_connections 2024-09-11T22:30:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 1 -- 172.21.15.5:0/238877783 wait complete. 2024-09-11T22:30:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.479+0000 7f4959a2f740 20 client.18132 trim_cache size 0 max 16384 2024-09-11T22:30:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f4959a2f740 1 Processor -- start 2024-09-11T22:30:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f4959a2f740 1 -- start start 2024-09-11T22:30:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 0x5586be87dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be896e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be87f0c0 0x5586be897380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f4959a2f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49501ab7b0 con 0x5586be8702d0 2024-09-11T22:30:52.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f4959a2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49501ad2f0 con 0x5586be87f0c0 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f4959a2f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49501abe80 con 0x5586be86f9b0 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f494ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be896e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f494ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be896e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36886/0 (socket says 172.21.15.5:36886) 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f494ffff640 1 -- 172.21.15.5:0/1835820214 learned_addr learned my addr 172.21.15.5:0/1835820214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f494ffff640 1 -- 172.21.15.5:0/1835820214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 msgr2=0x5586be87dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f495495c640 1 --2- 172.21.15.5:0/1835820214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 0x5586be87dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f494ffff640 1 --2- 172.21.15.5:0/1835820214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 0x5586be87dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f495515d640 1 --2- 172.21.15.5:0/1835820214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be87f0c0 0x5586be897380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f494ffff640 1 -- 172.21.15.5:0/1835820214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be87f0c0 msgr2=0x5586be897380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f494ffff640 1 --2- 172.21.15.5:0/1835820214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be87f0c0 0x5586be897380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f494ffff640 1 -- 172.21.15.5:0/1835820214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586be88dcc0 con 0x5586be8702d0 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f495495c640 1 --2- 172.21.15.5:0/1835820214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 0x5586be87dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:30:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.494+0000 7f494ffff640 1 --2- 172.21.15.5:0/1835820214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be896e40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f49400245c0 tx=0x7f49400244a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0958fcd93ffd2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f49467fc640 1 -- 172.21.15.5:0/1835820214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4940024830 con 0x5586be8702d0 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f49467fc640 1 -- 172.21.15.5:0/1835820214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49400249d0 con 0x5586be8702d0 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f49467fc640 1 -- 172.21.15.5:0/1835820214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49400048b0 con 0x5586be8702d0 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f4959a2f740 1 -- 172.21.15.5:0/1835820214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 msgr2=0x5586be896e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f4959a2f740 1 --2- 172.21.15.5:0/1835820214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be896e40 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f49400245c0 tx=0x7f49400244a0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f4959a2f740 1 -- 172.21.15.5:0/1835820214 shutdown_connections 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f4959a2f740 1 --2- 172.21.15.5:0/1835820214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be87f0c0 0x5586be897380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f4959a2f740 1 --2- 172.21.15.5:0/1835820214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be8702d0 0x5586be896e40 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f4959a2f740 1 --2- 172.21.15.5:0/1835820214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be86f9b0 0x5586be87dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f4959a2f740 1 -- 172.21.15.5:0/1835820214 >> 172.21.15.5:0/1835820214 conn(0x5586be86c060 msgr2=0x5586be87d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f4959a2f740 1 -- 172.21.15.5:0/1835820214 shutdown_connections 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.495+0000 7f4959a2f740 1 -- 172.21.15.5:0/1835820214 wait complete. 2024-09-11T22:30:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f4959a2f740 1 -- start start 2024-09-11T22:30:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f4959a2f740 1 Processor -- start 2024-09-11T22:30:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be86f9b0 0x5586be879090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be8702d0 0x5586be8795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f4959a2f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be87f0c0 0x5586be879b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f4959a2f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49501acc20 con 0x5586be87f0c0 2024-09-11T22:30:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f4959a2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49501ac550 con 0x5586be86f9b0 2024-09-11T22:30:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f4959a2f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49501a97d0 con 0x5586be8702d0 2024-09-11T22:30:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f4959a2f740 10 client.0 _ll_register_callbacks cb 0x7f49400094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:30:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f495515d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be87f0c0 0x5586be879b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f495515d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be87f0c0 0x5586be879b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36900/0 (socket says 172.21.15.5:36900) 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f495515d640 1 -- 172.21.15.5:0/3678962347 learned_addr learned my addr 172.21.15.5:0/3678962347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f495495c640 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be86f9b0 0x5586be879090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.496+0000 7f494ffff640 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be8702d0 0x5586be8795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f495515d640 1 -- 172.21.15.5:0/3678962347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be8702d0 msgr2=0x5586be8795d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f495515d640 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be8702d0 0x5586be8795d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f495515d640 1 -- 172.21.15.5:0/3678962347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be86f9b0 msgr2=0x5586be879090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f495515d640 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be86f9b0 0x5586be879090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f495515d640 1 -- 172.21.15.5:0/3678962347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586be8f8e60 con 0x5586be87f0c0 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f495515d640 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be87f0c0 0x5586be879b10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4948006010 tx=0x7f4948009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2e1bf35ba586c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f492ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f492ffff640 1 -- 172.21.15.5:0/3678962347 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948022070 con 0x5586be87f0c0 2024-09-11T22:30:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f492ffff640 1 -- 172.21.15.5:0/3678962347 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f4948013050 con 0x5586be87f0c0 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f4959a2f740 1 -- 172.21.15.5:0/3678962347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49400249d0 con 0x5586be87f0c0 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f492ffff640 1 -- 172.21.15.5:0/3678962347 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f494803b050 con 0x5586be87f0c0 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f4959a2f740 10 client.18138 fetch_fsmap learned FSMap version 40 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f4959a2f740 1 -- 172.21.15.5:0/3678962347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5586be8fa0a0 con 0x5586be87f0c0 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f492ffff640 1 -- 172.21.15.5:0/3678962347 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49400249d0 con 0x5586be87f0c0 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.497+0000 7f492ffff640 1 -- 172.21.15.5:0/3678962347 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f4948004030 con 0x5586be87f0c0 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.498+0000 7f4959a2f740 10 client.18138 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.498+0000 7f4959a2f740 10 client.18138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.498+0000 7f4959a2f740 20 client.18138 populate_metadata read hostname 'smithi005' 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.498+0000 7f4959a2f740 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5586be8fa530 0x5586be8fc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.498+0000 7f4959a2f740 4 client.18138 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.498+0000 7f4959a2f740 1 -- 172.21.15.5:0/3678962347 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5586be7dbd70 con 0x5586be8fa530 2024-09-11T22:30:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.498+0000 7f495495c640 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5586be8fa530 0x5586be8fc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.498+0000 7f495495c640 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5586be8fa530 0x5586be8fc950 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59158cb1c76f73c server_cookie=a07c073d5b910070 in_seq=0 out_seq=0 2024-09-11T22:30:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.498+0000 7f492ffff640 10 client.18138 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f492ffff640 1 -- 172.21.15.5:0/3678962347 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5586be7dbd70 con 0x5586be8fa530 2024-09-11T22:30:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f492ffff640 10 client.18138 handle_command_reply: tid=0 2024-09-11T22:30:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f4959a2f740 1 client.18138 shutdown 2024-09-11T22:30:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f4959a2f740 1 -- 172.21.15.5:0/3678962347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be87f0c0 msgr2=0x5586be879b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f4959a2f740 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be87f0c0 0x5586be879b10 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4948006010 tx=0x7f4948009e40 comp rx=0 tx=0).stop 2024-09-11T22:30:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f4959a2f740 1 -- 172.21.15.5:0/3678962347 shutdown_connections 2024-09-11T22:30:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f4959a2f740 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5586be8fa530 0x5586be8fc950 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f4959a2f740 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586be87f0c0 0x5586be879b10 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f4959a2f740 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586be8702d0 0x5586be8795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f492ffff640 0 client.18138 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:30:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f4959a2f740 1 --2- 172.21.15.5:0/3678962347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586be86f9b0 0x5586be879090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.560+0000 7f4959a2f740 1 -- 172.21.15.5:0/3678962347 >> 172.21.15.5:0/3678962347 conn(0x5586be86c060 msgr2=0x5586be87e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4959a2f740 1 -- 172.21.15.5:0/3678962347 shutdown_connections 2024-09-11T22:30:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4959a2f740 1 -- 172.21.15.5:0/3678962347 wait complete. 2024-09-11T22:30:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4959a2f740 20 client.18138 trim_cache size 0 max 16384 2024-09-11T22:30:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:30:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:30:52.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:30:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:30:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 -- 172.21.15.5:0/1638313374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f493c0a2da0 msgr2=0x7f493c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f493c0a2da0 0x7f493c0a5260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4948004000 tx=0x7f494801c000 comp rx=0 tx=0).stop 2024-09-11T22:30:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 -- 172.21.15.5:0/1638313374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49501009f0 msgr2=0x7f49501991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:52.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49501009f0 0x7f49501991f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4938009230 tx=0x7f4938004270 comp rx=0 tx=0).stop 2024-09-11T22:30:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 -- 172.21.15.5:0/1638313374 shutdown_connections 2024-09-11T22:30:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4950106a00 0x7f49501a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f493c0a2da0 0x7f493c0a5260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4950101390 0x7f4950199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 --2- 172.21.15.5:0/1638313374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49501009f0 0x7f49501991f0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 -- 172.21.15.5:0/1638313374 >> 172.21.15.5:0/1638313374 conn(0x7f49500fc6b0 msgr2=0x7f49500fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 -- 172.21.15.5:0/1638313374 shutdown_connections 2024-09-11T22:30:52.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:30:52.561+0000 7f4956bc0640 1 -- 172.21.15.5:0/1638313374 wait complete. 2024-09-11T22:30:53.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.50 MB/sec execute 311 sec latency 10684.358 ms 2024-09-11T22:30:54.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:53 smithi005 ceph-mon[30943]: pgmap v271: 97 pgs: 97 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2024-09-11T22:30:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:53 smithi159 ceph-mon[35534]: pgmap v271: 97 pgs: 97 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2024-09-11T22:30:54.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:53 smithi172 ceph-mon[35461]: pgmap v271: 97 pgs: 97 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2024-09-11T22:30:54.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.48 MB/sec execute 312 sec latency 11684.418 ms 2024-09-11T22:30:55.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.45 MB/sec execute 313 sec latency 12684.544 ms 2024-09-11T22:30:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:55 smithi005 ceph-mon[30943]: pgmap v272: 97 pgs: 97 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-09-11T22:30:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:55 smithi159 ceph-mon[35534]: pgmap v272: 97 pgs: 97 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-09-11T22:30:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:55 smithi172 ceph-mon[35461]: pgmap v272: 97 pgs: 97 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-09-11T22:30:56.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.43 MB/sec execute 314 sec latency 13684.684 ms 2024-09-11T22:30:57.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.41 MB/sec execute 315 sec latency 14684.783 ms 2024-09-11T22:30:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:57 smithi005 ceph-mon[30943]: pgmap v273: 97 pgs: 97 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-09-11T22:30:58.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:57 smithi159 ceph-mon[35534]: pgmap v273: 97 pgs: 97 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-09-11T22:30:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:57 smithi172 ceph-mon[35461]: pgmap v273: 97 pgs: 97 active+clean; 96 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-09-11T22:30:58.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.38 MB/sec execute 316 sec latency 15684.894 ms 2024-09-11T22:30:58.877 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:30:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:30:59.333 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.36 MB/sec execute 317 sec latency 16684.956 ms 2024-09-11T22:30:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.653+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/1154026032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d78102570 msgr2=0x7f2d781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.653+0000 7f2d7ef9c640 1 --2- 172.21.15.5:0/1154026032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d78102570 0x7f2d781029b0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009ca0 tx=0x7f2d6802a950 comp rx=0 tx=0).stop 2024-09-11T22:30:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.653+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/1154026032 shutdown_connections 2024-09-11T22:30:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.653+0000 7f2d7ef9c640 1 --2- 172.21.15.5:0/1154026032 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d78102ef0 0x7f2d7810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.653+0000 7f2d7ef9c640 1 --2- 172.21.15.5:0/1154026032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d78102570 0x7f2d781029b0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.653+0000 7f2d7ef9c640 1 --2- 172.21.15.5:0/1154026032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78100e20 0x7f2d78101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.653+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/1154026032 >> 172.21.15.5:0/1154026032 conn(0x7f2d780fc7d0 msgr2=0x7f2d780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.654+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/1154026032 shutdown_connections 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.654+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/1154026032 wait complete. 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.654+0000 7f2d7ef9c640 1 Processor -- start 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.654+0000 7f2d7ef9c640 1 -- start start 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.654+0000 7f2d7ef9c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d78100e20 0x7f2d781984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.654+0000 7f2d7ef9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78102570 0x7f2d78198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.654+0000 7f2d7ef9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d78102ef0 0x7f2d7819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.654+0000 7f2d7ef9c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78110f80 con 0x7f2d78102ef0 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.654+0000 7f2d7ef9c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78110e00 con 0x7f2d78100e20 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.654+0000 7f2d7ef9c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78111100 con 0x7f2d78102570 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7d539640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d78102ef0 0x7f2d7819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7cd38640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d78100e20 0x7f2d781984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7d539640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d78102ef0 0x7f2d7819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36918/0 (socket says 172.21.15.5:36918) 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7d539640 1 -- 172.21.15.5:0/4241732047 learned_addr learned my addr 172.21.15.5:0/4241732047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d77fff640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78102570 0x7f2d78198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7d539640 1 -- 172.21.15.5:0/4241732047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78102570 msgr2=0x7f2d78198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7d539640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78102570 0x7f2d78198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7d539640 1 -- 172.21.15.5:0/4241732047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d78100e20 msgr2=0x7f2d781984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7d539640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d78100e20 0x7f2d781984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7d539640 1 -- 172.21.15.5:0/4241732047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7819d4a0 con 0x7f2d78102ef0 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d77fff640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78102570 0x7f2d78198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7d539640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d78102ef0 0x7f2d7819cdc0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000c970 tx=0x7f2d7000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d75ffb640 1 -- 172.21.15.5:0/4241732047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70007bb0 con 0x7f2d78102ef0 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/4241732047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7819d790 con 0x7f2d78102ef0 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d75ffb640 1 -- 172.21.15.5:0/4241732047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d70007d50 con 0x7f2d78102ef0 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.655+0000 7f2d75ffb640 1 -- 172.21.15.5:0/4241732047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d70018800 con 0x7f2d78102ef0 2024-09-11T22:30:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.656+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/4241732047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d781a5070 con 0x7f2d78102ef0 2024-09-11T22:30:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.657+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/4241732047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d78101200 con 0x7f2d78102ef0 2024-09-11T22:30:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.659+0000 7f2d75ffb640 1 -- 172.21.15.5:0/4241732047 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2d7000fa50 con 0x7f2d78102ef0 2024-09-11T22:30:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.659+0000 7f2d75ffb640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d640a2cf0 0x7f2d640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:30:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.659+0000 7f2d75ffb640 1 -- 172.21.15.5:0/4241732047 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f2d700cf4c0 con 0x7f2d78102ef0 2024-09-11T22:30:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.659+0000 7f2d7cd38640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d640a2cf0 0x7f2d640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:30:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.660+0000 7f2d7cd38640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d640a2cf0 0x7f2d640a51b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2d600058c0 tx=0x7f2d6000a250 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:30:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.668+0000 7f2d75ffb640 1 -- 172.21.15.5:0/4241732047 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d70014030 con 0x7f2d78102ef0 2024-09-11T22:30:59.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:30:59 smithi005 ceph-mon[30943]: pgmap v274: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.1 MiB/s wr, 124 op/s 2024-09-11T22:30:59.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.885+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/4241732047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d7810a0a0 con 0x7f2d78102ef0 2024-09-11T22:30:59.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.886+0000 7f2d75ffb640 1 -- 172.21.15.5:0/4241732047 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11967 (secure 0 0 0) 0x7f2d7009c070 con 0x7f2d78102ef0 2024-09-11T22:30:59.887 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:30:59.887 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"btime":"2024-09-11T22:29:21:436884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:29:21.436856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26914":{"gid":26914,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1755675305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1755675305},{"type":"v1","addr":"172.21.15.159:6833","nonce":1755675305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:30:59.888 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 40 2024-09-11T22:30:59.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.888+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/4241732047 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d640a2cf0 msgr2=0x7f2d640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:59.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.888+0000 7f2d7ef9c640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d640a2cf0 0x7f2d640a51b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2d600058c0 tx=0x7f2d6000a250 comp rx=0 tx=0).stop 2024-09-11T22:30:59.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.888+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/4241732047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d78102ef0 msgr2=0x7f2d7819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:30:59.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.888+0000 7f2d7ef9c640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d78102ef0 0x7f2d7819cdc0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000c970 tx=0x7f2d7000ce30 comp rx=0 tx=0).stop 2024-09-11T22:30:59.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.888+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/4241732047 shutdown_connections 2024-09-11T22:30:59.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.888+0000 7f2d7ef9c640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d78102ef0 0x7f2d7819cdc0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:59.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.888+0000 7f2d7ef9c640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d78102570 0x7f2d78198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:59.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.888+0000 7f2d7ef9c640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2d640a2cf0 0x7f2d640a51b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:59.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.888+0000 7f2d7ef9c640 1 --2- 172.21.15.5:0/4241732047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d78100e20 0x7f2d781984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:30:59.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.888+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/4241732047 >> 172.21.15.5:0/4241732047 conn(0x7f2d780fc7d0 msgr2=0x7f2d7806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:30:59.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.889+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/4241732047 shutdown_connections 2024-09-11T22:30:59.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:30:59.889+0000 7f2d7ef9c640 1 -- 172.21.15.5:0/4241732047 wait complete. 2024-09-11T22:31:00.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:30:59 smithi159 ceph-mon[35534]: pgmap v274: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.1 MiB/s wr, 124 op/s 2024-09-11T22:31:00.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:30:59 smithi172 ceph-mon[35461]: pgmap v274: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 2.1 MiB/s wr, 124 op/s 2024-09-11T22:31:00.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.34 MB/sec execute 318 sec latency 17685.196 ms 2024-09-11T22:31:00.586 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 620b531d --release --await' 2024-09-11T22:31:00.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 620b531d --release --await 2024-09-11T22:31:00.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:00 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4241732047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:31:00.869 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:01.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:00 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4241732047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:31:01.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:00 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4241732047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:31:01.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 644248 7.31 MB/sec execute 319 sec latency 18685.223 ms 2024-09-11T22:31:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.384+0000 7f6637889640 1 -- 172.21.15.5:0/1421659682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6630100b20 msgr2=0x7f6630100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.384+0000 7f6637889640 1 --2- 172.21.15.5:0/1421659682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6630100b20 0x7f6630100f00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f6620009ca0 tx=0x7f662002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.384+0000 7f6637889640 1 -- 172.21.15.5:0/1421659682 shutdown_connections 2024-09-11T22:31:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.384+0000 7f6637889640 1 --2- 172.21.15.5:0/1421659682 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f663010e500 0x7f6630110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.384+0000 7f6637889640 1 --2- 172.21.15.5:0/1421659682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6630101440 0x7f663010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.384+0000 7f6637889640 1 --2- 172.21.15.5:0/1421659682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6630100b20 0x7f6630100f00 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.384+0000 7f6637889640 1 -- 172.21.15.5:0/1421659682 >> 172.21.15.5:0/1421659682 conn(0x7f66300fc7d0 msgr2=0x7f66300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.384+0000 7f6637889640 1 -- 172.21.15.5:0/1421659682 shutdown_connections 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.384+0000 7f6637889640 1 -- 172.21.15.5:0/1421659682 wait complete. 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6637889640 1 Processor -- start 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6637889640 1 -- start start 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6637889640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6630100b20 0x7f66301a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6637889640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6630101440 0x7f66301a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6637889640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f663010e500 0x7f663019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6637889640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6630114370 con 0x7f6630100b20 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6637889640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66301141f0 con 0x7f6630101440 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6637889640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66301144f0 con 0x7f663010e500 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6635625640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6630100b20 0x7f66301a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6635625640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6630100b20 0x7f66301a0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45180/0 (socket says 172.21.15.5:45180) 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6635625640 1 -- 172.21.15.5:0/1410579431 learned_addr learned my addr 172.21.15.5:0/1410579431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6634e24640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6630101440 0x7f66301a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6635625640 1 -- 172.21.15.5:0/1410579431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f663010e500 msgr2=0x7f663019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6635625640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f663010e500 0x7f663019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6635625640 1 -- 172.21.15.5:0/1410579431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6630101440 msgr2=0x7f66301a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.385+0000 7f6635625640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6630101440 0x7f66301a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.386+0000 7f6635625640 1 -- 172.21.15.5:0/1410579431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663019af90 con 0x7f6630100b20 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.386+0000 7f6634e24640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6630101440 0x7f66301a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.386+0000 7f6635625640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6630100b20 0x7f66301a0650 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f66200097b0 tx=0x7f6620031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.386+0000 7f662e7fc640 1 -- 172.21.15.5:0/1410579431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662002fba0 con 0x7f6630100b20 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.386+0000 7f662e7fc640 1 -- 172.21.15.5:0/1410579431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662002fd40 con 0x7f6630100b20 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.386+0000 7f6637889640 1 -- 172.21.15.5:0/1410579431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663019b220 con 0x7f6630100b20 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.386+0000 7f662e7fc640 1 -- 172.21.15.5:0/1410579431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662003b930 con 0x7f6630100b20 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.386+0000 7f6637889640 1 -- 172.21.15.5:0/1410579431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66301a7400 con 0x7f6630100b20 2024-09-11T22:31:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.388+0000 7f662e7fc640 1 -- 172.21.15.5:0/1410579431 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6620030cc0 con 0x7f6630100b20 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6639c62740 1 Processor -- start 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6639c62740 1 -- start start 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6639c62740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e609142060 0x55e609142440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6639c62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142980 0x55e60914cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6639c62740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60914d240 0x55e60914f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6639c62740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e6090a7620 con 0x55e609142060 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6639c62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e60904ee50 con 0x55e609142980 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6639c62740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e6090a35e0 con 0x55e60914d240 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6635e26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e609142060 0x55e609142440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6635e26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e609142060 0x55e609142440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45184/0 (socket says 172.21.15.5:45184) 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6635e26640 1 -- 172.21.15.5:0/2802051192 learned_addr learned my addr 172.21.15.5:0/2802051192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6635625640 1 --2- 172.21.15.5:0/2802051192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142980 0x55e60914cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6634e24640 1 --2- 172.21.15.5:0/2802051192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60914d240 0x55e60914f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f662e7fc640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f66180a2cf0 0x7f66180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6635e26640 1 -- 172.21.15.5:0/2802051192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60914d240 msgr2=0x55e60914f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6635e26640 1 --2- 172.21.15.5:0/2802051192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60914d240 0x55e60914f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6635e26640 1 -- 172.21.15.5:0/2802051192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142980 msgr2=0x55e60914cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6635e26640 1 --2- 172.21.15.5:0/2802051192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142980 0x55e60914cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.389+0000 7f6635e26640 1 -- 172.21.15.5:0/2802051192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60903fd40 con 0x55e609142060 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.390+0000 7f6635e26640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f66180a2cf0 0x7f66180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.390+0000 7f662e7fc640 1 -- 172.21.15.5:0/1410579431 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f66200f0b80 con 0x7f6630100b20 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.390+0000 7f6635e26640 1 --2- 172.21.15.5:0/2802051192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e609142060 0x55e609142440 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6628009930 tx=0x7f6628009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edde27257056418 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.390+0000 7f66177fe640 1 -- 172.21.15.5:0/2802051192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628017040 con 0x55e609142060 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.390+0000 7f66177fe640 1 -- 172.21.15.5:0/2802051192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662801ee00 con 0x55e609142060 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.390+0000 7f66177fe640 1 -- 172.21.15.5:0/2802051192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662800f040 con 0x55e609142060 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.390+0000 7f6635e26640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f66180a2cf0 0x7f66180a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f662801d000 tx=0x7f662801a000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 -- 172.21.15.5:0/2802051192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e609142060 msgr2=0x55e609142440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:01.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 --2- 172.21.15.5:0/2802051192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e609142060 0x55e609142440 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6628009930 tx=0x7f6628009df0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 -- 172.21.15.5:0/2802051192 shutdown_connections 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 --2- 172.21.15.5:0/2802051192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e60914d240 0x55e60914f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 --2- 172.21.15.5:0/2802051192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142980 0x55e60914cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 --2- 172.21.15.5:0/2802051192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e609142060 0x55e609142440 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 -- 172.21.15.5:0/2802051192 >> 172.21.15.5:0/2802051192 conn(0x55e60913e5f0 msgr2=0x55e60913e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 -- 172.21.15.5:0/2802051192 shutdown_connections 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 -- 172.21.15.5:0/2802051192 wait complete. 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 -- start start 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.391+0000 7f6639c62740 1 Processor -- start 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6639c62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e609149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6639c62740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e60914a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6639c62740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e60914aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6639c62740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e60914fe10 con 0x55e60914d240 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6639c62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e60902ff10 con 0x55e609142060 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6639c62740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e60914ff90 con 0x55e609142980 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6639c62740 10 client.0 _ll_register_callbacks cb 0x55e609053db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6635e26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e60914aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6635e26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e60914aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45198/0 (socket says 172.21.15.5:45198) 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6635e26640 1 -- 172.21.15.5:0/2586654745 learned_addr learned my addr 172.21.15.5:0/2586654745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6635625640 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e609149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6634e24640 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e60914a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6635e26640 1 -- 172.21.15.5:0/2586654745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 msgr2=0x55e60914a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6635e26640 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e60914a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6635e26640 1 -- 172.21.15.5:0/2586654745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 msgr2=0x55e609149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6635e26640 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e609149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6635e26640 1 -- 172.21.15.5:0/2586654745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e608db6580 con 0x55e60914d240 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6635e26640 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e60914aa60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f662801ed50 tx=0x7f662801ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2fe2bbdae07e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6614ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.392+0000 7f6614ff9640 1 -- 172.21.15.5:0/2586654745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628017050 con 0x55e60914d240 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6614ff9640 1 -- 172.21.15.5:0/2586654745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f6628013b20 con 0x55e60914d240 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6639c62740 1 -- 172.21.15.5:0/2586654745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662801ee00 con 0x55e60914d240 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6614ff9640 1 -- 172.21.15.5:0/2586654745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6628034050 con 0x55e60914d240 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6639c62740 10 client.18156 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6639c62740 1 -- 172.21.15.5:0/2586654745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e608e80960 con 0x55e60914d240 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6614ff9640 1 -- 172.21.15.5:0/2586654745 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662801ee00 con 0x55e60914d240 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6614ff9640 1 -- 172.21.15.5:0/2586654745 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f662801b040 con 0x55e60914d240 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6639c62740 10 client.18156 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:01.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6639c62740 10 client.18156 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:31:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6639c62740 20 client.18156 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6639c62740 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e60915cbb0 0x55e60915efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6639c62740 4 client.18156 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6639c62740 1 -- 172.21.15.5:0/2586654745 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6090b4b90 con 0x55e60915cbb0 2024-09-11T22:31:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.393+0000 7f6635625640 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e60915cbb0 0x55e60915efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.394+0000 7f6614ff9640 10 client.18156 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.394+0000 7f6635625640 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e60915cbb0 0x55e60915efa0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b191f649054d42e server_cookie=a6bc218e779056e9 in_seq=0 out_seq=0 2024-09-11T22:31:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6614ff9640 1 -- 172.21.15.5:0/2586654745 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6090b4b90 con 0x55e60915cbb0 2024-09-11T22:31:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6614ff9640 10 client.18156 handle_command_reply: tid=0 2024-09-11T22:31:01.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 client.18156 shutdown 2024-09-11T22:31:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 -- 172.21.15.5:0/2586654745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 msgr2=0x55e60914aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e60914aa60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f662801ed50 tx=0x7f662801ed80 comp rx=0 tx=0).stop 2024-09-11T22:31:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 -- 172.21.15.5:0/2586654745 shutdown_connections 2024-09-11T22:31:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e60915cbb0 0x55e60915efa0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e60914aa60 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e60914a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6614ff9640 0 client.18156 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:01.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 --2- 172.21.15.5:0/2586654745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e609149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 -- 172.21.15.5:0/2586654745 >> 172.21.15.5:0/2586654745 conn(0x55e60913e5f0 msgr2=0x55e60913d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 -- 172.21.15.5:0/2586654745 shutdown_connections 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 1 -- 172.21.15.5:0/2586654745 wait complete. 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.395+0000 7f6639c62740 20 client.18156 trim_cache size 0 max 16384 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6639c62740 1 Processor -- start 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6639c62740 1 -- start start 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6639c62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e609148e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6639c62740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e6091647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6639c62740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e609164d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6639c62740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66301afd60 con 0x55e60914d240 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6639c62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66301b18a0 con 0x55e609142060 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6639c62740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66301b0430 con 0x55e609142980 2024-09-11T22:31:01.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6635e26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e609164d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6635e26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e609164d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45204/0 (socket says 172.21.15.5:45204) 2024-09-11T22:31:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.412+0000 7f6635e26640 1 -- 172.21.15.5:0/2221415771 learned_addr learned my addr 172.21.15.5:0/2221415771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6635625640 1 --2- 172.21.15.5:0/2221415771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e609148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6634e24640 1 --2- 172.21.15.5:0/2221415771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e6091647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6635e26640 1 -- 172.21.15.5:0/2221415771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 msgr2=0x55e6091647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6635e26640 1 --2- 172.21.15.5:0/2221415771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e6091647d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6635e26640 1 -- 172.21.15.5:0/2221415771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 msgr2=0x55e609148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6635e26640 1 --2- 172.21.15.5:0/2221415771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e609148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6635e26640 1 -- 172.21.15.5:0/2221415771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60913e420 con 0x55e60914d240 2024-09-11T22:31:01.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6634e24640 1 --2- 172.21.15.5:0/2221415771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e6091647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:31:01.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6635e26640 1 --2- 172.21.15.5:0/2221415771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e609164d10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6628006c50 tx=0x7f6628009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c1ebc20db3be55 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:01.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6616ffd640 1 -- 172.21.15.5:0/2221415771 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628017040 con 0x55e60914d240 2024-09-11T22:31:01.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6616ffd640 1 -- 172.21.15.5:0/2221415771 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662802b070 con 0x55e60914d240 2024-09-11T22:31:01.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6616ffd640 1 -- 172.21.15.5:0/2221415771 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66280266c0 con 0x55e60914d240 2024-09-11T22:31:01.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6639c62740 1 -- 172.21.15.5:0/2221415771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 msgr2=0x55e609164d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:01.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.413+0000 7f6639c62740 1 --2- 172.21.15.5:0/2221415771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e609164d10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f6628006c50 tx=0x7f6628009690 comp rx=0 tx=0).stop 2024-09-11T22:31:01.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 -- 172.21.15.5:0/2221415771 shutdown_connections 2024-09-11T22:31:01.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 --2- 172.21.15.5:0/2221415771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e609164d10 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 --2- 172.21.15.5:0/2221415771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e6091647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 --2- 172.21.15.5:0/2221415771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e609148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 -- 172.21.15.5:0/2221415771 >> 172.21.15.5:0/2221415771 conn(0x55e60913e5f0 msgr2=0x55e609148780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 -- 172.21.15.5:0/2221415771 shutdown_connections 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 -- 172.21.15.5:0/2221415771 wait complete. 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 -- start start 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 Processor -- start 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e6091551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e609155730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e6091501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66301b11d0 con 0x55e60914d240 2024-09-11T22:31:01.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66301b0b00 con 0x55e609142060 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f66301add60 con 0x55e609142980 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.414+0000 7f6639c62740 10 client.0 _ll_register_callbacks cb 0x7f662400a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6635e26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e6091501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6635e26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e6091501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45216/0 (socket says 172.21.15.5:45216) 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6635e26640 1 -- 172.21.15.5:0/2284760706 learned_addr learned my addr 172.21.15.5:0/2284760706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6634e24640 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e609155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6635625640 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e6091551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6635e26640 1 -- 172.21.15.5:0/2284760706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 msgr2=0x55e609155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6635e26640 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e609155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6635e26640 1 -- 172.21.15.5:0/2284760706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 msgr2=0x55e6091551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:01.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6635e26640 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e6091551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6635e26640 1 -- 172.21.15.5:0/2284760706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e609150710 con 0x55e60914d240 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6635e26640 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e6091501a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f66280092d0 tx=0x7f6628026690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30181590b9ffe1e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6614ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6614ff9640 1 -- 172.21.15.5:0/2284760706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6628017070 con 0x55e60914d240 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6614ff9640 1 -- 172.21.15.5:0/2284760706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8629+0+0 (secure 0 0 0) 0x7f6628013610 con 0x55e60914d240 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6639c62740 1 -- 172.21.15.5:0/2284760706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f662802b070 con 0x55e60914d240 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6614ff9640 1 -- 172.21.15.5:0/2284760706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f662800b8e0 con 0x55e60914d240 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6639c62740 10 client.18162 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.415+0000 7f6639c62740 1 -- 172.21.15.5:0/2284760706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e6091c7f90 con 0x55e60914d240 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.416+0000 7f6614ff9640 1 -- 172.21.15.5:0/2284760706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f662802b070 con 0x55e60914d240 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.416+0000 7f6614ff9640 1 -- 172.21.15.5:0/2284760706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f662801b040 con 0x55e60914d240 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.416+0000 7f6639c62740 10 client.18162 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:01.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.416+0000 7f6639c62740 10 client.18162 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.416+0000 7f6639c62740 20 client.18162 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.416+0000 7f6639c62740 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e6091c85f0 0x55e6091ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.416+0000 7f6639c62740 4 client.18162 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "620b531d", "release": true, "await": true}] 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.416+0000 7f6639c62740 1 -- 172.21.15.5:0/2284760706 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "620b531d", "release": true, "await": true}) -- 0x55e6090af0a0 con 0x55e6091c85f0 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.416+0000 7f6635625640 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e6091c85f0 0x55e6091ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.417+0000 7f6614ff9640 10 client.18162 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.417+0000 7f6635625640 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e6091c85f0 0x55e6091ca9e0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db38418727e1f4b server_cookie=a9bcbe77a608f057 in_seq=0 out_seq=0 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6614ff9640 1 -- 172.21.15.5:0/2284760706 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e6090af0a0 con 0x55e6091c85f0 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6614ff9640 10 client.18162 handle_command_reply: tid=0 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 client.18162 shutdown 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 -- 172.21.15.5:0/2284760706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 msgr2=0x55e6091501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e6091501a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f66280092d0 tx=0x7f6628026690 comp rx=0 tx=0).stop 2024-09-11T22:31:01.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 -- 172.21.15.5:0/2284760706 shutdown_connections 2024-09-11T22:31:01.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e6091c85f0 0x55e6091ca9e0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.489 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:31:01.489 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 40, 2024-09-11T22:31:01.489 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:31:01.489 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 36, 2024-09-11T22:31:01.489 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:31:01.489 INFO:teuthology.orchestra.run.smithi005.stdout: "620b531d": { 2024-09-11T22:31:01.489 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 36, 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.4, 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:31:01.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:31:01.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:31:01.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:31:01.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:31:01.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:31:01.491 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:31:01.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e60914d240 0x55e6091501a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6614ff9640 0 client.18162 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e609142980 0x55e609155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 --2- 172.21.15.5:0/2284760706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e609142060 0x55e6091551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 -- 172.21.15.5:0/2284760706 >> 172.21.15.5:0/2284760706 conn(0x55e60913e5f0 msgr2=0x55e60914c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 -- 172.21.15.5:0/2284760706 shutdown_connections 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 1 -- 172.21.15.5:0/2284760706 wait complete. 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.431+0000 7f6639c62740 20 client.18162 trim_cache size 0 max 16384 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 -- 172.21.15.5:0/1410579431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f66180a2cf0 msgr2=0x7f66180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f66180a2cf0 0x7f66180a51b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f662801d000 tx=0x7f662801a000 comp rx=0 tx=0).stop 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 -- 172.21.15.5:0/1410579431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6630100b20 msgr2=0x7f66301a0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6630100b20 0x7f66301a0650 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f66200097b0 tx=0x7f6620031b10 comp rx=0 tx=0).stop 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 -- 172.21.15.5:0/1410579431 shutdown_connections 2024-09-11T22:31:01.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f663010e500 0x7f663019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f66180a2cf0 0x7f66180a51b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6630101440 0x7f66301a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 --2- 172.21.15.5:0/1410579431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6630100b20 0x7f66301a0650 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 -- 172.21.15.5:0/1410579431 >> 172.21.15.5:0/1410579431 conn(0x7f66300fc7d0 msgr2=0x7f66300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 -- 172.21.15.5:0/1410579431 shutdown_connections 2024-09-11T22:31:01.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:01.432+0000 7f6637889640 1 -- 172.21.15.5:0/1410579431 wait complete. 2024-09-11T22:31:02.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:01 smithi005 ceph-mon[30943]: pgmap v275: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.2 MiB/s wr, 79 op/s 2024-09-11T22:31:02.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:01 smithi159 ceph-mon[35534]: pgmap v275: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.2 MiB/s wr, 79 op/s 2024-09-11T22:31:02.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:01 smithi172 ceph-mon[35461]: pgmap v275: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.2 MiB/s wr, 79 op/s 2024-09-11T22:31:02.256 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '620b531d', total seconds elapsed: 22.8 2024-09-11T22:31:02.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 645677 7.31 MB/sec execute 320 sec latency 18786.993 ms 2024-09-11T22:31:03.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 647612 7.32 MB/sec execute 321 sec latency 18.082 ms 2024-09-11T22:31:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:03 smithi005 ceph-mon[30943]: pgmap v276: 97 pgs: 97 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-09-11T22:31:04.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:03 smithi159 ceph-mon[35534]: pgmap v276: 97 pgs: 97 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-09-11T22:31:04.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:03 smithi172 ceph-mon[35461]: pgmap v276: 97 pgs: 97 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-09-11T22:31:04.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650120 7.35 MB/sec execute 322 sec latency 14.021 ms 2024-09-11T22:31:05.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 651078 7.33 MB/sec execute 323 sec latency 25.016 ms 2024-09-11T22:31:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.704+0000 7f0fffd4b640 1 -- 172.21.15.5:0/1924842396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81095d0 msgr2=0x7f0ff8109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.704+0000 7f0fffd4b640 1 --2- 172.21.15.5:0/1924842396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81095d0 0x7f0ff8109a10 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009900 tx=0x7f0fe802a410 comp rx=0 tx=0).stop 2024-09-11T22:31:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.705+0000 7f0fffd4b640 1 -- 172.21.15.5:0/1924842396 shutdown_connections 2024-09-11T22:31:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.705+0000 7f0fffd4b640 1 --2- 172.21.15.5:0/1924842396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff811b140 0x7f0ff811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.705+0000 7f0fffd4b640 1 --2- 172.21.15.5:0/1924842396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81095d0 0x7f0ff8109a10 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.705+0000 7f0fffd4b640 1 --2- 172.21.15.5:0/1924842396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff81091f0 0x7f0ff8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.705+0000 7f0fffd4b640 1 -- 172.21.15.5:0/1924842396 >> 172.21.15.5:0/1924842396 conn(0x7f0ff806ce00 msgr2=0x7f0ff806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.705+0000 7f0fffd4b640 1 -- 172.21.15.5:0/1924842396 shutdown_connections 2024-09-11T22:31:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.705+0000 7f0fffd4b640 1 -- 172.21.15.5:0/1924842396 wait complete. 2024-09-11T22:31:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.706+0000 7f0fffd4b640 1 Processor -- start 2024-09-11T22:31:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.706+0000 7f0fffd4b640 1 -- start start 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0fffd4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff81091f0 0x7f0ff8118660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0fffd4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81095d0 0x7f0ff8118ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0fffd4b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff811b140 0x7f0ff8111870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0fffd4b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff8120cf0 con 0x7f0ff81095d0 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0fffd4b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff8120b70 con 0x7f0ff811b140 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0fffd4b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff8120e70 con 0x7f0ff81091f0 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0ffd2e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81095d0 0x7f0ff8118ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0ffd2e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81095d0 0x7f0ff8118ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45236/0 (socket says 172.21.15.5:45236) 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0ffd2e6640 1 -- 172.21.15.5:0/581125834 learned_addr learned my addr 172.21.15.5:0/581125834 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0ffd2e6640 1 -- 172.21.15.5:0/581125834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff81091f0 msgr2=0x7f0ff8118660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:31:05.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff81091f0 0x7f0ff8118660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0ffd2e6640 1 -- 172.21.15.5:0/581125834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff811b140 msgr2=0x7f0ff8111870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:31:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff811b140 0x7f0ff8111870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.707+0000 7f0ffd2e6640 1 -- 172.21.15.5:0/581125834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff8112130 con 0x7f0ff81095d0 2024-09-11T22:31:05.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.708+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81095d0 0x7f0ff8118ba0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8006010 tx=0x7f0fe8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.708+0000 7f0feeffd640 1 -- 172.21.15.5:0/581125834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe803f070 con 0x7f0ff81095d0 2024-09-11T22:31:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.708+0000 7f0fffd4b640 1 -- 172.21.15.5:0/581125834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff81123c0 con 0x7f0ff81095d0 2024-09-11T22:31:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.708+0000 7f0fffd4b640 1 -- 172.21.15.5:0/581125834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff8116360 con 0x7f0ff81095d0 2024-09-11T22:31:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.709+0000 7f0feeffd640 1 -- 172.21.15.5:0/581125834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe8031c90 con 0x7f0ff81095d0 2024-09-11T22:31:05.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.709+0000 7f0feeffd640 1 -- 172.21.15.5:0/581125834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8035030 con 0x7f0ff81095d0 2024-09-11T22:31:05.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.712+0000 7f1002bc3740 1 Processor -- start 2024-09-11T22:31:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.714+0000 7f1002bc3740 1 -- start start 2024-09-11T22:31:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.714+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc6ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.714+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 0x55cf4cc78770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.714+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc7b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.714+0000 7f1002bc3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cf4cbd22f0 con 0x55cf4cc78cb0 2024-09-11T22:31:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.714+0000 7f1002bc3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf4cb7a270 con 0x55cf4cc6e410 2024-09-11T22:31:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.714+0000 7f1002bc3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cf4cbce2b0 con 0x55cf4cc6daf0 2024-09-11T22:31:05.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.715+0000 7f0ffe2e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc6ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.715+0000 7f0ffdae7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 0x55cf4cc78770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.715+0000 7f0ffdae7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 0x55cf4cc78770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39302/0 (socket says 172.21.15.5:39302) 2024-09-11T22:31:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.715+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4041900882 learned_addr learned my addr 172.21.15.5:0/4041900882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:05.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.715+0000 7f0feeffd640 1 -- 172.21.15.5:0/581125834 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0fe8042070 con 0x7f0ff81095d0 2024-09-11T22:31:05.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.717+0000 7f0feeffd640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fe40a2f80 0x7f0fe40a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.717+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/4041900882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc7b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.717+0000 7f0feeffd640 1 -- 172.21.15.5:0/581125834 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8673+0+0 (secure 0 0 0) 0x7f0fe80efe90 con 0x7f0ff81095d0 2024-09-11T22:31:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.717+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4041900882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 msgr2=0x55cf4cc6ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.717+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/4041900882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc6ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.717+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4041900882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 msgr2=0x55cf4cc7b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.717+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/4041900882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc7b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.717+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4041900882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf4cb79340 con 0x55cf4cc6e410 2024-09-11T22:31:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.718+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/4041900882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc7b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:05.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.718+0000 7f0ffe2e8640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fe40a2f80 0x7f0fe40a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.719+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/4041900882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 0x55cf4cc78770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400ba80 tx=0x7f0ff400be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1d913b43dd8776b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.719+0000 7f0fe3fff640 1 -- 172.21.15.5:0/4041900882 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4016070 con 0x55cf4cc6e410 2024-09-11T22:31:05.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.720+0000 7f0fe3fff640 1 -- 172.21.15.5:0/4041900882 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff4005500 con 0x55cf4cc6e410 2024-09-11T22:31:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.720+0000 7f0fe3fff640 1 -- 172.21.15.5:0/4041900882 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4004e00 con 0x55cf4cc6e410 2024-09-11T22:31:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.720+0000 7f1002bc3740 1 -- 172.21.15.5:0/4041900882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 msgr2=0x55cf4cc78770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.720+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4041900882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 0x55cf4cc78770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400ba80 tx=0x7f0ff400be80 comp rx=0 tx=0).stop 2024-09-11T22:31:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.720+0000 7f0ffe2e8640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fe40a2f80 0x7f0fe40a5440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005fd0 tx=0x7f0ff0004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.720+0000 7f1002bc3740 1 -- 172.21.15.5:0/4041900882 shutdown_connections 2024-09-11T22:31:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.720+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4041900882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc7b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.720+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4041900882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 0x55cf4cc78770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.720+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4041900882 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc6ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.721+0000 7f1002bc3740 1 -- 172.21.15.5:0/4041900882 >> 172.21.15.5:0/4041900882 conn(0x55cf4cc6a060 msgr2=0x55cf4cc6a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.721+0000 7f1002bc3740 1 -- 172.21.15.5:0/4041900882 shutdown_connections 2024-09-11T22:31:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.721+0000 7f1002bc3740 1 -- 172.21.15.5:0/4041900882 wait complete. 2024-09-11T22:31:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.722+0000 7f1002bc3740 1 -- start start 2024-09-11T22:31:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.722+0000 7f1002bc3740 1 Processor -- start 2024-09-11T22:31:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.722+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.722+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 0x55cf4cc760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.722+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc76600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.722+0000 7f1002bc3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cf4cc7b880 con 0x55cf4cc78cb0 2024-09-11T22:31:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.722+0000 7f1002bc3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf4cb5b290 con 0x55cf4cc6e410 2024-09-11T22:31:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.722+0000 7f1002bc3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cf4cc7ba00 con 0x55cf4cc6daf0 2024-09-11T22:31:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.722+0000 7f1002bc3740 10 client.0 _ll_register_callbacks cb 0x55cf4cb8c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffdae7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffdae7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc75b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:38806/0 (socket says 172.21.15.5:38806) 2024-09-11T22:31:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4106719823 learned_addr learned my addr 172.21.15.5:0/4106719823 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffe2e8640 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc76600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 0x55cf4cc760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4106719823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 msgr2=0x55cf4cc760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 0x55cf4cc760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4106719823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 msgr2=0x55cf4cc76600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc76600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4106719823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf4c8e5210 con 0x55cf4cc6daf0 2024-09-11T22:31:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffe2e8640 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc76600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc75b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005d00 tx=0x7f0ff4004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4148a4c70b7d51c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.723+0000 7f0fe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:31:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.724+0000 7f0fe17fa640 1 -- 172.21.15.5:0/4106719823 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4016070 con 0x55cf4cc6daf0 2024-09-11T22:31:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.724+0000 7f1002bc3740 1 -- 172.21.15.5:0/4106719823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff4005500 con 0x55cf4cc6daf0 2024-09-11T22:31:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.724+0000 7f0fe17fa640 1 -- 172.21.15.5:0/4106719823 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8673+0+0 (secure 0 0 0) 0x7f0ff400d980 con 0x55cf4cc6daf0 2024-09-11T22:31:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.724+0000 7f0fe17fa640 1 -- 172.21.15.5:0/4106719823 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0ff400b650 con 0x55cf4cc6daf0 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.724+0000 7f1002bc3740 10 client.27355 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.724+0000 7f1002bc3740 1 -- 172.21.15.5:0/4106719823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cf4c9b1620 con 0x55cf4cc6daf0 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.725+0000 7f0fe17fa640 1 -- 172.21.15.5:0/4106719823 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff4005500 con 0x55cf4cc6daf0 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.726+0000 7f0fe17fa640 1 -- 172.21.15.5:0/4106719823 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f0ff40037f0 con 0x55cf4cc6daf0 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.726+0000 7f1002bc3740 10 client.27355 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.726+0000 7f1002bc3740 10 client.27355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.726+0000 7f1002bc3740 20 client.27355 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.726+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cf4cc88f60 0x55cf4cc8b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.726+0000 7f1002bc3740 4 client.27355 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.726+0000 7f1002bc3740 1 -- 172.21.15.5:0/4106719823 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf4cbdfa20 con 0x55cf4cc88f60 2024-09-11T22:31:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.726+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cf4cc88f60 0x55cf4cc8b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.727+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cf4cc88f60 0x55cf4cc8b3e0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c181fc6d748601 server_cookie=6b821097a21808d0 in_seq=0 out_seq=0 2024-09-11T22:31:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.727+0000 7f0fe17fa640 10 client.27355 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f0fe17fa640 1 -- 172.21.15.5:0/4106719823 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf4cbdfa20 con 0x55cf4cc88f60 2024-09-11T22:31:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f0fe17fa640 10 client.27355 handle_command_reply: tid=0 2024-09-11T22:31:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f1002bc3740 1 client.27355 shutdown 2024-09-11T22:31:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f1002bc3740 1 -- 172.21.15.5:0/4106719823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 msgr2=0x55cf4cc75b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc75b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005d00 tx=0x7f0ff4004db0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f1002bc3740 1 -- 172.21.15.5:0/4106719823 shutdown_connections 2024-09-11T22:31:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cf4cc88f60 0x55cf4cc8b3e0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc76600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6e410 0x55cf4cc760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4106719823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc75b80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f1002bc3740 1 -- 172.21.15.5:0/4106719823 >> 172.21.15.5:0/4106719823 conn(0x55cf4cc6a060 msgr2=0x55cf4cc691c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f0fe17fa640 0 client.27355 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.729+0000 7f1002bc3740 1 -- 172.21.15.5:0/4106719823 shutdown_connections 2024-09-11T22:31:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.730+0000 7f1002bc3740 1 -- 172.21.15.5:0/4106719823 wait complete. 2024-09-11T22:31:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.730+0000 7f1002bc3740 20 client.27355 trim_cache size 0 max 16384 2024-09-11T22:31:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.758+0000 7f1002bc3740 1 Processor -- start 2024-09-11T22:31:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.758+0000 7f1002bc3740 1 -- start start 2024-09-11T22:31:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.758+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc70560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.758+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc76400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.758+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc90cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.758+0000 7f1002bc3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc00eb30 con 0x55cf4cc78cb0 2024-09-11T22:31:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.758+0000 7f1002bc3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc010670 con 0x55cf4cc6daf0 2024-09-11T22:31:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.758+0000 7f1002bc3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc00f200 con 0x55cf4cc6e410 2024-09-11T22:31:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.759+0000 7f0ffd2e6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc76400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.759+0000 7f0ffdae7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc70560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.759+0000 7f0ffd2e6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc76400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:38820/0 (socket says 172.21.15.5:38820) 2024-09-11T22:31:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.759+0000 7f0ffd2e6640 1 -- 172.21.15.5:0/4150999291 learned_addr learned my addr 172.21.15.5:0/4150999291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.759+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4150999291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 msgr2=0x55cf4cc76400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.759+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/4150999291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc76400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.759+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4150999291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 msgr2=0x55cf4cc90cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.759+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/4150999291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc90cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.759+0000 7f0ffdae7640 1 -- 172.21.15.5:0/4150999291 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf4cc6e240 con 0x55cf4cc6daf0 2024-09-11T22:31:05.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.759+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/4150999291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc70560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400b9a0 tx=0x7f0ff4005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e074a3421145af73 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.760+0000 7f0fe37fe640 1 -- 172.21.15.5:0/4150999291 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4016040 con 0x55cf4cc6daf0 2024-09-11T22:31:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.760+0000 7f0fe37fe640 1 -- 172.21.15.5:0/4150999291 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff4005500 con 0x55cf4cc6daf0 2024-09-11T22:31:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.760+0000 7f0fe37fe640 1 -- 172.21.15.5:0/4150999291 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40035f0 con 0x55cf4cc6daf0 2024-09-11T22:31:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.761+0000 7f1002bc3740 1 -- 172.21.15.5:0/4150999291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 msgr2=0x55cf4cc70560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.761+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4150999291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc70560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0ff400b9a0 tx=0x7f0ff4005970 comp rx=0 tx=0).stop 2024-09-11T22:31:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.761+0000 7f1002bc3740 1 -- 172.21.15.5:0/4150999291 shutdown_connections 2024-09-11T22:31:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.761+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4150999291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc90cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.761+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4150999291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc76400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.761+0000 7f1002bc3740 1 --2- 172.21.15.5:0/4150999291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc70560 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.761+0000 7f1002bc3740 1 -- 172.21.15.5:0/4150999291 >> 172.21.15.5:0/4150999291 conn(0x55cf4cc6a060 msgr2=0x55cf4cc70e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.761+0000 7f1002bc3740 1 -- 172.21.15.5:0/4150999291 shutdown_connections 2024-09-11T22:31:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.761+0000 7f1002bc3740 1 -- 172.21.15.5:0/4150999291 wait complete. 2024-09-11T22:31:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.762+0000 7f1002bc3740 1 -- start start 2024-09-11T22:31:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.762+0000 7f1002bc3740 1 Processor -- start 2024-09-11T22:31:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.763+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc81430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.763+0000 7f1002bc3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc7bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.764+0000 7f0ffdae7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc81430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.764+0000 7f0ffd2e6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc7bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.764+0000 7f0ffd2e6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc7bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:38824/0 (socket says 172.21.15.5:38824) 2024-09-11T22:31:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.764+0000 7f0ffd2e6640 1 -- 172.21.15.5:0/898018050 learned_addr learned my addr 172.21.15.5:0/898018050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.764+0000 7f1002bc3740 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc7c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.764+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc00ffa0 con 0x55cf4cc78cb0 2024-09-11T22:31:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.764+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc00f8d0 con 0x55cf4cc6daf0 2024-09-11T22:31:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.764+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc00cb30 con 0x55cf4cc6e410 2024-09-11T22:31:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.764+0000 7f1002bc3740 10 client.0 _ll_register_callbacks cb 0x7f0ff400b8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.765+0000 7f0ffe2e8640 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc7c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.765+0000 7f0ffd2e6640 1 -- 172.21.15.5:0/898018050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 msgr2=0x55cf4cc81430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.765+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc81430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.765+0000 7f0ffd2e6640 1 -- 172.21.15.5:0/898018050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 msgr2=0x55cf4cc7c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.765+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc7c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.765+0000 7f0ffd2e6640 1 -- 172.21.15.5:0/898018050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf4cc7ca60 con 0x55cf4cc6e410 2024-09-11T22:31:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.766+0000 7f0ffd2e6640 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc7bc80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0fe803b970 tx=0x7f0fe8040b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8437ee15d70225e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.766+0000 7f0fe17fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:31:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.766+0000 7f0fe17fa640 1 -- 172.21.15.5:0/898018050 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe80cbe80 con 0x55cf4cc6e410 2024-09-11T22:31:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.766+0000 7f0fe17fa640 1 -- 172.21.15.5:0/898018050 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8673+0+0 (secure 0 0 0) 0x7f0fe80b8920 con 0x55cf4cc6e410 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.766+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff4005500 con 0x55cf4cc6e410 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.766+0000 7f0fe17fa640 1 -- 172.21.15.5:0/898018050 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0fe80bc2b0 con 0x55cf4cc6e410 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.766+0000 7f1002bc3740 10 client.27361 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.766+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cf4ccf2d90 con 0x55cf4cc6e410 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.767+0000 7f0fe17fa640 1 -- 172.21.15.5:0/898018050 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fe80c0660 con 0x55cf4cc6e410 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.767+0000 7f0fe17fa640 1 -- 172.21.15.5:0/898018050 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f0fe804c350 con 0x55cf4cc6e410 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.767+0000 7f1002bc3740 10 client.27361 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.767+0000 7f1002bc3740 10 client.27361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.767+0000 7f1002bc3740 20 client.27361 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.767+0000 7f1002bc3740 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cf4ccf3470 0x55cf4ccf5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.767+0000 7f1002bc3740 4 client.27361 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:31:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.767+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0ff4005500 con 0x55cf4ccf3470 2024-09-11T22:31:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.767+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cf4ccf3470 0x55cf4ccf5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.768+0000 7f0fe17fa640 10 client.27361 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:05.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.768+0000 7f0ffdae7640 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cf4ccf3470 0x55cf4ccf5890 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524881ea6371e849 server_cookie=60ab5eeea31f27d8 in_seq=0 out_seq=0 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f0fe17fa640 1 -- 172.21.15.5:0/898018050 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0ff4005500 con 0x55cf4ccf3470 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f0fe17fa640 10 client.27361 handle_command_reply: tid=0 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f1002bc3740 1 client.27361 shutdown 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 msgr2=0x55cf4cc7bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f1002bc3740 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc7bc80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0fe803b970 tx=0x7f0fe8040b60 comp rx=0 tx=0).stop 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 shutdown_connections 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f1002bc3740 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cf4ccf3470 0x55cf4ccf5890 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f1002bc3740 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf4cc78cb0 0x55cf4cc7c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f1002bc3740 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cf4cc6e410 0x55cf4cc7bc80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f1002bc3740 1 --2- 172.21.15.5:0/898018050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf4cc6daf0 0x55cf4cc81430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 >> 172.21.15.5:0/898018050 conn(0x55cf4cc6a060 msgr2=0x55cf4cc689a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.840+0000 7f0fe17fa640 0 client.27361 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.841+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 shutdown_connections 2024-09-11T22:31:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.841+0000 7f1002bc3740 1 -- 172.21.15.5:0/898018050 wait complete. 2024-09-11T22:31:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.841+0000 7f1002bc3740 20 client.27361 trim_cache size 0 max 16384 2024-09-11T22:31:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:31:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:31:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:31:05.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:31:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.841+0000 7f0fffd4b640 1 -- 172.21.15.5:0/581125834 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fe40a2f80 msgr2=0x7f0fe40a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.841+0000 7f0fffd4b640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fe40a2f80 0x7f0fe40a5440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005fd0 tx=0x7f0ff0004ab0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.841+0000 7f0fffd4b640 1 -- 172.21.15.5:0/581125834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81095d0 msgr2=0x7f0ff8118ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.841+0000 7f0fffd4b640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81095d0 0x7f0ff8118ba0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8006010 tx=0x7f0fe8003b90 comp rx=0 tx=0).stop 2024-09-11T22:31:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.842+0000 7f0fffd4b640 1 -- 172.21.15.5:0/581125834 shutdown_connections 2024-09-11T22:31:05.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.842+0000 7f0fffd4b640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0fe40a2f80 0x7f0fe40a5440 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.842+0000 7f0fffd4b640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ff811b140 0x7f0ff8111870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.842+0000 7f0fffd4b640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81095d0 0x7f0ff8118ba0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.842+0000 7f0fffd4b640 1 --2- 172.21.15.5:0/581125834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff81091f0 0x7f0ff8118660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.842+0000 7f0fffd4b640 1 -- 172.21.15.5:0/581125834 >> 172.21.15.5:0/581125834 conn(0x7f0ff806ce00 msgr2=0x7f0ff811b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.842+0000 7f0fffd4b640 1 -- 172.21.15.5:0/581125834 shutdown_connections 2024-09-11T22:31:05.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:05.842+0000 7f0fffd4b640 1 -- 172.21.15.5:0/581125834 wait complete. 2024-09-11T22:31:05.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:31:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:05 smithi005 ceph-mon[30943]: pgmap v277: 97 pgs: 97 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 664 KiB/s wr, 91 op/s 2024-09-11T22:31:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:05 smithi005 ceph-mon[30943]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T22:31:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:05 smithi159 ceph-mon[35534]: pgmap v277: 97 pgs: 97 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 664 KiB/s wr, 91 op/s 2024-09-11T22:31:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:05 smithi159 ceph-mon[35534]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T22:31:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:05 smithi172 ceph-mon[35461]: pgmap v277: 97 pgs: 97 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 664 KiB/s wr, 91 op/s 2024-09-11T22:31:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:05 smithi172 ceph-mon[35461]: osdmap e94: 12 total, 12 up, 12 in 2024-09-11T22:31:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:06.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 652858 7.34 MB/sec execute 324 sec latency 71.891 ms 2024-09-11T22:31:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.663+0000 7f1c8e83e640 1 -- 172.21.15.5:0/479978220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c88102ef0 msgr2=0x7f1c8810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.663+0000 7f1c8e83e640 1 --2- 172.21.15.5:0/479978220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c88102ef0 0x7f1c8810da40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009ca0 tx=0x7f1c8002a950 comp rx=0 tx=0).stop 2024-09-11T22:31:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.664+0000 7f1c8e83e640 1 -- 172.21.15.5:0/479978220 shutdown_connections 2024-09-11T22:31:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.664+0000 7f1c8e83e640 1 --2- 172.21.15.5:0/479978220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c88102ef0 0x7f1c8810da40 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.664+0000 7f1c8e83e640 1 --2- 172.21.15.5:0/479978220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c88102570 0x7f1c881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.664+0000 7f1c8e83e640 1 --2- 172.21.15.5:0/479978220 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c88100e20 0x7f1c88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.664+0000 7f1c8e83e640 1 -- 172.21.15.5:0/479978220 >> 172.21.15.5:0/479978220 conn(0x7f1c880fc7d0 msgr2=0x7f1c880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.664+0000 7f1c8e83e640 1 -- 172.21.15.5:0/479978220 shutdown_connections 2024-09-11T22:31:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.664+0000 7f1c8e83e640 1 -- 172.21.15.5:0/479978220 wait complete. 2024-09-11T22:31:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.664+0000 7f1c8e83e640 1 Processor -- start 2024-09-11T22:31:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.664+0000 7f1c8e83e640 1 -- start start 2024-09-11T22:31:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c8e83e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c88100e20 0x7f1c8819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c8e83e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c88102570 0x7f1c8819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c8e83e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c88102ef0 0x7f1c8819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c8e83e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c88110fd0 con 0x7f1c88100e20 2024-09-11T22:31:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c8e83e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1c88110e50 con 0x7f1c88102570 2024-09-11T22:31:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c8e83e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c88111150 con 0x7f1c88102ef0 2024-09-11T22:31:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c88100e20 0x7f1c8819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c88100e20 0x7f1c8819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45294/0 (socket says 172.21.15.5:45294) 2024-09-11T22:31:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c87fff640 1 -- 172.21.15.5:0/2623958551 learned_addr learned my addr 172.21.15.5:0/2623958551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c877fe640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c88102570 0x7f1c8819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c8cddb640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c88102ef0 0x7f1c8819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c87fff640 1 -- 172.21.15.5:0/2623958551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c88102ef0 msgr2=0x7f1c8819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c87fff640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c88102ef0 0x7f1c8819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c87fff640 1 -- 172.21.15.5:0/2623958551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c88102570 msgr2=0x7f1c8819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c87fff640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c88102570 0x7f1c8819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c87fff640 1 -- 172.21.15.5:0/2623958551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8819f600 con 0x7f1c88100e20 2024-09-11T22:31:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c87fff640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c88100e20 0x7f1c8819a650 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800e9f0 tx=0x7f1c7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.665+0000 7f1c857fa640 1 -- 172.21.15.5:0/2623958551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7800cd80 con 0x7f1c88100e20 2024-09-11T22:31:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.666+0000 7f1c8e83e640 1 -- 172.21.15.5:0/2623958551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8819f8f0 con 0x7f1c88100e20 2024-09-11T22:31:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.666+0000 7f1c857fa640 1 -- 172.21.15.5:0/2623958551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c78010910 con 0x7f1c88100e20 2024-09-11T22:31:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.666+0000 7f1c857fa640 1 -- 172.21.15.5:0/2623958551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c780188f0 con 0x7f1c88100e20 2024-09-11T22:31:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.666+0000 7f1c8e83e640 1 -- 172.21.15.5:0/2623958551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c881a71d0 con 0x7f1c88100e20 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.668+0000 7f1c857fa640 1 -- 172.21.15.5:0/2623958551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1c78010ac0 con 0x7f1c88100e20 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.668+0000 7f1c90c17740 1 Processor -- start 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.668+0000 7f1c90c17740 1 -- start start 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fd9f8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f8dc0 0x5617fda03140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fda03680 0x5617fda05a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c90c17740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617fd95e2e0 con 0x5617fd9f8dc0 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c90c17740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5617fd905ce0 con 0x5617fd9f84a0 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c90c17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5617fd95a2a0 con 0x5617fda03680 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c8cddb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fd9f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c8cddb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fd9f8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39382/0 (socket says 172.21.15.5:39382) 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c8cddb640 1 -- 172.21.15.5:0/2897691016 learned_addr learned my addr 172.21.15.5:0/2897691016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c857fa640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1c700a2cf0 0x7f1c700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c87fff640 1 --2- 172.21.15.5:0/2897691016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f8dc0 0x5617fda03140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c877fe640 1 --2- 172.21.15.5:0/2897691016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fda03680 0x5617fda05a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c8cddb640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1c700a2cf0 0x7f1c700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c857fa640 1 -- 172.21.15.5:0/2623958551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8673+0+0 (secure 0 0 0) 0x7f1c780cf930 con 0x7f1c88100e20 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c877fe640 1 -- 172.21.15.5:0/2897691016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 msgr2=0x5617fd9f8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c877fe640 1 --2- 172.21.15.5:0/2897691016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fd9f8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c877fe640 1 -- 172.21.15.5:0/2897691016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f8dc0 msgr2=0x5617fda03140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c877fe640 1 --2- 172.21.15.5:0/2897691016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f8dc0 0x5617fda03140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.669+0000 7f1c877fe640 1 -- 172.21.15.5:0/2897691016 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617fd8ea440 con 0x5617fda03680 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.670+0000 7f1c877fe640 1 --2- 172.21.15.5:0/2897691016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fda03680 0x5617fda05a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009bf0 tx=0x7f1c7c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d476bda62d75287 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.670+0000 7f1c6e7fc640 1 -- 172.21.15.5:0/2897691016 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c009da0 con 0x5617fda03680 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.670+0000 7f1c6e7fc640 1 -- 172.21.15.5:0/2897691016 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7c005020 con 0x5617fda03680 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.670+0000 7f1c8cddb640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1c700a2cf0 0x7f1c700a51b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009430 tx=0x7f1c8002a950 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.670+0000 7f1c6e7fc640 1 -- 172.21.15.5:0/2897691016 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c004bd0 con 0x5617fda03680 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 -- 172.21.15.5:0/2897691016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fda03680 msgr2=0x5617fda05a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 --2- 172.21.15.5:0/2897691016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fda03680 0x5617fda05a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009bf0 tx=0x7f1c7c011810 comp rx=0 tx=0).stop 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 -- 172.21.15.5:0/2897691016 shutdown_connections 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 --2- 172.21.15.5:0/2897691016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fda03680 0x5617fda05a70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 --2- 172.21.15.5:0/2897691016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f8dc0 0x5617fda03140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 --2- 172.21.15.5:0/2897691016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fd9f8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 -- 172.21.15.5:0/2897691016 >> 172.21.15.5:0/2897691016 conn(0x5617fd9f4a30 msgr2=0x5617fd9f4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 -- 172.21.15.5:0/2897691016 shutdown_connections 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 -- 172.21.15.5:0/2897691016 wait complete. 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 -- start start 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.671+0000 7f1c90c17740 1 Processor -- start 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fda02f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f8dc0 0x5617fda001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fda03680 0x5617fda00730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c90c17740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617fda06250 con 0x5617fda03680 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c90c17740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5617fd919050 con 0x5617fd9f84a0 2024-09-11T22:31:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c90c17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5617fda063d0 con 0x5617fd9f8dc0 2024-09-11T22:31:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c90c17740 10 client.0 _ll_register_callbacks cb 0x5617fd959160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fda02f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c8cddb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fda03680 0x5617fda00730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f8dc0 0x5617fda001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fda02f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39392/0 (socket says 172.21.15.5:39392) 2024-09-11T22:31:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c87fff640 1 -- 172.21.15.5:0/976406352 learned_addr learned my addr 172.21.15.5:0/976406352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c8cddb640 1 -- 172.21.15.5:0/976406352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f8dc0 msgr2=0x5617fda001f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c8cddb640 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f8dc0 0x5617fda001f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c8cddb640 1 -- 172.21.15.5:0/976406352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 msgr2=0x5617fda02f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c8cddb640 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fda02f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c8cddb640 1 -- 172.21.15.5:0/976406352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617fd6cf3c0 con 0x5617fda03680 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c87fff640 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fda02f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c8cddb640 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fda03680 0x5617fda00730 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1c80006010 tx=0x7f1c800095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=299b26bc6bac55f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.672+0000 7f1c677fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c677fe640 1 -- 172.21.15.5:0/976406352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80034040 con 0x5617fda03680 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c677fe640 1 -- 172.21.15.5:0/976406352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8673+0+0 (secure 0 0 0) 0x7f1c80004450 con 0x5617fda03680 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c90c17740 1 -- 172.21.15.5:0/976406352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7c005020 con 0x5617fda03680 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c677fe640 1 -- 172.21.15.5:0/976406352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1c80059050 con 0x5617fda03680 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c90c17740 10 client.18189 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c90c17740 1 -- 172.21.15.5:0/976406352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5617fda11ba0 con 0x5617fda03680 2024-09-11T22:31:06.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c677fe640 1 -- 172.21.15.5:0/976406352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7c005020 con 0x5617fda03680 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c677fe640 1 -- 172.21.15.5:0/976406352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f1c80049040 con 0x5617fda03680 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c90c17740 10 client.18189 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c90c17740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c90c17740 20 client.18189 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c90c17740 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5617fda13060 0x5617fda154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c90c17740 4 client.18189 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.673+0000 7f1c90c17740 1 -- 172.21.15.5:0/976406352 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617fd96b850 con 0x5617fda13060 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.674+0000 7f1c87fff640 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5617fda13060 0x5617fda154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.674+0000 7f1c677fe640 10 client.18189 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.674+0000 7f1c87fff640 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5617fda13060 0x5617fda154d0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=675afffef14f2ef9 server_cookie=5c5fa5072519f005 in_seq=0 out_seq=0 2024-09-11T22:31:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c677fe640 1 -- 172.21.15.5:0/976406352 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617fd96b850 con 0x5617fda13060 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c677fe640 10 client.18189 handle_command_reply: tid=0 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 client.18189 shutdown 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 -- 172.21.15.5:0/976406352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fda03680 msgr2=0x5617fda00730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fda03680 0x5617fda00730 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f1c80006010 tx=0x7f1c800095c0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 -- 172.21.15.5:0/976406352 shutdown_connections 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5617fda13060 0x5617fda154d0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fda03680 0x5617fda00730 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f8dc0 0x5617fda001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 --2- 172.21.15.5:0/976406352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f84a0 0x5617fda02f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 -- 172.21.15.5:0/976406352 >> 172.21.15.5:0/976406352 conn(0x5617fd9f4a30 msgr2=0x5617fd9fa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c677fe640 0 client.18189 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 -- 172.21.15.5:0/976406352 shutdown_connections 2024-09-11T22:31:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 1 -- 172.21.15.5:0/976406352 wait complete. 2024-09-11T22:31:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.675+0000 7f1c90c17740 20 client.18189 trim_cache size 0 max 16384 2024-09-11T22:31:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.694+0000 7f1c90c17740 1 Processor -- start 2024-09-11T22:31:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c90c17740 1 -- start start 2024-09-11T22:31:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f4a30 0x5617fda0f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda001b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f8dc0 0x5617fd9fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c90c17740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881afd10 con 0x5617fd9f84a0 2024-09-11T22:31:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c90c17740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881b1850 con 0x5617fd9f4a30 2024-09-11T22:31:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c90c17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881b03e0 con 0x5617fd9f8dc0 2024-09-11T22:31:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c87fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f4a30 0x5617fda0f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45310/0 (socket says 172.21.15.5:45310) 2024-09-11T22:31:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c877fe640 1 -- 172.21.15.5:0/1462864996 learned_addr learned my addr 172.21.15.5:0/1462864996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.695+0000 7f1c8cddb640 1 --2- 172.21.15.5:0/1462864996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f8dc0 0x5617fd9fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.696+0000 7f1c87fff640 1 -- 172.21.15.5:0/1462864996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f8dc0 msgr2=0x5617fd9fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.696+0000 7f1c87fff640 1 --2- 172.21.15.5:0/1462864996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f8dc0 0x5617fd9fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.696+0000 7f1c87fff640 1 -- 172.21.15.5:0/1462864996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 msgr2=0x5617fda001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.696+0000 7f1c87fff640 1 --2- 172.21.15.5:0/1462864996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.696+0000 7f1c87fff640 1 -- 172.21.15.5:0/1462864996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617fd9ff1a0 con 0x5617fd9f4a30 2024-09-11T22:31:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.696+0000 7f1c87fff640 1 --2- 172.21.15.5:0/1462864996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f4a30 0x5617fda0f4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c78098210 tx=0x7f1c7806eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdc3b911824b49fe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.696+0000 7f1c6dffb640 1 -- 172.21.15.5:0/1462864996 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c780a4a20 con 0x5617fd9f4a30 2024-09-11T22:31:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.696+0000 7f1c6dffb640 1 -- 172.21.15.5:0/1462864996 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7809a0b0 con 0x5617fd9f4a30 2024-09-11T22:31:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.696+0000 7f1c6dffb640 1 -- 172.21.15.5:0/1462864996 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78097d60 con 0x5617fd9f4a30 2024-09-11T22:31:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.697+0000 7f1c90c17740 1 -- 172.21.15.5:0/1462864996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f4a30 msgr2=0x5617fda0f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.697+0000 7f1c90c17740 1 --2- 172.21.15.5:0/1462864996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f4a30 0x5617fda0f4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1c78098210 tx=0x7f1c7806eee0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.697+0000 7f1c90c17740 1 -- 172.21.15.5:0/1462864996 shutdown_connections 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.697+0000 7f1c90c17740 1 --2- 172.21.15.5:0/1462864996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f8dc0 0x5617fd9fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.697+0000 7f1c90c17740 1 --2- 172.21.15.5:0/1462864996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.697+0000 7f1c90c17740 1 --2- 172.21.15.5:0/1462864996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f4a30 0x5617fda0f4b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.697+0000 7f1c90c17740 1 -- 172.21.15.5:0/1462864996 >> 172.21.15.5:0/1462864996 conn(0x5617fda00890 msgr2=0x5617fd9fb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.697+0000 7f1c90c17740 1 -- 172.21.15.5:0/1462864996 shutdown_connections 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.697+0000 7f1c90c17740 1 -- 172.21.15.5:0/1462864996 wait complete. 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.697+0000 7f1c90c17740 1 -- start start 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c90c17740 1 Processor -- start 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f4a30 0x5617fda0b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda0be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c90c17740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f8dc0 0x5617fda06d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c90c17740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881b1180 con 0x5617fd9f84a0 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c90c17740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881b0ab0 con 0x5617fd9f8dc0 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c90c17740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1c881add10 con 0x5617fd9f4a30 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c90c17740 10 client.0 _ll_register_callbacks cb 0x7f1c78005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda0be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c8cddb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f8dc0 0x5617fda06d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c8cddb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f8dc0 0x5617fda06d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39402/0 (socket says 172.21.15.5:39402) 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c8cddb640 1 -- 172.21.15.5:0/2880159409 learned_addr learned my addr 172.21.15.5:0/2880159409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c877fe640 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda0be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45314/0 (socket says 172.21.15.5:45314) 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.698+0000 7f1c87fff640 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f4a30 0x5617fda0b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c877fe640 1 -- 172.21.15.5:0/2880159409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f4a30 msgr2=0x5617fda0b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c877fe640 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f4a30 0x5617fda0b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c877fe640 1 -- 172.21.15.5:0/2880159409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f8dc0 msgr2=0x5617fda06d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c877fe640 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f8dc0 0x5617fda06d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c877fe640 1 -- 172.21.15.5:0/2880159409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617fda075a0 con 0x5617fd9f84a0 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c8cddb640 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f8dc0 0x5617fda06d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c87fff640 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f4a30 0x5617fda0b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c877fe640 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda0be80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c014040 tx=0x7f1c7c005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c77207f301c4ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c677fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c677fe640 1 -- 172.21.15.5:0/2880159409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c009e60 con 0x5617fd9f84a0 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c677fe640 1 -- 172.21.15.5:0/2880159409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8673+0+0 (secure 0 0 0) 0x7f1c7c003800 con 0x5617fd9f84a0 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c90c17740 1 -- 172.21.15.5:0/2880159409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c7809a0b0 con 0x5617fd9f84a0 2024-09-11T22:31:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c677fe640 1 -- 172.21.15.5:0/2880159409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1c7c003a00 con 0x5617fd9f84a0 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c90c17740 10 client.18195 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c90c17740 1 -- 172.21.15.5:0/2880159409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5617fda7d9b0 con 0x5617fd9f84a0 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.699+0000 7f1c677fe640 1 -- 172.21.15.5:0/2880159409 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c7809a0b0 con 0x5617fd9f84a0 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.700+0000 7f1c677fe640 1 -- 172.21.15.5:0/2880159409 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f1c7c024410 con 0x5617fd9f84a0 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.700+0000 7f1c90c17740 10 client.18195 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.700+0000 7f1c90c17740 10 client.18195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.700+0000 7f1c90c17740 20 client.18195 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.700+0000 7f1c90c17740 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5617fda03680 0x5617fda7fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.700+0000 7f1c90c17740 4 client.18195 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.700+0000 7f1c90c17740 1 -- 172.21.15.5:0/2880159409 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617fd965d60 con 0x5617fda03680 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.700+0000 7f1c87fff640 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5617fda03680 0x5617fda7fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.700+0000 7f1c87fff640 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5617fda03680 0x5617fda7fe40 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7347d1031301c9 server_cookie=9593ca075d1ca1a4 in_seq=0 out_seq=0 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.700+0000 7f1c677fe640 10 client.18195 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.701+0000 7f1c677fe640 1 -- 172.21.15.5:0/2880159409 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5617fd965d60 con 0x5617fda03680 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.701+0000 7f1c677fe640 10 client.18195 handle_command_reply: tid=0 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.701+0000 7f1c90c17740 1 client.18195 shutdown 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.701+0000 7f1c90c17740 1 -- 172.21.15.5:0/2880159409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 msgr2=0x5617fda0be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.701+0000 7f1c90c17740 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda0be80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c014040 tx=0x7f1c7c005610 comp rx=0 tx=0).stop 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.701+0000 7f1c90c17740 1 -- 172.21.15.5:0/2880159409 shutdown_connections 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c90c17740 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5617fda03680 0x5617fda7fe40 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c90c17740 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5617fd9f8dc0 0x5617fda06d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c90c17740 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617fd9f84a0 0x5617fda0be80 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c90c17740 1 --2- 172.21.15.5:0/2880159409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5617fd9f4a30 0x5617fda0b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c90c17740 1 -- 172.21.15.5:0/2880159409 >> 172.21.15.5:0/2880159409 conn(0x5617fda00890 msgr2=0x5617fd9fb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c677fe640 0 client.18195 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c90c17740 1 -- 172.21.15.5:0/2880159409 shutdown_connections 2024-09-11T22:31:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c90c17740 1 -- 172.21.15.5:0/2880159409 wait complete. 2024-09-11T22:31:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c90c17740 20 client.18195 trim_cache size 0 max 16384 2024-09-11T22:31:06.709 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:31:06.709 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:31:06.709 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:31:06.709 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:31:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c8e83e640 1 -- 172.21.15.5:0/2623958551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1c700a2cf0 msgr2=0x7f1c700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c8e83e640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1c700a2cf0 0x7f1c700a51b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009430 tx=0x7f1c8002a950 comp rx=0 tx=0).stop 2024-09-11T22:31:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c8e83e640 1 -- 172.21.15.5:0/2623958551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c88100e20 msgr2=0x7f1c8819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.702+0000 7f1c8e83e640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c88100e20 0x7f1c8819a650 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800e9f0 tx=0x7f1c7800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.703+0000 7f1c8e83e640 1 -- 172.21.15.5:0/2623958551 shutdown_connections 2024-09-11T22:31:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.703+0000 7f1c8e83e640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c88102ef0 0x7f1c8819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.703+0000 7f1c8e83e640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1c700a2cf0 0x7f1c700a51b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.703+0000 7f1c8e83e640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1c88102570 0x7f1c8819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.703+0000 7f1c8e83e640 1 --2- 172.21.15.5:0/2623958551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c88100e20 0x7f1c8819a650 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.703+0000 7f1c8e83e640 1 -- 172.21.15.5:0/2623958551 >> 172.21.15.5:0/2623958551 conn(0x7f1c880fc7d0 msgr2=0x7f1c8806a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.703+0000 7f1c8e83e640 1 -- 172.21.15.5:0/2623958551 shutdown_connections 2024-09-11T22:31:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:06.703+0000 7f1c8e83e640 1 -- 172.21.15.5:0/2623958551 wait complete. 2024-09-11T22:31:07.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 654720 7.35 MB/sec execute 325 sec latency 47.954 ms 2024-09-11T22:31:07.777 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:31:07.778 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:31:08.040 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:07 smithi005 ceph-mon[30943]: pgmap v279: 97 pgs: 97 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 797 KiB/s wr, 110 op/s 2024-09-11T22:31:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:07 smithi159 ceph-mon[35534]: pgmap v279: 97 pgs: 97 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 797 KiB/s wr, 110 op/s 2024-09-11T22:31:08.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:07 smithi172 ceph-mon[35461]: pgmap v279: 97 pgs: 97 active+clean; 84 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 797 KiB/s wr, 110 op/s 2024-09-11T22:31:08.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 657128 7.37 MB/sec execute 326 sec latency 19.821 ms 2024-09-11T22:31:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.542+0000 7f1d65590640 1 -- 172.21.15.5:0/510105887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60101de0 msgr2=0x7f1d601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.542+0000 7f1d65590640 1 --2- 172.21.15.5:0/510105887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60101de0 0x7f1d601021c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009900 tx=0x7f1d5002a410 comp rx=0 tx=0).stop 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.542+0000 7f1d65590640 1 -- 172.21.15.5:0/510105887 shutdown_connections 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.542+0000 7f1d65590640 1 --2- 172.21.15.5:0/510105887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d60106f10 0x7f1d6010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.542+0000 7f1d65590640 1 --2- 172.21.15.5:0/510105887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d60102790 0x7f1d601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.542+0000 7f1d65590640 1 --2- 172.21.15.5:0/510105887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60101de0 0x7f1d601021c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.542+0000 7f1d65590640 1 -- 172.21.15.5:0/510105887 >> 172.21.15.5:0/510105887 conn(0x7f1d600fd8c0 msgr2=0x7f1d600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.542+0000 7f1d65590640 1 -- 172.21.15.5:0/510105887 shutdown_connections 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.542+0000 7f1d65590640 1 -- 172.21.15.5:0/510105887 wait complete. 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.543+0000 7f1d65590640 1 Processor -- start 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.543+0000 7f1d65590640 1 -- start start 2024-09-11T22:31:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.543+0000 7f1d65590640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60101de0 0x7f1d60198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.543+0000 7f1d65590640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d60102790 0x7f1d60198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.543+0000 7f1d65590640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d60106f10 0x7f1d6019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.543+0000 7f1d65590640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60111f70 con 0x7f1d60101de0 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.543+0000 7f1d65590640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d60111df0 con 0x7f1d60102790 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.543+0000 7f1d65590640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1d601120f0 con 0x7f1d60106f10 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60101de0 0x7f1d60198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60101de0 0x7f1d60198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45330/0 (socket says 172.21.15.5:45330) 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5effd640 1 -- 172.21.15.5:0/1216734769 learned_addr learned my addr 172.21.15.5:0/1216734769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5e7fc640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d60102790 0x7f1d60198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5f7fe640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d60106f10 0x7f1d6019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5effd640 1 -- 172.21.15.5:0/1216734769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d60106f10 msgr2=0x7f1d6019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5effd640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d60106f10 0x7f1d6019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5effd640 1 -- 172.21.15.5:0/1216734769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d60102790 msgr2=0x7f1d60198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5effd640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d60102790 0x7f1d60198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5effd640 1 -- 172.21.15.5:0/1216734769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6019d400 con 0x7f1d60101de0 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5e7fc640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d60102790 0x7f1d60198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d5effd640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60101de0 0x7f1d60198450 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1d50006010 tx=0x7f1d50004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d47fff640 1 -- 172.21.15.5:0/1216734769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5003f030 con 0x7f1d60101de0 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d65590640 1 -- 172.21.15.5:0/1216734769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6019d690 con 0x7f1d60101de0 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d47fff640 1 -- 172.21.15.5:0/1216734769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d50004440 con 0x7f1d60101de0 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d47fff640 1 -- 172.21.15.5:0/1216734769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d50035070 con 0x7f1d60101de0 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.544+0000 7f1d65590640 1 -- 172.21.15.5:0/1216734769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d601a4fd0 con 0x7f1d60101de0 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.545+0000 7f1d65590640 1 -- 172.21.15.5:0/1216734769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d601032c0 con 0x7f1d60101de0 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.546+0000 7f1d47fff640 1 -- 172.21.15.5:0/1216734769 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1d50042030 con 0x7f1d60101de0 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.547+0000 7f1d47fff640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d480a2cf0 0x7f1d480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.547+0000 7f1d5e7fc640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d480a2cf0 0x7f1d480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.547+0000 7f1d47fff640 1 -- 172.21.15.5:0/1216734769 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8673+0+0 (secure 0 0 0) 0x7f1d500ef840 con 0x7f1d60101de0 2024-09-11T22:31:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.548+0000 7f1d5e7fc640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d480a2cf0 0x7f1d480a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1d600fdce0 tx=0x7f1d54005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.552+0000 7f1d47fff640 1 -- 172.21.15.5:0/1216734769 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1d500bbd70 con 0x7f1d60101de0 2024-09-11T22:31:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.769+0000 7f1d65590640 1 -- 172.21.15.5:0/1216734769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d6010a0f0 con 0x7f1d60101de0 2024-09-11T22:31:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.770+0000 7f1d47fff640 1 -- 172.21.15.5:0/1216734769 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11967 (secure 0 0 0) 0x7f1d500c0070 con 0x7f1d60101de0 2024-09-11T22:31:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:31:08.771 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":40,"btime":"2024-09-11T22:29:21:436884+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:29:21.436856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26914":{"gid":26914,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1755675305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1755675305},{"type":"v1","addr":"172.21.15.159:6833","nonce":1755675305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:31:08.772 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 40 2024-09-11T22:31:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 -- 172.21.15.5:0/1216734769 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d480a2cf0 msgr2=0x7f1d480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d480a2cf0 0x7f1d480a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1d600fdce0 tx=0x7f1d54005f90 comp rx=0 tx=0).stop 2024-09-11T22:31:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 -- 172.21.15.5:0/1216734769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60101de0 msgr2=0x7f1d60198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60101de0 0x7f1d60198450 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1d50006010 tx=0x7f1d50004040 comp rx=0 tx=0).stop 2024-09-11T22:31:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 -- 172.21.15.5:0/1216734769 shutdown_connections 2024-09-11T22:31:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d60106f10 0x7f1d6019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d480a2cf0 0x7f1d480a51b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d60102790 0x7f1d60198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 --2- 172.21.15.5:0/1216734769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d60101de0 0x7f1d60198450 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 -- 172.21.15.5:0/1216734769 >> 172.21.15.5:0/1216734769 conn(0x7f1d600fd8c0 msgr2=0x7f1d600fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 -- 172.21.15.5:0/1216734769 shutdown_connections 2024-09-11T22:31:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:08.773+0000 7f1d65590640 1 -- 172.21.15.5:0/1216734769 wait complete. 2024-09-11T22:31:09.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 658963 7.37 MB/sec execute 327 sec latency 38.592 ms 2024-09-11T22:31:09.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:31:09.751 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:09.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:09 smithi005 ceph-mon[30943]: pgmap v280: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-09-11T22:31:09.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:09 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1216734769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:31:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:09 smithi159 ceph-mon[35534]: pgmap v280: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-09-11T22:31:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:09 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1216734769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:31:10.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:09 smithi172 ceph-mon[35461]: pgmap v280: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-09-11T22:31:10.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:09 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1216734769' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:31:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.291+0000 7f3e22a97640 1 -- 172.21.15.5:0/94888072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c102ef0 msgr2=0x7f3e1c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.291+0000 7f3e22a97640 1 --2- 172.21.15.5:0/94888072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c102ef0 0x7f3e1c10da40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009ca0 tx=0x7f3e1402a950 comp rx=0 tx=0).stop 2024-09-11T22:31:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.291+0000 7f3e22a97640 1 -- 172.21.15.5:0/94888072 shutdown_connections 2024-09-11T22:31:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.291+0000 7f3e22a97640 1 --2- 172.21.15.5:0/94888072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c102ef0 0x7f3e1c10da40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.291+0000 7f3e22a97640 1 --2- 172.21.15.5:0/94888072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e1c102570 0x7f3e1c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.291+0000 7f3e22a97640 1 --2- 172.21.15.5:0/94888072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c100e20 0x7f3e1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.291+0000 7f3e22a97640 1 -- 172.21.15.5:0/94888072 >> 172.21.15.5:0/94888072 conn(0x7f3e1c0fc7d0 msgr2=0x7f3e1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.291+0000 7f3e22a97640 1 -- 172.21.15.5:0/94888072 shutdown_connections 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.291+0000 7f3e22a97640 1 -- 172.21.15.5:0/94888072 wait complete. 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.292+0000 7f3e22a97640 1 Processor -- start 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.292+0000 7f3e22a97640 1 -- start start 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e22a97640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e1c100e20 0x7f3e1c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e22a97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c102570 0x7f3e1c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e22a97640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c102ef0 0x7f3e1c19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e22a97640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c110f80 con 0x7f3e1c102570 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e22a97640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c110e00 con 0x7f3e1c100e20 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e22a97640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c111100 con 0x7f3e1c102ef0 2024-09-11T22:31:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e1bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c102570 0x7f3e1c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e1bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c102570 0x7f3e1c19ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45344/0 (socket says 172.21.15.5:45344) 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e1bfff640 1 -- 172.21.15.5:0/3612912707 learned_addr learned my addr 172.21.15.5:0/3612912707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e21034640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c102ef0 0x7f3e1c19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e20833640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e1c100e20 0x7f3e1c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e1bfff640 1 -- 172.21.15.5:0/3612912707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c102ef0 msgr2=0x7f3e1c19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c102ef0 0x7f3e1c19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e1bfff640 1 -- 172.21.15.5:0/3612912707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e1c100e20 msgr2=0x7f3e1c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e1c100e20 0x7f3e1c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e1bfff640 1 -- 172.21.15.5:0/3612912707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1c19f6f0 con 0x7f3e1c102570 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.293+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c102570 0x7f3e1c19ac80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800e9f0 tx=0x7f3e0800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.294+0000 7f3e19ffb640 1 -- 172.21.15.5:0/3612912707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800cd80 con 0x7f3e1c102570 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.294+0000 7f3e22a97640 1 -- 172.21.15.5:0/3612912707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1c19f9e0 con 0x7f3e1c102570 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.294+0000 7f3e19ffb640 1 -- 172.21.15.5:0/3612912707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e08010910 con 0x7f3e1c102570 2024-09-11T22:31:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.294+0000 7f3e19ffb640 1 -- 172.21.15.5:0/3612912707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e080188f0 con 0x7f3e1c102570 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.294+0000 7f3e22a97640 1 -- 172.21.15.5:0/3612912707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e1c1a72c0 con 0x7f3e1c102570 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.296+0000 7f3e19ffb640 1 -- 172.21.15.5:0/3612912707 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3e08010ac0 con 0x7f3e1c102570 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e24e70740 1 Processor -- start 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e24e70740 1 -- start start 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f57090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f575d0 0x564785f57a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 0x564785f64760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e24e70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564785ebd2f0 con 0x564785f62370 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e24e70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564785e64cf0 con 0x564785f58b40 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e24e70740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564785eb92b0 con 0x564785f575d0 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e21034640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f57090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e1bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 0x564785f64760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e21034640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f57090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39432/0 (socket says 172.21.15.5:39432) 2024-09-11T22:31:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e21034640 1 -- 172.21.15.5:0/4092832474 learned_addr learned my addr 172.21.15.5:0/4092832474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e20833640 1 --2- 172.21.15.5:0/4092832474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f575d0 0x564785f57a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.297+0000 7f3e19ffb640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e0c0a2d70 0x7f3e0c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e21034640 1 -- 172.21.15.5:0/4092832474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f575d0 msgr2=0x564785f57a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e21034640 1 --2- 172.21.15.5:0/4092832474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f575d0 0x564785f57a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e21034640 1 -- 172.21.15.5:0/4092832474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 msgr2=0x564785f64760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e21034640 1 --2- 172.21.15.5:0/4092832474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 0x564785f64760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e21034640 1 -- 172.21.15.5:0/4092832474 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564785e493a0 con 0x564785f58b40 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e19ffb640 1 -- 172.21.15.5:0/3612912707 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8673+0+0 (secure 0 0 0) 0x7f3e080cf7c0 con 0x7f3e1c102570 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e21034640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e0c0a2d70 0x7f3e0c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e21034640 1 --2- 172.21.15.5:0/4092832474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f57090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e14035000 tx=0x7f3e14005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c40f0e8a60d9f75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e06ffd640 1 -- 172.21.15.5:0/4092832474 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14034040 con 0x564785f58b40 2024-09-11T22:31:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e21034640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e0c0a2d70 0x7f3e0c0a5230 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e1404d000 tx=0x7f3e1404a000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e06ffd640 1 -- 172.21.15.5:0/4092832474 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e14003b70 con 0x564785f58b40 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.298+0000 7f3e06ffd640 1 -- 172.21.15.5:0/4092832474 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14046690 con 0x564785f58b40 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.299+0000 7f3e24e70740 1 -- 172.21.15.5:0/4092832474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 msgr2=0x564785f57090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.299+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4092832474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f57090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e14035000 tx=0x7f3e14005ef0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.299+0000 7f3e24e70740 1 -- 172.21.15.5:0/4092832474 shutdown_connections 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.299+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4092832474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 0x564785f64760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.299+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4092832474 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f575d0 0x564785f57a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.299+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4092832474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f57090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.299+0000 7f3e24e70740 1 -- 172.21.15.5:0/4092832474 >> 172.21.15.5:0/4092832474 conn(0x564785f20310 msgr2=0x564785f53c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.299+0000 7f3e24e70740 1 -- 172.21.15.5:0/4092832474 shutdown_connections 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.299+0000 7f3e24e70740 1 -- 172.21.15.5:0/4092832474 wait complete. 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e24e70740 1 -- start start 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e24e70740 1 Processor -- start 2024-09-11T22:31:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 0x564785f5f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 0x564785f5f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e24e70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564785f65070 con 0x564785f575d0 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e24e70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564785e78060 con 0x564785f62370 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e24e70740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564785f651f0 con 0x564785f58b40 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e24e70740 10 client.0 _ll_register_callbacks cb 0x564785bcb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e20833640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e1bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 0x564785f5f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e21034640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 0x564785f5f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e21034640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 0x564785f5f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39446/0 (socket says 172.21.15.5:39446) 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e20833640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45362/0 (socket says 172.21.15.5:45362) 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.300+0000 7f3e21034640 1 -- 172.21.15.5:0/1971019984 learned_addr learned my addr 172.21.15.5:0/1971019984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3e1bfff640 1 -- 172.21.15.5:0/1971019984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 msgr2=0x564785f5f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 0x564785f5f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3e1bfff640 1 -- 172.21.15.5:0/1971019984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 msgr2=0x564785f61df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3e1bfff640 1 -- 172.21.15.5:0/1971019984 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564785c2e3c0 con 0x564785f58b40 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 0x564785f5f0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e080cce70 tx=0x7f3e0809b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d034154dd5f8789a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3dfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3dfbfff640 1 -- 172.21.15.5:0/1971019984 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0809fff0 con 0x564785f58b40 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3dfbfff640 1 -- 172.21.15.5:0/1971019984 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8673+0+0 (secure 0 0 0) 0x7f3e080a4420 con 0x564785f58b40 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3e24e70740 1 -- 172.21.15.5:0/1971019984 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e14003b70 con 0x564785f58b40 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3dfbfff640 1 -- 172.21.15.5:0/1971019984 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3e080a48b0 con 0x564785f58b40 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3e24e70740 10 client.27397 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.301+0000 7f3e24e70740 1 -- 172.21.15.5:0/1971019984 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564785f70bf0 con 0x564785f58b40 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.302+0000 7f3dfbfff640 1 -- 172.21.15.5:0/1971019984 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e14003b70 con 0x564785f58b40 2024-09-11T22:31:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.302+0000 7f3dfbfff640 1 -- 172.21.15.5:0/1971019984 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f3e080a4b90 con 0x564785f58b40 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.302+0000 7f3e24e70740 10 client.27397 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.302+0000 7f3e24e70740 10 client.27397 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.302+0000 7f3e24e70740 20 client.27397 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.302+0000 7f3e24e70740 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x564785f720b0 0x564785f744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.302+0000 7f3e24e70740 4 client.27397 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.302+0000 7f3e24e70740 1 -- 172.21.15.5:0/1971019984 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564785eca860 con 0x564785f720b0 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.302+0000 7f3e20833640 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x564785f720b0 0x564785f744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3dfbfff640 10 client.27397 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3e20833640 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x564785f720b0 0x564785f744a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8a1f10f152a891e2 server_cookie=e221db1030e63cb7 in_seq=0 out_seq=0 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3dfbfff640 1 -- 172.21.15.5:0/1971019984 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564785eca860 con 0x564785f720b0 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3dfbfff640 10 client.27397 handle_command_reply: tid=0 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3e24e70740 1 client.27397 shutdown 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3e24e70740 1 -- 172.21.15.5:0/1971019984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 msgr2=0x564785f5f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3e24e70740 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 0x564785f5f0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3e080cce70 tx=0x7f3e0809b8f0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3e24e70740 1 -- 172.21.15.5:0/1971019984 shutdown_connections 2024-09-11T22:31:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3e24e70740 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x564785f720b0 0x564785f744a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3e24e70740 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 0x564785f5f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3e24e70740 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 0x564785f5f0a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3e24e70740 1 --2- 172.21.15.5:0/1971019984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3e24e70740 1 -- 172.21.15.5:0/1971019984 >> 172.21.15.5:0/1971019984 conn(0x564785f20310 msgr2=0x564785f59960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.303+0000 7f3dfbfff640 0 client.27397 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:31:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.304+0000 7f3e24e70740 1 -- 172.21.15.5:0/1971019984 shutdown_connections 2024-09-11T22:31:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.304+0000 7f3e24e70740 1 -- 172.21.15.5:0/1971019984 wait complete. 2024-09-11T22:31:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.304+0000 7f3e24e70740 20 client.27397 trim_cache size 0 max 16384 2024-09-11T22:31:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e24e70740 1 Processor -- start 2024-09-11T22:31:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e24e70740 1 -- start start 2024-09-11T22:31:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 0x564785f5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 0x564785f5a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e24e70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1ab840 con 0x564785f575d0 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e24e70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1ad380 con 0x564785f62370 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e24e70740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1abf10 con 0x564785f58b40 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e20833640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e20833640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45368/0 (socket says 172.21.15.5:45368) 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e20833640 1 -- 172.21.15.5:0/4078969896 learned_addr learned my addr 172.21.15.5:0/4078969896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e21034640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 0x564785f5a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/4078969896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 0x564785f5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e20833640 1 -- 172.21.15.5:0/4078969896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 msgr2=0x564785f5a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e20833640 1 --2- 172.21.15.5:0/4078969896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 0x564785f5a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e20833640 1 -- 172.21.15.5:0/4078969896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 msgr2=0x564785f5a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e20833640 1 --2- 172.21.15.5:0/4078969896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 0x564785f5a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.319+0000 7f3e20833640 1 -- 172.21.15.5:0/4078969896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564785f54dd0 con 0x564785f575d0 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.320+0000 7f3e20833640 1 --2- 172.21.15.5:0/4078969896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61120 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3e10000c00 tx=0x7f3e100108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eaea5403d0d2ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.320+0000 7f3e067fc640 1 -- 172.21.15.5:0/4078969896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10022020 con 0x564785f575d0 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.320+0000 7f3e067fc640 1 -- 172.21.15.5:0/4078969896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e10004250 con 0x564785f575d0 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.320+0000 7f3e067fc640 1 -- 172.21.15.5:0/4078969896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1001e820 con 0x564785f575d0 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.321+0000 7f3e24e70740 1 -- 172.21.15.5:0/4078969896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 msgr2=0x564785f61120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.321+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4078969896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61120 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3e10000c00 tx=0x7f3e100108c0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.321+0000 7f3e24e70740 1 -- 172.21.15.5:0/4078969896 shutdown_connections 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.321+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4078969896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f62370 0x564785f5a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.321+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4078969896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f58b40 0x564785f5a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.321+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4078969896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f575d0 0x564785f61120 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.321+0000 7f3e24e70740 1 -- 172.21.15.5:0/4078969896 >> 172.21.15.5:0/4078969896 conn(0x564785f20310 msgr2=0x564785f5e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.321+0000 7f3e24e70740 1 -- 172.21.15.5:0/4078969896 shutdown_connections 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.321+0000 7f3e24e70740 1 -- 172.21.15.5:0/4078969896 wait complete. 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e24e70740 1 -- start start 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e24e70740 1 Processor -- start 2024-09-11T22:31:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f575d0 0x564785f6a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e24e70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 0x564785f67450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e24e70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1accb0 con 0x564785f62370 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e24e70740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1ac5e0 con 0x564785f58b40 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e24e70740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1c1a9880 con 0x564785f575d0 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e24e70740 10 client.0 _ll_register_callbacks cb 0x7f3e0809bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e21034640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 0x564785f67450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e1bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e21034640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 0x564785f67450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45382/0 (socket says 172.21.15.5:45382) 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.322+0000 7f3e21034640 1 -- 172.21.15.5:0/4106947947 learned_addr learned my addr 172.21.15.5:0/4106947947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e20833640 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f575d0 0x564785f6a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e21034640 1 -- 172.21.15.5:0/4106947947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f575d0 msgr2=0x564785f6a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e21034640 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f575d0 0x564785f6a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e21034640 1 -- 172.21.15.5:0/4106947947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 msgr2=0x564785f6a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e21034640 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f6a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e21034640 1 -- 172.21.15.5:0/4106947947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564785f53510 con 0x564785f62370 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e1bfff640 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f6a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e21034640 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 0x564785f67450 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3e14006010 tx=0x7f3e140095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644a2801efea45b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3dfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3dfbfff640 1 -- 172.21.15.5:0/4106947947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e14046b00 con 0x564785f62370 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e24e70740 1 -- 172.21.15.5:0/4106947947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e10004250 con 0x564785f62370 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3dfbfff640 1 -- 172.21.15.5:0/4106947947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8673+0+0 (secure 0 0 0) 0x7f3e14046ca0 con 0x564785f62370 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3dfbfff640 1 -- 172.21.15.5:0/4106947947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f3e1402fdb0 con 0x564785f62370 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e24e70740 10 client.18219 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.323+0000 7f3e24e70740 1 -- 172.21.15.5:0/4106947947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564785fdd270 con 0x564785f62370 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.324+0000 7f3dfbfff640 1 -- 172.21.15.5:0/4106947947 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e10004250 con 0x564785f62370 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.324+0000 7f3dfbfff640 1 -- 172.21.15.5:0/4106947947 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f3e140306a0 con 0x564785f62370 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.324+0000 7f3e24e70740 10 client.18219 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.324+0000 7f3e24e70740 10 client.18219 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.324+0000 7f3e24e70740 20 client.18219 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.324+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x564785fdd5a0 0x564785fdf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.324+0000 7f3e24e70740 4 client.18219 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.324+0000 7f3e24e70740 1 -- 172.21.15.5:0/4106947947 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564785ec4d70 con 0x564785fdd5a0 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e20833640 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x564785fdd5a0 0x564785fdf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e20833640 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x564785fdd5a0 0x564785fdf990 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eb76031873b6323e server_cookie=c5218948a4f52327 in_seq=0 out_seq=0 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3dfbfff640 10 client.18219 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3dfbfff640 1 -- 172.21.15.5:0/4106947947 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564785ec4d70 con 0x564785fdd5a0 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3dfbfff640 10 client.18219 handle_command_reply: tid=0 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e24e70740 1 client.18219 shutdown 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e24e70740 1 -- 172.21.15.5:0/4106947947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 msgr2=0x564785f67450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 0x564785f67450 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3e14006010 tx=0x7f3e140095c0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e24e70740 1 -- 172.21.15.5:0/4106947947 shutdown_connections 2024-09-11T22:31:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x564785fdd5a0 0x564785fdf990 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564785f62370 0x564785f67450 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564785f58b40 0x564785f6a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e24e70740 1 --2- 172.21.15.5:0/4106947947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564785f575d0 0x564785f6a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3dfbfff640 0 client.18219 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:31:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.325+0000 7f3e24e70740 1 -- 172.21.15.5:0/4106947947 >> 172.21.15.5:0/4106947947 conn(0x564785f20310 msgr2=0x564785f52d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.326+0000 7f3e24e70740 1 -- 172.21.15.5:0/4106947947 shutdown_connections 2024-09-11T22:31:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.326+0000 7f3e24e70740 1 -- 172.21.15.5:0/4106947947 wait complete. 2024-09-11T22:31:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.326+0000 7f3e24e70740 20 client.18219 trim_cache size 0 max 16384 2024-09-11T22:31:10.332 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:31:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 -- 172.21.15.5:0/3612912707 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e0c0a2d70 msgr2=0x7f3e0c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e0c0a2d70 0x7f3e0c0a5230 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3e1404d000 tx=0x7f3e1404a000 comp rx=0 tx=0).stop 2024-09-11T22:31:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 -- 172.21.15.5:0/3612912707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c102570 msgr2=0x7f3e1c19ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c102570 0x7f3e1c19ac80 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800e9f0 tx=0x7f3e0800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 -- 172.21.15.5:0/3612912707 shutdown_connections 2024-09-11T22:31:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e1c102ef0 0x7f3e1c19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1c102570 0x7f3e1c19ac80 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3e0c0a2d70 0x7f3e0c0a5230 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 --2- 172.21.15.5:0/3612912707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3e1c100e20 0x7f3e1c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 -- 172.21.15.5:0/3612912707 >> 172.21.15.5:0/3612912707 conn(0x7f3e1c0fc7d0 msgr2=0x7f3e1c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 -- 172.21.15.5:0/3612912707 shutdown_connections 2024-09-11T22:31:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:10.327+0000 7f3e22a97640 1 -- 172.21.15.5:0/3612912707 wait complete. 2024-09-11T22:31:10.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 661010 7.39 MB/sec execute 328 sec latency 37.718 ms 2024-09-11T22:31:11.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:31:11.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 663306 7.40 MB/sec execute 329 sec latency 18.352 ms 2024-09-11T22:31:11.376 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:12.021 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:11 smithi005 ceph-mon[30943]: pgmap v281: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-09-11T22:31:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:11 smithi159 ceph-mon[35534]: pgmap v281: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-09-11T22:31:12.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:11 smithi172 ceph-mon[35461]: pgmap v281: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 174 op/s 2024-09-11T22:31:12.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 665042 7.40 MB/sec execute 330 sec latency 63.278 ms 2024-09-11T22:31:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.452+0000 7f12f9c5f640 1 -- 172.21.15.5:0/213169837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f4102790 msgr2=0x7f12f41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.452+0000 7f12f9c5f640 1 --2- 172.21.15.5:0/213169837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f4102790 0x7f12f41067e0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009900 tx=0x7f12e802a410 comp rx=0 tx=0).stop 2024-09-11T22:31:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.452+0000 7f12f9c5f640 1 -- 172.21.15.5:0/213169837 shutdown_connections 2024-09-11T22:31:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.452+0000 7f12f9c5f640 1 --2- 172.21.15.5:0/213169837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f4106f10 0x7f12f410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.452+0000 7f12f9c5f640 1 --2- 172.21.15.5:0/213169837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f4102790 0x7f12f41067e0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.452+0000 7f12f9c5f640 1 --2- 172.21.15.5:0/213169837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f4101de0 0x7f12f41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.452+0000 7f12f9c5f640 1 -- 172.21.15.5:0/213169837 >> 172.21.15.5:0/213169837 conn(0x7f12f40fd880 msgr2=0x7f12f40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.452+0000 7f12f9c5f640 1 -- 172.21.15.5:0/213169837 shutdown_connections 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.452+0000 7f12f9c5f640 1 -- 172.21.15.5:0/213169837 wait complete. 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.453+0000 7f12f9c5f640 1 Processor -- start 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.453+0000 7f12f9c5f640 1 -- start start 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f9c5f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f4101de0 0x7f12f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f9c5f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f4102790 0x7f12f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f9c5f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f4106f10 0x7f12f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f9c5f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4111f30 con 0x7f12f4106f10 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f4106f10 0x7f12f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f9c5f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4111db0 con 0x7f12f4102790 2024-09-11T22:31:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f4106f10 0x7f12f4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56836/0 (socket says 172.21.15.5:56836) 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f4101de0 0x7f12f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f9c5f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41120b0 con 0x7f12f4101de0 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f3fff640 1 -- 172.21.15.5:0/1783313464 learned_addr learned my addr 172.21.15.5:0/1783313464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.454+0000 7f12f2ffd640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f4102790 0x7f12f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.455+0000 7f12f37fe640 1 -- 172.21.15.5:0/1783313464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f4102790 msgr2=0x7f12f4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.455+0000 7f12f37fe640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f4102790 0x7f12f4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.455+0000 7f12f37fe640 1 -- 172.21.15.5:0/1783313464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f4106f10 msgr2=0x7f12f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.455+0000 7f12f37fe640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f4106f10 0x7f12f4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.455+0000 7f12f37fe640 1 -- 172.21.15.5:0/1783313464 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f41a68f0 con 0x7f12f4101de0 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.455+0000 7f12f3fff640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f4106f10 0x7f12f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.455+0000 7f12f37fe640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f4101de0 0x7f12f40744f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12e400e9f0 tx=0x7f12e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.459+0000 7f12f0ff9640 1 -- 172.21.15.5:0/1783313464 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e400ccc0 con 0x7f12f4101de0 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.459+0000 7f12f9c5f640 1 -- 172.21.15.5:0/1783313464 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f41a6be0 con 0x7f12f4101de0 2024-09-11T22:31:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.459+0000 7f12f0ff9640 1 -- 172.21.15.5:0/1783313464 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e4010910 con 0x7f12f4101de0 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.459+0000 7f12f0ff9640 1 -- 172.21.15.5:0/1783313464 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e40188e0 con 0x7f12f4101de0 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.459+0000 7f12f9c5f640 1 -- 172.21.15.5:0/1783313464 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f41a7140 con 0x7f12f4101de0 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.462+0000 7f12f0ff9640 1 -- 172.21.15.5:0/1783313464 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f12e4010430 con 0x7f12f4101de0 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.462+0000 7f12fc038740 1 Processor -- start 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.462+0000 7f12fc038740 1 -- start start 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f0ff9640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f12dc0a2cf0 0x7f12dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f3fff640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f12dc0a2cf0 0x7f12dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12fc038740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774cb40 0x5596f774b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12fc038740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 0x5596f774ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f0ff9640 1 -- 172.21.15.5:0/1783313464 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f12e40cf960 con 0x7f12f4101de0 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12fc038740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f7758760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12fc038740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596f76b12f0 con 0x5596f774cb40 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12fc038740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5596f7658cf0 con 0x5596f7756370 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12fc038740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5596f76ad2b0 con 0x5596f774b5d0 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f7758760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f7758760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47234/0 (socket says 172.21.15.5:47234) 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f3fff640 1 -- 172.21.15.5:0/2109149938 learned_addr learned my addr 172.21.15.5:0/2109149938 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f37fe640 1 --2- 172.21.15.5:0/2109149938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774cb40 0x5596f774b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f2ffd640 1 --2- 172.21.15.5:0/2109149938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 0x5596f774ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f3fff640 1 -- 172.21.15.5:0/2109149938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 msgr2=0x5596f774ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f3fff640 1 --2- 172.21.15.5:0/2109149938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 0x5596f774ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f3fff640 1 -- 172.21.15.5:0/2109149938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774cb40 msgr2=0x5596f774b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f3fff640 1 --2- 172.21.15.5:0/2109149938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774cb40 0x5596f774b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.463+0000 7f12f3fff640 1 -- 172.21.15.5:0/2109149938 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f763d3a0 con 0x5596f7756370 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.464+0000 7f12f37fe640 1 --2- 172.21.15.5:0/2109149938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774cb40 0x5596f774b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.464+0000 7f12f3fff640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f12dc0a2cf0 0x7f12dc0a51b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009fd0 tx=0x7f12ec015d40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.464+0000 7f12f3fff640 1 --2- 172.21.15.5:0/2109149938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f7758760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0099a0 tx=0x7f12ec01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a22896f9a5221f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.464+0000 7f12d9ffb640 1 -- 172.21.15.5:0/2109149938 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00fc30 con 0x5596f7756370 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.464+0000 7f12d9ffb640 1 -- 172.21.15.5:0/2109149938 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12ec00fdd0 con 0x5596f7756370 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.464+0000 7f12d9ffb640 1 -- 172.21.15.5:0/2109149938 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0205d0 con 0x5596f7756370 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.465+0000 7f12fc038740 1 -- 172.21.15.5:0/2109149938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 msgr2=0x5596f7758760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.465+0000 7f12fc038740 1 --2- 172.21.15.5:0/2109149938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f7758760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0099a0 tx=0x7f12ec01f000 comp rx=0 tx=0).stop 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.465+0000 7f12fc038740 1 -- 172.21.15.5:0/2109149938 shutdown_connections 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.465+0000 7f12fc038740 1 --2- 172.21.15.5:0/2109149938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f7758760 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.465+0000 7f12fc038740 1 --2- 172.21.15.5:0/2109149938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 0x5596f774ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.465+0000 7f12fc038740 1 --2- 172.21.15.5:0/2109149938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774cb40 0x5596f774b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.465+0000 7f12fc038740 1 -- 172.21.15.5:0/2109149938 >> 172.21.15.5:0/2109149938 conn(0x5596f7714310 msgr2=0x5596f7747c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.465+0000 7f12fc038740 1 -- 172.21.15.5:0/2109149938 shutdown_connections 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.465+0000 7f12fc038740 1 -- 172.21.15.5:0/2109149938 wait complete. 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.465+0000 7f12fc038740 1 -- start start 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12fc038740 1 Processor -- start 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12fc038740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12fc038740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774cb40 0x5596f7753050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12fc038740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f77537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12fc038740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596f7759010 con 0x5596f774b5d0 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12fc038740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5596f766c060 con 0x5596f7756370 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12fc038740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5596f7759190 con 0x5596f774cb40 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12fc038740 10 client.0 _ll_register_callbacks cb 0x5596f73bf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56848/0 (socket says 172.21.15.5:56848) 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12f2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774cb40 0x5596f7753050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12f37fe640 1 -- 172.21.15.5:0/415126612 learned_addr learned my addr 172.21.15.5:0/415126612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12f3fff640 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f77537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.466+0000 7f12f37fe640 1 -- 172.21.15.5:0/415126612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774cb40 msgr2=0x5596f7753050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12f37fe640 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774cb40 0x5596f7753050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12f37fe640 1 -- 172.21.15.5:0/415126612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 msgr2=0x5596f77537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12f37fe640 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f77537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12f37fe640 1 -- 172.21.15.5:0/415126612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f74223c0 con 0x5596f774b5d0 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12f2ffd640 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774cb40 0x5596f7753050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12f3fff640 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f77537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12f37fe640 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f12e40cd040 tx=0x7f12e4010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e6047fb2cde7dd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12d2ffd640 1 -- 172.21.15.5:0/415126612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e40a4a20 con 0x5596f774b5d0 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12d2ffd640 1 -- 172.21.15.5:0/415126612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f12e409b200 con 0x5596f774b5d0 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12fc038740 1 -- 172.21.15.5:0/415126612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12ec00fdd0 con 0x5596f774b5d0 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12d2ffd640 1 -- 172.21.15.5:0/415126612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f12e4010720 con 0x5596f774b5d0 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12fc038740 10 client.18231 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12fc038740 1 -- 172.21.15.5:0/415126612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5596f7764bd0 con 0x5596f774b5d0 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12d2ffd640 1 -- 172.21.15.5:0/415126612 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12ec00fdd0 con 0x5596f774b5d0 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.467+0000 7f12d2ffd640 1 -- 172.21.15.5:0/415126612 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f12e4086040 con 0x5596f774b5d0 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.468+0000 7f12fc038740 10 client.18231 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.468+0000 7f12fc038740 10 client.18231 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:31:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.468+0000 7f12fc038740 20 client.18231 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.468+0000 7f12fc038740 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5596f7766090 0x5596f7768480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.468+0000 7f12fc038740 4 client.18231 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.468+0000 7f12fc038740 1 -- 172.21.15.5:0/415126612 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596f76be860 con 0x5596f7766090 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.468+0000 7f12f2ffd640 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5596f7766090 0x5596f7768480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.468+0000 7f12f2ffd640 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5596f7766090 0x5596f7768480 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=be67f9f63efa3541 server_cookie=15d3614979e627a in_seq=0 out_seq=0 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.468+0000 7f12d2ffd640 10 client.18231 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12d2ffd640 1 -- 172.21.15.5:0/415126612 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596f76be860 con 0x5596f7766090 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12d2ffd640 10 client.18231 handle_command_reply: tid=0 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 client.18231 shutdown 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 -- 172.21.15.5:0/415126612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 msgr2=0x5596f7755da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f12e40cd040 tx=0x7f12e4010f00 comp rx=0 tx=0).stop 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 -- 172.21.15.5:0/415126612 shutdown_connections 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5596f7766090 0x5596f7768480 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f7756370 0x5596f77537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774cb40 0x5596f7753050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 --2- 172.21.15.5:0/415126612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12d2ffd640 0 client.18231 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 -- 172.21.15.5:0/415126612 >> 172.21.15.5:0/415126612 conn(0x5596f7714310 msgr2=0x5596f774d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 -- 172.21.15.5:0/415126612 shutdown_connections 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 1 -- 172.21.15.5:0/415126612 wait complete. 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.469+0000 7f12fc038740 20 client.18231 trim_cache size 0 max 16384 2024-09-11T22:31:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12fc038740 1 Processor -- start 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12fc038740 1 -- start start 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12fc038740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12fc038740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f774cb40 0x5596f774e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12fc038740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f7756370 0x5596f774e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12fc038740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41afbd0 con 0x5596f774b5d0 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12fc038740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41b1710 con 0x5596f774cb40 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12fc038740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41b02a0 con 0x5596f7756370 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12f2ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f774cb40 0x5596f774e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12f37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56856/0 (socket says 172.21.15.5:56856) 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12f37fe640 1 -- 172.21.15.5:0/1313282135 learned_addr learned my addr 172.21.15.5:0/1313282135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12f3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f7756370 0x5596f774e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12f37fe640 1 -- 172.21.15.5:0/1313282135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f7756370 msgr2=0x5596f774e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12f37fe640 1 --2- 172.21.15.5:0/1313282135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f7756370 0x5596f774e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12f37fe640 1 -- 172.21.15.5:0/1313282135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f774cb40 msgr2=0x5596f774e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12f37fe640 1 --2- 172.21.15.5:0/1313282135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f774cb40 0x5596f774e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.484+0000 7f12f37fe640 1 -- 172.21.15.5:0/1313282135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596f7768a60 con 0x5596f774b5d0 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.485+0000 7f12f37fe640 1 --2- 172.21.15.5:0/1313282135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f12e40a44f0 tx=0x7f12e400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8345439a63d22f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.485+0000 7f12d97fa640 1 -- 172.21.15.5:0/1313282135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4098420 con 0x5596f774b5d0 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.485+0000 7f12d97fa640 1 -- 172.21.15.5:0/1313282135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e409f030 con 0x5596f774b5d0 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.485+0000 7f12d97fa640 1 -- 172.21.15.5:0/1313282135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e409ac70 con 0x5596f774b5d0 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.486+0000 7f12fc038740 1 -- 172.21.15.5:0/1313282135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 msgr2=0x5596f7755da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.486+0000 7f12fc038740 1 --2- 172.21.15.5:0/1313282135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f12e40a44f0 tx=0x7f12e400cf60 comp rx=0 tx=0).stop 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.486+0000 7f12fc038740 1 -- 172.21.15.5:0/1313282135 shutdown_connections 2024-09-11T22:31:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.486+0000 7f12fc038740 1 --2- 172.21.15.5:0/1313282135 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f7756370 0x5596f774e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.486+0000 7f12fc038740 1 --2- 172.21.15.5:0/1313282135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f774cb40 0x5596f774e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.486+0000 7f12fc038740 1 --2- 172.21.15.5:0/1313282135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f774b5d0 0x5596f7755da0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.486+0000 7f12fc038740 1 -- 172.21.15.5:0/1313282135 >> 172.21.15.5:0/1313282135 conn(0x5596f7714310 msgr2=0x5596f7742a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.486+0000 7f12fc038740 1 -- 172.21.15.5:0/1313282135 shutdown_connections 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.486+0000 7f12fc038740 1 -- 172.21.15.5:0/1313282135 wait complete. 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.486+0000 7f12fc038740 1 -- start start 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12fc038740 1 Processor -- start 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12fc038740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 0x5596f775e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12fc038740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f774cb40 0x5596f775e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12fc038740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f7756370 0x5596f77598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12fc038740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41b1040 con 0x5596f7756370 2024-09-11T22:31:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12fc038740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41b0970 con 0x5596f774cb40 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 0x5596f775e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12fc038740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41adbd0 con 0x5596f774b5d0 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12f37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 0x5596f775e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33602/0 (socket says 172.21.15.5:33602) 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12f37fe640 1 -- 172.21.15.5:0/2611392065 learned_addr learned my addr 172.21.15.5:0/2611392065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12fc038740 10 client.0 _ll_register_callbacks cb 0x7f12e4097930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.487+0000 7f12f3fff640 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f7756370 0x5596f77598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12f37fe640 1 -- 172.21.15.5:0/2611392065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f774cb40 msgr2=0x5596f775e950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12f37fe640 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f774cb40 0x5596f775e950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12f37fe640 1 -- 172.21.15.5:0/2611392065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f7756370 msgr2=0x5596f77598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12f37fe640 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f7756370 0x5596f77598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12f37fe640 1 -- 172.21.15.5:0/2611392065 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596f775a120 con 0x5596f774b5d0 2024-09-11T22:31:12.524 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:31:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12f3fff640 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f7756370 0x5596f77598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:31:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12f37fe640 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 0x5596f775e410 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f12e400e9c0 tx=0x7f12e409aea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5db3ae5b082a5e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:31:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12d2ffd640 1 -- 172.21.15.5:0/2611392065 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e40a9060 con 0x5596f774b5d0 2024-09-11T22:31:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12d2ffd640 1 -- 172.21.15.5:0/2611392065 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f12e409b200 con 0x5596f774b5d0 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12fc038740 1 -- 172.21.15.5:0/2611392065 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e409f030 con 0x5596f774b5d0 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12d2ffd640 1 -- 172.21.15.5:0/2611392065 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f12e409b710 con 0x5596f774b5d0 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12fc038740 10 client.27421 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.488+0000 7f12fc038740 1 -- 172.21.15.5:0/2611392065 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5596f77d8e90 con 0x5596f774b5d0 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.489+0000 7f12d2ffd640 1 -- 172.21.15.5:0/2611392065 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e409f030 con 0x5596f774b5d0 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.489+0000 7f12d2ffd640 1 -- 172.21.15.5:0/2611392065 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f12e4081020 con 0x5596f774b5d0 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.489+0000 7f12fc038740 10 client.27421 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.489+0000 7f12fc038740 10 client.27421 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.489+0000 7f12fc038740 20 client.27421 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.489+0000 7f12fc038740 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5596f77d9270 0x5596f77db660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.489+0000 7f12fc038740 4 client.27421 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:31:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.489+0000 7f12fc038740 1 -- 172.21.15.5:0/2611392065 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596f76b8d70 con 0x5596f77d9270 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.489+0000 7f12f2ffd640 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5596f77d9270 0x5596f77db660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.489+0000 7f12d2ffd640 10 client.27421 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12f2ffd640 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5596f77d9270 0x5596f77db660 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=77aa8db1ddd0ce72 server_cookie=d263d170e50ac5f6 in_seq=0 out_seq=0 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12d2ffd640 1 -- 172.21.15.5:0/2611392065 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596f76b8d70 con 0x5596f77d9270 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12d2ffd640 10 client.27421 handle_command_reply: tid=0 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 client.27421 shutdown 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 -- 172.21.15.5:0/2611392065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 msgr2=0x5596f775e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 0x5596f775e410 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f12e400e9c0 tx=0x7f12e409aea0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 -- 172.21.15.5:0/2611392065 shutdown_connections 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f7756370 0x5596f77598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5596f77d9270 0x5596f77db660 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596f774cb40 0x5596f775e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12d2ffd640 0 client.27421 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 --2- 172.21.15.5:0/2611392065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f774b5d0 0x5596f775e410 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 -- 172.21.15.5:0/2611392065 >> 172.21.15.5:0/2611392065 conn(0x5596f7714310 msgr2=0x5596f7746500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 -- 172.21.15.5:0/2611392065 shutdown_connections 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.490+0000 7f12fc038740 1 -- 172.21.15.5:0/2611392065 wait complete. 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12fc038740 20 client.27421 trim_cache size 0 max 16384 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12f9c5f640 1 -- 172.21.15.5:0/1783313464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f12dc0a2cf0 msgr2=0x7f12dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12f9c5f640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f12dc0a2cf0 0x7f12dc0a51b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009fd0 tx=0x7f12ec015d40 comp rx=0 tx=0).stop 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12f9c5f640 1 -- 172.21.15.5:0/1783313464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f4101de0 msgr2=0x7f12f40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12f9c5f640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f4101de0 0x7f12f40744f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f12e400e9f0 tx=0x7f12e400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12f9c5f640 1 -- 172.21.15.5:0/1783313464 shutdown_connections 2024-09-11T22:31:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12f9c5f640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f4106f10 0x7f12f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12f9c5f640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f12dc0a2cf0 0x7f12dc0a51b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12f9c5f640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f4102790 0x7f12f4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12f9c5f640 1 --2- 172.21.15.5:0/1783313464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f12f4101de0 0x7f12f40744f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.491+0000 7f12f9c5f640 1 -- 172.21.15.5:0/1783313464 >> 172.21.15.5:0/1783313464 conn(0x7f12f40fd880 msgr2=0x7f12f40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.492+0000 7f12f9c5f640 1 -- 172.21.15.5:0/1783313464 shutdown_connections 2024-09-11T22:31:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:12.492+0000 7f12f9c5f640 1 -- 172.21.15.5:0/1783313464 wait complete. 2024-09-11T22:31:13.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 667003 7.42 MB/sec execute 331 sec latency 55.022 ms 2024-09-11T22:31:13.413 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:31:13.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:13 smithi005 ceph-mon[30943]: pgmap v282: 97 pgs: 97 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-11T22:31:13.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:13 smithi005 ceph-mon[30943]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T22:31:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:13 smithi159 ceph-mon[35534]: pgmap v282: 97 pgs: 97 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-11T22:31:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:13 smithi159 ceph-mon[35534]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T22:31:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:13 smithi172 ceph-mon[35461]: pgmap v282: 97 pgs: 97 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-11T22:31:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:13 smithi172 ceph-mon[35461]: osdmap e95: 12 total, 12 up, 12 in 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.189+0000 7f2d190e2640 1 -- 172.21.15.5:0/2195058661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d14101de0 msgr2=0x7f2d141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.189+0000 7f2d190e2640 1 --2- 172.21.15.5:0/2195058661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d14101de0 0x7f2d141021c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009900 tx=0x7f2d0402a410 comp rx=0 tx=0).stop 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.189+0000 7f2d190e2640 1 -- 172.21.15.5:0/2195058661 shutdown_connections 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.189+0000 7f2d190e2640 1 --2- 172.21.15.5:0/2195058661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d14106f10 0x7f2d1410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.189+0000 7f2d190e2640 1 --2- 172.21.15.5:0/2195058661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14102790 0x7f2d141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.189+0000 7f2d190e2640 1 --2- 172.21.15.5:0/2195058661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d14101de0 0x7f2d141021c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.189+0000 7f2d190e2640 1 -- 172.21.15.5:0/2195058661 >> 172.21.15.5:0/2195058661 conn(0x7f2d140fd8c0 msgr2=0x7f2d140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.189+0000 7f2d190e2640 1 -- 172.21.15.5:0/2195058661 shutdown_connections 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.189+0000 7f2d190e2640 1 -- 172.21.15.5:0/2195058661 wait complete. 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.190+0000 7f2d190e2640 1 Processor -- start 2024-09-11T22:31:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d190e2640 1 -- start start 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d190e2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d14101de0 0x7f2d14198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d190e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d14102790 0x7f2d14198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d190e2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14106f10 0x7f2d1419cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d190e2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14111f30 con 0x7f2d14102790 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d190e2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d14111db0 con 0x7f2d14101de0 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d190e2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141120b0 con 0x7f2d14106f10 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d14101de0 0x7f2d14198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d14102790 0x7f2d14198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d14101de0 0x7f2d14198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47264/0 (socket says 172.21.15.5:47264) 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d12d9d640 1 -- 172.21.15.5:0/2632492824 learned_addr learned my addr 172.21.15.5:0/2632492824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d1259c640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d14102790 0x7f2d14198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56886/0 (socket says 172.21.15.5:56886) 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d1359e640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14106f10 0x7f2d1419cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d12d9d640 1 -- 172.21.15.5:0/2632492824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14106f10 msgr2=0x7f2d1419cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14106f10 0x7f2d1419cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d12d9d640 1 -- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d14102790 msgr2=0x7f2d14198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d14102790 0x7f2d14198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d12d9d640 1 -- 172.21.15.5:0/2632492824 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1419d4c0 con 0x7f2d14101de0 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.191+0000 7f2d1259c640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d14102790 0x7f2d14198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.192+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d14101de0 0x7f2d14198510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2d04006010 tx=0x7f2d04004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.192+0000 7f2cfbfff640 1 -- 172.21.15.5:0/2632492824 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0403f030 con 0x7f2d14101de0 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.192+0000 7f2d190e2640 1 -- 172.21.15.5:0/2632492824 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1419d750 con 0x7f2d14101de0 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.192+0000 7f2cfbfff640 1 -- 172.21.15.5:0/2632492824 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d04004440 con 0x7f2d14101de0 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.192+0000 7f2cfbfff640 1 -- 172.21.15.5:0/2632492824 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04035070 con 0x7f2d14101de0 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.192+0000 7f2d190e2640 1 -- 172.21.15.5:0/2632492824 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d141a5090 con 0x7f2d14101de0 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.195+0000 7f2d1b4bb740 1 Processor -- start 2024-09-11T22:31:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.195+0000 7f2d1b4bb740 1 -- start start 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.195+0000 7f2cfbfff640 1 -- 172.21.15.5:0/2632492824 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2d04042030 con 0x7f2d14101de0 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d8b40 0x5569135d7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135e4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55691353d2f0 con 0x5569135d75d0 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5569134e4cf0 con 0x5569135e2370 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5569135392b0 con 0x5569135d8b40 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135e4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135e4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47272/0 (socket says 172.21.15.5:47272) 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135d7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56898/0 (socket says 172.21.15.5:56898) 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1259c640 1 -- 172.21.15.5:0/2149075493 learned_addr learned my addr 172.21.15.5:0/2149075493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1359e640 1 --2- 172.21.15.5:0/2149075493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d8b40 0x5569135d7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1259c640 1 -- 172.21.15.5:0/2149075493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d8b40 msgr2=0x5569135d7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1259c640 1 --2- 172.21.15.5:0/2149075493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d8b40 0x5569135d7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1259c640 1 -- 172.21.15.5:0/2149075493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 msgr2=0x5569135d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1259c640 1 --2- 172.21.15.5:0/2149075493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135d7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1259c640 1 -- 172.21.15.5:0/2149075493 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569134c93a0 con 0x5569135e2370 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2d1259c640 1 --2- 172.21.15.5:0/2149075493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135e4760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2d080114f0 tx=0x7f2d080119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccaea3fc0868bd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2cfbfff640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2cfc0a2cf0 0x7f2cfc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.196+0000 7f2cf97fa640 1 -- 172.21.15.5:0/2149075493 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0801e070 con 0x5569135e2370 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2cf97fa640 1 -- 172.21.15.5:0/2149075493 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d080035f0 con 0x5569135e2370 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2cf97fa640 1 -- 172.21.15.5:0/2149075493 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0800f820 con 0x5569135e2370 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1359e640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2cfc0a2cf0 0x7f2cfc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2cfbfff640 1 -- 172.21.15.5:0/2632492824 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f2d040ef810 con 0x7f2d14101de0 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/2149075493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 msgr2=0x5569135e4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1359e640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2cfc0a2cf0 0x7f2cfc0a51b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005fd0 tx=0x7f2d0c009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/2149075493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135e4760 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2d080114f0 tx=0x7f2d080119b0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/2149075493 shutdown_connections 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/2149075493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135e4760 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/2149075493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/2149075493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d8b40 0x5569135d7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/2149075493 >> 172.21.15.5:0/2149075493 conn(0x5569135a0310 msgr2=0x5569135d3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/2149075493 shutdown_connections 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.197+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/2149075493 wait complete. 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1b4bb740 1 -- start start 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1b4bb740 1 Processor -- start 2024-09-11T22:31:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d75d0 0x5569135e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 0x5569135defc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135df560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5569135e5070 con 0x5569135d8b40 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5569134f8060 con 0x5569135e2370 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5569135e51f0 con 0x5569135d75d0 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1b4bb740 10 client.0 _ll_register_callbacks cb 0x55691324b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 0x5569135defc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1359e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135df560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 0x5569135defc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56900/0 (socket says 172.21.15.5:56900) 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1259c640 1 -- 172.21.15.5:0/1570326730 learned_addr learned my addr 172.21.15.5:0/1570326730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d75d0 0x5569135e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1259c640 1 -- 172.21.15.5:0/1570326730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d75d0 msgr2=0x5569135e1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1259c640 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d75d0 0x5569135e1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1259c640 1 -- 172.21.15.5:0/1570326730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 msgr2=0x5569135df560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.198+0000 7f2d1259c640 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135df560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.199+0000 7f2d1259c640 1 -- 172.21.15.5:0/1570326730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569132ae3c0 con 0x5569135d8b40 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.199+0000 7f2d1259c640 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 0x5569135defc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2d08011eb0 tx=0x7f2d08003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f024ad283170cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.199+0000 7f2cf6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.199+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/1570326730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0801e040 con 0x5569135d8b40 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.199+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/1570326730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f2d08005040 con 0x5569135d8b40 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.199+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/1570326730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d080035f0 con 0x5569135d8b40 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.199+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/1570326730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2d08032050 con 0x5569135d8b40 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.199+0000 7f2d1b4bb740 10 client.18249 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.199+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/1570326730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5569135f4ed0 con 0x5569135d8b40 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.200+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/1570326730 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d08029070 con 0x5569135d8b40 2024-09-11T22:31:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.200+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/1570326730 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f2d08021030 con 0x5569135d8b40 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.200+0000 7f2d1b4bb740 10 client.18249 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.200+0000 7f2d1b4bb740 10 client.18249 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.200+0000 7f2d1b4bb740 20 client.18249 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.200+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5569135f62d0 0x5569135f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.200+0000 7f2d1b4bb740 4 client.18249 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.200+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/1570326730 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2d080035f0 con 0x5569135f62d0 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.200+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5569135f62d0 0x5569135f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.201+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5569135f62d0 0x5569135f86c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7db04a308aee4e56 server_cookie=7ce1b39f182a52f3 in_seq=0 out_seq=0 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.201+0000 7f2cf6ff5640 10 client.18249 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.202+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/1570326730 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2d080035f0 con 0x5569135f62d0 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.202+0000 7f2cf6ff5640 10 client.18249 handle_command_reply: tid=0 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.202+0000 7f2d1b4bb740 1 client.18249 shutdown 2024-09-11T22:31:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.202+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/1570326730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 msgr2=0x5569135defc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.202+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 0x5569135defc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f2d08011eb0 tx=0x7f2d08003a50 comp rx=0 tx=0).stop 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.203+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/1570326730 shutdown_connections 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.203+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5569135f62d0 0x5569135f86c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.203+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135df560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.203+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 0x5569135defc0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.203+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/1570326730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d75d0 0x5569135e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.203+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/1570326730 >> 172.21.15.5:0/1570326730 conn(0x5569135a0310 msgr2=0x5569135d9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.203+0000 7f2cf6ff5640 0 client.18249 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.203+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/1570326730 shutdown_connections 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.203+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/1570326730 wait complete. 2024-09-11T22:31:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.203+0000 7f2d1b4bb740 20 client.18249 trim_cache size 0 max 16384 2024-09-11T22:31:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d1b4bb740 1 Processor -- start 2024-09-11T22:31:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d1b4bb740 1 -- start start 2024-09-11T22:31:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135d75d0 0x5569135da120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 0x5569135da660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135e2370 0x5569135fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ab8c0 con 0x5569135d8b40 2024-09-11T22:31:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ad400 con 0x5569135d75d0 2024-09-11T22:31:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141abf90 con 0x5569135e2370 2024-09-11T22:31:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135d75d0 0x5569135da120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d1259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 0x5569135da660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135d75d0 0x5569135da120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47300/0 (socket says 172.21.15.5:47300) 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.218+0000 7f2d12d9d640 1 -- 172.21.15.5:0/514303051 learned_addr learned my addr 172.21.15.5:0/514303051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2d1359e640 1 --2- 172.21.15.5:0/514303051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135e2370 0x5569135fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2d12d9d640 1 -- 172.21.15.5:0/514303051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135e2370 msgr2=0x5569135fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/514303051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135e2370 0x5569135fde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2d12d9d640 1 -- 172.21.15.5:0/514303051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 msgr2=0x5569135da660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/514303051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 0x5569135da660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2d12d9d640 1 -- 172.21.15.5:0/514303051 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569135e1930 con 0x5569135d75d0 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2d1359e640 1 --2- 172.21.15.5:0/514303051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135e2370 0x5569135fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/514303051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135d75d0 0x5569135da120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d040b8740 tx=0x7f2d040efbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29ea47a780641b36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2cf8ff9640 1 -- 172.21.15.5:0/514303051 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04049040 con 0x5569135d75d0 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2cf8ff9640 1 -- 172.21.15.5:0/514303051 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d040bbd70 con 0x5569135d75d0 2024-09-11T22:31:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.219+0000 7f2cf8ff9640 1 -- 172.21.15.5:0/514303051 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d040ba100 con 0x5569135d75d0 2024-09-11T22:31:14.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.220+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/514303051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135d75d0 msgr2=0x5569135da120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.220+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/514303051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135d75d0 0x5569135da120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d040b8740 tx=0x7f2d040efbd0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.220+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/514303051 shutdown_connections 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.220+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/514303051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135e2370 0x5569135fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.220+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/514303051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d8b40 0x5569135da660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.220+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/514303051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135d75d0 0x5569135da120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.220+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/514303051 >> 172.21.15.5:0/514303051 conn(0x5569135a0310 msgr2=0x5569135e1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.220+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/514303051 shutdown_connections 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.220+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/514303051 wait complete. 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.220+0000 7f2d1b4bb740 1 -- start start 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d1b4bb740 1 Processor -- start 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d8b40 0x5569135dbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d1b4bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135dc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141acd30 con 0x5569135d75d0 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141ac660 con 0x5569135e2370 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d1b4bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2d141a98c0 con 0x5569135d8b40 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d1b4bb740 10 client.0 _ll_register_callbacks cb 0x7f2d0801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d12d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135eaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56910/0 (socket says 172.21.15.5:56910) 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d12d9d640 1 -- 172.21.15.5:0/3771616322 learned_addr learned my addr 172.21.15.5:0/3771616322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d1359e640 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135dc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d1259c640 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d8b40 0x5569135dbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d12d9d640 1 -- 172.21.15.5:0/3771616322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d8b40 msgr2=0x5569135dbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d8b40 0x5569135dbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d12d9d640 1 -- 172.21.15.5:0/3771616322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 msgr2=0x5569135dc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135dc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.221+0000 7f2d12d9d640 1 -- 172.21.15.5:0/3771616322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569135dc8b0 con 0x5569135d75d0 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2d12d9d640 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135eaac0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2d0403d040 tx=0x7f2d040bbc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e426ee375f2abd0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2cf6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/3771616322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d04049070 con 0x5569135d75d0 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/3771616322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f2d04003cd0 con 0x5569135d75d0 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/3771616322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d040bbd70 con 0x5569135d75d0 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/3771616322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2d040c0070 con 0x5569135d75d0 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2d1b4bb740 10 client.18255 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/3771616322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556913660d80 con 0x5569135d75d0 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/3771616322 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d040bbd70 con 0x5569135d75d0 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/3771616322 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f2d040bc060 con 0x5569135d75d0 2024-09-11T22:31:14.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2d1b4bb740 10 client.18255 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2d1b4bb740 10 client.18255 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2d1b4bb740 20 client.18255 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556913661420 0x556913663840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2d1b4bb740 4 client.18255 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.222+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/3771616322 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55691354a860 con 0x556913661420 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.223+0000 7f2d1259c640 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556913661420 0x556913663840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.223+0000 7f2d1259c640 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556913661420 0x556913663840 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4139668cba12d39d server_cookie=472c965b63d5988e in_seq=0 out_seq=0 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.223+0000 7f2cf6ff5640 10 client.18255 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.223+0000 7f2cf6ff5640 1 -- 172.21.15.5:0/3771616322 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55691354a860 con 0x556913661420 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.223+0000 7f2cf6ff5640 10 client.18255 handle_command_reply: tid=0 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.223+0000 7f2d1b4bb740 1 client.18255 shutdown 2024-09-11T22:31:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.223+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/3771616322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 msgr2=0x5569135eaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.223+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135eaac0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f2d0403d040 tx=0x7f2d040bbc50 comp rx=0 tx=0).stop 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/3771616322 shutdown_connections 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x556913661420 0x556913663840 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5569135e2370 0x5569135dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5569135d8b40 0x5569135dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d1b4bb740 1 --2- 172.21.15.5:0/3771616322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5569135d75d0 0x5569135eaac0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2cf6ff5640 0 client.18255 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/3771616322 >> 172.21.15.5:0/3771616322 conn(0x5569135a0310 msgr2=0x5569135d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/3771616322 shutdown_connections 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d1b4bb740 1 -- 172.21.15.5:0/3771616322 wait complete. 2024-09-11T22:31:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d1b4bb740 20 client.18255 trim_cache size 0 max 16384 2024-09-11T22:31:14.230 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:31:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d190e2640 1 -- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2cfc0a2cf0 msgr2=0x7f2cfc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d190e2640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2cfc0a2cf0 0x7f2cfc0a51b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005fd0 tx=0x7f2d0c009500 comp rx=0 tx=0).stop 2024-09-11T22:31:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d190e2640 1 -- 172.21.15.5:0/2632492824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d14101de0 msgr2=0x7f2d14198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d190e2640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d14101de0 0x7f2d14198510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2d04006010 tx=0x7f2d04004040 comp rx=0 tx=0).stop 2024-09-11T22:31:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d190e2640 1 -- 172.21.15.5:0/2632492824 shutdown_connections 2024-09-11T22:31:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d190e2640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d14106f10 0x7f2d1419cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d190e2640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d14102790 0x7f2d14198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d190e2640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2cfc0a2cf0 0x7f2cfc0a51b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d190e2640 1 --2- 172.21.15.5:0/2632492824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d14101de0 0x7f2d14198510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.224+0000 7f2d190e2640 1 -- 172.21.15.5:0/2632492824 >> 172.21.15.5:0/2632492824 conn(0x7f2d140fd8c0 msgr2=0x7f2d140fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.225+0000 7f2d190e2640 1 -- 172.21.15.5:0/2632492824 shutdown_connections 2024-09-11T22:31:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:14.225+0000 7f2d190e2640 1 -- 172.21.15.5:0/2632492824 wait complete. 2024-09-11T22:31:14.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 668874 7.42 MB/sec execute 332 sec latency 20.401 ms 2024-09-11T22:31:15.101 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:31:15.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 671378 7.45 MB/sec execute 333 sec latency 14.029 ms 2024-09-11T22:31:15.362 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:15 smithi005 ceph-mon[30943]: pgmap v284: 97 pgs: 97 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-09-11T22:31:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:15 smithi159 ceph-mon[35534]: pgmap v284: 97 pgs: 97 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-09-11T22:31:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:15 smithi172 ceph-mon[35461]: pgmap v284: 97 pgs: 97 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-09-11T22:31:16.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 -- 172.21.15.5:0/3842289060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f858c103760 msgr2=0x7f858c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:16.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 --2- 172.21.15.5:0/3842289060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f858c103760 0x7f858c107ba0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f857c009930 tx=0x7f857c02a420 comp rx=0 tx=0).stop 2024-09-11T22:31:16.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 -- 172.21.15.5:0/3842289060 shutdown_connections 2024-09-11T22:31:16.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 --2- 172.21.15.5:0/3842289060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f858c1082e0 0x7f858c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 --2- 172.21.15.5:0/3842289060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f858c103760 0x7f858c107ba0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 --2- 172.21.15.5:0/3842289060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f858c102db0 0x7f858c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 -- 172.21.15.5:0/3842289060 >> 172.21.15.5:0/3842289060 conn(0x7f858c0fc7d0 msgr2=0x7f858c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 -- 172.21.15.5:0/3842289060 shutdown_connections 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 -- 172.21.15.5:0/3842289060 wait complete. 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 Processor -- start 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.111+0000 7f8593d2c640 1 -- start start 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f8593d2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f858c102db0 0x7f858c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f8593d2c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f858c103760 0x7f858c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f8593d2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f858c1082e0 0x7f858c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f8593d2c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f858c112050 con 0x7f858c1082e0 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f8593d2c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f858c111ed0 con 0x7f858c103760 2024-09-11T22:31:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f8593d2c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f858c1121d0 con 0x7f858c102db0 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f8591ac8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f858c102db0 0x7f858c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85922c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f858c1082e0 0x7f858c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85912c7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f858c103760 0x7f858c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85922c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f858c1082e0 0x7f858c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56918/0 (socket says 172.21.15.5:56918) 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f8591ac8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f858c102db0 0x7f858c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33672/0 (socket says 172.21.15.5:33672) 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85922c9640 1 -- 172.21.15.5:0/2320329224 learned_addr learned my addr 172.21.15.5:0/2320329224 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85922c9640 1 -- 172.21.15.5:0/2320329224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f858c102db0 msgr2=0x7f858c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85922c9640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f858c102db0 0x7f858c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85922c9640 1 -- 172.21.15.5:0/2320329224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f858c103760 msgr2=0x7f858c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85922c9640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f858c103760 0x7f858c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85922c9640 1 -- 172.21.15.5:0/2320329224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858c19d560 con 0x7f858c1082e0 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f8591ac8640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f858c102db0 0x7f858c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85912c7640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f858c103760 0x7f858c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.112+0000 7f85922c9640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f858c1082e0 0x7f858c19ce80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f858400efd0 tx=0x7f858400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.113+0000 7f8582ffd640 1 -- 172.21.15.5:0/2320329224 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584003b70 con 0x7f858c1082e0 2024-09-11T22:31:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.113+0000 7f8593d2c640 1 -- 172.21.15.5:0/2320329224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858c19d850 con 0x7f858c1082e0 2024-09-11T22:31:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.113+0000 7f8582ffd640 1 -- 172.21.15.5:0/2320329224 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8584003d10 con 0x7f858c1082e0 2024-09-11T22:31:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.113+0000 7f8582ffd640 1 -- 172.21.15.5:0/2320329224 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85840109f0 con 0x7f858c1082e0 2024-09-11T22:31:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.113+0000 7f8593d2c640 1 -- 172.21.15.5:0/2320329224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f858c1a5130 con 0x7f858c1082e0 2024-09-11T22:31:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.115+0000 7f8582ffd640 1 -- 172.21.15.5:0/2320329224 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8584004020 con 0x7f858c1082e0 2024-09-11T22:31:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.115+0000 7f8596105740 1 Processor -- start 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f8596105740 1 -- start start 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f8596105740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c2b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f8596105740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f8596105740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f8596105740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4b912f0 con 0x55f4c4c2cb40 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f8596105740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4b38cf0 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f8596105740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4b8d2b0 con 0x55f4c4c36370 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f85912c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c2b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f85912c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c2b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56922/0 (socket says 172.21.15.5:56922) 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f85912c7640 1 -- 172.21.15.5:0/1664181808 learned_addr learned my addr 172.21.15.5:0/1664181808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f8591ac8640 1 --2- 172.21.15.5:0/1664181808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f85912c7640 1 --2- 172.21.15.5:0/1664181808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f8582ffd640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85780a2d70 0x7f85780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f85912c7640 1 -- 172.21.15.5:0/1664181808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 msgr2=0x55f4c4c38760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f85912c7640 1 --2- 172.21.15.5:0/1664181808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c38760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f85912c7640 1 -- 172.21.15.5:0/1664181808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 msgr2=0x55f4c4c2ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f85912c7640 1 --2- 172.21.15.5:0/1664181808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c2ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f85912c7640 1 -- 172.21.15.5:0/1664181808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c4b1d3a0 con 0x55f4c4c2cb40 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.116+0000 7f85922c9640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85780a2d70 0x7f85780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.117+0000 7f8582ffd640 1 -- 172.21.15.5:0/2320329224 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f85840cf800 con 0x7f858c1082e0 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.117+0000 7f85912c7640 1 --2- 172.21.15.5:0/1664181808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c2b090 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f857c005ec0 tx=0x7f857c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6518e7f377e656d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.117+0000 7f8577fff640 1 -- 172.21.15.5:0/1664181808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c035040 con 0x55f4c4c2cb40 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.117+0000 7f8577fff640 1 -- 172.21.15.5:0/1664181808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c003e50 con 0x55f4c4c2cb40 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.117+0000 7f8577fff640 1 -- 172.21.15.5:0/1664181808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c03d3f0 con 0x55f4c4c2cb40 2024-09-11T22:31:16.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 -- 172.21.15.5:0/1664181808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 msgr2=0x55f4c4c2b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f85922c9640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85780a2d70 0x7f85780a5230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f85840ccf90 tx=0x7f858406abe0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 --2- 172.21.15.5:0/1664181808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c2b090 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f857c005ec0 tx=0x7f857c005fd0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 -- 172.21.15.5:0/1664181808 shutdown_connections 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 --2- 172.21.15.5:0/1664181808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 --2- 172.21.15.5:0/1664181808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c2ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 --2- 172.21.15.5:0/1664181808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c2b090 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 -- 172.21.15.5:0/1664181808 >> 172.21.15.5:0/1664181808 conn(0x55f4c4bf4310 msgr2=0x55f4c4c27c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 -- 172.21.15.5:0/1664181808 shutdown_connections 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 -- 172.21.15.5:0/1664181808 wait complete. 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 -- start start 2024-09-11T22:31:16.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.118+0000 7f8596105740 1 Processor -- start 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8596105740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c35da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8596105740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c33050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8596105740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8596105740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4c38fb0 con 0x55f4c4c2cb40 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8596105740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4b4c060 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8596105740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4c4c39130 con 0x55f4c4c36370 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8596105740 10 client.0 _ll_register_callbacks cb 0x55f4c489f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8591ac8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c33050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f85912c7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c35da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8591ac8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c33050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56936/0 (socket says 172.21.15.5:56936) 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8591ac8640 1 -- 172.21.15.5:0/1484600538 learned_addr learned my addr 172.21.15.5:0/1484600538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f85912c7640 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8591ac8640 1 -- 172.21.15.5:0/1484600538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 msgr2=0x55f4c4c337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8591ac8640 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c337e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8591ac8640 1 -- 172.21.15.5:0/1484600538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 msgr2=0x55f4c4c35da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8591ac8640 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c35da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8591ac8640 1 -- 172.21.15.5:0/1484600538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c49023c0 con 0x55f4c4c2cb40 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f8591ac8640 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c33050 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8588012d20 tx=0x7f858800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189e70883162bbdd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.119+0000 7f85757fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f85757fa640 1 -- 172.21.15.5:0/1484600538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588012110 con 0x55f4c4c2cb40 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f85757fa640 1 -- 172.21.15.5:0/1484600538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f8588004050 con 0x55f4c4c2cb40 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f8596105740 1 -- 172.21.15.5:0/1484600538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f857c003e50 con 0x55f4c4c2cb40 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f85757fa640 1 -- 172.21.15.5:0/1484600538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f85880122b0 con 0x55f4c4c2cb40 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f8596105740 10 client.18267 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f8596105740 1 -- 172.21.15.5:0/1484600538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f4c4c44af0 con 0x55f4c4c2cb40 2024-09-11T22:31:16.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f85757fa640 1 -- 172.21.15.5:0/1484600538 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c003e50 con 0x55f4c4c2cb40 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f85757fa640 1 -- 172.21.15.5:0/1484600538 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f85880104c0 con 0x55f4c4c2cb40 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f8596105740 10 client.18267 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f8596105740 10 client.18267 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f8596105740 20 client.18267 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f8596105740 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55f4c4c45fb0 0x55f4c4c483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f8596105740 4 client.18267 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f8596105740 1 -- 172.21.15.5:0/1484600538 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4c4b9e860 con 0x55f4c4c45fb0 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.120+0000 7f85912c7640 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55f4c4c45fb0 0x55f4c4c483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.121+0000 7f85757fa640 10 client.18267 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.121+0000 7f85912c7640 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55f4c4c45fb0 0x55f4c4c483e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f49add53f41e88e server_cookie=922383e0eac2a37c in_seq=0 out_seq=0 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.121+0000 7f85757fa640 1 -- 172.21.15.5:0/1484600538 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4c4b9e860 con 0x55f4c4c45fb0 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f85757fa640 10 client.18267 handle_command_reply: tid=0 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 client.18267 shutdown 2024-09-11T22:31:16.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 -- 172.21.15.5:0/1484600538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 msgr2=0x55f4c4c33050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c33050 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f8588012d20 tx=0x7f858800f790 comp rx=0 tx=0).stop 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 -- 172.21.15.5:0/1484600538 shutdown_connections 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55f4c4c45fb0 0x55f4c4c483e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c33050 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 --2- 172.21.15.5:0/1484600538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c35da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f85757fa640 0 client.18267 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 -- 172.21.15.5:0/1484600538 >> 172.21.15.5:0/1484600538 conn(0x55f4c4bf4310 msgr2=0x55f4c4c2d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 -- 172.21.15.5:0/1484600538 shutdown_connections 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 1 -- 172.21.15.5:0/1484600538 wait complete. 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.122+0000 7f8596105740 20 client.18267 trim_cache size 0 max 16384 2024-09-11T22:31:16.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.135+0000 7f8596105740 1 Processor -- start 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f8596105740 1 -- start start 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f8596105740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c35da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f8596105740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c32910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f8596105740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c36370 0x55f4c4c4dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f8596105740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f858c1ad930 con 0x55f4c4c36370 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f8596105740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f858c1af470 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f8596105740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f858c1ae000 con 0x55f4c4c2cb40 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85912c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c36370 0x55f4c4c4dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85912c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c36370 0x55f4c4c4dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56944/0 (socket says 172.21.15.5:56944) 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85912c7640 1 -- 172.21.15.5:0/575386387 learned_addr learned my addr 172.21.15.5:0/575386387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85912c7640 1 --2- 172.21.15.5:0/575386387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c35da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f8591ac8640 1 --2- 172.21.15.5:0/575386387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c32910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85912c7640 1 -- 172.21.15.5:0/575386387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c2cb40 msgr2=0x55f4c4c32910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85912c7640 1 --2- 172.21.15.5:0/575386387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c32910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85912c7640 1 -- 172.21.15.5:0/575386387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 msgr2=0x55f4c4c35da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85912c7640 1 --2- 172.21.15.5:0/575386387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c35da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85912c7640 1 -- 172.21.15.5:0/575386387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4c4c42db0 con 0x55f4c4c36370 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85912c7640 1 --2- 172.21.15.5:0/575386387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c36370 0x55f4c4c4dcf0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f857c02a460 tx=0x7f857c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfc06224e003b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.136+0000 7f85777fe640 1 -- 172.21.15.5:0/575386387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c035040 con 0x55f4c4c36370 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f85777fe640 1 -- 172.21.15.5:0/575386387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c031780 con 0x55f4c4c36370 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f85777fe640 1 -- 172.21.15.5:0/575386387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c04e820 con 0x55f4c4c36370 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f8596105740 1 -- 172.21.15.5:0/575386387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c36370 msgr2=0x55f4c4c4dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f8596105740 1 --2- 172.21.15.5:0/575386387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c36370 0x55f4c4c4dcf0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f857c02a460 tx=0x7f857c005b50 comp rx=0 tx=0).stop 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f8596105740 1 -- 172.21.15.5:0/575386387 shutdown_connections 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f8596105740 1 --2- 172.21.15.5:0/575386387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c36370 0x55f4c4c4dcf0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f8596105740 1 --2- 172.21.15.5:0/575386387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c32910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f8596105740 1 --2- 172.21.15.5:0/575386387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c35da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f8596105740 1 -- 172.21.15.5:0/575386387 >> 172.21.15.5:0/575386387 conn(0x55f4c4bf4310 msgr2=0x55f4c4c31920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f8596105740 1 -- 172.21.15.5:0/575386387 shutdown_connections 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.137+0000 7f8596105740 1 -- 172.21.15.5:0/575386387 wait complete. 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f8596105740 1 -- start start 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f8596105740 1 Processor -- start 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f8596105740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c3ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f8596105740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c39f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f8596105740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c3a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f8596105740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f858c1aeda0 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f8596105740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f858c1ae6d0 con 0x55f4c4c2cb40 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f8596105740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f858c1ab930 con 0x55f4c4c36370 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f8596105740 10 client.0 _ll_register_callbacks cb 0x7f8588009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f85912c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c3ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f85912c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c3ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56946/0 (socket says 172.21.15.5:56946) 2024-09-11T22:31:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f85912c7640 1 -- 172.21.15.5:0/1626282000 learned_addr learned my addr 172.21.15.5:0/1626282000 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.138+0000 7f8591ac8640 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c39f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85912c7640 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c3a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85912c7640 1 -- 172.21.15.5:0/1626282000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 msgr2=0x55f4c4c3a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85912c7640 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c3a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85912c7640 1 -- 172.21.15.5:0/1626282000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2cb40 msgr2=0x55f4c4c39f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85912c7640 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c39f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85912c7640 1 -- 172.21.15.5:0/1626282000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4c4c3ad20 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f8591ac8640 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c39f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85912c7640 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c3ec00 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f857c004650 tx=0x7f857c050040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56978af89420c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85757fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85757fa640 1 -- 172.21.15.5:0/1626282000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c035030 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85757fa640 1 -- 172.21.15.5:0/1626282000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f857c005260 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f8596105740 1 -- 172.21.15.5:0/1626282000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f857c031780 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85757fa640 1 -- 172.21.15.5:0/1626282000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f857c005460 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f8596105740 10 client.18273 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f8596105740 1 -- 172.21.15.5:0/1626282000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f4c4cb0da0 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.139+0000 7f85757fa640 1 -- 172.21.15.5:0/1626282000 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c031780 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f85757fa640 1 -- 172.21.15.5:0/1626282000 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f857c056030 con 0x55f4c4c2b5d0 2024-09-11T22:31:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f8596105740 10 client.18273 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f8596105740 10 client.18273 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:31:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f8596105740 20 client.18273 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f8596105740 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55f4c4cb0fd0 0x55f4c4cb33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f8596105740 4 client.18273 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:31:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f8596105740 1 -- 172.21.15.5:0/1626282000 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4c4b98d70 con 0x55f4c4cb0fd0 2024-09-11T22:31:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f8591ac8640 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55f4c4cb0fd0 0x55f4c4cb33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f8591ac8640 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55f4c4cb0fd0 0x55f4c4cb33c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cac948c61e6fbbbb server_cookie=d7e972afdf2ea4d6 in_seq=0 out_seq=0 2024-09-11T22:31:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f85757fa640 10 client.18273 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:31:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f85757fa640 1 -- 172.21.15.5:0/1626282000 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4c4b98d70 con 0x55f4c4cb0fd0 2024-09-11T22:31:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f85757fa640 10 client.18273 handle_command_reply: tid=0 2024-09-11T22:31:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.140+0000 7f8596105740 1 client.18273 shutdown 2024-09-11T22:31:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 1 -- 172.21.15.5:0/1626282000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2b5d0 msgr2=0x55f4c4c3ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c3ec00 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f857c004650 tx=0x7f857c050040 comp rx=0 tx=0).stop 2024-09-11T22:31:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 1 -- 172.21.15.5:0/1626282000 shutdown_connections 2024-09-11T22:31:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55f4c4cb0fd0 0x55f4c4cb33c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4c4c36370 0x55f4c4c3a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4c4c2cb40 0x55f4c4c39f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f85757fa640 0 client.18273 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:31:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 1 --2- 172.21.15.5:0/1626282000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4c4c2b5d0 0x55f4c4c3ec00 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 1 -- 172.21.15.5:0/1626282000 >> 172.21.15.5:0/1626282000 conn(0x55f4c4bf4310 msgr2=0x55f4c4c348b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 1 -- 172.21.15.5:0/1626282000 shutdown_connections 2024-09-11T22:31:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 1 -- 172.21.15.5:0/1626282000 wait complete. 2024-09-11T22:31:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8596105740 20 client.18273 trim_cache size 0 max 16384 2024-09-11T22:31:16.161 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:31:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8593d2c640 1 -- 172.21.15.5:0/2320329224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85780a2d70 msgr2=0x7f85780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8593d2c640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85780a2d70 0x7f85780a5230 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f85840ccf90 tx=0x7f858406abe0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8593d2c640 1 -- 172.21.15.5:0/2320329224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f858c1082e0 msgr2=0x7f858c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.141+0000 7f8593d2c640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f858c1082e0 0x7f858c19ce80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f858400efd0 tx=0x7f858400c6a0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.142+0000 7f8593d2c640 1 -- 172.21.15.5:0/2320329224 shutdown_connections 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.142+0000 7f8593d2c640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f858c1082e0 0x7f858c19ce80 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.142+0000 7f8593d2c640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85780a2d70 0x7f85780a5230 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.142+0000 7f8593d2c640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f858c103760 0x7f858c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.142+0000 7f8593d2c640 1 --2- 172.21.15.5:0/2320329224 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f858c102db0 0x7f858c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.142+0000 7f8593d2c640 1 -- 172.21.15.5:0/2320329224 >> 172.21.15.5:0/2320329224 conn(0x7f858c0fc7d0 msgr2=0x7f858c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.142+0000 7f8593d2c640 1 -- 172.21.15.5:0/2320329224 shutdown_connections 2024-09-11T22:31:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:16.142+0000 7f8593d2c640 1 -- 172.21.15.5:0/2320329224 wait complete. 2024-09-11T22:31:16.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 673133 7.44 MB/sec execute 334 sec latency 41.154 ms 2024-09-11T22:31:16.959 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:31:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:17.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 675340 7.46 MB/sec execute 335 sec latency 13.480 ms 2024-09-11T22:31:17.644 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:17 smithi005 ceph-mon[30943]: pgmap v285: 97 pgs: 97 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-11T22:31:17.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.730+0000 7fed84a3c640 1 -- 172.21.15.5:0/2764341153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed801027a0 msgr2=0x7fed801067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:17.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.730+0000 7fed84a3c640 1 --2- 172.21.15.5:0/2764341153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed801027a0 0x7fed801067f0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fed68009900 tx=0x7fed6802a410 comp rx=0 tx=0).stop 2024-09-11T22:31:17.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.730+0000 7fed84a3c640 1 -- 172.21.15.5:0/2764341153 shutdown_connections 2024-09-11T22:31:17.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.730+0000 7fed84a3c640 1 --2- 172.21.15.5:0/2764341153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed80106f20 0x7fed8010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.730+0000 7fed84a3c640 1 --2- 172.21.15.5:0/2764341153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed801027a0 0x7fed801067f0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.730+0000 7fed84a3c640 1 --2- 172.21.15.5:0/2764341153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed80101df0 0x7fed801021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.730+0000 7fed84a3c640 1 -- 172.21.15.5:0/2764341153 >> 172.21.15.5:0/2764341153 conn(0x7fed800fd8f0 msgr2=0x7fed800ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.730+0000 7fed84a3c640 1 -- 172.21.15.5:0/2764341153 shutdown_connections 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.730+0000 7fed84a3c640 1 -- 172.21.15.5:0/2764341153 wait complete. 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed84a3c640 1 Processor -- start 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed84a3c640 1 -- start start 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed84a3c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed80101df0 0x7fed8019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed84a3c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed801027a0 0x7fed8019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed84a3c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed80106f20 0x7fed8019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed84a3c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed80111f80 con 0x7fed80101df0 2024-09-11T22:31:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed84a3c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fed80111e00 con 0x7fed80106f20 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed84a3c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fed80112100 con 0x7fed801027a0 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed7e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed80101df0 0x7fed8019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed7e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed80101df0 0x7fed8019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56966/0 (socket says 172.21.15.5:56966) 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed7e59c640 1 -- 172.21.15.5:0/1148272055 learned_addr learned my addr 172.21.15.5:0/1148272055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed77fff640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed801027a0 0x7fed8019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed7ed9d640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed80106f20 0x7fed8019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed7e59c640 1 -- 172.21.15.5:0/1148272055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed801027a0 msgr2=0x7fed8019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed7e59c640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed801027a0 0x7fed8019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed7e59c640 1 -- 172.21.15.5:0/1148272055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed80106f20 msgr2=0x7fed8019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed7e59c640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed80106f20 0x7fed8019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed7e59c640 1 -- 172.21.15.5:0/1148272055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8019f6a0 con 0x7fed80101df0 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.731+0000 7fed7e59c640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed80101df0 0x7fed8019a6f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fed7000e9f0 tx=0x7fed7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.732+0000 7fed777fe640 1 -- 172.21.15.5:0/1148272055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7000cd80 con 0x7fed80101df0 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.732+0000 7fed84a3c640 1 -- 172.21.15.5:0/1148272055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8019f990 con 0x7fed80101df0 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.732+0000 7fed777fe640 1 -- 172.21.15.5:0/1148272055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed70010910 con 0x7fed80101df0 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.732+0000 7fed777fe640 1 -- 172.21.15.5:0/1148272055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed700188f0 con 0x7fed80101df0 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.732+0000 7fed84a3c640 1 -- 172.21.15.5:0/1148272055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed801a7270 con 0x7fed80101df0 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.734+0000 7fed777fe640 1 -- 172.21.15.5:0/1148272055 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fed70010ac0 con 0x7fed80101df0 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.734+0000 7fed86e15740 1 Processor -- start 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed86e15740 1 -- start start 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed86e15740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e77ab40 0x55749e779090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed86e15740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e779a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed86e15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e784370 0x55749e786760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed86e15740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55749e6df2f0 con 0x55749e77ab40 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed86e15740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55749e686cf0 con 0x55749e7795d0 2024-09-11T22:31:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed86e15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55749e6db2b0 con 0x55749e784370 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed777fe640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fed6c0a2cf0 0x7fed6c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed7e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e779a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e784370 0x55749e786760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed7ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e77ab40 0x55749e779090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed7e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e779a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47374/0 (socket says 172.21.15.5:47374) 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed7e59c640 1 -- 172.21.15.5:0/1773041706 learned_addr learned my addr 172.21.15.5:0/1773041706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed7ed9d640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fed6c0a2cf0 0x7fed6c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed777fe640 1 -- 172.21.15.5:0/1148272055 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fed700cf8b0 con 0x7fed80101df0 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed7e59c640 1 -- 172.21.15.5:0/1773041706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e784370 msgr2=0x55749e786760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed7e59c640 1 --2- 172.21.15.5:0/1773041706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e784370 0x55749e786760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed7e59c640 1 -- 172.21.15.5:0/1773041706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e77ab40 msgr2=0x55749e779090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.735+0000 7fed7e59c640 1 --2- 172.21.15.5:0/1773041706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e77ab40 0x55749e779090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.736+0000 7fed7e59c640 1 -- 172.21.15.5:0/1773041706 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749e66b3a0 con 0x55749e7795d0 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.736+0000 7fed7e59c640 1 --2- 172.21.15.5:0/1773041706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e779a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed7000e9c0 tx=0x7fed7006eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70df8fb1076391b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.736+0000 7fed7ed9d640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fed6c0a2cf0 0x7fed6c0a51b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fed78009990 tx=0x7fed780096a0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.736+0000 7fed74ff9640 1 -- 172.21.15.5:0/1773041706 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70098420 con 0x55749e7795d0 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.736+0000 7fed74ff9640 1 -- 172.21.15.5:0/1773041706 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed700cd1b0 con 0x55749e7795d0 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.736+0000 7fed74ff9640 1 -- 172.21.15.5:0/1773041706 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7009b460 con 0x55749e7795d0 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 -- 172.21.15.5:0/1773041706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 msgr2=0x55749e779a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 --2- 172.21.15.5:0/1773041706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e779a10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fed7000e9c0 tx=0x7fed7006eee0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 -- 172.21.15.5:0/1773041706 shutdown_connections 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 --2- 172.21.15.5:0/1773041706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e784370 0x55749e786760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 --2- 172.21.15.5:0/1773041706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e779a10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 --2- 172.21.15.5:0/1773041706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e77ab40 0x55749e779090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 -- 172.21.15.5:0/1773041706 >> 172.21.15.5:0/1773041706 conn(0x55749e742310 msgr2=0x55749e775c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 -- 172.21.15.5:0/1773041706 shutdown_connections 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 -- 172.21.15.5:0/1773041706 wait complete. 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 -- start start 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 Processor -- start 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e7795d0 0x55749e783d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.737+0000 7fed86e15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e781030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed86e15740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e784370 0x55749e781640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed86e15740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55749e787010 con 0x55749e7795d0 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed86e15740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55749e69a060 con 0x55749e784370 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed86e15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55749e787190 con 0x55749e77ab40 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed86e15740 10 client.0 _ll_register_callbacks cb 0x55749e3ed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed7e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e7795d0 0x55749e783d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e781030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e781030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33750/0 (socket says 172.21.15.5:33750) 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed7ed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e784370 0x55749e781640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed7e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e7795d0 0x55749e783d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56986/0 (socket says 172.21.15.5:56986) 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed77fff640 1 -- 172.21.15.5:0/2134017742 learned_addr learned my addr 172.21.15.5:0/2134017742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed77fff640 1 -- 172.21.15.5:0/2134017742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e784370 msgr2=0x55749e781640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed77fff640 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e784370 0x55749e781640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed77fff640 1 -- 172.21.15.5:0/2134017742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e7795d0 msgr2=0x55749e783d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed77fff640 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e7795d0 0x55749e783d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed77fff640 1 -- 172.21.15.5:0/2134017742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55749e4503c0 con 0x55749e77ab40 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed7e59c640 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e7795d0 0x55749e783d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed7ed9d640 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e784370 0x55749e781640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed77fff640 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e781030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fed6802a930 tx=0x7fed68031850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84824ed9fc06ac7b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:31:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed65ffb640 1 -- 172.21.15.5:0/2134017742 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68035040 con 0x55749e77ab40 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.738+0000 7fed86e15740 1 -- 172.21.15.5:0/2134017742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed700cd1b0 con 0x55749e77ab40 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed65ffb640 1 -- 172.21.15.5:0/2134017742 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fed68004240 con 0x55749e77ab40 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed65ffb640 1 -- 172.21.15.5:0/2134017742 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fed68004440 con 0x55749e77ab40 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed86e15740 10 client.27469 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed86e15740 1 -- 172.21.15.5:0/2134017742 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55749e792b30 con 0x55749e77ab40 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed65ffb640 1 -- 172.21.15.5:0/2134017742 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed700cd1b0 con 0x55749e77ab40 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed65ffb640 1 -- 172.21.15.5:0/2134017742 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fed6803d5c0 con 0x55749e77ab40 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed86e15740 10 client.27469 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed86e15740 10 client.27469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed86e15740 20 client.27469 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed86e15740 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55749e793ff0 0x55749e796460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed86e15740 4 client.27469 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed86e15740 1 -- 172.21.15.5:0/2134017742 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55749e6ec860 con 0x55749e793ff0 2024-09-11T22:31:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.739+0000 7fed7e59c640 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55749e793ff0 0x55749e796460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.740+0000 7fed7e59c640 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55749e793ff0 0x55749e796460 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3103ec9b2d3878a server_cookie=424612ee83f2cf05 in_seq=0 out_seq=0 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.740+0000 7fed65ffb640 10 client.27469 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.740+0000 7fed65ffb640 1 -- 172.21.15.5:0/2134017742 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55749e6ec860 con 0x55749e793ff0 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.740+0000 7fed65ffb640 10 client.27469 handle_command_reply: tid=0 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.740+0000 7fed86e15740 1 client.27469 shutdown 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.740+0000 7fed86e15740 1 -- 172.21.15.5:0/2134017742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 msgr2=0x55749e781030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.740+0000 7fed86e15740 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e781030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fed6802a930 tx=0x7fed68031850 comp rx=0 tx=0).stop 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.741+0000 7fed86e15740 1 -- 172.21.15.5:0/2134017742 shutdown_connections 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.741+0000 7fed86e15740 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55749e793ff0 0x55749e796460 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.741+0000 7fed86e15740 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e784370 0x55749e781640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.741+0000 7fed86e15740 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e781030 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.741+0000 7fed65ffb640 0 client.27469 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.741+0000 7fed86e15740 1 --2- 172.21.15.5:0/2134017742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e7795d0 0x55749e783d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.741+0000 7fed86e15740 1 -- 172.21.15.5:0/2134017742 >> 172.21.15.5:0/2134017742 conn(0x55749e742310 msgr2=0x55749e77b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.741+0000 7fed86e15740 1 -- 172.21.15.5:0/2134017742 shutdown_connections 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.741+0000 7fed86e15740 1 -- 172.21.15.5:0/2134017742 wait complete. 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.741+0000 7fed86e15740 20 client.27469 trim_cache size 0 max 16384 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed86e15740 1 Processor -- start 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed86e15740 1 -- start start 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed86e15740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e782fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed86e15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e77c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed86e15740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed86e15740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed801afc30 con 0x55749e784370 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed86e15740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fed801b1770 con 0x55749e7795d0 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed86e15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fed801b0300 con 0x55749e77ab40 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed7ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed7ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56988/0 (socket says 172.21.15.5:56988) 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed7ed9d640 1 -- 172.21.15.5:0/4204287809 learned_addr learned my addr 172.21.15.5:0/4204287809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed7e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e782fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.754+0000 7fed77fff640 1 --2- 172.21.15.5:0/4204287809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e77c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.755+0000 7fed7ed9d640 1 -- 172.21.15.5:0/4204287809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 msgr2=0x55749e77c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.755+0000 7fed7ed9d640 1 --2- 172.21.15.5:0/4204287809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e77c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.755+0000 7fed7ed9d640 1 -- 172.21.15.5:0/4204287809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 msgr2=0x55749e782fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.755+0000 7fed7ed9d640 1 --2- 172.21.15.5:0/4204287809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e782fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.755+0000 7fed7ed9d640 1 -- 172.21.15.5:0/4204287809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55749e77cc30 con 0x55749e784370 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.755+0000 7fed7ed9d640 1 --2- 172.21.15.5:0/4204287809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77c6f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fed780039a0 tx=0x7fed78009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01d03ce215834f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.755+0000 7fed67fff640 1 -- 172.21.15.5:0/4204287809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78004b20 con 0x55749e784370 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.755+0000 7fed67fff640 1 -- 172.21.15.5:0/4204287809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed78004cc0 con 0x55749e784370 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.755+0000 7fed67fff640 1 -- 172.21.15.5:0/4204287809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed78004670 con 0x55749e784370 2024-09-11T22:31:17.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 -- 172.21.15.5:0/4204287809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 msgr2=0x55749e77c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 --2- 172.21.15.5:0/4204287809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77c6f0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fed780039a0 tx=0x7fed78009920 comp rx=0 tx=0).stop 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 -- 172.21.15.5:0/4204287809 shutdown_connections 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 --2- 172.21.15.5:0/4204287809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77c6f0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 --2- 172.21.15.5:0/4204287809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e77c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 --2- 172.21.15.5:0/4204287809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e782fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 -- 172.21.15.5:0/4204287809 >> 172.21.15.5:0/4204287809 conn(0x55749e742310 msgr2=0x55749e782890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 -- 172.21.15.5:0/4204287809 shutdown_connections 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 -- 172.21.15.5:0/4204287809 wait complete. 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 -- start start 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 Processor -- start 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e78cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e77de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed801b10a0 con 0x55749e784370 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fed801b09d0 con 0x55749e7795d0 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fed801adc30 con 0x55749e77ab40 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed86e15740 10 client.0 _ll_register_callbacks cb 0x7fed68005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed7e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e78cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.756+0000 7fed7ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed7e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e78cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47408/0 (socket says 172.21.15.5:47408) 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed7ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56992/0 (socket says 172.21.15.5:56992) 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed77fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e77de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed7e59c640 1 -- 172.21.15.5:0/447614572 learned_addr learned my addr 172.21.15.5:0/447614572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed77fff640 1 -- 172.21.15.5:0/447614572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 msgr2=0x55749e78cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed77fff640 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e78cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed77fff640 1 -- 172.21.15.5:0/447614572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 msgr2=0x55749e77e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:17.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed77fff640 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed77fff640 1 -- 172.21.15.5:0/447614572 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55749e77ec50 con 0x55749e77ab40 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed7ed9d640 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed77fff640 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e77de50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fed6803d990 tx=0x7fed68031af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d91bbf51f7cfae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed65ffb640 1 -- 172.21.15.5:0/447614572 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed68035040 con 0x55749e77ab40 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed86e15740 1 -- 172.21.15.5:0/447614572 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed78004cc0 con 0x55749e77ab40 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed65ffb640 1 -- 172.21.15.5:0/447614572 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fed68005040 con 0x55749e77ab40 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed65ffb640 1 -- 172.21.15.5:0/447614572 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fed68062050 con 0x55749e77ab40 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed86e15740 10 client.27475 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.757+0000 7fed86e15740 1 -- 172.21.15.5:0/447614572 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55749e7fee00 con 0x55749e77ab40 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed65ffb640 1 -- 172.21.15.5:0/447614572 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed78004cc0 con 0x55749e77ab40 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed65ffb640 1 -- 172.21.15.5:0/447614572 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fed6805d030 con 0x55749e77ab40 2024-09-11T22:31:17.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed86e15740 10 client.27475 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed86e15740 10 client.27475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed86e15740 20 client.27475 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed86e15740 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55749e793ff0 0x55749e8015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed86e15740 4 client.27475 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed86e15740 1 -- 172.21.15.5:0/447614572 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55749e6e6d70 con 0x55749e793ff0 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed7e59c640 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55749e793ff0 0x55749e8015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed7e59c640 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55749e793ff0 0x55749e8015e0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=809a41021bdd2009 server_cookie=ee1ecbb797298fee in_seq=0 out_seq=0 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.758+0000 7fed65ffb640 10 client.27475 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed65ffb640 1 -- 172.21.15.5:0/447614572 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55749e6e6d70 con 0x55749e793ff0 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed65ffb640 10 client.27475 handle_command_reply: tid=0 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 client.27475 shutdown 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 -- 172.21.15.5:0/447614572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 msgr2=0x55749e77de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e77de50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fed6803d990 tx=0x7fed68031af0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 -- 172.21.15.5:0/447614572 shutdown_connections 2024-09-11T22:31:17.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55749e793ff0 0x55749e8015e0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55749e784370 0x55749e77e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55749e77ab40 0x55749e77de50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed65ffb640 0 client.27475 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:17.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 --2- 172.21.15.5:0/447614572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55749e7795d0 0x55749e78cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 -- 172.21.15.5:0/447614572 >> 172.21.15.5:0/447614572 conn(0x55749e742310 msgr2=0x55749e7830b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:17.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 -- 172.21.15.5:0/447614572 shutdown_connections 2024-09-11T22:31:17.786 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 1 -- 172.21.15.5:0/447614572 wait complete. 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.759+0000 7fed86e15740 20 client.27475 trim_cache size 0 max 16384 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 -- 172.21.15.5:0/1148272055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fed6c0a2cf0 msgr2=0x7fed6c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fed6c0a2cf0 0x7fed6c0a51b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fed78009990 tx=0x7fed780096a0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 -- 172.21.15.5:0/1148272055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed80101df0 msgr2=0x7fed8019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed80101df0 0x7fed8019a6f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fed7000e9f0 tx=0x7fed7000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 -- 172.21.15.5:0/1148272055 shutdown_connections 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fed6c0a2cf0 0x7fed6c0a51b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed80106f20 0x7fed8019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed801027a0 0x7fed8019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 --2- 172.21.15.5:0/1148272055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed80101df0 0x7fed8019a6f0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 -- 172.21.15.5:0/1148272055 >> 172.21.15.5:0/1148272055 conn(0x7fed800fd8f0 msgr2=0x7fed800fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 -- 172.21.15.5:0/1148272055 shutdown_connections 2024-09-11T22:31:17.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:17.760+0000 7fed84a3c640 1 -- 172.21.15.5:0/1148272055 wait complete. 2024-09-11T22:31:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:17 smithi159 ceph-mon[35534]: pgmap v285: 97 pgs: 97 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-11T22:31:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:17 smithi172 ceph-mon[35461]: pgmap v285: 97 pgs: 97 active+clean; 94 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 220 op/s 2024-09-11T22:31:18.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 677645 7.47 MB/sec execute 336 sec latency 18.677 ms 2024-09-11T22:31:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.934+0000 7ff895952640 1 -- 172.21.15.5:0/1534050844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89010c0d0 msgr2=0x7ff89010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.934+0000 7ff895952640 1 --2- 172.21.15.5:0/1534050844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89010c0d0 0x7ff89010e550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff888009be0 tx=0x7ff88802a750 comp rx=0 tx=0).stop 2024-09-11T22:31:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.934+0000 7ff895952640 1 -- 172.21.15.5:0/1534050844 shutdown_connections 2024-09-11T22:31:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.934+0000 7ff895952640 1 --2- 172.21.15.5:0/1534050844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89010c0d0 0x7ff89010e550 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.934+0000 7ff895952640 1 --2- 172.21.15.5:0/1534050844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8901035a0 0x7ff89010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.934+0000 7ff895952640 1 --2- 172.21.15.5:0/1534050844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff890102c80 0x7ff890103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.934+0000 7ff895952640 1 -- 172.21.15.5:0/1534050844 >> 172.21.15.5:0/1534050844 conn(0x7ff890077990 msgr2=0x7ff8900fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.934+0000 7ff895952640 1 -- 172.21.15.5:0/1534050844 shutdown_connections 2024-09-11T22:31:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.934+0000 7ff895952640 1 -- 172.21.15.5:0/1534050844 wait complete. 2024-09-11T22:31:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff895952640 1 Processor -- start 2024-09-11T22:31:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff895952640 1 -- start start 2024-09-11T22:31:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff895952640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff890102c80 0x7ff890198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff895952640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8901035a0 0x7ff890198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff895952640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89010c0d0 0x7ff89019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff895952640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff890111e40 con 0x7ff8901035a0 2024-09-11T22:31:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff895952640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff890111cc0 con 0x7ff89010c0d0 2024-09-11T22:31:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff895952640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff890111fc0 con 0x7ff890102c80 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8901035a0 0x7ff890198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff88effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff890102c80 0x7ff890198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff88f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89010c0d0 0x7ff89019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8901035a0 0x7ff890198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57008/0 (socket says 172.21.15.5:57008) 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff88effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff890102c80 0x7ff890198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33792/0 (socket says 172.21.15.5:33792) 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.935+0000 7ff88e7fc640 1 -- 172.21.15.5:0/1001062491 learned_addr learned my addr 172.21.15.5:0/1001062491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff88f7fe640 1 -- 172.21.15.5:0/1001062491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff890102c80 msgr2=0x7ff890198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff88f7fe640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff890102c80 0x7ff890198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff88f7fe640 1 -- 172.21.15.5:0/1001062491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8901035a0 msgr2=0x7ff890198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff88f7fe640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8901035a0 0x7ff890198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff88f7fe640 1 -- 172.21.15.5:0/1001062491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff89019d340 con 0x7ff89010c0d0 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff88e7fc640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8901035a0 0x7ff890198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff88effd640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff890102c80 0x7ff890198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff88f7fe640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89010c0d0 0x7ff89019cc10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff888034040 tx=0x7ff8880305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff894950640 1 -- 172.21.15.5:0/1001062491 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888041020 con 0x7ff89010c0d0 2024-09-11T22:31:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff894950640 1 -- 172.21.15.5:0/1001062491 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff888037040 con 0x7ff89010c0d0 2024-09-11T22:31:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff895952640 1 -- 172.21.15.5:0/1001062491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff89019d5d0 con 0x7ff89010c0d0 2024-09-11T22:31:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff894950640 1 -- 172.21.15.5:0/1001062491 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888036070 con 0x7ff89010c0d0 2024-09-11T22:31:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.936+0000 7ff895952640 1 -- 172.21.15.5:0/1001062491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8901a4ec0 con 0x7ff89010c0d0 2024-09-11T22:31:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff894950640 1 -- 172.21.15.5:0/1001062491 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff888044020 con 0x7ff89010c0d0 2024-09-11T22:31:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff8987bf740 1 Processor -- start 2024-09-11T22:31:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff8987bf740 1 -- start start 2024-09-11T22:31:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab32af0 0x55b81ab32ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 0x55b81ab3d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab3dcb0 0x55b81ab400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff8987bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b81aa972f0 con 0x55b81ab32af0 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff8987bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b81aa3f270 con 0x55b81ab3dcb0 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff8987bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b81aa932b0 con 0x55b81ab33410 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab32af0 0x55b81ab32ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab32af0 0x55b81ab32ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57022/0 (socket says 172.21.15.5:57022) 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff88e7fc640 1 -- 172.21.15.5:0/1948181295 learned_addr learned my addr 172.21.15.5:0/1948181295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff88effd640 1 --2- 172.21.15.5:0/1948181295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 0x55b81ab3d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff88e7fc640 1 --2- 172.21.15.5:0/1948181295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab3dcb0 0x55b81ab400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.939+0000 7ff894950640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8780a2da0 0x7ff8780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff88effd640 1 -- 172.21.15.5:0/1948181295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab3dcb0 msgr2=0x55b81ab400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff88effd640 1 --2- 172.21.15.5:0/1948181295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab3dcb0 0x55b81ab400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff88effd640 1 -- 172.21.15.5:0/1948181295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab32af0 msgr2=0x55b81ab32ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff88effd640 1 --2- 172.21.15.5:0/1948181295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab32af0 0x55b81ab32ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff88effd640 1 -- 172.21.15.5:0/1948181295 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81aa3e340 con 0x55b81ab33410 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff88f7fe640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8780a2da0 0x7ff8780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff894950640 1 -- 172.21.15.5:0/1001062491 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7ff8880efe00 con 0x7ff89010c0d0 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff88effd640 1 --2- 172.21.15.5:0/1948181295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 0x55b81ab3d770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff880005fd0 tx=0x7ff8800118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9c345933f4e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff875ffb640 1 -- 172.21.15.5:0/1948181295 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880004020 con 0x55b81ab33410 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff875ffb640 1 -- 172.21.15.5:0/1948181295 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8800041c0 con 0x55b81ab33410 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff875ffb640 1 -- 172.21.15.5:0/1948181295 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff88000f8a0 con 0x55b81ab33410 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.940+0000 7ff88f7fe640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8780a2da0 0x7ff8780a5260 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ed590 tx=0x7ff88808e270 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 -- 172.21.15.5:0/1948181295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 msgr2=0x55b81ab3d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1948181295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 0x55b81ab3d770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff880005fd0 tx=0x7ff8800118d0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 -- 172.21.15.5:0/1948181295 shutdown_connections 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1948181295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab3dcb0 0x55b81ab400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1948181295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 0x55b81ab3d770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1948181295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab32af0 0x55b81ab32ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 -- 172.21.15.5:0/1948181295 >> 172.21.15.5:0/1948181295 conn(0x55b81ab2f060 msgr2=0x55b81ab2f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 -- 172.21.15.5:0/1948181295 shutdown_connections 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 -- 172.21.15.5:0/1948181295 wait complete. 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 -- start start 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.941+0000 7ff8987bf740 1 Processor -- start 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 0x55b81ab3ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab33410 0x55b81ab3b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab3dcb0 0x55b81ab3b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff8987bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b81ab40880 con 0x55b81ab33410 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff8987bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b81aa20290 con 0x55b81ab32af0 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff8987bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b81ab40a00 con 0x55b81ab3dcb0 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff8987bf740 10 client.0 _ll_register_callbacks cb 0x55b81aa51140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 0x55b81ab3ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab33410 0x55b81ab3b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 0x55b81ab3ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47440/0 (socket says 172.21.15.5:47440) 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88e7fc640 1 -- 172.21.15.5:0/1687582037 learned_addr learned my addr 172.21.15.5:0/1687582037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88e7fc640 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab3dcb0 0x55b81ab3b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88effd640 1 -- 172.21.15.5:0/1687582037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab3dcb0 msgr2=0x55b81ab3b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88effd640 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab3dcb0 0x55b81ab3b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88effd640 1 -- 172.21.15.5:0/1687582037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 msgr2=0x55b81ab3ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88effd640 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 0x55b81ab3ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88effd640 1 -- 172.21.15.5:0/1687582037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81a7aa210 con 0x55b81ab33410 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88e7fc640 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 0x55b81ab3ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff88effd640 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab33410 0x55b81ab3b0b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff880000c00 tx=0x7ff880003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b2d38ff3f7a3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff86effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff86effd640 1 -- 172.21.15.5:0/1687582037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880028020 con 0x55b81ab33410 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff86effd640 1 -- 172.21.15.5:0/1687582037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7ff880016040 con 0x55b81ab33410 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.942+0000 7ff8987bf740 1 -- 172.21.15.5:0/1687582037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8800041c0 con 0x55b81ab33410 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff86effd640 1 -- 172.21.15.5:0/1687582037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff880033050 con 0x55b81ab33410 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff8987bf740 10 client.18303 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff8987bf740 1 -- 172.21.15.5:0/1687582037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b81a876620 con 0x55b81ab33410 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff86effd640 1 -- 172.21.15.5:0/1687582037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8800041c0 con 0x55b81ab33410 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff86effd640 1 -- 172.21.15.5:0/1687582037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff8800049c0 con 0x55b81ab33410 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff8987bf740 10 client.18303 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff8987bf740 10 client.18303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff8987bf740 20 client.18303 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b81ab4dba0 0x55b81ab4ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff8987bf740 4 client.18303 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff8987bf740 1 -- 172.21.15.5:0/1687582037 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b81aaa4a20 con 0x55b81ab4dba0 2024-09-11T22:31:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.943+0000 7ff88e7fc640 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b81ab4dba0 0x55b81ab4ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.944+0000 7ff88e7fc640 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b81ab4dba0 0x55b81ab4ff90 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dbceeec86e0d87d server_cookie=8105799418ce30ab in_seq=0 out_seq=0 2024-09-11T22:31:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.944+0000 7ff86effd640 10 client.18303 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff86effd640 1 -- 172.21.15.5:0/1687582037 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b81aaa4a20 con 0x55b81ab4dba0 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff86effd640 10 client.18303 handle_command_reply: tid=0 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 client.18303 shutdown 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 -- 172.21.15.5:0/1687582037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab33410 msgr2=0x55b81ab3b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab33410 0x55b81ab3b0b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7ff880000c00 tx=0x7ff880003d30 comp rx=0 tx=0).stop 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 -- 172.21.15.5:0/1687582037 shutdown_connections 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b81ab4dba0 0x55b81ab4ff90 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab3dcb0 0x55b81ab3b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff86effd640 0 client.18303 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab33410 0x55b81ab3b0b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1687582037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 0x55b81ab3ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 -- 172.21.15.5:0/1687582037 >> 172.21.15.5:0/1687582037 conn(0x55b81ab2f060 msgr2=0x55b81ab34dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 -- 172.21.15.5:0/1687582037 shutdown_connections 2024-09-11T22:31:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 1 -- 172.21.15.5:0/1687582037 wait complete. 2024-09-11T22:31:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.946+0000 7ff8987bf740 20 client.18303 trim_cache size 0 max 16384 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.962+0000 7ff8987bf740 1 Processor -- start 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.962+0000 7ff8987bf740 1 -- start start 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.962+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab32af0 0x55b81ab35560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.962+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab33410 0x55b81ab3b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.962+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 0x55b81ab556b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.962+0000 7ff8987bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901ad980 con 0x55b81ab3dcb0 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.962+0000 7ff8987bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901af4c0 con 0x55b81ab33410 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.962+0000 7ff8987bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901ae050 con 0x55b81ab32af0 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab32af0 0x55b81ab35560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88effd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab33410 0x55b81ab3b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab32af0 0x55b81ab35560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33806/0 (socket says 172.21.15.5:33806) 2024-09-11T22:31:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88e7fc640 1 -- 172.21.15.5:0/1044271792 learned_addr learned my addr 172.21.15.5:0/1044271792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88effd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab33410 0x55b81ab3b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47448/0 (socket says 172.21.15.5:47448) 2024-09-11T22:31:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88e7fc640 1 --2- 172.21.15.5:0/1044271792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 0x55b81ab556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88effd640 1 -- 172.21.15.5:0/1044271792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab32af0 msgr2=0x55b81ab35560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88effd640 1 --2- 172.21.15.5:0/1044271792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab32af0 0x55b81ab35560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88effd640 1 -- 172.21.15.5:0/1044271792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 msgr2=0x55b81ab556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88effd640 1 --2- 172.21.15.5:0/1044271792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 0x55b81ab556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88effd640 1 -- 172.21.15.5:0/1044271792 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b81ab32870 con 0x55b81ab33410 2024-09-11T22:31:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff88effd640 1 --2- 172.21.15.5:0/1044271792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab33410 0x55b81ab3b420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff880004d60 tx=0x7ff880009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70bfc6fd3a44b807 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.963+0000 7ff8757fa640 1 -- 172.21.15.5:0/1044271792 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880028040 con 0x55b81ab33410 2024-09-11T22:31:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8757fa640 1 -- 172.21.15.5:0/1044271792 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff88000f6e0 con 0x55b81ab33410 2024-09-11T22:31:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8757fa640 1 -- 172.21.15.5:0/1044271792 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880004660 con 0x55b81ab33410 2024-09-11T22:31:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8987bf740 1 -- 172.21.15.5:0/1044271792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab33410 msgr2=0x55b81ab3b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1044271792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab33410 0x55b81ab3b420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff880004d60 tx=0x7ff880009530 comp rx=0 tx=0).stop 2024-09-11T22:31:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8987bf740 1 -- 172.21.15.5:0/1044271792 shutdown_connections 2024-09-11T22:31:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1044271792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 0x55b81ab556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1044271792 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab33410 0x55b81ab3b420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8987bf740 1 --2- 172.21.15.5:0/1044271792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab32af0 0x55b81ab35560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8987bf740 1 -- 172.21.15.5:0/1044271792 >> 172.21.15.5:0/1044271792 conn(0x55b81ab2f060 msgr2=0x55b81ab3cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8987bf740 1 -- 172.21.15.5:0/1044271792 shutdown_connections 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.964+0000 7ff8987bf740 1 -- 172.21.15.5:0/1044271792 wait complete. 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff8987bf740 1 -- start start 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff8987bf740 1 Processor -- start 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 0x55b81ab45cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 0x55b81ab46200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff8987bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 0x55b81ab37190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff8987bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901aedf0 con 0x55b81ab3dcb0 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff8987bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901ae720 con 0x55b81ab32af0 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff8987bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8901ab980 con 0x55b81ab33410 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff8987bf740 10 client.0 _ll_register_callbacks cb 0x7ff884009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 0x55b81ab45cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 0x55b81ab37190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff88e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 0x55b81ab37190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57034/0 (socket says 172.21.15.5:57034) 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff88e7fc640 1 -- 172.21.15.5:0/590279292 learned_addr learned my addr 172.21.15.5:0/590279292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.965+0000 7ff88effd640 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 0x55b81ab46200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff88e7fc640 1 -- 172.21.15.5:0/590279292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 msgr2=0x55b81ab46200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff88e7fc640 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 0x55b81ab46200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff88e7fc640 1 -- 172.21.15.5:0/590279292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 msgr2=0x55b81ab45cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff88e7fc640 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 0x55b81ab45cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff88e7fc640 1 -- 172.21.15.5:0/590279292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b81ab377d0 con 0x55b81ab3dcb0 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff88effd640 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 0x55b81ab46200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff88e7fc640 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 0x55b81ab37190 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff884009fd0 tx=0x7ff884004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebb332f2d508898 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff86effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff86effd640 1 -- 172.21.15.5:0/590279292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884005020 con 0x55b81ab3dcb0 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff86effd640 1 -- 172.21.15.5:0/590279292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7ff88401c040 con 0x55b81ab3dcb0 2024-09-11T22:31:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff8987bf740 1 -- 172.21.15.5:0/590279292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff88000f6e0 con 0x55b81ab3dcb0 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff86effd640 1 -- 172.21.15.5:0/590279292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff88403d050 con 0x55b81ab3dcb0 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff8987bf740 10 client.18309 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.966+0000 7ff8987bf740 1 -- 172.21.15.5:0/590279292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b81abb7f90 con 0x55b81ab3dcb0 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.967+0000 7ff86effd640 1 -- 172.21.15.5:0/590279292 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff88000f6e0 con 0x55b81ab3dcb0 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.967+0000 7ff86effd640 1 -- 172.21.15.5:0/590279292 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff8840047a0 con 0x55b81ab3dcb0 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.967+0000 7ff8987bf740 10 client.18309 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.967+0000 7ff8987bf740 10 client.18309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.967+0000 7ff8987bf740 20 client.18309 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.967+0000 7ff8987bf740 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b81abb86b0 0x55b81abbaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.967+0000 7ff8987bf740 4 client.18309 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.967+0000 7ff8987bf740 1 -- 172.21.15.5:0/590279292 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b81aa9ed70 con 0x55b81abb86b0 2024-09-11T22:31:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.967+0000 7ff88effd640 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b81abb86b0 0x55b81abbaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.967+0000 7ff86effd640 10 client.18309 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:18.968+0000 7ff88effd640 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b81abb86b0 0x55b81abbaad0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05e86967a34ff2b server_cookie=a81f9aa6c548a145 in_seq=0 out_seq=0 2024-09-11T22:31:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff86effd640 1 -- 172.21.15.5:0/590279292 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b81aa9ed70 con 0x55b81abb86b0 2024-09-11T22:31:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff86effd640 10 client.18309 handle_command_reply: tid=0 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 client.18309 shutdown 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 -- 172.21.15.5:0/590279292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 msgr2=0x55b81ab37190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 0x55b81ab37190 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ff884009fd0 tx=0x7ff884004730 comp rx=0 tx=0).stop 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 -- 172.21.15.5:0/590279292 shutdown_connections 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b81abb86b0 0x55b81abbaad0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b81ab3dcb0 0x55b81ab37190 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff86effd640 0 client.18309 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b81ab33410 0x55b81ab46200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 --2- 172.21.15.5:0/590279292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b81ab32af0 0x55b81ab45cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 -- 172.21.15.5:0/590279292 >> 172.21.15.5:0/590279292 conn(0x55b81ab2f060 msgr2=0x55b81ab2d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 -- 172.21.15.5:0/590279292 shutdown_connections 2024-09-11T22:31:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.089+0000 7ff8987bf740 1 -- 172.21.15.5:0/590279292 wait complete. 2024-09-11T22:31:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff8987bf740 20 client.18309 trim_cache size 0 max 16384 2024-09-11T22:31:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:31:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:31:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:31:19.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:31:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 -- 172.21.15.5:0/1001062491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8780a2da0 msgr2=0x7ff8780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8780a2da0 0x7ff8780a5260 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ed590 tx=0x7ff88808e270 comp rx=0 tx=0).stop 2024-09-11T22:31:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 -- 172.21.15.5:0/1001062491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89010c0d0 msgr2=0x7ff89019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89010c0d0 0x7ff89019cc10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff888034040 tx=0x7ff8880305d0 comp rx=0 tx=0).stop 2024-09-11T22:31:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 -- 172.21.15.5:0/1001062491 shutdown_connections 2024-09-11T22:31:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8780a2da0 0x7ff8780a5260 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff89010c0d0 0x7ff89019cc10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8901035a0 0x7ff890198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 --2- 172.21.15.5:0/1001062491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff890102c80 0x7ff890198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 -- 172.21.15.5:0/1001062491 >> 172.21.15.5:0/1001062491 conn(0x7ff890077990 msgr2=0x7ff8900fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 -- 172.21.15.5:0/1001062491 shutdown_connections 2024-09-11T22:31:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:19.090+0000 7ff895952640 1 -- 172.21.15.5:0/1001062491 wait complete. 2024-09-11T22:31:19.334 INFO:tasks.workunit.client.0.smithi005.stdout: 1 679132 7.48 MB/sec execute 337 sec latency 64.725 ms 2024-09-11T22:31:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:19 smithi005 ceph-mon[30943]: pgmap v286: 97 pgs: 97 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-11T22:31:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:19 smithi159 ceph-mon[35534]: pgmap v286: 97 pgs: 97 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-11T22:31:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:19 smithi172 ceph-mon[35461]: pgmap v286: 97 pgs: 97 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-11T22:31:19.985 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:31:19.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:31:20.251 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:20.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 681799 7.50 MB/sec execute 338 sec latency 16.568 ms 2024-09-11T22:31:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.945+0000 7f471f9b2640 1 -- 172.21.15.5:0/1852957052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718106f10 msgr2=0x7f471810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.945+0000 7f471f9b2640 1 --2- 172.21.15.5:0/1852957052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718106f10 0x7f471810ea90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f4710009ca0 tx=0x7f471002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.945+0000 7f471f9b2640 1 -- 172.21.15.5:0/1852957052 shutdown_connections 2024-09-11T22:31:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.945+0000 7f471f9b2640 1 --2- 172.21.15.5:0/1852957052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718106f10 0x7f471810ea90 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.945+0000 7f471f9b2640 1 --2- 172.21.15.5:0/1852957052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4718102790 0x7f47181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.945+0000 7f471f9b2640 1 --2- 172.21.15.5:0/1852957052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4718101de0 0x7f47181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.945+0000 7f471f9b2640 1 -- 172.21.15.5:0/1852957052 >> 172.21.15.5:0/1852957052 conn(0x7f47180fd8c0 msgr2=0x7f47180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.945+0000 7f471f9b2640 1 -- 172.21.15.5:0/1852957052 shutdown_connections 2024-09-11T22:31:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.945+0000 7f471f9b2640 1 -- 172.21.15.5:0/1852957052 wait complete. 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.946+0000 7f471f9b2640 1 Processor -- start 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.946+0000 7f471f9b2640 1 -- start start 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.946+0000 7f471f9b2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4718101de0 0x7f4718198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.946+0000 7f471f9b2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4718102790 0x7f4718198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.946+0000 7f471f9b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718106f10 0x7f471819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.946+0000 7f471f9b2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4718111f70 con 0x7f4718106f10 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.946+0000 7f471f9b2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4718111df0 con 0x7f4718102790 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.946+0000 7f471f9b2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f47181120f0 con 0x7f4718101de0 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471df4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718106f10 0x7f471819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471d74e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4718101de0 0x7f4718198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471df4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718106f10 0x7f471819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57046/0 (socket says 172.21.15.5:57046) 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471df4f640 1 -- 172.21.15.5:0/1672478080 learned_addr learned my addr 172.21.15.5:0/1672478080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471cf4d640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4718102790 0x7f4718198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471df4f640 1 -- 172.21.15.5:0/1672478080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4718101de0 msgr2=0x7f4718198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471df4f640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4718101de0 0x7f4718198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471df4f640 1 -- 172.21.15.5:0/1672478080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4718102790 msgr2=0x7f4718198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471df4f640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4718102790 0x7f4718198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471df4f640 1 -- 172.21.15.5:0/1672478080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471819d3c0 con 0x7f4718106f10 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471df4f640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718106f10 0x7f471819cce0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f4710031a20 tx=0x7f4710031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f470e7fc640 1 -- 172.21.15.5:0/1672478080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710003c30 con 0x7f4718106f10 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471f9b2640 1 -- 172.21.15.5:0/1672478080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471819d650 con 0x7f4718106f10 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f470e7fc640 1 -- 172.21.15.5:0/1672478080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4710003dd0 con 0x7f4718106f10 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f470e7fc640 1 -- 172.21.15.5:0/1672478080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471002fc40 con 0x7f4718106f10 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.947+0000 7f471f9b2640 1 -- 172.21.15.5:0/1672478080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47181a4f90 con 0x7f4718106f10 2024-09-11T22:31:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.950+0000 7f470e7fc640 1 -- 172.21.15.5:0/1672478080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4710004050 con 0x7f4718106f10 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f4721d8b740 1 Processor -- start 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f4721d8b740 1 -- start start 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d1180be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180c380 0x559d1181ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1181b150 0x559d1181d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f4721d8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559d11771280 con 0x559d1181b150 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f4721d8b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559d11718b10 con 0x559d1180ba60 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f4721d8b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559d1176d240 con 0x559d1180c380 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f471cf4d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d1180be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f471cf4d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d1180be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47488/0 (socket says 172.21.15.5:47488) 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f471cf4d640 1 -- 172.21.15.5:0/2959679943 learned_addr learned my addr 172.21.15.5:0/2959679943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f471cf4d640 1 --2- 172.21.15.5:0/2959679943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1181b150 0x559d1181d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f471d74e640 1 --2- 172.21.15.5:0/2959679943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180c380 0x559d1181ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f470e7fc640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f47040a2cf0 0x7f47040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f471cf4d640 1 -- 172.21.15.5:0/2959679943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180c380 msgr2=0x559d1181ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f471cf4d640 1 --2- 172.21.15.5:0/2959679943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180c380 0x559d1181ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f471cf4d640 1 -- 172.21.15.5:0/2959679943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1181b150 msgr2=0x559d1181d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f471cf4d640 1 --2- 172.21.15.5:0/2959679943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1181b150 0x559d1181d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.951+0000 7f471cf4d640 1 -- 172.21.15.5:0/2959679943 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d117251f0 con 0x559d1180ba60 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.952+0000 7f471df4f640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f47040a2cf0 0x7f47040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.952+0000 7f471cf4d640 1 --2- 172.21.15.5:0/2959679943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d1180be40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4708012720 tx=0x7f470800b6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18558f67d11e06ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.952+0000 7f470e7fc640 1 -- 172.21.15.5:0/1672478080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f47100f0590 con 0x7f4718106f10 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.952+0000 7f47037fe640 1 -- 172.21.15.5:0/2959679943 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708012ca0 con 0x559d1180ba60 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.952+0000 7f47037fe640 1 -- 172.21.15.5:0/2959679943 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4708012e40 con 0x559d1180ba60 2024-09-11T22:31:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.952+0000 7f47037fe640 1 -- 172.21.15.5:0/2959679943 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708004a00 con 0x559d1180ba60 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 -- 172.21.15.5:0/2959679943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 msgr2=0x559d1180be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 --2- 172.21.15.5:0/2959679943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d1180be40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4708012720 tx=0x7f470800b6e0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f471df4f640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f47040a2cf0 0x7f47040a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f47100b9100 tx=0x7f471002a8c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 -- 172.21.15.5:0/2959679943 shutdown_connections 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 --2- 172.21.15.5:0/2959679943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1181b150 0x559d1181d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 --2- 172.21.15.5:0/2959679943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180c380 0x559d1181ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 --2- 172.21.15.5:0/2959679943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d1180be40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 -- 172.21.15.5:0/2959679943 >> 172.21.15.5:0/2959679943 conn(0x559d118080f0 msgr2=0x559d118084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 -- 172.21.15.5:0/2959679943 shutdown_connections 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 -- 172.21.15.5:0/2959679943 wait complete. 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 -- start start 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.953+0000 7f4721d8b740 1 Processor -- start 2024-09-11T22:31:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d11818050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11818590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 0x559d11818ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4721d8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559d1181dd80 con 0x559d1180c380 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4721d8b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559d1172bf10 con 0x559d1180ba60 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4721d8b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559d1181df00 con 0x559d1181b150 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4721d8b740 10 client.0 _ll_register_callbacks cb 0x559d116f4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471d74e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11818590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471d74e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11818590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57056/0 (socket says 172.21.15.5:57056) 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471d74e640 1 -- 172.21.15.5:0/853786123 learned_addr learned my addr 172.21.15.5:0/853786123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471cf4d640 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d11818050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471cf4d640 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 0x559d11818ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471d74e640 1 -- 172.21.15.5:0/853786123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 msgr2=0x559d11818ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471d74e640 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 0x559d11818ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471d74e640 1 -- 172.21.15.5:0/853786123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 msgr2=0x559d11818050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471d74e640 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d11818050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471d74e640 1 -- 172.21.15.5:0/853786123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d114e26a0 con 0x559d1180c380 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471cf4d640 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 0x559d11818ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f471d74e640 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11818590 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4714012770 tx=0x7f4714012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d016b0c943099caf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4700ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4700ff9640 1 -- 172.21.15.5:0/853786123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471400f8c0 con 0x559d1180c380 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4700ff9640 1 -- 172.21.15.5:0/853786123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f471400fa60 con 0x559d1180c380 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.954+0000 7f4721d8b740 1 -- 172.21.15.5:0/853786123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559d1177e7f0 con 0x559d1180c380 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4700ff9640 1 -- 172.21.15.5:0/853786123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f471400fc60 con 0x559d1180c380 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4721d8b740 10 client.18321 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4721d8b740 1 -- 172.21.15.5:0/853786123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559d11829a80 con 0x559d1180c380 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4700ff9640 1 -- 172.21.15.5:0/853786123 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559d1177e7f0 con 0x559d1180c380 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4700ff9640 1 -- 172.21.15.5:0/853786123 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f4714023420 con 0x559d1180c380 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4721d8b740 10 client.18321 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4721d8b740 10 client.18321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4721d8b740 20 client.18321 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4721d8b740 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559d1182af90 0x559d1182d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4721d8b740 4 client.18321 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.955+0000 7f4721d8b740 1 -- 172.21.15.5:0/853786123 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d11778d00 con 0x559d1182af90 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.967+0000 7f471cf4d640 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559d1182af90 0x559d1182d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.967+0000 7f471cf4d640 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559d1182af90 0x559d1182d3f0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12a7e2d74dc8492 server_cookie=75d1035e0325d444 in_seq=0 out_seq=0 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.967+0000 7f4700ff9640 10 client.18321 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.968+0000 7f4700ff9640 1 -- 172.21.15.5:0/853786123 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d11778d00 con 0x559d1182af90 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.968+0000 7f4700ff9640 10 client.18321 handle_command_reply: tid=0 2024-09-11T22:31:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.968+0000 7f4721d8b740 1 client.18321 shutdown 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 1 -- 172.21.15.5:0/853786123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 msgr2=0x559d11818590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11818590 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4714012770 tx=0x7f4714012c30 comp rx=0 tx=0).stop 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 1 -- 172.21.15.5:0/853786123 shutdown_connections 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559d1182af90 0x559d1182d3f0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 0x559d11818ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11818590 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 1 --2- 172.21.15.5:0/853786123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d11818050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 1 -- 172.21.15.5:0/853786123 >> 172.21.15.5:0/853786123 conn(0x559d118080f0 msgr2=0x559d1180de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4700ff9640 0 client.18321 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 1 -- 172.21.15.5:0/853786123 shutdown_connections 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 1 -- 172.21.15.5:0/853786123 wait complete. 2024-09-11T22:31:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.969+0000 7f4721d8b740 20 client.18321 trim_cache size 0 max 16384 2024-09-11T22:31:20.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.991+0000 7f4721d8b740 1 Processor -- start 2024-09-11T22:31:20.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.991+0000 7f4721d8b740 1 -- start start 2024-09-11T22:31:20.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.991+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d11832cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.991+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11833200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.991+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 0x559d118152b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.991+0000 7f4721d8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47181ab8b0 con 0x559d1180c380 2024-09-11T22:31:20.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.991+0000 7f4721d8b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47181ad3f0 con 0x559d1180ba60 2024-09-11T22:31:20.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.991+0000 7f4721d8b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f47181abf80 con 0x559d1181b150 2024-09-11T22:31:20.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471d74e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11833200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471d74e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11833200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57070/0 (socket says 172.21.15.5:57070) 2024-09-11T22:31:20.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471d74e640 1 -- 172.21.15.5:0/46023445 learned_addr learned my addr 172.21.15.5:0/46023445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:20.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471cf4d640 1 --2- 172.21.15.5:0/46023445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d11832cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471cf4d640 1 --2- 172.21.15.5:0/46023445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 0x559d118152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471cf4d640 1 -- 172.21.15.5:0/46023445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 msgr2=0x559d118152b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471cf4d640 1 --2- 172.21.15.5:0/46023445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 0x559d118152b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471cf4d640 1 -- 172.21.15.5:0/46023445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 msgr2=0x559d11833200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471cf4d640 1 --2- 172.21.15.5:0/46023445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11833200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471cf4d640 1 -- 172.21.15.5:0/46023445 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d11803830 con 0x559d1180ba60 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471d74e640 1 --2- 172.21.15.5:0/46023445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11833200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f471cf4d640 1 --2- 172.21.15.5:0/46023445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d11832cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x559d1182dc60 tx=0x7f470800bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9fc254309f77e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.992+0000 7f4702ffd640 1 -- 172.21.15.5:0/46023445 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708032040 con 0x559d1180ba60 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.993+0000 7f4702ffd640 1 -- 172.21.15.5:0/46023445 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4708005250 con 0x559d1180ba60 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.993+0000 7f4702ffd640 1 -- 172.21.15.5:0/46023445 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470802d5c0 con 0x559d1180ba60 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.993+0000 7f4721d8b740 1 -- 172.21.15.5:0/46023445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 msgr2=0x559d11832cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.993+0000 7f4721d8b740 1 --2- 172.21.15.5:0/46023445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d11832cc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x559d1182dc60 tx=0x7f470800bb40 comp rx=0 tx=0).stop 2024-09-11T22:31:20.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.993+0000 7f4721d8b740 1 -- 172.21.15.5:0/46023445 shutdown_connections 2024-09-11T22:31:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.993+0000 7f4721d8b740 1 --2- 172.21.15.5:0/46023445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1181b150 0x559d118152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.993+0000 7f4721d8b740 1 --2- 172.21.15.5:0/46023445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1180c380 0x559d11833200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.993+0000 7f4721d8b740 1 --2- 172.21.15.5:0/46023445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180ba60 0x559d11832cc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.993+0000 7f4721d8b740 1 -- 172.21.15.5:0/46023445 >> 172.21.15.5:0/46023445 conn(0x559d118080f0 msgr2=0x559d11819fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.994+0000 7f4721d8b740 1 -- 172.21.15.5:0/46023445 shutdown_connections 2024-09-11T22:31:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.994+0000 7f4721d8b740 1 -- 172.21.15.5:0/46023445 wait complete. 2024-09-11T22:31:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.994+0000 7f4721d8b740 1 -- start start 2024-09-11T22:31:20.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.994+0000 7f4721d8b740 1 Processor -- start 2024-09-11T22:31:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.994+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180ba60 0x559d1181ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.994+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180c380 0x559d1181f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.994+0000 7f4721d8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1181b150 0x559d1181f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.994+0000 7f4721d8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47181acd20 con 0x559d1181b150 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.994+0000 7f4721d8b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f47181ac650 con 0x559d1180c380 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.994+0000 7f4721d8b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f47181a98b0 con 0x559d1180ba60 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f4721d8b740 10 client.0 _ll_register_callbacks cb 0x7f4714009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471cf4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180ba60 0x559d1181ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471d74e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180c380 0x559d1181f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471cf4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1181b150 0x559d1181f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471d74e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180c380 0x559d1181f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47516/0 (socket says 172.21.15.5:47516) 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471d74e640 1 -- 172.21.15.5:0/1023361327 learned_addr learned my addr 172.21.15.5:0/1023361327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471cf4d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180ba60 0x559d1181ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33868/0 (socket says 172.21.15.5:33868) 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471cf4d640 1 -- 172.21.15.5:0/1023361327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180c380 msgr2=0x559d1181f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471cf4d640 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180c380 0x559d1181f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:20.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471cf4d640 1 -- 172.21.15.5:0/1023361327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1181b150 msgr2=0x559d1181f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471cf4d640 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1181b150 0x559d1181f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471cf4d640 1 -- 172.21.15.5:0/1023361327 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d11894c90 con 0x559d1180ba60 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f471cf4d640 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180ba60 0x559d1181ee60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4708015040 tx=0x7f4708010f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f4187cb519207c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f4700ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f4700ff9640 1 -- 172.21.15.5:0/1023361327 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708010c50 con 0x559d1180ba60 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f4700ff9640 1 -- 172.21.15.5:0/1023361327 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f47080035f0 con 0x559d1180ba60 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.995+0000 7f4721d8b740 1 -- 172.21.15.5:0/1023361327 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4708005250 con 0x559d1180ba60 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4700ff9640 1 -- 172.21.15.5:0/1023361327 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f47080037f0 con 0x559d1180ba60 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4721d8b740 10 client.27511 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4721d8b740 1 -- 172.21.15.5:0/1023361327 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559d11895f10 con 0x559d1180ba60 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4700ff9640 1 -- 172.21.15.5:0/1023361327 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4708005250 con 0x559d1180ba60 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4700ff9640 1 -- 172.21.15.5:0/1023361327 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f470801f6f0 con 0x559d1180ba60 2024-09-11T22:31:21.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4721d8b740 10 client.27511 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:21.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4721d8b740 10 client.27511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:21.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4721d8b740 20 client.27511 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:21.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4721d8b740 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559d118963e0 0x559d11898800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:21.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4721d8b740 4 client.27511 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:31:21.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.996+0000 7f4721d8b740 1 -- 172.21.15.5:0/1023361327 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559d116f5110 con 0x559d118963e0 2024-09-11T22:31:21.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.997+0000 7f471d74e640 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559d118963e0 0x559d11898800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:21.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.997+0000 7f471d74e640 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559d118963e0 0x559d11898800 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60c4463d68af3022 server_cookie=ff7205392fa5c748 in_seq=0 out_seq=0 2024-09-11T22:31:21.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.997+0000 7f4700ff9640 10 client.27511 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.998+0000 7f4700ff9640 1 -- 172.21.15.5:0/1023361327 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559d116f5110 con 0x559d118963e0 2024-09-11T22:31:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.998+0000 7f4700ff9640 10 client.27511 handle_command_reply: tid=0 2024-09-11T22:31:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.998+0000 7f4721d8b740 1 client.27511 shutdown 2024-09-11T22:31:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.998+0000 7f4721d8b740 1 -- 172.21.15.5:0/1023361327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180ba60 msgr2=0x559d1181ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.998+0000 7f4721d8b740 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180ba60 0x559d1181ee60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4708015040 tx=0x7f4708010f80 comp rx=0 tx=0).stop 2024-09-11T22:31:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.998+0000 7f4721d8b740 1 -- 172.21.15.5:0/1023361327 shutdown_connections 2024-09-11T22:31:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.998+0000 7f4721d8b740 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559d118963e0 0x559d11898800 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:21.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f4721d8b740 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d1181b150 0x559d1181f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f4721d8b740 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559d1180c380 0x559d1181f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f4700ff9640 0 client.27511 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f4721d8b740 1 --2- 172.21.15.5:0/1023361327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559d1180ba60 0x559d1181ee60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f4721d8b740 1 -- 172.21.15.5:0/1023361327 >> 172.21.15.5:0/1023361327 conn(0x559d118080f0 msgr2=0x559d11806a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f4721d8b740 1 -- 172.21.15.5:0/1023361327 shutdown_connections 2024-09-11T22:31:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f4721d8b740 1 -- 172.21.15.5:0/1023361327 wait complete. 2024-09-11T22:31:21.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f4721d8b740 20 client.27511 trim_cache size 0 max 16384 2024-09-11T22:31:21.003 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:31:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:31:21.003 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "b38fa1f5-6bcb-4bc5-a7b0-1143c58c5357", 2024-09-11T22:31:21.004 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:31:21.004 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:31:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f471f9b2640 1 -- 172.21.15.5:0/1672478080 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f47040a2cf0 msgr2=0x7f47040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f471f9b2640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f47040a2cf0 0x7f47040a51b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f47100b9100 tx=0x7f471002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:31:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f471f9b2640 1 -- 172.21.15.5:0/1672478080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718106f10 msgr2=0x7f471819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:20.999+0000 7f471f9b2640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718106f10 0x7f471819cce0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f4710031a20 tx=0x7f4710031a50 comp rx=0 tx=0).stop 2024-09-11T22:31:21.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:21.000+0000 7f471f9b2640 1 -- 172.21.15.5:0/1672478080 shutdown_connections 2024-09-11T22:31:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:21.000+0000 7f471f9b2640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4718106f10 0x7f471819cce0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:21.000+0000 7f471f9b2640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f47040a2cf0 0x7f47040a51b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:21.000+0000 7f471f9b2640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4718102790 0x7f4718198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:21.000+0000 7f471f9b2640 1 --2- 172.21.15.5:0/1672478080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4718101de0 0x7f4718198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:21.000+0000 7f471f9b2640 1 -- 172.21.15.5:0/1672478080 >> 172.21.15.5:0/1672478080 conn(0x7f47180fd8c0 msgr2=0x7f47180feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:21.000+0000 7f471f9b2640 1 -- 172.21.15.5:0/1672478080 shutdown_connections 2024-09-11T22:31:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:21.000+0000 7f471f9b2640 1 -- 172.21.15.5:0/1672478080 wait complete. 2024-09-11T22:31:21.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 683366 7.50 MB/sec execute 339 sec latency 105.389 ms 2024-09-11T22:31:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:21 smithi005 ceph-mon[30943]: pgmap v287: 97 pgs: 97 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-11T22:31:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:21 smithi159 ceph-mon[35534]: pgmap v287: 97 pgs: 97 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-11T22:31:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:21 smithi172 ceph-mon[35461]: pgmap v287: 97 pgs: 97 active+clean; 102 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-11T22:31:22.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:31:22.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 685721 7.52 MB/sec execute 340 sec latency 14.255 ms 2024-09-11T22:31:22.376 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:22.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:22 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:31:22.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:22 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:31:22.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:22 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:31:22.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:22 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:31:22.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:22 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:31:22.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:22 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:31:22.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:22 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:31:22.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:22 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:31:22.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:22 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:31:22.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:22 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:31:22.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:22 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:31:22.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:22 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:31:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:22 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:31:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:22 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:31:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:22 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:31:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:22 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:31:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:22 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:31:22.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:22 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:31:22.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.946+0000 7fcf90363640 1 -- 172.21.15.5:0/2144028249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf88074590 msgr2=0x7fcf88074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:22.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.946+0000 7fcf90363640 1 --2- 172.21.15.5:0/2144028249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf88074590 0x7fcf88074970 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009ca0 tx=0x7fcf8402a950 comp rx=0 tx=0).stop 2024-09-11T22:31:22.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.947+0000 7fcf90363640 1 -- 172.21.15.5:0/2144028249 shutdown_connections 2024-09-11T22:31:22.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.947+0000 7fcf90363640 1 --2- 172.21.15.5:0/2144028249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf88112b90 0x7fcf88114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.947+0000 7fcf90363640 1 --2- 172.21.15.5:0/2144028249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf88074eb0 0x7fcf88112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.947+0000 7fcf90363640 1 --2- 172.21.15.5:0/2144028249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf88074590 0x7fcf88074970 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.947+0000 7fcf90363640 1 -- 172.21.15.5:0/2144028249 >> 172.21.15.5:0/2144028249 conn(0x7fcf880fd8c0 msgr2=0x7fcf880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.947+0000 7fcf90363640 1 -- 172.21.15.5:0/2144028249 shutdown_connections 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.947+0000 7fcf90363640 1 -- 172.21.15.5:0/2144028249 wait complete. 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.947+0000 7fcf90363640 1 Processor -- start 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.947+0000 7fcf90363640 1 -- start start 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf90363640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf88074590 0x7fcf881a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf90363640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf88074eb0 0x7fcf881a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf90363640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf88112b90 0x7fcf8819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf90363640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf88118570 con 0x7fcf88112b90 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf90363640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf881183f0 con 0x7fcf88074eb0 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf90363640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf881186f0 con 0x7fcf88074590 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf8e900640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf88112b90 0x7fcf8819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf8e900640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf88112b90 0x7fcf8819e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57972/0 (socket says 172.21.15.5:57972) 2024-09-11T22:31:22.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf8e900640 1 -- 172.21.15.5:0/3369915927 learned_addr learned my addr 172.21.15.5:0/3369915927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf88074eb0 0x7fcf881a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.948+0000 7fcf8e0ff640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf88074590 0x7fcf881a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf8e900640 1 -- 172.21.15.5:0/3369915927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf88074590 msgr2=0x7fcf881a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf8e900640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf88074590 0x7fcf881a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf8e900640 1 -- 172.21.15.5:0/3369915927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf88074eb0 msgr2=0x7fcf881a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf8e900640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf88074eb0 0x7fcf881a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf8e900640 1 -- 172.21.15.5:0/3369915927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8819f0d0 con 0x7fcf88112b90 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf88074eb0 0x7fcf881a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf8e900640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf88112b90 0x7fcf8819e9a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000e970 tx=0x7fcf8000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf7b7fe640 1 -- 172.21.15.5:0/3369915927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8000ccc0 con 0x7fcf88112b90 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf7b7fe640 1 -- 172.21.15.5:0/3369915927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf80005500 con 0x7fcf88112b90 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf7b7fe640 1 -- 172.21.15.5:0/3369915927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf80010690 con 0x7fcf88112b90 2024-09-11T22:31:22.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf90363640 1 -- 172.21.15.5:0/3369915927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8819f3c0 con 0x7fcf88112b90 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.949+0000 7fcf90363640 1 -- 172.21.15.5:0/3369915927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf881ab6b0 con 0x7fcf88112b90 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf7b7fe640 1 -- 172.21.15.5:0/3369915927 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fcf800046c0 con 0x7fcf88112b90 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf9273c740 1 Processor -- start 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf9273c740 1 -- start start 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d0dc0 0x55b9141db140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf9273c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b9141362e0 con 0x55b9141db680 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf9273c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b9140ddce0 con 0x55b9141d0dc0 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf9273c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b9141322a0 con 0x55b9141d04a0 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf8d8fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.953+0000 7fcf8e0ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d0dc0 0x55b9141db140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8e0ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d0dc0 0x55b9141db140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:32876/0 (socket says 172.21.15.5:32876) 2024-09-11T22:31:22.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/2366528687 learned_addr learned my addr 172.21.15.5:0/2366528687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:22.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/2366528687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/2366528687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 msgr2=0x55b9141d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:22.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8e0ff640 1 --2- 172.21.15.5:0/2366528687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/2366528687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 msgr2=0x55b9141dda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:22.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8e0ff640 1 --2- 172.21.15.5:0/2366528687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141dda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/2366528687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9140c2440 con 0x55b9141d0dc0 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/2366528687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8e0ff640 1 --2- 172.21.15.5:0/2366528687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d0dc0 0x55b9141db140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009ca0 tx=0x7fcf84009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f166987247ae725a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf7b7fe640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf740a2d70 0x7fcf740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf740a2d70 0x7fcf740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf78dea640 1 -- 172.21.15.5:0/2366528687 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84048070 con 0x55b9141d0dc0 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.954+0000 7fcf78dea640 1 -- 172.21.15.5:0/2366528687 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf8402f940 con 0x55b9141d0dc0 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf78dea640 1 -- 172.21.15.5:0/2366528687 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84034040 con 0x55b9141d0dc0 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf7b7fe640 1 -- 172.21.15.5:0/3369915927 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fcf800cf790 con 0x7fcf88112b90 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf740a2d70 0x7fcf740a5230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55b9141cddb0 tx=0x7fcf7c004730 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf9273c740 1 -- 172.21.15.5:0/2366528687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d0dc0 msgr2=0x55b9141db140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf9273c740 1 --2- 172.21.15.5:0/2366528687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d0dc0 0x55b9141db140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009ca0 tx=0x7fcf84009820 comp rx=0 tx=0).stop 2024-09-11T22:31:22.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf9273c740 1 -- 172.21.15.5:0/2366528687 shutdown_connections 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf9273c740 1 --2- 172.21.15.5:0/2366528687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf9273c740 1 --2- 172.21.15.5:0/2366528687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d0dc0 0x55b9141db140 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf9273c740 1 --2- 172.21.15.5:0/2366528687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf9273c740 1 -- 172.21.15.5:0/2366528687 >> 172.21.15.5:0/2366528687 conn(0x55b9141cca30 msgr2=0x55b9141cce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.955+0000 7fcf9273c740 1 -- 172.21.15.5:0/2366528687 shutdown_connections 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.956+0000 7fcf9273c740 1 -- 172.21.15.5:0/2366528687 wait complete. 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.956+0000 7fcf9273c740 1 -- start start 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.956+0000 7fcf9273c740 1 Processor -- start 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141daff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf9273c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b9141de250 con 0x55b9141db680 2024-09-11T22:31:22.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf9273c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b9140f1050 con 0x55b9141d04a0 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf9273c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b9141de3d0 con 0x55b9141d0dc0 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf9273c740 10 client.0 _ll_register_callbacks cb 0x55b914131160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8e0ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141daff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8e900640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8e0ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141daff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:32886/0 (socket says 172.21.15.5:32886) 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/1578801387 learned_addr learned my addr 172.21.15.5:0/1578801387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/1578801387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 msgr2=0x55b9141d82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8e0ff640 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141d82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/1578801387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 msgr2=0x55b9141d8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8e0ff640 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141d8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/1578801387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b913ea73c0 con 0x55b9141d04a0 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141d82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:22.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.957+0000 7fcf8e900640 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.958+0000 7fcf8e0ff640 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141daff0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf84000f80 tx=0x7fcf840095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f66061df89f577f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.958+0000 7fcf71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.958+0000 7fcf71ffb640 1 -- 172.21.15.5:0/1578801387 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84051060 con 0x55b9141d04a0 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.958+0000 7fcf71ffb640 1 -- 172.21.15.5:0/1578801387 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fcf84040050 con 0x55b9141d04a0 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.958+0000 7fcf9273c740 1 -- 172.21.15.5:0/1578801387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf8402f940 con 0x55b9141d04a0 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.958+0000 7fcf71ffb640 1 -- 172.21.15.5:0/1578801387 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcf84048030 con 0x55b9141d04a0 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.958+0000 7fcf9273c740 10 client.27614 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.958+0000 7fcf9273c740 1 -- 172.21.15.5:0/1578801387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b9141e9c20 con 0x55b9141d04a0 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.958+0000 7fcf71ffb640 1 -- 172.21.15.5:0/1578801387 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf8402f940 con 0x55b9141d04a0 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.959+0000 7fcf71ffb640 1 -- 172.21.15.5:0/1578801387 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fcf8403a430 con 0x55b9141d04a0 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.959+0000 7fcf9273c740 10 client.27614 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.959+0000 7fcf9273c740 10 client.27614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:22.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.959+0000 7fcf9273c740 20 client.27614 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.959+0000 7fcf9273c740 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b9141eaf70 0x55b9141ed360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.959+0000 7fcf9273c740 4 client.27614 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.959+0000 7fcf9273c740 1 -- 172.21.15.5:0/1578801387 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b914143850 con 0x55b9141eaf70 2024-09-11T22:31:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.959+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b9141eaf70 0x55b9141ed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.959+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b9141eaf70 0x55b9141ed360 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=169f507f903edc36 server_cookie=439d84d5268333e in_seq=0 out_seq=0 2024-09-11T22:31:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.959+0000 7fcf71ffb640 10 client.27614 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.960+0000 7fcf71ffb640 1 -- 172.21.15.5:0/1578801387 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b914143850 con 0x55b9141eaf70 2024-09-11T22:31:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.960+0000 7fcf71ffb640 10 client.27614 handle_command_reply: tid=0 2024-09-11T22:31:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.960+0000 7fcf9273c740 1 client.27614 shutdown 2024-09-11T22:31:22.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.960+0000 7fcf9273c740 1 -- 172.21.15.5:0/1578801387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 msgr2=0x55b9141daff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.960+0000 7fcf9273c740 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141daff0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf84000f80 tx=0x7fcf840095c0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.961+0000 7fcf9273c740 1 -- 172.21.15.5:0/1578801387 shutdown_connections 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.961+0000 7fcf9273c740 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b9141eaf70 0x55b9141ed360 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.961+0000 7fcf9273c740 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.961+0000 7fcf9273c740 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141d82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.961+0000 7fcf71ffb640 0 client.27614 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.961+0000 7fcf9273c740 1 --2- 172.21.15.5:0/1578801387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141daff0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.961+0000 7fcf9273c740 1 -- 172.21.15.5:0/1578801387 >> 172.21.15.5:0/1578801387 conn(0x55b9141cca30 msgr2=0x55b9141d2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.961+0000 7fcf9273c740 1 -- 172.21.15.5:0/1578801387 shutdown_connections 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.961+0000 7fcf9273c740 1 -- 172.21.15.5:0/1578801387 wait complete. 2024-09-11T22:31:22.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.961+0000 7fcf9273c740 20 client.27614 trim_cache size 0 max 16384 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.978+0000 7fcf9273c740 1 Processor -- start 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.978+0000 7fcf9273c740 1 -- start start 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141f0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141d5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf9273c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf881b41c0 con 0x55b9141db680 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf9273c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf881b5d00 con 0x55b9141d04a0 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf9273c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf881b4890 con 0x55b9141d0dc0 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8d8fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8e0ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8e900640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141d5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8d8fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141f08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:50554/0 (socket says 172.21.15.5:50554) 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8e0ff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141f0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:32902/0 (socket says 172.21.15.5:32902) 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8d8fe640 1 -- 172.21.15.5:0/418522456 learned_addr learned my addr 172.21.15.5:0/418522456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8d8fe640 1 -- 172.21.15.5:0/418522456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 msgr2=0x55b9141f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/418522456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8d8fe640 1 -- 172.21.15.5:0/418522456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 msgr2=0x55b9141d5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/418522456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141d5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8d8fe640 1 -- 172.21.15.5:0/418522456 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9141db360 con 0x55b9141d0dc0 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.979+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/418522456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141f08c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcf88100150 tx=0x7fcf7c012070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef7f3ba9566ef8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.980+0000 7fcf73fff640 1 -- 172.21.15.5:0/418522456 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c019040 con 0x55b9141d0dc0 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.980+0000 7fcf73fff640 1 -- 172.21.15.5:0/418522456 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf7c004490 con 0x55b9141d0dc0 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.980+0000 7fcf73fff640 1 -- 172.21.15.5:0/418522456 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c02c6d0 con 0x55b9141d0dc0 2024-09-11T22:31:22.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.980+0000 7fcf9273c740 1 -- 172.21.15.5:0/418522456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 msgr2=0x55b9141f08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.980+0000 7fcf9273c740 1 --2- 172.21.15.5:0/418522456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141f08c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcf88100150 tx=0x7fcf7c012070 comp rx=0 tx=0).stop 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.980+0000 7fcf9273c740 1 -- 172.21.15.5:0/418522456 shutdown_connections 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.980+0000 7fcf9273c740 1 --2- 172.21.15.5:0/418522456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141db680 0x55b9141d5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.980+0000 7fcf9273c740 1 --2- 172.21.15.5:0/418522456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d0dc0 0x55b9141f08c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.980+0000 7fcf9273c740 1 --2- 172.21.15.5:0/418522456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141d04a0 0x55b9141f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.980+0000 7fcf9273c740 1 -- 172.21.15.5:0/418522456 >> 172.21.15.5:0/418522456 conn(0x55b9141cca30 msgr2=0x55b9141d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 1 -- 172.21.15.5:0/418522456 shutdown_connections 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 1 -- 172.21.15.5:0/418522456 wait complete. 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 1 -- start start 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 1 Processor -- start 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141d0dc0 0x55b9141ee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141db680 0x55b9141eedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf881b5630 con 0x55b9141d0dc0 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf881b4f60 con 0x55b9141db680 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf881b21c0 con 0x55b9141d04a0 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf9273c740 10 client.0 _ll_register_callbacks cb 0x7fcf84046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf8e0ff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf8d8fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141d0dc0 0x55b9141ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf8e0ff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141ee350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:50558/0 (socket says 172.21.15.5:50558) 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/3507403676 learned_addr learned my addr 172.21.15.5:0/3507403676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.981+0000 7fcf8e900640 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141db680 0x55b9141eedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/3507403676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141db680 msgr2=0x55b9141eedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf8e0ff640 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141db680 0x55b9141eedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/3507403676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141d0dc0 msgr2=0x55b9141ee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf8e0ff640 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141d0dc0 0x55b9141ee890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf8e0ff640 1 -- 172.21.15.5:0/3507403676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b914254a50 con 0x55b9141d04a0 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf8e0ff640 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141ee350 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf8402b050 tx=0x7fcf8402b080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0847bb3319c132d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf71ffb640 1 -- 172.21.15.5:0/3507403676 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84051070 con 0x55b9141d04a0 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf9273c740 1 -- 172.21.15.5:0/3507403676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf7c004490 con 0x55b9141d04a0 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf71ffb640 1 -- 172.21.15.5:0/3507403676 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7fcf84040040 con 0x55b9141d04a0 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf71ffb640 1 -- 172.21.15.5:0/3507403676 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcf84034040 con 0x55b9141d04a0 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf9273c740 10 client.27529 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.982+0000 7fcf9273c740 1 -- 172.21.15.5:0/3507403676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b914255d40 con 0x55b9141d04a0 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf71ffb640 1 -- 172.21.15.5:0/3507403676 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf7c004490 con 0x55b9141d04a0 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf71ffb640 1 -- 172.21.15.5:0/3507403676 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fcf8404d470 con 0x55b9141d04a0 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf9273c740 10 client.27529 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:22.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf9273c740 10 client.27529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf9273c740 20 client.27529 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf9273c740 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b914256410 0x55b914258800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf9273c740 4 client.27529 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf9273c740 1 -- 172.21.15.5:0/3507403676 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b91413dd60 con 0x55b914256410 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b914256410 0x55b914258800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf71ffb640 10 client.27529 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.983+0000 7fcf8d8fe640 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b914256410 0x55b914258800 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b94ac8c6d45ea9c7 server_cookie=fe151cf03fed45c8 in_seq=0 out_seq=0 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf71ffb640 1 -- 172.21.15.5:0/3507403676 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b91413dd60 con 0x55b914256410 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf71ffb640 10 client.27529 handle_command_reply: tid=0 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 client.27529 shutdown 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 -- 172.21.15.5:0/3507403676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 msgr2=0x55b9141ee350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141ee350 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcf8402b050 tx=0x7fcf8402b080 comp rx=0 tx=0).stop 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 -- 172.21.15.5:0/3507403676 shutdown_connections 2024-09-11T22:31:22.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b914256410 0x55b914258800 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b9141db680 0x55b9141eedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b9141d0dc0 0x55b9141ee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 --2- 172.21.15.5:0/3507403676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b9141d04a0 0x55b9141ee350 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 -- 172.21.15.5:0/3507403676 >> 172.21.15.5:0/3507403676 conn(0x55b9141cca30 msgr2=0x55b9141d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf71ffb640 0 client.27529 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 -- 172.21.15.5:0/3507403676 shutdown_connections 2024-09-11T22:31:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 1 -- 172.21.15.5:0/3507403676 wait complete. 2024-09-11T22:31:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf9273c740 20 client.27529 trim_cache size 0 max 16384 2024-09-11T22:31:22.991 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:31:22.992 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:31:22.992 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:31:22.992 INFO:teuthology.orchestra.run.smithi005.stdout: "b38fa1f5-6bcb-4bc5-a7b0-1143c58c5357": { 2024-09-11T22:31:22.992 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:31:22.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "b38fa1f5-6bcb-4bc5-a7b0-1143c58c5357", 2024-09-11T22:31:22.992 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:31:22.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:31:22.992 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:31:22.992 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.984+0000 7fcf90363640 1 -- 172.21.15.5:0/3369915927 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf740a2d70 msgr2=0x7fcf740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf740a2d70 0x7fcf740a5230 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x55b9141cddb0 tx=0x7fcf7c004730 comp rx=0 tx=0).stop 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 -- 172.21.15.5:0/3369915927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf88112b90 msgr2=0x7fcf8819e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf88112b90 0x7fcf8819e9a0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000e970 tx=0x7fcf8000ee30 comp rx=0 tx=0).stop 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 -- 172.21.15.5:0/3369915927 shutdown_connections 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf88112b90 0x7fcf8819e9a0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf740a2d70 0x7fcf740a5230 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf88074eb0 0x7fcf881a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 --2- 172.21.15.5:0/3369915927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf88074590 0x7fcf881a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 -- 172.21.15.5:0/3369915927 >> 172.21.15.5:0/3369915927 conn(0x7fcf880fd8c0 msgr2=0x7fcf880ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 -- 172.21.15.5:0/3369915927 shutdown_connections 2024-09-11T22:31:22.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:22.985+0000 7fcf90363640 1 -- 172.21.15.5:0/3369915927 wait complete. 2024-09-11T22:31:23.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687777 7.52 MB/sec execute 341 sec latency 24.107 ms 2024-09-11T22:31:23.802 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:23 smithi005 ceph-mon[30943]: pgmap v288: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-09-11T22:31:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:23 smithi159 ceph-mon[35534]: pgmap v288: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-09-11T22:31:23.835 INFO:tasks.cephfs.filesystem:scrub status for tag:b38fa1f5-6bcb-4bc5-a7b0-1143c58c5357 - {'path': '/', 'tag': 'b38fa1f5-6bcb-4bc5-a7b0-1143c58c5357', 'options': 'recursive,force'} 2024-09-11T22:31:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:23 smithi172 ceph-mon[35461]: pgmap v288: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.7 MiB/s wr, 331 op/s 2024-09-11T22:31:24.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 689707 7.53 MB/sec execute 342 sec latency 14.604 ms 2024-09-11T22:31:25.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 692156 7.55 MB/sec execute 343 sec latency 15.853 ms 2024-09-11T22:31:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:25 smithi005 ceph-mon[30943]: pgmap v289: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.3 MiB/s wr, 284 op/s 2024-09-11T22:31:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:25 smithi159 ceph-mon[35534]: pgmap v289: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.3 MiB/s wr, 284 op/s 2024-09-11T22:31:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:25 smithi172 ceph-mon[35461]: pgmap v289: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.3 MiB/s wr, 284 op/s 2024-09-11T22:31:26.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 693435 7.55 MB/sec execute 344 sec latency 16.076 ms 2024-09-11T22:31:27.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 695679 7.56 MB/sec execute 345 sec latency 123.116 ms 2024-09-11T22:31:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:27 smithi005 ceph-mon[30943]: pgmap v290: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 276 op/s 2024-09-11T22:31:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:27 smithi159 ceph-mon[35534]: pgmap v290: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 276 op/s 2024-09-11T22:31:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:27 smithi172 ceph-mon[35461]: pgmap v290: 97 pgs: 97 active+clean; 90 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.2 MiB/s wr, 276 op/s 2024-09-11T22:31:28.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 696915 7.56 MB/sec execute 346 sec latency 16.507 ms 2024-09-11T22:31:29.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 699246 7.57 MB/sec execute 347 sec latency 69.689 ms 2024-09-11T22:31:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:29 smithi005 ceph-mon[30943]: pgmap v291: 97 pgs: 97 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.1 MiB/s wr, 355 op/s 2024-09-11T22:31:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:29 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:31:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:29 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:31:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:29 smithi159 ceph-mon[35534]: pgmap v291: 97 pgs: 97 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.1 MiB/s wr, 355 op/s 2024-09-11T22:31:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:29 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:31:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:29 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:31:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:29 smithi172 ceph-mon[35461]: pgmap v291: 97 pgs: 97 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.1 MiB/s wr, 355 op/s 2024-09-11T22:31:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:29 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:31:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:29 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:31:30.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 700841 7.57 MB/sec execute 348 sec latency 20.459 ms 2024-09-11T22:31:31.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 703190 7.60 MB/sec execute 349 sec latency 14.903 ms 2024-09-11T22:31:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:31 smithi005 ceph-mon[30943]: pgmap v292: 97 pgs: 97 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-09-11T22:31:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:31 smithi159 ceph-mon[35534]: pgmap v292: 97 pgs: 97 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-09-11T22:31:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:31 smithi172 ceph-mon[35461]: pgmap v292: 97 pgs: 97 active+clean; 99 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 253 op/s 2024-09-11T22:31:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.190+0000 7f455bbc4640 1 -- 172.21.15.5:0/1227097177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45541069c0 msgr2=0x7f4554106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.190+0000 7f455bbc4640 1 --2- 172.21.15.5:0/1227097177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45541069c0 0x7f4554106da0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f4550009cd0 tx=0x7f455002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.190+0000 7f455bbc4640 1 -- 172.21.15.5:0/1227097177 shutdown_connections 2024-09-11T22:31:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.190+0000 7f455bbc4640 1 --2- 172.21.15.5:0/1227097177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4554101350 0x7f455410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.190+0000 7f455bbc4640 1 --2- 172.21.15.5:0/1227097177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45541009b0 0x7f4554100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.190+0000 7f455bbc4640 1 --2- 172.21.15.5:0/1227097177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45541069c0 0x7f4554106da0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.190+0000 7f455bbc4640 1 -- 172.21.15.5:0/1227097177 >> 172.21.15.5:0/1227097177 conn(0x7f4554077990 msgr2=0x7f45540fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.190+0000 7f455bbc4640 1 -- 172.21.15.5:0/1227097177 shutdown_connections 2024-09-11T22:31:32.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.190+0000 7f455bbc4640 1 -- 172.21.15.5:0/1227097177 wait complete. 2024-09-11T22:31:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.191+0000 7f455bbc4640 1 Processor -- start 2024-09-11T22:31:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.191+0000 7f455bbc4640 1 -- start start 2024-09-11T22:31:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.191+0000 7f455bbc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45541009b0 0x7f45540fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.191+0000 7f455bbc4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554101350 0x7f45540fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f455bbc4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45541069c0 0x7f45540fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f455bbc4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4554110dc0 con 0x7f45541009b0 2024-09-11T22:31:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f455bbc4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4554110c40 con 0x7f45541069c0 2024-09-11T22:31:32.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f455bbc4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4554110f40 con 0x7f4554101350 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f4559960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45541009b0 0x7f45540fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f4559960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45541009b0 0x7f45540fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40886/0 (socket says 172.21.15.5:40886) 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f4559960640 1 -- 172.21.15.5:0/602347857 learned_addr learned my addr 172.21.15.5:0/602347857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f455a161640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45541069c0 0x7f45540fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f455915f640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554101350 0x7f45540fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f4559960640 1 -- 172.21.15.5:0/602347857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554101350 msgr2=0x7f45540fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f4559960640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554101350 0x7f45540fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f4559960640 1 -- 172.21.15.5:0/602347857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45541069c0 msgr2=0x7f45540fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f4559960640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45541069c0 0x7f45540fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f4559960640 1 -- 172.21.15.5:0/602347857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45541a6870 con 0x7f45541009b0 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f4559960640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45541009b0 0x7f45540fcde0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f455002ae00 tx=0x7f4550031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f454affd640 1 -- 172.21.15.5:0/602347857 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550003e00 con 0x7f45541009b0 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f454affd640 1 -- 172.21.15.5:0/602347857 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4550004520 con 0x7f45541009b0 2024-09-11T22:31:32.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f455bbc4640 1 -- 172.21.15.5:0/602347857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45541a6b00 con 0x7f45541009b0 2024-09-11T22:31:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.192+0000 7f454affd640 1 -- 172.21.15.5:0/602347857 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550030de0 con 0x7f45541009b0 2024-09-11T22:31:32.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.193+0000 7f455bbc4640 1 -- 172.21.15.5:0/602347857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541a7050 con 0x7f45541009b0 2024-09-11T22:31:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.195+0000 7f454affd640 1 -- 172.21.15.5:0/602347857 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f45500046d0 con 0x7f45541009b0 2024-09-11T22:31:32.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.195+0000 7f455ea38740 1 Processor -- start 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.195+0000 7f455ea38740 1 -- start start 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455ea38740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922aeaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455ea38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455ea38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922af5cb0 0x558922af80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455ea38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558922a4f2f0 con 0x558922aeb410 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455ea38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5589229f7270 con 0x558922af5cb0 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455ea38740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558922a4b2b0 con 0x558922aeaaf0 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f4559960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455a161640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922aeaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f4559960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40898/0 (socket says 172.21.15.5:40898) 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455a161640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922aeaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36446/0 (socket says 172.21.15.5:36446) 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f4559960640 1 -- 172.21.15.5:0/1224922846 learned_addr learned my addr 172.21.15.5:0/1224922846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:32.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455915f640 1 --2- 172.21.15.5:0/1224922846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922af5cb0 0x558922af80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455a161640 1 -- 172.21.15.5:0/1224922846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922af5cb0 msgr2=0x558922af80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455a161640 1 --2- 172.21.15.5:0/1224922846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922af5cb0 0x558922af80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455a161640 1 -- 172.21.15.5:0/1224922846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 msgr2=0x558922af5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455a161640 1 --2- 172.21.15.5:0/1224922846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455a161640 1 -- 172.21.15.5:0/1224922846 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589229f6340 con 0x558922aeaaf0 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f454affd640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45400a2da0 0x7f45400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.196+0000 7f455a161640 1 --2- 172.21.15.5:0/1224922846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922aeaed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f454c009b80 tx=0x7f454c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3aed4c3a76cf4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.197+0000 7f4559960640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45400a2da0 0x7f45400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.197+0000 7f453ffff640 1 -- 172.21.15.5:0/1224922846 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c004050 con 0x558922aeaaf0 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.197+0000 7f454affd640 1 -- 172.21.15.5:0/602347857 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f455002e030 con 0x7f45541009b0 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.197+0000 7f453ffff640 1 -- 172.21.15.5:0/1224922846 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c005500 con 0x558922aeaaf0 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.197+0000 7f453ffff640 1 -- 172.21.15.5:0/1224922846 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c004cd0 con 0x558922aeaaf0 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.197+0000 7f455ea38740 1 -- 172.21.15.5:0/1224922846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 msgr2=0x558922aeaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.197+0000 7f4559960640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45400a2da0 0x7f45400a5260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4550031040 tx=0x7f45500b6040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.197+0000 7f455ea38740 1 --2- 172.21.15.5:0/1224922846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922aeaed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f454c009b80 tx=0x7f454c011810 comp rx=0 tx=0).stop 2024-09-11T22:31:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 -- 172.21.15.5:0/1224922846 shutdown_connections 2024-09-11T22:31:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 --2- 172.21.15.5:0/1224922846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922af5cb0 0x558922af80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 --2- 172.21.15.5:0/1224922846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 --2- 172.21.15.5:0/1224922846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922aeaed0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 -- 172.21.15.5:0/1224922846 >> 172.21.15.5:0/1224922846 conn(0x558922ae7060 msgr2=0x558922ae7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 -- 172.21.15.5:0/1224922846 shutdown_connections 2024-09-11T22:31:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 -- 172.21.15.5:0/1224922846 wait complete. 2024-09-11T22:31:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 -- start start 2024-09-11T22:31:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 Processor -- start 2024-09-11T22:31:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922af2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922af5cb0 0x558922af35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558922af8880 con 0x558922aeb410 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5589229d8290 con 0x558922af5cb0 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.198+0000 7f455ea38740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558922af8a00 con 0x558922aeaaf0 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455ea38740 10 client.0 _ll_register_callbacks cb 0x558922a09140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455915f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455a161640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922af2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455915f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40906/0 (socket says 172.21.15.5:40906) 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455915f640 1 -- 172.21.15.5:0/343240572 learned_addr learned my addr 172.21.15.5:0/343240572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455a161640 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922af5cb0 0x558922af35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455915f640 1 -- 172.21.15.5:0/343240572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 msgr2=0x558922af2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455915f640 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922af2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455915f640 1 -- 172.21.15.5:0/343240572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922af5cb0 msgr2=0x558922af35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455915f640 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922af5cb0 0x558922af35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455915f640 1 -- 172.21.15.5:0/343240572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558922762210 con 0x558922aeb410 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455915f640 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af3060 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4544012730 tx=0x7f4544012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd681b99c63b3c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f453d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f453d7fa640 1 -- 172.21.15.5:0/343240572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454400f920 con 0x558922aeb410 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f455ea38740 1 -- 172.21.15.5:0/343240572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c005500 con 0x558922aeb410 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f453d7fa640 1 -- 172.21.15.5:0/343240572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f454400fac0 con 0x558922aeb410 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.199+0000 7f453d7fa640 1 -- 172.21.15.5:0/343240572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f454400fcc0 con 0x558922aeb410 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f455ea38740 10 client.18357 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f455ea38740 1 -- 172.21.15.5:0/343240572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55892282e620 con 0x558922aeb410 2024-09-11T22:31:32.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f453d7fa640 1 -- 172.21.15.5:0/343240572 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c005500 con 0x558922aeb410 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f453d7fa640 1 -- 172.21.15.5:0/343240572 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f454401c420 con 0x558922aeb410 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f455ea38740 10 client.18357 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f455ea38740 10 client.18357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f455ea38740 20 client.18357 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f455ea38740 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558922b05cd0 0x558922b080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f455ea38740 4 client.18357 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f455ea38740 1 -- 172.21.15.5:0/343240572 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558922a5ca20 con 0x558922b05cd0 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.200+0000 7f455a161640 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558922b05cd0 0x558922b080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.201+0000 7f453d7fa640 10 client.18357 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.201+0000 7f455a161640 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558922b05cd0 0x558922b080c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4efa2fb4daf4a85 server_cookie=c331438421cd68dd in_seq=0 out_seq=0 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f453d7fa640 1 -- 172.21.15.5:0/343240572 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558922a5ca20 con 0x558922b05cd0 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f453d7fa640 10 client.18357 handle_command_reply: tid=0 2024-09-11T22:31:32.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 client.18357 shutdown 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 -- 172.21.15.5:0/343240572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 msgr2=0x558922af3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af3060 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4544012730 tx=0x7f4544012bf0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 -- 172.21.15.5:0/343240572 shutdown_connections 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558922b05cd0 0x558922b080c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922af5cb0 0x558922af35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f453d7fa640 0 client.18357 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeb410 0x558922af3060 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 --2- 172.21.15.5:0/343240572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922af2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 -- 172.21.15.5:0/343240572 >> 172.21.15.5:0/343240572 conn(0x558922ae7060 msgr2=0x558922aecdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 -- 172.21.15.5:0/343240572 shutdown_connections 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 1 -- 172.21.15.5:0/343240572 wait complete. 2024-09-11T22:31:32.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.203+0000 7f455ea38740 20 client.18357 trim_cache size 0 max 16384 2024-09-11T22:31:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.219+0000 7f455ea38740 1 Processor -- start 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.219+0000 7f455ea38740 1 -- start start 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455ea38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeaaf0 0x558922af33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455ea38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 0x558922b0d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455ea38740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922af5cb0 0x558922b0dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455ea38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45541afc30 con 0x558922aeaaf0 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455ea38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45541b1770 con 0x558922aeb410 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455ea38740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f45541b0300 con 0x558922af5cb0 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455a161640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeaaf0 0x558922af33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455a161640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeaaf0 0x558922af33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40910/0 (socket says 172.21.15.5:40910) 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455a161640 1 -- 172.21.15.5:0/1042081173 learned_addr learned my addr 172.21.15.5:0/1042081173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455915f640 1 --2- 172.21.15.5:0/1042081173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 0x558922b0d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455a161640 1 --2- 172.21.15.5:0/1042081173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922af5cb0 0x558922b0dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455915f640 1 -- 172.21.15.5:0/1042081173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922af5cb0 msgr2=0x558922b0dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455915f640 1 --2- 172.21.15.5:0/1042081173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922af5cb0 0x558922b0dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455915f640 1 -- 172.21.15.5:0/1042081173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeaaf0 msgr2=0x558922af33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455915f640 1 --2- 172.21.15.5:0/1042081173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeaaf0 0x558922af33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455915f640 1 -- 172.21.15.5:0/1042081173 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558922af1ab0 con 0x558922aeb410 2024-09-11T22:31:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.220+0000 7f455a161640 1 --2- 172.21.15.5:0/1042081173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeaaf0 0x558922af33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.221+0000 7f455915f640 1 --2- 172.21.15.5:0/1042081173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 0x558922b0d960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f4544009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0f9b90cf1f8eeb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.221+0000 7f453f7fe640 1 -- 172.21.15.5:0/1042081173 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454400f9e0 con 0x558922aeb410 2024-09-11T22:31:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.221+0000 7f453f7fe640 1 -- 172.21.15.5:0/1042081173 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454401c420 con 0x558922aeb410 2024-09-11T22:31:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.221+0000 7f453f7fe640 1 -- 172.21.15.5:0/1042081173 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45440035f0 con 0x558922aeb410 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.221+0000 7f455ea38740 1 -- 172.21.15.5:0/1042081173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 msgr2=0x558922b0d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.221+0000 7f455ea38740 1 --2- 172.21.15.5:0/1042081173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 0x558922b0d960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4544000c00 tx=0x7f4544009f20 comp rx=0 tx=0).stop 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 -- 172.21.15.5:0/1042081173 shutdown_connections 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 --2- 172.21.15.5:0/1042081173 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922af5cb0 0x558922b0dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 --2- 172.21.15.5:0/1042081173 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 0x558922b0d960 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 --2- 172.21.15.5:0/1042081173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922aeaaf0 0x558922af33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 -- 172.21.15.5:0/1042081173 >> 172.21.15.5:0/1042081173 conn(0x558922ae7060 msgr2=0x558922af4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 -- 172.21.15.5:0/1042081173 shutdown_connections 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 -- 172.21.15.5:0/1042081173 wait complete. 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 -- start start 2024-09-11T22:31:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 Processor -- start 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922af9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 0x558922af9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922af5cb0 0x558922afa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45541b10a0 con 0x558922af5cb0 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45541b09d0 con 0x558922aeb410 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f45541adc30 con 0x558922aeaaf0 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.222+0000 7f455ea38740 10 client.0 _ll_register_callbacks cb 0x7f4544009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455a161640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922af5cb0 0x558922afa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455a161640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922af9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455a161640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922af5cb0 0x558922afa3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40922/0 (socket says 172.21.15.5:40922) 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455a161640 1 -- 172.21.15.5:0/2943707057 learned_addr learned my addr 172.21.15.5:0/2943707057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455915f640 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 0x558922af9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455a161640 1 -- 172.21.15.5:0/2943707057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 msgr2=0x558922af9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455a161640 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922af9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455a161640 1 -- 172.21.15.5:0/2943707057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 msgr2=0x558922af9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455a161640 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 0x558922af9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455a161640 1 -- 172.21.15.5:0/2943707057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558922aeb240 con 0x558922af5cb0 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455a161640 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922af5cb0 0x558922afa3f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f454c009b50 tx=0x7f454c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e194e1d8649e89f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f453d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f453d7fa640 1 -- 172.21.15.5:0/2943707057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0036b0 con 0x558922af5cb0 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f453d7fa640 1 -- 172.21.15.5:0/2943707057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8629+0+0 (secure 0 0 0) 0x7f454c003850 con 0x558922af5cb0 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.223+0000 7f455ea38740 1 -- 172.21.15.5:0/2943707057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454401c420 con 0x558922af5cb0 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f453d7fa640 1 -- 172.21.15.5:0/2943707057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f454c042050 con 0x558922af5cb0 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f455ea38740 10 client.18363 fetch_fsmap learned FSMap version 40 2024-09-11T22:31:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f455ea38740 1 -- 172.21.15.5:0/2943707057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558922b709b0 con 0x558922af5cb0 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f453d7fa640 1 -- 172.21.15.5:0/2943707057 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454401c420 con 0x558922af5cb0 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f453d7fa640 1 -- 172.21.15.5:0/2943707057 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f454c028c10 con 0x558922af5cb0 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f455ea38740 10 client.18363 fetch_fsmap finished waiting for FSMap version 40 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f455ea38740 10 client.18363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f455ea38740 20 client.18363 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f455ea38740 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558922b05cd0 0x558922b72e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f455ea38740 4 client.18363 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f455ea38740 1 -- 172.21.15.5:0/2943707057 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558922a56d70 con 0x558922b05cd0 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.224+0000 7f455915f640 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558922b05cd0 0x558922b72e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.225+0000 7f455915f640 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558922b05cd0 0x558922b72e90 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0f03390b00ee29 server_cookie=88e55cc4f0c7b65 in_seq=0 out_seq=0 2024-09-11T22:31:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.225+0000 7f453d7fa640 10 client.18363 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:32.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 704363 7.59 MB/sec execute 350 sec latency 49.814 ms 2024-09-11T22:31:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f453d7fa640 1 -- 172.21.15.5:0/2943707057 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558922a56d70 con 0x558922b05cd0 2024-09-11T22:31:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f453d7fa640 10 client.18363 handle_command_reply: tid=0 2024-09-11T22:31:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 client.18363 shutdown 2024-09-11T22:31:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 -- 172.21.15.5:0/2943707057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922af5cb0 msgr2=0x558922afa3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922af5cb0 0x558922afa3f0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f454c009b50 tx=0x7f454c0096a0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 -- 172.21.15.5:0/2943707057 shutdown_connections 2024-09-11T22:31:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558922b05cd0 0x558922b72e90 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558922af5cb0 0x558922afa3f0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558922aeb410 0x558922af9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 --2- 172.21.15.5:0/2943707057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558922aeaaf0 0x558922af9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 -- 172.21.15.5:0/2943707057 >> 172.21.15.5:0/2943707057 conn(0x558922ae7060 msgr2=0x558922ae59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f453d7fa640 0 client.18363 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 -- 172.21.15.5:0/2943707057 shutdown_connections 2024-09-11T22:31:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 1 -- 172.21.15.5:0/2943707057 wait complete. 2024-09-11T22:31:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.368+0000 7f455ea38740 20 client.18363 trim_cache size 0 max 16384 2024-09-11T22:31:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:31:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:31:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:31:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:31:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 -- 172.21.15.5:0/602347857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45400a2da0 msgr2=0x7f45400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:32.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45400a2da0 0x7f45400a5260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4550031040 tx=0x7f45500b6040 comp rx=0 tx=0).stop 2024-09-11T22:31:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 -- 172.21.15.5:0/602347857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45541009b0 msgr2=0x7f45540fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45541009b0 0x7f45540fcde0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f455002ae00 tx=0x7f4550031e60 comp rx=0 tx=0).stop 2024-09-11T22:31:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 -- 172.21.15.5:0/602347857 shutdown_connections 2024-09-11T22:31:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45400a2da0 0x7f45400a5260 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45541069c0 0x7f45540fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554101350 0x7f45540fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 --2- 172.21.15.5:0/602347857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45541009b0 0x7f45540fcde0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 -- 172.21.15.5:0/602347857 >> 172.21.15.5:0/602347857 conn(0x7f4554077990 msgr2=0x7f45540699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.369+0000 7f455bbc4640 1 -- 172.21.15.5:0/602347857 shutdown_connections 2024-09-11T22:31:32.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:32.370+0000 7f455bbc4640 1 -- 172.21.15.5:0/602347857 wait complete. 2024-09-11T22:31:32.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:32 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:31:32.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:32 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:31:32.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:32 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:31:32.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:32 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:31:32.829 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:31:32 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setuser ceph since I am not root 2024-09-11T22:31:32.829 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:31:32 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setgroup ceph since I am not root 2024-09-11T22:31:32.829 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:31:32 smithi159 ceph-mds[65034]: -- 172.21.15.159:0/1689604723 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d48fdb860 con 0x558d48fb9400 2024-09-11T22:31:32.829 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:31:32 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: starting mds.k at 2024-09-11T22:31:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:32 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:31:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:32 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:31:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:32 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:31:32.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:32 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:31:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:32 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:31:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:32 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:31:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:32 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:31:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:32 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:31:33.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 706683 7.61 MB/sec execute 351 sec latency 70.993 ms 2024-09-11T22:31:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:33 smithi005 ceph-mon[30943]: pgmap v293: 97 pgs: 97 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-09-11T22:31:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:33 smithi005 ceph-mon[30943]: daemon mds.k restarted 2024-09-11T22:31:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:33 smithi159 ceph-mon[35534]: pgmap v293: 97 pgs: 97 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-09-11T22:31:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:33 smithi159 ceph-mon[35534]: daemon mds.k restarted 2024-09-11T22:31:33.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:33 smithi172 ceph-mon[35461]: pgmap v293: 97 pgs: 97 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-09-11T22:31:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:33 smithi172 ceph-mon[35461]: daemon mds.k restarted 2024-09-11T22:31:34.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 708092 7.61 MB/sec execute 352 sec latency 47.450 ms 2024-09-11T22:31:34.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:34 smithi005 ceph-mon[30943]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T22:31:34.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:34 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6832/1751117477,v1:172.21.15.159:6833/1751117477] up:boot 2024-09-11T22:31:34.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:34 smithi005 ceph-mon[30943]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:31:34.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:31:34.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:34 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:31:34.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:34 smithi159 ceph-mon[35534]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T22:31:34.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:34 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6832/1751117477,v1:172.21.15.159:6833/1751117477] up:boot 2024-09-11T22:31:34.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:34 smithi159 ceph-mon[35534]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:31:34.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:31:34.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:34 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:31:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:34 smithi172 ceph-mon[35461]: osdmap e96: 12 total, 12 up, 12 in 2024-09-11T22:31:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:34 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6832/1751117477,v1:172.21.15.159:6833/1751117477] up:boot 2024-09-11T22:31:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:34 smithi172 ceph-mon[35461]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:31:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:31:34.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:34 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:31:35.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 710297 7.62 MB/sec execute 353 sec latency 18.242 ms 2024-09-11T22:31:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:35 smithi005 ceph-mon[30943]: pgmap v295: 97 pgs: 97 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T22:31:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:35 smithi159 ceph-mon[35534]: pgmap v295: 97 pgs: 97 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T22:31:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:35 smithi172 ceph-mon[35461]: pgmap v295: 97 pgs: 97 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T22:31:36.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 711862 7.62 MB/sec execute 354 sec latency 20.512 ms 2024-09-11T22:31:37.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 713911 7.63 MB/sec execute 355 sec latency 13.120 ms 2024-09-11T22:31:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:37 smithi005 ceph-mon[30943]: pgmap v296: 97 pgs: 97 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T22:31:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:37 smithi159 ceph-mon[35534]: pgmap v296: 97 pgs: 97 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T22:31:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:37 smithi172 ceph-mon[35461]: pgmap v296: 97 pgs: 97 active+clean; 106 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 181 op/s 2024-09-11T22:31:38.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 715732 7.63 MB/sec execute 356 sec latency 22.509 ms 2024-09-11T22:31:39.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 717456 7.64 MB/sec execute 357 sec latency 62.167 ms 2024-09-11T22:31:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:39 smithi005 ceph-mon[30943]: pgmap v297: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-09-11T22:31:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:39 smithi159 ceph-mon[35534]: pgmap v297: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-09-11T22:31:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:39 smithi172 ceph-mon[35461]: pgmap v297: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-09-11T22:31:40.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 719077 7.64 MB/sec execute 358 sec latency 21.996 ms 2024-09-11T22:31:41.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 721015 7.65 MB/sec execute 359 sec latency 13.929 ms 2024-09-11T22:31:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:41 smithi005 ceph-mon[30943]: pgmap v298: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-09-11T22:31:41.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:41 smithi159 ceph-mon[35534]: pgmap v298: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-09-11T22:31:41.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:41 smithi172 ceph-mon[35461]: pgmap v298: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-09-11T22:31:42.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 722682 7.65 MB/sec execute 360 sec latency 24.011 ms 2024-09-11T22:31:43.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 724617 7.67 MB/sec execute 361 sec latency 22.065 ms 2024-09-11T22:31:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:43 smithi005 ceph-mon[30943]: pgmap v299: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-09-11T22:31:43.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:43 smithi159 ceph-mon[35534]: pgmap v299: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-09-11T22:31:43.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:43 smithi172 ceph-mon[35461]: pgmap v299: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-09-11T22:31:44.335 INFO:tasks.workunit.client.0.smithi005.stdout: 1 726099 7.66 MB/sec execute 362 sec latency 50.071 ms 2024-09-11T22:31:45.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 728087 7.67 MB/sec execute 363 sec latency 30.416 ms 2024-09-11T22:31:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.464+0000 7fe0f3c23640 1 -- 172.21.15.5:0/3161879802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec101390 msgr2=0x7fe0ec10d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:45.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.464+0000 7fe0f3c23640 1 --2- 172.21.15.5:0/3161879802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec101390 0x7fe0ec10d8c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009930 tx=0x7fe0e402a420 comp rx=0 tx=0).stop 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.464+0000 7fe0f3c23640 1 -- 172.21.15.5:0/3161879802 shutdown_connections 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.464+0000 7fe0f3c23640 1 --2- 172.21.15.5:0/3161879802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec101390 0x7fe0ec10d8c0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.464+0000 7fe0f3c23640 1 --2- 172.21.15.5:0/3161879802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0ec1009f0 0x7fe0ec100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.464+0000 7fe0f3c23640 1 --2- 172.21.15.5:0/3161879802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec106a00 0x7fe0ec106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.464+0000 7fe0f3c23640 1 -- 172.21.15.5:0/3161879802 >> 172.21.15.5:0/3161879802 conn(0x7fe0ec0fc6b0 msgr2=0x7fe0ec0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.465+0000 7fe0f3c23640 1 -- 172.21.15.5:0/3161879802 shutdown_connections 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.465+0000 7fe0f3c23640 1 -- 172.21.15.5:0/3161879802 wait complete. 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.465+0000 7fe0f3c23640 1 Processor -- start 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.465+0000 7fe0f3c23640 1 -- start start 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.465+0000 7fe0f3c23640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec1009f0 0x7fe0ec198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.465+0000 7fe0f3c23640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0ec101390 0x7fe0ec1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.465+0000 7fe0f3c23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec106a00 0x7fe0ec19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.465+0000 7fe0f3c23640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec110e40 con 0x7fe0ec106a00 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.465+0000 7fe0f3c23640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec110cc0 con 0x7fe0ec101390 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.465+0000 7fe0f3c23640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec110fc0 con 0x7fe0ec1009f0 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f21c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec106a00 0x7fe0ec19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f19bf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec1009f0 0x7fe0ec198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f21c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec106a00 0x7fe0ec19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41224/0 (socket says 172.21.15.5:41224) 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f19bf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec1009f0 0x7fe0ec198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:41490/0 (socket says 172.21.15.5:41490) 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f21c0640 1 -- 172.21.15.5:0/2669194641 learned_addr learned my addr 172.21.15.5:0/2669194641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f11be640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0ec101390 0x7fe0ec1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f19bf640 1 -- 172.21.15.5:0/2669194641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0ec101390 msgr2=0x7fe0ec1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:45.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f19bf640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0ec101390 0x7fe0ec1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f19bf640 1 -- 172.21.15.5:0/2669194641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec106a00 msgr2=0x7fe0ec19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f19bf640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec106a00 0x7fe0ec19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f19bf640 1 -- 172.21.15.5:0/2669194641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0ec19d350 con 0x7fe0ec1009f0 2024-09-11T22:31:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f19bf640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec1009f0 0x7fe0ec198360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009fd0 tx=0x7fe0dc00ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0eaffd640 1 -- 172.21.15.5:0/2669194641 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc004b20 con 0x7fe0ec1009f0 2024-09-11T22:31:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0eaffd640 1 -- 172.21.15.5:0/2669194641 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0dc004cc0 con 0x7fe0ec1009f0 2024-09-11T22:31:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0eaffd640 1 -- 172.21.15.5:0/2669194641 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc0107b0 con 0x7fe0ec1009f0 2024-09-11T22:31:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f3c23640 1 -- 172.21.15.5:0/2669194641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0ec19d640 con 0x7fe0ec1009f0 2024-09-11T22:31:45.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.466+0000 7fe0f3c23640 1 -- 172.21.15.5:0/2669194641 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0ec1a4ed0 con 0x7fe0ec1009f0 2024-09-11T22:31:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.469+0000 7fe0eaffd640 1 -- 172.21.15.5:0/2669194641 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe0dc005020 con 0x7fe0ec1009f0 2024-09-11T22:31:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.469+0000 7fe0f69bd740 1 Processor -- start 2024-09-11T22:31:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.469+0000 7fe0f69bd740 1 -- start start 2024-09-11T22:31:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.469+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f621157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.469+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f621166b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.469+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211670c0 0x55f6211694b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.469+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f6210bc2f0 con 0x55f6211579f0 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.469+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f621064270 con 0x55f621158310 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.469+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f6210b82b0 con 0x55f6211670c0 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f21c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f621157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f21c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f621157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41228/0 (socket says 172.21.15.5:41228) 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f21c0640 1 -- 172.21.15.5:0/3812993279 learned_addr learned my addr 172.21.15.5:0/3812993279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f19bf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f621166b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f11be640 1 --2- 172.21.15.5:0/3812993279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211670c0 0x55f6211694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f21c0640 1 -- 172.21.15.5:0/3812993279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211670c0 msgr2=0x55f6211694b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/3812993279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211670c0 0x55f6211694b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f21c0640 1 -- 172.21.15.5:0/3812993279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 msgr2=0x55f621166b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/3812993279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f621166b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f21c0640 1 -- 172.21.15.5:0/3812993279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f621063340 con 0x55f6211579f0 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0eaffd640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0d40a2da0 0x7fe0d40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/3812993279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f621157dd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4005bd0 tx=0x7fe0e4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e13a95bc181abf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0f19bf640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0d40a2da0 0x7fe0d40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0d3fff640 1 -- 172.21.15.5:0/3812993279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4040070 con 0x55f6211579f0 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0eaffd640 1 -- 172.21.15.5:0/2669194641 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7fe0dc0cfba0 con 0x7fe0ec1009f0 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0d3fff640 1 -- 172.21.15.5:0/3812993279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e40318f0 con 0x55f6211579f0 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.470+0000 7fe0d3fff640 1 -- 172.21.15.5:0/3812993279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4035040 con 0x55f6211579f0 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.471+0000 7fe0f19bf640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0d40a2da0 0x7fe0d40a5260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0cd360 tx=0x7fe0dc0bc040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.471+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3812993279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 msgr2=0x55f621157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.471+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/3812993279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f621157dd0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4005bd0 tx=0x7fe0e4005c00 comp rx=0 tx=0).stop 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.471+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3812993279 shutdown_connections 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.471+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/3812993279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211670c0 0x55f6211694b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.471+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/3812993279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f621166b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.471+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/3812993279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f621157dd0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.471+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3812993279 >> 172.21.15.5:0/3812993279 conn(0x55f621154060 msgr2=0x55f621154440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.471+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3812993279 shutdown_connections 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.471+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3812993279 wait complete. 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f69bd740 1 -- start start 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f69bd740 1 Processor -- start 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211579f0 0x55f621163fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f621164520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f621164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f621169cc0 con 0x55f6211670c0 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f621045290 con 0x55f621158310 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f621169e40 con 0x55f6211579f0 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f69bd740 10 client.0 _ll_register_callbacks cb 0x55f621076140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f21c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f621164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f11be640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f621164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f21c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f621164a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41240/0 (socket says 172.21.15.5:41240) 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f21c0640 1 -- 172.21.15.5:0/3950243047 learned_addr learned my addr 172.21.15.5:0/3950243047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.472+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211579f0 0x55f621163fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0f11be640 1 -- 172.21.15.5:0/3950243047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211579f0 msgr2=0x55f621163fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0f11be640 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211579f0 0x55f621163fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0f11be640 1 -- 172.21.15.5:0/3950243047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 msgr2=0x55f621164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0f11be640 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f621164a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0f11be640 1 -- 172.21.15.5:0/3950243047 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f620dcf210 con 0x55f621158310 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f621164a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0f11be640 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f621164520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec1050d0 tx=0x7fe0e0012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ffe6a420feb319b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0d17fa640 1 -- 172.21.15.5:0/3950243047 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e000f920 con 0x55f621158310 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0d17fa640 1 -- 172.21.15.5:0/3950243047 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7fe0e000fac0 con 0x55f621158310 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3950243047 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e40318f0 con 0x55f621158310 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0d17fa640 1 -- 172.21.15.5:0/3950243047 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe0e000fcc0 con 0x55f621158310 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0f69bd740 10 client.27656 fetch_fsmap learned FSMap version 42 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.473+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3950243047 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f620e9b620 con 0x55f621158310 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0d17fa640 1 -- 172.21.15.5:0/3950243047 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e40318f0 con 0x55f621158310 2024-09-11T22:31:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0d17fa640 1 -- 172.21.15.5:0/3950243047 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fe0e0022030 con 0x55f621158310 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0f69bd740 10 client.27656 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0f69bd740 10 client.27656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0f69bd740 20 client.27656 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f6211771a0 0x55f621179590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0f69bd740 4 client.27656 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3950243047 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6210c9a20 con 0x55f6211771a0 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f6211771a0 0x55f621179590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0d17fa640 10 client.27656 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.474+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f6211771a0 0x55f621179590 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e3a51b9d4ae90e5 server_cookie=6df2a7baf3fae614 in_seq=0 out_seq=0 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0d17fa640 1 -- 172.21.15.5:0/3950243047 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6210c9a20 con 0x55f6211771a0 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0d17fa640 10 client.27656 handle_command_reply: tid=0 2024-09-11T22:31:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0f69bd740 1 client.27656 shutdown 2024-09-11T22:31:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3950243047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 msgr2=0x55f621164520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f621164520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec1050d0 tx=0x7fe0e0012b60 comp rx=0 tx=0).stop 2024-09-11T22:31:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3950243047 shutdown_connections 2024-09-11T22:31:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f6211771a0 0x55f621179590 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f621164a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f621164520 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0d17fa640 0 client.27656 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/3950243047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211579f0 0x55f621163fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.475+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3950243047 >> 172.21.15.5:0/3950243047 conn(0x55f621154060 msgr2=0x55f621159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.476+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3950243047 shutdown_connections 2024-09-11T22:31:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.476+0000 7fe0f69bd740 1 -- 172.21.15.5:0/3950243047 wait complete. 2024-09-11T22:31:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.476+0000 7fe0f69bd740 20 client.27656 trim_cache size 0 max 16384 2024-09-11T22:31:45.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f69bd740 1 Processor -- start 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f69bd740 1 -- start start 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211579f0 0x55f621159c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f62117ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f62117f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1ab760 con 0x55f6211670c0 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1ad2a0 con 0x55f621158310 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1abe30 con 0x55f6211579f0 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f21c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f62117f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f11be640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f62117ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f21c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f62117f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41248/0 (socket says 172.21.15.5:41248) 2024-09-11T22:31:45.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f21c0640 1 -- 172.21.15.5:0/2550007196 learned_addr learned my addr 172.21.15.5:0/2550007196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/2550007196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211579f0 0x55f621159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f21c0640 1 -- 172.21.15.5:0/2550007196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211579f0 msgr2=0x55f621159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.490+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/2550007196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211579f0 0x55f621159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.491+0000 7fe0f21c0640 1 -- 172.21.15.5:0/2550007196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 msgr2=0x55f62117ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.491+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/2550007196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f62117ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.491+0000 7fe0f21c0640 1 -- 172.21.15.5:0/2550007196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f621154fc0 con 0x55f6211670c0 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.491+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/2550007196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f62117f3a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006010 tx=0x7fe0e40053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc16de81127b640f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.491+0000 7fe0d37fe640 1 -- 172.21.15.5:0/2550007196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4035070 con 0x55f6211670c0 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.491+0000 7fe0d37fe640 1 -- 172.21.15.5:0/2550007196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e4031850 con 0x55f6211670c0 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.491+0000 7fe0d37fe640 1 -- 172.21.15.5:0/2550007196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4040040 con 0x55f6211670c0 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.491+0000 7fe0f69bd740 1 -- 172.21.15.5:0/2550007196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 msgr2=0x55f62117f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:45.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.491+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/2550007196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f62117f3a0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006010 tx=0x7fe0e40053c0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 -- 172.21.15.5:0/2550007196 shutdown_connections 2024-09-11T22:31:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/2550007196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211670c0 0x55f62117f3a0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/2550007196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f62117ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/2550007196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211579f0 0x55f621159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 -- 172.21.15.5:0/2550007196 >> 172.21.15.5:0/2550007196 conn(0x55f621154060 msgr2=0x55f621159530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 -- 172.21.15.5:0/2550007196 shutdown_connections 2024-09-11T22:31:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 -- 172.21.15.5:0/2550007196 wait complete. 2024-09-11T22:31:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 -- start start 2024-09-11T22:31:45.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 Processor -- start 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f62116f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f62116f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211670c0 0x55f62116a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1acbd0 con 0x55f6211579f0 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1ac500 con 0x55f621158310 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe0ec1a9740 con 0x55f6211670c0 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f69bd740 10 client.0 _ll_register_callbacks cb 0x7fe0e0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.492+0000 7fe0f21c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f62116f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f21c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f62116f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41258/0 (socket says 172.21.15.5:41258) 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f21c0640 1 -- 172.21.15.5:0/4007138733 learned_addr learned my addr 172.21.15.5:0/4007138733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:45.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f11be640 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f62116f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211670c0 0x55f62116a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f21c0640 1 -- 172.21.15.5:0/4007138733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211670c0 msgr2=0x55f62116a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211670c0 0x55f62116a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f21c0640 1 -- 172.21.15.5:0/4007138733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 msgr2=0x55f62116f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f62116f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f21c0640 1 -- 172.21.15.5:0/4007138733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62116ae40 con 0x55f6211579f0 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f21c0640 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f62116f1a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4005390 tx=0x7fe0e402ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929ee70c322c4f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0d17fa640 1 -- 172.21.15.5:0/4007138733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4048030 con 0x55f6211579f0 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0d17fa640 1 -- 172.21.15.5:0/4007138733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7fe0e40553d0 con 0x55f6211579f0 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f69bd740 1 -- 172.21.15.5:0/4007138733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e4031850 con 0x55f6211579f0 2024-09-11T22:31:45.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0d17fa640 1 -- 172.21.15.5:0/4007138733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe0e4040070 con 0x55f6211579f0 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f69bd740 10 client.18384 fetch_fsmap learned FSMap version 42 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0f69bd740 1 -- 172.21.15.5:0/4007138733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f6211e18a0 con 0x55f6211579f0 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.493+0000 7fe0d17fa640 1 -- 172.21.15.5:0/4007138733 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e4031850 con 0x55f6211579f0 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.494+0000 7fe0d17fa640 1 -- 172.21.15.5:0/4007138733 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fe0e4051070 con 0x55f6211579f0 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.494+0000 7fe0f69bd740 10 client.18384 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.494+0000 7fe0f69bd740 10 client.18384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.494+0000 7fe0f69bd740 20 client.18384 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.494+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f6211e1b20 0x55f6211e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.494+0000 7fe0f69bd740 4 client.18384 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.494+0000 7fe0f69bd740 1 -- 172.21.15.5:0/4007138733 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6210c3d70 con 0x55f6211e1b20 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.494+0000 7fe0f11be640 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f6211e1b20 0x55f6211e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.494+0000 7fe0f11be640 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f6211e1b20 0x55f6211e3f10 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5550bd647a428aa9 server_cookie=86edc4d02c73aa12 in_seq=0 out_seq=0 2024-09-11T22:31:45.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.494+0000 7fe0d17fa640 10 client.18384 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0d17fa640 1 -- 172.21.15.5:0/4007138733 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6210c3d70 con 0x55f6211e1b20 2024-09-11T22:31:45.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0d17fa640 10 client.18384 handle_command_reply: tid=0 2024-09-11T22:31:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 client.18384 shutdown 2024-09-11T22:31:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 -- 172.21.15.5:0/4007138733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 msgr2=0x55f62116f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f62116f1a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4005390 tx=0x7fe0e402ab60 comp rx=0 tx=0).stop 2024-09-11T22:31:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 -- 172.21.15.5:0/4007138733 shutdown_connections 2024-09-11T22:31:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f6211e1b20 0x55f6211e3f10 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f6211670c0 0x55f62116a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0d17fa640 0 client.18384 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:45.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f621158310 0x55f62116f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 --2- 172.21.15.5:0/4007138733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6211579f0 0x55f62116f1a0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 -- 172.21.15.5:0/4007138733 >> 172.21.15.5:0/4007138733 conn(0x55f621154060 msgr2=0x55f621159530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 -- 172.21.15.5:0/4007138733 shutdown_connections 2024-09-11T22:31:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.607+0000 7fe0f69bd740 1 -- 172.21.15.5:0/4007138733 wait complete. 2024-09-11T22:31:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.608+0000 7fe0f69bd740 20 client.18384 trim_cache size 0 max 16384 2024-09-11T22:31:45.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:31:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:31:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:31:45.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:31:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.608+0000 7fe0f3c23640 1 -- 172.21.15.5:0/2669194641 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0d40a2da0 msgr2=0x7fe0d40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.608+0000 7fe0f3c23640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0d40a2da0 0x7fe0d40a5260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0cd360 tx=0x7fe0dc0bc040 comp rx=0 tx=0).stop 2024-09-11T22:31:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.608+0000 7fe0f3c23640 1 -- 172.21.15.5:0/2669194641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec1009f0 msgr2=0x7fe0ec198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.608+0000 7fe0f3c23640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec1009f0 0x7fe0ec198360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009fd0 tx=0x7fe0dc00ef90 comp rx=0 tx=0).stop 2024-09-11T22:31:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.609+0000 7fe0f3c23640 1 -- 172.21.15.5:0/2669194641 shutdown_connections 2024-09-11T22:31:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.609+0000 7fe0f3c23640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0ec106a00 0x7fe0ec19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.609+0000 7fe0f3c23640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe0d40a2da0 0x7fe0d40a5260 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.609+0000 7fe0f3c23640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0ec101390 0x7fe0ec1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.609+0000 7fe0f3c23640 1 --2- 172.21.15.5:0/2669194641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0ec1009f0 0x7fe0ec198360 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.609+0000 7fe0f3c23640 1 -- 172.21.15.5:0/2669194641 >> 172.21.15.5:0/2669194641 conn(0x7fe0ec0fc6b0 msgr2=0x7fe0ec06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.609+0000 7fe0f3c23640 1 -- 172.21.15.5:0/2669194641 shutdown_connections 2024-09-11T22:31:45.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:45.609+0000 7fe0f3c23640 1 -- 172.21.15.5:0/2669194641 wait complete. 2024-09-11T22:31:45.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:45 smithi005 ceph-mon[30943]: pgmap v300: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-09-11T22:31:45.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:45 smithi159 ceph-mon[35534]: pgmap v300: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-09-11T22:31:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:45 smithi172 ceph-mon[35461]: pgmap v300: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-09-11T22:31:46.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 729586 7.67 MB/sec execute 364 sec latency 54.317 ms 2024-09-11T22:31:47.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 731651 7.68 MB/sec execute 365 sec latency 16.384 ms 2024-09-11T22:31:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:47 smithi005 ceph-mon[30943]: pgmap v301: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-11T22:31:47.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:47 smithi159 ceph-mon[35534]: pgmap v301: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-11T22:31:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:47 smithi172 ceph-mon[35461]: pgmap v301: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-11T22:31:48.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 733490 7.68 MB/sec execute 366 sec latency 18.770 ms 2024-09-11T22:31:49.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 735278 7.70 MB/sec execute 367 sec latency 93.789 ms 2024-09-11T22:31:49.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:49 smithi005 ceph-mon[30943]: pgmap v302: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 435 op/s 2024-09-11T22:31:49.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:49 smithi159 ceph-mon[35534]: pgmap v302: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 435 op/s 2024-09-11T22:31:49.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:49 smithi172 ceph-mon[35461]: pgmap v302: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 435 op/s 2024-09-11T22:31:50.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 737016 7.69 MB/sec execute 368 sec latency 19.272 ms 2024-09-11T22:31:51.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 738871 7.71 MB/sec execute 369 sec latency 17.628 ms 2024-09-11T22:31:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:51 smithi005 ceph-mon[30943]: pgmap v303: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-09-11T22:31:51.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:51 smithi159 ceph-mon[35534]: pgmap v303: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-09-11T22:31:51.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:51 smithi172 ceph-mon[35461]: pgmap v303: 97 pgs: 97 active+clean; 97 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 253 op/s 2024-09-11T22:31:52.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 740986 7.71 MB/sec execute 370 sec latency 14.932 ms 2024-09-11T22:31:53.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 742538 7.72 MB/sec execute 371 sec latency 13.236 ms 2024-09-11T22:31:53.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:53 smithi005 ceph-mon[30943]: pgmap v304: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 349 op/s 2024-09-11T22:31:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:53 smithi159 ceph-mon[35534]: pgmap v304: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 349 op/s 2024-09-11T22:31:53.836 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:31:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:53 smithi172 ceph-mon[35461]: pgmap v304: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 349 op/s 2024-09-11T22:31:54.122 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:54.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 744331 7.71 MB/sec execute 372 sec latency 110.192 ms 2024-09-11T22:31:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 -- 172.21.15.5:0/3896374791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0102d90 msgr2=0x7fd6a0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 --2- 172.21.15.5:0/3896374791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0102d90 0x7fd6a0103170 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd690009cd0 tx=0x7fd69002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 -- 172.21.15.5:0/3896374791 shutdown_connections 2024-09-11T22:31:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 --2- 172.21.15.5:0/3896374791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0108290 0x7fd6a010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 --2- 172.21.15.5:0/3896374791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0103740 0x7fd6a0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 --2- 172.21.15.5:0/3896374791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0102d90 0x7fd6a0103170 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 -- 172.21.15.5:0/3896374791 >> 172.21.15.5:0/3896374791 conn(0x7fd6a00fc7d0 msgr2=0x7fd6a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 -- 172.21.15.5:0/3896374791 shutdown_connections 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 -- 172.21.15.5:0/3896374791 wait complete. 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 Processor -- start 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.664+0000 7fd6a6d96640 1 -- start start 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd6a6d96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0102d90 0x7fd6a01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd6a6d96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0103740 0x7fd6a0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd6a6d96640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0108290 0x7fd6a0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd6a6d96640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01120d0 con 0x7fd6a0103740 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd6a6d96640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0111f50 con 0x7fd6a0108290 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd6a6d96640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0112250 con 0x7fd6a0102d90 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0103740 0x7fd6a0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0103740 0x7fd6a0100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38978/0 (socket says 172.21.15.5:38978) 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd69ffff640 1 -- 172.21.15.5:0/1696741929 learned_addr learned my addr 172.21.15.5:0/1696741929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd6a5333640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0108290 0x7fd6a0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd6a4b32640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0102d90 0x7fd6a01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd69ffff640 1 -- 172.21.15.5:0/1696741929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0102d90 msgr2=0x7fd6a01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd69ffff640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0102d90 0x7fd6a01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd69ffff640 1 -- 172.21.15.5:0/1696741929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0108290 msgr2=0x7fd6a0101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd69ffff640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0108290 0x7fd6a0101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd69ffff640 1 -- 172.21.15.5:0/1696741929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a01a6b00 con 0x7fd6a0103740 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.665+0000 7fd69ffff640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0103740 0x7fd6a0100f30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd68800b450 tx=0x7fd68800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.666+0000 7fd69dffb640 1 -- 172.21.15.5:0/1696741929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688015020 con 0x7fd6a0103740 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.666+0000 7fd6a6d96640 1 -- 172.21.15.5:0/1696741929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a01a6df0 con 0x7fd6a0103740 2024-09-11T22:31:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.666+0000 7fd69dffb640 1 -- 172.21.15.5:0/1696741929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd688011d70 con 0x7fd6a0103740 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.666+0000 7fd69dffb640 1 -- 172.21.15.5:0/1696741929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68800f340 con 0x7fd6a0103740 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.666+0000 7fd6a6d96640 1 -- 172.21.15.5:0/1696741929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a01a72a0 con 0x7fd6a0103740 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.668+0000 7fd69dffb640 1 -- 172.21.15.5:0/1696741929 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd688018020 con 0x7fd6a0103740 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a916f740 1 Processor -- start 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a916f740 1 -- start start 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 0x561bf5d7e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d89140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 0x561bf5d8ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a916f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561bf5ce42e0 con 0x561bf5d7edc0 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a916f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561bf5c8bce0 con 0x561bf5d89680 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a916f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561bf5ce02a0 con 0x561bf5d7e4a0 2024-09-11T22:31:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd69dffb640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd68c0a2cf0 0x7fd68c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a4b32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d89140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a4b32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d89140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38982/0 (socket says 172.21.15.5:38982) 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a4b32640 1 -- 172.21.15.5:0/2796744109 learned_addr learned my addr 172.21.15.5:0/2796744109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a5333640 1 --2- 172.21.15.5:0/2796744109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 0x561bf5d7e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd69dffb640 1 -- 172.21.15.5:0/1696741929 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7fd6880ced50 con 0x7fd6a0103740 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd69ffff640 1 --2- 172.21.15.5:0/2796744109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 0x561bf5d8ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.669+0000 7fd6a5333640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd68c0a2cf0 0x7fd68c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd6a5333640 1 -- 172.21.15.5:0/2796744109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 msgr2=0x561bf5d8ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd6a5333640 1 --2- 172.21.15.5:0/2796744109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 0x561bf5d8ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd6a5333640 1 -- 172.21.15.5:0/2796744109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 msgr2=0x561bf5d89140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd6a5333640 1 --2- 172.21.15.5:0/2796744109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d89140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd6a5333640 1 -- 172.21.15.5:0/2796744109 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bf5c70440 con 0x561bf5d7e4a0 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd69ffff640 1 --2- 172.21.15.5:0/2796744109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 0x561bf5d8ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd6a5333640 1 --2- 172.21.15.5:0/2796744109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 0x561bf5d7e880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd698011540 tx=0x7fd698011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea07da5d2b067813 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd6a5333640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd68c0a2cf0 0x7fd68c0a51b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd69801d000 tx=0x7fd69801f000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd696ffd640 1 -- 172.21.15.5:0/2796744109 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698004050 con 0x561bf5d7e4a0 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd696ffd640 1 -- 172.21.15.5:0/2796744109 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd698022020 con 0x561bf5d7e4a0 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.670+0000 7fd696ffd640 1 -- 172.21.15.5:0/2796744109 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69801e480 con 0x561bf5d7e4a0 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 -- 172.21.15.5:0/2796744109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 msgr2=0x561bf5d7e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 --2- 172.21.15.5:0/2796744109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 0x561bf5d7e880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd698011540 tx=0x7fd698011a00 comp rx=0 tx=0).stop 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 -- 172.21.15.5:0/2796744109 shutdown_connections 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 --2- 172.21.15.5:0/2796744109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 0x561bf5d8ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 --2- 172.21.15.5:0/2796744109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d89140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 --2- 172.21.15.5:0/2796744109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 0x561bf5d7e880 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 -- 172.21.15.5:0/2796744109 >> 172.21.15.5:0/2796744109 conn(0x561bf5d7aa30 msgr2=0x561bf5d7ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 -- 172.21.15.5:0/2796744109 shutdown_connections 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 -- 172.21.15.5:0/2796744109 wait complete. 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 -- start start 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.671+0000 7fd6a916f740 1 Processor -- start 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d7e4a0 0x561bf5d88fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d86270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 0x561bf5d867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd6a916f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561bf5d8c250 con 0x561bf5d7edc0 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd6a916f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561bf5c9f050 con 0x561bf5d7e4a0 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd6a916f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561bf5d8c3d0 con 0x561bf5d89680 2024-09-11T22:31:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd6a916f740 10 client.0 _ll_register_callbacks cb 0x561bf5cdf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d86270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d86270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38994/0 (socket says 172.21.15.5:38994) 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd69ffff640 1 -- 172.21.15.5:0/668275787 learned_addr learned my addr 172.21.15.5:0/668275787 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd6a5333640 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 0x561bf5d867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd6a4b32640 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d7e4a0 0x561bf5d88fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd69ffff640 1 -- 172.21.15.5:0/668275787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 msgr2=0x561bf5d867b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd69ffff640 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 0x561bf5d867b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd69ffff640 1 -- 172.21.15.5:0/668275787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d7e4a0 msgr2=0x561bf5d88fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd69ffff640 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d7e4a0 0x561bf5d88fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd69ffff640 1 -- 172.21.15.5:0/668275787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bf5a553c0 con 0x561bf5d7edc0 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.672+0000 7fd6a5333640 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 0x561bf5d867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd69ffff640 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d86270 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd68809b040 tx=0x7fd68809b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20de903e642610ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd67ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd67ffff640 1 -- 172.21.15.5:0/668275787 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68809b340 con 0x561bf5d7edc0 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd67ffff640 1 -- 172.21.15.5:0/668275787 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7fd6880ccbb0 con 0x561bf5d7edc0 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd6a916f740 1 -- 172.21.15.5:0/668275787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd698022020 con 0x561bf5d7edc0 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd67ffff640 1 -- 172.21.15.5:0/668275787 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd6880b4300 con 0x561bf5d7edc0 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd6a916f740 10 client.18396 fetch_fsmap learned FSMap version 42 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd6a916f740 1 -- 172.21.15.5:0/668275787 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561bf5d97ba0 con 0x561bf5d7edc0 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd67ffff640 1 -- 172.21.15.5:0/668275787 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd698022020 con 0x561bf5d7edc0 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd67ffff640 1 -- 172.21.15.5:0/668275787 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fd68809a790 con 0x561bf5d7edc0 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd6a916f740 10 client.18396 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd6a916f740 10 client.18396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd6a916f740 20 client.18396 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:54.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd6a916f740 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561bf5d99050 0x561bf5d9b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd6a916f740 4 client.18396 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.673+0000 7fd6a916f740 1 -- 172.21.15.5:0/668275787 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bf5cf1850 con 0x561bf5d99050 2024-09-11T22:31:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.674+0000 7fd6a4b32640 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561bf5d99050 0x561bf5d9b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.674+0000 7fd67ffff640 10 client.18396 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.674+0000 7fd6a4b32640 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561bf5d99050 0x561bf5d9b440 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9dfce83c18e673 server_cookie=9ecf5ce9847ec670 in_seq=0 out_seq=0 2024-09-11T22:31:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd67ffff640 1 -- 172.21.15.5:0/668275787 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561bf5cf1850 con 0x561bf5d99050 2024-09-11T22:31:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd67ffff640 10 client.18396 handle_command_reply: tid=0 2024-09-11T22:31:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 client.18396 shutdown 2024-09-11T22:31:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 -- 172.21.15.5:0/668275787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 msgr2=0x561bf5d86270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:54.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d86270 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fd68809b040 tx=0x7fd68809b1d0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 -- 172.21.15.5:0/668275787 shutdown_connections 2024-09-11T22:31:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561bf5d99050 0x561bf5d9b440 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 0x561bf5d867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d86270 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd67ffff640 0 client.18396 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 --2- 172.21.15.5:0/668275787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d7e4a0 0x561bf5d88fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 -- 172.21.15.5:0/668275787 >> 172.21.15.5:0/668275787 conn(0x561bf5d7aa30 msgr2=0x561bf5d79b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 -- 172.21.15.5:0/668275787 shutdown_connections 2024-09-11T22:31:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 1 -- 172.21.15.5:0/668275787 wait complete. 2024-09-11T22:31:54.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.675+0000 7fd6a916f740 20 client.18396 trim_cache size 0 max 16384 2024-09-11T22:31:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd6a916f740 1 Processor -- start 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd6a916f740 1 -- start start 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d7e4a0 0x561bf5d954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5da0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 0x561bf5da1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd6a916f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01afbf0 con 0x561bf5d7edc0 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd6a916f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01b1730 con 0x561bf5d7e4a0 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd6a916f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01b02c0 con 0x561bf5d89680 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5da0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5da0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39010/0 (socket says 172.21.15.5:39010) 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd69ffff640 1 -- 172.21.15.5:0/3903357478 learned_addr learned my addr 172.21.15.5:0/3903357478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd6a4b32640 1 --2- 172.21.15.5:0/3903357478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d7e4a0 0x561bf5d954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd6a5333640 1 --2- 172.21.15.5:0/3903357478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 0x561bf5da1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd69ffff640 1 -- 172.21.15.5:0/3903357478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 msgr2=0x561bf5da1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd69ffff640 1 --2- 172.21.15.5:0/3903357478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 0x561bf5da1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd69ffff640 1 -- 172.21.15.5:0/3903357478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d7e4a0 msgr2=0x561bf5d954e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd69ffff640 1 --2- 172.21.15.5:0/3903357478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d7e4a0 0x561bf5d954e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.694+0000 7fd69ffff640 1 -- 172.21.15.5:0/3903357478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bf5d7e2d0 con 0x561bf5d7edc0 2024-09-11T22:31:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.695+0000 7fd69ffff640 1 --2- 172.21.15.5:0/3903357478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5da0cc0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd6880093f0 tx=0x7fd688010780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad47674c20827ee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.695+0000 7fd6967fc640 1 -- 172.21.15.5:0/3903357478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68809b340 con 0x561bf5d7edc0 2024-09-11T22:31:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.695+0000 7fd6967fc640 1 -- 172.21.15.5:0/3903357478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68809f6f0 con 0x561bf5d7edc0 2024-09-11T22:31:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.695+0000 7fd6967fc640 1 -- 172.21.15.5:0/3903357478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6880bb440 con 0x561bf5d7edc0 2024-09-11T22:31:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 -- 172.21.15.5:0/3903357478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 msgr2=0x561bf5da0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 --2- 172.21.15.5:0/3903357478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5da0cc0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd6880093f0 tx=0x7fd688010780 comp rx=0 tx=0).stop 2024-09-11T22:31:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 -- 172.21.15.5:0/3903357478 shutdown_connections 2024-09-11T22:31:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 --2- 172.21.15.5:0/3903357478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d89680 0x561bf5da1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 --2- 172.21.15.5:0/3903357478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5da0cc0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 --2- 172.21.15.5:0/3903357478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d7e4a0 0x561bf5d954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 -- 172.21.15.5:0/3903357478 >> 172.21.15.5:0/3903357478 conn(0x561bf5d7aa30 msgr2=0x561bf5d84b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 -- 172.21.15.5:0/3903357478 shutdown_connections 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 -- 172.21.15.5:0/3903357478 wait complete. 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 -- start start 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 Processor -- start 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.696+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 0x561bf5d8d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d8db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd6a916f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 0x561bf5d8e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd6a916f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01b1060 con 0x561bf5d7edc0 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd6a916f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01b0990 con 0x561bf5d89680 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd6a916f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a01adbd0 con 0x561bf5d7e4a0 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd6a916f740 10 client.0 _ll_register_callbacks cb 0x7fd6880092c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:54.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d8db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd69ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d8db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39020/0 (socket says 172.21.15.5:39020) 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd69ffff640 1 -- 172.21.15.5:0/1843657065 learned_addr learned my addr 172.21.15.5:0/1843657065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd6a4b32640 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 0x561bf5d8d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd6a5333640 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 0x561bf5d8e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd69ffff640 1 -- 172.21.15.5:0/1843657065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 msgr2=0x561bf5d8d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd69ffff640 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 0x561bf5d8d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd69ffff640 1 -- 172.21.15.5:0/1843657065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 msgr2=0x561bf5d8e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd69ffff640 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 0x561bf5d8e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd69ffff640 1 -- 172.21.15.5:0/1843657065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bf5d872b0 con 0x561bf5d7edc0 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd6a4b32640 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 0x561bf5d8d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd69ffff640 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d8db00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd68809b040 tx=0x7fd6880047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff93bc977a39e296 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd67ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd67ffff640 1 -- 172.21.15.5:0/1843657065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6880cce40 con 0x561bf5d7edc0 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd67ffff640 1 -- 172.21.15.5:0/1843657065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7fd6880bb0b0 con 0x561bf5d7edc0 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.697+0000 7fd6a916f740 1 -- 172.21.15.5:0/1843657065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68809f6f0 con 0x561bf5d7edc0 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd67ffff640 1 -- 172.21.15.5:0/1843657065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd68809b340 con 0x561bf5d7edc0 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd6a916f740 10 client.18402 fetch_fsmap learned FSMap version 42 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd6a916f740 1 -- 172.21.15.5:0/1843657065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561bf5e03fb0 con 0x561bf5d7edc0 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd67ffff640 1 -- 172.21.15.5:0/1843657065 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68809f6f0 con 0x561bf5d7edc0 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd67ffff640 1 -- 172.21.15.5:0/1843657065 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fd6880a0960 con 0x561bf5d7edc0 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd6a916f740 10 client.18402 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd6a916f740 10 client.18402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd6a916f740 20 client.18402 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd6a916f740 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561bf5e04430 0x561bf5e06820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd6a916f740 4 client.18402 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd6a916f740 1 -- 172.21.15.5:0/1843657065 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561bf5d86b40 con 0x561bf5e04430 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.698+0000 7fd6a4b32640 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561bf5e04430 0x561bf5e06820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.699+0000 7fd6a4b32640 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561bf5e04430 0x561bf5e06820 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6148c8afddbd7d43 server_cookie=759f697f13123c9b in_seq=0 out_seq=0 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.699+0000 7fd67ffff640 10 client.18402 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.699+0000 7fd67ffff640 1 -- 172.21.15.5:0/1843657065 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561bf5d86b40 con 0x561bf5e04430 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.699+0000 7fd67ffff640 10 client.18402 handle_command_reply: tid=0 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.699+0000 7fd6a916f740 1 client.18402 shutdown 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.699+0000 7fd6a916f740 1 -- 172.21.15.5:0/1843657065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 msgr2=0x561bf5d8db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.699+0000 7fd6a916f740 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d8db00 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd68809b040 tx=0x7fd6880047d0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.699+0000 7fd6a916f740 1 -- 172.21.15.5:0/1843657065 shutdown_connections 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd6a916f740 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561bf5e04430 0x561bf5e06820 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd6a916f740 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561bf5d89680 0x561bf5d8e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd6a916f740 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561bf5d7edc0 0x561bf5d8db00 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd6a916f740 1 --2- 172.21.15.5:0/1843657065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561bf5d7e4a0 0x561bf5d8d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd6a916f740 1 -- 172.21.15.5:0/1843657065 >> 172.21.15.5:0/1843657065 conn(0x561bf5d7aa30 msgr2=0x561bf5d882f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd67ffff640 0 client.18402 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd6a916f740 1 -- 172.21.15.5:0/1843657065 shutdown_connections 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd6a916f740 1 -- 172.21.15.5:0/1843657065 wait complete. 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd6a916f740 20 client.18402 trim_cache size 0 max 16384 2024-09-11T22:31:54.706 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:31:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:31:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:31:54.707 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:31:54.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd6a6d96640 1 -- 172.21.15.5:0/1696741929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd68c0a2cf0 msgr2=0x7fd68c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.700+0000 7fd6a6d96640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd68c0a2cf0 0x7fd68c0a51b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd69801d000 tx=0x7fd69801f000 comp rx=0 tx=0).stop 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.701+0000 7fd6a6d96640 1 -- 172.21.15.5:0/1696741929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0103740 msgr2=0x7fd6a0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.701+0000 7fd6a6d96640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0103740 0x7fd6a0100f30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fd68800b450 tx=0x7fd68800b910 comp rx=0 tx=0).stop 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.701+0000 7fd6a6d96640 1 -- 172.21.15.5:0/1696741929 shutdown_connections 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.701+0000 7fd6a6d96640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd68c0a2cf0 0x7fd68c0a51b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.701+0000 7fd6a6d96640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6a0108290 0x7fd6a0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.701+0000 7fd6a6d96640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0103740 0x7fd6a0100f30 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.701+0000 7fd6a6d96640 1 --2- 172.21.15.5:0/1696741929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6a0102d90 0x7fd6a01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.701+0000 7fd6a6d96640 1 -- 172.21.15.5:0/1696741929 >> 172.21.15.5:0/1696741929 conn(0x7fd6a00fc7d0 msgr2=0x7fd6a0105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.701+0000 7fd6a6d96640 1 -- 172.21.15.5:0/1696741929 shutdown_connections 2024-09-11T22:31:54.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:54.701+0000 7fd6a6d96640 1 -- 172.21.15.5:0/1696741929 wait complete. 2024-09-11T22:31:55.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 745969 7.73 MB/sec execute 373 sec latency 20.679 ms 2024-09-11T22:31:55.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:55 smithi005 ceph-mon[30943]: pgmap v305: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-09-11T22:31:55.557 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:31:55.558 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:31:55.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:55 smithi159 ceph-mon[35534]: pgmap v305: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-09-11T22:31:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:55.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:55 smithi172 ceph-mon[35461]: pgmap v305: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-09-11T22:31:56.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 747925 7.72 MB/sec execute 374 sec latency 32.097 ms 2024-09-11T22:31:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.384+0000 7f78ed4ec640 1 -- 172.21.15.5:0/1101795155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78e81036e0 msgr2=0x7f78e8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.384+0000 7f78ed4ec640 1 --2- 172.21.15.5:0/1101795155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78e81036e0 0x7f78e8107b90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009930 tx=0x7f78d002a420 comp rx=0 tx=0).stop 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.384+0000 7f78ed4ec640 1 -- 172.21.15.5:0/1101795155 shutdown_connections 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.384+0000 7f78ed4ec640 1 --2- 172.21.15.5:0/1101795155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78e81082a0 0x7f78e810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.384+0000 7f78ed4ec640 1 --2- 172.21.15.5:0/1101795155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78e81036e0 0x7f78e8107b90 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.384+0000 7f78ed4ec640 1 --2- 172.21.15.5:0/1101795155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78e8102d30 0x7f78e8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.384+0000 7f78ed4ec640 1 -- 172.21.15.5:0/1101795155 >> 172.21.15.5:0/1101795155 conn(0x7f78e80fc7d0 msgr2=0x7f78e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.384+0000 7f78ed4ec640 1 -- 172.21.15.5:0/1101795155 shutdown_connections 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.384+0000 7f78ed4ec640 1 -- 172.21.15.5:0/1101795155 wait complete. 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78ed4ec640 1 Processor -- start 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78ed4ec640 1 -- start start 2024-09-11T22:31:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78ed4ec640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78e8102d30 0x7f78e8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78ed4ec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78e81036e0 0x7f78e8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78ed4ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78e81082a0 0x7f78e819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78ed4ec640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78e81120a0 con 0x7f78e81082a0 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78ed4ec640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f78e8111f20 con 0x7f78e8102d30 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78ed4ec640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f78e8112220 con 0x7f78e81036e0 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78e6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78e8102d30 0x7f78e8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78e6ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78e8102d30 0x7f78e8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36076/0 (socket says 172.21.15.5:36076) 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78e6ffd640 1 -- 172.21.15.5:0/3287042323 learned_addr learned my addr 172.21.15.5:0/3287042323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78e77fe640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78e81082a0 0x7f78e819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.385+0000 7f78dffff640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78e81036e0 0x7f78e8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78e6ffd640 1 -- 172.21.15.5:0/3287042323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78e81036e0 msgr2=0x7f78e8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78e6ffd640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78e81036e0 0x7f78e8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78e6ffd640 1 -- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78e81082a0 msgr2=0x7f78e819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78e6ffd640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78e81082a0 0x7f78e819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78e6ffd640 1 -- 172.21.15.5:0/3287042323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e819d5b0 con 0x7f78e8102d30 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78e77fe640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78e81082a0 0x7f78e819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78e6ffd640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78e8102d30 0x7f78e8198600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f78d800e9f0 tx=0x7f78d800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78e4ff9640 1 -- 172.21.15.5:0/3287042323 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d800cd80 con 0x7f78e8102d30 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78ed4ec640 1 -- 172.21.15.5:0/3287042323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e819d8a0 con 0x7f78e8102d30 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78e4ff9640 1 -- 172.21.15.5:0/3287042323 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78d8005500 con 0x7f78e8102d30 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78e4ff9640 1 -- 172.21.15.5:0/3287042323 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8010690 con 0x7f78e8102d30 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.386+0000 7f78ed4ec640 1 -- 172.21.15.5:0/3287042323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e81a5180 con 0x7f78e8102d30 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.387+0000 7f78ed4ec640 1 -- 172.21.15.5:0/3287042323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e8103110 con 0x7f78e8102d30 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.388+0000 7f78e4ff9640 1 -- 172.21.15.5:0/3287042323 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f78d80046c0 con 0x7f78e8102d30 2024-09-11T22:31:56.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.389+0000 7f78e4ff9640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78d40a2cf0 0x7f78d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.389+0000 7f78dffff640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78d40a2cf0 0x7f78d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.389+0000 7f78e4ff9640 1 -- 172.21.15.5:0/3287042323 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f78d80cf8d0 con 0x7f78e8102d30 2024-09-11T22:31:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.390+0000 7f78dffff640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78d40a2cf0 0x7f78d40a51b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f78d0005940 tx=0x7f78d00058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:56.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.394+0000 7f78e4ff9640 1 -- 172.21.15.5:0/3287042323 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78d809c450 con 0x7f78e8102d30 2024-09-11T22:31:56.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.612+0000 7f78ed4ec640 1 -- 172.21.15.5:0/3287042323 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78e810a2b0 con 0x7f78e8102d30 2024-09-11T22:31:56.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.613+0000 7f78e4ff9640 1 -- 172.21.15.5:0/3287042323 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11967 (secure 0 0 0) 0x7f78d809bd50 con 0x7f78e8102d30 2024-09-11T22:31:56.614 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:31:56.614 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-11T22:31:33:497496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:31:33.497479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27553":{"gid":27553,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1751117477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1751117477},{"type":"v1","addr":"172.21.15.159:6833","nonce":1751117477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:31:56.615 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-11T22:31:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 -- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78d40a2cf0 msgr2=0x7f78d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78d40a2cf0 0x7f78d40a51b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f78d0005940 tx=0x7f78d00058d0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 -- 172.21.15.5:0/3287042323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78e8102d30 msgr2=0x7f78e8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78e8102d30 0x7f78e8198600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f78d800e9f0 tx=0x7f78d800edf0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 -- 172.21.15.5:0/3287042323 shutdown_connections 2024-09-11T22:31:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78e81082a0 0x7f78e819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78e81036e0 0x7f78e8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78d40a2cf0 0x7f78d40a51b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 --2- 172.21.15.5:0/3287042323 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78e8102d30 0x7f78e8198600 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 -- 172.21.15.5:0/3287042323 >> 172.21.15.5:0/3287042323 conn(0x7f78e80fc7d0 msgr2=0x7f78e8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 -- 172.21.15.5:0/3287042323 shutdown_connections 2024-09-11T22:31:56.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:56.615+0000 7f78ed4ec640 1 -- 172.21.15.5:0/3287042323 wait complete. 2024-09-11T22:31:57.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 749458 7.74 MB/sec execute 375 sec latency 96.639 ms 2024-09-11T22:31:57.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:31:57.631 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:57.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:57 smithi005 ceph-mon[30943]: pgmap v306: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-09-11T22:31:57.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:57 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3287042323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:31:57.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:57 smithi159 ceph-mon[35534]: pgmap v306: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-09-11T22:31:57.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:57 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3287042323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:31:57.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:57 smithi172 ceph-mon[35461]: pgmap v306: 97 pgs: 97 active+clean; 103 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-09-11T22:31:57.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:57 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3287042323' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:31:58.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.157+0000 7f69ae6fc640 1 -- 172.21.15.5:0/80476185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69a8106b80 msgr2=0x7f69a8106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.157+0000 7f69ae6fc640 1 --2- 172.21.15.5:0/80476185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69a8106b80 0x7f69a8106f60 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6998009900 tx=0x7f699802a410 comp rx=0 tx=0).stop 2024-09-11T22:31:58.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.158+0000 7f69ae6fc640 1 -- 172.21.15.5:0/80476185 shutdown_connections 2024-09-11T22:31:58.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.158+0000 7f69ae6fc640 1 --2- 172.21.15.5:0/80476185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a8100aa0 0x7f69a810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.158+0000 7f69ae6fc640 1 --2- 172.21.15.5:0/80476185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69a8100100 0x7f69a8100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.158+0000 7f69ae6fc640 1 --2- 172.21.15.5:0/80476185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69a8106b80 0x7f69a8106f60 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.158+0000 7f69ae6fc640 1 -- 172.21.15.5:0/80476185 >> 172.21.15.5:0/80476185 conn(0x7f69a8077230 msgr2=0x7f69a80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.158+0000 7f69ae6fc640 1 -- 172.21.15.5:0/80476185 shutdown_connections 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.158+0000 7f69ae6fc640 1 -- 172.21.15.5:0/80476185 wait complete. 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.158+0000 7f69ae6fc640 1 Processor -- start 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.158+0000 7f69ae6fc640 1 -- start start 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69ae6fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a8100100 0x7f69a819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69ae6fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69a8100aa0 0x7f69a819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69ae6fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69a8106b80 0x7f69a819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69ae6fc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8112050 con 0x7f69a8100aa0 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69ae6fc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69a8111ed0 con 0x7f69a8106b80 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69ae6fc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81121d0 con 0x7f69a8100100 2024-09-11T22:31:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69a8100aa0 0x7f69a819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a8100100 0x7f69a819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a8100100 0x7f69a819a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59408/0 (socket says 172.21.15.5:59408) 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69a8100aa0 0x7f69a819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39066/0 (socket says 172.21.15.5:39066) 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a7fff640 1 -- 172.21.15.5:0/2399764864 learned_addr learned my addr 172.21.15.5:0/2399764864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69acc99640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69a8106b80 0x7f69a819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a77fe640 1 -- 172.21.15.5:0/2399764864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a8100100 msgr2=0x7f69a819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a77fe640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a8100100 0x7f69a819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a77fe640 1 -- 172.21.15.5:0/2399764864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69a8106b80 msgr2=0x7f69a819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a77fe640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69a8106b80 0x7f69a819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a77fe640 1 -- 172.21.15.5:0/2399764864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a819f6e0 con 0x7f69a8100aa0 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a77fe640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69a8100aa0 0x7f69a819ac70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f699c005fd0 tx=0x7f699c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.159+0000 7f69a57fa640 1 -- 172.21.15.5:0/2399764864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c009c30 con 0x7f69a8100aa0 2024-09-11T22:31:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.160+0000 7f69a57fa640 1 -- 172.21.15.5:0/2399764864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f699c009dd0 con 0x7f69a8100aa0 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.160+0000 7f69ae6fc640 1 -- 172.21.15.5:0/2399764864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a819f9d0 con 0x7f69a8100aa0 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.160+0000 7f69a57fa640 1 -- 172.21.15.5:0/2399764864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c010460 con 0x7f69a8100aa0 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.160+0000 7f69ae6fc640 1 -- 172.21.15.5:0/2399764864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a81a72b0 con 0x7f69a8100aa0 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.162+0000 7f69a57fa640 1 -- 172.21.15.5:0/2399764864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f699c003f50 con 0x7f69a8100aa0 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.162+0000 7f69b0ad5740 1 Processor -- start 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.162+0000 7f69b0ad5740 1 -- start start 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.162+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa8b40 0x5611e1aa7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.162+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aa7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.162+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 0x5611e1ab4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.162+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611e1a0d2f0 con 0x5611e1aa8b40 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.162+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5611e19b4cf0 con 0x5611e1ab2370 2024-09-11T22:31:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.162+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5611e1a092b0 con 0x5611e1aa75d0 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69acc99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa8b40 0x5611e1aa7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aa7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69acc99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa8b40 0x5611e1aa7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39068/0 (socket says 172.21.15.5:39068) 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 0x5611e1ab4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69acc99640 1 -- 172.21.15.5:0/2626037554 learned_addr learned my addr 172.21.15.5:0/2626037554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aa7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59422/0 (socket says 172.21.15.5:59422) 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a57fa640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69900a2cf0 0x7f69900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69acc99640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69900a2cf0 0x7f69900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a7fff640 1 -- 172.21.15.5:0/2626037554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 msgr2=0x5611e1ab4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a7fff640 1 --2- 172.21.15.5:0/2626037554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 0x5611e1ab4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a7fff640 1 -- 172.21.15.5:0/2626037554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa8b40 msgr2=0x5611e1aa7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a7fff640 1 --2- 172.21.15.5:0/2626037554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa8b40 0x5611e1aa7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a7fff640 1 -- 172.21.15.5:0/2626037554 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e19993a0 con 0x5611e1aa75d0 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69acc99640 1 --2- 172.21.15.5:0/2626037554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa8b40 0x5611e1aa7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a57fa640 1 -- 172.21.15.5:0/2399764864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f699c0cf020 con 0x7f69a8100aa0 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69a7fff640 1 --2- 172.21.15.5:0/2626037554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aa7a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6998009230 tx=0x7f6998005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98d7575e5ff70523 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f69acc99640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69900a2cf0 0x7f69900a51b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f69a0005c40 tx=0x7f69a0004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f698e7fc640 1 -- 172.21.15.5:0/2626037554 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998040070 con 0x5611e1aa75d0 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f698e7fc640 1 -- 172.21.15.5:0/2626037554 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69980317b0 con 0x5611e1aa75d0 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.163+0000 7f698e7fc640 1 -- 172.21.15.5:0/2626037554 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998035040 con 0x5611e1aa75d0 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.164+0000 7f69b0ad5740 1 -- 172.21.15.5:0/2626037554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 msgr2=0x5611e1aa7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.164+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/2626037554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aa7a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6998009230 tx=0x7f6998005e80 comp rx=0 tx=0).stop 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.164+0000 7f69b0ad5740 1 -- 172.21.15.5:0/2626037554 shutdown_connections 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.164+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/2626037554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 0x5611e1ab4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.164+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/2626037554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aa7a10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.164+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/2626037554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa8b40 0x5611e1aa7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.164+0000 7f69b0ad5740 1 -- 172.21.15.5:0/2626037554 >> 172.21.15.5:0/2626037554 conn(0x5611e1a70310 msgr2=0x5611e1aa3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.164+0000 7f69b0ad5740 1 -- 172.21.15.5:0/2626037554 shutdown_connections 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.164+0000 7f69b0ad5740 1 -- 172.21.15.5:0/2626037554 wait complete. 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69b0ad5740 1 -- start start 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69b0ad5740 1 Processor -- start 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1ab1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aaf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1ab2370 0x5611e1aaf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611e1ab5010 con 0x5611e1aa75d0 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5611e19c8060 con 0x5611e1aa8b40 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5611e1ab5190 con 0x5611e1ab2370 2024-09-11T22:31:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69b0ad5740 10 client.0 _ll_register_callbacks cb 0x5611e171b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1ab1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69a77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aaf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69a77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aaf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36104/0 (socket says 172.21.15.5:36104) 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1ab1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39072/0 (socket says 172.21.15.5:39072) 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69a77fe640 1 -- 172.21.15.5:0/1711951875 learned_addr learned my addr 172.21.15.5:0/1711951875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.165+0000 7f69acc99640 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1ab2370 0x5611e1aaf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69a77fe640 1 -- 172.21.15.5:0/1711951875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1ab2370 msgr2=0x5611e1aaf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69a77fe640 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1ab2370 0x5611e1aaf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69a77fe640 1 -- 172.21.15.5:0/1711951875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 msgr2=0x5611e1ab1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69a77fe640 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1ab1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69a77fe640 1 -- 172.21.15.5:0/1711951875 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e177e3c0 con 0x5611e1aa8b40 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69a77fe640 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aaf050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f699c0979b0 tx=0x7f699c099a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d32e47acba11a81e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69877fe640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69877fe640 1 -- 172.21.15.5:0/1711951875 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c020040 con 0x5611e1aa8b40 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69b0ad5740 1 -- 172.21.15.5:0/1711951875 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69980317b0 con 0x5611e1aa8b40 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69877fe640 1 -- 172.21.15.5:0/1711951875 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f699c0972a0 con 0x5611e1aa8b40 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69877fe640 1 -- 172.21.15.5:0/1711951875 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f699c09f850 con 0x5611e1aa8b40 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69b0ad5740 10 client.27695 fetch_fsmap learned FSMap version 42 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69b0ad5740 1 -- 172.21.15.5:0/1711951875 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5611e1ac0b40 con 0x5611e1aa8b40 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.166+0000 7f69877fe640 1 -- 172.21.15.5:0/1711951875 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69980317b0 con 0x5611e1aa8b40 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.167+0000 7f69877fe640 1 -- 172.21.15.5:0/1711951875 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f699c0d1070 con 0x5611e1aa8b40 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.167+0000 7f69b0ad5740 10 client.27695 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.167+0000 7f69b0ad5740 10 client.27695 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.167+0000 7f69b0ad5740 20 client.27695 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.167+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5611e1ac1f50 0x5611e1ac4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.167+0000 7f69b0ad5740 4 client.27695 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.167+0000 7f69b0ad5740 1 -- 172.21.15.5:0/1711951875 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611e1a1a860 con 0x5611e1ac1f50 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.167+0000 7f69a7fff640 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5611e1ac1f50 0x5611e1ac4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.167+0000 7f69a7fff640 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5611e1ac1f50 0x5611e1ac4360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ae6902fe26e59783 server_cookie=ad07fbcd1a2b6eef in_seq=0 out_seq=0 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.167+0000 7f69877fe640 10 client.27695 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69877fe640 1 -- 172.21.15.5:0/1711951875 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f699804fe80 con 0x5611e1ac1f50 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69877fe640 10 client.27695 handle_command_reply: tid=0 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 client.27695 shutdown 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 -- 172.21.15.5:0/1711951875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 msgr2=0x5611e1aaf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aaf050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f699c0979b0 tx=0x7f699c099a60 comp rx=0 tx=0).stop 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 -- 172.21.15.5:0/1711951875 shutdown_connections 2024-09-11T22:31:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5611e1ac1f50 0x5611e1ac4360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1ab2370 0x5611e1aaf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aaf050 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/1711951875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1ab1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69877fe640 0 client.27695 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:31:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 -- 172.21.15.5:0/1711951875 >> 172.21.15.5:0/1711951875 conn(0x5611e1a70310 msgr2=0x5611e1aa9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 -- 172.21.15.5:0/1711951875 shutdown_connections 2024-09-11T22:31:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.168+0000 7f69b0ad5740 1 -- 172.21.15.5:0/1711951875 wait complete. 2024-09-11T22:31:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.169+0000 7f69b0ad5740 20 client.27695 trim_cache size 0 max 16384 2024-09-11T22:31:58.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69b0ad5740 1 Processor -- start 2024-09-11T22:31:58.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69b0ad5740 1 -- start start 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1aa8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa8b40 0x5611e1aa3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 0x5611e1aa3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81afba0 con 0x5611e1aa75d0 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81b16e0 con 0x5611e1ab2370 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81b0270 con 0x5611e1aa8b40 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1aa8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69acc99640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 0x5611e1aa3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1aa8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39076/0 (socket says 172.21.15.5:39076) 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69a7fff640 1 -- 172.21.15.5:0/3346067979 learned_addr learned my addr 172.21.15.5:0/3346067979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69acc99640 1 --2- 172.21.15.5:0/3346067979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 0x5611e1aa3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36120/0 (socket says 172.21.15.5:36120) 2024-09-11T22:31:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69a77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa8b40 0x5611e1aa3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69a7fff640 1 -- 172.21.15.5:0/3346067979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa8b40 msgr2=0x5611e1aa3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69a7fff640 1 --2- 172.21.15.5:0/3346067979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa8b40 0x5611e1aa3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69a7fff640 1 -- 172.21.15.5:0/3346067979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 msgr2=0x5611e1aa3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69a7fff640 1 --2- 172.21.15.5:0/3346067979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 0x5611e1aa3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69a7fff640 1 -- 172.21.15.5:0/3346067979 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611e1ac4990 con 0x5611e1aa75d0 2024-09-11T22:31:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.183+0000 7f69acc99640 1 --2- 172.21.15.5:0/3346067979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 0x5611e1aa3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.184+0000 7f69a7fff640 1 --2- 172.21.15.5:0/3346067979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1aa8300 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6998006010 tx=0x7f6998005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107cb17e92df5ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.184+0000 7f698dffb640 1 -- 172.21.15.5:0/3346067979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998035040 con 0x5611e1aa75d0 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.184+0000 7f698dffb640 1 -- 172.21.15.5:0/3346067979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f699804a040 con 0x5611e1aa75d0 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.184+0000 7f698dffb640 1 -- 172.21.15.5:0/3346067979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998040040 con 0x5611e1aa75d0 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3346067979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 msgr2=0x5611e1aa8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/3346067979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1aa8300 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f6998006010 tx=0x7f6998005260 comp rx=0 tx=0).stop 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3346067979 shutdown_connections 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/3346067979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1ab2370 0x5611e1aa3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/3346067979 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa8b40 0x5611e1aa3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/3346067979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1aa75d0 0x5611e1aa8300 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3346067979 >> 172.21.15.5:0/3346067979 conn(0x5611e1a70310 msgr2=0x5611e1a9ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3346067979 shutdown_connections 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3346067979 wait complete. 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 -- start start 2024-09-11T22:31:58.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.185+0000 7f69b0ad5740 1 Processor -- start 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aac270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aac7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69b0ad5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1ab2370 0x5611e1aaccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81b1010 con 0x5611e1ab2370 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81b0940 con 0x5611e1aa8b40 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69b0ad5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69a81adb80 con 0x5611e1aa75d0 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69b0ad5740 10 client.0 _ll_register_callbacks cb 0x7f699c00df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aac7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69acc99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1ab2370 0x5611e1aaccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aac7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36126/0 (socket says 172.21.15.5:36126) 2024-09-11T22:31:58.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59452/0 (socket says 172.21.15.5:59452) 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a77fe640 1 -- 172.21.15.5:0/3953247803 learned_addr learned my addr 172.21.15.5:0/3953247803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a7fff640 1 -- 172.21.15.5:0/3953247803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 msgr2=0x5611e1aac7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a7fff640 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aac7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a7fff640 1 -- 172.21.15.5:0/3953247803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1ab2370 msgr2=0x5611e1aaccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a7fff640 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1ab2370 0x5611e1aaccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a7fff640 1 -- 172.21.15.5:0/3953247803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611e1aaa120 con 0x5611e1aa75d0 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a77fe640 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69a7fff640 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aac270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6998005230 tx=0x7f6998001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2126fa11073ba138 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.186+0000 7f69877fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69877fe640 1 -- 172.21.15.5:0/3953247803 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6998040070 con 0x5611e1aa75d0 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69877fe640 1 -- 172.21.15.5:0/3953247803 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f69980316f0 con 0x5611e1aa75d0 2024-09-11T22:31:58.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3953247803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f699804a040 con 0x5611e1aa75d0 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69877fe640 1 -- 172.21.15.5:0/3953247803 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6998066050 con 0x5611e1aa75d0 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69b0ad5740 10 client.27610 fetch_fsmap learned FSMap version 42 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3953247803 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5611e1aa4dd0 con 0x5611e1aa75d0 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69877fe640 1 -- 172.21.15.5:0/3953247803 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f699804a040 con 0x5611e1aa75d0 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69877fe640 1 -- 172.21.15.5:0/3953247803 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f6998003a10 con 0x5611e1aa75d0 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69b0ad5740 10 client.27610 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69b0ad5740 10 client.27610 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69b0ad5740 20 client.27610 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5611e1b2cef0 0x5611e1b2f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69b0ad5740 4 client.27610 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.187+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3953247803 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611e1a14d70 con 0x5611e1b2cef0 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69a77fe640 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5611e1b2cef0 0x5611e1b2f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69a77fe640 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5611e1b2cef0 0x5611e1b2f2e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6a83cb77bb38ac8b server_cookie=ae1a2813b6144fe9 in_seq=0 out_seq=0 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69877fe640 10 client.27610 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:31:58.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69877fe640 1 -- 172.21.15.5:0/3953247803 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611e1a14d70 con 0x5611e1b2cef0 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69877fe640 10 client.27610 handle_command_reply: tid=0 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 client.27610 shutdown 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3953247803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 msgr2=0x5611e1aac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aac270 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6998005230 tx=0x7f6998001e70 comp rx=0 tx=0).stop 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3953247803 shutdown_connections 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5611e1b2cef0 0x5611e1b2f2e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611e1ab2370 0x5611e1aaccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611e1aa8b40 0x5611e1aac7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69877fe640 0 client.27610 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 --2- 172.21.15.5:0/3953247803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611e1aa75d0 0x5611e1aac270 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3953247803 >> 172.21.15.5:0/3953247803 conn(0x5611e1a70310 msgr2=0x5611e1aa3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3953247803 shutdown_connections 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 1 -- 172.21.15.5:0/3953247803 wait complete. 2024-09-11T22:31:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.188+0000 7f69b0ad5740 20 client.27610 trim_cache size 0 max 16384 2024-09-11T22:31:58.202 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:31:58.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 -- 172.21.15.5:0/2399764864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69900a2cf0 msgr2=0x7f69900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69900a2cf0 0x7f69900a51b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f69a0005c40 tx=0x7f69a0004ab0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 -- 172.21.15.5:0/2399764864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69a8100aa0 msgr2=0x7f69a819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69a8100aa0 0x7f69a819ac70 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f699c005fd0 tx=0x7f699c00dde0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 -- 172.21.15.5:0/2399764864 shutdown_connections 2024-09-11T22:31:58.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69900a2cf0 0x7f69900a51b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69a8106b80 0x7f69a819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69a8100aa0 0x7f69a819ac70 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 --2- 172.21.15.5:0/2399764864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69a8100100 0x7f69a819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 -- 172.21.15.5:0/2399764864 >> 172.21.15.5:0/2399764864 conn(0x7f69a8077230 msgr2=0x7f69a8104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 -- 172.21.15.5:0/2399764864 shutdown_connections 2024-09-11T22:31:58.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:58.189+0000 7f69ae6fc640 1 -- 172.21.15.5:0/2399764864 wait complete. 2024-09-11T22:31:58.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 751360 7.73 MB/sec execute 376 sec latency 40.849 ms 2024-09-11T22:31:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.705+0000 7f8615c2e640 1 -- 172.21.15.5:0/2852680680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8610101ae0 msgr2=0x7f8610101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.705+0000 7f8615c2e640 1 --2- 172.21.15.5:0/2852680680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8610101ae0 0x7f8610101f40 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8604009930 tx=0x7f860402a420 comp rx=0 tx=0).stop 2024-09-11T22:31:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.705+0000 7f8615c2e640 1 -- 172.21.15.5:0/2852680680 shutdown_connections 2024-09-11T22:31:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.705+0000 7f8615c2e640 1 --2- 172.21.15.5:0/2852680680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8610102480 0x7f861010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.705+0000 7f8615c2e640 1 --2- 172.21.15.5:0/2852680680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8610101ae0 0x7f8610101f40 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.705+0000 7f8615c2e640 1 --2- 172.21.15.5:0/2852680680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8610107b80 0x7f8610107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.705+0000 7f8615c2e640 1 -- 172.21.15.5:0/2852680680 >> 172.21.15.5:0/2852680680 conn(0x7f86100fd7e0 msgr2=0x7f86100ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.705+0000 7f8615c2e640 1 -- 172.21.15.5:0/2852680680 shutdown_connections 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.705+0000 7f8615c2e640 1 -- 172.21.15.5:0/2852680680 wait complete. 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f8615c2e640 1 Processor -- start 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f8615c2e640 1 -- start start 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f8615c2e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8610101ae0 0x7f86100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f8615c2e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8610102480 0x7f8610074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f8615c2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8610107b80 0x7f8610074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f8615c2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8610111f30 con 0x7f8610107b80 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f8615c2e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8610111db0 con 0x7f8610101ae0 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f8615c2e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f86101120b0 con 0x7f8610102480 2024-09-11T22:31:58.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f860ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8610107b80 0x7f8610074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f860effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8610102480 0x7f8610074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f860f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8610101ae0 0x7f86100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f860ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8610107b80 0x7f8610074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39092/0 (socket says 172.21.15.5:39092) 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f860effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8610102480 0x7f8610074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59470/0 (socket says 172.21.15.5:59470) 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f860f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8610101ae0 0x7f86100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36140/0 (socket says 172.21.15.5:36140) 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f860ffff640 1 -- 172.21.15.5:0/2713737749 learned_addr learned my addr 172.21.15.5:0/2713737749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f860ffff640 1 -- 172.21.15.5:0/2713737749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8610102480 msgr2=0x7f8610074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.706+0000 7f860ffff640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8610102480 0x7f8610074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.707+0000 7f860ffff640 1 -- 172.21.15.5:0/2713737749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8610101ae0 msgr2=0x7f86100744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.707+0000 7f860ffff640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8610101ae0 0x7f86100744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.707+0000 7f860ffff640 1 -- 172.21.15.5:0/2713737749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86101a68a0 con 0x7f8610107b80 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.707+0000 7f860f7fe640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8610101ae0 0x7f86100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:58.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.707+0000 7f860ffff640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8610107b80 0x7f8610074f70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f860800efd0 tx=0x7f860800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.707+0000 7f860cff9640 1 -- 172.21.15.5:0/2713737749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608003b70 con 0x7f8610107b80 2024-09-11T22:31:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.707+0000 7f860cff9640 1 -- 172.21.15.5:0/2713737749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608003d10 con 0x7f8610107b80 2024-09-11T22:31:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.707+0000 7f860cff9640 1 -- 172.21.15.5:0/2713737749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86080109f0 con 0x7f8610107b80 2024-09-11T22:31:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.707+0000 7f8615c2e640 1 -- 172.21.15.5:0/2713737749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86101a6b90 con 0x7f8610107b80 2024-09-11T22:31:58.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.707+0000 7f8615c2e640 1 -- 172.21.15.5:0/2713737749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86101a7060 con 0x7f8610107b80 2024-09-11T22:31:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f86189cb740 1 Processor -- start 2024-09-11T22:31:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f860cff9640 1 -- 172.21.15.5:0/2713737749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8608004020 con 0x7f8610107b80 2024-09-11T22:31:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f86189cb740 1 -- start start 2024-09-11T22:31:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f86189cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e68faf0 0x55d94e68fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f86189cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e69a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f86189cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 0x55d94e69d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f86189cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d94e5f42f0 con 0x55d94e690410 2024-09-11T22:31:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f86189cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d94e59c270 con 0x55d94e68faf0 2024-09-11T22:31:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f86189cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d94e5f02b0 con 0x55d94e69acb0 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f860effd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e68faf0 0x55d94e68fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f860f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e69a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f860effd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e68faf0 0x55d94e68fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36152/0 (socket says 172.21.15.5:36152) 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.710+0000 7f860effd640 1 -- 172.21.15.5:0/1991045604 learned_addr learned my addr 172.21.15.5:0/1991045604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f860effd640 1 --2- 172.21.15.5:0/1991045604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 0x55d94e69d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f860f7fe640 1 -- 172.21.15.5:0/1991045604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 msgr2=0x55d94e69d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f860f7fe640 1 --2- 172.21.15.5:0/1991045604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 0x55d94e69d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f860f7fe640 1 -- 172.21.15.5:0/1991045604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e68faf0 msgr2=0x55d94e68fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f860f7fe640 1 --2- 172.21.15.5:0/1991045604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e68faf0 0x55d94e68fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f860f7fe640 1 -- 172.21.15.5:0/1991045604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d94e59b340 con 0x55d94e690410 2024-09-11T22:31:58.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f860cff9640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85f80a2da0 0x7f85f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f860effd640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85f80a2da0 0x7f85f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f860cff9640 1 -- 172.21.15.5:0/2713737749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f86080cf8b0 con 0x7f8610107b80 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f860f7fe640 1 --2- 172.21.15.5:0/1991045604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e69a770 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f860000bf70 tx=0x7f8600010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6247d8132c57db8c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f85f5ffb640 1 -- 172.21.15.5:0/1991045604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860001c680 con 0x55d94e690410 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f85f5ffb640 1 -- 172.21.15.5:0/1991045604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f860001cd00 con 0x55d94e690410 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.711+0000 7f85f5ffb640 1 -- 172.21.15.5:0/1991045604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600011e30 con 0x55d94e690410 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f86189cb740 1 -- 172.21.15.5:0/1991045604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 msgr2=0x55d94e69a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f86189cb740 1 --2- 172.21.15.5:0/1991045604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e69a770 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f860000bf70 tx=0x7f8600010860 comp rx=0 tx=0).stop 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f860effd640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85f80a2da0 0x7f85f80a5260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8604000c00 tx=0x7f860402a420 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f86189cb740 1 -- 172.21.15.5:0/1991045604 shutdown_connections 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f86189cb740 1 --2- 172.21.15.5:0/1991045604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 0x55d94e69d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f86189cb740 1 --2- 172.21.15.5:0/1991045604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e69a770 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f86189cb740 1 --2- 172.21.15.5:0/1991045604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e68faf0 0x55d94e68fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f86189cb740 1 -- 172.21.15.5:0/1991045604 >> 172.21.15.5:0/1991045604 conn(0x55d94e68c060 msgr2=0x55d94e68c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f86189cb740 1 -- 172.21.15.5:0/1991045604 shutdown_connections 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f86189cb740 1 -- 172.21.15.5:0/1991045604 wait complete. 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.712+0000 7f86189cb740 1 -- start start 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f86189cb740 1 Processor -- start 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f86189cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e68faf0 0x55d94e697b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f86189cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e698070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f86189cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 0x55d94e6985b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f86189cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d94e69d880 con 0x55d94e690410 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f86189cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d94e57d290 con 0x55d94e69acb0 2024-09-11T22:31:58.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f86189cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d94e69da00 con 0x55d94e68faf0 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f860effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e698070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f86189cb740 10 client.0 _ll_register_callbacks cb 0x55d94e5ae140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f860effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e698070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39116/0 (socket says 172.21.15.5:39116) 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f860effd640 1 -- 172.21.15.5:0/724556544 learned_addr learned my addr 172.21.15.5:0/724556544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f860f7fe640 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e68faf0 0x55d94e697b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.713+0000 7f860ffff640 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 0x55d94e6985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f860effd640 1 -- 172.21.15.5:0/724556544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e68faf0 msgr2=0x55d94e697b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f860effd640 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e68faf0 0x55d94e697b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f860effd640 1 -- 172.21.15.5:0/724556544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 msgr2=0x55d94e6985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f860effd640 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 0x55d94e6985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f860effd640 1 -- 172.21.15.5:0/724556544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d94e307210 con 0x55d94e690410 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f860ffff640 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 0x55d94e6985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f860effd640 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e698070 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8604005940 tx=0x7f86040058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1399905f3f7b4d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f85eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f85eeffd640 1 -- 172.21.15.5:0/724556544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8604035040 con 0x55d94e690410 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f85eeffd640 1 -- 172.21.15.5:0/724556544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f8604055480 con 0x55d94e690410 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f86189cb740 1 -- 172.21.15.5:0/724556544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f860001cd00 con 0x55d94e690410 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f85eeffd640 1 -- 172.21.15.5:0/724556544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8604055680 con 0x55d94e690410 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f86189cb740 10 client.18438 fetch_fsmap learned FSMap version 42 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.714+0000 7f86189cb740 1 -- 172.21.15.5:0/724556544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d94e3d3620 con 0x55d94e690410 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.715+0000 7f85eeffd640 1 -- 172.21.15.5:0/724556544 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8604055d00 con 0x55d94e690410 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.715+0000 7f85eeffd640 1 -- 172.21.15.5:0/724556544 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f860403ca70 con 0x55d94e690410 2024-09-11T22:31:58.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.715+0000 7f86189cb740 10 client.18438 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:31:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.715+0000 7f86189cb740 10 client.18438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.715+0000 7f86189cb740 20 client.18438 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.715+0000 7f86189cb740 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d94e6aabb0 0x55d94e6acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.715+0000 7f86189cb740 4 client.18438 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.715+0000 7f86189cb740 1 -- 172.21.15.5:0/724556544 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f860001cd00 con 0x55d94e6aabb0 2024-09-11T22:31:58.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.715+0000 7f860f7fe640 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d94e6aabb0 0x55d94e6acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.716+0000 7f860f7fe640 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d94e6aabb0 0x55d94e6acfa0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36291a69306659da server_cookie=e376489ec75dd4c5 in_seq=0 out_seq=0 2024-09-11T22:31:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.716+0000 7f85eeffd640 10 client.18438 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.716+0000 7f85eeffd640 1 -- 172.21.15.5:0/724556544 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f860001cd00 con 0x55d94e6aabb0 2024-09-11T22:31:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.716+0000 7f85eeffd640 10 client.18438 handle_command_reply: tid=0 2024-09-11T22:31:58.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.716+0000 7f86189cb740 1 client.18438 shutdown 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 1 -- 172.21.15.5:0/724556544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 msgr2=0x55d94e698070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e698070 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8604005940 tx=0x7f86040058d0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 1 -- 172.21.15.5:0/724556544 shutdown_connections 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d94e6aabb0 0x55d94e6acfa0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 0x55d94e6985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e690410 0x55d94e698070 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f85eeffd640 0 client.18438 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 1 --2- 172.21.15.5:0/724556544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e68faf0 0x55d94e697b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 1 -- 172.21.15.5:0/724556544 >> 172.21.15.5:0/724556544 conn(0x55d94e68c060 msgr2=0x55d94e691dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 1 -- 172.21.15.5:0/724556544 shutdown_connections 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 1 -- 172.21.15.5:0/724556544 wait complete. 2024-09-11T22:31:58.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.717+0000 7f86189cb740 20 client.18438 trim_cache size 0 max 16384 2024-09-11T22:31:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.732+0000 7f86189cb740 1 Processor -- start 2024-09-11T22:31:58.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.732+0000 7f86189cb740 1 -- start start 2024-09-11T22:31:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.732+0000 7f86189cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f86189cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e690410 0x55d94e6b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f86189cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 0x55d94e6b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f86189cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86101afb40 con 0x55d94e68faf0 2024-09-11T22:31:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f86189cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f86101b1680 con 0x55d94e690410 2024-09-11T22:31:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f86189cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f86101b0210 con 0x55d94e69acb0 2024-09-11T22:31:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39118/0 (socket says 172.21.15.5:39118) 2024-09-11T22:31:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860f7fe640 1 -- 172.21.15.5:0/2789936150 learned_addr learned my addr 172.21.15.5:0/2789936150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:58.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860effd640 1 --2- 172.21.15.5:0/2789936150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e690410 0x55d94e6b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860ffff640 1 --2- 172.21.15.5:0/2789936150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 0x55d94e6b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860f7fe640 1 -- 172.21.15.5:0/2789936150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 msgr2=0x55d94e6b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860f7fe640 1 --2- 172.21.15.5:0/2789936150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 0x55d94e6b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860f7fe640 1 -- 172.21.15.5:0/2789936150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e690410 msgr2=0x55d94e6b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860f7fe640 1 --2- 172.21.15.5:0/2789936150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e690410 0x55d94e6b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860f7fe640 1 -- 172.21.15.5:0/2789936150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d94e690240 con 0x55d94e68faf0 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860ffff640 1 --2- 172.21.15.5:0/2789936150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 0x55d94e6b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f860f7fe640 1 --2- 172.21.15.5:0/2789936150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6983e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8600010d60 tx=0x7f8600010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f6387806136664 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f85f57fa640 1 -- 172.21.15.5:0/2789936150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860002c070 con 0x55d94e68faf0 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f85f57fa640 1 -- 172.21.15.5:0/2789936150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86000050c0 con 0x55d94e68faf0 2024-09-11T22:31:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.733+0000 7f85f57fa640 1 -- 172.21.15.5:0/2789936150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600003ad0 con 0x55d94e68faf0 2024-09-11T22:31:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.734+0000 7f86189cb740 1 -- 172.21.15.5:0/2789936150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 msgr2=0x55d94e6983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.734+0000 7f86189cb740 1 --2- 172.21.15.5:0/2789936150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6983e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f8600010d60 tx=0x7f8600010860 comp rx=0 tx=0).stop 2024-09-11T22:31:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.734+0000 7f86189cb740 1 -- 172.21.15.5:0/2789936150 shutdown_connections 2024-09-11T22:31:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.734+0000 7f86189cb740 1 --2- 172.21.15.5:0/2789936150 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e69acb0 0x55d94e6b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.734+0000 7f86189cb740 1 --2- 172.21.15.5:0/2789936150 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e690410 0x55d94e6b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.734+0000 7f86189cb740 1 --2- 172.21.15.5:0/2789936150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6983e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.734+0000 7f86189cb740 1 -- 172.21.15.5:0/2789936150 >> 172.21.15.5:0/2789936150 conn(0x55d94e68c060 msgr2=0x55d94e699bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 1 -- 172.21.15.5:0/2789936150 shutdown_connections 2024-09-11T22:31:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 1 -- 172.21.15.5:0/2789936150 wait complete. 2024-09-11T22:31:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 1 -- start start 2024-09-11T22:31:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 1 Processor -- start 2024-09-11T22:31:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e690410 0x55d94e694690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 0x55d94e694bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86101b0fb0 con 0x55d94e68faf0 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f86101b08e0 con 0x55d94e69acb0 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f86101adb20 con 0x55d94e690410 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.735+0000 7f86189cb740 10 client.0 _ll_register_callbacks cb 0x7f86040314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6a3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39128/0 (socket says 172.21.15.5:39128) 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e690410 0x55d94e694690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 0x55d94e694bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860f7fe640 1 -- 172.21.15.5:0/2700470542 learned_addr learned my addr 172.21.15.5:0/2700470542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860f7fe640 1 -- 172.21.15.5:0/2700470542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e690410 msgr2=0x55d94e694690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860f7fe640 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e690410 0x55d94e694690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860f7fe640 1 -- 172.21.15.5:0/2700470542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 msgr2=0x55d94e694bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860f7fe640 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 0x55d94e694bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860f7fe640 1 -- 172.21.15.5:0/2700470542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d94e6953d0 con 0x55d94e68faf0 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860ffff640 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 0x55d94e694bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f860f7fe640 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6a3320 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8600005090 tx=0x7f8600003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aafc60c0929bd45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f85eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f85eeffd640 1 -- 172.21.15.5:0/2700470542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600035060 con 0x55d94e68faf0 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f85eeffd640 1 -- 172.21.15.5:0/2700470542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f8600005470 con 0x55d94e68faf0 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f86189cb740 1 -- 172.21.15.5:0/2700470542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86000050c0 con 0x55d94e68faf0 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.736+0000 7f85eeffd640 1 -- 172.21.15.5:0/2700470542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f860002c070 con 0x55d94e68faf0 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f86189cb740 10 client.18444 fetch_fsmap learned FSMap version 42 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f86189cb740 1 -- 172.21.15.5:0/2700470542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d94e715880 con 0x55d94e68faf0 2024-09-11T22:31:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f85eeffd640 1 -- 172.21.15.5:0/2700470542 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86000050c0 con 0x55d94e68faf0 2024-09-11T22:31:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f85eeffd640 1 -- 172.21.15.5:0/2700470542 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f860001c480 con 0x55d94e68faf0 2024-09-11T22:31:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f86189cb740 10 client.18444 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:31:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f86189cb740 10 client.18444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:31:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f86189cb740 20 client.18444 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f86189cb740 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d94e715b60 0x55d94e717f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f86189cb740 4 client.18444 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:31:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f86189cb740 1 -- 172.21.15.5:0/2700470542 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d94e601a20 con 0x55d94e715b60 2024-09-11T22:31:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.737+0000 7f860effd640 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d94e715b60 0x55d94e717f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.738+0000 7f85eeffd640 10 client.18444 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:58.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.738+0000 7f860effd640 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d94e715b60 0x55d94e717f50 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f4062703afcca33 server_cookie=212cdfc38d7a5e57 in_seq=0 out_seq=0 2024-09-11T22:31:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.846+0000 7f85eeffd640 1 -- 172.21.15.5:0/2700470542 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d94e601a20 con 0x55d94e715b60 2024-09-11T22:31:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.846+0000 7f85eeffd640 10 client.18444 handle_command_reply: tid=0 2024-09-11T22:31:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.846+0000 7f86189cb740 1 client.18444 shutdown 2024-09-11T22:31:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 1 -- 172.21.15.5:0/2700470542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 msgr2=0x55d94e6a3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6a3320 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8600005090 tx=0x7f8600003820 comp rx=0 tx=0).stop 2024-09-11T22:31:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 1 -- 172.21.15.5:0/2700470542 shutdown_connections 2024-09-11T22:31:58.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d94e715b60 0x55d94e717f50 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d94e69acb0 0x55d94e694bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d94e690410 0x55d94e694690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 1 --2- 172.21.15.5:0/2700470542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d94e68faf0 0x55d94e6a3320 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 1 -- 172.21.15.5:0/2700470542 >> 172.21.15.5:0/2700470542 conn(0x55d94e68c060 msgr2=0x55d94e68a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f85eeffd640 0 client.18444 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:31:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 1 -- 172.21.15.5:0/2700470542 shutdown_connections 2024-09-11T22:31:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 1 -- 172.21.15.5:0/2700470542 wait complete. 2024-09-11T22:31:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.847+0000 7f86189cb740 20 client.18444 trim_cache size 0 max 16384 2024-09-11T22:31:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:31:58.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:31:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:31:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:31:58.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 -- 172.21.15.5:0/2713737749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85f80a2da0 msgr2=0x7f85f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85f80a2da0 0x7f85f80a5260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8604000c00 tx=0x7f860402a420 comp rx=0 tx=0).stop 2024-09-11T22:31:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 -- 172.21.15.5:0/2713737749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8610107b80 msgr2=0x7f8610074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8610107b80 0x7f8610074f70 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f860800efd0 tx=0x7f860800c6a0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 -- 172.21.15.5:0/2713737749 shutdown_connections 2024-09-11T22:31:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8610107b80 0x7f8610074f70 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8610102480 0x7f8610074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85f80a2da0 0x7f85f80a5260 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 --2- 172.21.15.5:0/2713737749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8610101ae0 0x7f86100744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 -- 172.21.15.5:0/2713737749 >> 172.21.15.5:0/2713737749 conn(0x7f86100fd7e0 msgr2=0x7f86100fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:58.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 -- 172.21.15.5:0/2713737749 shutdown_connections 2024-09-11T22:31:58.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:31:58.848+0000 7f8615c2e640 1 -- 172.21.15.5:0/2713737749 wait complete. 2024-09-11T22:31:58.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:31:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:31:59.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 752909 7.74 MB/sec execute 377 sec latency 102.828 ms 2024-09-11T22:31:59.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:31:59 smithi005 ceph-mon[30943]: pgmap v307: 97 pgs: 97 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 356 op/s 2024-09-11T22:31:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.725+0000 7f459e137640 1 -- 172.21.15.5:0/4034570119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4598102570 msgr2=0x7f45981029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.725+0000 7f459e137640 1 --2- 172.21.15.5:0/4034570119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4598102570 0x7f45981029b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f458c009900 tx=0x7f458c02a480 comp rx=0 tx=0).stop 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.725+0000 7f459e137640 1 -- 172.21.15.5:0/4034570119 shutdown_connections 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.725+0000 7f459e137640 1 --2- 172.21.15.5:0/4034570119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598102ef0 0x7f459810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.725+0000 7f459e137640 1 --2- 172.21.15.5:0/4034570119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4598102570 0x7f45981029b0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.725+0000 7f459e137640 1 --2- 172.21.15.5:0/4034570119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598100e20 0x7f4598101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.725+0000 7f459e137640 1 -- 172.21.15.5:0/4034570119 >> 172.21.15.5:0/4034570119 conn(0x7f45980fc7d0 msgr2=0x7f45980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.725+0000 7f459e137640 1 -- 172.21.15.5:0/4034570119 shutdown_connections 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.725+0000 7f459e137640 1 -- 172.21.15.5:0/4034570119 wait complete. 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f459e137640 1 Processor -- start 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f459e137640 1 -- start start 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f459e137640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598100e20 0x7f45981984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f459e137640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4598102570 0x7f4598198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f459e137640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598102ef0 0x7f459819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f459e137640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45981120d0 con 0x7f4598102570 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f459e137640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4598111f50 con 0x7f4598100e20 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f459e137640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4598112250 con 0x7f4598102ef0 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f4596ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4598102570 0x7f4598198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f45977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598100e20 0x7f45981984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f4596ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4598102570 0x7f4598198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39152/0 (socket says 172.21.15.5:39152) 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f4596ffd640 1 -- 172.21.15.5:0/1375690857 learned_addr learned my addr 172.21.15.5:0/1375690857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.726+0000 7f4597fff640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598102ef0 0x7f459819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f4596ffd640 1 -- 172.21.15.5:0/1375690857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598102ef0 msgr2=0x7f459819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f4596ffd640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598102ef0 0x7f459819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f4596ffd640 1 -- 172.21.15.5:0/1375690857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598100e20 msgr2=0x7f45981984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f4596ffd640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598100e20 0x7f45981984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f4596ffd640 1 -- 172.21.15.5:0/1375690857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459819d480 con 0x7f4598102570 2024-09-11T22:31:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f45977fe640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598100e20 0x7f45981984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f4596ffd640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4598102570 0x7f4598198a10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f458c02aa70 tx=0x7f458c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f4594ff9640 1 -- 172.21.15.5:0/1375690857 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c03f070 con 0x7f4598102570 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f4594ff9640 1 -- 172.21.15.5:0/1375690857 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458c031c30 con 0x7f4598102570 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f4594ff9640 1 -- 172.21.15.5:0/1375690857 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c035030 con 0x7f4598102570 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f459e137640 1 -- 172.21.15.5:0/1375690857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459819d710 con 0x7f4598102570 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.727+0000 7f459e137640 1 -- 172.21.15.5:0/1375690857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45981a5050 con 0x7f4598102570 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.730+0000 7f4594ff9640 1 -- 172.21.15.5:0/1375690857 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f458c042070 con 0x7f4598102570 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.730+0000 7f45a0510740 1 Processor -- start 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.730+0000 7f45a0510740 1 -- start start 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.730+0000 7f45a0510740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf19b40 0x555d5cf18090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.730+0000 7f45a0510740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf18a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.730+0000 7f45a0510740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf23370 0x555d5cf25760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.730+0000 7f45a0510740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d5ce7e2f0 con 0x555d5cf23370 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.730+0000 7f45a0510740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d5ce25cf0 con 0x555d5cf185d0 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.730+0000 7f45a0510740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555d5ce7a2b0 con 0x555d5cf19b40 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4597fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf19b40 0x555d5cf18090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4596ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf23370 0x555d5cf25760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4597fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf19b40 0x555d5cf18090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59534/0 (socket says 172.21.15.5:59534) 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f45977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf18a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4597fff640 1 -- 172.21.15.5:0/1137232024 learned_addr learned my addr 172.21.15.5:0/1137232024 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4597fff640 1 -- 172.21.15.5:0/1137232024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 msgr2=0x555d5cf18a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4597fff640 1 --2- 172.21.15.5:0/1137232024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf18a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4597fff640 1 -- 172.21.15.5:0/1137232024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf23370 msgr2=0x555d5cf25760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4597fff640 1 --2- 172.21.15.5:0/1137232024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf23370 0x555d5cf25760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4597fff640 1 -- 172.21.15.5:0/1137232024 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d5ce0a3a0 con 0x555d5cf19b40 2024-09-11T22:31:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4594ff9640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45800a2cf0 0x7f45800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f45977fe640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45800a2cf0 0x7f45800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4597fff640 1 --2- 172.21.15.5:0/1137232024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf19b40 0x555d5cf18090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4590009fd0 tx=0x7f45900118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8bba097896d50af server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f4594ff9640 1 -- 172.21.15.5:0/1375690857 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f458c0ef710 con 0x7f4598102570 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f457dffb640 1 -- 172.21.15.5:0/1137232024 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590004d20 con 0x555d5cf19b40 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f457dffb640 1 -- 172.21.15.5:0/1137232024 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4590004050 con 0x555d5cf19b40 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.731+0000 7f457dffb640 1 -- 172.21.15.5:0/1137232024 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590005280 con 0x555d5cf19b40 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.732+0000 7f45a0510740 1 -- 172.21.15.5:0/1137232024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf19b40 msgr2=0x555d5cf18090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.732+0000 7f45a0510740 1 --2- 172.21.15.5:0/1137232024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf19b40 0x555d5cf18090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4590009fd0 tx=0x7f45900118d0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.732+0000 7f45977fe640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45800a2cf0 0x7f45800a51b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x555d5cf15ac0 tx=0x7f4588005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.732+0000 7f45a0510740 1 -- 172.21.15.5:0/1137232024 shutdown_connections 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.732+0000 7f45a0510740 1 --2- 172.21.15.5:0/1137232024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf23370 0x555d5cf25760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.732+0000 7f45a0510740 1 --2- 172.21.15.5:0/1137232024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf18a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.732+0000 7f45a0510740 1 --2- 172.21.15.5:0/1137232024 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf19b40 0x555d5cf18090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.732+0000 7f45a0510740 1 -- 172.21.15.5:0/1137232024 >> 172.21.15.5:0/1137232024 conn(0x555d5cee1310 msgr2=0x555d5cf14c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.732+0000 7f45a0510740 1 -- 172.21.15.5:0/1137232024 shutdown_connections 2024-09-11T22:31:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.732+0000 7f45a0510740 1 -- 172.21.15.5:0/1137232024 wait complete. 2024-09-11T22:31:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.733+0000 7f45a0510740 1 -- start start 2024-09-11T22:31:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.733+0000 7f45a0510740 1 Processor -- start 2024-09-11T22:31:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.733+0000 7f45a0510740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf22cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.733+0000 7f45a0510740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf1ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.733+0000 7f45a0510740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf204d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.733+0000 7f45a0510740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d5cf26070 con 0x555d5cf19b40 2024-09-11T22:31:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.733+0000 7f45a0510740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d5ce39060 con 0x555d5cf185d0 2024-09-11T22:31:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.733+0000 7f45a0510740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555d5cf261f0 con 0x555d5cf23370 2024-09-11T22:31:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.733+0000 7f45a0510740 10 client.0 _ll_register_callbacks cb 0x555d5cb8c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f45977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf22cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4597fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf204d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4596ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf1ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4597fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf204d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59542/0 (socket says 172.21.15.5:59542) 2024-09-11T22:31:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4597fff640 1 -- 172.21.15.5:0/845720112 learned_addr learned my addr 172.21.15.5:0/845720112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f45977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf22cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36226/0 (socket says 172.21.15.5:36226) 2024-09-11T22:31:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4597fff640 1 -- 172.21.15.5:0/845720112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 msgr2=0x555d5cf22cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4597fff640 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf22cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4597fff640 1 -- 172.21.15.5:0/845720112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 msgr2=0x555d5cf1ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4597fff640 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf1ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4597fff640 1 -- 172.21.15.5:0/845720112 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d5cbef3c0 con 0x555d5cf23370 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4597fff640 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf204d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f459001b040 tx=0x7f4590003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26f98a49f5e682ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4576ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4576ffd640 1 -- 172.21.15.5:0/845720112 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4590029020 con 0x555d5cf23370 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f45a0510740 1 -- 172.21.15.5:0/845720112 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4590004050 con 0x555d5cf23370 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4576ffd640 1 -- 172.21.15.5:0/845720112 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f4590004400 con 0x555d5cf23370 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f4576ffd640 1 -- 172.21.15.5:0/845720112 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4590034050 con 0x555d5cf23370 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.734+0000 7f45a0510740 10 client.27637 fetch_fsmap learned FSMap version 42 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.735+0000 7f45a0510740 1 -- 172.21.15.5:0/845720112 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555d5cf31a80 con 0x555d5cf23370 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.735+0000 7f4576ffd640 1 -- 172.21.15.5:0/845720112 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4590004050 con 0x555d5cf23370 2024-09-11T22:31:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.735+0000 7f4576ffd640 1 -- 172.21.15.5:0/845720112 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f4590025570 con 0x555d5cf23370 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.735+0000 7f45a0510740 10 client.27637 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.735+0000 7f45a0510740 10 client.27637 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.735+0000 7f45a0510740 20 client.27637 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.735+0000 7f45a0510740 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555d5cf32e90 0x555d5cf35280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.735+0000 7f45a0510740 4 client.27637 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.735+0000 7f45a0510740 1 -- 172.21.15.5:0/845720112 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d5ce8b860 con 0x555d5cf32e90 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.735+0000 7f45977fe640 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555d5cf32e90 0x555d5cf35280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.736+0000 7f4576ffd640 10 client.27637 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.736+0000 7f45977fe640 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555d5cf32e90 0x555d5cf35280 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=642fa699d80863e1 server_cookie=1aa98efc7930717 in_seq=0 out_seq=0 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.736+0000 7f4576ffd640 1 -- 172.21.15.5:0/845720112 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d5ce8b860 con 0x555d5cf32e90 2024-09-11T22:31:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.736+0000 7f4576ffd640 10 client.27637 handle_command_reply: tid=0 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.736+0000 7f45a0510740 1 client.27637 shutdown 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 1 -- 172.21.15.5:0/845720112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 msgr2=0x555d5cf204d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf204d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f459001b040 tx=0x7f4590003ce0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 1 -- 172.21.15.5:0/845720112 shutdown_connections 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf204d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf1ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555d5cf32e90 0x555d5cf35280 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 1 --2- 172.21.15.5:0/845720112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f4576ffd640 0 client.27637 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 1 -- 172.21.15.5:0/845720112 >> 172.21.15.5:0/845720112 conn(0x555d5cee1310 msgr2=0x555d5cf1a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 1 -- 172.21.15.5:0/845720112 shutdown_connections 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 1 -- 172.21.15.5:0/845720112 wait complete. 2024-09-11T22:31:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.737+0000 7f45a0510740 20 client.27637 trim_cache size 0 max 16384 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f45a0510740 1 Processor -- start 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f45a0510740 1 -- start start 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f45a0510740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf185d0 0x555d5cf2f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f45a0510740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf1b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f45a0510740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf23370 0x555d5cf1b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f45a0510740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45981ad960 con 0x555d5cf19b40 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f45a0510740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45981af4a0 con 0x555d5cf23370 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f45a0510740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f45981ae030 con 0x555d5cf185d0 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f4596ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf1b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f4596ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf1b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39180/0 (socket says 172.21.15.5:39180) 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f4596ffd640 1 -- 172.21.15.5:0/2490978958 learned_addr learned my addr 172.21.15.5:0/2490978958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:59.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.750+0000 7f4597fff640 1 --2- 172.21.15.5:0/2490978958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf23370 0x555d5cf1b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.751+0000 7f45977fe640 1 --2- 172.21.15.5:0/2490978958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf185d0 0x555d5cf2f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.751+0000 7f4596ffd640 1 -- 172.21.15.5:0/2490978958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf185d0 msgr2=0x555d5cf2f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.751+0000 7f4596ffd640 1 --2- 172.21.15.5:0/2490978958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf185d0 0x555d5cf2f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.751+0000 7f4596ffd640 1 -- 172.21.15.5:0/2490978958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf23370 msgr2=0x555d5cf1b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.751+0000 7f4596ffd640 1 --2- 172.21.15.5:0/2490978958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf23370 0x555d5cf1b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.751+0000 7f4596ffd640 1 -- 172.21.15.5:0/2490978958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d5cf1e4c0 con 0x555d5cf19b40 2024-09-11T22:31:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.751+0000 7f4596ffd640 1 --2- 172.21.15.5:0/2490978958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf1b120 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f458c006010 tx=0x7f458c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a9c0434e7587c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.751+0000 7f457d7fa640 1 -- 172.21.15.5:0/2490978958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c049030 con 0x555d5cf19b40 2024-09-11T22:31:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.751+0000 7f457d7fa640 1 -- 172.21.15.5:0/2490978958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458c03b460 con 0x555d5cf19b40 2024-09-11T22:31:59.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.751+0000 7f457d7fa640 1 -- 172.21.15.5:0/2490978958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c0bffa0 con 0x555d5cf19b40 2024-09-11T22:31:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 -- 172.21.15.5:0/2490978958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 msgr2=0x555d5cf1b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 --2- 172.21.15.5:0/2490978958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf1b120 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f458c006010 tx=0x7f458c02a480 comp rx=0 tx=0).stop 2024-09-11T22:31:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 -- 172.21.15.5:0/2490978958 shutdown_connections 2024-09-11T22:31:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 --2- 172.21.15.5:0/2490978958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf23370 0x555d5cf1b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 --2- 172.21.15.5:0/2490978958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf1b120 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 --2- 172.21.15.5:0/2490978958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf185d0 0x555d5cf2f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 -- 172.21.15.5:0/2490978958 >> 172.21.15.5:0/2490978958 conn(0x555d5cee1310 msgr2=0x555d5cf21ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:59.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 -- 172.21.15.5:0/2490978958 shutdown_connections 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 -- 172.21.15.5:0/2490978958 wait complete. 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 -- start start 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.752+0000 7f45a0510740 1 Processor -- start 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45a0510740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf2b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45a0510740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf26d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45a0510740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf27240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45a0510740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45981aedd0 con 0x555d5cf19b40 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45a0510740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45981ae700 con 0x555d5cf185d0 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45a0510740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f45981ab940 con 0x555d5cf23370 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45a0510740 10 client.0 _ll_register_callbacks cb 0x7f458c0f5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf2b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f4596ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf26d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f4596ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf26d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39182/0 (socket says 172.21.15.5:39182) 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f4596ffd640 1 -- 172.21.15.5:0/253868766 learned_addr learned my addr 172.21.15.5:0/253868766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f4597fff640 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf27240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45977fe640 1 -- 172.21.15.5:0/253868766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 msgr2=0x555d5cf27240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45977fe640 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf27240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45977fe640 1 -- 172.21.15.5:0/253868766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 msgr2=0x555d5cf26d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45977fe640 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf26d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f45977fe640 1 -- 172.21.15.5:0/253868766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d5cf27830 con 0x555d5cf185d0 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f4597fff640 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf27240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.753+0000 7f4596ffd640 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf26d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f45977fe640 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf2b9d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x555d5cf35980 tx=0x7f4588017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b2646a8ef4e4c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f4576ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f4576ffd640 1 -- 172.21.15.5:0/253868766 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458800fe30 con 0x555d5cf185d0 2024-09-11T22:31:59.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f45a0510740 1 -- 172.21.15.5:0/253868766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555d5ce85d70 con 0x555d5cf185d0 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f4576ffd640 1 -- 172.21.15.5:0/253868766 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f458801d7e0 con 0x555d5cf185d0 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f4576ffd640 1 -- 172.21.15.5:0/253868766 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f458801d9e0 con 0x555d5cf185d0 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f45a0510740 10 client.27737 fetch_fsmap learned FSMap version 42 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f45a0510740 1 -- 172.21.15.5:0/253868766 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555d5cf9d870 con 0x555d5cf185d0 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f4576ffd640 1 -- 172.21.15.5:0/253868766 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555d5ce85d70 con 0x555d5cf185d0 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f4576ffd640 1 -- 172.21.15.5:0/253868766 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f4588033430 con 0x555d5cf185d0 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f45a0510740 10 client.27737 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f45a0510740 10 client.27737 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f45a0510740 20 client.27737 populate_metadata read hostname 'smithi005' 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f45a0510740 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555d5cf9df90 0x555d5cfa03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f45a0510740 4 client.27737 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:31:59.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.754+0000 7f45a0510740 1 -- 172.21.15.5:0/253868766 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d5ce02170 con 0x555d5cf9df90 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f4596ffd640 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555d5cf9df90 0x555d5cfa03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f4596ffd640 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555d5cf9df90 0x555d5cfa03b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8324b48430fe0c38 server_cookie=5d2f3a479a7b16ff in_seq=0 out_seq=0 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f4576ffd640 10 client.27737 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f4576ffd640 1 -- 172.21.15.5:0/253868766 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d5ce02170 con 0x555d5cf9df90 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f4576ffd640 10 client.27737 handle_command_reply: tid=0 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f45a0510740 1 client.27737 shutdown 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f45a0510740 1 -- 172.21.15.5:0/253868766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 msgr2=0x555d5cf2b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f45a0510740 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf2b9d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x555d5cf35980 tx=0x7f4588017000 comp rx=0 tx=0).stop 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f45a0510740 1 -- 172.21.15.5:0/253868766 shutdown_connections 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f45a0510740 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555d5cf23370 0x555d5cf27240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f45a0510740 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d5cf19b40 0x555d5cf26d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f45a0510740 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x555d5cf9df90 0x555d5cfa03b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.755+0000 7f45a0510740 1 --2- 172.21.15.5:0/253868766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d5cf185d0 0x555d5cf2b9d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f4576ffd640 0 client.27737 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:31:59.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f45a0510740 1 -- 172.21.15.5:0/253868766 >> 172.21.15.5:0/253868766 conn(0x555d5cee1310 msgr2=0x555d5cf1e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:59.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f45a0510740 1 -- 172.21.15.5:0/253868766 shutdown_connections 2024-09-11T22:31:59.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f45a0510740 1 -- 172.21.15.5:0/253868766 wait complete. 2024-09-11T22:31:59.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f45a0510740 20 client.27737 trim_cache size 0 max 16384 2024-09-11T22:31:59.762 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:31:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f459e137640 1 -- 172.21.15.5:0/1375690857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45800a2cf0 msgr2=0x7f45800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f459e137640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45800a2cf0 0x7f45800a51b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x555d5cf15ac0 tx=0x7f4588005f90 comp rx=0 tx=0).stop 2024-09-11T22:31:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f459e137640 1 -- 172.21.15.5:0/1375690857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4598102570 msgr2=0x7f4598198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:31:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f459e137640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4598102570 0x7f4598198a10 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f458c02aa70 tx=0x7f458c003c10 comp rx=0 tx=0).stop 2024-09-11T22:31:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f459e137640 1 -- 172.21.15.5:0/1375690857 shutdown_connections 2024-09-11T22:31:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f459e137640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4598102ef0 0x7f459819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f459e137640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4598102570 0x7f4598198a10 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f459e137640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f45800a2cf0 0x7f45800a51b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f459e137640 1 --2- 172.21.15.5:0/1375690857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4598100e20 0x7f45981984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:31:59.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.756+0000 7f459e137640 1 -- 172.21.15.5:0/1375690857 >> 172.21.15.5:0/1375690857 conn(0x7f45980fc7d0 msgr2=0x7f4598106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:31:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.757+0000 7f459e137640 1 -- 172.21.15.5:0/1375690857 shutdown_connections 2024-09-11T22:31:59.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:31:59.757+0000 7f459e137640 1 -- 172.21.15.5:0/1375690857 wait complete. 2024-09-11T22:31:59.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:31:59 smithi159 ceph-mon[35534]: pgmap v307: 97 pgs: 97 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 356 op/s 2024-09-11T22:31:59.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:31:59 smithi172 ceph-mon[35461]: pgmap v307: 97 pgs: 97 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.3 MiB/s wr, 356 op/s 2024-09-11T22:32:00.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 754630 7.74 MB/sec execute 378 sec latency 19.623 ms 2024-09-11T22:32:00.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:32:00.886 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:01.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 756328 7.75 MB/sec execute 379 sec latency 24.154 ms 2024-09-11T22:32:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.562+0000 7f8033d9a640 1 -- 172.21.15.5:0/1471610259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f802c102ef0 msgr2=0x7f802c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.562+0000 7f8033d9a640 1 --2- 172.21.15.5:0/1471610259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f802c102ef0 0x7f802c10da40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8024009930 tx=0x7f802402a4d0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.562+0000 7f8033d9a640 1 -- 172.21.15.5:0/1471610259 shutdown_connections 2024-09-11T22:32:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.562+0000 7f8033d9a640 1 --2- 172.21.15.5:0/1471610259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f802c102ef0 0x7f802c10da40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.562+0000 7f8033d9a640 1 --2- 172.21.15.5:0/1471610259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f802c102570 0x7f802c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.562+0000 7f8033d9a640 1 --2- 172.21.15.5:0/1471610259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f802c100e20 0x7f802c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.562+0000 7f8033d9a640 1 -- 172.21.15.5:0/1471610259 >> 172.21.15.5:0/1471610259 conn(0x7f802c0fc7d0 msgr2=0x7f802c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.562+0000 7f8033d9a640 1 -- 172.21.15.5:0/1471610259 shutdown_connections 2024-09-11T22:32:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.562+0000 7f8033d9a640 1 -- 172.21.15.5:0/1471610259 wait complete. 2024-09-11T22:32:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8033d9a640 1 Processor -- start 2024-09-11T22:32:01.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8033d9a640 1 -- start start 2024-09-11T22:32:01.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8033d9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f802c100e20 0x7f802c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8033d9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f802c102570 0x7f802c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8033d9a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f802c102ef0 0x7f802c1a4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8033d9a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f802c110fd0 con 0x7f802c100e20 2024-09-11T22:32:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8033d9a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f802c110e50 con 0x7f802c102ef0 2024-09-11T22:32:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8033d9a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f802c111150 con 0x7f802c102570 2024-09-11T22:32:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8031b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f802c100e20 0x7f802c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8031335640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f802c102570 0x7f802c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8031b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f802c100e20 0x7f802c1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35378/0 (socket says 172.21.15.5:35378) 2024-09-11T22:32:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8031b36640 1 -- 172.21.15.5:0/780780747 learned_addr learned my addr 172.21.15.5:0/780780747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8032337640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f802c102ef0 0x7f802c1a4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8031335640 1 -- 172.21.15.5:0/780780747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f802c102ef0 msgr2=0x7f802c1a4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8031335640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f802c102ef0 0x7f802c1a4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8031335640 1 -- 172.21.15.5:0/780780747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f802c100e20 msgr2=0x7f802c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.563+0000 7f8031335640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f802c100e20 0x7f802c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.564+0000 7f8031335640 1 -- 172.21.15.5:0/780780747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c1a48a0 con 0x7f802c102570 2024-09-11T22:32:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.564+0000 7f8031b36640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f802c100e20 0x7f802c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.564+0000 7f8031335640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f802c102570 0x7f802c199820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f802000e9f0 tx=0x7f802000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.564+0000 7f802affd640 1 -- 172.21.15.5:0/780780747 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802000ccc0 con 0x7f802c102570 2024-09-11T22:32:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.564+0000 7f802affd640 1 -- 172.21.15.5:0/780780747 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8020010910 con 0x7f802c102570 2024-09-11T22:32:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.564+0000 7f802affd640 1 -- 172.21.15.5:0/780780747 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020018960 con 0x7f802c102570 2024-09-11T22:32:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.564+0000 7f8033d9a640 1 -- 172.21.15.5:0/780780747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802c1a4a70 con 0x7f802c102570 2024-09-11T22:32:01.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.564+0000 7f8033d9a640 1 -- 172.21.15.5:0/780780747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802c1a4f60 con 0x7f802c102570 2024-09-11T22:32:01.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.566+0000 7f802affd640 1 -- 172.21.15.5:0/780780747 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8020010ac0 con 0x7f802c102570 2024-09-11T22:32:01.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8036173740 1 Processor -- start 2024-09-11T22:32:01.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8036173740 1 -- start start 2024-09-11T22:32:01.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8036173740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155eb40 0x56294155d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8036173740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155d5d0 0x56294155da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8036173740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 0x56294156a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8036173740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5629414c32f0 con 0x56294155eb40 2024-09-11T22:32:01.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8036173740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56294146acf0 con 0x56294155d5d0 2024-09-11T22:32:01.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8036173740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5629414bf2b0 con 0x562941568370 2024-09-11T22:32:01.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f802affd640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80140a2cf0 0x7f80140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8031b36640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155d5d0 0x56294155da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8031b36640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155d5d0 0x56294155da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33408/0 (socket says 172.21.15.5:33408) 2024-09-11T22:32:01.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8031b36640 1 -- 172.21.15.5:0/2759937719 learned_addr learned my addr 172.21.15.5:0/2759937719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:01.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.567+0000 7f8031335640 1 --2- 172.21.15.5:0/2759937719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 0x56294156a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8032337640 1 --2- 172.21.15.5:0/2759937719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155eb40 0x56294155d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8032337640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80140a2cf0 0x7f80140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8031b36640 1 -- 172.21.15.5:0/2759937719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 msgr2=0x56294156a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:01.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8031b36640 1 --2- 172.21.15.5:0/2759937719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 0x56294156a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8031b36640 1 -- 172.21.15.5:0/2759937719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155eb40 msgr2=0x56294155d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:01.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8031b36640 1 --2- 172.21.15.5:0/2759937719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155eb40 0x56294155d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f802affd640 1 -- 172.21.15.5:0/780780747 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f80200cf810 con 0x7f802c102570 2024-09-11T22:32:01.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8031b36640 1 -- 172.21.15.5:0/2759937719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56294144f3a0 con 0x56294155d5d0 2024-09-11T22:32:01.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8031b36640 1 --2- 172.21.15.5:0/2759937719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155d5d0 0x56294155da10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f801c00f8d0 tx=0x7f801c00fd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=115183e7380fa930 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:01.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8032337640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80140a2cf0 0x7f80140a51b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8024004770 tx=0x7f80240046c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:01.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8013fff640 1 -- 172.21.15.5:0/2759937719 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c011c20 con 0x56294155d5d0 2024-09-11T22:32:01.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8013fff640 1 -- 172.21.15.5:0/2759937719 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f801c011dc0 con 0x56294155d5d0 2024-09-11T22:32:01.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.568+0000 7f8013fff640 1 -- 172.21.15.5:0/2759937719 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c018830 con 0x56294155d5d0 2024-09-11T22:32:01.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.569+0000 7f8036173740 1 -- 172.21.15.5:0/2759937719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155d5d0 msgr2=0x56294155da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:01.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.569+0000 7f8036173740 1 --2- 172.21.15.5:0/2759937719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155d5d0 0x56294155da10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f801c00f8d0 tx=0x7f801c00fd90 comp rx=0 tx=0).stop 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.569+0000 7f8036173740 1 -- 172.21.15.5:0/2759937719 shutdown_connections 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.569+0000 7f8036173740 1 --2- 172.21.15.5:0/2759937719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 0x56294156a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.569+0000 7f8036173740 1 --2- 172.21.15.5:0/2759937719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155d5d0 0x56294155da10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.569+0000 7f8036173740 1 --2- 172.21.15.5:0/2759937719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155eb40 0x56294155d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.569+0000 7f8036173740 1 -- 172.21.15.5:0/2759937719 >> 172.21.15.5:0/2759937719 conn(0x562941526310 msgr2=0x562941559c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.569+0000 7f8036173740 1 -- 172.21.15.5:0/2759937719 shutdown_connections 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.569+0000 7f8036173740 1 -- 172.21.15.5:0/2759937719 wait complete. 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.570+0000 7f8036173740 1 -- start start 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.570+0000 7f8036173740 1 Processor -- start 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.570+0000 7f8036173740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941567cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.570+0000 7f8036173740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155eb40 0x562941564fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.570+0000 7f8036173740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562941568370 0x562941565520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.570+0000 7f8036173740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56294156b070 con 0x56294155d5d0 2024-09-11T22:32:01.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.570+0000 7f8036173740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56294147e060 con 0x562941568370 2024-09-11T22:32:01.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.570+0000 7f8036173740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56294156b1f0 con 0x56294155eb40 2024-09-11T22:32:01.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.570+0000 7f8036173740 10 client.0 _ll_register_callbacks cb 0x5629411d18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:01.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8031b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941567cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8031335640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155eb40 0x562941564fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8032337640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562941568370 0x562941565520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8031b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941567cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35396/0 (socket says 172.21.15.5:35396) 2024-09-11T22:32:01.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8031b36640 1 -- 172.21.15.5:0/244776258 learned_addr learned my addr 172.21.15.5:0/244776258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8031335640 1 -- 172.21.15.5:0/244776258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562941568370 msgr2=0x562941565520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8031335640 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562941568370 0x562941565520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8031335640 1 -- 172.21.15.5:0/244776258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 msgr2=0x562941567cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8031335640 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941567cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8031335640 1 -- 172.21.15.5:0/244776258 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629412343c0 con 0x56294155eb40 2024-09-11T22:32:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8031335640 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155eb40 0x562941564fa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f80200ccec0 tx=0x7f802006bba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf0c5e9d8e0104d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f80117fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:32:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f80117fa640 1 -- 172.21.15.5:0/244776258 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200cd350 con 0x56294155eb40 2024-09-11T22:32:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f80117fa640 1 -- 172.21.15.5:0/244776258 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f802009b390 con 0x56294155eb40 2024-09-11T22:32:01.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8036173740 1 -- 172.21.15.5:0/244776258 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801c011dc0 con 0x56294155eb40 2024-09-11T22:32:01.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f80117fa640 1 -- 172.21.15.5:0/244776258 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f80200a0040 con 0x56294155eb40 2024-09-11T22:32:01.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8036173740 10 client.27655 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:01.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.571+0000 7f8036173740 1 -- 172.21.15.5:0/244776258 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562941576aa0 con 0x56294155eb40 2024-09-11T22:32:01.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f80117fa640 1 -- 172.21.15.5:0/244776258 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f801c011dc0 con 0x56294155eb40 2024-09-11T22:32:01.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f80117fa640 1 -- 172.21.15.5:0/244776258 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f8020097bb0 con 0x56294155eb40 2024-09-11T22:32:01.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f8036173740 10 client.27655 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:01.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f8036173740 10 client.27655 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:32:01.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f8036173740 20 client.27655 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:01.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f8036173740 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562941577f60 0x56294157a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f8036173740 4 client.27655 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:01.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f8036173740 1 -- 172.21.15.5:0/244776258 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629414d0860 con 0x562941577f60 2024-09-11T22:32:01.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f8031b36640 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562941577f60 0x56294157a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f8031b36640 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562941577f60 0x56294157a350 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f39f21fada7c587 server_cookie=cc7059dd16797313 in_seq=0 out_seq=0 2024-09-11T22:32:01.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.572+0000 7f80117fa640 10 client.27655 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:32:01.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f80117fa640 1 -- 172.21.15.5:0/244776258 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629414d0860 con 0x562941577f60 2024-09-11T22:32:01.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f80117fa640 10 client.27655 handle_command_reply: tid=0 2024-09-11T22:32:01.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f8036173740 1 client.27655 shutdown 2024-09-11T22:32:01.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f8036173740 1 -- 172.21.15.5:0/244776258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155eb40 msgr2=0x562941564fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:01.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f8036173740 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155eb40 0x562941564fa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f80200ccec0 tx=0x7f802006bba0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f8036173740 1 -- 172.21.15.5:0/244776258 shutdown_connections 2024-09-11T22:32:01.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f8036173740 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562941577f60 0x56294157a350 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f8036173740 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562941568370 0x562941565520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f8036173740 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155eb40 0x562941564fa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f8036173740 1 --2- 172.21.15.5:0/244776258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941567cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f80117fa640 0 client.27655 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:32:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f8036173740 1 -- 172.21.15.5:0/244776258 >> 172.21.15.5:0/244776258 conn(0x562941526310 msgr2=0x56294155f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.573+0000 7f8036173740 1 -- 172.21.15.5:0/244776258 shutdown_connections 2024-09-11T22:32:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.574+0000 7f8036173740 1 -- 172.21.15.5:0/244776258 wait complete. 2024-09-11T22:32:01.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.574+0000 7f8036173740 20 client.27655 trim_cache size 0 max 16384 2024-09-11T22:32:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8036173740 1 Processor -- start 2024-09-11T22:32:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8036173740 1 -- start start 2024-09-11T22:32:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8036173740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155d5d0 0x562941559e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8036173740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155eb40 0x562941574310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8036173740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562941568370 0x56294157fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8036173740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1ab930 con 0x562941568370 2024-09-11T22:32:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8036173740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1ad470 con 0x56294155eb40 2024-09-11T22:32:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8036173740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1ac000 con 0x56294155d5d0 2024-09-11T22:32:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8032337640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562941568370 0x56294157fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8031b36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155d5d0 0x562941559e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8031335640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155eb40 0x562941574310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8031b36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155d5d0 0x562941559e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46840/0 (socket says 172.21.15.5:46840) 2024-09-11T22:32:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.587+0000 7f8031b36640 1 -- 172.21.15.5:0/3727454717 learned_addr learned my addr 172.21.15.5:0/3727454717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.588+0000 7f8031b36640 1 -- 172.21.15.5:0/3727454717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155eb40 msgr2=0x562941574310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.588+0000 7f8031b36640 1 --2- 172.21.15.5:0/3727454717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155eb40 0x562941574310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.588+0000 7f8031b36640 1 -- 172.21.15.5:0/3727454717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562941568370 msgr2=0x56294157fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.588+0000 7f8031b36640 1 --2- 172.21.15.5:0/3727454717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562941568370 0x56294157fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.588+0000 7f8031b36640 1 -- 172.21.15.5:0/3727454717 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562941568060 con 0x56294155d5d0 2024-09-11T22:32:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.588+0000 7f8031b36640 1 --2- 172.21.15.5:0/3727454717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155d5d0 0x562941559e10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f801c00ffe0 tx=0x7f801c0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b71be4fcb4407126 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.588+0000 7f80137fe640 1 -- 172.21.15.5:0/3727454717 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c003b70 con 0x56294155d5d0 2024-09-11T22:32:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.588+0000 7f80137fe640 1 -- 172.21.15.5:0/3727454717 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f801c003d10 con 0x56294155d5d0 2024-09-11T22:32:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.588+0000 7f80137fe640 1 -- 172.21.15.5:0/3727454717 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c019640 con 0x56294155d5d0 2024-09-11T22:32:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 -- 172.21.15.5:0/3727454717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155d5d0 msgr2=0x562941559e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 --2- 172.21.15.5:0/3727454717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155d5d0 0x562941559e10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f801c00ffe0 tx=0x7f801c0096c0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 -- 172.21.15.5:0/3727454717 shutdown_connections 2024-09-11T22:32:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 --2- 172.21.15.5:0/3727454717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562941568370 0x56294157fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 --2- 172.21.15.5:0/3727454717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155eb40 0x562941574310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 --2- 172.21.15.5:0/3727454717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56294155d5d0 0x562941559e10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 -- 172.21.15.5:0/3727454717 >> 172.21.15.5:0/3727454717 conn(0x562941526310 msgr2=0x562941559700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:01.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 -- 172.21.15.5:0/3727454717 shutdown_connections 2024-09-11T22:32:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 -- 172.21.15.5:0/3727454717 wait complete. 2024-09-11T22:32:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 -- start start 2024-09-11T22:32:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.589+0000 7f8036173740 1 Processor -- start 2024-09-11T22:32:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8036173740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941562140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8036173740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155eb40 0x562941562680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8036173740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 0x562941562bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8036173740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1acda0 con 0x56294155d5d0 2024-09-11T22:32:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8036173740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1ac6d0 con 0x56294155eb40 2024-09-11T22:32:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8036173740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1a9910 con 0x562941568370 2024-09-11T22:32:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8036173740 10 client.0 _ll_register_callbacks cb 0x7f802009b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8031b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941562140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8032337640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 0x562941562bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8031335640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155eb40 0x562941562680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8032337640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 0x562941562bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46856/0 (socket says 172.21.15.5:46856) 2024-09-11T22:32:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8031b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941562140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35418/0 (socket says 172.21.15.5:35418) 2024-09-11T22:32:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8032337640 1 -- 172.21.15.5:0/2780152579 learned_addr learned my addr 172.21.15.5:0/2780152579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8031b36640 1 -- 172.21.15.5:0/2780152579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 msgr2=0x562941562bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8031b36640 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 0x562941562bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8031b36640 1 -- 172.21.15.5:0/2780152579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155eb40 msgr2=0x562941562680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8031b36640 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155eb40 0x562941562680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8031b36640 1 -- 172.21.15.5:0/2780152579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629415e1cb0 con 0x56294155d5d0 2024-09-11T22:32:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.590+0000 7f8032337640 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 0x562941562bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f8031b36640 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941562140 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f801c005040 tx=0x7f801c00f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ebab213cb56572 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f80117fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f80117fa640 1 -- 172.21.15.5:0/2780152579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c030070 con 0x56294155d5d0 2024-09-11T22:32:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f8036173740 1 -- 172.21.15.5:0/2780152579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801c003d10 con 0x56294155d5d0 2024-09-11T22:32:01.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f80117fa640 1 -- 172.21.15.5:0/2780152579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f801c0052d0 con 0x56294155d5d0 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f80117fa640 1 -- 172.21.15.5:0/2780152579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f801c011ca0 con 0x56294155d5d0 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f8036173740 10 client.18480 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f8036173740 1 -- 172.21.15.5:0/2780152579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5629415e2f70 con 0x56294155d5d0 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f80117fa640 1 -- 172.21.15.5:0/2780152579 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f801c003d10 con 0x56294155d5d0 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f80117fa640 1 -- 172.21.15.5:0/2780152579 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f801c019ba0 con 0x56294155d5d0 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f8036173740 10 client.18480 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f8036173740 10 client.18480 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f8036173740 20 client.18480 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f8036173740 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5629415e33f0 0x5629415e57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f8036173740 4 client.18480 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.591+0000 7f8036173740 1 -- 172.21.15.5:0/2780152579 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629414cad70 con 0x5629415e33f0 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8031335640 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5629415e33f0 0x5629415e57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f80117fa640 10 client.18480 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8031335640 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5629415e33f0 0x5629415e57e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d064d9fbd8cc6ae server_cookie=22a7a8adcfc8317f in_seq=0 out_seq=0 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f80117fa640 1 -- 172.21.15.5:0/2780152579 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629414cad70 con 0x5629415e33f0 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f80117fa640 10 client.18480 handle_command_reply: tid=0 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 client.18480 shutdown 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 -- 172.21.15.5:0/2780152579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 msgr2=0x562941562140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941562140 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f801c005040 tx=0x7f801c00f650 comp rx=0 tx=0).stop 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 -- 172.21.15.5:0/2780152579 shutdown_connections 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5629415e33f0 0x5629415e57e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562941568370 0x562941562bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f80117fa640 0 client.18480 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56294155eb40 0x562941562680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 --2- 172.21.15.5:0/2780152579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56294155d5d0 0x562941562140 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 -- 172.21.15.5:0/2780152579 >> 172.21.15.5:0/2780152579 conn(0x562941526310 msgr2=0x5629415584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 -- 172.21.15.5:0/2780152579 shutdown_connections 2024-09-11T22:32:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.592+0000 7f8036173740 1 -- 172.21.15.5:0/2780152579 wait complete. 2024-09-11T22:32:01.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8036173740 20 client.18480 trim_cache size 0 max 16384 2024-09-11T22:32:01.623 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 -- 172.21.15.5:0/780780747 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80140a2cf0 msgr2=0x7f80140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80140a2cf0 0x7f80140a51b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8024004770 tx=0x7f80240046c0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 -- 172.21.15.5:0/780780747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f802c102570 msgr2=0x7f802c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f802c102570 0x7f802c199820 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f802000e9f0 tx=0x7f802000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 -- 172.21.15.5:0/780780747 shutdown_connections 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f80140a2cf0 0x7f80140a51b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f802c102ef0 0x7f802c1a4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f802c102570 0x7f802c199820 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 --2- 172.21.15.5:0/780780747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f802c100e20 0x7f802c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 -- 172.21.15.5:0/780780747 >> 172.21.15.5:0/780780747 conn(0x7f802c0fc7d0 msgr2=0x7f802c1047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:01.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 -- 172.21.15.5:0/780780747 shutdown_connections 2024-09-11T22:32:01.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:01.593+0000 7f8033d9a640 1 -- 172.21.15.5:0/780780747 wait complete. 2024-09-11T22:32:01.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:01 smithi005 ceph-mon[30943]: pgmap v308: 97 pgs: 97 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2024-09-11T22:32:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:01 smithi159 ceph-mon[35534]: pgmap v308: 97 pgs: 97 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2024-09-11T22:32:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:01 smithi172 ceph-mon[35461]: pgmap v308: 97 pgs: 97 active+clean; 108 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 192 op/s 2024-09-11T22:32:02.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 757836 7.75 MB/sec execute 380 sec latency 18.066 ms 2024-09-11T22:32:02.413 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:32:02.690 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.248+0000 7f3ed9f06640 1 -- 172.21.15.5:0/4229198263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4102ef0 msgr2=0x7f3ed410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.248+0000 7f3ed9f06640 1 --2- 172.21.15.5:0/4229198263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4102ef0 0x7f3ed410da40 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009ca0 tx=0x7f3ecc02a950 comp rx=0 tx=0).stop 2024-09-11T22:32:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.248+0000 7f3ed9f06640 1 -- 172.21.15.5:0/4229198263 shutdown_connections 2024-09-11T22:32:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.248+0000 7f3ed9f06640 1 --2- 172.21.15.5:0/4229198263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4102ef0 0x7f3ed410da40 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.248+0000 7f3ed9f06640 1 --2- 172.21.15.5:0/4229198263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ed4102570 0x7f3ed41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.248+0000 7f3ed9f06640 1 --2- 172.21.15.5:0/4229198263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ed4100e20 0x7f3ed4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.248+0000 7f3ed9f06640 1 -- 172.21.15.5:0/4229198263 >> 172.21.15.5:0/4229198263 conn(0x7f3ed40fc7d0 msgr2=0x7f3ed40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.248+0000 7f3ed9f06640 1 -- 172.21.15.5:0/4229198263 shutdown_connections 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.248+0000 7f3ed9f06640 1 -- 172.21.15.5:0/4229198263 wait complete. 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.249+0000 7f3ed9f06640 1 Processor -- start 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.249+0000 7f3ed9f06640 1 -- start start 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed9f06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4100e20 0x7f3ed41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed9f06640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ed4102570 0x7f3ed4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed9f06640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ed4102ef0 0x7f3ed419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed9f06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4110f80 con 0x7f3ed4100e20 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed9f06640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4110e00 con 0x7f3ed4102570 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed9f06640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed4111100 con 0x7f3ed4102ef0 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4100e20 0x7f3ed41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4100e20 0x7f3ed41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35434/0 (socket says 172.21.15.5:35434) 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed37fe640 1 -- 172.21.15.5:0/1303861751 learned_addr learned my addr 172.21.15.5:0/1303861751 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ed4102570 0x7f3ed4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed3fff640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ed4102ef0 0x7f3ed419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1303861751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ed4102ef0 msgr2=0x7f3ed419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ed4102ef0 0x7f3ed419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1303861751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4100e20 msgr2=0x7f3ed41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4100e20 0x7f3ed41984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1303861751 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed419d480 con 0x7f3ed4102570 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4100e20 0x7f3ed41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed3fff640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ed4102ef0 0x7f3ed419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.250+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ed4102570 0x7f3ed4198a10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800e9f0 tx=0x7f3ec800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.251+0000 7f3ed0ff9640 1 -- 172.21.15.5:0/1303861751 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec800ccc0 con 0x7f3ed4102570 2024-09-11T22:32:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.251+0000 7f3ed9f06640 1 -- 172.21.15.5:0/1303861751 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed419d770 con 0x7f3ed4102570 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.251+0000 7f3ed0ff9640 1 -- 172.21.15.5:0/1303861751 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec8005020 con 0x7f3ed4102570 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.251+0000 7f3ed0ff9640 1 -- 172.21.15.5:0/1303861751 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec80049e0 con 0x7f3ed4102570 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.251+0000 7f3ed9f06640 1 -- 172.21.15.5:0/1303861751 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed41a5050 con 0x7f3ed4102570 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3edc2df740 1 Processor -- start 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3edc2df740 1 -- start start 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3ed0ff9640 1 -- 172.21.15.5:0/1303861751 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3ec8005560 con 0x7f3ed4102570 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cf6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 0x55bb01cf6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 0x55bb01d03760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3edc2df740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb01c5c2f0 con 0x55bb01d01370 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3edc2df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bb01c03cf0 con 0x55bb01cf65d0 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3edc2df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bb01c582b0 con 0x55bb01cf7b40 2024-09-11T22:32:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 0x55bb01d03760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.253+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cf6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed37fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 0x55bb01cf6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cf6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46868/0 (socket says 172.21.15.5:46868) 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 0x55bb01d03760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35442/0 (socket says 172.21.15.5:35442) 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed3fff640 1 -- 172.21.15.5:0/1289170879 learned_addr learned my addr 172.21.15.5:0/1289170879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed37fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 0x55bb01cf6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33470/0 (socket says 172.21.15.5:33470) 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed37fe640 1 -- 172.21.15.5:0/1289170879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 msgr2=0x55bb01cf6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/1289170879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cf6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed37fe640 1 -- 172.21.15.5:0/1289170879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 msgr2=0x55bb01d03760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/1289170879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 0x55bb01d03760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed37fe640 1 -- 172.21.15.5:0/1289170879 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb01be83a0 con 0x55bb01cf65d0 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed3fff640 1 --2- 172.21.15.5:0/1289170879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cf6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed0ff9640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ebc0a2cf0 0x7f3ebc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/1289170879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 0x55bb01cf6a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40114f0 tx=0x7f3ec40119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86147d913e2fd3fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed3fff640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ebc0a2cf0 0x7f3ebc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3ed0ff9640 1 -- 172.21.15.5:0/1303861751 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f3ec80cf3e0 con 0x7f3ed4102570 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3eb9ffb640 1 -- 172.21.15.5:0/1289170879 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec401e070 con 0x55bb01cf65d0 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3eb9ffb640 1 -- 172.21.15.5:0/1289170879 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec40035f0 con 0x55bb01cf65d0 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.254+0000 7f3eb9ffb640 1 -- 172.21.15.5:0/1289170879 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec400f820 con 0x55bb01cf65d0 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.255+0000 7f3edc2df740 1 -- 172.21.15.5:0/1289170879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 msgr2=0x55bb01cf6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.255+0000 7f3ed3fff640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ebc0a2cf0 0x7f3ebc0a51b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc02fd80 tx=0x7f3ecc005c50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.255+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1289170879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 0x55bb01cf6a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40114f0 tx=0x7f3ec40119b0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.255+0000 7f3edc2df740 1 -- 172.21.15.5:0/1289170879 shutdown_connections 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.255+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1289170879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 0x55bb01d03760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.255+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1289170879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 0x55bb01cf6a10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.255+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1289170879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cf6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.255+0000 7f3edc2df740 1 -- 172.21.15.5:0/1289170879 >> 172.21.15.5:0/1289170879 conn(0x55bb01cbf310 msgr2=0x55bb01cf2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.255+0000 7f3edc2df740 1 -- 172.21.15.5:0/1289170879 shutdown_connections 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.255+0000 7f3edc2df740 1 -- 172.21.15.5:0/1289170879 wait complete. 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3edc2df740 1 -- start start 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3edc2df740 1 Processor -- start 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 0x55bb01d00d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cfdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 0x55bb01cfe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3edc2df740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb01d04070 con 0x55bb01d01370 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3edc2df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bb01c17060 con 0x55bb01cf65d0 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3edc2df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bb01d041f0 con 0x55bb01cf7b40 2024-09-11T22:32:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3edc2df740 10 client.0 _ll_register_callbacks cb 0x55bb0196a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cfdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 0x55bb01cfe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cfdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46872/0 (socket says 172.21.15.5:46872) 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1485013729 learned_addr learned my addr 172.21.15.5:0/1485013729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 0x55bb01cfe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35452/0 (socket says 172.21.15.5:35452) 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.256+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 0x55bb01d00d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1485013729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 msgr2=0x55bb01d00d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 0x55bb01d00d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1485013729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 msgr2=0x55bb01cfe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 0x55bb01cfe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1485013729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb019cd3c0 con 0x55bb01cf7b40 2024-09-11T22:32:03.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cfdff0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3ec809b5c0 tx=0x7f3ec80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9a0f38585638a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3eb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1485013729 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec801c070 con 0x55bb01cf7b40 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1485013729 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f3ec8004c60 con 0x55bb01cf7b40 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3edc2df740 1 -- 172.21.15.5:0/1485013729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec40035f0 con 0x55bb01cf7b40 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1485013729 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3ec809b220 con 0x55bb01cf7b40 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3edc2df740 10 client.27673 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3edc2df740 1 -- 172.21.15.5:0/1485013729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bb01d0fb40 con 0x55bb01cf7b40 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1485013729 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec40035f0 con 0x55bb01cf7b40 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.257+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1485013729 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f3ec809bb20 con 0x55bb01cf7b40 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.258+0000 7f3edc2df740 10 client.27673 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.258+0000 7f3edc2df740 10 client.27673 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.258+0000 7f3edc2df740 20 client.27673 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.258+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55bb01d10fa0 0x55bb01d13390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.258+0000 7f3edc2df740 4 client.27673 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.258+0000 7f3edc2df740 1 -- 172.21.15.5:0/1485013729 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb01c69860 con 0x55bb01d10fa0 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.258+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55bb01d10fa0 0x55bb01d13390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.258+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55bb01d10fa0 0x55bb01d13390 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e14f32cea7eb284 server_cookie=813884bfc646adf2 in_seq=0 out_seq=0 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.258+0000 7f3eb2ffd640 10 client.27673 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1485013729 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb01c69860 con 0x55bb01d10fa0 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3eb2ffd640 10 client.27673 handle_command_reply: tid=0 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3edc2df740 1 client.27673 shutdown 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3edc2df740 1 -- 172.21.15.5:0/1485013729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 msgr2=0x55bb01cfdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cfdff0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3ec809b5c0 tx=0x7f3ec80046c0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3edc2df740 1 -- 172.21.15.5:0/1485013729 shutdown_connections 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55bb01d10fa0 0x55bb01d13390 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01d01370 0x55bb01cfe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf7b40 0x55bb01cfdff0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1485013729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf65d0 0x55bb01d00d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3edc2df740 1 -- 172.21.15.5:0/1485013729 >> 172.21.15.5:0/1485013729 conn(0x55bb01cbf310 msgr2=0x55bb01cf8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.259+0000 7f3eb2ffd640 0 client.27673 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:32:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.260+0000 7f3edc2df740 1 -- 172.21.15.5:0/1485013729 shutdown_connections 2024-09-11T22:32:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.260+0000 7f3edc2df740 1 -- 172.21.15.5:0/1485013729 wait complete. 2024-09-11T22:32:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.260+0000 7f3edc2df740 20 client.27673 trim_cache size 0 max 16384 2024-09-11T22:32:03.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3edc2df740 1 Processor -- start 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3edc2df740 1 -- start start 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf65d0 0x55bb01d0d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf7b40 0x55bb01cfd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01d01370 0x55bb01d02760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3edc2df740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41ada30 con 0x55bb01cf65d0 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3edc2df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41af570 con 0x55bb01cf7b40 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3edc2df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41ae100 con 0x55bb01d01370 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3ed37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf65d0 0x55bb01d0d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf7b40 0x55bb01cfd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3ed37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf65d0 0x55bb01d0d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35466/0 (socket says 172.21.15.5:35466) 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3ed37fe640 1 -- 172.21.15.5:0/3026851226 learned_addr learned my addr 172.21.15.5:0/3026851226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3ed3fff640 1 --2- 172.21.15.5:0/3026851226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01d01370 0x55bb01d02760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3ed37fe640 1 -- 172.21.15.5:0/3026851226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01d01370 msgr2=0x55bb01d02760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/3026851226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01d01370 0x55bb01d02760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3ed37fe640 1 -- 172.21.15.5:0/3026851226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf7b40 msgr2=0x55bb01cfd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/3026851226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf7b40 0x55bb01cfd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.273+0000 7f3ed37fe640 1 -- 172.21.15.5:0/3026851226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb01d010b0 con 0x55bb01cf65d0 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.274+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/3026851226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf7b40 0x55bb01cfd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.274+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/3026851226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf65d0 0x55bb01d0d310 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4003fd0 tx=0x7f3ec4004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdde0973813f4b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.274+0000 7f3eb97fa640 1 -- 172.21.15.5:0/3026851226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec401e040 con 0x55bb01cf65d0 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.274+0000 7f3eb97fa640 1 -- 172.21.15.5:0/3026851226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec4005510 con 0x55bb01cf65d0 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.274+0000 7f3eb97fa640 1 -- 172.21.15.5:0/3026851226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4003c00 con 0x55bb01cf65d0 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.275+0000 7f3edc2df740 1 -- 172.21.15.5:0/3026851226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf65d0 msgr2=0x55bb01d0d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.275+0000 7f3edc2df740 1 --2- 172.21.15.5:0/3026851226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf65d0 0x55bb01d0d310 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4003fd0 tx=0x7f3ec4004140 comp rx=0 tx=0).stop 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.275+0000 7f3edc2df740 1 -- 172.21.15.5:0/3026851226 shutdown_connections 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.275+0000 7f3edc2df740 1 --2- 172.21.15.5:0/3026851226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01d01370 0x55bb01d02760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.275+0000 7f3edc2df740 1 --2- 172.21.15.5:0/3026851226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01cf7b40 0x55bb01cfd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.275+0000 7f3edc2df740 1 --2- 172.21.15.5:0/3026851226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf65d0 0x55bb01d0d310 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.275+0000 7f3edc2df740 1 -- 172.21.15.5:0/3026851226 >> 172.21.15.5:0/3026851226 conn(0x55bb01cbf310 msgr2=0x55bb01cfc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.275+0000 7f3edc2df740 1 -- 172.21.15.5:0/3026851226 shutdown_connections 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.275+0000 7f3edc2df740 1 -- 172.21.15.5:0/3026851226 wait complete. 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.275+0000 7f3edc2df740 1 -- start start 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3edc2df740 1 Processor -- start 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf65d0 0x55bb01cfb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf7b40 0x55bb01d09470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3edc2df740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01d01370 0x55bb01d099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3edc2df740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41aeea0 con 0x55bb01cf7b40 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3edc2df740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41ae7d0 con 0x55bb01d01370 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3edc2df740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed41aba10 con 0x55bb01cf65d0 2024-09-11T22:32:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3edc2df740 10 client.0 _ll_register_callbacks cb 0x7f3ec800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf7b40 0x55bb01d09470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3ed37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf65d0 0x55bb01cfb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3ed3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01d01370 0x55bb01d099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3ed2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf7b40 0x55bb01d09470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35480/0 (socket says 172.21.15.5:35480) 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1221755167 learned_addr learned my addr 172.21.15.5:0/1221755167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1221755167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf65d0 msgr2=0x55bb01cfb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf65d0 0x55bb01cfb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1221755167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01d01370 msgr2=0x55bb01d099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01d01370 0x55bb01d099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.276+0000 7f3ed2ffd640 1 -- 172.21.15.5:0/1221755167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb01d09ef0 con 0x55bb01cf7b40 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3ed2ffd640 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf7b40 0x55bb01d09470 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800c6a0 tx=0x7f3ec809fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6277e3d1d0720ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3eb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1221755167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec801c040 con 0x55bb01cf7b40 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3edc2df740 1 -- 172.21.15.5:0/1221755167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec4005510 con 0x55bb01cf7b40 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1221755167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8585+0+0 (secure 0 0 0) 0x7f3ec8084e00 con 0x55bb01cf7b40 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1221755167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3ec8081050 con 0x55bb01cf7b40 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3edc2df740 10 client.18498 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3edc2df740 1 -- 172.21.15.5:0/1221755167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bb01d84830 con 0x55bb01cf7b40 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1221755167 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec4005510 con 0x55bb01cf7b40 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1221755167 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f3ec809ad10 con 0x55bb01cf7b40 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3edc2df740 10 client.18498 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3edc2df740 10 client.18498 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3edc2df740 20 client.18498 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.277+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55bb01d84dc0 0x55bb01d871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.278+0000 7f3edc2df740 4 client.18498 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.278+0000 7f3edc2df740 1 -- 172.21.15.5:0/1221755167 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb01c63d70 con 0x55bb01d84dc0 2024-09-11T22:32:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.278+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55bb01d84dc0 0x55bb01d871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.278+0000 7f3eb2ffd640 10 client.18498 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.278+0000 7f3ed37fe640 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55bb01d84dc0 0x55bb01d871e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f227fc07758a7b32 server_cookie=eb245e2394b32418 in_seq=0 out_seq=0 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.278+0000 7f3eb2ffd640 1 -- 172.21.15.5:0/1221755167 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb01c63d70 con 0x55bb01d84dc0 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.278+0000 7f3eb2ffd640 10 client.18498 handle_command_reply: tid=0 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.278+0000 7f3edc2df740 1 client.18498 shutdown 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 1 -- 172.21.15.5:0/1221755167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf7b40 msgr2=0x55bb01d09470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf7b40 0x55bb01d09470 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800c6a0 tx=0x7f3ec809fc70 comp rx=0 tx=0).stop 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 1 -- 172.21.15.5:0/1221755167 shutdown_connections 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55bb01d84dc0 0x55bb01d871e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb01d01370 0x55bb01d099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb01cf7b40 0x55bb01d09470 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 1 --2- 172.21.15.5:0/1221755167 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb01cf65d0 0x55bb01cfb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 1 -- 172.21.15.5:0/1221755167 >> 172.21.15.5:0/1221755167 conn(0x55bb01cbf310 msgr2=0x55bb01d00070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:03.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3eb2ffd640 0 client.18498 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:32:03.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 1 -- 172.21.15.5:0/1221755167 shutdown_connections 2024-09-11T22:32:03.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 1 -- 172.21.15.5:0/1221755167 wait complete. 2024-09-11T22:32:03.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3edc2df740 20 client.18498 trim_cache size 0 max 16384 2024-09-11T22:32:03.286 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3ed9f06640 1 -- 172.21.15.5:0/1303861751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ebc0a2cf0 msgr2=0x7f3ebc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3ed9f06640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ebc0a2cf0 0x7f3ebc0a51b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc02fd80 tx=0x7f3ecc005c50 comp rx=0 tx=0).stop 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.279+0000 7f3ed9f06640 1 -- 172.21.15.5:0/1303861751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ed4102570 msgr2=0x7f3ed4198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.280+0000 7f3ed9f06640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ed4102570 0x7f3ed4198a10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3ec800e9f0 tx=0x7f3ec800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.280+0000 7f3ed9f06640 1 -- 172.21.15.5:0/1303861751 shutdown_connections 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.280+0000 7f3ed9f06640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ed4102ef0 0x7f3ed419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.280+0000 7f3ed9f06640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ebc0a2cf0 0x7f3ebc0a51b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.280+0000 7f3ed9f06640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ed4102570 0x7f3ed4198a10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.280+0000 7f3ed9f06640 1 --2- 172.21.15.5:0/1303861751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ed4100e20 0x7f3ed41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.280+0000 7f3ed9f06640 1 -- 172.21.15.5:0/1303861751 >> 172.21.15.5:0/1303861751 conn(0x7f3ed40fc7d0 msgr2=0x7f3ed406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.280+0000 7f3ed9f06640 1 -- 172.21.15.5:0/1303861751 shutdown_connections 2024-09-11T22:32:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:03.280+0000 7f3ed9f06640 1 -- 172.21.15.5:0/1303861751 wait complete. 2024-09-11T22:32:03.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 759952 7.76 MB/sec execute 381 sec latency 29.739 ms 2024-09-11T22:32:03.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:03 smithi005 ceph-mon[30943]: pgmap v309: 97 pgs: 97 active+clean; 98 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 390 op/s 2024-09-11T22:32:03.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:03 smithi159 ceph-mon[35534]: pgmap v309: 97 pgs: 97 active+clean; 98 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 390 op/s 2024-09-11T22:32:03.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:03 smithi172 ceph-mon[35461]: pgmap v309: 97 pgs: 97 active+clean; 98 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.3 MiB/s wr, 390 op/s 2024-09-11T22:32:04.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 761396 7.76 MB/sec execute 382 sec latency 35.537 ms 2024-09-11T22:32:04.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:32:05.055 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:05.336 INFO:tasks.workunit.client.0.smithi005.stdout: 1 762584 7.75 MB/sec execute 383 sec latency 78.061 ms 2024-09-11T22:32:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.700+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2114414958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c41017e0 msgr2=0x7fd2c4101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.700+0000 7fd2c92b3640 1 --2- 172.21.15.5:0/2114414958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c41017e0 0x7fd2c4101bc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009900 tx=0x7fd2b402a410 comp rx=0 tx=0).stop 2024-09-11T22:32:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.700+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2114414958 shutdown_connections 2024-09-11T22:32:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.700+0000 7fd2c92b3640 1 --2- 172.21.15.5:0/2114414958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c4105e00 0x7fd2c410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.700+0000 7fd2c92b3640 1 --2- 172.21.15.5:0/2114414958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c4102190 0x7fd2c41056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.700+0000 7fd2c92b3640 1 --2- 172.21.15.5:0/2114414958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c41017e0 0x7fd2c4101bc0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.700+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2114414958 >> 172.21.15.5:0/2114414958 conn(0x7fd2c40fba90 msgr2=0x7fd2c40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:05.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.700+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2114414958 shutdown_connections 2024-09-11T22:32:05.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.700+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2114414958 wait complete. 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.701+0000 7fd2c92b3640 1 Processor -- start 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c92b3640 1 -- start start 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c92b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c41017e0 0x7fd2c419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c92b3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c4102190 0x7fd2c419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c92b3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c4105e00 0x7fd2c410edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c92b3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4111f80 con 0x7fd2c41017e0 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c92b3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4111e00 con 0x7fd2c4105e00 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c92b3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4112100 con 0x7fd2c4102190 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c4102190 0x7fd2c419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c41017e0 0x7fd2c419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c4102190 0x7fd2c419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46910/0 (socket says 172.21.15.5:46910) 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c27fc640 1 -- 172.21.15.5:0/2179938878 learned_addr learned my addr 172.21.15.5:0/2179938878 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.702+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c4105e00 0x7fd2c410edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2c2ffd640 1 -- 172.21.15.5:0/2179938878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c4102190 msgr2=0x7fd2c419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2c2ffd640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c4102190 0x7fd2c419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2c2ffd640 1 -- 172.21.15.5:0/2179938878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c4105e00 msgr2=0x7fd2c410edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2c2ffd640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c4105e00 0x7fd2c410edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2c2ffd640 1 -- 172.21.15.5:0/2179938878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c410f6b0 con 0x7fd2c41017e0 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2c2ffd640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c41017e0 0x7fd2c419f100 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4005940 tx=0x7fd2b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2abfff640 1 -- 172.21.15.5:0/2179938878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b403f030 con 0x7fd2c41017e0 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2abfff640 1 -- 172.21.15.5:0/2179938878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2b4003ad0 con 0x7fd2c41017e0 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2abfff640 1 -- 172.21.15.5:0/2179938878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4035070 con 0x7fd2c41017e0 2024-09-11T22:32:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2179938878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c410f940 con 0x7fd2c41017e0 2024-09-11T22:32:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.703+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2179938878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c419abb0 con 0x7fd2c41017e0 2024-09-11T22:32:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.707+0000 7fd2cb68c740 1 Processor -- start 2024-09-11T22:32:05.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.707+0000 7fd2cb68c740 1 -- start start 2024-09-11T22:32:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.707+0000 7fd2abfff640 1 -- 172.21.15.5:0/2179938878 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd2b4042030 con 0x7fd2c41017e0 2024-09-11T22:32:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679fb40 0x564e1679e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e1679ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e167a9370 0x564e167ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2cb68c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e167042f0 con 0x564e167a9370 2024-09-11T22:32:05.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2cb68c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e166abcf0 con 0x564e1679e5d0 2024-09-11T22:32:05.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2cb68c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564e167002b0 con 0x564e1679fb40 2024-09-11T22:32:05.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679fb40 0x564e1679e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679fb40 0x564e1679e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46912/0 (socket says 172.21.15.5:46912) 2024-09-11T22:32:05.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c37fe640 1 -- 172.21.15.5:0/1941823593 learned_addr learned my addr 172.21.15.5:0/1941823593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:05.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c2ffd640 1 --2- 172.21.15.5:0/1941823593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e1679ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c27fc640 1 --2- 172.21.15.5:0/1941823593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e167a9370 0x564e167ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c37fe640 1 -- 172.21.15.5:0/1941823593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 msgr2=0x564e1679ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/1941823593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e1679ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c37fe640 1 -- 172.21.15.5:0/1941823593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e167a9370 msgr2=0x564e167ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/1941823593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e167a9370 0x564e167ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c37fe640 1 -- 172.21.15.5:0/1941823593 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e166903a0 con 0x564e1679fb40 2024-09-11T22:32:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.708+0000 7fd2c27fc640 1 --2- 172.21.15.5:0/1941823593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e167a9370 0x564e167ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.709+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/1941823593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679fb40 0x564e1679e090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009fd0 tx=0x7fd2bc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=202d7a001293e84e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.709+0000 7fd2abfff640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2ac0a2d70 0x7fd2ac0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.709+0000 7fd2a97fa640 1 -- 172.21.15.5:0/1941823593 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc004d20 con 0x564e1679fb40 2024-09-11T22:32:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.709+0000 7fd2a97fa640 1 -- 172.21.15.5:0/1941823593 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc004050 con 0x564e1679fb40 2024-09-11T22:32:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.709+0000 7fd2abfff640 1 -- 172.21.15.5:0/2179938878 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7fd2b40efad0 con 0x7fd2c41017e0 2024-09-11T22:32:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2a97fa640 1 -- 172.21.15.5:0/1941823593 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc005260 con 0x564e1679fb40 2024-09-11T22:32:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2c2ffd640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2ac0a2d70 0x7fd2ac0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2cb68c740 1 -- 172.21.15.5:0/1941823593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679fb40 msgr2=0x564e1679e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/1941823593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679fb40 0x564e1679e090 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009fd0 tx=0x7fd2bc0118d0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2cb68c740 1 -- 172.21.15.5:0/1941823593 shutdown_connections 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/1941823593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e167a9370 0x564e167ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/1941823593 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e1679ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/1941823593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679fb40 0x564e1679e090 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2cb68c740 1 -- 172.21.15.5:0/1941823593 >> 172.21.15.5:0/1941823593 conn(0x564e16767310 msgr2=0x564e1679ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2cb68c740 1 -- 172.21.15.5:0/1941823593 shutdown_connections 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2c2ffd640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2ac0a2d70 0x7fd2ac0a5230 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x564e1679bac0 tx=0x7fd2b40f3580 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.710+0000 7fd2cb68c740 1 -- 172.21.15.5:0/1941823593 wait complete. 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.711+0000 7fd2cb68c740 1 -- start start 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.711+0000 7fd2cb68c740 1 Processor -- start 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679e5d0 0x564e167a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e167a9370 0x564e167a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2cb68c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e167ac110 con 0x564e1679fb40 2024-09-11T22:32:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2cb68c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e166bf060 con 0x564e167a9370 2024-09-11T22:32:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2cb68c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564e167ac290 con 0x564e1679e5d0 2024-09-11T22:32:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2cb68c740 10 client.0 _ll_register_callbacks cb 0x564e164128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679e5d0 0x564e167a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679e5d0 0x564e167a8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46926/0 (socket says 172.21.15.5:46926) 2024-09-11T22:32:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2c37fe640 1 -- 172.21.15.5:0/2889320739 learned_addr learned my addr 172.21.15.5:0/2889320739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e167a9370 0x564e167a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2c37fe640 1 -- 172.21.15.5:0/2889320739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e167a9370 msgr2=0x564e167a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e167a9370 0x564e167a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2c37fe640 1 -- 172.21.15.5:0/2889320739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 msgr2=0x564e167a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.712+0000 7fd2c37fe640 1 -- 172.21.15.5:0/2889320739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e164753c0 con 0x564e1679e5d0 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.713+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679e5d0 0x564e167a8c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009de0 tx=0x7fd2bc009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=401bcd70c22b3c60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.713+0000 7fd2a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.713+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2889320739 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00f4e0 con 0x564e1679e5d0 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.713+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2889320739 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7fd2bc00f680 con 0x564e1679e5d0 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.713+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2889320739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2bc004050 con 0x564e1679e5d0 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.713+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2889320739 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd2bc036050 con 0x564e1679e5d0 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.713+0000 7fd2cb68c740 10 client.27691 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.713+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2889320739 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564e167b7b50 con 0x564e1679e5d0 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.714+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2889320739 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc004050 con 0x564e1679e5d0 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.714+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2889320739 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fd2bc025040 con 0x564e1679e5d0 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.714+0000 7fd2cb68c740 10 client.27691 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.714+0000 7fd2cb68c740 10 client.27691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:05.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.714+0000 7fd2cb68c740 20 client.27691 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:05.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.714+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564e167b8f50 0x564e167bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.714+0000 7fd2cb68c740 4 client.27691 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:05.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.714+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2889320739 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e16711860 con 0x564e167b8f50 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.715+0000 7fd2c27fc640 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564e167b8f50 0x564e167bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.716+0000 7fd2a6ff5640 10 client.27691 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.716+0000 7fd2c27fc640 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564e167b8f50 0x564e167bb340 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba381c548aba3a21 server_cookie=3ea76581c55831b in_seq=0 out_seq=0 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2889320739 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e16711860 con 0x564e167b8f50 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2a6ff5640 10 client.27691 handle_command_reply: tid=0 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 client.27691 shutdown 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2889320739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679e5d0 msgr2=0x564e167a8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679e5d0 0x564e167a8c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009de0 tx=0x7fd2bc009be0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2889320739 shutdown_connections 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564e167b8f50 0x564e167bb340 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2a6ff5640 0 client.27691 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e167a9370 0x564e167a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2889320739 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e1679e5d0 0x564e167a8c40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2889320739 >> 172.21.15.5:0/2889320739 conn(0x564e16767310 msgr2=0x564e167a0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2889320739 shutdown_connections 2024-09-11T22:32:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2889320739 wait complete. 2024-09-11T22:32:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.718+0000 7fd2cb68c740 20 client.27691 trim_cache size 0 max 16384 2024-09-11T22:32:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.734+0000 7fd2cb68c740 1 Processor -- start 2024-09-11T22:32:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.734+0000 7fd2cb68c740 1 -- start start 2024-09-11T22:32:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e167b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e167a9370 0x564e167c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2cb68c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41bc9e0 con 0x564e1679fb40 2024-09-11T22:32:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2cb68c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41be520 con 0x564e1679e5d0 2024-09-11T22:32:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2cb68c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41bd0b0 con 0x564e167a9370 2024-09-11T22:32:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35526/0 (socket says 172.21.15.5:35526) 2024-09-11T22:32:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c27fc640 1 -- 172.21.15.5:0/823032938 learned_addr learned my addr 172.21.15.5:0/823032938 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/823032938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e167b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/823032938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e167a9370 0x564e167c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c27fc640 1 -- 172.21.15.5:0/823032938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e167a9370 msgr2=0x564e167c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c27fc640 1 --2- 172.21.15.5:0/823032938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e167a9370 0x564e167c1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c27fc640 1 -- 172.21.15.5:0/823032938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 msgr2=0x564e167b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c27fc640 1 --2- 172.21.15.5:0/823032938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e167b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c27fc640 1 -- 172.21.15.5:0/823032938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e16717e70 con 0x564e1679fb40 2024-09-11T22:32:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/823032938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e167b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.735+0000 7fd2c27fc640 1 --2- 172.21.15.5:0/823032938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167c0b00 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8004810 tx=0x7fd2b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d140a64111c6b8eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.736+0000 7fd2a8ff9640 1 -- 172.21.15.5:0/823032938 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8017e40 con 0x564e1679fb40 2024-09-11T22:32:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.736+0000 7fd2a8ff9640 1 -- 172.21.15.5:0/823032938 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2b8021030 con 0x564e1679fb40 2024-09-11T22:32:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.736+0000 7fd2a8ff9640 1 -- 172.21.15.5:0/823032938 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b801d5b0 con 0x564e1679fb40 2024-09-11T22:32:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.736+0000 7fd2cb68c740 1 -- 172.21.15.5:0/823032938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 msgr2=0x564e167c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.736+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/823032938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167c0b00 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8004810 tx=0x7fd2b8008040 comp rx=0 tx=0).stop 2024-09-11T22:32:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.737+0000 7fd2cb68c740 1 -- 172.21.15.5:0/823032938 shutdown_connections 2024-09-11T22:32:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.737+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/823032938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e167a9370 0x564e167c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.737+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/823032938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167c0b00 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.737+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/823032938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e167b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.737+0000 7fd2cb68c740 1 -- 172.21.15.5:0/823032938 >> 172.21.15.5:0/823032938 conn(0x564e16767310 msgr2=0x564e167a4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:05.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.737+0000 7fd2cb68c740 1 -- 172.21.15.5:0/823032938 shutdown_connections 2024-09-11T22:32:05.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.737+0000 7fd2cb68c740 1 -- 172.21.15.5:0/823032938 wait complete. 2024-09-11T22:32:05.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.737+0000 7fd2cb68c740 1 -- start start 2024-09-11T22:32:05.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2cb68c740 1 Processor -- start 2024-09-11T22:32:05.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e167b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167b1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35536/0 (socket says 172.21.15.5:35536) 2024-09-11T22:32:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2cb68c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e167a9370 0x564e167aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e167b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2c27fc640 1 -- 172.21.15.5:0/2898692277 learned_addr learned my addr 172.21.15.5:0/2898692277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41bde50 con 0x564e1679fb40 2024-09-11T22:32:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41bd780 con 0x564e1679e5d0 2024-09-11T22:32:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41ba9e0 con 0x564e167a9370 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.738+0000 7fd2cb68c740 10 client.0 _ll_register_callbacks cb 0x7fd2b40f3450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e167a9370 0x564e167aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2c37fe640 1 -- 172.21.15.5:0/2898692277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e167a9370 msgr2=0x564e167aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e167a9370 0x564e167aced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2c37fe640 1 -- 172.21.15.5:0/2898692277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 msgr2=0x564e167b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2c37fe640 1 -- 172.21.15.5:0/2898692277 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e167ad700 con 0x564e1679e5d0 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2c27fc640 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2c37fe640 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e167b1950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00a010 tx=0x7fd2bc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6764864353c4caaa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2898692277 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc042020 con 0x564e1679e5d0 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2898692277 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7fd2bc028480 con 0x564e1679e5d0 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.739+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2b8021030 con 0x564e1679e5d0 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2898692277 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd2bc04a050 con 0x564e1679e5d0 2024-09-11T22:32:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2cb68c740 10 client.27788 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564e16823720 con 0x564e1679e5d0 2024-09-11T22:32:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2898692277 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2b8021030 con 0x564e1679e5d0 2024-09-11T22:32:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2898692277 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fd2bc00f4e0 con 0x564e1679e5d0 2024-09-11T22:32:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2cb68c740 10 client.27788 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2cb68c740 10 client.27788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2cb68c740 20 client.27788 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564e16823d20 0x564e16826140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2cb68c740 4 client.27788 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:32:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.740+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e1670bd70 con 0x564e16823d20 2024-09-11T22:32:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.741+0000 7fd2c27fc640 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564e16823d20 0x564e16826140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2a6ff5640 10 client.27788 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2c27fc640 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564e16823d20 0x564e16826140 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=438edbbe21bf0ad1 server_cookie=257abe44a4d8b70b in_seq=0 out_seq=0 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2a6ff5640 1 -- 172.21.15.5:0/2898692277 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e1670bd70 con 0x564e16823d20 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2a6ff5640 10 client.27788 handle_command_reply: tid=0 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2cb68c740 1 client.27788 shutdown 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 msgr2=0x564e167b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e167b1950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00a010 tx=0x7fd2bc005fd0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 shutdown_connections 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564e16823d20 0x564e16826140 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564e167a9370 0x564e167aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2a6ff5640 0 client.27788 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e1679fb40 0x564e167b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2cb68c740 1 --2- 172.21.15.5:0/2898692277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e1679e5d0 0x564e167b1950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.742+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 >> 172.21.15.5:0/2898692277 conn(0x564e16767310 msgr2=0x564e167a46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.743+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 shutdown_connections 2024-09-11T22:32:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.743+0000 7fd2cb68c740 1 -- 172.21.15.5:0/2898692277 wait complete. 2024-09-11T22:32:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.743+0000 7fd2cb68c740 20 client.27788 trim_cache size 0 max 16384 2024-09-11T22:32:05.750 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:32:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.743+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2179938878 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2ac0a2d70 msgr2=0x7fd2ac0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.743+0000 7fd2c92b3640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2ac0a2d70 0x7fd2ac0a5230 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x564e1679bac0 tx=0x7fd2b40f3580 comp rx=0 tx=0).stop 2024-09-11T22:32:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.743+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2179938878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c41017e0 msgr2=0x7fd2c419f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.743+0000 7fd2c92b3640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c41017e0 0x7fd2c419f100 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4005940 tx=0x7fd2b4004270 comp rx=0 tx=0).stop 2024-09-11T22:32:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.743+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2179938878 shutdown_connections 2024-09-11T22:32:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.744+0000 7fd2c92b3640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2ac0a2d70 0x7fd2ac0a5230 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.744+0000 7fd2c92b3640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2c4105e00 0x7fd2c410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.744+0000 7fd2c92b3640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2c4102190 0x7fd2c419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.744+0000 7fd2c92b3640 1 --2- 172.21.15.5:0/2179938878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c41017e0 0x7fd2c419f100 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.744+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2179938878 >> 172.21.15.5:0/2179938878 conn(0x7fd2c40fba90 msgr2=0x7fd2c40fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.744+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2179938878 shutdown_connections 2024-09-11T22:32:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:05.744+0000 7fd2c92b3640 1 -- 172.21.15.5:0/2179938878 wait complete. 2024-09-11T22:32:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:05 smithi005 ceph-mon[30943]: pgmap v310: 97 pgs: 97 active+clean; 98 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 295 op/s 2024-09-11T22:32:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:05 smithi005 ceph-mon[30943]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T22:32:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:05 smithi159 ceph-mon[35534]: pgmap v310: 97 pgs: 97 active+clean; 98 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 295 op/s 2024-09-11T22:32:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:05 smithi159 ceph-mon[35534]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T22:32:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:05 smithi172 ceph-mon[35461]: pgmap v310: 97 pgs: 97 active+clean; 98 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 295 op/s 2024-09-11T22:32:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:05 smithi172 ceph-mon[35461]: osdmap e97: 12 total, 12 up, 12 in 2024-09-11T22:32:06.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 763657 7.75 MB/sec execute 384 sec latency 31.544 ms 2024-09-11T22:32:07.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 765501 7.75 MB/sec execute 385 sec latency 25.847 ms 2024-09-11T22:32:07.679 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:32:07.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:32:07.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:07 smithi005 ceph-mon[30943]: pgmap v312: 97 pgs: 97 active+clean; 98 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 354 op/s 2024-09-11T22:32:07.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:07 smithi159 ceph-mon[35534]: pgmap v312: 97 pgs: 97 active+clean; 98 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 354 op/s 2024-09-11T22:32:07.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:07 smithi172 ceph-mon[35461]: pgmap v312: 97 pgs: 97 active+clean; 98 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 354 op/s 2024-09-11T22:32:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:08.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 767258 7.76 MB/sec execute 386 sec latency 39.446 ms 2024-09-11T22:32:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.403+0000 7f2e02adb640 1 -- 172.21.15.5:0/92146430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc1036e0 msgr2=0x7f2dfc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.403+0000 7f2e02adb640 1 --2- 172.21.15.5:0/92146430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc1036e0 0x7f2dfc107b90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009cd0 tx=0x7f2dec02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.403+0000 7f2e02adb640 1 -- 172.21.15.5:0/92146430 shutdown_connections 2024-09-11T22:32:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.403+0000 7f2e02adb640 1 --2- 172.21.15.5:0/92146430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dfc1082a0 0x7f2dfc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.403+0000 7f2e02adb640 1 --2- 172.21.15.5:0/92146430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc1036e0 0x7f2dfc107b90 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.403+0000 7f2e02adb640 1 --2- 172.21.15.5:0/92146430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dfc102d30 0x7f2dfc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.403+0000 7f2e02adb640 1 -- 172.21.15.5:0/92146430 >> 172.21.15.5:0/92146430 conn(0x7f2dfc0fc7d0 msgr2=0x7f2dfc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.403+0000 7f2e02adb640 1 -- 172.21.15.5:0/92146430 shutdown_connections 2024-09-11T22:32:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.403+0000 7f2e02adb640 1 -- 172.21.15.5:0/92146430 wait complete. 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.403+0000 7f2e02adb640 1 Processor -- start 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e02adb640 1 -- start start 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e02adb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dfc102d30 0x7f2dfc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e02adb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dfc1036e0 0x7f2dfc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e02adb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc1082a0 0x7f2dfc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e02adb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1120a0 con 0x7f2dfc1082a0 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e02adb640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc111f20 con 0x7f2dfc102d30 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e02adb640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc112220 con 0x7f2dfc1036e0 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e01078640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc1082a0 0x7f2dfc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e00877640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dfc102d30 0x7f2dfc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e00877640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dfc102d30 0x7f2dfc100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33564/0 (socket says 172.21.15.5:33564) 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e00877640 1 -- 172.21.15.5:0/324797172 learned_addr learned my addr 172.21.15.5:0/324797172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dfc1036e0 0x7f2dfc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e00877640 1 -- 172.21.15.5:0/324797172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dfc1036e0 msgr2=0x7f2dfc100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e00877640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dfc1036e0 0x7f2dfc100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e00877640 1 -- 172.21.15.5:0/324797172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc1082a0 msgr2=0x7f2dfc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e00877640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc1082a0 0x7f2dfc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e00877640 1 -- 172.21.15.5:0/324797172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc1a6b20 con 0x7f2dfc102d30 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.404+0000 7f2e01078640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc1082a0 0x7f2dfc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.405+0000 7f2e00877640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dfc102d30 0x7f2dfc100990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2de400e9f0 tx=0x7f2de400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.405+0000 7f2df9ffb640 1 -- 172.21.15.5:0/324797172 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de400cd80 con 0x7f2dfc102d30 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.405+0000 7f2df9ffb640 1 -- 172.21.15.5:0/324797172 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de4005500 con 0x7f2dfc102d30 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.405+0000 7f2e02adb640 1 -- 172.21.15.5:0/324797172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dfc1a6e10 con 0x7f2dfc102d30 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.405+0000 7f2df9ffb640 1 -- 172.21.15.5:0/324797172 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de4010690 con 0x7f2dfc102d30 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.405+0000 7f2e02adb640 1 -- 172.21.15.5:0/324797172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dfc1a7270 con 0x7f2dfc102d30 2024-09-11T22:32:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.408+0000 7f2e04eb4740 1 Processor -- start 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.408+0000 7f2e04eb4740 1 -- start start 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.408+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 0x560a7c920f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.408+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c921480 0x560a7c92b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.408+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c92bd40 0x560a7c92e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.408+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560a7c886280 con 0x560a7c92bd40 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.408+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560a7c82db10 con 0x560a7c920b60 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560a7c882240 con 0x560a7c921480 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2df9ffb640 1 -- 172.21.15.5:0/324797172 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2de40046c0 con 0x7f2dfc102d30 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2e01078640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 0x560a7c920f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2dfbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c92bd40 0x560a7c92e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2dfbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c92bd40 0x560a7c92e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35572/0 (socket says 172.21.15.5:35572) 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2dfbfff640 1 -- 172.21.15.5:0/1130950313 learned_addr learned my addr 172.21.15.5:0/1130950313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2e01078640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 0x560a7c920f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33580/0 (socket says 172.21.15.5:33580) 2024-09-11T22:32:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2e00877640 1 --2- 172.21.15.5:0/1130950313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c921480 0x560a7c92b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2dfbfff640 1 -- 172.21.15.5:0/1130950313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c921480 msgr2=0x560a7c92b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/1130950313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c921480 0x560a7c92b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2dfbfff640 1 -- 172.21.15.5:0/1130950313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 msgr2=0x560a7c920f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/1130950313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 0x560a7c920f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2dfbfff640 1 -- 172.21.15.5:0/1130950313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7c83a1f0 con 0x560a7c92bd40 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2e01078640 1 --2- 172.21.15.5:0/1130950313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 0x560a7c920f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/1130950313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c92bd40 0x560a7c92e130 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009ca0 tx=0x7f2dec009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370b447cad4e7ee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.409+0000 7f2df9ffb640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2de80a2d70 0x7f2de80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2df2ffd640 1 -- 172.21.15.5:0/1130950313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec048070 con 0x560a7c92bd40 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2e01078640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2de80a2d70 0x7f2de80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2df9ffb640 1 -- 172.21.15.5:0/324797172 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7f2de40cf840 con 0x7f2dfc102d30 2024-09-11T22:32:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2df2ffd640 1 -- 172.21.15.5:0/1130950313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec036070 con 0x560a7c92bd40 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2df2ffd640 1 -- 172.21.15.5:0/1130950313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec004380 con 0x560a7c92bd40 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1130950313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c92bd40 msgr2=0x560a7c92e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2e01078640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2de80a2d70 0x7f2de80a5230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009fd0 tx=0x7f2df4005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/1130950313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c92bd40 0x560a7c92e130 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2dec009ca0 tx=0x7f2dec009a20 comp rx=0 tx=0).stop 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1130950313 shutdown_connections 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/1130950313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c92bd40 0x560a7c92e130 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/1130950313 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c921480 0x560a7c92b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/1130950313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 0x560a7c920f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1130950313 >> 172.21.15.5:0/1130950313 conn(0x560a7c91d0f0 msgr2=0x560a7c91d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.410+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1130950313 shutdown_connections 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1130950313 wait complete. 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2e04eb4740 1 -- start start 2024-09-11T22:32:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2e04eb4740 1 Processor -- start 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 0x560a7c928ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c92bd40 0x560a7c929620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560a7c92e910 con 0x560a7c921480 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560a7c840f10 con 0x560a7c920b60 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560a7c92ea90 con 0x560a7c92bd40 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2e04eb4740 10 client.0 _ll_register_callbacks cb 0x560a7c809a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.411+0000 7f2dfbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2dfbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9290e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35580/0 (socket says 172.21.15.5:35580) 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2dfbfff640 1 -- 172.21.15.5:0/1853089553 learned_addr learned my addr 172.21.15.5:0/1853089553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2e00877640 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 0x560a7c928ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2e01078640 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c92bd40 0x560a7c929620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2dfbfff640 1 -- 172.21.15.5:0/1853089553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c92bd40 msgr2=0x560a7c929620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c92bd40 0x560a7c929620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2dfbfff640 1 -- 172.21.15.5:0/1853089553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 msgr2=0x560a7c928ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 0x560a7c928ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2dfbfff640 1 -- 172.21.15.5:0/1853089553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7c5f76a0 con 0x560a7c921480 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9290e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2dec046040 tx=0x7f2dec02b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8223801dc9052b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2ddbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2ddbfff640 1 -- 172.21.15.5:0/1853089553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec048040 con 0x560a7c921480 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2ddbfff640 1 -- 172.21.15.5:0/1853089553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7f2dec003b80 con 0x560a7c921480 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1853089553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dec036070 con 0x560a7c921480 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2ddbfff640 1 -- 172.21.15.5:0/1853089553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2dec05d050 con 0x560a7c921480 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.412+0000 7f2e04eb4740 10 client.18528 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1853089553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560a7c92e910 con 0x560a7c921480 2024-09-11T22:32:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2ddbfff640 1 -- 172.21.15.5:0/1853089553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec036070 con 0x560a7c921480 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2ddbfff640 1 -- 172.21.15.5:0/1853089553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f2dec042030 con 0x560a7c921480 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2e04eb4740 10 client.18528 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2e04eb4740 10 client.18528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2e04eb4740 20 client.18528 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560a7c93b620 0x560a7c93da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2e04eb4740 4 client.18528 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1853089553 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a7c8937f0 con 0x560a7c93b620 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2e00877640 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560a7c93b620 0x560a7c93da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2e00877640 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560a7c93b620 0x560a7c93da10 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f300dd79afaf0ad server_cookie=2ce251b5493beea8 in_seq=0 out_seq=0 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.413+0000 7f2ddbfff640 10 client.18528 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2ddbfff640 1 -- 172.21.15.5:0/1853089553 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a7c8937f0 con 0x560a7c93b620 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2ddbfff640 10 client.18528 handle_command_reply: tid=0 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 client.18528 shutdown 2024-09-11T22:32:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1853089553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 msgr2=0x560a7c9290e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9290e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f2dec046040 tx=0x7f2dec02b020 comp rx=0 tx=0).stop 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1853089553 shutdown_connections 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560a7c93b620 0x560a7c93da10 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c92bd40 0x560a7c929620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9290e0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/1853089553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c920b60 0x560a7c928ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1853089553 >> 172.21.15.5:0/1853089553 conn(0x560a7c91d0f0 msgr2=0x560a7c922dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2ddbfff640 0 client.18528 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1853089553 shutdown_connections 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 1 -- 172.21.15.5:0/1853089553 wait complete. 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.415+0000 7f2e04eb4740 20 client.18528 trim_cache size 0 max 16384 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e04eb4740 1 Processor -- start 2024-09-11T22:32:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e04eb4740 1 -- start start 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c91d0f0 0x560a7c927b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c920b60 0x560a7c9431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c921480 0x560a7c943720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1afc70 con 0x560a7c91d0f0 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1b17b0 con 0x560a7c921480 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1b0340 con 0x560a7c920b60 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e00877640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c91d0f0 0x560a7c927b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e00877640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c91d0f0 0x560a7c927b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35584/0 (socket says 172.21.15.5:35584) 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e00877640 1 -- 172.21.15.5:0/2613880960 learned_addr learned my addr 172.21.15.5:0/2613880960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.433+0000 7f2e01078640 1 --2- 172.21.15.5:0/2613880960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c921480 0x560a7c943720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/2613880960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c920b60 0x560a7c9431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2e00877640 1 -- 172.21.15.5:0/2613880960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c920b60 msgr2=0x560a7c9431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2e00877640 1 --2- 172.21.15.5:0/2613880960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c920b60 0x560a7c9431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2e00877640 1 -- 172.21.15.5:0/2613880960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c921480 msgr2=0x560a7c943720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2e00877640 1 --2- 172.21.15.5:0/2613880960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c921480 0x560a7c943720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2e00877640 1 -- 172.21.15.5:0/2613880960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7c91cf20 con 0x560a7c91d0f0 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2e01078640 1 --2- 172.21.15.5:0/2613880960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c921480 0x560a7c943720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2e00877640 1 --2- 172.21.15.5:0/2613880960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c91d0f0 0x560a7c927b00 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2de4004c00 tx=0x7f2de4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94a124496c5b443 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2df27fc640 1 -- 172.21.15.5:0/2613880960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de409c3c0 con 0x560a7c91d0f0 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2df27fc640 1 -- 172.21.15.5:0/2613880960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de409bcc0 con 0x560a7c91d0f0 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.434+0000 7f2df27fc640 1 -- 172.21.15.5:0/2613880960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de40a44a0 con 0x560a7c91d0f0 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 -- 172.21.15.5:0/2613880960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c91d0f0 msgr2=0x560a7c927b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/2613880960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c91d0f0 0x560a7c927b00 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2de4004c00 tx=0x7f2de4004a40 comp rx=0 tx=0).stop 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 -- 172.21.15.5:0/2613880960 shutdown_connections 2024-09-11T22:32:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/2613880960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c921480 0x560a7c943720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/2613880960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c920b60 0x560a7c9431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/2613880960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c91d0f0 0x560a7c927b00 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 -- 172.21.15.5:0/2613880960 >> 172.21.15.5:0/2613880960 conn(0x560a7c91db70 msgr2=0x560a7c922dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 -- 172.21.15.5:0/2613880960 shutdown_connections 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 -- 172.21.15.5:0/2613880960 wait complete. 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 -- start start 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.435+0000 7f2e04eb4740 1 Processor -- start 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c91d0f0 0x560a7c934140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c920b60 0x560a7c930cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e04eb4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1b10e0 con 0x560a7c921480 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1b0a10 con 0x560a7c91d0f0 2024-09-11T22:32:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e04eb4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1adc70 con 0x560a7c920b60 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e04eb4740 10 client.0 _ll_register_callbacks cb 0x7f2dec046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e00877640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c91d0f0 0x560a7c934140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e01078640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e00877640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c91d0f0 0x560a7c934140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33600/0 (socket says 172.21.15.5:33600) 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e00877640 1 -- 172.21.15.5:0/3015517022 learned_addr learned my addr 172.21.15.5:0/3015517022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2dfbfff640 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c920b60 0x560a7c930cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e01078640 1 -- 172.21.15.5:0/3015517022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c920b60 msgr2=0x560a7c930cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e01078640 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c920b60 0x560a7c930cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e01078640 1 -- 172.21.15.5:0/3015517022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c91d0f0 msgr2=0x560a7c934140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e01078640 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c91d0f0 0x560a7c934140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e01078640 1 -- 172.21.15.5:0/3015517022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7c931730 con 0x560a7c921480 2024-09-11T22:32:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2e01078640 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9311f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2df4012fa0 tx=0x7f2df40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba5a129ff5be1cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2ddbfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.436+0000 7f2ddbfff640 1 -- 172.21.15.5:0/3015517022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4004320 con 0x560a7c921480 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2ddbfff640 1 -- 172.21.15.5:0/3015517022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7f2df40035f0 con 0x560a7c921480 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2e04eb4740 1 -- 172.21.15.5:0/3015517022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de409bcc0 con 0x560a7c921480 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2ddbfff640 1 -- 172.21.15.5:0/3015517022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2df40044c0 con 0x560a7c921480 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2e04eb4740 10 client.18534 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2e04eb4740 1 -- 172.21.15.5:0/3015517022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560a7c9a6f70 con 0x560a7c921480 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2ddbfff640 1 -- 172.21.15.5:0/3015517022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de409bcc0 con 0x560a7c921480 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2ddbfff640 1 -- 172.21.15.5:0/3015517022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f2df4031a90 con 0x560a7c921480 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2e04eb4740 10 client.18534 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2e04eb4740 10 client.18534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:08.506 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:32:08.506 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:32:08.506 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "b53494d7-eab6-4104-93e0-1451305f3ffc", 2024-09-11T22:32:08.506 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:32:08.506 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:32:08.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2e04eb4740 20 client.18534 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560a7c92bd40 0x560a7c9a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2e04eb4740 4 client.18534 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2e04eb4740 1 -- 172.21.15.5:0/3015517022 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560a7c88dd00 con 0x560a7c92bd40 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.437+0000 7f2e00877640 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560a7c92bd40 0x560a7c9a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.438+0000 7f2e00877640 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560a7c92bd40 0x560a7c9a97b0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b93cdbec30246310 server_cookie=671642ce30a06b88 in_seq=0 out_seq=0 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.438+0000 7f2ddbfff640 10 client.18534 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2ddbfff640 1 -- 172.21.15.5:0/3015517022 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560a7c88dd00 con 0x560a7c92bd40 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2ddbfff640 10 client.18534 handle_command_reply: tid=0 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 client.18534 shutdown 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 -- 172.21.15.5:0/3015517022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 msgr2=0x560a7c9311f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9311f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2df4012fa0 tx=0x7f2df40046c0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 -- 172.21.15.5:0/3015517022 shutdown_connections 2024-09-11T22:32:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560a7c92bd40 0x560a7c9a97b0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a7c921480 0x560a7c9311f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560a7c920b60 0x560a7c930cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 --2- 172.21.15.5:0/3015517022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560a7c91d0f0 0x560a7c934140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 -- 172.21.15.5:0/3015517022 >> 172.21.15.5:0/3015517022 conn(0x560a7c91db70 msgr2=0x560a7c928ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2ddbfff640 0 client.18534 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 -- 172.21.15.5:0/3015517022 shutdown_connections 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 1 -- 172.21.15.5:0/3015517022 wait complete. 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.440+0000 7f2e04eb4740 20 client.18534 trim_cache size 0 max 16384 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 -- 172.21.15.5:0/324797172 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2de80a2d70 msgr2=0x7f2de80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2de80a2d70 0x7f2de80a5230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009fd0 tx=0x7f2df4005f90 comp rx=0 tx=0).stop 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 -- 172.21.15.5:0/324797172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dfc102d30 msgr2=0x7f2dfc100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dfc102d30 0x7f2dfc100990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2de400e9f0 tx=0x7f2de400edf0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 -- 172.21.15.5:0/324797172 shutdown_connections 2024-09-11T22:32:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2dfc1082a0 0x7f2dfc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dfc1036e0 0x7f2dfc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2de80a2d70 0x7f2de80a5230 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 --2- 172.21.15.5:0/324797172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2dfc102d30 0x7f2dfc100990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 -- 172.21.15.5:0/324797172 >> 172.21.15.5:0/324797172 conn(0x7f2dfc0fc7d0 msgr2=0x7f2dfc105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 -- 172.21.15.5:0/324797172 shutdown_connections 2024-09-11T22:32:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:08.441+0000 7f2e02adb640 1 -- 172.21.15.5:0/324797172 wait complete. 2024-09-11T22:32:09.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 769335 7.76 MB/sec execute 387 sec latency 25.976 ms 2024-09-11T22:32:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:09 smithi005 ceph-mon[30943]: pgmap v313: 97 pgs: 97 active+clean; 101 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 336 op/s 2024-09-11T22:32:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:09 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:32:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:09 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:09 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:32:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:09 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:32:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:09 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:09 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:09.741 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:32:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:09 smithi159 ceph-mon[35534]: pgmap v313: 97 pgs: 97 active+clean; 101 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 336 op/s 2024-09-11T22:32:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:09 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:32:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:09 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:09 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:32:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:09 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:32:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:09 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:09.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:09 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:09 smithi172 ceph-mon[35461]: pgmap v313: 97 pgs: 97 active+clean; 101 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 336 op/s 2024-09-11T22:32:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:09 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:32:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:09 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:09 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:32:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:09 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:32:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:09 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:09 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:10.002 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:10.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 770762 7.77 MB/sec execute 388 sec latency 37.235 ms 2024-09-11T22:32:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.531+0000 7f9c5cc84640 1 -- 172.21.15.5:0/2037620603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c581082a0 msgr2=0x7f9c5810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.531+0000 7f9c5cc84640 1 --2- 172.21.15.5:0/2037620603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c581082a0 0x7f9c5810ec40 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009cd0 tx=0x7f9c5002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.531+0000 7f9c5cc84640 1 -- 172.21.15.5:0/2037620603 shutdown_connections 2024-09-11T22:32:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.531+0000 7f9c5cc84640 1 --2- 172.21.15.5:0/2037620603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c581082a0 0x7f9c5810ec40 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.531+0000 7f9c5cc84640 1 --2- 172.21.15.5:0/2037620603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c581036e0 0x7f9c58107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.531+0000 7f9c5cc84640 1 --2- 172.21.15.5:0/2037620603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58102d30 0x7f9c58103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.531+0000 7f9c5cc84640 1 -- 172.21.15.5:0/2037620603 >> 172.21.15.5:0/2037620603 conn(0x7f9c580fc7d0 msgr2=0x7f9c580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.531+0000 7f9c5cc84640 1 -- 172.21.15.5:0/2037620603 shutdown_connections 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.531+0000 7f9c5cc84640 1 -- 172.21.15.5:0/2037620603 wait complete. 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.532+0000 7f9c5cc84640 1 Processor -- start 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.532+0000 7f9c5cc84640 1 -- start start 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.532+0000 7f9c5cc84640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58102d30 0x7f9c581a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.532+0000 7f9c5cc84640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c581036e0 0x7f9c581a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.532+0000 7f9c5cc84640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c581082a0 0x7f9c5819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.532+0000 7f9c5cc84640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581120a0 con 0x7f9c581082a0 2024-09-11T22:32:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c5cc84640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58111f20 con 0x7f9c58102d30 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c5cc84640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9c58112220 con 0x7f9c581036e0 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c56d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c581082a0 0x7f9c5819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c5659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58102d30 0x7f9c581a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c5659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58102d30 0x7f9c581a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33622/0 (socket says 172.21.15.5:33622) 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c5659c640 1 -- 172.21.15.5:0/3714807945 learned_addr learned my addr 172.21.15.5:0/3714807945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c581036e0 0x7f9c581a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c56d9d640 1 -- 172.21.15.5:0/3714807945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c581036e0 msgr2=0x7f9c581a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c56d9d640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c581036e0 0x7f9c581a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c56d9d640 1 -- 172.21.15.5:0/3714807945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58102d30 msgr2=0x7f9c581a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c56d9d640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58102d30 0x7f9c581a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c56d9d640 1 -- 172.21.15.5:0/3714807945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5819ace0 con 0x7f9c581082a0 2024-09-11T22:32:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c5659c640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58102d30 0x7f9c581a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c581036e0 0x7f9c581a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c56d9d640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c581082a0 0x7f9c5819a5b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9c5002ae00 tx=0x7f9c50031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.533+0000 7f9c477fe640 1 -- 172.21.15.5:0/3714807945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50004040 con 0x7f9c581082a0 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.534+0000 7f9c477fe640 1 -- 172.21.15.5:0/3714807945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c500046c0 con 0x7f9c581082a0 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.534+0000 7f9c477fe640 1 -- 172.21.15.5:0/3714807945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50030e40 con 0x7f9c581082a0 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.534+0000 7f9c5cc84640 1 -- 172.21.15.5:0/3714807945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5819af70 con 0x7f9c581082a0 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.534+0000 7f9c5cc84640 1 -- 172.21.15.5:0/3714807945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c581a72c0 con 0x7f9c581082a0 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.536+0000 7f9c477fe640 1 -- 172.21.15.5:0/3714807945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9c500305a0 con 0x7f9c581082a0 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.536+0000 7f9c5f05d740 1 Processor -- start 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c5f05d740 1 -- start start 2024-09-11T22:32:10.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c7e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c89140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c89680 0x55ae28c8ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae28be42e0 con 0x55ae28c7edc0 2024-09-11T22:32:10.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae28b8bce0 con 0x55ae28c7e4a0 2024-09-11T22:32:10.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ae28be02a0 con 0x55ae28c89680 2024-09-11T22:32:10.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c7e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c7e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33638/0 (socket says 172.21.15.5:33638) 2024-09-11T22:32:10.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c55d9b640 1 -- 172.21.15.5:0/4075427336 learned_addr learned my addr 172.21.15.5:0/4075427336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:10.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c5659c640 1 --2- 172.21.15.5:0/4075427336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c89140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/4075427336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c89680 0x55ae28c8ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.537+0000 7f9c477fe640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9c400a2d70 0x7f9c400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c55d9b640 1 -- 172.21.15.5:0/4075427336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c89680 msgr2=0x55ae28c8ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c56d9d640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9c400a2d70 0x7f9c400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/4075427336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c89680 0x55ae28c8ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c55d9b640 1 -- 172.21.15.5:0/4075427336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 msgr2=0x55ae28c89140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/4075427336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c89140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c55d9b640 1 -- 172.21.15.5:0/4075427336 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae28b70440 con 0x55ae28c7e4a0 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c477fe640 1 -- 172.21.15.5:0/3714807945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7f9c5002e030 con 0x7f9c581082a0 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/4075427336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c7e880 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55ae28c7bdb0 tx=0x7f9c4c0096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c1e4a5580e364a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c56d9d640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9c400a2d70 0x7f9c400a5230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c500ee040 tx=0x7f9c5008bc90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c44ff9640 1 -- 172.21.15.5:0/4075427336 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c012c40 con 0x55ae28c7e4a0 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c44ff9640 1 -- 172.21.15.5:0/4075427336 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c4c012de0 con 0x55ae28c7e4a0 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.538+0000 7f9c44ff9640 1 -- 172.21.15.5:0/4075427336 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c01fc20 con 0x55ae28c7e4a0 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.539+0000 7f9c5f05d740 1 -- 172.21.15.5:0/4075427336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 msgr2=0x55ae28c7e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.539+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/4075427336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c7e880 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55ae28c7bdb0 tx=0x7f9c4c0096d0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.539+0000 7f9c5f05d740 1 -- 172.21.15.5:0/4075427336 shutdown_connections 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.539+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/4075427336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c89680 0x55ae28c8ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.539+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/4075427336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c89140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.539+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/4075427336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c7e880 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.539+0000 7f9c5f05d740 1 -- 172.21.15.5:0/4075427336 >> 172.21.15.5:0/4075427336 conn(0x55ae28c7aa30 msgr2=0x55ae28c7ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.539+0000 7f9c5f05d740 1 -- 172.21.15.5:0/4075427336 shutdown_connections 2024-09-11T22:32:10.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.539+0000 7f9c5f05d740 1 -- 172.21.15.5:0/4075427336 wait complete. 2024-09-11T22:32:10.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5f05d740 1 -- start start 2024-09-11T22:32:10.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5f05d740 1 Processor -- start 2024-09-11T22:32:10.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c89000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c89680 0x55ae28c86830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae28c8c250 con 0x55ae28c7edc0 2024-09-11T22:32:10.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae28b9f050 con 0x55ae28c7e4a0 2024-09-11T22:32:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ae28c8c3d0 con 0x55ae28c89680 2024-09-11T22:32:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5f05d740 10 client.0 _ll_register_callbacks cb 0x55ae28bdf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c89000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c89000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33652/0 (socket says 172.21.15.5:33652) 2024-09-11T22:32:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c862b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35608/0 (socket says 172.21.15.5:35608) 2024-09-11T22:32:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c55d9b640 1 -- 172.21.15.5:0/1294574624 learned_addr learned my addr 172.21.15.5:0/1294574624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.540+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c89680 0x55ae28c86830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.541+0000 7f9c55d9b640 1 -- 172.21.15.5:0/1294574624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c89680 msgr2=0x55ae28c86830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:10.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.541+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c89680 0x55ae28c86830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.541+0000 7f9c55d9b640 1 -- 172.21.15.5:0/1294574624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 msgr2=0x55ae28c862b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.541+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c862b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.541+0000 7f9c55d9b640 1 -- 172.21.15.5:0/1294574624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae289553c0 con 0x55ae28c7e4a0 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.541+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c89000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009fd0 tx=0x7f9c4c014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb0276d65192ead server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.541+0000 7f9c3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.542+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1294574624 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0109b0 con 0x55ae28c7e4a0 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.542+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1294574624 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7f9c4c010b50 con 0x55ae28c7e4a0 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.542+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1294574624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c4c012de0 con 0x55ae28c7e4a0 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.542+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1294574624 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9c4c033050 con 0x55ae28c7e4a0 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.542+0000 7f9c5f05d740 10 client.27818 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.542+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1294574624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ae28c97c40 con 0x55ae28c7e4a0 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.543+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1294574624 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c4c012de0 con 0x55ae28c7e4a0 2024-09-11T22:32:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.543+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1294574624 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f9c4c004a20 con 0x55ae28c7e4a0 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.543+0000 7f9c5f05d740 10 client.27818 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.543+0000 7f9c5f05d740 10 client.27818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.543+0000 7f9c5f05d740 20 client.27818 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.543+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ae28c98f90 0x55ae28c9b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.543+0000 7f9c5f05d740 4 client.27818 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.543+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1294574624 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae28bf1850 con 0x55ae28c98f90 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.543+0000 7f9c5659c640 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ae28c98f90 0x55ae28c9b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.544+0000 7f9c5659c640 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ae28c98f90 0x55ae28c9b380 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73d0bebd4bf3da6 server_cookie=17b2d7e0de2ae400 in_seq=0 out_seq=0 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.544+0000 7f9c3dffb640 10 client.27818 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1294574624 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae28bf1850 con 0x55ae28c98f90 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c3dffb640 10 client.27818 handle_command_reply: tid=0 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 client.27818 shutdown 2024-09-11T22:32:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1294574624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 msgr2=0x55ae28c89000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c89000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009fd0 tx=0x7f9c4c014040 comp rx=0 tx=0).stop 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1294574624 shutdown_connections 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ae28c98f90 0x55ae28c9b380 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c89680 0x55ae28c86830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1294574624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7e4a0 0x55ae28c89000 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c3dffb640 0 client.27818 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1294574624 >> 172.21.15.5:0/1294574624 conn(0x55ae28c7aa30 msgr2=0x55ae28c80760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1294574624 shutdown_connections 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1294574624 wait complete. 2024-09-11T22:32:10.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.545+0000 7f9c5f05d740 20 client.27818 trim_cache size 0 max 16384 2024-09-11T22:32:10.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.564+0000 7f9c5f05d740 1 Processor -- start 2024-09-11T22:32:10.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.564+0000 7f9c5f05d740 1 -- start start 2024-09-11T22:32:10.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 0x55ae28c84b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7edc0 0x55ae28ca0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c89680 0x55ae28ca1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581ab910 con 0x55ae28c89680 2024-09-11T22:32:10.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581ad450 con 0x55ae28c7edc0 2024-09-11T22:32:10.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581abfe0 con 0x55ae28c7e4a0 2024-09-11T22:32:10.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 0x55ae28c84b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c89680 0x55ae28ca1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c5659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7edc0 0x55ae28ca0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 0x55ae28c84b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47034/0 (socket says 172.21.15.5:47034) 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c5659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7edc0 0x55ae28ca0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33666/0 (socket says 172.21.15.5:33666) 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c55d9b640 1 -- 172.21.15.5:0/3985244929 learned_addr learned my addr 172.21.15.5:0/3985244929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c55d9b640 1 -- 172.21.15.5:0/3985244929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7edc0 msgr2=0x55ae28ca0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/3985244929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7edc0 0x55ae28ca0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c55d9b640 1 -- 172.21.15.5:0/3985244929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c89680 msgr2=0x55ae28ca1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/3985244929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c89680 0x55ae28ca1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c55d9b640 1 -- 172.21.15.5:0/3985244929 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae28c85260 con 0x55ae28c7e4a0 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.565+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/3985244929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 0x55ae28c84b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0124f0 tx=0x7f9c4c01f970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28e3fe21fcf38912 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:10.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.566+0000 7f9c3ffff640 1 -- 172.21.15.5:0/3985244929 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c01fd10 con 0x55ae28c7e4a0 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.566+0000 7f9c3ffff640 1 -- 172.21.15.5:0/3985244929 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c4c009a40 con 0x55ae28c7e4a0 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.566+0000 7f9c3ffff640 1 -- 172.21.15.5:0/3985244929 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c012320 con 0x55ae28c7e4a0 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.566+0000 7f9c5f05d740 1 -- 172.21.15.5:0/3985244929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 msgr2=0x55ae28c84b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.566+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/3985244929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 0x55ae28c84b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0124f0 tx=0x7f9c4c01f970 comp rx=0 tx=0).stop 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.566+0000 7f9c5f05d740 1 -- 172.21.15.5:0/3985244929 shutdown_connections 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.566+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/3985244929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c89680 0x55ae28ca1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.566+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/3985244929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c7edc0 0x55ae28ca0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.566+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/3985244929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 0x55ae28c84b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 -- 172.21.15.5:0/3985244929 >> 172.21.15.5:0/3985244929 conn(0x55ae28c7aa30 msgr2=0x55ae28c79b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 -- 172.21.15.5:0/3985244929 shutdown_connections 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 -- 172.21.15.5:0/3985244929 wait complete. 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 -- start start 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 Processor -- start 2024-09-11T22:32:10.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 0x55ae28c91ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c8f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c89680 0x55ae28c8f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581acd80 con 0x55ae28c7edc0 2024-09-11T22:32:10.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581ac6b0 con 0x55ae28c89680 2024-09-11T22:32:10.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9c581a9930 con 0x55ae28c7e4a0 2024-09-11T22:32:10.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5f05d740 10 client.0 _ll_register_callbacks cb 0x7f9c4c009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:10.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c8f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c8f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35630/0 (socket says 172.21.15.5:35630) 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c55d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c89680 0x55ae28c8f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c5659c640 1 -- 172.21.15.5:0/1102694553 learned_addr learned my addr 172.21.15.5:0/1102694553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.567+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 0x55ae28c91ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c5659c640 1 -- 172.21.15.5:0/1102694553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 msgr2=0x55ae28c91ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c5659c640 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 0x55ae28c91ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c5659c640 1 -- 172.21.15.5:0/1102694553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c89680 msgr2=0x55ae28c8f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c5659c640 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c89680 0x55ae28c8f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c5659c640 1 -- 172.21.15.5:0/1102694553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae28c9b6f0 con 0x55ae28c7edc0 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c5659c640 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c8f210 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x55ae28c85b40 tx=0x7f9c480045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381937561e9bbdb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:10.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1102694553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48003a00 con 0x55ae28c7edc0 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1102694553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7f9c48003ba0 con 0x55ae28c7edc0 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1102694553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c4c009a40 con 0x55ae28c7edc0 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1102694553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9c48003da0 con 0x55ae28c7edc0 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c5f05d740 10 client.18552 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1102694553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ae28c86ba0 con 0x55ae28c7edc0 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.568+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1102694553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c4c009a40 con 0x55ae28c7edc0 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.569+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1102694553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f9c480245d0 con 0x55ae28c7edc0 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.569+0000 7f9c5f05d740 10 client.18552 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.569+0000 7f9c5f05d740 10 client.18552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.569+0000 7f9c5f05d740 20 client.18552 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.569+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ae28d04090 0x55ae28d064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.569+0000 7f9c5f05d740 4 client.18552 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:32:10.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.569+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1102694553 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae28bebd60 con 0x55ae28d04090 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.569+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ae28d04090 0x55ae28d064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.569+0000 7f9c55d9b640 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ae28d04090 0x55ae28d064b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2853ced50b11753c server_cookie=b3bf23ba448b88fa in_seq=0 out_seq=0 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.569+0000 7f9c3dffb640 10 client.18552 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c3dffb640 1 -- 172.21.15.5:0/1102694553 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ae28bebd60 con 0x55ae28d04090 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c3dffb640 10 client.18552 handle_command_reply: tid=0 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c5f05d740 1 client.18552 shutdown 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1102694553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 msgr2=0x55ae28c8f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c8f210 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x55ae28c85b40 tx=0x7f9c480045c0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1102694553 shutdown_connections 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55ae28d04090 0x55ae28d064b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae28c89680 0x55ae28c8f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae28c7edc0 0x55ae28c8f210 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c5f05d740 1 --2- 172.21.15.5:0/1102694553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae28c7e4a0 0x55ae28c91ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.570+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1102694553 >> 172.21.15.5:0/1102694553 conn(0x55ae28c7aa30 msgr2=0x55ae28c79b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:10.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c3dffb640 0 client.18552 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:10.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1102694553 shutdown_connections 2024-09-11T22:32:10.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5f05d740 1 -- 172.21.15.5:0/1102694553 wait complete. 2024-09-11T22:32:10.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5f05d740 20 client.18552 trim_cache size 0 max 16384 2024-09-11T22:32:10.584 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:32:10.584 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:32:10.584 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:32:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "b53494d7-eab6-4104-93e0-1451305f3ffc": { 2024-09-11T22:32:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:32:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "b53494d7-eab6-4104-93e0-1451305f3ffc", 2024-09-11T22:32:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:32:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:10.585 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 -- 172.21.15.5:0/3714807945 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9c400a2d70 msgr2=0x7f9c400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9c400a2d70 0x7f9c400a5230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c500ee040 tx=0x7f9c5008bc90 comp rx=0 tx=0).stop 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 -- 172.21.15.5:0/3714807945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c581082a0 msgr2=0x7f9c5819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c581082a0 0x7f9c5819a5b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9c5002ae00 tx=0x7f9c50031e60 comp rx=0 tx=0).stop 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 -- 172.21.15.5:0/3714807945 shutdown_connections 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c581082a0 0x7f9c5819a5b0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c581036e0 0x7f9c581a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9c400a2d70 0x7f9c400a5230 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 --2- 172.21.15.5:0/3714807945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c58102d30 0x7f9c581a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 -- 172.21.15.5:0/3714807945 >> 172.21.15.5:0/3714807945 conn(0x7f9c580fc7d0 msgr2=0x7f9c581008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:10.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 -- 172.21.15.5:0/3714807945 shutdown_connections 2024-09-11T22:32:10.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:10.571+0000 7f9c5cc84640 1 -- 172.21.15.5:0/3714807945 wait complete. 2024-09-11T22:32:11.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 772640 7.77 MB/sec execute 389 sec latency 19.713 ms 2024-09-11T22:32:11.422 INFO:tasks.cephfs.filesystem:scrub status for tag:b53494d7-eab6-4104-93e0-1451305f3ffc - {'path': '/', 'tag': 'b53494d7-eab6-4104-93e0-1451305f3ffc', 'options': 'recursive,force'} 2024-09-11T22:32:11.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:11 smithi005 ceph-mon[30943]: pgmap v314: 97 pgs: 97 active+clean; 101 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 336 op/s 2024-09-11T22:32:11.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:11 smithi159 ceph-mon[35534]: pgmap v314: 97 pgs: 97 active+clean; 101 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 336 op/s 2024-09-11T22:32:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:11 smithi172 ceph-mon[35461]: pgmap v314: 97 pgs: 97 active+clean; 101 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 336 op/s 2024-09-11T22:32:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.942+0000 7f0fae6c9640 1 -- 172.21.15.5:0/3360558335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa8102c10 msgr2=0x7f0fa8102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.942+0000 7f0fae6c9640 1 --2- 172.21.15.5:0/3360558335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa8102c10 0x7f0fa8102ff0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009900 tx=0x7f0f9802a3f0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.942+0000 7f0fae6c9640 1 -- 172.21.15.5:0/3360558335 shutdown_connections 2024-09-11T22:32:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.942+0000 7f0fae6c9640 1 --2- 172.21.15.5:0/3360558335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fa810c110 0x7f0fa810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.942+0000 7f0fae6c9640 1 --2- 172.21.15.5:0/3360558335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa81035c0 0x7f0fa810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.942+0000 7f0fae6c9640 1 --2- 172.21.15.5:0/3360558335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa8102c10 0x7f0fa8102ff0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.942+0000 7f0fae6c9640 1 -- 172.21.15.5:0/3360558335 >> 172.21.15.5:0/3360558335 conn(0x7f0fa80fc6b0 msgr2=0x7f0fa80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.942+0000 7f0fae6c9640 1 -- 172.21.15.5:0/3360558335 shutdown_connections 2024-09-11T22:32:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.942+0000 7f0fae6c9640 1 -- 172.21.15.5:0/3360558335 wait complete. 2024-09-11T22:32:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.942+0000 7f0fae6c9640 1 Processor -- start 2024-09-11T22:32:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fae6c9640 1 -- start start 2024-09-11T22:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fae6c9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa8102c10 0x7f0fa8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fae6c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa81035c0 0x7f0fa8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fae6c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fa810c110 0x7f0fa819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fae6c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8111f10 con 0x7f0fa81035c0 2024-09-11T22:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fae6c9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8111d90 con 0x7f0fa8102c10 2024-09-11T22:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fae6c9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8112090 con 0x7f0fa810c110 2024-09-11T22:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa81035c0 0x7f0fa8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa81035c0 0x7f0fa8198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57698/0 (socket says 172.21.15.5:57698) 2024-09-11T22:32:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fa77fe640 1 -- 172.21.15.5:0/2445553478 learned_addr learned my addr 172.21.15.5:0/2445553478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fa7fff640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa8102c10 0x7f0fa8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0facc66640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fa810c110 0x7f0fa819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fa77fe640 1 -- 172.21.15.5:0/2445553478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fa810c110 msgr2=0x7f0fa819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fa810c110 0x7f0fa819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fa77fe640 1 -- 172.21.15.5:0/2445553478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa8102c10 msgr2=0x7f0fa8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa8102c10 0x7f0fa8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.943+0000 7f0fa77fe640 1 -- 172.21.15.5:0/2445553478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa819d330 con 0x7f0fa81035c0 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.944+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa81035c0 0x7f0fa8198880 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e9f0 tx=0x7f0f9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.944+0000 7f0fa57fa640 1 -- 172.21.15.5:0/2445553478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c00cd80 con 0x7f0fa81035c0 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.944+0000 7f0fae6c9640 1 -- 172.21.15.5:0/2445553478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa819d620 con 0x7f0fa81035c0 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.944+0000 7f0fa57fa640 1 -- 172.21.15.5:0/2445553478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9c010910 con 0x7f0fa81035c0 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.944+0000 7f0fa57fa640 1 -- 172.21.15.5:0/2445553478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c018990 con 0x7f0fa81035c0 2024-09-11T22:32:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.944+0000 7f0fae6c9640 1 -- 172.21.15.5:0/2445553478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa81a4eb0 con 0x7f0fa81035c0 2024-09-11T22:32:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.946+0000 7f0fb1461740 1 Processor -- start 2024-09-11T22:32:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.946+0000 7f0fa57fa640 1 -- 172.21.15.5:0/2445553478 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0f9c010ac0 con 0x7f0fa81035c0 2024-09-11T22:32:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.946+0000 7f0fb1461740 1 -- start start 2024-09-11T22:32:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.946+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7519f0 0x5649ea751dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea760b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7610c0 0x5649ea7634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fb1461740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5649ea6b62f0 con 0x5649ea752310 2024-09-11T22:32:11.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fb1461740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5649ea65e270 con 0x5649ea7519f0 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fb1461740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5649ea6b22b0 con 0x5649ea7610c0 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea760b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0facc66640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7519f0 0x5649ea751dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea760b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57708/0 (socket says 172.21.15.5:57708) 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0facc66640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7519f0 0x5649ea751dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46162/0 (socket says 172.21.15.5:46162) 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa7fff640 1 -- 172.21.15.5:0/2783878180 learned_addr learned my addr 172.21.15.5:0/2783878180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/2783878180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7610c0 0x5649ea7634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa7fff640 1 -- 172.21.15.5:0/2783878180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7610c0 msgr2=0x5649ea7634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa7fff640 1 --2- 172.21.15.5:0/2783878180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7610c0 0x5649ea7634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa7fff640 1 -- 172.21.15.5:0/2783878180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7519f0 msgr2=0x5649ea751dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa7fff640 1 --2- 172.21.15.5:0/2783878180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7519f0 0x5649ea751dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa7fff640 1 -- 172.21.15.5:0/2783878180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ea65d340 con 0x5649ea752310 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0facc66640 1 --2- 172.21.15.5:0/2783878180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7519f0 0x5649ea751dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa57fa640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0f900a2da0 0x7f0f900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0facc66640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0f900a2da0 0x7f0f900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa7fff640 1 --2- 172.21.15.5:0/2783878180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea760b80 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009900 tx=0x7f0f98005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626f4058e217904b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0fa57fa640 1 -- 172.21.15.5:0/2445553478 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7f0f9c0cf890 con 0x7f0fa81035c0 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0f8e7fc640 1 -- 172.21.15.5:0/2783878180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98035040 con 0x5649ea752310 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.947+0000 7f0f8e7fc640 1 -- 172.21.15.5:0/2783878180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f98004400 con 0x5649ea752310 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0f8e7fc640 1 -- 172.21.15.5:0/2783878180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f980035f0 con 0x5649ea752310 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0fb1461740 1 -- 172.21.15.5:0/2783878180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 msgr2=0x5649ea760b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0facc66640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0f900a2da0 0x7f0f900a5260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0005fd0 tx=0x7f0fa0009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0fb1461740 1 --2- 172.21.15.5:0/2783878180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea760b80 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009900 tx=0x7f0f98005de0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0fb1461740 1 -- 172.21.15.5:0/2783878180 shutdown_connections 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0fb1461740 1 --2- 172.21.15.5:0/2783878180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7610c0 0x5649ea7634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0fb1461740 1 --2- 172.21.15.5:0/2783878180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea760b80 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0fb1461740 1 --2- 172.21.15.5:0/2783878180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7519f0 0x5649ea751dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0fb1461740 1 -- 172.21.15.5:0/2783878180 >> 172.21.15.5:0/2783878180 conn(0x5649ea74e060 msgr2=0x5649ea74e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0fb1461740 1 -- 172.21.15.5:0/2783878180 shutdown_connections 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.948+0000 7f0fb1461740 1 -- 172.21.15.5:0/2783878180 wait complete. 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fb1461740 1 -- start start 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fb1461740 1 Processor -- start 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fb1461740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5649ea763cc0 con 0x5649ea752310 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fb1461740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5649ea63f290 con 0x5649ea7610c0 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fb1461740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5649ea763e40 con 0x5649ea7519f0 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fb1461740 10 client.0 _ll_register_callbacks cb 0x5649ea670140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fa7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0facc66640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fa7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47860/0 (socket says 172.21.15.5:47860) 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57724/0 (socket says 172.21.15.5:57724) 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0facc66640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46166/0 (socket says 172.21.15.5:46166) 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.949+0000 7f0fa7fff640 1 -- 172.21.15.5:0/1228678894 learned_addr learned my addr 172.21.15.5:0/1228678894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0fa7fff640 1 -- 172.21.15.5:0/1228678894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 msgr2=0x5649ea75eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0fa7fff640 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0fa7fff640 1 -- 172.21.15.5:0/1228678894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 msgr2=0x5649ea75e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0fa7fff640 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0fa7fff640 1 -- 172.21.15.5:0/1228678894 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ea3c9210 con 0x5649ea7519f0 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0fa7fff640 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75e020 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f98004920 tx=0x7f0f9802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9deef384edee246 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0f877fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0f877fe640 1 -- 172.21.15.5:0/1228678894 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98052060 con 0x5649ea7519f0 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0fb1461740 1 -- 172.21.15.5:0/1228678894 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f98004400 con 0x5649ea7519f0 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0f877fe640 1 -- 172.21.15.5:0/1228678894 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7f0f98004200 con 0x5649ea7519f0 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0f877fe640 1 -- 172.21.15.5:0/1228678894 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0f98035070 con 0x5649ea7519f0 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0fb1461740 10 client.27745 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:11.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0fb1461740 1 -- 172.21.15.5:0/1228678894 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5649ea495620 con 0x5649ea7519f0 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.950+0000 7f0f877fe640 1 -- 172.21.15.5:0/1228678894 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f98004400 con 0x5649ea7519f0 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.951+0000 7f0f877fe640 1 -- 172.21.15.5:0/1228678894 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f0f98031a80 con 0x5649ea7519f0 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.951+0000 7f0fb1461740 10 client.27745 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.951+0000 7f0fb1461740 10 client.27745 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.951+0000 7f0fb1461740 20 client.27745 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.951+0000 7f0fb1461740 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5649ea770f40 0x5649ea773330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.951+0000 7f0fb1461740 4 client.27745 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.951+0000 7f0fb1461740 1 -- 172.21.15.5:0/1228678894 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649ea6c3a20 con 0x5649ea770f40 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.951+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5649ea770f40 0x5649ea773330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.951+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5649ea770f40 0x5649ea773330 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6590d576791e9f5 server_cookie=8e081344b033fc88 in_seq=0 out_seq=0 2024-09-11T22:32:11.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.951+0000 7f0f877fe640 10 client.27745 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0f877fe640 1 -- 172.21.15.5:0/1228678894 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649ea6c3a20 con 0x5649ea770f40 2024-09-11T22:32:11.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0f877fe640 10 client.27745 handle_command_reply: tid=0 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 client.27745 shutdown 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 -- 172.21.15.5:0/1228678894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 msgr2=0x5649ea75e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75e020 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f98004920 tx=0x7f0f9802a3f0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 -- 172.21.15.5:0/1228678894 shutdown_connections 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5649ea770f40 0x5649ea773330 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 --2- 172.21.15.5:0/1228678894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75e020 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 -- 172.21.15.5:0/1228678894 >> 172.21.15.5:0/1228678894 conn(0x5649ea74e060 msgr2=0x5649ea753d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0f877fe640 0 client.27745 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:11.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 -- 172.21.15.5:0/1228678894 shutdown_connections 2024-09-11T22:32:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 1 -- 172.21.15.5:0/1228678894 wait complete. 2024-09-11T22:32:11.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.953+0000 7f0fb1461740 20 client.27745 trim_cache size 0 max 16384 2024-09-11T22:32:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fb1461740 1 Processor -- start 2024-09-11T22:32:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fb1461740 1 -- start start 2024-09-11T22:32:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea7788d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea778e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fb1461740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81ab8d0 con 0x5649ea752310 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fb1461740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81ad410 con 0x5649ea7610c0 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fb1461740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81abfa0 con 0x5649ea7519f0 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea778e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea778e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57732/0 (socket says 172.21.15.5:57732) 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fa77fe640 1 -- 172.21.15.5:0/2284700199 learned_addr learned my addr 172.21.15.5:0/2284700199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fa7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea7788d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0facc66640 1 --2- 172.21.15.5:0/2284700199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fa77fe640 1 -- 172.21.15.5:0/2284700199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 msgr2=0x5649ea7788d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/2284700199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea7788d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fa77fe640 1 -- 172.21.15.5:0/2284700199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 msgr2=0x5649ea75f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/2284700199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.969+0000 7f0fa77fe640 1 -- 172.21.15.5:0/2284700199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649ea74de90 con 0x5649ea752310 2024-09-11T22:32:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.970+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/2284700199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea778e10 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00c6a0 tx=0x7f0f9c0d1da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c632eadb5905022 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.970+0000 7f0f8dffb640 1 -- 172.21.15.5:0/2284700199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c09b670 con 0x5649ea752310 2024-09-11T22:32:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.970+0000 7f0f8dffb640 1 -- 172.21.15.5:0/2284700199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9c09c410 con 0x5649ea752310 2024-09-11T22:32:11.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.970+0000 7f0f8dffb640 1 -- 172.21.15.5:0/2284700199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c018b30 con 0x5649ea752310 2024-09-11T22:32:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.970+0000 7f0fb1461740 1 -- 172.21.15.5:0/2284700199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 msgr2=0x5649ea778e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.970+0000 7f0fb1461740 1 --2- 172.21.15.5:0/2284700199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea778e10 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00c6a0 tx=0x7f0f9c0d1da0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.970+0000 7f0fb1461740 1 -- 172.21.15.5:0/2284700199 shutdown_connections 2024-09-11T22:32:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.970+0000 7f0fb1461740 1 --2- 172.21.15.5:0/2284700199 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.970+0000 7f0fb1461740 1 --2- 172.21.15.5:0/2284700199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea778e10 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.970+0000 7f0fb1461740 1 --2- 172.21.15.5:0/2284700199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea7788d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 -- 172.21.15.5:0/2284700199 >> 172.21.15.5:0/2284700199 conn(0x5649ea74e060 msgr2=0x5649ea75d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 -- 172.21.15.5:0/2284700199 shutdown_connections 2024-09-11T22:32:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 -- 172.21.15.5:0/2284700199 wait complete. 2024-09-11T22:32:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 -- start start 2024-09-11T22:32:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 Processor -- start 2024-09-11T22:32:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81acd40 con 0x5649ea752310 2024-09-11T22:32:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81ac670 con 0x5649ea7610c0 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81a98d0 con 0x5649ea7519f0 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.971+0000 7f0fb1461740 10 client.0 _ll_register_callbacks cb 0x7f0fa00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57748/0 (socket says 172.21.15.5:57748) 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa77fe640 1 -- 172.21.15.5:0/968127725 learned_addr learned my addr 172.21.15.5:0/968127725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47886/0 (socket says 172.21.15.5:47886) 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0facc66640 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa77fe640 1 -- 172.21.15.5:0/968127725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 msgr2=0x5649ea75afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa77fe640 1 -- 172.21.15.5:0/968127725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 msgr2=0x5649ea75ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa77fe640 1 -- 172.21.15.5:0/968127725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649ea763850 con 0x5649ea752310 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fa77fe640 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75b510 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c09bca0 tx=0x7f0f9c0d5330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49e2257014b2832 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0f877fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0f877fe640 1 -- 172.21.15.5:0/968127725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0b8020 con 0x5649ea752310 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0f877fe640 1 -- 172.21.15.5:0/968127725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8673+0+0 (secure 0 0 0) 0x7f0f9c0a4bd0 con 0x5649ea752310 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fb1461740 1 -- 172.21.15.5:0/968127725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f9c09c410 con 0x5649ea752310 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0f877fe640 1 -- 172.21.15.5:0/968127725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0f9c0a00c0 con 0x5649ea752310 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fb1461740 10 client.18570 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0fb1461740 1 -- 172.21.15.5:0/968127725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5649ea760bb0 con 0x5649ea752310 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.972+0000 7f0f877fe640 1 -- 172.21.15.5:0/968127725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9c09c410 con 0x5649ea752310 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.973+0000 7f0f877fe640 1 -- 172.21.15.5:0/968127725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f0f9c09b1f0 con 0x5649ea752310 2024-09-11T22:32:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.973+0000 7f0fb1461740 10 client.18570 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.973+0000 7f0fb1461740 10 client.18570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.973+0000 7f0fb1461740 20 client.18570 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.973+0000 7f0fb1461740 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5649ea7dbdd0 0x5649ea7de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.973+0000 7f0fb1461740 4 client.18570 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:32:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.973+0000 7f0fb1461740 1 -- 172.21.15.5:0/968127725 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5649ea6bdd70 con 0x5649ea7dbdd0 2024-09-11T22:32:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.973+0000 7f0fa7fff640 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5649ea7dbdd0 0x5649ea7de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.973+0000 7f0fa7fff640 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5649ea7dbdd0 0x5649ea7de1c0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879250948f899f08 server_cookie=8255b78189b0a12b in_seq=0 out_seq=0 2024-09-11T22:32:11.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:11.973+0000 7f0f877fe640 10 client.18570 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0f877fe640 1 -- 172.21.15.5:0/968127725 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5649ea6bdd70 con 0x5649ea7dbdd0 2024-09-11T22:32:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0f877fe640 10 client.18570 handle_command_reply: tid=0 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0fb1461740 1 client.18570 shutdown 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0fb1461740 1 -- 172.21.15.5:0/968127725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 msgr2=0x5649ea75b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0fb1461740 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75b510 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c09bca0 tx=0x7f0f9c0d5330 comp rx=0 tx=0).stop 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0fb1461740 1 -- 172.21.15.5:0/968127725 shutdown_connections 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0fb1461740 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5649ea7dbdd0 0x5649ea7de1c0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0fb1461740 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5649ea7610c0 0x5649ea75ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0fb1461740 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649ea752310 0x5649ea75b510 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0fb1461740 1 --2- 172.21.15.5:0/968127725 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5649ea7519f0 0x5649ea75afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0f877fe640 0 client.18570 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.090+0000 7f0fb1461740 1 -- 172.21.15.5:0/968127725 >> 172.21.15.5:0/968127725 conn(0x5649ea74e060 msgr2=0x5649ea75f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.091+0000 7f0fb1461740 1 -- 172.21.15.5:0/968127725 shutdown_connections 2024-09-11T22:32:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.091+0000 7f0fb1461740 1 -- 172.21.15.5:0/968127725 wait complete. 2024-09-11T22:32:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.091+0000 7f0fb1461740 20 client.18570 trim_cache size 0 max 16384 2024-09-11T22:32:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:32:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:32:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:32:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:32:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 -- 172.21.15.5:0/2445553478 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0f900a2da0 msgr2=0x7f0f900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0f900a2da0 0x7f0f900a5260 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0005fd0 tx=0x7f0fa0009500 comp rx=0 tx=0).stop 2024-09-11T22:32:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 -- 172.21.15.5:0/2445553478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa81035c0 msgr2=0x7f0fa8198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa81035c0 0x7f0fa8198880 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e9f0 tx=0x7f0f9c00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:32:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 -- 172.21.15.5:0/2445553478 shutdown_connections 2024-09-11T22:32:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0fa810c110 0x7f0fa819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fa81035c0 0x7f0fa8198880 secure :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e9f0 tx=0x7f0f9c00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:32:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0f900a2da0 0x7f0f900a5260 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 --2- 172.21.15.5:0/2445553478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0fa8102c10 0x7f0fa8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 -- 172.21.15.5:0/2445553478 >> 172.21.15.5:0/2445553478 conn(0x7f0fa80fc6b0 msgr2=0x7f0fa80fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 -- 172.21.15.5:0/2445553478 shutdown_connections 2024-09-11T22:32:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:12.092+0000 7f0fae6c9640 1 -- 172.21.15.5:0/2445553478 wait complete. 2024-09-11T22:32:12.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 774427 7.78 MB/sec execute 390 sec latency 52.226 ms 2024-09-11T22:32:13.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 776489 7.78 MB/sec execute 391 sec latency 20.131 ms 2024-09-11T22:32:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:13 smithi005 ceph-mon[30943]: pgmap v315: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 297 op/s 2024-09-11T22:32:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:13 smithi005 ceph-mon[30943]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T22:32:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:13 smithi159 ceph-mon[35534]: pgmap v315: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 297 op/s 2024-09-11T22:32:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:13 smithi159 ceph-mon[35534]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T22:32:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:13 smithi172 ceph-mon[35461]: pgmap v315: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 297 op/s 2024-09-11T22:32:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:13 smithi172 ceph-mon[35461]: osdmap e98: 12 total, 12 up, 12 in 2024-09-11T22:32:14.314 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T22:32:14.314 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-11T22:32:14.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:32:14.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 777654 7.79 MB/sec execute 392 sec latency 465.099 ms 2024-09-11T22:32:14.612 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:14.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:14 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:32:14.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:14 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:32:14.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:14 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:32:14.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:14 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:32:14.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:14 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:32:14.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:14 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:32:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.129+0000 7f0efb9fe640 1 -- 172.21.15.5:0/3643964927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4103660 msgr2=0x7f0ef410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.129+0000 7f0efb9fe640 1 --2- 172.21.15.5:0/3643964927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4103660 0x7f0ef410bb50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009900 tx=0x7f0ee402a480 comp rx=0 tx=0).stop 2024-09-11T22:32:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.129+0000 7f0efb9fe640 1 -- 172.21.15.5:0/3643964927 shutdown_connections 2024-09-11T22:32:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.129+0000 7f0efb9fe640 1 --2- 172.21.15.5:0/3643964927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef410c170 0x7f0ef410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.129+0000 7f0efb9fe640 1 --2- 172.21.15.5:0/3643964927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef4103660 0x7f0ef410bb50 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.129+0000 7f0efb9fe640 1 --2- 172.21.15.5:0/3643964927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ef4102d40 0x7f0ef4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.129+0000 7f0efb9fe640 1 -- 172.21.15.5:0/3643964927 >> 172.21.15.5:0/3643964927 conn(0x7f0ef40fc7a0 msgr2=0x7f0ef40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.129+0000 7f0efb9fe640 1 -- 172.21.15.5:0/3643964927 shutdown_connections 2024-09-11T22:32:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.129+0000 7f0efb9fe640 1 -- 172.21.15.5:0/3643964927 wait complete. 2024-09-11T22:32:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0efb9fe640 1 Processor -- start 2024-09-11T22:32:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0efb9fe640 1 -- start start 2024-09-11T22:32:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0efb9fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ef4102d40 0x7f0ef41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0efb9fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef4103660 0x7f0ef4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0efb9fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef410c170 0x7f0ef4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0efb9fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4111f70 con 0x7f0ef410c170 2024-09-11T22:32:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0efb9fe640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef4111df0 con 0x7f0ef4103660 2024-09-11T22:32:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0efb9fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef41120f0 con 0x7f0ef4102d40 2024-09-11T22:32:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0ef9f9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef410c170 0x7f0ef4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0ef9f9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef410c170 0x7f0ef4101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57752/0 (socket says 172.21.15.5:57752) 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.130+0000 7f0ef9f9b640 1 -- 172.21.15.5:0/1526462403 learned_addr learned my addr 172.21.15.5:0/1526462403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0ef8f99640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef4103660 0x7f0ef4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0ef979a640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ef4102d40 0x7f0ef41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0ef9f9b640 1 -- 172.21.15.5:0/1526462403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ef4102d40 msgr2=0x7f0ef41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0ef9f9b640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ef4102d40 0x7f0ef41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0ef9f9b640 1 -- 172.21.15.5:0/1526462403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef4103660 msgr2=0x7f0ef4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0ef9f9b640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef4103660 0x7f0ef4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0ef9f9b640 1 -- 172.21.15.5:0/1526462403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef41a6990 con 0x7f0ef410c170 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0ef9f9b640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef410c170 0x7f0ef4101420 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00eaa0 tx=0x7f0eec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0eea7fc640 1 -- 172.21.15.5:0/1526462403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec00cdb0 con 0x7f0ef410c170 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0eea7fc640 1 -- 172.21.15.5:0/1526462403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eec005020 con 0x7f0ef410c170 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0efb9fe640 1 -- 172.21.15.5:0/1526462403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef41a6c80 con 0x7f0ef410c170 2024-09-11T22:32:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0eea7fc640 1 -- 172.21.15.5:0/1526462403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0116b0 con 0x7f0ef410c170 2024-09-11T22:32:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.131+0000 7f0efb9fe640 1 -- 172.21.15.5:0/1526462403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef41a7140 con 0x7f0ef410c170 2024-09-11T22:32:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.132+0000 7f0efb9fe640 1 -- 172.21.15.5:0/1526462403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef4103120 con 0x7f0ef410c170 2024-09-11T22:32:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.133+0000 7f0eea7fc640 1 -- 172.21.15.5:0/1526462403 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0eec005560 con 0x7f0ef410c170 2024-09-11T22:32:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.134+0000 7f0eea7fc640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ee00a2cf0 0x7f0ee00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.134+0000 7f0eea7fc640 1 -- 172.21.15.5:0/1526462403 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8629+0+0 (secure 0 0 0) 0x7f0eec0cf9b0 con 0x7f0ef410c170 2024-09-11T22:32:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.134+0000 7f0ef979a640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ee00a2cf0 0x7f0ee00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.135+0000 7f0ef979a640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ee00a2cf0 0x7f0ee00a51b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009bd0 tx=0x7f0ef0005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.138+0000 7f0eea7fc640 1 -- 172.21.15.5:0/1526462403 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0eec016030 con 0x7f0ef410c170 2024-09-11T22:32:15.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 778774 7.78 MB/sec execute 393 sec latency 88.402 ms 2024-09-11T22:32:15.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.360+0000 7f0efb9fe640 1 -- 172.21.15.5:0/1526462403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ef41071e0 con 0x7f0ef410c170 2024-09-11T22:32:15.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.361+0000 7f0eea7fc640 1 -- 172.21.15.5:0/1526462403 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11967 (secure 0 0 0) 0x7f0eec09bf10 con 0x7f0ef410c170 2024-09-11T22:32:15.362 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:32:15.362 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-11T22:31:33:497496+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:31:33.497479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27553":{"gid":27553,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1751117477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1751117477},{"type":"v1","addr":"172.21.15.159:6833","nonce":1751117477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:32:15.362 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-11T22:32:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 -- 172.21.15.5:0/1526462403 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ee00a2cf0 msgr2=0x7f0ee00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ee00a2cf0 0x7f0ee00a51b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0009bd0 tx=0x7f0ef0005f90 comp rx=0 tx=0).stop 2024-09-11T22:32:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 -- 172.21.15.5:0/1526462403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef410c170 msgr2=0x7f0ef4101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef410c170 0x7f0ef4101420 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f0eec00eaa0 tx=0x7f0eec00ef60 comp rx=0 tx=0).stop 2024-09-11T22:32:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 -- 172.21.15.5:0/1526462403 shutdown_connections 2024-09-11T22:32:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ef410c170 0x7f0ef4101420 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ee00a2cf0 0x7f0ee00a51b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ef4103660 0x7f0ef4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 --2- 172.21.15.5:0/1526462403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ef4102d40 0x7f0ef41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:15.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 -- 172.21.15.5:0/1526462403 >> 172.21.15.5:0/1526462403 conn(0x7f0ef40fc7a0 msgr2=0x7f0ef40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:15.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 -- 172.21.15.5:0/1526462403 shutdown_connections 2024-09-11T22:32:15.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:15.363+0000 7f0efb9fe640 1 -- 172.21.15.5:0/1526462403 wait complete. 2024-09-11T22:32:15.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:15 smithi005 ceph-mon[30943]: pgmap v317: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 310 op/s 2024-09-11T22:32:15.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:15 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1526462403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:15 smithi159 ceph-mon[35534]: pgmap v317: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 310 op/s 2024-09-11T22:32:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:15 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1526462403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:15 smithi172 ceph-mon[35461]: pgmap v317: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 310 op/s 2024-09-11T22:32:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:15 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1526462403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:16.196 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-11T22:32:16.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-11T22:32:16.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 780584 7.78 MB/sec execute 394 sec latency 112.159 ms 2024-09-11T22:32:16.500 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:17.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.990+0000 7f60b45f8640 1 -- 172.21.15.5:0/670955575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac103660 msgr2=0x7f60ac10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.990+0000 7f60b45f8640 1 --2- 172.21.15.5:0/670955575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac103660 0x7f60ac10bb50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f609c009900 tx=0x7f609c02a480 comp rx=0 tx=0).stop 2024-09-11T22:32:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.990+0000 7f60b45f8640 1 -- 172.21.15.5:0/670955575 shutdown_connections 2024-09-11T22:32:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.990+0000 7f60b45f8640 1 --2- 172.21.15.5:0/670955575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac10c170 0x7f60ac10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.990+0000 7f60b45f8640 1 --2- 172.21.15.5:0/670955575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac103660 0x7f60ac10bb50 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.990+0000 7f60b45f8640 1 --2- 172.21.15.5:0/670955575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac102d40 0x7f60ac103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.990+0000 7f60b45f8640 1 -- 172.21.15.5:0/670955575 >> 172.21.15.5:0/670955575 conn(0x7f60ac0fc7a0 msgr2=0x7f60ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.990+0000 7f60b45f8640 1 -- 172.21.15.5:0/670955575 shutdown_connections 2024-09-11T22:32:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.990+0000 7f60b45f8640 1 -- 172.21.15.5:0/670955575 wait complete. 2024-09-11T22:32:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b45f8640 1 Processor -- start 2024-09-11T22:32:17.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b45f8640 1 -- start start 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b45f8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac102d40 0x7f60ac1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b45f8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac103660 0x7f60ac100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b45f8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac10c170 0x7f60ac101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b45f8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac111f70 con 0x7f60ac103660 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b45f8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac111df0 con 0x7f60ac102d40 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b45f8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1120f0 con 0x7f60ac10c170 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b1b93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac103660 0x7f60ac100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b2b95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac10c170 0x7f60ac101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b2b95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac10c170 0x7f60ac101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47908/0 (socket says 172.21.15.5:47908) 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b2b95640 1 -- 172.21.15.5:0/3846940227 learned_addr learned my addr 172.21.15.5:0/3846940227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b2394640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac102d40 0x7f60ac1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b1b93640 1 -- 172.21.15.5:0/3846940227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac10c170 msgr2=0x7f60ac101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:17.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b1b93640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac10c170 0x7f60ac101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b1b93640 1 -- 172.21.15.5:0/3846940227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac102d40 msgr2=0x7f60ac1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b1b93640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac102d40 0x7f60ac1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.991+0000 7f60b1b93640 1 -- 172.21.15.5:0/3846940227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ac1a6970 con 0x7f60ac103660 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.992+0000 7f60b1b93640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac103660 0x7f60ac100ee0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f609c005940 tx=0x7f609c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.992+0000 7f60a37fe640 1 -- 172.21.15.5:0/3846940227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c03f030 con 0x7f60ac103660 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.992+0000 7f60a37fe640 1 -- 172.21.15.5:0/3846940227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609c004440 con 0x7f60ac103660 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.992+0000 7f60b45f8640 1 -- 172.21.15.5:0/3846940227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ac1a6c00 con 0x7f60ac103660 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.992+0000 7f60a37fe640 1 -- 172.21.15.5:0/3846940227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c035070 con 0x7f60ac103660 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.992+0000 7f60b45f8640 1 -- 172.21.15.5:0/3846940227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ac1a7130 con 0x7f60ac103660 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.994+0000 7f60a37fe640 1 -- 172.21.15.5:0/3846940227 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f609c042030 con 0x7f60ac103660 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.994+0000 7f60b69d1740 1 Processor -- start 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b69d1740 1 -- start start 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 0x55c6534e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 0x55c6534f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b69d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c65344d160 con 0x55c6534f3a30 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b69d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c6533f4720 con 0x55c6534e9170 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b69d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c653449120 con 0x55c6534e8850 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b2394640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 0x55c6534f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b2b95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 0x55c6534e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b1b93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b2394640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 0x55c6534f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46218/0 (socket says 172.21.15.5:46218) 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b2b95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 0x55c6534e8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47920/0 (socket says 172.21.15.5:47920) 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60b2394640 1 -- 172.21.15.5:0/1487047714 learned_addr learned my addr 172.21.15.5:0/1487047714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.995+0000 7f60a37fe640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2d70 0x7f60980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60a37fe640 1 -- 172.21.15.5:0/3846940227 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8629+0+0 (secure 0 0 0) 0x7f609c0ef6a0 con 0x7f60ac103660 2024-09-11T22:32:17.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60b1b93640 1 -- 172.21.15.5:0/1487047714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 msgr2=0x55c6534e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60b2b95640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2d70 0x7f60980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60b1b93640 1 --2- 172.21.15.5:0/1487047714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 0x55c6534e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60b1b93640 1 -- 172.21.15.5:0/1487047714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 msgr2=0x55c6534f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60b1b93640 1 --2- 172.21.15.5:0/1487047714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 0x55c6534f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60b1b93640 1 -- 172.21.15.5:0/1487047714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6531beba0 con 0x55c6534f3a30 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60b1b93640 1 --2- 172.21.15.5:0/1487047714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f5e20 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f609c0ecd20 tx=0x7f609c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214023b7eed1def2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60a0ff9640 1 -- 172.21.15.5:0/1487047714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c049070 con 0x55c6534f3a30 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60a0ff9640 1 -- 172.21.15.5:0/1487047714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609c0f5070 con 0x55c6534f3a30 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60a0ff9640 1 -- 172.21.15.5:0/1487047714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c0b8360 con 0x55c6534f3a30 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.996+0000 7f60b2b95640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2d70 0x7f60980a5230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009810 tx=0x7f60a4004730 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 -- 172.21.15.5:0/1487047714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 msgr2=0x55c6534f5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 --2- 172.21.15.5:0/1487047714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f5e20 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f609c0ecd20 tx=0x7f609c009680 comp rx=0 tx=0).stop 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 -- 172.21.15.5:0/1487047714 shutdown_connections 2024-09-11T22:32:17.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 --2- 172.21.15.5:0/1487047714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f5e20 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 --2- 172.21.15.5:0/1487047714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 0x55c6534f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 --2- 172.21.15.5:0/1487047714 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 0x55c6534e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 -- 172.21.15.5:0/1487047714 >> 172.21.15.5:0/1487047714 conn(0x55c6534e4de0 msgr2=0x55c6534e51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 -- 172.21.15.5:0/1487047714 shutdown_connections 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 -- 172.21.15.5:0/1487047714 wait complete. 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 -- start start 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.997+0000 7f60b69d1740 1 Processor -- start 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 0x55c6534f0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 0x55c6534f0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b69d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c6534f63c0 con 0x55c6534f3a30 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b69d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c6533d5900 con 0x55c6534e9170 2024-09-11T22:32:17.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b69d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c6534f6540 con 0x55c6534e8850 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b69d1740 10 client.0 _ll_register_callbacks cb 0x55c6533d3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2b95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2394640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 0x55c6534f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b1b93640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 0x55c6534f0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2b95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57778/0 (socket says 172.21.15.5:57778) 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2b95640 1 -- 172.21.15.5:0/1599229094 learned_addr learned my addr 172.21.15.5:0/1599229094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2b95640 1 -- 172.21.15.5:0/1599229094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 msgr2=0x55c6534f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2b95640 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 0x55c6534f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2b95640 1 -- 172.21.15.5:0/1599229094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 msgr2=0x55c6534f0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2b95640 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 0x55c6534f0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2b95640 1 -- 172.21.15.5:0/1599229094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c653415c50 con 0x55c6534f3a30 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2394640 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 0x55c6534f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f60b2b95640 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f13c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f60a4011b90 tx=0x7f60a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf1beb7327a7de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:17.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.998+0000 7f6095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f6095ffb640 1 -- 172.21.15.5:0/1599229094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a4026070 con 0x55c6534f3a30 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f6095ffb640 1 -- 172.21.15.5:0/1599229094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8629+0+0 (secure 0 0 0) 0x7f60a4022480 con 0x55c6534f3a30 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f60b69d1740 1 -- 172.21.15.5:0/1599229094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f609c0f5070 con 0x55c6534f3a30 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f6095ffb640 1 -- 172.21.15.5:0/1599229094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f60a4022680 con 0x55c6534f3a30 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f60b69d1740 10 client.18588 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f60b69d1740 1 -- 172.21.15.5:0/1599229094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c653502070 con 0x55c6534f3a30 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f6095ffb640 1 -- 172.21.15.5:0/1599229094 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609c0f5070 con 0x55c6534f3a30 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f6095ffb640 1 -- 172.21.15.5:0/1599229094 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f60a401f050 con 0x55c6534f3a30 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f60b69d1740 10 client.18588 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f60b69d1740 10 client.18588 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f60b69d1740 20 client.18588 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f60b69d1740 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c653503480 0x55c653505890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f60b69d1740 4 client.18588 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:16.999+0000 7f60b69d1740 1 -- 172.21.15.5:0/1599229094 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c65345a890 con 0x55c653503480 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.000+0000 7f60b2394640 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c653503480 0x55c653505890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.000+0000 7f60b2394640 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c653503480 0x55c653505890 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908e761a2f9fd156 server_cookie=c4843be778e4217c in_seq=0 out_seq=0 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.000+0000 7f6095ffb640 10 client.18588 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.001+0000 7f6095ffb640 1 -- 172.21.15.5:0/1599229094 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c65345a890 con 0x55c653503480 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.001+0000 7f6095ffb640 10 client.18588 handle_command_reply: tid=0 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.001+0000 7f60b69d1740 1 client.18588 shutdown 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.001+0000 7f60b69d1740 1 -- 172.21.15.5:0/1599229094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 msgr2=0x55c6534f13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.001+0000 7f60b69d1740 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f13c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f60a4011b90 tx=0x7f60a40046c0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.001+0000 7f60b69d1740 1 -- 172.21.15.5:0/1599229094 shutdown_connections 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.001+0000 7f60b69d1740 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c653503480 0x55c653505890 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.002+0000 7f60b69d1740 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534f3a30 0x55c6534f13c0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.002+0000 7f6095ffb640 0 client.18588 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.002+0000 7f60b69d1740 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e9170 0x55c6534f0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.002+0000 7f60b69d1740 1 --2- 172.21.15.5:0/1599229094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534e8850 0x55c6534f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.002+0000 7f60b69d1740 1 -- 172.21.15.5:0/1599229094 >> 172.21.15.5:0/1599229094 conn(0x55c6534e4de0 msgr2=0x55c6534eaac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.002+0000 7f60b69d1740 1 -- 172.21.15.5:0/1599229094 shutdown_connections 2024-09-11T22:32:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.002+0000 7f60b69d1740 1 -- 172.21.15.5:0/1599229094 wait complete. 2024-09-11T22:32:17.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.002+0000 7f60b69d1740 20 client.18588 trim_cache size 0 max 16384 2024-09-11T22:32:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.019+0000 7f60b69d1740 1 Processor -- start 2024-09-11T22:32:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.019+0000 7f60b69d1740 1 -- start start 2024-09-11T22:32:17.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.019+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 0x55c65350b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.019+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c65350b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.019+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 0x55c6534edba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.019+0000 7f60b69d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1afcf0 con 0x55c6534e9170 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.019+0000 7f60b69d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1b1830 con 0x55c6534e8850 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.019+0000 7f60b69d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1b03c0 con 0x55c6534f3a30 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.019+0000 7f60b1b93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c65350b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b2394640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 0x55c65350b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b2b95640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 0x55c6534edba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b1b93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c65350b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57782/0 (socket says 172.21.15.5:57782) 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b1b93640 1 -- 172.21.15.5:0/2319144392 learned_addr learned my addr 172.21.15.5:0/2319144392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b1b93640 1 -- 172.21.15.5:0/2319144392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 msgr2=0x55c6534edba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b1b93640 1 --2- 172.21.15.5:0/2319144392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 0x55c6534edba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b1b93640 1 -- 172.21.15.5:0/2319144392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 msgr2=0x55c65350b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b1b93640 1 --2- 172.21.15.5:0/2319144392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 0x55c65350b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b1b93640 1 -- 172.21.15.5:0/2319144392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6534e03f0 con 0x55c6534e9170 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b2b95640 1 --2- 172.21.15.5:0/2319144392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 0x55c6534edba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b2394640 1 --2- 172.21.15.5:0/2319144392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 0x55c65350b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f60b1b93640 1 --2- 172.21.15.5:0/2319144392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c65350b6a0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f609c0b8570 tx=0x7f609c0b7ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2662de1fb042d71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f6097fff640 1 -- 172.21.15.5:0/2319144392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c049040 con 0x55c6534e9170 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f6097fff640 1 -- 172.21.15.5:0/2319144392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609c0f5040 con 0x55c6534e9170 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.020+0000 7f6097fff640 1 -- 172.21.15.5:0/2319144392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c0bbe00 con 0x55c6534e9170 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 -- 172.21.15.5:0/2319144392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 msgr2=0x55c65350b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 --2- 172.21.15.5:0/2319144392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c65350b6a0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f609c0b8570 tx=0x7f609c0b7ec0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 -- 172.21.15.5:0/2319144392 shutdown_connections 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 --2- 172.21.15.5:0/2319144392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 0x55c6534edba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 --2- 172.21.15.5:0/2319144392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c65350b6a0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 --2- 172.21.15.5:0/2319144392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 0x55c65350b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 -- 172.21.15.5:0/2319144392 >> 172.21.15.5:0/2319144392 conn(0x55c6534e4de0 msgr2=0x55c6534f29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 -- 172.21.15.5:0/2319144392 shutdown_connections 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 -- 172.21.15.5:0/2319144392 wait complete. 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 -- start start 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.021+0000 7f60b69d1740 1 Processor -- start 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 0x55c6534f7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c6534f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b69d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 0x55c6534f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b69d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1b1160 con 0x55c6534e9170 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b69d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1b0a90 con 0x55c6534e8850 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b69d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60ac1adcf0 con 0x55c6534f3a30 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b1b93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c6534f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b2394640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 0x55c6534f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b1b93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c6534f7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57784/0 (socket says 172.21.15.5:57784) 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b1b93640 1 -- 172.21.15.5:0/2051898999 learned_addr learned my addr 172.21.15.5:0/2051898999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b69d1740 10 client.0 _ll_register_callbacks cb 0x7f60a4009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b2b95640 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 0x55c6534f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b1b93640 1 -- 172.21.15.5:0/2051898999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 msgr2=0x55c6534f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b1b93640 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 0x55c6534f80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b1b93640 1 -- 172.21.15.5:0/2051898999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 msgr2=0x55c6534f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b1b93640 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 0x55c6534f7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b1b93640 1 -- 172.21.15.5:0/2051898999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c65356d380 con 0x55c6534e9170 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b2394640 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 0x55c6534f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f60b1b93640 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c6534f7bb0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f609c03d040 tx=0x7f609c003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3bef061ea6decd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.022+0000 7f6095ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f6095ffb640 1 -- 172.21.15.5:0/2051898999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609c049070 con 0x55c6534e9170 2024-09-11T22:32:17.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f60b69d1740 1 -- 172.21.15.5:0/2051898999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f609c0f5040 con 0x55c6534e9170 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f6095ffb640 1 -- 172.21.15.5:0/2051898999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8629+0+0 (secure 0 0 0) 0x7f609c0ecfa0 con 0x55c6534e9170 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f6095ffb640 1 -- 172.21.15.5:0/2051898999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f609c0bfea0 con 0x55c6534e9170 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f60b69d1740 10 client.18594 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f60b69d1740 1 -- 172.21.15.5:0/2051898999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c65356e640 con 0x55c6534e9170 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f6095ffb640 1 -- 172.21.15.5:0/2051898999 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f609c0f5040 con 0x55c6534e9170 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f6095ffb640 1 -- 172.21.15.5:0/2051898999 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f609c0c42d0 con 0x55c6534e9170 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f60b69d1740 10 client.18594 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f60b69d1740 10 client.18594 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f60b69d1740 20 client.18594 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f60b69d1740 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c653503480 0x55c6535709c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f60b69d1740 4 client.18594 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f60b69d1740 1 -- 172.21.15.5:0/2051898999 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x55c653454be0 con 0x55c653503480 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.023+0000 7f60b2394640 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c653503480 0x55c6535709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:17.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.024+0000 7f6095ffb640 10 client.18594 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:17.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.024+0000 7f60b2394640 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c653503480 0x55c6535709c0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae30b387c5f4fb88 server_cookie=c6e65cd1601923ec in_seq=0 out_seq=0 2024-09-11T22:32:17.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f6095ffb640 1 -- 172.21.15.5:0/2051898999 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c653454be0 con 0x55c653503480 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f6095ffb640 10 client.18594 handle_command_reply: tid=0 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 client.18594 shutdown 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 -- 172.21.15.5:0/2051898999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 msgr2=0x55c6534f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c6534f7bb0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f609c03d040 tx=0x7f609c003f70 comp rx=0 tx=0).stop 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 -- 172.21.15.5:0/2051898999 shutdown_connections 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c653503480 0x55c6535709c0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6534f3a30 0x55c6534f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c6534e9170 0x55c6534f7bb0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 --2- 172.21.15.5:0/2051898999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c6534e8850 0x55c6534f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 -- 172.21.15.5:0/2051898999 >> 172.21.15.5:0/2051898999 conn(0x55c6534e4de0 msgr2=0x55c6534f0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f6095ffb640 0 client.18594 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 -- 172.21.15.5:0/2051898999 shutdown_connections 2024-09-11T22:32:17.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.151+0000 7f60b69d1740 1 -- 172.21.15.5:0/2051898999 wait complete. 2024-09-11T22:32:17.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b69d1740 20 client.18594 trim_cache size 0 max 16384 2024-09-11T22:32:17.154 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:32:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-11T22:32:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:32:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 38, 2024-09-11T22:32:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:32:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "70839249": { 2024-09-11T22:32:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 38, 2024-09-11T22:32:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:32:17.154 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.6, 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:17.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:17.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:17.156 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:32:17.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 -- 172.21.15.5:0/3846940227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2d70 msgr2=0x7f60980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:17.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2d70 0x7f60980a5230 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009810 tx=0x7f60a4004730 comp rx=0 tx=0).stop 2024-09-11T22:32:17.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 -- 172.21.15.5:0/3846940227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac103660 msgr2=0x7f60ac100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:17.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac103660 0x7f60ac100ee0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f609c005940 tx=0x7f609c004040 comp rx=0 tx=0).stop 2024-09-11T22:32:17.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 -- 172.21.15.5:0/3846940227 shutdown_connections 2024-09-11T22:32:17.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60ac10c170 0x7f60ac101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60ac103660 0x7f60ac100ee0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60980a2d70 0x7f60980a5230 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 --2- 172.21.15.5:0/3846940227 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60ac102d40 0x7f60ac1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:17.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 -- 172.21.15.5:0/3846940227 >> 172.21.15.5:0/3846940227 conn(0x7f60ac0fc7a0 msgr2=0x7f60ac0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:17.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 -- 172.21.15.5:0/3846940227 shutdown_connections 2024-09-11T22:32:17.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:17.152+0000 7f60b45f8640 1 -- 172.21.15.5:0/3846940227 wait complete. 2024-09-11T22:32:17.337 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.78 MB/sec execute 395 sec latency 291.794 ms 2024-09-11T22:32:17.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:17 smithi005 ceph-mon[30943]: pgmap v318: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 297 op/s 2024-09-11T22:32:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:17 smithi159 ceph-mon[35534]: pgmap v318: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 297 op/s 2024-09-11T22:32:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:17 smithi172 ceph-mon[35461]: pgmap v318: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 297 op/s 2024-09-11T22:32:18.166 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '70839249', quiesce took 3.9 seconds. Will release after: 18.650000000000002 2024-09-11T22:32:18.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.76 MB/sec execute 396 sec latency 1291.908 ms 2024-09-11T22:32:19.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.74 MB/sec execute 397 sec latency 2292.009 ms 2024-09-11T22:32:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:19 smithi005 ceph-mon[30943]: pgmap v319: 97 pgs: 97 active+clean; 100 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-09-11T22:32:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:19 smithi159 ceph-mon[35534]: pgmap v319: 97 pgs: 97 active+clean; 100 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-09-11T22:32:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:19 smithi172 ceph-mon[35461]: pgmap v319: 97 pgs: 97 active+clean; 100 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-09-11T22:32:20.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.72 MB/sec execute 398 sec latency 3292.113 ms 2024-09-11T22:32:21.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.70 MB/sec execute 399 sec latency 4292.216 ms 2024-09-11T22:32:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:21 smithi005 ceph-mon[30943]: pgmap v320: 97 pgs: 97 active+clean; 100 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-09-11T22:32:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:21 smithi159 ceph-mon[35534]: pgmap v320: 97 pgs: 97 active+clean; 100 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-09-11T22:32:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:21 smithi172 ceph-mon[35461]: pgmap v320: 97 pgs: 97 active+clean; 100 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-09-11T22:32:22.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.68 MB/sec execute 400 sec latency 5292.353 ms 2024-09-11T22:32:23.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.66 MB/sec execute 401 sec latency 6292.460 ms 2024-09-11T22:32:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:23 smithi005 ceph-mon[30943]: pgmap v321: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-11T22:32:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:23 smithi159 ceph-mon[35534]: pgmap v321: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-11T22:32:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:23 smithi172 ceph-mon[35461]: pgmap v321: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 134 op/s 2024-09-11T22:32:24.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.64 MB/sec execute 402 sec latency 7292.566 ms 2024-09-11T22:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.188+0000 7fe209cf1640 1 -- 172.21.15.5:0/3894811781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe20410c130 msgr2=0x7fe20410e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.188+0000 7fe209cf1640 1 --2- 172.21.15.5:0/3894811781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe20410c130 0x7fe20410e5a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009ca0 tx=0x7fe1fc02a950 comp rx=0 tx=0).stop 2024-09-11T22:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.188+0000 7fe209cf1640 1 -- 172.21.15.5:0/3894811781 shutdown_connections 2024-09-11T22:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.188+0000 7fe209cf1640 1 --2- 172.21.15.5:0/3894811781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe20410c130 0x7fe20410e5a0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.188+0000 7fe209cf1640 1 --2- 172.21.15.5:0/3894811781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe204103680 0x7fe20410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.188+0000 7fe209cf1640 1 --2- 172.21.15.5:0/3894811781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe204102d60 0x7fe204103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.188+0000 7fe209cf1640 1 -- 172.21.15.5:0/3894811781 >> 172.21.15.5:0/3894811781 conn(0x7fe2040fc6b0 msgr2=0x7fe2040fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.188+0000 7fe209cf1640 1 -- 172.21.15.5:0/3894811781 shutdown_connections 2024-09-11T22:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.188+0000 7fe209cf1640 1 -- 172.21.15.5:0/3894811781 wait complete. 2024-09-11T22:32:25.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.188+0000 7fe209cf1640 1 Processor -- start 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe209cf1640 1 -- start start 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe209cf1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe204102d60 0x7fe204198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe209cf1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204103680 0x7fe2041988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe209cf1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe20410c130 0x7fe20419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe209cf1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe204111f40 con 0x7fe204103680 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe209cf1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe204111dc0 con 0x7fe204102d60 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe209cf1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041120c0 con 0x7fe20410c130 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe202ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204103680 0x7fe2041988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe2037fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe204102d60 0x7fe204198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe202ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204103680 0x7fe2041988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48280/0 (socket says 172.21.15.5:48280) 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe202ffd640 1 -- 172.21.15.5:0/2920272069 learned_addr learned my addr 172.21.15.5:0/2920272069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:25.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe203fff640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe20410c130 0x7fe20419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe202ffd640 1 -- 172.21.15.5:0/2920272069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe20410c130 msgr2=0x7fe20419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe202ffd640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe20410c130 0x7fe20419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe202ffd640 1 -- 172.21.15.5:0/2920272069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe204102d60 msgr2=0x7fe204198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe202ffd640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe204102d60 0x7fe204198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.189+0000 7fe202ffd640 1 -- 172.21.15.5:0/2920272069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20419d380 con 0x7fe204103680 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.190+0000 7fe202ffd640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204103680 0x7fe2041988d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800b4a0 tx=0x7fe1f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.190+0000 7fe200ff9640 1 -- 172.21.15.5:0/2920272069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8014070 con 0x7fe204103680 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.190+0000 7fe200ff9640 1 -- 172.21.15.5:0/2920272069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1f8010af0 con 0x7fe204103680 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.190+0000 7fe209cf1640 1 -- 172.21.15.5:0/2920272069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20419d670 con 0x7fe204103680 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.190+0000 7fe200ff9640 1 -- 172.21.15.5:0/2920272069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8017d10 con 0x7fe204103680 2024-09-11T22:32:25.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.190+0000 7fe209cf1640 1 -- 172.21.15.5:0/2920272069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2041a4f00 con 0x7fe204103680 2024-09-11T22:32:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.192+0000 7fe200ff9640 1 -- 172.21.15.5:0/2920272069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe1f8017070 con 0x7fe204103680 2024-09-11T22:32:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.192+0000 7fe20ca8d740 1 Processor -- start 2024-09-11T22:32:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe20ca8d740 1 -- start start 2024-09-11T22:32:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe200ff9640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1ec0a2da0 0x7fe1ec0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5a9d0 0x5573faa5adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 0x5573faa69b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa6a0c0 0x5573faa6c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573fa9bf2f0 con 0x5573faa5a9d0 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5573fa967270 con 0x5573faa5b2f0 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5573fa9bb2b0 con 0x5573faa6a0c0 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe2037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5a9d0 0x5573faa5adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe203fff640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1ec0a2da0 0x7fe1ec0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe200ff9640 1 -- 172.21.15.5:0/2920272069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8629+0+0 (secure 0 0 0) 0x7fe1f80ce7e0 con 0x7fe204103680 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe2037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5a9d0 0x5573faa5adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48290/0 (socket says 172.21.15.5:48290) 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe2037fe640 1 -- 172.21.15.5:0/1461375143 learned_addr learned my addr 172.21.15.5:0/1461375143 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe202ffd640 1 --2- 172.21.15.5:0/1461375143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 0x5573faa69b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe203fff640 1 --2- 172.21.15.5:0/1461375143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa6a0c0 0x5573faa6c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe2037fe640 1 -- 172.21.15.5:0/1461375143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa6a0c0 msgr2=0x5573faa6c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe2037fe640 1 --2- 172.21.15.5:0/1461375143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa6a0c0 0x5573faa6c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe2037fe640 1 -- 172.21.15.5:0/1461375143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 msgr2=0x5573faa69b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe2037fe640 1 --2- 172.21.15.5:0/1461375143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 0x5573faa69b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe2037fe640 1 -- 172.21.15.5:0/1461375143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573fa966340 con 0x5573faa5a9d0 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe203fff640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1ec0a2da0 0x7fe1ec0a5260 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc02ae70 tx=0x7fe1fc0099f0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.193+0000 7fe2037fe640 1 --2- 172.21.15.5:0/1461375143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5a9d0 0x5573faa5adb0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400ac50 tx=0x7fe1f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1708e8753787abbf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe1e9ffb640 1 -- 172.21.15.5:0/1461375143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f401d070 con 0x5573faa5a9d0 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe1e9ffb640 1 -- 172.21.15.5:0/1461375143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1f4018b20 con 0x5573faa5a9d0 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe1e9ffb640 1 -- 172.21.15.5:0/1461375143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4010b30 con 0x5573faa5a9d0 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe20ca8d740 1 -- 172.21.15.5:0/1461375143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5a9d0 msgr2=0x5573faa5adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/1461375143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5a9d0 0x5573faa5adb0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400ac50 tx=0x7fe1f400f790 comp rx=0 tx=0).stop 2024-09-11T22:32:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe20ca8d740 1 -- 172.21.15.5:0/1461375143 shutdown_connections 2024-09-11T22:32:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/1461375143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa6a0c0 0x5573faa6c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/1461375143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 0x5573faa69b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/1461375143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5a9d0 0x5573faa5adb0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe20ca8d740 1 -- 172.21.15.5:0/1461375143 >> 172.21.15.5:0/1461375143 conn(0x5573faa57060 msgr2=0x5573faa57440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe20ca8d740 1 -- 172.21.15.5:0/1461375143 shutdown_connections 2024-09-11T22:32:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.194+0000 7fe20ca8d740 1 -- 172.21.15.5:0/1461375143 wait complete. 2024-09-11T22:32:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe20ca8d740 1 -- start start 2024-09-11T22:32:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe20ca8d740 1 Processor -- start 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5a9d0 0x5573faa67060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5b2f0 0x5573faa675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa6a0c0 0x5573faa67ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573faa6ccf0 con 0x5573faa5b2f0 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5573fa948290 con 0x5573faa6a0c0 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5573faa6ce70 con 0x5573faa5a9d0 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe20ca8d740 10 client.0 _ll_register_callbacks cb 0x5573fa979140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe202ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5b2f0 0x5573faa675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe202ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5b2f0 0x5573faa675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48306/0 (socket says 172.21.15.5:48306) 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.195+0000 7fe202ffd640 1 -- 172.21.15.5:0/494638517 learned_addr learned my addr 172.21.15.5:0/494638517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe203fff640 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa6a0c0 0x5573faa67ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe2037fe640 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5a9d0 0x5573faa67060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe202ffd640 1 -- 172.21.15.5:0/494638517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5a9d0 msgr2=0x5573faa67060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe202ffd640 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5a9d0 0x5573faa67060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe202ffd640 1 -- 172.21.15.5:0/494638517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa6a0c0 msgr2=0x5573faa67ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe202ffd640 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa6a0c0 0x5573faa67ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe202ffd640 1 -- 172.21.15.5:0/494638517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573fa6d2210 con 0x5573faa5b2f0 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe202ffd640 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5b2f0 0x5573faa675a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80971a0 tx=0x7fe1f809ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86507b48825c11c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe1e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/494638517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f801e040 con 0x5573faa5b2f0 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/494638517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8629+0+0 (secure 0 0 0) 0x7fe1f80973b0 con 0x5573faa5b2f0 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe20ca8d740 1 -- 172.21.15.5:0/494638517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1f4018b20 con 0x5573faa5b2f0 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/494638517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe1f8078180 con 0x5573faa5b2f0 2024-09-11T22:32:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe20ca8d740 10 client.18606 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe20ca8d740 1 -- 172.21.15.5:0/494638517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5573fa79e620 con 0x5573faa5b2f0 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.196+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/494638517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1f4018b20 con 0x5573faa5b2f0 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.197+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/494638517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fe1f80b1080 con 0x5573faa5b2f0 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.197+0000 7fe20ca8d740 10 client.18606 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.197+0000 7fe20ca8d740 10 client.18606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.197+0000 7fe20ca8d740 20 client.18606 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.197+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573faa7a120 0x5573faa7c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.197+0000 7fe20ca8d740 4 client.18606 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.197+0000 7fe20ca8d740 1 -- 172.21.15.5:0/494638517 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573fa9cca20 con 0x5573faa7a120 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.197+0000 7fe2037fe640 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573faa7a120 0x5573faa7c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.197+0000 7fe2037fe640 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573faa7a120 0x5573faa7c530 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d033fef8ca848508 server_cookie=12e8a8f685b74fcd in_seq=0 out_seq=0 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.197+0000 7fe1e2ffd640 10 client.18606 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/494638517 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573fa9cca20 con 0x5573faa7a120 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe1e2ffd640 10 client.18606 handle_command_reply: tid=0 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 client.18606 shutdown 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 -- 172.21.15.5:0/494638517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5b2f0 msgr2=0x5573faa675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5b2f0 0x5573faa675a0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80971a0 tx=0x7fe1f809ac40 comp rx=0 tx=0).stop 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 -- 172.21.15.5:0/494638517 shutdown_connections 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573faa7a120 0x5573faa7c530 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa6a0c0 0x5573faa67ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa5b2f0 0x5573faa675a0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/494638517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5a9d0 0x5573faa67060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe1e2ffd640 0 client.18606 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 -- 172.21.15.5:0/494638517 >> 172.21.15.5:0/494638517 conn(0x5573faa57060 msgr2=0x5573faa5cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 -- 172.21.15.5:0/494638517 shutdown_connections 2024-09-11T22:32:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 1 -- 172.21.15.5:0/494638517 wait complete. 2024-09-11T22:32:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.198+0000 7fe20ca8d740 20 client.18606 trim_cache size 0 max 16384 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe20ca8d740 1 Processor -- start 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe20ca8d740 1 -- start start 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5a9d0 0x5573faa68fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 0x5573faa81e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa82380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041ab7c0 con 0x5573faa6a0c0 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041ad300 con 0x5573faa5b2f0 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041abe90 con 0x5573faa5a9d0 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe203fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa82380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe203fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa82380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48322/0 (socket says 172.21.15.5:48322) 2024-09-11T22:32:25.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe203fff640 1 -- 172.21.15.5:0/2544745004 learned_addr learned my addr 172.21.15.5:0/2544745004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe2037fe640 1 --2- 172.21.15.5:0/2544745004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5a9d0 0x5573faa68fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe202ffd640 1 --2- 172.21.15.5:0/2544745004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 0x5573faa81e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe203fff640 1 -- 172.21.15.5:0/2544745004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5a9d0 msgr2=0x5573faa68fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe203fff640 1 --2- 172.21.15.5:0/2544745004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5a9d0 0x5573faa68fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe203fff640 1 -- 172.21.15.5:0/2544745004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 msgr2=0x5573faa81e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe203fff640 1 --2- 172.21.15.5:0/2544745004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 0x5573faa81e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe203fff640 1 -- 172.21.15.5:0/2544745004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573faa56d30 con 0x5573faa6a0c0 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe202ffd640 1 --2- 172.21.15.5:0/2544745004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 0x5573faa81e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.213+0000 7fe203fff640 1 --2- 172.21.15.5:0/2544745004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa82380 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc035000 tx=0x7fe1fc009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f0e92c3ef594ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe1e97fa640 1 -- 172.21.15.5:0/2544745004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc04d070 con 0x5573faa6a0c0 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe1e97fa640 1 -- 172.21.15.5:0/2544745004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1fc003e50 con 0x5573faa6a0c0 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe1e97fa640 1 -- 172.21.15.5:0/2544745004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc034040 con 0x5573faa6a0c0 2024-09-11T22:32:25.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2544745004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 msgr2=0x5573faa82380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/2544745004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa82380 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc035000 tx=0x7fe1fc009820 comp rx=0 tx=0).stop 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2544745004 shutdown_connections 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/2544745004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa82380 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/2544745004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5b2f0 0x5573faa81e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/2544745004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5a9d0 0x5573faa68fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2544745004 >> 172.21.15.5:0/2544745004 conn(0x5573faa57060 msgr2=0x5573faa688c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2544745004 shutdown_connections 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2544745004 wait complete. 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.214+0000 7fe20ca8d740 1 -- start start 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe20ca8d740 1 Processor -- start 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5a9d0 0x5573faa720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5b2f0 0x5573faa72600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe20ca8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa6f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041acc30 con 0x5573faa6a0c0 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041ac560 con 0x5573faa5a9d0 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe20ca8d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041a97c0 con 0x5573faa5b2f0 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe20ca8d740 10 client.0 _ll_register_callbacks cb 0x7fe1f800b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe203fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa6f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe203fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa6f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48330/0 (socket says 172.21.15.5:48330) 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe2037fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5a9d0 0x5573faa720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe203fff640 1 -- 172.21.15.5:0/2925697073 learned_addr learned my addr 172.21.15.5:0/2925697073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe202ffd640 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5b2f0 0x5573faa72600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe203fff640 1 -- 172.21.15.5:0/2925697073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5b2f0 msgr2=0x5573faa72600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe203fff640 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5b2f0 0x5573faa72600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe203fff640 1 -- 172.21.15.5:0/2925697073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5a9d0 msgr2=0x5573faa720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe203fff640 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5a9d0 0x5573faa720c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe203fff640 1 -- 172.21.15.5:0/2925697073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573faa72cf0 con 0x5573faa6a0c0 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.215+0000 7fe2037fe640 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5a9d0 0x5573faa720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe203fff640 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa6f090 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc005fc0 tx=0x7fe1fc02fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6233848abfe681f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe1e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/2925697073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc056020 con 0x5573faa6a0c0 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/2925697073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8629+0+0 (secure 0 0 0) 0x7fe1fc004030 con 0x5573faa6a0c0 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2925697073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1fc003e50 con 0x5573faa6a0c0 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/2925697073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fe1fc061050 con 0x5573faa6a0c0 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe20ca8d740 10 client.18612 fetch_fsmap learned FSMap version 42 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2925697073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5573faae5430 con 0x5573faa6a0c0 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/2925697073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1fc003e50 con 0x5573faa6a0c0 2024-09-11T22:32:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/2925697073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fe1fc052450 con 0x5573faa6a0c0 2024-09-11T22:32:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe20ca8d740 10 client.18612 fetch_fsmap finished waiting for FSMap version 42 2024-09-11T22:32:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe20ca8d740 10 client.18612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe20ca8d740 20 client.18612 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573faae5c30 0x5573faae8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe20ca8d740 4 client.18612 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:32:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.216+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2925697073 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573fa9c6d70 con 0x5573faae5c30 2024-09-11T22:32:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.217+0000 7fe2037fe640 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573faae5c30 0x5573faae8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.217+0000 7fe2037fe640 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573faae5c30 0x5573faae8050 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4e0880faeae73d server_cookie=99be4c7f5229c49a in_seq=0 out_seq=0 2024-09-11T22:32:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.217+0000 7fe1e2ffd640 10 client.18612 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe1e2ffd640 1 -- 172.21.15.5:0/2925697073 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5573fa9c6d70 con 0x5573faae5c30 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe1e2ffd640 10 client.18612 handle_command_reply: tid=0 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 client.18612 shutdown 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2925697073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 msgr2=0x5573faa6f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa6f090 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc005fc0 tx=0x7fe1fc02fef0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2925697073 shutdown_connections 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5573faae5c30 0x5573faae8050 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573faa6a0c0 0x5573faa6f090 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5573faa5b2f0 0x5573faa72600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 --2- 172.21.15.5:0/2925697073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5573faa5a9d0 0x5573faa720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe1e2ffd640 0 client.18612 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2925697073 >> 172.21.15.5:0/2925697073 conn(0x5573faa57060 msgr2=0x5573faa67840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2925697073 shutdown_connections 2024-09-11T22:32:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.308+0000 7fe20ca8d740 1 -- 172.21.15.5:0/2925697073 wait complete. 2024-09-11T22:32:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe20ca8d740 20 client.18612 trim_cache size 0 max 16384 2024-09-11T22:32:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:32:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:32:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:32:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 -- 172.21.15.5:0/2920272069 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1ec0a2da0 msgr2=0x7fe1ec0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1ec0a2da0 0x7fe1ec0a5260 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc02ae70 tx=0x7fe1fc0099f0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 -- 172.21.15.5:0/2920272069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204103680 msgr2=0x7fe2041988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204103680 0x7fe2041988d0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800b4a0 tx=0x7fe1f800b960 comp rx=0 tx=0).stop 2024-09-11T22:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 -- 172.21.15.5:0/2920272069 shutdown_connections 2024-09-11T22:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe20410c130 0x7fe20419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204103680 0x7fe2041988d0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe1ec0a2da0 0x7fe1ec0a5260 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 --2- 172.21.15.5:0/2920272069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe204102d60 0x7fe204198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 -- 172.21.15.5:0/2920272069 >> 172.21.15.5:0/2920272069 conn(0x7fe2040fc6b0 msgr2=0x7fe2040fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 -- 172.21.15.5:0/2920272069 shutdown_connections 2024-09-11T22:32:25.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:25.309+0000 7fe209cf1640 1 -- 172.21.15.5:0/2920272069 wait complete. 2024-09-11T22:32:25.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.63 MB/sec execute 403 sec latency 8292.683 ms 2024-09-11T22:32:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:25 smithi005 ceph-mon[30943]: pgmap v322: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 115 op/s 2024-09-11T22:32:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:25 smithi159 ceph-mon[35534]: pgmap v322: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 115 op/s 2024-09-11T22:32:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:25 smithi172 ceph-mon[35461]: pgmap v322: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 115 op/s 2024-09-11T22:32:26.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.61 MB/sec execute 404 sec latency 9292.806 ms 2024-09-11T22:32:26.493 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:32:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setuser ceph since I am not root 2024-09-11T22:32:26.494 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:32:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setgroup ceph since I am not root 2024-09-11T22:32:26.494 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:32:26 smithi159 ceph-mds[65034]: -- 172.21.15.159:0/2670613920 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c54e1fc20 con 0x555c54dfd400 2024-09-11T22:32:26.494 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:32:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: starting mds.k at 2024-09-11T22:32:26.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:26 smithi005 ceph-mon[30943]: daemon mds.k restarted 2024-09-11T22:32:26.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:26 smithi159 ceph-mon[35534]: daemon mds.k restarted 2024-09-11T22:32:26.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:26 smithi172 ceph-mon[35461]: daemon mds.k restarted 2024-09-11T22:32:27.339 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.59 MB/sec execute 405 sec latency 10292.903 ms 2024-09-11T22:32:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:27 smithi005 ceph-mon[30943]: pgmap v323: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 112 op/s 2024-09-11T22:32:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:27 smithi005 ceph-mon[30943]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T22:32:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:27 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6832/1984462744,v1:172.21.15.159:6833/1984462744] up:boot 2024-09-11T22:32:27.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:27 smithi005 ceph-mon[30943]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:32:27.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:27 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:32:27.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:27 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:32:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:27 smithi159 ceph-mon[35534]: pgmap v323: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 112 op/s 2024-09-11T22:32:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:27 smithi159 ceph-mon[35534]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T22:32:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:27 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6832/1984462744,v1:172.21.15.159:6833/1984462744] up:boot 2024-09-11T22:32:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:27 smithi159 ceph-mon[35534]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:32:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:27 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:32:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:27 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:32:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:27 smithi172 ceph-mon[35461]: pgmap v323: 97 pgs: 97 active+clean; 108 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 112 op/s 2024-09-11T22:32:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:27 smithi172 ceph-mon[35461]: osdmap e99: 12 total, 12 up, 12 in 2024-09-11T22:32:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:27 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6832/1984462744,v1:172.21.15.159:6833/1984462744] up:boot 2024-09-11T22:32:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:27 smithi172 ceph-mon[35461]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:32:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:27 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:32:27.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:27 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:32:28.338 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.57 MB/sec execute 406 sec latency 11292.957 ms 2024-09-11T22:32:29.339 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.55 MB/sec execute 407 sec latency 12293.063 ms 2024-09-11T22:32:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:29 smithi005 ceph-mon[30943]: pgmap v325: 97 pgs: 97 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-11T22:32:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:29 smithi159 ceph-mon[35534]: pgmap v325: 97 pgs: 97 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-11T22:32:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:29 smithi172 ceph-mon[35461]: pgmap v325: 97 pgs: 97 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-11T22:32:30.339 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.53 MB/sec execute 408 sec latency 13293.186 ms 2024-09-11T22:32:31.339 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.51 MB/sec execute 409 sec latency 14293.280 ms 2024-09-11T22:32:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:31 smithi005 ceph-mon[30943]: pgmap v326: 97 pgs: 97 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-11T22:32:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:31 smithi159 ceph-mon[35534]: pgmap v326: 97 pgs: 97 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-11T22:32:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:31 smithi172 ceph-mon[35461]: pgmap v326: 97 pgs: 97 active+clean; 94 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-11T22:32:32.339 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.50 MB/sec execute 410 sec latency 15293.407 ms 2024-09-11T22:32:32.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:32 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:32:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:32 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:32:32.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:32 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:32:33.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.48 MB/sec execute 411 sec latency 16293.515 ms 2024-09-11T22:32:33.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:33 smithi172 ceph-mon[35461]: pgmap v327: 97 pgs: 97 active+clean; 96 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-11T22:32:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:33 smithi005 ceph-mon[30943]: pgmap v327: 97 pgs: 97 active+clean; 96 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-11T22:32:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:33 smithi159 ceph-mon[35534]: pgmap v327: 97 pgs: 97 active+clean; 96 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-11T22:32:34.339 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.46 MB/sec execute 412 sec latency 17293.576 ms 2024-09-11T22:32:35.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:32:35.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:32:35.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:32:35.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.339 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.44 MB/sec execute 413 sec latency 18293.694 ms 2024-09-11T22:32:35.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:35.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:32:35.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:32:35.362 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:32:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:35 smithi005 ceph-mon[30943]: pgmap v328: 97 pgs: 97 active+clean; 96 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-11T22:32:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:35 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-11T22:32:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:35 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-11T22:32:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:35 smithi005 ceph-mon[30943]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-11T22:32:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:35 smithi005 ceph-mon[30943]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T22:32:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:35 smithi005 ceph-mon[30943]: Unable to set osd_memory_target on smithi172 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T22:32:36.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:35 smithi005 ceph-mon[30943]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T22:32:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:35 smithi159 ceph-mon[35534]: pgmap v328: 97 pgs: 97 active+clean; 96 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-11T22:32:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:35 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-11T22:32:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:35 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-11T22:32:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:35 smithi159 ceph-mon[35534]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-11T22:32:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:35 smithi159 ceph-mon[35534]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T22:32:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:35 smithi159 ceph-mon[35534]: Unable to set osd_memory_target on smithi172 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T22:32:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:35 smithi159 ceph-mon[35534]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T22:32:36.339 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.42 MB/sec execute 414 sec latency 19293.817 ms 2024-09-11T22:32:36.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:35 smithi172 ceph-mon[35461]: pgmap v328: 97 pgs: 97 active+clean; 96 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-11T22:32:36.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:35 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-11T22:32:36.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:35 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi172 to 176.0M 2024-09-11T22:32:36.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:35 smithi172 ceph-mon[35461]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-11T22:32:36.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:35 smithi172 ceph-mon[35461]: Unable to set osd_memory_target on smithi005 to 185689190: error parsing value: Value '185689190' is below minimum 939524096 2024-09-11T22:32:36.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:35 smithi172 ceph-mon[35461]: Unable to set osd_memory_target on smithi172 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-11T22:32:36.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:35 smithi172 ceph-mon[35461]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-11T22:32:36.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:32:37.089 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:37.339 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.41 MB/sec execute 415 sec latency 20293.936 ms 2024-09-11T22:32:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.605+0000 7f8325950640 1 -- 172.21.15.5:0/2608635144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320102dd0 msgr2=0x7f83201031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.605+0000 7f8325950640 1 --2- 172.21.15.5:0/2608635144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320102dd0 0x7f83201031b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8310009cd0 tx=0x7f831002a8d0 comp rx=0 tx=0).stop 2024-09-11T22:32:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.605+0000 7f8325950640 1 -- 172.21.15.5:0/2608635144 shutdown_connections 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.605+0000 7f8325950640 1 --2- 172.21.15.5:0/2608635144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8320108300 0x7f832010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.605+0000 7f8325950640 1 --2- 172.21.15.5:0/2608635144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8320103780 0x7f8320107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.605+0000 7f8325950640 1 --2- 172.21.15.5:0/2608635144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320102dd0 0x7f83201031b0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.605+0000 7f8325950640 1 -- 172.21.15.5:0/2608635144 >> 172.21.15.5:0/2608635144 conn(0x7f83200fc7d0 msgr2=0x7f83200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.605+0000 7f8325950640 1 -- 172.21.15.5:0/2608635144 shutdown_connections 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.605+0000 7f8325950640 1 -- 172.21.15.5:0/2608635144 wait complete. 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f8325950640 1 Processor -- start 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f8325950640 1 -- start start 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f8325950640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8320102dd0 0x7f8320100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f8325950640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8320103780 0x7f8320100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f8325950640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320108300 0x7f83201014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f8325950640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83201120b0 con 0x7f8320108300 2024-09-11T22:32:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f8325950640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8320111f30 con 0x7f8320102dd0 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f8325950640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8320112230 con 0x7f8320103780 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320108300 0x7f83201014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f831f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320108300 0x7f83201014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55982/0 (socket says 172.21.15.5:55982) 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f831f7fe640 1 -- 172.21.15.5:0/391206331 learned_addr learned my addr 172.21.15.5:0/391206331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f831effd640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8320102dd0 0x7f8320100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.606+0000 7f831e7fc640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8320103780 0x7f8320100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f831f7fe640 1 -- 172.21.15.5:0/391206331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8320103780 msgr2=0x7f8320100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f831f7fe640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8320103780 0x7f8320100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f831f7fe640 1 -- 172.21.15.5:0/391206331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8320102dd0 msgr2=0x7f8320100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f831f7fe640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8320102dd0 0x7f8320100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f831f7fe640 1 -- 172.21.15.5:0/391206331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83201a6b30 con 0x7f8320108300 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f831effd640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8320102dd0 0x7f8320100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f831f7fe640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320108300 0x7f83201014b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f831800d750 tx=0x7f831800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f832494e640 1 -- 172.21.15.5:0/391206331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318017070 con 0x7f8320108300 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f832494e640 1 -- 172.21.15.5:0/391206331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8318005500 con 0x7f8320108300 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f8325950640 1 -- 172.21.15.5:0/391206331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83201a6e20 con 0x7f8320108300 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f832494e640 1 -- 172.21.15.5:0/391206331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318004d90 con 0x7f8320108300 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.607+0000 7f8325950640 1 -- 172.21.15.5:0/391206331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83201a7380 con 0x7f8320108300 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.608+0000 7f8325950640 1 -- 172.21.15.5:0/391206331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f832006a7b0 con 0x7f8320108300 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.609+0000 7f832494e640 1 -- 172.21.15.5:0/391206331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8318004f30 con 0x7f8320108300 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.610+0000 7f832494e640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83080a2d70 0x7f83080a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.610+0000 7f831effd640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83080a2d70 0x7f83080a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.610+0000 7f832494e640 1 -- 172.21.15.5:0/391206331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f83180cf120 con 0x7f8320108300 2024-09-11T22:32:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.611+0000 7f831effd640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83080a2d70 0x7f83080a5230 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8310000f80 tx=0x7f8310009360 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.613+0000 7f832494e640 1 -- 172.21.15.5:0/391206331 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f831809bd80 con 0x7f8320108300 2024-09-11T22:32:37.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.829+0000 7f8325950640 1 -- 172.21.15.5:0/391206331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f832010a290 con 0x7f8320108300 2024-09-11T22:32:37.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.829+0000 7f832494e640 1 -- 172.21.15.5:0/391206331 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11967 (secure 0 0 0) 0x7f831809b680 con 0x7f8320108300 2024-09-11T22:32:37.830 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:32:37.830 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":44,"btime":"2024-09-11T22:32:26:501017+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:32:26.500999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27887":{"gid":27887,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1984462744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1984462744},{"type":"v1","addr":"172.21.15.159:6833","nonce":1984462744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:32:37.830 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 44 2024-09-11T22:32:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 -- 172.21.15.5:0/391206331 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83080a2d70 msgr2=0x7f83080a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83080a2d70 0x7f83080a5230 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8310000f80 tx=0x7f8310009360 comp rx=0 tx=0).stop 2024-09-11T22:32:37.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 -- 172.21.15.5:0/391206331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320108300 msgr2=0x7f83201014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320108300 0x7f83201014b0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f831800d750 tx=0x7f831800dc10 comp rx=0 tx=0).stop 2024-09-11T22:32:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 -- 172.21.15.5:0/391206331 shutdown_connections 2024-09-11T22:32:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8320108300 0x7f83201014b0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8320103780 0x7f8320100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83080a2d70 0x7f83080a5230 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 --2- 172.21.15.5:0/391206331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8320102dd0 0x7f8320100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 -- 172.21.15.5:0/391206331 >> 172.21.15.5:0/391206331 conn(0x7f83200fc7d0 msgr2=0x7f8320105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.831+0000 7f8325950640 1 -- 172.21.15.5:0/391206331 shutdown_connections 2024-09-11T22:32:37.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:37.832+0000 7f8325950640 1 -- 172.21.15.5:0/391206331 wait complete. 2024-09-11T22:32:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:37 smithi005 ceph-mon[30943]: pgmap v329: 97 pgs: 97 active+clean; 96 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-11T22:32:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:37 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/391206331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:38.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:37 smithi159 ceph-mon[35534]: pgmap v329: 97 pgs: 97 active+clean; 96 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-11T22:32:38.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:37 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/391206331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:38.339 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.39 MB/sec execute 416 sec latency 21293.993 ms 2024-09-11T22:32:38.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:37 smithi172 ceph-mon[35461]: pgmap v329: 97 pgs: 97 active+clean; 96 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-09-11T22:32:38.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:37 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/391206331' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.402+0000 7f38a78a5640 1 -- 172.21.15.5:0/3190368594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38a0101210 msgr2=0x7f38a010ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.402+0000 7f38a78a5640 1 --2- 172.21.15.5:0/3190368594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38a0101210 0x7f38a010ded0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f3890009900 tx=0x7f389002a410 comp rx=0 tx=0).stop 2024-09-11T22:32:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.402+0000 7f38a78a5640 1 -- 172.21.15.5:0/3190368594 shutdown_connections 2024-09-11T22:32:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.402+0000 7f38a78a5640 1 --2- 172.21.15.5:0/3190368594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a010e410 0x7f38a0110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.402+0000 7f38a78a5640 1 --2- 172.21.15.5:0/3190368594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38a0101210 0x7f38a010ded0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.402+0000 7f38a78a5640 1 --2- 172.21.15.5:0/3190368594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a0100860 0x7f38a0100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.402+0000 7f38a78a5640 1 -- 172.21.15.5:0/3190368594 >> 172.21.15.5:0/3190368594 conn(0x7f38a0077990 msgr2=0x7f38a00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.402+0000 7f38a78a5640 1 -- 172.21.15.5:0/3190368594 shutdown_connections 2024-09-11T22:32:38.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.402+0000 7f38a78a5640 1 -- 172.21.15.5:0/3190368594 wait complete. 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a78a5640 1 Processor -- start 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a78a5640 1 -- start start 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a78a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38a0100860 0x7f38a019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a78a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a0101210 0x7f38a019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a78a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a010e410 0x7f38a019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a78a5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0114060 con 0x7f38a0100860 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a78a5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0113ee0 con 0x7f38a0101210 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a78a5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01141e0 con 0x7f38a010e410 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a5641640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38a0100860 0x7f38a019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a5641640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38a0100860 0x7f38a019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56000/0 (socket says 172.21.15.5:56000) 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.403+0000 7f38a5641640 1 -- 172.21.15.5:0/344494942 learned_addr learned my addr 172.21.15.5:0/344494942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:38.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38a4e40640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a0101210 0x7f38a019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38a5641640 1 -- 172.21.15.5:0/344494942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a010e410 msgr2=0x7f38a019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38a5641640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a010e410 0x7f38a019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38a5641640 1 -- 172.21.15.5:0/344494942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a0101210 msgr2=0x7f38a019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38a5641640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a0101210 0x7f38a019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38a5641640 1 -- 172.21.15.5:0/344494942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a019f550 con 0x7f38a0100860 2024-09-11T22:32:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38a4e40640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a0101210 0x7f38a019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:38.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38a5641640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38a0100860 0x7f38a019a560 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f389c00d950 tx=0x7f389c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38967fc640 1 -- 172.21.15.5:0/344494942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c017070 con 0x7f38a0100860 2024-09-11T22:32:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38967fc640 1 -- 172.21.15.5:0/344494942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c009c30 con 0x7f38a0100860 2024-09-11T22:32:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38a78a5640 1 -- 172.21.15.5:0/344494942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a019f840 con 0x7f38a0100860 2024-09-11T22:32:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38967fc640 1 -- 172.21.15.5:0/344494942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c00f6b0 con 0x7f38a0100860 2024-09-11T22:32:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.404+0000 7f38a78a5640 1 -- 172.21.15.5:0/344494942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a01a70d0 con 0x7f38a0100860 2024-09-11T22:32:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.406+0000 7f38967fc640 1 -- 172.21.15.5:0/344494942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f389c0109d0 con 0x7f38a0100860 2024-09-11T22:32:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38aa647740 1 Processor -- start 2024-09-11T22:32:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38aa647740 1 -- start start 2024-09-11T22:32:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38aa647740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 0x5575c03f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38aa647740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38aa647740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03fdcb0 0x5575c04000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38aa647740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5575c03572f0 con 0x5575c03f3410 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38aa647740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5575c02ff270 con 0x5575c03f2af0 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38aa647740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5575c03532b0 con 0x5575c03fdcb0 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a5641640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a5641640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56006/0 (socket says 172.21.15.5:56006) 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a5641640 1 -- 172.21.15.5:0/3703394577 learned_addr learned my addr 172.21.15.5:0/3703394577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38967fc640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f388c0a2da0 0x7f388c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a5e42640 1 --2- 172.21.15.5:0/3703394577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 0x5575c03f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a4e40640 1 --2- 172.21.15.5:0/3703394577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03fdcb0 0x5575c04000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a5e42640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f388c0a2da0 0x7f388c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38967fc640 1 -- 172.21.15.5:0/344494942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f389c0cf1f0 con 0x7f38a0100860 2024-09-11T22:32:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a5641640 1 -- 172.21.15.5:0/3703394577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03fdcb0 msgr2=0x5575c04000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a5641640 1 --2- 172.21.15.5:0/3703394577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03fdcb0 0x5575c04000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a5641640 1 -- 172.21.15.5:0/3703394577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 msgr2=0x5575c03f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a5641640 1 --2- 172.21.15.5:0/3703394577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 0x5575c03f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.407+0000 7f38a5641640 1 -- 172.21.15.5:0/3703394577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575c02fe340 con 0x5575c03f3410 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.408+0000 7f38a5641640 1 --2- 172.21.15.5:0/3703394577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fd770 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f389c09b370 tx=0x7f389c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e4dbb80e997608 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.408+0000 7f38a5e42640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f388c0a2da0 0x7f388c0a5260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3898005c40 tx=0x7f3898009200 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.408+0000 7f388b7fe640 1 -- 172.21.15.5:0/3703394577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c020070 con 0x5575c03f3410 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.408+0000 7f388b7fe640 1 -- 172.21.15.5:0/3703394577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c009de0 con 0x5575c03f3410 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.408+0000 7f388b7fe640 1 -- 172.21.15.5:0/3703394577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c0973c0 con 0x5575c03f3410 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 -- 172.21.15.5:0/3703394577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 msgr2=0x5575c03fd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 --2- 172.21.15.5:0/3703394577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fd770 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f389c09b370 tx=0x7f389c00bf60 comp rx=0 tx=0).stop 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 -- 172.21.15.5:0/3703394577 shutdown_connections 2024-09-11T22:32:38.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 --2- 172.21.15.5:0/3703394577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03fdcb0 0x5575c04000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 --2- 172.21.15.5:0/3703394577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fd770 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 --2- 172.21.15.5:0/3703394577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 0x5575c03f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 -- 172.21.15.5:0/3703394577 >> 172.21.15.5:0/3703394577 conn(0x5575c03ef060 msgr2=0x5575c03ef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 -- 172.21.15.5:0/3703394577 shutdown_connections 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 -- 172.21.15.5:0/3703394577 wait complete. 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 -- start start 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 Processor -- start 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 0x5575c03fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03fdcb0 0x5575c03fb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5575c0400880 con 0x5575c03f3410 2024-09-11T22:32:38.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5575c02e0290 con 0x5575c03f2af0 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.409+0000 7f38aa647740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5575c0400a00 con 0x5575c03fdcb0 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38aa647740 10 client.0 _ll_register_callbacks cb 0x5575c0311140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a4e40640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a4e40640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56014/0 (socket says 172.21.15.5:56014) 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a4e40640 1 -- 172.21.15.5:0/810880642 learned_addr learned my addr 172.21.15.5:0/810880642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a5641640 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 0x5575c03fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a4e40640 1 -- 172.21.15.5:0/810880642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03fdcb0 msgr2=0x5575c03fb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a4e40640 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03fdcb0 0x5575c03fb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a4e40640 1 -- 172.21.15.5:0/810880642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 msgr2=0x5575c03fabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a4e40640 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 0x5575c03fabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a4e40640 1 -- 172.21.15.5:0/810880642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575c006a210 con 0x5575c03f3410 2024-09-11T22:32:38.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a5641640 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 0x5575c03fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38a4e40640 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fb130 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3890005ab0 tx=0x7f3890031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e65b176d367107 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f3888ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f3888ff9640 1 -- 172.21.15.5:0/810880642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389003f070 con 0x5575c03f3410 2024-09-11T22:32:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f3888ff9640 1 -- 172.21.15.5:0/810880642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f3890004450 con 0x5575c03f3410 2024-09-11T22:32:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38aa647740 1 -- 172.21.15.5:0/810880642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389c009de0 con 0x5575c03f3410 2024-09-11T22:32:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f3888ff9640 1 -- 172.21.15.5:0/810880642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3890035070 con 0x5575c03f3410 2024-09-11T22:32:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38aa647740 10 client.18636 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.410+0000 7f38aa647740 1 -- 172.21.15.5:0/810880642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5575c0136620 con 0x5575c03f3410 2024-09-11T22:32:38.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.411+0000 7f3888ff9640 1 -- 172.21.15.5:0/810880642 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c009de0 con 0x5575c03f3410 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.411+0000 7f3888ff9640 1 -- 172.21.15.5:0/810880642 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f38900035f0 con 0x5575c03f3410 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.411+0000 7f38aa647740 10 client.18636 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.411+0000 7f38aa647740 10 client.18636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.411+0000 7f38aa647740 20 client.18636 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.411+0000 7f38aa647740 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5575c040dc50 0x5575c04100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.411+0000 7f38aa647740 4 client.18636 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.411+0000 7f38aa647740 1 -- 172.21.15.5:0/810880642 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575c0364a20 con 0x5575c040dc50 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.411+0000 7f38a5641640 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5575c040dc50 0x5575c04100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f3888ff9640 10 client.18636 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f38a5641640 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5575c040dc50 0x5575c04100e0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb988023e3e0b1f server_cookie=7387361895c5eb54 in_seq=0 out_seq=0 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f3888ff9640 1 -- 172.21.15.5:0/810880642 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575c0364a20 con 0x5575c040dc50 2024-09-11T22:32:38.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f3888ff9640 10 client.18636 handle_command_reply: tid=0 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f38aa647740 1 client.18636 shutdown 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f38aa647740 1 -- 172.21.15.5:0/810880642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 msgr2=0x5575c03fb130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f38aa647740 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fb130 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3890005ab0 tx=0x7f3890031860 comp rx=0 tx=0).stop 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f38aa647740 1 -- 172.21.15.5:0/810880642 shutdown_connections 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f38aa647740 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5575c040dc50 0x5575c04100e0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f38aa647740 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03fdcb0 0x5575c03fb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f3888ff9640 0 client.18636 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f38aa647740 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03fb130 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f38aa647740 1 --2- 172.21.15.5:0/810880642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f2af0 0x5575c03fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.412+0000 7f38aa647740 1 -- 172.21.15.5:0/810880642 >> 172.21.15.5:0/810880642 conn(0x5575c03ef060 msgr2=0x5575c03f4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.413+0000 7f38aa647740 1 -- 172.21.15.5:0/810880642 shutdown_connections 2024-09-11T22:32:38.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.413+0000 7f38aa647740 1 -- 172.21.15.5:0/810880642 wait complete. 2024-09-11T22:32:38.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.413+0000 7f38aa647740 20 client.18636 trim_cache size 0 max 16384 2024-09-11T22:32:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38aa647740 1 Processor -- start 2024-09-11T22:32:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38aa647740 1 -- start start 2024-09-11T22:32:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38aa647740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 0x5575c03fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38aa647740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f3410 0x5575c03fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38aa647740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03fdcb0 0x5575c04159f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38aa647740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01afc20 con 0x5575c03fdcb0 2024-09-11T22:32:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38aa647740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b1760 con 0x5575c03f3410 2024-09-11T22:32:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38aa647740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b02f0 con 0x5575c03f2af0 2024-09-11T22:32:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a5e42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03fdcb0 0x5575c04159f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a5e42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03fdcb0 0x5575c04159f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56024/0 (socket says 172.21.15.5:56024) 2024-09-11T22:32:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a5e42640 1 -- 172.21.15.5:0/534540361 learned_addr learned my addr 172.21.15.5:0/534540361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a4e40640 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f3410 0x5575c03fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a5e42640 1 -- 172.21.15.5:0/534540361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 msgr2=0x5575c03fb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a5641640 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 0x5575c03fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a5e42640 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 0x5575c03fb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a5e42640 1 -- 172.21.15.5:0/534540361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f3410 msgr2=0x5575c03fcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a5e42640 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f3410 0x5575c03fcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a5e42640 1 -- 172.21.15.5:0/534540361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575c03f3240 con 0x5575c03fdcb0 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a4e40640 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f3410 0x5575c03fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.428+0000 7f38a5641640 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 0x5575c03fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f38a5e42640 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03fdcb0 0x5575c04159f0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f38980099e0 tx=0x7f38980095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e9df1d16c8bcc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f388affd640 1 -- 172.21.15.5:0/534540361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898028680 con 0x5575c03fdcb0 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f388affd640 1 -- 172.21.15.5:0/534540361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389800fe50 con 0x5575c03fdcb0 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f388affd640 1 -- 172.21.15.5:0/534540361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389801cc10 con 0x5575c03fdcb0 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f38aa647740 1 -- 172.21.15.5:0/534540361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03fdcb0 msgr2=0x5575c04159f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f38aa647740 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03fdcb0 0x5575c04159f0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f38980099e0 tx=0x7f38980095c0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f38aa647740 1 -- 172.21.15.5:0/534540361 shutdown_connections 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f38aa647740 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03fdcb0 0x5575c04159f0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f38aa647740 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03f3410 0x5575c03fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f38aa647740 1 --2- 172.21.15.5:0/534540361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 0x5575c03fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.429+0000 7f38aa647740 1 -- 172.21.15.5:0/534540361 >> 172.21.15.5:0/534540361 conn(0x5575c03ef060 msgr2=0x5575c03fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 1 -- 172.21.15.5:0/534540361 shutdown_connections 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 1 -- 172.21.15.5:0/534540361 wait complete. 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 1 -- start start 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 1 Processor -- start 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 0x5575c0406240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03f7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03fdcb0 0x5575c03f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b1090 con 0x5575c03f3410 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01b09c0 con 0x5575c03fdcb0 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01adc00 con 0x5575c03f2af0 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38aa647740 10 client.0 _ll_register_callbacks cb 0x7f3890009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38a4e40640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38a4e40640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03f7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56032/0 (socket says 172.21.15.5:56032) 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.430+0000 7f38a4e40640 1 -- 172.21.15.5:0/308381130 learned_addr learned my addr 172.21.15.5:0/308381130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38a5641640 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 0x5575c0406240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38a5e42640 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03fdcb0 0x5575c03f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38a4e40640 1 -- 172.21.15.5:0/308381130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 msgr2=0x5575c0406240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38a4e40640 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 0x5575c0406240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38a4e40640 1 -- 172.21.15.5:0/308381130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03fdcb0 msgr2=0x5575c03f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38a4e40640 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03fdcb0 0x5575c03f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38a4e40640 1 -- 172.21.15.5:0/308381130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575c03f82b0 con 0x5575c03f3410 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38a5641640 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 0x5575c0406240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38a4e40640 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03f7700 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x5575c03fc230 tx=0x7f3890005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e275c3b7bc6d67 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f3888ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f3888ff9640 1 -- 172.21.15.5:0/308381130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389003f040 con 0x5575c03f3410 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f3888ff9640 1 -- 172.21.15.5:0/308381130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f3890031b10 con 0x5575c03f3410 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38aa647740 1 -- 172.21.15.5:0/308381130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389800fe50 con 0x5575c03f3410 2024-09-11T22:32:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f3888ff9640 1 -- 172.21.15.5:0/308381130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f389002aa90 con 0x5575c03f3410 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38aa647740 10 client.18642 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f38aa647740 1 -- 172.21.15.5:0/308381130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5575c04787b0 con 0x5575c03f3410 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.431+0000 7f3888ff9640 1 -- 172.21.15.5:0/308381130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389800fe50 con 0x5575c03f3410 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.432+0000 7f3888ff9640 1 -- 172.21.15.5:0/308381130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f38900049c0 con 0x5575c03f3410 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.432+0000 7f38aa647740 10 client.18642 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.432+0000 7f38aa647740 10 client.18642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.432+0000 7f38aa647740 20 client.18642 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.432+0000 7f38aa647740 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5575c040dc50 0x5575c047aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.432+0000 7f38aa647740 4 client.18642 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.432+0000 7f38aa647740 1 -- 172.21.15.5:0/308381130 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575c035ed70 con 0x5575c040dc50 2024-09-11T22:32:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.432+0000 7f38a5641640 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5575c040dc50 0x5575c047aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.432+0000 7f3888ff9640 10 client.18642 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.432+0000 7f38a5641640 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5575c040dc50 0x5575c047aeb0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4cdb9ad097df9a8 server_cookie=aa8f30c3935adf27 in_seq=0 out_seq=0 2024-09-11T22:32:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.482+0000 7f3888ff9640 1 -- 172.21.15.5:0/308381130 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575c035ed70 con 0x5575c040dc50 2024-09-11T22:32:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.482+0000 7f3888ff9640 10 client.18642 handle_command_reply: tid=0 2024-09-11T22:32:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.482+0000 7f38aa647740 1 client.18642 shutdown 2024-09-11T22:32:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 1 -- 172.21.15.5:0/308381130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 msgr2=0x5575c03f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03f7700 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x5575c03fc230 tx=0x7f3890005e20 comp rx=0 tx=0).stop 2024-09-11T22:32:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 1 -- 172.21.15.5:0/308381130 shutdown_connections 2024-09-11T22:32:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5575c040dc50 0x5575c047aeb0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575c03fdcb0 0x5575c03f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575c03f3410 0x5575c03f7700 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f3888ff9640 0 client.18642 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 1 --2- 172.21.15.5:0/308381130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5575c03f2af0 0x5575c0406240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 1 -- 172.21.15.5:0/308381130 >> 172.21.15.5:0/308381130 conn(0x5575c03ef060 msgr2=0x5575c03f45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 1 -- 172.21.15.5:0/308381130 shutdown_connections 2024-09-11T22:32:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 1 -- 172.21.15.5:0/308381130 wait complete. 2024-09-11T22:32:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38aa647740 20 client.18642 trim_cache size 0 max 16384 2024-09-11T22:32:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:32:38.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:32:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:32:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:32:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38a78a5640 1 -- 172.21.15.5:0/344494942 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f388c0a2da0 msgr2=0x7f388c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:38.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38a78a5640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f388c0a2da0 0x7f388c0a5260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3898005c40 tx=0x7f3898009200 comp rx=0 tx=0).stop 2024-09-11T22:32:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38a78a5640 1 -- 172.21.15.5:0/344494942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38a0100860 msgr2=0x7f38a019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.483+0000 7f38a78a5640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38a0100860 0x7f38a019a560 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f389c00d950 tx=0x7f389c00de10 comp rx=0 tx=0).stop 2024-09-11T22:32:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.484+0000 7f38a78a5640 1 -- 172.21.15.5:0/344494942 shutdown_connections 2024-09-11T22:32:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.484+0000 7f38a78a5640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a010e410 0x7f38a019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.484+0000 7f38a78a5640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f388c0a2da0 0x7f388c0a5260 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.484+0000 7f38a78a5640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38a0101210 0x7f38a019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.484+0000 7f38a78a5640 1 --2- 172.21.15.5:0/344494942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38a0100860 0x7f38a019a560 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.484+0000 7f38a78a5640 1 -- 172.21.15.5:0/344494942 >> 172.21.15.5:0/344494942 conn(0x7f38a0077990 msgr2=0x7f38a00ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.484+0000 7f38a78a5640 1 -- 172.21.15.5:0/344494942 shutdown_connections 2024-09-11T22:32:38.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:38.484+0000 7f38a78a5640 1 -- 172.21.15.5:0/344494942 wait complete. 2024-09-11T22:32:38.674 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 70839249 --release --await' 2024-09-11T22:32:38.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 70839249 --release --await 2024-09-11T22:32:38.932 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:39.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781371 7.37 MB/sec execute 417 sec latency 22294.035 ms 2024-09-11T22:32:39.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.406+0000 7ff880372640 1 -- 172.21.15.5:0/2945789847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff878106f10 msgr2=0x7ff87810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:39.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.406+0000 7ff880372640 1 --2- 172.21.15.5:0/2945789847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff878106f10 0x7ff87810ea90 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff870009ca0 tx=0x7ff87002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.406+0000 7ff880372640 1 -- 172.21.15.5:0/2945789847 shutdown_connections 2024-09-11T22:32:39.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.406+0000 7ff880372640 1 --2- 172.21.15.5:0/2945789847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff878106f10 0x7ff87810ea90 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.406+0000 7ff880372640 1 --2- 172.21.15.5:0/2945789847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff878102790 0x7ff8781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.406+0000 7ff880372640 1 --2- 172.21.15.5:0/2945789847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff878101de0 0x7ff8781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.406+0000 7ff880372640 1 -- 172.21.15.5:0/2945789847 >> 172.21.15.5:0/2945789847 conn(0x7ff8780fd8c0 msgr2=0x7ff8780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.406+0000 7ff880372640 1 -- 172.21.15.5:0/2945789847 shutdown_connections 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.406+0000 7ff880372640 1 -- 172.21.15.5:0/2945789847 wait complete. 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff880372640 1 Processor -- start 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff880372640 1 -- start start 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff880372640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff878101de0 0x7ff878198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff880372640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff878102790 0x7ff878198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff880372640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff878106f10 0x7ff87819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff880372640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff878111f90 con 0x7ff878106f10 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff880372640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff878111e10 con 0x7ff878102790 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff880372640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff878112110 con 0x7ff878101de0 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff87e90f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff878106f10 0x7ff87819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff87e90f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff878106f10 0x7ff87819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56060/0 (socket says 172.21.15.5:56060) 2024-09-11T22:32:39.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff87e90f640 1 -- 172.21.15.5:0/3847212840 learned_addr learned my addr 172.21.15.5:0/3847212840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff87d90d640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff878102790 0x7ff878198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff87e90f640 1 -- 172.21.15.5:0/3847212840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff878101de0 msgr2=0x7ff878198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff87e90f640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff878101de0 0x7ff878198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff87e90f640 1 -- 172.21.15.5:0/3847212840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff878102790 msgr2=0x7ff878198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff87e90f640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff878102790 0x7ff878198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.407+0000 7ff87e90f640 1 -- 172.21.15.5:0/3847212840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87819d400 con 0x7ff878106f10 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.408+0000 7ff87d90d640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff878102790 0x7ff878198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.408+0000 7ff87e90f640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff878106f10 0x7ff87819cd20 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff870031a20 tx=0x7ff870031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.408+0000 7ff86f7fe640 1 -- 172.21.15.5:0/3847212840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870003c30 con 0x7ff878106f10 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.408+0000 7ff86f7fe640 1 -- 172.21.15.5:0/3847212840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff870003dd0 con 0x7ff878106f10 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.408+0000 7ff880372640 1 -- 172.21.15.5:0/3847212840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87819d690 con 0x7ff878106f10 2024-09-11T22:32:39.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.408+0000 7ff86f7fe640 1 -- 172.21.15.5:0/3847212840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87002fcf0 con 0x7ff878106f10 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.408+0000 7ff880372640 1 -- 172.21.15.5:0/3847212840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8781a4fd0 con 0x7ff878106f10 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff86f7fe640 1 -- 172.21.15.5:0/3847212840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff870004050 con 0x7ff878106f10 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff88274b740 1 Processor -- start 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff88274b740 1 -- start start 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff88274b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6da470 0x564a3d6da850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff88274b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6dad90 0x564a3d6e5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff88274b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e5650 0x564a3d6e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff88274b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a3d63b620 con 0x564a3d6dad90 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff88274b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564a3d5e2e50 con 0x564a3d6e5650 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff88274b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a3d6375e0 con 0x564a3d6da470 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff87e10e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6dad90 0x564a3d6e5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff87e10e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6dad90 0x564a3d6e5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56072/0 (socket says 172.21.15.5:56072) 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff87e10e640 1 -- 172.21.15.5:0/444377184 learned_addr learned my addr 172.21.15.5:0/444377184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff87d90d640 1 --2- 172.21.15.5:0/444377184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6da470 0x564a3d6da850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff86f7fe640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8640a2d70 0x7ff8640a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.411+0000 7ff87d90d640 1 --2- 172.21.15.5:0/444377184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e5650 0x564a3d6e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff87e90f640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8640a2d70 0x7ff8640a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff86f7fe640 1 -- 172.21.15.5:0/3847212840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7ff8700f09f0 con 0x7ff878106f10 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff87e10e640 1 -- 172.21.15.5:0/444377184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6da470 msgr2=0x564a3d6da850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff87e10e640 1 --2- 172.21.15.5:0/444377184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6da470 0x564a3d6da850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff87e10e640 1 -- 172.21.15.5:0/444377184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e5650 msgr2=0x564a3d6e7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff87e10e640 1 --2- 172.21.15.5:0/444377184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e5650 0x564a3d6e7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff87e10e640 1 -- 172.21.15.5:0/444377184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a3d5d3d40 con 0x564a3d6dad90 2024-09-11T22:32:39.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff87d90d640 1 --2- 172.21.15.5:0/444377184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6da470 0x564a3d6da850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff87e10e640 1 --2- 172.21.15.5:0/444377184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6dad90 0x564a3d6e5110 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff87400bb10 tx=0x7ff87400d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95176d4395b1847c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff87e90f640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8640a2d70 0x7ff8640a5230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff8700b9470 tx=0x7ff870033040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff86cdea640 1 -- 172.21.15.5:0/444377184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874016070 con 0x564a3d6dad90 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff86cdea640 1 -- 172.21.15.5:0/444377184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff874005500 con 0x564a3d6dad90 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.412+0000 7ff86cdea640 1 -- 172.21.15.5:0/444377184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874010460 con 0x564a3d6dad90 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.413+0000 7ff88274b740 1 -- 172.21.15.5:0/444377184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6dad90 msgr2=0x564a3d6e5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.413+0000 7ff88274b740 1 --2- 172.21.15.5:0/444377184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6dad90 0x564a3d6e5110 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff87400bb10 tx=0x7ff87400d2e0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.413+0000 7ff88274b740 1 -- 172.21.15.5:0/444377184 shutdown_connections 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.413+0000 7ff88274b740 1 --2- 172.21.15.5:0/444377184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e5650 0x564a3d6e7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.413+0000 7ff88274b740 1 --2- 172.21.15.5:0/444377184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6dad90 0x564a3d6e5110 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.413+0000 7ff88274b740 1 --2- 172.21.15.5:0/444377184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6da470 0x564a3d6da850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.413+0000 7ff88274b740 1 -- 172.21.15.5:0/444377184 >> 172.21.15.5:0/444377184 conn(0x564a3d6d6a00 msgr2=0x564a3d6d6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:39.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.413+0000 7ff88274b740 1 -- 172.21.15.5:0/444377184 shutdown_connections 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.413+0000 7ff88274b740 1 -- 172.21.15.5:0/444377184 wait complete. 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff88274b740 1 -- start start 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff88274b740 1 Processor -- start 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff88274b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6da470 0x564a3d6e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff88274b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6e2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff88274b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e5650 0x564a3d6e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff88274b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a3d6e8220 con 0x564a3d6da470 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff88274b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564a3d5c3f10 con 0x564a3d6e5650 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff88274b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a3d6e83a0 con 0x564a3d6dad90 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff87d90d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6da470 0x564a3d6e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff88274b740 10 client.0 _ll_register_callbacks cb 0x564a3d5e7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff87d90d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6da470 0x564a3d6e2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56084/0 (socket says 172.21.15.5:56084) 2024-09-11T22:32:39.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff87d90d640 1 -- 172.21.15.5:0/3728865577 learned_addr learned my addr 172.21.15.5:0/3728865577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff87e10e640 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6e2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.414+0000 7ff87d90d640 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e5650 0x564a3d6e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff87d90d640 1 -- 172.21.15.5:0/3728865577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 msgr2=0x564a3d6e2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff87d90d640 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6e2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff87d90d640 1 -- 172.21.15.5:0/3728865577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e5650 msgr2=0x564a3d6e2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff87d90d640 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e5650 0x564a3d6e2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff87d90d640 1 -- 172.21.15.5:0/3728865577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3d34a580 con 0x564a3d6da470 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff87d90d640 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6da470 0x564a3d6e2550 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff868006eb0 tx=0x7ff8680047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fe8f36cce2022d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff861ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff861ffb640 1 -- 172.21.15.5:0/3728865577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86800f040 con 0x564a3d6da470 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff861ffb640 1 -- 172.21.15.5:0/3728865577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7ff868019230 con 0x564a3d6da470 2024-09-11T22:32:39.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff88274b740 1 -- 172.21.15.5:0/3728865577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff874005500 con 0x564a3d6da470 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.415+0000 7ff861ffb640 1 -- 172.21.15.5:0/3728865577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff868028050 con 0x564a3d6da470 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff88274b740 10 client.18654 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff88274b740 1 -- 172.21.15.5:0/3728865577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564a3d414960 con 0x564a3d6da470 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff861ffb640 1 -- 172.21.15.5:0/3728865577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff874005500 con 0x564a3d6da470 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff861ffb640 1 -- 172.21.15.5:0/3728865577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7ff868015040 con 0x564a3d6da470 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff88274b740 10 client.18654 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff88274b740 10 client.18654 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff88274b740 20 client.18654 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff88274b740 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3d6f51d0 0x564a3d6f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff88274b740 4 client.18654 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff88274b740 1 -- 172.21.15.5:0/3728865577 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a3d648b90 con 0x564a3d6f51d0 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.416+0000 7ff87e10e640 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3d6f51d0 0x564a3d6f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.417+0000 7ff87e10e640 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3d6f51d0 0x564a3d6f7600 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21aaf14cb4e48b72 server_cookie=77a306b80191dc26 in_seq=0 out_seq=0 2024-09-11T22:32:39.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.417+0000 7ff861ffb640 10 client.18654 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff861ffb640 1 -- 172.21.15.5:0/3728865577 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a3d648b90 con 0x564a3d6f51d0 2024-09-11T22:32:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff861ffb640 10 client.18654 handle_command_reply: tid=0 2024-09-11T22:32:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 client.18654 shutdown 2024-09-11T22:32:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 -- 172.21.15.5:0/3728865577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6da470 msgr2=0x564a3d6e2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6da470 0x564a3d6e2550 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff868006eb0 tx=0x7ff8680047d0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 -- 172.21.15.5:0/3728865577 shutdown_connections 2024-09-11T22:32:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3d6f51d0 0x564a3d6f7600 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e5650 0x564a3d6e2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6e2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 --2- 172.21.15.5:0/3728865577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6da470 0x564a3d6e2550 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 -- 172.21.15.5:0/3728865577 >> 172.21.15.5:0/3728865577 conn(0x564a3d6d6a00 msgr2=0x564a3d6d1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:39.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff861ffb640 0 client.18654 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:39.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 -- 172.21.15.5:0/3728865577 shutdown_connections 2024-09-11T22:32:39.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.418+0000 7ff88274b740 1 -- 172.21.15.5:0/3728865577 wait complete. 2024-09-11T22:32:39.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.419+0000 7ff88274b740 20 client.18654 trim_cache size 0 max 16384 2024-09-11T22:32:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.443+0000 7ff88274b740 1 Processor -- start 2024-09-11T22:32:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.443+0000 7ff88274b740 1 -- start start 2024-09-11T22:32:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.443+0000 7ff88274b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6dced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.443+0000 7ff88274b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 0x564a3d6fced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.443+0000 7ff88274b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e44c0 0x564a3d6fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.443+0000 7ff88274b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781ab860 con 0x564a3d6e5650 2024-09-11T22:32:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.443+0000 7ff88274b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781ad3a0 con 0x564a3d6e44c0 2024-09-11T22:32:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.443+0000 7ff88274b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781abf30 con 0x564a3d6dad90 2024-09-11T22:32:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.444+0000 7ff87e10e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 0x564a3d6fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.444+0000 7ff87e10e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 0x564a3d6fced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56092/0 (socket says 172.21.15.5:56092) 2024-09-11T22:32:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.444+0000 7ff87e10e640 1 -- 172.21.15.5:0/948793134 learned_addr learned my addr 172.21.15.5:0/948793134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.444+0000 7ff87d90d640 1 --2- 172.21.15.5:0/948793134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6dced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.444+0000 7ff87e10e640 1 -- 172.21.15.5:0/948793134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 msgr2=0x564a3d6dced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.444+0000 7ff87e10e640 1 --2- 172.21.15.5:0/948793134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6dced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.444+0000 7ff87e10e640 1 -- 172.21.15.5:0/948793134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e44c0 msgr2=0x564a3d6fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.444+0000 7ff87e10e640 1 --2- 172.21.15.5:0/948793134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e44c0 0x564a3d6fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.444+0000 7ff87e10e640 1 -- 172.21.15.5:0/948793134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a3d6e1500 con 0x564a3d6e5650 2024-09-11T22:32:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.444+0000 7ff87e10e640 1 --2- 172.21.15.5:0/948793134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 0x564a3d6fced0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff8740047c0 tx=0x7ff874004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2445bcf8abdc71d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:39.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.445+0000 7ff863fff640 1 -- 172.21.15.5:0/948793134 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874016040 con 0x564a3d6e5650 2024-09-11T22:32:39.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.445+0000 7ff863fff640 1 -- 172.21.15.5:0/948793134 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8740048f0 con 0x564a3d6e5650 2024-09-11T22:32:39.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.445+0000 7ff863fff640 1 -- 172.21.15.5:0/948793134 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874003850 con 0x564a3d6e5650 2024-09-11T22:32:39.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.445+0000 7ff88274b740 1 -- 172.21.15.5:0/948793134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 msgr2=0x564a3d6fced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:39.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.445+0000 7ff88274b740 1 --2- 172.21.15.5:0/948793134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 0x564a3d6fced0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff8740047c0 tx=0x7ff874004ba0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.446+0000 7ff88274b740 1 -- 172.21.15.5:0/948793134 shutdown_connections 2024-09-11T22:32:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.446+0000 7ff88274b740 1 --2- 172.21.15.5:0/948793134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e44c0 0x564a3d6fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.446+0000 7ff88274b740 1 --2- 172.21.15.5:0/948793134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 0x564a3d6fced0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.446+0000 7ff88274b740 1 --2- 172.21.15.5:0/948793134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6dced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.446+0000 7ff88274b740 1 -- 172.21.15.5:0/948793134 >> 172.21.15.5:0/948793134 conn(0x564a3d6d6a00 msgr2=0x564a3d6e3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.446+0000 7ff88274b740 1 -- 172.21.15.5:0/948793134 shutdown_connections 2024-09-11T22:32:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.446+0000 7ff88274b740 1 -- 172.21.15.5:0/948793134 wait complete. 2024-09-11T22:32:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.446+0000 7ff88274b740 1 -- start start 2024-09-11T22:32:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.447+0000 7ff88274b740 1 Processor -- start 2024-09-11T22:32:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.447+0000 7ff88274b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6edb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.447+0000 7ff88274b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e44c0 0x564a3d6e9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.447+0000 7ff88274b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 0x564a3d6ea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.447+0000 7ff87d90d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6edb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.447+0000 7ff87d90d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6edb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:55606/0 (socket says 172.21.15.5:55606) 2024-09-11T22:32:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.447+0000 7ff87d90d640 1 -- 172.21.15.5:0/287829005 learned_addr learned my addr 172.21.15.5:0/287829005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.447+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781accd0 con 0x564a3d6e5650 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.447+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781ac600 con 0x564a3d6e44c0 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.447+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff8781a9840 con 0x564a3d6dad90 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.448+0000 7ff87e10e640 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e44c0 0x564a3d6e9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.448+0000 7ff87d90d640 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 0x564a3d6ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.448+0000 7ff88274b740 10 client.0 _ll_register_callbacks cb 0x7ff868019af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.448+0000 7ff87d90d640 1 -- 172.21.15.5:0/287829005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e44c0 msgr2=0x564a3d6e9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.448+0000 7ff87d90d640 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e44c0 0x564a3d6e9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.448+0000 7ff87d90d640 1 -- 172.21.15.5:0/287829005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 msgr2=0x564a3d6ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.448+0000 7ff87d90d640 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 0x564a3d6ea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.448+0000 7ff87d90d640 1 -- 172.21.15.5:0/287829005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3d6eaa00 con 0x564a3d6dad90 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.449+0000 7ff87d90d640 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6edb40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff868019c00 tx=0x7ff868019830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55b5ff45e83dfac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.449+0000 7ff861ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:32:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.449+0000 7ff861ffb640 1 -- 172.21.15.5:0/287829005 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868017200 con 0x564a3d6dad90 2024-09-11T22:32:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.449+0000 7ff861ffb640 1 -- 172.21.15.5:0/287829005 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7ff868019030 con 0x564a3d6dad90 2024-09-11T22:32:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.449+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8740048f0 con 0x564a3d6dad90 2024-09-11T22:32:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.449+0000 7ff861ffb640 1 -- 172.21.15.5:0/287829005 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff86803e050 con 0x564a3d6dad90 2024-09-11T22:32:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.449+0000 7ff88274b740 10 client.27820 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.449+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564a3d760cd0 con 0x564a3d6dad90 2024-09-11T22:32:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.450+0000 7ff861ffb640 1 -- 172.21.15.5:0/287829005 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8740048f0 con 0x564a3d6dad90 2024-09-11T22:32:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.450+0000 7ff861ffb640 1 -- 172.21.15.5:0/287829005 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7ff8680175f0 con 0x564a3d6dad90 2024-09-11T22:32:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.450+0000 7ff88274b740 10 client.27820 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.450+0000 7ff88274b740 10 client.27820 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:32:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.450+0000 7ff88274b740 20 client.27820 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.450+0000 7ff88274b740 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3d761330 0x564a3d763750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:39.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.450+0000 7ff88274b740 4 client.27820 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "70839249", "release": true, "await": true}] 2024-09-11T22:32:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.450+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "70839249", "release": true, "await": true}) -- 0x564a3d6430a0 con 0x564a3d761330 2024-09-11T22:32:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.451+0000 7ff87e10e640 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3d761330 0x564a3d763750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.451+0000 7ff87e10e640 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3d761330 0x564a3d763750 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bd7abe7694f85a1 server_cookie=cb6d73be4fe11013 in_seq=0 out_seq=0 2024-09-11T22:32:39.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.451+0000 7ff861ffb640 10 client.27820 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.466+0000 7ff861ffb640 1 -- 172.21.15.5:0/287829005 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564a3d6430a0 con 0x564a3d761330 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.466+0000 7ff861ffb640 10 client.27820 handle_command_reply: tid=0 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.466+0000 7ff88274b740 1 client.27820 shutdown 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 msgr2=0x564a3d6edb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6edb40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff868019c00 tx=0x7ff868019830 comp rx=0 tx=0).stop 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 shutdown_connections 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3d761330 0x564a3d763750 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3d6e5650 0x564a3d6ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3d6e44c0 0x564a3d6e9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff861ffb640 0 client.27820 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:39.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 1 --2- 172.21.15.5:0/287829005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3d6dad90 0x564a3d6edb40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 >> 172.21.15.5:0/287829005 conn(0x564a3d6d6a00 msgr2=0x564a3d6e2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:39.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 shutdown_connections 2024-09-11T22:32:39.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 1 -- 172.21.15.5:0/287829005 wait complete. 2024-09-11T22:32:39.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.467+0000 7ff88274b740 20 client.27820 trim_cache size 0 max 16384 2024-09-11T22:32:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:32:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 44, 2024-09-11T22:32:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:32:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 40, 2024-09-11T22:32:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:32:39.469 INFO:teuthology.orchestra.run.smithi005.stdout: "70839249": { 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 40, 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.6, 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:32:39.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:32:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:32:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:39.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:32:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 -- 172.21.15.5:0/3847212840 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8640a2d70 msgr2=0x7ff8640a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8640a2d70 0x7ff8640a5230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff8700b9470 tx=0x7ff870033040 comp rx=0 tx=0).stop 2024-09-11T22:32:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 -- 172.21.15.5:0/3847212840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff878106f10 msgr2=0x7ff87819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff878106f10 0x7ff87819cd20 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff870031a20 tx=0x7ff870031a50 comp rx=0 tx=0).stop 2024-09-11T22:32:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 -- 172.21.15.5:0/3847212840 shutdown_connections 2024-09-11T22:32:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff878106f10 0x7ff87819cd20 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff8640a2d70 0x7ff8640a5230 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff878102790 0x7ff878198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 --2- 172.21.15.5:0/3847212840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff878101de0 0x7ff878198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 -- 172.21.15.5:0/3847212840 >> 172.21.15.5:0/3847212840 conn(0x7ff8780fd8c0 msgr2=0x7ff8780fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 -- 172.21.15.5:0/3847212840 shutdown_connections 2024-09-11T22:32:39.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:39.468+0000 7ff880372640 1 -- 172.21.15.5:0/3847212840 wait complete. 2024-09-11T22:32:40.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:39 smithi005 ceph-mon[30943]: pgmap v330: 97 pgs: 97 active+clean; 93 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-09-11T22:32:40.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:39 smithi159 ceph-mon[35534]: pgmap v330: 97 pgs: 97 active+clean; 93 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-09-11T22:32:40.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 782748 7.37 MB/sec execute 418 sec latency 22425.990 ms 2024-09-11T22:32:40.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:39 smithi172 ceph-mon[35461]: pgmap v330: 97 pgs: 97 active+clean; 93 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.8 MiB/s wr, 120 op/s 2024-09-11T22:32:40.370 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '70839249', total seconds elapsed: 26.1 2024-09-11T22:32:40.371 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-11T22:32:40.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:32:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:41.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.170+0000 7f7485045640 1 -- 172.21.15.5:0/1136403940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7480102ef0 msgr2=0x7f748010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:41.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.170+0000 7f7485045640 1 --2- 172.21.15.5:0/1136403940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7480102ef0 0x7f748010da40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f7478009ca0 tx=0x7f747802a950 comp rx=0 tx=0).stop 2024-09-11T22:32:41.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.170+0000 7f7485045640 1 -- 172.21.15.5:0/1136403940 shutdown_connections 2024-09-11T22:32:41.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.170+0000 7f7485045640 1 --2- 172.21.15.5:0/1136403940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7480102ef0 0x7f748010da40 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:41.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.170+0000 7f7485045640 1 --2- 172.21.15.5:0/1136403940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7480102570 0x7f74801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:41.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.170+0000 7f7485045640 1 --2- 172.21.15.5:0/1136403940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7480100e20 0x7f7480101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:41.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.170+0000 7f7485045640 1 -- 172.21.15.5:0/1136403940 >> 172.21.15.5:0/1136403940 conn(0x7f74800fc7d0 msgr2=0x7f74800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.170+0000 7f7485045640 1 -- 172.21.15.5:0/1136403940 shutdown_connections 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.170+0000 7f7485045640 1 -- 172.21.15.5:0/1136403940 wait complete. 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.171+0000 7f7485045640 1 Processor -- start 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.171+0000 7f7485045640 1 -- start start 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f7485045640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7480100e20 0x7f7480198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f7485045640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7480102570 0x7f74801989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f7485045640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7480102ef0 0x7f748019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f7485045640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7480110fd0 con 0x7f7480100e20 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f7485045640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7480110e50 con 0x7f7480102ef0 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f7485045640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7480111150 con 0x7f7480102570 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7480100e20 0x7f7480198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:41.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7480100e20 0x7f7480198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56118/0 (socket says 172.21.15.5:56118) 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747ed9d640 1 -- 172.21.15.5:0/868734495 learned_addr learned my addr 172.21.15.5:0/868734495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747e59c640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7480102570 0x7f74801989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747f59e640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7480102ef0 0x7f748019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747e59c640 1 -- 172.21.15.5:0/868734495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7480102ef0 msgr2=0x7f748019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747e59c640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7480102ef0 0x7f748019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747e59c640 1 -- 172.21.15.5:0/868734495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7480100e20 msgr2=0x7f7480198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747e59c640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7480100e20 0x7f7480198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747e59c640 1 -- 172.21.15.5:0/868734495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748019d410 con 0x7f7480102570 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747ed9d640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7480100e20 0x7f7480198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.172+0000 7f747e59c640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7480102570 0x7f74801989a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f747400e9f0 tx=0x7f747400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.173+0000 7f7467fff640 1 -- 172.21.15.5:0/868734495 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747400ccc0 con 0x7f7480102570 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.173+0000 7f7485045640 1 -- 172.21.15.5:0/868734495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748019d700 con 0x7f7480102570 2024-09-11T22:32:41.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.173+0000 7f7467fff640 1 -- 172.21.15.5:0/868734495 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7474010910 con 0x7f7480102570 2024-09-11T22:32:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.173+0000 7f7467fff640 1 -- 172.21.15.5:0/868734495 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474018960 con 0x7f7480102570 2024-09-11T22:32:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.173+0000 7f7485045640 1 -- 172.21.15.5:0/868734495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74801a4fe0 con 0x7f7480102570 2024-09-11T22:32:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.174+0000 7f7485045640 1 -- 172.21.15.5:0/868734495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7480101200 con 0x7f7480102570 2024-09-11T22:32:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.175+0000 7f7467fff640 1 -- 172.21.15.5:0/868734495 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7474010ac0 con 0x7f7480102570 2024-09-11T22:32:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.175+0000 7f7467fff640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f74680a2cf0 0x7f74680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.176+0000 7f747ed9d640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f74680a2cf0 0x7f74680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.176+0000 7f7467fff640 1 -- 172.21.15.5:0/868734495 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f74740cfb80 con 0x7f7480102570 2024-09-11T22:32:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.176+0000 7f747ed9d640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f74680a2cf0 0x7f74680a51b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7470011500 tx=0x7f7470005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:41.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.179+0000 7f7467fff640 1 -- 172.21.15.5:0/868734495 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f747409c0b0 con 0x7f7480102570 2024-09-11T22:32:41.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 784714 7.38 MB/sec execute 419 sec latency 50.158 ms 2024-09-11T22:32:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.397+0000 7f7485045640 1 -- 172.21.15.5:0/868734495 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74801997c0 con 0x7f7480102570 2024-09-11T22:32:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.398+0000 7f7467fff640 1 -- 172.21.15.5:0/868734495 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11967 (secure 0 0 0) 0x7f74801997c0 con 0x7f7480102570 2024-09-11T22:32:41.399 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:32:41.399 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":44,"btime":"2024-09-11T22:32:26:501017+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:32:26.500999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27887":{"gid":27887,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1984462744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1984462744},{"type":"v1","addr":"172.21.15.159:6833","nonce":1984462744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:32:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 44 2024-09-11T22:32:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.400+0000 7f7485045640 1 -- 172.21.15.5:0/868734495 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f74680a2cf0 msgr2=0x7f74680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.400+0000 7f7485045640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f74680a2cf0 0x7f74680a51b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7470011500 tx=0x7f7470005f90 comp rx=0 tx=0).stop 2024-09-11T22:32:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.400+0000 7f7485045640 1 -- 172.21.15.5:0/868734495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7480102570 msgr2=0x7f74801989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.400+0000 7f7485045640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7480102570 0x7f74801989a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f747400e9f0 tx=0x7f747400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:32:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.400+0000 7f7485045640 1 -- 172.21.15.5:0/868734495 shutdown_connections 2024-09-11T22:32:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.400+0000 7f7485045640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f74680a2cf0 0x7f74680a51b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.400+0000 7f7485045640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7480102ef0 0x7f748019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.400+0000 7f7485045640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7480102570 0x7f74801989a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.400+0000 7f7485045640 1 --2- 172.21.15.5:0/868734495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7480100e20 0x7f7480198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.401+0000 7f7485045640 1 -- 172.21.15.5:0/868734495 >> 172.21.15.5:0/868734495 conn(0x7f74800fc7d0 msgr2=0x7f748006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.401+0000 7f7485045640 1 -- 172.21.15.5:0/868734495 shutdown_connections 2024-09-11T22:32:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:41.401+0000 7f7485045640 1 -- 172.21.15.5:0/868734495 wait complete. 2024-09-11T22:32:41.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:32:42.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:41 smithi005 ceph-mon[30943]: pgmap v331: 97 pgs: 97 active+clean; 93 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 944 KiB/s wr, 41 op/s 2024-09-11T22:32:42.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:41 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/868734495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:42.171 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-11T22:32:42.171 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-11T22:32:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:41 smithi159 ceph-mon[35534]: pgmap v331: 97 pgs: 97 active+clean; 93 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 944 KiB/s wr, 41 op/s 2024-09-11T22:32:42.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:41 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/868734495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:42.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 786174 7.38 MB/sec execute 420 sec latency 31.549 ms 2024-09-11T22:32:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:41 smithi172 ceph-mon[35461]: pgmap v331: 97 pgs: 97 active+clean; 93 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 944 KiB/s wr, 41 op/s 2024-09-11T22:32:42.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:41 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/868734495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:42.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.760+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1577970893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e8102570 msgr2=0x7fd3e81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:42.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.760+0000 7fd3ee0fc640 1 --2- 172.21.15.5:0/1577970893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e8102570 0x7fd3e81029b0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009900 tx=0x7fd3d002a480 comp rx=0 tx=0).stop 2024-09-11T22:32:42.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.760+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1577970893 shutdown_connections 2024-09-11T22:32:42.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.760+0000 7fd3ee0fc640 1 --2- 172.21.15.5:0/1577970893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e8102ef0 0x7fd3e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.760+0000 7fd3ee0fc640 1 --2- 172.21.15.5:0/1577970893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e8102570 0x7fd3e81029b0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.760+0000 7fd3ee0fc640 1 --2- 172.21.15.5:0/1577970893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e8100e20 0x7fd3e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.760+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1577970893 >> 172.21.15.5:0/1577970893 conn(0x7fd3e80fc7d0 msgr2=0x7fd3e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:42.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.760+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1577970893 shutdown_connections 2024-09-11T22:32:42.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.760+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1577970893 wait complete. 2024-09-11T22:32:42.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3ee0fc640 1 Processor -- start 2024-09-11T22:32:42.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3ee0fc640 1 -- start start 2024-09-11T22:32:42.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3ee0fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e8100e20 0x7fd3e81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3ee0fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e8102570 0x7fd3e8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3ee0fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e8102ef0 0x7fd3e819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3ee0fc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8110f80 con 0x7fd3e8100e20 2024-09-11T22:32:42.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3ee0fc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8110e00 con 0x7fd3e8102570 2024-09-11T22:32:42.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3ee0fc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e8111100 con 0x7fd3e8102ef0 2024-09-11T22:32:42.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e8100e20 0x7fd3e81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3deffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e8102570 0x7fd3e8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e8100e20 0x7fd3e81984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51552/0 (socket says 172.21.15.5:51552) 2024-09-11T22:32:42.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3e7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e8102ef0 0x7fd3e819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.761+0000 7fd3e77fe640 1 -- 172.21.15.5:0/1520398577 learned_addr learned my addr 172.21.15.5:0/1520398577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:42.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3e77fe640 1 -- 172.21.15.5:0/1520398577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e8102ef0 msgr2=0x7fd3e819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:42.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3e77fe640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e8102ef0 0x7fd3e819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3e77fe640 1 -- 172.21.15.5:0/1520398577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e8102570 msgr2=0x7fd3e8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:42.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3e77fe640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e8102570 0x7fd3e8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3e77fe640 1 -- 172.21.15.5:0/1520398577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3e819d490 con 0x7fd3e8100e20 2024-09-11T22:32:42.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3deffd640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e8102570 0x7fd3e8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:42.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e8102ef0 0x7fd3e819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:42.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3e77fe640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e8100e20 0x7fd3e81984e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800e9f0 tx=0x7fd3d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:42.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3e57fa640 1 -- 172.21.15.5:0/1520398577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800cd80 con 0x7fd3e8100e20 2024-09-11T22:32:42.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3e57fa640 1 -- 172.21.15.5:0/1520398577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d8010910 con 0x7fd3e8100e20 2024-09-11T22:32:42.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1520398577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3e819d780 con 0x7fd3e8100e20 2024-09-11T22:32:42.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3e57fa640 1 -- 172.21.15.5:0/1520398577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8018990 con 0x7fd3e8100e20 2024-09-11T22:32:42.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.762+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1520398577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3e81a5060 con 0x7fd3e8100e20 2024-09-11T22:32:42.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.764+0000 7fd3e57fa640 1 -- 172.21.15.5:0/1520398577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd3d8010ac0 con 0x7fd3e8100e20 2024-09-11T22:32:42.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3f04d5740 1 Processor -- start 2024-09-11T22:32:42.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3f04d5740 1 -- start start 2024-09-11T22:32:42.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0ee4a0 0x55f79b0ee880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b0f9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0f9680 0x55f79b0fba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f79b0542e0 con 0x55f79b0f9680 2024-09-11T22:32:42.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f79affbce0 con 0x55f79b0ee4a0 2024-09-11T22:32:42.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f79b0502a0 con 0x55f79b0eedc0 2024-09-11T22:32:42.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0f9680 0x55f79b0fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3e7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0ee4a0 0x55f79b0ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b0f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3e7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0ee4a0 0x55f79b0ee880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37240/0 (socket says 172.21.15.5:37240) 2024-09-11T22:32:42.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3e7fff640 1 -- 172.21.15.5:0/4249393134 learned_addr learned my addr 172.21.15.5:0/4249393134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:42.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.765+0000 7fd3e57fa640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd3d40a2cf0 0x7fd3d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3e7fff640 1 -- 172.21.15.5:0/4249393134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 msgr2=0x55f79b0f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:42.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/4249393134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b0f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3e7fff640 1 -- 172.21.15.5:0/4249393134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0f9680 msgr2=0x55f79b0fba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:42.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/4249393134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0f9680 0x55f79b0fba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3e7fff640 1 -- 172.21.15.5:0/4249393134 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f79afe0440 con 0x55f79b0ee4a0 2024-09-11T22:32:42.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3deffd640 1 --2- 172.21.15.5:0/4249393134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0f9680 0x55f79b0fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:42.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3e77fe640 1 --2- 172.21.15.5:0/4249393134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b0f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:42.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3e57fa640 1 -- 172.21.15.5:0/1520398577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fd3d80cf7a0 con 0x7fd3e8100e20 2024-09-11T22:32:42.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd3d40a2cf0 0x7fd3d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/4249393134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0ee4a0 0x55f79b0ee880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00108d0 tx=0x7fd3e0010d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d242092f91cd5e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:42.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3ddffb640 1 -- 172.21.15.5:0/4249393134 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000ee50 con 0x55f79b0ee4a0 2024-09-11T22:32:42.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd3d40a2cf0 0x7fd3d40a51b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0021000 tx=0x7fd3e001e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:42.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3ddffb640 1 -- 172.21.15.5:0/4249393134 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e000e930 con 0x55f79b0ee4a0 2024-09-11T22:32:42.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.766+0000 7fd3ddffb640 1 -- 172.21.15.5:0/4249393134 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0011d80 con 0x55f79b0ee4a0 2024-09-11T22:32:42.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.767+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4249393134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0ee4a0 msgr2=0x55f79b0ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:42.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.767+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/4249393134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0ee4a0 0x55f79b0ee880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd3e00108d0 tx=0x7fd3e0010d90 comp rx=0 tx=0).stop 2024-09-11T22:32:42.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.767+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4249393134 shutdown_connections 2024-09-11T22:32:42.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.767+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/4249393134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0f9680 0x55f79b0fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.767+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/4249393134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b0f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.767+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/4249393134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0ee4a0 0x55f79b0ee880 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.767+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4249393134 >> 172.21.15.5:0/4249393134 conn(0x55f79b0eaa30 msgr2=0x55f79b0eae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:42.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.767+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4249393134 shutdown_connections 2024-09-11T22:32:42.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.767+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4249393134 wait complete. 2024-09-11T22:32:42.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3f04d5740 1 -- start start 2024-09-11T22:32:42.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3f04d5740 1 Processor -- start 2024-09-11T22:32:42.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b0f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b0f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b0f6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f79b0fc250 con 0x55f79b0ee4a0 2024-09-11T22:32:42.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f79b00f050 con 0x55f79b0f9680 2024-09-11T22:32:42.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f79b0fc3d0 con 0x55f79b0eedc0 2024-09-11T22:32:42.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3f04d5740 10 client.0 _ll_register_callbacks cb 0x55f79b04f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:42.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b0f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b0f8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51578/0 (socket says 172.21.15.5:51578) 2024-09-11T22:32:42.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3e77fe640 1 -- 172.21.15.5:0/4042887166 learned_addr learned my addr 172.21.15.5:0/4042887166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:42.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b0f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.768+0000 7fd3deffd640 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b0f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3e77fe640 1 -- 172.21.15.5:0/4042887166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 msgr2=0x55f79b0f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3e77fe640 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b0f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3e77fe640 1 -- 172.21.15.5:0/4042887166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 msgr2=0x55f79b0f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3e77fe640 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b0f6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3e77fe640 1 -- 172.21.15.5:0/4042887166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f79adc53c0 con 0x55f79b0ee4a0 2024-09-11T22:32:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3e77fe640 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b0f8ff0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800c6a0 tx=0x7fd3d8098660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a740842d5c4bf4c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/4042887166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d809b870 con 0x55f79b0ee4a0 2024-09-11T22:32:42.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/4042887166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fd3d809b390 con 0x55f79b0ee4a0 2024-09-11T22:32:42.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4042887166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f79b061850 con 0x55f79b0ee4a0 2024-09-11T22:32:42.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/4042887166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd3d809b690 con 0x55f79b0ee4a0 2024-09-11T22:32:42.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3f04d5740 10 client.18678 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:42.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4042887166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f79b107c20 con 0x55f79b0ee4a0 2024-09-11T22:32:42.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/4042887166 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f79b061850 con 0x55f79b0ee4a0 2024-09-11T22:32:42.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/4042887166 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fd3d8097a80 con 0x55f79b0ee4a0 2024-09-11T22:32:42.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3f04d5740 10 client.18678 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:42.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.769+0000 7fd3f04d5740 10 client.18678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.770+0000 7fd3f04d5740 20 client.18678 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.770+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f79b109030 0x55f79b10b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.770+0000 7fd3f04d5740 4 client.18678 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.770+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4042887166 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f79b05bd60 con 0x55f79b109030 2024-09-11T22:32:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.770+0000 7fd3deffd640 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f79b109030 0x55f79b10b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.770+0000 7fd3c6ffd640 10 client.18678 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.770+0000 7fd3deffd640 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f79b109030 0x55f79b10b420 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df98295f7f2b49f2 server_cookie=b3a924487c785bef in_seq=0 out_seq=0 2024-09-11T22:32:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/4042887166 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f79b05bd60 con 0x55f79b109030 2024-09-11T22:32:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3c6ffd640 10 client.18678 handle_command_reply: tid=0 2024-09-11T22:32:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 client.18678 shutdown 2024-09-11T22:32:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4042887166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 msgr2=0x55f79b0f8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b0f8ff0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800c6a0 tx=0x7fd3d8098660 comp rx=0 tx=0).stop 2024-09-11T22:32:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4042887166 shutdown_connections 2024-09-11T22:32:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f79b109030 0x55f79b10b420 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b0f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b0f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/4042887166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b0f8ff0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4042887166 >> 172.21.15.5:0/4042887166 conn(0x55f79b0eaa30 msgr2=0x55f79b0f0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3c6ffd640 0 client.18678 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:42.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4042887166 shutdown_connections 2024-09-11T22:32:42.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 1 -- 172.21.15.5:0/4042887166 wait complete. 2024-09-11T22:32:42.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.771+0000 7fd3f04d5740 20 client.18678 trim_cache size 0 max 16384 2024-09-11T22:32:42.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.789+0000 7fd3f04d5740 1 Processor -- start 2024-09-11T22:32:42.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.789+0000 7fd3f04d5740 1 -- start start 2024-09-11T22:32:42.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.789+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b10b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.789+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b110c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.789+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b1111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.789+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81ad9c0 con 0x55f79b0ee4a0 2024-09-11T22:32:42.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.789+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81af500 con 0x55f79b0f9680 2024-09-11T22:32:42.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.789+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81ae090 con 0x55f79b0eedc0 2024-09-11T22:32:42.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b10b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3e7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b1111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3e77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b10b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51582/0 (socket says 172.21.15.5:51582) 2024-09-11T22:32:42.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3e77fe640 1 -- 172.21.15.5:0/755777332 learned_addr learned my addr 172.21.15.5:0/755777332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:42.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3deffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b110c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3e7fff640 1 -- 172.21.15.5:0/755777332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 msgr2=0x55f79b110c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:42.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/755777332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b110c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3e7fff640 1 -- 172.21.15.5:0/755777332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 msgr2=0x55f79b10b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/755777332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b10b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3e7fff640 1 -- 172.21.15.5:0/755777332 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f79b0f5250 con 0x55f79b0f9680 2024-09-11T22:32:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3deffd640 1 --2- 172.21.15.5:0/755777332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/755777332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b1111b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0009860 tx=0x7fd3e0011a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75210a6f43e5d31f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.790+0000 7fd3dd7fa640 1 -- 172.21.15.5:0/755777332 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000ee50 con 0x55f79b0f9680 2024-09-11T22:32:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.791+0000 7fd3dd7fa640 1 -- 172.21.15.5:0/755777332 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e0011ab0 con 0x55f79b0f9680 2024-09-11T22:32:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.791+0000 7fd3dd7fa640 1 -- 172.21.15.5:0/755777332 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e001dc60 con 0x55f79b0f9680 2024-09-11T22:32:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.791+0000 7fd3f04d5740 1 -- 172.21.15.5:0/755777332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 msgr2=0x55f79b1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.791+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/755777332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b1111b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0009860 tx=0x7fd3e0011a40 comp rx=0 tx=0).stop 2024-09-11T22:32:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.791+0000 7fd3f04d5740 1 -- 172.21.15.5:0/755777332 shutdown_connections 2024-09-11T22:32:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.791+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/755777332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b1111b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.791+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/755777332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0eedc0 0x55f79b110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.791+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/755777332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0ee4a0 0x55f79b10b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.791+0000 7fd3f04d5740 1 -- 172.21.15.5:0/755777332 >> 172.21.15.5:0/755777332 conn(0x55f79b0eaa30 msgr2=0x55f79b0e9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.791+0000 7fd3f04d5740 1 -- 172.21.15.5:0/755777332 shutdown_connections 2024-09-11T22:32:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.792+0000 7fd3f04d5740 1 -- 172.21.15.5:0/755777332 wait complete. 2024-09-11T22:32:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.792+0000 7fd3f04d5740 1 -- start start 2024-09-11T22:32:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.792+0000 7fd3f04d5740 1 Processor -- start 2024-09-11T22:32:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0ee4a0 0x55f79b101c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0eedc0 0x55f79b0fef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3f04d5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b0ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81aee30 con 0x55f79b0eedc0 2024-09-11T22:32:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81ae760 con 0x55f79b0f9680 2024-09-11T22:32:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3f04d5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd3e81ab9a0 con 0x55f79b0ee4a0 2024-09-11T22:32:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3f04d5740 10 client.0 _ll_register_callbacks cb 0x7fd3d800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0eedc0 0x55f79b0fef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3e77fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0ee4a0 0x55f79b101c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3deffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0eedc0 0x55f79b0fef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51594/0 (socket says 172.21.15.5:51594) 2024-09-11T22:32:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3deffd640 1 -- 172.21.15.5:0/1879237367 learned_addr learned my addr 172.21.15.5:0/1879237367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3e7fff640 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b0ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3e77fe640 1 -- 172.21.15.5:0/1879237367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 msgr2=0x55f79b0ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3e77fe640 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b0ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3e77fe640 1 -- 172.21.15.5:0/1879237367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0eedc0 msgr2=0x55f79b0fef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3e77fe640 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0eedc0 0x55f79b0fef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3e77fe640 1 -- 172.21.15.5:0/1879237367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f79b0ffb40 con 0x55f79b0ee4a0 2024-09-11T22:32:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3e77fe640 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0ee4a0 0x55f79b101c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8098700 tx=0x7fd3d809b080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a25ce28676ebade server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.793+0000 7fd3c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:32:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/1879237367 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d809b870 con 0x55f79b0ee4a0 2024-09-11T22:32:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/1879237367 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fd3d8010430 con 0x55f79b0ee4a0 2024-09-11T22:32:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3f04d5740 1 -- 172.21.15.5:0/1879237367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3e0011ab0 con 0x55f79b0ee4a0 2024-09-11T22:32:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/1879237367 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd3d80d5270 con 0x55f79b0ee4a0 2024-09-11T22:32:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3f04d5740 10 client.27844 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3f04d5740 1 -- 172.21.15.5:0/1879237367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55f79b173af0 con 0x55f79b0ee4a0 2024-09-11T22:32:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/1879237367 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e0011ab0 con 0x55f79b0ee4a0 2024-09-11T22:32:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/1879237367 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fd3d80ba020 con 0x55f79b0ee4a0 2024-09-11T22:32:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3f04d5740 10 client.27844 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3f04d5740 10 client.27844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3f04d5740 20 client.27844 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f79b174080 0x55f79b176470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3f04d5740 4 client.27844 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:32:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.794+0000 7fd3f04d5740 1 -- 172.21.15.5:0/1879237367 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f79afd8210 con 0x55f79b174080 2024-09-11T22:32:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.795+0000 7fd3deffd640 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f79b174080 0x55f79b176470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.795+0000 7fd3deffd640 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f79b174080 0x55f79b176470 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbcef832b2d9ac2c server_cookie=d933547a0d14488c in_seq=0 out_seq=0 2024-09-11T22:32:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.795+0000 7fd3c6ffd640 10 client.27844 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:42.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.796+0000 7fd3c6ffd640 1 -- 172.21.15.5:0/1879237367 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f79afd8210 con 0x55f79b174080 2024-09-11T22:32:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.796+0000 7fd3c6ffd640 10 client.27844 handle_command_reply: tid=0 2024-09-11T22:32:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.796+0000 7fd3f04d5740 1 client.27844 shutdown 2024-09-11T22:32:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 1 -- 172.21.15.5:0/1879237367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0ee4a0 msgr2=0x55f79b101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0ee4a0 0x55f79b101c10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd3d8098700 tx=0x7fd3d809b080 comp rx=0 tx=0).stop 2024-09-11T22:32:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 1 -- 172.21.15.5:0/1879237367 shutdown_connections 2024-09-11T22:32:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f79b174080 0x55f79b176470 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f79b0f9680 0x55f79b0ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3c6ffd640 0 client.27844 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:42.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f79b0eedc0 0x55f79b0fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 1 --2- 172.21.15.5:0/1879237367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f79b0ee4a0 0x55f79b101c10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 1 -- 172.21.15.5:0/1879237367 >> 172.21.15.5:0/1879237367 conn(0x55f79b0eaa30 msgr2=0x55f79b0f7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:42.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 1 -- 172.21.15.5:0/1879237367 shutdown_connections 2024-09-11T22:32:42.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 1 -- 172.21.15.5:0/1879237367 wait complete. 2024-09-11T22:32:42.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3f04d5740 20 client.27844 trim_cache size 0 max 16384 2024-09-11T22:32:42.810 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:32:42.810 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:32:42.811 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:32:42.811 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:32:42.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1520398577 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd3d40a2cf0 msgr2=0x7fd3d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:42.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.797+0000 7fd3ee0fc640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd3d40a2cf0 0x7fd3d40a51b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0021000 tx=0x7fd3e001e000 comp rx=0 tx=0).stop 2024-09-11T22:32:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.798+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1520398577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e8100e20 msgr2=0x7fd3e81984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.798+0000 7fd3ee0fc640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e8100e20 0x7fd3e81984e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800e9f0 tx=0x7fd3d800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.798+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1520398577 shutdown_connections 2024-09-11T22:32:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.798+0000 7fd3ee0fc640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3e8102ef0 0x7fd3e819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.798+0000 7fd3ee0fc640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd3d40a2cf0 0x7fd3d40a51b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.798+0000 7fd3ee0fc640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3e8102570 0x7fd3e8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.798+0000 7fd3ee0fc640 1 --2- 172.21.15.5:0/1520398577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3e8100e20 0x7fd3e81984e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.798+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1520398577 >> 172.21.15.5:0/1520398577 conn(0x7fd3e80fc7d0 msgr2=0x7fd3e806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.798+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1520398577 shutdown_connections 2024-09-11T22:32:42.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:42.798+0000 7fd3ee0fc640 1 -- 172.21.15.5:0/1520398577 wait complete. 2024-09-11T22:32:43.054 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:43.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 788239 7.39 MB/sec execute 421 sec latency 43.278 ms 2024-09-11T22:32:43.605 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:32:43.605 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:32:44.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:43 smithi005 ceph-mon[30943]: pgmap v332: 97 pgs: 97 active+clean; 92 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 119 op/s 2024-09-11T22:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 -- 172.21.15.5:0/2600843207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f362c102d40 msgr2=0x7f362c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 --2- 172.21.15.5:0/2600843207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f362c102d40 0x7f362c103120 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3620009900 tx=0x7f362002a480 comp rx=0 tx=0).stop 2024-09-11T22:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 -- 172.21.15.5:0/2600843207 shutdown_connections 2024-09-11T22:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 --2- 172.21.15.5:0/2600843207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f362c10c170 0x7f362c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 --2- 172.21.15.5:0/2600843207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f362c103660 0x7f362c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 --2- 172.21.15.5:0/2600843207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f362c102d40 0x7f362c103120 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 -- 172.21.15.5:0/2600843207 >> 172.21.15.5:0/2600843207 conn(0x7f362c0fc7a0 msgr2=0x7f362c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 -- 172.21.15.5:0/2600843207 shutdown_connections 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 -- 172.21.15.5:0/2600843207 wait complete. 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 Processor -- start 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 -- start start 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.246+0000 7f36331fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f362c102d40 0x7f362c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f36331fd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f362c103660 0x7f362c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f36331fd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f362c10c170 0x7f362c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f36331fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f362c111f70 con 0x7f362c102d40 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f36331fd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f362c111df0 con 0x7f362c10c170 2024-09-11T22:32:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f36331fd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1120f0 con 0x7f362c103660 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3630f99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f362c102d40 0x7f362c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3630f99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f362c102d40 0x7f362c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51622/0 (socket says 172.21.15.5:51622) 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3630f99640 1 -- 172.21.15.5:0/2445969171 learned_addr learned my addr 172.21.15.5:0/2445969171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f362bfff640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f362c103660 0x7f362c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f363179a640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f362c10c170 0x7f362c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3630f99640 1 -- 172.21.15.5:0/2445969171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f362c103660 msgr2=0x7f362c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3630f99640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f362c103660 0x7f362c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3630f99640 1 -- 172.21.15.5:0/2445969171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f362c10c170 msgr2=0x7f362c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3630f99640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f362c10c170 0x7f362c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3630f99640 1 -- 172.21.15.5:0/2445969171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c19d460 con 0x7f362c102d40 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3630f99640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f362c102d40 0x7f362c1984b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3620006010 tx=0x7f3620004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3629ffb640 1 -- 172.21.15.5:0/2445969171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362003f030 con 0x7f362c102d40 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3629ffb640 1 -- 172.21.15.5:0/2445969171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3620004440 con 0x7f362c102d40 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f36331fd640 1 -- 172.21.15.5:0/2445969171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362c19d6f0 con 0x7f362c102d40 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.247+0000 7f3629ffb640 1 -- 172.21.15.5:0/2445969171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3620035070 con 0x7f362c102d40 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.248+0000 7f36331fd640 1 -- 172.21.15.5:0/2445969171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362c1a5030 con 0x7f362c102d40 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.250+0000 7f3629ffb640 1 -- 172.21.15.5:0/2445969171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3620042030 con 0x7f362c102d40 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.250+0000 7f36355d6740 1 Processor -- start 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.250+0000 7f36355d6740 1 -- start start 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.250+0000 7f36355d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601079850 0x560601079c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.250+0000 7f36355d6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x5606010844f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f36355d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 0x560601086e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f36355d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560600fde160 con 0x560601079850 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f36355d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560600f85720 con 0x560601084a30 2024-09-11T22:32:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f36355d6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560600fda120 con 0x56060107a170 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f363179a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601079850 0x560601079c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f3630f99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x5606010844f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f363179a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601079850 0x560601079c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51632/0 (socket says 172.21.15.5:51632) 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f363179a640 1 -- 172.21.15.5:0/2746008483 learned_addr learned my addr 172.21.15.5:0/2746008483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f3629ffb640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f361c0a2cf0 0x7f361c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f362bfff640 1 --2- 172.21.15.5:0/2746008483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 0x560601086e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f363179a640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f361c0a2cf0 0x7f361c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f3630f99640 1 -- 172.21.15.5:0/2746008483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 msgr2=0x560601086e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f3630f99640 1 --2- 172.21.15.5:0/2746008483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 0x560601086e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f3630f99640 1 -- 172.21.15.5:0/2746008483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601079850 msgr2=0x560601079c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f3630f99640 1 --2- 172.21.15.5:0/2746008483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601079850 0x560601079c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f3630f99640 1 -- 172.21.15.5:0/2746008483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560600d4fba0 con 0x56060107a170 2024-09-11T22:32:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f362bfff640 1 --2- 172.21.15.5:0/2746008483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 0x560601086e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f3629ffb640 1 -- 172.21.15.5:0/2445969171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f36200ef6f0 con 0x7f362c102d40 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f3630f99640 1 --2- 172.21.15.5:0/2746008483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x5606010844f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f36200ece00 tx=0x7f362004b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3996565293a1e9b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.251+0000 7f3616ffd640 1 -- 172.21.15.5:0/2746008483 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36200bc2d0 con 0x56060107a170 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f363179a640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f361c0a2cf0 0x7f361c0a51b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3624009fd0 tx=0x7f3624004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f3616ffd640 1 -- 172.21.15.5:0/2746008483 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36200bff80 con 0x56060107a170 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f3616ffd640 1 -- 172.21.15.5:0/2746008483 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36200bbbd0 con 0x56060107a170 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f36355d6740 1 -- 172.21.15.5:0/2746008483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 msgr2=0x5606010844f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f36355d6740 1 --2- 172.21.15.5:0/2746008483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x5606010844f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f36200ece00 tx=0x7f362004b000 comp rx=0 tx=0).stop 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f36355d6740 1 -- 172.21.15.5:0/2746008483 shutdown_connections 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f36355d6740 1 --2- 172.21.15.5:0/2746008483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 0x560601086e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f36355d6740 1 --2- 172.21.15.5:0/2746008483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x5606010844f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f36355d6740 1 --2- 172.21.15.5:0/2746008483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601079850 0x560601079c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f36355d6740 1 -- 172.21.15.5:0/2746008483 >> 172.21.15.5:0/2746008483 conn(0x560601075de0 msgr2=0x5606010761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f36355d6740 1 -- 172.21.15.5:0/2746008483 shutdown_connections 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.252+0000 7f36355d6740 1 -- 172.21.15.5:0/2746008483 wait complete. 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f36355d6740 1 -- start start 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f36355d6740 1 Processor -- start 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f36355d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601079850 0x5606010818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f36355d6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x560601081e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f36355d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 0x560601082370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f36355d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5606010873c0 con 0x560601084a30 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f36355d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560600f66900 con 0x560601079850 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f36355d6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560601087540 con 0x56060107a170 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f36355d6740 10 client.0 _ll_register_callbacks cb 0x560600f64780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:44.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f363179a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 0x560601082370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f3630f99640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601079850 0x5606010818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f362bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x560601081e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f362bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x560601081e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:42644/0 (socket says 172.21.15.5:42644) 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.253+0000 7f363179a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 0x560601082370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51642/0 (socket says 172.21.15.5:51642) 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f362bfff640 1 -- 172.21.15.5:0/602784619 learned_addr learned my addr 172.21.15.5:0/602784619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f362bfff640 1 -- 172.21.15.5:0/602784619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601079850 msgr2=0x5606010818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f362bfff640 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601079850 0x5606010818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f362bfff640 1 -- 172.21.15.5:0/602784619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 msgr2=0x560601082370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f362bfff640 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 0x560601082370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f362bfff640 1 -- 172.21.15.5:0/602784619 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560600fa6c50 con 0x56060107a170 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f362bfff640 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x560601081e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3618012700 tx=0x7f3618012b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d0bc6f5ec38cb26 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f360bfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:32:44.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f360bfff640 1 -- 172.21.15.5:0/602784619 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361800f640 con 0x56060107a170 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f360bfff640 1 -- 172.21.15.5:0/602784619 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f361800f7e0 con 0x56060107a170 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f36355d6740 1 -- 172.21.15.5:0/602784619 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36200bff80 con 0x56060107a170 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f360bfff640 1 -- 172.21.15.5:0/602784619 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f361800f9e0 con 0x56060107a170 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f36355d6740 10 client.27856 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.254+0000 7f36355d6740 1 -- 172.21.15.5:0/602784619 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560601093010 con 0x56060107a170 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f360bfff640 1 -- 172.21.15.5:0/602784619 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36200bff80 con 0x56060107a170 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f360bfff640 1 -- 172.21.15.5:0/602784619 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f3618022030 con 0x56060107a170 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f36355d6740 10 client.27856 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f36355d6740 10 client.27856 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f36355d6740 20 client.27856 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f36355d6740 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560601094570 0x560601096990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f36355d6740 4 client.27856 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:44.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f36355d6740 1 -- 172.21.15.5:0/602784619 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560600feb890 con 0x560601094570 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f3630f99640 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560601094570 0x560601096990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f360bfff640 10 client.27856 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.255+0000 7f3630f99640 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560601094570 0x560601096990 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37878496e60654e8 server_cookie=42f1fb752da15d7 in_seq=0 out_seq=0 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.256+0000 7f360bfff640 1 -- 172.21.15.5:0/602784619 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560600feb890 con 0x560601094570 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.256+0000 7f360bfff640 10 client.27856 handle_command_reply: tid=0 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.256+0000 7f36355d6740 1 client.27856 shutdown 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.256+0000 7f36355d6740 1 -- 172.21.15.5:0/602784619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 msgr2=0x560601081e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.256+0000 7f36355d6740 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x560601081e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3618012700 tx=0x7f3618012b00 comp rx=0 tx=0).stop 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.257+0000 7f36355d6740 1 -- 172.21.15.5:0/602784619 shutdown_connections 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.257+0000 7f36355d6740 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560601094570 0x560601096990 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.257+0000 7f36355d6740 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 0x560601082370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.257+0000 7f36355d6740 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56060107a170 0x560601081e30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.257+0000 7f360bfff640 0 client.27856 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:44.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.257+0000 7f36355d6740 1 --2- 172.21.15.5:0/602784619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601079850 0x5606010818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.257+0000 7f36355d6740 1 -- 172.21.15.5:0/602784619 >> 172.21.15.5:0/602784619 conn(0x560601075de0 msgr2=0x56060107bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:44.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.257+0000 7f36355d6740 1 -- 172.21.15.5:0/602784619 shutdown_connections 2024-09-11T22:32:44.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.257+0000 7f36355d6740 1 -- 172.21.15.5:0/602784619 wait complete. 2024-09-11T22:32:44.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.257+0000 7f36355d6740 20 client.27856 trim_cache size 0 max 16384 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.281+0000 7f36355d6740 1 Processor -- start 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.281+0000 7f36355d6740 1 -- start start 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f36355d6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x5606010821c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f36355d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56060107a170 0x56060109c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f36355d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 0x56060109c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f36355d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ada30 con 0x560601084a30 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f36355d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1af570 con 0x56060107a170 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f36355d6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ae100 con 0x560601079850 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f3630f99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x5606010821c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f3630f99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x5606010821c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:42650/0 (socket says 172.21.15.5:42650) 2024-09-11T22:32:44.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f362bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56060107a170 0x56060109c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f3630f99640 1 -- 172.21.15.5:0/3072697255 learned_addr learned my addr 172.21.15.5:0/3072697255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f363179a640 1 --2- 172.21.15.5:0/3072697255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 0x56060109c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f362bfff640 1 -- 172.21.15.5:0/3072697255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 msgr2=0x5606010821c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f362bfff640 1 --2- 172.21.15.5:0/3072697255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x5606010821c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f362bfff640 1 -- 172.21.15.5:0/3072697255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 msgr2=0x56060109c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f362bfff640 1 --2- 172.21.15.5:0/3072697255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 0x56060109c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.282+0000 7f362bfff640 1 -- 172.21.15.5:0/3072697255 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560601075c10 con 0x56060107a170 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.283+0000 7f362bfff640 1 --2- 172.21.15.5:0/3072697255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56060107a170 0x56060109c260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3618010e80 tx=0x7f3618009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56d230da27c34e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.283+0000 7f36167fc640 1 -- 172.21.15.5:0/3072697255 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361801c420 con 0x56060107a170 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.283+0000 7f36167fc640 1 -- 172.21.15.5:0/3072697255 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f361800f640 con 0x56060107a170 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.283+0000 7f36167fc640 1 -- 172.21.15.5:0/3072697255 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36180106b0 con 0x56060107a170 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.283+0000 7f36355d6740 1 -- 172.21.15.5:0/3072697255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56060107a170 msgr2=0x56060109c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:44.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 --2- 172.21.15.5:0/3072697255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56060107a170 0x56060109c260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3618010e80 tx=0x7f3618009e90 comp rx=0 tx=0).stop 2024-09-11T22:32:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 -- 172.21.15.5:0/3072697255 shutdown_connections 2024-09-11T22:32:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 --2- 172.21.15.5:0/3072697255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560601084a30 0x56060109c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 --2- 172.21.15.5:0/3072697255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56060107a170 0x56060109c260 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 --2- 172.21.15.5:0/3072697255 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x5606010821c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 -- 172.21.15.5:0/3072697255 >> 172.21.15.5:0/3072697255 conn(0x560601075de0 msgr2=0x560601083970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 -- 172.21.15.5:0/3072697255 shutdown_connections 2024-09-11T22:32:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 -- 172.21.15.5:0/3072697255 wait complete. 2024-09-11T22:32:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 -- start start 2024-09-11T22:32:44.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 Processor -- start 2024-09-11T22:32:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.284+0000 7f36355d6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x56060107e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f36355d6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56060107a170 0x56060107ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f36355d6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 0x56060107f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f36355d6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1aeea0 con 0x56060107a170 2024-09-11T22:32:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f36355d6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1ae7d0 con 0x560601084a30 2024-09-11T22:32:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f36355d6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f362c1aba10 con 0x560601079850 2024-09-11T22:32:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f3630f99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x56060107e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f36355d6740 10 client.0 _ll_register_callbacks cb 0x7f3618009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f3630f99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x56060107e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:42666/0 (socket says 172.21.15.5:42666) 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f3630f99640 1 -- 172.21.15.5:0/1848679624 learned_addr learned my addr 172.21.15.5:0/1848679624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f362bfff640 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56060107a170 0x56060107ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f363179a640 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 0x56060107f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f3630f99640 1 -- 172.21.15.5:0/1848679624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 msgr2=0x56060107f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f3630f99640 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 0x56060107f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f3630f99640 1 -- 172.21.15.5:0/1848679624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56060107a170 msgr2=0x56060107ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f3630f99640 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56060107a170 0x56060107ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f3630f99640 1 -- 172.21.15.5:0/1848679624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560601079680 con 0x560601079850 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f3630f99640 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x56060107e9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36200ed400 tx=0x7f362002a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbd0289d4ef82a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f360bfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:32:44.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f360bfff640 1 -- 172.21.15.5:0/1848679624 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36200bbbd0 con 0x560601079850 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f360bfff640 1 -- 172.21.15.5:0/1848679624 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f36200ed080 con 0x560601079850 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.285+0000 7f36355d6740 1 -- 172.21.15.5:0/1848679624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f361800f640 con 0x560601079850 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f360bfff640 1 -- 172.21.15.5:0/1848679624 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f36200bff80 con 0x560601079850 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f36355d6740 10 client.27862 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f36355d6740 1 -- 172.21.15.5:0/1848679624 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560601107be0 con 0x560601079850 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f360bfff640 1 -- 172.21.15.5:0/1848679624 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36200d2020 con 0x560601079850 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f360bfff640 1 -- 172.21.15.5:0/1848679624 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f36200efba0 con 0x560601079850 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f36355d6740 10 client.27862 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f36355d6740 10 client.27862 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f36355d6740 20 client.27862 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f36355d6740 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560601108260 0x56060110a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f36355d6740 4 client.27862 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f36355d6740 1 -- 172.21.15.5:0/1848679624 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x560600fe5be0 con 0x560601108260 2024-09-11T22:32:44.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.286+0000 7f362bfff640 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560601108260 0x56060110a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.287+0000 7f360bfff640 10 client.27862 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.287+0000 7f362bfff640 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560601108260 0x56060110a680 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ff922d57b1081d server_cookie=749705577b6511a1 in_seq=0 out_seq=0 2024-09-11T22:32:44.303 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:44.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:43 smithi159 ceph-mon[35534]: pgmap v332: 97 pgs: 97 active+clean; 92 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 119 op/s 2024-09-11T22:32:44.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789614 7.39 MB/sec execute 422 sec latency 44.400 ms 2024-09-11T22:32:44.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:43 smithi172 ceph-mon[35461]: pgmap v332: 97 pgs: 97 active+clean; 92 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 119 op/s 2024-09-11T22:32:44.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f360bfff640 1 -- 172.21.15.5:0/1848679624 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560600fe5be0 con 0x560601108260 2024-09-11T22:32:44.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f360bfff640 10 client.27862 handle_command_reply: tid=0 2024-09-11T22:32:44.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 client.27862 shutdown 2024-09-11T22:32:44.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 -- 172.21.15.5:0/1848679624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 msgr2=0x56060107e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:44.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x56060107e9c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f36200ed400 tx=0x7f362002a480 comp rx=0 tx=0).stop 2024-09-11T22:32:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 -- 172.21.15.5:0/1848679624 shutdown_connections 2024-09-11T22:32:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560601108260 0x56060110a680 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560601084a30 0x56060107f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56060107a170 0x56060107ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 --2- 172.21.15.5:0/1848679624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560601079850 0x56060107e9c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f360bfff640 0 client.27862 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 -- 172.21.15.5:0/1848679624 >> 172.21.15.5:0/1848679624 conn(0x560601075de0 msgr2=0x5606010746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 -- 172.21.15.5:0/1848679624 shutdown_connections 2024-09-11T22:32:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 1 -- 172.21.15.5:0/1848679624 wait complete. 2024-09-11T22:32:44.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.410+0000 7f36355d6740 20 client.27862 trim_cache size 0 max 16384 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 44, 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 42, 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "9f4d187b": { 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 42, 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:32:44.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.6, 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:44.414 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:44.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:44.415 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 -- 172.21.15.5:0/2445969171 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f361c0a2cf0 msgr2=0x7f361c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f361c0a2cf0 0x7f361c0a51b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3624009fd0 tx=0x7f3624004ab0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 -- 172.21.15.5:0/2445969171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f362c102d40 msgr2=0x7f362c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f362c102d40 0x7f362c1984b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3620006010 tx=0x7f3620004040 comp rx=0 tx=0).stop 2024-09-11T22:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 -- 172.21.15.5:0/2445969171 shutdown_connections 2024-09-11T22:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f361c0a2cf0 0x7f361c0a51b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f362c10c170 0x7f362c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f362c103660 0x7f362c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 --2- 172.21.15.5:0/2445969171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f362c102d40 0x7f362c1984b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 -- 172.21.15.5:0/2445969171 >> 172.21.15.5:0/2445969171 conn(0x7f362c0fc7a0 msgr2=0x7f362c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 -- 172.21.15.5:0/2445969171 shutdown_connections 2024-09-11T22:32:44.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:44.411+0000 7f36331fd640 1 -- 172.21.15.5:0/2445969171 wait complete. 2024-09-11T22:32:45.329 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9f4d187b', quiesce took 5.0 seconds. Will release after: 17.55 2024-09-11T22:32:45.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.37 MB/sec execute 423 sec latency 909.960 ms 2024-09-11T22:32:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.513+0000 7f9246fb9640 1 -- 172.21.15.5:0/147215251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9240102570 msgr2=0x7f92401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.513+0000 7f9246fb9640 1 --2- 172.21.15.5:0/147215251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9240102570 0x7f92401029b0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f9234009900 tx=0x7f923402a480 comp rx=0 tx=0).stop 2024-09-11T22:32:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.513+0000 7f9246fb9640 1 -- 172.21.15.5:0/147215251 shutdown_connections 2024-09-11T22:32:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.513+0000 7f9246fb9640 1 --2- 172.21.15.5:0/147215251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9240102ef0 0x7f924010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.513+0000 7f9246fb9640 1 --2- 172.21.15.5:0/147215251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9240102570 0x7f92401029b0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.513+0000 7f9246fb9640 1 --2- 172.21.15.5:0/147215251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9240100e20 0x7f9240101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.513+0000 7f9246fb9640 1 -- 172.21.15.5:0/147215251 >> 172.21.15.5:0/147215251 conn(0x7f92400fc7d0 msgr2=0x7f92400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.513+0000 7f9246fb9640 1 -- 172.21.15.5:0/147215251 shutdown_connections 2024-09-11T22:32:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.513+0000 7f9246fb9640 1 -- 172.21.15.5:0/147215251 wait complete. 2024-09-11T22:32:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9246fb9640 1 Processor -- start 2024-09-11T22:32:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9246fb9640 1 -- start start 2024-09-11T22:32:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9246fb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9240100e20 0x7f9240197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9246fb9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9240102570 0x7f9240197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9246fb9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9240102ef0 0x7f924019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9246fb9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9240110f50 con 0x7f9240100e20 2024-09-11T22:32:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9246fb9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9240110dd0 con 0x7f9240102570 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9246fb9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f92401110d0 con 0x7f9240102ef0 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9244d55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9240100e20 0x7f9240197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9244d55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9240100e20 0x7f9240197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51684/0 (socket says 172.21.15.5:51684) 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9244d55640 1 -- 172.21.15.5:0/3500926105 learned_addr learned my addr 172.21.15.5:0/3500926105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f923ffff640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9240102570 0x7f9240197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.514+0000 7f9245556640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9240102ef0 0x7f924019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f9244d55640 1 -- 172.21.15.5:0/3500926105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9240102ef0 msgr2=0x7f924019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f9244d55640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9240102ef0 0x7f924019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f9244d55640 1 -- 172.21.15.5:0/3500926105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9240102570 msgr2=0x7f9240197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f9244d55640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9240102570 0x7f9240197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f9244d55640 1 -- 172.21.15.5:0/3500926105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924019c3e0 con 0x7f9240100e20 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f9245556640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9240102ef0 0x7f924019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f9244d55640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9240100e20 0x7f9240197430 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f923000cce0 tx=0x7f923000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f923dffb640 1 -- 172.21.15.5:0/3500926105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230004b20 con 0x7f9240100e20 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f923dffb640 1 -- 172.21.15.5:0/3500926105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9230004cc0 con 0x7f9240100e20 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f9246fb9640 1 -- 172.21.15.5:0/3500926105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924019c6d0 con 0x7f9240100e20 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f923dffb640 1 -- 172.21.15.5:0/3500926105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923000f7d0 con 0x7f9240100e20 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.515+0000 7f9246fb9640 1 -- 172.21.15.5:0/3500926105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92401a3fb0 con 0x7f9240100e20 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.516+0000 7f9246fb9640 1 -- 172.21.15.5:0/3500926105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9240101200 con 0x7f9240100e20 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.517+0000 7f923dffb640 1 -- 172.21.15.5:0/3500926105 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f923000f970 con 0x7f9240100e20 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.518+0000 7f923dffb640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f92280a2d70 0x7f92280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.518+0000 7f923ffff640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f92280a2d70 0x7f92280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.518+0000 7f923dffb640 1 -- 172.21.15.5:0/3500926105 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f92300cf550 con 0x7f9240100e20 2024-09-11T22:32:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.519+0000 7f923ffff640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f92280a2d70 0x7f92280a5230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9234009230 tx=0x7f92340058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.522+0000 7f923dffb640 1 -- 172.21.15.5:0/3500926105 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9230014030 con 0x7f9240100e20 2024-09-11T22:32:45.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.735+0000 7f9246fb9640 1 -- 172.21.15.5:0/3500926105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9240109090 con 0x7f9240100e20 2024-09-11T22:32:45.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.736+0000 7f923dffb640 1 -- 172.21.15.5:0/3500926105 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11967 (secure 0 0 0) 0x7f923009c0d0 con 0x7f9240100e20 2024-09-11T22:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:32:45.737 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":44,"btime":"2024-09-11T22:32:26:501017+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:32:26.500999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27887":{"gid":27887,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1984462744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1984462744},{"type":"v1","addr":"172.21.15.159:6833","nonce":1984462744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:32:45.737 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 44 2024-09-11T22:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 -- 172.21.15.5:0/3500926105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f92280a2d70 msgr2=0x7f92280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f92280a2d70 0x7f92280a5230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9234009230 tx=0x7f92340058d0 comp rx=0 tx=0).stop 2024-09-11T22:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 -- 172.21.15.5:0/3500926105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9240100e20 msgr2=0x7f9240197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9240100e20 0x7f9240197430 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f923000cce0 tx=0x7f923000a620 comp rx=0 tx=0).stop 2024-09-11T22:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 -- 172.21.15.5:0/3500926105 shutdown_connections 2024-09-11T22:32:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9240102ef0 0x7f924019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f92280a2d70 0x7f92280a5230 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9240102570 0x7f9240197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 --2- 172.21.15.5:0/3500926105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9240100e20 0x7f9240197430 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 -- 172.21.15.5:0/3500926105 >> 172.21.15.5:0/3500926105 conn(0x7f92400fc7d0 msgr2=0x7f92400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 -- 172.21.15.5:0/3500926105 shutdown_connections 2024-09-11T22:32:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:45.738+0000 7f9246fb9640 1 -- 172.21.15.5:0/3500926105 wait complete. 2024-09-11T22:32:46.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:45 smithi005 ceph-mon[30943]: pgmap v333: 97 pgs: 97 active+clean; 92 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-11T22:32:46.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:45 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3500926105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:46.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:45 smithi159 ceph-mon[35534]: pgmap v333: 97 pgs: 97 active+clean; 92 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-11T22:32:46.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:45 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3500926105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:46.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.35 MB/sec execute 424 sec latency 1910.021 ms 2024-09-11T22:32:46.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:45 smithi172 ceph-mon[35461]: pgmap v333: 97 pgs: 97 active+clean; 92 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-11T22:32:46.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:45 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3500926105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:32:46.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:32:46.947 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:47.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.33 MB/sec execute 425 sec latency 2910.123 ms 2024-09-11T22:32:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.462+0000 7fd8986b6640 1 -- 172.21.15.5:0/1814356079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd890101420 msgr2=0x7fd89010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.462+0000 7fd8986b6640 1 --2- 172.21.15.5:0/1814356079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd890101420 0x7fd89010d950 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fd888009930 tx=0x7fd88802a420 comp rx=0 tx=0).stop 2024-09-11T22:32:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.462+0000 7fd8986b6640 1 -- 172.21.15.5:0/1814356079 shutdown_connections 2024-09-11T22:32:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.462+0000 7fd8986b6640 1 --2- 172.21.15.5:0/1814356079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd890101420 0x7fd89010d950 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.462+0000 7fd8986b6640 1 --2- 172.21.15.5:0/1814356079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd890100a80 0x7fd890100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.462+0000 7fd8986b6640 1 --2- 172.21.15.5:0/1814356079 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd890106a80 0x7fd890106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.462+0000 7fd8986b6640 1 -- 172.21.15.5:0/1814356079 >> 172.21.15.5:0/1814356079 conn(0x7fd8900fc7a0 msgr2=0x7fd8900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd8986b6640 1 -- 172.21.15.5:0/1814356079 shutdown_connections 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd8986b6640 1 -- 172.21.15.5:0/1814356079 wait complete. 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd8986b6640 1 Processor -- start 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd8986b6640 1 -- start start 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd8986b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd890100a80 0x7fd890197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd8986b6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd890101420 0x7fd8901977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd8986b6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd890106a80 0x7fd89019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd8986b6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd890110ed0 con 0x7fd890100a80 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd8986b6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd890110d50 con 0x7fd890101420 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd8986b6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd890111050 con 0x7fd890106a80 2024-09-11T22:32:47.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd896452640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd890100a80 0x7fd890197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd896452640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd890100a80 0x7fd890197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51694/0 (socket says 172.21.15.5:51694) 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.463+0000 7fd896452640 1 -- 172.21.15.5:0/296570750 learned_addr learned my addr 172.21.15.5:0/296570750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd896c53640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd890106a80 0x7fd89019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd895c51640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd890101420 0x7fd8901977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd896452640 1 -- 172.21.15.5:0/296570750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd890106a80 msgr2=0x7fd89019bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd896452640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd890106a80 0x7fd89019bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd896452640 1 -- 172.21.15.5:0/296570750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd890101420 msgr2=0x7fd8901977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd896452640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd890101420 0x7fd8901977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd896452640 1 -- 172.21.15.5:0/296570750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89019c240 con 0x7fd890100a80 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd896452640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd890100a80 0x7fd890197290 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd88000eab0 tx=0x7fd88000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd88f7fe640 1 -- 172.21.15.5:0/296570750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88000cd80 con 0x7fd890100a80 2024-09-11T22:32:47.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd88f7fe640 1 -- 172.21.15.5:0/296570750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd880011950 con 0x7fd890100a80 2024-09-11T22:32:47.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd8986b6640 1 -- 172.21.15.5:0/296570750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89019c530 con 0x7fd890100a80 2024-09-11T22:32:47.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd88f7fe640 1 -- 172.21.15.5:0/296570750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880010a00 con 0x7fd890100a80 2024-09-11T22:32:47.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.464+0000 7fd8986b6640 1 -- 172.21.15.5:0/296570750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8901a3e10 con 0x7fd890100a80 2024-09-11T22:32:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.467+0000 7fd89aa8f740 1 Processor -- start 2024-09-11T22:32:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.467+0000 7fd88f7fe640 1 -- 172.21.15.5:0/296570750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd880011b00 con 0x7fd890100a80 2024-09-11T22:32:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.467+0000 7fd89aa8f740 1 -- start start 2024-09-11T22:32:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.467+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f465090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.467+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 0x55ff4f465a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.467+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f474970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff4f3cb2f0 con 0x55ff4f472580 2024-09-11T22:32:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff4f372cf0 con 0x55ff4f466b40 2024-09-11T22:32:47.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff4f3c72b0 con 0x55ff4f4655d0 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd895c51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f474970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd895c51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f474970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51700/0 (socket says 172.21.15.5:51700) 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd895c51640 1 -- 172.21.15.5:0/2892897746 learned_addr learned my addr 172.21.15.5:0/2892897746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd896452640 1 --2- 172.21.15.5:0/2892897746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 0x55ff4f465a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd896c53640 1 --2- 172.21.15.5:0/2892897746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f465090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd895c51640 1 -- 172.21.15.5:0/2892897746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 msgr2=0x55ff4f465a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd88f7fe640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd8780a2d70 0x7fd8780a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd895c51640 1 --2- 172.21.15.5:0/2892897746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 0x55ff4f465a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd895c51640 1 -- 172.21.15.5:0/2892897746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 msgr2=0x55ff4f465090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd895c51640 1 --2- 172.21.15.5:0/2892897746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f465090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd895c51640 1 -- 172.21.15.5:0/2892897746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff4f3573a0 con 0x55ff4f472580 2024-09-11T22:32:47.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd896c53640 1 --2- 172.21.15.5:0/2892897746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f465090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd88f7fe640 1 -- 172.21.15.5:0/296570750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fd8800cf960 con 0x7fd890100a80 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd895c51640 1 --2- 172.21.15.5:0/2892897746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f474970 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd88400aca0 tx=0x7fd88400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c494ce2af91bb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd896c53640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd8780a2d70 0x7fd8780a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.468+0000 7fd88cff9640 1 -- 172.21.15.5:0/2892897746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88401d070 con 0x55ff4f472580 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd88cff9640 1 -- 172.21.15.5:0/2892897746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd884018b20 con 0x55ff4f472580 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd88cff9640 1 -- 172.21.15.5:0/2892897746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884010bd0 con 0x55ff4f472580 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd896c53640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd8780a2d70 0x7fd8780a5230 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd888004770 tx=0x7fd888009260 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd89aa8f740 1 -- 172.21.15.5:0/2892897746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 msgr2=0x55ff4f474970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/2892897746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f474970 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd88400aca0 tx=0x7fd88400f790 comp rx=0 tx=0).stop 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd89aa8f740 1 -- 172.21.15.5:0/2892897746 shutdown_connections 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/2892897746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f474970 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/2892897746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 0x55ff4f465a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/2892897746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f465090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd89aa8f740 1 -- 172.21.15.5:0/2892897746 >> 172.21.15.5:0/2892897746 conn(0x55ff4f42e310 msgr2=0x55ff4f461c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.469+0000 7fd89aa8f740 1 -- 172.21.15.5:0/2892897746 shutdown_connections 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd89aa8f740 1 -- 172.21.15.5:0/2892897746 wait complete. 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd89aa8f740 1 -- start start 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd89aa8f740 1 Processor -- start 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 0x55ff4f471f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f46f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f46f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ff4f475280 con 0x55ff4f472580 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ff4f386060 con 0x55ff4f466b40 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ff4f475400 con 0x55ff4f4655d0 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd89aa8f740 10 client.0 _ll_register_callbacks cb 0x55ff4f0d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:47.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd896c53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f46f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd895c51640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f46f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd896452640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 0x55ff4f471f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.470+0000 7fd895c51640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f46f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37362/0 (socket says 172.21.15.5:37362) 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd895c51640 1 -- 172.21.15.5:0/1123006284 learned_addr learned my addr 172.21.15.5:0/1123006284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd896452640 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 0x55ff4f471f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:42726/0 (socket says 172.21.15.5:42726) 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd895c51640 1 -- 172.21.15.5:0/1123006284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 msgr2=0x55ff4f471f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd895c51640 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 0x55ff4f471f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd895c51640 1 -- 172.21.15.5:0/1123006284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 msgr2=0x55ff4f46f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd895c51640 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f46f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd895c51640 1 -- 172.21.15.5:0/1123006284 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff4f13c3c0 con 0x55ff4f466b40 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd895c51640 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f46f1f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd88400a8a0 tx=0x7fd88400a9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dccbd51620fce47 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:32:47.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd875ffb640 1 -- 172.21.15.5:0/1123006284 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88401d050 con 0x55ff4f466b40 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1123006284 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd884018b20 con 0x55ff4f466b40 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd875ffb640 1 -- 172.21.15.5:0/1123006284 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fd8840108d0 con 0x55ff4f466b40 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd875ffb640 1 -- 172.21.15.5:0/1123006284 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd884032050 con 0x55ff4f466b40 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd89aa8f740 10 client.27974 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.471+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1123006284 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ff4f480ce0 con 0x55ff4f466b40 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.472+0000 7fd875ffb640 1 -- 172.21.15.5:0/1123006284 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd884018b20 con 0x55ff4f466b40 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.472+0000 7fd875ffb640 1 -- 172.21.15.5:0/1123006284 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fd8840046c0 con 0x55ff4f466b40 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.472+0000 7fd89aa8f740 10 client.27974 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.472+0000 7fd89aa8f740 10 client.27974 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.472+0000 7fd89aa8f740 20 client.27974 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.472+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ff4f482090 0x55ff4f484480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.472+0000 7fd89aa8f740 4 client.27974 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.472+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1123006284 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff4f3d8860 con 0x55ff4f482090 2024-09-11T22:32:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.472+0000 7fd896452640 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ff4f482090 0x55ff4f484480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.473+0000 7fd896452640 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ff4f482090 0x55ff4f484480 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bdd6008b62269c7 server_cookie=a375b8787d6fb131 in_seq=0 out_seq=0 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.473+0000 7fd875ffb640 10 client.27974 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd875ffb640 1 -- 172.21.15.5:0/1123006284 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ff4f3d8860 con 0x55ff4f482090 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd875ffb640 10 client.27974 handle_command_reply: tid=0 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 client.27974 shutdown 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1123006284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 msgr2=0x55ff4f46f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f46f1f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd88400a8a0 tx=0x7fd88400a9f0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1123006284 shutdown_connections 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ff4f482090 0x55ff4f484480 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f46f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f466b40 0x55ff4f46f1f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd875ffb640 0 client.27974 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:32:47.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1123006284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f4655d0 0x55ff4f471f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1123006284 >> 172.21.15.5:0/1123006284 conn(0x55ff4f42e310 msgr2=0x55ff4f467960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1123006284 shutdown_connections 2024-09-11T22:32:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1123006284 wait complete. 2024-09-11T22:32:47.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.474+0000 7fd89aa8f740 20 client.27974 trim_cache size 0 max 16384 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd89aa8f740 1 Processor -- start 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd89aa8f740 1 -- start start 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f4655d0 0x55ff4f47e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f489b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f48a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901ab6f0 con 0x55ff4f472580 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901ad230 con 0x55ff4f4655d0 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901abdc0 con 0x55ff4f466b40 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd896c53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f48a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd896c53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f48a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51722/0 (socket says 172.21.15.5:51722) 2024-09-11T22:32:47.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.487+0000 7fd896c53640 1 -- 172.21.15.5:0/3253942516 learned_addr learned my addr 172.21.15.5:0/3253942516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd896452640 1 --2- 172.21.15.5:0/3253942516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f4655d0 0x55ff4f47e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd895c51640 1 --2- 172.21.15.5:0/3253942516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f489b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd896c53640 1 -- 172.21.15.5:0/3253942516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 msgr2=0x55ff4f489b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd896c53640 1 --2- 172.21.15.5:0/3253942516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f489b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd896c53640 1 -- 172.21.15.5:0/3253942516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f4655d0 msgr2=0x55ff4f47e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd896c53640 1 --2- 172.21.15.5:0/3253942516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f4655d0 0x55ff4f47e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd896c53640 1 -- 172.21.15.5:0/3253942516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff4f46fb50 con 0x55ff4f472580 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd895c51640 1 --2- 172.21.15.5:0/3253942516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f489b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd896c53640 1 --2- 172.21.15.5:0/3253942516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f48a050 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd888009900 tx=0x7fd88803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aea48442ab65bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd877fff640 1 -- 172.21.15.5:0/3253942516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888035040 con 0x55ff4f472580 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd877fff640 1 -- 172.21.15.5:0/3253942516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88804e070 con 0x55ff4f472580 2024-09-11T22:32:47.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.488+0000 7fd877fff640 1 -- 172.21.15.5:0/3253942516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888053a60 con 0x55ff4f472580 2024-09-11T22:32:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 -- 172.21.15.5:0/3253942516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 msgr2=0x55ff4f48a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/3253942516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f48a050 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd888009900 tx=0x7fd88803f000 comp rx=0 tx=0).stop 2024-09-11T22:32:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 -- 172.21.15.5:0/3253942516 shutdown_connections 2024-09-11T22:32:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/3253942516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f472580 0x55ff4f48a050 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/3253942516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f489b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/3253942516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f4655d0 0x55ff4f47e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 -- 172.21.15.5:0/3253942516 >> 172.21.15.5:0/3253942516 conn(0x55ff4f42e310 msgr2=0x55ff4f471270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 -- 172.21.15.5:0/3253942516 shutdown_connections 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 -- 172.21.15.5:0/3253942516 wait complete. 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 -- start start 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 Processor -- start 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f4655d0 0x55ff4f47ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.489+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f46be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd89aa8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f472580 0x55ff4f46c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901acb60 con 0x55ff4f4655d0 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901ac490 con 0x55ff4f472580 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd89aa8f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901a96d0 con 0x55ff4f466b40 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896452640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f4655d0 0x55ff4f47ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd895c51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f46be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896452640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f4655d0 0x55ff4f47ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51724/0 (socket says 172.21.15.5:51724) 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd895c51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f46be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:42738/0 (socket says 172.21.15.5:42738) 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896452640 1 -- 172.21.15.5:0/1873072251 learned_addr learned my addr 172.21.15.5:0/1873072251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd89aa8f740 10 client.0 _ll_register_callbacks cb 0x7fd88401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896c53640 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f472580 0x55ff4f46c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896452640 1 -- 172.21.15.5:0/1873072251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 msgr2=0x55ff4f46be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896452640 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f46be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896452640 1 -- 172.21.15.5:0/1873072251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f472580 msgr2=0x55ff4f46c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896452640 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f472580 0x55ff4f46c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896452640 1 -- 172.21.15.5:0/1873072251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff4f46cbd0 con 0x55ff4f4655d0 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd895c51640 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f46be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896c53640 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f472580 0x55ff4f46c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd896452640 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f4655d0 0x55ff4f47ad60 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd88000c880 tx=0x7fd880000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362c10853531d1f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.490+0000 7fd875ffb640 1 -- 172.21.15.5:0/1873072251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800cd490 con 0x55ff4f4655d0 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd875ffb640 1 -- 172.21.15.5:0/1873072251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fd8800a45c0 con 0x55ff4f4655d0 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1873072251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd88804e070 con 0x55ff4f4655d0 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd875ffb640 1 -- 172.21.15.5:0/1873072251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd8800a0190 con 0x55ff4f4655d0 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd89aa8f740 10 client.18726 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1873072251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55ff4f4eca50 con 0x55ff4f4655d0 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd875ffb640 1 -- 172.21.15.5:0/1873072251 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd88804e070 con 0x55ff4f4655d0 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd875ffb640 1 -- 172.21.15.5:0/1873072251 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fd884019ce0 con 0x55ff4f4655d0 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd89aa8f740 10 client.18726 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd89aa8f740 10 client.18726 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd89aa8f740 20 client.18726 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ff4f4ed1a0 0x55ff4f4ef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd89aa8f740 4 client.18726 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1873072251 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff4f3d2d70 con 0x55ff4f4ed1a0 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.491+0000 7fd895c51640 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ff4f4ed1a0 0x55ff4f4ef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd895c51640 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ff4f4ed1a0 0x55ff4f4ef5c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4f9e1dff6d0fc29f server_cookie=b33f60ea3eecc857 in_seq=0 out_seq=0 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd875ffb640 10 client.18726 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd875ffb640 1 -- 172.21.15.5:0/1873072251 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff4f3d2d70 con 0x55ff4f4ed1a0 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd875ffb640 10 client.18726 handle_command_reply: tid=0 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 client.18726 shutdown 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1873072251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f4655d0 msgr2=0x55ff4f47ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f4655d0 0x55ff4f47ad60 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd88000c880 tx=0x7fd880000f40 comp rx=0 tx=0).stop 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1873072251 shutdown_connections 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55ff4f4ed1a0 0x55ff4f4ef5c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ff4f472580 0x55ff4f46c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ff4f466b40 0x55ff4f46be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 --2- 172.21.15.5:0/1873072251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ff4f4655d0 0x55ff4f47ad60 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1873072251 >> 172.21.15.5:0/1873072251 conn(0x55ff4f42e310 msgr2=0x55ff4f470a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd875ffb640 0 client.18726 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:32:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1873072251 shutdown_connections 2024-09-11T22:32:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.492+0000 7fd89aa8f740 1 -- 172.21.15.5:0/1873072251 wait complete. 2024-09-11T22:32:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd89aa8f740 20 client.18726 trim_cache size 0 max 16384 2024-09-11T22:32:47.499 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:32:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 -- 172.21.15.5:0/296570750 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd8780a2d70 msgr2=0x7fd8780a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd8780a2d70 0x7fd8780a5230 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd888004770 tx=0x7fd888009260 comp rx=0 tx=0).stop 2024-09-11T22:32:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 -- 172.21.15.5:0/296570750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd890100a80 msgr2=0x7fd890197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd890100a80 0x7fd890197290 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd88000eab0 tx=0x7fd88000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 -- 172.21.15.5:0/296570750 shutdown_connections 2024-09-11T22:32:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd890106a80 0x7fd89019bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd8780a2d70 0x7fd8780a5230 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd890101420 0x7fd8901977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 --2- 172.21.15.5:0/296570750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd890100a80 0x7fd890197290 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 -- 172.21.15.5:0/296570750 >> 172.21.15.5:0/296570750 conn(0x7fd8900fc7a0 msgr2=0x7fd8900fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:47.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 -- 172.21.15.5:0/296570750 shutdown_connections 2024-09-11T22:32:47.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:47.493+0000 7fd8986b6640 1 -- 172.21.15.5:0/296570750 wait complete. 2024-09-11T22:32:48.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:47 smithi005 ceph-mon[30943]: pgmap v334: 97 pgs: 97 active+clean; 92 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-11T22:32:48.210 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:32:48.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:47 smithi159 ceph-mon[35534]: pgmap v334: 97 pgs: 97 active+clean; 92 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-11T22:32:48.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.32 MB/sec execute 426 sec latency 3910.195 ms 2024-09-11T22:32:48.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:47 smithi172 ceph-mon[35461]: pgmap v334: 97 pgs: 97 active+clean; 92 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-11T22:32:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.020+0000 7ff578a71640 1 -- 172.21.15.5:0/23264830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574106f10 msgr2=0x7ff57410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.020+0000 7ff578a71640 1 --2- 172.21.15.5:0/23264830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574106f10 0x7ff57410ea90 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009ca0 tx=0x7ff56c02a8c0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.020+0000 7ff578a71640 1 -- 172.21.15.5:0/23264830 shutdown_connections 2024-09-11T22:32:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.020+0000 7ff578a71640 1 --2- 172.21.15.5:0/23264830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574106f10 0x7ff57410ea90 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.020+0000 7ff578a71640 1 --2- 172.21.15.5:0/23264830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff574102790 0x7ff5741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.020+0000 7ff578a71640 1 --2- 172.21.15.5:0/23264830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff574101de0 0x7ff5741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.020+0000 7ff578a71640 1 -- 172.21.15.5:0/23264830 >> 172.21.15.5:0/23264830 conn(0x7ff5740fd8c0 msgr2=0x7ff5740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.020+0000 7ff578a71640 1 -- 172.21.15.5:0/23264830 shutdown_connections 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.020+0000 7ff578a71640 1 -- 172.21.15.5:0/23264830 wait complete. 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff578a71640 1 Processor -- start 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff578a71640 1 -- start start 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff578a71640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff574101de0 0x7ff5740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff578a71640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff574102790 0x7ff574074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff578a71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574106f10 0x7ff574074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff578a71640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff574111f90 con 0x7ff574106f10 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff578a71640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff574111e10 con 0x7ff574102790 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff578a71640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff574112110 con 0x7ff574101de0 2024-09-11T22:32:49.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff572d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574106f10 0x7ff574074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff572d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574106f10 0x7ff574074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51742/0 (socket says 172.21.15.5:51742) 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.021+0000 7ff572d9d640 1 -- 172.21.15.5:0/305864563 learned_addr learned my addr 172.21.15.5:0/305864563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff57259c640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff574101de0 0x7ff5740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff571d9b640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff574102790 0x7ff574074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff572d9d640 1 -- 172.21.15.5:0/305864563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff574101de0 msgr2=0x7ff5740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff572d9d640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff574101de0 0x7ff5740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff572d9d640 1 -- 172.21.15.5:0/305864563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff574102790 msgr2=0x7ff574074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff572d9d640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff574102790 0x7ff574074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff572d9d640 1 -- 172.21.15.5:0/305864563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5741a6930 con 0x7ff574106f10 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff571d9b640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff574102790 0x7ff574074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:49.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff572d9d640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574106f10 0x7ff574074f70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff56c031a20 tx=0x7ff56c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff5637fe640 1 -- 172.21.15.5:0/305864563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c003c30 con 0x7ff574106f10 2024-09-11T22:32:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff5637fe640 1 -- 172.21.15.5:0/305864563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56c003dd0 con 0x7ff574106f10 2024-09-11T22:32:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff5637fe640 1 -- 172.21.15.5:0/305864563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c02fcf0 con 0x7ff574106f10 2024-09-11T22:32:49.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff578a71640 1 -- 172.21.15.5:0/305864563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5741a6bc0 con 0x7ff574106f10 2024-09-11T22:32:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.022+0000 7ff578a71640 1 -- 172.21.15.5:0/305864563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5741a7150 con 0x7ff574106f10 2024-09-11T22:32:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.025+0000 7ff5637fe640 1 -- 172.21.15.5:0/305864563 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff56c004050 con 0x7ff574106f10 2024-09-11T22:32:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.025+0000 7ff57ae4a740 1 Processor -- start 2024-09-11T22:32:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.025+0000 7ff5637fe640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff55c0a2d70 0x7ff55c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.025+0000 7ff5637fe640 1 -- 172.21.15.5:0/305864563 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7ff56c0f0690 con 0x7ff574106f10 2024-09-11T22:32:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.025+0000 7ff571d9b640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff55c0a2d70 0x7ff55c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.025+0000 7ff57ae4a740 1 -- start start 2024-09-11T22:32:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1bbb40 0x5603ee1ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1ba5d0 0x5603ee1baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5603ee1202f0 con 0x5603ee1ba5d0 2024-09-11T22:32:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5603ee0c7cf0 con 0x5603ee1bbb40 2024-09-11T22:32:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5603ee11c2b0 con 0x5603ee1c5370 2024-09-11T22:32:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff571d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1ba5d0 0x5603ee1baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff571d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1ba5d0 0x5603ee1baa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51750/0 (socket says 172.21.15.5:51750) 2024-09-11T22:32:49.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff571d9b640 1 -- 172.21.15.5:0/2867370193 learned_addr learned my addr 172.21.15.5:0/2867370193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff572d9d640 1 --2- 172.21.15.5:0/2867370193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff57259c640 1 --2- 172.21.15.5:0/2867370193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1bbb40 0x5603ee1ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff571d9b640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff55c0a2d70 0x7ff55c0a5230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff568006fd0 tx=0x7ff56800ad10 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff571d9b640 1 -- 172.21.15.5:0/2867370193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 msgr2=0x5603ee1c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:49.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff571d9b640 1 --2- 172.21.15.5:0/2867370193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff571d9b640 1 -- 172.21.15.5:0/2867370193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1bbb40 msgr2=0x5603ee1ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff571d9b640 1 --2- 172.21.15.5:0/2867370193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1bbb40 0x5603ee1ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff571d9b640 1 -- 172.21.15.5:0/2867370193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ee0ac3a0 con 0x5603ee1ba5d0 2024-09-11T22:32:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.026+0000 7ff571d9b640 1 --2- 172.21.15.5:0/2867370193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1ba5d0 0x5603ee1baa10 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff56800cc30 tx=0x7ff568017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a23f286538105f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.027+0000 7ff560ff9640 1 -- 172.21.15.5:0/2867370193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568023070 con 0x5603ee1ba5d0 2024-09-11T22:32:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.027+0000 7ff560ff9640 1 -- 172.21.15.5:0/2867370193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56801a040 con 0x5603ee1ba5d0 2024-09-11T22:32:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.027+0000 7ff560ff9640 1 -- 172.21.15.5:0/2867370193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568015680 con 0x5603ee1ba5d0 2024-09-11T22:32:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.027+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2867370193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1ba5d0 msgr2=0x5603ee1baa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:49.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.027+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/2867370193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1ba5d0 0x5603ee1baa10 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7ff56800cc30 tx=0x7ff568017000 comp rx=0 tx=0).stop 2024-09-11T22:32:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.027+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2867370193 shutdown_connections 2024-09-11T22:32:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.027+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/2867370193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.027+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/2867370193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1ba5d0 0x5603ee1baa10 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.027+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/2867370193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1bbb40 0x5603ee1ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.027+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2867370193 >> 172.21.15.5:0/2867370193 conn(0x5603ee183310 msgr2=0x5603ee1b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2867370193 shutdown_connections 2024-09-11T22:32:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2867370193 wait complete. 2024-09-11T22:32:49.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 1 -- start start 2024-09-11T22:32:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 1 Processor -- start 2024-09-11T22:32:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1ba5d0 0x5603ee1c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1c5370 0x5603ee1c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5603ee1c8070 con 0x5603ee1bbb40 2024-09-11T22:32:49.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5603ee0db060 con 0x5603ee1c5370 2024-09-11T22:32:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5603ee1c81f0 con 0x5603ee1ba5d0 2024-09-11T22:32:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff57ae4a740 10 client.0 _ll_register_callbacks cb 0x5603ede2e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff571d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.028+0000 7ff571d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1c1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51762/0 (socket says 172.21.15.5:51762) 2024-09-11T22:32:49.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff571d9b640 1 -- 172.21.15.5:0/526052649 learned_addr learned my addr 172.21.15.5:0/526052649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff57259c640 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1ba5d0 0x5603ee1c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff572d9d640 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1c5370 0x5603ee1c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff571d9b640 1 -- 172.21.15.5:0/526052649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1ba5d0 msgr2=0x5603ee1c4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff571d9b640 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1ba5d0 0x5603ee1c4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff571d9b640 1 -- 172.21.15.5:0/526052649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1c5370 msgr2=0x5603ee1c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff571d9b640 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1c5370 0x5603ee1c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff571d9b640 1 -- 172.21.15.5:0/526052649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ede913c0 con 0x5603ee1bbb40 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff572d9d640 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1c5370 0x5603ee1c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff571d9b640 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1c1fe0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff568021040 tx=0x7ff56801e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6016d543f5bcdad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:49.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff559ffb640 1 -- 172.21.15.5:0/526052649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56802e020 con 0x5603ee1bbb40 2024-09-11T22:32:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff57ae4a740 1 -- 172.21.15.5:0/526052649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff56801a040 con 0x5603ee1bbb40 2024-09-11T22:32:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff559ffb640 1 -- 172.21.15.5:0/526052649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7ff56802a680 con 0x5603ee1bbb40 2024-09-11T22:32:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff559ffb640 1 -- 172.21.15.5:0/526052649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff568039050 con 0x5603ee1bbb40 2024-09-11T22:32:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff57ae4a740 10 client.18738 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.029+0000 7ff57ae4a740 1 -- 172.21.15.5:0/526052649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5603ee1d3ad0 con 0x5603ee1bbb40 2024-09-11T22:32:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.030+0000 7ff559ffb640 1 -- 172.21.15.5:0/526052649 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff56801a040 con 0x5603ee1bbb40 2024-09-11T22:32:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.030+0000 7ff559ffb640 1 -- 172.21.15.5:0/526052649 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7ff568012040 con 0x5603ee1bbb40 2024-09-11T22:32:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.030+0000 7ff57ae4a740 10 client.18738 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.030+0000 7ff57ae4a740 10 client.18738 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:32:49.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.030+0000 7ff57ae4a740 20 client.18738 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.030+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5603ee1d4ed0 0x5603ee1d72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.030+0000 7ff57ae4a740 4 client.18738 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.030+0000 7ff57ae4a740 1 -- 172.21.15.5:0/526052649 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603ee12d860 con 0x5603ee1d4ed0 2024-09-11T22:32:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.030+0000 7ff57259c640 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5603ee1d4ed0 0x5603ee1d72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.031+0000 7ff57259c640 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5603ee1d4ed0 0x5603ee1d72c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8e0b510ad9f78de server_cookie=698143c9a50f6936 in_seq=0 out_seq=0 2024-09-11T22:32:49.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.031+0000 7ff559ffb640 10 client.18738 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.031+0000 7ff559ffb640 1 -- 172.21.15.5:0/526052649 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603ee12d860 con 0x5603ee1d4ed0 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.031+0000 7ff559ffb640 10 client.18738 handle_command_reply: tid=0 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.031+0000 7ff57ae4a740 1 client.18738 shutdown 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 1 -- 172.21.15.5:0/526052649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 msgr2=0x5603ee1c1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1c1fe0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff568021040 tx=0x7ff56801e660 comp rx=0 tx=0).stop 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 1 -- 172.21.15.5:0/526052649 shutdown_connections 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5603ee1d4ed0 0x5603ee1d72c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1c5370 0x5603ee1c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1c1fe0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff559ffb640 0 client.18738 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/526052649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1ba5d0 0x5603ee1c4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 1 -- 172.21.15.5:0/526052649 >> 172.21.15.5:0/526052649 conn(0x5603ee183310 msgr2=0x5603ee1bc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:49.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 1 -- 172.21.15.5:0/526052649 shutdown_connections 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 1 -- 172.21.15.5:0/526052649 wait complete. 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.032+0000 7ff57ae4a740 20 client.18738 trim_cache size 0 max 16384 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.045+0000 7ff57ae4a740 1 Processor -- start 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.045+0000 7ff57ae4a740 1 -- start start 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1ba5d0 0x5603ee1b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741afbb0 con 0x5603ee1bbb40 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741b16f0 con 0x5603ee1ba5d0 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741b0280 con 0x5603ee1c5370 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff571d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff571d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51774/0 (socket says 172.21.15.5:51774) 2024-09-11T22:32:49.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff571d9b640 1 -- 172.21.15.5:0/3925568072 learned_addr learned my addr 172.21.15.5:0/3925568072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff57259c640 1 --2- 172.21.15.5:0/3925568072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1ba5d0 0x5603ee1b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff571d9b640 1 -- 172.21.15.5:0/3925568072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 msgr2=0x5603ee1c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff571d9b640 1 --2- 172.21.15.5:0/3925568072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff571d9b640 1 -- 172.21.15.5:0/3925568072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1ba5d0 msgr2=0x5603ee1b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff571d9b640 1 --2- 172.21.15.5:0/3925568072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1ba5d0 0x5603ee1b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff571d9b640 1 -- 172.21.15.5:0/3925568072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ee1b7730 con 0x5603ee1bbb40 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff571d9b640 1 --2- 172.21.15.5:0/3925568072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1d1310 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff568015610 tx=0x7ff56800a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71005915d7b1be84 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff55bfff640 1 -- 172.21.15.5:0/3925568072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568042070 con 0x5603ee1bbb40 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff55bfff640 1 -- 172.21.15.5:0/3925568072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff568024040 con 0x5603ee1bbb40 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.046+0000 7ff55bfff640 1 -- 172.21.15.5:0/3925568072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56802e040 con 0x5603ee1bbb40 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.047+0000 7ff57ae4a740 1 -- 172.21.15.5:0/3925568072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 msgr2=0x5603ee1d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.047+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/3925568072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1d1310 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff568015610 tx=0x7ff56800a640 comp rx=0 tx=0).stop 2024-09-11T22:32:49.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.047+0000 7ff57ae4a740 1 -- 172.21.15.5:0/3925568072 shutdown_connections 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.047+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/3925568072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.047+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/3925568072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1d1310 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.047+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/3925568072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1ba5d0 0x5603ee1b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.047+0000 7ff57ae4a740 1 -- 172.21.15.5:0/3925568072 >> 172.21.15.5:0/3925568072 conn(0x5603ee183310 msgr2=0x5603ee1c3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.047+0000 7ff57ae4a740 1 -- 172.21.15.5:0/3925568072 shutdown_connections 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.047+0000 7ff57ae4a740 1 -- 172.21.15.5:0/3925568072 wait complete. 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.047+0000 7ff57ae4a740 1 -- start start 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff57ae4a740 1 Processor -- start 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1ba5d0 0x5603ee1cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff57ae4a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1be610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741b1020 con 0x5603ee1bbb40 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741b0950 con 0x5603ee1ba5d0 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff57ae4a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff5741adb90 con 0x5603ee1c5370 2024-09-11T22:32:49.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff57ae4a740 10 client.0 _ll_register_callbacks cb 0x7ff568021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff571d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff571d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51786/0 (socket says 172.21.15.5:51786) 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff571d9b640 1 -- 172.21.15.5:0/2713312813 learned_addr learned my addr 172.21.15.5:0/2713312813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff57259c640 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1ba5d0 0x5603ee1cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff571d9b640 1 -- 172.21.15.5:0/2713312813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 msgr2=0x5603ee1be610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff572d9d640 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1be610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff571d9b640 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1be610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff571d9b640 1 -- 172.21.15.5:0/2713312813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1ba5d0 msgr2=0x5603ee1cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff571d9b640 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1ba5d0 0x5603ee1cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff571d9b640 1 -- 172.21.15.5:0/2713312813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ee1beb50 con 0x5603ee1bbb40 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff571d9b640 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1cd9c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff568015780 tx=0x7ff568004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17598c3792dfcd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:49.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.048+0000 7ff559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff559ffb640 1 -- 172.21.15.5:0/2713312813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff568042070 con 0x5603ee1bbb40 2024-09-11T22:32:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff559ffb640 1 -- 172.21.15.5:0/2713312813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7ff56801e990 con 0x5603ee1bbb40 2024-09-11T22:32:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2713312813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff568024040 con 0x5603ee1bbb40 2024-09-11T22:32:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff559ffb640 1 -- 172.21.15.5:0/2713312813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7ff56802e040 con 0x5603ee1bbb40 2024-09-11T22:32:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff57ae4a740 10 client.18744 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2713312813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5603ee23ef80 con 0x5603ee1bbb40 2024-09-11T22:32:49.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff559ffb640 1 -- 172.21.15.5:0/2713312813 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff568024040 con 0x5603ee1bbb40 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff559ffb640 1 -- 172.21.15.5:0/2713312813 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7ff568012050 con 0x5603ee1bbb40 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff57ae4a740 10 client.18744 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff57ae4a740 10 client.18744 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff57ae4a740 20 client.18744 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5603ee23f5d0 0x5603ee2419f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff57ae4a740 4 client.18744 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.049+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2713312813 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603ee127d70 con 0x5603ee23f5d0 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57259c640 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5603ee23f5d0 0x5603ee2419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57259c640 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5603ee23f5d0 0x5603ee2419f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e6be0590efca8dbc server_cookie=4eabe00ab15579d4 in_seq=0 out_seq=0 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff559ffb640 10 client.18744 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff559ffb640 1 -- 172.21.15.5:0/2713312813 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603ee127d70 con 0x5603ee23f5d0 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff559ffb640 10 client.18744 handle_command_reply: tid=0 2024-09-11T22:32:49.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57ae4a740 1 client.18744 shutdown 2024-09-11T22:32:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2713312813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 msgr2=0x5603ee1cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1cd9c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7ff568015780 tx=0x7ff568004660 comp rx=0 tx=0).stop 2024-09-11T22:32:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2713312813 shutdown_connections 2024-09-11T22:32:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603ee1c5370 0x5603ee1be610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5603ee1bbb40 0x5603ee1cd9c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5603ee23f5d0 0x5603ee2419f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57ae4a740 1 --2- 172.21.15.5:0/2713312813 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5603ee1ba5d0 0x5603ee1cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff559ffb640 0 client.18744 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:32:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2713312813 >> 172.21.15.5:0/2713312813 conn(0x5603ee183310 msgr2=0x5603ee1c4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.050+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2713312813 shutdown_connections 2024-09-11T22:32:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff57ae4a740 1 -- 172.21.15.5:0/2713312813 wait complete. 2024-09-11T22:32:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff57ae4a740 20 client.18744 trim_cache size 0 max 16384 2024-09-11T22:32:49.077 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:32:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 -- 172.21.15.5:0/305864563 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff55c0a2d70 msgr2=0x7ff55c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff55c0a2d70 0x7ff55c0a5230 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff568006fd0 tx=0x7ff56800ad10 comp rx=0 tx=0).stop 2024-09-11T22:32:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 -- 172.21.15.5:0/305864563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574106f10 msgr2=0x7ff574074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574106f10 0x7ff574074f70 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff56c031a20 tx=0x7ff56c031a50 comp rx=0 tx=0).stop 2024-09-11T22:32:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 -- 172.21.15.5:0/305864563 shutdown_connections 2024-09-11T22:32:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff574106f10 0x7ff574074f70 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff55c0a2d70 0x7ff55c0a5230 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff574102790 0x7ff574074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 --2- 172.21.15.5:0/305864563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff574101de0 0x7ff5740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 -- 172.21.15.5:0/305864563 >> 172.21.15.5:0/305864563 conn(0x7ff5740fd8c0 msgr2=0x7ff5740fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 -- 172.21.15.5:0/305864563 shutdown_connections 2024-09-11T22:32:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:49.051+0000 7ff578a71640 1 -- 172.21.15.5:0/305864563 wait complete. 2024-09-11T22:32:49.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.30 MB/sec execute 427 sec latency 4910.256 ms 2024-09-11T22:32:49.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:32:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:49 smithi005 ceph-mon[30943]: pgmap v335: 97 pgs: 97 active+clean; 98 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-09-11T22:32:50.109 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:50.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:49 smithi159 ceph-mon[35534]: pgmap v335: 97 pgs: 97 active+clean; 98 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-09-11T22:32:50.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.28 MB/sec execute 428 sec latency 5910.307 ms 2024-09-11T22:32:50.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:49 smithi172 ceph-mon[35461]: pgmap v335: 97 pgs: 97 active+clean; 98 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-09-11T22:32:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/3742342799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4102ef0 msgr2=0x7f3cd410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 --2- 172.21.15.5:0/3742342799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4102ef0 0x7f3cd410da40 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009ca0 tx=0x7f3cc402a950 comp rx=0 tx=0).stop 2024-09-11T22:32:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/3742342799 shutdown_connections 2024-09-11T22:32:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 --2- 172.21.15.5:0/3742342799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4102ef0 0x7f3cd410da40 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 --2- 172.21.15.5:0/3742342799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cd4102570 0x7f3cd41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 --2- 172.21.15.5:0/3742342799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4100e20 0x7f3cd4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/3742342799 >> 172.21.15.5:0/3742342799 conn(0x7f3cd40fc7d0 msgr2=0x7f3cd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/3742342799 shutdown_connections 2024-09-11T22:32:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/3742342799 wait complete. 2024-09-11T22:32:50.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 Processor -- start 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.630+0000 7f3cdbbd9640 1 -- start start 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cdbbd9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4100e20 0x7f3cd40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cdbbd9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4102570 0x7f3cd40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cdbbd9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cd4102ef0 0x7f3cd40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cdbbd9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4110fd0 con 0x7f3cd4100e20 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cdbbd9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4110e50 con 0x7f3cd4102ef0 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cdbbd9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4111150 con 0x7f3cd4102570 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9975640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4100e20 0x7f3cd40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9975640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4100e20 0x7f3cd40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51812/0 (socket says 172.21.15.5:51812) 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9975640 1 -- 172.21.15.5:0/2342902366 learned_addr learned my addr 172.21.15.5:0/2342902366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:50.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9174640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4102570 0x7f3cd40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cda176640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cd4102ef0 0x7f3cd40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9975640 1 -- 172.21.15.5:0/2342902366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4102570 msgr2=0x7f3cd40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9975640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4102570 0x7f3cd40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9975640 1 -- 172.21.15.5:0/2342902366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cd4102ef0 msgr2=0x7f3cd40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9975640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cd4102ef0 0x7f3cd40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9975640 1 -- 172.21.15.5:0/2342902366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd41a69c0 con 0x7f3cd4100e20 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9174640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4102570 0x7f3cd40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.631+0000 7f3cd9975640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4100e20 0x7f3cd40fcc80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00d770 tx=0x7f3ccc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.632+0000 7f3cd2ffd640 1 -- 172.21.15.5:0/2342902366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc017070 con 0x7f3cd4100e20 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.632+0000 7f3cd2ffd640 1 -- 172.21.15.5:0/2342902366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ccc005500 con 0x7f3cd4100e20 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.632+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/2342902366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd41a6cb0 con 0x7f3cd4100e20 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.632+0000 7f3cd2ffd640 1 -- 172.21.15.5:0/2342902366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc004e00 con 0x7f3cd4100e20 2024-09-11T22:32:50.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.632+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/2342902366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd41a71a0 con 0x7f3cd4100e20 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.634+0000 7f3cd2ffd640 1 -- 172.21.15.5:0/2342902366 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3ccc004fa0 con 0x7f3cd4100e20 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.634+0000 7f3cddfb2740 1 Processor -- start 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.634+0000 7f3cddfb2740 1 -- start start 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aace090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aacea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aadb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56075aa342f0 con 0x56075aacfb40 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56075a9dbcf0 con 0x56075aace5d0 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56075aa302b0 con 0x56075aad9370 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cd2ffd640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3cbc0a2cf0 0x7f3cbc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cda176640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aace090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cda176640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aace090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51822/0 (socket says 172.21.15.5:51822) 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cda176640 1 -- 172.21.15.5:0/2425132392 learned_addr learned my addr 172.21.15.5:0/2425132392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cd9975640 1 --2- 172.21.15.5:0/2425132392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aacea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cd2ffd640 1 -- 172.21.15.5:0/2342902366 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f3ccc0cef70 con 0x7f3cd4100e20 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cda176640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3cbc0a2cf0 0x7f3cbc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cd9174640 1 --2- 172.21.15.5:0/2425132392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aadb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cda176640 1 -- 172.21.15.5:0/2425132392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 msgr2=0x56075aadb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cda176640 1 --2- 172.21.15.5:0/2425132392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aadb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cda176640 1 -- 172.21.15.5:0/2425132392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 msgr2=0x56075aacea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cda176640 1 --2- 172.21.15.5:0/2425132392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aacea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cda176640 1 -- 172.21.15.5:0/2425132392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075a9c03a0 con 0x56075aacfb40 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.635+0000 7f3cda176640 1 --2- 172.21.15.5:0/2425132392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aace090 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3cc402ae70 tx=0x7f3cc4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21fff723cfae709 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cbbfff640 1 -- 172.21.15.5:0/2425132392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4034040 con 0x56075aacfb40 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cbbfff640 1 -- 172.21.15.5:0/2425132392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc404d070 con 0x56075aacfb40 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cbbfff640 1 -- 172.21.15.5:0/2425132392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4030440 con 0x56075aacfb40 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cda176640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3cbc0a2cf0 0x7f3cbc0a51b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009a60 tx=0x7f3cc4047000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cddfb2740 1 -- 172.21.15.5:0/2425132392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 msgr2=0x56075aace090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/2425132392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aace090 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3cc402ae70 tx=0x7f3cc4003970 comp rx=0 tx=0).stop 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cddfb2740 1 -- 172.21.15.5:0/2425132392 shutdown_connections 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/2425132392 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aadb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/2425132392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aacea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/2425132392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aace090 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.636+0000 7f3cddfb2740 1 -- 172.21.15.5:0/2425132392 >> 172.21.15.5:0/2425132392 conn(0x56075aa97310 msgr2=0x56075aacac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.637+0000 7f3cddfb2740 1 -- 172.21.15.5:0/2425132392 shutdown_connections 2024-09-11T22:32:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.637+0000 7f3cddfb2740 1 -- 172.21.15.5:0/2425132392 wait complete. 2024-09-11T22:32:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.637+0000 7f3cddfb2740 1 -- start start 2024-09-11T22:32:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.637+0000 7f3cddfb2740 1 Processor -- start 2024-09-11T22:32:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.637+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aace5d0 0x56075aad8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.637+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aacfb40 0x56075aad5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aad6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56075aadc070 con 0x56075aace5d0 2024-09-11T22:32:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56075a9ef060 con 0x56075aacfb40 2024-09-11T22:32:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56075aadc1f0 con 0x56075aad9370 2024-09-11T22:32:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cddfb2740 10 client.0 _ll_register_callbacks cb 0x56075a7428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cd9975640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aace5d0 0x56075aad8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cd9975640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aace5d0 0x56075aad8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51838/0 (socket says 172.21.15.5:51838) 2024-09-11T22:32:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cd9975640 1 -- 172.21.15.5:0/190464177 learned_addr learned my addr 172.21.15.5:0/190464177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cd9174640 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aacfb40 0x56075aad5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cda176640 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aad6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cd9975640 1 -- 172.21.15.5:0/190464177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 msgr2=0x56075aad6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cd9975640 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aad6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cd9975640 1 -- 172.21.15.5:0/190464177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aacfb40 msgr2=0x56075aad5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cd9975640 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aacfb40 0x56075aad5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cd9975640 1 -- 172.21.15.5:0/190464177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56075a7a53c0 con 0x56075aace5d0 2024-09-11T22:32:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cda176640 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aad6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cd9975640 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aace5d0 0x56075aad8d00 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc005760 tx=0x7f3ccc09ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb02c0461732c6f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cb97fa640 1 -- 172.21.15.5:0/190464177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc020040 con 0x56075aace5d0 2024-09-11T22:32:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cb97fa640 1 -- 172.21.15.5:0/190464177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f3ccc097a60 con 0x56075aace5d0 2024-09-11T22:32:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cddfb2740 1 -- 172.21.15.5:0/190464177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc404d070 con 0x56075aace5d0 2024-09-11T22:32:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.638+0000 7f3cb97fa640 1 -- 172.21.15.5:0/190464177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3ccc09b3f0 con 0x56075aace5d0 2024-09-11T22:32:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cddfb2740 10 client.18756 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cddfb2740 1 -- 172.21.15.5:0/190464177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56075aae7af0 con 0x56075aace5d0 2024-09-11T22:32:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cb97fa640 1 -- 172.21.15.5:0/190464177 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc404d070 con 0x56075aace5d0 2024-09-11T22:32:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cb97fa640 1 -- 172.21.15.5:0/190464177 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f3ccc016170 con 0x56075aace5d0 2024-09-11T22:32:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cddfb2740 10 client.18756 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cddfb2740 10 client.18756 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cddfb2740 20 client.18756 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x56075aae8fb0 0x56075aaeb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cddfb2740 4 client.18756 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cddfb2740 1 -- 172.21.15.5:0/190464177 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56075aa41860 con 0x56075aae8fb0 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.639+0000 7f3cd9174640 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x56075aae8fb0 0x56075aaeb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.640+0000 7f3cb97fa640 10 client.18756 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.640+0000 7f3cd9174640 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x56075aae8fb0 0x56075aaeb3c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3eff9e2203cfdfbf server_cookie=246032ebbddde32f in_seq=0 out_seq=0 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.640+0000 7f3cb97fa640 1 -- 172.21.15.5:0/190464177 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56075aa41860 con 0x56075aae8fb0 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.640+0000 7f3cb97fa640 10 client.18756 handle_command_reply: tid=0 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.640+0000 7f3cddfb2740 1 client.18756 shutdown 2024-09-11T22:32:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 1 -- 172.21.15.5:0/190464177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aace5d0 msgr2=0x56075aad8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aace5d0 0x56075aad8d00 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc005760 tx=0x7f3ccc09ad50 comp rx=0 tx=0).stop 2024-09-11T22:32:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 1 -- 172.21.15.5:0/190464177 shutdown_connections 2024-09-11T22:32:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x56075aae8fb0 0x56075aaeb3c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aad6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aacfb40 0x56075aad5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/190464177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aace5d0 0x56075aad8d00 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 1 -- 172.21.15.5:0/190464177 >> 172.21.15.5:0/190464177 conn(0x56075aa97310 msgr2=0x56075aad0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cb97fa640 0 client.18756 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:32:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 1 -- 172.21.15.5:0/190464177 shutdown_connections 2024-09-11T22:32:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 1 -- 172.21.15.5:0/190464177 wait complete. 2024-09-11T22:32:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.641+0000 7f3cddfb2740 20 client.18756 trim_cache size 0 max 16384 2024-09-11T22:32:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cddfb2740 1 Processor -- start 2024-09-11T22:32:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cddfb2740 1 -- start start 2024-09-11T22:32:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aaeafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aac9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aad1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41afb60 con 0x56075aacfb40 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41b16a0 con 0x56075aace5d0 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41b0230 con 0x56075aad9370 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cd9174640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aac9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cd9174640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aac9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51846/0 (socket says 172.21.15.5:51846) 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cd9174640 1 -- 172.21.15.5:0/12514781 learned_addr learned my addr 172.21.15.5:0/12514781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cd9975640 1 --2- 172.21.15.5:0/12514781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aaeafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.654+0000 7f3cda176640 1 --2- 172.21.15.5:0/12514781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aad1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.655+0000 7f3cd9174640 1 -- 172.21.15.5:0/12514781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 msgr2=0x56075aad1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.655+0000 7f3cd9174640 1 --2- 172.21.15.5:0/12514781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aad1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.655+0000 7f3cd9174640 1 -- 172.21.15.5:0/12514781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 msgr2=0x56075aaeafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.655+0000 7f3cd9174640 1 --2- 172.21.15.5:0/12514781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aaeafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.655+0000 7f3cd9174640 1 -- 172.21.15.5:0/12514781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56075aacb730 con 0x56075aacfb40 2024-09-11T22:32:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.655+0000 7f3cd9975640 1 --2- 172.21.15.5:0/12514781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aaeafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.655+0000 7f3cd9174640 1 --2- 172.21.15.5:0/12514781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aac9bf0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x56075aacf070 tx=0x7f3cc8014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafaa3914e2a9023 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.655+0000 7f3cbb7fe640 1 -- 172.21.15.5:0/12514781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8014190 con 0x56075aacfb40 2024-09-11T22:32:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.655+0000 7f3cbb7fe640 1 -- 172.21.15.5:0/12514781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc8012930 con 0x56075aacfb40 2024-09-11T22:32:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.655+0000 7f3cbb7fe640 1 -- 172.21.15.5:0/12514781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc801d940 con 0x56075aacfb40 2024-09-11T22:32:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 -- 172.21.15.5:0/12514781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 msgr2=0x56075aac9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/12514781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aac9bf0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x56075aacf070 tx=0x7f3cc8014840 comp rx=0 tx=0).stop 2024-09-11T22:32:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 -- 172.21.15.5:0/12514781 shutdown_connections 2024-09-11T22:32:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/12514781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aad9370 0x56075aad1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/12514781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aacfb40 0x56075aac9bf0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/12514781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aaeafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 -- 172.21.15.5:0/12514781 >> 172.21.15.5:0/12514781 conn(0x56075aa97310 msgr2=0x56075aac94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 -- 172.21.15.5:0/12514781 shutdown_connections 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 -- 172.21.15.5:0/12514781 wait complete. 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 -- start start 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.656+0000 7f3cddfb2740 1 Processor -- start 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aadd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aacfb40 0x56075aadd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cddfb2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aad9370 0x56075aaddc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41b0fd0 con 0x56075aad9370 2024-09-11T22:32:50.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41b0900 con 0x56075aace5d0 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cddfb2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd41adb40 con 0x56075aacfb40 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cddfb2740 10 client.0 _ll_register_callbacks cb 0x7f3ccc097000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cda176640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aad9370 0x56075aaddc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cda176640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aad9370 0x56075aaddc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51850/0 (socket says 172.21.15.5:51850) 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cda176640 1 -- 172.21.15.5:0/803373544 learned_addr learned my addr 172.21.15.5:0/803373544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cd9174640 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aacfb40 0x56075aadd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cd9975640 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aadd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cda176640 1 -- 172.21.15.5:0/803373544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aacfb40 msgr2=0x56075aadd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cda176640 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aacfb40 0x56075aadd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cda176640 1 -- 172.21.15.5:0/803373544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 msgr2=0x56075aadd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cda176640 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aadd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cda176640 1 -- 172.21.15.5:0/803373544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56075ab52c10 con 0x56075aad9370 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.657+0000 7f3cd9975640 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aadd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cda176640 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aad9370 0x56075aaddc20 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3cc402ffe0 tx=0x7f3cc4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5068a86accc777 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cb97fa640 1 -- 172.21.15.5:0/803373544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4058070 con 0x56075aad9370 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cb97fa640 1 -- 172.21.15.5:0/803373544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f3cc4030970 con 0x56075aad9370 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cddfb2740 1 -- 172.21.15.5:0/803373544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cc8012930 con 0x56075aad9370 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cb97fa640 1 -- 172.21.15.5:0/803373544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3cc4030440 con 0x56075aad9370 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cddfb2740 10 client.18762 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cddfb2740 1 -- 172.21.15.5:0/803373544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x56075ab53ec0 con 0x56075aad9370 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cb97fa640 1 -- 172.21.15.5:0/803373544 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cc8012930 con 0x56075aad9370 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cb97fa640 1 -- 172.21.15.5:0/803373544 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f3cc4065020 con 0x56075aad9370 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cddfb2740 10 client.18762 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cddfb2740 10 client.18762 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cddfb2740 20 client.18762 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x56075ab54210 0x56075ab56630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cddfb2740 4 client.18762 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.658+0000 7f3cddfb2740 1 -- 172.21.15.5:0/803373544 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56075aa3bd70 con 0x56075ab54210 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cd9975640 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x56075ab54210 0x56075ab56630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cd9975640 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x56075ab54210 0x56075ab56630 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8465895e3aa72285 server_cookie=ea4de94a98ea5182 in_seq=0 out_seq=0 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cb97fa640 10 client.18762 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cb97fa640 1 -- 172.21.15.5:0/803373544 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56075aa3bd70 con 0x56075ab54210 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cb97fa640 10 client.18762 handle_command_reply: tid=0 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cddfb2740 1 client.18762 shutdown 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cddfb2740 1 -- 172.21.15.5:0/803373544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aad9370 msgr2=0x56075aaddc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aad9370 0x56075aaddc20 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3cc402ffe0 tx=0x7f3cc4009800 comp rx=0 tx=0).stop 2024-09-11T22:32:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cddfb2740 1 -- 172.21.15.5:0/803373544 shutdown_connections 2024-09-11T22:32:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x56075ab54210 0x56075ab56630 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56075aad9370 0x56075aaddc20 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56075aacfb40 0x56075aadd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cb97fa640 0 client.18762 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:32:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cddfb2740 1 --2- 172.21.15.5:0/803373544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56075aace5d0 0x56075aadd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.659+0000 7f3cddfb2740 1 -- 172.21.15.5:0/803373544 >> 172.21.15.5:0/803373544 conn(0x56075aa97310 msgr2=0x56075aac9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cddfb2740 1 -- 172.21.15.5:0/803373544 shutdown_connections 2024-09-11T22:32:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cddfb2740 1 -- 172.21.15.5:0/803373544 wait complete. 2024-09-11T22:32:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cddfb2740 20 client.18762 trim_cache size 0 max 16384 2024-09-11T22:32:50.674 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:32:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/2342902366 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3cbc0a2cf0 msgr2=0x7f3cbc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3cbc0a2cf0 0x7f3cbc0a51b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4009a60 tx=0x7f3cc4047000 comp rx=0 tx=0).stop 2024-09-11T22:32:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/2342902366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4100e20 msgr2=0x7f3cd40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4100e20 0x7f3cd40fcc80 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00d770 tx=0x7f3ccc00dc30 comp rx=0 tx=0).stop 2024-09-11T22:32:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/2342902366 shutdown_connections 2024-09-11T22:32:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3cbc0a2cf0 0x7f3cbc0a51b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cd4102ef0 0x7f3cd40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3cd4102570 0x7f3cd40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 --2- 172.21.15.5:0/2342902366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cd4100e20 0x7f3cd40fcc80 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/2342902366 >> 172.21.15.5:0/2342902366 conn(0x7f3cd40fc7d0 msgr2=0x7f3cd406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/2342902366 shutdown_connections 2024-09-11T22:32:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:50.660+0000 7f3cdbbd9640 1 -- 172.21.15.5:0/2342902366 wait complete. 2024-09-11T22:32:51.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.27 MB/sec execute 429 sec latency 6910.364 ms 2024-09-11T22:32:51.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:32:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.579+0000 7fd08221e640 1 -- 172.21.15.5:0/3674532636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c107b70 msgr2=0x7fd07c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.579+0000 7fd08221e640 1 --2- 172.21.15.5:0/3674532636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c107b70 0x7fd07c107f50 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009930 tx=0x7fd06c02a420 comp rx=0 tx=0).stop 2024-09-11T22:32:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.580+0000 7fd08221e640 1 -- 172.21.15.5:0/3674532636 shutdown_connections 2024-09-11T22:32:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.580+0000 7fd08221e640 1 --2- 172.21.15.5:0/3674532636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd07c102470 0x7fd07c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.580+0000 7fd08221e640 1 --2- 172.21.15.5:0/3674532636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd07c101ad0 0x7fd07c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.580+0000 7fd08221e640 1 --2- 172.21.15.5:0/3674532636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c107b70 0x7fd07c107f50 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.580+0000 7fd08221e640 1 -- 172.21.15.5:0/3674532636 >> 172.21.15.5:0/3674532636 conn(0x7fd07c0fd7d0 msgr2=0x7fd07c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.580+0000 7fd08221e640 1 -- 172.21.15.5:0/3674532636 shutdown_connections 2024-09-11T22:32:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.580+0000 7fd08221e640 1 -- 172.21.15.5:0/3674532636 wait complete. 2024-09-11T22:32:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.580+0000 7fd08221e640 1 Processor -- start 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.580+0000 7fd08221e640 1 -- start start 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd08221e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd07c101ad0 0x7fd07c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd08221e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd07c102470 0x7fd07c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd08221e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c107b70 0x7fd07c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd08221e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c111ee0 con 0x7fd07c107b70 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd08221e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c111d60 con 0x7fd07c101ad0 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd08221e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c112060 con 0x7fd07c102470 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c107b70 0x7fd07c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c107b70 0x7fd07c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47594/0 (socket says 172.21.15.5:47594) 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07bfff640 1 -- 172.21.15.5:0/2166934445 learned_addr learned my addr 172.21.15.5:0/2166934445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:51.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd07c101ad0 0x7fd07c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07affd640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd07c102470 0x7fd07c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07bfff640 1 -- 172.21.15.5:0/2166934445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd07c102470 msgr2=0x7fd07c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07bfff640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd07c102470 0x7fd07c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07bfff640 1 -- 172.21.15.5:0/2166934445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd07c101ad0 msgr2=0x7fd07c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07bfff640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd07c101ad0 0x7fd07c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07bfff640 1 -- 172.21.15.5:0/2166934445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07c19d360 con 0x7fd07c107b70 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.581+0000 7fd07bfff640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c107b70 0x7fd07c19cc30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd07400cad0 tx=0x7fd07400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.582+0000 7fd078ff9640 1 -- 172.21.15.5:0/2166934445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07400ae20 con 0x7fd07c107b70 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.582+0000 7fd078ff9640 1 -- 172.21.15.5:0/2166934445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd074005020 con 0x7fd07c107b70 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.582+0000 7fd078ff9640 1 -- 172.21.15.5:0/2166934445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07400f6f0 con 0x7fd07c107b70 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.582+0000 7fd08221e640 1 -- 172.21.15.5:0/2166934445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07c19d650 con 0x7fd07c107b70 2024-09-11T22:32:51.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.582+0000 7fd08221e640 1 -- 172.21.15.5:0/2166934445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07c1a4ee0 con 0x7fd07c107b70 2024-09-11T22:32:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.584+0000 7fd085090740 1 Processor -- start 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.584+0000 7fd078ff9640 1 -- 172.21.15.5:0/2166934445 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd0740051d0 con 0x7fd07c107b70 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd085090740 1 -- start start 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd085090740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823445af0 0x55c823445ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd085090740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c823450770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd085090740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823450cb0 0x55c8234530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd085090740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c8233aa2f0 con 0x55c823446410 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd085090740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c823352270 con 0x55c823450cb0 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd085090740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c8233a62b0 con 0x55c823445af0 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd07b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c823450770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd07b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c823450770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47608/0 (socket says 172.21.15.5:47608) 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd07b7fe640 1 -- 172.21.15.5:0/3282780231 learned_addr learned my addr 172.21.15.5:0/3282780231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd07affd640 1 --2- 172.21.15.5:0/3282780231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823450cb0 0x55c8234530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd07affd640 1 --2- 172.21.15.5:0/3282780231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823445af0 0x55c823445ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd07b7fe640 1 -- 172.21.15.5:0/3282780231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823445af0 msgr2=0x55c823445ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/3282780231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823445af0 0x55c823445ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd07b7fe640 1 -- 172.21.15.5:0/3282780231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823450cb0 msgr2=0x55c8234530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/3282780231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823450cb0 0x55c8234530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd07b7fe640 1 -- 172.21.15.5:0/3282780231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c823351340 con 0x55c823446410 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.585+0000 7fd078ff9640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0640a2da0 0x7fd0640a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.586+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/3282780231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c823450770 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009930 tx=0x7fd06c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35836825b6802e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.586+0000 7fd07affd640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0640a2da0 0x7fd0640a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.586+0000 7fd078ff9640 1 -- 172.21.15.5:0/2166934445 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fd0740cf590 con 0x7fd07c107b70 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.586+0000 7fd061ffb640 1 -- 172.21.15.5:0/3282780231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c03f070 con 0x55c823446410 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.586+0000 7fd061ffb640 1 -- 172.21.15.5:0/3282780231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd06c0317b0 con 0x55c823446410 2024-09-11T22:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.586+0000 7fd061ffb640 1 -- 172.21.15.5:0/3282780231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c035070 con 0x55c823446410 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 -- 172.21.15.5:0/3282780231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 msgr2=0x55c823450770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd07affd640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0640a2da0 0x7fd0640a5260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd07000d010 tx=0x7fd070016040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 --2- 172.21.15.5:0/3282780231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c823450770 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fd06c009930 tx=0x7fd06c032ce0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 -- 172.21.15.5:0/3282780231 shutdown_connections 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 --2- 172.21.15.5:0/3282780231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823450cb0 0x55c8234530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 --2- 172.21.15.5:0/3282780231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c823450770 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 --2- 172.21.15.5:0/3282780231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823445af0 0x55c823445ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 -- 172.21.15.5:0/3282780231 >> 172.21.15.5:0/3282780231 conn(0x55c823442060 msgr2=0x55c823442440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 -- 172.21.15.5:0/3282780231 shutdown_connections 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 -- 172.21.15.5:0/3282780231 wait complete. 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 -- start start 2024-09-11T22:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.587+0000 7fd085090740 1 Processor -- start 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd085090740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823445af0 0x55c82344dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd085090740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 0x55c82344e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd085090740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823450cb0 0x55c82344e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd085090740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c823453880 con 0x55c823450cb0 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd085090740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c823333290 con 0x55c823446410 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd085090740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c823453a00 con 0x55c823445af0 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd085090740 10 client.0 _ll_register_callbacks cb 0x55c823364140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823450cb0 0x55c82344e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823450cb0 0x55c82344e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47618/0 (socket says 172.21.15.5:47618) 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07bfff640 1 -- 172.21.15.5:0/4234369721 learned_addr learned my addr 172.21.15.5:0/4234369721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823445af0 0x55c82344dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07affd640 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 0x55c82344e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07bfff640 1 -- 172.21.15.5:0/4234369721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823445af0 msgr2=0x55c82344dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07bfff640 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823445af0 0x55c82344dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07bfff640 1 -- 172.21.15.5:0/4234369721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 msgr2=0x55c82344e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07bfff640 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 0x55c82344e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07bfff640 1 -- 172.21.15.5:0/4234369721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8230bd210 con 0x55c823450cb0 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd07bfff640 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823450cb0 0x55c82344e670 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd0740a4150 tx=0x7fd07409c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4821a54c13d52a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd05affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd05affd640 1 -- 172.21.15.5:0/4234369721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07401d040 con 0x55c823450cb0 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd05affd640 1 -- 172.21.15.5:0/4234369721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fd07409ae60 con 0x55c823450cb0 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.588+0000 7fd085090740 1 -- 172.21.15.5:0/4234369721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd06c0317b0 con 0x55c823450cb0 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd05affd640 1 -- 172.21.15.5:0/4234369721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd07409b060 con 0x55c823450cb0 2024-09-11T22:32:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd085090740 10 client.18774 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd085090740 1 -- 172.21.15.5:0/4234369721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c823189620 con 0x55c823450cb0 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd05affd640 1 -- 172.21.15.5:0/4234369721 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd06c0317b0 con 0x55c823450cb0 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd05affd640 1 -- 172.21.15.5:0/4234369721 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fd074097940 con 0x55c823450cb0 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd085090740 10 client.18774 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd085090740 10 client.18774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd085090740 20 client.18774 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd085090740 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c823460cb0 0x55c8234630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd085090740 4 client.18774 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd085090740 1 -- 172.21.15.5:0/4234369721 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8233b7a20 con 0x55c823460cb0 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.589+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c823460cb0 0x55c8234630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.590+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c823460cb0 0x55c8234630e0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64e757b78cf841a server_cookie=b70a92a2d21a9510 in_seq=0 out_seq=0 2024-09-11T22:32:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.590+0000 7fd05affd640 10 client.18774 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd05affd640 1 -- 172.21.15.5:0/4234369721 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fd06c04ee80 con 0x55c823460cb0 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd05affd640 10 client.18774 handle_command_reply: tid=0 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 client.18774 shutdown 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 -- 172.21.15.5:0/4234369721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823450cb0 msgr2=0x55c82344e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823450cb0 0x55c82344e670 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fd0740a4150 tx=0x7fd07409c250 comp rx=0 tx=0).stop 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 -- 172.21.15.5:0/4234369721 shutdown_connections 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c823460cb0 0x55c8234630e0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823450cb0 0x55c82344e670 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 0x55c82344e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 --2- 172.21.15.5:0/4234369721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823445af0 0x55c82344dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd05affd640 0 client.18774 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 -- 172.21.15.5:0/4234369721 >> 172.21.15.5:0/4234369721 conn(0x55c823442060 msgr2=0x55c823447d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 -- 172.21.15.5:0/4234369721 shutdown_connections 2024-09-11T22:32:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 1 -- 172.21.15.5:0/4234369721 wait complete. 2024-09-11T22:32:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.591+0000 7fd085090740 20 client.18774 trim_cache size 0 max 16384 2024-09-11T22:32:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.606+0000 7fd085090740 1 Processor -- start 2024-09-11T22:32:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.606+0000 7fd085090740 1 -- start start 2024-09-11T22:32:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.606+0000 7fd085090740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823445af0 0x55c82344e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.606+0000 7fd085090740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c82344fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.606+0000 7fd085090740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 0x55c8234689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.606+0000 7fd085090740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1ad9a0 con 0x55c823446410 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.606+0000 7fd085090740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1af4e0 con 0x55c823445af0 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.606+0000 7fd085090740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1ae070 con 0x55c823450cb0 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c82344fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c82344fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47632/0 (socket says 172.21.15.5:47632) 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07affd640 1 -- 172.21.15.5:0/2806066776 learned_addr learned my addr 172.21.15.5:0/2806066776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/2806066776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823445af0 0x55c82344e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07bfff640 1 --2- 172.21.15.5:0/2806066776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 0x55c8234689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07affd640 1 -- 172.21.15.5:0/2806066776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 msgr2=0x55c8234689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:51.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07affd640 1 --2- 172.21.15.5:0/2806066776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 0x55c8234689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07affd640 1 -- 172.21.15.5:0/2806066776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823445af0 msgr2=0x55c82344e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07affd640 1 --2- 172.21.15.5:0/2806066776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823445af0 0x55c82344e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07affd640 1 -- 172.21.15.5:0/2806066776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c823450740 con 0x55c823446410 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07bfff640 1 --2- 172.21.15.5:0/2806066776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 0x55c8234689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd07affd640 1 --2- 172.21.15.5:0/2806066776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c82344fb60 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x55c823448690 tx=0x7fd07000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12e9c067153e4d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd0617fa640 1 -- 172.21.15.5:0/2806066776 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07001b070 con 0x55c823446410 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd0617fa640 1 -- 172.21.15.5:0/2806066776 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0700047a0 con 0x55c823446410 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.607+0000 7fd0617fa640 1 -- 172.21.15.5:0/2806066776 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07000fb90 con 0x55c823446410 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.608+0000 7fd085090740 1 -- 172.21.15.5:0/2806066776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 msgr2=0x55c82344fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.608+0000 7fd085090740 1 --2- 172.21.15.5:0/2806066776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c82344fb60 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x55c823448690 tx=0x7fd07000c450 comp rx=0 tx=0).stop 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.608+0000 7fd085090740 1 -- 172.21.15.5:0/2806066776 shutdown_connections 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.608+0000 7fd085090740 1 --2- 172.21.15.5:0/2806066776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 0x55c8234689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.608+0000 7fd085090740 1 --2- 172.21.15.5:0/2806066776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823446410 0x55c82344fb60 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.608+0000 7fd085090740 1 --2- 172.21.15.5:0/2806066776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823445af0 0x55c82344e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.608+0000 7fd085090740 1 -- 172.21.15.5:0/2806066776 >> 172.21.15.5:0/2806066776 conn(0x55c823442060 msgr2=0x55c82344f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.608+0000 7fd085090740 1 -- 172.21.15.5:0/2806066776 shutdown_connections 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.608+0000 7fd085090740 1 -- 172.21.15.5:0/2806066776 wait complete. 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.608+0000 7fd085090740 1 -- start start 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd085090740 1 Processor -- start 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd085090740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823445af0 0x55c823459580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd085090740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 0x55c82344aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd085090740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 0x55c82344b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd085090740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1aee10 con 0x55c823445af0 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd085090740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1ae740 con 0x55c823446410 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd085090740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd07c1ab980 con 0x55c823450cb0 2024-09-11T22:32:51.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd085090740 10 client.0 _ll_register_callbacks cb 0x7fd074097cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd07b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823445af0 0x55c823459580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd07b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823445af0 0x55c823459580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47646/0 (socket says 172.21.15.5:47646) 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd07b7fe640 1 -- 172.21.15.5:0/2720588443 learned_addr learned my addr 172.21.15.5:0/2720588443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd07affd640 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 0x55c82344aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd07b7fe640 1 -- 172.21.15.5:0/2720588443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 msgr2=0x55c82344b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.609+0000 7fd07bfff640 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 0x55c82344b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 0x55c82344b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd07b7fe640 1 -- 172.21.15.5:0/2720588443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 msgr2=0x55c82344aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 0x55c82344aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd07b7fe640 1 -- 172.21.15.5:0/2720588443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8234ca9c0 con 0x55c823445af0 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd07affd640 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 0x55c82344aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd07bfff640 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 0x55c82344b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:51.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd07b7fe640 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823445af0 0x55c823459580 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd06c02a940 tx=0x7fd06c031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ca01b5ff4663ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd05affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd05affd640 1 -- 172.21.15.5:0/2720588443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c051020 con 0x55c823445af0 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd05affd640 1 -- 172.21.15.5:0/2720588443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fd06c0055b0 con 0x55c823445af0 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd085090740 1 -- 172.21.15.5:0/2720588443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0700047a0 con 0x55c823445af0 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd05affd640 1 -- 172.21.15.5:0/2720588443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fd06c05c050 con 0x55c823445af0 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd085090740 10 client.18780 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd085090740 1 -- 172.21.15.5:0/2720588443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55c8234cbae0 con 0x55c823445af0 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.610+0000 7fd05affd640 1 -- 172.21.15.5:0/2720588443 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0700047a0 con 0x55c823445af0 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.611+0000 7fd05affd640 1 -- 172.21.15.5:0/2720588443 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fd06c003ab0 con 0x55c823445af0 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.611+0000 7fd085090740 10 client.18780 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.611+0000 7fd085090740 10 client.18780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:51.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.611+0000 7fd085090740 20 client.18780 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.611+0000 7fd085090740 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8234cbe20 0x55c8234ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.611+0000 7fd085090740 4 client.18780 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:32:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.611+0000 7fd085090740 1 -- 172.21.15.5:0/2720588443 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8233b1d70 con 0x55c8234cbe20 2024-09-11T22:32:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.611+0000 7fd07affd640 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8234cbe20 0x55c8234ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.611+0000 7fd05affd640 10 client.18780 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:51.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.611+0000 7fd07affd640 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8234cbe20 0x55c8234ce210 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b8957d8fdc31d3 server_cookie=422e19d5388cc10d in_seq=0 out_seq=0 2024-09-11T22:32:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd05affd640 1 -- 172.21.15.5:0/2720588443 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c8233b1d70 con 0x55c8234cbe20 2024-09-11T22:32:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd05affd640 10 client.18780 handle_command_reply: tid=0 2024-09-11T22:32:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd085090740 1 client.18780 shutdown 2024-09-11T22:32:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd085090740 1 -- 172.21.15.5:0/2720588443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823445af0 msgr2=0x55c823459580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd085090740 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823445af0 0x55c823459580 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fd06c02a940 tx=0x7fd06c031b50 comp rx=0 tx=0).stop 2024-09-11T22:32:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd085090740 1 -- 172.21.15.5:0/2720588443 shutdown_connections 2024-09-11T22:32:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd085090740 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8234cbe20 0x55c8234ce210 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd085090740 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c823450cb0 0x55c82344b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd085090740 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c823446410 0x55c82344aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd085090740 1 --2- 172.21.15.5:0/2720588443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c823445af0 0x55c823459580 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd085090740 1 -- 172.21.15.5:0/2720588443 >> 172.21.15.5:0/2720588443 conn(0x55c823442060 msgr2=0x55c823447550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.683+0000 7fd05affd640 0 client.18780 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd085090740 1 -- 172.21.15.5:0/2720588443 shutdown_connections 2024-09-11T22:32:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd085090740 1 -- 172.21.15.5:0/2720588443 wait complete. 2024-09-11T22:32:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd085090740 20 client.18780 trim_cache size 0 max 16384 2024-09-11T22:32:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:32:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:32:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:32:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:32:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd08221e640 1 -- 172.21.15.5:0/2166934445 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0640a2da0 msgr2=0x7fd0640a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd08221e640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0640a2da0 0x7fd0640a5260 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd07000d010 tx=0x7fd070016040 comp rx=0 tx=0).stop 2024-09-11T22:32:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd08221e640 1 -- 172.21.15.5:0/2166934445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c107b70 msgr2=0x7fd07c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd08221e640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c107b70 0x7fd07c19cc30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fd07400cad0 tx=0x7fd07400cf90 comp rx=0 tx=0).stop 2024-09-11T22:32:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd08221e640 1 -- 172.21.15.5:0/2166934445 shutdown_connections 2024-09-11T22:32:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd08221e640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd07c107b70 0x7fd07c19cc30 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd08221e640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd07c102470 0x7fd07c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd08221e640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0640a2da0 0x7fd0640a5260 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd08221e640 1 --2- 172.21.15.5:0/2166934445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd07c101ad0 0x7fd07c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.684+0000 7fd08221e640 1 -- 172.21.15.5:0/2166934445 >> 172.21.15.5:0/2166934445 conn(0x7fd07c0fd7d0 msgr2=0x7fd07c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.685+0000 7fd08221e640 1 -- 172.21.15.5:0/2166934445 shutdown_connections 2024-09-11T22:32:51.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:32:51.685+0000 7fd08221e640 1 -- 172.21.15.5:0/2166934445 wait complete. 2024-09-11T22:32:51.851 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:52.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:51 smithi005 ceph-mon[30943]: pgmap v336: 97 pgs: 97 active+clean; 98 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 133 op/s 2024-09-11T22:32:52.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:51 smithi159 ceph-mon[35534]: pgmap v336: 97 pgs: 97 active+clean; 98 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 133 op/s 2024-09-11T22:32:52.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.25 MB/sec execute 430 sec latency 7910.409 ms 2024-09-11T22:32:52.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 -- 172.21.15.5:0/918919120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93d4074590 msgr2=0x7f93d4074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 --2- 172.21.15.5:0/918919120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93d4074590 0x7f93d4074970 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009900 tx=0x7f93d002a480 comp rx=0 tx=0).stop 2024-09-11T22:32:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 -- 172.21.15.5:0/918919120 shutdown_connections 2024-09-11T22:32:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 --2- 172.21.15.5:0/918919120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d4112b90 0x7f93d4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 --2- 172.21.15.5:0/918919120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93d4074eb0 0x7f93d4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 --2- 172.21.15.5:0/918919120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93d4074590 0x7f93d4074970 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 -- 172.21.15.5:0/918919120 >> 172.21.15.5:0/918919120 conn(0x7f93d40fd8c0 msgr2=0x7f93d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 -- 172.21.15.5:0/918919120 shutdown_connections 2024-09-11T22:32:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 -- 172.21.15.5:0/918919120 wait complete. 2024-09-11T22:32:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 Processor -- start 2024-09-11T22:32:52.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.341+0000 7f93dbc74640 1 -- start start 2024-09-11T22:32:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93dbc74640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d4074590 0x7f93d41a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93dbc74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93d4074eb0 0x7f93d41a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93dbc74640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93d4112b90 0x7f93d419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93dbc74640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93d41185c0 con 0x7f93d4074eb0 2024-09-11T22:32:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93dbc74640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93d4118440 con 0x7f93d4112b90 2024-09-11T22:32:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93dbc74640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93d4118740 con 0x7f93d4074590 2024-09-11T22:32:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93d920f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93d4074eb0 0x7f93d41a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93d920f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93d4074eb0 0x7f93d41a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47672/0 (socket says 172.21.15.5:47672) 2024-09-11T22:32:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93d920f640 1 -- 172.21.15.5:0/440544404 learned_addr learned my addr 172.21.15.5:0/440544404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:52.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93d9a10640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d4074590 0x7f93d41a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93da211640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93d4112b90 0x7f93d419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93d920f640 1 -- 172.21.15.5:0/440544404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d4074590 msgr2=0x7f93d41a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:52.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93d920f640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d4074590 0x7f93d41a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93d920f640 1 -- 172.21.15.5:0/440544404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93d4112b90 msgr2=0x7f93d419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:52.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93d920f640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93d4112b90 0x7f93d419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93d920f640 1 -- 172.21.15.5:0/440544404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d419efe0 con 0x7f93d4074eb0 2024-09-11T22:32:52.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93da211640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93d4112b90 0x7f93d419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:52.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.342+0000 7f93d920f640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93d4074eb0 0x7f93d41a4c30 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f93c400d950 tx=0x7f93c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.343+0000 7f93caffd640 1 -- 172.21.15.5:0/440544404 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4017030 con 0x7f93d4074eb0 2024-09-11T22:32:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.343+0000 7f93dbc74640 1 -- 172.21.15.5:0/440544404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d419f2d0 con 0x7f93d4074eb0 2024-09-11T22:32:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.343+0000 7f93caffd640 1 -- 172.21.15.5:0/440544404 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c4004020 con 0x7f93d4074eb0 2024-09-11T22:32:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.343+0000 7f93caffd640 1 -- 172.21.15.5:0/440544404 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c40047a0 con 0x7f93d4074eb0 2024-09-11T22:32:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.343+0000 7f93dbc74640 1 -- 172.21.15.5:0/440544404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d41ab5c0 con 0x7f93d4074eb0 2024-09-11T22:32:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.345+0000 7f93caffd640 1 -- 172.21.15.5:0/440544404 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f93c40041d0 con 0x7f93d4074eb0 2024-09-11T22:32:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.345+0000 7f93de04d740 1 Processor -- start 2024-09-11T22:32:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.345+0000 7f93de04d740 1 -- start start 2024-09-11T22:32:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93de04d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 0x55da52fe4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93de04d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe45d0 0x55da52fe4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93de04d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fef370 0x55da52ff1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93de04d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55da52f4a2f0 con 0x55da52fef370 2024-09-11T22:32:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93de04d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55da52ef1cf0 con 0x55da52fe5b40 2024-09-11T22:32:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93caffd640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93c00a2cf0 0x7f93c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93de04d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55da52f462b0 con 0x55da52fe45d0 2024-09-11T22:32:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93da211640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93c00a2cf0 0x7f93c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93d920f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fef370 0x55da52ff1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93d9a10640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe45d0 0x55da52fe4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93d9a10640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe45d0 0x55da52fe4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:50172/0 (socket says 172.21.15.5:50172) 2024-09-11T22:32:52.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93da211640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 0x55da52fe4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93d9a10640 1 -- 172.21.15.5:0/541721003 learned_addr learned my addr 172.21.15.5:0/541721003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93caffd640 1 -- 172.21.15.5:0/440544404 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f93c40cef90 con 0x7f93d4074eb0 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93d920f640 1 -- 172.21.15.5:0/541721003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe45d0 msgr2=0x55da52fe4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93d920f640 1 --2- 172.21.15.5:0/541721003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe45d0 0x55da52fe4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93d920f640 1 -- 172.21.15.5:0/541721003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 msgr2=0x55da52fe4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93d920f640 1 --2- 172.21.15.5:0/541721003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 0x55da52fe4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93d920f640 1 -- 172.21.15.5:0/541721003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da52ed63a0 con 0x55da52fef370 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.346+0000 7f93da211640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93c00a2cf0 0x7f93c00a51b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f93cc005c40 tx=0x7f93cc00ac50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.347+0000 7f93d920f640 1 --2- 172.21.15.5:0/541721003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fef370 0x55da52ff1760 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f93c40cc640 tx=0x7f93c406d3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c30feac589ef0f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.347+0000 7f93bffff640 1 -- 172.21.15.5:0/541721003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c409ad70 con 0x55da52fef370 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.347+0000 7f93bffff640 1 -- 172.21.15.5:0/541721003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c409bb10 con 0x55da52fef370 2024-09-11T22:32:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.347+0000 7f93bffff640 1 -- 172.21.15.5:0/541721003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c40c7530 con 0x55da52fef370 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.347+0000 7f93de04d740 1 -- 172.21.15.5:0/541721003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fef370 msgr2=0x55da52ff1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.347+0000 7f93de04d740 1 --2- 172.21.15.5:0/541721003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fef370 0x55da52ff1760 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f93c40cc640 tx=0x7f93c406d3d0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.348+0000 7f93de04d740 1 -- 172.21.15.5:0/541721003 shutdown_connections 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.348+0000 7f93de04d740 1 --2- 172.21.15.5:0/541721003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fef370 0x55da52ff1760 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.348+0000 7f93de04d740 1 --2- 172.21.15.5:0/541721003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe45d0 0x55da52fe4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.348+0000 7f93de04d740 1 --2- 172.21.15.5:0/541721003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 0x55da52fe4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.348+0000 7f93de04d740 1 -- 172.21.15.5:0/541721003 >> 172.21.15.5:0/541721003 conn(0x55da52fad310 msgr2=0x55da52fe0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.348+0000 7f93de04d740 1 -- 172.21.15.5:0/541721003 shutdown_connections 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.348+0000 7f93de04d740 1 -- 172.21.15.5:0/541721003 wait complete. 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.348+0000 7f93de04d740 1 -- start start 2024-09-11T22:32:52.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.348+0000 7f93de04d740 1 Processor -- start 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93de04d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe45d0 0x55da52feee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93de04d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe5b40 0x55da52fec0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93de04d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 0x55da52fec860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93de04d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55da52ff2070 con 0x55da52fe5b40 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93de04d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55da52f05060 con 0x55da52fe45d0 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93de04d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55da52ff21f0 con 0x55da52fef370 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93de04d740 10 client.0 _ll_register_callbacks cb 0x55da52c588c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93d920f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe5b40 0x55da52fec0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93d9a10640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe45d0 0x55da52feee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93d9a10640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe45d0 0x55da52feee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49740/0 (socket says 172.21.15.5:49740) 2024-09-11T22:32:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93d9a10640 1 -- 172.21.15.5:0/2600997113 learned_addr learned my addr 172.21.15.5:0/2600997113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93da211640 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 0x55da52fec860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93da211640 1 -- 172.21.15.5:0/2600997113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe45d0 msgr2=0x55da52feee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93da211640 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe45d0 0x55da52feee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.349+0000 7f93da211640 1 -- 172.21.15.5:0/2600997113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe5b40 msgr2=0x55da52fec0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.350+0000 7f93da211640 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe5b40 0x55da52fec0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.350+0000 7f93da211640 1 -- 172.21.15.5:0/2600997113 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da52cbb3c0 con 0x55da52fef370 2024-09-11T22:32:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.350+0000 7f93d920f640 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe5b40 0x55da52fec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.350+0000 7f93da211640 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 0x55da52fec860 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93d4076600 tx=0x7f93cc019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b209cd25fad282cc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.350+0000 7f93bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.350+0000 7f93bd7fa640 1 -- 172.21.15.5:0/2600997113 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc023070 con 0x55da52fef370 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.350+0000 7f93de04d740 1 -- 172.21.15.5:0/2600997113 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c409bb10 con 0x55da52fef370 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.350+0000 7f93bd7fa640 1 -- 172.21.15.5:0/2600997113 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f93cc011e00 con 0x55da52fef370 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93bd7fa640 1 -- 172.21.15.5:0/2600997113 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f93cc012340 con 0x55da52fef370 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93de04d740 10 client.27928 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93de04d740 1 -- 172.21.15.5:0/2600997113 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55da52ffdc30 con 0x55da52fef370 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93bd7fa640 1 -- 172.21.15.5:0/2600997113 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c409bb10 con 0x55da52fef370 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93bd7fa640 1 -- 172.21.15.5:0/2600997113 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f93cc0046a0 con 0x55da52fef370 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93de04d740 10 client.27928 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93de04d740 10 client.27928 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:32:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93de04d740 20 client.27928 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:52.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93de04d740 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55da52fff040 0x55da53001470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93de04d740 4 client.27928 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:52.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93de04d740 1 -- 172.21.15.5:0/2600997113 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da52f57860 con 0x55da52fff040 2024-09-11T22:32:52.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.351+0000 7f93d9a10640 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55da52fff040 0x55da53001470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.352+0000 7f93d9a10640 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55da52fff040 0x55da53001470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24e252a279b8028 server_cookie=4ee6c6dee67af0fe in_seq=0 out_seq=0 2024-09-11T22:32:52.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.352+0000 7f93bd7fa640 10 client.27928 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:32:52.367 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:51 smithi172 ceph-mon[35461]: pgmap v336: 97 pgs: 97 active+clean; 98 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 133 op/s 2024-09-11T22:32:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93bd7fa640 1 -- 172.21.15.5:0/2600997113 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da52f57860 con 0x55da52fff040 2024-09-11T22:32:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93bd7fa640 10 client.27928 handle_command_reply: tid=0 2024-09-11T22:32:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 client.27928 shutdown 2024-09-11T22:32:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 -- 172.21.15.5:0/2600997113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 msgr2=0x55da52fec860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 0x55da52fec860 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93d4076600 tx=0x7f93cc019000 comp rx=0 tx=0).stop 2024-09-11T22:32:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 -- 172.21.15.5:0/2600997113 shutdown_connections 2024-09-11T22:32:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55da52fff040 0x55da53001470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 0x55da52fec860 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe5b40 0x55da52fec0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 --2- 172.21.15.5:0/2600997113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe45d0 0x55da52feee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93bd7fa640 0 client.27928 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 -- 172.21.15.5:0/2600997113 >> 172.21.15.5:0/2600997113 conn(0x55da52fad310 msgr2=0x55da52fe6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 -- 172.21.15.5:0/2600997113 shutdown_connections 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 1 -- 172.21.15.5:0/2600997113 wait complete. 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.353+0000 7f93de04d740 20 client.27928 trim_cache size 0 max 16384 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93de04d740 1 Processor -- start 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93de04d740 1 -- start start 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93de04d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52fe0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93de04d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe5b40 0x55da52feb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93de04d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fef370 0x55da52fe7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93de04d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93d41b4130 con 0x55da52fe45d0 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93de04d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93d41b5c70 con 0x55da52fef370 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93de04d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93d41b4800 con 0x55da52fe5b40 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93d9a10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52fe0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93d9a10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52fe0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47698/0 (socket says 172.21.15.5:47698) 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93d9a10640 1 -- 172.21.15.5:0/2605830351 learned_addr learned my addr 172.21.15.5:0/2605830351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93da211640 1 --2- 172.21.15.5:0/2605830351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fef370 0x55da52fe7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.366+0000 7f93d920f640 1 --2- 172.21.15.5:0/2605830351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe5b40 0x55da52feb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.367+0000 7f93d9a10640 1 -- 172.21.15.5:0/2605830351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe5b40 msgr2=0x55da52feb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.367+0000 7f93d9a10640 1 --2- 172.21.15.5:0/2605830351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe5b40 0x55da52feb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.367+0000 7f93d9a10640 1 -- 172.21.15.5:0/2605830351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fef370 msgr2=0x55da52fe7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.367+0000 7f93d9a10640 1 --2- 172.21.15.5:0/2605830351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fef370 0x55da52fe7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.367+0000 7f93d9a10640 1 -- 172.21.15.5:0/2605830351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da52fe5350 con 0x55da52fe45d0 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.367+0000 7f93da211640 1 --2- 172.21.15.5:0/2605830351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fef370 0x55da52fe7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.367+0000 7f93d9a10640 1 --2- 172.21.15.5:0/2605830351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52fe0e10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009230 tx=0x7f93d0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3371e88e2c9cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.367+0000 7f93bf7fe640 1 -- 172.21.15.5:0/2605830351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0035040 con 0x55da52fe45d0 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.367+0000 7f93bf7fe640 1 -- 172.21.15.5:0/2605830351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93d004a070 con 0x55da52fe45d0 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.367+0000 7f93bf7fe640 1 -- 172.21.15.5:0/2605830351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0031810 con 0x55da52fe45d0 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 -- 172.21.15.5:0/2605830351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 msgr2=0x55da52fe0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 --2- 172.21.15.5:0/2605830351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52fe0e10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009230 tx=0x7f93d0031400 comp rx=0 tx=0).stop 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 -- 172.21.15.5:0/2605830351 shutdown_connections 2024-09-11T22:32:52.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 --2- 172.21.15.5:0/2605830351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fef370 0x55da52fe7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 --2- 172.21.15.5:0/2605830351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fe5b40 0x55da52feb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 --2- 172.21.15.5:0/2605830351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52fe0e10 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 -- 172.21.15.5:0/2605830351 >> 172.21.15.5:0/2605830351 conn(0x55da52fad310 msgr2=0x55da52fe0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 -- 172.21.15.5:0/2605830351 shutdown_connections 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 -- 172.21.15.5:0/2605830351 wait complete. 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 -- start start 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 Processor -- start 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52ff76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 0x55da52ff7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.368+0000 7f93de04d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 0x55da52fe8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93de04d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93d41b55a0 con 0x55da52fe45d0 2024-09-11T22:32:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93de04d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93d41b4ed0 con 0x55da52fe5b40 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93de04d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93d41b2110 con 0x55da52fef370 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93de04d740 10 client.0 _ll_register_callbacks cb 0x7f93cc00ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d9a10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52ff76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d9a10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52ff76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47706/0 (socket says 172.21.15.5:47706) 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d9a10640 1 -- 172.21.15.5:0/1329016853 learned_addr learned my addr 172.21.15.5:0/1329016853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93da211640 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 0x55da52fe8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d9a10640 1 -- 172.21.15.5:0/1329016853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 msgr2=0x55da52fe8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d9a10640 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 0x55da52fe8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d920f640 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 0x55da52ff7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d9a10640 1 -- 172.21.15.5:0/1329016853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 msgr2=0x55da52ff7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d9a10640 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 0x55da52ff7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d9a10640 1 -- 172.21.15.5:0/1329016853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da52fe9140 con 0x55da52fe45d0 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93da211640 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 0x55da52fe8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d920f640 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 0x55da52ff7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93d9a10640 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52ff76d0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f93d0006010 tx=0x7f93d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b70d50e0ba51407 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93bd7fa640 1 -- 172.21.15.5:0/1329016853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0052070 con 0x55da52fe45d0 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93bd7fa640 1 -- 172.21.15.5:0/1329016853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f93d00050a0 con 0x55da52fe45d0 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.369+0000 7f93de04d740 1 -- 172.21.15.5:0/1329016853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93d004a070 con 0x55da52fe45d0 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93bd7fa640 1 -- 172.21.15.5:0/1329016853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f93d0035050 con 0x55da52fe45d0 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93de04d740 10 client.18795 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93de04d740 1 -- 172.21.15.5:0/1329016853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55da53069810 con 0x55da52fe45d0 2024-09-11T22:32:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93bd7fa640 1 -- 172.21.15.5:0/1329016853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93d004a070 con 0x55da52fe45d0 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93bd7fa640 1 -- 172.21.15.5:0/1329016853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f93d005c020 con 0x55da52fe45d0 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93de04d740 10 client.18795 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93de04d740 10 client.18795 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93de04d740 20 client.18795 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93de04d740 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55da53069ee0 0x55da5306c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93de04d740 4 client.18795 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93de04d740 1 -- 172.21.15.5:0/1329016853 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da52f51d70 con 0x55da53069ee0 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.370+0000 7f93d920f640 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55da53069ee0 0x55da5306c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93d920f640 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55da53069ee0 0x55da5306c300 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5302158a0fa133a2 server_cookie=f5b30e57261c784f in_seq=0 out_seq=0 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93bd7fa640 10 client.18795 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:32:52.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93bd7fa640 1 -- 172.21.15.5:0/1329016853 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da52f51d70 con 0x55da53069ee0 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93bd7fa640 10 client.18795 handle_command_reply: tid=0 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 client.18795 shutdown 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 -- 172.21.15.5:0/1329016853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 msgr2=0x55da52ff76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52ff76d0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f93d0006010 tx=0x7f93d0004750 comp rx=0 tx=0).stop 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 -- 172.21.15.5:0/1329016853 shutdown_connections 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55da53069ee0 0x55da5306c300 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da52fef370 0x55da52fe8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da52fe5b40 0x55da52ff7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93bd7fa640 0 client.18795 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 --2- 172.21.15.5:0/1329016853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da52fe45d0 0x55da52ff76d0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 -- 172.21.15.5:0/1329016853 >> 172.21.15.5:0/1329016853 conn(0x55da52fad310 msgr2=0x55da52fed930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 -- 172.21.15.5:0/1329016853 shutdown_connections 2024-09-11T22:32:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 1 -- 172.21.15.5:0/1329016853 wait complete. 2024-09-11T22:32:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.371+0000 7f93de04d740 20 client.18795 trim_cache size 0 max 16384 2024-09-11T22:32:52.382 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:32:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 -- 172.21.15.5:0/440544404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93c00a2cf0 msgr2=0x7f93c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93c00a2cf0 0x7f93c00a51b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f93cc005c40 tx=0x7f93cc00ac50 comp rx=0 tx=0).stop 2024-09-11T22:32:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 -- 172.21.15.5:0/440544404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93d4074eb0 msgr2=0x7f93d41a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93d4074eb0 0x7f93d41a4c30 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f93c400d950 tx=0x7f93c400de10 comp rx=0 tx=0).stop 2024-09-11T22:32:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 -- 172.21.15.5:0/440544404 shutdown_connections 2024-09-11T22:32:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93c00a2cf0 0x7f93c00a51b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93d4112b90 0x7f93d419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93d4074eb0 0x7f93d41a4c30 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 --2- 172.21.15.5:0/440544404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93d4074590 0x7f93d41a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 -- 172.21.15.5:0/440544404 >> 172.21.15.5:0/440544404 conn(0x7f93d40fd8c0 msgr2=0x7f93d40fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:52.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 -- 172.21.15.5:0/440544404 shutdown_connections 2024-09-11T22:32:52.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:52.372+0000 7f93dbc74640 1 -- 172.21.15.5:0/440544404 wait complete. 2024-09-11T22:32:53.100 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:32:53.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.23 MB/sec execute 431 sec latency 8910.445 ms 2024-09-11T22:32:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.884+0000 7f73705c1640 1 -- 172.21.15.5:0/3652099125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7368100e20 msgr2=0x7f7368101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.884+0000 7f73705c1640 1 --2- 172.21.15.5:0/3652099125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7368100e20 0x7f7368101200 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f7364009ca0 tx=0x7f736402a950 comp rx=0 tx=0).stop 2024-09-11T22:32:53.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 -- 172.21.15.5:0/3652099125 shutdown_connections 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 --2- 172.21.15.5:0/3652099125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7368102ef0 0x7f736810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 --2- 172.21.15.5:0/3652099125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7368102570 0x7f73681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 --2- 172.21.15.5:0/3652099125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7368100e20 0x7f7368101200 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 -- 172.21.15.5:0/3652099125 >> 172.21.15.5:0/3652099125 conn(0x7f73680fc7d0 msgr2=0x7f73680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 -- 172.21.15.5:0/3652099125 shutdown_connections 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 -- 172.21.15.5:0/3652099125 wait complete. 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 Processor -- start 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 -- start start 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7368100e20 0x7f73681986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7368102570 0x7f7368198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7368102ef0 0x7f736819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7368111020 con 0x7f7368100e20 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7368110ea0 con 0x7f7368102570 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.885+0000 7f73705c1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f73681111a0 con 0x7f7368102ef0 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736e35d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7368100e20 0x7f73681986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736e35d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7368100e20 0x7f73681986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47720/0 (socket says 172.21.15.5:47720) 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736e35d640 1 -- 172.21.15.5:0/3794620706 learned_addr learned my addr 172.21.15.5:0/3794620706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736db5c640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7368102570 0x7f7368198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736eb5e640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7368102ef0 0x7f736819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736e35d640 1 -- 172.21.15.5:0/3794620706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7368102ef0 msgr2=0x7f736819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736e35d640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7368102ef0 0x7f736819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736e35d640 1 -- 172.21.15.5:0/3794620706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7368102570 msgr2=0x7f7368198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736e35d640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7368102570 0x7f7368198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736e35d640 1 -- 172.21.15.5:0/3794620706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736819d650 con 0x7f7368100e20 2024-09-11T22:32:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736eb5e640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7368102ef0 0x7f736819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f736e35d640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7368100e20 0x7f73681986a0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7364005cc0 tx=0x7f736402f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f735f7fe640 1 -- 172.21.15.5:0/3794620706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364034040 con 0x7f7368100e20 2024-09-11T22:32:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f735f7fe640 1 -- 172.21.15.5:0/3794620706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7364003e40 con 0x7f7368100e20 2024-09-11T22:32:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f73705c1640 1 -- 172.21.15.5:0/3794620706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736819d8e0 con 0x7f7368100e20 2024-09-11T22:32:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f735f7fe640 1 -- 172.21.15.5:0/3794620706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7364045480 con 0x7f7368100e20 2024-09-11T22:32:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.886+0000 7f73705c1640 1 -- 172.21.15.5:0/3794620706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73681a5220 con 0x7f7368100e20 2024-09-11T22:32:53.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.889+0000 7f735f7fe640 1 -- 172.21.15.5:0/3794620706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7364037020 con 0x7f7368100e20 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.889+0000 7f737299a740 1 Processor -- start 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f737299a740 1 -- start start 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f737299a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdeb40 0x5647d9fdd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f737299a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9fdda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f737299a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fe8370 0x5647d9fea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f737299a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647d9f432f0 con 0x5647d9fdd5d0 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f737299a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5647d9eeacf0 con 0x5647d9fe8370 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f737299a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5647d9f3f2b0 con 0x5647d9fdeb40 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736e35d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9fdda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736e35d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9fdda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47734/0 (socket says 172.21.15.5:47734) 2024-09-11T22:32:53.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736e35d640 1 -- 172.21.15.5:0/859000943 learned_addr learned my addr 172.21.15.5:0/859000943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736db5c640 1 --2- 172.21.15.5:0/859000943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fe8370 0x5647d9fea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736eb5e640 1 --2- 172.21.15.5:0/859000943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdeb40 0x5647d9fdd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736e35d640 1 -- 172.21.15.5:0/859000943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdeb40 msgr2=0x5647d9fdd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736e35d640 1 --2- 172.21.15.5:0/859000943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdeb40 0x5647d9fdd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f735f7fe640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73540a2d70 0x7f73540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736e35d640 1 -- 172.21.15.5:0/859000943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fe8370 msgr2=0x5647d9fea760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736e35d640 1 --2- 172.21.15.5:0/859000943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fe8370 0x5647d9fea760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736e35d640 1 -- 172.21.15.5:0/859000943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d9ecf3a0 con 0x5647d9fdd5d0 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.890+0000 7f736eb5e640 1 --2- 172.21.15.5:0/859000943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdeb40 0x5647d9fdd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.891+0000 7f736eb5e640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73540a2d70 0x7f73540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.891+0000 7f736e35d640 1 --2- 172.21.15.5:0/859000943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9fdda10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f73640edd30 tx=0x7f73640b9230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342f58204fc8d3b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.891+0000 7f735f7fe640 1 -- 172.21.15.5:0/3794620706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f73640f0630 con 0x7f7368100e20 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.891+0000 7f735cff9640 1 -- 172.21.15.5:0/859000943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640bc990 con 0x5647d9fdd5d0 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.891+0000 7f735cff9640 1 -- 172.21.15.5:0/859000943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f736402c030 con 0x5647d9fdd5d0 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.891+0000 7f735cff9640 1 -- 172.21.15.5:0/859000943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640cb840 con 0x5647d9fdd5d0 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 -- 172.21.15.5:0/859000943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 msgr2=0x5647d9fdda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f736eb5e640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73540a2d70 0x7f73540a5230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f736000b880 tx=0x7f7360008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 --2- 172.21.15.5:0/859000943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9fdda10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f73640edd30 tx=0x7f73640b9230 comp rx=0 tx=0).stop 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 -- 172.21.15.5:0/859000943 shutdown_connections 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 --2- 172.21.15.5:0/859000943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fe8370 0x5647d9fea760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 --2- 172.21.15.5:0/859000943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9fdda10 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 --2- 172.21.15.5:0/859000943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdeb40 0x5647d9fdd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 -- 172.21.15.5:0/859000943 >> 172.21.15.5:0/859000943 conn(0x5647d9fa6310 msgr2=0x5647d9fd9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 -- 172.21.15.5:0/859000943 shutdown_connections 2024-09-11T22:32:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 -- 172.21.15.5:0/859000943 wait complete. 2024-09-11T22:32:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 -- start start 2024-09-11T22:32:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 Processor -- start 2024-09-11T22:32:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 0x5647d9fe7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdeb40 0x5647d9fe5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fe8370 0x5647d9fe5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647d9feae30 con 0x5647d9fdeb40 2024-09-11T22:32:53.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5647d9efe060 con 0x5647d9fe8370 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5647d9feafb0 con 0x5647d9fdd5d0 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.892+0000 7f737299a740 10 client.0 _ll_register_callbacks cb 0x5647d9c518c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736db5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdeb40 0x5647d9fe5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736db5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdeb40 0x5647d9fe5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47738/0 (socket says 172.21.15.5:47738) 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736db5c640 1 -- 172.21.15.5:0/1600517740 learned_addr learned my addr 172.21.15.5:0/1600517740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736e35d640 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 0x5647d9fe7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736eb5e640 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fe8370 0x5647d9fe5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736db5c640 1 -- 172.21.15.5:0/1600517740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 msgr2=0x5647d9fe7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736db5c640 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 0x5647d9fe7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736db5c640 1 -- 172.21.15.5:0/1600517740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fe8370 msgr2=0x5647d9fe5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:53.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736db5c640 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fe8370 0x5647d9fe5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736db5c640 1 -- 172.21.15.5:0/1600517740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d9cb43c0 con 0x5647d9fdeb40 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736e35d640 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 0x5647d9fe7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f736db5c640 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdeb40 0x5647d9fe5080 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7358010500 tx=0x7f73580109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11ef46faa0b0d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f7351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f7351ffb640 1 -- 172.21.15.5:0/1600517740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f735801a020 con 0x5647d9fdeb40 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f7351ffb640 1 -- 172.21.15.5:0/1600517740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f7358016750 con 0x5647d9fdeb40 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f737299a740 1 -- 172.21.15.5:0/1600517740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f736402c030 con 0x5647d9fdeb40 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.893+0000 7f7351ffb640 1 -- 172.21.15.5:0/1600517740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f7358025050 con 0x5647d9fdeb40 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.894+0000 7f737299a740 10 client.18807 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.894+0000 7f737299a740 1 -- 172.21.15.5:0/1600517740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5647d9ff68b0 con 0x5647d9fdeb40 2024-09-11T22:32:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.894+0000 7f7351ffb640 1 -- 172.21.15.5:0/1600517740 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f736402c030 con 0x5647d9fdeb40 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.894+0000 7f7351ffb640 1 -- 172.21.15.5:0/1600517740 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f7358014b60 con 0x5647d9fdeb40 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.894+0000 7f737299a740 10 client.18807 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.894+0000 7f737299a740 10 client.18807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.894+0000 7f737299a740 20 client.18807 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.894+0000 7f737299a740 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647d9ff7d70 0x5647d9ffa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.894+0000 7f737299a740 4 client.18807 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.894+0000 7f737299a740 1 -- 172.21.15.5:0/1600517740 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647d9f50860 con 0x5647d9ff7d70 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.895+0000 7f736e35d640 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647d9ff7d70 0x5647d9ffa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.895+0000 7f736e35d640 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647d9ff7d70 0x5647d9ffa1e0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b91aa282252c1b server_cookie=5d22e5e4c7bc0954 in_seq=0 out_seq=0 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.895+0000 7f7351ffb640 10 client.18807 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f7351ffb640 1 -- 172.21.15.5:0/1600517740 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647d9f50860 con 0x5647d9ff7d70 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f7351ffb640 10 client.18807 handle_command_reply: tid=0 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 client.18807 shutdown 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 -- 172.21.15.5:0/1600517740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdeb40 msgr2=0x5647d9fe5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdeb40 0x5647d9fe5080 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7358010500 tx=0x7f73580109c0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 -- 172.21.15.5:0/1600517740 shutdown_connections 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647d9ff7d70 0x5647d9ffa1e0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fe8370 0x5647d9fe5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdeb40 0x5647d9fe5080 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 --2- 172.21.15.5:0/1600517740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 0x5647d9fe7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 -- 172.21.15.5:0/1600517740 >> 172.21.15.5:0/1600517740 conn(0x5647d9fa6310 msgr2=0x5647d9fdf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f7351ffb640 0 client.18807 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 -- 172.21.15.5:0/1600517740 shutdown_connections 2024-09-11T22:32:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 1 -- 172.21.15.5:0/1600517740 wait complete. 2024-09-11T22:32:53.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.896+0000 7f737299a740 20 client.18807 trim_cache size 0 max 16384 2024-09-11T22:32:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f737299a740 1 Processor -- start 2024-09-11T22:32:53.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f737299a740 1 -- start start 2024-09-11T22:32:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f737299a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 0x5647d9fd9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f737299a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 0x5647d9fd9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f737299a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fe8370 0x5647d9fffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f737299a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73681adad0 con 0x5647d9fe8370 2024-09-11T22:32:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f737299a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73681af610 con 0x5647d9fdeb40 2024-09-11T22:32:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f737299a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f73681ae1a0 con 0x5647d9fdd5d0 2024-09-11T22:32:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f736eb5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fe8370 0x5647d9fffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f736eb5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fe8370 0x5647d9fffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47746/0 (socket says 172.21.15.5:47746) 2024-09-11T22:32:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f736eb5e640 1 -- 172.21.15.5:0/740345967 learned_addr learned my addr 172.21.15.5:0/740345967 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:53.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f736db5c640 1 --2- 172.21.15.5:0/740345967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 0x5647d9fd9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f736e35d640 1 --2- 172.21.15.5:0/740345967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 0x5647d9fd9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f736eb5e640 1 -- 172.21.15.5:0/740345967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 msgr2=0x5647d9fd9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f736eb5e640 1 --2- 172.21.15.5:0/740345967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 0x5647d9fd9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f736eb5e640 1 -- 172.21.15.5:0/740345967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 msgr2=0x5647d9fd9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f736eb5e640 1 --2- 172.21.15.5:0/740345967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 0x5647d9fd9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.910+0000 7f736eb5e640 1 -- 172.21.15.5:0/740345967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d9fda180 con 0x5647d9fe8370 2024-09-11T22:32:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.911+0000 7f736eb5e640 1 --2- 172.21.15.5:0/740345967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fe8370 0x5647d9fffaf0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f736000dbd0 tx=0x7f7360010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20c81175db021b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.911+0000 7f7353fff640 1 -- 172.21.15.5:0/740345967 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360019030 con 0x5647d9fe8370 2024-09-11T22:32:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.911+0000 7f7353fff640 1 -- 172.21.15.5:0/740345967 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7360024070 con 0x5647d9fe8370 2024-09-11T22:32:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.911+0000 7f7353fff640 1 -- 172.21.15.5:0/740345967 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7360013650 con 0x5647d9fe8370 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.911+0000 7f737299a740 1 -- 172.21.15.5:0/740345967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fe8370 msgr2=0x5647d9fffaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.911+0000 7f737299a740 1 --2- 172.21.15.5:0/740345967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fe8370 0x5647d9fffaf0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f736000dbd0 tx=0x7f7360010000 comp rx=0 tx=0).stop 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 -- 172.21.15.5:0/740345967 shutdown_connections 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 --2- 172.21.15.5:0/740345967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fe8370 0x5647d9fffaf0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 --2- 172.21.15.5:0/740345967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 0x5647d9fd9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 --2- 172.21.15.5:0/740345967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fdd5d0 0x5647d9fd9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 -- 172.21.15.5:0/740345967 >> 172.21.15.5:0/740345967 conn(0x5647d9fa6310 msgr2=0x5647d9fd4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 -- 172.21.15.5:0/740345967 shutdown_connections 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 -- 172.21.15.5:0/740345967 wait complete. 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 -- start start 2024-09-11T22:32:53.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 Processor -- start 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9ff0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 0x5647d9ff0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fe8370 0x5647d9fed1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73681aef40 con 0x5647d9fdd5d0 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73681ae870 con 0x5647d9fdeb40 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f73681abab0 con 0x5647d9fe8370 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.912+0000 7f737299a740 10 client.0 _ll_register_callbacks cb 0x7f735800b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736e35d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9ff0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736e35d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9ff0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47752/0 (socket says 172.21.15.5:47752) 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736e35d640 1 -- 172.21.15.5:0/852972998 learned_addr learned my addr 172.21.15.5:0/852972998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736eb5e640 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fe8370 0x5647d9fed1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736db5c640 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 0x5647d9ff0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736e35d640 1 -- 172.21.15.5:0/852972998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fe8370 msgr2=0x5647d9fed1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736e35d640 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fe8370 0x5647d9fed1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736e35d640 1 -- 172.21.15.5:0/852972998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 msgr2=0x5647d9ff0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736e35d640 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 0x5647d9ff0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736e35d640 1 -- 172.21.15.5:0/852972998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d9fed710 con 0x5647d9fdd5d0 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736db5c640 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 0x5647d9ff0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f736e35d640 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9ff0200 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f73640bcb50 tx=0x7f73640095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8205264e384ddf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f7351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f7351ffb640 1 -- 172.21.15.5:0/852972998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73640cba40 con 0x5647d9fdd5d0 2024-09-11T22:32:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f7351ffb640 1 -- 172.21.15.5:0/852972998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f73640f66d0 con 0x5647d9fdd5d0 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f737299a740 1 -- 172.21.15.5:0/852972998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7360024070 con 0x5647d9fdd5d0 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f7351ffb640 1 -- 172.21.15.5:0/852972998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f73640da050 con 0x5647d9fdd5d0 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f737299a740 10 client.18813 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.913+0000 7f737299a740 1 -- 172.21.15.5:0/852972998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5647da0631b0 con 0x5647d9fdd5d0 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.914+0000 7f7351ffb640 1 -- 172.21.15.5:0/852972998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7360024070 con 0x5647d9fdd5d0 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.914+0000 7f7351ffb640 1 -- 172.21.15.5:0/852972998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f736404d030 con 0x5647d9fdd5d0 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.914+0000 7f737299a740 10 client.18813 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.914+0000 7f737299a740 10 client.18813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.914+0000 7f737299a740 20 client.18813 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.914+0000 7f737299a740 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647da063680 0x5647da065a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.914+0000 7f737299a740 4 client.18813 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.914+0000 7f737299a740 1 -- 172.21.15.5:0/852972998 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647d9f4ad70 con 0x5647da063680 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.914+0000 7f736db5c640 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647da063680 0x5647da065a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f7351ffb640 10 client.18813 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f736db5c640 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647da063680 0x5647da065a70 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f5e4d04d2ad3ade server_cookie=238ce6e560e294f5 in_seq=0 out_seq=0 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f7351ffb640 1 -- 172.21.15.5:0/852972998 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647d9f4ad70 con 0x5647da063680 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f7351ffb640 10 client.18813 handle_command_reply: tid=0 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 client.18813 shutdown 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 -- 172.21.15.5:0/852972998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 msgr2=0x5647d9ff0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9ff0200 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f73640bcb50 tx=0x7f73640095c0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 -- 172.21.15.5:0/852972998 shutdown_connections 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5647da063680 0x5647da065a70 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647d9fe8370 0x5647d9fed1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5647d9fdeb40 0x5647d9ff0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f7351ffb640 0 client.18813 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 --2- 172.21.15.5:0/852972998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647d9fdd5d0 0x5647d9ff0200 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 -- 172.21.15.5:0/852972998 >> 172.21.15.5:0/852972998 conn(0x5647d9fa6310 msgr2=0x5647d9fd9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 -- 172.21.15.5:0/852972998 shutdown_connections 2024-09-11T22:32:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 1 -- 172.21.15.5:0/852972998 wait complete. 2024-09-11T22:32:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.915+0000 7f737299a740 20 client.18813 trim_cache size 0 max 16384 2024-09-11T22:32:53.934 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:32:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 -- 172.21.15.5:0/3794620706 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73540a2d70 msgr2=0x7f73540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73540a2d70 0x7f73540a5230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f736000b880 tx=0x7f7360008040 comp rx=0 tx=0).stop 2024-09-11T22:32:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 -- 172.21.15.5:0/3794620706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7368100e20 msgr2=0x7f73681986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7368100e20 0x7f73681986a0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7364005cc0 tx=0x7f736402f6c0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 -- 172.21.15.5:0/3794620706 shutdown_connections 2024-09-11T22:32:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7368102ef0 0x7f736819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73540a2d70 0x7f73540a5230 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7368102570 0x7f7368198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 --2- 172.21.15.5:0/3794620706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7368100e20 0x7f73681986a0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 -- 172.21.15.5:0/3794620706 >> 172.21.15.5:0/3794620706 conn(0x7f73680fc7d0 msgr2=0x7f736806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 -- 172.21.15.5:0/3794620706 shutdown_connections 2024-09-11T22:32:53.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:53.916+0000 7f73705c1640 1 -- 172.21.15.5:0/3794620706 wait complete. 2024-09-11T22:32:54.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:53 smithi005 ceph-mon[30943]: pgmap v337: 97 pgs: 97 active+clean; 89 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 181 op/s 2024-09-11T22:32:54.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:53 smithi159 ceph-mon[35534]: pgmap v337: 97 pgs: 97 active+clean; 89 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 181 op/s 2024-09-11T22:32:54.340 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.22 MB/sec execute 432 sec latency 9910.475 ms 2024-09-11T22:32:54.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:53 smithi172 ceph-mon[35461]: pgmap v337: 97 pgs: 97 active+clean; 89 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 181 op/s 2024-09-11T22:32:55.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.20 MB/sec execute 433 sec latency 10910.583 ms 2024-09-11T22:32:55.934 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:32:55.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:32:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:56.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:55 smithi005 ceph-mon[30943]: pgmap v338: 97 pgs: 97 active+clean; 89 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-11T22:32:56.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:55 smithi159 ceph-mon[35534]: pgmap v338: 97 pgs: 97 active+clean; 89 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-11T22:32:56.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.18 MB/sec execute 434 sec latency 11910.679 ms 2024-09-11T22:32:56.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:55 smithi172 ceph-mon[35461]: pgmap v338: 97 pgs: 97 active+clean; 89 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-11T22:32:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.706+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3094773766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c74102790 msgr2=0x7f8c741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.706+0000 7f8c7bc41640 1 --2- 172.21.15.5:0/3094773766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c74102790 0x7f8c741067e0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f8c64009900 tx=0x7f8c6402a410 comp rx=0 tx=0).stop 2024-09-11T22:32:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.706+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3094773766 shutdown_connections 2024-09-11T22:32:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.706+0000 7f8c7bc41640 1 --2- 172.21.15.5:0/3094773766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c74106f10 0x7f8c7410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.706+0000 7f8c7bc41640 1 --2- 172.21.15.5:0/3094773766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c74102790 0x7f8c741067e0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.706+0000 7f8c7bc41640 1 --2- 172.21.15.5:0/3094773766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c74101de0 0x7f8c741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.706+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3094773766 >> 172.21.15.5:0/3094773766 conn(0x7f8c740fd8c0 msgr2=0x7f8c740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.706+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3094773766 shutdown_connections 2024-09-11T22:32:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.706+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3094773766 wait complete. 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c7bc41640 1 Processor -- start 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c7bc41640 1 -- start start 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c7bc41640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c74101de0 0x7f8c740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c7bc41640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c74102790 0x7f8c74074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c7bc41640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c74106f10 0x7f8c74074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c7bc41640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c74111f90 con 0x7f8c74102790 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c7bc41640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8c74111e10 con 0x7f8c74101de0 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c7bc41640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8c74112110 con 0x7f8c74106f10 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c791dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c74102790 0x7f8c74074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c791dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c74102790 0x7f8c74074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47772/0 (socket says 172.21.15.5:47772) 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c791dc640 1 -- 172.21.15.5:0/3568990181 learned_addr learned my addr 172.21.15.5:0/3568990181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c74106f10 0x7f8c74074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.707+0000 7f8c799dd640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c74101de0 0x7f8c740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c791dc640 1 -- 172.21.15.5:0/3568990181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c74106f10 msgr2=0x7f8c74074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c791dc640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c74106f10 0x7f8c74074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c791dc640 1 -- 172.21.15.5:0/3568990181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c74101de0 msgr2=0x7f8c740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c791dc640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c74101de0 0x7f8c740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c791dc640 1 -- 172.21.15.5:0/3568990181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c741a6930 con 0x7f8c74102790 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c799dd640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c74101de0 0x7f8c740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c791dc640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c74102790 0x7f8c74074a30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8c64005ab0 tx=0x7f8c640042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c6affd640 1 -- 172.21.15.5:0/3568990181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6403f030 con 0x7f8c74102790 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c6affd640 1 -- 172.21.15.5:0/3568990181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c64003ce0 con 0x7f8c74102790 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3568990181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c741a6bc0 con 0x7f8c74102790 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c6affd640 1 -- 172.21.15.5:0/3568990181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64035070 con 0x7f8c74102790 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.708+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3568990181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c741a70a0 con 0x7f8c74102790 2024-09-11T22:32:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.710+0000 7f8c6affd640 1 -- 172.21.15.5:0/3568990181 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8c64042030 con 0x7f8c74102790 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.710+0000 7f8c7e01a740 1 Processor -- start 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c7e01a740 1 -- start start 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947bee8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947bef3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55947be4e280 con 0x55947bef3d40 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55947bdf5b10 con 0x55947bee8b60 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55947be4a240 con 0x55947bee9480 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c6affd640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8c600a2cf0 0x7f8c600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c791dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c791dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47782/0 (socket says 172.21.15.5:47782) 2024-09-11T22:32:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c791dc640 1 -- 172.21.15.5:0/2001211668 learned_addr learned my addr 172.21.15.5:0/2001211668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/2001211668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947bee8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8c600a2cf0 0x7f8c600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c799dd640 1 --2- 172.21.15.5:0/2001211668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947bef3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.711+0000 7f8c6affd640 1 -- 172.21.15.5:0/3568990181 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f8c640efe50 con 0x7f8c74102790 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.712+0000 7f8c7a1de640 1 -- 172.21.15.5:0/2001211668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 msgr2=0x55947bef3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.712+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/2001211668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947bef3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.712+0000 7f8c7a1de640 1 -- 172.21.15.5:0/2001211668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 msgr2=0x55947bef6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.712+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/2001211668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.712+0000 7f8c7a1de640 1 -- 172.21.15.5:0/2001211668 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947be021f0 con 0x55947bee8b60 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.712+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8c600a2cf0 0x7f8c600a51b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c016c20 tx=0x7f8c6c01a000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.712+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/2001211668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947bee8f40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c014b70 tx=0x7f8c6c016720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b4156196ff3a622 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.712+0000 7f8c5ffff640 1 -- 172.21.15.5:0/2001211668 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c01c070 con 0x55947bee8b60 2024-09-11T22:32:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.712+0000 7f8c5ffff640 1 -- 172.21.15.5:0/2001211668 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c6c006740 con 0x55947bee8b60 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.712+0000 7f8c5ffff640 1 -- 172.21.15.5:0/2001211668 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0173c0 con 0x55947bee8b60 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 -- 172.21.15.5:0/2001211668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 msgr2=0x55947bee8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/2001211668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947bee8f40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c014b70 tx=0x7f8c6c016720 comp rx=0 tx=0).stop 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 -- 172.21.15.5:0/2001211668 shutdown_connections 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/2001211668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/2001211668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947bef3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/2001211668 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947bee8f40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 -- 172.21.15.5:0/2001211668 >> 172.21.15.5:0/2001211668 conn(0x55947bee50f0 msgr2=0x55947bee54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 -- 172.21.15.5:0/2001211668 shutdown_connections 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 -- 172.21.15.5:0/2001211668 wait complete. 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 -- start start 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.713+0000 7f8c7e01a740 1 Processor -- start 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947bef0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947bef1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55947bef68b0 con 0x55947bef3d40 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55947be08f10 con 0x55947bee8b60 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55947bef6a30 con 0x55947bee9480 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7e01a740 10 client.0 _ll_register_callbacks cb 0x55947bdd1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7a1de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7a1de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47784/0 (socket says 172.21.15.5:47784) 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7a1de640 1 -- 172.21.15.5:0/3534689235 learned_addr learned my addr 172.21.15.5:0/3534689235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c799dd640 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947bef0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c791dc640 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947bef1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7a1de640 1 -- 172.21.15.5:0/3534689235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 msgr2=0x55947bef1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947bef1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7a1de640 1 -- 172.21.15.5:0/3534689235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 msgr2=0x55947bef0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947bef0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c7a1de640 1 -- 172.21.15.5:0/3534689235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947bbbf6a0 con 0x55947bef3d40 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.714+0000 7f8c799dd640 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947bef0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef1650 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c006210 tx=0x7f8c6c014e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a83271646eb3372 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/3534689235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c030020 con 0x55947bef3d40 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/3534689235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f8c6c020610 con 0x55947bef3d40 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c7e01a740 1 -- 172.21.15.5:0/3534689235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55947be5b7f0 con 0x55947bef3d40 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/3534689235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8c6c03b050 con 0x55947bef3d40 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c7e01a740 10 client.18825 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c7e01a740 1 -- 172.21.15.5:0/3534689235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55947bf02490 con 0x55947bef3d40 2024-09-11T22:32:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/3534689235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55947be5b7f0 con 0x55947bef3d40 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/3534689235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f8c6c02cce0 con 0x55947bef3d40 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c7e01a740 10 client.18825 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c7e01a740 10 client.18825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c7e01a740 20 client.18825 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.715+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55947bf03950 0x55947bf05d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.716+0000 7f8c7e01a740 4 client.18825 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.716+0000 7f8c7e01a740 1 -- 172.21.15.5:0/3534689235 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55947be55d00 con 0x55947bf03950 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.716+0000 7f8c799dd640 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55947bf03950 0x55947bf05d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.716+0000 7f8c799dd640 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55947bf03950 0x55947bf05d40 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49fa5d82803a21b4 server_cookie=76d8f192de55cfa5 in_seq=0 out_seq=0 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.716+0000 7f8c5d7fa640 10 client.18825 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/3534689235 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55947be55d00 con 0x55947bf03950 2024-09-11T22:32:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c5d7fa640 10 client.18825 handle_command_reply: tid=0 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 client.18825 shutdown 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 -- 172.21.15.5:0/3534689235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 msgr2=0x55947bef1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef1650 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c006210 tx=0x7f8c6c014e90 comp rx=0 tx=0).stop 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 -- 172.21.15.5:0/3534689235 shutdown_connections 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55947bf03950 0x55947bf05d40 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bef1650 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947bef1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c5d7fa640 0 client.18825 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/3534689235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947bef0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 -- 172.21.15.5:0/3534689235 >> 172.21.15.5:0/3534689235 conn(0x55947bee50f0 msgr2=0x55947beeae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 -- 172.21.15.5:0/3534689235 shutdown_connections 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 1 -- 172.21.15.5:0/3534689235 wait complete. 2024-09-11T22:32:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.717+0000 7f8c7e01a740 20 client.18825 trim_cache size 0 max 16384 2024-09-11T22:32:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.735+0000 7f8c7e01a740 1 Processor -- start 2024-09-11T22:32:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.735+0000 7f8c7e01a740 1 -- start start 2024-09-11T22:32:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.735+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947beeb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947beefaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bf0b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741afc30 con 0x55947bef3d40 2024-09-11T22:32:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741b1770 con 0x55947bee8b60 2024-09-11T22:32:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741b0300 con 0x55947bee9480 2024-09-11T22:32:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7a1de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bf0b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7a1de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bf0b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47794/0 (socket says 172.21.15.5:47794) 2024-09-11T22:32:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7a1de640 1 -- 172.21.15.5:0/2537907703 learned_addr learned my addr 172.21.15.5:0/2537907703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c799dd640 1 --2- 172.21.15.5:0/2537907703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947beeb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c791dc640 1 --2- 172.21.15.5:0/2537907703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947beefaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7a1de640 1 -- 172.21.15.5:0/2537907703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 msgr2=0x55947beefaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/2537907703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947beefaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7a1de640 1 -- 172.21.15.5:0/2537907703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 msgr2=0x55947beeb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/2537907703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947beeb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7a1de640 1 -- 172.21.15.5:0/2537907703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947bee61a0 con 0x55947bef3d40 2024-09-11T22:32:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c799dd640 1 --2- 172.21.15.5:0/2537907703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947beeb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:32:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/2537907703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bf0b370 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c006d20 tx=0x7f8c6c016720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9225f187a992891f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.736+0000 7f8c5f7fe640 1 -- 172.21.15.5:0/2537907703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c043020 con 0x55947bef3d40 2024-09-11T22:32:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c5f7fe640 1 -- 172.21.15.5:0/2537907703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c6c005f40 con 0x55947bef3d40 2024-09-11T22:32:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c5f7fe640 1 -- 172.21.15.5:0/2537907703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c030040 con 0x55947bef3d40 2024-09-11T22:32:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c7e01a740 1 -- 172.21.15.5:0/2537907703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 msgr2=0x55947bf0b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/2537907703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bf0b370 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c006d20 tx=0x7f8c6c016720 comp rx=0 tx=0).stop 2024-09-11T22:32:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c7e01a740 1 -- 172.21.15.5:0/2537907703 shutdown_connections 2024-09-11T22:32:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/2537907703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947bf0b370 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/2537907703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947beefaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/2537907703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947beeb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c7e01a740 1 -- 172.21.15.5:0/2537907703 >> 172.21.15.5:0/2537907703 conn(0x55947bee50f0 msgr2=0x55947beef390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c7e01a740 1 -- 172.21.15.5:0/2537907703 shutdown_connections 2024-09-11T22:32:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.737+0000 7f8c7e01a740 1 -- 172.21.15.5:0/2537907703 wait complete. 2024-09-11T22:32:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.738+0000 7f8c7e01a740 1 -- start start 2024-09-11T22:32:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.738+0000 7f8c7e01a740 1 Processor -- start 2024-09-11T22:32:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.738+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947befc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.738+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947beed9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.738+0000 7f8c7e01a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947beedfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.738+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741b10a0 con 0x55947bef3d40 2024-09-11T22:32:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.738+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741b09d0 con 0x55947bee8b60 2024-09-11T22:32:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.738+0000 7f8c7e01a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8c741adc10 con 0x55947bee9480 2024-09-11T22:32:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.738+0000 7f8c7e01a740 10 client.0 _ll_register_callbacks cb 0x7f8c6c008070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7a1de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947beedfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7a1de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947beedfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47800/0 (socket says 172.21.15.5:47800) 2024-09-11T22:32:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7a1de640 1 -- 172.21.15.5:0/676270845 learned_addr learned my addr 172.21.15.5:0/676270845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c791dc640 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947beed9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c799dd640 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947befc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7a1de640 1 -- 172.21.15.5:0/676270845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 msgr2=0x55947beed9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947beed9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7a1de640 1 -- 172.21.15.5:0/676270845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 msgr2=0x55947befc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947befc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7a1de640 1 -- 172.21.15.5:0/676270845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947beee810 con 0x55947bef3d40 2024-09-11T22:32:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7a1de640 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947beedfe0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c02b6a0 tx=0x7f8c6c006bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9733014ecf4255 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/676270845 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c043030 con 0x55947bef3d40 2024-09-11T22:32:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/676270845 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f8c6c02be40 con 0x55947bef3d40 2024-09-11T22:32:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7e01a740 1 -- 172.21.15.5:0/676270845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c6c005f40 con 0x55947bef3d40 2024-09-11T22:32:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/676270845 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8c6c056050 con 0x55947bef3d40 2024-09-11T22:32:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7e01a740 10 client.18831 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.739+0000 7f8c7e01a740 1 -- 172.21.15.5:0/676270845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55947bf6e460 con 0x55947bef3d40 2024-09-11T22:32:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.740+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/676270845 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c6c005f40 con 0x55947bef3d40 2024-09-11T22:32:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.740+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/676270845 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f8c6c02c8a0 con 0x55947bef3d40 2024-09-11T22:32:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.740+0000 7f8c7e01a740 10 client.18831 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.740+0000 7f8c7e01a740 10 client.18831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.740+0000 7f8c7e01a740 20 client.18831 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.740+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55947bf6eac0 0x55947bf70eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.740+0000 7f8c7e01a740 4 client.18831 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:32:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.740+0000 7f8c7e01a740 1 -- 172.21.15.5:0/676270845 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55947bdd2110 con 0x55947bf6eac0 2024-09-11T22:32:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.740+0000 7f8c799dd640 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55947bf6eac0 0x55947bf70eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.741+0000 7f8c799dd640 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55947bf6eac0 0x55947bf70eb0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db8109ea7b73fa2 server_cookie=9be0f98baebd31e8 in_seq=0 out_seq=0 2024-09-11T22:32:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.741+0000 7f8c5d7fa640 10 client.18831 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c5d7fa640 1 -- 172.21.15.5:0/676270845 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55947bdd2110 con 0x55947bf6eac0 2024-09-11T22:32:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c5d7fa640 10 client.18831 handle_command_reply: tid=0 2024-09-11T22:32:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 client.18831 shutdown 2024-09-11T22:32:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 -- 172.21.15.5:0/676270845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 msgr2=0x55947beedfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947beedfe0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c02b6a0 tx=0x7f8c6c006bb0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 -- 172.21.15.5:0/676270845 shutdown_connections 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55947bf6eac0 0x55947bf70eb0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55947bef3d40 0x55947beedfe0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55947bee9480 0x55947beed9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c5d7fa640 0 client.18831 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 --2- 172.21.15.5:0/676270845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55947bee8b60 0x55947befc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 -- 172.21.15.5:0/676270845 >> 172.21.15.5:0/676270845 conn(0x55947bee50f0 msgr2=0x55947bee4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 -- 172.21.15.5:0/676270845 shutdown_connections 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 1 -- 172.21.15.5:0/676270845 wait complete. 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.742+0000 7f8c7e01a740 20 client.18831 trim_cache size 0 max 16384 2024-09-11T22:32:56.750 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:32:56.751 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:32:56.751 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "160dcccb-4e80-48a2-9ff8-cc276faf3ca6", 2024-09-11T22:32:56.751 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:32:56.751 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:32:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3568990181 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8c600a2cf0 msgr2=0x7f8c600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8c600a2cf0 0x7f8c600a51b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c016c20 tx=0x7f8c6c01a000 comp rx=0 tx=0).stop 2024-09-11T22:32:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3568990181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c74102790 msgr2=0x7f8c74074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c74102790 0x7f8c74074a30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8c64005ab0 tx=0x7f8c640042f0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3568990181 shutdown_connections 2024-09-11T22:32:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c74106f10 0x7f8c74074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c74102790 0x7f8c74074a30 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8c600a2cf0 0x7f8c600a51b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 --2- 172.21.15.5:0/3568990181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c74101de0 0x7f8c740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3568990181 >> 172.21.15.5:0/3568990181 conn(0x7f8c740fd8c0 msgr2=0x7f8c740fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3568990181 shutdown_connections 2024-09-11T22:32:56.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:56.743+0000 7f8c7bc41640 1 -- 172.21.15.5:0/3568990181 wait complete. 2024-09-11T22:32:57.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:56 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:32:57.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:56 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:57.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:56 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:32:57.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:56 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:32:57.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:56 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:57.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:56 smithi159 ceph-mon[35534]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:32:57.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:56 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:57.080 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:32:56 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:32:56.757+0000 7f54cdab7640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...10,head] /volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{1=1,2=1,3=1,4=1} v162361 ap=3 RANDEPHEMERALPIN f(v4645 m2024-09-11T22:32:43.898585+0000 3=3+0) n(v52 rc2024-09-11T22:32:43.898585+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:966 nref=3)) caps={24841=pAsLsXsFs/pAsLsXsFsx@9417} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x556c7fef9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v2606>,<0x10000000008/~dmtmp v158>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v14>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v30>,<0x10000000001/sv_1 v53>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v162361>,<0x10000000008/~dmtmp v10196>,<0x10000000007/client0 v898>,<0x10000000006/clients v811>,<0x10000000005/tmp v808>,<0x100000001fb/client.0 v811>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v816>,<0x10000000001/sv_1 v827>,<0x10000000000/qa v795>,<0x1/volumes v788>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T22:32:43.132601+0000 9=9+0)","ondisk_value.rstat":"n(v0 rc2024-09-11T22:32:43.898585+0000 b780288 4=3+1)","memory_value.dirstat":"f(v4645 m2024-09-11T22:32:43.898585+0000 3=3+0)","memory_value.rstat":"n(v52 rc2024-09-11T22:32:43.898585+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-11T22:32:57.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:56 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:32:57.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:56 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:57.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:56 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:32:57.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:56 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:32:57.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:56 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:57.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:56 smithi005 ceph-mon[30943]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:32:57.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:56 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:57.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.17 MB/sec execute 435 sec latency 12910.788 ms 2024-09-11T22:32:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:56 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:32:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:56 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:56 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:32:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:56 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:32:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:56 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:56 smithi172 ceph-mon[35461]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:32:57.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:56 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:32:57.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:32:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:32:58.173 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:57 smithi005 ceph-mon[30943]: pgmap v339: 97 pgs: 97 active+clean; 89 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-11T22:32:58.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:57 smithi159 ceph-mon[35534]: pgmap v339: 97 pgs: 97 active+clean; 89 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-11T22:32:58.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.15 MB/sec execute 436 sec latency 13910.847 ms 2024-09-11T22:32:58.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:57 smithi172 ceph-mon[35461]: pgmap v339: 97 pgs: 97 active+clean; 89 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-11T22:32:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.416+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2323360281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf78106a80 msgr2=0x7fcf78106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.416+0000 7fcf7dde6640 1 --2- 172.21.15.5:0/2323360281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf78106a80 0x7fcf78106e60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009930 tx=0x7fcf6802a420 comp rx=0 tx=0).stop 2024-09-11T22:32:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.417+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2323360281 shutdown_connections 2024-09-11T22:32:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.417+0000 7fcf7dde6640 1 --2- 172.21.15.5:0/2323360281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf78101420 0x7fcf7810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.417+0000 7fcf7dde6640 1 --2- 172.21.15.5:0/2323360281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf78100a80 0x7fcf78100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.417+0000 7fcf7dde6640 1 --2- 172.21.15.5:0/2323360281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf78106a80 0x7fcf78106e60 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.417+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2323360281 >> 172.21.15.5:0/2323360281 conn(0x7fcf780fc7a0 msgr2=0x7fcf780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.417+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2323360281 shutdown_connections 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.417+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2323360281 wait complete. 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.417+0000 7fcf7dde6640 1 Processor -- start 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.417+0000 7fcf7dde6640 1 -- start start 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf7dde6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf78100a80 0x7fcf780fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf7dde6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf78101420 0x7fcf780fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf7dde6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf78106a80 0x7fcf780fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf7dde6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78110e40 con 0x7fcf78101420 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf7dde6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78110cc0 con 0x7fcf78106a80 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf7dde6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78110fc0 con 0x7fcf78100a80 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf76ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf78101420 0x7fcf780fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf76ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf78101420 0x7fcf780fd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47816/0 (socket says 172.21.15.5:47816) 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf76ffd640 1 -- 172.21.15.5:0/2933608926 learned_addr learned my addr 172.21.15.5:0/2933608926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf777fe640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf78100a80 0x7fcf780fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf77fff640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf78106a80 0x7fcf780fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf76ffd640 1 -- 172.21.15.5:0/2933608926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf78100a80 msgr2=0x7fcf780fcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf76ffd640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf78100a80 0x7fcf780fcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf76ffd640 1 -- 172.21.15.5:0/2933608926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf78106a80 msgr2=0x7fcf780fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf76ffd640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf78106a80 0x7fcf780fd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf76ffd640 1 -- 172.21.15.5:0/2933608926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf781a6870 con 0x7fcf78101420 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf77fff640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf78106a80 0x7fcf780fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf76ffd640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf78101420 0x7fcf780fd280 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c920 tx=0x7fcf6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.418+0000 7fcf74ff9640 1 -- 172.21.15.5:0/2933608926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c007c70 con 0x7fcf78101420 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.419+0000 7fcf74ff9640 1 -- 172.21.15.5:0/2933608926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf6c007e10 con 0x7fcf78101420 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.419+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2933608926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf781a6b60 con 0x7fcf78101420 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.419+0000 7fcf74ff9640 1 -- 172.21.15.5:0/2933608926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c018890 con 0x7fcf78101420 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.419+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2933608926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf781a70f0 con 0x7fcf78101420 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.421+0000 7fcf74ff9640 1 -- 172.21.15.5:0/2933608926 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fcf6c018a70 con 0x7fcf78101420 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.421+0000 7fcf801bf740 1 Processor -- start 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.421+0000 7fcf801bf740 1 -- start start 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.421+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24ea4a0 0x55a7b24ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.421+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 0x55a7b24f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.421+0000 7fcf74ff9640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf600a2d70 0x7fcf600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.421+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 0x55a7b24f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.421+0000 7fcf801bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a7b24502e0 con 0x55a7b24f5680 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.421+0000 7fcf801bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a7b23f7ce0 con 0x55a7b24eadc0 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.421+0000 7fcf801bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7b244c2a0 con 0x55a7b24ea4a0 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf76ffd640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf600a2d70 0x7fcf600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf777fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 0x55a7b24f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 0x55a7b24f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf777fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 0x55a7b24f5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49916/0 (socket says 172.21.15.5:49916) 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf777fe640 1 -- 172.21.15.5:0/1242883963 learned_addr learned my addr 172.21.15.5:0/1242883963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf74ff9640 1 -- 172.21.15.5:0/2933608926 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fcf6c0cf5f0 con 0x7fcf78101420 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf77fff640 1 --2- 172.21.15.5:0/1242883963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24ea4a0 0x55a7b24ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf777fe640 1 -- 172.21.15.5:0/1242883963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24ea4a0 msgr2=0x55a7b24ea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf777fe640 1 --2- 172.21.15.5:0/1242883963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24ea4a0 0x55a7b24ea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf777fe640 1 -- 172.21.15.5:0/1242883963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 msgr2=0x55a7b24f7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf777fe640 1 --2- 172.21.15.5:0/1242883963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 0x55a7b24f7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf777fe640 1 -- 172.21.15.5:0/1242883963 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b23dc440 con 0x55a7b24eadc0 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf76ffd640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf600a2d70 0x7fcf600a5230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c09b7d0 tx=0x7fcf6c090f60 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.422+0000 7fcf777fe640 1 --2- 172.21.15.5:0/1242883963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 0x55a7b24f5140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf68005ab0 tx=0x7fcf68031630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b80013753a00dfe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.423+0000 7fcf5dffb640 1 -- 172.21.15.5:0/1242883963 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6803f070 con 0x55a7b24eadc0 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.423+0000 7fcf5dffb640 1 -- 172.21.15.5:0/1242883963 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf68031a20 con 0x55a7b24eadc0 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.423+0000 7fcf5dffb640 1 -- 172.21.15.5:0/1242883963 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68035070 con 0x55a7b24eadc0 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 -- 172.21.15.5:0/1242883963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 msgr2=0x55a7b24f5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 --2- 172.21.15.5:0/1242883963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 0x55a7b24f5140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf68005ab0 tx=0x7fcf68031630 comp rx=0 tx=0).stop 2024-09-11T22:32:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 -- 172.21.15.5:0/1242883963 shutdown_connections 2024-09-11T22:32:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 --2- 172.21.15.5:0/1242883963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 0x55a7b24f7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 --2- 172.21.15.5:0/1242883963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 0x55a7b24f5140 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 --2- 172.21.15.5:0/1242883963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24ea4a0 0x55a7b24ea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 -- 172.21.15.5:0/1242883963 >> 172.21.15.5:0/1242883963 conn(0x55a7b24e6a30 msgr2=0x55a7b24e6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 -- 172.21.15.5:0/1242883963 shutdown_connections 2024-09-11T22:32:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 -- 172.21.15.5:0/1242883963 wait complete. 2024-09-11T22:32:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 -- start start 2024-09-11T22:32:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 Processor -- start 2024-09-11T22:32:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24ea4a0 0x55a7b24f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.424+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24eadc0 0x55a7b24f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 0x55a7b24f2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf801bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a7b24f8250 con 0x55a7b24f5680 2024-09-11T22:32:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf801bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a7b240b050 con 0x55a7b24ea4a0 2024-09-11T22:32:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf801bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a7b24f83d0 con 0x55a7b24eadc0 2024-09-11T22:32:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf801bf740 10 client.0 _ll_register_callbacks cb 0x55a7b244b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 0x55a7b24f2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 0x55a7b24f2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47824/0 (socket says 172.21.15.5:47824) 2024-09-11T22:32:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf77fff640 1 -- 172.21.15.5:0/4286843 learned_addr learned my addr 172.21.15.5:0/4286843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf777fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24eadc0 0x55a7b24f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf77fff640 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24ea4a0 0x55a7b24f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf77fff640 1 -- 172.21.15.5:0/4286843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24eadc0 msgr2=0x55a7b24f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf77fff640 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24eadc0 0x55a7b24f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf77fff640 1 -- 172.21.15.5:0/4286843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24ea4a0 msgr2=0x55a7b24f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf77fff640 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24ea4a0 0x55a7b24f4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf77fff640 1 -- 172.21.15.5:0/4286843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b21c13c0 con 0x55a7b24f5680 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf77fff640 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 0x55a7b24f2750 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fcf70012f00 tx=0x7fcf7000aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a7244d31e4e557 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf56ffd640 1 -- 172.21.15.5:0/4286843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70021060 con 0x55a7b24f5680 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf56ffd640 1 -- 172.21.15.5:0/4286843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fcf7000f680 con 0x55a7b24f5680 2024-09-11T22:32:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.425+0000 7fcf801bf740 1 -- 172.21.15.5:0/4286843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf68031a20 con 0x55a7b24f5680 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf56ffd640 1 -- 172.21.15.5:0/4286843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcf7000f880 con 0x55a7b24f5680 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf801bf740 10 client.18843 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf801bf740 1 -- 172.21.15.5:0/4286843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a7b2503bf0 con 0x55a7b24f5680 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf56ffd640 1 -- 172.21.15.5:0/4286843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf68031a20 con 0x55a7b24f5680 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf56ffd640 1 -- 172.21.15.5:0/4286843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fcf7001b030 con 0x55a7b24f5680 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf801bf740 10 client.18843 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf801bf740 10 client.18843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf801bf740 20 client.18843 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf801bf740 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a7b25050b0 0x55a7b25074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf801bf740 4 client.18843 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:32:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf801bf740 1 -- 172.21.15.5:0/4286843 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7b245d850 con 0x55a7b25050b0 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.426+0000 7fcf777fe640 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a7b25050b0 0x55a7b25074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.427+0000 7fcf56ffd640 10 client.18843 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.427+0000 7fcf777fe640 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a7b25050b0 0x55a7b25074a0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17f2c248a597de52 server_cookie=b4bcddede42bb761 in_seq=0 out_seq=0 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.427+0000 7fcf56ffd640 1 -- 172.21.15.5:0/4286843 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a7b245d850 con 0x55a7b25050b0 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.427+0000 7fcf56ffd640 10 client.18843 handle_command_reply: tid=0 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 client.18843 shutdown 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 -- 172.21.15.5:0/4286843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 msgr2=0x55a7b24f2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 0x55a7b24f2750 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fcf70012f00 tx=0x7fcf7000aca0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 -- 172.21.15.5:0/4286843 shutdown_connections 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a7b25050b0 0x55a7b25074a0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24f5680 0x55a7b24f2750 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24eadc0 0x55a7b24f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 --2- 172.21.15.5:0/4286843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24ea4a0 0x55a7b24f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 -- 172.21.15.5:0/4286843 >> 172.21.15.5:0/4286843 conn(0x55a7b24e6a30 msgr2=0x55a7b24ec710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf56ffd640 0 client.18843 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 -- 172.21.15.5:0/4286843 shutdown_connections 2024-09-11T22:32:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 1 -- 172.21.15.5:0/4286843 wait complete. 2024-09-11T22:32:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.428+0000 7fcf801bf740 20 client.18843 trim_cache size 0 max 16384 2024-09-11T22:32:58.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.450+0000 7fcf801bf740 1 Processor -- start 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.450+0000 7fcf801bf740 1 -- start start 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.450+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24e6a30 0x55a7b24f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.450+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24ea4a0 0x55a7b250cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.450+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 0x55a7b250d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.450+0000 7fcf801bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781afad0 con 0x55a7b24e6a30 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.450+0000 7fcf801bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781b1610 con 0x55a7b24eadc0 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.450+0000 7fcf801bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781b01a0 con 0x55a7b24ea4a0 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.450+0000 7fcf77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24e6a30 0x55a7b24f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.450+0000 7fcf77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24e6a30 0x55a7b24f0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47832/0 (socket says 172.21.15.5:47832) 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf77fff640 1 -- 172.21.15.5:0/1782757828 learned_addr learned my addr 172.21.15.5:0/1782757828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:58.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf777fe640 1 --2- 172.21.15.5:0/1782757828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24ea4a0 0x55a7b250cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf77fff640 1 -- 172.21.15.5:0/1782757828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24ea4a0 msgr2=0x55a7b250cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf77fff640 1 --2- 172.21.15.5:0/1782757828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24ea4a0 0x55a7b250cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf77fff640 1 -- 172.21.15.5:0/1782757828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 msgr2=0x55a7b250d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf77fff640 1 --2- 172.21.15.5:0/1782757828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 0x55a7b250d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf77fff640 1 -- 172.21.15.5:0/1782757828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7b24f11c0 con 0x55a7b24e6a30 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf77fff640 1 --2- 172.21.15.5:0/1782757828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24e6a30 0x55a7b24f0ae0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000a5e0 tx=0x7fcf70012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf5dc9ee4fa9513 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf5d7fa640 1 -- 172.21.15.5:0/1782757828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70032070 con 0x55a7b24e6a30 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf5d7fa640 1 -- 172.21.15.5:0/1782757828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf7000f680 con 0x55a7b24e6a30 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.451+0000 7fcf5d7fa640 1 -- 172.21.15.5:0/1782757828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70021040 con 0x55a7b24e6a30 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.452+0000 7fcf801bf740 1 -- 172.21.15.5:0/1782757828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24e6a30 msgr2=0x55a7b24f0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:58.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.452+0000 7fcf801bf740 1 --2- 172.21.15.5:0/1782757828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24e6a30 0x55a7b24f0ae0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000a5e0 tx=0x7fcf70012b20 comp rx=0 tx=0).stop 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.452+0000 7fcf801bf740 1 -- 172.21.15.5:0/1782757828 shutdown_connections 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.452+0000 7fcf801bf740 1 --2- 172.21.15.5:0/1782757828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24eadc0 0x55a7b250d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.452+0000 7fcf801bf740 1 --2- 172.21.15.5:0/1782757828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24ea4a0 0x55a7b250cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.452+0000 7fcf801bf740 1 --2- 172.21.15.5:0/1782757828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24e6a30 0x55a7b24f0ae0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.452+0000 7fcf801bf740 1 -- 172.21.15.5:0/1782757828 >> 172.21.15.5:0/1782757828 conn(0x55a7b24f28b0 msgr2=0x55a7b24f4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.452+0000 7fcf801bf740 1 -- 172.21.15.5:0/1782757828 shutdown_connections 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.452+0000 7fcf801bf740 1 -- 172.21.15.5:0/1782757828 wait complete. 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.452+0000 7fcf801bf740 1 -- start start 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf801bf740 1 Processor -- start 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24e6a30 0x55a7b24fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24ea4a0 0x55a7b24fc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf801bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24eadc0 0x55a7b256ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf801bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781b0f40 con 0x55a7b24eadc0 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf801bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781b0870 con 0x55a7b24ea4a0 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf801bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781adab0 con 0x55a7b24e6a30 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf801bf740 10 client.0 _ll_register_callbacks cb 0x7fcf70005f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24eadc0 0x55a7b256ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24eadc0 0x55a7b256ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47834/0 (socket says 172.21.15.5:47834) 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf77fff640 1 -- 172.21.15.5:0/999877851 learned_addr learned my addr 172.21.15.5:0/999877851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf777fe640 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24ea4a0 0x55a7b24fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf77fff640 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24e6a30 0x55a7b24fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf77fff640 1 -- 172.21.15.5:0/999877851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24e6a30 msgr2=0x55a7b24fbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf77fff640 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24e6a30 0x55a7b24fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf77fff640 1 -- 172.21.15.5:0/999877851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24ea4a0 msgr2=0x55a7b24fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf77fff640 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24ea4a0 0x55a7b24fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf77fff640 1 -- 172.21.15.5:0/999877851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7b24f52d0 con 0x55a7b24eadc0 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf77fff640 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24eadc0 0x55a7b256ed10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000f850 tx=0x7fcf7001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb975f868fe0a57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.453+0000 7fcf56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf56ffd640 1 -- 172.21.15.5:0/999877851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf70032040 con 0x55a7b24eadc0 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf801bf740 1 -- 172.21.15.5:0/999877851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf7000f680 con 0x55a7b24eadc0 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf56ffd640 1 -- 172.21.15.5:0/999877851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fcf70004c30 con 0x55a7b24eadc0 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf56ffd640 1 -- 172.21.15.5:0/999877851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fcf700243f0 con 0x55a7b24eadc0 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf801bf740 10 client.18849 fetch_fsmap learned FSMap version 44 2024-09-11T22:32:58.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf801bf740 1 -- 172.21.15.5:0/999877851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55a7b2570450 con 0x55a7b24eadc0 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf56ffd640 1 -- 172.21.15.5:0/999877851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf7000f680 con 0x55a7b24eadc0 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf56ffd640 1 -- 172.21.15.5:0/999877851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7fcf7001b070 con 0x55a7b24eadc0 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf801bf740 10 client.18849 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf801bf740 10 client.18849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf801bf740 20 client.18849 populate_metadata read hostname 'smithi005' 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf801bf740 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a7b24f5680 0x55a7b2572b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf801bf740 4 client.18849 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.454+0000 7fcf801bf740 1 -- 172.21.15.5:0/999877851 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a7b2457d60 con 0x55a7b24f5680 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf777fe640 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a7b24f5680 0x55a7b2572b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf777fe640 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a7b24f5680 0x55a7b2572b20 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98b2cf1fa3afa76d server_cookie=c9c5baa781af0a60 in_seq=0 out_seq=0 2024-09-11T22:32:58.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf56ffd640 10 client.18849 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf56ffd640 1 -- 172.21.15.5:0/999877851 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a7b2457d60 con 0x55a7b24f5680 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf56ffd640 10 client.18849 handle_command_reply: tid=0 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf801bf740 1 client.18849 shutdown 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf801bf740 1 -- 172.21.15.5:0/999877851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24eadc0 msgr2=0x55a7b256ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf801bf740 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24eadc0 0x55a7b256ed10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000f850 tx=0x7fcf7001e040 comp rx=0 tx=0).stop 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf801bf740 1 -- 172.21.15.5:0/999877851 shutdown_connections 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf801bf740 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a7b24f5680 0x55a7b2572b20 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf801bf740 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7b24eadc0 0x55a7b256ed10 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf801bf740 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7b24ea4a0 0x55a7b24fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf801bf740 1 --2- 172.21.15.5:0/999877851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7b24e6a30 0x55a7b24fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf801bf740 1 -- 172.21.15.5:0/999877851 >> 172.21.15.5:0/999877851 conn(0x55a7b24f28b0 msgr2=0x55a7b24f4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:58.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.455+0000 7fcf56ffd640 0 client.18849 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:32:58.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf801bf740 1 -- 172.21.15.5:0/999877851 shutdown_connections 2024-09-11T22:32:58.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf801bf740 1 -- 172.21.15.5:0/999877851 wait complete. 2024-09-11T22:32:58.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf801bf740 20 client.18849 trim_cache size 0 max 16384 2024-09-11T22:32:58.466 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:32:58.466 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:32:58.467 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:32:58.467 INFO:teuthology.orchestra.run.smithi005.stdout: "160dcccb-4e80-48a2-9ff8-cc276faf3ca6": { 2024-09-11T22:32:58.467 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:32:58.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "160dcccb-4e80-48a2-9ff8-cc276faf3ca6", 2024-09-11T22:32:58.467 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:32:58.467 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:58.467 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:32:58.467 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:32:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2933608926 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf600a2d70 msgr2=0x7fcf600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf7dde6640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf600a2d70 0x7fcf600a5230 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c09b7d0 tx=0x7fcf6c090f60 comp rx=0 tx=0).stop 2024-09-11T22:32:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2933608926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf78101420 msgr2=0x7fcf780fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:32:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf7dde6640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf78101420 0x7fcf780fd280 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00c920 tx=0x7fcf6c00cde0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2933608926 shutdown_connections 2024-09-11T22:32:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf7dde6640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcf600a2d70 0x7fcf600a5230 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf7dde6640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf78106a80 0x7fcf780fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf7dde6640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf78101420 0x7fcf780fd280 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf7dde6640 1 --2- 172.21.15.5:0/2933608926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf78100a80 0x7fcf780fcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:32:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.456+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2933608926 >> 172.21.15.5:0/2933608926 conn(0x7fcf780fc7a0 msgr2=0x7fcf7806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:32:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.457+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2933608926 shutdown_connections 2024-09-11T22:32:58.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:32:58.457+0000 7fcf7dde6640 1 -- 172.21.15.5:0/2933608926 wait complete. 2024-09-11T22:32:59.308 INFO:tasks.cephfs.filesystem:scrub status for tag:160dcccb-4e80-48a2-9ff8-cc276faf3ca6 - {'path': '/', 'tag': '160dcccb-4e80-48a2-9ff8-cc276faf3ca6', 'options': 'recursive,force'} 2024-09-11T22:32:59.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.13 MB/sec execute 437 sec latency 14910.934 ms 2024-09-11T22:33:00.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:32:59 smithi005 ceph-mon[30943]: pgmap v340: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-11T22:33:00.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:32:59 smithi159 ceph-mon[35534]: pgmap v340: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-11T22:33:00.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.12 MB/sec execute 438 sec latency 15911.051 ms 2024-09-11T22:33:00.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:32:59 smithi172 ceph-mon[35461]: pgmap v340: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 1.2 MiB/s wr, 118 op/s 2024-09-11T22:33:01.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.10 MB/sec execute 439 sec latency 16911.159 ms 2024-09-11T22:33:02.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:01 smithi159 ceph-mon[35534]: pgmap v341: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 259 KiB/s wr, 63 op/s 2024-09-11T22:33:02.175 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:01 smithi172 ceph-mon[35461]: pgmap v341: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 259 KiB/s wr, 63 op/s 2024-09-11T22:33:02.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:01 smithi005 ceph-mon[30943]: pgmap v341: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 259 KiB/s wr, 63 op/s 2024-09-11T22:33:02.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.08 MB/sec execute 440 sec latency 17911.288 ms 2024-09-11T22:33:02.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:33:03.151 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:03.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.07 MB/sec execute 441 sec latency 18911.402 ms 2024-09-11T22:33:03.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.776+0000 7fbde7109640 1 -- 172.21.15.5:0/1664026797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde0102ef0 msgr2=0x7fbde010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:03.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.776+0000 7fbde7109640 1 --2- 172.21.15.5:0/1664026797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde0102ef0 0x7fbde010da40 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009ca0 tx=0x7fbdd802a950 comp rx=0 tx=0).stop 2024-09-11T22:33:03.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.776+0000 7fbde7109640 1 -- 172.21.15.5:0/1664026797 shutdown_connections 2024-09-11T22:33:03.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.776+0000 7fbde7109640 1 --2- 172.21.15.5:0/1664026797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde0102ef0 0x7fbde010da40 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:03.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.776+0000 7fbde7109640 1 --2- 172.21.15.5:0/1664026797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0102570 0x7fbde01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.776+0000 7fbde7109640 1 --2- 172.21.15.5:0/1664026797 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde0100e20 0x7fbde0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.776+0000 7fbde7109640 1 -- 172.21.15.5:0/1664026797 >> 172.21.15.5:0/1664026797 conn(0x7fbde00fc7d0 msgr2=0x7fbde00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.776+0000 7fbde7109640 1 -- 172.21.15.5:0/1664026797 shutdown_connections 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.776+0000 7fbde7109640 1 -- 172.21.15.5:0/1664026797 wait complete. 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde7109640 1 Processor -- start 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde7109640 1 -- start start 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde7109640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0100e20 0x7fbde01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde7109640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde0102570 0x7fbde0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde7109640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde0102ef0 0x7fbde019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde7109640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0110f80 con 0x7fbde0102ef0 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde7109640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0110e00 con 0x7fbde0100e20 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde7109640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0111100 con 0x7fbde0102570 2024-09-11T22:33:03.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde56a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde0102ef0 0x7fbde019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde56a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde0102ef0 0x7fbde019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35386/0 (socket says 172.21.15.5:35386) 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde56a6640 1 -- 172.21.15.5:0/3242441838 learned_addr learned my addr 172.21.15.5:0/3242441838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde4ea5640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0100e20 0x7fbde01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbddffff640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde0102570 0x7fbde0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde56a6640 1 -- 172.21.15.5:0/3242441838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde0102570 msgr2=0x7fbde0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde56a6640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde0102570 0x7fbde0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde56a6640 1 -- 172.21.15.5:0/3242441838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0100e20 msgr2=0x7fbde01984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde56a6640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0100e20 0x7fbde01984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.777+0000 7fbde56a6640 1 -- 172.21.15.5:0/3242441838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde019d490 con 0x7fbde0102ef0 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.778+0000 7fbde56a6640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde0102ef0 0x7fbde019cdb0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fbdd802ae70 tx=0x7fbdd802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.778+0000 7fbdddffb640 1 -- 172.21.15.5:0/3242441838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8034040 con 0x7fbde0102ef0 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.778+0000 7fbde7109640 1 -- 172.21.15.5:0/3242441838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde019d720 con 0x7fbde0102ef0 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.778+0000 7fbdddffb640 1 -- 172.21.15.5:0/3242441838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdd8030920 con 0x7fbde0102ef0 2024-09-11T22:33:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.778+0000 7fbdddffb640 1 -- 172.21.15.5:0/3242441838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8045480 con 0x7fbde0102ef0 2024-09-11T22:33:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.778+0000 7fbde7109640 1 -- 172.21.15.5:0/3242441838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde01a5060 con 0x7fbde0102ef0 2024-09-11T22:33:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.779+0000 7fbde7109640 1 -- 172.21.15.5:0/3242441838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde0101200 con 0x7fbde0102ef0 2024-09-11T22:33:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.780+0000 7fbdddffb640 1 -- 172.21.15.5:0/3242441838 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbdd8037020 con 0x7fbde0102ef0 2024-09-11T22:33:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.781+0000 7fbdddffb640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdd00a2cf0 0x7fbdd00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.781+0000 7fbdddffb640 1 -- 172.21.15.5:0/3242441838 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7fbdd80f0440 con 0x7fbde0102ef0 2024-09-11T22:33:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.781+0000 7fbde4ea5640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdd00a2cf0 0x7fbdd00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.782+0000 7fbde4ea5640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdd00a2cf0 0x7fbdd00a51b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400bbe0 tx=0x7fbdd400b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:03.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.784+0000 7fbdddffb640 1 -- 172.21.15.5:0/3242441838 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbdd80811b0 con 0x7fbde0102ef0 2024-09-11T22:33:03.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.997+0000 7fbde7109640 1 -- 172.21.15.5:0/3242441838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbde0199840 con 0x7fbde0102ef0 2024-09-11T22:33:03.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:03.998+0000 7fbdddffb640 1 -- 172.21.15.5:0/3242441838 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11967 (secure 0 0 0) 0x7fbde0199840 con 0x7fbde0102ef0 2024-09-11T22:33:03.999 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:33:03.999 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":44,"btime":"2024-09-11T22:32:26:501017+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:32:26.500999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27887":{"gid":27887,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1984462744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1984462744},{"type":"v1","addr":"172.21.15.159:6833","nonce":1984462744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:33:03.999 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 44 2024-09-11T22:33:04.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 -- 172.21.15.5:0/3242441838 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdd00a2cf0 msgr2=0x7fbdd00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdd00a2cf0 0x7fbdd00a51b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400bbe0 tx=0x7fbdd400b390 comp rx=0 tx=0).stop 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 -- 172.21.15.5:0/3242441838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde0102ef0 msgr2=0x7fbde019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde0102ef0 0x7fbde019cdb0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fbdd802ae70 tx=0x7fbdd802f5d0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 -- 172.21.15.5:0/3242441838 shutdown_connections 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde0102ef0 0x7fbde019cdb0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde0102570 0x7fbde0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdd00a2cf0 0x7fbdd00a51b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 --2- 172.21.15.5:0/3242441838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0100e20 0x7fbde01984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 -- 172.21.15.5:0/3242441838 >> 172.21.15.5:0/3242441838 conn(0x7fbde00fc7d0 msgr2=0x7fbde006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 -- 172.21.15.5:0/3242441838 shutdown_connections 2024-09-11T22:33:04.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:04.000+0000 7fbde7109640 1 -- 172.21.15.5:0/3242441838 wait complete. 2024-09-11T22:33:04.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:03 smithi005 ceph-mon[30943]: pgmap v342: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 308 KiB/s wr, 85 op/s 2024-09-11T22:33:04.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:03 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:33:04.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:03 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:33:04.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:03 smithi159 ceph-mon[35534]: pgmap v342: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 308 KiB/s wr, 85 op/s 2024-09-11T22:33:04.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:03 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:33:04.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:03 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:33:04.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.05 MB/sec execute 442 sec latency 19911.456 ms 2024-09-11T22:33:04.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:03 smithi172 ceph-mon[35461]: pgmap v342: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 308 KiB/s wr, 85 op/s 2024-09-11T22:33:04.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:03 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:33:04.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:03 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:33:04.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.785+0000 7f488fd31640 1 -- 172.21.15.5:0/1345507783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48881009b0 msgr2=0x7f4888100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.785+0000 7f488fd31640 1 --2- 172.21.15.5:0/1345507783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48881009b0 0x7f4888100e10 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4878009930 tx=0x7f487802a420 comp rx=0 tx=0).stop 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.785+0000 7f488fd31640 1 -- 172.21.15.5:0/1345507783 shutdown_connections 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.785+0000 7f488fd31640 1 --2- 172.21.15.5:0/1345507783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4888101350 0x7f488810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.785+0000 7f488fd31640 1 --2- 172.21.15.5:0/1345507783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48881009b0 0x7f4888100e10 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.785+0000 7f488fd31640 1 --2- 172.21.15.5:0/1345507783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48881069c0 0x7f4888106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.785+0000 7f488fd31640 1 -- 172.21.15.5:0/1345507783 >> 172.21.15.5:0/1345507783 conn(0x7f4888077990 msgr2=0x7f48880fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.785+0000 7f488fd31640 1 -- 172.21.15.5:0/1345507783 shutdown_connections 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.786+0000 7f488fd31640 1 -- 172.21.15.5:0/1345507783 wait complete. 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.786+0000 7f488fd31640 1 Processor -- start 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.786+0000 7f488fd31640 1 -- start start 2024-09-11T22:33:04.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.786+0000 7f488fd31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48881009b0 0x7f4888199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.786+0000 7f488fd31640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4888101350 0x7f4888199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.786+0000 7f488fd31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48881069c0 0x7f48881a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.786+0000 7f488fd31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4888110e00 con 0x7f48881069c0 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.786+0000 7f488fd31640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4888110c80 con 0x7f4888101350 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.786+0000 7f488fd31640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4888110f80 con 0x7f48881009b0 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488e2ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48881069c0 0x7f48881a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488e2ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48881069c0 0x7f48881a41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35416/0 (socket says 172.21.15.5:35416) 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488e2ce640 1 -- 172.21.15.5:0/2161960144 learned_addr learned my addr 172.21.15.5:0/2161960144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488dacd640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48881009b0 0x7f4888199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488d2cc640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4888101350 0x7f4888199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488e2ce640 1 -- 172.21.15.5:0/2161960144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48881009b0 msgr2=0x7f4888199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488e2ce640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48881009b0 0x7f4888199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488e2ce640 1 -- 172.21.15.5:0/2161960144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4888101350 msgr2=0x7f4888199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488e2ce640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4888101350 0x7f4888199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488e2ce640 1 -- 172.21.15.5:0/2161960144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48881a4710 con 0x7f48881069c0 2024-09-11T22:33:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488d2cc640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4888101350 0x7f4888199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488e2ce640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48881069c0 0x7f48881a41d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f488000cce0 tx=0x7f488000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f487effd640 1 -- 172.21.15.5:0/2161960144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488000f450 con 0x7f48881069c0 2024-09-11T22:33:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f487effd640 1 -- 172.21.15.5:0/2161960144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488000fad0 con 0x7f48881069c0 2024-09-11T22:33:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488fd31640 1 -- 172.21.15.5:0/2161960144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48881a48e0 con 0x7f48881069c0 2024-09-11T22:33:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f487effd640 1 -- 172.21.15.5:0/2161960144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4880018c10 con 0x7f48881069c0 2024-09-11T22:33:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.787+0000 7f488fd31640 1 -- 172.21.15.5:0/2161960144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48881a4e00 con 0x7f48881069c0 2024-09-11T22:33:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.790+0000 7f487effd640 1 -- 172.21.15.5:0/2161960144 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f488000f5f0 con 0x7f48881069c0 2024-09-11T22:33:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.791+0000 7f4892ad0740 1 Processor -- start 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.791+0000 7f4892ad0740 1 -- start start 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.791+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 0x564df1fe1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.791+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1ff0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.791+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1ff34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.791+0000 7f4892ad0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564df1f462f0 con 0x564df1fe2310 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.791+0000 7f4892ad0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564df1eee270 con 0x564df1fe19f0 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.791+0000 7f4892ad0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564df1f422b0 con 0x564df1ff10c0 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488dacd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1ff0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488dacd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1ff0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35418/0 (socket says 172.21.15.5:35418) 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488dacd640 1 -- 172.21.15.5:0/4094199478 learned_addr learned my addr 172.21.15.5:0/4094199478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f487effd640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f48740a2e20 0x7f48740a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488d2cc640 1 --2- 172.21.15.5:0/4094199478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 0x564df1fe1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488d2cc640 1 --2- 172.21.15.5:0/4094199478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1ff34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488e2ce640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f48740a2e20 0x7f48740a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488dacd640 1 -- 172.21.15.5:0/4094199478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 msgr2=0x564df1ff34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488dacd640 1 --2- 172.21.15.5:0/4094199478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1ff34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488dacd640 1 -- 172.21.15.5:0/4094199478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 msgr2=0x564df1fe1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488dacd640 1 --2- 172.21.15.5:0/4094199478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 0x564df1fe1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488dacd640 1 -- 172.21.15.5:0/4094199478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564df1eed340 con 0x564df1fe2310 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488d2cc640 1 --2- 172.21.15.5:0/4094199478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 0x564df1fe1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f487effd640 1 -- 172.21.15.5:0/2161960144 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f48800cfb00 con 0x7f48881069c0 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f488dacd640 1 --2- 172.21.15.5:0/4094199478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1ff0b80 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f488400aba0 tx=0x7f488400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cda7d13e542822 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.792+0000 7f4873fff640 1 -- 172.21.15.5:0/4094199478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488401d070 con 0x564df1fe2310 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f488e2ce640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f48740a2e20 0x7f48740a52e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f48800a4720 tx=0x7f4880066b60 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f4873fff640 1 -- 172.21.15.5:0/4094199478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4884018b20 con 0x564df1fe2310 2024-09-11T22:33:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f4873fff640 1 -- 172.21.15.5:0/4094199478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884010b30 con 0x564df1fe2310 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f4892ad0740 1 -- 172.21.15.5:0/4094199478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 msgr2=0x564df1ff0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f4892ad0740 1 --2- 172.21.15.5:0/4094199478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1ff0b80 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f488400aba0 tx=0x7f488400f790 comp rx=0 tx=0).stop 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f4892ad0740 1 -- 172.21.15.5:0/4094199478 shutdown_connections 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f4892ad0740 1 --2- 172.21.15.5:0/4094199478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1ff34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f4892ad0740 1 --2- 172.21.15.5:0/4094199478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1ff0b80 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f4892ad0740 1 --2- 172.21.15.5:0/4094199478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 0x564df1fe1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f4892ad0740 1 -- 172.21.15.5:0/4094199478 >> 172.21.15.5:0/4094199478 conn(0x564df1fde060 msgr2=0x564df1fde440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.793+0000 7f4892ad0740 1 -- 172.21.15.5:0/4094199478 shutdown_connections 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.794+0000 7f4892ad0740 1 -- 172.21.15.5:0/4094199478 wait complete. 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.794+0000 7f4892ad0740 1 -- start start 2024-09-11T22:33:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.794+0000 7f4892ad0740 1 Processor -- start 2024-09-11T22:33:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.794+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 0x564df1fedf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.794+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1fee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.794+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1feea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.794+0000 7f4892ad0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564df1ff3d20 con 0x564df1fe2310 2024-09-11T22:33:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.794+0000 7f4892ad0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564df1ecf290 con 0x564df1fe19f0 2024-09-11T22:33:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.794+0000 7f4892ad0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564df1ff3ea0 con 0x564df1ff10c0 2024-09-11T22:33:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f4892ad0740 10 client.0 _ll_register_callbacks cb 0x564df1f00140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488dacd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1fee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488dacd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1fee4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35422/0 (socket says 172.21.15.5:35422) 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488dacd640 1 -- 172.21.15.5:0/940835970 learned_addr learned my addr 172.21.15.5:0/940835970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488d2cc640 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 0x564df1fedf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488dacd640 1 -- 172.21.15.5:0/940835970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 msgr2=0x564df1feea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488d2cc640 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1feea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488dacd640 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1feea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488dacd640 1 -- 172.21.15.5:0/940835970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 msgr2=0x564df1fedf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488dacd640 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 0x564df1fedf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488dacd640 1 -- 172.21.15.5:0/940835970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df1c59210 con 0x564df1fe2310 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488d2cc640 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1feea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f488dacd640 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1fee4c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f4884000c00 tx=0x7f48840186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9c5e1ce95fd039 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f48717fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:04.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f48717fa640 1 -- 172.21.15.5:0/940835970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488401d050 con 0x564df1fe2310 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f4892ad0740 1 -- 172.21.15.5:0/940835970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4884018b20 con 0x564df1fe2310 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f48717fa640 1 -- 172.21.15.5:0/940835970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f4884010a70 con 0x564df1fe2310 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.795+0000 7f48717fa640 1 -- 172.21.15.5:0/940835970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f4884032050 con 0x564df1fe2310 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f4892ad0740 10 client.18867 fetch_fsmap learned FSMap version 44 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f4892ad0740 1 -- 172.21.15.5:0/940835970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564df1d25620 con 0x564df1fe2310 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f48717fa640 1 -- 172.21.15.5:0/940835970 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4884018b20 con 0x564df1fe2310 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f48717fa640 1 -- 172.21.15.5:0/940835970 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f48840046c0 con 0x564df1fe2310 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f4892ad0740 10 client.18867 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f4892ad0740 10 client.18867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f4892ad0740 20 client.18867 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f4892ad0740 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564df2000fb0 0x564df20033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f4892ad0740 4 client.18867 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f4892ad0740 1 -- 172.21.15.5:0/940835970 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564df1f53a20 con 0x564df2000fb0 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.796+0000 7f488d2cc640 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564df2000fb0 0x564df20033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.797+0000 7f48717fa640 10 client.18867 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.797+0000 7f488d2cc640 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564df2000fb0 0x564df20033a0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98331bdfa60c166a server_cookie=dbcc775caa3289d5 in_seq=0 out_seq=0 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.797+0000 7f48717fa640 1 -- 172.21.15.5:0/940835970 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564df1f53a20 con 0x564df2000fb0 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.797+0000 7f48717fa640 10 client.18867 handle_command_reply: tid=0 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 client.18867 shutdown 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 -- 172.21.15.5:0/940835970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 msgr2=0x564df1fee4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1fee4c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f4884000c00 tx=0x7f48840186f0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 -- 172.21.15.5:0/940835970 shutdown_connections 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564df2000fb0 0x564df20033a0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1feea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df1fee4c0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f48717fa640 0 client.18867 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 --2- 172.21.15.5:0/940835970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe19f0 0x564df1fedf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 -- 172.21.15.5:0/940835970 >> 172.21.15.5:0/940835970 conn(0x564df1fde060 msgr2=0x564df1fe3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 -- 172.21.15.5:0/940835970 shutdown_connections 2024-09-11T22:33:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 1 -- 172.21.15.5:0/940835970 wait complete. 2024-09-11T22:33:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.798+0000 7f4892ad0740 20 client.18867 trim_cache size 0 max 16384 2024-09-11T22:33:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f4892ad0740 1 Processor -- start 2024-09-11T22:33:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f4892ad0740 1 -- start start 2024-09-11T22:33:04.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1fe19f0 0x564df1fefef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df2008970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1ff10c0 0x564df2008eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f4892ad0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48881ab610 con 0x564df1fe2310 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f4892ad0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f48881ad150 con 0x564df1ff10c0 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f4892ad0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f48881abce0 con 0x564df1fe19f0 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f488dacd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df2008970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f488dacd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df2008970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35436/0 (socket says 172.21.15.5:35436) 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f488dacd640 1 -- 172.21.15.5:0/3674118364 learned_addr learned my addr 172.21.15.5:0/3674118364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f488d2cc640 1 --2- 172.21.15.5:0/3674118364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1ff10c0 0x564df2008eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f488dacd640 1 -- 172.21.15.5:0/3674118364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1fe19f0 msgr2=0x564df1fefef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f488dacd640 1 --2- 172.21.15.5:0/3674118364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1fe19f0 0x564df1fefef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f488dacd640 1 -- 172.21.15.5:0/3674118364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1ff10c0 msgr2=0x564df2008eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f488dacd640 1 --2- 172.21.15.5:0/3674118364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1ff10c0 0x564df2008eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f488dacd640 1 -- 172.21.15.5:0/3674118364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564df1fe2140 con 0x564df1fe2310 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.813+0000 7f488d2cc640 1 --2- 172.21.15.5:0/3674118364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1ff10c0 0x564df2008eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f488dacd640 1 --2- 172.21.15.5:0/3674118364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df2008970 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f488400a610 tx=0x7f488400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f0cc46de262436 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f48737fe640 1 -- 172.21.15.5:0/3674118364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488401d040 con 0x564df1fe2310 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f48737fe640 1 -- 172.21.15.5:0/3674118364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4884029070 con 0x564df1fe2310 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f48737fe640 1 -- 172.21.15.5:0/3674118364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4884010a30 con 0x564df1fe2310 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f4892ad0740 1 -- 172.21.15.5:0/3674118364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 msgr2=0x564df2008970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f4892ad0740 1 --2- 172.21.15.5:0/3674118364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df2008970 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f488400a610 tx=0x7f488400f790 comp rx=0 tx=0).stop 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f4892ad0740 1 -- 172.21.15.5:0/3674118364 shutdown_connections 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f4892ad0740 1 --2- 172.21.15.5:0/3674118364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1ff10c0 0x564df2008eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f4892ad0740 1 --2- 172.21.15.5:0/3674118364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe2310 0x564df2008970 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f4892ad0740 1 --2- 172.21.15.5:0/3674118364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1fe19f0 0x564df1fefef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.814+0000 7f4892ad0740 1 -- 172.21.15.5:0/3674118364 >> 172.21.15.5:0/3674118364 conn(0x564df1fde060 msgr2=0x564df1fef7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 1 -- 172.21.15.5:0/3674118364 shutdown_connections 2024-09-11T22:33:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 1 -- 172.21.15.5:0/3674118364 wait complete. 2024-09-11T22:33:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 1 -- start start 2024-09-11T22:33:04.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 1 Processor -- start 2024-09-11T22:33:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe19f0 0x564df2063af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe2310 0x564df2064030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1ff8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48881aca80 con 0x564df1fe19f0 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f48881ac3b0 con 0x564df1fe2310 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f48881a9610 con 0x564df1ff10c0 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f4892ad0740 10 client.0 _ll_register_callbacks cb 0x7f488401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.815+0000 7f488d2cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe19f0 0x564df2063af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f488d2cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe19f0 0x564df2063af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35442/0 (socket says 172.21.15.5:35442) 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f488d2cc640 1 -- 172.21.15.5:0/2460252852 learned_addr learned my addr 172.21.15.5:0/2460252852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f488dacd640 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe2310 0x564df2064030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f488d2cc640 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1ff8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f488d2cc640 1 -- 172.21.15.5:0/2460252852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 msgr2=0x564df1ff8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f488d2cc640 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1ff8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f488d2cc640 1 -- 172.21.15.5:0/2460252852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe2310 msgr2=0x564df2064030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:04.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f488d2cc640 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe2310 0x564df2064030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f488d2cc640 1 -- 172.21.15.5:0/2460252852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564df1ff94e0 con 0x564df1fe19f0 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f488d2cc640 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe19f0 0x564df2063af0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f4878009930 tx=0x7f4878000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d584d9413396bcf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f48717fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f48717fa640 1 -- 172.21.15.5:0/2460252852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878035040 con 0x564df1fe19f0 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f4892ad0740 1 -- 172.21.15.5:0/2460252852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4884029070 con 0x564df1fe19f0 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f48717fa640 1 -- 172.21.15.5:0/2460252852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f48780042d0 con 0x564df1fe19f0 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f48717fa640 1 -- 172.21.15.5:0/2460252852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f48780044d0 con 0x564df1fe19f0 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.816+0000 7f4892ad0740 10 client.18873 fetch_fsmap learned FSMap version 44 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.817+0000 7f4892ad0740 1 -- 172.21.15.5:0/2460252852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x564df206b380 con 0x564df1fe19f0 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.817+0000 7f48717fa640 1 -- 172.21.15.5:0/2460252852 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4884029070 con 0x564df1fe19f0 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.817+0000 7f48717fa640 1 -- 172.21.15.5:0/2460252852 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f4884019ce0 con 0x564df1fe19f0 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.817+0000 7f4892ad0740 10 client.18873 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:33:04.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.817+0000 7f4892ad0740 10 client.18873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.817+0000 7f4892ad0740 20 client.18873 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.817+0000 7f4892ad0740 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564df206bb30 0x564df206df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.817+0000 7f4892ad0740 4 client.18873 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:33:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.817+0000 7f4892ad0740 1 -- 172.21.15.5:0/2460252852 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564df1f4dd70 con 0x564df206bb30 2024-09-11T22:33:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.817+0000 7f488dacd640 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564df206bb30 0x564df206df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.818+0000 7f488dacd640 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564df206bb30 0x564df206df50 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae979efb8e31677a server_cookie=2da9cabd95d4456a in_seq=0 out_seq=0 2024-09-11T22:33:04.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.818+0000 7f48717fa640 10 client.18873 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.836+0000 7f48717fa640 1 -- 172.21.15.5:0/2460252852 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564df1f4dd70 con 0x564df206bb30 2024-09-11T22:33:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.836+0000 7f48717fa640 10 client.18873 handle_command_reply: tid=0 2024-09-11T22:33:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.836+0000 7f4892ad0740 1 client.18873 shutdown 2024-09-11T22:33:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 1 -- 172.21.15.5:0/2460252852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe19f0 msgr2=0x564df2063af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe19f0 0x564df2063af0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f4878009930 tx=0x7f4878000bc0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 1 -- 172.21.15.5:0/2460252852 shutdown_connections 2024-09-11T22:33:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564df206bb30 0x564df206df50 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564df1ff10c0 0x564df1ff8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564df1fe2310 0x564df2064030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 1 --2- 172.21.15.5:0/2460252852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564df1fe19f0 0x564df2063af0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 1 -- 172.21.15.5:0/2460252852 >> 172.21.15.5:0/2460252852 conn(0x564df1fde060 msgr2=0x564df1fee740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f48717fa640 0 client.18873 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 1 -- 172.21.15.5:0/2460252852 shutdown_connections 2024-09-11T22:33:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 1 -- 172.21.15.5:0/2460252852 wait complete. 2024-09-11T22:33:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.837+0000 7f4892ad0740 20 client.18873 trim_cache size 0 max 16384 2024-09-11T22:33:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:33:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:33:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:33:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:33:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 -- 172.21.15.5:0/2161960144 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f48740a2e20 msgr2=0x7f48740a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f48740a2e20 0x7f48740a52e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f48800a4720 tx=0x7f4880066b60 comp rx=0 tx=0).stop 2024-09-11T22:33:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 -- 172.21.15.5:0/2161960144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48881069c0 msgr2=0x7f48881a41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48881069c0 0x7f48881a41d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f488000cce0 tx=0x7f488000a620 comp rx=0 tx=0).stop 2024-09-11T22:33:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 -- 172.21.15.5:0/2161960144 shutdown_connections 2024-09-11T22:33:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48881069c0 0x7f48881a41d0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f48740a2e20 0x7f48740a52e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4888101350 0x7f4888199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 --2- 172.21.15.5:0/2161960144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48881009b0 0x7f4888199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 -- 172.21.15.5:0/2161960144 >> 172.21.15.5:0/2161960144 conn(0x7f4888077990 msgr2=0x7f48881036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 -- 172.21.15.5:0/2161960144 shutdown_connections 2024-09-11T22:33:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:04.838+0000 7f488fd31640 1 -- 172.21.15.5:0/2161960144 wait complete. 2024-09-11T22:33:04.911 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 9f4d187b --release --await' 2024-09-11T22:33:04.911 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 9f4d187b --release --await 2024-09-11T22:33:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:05.288 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:04 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3242441838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:33:05.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:04 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3242441838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:33:05.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789632 7.04 MB/sec execute 443 sec latency 20911.541 ms 2024-09-11T22:33:05.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:04 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3242441838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:33:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 -- 172.21.15.5:0/1614361952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc1082a0 msgr2=0x7f61dc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 --2- 172.21.15.5:0/1614361952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc1082a0 0x7f61dc10ec40 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009930 tx=0x7f61d402a420 comp rx=0 tx=0).stop 2024-09-11T22:33:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 -- 172.21.15.5:0/1614361952 shutdown_connections 2024-09-11T22:33:05.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 --2- 172.21.15.5:0/1614361952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc1082a0 0x7f61dc10ec40 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 --2- 172.21.15.5:0/1614361952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61dc1036e0 0x7f61dc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 --2- 172.21.15.5:0/1614361952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61dc102d30 0x7f61dc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 -- 172.21.15.5:0/1614361952 >> 172.21.15.5:0/1614361952 conn(0x7f61dc0fc7d0 msgr2=0x7f61dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 -- 172.21.15.5:0/1614361952 shutdown_connections 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 -- 172.21.15.5:0/1614361952 wait complete. 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 Processor -- start 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.695+0000 7f61e38ef640 1 -- start start 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e38ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102d30 0x7f61dc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e38ef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61dc1036e0 0x7f61dc198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e38ef640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61dc1082a0 0x7f61dc19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e38ef640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1120a0 con 0x7f61dc102d30 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e38ef640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc111f20 con 0x7f61dc1082a0 2024-09-11T22:33:05.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e38ef640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc112220 con 0x7f61dc1036e0 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e0e8a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61dc1036e0 0x7f61dc198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e168b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102d30 0x7f61dc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e0e8a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61dc1036e0 0x7f61dc198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:50226/0 (socket says 172.21.15.5:50226) 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e0e8a640 1 -- 172.21.15.5:0/2918104657 learned_addr learned my addr 172.21.15.5:0/2918104657 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61dc1082a0 0x7f61dc19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e168b640 1 -- 172.21.15.5:0/2918104657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61dc1036e0 msgr2=0x7f61dc198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e168b640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61dc1036e0 0x7f61dc198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e168b640 1 -- 172.21.15.5:0/2918104657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61dc1082a0 msgr2=0x7f61dc19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e168b640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61dc1082a0 0x7f61dc19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.696+0000 7f61e168b640 1 -- 172.21.15.5:0/2918104657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc19d570 con 0x7f61dc102d30 2024-09-11T22:33:05.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.697+0000 7f61e168b640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102d30 0x7f61dc1985c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00cce0 tx=0x7f61cc0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.697+0000 7f61da7fc640 1 -- 172.21.15.5:0/2918104657 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc004b20 con 0x7f61dc102d30 2024-09-11T22:33:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.697+0000 7f61da7fc640 1 -- 172.21.15.5:0/2918104657 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61cc004cc0 con 0x7f61dc102d30 2024-09-11T22:33:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.697+0000 7f61e38ef640 1 -- 172.21.15.5:0/2918104657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc19d860 con 0x7f61dc102d30 2024-09-11T22:33:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.697+0000 7f61da7fc640 1 -- 172.21.15.5:0/2918104657 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00f6b0 con 0x7f61dc102d30 2024-09-11T22:33:05.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.697+0000 7f61e38ef640 1 -- 172.21.15.5:0/2918104657 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1008c0 con 0x7f61dc102d30 2024-09-11T22:33:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.700+0000 7f61da7fc640 1 -- 172.21.15.5:0/2918104657 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f61cc005000 con 0x7f61dc102d30 2024-09-11T22:33:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.700+0000 7f61e5cc8740 1 Processor -- start 2024-09-11T22:33:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.700+0000 7f61e5cc8740 1 -- start start 2024-09-11T22:33:05.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.700+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efd2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.700+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2980 0x557f5efdcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.700+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 0x557f5efdf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.700+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f5ef37620 con 0x557f5efd2060 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.700+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557f5eedee50 con 0x557f5efdd240 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.700+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557f5ef335e0 con 0x557f5efd2980 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.700+0000 7f61e1e8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efd2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e1e8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efd2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35468/0 (socket says 172.21.15.5:35468) 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e1e8c640 1 -- 172.21.15.5:0/2886407483 learned_addr learned my addr 172.21.15.5:0/2886407483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e168b640 1 --2- 172.21.15.5:0/2886407483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2980 0x557f5efdcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61da7fc640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61c40a2cf0 0x7f61c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e0e8a640 1 --2- 172.21.15.5:0/2886407483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 0x557f5efdf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e1e8c640 1 -- 172.21.15.5:0/2886407483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2980 msgr2=0x557f5efdcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/2886407483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2980 0x557f5efdcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e1e8c640 1 -- 172.21.15.5:0/2886407483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 msgr2=0x557f5efdf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/2886407483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 0x557f5efdf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e1e8c640 1 -- 172.21.15.5:0/2886407483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f5eecfd40 con 0x557f5efd2060 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e0e8a640 1 --2- 172.21.15.5:0/2886407483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 0x557f5efdf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61c40a2cf0 0x7f61c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61da7fc640 1 -- 172.21.15.5:0/2918104657 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f61cc0cf770 con 0x7f61dc102d30 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/2886407483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efd2440 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009930 tx=0x7f61d4031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437731b27ee400e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61c37fe640 1 -- 172.21.15.5:0/2886407483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4035040 con 0x557f5efd2060 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61c37fe640 1 -- 172.21.15.5:0/2886407483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4003c10 con 0x557f5efd2060 2024-09-11T22:33:05.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.701+0000 7f61c37fe640 1 -- 172.21.15.5:0/2886407483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4046c00 con 0x557f5efd2060 2024-09-11T22:33:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.702+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2886407483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 msgr2=0x557f5efd2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.702+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61c40a2cf0 0x7f61c40a51b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f61d404b000 tx=0x7f61d4048000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.702+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2886407483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efd2440 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009930 tx=0x7f61d4031680 comp rx=0 tx=0).stop 2024-09-11T22:33:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.702+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2886407483 shutdown_connections 2024-09-11T22:33:05.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.702+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2886407483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 0x557f5efdf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.702+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2886407483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2980 0x557f5efdcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.702+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2886407483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efd2440 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.702+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2886407483 >> 172.21.15.5:0/2886407483 conn(0x557f5efce5f0 msgr2=0x557f5efce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.702+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2886407483 shutdown_connections 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.702+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2886407483 wait complete. 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e5cc8740 1 -- start start 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e5cc8740 1 Processor -- start 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2060 0x557f5efda170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2980 0x557f5efda6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 0x557f5efdabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557f5efdfe10 con 0x557f5efd2980 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557f5eebff10 con 0x557f5efdd240 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557f5efdff90 con 0x557f5efd2060 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e5cc8740 10 client.0 _ll_register_callbacks cb 0x557f5eee3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e0e8a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2980 0x557f5efda6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e0e8a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2980 0x557f5efda6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35472/0 (socket says 172.21.15.5:35472) 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e0e8a640 1 -- 172.21.15.5:0/1861248458 learned_addr learned my addr 172.21.15.5:0/1861248458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.703+0000 7f61e168b640 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2060 0x557f5efda170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 0x557f5efdabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e0e8a640 1 -- 172.21.15.5:0/1861248458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2060 msgr2=0x557f5efda170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e0e8a640 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2060 0x557f5efda170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e0e8a640 1 -- 172.21.15.5:0/1861248458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 msgr2=0x557f5efdabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e0e8a640 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 0x557f5efdabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e0e8a640 1 -- 172.21.15.5:0/1861248458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f5ec46580 con 0x557f5efd2980 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 0x557f5efdabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e0e8a640 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2980 0x557f5efda6b0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f61d000bae0 tx=0x7f61d00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e5905fb93e3868 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61c0ff9640 1 -- 172.21.15.5:0/1861248458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0011440 con 0x557f5efd2980 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61c0ff9640 1 -- 172.21.15.5:0/1861248458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f61d00115e0 con 0x557f5efd2980 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e5cc8740 1 -- 172.21.15.5:0/1861248458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4003c10 con 0x557f5efd2980 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61c0ff9640 1 -- 172.21.15.5:0/1861248458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f61d00117e0 con 0x557f5efd2980 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e5cc8740 10 client.18885 fetch_fsmap learned FSMap version 44 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61e5cc8740 1 -- 172.21.15.5:0/1861248458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557f5ed10960 con 0x557f5efd2980 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.704+0000 7f61c0ff9640 1 -- 172.21.15.5:0/1861248458 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4003c10 con 0x557f5efd2980 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.705+0000 7f61c0ff9640 1 -- 172.21.15.5:0/1861248458 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f61d0010c80 con 0x557f5efd2980 2024-09-11T22:33:05.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.705+0000 7f61e5cc8740 10 client.18885 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.705+0000 7f61e5cc8740 10 client.18885 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.705+0000 7f61e5cc8740 20 client.18885 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.705+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557f5efecf00 0x557f5efef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.705+0000 7f61e5cc8740 4 client.18885 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.705+0000 7f61e5cc8740 1 -- 172.21.15.5:0/1861248458 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f5ef44b90 con 0x557f5efecf00 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.705+0000 7f61e168b640 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557f5efecf00 0x557f5efef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.706+0000 7f61e168b640 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557f5efecf00 0x557f5efef2f0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e9d3e16503adda7 server_cookie=89d337ee6caf8f5c in_seq=0 out_seq=0 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.706+0000 7f61c0ff9640 10 client.18885 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.706+0000 7f61c0ff9640 1 -- 172.21.15.5:0/1861248458 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557f5ef44b90 con 0x557f5efecf00 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.706+0000 7f61c0ff9640 10 client.18885 handle_command_reply: tid=0 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.706+0000 7f61e5cc8740 1 client.18885 shutdown 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.706+0000 7f61e5cc8740 1 -- 172.21.15.5:0/1861248458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2980 msgr2=0x557f5efda6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:05.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.706+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2980 0x557f5efda6b0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f61d000bae0 tx=0x7f61d00044c0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.706+0000 7f61e5cc8740 1 -- 172.21.15.5:0/1861248458 shutdown_connections 2024-09-11T22:33:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.706+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557f5efecf00 0x557f5efef2f0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.707+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efdd240 0x557f5efdabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.707+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2980 0x557f5efda6b0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.707+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/1861248458 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2060 0x557f5efda170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.707+0000 7f61e5cc8740 1 -- 172.21.15.5:0/1861248458 >> 172.21.15.5:0/1861248458 conn(0x557f5efce5f0 msgr2=0x557f5efd42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.707+0000 7f61c0ff9640 0 client.18885 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.707+0000 7f61e5cc8740 1 -- 172.21.15.5:0/1861248458 shutdown_connections 2024-09-11T22:33:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.707+0000 7f61e5cc8740 1 -- 172.21.15.5:0/1861248458 wait complete. 2024-09-11T22:33:05.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.707+0000 7f61e5cc8740 20 client.18885 trim_cache size 0 max 16384 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e5cc8740 1 Processor -- start 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e5cc8740 1 -- start start 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2060 0x557f5efcd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 0x557f5eff4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efdd240 0x557f5eff5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1afd20 con 0x557f5efdd240 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1b1860 con 0x557f5efd2980 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1b03f0 con 0x557f5efd2060 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e1e8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efdd240 0x557f5eff5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e1e8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efdd240 0x557f5eff5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35476/0 (socket says 172.21.15.5:35476) 2024-09-11T22:33:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e1e8c640 1 -- 172.21.15.5:0/2675507492 learned_addr learned my addr 172.21.15.5:0/2675507492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e0e8a640 1 --2- 172.21.15.5:0/2675507492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 0x557f5eff4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e168b640 1 --2- 172.21.15.5:0/2675507492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2060 0x557f5efcd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e1e8c640 1 -- 172.21.15.5:0/2675507492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2060 msgr2=0x557f5efcd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/2675507492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2060 0x557f5efcd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e1e8c640 1 -- 172.21.15.5:0/2675507492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 msgr2=0x557f5eff4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/2675507492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 0x557f5eff4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e1e8c640 1 -- 172.21.15.5:0/2675507492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f5efd86c0 con 0x557f5efdd240 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.724+0000 7f61e0e8a640 1 --2- 172.21.15.5:0/2675507492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 0x557f5eff4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/2675507492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efdd240 0x557f5eff5140 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f61d4005940 tx=0x7f61d402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7339c6cfe14003 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61c2ffd640 1 -- 172.21.15.5:0/2675507492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d405a070 con 0x557f5efdd240 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61c2ffd640 1 -- 172.21.15.5:0/2675507492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4003c10 con 0x557f5efdd240 2024-09-11T22:33:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61c2ffd640 1 -- 172.21.15.5:0/2675507492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4035030 con 0x557f5efdd240 2024-09-11T22:33:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2675507492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efdd240 msgr2=0x557f5eff5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2675507492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efdd240 0x557f5eff5140 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f61d4005940 tx=0x7f61d402a420 comp rx=0 tx=0).stop 2024-09-11T22:33:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2675507492 shutdown_connections 2024-09-11T22:33:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2675507492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efdd240 0x557f5eff5140 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2675507492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 0x557f5eff4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2675507492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efd2060 0x557f5efcd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.725+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2675507492 >> 172.21.15.5:0/2675507492 conn(0x557f5efce5f0 msgr2=0x557f5efccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2675507492 shutdown_connections 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2675507492 wait complete. 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 1 -- start start 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 1 Processor -- start 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efe1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 0x557f5efe1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efdd240 0x557f5efe5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1b1190 con 0x557f5efd2060 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1b0ac0 con 0x557f5efd2980 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1add00 con 0x557f5efdd240 2024-09-11T22:33:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e5cc8740 10 client.0 _ll_register_callbacks cb 0x7f61d0004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.726+0000 7f61e168b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efe1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e168b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efe1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35482/0 (socket says 172.21.15.5:35482) 2024-09-11T22:33:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e168b640 1 -- 172.21.15.5:0/2135142827 learned_addr learned my addr 172.21.15.5:0/2135142827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e1e8c640 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efdd240 0x557f5efe5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e0e8a640 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 0x557f5efe1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e168b640 1 -- 172.21.15.5:0/2135142827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efdd240 msgr2=0x557f5efe5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e168b640 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efdd240 0x557f5efe5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e168b640 1 -- 172.21.15.5:0/2135142827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 msgr2=0x557f5efe1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e168b640 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 0x557f5efe1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e168b640 1 -- 172.21.15.5:0/2135142827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f5efe56d0 con 0x557f5efd2060 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e168b640 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efe1a10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f61cc098100 tx=0x7f61cc0d5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903d6cf92089161 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61c0ff9640 1 -- 172.21.15.5:0/2135142827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61cc01d040 con 0x557f5efd2060 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61c0ff9640 1 -- 172.21.15.5:0/2135142827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8585+0+0 (secure 0 0 0) 0x7f61cc098310 con 0x557f5efd2060 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2135142827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d4003c10 con 0x557f5efd2060 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61c0ff9640 1 -- 172.21.15.5:0/2135142827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f61cc09b7e0 con 0x557f5efd2060 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e5cc8740 10 client.18891 fetch_fsmap learned FSMap version 44 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.727+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2135142827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557f5f0587a0 con 0x557f5efd2060 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.728+0000 7f61c0ff9640 1 -- 172.21.15.5:0/2135142827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d4003c10 con 0x557f5efd2060 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.728+0000 7f61c0ff9640 1 -- 172.21.15.5:0/2135142827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f61cc0a43d0 con 0x557f5efd2060 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.728+0000 7f61e5cc8740 10 client.18891 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:33:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.728+0000 7f61e5cc8740 10 client.18891 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:33:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.728+0000 7f61e5cc8740 20 client.18891 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.728+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557f5f058ec0 0x557f5f05b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.728+0000 7f61e5cc8740 4 client.18891 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9f4d187b", "release": true, "await": true}] 2024-09-11T22:33:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.728+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2135142827 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9f4d187b", "release": true, "await": true}) -- 0x557f5ef3f0a0 con 0x557f5f058ec0 2024-09-11T22:33:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.728+0000 7f61e0e8a640 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557f5f058ec0 0x557f5f05b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.729+0000 7f61c0ff9640 10 client.18891 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.729+0000 7f61e0e8a640 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557f5f058ec0 0x557f5f05b2b0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=900bcad6ac36c4cd server_cookie=63c3342cba2188f0 in_seq=0 out_seq=0 2024-09-11T22:33:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.741+0000 7f61c0ff9640 1 -- 172.21.15.5:0/2135142827 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557f5ef3f0a0 con 0x557f5f058ec0 2024-09-11T22:33:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.741+0000 7f61c0ff9640 10 client.18891 handle_command_reply: tid=0 2024-09-11T22:33:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.741+0000 7f61e5cc8740 1 client.18891 shutdown 2024-09-11T22:33:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2135142827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 msgr2=0x557f5efe1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efe1a10 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f61cc098100 tx=0x7f61cc0d5530 comp rx=0 tx=0).stop 2024-09-11T22:33:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2135142827 shutdown_connections 2024-09-11T22:33:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557f5f058ec0 0x557f5f05b2b0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557f5efdd240 0x557f5efe5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557f5efd2980 0x557f5efe1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 1 --2- 172.21.15.5:0/2135142827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f5efd2060 0x557f5efe1a10 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2135142827 >> 172.21.15.5:0/2135142827 conn(0x557f5efce5f0 msgr2=0x557f5efcd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61c0ff9640 0 client.18891 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2135142827 shutdown_connections 2024-09-11T22:33:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 1 -- 172.21.15.5:0/2135142827 wait complete. 2024-09-11T22:33:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.742+0000 7f61e5cc8740 20 client.18891 trim_cache size 0 max 16384 2024-09-11T22:33:05.744 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 44, 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 44, 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "9f4d187b": { 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 44, 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:33:05.745 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.6, 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:33:05.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:33:05.747 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:33:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 -- 172.21.15.5:0/2918104657 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61c40a2cf0 msgr2=0x7f61c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61c40a2cf0 0x7f61c40a51b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f61d404b000 tx=0x7f61d4048000 comp rx=0 tx=0).stop 2024-09-11T22:33:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 -- 172.21.15.5:0/2918104657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102d30 msgr2=0x7f61dc1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102d30 0x7f61dc1985c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f61cc00cce0 tx=0x7f61cc0075f0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 -- 172.21.15.5:0/2918104657 shutdown_connections 2024-09-11T22:33:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f61c40a2cf0 0x7f61c40a51b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61dc1082a0 0x7f61dc19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61dc1036e0 0x7f61dc198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 --2- 172.21.15.5:0/2918104657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61dc102d30 0x7f61dc1985c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 -- 172.21.15.5:0/2918104657 >> 172.21.15.5:0/2918104657 conn(0x7f61dc0fc7d0 msgr2=0x7f61dc106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 -- 172.21.15.5:0/2918104657 shutdown_connections 2024-09-11T22:33:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:05.743+0000 7f61e38ef640 1 -- 172.21.15.5:0/2918104657 wait complete. 2024-09-11T22:33:06.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:05 smithi005 ceph-mon[30943]: pgmap v343: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-11T22:33:06.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:05 smithi159 ceph-mon[35534]: pgmap v343: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-11T22:33:06.341 INFO:tasks.workunit.client.0.smithi005.stdout: 1 791007 7.03 MB/sec execute 444 sec latency 21314.413 ms 2024-09-11T22:33:06.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:05 smithi172 ceph-mon[35461]: pgmap v343: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-11T22:33:06.590 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9f4d187b', total seconds elapsed: 26.2 2024-09-11T22:33:07.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 792392 7.04 MB/sec execute 445 sec latency 19.627 ms 2024-09-11T22:33:08.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:07 smithi005 ceph-mon[30943]: pgmap v344: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-11T22:33:08.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:07 smithi159 ceph-mon[35534]: pgmap v344: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-11T22:33:08.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 794776 7.05 MB/sec execute 446 sec latency 23.520 ms 2024-09-11T22:33:08.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:07 smithi172 ceph-mon[35461]: pgmap v344: 97 pgs: 97 active+clean; 86 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-09-11T22:33:09.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 795628 7.05 MB/sec execute 447 sec latency 84.102 ms 2024-09-11T22:33:10.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:09 smithi005 ceph-mon[30943]: pgmap v345: 97 pgs: 97 active+clean; 88 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 217 KiB/s wr, 63 op/s 2024-09-11T22:33:10.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:09 smithi005 ceph-mon[30943]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T22:33:10.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:09 smithi159 ceph-mon[35534]: pgmap v345: 97 pgs: 97 active+clean; 88 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 217 KiB/s wr, 63 op/s 2024-09-11T22:33:10.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:09 smithi159 ceph-mon[35534]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T22:33:10.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 796500 7.04 MB/sec execute 448 sec latency 67.618 ms 2024-09-11T22:33:10.359 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:09 smithi172 ceph-mon[35461]: pgmap v345: 97 pgs: 97 active+clean; 88 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 217 KiB/s wr, 63 op/s 2024-09-11T22:33:10.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:09 smithi172 ceph-mon[35461]: osdmap e100: 12 total, 12 up, 12 in 2024-09-11T22:33:11.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:10 smithi005 ceph-mon[30943]: pgmap v347: 97 pgs: 97 active+clean; 88 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 256 KiB/s wr, 56 op/s 2024-09-11T22:33:11.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:10 smithi159 ceph-mon[35534]: pgmap v347: 97 pgs: 97 active+clean; 88 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 256 KiB/s wr, 56 op/s 2024-09-11T22:33:11.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 798682 7.05 MB/sec execute 449 sec latency 20.450 ms 2024-09-11T22:33:11.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:10 smithi172 ceph-mon[35461]: pgmap v347: 97 pgs: 97 active+clean; 88 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 256 KiB/s wr, 56 op/s 2024-09-11T22:33:12.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 799652 7.04 MB/sec execute 450 sec latency 30.216 ms 2024-09-11T22:33:13.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 802058 7.06 MB/sec execute 451 sec latency 15.728 ms 2024-09-11T22:33:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:13 smithi005 ceph-mon[30943]: pgmap v348: 97 pgs: 97 active+clean; 90 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-11T22:33:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:13 smithi005 ceph-mon[30943]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T22:33:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:13 smithi159 ceph-mon[35534]: pgmap v348: 97 pgs: 97 active+clean; 90 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-11T22:33:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:13 smithi159 ceph-mon[35534]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T22:33:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:13 smithi172 ceph-mon[35461]: pgmap v348: 97 pgs: 97 active+clean; 90 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-11T22:33:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:13 smithi172 ceph-mon[35461]: osdmap e101: 12 total, 12 up, 12 in 2024-09-11T22:33:14.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 803144 7.05 MB/sec execute 452 sec latency 70.102 ms 2024-09-11T22:33:15.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 805584 7.07 MB/sec execute 453 sec latency 15.440 ms 2024-09-11T22:33:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:15 smithi005 ceph-mon[30943]: pgmap v350: 97 pgs: 97 active+clean; 90 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-11T22:33:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:15 smithi159 ceph-mon[35534]: pgmap v350: 97 pgs: 97 active+clean; 90 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-11T22:33:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:15 smithi172 ceph-mon[35461]: pgmap v350: 97 pgs: 97 active+clean; 90 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-09-11T22:33:16.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 806687 7.07 MB/sec execute 454 sec latency 22.704 ms 2024-09-11T22:33:17.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 808966 7.08 MB/sec execute 455 sec latency 14.757 ms 2024-09-11T22:33:17.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:17 smithi005 ceph-mon[30943]: pgmap v351: 97 pgs: 97 active+clean; 90 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-09-11T22:33:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:17 smithi159 ceph-mon[35534]: pgmap v351: 97 pgs: 97 active+clean; 90 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-09-11T22:33:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:17 smithi172 ceph-mon[35461]: pgmap v351: 97 pgs: 97 active+clean; 90 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-09-11T22:33:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.927+0000 7feb4179f640 1 -- 172.21.15.5:0/4031209286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c100240 msgr2=0x7feb3c100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.927+0000 7feb4179f640 1 --2- 172.21.15.5:0/4031209286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c100240 0x7feb3c100620 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009900 tx=0x7feb2c02a410 comp rx=0 tx=0).stop 2024-09-11T22:33:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.927+0000 7feb4179f640 1 -- 172.21.15.5:0/4031209286 shutdown_connections 2024-09-11T22:33:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.927+0000 7feb4179f640 1 --2- 172.21.15.5:0/4031209286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c106e80 0x7feb3c10e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.927+0000 7feb4179f640 1 --2- 172.21.15.5:0/4031209286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3c100bf0 0x7feb3c106720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.927+0000 7feb4179f640 1 --2- 172.21.15.5:0/4031209286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c100240 0x7feb3c100620 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.927+0000 7feb4179f640 1 -- 172.21.15.5:0/4031209286 >> 172.21.15.5:0/4031209286 conn(0x7feb3c077160 msgr2=0x7feb3c0fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.928+0000 7feb4179f640 1 -- 172.21.15.5:0/4031209286 shutdown_connections 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.928+0000 7feb4179f640 1 -- 172.21.15.5:0/4031209286 wait complete. 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.928+0000 7feb4179f640 1 Processor -- start 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.928+0000 7feb4179f640 1 -- start start 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb4179f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c100240 0x7feb3c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb4179f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3c106e80 0x7feb3c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb4179f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c19cc20 0x7feb3c19d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb4179f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c111ee0 con 0x7feb3c19cc20 2024-09-11T22:33:17.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb4179f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c111d60 con 0x7feb3c106e80 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb4179f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c112060 con 0x7feb3c100240 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c19cc20 0x7feb3c19d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c100240 0x7feb3c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb327fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3c106e80 0x7feb3c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c19cc20 0x7feb3c19d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50064/0 (socket says 172.21.15.5:50064) 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb327fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3c106e80 0x7feb3c1988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37742/0 (socket says 172.21.15.5:37742) 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c100240 0x7feb3c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46198/0 (socket says 172.21.15.5:46198) 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3b7fe640 1 -- 172.21.15.5:0/2350211679 learned_addr learned my addr 172.21.15.5:0/2350211679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3b7fe640 1 -- 172.21.15.5:0/2350211679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c100240 msgr2=0x7feb3c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3b7fe640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c100240 0x7feb3c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3b7fe640 1 -- 172.21.15.5:0/2350211679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3c106e80 msgr2=0x7feb3c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:17.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3b7fe640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3c106e80 0x7feb3c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3b7fe640 1 -- 172.21.15.5:0/2350211679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c19d710 con 0x7feb3c19cc20 2024-09-11T22:33:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb3b7fe640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c19cc20 0x7feb3c19d0d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7feb3400eb10 tx=0x7feb3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb38ff9640 1 -- 172.21.15.5:0/2350211679 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34004b20 con 0x7feb3c19cc20 2024-09-11T22:33:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb38ff9640 1 -- 172.21.15.5:0/2350211679 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb34004cc0 con 0x7feb3c19cc20 2024-09-11T22:33:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb4179f640 1 -- 172.21.15.5:0/2350211679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3c19da00 con 0x7feb3c19cc20 2024-09-11T22:33:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb38ff9640 1 -- 172.21.15.5:0/2350211679 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34010760 con 0x7feb3c19cc20 2024-09-11T22:33:17.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.929+0000 7feb4179f640 1 -- 172.21.15.5:0/2350211679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3c1a51f0 con 0x7feb3c19cc20 2024-09-11T22:33:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.932+0000 7feb38ff9640 1 -- 172.21.15.5:0/2350211679 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7feb34005020 con 0x7feb3c19cc20 2024-09-11T22:33:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.932+0000 7feb4461b740 1 Processor -- start 2024-09-11T22:33:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.932+0000 7feb4461b740 1 -- start start 2024-09-11T22:33:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.932+0000 7feb4461b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e0af0 0x55d7d45e0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.932+0000 7feb4461b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e1410 0x55d7d45eb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.932+0000 7feb4461b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d45ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.932+0000 7feb4461b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d7d45452f0 con 0x55d7d45ebcb0 2024-09-11T22:33:17.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.932+0000 7feb4461b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d7d44ed270 con 0x55d7d45e0af0 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.932+0000 7feb4461b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d7d45412b0 con 0x55d7d45e1410 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.932+0000 7feb327fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d45ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb327fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d45ee0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50074/0 (socket says 172.21.15.5:50074) 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb327fc640 1 -- 172.21.15.5:0/1484257432 learned_addr learned my addr 172.21.15.5:0/1484257432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb327fc640 1 --2- 172.21.15.5:0/1484257432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e0af0 0x55d7d45e0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb38ff9640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7feb280a2da0 0x7feb280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb3affd640 1 --2- 172.21.15.5:0/1484257432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e1410 0x55d7d45eb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb3b7fe640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7feb280a2da0 0x7feb280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb327fc640 1 -- 172.21.15.5:0/1484257432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e1410 msgr2=0x55d7d45eb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb327fc640 1 --2- 172.21.15.5:0/1484257432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e1410 0x55d7d45eb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb327fc640 1 -- 172.21.15.5:0/1484257432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e0af0 msgr2=0x55d7d45e0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:17.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb327fc640 1 --2- 172.21.15.5:0/1484257432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e0af0 0x55d7d45e0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb327fc640 1 -- 172.21.15.5:0/1484257432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7d44ec340 con 0x55d7d45ebcb0 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb38ff9640 1 -- 172.21.15.5:0/2350211679 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7feb340cfb20 con 0x7feb3c19cc20 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb3affd640 1 --2- 172.21.15.5:0/1484257432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e1410 0x55d7d45eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb327fc640 1 --2- 172.21.15.5:0/1484257432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d45ee0a0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7feb240058c0 tx=0x7feb24005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4fbacc11a4bd69 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb317fa640 1 -- 172.21.15.5:0/1484257432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24004030 con 0x55d7d45ebcb0 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb317fa640 1 -- 172.21.15.5:0/1484257432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb2400fc70 con 0x55d7d45ebcb0 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb317fa640 1 -- 172.21.15.5:0/1484257432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24003ab0 con 0x55d7d45ebcb0 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.933+0000 7feb3b7fe640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7feb280a2da0 0x7feb280a5260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb3409bcd0 tx=0x7feb340bc040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.934+0000 7feb4461b740 1 -- 172.21.15.5:0/1484257432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 msgr2=0x55d7d45ee0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.934+0000 7feb4461b740 1 --2- 172.21.15.5:0/1484257432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d45ee0a0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7feb240058c0 tx=0x7feb24005850 comp rx=0 tx=0).stop 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.934+0000 7feb4461b740 1 -- 172.21.15.5:0/1484257432 shutdown_connections 2024-09-11T22:33:17.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.934+0000 7feb4461b740 1 --2- 172.21.15.5:0/1484257432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d45ee0a0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.934+0000 7feb4461b740 1 --2- 172.21.15.5:0/1484257432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e1410 0x55d7d45eb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.934+0000 7feb4461b740 1 --2- 172.21.15.5:0/1484257432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e0af0 0x55d7d45e0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.934+0000 7feb4461b740 1 -- 172.21.15.5:0/1484257432 >> 172.21.15.5:0/1484257432 conn(0x55d7d45dd060 msgr2=0x55d7d45dd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.934+0000 7feb4461b740 1 -- 172.21.15.5:0/1484257432 shutdown_connections 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.934+0000 7feb4461b740 1 -- 172.21.15.5:0/1484257432 wait complete. 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.934+0000 7feb4461b740 1 -- start start 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb4461b740 1 Processor -- start 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb4461b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e0af0 0x55d7d45e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb4461b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e1410 0x55d7d45e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb4461b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45ebcb0 0x55d7d45e9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb4461b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d7d45ee880 con 0x55d7d45e1410 2024-09-11T22:33:17.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb4461b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d7d44ce290 con 0x55d7d45ebcb0 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb4461b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d7d45eea00 con 0x55d7d45e0af0 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb4461b740 10 client.0 _ll_register_callbacks cb 0x55d7d44ff140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb3affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e1410 0x55d7d45e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb327fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e0af0 0x55d7d45e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb3affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e1410 0x55d7d45e90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50076/0 (socket says 172.21.15.5:50076) 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb3affd640 1 -- 172.21.15.5:0/3324858985 learned_addr learned my addr 172.21.15.5:0/3324858985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb327fc640 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45ebcb0 0x55d7d45e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb3affd640 1 -- 172.21.15.5:0/3324858985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e0af0 msgr2=0x55d7d45e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb3affd640 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e0af0 0x55d7d45e8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb3affd640 1 -- 172.21.15.5:0/3324858985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45ebcb0 msgr2=0x55d7d45e9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb3affd640 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45ebcb0 0x55d7d45e9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb3affd640 1 -- 172.21.15.5:0/3324858985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7d4258210 con 0x55d7d45e1410 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb3affd640 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e1410 0x55d7d45e90e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7feb2c02a930 tx=0x7feb2c031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e099210b4a8cd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.935+0000 7feb227fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb227fc640 1 -- 172.21.15.5:0/3324858985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c035040 con 0x55d7d45e1410 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb227fc640 1 -- 172.21.15.5:0/3324858985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7feb2c004040 con 0x55d7d45e1410 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb4461b740 1 -- 172.21.15.5:0/3324858985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2400fc70 con 0x55d7d45e1410 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb227fc640 1 -- 172.21.15.5:0/3324858985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7feb2c004240 con 0x55d7d45e1410 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb4461b740 10 client.18903 fetch_fsmap learned FSMap version 44 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb4461b740 1 -- 172.21.15.5:0/3324858985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d7d4324620 con 0x55d7d45e1410 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb227fc640 1 -- 172.21.15.5:0/3324858985 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb2400fc70 con 0x55d7d45e1410 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb227fc640 1 -- 172.21.15.5:0/3324858985 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7feb2c004b10 con 0x55d7d45e1410 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb4461b740 10 client.18903 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:33:17.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb4461b740 10 client.18903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb4461b740 20 client.18903 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb4461b740 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d7d45fbc30 0x55d7d45fe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb4461b740 4 client.18903 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb4461b740 1 -- 172.21.15.5:0/3324858985 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7d4552a20 con 0x55d7d45fbc30 2024-09-11T22:33:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.936+0000 7feb327fc640 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d7d45fbc30 0x55d7d45fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.937+0000 7feb227fc640 10 client.18903 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.937+0000 7feb327fc640 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d7d45fbc30 0x55d7d45fe020 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9043c7fc62f0641 server_cookie=30815ec5b8e52aa4 in_seq=0 out_seq=0 2024-09-11T22:33:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb227fc640 1 -- 172.21.15.5:0/3324858985 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7d4552a20 con 0x55d7d45fbc30 2024-09-11T22:33:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb227fc640 10 client.18903 handle_command_reply: tid=0 2024-09-11T22:33:17.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 client.18903 shutdown 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 -- 172.21.15.5:0/3324858985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e1410 msgr2=0x55d7d45e90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e1410 0x55d7d45e90e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7feb2c02a930 tx=0x7feb2c031860 comp rx=0 tx=0).stop 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 -- 172.21.15.5:0/3324858985 shutdown_connections 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d7d45fbc30 0x55d7d45fe020 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45ebcb0 0x55d7d45e9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e1410 0x55d7d45e90e0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 --2- 172.21.15.5:0/3324858985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e0af0 0x55d7d45e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 -- 172.21.15.5:0/3324858985 >> 172.21.15.5:0/3324858985 conn(0x55d7d45dd060 msgr2=0x55d7d45e2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb227fc640 0 client.18903 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 -- 172.21.15.5:0/3324858985 shutdown_connections 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 1 -- 172.21.15.5:0/3324858985 wait complete. 2024-09-11T22:33:17.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.939+0000 7feb4461b740 20 client.18903 trim_cache size 0 max 16384 2024-09-11T22:33:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.953+0000 7feb4461b740 1 Processor -- start 2024-09-11T22:33:17.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.953+0000 7feb4461b740 1 -- start start 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb4461b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e0af0 0x55d7d45eab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb4461b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 0x55d7d45e9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb4461b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45ebcb0 0x55d7d46038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb4461b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1adae0 con 0x55d7d45e0af0 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb4461b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1af620 con 0x55d7d45e1410 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb4461b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1ae1b0 con 0x55d7d45ebcb0 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb327fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e0af0 0x55d7d45eab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb327fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e0af0 0x55d7d45eab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50080/0 (socket says 172.21.15.5:50080) 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb327fc640 1 -- 172.21.15.5:0/3321539342 learned_addr learned my addr 172.21.15.5:0/3321539342 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb3affd640 1 --2- 172.21.15.5:0/3321539342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 0x55d7d45e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb327fc640 1 -- 172.21.15.5:0/3321539342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45ebcb0 msgr2=0x55d7d46038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:17.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb327fc640 1 --2- 172.21.15.5:0/3321539342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45ebcb0 0x55d7d46038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb327fc640 1 -- 172.21.15.5:0/3321539342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 msgr2=0x55d7d45e9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb327fc640 1 --2- 172.21.15.5:0/3321539342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 0x55d7d45e9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb327fc640 1 -- 172.21.15.5:0/3321539342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7d45dce90 con 0x55d7d45e0af0 2024-09-11T22:33:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.954+0000 7feb327fc640 1 --2- 172.21.15.5:0/3321539342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e0af0 0x55d7d45eab10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x55d7d45e3690 tx=0x7feb24005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf381e8b9e0975c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb30ff9640 1 -- 172.21.15.5:0/3321539342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24005070 con 0x55d7d45e0af0 2024-09-11T22:33:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb30ff9640 1 -- 172.21.15.5:0/3321539342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb2400fc70 con 0x55d7d45e0af0 2024-09-11T22:33:17.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb30ff9640 1 -- 172.21.15.5:0/3321539342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24004900 con 0x55d7d45e0af0 2024-09-11T22:33:17.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb4461b740 1 -- 172.21.15.5:0/3321539342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e0af0 msgr2=0x55d7d45eab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb4461b740 1 --2- 172.21.15.5:0/3321539342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e0af0 0x55d7d45eab10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x55d7d45e3690 tx=0x7feb24005850 comp rx=0 tx=0).stop 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb4461b740 1 -- 172.21.15.5:0/3321539342 shutdown_connections 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb4461b740 1 --2- 172.21.15.5:0/3321539342 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45ebcb0 0x55d7d46038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb4461b740 1 --2- 172.21.15.5:0/3321539342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 0x55d7d45e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb4461b740 1 --2- 172.21.15.5:0/3321539342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45e0af0 0x55d7d45eab10 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb4461b740 1 -- 172.21.15.5:0/3321539342 >> 172.21.15.5:0/3321539342 conn(0x55d7d45dd060 msgr2=0x55d7d45ea400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb4461b740 1 -- 172.21.15.5:0/3321539342 shutdown_connections 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.955+0000 7feb4461b740 1 -- 172.21.15.5:0/3321539342 wait complete. 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb4461b740 1 -- start start 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb4461b740 1 Processor -- start 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb4461b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e0af0 0x55d7d45ef400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb4461b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 0x55d7d45ef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb4461b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d465e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb4461b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1aef50 con 0x55d7d45ebcb0 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb4461b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1ae880 con 0x55d7d45e1410 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb4461b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1abae0 con 0x55d7d45e0af0 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb4461b740 10 client.0 _ll_register_callbacks cb 0x7feb340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb327fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d465e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb327fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d465e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50086/0 (socket says 172.21.15.5:50086) 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb327fc640 1 -- 172.21.15.5:0/1739998745 learned_addr learned my addr 172.21.15.5:0/1739998745 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb3affd640 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 0x55d7d45ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb327fc640 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e0af0 0x55d7d45ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb3affd640 1 -- 172.21.15.5:0/1739998745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e0af0 msgr2=0x55d7d45ef400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.956+0000 7feb3affd640 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e0af0 0x55d7d45ef400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb3affd640 1 -- 172.21.15.5:0/1739998745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 msgr2=0x55d7d465e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:17.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb3affd640 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d465e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb3affd640 1 -- 172.21.15.5:0/1739998745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7d465eec0 con 0x55d7d45e1410 2024-09-11T22:33:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb327fc640 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d465e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb3affd640 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 0x55d7d45ef940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb2c031550 tx=0x7feb2c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc836c403e6727f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb227fc640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:33:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb227fc640 1 -- 172.21.15.5:0/1739998745 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c05a060 con 0x55d7d45e1410 2024-09-11T22:33:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb4461b740 1 -- 172.21.15.5:0/1739998745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2400fc70 con 0x55d7d45e1410 2024-09-11T22:33:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb227fc640 1 -- 172.21.15.5:0/1739998745 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7feb2c0318d0 con 0x55d7d45e1410 2024-09-11T22:33:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb227fc640 1 -- 172.21.15.5:0/1739998745 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7feb2c035070 con 0x55d7d45e1410 2024-09-11T22:33:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb4461b740 10 client.28118 fetch_fsmap learned FSMap version 44 2024-09-11T22:33:17.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb4461b740 1 -- 172.21.15.5:0/1739998745 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55d7d4666260 con 0x55d7d45e1410 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb227fc640 1 -- 172.21.15.5:0/1739998745 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb2400fc70 con 0x55d7d45e1410 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.957+0000 7feb227fc640 1 -- 172.21.15.5:0/1739998745 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7feb2c054030 con 0x55d7d45e1410 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.958+0000 7feb4461b740 10 client.28118 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.958+0000 7feb4461b740 10 client.28118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.958+0000 7feb4461b740 20 client.28118 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.958+0000 7feb4461b740 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d7d46669c0 0x55d7d4668de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.958+0000 7feb4461b740 4 client.28118 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.958+0000 7feb4461b740 1 -- 172.21.15.5:0/1739998745 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7d454cd70 con 0x55d7d46669c0 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.958+0000 7feb327fc640 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d7d46669c0 0x55d7d4668de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.958+0000 7feb227fc640 10 client.28118 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:17.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:17.958+0000 7feb327fc640 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d7d46669c0 0x55d7d4668de0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0121c66ca02b82 server_cookie=9de2d5c836f9d2f4 in_seq=0 out_seq=0 2024-09-11T22:33:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb227fc640 1 -- 172.21.15.5:0/1739998745 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7d454cd70 con 0x55d7d46669c0 2024-09-11T22:33:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb227fc640 10 client.28118 handle_command_reply: tid=0 2024-09-11T22:33:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 client.28118 shutdown 2024-09-11T22:33:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 -- 172.21.15.5:0/1739998745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 msgr2=0x55d7d45ef940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 0x55d7d45ef940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feb2c031550 tx=0x7feb2c005a20 comp rx=0 tx=0).stop 2024-09-11T22:33:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 -- 172.21.15.5:0/1739998745 shutdown_connections 2024-09-11T22:33:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d7d46669c0 0x55d7d4668de0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7d45ebcb0 0x55d7d465e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7d45e1410 0x55d7d45ef940 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb227fc640 0 client.28118 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 --2- 172.21.15.5:0/1739998745 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d7d45e0af0 0x55d7d45ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 -- 172.21.15.5:0/1739998745 >> 172.21.15.5:0/1739998745 conn(0x55d7d45dd060 msgr2=0x55d7d45dc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 -- 172.21.15.5:0/1739998745 shutdown_connections 2024-09-11T22:33:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 1 -- 172.21.15.5:0/1739998745 wait complete. 2024-09-11T22:33:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.072+0000 7feb4461b740 20 client.28118 trim_cache size 0 max 16384 2024-09-11T22:33:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:33:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:33:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:33:18.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:33:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 -- 172.21.15.5:0/2350211679 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7feb280a2da0 msgr2=0x7feb280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7feb280a2da0 0x7feb280a5260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb3409bcd0 tx=0x7feb340bc040 comp rx=0 tx=0).stop 2024-09-11T22:33:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 -- 172.21.15.5:0/2350211679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c19cc20 msgr2=0x7feb3c19d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c19cc20 0x7feb3c19d0d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7feb3400eb10 tx=0x7feb3400c2d0 comp rx=0 tx=0).stop 2024-09-11T22:33:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 -- 172.21.15.5:0/2350211679 shutdown_connections 2024-09-11T22:33:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c19cc20 0x7feb3c19d0d0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:18.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7feb280a2da0 0x7feb280a5260 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3c106e80 0x7feb3c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 --2- 172.21.15.5:0/2350211679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c100240 0x7feb3c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 -- 172.21.15.5:0/2350211679 >> 172.21.15.5:0/2350211679 conn(0x7feb3c077160 msgr2=0x7feb3c105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 -- 172.21.15.5:0/2350211679 shutdown_connections 2024-09-11T22:33:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:18.073+0000 7feb4179f640 1 -- 172.21.15.5:0/2350211679 wait complete. 2024-09-11T22:33:18.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 810101 7.08 MB/sec execute 456 sec latency 68.270 ms 2024-09-11T22:33:19.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 812422 7.09 MB/sec execute 457 sec latency 17.343 ms 2024-09-11T22:33:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:19 smithi005 ceph-mon[30943]: pgmap v352: 97 pgs: 97 active+clean; 98 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-11T22:33:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:19 smithi159 ceph-mon[35534]: pgmap v352: 97 pgs: 97 active+clean; 98 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-11T22:33:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:19 smithi172 ceph-mon[35461]: pgmap v352: 97 pgs: 97 active+clean; 98 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-11T22:33:20.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 813705 7.09 MB/sec execute 458 sec latency 14.813 ms 2024-09-11T22:33:21.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 816026 7.10 MB/sec execute 459 sec latency 21.634 ms 2024-09-11T22:33:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:21 smithi005 ceph-mon[30943]: pgmap v353: 97 pgs: 97 active+clean; 98 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 179 op/s 2024-09-11T22:33:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:21 smithi159 ceph-mon[35534]: pgmap v353: 97 pgs: 97 active+clean; 98 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 179 op/s 2024-09-11T22:33:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:21 smithi172 ceph-mon[35461]: pgmap v353: 97 pgs: 97 active+clean; 98 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 179 op/s 2024-09-11T22:33:22.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 817311 7.10 MB/sec execute 460 sec latency 16.436 ms 2024-09-11T22:33:23.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 819091 7.10 MB/sec execute 461 sec latency 175.398 ms 2024-09-11T22:33:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:23 smithi005 ceph-mon[30943]: pgmap v354: 97 pgs: 97 active+clean; 93 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 259 op/s 2024-09-11T22:33:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:23 smithi159 ceph-mon[35534]: pgmap v354: 97 pgs: 97 active+clean; 93 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 259 op/s 2024-09-11T22:33:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:23 smithi172 ceph-mon[35461]: pgmap v354: 97 pgs: 97 active+clean; 93 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 259 op/s 2024-09-11T22:33:24.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 820282 7.11 MB/sec execute 462 sec latency 110.971 ms 2024-09-11T22:33:25.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 821729 7.10 MB/sec execute 463 sec latency 79.935 ms 2024-09-11T22:33:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:25 smithi005 ceph-mon[30943]: pgmap v355: 97 pgs: 97 active+clean; 93 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-09-11T22:33:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:25 smithi159 ceph-mon[35534]: pgmap v355: 97 pgs: 97 active+clean; 93 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-09-11T22:33:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:25 smithi172 ceph-mon[35461]: pgmap v355: 97 pgs: 97 active+clean; 93 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-09-11T22:33:26.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 823752 7.12 MB/sec execute 464 sec latency 54.376 ms 2024-09-11T22:33:27.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 825137 7.11 MB/sec execute 465 sec latency 16.662 ms 2024-09-11T22:33:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:27 smithi005 ceph-mon[30943]: pgmap v356: 97 pgs: 97 active+clean; 93 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-09-11T22:33:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:27 smithi159 ceph-mon[35534]: pgmap v356: 97 pgs: 97 active+clean; 93 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-09-11T22:33:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:27 smithi172 ceph-mon[35461]: pgmap v356: 97 pgs: 97 active+clean; 93 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-09-11T22:33:28.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 827290 7.13 MB/sec execute 466 sec latency 24.536 ms 2024-09-11T22:33:29.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:33:29.342 INFO:tasks.workunit.client.0.smithi005.stdout: 1 828683 7.12 MB/sec execute 467 sec latency 14.981 ms 2024-09-11T22:33:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:29.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:29 smithi005 ceph-mon[30943]: pgmap v357: 97 pgs: 97 active+clean; 100 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-09-11T22:33:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:29 smithi159 ceph-mon[35534]: pgmap v357: 97 pgs: 97 active+clean; 100 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-09-11T22:33:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:29 smithi172 ceph-mon[35461]: pgmap v357: 97 pgs: 97 active+clean; 100 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-09-11T22:33:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.084+0000 7f31106bf640 1 -- 172.21.15.5:0/300086891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3108100e20 msgr2=0x7f3108101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.084+0000 7f31106bf640 1 --2- 172.21.15.5:0/300086891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3108100e20 0x7f3108101200 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3104009890 tx=0x7f310402a410 comp rx=0 tx=0).stop 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.084+0000 7f31106bf640 1 -- 172.21.15.5:0/300086891 shutdown_connections 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.084+0000 7f31106bf640 1 --2- 172.21.15.5:0/300086891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108102ef0 0x7f310810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.084+0000 7f31106bf640 1 --2- 172.21.15.5:0/300086891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3108102570 0x7f31081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.084+0000 7f31106bf640 1 --2- 172.21.15.5:0/300086891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3108100e20 0x7f3108101200 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.084+0000 7f31106bf640 1 -- 172.21.15.5:0/300086891 >> 172.21.15.5:0/300086891 conn(0x7f31080fc7d0 msgr2=0x7f31080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.084+0000 7f31106bf640 1 -- 172.21.15.5:0/300086891 shutdown_connections 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.084+0000 7f31106bf640 1 -- 172.21.15.5:0/300086891 wait complete. 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.084+0000 7f31106bf640 1 Processor -- start 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f31106bf640 1 -- start start 2024-09-11T22:33:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f31106bf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3108100e20 0x7f3108198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f31106bf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3108102570 0x7f3108198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f31106bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108102ef0 0x7f310819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f31106bf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3108110f80 con 0x7f3108102ef0 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f31106bf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3108110e00 con 0x7f3108102570 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f31106bf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3108111100 con 0x7f3108100e20 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310ec5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108102ef0 0x7f310819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310dc5a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3108102570 0x7f3108198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310e45b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3108100e20 0x7f3108198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310ec5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108102ef0 0x7f310819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48510/0 (socket says 172.21.15.5:48510) 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310ec5c640 1 -- 172.21.15.5:0/3492841330 learned_addr learned my addr 172.21.15.5:0/3492841330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310dc5a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3108102570 0x7f3108198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48420/0 (socket says 172.21.15.5:48420) 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310e45b640 1 -- 172.21.15.5:0/3492841330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3108102570 msgr2=0x7f3108198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310e45b640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3108102570 0x7f3108198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310e45b640 1 -- 172.21.15.5:0/3492841330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108102ef0 msgr2=0x7f310819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310e45b640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108102ef0 0x7f310819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310e45b640 1 -- 172.21.15.5:0/3492841330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310819d4f0 con 0x7f3108100e20 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.085+0000 7f310ec5c640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108102ef0 0x7f310819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.086+0000 7f310e45b640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3108100e20 0x7f3108198540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3104034000 tx=0x7f31040305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.086+0000 7f30ff7fe640 1 -- 172.21.15.5:0/3492841330 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104005460 con 0x7f3108100e20 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.086+0000 7f31106bf640 1 -- 172.21.15.5:0/3492841330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310819d780 con 0x7f3108100e20 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.086+0000 7f30ff7fe640 1 -- 172.21.15.5:0/3492841330 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31040308b0 con 0x7f3108100e20 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.086+0000 7f30ff7fe640 1 -- 172.21.15.5:0/3492841330 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104004c30 con 0x7f3108100e20 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.086+0000 7f31106bf640 1 -- 172.21.15.5:0/3492841330 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31080fce30 con 0x7f3108100e20 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.088+0000 7f30ff7fe640 1 -- 172.21.15.5:0/3492841330 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3104039040 con 0x7f3108100e20 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.088+0000 7f3112a98740 1 Processor -- start 2024-09-11T22:33:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f3112a98740 1 -- start start 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f3112a98740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c24a0 0x5604344c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f3112a98740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 0x5604344cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f3112a98740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344cd680 0x5604344cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f3112a98740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5604344282e0 con 0x5604344c24a0 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f3112a98740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5604343cfce0 con 0x5604344c2dc0 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f3112a98740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5604344242a0 con 0x5604344cd680 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310ec5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c24a0 0x5604344c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310e45b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 0x5604344cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310ec5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c24a0 0x5604344c2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48522/0 (socket says 172.21.15.5:48522) 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310ec5c640 1 -- 172.21.15.5:0/3738857085 learned_addr learned my addr 172.21.15.5:0/3738857085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310dc5a640 1 --2- 172.21.15.5:0/3738857085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344cd680 0x5604344cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f30ff7fe640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30f40a2cf0 0x7f30f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310ec5c640 1 -- 172.21.15.5:0/3738857085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344cd680 msgr2=0x5604344cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310ec5c640 1 --2- 172.21.15.5:0/3738857085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344cd680 0x5604344cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310ec5c640 1 -- 172.21.15.5:0/3738857085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 msgr2=0x5604344cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310ec5c640 1 --2- 172.21.15.5:0/3738857085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 0x5604344cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310ec5c640 1 -- 172.21.15.5:0/3738857085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604343b4440 con 0x5604344c24a0 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.089+0000 7f310e45b640 1 --2- 172.21.15.5:0/3738857085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 0x5604344cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.090+0000 7f30ff7fe640 1 -- 172.21.15.5:0/3492841330 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7f31040f01c0 con 0x7f3108100e20 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.090+0000 7f310ec5c640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30f40a2cf0 0x7f30f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.090+0000 7f310ec5c640 1 --2- 172.21.15.5:0/3738857085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c24a0 0x5604344c2880 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f310000e8d0 tx=0x7f310000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9992b7d878a9658 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.090+0000 7f30fcff9640 1 -- 172.21.15.5:0/3738857085 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31000185f0 con 0x5604344c24a0 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.090+0000 7f30fcff9640 1 -- 172.21.15.5:0/3738857085 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f310000ce40 con 0x5604344c24a0 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.090+0000 7f30fcff9640 1 -- 172.21.15.5:0/3738857085 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31000249c0 con 0x5604344c24a0 2024-09-11T22:33:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 -- 172.21.15.5:0/3738857085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c24a0 msgr2=0x5604344c2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f310ec5c640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30f40a2cf0 0x7f30f40a51b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3100021000 tx=0x7f3100022000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 --2- 172.21.15.5:0/3738857085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c24a0 0x5604344c2880 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f310000e8d0 tx=0x7f310000ed90 comp rx=0 tx=0).stop 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 -- 172.21.15.5:0/3738857085 shutdown_connections 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 --2- 172.21.15.5:0/3738857085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344cd680 0x5604344cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 --2- 172.21.15.5:0/3738857085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 0x5604344cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 --2- 172.21.15.5:0/3738857085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c24a0 0x5604344c2880 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 -- 172.21.15.5:0/3738857085 >> 172.21.15.5:0/3738857085 conn(0x5604344bea30 msgr2=0x5604344bee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 -- 172.21.15.5:0/3738857085 shutdown_connections 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 -- 172.21.15.5:0/3738857085 wait complete. 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 -- start start 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.091+0000 7f3112a98740 1 Processor -- start 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f3112a98740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 0x5604344ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f3112a98740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 0x5604344ca220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f3112a98740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x5604344ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f3112a98740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5604344d0250 con 0x5604344cd680 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f3112a98740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5604343e3050 con 0x5604344c2dc0 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f3112a98740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5604344d03d0 con 0x5604344c24a0 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f3112a98740 10 client.0 _ll_register_callbacks cb 0x560434423160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310ec5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x5604344ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310ec5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x5604344ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48534/0 (socket says 172.21.15.5:48534) 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310ec5c640 1 -- 172.21.15.5:0/1758737145 learned_addr learned my addr 172.21.15.5:0/1758737145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310dc5a640 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 0x5604344ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310ec5c640 1 -- 172.21.15.5:0/1758737145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 msgr2=0x5604344ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310e45b640 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 0x5604344ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310ec5c640 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 0x5604344ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310ec5c640 1 -- 172.21.15.5:0/1758737145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 msgr2=0x5604344ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:30.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310ec5c640 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 0x5604344ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310ec5c640 1 -- 172.21.15.5:0/1758737145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604341993c0 con 0x5604344cd680 2024-09-11T22:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310e45b640 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 0x5604344ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310dc5a640 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 0x5604344ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f310ec5c640 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x5604344ca760 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f3100024720 tx=0x7f31000097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2eea452774baef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.092+0000 7f30f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1758737145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31000185f0 con 0x5604344cd680 2024-09-11T22:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1758737145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7f310000b570 con 0x5604344cd680 2024-09-11T22:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f3112a98740 1 -- 172.21.15.5:0/1758737145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560434435850 con 0x5604344cd680 2024-09-11T22:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1758737145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f310003b050 con 0x5604344cd680 2024-09-11T22:33:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f3112a98740 10 client.18921 fetch_fsmap learned FSMap version 44 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f3112a98740 1 -- 172.21.15.5:0/1758737145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5604344dbbc0 con 0x5604344cd680 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1758737145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560434435850 con 0x5604344cd680 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1758737145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f310000b040 con 0x5604344cd680 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f3112a98740 10 client.18921 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f3112a98740 10 client.18921 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f3112a98740 20 client.18921 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f3112a98740 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5604344dd020 0x5604344df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f3112a98740 4 client.18921 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f3112a98740 1 -- 172.21.15.5:0/1758737145 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56043442fd60 con 0x5604344dd020 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.093+0000 7f310e45b640 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5604344dd020 0x5604344df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.094+0000 7f30f1ffb640 10 client.18921 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.094+0000 7f310e45b640 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5604344dd020 0x5604344df410 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=758046bc1b48671d server_cookie=490cf9c5c191951e in_seq=0 out_seq=0 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.094+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1758737145 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56043442fd60 con 0x5604344dd020 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.094+0000 7f30f1ffb640 10 client.18921 handle_command_reply: tid=0 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.094+0000 7f3112a98740 1 client.18921 shutdown 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 1 -- 172.21.15.5:0/1758737145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 msgr2=0x5604344ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x5604344ca760 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f3100024720 tx=0x7f31000097e0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 1 -- 172.21.15.5:0/1758737145 shutdown_connections 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5604344dd020 0x5604344df410 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x5604344ca760 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f30f1ffb640 0 client.18921 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c2dc0 0x5604344ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 1 --2- 172.21.15.5:0/1758737145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 0x5604344ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 1 -- 172.21.15.5:0/1758737145 >> 172.21.15.5:0/1758737145 conn(0x5604344bea30 msgr2=0x5604344c4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 1 -- 172.21.15.5:0/1758737145 shutdown_connections 2024-09-11T22:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 1 -- 172.21.15.5:0/1758737145 wait complete. 2024-09-11T22:33:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.095+0000 7f3112a98740 20 client.18921 trim_cache size 0 max 16384 2024-09-11T22:33:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.112+0000 7f3112a98740 1 Processor -- start 2024-09-11T22:33:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.112+0000 7f3112a98740 1 -- start start 2024-09-11T22:33:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.112+0000 7f3112a98740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 0x5604344d94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.112+0000 7f3112a98740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c2dc0 0x5604344e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f3112a98740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344cd680 0x5604344e5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f3112a98740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31081afc40 con 0x5604344c2dc0 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f3112a98740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b1780 con 0x5604344cd680 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f3112a98740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b0310 con 0x5604344c24a0 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310dc5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c2dc0 0x5604344e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310e45b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 0x5604344d94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310dc5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c2dc0 0x5604344e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48548/0 (socket says 172.21.15.5:48548) 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310dc5a640 1 -- 172.21.15.5:0/1749899854 learned_addr learned my addr 172.21.15.5:0/1749899854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310ec5c640 1 --2- 172.21.15.5:0/1749899854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344cd680 0x5604344e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310e45b640 1 -- 172.21.15.5:0/1749899854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344cd680 msgr2=0x5604344e5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310e45b640 1 --2- 172.21.15.5:0/1749899854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344cd680 0x5604344e5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310e45b640 1 -- 172.21.15.5:0/1749899854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c2dc0 msgr2=0x5604344e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310e45b640 1 --2- 172.21.15.5:0/1749899854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c2dc0 0x5604344e4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310e45b640 1 -- 172.21.15.5:0/1749899854 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604344c3330 con 0x5604344c24a0 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310ec5c640 1 --2- 172.21.15.5:0/1749899854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344cd680 0x5604344e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f310e45b640 1 --2- 172.21.15.5:0/1749899854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 0x5604344d94b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f31040ed8e0 tx=0x7f31040a7040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20a6d82650d691c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f30f3fff640 1 -- 172.21.15.5:0/1749899854 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31040bc480 con 0x5604344c24a0 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f30f3fff640 1 -- 172.21.15.5:0/1749899854 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31040bcdb0 con 0x5604344c24a0 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.113+0000 7f30f3fff640 1 -- 172.21.15.5:0/1749899854 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31040a70b0 con 0x5604344c24a0 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.114+0000 7f3112a98740 1 -- 172.21.15.5:0/1749899854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 msgr2=0x5604344d94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.114+0000 7f3112a98740 1 --2- 172.21.15.5:0/1749899854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 0x5604344d94b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f31040ed8e0 tx=0x7f31040a7040 comp rx=0 tx=0).stop 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.114+0000 7f3112a98740 1 -- 172.21.15.5:0/1749899854 shutdown_connections 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.114+0000 7f3112a98740 1 --2- 172.21.15.5:0/1749899854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344cd680 0x5604344e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.114+0000 7f3112a98740 1 --2- 172.21.15.5:0/1749899854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344c2dc0 0x5604344e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.114+0000 7f3112a98740 1 --2- 172.21.15.5:0/1749899854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c24a0 0x5604344d94b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.114+0000 7f3112a98740 1 -- 172.21.15.5:0/1749899854 >> 172.21.15.5:0/1749899854 conn(0x5604344bea30 msgr2=0x5604344c9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.114+0000 7f3112a98740 1 -- 172.21.15.5:0/1749899854 shutdown_connections 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.114+0000 7f3112a98740 1 -- 172.21.15.5:0/1749899854 wait complete. 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f3112a98740 1 -- start start 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f3112a98740 1 Processor -- start 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f3112a98740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c24a0 0x5604344d29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f3112a98740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c2dc0 0x5604344d2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f3112a98740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x560434545ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f3112a98740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b10b0 con 0x5604344cd680 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f3112a98740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b09e0 con 0x5604344c24a0 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f3112a98740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f31081adc40 con 0x5604344c2dc0 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f3112a98740 10 client.0 _ll_register_callbacks cb 0x7f31040f3c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f310ec5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x560434545ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f310e45b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c24a0 0x5604344d29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f310e45b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c24a0 0x5604344d29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48462/0 (socket says 172.21.15.5:48462) 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f310e45b640 1 -- 172.21.15.5:0/1506404707 learned_addr learned my addr 172.21.15.5:0/1506404707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f310dc5a640 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c2dc0 0x5604344d2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f310ec5c640 1 -- 172.21.15.5:0/1506404707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c2dc0 msgr2=0x5604344d2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f310ec5c640 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c2dc0 0x5604344d2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f310ec5c640 1 -- 172.21.15.5:0/1506404707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c24a0 msgr2=0x5604344d29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f310ec5c640 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c24a0 0x5604344d29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.115+0000 7f310ec5c640 1 -- 172.21.15.5:0/1506404707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560434546010 con 0x5604344cd680 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.116+0000 7f310ec5c640 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x560434545ad0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f310002db40 tx=0x7f3100009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1f88f9243864d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.116+0000 7f30f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.116+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1506404707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31000185f0 con 0x5604344cd680 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.116+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1506404707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7f310002d3e0 con 0x5604344cd680 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.116+0000 7f3112a98740 1 -- 172.21.15.5:0/1506404707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31040bcdb0 con 0x5604344cd680 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.116+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1506404707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f310002d5e0 con 0x5604344cd680 2024-09-11T22:33:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.116+0000 7f3112a98740 10 client.18927 fetch_fsmap learned FSMap version 44 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.116+0000 7f3112a98740 1 -- 172.21.15.5:0/1506404707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x560434547230 con 0x5604344cd680 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.116+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1506404707 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31040bcdb0 con 0x5604344cd680 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1506404707 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f310000b040 con 0x5604344cd680 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f3112a98740 10 client.18927 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f3112a98740 10 client.18927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f3112a98740 20 client.18927 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f3112a98740 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5604345478e0 0x560434549d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f3112a98740 4 client.18927 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f3112a98740 1 -- 172.21.15.5:0/1506404707 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5604343ac210 con 0x5604345478e0 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f310e45b640 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5604345478e0 0x560434549d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f310e45b640 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5604345478e0 0x560434549d00 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b27a535cb0073280 server_cookie=38a0d2a22a46c7dc in_seq=0 out_seq=0 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f30f1ffb640 10 client.18927 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f30f1ffb640 1 -- 172.21.15.5:0/1506404707 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5604343ac210 con 0x5604345478e0 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f30f1ffb640 10 client.18927 handle_command_reply: tid=0 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.117+0000 7f3112a98740 1 client.18927 shutdown 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 1 -- 172.21.15.5:0/1506404707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 msgr2=0x560434545ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x560434545ad0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f310002db40 tx=0x7f3100009d30 comp rx=0 tx=0).stop 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 1 -- 172.21.15.5:0/1506404707 shutdown_connections 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5604345478e0 0x560434549d00 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604344cd680 0x560434545ad0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604344c2dc0 0x5604344d2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f30f1ffb640 0 client.18927 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 1 --2- 172.21.15.5:0/1506404707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604344c24a0 0x5604344d29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 1 -- 172.21.15.5:0/1506404707 >> 172.21.15.5:0/1506404707 conn(0x5604344bea30 msgr2=0x5604344cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 1 -- 172.21.15.5:0/1506404707 shutdown_connections 2024-09-11T22:33:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 1 -- 172.21.15.5:0/1506404707 wait complete. 2024-09-11T22:33:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f3112a98740 20 client.18927 trim_cache size 0 max 16384 2024-09-11T22:33:30.129 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:33:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:33:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:33:30.129 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f31106bf640 1 -- 172.21.15.5:0/3492841330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30f40a2cf0 msgr2=0x7f30f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f31106bf640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30f40a2cf0 0x7f30f40a51b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3100021000 tx=0x7f3100022000 comp rx=0 tx=0).stop 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f31106bf640 1 -- 172.21.15.5:0/3492841330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3108100e20 msgr2=0x7f3108198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f31106bf640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3108100e20 0x7f3108198540 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3104034000 tx=0x7f31040305d0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f31106bf640 1 -- 172.21.15.5:0/3492841330 shutdown_connections 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f31106bf640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3108102ef0 0x7f310819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f31106bf640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f30f40a2cf0 0x7f30f40a51b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f31106bf640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3108102570 0x7f3108198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f31106bf640 1 --2- 172.21.15.5:0/3492841330 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3108100e20 0x7f3108198540 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.118+0000 7f31106bf640 1 -- 172.21.15.5:0/3492841330 >> 172.21.15.5:0/3492841330 conn(0x7f31080fc7d0 msgr2=0x7f310806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.119+0000 7f31106bf640 1 -- 172.21.15.5:0/3492841330 shutdown_connections 2024-09-11T22:33:30.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:30.119+0000 7f31106bf640 1 -- 172.21.15.5:0/3492841330 wait complete. 2024-09-11T22:33:30.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 830578 7.13 MB/sec execute 468 sec latency 159.167 ms 2024-09-11T22:33:31.028 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:33:31.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:33:31.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.167+0000 7f3770053640 1 -- 172.21.15.5:0/1587136881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3768101350 msgr2=0x7f376810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.167+0000 7f3770053640 1 --2- 172.21.15.5:0/1587136881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3768101350 0x7f376810d880 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3760009930 tx=0x7f376002a420 comp rx=0 tx=0).stop 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.167+0000 7f3770053640 1 -- 172.21.15.5:0/1587136881 shutdown_connections 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.167+0000 7f3770053640 1 --2- 172.21.15.5:0/1587136881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3768101350 0x7f376810d880 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.167+0000 7f3770053640 1 --2- 172.21.15.5:0/1587136881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37681009b0 0x7f3768100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.167+0000 7f3770053640 1 --2- 172.21.15.5:0/1587136881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37681069c0 0x7f3768106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.167+0000 7f3770053640 1 -- 172.21.15.5:0/1587136881 >> 172.21.15.5:0/1587136881 conn(0x7f3768077990 msgr2=0x7f37680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.167+0000 7f3770053640 1 -- 172.21.15.5:0/1587136881 shutdown_connections 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.167+0000 7f3770053640 1 -- 172.21.15.5:0/1587136881 wait complete. 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f3770053640 1 Processor -- start 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f3770053640 1 -- start start 2024-09-11T22:33:31.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f3770053640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37681009b0 0x7f3768199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f3770053640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3768101350 0x7f37681996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f3770053640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37681069c0 0x7f37681a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f3770053640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3768110e00 con 0x7f37681069c0 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f3770053640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3768110c80 con 0x7f3768101350 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f3770053640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3768110f80 con 0x7f37681009b0 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f376e5f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37681069c0 0x7f37681a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f376d5ee640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3768101350 0x7f37681996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.168+0000 7f376ddef640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37681009b0 0x7f3768199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f376d5ee640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3768101350 0x7f37681996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48472/0 (socket says 172.21.15.5:48472) 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f376d5ee640 1 -- 172.21.15.5:0/1904893292 learned_addr learned my addr 172.21.15.5:0/1904893292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:31.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f376d5ee640 1 -- 172.21.15.5:0/1904893292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37681009b0 msgr2=0x7f3768199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f376d5ee640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37681009b0 0x7f3768199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f376d5ee640 1 -- 172.21.15.5:0/1904893292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37681069c0 msgr2=0x7f37681a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f376d5ee640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37681069c0 0x7f37681a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f376d5ee640 1 -- 172.21.15.5:0/1904893292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37681a4730 con 0x7f3768101350 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f376e5f0640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37681069c0 0x7f37681a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f376ddef640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37681009b0 0x7f3768199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f376d5ee640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3768101350 0x7f37681996b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f375c00eb90 tx=0x7f375c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f375affd640 1 -- 172.21.15.5:0/1904893292 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c004d20 con 0x7f3768101350 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f375affd640 1 -- 172.21.15.5:0/1904893292 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375c00ee90 con 0x7f3768101350 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f3770053640 1 -- 172.21.15.5:0/1904893292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37681a4900 con 0x7f3768101350 2024-09-11T22:33:31.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f375affd640 1 -- 172.21.15.5:0/1904893292 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c010830 con 0x7f3768101350 2024-09-11T22:33:31.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.169+0000 7f3770053640 1 -- 172.21.15.5:0/1904893292 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37681a4dd0 con 0x7f3768101350 2024-09-11T22:33:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f3772de8740 1 Processor -- start 2024-09-11T22:33:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f3772de8740 1 -- start start 2024-09-11T22:33:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f375affd640 1 -- 172.21.15.5:0/1904893292 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f375c005020 con 0x7f3768101350 2024-09-11T22:33:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f3772de8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 0x557b06ce4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f3772de8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 0x557b06cef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f3772de8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06cf20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f3772de8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557b06c492f0 con 0x557b06cefcb0 2024-09-11T22:33:31.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f3772de8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557b06bf1270 con 0x557b06ce5410 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f3772de8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557b06c452b0 con 0x557b06ce4af0 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f376d5ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06cf20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f376ddef640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 0x557b06cef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f376d5ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06cf20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48588/0 (socket says 172.21.15.5:48588) 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f376d5ee640 1 -- 172.21.15.5:0/505286829 learned_addr learned my addr 172.21.15.5:0/505286829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f376ddef640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 0x557b06cef770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48488/0 (socket says 172.21.15.5:48488) 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.172+0000 7f376e5f0640 1 --2- 172.21.15.5:0/505286829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 0x557b06ce4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f376d5ee640 1 -- 172.21.15.5:0/505286829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 msgr2=0x557b06ce4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f376d5ee640 1 --2- 172.21.15.5:0/505286829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 0x557b06ce4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f376d5ee640 1 -- 172.21.15.5:0/505286829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 msgr2=0x557b06cef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f376d5ee640 1 --2- 172.21.15.5:0/505286829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 0x557b06cef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f376d5ee640 1 -- 172.21.15.5:0/505286829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b06bf0340 con 0x557b06cefcb0 2024-09-11T22:33:31.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f376d5ee640 1 --2- 172.21.15.5:0/505286829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06cf20a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f375c0cd160 tx=0x7f375c0d3c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca156fb86261149c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f375affd640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f37540a2da0 0x7f37540a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f3753fff640 1 -- 172.21.15.5:0/505286829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0df020 con 0x557b06cefcb0 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f376e5f0640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f37540a2da0 0x7f37540a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f3753fff640 1 -- 172.21.15.5:0/505286829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375c09c6c0 con 0x557b06cefcb0 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f3753fff640 1 -- 172.21.15.5:0/505286829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0d4040 con 0x557b06cefcb0 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.173+0000 7f375affd640 1 -- 172.21.15.5:0/1904893292 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7f375c0cfb10 con 0x7f3768101350 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.174+0000 7f3772de8740 1 -- 172.21.15.5:0/505286829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 msgr2=0x557b06cf20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.174+0000 7f376e5f0640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f37540a2da0 0x7f37540a5260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3760006010 tx=0x7f3760009560 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.174+0000 7f3772de8740 1 --2- 172.21.15.5:0/505286829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06cf20a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f375c0cd160 tx=0x7f375c0d3c50 comp rx=0 tx=0).stop 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.174+0000 7f3772de8740 1 -- 172.21.15.5:0/505286829 shutdown_connections 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.174+0000 7f3772de8740 1 --2- 172.21.15.5:0/505286829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06cf20a0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.174+0000 7f3772de8740 1 --2- 172.21.15.5:0/505286829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 0x557b06cef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.174+0000 7f3772de8740 1 --2- 172.21.15.5:0/505286829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 0x557b06ce4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.174+0000 7f3772de8740 1 -- 172.21.15.5:0/505286829 >> 172.21.15.5:0/505286829 conn(0x557b06ce1060 msgr2=0x557b06ce1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.174+0000 7f3772de8740 1 -- 172.21.15.5:0/505286829 shutdown_connections 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.174+0000 7f3772de8740 1 -- 172.21.15.5:0/505286829 wait complete. 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f3772de8740 1 -- start start 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f3772de8740 1 Processor -- start 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f3772de8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f3772de8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 0x557b06ced050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f3772de8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ced590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f3772de8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557b06cf2880 con 0x557b06cefcb0 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f3772de8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557b06bd2290 con 0x557b06ce4af0 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f3772de8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557b06cf2a00 con 0x557b06ce5410 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f3772de8740 10 client.0 _ll_register_callbacks cb 0x557b06c03140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f376e5f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ced590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f376ddef640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f376e5f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ced590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48590/0 (socket says 172.21.15.5:48590) 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f376e5f0640 1 -- 172.21.15.5:0/3064467248 learned_addr learned my addr 172.21.15.5:0/3064467248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f376ddef640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cecb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48496/0 (socket says 172.21.15.5:48496) 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.175+0000 7f376d5ee640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 0x557b06ced050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f376d5ee640 1 -- 172.21.15.5:0/3064467248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 msgr2=0x557b06cecb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f376d5ee640 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cecb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f376d5ee640 1 -- 172.21.15.5:0/3064467248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 msgr2=0x557b06ced590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f376d5ee640 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ced590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f376d5ee640 1 -- 172.21.15.5:0/3064467248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b0695c210 con 0x557b06ce5410 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f376d5ee640 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 0x557b06ced050 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f375c0cd160 tx=0x7f375c009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46fa72222ccdb33b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f37517fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f37517fa640 1 -- 172.21.15.5:0/3064467248 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0df020 con 0x557b06ce5410 2024-09-11T22:33:31.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f3772de8740 1 -- 172.21.15.5:0/3064467248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f375c09c6c0 con 0x557b06ce5410 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f37517fa640 1 -- 172.21.15.5:0/3064467248 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7f375c09a2c0 con 0x557b06ce5410 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f37517fa640 1 -- 172.21.15.5:0/3064467248 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f375c07afb0 con 0x557b06ce5410 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f3772de8740 10 client.28039 fetch_fsmap learned FSMap version 44 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f3772de8740 1 -- 172.21.15.5:0/3064467248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557b06a28620 con 0x557b06ce5410 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.176+0000 7f37517fa640 1 -- 172.21.15.5:0/3064467248 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375c09c6c0 con 0x557b06ce5410 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.177+0000 7f37517fa640 1 -- 172.21.15.5:0/3064467248 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f375c09e070 con 0x557b06ce5410 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.177+0000 7f3772de8740 10 client.28039 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.177+0000 7f3772de8740 10 client.28039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.177+0000 7f3772de8740 20 client.28039 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.177+0000 7f3772de8740 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557b06cffa00 0x557b06d01df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.177+0000 7f3772de8740 4 client.28039 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:31.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.177+0000 7f3772de8740 1 -- 172.21.15.5:0/3064467248 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b06c56a20 con 0x557b06cffa00 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.177+0000 7f376ddef640 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557b06cffa00 0x557b06d01df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.177+0000 7f376ddef640 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557b06cffa00 0x557b06d01df0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61bdf445f4e9d917 server_cookie=5b82c6f4df9e4649 in_seq=0 out_seq=0 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.177+0000 7f37517fa640 10 client.28039 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f37517fa640 1 -- 172.21.15.5:0/3064467248 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b06c56a20 con 0x557b06cffa00 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f37517fa640 10 client.28039 handle_command_reply: tid=0 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 client.28039 shutdown 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 -- 172.21.15.5:0/3064467248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 msgr2=0x557b06ced050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 0x557b06ced050 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f375c0cd160 tx=0x7f375c009200 comp rx=0 tx=0).stop 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 -- 172.21.15.5:0/3064467248 shutdown_connections 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557b06cffa00 0x557b06d01df0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ced590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f37517fa640 0 client.28039 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:31.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 0x557b06ced050 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 --2- 172.21.15.5:0/3064467248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cecb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 -- 172.21.15.5:0/3064467248 >> 172.21.15.5:0/3064467248 conn(0x557b06ce1060 msgr2=0x557b06ce6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 -- 172.21.15.5:0/3064467248 shutdown_connections 2024-09-11T22:33:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 1 -- 172.21.15.5:0/3064467248 wait complete. 2024-09-11T22:33:31.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.179+0000 7f3772de8740 20 client.28039 trim_cache size 0 max 16384 2024-09-11T22:33:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f3772de8740 1 Processor -- start 2024-09-11T22:33:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f3772de8740 1 -- start start 2024-09-11T22:33:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f3772de8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 0x557b06ced3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f3772de8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 0x557b06d073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f3772de8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06d07900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f3772de8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37681ab800 con 0x557b06cefcb0 2024-09-11T22:33:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f3772de8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37681ad340 con 0x557b06ce5410 2024-09-11T22:33:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f3772de8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37681abed0 con 0x557b06ce4af0 2024-09-11T22:33:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f376e5f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06d07900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f376d5ee640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 0x557b06d073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f376e5f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06d07900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48602/0 (socket says 172.21.15.5:48602) 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f376e5f0640 1 -- 172.21.15.5:0/2082003120 learned_addr learned my addr 172.21.15.5:0/2082003120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.194+0000 7f376ddef640 1 --2- 172.21.15.5:0/2082003120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 0x557b06ced3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.195+0000 7f376e5f0640 1 -- 172.21.15.5:0/2082003120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 msgr2=0x557b06ced3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.195+0000 7f376e5f0640 1 --2- 172.21.15.5:0/2082003120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 0x557b06ced3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.195+0000 7f376e5f0640 1 -- 172.21.15.5:0/2082003120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 msgr2=0x557b06d073c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.195+0000 7f376e5f0640 1 --2- 172.21.15.5:0/2082003120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 0x557b06d073c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.195+0000 7f376e5f0640 1 -- 172.21.15.5:0/2082003120 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b06ce5240 con 0x557b06cefcb0 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.195+0000 7f376ddef640 1 --2- 172.21.15.5:0/2082003120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 0x557b06ced3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.195+0000 7f376e5f0640 1 --2- 172.21.15.5:0/2082003120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06d07900 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f3760004040 tx=0x7f376003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07014f89b249fa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:31.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.195+0000 7f37537fe640 1 -- 172.21.15.5:0/2082003120 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760043070 con 0x557b06cefcb0 2024-09-11T22:33:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.195+0000 7f37537fe640 1 -- 172.21.15.5:0/2082003120 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3760005440 con 0x557b06cefcb0 2024-09-11T22:33:31.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.195+0000 7f37537fe640 1 -- 172.21.15.5:0/2082003120 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3760035040 con 0x557b06cefcb0 2024-09-11T22:33:31.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 -- 172.21.15.5:0/2082003120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 msgr2=0x557b06d07900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 --2- 172.21.15.5:0/2082003120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06d07900 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f3760004040 tx=0x7f376003f000 comp rx=0 tx=0).stop 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 -- 172.21.15.5:0/2082003120 shutdown_connections 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 --2- 172.21.15.5:0/2082003120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06d07900 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 --2- 172.21.15.5:0/2082003120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce5410 0x557b06d073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 --2- 172.21.15.5:0/2082003120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce4af0 0x557b06ced3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 -- 172.21.15.5:0/2082003120 >> 172.21.15.5:0/2082003120 conn(0x557b06ce1060 msgr2=0x557b06ceeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 -- 172.21.15.5:0/2082003120 shutdown_connections 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 -- 172.21.15.5:0/2082003120 wait complete. 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 -- start start 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.196+0000 7f3772de8740 1 Processor -- start 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f3772de8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cf82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f3772de8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 0x557b06ce9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f3772de8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ce9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f3772de8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37681acc70 con 0x557b06cefcb0 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f3772de8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37681ac5a0 con 0x557b06ce4af0 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f3772de8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f37681a9800 con 0x557b06ce5410 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f3772de8740 10 client.0 _ll_register_callbacks cb 0x7f3764009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376e5f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ce9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376ddef640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cf82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376ddef640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cf82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48524/0 (socket says 172.21.15.5:48524) 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376e5f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ce9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48610/0 (socket says 172.21.15.5:48610) 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376ddef640 1 -- 172.21.15.5:0/2573210842 learned_addr learned my addr 172.21.15.5:0/2573210842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376d5ee640 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 0x557b06ce9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376e5f0640 1 -- 172.21.15.5:0/2573210842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 msgr2=0x557b06ce9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376e5f0640 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 0x557b06ce9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376e5f0640 1 -- 172.21.15.5:0/2573210842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 msgr2=0x557b06cf82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376e5f0640 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cf82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376e5f0640 1 -- 172.21.15.5:0/2573210842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b06cea4f0 con 0x557b06cefcb0 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.197+0000 7f376ddef640 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cf82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f376e5f0640 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ce9cc0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3760004500 tx=0x7f3760003bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f04f3281fdc7517 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f37517fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f37517fa640 1 -- 172.21.15.5:0/2573210842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376005b070 con 0x557b06cefcb0 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f37517fa640 1 -- 172.21.15.5:0/2573210842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7f37600055f0 con 0x557b06cefcb0 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f3772de8740 1 -- 172.21.15.5:0/2573210842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3760005440 con 0x557b06cefcb0 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f37517fa640 1 -- 172.21.15.5:0/2573210842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3760067050 con 0x557b06cefcb0 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f3772de8740 10 client.18945 fetch_fsmap learned FSMap version 44 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f3772de8740 1 -- 172.21.15.5:0/2573210842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x557b06d6a390 con 0x557b06cefcb0 2024-09-11T22:33:31.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f37517fa640 1 -- 172.21.15.5:0/2573210842 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3760005440 con 0x557b06cefcb0 2024-09-11T22:33:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f37517fa640 1 -- 172.21.15.5:0/2573210842 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) ==== 8362+0+0 (secure 0 0 0) 0x7f376003d720 con 0x557b06cefcb0 2024-09-11T22:33:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f3772de8740 10 client.18945 fetch_fsmap finished waiting for FSMap version 44 2024-09-11T22:33:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f3772de8740 10 client.18945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f3772de8740 20 client.18945 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f3772de8740 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557b06d6a9f0 0x557b06d6cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f3772de8740 4 client.18945 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:33:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.198+0000 7f3772de8740 1 -- 172.21.15.5:0/2573210842 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b06c50d70 con 0x557b06d6a9f0 2024-09-11T22:33:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.199+0000 7f376ddef640 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557b06d6a9f0 0x557b06d6cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.199+0000 7f37517fa640 10 client.18945 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:31.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.199+0000 7f376ddef640 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557b06d6a9f0 0x557b06d6cde0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82f1a6e9829ec9 server_cookie=4f3f0121a5cbd011 in_seq=0 out_seq=0 2024-09-11T22:33:31.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 831454 7.12 MB/sec execute 469 sec latency 62.513 ms 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.369+0000 7f37517fa640 1 -- 172.21.15.5:0/2573210842 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557b06c50d70 con 0x557b06d6a9f0 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.369+0000 7f37517fa640 10 client.18945 handle_command_reply: tid=0 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.369+0000 7f3772de8740 1 client.18945 shutdown 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.369+0000 7f3772de8740 1 -- 172.21.15.5:0/2573210842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 msgr2=0x557b06ce9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.369+0000 7f3772de8740 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ce9cc0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3760004500 tx=0x7f3760003bb0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.369+0000 7f3772de8740 1 -- 172.21.15.5:0/2573210842 shutdown_connections 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.369+0000 7f3772de8740 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557b06d6a9f0 0x557b06d6cde0 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.370+0000 7f3772de8740 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b06cefcb0 0x557b06ce9cc0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.370+0000 7f3772de8740 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557b06ce5410 0x557b06ce9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.370+0000 7f3772de8740 1 --2- 172.21.15.5:0/2573210842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b06ce4af0 0x557b06cf82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.370+0000 7f3772de8740 1 -- 172.21.15.5:0/2573210842 >> 172.21.15.5:0/2573210842 conn(0x557b06ce1060 msgr2=0x557b06cdf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.370+0000 7f37517fa640 0 client.18945 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.370+0000 7f3772de8740 1 -- 172.21.15.5:0/2573210842 shutdown_connections 2024-09-11T22:33:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.370+0000 7f3772de8740 1 -- 172.21.15.5:0/2573210842 wait complete. 2024-09-11T22:33:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.370+0000 7f3772de8740 20 client.18945 trim_cache size 0 max 16384 2024-09-11T22:33:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:33:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:33:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:33:31.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:33:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.370+0000 7f3770053640 1 -- 172.21.15.5:0/1904893292 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f37540a2da0 msgr2=0x7f37540a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.370+0000 7f3770053640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f37540a2da0 0x7f37540a5260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3760006010 tx=0x7f3760009560 comp rx=0 tx=0).stop 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.371+0000 7f3770053640 1 -- 172.21.15.5:0/1904893292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3768101350 msgr2=0x7f37681996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.371+0000 7f3770053640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3768101350 0x7f37681996b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f375c00eb90 tx=0x7f375c00c6a0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.371+0000 7f3770053640 1 -- 172.21.15.5:0/1904893292 shutdown_connections 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.371+0000 7f3770053640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37681069c0 0x7f37681a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.371+0000 7f3770053640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f37540a2da0 0x7f37540a5260 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.371+0000 7f3770053640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3768101350 0x7f37681996b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.371+0000 7f3770053640 1 --2- 172.21.15.5:0/1904893292 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f37681009b0 0x7f3768199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.371+0000 7f3770053640 1 -- 172.21.15.5:0/1904893292 >> 172.21.15.5:0/1904893292 conn(0x7f3768077990 msgr2=0x7f3768103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.371+0000 7f3770053640 1 -- 172.21.15.5:0/1904893292 shutdown_connections 2024-09-11T22:33:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:31.371+0000 7f3770053640 1 -- 172.21.15.5:0/1904893292 wait complete. 2024-09-11T22:33:31.389 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:31 smithi005 ceph-mon[30943]: pgmap v358: 97 pgs: 97 active+clean; 100 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-09-11T22:33:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:31 smithi159 ceph-mon[35534]: pgmap v358: 97 pgs: 97 active+clean; 100 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-09-11T22:33:31.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:31 smithi172 ceph-mon[35461]: pgmap v358: 97 pgs: 97 active+clean; 100 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-09-11T22:33:32.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.149+0000 7f5a9bd92640 1 -- 172.21.15.5:0/642158054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a94101de0 msgr2=0x7f5a941021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:32.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.149+0000 7f5a9bd92640 1 --2- 172.21.15.5:0/642158054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a94101de0 0x7f5a941021c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009900 tx=0x7f5a9002a410 comp rx=0 tx=0).stop 2024-09-11T22:33:32.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.149+0000 7f5a9bd92640 1 -- 172.21.15.5:0/642158054 shutdown_connections 2024-09-11T22:33:32.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.149+0000 7f5a9bd92640 1 --2- 172.21.15.5:0/642158054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a94106f10 0x7f5a9410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:32.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.149+0000 7f5a9bd92640 1 --2- 172.21.15.5:0/642158054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a94102790 0x7f5a941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:32.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.149+0000 7f5a9bd92640 1 --2- 172.21.15.5:0/642158054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a94101de0 0x7f5a941021c0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:32.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.149+0000 7f5a9bd92640 1 -- 172.21.15.5:0/642158054 >> 172.21.15.5:0/642158054 conn(0x7f5a940fd8c0 msgr2=0x7f5a940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:32.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.149+0000 7f5a9bd92640 1 -- 172.21.15.5:0/642158054 shutdown_connections 2024-09-11T22:33:32.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.149+0000 7f5a9bd92640 1 -- 172.21.15.5:0/642158054 wait complete. 2024-09-11T22:33:32.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a9bd92640 1 Processor -- start 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a9bd92640 1 -- start start 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a9bd92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a94101de0 0x7f5a94198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a9bd92640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a94102790 0x7f5a941989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a9bd92640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a94106f10 0x7f5a9419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a9bd92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94111f70 con 0x7f5a94101de0 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a9bd92640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94111df0 con 0x7f5a94102790 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a9bd92640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5a941120f0 con 0x7f5a94106f10 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a99b2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a94101de0 0x7f5a94198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a9932d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a94102790 0x7f5a941989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a99b2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a94101de0 0x7f5a94198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60642/0 (socket says 172.21.15.5:60642) 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a99b2e640 1 -- 172.21.15.5:0/3252963002 learned_addr learned my addr 172.21.15.5:0/3252963002 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.150+0000 7f5a9a32f640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a94106f10 0x7f5a9419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:32.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a99b2e640 1 -- 172.21.15.5:0/3252963002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a94106f10 msgr2=0x7f5a9419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a99b2e640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a94106f10 0x7f5a9419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a99b2e640 1 -- 172.21.15.5:0/3252963002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a94102790 msgr2=0x7f5a941989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a99b2e640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a94102790 0x7f5a941989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a99b2e640 1 -- 172.21.15.5:0/3252963002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9419d440 con 0x7f5a94101de0 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a9932d640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a94102790 0x7f5a941989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a9a32f640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a94106f10 0x7f5a9419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a99b2e640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a94101de0 0x7f5a94198490 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f5a90006010 tx=0x7f5a90004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a8affd640 1 -- 172.21.15.5:0/3252963002 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9003f030 con 0x7f5a94101de0 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a8affd640 1 -- 172.21.15.5:0/3252963002 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a90004440 con 0x7f5a94101de0 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a9bd92640 1 -- 172.21.15.5:0/3252963002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9419d6d0 con 0x7f5a94101de0 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a8affd640 1 -- 172.21.15.5:0/3252963002 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90035070 con 0x7f5a94101de0 2024-09-11T22:33:32.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.151+0000 7f5a9bd92640 1 -- 172.21.15.5:0/3252963002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a941a5010 con 0x7f5a94101de0 2024-09-11T22:33:32.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.153+0000 7f5a9bd92640 1 -- 172.21.15.5:0/3252963002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a941032c0 con 0x7f5a94101de0 2024-09-11T22:33:32.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.154+0000 7f5a8affd640 1 -- 172.21.15.5:0/3252963002 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f5a90042030 con 0x7f5a94101de0 2024-09-11T22:33:32.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.154+0000 7f5a8affd640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a800a2cf0 0x7f5a800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:32.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.155+0000 7f5a9932d640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a800a2cf0 0x7f5a800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:32.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.155+0000 7f5a8affd640 1 -- 172.21.15.5:0/3252963002 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8629+0+0 (secure 0 0 0) 0x7f5a900ef810 con 0x7f5a94101de0 2024-09-11T22:33:32.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.155+0000 7f5a9932d640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a800a2cf0 0x7f5a800a51b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5a940fdd70 tx=0x7f5a84005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:32.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.159+0000 7f5a8affd640 1 -- 172.21.15.5:0/3252963002 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a900bbd70 con 0x7f5a94101de0 2024-09-11T22:33:32.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:33:31 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setuser ceph since I am not root 2024-09-11T22:33:32.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:33:31 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setgroup ceph since I am not root 2024-09-11T22:33:32.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:33:31 smithi159 ceph-mds[65034]: -- 172.21.15.159:0/4078046509 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55934a171860 con 0x55934a14f400 2024-09-11T22:33:32.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:33:31 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: starting mds.k at 2024-09-11T22:33:32.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 833748 7.14 MB/sec execute 470 sec latency 17.253 ms 2024-09-11T22:33:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.388+0000 7f5a9bd92640 1 -- 172.21.15.5:0/3252963002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a9410a0f0 con 0x7f5a94101de0 2024-09-11T22:33:32.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.389+0000 7f5a8affd640 1 -- 172.21.15.5:0/3252963002 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11967 (secure 0 0 0) 0x7f5a900c0070 con 0x7f5a94101de0 2024-09-11T22:33:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:33:32.390 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":44,"btime":"2024-09-11T22:32:26:501017+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:32:26.500999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27887":{"gid":27887,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/1984462744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":1984462744},{"type":"v1","addr":"172.21.15.159:6833","nonce":1984462744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:33:32.390 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 44 2024-09-11T22:33:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 -- 172.21.15.5:0/3252963002 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a800a2cf0 msgr2=0x7f5a800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a800a2cf0 0x7f5a800a51b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5a940fdd70 tx=0x7f5a84005f90 comp rx=0 tx=0).stop 2024-09-11T22:33:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 -- 172.21.15.5:0/3252963002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a94101de0 msgr2=0x7f5a94198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a94101de0 0x7f5a94198490 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f5a90006010 tx=0x7f5a90004040 comp rx=0 tx=0).stop 2024-09-11T22:33:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 -- 172.21.15.5:0/3252963002 shutdown_connections 2024-09-11T22:33:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a94106f10 0x7f5a9419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a800a2cf0 0x7f5a800a51b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a94102790 0x7f5a941989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 --2- 172.21.15.5:0/3252963002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a94101de0 0x7f5a94198490 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 -- 172.21.15.5:0/3252963002 >> 172.21.15.5:0/3252963002 conn(0x7f5a940fd8c0 msgr2=0x7f5a940fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 -- 172.21.15.5:0/3252963002 shutdown_connections 2024-09-11T22:33:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:32.391+0000 7f5a9bd92640 1 -- 172.21.15.5:0/3252963002 wait complete. 2024-09-11T22:33:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:32 smithi005 ceph-mon[30943]: daemon mds.k restarted 2024-09-11T22:33:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:32 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3252963002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:33:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:32 smithi005 ceph-mon[30943]: osdmap e102: 12 total, 12 up, 12 in 2024-09-11T22:33:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:32 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6832/2346038927,v1:172.21.15.159:6833/2346038927] up:boot 2024-09-11T22:33:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:32 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:33:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:32 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:33:32.760 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:32 smithi172 ceph-mon[35461]: daemon mds.k restarted 2024-09-11T22:33:32.760 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:32 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3252963002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:33:32.760 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:32 smithi172 ceph-mon[35461]: osdmap e102: 12 total, 12 up, 12 in 2024-09-11T22:33:32.761 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:32 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6832/2346038927,v1:172.21.15.159:6833/2346038927] up:boot 2024-09-11T22:33:32.761 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:32 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:33:32.761 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:32 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:33:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:32 smithi159 ceph-mon[35534]: daemon mds.k restarted 2024-09-11T22:33:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:32 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3252963002' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:33:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:32 smithi159 ceph-mon[35534]: osdmap e102: 12 total, 12 up, 12 in 2024-09-11T22:33:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:32 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6832/2346038927,v1:172.21.15.159:6833/2346038927] up:boot 2024-09-11T22:33:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:32 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:33:32.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:32 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:33:33.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:33:33.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 835002 7.13 MB/sec execute 471 sec latency 24.145 ms 2024-09-11T22:33:33.402 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:33 smithi005 ceph-mon[30943]: pgmap v359: 97 pgs: 97 active+clean; 91 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T22:33:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:33 smithi159 ceph-mon[35534]: pgmap v359: 97 pgs: 97 active+clean; 91 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T22:33:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:33 smithi172 ceph-mon[35461]: pgmap v359: 97 pgs: 97 active+clean; 91 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.1 MiB/s wr, 337 op/s 2024-09-11T22:33:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 -- 172.21.15.5:0/313227356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe928100e20 msgr2=0x7fe928101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 --2- 172.21.15.5:0/313227356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe928100e20 0x7fe928101200 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe918009900 tx=0x7fe91802a480 comp rx=0 tx=0).stop 2024-09-11T22:33:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 -- 172.21.15.5:0/313227356 shutdown_connections 2024-09-11T22:33:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 --2- 172.21.15.5:0/313227356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe928102ef0 0x7fe92810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 --2- 172.21.15.5:0/313227356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe928102570 0x7fe9281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 --2- 172.21.15.5:0/313227356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe928100e20 0x7fe928101200 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 -- 172.21.15.5:0/313227356 >> 172.21.15.5:0/313227356 conn(0x7fe9280fc7d0 msgr2=0x7fe9280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 -- 172.21.15.5:0/313227356 shutdown_connections 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 -- 172.21.15.5:0/313227356 wait complete. 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 Processor -- start 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.933+0000 7fe92df9f640 1 -- start start 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe92df9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe928100e20 0x7fe928103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe92df9f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe928102570 0x7fe928103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe92df9f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe928102ef0 0x7fe9281041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe92df9f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe928110f80 con 0x7fe928100e20 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe92df9f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe928110e00 con 0x7fe928102ef0 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe92df9f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe928111100 con 0x7fe928102570 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe928100e20 0x7fe928103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe928100e20 0x7fe928103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60650/0 (socket says 172.21.15.5:60650) 2024-09-11T22:33:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9277fe640 1 -- 172.21.15.5:0/1006376386 learned_addr learned my addr 172.21.15.5:0/1006376386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe927fff640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe928102ef0 0x7fe9281041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9277fe640 1 -- 172.21.15.5:0/1006376386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe928102570 msgr2=0x7fe928103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9277fe640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe928102570 0x7fe928103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9277fe640 1 -- 172.21.15.5:0/1006376386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe928102ef0 msgr2=0x7fe9281041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9277fe640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe928102ef0 0x7fe9281041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9277fe640 1 -- 172.21.15.5:0/1006376386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9281982d0 con 0x7fe928100e20 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9277fe640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe928100e20 0x7fe928103750 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe918006010 tx=0x7fe918003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9257fa640 1 -- 172.21.15.5:0/1006376386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91803f070 con 0x7fe928100e20 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9257fa640 1 -- 172.21.15.5:0/1006376386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe918031c30 con 0x7fe928100e20 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.934+0000 7fe9257fa640 1 -- 172.21.15.5:0/1006376386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918035030 con 0x7fe928100e20 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.935+0000 7fe92df9f640 1 -- 172.21.15.5:0/1006376386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe928198560 con 0x7fe928100e20 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.935+0000 7fe92df9f640 1 -- 172.21.15.5:0/1006376386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9281989f0 con 0x7fe928100e20 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.937+0000 7fe9257fa640 1 -- 172.21.15.5:0/1006376386 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe918042070 con 0x7fe928100e20 2024-09-11T22:33:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe930378740 1 Processor -- start 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe930378740 1 -- start start 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe9257fa640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9140a2cf0 0x7fe9140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe930378740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe930378740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 0x5601a40d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe930378740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 0x5601a40df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe930378740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5601a40382f0 con 0x5601a40d3b40 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe930378740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5601a3fdfcf0 con 0x5601a40dd370 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe930378740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5601a40342b0 con 0x5601a40d25d0 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe927fff640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9140a2cf0 0x7fe9140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe9257fa640 1 -- 172.21.15.5:0/1006376386 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fe9180efc00 con 0x7fe928100e20 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60662/0 (socket says 172.21.15.5:60662) 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe9277fe640 1 -- 172.21.15.5:0/2042154377 learned_addr learned my addr 172.21.15.5:0/2042154377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.938+0000 7fe91ffff640 1 --2- 172.21.15.5:0/2042154377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 0x5601a40d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe927fff640 1 --2- 172.21.15.5:0/2042154377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 0x5601a40df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe9277fe640 1 -- 172.21.15.5:0/2042154377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 msgr2=0x5601a40d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe9277fe640 1 --2- 172.21.15.5:0/2042154377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 0x5601a40d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe9277fe640 1 -- 172.21.15.5:0/2042154377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 msgr2=0x5601a40df760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe9277fe640 1 --2- 172.21.15.5:0/2042154377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 0x5601a40df760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe9277fe640 1 -- 172.21.15.5:0/2042154377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a3fc43a0 con 0x5601a40d3b40 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe91ffff640 1 --2- 172.21.15.5:0/2042154377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 0x5601a40d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe927fff640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9140a2cf0 0x7fe9140a51b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe920005fd0 tx=0x7fe920011b40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe9277fe640 1 --2- 172.21.15.5:0/2042154377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d2090 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe9180c47c0 tx=0x7fe9180b7d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5017d419589804a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe91dffb640 1 -- 172.21.15.5:0/2042154377 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918049070 con 0x5601a40d3b40 2024-09-11T22:33:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe91dffb640 1 -- 172.21.15.5:0/2042154377 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe918031de0 con 0x5601a40d3b40 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.939+0000 7fe91dffb640 1 -- 172.21.15.5:0/2042154377 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9180ba380 con 0x5601a40d3b40 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.940+0000 7fe930378740 1 -- 172.21.15.5:0/2042154377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 msgr2=0x5601a40d2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.940+0000 7fe930378740 1 --2- 172.21.15.5:0/2042154377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d2090 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fe9180c47c0 tx=0x7fe9180b7d20 comp rx=0 tx=0).stop 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.940+0000 7fe930378740 1 -- 172.21.15.5:0/2042154377 shutdown_connections 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.940+0000 7fe930378740 1 --2- 172.21.15.5:0/2042154377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 0x5601a40df760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.940+0000 7fe930378740 1 --2- 172.21.15.5:0/2042154377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 0x5601a40d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.940+0000 7fe930378740 1 --2- 172.21.15.5:0/2042154377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d2090 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.940+0000 7fe930378740 1 -- 172.21.15.5:0/2042154377 >> 172.21.15.5:0/2042154377 conn(0x5601a409b310 msgr2=0x5601a40cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.940+0000 7fe930378740 1 -- 172.21.15.5:0/2042154377 shutdown_connections 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.940+0000 7fe930378740 1 -- 172.21.15.5:0/2042154377 wait complete. 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.940+0000 7fe930378740 1 -- start start 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe930378740 1 Processor -- start 2024-09-11T22:33:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe930378740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40d25d0 0x5601a40dcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe930378740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40da070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe930378740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 0x5601a40da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe930378740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5601a40dfe30 con 0x5601a40d3b40 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe930378740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5601a3ff3060 con 0x5601a40d25d0 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe930378740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5601a40dffb0 con 0x5601a40dd370 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe930378740 10 client.0 _ll_register_callbacks cb 0x5601a3d468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe91ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40da070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe91ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40da070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60668/0 (socket says 172.21.15.5:60668) 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe91ffff640 1 -- 172.21.15.5:0/1012432068 learned_addr learned my addr 172.21.15.5:0/1012432068 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe9277fe640 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40d25d0 0x5601a40dcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.941+0000 7fe927fff640 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 0x5601a40da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe91ffff640 1 -- 172.21.15.5:0/1012432068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 msgr2=0x5601a40da6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe91ffff640 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 0x5601a40da6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe91ffff640 1 -- 172.21.15.5:0/1012432068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40d25d0 msgr2=0x5601a40dcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe91ffff640 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40d25d0 0x5601a40dcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe91ffff640 1 -- 172.21.15.5:0/1012432068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a3da93c0 con 0x5601a40d3b40 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe91ffff640 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40da070 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe9100116f0 tx=0x7fe910011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd18ea6a692bd67 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe906ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe906ffd640 1 -- 172.21.15.5:0/1012432068 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91001e060 con 0x5601a40d3b40 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe930378740 1 -- 172.21.15.5:0/1012432068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe918031de0 con 0x5601a40d3b40 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe906ffd640 1 -- 172.21.15.5:0/1012432068 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fe91000fd30 con 0x5601a40d3b40 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe906ffd640 1 -- 172.21.15.5:0/1012432068 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe910019430 con 0x5601a40d3b40 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe930378740 10 client.18966 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe930378740 1 -- 172.21.15.5:0/1012432068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5601a40eb8b0 con 0x5601a40d3b40 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe906ffd640 1 -- 172.21.15.5:0/1012432068 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe918031de0 con 0x5601a40d3b40 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.942+0000 7fe906ffd640 1 -- 172.21.15.5:0/1012432068 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fe910011ce0 con 0x5601a40d3b40 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.943+0000 7fe930378740 10 client.18966 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.943+0000 7fe930378740 10 client.18966 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.943+0000 7fe930378740 20 client.18966 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.943+0000 7fe930378740 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5601a40ecd70 0x5601a40ef160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.943+0000 7fe930378740 4 client.18966 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.943+0000 7fe930378740 1 -- 172.21.15.5:0/1012432068 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601a4045860 con 0x5601a40ecd70 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.943+0000 7fe9277fe640 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5601a40ecd70 0x5601a40ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.943+0000 7fe9277fe640 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5601a40ecd70 0x5601a40ef160 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1f1a1225d4d3128a server_cookie=87c7a7836a710d55 in_seq=0 out_seq=0 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.943+0000 7fe906ffd640 10 client.18966 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe906ffd640 1 -- 172.21.15.5:0/1012432068 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601a4045860 con 0x5601a40ecd70 2024-09-11T22:33:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe906ffd640 10 client.18966 handle_command_reply: tid=0 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe930378740 1 client.18966 shutdown 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe930378740 1 -- 172.21.15.5:0/1012432068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 msgr2=0x5601a40da070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe930378740 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40da070 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe9100116f0 tx=0x7fe910011bb0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe930378740 1 -- 172.21.15.5:0/1012432068 shutdown_connections 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe930378740 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5601a40ecd70 0x5601a40ef160 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe930378740 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 0x5601a40da6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe930378740 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40da070 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe906ffd640 0 client.18966 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe930378740 1 --2- 172.21.15.5:0/1012432068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40d25d0 0x5601a40dcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.944+0000 7fe930378740 1 -- 172.21.15.5:0/1012432068 >> 172.21.15.5:0/1012432068 conn(0x5601a409b310 msgr2=0x5601a40d4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.945+0000 7fe930378740 1 -- 172.21.15.5:0/1012432068 shutdown_connections 2024-09-11T22:33:33.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.945+0000 7fe930378740 1 -- 172.21.15.5:0/1012432068 wait complete. 2024-09-11T22:33:33.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.945+0000 7fe930378740 20 client.18966 trim_cache size 0 max 16384 2024-09-11T22:33:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.958+0000 7fe930378740 1 Processor -- start 2024-09-11T22:33:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.958+0000 7fe930378740 1 -- start start 2024-09-11T22:33:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe930378740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 0x5601a40dbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe930378740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe930378740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 0x5601a40d56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe930378740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281b4150 con 0x5601a40d3b40 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe930378740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281b5c90 con 0x5601a40dd370 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe930378740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281b4820 con 0x5601a40d25d0 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe91ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe91ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60672/0 (socket says 172.21.15.5:60672) 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe927fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 0x5601a40d56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe91ffff640 1 -- 172.21.15.5:0/924519954 learned_addr learned my addr 172.21.15.5:0/924519954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe9277fe640 1 --2- 172.21.15.5:0/924519954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 0x5601a40dbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe91ffff640 1 -- 172.21.15.5:0/924519954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 msgr2=0x5601a40dbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe91ffff640 1 --2- 172.21.15.5:0/924519954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 0x5601a40dbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe91ffff640 1 -- 172.21.15.5:0/924519954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 msgr2=0x5601a40d56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe91ffff640 1 --2- 172.21.15.5:0/924519954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 0x5601a40d56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe91ffff640 1 -- 172.21.15.5:0/924519954 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a40ef7c0 con 0x5601a40d3b40 2024-09-11T22:33:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe9277fe640 1 --2- 172.21.15.5:0/924519954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 0x5601a40dbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe927fff640 1 --2- 172.21.15.5:0/924519954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 0x5601a40d56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe91ffff640 1 --2- 172.21.15.5:0/924519954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d5180 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe91000e7b0 tx=0x7fe9100092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5585a58e1c9140e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.959+0000 7fe91d7fa640 1 -- 172.21.15.5:0/924519954 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91002e020 con 0x5601a40d3b40 2024-09-11T22:33:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe91d7fa640 1 -- 172.21.15.5:0/924519954 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe910005bf0 con 0x5601a40d3b40 2024-09-11T22:33:33.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe91d7fa640 1 -- 172.21.15.5:0/924519954 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe910019950 con 0x5601a40d3b40 2024-09-11T22:33:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe930378740 1 -- 172.21.15.5:0/924519954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 msgr2=0x5601a40d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe930378740 1 --2- 172.21.15.5:0/924519954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d5180 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe91000e7b0 tx=0x7fe9100092e0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe930378740 1 -- 172.21.15.5:0/924519954 shutdown_connections 2024-09-11T22:33:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe930378740 1 --2- 172.21.15.5:0/924519954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40dd370 0x5601a40d56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe930378740 1 --2- 172.21.15.5:0/924519954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d3b40 0x5601a40d5180 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe930378740 1 --2- 172.21.15.5:0/924519954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40d25d0 0x5601a40dbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe930378740 1 -- 172.21.15.5:0/924519954 >> 172.21.15.5:0/924519954 conn(0x5601a409b310 msgr2=0x5601a40db8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe930378740 1 -- 172.21.15.5:0/924519954 shutdown_connections 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.960+0000 7fe930378740 1 -- 172.21.15.5:0/924519954 wait complete. 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.961+0000 7fe930378740 1 -- start start 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.961+0000 7fe930378740 1 Processor -- start 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.961+0000 7fe930378740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d25d0 0x5601a40e5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.961+0000 7fe930378740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40d3b40 0x5601a40e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.961+0000 7fe930378740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 0x5601a40e12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.961+0000 7fe930378740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281b55c0 con 0x5601a40d25d0 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.961+0000 7fe930378740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281b4ef0 con 0x5601a40d3b40 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.961+0000 7fe930378740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9281b2170 con 0x5601a40dd370 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.961+0000 7fe930378740 10 client.0 _ll_register_callbacks cb 0x7fe9100096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.961+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d25d0 0x5601a40e5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe9277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d25d0 0x5601a40e5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60678/0 (socket says 172.21.15.5:60678) 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe927fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 0x5601a40e12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe9277fe640 1 -- 172.21.15.5:0/2971746859 learned_addr learned my addr 172.21.15.5:0/2971746859 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe91ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40d3b40 0x5601a40e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe9277fe640 1 -- 172.21.15.5:0/2971746859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 msgr2=0x5601a40e12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe9277fe640 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 0x5601a40e12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe9277fe640 1 -- 172.21.15.5:0/2971746859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40d3b40 msgr2=0x5601a40e0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe9277fe640 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40d3b40 0x5601a40e0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe9277fe640 1 -- 172.21.15.5:0/2971746859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a40e1ad0 con 0x5601a40d25d0 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe927fff640 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 0x5601a40e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe9277fe640 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d25d0 0x5601a40e5960 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe918000c00 tx=0x7fe91802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9895567486a2675 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:33.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe906ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe906ffd640 1 -- 172.21.15.5:0/2971746859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe918049040 con 0x5601a40d25d0 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe906ffd640 1 -- 172.21.15.5:0/2971746859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fe9180a52a0 con 0x5601a40d25d0 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe930378740 1 -- 172.21.15.5:0/2971746859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe910005bf0 con 0x5601a40d25d0 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe906ffd640 1 -- 172.21.15.5:0/2971746859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe918031de0 con 0x5601a40d25d0 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe930378740 10 client.18972 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.962+0000 7fe930378740 1 -- 172.21.15.5:0/2971746859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5601a4157b30 con 0x5601a40d25d0 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe906ffd640 1 -- 172.21.15.5:0/2971746859 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe910005bf0 con 0x5601a40d25d0 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe906ffd640 1 -- 172.21.15.5:0/2971746859 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fe9180c00b0 con 0x5601a40d25d0 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe930378740 10 client.18972 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe930378740 10 client.18972 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe930378740 20 client.18972 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe930378740 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5601a4158100 0x5601a415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe930378740 4 client.18972 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe930378740 1 -- 172.21.15.5:0/2971746859 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5601a403fd70 con 0x5601a4158100 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe91ffff640 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5601a4158100 0x5601a415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe906ffd640 10 client.18972 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.963+0000 7fe91ffff640 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5601a4158100 0x5601a415a4f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3c214b566b2e0d3 server_cookie=84e549dcbf158319 in_seq=0 out_seq=0 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe906ffd640 1 -- 172.21.15.5:0/2971746859 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5601a403fd70 con 0x5601a4158100 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe906ffd640 10 client.18972 handle_command_reply: tid=0 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 client.18972 shutdown 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 -- 172.21.15.5:0/2971746859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d25d0 msgr2=0x5601a40e5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d25d0 0x5601a40e5960 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe918000c00 tx=0x7fe91802a480 comp rx=0 tx=0).stop 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 -- 172.21.15.5:0/2971746859 shutdown_connections 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5601a4158100 0x5601a415a4f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5601a40dd370 0x5601a40e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601a40d3b40 0x5601a40e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe906ffd640 0 client.18972 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:33:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 --2- 172.21.15.5:0/2971746859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601a40d25d0 0x5601a40e5960 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 -- 172.21.15.5:0/2971746859 >> 172.21.15.5:0/2971746859 conn(0x5601a409b310 msgr2=0x5601a40dc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 -- 172.21.15.5:0/2971746859 shutdown_connections 2024-09-11T22:33:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 1 -- 172.21.15.5:0/2971746859 wait complete. 2024-09-11T22:33:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.964+0000 7fe930378740 20 client.18972 trim_cache size 0 max 16384 2024-09-11T22:33:33.971 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:33:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 -- 172.21.15.5:0/1006376386 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9140a2cf0 msgr2=0x7fe9140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9140a2cf0 0x7fe9140a51b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe920005fd0 tx=0x7fe920011b40 comp rx=0 tx=0).stop 2024-09-11T22:33:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 -- 172.21.15.5:0/1006376386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe928100e20 msgr2=0x7fe928103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe928100e20 0x7fe928103750 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fe918006010 tx=0x7fe918003c10 comp rx=0 tx=0).stop 2024-09-11T22:33:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 -- 172.21.15.5:0/1006376386 shutdown_connections 2024-09-11T22:33:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9140a2cf0 0x7fe9140a51b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe928102ef0 0x7fe9281041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe928102570 0x7fe928103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 --2- 172.21.15.5:0/1006376386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe928100e20 0x7fe928103750 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:33.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 -- 172.21.15.5:0/1006376386 >> 172.21.15.5:0/1006376386 conn(0x7fe9280fc7d0 msgr2=0x7fe92806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:33.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 -- 172.21.15.5:0/1006376386 shutdown_connections 2024-09-11T22:33:33.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:33.965+0000 7fe92df9f640 1 -- 172.21.15.5:0/1006376386 wait complete. 2024-09-11T22:33:34.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 837286 7.15 MB/sec execute 472 sec latency 33.259 ms 2024-09-11T22:33:34.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:34 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:33:34.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:34 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:33:34.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:34 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:33:34.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:33:35.166 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:35.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 838500 7.14 MB/sec execute 473 sec latency 15.832 ms 2024-09-11T22:33:35.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:35 smithi005 ceph-mon[30943]: pgmap v361: 97 pgs: 97 active+clean; 91 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 244 op/s 2024-09-11T22:33:35.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:35 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:33:35.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:35 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.713+0000 7f9060f28640 1 -- 172.21.15.5:0/2888951596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f905c100920 msgr2=0x7f905c100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.713+0000 7f9060f28640 1 --2- 172.21.15.5:0/2888951596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f905c100920 0x7f905c100d00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f904c009860 tx=0x7f904c02a340 comp rx=0 tx=0).stop 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.713+0000 7f9060f28640 1 -- 172.21.15.5:0/2888951596 shutdown_connections 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.713+0000 7f9060f28640 1 --2- 172.21.15.5:0/2888951596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c10e4d0 0x7f905c1108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.713+0000 7f9060f28640 1 --2- 172.21.15.5:0/2888951596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c1012d0 0x7f905c10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.713+0000 7f9060f28640 1 --2- 172.21.15.5:0/2888951596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f905c100920 0x7f905c100d00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.713+0000 7f9060f28640 1 -- 172.21.15.5:0/2888951596 >> 172.21.15.5:0/2888951596 conn(0x7f905c0fc790 msgr2=0x7f905c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.713+0000 7f9060f28640 1 -- 172.21.15.5:0/2888951596 shutdown_connections 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.713+0000 7f9060f28640 1 -- 172.21.15.5:0/2888951596 wait complete. 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9060f28640 1 Processor -- start 2024-09-11T22:33:35.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9060f28640 1 -- start start 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9060f28640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c100920 0x7f905c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9060f28640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c1012d0 0x7f905c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9060f28640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f905c10e4d0 0x7f905c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9060f28640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f905c114170 con 0x7f905c1012d0 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9060f28640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f905c113ff0 con 0x7f905c100920 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9060f28640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f905c1142f0 con 0x7f905c10e4d0 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9059d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c1012d0 0x7f905c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9059d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c1012d0 0x7f905c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60702/0 (socket says 172.21.15.5:60702) 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f9059d9b640 1 -- 172.21.15.5:0/2603175846 learned_addr learned my addr 172.21.15.5:0/2603175846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f905a59c640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c100920 0x7f905c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.714+0000 7f905ad9d640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f905c10e4d0 0x7f905c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f9059d9b640 1 -- 172.21.15.5:0/2603175846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f905c10e4d0 msgr2=0x7f905c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f9059d9b640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f905c10e4d0 0x7f905c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f9059d9b640 1 -- 172.21.15.5:0/2603175846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c100920 msgr2=0x7f905c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f9059d9b640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c100920 0x7f905c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f9059d9b640 1 -- 172.21.15.5:0/2603175846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905c19f600 con 0x7f905c1012d0 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f9059d9b640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c1012d0 0x7f905c19ab90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f905000e9c0 tx=0x7f905000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f904b7fe640 1 -- 172.21.15.5:0/2603175846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905000cde0 con 0x7f905c1012d0 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f9060f28640 1 -- 172.21.15.5:0/2603175846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905c19f8f0 con 0x7f905c1012d0 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f904b7fe640 1 -- 172.21.15.5:0/2603175846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90500035f0 con 0x7f905c1012d0 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f9060f28640 1 -- 172.21.15.5:0/2603175846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905c1a71d0 con 0x7f905c1012d0 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.715+0000 7f904b7fe640 1 -- 172.21.15.5:0/2603175846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050010650 con 0x7f905c1012d0 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.717+0000 7f904b7fe640 1 -- 172.21.15.5:0/2603175846 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f90500037a0 con 0x7f905c1012d0 2024-09-11T22:33:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f9063301740 1 Processor -- start 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f9063301740 1 -- start start 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f904b7fe640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90440a2cf0 0x7f90440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f9063301740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda2f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f9063301740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda2fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f904b7fe640 1 -- 172.21.15.5:0/2603175846 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f90500cf830 con 0x7f905c1012d0 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f9063301740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda3c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f9063301740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564bed9952f0 con 0x564beda30b40 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f9063301740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564bed93ccf0 con 0x564beda3a370 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f9063301740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564bed9912b0 con 0x564beda2f5d0 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.718+0000 7f905ad9d640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90440a2cf0 0x7f90440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f905a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda2f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f905a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda2f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60718/0 (socket says 172.21.15.5:60718) 2024-09-11T22:33:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f905a59c640 1 -- 172.21.15.5:0/2581181973 learned_addr learned my addr 172.21.15.5:0/2581181973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f9059d9b640 1 --2- 172.21.15.5:0/2581181973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda2fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f905ad9d640 1 --2- 172.21.15.5:0/2581181973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda3c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f9059d9b640 1 -- 172.21.15.5:0/2581181973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 msgr2=0x564beda3c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f9059d9b640 1 --2- 172.21.15.5:0/2581181973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda3c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f9059d9b640 1 -- 172.21.15.5:0/2581181973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 msgr2=0x564beda2f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f9059d9b640 1 --2- 172.21.15.5:0/2581181973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda2f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f9059d9b640 1 -- 172.21.15.5:0/2581181973 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bed9213a0 con 0x564beda2f5d0 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f905a59c640 1 --2- 172.21.15.5:0/2581181973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda2f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f905ad9d640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90440a2cf0 0x7f90440a51b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f905c102b00 tx=0x7f9054013ae0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f9059d9b640 1 --2- 172.21.15.5:0/2581181973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda2fa10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f90500ccee0 tx=0x7f905009bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e497d35ec510316e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f9048ff9640 1 -- 172.21.15.5:0/2581181973 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90500a0060 con 0x564beda2f5d0 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f9048ff9640 1 -- 172.21.15.5:0/2581181973 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90500a4920 con 0x564beda2f5d0 2024-09-11T22:33:35.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.719+0000 7f9048ff9640 1 -- 172.21.15.5:0/2581181973 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905009b100 con 0x564beda2f5d0 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.720+0000 7f9063301740 1 -- 172.21.15.5:0/2581181973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 msgr2=0x564beda2fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.720+0000 7f9063301740 1 --2- 172.21.15.5:0/2581181973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda2fa10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f90500ccee0 tx=0x7f905009bbb0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.720+0000 7f9063301740 1 -- 172.21.15.5:0/2581181973 shutdown_connections 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.720+0000 7f9063301740 1 --2- 172.21.15.5:0/2581181973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda3c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.720+0000 7f9063301740 1 --2- 172.21.15.5:0/2581181973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda2fa10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.720+0000 7f9063301740 1 --2- 172.21.15.5:0/2581181973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda2f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.720+0000 7f9063301740 1 -- 172.21.15.5:0/2581181973 >> 172.21.15.5:0/2581181973 conn(0x564bed9f8310 msgr2=0x564beda2bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.720+0000 7f9063301740 1 -- 172.21.15.5:0/2581181973 shutdown_connections 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.720+0000 7f9063301740 1 -- 172.21.15.5:0/2581181973 wait complete. 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9063301740 1 -- start start 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9063301740 1 Processor -- start 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9063301740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda39da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9063301740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda37050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9063301740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9063301740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564beda3d010 con 0x564beda30b40 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9063301740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564bed950060 con 0x564beda3a370 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9063301740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564beda3d190 con 0x564beda2f5d0 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9063301740 10 client.0 _ll_register_callbacks cb 0x564bed6a38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9059d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda37050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9059d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda37050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60724/0 (socket says 172.21.15.5:60724) 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9059d9b640 1 -- 172.21.15.5:0/709217799 learned_addr learned my addr 172.21.15.5:0/709217799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f905a59c640 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda39da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f905ad9d640 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9059d9b640 1 -- 172.21.15.5:0/709217799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 msgr2=0x564beda39da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9059d9b640 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda39da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9059d9b640 1 -- 172.21.15.5:0/709217799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 msgr2=0x564beda377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9059d9b640 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda377e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.721+0000 7f9059d9b640 1 -- 172.21.15.5:0/709217799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bed7063c0 con 0x564beda30b40 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9059d9b640 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda37050 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9050099f60 tx=0x7f9050004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b705c22e5a0ab5de server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9041ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9041ffb640 1 -- 172.21.15.5:0/709217799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90500cd160 con 0x564beda30b40 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9041ffb640 1 -- 172.21.15.5:0/709217799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f905009b100 con 0x564beda30b40 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9063301740 1 -- 172.21.15.5:0/709217799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90500a4920 con 0x564beda30b40 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9041ffb640 1 -- 172.21.15.5:0/709217799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f905009b300 con 0x564beda30b40 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9063301740 10 client.18984 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9063301740 1 -- 172.21.15.5:0/709217799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564beda48af0 con 0x564beda30b40 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9041ffb640 1 -- 172.21.15.5:0/709217799 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90500a4920 con 0x564beda30b40 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9041ffb640 1 -- 172.21.15.5:0/709217799 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f9050082030 con 0x564beda30b40 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9063301740 10 client.18984 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9063301740 10 client.18984 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9063301740 20 client.18984 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9063301740 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x564beda49e40 0x564beda4c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9063301740 4 client.18984 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.722+0000 7f9063301740 1 -- 172.21.15.5:0/709217799 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bed9a2860 con 0x564beda49e40 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.723+0000 7f905a59c640 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x564beda49e40 0x564beda4c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.723+0000 7f905a59c640 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x564beda49e40 0x564beda4c230 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15cde052274d25b8 server_cookie=be9a9c9336d38dd7 in_seq=0 out_seq=0 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.723+0000 7f9041ffb640 10 client.18984 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9041ffb640 1 -- 172.21.15.5:0/709217799 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bed9a2860 con 0x564beda49e40 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9041ffb640 10 client.18984 handle_command_reply: tid=0 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 client.18984 shutdown 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 -- 172.21.15.5:0/709217799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 msgr2=0x564beda37050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda37050 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9050099f60 tx=0x7f9050004560 comp rx=0 tx=0).stop 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 -- 172.21.15.5:0/709217799 shutdown_connections 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x564beda49e40 0x564beda4c230 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda37050 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9041ffb640 0 client.18984 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 --2- 172.21.15.5:0/709217799 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda39da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 -- 172.21.15.5:0/709217799 >> 172.21.15.5:0/709217799 conn(0x564bed9f8310 msgr2=0x564beda31960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 -- 172.21.15.5:0/709217799 shutdown_connections 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 1 -- 172.21.15.5:0/709217799 wait complete. 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.724+0000 7f9063301740 20 client.18984 trim_cache size 0 max 16384 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f9063301740 1 Processor -- start 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f9063301740 1 -- start start 2024-09-11T22:33:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f9063301740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda2bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f9063301740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda30b40 0x564beda45b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f9063301740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda3a370 0x564beda4f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f9063301740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f905c1afd80 con 0x564beda3a370 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f9063301740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f905c1b18c0 con 0x564beda30b40 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f9063301740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f905c1b0450 con 0x564beda2f5d0 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f905ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda3a370 0x564beda4f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f905ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda3a370 0x564beda4f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60738/0 (socket says 172.21.15.5:60738) 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f905ad9d640 1 -- 172.21.15.5:0/3644232781 learned_addr learned my addr 172.21.15.5:0/3644232781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f905a59c640 1 --2- 172.21.15.5:0/3644232781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda2bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f9059d9b640 1 --2- 172.21.15.5:0/3644232781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda30b40 0x564beda45b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f905ad9d640 1 -- 172.21.15.5:0/3644232781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 msgr2=0x564beda2bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f905ad9d640 1 --2- 172.21.15.5:0/3644232781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda2bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f905ad9d640 1 -- 172.21.15.5:0/3644232781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda30b40 msgr2=0x564beda45b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f905ad9d640 1 --2- 172.21.15.5:0/3644232781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda30b40 0x564beda45b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.738+0000 7f905ad9d640 1 -- 172.21.15.5:0/3644232781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564beda2c960 con 0x564beda3a370 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f905ad9d640 1 --2- 172.21.15.5:0/3644232781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda3a370 0x564beda4f250 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f90540123f0 tx=0x7f9054009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e1e0297d0062a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f9043fff640 1 -- 172.21.15.5:0/3644232781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905400fe30 con 0x564beda3a370 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f9043fff640 1 -- 172.21.15.5:0/3644232781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9054025070 con 0x564beda3a370 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f9043fff640 1 -- 172.21.15.5:0/3644232781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054010ad0 con 0x564beda3a370 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f9063301740 1 -- 172.21.15.5:0/3644232781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda3a370 msgr2=0x564beda4f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f9063301740 1 --2- 172.21.15.5:0/3644232781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda3a370 0x564beda4f250 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f90540123f0 tx=0x7f9054009500 comp rx=0 tx=0).stop 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f9063301740 1 -- 172.21.15.5:0/3644232781 shutdown_connections 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f9063301740 1 --2- 172.21.15.5:0/3644232781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda3a370 0x564beda4f250 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f9063301740 1 --2- 172.21.15.5:0/3644232781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda30b40 0x564beda45b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f9063301740 1 --2- 172.21.15.5:0/3644232781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda2bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.739+0000 7f9063301740 1 -- 172.21.15.5:0/3644232781 >> 172.21.15.5:0/3644232781 conn(0x564bed9f8310 msgr2=0x564beda45420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 1 -- 172.21.15.5:0/3644232781 shutdown_connections 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 1 -- 172.21.15.5:0/3644232781 wait complete. 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 1 -- start start 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 1 Processor -- start 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda4fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda4ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda4d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f905c1b11f0 con 0x564beda30b40 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f905c1b0b20 con 0x564beda3a370 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f905c1add60 con 0x564beda2f5d0 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9063301740 10 client.0 _ll_register_callbacks cb 0x7f9050005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9059d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda4ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9059d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda4ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60750/0 (socket says 172.21.15.5:60750) 2024-09-11T22:33:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f9059d9b640 1 -- 172.21.15.5:0/2903332847 learned_addr learned my addr 172.21.15.5:0/2903332847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f905a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda4fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.740+0000 7f905ad9d640 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda4d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9059d9b640 1 -- 172.21.15.5:0/2903332847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 msgr2=0x564beda4fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9059d9b640 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda4fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9059d9b640 1 -- 172.21.15.5:0/2903332847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 msgr2=0x564beda4d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9059d9b640 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda4d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9059d9b640 1 -- 172.21.15.5:0/2903332847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564beda4dc40 con 0x564beda30b40 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9059d9b640 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda4ce20 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f9050098680 tx=0x7f905007ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c0cd79c6e4a389 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9041ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9041ffb640 1 -- 172.21.15.5:0/2903332847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9050097f50 con 0x564beda30b40 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9041ffb640 1 -- 172.21.15.5:0/2903332847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f905009b100 con 0x564beda30b40 2024-09-11T22:33:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9063301740 1 -- 172.21.15.5:0/2903332847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9054025070 con 0x564beda30b40 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9041ffb640 1 -- 172.21.15.5:0/2903332847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f90500a4920 con 0x564beda30b40 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9063301740 10 client.18990 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9063301740 1 -- 172.21.15.5:0/2903332847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564beda4dc40 con 0x564beda30b40 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9041ffb640 1 -- 172.21.15.5:0/2903332847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9054025070 con 0x564beda30b40 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9041ffb640 1 -- 172.21.15.5:0/2903332847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f9050082030 con 0x564beda30b40 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9063301740 10 client.18990 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9063301740 10 client.18990 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.741+0000 7f9063301740 20 client.18990 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f9063301740 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x564bedab4be0 0x564bedab7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f9063301740 4 client.18990 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f9063301740 1 -- 172.21.15.5:0/2903332847 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bed99cd70 con 0x564bedab4be0 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f905a59c640 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x564bedab4be0 0x564bedab7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f905a59c640 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x564bedab4be0 0x564bedab7000 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=75301ab07d91c8f8 server_cookie=c2967e46c221ebf7 in_seq=0 out_seq=0 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f9041ffb640 10 client.18990 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f9041ffb640 1 -- 172.21.15.5:0/2903332847 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bed99cd70 con 0x564bedab4be0 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f9041ffb640 10 client.18990 handle_command_reply: tid=0 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f9063301740 1 client.18990 shutdown 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f9063301740 1 -- 172.21.15.5:0/2903332847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 msgr2=0x564beda4ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.742+0000 7f9063301740 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda4ce20 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f9050098680 tx=0x7f905007ad00 comp rx=0 tx=0).stop 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9063301740 1 -- 172.21.15.5:0/2903332847 shutdown_connections 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9063301740 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x564bedab4be0 0x564bedab7000 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9063301740 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564beda3a370 0x564beda4d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9063301740 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564beda30b40 0x564beda4ce20 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9063301740 1 --2- 172.21.15.5:0/2903332847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564beda2f5d0 0x564beda4fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9063301740 1 -- 172.21.15.5:0/2903332847 >> 172.21.15.5:0/2903332847 conn(0x564bed9f8310 msgr2=0x564beda45c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:35.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9041ffb640 0 client.18990 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:33:35.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9063301740 1 -- 172.21.15.5:0/2903332847 shutdown_connections 2024-09-11T22:33:35.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9063301740 1 -- 172.21.15.5:0/2903332847 wait complete. 2024-09-11T22:33:35.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9063301740 20 client.18990 trim_cache size 0 max 16384 2024-09-11T22:33:35.761 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9060f28640 1 -- 172.21.15.5:0/2603175846 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90440a2cf0 msgr2=0x7f90440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9060f28640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90440a2cf0 0x7f90440a51b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f905c102b00 tx=0x7f9054013ae0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9060f28640 1 -- 172.21.15.5:0/2603175846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c1012d0 msgr2=0x7f905c19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9060f28640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c1012d0 0x7f905c19ab90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f905000e9c0 tx=0x7f905000ee80 comp rx=0 tx=0).stop 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9060f28640 1 -- 172.21.15.5:0/2603175846 shutdown_connections 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9060f28640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f905c10e4d0 0x7f905c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9060f28640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f905c1012d0 0x7f905c19ab90 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9060f28640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f90440a2cf0 0x7f90440a51b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9060f28640 1 --2- 172.21.15.5:0/2603175846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f905c100920 0x7f905c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.743+0000 7f9060f28640 1 -- 172.21.15.5:0/2603175846 >> 172.21.15.5:0/2603175846 conn(0x7f905c0fc790 msgr2=0x7f905c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.744+0000 7f9060f28640 1 -- 172.21.15.5:0/2603175846 shutdown_connections 2024-09-11T22:33:35.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:35.744+0000 7f9060f28640 1 -- 172.21.15.5:0/2603175846 wait complete. 2024-09-11T22:33:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:35 smithi159 ceph-mon[35534]: pgmap v361: 97 pgs: 97 active+clean; 91 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 244 op/s 2024-09-11T22:33:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:35 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:33:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:35 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:33:35.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:35 smithi172 ceph-mon[35461]: pgmap v361: 97 pgs: 97 active+clean; 91 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 244 op/s 2024-09-11T22:33:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:35 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:33:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:35 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:33:36.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 840406 7.15 MB/sec execute 474 sec latency 37.984 ms 2024-09-11T22:33:36.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:33:36.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:33:36.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:33:36.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:33:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:33:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:33:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:33:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:33:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:33:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:33:36.871 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:37.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 841819 7.16 MB/sec execute 475 sec latency 26.051 ms 2024-09-11T22:33:37.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.638+0000 7fdccb93a640 1 -- 172.21.15.5:0/315719042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc4102d40 msgr2=0x7fdcc4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:37.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.638+0000 7fdccb93a640 1 --2- 172.21.15.5:0/315719042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc4102d40 0x7fdcc4103120 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009900 tx=0x7fdcc002a480 comp rx=0 tx=0).stop 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 -- 172.21.15.5:0/315719042 shutdown_connections 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 --2- 172.21.15.5:0/315719042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc410c170 0x7fdcc410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 --2- 172.21.15.5:0/315719042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc4103660 0x7fdcc410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 --2- 172.21.15.5:0/315719042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc4102d40 0x7fdcc4103120 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 -- 172.21.15.5:0/315719042 >> 172.21.15.5:0/315719042 conn(0x7fdcc40fc7a0 msgr2=0x7fdcc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 -- 172.21.15.5:0/315719042 shutdown_connections 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 -- 172.21.15.5:0/315719042 wait complete. 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 Processor -- start 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 -- start start 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc4102d40 0x7fdcc4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc4103660 0x7fdcc4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.639+0000 7fdccb93a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc410c170 0x7fdcc419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdccb93a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4111fc0 con 0x7fdcc4103660 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdccb93a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4111e40 con 0x7fdcc4102d40 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdccb93a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4112140 con 0x7fdcc410c170 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc8ed5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc4103660 0x7fdcc4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc8ed5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc4103660 0x7fdcc4198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60762/0 (socket says 172.21.15.5:60762) 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc8ed5640 1 -- 172.21.15.5:0/4026623576 learned_addr learned my addr 172.21.15.5:0/4026623576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc96d6640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc4102d40 0x7fdcc4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc8ed5640 1 -- 172.21.15.5:0/4026623576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc410c170 msgr2=0x7fdcc419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc410c170 0x7fdcc419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc8ed5640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc410c170 0x7fdcc419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc8ed5640 1 -- 172.21.15.5:0/4026623576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc4102d40 msgr2=0x7fdcc4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc8ed5640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc4102d40 0x7fdcc4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc8ed5640 1 -- 172.21.15.5:0/4026623576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc419d3f0 con 0x7fdcc4103660 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcc8ed5640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc4103660 0x7fdcc4198980 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9f0 tx=0x7fdcb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcba7fc640 1 -- 172.21.15.5:0/4026623576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb400ccc0 con 0x7fdcc4103660 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcba7fc640 1 -- 172.21.15.5:0/4026623576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb4010910 con 0x7fdcc4103660 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdcba7fc640 1 -- 172.21.15.5:0/4026623576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4018920 con 0x7fdcc4103660 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdccb93a640 1 -- 172.21.15.5:0/4026623576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc419d6e0 con 0x7fdcc4103660 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.640+0000 7fdccb93a640 1 -- 172.21.15.5:0/4026623576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc41a4fc0 con 0x7fdcc4103660 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdccdd13740 1 Processor -- start 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdcba7fc640 1 -- 172.21.15.5:0/4026623576 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fdcb4010430 con 0x7fdcc4103660 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdccdd13740 1 -- start start 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdccdd13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5555824282f0 con 0x5555824c25d0 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdccdd13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5555823cfcf0 con 0x5555824cd370 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdccdd13740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555824242b0 con 0x5555824c3b40 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdcc96d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.643+0000 7fdcc9ed7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc96d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824c2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60778/0 (socket says 172.21.15.5:60778) 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc96d6640 1 -- 172.21.15.5:0/104890136 learned_addr learned my addr 172.21.15.5:0/104890136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc8ed5640 1 --2- 172.21.15.5:0/104890136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/104890136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 msgr2=0x5555824cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/104890136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/104890136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 msgr2=0x5555824c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/104890136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824c2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcba7fc640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdcb00a2cf0 0x7fdcb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/104890136 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555823b43a0 con 0x5555824c3b40 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc96d6640 1 --2- 172.21.15.5:0/104890136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc8ed5640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdcb00a2cf0 0x7fdcb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/104890136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824c2090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009d90 tx=0x7fdcbc00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa1ff53d07bdd654 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcba7fc640 1 -- 172.21.15.5:0/4026623576 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fdcb40cf7b0 con 0x7fdcc4103660 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcaf7fe640 1 -- 172.21.15.5:0/104890136 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc003970 con 0x5555824c3b40 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcaf7fe640 1 -- 172.21.15.5:0/104890136 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc004030 con 0x5555824c3b40 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.644+0000 7fdcaf7fe640 1 -- 172.21.15.5:0/104890136 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc011800 con 0x5555824c3b40 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.645+0000 7fdccdd13740 1 -- 172.21.15.5:0/104890136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 msgr2=0x5555824c2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.645+0000 7fdccdd13740 1 --2- 172.21.15.5:0/104890136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824c2090 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009d90 tx=0x7fdcbc00f730 comp rx=0 tx=0).stop 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.645+0000 7fdcc8ed5640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdcb00a2cf0 0x7fdcb00a51b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40ccf70 tx=0x7fdcb40d5550 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:37.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.645+0000 7fdccdd13740 1 -- 172.21.15.5:0/104890136 shutdown_connections 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.645+0000 7fdccdd13740 1 --2- 172.21.15.5:0/104890136 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.645+0000 7fdccdd13740 1 --2- 172.21.15.5:0/104890136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.645+0000 7fdccdd13740 1 --2- 172.21.15.5:0/104890136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824c2090 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.645+0000 7fdccdd13740 1 -- 172.21.15.5:0/104890136 >> 172.21.15.5:0/104890136 conn(0x55558248b310 msgr2=0x5555824bec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.645+0000 7fdccdd13740 1 -- 172.21.15.5:0/104890136 shutdown_connections 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.645+0000 7fdccdd13740 1 -- 172.21.15.5:0/104890136 wait complete. 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdccdd13740 1 -- start start 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdccdd13740 1 Processor -- start 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdccdd13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5555824d0070 con 0x5555824c25d0 2024-09-11T22:33:37.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdccdd13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5555823e3060 con 0x5555824cd370 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdccdd13740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5555824d01f0 con 0x5555824c3b40 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdccdd13740 10 client.0 _ll_register_callbacks cb 0x5555821368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdcc9ed7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.646+0000 7fdcc9ed7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60784/0 (socket says 172.21.15.5:60784) 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/2639975190 learned_addr learned my addr 172.21.15.5:0/2639975190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcc96d6640 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcc96d6640 1 -- 172.21.15.5:0/2639975190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 msgr2=0x5555824ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcc96d6640 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcc96d6640 1 -- 172.21.15.5:0/2639975190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 msgr2=0x5555824ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcc96d6640 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcc96d6640 1 -- 172.21.15.5:0/2639975190 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555821993c0 con 0x5555824c3b40 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcc96d6640 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824c9fd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0031550 tx=0x7fdcc00042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd5f67b68fabffe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcacff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcacff9640 1 -- 172.21.15.5:0/2639975190 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0035040 con 0x5555824c3b40 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdccdd13740 1 -- 172.21.15.5:0/2639975190 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcbc004030 con 0x5555824c3b40 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcacff9640 1 -- 172.21.15.5:0/2639975190 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fdcc0005450 con 0x5555824c3b40 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdcacff9640 1 -- 172.21.15.5:0/2639975190 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdcc0005650 con 0x5555824c3b40 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdccdd13740 10 client.28099 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.647+0000 7fdccdd13740 1 -- 172.21.15.5:0/2639975190 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5555824dbb10 con 0x5555824c3b40 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdcacff9640 1 -- 172.21.15.5:0/2639975190 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc004030 con 0x5555824c3b40 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdcacff9640 1 -- 172.21.15.5:0/2639975190 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fdcc0003930 con 0x5555824c3b40 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdccdd13740 10 client.28099 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdccdd13740 10 client.28099 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdccdd13740 20 client.28099 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdccdd13740 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5555824dcfd0 0x5555824df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdccdd13740 4 client.28099 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdccdd13740 1 -- 172.21.15.5:0/2639975190 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555582435860 con 0x5555824dcfd0 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5555824dcfd0 0x5555824df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5555824dcfd0 0x5555824df3e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7aa7cf445393bd5c server_cookie=ae072b917960cc42 in_seq=0 out_seq=0 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.648+0000 7fdcacff9640 10 client.28099 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdcacff9640 1 -- 172.21.15.5:0/2639975190 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555582435860 con 0x5555824dcfd0 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdcacff9640 10 client.28099 handle_command_reply: tid=0 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdccdd13740 1 client.28099 shutdown 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdccdd13740 1 -- 172.21.15.5:0/2639975190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 msgr2=0x5555824c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdccdd13740 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824c9fd0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0031550 tx=0x7fdcc00042e0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdccdd13740 1 -- 172.21.15.5:0/2639975190 shutdown_connections 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdccdd13740 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5555824dcfd0 0x5555824df3e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdccdd13740 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdccdd13740 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824c9fd0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdccdd13740 1 --2- 172.21.15.5:0/2639975190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdcacff9640 0 client.28099 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.649+0000 7fdccdd13740 1 -- 172.21.15.5:0/2639975190 >> 172.21.15.5:0/2639975190 conn(0x55558248b310 msgr2=0x5555824c4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.650+0000 7fdccdd13740 1 -- 172.21.15.5:0/2639975190 shutdown_connections 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.650+0000 7fdccdd13740 1 -- 172.21.15.5:0/2639975190 wait complete. 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.650+0000 7fdccdd13740 20 client.28099 trim_cache size 0 max 16384 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.664+0000 7fdccdd13740 1 Processor -- start 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.664+0000 7fdccdd13740 1 -- start start 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824c25d0 0x5555824c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824d9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824cd370 0x5555824c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdccdd13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41ad970 con 0x5555824cd370 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdccdd13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41af4b0 con 0x5555824c25d0 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdccdd13740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41ae040 con 0x5555824c3b40 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc9ed7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824cd370 0x5555824c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc9ed7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824cd370 0x5555824c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60792/0 (socket says 172.21.15.5:60792) 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/1583998563 learned_addr learned my addr 172.21.15.5:0/1583998563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc96d6640 1 --2- 172.21.15.5:0/1583998563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/1583998563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824c25d0 0x5555824c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/1583998563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 msgr2=0x5555824d9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/1583998563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824d9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/1583998563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824c25d0 msgr2=0x5555824c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/1583998563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824c25d0 0x5555824c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/1583998563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555824cd090 con 0x5555824cd370 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/1583998563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824cd370 0x5555824c5120 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009540 tx=0x7fdcbc0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b00fafa6ef0595 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.665+0000 7fdcaeffd640 1 -- 172.21.15.5:0/1583998563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc031030 con 0x5555824cd370 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdcaeffd640 1 -- 172.21.15.5:0/1583998563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc026040 con 0x5555824cd370 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdcaeffd640 1 -- 172.21.15.5:0/1583998563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc025040 con 0x5555824cd370 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdccdd13740 1 -- 172.21.15.5:0/1583998563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824cd370 msgr2=0x5555824c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdccdd13740 1 --2- 172.21.15.5:0/1583998563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824cd370 0x5555824c5120 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009540 tx=0x7fdcbc0092a0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdccdd13740 1 -- 172.21.15.5:0/1583998563 shutdown_connections 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdccdd13740 1 --2- 172.21.15.5:0/1583998563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824cd370 0x5555824c5120 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdccdd13740 1 --2- 172.21.15.5:0/1583998563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824d9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdccdd13740 1 --2- 172.21.15.5:0/1583998563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824c25d0 0x5555824c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdccdd13740 1 -- 172.21.15.5:0/1583998563 >> 172.21.15.5:0/1583998563 conn(0x55558248b310 msgr2=0x5555824c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdccdd13740 1 -- 172.21.15.5:0/1583998563 shutdown_connections 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.666+0000 7fdccdd13740 1 -- 172.21.15.5:0/1583998563 wait complete. 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdccdd13740 1 -- start start 2024-09-11T22:33:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdccdd13740 1 Processor -- start 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdccdd13740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdccdd13740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41aede0 con 0x5555824c25d0 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdccdd13740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41ae710 con 0x5555824cd370 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdccdd13740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41ab950 con 0x5555824c3b40 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdccdd13740 10 client.0 _ll_register_callbacks cb 0x7fdcc0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdcc9ed7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdcc9ed7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824d5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60808/0 (socket says 172.21.15.5:60808) 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/4286444827 learned_addr learned my addr 172.21.15.5:0/4286444827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.667+0000 7fdcc96d6640 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/4286444827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 msgr2=0x5555824d0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:37.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824d0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/4286444827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 msgr2=0x5555824d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824d1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcc9ed7640 1 -- 172.21.15.5:0/4286444827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555824d16b0 con 0x5555824c25d0 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcc9ed7640 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824d5900 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc006010 tx=0x7fdcbc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02c6a05c71acbb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcacff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcacff9640 1 -- 172.21.15.5:0/4286444827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc031070 con 0x5555824c25d0 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcacff9640 1 -- 172.21.15.5:0/4286444827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fdcbc003d80 con 0x5555824c25d0 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdccdd13740 1 -- 172.21.15.5:0/4286444827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcbc026040 con 0x5555824c25d0 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcacff9640 1 -- 172.21.15.5:0/4286444827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc025040 con 0x5555824c25d0 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdccdd13740 10 client.19008 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdccdd13740 1 -- 172.21.15.5:0/4286444827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5555825479b0 con 0x5555824c25d0 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.668+0000 7fdcacff9640 1 -- 172.21.15.5:0/4286444827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc026040 con 0x5555824c25d0 2024-09-11T22:33:37.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdcacff9640 1 -- 172.21.15.5:0/4286444827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fdcbc02b030 con 0x5555824c25d0 2024-09-11T22:33:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdccdd13740 10 client.19008 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdccdd13740 10 client.19008 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:33:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdccdd13740 20 client.19008 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdccdd13740 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555582547ed0 0x55558254a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdccdd13740 4 client.19008 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:33:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdccdd13740 1 -- 172.21.15.5:0/4286444827 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55558242fd70 con 0x555582547ed0 2024-09-11T22:33:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdcc96d6640 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555582547ed0 0x55558254a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdcc96d6640 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555582547ed0 0x55558254a2c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8fe8499d7703f87 server_cookie=50b9fda902922741 in_seq=0 out_seq=0 2024-09-11T22:33:37.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdcacff9640 10 client.19008 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:33:37.687 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:33:37.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdcacff9640 1 -- 172.21.15.5:0/4286444827 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55558242fd70 con 0x555582547ed0 2024-09-11T22:33:37.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.669+0000 7fdcacff9640 10 client.19008 handle_command_reply: tid=0 2024-09-11T22:33:37.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 client.19008 shutdown 2024-09-11T22:33:37.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 -- 172.21.15.5:0/4286444827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 msgr2=0x5555824d5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:37.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824d5900 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc006010 tx=0x7fdcbc009310 comp rx=0 tx=0).stop 2024-09-11T22:33:37.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 -- 172.21.15.5:0/4286444827 shutdown_connections 2024-09-11T22:33:37.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x555582547ed0 0x55558254a2c0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5555824cd370 0x5555824d1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5555824c3b40 0x5555824d0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 --2- 172.21.15.5:0/4286444827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5555824c25d0 0x5555824d5900 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 -- 172.21.15.5:0/4286444827 >> 172.21.15.5:0/4286444827 conn(0x55558248b310 msgr2=0x5555824cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdcacff9640 0 client.19008 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 -- 172.21.15.5:0/4286444827 shutdown_connections 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 1 -- 172.21.15.5:0/4286444827 wait complete. 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.670+0000 7fdccdd13740 20 client.19008 trim_cache size 0 max 16384 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.671+0000 7fdccb93a640 1 -- 172.21.15.5:0/4026623576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdcb00a2cf0 msgr2=0x7fdcb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.671+0000 7fdccb93a640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdcb00a2cf0 0x7fdcb00a51b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40ccf70 tx=0x7fdcb40d5550 comp rx=0 tx=0).stop 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.671+0000 7fdccb93a640 1 -- 172.21.15.5:0/4026623576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc4103660 msgr2=0x7fdcc4198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.671+0000 7fdccb93a640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc4103660 0x7fdcc4198980 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9f0 tx=0x7fdcb400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.671+0000 7fdccb93a640 1 -- 172.21.15.5:0/4026623576 shutdown_connections 2024-09-11T22:33:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.671+0000 7fdccb93a640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdcc410c170 0x7fdcc419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.671+0000 7fdccb93a640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcc4103660 0x7fdcc4198980 secure :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9f0 tx=0x7fdcb400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.672+0000 7fdccb93a640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdcb00a2cf0 0x7fdcb00a51b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.672+0000 7fdccb93a640 1 --2- 172.21.15.5:0/4026623576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcc4102d40 0x7fdcc4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.672+0000 7fdccb93a640 1 -- 172.21.15.5:0/4026623576 >> 172.21.15.5:0/4026623576 conn(0x7fdcc40fc7a0 msgr2=0x7fdcc40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.672+0000 7fdccb93a640 1 -- 172.21.15.5:0/4026623576 shutdown_connections 2024-09-11T22:33:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:37.672+0000 7fdccb93a640 1 -- 172.21.15.5:0/4026623576 wait complete. 2024-09-11T22:33:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:37 smithi005 ceph-mon[30943]: pgmap v362: 97 pgs: 97 active+clean; 91 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 244 op/s 2024-09-11T22:33:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:37 smithi159 ceph-mon[35534]: pgmap v362: 97 pgs: 97 active+clean; 91 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 244 op/s 2024-09-11T22:33:38.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:37 smithi172 ceph-mon[35461]: pgmap v362: 97 pgs: 97 active+clean; 91 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 244 op/s 2024-09-11T22:33:38.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 843427 7.15 MB/sec execute 476 sec latency 55.290 ms 2024-09-11T22:33:38.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:33:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:39.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 845158 7.16 MB/sec execute 477 sec latency 123.707 ms 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.515+0000 7f49fd16a640 1 -- 172.21.15.5:0/4129075186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f81036e0 msgr2=0x7f49f8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.515+0000 7f49fd16a640 1 --2- 172.21.15.5:0/4129075186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f81036e0 0x7f49f8107b90 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009930 tx=0x7f49e002a420 comp rx=0 tx=0).stop 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.515+0000 7f49fd16a640 1 -- 172.21.15.5:0/4129075186 shutdown_connections 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.515+0000 7f49fd16a640 1 --2- 172.21.15.5:0/4129075186 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f81082a0 0x7f49f810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.515+0000 7f49fd16a640 1 --2- 172.21.15.5:0/4129075186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f81036e0 0x7f49f8107b90 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.515+0000 7f49fd16a640 1 --2- 172.21.15.5:0/4129075186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f8102d30 0x7f49f8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.515+0000 7f49fd16a640 1 -- 172.21.15.5:0/4129075186 >> 172.21.15.5:0/4129075186 conn(0x7f49f80fc7d0 msgr2=0x7f49f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.515+0000 7f49fd16a640 1 -- 172.21.15.5:0/4129075186 shutdown_connections 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.515+0000 7f49fd16a640 1 -- 172.21.15.5:0/4129075186 wait complete. 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49fd16a640 1 Processor -- start 2024-09-11T22:33:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49fd16a640 1 -- start start 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49fd16a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f8102d30 0x7f49f81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49fd16a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f81036e0 0x7f49f8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49fd16a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f81082a0 0x7f49f819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49fd16a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81120b0 con 0x7f49f8102d30 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49fd16a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8111f30 con 0x7f49f81036e0 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49fd16a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49f8112230 con 0x7f49f81082a0 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f8102d30 0x7f49f81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f8102d30 0x7f49f81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60820/0 (socket says 172.21.15.5:60820) 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49f6d9d640 1 -- 172.21.15.5:0/419535843 learned_addr learned my addr 172.21.15.5:0/419535843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49effff640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f81036e0 0x7f49f8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.516+0000 7f49f759e640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f81082a0 0x7f49f819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49f6d9d640 1 -- 172.21.15.5:0/419535843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f81082a0 msgr2=0x7f49f819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f81082a0 0x7f49f819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49f6d9d640 1 -- 172.21.15.5:0/419535843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f81036e0 msgr2=0x7f49f8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f81036e0 0x7f49f8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49f6d9d640 1 -- 172.21.15.5:0/419535843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f819d570 con 0x7f49f8102d30 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f8102d30 0x7f49f81985c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f49e800e9f0 tx=0x7f49e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49f4d99640 1 -- 172.21.15.5:0/419535843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e800cd80 con 0x7f49f8102d30 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49fd16a640 1 -- 172.21.15.5:0/419535843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f819d860 con 0x7f49f8102d30 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49f4d99640 1 -- 172.21.15.5:0/419535843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e8010910 con 0x7f49f8102d30 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49f4d99640 1 -- 172.21.15.5:0/419535843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80188f0 con 0x7f49f8102d30 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.517+0000 7f49fd16a640 1 -- 172.21.15.5:0/419535843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f81a5140 con 0x7f49f8102d30 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.520+0000 7f49f4d99640 1 -- 172.21.15.5:0/419535843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f49e8010ac0 con 0x7f49f8102d30 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.520+0000 7f49ff543740 1 Processor -- start 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.520+0000 7f49ff543740 1 -- start start 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49ff543740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6eb40 0x555c47f6d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49ff543740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6d5d0 0x555c47f6da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49ff543740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f78370 0x555c47f7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49ff543740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c47ed32f0 con 0x555c47f6eb40 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49ff543740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555c47e7acf0 con 0x555c47f78370 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49ff543740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555c47ecf2b0 con 0x555c47f6d5d0 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49f759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6eb40 0x555c47f6d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49f4d99640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49e40a2cf0 0x7f49e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49effff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f78370 0x555c47f7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6d5d0 0x555c47f6da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49effff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f78370 0x555c47f7a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39730/0 (socket says 172.21.15.5:39730) 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49effff640 1 -- 172.21.15.5:0/2035196487 learned_addr learned my addr 172.21.15.5:0/2035196487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49f759e640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49e40a2cf0 0x7f49e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49f4d99640 1 -- 172.21.15.5:0/419535843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f49e80cf8b0 con 0x7f49f8102d30 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49effff640 1 -- 172.21.15.5:0/2035196487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6d5d0 msgr2=0x555c47f6da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49effff640 1 --2- 172.21.15.5:0/2035196487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6d5d0 0x555c47f6da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49effff640 1 -- 172.21.15.5:0/2035196487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6eb40 msgr2=0x555c47f6d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49effff640 1 --2- 172.21.15.5:0/2035196487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6eb40 0x555c47f6d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49effff640 1 -- 172.21.15.5:0/2035196487 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c47e5f3a0 con 0x555c47f78370 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.521+0000 7f49f759e640 1 --2- 172.21.15.5:0/2035196487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6eb40 0x555c47f6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.522+0000 7f49f759e640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49e40a2cf0 0x7f49e40a51b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009fd0 tx=0x7f49f0004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.522+0000 7f49effff640 1 --2- 172.21.15.5:0/2035196487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f78370 0x555c47f7a760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f49e0005ab0 tx=0x7f49e0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9634ce627119d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.522+0000 7f49ed7fa640 1 -- 172.21.15.5:0/2035196487 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0040070 con 0x555c47f78370 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.522+0000 7f49ed7fa640 1 -- 172.21.15.5:0/2035196487 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e0004400 con 0x555c47f78370 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.522+0000 7f49ed7fa640 1 -- 172.21.15.5:0/2035196487 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0035040 con 0x555c47f78370 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 -- 172.21.15.5:0/2035196487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f78370 msgr2=0x555c47f7a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 --2- 172.21.15.5:0/2035196487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f78370 0x555c47f7a760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f49e0005ab0 tx=0x7f49e0005c00 comp rx=0 tx=0).stop 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 -- 172.21.15.5:0/2035196487 shutdown_connections 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 --2- 172.21.15.5:0/2035196487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f78370 0x555c47f7a760 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 --2- 172.21.15.5:0/2035196487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6d5d0 0x555c47f6da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 --2- 172.21.15.5:0/2035196487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6eb40 0x555c47f6d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 -- 172.21.15.5:0/2035196487 >> 172.21.15.5:0/2035196487 conn(0x555c47f36310 msgr2=0x555c47f69c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 -- 172.21.15.5:0/2035196487 shutdown_connections 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 -- 172.21.15.5:0/2035196487 wait complete. 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 -- start start 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.523+0000 7f49ff543740 1 Processor -- start 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49ff543740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6d5d0 0x555c47f77d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49ff543740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6eb40 0x555c47f75010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49ff543740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f78370 0x555c47f75600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49ff543740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c47f7b010 con 0x555c47f78370 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49ff543740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555c47e8e060 con 0x555c47f6d5d0 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49ff543740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555c47f7b190 con 0x555c47f6eb40 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49ff543740 10 client.0 _ll_register_callbacks cb 0x555c47be18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6d5d0 0x555c47f77d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49f759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f78370 0x555c47f75600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6d5d0 0x555c47f77d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39736/0 (socket says 172.21.15.5:39736) 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49f6d9d640 1 -- 172.21.15.5:0/4248101066 learned_addr learned my addr 172.21.15.5:0/4248101066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49effff640 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6eb40 0x555c47f75010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49f759e640 1 -- 172.21.15.5:0/4248101066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6eb40 msgr2=0x555c47f75010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49f759e640 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6eb40 0x555c47f75010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49f759e640 1 -- 172.21.15.5:0/4248101066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6d5d0 msgr2=0x555c47f77d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49f759e640 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6d5d0 0x555c47f77d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49f759e640 1 -- 172.21.15.5:0/4248101066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c47c443c0 con 0x555c47f78370 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49effff640 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6eb40 0x555c47f75010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49f759e640 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f78370 0x555c47f75600 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f49f00106e0 tx=0x7f49f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29ae7b1e55ada71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.524+0000 7f49de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49de7fc640 1 -- 172.21.15.5:0/4248101066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0004b20 con 0x555c47f78370 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49de7fc640 1 -- 172.21.15.5:0/4248101066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f49f0004cc0 con 0x555c47f78370 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49ff543740 1 -- 172.21.15.5:0/4248101066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e0004400 con 0x555c47f78370 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49de7fc640 1 -- 172.21.15.5:0/4248101066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f49f0005020 con 0x555c47f78370 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49ff543740 10 client.19020 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49ff543740 1 -- 172.21.15.5:0/4248101066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555c47f86b10 con 0x555c47f78370 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49de7fc640 1 -- 172.21.15.5:0/4248101066 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e0004400 con 0x555c47f78370 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49de7fc640 1 -- 172.21.15.5:0/4248101066 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f49f0029610 con 0x555c47f78370 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49ff543740 10 client.19020 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49ff543740 10 client.19020 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49ff543740 20 client.19020 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49ff543740 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555c47f87fd0 0x555c47f8a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49ff543740 4 client.19020 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.525+0000 7f49ff543740 1 -- 172.21.15.5:0/4248101066 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c47ee0860 con 0x555c47f87fd0 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.526+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555c47f87fd0 0x555c47f8a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.526+0000 7f49de7fc640 10 client.19020 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.526+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555c47f87fd0 0x555c47f8a3e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ee50e7fe4a274d0 server_cookie=33aef68178f89d in_seq=0 out_seq=0 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49de7fc640 1 -- 172.21.15.5:0/4248101066 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c47ee0860 con 0x555c47f87fd0 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49de7fc640 10 client.19020 handle_command_reply: tid=0 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 client.19020 shutdown 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 -- 172.21.15.5:0/4248101066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f78370 msgr2=0x555c47f75600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f78370 0x555c47f75600 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f49f00106e0 tx=0x7f49f0004a40 comp rx=0 tx=0).stop 2024-09-11T22:33:39.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 -- 172.21.15.5:0/4248101066 shutdown_connections 2024-09-11T22:33:39.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555c47f87fd0 0x555c47f8a3e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f78370 0x555c47f75600 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f6eb40 0x555c47f75010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 --2- 172.21.15.5:0/4248101066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6d5d0 0x555c47f77d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49de7fc640 0 client.19020 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:33:39.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 -- 172.21.15.5:0/4248101066 >> 172.21.15.5:0/4248101066 conn(0x555c47f36310 msgr2=0x555c47f6f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:39.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 -- 172.21.15.5:0/4248101066 shutdown_connections 2024-09-11T22:33:39.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 1 -- 172.21.15.5:0/4248101066 wait complete. 2024-09-11T22:33:39.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.527+0000 7f49ff543740 20 client.19020 trim_cache size 0 max 16384 2024-09-11T22:33:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49ff543740 1 Processor -- start 2024-09-11T22:33:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49ff543740 1 -- start start 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49ff543740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f76f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49ff543740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 0x555c47f701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49ff543740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 0x555c47f706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49ff543740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81ada50 con 0x555c47f6d5d0 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49ff543740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81af590 con 0x555c47f6eb40 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49ff543740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81ae120 con 0x555c47f78370 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f76f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f76f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60838/0 (socket says 172.21.15.5:60838) 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49f6d9d640 1 -- 172.21.15.5:0/3012397426 learned_addr learned my addr 172.21.15.5:0/3012397426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49effff640 1 --2- 172.21.15.5:0/3012397426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 0x555c47f701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.541+0000 7f49f759e640 1 --2- 172.21.15.5:0/3012397426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 0x555c47f706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.542+0000 7f49f6d9d640 1 -- 172.21.15.5:0/3012397426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 msgr2=0x555c47f706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.542+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/3012397426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 0x555c47f706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.542+0000 7f49f6d9d640 1 -- 172.21.15.5:0/3012397426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 msgr2=0x555c47f701b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.542+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/3012397426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 0x555c47f701b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.542+0000 7f49f6d9d640 1 -- 172.21.15.5:0/3012397426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c47f76390 con 0x555c47f6d5d0 2024-09-11T22:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.542+0000 7f49effff640 1 --2- 172.21.15.5:0/3012397426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 0x555c47f701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.542+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/3012397426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f76f80 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009230 tx=0x7f49e80d1d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b218a82c1931e497 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.542+0000 7f49ecff9640 1 -- 172.21.15.5:0/3012397426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e809a0e0 con 0x555c47f6d5d0 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.542+0000 7f49ecff9640 1 -- 172.21.15.5:0/3012397426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e8098450 con 0x555c47f6d5d0 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.542+0000 7f49ecff9640 1 -- 172.21.15.5:0/3012397426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e8097b30 con 0x555c47f6d5d0 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 -- 172.21.15.5:0/3012397426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 msgr2=0x555c47f76f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 --2- 172.21.15.5:0/3012397426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f76f80 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009230 tx=0x7f49e80d1d00 comp rx=0 tx=0).stop 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 -- 172.21.15.5:0/3012397426 shutdown_connections 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 --2- 172.21.15.5:0/3012397426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 0x555c47f706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 --2- 172.21.15.5:0/3012397426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 0x555c47f701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 --2- 172.21.15.5:0/3012397426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f76f80 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 -- 172.21.15.5:0/3012397426 >> 172.21.15.5:0/3012397426 conn(0x555c47f36310 msgr2=0x555c47f76870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 -- 172.21.15.5:0/3012397426 shutdown_connections 2024-09-11T22:33:39.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 -- 172.21.15.5:0/3012397426 wait complete. 2024-09-11T22:33:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 -- start start 2024-09-11T22:33:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.543+0000 7f49ff543740 1 Processor -- start 2024-09-11T22:33:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49ff543740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f80ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49ff543740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 0x555c47f79260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49ff543740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 0x555c47f797a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49ff543740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81aeec0 con 0x555c47f6d5d0 2024-09-11T22:33:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49ff543740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81ae7f0 con 0x555c47f6eb40 2024-09-11T22:33:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49ff543740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49f81aba30 con 0x555c47f78370 2024-09-11T22:33:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49ff543740 10 client.0 _ll_register_callbacks cb 0x7f49f000ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f80ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 0x555c47f797a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f80ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60844/0 (socket says 172.21.15.5:60844) 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 0x555c47f797a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46636/0 (socket says 172.21.15.5:46636) 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f6d9d640 1 -- 172.21.15.5:0/683198165 learned_addr learned my addr 172.21.15.5:0/683198165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49effff640 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 0x555c47f79260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f6d9d640 1 -- 172.21.15.5:0/683198165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 msgr2=0x555c47f797a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 0x555c47f797a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f6d9d640 1 -- 172.21.15.5:0/683198165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 msgr2=0x555c47f79260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 0x555c47f79260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f6d9d640 1 -- 172.21.15.5:0/683198165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c47f79fa0 con 0x555c47f6d5d0 2024-09-11T22:33:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49f6d9d640 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f80ad0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f49e800cfd0 tx=0x7f49e809b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9b33d55c6608d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.544+0000 7f49de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49de7fc640 1 -- 172.21.15.5:0/683198165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e80cd1e0 con 0x555c47f6d5d0 2024-09-11T22:33:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49de7fc640 1 -- 172.21.15.5:0/683198165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f49e80cd380 con 0x555c47f6d5d0 2024-09-11T22:33:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49ff543740 1 -- 172.21.15.5:0/683198165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e8098450 con 0x555c47f6d5d0 2024-09-11T22:33:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49de7fc640 1 -- 172.21.15.5:0/683198165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f49e8010430 con 0x555c47f6d5d0 2024-09-11T22:33:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49ff543740 10 client.19026 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49ff543740 1 -- 172.21.15.5:0/683198165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555c47ff2c80 con 0x555c47f6d5d0 2024-09-11T22:33:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49de7fc640 1 -- 172.21.15.5:0/683198165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49e8098450 con 0x555c47f6d5d0 2024-09-11T22:33:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49de7fc640 1 -- 172.21.15.5:0/683198165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f49e80ba020 con 0x555c47f6d5d0 2024-09-11T22:33:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49ff543740 10 client.19026 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49ff543740 10 client.19026 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:33:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49ff543740 20 client.19026 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49ff543740 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555c47ff3340 0x555c47ff5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49ff543740 4 client.19026 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:33:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49ff543740 1 -- 172.21.15.5:0/683198165 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c47edad70 con 0x555c47ff3340 2024-09-11T22:33:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.545+0000 7f49effff640 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555c47ff3340 0x555c47ff5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49effff640 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555c47ff3340 0x555c47ff5730 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4529893a3f6ff6a server_cookie=581550bf5e7cb6ce in_seq=0 out_seq=0 2024-09-11T22:33:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49de7fc640 10 client.19026 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:33:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49de7fc640 1 -- 172.21.15.5:0/683198165 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c47edad70 con 0x555c47ff3340 2024-09-11T22:33:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49de7fc640 10 client.19026 handle_command_reply: tid=0 2024-09-11T22:33:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 client.19026 shutdown 2024-09-11T22:33:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 -- 172.21.15.5:0/683198165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 msgr2=0x555c47f80ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f80ad0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f49e800cfd0 tx=0x7f49e809b230 comp rx=0 tx=0).stop 2024-09-11T22:33:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 -- 172.21.15.5:0/683198165 shutdown_connections 2024-09-11T22:33:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555c47ff3340 0x555c47ff5730 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555c47f78370 0x555c47f797a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c47f6eb40 0x555c47f79260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 --2- 172.21.15.5:0/683198165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c47f6d5d0 0x555c47f80ad0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49de7fc640 0 client.19026 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:33:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 -- 172.21.15.5:0/683198165 >> 172.21.15.5:0/683198165 conn(0x555c47f36310 msgr2=0x555c47f77090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 -- 172.21.15.5:0/683198165 shutdown_connections 2024-09-11T22:33:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.546+0000 7f49ff543740 1 -- 172.21.15.5:0/683198165 wait complete. 2024-09-11T22:33:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49ff543740 20 client.19026 trim_cache size 0 max 16384 2024-09-11T22:33:39.556 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49fd16a640 1 -- 172.21.15.5:0/419535843 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49e40a2cf0 msgr2=0x7f49e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49fd16a640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49e40a2cf0 0x7f49e40a51b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009fd0 tx=0x7f49f0004ab0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49fd16a640 1 -- 172.21.15.5:0/419535843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f8102d30 msgr2=0x7f49f81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49fd16a640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f8102d30 0x7f49f81985c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f49e800e9f0 tx=0x7f49e800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49fd16a640 1 -- 172.21.15.5:0/419535843 shutdown_connections 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49fd16a640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49f81082a0 0x7f49f819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49fd16a640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49e40a2cf0 0x7f49e40a51b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49fd16a640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49f81036e0 0x7f49f8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49fd16a640 1 --2- 172.21.15.5:0/419535843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f8102d30 0x7f49f81985c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.547+0000 7f49fd16a640 1 -- 172.21.15.5:0/419535843 >> 172.21.15.5:0/419535843 conn(0x7f49f80fc7d0 msgr2=0x7f49f8105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.548+0000 7f49fd16a640 1 -- 172.21.15.5:0/419535843 shutdown_connections 2024-09-11T22:33:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:39.548+0000 7f49fd16a640 1 -- 172.21.15.5:0/419535843 wait complete. 2024-09-11T22:33:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:39 smithi005 ceph-mon[30943]: pgmap v363: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-09-11T22:33:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:39 smithi159 ceph-mon[35534]: pgmap v363: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-09-11T22:33:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:39 smithi172 ceph-mon[35461]: pgmap v363: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-09-11T22:33:40.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 846715 7.16 MB/sec execute 478 sec latency 51.557 ms 2024-09-11T22:33:40.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:33:40.614 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.127+0000 7f70b6f9d640 1 -- 172.21.15.5:0/1615175284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70b0101de0 msgr2=0x7f70b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.127+0000 7f70b6f9d640 1 --2- 172.21.15.5:0/1615175284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70b0101de0 0x7f70b01021c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009ca0 tx=0x7f70a002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.127+0000 7f70b6f9d640 1 -- 172.21.15.5:0/1615175284 shutdown_connections 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.127+0000 7f70b6f9d640 1 --2- 172.21.15.5:0/1615175284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0106f10 0x7f70b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.127+0000 7f70b6f9d640 1 --2- 172.21.15.5:0/1615175284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70b0102790 0x7f70b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.127+0000 7f70b6f9d640 1 --2- 172.21.15.5:0/1615175284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70b0101de0 0x7f70b01021c0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.127+0000 7f70b6f9d640 1 -- 172.21.15.5:0/1615175284 >> 172.21.15.5:0/1615175284 conn(0x7f70b00fd8c0 msgr2=0x7f70b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.127+0000 7f70b6f9d640 1 -- 172.21.15.5:0/1615175284 shutdown_connections 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.127+0000 7f70b6f9d640 1 -- 172.21.15.5:0/1615175284 wait complete. 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b6f9d640 1 Processor -- start 2024-09-11T22:33:41.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b6f9d640 1 -- start start 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b6f9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70b0101de0 0x7f70b01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b6f9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70b0102790 0x7f70b01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b6f9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0106f10 0x7f70b019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b6f9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0111f70 con 0x7f70b0101de0 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b6f9d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f70b0111df0 con 0x7f70b0106f10 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b6f9d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01120f0 con 0x7f70b0102790 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b4d39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70b0101de0 0x7f70b01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b4d39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70b0101de0 0x7f70b01984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60866/0 (socket says 172.21.15.5:60866) 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.128+0000 7f70b4d39640 1 -- 172.21.15.5:0/145656090 learned_addr learned my addr 172.21.15.5:0/145656090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70affff640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70b0102790 0x7f70b01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70b553a640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0106f10 0x7f70b019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70b4d39640 1 -- 172.21.15.5:0/145656090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70b0102790 msgr2=0x7f70b01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70b4d39640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70b0102790 0x7f70b01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70b4d39640 1 -- 172.21.15.5:0/145656090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0106f10 msgr2=0x7f70b019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70b4d39640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0106f10 0x7f70b019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70b4d39640 1 -- 172.21.15.5:0/145656090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b019d460 con 0x7f70b0101de0 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70b4d39640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70b0101de0 0x7f70b01984b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f70a002ade0 tx=0x7f70a00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70adffb640 1 -- 172.21.15.5:0/145656090 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a002fa20 con 0x7f70b0101de0 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70adffb640 1 -- 172.21.15.5:0/145656090 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a002fbc0 con 0x7f70b0101de0 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70b6f9d640 1 -- 172.21.15.5:0/145656090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b019d6f0 con 0x7f70b0101de0 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.129+0000 7f70adffb640 1 -- 172.21.15.5:0/145656090 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a00466e0 con 0x7f70b0101de0 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.130+0000 7f70b6f9d640 1 -- 172.21.15.5:0/145656090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b01a5030 con 0x7f70b0101de0 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.132+0000 7f70adffb640 1 -- 172.21.15.5:0/145656090 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f70a00305c0 con 0x7f70b0101de0 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.132+0000 7f70b9376740 1 Processor -- start 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.132+0000 7f70b9376740 1 -- start start 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70b9376740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fa9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70b9376740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fa9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70b9376740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fb6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70b9376740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5600b3f0f2f0 con 0x5600b3fa95d0 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70b9376740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600b3eb6cf0 con 0x5600b3faab40 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70b9376740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5600b3f0b2b0 con 0x5600b3fb4370 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70adffb640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f709c0a2cf0 0x7f709c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70affff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fb6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70b4d39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fa9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70affff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fb6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46650/0 (socket says 172.21.15.5:46650) 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70affff640 1 -- 172.21.15.5:0/3976477304 learned_addr learned my addr 172.21.15.5:0/3976477304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70b553a640 1 --2- 172.21.15.5:0/3976477304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fa9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70b553a640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f709c0a2cf0 0x7f709c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70affff640 1 -- 172.21.15.5:0/3976477304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 msgr2=0x5600b3fa9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70affff640 1 --2- 172.21.15.5:0/3976477304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fa9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70affff640 1 -- 172.21.15.5:0/3976477304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 msgr2=0x5600b3fa9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70affff640 1 --2- 172.21.15.5:0/3976477304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fa9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70affff640 1 -- 172.21.15.5:0/3976477304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b3e9b3a0 con 0x5600b3fb4370 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70adffb640 1 -- 172.21.15.5:0/145656090 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f70a00f0730 con 0x7f70b0101de0 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70affff640 1 --2- 172.21.15.5:0/3976477304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fb6760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7098009ca0 tx=0x7f7098011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce12536871fdf993 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.133+0000 7f70b553a640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f709c0a2cf0 0x7f709c0a51b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f70a8005f60 tx=0x7f70a8017040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70a6ffd640 1 -- 172.21.15.5:0/3976477304 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709801e070 con 0x5600b3fb4370 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70a6ffd640 1 -- 172.21.15.5:0/3976477304 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7098019b10 con 0x5600b3fb4370 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70a6ffd640 1 -- 172.21.15.5:0/3976477304 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709800eae0 con 0x5600b3fb4370 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70b9376740 1 -- 172.21.15.5:0/3976477304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 msgr2=0x5600b3fb6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70b9376740 1 --2- 172.21.15.5:0/3976477304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fb6760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7098009ca0 tx=0x7f7098011810 comp rx=0 tx=0).stop 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70b9376740 1 -- 172.21.15.5:0/3976477304 shutdown_connections 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70b9376740 1 --2- 172.21.15.5:0/3976477304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fb6760 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70b9376740 1 --2- 172.21.15.5:0/3976477304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fa9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70b9376740 1 --2- 172.21.15.5:0/3976477304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fa9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70b9376740 1 -- 172.21.15.5:0/3976477304 >> 172.21.15.5:0/3976477304 conn(0x5600b3f72310 msgr2=0x5600b3fa5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70b9376740 1 -- 172.21.15.5:0/3976477304 shutdown_connections 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.134+0000 7f70b9376740 1 -- 172.21.15.5:0/3976477304 wait complete. 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.135+0000 7f70b9376740 1 -- start start 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.135+0000 7f70b9376740 1 Processor -- start 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.135+0000 7f70b9376740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fb3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.135+0000 7f70b9376740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fb1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.135+0000 7f70b9376740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fb1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.135+0000 7f70b9376740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5600b3fb7070 con 0x5600b3fa95d0 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.135+0000 7f70b9376740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600b3eca060 con 0x5600b3faab40 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.135+0000 7f70b9376740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5600b3fb71f0 con 0x5600b3fb4370 2024-09-11T22:33:41.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.135+0000 7f70b9376740 10 client.0 _ll_register_callbacks cb 0x5600b3c1d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b4d39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fb3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70affff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fb1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b4d39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fb3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60882/0 (socket says 172.21.15.5:60882) 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b4d39640 1 -- 172.21.15.5:0/3055639944 learned_addr learned my addr 172.21.15.5:0/3055639944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b553a640 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fb1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b4d39640 1 -- 172.21.15.5:0/3055639944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 msgr2=0x5600b3fb1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b4d39640 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fb1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b4d39640 1 -- 172.21.15.5:0/3055639944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 msgr2=0x5600b3fb1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b4d39640 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fb1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b4d39640 1 -- 172.21.15.5:0/3055639944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b3c803c0 con 0x5600b3fa95d0 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70affff640 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fb1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b4d39640 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fb3d70 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f70a0031f80 tx=0x7f70a00099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c8b3eafe17fd40 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f708ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:41.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f708ffff640 1 -- 172.21.15.5:0/3055639944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a00bd3c0 con 0x5600b3fa95d0 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f708ffff640 1 -- 172.21.15.5:0/3055639944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f70a00ee060 con 0x5600b3fa95d0 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b9376740 1 -- 172.21.15.5:0/3055639944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7098019b10 con 0x5600b3fa95d0 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f708ffff640 1 -- 172.21.15.5:0/3055639944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f70a00ee260 con 0x5600b3fa95d0 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b9376740 10 client.19038 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.136+0000 7f70b9376740 1 -- 172.21.15.5:0/3055639944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5600b3fb7070 con 0x5600b3fa95d0 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f708ffff640 1 -- 172.21.15.5:0/3055639944 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7098019b10 con 0x5600b3fa95d0 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f708ffff640 1 -- 172.21.15.5:0/3055639944 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f70a004e030 con 0x5600b3fa95d0 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f70b9376740 10 client.19038 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f70b9376740 10 client.19038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f70b9376740 20 client.19038 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f70b9376740 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600b3fc3e40 0x5600b3fc62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f70b9376740 4 client.19038 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:41.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f70b9376740 1 -- 172.21.15.5:0/3055639944 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600b3f1c860 con 0x5600b3fc3e40 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f70affff640 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600b3fc3e40 0x5600b3fc62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f70affff640 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600b3fc3e40 0x5600b3fc62b0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10f62d126039d1e6 server_cookie=5d6d49066df6d5cf in_seq=0 out_seq=0 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.137+0000 7f708ffff640 10 client.19038 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f708ffff640 1 -- 172.21.15.5:0/3055639944 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600b3f1c860 con 0x5600b3fc3e40 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f708ffff640 10 client.19038 handle_command_reply: tid=0 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f70b9376740 1 client.19038 shutdown 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f70b9376740 1 -- 172.21.15.5:0/3055639944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 msgr2=0x5600b3fb3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f70b9376740 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fb3d70 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f70a0031f80 tx=0x7f70a00099c0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f70b9376740 1 -- 172.21.15.5:0/3055639944 shutdown_connections 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f70b9376740 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600b3fc3e40 0x5600b3fc62b0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f70b9376740 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fb1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f70b9376740 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fb1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f70b9376740 1 --2- 172.21.15.5:0/3055639944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fb3d70 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.139+0000 7f70b9376740 1 -- 172.21.15.5:0/3055639944 >> 172.21.15.5:0/3055639944 conn(0x5600b3f72310 msgr2=0x5600b3fab960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.140+0000 7f708ffff640 0 client.19038 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.140+0000 7f70b9376740 1 -- 172.21.15.5:0/3055639944 shutdown_connections 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.140+0000 7f70b9376740 1 -- 172.21.15.5:0/3055639944 wait complete. 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.140+0000 7f70b9376740 20 client.19038 trim_cache size 0 max 16384 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.153+0000 7f70b9376740 1 Processor -- start 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.153+0000 7f70b9376740 1 -- start start 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.153+0000 7f70b9376740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fc0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.153+0000 7f70b9376740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.153+0000 7f70b9376740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.153+0000 7f70b9376740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01ab700 con 0x5600b3fa95d0 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.153+0000 7f70b9376740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01ad240 con 0x5600b3faab40 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.153+0000 7f70b9376740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01abdd0 con 0x5600b3fb4370 2024-09-11T22:33:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b4d39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fc0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b553a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b4d39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fc0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60890/0 (socket says 172.21.15.5:60890) 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b4d39640 1 -- 172.21.15.5:0/2809870970 learned_addr learned my addr 172.21.15.5:0/2809870970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70affff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b4d39640 1 -- 172.21.15.5:0/2809870970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 msgr2=0x5600b3fac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b4d39640 1 --2- 172.21.15.5:0/2809870970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b4d39640 1 -- 172.21.15.5:0/2809870970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 msgr2=0x5600b3fac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b4d39640 1 --2- 172.21.15.5:0/2809870970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b4d39640 1 -- 172.21.15.5:0/2809870970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600b3faf570 con 0x5600b3fa95d0 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b553a640 1 --2- 172.21.15.5:0/2809870970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70b4d39640 1 --2- 172.21.15.5:0/2809870970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fc0310 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f70a00b9690 tx=0x7f70a0030c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d5cb40c0d1cc86 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70a67fc640 1 -- 172.21.15.5:0/2809870970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a00bc620 con 0x5600b3fa95d0 2024-09-11T22:33:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70a67fc640 1 -- 172.21.15.5:0/2809870970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a00a0070 con 0x5600b3fa95d0 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.154+0000 7f70a67fc640 1 -- 172.21.15.5:0/2809870970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a00bc110 con 0x5600b3fa95d0 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 -- 172.21.15.5:0/2809870970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 msgr2=0x5600b3fc0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 --2- 172.21.15.5:0/2809870970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fc0310 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f70a00b9690 tx=0x7f70a0030c80 comp rx=0 tx=0).stop 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 -- 172.21.15.5:0/2809870970 shutdown_connections 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 --2- 172.21.15.5:0/2809870970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fb4370 0x5600b3fac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 --2- 172.21.15.5:0/2809870970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 --2- 172.21.15.5:0/2809870970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fa95d0 0x5600b3fc0310 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 -- 172.21.15.5:0/2809870970 >> 172.21.15.5:0/2809870970 conn(0x5600b3f72310 msgr2=0x5600b3fb0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 -- 172.21.15.5:0/2809870970 shutdown_connections 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 -- 172.21.15.5:0/2809870970 wait complete. 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 -- start start 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.155+0000 7f70b9376740 1 Processor -- start 2024-09-11T22:33:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b9376740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fa95d0 0x5600b3fbcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b9376740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fb7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b9376740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fb4370 0x5600b3fb8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b9376740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01acb70 con 0x5600b3fb4370 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b9376740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01ac4a0 con 0x5600b3faab40 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b9376740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70b01a96e0 con 0x5600b3fa95d0 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b9376740 10 client.0 _ll_register_callbacks cb 0x7f70a0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b553a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fb4370 0x5600b3fb8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b4d39640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fa95d0 0x5600b3fbcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b553a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fb4370 0x5600b3fb8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60894/0 (socket says 172.21.15.5:60894) 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b4d39640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fa95d0 0x5600b3fbcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46684/0 (socket says 172.21.15.5:46684) 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b553a640 1 -- 172.21.15.5:0/1231046099 learned_addr learned my addr 172.21.15.5:0/1231046099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70affff640 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fb7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b553a640 1 -- 172.21.15.5:0/1231046099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fa95d0 msgr2=0x5600b3fbcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b553a640 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fa95d0 0x5600b3fbcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b553a640 1 -- 172.21.15.5:0/1231046099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 msgr2=0x5600b3fb7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b553a640 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fb7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b553a640 1 -- 172.21.15.5:0/1231046099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600b3fb8a00 con 0x5600b3fb4370 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b4d39640 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fa95d0 0x5600b3fbcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f70b553a640 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fb4370 0x5600b3fb8310 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f70a800fba0 tx=0x7f70a8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4ff85a053696fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.156+0000 7f708ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f708ffff640 1 -- 172.21.15.5:0/1231046099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8033060 con 0x5600b3fb4370 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f708ffff640 1 -- 172.21.15.5:0/1231046099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f70a8003b70 con 0x5600b3fb4370 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f70b9376740 1 -- 172.21.15.5:0/1231046099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a00a0070 con 0x5600b3fb4370 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f708ffff640 1 -- 172.21.15.5:0/1231046099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f70a8003d70 con 0x5600b3fb4370 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f70b9376740 10 client.19044 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f70b9376740 1 -- 172.21.15.5:0/1231046099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5600b402ea10 con 0x5600b3fb4370 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f708ffff640 1 -- 172.21.15.5:0/1231046099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a00a0070 con 0x5600b3fb4370 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f708ffff640 1 -- 172.21.15.5:0/1231046099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f70a8023530 con 0x5600b3fb4370 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f70b9376740 10 client.19044 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f70b9376740 10 client.19044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f70b9376740 20 client.19044 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f70b9376740 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600b402f190 0x5600b40315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f70b9376740 4 client.19044 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f70b9376740 1 -- 172.21.15.5:0/1231046099 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600b3f16d70 con 0x5600b402f190 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.157+0000 7f70b4d39640 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600b402f190 0x5600b40315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.158+0000 7f70b4d39640 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600b402f190 0x5600b40315b0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc61ee5fd1500ad server_cookie=4a602560b2ec8248 in_seq=0 out_seq=0 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.158+0000 7f708ffff640 10 client.19044 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.158+0000 7f708ffff640 1 -- 172.21.15.5:0/1231046099 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600b3f16d70 con 0x5600b402f190 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.158+0000 7f708ffff640 10 client.19044 handle_command_reply: tid=0 2024-09-11T22:33:41.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 client.19044 shutdown 2024-09-11T22:33:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 -- 172.21.15.5:0/1231046099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fb4370 msgr2=0x5600b3fb8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:41.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fb4370 0x5600b3fb8310 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f70a800fba0 tx=0x7f70a8005f90 comp rx=0 tx=0).stop 2024-09-11T22:33:41.177 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 -- 172.21.15.5:0/1231046099 shutdown_connections 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600b402f190 0x5600b40315b0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600b3fb4370 0x5600b3fb8310 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f708ffff640 0 client.19044 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600b3faab40 0x5600b3fb7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 --2- 172.21.15.5:0/1231046099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600b3fa95d0 0x5600b3fbcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 -- 172.21.15.5:0/1231046099 >> 172.21.15.5:0/1231046099 conn(0x5600b3f72310 msgr2=0x5600b3fb2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 -- 172.21.15.5:0/1231046099 shutdown_connections 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 1 -- 172.21.15.5:0/1231046099 wait complete. 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.159+0000 7f70b9376740 20 client.19044 trim_cache size 0 max 16384 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 -- 172.21.15.5:0/145656090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f709c0a2cf0 msgr2=0x7f709c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f709c0a2cf0 0x7f709c0a51b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f70a8005f60 tx=0x7f70a8017040 comp rx=0 tx=0).stop 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 -- 172.21.15.5:0/145656090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70b0101de0 msgr2=0x7f70b01984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:41.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70b0101de0 0x7f70b01984b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f70a002ade0 tx=0x7f70a00319f0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 -- 172.21.15.5:0/145656090 shutdown_connections 2024-09-11T22:33:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f709c0a2cf0 0x7f709c0a51b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70b0106f10 0x7f70b019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70b0102790 0x7f70b01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 --2- 172.21.15.5:0/145656090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70b0101de0 0x7f70b01984b0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 -- 172.21.15.5:0/145656090 >> 172.21.15.5:0/145656090 conn(0x7f70b00fd8c0 msgr2=0x7f70b00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 -- 172.21.15.5:0/145656090 shutdown_connections 2024-09-11T22:33:41.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:41.160+0000 7f70b6f9d640 1 -- 172.21.15.5:0/145656090 wait complete. 2024-09-11T22:33:41.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 848671 7.17 MB/sec execute 479 sec latency 42.376 ms 2024-09-11T22:33:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:41 smithi005 ceph-mon[30943]: pgmap v364: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-09-11T22:33:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:41 smithi159 ceph-mon[35534]: pgmap v364: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-09-11T22:33:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:41 smithi172 ceph-mon[35461]: pgmap v364: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-09-11T22:33:42.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 850082 7.17 MB/sec execute 480 sec latency 42.755 ms 2024-09-11T22:33:42.961 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:33:42.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:33:43.227 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:43.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 852186 7.18 MB/sec execute 481 sec latency 27.155 ms 2024-09-11T22:33:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.751+0000 7f8986627640 1 -- 172.21.15.5:0/3243693690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8980106f10 msgr2=0x7f898010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.751+0000 7f8986627640 1 --2- 172.21.15.5:0/3243693690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8980106f10 0x7f898010ea90 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8978009ca0 tx=0x7f897802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.751+0000 7f8986627640 1 -- 172.21.15.5:0/3243693690 shutdown_connections 2024-09-11T22:33:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.751+0000 7f8986627640 1 --2- 172.21.15.5:0/3243693690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8980106f10 0x7f898010ea90 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.751+0000 7f8986627640 1 --2- 172.21.15.5:0/3243693690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8980102790 0x7f89801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.751+0000 7f8986627640 1 --2- 172.21.15.5:0/3243693690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8980101de0 0x7f89801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.751+0000 7f8986627640 1 -- 172.21.15.5:0/3243693690 >> 172.21.15.5:0/3243693690 conn(0x7f89800fd8c0 msgr2=0x7f89800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.751+0000 7f8986627640 1 -- 172.21.15.5:0/3243693690 shutdown_connections 2024-09-11T22:33:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.751+0000 7f8986627640 1 -- 172.21.15.5:0/3243693690 wait complete. 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.752+0000 7f8986627640 1 Processor -- start 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.752+0000 7f8986627640 1 -- start start 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.752+0000 7f8986627640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8980101de0 0x7f89801983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.752+0000 7f8986627640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8980102790 0x7f8980198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.752+0000 7f8986627640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8980106f10 0x7f898019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f8986627640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8980111f70 con 0x7f8980101de0 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f8986627640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8980111df0 con 0x7f8980106f10 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f8986627640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f89801120f0 con 0x7f8980102790 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8980101de0 0x7f89801983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8980102790 0x7f8980198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8980101de0 0x7f89801983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56728/0 (socket says 172.21.15.5:56728) 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8980102790 0x7f8980198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:35328/0 (socket says 172.21.15.5:35328) 2024-09-11T22:33:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897ffff640 1 -- 172.21.15.5:0/2506222414 learned_addr learned my addr 172.21.15.5:0/2506222414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f8984bc4640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8980106f10 0x7f898019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897f7fe640 1 -- 172.21.15.5:0/2506222414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8980106f10 msgr2=0x7f898019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897f7fe640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8980106f10 0x7f898019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897f7fe640 1 -- 172.21.15.5:0/2506222414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8980101de0 msgr2=0x7f89801983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897f7fe640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8980101de0 0x7f89801983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897f7fe640 1 -- 172.21.15.5:0/2506222414 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898019d3a0 con 0x7f8980102790 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897ffff640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8980101de0 0x7f89801983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.753+0000 7f897f7fe640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8980102790 0x7f8980198930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f896800e9f0 tx=0x7f896800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.754+0000 7f897d7fa640 1 -- 172.21.15.5:0/2506222414 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896800ccc0 con 0x7f8980102790 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.754+0000 7f8986627640 1 -- 172.21.15.5:0/2506222414 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898019d690 con 0x7f8980102790 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.754+0000 7f897d7fa640 1 -- 172.21.15.5:0/2506222414 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8968005020 con 0x7f8980102790 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.754+0000 7f897d7fa640 1 -- 172.21.15.5:0/2506222414 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89680049e0 con 0x7f8980102790 2024-09-11T22:33:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.754+0000 7f8986627640 1 -- 172.21.15.5:0/2506222414 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89801a4f70 con 0x7f8980102790 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.756+0000 7f897d7fa640 1 -- 172.21.15.5:0/2506222414 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8968005560 con 0x7f8980102790 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.756+0000 7f8988a00740 1 Processor -- start 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.756+0000 7f8988a00740 1 -- start start 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8988a00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f375b60 0x55987f375f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8988a00740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f376480 0x55987f380800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8988a00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f380d40 0x55987f383130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8988a00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55987f2db280 con 0x55987f375b60 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8988a00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55987f282b10 con 0x55987f380d40 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8988a00740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55987f2d7240 con 0x55987f376480 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8984bc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f375b60 0x55987f375f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8984bc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f375b60 0x55987f375f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56730/0 (socket says 172.21.15.5:56730) 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8984bc4640 1 -- 172.21.15.5:0/2778016352 learned_addr learned my addr 172.21.15.5:0/2778016352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f897d7fa640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f896c0a2d70 0x7f896c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f897f7fe640 1 --2- 172.21.15.5:0/2778016352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f380d40 0x55987f383130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f897ffff640 1 --2- 172.21.15.5:0/2778016352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f376480 0x55987f380800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8984bc4640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f896c0a2d70 0x7f896c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f897ffff640 1 -- 172.21.15.5:0/2778016352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f380d40 msgr2=0x55987f383130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f897ffff640 1 --2- 172.21.15.5:0/2778016352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f380d40 0x55987f383130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f897ffff640 1 -- 172.21.15.5:0/2778016352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f375b60 msgr2=0x55987f375f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f897ffff640 1 --2- 172.21.15.5:0/2778016352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f375b60 0x55987f375f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f897d7fa640 1 -- 172.21.15.5:0/2506222414 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f89680cf360 con 0x7f8980102790 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f897ffff640 1 -- 172.21.15.5:0/2778016352 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987f28f1f0 con 0x55987f376480 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f8984bc4640 1 --2- 172.21.15.5:0/2778016352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f375b60 0x55987f375f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.757+0000 7f897ffff640 1 --2- 172.21.15.5:0/2778016352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f376480 0x55987f380800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f89700114f0 tx=0x7f89700119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f4e9b35a62695e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f8984bc4640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f896c0a2d70 0x7f896c0a5230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8978004af0 tx=0x7f897802a8c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f89767fc640 1 -- 172.21.15.5:0/2778016352 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897001e070 con 0x55987f376480 2024-09-11T22:33:43.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f89767fc640 1 -- 172.21.15.5:0/2778016352 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89700035f0 con 0x55987f376480 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f89767fc640 1 -- 172.21.15.5:0/2778016352 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897000f820 con 0x55987f376480 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f8988a00740 1 -- 172.21.15.5:0/2778016352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f376480 msgr2=0x55987f380800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f8988a00740 1 --2- 172.21.15.5:0/2778016352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f376480 0x55987f380800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f89700114f0 tx=0x7f89700119b0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f8988a00740 1 -- 172.21.15.5:0/2778016352 shutdown_connections 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f8988a00740 1 --2- 172.21.15.5:0/2778016352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f380d40 0x55987f383130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f8988a00740 1 --2- 172.21.15.5:0/2778016352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f376480 0x55987f380800 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f8988a00740 1 --2- 172.21.15.5:0/2778016352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f375b60 0x55987f375f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f8988a00740 1 -- 172.21.15.5:0/2778016352 >> 172.21.15.5:0/2778016352 conn(0x55987f3720f0 msgr2=0x55987f3724d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f8988a00740 1 -- 172.21.15.5:0/2778016352 shutdown_connections 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.758+0000 7f8988a00740 1 -- 172.21.15.5:0/2778016352 wait complete. 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f8988a00740 1 -- start start 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f8988a00740 1 Processor -- start 2024-09-11T22:33:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f8988a00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f375b60 0x55987f37dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f8988a00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f37e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f8988a00740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f380d40 0x55987f37e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f8988a00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55987f383910 con 0x55987f376480 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f8988a00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55987f295f10 con 0x55987f375b60 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f8988a00740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55987f383a90 con 0x55987f380d40 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f8988a00740 10 client.0 _ll_register_callbacks cb 0x55987f25ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f897f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f37e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f897f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f37e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56736/0 (socket says 172.21.15.5:56736) 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f8984bc4640 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f380d40 0x55987f37e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.759+0000 7f897f7fe640 1 -- 172.21.15.5:0/3460673333 learned_addr learned my addr 172.21.15.5:0/3460673333 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f897ffff640 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f375b60 0x55987f37dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f897f7fe640 1 -- 172.21.15.5:0/3460673333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f380d40 msgr2=0x55987f37e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f897f7fe640 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f380d40 0x55987f37e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f897f7fe640 1 -- 172.21.15.5:0/3460673333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f375b60 msgr2=0x55987f37dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f897f7fe640 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f375b60 0x55987f37dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f897f7fe640 1 -- 172.21.15.5:0/3460673333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987f04c6a0 con 0x55987f376480 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f897ffff640 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f375b60 0x55987f37dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f8984bc4640 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f380d40 0x55987f37e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f897f7fe640 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f37e100 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f896800c6a0 tx=0x7f896809b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7969ae7cacd82beb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f895f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f895f7fe640 1 -- 172.21.15.5:0/3460673333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896801c040 con 0x55987f376480 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f895f7fe640 1 -- 172.21.15.5:0/3460673333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f89680ccc90 con 0x55987f376480 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f8988a00740 1 -- 172.21.15.5:0/3460673333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89700035f0 con 0x55987f376480 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f895f7fe640 1 -- 172.21.15.5:0/3460673333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f89680cce90 con 0x55987f376480 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f8988a00740 10 client.19056 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.760+0000 7f8988a00740 1 -- 172.21.15.5:0/3460673333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55987f38f4c0 con 0x55987f376480 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.761+0000 7f895f7fe640 1 -- 172.21.15.5:0/3460673333 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89700035f0 con 0x55987f376480 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.761+0000 7f895f7fe640 1 -- 172.21.15.5:0/3460673333 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f89680a3fc0 con 0x55987f376480 2024-09-11T22:33:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.761+0000 7f8988a00740 10 client.19056 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.761+0000 7f8988a00740 10 client.19056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.761+0000 7f8988a00740 20 client.19056 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.761+0000 7f8988a00740 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55987f390860 0x55987f392c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.761+0000 7f8988a00740 4 client.19056 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.761+0000 7f8988a00740 1 -- 172.21.15.5:0/3460673333 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55987f2e87f0 con 0x55987f390860 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.761+0000 7f897ffff640 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55987f390860 0x55987f392c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.762+0000 7f897ffff640 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55987f390860 0x55987f392c70 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4bf8ee88583b36c server_cookie=7906dccd7efac04a in_seq=0 out_seq=0 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.762+0000 7f895f7fe640 10 client.19056 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f895f7fe640 1 -- 172.21.15.5:0/3460673333 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55987f2e87f0 con 0x55987f390860 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f895f7fe640 10 client.19056 handle_command_reply: tid=0 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f8988a00740 1 client.19056 shutdown 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f8988a00740 1 -- 172.21.15.5:0/3460673333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 msgr2=0x55987f37e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f8988a00740 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f37e100 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f896800c6a0 tx=0x7f896809b140 comp rx=0 tx=0).stop 2024-09-11T22:33:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f8988a00740 1 -- 172.21.15.5:0/3460673333 shutdown_connections 2024-09-11T22:33:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f8988a00740 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55987f390860 0x55987f392c70 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f8988a00740 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f380d40 0x55987f37e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f8988a00740 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f37e100 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f895f7fe640 0 client.19056 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f8988a00740 1 --2- 172.21.15.5:0/3460673333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f375b60 0x55987f37dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.763+0000 7f8988a00740 1 -- 172.21.15.5:0/3460673333 >> 172.21.15.5:0/3460673333 conn(0x55987f3720f0 msgr2=0x55987f377e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.764+0000 7f8988a00740 1 -- 172.21.15.5:0/3460673333 shutdown_connections 2024-09-11T22:33:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.764+0000 7f8988a00740 1 -- 172.21.15.5:0/3460673333 wait complete. 2024-09-11T22:33:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.764+0000 7f8988a00740 20 client.19056 trim_cache size 0 max 16384 2024-09-11T22:33:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.781+0000 7f8988a00740 1 Processor -- start 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.781+0000 7f8988a00740 1 -- start start 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.781+0000 7f8988a00740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 0x55987f377d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.781+0000 7f8988a00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f398540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f8988a00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f380d40 0x55987f398a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f8988a00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89801ada80 con 0x55987f376480 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f8988a00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f89801af5c0 con 0x55987f380d40 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f8988a00740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f89801ae150 con 0x55987f375b60 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f897f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f398540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f897ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 0x55987f377d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f897f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f398540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56742/0 (socket says 172.21.15.5:56742) 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f897f7fe640 1 -- 172.21.15.5:0/1028651741 learned_addr learned my addr 172.21.15.5:0/1028651741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f8984bc4640 1 --2- 172.21.15.5:0/1028651741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f380d40 0x55987f398a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f897f7fe640 1 -- 172.21.15.5:0/1028651741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 msgr2=0x55987f377d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f897f7fe640 1 --2- 172.21.15.5:0/1028651741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 0x55987f377d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f897f7fe640 1 -- 172.21.15.5:0/1028651741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f380d40 msgr2=0x55987f398a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f897f7fe640 1 --2- 172.21.15.5:0/1028651741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f380d40 0x55987f398a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f897f7fe640 1 -- 172.21.15.5:0/1028651741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987f371f20 con 0x55987f376480 2024-09-11T22:33:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f897f7fe640 1 --2- 172.21.15.5:0/1028651741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f398540 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f89680a3f20 tx=0x7f89680980d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8cf27e36d4ec39d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f8975ffb640 1 -- 172.21.15.5:0/1028651741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89680a4450 con 0x55987f376480 2024-09-11T22:33:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f8975ffb640 1 -- 172.21.15.5:0/1028651741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f896809b7e0 con 0x55987f376480 2024-09-11T22:33:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.782+0000 7f8975ffb640 1 -- 172.21.15.5:0/1028651741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896809ac80 con 0x55987f376480 2024-09-11T22:33:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.783+0000 7f8988a00740 1 -- 172.21.15.5:0/1028651741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 msgr2=0x55987f398540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.783+0000 7f8988a00740 1 --2- 172.21.15.5:0/1028651741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f398540 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f89680a3f20 tx=0x7f89680980d0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.783+0000 7f8988a00740 1 -- 172.21.15.5:0/1028651741 shutdown_connections 2024-09-11T22:33:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.783+0000 7f8988a00740 1 --2- 172.21.15.5:0/1028651741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f380d40 0x55987f398a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.783+0000 7f8988a00740 1 --2- 172.21.15.5:0/1028651741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f376480 0x55987f398540 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.783+0000 7f8988a00740 1 --2- 172.21.15.5:0/1028651741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 0x55987f377d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.783+0000 7f8988a00740 1 -- 172.21.15.5:0/1028651741 >> 172.21.15.5:0/1028651741 conn(0x55987f3720f0 msgr2=0x55987f377600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.783+0000 7f8988a00740 1 -- 172.21.15.5:0/1028651741 shutdown_connections 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.783+0000 7f8988a00740 1 -- 172.21.15.5:0/1028651741 wait complete. 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8988a00740 1 -- start start 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8988a00740 1 Processor -- start 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8988a00740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 0x55987f388ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8988a00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f376480 0x55987f389220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8988a00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f380d40 0x55987f385cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8988a00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89801aeef0 con 0x55987f380d40 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8988a00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f89801ae820 con 0x55987f376480 2024-09-11T22:33:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8988a00740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f89801aba60 con 0x55987f375b60 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8988a00740 10 client.0 _ll_register_callbacks cb 0x7f8968004b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8984bc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f380d40 0x55987f385cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8984bc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f380d40 0x55987f385cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56744/0 (socket says 172.21.15.5:56744) 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f8984bc4640 1 -- 172.21.15.5:0/3497678880 learned_addr learned my addr 172.21.15.5:0/3497678880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f897f7fe640 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f376480 0x55987f389220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.784+0000 7f897ffff640 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 0x55987f388ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f8984bc4640 1 -- 172.21.15.5:0/3497678880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 msgr2=0x55987f388ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f8984bc4640 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 0x55987f388ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f8984bc4640 1 -- 172.21.15.5:0/3497678880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f376480 msgr2=0x55987f389220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f8984bc4640 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f376480 0x55987f389220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f8984bc4640 1 -- 172.21.15.5:0/3497678880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987f3861f0 con 0x55987f380d40 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f897f7fe640 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f376480 0x55987f389220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f897ffff640 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 0x55987f388ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f8984bc4640 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f380d40 0x55987f385cb0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8978035000 tx=0x7f89780095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb971f610ec76f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f895f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f895f7fe640 1 -- 172.21.15.5:0/3497678880 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8978030d40 con 0x55987f380d40 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f895f7fe640 1 -- 172.21.15.5:0/3497678880 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7f897804d440 con 0x55987f380d40 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f8988a00740 1 -- 172.21.15.5:0/3497678880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f896809b7e0 con 0x55987f380d40 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f895f7fe640 1 -- 172.21.15.5:0/3497678880 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f897804d640 con 0x55987f380d40 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f8988a00740 10 client.19062 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.785+0000 7f8988a00740 1 -- 172.21.15.5:0/3497678880 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55987f3fbd00 con 0x55987f380d40 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f895f7fe640 1 -- 172.21.15.5:0/3497678880 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f896809b7e0 con 0x55987f380d40 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f895f7fe640 1 -- 172.21.15.5:0/3497678880 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f897804a040 con 0x55987f380d40 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f8988a00740 10 client.19062 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f8988a00740 10 client.19062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f8988a00740 20 client.19062 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f8988a00740 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55987f3fc500 0x55987f3fe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:43.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f8988a00740 4 client.19062 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:33:43.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f8988a00740 1 -- 172.21.15.5:0/3497678880 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55987f2e2d00 con 0x55987f3fc500 2024-09-11T22:33:43.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f897ffff640 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55987f3fc500 0x55987f3fe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:43.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f897ffff640 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55987f3fc500 0x55987f3fe920 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e5568e7795f2a2f server_cookie=a74fbd25aad99626 in_seq=0 out_seq=0 2024-09-11T22:33:43.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.786+0000 7f895f7fe640 10 client.19062 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.787+0000 7f895f7fe640 1 -- 172.21.15.5:0/3497678880 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55987f2e2d00 con 0x55987f3fc500 2024-09-11T22:33:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.787+0000 7f895f7fe640 10 client.19062 handle_command_reply: tid=0 2024-09-11T22:33:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.787+0000 7f8988a00740 1 client.19062 shutdown 2024-09-11T22:33:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 1 -- 172.21.15.5:0/3497678880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f380d40 msgr2=0x55987f385cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f380d40 0x55987f385cb0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8978035000 tx=0x7f89780095c0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 1 -- 172.21.15.5:0/3497678880 shutdown_connections 2024-09-11T22:33:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55987f3fc500 0x55987f3fe920 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987f380d40 0x55987f385cb0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55987f376480 0x55987f389220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f895f7fe640 0 client.19062 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:43.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 1 --2- 172.21.15.5:0/3497678880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55987f375b60 0x55987f388ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 1 -- 172.21.15.5:0/3497678880 >> 172.21.15.5:0/3497678880 conn(0x55987f3720f0 msgr2=0x55987f371270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:43.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 1 -- 172.21.15.5:0/3497678880 shutdown_connections 2024-09-11T22:33:43.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 1 -- 172.21.15.5:0/3497678880 wait complete. 2024-09-11T22:33:43.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8988a00740 20 client.19062 trim_cache size 0 max 16384 2024-09-11T22:33:43.793 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:33:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:33:43.793 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "7f65598c-6263-4f1b-ac8b-c9350044ad61", 2024-09-11T22:33:43.794 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:33:43.794 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:33:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8986627640 1 -- 172.21.15.5:0/2506222414 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f896c0a2d70 msgr2=0x7f896c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8986627640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f896c0a2d70 0x7f896c0a5230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8978004af0 tx=0x7f897802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8986627640 1 -- 172.21.15.5:0/2506222414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8980102790 msgr2=0x7f8980198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.788+0000 7f8986627640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8980102790 0x7f8980198930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f896800e9f0 tx=0x7f896800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.789+0000 7f8986627640 1 -- 172.21.15.5:0/2506222414 shutdown_connections 2024-09-11T22:33:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.789+0000 7f8986627640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f896c0a2d70 0x7f896c0a5230 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.789+0000 7f8986627640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8980106f10 0x7f898019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.789+0000 7f8986627640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8980102790 0x7f8980198930 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.789+0000 7f8986627640 1 --2- 172.21.15.5:0/2506222414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8980101de0 0x7f89801983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.789+0000 7f8986627640 1 -- 172.21.15.5:0/2506222414 >> 172.21.15.5:0/2506222414 conn(0x7f89800fd8c0 msgr2=0x7f89800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.789+0000 7f8986627640 1 -- 172.21.15.5:0/2506222414 shutdown_connections 2024-09-11T22:33:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:43.789+0000 7f8986627640 1 -- 172.21.15.5:0/2506222414 wait complete. 2024-09-11T22:33:44.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:43 smithi005 ceph-mon[30943]: pgmap v365: 97 pgs: 97 active+clean; 111 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-11T22:33:44.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:43 smithi159 ceph-mon[35534]: pgmap v365: 97 pgs: 97 active+clean; 111 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-11T22:33:44.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:43 smithi172 ceph-mon[35461]: pgmap v365: 97 pgs: 97 active+clean; 111 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 178 op/s 2024-09-11T22:33:44.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 853564 7.18 MB/sec execute 482 sec latency 37.254 ms 2024-09-11T22:33:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 -- 172.21.15.5:0/894346014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b0107b70 msgr2=0x7fa4b0107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 --2- 172.21.15.5:0/894346014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b0107b70 0x7fa4b0107f50 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009930 tx=0x7fa4ac02a420 comp rx=0 tx=0).stop 2024-09-11T22:33:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 -- 172.21.15.5:0/894346014 shutdown_connections 2024-09-11T22:33:44.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 --2- 172.21.15.5:0/894346014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4b0102470 0x7fa4b010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 --2- 172.21.15.5:0/894346014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0101ad0 0x7fa4b0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 --2- 172.21.15.5:0/894346014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b0107b70 0x7fa4b0107f50 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 -- 172.21.15.5:0/894346014 >> 172.21.15.5:0/894346014 conn(0x7fa4b00fd7d0 msgr2=0x7fa4b00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 -- 172.21.15.5:0/894346014 shutdown_connections 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 -- 172.21.15.5:0/894346014 wait complete. 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 Processor -- start 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.466+0000 7fa4b82e6640 1 -- start start 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b82e6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0101ad0 0x7fa4b0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b82e6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4b0102470 0x7fa4b01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b82e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b0107b70 0x7fa4b019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b82e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0111f20 con 0x7fa4b0107b70 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b82e6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b0111da0 con 0x7fa4b0102470 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b82e6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01120a0 con 0x7fa4b0101ad0 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6883640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b0107b70 0x7fa4b019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6082640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0101ad0 0x7fa4b0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6883640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b0107b70 0x7fa4b019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56766/0 (socket says 172.21.15.5:56766) 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6883640 1 -- 172.21.15.5:0/1039036746 learned_addr learned my addr 172.21.15.5:0/1039036746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b5881640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4b0102470 0x7fa4b01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6883640 1 -- 172.21.15.5:0/1039036746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0101ad0 msgr2=0x7fa4b0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6883640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0101ad0 0x7fa4b0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6883640 1 -- 172.21.15.5:0/1039036746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4b0102470 msgr2=0x7fa4b01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6883640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4b0102470 0x7fa4b01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6883640 1 -- 172.21.15.5:0/1039036746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b019d380 con 0x7fa4b0107b70 2024-09-11T22:33:44.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6082640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0101ad0 0x7fa4b0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b6883640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b0107b70 0x7fa4b019cc50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800eb10 tx=0x7fa4a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4a77fe640 1 -- 172.21.15.5:0/1039036746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8004b20 con 0x7fa4b0107b70 2024-09-11T22:33:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4a77fe640 1 -- 172.21.15.5:0/1039036746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a8004cc0 con 0x7fa4b0107b70 2024-09-11T22:33:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4b82e6640 1 -- 172.21.15.5:0/1039036746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b019d670 con 0x7fa4b0107b70 2024-09-11T22:33:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.467+0000 7fa4a77fe640 1 -- 172.21.15.5:0/1039036746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8010760 con 0x7fa4b0107b70 2024-09-11T22:33:44.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.468+0000 7fa4b82e6640 1 -- 172.21.15.5:0/1039036746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b01a4f00 con 0x7fa4b0107b70 2024-09-11T22:33:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4a77fe640 1 -- 172.21.15.5:0/1039036746 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa4a8005020 con 0x7fa4b0107b70 2024-09-11T22:33:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4bb07f740 1 Processor -- start 2024-09-11T22:33:44.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4bb07f740 1 -- start start 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 0x55a38d7eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d7ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a38d7522f0 con 0x55a38d7fd0c0 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a38d6fa270 con 0x55a38d7ee2f0 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a38d74e2b0 con 0x55a38d7ed9d0 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4b6082640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4b5881640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d7ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4b6082640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52460/0 (socket says 172.21.15.5:52460) 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.471+0000 7fa4b6082640 1 -- 172.21.15.5:0/3295629441 learned_addr learned my addr 172.21.15.5:0/3295629441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4b5881640 1 --2- 172.21.15.5:0/3295629441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 0x55a38d7eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4b5881640 1 -- 172.21.15.5:0/3295629441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 msgr2=0x55a38d7eddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:44.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4b5881640 1 --2- 172.21.15.5:0/3295629441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 0x55a38d7eddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4b5881640 1 -- 172.21.15.5:0/3295629441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 msgr2=0x55a38d7fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4b5881640 1 --2- 172.21.15.5:0/3295629441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fcb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4b5881640 1 -- 172.21.15.5:0/3295629441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38d6f9340 con 0x55a38d7fd0c0 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4b6082640 1 --2- 172.21.15.5:0/3295629441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4b5881640 1 --2- 172.21.15.5:0/3295629441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d7ff4b0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0011740 tx=0x7fa4a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6ae0c093eb4c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4a4bfe640 1 -- 172.21.15.5:0/3295629441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0004050 con 0x55a38d7fd0c0 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4a4bfe640 1 -- 172.21.15.5:0/3295629441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a000fca0 con 0x55a38d7fd0c0 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4a4bfe640 1 -- 172.21.15.5:0/3295629441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0003950 con 0x55a38d7fd0c0 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.472+0000 7fa4a77fe640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa49c0a2da0 0x7fa49c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4b6082640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa49c0a2da0 0x7fa49c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4a77fe640 1 -- 172.21.15.5:0/1039036746 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fa4a80cf980 con 0x7fa4b0107b70 2024-09-11T22:33:44.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3295629441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 msgr2=0x55a38d7ff4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3295629441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d7ff4b0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0011740 tx=0x7fa4a0004730 comp rx=0 tx=0).stop 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4b6082640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa49c0a2da0 0x7fa49c0a5260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55a38d7f06c0 tx=0x7fa4ac0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3295629441 shutdown_connections 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3295629441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d7ff4b0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3295629441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3295629441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 0x55a38d7eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3295629441 >> 172.21.15.5:0/3295629441 conn(0x55a38d7ea060 msgr2=0x55a38d7ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3295629441 shutdown_connections 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.473+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3295629441 wait complete. 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4bb07f740 1 -- start start 2024-09-11T22:33:44.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4bb07f740 1 Processor -- start 2024-09-11T22:33:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7fd0c0 0x55a38d7faad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a38d7ffcf0 con 0x55a38d7ee2f0 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a38d6db290 con 0x55a38d7fd0c0 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a38d7ffe70 con 0x55a38d7ed9d0 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4bb07f740 10 client.0 _ll_register_callbacks cb 0x55a38d70c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4b5881640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4b6082640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4b6082640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:35404/0 (socket says 172.21.15.5:35404) 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4b6082640 1 -- 172.21.15.5:0/2246673838 learned_addr learned my addr 172.21.15.5:0/2246673838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4b5881640 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56774/0 (socket says 172.21.15.5:56774) 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.474+0000 7fa4b6883640 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7fd0c0 0x55a38d7faad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa4b6082640 1 -- 172.21.15.5:0/2246673838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7fd0c0 msgr2=0x55a38d7faad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa4b6082640 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7fd0c0 0x55a38d7faad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa4b6082640 1 -- 172.21.15.5:0/2246673838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ee2f0 msgr2=0x55a38d7fa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:44.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa4b6082640 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa4b6082640 1 -- 172.21.15.5:0/2246673838 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a38d465210 con 0x55a38d7ed9d0 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa4b5881640 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa4b6082640 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fa050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009230 tx=0x7fa4ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e78d33c2b0513d5f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa499ffb640 1 -- 172.21.15.5:0/2246673838 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac04d070 con 0x55a38d7ed9d0 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa499ffb640 1 -- 172.21.15.5:0/2246673838 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fa4ac003c40 con 0x55a38d7ed9d0 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa4bb07f740 1 -- 172.21.15.5:0/2246673838 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a000fca0 con 0x55a38d7ed9d0 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa499ffb640 1 -- 172.21.15.5:0/2246673838 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa4ac043070 con 0x55a38d7ed9d0 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa4bb07f740 10 client.28171 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.475+0000 7fa4bb07f740 1 -- 172.21.15.5:0/2246673838 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a38d531620 con 0x55a38d7ed9d0 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.476+0000 7fa499ffb640 1 -- 172.21.15.5:0/2246673838 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a000fca0 con 0x55a38d7ed9d0 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.476+0000 7fa499ffb640 1 -- 172.21.15.5:0/2246673838 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fa4ac049420 con 0x55a38d7ed9d0 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.476+0000 7fa4bb07f740 10 client.28171 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.476+0000 7fa4bb07f740 10 client.28171 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:44.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.476+0000 7fa4bb07f740 20 client.28171 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.476+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a38d80d100 0x55a38d80f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.476+0000 7fa4bb07f740 4 client.28171 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.476+0000 7fa4bb07f740 1 -- 172.21.15.5:0/2246673838 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a38d75fa20 con 0x55a38d80d100 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.476+0000 7fa4b5881640 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a38d80d100 0x55a38d80f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.477+0000 7fa499ffb640 10 client.28171 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.477+0000 7fa4b5881640 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a38d80d100 0x55a38d80f4f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70fe0358a55ff86 server_cookie=acc59df790f80e74 in_seq=0 out_seq=0 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa499ffb640 1 -- 172.21.15.5:0/2246673838 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a38d75fa20 con 0x55a38d80d100 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa499ffb640 10 client.28171 handle_command_reply: tid=0 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 client.28171 shutdown 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 -- 172.21.15.5:0/2246673838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 msgr2=0x55a38d7fa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fa050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009230 tx=0x7fa4ac009260 comp rx=0 tx=0).stop 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 -- 172.21.15.5:0/2246673838 shutdown_connections 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a38d80d100 0x55a38d80f4f0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7fd0c0 0x55a38d7faad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ee2f0 0x55a38d7fa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa499ffb640 0 client.28171 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/2246673838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fa050 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 -- 172.21.15.5:0/2246673838 >> 172.21.15.5:0/2246673838 conn(0x55a38d7ea060 msgr2=0x55a38d7efdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 -- 172.21.15.5:0/2246673838 shutdown_connections 2024-09-11T22:33:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 1 -- 172.21.15.5:0/2246673838 wait complete. 2024-09-11T22:33:44.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.478+0000 7fa4bb07f740 20 client.28171 trim_cache size 0 max 16384 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4bb07f740 1 Processor -- start 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4bb07f740 1 -- start start 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 0x55a38d814db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7fd0c0 0x55a38d8152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01ab700 con 0x55a38d7ed9d0 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01ad240 con 0x55a38d7ee2f0 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01abdd0 con 0x55a38d7fd0c0 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4b6082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4b6082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56778/0 (socket says 172.21.15.5:56778) 2024-09-11T22:33:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4b6082640 1 -- 172.21.15.5:0/3544781209 learned_addr learned my addr 172.21.15.5:0/3544781209 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4b5881640 1 --2- 172.21.15.5:0/3544781209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 0x55a38d814db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.493+0000 7fa4b6883640 1 --2- 172.21.15.5:0/3544781209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7fd0c0 0x55a38d8152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.494+0000 7fa4b6082640 1 -- 172.21.15.5:0/3544781209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7fd0c0 msgr2=0x55a38d8152f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.494+0000 7fa4b6082640 1 --2- 172.21.15.5:0/3544781209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7fd0c0 0x55a38d8152f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.494+0000 7fa4b6082640 1 -- 172.21.15.5:0/3544781209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 msgr2=0x55a38d814db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.494+0000 7fa4b6082640 1 --2- 172.21.15.5:0/3544781209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 0x55a38d814db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.494+0000 7fa4b6082640 1 -- 172.21.15.5:0/3544781209 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38d7f8fe0 con 0x55a38d7ed9d0 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.494+0000 7fa4b6082640 1 --2- 172.21.15.5:0/3544781209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fc8c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac033000 tx=0x7fa4ac02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcc0df7a8226251 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.494+0000 7fa49bfff640 1 -- 172.21.15.5:0/3544781209 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac04d070 con 0x55a38d7ed9d0 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.494+0000 7fa49bfff640 1 -- 172.21.15.5:0/3544781209 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac049cb0 con 0x55a38d7ed9d0 2024-09-11T22:33:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.494+0000 7fa49bfff640 1 -- 172.21.15.5:0/3544781209 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac049e60 con 0x55a38d7ed9d0 2024-09-11T22:33:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3544781209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ed9d0 msgr2=0x55a38d7fc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3544781209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fc8c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac033000 tx=0x7fa4ac02a420 comp rx=0 tx=0).stop 2024-09-11T22:33:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3544781209 shutdown_connections 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3544781209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7fd0c0 0x55a38d8152f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3544781209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ee2f0 0x55a38d814db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3544781209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7ed9d0 0x55a38d7fc8c0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3544781209 >> 172.21.15.5:0/3544781209 conn(0x55a38d7ea060 msgr2=0x55a38d7efdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3544781209 shutdown_connections 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3544781209 wait complete. 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 -- start start 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.495+0000 7fa4bb07f740 1 Processor -- start 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ed9d0 0x55a38d800e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ee2f0 0x55a38d801350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4bb07f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d801890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01acb70 con 0x55a38d7fd0c0 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01ac4a0 con 0x55a38d7ed9d0 2024-09-11T22:33:44.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4bb07f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b01a9700 con 0x55a38d7ee2f0 2024-09-11T22:33:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4bb07f740 10 client.0 _ll_register_callbacks cb 0x7fa4ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b6883640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d801890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b6883640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d801890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56790/0 (socket says 172.21.15.5:56790) 2024-09-11T22:33:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b5881640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ee2f0 0x55a38d801350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b6883640 1 -- 172.21.15.5:0/3108824605 learned_addr learned my addr 172.21.15.5:0/3108824605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b6082640 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ed9d0 0x55a38d800e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b6883640 1 -- 172.21.15.5:0/3108824605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ee2f0 msgr2=0x55a38d801350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b6883640 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ee2f0 0x55a38d801350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b6883640 1 -- 172.21.15.5:0/3108824605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ed9d0 msgr2=0x55a38d800e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:44.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b6883640 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ed9d0 0x55a38d800e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b6883640 1 -- 172.21.15.5:0/3108824605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a38d876d80 con 0x55a38d7fd0c0 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4b6883640 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d801890 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80982c0 tx=0x7fa4a809ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1317f50d475313a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa499ffb640 1 -- 172.21.15.5:0/3108824605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a809c2c0 con 0x55a38d7fd0c0 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa499ffb640 1 -- 172.21.15.5:0/3108824605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fa4a80a4640 con 0x55a38d7fd0c0 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3108824605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4ac049cb0 con 0x55a38d7fd0c0 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa499ffb640 1 -- 172.21.15.5:0/3108824605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa4a80a0210 con 0x55a38d7fd0c0 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4bb07f740 10 client.19080 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.496+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3108824605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a38d8780d0 con 0x55a38d7fd0c0 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.497+0000 7fa499ffb640 1 -- 172.21.15.5:0/3108824605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ac049cb0 con 0x55a38d7fd0c0 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.497+0000 7fa499ffb640 1 -- 172.21.15.5:0/3108824605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fa4a80053f0 con 0x55a38d7fd0c0 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.497+0000 7fa4bb07f740 10 client.19080 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.497+0000 7fa4bb07f740 10 client.19080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:44.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.497+0000 7fa4bb07f740 20 client.19080 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.497+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a38d8787f0 0x55a38d87ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.497+0000 7fa4bb07f740 4 client.19080 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:33:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.497+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3108824605 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a38d759d70 con 0x55a38d8787f0 2024-09-11T22:33:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.497+0000 7fa4b6082640 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a38d8787f0 0x55a38d87ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.498+0000 7fa4b6082640 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a38d8787f0 0x55a38d87ac10 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d72a9e038130e4b5 server_cookie=3afc8681c5d42762 in_seq=0 out_seq=0 2024-09-11T22:33:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.498+0000 7fa499ffb640 10 client.19080 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa499ffb640 1 -- 172.21.15.5:0/3108824605 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a38d759d70 con 0x55a38d8787f0 2024-09-11T22:33:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa499ffb640 10 client.19080 handle_command_reply: tid=0 2024-09-11T22:33:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 client.19080 shutdown 2024-09-11T22:33:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3108824605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 msgr2=0x55a38d801890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:44.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d801890 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fa4a80982c0 tx=0x7fa4a809ba60 comp rx=0 tx=0).stop 2024-09-11T22:33:44.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3108824605 shutdown_connections 2024-09-11T22:33:44.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a38d8787f0 0x55a38d87ac10 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a38d7fd0c0 0x55a38d801890 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa499ffb640 0 client.19080 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:44.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a38d7ee2f0 0x55a38d801350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 --2- 172.21.15.5:0/3108824605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a38d7ed9d0 0x55a38d800e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3108824605 >> 172.21.15.5:0/3108824605 conn(0x55a38d7ea060 msgr2=0x55a38d7e89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:44.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3108824605 shutdown_connections 2024-09-11T22:33:44.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 1 -- 172.21.15.5:0/3108824605 wait complete. 2024-09-11T22:33:44.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.628+0000 7fa4bb07f740 20 client.19080 trim_cache size 0 max 16384 2024-09-11T22:33:44.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:33:44.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:33:44.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:33:44.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 -- 172.21.15.5:0/1039036746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa49c0a2da0 msgr2=0x7fa49c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa49c0a2da0 0x7fa49c0a5260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55a38d7f06c0 tx=0x7fa4ac0058d0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 -- 172.21.15.5:0/1039036746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b0107b70 msgr2=0x7fa4b019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b0107b70 0x7fa4b019cc50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800eb10 tx=0x7fa4a800c2d0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 -- 172.21.15.5:0/1039036746 shutdown_connections 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b0107b70 0x7fa4b019cc50 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa49c0a2da0 0x7fa49c0a5260 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4b0102470 0x7fa4b01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 --2- 172.21.15.5:0/1039036746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0101ad0 0x7fa4b0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 -- 172.21.15.5:0/1039036746 >> 172.21.15.5:0/1039036746 conn(0x7fa4b00fd7d0 msgr2=0x7fa4b00fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:44.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 -- 172.21.15.5:0/1039036746 shutdown_connections 2024-09-11T22:33:44.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:44.629+0000 7fa4b82e6640 1 -- 172.21.15.5:0/1039036746 wait complete. 2024-09-11T22:33:44.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:33:44.962 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:33:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:44 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:33:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:44 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:33:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:44 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:33:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:44 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:33:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:44 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:33:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:44 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:33:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:44 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:33:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:44 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:33:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:44 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:33:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:44 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:33:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:44 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:33:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:44 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:33:45.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:44 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:33:45.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:44 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:33:45.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:44 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:33:45.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:44 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:33:45.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:44 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:33:45.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:44 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:33:45.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 855599 7.19 MB/sec execute 483 sec latency 100.019 ms 2024-09-11T22:33:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 -- 172.21.15.5:0/3249740000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdc8112640 msgr2=0x7fbdc8112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 --2- 172.21.15.5:0/3249740000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdc8112640 0x7fbdc8112a20 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009900 tx=0x7fbdb802a410 comp rx=0 tx=0).stop 2024-09-11T22:33:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 -- 172.21.15.5:0/3249740000 shutdown_connections 2024-09-11T22:33:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 --2- 172.21.15.5:0/3249740000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc810cf30 0x7fbdc8119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 --2- 172.21.15.5:0/3249740000 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc810c590 0x7fbdc810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 --2- 172.21.15.5:0/3249740000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdc8112640 0x7fbdc8112a20 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 -- 172.21.15.5:0/3249740000 >> 172.21.15.5:0/3249740000 conn(0x7fbdc806cdc0 msgr2=0x7fbdc806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 -- 172.21.15.5:0/3249740000 shutdown_connections 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 -- 172.21.15.5:0/3249740000 wait complete. 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 Processor -- start 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.455+0000 7fbdcea62640 1 -- start start 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdcea62640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdc810c590 0x7fbdc81a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdcea62640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc810cf30 0x7fbdc81a3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdcea62640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc8112640 0x7fbdc81a7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdcea62640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc811c9a0 con 0x7fbdc810c590 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdcea62640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc811c820 con 0x7fbdc810cf30 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdcea62640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc811cb20 con 0x7fbdc8112640 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdc810c590 0x7fbdc81a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdc810c590 0x7fbdc81a2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56820/0 (socket says 172.21.15.5:56820) 2024-09-11T22:33:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdc7fff640 1 -- 172.21.15.5:0/3613835776 learned_addr learned my addr 172.21.15.5:0/3613835776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdc77fe640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc810cf30 0x7fbdc81a3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdccfff640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc8112640 0x7fbdc81a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdc7fff640 1 -- 172.21.15.5:0/3613835776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc8112640 msgr2=0x7fbdc81a7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc8112640 0x7fbdc81a7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdc7fff640 1 -- 172.21.15.5:0/3613835776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc810cf30 msgr2=0x7fbdc81a3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc810cf30 0x7fbdc81a3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdc7fff640 1 -- 172.21.15.5:0/3613835776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc81a7d70 con 0x7fbdc810c590 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.456+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdc810c590 0x7fbdc81a2dc0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8006010 tx=0x7fbdb8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.457+0000 7fbdc57fa640 1 -- 172.21.15.5:0/3613835776 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb803f070 con 0x7fbdc810c590 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.457+0000 7fbdc57fa640 1 -- 172.21.15.5:0/3613835776 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb8031c30 con 0x7fbdc810c590 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.457+0000 7fbdcea62640 1 -- 172.21.15.5:0/3613835776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc81a8000 con 0x7fbdc810c590 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.457+0000 7fbdc57fa640 1 -- 172.21.15.5:0/3613835776 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8035030 con 0x7fbdc810c590 2024-09-11T22:33:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.457+0000 7fbdcea62640 1 -- 172.21.15.5:0/3613835776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc81af940 con 0x7fbdc810c590 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.459+0000 7fbdc57fa640 1 -- 172.21.15.5:0/3613835776 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbdb8042070 con 0x7fbdc810c590 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.459+0000 7fbdd0e3b740 1 Processor -- start 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.459+0000 7fbdd0e3b740 1 -- start start 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3cedc0 0x55889b3d9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b3dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55889b3342e0 con 0x55889b3ce4a0 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55889b2dbce0 con 0x55889b3d9680 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55889b3302a0 con 0x55889b3cedc0 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3cedc0 0x55889b3d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdccfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3cedc0 0x55889b3d9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:35454/0 (socket says 172.21.15.5:35454) 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc7fff640 1 -- 172.21.15.5:0/2596303913 learned_addr learned my addr 172.21.15.5:0/2596303913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdccfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56836/0 (socket says 172.21.15.5:56836) 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc77fe640 1 --2- 172.21.15.5:0/2596303913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b3dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc57fa640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdb00a2d70 0x7fbdb00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdccfff640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdb00a2d70 0x7fbdb00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc7fff640 1 -- 172.21.15.5:0/2596303913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 msgr2=0x55889b3dba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/2596303913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b3dba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc7fff640 1 -- 172.21.15.5:0/2596303913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 msgr2=0x55889b3ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/2596303913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc57fa640 1 -- 172.21.15.5:0/3613835776 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fbdb80efa70 con 0x7fbdc810c590 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc7fff640 1 -- 172.21.15.5:0/2596303913 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889b2c0440 con 0x55889b3cedc0 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc77fe640 1 --2- 172.21.15.5:0/2596303913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b3dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdccfff640 1 --2- 172.21.15.5:0/2596303913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.460+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/2596303913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3cedc0 0x55889b3d9140 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80b8460 tx=0x7fbdb802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dfd02d8a3983e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.461+0000 7fbdccfff640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdb00a2d70 0x7fbdb00a5230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0005c40 tx=0x7fbdc0004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.461+0000 7fbdae7fc640 1 -- 172.21.15.5:0/2596303913 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8049030 con 0x55889b3cedc0 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.461+0000 7fbdae7fc640 1 -- 172.21.15.5:0/2596303913 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb8031de0 con 0x55889b3cedc0 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.461+0000 7fbdae7fc640 1 -- 172.21.15.5:0/2596303913 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8045e60 con 0x55889b3cedc0 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.461+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/2596303913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3cedc0 msgr2=0x55889b3d9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.461+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/2596303913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3cedc0 0x55889b3d9140 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80b8460 tx=0x7fbdb802a410 comp rx=0 tx=0).stop 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/2596303913 shutdown_connections 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/2596303913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b3dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/2596303913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3cedc0 0x55889b3d9140 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/2596303913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/2596303913 >> 172.21.15.5:0/2596303913 conn(0x55889b3caa30 msgr2=0x55889b3cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/2596303913 shutdown_connections 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/2596303913 wait complete. 2024-09-11T22:33:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 -- start start 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 Processor -- start 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3d8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3cedc0 0x55889b3d6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3d9680 0x55889b3d6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55889b3dc010 con 0x55889b3ce4a0 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55889b2ef050 con 0x55889b3cedc0 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55889b3dc190 con 0x55889b3d9680 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.462+0000 7fbdd0e3b740 10 client.0 _ll_register_callbacks cb 0x55889b32f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3d8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3cedc0 0x55889b3d6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3d8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56850/0 (socket says 172.21.15.5:56850) 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc7fff640 1 -- 172.21.15.5:0/1452062375 learned_addr learned my addr 172.21.15.5:0/1452062375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdccfff640 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3d9680 0x55889b3d6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc7fff640 1 -- 172.21.15.5:0/1452062375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3d9680 msgr2=0x55889b3d6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3d9680 0x55889b3d6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc7fff640 1 -- 172.21.15.5:0/1452062375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3cedc0 msgr2=0x55889b3d6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3cedc0 0x55889b3d6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc7fff640 1 -- 172.21.15.5:0/1452062375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889b0a53c0 con 0x55889b3ce4a0 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc77fe640 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3cedc0 0x55889b3d6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3d8f50 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80bbd00 tx=0x7fbdb803b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580da032ecebc8a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbda77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbda77fe640 1 -- 172.21.15.5:0/1452062375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8049040 con 0x55889b3ce4a0 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbda77fe640 1 -- 172.21.15.5:0/1452062375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fbdb80b8670 con 0x55889b3ce4a0 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1452062375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb8031de0 con 0x55889b3ce4a0 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbda77fe640 1 -- 172.21.15.5:0/1452062375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbdb80bb960 con 0x55889b3ce4a0 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdd0e3b740 10 client.19092 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.463+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1452062375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55889b3dc010 con 0x55889b3ce4a0 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbda77fe640 1 -- 172.21.15.5:0/1452062375 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb8031de0 con 0x55889b3ce4a0 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbda77fe640 1 -- 172.21.15.5:0/1452062375 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fbdb80d8070 con 0x55889b3ce4a0 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbdd0e3b740 10 client.19092 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbdd0e3b740 10 client.19092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbdd0e3b740 20 client.19092 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55889b3e8b40 0x55889b3eafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbdd0e3b740 4 client.19092 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1452062375 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55889b341850 con 0x55889b3e8b40 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbdc77fe640 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55889b3e8b40 0x55889b3eafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbdc77fe640 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55889b3e8b40 0x55889b3eafa0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3296e0efd4f48e5 server_cookie=dea51296543dd7df in_seq=0 out_seq=0 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.464+0000 7fbda77fe640 10 client.19092 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbda77fe640 1 -- 172.21.15.5:0/1452062375 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55889b341850 con 0x55889b3e8b40 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbda77fe640 10 client.19092 handle_command_reply: tid=0 2024-09-11T22:33:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 client.19092 shutdown 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1452062375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 msgr2=0x55889b3d8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3d8f50 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80bbd00 tx=0x7fbdb803b690 comp rx=0 tx=0).stop 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1452062375 shutdown_connections 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55889b3e8b40 0x55889b3eafa0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbda77fe640 0 client.19092 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3d9680 0x55889b3d6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3cedc0 0x55889b3d6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1452062375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3ce4a0 0x55889b3d8f50 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1452062375 >> 172.21.15.5:0/1452062375 conn(0x55889b3caa30 msgr2=0x55889b3d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1452062375 shutdown_connections 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1452062375 wait complete. 2024-09-11T22:33:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.466+0000 7fbdd0e3b740 20 client.19092 trim_cache size 0 max 16384 2024-09-11T22:33:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdd0e3b740 1 Processor -- start 2024-09-11T22:33:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdd0e3b740 1 -- start start 2024-09-11T22:33:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 0x55889b3e5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3f0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b3f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81b83f0 con 0x55889b3cedc0 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81b9f30 con 0x55889b3d9680 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81b8ac0 con 0x55889b3ce4a0 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdc77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3f0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdc77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3f0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56862/0 (socket says 172.21.15.5:56862) 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 0x55889b3e5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdc77fe640 1 -- 172.21.15.5:0/805478885 learned_addr learned my addr 172.21.15.5:0/805478885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdccfff640 1 --2- 172.21.15.5:0/805478885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b3f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdc7fff640 1 -- 172.21.15.5:0/805478885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 msgr2=0x55889b3f0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:45.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.484+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/805478885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b3f0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.485+0000 7fbdc7fff640 1 -- 172.21.15.5:0/805478885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 msgr2=0x55889b3f0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.485+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/805478885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3f0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.485+0000 7fbdc7fff640 1 -- 172.21.15.5:0/805478885 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889b3ca860 con 0x55889b3ce4a0 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.485+0000 7fbdc77fe640 1 --2- 172.21.15.5:0/805478885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3f0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.485+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/805478885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 0x55889b3e5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80bbbc0 tx=0x7fbdb803b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8597fd93e4b41bb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.485+0000 7fbdadffb640 1 -- 172.21.15.5:0/805478885 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8049070 con 0x55889b3ce4a0 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.485+0000 7fbdadffb640 1 -- 172.21.15.5:0/805478885 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb80bc4e0 con 0x55889b3ce4a0 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.485+0000 7fbdadffb640 1 -- 172.21.15.5:0/805478885 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80c4730 con 0x55889b3ce4a0 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/805478885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 msgr2=0x55889b3e5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/805478885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 0x55889b3e5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80bbbc0 tx=0x7fbdb803b700 comp rx=0 tx=0).stop 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/805478885 shutdown_connections 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/805478885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b3f0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/805478885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3f0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/805478885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 0x55889b3e5170 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/805478885 >> 172.21.15.5:0/805478885 conn(0x55889b3caa30 msgr2=0x55889b3d82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/805478885 shutdown_connections 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/805478885 wait complete. 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 -- start start 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.486+0000 7fbdd0e3b740 1 Processor -- start 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 0x55889b3deba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdd0e3b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b44a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81b9860 con 0x55889b3cedc0 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81b9190 con 0x55889b3d9680 2024-09-11T22:33:45.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdd0e3b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81b63d0 con 0x55889b3ce4a0 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdd0e3b740 10 client.0 _ll_register_callbacks cb 0x7fbdc0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdc77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdc7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 0x55889b3deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdc77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3df0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56878/0 (socket says 172.21.15.5:56878) 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdc77fe640 1 -- 172.21.15.5:0/1810858305 learned_addr learned my addr 172.21.15.5:0/1810858305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdccfff640 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b44a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdc7fff640 1 -- 172.21.15.5:0/1810858305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 msgr2=0x55889b44a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b44a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdc7fff640 1 -- 172.21.15.5:0/1810858305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 msgr2=0x55889b3df0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3df0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdc7fff640 1 -- 172.21.15.5:0/1810858305 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889b44aea0 con 0x55889b3ce4a0 2024-09-11T22:33:45.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdc7fff640 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 0x55889b3deba0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80b8460 tx=0x7fbdb80ed680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7b1ae1985ae10e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbda77fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbda77fe640 1 -- 172.21.15.5:0/1810858305 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8049070 con 0x55889b3ce4a0 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbda77fe640 1 -- 172.21.15.5:0/1810858305 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fbdb80c4940 con 0x55889b3ce4a0 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.487+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1810858305 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb80bc4e0 con 0x55889b3ce4a0 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbda77fe640 1 -- 172.21.15.5:0/1810858305 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbdb8031de0 con 0x55889b3ce4a0 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbdd0e3b740 10 client.28195 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1810858305 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55889b44aea0 con 0x55889b3ce4a0 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbda77fe640 1 -- 172.21.15.5:0/1810858305 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdb80bc4e0 con 0x55889b3ce4a0 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbda77fe640 1 -- 172.21.15.5:0/1810858305 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fbdb80b6b70 con 0x55889b3ce4a0 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbdd0e3b740 10 client.28195 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbdd0e3b740 10 client.28195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbdd0e3b740 20 client.28195 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55889b3e8b40 0x55889b455700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:45.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbdd0e3b740 4 client.28195 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:33:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1810858305 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55889b33bd60 con 0x55889b3e8b40 2024-09-11T22:33:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.488+0000 7fbdc77fe640 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55889b3e8b40 0x55889b455700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.489+0000 7fbda77fe640 10 client.28195 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:45.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.489+0000 7fbdc77fe640 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55889b3e8b40 0x55889b455700 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e742b529fc9493 server_cookie=a7d10a67fac2f364 in_seq=0 out_seq=0 2024-09-11T22:33:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbda77fe640 1 -- 172.21.15.5:0/1810858305 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55889b33bd60 con 0x55889b3e8b40 2024-09-11T22:33:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbda77fe640 10 client.28195 handle_command_reply: tid=0 2024-09-11T22:33:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 client.28195 shutdown 2024-09-11T22:33:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1810858305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 msgr2=0x55889b3deba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 0x55889b3deba0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80b8460 tx=0x7fbdb80ed680 comp rx=0 tx=0).stop 2024-09-11T22:33:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1810858305 shutdown_connections 2024-09-11T22:33:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55889b3e8b40 0x55889b455700 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55889b3d9680 0x55889b44a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55889b3cedc0 0x55889b3df0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbda77fe640 0 client.28195 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 --2- 172.21.15.5:0/1810858305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55889b3ce4a0 0x55889b3deba0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1810858305 >> 172.21.15.5:0/1810858305 conn(0x55889b3caa30 msgr2=0x55889b3d49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1810858305 shutdown_connections 2024-09-11T22:33:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 1 -- 172.21.15.5:0/1810858305 wait complete. 2024-09-11T22:33:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.490+0000 7fbdd0e3b740 20 client.28195 trim_cache size 0 max 16384 2024-09-11T22:33:45.497 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:33:45.497 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:33:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:33:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "7f65598c-6263-4f1b-ac8b-c9350044ad61": { 2024-09-11T22:33:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:33:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "7f65598c-6263-4f1b-ac8b-c9350044ad61", 2024-09-11T22:33:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:33:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:33:45.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:33:45.498 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 -- 172.21.15.5:0/3613835776 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdb00a2d70 msgr2=0x7fbdb00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdb00a2d70 0x7fbdb00a5230 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0005c40 tx=0x7fbdc0004ab0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 -- 172.21.15.5:0/3613835776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdc810c590 msgr2=0x7fbdc81a2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdc810c590 0x7fbdc81a2dc0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8006010 tx=0x7fbdb8003c10 comp rx=0 tx=0).stop 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 -- 172.21.15.5:0/3613835776 shutdown_connections 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc8112640 0x7fbdc81a7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbdb00a2d70 0x7fbdb00a5230 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdc810cf30 0x7fbdc81a3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 --2- 172.21.15.5:0/3613835776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdc810c590 0x7fbdc81a2dc0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 -- 172.21.15.5:0/3613835776 >> 172.21.15.5:0/3613835776 conn(0x7fbdc806cdc0 msgr2=0x7fbdc810aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:45.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 -- 172.21.15.5:0/3613835776 shutdown_connections 2024-09-11T22:33:45.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:33:45.491+0000 7fbdcea62640 1 -- 172.21.15.5:0/3613835776 wait complete. 2024-09-11T22:33:46.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:45 smithi005 ceph-mon[30943]: pgmap v366: 97 pgs: 97 active+clean; 111 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-09-11T22:33:46.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:45 smithi159 ceph-mon[35534]: pgmap v366: 97 pgs: 97 active+clean; 111 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-09-11T22:33:46.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:45 smithi172 ceph-mon[35461]: pgmap v366: 97 pgs: 97 active+clean; 111 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-09-11T22:33:46.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 856885 7.18 MB/sec execute 484 sec latency 21.425 ms 2024-09-11T22:33:46.511 INFO:tasks.cephfs.filesystem:scrub status for tag:7f65598c-6263-4f1b-ac8b-c9350044ad61 - {'path': '/', 'tag': '7f65598c-6263-4f1b-ac8b-c9350044ad61', 'options': 'recursive,force'} 2024-09-11T22:33:47.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859166 7.20 MB/sec execute 485 sec latency 90.740 ms 2024-09-11T22:33:48.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:47 smithi005 ceph-mon[30943]: pgmap v367: 97 pgs: 97 active+clean; 111 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 148 op/s 2024-09-11T22:33:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:47 smithi159 ceph-mon[35534]: pgmap v367: 97 pgs: 97 active+clean; 111 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 148 op/s 2024-09-11T22:33:48.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:47 smithi172 ceph-mon[35461]: pgmap v367: 97 pgs: 97 active+clean; 111 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 148 op/s 2024-09-11T22:33:48.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 860182 7.19 MB/sec execute 486 sec latency 29.607 ms 2024-09-11T22:33:49.343 INFO:tasks.workunit.client.0.smithi005.stdout: 1 862689 7.21 MB/sec execute 487 sec latency 24.078 ms 2024-09-11T22:33:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:49 smithi005 ceph-mon[30943]: pgmap v368: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-09-11T22:33:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:49 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:33:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:49 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:33:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:49 smithi159 ceph-mon[35534]: pgmap v368: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-09-11T22:33:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:49 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:33:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:49 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:33:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:49 smithi172 ceph-mon[35461]: pgmap v368: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.3 MiB/s wr, 320 op/s 2024-09-11T22:33:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:49 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:33:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:49 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:33:50.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 863932 7.20 MB/sec execute 488 sec latency 17.579 ms 2024-09-11T22:33:51.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866314 7.22 MB/sec execute 489 sec latency 15.856 ms 2024-09-11T22:33:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:51 smithi005 ceph-mon[30943]: pgmap v369: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-09-11T22:33:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:51 smithi159 ceph-mon[35534]: pgmap v369: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-09-11T22:33:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:51 smithi172 ceph-mon[35461]: pgmap v369: 97 pgs: 97 active+clean; 106 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-09-11T22:33:52.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 867718 7.22 MB/sec execute 490 sec latency 31.483 ms 2024-09-11T22:33:53.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 869898 7.23 MB/sec execute 491 sec latency 24.401 ms 2024-09-11T22:33:54.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:53 smithi005 ceph-mon[30943]: pgmap v370: 97 pgs: 97 active+clean; 114 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-11T22:33:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:53 smithi159 ceph-mon[35534]: pgmap v370: 97 pgs: 97 active+clean; 114 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-11T22:33:54.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:53 smithi172 ceph-mon[35461]: pgmap v370: 97 pgs: 97 active+clean; 114 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-11T22:33:54.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 871394 7.23 MB/sec execute 492 sec latency 15.867 ms 2024-09-11T22:33:55.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 873394 7.24 MB/sec execute 493 sec latency 186.729 ms 2024-09-11T22:33:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:55 smithi005 ceph-mon[30943]: pgmap v371: 97 pgs: 97 active+clean; 114 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-09-11T22:33:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:55 smithi159 ceph-mon[35534]: pgmap v371: 97 pgs: 97 active+clean; 114 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-09-11T22:33:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:55 smithi172 ceph-mon[35461]: pgmap v371: 97 pgs: 97 active+clean; 114 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-09-11T22:33:56.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 874943 7.24 MB/sec execute 494 sec latency 14.139 ms 2024-09-11T22:33:57.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 877017 7.25 MB/sec execute 495 sec latency 46.163 ms 2024-09-11T22:33:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.749+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/2467079218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd44102d40 msgr2=0x7fbd44103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.749+0000 7fbd4a3ed640 1 --2- 172.21.15.5:0/2467079218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd44102d40 0x7fbd44103120 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009900 tx=0x7fbd3402a480 comp rx=0 tx=0).stop 2024-09-11T22:33:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.750+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/2467079218 shutdown_connections 2024-09-11T22:33:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.750+0000 7fbd4a3ed640 1 --2- 172.21.15.5:0/2467079218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd4410c0d0 0x7fbd4410e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.750+0000 7fbd4a3ed640 1 --2- 172.21.15.5:0/2467079218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd44103660 0x7fbd4410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.750+0000 7fbd4a3ed640 1 --2- 172.21.15.5:0/2467079218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd44102d40 0x7fbd44103120 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.750+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/2467079218 >> 172.21.15.5:0/2467079218 conn(0x7fbd440fc6f0 msgr2=0x7fbd440feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.750+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/2467079218 shutdown_connections 2024-09-11T22:33:57.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.750+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/2467079218 wait complete. 2024-09-11T22:33:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd4a3ed640 1 Processor -- start 2024-09-11T22:33:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd4a3ed640 1 -- start start 2024-09-11T22:33:57.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd4a3ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd44102d40 0x7fbd44198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd4a3ed640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd44103660 0x7fbd44198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd4a3ed640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd4410c0d0 0x7fbd4419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd4a3ed640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44111ed0 con 0x7fbd44102d40 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd4a3ed640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44111d50 con 0x7fbd4410c0d0 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd4a3ed640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44112050 con 0x7fbd44103660 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd44102d40 0x7fbd44198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd44102d40 0x7fbd44198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48288/0 (socket says 172.21.15.5:48288) 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd43fff640 1 -- 172.21.15.5:0/1855272723 learned_addr learned my addr 172.21.15.5:0/1855272723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.751+0000 7fbd437fe640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd44103660 0x7fbd44198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd4898a640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd4410c0d0 0x7fbd4419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd437fe640 1 -- 172.21.15.5:0/1855272723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd4410c0d0 msgr2=0x7fbd4419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd437fe640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd4410c0d0 0x7fbd4419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd437fe640 1 -- 172.21.15.5:0/1855272723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd44102d40 msgr2=0x7fbd44198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd437fe640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd44102d40 0x7fbd44198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd437fe640 1 -- 172.21.15.5:0/1855272723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4419d300 con 0x7fbd44103660 2024-09-11T22:33:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd43fff640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd44102d40 0x7fbd44198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd437fe640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd44103660 0x7fbd44198850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800e9f0 tx=0x7fbd3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd417fa640 1 -- 172.21.15.5:0/1855272723 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3800ccc0 con 0x7fbd44103660 2024-09-11T22:33:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd417fa640 1 -- 172.21.15.5:0/1855272723 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd38010910 con 0x7fbd44103660 2024-09-11T22:33:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd417fa640 1 -- 172.21.15.5:0/1855272723 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd38018960 con 0x7fbd44103660 2024-09-11T22:33:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.752+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/1855272723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4419d5f0 con 0x7fbd44103660 2024-09-11T22:33:57.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.753+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/1855272723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd441008c0 con 0x7fbd44103660 2024-09-11T22:33:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.756+0000 7fbd417fa640 1 -- 172.21.15.5:0/1855272723 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbd3801f020 con 0x7fbd44103660 2024-09-11T22:33:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.756+0000 7fbd4d25d740 1 Processor -- start 2024-09-11T22:33:57.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.757+0000 7fbd417fa640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd2c0a7230 0x7fbd2c0a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.757+0000 7fbd4d25d740 1 -- start start 2024-09-11T22:33:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.757+0000 7fbd4898a640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd2c0a7230 0x7fbd2c0a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.757+0000 7fbd417fa640 1 -- 172.21.15.5:0/1855272723 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fbd380cf6c0 con 0x7fbd44103660 2024-09-11T22:33:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.757+0000 7fbd4898a640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd2c0a7230 0x7fbd2c0a96f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c005fd0 tx=0x7fbd3c009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.757+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 0x5611f04cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.757+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04d8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.757+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04db0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611f04322f0 con 0x5611f04d8cb0 2024-09-11T22:33:57.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5611f03da270 con 0x5611f04ce410 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5611f042e2b0 con 0x5611f04cdaf0 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd43fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 0x5611f04cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd437fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04d8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd4898a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04db0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd437fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04d8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57872/0 (socket says 172.21.15.5:57872) 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd437fe640 1 -- 172.21.15.5:0/3534148512 learned_addr learned my addr 172.21.15.5:0/3534148512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd437fe640 1 -- 172.21.15.5:0/3534148512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 msgr2=0x5611f04cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd437fe640 1 --2- 172.21.15.5:0/3534148512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 0x5611f04cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd437fe640 1 -- 172.21.15.5:0/3534148512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 msgr2=0x5611f04db0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd437fe640 1 --2- 172.21.15.5:0/3534148512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04db0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd437fe640 1 -- 172.21.15.5:0/3534148512 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611f03d9340 con 0x5611f04ce410 2024-09-11T22:33:57.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd4898a640 1 --2- 172.21.15.5:0/3534148512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd437fe640 1 --2- 172.21.15.5:0/3534148512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04d8770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbd38098570 tx=0x7fbd3809c2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad01d54472ac71f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd2a7fc640 1 -- 172.21.15.5:0/3534148512 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3809ff50 con 0x5611f04ce410 2024-09-11T22:33:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd2a7fc640 1 -- 172.21.15.5:0/3534148512 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd380a4810 con 0x5611f04ce410 2024-09-11T22:33:57.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.758+0000 7fbd2a7fc640 1 -- 172.21.15.5:0/3534148512 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3809aff0 con 0x5611f04ce410 2024-09-11T22:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.759+0000 7fbd4d25d740 1 -- 172.21.15.5:0/3534148512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 msgr2=0x5611f04d8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.759+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/3534148512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04d8770 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbd38098570 tx=0x7fbd3809c2a0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.759+0000 7fbd4d25d740 1 -- 172.21.15.5:0/3534148512 shutdown_connections 2024-09-11T22:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.759+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/3534148512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04db0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.759+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/3534148512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04d8770 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.759+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/3534148512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 0x5611f04cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.759+0000 7fbd4d25d740 1 -- 172.21.15.5:0/3534148512 >> 172.21.15.5:0/3534148512 conn(0x5611f04ca060 msgr2=0x5611f04ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:57.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.759+0000 7fbd4d25d740 1 -- 172.21.15.5:0/3534148512 shutdown_connections 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.759+0000 7fbd4d25d740 1 -- 172.21.15.5:0/3534148512 wait complete. 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd4d25d740 1 -- start start 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd4d25d740 1 Processor -- start 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04cdaf0 0x5611f04d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04d8cb0 0x5611f04d66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5611f04db880 con 0x5611f04cdaf0 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5611f03bb290 con 0x5611f04d8cb0 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5611f04dba00 con 0x5611f04ce410 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd4d25d740 10 client.0 _ll_register_callbacks cb 0x5611f03ec140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04cdaf0 0x5611f04d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04cdaf0 0x5611f04d5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48306/0 (socket says 172.21.15.5:48306) 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd43fff640 1 -- 172.21.15.5:0/1878074180 learned_addr learned my addr 172.21.15.5:0/1878074180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd4898a640 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04d8cb0 0x5611f04d66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.760+0000 7fbd437fe640 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd43fff640 1 -- 172.21.15.5:0/1878074180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 msgr2=0x5611f04d6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd43fff640 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd43fff640 1 -- 172.21.15.5:0/1878074180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04d8cb0 msgr2=0x5611f04d66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd43fff640 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04d8cb0 0x5611f04d66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd43fff640 1 -- 172.21.15.5:0/1878074180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611f0145210 con 0x5611f04cdaf0 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd437fe640 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd4898a640 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04d8cb0 0x5611f04d66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd43fff640 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04cdaf0 0x5611f04d5c30 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009a20 tx=0x7fbd34031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeed1d1f667550d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd237fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:33:57.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd237fe640 1 -- 172.21.15.5:0/1878074180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34035040 con 0x5611f04cdaf0 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd4d25d740 1 -- 172.21.15.5:0/1878074180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd380a4810 con 0x5611f04cdaf0 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.761+0000 7fbd237fe640 1 -- 172.21.15.5:0/1878074180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fbd34004240 con 0x5611f04cdaf0 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd237fe640 1 -- 172.21.15.5:0/1878074180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbd34004440 con 0x5611f04cdaf0 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd4d25d740 10 client.19110 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd4d25d740 1 -- 172.21.15.5:0/1878074180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5611f0211620 con 0x5611f04cdaf0 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd237fe640 1 -- 172.21.15.5:0/1878074180 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd380a4810 con 0x5611f04cdaf0 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd237fe640 1 -- 172.21.15.5:0/1878074180 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fbd34004c60 con 0x5611f04cdaf0 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd4d25d740 10 client.19110 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd4d25d740 10 client.19110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd4d25d740 20 client.19110 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611f04e8c90 0x5611f04eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd4d25d740 4 client.19110 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd4d25d740 1 -- 172.21.15.5:0/1878074180 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611f043fa20 con 0x5611f04e8c90 2024-09-11T22:33:57.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.762+0000 7fbd437fe640 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611f04e8c90 0x5611f04eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.763+0000 7fbd237fe640 10 client.19110 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:57.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.763+0000 7fbd437fe640 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611f04e8c90 0x5611f04eb080 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ca3081314446a4b server_cookie=a533fcd35b69c4a3 in_seq=0 out_seq=0 2024-09-11T22:33:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd237fe640 1 -- 172.21.15.5:0/1878074180 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611f043fa20 con 0x5611f04e8c90 2024-09-11T22:33:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd237fe640 10 client.19110 handle_command_reply: tid=0 2024-09-11T22:33:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 client.19110 shutdown 2024-09-11T22:33:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 -- 172.21.15.5:0/1878074180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04cdaf0 msgr2=0x5611f04d5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04cdaf0 0x5611f04d5c30 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fbd34009a20 tx=0x7fbd34031850 comp rx=0 tx=0).stop 2024-09-11T22:33:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 -- 172.21.15.5:0/1878074180 shutdown_connections 2024-09-11T22:33:57.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611f04e8c90 0x5611f04eb080 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04d8cb0 0x5611f04d66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/1878074180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04cdaf0 0x5611f04d5c30 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 -- 172.21.15.5:0/1878074180 >> 172.21.15.5:0/1878074180 conn(0x5611f04ca060 msgr2=0x5611f04cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd237fe640 0 client.19110 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 -- 172.21.15.5:0/1878074180 shutdown_connections 2024-09-11T22:33:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.765+0000 7fbd4d25d740 1 -- 172.21.15.5:0/1878074180 wait complete. 2024-09-11T22:33:57.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.766+0000 7fbd4d25d740 20 client.19110 trim_cache size 0 max 16384 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4d25d740 1 Processor -- start 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4d25d740 1 -- start start 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04cdaf0 0x5611f04d44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04f0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441ab7a0 con 0x5611f04d8cb0 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441ad2e0 con 0x5611f04cdaf0 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441abe70 con 0x5611f04ce410 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4898a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04f0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd437fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd43fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04cdaf0 0x5611f04d44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4898a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04f0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48320/0 (socket says 172.21.15.5:48320) 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd437fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33778/0 (socket says 172.21.15.5:33778) 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.781+0000 7fbd4898a640 1 -- 172.21.15.5:0/2329214303 learned_addr learned my addr 172.21.15.5:0/2329214303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd437fe640 1 -- 172.21.15.5:0/2329214303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04cdaf0 msgr2=0x5611f04d44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd437fe640 1 --2- 172.21.15.5:0/2329214303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04cdaf0 0x5611f04d44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd437fe640 1 -- 172.21.15.5:0/2329214303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 msgr2=0x5611f04f0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd437fe640 1 --2- 172.21.15.5:0/2329214303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04f0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd437fe640 1 -- 172.21.15.5:0/2329214303 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611f04d4bc0 con 0x5611f04ce410 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd4898a640 1 --2- 172.21.15.5:0/2329214303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd43fff640 1 --2- 172.21.15.5:0/2329214303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04cdaf0 0x5611f04d44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd437fe640 1 --2- 172.21.15.5:0/2329214303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d64e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbd38098520 tx=0x7fbd3800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3993ac5d35e469e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd29ffb640 1 -- 172.21.15.5:0/2329214303 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd380b8020 con 0x5611f04ce410 2024-09-11T22:33:57.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd29ffb640 1 -- 172.21.15.5:0/2329214303 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd380a4810 con 0x5611f04ce410 2024-09-11T22:33:57.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.782+0000 7fbd29ffb640 1 -- 172.21.15.5:0/2329214303 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3809b250 con 0x5611f04ce410 2024-09-11T22:33:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.783+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2329214303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 msgr2=0x5611f04d64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.783+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/2329214303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d64e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbd38098520 tx=0x7fbd3800cf60 comp rx=0 tx=0).stop 2024-09-11T22:33:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.783+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2329214303 shutdown_connections 2024-09-11T22:33:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.783+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/2329214303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04f0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.783+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/2329214303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04ce410 0x5611f04d64e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.783+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/2329214303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04cdaf0 0x5611f04d44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.783+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2329214303 >> 172.21.15.5:0/2329214303 conn(0x5611f04ca060 msgr2=0x5611f04d7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.783+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2329214303 shutdown_connections 2024-09-11T22:33:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.783+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2329214303 wait complete. 2024-09-11T22:33:57.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.783+0000 7fbd4d25d740 1 -- start start 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd4d25d740 1 Processor -- start 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 0x5611f04e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04e1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd4d25d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441acc10 con 0x5611f04d8cb0 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441ac540 con 0x5611f04ce410 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd4d25d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441a9780 con 0x5611f04cdaf0 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd4d25d740 10 client.0 _ll_register_callbacks cb 0x7fbd34009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd4898a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd43fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 0x5611f04e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd437fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd43fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 0x5611f04e0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33786/0 (socket says 172.21.15.5:33786) 2024-09-11T22:33:57.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd4898a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04dc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48334/0 (socket says 172.21.15.5:48334) 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd437fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04e1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57910/0 (socket says 172.21.15.5:57910) 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd43fff640 1 -- 172.21.15.5:0/2906872169 learned_addr learned my addr 172.21.15.5:0/2906872169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd43fff640 1 -- 172.21.15.5:0/2906872169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 msgr2=0x5611f04e1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd43fff640 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04e1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd43fff640 1 -- 172.21.15.5:0/2906872169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 msgr2=0x5611f04dc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd43fff640 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04dc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.784+0000 7fbd43fff640 1 -- 172.21.15.5:0/2906872169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611f04dca90 con 0x5611f04cdaf0 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd437fe640 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04e1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd43fff640 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 0x5611f04e0d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbd34005ab0 tx=0x7fbd34003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57e638b437922270 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd237fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd237fe640 1 -- 172.21.15.5:0/2906872169 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3405b060 con 0x5611f04cdaf0 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2906872169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd380a4810 con 0x5611f04cdaf0 2024-09-11T22:33:57.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd237fe640 1 -- 172.21.15.5:0/2906872169 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8585+0+0 (secure 0 0 0) 0x7fbd34005000 con 0x5611f04cdaf0 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd237fe640 1 -- 172.21.15.5:0/2906872169 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbd34035030 con 0x5611f04cdaf0 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd4d25d740 10 client.28213 fetch_fsmap learned FSMap version 45 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2906872169 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5611f0553380 con 0x5611f04cdaf0 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd237fe640 1 -- 172.21.15.5:0/2906872169 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd380a4810 con 0x5611f04cdaf0 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd237fe640 1 -- 172.21.15.5:0/2906872169 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fbd340049d0 con 0x5611f04cdaf0 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd4d25d740 10 client.28213 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd4d25d740 10 client.28213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd4d25d740 20 client.28213 populate_metadata read hostname 'smithi005' 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611f0553ae0 0x5611f0555f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd4d25d740 4 client.28213 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.785+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2906872169 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611f0439d70 con 0x5611f0553ae0 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.786+0000 7fbd437fe640 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611f0553ae0 0x5611f0555f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:33:57.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.786+0000 7fbd237fe640 10 client.28213 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:57.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.786+0000 7fbd437fe640 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611f0553ae0 0x5611f0555f00 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28f985dd07a17636 server_cookie=d176d343da784501 in_seq=0 out_seq=0 2024-09-11T22:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.907+0000 7fbd237fe640 1 -- 172.21.15.5:0/2906872169 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611f0439d70 con 0x5611f0553ae0 2024-09-11T22:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.907+0000 7fbd237fe640 10 client.28213 handle_command_reply: tid=0 2024-09-11T22:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.907+0000 7fbd4d25d740 1 client.28213 shutdown 2024-09-11T22:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.907+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2906872169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 msgr2=0x5611f04e0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.907+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 0x5611f04e0d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbd34005ab0 tx=0x7fbd34003ed0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.907+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2906872169 shutdown_connections 2024-09-11T22:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.907+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5611f0553ae0 0x5611f0555f00 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.907+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5611f04d8cb0 0x5611f04dc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.907+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5611f04ce410 0x5611f04e1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.907+0000 7fbd4d25d740 1 --2- 172.21.15.5:0/2906872169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5611f04cdaf0 0x5611f04e0d40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.908+0000 7fbd237fe640 0 client.28213 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.908+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2906872169 >> 172.21.15.5:0/2906872169 conn(0x5611f04ca060 msgr2=0x5611f04d7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.908+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2906872169 shutdown_connections 2024-09-11T22:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.908+0000 7fbd4d25d740 1 -- 172.21.15.5:0/2906872169 wait complete. 2024-09-11T22:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.908+0000 7fbd4d25d740 20 client.28213 trim_cache size 0 max 16384 2024-09-11T22:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:33:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/1855272723 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd2c0a7230 msgr2=0x7fbd2c0a96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd2c0a7230 0x7fbd2c0a96f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c005fd0 tx=0x7fbd3c009500 comp rx=0 tx=0).stop 2024-09-11T22:33:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/1855272723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd44103660 msgr2=0x7fbd44198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:33:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd44103660 0x7fbd44198850 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800e9f0 tx=0x7fbd3800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/1855272723 shutdown_connections 2024-09-11T22:33:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbd2c0a7230 0x7fbd2c0a96f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd4410c0d0 0x7fbd4419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd44103660 0x7fbd44198850 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 --2- 172.21.15.5:0/1855272723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd44102d40 0x7fbd44198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:33:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/1855272723 >> 172.21.15.5:0/1855272723 conn(0x7fbd440fc6f0 msgr2=0x7fbd440fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:33:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/1855272723 shutdown_connections 2024-09-11T22:33:57.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:33:57.909+0000 7fbd4a3ed640 1 -- 172.21.15.5:0/1855272723 wait complete. 2024-09-11T22:33:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:57 smithi005 ceph-mon[30943]: pgmap v372: 97 pgs: 97 active+clean; 114 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-09-11T22:33:58.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:57 smithi159 ceph-mon[35534]: pgmap v372: 97 pgs: 97 active+clean; 114 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-09-11T22:33:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:57 smithi172 ceph-mon[35461]: pgmap v372: 97 pgs: 97 active+clean; 114 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-09-11T22:33:58.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 878337 7.25 MB/sec execute 496 sec latency 48.111 ms 2024-09-11T22:33:59.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 880534 7.26 MB/sec execute 497 sec latency 15.419 ms 2024-09-11T22:34:00.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:33:59 smithi005 ceph-mon[30943]: pgmap v373: 97 pgs: 97 active+clean; 118 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 354 op/s 2024-09-11T22:34:00.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:33:59 smithi159 ceph-mon[35534]: pgmap v373: 97 pgs: 97 active+clean; 118 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 354 op/s 2024-09-11T22:34:00.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:33:59 smithi172 ceph-mon[35461]: pgmap v373: 97 pgs: 97 active+clean; 118 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 354 op/s 2024-09-11T22:34:00.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 881929 7.26 MB/sec execute 498 sec latency 78.879 ms 2024-09-11T22:34:01.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 883732 7.26 MB/sec execute 499 sec latency 60.324 ms 2024-09-11T22:34:02.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:01 smithi005 ceph-mon[30943]: pgmap v374: 97 pgs: 97 active+clean; 118 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2024-09-11T22:34:02.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:01 smithi159 ceph-mon[35534]: pgmap v374: 97 pgs: 97 active+clean; 118 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2024-09-11T22:34:02.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:01 smithi172 ceph-mon[35461]: pgmap v374: 97 pgs: 97 active+clean; 118 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 183 op/s 2024-09-11T22:34:02.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 884696 7.26 MB/sec execute 500 sec latency 35.893 ms 2024-09-11T22:34:03.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 886950 7.27 MB/sec execute 501 sec latency 27.084 ms 2024-09-11T22:34:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:03 smithi005 ceph-mon[30943]: pgmap v375: 97 pgs: 97 active+clean; 104 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 367 op/s 2024-09-11T22:34:04.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:03 smithi159 ceph-mon[35534]: pgmap v375: 97 pgs: 97 active+clean; 104 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 367 op/s 2024-09-11T22:34:04.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:03 smithi172 ceph-mon[35461]: pgmap v375: 97 pgs: 97 active+clean; 104 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 367 op/s 2024-09-11T22:34:04.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 888284 7.27 MB/sec execute 502 sec latency 20.446 ms 2024-09-11T22:34:05.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 889522 7.26 MB/sec execute 503 sec latency 48.785 ms 2024-09-11T22:34:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:05 smithi005 ceph-mon[30943]: pgmap v376: 97 pgs: 97 active+clean; 104 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-09-11T22:34:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:05 smithi005 ceph-mon[30943]: osdmap e103: 12 total, 12 up, 12 in 2024-09-11T22:34:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:05 smithi159 ceph-mon[35534]: pgmap v376: 97 pgs: 97 active+clean; 104 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-09-11T22:34:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:05 smithi159 ceph-mon[35534]: osdmap e103: 12 total, 12 up, 12 in 2024-09-11T22:34:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:05 smithi172 ceph-mon[35461]: pgmap v376: 97 pgs: 97 active+clean; 104 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.2 MiB/s wr, 278 op/s 2024-09-11T22:34:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:05 smithi172 ceph-mon[35461]: osdmap e103: 12 total, 12 up, 12 in 2024-09-11T22:34:06.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 891198 7.27 MB/sec execute 504 sec latency 37.077 ms 2024-09-11T22:34:07.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 892535 7.27 MB/sec execute 505 sec latency 98.077 ms 2024-09-11T22:34:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:07 smithi005 ceph-mon[30943]: pgmap v378: 97 pgs: 97 active+clean; 104 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 333 op/s 2024-09-11T22:34:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:07 smithi159 ceph-mon[35534]: pgmap v378: 97 pgs: 97 active+clean; 104 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 333 op/s 2024-09-11T22:34:08.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:07 smithi172 ceph-mon[35461]: pgmap v378: 97 pgs: 97 active+clean; 104 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 333 op/s 2024-09-11T22:34:08.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 894666 7.28 MB/sec execute 506 sec latency 32.593 ms 2024-09-11T22:34:09.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 896110 7.28 MB/sec execute 507 sec latency 16.943 ms 2024-09-11T22:34:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:09 smithi005 ceph-mon[30943]: pgmap v379: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-09-11T22:34:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:09 smithi159 ceph-mon[35534]: pgmap v379: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-09-11T22:34:10.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:09 smithi172 ceph-mon[35461]: pgmap v379: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-09-11T22:34:10.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 898284 7.29 MB/sec execute 508 sec latency 14.947 ms 2024-09-11T22:34:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.009+0000 7f341681e640 1 -- 172.21.15.5:0/3085273855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410102d00 msgr2=0x7f34101030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.009+0000 7f341681e640 1 --2- 172.21.15.5:0/3085273855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410102d00 0x7f34101030e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f3400009900 tx=0x7f340002a480 comp rx=0 tx=0).stop 2024-09-11T22:34:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.009+0000 7f341681e640 1 -- 172.21.15.5:0/3085273855 shutdown_connections 2024-09-11T22:34:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.009+0000 7f341681e640 1 --2- 172.21.15.5:0/3085273855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341010c0d0 0x7f341010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.009+0000 7f341681e640 1 --2- 172.21.15.5:0/3085273855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410103620 0x7f341010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.009+0000 7f341681e640 1 --2- 172.21.15.5:0/3085273855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410102d00 0x7f34101030e0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.009+0000 7f341681e640 1 -- 172.21.15.5:0/3085273855 >> 172.21.15.5:0/3085273855 conn(0x7f3410077990 msgr2=0x7f34100fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.009+0000 7f341681e640 1 -- 172.21.15.5:0/3085273855 shutdown_connections 2024-09-11T22:34:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.009+0000 7f341681e640 1 -- 172.21.15.5:0/3085273855 wait complete. 2024-09-11T22:34:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f341681e640 1 Processor -- start 2024-09-11T22:34:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f341681e640 1 -- start start 2024-09-11T22:34:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f341681e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410102d00 0x7f3410198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f341681e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410103620 0x7f3410198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f341681e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341010c0d0 0x7f341019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f341681e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3410111eb0 con 0x7f3410103620 2024-09-11T22:34:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f341681e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3410111d30 con 0x7f3410102d00 2024-09-11T22:34:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f341681e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3410112030 con 0x7f341010c0d0 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f340f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410103620 0x7f3410198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f340ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410102d00 0x7f3410198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f3414dbb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341010c0d0 0x7f341019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f340ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410102d00 0x7f3410198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60644/0 (socket says 172.21.15.5:60644) 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f340f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410103620 0x7f3410198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34940/0 (socket says 172.21.15.5:34940) 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.010+0000 7f340ffff640 1 -- 172.21.15.5:0/1623447354 learned_addr learned my addr 172.21.15.5:0/1623447354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f340ffff640 1 -- 172.21.15.5:0/1623447354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341010c0d0 msgr2=0x7f341019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f340ffff640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341010c0d0 0x7f341019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f340ffff640 1 -- 172.21.15.5:0/1623447354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410103620 msgr2=0x7f3410198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f340ffff640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410103620 0x7f3410198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f340ffff640 1 -- 172.21.15.5:0/1623447354 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341019d310 con 0x7f3410102d00 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f340f7fe640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410103620 0x7f3410198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f340ffff640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410102d00 0x7f3410198320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3400006010 tx=0x7f3400004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f340d7fa640 1 -- 172.21.15.5:0/1623447354 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340003f030 con 0x7f3410102d00 2024-09-11T22:34:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f341681e640 1 -- 172.21.15.5:0/1623447354 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341019d5a0 con 0x7f3410102d00 2024-09-11T22:34:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f340d7fa640 1 -- 172.21.15.5:0/1623447354 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3400003ad0 con 0x7f3410102d00 2024-09-11T22:34:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f340d7fa640 1 -- 172.21.15.5:0/1623447354 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400035070 con 0x7f3410102d00 2024-09-11T22:34:11.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.011+0000 7f341681e640 1 -- 172.21.15.5:0/1623447354 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34101a4e90 con 0x7f3410102d00 2024-09-11T22:34:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.014+0000 7f340d7fa640 1 -- 172.21.15.5:0/1623447354 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3400042030 con 0x7f3410102d00 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.014+0000 7f341968b740 1 Processor -- start 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.014+0000 7f341968b740 1 -- start start 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.014+0000 7f341968b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x56010105ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.014+0000 7f341968b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 0x560101067770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f341968b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560101067cb0 0x56010106a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f341968b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560100fc12f0 con 0x56010105d410 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f341968b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560100f69270 con 0x56010105caf0 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f341968b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560100fbd2b0 con 0x560101067cb0 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f3414dbb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x56010105ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f3414dbb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x56010105ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60650/0 (socket says 172.21.15.5:60650) 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f3414dbb640 1 -- 172.21.15.5:0/1203878383 learned_addr learned my addr 172.21.15.5:0/1203878383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f340ffff640 1 --2- 172.21.15.5:0/1203878383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 0x560101067770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f340f7fe640 1 --2- 172.21.15.5:0/1203878383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560101067cb0 0x56010106a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f340d7fa640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33f80a2da0 0x7f33f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f3414dbb640 1 -- 172.21.15.5:0/1203878383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560101067cb0 msgr2=0x56010106a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f3414dbb640 1 --2- 172.21.15.5:0/1203878383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560101067cb0 0x56010106a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f3414dbb640 1 -- 172.21.15.5:0/1203878383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 msgr2=0x560101067770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f3414dbb640 1 --2- 172.21.15.5:0/1203878383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 0x560101067770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f3414dbb640 1 -- 172.21.15.5:0/1203878383 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560100f68340 con 0x56010105caf0 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f340d7fa640 1 -- 172.21.15.5:0/1623447354 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8673+0+0 (secure 0 0 0) 0x7f34000efb70 con 0x7f3410102d00 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f3414dbb640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33f80a2da0 0x7f33f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.015+0000 7f3414dbb640 1 --2- 172.21.15.5:0/1203878383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x56010105ced0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3408009fc0 tx=0x7f34080108d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c2fc0b015686316 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f33f67fc640 1 -- 172.21.15.5:0/1203878383 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408021da0 con 0x56010105caf0 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f3414dbb640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33f80a2da0 0x7f33f80a5260 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3408025000 tx=0x7f3408022000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f33f67fc640 1 -- 172.21.15.5:0/1203878383 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3408020920 con 0x56010105caf0 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f33f67fc640 1 -- 172.21.15.5:0/1203878383 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408011800 con 0x56010105caf0 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f341968b740 1 -- 172.21.15.5:0/1203878383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 msgr2=0x56010105ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f341968b740 1 --2- 172.21.15.5:0/1203878383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x56010105ced0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3408009fc0 tx=0x7f34080108d0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f341968b740 1 -- 172.21.15.5:0/1203878383 shutdown_connections 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f341968b740 1 --2- 172.21.15.5:0/1203878383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560101067cb0 0x56010106a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f341968b740 1 --2- 172.21.15.5:0/1203878383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 0x560101067770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f341968b740 1 --2- 172.21.15.5:0/1203878383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x56010105ced0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.016+0000 7f341968b740 1 -- 172.21.15.5:0/1203878383 >> 172.21.15.5:0/1203878383 conn(0x560101059060 msgr2=0x560101059440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.017+0000 7f341968b740 1 -- 172.21.15.5:0/1203878383 shutdown_connections 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.017+0000 7f341968b740 1 -- 172.21.15.5:0/1203878383 wait complete. 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.017+0000 7f341968b740 1 -- start start 2024-09-11T22:34:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.017+0000 7f341968b740 1 Processor -- start 2024-09-11T22:34:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.017+0000 7f341968b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.017+0000 7f341968b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 0x5601010650f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.017+0000 7f341968b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560101067cb0 0x560101065630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f341968b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56010106a880 con 0x56010105d410 2024-09-11T22:34:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f341968b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560100f4a290 con 0x56010105caf0 2024-09-11T22:34:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f341968b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56010106aa00 con 0x560101067cb0 2024-09-11T22:34:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f341968b740 10 client.0 _ll_register_callbacks cb 0x560100f7b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:11.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f340f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 0x5601010650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f340ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f340f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 0x5601010650f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34962/0 (socket says 172.21.15.5:34962) 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f340f7fe640 1 -- 172.21.15.5:0/3634238121 learned_addr learned my addr 172.21.15.5:0/3634238121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f3414dbb640 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560101067cb0 0x560101065630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f340ffff640 1 -- 172.21.15.5:0/3634238121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560101067cb0 msgr2=0x560101065630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f340ffff640 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560101067cb0 0x560101065630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f340ffff640 1 -- 172.21.15.5:0/3634238121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 msgr2=0x5601010650f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f340ffff640 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 0x5601010650f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f340ffff640 1 -- 172.21.15.5:0/3634238121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560100cd4210 con 0x56010105caf0 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f340ffff640 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101064bb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f34000ed1f0 tx=0x7f34000f32b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce9f4466ba0d9d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f33ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f33ef7fe640 1 -- 172.21.15.5:0/3634238121 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400049070 con 0x56010105caf0 2024-09-11T22:34:11.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f33ef7fe640 1 -- 172.21.15.5:0/3634238121 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8673+0+0 (secure 0 0 0) 0x7f34000ed4a0 con 0x56010105caf0 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.018+0000 7f341968b740 1 -- 172.21.15.5:0/3634238121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3408020920 con 0x56010105caf0 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f33ef7fe640 1 -- 172.21.15.5:0/3634238121 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f34000bbca0 con 0x56010105caf0 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f341968b740 10 client.28340 fetch_fsmap learned FSMap version 45 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f341968b740 1 -- 172.21.15.5:0/3634238121 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560100da0620 con 0x56010105caf0 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f33ef7fe640 1 -- 172.21.15.5:0/3634238121 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3408020920 con 0x56010105caf0 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f33ef7fe640 1 -- 172.21.15.5:0/3634238121 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f34000b87d0 con 0x56010105caf0 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f341968b740 10 client.28340 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f341968b740 10 client.28340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f341968b740 20 client.28340 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f341968b740 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560101077c60 0x56010107a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f341968b740 4 client.28340 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.019+0000 7f341968b740 1 -- 172.21.15.5:0/3634238121 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560100fcea20 con 0x560101077c60 2024-09-11T22:34:11.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.020+0000 7f340f7fe640 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560101077c60 0x56010107a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.020+0000 7f33ef7fe640 10 client.28340 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.020+0000 7f340f7fe640 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560101077c60 0x56010107a100 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709db775c27a25df server_cookie=eb22d60ecb9f0f04 in_seq=0 out_seq=0 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f33ef7fe640 1 -- 172.21.15.5:0/3634238121 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560100fcea20 con 0x560101077c60 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f33ef7fe640 10 client.28340 handle_command_reply: tid=0 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 client.28340 shutdown 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 -- 172.21.15.5:0/3634238121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 msgr2=0x560101064bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101064bb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f34000ed1f0 tx=0x7f34000f32b0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 -- 172.21.15.5:0/3634238121 shutdown_connections 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560101077c60 0x56010107a100 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560101067cb0 0x560101065630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010105d410 0x5601010650f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 --2- 172.21.15.5:0/3634238121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101064bb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f33ef7fe640 0 client.28340 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 -- 172.21.15.5:0/3634238121 >> 172.21.15.5:0/3634238121 conn(0x560101059060 msgr2=0x56010105edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 -- 172.21.15.5:0/3634238121 shutdown_connections 2024-09-11T22:34:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 1 -- 172.21.15.5:0/3634238121 wait complete. 2024-09-11T22:34:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.021+0000 7f341968b740 20 client.28340 trim_cache size 0 max 16384 2024-09-11T22:34:11.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.039+0000 7f341968b740 1 Processor -- start 2024-09-11T22:34:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.039+0000 7f341968b740 1 -- start start 2024-09-11T22:34:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f341968b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101066ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f341968b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 0x560101065460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f341968b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 0x56010107f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f341968b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34101ab810 con 0x560101067cb0 2024-09-11T22:34:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f341968b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34101ad350 con 0x56010105caf0 2024-09-11T22:34:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f341968b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34101abee0 con 0x56010105d410 2024-09-11T22:34:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f3414dbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 0x56010107f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f340ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101066ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f340f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 0x560101065460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f340ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101066ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60668/0 (socket says 172.21.15.5:60668) 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f340ffff640 1 -- 172.21.15.5:0/182238016 learned_addr learned my addr 172.21.15.5:0/182238016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f340ffff640 1 -- 172.21.15.5:0/182238016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 msgr2=0x560101065460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f340ffff640 1 --2- 172.21.15.5:0/182238016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 0x560101065460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f340ffff640 1 -- 172.21.15.5:0/182238016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 msgr2=0x56010107f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f340ffff640 1 --2- 172.21.15.5:0/182238016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 0x56010107f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f340ffff640 1 -- 172.21.15.5:0/182238016 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56010105d240 con 0x56010105caf0 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.040+0000 7f340ffff640 1 --2- 172.21.15.5:0/182238016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101066ee0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f34000ed820 tx=0x7f34000ed240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9673e6904c7efd24 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f33f5ffb640 1 -- 172.21.15.5:0/182238016 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400049070 con 0x56010105caf0 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f33f5ffb640 1 -- 172.21.15.5:0/182238016 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f340003b460 con 0x56010105caf0 2024-09-11T22:34:11.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f33f5ffb640 1 -- 172.21.15.5:0/182238016 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34000bbac0 con 0x56010105caf0 2024-09-11T22:34:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f341968b740 1 -- 172.21.15.5:0/182238016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 msgr2=0x560101066ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f341968b740 1 --2- 172.21.15.5:0/182238016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101066ee0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f34000ed820 tx=0x7f34000ed240 comp rx=0 tx=0).stop 2024-09-11T22:34:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f341968b740 1 -- 172.21.15.5:0/182238016 shutdown_connections 2024-09-11T22:34:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f341968b740 1 --2- 172.21.15.5:0/182238016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 0x56010107f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f341968b740 1 --2- 172.21.15.5:0/182238016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 0x560101065460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f341968b740 1 --2- 172.21.15.5:0/182238016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101066ee0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f341968b740 1 -- 172.21.15.5:0/182238016 >> 172.21.15.5:0/182238016 conn(0x560101059060 msgr2=0x5601010581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f341968b740 1 -- 172.21.15.5:0/182238016 shutdown_connections 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.041+0000 7f341968b740 1 -- 172.21.15.5:0/182238016 wait complete. 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.042+0000 7f341968b740 1 -- start start 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.042+0000 7f341968b740 1 Processor -- start 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.042+0000 7f341968b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101061b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.042+0000 7f341968b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 0x560101062060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.042+0000 7f341968b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 0x5601010625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.042+0000 7f341968b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34101acc80 con 0x560101067cb0 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.042+0000 7f341968b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34101ac5b0 con 0x56010105caf0 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.042+0000 7f341968b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34101a9810 con 0x56010105d410 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.042+0000 7f341968b740 10 client.0 _ll_register_callbacks cb 0x7f34000f33b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f3414dbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 0x5601010625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f340f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 0x560101062060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f3414dbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 0x5601010625a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34984/0 (socket says 172.21.15.5:34984) 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f3414dbb640 1 -- 172.21.15.5:0/2662472571 learned_addr learned my addr 172.21.15.5:0/2662472571 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f340ffff640 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101061b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f3414dbb640 1 -- 172.21.15.5:0/2662472571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 msgr2=0x560101062060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f3414dbb640 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 0x560101062060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f3414dbb640 1 -- 172.21.15.5:0/2662472571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 msgr2=0x560101061b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f3414dbb640 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101061b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f3414dbb640 1 -- 172.21.15.5:0/2662472571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601010e19a0 con 0x560101067cb0 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f340ffff640 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101061b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f340f7fe640 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 0x560101062060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f3414dbb640 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 0x5601010625a0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3408020fd0 tx=0x7f3408017c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536471a98133f3a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f33ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f33ef7fe640 1 -- 172.21.15.5:0/2662472571 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340803d060 con 0x560101067cb0 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f33ef7fe640 1 -- 172.21.15.5:0/2662472571 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8673+0+0 (secure 0 0 0) 0x7f3408011720 con 0x560101067cb0 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.043+0000 7f341968b740 1 -- 172.21.15.5:0/2662472571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560100fc8d70 con 0x560101067cb0 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f33ef7fe640 1 -- 172.21.15.5:0/2662472571 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3408044050 con 0x560101067cb0 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f341968b740 10 client.19134 fetch_fsmap learned FSMap version 45 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f341968b740 1 -- 172.21.15.5:0/2662472571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5601010e2be0 con 0x560101067cb0 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f33ef7fe640 1 -- 172.21.15.5:0/2662472571 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560100fc8d70 con 0x560101067cb0 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f33ef7fe640 1 -- 172.21.15.5:0/2662472571 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f340800f040 con 0x560101067cb0 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f341968b740 10 client.19134 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f341968b740 10 client.19134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f341968b740 20 client.19134 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f341968b740 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5601010e3190 0x5601010e55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f341968b740 4 client.19134 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:34:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f341968b740 1 -- 172.21.15.5:0/2662472571 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560100f45900 con 0x5601010e3190 2024-09-11T22:34:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.044+0000 7f340ffff640 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5601010e3190 0x5601010e55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.045+0000 7f33ef7fe640 10 client.19134 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:11.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.045+0000 7f340ffff640 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5601010e3190 0x5601010e55b0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe7346b08aa9924 server_cookie=725470d442271872 in_seq=0 out_seq=0 2024-09-11T22:34:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.177+0000 7f33ef7fe640 1 -- 172.21.15.5:0/2662472571 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560100f45900 con 0x5601010e3190 2024-09-11T22:34:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.177+0000 7f33ef7fe640 10 client.19134 handle_command_reply: tid=0 2024-09-11T22:34:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.177+0000 7f341968b740 1 client.19134 shutdown 2024-09-11T22:34:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.177+0000 7f341968b740 1 -- 172.21.15.5:0/2662472571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 msgr2=0x5601010625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.177+0000 7f341968b740 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 0x5601010625a0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f3408020fd0 tx=0x7f3408017c00 comp rx=0 tx=0).stop 2024-09-11T22:34:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.177+0000 7f341968b740 1 -- 172.21.15.5:0/2662472571 shutdown_connections 2024-09-11T22:34:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.177+0000 7f341968b740 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5601010e3190 0x5601010e55b0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.178+0000 7f341968b740 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560101067cb0 0x5601010625a0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.178+0000 7f33ef7fe640 0 client.19134 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.178+0000 7f341968b740 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56010105d410 0x560101062060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.178+0000 7f341968b740 1 --2- 172.21.15.5:0/2662472571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56010105caf0 0x560101061b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.178+0000 7f341968b740 1 -- 172.21.15.5:0/2662472571 >> 172.21.15.5:0/2662472571 conn(0x560101059060 msgr2=0x5601010581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.178+0000 7f341968b740 1 -- 172.21.15.5:0/2662472571 shutdown_connections 2024-09-11T22:34:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.178+0000 7f341968b740 1 -- 172.21.15.5:0/2662472571 wait complete. 2024-09-11T22:34:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.178+0000 7f341968b740 20 client.19134 trim_cache size 0 max 16384 2024-09-11T22:34:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:34:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:34:11.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:34:11.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:34:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 -- 172.21.15.5:0/1623447354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33f80a2da0 msgr2=0x7f33f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33f80a2da0 0x7f33f80a5260 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3408025000 tx=0x7f3408022000 comp rx=0 tx=0).stop 2024-09-11T22:34:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 -- 172.21.15.5:0/1623447354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410102d00 msgr2=0x7f3410198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410102d00 0x7f3410198320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3400006010 tx=0x7f3400004270 comp rx=0 tx=0).stop 2024-09-11T22:34:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 -- 172.21.15.5:0/1623447354 shutdown_connections 2024-09-11T22:34:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f341010c0d0 0x7f341019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3410103620 0x7f3410198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f33f80a2da0 0x7f33f80a5260 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 --2- 172.21.15.5:0/1623447354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3410102d00 0x7f3410198320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:11.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 -- 172.21.15.5:0/1623447354 >> 172.21.15.5:0/1623447354 conn(0x7f3410077990 msgr2=0x7f34100fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 -- 172.21.15.5:0/1623447354 shutdown_connections 2024-09-11T22:34:11.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:11.179+0000 7f341681e640 1 -- 172.21.15.5:0/1623447354 wait complete. 2024-09-11T22:34:11.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 899584 7.29 MB/sec execute 509 sec latency 55.453 ms 2024-09-11T22:34:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:11 smithi005 ceph-mon[30943]: pgmap v380: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-09-11T22:34:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:11 smithi159 ceph-mon[35534]: pgmap v380: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-09-11T22:34:12.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:11 smithi172 ceph-mon[35461]: pgmap v380: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 321 op/s 2024-09-11T22:34:12.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 901752 7.30 MB/sec execute 510 sec latency 39.466 ms 2024-09-11T22:34:13.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 902927 7.29 MB/sec execute 511 sec latency 90.577 ms 2024-09-11T22:34:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:13 smithi005 ceph-mon[30943]: pgmap v381: 97 pgs: 97 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 306 op/s 2024-09-11T22:34:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:13 smithi005 ceph-mon[30943]: osdmap e104: 12 total, 12 up, 12 in 2024-09-11T22:34:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:13 smithi159 ceph-mon[35534]: pgmap v381: 97 pgs: 97 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 306 op/s 2024-09-11T22:34:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:13 smithi159 ceph-mon[35534]: osdmap e104: 12 total, 12 up, 12 in 2024-09-11T22:34:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:13 smithi172 ceph-mon[35461]: pgmap v381: 97 pgs: 97 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 306 op/s 2024-09-11T22:34:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:13 smithi172 ceph-mon[35461]: osdmap e104: 12 total, 12 up, 12 in 2024-09-11T22:34:14.344 INFO:tasks.workunit.client.0.smithi005.stdout: 1 905256 7.31 MB/sec execute 512 sec latency 26.783 ms 2024-09-11T22:34:15.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 906240 7.30 MB/sec execute 513 sec latency 23.090 ms 2024-09-11T22:34:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:15 smithi005 ceph-mon[30943]: pgmap v383: 97 pgs: 97 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-11T22:34:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:15 smithi159 ceph-mon[35534]: pgmap v383: 97 pgs: 97 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-11T22:34:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:15 smithi172 ceph-mon[35461]: pgmap v383: 97 pgs: 97 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-11T22:34:16.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 908780 7.32 MB/sec execute 514 sec latency 18.653 ms 2024-09-11T22:34:16.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:34:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:17.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.298+0000 7f27976a6640 1 -- 172.21.15.5:0/1851225536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2790101de0 msgr2=0x7f27901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:17.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.298+0000 7f27976a6640 1 --2- 172.21.15.5:0/1851225536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2790101de0 0x7f27901021c0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f278c009900 tx=0x7f278c02a410 comp rx=0 tx=0).stop 2024-09-11T22:34:17.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.299+0000 7f27976a6640 1 -- 172.21.15.5:0/1851225536 shutdown_connections 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.299+0000 7f27976a6640 1 --2- 172.21.15.5:0/1851225536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790106f10 0x7f279010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.299+0000 7f27976a6640 1 --2- 172.21.15.5:0/1851225536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2790102790 0x7f27901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.299+0000 7f27976a6640 1 --2- 172.21.15.5:0/1851225536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2790101de0 0x7f27901021c0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.299+0000 7f27976a6640 1 -- 172.21.15.5:0/1851225536 >> 172.21.15.5:0/1851225536 conn(0x7f27900fd8c0 msgr2=0x7f27900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.299+0000 7f27976a6640 1 -- 172.21.15.5:0/1851225536 shutdown_connections 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.299+0000 7f27976a6640 1 -- 172.21.15.5:0/1851225536 wait complete. 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.299+0000 7f27976a6640 1 Processor -- start 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.299+0000 7f27976a6640 1 -- start start 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f27976a6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2790101de0 0x7f27901984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f27976a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790102790 0x7f2790198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f27976a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2790106f10 0x7f279019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f27976a6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2790111f70 con 0x7f2790106f10 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f27976a6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2790111df0 con 0x7f2790101de0 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f27976a6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27901120f0 con 0x7f2790102790 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f2795c43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2790106f10 0x7f279019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f2795c43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2790106f10 0x7f279019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43994/0 (socket says 172.21.15.5:43994) 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f2795c43640 1 -- 172.21.15.5:0/2305844329 learned_addr learned my addr 172.21.15.5:0/2305844329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f2795442640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2790101de0 0x7f27901984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f2794c41640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790102790 0x7f2790198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f2795c43640 1 -- 172.21.15.5:0/2305844329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790102790 msgr2=0x7f2790198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f2795c43640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790102790 0x7f2790198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f2795c43640 1 -- 172.21.15.5:0/2305844329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2790101de0 msgr2=0x7f27901984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f2795c43640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2790101de0 0x7f27901984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.300+0000 7f2795c43640 1 -- 172.21.15.5:0/2305844329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279019d480 con 0x7f2790106f10 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.301+0000 7f2795c43640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2790106f10 0x7f279019cda0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f278800eaa0 tx=0x7f278800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.301+0000 7f27867fc640 1 -- 172.21.15.5:0/2305844329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278800cdb0 con 0x7f2790106f10 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.301+0000 7f27976a6640 1 -- 172.21.15.5:0/2305844329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279019d770 con 0x7f2790106f10 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.301+0000 7f27867fc640 1 -- 172.21.15.5:0/2305844329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2788005020 con 0x7f2790106f10 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.301+0000 7f27867fc640 1 -- 172.21.15.5:0/2305844329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27880116d0 con 0x7f2790106f10 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.301+0000 7f27976a6640 1 -- 172.21.15.5:0/2305844329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901a5050 con 0x7f2790106f10 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.303+0000 7f27867fc640 1 -- 172.21.15.5:0/2305844329 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f27880054d0 con 0x7f2790106f10 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2799a7f740 1 Processor -- start 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2799a7f740 1 -- start start 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f27867fc640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f277c0a2cf0 0x7f277c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64227760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a64236550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a64238e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2799a7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a6418d2e0 con 0x562a64227380 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2799a7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a64134ce0 con 0x562a64227ca0 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2799a7f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562a641892a0 con 0x562a64236a90 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f27867fc640 1 -- 172.21.15.5:0/2305844329 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f27880cf610 con 0x7f2790106f10 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2794c41640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f277c0a2cf0 0x7f277c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2795442640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64227760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2794c41640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a64236550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2795442640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64227760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43998/0 (socket says 172.21.15.5:43998) 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2795442640 1 -- 172.21.15.5:0/4206465293 learned_addr learned my addr 172.21.15.5:0/4206465293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.304+0000 7f2795c43640 1 --2- 172.21.15.5:0/4206465293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a64238e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.305+0000 7f2795442640 1 -- 172.21.15.5:0/4206465293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 msgr2=0x562a64238e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.305+0000 7f2795442640 1 --2- 172.21.15.5:0/4206465293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a64238e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.305+0000 7f2795442640 1 -- 172.21.15.5:0/4206465293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 msgr2=0x562a64236550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.305+0000 7f2795442640 1 --2- 172.21.15.5:0/4206465293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a64236550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.305+0000 7f2795442640 1 -- 172.21.15.5:0/4206465293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a64119440 con 0x562a64227380 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.305+0000 7f2794c41640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f277c0a2cf0 0x7f277c0a51b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2790108900 tx=0x7f2780009b60 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.305+0000 7f2795442640 1 --2- 172.21.15.5:0/4206465293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64227760 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f278c009900 tx=0x7f278c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ccf5df1111ed76 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.305+0000 7f277b7fe640 1 -- 172.21.15.5:0/4206465293 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c040070 con 0x562a64227380 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.305+0000 7f277b7fe640 1 -- 172.21.15.5:0/4206465293 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f278c0318f0 con 0x562a64227380 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.305+0000 7f277b7fe640 1 -- 172.21.15.5:0/4206465293 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c035040 con 0x562a64227380 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.306+0000 7f2799a7f740 1 -- 172.21.15.5:0/4206465293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 msgr2=0x562a64227760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.306+0000 7f2799a7f740 1 --2- 172.21.15.5:0/4206465293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64227760 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f278c009900 tx=0x7f278c005e00 comp rx=0 tx=0).stop 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.306+0000 7f2799a7f740 1 -- 172.21.15.5:0/4206465293 shutdown_connections 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.306+0000 7f2799a7f740 1 --2- 172.21.15.5:0/4206465293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a64238e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.306+0000 7f2799a7f740 1 --2- 172.21.15.5:0/4206465293 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a64236550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.306+0000 7f2799a7f740 1 --2- 172.21.15.5:0/4206465293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64227760 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.306+0000 7f2799a7f740 1 -- 172.21.15.5:0/4206465293 >> 172.21.15.5:0/4206465293 conn(0x562a64223a30 msgr2=0x562a64223e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.306+0000 7f2799a7f740 1 -- 172.21.15.5:0/4206465293 shutdown_connections 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.306+0000 7f2799a7f740 1 -- 172.21.15.5:0/4206465293 wait complete. 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.307+0000 7f2799a7f740 1 -- start start 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.307+0000 7f2799a7f740 1 Processor -- start 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.307+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64236330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.307+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a642335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.307+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a64233b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.307+0000 7f2799a7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a64239660 con 0x562a64227380 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.307+0000 7f2799a7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a64148050 con 0x562a64227ca0 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.307+0000 7f2799a7f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562a642397e0 con 0x562a64236a90 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.307+0000 7f2799a7f740 10 client.0 _ll_register_callbacks cb 0x562a64188160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:17.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795442640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64236330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2794c41640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a642335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795442640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64236330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44004/0 (socket says 172.21.15.5:44004) 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795442640 1 -- 172.21.15.5:0/3619390448 learned_addr learned my addr 172.21.15.5:0/3619390448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795c43640 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a64233b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795442640 1 -- 172.21.15.5:0/3619390448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 msgr2=0x562a64233b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795442640 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a64233b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795442640 1 -- 172.21.15.5:0/3619390448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 msgr2=0x562a642335e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795442640 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a642335e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795442640 1 -- 172.21.15.5:0/3619390448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a63efe3c0 con 0x562a64227380 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795c43640 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a64233b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2794c41640 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a642335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2795442640 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64236330 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f278c031440 tx=0x7f278c0052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda784104b04121f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:17.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2778ff9640 1 -- 172.21.15.5:0/3619390448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c04b070 con 0x562a64227380 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2778ff9640 1 -- 172.21.15.5:0/3619390448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f278c004250 con 0x562a64227380 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2799a7f740 1 -- 172.21.15.5:0/3619390448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f278c0318f0 con 0x562a64227380 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.308+0000 7f2778ff9640 1 -- 172.21.15.5:0/3619390448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f278c057050 con 0x562a64227380 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2799a7f740 10 client.19146 fetch_fsmap learned FSMap version 45 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2799a7f740 1 -- 172.21.15.5:0/3619390448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562a64244fc0 con 0x562a64227380 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2778ff9640 1 -- 172.21.15.5:0/3619390448 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f278c0318f0 con 0x562a64227380 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2778ff9640 1 -- 172.21.15.5:0/3619390448 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f278c0035f0 con 0x562a64227380 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2799a7f740 10 client.19146 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2799a7f740 10 client.19146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2799a7f740 20 client.19146 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2799a7f740 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a64246310 0x562a64248700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2799a7f740 4 client.19146 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:17.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2799a7f740 1 -- 172.21.15.5:0/3619390448 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a6419a850 con 0x562a64246310 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.309+0000 7f2794c41640 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a64246310 0x562a64248700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.310+0000 7f2794c41640 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a64246310 0x562a64248700 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b530b0d5ca9366e8 server_cookie=45787dfb3d4d2969 in_seq=0 out_seq=0 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.310+0000 7f2778ff9640 10 client.19146 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2778ff9640 1 -- 172.21.15.5:0/3619390448 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a6419a850 con 0x562a64246310 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2778ff9640 10 client.19146 handle_command_reply: tid=0 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2799a7f740 1 client.19146 shutdown 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2799a7f740 1 -- 172.21.15.5:0/3619390448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 msgr2=0x562a64236330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2799a7f740 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64236330 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f278c031440 tx=0x7f278c0052d0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2799a7f740 1 -- 172.21.15.5:0/3619390448 shutdown_connections 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2799a7f740 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a64246310 0x562a64248700 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2799a7f740 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a64233b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2799a7f740 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a642335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2799a7f740 1 --2- 172.21.15.5:0/3619390448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a64236330 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2799a7f740 1 -- 172.21.15.5:0/3619390448 >> 172.21.15.5:0/3619390448 conn(0x562a64223a30 msgr2=0x562a64229790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2778ff9640 0 client.19146 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.311+0000 7f2799a7f740 1 -- 172.21.15.5:0/3619390448 shutdown_connections 2024-09-11T22:34:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.312+0000 7f2799a7f740 1 -- 172.21.15.5:0/3619390448 wait complete. 2024-09-11T22:34:17.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.312+0000 7f2799a7f740 20 client.19146 trim_cache size 0 max 16384 2024-09-11T22:34:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2799a7f740 1 Processor -- start 2024-09-11T22:34:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2799a7f740 1 -- start start 2024-09-11T22:34:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227380 0x562a64242900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64227ca0 0x562a64231eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64236a90 0x562a6422ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2799a7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27901ab910 con 0x562a64236a90 2024-09-11T22:34:17.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2799a7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27901ad450 con 0x562a64227380 2024-09-11T22:34:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2799a7f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27901abfe0 con 0x562a64227ca0 2024-09-11T22:34:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2795c43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64236a90 0x562a6422ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2795442640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227380 0x562a64242900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2795442640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227380 0x562a64242900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35112/0 (socket says 172.21.15.5:35112) 2024-09-11T22:34:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2794c41640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64227ca0 0x562a64231eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.332+0000 7f2795442640 1 -- 172.21.15.5:0/4221704965 learned_addr learned my addr 172.21.15.5:0/4221704965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f2794c41640 1 -- 172.21.15.5:0/4221704965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227380 msgr2=0x562a64242900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f2794c41640 1 --2- 172.21.15.5:0/4221704965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227380 0x562a64242900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f2794c41640 1 -- 172.21.15.5:0/4221704965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64236a90 msgr2=0x562a6422ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f2794c41640 1 --2- 172.21.15.5:0/4221704965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64236a90 0x562a6422ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f2794c41640 1 -- 172.21.15.5:0/4221704965 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a64232590 con 0x562a64227ca0 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f2795442640 1 --2- 172.21.15.5:0/4221704965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227380 0x562a64242900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f2795c43640 1 --2- 172.21.15.5:0/4221704965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64236a90 0x562a6422ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f2794c41640 1 --2- 172.21.15.5:0/4221704965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64227ca0 0x562a64231eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x562a64234b90 tx=0x7f2780021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd6c0a70aa95f938 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f277affd640 1 -- 172.21.15.5:0/4221704965 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800097a0 con 0x562a64227ca0 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f277affd640 1 -- 172.21.15.5:0/4221704965 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2780005500 con 0x562a64227ca0 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.333+0000 7f277affd640 1 -- 172.21.15.5:0/4221704965 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780004c40 con 0x562a64227ca0 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.334+0000 7f2799a7f740 1 -- 172.21.15.5:0/4221704965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64227ca0 msgr2=0x562a64231eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.334+0000 7f2799a7f740 1 --2- 172.21.15.5:0/4221704965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64227ca0 0x562a64231eb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x562a64234b90 tx=0x7f2780021000 comp rx=0 tx=0).stop 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.334+0000 7f2799a7f740 1 -- 172.21.15.5:0/4221704965 shutdown_connections 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.334+0000 7f2799a7f740 1 --2- 172.21.15.5:0/4221704965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64236a90 0x562a6422ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.334+0000 7f2799a7f740 1 --2- 172.21.15.5:0/4221704965 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64227ca0 0x562a64231eb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.334+0000 7f2799a7f740 1 --2- 172.21.15.5:0/4221704965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227380 0x562a64242900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.334+0000 7f2799a7f740 1 -- 172.21.15.5:0/4221704965 >> 172.21.15.5:0/4221704965 conn(0x562a64223a30 msgr2=0x562a6422e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.334+0000 7f2799a7f740 1 -- 172.21.15.5:0/4221704965 shutdown_connections 2024-09-11T22:34:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.334+0000 7f2799a7f740 1 -- 172.21.15.5:0/4221704965 wait complete. 2024-09-11T22:34:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2799a7f740 1 -- start start 2024-09-11T22:34:17.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2799a7f740 1 Processor -- start 2024-09-11T22:34:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a6423ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a6423f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2799a7f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a6423a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2799a7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27901acd80 con 0x562a64227380 2024-09-11T22:34:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2799a7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27901ac6b0 con 0x562a64227ca0 2024-09-11T22:34:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2799a7f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27901a9910 con 0x562a64236a90 2024-09-11T22:34:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2799a7f740 10 client.0 _ll_register_callbacks cb 0x7f27800114c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2795442640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a6423ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2795442640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a6423ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44012/0 (socket says 172.21.15.5:44012) 2024-09-11T22:34:17.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2795442640 1 -- 172.21.15.5:0/2151691051 learned_addr learned my addr 172.21.15.5:0/2151691051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.335+0000 7f2795c43640 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a6423a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2794c41640 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a6423f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2795442640 1 -- 172.21.15.5:0/2151691051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 msgr2=0x562a6423a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2795442640 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a6423a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2795442640 1 -- 172.21.15.5:0/2151691051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 msgr2=0x562a6423f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2795442640 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a6423f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2795442640 1 -- 172.21.15.5:0/2151691051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a642366a0 con 0x562a64227380 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2794c41640 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a6423f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2795442640 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a6423ed50 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f278c031440 tx=0x7f278c005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e523f63da4eb90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2778ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2778ff9640 1 -- 172.21.15.5:0/2151691051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c04b040 con 0x562a64227380 2024-09-11T22:34:17.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2778ff9640 1 -- 172.21.15.5:0/2151691051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f278c003e00 con 0x562a64227380 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2799a7f740 1 -- 172.21.15.5:0/2151691051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2780005500 con 0x562a64227380 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2778ff9640 1 -- 172.21.15.5:0/2151691051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f278c035030 con 0x562a64227380 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2799a7f740 10 client.19152 fetch_fsmap learned FSMap version 45 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.336+0000 7f2799a7f740 1 -- 172.21.15.5:0/2151691051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x562a642b06c0 con 0x562a64227380 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2778ff9640 1 -- 172.21.15.5:0/2151691051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2780005500 con 0x562a64227380 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2778ff9640 1 -- 172.21.15.5:0/2151691051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f278c004900 con 0x562a64227380 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2799a7f740 10 client.19152 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2799a7f740 10 client.19152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2799a7f740 20 client.19152 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2799a7f740 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a642b0d30 0x562a642b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2799a7f740 4 client.19152 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2799a7f740 1 -- 172.21.15.5:0/2151691051 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562a64194d60 con 0x562a642b0d30 2024-09-11T22:34:17.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2794c41640 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a642b0d30 0x562a642b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2794c41640 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a642b0d30 0x562a642b3120 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30bdd3d73a78330 server_cookie=bc21d601324d6f1a in_seq=0 out_seq=0 2024-09-11T22:34:17.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.337+0000 7f2778ff9640 10 client.19152 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.339+0000 7f2778ff9640 1 -- 172.21.15.5:0/2151691051 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562a64194d60 con 0x562a642b0d30 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.339+0000 7f2778ff9640 10 client.19152 handle_command_reply: tid=0 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.339+0000 7f2799a7f740 1 client.19152 shutdown 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.339+0000 7f2799a7f740 1 -- 172.21.15.5:0/2151691051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 msgr2=0x562a6423ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.339+0000 7f2799a7f740 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a6423ed50 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f278c031440 tx=0x7f278c005340 comp rx=0 tx=0).stop 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.339+0000 7f2799a7f740 1 -- 172.21.15.5:0/2151691051 shutdown_connections 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.339+0000 7f2799a7f740 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a642b0d30 0x562a642b3120 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f2799a7f740 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a64236a90 0x562a6423a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f2799a7f740 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a64227ca0 0x562a6423f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f2778ff9640 0 client.19152 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f2799a7f740 1 --2- 172.21.15.5:0/2151691051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a64227380 0x562a6423ed50 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f2799a7f740 1 -- 172.21.15.5:0/2151691051 >> 172.21.15.5:0/2151691051 conn(0x562a64223a30 msgr2=0x562a6422e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f2799a7f740 1 -- 172.21.15.5:0/2151691051 shutdown_connections 2024-09-11T22:34:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f2799a7f740 1 -- 172.21.15.5:0/2151691051 wait complete. 2024-09-11T22:34:17.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f2799a7f740 20 client.19152 trim_cache size 0 max 16384 2024-09-11T22:34:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:34:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:34:17.344 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:34:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:34:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f27976a6640 1 -- 172.21.15.5:0/2305844329 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f277c0a2cf0 msgr2=0x7f277c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f27976a6640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f277c0a2cf0 0x7f277c0a51b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2790108900 tx=0x7f2780009b60 comp rx=0 tx=0).stop 2024-09-11T22:34:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f27976a6640 1 -- 172.21.15.5:0/2305844329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2790106f10 msgr2=0x7f279019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.340+0000 7f27976a6640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2790106f10 0x7f279019cda0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f278800eaa0 tx=0x7f278800ef60 comp rx=0 tx=0).stop 2024-09-11T22:34:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.341+0000 7f27976a6640 1 -- 172.21.15.5:0/2305844329 shutdown_connections 2024-09-11T22:34:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.341+0000 7f27976a6640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2790106f10 0x7f279019cda0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.341+0000 7f27976a6640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2790102790 0x7f2790198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.341+0000 7f27976a6640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f277c0a2cf0 0x7f277c0a51b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.341+0000 7f27976a6640 1 --2- 172.21.15.5:0/2305844329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2790101de0 0x7f27901984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:17.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.341+0000 7f27976a6640 1 -- 172.21.15.5:0/2305844329 >> 172.21.15.5:0/2305844329 conn(0x7f27900fd8c0 msgr2=0x7f27900fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:17.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.341+0000 7f27976a6640 1 -- 172.21.15.5:0/2305844329 shutdown_connections 2024-09-11T22:34:17.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:17.341+0000 7f27976a6640 1 -- 172.21.15.5:0/2305844329 wait complete. 2024-09-11T22:34:17.346 INFO:tasks.workunit.client.0.smithi005.stdout: 1 909820 7.31 MB/sec execute 515 sec latency 31.765 ms 2024-09-11T22:34:17.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:17 smithi005 ceph-mon[30943]: pgmap v384: 97 pgs: 97 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 306 op/s 2024-09-11T22:34:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:17 smithi159 ceph-mon[35534]: pgmap v384: 97 pgs: 97 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 306 op/s 2024-09-11T22:34:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:17 smithi172 ceph-mon[35461]: pgmap v384: 97 pgs: 97 active+clean; 80 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.1 MiB/s wr, 306 op/s 2024-09-11T22:34:18.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 911820 7.31 MB/sec execute 516 sec latency 44.074 ms 2024-09-11T22:34:18.401 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:34:18.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:34:18.673 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:19.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 913114 7.31 MB/sec execute 517 sec latency 17.605 ms 2024-09-11T22:34:19.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.331+0000 7fc524835640 1 -- 172.21.15.5:0/3730876734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5201036e0 msgr2=0x7fc520107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.331+0000 7fc524835640 1 --2- 172.21.15.5:0/3730876734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5201036e0 0x7fc520107b90 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fc508009930 tx=0x7fc50802a420 comp rx=0 tx=0).stop 2024-09-11T22:34:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.331+0000 7fc524835640 1 -- 172.21.15.5:0/3730876734 shutdown_connections 2024-09-11T22:34:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.331+0000 7fc524835640 1 --2- 172.21.15.5:0/3730876734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5201082a0 0x7fc52010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.331+0000 7fc524835640 1 --2- 172.21.15.5:0/3730876734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5201036e0 0x7fc520107b90 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.331+0000 7fc524835640 1 --2- 172.21.15.5:0/3730876734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc520102d30 0x7fc520103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.331+0000 7fc524835640 1 -- 172.21.15.5:0/3730876734 >> 172.21.15.5:0/3730876734 conn(0x7fc5200fc7d0 msgr2=0x7fc5200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.331+0000 7fc524835640 1 -- 172.21.15.5:0/3730876734 shutdown_connections 2024-09-11T22:34:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.331+0000 7fc524835640 1 -- 172.21.15.5:0/3730876734 wait complete. 2024-09-11T22:34:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc524835640 1 Processor -- start 2024-09-11T22:34:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc524835640 1 -- start start 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc524835640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc520102d30 0x7fc520100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc524835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5201036e0 0x7fc520100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc524835640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5201082a0 0x7fc520101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc524835640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc520112060 con 0x7fc5201036e0 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc524835640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc520111ee0 con 0x7fc5201082a0 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc524835640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201121e0 con 0x7fc520102d30 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc517fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5201036e0 0x7fc520100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc517fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5201036e0 0x7fc520100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44024/0 (socket says 172.21.15.5:44024) 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc517fff640 1 -- 172.21.15.5:0/708548154 learned_addr learned my addr 172.21.15.5:0/708548154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc51e59c640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc520102d30 0x7fc520100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc51ed9d640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5201082a0 0x7fc520101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc517fff640 1 -- 172.21.15.5:0/708548154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc520102d30 msgr2=0x7fc520100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc517fff640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc520102d30 0x7fc520100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc517fff640 1 -- 172.21.15.5:0/708548154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5201082a0 msgr2=0x7fc520101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc517fff640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5201082a0 0x7fc520101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc517fff640 1 -- 172.21.15.5:0/708548154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5201a6a00 con 0x7fc5201036e0 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.332+0000 7fc51ed9d640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5201082a0 0x7fc520101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.333+0000 7fc517fff640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5201036e0 0x7fc520100ed0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc508005940 tx=0x7fc5080040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.333+0000 7fc5177fe640 1 -- 172.21.15.5:0/708548154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50803f030 con 0x7fc5201036e0 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.333+0000 7fc5177fe640 1 -- 172.21.15.5:0/708548154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5080044c0 con 0x7fc5201036e0 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.333+0000 7fc524835640 1 -- 172.21.15.5:0/708548154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5201a6c90 con 0x7fc5201036e0 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.333+0000 7fc5177fe640 1 -- 172.21.15.5:0/708548154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508035070 con 0x7fc5201036e0 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.333+0000 7fc524835640 1 -- 172.21.15.5:0/708548154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5201a71c0 con 0x7fc5201036e0 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.334+0000 7fc524835640 1 -- 172.21.15.5:0/708548154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc520103110 con 0x7fc5201036e0 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.335+0000 7fc5177fe640 1 -- 172.21.15.5:0/708548154 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc508042030 con 0x7fc5201036e0 2024-09-11T22:34:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.336+0000 7fc5177fe640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc50c0a2d70 0x7fc50c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:19.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.336+0000 7fc51e59c640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc50c0a2d70 0x7fc50c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:19.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.336+0000 7fc5177fe640 1 -- 172.21.15.5:0/708548154 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fc5080ef750 con 0x7fc5201036e0 2024-09-11T22:34:19.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.336+0000 7fc51e59c640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc50c0a2d70 0x7fc50c0a5230 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc51000abe0 tx=0x7fc51000a2d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:19.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.341+0000 7fc5177fe640 1 -- 172.21.15.5:0/708548154 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5080bc300 con 0x7fc5201036e0 2024-09-11T22:34:19.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.566+0000 7fc524835640 1 -- 172.21.15.5:0/708548154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc52010a2b0 con 0x7fc5201036e0 2024-09-11T22:34:19.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.567+0000 7fc5177fe640 1 -- 172.21.15.5:0/708548154 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11967 (secure 0 0 0) 0x7fc5080bbc00 con 0x7fc5201036e0 2024-09-11T22:34:19.568 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:34:19.569 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":45,"btime":"2024-09-11T22:33:32:484791+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:33:32.484772+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28160":{"gid":28160,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2346038927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2346038927},{"type":"v1","addr":"172.21.15.159:6833","nonce":2346038927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:34:19.569 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 45 2024-09-11T22:34:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.569+0000 7fc524835640 1 -- 172.21.15.5:0/708548154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc50c0a2d70 msgr2=0x7fc50c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:19.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.569+0000 7fc524835640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc50c0a2d70 0x7fc50c0a5230 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc51000abe0 tx=0x7fc51000a2d0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.569+0000 7fc524835640 1 -- 172.21.15.5:0/708548154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5201036e0 msgr2=0x7fc520100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.569+0000 7fc524835640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5201036e0 0x7fc520100ed0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fc508005940 tx=0x7fc5080040c0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.570+0000 7fc524835640 1 -- 172.21.15.5:0/708548154 shutdown_connections 2024-09-11T22:34:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.570+0000 7fc524835640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc50c0a2d70 0x7fc50c0a5230 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.570+0000 7fc524835640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5201082a0 0x7fc520101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.570+0000 7fc524835640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5201036e0 0x7fc520100ed0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.570+0000 7fc524835640 1 --2- 172.21.15.5:0/708548154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc520102d30 0x7fc520100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.570+0000 7fc524835640 1 -- 172.21.15.5:0/708548154 >> 172.21.15.5:0/708548154 conn(0x7fc5200fc7d0 msgr2=0x7fc520105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.570+0000 7fc524835640 1 -- 172.21.15.5:0/708548154 shutdown_connections 2024-09-11T22:34:19.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:19.570+0000 7fc524835640 1 -- 172.21.15.5:0/708548154 wait complete. 2024-09-11T22:34:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:19 smithi005 ceph-mon[30943]: pgmap v385: 97 pgs: 97 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-11T22:34:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:19 smithi159 ceph-mon[35534]: pgmap v385: 97 pgs: 97 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-11T22:34:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:19 smithi172 ceph-mon[35461]: pgmap v385: 97 pgs: 97 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-11T22:34:20.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 915275 7.32 MB/sec execute 518 sec latency 79.361 ms 2024-09-11T22:34:20.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:34:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:20.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:20 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/708548154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:34:20.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:20 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/708548154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:34:20.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:20 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/708548154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:34:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.153+0000 7f357843a640 1 -- 172.21.15.5:0/1680884042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102790 msgr2=0x7f35701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.153+0000 7f357843a640 1 --2- 172.21.15.5:0/1680884042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102790 0x7f35701067e0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f3560009900 tx=0x7f356002a410 comp rx=0 tx=0).stop 2024-09-11T22:34:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.154+0000 7f357843a640 1 -- 172.21.15.5:0/1680884042 shutdown_connections 2024-09-11T22:34:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.154+0000 7f357843a640 1 --2- 172.21.15.5:0/1680884042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3570106f10 0x7f357010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.154+0000 7f357843a640 1 --2- 172.21.15.5:0/1680884042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102790 0x7f35701067e0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.154+0000 7f357843a640 1 --2- 172.21.15.5:0/1680884042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3570101de0 0x7f35701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.154+0000 7f357843a640 1 -- 172.21.15.5:0/1680884042 >> 172.21.15.5:0/1680884042 conn(0x7f35700fd8c0 msgr2=0x7f35700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.154+0000 7f357843a640 1 -- 172.21.15.5:0/1680884042 shutdown_connections 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.154+0000 7f357843a640 1 -- 172.21.15.5:0/1680884042 wait complete. 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.154+0000 7f357843a640 1 Processor -- start 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.154+0000 7f357843a640 1 -- start start 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.154+0000 7f357843a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3570101de0 0x7f357019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f357843a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102790 0x7f357019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f357843a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3570106f10 0x7f357019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f357843a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3570111f70 con 0x7f3570102790 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f357843a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3570111df0 con 0x7f3570106f10 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f357843a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f35701120f0 con 0x7f3570101de0 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35759d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102790 0x7f357019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35759d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102790 0x7f357019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44050/0 (socket says 172.21.15.5:44050) 2024-09-11T22:34:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35759d5640 1 -- 172.21.15.5:0/1228031102 learned_addr learned my addr 172.21.15.5:0/1228031102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35761d6640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3570101de0 0x7f357019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35769d7640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3570106f10 0x7f357019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35759d5640 1 -- 172.21.15.5:0/1228031102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3570101de0 msgr2=0x7f357019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35759d5640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3570101de0 0x7f357019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35759d5640 1 -- 172.21.15.5:0/1228031102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3570106f10 msgr2=0x7f357019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35759d5640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3570106f10 0x7f357019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35759d5640 1 -- 172.21.15.5:0/1228031102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357019f5f0 con 0x7f3570102790 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35759d5640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102790 0x7f357019ab80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3560006010 tx=0x7f35600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35677fe640 1 -- 172.21.15.5:0/1228031102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356003f030 con 0x7f3570102790 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f357843a640 1 -- 172.21.15.5:0/1228031102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357019f880 con 0x7f3570102790 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35677fe640 1 -- 172.21.15.5:0/1228031102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3560003ce0 con 0x7f3570102790 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.155+0000 7f35677fe640 1 -- 172.21.15.5:0/1228031102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560035070 con 0x7f3570102790 2024-09-11T22:34:21.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.156+0000 7f357843a640 1 -- 172.21.15.5:0/1228031102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35701a71c0 con 0x7f3570102790 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.158+0000 7f35677fe640 1 -- 172.21.15.5:0/1228031102 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3560042030 con 0x7f3570102790 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.158+0000 7f357a813740 1 Processor -- start 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.158+0000 7f357a813740 1 -- start start 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f357a813740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b94090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f357a813740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b945d0 0x55e146b94a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35677fe640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f355c0a2cf0 0x7f355c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f357a813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146ba1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f357a813740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e146afa2f0 con 0x55e146b95b40 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f357a813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e146aa1cf0 con 0x55e146b9f370 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f357a813740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e146af62b0 con 0x55e146b945d0 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35769d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b94090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35769d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b94090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44060/0 (socket says 172.21.15.5:44060) 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35769d7640 1 -- 172.21.15.5:0/1388145877 learned_addr learned my addr 172.21.15.5:0/1388145877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35759d5640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f355c0a2cf0 0x7f355c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35769d7640 1 --2- 172.21.15.5:0/1388145877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146ba1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35677fe640 1 -- 172.21.15.5:0/1228031102 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f35600ef920 con 0x7f3570102790 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35761d6640 1 --2- 172.21.15.5:0/1388145877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b945d0 0x55e146b94a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35769d7640 1 -- 172.21.15.5:0/1388145877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b945d0 msgr2=0x55e146b94a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35769d7640 1 --2- 172.21.15.5:0/1388145877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b945d0 0x55e146b94a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35769d7640 1 -- 172.21.15.5:0/1388145877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 msgr2=0x55e146ba1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35769d7640 1 --2- 172.21.15.5:0/1388145877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146ba1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35769d7640 1 -- 172.21.15.5:0/1388145877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e146a863a0 con 0x55e146b95b40 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.159+0000 7f35759d5640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f355c0a2cf0 0x7f355c0a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f35600bbb80 tx=0x7f3560031a60 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.160+0000 7f35769d7640 1 --2- 172.21.15.5:0/1388145877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b94090 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x55e146b95070 tx=0x7f3568005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101fb719e3dd8db5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.160+0000 7f3564dea640 1 -- 172.21.15.5:0/1388145877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568013130 con 0x55e146b95b40 2024-09-11T22:34:21.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.160+0000 7f3564dea640 1 -- 172.21.15.5:0/1388145877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35680132d0 con 0x55e146b95b40 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.160+0000 7f3564dea640 1 -- 172.21.15.5:0/1388145877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35680053e0 con 0x55e146b95b40 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 -- 172.21.15.5:0/1388145877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 msgr2=0x55e146b94090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 --2- 172.21.15.5:0/1388145877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b94090 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x55e146b95070 tx=0x7f3568005bd0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 -- 172.21.15.5:0/1388145877 shutdown_connections 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 --2- 172.21.15.5:0/1388145877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146ba1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 --2- 172.21.15.5:0/1388145877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b945d0 0x55e146b94a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 --2- 172.21.15.5:0/1388145877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b94090 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 -- 172.21.15.5:0/1388145877 >> 172.21.15.5:0/1388145877 conn(0x55e146b5d310 msgr2=0x55e146b90c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 -- 172.21.15.5:0/1388145877 shutdown_connections 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 -- 172.21.15.5:0/1388145877 wait complete. 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 -- start start 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.161+0000 7f357a813740 1 Processor -- start 2024-09-11T22:34:21.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f357a813740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146b9ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f357a813740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b95b40 0x55e146b9bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f357a813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146b9c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f357a813740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e146ba2010 con 0x55e146b945d0 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f357a813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e146ab5060 con 0x55e146b9f370 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f357a813740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e146ba2190 con 0x55e146b95b40 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f357a813740 10 client.0 _ll_register_callbacks cb 0x55e1468088c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35769d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146b9ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35769d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146b9ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44076/0 (socket says 172.21.15.5:44076) 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35769d7640 1 -- 172.21.15.5:0/666175481 learned_addr learned my addr 172.21.15.5:0/666175481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35761d6640 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b95b40 0x55e146b9bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35769d7640 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146b9c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35769d7640 1 -- 172.21.15.5:0/666175481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b95b40 msgr2=0x55e146b9bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35769d7640 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b95b40 0x55e146b9bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35769d7640 1 -- 172.21.15.5:0/666175481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 msgr2=0x55e146b9c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35769d7640 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146b9c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35769d7640 1 -- 172.21.15.5:0/666175481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e14686b3c0 con 0x55e146b945d0 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f35769d7640 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146b9ed20 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f3568005cc0 tx=0x7f3568005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998e43cbafd36606 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f3559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f3559ffb640 1 -- 172.21.15.5:0/666175481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3568013130 con 0x55e146b945d0 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f3559ffb640 1 -- 172.21.15.5:0/666175481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f3568005320 con 0x55e146b945d0 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.162+0000 7f357a813740 1 -- 172.21.15.5:0/666175481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35680132d0 con 0x55e146b945d0 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f3559ffb640 1 -- 172.21.15.5:0/666175481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3568005520 con 0x55e146b945d0 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f357a813740 10 client.19170 fetch_fsmap learned FSMap version 45 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f357a813740 1 -- 172.21.15.5:0/666175481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e146bada60 con 0x55e146b945d0 2024-09-11T22:34:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f3559ffb640 1 -- 172.21.15.5:0/666175481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35680132d0 con 0x55e146b945d0 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f3559ffb640 1 -- 172.21.15.5:0/666175481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f3568021c90 con 0x55e146b945d0 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f357a813740 10 client.19170 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f357a813740 10 client.19170 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f357a813740 20 client.19170 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f357a813740 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55e146baedb0 0x55e146bb11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f357a813740 4 client.19170 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f357a813740 1 -- 172.21.15.5:0/666175481 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e146b07860 con 0x55e146baedb0 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.163+0000 7f35761d6640 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55e146baedb0 0x55e146bb11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.164+0000 7f35761d6640 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55e146baedb0 0x55e146bb11a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=acb25b1eb031b0a6 server_cookie=a788c9886cadd3ff in_seq=0 out_seq=0 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.164+0000 7f3559ffb640 10 client.19170 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.164+0000 7f3559ffb640 1 -- 172.21.15.5:0/666175481 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e146b07860 con 0x55e146baedb0 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.164+0000 7f3559ffb640 10 client.19170 handle_command_reply: tid=0 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.164+0000 7f357a813740 1 client.19170 shutdown 2024-09-11T22:34:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 1 -- 172.21.15.5:0/666175481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 msgr2=0x55e146b9ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146b9ed20 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f3568005cc0 tx=0x7f3568005020 comp rx=0 tx=0).stop 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 1 -- 172.21.15.5:0/666175481 shutdown_connections 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55e146baedb0 0x55e146bb11a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146b9c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b95b40 0x55e146b9bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f3559ffb640 0 client.19170 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 1 --2- 172.21.15.5:0/666175481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146b9ed20 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 1 -- 172.21.15.5:0/666175481 >> 172.21.15.5:0/666175481 conn(0x55e146b5d310 msgr2=0x55e146b96960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 1 -- 172.21.15.5:0/666175481 shutdown_connections 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 1 -- 172.21.15.5:0/666175481 wait complete. 2024-09-11T22:34:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.165+0000 7f357a813740 20 client.19170 trim_cache size 0 max 16384 2024-09-11T22:34:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.178+0000 7f357a813740 1 Processor -- start 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f357a813740 1 -- start start 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f357a813740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b945d0 0x55e146bab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f357a813740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b93c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f357a813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146bb68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f357a813740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35701afd00 con 0x55e146b95b40 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f357a813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35701b1840 con 0x55e146b9f370 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f357a813740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f35701b03d0 con 0x55e146b945d0 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35761d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b93c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35761d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b93c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44082/0 (socket says 172.21.15.5:44082) 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35761d6640 1 -- 172.21.15.5:0/3110040790 learned_addr learned my addr 172.21.15.5:0/3110040790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35769d7640 1 --2- 172.21.15.5:0/3110040790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146bb68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35769d7640 1 --2- 172.21.15.5:0/3110040790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b945d0 0x55e146bab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35761d6640 1 -- 172.21.15.5:0/3110040790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b945d0 msgr2=0x55e146bab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35761d6640 1 --2- 172.21.15.5:0/3110040790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b945d0 0x55e146bab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35761d6640 1 -- 172.21.15.5:0/3110040790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 msgr2=0x55e146bb68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35761d6640 1 --2- 172.21.15.5:0/3110040790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146bb68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35761d6640 1 -- 172.21.15.5:0/3110040790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e146b95750 con 0x55e146b95b40 2024-09-11T22:34:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.179+0000 7f35761d6640 1 --2- 172.21.15.5:0/3110040790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b93c90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x55e146b9d610 tx=0x7f356c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db91d76699656a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.180+0000 7f355bfff640 1 -- 172.21.15.5:0/3110040790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c003c80 con 0x55e146b95b40 2024-09-11T22:34:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.180+0000 7f355bfff640 1 -- 172.21.15.5:0/3110040790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f356c003e20 con 0x55e146b95b40 2024-09-11T22:34:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.180+0000 7f355bfff640 1 -- 172.21.15.5:0/3110040790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356c026800 con 0x55e146b95b40 2024-09-11T22:34:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.180+0000 7f357a813740 1 -- 172.21.15.5:0/3110040790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 msgr2=0x55e146b93c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.180+0000 7f357a813740 1 --2- 172.21.15.5:0/3110040790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b93c90 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x55e146b9d610 tx=0x7f356c00b390 comp rx=0 tx=0).stop 2024-09-11T22:34:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 -- 172.21.15.5:0/3110040790 shutdown_connections 2024-09-11T22:34:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 --2- 172.21.15.5:0/3110040790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b9f370 0x55e146bb68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 --2- 172.21.15.5:0/3110040790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b95b40 0x55e146b93c90 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 --2- 172.21.15.5:0/3110040790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b945d0 0x55e146bab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 -- 172.21.15.5:0/3110040790 >> 172.21.15.5:0/3110040790 conn(0x55e146b5d310 msgr2=0x55e146b8fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 -- 172.21.15.5:0/3110040790 shutdown_connections 2024-09-11T22:34:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 -- 172.21.15.5:0/3110040790 wait complete. 2024-09-11T22:34:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 -- start start 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 Processor -- start 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146ba37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b95b40 0x55e146ba3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b9f370 0x55e146c18890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35701b1170 con 0x55e146b945d0 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35701b0aa0 con 0x55e146b95b40 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f35701add00 con 0x55e146b9f370 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f357a813740 10 client.0 _ll_register_callbacks cb 0x7f35680135c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f35769d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146ba37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f35761d6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b95b40 0x55e146ba3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f35769d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146ba37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44088/0 (socket says 172.21.15.5:44088) 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.181+0000 7f35769d7640 1 -- 172.21.15.5:0/3937552671 learned_addr learned my addr 172.21.15.5:0/3937552671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f35769d7640 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b9f370 0x55e146c18890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f35769d7640 1 -- 172.21.15.5:0/3937552671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b9f370 msgr2=0x55e146c18890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f35769d7640 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b9f370 0x55e146c18890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f35769d7640 1 -- 172.21.15.5:0/3937552671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b95b40 msgr2=0x55e146ba3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f35769d7640 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b95b40 0x55e146ba3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f35769d7640 1 -- 172.21.15.5:0/3937552671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e146c18dd0 con 0x55e146b945d0 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f35769d7640 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146ba37f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f356802a670 tx=0x7f3568005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb658b07a1831c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f3559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f3559ffb640 1 -- 172.21.15.5:0/3937552671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35680052a0 con 0x55e146b945d0 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f3559ffb640 1 -- 172.21.15.5:0/3937552671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f35680056c0 con 0x55e146b945d0 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f357a813740 1 -- 172.21.15.5:0/3937552671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f356c003e20 con 0x55e146b945d0 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f3559ffb640 1 -- 172.21.15.5:0/3937552671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f356803d050 con 0x55e146b945d0 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f357a813740 10 client.19176 fetch_fsmap learned FSMap version 45 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.182+0000 7f357a813740 1 -- 172.21.15.5:0/3937552671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e146c1a010 con 0x55e146b945d0 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.183+0000 7f3559ffb640 1 -- 172.21.15.5:0/3937552671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f356c003e20 con 0x55e146b945d0 2024-09-11T22:34:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.183+0000 7f3559ffb640 1 -- 172.21.15.5:0/3937552671 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f3568003b30 con 0x55e146b945d0 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.183+0000 7f357a813740 10 client.19176 fetch_fsmap finished waiting for FSMap version 45 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.183+0000 7f357a813740 10 client.19176 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.183+0000 7f357a813740 20 client.19176 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.183+0000 7f357a813740 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55e146c1a520 0x55e146c1c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.183+0000 7f357a813740 4 client.19176 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.183+0000 7f357a813740 1 -- 172.21.15.5:0/3937552671 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e146b01d70 con 0x55e146c1a520 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.183+0000 7f35761d6640 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55e146c1a520 0x55e146c1c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f3559ffb640 10 client.19176 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f35761d6640 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55e146c1a520 0x55e146c1c910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=87fded9d40005d6b server_cookie=6c63bd0b5e933f40 in_seq=0 out_seq=0 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f3559ffb640 1 -- 172.21.15.5:0/3937552671 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e146b01d70 con 0x55e146c1a520 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f3559ffb640 10 client.19176 handle_command_reply: tid=0 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 client.19176 shutdown 2024-09-11T22:34:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 -- 172.21.15.5:0/3937552671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 msgr2=0x55e146ba37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146ba37f0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f356802a670 tx=0x7f3568005020 comp rx=0 tx=0).stop 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 -- 172.21.15.5:0/3937552671 shutdown_connections 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55e146c1a520 0x55e146c1c910 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e146b9f370 0x55e146c18890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e146b95b40 0x55e146ba3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 --2- 172.21.15.5:0/3937552671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e146b945d0 0x55e146ba37f0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 -- 172.21.15.5:0/3937552671 >> 172.21.15.5:0/3937552671 conn(0x55e146b5d310 msgr2=0x55e146b9a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f3559ffb640 0 client.19176 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 -- 172.21.15.5:0/3937552671 shutdown_connections 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 1 -- 172.21.15.5:0/3937552671 wait complete. 2024-09-11T22:34:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.184+0000 7f357a813740 20 client.19176 trim_cache size 0 max 16384 2024-09-11T22:34:21.191 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:34:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 -- 172.21.15.5:0/1228031102 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f355c0a2cf0 msgr2=0x7f355c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f355c0a2cf0 0x7f355c0a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f35600bbb80 tx=0x7f3560031a60 comp rx=0 tx=0).stop 2024-09-11T22:34:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 -- 172.21.15.5:0/1228031102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102790 msgr2=0x7f357019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102790 0x7f357019ab80 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f3560006010 tx=0x7f35600042f0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 -- 172.21.15.5:0/1228031102 shutdown_connections 2024-09-11T22:34:21.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f355c0a2cf0 0x7f355c0a51b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3570106f10 0x7f357019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3570102790 0x7f357019ab80 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 --2- 172.21.15.5:0/1228031102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3570101de0 0x7f357019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:21.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 -- 172.21.15.5:0/1228031102 >> 172.21.15.5:0/1228031102 conn(0x7f35700fd8c0 msgr2=0x7f35700feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:21.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 -- 172.21.15.5:0/1228031102 shutdown_connections 2024-09-11T22:34:21.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:21.185+0000 7f357843a640 1 -- 172.21.15.5:0/1228031102 wait complete. 2024-09-11T22:34:21.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 916496 7.32 MB/sec execute 519 sec latency 68.312 ms 2024-09-11T22:34:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:21 smithi005 ceph-mon[30943]: pgmap v386: 97 pgs: 97 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-11T22:34:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:21 smithi159 ceph-mon[35534]: pgmap v386: 97 pgs: 97 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-11T22:34:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:21 smithi172 ceph-mon[35461]: pgmap v386: 97 pgs: 97 active+clean; 87 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-09-11T22:34:22.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:34:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:22.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 917658 7.35 MB/sec execute 520 sec latency 65.255 ms 2024-09-11T22:34:22.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.807+0000 7fa05ad9a640 1 -- 172.21.15.5:0/633356034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa054102ef0 msgr2=0x7fa05410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.807+0000 7fa05ad9a640 1 --2- 172.21.15.5:0/633356034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa054102ef0 0x7fa05410da40 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009ca0 tx=0x7fa04c02a950 comp rx=0 tx=0).stop 2024-09-11T22:34:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.807+0000 7fa05ad9a640 1 -- 172.21.15.5:0/633356034 shutdown_connections 2024-09-11T22:34:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.807+0000 7fa05ad9a640 1 --2- 172.21.15.5:0/633356034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa054102ef0 0x7fa05410da40 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.807+0000 7fa05ad9a640 1 --2- 172.21.15.5:0/633356034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa054102570 0x7fa0541029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.807+0000 7fa05ad9a640 1 --2- 172.21.15.5:0/633356034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa054100e20 0x7fa054101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.807+0000 7fa05ad9a640 1 -- 172.21.15.5:0/633356034 >> 172.21.15.5:0/633356034 conn(0x7fa0540fc7d0 msgr2=0x7fa0540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.807+0000 7fa05ad9a640 1 -- 172.21.15.5:0/633356034 shutdown_connections 2024-09-11T22:34:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.807+0000 7fa05ad9a640 1 -- 172.21.15.5:0/633356034 wait complete. 2024-09-11T22:34:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.808+0000 7fa05ad9a640 1 Processor -- start 2024-09-11T22:34:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.808+0000 7fa05ad9a640 1 -- start start 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.808+0000 7fa05ad9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa054100e20 0x7fa054198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.808+0000 7fa05ad9a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa054102570 0x7fa054198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.808+0000 7fa05ad9a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa054102ef0 0x7fa05419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.808+0000 7fa05ad9a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa054110f80 con 0x7fa054100e20 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.808+0000 7fa05ad9a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa054110e00 con 0x7fa054102ef0 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.808+0000 7fa05ad9a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa054111100 con 0x7fa054102570 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa058b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa054100e20 0x7fa054198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa058b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa054100e20 0x7fa054198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49864/0 (socket says 172.21.15.5:49864) 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa058b36640 1 -- 172.21.15.5:0/1002817852 learned_addr learned my addr 172.21.15.5:0/1002817852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa053fff640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa054102570 0x7fa054198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa059337640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa054102ef0 0x7fa05419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa053fff640 1 -- 172.21.15.5:0/1002817852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa054102ef0 msgr2=0x7fa05419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa053fff640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa054102ef0 0x7fa05419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa053fff640 1 -- 172.21.15.5:0/1002817852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa054100e20 msgr2=0x7fa054198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa053fff640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa054100e20 0x7fa054198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa053fff640 1 -- 172.21.15.5:0/1002817852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05419d400 con 0x7fa054102570 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.809+0000 7fa053fff640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa054102570 0x7fa054198990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa04000e9f0 tx=0x7fa04000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.810+0000 7fa051ffb640 1 -- 172.21.15.5:0/1002817852 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04000ccc0 con 0x7fa054102570 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.810+0000 7fa051ffb640 1 -- 172.21.15.5:0/1002817852 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa040010910 con 0x7fa054102570 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.810+0000 7fa05ad9a640 1 -- 172.21.15.5:0/1002817852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05419d6f0 con 0x7fa054102570 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.810+0000 7fa051ffb640 1 -- 172.21.15.5:0/1002817852 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040018960 con 0x7fa054102570 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.810+0000 7fa05ad9a640 1 -- 172.21.15.5:0/1002817852 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0541a4fd0 con 0x7fa054102570 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.813+0000 7fa051ffb640 1 -- 172.21.15.5:0/1002817852 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa040010ac0 con 0x7fa054102570 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.813+0000 7fa05d173740 1 Processor -- start 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa05d173740 1 -- start start 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa051ffb640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0440a2cf0 0x7fa0440a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa05d173740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa05d173740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 0x557db969ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa05d173740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 0x557db96a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa05d173740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557db96012f0 con 0x557db96a6370 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa05d173740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557db95a8cf0 con 0x557db969b5d0 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa05d173740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557db95fd2b0 con 0x557db969cb40 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa051ffb640 1 -- 172.21.15.5:0/1002817852 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fa0400cf850 con 0x7fa054102570 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa059337640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0440a2cf0 0x7fa0440a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa058b36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa058b36640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60166/0 (socket says 172.21.15.5:60166) 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa058b36640 1 -- 172.21.15.5:0/3122517621 learned_addr learned my addr 172.21.15.5:0/3122517621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa059337640 1 --2- 172.21.15.5:0/3122517621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 0x557db96a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.814+0000 7fa053fff640 1 --2- 172.21.15.5:0/3122517621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 0x557db969ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.815+0000 7fa058b36640 1 -- 172.21.15.5:0/3122517621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 msgr2=0x557db969ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.815+0000 7fa058b36640 1 --2- 172.21.15.5:0/3122517621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 0x557db969ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.815+0000 7fa058b36640 1 -- 172.21.15.5:0/3122517621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 msgr2=0x557db96a8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.815+0000 7fa058b36640 1 --2- 172.21.15.5:0/3122517621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 0x557db96a8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.815+0000 7fa058b36640 1 -- 172.21.15.5:0/3122517621 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557db958d3a0 con 0x557db969cb40 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.815+0000 7fa059337640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0440a2cf0 0x7fa0440a51b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa04c005e30 tx=0x7fa04c02ffc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.815+0000 7fa058b36640 1 --2- 172.21.15.5:0/3122517621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969b090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa048009dc0 tx=0x7fa048011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b08ec1fc968b0ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.815+0000 7fa03effd640 1 -- 172.21.15.5:0/3122517621 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048005220 con 0x557db969cb40 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.815+0000 7fa03effd640 1 -- 172.21.15.5:0/3122517621 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa048004c80 con 0x557db969cb40 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.815+0000 7fa03effd640 1 -- 172.21.15.5:0/3122517621 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0480168c0 con 0x557db969cb40 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.816+0000 7fa05d173740 1 -- 172.21.15.5:0/3122517621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 msgr2=0x557db969b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.816+0000 7fa05d173740 1 --2- 172.21.15.5:0/3122517621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969b090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa048009dc0 tx=0x7fa048011810 comp rx=0 tx=0).stop 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.816+0000 7fa05d173740 1 -- 172.21.15.5:0/3122517621 shutdown_connections 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.816+0000 7fa05d173740 1 --2- 172.21.15.5:0/3122517621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 0x557db96a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.816+0000 7fa05d173740 1 --2- 172.21.15.5:0/3122517621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 0x557db969ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.816+0000 7fa05d173740 1 --2- 172.21.15.5:0/3122517621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969b090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.816+0000 7fa05d173740 1 -- 172.21.15.5:0/3122517621 >> 172.21.15.5:0/3122517621 conn(0x557db9664310 msgr2=0x557db9697c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.817+0000 7fa05d173740 1 -- 172.21.15.5:0/3122517621 shutdown_connections 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.817+0000 7fa05d173740 1 -- 172.21.15.5:0/3122517621 wait complete. 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.818+0000 7fa05d173740 1 -- start start 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.818+0000 7fa05d173740 1 Processor -- start 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa05d173740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa05d173740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db96a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa05d173740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db96a6370 0x557db96a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa05d173740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557db96a9070 con 0x557db969b5d0 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa05d173740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557db95bc060 con 0x557db96a6370 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa05d173740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557db96a91f0 con 0x557db969cb40 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa05d173740 10 client.0 _ll_register_callbacks cb 0x557db930f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa058b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa058b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49882/0 (socket says 172.21.15.5:49882) 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa058b36640 1 -- 172.21.15.5:0/2261197981 learned_addr learned my addr 172.21.15.5:0/2261197981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa059337640 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db96a6370 0x557db96a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa058b36640 1 -- 172.21.15.5:0/2261197981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 msgr2=0x557db96a2ff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa058b36640 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db96a2ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa058b36640 1 -- 172.21.15.5:0/2261197981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db96a6370 msgr2=0x557db96a35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa058b36640 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db96a6370 0x557db96a35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.819+0000 7fa058b36640 1 -- 172.21.15.5:0/2261197981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557db93723c0 con 0x557db969b5d0 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.820+0000 7fa058b36640 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a5d40 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa048000c00 tx=0x7fa048004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4167ac4a7522f513 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.820+0000 7fa033fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.820+0000 7fa033fff640 1 -- 172.21.15.5:0/2261197981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04802c020 con 0x557db969b5d0 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.820+0000 7fa05d173740 1 -- 172.21.15.5:0/2261197981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa048004c80 con 0x557db969b5d0 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.821+0000 7fa033fff640 1 -- 172.21.15.5:0/2261197981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fa048004450 con 0x557db969b5d0 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.821+0000 7fa033fff640 1 -- 172.21.15.5:0/2261197981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa048037050 con 0x557db969b5d0 2024-09-11T22:34:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.821+0000 7fa05d173740 10 client.19188 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.821+0000 7fa05d173740 1 -- 172.21.15.5:0/2261197981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557db96b4ae0 con 0x557db969b5d0 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.822+0000 7fa033fff640 1 -- 172.21.15.5:0/2261197981 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa048004c80 con 0x557db969b5d0 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.822+0000 7fa033fff640 1 -- 172.21.15.5:0/2261197981 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fa048003c80 con 0x557db969b5d0 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.822+0000 7fa05d173740 10 client.19188 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.822+0000 7fa05d173740 10 client.19188 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.822+0000 7fa05d173740 20 client.19188 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.822+0000 7fa05d173740 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557db96b5e90 0x557db96b8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.822+0000 7fa05d173740 4 client.19188 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.822+0000 7fa05d173740 1 -- 172.21.15.5:0/2261197981 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557db960e860 con 0x557db96b5e90 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.822+0000 7fa053fff640 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557db96b5e90 0x557db96b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.823+0000 7fa053fff640 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557db96b5e90 0x557db96b8280 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d98ff1248c04d1b4 server_cookie=f96c609e8ec8db4c in_seq=0 out_seq=0 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.823+0000 7fa033fff640 10 client.19188 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.824+0000 7fa033fff640 1 -- 172.21.15.5:0/2261197981 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557db960e860 con 0x557db96b5e90 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.824+0000 7fa033fff640 10 client.19188 handle_command_reply: tid=0 2024-09-11T22:34:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.824+0000 7fa05d173740 1 client.19188 shutdown 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.825+0000 7fa05d173740 1 -- 172.21.15.5:0/2261197981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 msgr2=0x557db96a5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.825+0000 7fa05d173740 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a5d40 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa048000c00 tx=0x7fa048004bc0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.825+0000 7fa05d173740 1 -- 172.21.15.5:0/2261197981 shutdown_connections 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.825+0000 7fa05d173740 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557db96b5e90 0x557db96b8280 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.825+0000 7fa05d173740 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db96a6370 0x557db96a35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.825+0000 7fa05d173740 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db96a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.825+0000 7fa05d173740 1 --2- 172.21.15.5:0/2261197981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a5d40 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.825+0000 7fa05d173740 1 -- 172.21.15.5:0/2261197981 >> 172.21.15.5:0/2261197981 conn(0x557db9664310 msgr2=0x557db969d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.825+0000 7fa033fff640 0 client.19188 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.825+0000 7fa05d173740 1 -- 172.21.15.5:0/2261197981 shutdown_connections 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.826+0000 7fa05d173740 1 -- 172.21.15.5:0/2261197981 wait complete. 2024-09-11T22:34:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.826+0000 7fa05d173740 20 client.19188 trim_cache size 0 max 16384 2024-09-11T22:34:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.845+0000 7fa05d173740 1 Processor -- start 2024-09-11T22:34:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.845+0000 7fa05d173740 1 -- start start 2024-09-11T22:34:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa05d173740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 0x557db96a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa05d173740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa05d173740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 0x557db969e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa05d173740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541ab900 con 0x557db96a6370 2024-09-11T22:34:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa05d173740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541ad470 con 0x557db969b5d0 2024-09-11T22:34:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa05d173740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541abfd0 con 0x557db969cb40 2024-09-11T22:34:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa053fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa059337640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 0x557db969e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa058b36640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 0x557db96a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa053fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60186/0 (socket says 172.21.15.5:60186) 2024-09-11T22:34:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa053fff640 1 -- 172.21.15.5:0/3679600830 learned_addr learned my addr 172.21.15.5:0/3679600830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa058b36640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 0x557db96a27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56172/0 (socket says 172.21.15.5:56172) 2024-09-11T22:34:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa059337640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 0x557db969e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49892/0 (socket says 172.21.15.5:49892) 2024-09-11T22:34:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa053fff640 1 -- 172.21.15.5:0/3679600830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 msgr2=0x557db96a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa053fff640 1 --2- 172.21.15.5:0/3679600830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 0x557db96a27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa053fff640 1 -- 172.21.15.5:0/3679600830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 msgr2=0x557db969e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa053fff640 1 --2- 172.21.15.5:0/3679600830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 0x557db969e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa053fff640 1 -- 172.21.15.5:0/3679600830 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557db9698960 con 0x557db969cb40 2024-09-11T22:34:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.846+0000 7fa053fff640 1 --2- 172.21.15.5:0/3679600830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969e120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa04009bd40 tx=0x7fa040010e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4480c96bb44d8870 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.847+0000 7fa03e7fc640 1 -- 172.21.15.5:0/3679600830 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04009a0b0 con 0x557db969cb40 2024-09-11T22:34:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.847+0000 7fa03e7fc640 1 -- 172.21.15.5:0/3679600830 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa040098420 con 0x557db969cb40 2024-09-11T22:34:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.847+0000 7fa03e7fc640 1 -- 172.21.15.5:0/3679600830 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0400a44e0 con 0x557db969cb40 2024-09-11T22:34:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 -- 172.21.15.5:0/3679600830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 msgr2=0x557db969e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 --2- 172.21.15.5:0/3679600830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969e120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa04009bd40 tx=0x7fa040010e30 comp rx=0 tx=0).stop 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 -- 172.21.15.5:0/3679600830 shutdown_connections 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 --2- 172.21.15.5:0/3679600830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db96a6370 0x557db969e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 --2- 172.21.15.5:0/3679600830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db969cb40 0x557db969e120 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 --2- 172.21.15.5:0/3679600830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969b5d0 0x557db96a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 -- 172.21.15.5:0/3679600830 >> 172.21.15.5:0/3679600830 conn(0x557db9664310 msgr2=0x557db9696d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 -- 172.21.15.5:0/3679600830 shutdown_connections 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 -- 172.21.15.5:0/3679600830 wait complete. 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 -- start start 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.848+0000 7fa05d173740 1 Processor -- start 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa05d173740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa05d173740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969cb40 0x557db96a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa05d173740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db96a6370 0x557db971e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa058b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa058b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49904/0 (socket says 172.21.15.5:49904) 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa058b36640 1 -- 172.21.15.5:0/2984150022 learned_addr learned my addr 172.21.15.5:0/2984150022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa053fff640 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969cb40 0x557db96a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541acda0 con 0x557db969b5d0 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541ac6d0 con 0x557db969cb40 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa0541a9900 con 0x557db96a6370 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa059337640 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db96a6370 0x557db971e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa05d173740 10 client.0 _ll_register_callbacks cb 0x7fa040005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa058b36640 1 -- 172.21.15.5:0/2984150022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db96a6370 msgr2=0x557db971e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa058b36640 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db96a6370 0x557db971e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa058b36640 1 -- 172.21.15.5:0/2984150022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969cb40 msgr2=0x557db96a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa058b36640 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969cb40 0x557db96a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa058b36640 1 -- 172.21.15.5:0/2984150022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557db971edb0 con 0x557db969b5d0 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa053fff640 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969cb40 0x557db96a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.849+0000 7fa059337640 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db96a6370 0x557db971e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.850+0000 7fa058b36640 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a99e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa048009250 tx=0x7fa048009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a755f699dc0c8db4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.850+0000 7fa033fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.850+0000 7fa033fff640 1 -- 172.21.15.5:0/2984150022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048040060 con 0x557db969b5d0 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.850+0000 7fa033fff640 1 -- 172.21.15.5:0/2984150022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fa048004250 con 0x557db969b5d0 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.850+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa040098420 con 0x557db969b5d0 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.850+0000 7fa033fff640 1 -- 172.21.15.5:0/2984150022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa04802c070 con 0x557db969b5d0 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.850+0000 7fa05d173740 10 client.19194 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.850+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x557db971fff0 con 0x557db969b5d0 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.850+0000 7fa033fff640 1 -- 172.21.15.5:0/2984150022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa040098420 con 0x557db969b5d0 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.850+0000 7fa033fff640 1 -- 172.21.15.5:0/2984150022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fa0480035f0 con 0x557db969b5d0 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.851+0000 7fa05d173740 10 client.19194 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.851+0000 7fa05d173740 10 client.19194 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:34:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.851+0000 7fa05d173740 20 client.19194 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.851+0000 7fa05d173740 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557db97206b0 0x557db9722aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.851+0000 7fa05d173740 4 client.19194 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:34:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.851+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557db9608d70 con 0x557db97206b0 2024-09-11T22:34:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.851+0000 7fa053fff640 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557db97206b0 0x557db9722aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.852+0000 7fa033fff640 10 client.19194 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:34:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.852+0000 7fa053fff640 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557db97206b0 0x557db9722aa0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b1b573b6b8ac99ba server_cookie=f7f4d429f55ae45 in_seq=0 out_seq=0 2024-09-11T22:34:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.852+0000 7fa033fff640 1 -- 172.21.15.5:0/2984150022 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557db9608d70 con 0x557db97206b0 2024-09-11T22:34:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.852+0000 7fa033fff640 10 client.19194 handle_command_reply: tid=0 2024-09-11T22:34:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.852+0000 7fa05d173740 1 client.19194 shutdown 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.852+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 msgr2=0x557db96a99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.852+0000 7fa05d173740 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a99e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fa048009250 tx=0x7fa048009c10 comp rx=0 tx=0).stop 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 shutdown_connections 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05d173740 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557db96a6370 0x557db971e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05d173740 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x557db97206b0 0x557db9722aa0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05d173740 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557db969cb40 0x557db96a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05d173740 1 --2- 172.21.15.5:0/2984150022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557db969b5d0 0x557db96a99e0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 >> 172.21.15.5:0/2984150022 conn(0x557db9664310 msgr2=0x557db96a4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa033fff640 0 client.19194 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 shutdown_connections 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05d173740 1 -- 172.21.15.5:0/2984150022 wait complete. 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05d173740 20 client.19194 trim_cache size 0 max 16384 2024-09-11T22:34:22.858 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05ad9a640 1 -- 172.21.15.5:0/1002817852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0440a2cf0 msgr2=0x7fa0440a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05ad9a640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0440a2cf0 0x7fa0440a51b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa04c005e30 tx=0x7fa04c02ffc0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05ad9a640 1 -- 172.21.15.5:0/1002817852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa054102570 msgr2=0x7fa054198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.853+0000 7fa05ad9a640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa054102570 0x7fa054198990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa04000e9f0 tx=0x7fa04000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.854+0000 7fa05ad9a640 1 -- 172.21.15.5:0/1002817852 shutdown_connections 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.854+0000 7fa05ad9a640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa0440a2cf0 0x7fa0440a51b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.854+0000 7fa05ad9a640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa054102ef0 0x7fa05419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.854+0000 7fa05ad9a640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa054102570 0x7fa054198990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.854+0000 7fa05ad9a640 1 --2- 172.21.15.5:0/1002817852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa054100e20 0x7fa054198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.854+0000 7fa05ad9a640 1 -- 172.21.15.5:0/1002817852 >> 172.21.15.5:0/1002817852 conn(0x7fa0540fc7d0 msgr2=0x7fa05406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.854+0000 7fa05ad9a640 1 -- 172.21.15.5:0/1002817852 shutdown_connections 2024-09-11T22:34:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:22.854+0000 7fa05ad9a640 1 -- 172.21.15.5:0/1002817852 wait complete. 2024-09-11T22:34:23.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 919677 7.36 MB/sec execute 521 sec latency 104.175 ms 2024-09-11T22:34:23.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:34:24.283 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:23 smithi172 ceph-mon[35461]: pgmap v387: 97 pgs: 97 active+clean; 93 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 217 op/s 2024-09-11T22:34:24.284 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:23 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:34:24.284 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:23 smithi159 ceph-mon[35534]: pgmap v387: 97 pgs: 97 active+clean; 93 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 217 op/s 2024-09-11T22:34:24.284 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:23 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:34:24.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:23 smithi005 ceph-mon[30943]: pgmap v387: 97 pgs: 97 active+clean; 93 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 217 op/s 2024-09-11T22:34:24.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:23 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:34:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.276+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3375961580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1d8101350 msgr2=0x7fd1d810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.276+0000 7fd1dd7c4640 1 --2- 172.21.15.5:0/3375961580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1d8101350 0x7fd1d810d880 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009c10 tx=0x7fd1d002a6e0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.276+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3375961580 shutdown_connections 2024-09-11T22:34:24.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.276+0000 7fd1dd7c4640 1 --2- 172.21.15.5:0/3375961580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1d8101350 0x7fd1d810d880 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.276+0000 7fd1dd7c4640 1 --2- 172.21.15.5:0/3375961580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1d81009b0 0x7fd1d8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.276+0000 7fd1dd7c4640 1 --2- 172.21.15.5:0/3375961580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1d81069c0 0x7fd1d8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.276+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3375961580 >> 172.21.15.5:0/3375961580 conn(0x7fd1d8077990 msgr2=0x7fd1d80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.276+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3375961580 shutdown_connections 2024-09-11T22:34:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.276+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3375961580 wait complete. 2024-09-11T22:34:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.276+0000 7fd1dd7c4640 1 Processor -- start 2024-09-11T22:34:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1dd7c4640 1 -- start start 2024-09-11T22:34:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1dd7c4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1d81009b0 0x7fd1d8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1dd7c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1d8101350 0x7fd1d8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1dd7c4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1d81069c0 0x7fd1d819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1dd7c4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d8110db0 con 0x7fd1d8101350 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1dd7c4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d8110c30 con 0x7fd1d81069c0 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1dd7c4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d8110f30 con 0x7fd1d81009b0 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1d8101350 0x7fd1d8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1d81009b0 0x7fd1d8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1d8101350 0x7fd1d8198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49934/0 (socket says 172.21.15.5:49934) 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d67fc640 1 -- 172.21.15.5:0/3424620852 learned_addr learned my addr 172.21.15.5:0/3424620852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1d81069c0 0x7fd1d819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d67fc640 1 -- 172.21.15.5:0/3424620852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1d81009b0 msgr2=0x7fd1d8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d67fc640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1d81009b0 0x7fd1d8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d67fc640 1 -- 172.21.15.5:0/3424620852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1d81069c0 msgr2=0x7fd1d819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:24.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d67fc640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1d81069c0 0x7fd1d819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d67fc640 1 -- 172.21.15.5:0/3424620852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d819d330 con 0x7fd1d8101350 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.277+0000 7fd1d67fc640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1d8101350 0x7fd1d8198880 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000e9f0 tx=0x7fd1c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.278+0000 7fd1cffff640 1 -- 172.21.15.5:0/3424620852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c000cd80 con 0x7fd1d8101350 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.278+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3424620852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d819d620 con 0x7fd1d8101350 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.278+0000 7fd1cffff640 1 -- 172.21.15.5:0/3424620852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c0005500 con 0x7fd1d8101350 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.278+0000 7fd1cffff640 1 -- 172.21.15.5:0/3424620852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0010690 con 0x7fd1d8101350 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.278+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3424620852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d81a4eb0 con 0x7fd1d8101350 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.280+0000 7fd1cffff640 1 -- 172.21.15.5:0/3424620852 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd1c00046c0 con 0x7fd1d8101350 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.280+0000 7fd1e063a740 1 Processor -- start 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.280+0000 7fd1e063a740 1 -- start start 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 0x561eb7199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb71a4cb0 0x561eb71a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1e063a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561eb70fe2f0 con 0x561eb719a410 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1e063a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561eb70a6270 con 0x561eb7199af0 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1e063a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561eb70fa2b0 con 0x561eb71a4cb0 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 0x561eb7199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d67fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb71a4cb0 0x561eb71a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49942/0 (socket says 172.21.15.5:49942) 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d6ffd640 1 -- 172.21.15.5:0/1000312073 learned_addr learned my addr 172.21.15.5:0/1000312073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1cffff640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1c40a2da0 0x7fd1c40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d6ffd640 1 -- 172.21.15.5:0/1000312073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb71a4cb0 msgr2=0x561eb71a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/1000312073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb71a4cb0 0x561eb71a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d6ffd640 1 -- 172.21.15.5:0/1000312073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 msgr2=0x561eb7199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/1000312073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 0x561eb7199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d6ffd640 1 -- 172.21.15.5:0/1000312073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb70a5340 con 0x561eb719a410 2024-09-11T22:34:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/1000312073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 0x561eb7199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:34:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1c40a2da0 0x7fd1c40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1cffff640 1 -- 172.21.15.5:0/3424620852 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fd1c00cf8e0 con 0x7fd1d8101350 2024-09-11T22:34:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.281+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/1000312073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a4770 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009c60 tx=0x7fd1c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440a126cd258ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1cd7fa640 1 -- 172.21.15.5:0/1000312073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8005220 con 0x561eb719a410 2024-09-11T22:34:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1cd7fa640 1 -- 172.21.15.5:0/1000312073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c8009e10 con 0x561eb719a410 2024-09-11T22:34:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1cd7fa640 1 -- 172.21.15.5:0/1000312073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8019810 con 0x561eb719a410 2024-09-11T22:34:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1e063a740 1 -- 172.21.15.5:0/1000312073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 msgr2=0x561eb71a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:24.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1c40a2da0 0x7fd1c40a5260 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0004af0 tx=0x7fd1d0009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1e063a740 1 --2- 172.21.15.5:0/1000312073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a4770 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009c60 tx=0x7fd1c8011810 comp rx=0 tx=0).stop 2024-09-11T22:34:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1e063a740 1 -- 172.21.15.5:0/1000312073 shutdown_connections 2024-09-11T22:34:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1e063a740 1 --2- 172.21.15.5:0/1000312073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb71a4cb0 0x561eb71a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1e063a740 1 --2- 172.21.15.5:0/1000312073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a4770 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1e063a740 1 --2- 172.21.15.5:0/1000312073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 0x561eb7199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.282+0000 7fd1e063a740 1 -- 172.21.15.5:0/1000312073 >> 172.21.15.5:0/1000312073 conn(0x561eb7196060 msgr2=0x561eb7196440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 1 -- 172.21.15.5:0/1000312073 shutdown_connections 2024-09-11T22:34:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 1 -- 172.21.15.5:0/1000312073 wait complete. 2024-09-11T22:34:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 1 -- start start 2024-09-11T22:34:24.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 1 Processor -- start 2024-09-11T22:34:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb71a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb719a410 0x561eb71a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb71a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561eb71a7880 con 0x561eb71a4cb0 2024-09-11T22:34:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561eb7087290 con 0x561eb719a410 2024-09-11T22:34:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x561eb71a7a00 con 0x561eb7199af0 2024-09-11T22:34:24.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1e063a740 10 client.0 _ll_register_callbacks cb 0x561eb70b8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb71a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb71a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb71a25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49958/0 (socket says 172.21.15.5:49958) 2024-09-11T22:34:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1d6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb71a1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60244/0 (socket says 172.21.15.5:60244) 2024-09-11T22:34:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.283+0000 7fd1d77fe640 1 -- 172.21.15.5:0/841550341 learned_addr learned my addr 172.21.15.5:0/841550341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1d67fc640 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb719a410 0x561eb71a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1d77fe640 1 -- 172.21.15.5:0/841550341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 msgr2=0x561eb71a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:24.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb71a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1d77fe640 1 -- 172.21.15.5:0/841550341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb719a410 msgr2=0x561eb71a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb719a410 0x561eb71a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1d77fe640 1 -- 172.21.15.5:0/841550341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb6e11210 con 0x561eb71a4cb0 2024-09-11T22:34:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb71a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1d67fc640 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb719a410 0x561eb71a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb71a25d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009c10 tx=0x7fd1d0009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a21adb2c9a7a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:24.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1be7fc640 1 -- 172.21.15.5:0/841550341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d004f070 con 0x561eb71a4cb0 2024-09-11T22:34:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1e063a740 1 -- 172.21.15.5:0/841550341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c8009e10 con 0x561eb71a4cb0 2024-09-11T22:34:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1be7fc640 1 -- 172.21.15.5:0/841550341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fd1d0003d80 con 0x561eb71a4cb0 2024-09-11T22:34:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1be7fc640 1 -- 172.21.15.5:0/841550341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd1d005b050 con 0x561eb71a4cb0 2024-09-11T22:34:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1e063a740 10 client.19206 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.284+0000 7fd1e063a740 1 -- 172.21.15.5:0/841550341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x561eb6edd620 con 0x561eb71a4cb0 2024-09-11T22:34:24.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1be7fc640 1 -- 172.21.15.5:0/841550341 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c8009e10 con 0x561eb71a4cb0 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1be7fc640 1 -- 172.21.15.5:0/841550341 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fd1d00317b0 con 0x561eb71a4cb0 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1e063a740 10 client.19206 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1e063a740 10 client.19206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1e063a740 20 client.19206 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1e063a740 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561eb71b4b70 0x561eb71b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1e063a740 4 client.19206 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1e063a740 1 -- 172.21.15.5:0/841550341 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561eb710ba20 con 0x561eb71b4b70 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561eb71b4b70 0x561eb71b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1be7fc640 10 client.19206 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.285+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561eb71b4b70 0x561eb71b6fe0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67917a01e17dbb69 server_cookie=908ff3967344ccd5 in_seq=0 out_seq=0 2024-09-11T22:34:24.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1be7fc640 1 -- 172.21.15.5:0/841550341 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561eb710ba20 con 0x561eb71b4b70 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1be7fc640 10 client.19206 handle_command_reply: tid=0 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 client.19206 shutdown 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 -- 172.21.15.5:0/841550341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 msgr2=0x561eb71a25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb71a25d0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009c10 tx=0x7fd1d0009280 comp rx=0 tx=0).stop 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 -- 172.21.15.5:0/841550341 shutdown_connections 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561eb71b4b70 0x561eb71b6fe0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb71a25d0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1be7fc640 0 client.19206 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb719a410 0x561eb71a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 --2- 172.21.15.5:0/841550341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb71a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 -- 172.21.15.5:0/841550341 >> 172.21.15.5:0/841550341 conn(0x561eb7196060 msgr2=0x561eb719bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 -- 172.21.15.5:0/841550341 shutdown_connections 2024-09-11T22:34:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 1 -- 172.21.15.5:0/841550341 wait complete. 2024-09-11T22:34:24.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.286+0000 7fd1e063a740 20 client.19206 trim_cache size 0 max 16384 2024-09-11T22:34:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.301+0000 7fd1e063a740 1 Processor -- start 2024-09-11T22:34:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.301+0000 7fd1e063a740 1 -- start start 2024-09-11T22:34:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.301+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb719c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.301+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.301+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb71a4cb0 0x561eb71bc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1e063a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81ad900 con 0x561eb719a410 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1e063a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81af470 con 0x561eb71a4cb0 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1e063a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81adfd0 con 0x561eb7199af0 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49974/0 (socket says 172.21.15.5:49974) 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d67fc640 1 -- 172.21.15.5:0/2989007080 learned_addr learned my addr 172.21.15.5:0/2989007080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb71a4cb0 0x561eb71bc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb719c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d77fe640 1 -- 172.21.15.5:0/2989007080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 msgr2=0x561eb719c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb719c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d77fe640 1 -- 172.21.15.5:0/2989007080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 msgr2=0x561eb71a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:24.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d77fe640 1 -- 172.21.15.5:0/2989007080 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb7199870 con 0x561eb71a4cb0 2024-09-11T22:34:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb719c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d67fc640 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.302+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb71a4cb0 0x561eb71bc940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0031c00 tx=0x7fd1d0030770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0996d1cef52631 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1ccff9640 1 -- 172.21.15.5:0/2989007080 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d004f040 con 0x561eb71a4cb0 2024-09-11T22:34:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1ccff9640 1 -- 172.21.15.5:0/2989007080 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d0037040 con 0x561eb71a4cb0 2024-09-11T22:34:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1ccff9640 1 -- 172.21.15.5:0/2989007080 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0036040 con 0x561eb71a4cb0 2024-09-11T22:34:24.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1e063a740 1 -- 172.21.15.5:0/2989007080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb71a4cb0 msgr2=0x561eb71bc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1e063a740 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb71a4cb0 0x561eb71bc940 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0031c00 tx=0x7fd1d0030770 comp rx=0 tx=0).stop 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1e063a740 1 -- 172.21.15.5:0/2989007080 shutdown_connections 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1e063a740 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb71a4cb0 0x561eb71bc940 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1e063a740 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb719a410 0x561eb71a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1e063a740 1 --2- 172.21.15.5:0/2989007080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb7199af0 0x561eb719c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1e063a740 1 -- 172.21.15.5:0/2989007080 >> 172.21.15.5:0/2989007080 conn(0x561eb7196060 msgr2=0x561eb71a3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1e063a740 1 -- 172.21.15.5:0/2989007080 shutdown_connections 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.303+0000 7fd1e063a740 1 -- 172.21.15.5:0/2989007080 wait complete. 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1e063a740 1 -- start start 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1e063a740 1 Processor -- start 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 0x561eb71ace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb719a410 0x561eb71ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1e063a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb719e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1e063a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81aeda0 con 0x561eb71a4cb0 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1e063a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81ae6d0 con 0x561eb7199af0 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1e063a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d81ab900 con 0x561eb719a410 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1e063a740 10 client.0 _ll_register_callbacks cb 0x7fd1c8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb719e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb719e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49986/0 (socket says 172.21.15.5:49986) 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1d77fe640 1 -- 172.21.15.5:0/2465656871 learned_addr learned my addr 172.21.15.5:0/2465656871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 0x561eb71ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.304+0000 7fd1d67fc640 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb719a410 0x561eb71ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1d77fe640 1 -- 172.21.15.5:0/2465656871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb719a410 msgr2=0x561eb71ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb719a410 0x561eb71ad3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1d77fe640 1 -- 172.21.15.5:0/2465656871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 msgr2=0x561eb71ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 0x561eb71ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1d77fe640 1 -- 172.21.15.5:0/2465656871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb719ebd0 con 0x561eb71a4cb0 2024-09-11T22:34:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1d77fe640 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb719e3a0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0004080 tx=0x7fd1d0003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f526b5c22c36df53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1be7fc640 1 -- 172.21.15.5:0/2465656871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0036070 con 0x561eb71a4cb0 2024-09-11T22:34:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1be7fc640 1 -- 172.21.15.5:0/2465656871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fd1d0031850 con 0x561eb71a4cb0 2024-09-11T22:34:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1e063a740 1 -- 172.21.15.5:0/2465656871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1d0037040 con 0x561eb71a4cb0 2024-09-11T22:34:24.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1be7fc640 1 -- 172.21.15.5:0/2465656871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd1d004f040 con 0x561eb71a4cb0 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1e063a740 10 client.19212 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1e063a740 1 -- 172.21.15.5:0/2465656871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x561eb721f4a0 con 0x561eb71a4cb0 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1be7fc640 1 -- 172.21.15.5:0/2465656871 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1d0037040 con 0x561eb71a4cb0 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.305+0000 7fd1be7fc640 1 -- 172.21.15.5:0/2465656871 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fd1d0070750 con 0x561eb71a4cb0 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.306+0000 7fd1e063a740 10 client.19212 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.306+0000 7fd1e063a740 10 client.19212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.306+0000 7fd1e063a740 20 client.19212 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.306+0000 7fd1e063a740 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561eb721fb50 0x561eb7221f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.306+0000 7fd1e063a740 4 client.19212 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.306+0000 7fd1e063a740 1 -- 172.21.15.5:0/2465656871 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561eb7105d70 con 0x561eb721fb50 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.306+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561eb721fb50 0x561eb7221f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.306+0000 7fd1be7fc640 10 client.19212 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:24.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.306+0000 7fd1d6ffd640 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561eb721fb50 0x561eb7221f40 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e2cb8c0bb8c9c7 server_cookie=12025d426a6850d9 in_seq=0 out_seq=0 2024-09-11T22:34:24.359 INFO:tasks.workunit.client.0.smithi005.stdout: 1 921320 7.36 MB/sec execute 522 sec latency 28.693 ms 2024-09-11T22:34:24.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.461+0000 7fd1be7fc640 1 -- 172.21.15.5:0/2465656871 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561eb7105d70 con 0x561eb721fb50 2024-09-11T22:34:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1be7fc640 10 client.19212 handle_command_reply: tid=0 2024-09-11T22:34:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 client.19212 shutdown 2024-09-11T22:34:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 -- 172.21.15.5:0/2465656871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 msgr2=0x561eb719e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb719e3a0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0004080 tx=0x7fd1d0003e30 comp rx=0 tx=0).stop 2024-09-11T22:34:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 -- 172.21.15.5:0/2465656871 shutdown_connections 2024-09-11T22:34:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x561eb721fb50 0x561eb7221f40 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561eb71a4cb0 0x561eb719e3a0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561eb719a410 0x561eb71ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 --2- 172.21.15.5:0/2465656871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561eb7199af0 0x561eb71ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 -- 172.21.15.5:0/2465656871 >> 172.21.15.5:0/2465656871 conn(0x561eb7196060 msgr2=0x561eb71949a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1be7fc640 0 client.19212 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 -- 172.21.15.5:0/2465656871 shutdown_connections 2024-09-11T22:34:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 1 -- 172.21.15.5:0/2465656871 wait complete. 2024-09-11T22:34:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.462+0000 7fd1e063a740 20 client.19212 trim_cache size 0 max 16384 2024-09-11T22:34:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:34:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:34:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:34:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:34:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.463+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3424620852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1c40a2da0 msgr2=0x7fd1c40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.463+0000 7fd1dd7c4640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1c40a2da0 0x7fd1c40a5260 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0004af0 tx=0x7fd1d0009500 comp rx=0 tx=0).stop 2024-09-11T22:34:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.463+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3424620852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1d8101350 msgr2=0x7fd1d8198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.463+0000 7fd1dd7c4640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1d8101350 0x7fd1d8198880 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000e9f0 tx=0x7fd1c000edf0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.463+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3424620852 shutdown_connections 2024-09-11T22:34:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.463+0000 7fd1dd7c4640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd1c40a2da0 0x7fd1c40a5260 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.463+0000 7fd1dd7c4640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1d81069c0 0x7fd1d819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.463+0000 7fd1dd7c4640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1d8101350 0x7fd1d8198880 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.463+0000 7fd1dd7c4640 1 --2- 172.21.15.5:0/3424620852 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1d81009b0 0x7fd1d8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.463+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3424620852 >> 172.21.15.5:0/3424620852 conn(0x7fd1d8077990 msgr2=0x7fd1d8069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.464+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3424620852 shutdown_connections 2024-09-11T22:34:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:24.464+0000 7fd1dd7c4640 1 -- 172.21.15.5:0/3424620852 wait complete. 2024-09-11T22:34:24.604 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:25.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 923174 7.37 MB/sec execute 523 sec latency 101.863 ms 2024-09-11T22:34:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.635+0000 7fe741316640 1 -- 172.21.15.5:0/884139141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe73c10e440 msgr2=0x7fe73c110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.635+0000 7fe741316640 1 --2- 172.21.15.5:0/884139141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe73c10e440 0x7fe73c110830 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fe734009ca0 tx=0x7fe73402a950 comp rx=0 tx=0).stop 2024-09-11T22:34:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.635+0000 7fe741316640 1 -- 172.21.15.5:0/884139141 shutdown_connections 2024-09-11T22:34:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.635+0000 7fe741316640 1 --2- 172.21.15.5:0/884139141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe73c10e440 0x7fe73c110830 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.635+0000 7fe741316640 1 --2- 172.21.15.5:0/884139141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c074e30 0x7fe73c10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.635+0000 7fe741316640 1 --2- 172.21.15.5:0/884139141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe73c074510 0x7fe73c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.635+0000 7fe741316640 1 -- 172.21.15.5:0/884139141 >> 172.21.15.5:0/884139141 conn(0x7fe73c0fd8c0 msgr2=0x7fe73c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.635+0000 7fe741316640 1 -- 172.21.15.5:0/884139141 shutdown_connections 2024-09-11T22:34:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.635+0000 7fe741316640 1 -- 172.21.15.5:0/884139141 wait complete. 2024-09-11T22:34:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.636+0000 7fe741316640 1 Processor -- start 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.636+0000 7fe741316640 1 -- start start 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe741316640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe73c074510 0x7fe73c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe741316640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c074e30 0x7fe73c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe741316640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe73c10e440 0x7fe73c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe741316640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c114160 con 0x7fe73c10e440 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe741316640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c113fe0 con 0x7fe73c074e30 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe741316640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1142e0 con 0x7fe73c074510 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe73c074510 0x7fe73c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe73c10e440 0x7fe73c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe73c074510 0x7fe73c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60268/0 (socket says 172.21.15.5:60268) 2024-09-11T22:34:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73affd640 1 -- 172.21.15.5:0/4223913460 learned_addr learned my addr 172.21.15.5:0/4223913460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c074e30 0x7fe73c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73b7fe640 1 -- 172.21.15.5:0/4223913460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe73c074510 msgr2=0x7fe73c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73b7fe640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe73c074510 0x7fe73c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73b7fe640 1 -- 172.21.15.5:0/4223913460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c074e30 msgr2=0x7fe73c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73b7fe640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c074e30 0x7fe73c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73b7fe640 1 -- 172.21.15.5:0/4223913460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c19f5f0 con 0x7fe73c10e440 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73affd640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe73c074510 0x7fe73c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe73b7fe640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe73c10e440 0x7fe73c19ef10 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe73402ae70 tx=0x7fe73402f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.637+0000 7fe723fff640 1 -- 172.21.15.5:0/4223913460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734034040 con 0x7fe73c10e440 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.638+0000 7fe741316640 1 -- 172.21.15.5:0/4223913460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c19f880 con 0x7fe73c10e440 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.638+0000 7fe723fff640 1 -- 172.21.15.5:0/4223913460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe734030920 con 0x7fe73c10e440 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.638+0000 7fe723fff640 1 -- 172.21.15.5:0/4223913460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7340045b0 con 0x7fe73c10e440 2024-09-11T22:34:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.638+0000 7fe741316640 1 -- 172.21.15.5:0/4223913460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c1a71c0 con 0x7fe73c10e440 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.640+0000 7fe723fff640 1 -- 172.21.15.5:0/4223913460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe734037020 con 0x7fe73c10e440 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.640+0000 7fe7436ef740 1 Processor -- start 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.640+0000 7fe7436ef740 1 -- start start 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.640+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340df2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.640+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df25d0 0x55e340df2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.640+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340dfd370 0x55e340dff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.640+0000 7fe7436ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e340d582f0 con 0x55e340df3b40 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.640+0000 7fe7436ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e340cffcf0 con 0x55e340dfd370 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.640+0000 7fe7436ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e340d542b0 con 0x55e340df25d0 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe723fff640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe7240a2cf0 0x7fe7240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df25d0 0x55e340df2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340df2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df25d0 0x55e340df2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60282/0 (socket says 172.21.15.5:60282) 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340df2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50020/0 (socket says 172.21.15.5:50020) 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73affd640 1 -- 172.21.15.5:0/1218646301 learned_addr learned my addr 172.21.15.5:0/1218646301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73a7fc640 1 --2- 172.21.15.5:0/1218646301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340dfd370 0x55e340dff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe723fff640 1 -- 172.21.15.5:0/4223913460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fe7340f00e0 con 0x7fe73c10e440 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73b7fe640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe7240a2cf0 0x7fe7240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73affd640 1 -- 172.21.15.5:0/1218646301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340dfd370 msgr2=0x55e340dff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73affd640 1 --2- 172.21.15.5:0/1218646301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340dfd370 0x55e340dff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73affd640 1 -- 172.21.15.5:0/1218646301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 msgr2=0x55e340df2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73affd640 1 --2- 172.21.15.5:0/1218646301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340df2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73affd640 1 -- 172.21.15.5:0/1218646301 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e340ce43a0 con 0x55e340df25d0 2024-09-11T22:34:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73a7fc640 1 --2- 172.21.15.5:0/1218646301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340df2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73affd640 1 --2- 172.21.15.5:0/1218646301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df25d0 0x55e340df2a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe72c011630 tx=0x7fe72c011af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea682f341249039c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.641+0000 7fe73b7fe640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe7240a2cf0 0x7fe7240a51b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe734006010 tx=0x7fe73402a950 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7217fa640 1 -- 172.21.15.5:0/1218646301 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c01e070 con 0x55e340df25d0 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7217fa640 1 -- 172.21.15.5:0/1218646301 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c00fbd0 con 0x55e340df25d0 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7217fa640 1 -- 172.21.15.5:0/1218646301 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c019c90 con 0x55e340df25d0 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7436ef740 1 -- 172.21.15.5:0/1218646301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df25d0 msgr2=0x55e340df2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7436ef740 1 --2- 172.21.15.5:0/1218646301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df25d0 0x55e340df2a10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe72c011630 tx=0x7fe72c011af0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7436ef740 1 -- 172.21.15.5:0/1218646301 shutdown_connections 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7436ef740 1 --2- 172.21.15.5:0/1218646301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340dfd370 0x55e340dff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7436ef740 1 --2- 172.21.15.5:0/1218646301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df25d0 0x55e340df2a10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7436ef740 1 --2- 172.21.15.5:0/1218646301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340df2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7436ef740 1 -- 172.21.15.5:0/1218646301 >> 172.21.15.5:0/1218646301 conn(0x55e340dbb310 msgr2=0x55e340deec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7436ef740 1 -- 172.21.15.5:0/1218646301 shutdown_connections 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.642+0000 7fe7436ef740 1 -- 172.21.15.5:0/1218646301 wait complete. 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe7436ef740 1 -- start start 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe7436ef740 1 Processor -- start 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df25d0 0x55e340dfcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df3b40 0x55e340dfa020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340dfd370 0x55e340dfa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe7436ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e340e00070 con 0x55e340df25d0 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe7436ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e340d13060 con 0x55e340df3b40 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe7436ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e340e001f0 con 0x55e340dfd370 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe7436ef740 10 client.0 _ll_register_callbacks cb 0x55e340a668c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df25d0 0x55e340dfcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe73affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df3b40 0x55e340dfa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df25d0 0x55e340dfcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50036/0 (socket says 172.21.15.5:50036) 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe73a7fc640 1 -- 172.21.15.5:0/232878591 learned_addr learned my addr 172.21.15.5:0/232878591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe73affd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df3b40 0x55e340dfa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56292/0 (socket says 172.21.15.5:56292) 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.643+0000 7fe73a7fc640 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340dfd370 0x55e340dfa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe73a7fc640 1 -- 172.21.15.5:0/232878591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340dfd370 msgr2=0x55e340dfa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe73a7fc640 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340dfd370 0x55e340dfa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe73a7fc640 1 -- 172.21.15.5:0/232878591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df3b40 msgr2=0x55e340dfa020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe73a7fc640 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df3b40 0x55e340dfa020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe73a7fc640 1 -- 172.21.15.5:0/232878591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e340ac93c0 con 0x55e340df25d0 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe73affd640 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df3b40 0x55e340dfa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe73a7fc640 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df25d0 0x55e340dfcd70 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe73000f430 tx=0x7fe730009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0819957f6fa697 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe71eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe71eff5640 1 -- 172.21.15.5:0/232878591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe730005220 con 0x55e340df25d0 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe71eff5640 1 -- 172.21.15.5:0/232878591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fe7300053c0 con 0x55e340df25d0 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe7436ef740 1 -- 172.21.15.5:0/232878591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe72c00fbd0 con 0x55e340df25d0 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe71eff5640 1 -- 172.21.15.5:0/232878591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe73002c050 con 0x55e340df25d0 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe7436ef740 10 client.19224 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe7436ef740 1 -- 172.21.15.5:0/232878591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e340e0bb10 con 0x55e340df25d0 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe71eff5640 1 -- 172.21.15.5:0/232878591 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c00fbd0 con 0x55e340df25d0 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.644+0000 7fe71eff5640 1 -- 172.21.15.5:0/232878591 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fe73000e650 con 0x55e340df25d0 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.645+0000 7fe7436ef740 10 client.19224 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.645+0000 7fe7436ef740 10 client.19224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.645+0000 7fe7436ef740 20 client.19224 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.645+0000 7fe7436ef740 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55e340e0cfd0 0x55e340e0f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.645+0000 7fe7436ef740 4 client.19224 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.645+0000 7fe7436ef740 1 -- 172.21.15.5:0/232878591 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e340d65860 con 0x55e340e0cfd0 2024-09-11T22:34:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.645+0000 7fe73affd640 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55e340e0cfd0 0x55e340e0f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.645+0000 7fe71eff5640 10 client.19224 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.645+0000 7fe73affd640 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55e340e0cfd0 0x55e340e0f430 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ab98d0f4cd1c4161 server_cookie=d6533f188d5d32ea in_seq=0 out_seq=0 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe71eff5640 1 -- 172.21.15.5:0/232878591 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e340d65860 con 0x55e340e0cfd0 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe71eff5640 10 client.19224 handle_command_reply: tid=0 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 client.19224 shutdown 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 -- 172.21.15.5:0/232878591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df25d0 msgr2=0x55e340dfcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df25d0 0x55e340dfcd70 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe73000f430 tx=0x7fe730009f90 comp rx=0 tx=0).stop 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 -- 172.21.15.5:0/232878591 shutdown_connections 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55e340e0cfd0 0x55e340e0f430 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340dfd370 0x55e340dfa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df3b40 0x55e340dfa020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 --2- 172.21.15.5:0/232878591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df25d0 0x55e340dfcd70 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 -- 172.21.15.5:0/232878591 >> 172.21.15.5:0/232878591 conn(0x55e340dbb310 msgr2=0x55e340df4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe71eff5640 0 client.19224 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:34:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 -- 172.21.15.5:0/232878591 shutdown_connections 2024-09-11T22:34:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 1 -- 172.21.15.5:0/232878591 wait complete. 2024-09-11T22:34:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.646+0000 7fe7436ef740 20 client.19224 trim_cache size 0 max 16384 2024-09-11T22:34:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.659+0000 7fe7436ef740 1 Processor -- start 2024-09-11T22:34:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.659+0000 7fe7436ef740 1 -- start start 2024-09-11T22:34:25.661 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:25 smithi005 ceph-mon[30943]: pgmap v388: 97 pgs: 97 active+clean; 93 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-11T22:34:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 0x55e340df3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340e09310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340dfd370 0x55e340df5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe7436ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1afbf0 con 0x55e340df3b40 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe7436ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1b1760 con 0x55e340df25d0 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe7436ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1b02c0 con 0x55e340dfd370 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe73affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340e09310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 0x55e340df3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340dfd370 0x55e340df5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe73affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340e09310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50052/0 (socket says 172.21.15.5:50052) 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe73affd640 1 -- 172.21.15.5:0/3260018486 learned_addr learned my addr 172.21.15.5:0/3260018486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe73affd640 1 -- 172.21.15.5:0/3260018486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340dfd370 msgr2=0x55e340df5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe73affd640 1 --2- 172.21.15.5:0/3260018486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340dfd370 0x55e340df5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe73affd640 1 -- 172.21.15.5:0/3260018486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 msgr2=0x55e340df3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe73affd640 1 --2- 172.21.15.5:0/3260018486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 0x55e340df3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.660+0000 7fe73affd640 1 -- 172.21.15.5:0/3260018486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e340df8590 con 0x55e340df3b40 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe73affd640 1 --2- 172.21.15.5:0/3260018486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340e09310 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe72c011630 tx=0x7fe72c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350b0963fb01269 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe720ff9640 1 -- 172.21.15.5:0/3260018486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c01e040 con 0x55e340df3b40 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe720ff9640 1 -- 172.21.15.5:0/3260018486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c00fb60 con 0x55e340df3b40 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe720ff9640 1 -- 172.21.15.5:0/3260018486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c019ab0 con 0x55e340df3b40 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe7436ef740 1 -- 172.21.15.5:0/3260018486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 msgr2=0x55e340e09310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe7436ef740 1 --2- 172.21.15.5:0/3260018486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340e09310 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe72c011630 tx=0x7fe72c00fa00 comp rx=0 tx=0).stop 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe7436ef740 1 -- 172.21.15.5:0/3260018486 shutdown_connections 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe7436ef740 1 --2- 172.21.15.5:0/3260018486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340dfd370 0x55e340df5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe7436ef740 1 --2- 172.21.15.5:0/3260018486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340df3b40 0x55e340e09310 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe7436ef740 1 --2- 172.21.15.5:0/3260018486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 0x55e340df3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.661+0000 7fe7436ef740 1 -- 172.21.15.5:0/3260018486 >> 172.21.15.5:0/3260018486 conn(0x55e340dbb310 msgr2=0x55e340de9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 1 -- 172.21.15.5:0/3260018486 shutdown_connections 2024-09-11T22:34:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 1 -- 172.21.15.5:0/3260018486 wait complete. 2024-09-11T22:34:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 1 -- start start 2024-09-11T22:34:25.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 1 Processor -- start 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 0x55e340e05660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df3b40 0x55e340e05ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340dfd370 0x55e340df6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1b1090 con 0x55e340dfd370 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1b09c0 con 0x55e340df25d0 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1adbf0 con 0x55e340df3b40 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe7436ef740 10 client.0 _ll_register_callbacks cb 0x7fe730009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe73affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df3b40 0x55e340e05ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.662+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 0x55e340e05660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73a7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 0x55e340e05660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56306/0 (socket says 172.21.15.5:56306) 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df3b40 0x55e340e05ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60320/0 (socket says 172.21.15.5:60320) 2024-09-11T22:34:25.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73a7fc640 1 -- 172.21.15.5:0/1518945045 learned_addr learned my addr 172.21.15.5:0/1518945045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73a7fc640 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340dfd370 0x55e340df6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73affd640 1 -- 172.21.15.5:0/1518945045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 msgr2=0x55e340e05660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73affd640 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 0x55e340e05660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73affd640 1 -- 172.21.15.5:0/1518945045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340dfd370 msgr2=0x55e340df6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73affd640 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340dfd370 0x55e340df6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73affd640 1 -- 172.21.15.5:0/1518945045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e340df7050 con 0x55e340df3b40 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73a7fc640 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 0x55e340e05660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe73affd640 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df3b40 0x55e340e05ba0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55e340e0bae0 tx=0x7fe72c00fd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7aaf2a2719603a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe71eff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe71eff5640 1 -- 172.21.15.5:0/1518945045 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c035070 con 0x55e340df3b40 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe7436ef740 1 -- 172.21.15.5:0/1518945045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe72c00fb60 con 0x55e340df3b40 2024-09-11T22:34:25.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.663+0000 7fe71eff5640 1 -- 172.21.15.5:0/1518945045 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fe72c019410 con 0x55e340df3b40 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe71eff5640 1 -- 172.21.15.5:0/1518945045 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fe72c01e040 con 0x55e340df3b40 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe7436ef740 10 client.28321 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe7436ef740 1 -- 172.21.15.5:0/1518945045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e340e77760 con 0x55e340df3b40 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe71eff5640 1 -- 172.21.15.5:0/1518945045 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c00fb60 con 0x55e340df3b40 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe71eff5640 1 -- 172.21.15.5:0/1518945045 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fe72c004680 con 0x55e340df3b40 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe7436ef740 10 client.28321 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe7436ef740 10 client.28321 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe7436ef740 20 client.28321 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe7436ef740 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55e340e77c80 0x55e340e7a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe7436ef740 4 client.28321 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe7436ef740 1 -- 172.21.15.5:0/1518945045 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e340d5fd70 con 0x55e340e77c80 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.664+0000 7fe73a7fc640 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55e340e77c80 0x55e340e7a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe73a7fc640 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55e340e77c80 0x55e340e7a070 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed7db4f1f9f1e9b2 server_cookie=6b00cdc4d788fff3 in_seq=0 out_seq=0 2024-09-11T22:34:25.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe71eff5640 10 client.28321 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe71eff5640 1 -- 172.21.15.5:0/1518945045 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e340d5fd70 con 0x55e340e77c80 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe71eff5640 10 client.28321 handle_command_reply: tid=0 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 client.28321 shutdown 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 -- 172.21.15.5:0/1518945045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df3b40 msgr2=0x55e340e05ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df3b40 0x55e340e05ba0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55e340e0bae0 tx=0x7fe72c00fd10 comp rx=0 tx=0).stop 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 -- 172.21.15.5:0/1518945045 shutdown_connections 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55e340e77c80 0x55e340e7a070 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e340dfd370 0x55e340df6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e340df3b40 0x55e340e05ba0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 --2- 172.21.15.5:0/1518945045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e340df25d0 0x55e340e05660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 -- 172.21.15.5:0/1518945045 >> 172.21.15.5:0/1518945045 conn(0x55e340dbb310 msgr2=0x55e340ded4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe71eff5640 0 client.28321 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:34:25.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 -- 172.21.15.5:0/1518945045 shutdown_connections 2024-09-11T22:34:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 1 -- 172.21.15.5:0/1518945045 wait complete. 2024-09-11T22:34:25.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.665+0000 7fe7436ef740 20 client.28321 trim_cache size 0 max 16384 2024-09-11T22:34:25.672 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 -- 172.21.15.5:0/4223913460 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe7240a2cf0 msgr2=0x7fe7240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe7240a2cf0 0x7fe7240a51b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe734006010 tx=0x7fe73402a950 comp rx=0 tx=0).stop 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 -- 172.21.15.5:0/4223913460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe73c10e440 msgr2=0x7fe73c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe73c10e440 0x7fe73c19ef10 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fe73402ae70 tx=0x7fe73402f3a0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 -- 172.21.15.5:0/4223913460 shutdown_connections 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe73c10e440 0x7fe73c19ef10 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe7240a2cf0 0x7fe7240a51b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe73c074e30 0x7fe73c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 --2- 172.21.15.5:0/4223913460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe73c074510 0x7fe73c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 -- 172.21.15.5:0/4223913460 >> 172.21.15.5:0/4223913460 conn(0x7fe73c0fd8c0 msgr2=0x7fe73c0fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 -- 172.21.15.5:0/4223913460 shutdown_connections 2024-09-11T22:34:25.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:25.666+0000 7fe741316640 1 -- 172.21.15.5:0/4223913460 wait complete. 2024-09-11T22:34:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:25 smithi159 ceph-mon[35534]: pgmap v388: 97 pgs: 97 active+clean; 93 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-11T22:34:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:25 smithi172 ceph-mon[35461]: pgmap v388: 97 pgs: 97 active+clean; 93 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.8 MiB/s wr, 186 op/s 2024-09-11T22:34:26.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 924897 7.37 MB/sec execute 524 sec latency 13.518 ms 2024-09-11T22:34:26.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:34:26.736 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.242+0000 7efde9309640 1 -- 172.21.15.5:0/2310514854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4101430 msgr2=0x7efde410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.242+0000 7efde9309640 1 --2- 172.21.15.5:0/2310514854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4101430 0x7efde410d960 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7efddc009cd0 tx=0x7efddc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.242+0000 7efde9309640 1 -- 172.21.15.5:0/2310514854 shutdown_connections 2024-09-11T22:34:27.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.242+0000 7efde9309640 1 --2- 172.21.15.5:0/2310514854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4101430 0x7efde410d960 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.242+0000 7efde9309640 1 --2- 172.21.15.5:0/2310514854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4100a90 0x7efde4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.242+0000 7efde9309640 1 --2- 172.21.15.5:0/2310514854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde4106a90 0x7efde4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.242+0000 7efde9309640 1 -- 172.21.15.5:0/2310514854 >> 172.21.15.5:0/2310514854 conn(0x7efde40fc7d0 msgr2=0x7efde40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.242+0000 7efde9309640 1 -- 172.21.15.5:0/2310514854 shutdown_connections 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.242+0000 7efde9309640 1 -- 172.21.15.5:0/2310514854 wait complete. 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.243+0000 7efde9309640 1 Processor -- start 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.243+0000 7efde9309640 1 -- start start 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde9309640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde4100a90 0x7efde41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde9309640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4101430 0x7efde4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde9309640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4106a90 0x7efde419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde9309640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efde4110e90 con 0x7efde4106a90 2024-09-11T22:34:27.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde9309640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efde4110d10 con 0x7efde4101430 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde9309640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efde4111010 con 0x7efde4100a90 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4106a90 0x7efde419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4106a90 0x7efde419cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50082/0 (socket says 172.21.15.5:50082) 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde27fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4101430 0x7efde4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde37fe640 1 -- 172.21.15.5:0/3368882338 learned_addr learned my addr 172.21.15.5:0/3368882338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde2ffd640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde4100a90 0x7efde41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde37fe640 1 -- 172.21.15.5:0/3368882338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde4100a90 msgr2=0x7efde41984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde37fe640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde4100a90 0x7efde41984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde37fe640 1 -- 172.21.15.5:0/3368882338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4101430 msgr2=0x7efde4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde37fe640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4101430 0x7efde4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde37fe640 1 -- 172.21.15.5:0/3368882338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde419d480 con 0x7efde4106a90 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde2ffd640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde4100a90 0x7efde41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde27fc640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4101430 0x7efde4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.244+0000 7efde37fe640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4106a90 0x7efde419cda0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7efddc005cc0 tx=0x7efddc0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.245+0000 7efdcbfff640 1 -- 172.21.15.5:0/3368882338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc003e00 con 0x7efde4106a90 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.245+0000 7efdcbfff640 1 -- 172.21.15.5:0/3368882338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efddc004520 con 0x7efde4106a90 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.245+0000 7efde9309640 1 -- 172.21.15.5:0/3368882338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde419d710 con 0x7efde4106a90 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.245+0000 7efdcbfff640 1 -- 172.21.15.5:0/3368882338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc030db0 con 0x7efde4106a90 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.245+0000 7efde9309640 1 -- 172.21.15.5:0/3368882338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde40fcca0 con 0x7efde4106a90 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efdcbfff640 1 -- 172.21.15.5:0/3368882338 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7efddc0046d0 con 0x7efde4106a90 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efdeb6e2740 1 Processor -- start 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efdeb6e2740 1 -- start start 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 0x55d0219b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219bd370 0x55d0219bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0219182f0 con 0x55d0219b25d0 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d0218bfcf0 con 0x55d0219bd370 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d0219142b0 con 0x55d0219b3b40 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efde2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efde27fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 0x55d0219b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efde2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50096/0 (socket says 172.21.15.5:50096) 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efde2ffd640 1 -- 172.21.15.5:0/2258400651 learned_addr learned my addr 172.21.15.5:0/2258400651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efdcbfff640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdcc0a2d70 0x7efdcc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.248+0000 7efde27fc640 1 --2- 172.21.15.5:0/2258400651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219bd370 0x55d0219bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efde2ffd640 1 -- 172.21.15.5:0/2258400651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 msgr2=0x55d0219b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efde2ffd640 1 --2- 172.21.15.5:0/2258400651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 0x55d0219b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efde2ffd640 1 -- 172.21.15.5:0/2258400651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219bd370 msgr2=0x55d0219bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efde2ffd640 1 --2- 172.21.15.5:0/2258400651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219bd370 0x55d0219bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efde37fe640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdcc0a2d70 0x7efdcc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efde2ffd640 1 -- 172.21.15.5:0/2258400651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0218a43a0 con 0x55d0219b25d0 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efde27fc640 1 --2- 172.21.15.5:0/2258400651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 0x55d0219b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efdcbfff640 1 -- 172.21.15.5:0/3368882338 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7efddc02e030 con 0x7efde4106a90 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efde2ffd640 1 --2- 172.21.15.5:0/2258400651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219b2a10 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009ca0 tx=0x7efdd40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c047b7b1c7be4387 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efdc97fa640 1 -- 172.21.15.5:0/2258400651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4012450 con 0x55d0219b25d0 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efdc97fa640 1 -- 172.21.15.5:0/2258400651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd4012ad0 con 0x55d0219b25d0 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.249+0000 7efdc97fa640 1 -- 172.21.15.5:0/2258400651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd400fb50 con 0x55d0219b25d0 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2258400651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 msgr2=0x55d0219b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/2258400651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219b2a10 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009ca0 tx=0x7efdd40117b0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efde37fe640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdcc0a2d70 0x7efdcc0a5230 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efddc0ee2e0 tx=0x7efddc049000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2258400651 shutdown_connections 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/2258400651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219bd370 0x55d0219bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/2258400651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219b2a10 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/2258400651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 0x55d0219b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2258400651 >> 172.21.15.5:0/2258400651 conn(0x55d02197b310 msgr2=0x55d0219aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2258400651 shutdown_connections 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2258400651 wait complete. 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.250+0000 7efdeb6e2740 1 -- start start 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efdeb6e2740 1 Processor -- start 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 0x55d0219b9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219bd370 0x55d0219ba570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0219c0070 con 0x55d0219b25d0 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d0218d3060 con 0x55d0219bd370 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d0219c01f0 con 0x55d0219b3b40 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efdeb6e2740 10 client.0 _ll_register_callbacks cb 0x55d0216268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219bcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219bcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50104/0 (socket says 172.21.15.5:50104) 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde27fc640 1 -- 172.21.15.5:0/2074154124 learned_addr learned my addr 172.21.15.5:0/2074154124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde2ffd640 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 0x55d0219b9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde27fc640 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219bd370 0x55d0219ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde27fc640 1 -- 172.21.15.5:0/2074154124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 msgr2=0x55d0219b9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde27fc640 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 0x55d0219b9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde27fc640 1 -- 172.21.15.5:0/2074154124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219bd370 msgr2=0x55d0219ba570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde27fc640 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219bd370 0x55d0219ba570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde27fc640 1 -- 172.21.15.5:0/2074154124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0216893c0 con 0x55d0219b25d0 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efde27fc640 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219bcd20 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7efdd8004530 tx=0x7efdd8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d17cd11f7013d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.251+0000 7efdc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdc6ff5640 1 -- 172.21.15.5:0/2074154124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8003d00 con 0x55d0219b25d0 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdc6ff5640 1 -- 172.21.15.5:0/2074154124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7efdd8005020 con 0x55d0219b25d0 2024-09-11T22:34:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2074154124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd4012ad0 con 0x55d0219b25d0 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdc6ff5640 1 -- 172.21.15.5:0/2074154124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7efdd8005220 con 0x55d0219b25d0 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdeb6e2740 10 client.19242 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2074154124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d0219cbb70 con 0x55d0219b25d0 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdc6ff5640 1 -- 172.21.15.5:0/2074154124 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd4012ad0 con 0x55d0219b25d0 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdc6ff5640 1 -- 172.21.15.5:0/2074154124 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7efdd8021450 con 0x55d0219b25d0 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdeb6e2740 10 client.19242 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdeb6e2740 10 client.19242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdeb6e2740 20 client.19242 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d0219cd030 0x55d0219cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdeb6e2740 4 client.19242 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.252+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2074154124 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d021925860 con 0x55d0219cd030 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.253+0000 7efde2ffd640 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d0219cd030 0x55d0219cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.253+0000 7efdc6ff5640 10 client.19242 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.253+0000 7efde2ffd640 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d0219cd030 0x55d0219cf420 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8d9ebbff2a536e5 server_cookie=893457fdf22ac1b4 in_seq=0 out_seq=0 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdc6ff5640 1 -- 172.21.15.5:0/2074154124 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d021925860 con 0x55d0219cd030 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdc6ff5640 10 client.19242 handle_command_reply: tid=0 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 client.19242 shutdown 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2074154124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 msgr2=0x55d0219bcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219bcd20 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7efdd8004530 tx=0x7efdd8005f90 comp rx=0 tx=0).stop 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2074154124 shutdown_connections 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d0219cd030 0x55d0219cf420 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219bd370 0x55d0219ba570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219b3b40 0x55d0219b9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdc6ff5640 0 client.19242 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/2074154124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219bcd20 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2074154124 >> 172.21.15.5:0/2074154124 conn(0x55d02197b310 msgr2=0x55d0219b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2074154124 shutdown_connections 2024-09-11T22:34:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 1 -- 172.21.15.5:0/2074154124 wait complete. 2024-09-11T22:34:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.254+0000 7efdeb6e2740 20 client.19242 trim_cache size 0 max 16384 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efdeb6e2740 1 Processor -- start 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efdeb6e2740 1 -- start start 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 0x55d0219b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219bd370 0x55d0219b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efde41afbb0 con 0x55d0219b25d0 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efde41b1720 con 0x55d0219b3b40 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efde41b0280 con 0x55d0219bd370 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efde27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efde27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50108/0 (socket says 172.21.15.5:50108) 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efde27fc640 1 -- 172.21.15.5:0/4038044367 learned_addr learned my addr 172.21.15.5:0/4038044367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efde2ffd640 1 --2- 172.21.15.5:0/4038044367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 0x55d0219b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.269+0000 7efde27fc640 1 --2- 172.21.15.5:0/4038044367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219bd370 0x55d0219b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.270+0000 7efde2ffd640 1 -- 172.21.15.5:0/4038044367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219bd370 msgr2=0x55d0219b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.270+0000 7efde2ffd640 1 --2- 172.21.15.5:0/4038044367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219bd370 0x55d0219b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.270+0000 7efde2ffd640 1 -- 172.21.15.5:0/4038044367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 msgr2=0x55d0219adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.270+0000 7efde2ffd640 1 --2- 172.21.15.5:0/4038044367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.270+0000 7efde2ffd640 1 -- 172.21.15.5:0/4038044367 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0219aeed0 con 0x55d0219b3b40 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.270+0000 7efde27fc640 1 --2- 172.21.15.5:0/4038044367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.270+0000 7efde2ffd640 1 --2- 172.21.15.5:0/4038044367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 0x55d0219b5120 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efdd401c040 tx=0x7efdd4009860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb1d1326773709c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.270+0000 7efdc8ff9640 1 -- 172.21.15.5:0/4038044367 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4031020 con 0x55d0219b3b40 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.270+0000 7efdc8ff9640 1 -- 172.21.15.5:0/4038044367 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd40098d0 con 0x55d0219b3b40 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.270+0000 7efdc8ff9640 1 -- 172.21.15.5:0/4038044367 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4005440 con 0x55d0219b3b40 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.271+0000 7efdeb6e2740 1 -- 172.21.15.5:0/4038044367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 msgr2=0x55d0219b5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.271+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/4038044367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 0x55d0219b5120 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efdd401c040 tx=0x7efdd4009860 comp rx=0 tx=0).stop 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.271+0000 7efdeb6e2740 1 -- 172.21.15.5:0/4038044367 shutdown_connections 2024-09-11T22:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.271+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/4038044367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219bd370 0x55d0219b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.271+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/4038044367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 0x55d0219b5120 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.271+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/4038044367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.271+0000 7efdeb6e2740 1 -- 172.21.15.5:0/4038044367 >> 172.21.15.5:0/4038044367 conn(0x55d02197b310 msgr2=0x55d0219ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.271+0000 7efdeb6e2740 1 -- 172.21.15.5:0/4038044367 shutdown_connections 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.271+0000 7efdeb6e2740 1 -- 172.21.15.5:0/4038044367 wait complete. 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.271+0000 7efdeb6e2740 1 -- start start 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efdeb6e2740 1 Processor -- start 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 0x55d0219c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efdeb6e2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219bd370 0x55d0219b6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efde41b1050 con 0x55d0219b25d0 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efde41b0980 con 0x55d0219b3b40 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efdeb6e2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7efde41adbb0 con 0x55d0219bd370 2024-09-11T22:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efdeb6e2740 10 client.0 _ll_register_callbacks cb 0x7efdd8004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219c5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50116/0 (socket says 172.21.15.5:50116) 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde27fc640 1 -- 172.21.15.5:0/3550253160 learned_addr learned my addr 172.21.15.5:0/3550253160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde2ffd640 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 0x55d0219c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde27fc640 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219bd370 0x55d0219b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde27fc640 1 -- 172.21.15.5:0/3550253160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219bd370 msgr2=0x55d0219b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde27fc640 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219bd370 0x55d0219b6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde27fc640 1 -- 172.21.15.5:0/3550253160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 msgr2=0x55d0219c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde27fc640 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 0x55d0219c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde27fc640 1 -- 172.21.15.5:0/3550253160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0219b67a0 con 0x55d0219b25d0 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde2ffd640 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 0x55d0219c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.272+0000 7efde27fc640 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219c5480 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7efdd80218a0 tx=0x7efdd8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53093189b53cbcf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdc6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdc6ff5640 1 -- 172.21.15.5:0/3550253160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd803d060 con 0x55d0219b25d0 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdc6ff5640 1 -- 172.21.15.5:0/3550253160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7efdd8003d00 con 0x55d0219b25d0 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdeb6e2740 1 -- 172.21.15.5:0/3550253160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d02191fd70 con 0x55d0219b25d0 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdc6ff5640 1 -- 172.21.15.5:0/3550253160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7efdd802d040 con 0x55d0219b25d0 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdeb6e2740 10 client.19248 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdeb6e2740 1 -- 172.21.15.5:0/3550253160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d0219b7aa0 con 0x55d0219b25d0 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdc6ff5640 1 -- 172.21.15.5:0/3550253160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d02191fd70 con 0x55d0219b25d0 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdc6ff5640 1 -- 172.21.15.5:0/3550253160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7efdd8033050 con 0x55d0219b25d0 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdeb6e2740 10 client.19248 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdeb6e2740 10 client.19248 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdeb6e2740 20 client.19248 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d021a38cd0 0x55d021a3b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdeb6e2740 4 client.19248 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.273+0000 7efdeb6e2740 1 -- 172.21.15.5:0/3550253160 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d02189c170 con 0x55d021a38cd0 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efde2ffd640 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d021a38cd0 0x55d021a3b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efde2ffd640 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d021a38cd0 0x55d021a3b0c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b3008b8f05954923 server_cookie=193e3f63266e98cf in_seq=0 out_seq=0 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdc6ff5640 10 client.19248 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdc6ff5640 1 -- 172.21.15.5:0/3550253160 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d02189c170 con 0x55d021a38cd0 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdc6ff5640 10 client.19248 handle_command_reply: tid=0 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdeb6e2740 1 client.19248 shutdown 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdeb6e2740 1 -- 172.21.15.5:0/3550253160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 msgr2=0x55d0219c5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219c5480 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7efdd80218a0 tx=0x7efdd8001e50 comp rx=0 tx=0).stop 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdeb6e2740 1 -- 172.21.15.5:0/3550253160 shutdown_connections 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55d021a38cd0 0x55d021a3b0c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0219bd370 0x55d0219b6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0219b3b40 0x55d0219c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdeb6e2740 1 --2- 172.21.15.5:0/3550253160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0219b25d0 0x55d0219c5480 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdeb6e2740 1 -- 172.21.15.5:0/3550253160 >> 172.21.15.5:0/3550253160 conn(0x55d02197b310 msgr2=0x55d0219add00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdc6ff5640 0 client.19248 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:34:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.274+0000 7efdeb6e2740 1 -- 172.21.15.5:0/3550253160 shutdown_connections 2024-09-11T22:34:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efdeb6e2740 1 -- 172.21.15.5:0/3550253160 wait complete. 2024-09-11T22:34:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efdeb6e2740 20 client.19248 trim_cache size 0 max 16384 2024-09-11T22:34:27.283 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:34:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efde9309640 1 -- 172.21.15.5:0/3368882338 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdcc0a2d70 msgr2=0x7efdcc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efde9309640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdcc0a2d70 0x7efdcc0a5230 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efddc0ee2e0 tx=0x7efddc049000 comp rx=0 tx=0).stop 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efde9309640 1 -- 172.21.15.5:0/3368882338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4106a90 msgr2=0x7efde419cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efde9309640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4106a90 0x7efde419cda0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7efddc005cc0 tx=0x7efddc0093d0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efde9309640 1 -- 172.21.15.5:0/3368882338 shutdown_connections 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efde9309640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efde4106a90 0x7efde419cda0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efde9309640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7efdcc0a2d70 0x7efdcc0a5230 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efde9309640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efde4101430 0x7efde4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efde9309640 1 --2- 172.21.15.5:0/3368882338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde4100a90 0x7efde41984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.275+0000 7efde9309640 1 -- 172.21.15.5:0/3368882338 >> 172.21.15.5:0/3368882338 conn(0x7efde40fc7d0 msgr2=0x7efde406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.276+0000 7efde9309640 1 -- 172.21.15.5:0/3368882338 shutdown_connections 2024-09-11T22:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:27.276+0000 7efde9309640 1 -- 172.21.15.5:0/3368882338 wait complete. 2024-09-11T22:34:27.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 927092 7.38 MB/sec execute 525 sec latency 94.803 ms 2024-09-11T22:34:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:27 smithi005 ceph-mon[30943]: pgmap v389: 97 pgs: 97 active+clean; 93 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-09-11T22:34:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:27 smithi159 ceph-mon[35534]: pgmap v389: 97 pgs: 97 active+clean; 93 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-09-11T22:34:27.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:27 smithi172 ceph-mon[35461]: pgmap v389: 97 pgs: 97 active+clean; 93 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 181 op/s 2024-09-11T22:34:28.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:34:28.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 928558 7.38 MB/sec execute 526 sec latency 44.106 ms 2024-09-11T22:34:28.383 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.926+0000 7fdfc457a640 1 -- 172.21.15.5:0/3624469851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc100e20 msgr2=0x7fdfbc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.926+0000 7fdfc457a640 1 --2- 172.21.15.5:0/3624469851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc100e20 0x7fdfbc101200 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009900 tx=0x7fdfac02a480 comp rx=0 tx=0).stop 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.926+0000 7fdfc457a640 1 -- 172.21.15.5:0/3624469851 shutdown_connections 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.926+0000 7fdfc457a640 1 --2- 172.21.15.5:0/3624469851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfbc102ef0 0x7fdfbc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.926+0000 7fdfc457a640 1 --2- 172.21.15.5:0/3624469851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfbc102570 0x7fdfbc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.926+0000 7fdfc457a640 1 --2- 172.21.15.5:0/3624469851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc100e20 0x7fdfbc101200 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.926+0000 7fdfc457a640 1 -- 172.21.15.5:0/3624469851 >> 172.21.15.5:0/3624469851 conn(0x7fdfbc0fc7d0 msgr2=0x7fdfbc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.926+0000 7fdfc457a640 1 -- 172.21.15.5:0/3624469851 shutdown_connections 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.926+0000 7fdfc457a640 1 -- 172.21.15.5:0/3624469851 wait complete. 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc457a640 1 Processor -- start 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc457a640 1 -- start start 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc457a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc100e20 0x7fdfbc1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc457a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfbc102570 0x7fdfbc1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc457a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfbc102ef0 0x7fdfbc19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc457a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc110f80 con 0x7fdfbc100e20 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc457a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc110e00 con 0x7fdfbc102570 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc457a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc111100 con 0x7fdfbc102ef0 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc2316640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc100e20 0x7fdfbc1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc2316640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc100e20 0x7fdfbc1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50124/0 (socket says 172.21.15.5:50124) 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc2316640 1 -- 172.21.15.5:0/4019221683 learned_addr learned my addr 172.21.15.5:0/4019221683 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.927+0000 7fdfc1b15640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfbc102570 0x7fdfbc1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfbc102ef0 0x7fdfbc19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfc2316640 1 -- 172.21.15.5:0/4019221683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfbc102ef0 msgr2=0x7fdfbc19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfc2316640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfbc102ef0 0x7fdfbc19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfc2316640 1 -- 172.21.15.5:0/4019221683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfbc102570 msgr2=0x7fdfbc1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfc2316640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfbc102570 0x7fdfbc1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfc2316640 1 -- 172.21.15.5:0/4019221683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfbc19d460 con 0x7fdfbc100e20 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfc2316640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc100e20 0x7fdfbc1984b0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006010 tx=0x7fdfac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfbb7fe640 1 -- 172.21.15.5:0/4019221683 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac03f070 con 0x7fdfbc100e20 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfc457a640 1 -- 172.21.15.5:0/4019221683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfbc19d6f0 con 0x7fdfbc100e20 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfbb7fe640 1 -- 172.21.15.5:0/4019221683 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfac031c30 con 0x7fdfbc100e20 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfbb7fe640 1 -- 172.21.15.5:0/4019221683 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac035030 con 0x7fdfbc100e20 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.928+0000 7fdfc457a640 1 -- 172.21.15.5:0/4019221683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfbc1a5030 con 0x7fdfbc100e20 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.931+0000 7fdfbb7fe640 1 -- 172.21.15.5:0/4019221683 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fdfac042070 con 0x7fdfbc100e20 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.931+0000 7fdfc6953740 1 Processor -- start 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.931+0000 7fdfc6953740 1 -- start start 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.931+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 0x55fdd247e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.931+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247e5d0 0x55fdd247ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.931+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 0x55fdd248b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.931+0000 7fdfc6953740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdd23e42f0 con 0x55fdd2489370 2024-09-11T22:34:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.931+0000 7fdfc6953740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fdd238bcf0 con 0x55fdd247e5d0 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.931+0000 7fdfc6953740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fdd23e02b0 con 0x55fdd247fb40 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2b17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 0x55fdd247e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2316640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247e5d0 0x55fdd247ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2b17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 0x55fdd247e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60386/0 (socket says 172.21.15.5:60386) 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2b17640 1 -- 172.21.15.5:0/2466314837 learned_addr learned my addr 172.21.15.5:0/2466314837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc1b15640 1 --2- 172.21.15.5:0/2466314837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 0x55fdd248b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfbb7fe640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfa40a2cf0 0x7fdfa40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2b17640 1 -- 172.21.15.5:0/2466314837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247e5d0 msgr2=0x55fdd247ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/2466314837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247e5d0 0x55fdd247ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2b17640 1 -- 172.21.15.5:0/2466314837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 msgr2=0x55fdd248b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/2466314837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 0x55fdd248b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2b17640 1 -- 172.21.15.5:0/2466314837 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd23703a0 con 0x55fdd247fb40 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfbb7fe640 1 -- 172.21.15.5:0/4019221683 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fdfac0efcd0 con 0x7fdfbc100e20 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfa40a2cf0 0x7fdfa40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/2466314837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 0x55fdd247e090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4010510 tx=0x7fdfb4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f78e44775c2cc11c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.932+0000 7fdfb8ff9640 1 -- 172.21.15.5:0/2466314837 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4023070 con 0x55fdd247fb40 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfb8ff9640 1 -- 172.21.15.5:0/2466314837 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb401dae0 con 0x55fdd247fb40 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfb8ff9640 1 -- 172.21.15.5:0/2466314837 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb401c950 con 0x55fdd247fb40 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfc6953740 1 -- 172.21.15.5:0/2466314837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 msgr2=0x55fdd247e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfa40a2cf0 0x7fdfa40a51b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4020000 tx=0x7fdfb401b000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfc6953740 1 --2- 172.21.15.5:0/2466314837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 0x55fdd247e090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4010510 tx=0x7fdfb4009f90 comp rx=0 tx=0).stop 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfc6953740 1 -- 172.21.15.5:0/2466314837 shutdown_connections 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfc6953740 1 --2- 172.21.15.5:0/2466314837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 0x55fdd248b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfc6953740 1 --2- 172.21.15.5:0/2466314837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247e5d0 0x55fdd247ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfc6953740 1 --2- 172.21.15.5:0/2466314837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 0x55fdd247e090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfc6953740 1 -- 172.21.15.5:0/2466314837 >> 172.21.15.5:0/2466314837 conn(0x55fdd2447310 msgr2=0x55fdd247ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfc6953740 1 -- 172.21.15.5:0/2466314837 shutdown_connections 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.933+0000 7fdfc6953740 1 -- 172.21.15.5:0/2466314837 wait complete. 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc6953740 1 -- start start 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc6953740 1 Processor -- start 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247e5d0 0x55fdd2488d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247fb40 0x55fdd2486010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 0x55fdd2486600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc6953740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdd248c070 con 0x55fdd2489370 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc6953740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fdd239f060 con 0x55fdd247fb40 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc6953740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fdd248c1f0 con 0x55fdd247e5d0 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc6953740 10 client.0 _ll_register_callbacks cb 0x55fdd20f28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc2b17640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 0x55fdd2486600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc2316640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247e5d0 0x55fdd2488d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc2b17640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 0x55fdd2486600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50154/0 (socket says 172.21.15.5:50154) 2024-09-11T22:34:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc1b15640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247fb40 0x55fdd2486010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.934+0000 7fdfc2b17640 1 -- 172.21.15.5:0/1672803626 learned_addr learned my addr 172.21.15.5:0/1672803626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc2b17640 1 -- 172.21.15.5:0/1672803626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247e5d0 msgr2=0x55fdd2488d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247e5d0 0x55fdd2488d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc2b17640 1 -- 172.21.15.5:0/1672803626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247fb40 msgr2=0x55fdd2486010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247fb40 0x55fdd2486010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc2b17640 1 -- 172.21.15.5:0/1672803626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdd21553c0 con 0x55fdd2489370 2024-09-11T22:34:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc2316640 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247e5d0 0x55fdd2488d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc1b15640 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247fb40 0x55fdd2486010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 0x55fdd2486600 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009640 tx=0x7fdfb4009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515407dd8251f88c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/1672803626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4023050 con 0x55fdd2489370 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc6953740 1 -- 172.21.15.5:0/1672803626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfb401dae0 con 0x55fdd2489370 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/1672803626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fdfb4003b70 con 0x55fdd2489370 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/1672803626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fdfb4003d70 con 0x55fdd2489370 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc6953740 10 client.19260 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc6953740 1 -- 172.21.15.5:0/1672803626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fdd2497b10 con 0x55fdd2489370 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/1672803626 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb401dae0 con 0x55fdd2489370 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/1672803626 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fdfb402e070 con 0x55fdd2489370 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc6953740 10 client.19260 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.935+0000 7fdfc6953740 10 client.19260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.936+0000 7fdfc6953740 20 client.19260 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.936+0000 7fdfc6953740 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fdd2499010 0x55fdd249b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.936+0000 7fdfc6953740 4 client.19260 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.936+0000 7fdfc6953740 1 -- 172.21.15.5:0/1672803626 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdd23f1860 con 0x55fdd2499010 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.936+0000 7fdfc2316640 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fdd2499010 0x55fdd249b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.936+0000 7fdfc2316640 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fdd2499010 0x55fdd249b460 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baaf4553a726d2f7 server_cookie=b32aaf6b221d3ae6 in_seq=0 out_seq=0 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.936+0000 7fdfa1ffb640 10 client.19260 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.937+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/1672803626 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdd23f1860 con 0x55fdd2499010 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.937+0000 7fdfa1ffb640 10 client.19260 handle_command_reply: tid=0 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 client.19260 shutdown 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 -- 172.21.15.5:0/1672803626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 msgr2=0x55fdd2486600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 0x55fdd2486600 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009640 tx=0x7fdfb4009b10 comp rx=0 tx=0).stop 2024-09-11T22:34:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 -- 172.21.15.5:0/1672803626 shutdown_connections 2024-09-11T22:34:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fdd2499010 0x55fdd249b460 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd2489370 0x55fdd2486600 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247fb40 0x55fdd2486010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfa1ffb640 0 client.19260 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 --2- 172.21.15.5:0/1672803626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247e5d0 0x55fdd2488d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 -- 172.21.15.5:0/1672803626 >> 172.21.15.5:0/1672803626 conn(0x55fdd2447310 msgr2=0x55fdd2480960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 -- 172.21.15.5:0/1672803626 shutdown_connections 2024-09-11T22:34:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 1 -- 172.21.15.5:0/1672803626 wait complete. 2024-09-11T22:34:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.938+0000 7fdfc6953740 20 client.19260 trim_cache size 0 max 16384 2024-09-11T22:34:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.952+0000 7fdfc6953740 1 Processor -- start 2024-09-11T22:34:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.952+0000 7fdfc6953740 1 -- start start 2024-09-11T22:34:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.952+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247e5d0 0x55fdd2481120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.952+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 0x55fdd2481660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.952+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd2489370 0x55fdd2483290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.952+0000 7fdfc6953740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1ad9b0 con 0x55fdd247e5d0 2024-09-11T22:34:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.952+0000 7fdfc6953740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1af520 con 0x55fdd2489370 2024-09-11T22:34:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.952+0000 7fdfc6953740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1ae080 con 0x55fdd247fb40 2024-09-11T22:34:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2316640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247e5d0 0x55fdd2481120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2b17640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd2489370 0x55fdd2483290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc1b15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 0x55fdd2481660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2316640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247e5d0 0x55fdd2481120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50168/0 (socket says 172.21.15.5:50168) 2024-09-11T22:34:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2316640 1 -- 172.21.15.5:0/2752084516 learned_addr learned my addr 172.21.15.5:0/2752084516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2b17640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd2489370 0x55fdd2483290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56418/0 (socket says 172.21.15.5:56418) 2024-09-11T22:34:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2316640 1 -- 172.21.15.5:0/2752084516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 msgr2=0x55fdd2481660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2316640 1 --2- 172.21.15.5:0/2752084516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 0x55fdd2481660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2316640 1 -- 172.21.15.5:0/2752084516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd2489370 msgr2=0x55fdd2483290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2316640 1 --2- 172.21.15.5:0/2752084516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd2489370 0x55fdd2483290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2316640 1 -- 172.21.15.5:0/2752084516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdd24890d0 con 0x55fdd247e5d0 2024-09-11T22:34:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/2752084516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd2489370 0x55fdd2483290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfc2316640 1 --2- 172.21.15.5:0/2752084516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247e5d0 0x55fdd2481120 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0bc160 tx=0x7fdfac0f2120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eb7334d4057c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:28.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfa3fff640 1 -- 172.21.15.5:0/2752084516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac049040 con 0x55fdd247e5d0 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfa3fff640 1 -- 172.21.15.5:0/2752084516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfac031de0 con 0x55fdd247e5d0 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.953+0000 7fdfa3fff640 1 -- 172.21.15.5:0/2752084516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0dc0b0 con 0x55fdd247e5d0 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 -- 172.21.15.5:0/2752084516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247e5d0 msgr2=0x55fdd2481120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 --2- 172.21.15.5:0/2752084516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247e5d0 0x55fdd2481120 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0bc160 tx=0x7fdfac0f2120 comp rx=0 tx=0).stop 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 -- 172.21.15.5:0/2752084516 shutdown_connections 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 --2- 172.21.15.5:0/2752084516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd2489370 0x55fdd2483290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 --2- 172.21.15.5:0/2752084516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd247fb40 0x55fdd2481660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 --2- 172.21.15.5:0/2752084516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247e5d0 0x55fdd2481120 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 -- 172.21.15.5:0/2752084516 >> 172.21.15.5:0/2752084516 conn(0x55fdd2447310 msgr2=0x55fdd2485100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 -- 172.21.15.5:0/2752084516 shutdown_connections 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 -- 172.21.15.5:0/2752084516 wait complete. 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 -- start start 2024-09-11T22:34:28.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.954+0000 7fdfc6953740 1 Processor -- start 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247e5d0 0x55fdd24fbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247fb40 0x55fdd24fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc6953740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd2489370 0x55fdd2506020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc6953740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1aee50 con 0x55fdd247fb40 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc6953740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1ae780 con 0x55fdd247e5d0 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc6953740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdfbc1ab9b0 con 0x55fdd2489370 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc6953740 10 client.0 _ll_register_callbacks cb 0x7fdfb4021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc1b15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247fb40 0x55fdd24fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc2316640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247e5d0 0x55fdd24fbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc1b15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247fb40 0x55fdd24fc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50174/0 (socket says 172.21.15.5:50174) 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc1b15640 1 -- 172.21.15.5:0/3001582538 learned_addr learned my addr 172.21.15.5:0/3001582538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc2b17640 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd2489370 0x55fdd2506020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc1b15640 1 -- 172.21.15.5:0/3001582538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd2489370 msgr2=0x55fdd2506020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc1b15640 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd2489370 0x55fdd2506020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc1b15640 1 -- 172.21.15.5:0/3001582538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247e5d0 msgr2=0x55fdd24fbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc1b15640 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247e5d0 0x55fdd24fbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc1b15640 1 -- 172.21.15.5:0/3001582538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdd24fcda0 con 0x55fdd247fb40 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfc1b15640 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247fb40 0x55fdd24fc4e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x55fdd2489340 tx=0x7fdfb00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3d53bde1972bc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.955+0000 7fdfa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/3001582538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0021020 con 0x55fdd247fb40 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/3001582538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fdfb0005020 con 0x55fdd247fb40 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfc6953740 1 -- 172.21.15.5:0/3001582538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfac031de0 con 0x55fdd247fb40 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/3001582538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fdfb0029050 con 0x55fdd247fb40 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfc6953740 10 client.19266 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfc6953740 1 -- 172.21.15.5:0/3001582538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55fdd25076f0 con 0x55fdd247fb40 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/3001582538 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfac031de0 con 0x55fdd247fb40 2024-09-11T22:34:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/3001582538 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fdfb00116c0 con 0x55fdd247fb40 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfc6953740 10 client.19266 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfc6953740 10 client.19266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfc6953740 20 client.19266 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfc6953740 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fdd2507ef0 0x55fdd250a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfc6953740 4 client.19266 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfc6953740 1 -- 172.21.15.5:0/3001582538 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fdd23ebd70 con 0x55fdd2507ef0 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.956+0000 7fdfc2316640 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fdd2507ef0 0x55fdd250a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfa1ffb640 10 client.19266 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc2316640 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fdd2507ef0 0x55fdd250a310 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6a1be676f33bf20 server_cookie=1cc765a25aa5e7a5 in_seq=0 out_seq=0 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfa1ffb640 1 -- 172.21.15.5:0/3001582538 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fdd23ebd70 con 0x55fdd2507ef0 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfa1ffb640 10 client.19266 handle_command_reply: tid=0 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 client.19266 shutdown 2024-09-11T22:34:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 -- 172.21.15.5:0/3001582538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247fb40 msgr2=0x55fdd24fc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247fb40 0x55fdd24fc4e0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x55fdd2489340 tx=0x7fdfb00039c0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 -- 172.21.15.5:0/3001582538 shutdown_connections 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fdd2507ef0 0x55fdd250a310 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fdd2489370 0x55fdd2506020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdd247fb40 0x55fdd24fc4e0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfa1ffb640 0 client.19266 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 --2- 172.21.15.5:0/3001582538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdd247e5d0 0x55fdd24fbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 -- 172.21.15.5:0/3001582538 >> 172.21.15.5:0/3001582538 conn(0x55fdd2447310 msgr2=0x55fdd2487870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 -- 172.21.15.5:0/3001582538 shutdown_connections 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 1 -- 172.21.15.5:0/3001582538 wait complete. 2024-09-11T22:34:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.957+0000 7fdfc6953740 20 client.19266 trim_cache size 0 max 16384 2024-09-11T22:34:28.964 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:34:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 -- 172.21.15.5:0/4019221683 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfa40a2cf0 msgr2=0x7fdfa40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfa40a2cf0 0x7fdfa40a51b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4020000 tx=0x7fdfb401b000 comp rx=0 tx=0).stop 2024-09-11T22:34:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 -- 172.21.15.5:0/4019221683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc100e20 msgr2=0x7fdfbc1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc100e20 0x7fdfbc1984b0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fdfac006010 tx=0x7fdfac003c10 comp rx=0 tx=0).stop 2024-09-11T22:34:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 -- 172.21.15.5:0/4019221683 shutdown_connections 2024-09-11T22:34:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfbc102ef0 0x7fdfbc19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdfa40a2cf0 0x7fdfa40a51b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdfbc102570 0x7fdfbc1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 --2- 172.21.15.5:0/4019221683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfbc100e20 0x7fdfbc1984b0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 -- 172.21.15.5:0/4019221683 >> 172.21.15.5:0/4019221683 conn(0x7fdfbc0fc7d0 msgr2=0x7fdfbc06a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 -- 172.21.15.5:0/4019221683 shutdown_connections 2024-09-11T22:34:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:28.958+0000 7fdfc457a640 1 -- 172.21.15.5:0/4019221683 wait complete. 2024-09-11T22:34:29.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 930882 7.39 MB/sec execute 527 sec latency 38.883 ms 2024-09-11T22:34:29.773 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:29 smithi005 ceph-mon[30943]: pgmap v390: 97 pgs: 97 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 359 op/s 2024-09-11T22:34:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:29 smithi159 ceph-mon[35534]: pgmap v390: 97 pgs: 97 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 359 op/s 2024-09-11T22:34:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:29 smithi172 ceph-mon[35461]: pgmap v390: 97 pgs: 97 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 359 op/s 2024-09-11T22:34:30.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 932542 7.39 MB/sec execute 528 sec latency 14.394 ms 2024-09-11T22:34:30.348 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-11T22:34:30.348 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:34:30.606 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:30.801 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:34:30.802 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:34:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.110+0000 7f2b320a3640 1 -- 172.21.15.5:0/2271674315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2c100a60 msgr2=0x7f2b2c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.110+0000 7f2b320a3640 1 --2- 172.21.15.5:0/2271674315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2c100a60 0x7f2b2c100ec0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009930 tx=0x7f2b2002a420 comp rx=0 tx=0).stop 2024-09-11T22:34:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.110+0000 7f2b320a3640 1 -- 172.21.15.5:0/2271674315 shutdown_connections 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.110+0000 7f2b320a3640 1 --2- 172.21.15.5:0/2271674315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2c101400 0x7f2b2c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.110+0000 7f2b320a3640 1 --2- 172.21.15.5:0/2271674315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2c100a60 0x7f2b2c100ec0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.110+0000 7f2b320a3640 1 --2- 172.21.15.5:0/2271674315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b2c106a60 0x7f2b2c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.110+0000 7f2b320a3640 1 -- 172.21.15.5:0/2271674315 >> 172.21.15.5:0/2271674315 conn(0x7f2b2c0fc780 msgr2=0x7f2b2c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.110+0000 7f2b320a3640 1 -- 172.21.15.5:0/2271674315 shutdown_connections 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.110+0000 7f2b320a3640 1 -- 172.21.15.5:0/2271674315 wait complete. 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.110+0000 7f2b320a3640 1 Processor -- start 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b320a3640 1 -- start start 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b320a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b2c100a60 0x7f2b2c102630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b320a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2c101400 0x7f2b2c102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b320a3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2c106a60 0x7f2b2c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b320a3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c110e60 con 0x7f2b2c101400 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b320a3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c110ce0 con 0x7f2b2c106a60 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b320a3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2c110fe0 con 0x7f2b2c100a60 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b2affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2c101400 0x7f2b2c102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b2b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b2c100a60 0x7f2b2c102630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b2affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2c101400 0x7f2b2c102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50200/0 (socket says 172.21.15.5:50200) 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b2affd640 1 -- 172.21.15.5:0/2521413108 learned_addr learned my addr 172.21.15.5:0/2521413108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b2bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2c106a60 0x7f2b2c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b2affd640 1 -- 172.21.15.5:0/2521413108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b2c100a60 msgr2=0x7f2b2c102630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b2affd640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b2c100a60 0x7f2b2c102630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b2affd640 1 -- 172.21.15.5:0/2521413108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2c106a60 msgr2=0x7f2b2c1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b2affd640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2c106a60 0x7f2b2c1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.111+0000 7f2b2affd640 1 -- 172.21.15.5:0/2521413108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2c1a6940 con 0x7f2b2c101400 2024-09-11T22:34:31.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.112+0000 7f2b2affd640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2c101400 0x7f2b2c102b70 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009230 tx=0x7f2b20004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.112+0000 7f2b28ff9640 1 -- 172.21.15.5:0/2521413108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2003f030 con 0x7f2b2c101400 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.112+0000 7f2b320a3640 1 -- 172.21.15.5:0/2521413108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2c1a6bd0 con 0x7f2b2c101400 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.112+0000 7f2b28ff9640 1 -- 172.21.15.5:0/2521413108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b20003ad0 con 0x7f2b2c101400 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.112+0000 7f2b28ff9640 1 -- 172.21.15.5:0/2521413108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20035070 con 0x7f2b2c101400 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.112+0000 7f2b320a3640 1 -- 172.21.15.5:0/2521413108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b2c1a70b0 con 0x7f2b2c101400 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.113+0000 7f2b320a3640 1 -- 172.21.15.5:0/2521413108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2c100ec0 con 0x7f2b2c101400 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.114+0000 7f2b28ff9640 1 -- 172.21.15.5:0/2521413108 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2b20042030 con 0x7f2b2c101400 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.115+0000 7f2b28ff9640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b140a2cf0 0x7f2b140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.115+0000 7f2b2b7fe640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b140a2cf0 0x7f2b140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.115+0000 7f2b28ff9640 1 -- 172.21.15.5:0/2521413108 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f2b200efa50 con 0x7f2b2c101400 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.115+0000 7f2b2b7fe640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b140a2cf0 0x7f2b140a51b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00bb60 tx=0x7f2b1c00b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:31.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.119+0000 7f2b28ff9640 1 -- 172.21.15.5:0/2521413108 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b200bbfb0 con 0x7f2b2c101400 2024-09-11T22:34:31.203 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:31.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.339+0000 7f2b320a3640 1 -- 172.21.15.5:0/2521413108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b2c1a7280 con 0x7f2b2c101400 2024-09-11T22:34:31.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.339+0000 7f2b28ff9640 1 -- 172.21.15.5:0/2521413108 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11968 (secure 0 0 0) 0x7f2b200c02b0 con 0x7f2b2c101400 2024-09-11T22:34:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:34:31.341 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":46,"btime":"2024-09-11T22:34:22:505619+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:34:21.776249+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28160":{"gid":28160,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2346038927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2346038927},{"type":"v1","addr":"172.21.15.159:6833","nonce":2346038927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:34:31.341 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 46 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 -- 172.21.15.5:0/2521413108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b140a2cf0 msgr2=0x7f2b140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b140a2cf0 0x7f2b140a51b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00bb60 tx=0x7f2b1c00b390 comp rx=0 tx=0).stop 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 -- 172.21.15.5:0/2521413108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2c101400 msgr2=0x7f2b2c102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2c101400 0x7f2b2c102b70 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009230 tx=0x7f2b20004270 comp rx=0 tx=0).stop 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 -- 172.21.15.5:0/2521413108 shutdown_connections 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b140a2cf0 0x7f2b140a51b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2c106a60 0x7f2b2c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2c101400 0x7f2b2c102b70 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 --2- 172.21.15.5:0/2521413108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b2c100a60 0x7f2b2c102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 -- 172.21.15.5:0/2521413108 >> 172.21.15.5:0/2521413108 conn(0x7f2b2c0fc780 msgr2=0x7f2b2c104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:31.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 -- 172.21.15.5:0/2521413108 shutdown_connections 2024-09-11T22:34:31.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.342+0000 7f2b320a3640 1 -- 172.21.15.5:0/2521413108 wait complete. 2024-09-11T22:34:31.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 934819 7.40 MB/sec execute 529 sec latency 13.856 ms 2024-09-11T22:34:31.581 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:31 smithi005 ceph-mon[30943]: pgmap v391: 97 pgs: 97 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 268 op/s 2024-09-11T22:34:31.581 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:31 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2521413108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:34:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:31 smithi159 ceph-mon[35534]: pgmap v391: 97 pgs: 97 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 268 op/s 2024-09-11T22:34:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:31 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2521413108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:34:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:31 smithi172 ceph-mon[35461]: pgmap v391: 97 pgs: 97 active+clean; 83 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 268 op/s 2024-09-11T22:34:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:31 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2521413108' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 -- 172.21.15.5:0/3597876787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813c100e20 msgr2=0x7f813c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 --2- 172.21.15.5:0/3597876787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813c100e20 0x7f813c101200 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f8138009900 tx=0x7f813802a480 comp rx=0 tx=0).stop 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 -- 172.21.15.5:0/3597876787 shutdown_connections 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 --2- 172.21.15.5:0/3597876787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f813c102ef0 0x7f813c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 --2- 172.21.15.5:0/3597876787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c102570 0x7f813c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 --2- 172.21.15.5:0/3597876787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813c100e20 0x7f813c101200 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 -- 172.21.15.5:0/3597876787 >> 172.21.15.5:0/3597876787 conn(0x7f813c0fc7d0 msgr2=0x7f813c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 -- 172.21.15.5:0/3597876787 shutdown_connections 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 -- 172.21.15.5:0/3597876787 wait complete. 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 Processor -- start 2024-09-11T22:34:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.979+0000 7f8143f79640 1 -- start start 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8143f79640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c100e20 0x7f813c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8143f79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813c102570 0x7f813c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8143f79640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f813c102ef0 0x7f813c1a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8143f79640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f813c110fc0 con 0x7f813c102570 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8143f79640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f813c110e40 con 0x7f813c102ef0 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8143f79640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f813c111140 con 0x7f813c100e20 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8141514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813c102570 0x7f813c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8141514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813c102570 0x7f813c199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36772/0 (socket says 172.21.15.5:36772) 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8141514640 1 -- 172.21.15.5:0/1652447401 learned_addr learned my addr 172.21.15.5:0/1652447401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8141d15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c100e20 0x7f813c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8142516640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f813c102ef0 0x7f813c1a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8141514640 1 -- 172.21.15.5:0/1652447401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c100e20 msgr2=0x7f813c199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8141514640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c100e20 0x7f813c199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8141514640 1 -- 172.21.15.5:0/1652447401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f813c102ef0 msgr2=0x7f813c1a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8141514640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f813c102ef0 0x7f813c1a4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8141514640 1 -- 172.21.15.5:0/1652447401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813c1a49d0 con 0x7f813c102570 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8141514640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813c102570 0x7f813c199950 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f812c00e9f0 tx=0x7f812c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.980+0000 7f8132ffd640 1 -- 172.21.15.5:0/1652447401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c00cd80 con 0x7f813c102570 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.981+0000 7f8143f79640 1 -- 172.21.15.5:0/1652447401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813c1a4ba0 con 0x7f813c102570 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.981+0000 7f8132ffd640 1 -- 172.21.15.5:0/1652447401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812c005500 con 0x7f813c102570 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.981+0000 7f8132ffd640 1 -- 172.21.15.5:0/1652447401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c010690 con 0x7f813c102570 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.981+0000 7f8143f79640 1 -- 172.21.15.5:0/1652447401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f813c1a50b0 con 0x7f813c102570 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.983+0000 7f8132ffd640 1 -- 172.21.15.5:0/1652447401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f812c0046c0 con 0x7f813c102570 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.983+0000 7f8146352740 1 Processor -- start 2024-09-11T22:34:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.983+0000 7f8146352740 1 -- start start 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8146352740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfdb60 0x55bab3bfdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8146352740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 0x55bab3c08800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8146352740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3c08d40 0x55bab3c0b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8146352740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bab3b63280 con 0x55bab3bfdb60 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8146352740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bab3b0ab10 con 0x55bab3bfe480 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8146352740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bab3b5f240 con 0x55bab3c08d40 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8142516640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfdb60 0x55bab3bfdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8141d15640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 0x55bab3c08800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8142516640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfdb60 0x55bab3bfdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36776/0 (socket says 172.21.15.5:36776) 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8142516640 1 -- 172.21.15.5:0/2163148191 learned_addr learned my addr 172.21.15.5:0/2163148191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8141514640 1 --2- 172.21.15.5:0/2163148191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3c08d40 0x55bab3c0b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8132ffd640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81280a2cf0 0x7f81280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8141d15640 1 -- 172.21.15.5:0/2163148191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3c08d40 msgr2=0x55bab3c0b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8141d15640 1 --2- 172.21.15.5:0/2163148191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3c08d40 0x55bab3c0b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8141d15640 1 -- 172.21.15.5:0/2163148191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfdb60 msgr2=0x55bab3bfdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8141d15640 1 --2- 172.21.15.5:0/2163148191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfdb60 0x55bab3bfdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8141d15640 1 -- 172.21.15.5:0/2163148191 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bab3b171f0 con 0x55bab3bfe480 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8142516640 1 --2- 172.21.15.5:0/2163148191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfdb60 0x55bab3bfdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8142516640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81280a2cf0 0x7f81280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.984+0000 7f8132ffd640 1 -- 172.21.15.5:0/1652447401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f812c0cf8e0 con 0x7f813c102570 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.985+0000 7f8141d15640 1 --2- 172.21.15.5:0/2163148191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 0x55bab3c08800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8138005940 tx=0x7f81380040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d40887032790f9d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.985+0000 7f8127fff640 1 -- 172.21.15.5:0/2163148191 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138035030 con 0x55bab3bfe480 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.985+0000 7f8127fff640 1 -- 172.21.15.5:0/2163148191 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8138040070 con 0x55bab3bfe480 2024-09-11T22:34:31.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.985+0000 7f8127fff640 1 -- 172.21.15.5:0/2163148191 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380319d0 con 0x55bab3bfe480 2024-09-11T22:34:31.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 -- 172.21.15.5:0/2163148191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 msgr2=0x55bab3c08800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:31.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8142516640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81280a2cf0 0x7f81280a51b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f813400a720 tx=0x7f8134004b50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:31.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 --2- 172.21.15.5:0/2163148191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 0x55bab3c08800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8138005940 tx=0x7f81380040c0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 -- 172.21.15.5:0/2163148191 shutdown_connections 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 --2- 172.21.15.5:0/2163148191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3c08d40 0x55bab3c0b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 --2- 172.21.15.5:0/2163148191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 0x55bab3c08800 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 --2- 172.21.15.5:0/2163148191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfdb60 0x55bab3bfdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 -- 172.21.15.5:0/2163148191 >> 172.21.15.5:0/2163148191 conn(0x55bab3bfa0f0 msgr2=0x55bab3bfa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 -- 172.21.15.5:0/2163148191 shutdown_connections 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 -- 172.21.15.5:0/2163148191 wait complete. 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 -- start start 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.986+0000 7f8146352740 1 Processor -- start 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8146352740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c05be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8146352740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c06120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8146352740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3c08d40 0x55bab3c06660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8146352740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bab3c0b910 con 0x55bab3bfe480 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8146352740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bab3b1df10 con 0x55bab3c08d40 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8146352740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bab3c0ba90 con 0x55bab3bfdb60 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8146352740 10 client.0 _ll_register_callbacks cb 0x55bab3ae6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8141514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c06120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8141514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c06120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36778/0 (socket says 172.21.15.5:36778) 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8141514640 1 -- 172.21.15.5:0/2802353699 learned_addr learned my addr 172.21.15.5:0/2802353699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8141d15640 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c05be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8141514640 1 -- 172.21.15.5:0/2802353699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 msgr2=0x55bab3c05be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8141514640 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c05be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8141514640 1 -- 172.21.15.5:0/2802353699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3c08d40 msgr2=0x55bab3c06660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8141514640 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3c08d40 0x55bab3c06660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8141514640 1 -- 172.21.15.5:0/2802353699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bab38d46a0 con 0x55bab3bfe480 2024-09-11T22:34:31.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f8141514640 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c06120 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f812c00c6a0 tx=0x7f812c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0b253525438fc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.987+0000 7f81257fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f81257fa640 1 -- 172.21.15.5:0/2802353699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c09b8d0 con 0x55bab3bfe480 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f81257fa640 1 -- 172.21.15.5:0/2802353699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f812c0a4770 con 0x55bab3bfe480 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f8146352740 1 -- 172.21.15.5:0/2802353699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8138040070 con 0x55bab3bfe480 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f81257fa640 1 -- 172.21.15.5:0/2802353699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f812c09b6f0 con 0x55bab3bfe480 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f8146352740 10 client.19284 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f8146352740 1 -- 172.21.15.5:0/2802353699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bab3c17540 con 0x55bab3bfe480 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f81257fa640 1 -- 172.21.15.5:0/2802353699 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8138040070 con 0x55bab3bfe480 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f81257fa640 1 -- 172.21.15.5:0/2802353699 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f812c09b1e0 con 0x55bab3bfe480 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f8146352740 10 client.19284 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f8146352740 10 client.19284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f8146352740 20 client.19284 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:31.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f8146352740 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bab3c189a0 0x55bab3c1ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f8146352740 4 client.19284 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.988+0000 7f8146352740 1 -- 172.21.15.5:0/2802353699 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bab3b707f0 con 0x55bab3c189a0 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.989+0000 7f8141d15640 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bab3c189a0 0x55bab3c1ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.989+0000 7f8141d15640 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bab3c189a0 0x55bab3c1ae10 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24386937565d9b7 server_cookie=fa92834de78eb1b5 in_seq=0 out_seq=0 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.989+0000 7f81257fa640 10 client.19284 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.990+0000 7f81257fa640 1 -- 172.21.15.5:0/2802353699 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bab3b707f0 con 0x55bab3c189a0 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.990+0000 7f81257fa640 10 client.19284 handle_command_reply: tid=0 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.990+0000 7f8146352740 1 client.19284 shutdown 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 1 -- 172.21.15.5:0/2802353699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 msgr2=0x55bab3c06120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c06120 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f812c00c6a0 tx=0x7f812c009310 comp rx=0 tx=0).stop 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 1 -- 172.21.15.5:0/2802353699 shutdown_connections 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bab3c189a0 0x55bab3c1ae10 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3c08d40 0x55bab3c06660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c06120 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f81257fa640 0 client.19284 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:31.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 1 --2- 172.21.15.5:0/2802353699 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c05be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:31.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 1 -- 172.21.15.5:0/2802353699 >> 172.21.15.5:0/2802353699 conn(0x55bab3bfa0f0 msgr2=0x55bab3bffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:31.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 1 -- 172.21.15.5:0/2802353699 shutdown_connections 2024-09-11T22:34:31.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 1 -- 172.21.15.5:0/2802353699 wait complete. 2024-09-11T22:34:31.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:31.991+0000 7f8146352740 20 client.19284 trim_cache size 0 max 16384 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.009+0000 7f8146352740 1 Processor -- start 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.009+0000 7f8146352740 1 -- start start 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.009+0000 7f8146352740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c00610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.009+0000 7f8146352740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c07e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.009+0000 7f8146352740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3c08d40 0x55bab3c206e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.009+0000 7f8146352740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f813c1adab0 con 0x55bab3bfe480 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.009+0000 7f8146352740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f813c1af620 con 0x55bab3c08d40 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.009+0000 7f8146352740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f813c1ae180 con 0x55bab3bfdb60 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c07e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c07e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36792/0 (socket says 172.21.15.5:36792) 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141514640 1 -- 172.21.15.5:0/2943174017 learned_addr learned my addr 172.21.15.5:0/2943174017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:32.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8142516640 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3c08d40 0x55bab3c206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:32.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141d15640 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c00610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:32.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141514640 1 -- 172.21.15.5:0/2943174017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 msgr2=0x55bab3c00610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:32.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141514640 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c00610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141514640 1 -- 172.21.15.5:0/2943174017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3c08d40 msgr2=0x55bab3c206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:32.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141514640 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3c08d40 0x55bab3c206e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141514640 1 -- 172.21.15.5:0/2943174017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bab3bf9f20 con 0x55bab3bfe480 2024-09-11T22:34:32.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8142516640 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3c08d40 0x55bab3c206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:34:32.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141d15640 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c00610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:34:32.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f8141514640 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c07e00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f812c0981f0 tx=0x7f812c0cfc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af15cc6643e849ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:32.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f81277fe640 1 -- 172.21.15.5:0/2943174017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c09b8d0 con 0x55bab3bfe480 2024-09-11T22:34:32.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f81277fe640 1 -- 172.21.15.5:0/2943174017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812c0a9070 con 0x55bab3bfe480 2024-09-11T22:34:32.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.010+0000 7f81277fe640 1 -- 172.21.15.5:0/2943174017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c0a47b0 con 0x55bab3bfe480 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.011+0000 7f8146352740 1 -- 172.21.15.5:0/2943174017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 msgr2=0x55bab3c07e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.011+0000 7f8146352740 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c07e00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f812c0981f0 tx=0x7f812c0cfc70 comp rx=0 tx=0).stop 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.011+0000 7f8146352740 1 -- 172.21.15.5:0/2943174017 shutdown_connections 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.011+0000 7f8146352740 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3c08d40 0x55bab3c206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.011+0000 7f8146352740 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3bfe480 0x55bab3c07e00 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.011+0000 7f8146352740 1 --2- 172.21.15.5:0/2943174017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c00610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.011+0000 7f8146352740 1 -- 172.21.15.5:0/2943174017 >> 172.21.15.5:0/2943174017 conn(0x55bab3bfa0f0 msgr2=0x55bab3c04ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.011+0000 7f8146352740 1 -- 172.21.15.5:0/2943174017 shutdown_connections 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.011+0000 7f8146352740 1 -- 172.21.15.5:0/2943174017 wait complete. 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8146352740 1 -- start start 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8146352740 1 Processor -- start 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8146352740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c0cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:32.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8146352740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 0x55bab3c0d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8146352740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3c08d40 0x55bab3c0d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8146352740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f813c1aef50 con 0x55bab3c08d40 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8146352740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f813c1ae880 con 0x55bab3bfe480 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8146352740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f813c1abab0 con 0x55bab3bfdb60 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8146352740 10 client.0 _ll_register_callbacks cb 0x7f812c0d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8141d15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c0cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8142516640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3c08d40 0x55bab3c0d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8141d15640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c0cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36080/0 (socket says 172.21.15.5:36080) 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.012+0000 7f8141d15640 1 -- 172.21.15.5:0/2318587975 learned_addr learned my addr 172.21.15.5:0/2318587975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f8141514640 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 0x55bab3c0d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f8141d15640 1 -- 172.21.15.5:0/2318587975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 msgr2=0x55bab3c0d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f8141d15640 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 0x55bab3c0d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f8141d15640 1 -- 172.21.15.5:0/2318587975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3c08d40 msgr2=0x55bab3c0d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f8141d15640 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3c08d40 0x55bab3c0d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f8141d15640 1 -- 172.21.15.5:0/2318587975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bab3c82980 con 0x55bab3bfdb60 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f8141d15640 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c0cd60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8138009900 tx=0x7f8138005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99bfa4b282af5c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f81257fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f81257fa640 1 -- 172.21.15.5:0/2318587975 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138056020 con 0x55bab3bfdb60 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f8146352740 1 -- 172.21.15.5:0/2318587975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f812c0a9070 con 0x55bab3bfdb60 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f81257fa640 1 -- 172.21.15.5:0/2318587975 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f81380037f0 con 0x55bab3bfdb60 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f81257fa640 1 -- 172.21.15.5:0/2318587975 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8138035040 con 0x55bab3bfdb60 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f8146352740 10 client.28381 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.013+0000 7f8146352740 1 -- 172.21.15.5:0/2318587975 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55bab3c83cc0 con 0x55bab3bfdb60 2024-09-11T22:34:32.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f81257fa640 1 -- 172.21.15.5:0/2318587975 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812c0a9070 con 0x55bab3bfdb60 2024-09-11T22:34:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f81257fa640 1 -- 172.21.15.5:0/2318587975 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f8138004750 con 0x55bab3bfdb60 2024-09-11T22:34:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f8146352740 10 client.28381 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f8146352740 10 client.28381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f8146352740 20 client.28381 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f8146352740 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bab3c84070 0x55bab3c86460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f8146352740 4 client.28381 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:34:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f8146352740 1 -- 172.21.15.5:0/2318587975 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bab3b6ad00 con 0x55bab3c84070 2024-09-11T22:34:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f8141514640 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bab3c84070 0x55bab3c86460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f8141514640 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bab3c84070 0x55bab3c86460 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33eeb2ae2896e6c server_cookie=d0c209f82334304f in_seq=0 out_seq=0 2024-09-11T22:34:32.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.014+0000 7f81257fa640 10 client.28381 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f81257fa640 1 -- 172.21.15.5:0/2318587975 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bab3b6ad00 con 0x55bab3c84070 2024-09-11T22:34:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f81257fa640 10 client.28381 handle_command_reply: tid=0 2024-09-11T22:34:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 client.28381 shutdown 2024-09-11T22:34:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 -- 172.21.15.5:0/2318587975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 msgr2=0x55bab3c0cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:32.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c0cd60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8138009900 tx=0x7f8138005850 comp rx=0 tx=0).stop 2024-09-11T22:34:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 -- 172.21.15.5:0/2318587975 shutdown_connections 2024-09-11T22:34:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bab3c84070 0x55bab3c86460 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bab3c08d40 0x55bab3c0d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bab3bfe480 0x55bab3c0d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f81257fa640 0 client.28381 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 --2- 172.21.15.5:0/2318587975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bab3bfdb60 0x55bab3c0cd60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 -- 172.21.15.5:0/2318587975 >> 172.21.15.5:0/2318587975 conn(0x55bab3bfa0f0 msgr2=0x55bab3bf8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 -- 172.21.15.5:0/2318587975 shutdown_connections 2024-09-11T22:34:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 1 -- 172.21.15.5:0/2318587975 wait complete. 2024-09-11T22:34:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.016+0000 7f8146352740 20 client.28381 trim_cache size 0 max 16384 2024-09-11T22:34:32.022 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:34:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:34:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "133b49b4-3a55-4acd-8fc2-af33f4b5ce0a", 2024-09-11T22:34:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:34:32.022 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:34:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 -- 172.21.15.5:0/1652447401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81280a2cf0 msgr2=0x7f81280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81280a2cf0 0x7f81280a51b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f813400a720 tx=0x7f8134004b50 comp rx=0 tx=0).stop 2024-09-11T22:34:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 -- 172.21.15.5:0/1652447401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813c102570 msgr2=0x7f813c199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813c102570 0x7f813c199950 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f812c00e9f0 tx=0x7f812c00edf0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 -- 172.21.15.5:0/1652447401 shutdown_connections 2024-09-11T22:34:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81280a2cf0 0x7f81280a51b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f813c102ef0 0x7f813c1a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813c102570 0x7f813c199950 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 --2- 172.21.15.5:0/1652447401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f813c100e20 0x7f813c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 -- 172.21.15.5:0/1652447401 >> 172.21.15.5:0/1652447401 conn(0x7f813c0fc7d0 msgr2=0x7f813c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 -- 172.21.15.5:0/1652447401 shutdown_connections 2024-09-11T22:34:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:32.017+0000 7f8143f79640 1 -- 172.21.15.5:0/1652447401 wait complete. 2024-09-11T22:34:32.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 936912 7.40 MB/sec execute 530 sec latency 26.120 ms 2024-09-11T22:34:32.621 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-11T22:34:32.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-11T22:34:33.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:34:33.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 938632 7.41 MB/sec execute 531 sec latency 74.440 ms 2024-09-11T22:34:33.369 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:33.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:33 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:34:33.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:33 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:34:33.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:33 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:34:33.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:33 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:34:33.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:33 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:34:33.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:33 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:34:33.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:33 smithi005 ceph-mon[30943]: pgmap v392: 97 pgs: 97 active+clean; 90 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-09-11T22:34:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:33 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:34:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:33 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:34:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:33 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:34:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:33 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:34:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:33 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:34:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:33 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:34:33.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:33 smithi159 ceph-mon[35534]: pgmap v392: 97 pgs: 97 active+clean; 90 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-09-11T22:34:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:33 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:34:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:33 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:34:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:33 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:34:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:33 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:34:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:33 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:34:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:33 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:34:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:33 smithi172 ceph-mon[35461]: pgmap v392: 97 pgs: 97 active+clean; 90 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-09-11T22:34:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.929+0000 7f70f383f640 1 -- 172.21.15.5:0/2476803696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70ec1027c0 msgr2=0x7f70ec106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.929+0000 7f70f383f640 1 --2- 172.21.15.5:0/2476803696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70ec1027c0 0x7f70ec106810 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009900 tx=0x7f70dc02a410 comp rx=0 tx=0).stop 2024-09-11T22:34:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.929+0000 7f70f383f640 1 -- 172.21.15.5:0/2476803696 shutdown_connections 2024-09-11T22:34:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.929+0000 7f70f383f640 1 --2- 172.21.15.5:0/2476803696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec106f40 0x7f70ec10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.929+0000 7f70f383f640 1 --2- 172.21.15.5:0/2476803696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70ec1027c0 0x7f70ec106810 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.929+0000 7f70f383f640 1 --2- 172.21.15.5:0/2476803696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70ec101e10 0x7f70ec1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.929+0000 7f70f383f640 1 -- 172.21.15.5:0/2476803696 >> 172.21.15.5:0/2476803696 conn(0x7f70ec0fd8d0 msgr2=0x7f70ec0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.929+0000 7f70f383f640 1 -- 172.21.15.5:0/2476803696 shutdown_connections 2024-09-11T22:34:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.929+0000 7f70f383f640 1 -- 172.21.15.5:0/2476803696 wait complete. 2024-09-11T22:34:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f383f640 1 Processor -- start 2024-09-11T22:34:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f383f640 1 -- start start 2024-09-11T22:34:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f383f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70ec101e10 0x7f70ec073b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f383f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec1027c0 0x7f70ec06dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f383f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70ec106f40 0x7f70ec06e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f383f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec06c620 con 0x7f70ec106f40 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f383f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec06c4a0 con 0x7f70ec101e10 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f383f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec06c7a0 con 0x7f70ec1027c0 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f0dda640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec1027c0 0x7f70ec06dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f1ddc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70ec106f40 0x7f70ec06e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f15db640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70ec101e10 0x7f70ec073b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f1ddc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70ec106f40 0x7f70ec06e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36836/0 (socket says 172.21.15.5:36836) 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f1ddc640 1 -- 172.21.15.5:0/946378600 learned_addr learned my addr 172.21.15.5:0/946378600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f15db640 1 -- 172.21.15.5:0/946378600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec1027c0 msgr2=0x7f70ec06dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f15db640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec1027c0 0x7f70ec06dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f15db640 1 -- 172.21.15.5:0/946378600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70ec106f40 msgr2=0x7f70ec06e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.930+0000 7f70f15db640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70ec106f40 0x7f70ec06e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.931+0000 7f70f15db640 1 -- 172.21.15.5:0/946378600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ec06e810 con 0x7f70ec101e10 2024-09-11T22:34:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.931+0000 7f70f15db640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70ec101e10 0x7f70ec073b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f70e800e9f0 tx=0x7f70e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.931+0000 7f70e27fc640 1 -- 172.21.15.5:0/946378600 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e800cd80 con 0x7f70ec101e10 2024-09-11T22:34:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.931+0000 7f70e27fc640 1 -- 172.21.15.5:0/946378600 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e80035f0 con 0x7f70ec101e10 2024-09-11T22:34:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.931+0000 7f70f383f640 1 -- 172.21.15.5:0/946378600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ec1ad2f0 con 0x7f70ec101e10 2024-09-11T22:34:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.931+0000 7f70e27fc640 1 -- 172.21.15.5:0/946378600 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8010690 con 0x7f70ec101e10 2024-09-11T22:34:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.931+0000 7f70f383f640 1 -- 172.21.15.5:0/946378600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ec1ad740 con 0x7f70ec101e10 2024-09-11T22:34:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f5c18740 1 Processor -- start 2024-09-11T22:34:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f5c18740 1 -- start start 2024-09-11T22:34:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70e27fc640 1 -- 172.21.15.5:0/946378600 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f70e8003b30 con 0x7f70ec101e10 2024-09-11T22:34:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 0x55c190c7da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7df60 0x55c190c8c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c8cd50 0x55c190c8f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f5c18740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c190be32a0 con 0x55c190c8cd50 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f5c18740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c190b8ab10 con 0x55c190c7df60 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f5c18740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c190bdf260 con 0x55c190c7d640 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f1ddc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 0x55c190c7da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f1ddc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 0x55c190c7da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36086/0 (socket says 172.21.15.5:36086) 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f1ddc640 1 -- 172.21.15.5:0/2360525771 learned_addr learned my addr 172.21.15.5:0/2360525771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f0dda640 1 --2- 172.21.15.5:0/2360525771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c8cd50 0x55c190c8f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.934+0000 7f70f15db640 1 --2- 172.21.15.5:0/2360525771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7df60 0x55c190c8c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.935+0000 7f70f1ddc640 1 -- 172.21.15.5:0/2360525771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7df60 msgr2=0x55c190c8c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.935+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/2360525771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7df60 0x55c190c8c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.935+0000 7f70f1ddc640 1 -- 172.21.15.5:0/2360525771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c8cd50 msgr2=0x55c190c8f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.935+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/2360525771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c8cd50 0x55c190c8f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.935+0000 7f70f1ddc640 1 -- 172.21.15.5:0/2360525771 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c190954b90 con 0x55c190c7d640 2024-09-11T22:34:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.935+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/2360525771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 0x55c190c7da20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70e4011470 tx=0x7f70e4011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84ce6317005fbd6b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.935+0000 7f70d77fe640 1 -- 172.21.15.5:0/2360525771 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4004b20 con 0x55c190c7d640 2024-09-11T22:34:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.935+0000 7f70d77fe640 1 -- 172.21.15.5:0/2360525771 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e4004cc0 con 0x55c190c7d640 2024-09-11T22:34:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.935+0000 7f70d77fe640 1 -- 172.21.15.5:0/2360525771 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e40053f0 con 0x55c190c7d640 2024-09-11T22:34:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.935+0000 7f70e27fc640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f70d80a2dc0 0x7f70d80a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f15db640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f70d80a2dc0 0x7f70d80a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70e27fc640 1 -- 172.21.15.5:0/946378600 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f70e80cf6e0 con 0x7f70ec101e10 2024-09-11T22:34:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f5c18740 1 -- 172.21.15.5:0/2360525771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 msgr2=0x55c190c7da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f5c18740 1 --2- 172.21.15.5:0/2360525771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 0x55c190c7da20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f70e4011470 tx=0x7f70e4011930 comp rx=0 tx=0).stop 2024-09-11T22:34:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f15db640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f70d80a2dc0 0x7f70d80a5280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f70e80cce70 tx=0x7f70e80d5550 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f5c18740 1 -- 172.21.15.5:0/2360525771 shutdown_connections 2024-09-11T22:34:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f5c18740 1 --2- 172.21.15.5:0/2360525771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c8cd50 0x55c190c8f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f5c18740 1 --2- 172.21.15.5:0/2360525771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7df60 0x55c190c8c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f5c18740 1 --2- 172.21.15.5:0/2360525771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 0x55c190c7da20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f5c18740 1 -- 172.21.15.5:0/2360525771 >> 172.21.15.5:0/2360525771 conn(0x55c190c79cf0 msgr2=0x55c190c7a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f5c18740 1 -- 172.21.15.5:0/2360525771 shutdown_connections 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.936+0000 7f70f5c18740 1 -- 172.21.15.5:0/2360525771 wait complete. 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f5c18740 1 -- start start 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f5c18740 1 Processor -- start 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 0x55c190c89b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190c8a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c8cd50 0x55c190c8a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f5c18740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c190c8f920 con 0x55c190c7df60 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f5c18740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c190b6b6e0 con 0x55c190c8cd50 2024-09-11T22:34:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f5c18740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c190c8faa0 con 0x55c190c7d640 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f5c18740 10 client.0 _ll_register_callbacks cb 0x55c190b8fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f0dda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190c8a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f0dda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190c8a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36846/0 (socket says 172.21.15.5:36846) 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f0dda640 1 -- 172.21.15.5:0/981178765 learned_addr learned my addr 172.21.15.5:0/981178765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f1ddc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 0x55c190c89b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.937+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c8cd50 0x55c190c8a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70f0dda640 1 -- 172.21.15.5:0/981178765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 msgr2=0x55c190c89b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70f0dda640 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 0x55c190c89b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70f0dda640 1 -- 172.21.15.5:0/981178765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c8cd50 msgr2=0x55c190c8a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70f0dda640 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c8cd50 0x55c190c8a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70f0dda640 1 -- 172.21.15.5:0/981178765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c190c98dd0 con 0x55c190c7df60 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70f0dda640 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190c8a060 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009900 tx=0x7f70dc004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dc91b13e53588a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70d4ff9640 1 -- 172.21.15.5:0/981178765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc035040 con 0x55c190c7df60 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70d4ff9640 1 -- 172.21.15.5:0/981178765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f70dc005450 con 0x55c190c7df60 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70f5c18740 1 -- 172.21.15.5:0/981178765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70e4004cc0 con 0x55c190c7df60 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70d4ff9640 1 -- 172.21.15.5:0/981178765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f70dc005650 con 0x55c190c7df60 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70f5c18740 10 client.19302 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.938+0000 7f70f5c18740 1 -- 172.21.15.5:0/981178765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c190c9b6c0 con 0x55c190c7df60 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70d4ff9640 1 -- 172.21.15.5:0/981178765 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e4004cc0 con 0x55c190c7df60 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70d4ff9640 1 -- 172.21.15.5:0/981178765 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f70dc003a30 con 0x55c190c7df60 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70f5c18740 10 client.19302 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70f5c18740 10 client.19302 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70f5c18740 20 client.19302 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70f5c18740 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c190c9cb80 0x55c190c9efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70f5c18740 4 client.19302 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70f5c18740 1 -- 172.21.15.5:0/981178765 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c190bf09d0 con 0x55c190c9cb80 2024-09-11T22:34:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c190c9cb80 0x55c190c9efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c190c9cb80 0x55c190c9efa0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99448d806276894 server_cookie=34c88a4360b93b74 in_seq=0 out_seq=0 2024-09-11T22:34:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.939+0000 7f70d4ff9640 10 client.19302 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.940+0000 7f70d4ff9640 1 -- 172.21.15.5:0/981178765 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c190bf09d0 con 0x55c190c9cb80 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.940+0000 7f70d4ff9640 10 client.19302 handle_command_reply: tid=0 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.940+0000 7f70f5c18740 1 client.19302 shutdown 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 1 -- 172.21.15.5:0/981178765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 msgr2=0x55c190c8a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190c8a060 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009900 tx=0x7f70dc004340 comp rx=0 tx=0).stop 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 1 -- 172.21.15.5:0/981178765 shutdown_connections 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c190c9cb80 0x55c190c9efa0 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c8cd50 0x55c190c8a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190c8a060 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 1 --2- 172.21.15.5:0/981178765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7d640 0x55c190c89b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 1 -- 172.21.15.5:0/981178765 >> 172.21.15.5:0/981178765 conn(0x55c190c79cf0 msgr2=0x55c190c7fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70d4ff9640 0 client.19302 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 1 -- 172.21.15.5:0/981178765 shutdown_connections 2024-09-11T22:34:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 1 -- 172.21.15.5:0/981178765 wait complete. 2024-09-11T22:34:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.941+0000 7f70f5c18740 20 client.19302 trim_cache size 0 max 16384 2024-09-11T22:34:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f5c18740 1 Processor -- start 2024-09-11T22:34:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f5c18740 1 -- start start 2024-09-11T22:34:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 0x55c190c8a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190ca4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c8cd50 0x55c190ca4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f5c18740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b4060 con 0x55c190c7df60 2024-09-11T22:34:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f5c18740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b5bd0 con 0x55c190c7d640 2024-09-11T22:34:33.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f5c18740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b4730 con 0x55c190c8cd50 2024-09-11T22:34:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f0dda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190ca4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f0dda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190ca4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36862/0 (socket says 172.21.15.5:36862) 2024-09-11T22:34:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f0dda640 1 -- 172.21.15.5:0/1271168217 learned_addr learned my addr 172.21.15.5:0/1271168217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:33.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f1ddc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 0x55c190c8a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.963+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/1271168217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c8cd50 0x55c190ca4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.964+0000 7f70f1ddc640 1 -- 172.21.15.5:0/1271168217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c8cd50 msgr2=0x55c190ca4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.964+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/1271168217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c8cd50 0x55c190ca4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.964+0000 7f70f1ddc640 1 -- 172.21.15.5:0/1271168217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 msgr2=0x55c190ca4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.964+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/1271168217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190ca4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.964+0000 7f70f1ddc640 1 -- 172.21.15.5:0/1271168217 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c190c75260 con 0x55c190c7d640 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.964+0000 7f70f0dda640 1 --2- 172.21.15.5:0/1271168217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190ca4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.964+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/1271168217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 0x55c190c8a3f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70e4011e30 tx=0x7f70e400ff10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47fa0915c0edf124 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.964+0000 7f70d6ffd640 1 -- 172.21.15.5:0/1271168217 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e402c030 con 0x55c190c7d640 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.964+0000 7f70d6ffd640 1 -- 172.21.15.5:0/1271168217 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e400fcf0 con 0x55c190c7d640 2024-09-11T22:34:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.964+0000 7f70d6ffd640 1 -- 172.21.15.5:0/1271168217 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e4005600 con 0x55c190c7d640 2024-09-11T22:34:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.965+0000 7f70f5c18740 1 -- 172.21.15.5:0/1271168217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 msgr2=0x55c190c8a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.965+0000 7f70f5c18740 1 --2- 172.21.15.5:0/1271168217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 0x55c190c8a3f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f70e4011e30 tx=0x7f70e400ff10 comp rx=0 tx=0).stop 2024-09-11T22:34:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.965+0000 7f70f5c18740 1 -- 172.21.15.5:0/1271168217 shutdown_connections 2024-09-11T22:34:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.965+0000 7f70f5c18740 1 --2- 172.21.15.5:0/1271168217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c8cd50 0x55c190ca4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.965+0000 7f70f5c18740 1 --2- 172.21.15.5:0/1271168217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c7df60 0x55c190ca4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.965+0000 7f70f5c18740 1 --2- 172.21.15.5:0/1271168217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 0x55c190c8a3f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.965+0000 7f70f5c18740 1 -- 172.21.15.5:0/1271168217 >> 172.21.15.5:0/1271168217 conn(0x55c190c79cf0 msgr2=0x55c190c8b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.965+0000 7f70f5c18740 1 -- 172.21.15.5:0/1271168217 shutdown_connections 2024-09-11T22:34:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.965+0000 7f70f5c18740 1 -- 172.21.15.5:0/1271168217 wait complete. 2024-09-11T22:34:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f5c18740 1 -- start start 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f5c18740 1 Processor -- start 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 0x55c190c94d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7df60 0x55c190c95280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f5c18740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c8cd50 0x55c190c8fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f5c18740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b5500 con 0x55c190c8cd50 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f5c18740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b4e30 con 0x55c190c7d640 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f5c18740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f70ec1b2060 con 0x55c190c7df60 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f0dda640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7df60 0x55c190c95280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f1ddc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 0x55c190c94d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f0dda640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7df60 0x55c190c95280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36118/0 (socket says 172.21.15.5:36118) 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.966+0000 7f70f0dda640 1 -- 172.21.15.5:0/794403628 learned_addr learned my addr 172.21.15.5:0/794403628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70f5c18740 10 client.0 _ll_register_callbacks cb 0x7f70dc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c8cd50 0x55c190c8fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70f0dda640 1 -- 172.21.15.5:0/794403628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 msgr2=0x55c190c94d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70f0dda640 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 0x55c190c94d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70f0dda640 1 -- 172.21.15.5:0/794403628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c8cd50 msgr2=0x55c190c8fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70f0dda640 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c8cd50 0x55c190c8fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70f0dda640 1 -- 172.21.15.5:0/794403628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c190c90230 con 0x55c190c7df60 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 0x55c190c94d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70f0dda640 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7df60 0x55c190c95280 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009900 tx=0x7f70dc032c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d59725ff962625ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70d4ff9640 1 -- 172.21.15.5:0/794403628 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70dc035040 con 0x55c190c7df60 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70d4ff9640 1 -- 172.21.15.5:0/794403628 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f70dc0037f0 con 0x55c190c7df60 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.967+0000 7f70f5c18740 1 -- 172.21.15.5:0/794403628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70e400fcf0 con 0x55c190c7df60 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70d4ff9640 1 -- 172.21.15.5:0/794403628 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f70dc02aa90 con 0x55c190c7df60 2024-09-11T22:34:33.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70f5c18740 10 client.28399 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70f5c18740 1 -- 172.21.15.5:0/794403628 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c190d07fa0 con 0x55c190c7df60 2024-09-11T22:34:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70d4ff9640 1 -- 172.21.15.5:0/794403628 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70dc041070 con 0x55c190c7df60 2024-09-11T22:34:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70d4ff9640 1 -- 172.21.15.5:0/794403628 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f70dc004ba0 con 0x55c190c7df60 2024-09-11T22:34:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70f5c18740 10 client.28399 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70f5c18740 10 client.28399 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:34:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70f5c18740 20 client.28399 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70f5c18740 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c190d08810 0x55c190d0ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:33.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70f5c18740 4 client.28399 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-11T22:34:33.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70f5c18740 1 -- 172.21.15.5:0/794403628 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x7f70e400fcf0 con 0x55c190d08810 2024-09-11T22:34:33.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.968+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c190d08810 0x55c190d0ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:33.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.969+0000 7f70d4ff9640 10 client.28399 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:33.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:33.969+0000 7f70f1ddc640 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c190d08810 0x55c190d0ac30 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff64aa1056237260 server_cookie=29bcdfbfb71859f7 in_seq=0 out_seq=0 2024-09-11T22:34:33.973 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:34.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70d4ff9640 1 -- 172.21.15.5:0/794403628 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7f70e400fcf0 con 0x55c190d08810 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70d4ff9640 10 client.28399 handle_command_reply: tid=0 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70f5c18740 1 client.28399 shutdown 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70f5c18740 1 -- 172.21.15.5:0/794403628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7df60 msgr2=0x55c190c95280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70f5c18740 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7df60 0x55c190c95280 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009900 tx=0x7f70dc032c30 comp rx=0 tx=0).stop 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70f5c18740 1 -- 172.21.15.5:0/794403628 shutdown_connections 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70f5c18740 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c190d08810 0x55c190d0ac30 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70f5c18740 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c190c8cd50 0x55c190c8fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70f5c18740 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c190c7df60 0x55c190c95280 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70f5c18740 1 --2- 172.21.15.5:0/794403628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c190c7d640 0x55c190c94d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70d4ff9640 0 client.28399 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:34.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.087+0000 7f70f5c18740 1 -- 172.21.15.5:0/794403628 >> 172.21.15.5:0/794403628 conn(0x55c190c79cf0 msgr2=0x55c190c78650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f5c18740 1 -- 172.21.15.5:0/794403628 shutdown_connections 2024-09-11T22:34:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f5c18740 1 -- 172.21.15.5:0/794403628 wait complete. 2024-09-11T22:34:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f5c18740 20 client.28399 trim_cache size 0 max 16384 2024-09-11T22:34:34.090 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:34:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 46, 2024-09-11T22:34:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:34:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 46, 2024-09-11T22:34:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:34:34.090 INFO:teuthology.orchestra.run.smithi005.stdout: "771318f": { 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 46, 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 147.2, 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:34:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:34:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:34:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:34:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:34:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:34:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:34:34.092 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:34:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 -- 172.21.15.5:0/946378600 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f70d80a2dc0 msgr2=0x7f70d80a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f70d80a2dc0 0x7f70d80a5280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f70e80cce70 tx=0x7f70e80d5550 comp rx=0 tx=0).stop 2024-09-11T22:34:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 -- 172.21.15.5:0/946378600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70ec101e10 msgr2=0x7f70ec073b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70ec101e10 0x7f70ec073b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f70e800e9f0 tx=0x7f70e800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:34:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 -- 172.21.15.5:0/946378600 shutdown_connections 2024-09-11T22:34:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70ec106f40 0x7f70ec06e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:34.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec1027c0 0x7f70ec06dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:34.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f70d80a2dc0 0x7f70d80a5280 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:34.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 --2- 172.21.15.5:0/946378600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f70ec101e10 0x7f70ec073b20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:34.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 -- 172.21.15.5:0/946378600 >> 172.21.15.5:0/946378600 conn(0x7f70ec0fd8d0 msgr2=0x7f70ec0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:34.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 -- 172.21.15.5:0/946378600 shutdown_connections 2024-09-11T22:34:34.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:34.088+0000 7f70f383f640 1 -- 172.21.15.5:0/946378600 wait complete. 2024-09-11T22:34:34.345 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.41 MB/sec execute 532 sec latency 263.085 ms 2024-09-11T22:34:35.026 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '771318f', quiesce took 4.7 seconds. Will release after: 22.5 2024-09-11T22:34:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.208+0000 7f5abb236640 1 -- 172.21.15.5:0/1963430377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab4102790 msgr2=0x7f5ab41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.208+0000 7f5abb236640 1 --2- 172.21.15.5:0/1963430377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab4102790 0x7f5ab41067e0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009900 tx=0x7f5aa002a3f0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.208+0000 7f5abb236640 1 -- 172.21.15.5:0/1963430377 shutdown_connections 2024-09-11T22:34:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.208+0000 7f5abb236640 1 --2- 172.21.15.5:0/1963430377 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab4106f10 0x7f5ab410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.208+0000 7f5abb236640 1 --2- 172.21.15.5:0/1963430377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab4102790 0x7f5ab41067e0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.208+0000 7f5abb236640 1 --2- 172.21.15.5:0/1963430377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab4101de0 0x7f5ab41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.208+0000 7f5abb236640 1 -- 172.21.15.5:0/1963430377 >> 172.21.15.5:0/1963430377 conn(0x7f5ab40fd8c0 msgr2=0x7f5ab40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.208+0000 7f5abb236640 1 -- 172.21.15.5:0/1963430377 shutdown_connections 2024-09-11T22:34:35.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.208+0000 7f5abb236640 1 -- 172.21.15.5:0/1963430377 wait complete. 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5abb236640 1 Processor -- start 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5abb236640 1 -- start start 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5abb236640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab4101de0 0x7f5ab419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5abb236640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab4102790 0x7f5ab419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5abb236640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab4106f10 0x7f5ab419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5abb236640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4111f70 con 0x7f5ab4101de0 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5abb236640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab4111df0 con 0x7f5ab4102790 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5abb236640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41120f0 con 0x7f5ab4106f10 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5ab8fd2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab4101de0 0x7f5ab419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5ab8fd2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab4101de0 0x7f5ab419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36902/0 (socket says 172.21.15.5:36902) 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.209+0000 7f5ab8fd2640 1 -- 172.21.15.5:0/2334247354 learned_addr learned my addr 172.21.15.5:0/2334247354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab3fff640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab4102790 0x7f5ab419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab4106f10 0x7f5ab419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab8fd2640 1 -- 172.21.15.5:0/2334247354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab4106f10 msgr2=0x7f5ab419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab4106f10 0x7f5ab419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab8fd2640 1 -- 172.21.15.5:0/2334247354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab4102790 msgr2=0x7f5ab419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab4102790 0x7f5ab419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab8fd2640 1 -- 172.21.15.5:0/2334247354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab419f570 con 0x7f5ab4101de0 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab4106f10 0x7f5ab419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab4101de0 0x7f5ab419a5c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800d950 tx=0x7f5aa800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab1ffb640 1 -- 172.21.15.5:0/2334247354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8017070 con 0x7f5ab4101de0 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab1ffb640 1 -- 172.21.15.5:0/2334247354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8009c30 con 0x7f5ab4101de0 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5abb236640 1 -- 172.21.15.5:0/2334247354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab419f860 con 0x7f5ab4101de0 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5ab1ffb640 1 -- 172.21.15.5:0/2334247354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa800f6b0 con 0x7f5ab4101de0 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.210+0000 7f5abb236640 1 -- 172.21.15.5:0/2334247354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab41a7140 con 0x7f5ab4101de0 2024-09-11T22:34:35.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.212+0000 7f5ab1ffb640 1 -- 172.21.15.5:0/2334247354 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f5aa80109d0 con 0x7f5ab4101de0 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5abd60f740 1 Processor -- start 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5abd60f740 1 -- start start 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5ab1ffb640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5aa40a2d70 0x7f5aa40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cf3c0 0x55b0c99cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cfce0 0x55b0c99e2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5ab1ffb640 1 -- 172.21.15.5:0/2334247354 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f5aa80cf160 con 0x7f5ab4101de0 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5aa40a2d70 0x7f5aa40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5abd60f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b0c99352e0 con 0x55b0c99e2e90 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5abd60f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b0c98dcce0 con 0x55b0c99cf3c0 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5abd60f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b0c99312a0 con 0x55b0c99cfce0 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5ab97d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.213+0000 7f5ab97d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36906/0 (socket says 172.21.15.5:36906) 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1281827574 learned_addr learned my addr 172.21.15.5:0/1281827574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab3fff640 1 --2- 172.21.15.5:0/1281827574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cfce0 0x55b0c99e2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/1281827574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cf3c0 0x55b0c99cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5aa40a2d70 0x7f5aa40a5230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5aac008fd0 tx=0x7f5aac014ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1281827574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cfce0 msgr2=0x55b0c99e2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/1281827574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cfce0 0x55b0c99e2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1281827574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cf3c0 msgr2=0x55b0c99cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/1281827574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cf3c0 0x55b0c99cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1281827574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0c98c1440 con 0x55b0c99e2e90 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/1281827574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cf3c0 0x55b0c99cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/1281827574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e5280 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f5aac008bc0 tx=0x7f5aac01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1187b1e184af37 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5a9effd640 1 -- 172.21.15.5:0/1281827574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac023070 con 0x55b0c99e2e90 2024-09-11T22:34:35.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5a9effd640 1 -- 172.21.15.5:0/1281827574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aac01e450 con 0x55b0c99e2e90 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.214+0000 7f5a9effd640 1 -- 172.21.15.5:0/1281827574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac011e60 con 0x55b0c99e2e90 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 -- 172.21.15.5:0/1281827574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 msgr2=0x55b0c99e5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1281827574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e5280 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f5aac008bc0 tx=0x7f5aac01b000 comp rx=0 tx=0).stop 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 -- 172.21.15.5:0/1281827574 shutdown_connections 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1281827574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e5280 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1281827574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cfce0 0x55b0c99e2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1281827574 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cf3c0 0x55b0c99cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 -- 172.21.15.5:0/1281827574 >> 172.21.15.5:0/1281827574 conn(0x55b0c99cba30 msgr2=0x55b0c99cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 -- 172.21.15.5:0/1281827574 shutdown_connections 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 -- 172.21.15.5:0/1281827574 wait complete. 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 -- start start 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.215+0000 7f5abd60f740 1 Processor -- start 2024-09-11T22:34:35.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99e2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 0x55b0c99dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5abd60f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b0c99e59a0 con 0x55b0c99e2e90 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5abd60f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b0c98f0050 con 0x55b0c99cfce0 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5abd60f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b0c99e5b20 con 0x55b0c99cf3c0 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5abd60f740 10 client.0 _ll_register_callbacks cb 0x55b0c9930160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5ab97d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 0x55b0c99dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5ab97d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36918/0 (socket says 172.21.15.5:36918) 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1635331440 learned_addr learned my addr 172.21.15.5:0/1635331440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1635331440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 msgr2=0x55b0c99e2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99e2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1635331440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 msgr2=0x55b0c99dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 0x55b0c99dfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.216+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1635331440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0c96a63c0 con 0x55b0c99e2e90 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e01d0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f5aac01ee10 tx=0x7f5aac004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cc84d53d1b2686 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5a93fff640 1 -- 172.21.15.5:0/1635331440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac02c060 con 0x55b0c99e2e90 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5a93fff640 1 -- 172.21.15.5:0/1635331440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f5aac00f7e0 con 0x55b0c99e2e90 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5abd60f740 1 -- 172.21.15.5:0/1635331440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aac01e450 con 0x55b0c99e2e90 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5a93fff640 1 -- 172.21.15.5:0/1635331440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5aac023070 con 0x55b0c99e2e90 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5abd60f740 10 client.19320 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5abd60f740 1 -- 172.21.15.5:0/1635331440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b0c99f1470 con 0x55b0c99e2e90 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5a93fff640 1 -- 172.21.15.5:0/1635331440 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aac01e450 con 0x55b0c99e2e90 2024-09-11T22:34:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5a93fff640 1 -- 172.21.15.5:0/1635331440 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f5aac005370 con 0x55b0c99e2e90 2024-09-11T22:34:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5abd60f740 10 client.19320 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5abd60f740 10 client.19320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5abd60f740 20 client.19320 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b0c99f2880 0x55b0c99f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5abd60f740 4 client.19320 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.217+0000 7f5abd60f740 1 -- 172.21.15.5:0/1635331440 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0c9942850 con 0x55b0c99f2880 2024-09-11T22:34:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.218+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b0c99f2880 0x55b0c99f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.218+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b0c99f2880 0x55b0c99f4c70 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8452ba4a957dd7f server_cookie=c9940c5b07218290 in_seq=0 out_seq=0 2024-09-11T22:34:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.218+0000 7f5a93fff640 10 client.19320 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5a93fff640 1 -- 172.21.15.5:0/1635331440 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0c9942850 con 0x55b0c99f2880 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5a93fff640 10 client.19320 handle_command_reply: tid=0 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 client.19320 shutdown 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 -- 172.21.15.5:0/1635331440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 msgr2=0x55b0c99e01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e01d0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f5aac01ee10 tx=0x7f5aac004f80 comp rx=0 tx=0).stop 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 -- 172.21.15.5:0/1635331440 shutdown_connections 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b0c99f2880 0x55b0c99f4c70 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99e01d0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 0x55b0c99dfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5a93fff640 0 client.19320 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1635331440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99e2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 -- 172.21.15.5:0/1635331440 >> 172.21.15.5:0/1635331440 conn(0x55b0c99cba30 msgr2=0x55b0c99d16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 -- 172.21.15.5:0/1635331440 shutdown_connections 2024-09-11T22:34:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 1 -- 172.21.15.5:0/1635331440 wait complete. 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.219+0000 7f5abd60f740 20 client.19320 trim_cache size 0 max 16384 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.236+0000 7f5abd60f740 1 Processor -- start 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.236+0000 7f5abd60f740 1 -- start start 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99eedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99cfce0 0x55b0c99f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99e2e90 0x55b0c99f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5abd60f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41ab8d0 con 0x55b0c99cfce0 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5abd60f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41ad440 con 0x55b0c99e2e90 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5abd60f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41abfa0 con 0x55b0c99cf3c0 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99cfce0 0x55b0c99f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99cfce0 0x55b0c99f7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36930/0 (socket says 172.21.15.5:36930) 2024-09-11T22:34:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab3fff640 1 -- 172.21.15.5:0/178747351 learned_addr learned my addr 172.21.15.5:0/178747351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/178747351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99e2e90 0x55b0c99f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab3fff640 1 -- 172.21.15.5:0/178747351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 msgr2=0x55b0c99eedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab3fff640 1 --2- 172.21.15.5:0/178747351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99eedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab3fff640 1 -- 172.21.15.5:0/178747351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99e2e90 msgr2=0x55b0c99f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab3fff640 1 --2- 172.21.15.5:0/178747351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99e2e90 0x55b0c99f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab3fff640 1 -- 172.21.15.5:0/178747351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0c99de0c0 con 0x55b0c99cfce0 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/178747351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99e2e90 0x55b0c99f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5ab3fff640 1 --2- 172.21.15.5:0/178747351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99cfce0 0x55b0c99f7c90 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f5aa002a910 tx=0x7f5aa0032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00ec3ca32785c4b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5a9e7fc640 1 -- 172.21.15.5:0/178747351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0035040 con 0x55b0c99cfce0 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5a9e7fc640 1 -- 172.21.15.5:0/178747351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa0031830 con 0x55b0c99cfce0 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.237+0000 7f5a9e7fc640 1 -- 172.21.15.5:0/178747351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00048c0 con 0x55b0c99cfce0 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 -- 172.21.15.5:0/178747351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99cfce0 msgr2=0x55b0c99f7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 --2- 172.21.15.5:0/178747351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99cfce0 0x55b0c99f7c90 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f5aa002a910 tx=0x7f5aa0032f00 comp rx=0 tx=0).stop 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 -- 172.21.15.5:0/178747351 shutdown_connections 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 --2- 172.21.15.5:0/178747351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99e2e90 0x55b0c99f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 --2- 172.21.15.5:0/178747351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99cfce0 0x55b0c99f7c90 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 --2- 172.21.15.5:0/178747351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99eedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 -- 172.21.15.5:0/178747351 >> 172.21.15.5:0/178747351 conn(0x55b0c99cba30 msgr2=0x55b0c99cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 -- 172.21.15.5:0/178747351 shutdown_connections 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 -- 172.21.15.5:0/178747351 wait complete. 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 -- start start 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.238+0000 7f5abd60f740 1 Processor -- start 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 0x55b0c99f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5abd60f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99f6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5abd60f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41acd70 con 0x55b0c99e2e90 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5abd60f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41ac6a0 con 0x55b0c99cfce0 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5abd60f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab41a9910 con 0x55b0c99cf3c0 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5abd60f740 10 client.0 _ll_register_callbacks cb 0x7f5aac021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab97d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab97d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99f6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36946/0 (socket says 172.21.15.5:36946) 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1993448988 learned_addr learned my addr 172.21.15.5:0/1993448988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab3fff640 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 0x55b0c99f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1993448988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 msgr2=0x55b0c99f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1993448988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 msgr2=0x55b0c99f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 0x55b0c99f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab97d3640 1 -- 172.21.15.5:0/1993448988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0c9a5c410 con 0x55b0c99e2e90 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab3fff640 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 0x55b0c99f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5ab97d3640 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99f6700 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f5aac008660 tx=0x7f5aac0082b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873a1df3d4229923 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.239+0000 7f5a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5a93fff640 1 -- 172.21.15.5:0/1993448988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac02c070 con 0x55b0c99e2e90 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5a93fff640 1 -- 172.21.15.5:0/1993448988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f5aac005410 con 0x55b0c99e2e90 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5abd60f740 1 -- 172.21.15.5:0/1993448988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa0031830 con 0x55b0c99e2e90 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5a93fff640 1 -- 172.21.15.5:0/1993448988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5aac008c50 con 0x55b0c99e2e90 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5abd60f740 10 client.19326 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5abd60f740 1 -- 172.21.15.5:0/1993448988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b0c9a5d690 con 0x55b0c99e2e90 2024-09-11T22:34:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5a93fff640 1 -- 172.21.15.5:0/1993448988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa0031830 con 0x55b0c99e2e90 2024-09-11T22:34:35.267 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:34:35.267 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:34:35.267 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:34:35.267 INFO:teuthology.orchestra.run.smithi005.stdout: "133b49b4-3a55-4acd-8fc2-af33f4b5ce0a": { 2024-09-11T22:34:35.267 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:34:35.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "133b49b4-3a55-4acd-8fc2-af33f4b5ce0a", 2024-09-11T22:34:35.267 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:34:35.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:34:35.267 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:34:35.267 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5a93fff640 1 -- 172.21.15.5:0/1993448988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f5aac036020 con 0x55b0c99e2e90 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5abd60f740 10 client.19326 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5abd60f740 10 client.19326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5abd60f740 20 client.19326 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b0c9a5dd50 0x55b0c9a60140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5abd60f740 4 client.19326 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5abd60f740 1 -- 172.21.15.5:0/1993448988 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0c99cb300 con 0x55b0c9a5dd50 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.240+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b0c9a5dd50 0x55b0c9a60140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5a93fff640 10 client.19326 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5ab8fd2640 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b0c9a5dd50 0x55b0c9a60140 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7234679dd780ab server_cookie=7bf48871f68559f in_seq=0 out_seq=0 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5a93fff640 1 -- 172.21.15.5:0/1993448988 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b0c99cb300 con 0x55b0c9a5dd50 2024-09-11T22:34:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5a93fff640 10 client.19326 handle_command_reply: tid=0 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 client.19326 shutdown 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 -- 172.21.15.5:0/1993448988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 msgr2=0x55b0c99f6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99f6700 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f5aac008660 tx=0x7f5aac0082b0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 -- 172.21.15.5:0/1993448988 shutdown_connections 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b0c9a5dd50 0x55b0c9a60140 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0c99e2e90 0x55b0c99f6700 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5a93fff640 0 client.19326 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0c99cfce0 0x55b0c99f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 --2- 172.21.15.5:0/1993448988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0c99cf3c0 0x55b0c99f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 -- 172.21.15.5:0/1993448988 >> 172.21.15.5:0/1993448988 conn(0x55b0c99cba30 msgr2=0x55b0c99e13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 -- 172.21.15.5:0/1993448988 shutdown_connections 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.241+0000 7f5abd60f740 1 -- 172.21.15.5:0/1993448988 wait complete. 2024-09-11T22:34:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abd60f740 20 client.19326 trim_cache size 0 max 16384 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 -- 172.21.15.5:0/2334247354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5aa40a2d70 msgr2=0x7f5aa40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5aa40a2d70 0x7f5aa40a5230 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5aac008fd0 tx=0x7f5aac014ab0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 -- 172.21.15.5:0/2334247354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab4101de0 msgr2=0x7f5ab419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab4101de0 0x7f5ab419a5c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800d950 tx=0x7f5aa800de10 comp rx=0 tx=0).stop 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 -- 172.21.15.5:0/2334247354 shutdown_connections 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ab4106f10 0x7f5ab419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5aa40a2d70 0x7f5aa40a5230 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ab4102790 0x7f5ab419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 --2- 172.21.15.5:0/2334247354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ab4101de0 0x7f5ab419a5c0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 -- 172.21.15.5:0/2334247354 >> 172.21.15.5:0/2334247354 conn(0x7f5ab40fd8c0 msgr2=0x7f5ab40fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 -- 172.21.15.5:0/2334247354 shutdown_connections 2024-09-11T22:34:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:35.242+0000 7f5abb236640 1 -- 172.21.15.5:0/2334247354 wait complete. 2024-09-11T22:34:35.346 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.40 MB/sec execute 533 sec latency 1263.165 ms 2024-09-11T22:34:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:35 smithi005 ceph-mon[30943]: pgmap v393: 97 pgs: 97 active+clean; 90 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-09-11T22:34:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:35 smithi159 ceph-mon[35534]: pgmap v393: 97 pgs: 97 active+clean; 90 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-09-11T22:34:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:35 smithi172 ceph-mon[35461]: pgmap v393: 97 pgs: 97 active+clean; 90 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-09-11T22:34:36.065 INFO:tasks.cephfs.filesystem:scrub status for tag:133b49b4-3a55-4acd-8fc2-af33f4b5ce0a - {'path': '/', 'tag': '133b49b4-3a55-4acd-8fc2-af33f4b5ce0a', 'options': 'recursive,force'} 2024-09-11T22:34:36.346 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.38 MB/sec execute 534 sec latency 2263.277 ms 2024-09-11T22:34:36.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:34:36.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:34:36.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:34:36.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:36 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:34:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:34:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:34:36.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:36 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:34:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:34:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:34:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:36 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:34:37.346 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.37 MB/sec execute 535 sec latency 3263.388 ms 2024-09-11T22:34:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.569+0000 7f783bbd8640 1 -- 172.21.15.5:0/528456437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7834106a00 msgr2=0x7f7834106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.569+0000 7f783bbd8640 1 --2- 172.21.15.5:0/528456437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7834106a00 0x7f7834106de0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7830009930 tx=0x7f783002a420 comp rx=0 tx=0).stop 2024-09-11T22:34:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.569+0000 7f783bbd8640 1 -- 172.21.15.5:0/528456437 shutdown_connections 2024-09-11T22:34:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.569+0000 7f783bbd8640 1 --2- 172.21.15.5:0/528456437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7834101390 0x7f783410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.569+0000 7f783bbd8640 1 --2- 172.21.15.5:0/528456437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78341009f0 0x7f7834100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.569+0000 7f783bbd8640 1 --2- 172.21.15.5:0/528456437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7834106a00 0x7f7834106de0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.569+0000 7f783bbd8640 1 -- 172.21.15.5:0/528456437 >> 172.21.15.5:0/528456437 conn(0x7f78340fc6b0 msgr2=0x7f78340fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.569+0000 7f783bbd8640 1 -- 172.21.15.5:0/528456437 shutdown_connections 2024-09-11T22:34:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.569+0000 7f783bbd8640 1 -- 172.21.15.5:0/528456437 wait complete. 2024-09-11T22:34:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783bbd8640 1 Processor -- start 2024-09-11T22:34:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783bbd8640 1 -- start start 2024-09-11T22:34:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783bbd8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78341009f0 0x7f78340fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783bbd8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7834101390 0x7f78340fd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783bbd8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7834106a00 0x7f78340fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783bbd8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7834110e40 con 0x7f7834106a00 2024-09-11T22:34:37.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783bbd8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7834110cc0 con 0x7f7834101390 2024-09-11T22:34:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783bbd8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7834110fc0 con 0x7f78341009f0 2024-09-11T22:34:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783a175640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7834106a00 0x7f78340fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783a175640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7834106a00 0x7f78340fd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36958/0 (socket says 172.21.15.5:36958) 2024-09-11T22:34:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783a175640 1 -- 172.21.15.5:0/3317811520 learned_addr learned my addr 172.21.15.5:0/3317811520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f7839173640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7834101390 0x7f78340fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f7839974640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78341009f0 0x7f78340fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783a175640 1 -- 172.21.15.5:0/3317811520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78341009f0 msgr2=0x7f78340fcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:37.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783a175640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78341009f0 0x7f78340fcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.570+0000 7f783a175640 1 -- 172.21.15.5:0/3317811520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7834101390 msgr2=0x7f78340fd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.571+0000 7f783a175640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7834101390 0x7f78340fd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.571+0000 7f783a175640 1 -- 172.21.15.5:0/3317811520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78341a6860 con 0x7f7834106a00 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.571+0000 7f7839974640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78341009f0 0x7f78340fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.571+0000 7f7839173640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7834101390 0x7f78340fd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.571+0000 7f783a175640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7834106a00 0x7f78340fd6d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f782c00cce0 tx=0x7f782c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.571+0000 7f782affd640 1 -- 172.21.15.5:0/3317811520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c00f450 con 0x7f7834106a00 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.571+0000 7f782affd640 1 -- 172.21.15.5:0/3317811520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782c00fad0 con 0x7f7834106a00 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.571+0000 7f783bbd8640 1 -- 172.21.15.5:0/3317811520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78341a6b50 con 0x7f7834106a00 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.571+0000 7f782affd640 1 -- 172.21.15.5:0/3317811520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c018c10 con 0x7f7834106a00 2024-09-11T22:34:37.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.571+0000 7f783bbd8640 1 -- 172.21.15.5:0/3317811520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78341a70a0 con 0x7f7834106a00 2024-09-11T22:34:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f782affd640 1 -- 172.21.15.5:0/3317811520 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f782c00f5f0 con 0x7f7834106a00 2024-09-11T22:34:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f783ea48740 1 Processor -- start 2024-09-11T22:34:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f783ea48740 1 -- start start 2024-09-11T22:34:37.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f783ea48740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f783ea48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f783ea48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f783ea48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd9162f0 con 0x55dcdd9b2410 2024-09-11T22:34:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f783ea48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd8be270 con 0x55dcdd9bccb0 2024-09-11T22:34:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f783ea48740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd9122b0 con 0x55dcdd9b1af0 2024-09-11T22:34:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f7839974640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f7839974640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9bc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36968/0 (socket says 172.21.15.5:36968) 2024-09-11T22:34:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f7839974640 1 -- 172.21.15.5:0/3182928833 learned_addr learned my addr 172.21.15.5:0/3182928833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.574+0000 7f7839173640 1 --2- 172.21.15.5:0/3182928833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f7839173640 1 --2- 172.21.15.5:0/3182928833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f7839974640 1 -- 172.21.15.5:0/3182928833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9b1af0 msgr2=0x55dcdd9b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f7839974640 1 --2- 172.21.15.5:0/3182928833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9b1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f7839974640 1 -- 172.21.15.5:0/3182928833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9bccb0 msgr2=0x55dcdd9bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f7839974640 1 --2- 172.21.15.5:0/3182928833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f7839974640 1 -- 172.21.15.5:0/3182928833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcdd8bd340 con 0x55dcdd9b2410 2024-09-11T22:34:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f782affd640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78200a2e20 0x7f78200a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f7839173640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78200a2e20 0x7f78200a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f7839974640 1 --2- 172.21.15.5:0/3182928833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9bc770 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f7830005ab0 tx=0x7f7830009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fca1120572e6038 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f782affd640 1 -- 172.21.15.5:0/3317811520 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f782c0cfbb0 con 0x7f7834106a00 2024-09-11T22:34:37.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f781ffff640 1 -- 172.21.15.5:0/3182928833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783003f070 con 0x55dcdd9b2410 2024-09-11T22:34:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f781ffff640 1 -- 172.21.15.5:0/3182928833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78300317b0 con 0x55dcdd9b2410 2024-09-11T22:34:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.575+0000 7f781ffff640 1 -- 172.21.15.5:0/3182928833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7830035070 con 0x55dcdd9b2410 2024-09-11T22:34:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.576+0000 7f783ea48740 1 -- 172.21.15.5:0/3182928833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 msgr2=0x55dcdd9bc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.576+0000 7f7839173640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78200a2e20 0x7f78200a52e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55dcdd9af3e0 tx=0x7f782400a2b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.576+0000 7f783ea48740 1 --2- 172.21.15.5:0/3182928833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9bc770 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f7830005ab0 tx=0x7f7830009680 comp rx=0 tx=0).stop 2024-09-11T22:34:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.576+0000 7f783ea48740 1 -- 172.21.15.5:0/3182928833 shutdown_connections 2024-09-11T22:34:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.576+0000 7f783ea48740 1 --2- 172.21.15.5:0/3182928833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.576+0000 7f783ea48740 1 --2- 172.21.15.5:0/3182928833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9bc770 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.576+0000 7f783ea48740 1 --2- 172.21.15.5:0/3182928833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9b1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.576+0000 7f783ea48740 1 -- 172.21.15.5:0/3182928833 >> 172.21.15.5:0/3182928833 conn(0x55dcdd9ae060 msgr2=0x55dcdd9ae440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.576+0000 7f783ea48740 1 -- 172.21.15.5:0/3182928833 shutdown_connections 2024-09-11T22:34:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.576+0000 7f783ea48740 1 -- 172.21.15.5:0/3182928833 wait complete. 2024-09-11T22:34:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f783ea48740 1 -- start start 2024-09-11T22:34:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f783ea48740 1 Processor -- start 2024-09-11T22:34:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f783ea48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9b9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f783ea48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b2410 0x55dcdd9ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f783ea48740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9ba540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f783ea48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd9bf880 con 0x55dcdd9b1af0 2024-09-11T22:34:37.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f783ea48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd89f290 con 0x55dcdd9b2410 2024-09-11T22:34:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f783ea48740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dcdd9bfa00 con 0x55dcdd9bccb0 2024-09-11T22:34:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f783ea48740 10 client.0 _ll_register_callbacks cb 0x55dcdd8d0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f7839974640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9b9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f7839974640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9b9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36974/0 (socket says 172.21.15.5:36974) 2024-09-11T22:34:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f7839974640 1 -- 172.21.15.5:0/104681838 learned_addr learned my addr 172.21.15.5:0/104681838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f7839173640 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b2410 0x55dcdd9ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.577+0000 7f783a175640 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f7839974640 1 -- 172.21.15.5:0/104681838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 msgr2=0x55dcdd9ba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f7839974640 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9ba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f7839974640 1 -- 172.21.15.5:0/104681838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b2410 msgr2=0x55dcdd9ba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f7839974640 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b2410 0x55dcdd9ba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f7839974640 1 -- 172.21.15.5:0/104681838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcdd629210 con 0x55dcdd9b1af0 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f7839974640 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9b9ac0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f7830031550 tx=0x7f7830005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff423cf4ae5ad5ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f781d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f781d7fa640 1 -- 172.21.15.5:0/104681838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783003f050 con 0x55dcdd9b1af0 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f781d7fa640 1 -- 172.21.15.5:0/104681838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f7830004240 con 0x55dcdd9b1af0 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f783ea48740 1 -- 172.21.15.5:0/104681838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78300317b0 con 0x55dcdd9b1af0 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f781d7fa640 1 -- 172.21.15.5:0/104681838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f7830035040 con 0x55dcdd9b1af0 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f783ea48740 10 client.19338 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f783ea48740 1 -- 172.21.15.5:0/104681838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55dcdd6f5620 con 0x55dcdd9b1af0 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.578+0000 7f781d7fa640 1 -- 172.21.15.5:0/104681838 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78300317b0 con 0x55dcdd9b1af0 2024-09-11T22:34:37.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.579+0000 7f781d7fa640 1 -- 172.21.15.5:0/104681838 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f7830004850 con 0x55dcdd9b1af0 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.579+0000 7f783ea48740 10 client.19338 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.579+0000 7f783ea48740 10 client.19338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.579+0000 7f783ea48740 20 client.19338 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.579+0000 7f783ea48740 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55dcdd9cca10 0x55dcdd9cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.579+0000 7f783ea48740 4 client.19338 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.579+0000 7f783ea48740 1 -- 172.21.15.5:0/104681838 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcdd923a20 con 0x55dcdd9cca10 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.579+0000 7f7839173640 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55dcdd9cca10 0x55dcdd9cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.579+0000 7f781d7fa640 10 client.19338 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.579+0000 7f7839173640 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55dcdd9cca10 0x55dcdd9cee00 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bf83fb11e5ba2bf server_cookie=484298aef763004e in_seq=0 out_seq=0 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f781d7fa640 1 -- 172.21.15.5:0/104681838 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcdd923a20 con 0x55dcdd9cca10 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f781d7fa640 10 client.19338 handle_command_reply: tid=0 2024-09-11T22:34:37.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f783ea48740 1 client.19338 shutdown 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f783ea48740 1 -- 172.21.15.5:0/104681838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b1af0 msgr2=0x55dcdd9b9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f783ea48740 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9b9ac0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f7830031550 tx=0x7f7830005340 comp rx=0 tx=0).stop 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f783ea48740 1 -- 172.21.15.5:0/104681838 shutdown_connections 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f783ea48740 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55dcdd9cca10 0x55dcdd9cee00 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f783ea48740 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f783ea48740 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b2410 0x55dcdd9ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f783ea48740 1 --2- 172.21.15.5:0/104681838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9b9ac0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f781d7fa640 0 client.19338 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f783ea48740 1 -- 172.21.15.5:0/104681838 >> 172.21.15.5:0/104681838 conn(0x55dcdd9ae060 msgr2=0x55dcdd9b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.580+0000 7f783ea48740 1 -- 172.21.15.5:0/104681838 shutdown_connections 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.581+0000 7f783ea48740 1 -- 172.21.15.5:0/104681838 wait complete. 2024-09-11T22:34:37.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.581+0000 7f783ea48740 20 client.19338 trim_cache size 0 max 16384 2024-09-11T22:34:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783ea48740 1 Processor -- start 2024-09-11T22:34:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783ea48740 1 -- start start 2024-09-11T22:34:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783ea48740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9ba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783ea48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b2410 0x55dcdd9d4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783ea48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783ea48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78341afb40 con 0x55dcdd9bccb0 2024-09-11T22:34:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783ea48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f78341b16b0 con 0x55dcdd9b2410 2024-09-11T22:34:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783ea48740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f78341b0210 con 0x55dcdd9b1af0 2024-09-11T22:34:37.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783a175640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783a175640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9d49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36978/0 (socket says 172.21.15.5:36978) 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f783a175640 1 -- 172.21.15.5:0/1811580925 learned_addr learned my addr 172.21.15.5:0/1811580925 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.595+0000 7f7839173640 1 --2- 172.21.15.5:0/1811580925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b2410 0x55dcdd9d4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.596+0000 7f783a175640 1 -- 172.21.15.5:0/1811580925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9b1af0 msgr2=0x55dcdd9ba370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.596+0000 7f783a175640 1 --2- 172.21.15.5:0/1811580925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9ba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.596+0000 7f783a175640 1 -- 172.21.15.5:0/1811580925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b2410 msgr2=0x55dcdd9d4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.596+0000 7f783a175640 1 --2- 172.21.15.5:0/1811580925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b2410 0x55dcdd9d4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.596+0000 7f783a175640 1 -- 172.21.15.5:0/1811580925 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcdd9ade90 con 0x55dcdd9bccb0 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.596+0000 7f783a175640 1 --2- 172.21.15.5:0/1811580925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9d49d0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f782c018550 tx=0x7f782c018400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213ed7d1449afad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.596+0000 7f781f7fe640 1 -- 172.21.15.5:0/1811580925 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c0b5020 con 0x55dcdd9bccb0 2024-09-11T22:34:37.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.596+0000 7f781f7fe640 1 -- 172.21.15.5:0/1811580925 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782c0a0410 con 0x55dcdd9bccb0 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.596+0000 7f781f7fe640 1 -- 172.21.15.5:0/1811580925 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c00fc80 con 0x55dcdd9bccb0 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 -- 172.21.15.5:0/1811580925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9bccb0 msgr2=0x55dcdd9d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 --2- 172.21.15.5:0/1811580925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9d49d0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f782c018550 tx=0x7f782c018400 comp rx=0 tx=0).stop 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 -- 172.21.15.5:0/1811580925 shutdown_connections 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 --2- 172.21.15.5:0/1811580925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9d49d0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 --2- 172.21.15.5:0/1811580925 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b2410 0x55dcdd9d4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 --2- 172.21.15.5:0/1811580925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9ba370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 -- 172.21.15.5:0/1811580925 >> 172.21.15.5:0/1811580925 conn(0x55dcdd9ae060 msgr2=0x55dcdd9bbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 -- 172.21.15.5:0/1811580925 shutdown_connections 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 -- 172.21.15.5:0/1811580925 wait complete. 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 -- start start 2024-09-11T22:34:37.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 Processor -- start 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9bfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78341b0fe0 con 0x55dcdd9b2410 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f78341b0910 con 0x55dcdd9b1af0 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f78341adb40 con 0x55dcdd9bccb0 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.597+0000 7f783ea48740 10 client.0 _ll_register_callbacks cb 0x7f782400a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f7839173640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f7839173640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9c5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36994/0 (socket says 172.21.15.5:36994) 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f7839173640 1 -- 172.21.15.5:0/2588453920 learned_addr learned my addr 172.21.15.5:0/2588453920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:37.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f7839974640 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f783a175640 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9bfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f7839173640 1 -- 172.21.15.5:0/2588453920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 msgr2=0x55dcdd9bfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f7839173640 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9bfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f7839173640 1 -- 172.21.15.5:0/2588453920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b1af0 msgr2=0x55dcdd9c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f7839173640 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f7839173640 1 -- 172.21.15.5:0/2588453920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcdd9c01f0 con 0x55dcdd9b2410 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f783a175640 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9bfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f7839173640 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9c5240 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x55dcdd9b4690 tx=0x7f78240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96aab402e7a4d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f781d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f781d7fa640 1 -- 172.21.15.5:0/2588453920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824031020 con 0x55dcdd9b2410 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f781d7fa640 1 -- 172.21.15.5:0/2588453920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f7824025410 con 0x55dcdd9b2410 2024-09-11T22:34:37.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f783ea48740 1 -- 172.21.15.5:0/2588453920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f782c0a0410 con 0x55dcdd9b2410 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f781d7fa640 1 -- 172.21.15.5:0/2588453920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f782403c050 con 0x55dcdd9b2410 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f783ea48740 10 client.19344 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.598+0000 7f783ea48740 1 -- 172.21.15.5:0/2588453920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55dcdda37c80 con 0x55dcdd9b2410 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.599+0000 7f781d7fa640 1 -- 172.21.15.5:0/2588453920 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782c0a0410 con 0x55dcdd9b2410 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.599+0000 7f781d7fa640 1 -- 172.21.15.5:0/2588453920 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f7824015040 con 0x55dcdd9b2410 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.599+0000 7f783ea48740 10 client.19344 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.599+0000 7f783ea48740 10 client.19344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.599+0000 7f783ea48740 20 client.19344 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.599+0000 7f783ea48740 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55dcdda38430 0x55dcdda3a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.599+0000 7f783ea48740 4 client.19344 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:34:37.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.599+0000 7f783ea48740 1 -- 172.21.15.5:0/2588453920 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dcdd91dd70 con 0x55dcdda38430 2024-09-11T22:34:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.599+0000 7f7839974640 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55dcdda38430 0x55dcdda3a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.600+0000 7f7839974640 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55dcdda38430 0x55dcdda3a850 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c070eb76e92509a3 server_cookie=bc2afd52e4118132 in_seq=0 out_seq=0 2024-09-11T22:34:37.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.600+0000 7f781d7fa640 10 client.19344 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f781d7fa640 1 -- 172.21.15.5:0/2588453920 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dcdd91dd70 con 0x55dcdda38430 2024-09-11T22:34:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f781d7fa640 10 client.19344 handle_command_reply: tid=0 2024-09-11T22:34:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 client.19344 shutdown 2024-09-11T22:34:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 -- 172.21.15.5:0/2588453920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 msgr2=0x55dcdd9c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9c5240 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x55dcdd9b4690 tx=0x7f78240046c0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 -- 172.21.15.5:0/2588453920 shutdown_connections 2024-09-11T22:34:37.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55dcdda38430 0x55dcdda3a850 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dcdd9bccb0 0x55dcdd9bfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcdd9b2410 0x55dcdd9c5240 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 --2- 172.21.15.5:0/2588453920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dcdd9b1af0 0x55dcdd9c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f781d7fa640 0 client.19344 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 -- 172.21.15.5:0/2588453920 >> 172.21.15.5:0/2588453920 conn(0x55dcdd9ae060 msgr2=0x55dcdd9ac9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 -- 172.21.15.5:0/2588453920 shutdown_connections 2024-09-11T22:34:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 1 -- 172.21.15.5:0/2588453920 wait complete. 2024-09-11T22:34:37.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783ea48740 20 client.19344 trim_cache size 0 max 16384 2024-09-11T22:34:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:34:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:34:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:34:37.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:34:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 -- 172.21.15.5:0/3317811520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78200a2e20 msgr2=0x7f78200a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78200a2e20 0x7f78200a52e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55dcdd9af3e0 tx=0x7f782400a2b0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 -- 172.21.15.5:0/3317811520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7834106a00 msgr2=0x7f78340fd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7834106a00 0x7f78340fd6d0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f782c00cce0 tx=0x7f782c00a620 comp rx=0 tx=0).stop 2024-09-11T22:34:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 -- 172.21.15.5:0/3317811520 shutdown_connections 2024-09-11T22:34:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7834106a00 0x7f78340fd6d0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f78200a2e20 0x7f78200a52e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7834101390 0x7f78340fd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 --2- 172.21.15.5:0/3317811520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78341009f0 0x7f78340fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 -- 172.21.15.5:0/3317811520 >> 172.21.15.5:0/3317811520 conn(0x7f78340fc6b0 msgr2=0x7f783406a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 -- 172.21.15.5:0/3317811520 shutdown_connections 2024-09-11T22:34:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:37.713+0000 7f783bbd8640 1 -- 172.21.15.5:0/3317811520 wait complete. 2024-09-11T22:34:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:37 smithi005 ceph-mon[30943]: pgmap v394: 97 pgs: 97 active+clean; 90 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-09-11T22:34:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:37 smithi159 ceph-mon[35534]: pgmap v394: 97 pgs: 97 active+clean; 90 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-09-11T22:34:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:37 smithi172 ceph-mon[35461]: pgmap v394: 97 pgs: 97 active+clean; 90 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-09-11T22:34:38.346 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.35 MB/sec execute 536 sec latency 4263.492 ms 2024-09-11T22:34:39.346 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.34 MB/sec execute 537 sec latency 5263.586 ms 2024-09-11T22:34:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:39 smithi005 ceph-mon[30943]: pgmap v395: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-11T22:34:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:34:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:34:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:39 smithi159 ceph-mon[35534]: pgmap v395: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-11T22:34:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:34:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:34:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:39 smithi172 ceph-mon[35461]: pgmap v395: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 336 op/s 2024-09-11T22:34:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:34:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:34:40.346 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.33 MB/sec execute 538 sec latency 6263.713 ms 2024-09-11T22:34:41.346 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.31 MB/sec execute 539 sec latency 7263.819 ms 2024-09-11T22:34:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:41 smithi005 ceph-mon[30943]: pgmap v396: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-09-11T22:34:41.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:41 smithi159 ceph-mon[35534]: pgmap v396: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-09-11T22:34:41.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:41 smithi172 ceph-mon[35461]: pgmap v396: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 2.6 MiB/s wr, 158 op/s 2024-09-11T22:34:42.346 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.30 MB/sec execute 540 sec latency 8263.928 ms 2024-09-11T22:34:43.346 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.29 MB/sec execute 541 sec latency 9264.032 ms 2024-09-11T22:34:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:43 smithi005 ceph-mon[30943]: pgmap v397: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-09-11T22:34:43.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:43 smithi159 ceph-mon[35534]: pgmap v397: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-09-11T22:34:43.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:43 smithi172 ceph-mon[35461]: pgmap v397: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-09-11T22:34:44.347 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.27 MB/sec execute 542 sec latency 10264.153 ms 2024-09-11T22:34:45.347 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.26 MB/sec execute 543 sec latency 11264.266 ms 2024-09-11T22:34:45.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:45 smithi005 ceph-mon[30943]: pgmap v398: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.8 MiB/s wr, 137 op/s 2024-09-11T22:34:45.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:45 smithi159 ceph-mon[35534]: pgmap v398: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.8 MiB/s wr, 137 op/s 2024-09-11T22:34:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:45 smithi172 ceph-mon[35461]: pgmap v398: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.8 MiB/s wr, 137 op/s 2024-09-11T22:34:46.347 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.25 MB/sec execute 544 sec latency 12264.393 ms 2024-09-11T22:34:47.347 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.23 MB/sec execute 545 sec latency 13264.525 ms 2024-09-11T22:34:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:47 smithi005 ceph-mon[30943]: pgmap v399: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.8 MiB/s wr, 137 op/s 2024-09-11T22:34:47.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:47 smithi159 ceph-mon[35534]: pgmap v399: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.8 MiB/s wr, 137 op/s 2024-09-11T22:34:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:47 smithi172 ceph-mon[35461]: pgmap v399: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 1.8 MiB/s wr, 137 op/s 2024-09-11T22:34:48.347 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.22 MB/sec execute 546 sec latency 14264.629 ms 2024-09-11T22:34:49.347 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.21 MB/sec execute 547 sec latency 15264.748 ms 2024-09-11T22:34:49.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:49 smithi005 ceph-mon[30943]: pgmap v400: 97 pgs: 97 active+clean; 87 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-09-11T22:34:49.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:49 smithi159 ceph-mon[35534]: pgmap v400: 97 pgs: 97 active+clean; 87 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-09-11T22:34:49.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:49 smithi172 ceph-mon[35461]: pgmap v400: 97 pgs: 97 active+clean; 87 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-09-11T22:34:50.347 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.19 MB/sec execute 548 sec latency 16264.875 ms 2024-09-11T22:34:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.812+0000 7ffbfb4af640 1 -- 172.21.15.5:0/1957774966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4101350 msgr2=0x7ffbf410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.812+0000 7ffbfb4af640 1 --2- 172.21.15.5:0/1957774966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4101350 0x7ffbf410d880 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ffbec009cd0 tx=0x7ffbec02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.812+0000 7ffbfb4af640 1 -- 172.21.15.5:0/1957774966 shutdown_connections 2024-09-11T22:34:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.812+0000 7ffbfb4af640 1 --2- 172.21.15.5:0/1957774966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf4101350 0x7ffbf410d880 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.812+0000 7ffbfb4af640 1 --2- 172.21.15.5:0/1957774966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf41009b0 0x7ffbf4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.812+0000 7ffbfb4af640 1 --2- 172.21.15.5:0/1957774966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf41069c0 0x7ffbf4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.812+0000 7ffbfb4af640 1 -- 172.21.15.5:0/1957774966 >> 172.21.15.5:0/1957774966 conn(0x7ffbf4077990 msgr2=0x7ffbf40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.812+0000 7ffbfb4af640 1 -- 172.21.15.5:0/1957774966 shutdown_connections 2024-09-11T22:34:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.812+0000 7ffbfb4af640 1 -- 172.21.15.5:0/1957774966 wait complete. 2024-09-11T22:34:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbfb4af640 1 Processor -- start 2024-09-11T22:34:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbfb4af640 1 -- start start 2024-09-11T22:34:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbfb4af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf41009b0 0x7ffbf41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbfb4af640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4101350 0x7ffbf4198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbfb4af640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf41069c0 0x7ffbf419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbfb4af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf4111ef0 con 0x7ffbf41009b0 2024-09-11T22:34:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbfb4af640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf4111d70 con 0x7ffbf4101350 2024-09-11T22:34:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbfb4af640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf4112070 con 0x7ffbf41069c0 2024-09-11T22:34:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbf924b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf41009b0 0x7ffbf41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbf924b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf41009b0 0x7ffbf41982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52754/0 (socket says 172.21.15.5:52754) 2024-09-11T22:34:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.813+0000 7ffbf924b640 1 -- 172.21.15.5:0/3035998573 learned_addr learned my addr 172.21.15.5:0/3035998573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbf8a4a640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4101350 0x7ffbf4198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf41069c0 0x7ffbf419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbf924b640 1 -- 172.21.15.5:0/3035998573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf41069c0 msgr2=0x7ffbf419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbf924b640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf41069c0 0x7ffbf419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbf924b640 1 -- 172.21.15.5:0/3035998573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4101350 msgr2=0x7ffbf4198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbf924b640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4101350 0x7ffbf4198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbf924b640 1 -- 172.21.15.5:0/3035998573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbf419d2d0 con 0x7ffbf41009b0 2024-09-11T22:34:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbf924b640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf41009b0 0x7ffbf41982e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ffbf000d950 tx=0x7ffbf000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbea7fc640 1 -- 172.21.15.5:0/3035998573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbf0017070 con 0x7ffbf41009b0 2024-09-11T22:34:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbea7fc640 1 -- 172.21.15.5:0/3035998573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbf0009c30 con 0x7ffbf41009b0 2024-09-11T22:34:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbfb4af640 1 -- 172.21.15.5:0/3035998573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbf419d5c0 con 0x7ffbf41009b0 2024-09-11T22:34:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbea7fc640 1 -- 172.21.15.5:0/3035998573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbf000f6b0 con 0x7ffbf41009b0 2024-09-11T22:34:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.814+0000 7ffbfb4af640 1 -- 172.21.15.5:0/3035998573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbf41a4e50 con 0x7ffbf41009b0 2024-09-11T22:34:50.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.817+0000 7ffbea7fc640 1 -- 172.21.15.5:0/3035998573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ffbf00109d0 con 0x7ffbf41009b0 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.817+0000 7ffbfe23f740 1 Processor -- start 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.817+0000 7ffbfe23f740 1 -- start start 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.817+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faed7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.817+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faee2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.817+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faee50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.817+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5600fae3c2f0 con 0x5600faed7af0 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.817+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600fade4270 con 0x5600faee2cb0 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.817+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5600fae382b0 con 0x5600faed8410 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf9a4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faed7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf9a4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faed7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52770/0 (socket says 172.21.15.5:52770) 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf9a4c640 1 -- 172.21.15.5:0/2149885192 learned_addr learned my addr 172.21.15.5:0/2149885192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf924b640 1 --2- 172.21.15.5:0/2149885192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faee2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbea7fc640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2e20 0x7ffbe00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf8a4a640 1 --2- 172.21.15.5:0/2149885192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faee50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf9a4c640 1 -- 172.21.15.5:0/2149885192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 msgr2=0x5600faee2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/2149885192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faee2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf9a4c640 1 -- 172.21.15.5:0/2149885192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 msgr2=0x5600faee50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/2149885192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faee50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf9a4c640 1 -- 172.21.15.5:0/2149885192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600fade3340 con 0x5600faed7af0 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbea7fc640 1 -- 172.21.15.5:0/3035998573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7ffbf00cf0b0 con 0x7ffbf41009b0 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2e20 0x7ffbe00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.818+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/2149885192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faed7ed0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ffbec009a60 tx=0x7ffbec005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12af22d705ab0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbdf7fe640 1 -- 172.21.15.5:0/2149885192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec0315b0 con 0x5600faed7af0 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbdf7fe640 1 -- 172.21.15.5:0/2149885192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbec036040 con 0x5600faed7af0 2024-09-11T22:34:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbdf7fe640 1 -- 172.21.15.5:0/2149885192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec0305a0 con 0x5600faed7af0 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbfe23f740 1 -- 172.21.15.5:0/2149885192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 msgr2=0x5600faed7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/2149885192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faed7ed0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7ffbec009a60 tx=0x7ffbec005f70 comp rx=0 tx=0).stop 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2e20 0x7ffbe00a52e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffbec009430 tx=0x7ffbec048000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbfe23f740 1 -- 172.21.15.5:0/2149885192 shutdown_connections 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/2149885192 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faee50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/2149885192 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faee2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/2149885192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faed7ed0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.819+0000 7ffbfe23f740 1 -- 172.21.15.5:0/2149885192 >> 172.21.15.5:0/2149885192 conn(0x5600faed4060 msgr2=0x5600faed4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.820+0000 7ffbfe23f740 1 -- 172.21.15.5:0/2149885192 shutdown_connections 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.820+0000 7ffbfe23f740 1 -- 172.21.15.5:0/2149885192 wait complete. 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.820+0000 7ffbfe23f740 1 -- start start 2024-09-11T22:34:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.820+0000 7ffbfe23f740 1 Processor -- start 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.820+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faedfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.820+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faee0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.820+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faee05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.820+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5600faee5880 con 0x5600faed7af0 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.820+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600fadc5290 con 0x5600faee2cb0 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.820+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5600faee5a00 con 0x5600faed8410 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbfe23f740 10 client.0 _ll_register_callbacks cb 0x5600fadf6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf924b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faedfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf924b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faedfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52784/0 (socket says 172.21.15.5:52784) 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf924b640 1 -- 172.21.15.5:0/1824096320 learned_addr learned my addr 172.21.15.5:0/1824096320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf8a4a640 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faee0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faee05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf924b640 1 -- 172.21.15.5:0/1824096320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 msgr2=0x5600faee0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf924b640 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faee0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf924b640 1 -- 172.21.15.5:0/1824096320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 msgr2=0x5600faee05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf924b640 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faee05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf924b640 1 -- 172.21.15.5:0/1824096320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600fab4f210 con 0x5600faed7af0 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faee05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbf924b640 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faedfb40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ffbf0010fd0 tx=0x7ffbf000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8822d394ac587d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1824096320 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbf0020070 con 0x5600faed7af0 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1824096320 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7ffbf00cc9f0 con 0x5600faed7af0 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1824096320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbec036040 con 0x5600faed7af0 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.821+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1824096320 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ffbf00ccbf0 con 0x5600faed7af0 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbfe23f740 10 client.19356 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:50.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1824096320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5600fac1b620 con 0x5600faed7af0 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1824096320 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbec036040 con 0x5600faed7af0 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1824096320 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7ffbf00d1110 con 0x5600faed7af0 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbfe23f740 10 client.19356 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbfe23f740 10 client.19356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbfe23f740 20 client.19356 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600faef2ba0 0x5600faef4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbfe23f740 4 client.19356 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1824096320 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600fae49a20 con 0x5600faef2ba0 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.822+0000 7ffbf8a4a640 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600faef2ba0 0x5600faef4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.823+0000 7ffbdcff9640 10 client.19356 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.823+0000 7ffbf8a4a640 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600faef2ba0 0x5600faef4fd0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6de217ec75d189a server_cookie=ea067448f362ba4a in_seq=0 out_seq=0 2024-09-11T22:34:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.823+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1824096320 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600fae49a20 con 0x5600faef2ba0 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.823+0000 7ffbdcff9640 10 client.19356 handle_command_reply: tid=0 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 client.19356 shutdown 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1824096320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 msgr2=0x5600faedfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faedfb40 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ffbf0010fd0 tx=0x7ffbf000bf60 comp rx=0 tx=0).stop 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1824096320 shutdown_connections 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600faef2ba0 0x5600faef4fd0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faee05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faee0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbdcff9640 0 client.19356 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1824096320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faedfb40 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1824096320 >> 172.21.15.5:0/1824096320 conn(0x5600faed4060 msgr2=0x5600faed9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1824096320 shutdown_connections 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1824096320 wait complete. 2024-09-11T22:34:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.824+0000 7ffbfe23f740 20 client.19356 trim_cache size 0 max 16384 2024-09-11T22:34:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbfe23f740 1 Processor -- start 2024-09-11T22:34:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbfe23f740 1 -- start start 2024-09-11T22:34:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed7af0 0x5600faedea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faed8410 0x5600faee03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faee2cb0 0x5600faefa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41ada00 con 0x5600faee2cb0 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41af570 con 0x5600faed8410 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41ae0d0 con 0x5600faed7af0 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf9a4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faee2cb0 0x5600faefa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf9a4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faee2cb0 0x5600faefa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52788/0 (socket says 172.21.15.5:52788) 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf9a4c640 1 -- 172.21.15.5:0/2112051836 learned_addr learned my addr 172.21.15.5:0/2112051836 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf8a4a640 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faed8410 0x5600faee03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf924b640 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed7af0 0x5600faedea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf9a4c640 1 -- 172.21.15.5:0/2112051836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed7af0 msgr2=0x5600faedea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed7af0 0x5600faedea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf9a4c640 1 -- 172.21.15.5:0/2112051836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faed8410 msgr2=0x5600faee03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faed8410 0x5600faee03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf9a4c640 1 -- 172.21.15.5:0/2112051836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600faed3e90 con 0x5600faee2cb0 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf924b640 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed7af0 0x5600faedea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf8a4a640 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faed8410 0x5600faee03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.839+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faee2cb0 0x5600faefa8e0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ffbec02a920 tx=0x7ffbec03e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc7d0f191b0a851 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbdeffd640 1 -- 172.21.15.5:0/2112051836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec059070 con 0x5600faee2cb0 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbdeffd640 1 -- 172.21.15.5:0/2112051836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbec036030 con 0x5600faee2cb0 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbdeffd640 1 -- 172.21.15.5:0/2112051836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbec0042a0 con 0x5600faee2cb0 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbfe23f740 1 -- 172.21.15.5:0/2112051836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faee2cb0 msgr2=0x5600faefa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faee2cb0 0x5600faefa8e0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ffbec02a920 tx=0x7ffbec03e040 comp rx=0 tx=0).stop 2024-09-11T22:34:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbfe23f740 1 -- 172.21.15.5:0/2112051836 shutdown_connections 2024-09-11T22:34:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faee2cb0 0x5600faefa8e0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faed8410 0x5600faee03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/2112051836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed7af0 0x5600faedea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbfe23f740 1 -- 172.21.15.5:0/2112051836 >> 172.21.15.5:0/2112051836 conn(0x5600faed4060 msgr2=0x5600faee1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbfe23f740 1 -- 172.21.15.5:0/2112051836 shutdown_connections 2024-09-11T22:34:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.840+0000 7ffbfe23f740 1 -- 172.21.15.5:0/2112051836 wait complete. 2024-09-11T22:34:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbfe23f740 1 -- start start 2024-09-11T22:34:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbfe23f740 1 Processor -- start 2024-09-11T22:34:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faeeb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faedc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbfe23f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faedcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41aeea0 con 0x5600faed7af0 2024-09-11T22:34:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41ae7d0 con 0x5600faee2cb0 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbfe23f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ffbf41aba00 con 0x5600faed8410 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbfe23f740 10 client.0 _ll_register_callbacks cb 0x7ffbf000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbf924b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faeeb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbf924b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faeeb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52804/0 (socket says 172.21.15.5:52804) 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.841+0000 7ffbf924b640 1 -- 172.21.15.5:0/1598059034 learned_addr learned my addr 172.21.15.5:0/1598059034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbf9a4c640 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faedcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbf8a4a640 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faedc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbf924b640 1 -- 172.21.15.5:0/1598059034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 msgr2=0x5600faedc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbf924b640 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faedc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbf924b640 1 -- 172.21.15.5:0/1598059034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 msgr2=0x5600faedcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbf924b640 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faedcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbf924b640 1 -- 172.21.15.5:0/1598059034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600faecf2d0 con 0x5600faed7af0 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbf924b640 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faeeb330 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ffbf00058c0 tx=0x7ffbf00a4310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6147e219eb8711 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1598059034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbf009b310 con 0x5600faed7af0 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1598059034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7ffbf0097290 con 0x5600faed7af0 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1598059034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbec036030 con 0x5600faed7af0 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1598059034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7ffbf00977e0 con 0x5600faed7af0 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbfe23f740 10 client.19362 fetch_fsmap learned FSMap version 46 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1598059034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5600faed4cf0 con 0x5600faed7af0 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.842+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1598059034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbec036030 con 0x5600faed7af0 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.843+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1598059034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7ffbf009a990 con 0x5600faed7af0 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.843+0000 7ffbfe23f740 10 client.19362 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.843+0000 7ffbfe23f740 10 client.19362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:34:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.843+0000 7ffbfe23f740 20 client.19362 populate_metadata read hostname 'smithi005' 2024-09-11T22:34:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.843+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600faf664b0 0x5600faf688a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.843+0000 7ffbfe23f740 4 client.19362 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:34:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.843+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1598059034 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600fae43d70 con 0x5600faf664b0 2024-09-11T22:34:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.843+0000 7ffbf8a4a640 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600faf664b0 0x5600faf688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.843+0000 7ffbdcff9640 10 client.19362 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.843+0000 7ffbf8a4a640 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600faf664b0 0x5600faf688a0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d37fd7e50e7200a1 server_cookie=3a8f7d04da7471d0 in_seq=0 out_seq=0 2024-09-11T22:34:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.877+0000 7ffbdcff9640 1 -- 172.21.15.5:0/1598059034 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600fae43d70 con 0x5600faf664b0 2024-09-11T22:34:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.877+0000 7ffbdcff9640 10 client.19362 handle_command_reply: tid=0 2024-09-11T22:34:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.877+0000 7ffbfe23f740 1 client.19362 shutdown 2024-09-11T22:34:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1598059034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 msgr2=0x5600faeeb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faeeb330 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7ffbf00058c0 tx=0x7ffbf00a4310 comp rx=0 tx=0).stop 2024-09-11T22:34:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1598059034 shutdown_connections 2024-09-11T22:34:50.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600faf664b0 0x5600faf688a0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600faee2cb0 0x5600faedcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600faed8410 0x5600faedc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbdcff9640 0 client.19362 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:34:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 1 --2- 172.21.15.5:0/1598059034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600faed7af0 0x5600faeeb330 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1598059034 >> 172.21.15.5:0/1598059034 conn(0x5600faed4060 msgr2=0x5600faed29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1598059034 shutdown_connections 2024-09-11T22:34:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 1 -- 172.21.15.5:0/1598059034 wait complete. 2024-09-11T22:34:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.878+0000 7ffbfe23f740 20 client.19362 trim_cache size 0 max 16384 2024-09-11T22:34:50.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:34:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:34:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:34:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:34:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 -- 172.21.15.5:0/3035998573 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2e20 msgr2=0x7ffbe00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:50.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2e20 0x7ffbe00a52e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffbec009430 tx=0x7ffbec048000 comp rx=0 tx=0).stop 2024-09-11T22:34:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 -- 172.21.15.5:0/3035998573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf41009b0 msgr2=0x7ffbf41982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf41009b0 0x7ffbf41982e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ffbf000d950 tx=0x7ffbf000de10 comp rx=0 tx=0).stop 2024-09-11T22:34:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 -- 172.21.15.5:0/3035998573 shutdown_connections 2024-09-11T22:34:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffbf41069c0 0x7ffbf419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ffbe00a2e20 0x7ffbe00a52e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffbf4101350 0x7ffbf4198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 --2- 172.21.15.5:0/3035998573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbf41009b0 0x7ffbf41982e0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 -- 172.21.15.5:0/3035998573 >> 172.21.15.5:0/3035998573 conn(0x7ffbf4077990 msgr2=0x7ffbf40fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 -- 172.21.15.5:0/3035998573 shutdown_connections 2024-09-11T22:34:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:34:50.879+0000 7ffbfb4af640 1 -- 172.21.15.5:0/3035998573 wait complete. 2024-09-11T22:34:51.347 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.18 MB/sec execute 549 sec latency 17264.960 ms 2024-09-11T22:34:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:51 smithi005 ceph-mon[30943]: pgmap v401: 97 pgs: 97 active+clean; 87 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-09-11T22:34:51.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:51 smithi159 ceph-mon[35534]: pgmap v401: 97 pgs: 97 active+clean; 87 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-09-11T22:34:51.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:51 smithi172 ceph-mon[35461]: pgmap v401: 97 pgs: 97 active+clean; 87 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.3 MiB/s wr, 106 op/s 2024-09-11T22:34:52.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.17 MB/sec execute 550 sec latency 18265.065 ms 2024-09-11T22:34:53.347 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.15 MB/sec execute 551 sec latency 19265.115 ms 2024-09-11T22:34:53.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:53 smithi005 ceph-mon[30943]: pgmap v402: 97 pgs: 97 active+clean; 84 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.8 MiB/s wr, 140 op/s 2024-09-11T22:34:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:53 smithi159 ceph-mon[35534]: pgmap v402: 97 pgs: 97 active+clean; 84 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.8 MiB/s wr, 140 op/s 2024-09-11T22:34:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:53 smithi172 ceph-mon[35461]: pgmap v402: 97 pgs: 97 active+clean; 84 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 1.8 MiB/s wr, 140 op/s 2024-09-11T22:34:54.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.14 MB/sec execute 552 sec latency 20265.218 ms 2024-09-11T22:34:55.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.13 MB/sec execute 553 sec latency 21265.312 ms 2024-09-11T22:34:55.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:55 smithi005 ceph-mon[30943]: pgmap v403: 97 pgs: 97 active+clean; 84 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-09-11T22:34:55.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:55 smithi159 ceph-mon[35534]: pgmap v403: 97 pgs: 97 active+clean; 84 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-09-11T22:34:55.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:55 smithi172 ceph-mon[35461]: pgmap v403: 97 pgs: 97 active+clean; 84 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-09-11T22:34:56.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.11 MB/sec execute 554 sec latency 22265.447 ms 2024-09-11T22:34:57.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.10 MB/sec execute 555 sec latency 23265.546 ms 2024-09-11T22:34:57.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:34:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:57.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:57 smithi005 ceph-mon[30943]: pgmap v404: 97 pgs: 97 active+clean; 84 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-09-11T22:34:57.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:57 smithi159 ceph-mon[35534]: pgmap v404: 97 pgs: 97 active+clean; 84 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-09-11T22:34:57.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:57 smithi172 ceph-mon[35461]: pgmap v404: 97 pgs: 97 active+clean; 84 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 58 op/s 2024-09-11T22:34:58.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.09 MB/sec execute 556 sec latency 24265.590 ms 2024-09-11T22:34:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.339+0000 7f77b8022640 1 -- 172.21.15.5:0/3699160156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77b0100a80 msgr2=0x7f77b0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.339+0000 7f77b8022640 1 --2- 172.21.15.5:0/3699160156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77b0100a80 0x7f77b0100ee0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009930 tx=0x7f77a402a420 comp rx=0 tx=0).stop 2024-09-11T22:34:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.339+0000 7f77b8022640 1 -- 172.21.15.5:0/3699160156 shutdown_connections 2024-09-11T22:34:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.339+0000 7f77b8022640 1 --2- 172.21.15.5:0/3699160156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77b0101420 0x7f77b010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.339+0000 7f77b8022640 1 --2- 172.21.15.5:0/3699160156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77b0100a80 0x7f77b0100ee0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.339+0000 7f77b8022640 1 --2- 172.21.15.5:0/3699160156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77b0106a80 0x7f77b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.339+0000 7f77b8022640 1 -- 172.21.15.5:0/3699160156 >> 172.21.15.5:0/3699160156 conn(0x7f77b00fc7a0 msgr2=0x7f77b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.339+0000 7f77b8022640 1 -- 172.21.15.5:0/3699160156 shutdown_connections 2024-09-11T22:34:58.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.339+0000 7f77b8022640 1 -- 172.21.15.5:0/3699160156 wait complete. 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b8022640 1 Processor -- start 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b8022640 1 -- start start 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b8022640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77b0100a80 0x7f77b0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b8022640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77b0101420 0x7f77b0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b8022640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77b0106a80 0x7f77b019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b8022640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0110ea0 con 0x7f77b0106a80 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b8022640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0110d20 con 0x7f77b0100a80 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b8022640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f77b0111020 con 0x7f77b0101420 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b65bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77b0106a80 0x7f77b019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b65bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77b0106a80 0x7f77b019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59220/0 (socket says 172.21.15.5:59220) 2024-09-11T22:34:58.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b65bf640 1 -- 172.21.15.5:0/459026416 learned_addr learned my addr 172.21.15.5:0/459026416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b55bd640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77b0101420 0x7f77b0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b5dbe640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77b0100a80 0x7f77b0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b65bf640 1 -- 172.21.15.5:0/459026416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77b0101420 msgr2=0x7f77b0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b65bf640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77b0101420 0x7f77b0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b65bf640 1 -- 172.21.15.5:0/459026416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77b0100a80 msgr2=0x7f77b0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b65bf640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77b0100a80 0x7f77b0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.340+0000 7f77b65bf640 1 -- 172.21.15.5:0/459026416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b019d3f0 con 0x7f77b0106a80 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.341+0000 7f77b65bf640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77b0106a80 0x7f77b019cd10 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f77a800b970 tx=0x7f77a800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.341+0000 7f77a2ffd640 1 -- 172.21.15.5:0/459026416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8019070 con 0x7f77b0106a80 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.341+0000 7f77a2ffd640 1 -- 172.21.15.5:0/459026416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a8014440 con 0x7f77b0106a80 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.341+0000 7f77b8022640 1 -- 172.21.15.5:0/459026416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b019d6e0 con 0x7f77b0106a80 2024-09-11T22:34:58.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.341+0000 7f77a2ffd640 1 -- 172.21.15.5:0/459026416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a80075d0 con 0x7f77b0106a80 2024-09-11T22:34:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.341+0000 7f77b8022640 1 -- 172.21.15.5:0/459026416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b01a4fc0 con 0x7f77b0106a80 2024-09-11T22:34:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.342+0000 7f77b8022640 1 -- 172.21.15.5:0/459026416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b0100ee0 con 0x7f77b0106a80 2024-09-11T22:34:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.344+0000 7f77a2ffd640 1 -- 172.21.15.5:0/459026416 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f77a801c070 con 0x7f77b0106a80 2024-09-11T22:34:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.345+0000 7f77a2ffd640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f779c0a2cf0 0x7f779c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:34:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.345+0000 7f77b5dbe640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f779c0a2cf0 0x7f779c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:34:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.345+0000 7f77a2ffd640 1 -- 172.21.15.5:0/459026416 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7f77a80cee30 con 0x7f77b0106a80 2024-09-11T22:34:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.345+0000 7f77b5dbe640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f779c0a2cf0 0x7f779c0a51b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00abe0 tx=0x7f77ac00a2d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:34:58.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.348+0000 7f77a2ffd640 1 -- 172.21.15.5:0/459026416 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77a809b440 con 0x7f77b0106a80 2024-09-11T22:34:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.589+0000 7f77b8022640 1 -- 172.21.15.5:0/459026416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77b010a080 con 0x7f77b0106a80 2024-09-11T22:34:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.590+0000 7f77a2ffd640 1 -- 172.21.15.5:0/459026416 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11968 (secure 0 0 0) 0x7f77a809f740 con 0x7f77b0106a80 2024-09-11T22:34:58.591 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:34:58.591 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":46,"btime":"2024-09-11T22:34:22:505619+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:34:21.776249+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28160":{"gid":28160,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/2346038927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":2346038927},{"type":"v1","addr":"172.21.15.159:6833","nonce":2346038927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:34:58.591 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 46 2024-09-11T22:34:58.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 -- 172.21.15.5:0/459026416 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f779c0a2cf0 msgr2=0x7f779c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:58.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f779c0a2cf0 0x7f779c0a51b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00abe0 tx=0x7f77ac00a2d0 comp rx=0 tx=0).stop 2024-09-11T22:34:58.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 -- 172.21.15.5:0/459026416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77b0106a80 msgr2=0x7f77b019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:34:58.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77b0106a80 0x7f77b019cd10 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f77a800b970 tx=0x7f77a800bd70 comp rx=0 tx=0).stop 2024-09-11T22:34:58.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 -- 172.21.15.5:0/459026416 shutdown_connections 2024-09-11T22:34:58.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77b0106a80 0x7f77b019cd10 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:58.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77b0101420 0x7f77b0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:58.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f779c0a2cf0 0x7f779c0a51b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:58.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 --2- 172.21.15.5:0/459026416 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77b0100a80 0x7f77b0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:34:58.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 -- 172.21.15.5:0/459026416 >> 172.21.15.5:0/459026416 conn(0x7f77b00fc7a0 msgr2=0x7f77b006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:34:58.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 -- 172.21.15.5:0/459026416 shutdown_connections 2024-09-11T22:34:58.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:34:58.592+0000 7f77b8022640 1 -- 172.21.15.5:0/459026416 wait complete. 2024-09-11T22:34:59.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 939863 7.08 MB/sec execute 557 sec latency 25265.710 ms 2024-09-11T22:34:59.425 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 771318f --release --await' 2024-09-11T22:34:59.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 771318f --release --await 2024-09-11T22:34:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:34:59.817 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:59 smithi005 ceph-mon[30943]: pgmap v405: 97 pgs: 97 active+clean; 98 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 2.9 MiB/s wr, 86 op/s 2024-09-11T22:34:59.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:34:59 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/459026416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:34:59.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:59 smithi159 ceph-mon[35534]: pgmap v405: 97 pgs: 97 active+clean; 98 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 2.9 MiB/s wr, 86 op/s 2024-09-11T22:34:59.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:34:59 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/459026416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:34:59.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:59 smithi172 ceph-mon[35461]: pgmap v405: 97 pgs: 97 active+clean; 98 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 2.9 MiB/s wr, 86 op/s 2024-09-11T22:34:59.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:34:59 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/459026416' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:35:00.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.199+0000 7fa013307640 1 -- 172.21.15.5:0/2482077354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa00c101430 msgr2=0x7fa00c10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:00.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.199+0000 7fa013307640 1 --2- 172.21.15.5:0/2482077354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa00c101430 0x7fa00c10d960 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc009cd0 tx=0x7f9ffc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.199+0000 7fa013307640 1 -- 172.21.15.5:0/2482077354 shutdown_connections 2024-09-11T22:35:00.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.199+0000 7fa013307640 1 --2- 172.21.15.5:0/2482077354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa00c101430 0x7fa00c10d960 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.199+0000 7fa013307640 1 --2- 172.21.15.5:0/2482077354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa00c100a90 0x7fa00c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.199+0000 7fa013307640 1 --2- 172.21.15.5:0/2482077354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa00c106a90 0x7fa00c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.199+0000 7fa013307640 1 -- 172.21.15.5:0/2482077354 >> 172.21.15.5:0/2482077354 conn(0x7fa00c0fc7d0 msgr2=0x7fa00c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.199+0000 7fa013307640 1 -- 172.21.15.5:0/2482077354 shutdown_connections 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.199+0000 7fa013307640 1 -- 172.21.15.5:0/2482077354 wait complete. 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa013307640 1 Processor -- start 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa013307640 1 -- start start 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa013307640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa00c100a90 0x7fa00c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa013307640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa00c101430 0x7fa00c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa013307640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa00c106a90 0x7fa00c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa013307640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c110e90 con 0x7fa00c101430 2024-09-11T22:35:00.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa013307640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c110d10 con 0x7fa00c106a90 2024-09-11T22:35:00.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa013307640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c111010 con 0x7fa00c100a90 2024-09-11T22:35:00.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa0110a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa00c100a90 0x7fa00c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa0110a3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa00c100a90 0x7fa00c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:42508/0 (socket says 172.21.15.5:42508) 2024-09-11T22:35:00.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.200+0000 7fa0110a3640 1 -- 172.21.15.5:0/1301835551 learned_addr learned my addr 172.21.15.5:0/1301835551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:00.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa0118a4640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa00c106a90 0x7fa00c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa0108a2640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa00c101430 0x7fa00c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa0110a3640 1 -- 172.21.15.5:0/1301835551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa00c106a90 msgr2=0x7fa00c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:00.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa0110a3640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa00c106a90 0x7fa00c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa0110a3640 1 -- 172.21.15.5:0/1301835551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa00c101430 msgr2=0x7fa00c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:00.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa0110a3640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa00c101430 0x7fa00c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa0110a3640 1 -- 172.21.15.5:0/1301835551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00c19f600 con 0x7fa00c100a90 2024-09-11T22:35:00.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa0110a3640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa00c100a90 0x7fa00c19a650 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa00400d950 tx=0x7fa00400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:00.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa00a7fc640 1 -- 172.21.15.5:0/1301835551 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004017070 con 0x7fa00c100a90 2024-09-11T22:35:00.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa013307640 1 -- 172.21.15.5:0/1301835551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00c19f8f0 con 0x7fa00c100a90 2024-09-11T22:35:00.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa00a7fc640 1 -- 172.21.15.5:0/1301835551 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa004009c30 con 0x7fa00c100a90 2024-09-11T22:35:00.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa013307640 1 -- 172.21.15.5:0/1301835551 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa00c1a71d0 con 0x7fa00c100a90 2024-09-11T22:35:00.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.201+0000 7fa00a7fc640 1 -- 172.21.15.5:0/1301835551 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00400f6b0 con 0x7fa00c100a90 2024-09-11T22:35:00.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.204+0000 7fa0156e0740 1 Processor -- start 2024-09-11T22:35:00.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.204+0000 7fa0156e0740 1 -- start start 2024-09-11T22:35:00.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.204+0000 7fa00a7fc640 1 -- 172.21.15.5:0/1301835551 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa0040109d0 con 0x7fa00c100a90 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.204+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.204+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 0x55e94ff34d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.204+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff37630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.204+0000 7fa0156e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e94fe8f620 con 0x55e94ff2a060 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.204+0000 7fa0156e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e94fe36e50 con 0x55e94ff2a980 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.204+0000 7fa0156e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e94fe8b5e0 con 0x55e94ff35240 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0118a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0118a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59246/0 (socket says 172.21.15.5:59246) 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0118a4640 1 -- 172.21.15.5:0/608615710 learned_addr learned my addr 172.21.15.5:0/608615710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0108a2640 1 --2- 172.21.15.5:0/608615710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff37630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0110a3640 1 --2- 172.21.15.5:0/608615710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 0x55e94ff34d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0118a4640 1 -- 172.21.15.5:0/608615710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 msgr2=0x55e94ff37630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0118a4640 1 --2- 172.21.15.5:0/608615710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff37630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0118a4640 1 -- 172.21.15.5:0/608615710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 msgr2=0x55e94ff34d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0118a4640 1 --2- 172.21.15.5:0/608615710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 0x55e94ff34d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0118a4640 1 -- 172.21.15.5:0/608615710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e94fe27d40 con 0x55e94ff2a060 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa00a7fc640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ff40a2cf0 0x7f9ff40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0118a4640 1 --2- 172.21.15.5:0/608615710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2a440 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc035000 tx=0x7f9ffc031570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd2e1530842fb3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa0110a3640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ff40a2cf0 0x7f9ff40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7f9ff37fe640 1 -- 172.21.15.5:0/608615710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc048070 con 0x55e94ff2a060 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7f9ff37fe640 1 -- 172.21.15.5:0/608615710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ffc036070 con 0x55e94ff2a060 2024-09-11T22:35:00.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7fa00a7fc640 1 -- 172.21.15.5:0/1301835551 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fa0040cf360 con 0x7fa00c100a90 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.205+0000 7f9ff37fe640 1 -- 172.21.15.5:0/608615710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc004600 con 0x55e94ff2a060 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.206+0000 7fa0156e0740 1 -- 172.21.15.5:0/608615710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 msgr2=0x55e94ff2a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.206+0000 7fa0110a3640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ff40a2cf0 0x7f9ff40a51b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa0040ccaf0 tx=0x7fa0040d2eb0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.206+0000 7fa0156e0740 1 --2- 172.21.15.5:0/608615710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2a440 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc035000 tx=0x7f9ffc031570 comp rx=0 tx=0).stop 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.206+0000 7fa0156e0740 1 -- 172.21.15.5:0/608615710 shutdown_connections 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.206+0000 7fa0156e0740 1 --2- 172.21.15.5:0/608615710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff37630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.206+0000 7fa0156e0740 1 --2- 172.21.15.5:0/608615710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 0x55e94ff34d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.206+0000 7fa0156e0740 1 --2- 172.21.15.5:0/608615710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2a440 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.206+0000 7fa0156e0740 1 -- 172.21.15.5:0/608615710 >> 172.21.15.5:0/608615710 conn(0x55e94ff265f0 msgr2=0x55e94ff269d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.206+0000 7fa0156e0740 1 -- 172.21.15.5:0/608615710 shutdown_connections 2024-09-11T22:35:00.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.206+0000 7fa0156e0740 1 -- 172.21.15.5:0/608615710 wait complete. 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0156e0740 1 -- start start 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0156e0740 1 Processor -- start 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a060 0x55e94ff32180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff326c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff32c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0156e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e94ff37e10 con 0x55e94ff2a980 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0156e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e94fe17f10 con 0x55e94ff2a060 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0156e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e94ff37f90 con 0x55e94ff35240 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0156e0740 10 client.0 _ll_register_callbacks cb 0x55e94fe3bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0108a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff326c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.207+0000 7fa0108a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff326c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59262/0 (socket says 172.21.15.5:59262) 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0108a2640 1 -- 172.21.15.5:0/196399587 learned_addr learned my addr 172.21.15.5:0/196399587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0118a4640 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff32c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0118a4640 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a060 0x55e94ff32180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0108a2640 1 -- 172.21.15.5:0/196399587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 msgr2=0x55e94ff32c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0108a2640 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff32c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0108a2640 1 -- 172.21.15.5:0/196399587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a060 msgr2=0x55e94ff32180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0108a2640 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a060 0x55e94ff32180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0108a2640 1 -- 172.21.15.5:0/196399587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e94fb9e580 con 0x55e94ff2a980 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0108a2640 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff326c0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa000010770 tx=0x7fa000010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d44024682d4849d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7f9ff0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:00.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/196399587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00000fb90 con 0x55e94ff2a980 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/196399587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fa00000fd30 con 0x55e94ff2a980 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0156e0740 1 -- 172.21.15.5:0/196399587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ffc036070 con 0x55e94ff2a980 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/196399587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa000004020 con 0x55e94ff2a980 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0156e0740 10 client.19380 fetch_fsmap learned FSMap version 46 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7fa0156e0740 1 -- 172.21.15.5:0/196399587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e94fc68960 con 0x55e94ff2a980 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.208+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/196399587 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ffc036070 con 0x55e94ff2a980 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.209+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/196399587 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fa000019410 con 0x55e94ff2a980 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.209+0000 7fa0156e0740 10 client.19380 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.209+0000 7fa0156e0740 10 client.19380 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.209+0000 7fa0156e0740 20 client.19380 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:00.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.209+0000 7fa0156e0740 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e94ff44e40 0x55e94ff472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.209+0000 7fa0156e0740 4 client.19380 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.209+0000 7fa0156e0740 1 -- 172.21.15.5:0/196399587 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e94fe9cb90 con 0x55e94ff44e40 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.209+0000 7fa0118a4640 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e94ff44e40 0x55e94ff472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.209+0000 7f9ff0ff9640 10 client.19380 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.209+0000 7fa0118a4640 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e94ff44e40 0x55e94ff472a0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f05e8e82de460390 server_cookie=572aa8110656a7ec in_seq=0 out_seq=0 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/196399587 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e94fe9cb90 con 0x55e94ff44e40 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7f9ff0ff9640 10 client.19380 handle_command_reply: tid=0 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7fa0156e0740 1 client.19380 shutdown 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7fa0156e0740 1 -- 172.21.15.5:0/196399587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 msgr2=0x55e94ff326c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7fa0156e0740 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff326c0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa000010770 tx=0x7fa000010c30 comp rx=0 tx=0).stop 2024-09-11T22:35:00.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7fa0156e0740 1 -- 172.21.15.5:0/196399587 shutdown_connections 2024-09-11T22:35:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7fa0156e0740 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e94ff44e40 0x55e94ff472a0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7fa0156e0740 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff32c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7fa0156e0740 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff326c0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7fa0156e0740 1 --2- 172.21.15.5:0/196399587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a060 0x55e94ff32180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7f9ff0ff9640 0 client.19380 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.210+0000 7fa0156e0740 1 -- 172.21.15.5:0/196399587 >> 172.21.15.5:0/196399587 conn(0x55e94ff265f0 msgr2=0x55e94ff25750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.211+0000 7fa0156e0740 1 -- 172.21.15.5:0/196399587 shutdown_connections 2024-09-11T22:35:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.211+0000 7fa0156e0740 1 -- 172.21.15.5:0/196399587 wait complete. 2024-09-11T22:35:00.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.211+0000 7fa0156e0740 20 client.19380 trim_cache size 0 max 16384 2024-09-11T22:35:00.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0156e0740 1 Processor -- start 2024-09-11T22:35:00.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0156e0740 1 -- start start 2024-09-11T22:35:00.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 0x55e94ff4cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff4d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0156e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c1afd60 con 0x55e94ff2a060 2024-09-11T22:35:00.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0156e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c1b18d0 con 0x55e94ff2a980 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0156e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c1b0430 con 0x55e94ff35240 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0118a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0118a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59270/0 (socket says 172.21.15.5:59270) 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.228+0000 7fa0118a4640 1 -- 172.21.15.5:0/984623708 learned_addr learned my addr 172.21.15.5:0/984623708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7fa0108a2640 1 --2- 172.21.15.5:0/984623708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 0x55e94ff4cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7fa0118a4640 1 -- 172.21.15.5:0/984623708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 msgr2=0x55e94ff4d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7fa0118a4640 1 --2- 172.21.15.5:0/984623708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff4d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7fa0118a4640 1 -- 172.21.15.5:0/984623708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 msgr2=0x55e94ff4cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7fa0118a4640 1 --2- 172.21.15.5:0/984623708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 0x55e94ff4cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7fa0118a4640 1 -- 172.21.15.5:0/984623708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e94ff276a0 con 0x55e94ff2a060 2024-09-11T22:35:00.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7fa0108a2640 1 --2- 172.21.15.5:0/984623708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 0x55e94ff4cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7fa0118a4640 1 --2- 172.21.15.5:0/984623708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2c210 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0040b0 tx=0x7f9ffc004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9142612ef52193d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7f9ff2ffd640 1 -- 172.21.15.5:0/984623708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc048040 con 0x55e94ff2a060 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7f9ff2ffd640 1 -- 172.21.15.5:0/984623708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ffc036040 con 0x55e94ff2a060 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.229+0000 7f9ff2ffd640 1 -- 172.21.15.5:0/984623708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc0305a0 con 0x55e94ff2a060 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 -- 172.21.15.5:0/984623708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 msgr2=0x55e94ff2c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 --2- 172.21.15.5:0/984623708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2c210 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc0040b0 tx=0x7f9ffc004180 comp rx=0 tx=0).stop 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 -- 172.21.15.5:0/984623708 shutdown_connections 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 --2- 172.21.15.5:0/984623708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff35240 0x55e94ff4d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 --2- 172.21.15.5:0/984623708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff2a980 0x55e94ff4cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 --2- 172.21.15.5:0/984623708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a060 0x55e94ff2c210 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 -- 172.21.15.5:0/984623708 >> 172.21.15.5:0/984623708 conn(0x55e94ff265f0 msgr2=0x55e94ff2bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:00.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 -- 172.21.15.5:0/984623708 shutdown_connections 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 -- 172.21.15.5:0/984623708 wait complete. 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 -- start start 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.230+0000 7fa0156e0740 1 Processor -- start 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff2a060 0x55e94ff3d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff39bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0156e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff35240 0x55e94ff3a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0156e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c1b1200 con 0x55e94ff2a980 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0156e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c1b0b30 con 0x55e94ff35240 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0156e0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa00c1add60 con 0x55e94ff2a060 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0156e0740 10 client.0 _ll_register_callbacks cb 0x7fa00000abf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:00.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0108a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff39bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0108a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff39bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59274/0 (socket says 172.21.15.5:59274) 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0108a2640 1 -- 172.21.15.5:0/1171954435 learned_addr learned my addr 172.21.15.5:0/1171954435 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0118a4640 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff35240 0x55e94ff3a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0118a4640 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff2a060 0x55e94ff3d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0108a2640 1 -- 172.21.15.5:0/1171954435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff2a060 msgr2=0x55e94ff3d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0108a2640 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff2a060 0x55e94ff3d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0108a2640 1 -- 172.21.15.5:0/1171954435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff35240 msgr2=0x55e94ff3a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0108a2640 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff35240 0x55e94ff3a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0108a2640 1 -- 172.21.15.5:0/1171954435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e94ff3a960 con 0x55e94ff2a980 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0118a4640 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff2a060 0x55e94ff3d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:35:00.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7fa0108a2640 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff39bf0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fa000000c00 tx=0x7fa00001af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3788d0b311f504 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.231+0000 7f9ff0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/1171954435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00002e020 con 0x55e94ff2a980 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/1171954435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fa0000195f0 con 0x55e94ff2a980 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7fa0156e0740 1 -- 172.21.15.5:0/1171954435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ffc036040 con 0x55e94ff2a980 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/1171954435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa000039050 con 0x55e94ff2a980 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7fa0156e0740 10 client.19386 fetch_fsmap learned FSMap version 46 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7fa0156e0740 1 -- 172.21.15.5:0/1171954435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55e94ffb0ad0 con 0x55e94ff2a980 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/1171954435 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ffc036040 con 0x55e94ff2a980 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/1171954435 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fa000003b60 con 0x55e94ff2a980 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7fa0156e0740 10 client.19386 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7fa0156e0740 10 client.19386 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:35:00.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7fa0156e0740 20 client.19386 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:00.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7fa0156e0740 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e94ffb12a0 0x55e94ffb36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:00.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7fa0156e0740 4 client.19386 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "771318f", "release": true, "await": true}] 2024-09-11T22:35:00.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.232+0000 7fa0156e0740 1 -- 172.21.15.5:0/1171954435 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "771318f", "release": true, "await": true}) -- 0x55e94fe970a0 con 0x55e94ffb12a0 2024-09-11T22:35:00.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.233+0000 7fa0118a4640 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e94ffb12a0 0x55e94ffb36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:00.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.233+0000 7fa0118a4640 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e94ffb12a0 0x55e94ffb36c0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec4113f1981909a server_cookie=67e27338e86d2c48 in_seq=0 out_seq=0 2024-09-11T22:35:00.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.233+0000 7f9ff0ff9640 10 client.19386 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:00.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.245+0000 7f9ff0ff9640 1 -- 172.21.15.5:0/1171954435 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7f9ffc054260 con 0x55e94ffb12a0 2024-09-11T22:35:00.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.245+0000 7f9ff0ff9640 10 client.19386 handle_command_reply: tid=0 2024-09-11T22:35:00.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.245+0000 7fa0156e0740 1 client.19386 shutdown 2024-09-11T22:35:00.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 1 -- 172.21.15.5:0/1171954435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 msgr2=0x55e94ff39bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:00.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff39bf0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fa000000c00 tx=0x7fa00001af30 comp rx=0 tx=0).stop 2024-09-11T22:35:00.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 1 -- 172.21.15.5:0/1171954435 shutdown_connections 2024-09-11T22:35:00.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e94ffb12a0 0x55e94ffb36c0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e94ff35240 0x55e94ff3a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e94ff2a980 0x55e94ff39bf0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 1 --2- 172.21.15.5:0/1171954435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e94ff2a060 0x55e94ff3d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7f9ff0ff9640 0 client.19386 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:00.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 1 -- 172.21.15.5:0/1171954435 >> 172.21.15.5:0/1171954435 conn(0x55e94ff265f0 msgr2=0x55e94ff2bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:00.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 1 -- 172.21.15.5:0/1171954435 shutdown_connections 2024-09-11T22:35:00.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 1 -- 172.21.15.5:0/1171954435 wait complete. 2024-09-11T22:35:00.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa0156e0740 20 client.19386 trim_cache size 0 max 16384 2024-09-11T22:35:00.248 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 46, 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 48, 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: "771318f": { 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 48, 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:35:00.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 147.2, 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:00.250 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:00.251 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:00.251 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:35:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa013307640 1 -- 172.21.15.5:0/1301835551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ff40a2cf0 msgr2=0x7f9ff40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa013307640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ff40a2cf0 0x7f9ff40a51b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa0040ccaf0 tx=0x7fa0040d2eb0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa013307640 1 -- 172.21.15.5:0/1301835551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa00c100a90 msgr2=0x7fa00c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:00.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa013307640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa00c100a90 0x7fa00c19a650 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa00400d950 tx=0x7fa00400de10 comp rx=0 tx=0).stop 2024-09-11T22:35:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa013307640 1 -- 172.21.15.5:0/1301835551 shutdown_connections 2024-09-11T22:35:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa013307640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ff40a2cf0 0x7f9ff40a51b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa013307640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa00c106a90 0x7fa00c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.246+0000 7fa013307640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa00c101430 0x7fa00c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.247+0000 7fa013307640 1 --2- 172.21.15.5:0/1301835551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa00c100a90 0x7fa00c19a650 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.247+0000 7fa013307640 1 -- 172.21.15.5:0/1301835551 >> 172.21.15.5:0/1301835551 conn(0x7fa00c0fc7d0 msgr2=0x7fa00c06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.247+0000 7fa013307640 1 -- 172.21.15.5:0/1301835551 shutdown_connections 2024-09-11T22:35:00.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:00.247+0000 7fa013307640 1 -- 172.21.15.5:0/1301835551 wait complete. 2024-09-11T22:35:00.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 940153 7.06 MB/sec execute 558 sec latency 26162.070 ms 2024-09-11T22:35:01.024 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '771318f', total seconds elapsed: 30.7 2024-09-11T22:35:01.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 941738 7.07 MB/sec execute 559 sec latency 23.538 ms 2024-09-11T22:35:01.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:01 smithi005 ceph-mon[30943]: pgmap v406: 97 pgs: 97 active+clean; 98 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.3 MiB/s wr, 62 op/s 2024-09-11T22:35:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:01 smithi159 ceph-mon[35534]: pgmap v406: 97 pgs: 97 active+clean; 98 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.3 MiB/s wr, 62 op/s 2024-09-11T22:35:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:01 smithi172 ceph-mon[35461]: pgmap v406: 97 pgs: 97 active+clean; 98 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.3 MiB/s wr, 62 op/s 2024-09-11T22:35:02.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 943086 7.07 MB/sec execute 560 sec latency 28.187 ms 2024-09-11T22:35:03.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 945214 7.08 MB/sec execute 561 sec latency 15.370 ms 2024-09-11T22:35:03.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:03 smithi005 ceph-mon[30943]: pgmap v407: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.3 MiB/s wr, 147 op/s 2024-09-11T22:35:03.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:03 smithi159 ceph-mon[35534]: pgmap v407: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.3 MiB/s wr, 147 op/s 2024-09-11T22:35:03.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:03 smithi172 ceph-mon[35461]: pgmap v407: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 3.3 MiB/s wr, 147 op/s 2024-09-11T22:35:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.972+0000 7fd29b623640 1 -- 172.21.15.5:0/2633943328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd294101350 msgr2=0x7fd29410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.972+0000 7fd29b623640 1 --2- 172.21.15.5:0/2633943328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd294101350 0x7fd29410d880 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009cd0 tx=0x7fd28c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.972+0000 7fd29b623640 1 -- 172.21.15.5:0/2633943328 shutdown_connections 2024-09-11T22:35:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.972+0000 7fd29b623640 1 --2- 172.21.15.5:0/2633943328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd294101350 0x7fd29410d880 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.972+0000 7fd29b623640 1 --2- 172.21.15.5:0/2633943328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2941009b0 0x7fd294100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.972+0000 7fd29b623640 1 --2- 172.21.15.5:0/2633943328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2941069c0 0x7fd294106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.972+0000 7fd29b623640 1 -- 172.21.15.5:0/2633943328 >> 172.21.15.5:0/2633943328 conn(0x7fd294077990 msgr2=0x7fd2940fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.972+0000 7fd29b623640 1 -- 172.21.15.5:0/2633943328 shutdown_connections 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.972+0000 7fd29b623640 1 -- 172.21.15.5:0/2633943328 wait complete. 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd29b623640 1 Processor -- start 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd29b623640 1 -- start start 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd29b623640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2941009b0 0x7fd2941982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd29b623640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd294101350 0x7fd294198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd29b623640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2941069c0 0x7fd29419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd29b623640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd294110e00 con 0x7fd2941009b0 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd29b623640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd294110c80 con 0x7fd294101350 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd29b623640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd294110f80 con 0x7fd2941069c0 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd2993bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2941009b0 0x7fd2941982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd2993bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2941009b0 0x7fd2941982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51980/0 (socket says 172.21.15.5:51980) 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd2993bf640 1 -- 172.21.15.5:0/66792611 learned_addr learned my addr 172.21.15.5:0/66792611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd298bbe640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd294101350 0x7fd294198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.973+0000 7fd299bc0640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2941069c0 0x7fd29419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd2993bf640 1 -- 172.21.15.5:0/66792611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2941069c0 msgr2=0x7fd29419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd2993bf640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2941069c0 0x7fd29419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd2993bf640 1 -- 172.21.15.5:0/66792611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd294101350 msgr2=0x7fd294198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd2993bf640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd294101350 0x7fd294198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd2993bf640 1 -- 172.21.15.5:0/66792611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29419d2b0 con 0x7fd2941009b0 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd2993bf640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2941009b0 0x7fd2941982c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd29000e9f0 tx=0x7fd29000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd28a7fc640 1 -- 172.21.15.5:0/66792611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29000cd80 con 0x7fd2941009b0 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd29b623640 1 -- 172.21.15.5:0/66792611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29419d5a0 con 0x7fd2941009b0 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd28a7fc640 1 -- 172.21.15.5:0/66792611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd290005500 con 0x7fd2941009b0 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd28a7fc640 1 -- 172.21.15.5:0/66792611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290010690 con 0x7fd2941009b0 2024-09-11T22:35:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.974+0000 7fd29b623640 1 -- 172.21.15.5:0/66792611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2941a4e30 con 0x7fd2941009b0 2024-09-11T22:35:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.976+0000 7fd28a7fc640 1 -- 172.21.15.5:0/66792611 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd2900046c0 con 0x7fd2941009b0 2024-09-11T22:35:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd29e3bc740 1 Processor -- start 2024-09-11T22:35:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd29e3bc740 1 -- start start 2024-09-11T22:35:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a43ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 0x55d0e7a4e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0e79a82f0 con 0x55d0e7a4ecb0 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d0e7950270 con 0x55d0e7a44410 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d0e79a42b0 con 0x55d0e7a43af0 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd298bbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd2993bf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 0x55d0e7a4e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.977+0000 7fd298bbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51988/0 (socket says 172.21.15.5:51988) 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd298bbe640 1 -- 172.21.15.5:0/1257168891 learned_addr learned my addr 172.21.15.5:0/1257168891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd28a7fc640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2800a2da0 0x7fd2800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd299bc0640 1 --2- 172.21.15.5:0/1257168891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a43ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd299bc0640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2800a2da0 0x7fd2800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd2993bf640 1 -- 172.21.15.5:0/1257168891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 msgr2=0x55d0e7a43ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd2993bf640 1 --2- 172.21.15.5:0/1257168891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a43ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd2993bf640 1 -- 172.21.15.5:0/1257168891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 msgr2=0x55d0e7a510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd2993bf640 1 --2- 172.21.15.5:0/1257168891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd2993bf640 1 -- 172.21.15.5:0/1257168891 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e794f340 con 0x55d0e7a44410 2024-09-11T22:35:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd298bbe640 1 --2- 172.21.15.5:0/1257168891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd28a7fc640 1 -- 172.21.15.5:0/66792611 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fd2900cf8c0 con 0x7fd2941009b0 2024-09-11T22:35:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd2993bf640 1 --2- 172.21.15.5:0/1257168891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 0x55d0e7a4e770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd2900ccf40 tx=0x7fd290005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7517fc651f18a64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd299bc0640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2800a2da0 0x7fd2800a5260 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd28c004ab0 tx=0x7fd28c004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd27f7fe640 1 -- 172.21.15.5:0/1257168891 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29009b670 con 0x55d0e7a44410 2024-09-11T22:35:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd27f7fe640 1 -- 172.21.15.5:0/1257168891 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29009c410 con 0x55d0e7a44410 2024-09-11T22:35:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.978+0000 7fd27f7fe640 1 -- 172.21.15.5:0/1257168891 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2900cd1c0 con 0x55d0e7a44410 2024-09-11T22:35:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.979+0000 7fd29e3bc740 1 -- 172.21.15.5:0/1257168891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 msgr2=0x55d0e7a4e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.979+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/1257168891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 0x55d0e7a4e770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd2900ccf40 tx=0x7fd290005f10 comp rx=0 tx=0).stop 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.979+0000 7fd29e3bc740 1 -- 172.21.15.5:0/1257168891 shutdown_connections 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.979+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/1257168891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.979+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/1257168891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 0x55d0e7a4e770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.979+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/1257168891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a43ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.979+0000 7fd29e3bc740 1 -- 172.21.15.5:0/1257168891 >> 172.21.15.5:0/1257168891 conn(0x55d0e7a40060 msgr2=0x55d0e7a40440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.979+0000 7fd29e3bc740 1 -- 172.21.15.5:0/1257168891 shutdown_connections 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.979+0000 7fd29e3bc740 1 -- 172.21.15.5:0/1257168891 wait complete. 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.979+0000 7fd29e3bc740 1 -- start start 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd29e3bc740 1 Processor -- start 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a43af0 0x55d0e7a4bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a44410 0x55d0e7a4c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a4c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d0e7a51880 con 0x55d0e7a4ecb0 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d0e7931290 con 0x55d0e7a43af0 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d0e7a51a00 con 0x55d0e7a44410 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd29e3bc740 10 client.0 _ll_register_callbacks cb 0x55d0e7962140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd299bc0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a4c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd299bc0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a4c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52004/0 (socket says 172.21.15.5:52004) 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd299bc0640 1 -- 172.21.15.5:0/2867452277 learned_addr learned my addr 172.21.15.5:0/2867452277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd2993bf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a43af0 0x55d0e7a4bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd299bc0640 1 -- 172.21.15.5:0/2867452277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a44410 msgr2=0x55d0e7a4c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.980+0000 7fd298bbe640 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a44410 0x55d0e7a4c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd299bc0640 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a44410 0x55d0e7a4c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd299bc0640 1 -- 172.21.15.5:0/2867452277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a43af0 msgr2=0x55d0e7a4bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd299bc0640 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a43af0 0x55d0e7a4bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd299bc0640 1 -- 172.21.15.5:0/2867452277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e76bb210 con 0x55d0e7a4ecb0 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd2993bf640 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a43af0 0x55d0e7a4bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd298bbe640 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a44410 0x55d0e7a4c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd299bc0640 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a4c5e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd28c003bf0 tx=0x7fd28c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6240fc55a7ea5e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd27cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd27cff9640 1 -- 172.21.15.5:0/2867452277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c004640 con 0x55d0e7a4ecb0 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd27cff9640 1 -- 172.21.15.5:0/2867452277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fd28c0047e0 con 0x55d0e7a4ecb0 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2867452277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd29009c410 con 0x55d0e7a4ecb0 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd27cff9640 1 -- 172.21.15.5:0/2867452277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd28c0305a0 con 0x55d0e7a4ecb0 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd29e3bc740 10 client.19398 fetch_fsmap learned FSMap version 46 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2867452277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d0e7787620 con 0x55d0e7a4ecb0 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd27cff9640 1 -- 172.21.15.5:0/2867452277 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29009c410 con 0x55d0e7a4ecb0 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd27cff9640 1 -- 172.21.15.5:0/2867452277 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fd28c030070 con 0x55d0e7a4ecb0 2024-09-11T22:35:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd29e3bc740 10 client.19398 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:35:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.981+0000 7fd29e3bc740 10 client.19398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.982+0000 7fd29e3bc740 20 client.19398 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.982+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d0e7a5ebc0 0x55d0e7a60ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.982+0000 7fd29e3bc740 4 client.19398 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.982+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2867452277 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0e79b5a20 con 0x55d0e7a5ebc0 2024-09-11T22:35:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.982+0000 7fd2993bf640 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d0e7a5ebc0 0x55d0e7a60ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.982+0000 7fd27cff9640 10 client.19398 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.982+0000 7fd2993bf640 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d0e7a5ebc0 0x55d0e7a60ff0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4bc1454bcc8ca46 server_cookie=351a338cc1f4c787 in_seq=0 out_seq=0 2024-09-11T22:35:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd27cff9640 1 -- 172.21.15.5:0/2867452277 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0e79b5a20 con 0x55d0e7a5ebc0 2024-09-11T22:35:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd27cff9640 10 client.19398 handle_command_reply: tid=0 2024-09-11T22:35:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd29e3bc740 1 client.19398 shutdown 2024-09-11T22:35:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2867452277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 msgr2=0x55d0e7a4c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a4c5e0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fd28c003bf0 tx=0x7fd28c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2867452277 shutdown_connections 2024-09-11T22:35:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d0e7a5ebc0 0x55d0e7a60ff0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a4c5e0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd27cff9640 0 client.19398 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a44410 0x55d0e7a4c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2867452277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a43af0 0x55d0e7a4bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.984+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2867452277 >> 172.21.15.5:0/2867452277 conn(0x55d0e7a40060 msgr2=0x55d0e7a45dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.985+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2867452277 shutdown_connections 2024-09-11T22:35:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.985+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2867452277 wait complete. 2024-09-11T22:35:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.985+0000 7fd29e3bc740 20 client.19398 trim_cache size 0 max 16384 2024-09-11T22:35:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.999+0000 7fd29e3bc740 1 Processor -- start 2024-09-11T22:35:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:03.999+0000 7fd29e3bc740 1 -- start start 2024-09-11T22:35:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a4c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a44410 0x55d0e7a4dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a66900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941ada10 con 0x55d0e7a44410 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941af580 con 0x55d0e7a4ecb0 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941ae0e0 con 0x55d0e7a43af0 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd298bbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a44410 0x55d0e7a4dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd298bbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a44410 0x55d0e7a4dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52016/0 (socket says 172.21.15.5:52016) 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd299bc0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a66900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd298bbe640 1 -- 172.21.15.5:0/2855741205 learned_addr learned my addr 172.21.15.5:0/2855741205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd2993bf640 1 --2- 172.21.15.5:0/2855741205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a4c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd298bbe640 1 -- 172.21.15.5:0/2855741205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 msgr2=0x55d0e7a4c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd298bbe640 1 --2- 172.21.15.5:0/2855741205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a4c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd298bbe640 1 -- 172.21.15.5:0/2855741205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a4ecb0 msgr2=0x55d0e7a66900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd298bbe640 1 --2- 172.21.15.5:0/2855741205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a66900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd298bbe640 1 -- 172.21.15.5:0/2855741205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e7a3fe90 con 0x55d0e7a44410 2024-09-11T22:35:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd2993bf640 1 --2- 172.21.15.5:0/2855741205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a4c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd298bbe640 1 --2- 172.21.15.5:0/2855741205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a44410 0x55d0e7a4dad0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x55d0e7a46690 tx=0x7fd284010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e399c4f57a596ca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.000+0000 7fd27effd640 1 -- 172.21.15.5:0/2855741205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284020070 con 0x55d0e7a44410 2024-09-11T22:35:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd27effd640 1 -- 172.21.15.5:0/2855741205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2840035f0 con 0x55d0e7a44410 2024-09-11T22:35:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd27effd640 1 -- 172.21.15.5:0/2855741205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28401b660 con 0x55d0e7a44410 2024-09-11T22:35:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2855741205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a44410 msgr2=0x55d0e7a4dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2855741205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a44410 0x55d0e7a4dad0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x55d0e7a46690 tx=0x7fd284010800 comp rx=0 tx=0).stop 2024-09-11T22:35:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2855741205 shutdown_connections 2024-09-11T22:35:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2855741205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a66900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2855741205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a44410 0x55d0e7a4dad0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2855741205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a4c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2855741205 >> 172.21.15.5:0/2855741205 conn(0x55d0e7a40060 msgr2=0x55d0e7a4d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2855741205 shutdown_connections 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.001+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2855741205 wait complete. 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd29e3bc740 1 -- start start 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd29e3bc740 1 Processor -- start 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a57090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 0x55d0e7a48570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd29e3bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a48ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941aeeb0 con 0x55d0e7a4ecb0 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941ae7e0 con 0x55d0e7a44410 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd29e3bc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd2941aba10 con 0x55d0e7a43af0 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd29e3bc740 10 client.0 _ll_register_callbacks cb 0x7fd290005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd299bc0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a48ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd299bc0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a48ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52026/0 (socket says 172.21.15.5:52026) 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd298bbe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 0x55d0e7a48570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd299bc0640 1 -- 172.21.15.5:0/2143186309 learned_addr learned my addr 172.21.15.5:0/2143186309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.002+0000 7fd2993bf640 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a57090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd299bc0640 1 -- 172.21.15.5:0/2143186309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 msgr2=0x55d0e7a57090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd299bc0640 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a57090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd299bc0640 1 -- 172.21.15.5:0/2143186309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 msgr2=0x55d0e7a48570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd299bc0640 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 0x55d0e7a48570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd299bc0640 1 -- 172.21.15.5:0/2143186309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e7a48ff0 con 0x55d0e7a4ecb0 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd299bc0640 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a48ab0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0528f0 tx=0x7fd28c009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79209c26b38b6ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd27cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd27cff9640 1 -- 172.21.15.5:0/2143186309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c004410 con 0x55d0e7a4ecb0 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd27cff9640 1 -- 172.21.15.5:0/2143186309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8629+0+0 (secure 0 0 0) 0x7fd28c0045b0 con 0x55d0e7a4ecb0 2024-09-11T22:35:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2143186309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2840035f0 con 0x55d0e7a4ecb0 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd27cff9640 1 -- 172.21.15.5:0/2143186309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fd28c067050 con 0x55d0e7a4ecb0 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd29e3bc740 10 client.19404 fetch_fsmap learned FSMap version 46 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2143186309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55d0e7ac95b0 con 0x55d0e7a4ecb0 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.003+0000 7fd27cff9640 1 -- 172.21.15.5:0/2143186309 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2840035f0 con 0x55d0e7a4ecb0 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.004+0000 7fd27cff9640 1 -- 172.21.15.5:0/2143186309 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7fd28c005140 con 0x55d0e7a4ecb0 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.004+0000 7fd29e3bc740 10 client.19404 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.004+0000 7fd29e3bc740 10 client.19404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.004+0000 7fd29e3bc740 20 client.19404 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.004+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d0e7ac9cc0 0x55d0e7acc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.004+0000 7fd29e3bc740 4 client.19404 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.004+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2143186309 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0e79afd70 con 0x55d0e7ac9cc0 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.004+0000 7fd2993bf640 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d0e7ac9cc0 0x55d0e7acc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:04.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.004+0000 7fd27cff9640 10 client.19404 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.004+0000 7fd2993bf640 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d0e7ac9cc0 0x55d0e7acc0e0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471313284e525110 server_cookie=e07af52ccf045cc2 in_seq=0 out_seq=0 2024-09-11T22:35:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd27cff9640 1 -- 172.21.15.5:0/2143186309 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0e79afd70 con 0x55d0e7ac9cc0 2024-09-11T22:35:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd27cff9640 10 client.19404 handle_command_reply: tid=0 2024-09-11T22:35:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 client.19404 shutdown 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2143186309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 msgr2=0x55d0e7a48ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a48ab0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0528f0 tx=0x7fd28c009c60 comp rx=0 tx=0).stop 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2143186309 shutdown_connections 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d0e7ac9cc0 0x55d0e7acc0e0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0e7a4ecb0 0x55d0e7a48ab0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d0e7a44410 0x55d0e7a48570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 --2- 172.21.15.5:0/2143186309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d0e7a43af0 0x55d0e7a57090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2143186309 >> 172.21.15.5:0/2143186309 conn(0x55d0e7a40060 msgr2=0x55d0e7a3f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd27cff9640 0 client.19404 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2143186309 shutdown_connections 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 1 -- 172.21.15.5:0/2143186309 wait complete. 2024-09-11T22:35:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.109+0000 7fd29e3bc740 20 client.19404 trim_cache size 0 max 16384 2024-09-11T22:35:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:35:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:35:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:35:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 -- 172.21.15.5:0/66792611 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2800a2da0 msgr2=0x7fd2800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2800a2da0 0x7fd2800a5260 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd28c004ab0 tx=0x7fd28c004a40 comp rx=0 tx=0).stop 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 -- 172.21.15.5:0/66792611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2941009b0 msgr2=0x7fd2941982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2941009b0 0x7fd2941982c0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fd29000e9f0 tx=0x7fd29000edf0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 -- 172.21.15.5:0/66792611 shutdown_connections 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2941069c0 0x7fd29419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd2800a2da0 0x7fd2800a5260 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd294101350 0x7fd294198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 --2- 172.21.15.5:0/66792611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2941009b0 0x7fd2941982c0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 -- 172.21.15.5:0/66792611 >> 172.21.15.5:0/66792611 conn(0x7fd294077990 msgr2=0x7fd2940ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:04.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.110+0000 7fd29b623640 1 -- 172.21.15.5:0/66792611 shutdown_connections 2024-09-11T22:35:04.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:04.111+0000 7fd29b623640 1 -- 172.21.15.5:0/66792611 wait complete. 2024-09-11T22:35:04.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 946362 7.08 MB/sec execute 562 sec latency 71.727 ms 2024-09-11T22:35:05.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:35:04 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setuser ceph since I am not root 2024-09-11T22:35:05.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:35:04 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setgroup ceph since I am not root 2024-09-11T22:35:05.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:35:04 smithi159 ceph-mds[65034]: -- 172.21.15.159:0/1691580461 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5607dd298680 con 0x5607dd26c800 2024-09-11T22:35:05.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:35:04 smithi159 ceph-mds[65034]: -- 172.21.15.159:0/1691580461 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607dd28f860 con 0x5607dd26c800 2024-09-11T22:35:05.079 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:35:04 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: starting mds.k at 2024-09-11T22:35:05.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 947523 7.07 MB/sec execute 563 sec latency 58.057 ms 2024-09-11T22:35:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:05 smithi159 ceph-mon[35534]: pgmap v408: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-09-11T22:35:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:05 smithi159 ceph-mon[35534]: osdmap e105: 12 total, 12 up, 12 in 2024-09-11T22:35:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:05 smithi159 ceph-mon[35534]: daemon mds.k restarted 2024-09-11T22:35:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:05 smithi172 ceph-mon[35461]: pgmap v408: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-09-11T22:35:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:05 smithi172 ceph-mon[35461]: osdmap e105: 12 total, 12 up, 12 in 2024-09-11T22:35:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:05 smithi172 ceph-mon[35461]: daemon mds.k restarted 2024-09-11T22:35:06.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:35:06.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:05 smithi005 ceph-mon[30943]: pgmap v408: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 2.8 MiB/s wr, 113 op/s 2024-09-11T22:35:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:05 smithi005 ceph-mon[30943]: osdmap e105: 12 total, 12 up, 12 in 2024-09-11T22:35:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:05 smithi005 ceph-mon[30943]: daemon mds.k restarted 2024-09-11T22:35:06.328 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:06.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 948789 7.08 MB/sec execute 564 sec latency 21.451 ms 2024-09-11T22:35:06.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:06 smithi005 ceph-mon[30943]: osdmap e106: 12 total, 12 up, 12 in 2024-09-11T22:35:06.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:06 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6832/4216563934,v1:172.21.15.159:6833/4216563934] up:boot 2024-09-11T22:35:06.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:06 smithi005 ceph-mon[30943]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:35:06.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:06 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:35:06.772 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:06 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:35:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:06 smithi159 ceph-mon[35534]: osdmap e106: 12 total, 12 up, 12 in 2024-09-11T22:35:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:06 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6832/4216563934,v1:172.21.15.159:6833/4216563934] up:boot 2024-09-11T22:35:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:06 smithi159 ceph-mon[35534]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:35:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:06 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:35:06.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:06 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:35:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:06 smithi172 ceph-mon[35461]: osdmap e106: 12 total, 12 up, 12 in 2024-09-11T22:35:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:06 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6832/4216563934,v1:172.21.15.159:6833/4216563934] up:boot 2024-09-11T22:35:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:06 smithi172 ceph-mon[35461]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-11T22:35:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:06 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:35:06.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:06 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:35:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.867+0000 7fdc65b6f640 1 -- 172.21.15.5:0/2006768306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc60100e20 msgr2=0x7fdc60101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.867+0000 7fdc65b6f640 1 --2- 172.21.15.5:0/2006768306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc60100e20 0x7fdc60101200 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdc50009900 tx=0x7fdc5002a480 comp rx=0 tx=0).stop 2024-09-11T22:35:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.867+0000 7fdc65b6f640 1 -- 172.21.15.5:0/2006768306 shutdown_connections 2024-09-11T22:35:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.867+0000 7fdc65b6f640 1 --2- 172.21.15.5:0/2006768306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60102ef0 0x7fdc6010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.867+0000 7fdc65b6f640 1 --2- 172.21.15.5:0/2006768306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60102570 0x7fdc601029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.867+0000 7fdc65b6f640 1 --2- 172.21.15.5:0/2006768306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc60100e20 0x7fdc60101200 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.867+0000 7fdc65b6f640 1 -- 172.21.15.5:0/2006768306 >> 172.21.15.5:0/2006768306 conn(0x7fdc600fc7d0 msgr2=0x7fdc600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.867+0000 7fdc65b6f640 1 -- 172.21.15.5:0/2006768306 shutdown_connections 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.867+0000 7fdc65b6f640 1 -- 172.21.15.5:0/2006768306 wait complete. 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc65b6f640 1 Processor -- start 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc65b6f640 1 -- start start 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc65b6f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60100e20 0x7fdc60103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc65b6f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60102570 0x7fdc60103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc65b6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc60102ef0 0x7fdc601041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc65b6f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60110f80 con 0x7fdc60102ef0 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc65b6f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60110e00 con 0x7fdc60102570 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc65b6f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60111100 con 0x7fdc60100e20 2024-09-11T22:35:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc60102ef0 0x7fdc601041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc56ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60102570 0x7fdc60103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc60102ef0 0x7fdc601041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52052/0 (socket says 172.21.15.5:52052) 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60100e20 0x7fdc60103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.868+0000 7fdc5ffff640 1 -- 172.21.15.5:0/293504703 learned_addr learned my addr 172.21.15.5:0/293504703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc5ffff640 1 -- 172.21.15.5:0/293504703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60100e20 msgr2=0x7fdc60103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc5ffff640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60100e20 0x7fdc60103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc5ffff640 1 -- 172.21.15.5:0/293504703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60102570 msgr2=0x7fdc60103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc5ffff640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60102570 0x7fdc60103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc5ffff640 1 -- 172.21.15.5:0/293504703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc601a69c0 con 0x7fdc60102ef0 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60102570 0x7fdc60103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc5ffff640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc60102ef0 0x7fdc601041d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800eb10 tx=0x7fdc5800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc5d7fa640 1 -- 172.21.15.5:0/293504703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58004b20 con 0x7fdc60102ef0 2024-09-11T22:35:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc65b6f640 1 -- 172.21.15.5:0/293504703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc601a6cb0 con 0x7fdc60102ef0 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc5d7fa640 1 -- 172.21.15.5:0/293504703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc58004cc0 con 0x7fdc60102ef0 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc5d7fa640 1 -- 172.21.15.5:0/293504703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58010760 con 0x7fdc60102ef0 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.869+0000 7fdc65b6f640 1 -- 172.21.15.5:0/293504703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601a71f0 con 0x7fdc60102ef0 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.872+0000 7fdc67f48740 1 Processor -- start 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc5d7fa640 1 -- 172.21.15.5:0/293504703 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fdc58005020 con 0x7fdc60102ef0 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc67f48740 1 -- start start 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e972624a0 0x560e97262880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 0x560e9726d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e9726d680 0x560e9726fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc67f48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560e971c82e0 con 0x560e972624a0 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc67f48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560e9716fce0 con 0x560e9726d680 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc67f48740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560e971c42a0 con 0x560e97262dc0 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 0x560e9726d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 0x560e9726d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46810/0 (socket says 172.21.15.5:46810) 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/2814769299 learned_addr learned my addr 172.21.15.5:0/2814769299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/2814769299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e972624a0 0x560e97262880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.873+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/2814769299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e9726d680 0x560e9726fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/2814769299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e9726d680 msgr2=0x560e9726fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/2814769299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e9726d680 0x560e9726fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/2814769299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e972624a0 msgr2=0x560e97262880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/2814769299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e972624a0 0x560e97262880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/2814769299 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e97154440 con 0x560e97262dc0 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/2814769299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e972624a0 0x560e97262880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/2814769299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 0x560e9726d140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc5002a9a0 tx=0x7fdc50005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69cceeca7371e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc5d7fa640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdc4c0a2cf0 0x7fdc4c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc55ffb640 1 -- 172.21.15.5:0/2814769299 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50035030 con 0x560e97262dc0 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdc4c0a2cf0 0x7fdc4c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc5d7fa640 1 -- 172.21.15.5:0/293504703 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7fdc580cf9b0 con 0x7fdc60102ef0 2024-09-11T22:35:06.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc55ffb640 1 -- 172.21.15.5:0/2814769299 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc50040070 con 0x560e97262dc0 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.874+0000 7fdc55ffb640 1 -- 172.21.15.5:0/2814769299 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500318a0 con 0x560e97262dc0 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.875+0000 7fdc67f48740 1 -- 172.21.15.5:0/2814769299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 msgr2=0x560e9726d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.875+0000 7fdc67f48740 1 --2- 172.21.15.5:0/2814769299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 0x560e9726d140 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc5002a9a0 tx=0x7fdc50005dd0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.875+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdc4c0a2cf0 0x7fdc4c0a51b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdc480058c0 tx=0x7fdc48005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.875+0000 7fdc67f48740 1 -- 172.21.15.5:0/2814769299 shutdown_connections 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.875+0000 7fdc67f48740 1 --2- 172.21.15.5:0/2814769299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e9726d680 0x560e9726fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.875+0000 7fdc67f48740 1 --2- 172.21.15.5:0/2814769299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 0x560e9726d140 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.875+0000 7fdc67f48740 1 --2- 172.21.15.5:0/2814769299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e972624a0 0x560e97262880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.875+0000 7fdc67f48740 1 -- 172.21.15.5:0/2814769299 >> 172.21.15.5:0/2814769299 conn(0x560e9725ea30 msgr2=0x560e9725ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.875+0000 7fdc67f48740 1 -- 172.21.15.5:0/2814769299 shutdown_connections 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.875+0000 7fdc67f48740 1 -- 172.21.15.5:0/2814769299 wait complete. 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.876+0000 7fdc67f48740 1 -- start start 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.876+0000 7fdc67f48740 1 Processor -- start 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e9726cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 0x560e9726a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 0x560e9726a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc67f48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560e97270250 con 0x560e9726d680 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc67f48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560e97183050 con 0x560e972624a0 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc67f48740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560e972703d0 con 0x560e97262dc0 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e9726cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 0x560e9726a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e9726cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38266/0 (socket says 172.21.15.5:38266) 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc56ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 0x560e9726a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1163830669 learned_addr learned my addr 172.21.15.5:0/1163830669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc67f48740 10 client.0 _ll_register_callbacks cb 0x560e971c3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1163830669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 msgr2=0x560e9726a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 0x560e9726a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1163830669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 msgr2=0x560e9726a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:06.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 0x560e9726a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/1163830669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e96f393c0 con 0x560e972624a0 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5ffff640 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 0x560e9726a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e9726cf60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc5002a4c0 tx=0x7fdc50004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39b82311692cd610 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.877+0000 7fdc3effd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.878+0000 7fdc3effd640 1 -- 172.21.15.5:0/1163830669 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5004e060 con 0x560e972624a0 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.878+0000 7fdc3effd640 1 -- 172.21.15.5:0/1163830669 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7fdc50005440 con 0x560e972624a0 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.878+0000 7fdc67f48740 1 -- 172.21.15.5:0/1163830669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc50040070 con 0x560e972624a0 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.878+0000 7fdc3effd640 1 -- 172.21.15.5:0/1163830669 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdc50035070 con 0x560e972624a0 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.878+0000 7fdc67f48740 10 client.28613 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.878+0000 7fdc67f48740 1 -- 172.21.15.5:0/1163830669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560e9727bc00 con 0x560e972624a0 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.878+0000 7fdc3effd640 1 -- 172.21.15.5:0/1163830669 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc50040070 con 0x560e972624a0 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.878+0000 7fdc3effd640 1 -- 172.21.15.5:0/1163830669 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fdc500039c0 con 0x560e972624a0 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.878+0000 7fdc67f48740 10 client.28613 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:06.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.878+0000 7fdc67f48740 10 client.28613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.879+0000 7fdc67f48740 20 client.28613 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.879+0000 7fdc67f48740 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560e9727cf50 0x560e9727f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.879+0000 7fdc67f48740 4 client.28613 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.879+0000 7fdc67f48740 1 -- 172.21.15.5:0/1163830669 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e971d5850 con 0x560e9727cf50 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.879+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560e9727cf50 0x560e9727f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.879+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560e9727cf50 0x560e9727f340 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e6e8a7b5cd19b11 server_cookie=9553c63088a4803c in_seq=0 out_seq=0 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.879+0000 7fdc3effd640 10 client.28613 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.880+0000 7fdc3effd640 1 -- 172.21.15.5:0/1163830669 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e971d5850 con 0x560e9727cf50 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.880+0000 7fdc3effd640 10 client.28613 handle_command_reply: tid=0 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.880+0000 7fdc67f48740 1 client.28613 shutdown 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 1 -- 172.21.15.5:0/1163830669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 msgr2=0x560e9726cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e9726cf60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc5002a4c0 tx=0x7fdc50004430 comp rx=0 tx=0).stop 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 1 -- 172.21.15.5:0/1163830669 shutdown_connections 2024-09-11T22:35:06.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560e9727cf50 0x560e9727f340 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 0x560e9726a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e97262dc0 0x560e9726a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 1 --2- 172.21.15.5:0/1163830669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e9726cf60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 1 -- 172.21.15.5:0/1163830669 >> 172.21.15.5:0/1163830669 conn(0x560e9725ea30 msgr2=0x560e9725db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:06.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc3effd640 0 client.28613 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:06.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 1 -- 172.21.15.5:0/1163830669 shutdown_connections 2024-09-11T22:35:06.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 1 -- 172.21.15.5:0/1163830669 wait complete. 2024-09-11T22:35:06.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.881+0000 7fdc67f48740 20 client.28613 trim_cache size 0 max 16384 2024-09-11T22:35:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.900+0000 7fdc67f48740 1 Processor -- start 2024-09-11T22:35:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.900+0000 7fdc67f48740 1 -- start start 2024-09-11T22:35:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e97265900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e97262dc0 0x560e972691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e9726d680 0x560e97281f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc67f48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601afd30 con 0x560e97262dc0 2024-09-11T22:35:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc67f48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601b18a0 con 0x560e972624a0 2024-09-11T22:35:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc67f48740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601b0400 con 0x560e9726d680 2024-09-11T22:35:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc56ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e97262dc0 0x560e972691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc56ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e97262dc0 0x560e972691f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52084/0 (socket says 172.21.15.5:52084) 2024-09-11T22:35:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc56ffd640 1 -- 172.21.15.5:0/207391384 learned_addr learned my addr 172.21.15.5:0/207391384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/207391384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e97265900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc5ffff640 1 --2- 172.21.15.5:0/207391384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e9726d680 0x560e97281f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc56ffd640 1 -- 172.21.15.5:0/207391384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e9726d680 msgr2=0x560e97281f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/207391384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e9726d680 0x560e97281f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc56ffd640 1 -- 172.21.15.5:0/207391384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 msgr2=0x560e97265900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/207391384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e97265900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc56ffd640 1 -- 172.21.15.5:0/207391384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e9725f570 con 0x560e97262dc0 2024-09-11T22:35:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/207391384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e97265900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/207391384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e97262dc0 0x560e972691f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009fd0 tx=0x7fdc48019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0793fd58edee72 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.901+0000 7fdc557fa640 1 -- 172.21.15.5:0/207391384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48005220 con 0x560e97262dc0 2024-09-11T22:35:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc557fa640 1 -- 172.21.15.5:0/207391384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc48004c80 con 0x560e97262dc0 2024-09-11T22:35:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc557fa640 1 -- 172.21.15.5:0/207391384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48020b00 con 0x560e97262dc0 2024-09-11T22:35:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc67f48740 1 -- 172.21.15.5:0/207391384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e97262dc0 msgr2=0x560e972691f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc67f48740 1 --2- 172.21.15.5:0/207391384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e97262dc0 0x560e972691f0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009fd0 tx=0x7fdc48019000 comp rx=0 tx=0).stop 2024-09-11T22:35:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc67f48740 1 -- 172.21.15.5:0/207391384 shutdown_connections 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc67f48740 1 --2- 172.21.15.5:0/207391384 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e9726d680 0x560e97281f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc67f48740 1 --2- 172.21.15.5:0/207391384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e97262dc0 0x560e972691f0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc67f48740 1 --2- 172.21.15.5:0/207391384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e972624a0 0x560e97265900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc67f48740 1 -- 172.21.15.5:0/207391384 >> 172.21.15.5:0/207391384 conn(0x560e9725ea30 msgr2=0x560e97268ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc67f48740 1 -- 172.21.15.5:0/207391384 shutdown_connections 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.902+0000 7fdc67f48740 1 -- 172.21.15.5:0/207391384 wait complete. 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc67f48740 1 -- start start 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc67f48740 1 Processor -- start 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e972624a0 0x560e9727f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e97262dc0 0x560e9727fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc67f48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 0x560e972df850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc67f48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601b11d0 con 0x560e9726d680 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc67f48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601b0b00 con 0x560e97262dc0 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc67f48740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601add10 con 0x560e972624a0 2024-09-11T22:35:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc67f48740 10 client.0 _ll_register_callbacks cb 0x7fdc50009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e972624a0 0x560e9727f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc56ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e97262dc0 0x560e9727fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 0x560e972df850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e972624a0 0x560e9727f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46844/0 (socket says 172.21.15.5:46844) 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.903+0000 7fdc5f7fe640 1 -- 172.21.15.5:0/2134946485 learned_addr learned my addr 172.21.15.5:0/2134946485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc56ffd640 1 -- 172.21.15.5:0/2134946485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e972624a0 msgr2=0x560e9727f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e972624a0 0x560e9727f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc56ffd640 1 -- 172.21.15.5:0/2134946485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 msgr2=0x560e972df850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 0x560e972df850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc56ffd640 1 -- 172.21.15.5:0/2134946485 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e972dfef0 con 0x560e97262dc0 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc5ffff640 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 0x560e972df850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc56ffd640 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e97262dc0 0x560e9727fbc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x560e97270580 tx=0x7fdc48005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68cca75184c8c5cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc3effd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc3effd640 1 -- 172.21.15.5:0/2134946485 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48039030 con 0x560e97262dc0 2024-09-11T22:35:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc67f48740 1 -- 172.21.15.5:0/2134946485 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc48004c80 con 0x560e97262dc0 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc3effd640 1 -- 172.21.15.5:0/2134946485 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7fdc48010d90 con 0x560e97262dc0 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc3effd640 1 -- 172.21.15.5:0/2134946485 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdc480046c0 con 0x560e97262dc0 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc67f48740 10 client.28619 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.904+0000 7fdc67f48740 1 -- 172.21.15.5:0/2134946485 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560e972e72b0 con 0x560e97262dc0 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc3effd640 1 -- 172.21.15.5:0/2134946485 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc48004c80 con 0x560e97262dc0 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc3effd640 1 -- 172.21.15.5:0/2134946485 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fdc480209b0 con 0x560e97262dc0 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc67f48740 10 client.28619 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc67f48740 10 client.28619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc67f48740 20 client.28619 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc67f48740 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560e972e79a0 0x560e972e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc67f48740 4 client.28619 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc67f48740 1 -- 172.21.15.5:0/2134946485 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e9726aae0 con 0x560e972e79a0 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560e972e79a0 0x560e972e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc3effd640 10 client.28619 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.905+0000 7fdc5f7fe640 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560e972e79a0 0x560e972e9dc0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e617cdceb13f22b server_cookie=7d2f0e2343d8096a in_seq=0 out_seq=0 2024-09-11T22:35:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc3effd640 1 -- 172.21.15.5:0/2134946485 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e9726aae0 con 0x560e972e79a0 2024-09-11T22:35:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc3effd640 10 client.28619 handle_command_reply: tid=0 2024-09-11T22:35:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 client.28619 shutdown 2024-09-11T22:35:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 -- 172.21.15.5:0/2134946485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e97262dc0 msgr2=0x560e9727fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e97262dc0 0x560e9727fbc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x560e97270580 tx=0x7fdc48005fd0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 -- 172.21.15.5:0/2134946485 shutdown_connections 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560e972e79a0 0x560e972e9dc0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e9726d680 0x560e972df850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e97262dc0 0x560e9727fbc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc3effd640 0 client.28619 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 --2- 172.21.15.5:0/2134946485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560e972624a0 0x560e9727f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 -- 172.21.15.5:0/2134946485 >> 172.21.15.5:0/2134946485 conn(0x560e9725ea30 msgr2=0x560e9725d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 -- 172.21.15.5:0/2134946485 shutdown_connections 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 1 -- 172.21.15.5:0/2134946485 wait complete. 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.906+0000 7fdc67f48740 20 client.28619 trim_cache size 0 max 16384 2024-09-11T22:35:06.911 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:35:06.912 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:35:06.912 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:35:06.912 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:35:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 -- 172.21.15.5:0/293504703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdc4c0a2cf0 msgr2=0x7fdc4c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdc4c0a2cf0 0x7fdc4c0a51b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdc480058c0 tx=0x7fdc48005850 comp rx=0 tx=0).stop 2024-09-11T22:35:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 -- 172.21.15.5:0/293504703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc60102ef0 msgr2=0x7fdc601041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc60102ef0 0x7fdc601041d0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800eb10 tx=0x7fdc5800c2d0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 -- 172.21.15.5:0/293504703 shutdown_connections 2024-09-11T22:35:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdc60102ef0 0x7fdc601041d0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdc4c0a2cf0 0x7fdc4c0a51b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdc60102570 0x7fdc60103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 --2- 172.21.15.5:0/293504703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc60100e20 0x7fdc60103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 -- 172.21.15.5:0/293504703 >> 172.21.15.5:0/293504703 conn(0x7fdc600fc7d0 msgr2=0x7fdc6006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 -- 172.21.15.5:0/293504703 shutdown_connections 2024-09-11T22:35:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:06.907+0000 7fdc65b6f640 1 -- 172.21.15.5:0/293504703 wait complete. 2024-09-11T22:35:07.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 950356 7.08 MB/sec execute 565 sec latency 50.581 ms 2024-09-11T22:35:07.663 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:35:07.663 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:35:07.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:07 smithi005 ceph-mon[30943]: pgmap v411: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-09-11T22:35:07.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:07 smithi172 ceph-mon[35461]: pgmap v411: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-09-11T22:35:07.976 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:07 smithi159 ceph-mon[35534]: pgmap v411: 97 pgs: 97 active+clean; 97 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-09-11T22:35:08.348 INFO:tasks.workunit.client.0.smithi005.stdout: 1 952360 7.09 MB/sec execute 566 sec latency 49.172 ms 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.430+0000 7fd0492e7640 1 -- 172.21.15.5:0/2960952319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0441037a0 msgr2=0x7fd04410bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.430+0000 7fd0492e7640 1 --2- 172.21.15.5:0/2960952319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0441037a0 0x7fd04410bc00 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009900 tx=0x7fd02c02a410 comp rx=0 tx=0).stop 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.430+0000 7fd0492e7640 1 -- 172.21.15.5:0/2960952319 shutdown_connections 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.430+0000 7fd0492e7640 1 --2- 172.21.15.5:0/2960952319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd04410c2f0 0x7fd04410e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.430+0000 7fd0492e7640 1 --2- 172.21.15.5:0/2960952319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0441037a0 0x7fd04410bc00 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.430+0000 7fd0492e7640 1 --2- 172.21.15.5:0/2960952319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044102df0 0x7fd0441031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.430+0000 7fd0492e7640 1 -- 172.21.15.5:0/2960952319 >> 172.21.15.5:0/2960952319 conn(0x7fd0440fc7d0 msgr2=0x7fd0440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.430+0000 7fd0492e7640 1 -- 172.21.15.5:0/2960952319 shutdown_connections 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.430+0000 7fd0492e7640 1 -- 172.21.15.5:0/2960952319 wait complete. 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0492e7640 1 Processor -- start 2024-09-11T22:35:08.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0492e7640 1 -- start start 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0492e7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044102df0 0x7fd044198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0492e7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0441037a0 0x7fd044198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0492e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04410c2f0 0x7fd04419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0492e7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0441120b0 con 0x7fd04410c2f0 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0492e7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd044111f30 con 0x7fd0441037a0 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0492e7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd044112230 con 0x7fd044102df0 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04410c2f0 0x7fd04419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0427fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0441037a0 0x7fd044198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04410c2f0 0x7fd04419cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52098/0 (socket says 172.21.15.5:52098) 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0437fe640 1 -- 172.21.15.5:0/3726549503 learned_addr learned my addr 172.21.15.5:0/3726549503 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd042ffd640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044102df0 0x7fd044198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0437fe640 1 -- 172.21.15.5:0/3726549503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044102df0 msgr2=0x7fd044198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0437fe640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044102df0 0x7fd044198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:08.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0437fe640 1 -- 172.21.15.5:0/3726549503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0441037a0 msgr2=0x7fd044198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.431+0000 7fd0437fe640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0441037a0 0x7fd044198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.432+0000 7fd0437fe640 1 -- 172.21.15.5:0/3726549503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04419d5e0 con 0x7fd04410c2f0 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.432+0000 7fd042ffd640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044102df0 0x7fd044198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.432+0000 7fd0437fe640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04410c2f0 0x7fd04419cf00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00e970 tx=0x7fd03c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.432+0000 7fd03bfff640 1 -- 172.21.15.5:0/3726549503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c00ccc0 con 0x7fd04410c2f0 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.432+0000 7fd0492e7640 1 -- 172.21.15.5:0/3726549503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04419d8d0 con 0x7fd04410c2f0 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.432+0000 7fd03bfff640 1 -- 172.21.15.5:0/3726549503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd03c00ce60 con 0x7fd04410c2f0 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.432+0000 7fd03bfff640 1 -- 172.21.15.5:0/3726549503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c010490 con 0x7fd04410c2f0 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.432+0000 7fd0492e7640 1 -- 172.21.15.5:0/3726549503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0441a51b0 con 0x7fd04410c2f0 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.433+0000 7fd0492e7640 1 -- 172.21.15.5:0/3726549503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0441031d0 con 0x7fd04410c2f0 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.434+0000 7fd03bfff640 1 -- 172.21.15.5:0/3726549503 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd03c005020 con 0x7fd04410c2f0 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.435+0000 7fd03bfff640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0300a2cf0 0x7fd0300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.435+0000 7fd042ffd640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0300a2cf0 0x7fd0300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:08.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.435+0000 7fd03bfff640 1 -- 172.21.15.5:0/3726549503 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7fd03c0cf9f0 con 0x7fd04410c2f0 2024-09-11T22:35:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.435+0000 7fd042ffd640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0300a2cf0 0x7fd0300a51b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd034005fd0 tx=0x7fd034009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:08.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.440+0000 7fd03bfff640 1 -- 172.21.15.5:0/3726549503 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd03c09c540 con 0x7fd04410c2f0 2024-09-11T22:35:08.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.663+0000 7fd0492e7640 1 -- 172.21.15.5:0/3726549503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0441072c0 con 0x7fd04410c2f0 2024-09-11T22:35:08.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.664+0000 7fd03bfff640 1 -- 172.21.15.5:0/3726549503 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11968 (secure 0 0 0) 0x7fd03c09be40 con 0x7fd04410c2f0 2024-09-11T22:35:08.665 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:35:08.665 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":48,"btime":"2024-09-11T22:35:05:588347+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:35:05.588328+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:35:08.665 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 48 2024-09-11T22:35:08.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 -- 172.21.15.5:0/3726549503 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0300a2cf0 msgr2=0x7fd0300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:08.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0300a2cf0 0x7fd0300a51b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd034005fd0 tx=0x7fd034009500 comp rx=0 tx=0).stop 2024-09-11T22:35:08.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 -- 172.21.15.5:0/3726549503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04410c2f0 msgr2=0x7fd04419cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:08.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04410c2f0 0x7fd04419cf00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00e970 tx=0x7fd03c00ed70 comp rx=0 tx=0).stop 2024-09-11T22:35:08.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 -- 172.21.15.5:0/3726549503 shutdown_connections 2024-09-11T22:35:08.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04410c2f0 0x7fd04419cf00 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:08.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0300a2cf0 0x7fd0300a51b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:08.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0441037a0 0x7fd044198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:08.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 --2- 172.21.15.5:0/3726549503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044102df0 0x7fd044198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:08.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 -- 172.21.15.5:0/3726549503 >> 172.21.15.5:0/3726549503 conn(0x7fd0440fc7d0 msgr2=0x7fd0440fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:08.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 -- 172.21.15.5:0/3726549503 shutdown_connections 2024-09-11T22:35:08.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:08.666+0000 7fd0492e7640 1 -- 172.21.15.5:0/3726549503 wait complete. 2024-09-11T22:35:09.349 INFO:tasks.workunit.client.0.smithi005.stdout: 1 953900 7.09 MB/sec execute 567 sec latency 18.208 ms 2024-09-11T22:35:09.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:35:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:09 smithi172 ceph-mon[35461]: pgmap v412: 97 pgs: 97 active+clean; 94 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 309 op/s 2024-09-11T22:35:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:09 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3726549503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:35:09.882 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:10.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:09 smithi005 ceph-mon[30943]: pgmap v412: 97 pgs: 97 active+clean; 94 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 309 op/s 2024-09-11T22:35:10.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:09 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3726549503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:35:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:09 smithi159 ceph-mon[35534]: pgmap v412: 97 pgs: 97 active+clean; 94 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 309 op/s 2024-09-11T22:35:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:09 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3726549503' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:35:10.349 INFO:tasks.workunit.client.0.smithi005.stdout: 1 955858 7.10 MB/sec execute 568 sec latency 51.347 ms 2024-09-11T22:35:10.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 -- 172.21.15.5:0/813374678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8db0100a80 msgr2=0x7f8db0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 --2- 172.21.15.5:0/813374678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8db0100a80 0x7f8db0100ee0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009930 tx=0x7f8da402a420 comp rx=0 tx=0).stop 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 -- 172.21.15.5:0/813374678 shutdown_connections 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 --2- 172.21.15.5:0/813374678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db0101420 0x7f8db010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 --2- 172.21.15.5:0/813374678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8db0100a80 0x7f8db0100ee0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 --2- 172.21.15.5:0/813374678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db0106a80 0x7f8db0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 -- 172.21.15.5:0/813374678 >> 172.21.15.5:0/813374678 conn(0x7f8db00fc7a0 msgr2=0x7f8db00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 -- 172.21.15.5:0/813374678 shutdown_connections 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 -- 172.21.15.5:0/813374678 wait complete. 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 Processor -- start 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.594+0000 7f8db6540640 1 -- start start 2024-09-11T22:35:10.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8db6540640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db0100a80 0x7f8db00fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8db6540640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8db0101420 0x7f8db00fd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8db6540640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db0106a80 0x7f8db00fd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8db6540640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8db0110e40 con 0x7f8db0101420 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8db6540640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8db0110cc0 con 0x7f8db0106a80 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8db6540640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8db0110fc0 con 0x7f8db0100a80 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8daf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8db0101420 0x7f8db00fd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8daf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8db0101420 0x7f8db00fd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52118/0 (socket says 172.21.15.5:52118) 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8daf7fe640 1 -- 172.21.15.5:0/743532182 learned_addr learned my addr 172.21.15.5:0/743532182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8daffff640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db0100a80 0x7f8db00fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8db4add640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db0106a80 0x7f8db00fd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8daf7fe640 1 -- 172.21.15.5:0/743532182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db0100a80 msgr2=0x7f8db00fcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:10.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8daf7fe640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db0100a80 0x7f8db00fcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8daf7fe640 1 -- 172.21.15.5:0/743532182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db0106a80 msgr2=0x7f8db00fd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8daf7fe640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db0106a80 0x7f8db00fd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8daf7fe640 1 -- 172.21.15.5:0/743532182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db01a68a0 con 0x7f8db0101420 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.595+0000 7f8db4add640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db0106a80 0x7f8db00fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.596+0000 7f8daf7fe640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8db0101420 0x7f8db00fd270 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009230 tx=0x7f8da4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.596+0000 7f8dad7fa640 1 -- 172.21.15.5:0/743532182 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da403f030 con 0x7f8db0101420 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.596+0000 7f8dad7fa640 1 -- 172.21.15.5:0/743532182 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da4003ad0 con 0x7f8db0101420 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.597+0000 7f8dad7fa640 1 -- 172.21.15.5:0/743532182 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da4035070 con 0x7f8db0101420 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.597+0000 7f8db6540640 1 -- 172.21.15.5:0/743532182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db01a6b30 con 0x7f8db0101420 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.597+0000 7f8db6540640 1 -- 172.21.15.5:0/743532182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db01a6f90 con 0x7f8db0101420 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.599+0000 7f8dad7fa640 1 -- 172.21.15.5:0/743532182 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8da4042030 con 0x7f8db0101420 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.599+0000 7f8db8919740 1 Processor -- start 2024-09-11T22:35:10.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.599+0000 7f8db8919740 1 -- start start 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8db8919740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b0a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8db8919740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b0aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8db8919740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b17760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8db8919740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e72a702f0 con 0x563e72b0a5d0 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8db8919740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563e72a17cf0 con 0x563e72b15370 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8db8919740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563e72a6c2b0 con 0x563e72b0bb40 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8dad7fa640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8d980a2bd0 0x7f8d980a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b0aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8db4add640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b0a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daf7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b17760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b0aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52122/0 (socket says 172.21.15.5:52122) 2024-09-11T22:35:10.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daffff640 1 -- 172.21.15.5:0/1830805100 learned_addr learned my addr 172.21.15.5:0/1830805100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8db4add640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8d980a2bd0 0x7f8d980a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8dad7fa640 1 -- 172.21.15.5:0/743532182 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7f8da40efd60 con 0x7f8db0101420 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daffff640 1 -- 172.21.15.5:0/1830805100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 msgr2=0x563e72b0a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daffff640 1 --2- 172.21.15.5:0/1830805100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b0a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daffff640 1 -- 172.21.15.5:0/1830805100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 msgr2=0x563e72b17760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daffff640 1 --2- 172.21.15.5:0/1830805100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b17760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daffff640 1 -- 172.21.15.5:0/1830805100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e729fc3a0 con 0x563e72b0a5d0 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8db4add640 1 --2- 172.21.15.5:0/1830805100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b0a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daf7fe640 1 --2- 172.21.15.5:0/1830805100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b17760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.600+0000 7f8daffff640 1 --2- 172.21.15.5:0/1830805100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b0aa10 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8da000ac90 tx=0x7f8da000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5785f330dbc45e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8db4add640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8d980a2bd0 0x7f8d980a5090 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8da8005c40 tx=0x7f8da8009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:10.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8d967fc640 1 -- 172.21.15.5:0/1830805100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da001d070 con 0x563e72b0a5d0 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8d967fc640 1 -- 172.21.15.5:0/1830805100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da0018b20 con 0x563e72b0a5d0 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8d967fc640 1 -- 172.21.15.5:0/1830805100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da0010b40 con 0x563e72b0a5d0 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8db8919740 1 -- 172.21.15.5:0/1830805100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 msgr2=0x563e72b0aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8db8919740 1 --2- 172.21.15.5:0/1830805100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b0aa10 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f8da000ac90 tx=0x7f8da000f790 comp rx=0 tx=0).stop 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8db8919740 1 -- 172.21.15.5:0/1830805100 shutdown_connections 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8db8919740 1 --2- 172.21.15.5:0/1830805100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b17760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8db8919740 1 --2- 172.21.15.5:0/1830805100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b0aa10 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8db8919740 1 --2- 172.21.15.5:0/1830805100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b0a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.601+0000 7f8db8919740 1 -- 172.21.15.5:0/1830805100 >> 172.21.15.5:0/1830805100 conn(0x563e72ad3310 msgr2=0x563e72b06c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 1 -- 172.21.15.5:0/1830805100 shutdown_connections 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 1 -- 172.21.15.5:0/1830805100 wait complete. 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 1 -- start start 2024-09-11T22:35:10.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 1 Processor -- start 2024-09-11T22:35:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b14d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b11ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e72b18070 con 0x563e72b0a5d0 2024-09-11T22:35:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563e72a2b060 con 0x563e72b15370 2024-09-11T22:35:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563e72b181f0 con 0x563e72b0bb40 2024-09-11T22:35:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8db8919740 10 client.0 _ll_register_callbacks cb 0x563e7277e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8daffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b14d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8daffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b14d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52132/0 (socket says 172.21.15.5:52132) 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.602+0000 7f8daffff640 1 -- 172.21.15.5:0/4044432235 learned_addr learned my addr 172.21.15.5:0/4044432235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8daf7fe640 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b11ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8db4add640 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8daffff640 1 -- 172.21.15.5:0/4044432235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 msgr2=0x563e72b11ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8daffff640 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b11ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8daffff640 1 -- 172.21.15.5:0/4044432235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 msgr2=0x563e72b125e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8daffff640 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b125e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8daffff640 1 -- 172.21.15.5:0/4044432235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e727e13c0 con 0x563e72b0a5d0 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8daffff640 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b14d40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8da001b040 tx=0x7f8da0005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbe72c579c9a24b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8d8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:10.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/4044432235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da001d040 con 0x563e72b0a5d0 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/4044432235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7f8da00108e0 con 0x563e72b0a5d0 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8db8919740 1 -- 172.21.15.5:0/4044432235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8da0018b20 con 0x563e72b0a5d0 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/4044432235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8da0031050 con 0x563e72b0a5d0 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8db8919740 10 client.19443 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.603+0000 7f8db8919740 1 -- 172.21.15.5:0/4044432235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563e72b23ae0 con 0x563e72b0a5d0 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/4044432235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da0018b20 con 0x563e72b0a5d0 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/4044432235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f8da0020030 con 0x563e72b0a5d0 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8db8919740 10 client.19443 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8db8919740 10 client.19443 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8db8919740 20 client.19443 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8db8919740 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x563e72b24ee0 0x563e72b272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8db8919740 4 client.19443 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8db8919740 1 -- 172.21.15.5:0/4044432235 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e72a7d860 con 0x563e72b24ee0 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8daf7fe640 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x563e72b24ee0 0x563e72b272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8d8f7fe640 10 client.19443 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.604+0000 7f8daf7fe640 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x563e72b24ee0 0x563e72b272d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a1a484f5e99e17f server_cookie=9a78ff9a91c85c27 in_seq=0 out_seq=0 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/4044432235 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e72a7d860 con 0x563e72b24ee0 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8d8f7fe640 10 client.19443 handle_command_reply: tid=0 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 client.19443 shutdown 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 -- 172.21.15.5:0/4044432235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 msgr2=0x563e72b14d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b14d40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f8da001b040 tx=0x7f8da0005ed0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 -- 172.21.15.5:0/4044432235 shutdown_connections 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x563e72b24ee0 0x563e72b272d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b11ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 --2- 172.21.15.5:0/4044432235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b14d40 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8d8f7fe640 0 client.19443 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:35:10.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 -- 172.21.15.5:0/4044432235 >> 172.21.15.5:0/4044432235 conn(0x563e72ad3310 msgr2=0x563e72b0c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:10.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 -- 172.21.15.5:0/4044432235 shutdown_connections 2024-09-11T22:35:10.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.605+0000 7f8db8919740 1 -- 172.21.15.5:0/4044432235 wait complete. 2024-09-11T22:35:10.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.606+0000 7f8db8919740 20 client.19443 trim_cache size 0 max 16384 2024-09-11T22:35:10.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8db8919740 1 Processor -- start 2024-09-11T22:35:10.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8db8919740 1 -- start start 2024-09-11T22:35:10.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8db8919740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b21310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8db8919740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b0bb40 0x563e72b0d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8db8919740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b15370 0x563e72b0d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8db8919740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8db01af990 con 0x563e72b0a5d0 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8db8919740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8db01b1500 con 0x563e72b0bb40 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8db8919740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8db01b0060 con 0x563e72b15370 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8daffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b21310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8daf7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b0bb40 0x563e72b0d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8daffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b21310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52146/0 (socket says 172.21.15.5:52146) 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8daffff640 1 -- 172.21.15.5:0/2895813690 learned_addr learned my addr 172.21.15.5:0/2895813690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8db4add640 1 --2- 172.21.15.5:0/2895813690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b15370 0x563e72b0d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8daffff640 1 -- 172.21.15.5:0/2895813690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b15370 msgr2=0x563e72b0d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8daffff640 1 --2- 172.21.15.5:0/2895813690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b15370 0x563e72b0d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8daffff640 1 -- 172.21.15.5:0/2895813690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b0bb40 msgr2=0x563e72b0d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8daffff640 1 --2- 172.21.15.5:0/2895813690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b0bb40 0x563e72b0d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.619+0000 7f8daffff640 1 -- 172.21.15.5:0/2895813690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e72b10540 con 0x563e72b0a5d0 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8daffff640 1 --2- 172.21.15.5:0/2895813690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b21310 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8da000a350 tx=0x7f8da000a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2241c8c99461ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8d95ffb640 1 -- 172.21.15.5:0/2895813690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da003a070 con 0x563e72b0a5d0 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8d95ffb640 1 -- 172.21.15.5:0/2895813690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da0028070 con 0x563e72b0a5d0 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8d95ffb640 1 -- 172.21.15.5:0/2895813690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da001d070 con 0x563e72b0a5d0 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8db8919740 1 -- 172.21.15.5:0/2895813690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 msgr2=0x563e72b21310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8db8919740 1 --2- 172.21.15.5:0/2895813690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b21310 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f8da000a350 tx=0x7f8da000a6d0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8db8919740 1 -- 172.21.15.5:0/2895813690 shutdown_connections 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8db8919740 1 --2- 172.21.15.5:0/2895813690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b15370 0x563e72b0d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8db8919740 1 --2- 172.21.15.5:0/2895813690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b0bb40 0x563e72b0d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8db8919740 1 --2- 172.21.15.5:0/2895813690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b21310 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.620+0000 7f8db8919740 1 -- 172.21.15.5:0/2895813690 >> 172.21.15.5:0/2895813690 conn(0x563e72ad3310 msgr2=0x563e72b110e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.621+0000 7f8db8919740 1 -- 172.21.15.5:0/2895813690 shutdown_connections 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.621+0000 7f8db8919740 1 -- 172.21.15.5:0/2895813690 wait complete. 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.621+0000 7f8db8919740 1 -- start start 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.621+0000 7f8db8919740 1 Processor -- start 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8db8919740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b1d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8db8919740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b0e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8db8919740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b0eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8db8919740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8db01b0e30 con 0x563e72b0a5d0 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8db8919740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8db01b0760 con 0x563e72b15370 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8db8919740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8db01ad970 con 0x563e72b0bb40 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8db8919740 10 client.0 _ll_register_callbacks cb 0x7f8da001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8daffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b1d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8daffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b1d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52158/0 (socket says 172.21.15.5:52158) 2024-09-11T22:35:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8daffff640 1 -- 172.21.15.5:0/2061586052 learned_addr learned my addr 172.21.15.5:0/2061586052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8daf7fe640 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b0e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8db4add640 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b0eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8daffff640 1 -- 172.21.15.5:0/2061586052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 msgr2=0x563e72b0e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8daffff640 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b0e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8daffff640 1 -- 172.21.15.5:0/2061586052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 msgr2=0x563e72b0eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8daffff640 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b0eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8daffff640 1 -- 172.21.15.5:0/2061586052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e72b0f410 con 0x563e72b0a5d0 2024-09-11T22:35:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8daffff640 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b1d8a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8da000abb0 tx=0x7f8da0010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe1144f04770d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8d8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.622+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/2061586052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da003a020 con 0x563e72b0a5d0 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/2061586052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7f8da0018900 con 0x563e72b0a5d0 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8db8919740 1 -- 172.21.15.5:0/2061586052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8da0028070 con 0x563e72b0a5d0 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/2061586052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8da0045050 con 0x563e72b0a5d0 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8db8919740 10 client.19449 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8db8919740 1 -- 172.21.15.5:0/2061586052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563e72b8f6a0 con 0x563e72b0a5d0 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/2061586052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8da0028070 con 0x563e72b0a5d0 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/2061586052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f8da0020040 con 0x563e72b0a5d0 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8db8919740 10 client.19449 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8db8919740 10 client.19449 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8db8919740 20 client.19449 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8db8919740 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x563e72b8fcf0 0x563e72b920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8db8919740 4 client.19449 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8db8919740 1 -- 172.21.15.5:0/2061586052 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e72a77d70 con 0x563e72b8fcf0 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.623+0000 7f8daf7fe640 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x563e72b8fcf0 0x563e72b920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8daf7fe640 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x563e72b8fcf0 0x563e72b920e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f47725f3a2e605 server_cookie=67b607cd209ddae2 in_seq=0 out_seq=0 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8d8f7fe640 10 client.19449 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8d8f7fe640 1 -- 172.21.15.5:0/2061586052 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e72a77d70 con 0x563e72b8fcf0 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8d8f7fe640 10 client.19449 handle_command_reply: tid=0 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8db8919740 1 client.19449 shutdown 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8db8919740 1 -- 172.21.15.5:0/2061586052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 msgr2=0x563e72b1d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8db8919740 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b1d8a0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f8da000abb0 tx=0x7f8da0010470 comp rx=0 tx=0).stop 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8db8919740 1 -- 172.21.15.5:0/2061586052 shutdown_connections 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8db8919740 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x563e72b8fcf0 0x563e72b920e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8d8f7fe640 0 client.19449 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:35:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8db8919740 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e72b15370 0x563e72b0eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8db8919740 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e72b0bb40 0x563e72b0e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8db8919740 1 --2- 172.21.15.5:0/2061586052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e72b0a5d0 0x563e72b1d8a0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.624+0000 7f8db8919740 1 -- 172.21.15.5:0/2061586052 >> 172.21.15.5:0/2061586052 conn(0x563e72ad3310 msgr2=0x563e72b054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db8919740 1 -- 172.21.15.5:0/2061586052 shutdown_connections 2024-09-11T22:35:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db8919740 1 -- 172.21.15.5:0/2061586052 wait complete. 2024-09-11T22:35:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db8919740 20 client.19449 trim_cache size 0 max 16384 2024-09-11T22:35:10.631 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:35:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 -- 172.21.15.5:0/743532182 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8d980a2bd0 msgr2=0x7f8d980a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8d980a2bd0 0x7f8d980a5090 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8da8005c40 tx=0x7f8da8009500 comp rx=0 tx=0).stop 2024-09-11T22:35:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 -- 172.21.15.5:0/743532182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8db0101420 msgr2=0x7f8db00fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8db0101420 0x7f8db00fd270 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009230 tx=0x7f8da4004270 comp rx=0 tx=0).stop 2024-09-11T22:35:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 -- 172.21.15.5:0/743532182 shutdown_connections 2024-09-11T22:35:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8d980a2bd0 0x7f8d980a5090 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8db0106a80 0x7f8db00fd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8db0101420 0x7f8db00fd270 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 --2- 172.21.15.5:0/743532182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8db0100a80 0x7f8db00fcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 -- 172.21.15.5:0/743532182 >> 172.21.15.5:0/743532182 conn(0x7f8db00fc7a0 msgr2=0x7f8db006a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 -- 172.21.15.5:0/743532182 shutdown_connections 2024-09-11T22:35:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:10.625+0000 7f8db6540640 1 -- 172.21.15.5:0/743532182 wait complete. 2024-09-11T22:35:11.349 INFO:tasks.workunit.client.0.smithi005.stdout: 1 957306 7.09 MB/sec execute 569 sec latency 24.394 ms 2024-09-11T22:35:11.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:35:11.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:11 smithi172 ceph-mon[35461]: pgmap v413: 97 pgs: 97 active+clean; 94 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-09-11T22:35:11.962 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:11 smithi005 ceph-mon[30943]: pgmap v413: 97 pgs: 97 active+clean; 94 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-09-11T22:35:12.078 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:11 smithi159 ceph-mon[35534]: pgmap v413: 97 pgs: 97 active+clean; 94 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-09-11T22:35:12.349 INFO:tasks.workunit.client.0.smithi005.stdout: 1 959366 7.10 MB/sec execute 570 sec latency 48.396 ms 2024-09-11T22:35:12.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.491+0000 7fac006fe640 1 -- 172.21.15.5:0/385679543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf8103740 msgr2=0x7fabf8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:12.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.491+0000 7fac006fe640 1 --2- 172.21.15.5:0/385679543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf8103740 0x7fabf8107b80 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fabec009930 tx=0x7fabec02a420 comp rx=0 tx=0).stop 2024-09-11T22:35:12.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.491+0000 7fac006fe640 1 -- 172.21.15.5:0/385679543 shutdown_connections 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.491+0000 7fac006fe640 1 --2- 172.21.15.5:0/385679543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf8108290 0x7fabf810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.491+0000 7fac006fe640 1 --2- 172.21.15.5:0/385679543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf8103740 0x7fabf8107b80 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.491+0000 7fac006fe640 1 --2- 172.21.15.5:0/385679543 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf8102d90 0x7fabf8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.491+0000 7fac006fe640 1 -- 172.21.15.5:0/385679543 >> 172.21.15.5:0/385679543 conn(0x7fabf80fc7d0 msgr2=0x7fabf80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.491+0000 7fac006fe640 1 -- 172.21.15.5:0/385679543 shutdown_connections 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.491+0000 7fac006fe640 1 -- 172.21.15.5:0/385679543 wait complete. 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.491+0000 7fac006fe640 1 Processor -- start 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fac006fe640 1 -- start start 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fac006fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf8102d90 0x7fabf81009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fac006fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf8103740 0x7fabf8100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fac006fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf8108290 0x7fabf8101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fac006fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8112090 con 0x7fabf8102d90 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fac006fe640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8111f10 con 0x7fabf8108290 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fac006fe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fabf8112210 con 0x7fabf8103740 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fabfe49a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf8102d90 0x7fabf81009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fabfdc99640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf8103740 0x7fabf8100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fabfe49a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf8102d90 0x7fabf81009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48220/0 (socket says 172.21.15.5:48220) 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fabfec9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf8108290 0x7fabf8101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fabfe49a640 1 -- 172.21.15.5:0/1137476964 learned_addr learned my addr 172.21.15.5:0/1137476964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fabfe49a640 1 -- 172.21.15.5:0/1137476964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf8103740 msgr2=0x7fabf8100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf8103740 0x7fabf8100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fabfe49a640 1 -- 172.21.15.5:0/1137476964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf8108290 msgr2=0x7fabf8101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf8108290 0x7fabf8101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.492+0000 7fabfe49a640 1 -- 172.21.15.5:0/1137476964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf81a6a60 con 0x7fabf8102d90 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.493+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf8102d90 0x7fabf81009f0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fabe800ebc0 tx=0x7fabe800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.493+0000 7fabf77fe640 1 -- 172.21.15.5:0/1137476964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8010630 con 0x7fabf8102d90 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.493+0000 7fac006fe640 1 -- 172.21.15.5:0/1137476964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf81a6d50 con 0x7fabf8102d90 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.493+0000 7fabf77fe640 1 -- 172.21.15.5:0/1137476964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe8010cb0 con 0x7fabf8102d90 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.493+0000 7fabf77fe640 1 -- 172.21.15.5:0/1137476964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8018bd0 con 0x7fabf8102d90 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.493+0000 7fac006fe640 1 -- 172.21.15.5:0/1137476964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf81a7200 con 0x7fabf8102d90 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.495+0000 7fabf77fe640 1 -- 172.21.15.5:0/1137476964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fabe80107d0 con 0x7fabf8102d90 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fac02ad7740 1 Processor -- start 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fac02ad7740 1 -- start start 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be2b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 0x55916be2ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 0x55916be38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fac02ad7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55916bd912f0 con 0x55916be36370 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fac02ad7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55916bd38cf0 con 0x55916be2cb40 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fac02ad7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55916bd8d2b0 con 0x55916be2b5d0 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fabf77fe640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fabe00a2cf0 0x7fabe00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fabfec9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be2b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fabfec9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be2b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57160/0 (socket says 172.21.15.5:57160) 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.496+0000 7fabfec9b640 1 -- 172.21.15.5:0/1118324886 learned_addr learned my addr 172.21.15.5:0/1118324886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfec9b640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fabe00a2cf0 0x7fabe00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfdc99640 1 --2- 172.21.15.5:0/1118324886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 0x55916be38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabf77fe640 1 -- 172.21.15.5:0/1137476964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7fabe80cfae0 con 0x7fabf8102d90 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1118324886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 0x55916be2ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfec9b640 1 -- 172.21.15.5:0/1118324886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 msgr2=0x55916be2ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfec9b640 1 --2- 172.21.15.5:0/1118324886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 0x55916be2ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfec9b640 1 -- 172.21.15.5:0/1118324886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 msgr2=0x55916be38760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfec9b640 1 --2- 172.21.15.5:0/1118324886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 0x55916be38760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfec9b640 1 -- 172.21.15.5:0/1118324886 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916bd1d3a0 con 0x55916be2cb40 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1118324886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 0x55916be2ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfec9b640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fabe00a2cf0 0x7fabe00a51b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fabf000f790 tx=0x7fabf001e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabfec9b640 1 --2- 172.21.15.5:0/1118324886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be2b090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fabf0011a00 tx=0x7fabf0011ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab7b20db936a8f09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabf4ff9640 1 -- 172.21.15.5:0/1118324886 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0004230 con 0x55916be2cb40 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabf4ff9640 1 -- 172.21.15.5:0/1118324886 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf0011560 con 0x55916be2cb40 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.497+0000 7fabf4ff9640 1 -- 172.21.15.5:0/1118324886 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0003950 con 0x55916be2cb40 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.498+0000 7fac02ad7740 1 -- 172.21.15.5:0/1118324886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 msgr2=0x55916be2b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.498+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1118324886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be2b090 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fabf0011a00 tx=0x7fabf0011ec0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.498+0000 7fac02ad7740 1 -- 172.21.15.5:0/1118324886 shutdown_connections 2024-09-11T22:35:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.498+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1118324886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 0x55916be38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.498+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1118324886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 0x55916be2ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.498+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1118324886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be2b090 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.498+0000 7fac02ad7740 1 -- 172.21.15.5:0/1118324886 >> 172.21.15.5:0/1118324886 conn(0x55916bdf4310 msgr2=0x55916be27c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.498+0000 7fac02ad7740 1 -- 172.21.15.5:0/1118324886 shutdown_connections 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.498+0000 7fac02ad7740 1 -- 172.21.15.5:0/1118324886 wait complete. 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fac02ad7740 1 -- start start 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fac02ad7740 1 Processor -- start 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 0x55916be35d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be33020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 0x55916be33610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fac02ad7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55916be39010 con 0x55916be36370 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fac02ad7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55916bd4c060 con 0x55916be2cb40 2024-09-11T22:35:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fac02ad7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55916be39190 con 0x55916be2b5d0 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fac02ad7740 10 client.0 _ll_register_callbacks cb 0x55916ba9f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fabfec9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 0x55916be33610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fabfe49a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 0x55916be35d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fabfec9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 0x55916be33610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48242/0 (socket says 172.21.15.5:48242) 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fabfec9b640 1 -- 172.21.15.5:0/534080434 learned_addr learned my addr 172.21.15.5:0/534080434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.499+0000 7fabfdc99640 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be33020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabfec9b640 1 -- 172.21.15.5:0/534080434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 msgr2=0x55916be35d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabfec9b640 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 0x55916be35d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabfec9b640 1 -- 172.21.15.5:0/534080434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 msgr2=0x55916be33020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabfec9b640 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be33020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabfec9b640 1 -- 172.21.15.5:0/534080434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916bb023c0 con 0x55916be36370 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabfe49a640 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 0x55916be35d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabfec9b640 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 0x55916be33610 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fabf000ff90 tx=0x7fabf001dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef91fa70f385bf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabddffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabddffb640 1 -- 172.21.15.5:0/534080434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0009e40 con 0x55916be36370 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabddffb640 1 -- 172.21.15.5:0/534080434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8629+0+0 (secure 0 0 0) 0x7fabf0005260 con 0x55916be36370 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fac02ad7740 1 -- 172.21.15.5:0/534080434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabf0011560 con 0x55916be36370 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fabddffb640 1 -- 172.21.15.5:0/534080434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fabf0005460 con 0x55916be36370 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fac02ad7740 10 client.19461 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.500+0000 7fac02ad7740 1 -- 172.21.15.5:0/534080434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55916be44a70 con 0x55916be36370 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fabddffb640 1 -- 172.21.15.5:0/534080434 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf0011560 con 0x55916be36370 2024-09-11T22:35:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fabddffb640 1 -- 172.21.15.5:0/534080434 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fabf0026610 con 0x55916be36370 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fac02ad7740 10 client.19461 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fac02ad7740 10 client.19461 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fac02ad7740 20 client.19461 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fac02ad7740 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55916be45f30 0x55916be48320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fac02ad7740 4 client.19461 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fac02ad7740 1 -- 172.21.15.5:0/534080434 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55916bd9e860 con 0x55916be45f30 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fabfe49a640 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55916be45f30 0x55916be48320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fabfe49a640 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55916be45f30 0x55916be48320 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2a77c8b3928dbce server_cookie=7cf4c7016099135a in_seq=0 out_seq=0 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.501+0000 7fabddffb640 10 client.19461 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.502+0000 7fabddffb640 1 -- 172.21.15.5:0/534080434 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55916bd9e860 con 0x55916be45f30 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.502+0000 7fabddffb640 10 client.19461 handle_command_reply: tid=0 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.502+0000 7fac02ad7740 1 client.19461 shutdown 2024-09-11T22:35:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 1 -- 172.21.15.5:0/534080434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 msgr2=0x55916be33610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 0x55916be33610 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fabf000ff90 tx=0x7fabf001dd90 comp rx=0 tx=0).stop 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 1 -- 172.21.15.5:0/534080434 shutdown_connections 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be36370 0x55916be33610 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55916be45f30 0x55916be48320 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be33020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 1 --2- 172.21.15.5:0/534080434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be2b5d0 0x55916be35d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fabddffb640 0 client.19461 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 1 -- 172.21.15.5:0/534080434 >> 172.21.15.5:0/534080434 conn(0x55916bdf4310 msgr2=0x55916be2d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 1 -- 172.21.15.5:0/534080434 shutdown_connections 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 1 -- 172.21.15.5:0/534080434 wait complete. 2024-09-11T22:35:12.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.503+0000 7fac02ad7740 20 client.19461 trim_cache size 0 max 16384 2024-09-11T22:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fac02ad7740 1 Processor -- start 2024-09-11T22:35:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fac02ad7740 1 -- start start 2024-09-11T22:35:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be32000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 0x55916be4da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fac02ad7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81afcd0 con 0x55916be2b5d0 2024-09-11T22:35:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fac02ad7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81b1840 con 0x55916be2cb40 2024-09-11T22:35:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fac02ad7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81b03a0 con 0x55916be36370 2024-09-11T22:35:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fabfe49a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fabfe49a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be422b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48248/0 (socket says 172.21.15.5:48248) 2024-09-11T22:35:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fabfe49a640 1 -- 172.21.15.5:0/1931495198 learned_addr learned my addr 172.21.15.5:0/1931495198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fabfdc99640 1 --2- 172.21.15.5:0/1931495198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be32000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.519+0000 7fabfec9b640 1 --2- 172.21.15.5:0/1931495198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 0x55916be4da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.520+0000 7fabfe49a640 1 -- 172.21.15.5:0/1931495198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 msgr2=0x55916be4da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.520+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1931495198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 0x55916be4da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.520+0000 7fabfe49a640 1 -- 172.21.15.5:0/1931495198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 msgr2=0x55916be32000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.520+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1931495198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be32000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.520+0000 7fabfe49a640 1 -- 172.21.15.5:0/1931495198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916be28730 con 0x55916be2b5d0 2024-09-11T22:35:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.520+0000 7fabfdc99640 1 --2- 172.21.15.5:0/1931495198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be32000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.520+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1931495198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be422b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fabe80cd7c0 tx=0x7fabe8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df024640a0a52ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.520+0000 7fabdffff640 1 -- 172.21.15.5:0/1931495198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe80b4070 con 0x55916be2b5d0 2024-09-11T22:35:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.520+0000 7fabdffff640 1 -- 172.21.15.5:0/1931495198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe8010ac0 con 0x55916be2b5d0 2024-09-11T22:35:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.520+0000 7fabdffff640 1 -- 172.21.15.5:0/1931495198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe807ae60 con 0x55916be2b5d0 2024-09-11T22:35:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.521+0000 7fac02ad7740 1 -- 172.21.15.5:0/1931495198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 msgr2=0x55916be422b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.521+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1931495198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be422b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fabe80cd7c0 tx=0x7fabe8000f40 comp rx=0 tx=0).stop 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.521+0000 7fac02ad7740 1 -- 172.21.15.5:0/1931495198 shutdown_connections 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.521+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1931495198 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 0x55916be4da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.521+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1931495198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be32000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.521+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1931495198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be422b0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.521+0000 7fac02ad7740 1 -- 172.21.15.5:0/1931495198 >> 172.21.15.5:0/1931495198 conn(0x55916bdf4310 msgr2=0x55916be318f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.521+0000 7fac02ad7740 1 -- 172.21.15.5:0/1931495198 shutdown_connections 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.521+0000 7fac02ad7740 1 -- 172.21.15.5:0/1931495198 wait complete. 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fac02ad7740 1 -- start start 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fac02ad7740 1 Processor -- start 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be3e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be3e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fac02ad7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 0x55916be3b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fac02ad7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81b1170 con 0x55916be2b5d0 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fac02ad7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81b0aa0 con 0x55916be2cb40 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fac02ad7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fabf81adcb0 con 0x55916be36370 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fac02ad7740 10 client.0 _ll_register_callbacks cb 0x7fabe80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fabfe49a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be3e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fabfe49a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be3e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48262/0 (socket says 172.21.15.5:48262) 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fabfe49a640 1 -- 172.21.15.5:0/1770546567 learned_addr learned my addr 172.21.15.5:0/1770546567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fabfdc99640 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be3e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.522+0000 7fabfec9b640 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 0x55916be3b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.523+0000 7fabfe49a640 1 -- 172.21.15.5:0/1770546567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 msgr2=0x55916be3b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.523+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 0x55916be3b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.523+0000 7fabfe49a640 1 -- 172.21.15.5:0/1770546567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 msgr2=0x55916be3e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.523+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be3e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.523+0000 7fabfe49a640 1 -- 172.21.15.5:0/1770546567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55916be3b930 con 0x55916be2b5d0 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.523+0000 7fabfec9b640 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 0x55916be3b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.523+0000 7fabfdc99640 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be3e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.523+0000 7fabfe49a640 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be3e420 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fabe809bfd0 tx=0x7fabe8009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363793a01c9c4729 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.523+0000 7fabddffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fabddffb640 1 -- 172.21.15.5:0/1770546567 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe807ae60 con 0x55916be2b5d0 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fabddffb640 1 -- 172.21.15.5:0/1770546567 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7fabe809a340 con 0x55916be2b5d0 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fac02ad7740 1 -- 172.21.15.5:0/1770546567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe8010ac0 con 0x55916be2b5d0 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fabddffb640 1 -- 172.21.15.5:0/1770546567 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fabe80a9050 con 0x55916be2b5d0 2024-09-11T22:35:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fac02ad7740 10 client.19467 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fac02ad7740 1 -- 172.21.15.5:0/1770546567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55916beb10e0 con 0x55916be2b5d0 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fabddffb640 1 -- 172.21.15.5:0/1770546567 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe8010ac0 con 0x55916be2b5d0 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fabddffb640 1 -- 172.21.15.5:0/1770546567 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fabe8096bb0 con 0x55916be2b5d0 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fac02ad7740 10 client.19467 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fac02ad7740 10 client.19467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fac02ad7740 20 client.19467 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55916beb1790 0x55916beb3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fac02ad7740 4 client.19467 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fac02ad7740 1 -- 172.21.15.5:0/1770546567 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55916bd98d70 con 0x55916beb1790 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.524+0000 7fabfdc99640 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55916beb1790 0x55916beb3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fabddffb640 10 client.19467 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fabfdc99640 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55916beb1790 0x55916beb3b80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0110fd8a00d1fdf server_cookie=37a46589357e44d in_seq=0 out_seq=0 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fabddffb640 1 -- 172.21.15.5:0/1770546567 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55916bd98d70 con 0x55916beb1790 2024-09-11T22:35:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fabddffb640 10 client.19467 handle_command_reply: tid=0 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fac02ad7740 1 client.19467 shutdown 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fac02ad7740 1 -- 172.21.15.5:0/1770546567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 msgr2=0x55916be3e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be3e420 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fabe809bfd0 tx=0x7fabe8009a40 comp rx=0 tx=0).stop 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fac02ad7740 1 -- 172.21.15.5:0/1770546567 shutdown_connections 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55916be36370 0x55916be3b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55916beb1790 0x55916beb3b80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55916be2cb40 0x55916be3e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fac02ad7740 1 --2- 172.21.15.5:0/1770546567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916be2b5d0 0x55916be3e420 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fac02ad7740 1 -- 172.21.15.5:0/1770546567 >> 172.21.15.5:0/1770546567 conn(0x55916bdf4310 msgr2=0x55916be32110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fabddffb640 0 client.19467 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.525+0000 7fac02ad7740 1 -- 172.21.15.5:0/1770546567 shutdown_connections 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac02ad7740 1 -- 172.21.15.5:0/1770546567 wait complete. 2024-09-11T22:35:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac02ad7740 20 client.19467 trim_cache size 0 max 16384 2024-09-11T22:35:12.532 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:35:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 -- 172.21.15.5:0/1137476964 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fabe00a2cf0 msgr2=0x7fabe00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fabe00a2cf0 0x7fabe00a51b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fabf000f790 tx=0x7fabf001e000 comp rx=0 tx=0).stop 2024-09-11T22:35:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 -- 172.21.15.5:0/1137476964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf8102d90 msgr2=0x7fabf81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf8102d90 0x7fabf81009f0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fabe800ebc0 tx=0x7fabe800c6a0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 -- 172.21.15.5:0/1137476964 shutdown_connections 2024-09-11T22:35:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fabe00a2cf0 0x7fabe00a51b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf8108290 0x7fabf8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fabf8103740 0x7fabf8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 --2- 172.21.15.5:0/1137476964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf8102d90 0x7fabf81009f0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 -- 172.21.15.5:0/1137476964 >> 172.21.15.5:0/1137476964 conn(0x7fabf80fc7d0 msgr2=0x7fabf8105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 -- 172.21.15.5:0/1137476964 shutdown_connections 2024-09-11T22:35:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:12.526+0000 7fac006fe640 1 -- 172.21.15.5:0/1137476964 wait complete. 2024-09-11T22:35:13.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:35:13.349 INFO:tasks.workunit.client.0.smithi005.stdout: 1 960791 7.10 MB/sec execute 571 sec latency 14.585 ms 2024-09-11T22:35:13.627 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:13.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:13 smithi005 ceph-mon[30943]: pgmap v414: 97 pgs: 97 active+clean; 101 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-09-11T22:35:13.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:13 smithi005 ceph-mon[30943]: osdmap e107: 12 total, 12 up, 12 in 2024-09-11T22:35:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:13 smithi159 ceph-mon[35534]: pgmap v414: 97 pgs: 97 active+clean; 101 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-09-11T22:35:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:13 smithi159 ceph-mon[35534]: osdmap e107: 12 total, 12 up, 12 in 2024-09-11T22:35:13.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:13 smithi172 ceph-mon[35461]: pgmap v414: 97 pgs: 97 active+clean; 101 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.1 MiB/s wr, 291 op/s 2024-09-11T22:35:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:13 smithi172 ceph-mon[35461]: osdmap e107: 12 total, 12 up, 12 in 2024-09-11T22:35:14.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.174+0000 7f8441c0b640 1 -- 172.21.15.5:0/1369036237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f843c102570 msgr2=0x7f843c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:14.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.174+0000 7f8441c0b640 1 --2- 172.21.15.5:0/1369036237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f843c102570 0x7f843c1029b0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f8430009900 tx=0x7f843002a480 comp rx=0 tx=0).stop 2024-09-11T22:35:14.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.174+0000 7f8441c0b640 1 -- 172.21.15.5:0/1369036237 shutdown_connections 2024-09-11T22:35:14.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.174+0000 7f8441c0b640 1 --2- 172.21.15.5:0/1369036237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f843c102ef0 0x7f843c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.174+0000 7f8441c0b640 1 --2- 172.21.15.5:0/1369036237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f843c102570 0x7f843c1029b0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.174+0000 7f8441c0b640 1 --2- 172.21.15.5:0/1369036237 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f843c100e20 0x7f843c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.174+0000 7f8441c0b640 1 -- 172.21.15.5:0/1369036237 >> 172.21.15.5:0/1369036237 conn(0x7f843c0fc7d0 msgr2=0x7f843c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.174+0000 7f8441c0b640 1 -- 172.21.15.5:0/1369036237 shutdown_connections 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.174+0000 7f8441c0b640 1 -- 172.21.15.5:0/1369036237 wait complete. 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f8441c0b640 1 Processor -- start 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f8441c0b640 1 -- start start 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f8441c0b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f843c100e20 0x7f843c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f8441c0b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f843c102570 0x7f843c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f8441c0b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f843c102ef0 0x7f843c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f8441c0b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f843c110f80 con 0x7f843c102ef0 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f8441c0b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f843c110e00 con 0x7f843c102570 2024-09-11T22:35:14.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f8441c0b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f843c111100 con 0x7f843c100e20 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f843bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f843c102ef0 0x7f843c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f843b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f843c100e20 0x7f843c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f843b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f843c100e20 0x7f843c0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36496/0 (socket says 172.21.15.5:36496) 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f843bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f843c102ef0 0x7f843c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48276/0 (socket says 172.21.15.5:48276) 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f843b7fe640 1 -- 172.21.15.5:0/3052495407 learned_addr learned my addr 172.21.15.5:0/3052495407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.175+0000 7f843affd640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f843c102570 0x7f843c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f843b7fe640 1 -- 172.21.15.5:0/3052495407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f843c102570 msgr2=0x7f843c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f843b7fe640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f843c102570 0x7f843c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f843b7fe640 1 -- 172.21.15.5:0/3052495407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f843c102ef0 msgr2=0x7f843c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f843b7fe640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f843c102ef0 0x7f843c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f843b7fe640 1 -- 172.21.15.5:0/3052495407 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843c1a69a0 con 0x7f843c100e20 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f843bfff640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f843c102ef0 0x7f843c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f843b7fe640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f843c100e20 0x7f843c0fcc80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f842c00e9f0 tx=0x7f842c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f8438ff9640 1 -- 172.21.15.5:0/3052495407 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c00cd80 con 0x7f843c100e20 2024-09-11T22:35:14.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f8441c0b640 1 -- 172.21.15.5:0/3052495407 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843c1a6c90 con 0x7f843c100e20 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f8438ff9640 1 -- 172.21.15.5:0/3052495407 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f842c010910 con 0x7f843c100e20 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f8438ff9640 1 -- 172.21.15.5:0/3052495407 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c0188f0 con 0x7f843c100e20 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.176+0000 7f8441c0b640 1 -- 172.21.15.5:0/3052495407 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f843c1a71a0 con 0x7f843c100e20 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.178+0000 7f8438ff9640 1 -- 172.21.15.5:0/3052495407 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f842c010ac0 con 0x7f843c100e20 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f8443fe4740 1 Processor -- start 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f8443fe4740 1 -- start start 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc105b40 0x55fadc104090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc1045d0 0x55fadc104a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc10f370 0x55fadc111760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f8443fe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fadc06a2f0 con 0x55fadc10f370 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f8443fe4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fadc011cf0 con 0x55fadc105b40 2024-09-11T22:35:14.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f8443fe4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fadc0662b0 con 0x55fadc1045d0 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f843bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc105b40 0x55fadc104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f843b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc1045d0 0x55fadc104a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f843bfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc105b40 0x55fadc104090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57194/0 (socket says 172.21.15.5:57194) 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f843bfff640 1 -- 172.21.15.5:0/340479147 learned_addr learned my addr 172.21.15.5:0/340479147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f843affd640 1 --2- 172.21.15.5:0/340479147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc10f370 0x55fadc111760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f8438ff9640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84240a2cf0 0x7f84240a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f843bfff640 1 -- 172.21.15.5:0/340479147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc1045d0 msgr2=0x55fadc104a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f843bfff640 1 --2- 172.21.15.5:0/340479147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc1045d0 0x55fadc104a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f843bfff640 1 -- 172.21.15.5:0/340479147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc10f370 msgr2=0x55fadc111760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f843bfff640 1 --2- 172.21.15.5:0/340479147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc10f370 0x55fadc111760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.179+0000 7f843bfff640 1 -- 172.21.15.5:0/340479147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fadbff63a0 con 0x55fadc105b40 2024-09-11T22:35:14.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.180+0000 7f8438ff9640 1 -- 172.21.15.5:0/3052495407 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f842c0cf8d0 con 0x7f843c100e20 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.180+0000 7f843bfff640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84240a2cf0 0x7f84240a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.180+0000 7f843bfff640 1 --2- 172.21.15.5:0/340479147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc105b40 0x55fadc104090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8434010710 tx=0x7f8434010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3c2c0faff0ff551 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.180+0000 7f8421ffb640 1 -- 172.21.15.5:0/340479147 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843400ed50 con 0x55fadc105b40 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.180+0000 7f8421ffb640 1 -- 172.21.15.5:0/340479147 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8434010de0 con 0x55fadc105b40 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.180+0000 7f8421ffb640 1 -- 172.21.15.5:0/340479147 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434011620 con 0x55fadc105b40 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 -- 172.21.15.5:0/340479147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc105b40 msgr2=0x55fadc104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 --2- 172.21.15.5:0/340479147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc105b40 0x55fadc104090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8434010710 tx=0x7f8434010bd0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f843bfff640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84240a2cf0 0x7f84240a51b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f843401f000 tx=0x7f8434022000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 -- 172.21.15.5:0/340479147 shutdown_connections 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 --2- 172.21.15.5:0/340479147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc10f370 0x55fadc111760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 --2- 172.21.15.5:0/340479147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc1045d0 0x55fadc104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 --2- 172.21.15.5:0/340479147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc105b40 0x55fadc104090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 -- 172.21.15.5:0/340479147 >> 172.21.15.5:0/340479147 conn(0x55fadc0cd310 msgr2=0x55fadc100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 -- 172.21.15.5:0/340479147 shutdown_connections 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 -- 172.21.15.5:0/340479147 wait complete. 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 -- start start 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.181+0000 7f8443fe4740 1 Processor -- start 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 0x55fadc10ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc10c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 0x55fadc10c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f8443fe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fadc112070 con 0x55fadc105b40 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f8443fe4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fadc025060 con 0x55fadc1045d0 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f8443fe4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fadc1121f0 con 0x55fadc10f370 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f8443fe4740 10 client.0 _ll_register_callbacks cb 0x55fadbd788c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:14.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc10c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc10c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48302/0 (socket says 172.21.15.5:48302) 2024-09-11T22:35:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843affd640 1 -- 172.21.15.5:0/2892743030 learned_addr learned my addr 172.21.15.5:0/2892743030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843bfff640 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 0x55fadc10c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843b7fe640 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 0x55fadc10ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843affd640 1 -- 172.21.15.5:0/2892743030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 msgr2=0x55fadc10c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843affd640 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 0x55fadc10c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843affd640 1 -- 172.21.15.5:0/2892743030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 msgr2=0x55fadc10ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:14.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843affd640 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 0x55fadc10ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843affd640 1 -- 172.21.15.5:0/2892743030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fadbddb3c0 con 0x55fadc105b40 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843b7fe640 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 0x55fadc10ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f843affd640 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc10c010 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8430000c00 tx=0x7f8430004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe196b207550495 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f841affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f841affd640 1 -- 172.21.15.5:0/2892743030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430035040 con 0x55fadc105b40 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.182+0000 7f841affd640 1 -- 172.21.15.5:0/2892743030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f8430003ac0 con 0x55fadc105b40 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f8443fe4740 1 -- 172.21.15.5:0/2892743030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8434010de0 con 0x55fadc105b40 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f841affd640 1 -- 172.21.15.5:0/2892743030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8430003cc0 con 0x55fadc105b40 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f8443fe4740 10 client.19479 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f8443fe4740 1 -- 172.21.15.5:0/2892743030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55fadc11db60 con 0x55fadc105b40 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f841affd640 1 -- 172.21.15.5:0/2892743030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8434010de0 con 0x55fadc105b40 2024-09-11T22:35:14.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f841affd640 1 -- 172.21.15.5:0/2892743030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f843003c6a0 con 0x55fadc105b40 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f8443fe4740 10 client.19479 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f8443fe4740 10 client.19479 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f8443fe4740 20 client.19479 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55fadc11f070 0x55fadc1214f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f8443fe4740 4 client.19479 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f8443fe4740 1 -- 172.21.15.5:0/2892743030 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fadc077860 con 0x55fadc11f070 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.183+0000 7f843b7fe640 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55fadc11f070 0x55fadc1214f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.184+0000 7f841affd640 10 client.19479 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.184+0000 7f843b7fe640 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55fadc11f070 0x55fadc1214f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ee2895139c17bed server_cookie=31c078695f418154 in_seq=0 out_seq=0 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.184+0000 7f841affd640 1 -- 172.21.15.5:0/2892743030 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fadc077860 con 0x55fadc11f070 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.184+0000 7f841affd640 10 client.19479 handle_command_reply: tid=0 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.184+0000 7f8443fe4740 1 client.19479 shutdown 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 1 -- 172.21.15.5:0/2892743030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 msgr2=0x55fadc10c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:14.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc10c010 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8430000c00 tx=0x7f8430004440 comp rx=0 tx=0).stop 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 1 -- 172.21.15.5:0/2892743030 shutdown_connections 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55fadc11f070 0x55fadc1214f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 0x55fadc10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc10c010 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2892743030 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 0x55fadc10ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f841affd640 0 client.19479 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 1 -- 172.21.15.5:0/2892743030 >> 172.21.15.5:0/2892743030 conn(0x55fadc0cd310 msgr2=0x55fadc106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 1 -- 172.21.15.5:0/2892743030 shutdown_connections 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 1 -- 172.21.15.5:0/2892743030 wait complete. 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.185+0000 7f8443fe4740 20 client.19479 trim_cache size 0 max 16384 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.198+0000 7f8443fe4740 1 Processor -- start 2024-09-11T22:35:14.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.198+0000 7f8443fe4740 1 -- start start 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc1045d0 0x55fadc100e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc105b40 0x55fadc11b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc10f370 0x55fadc107120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f8443fe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1afc70 con 0x55fadc1045d0 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f8443fe4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1b17e0 con 0x55fadc10f370 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc1045d0 0x55fadc100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f8443fe4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1b0340 con 0x55fadc105b40 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc105b40 0x55fadc11b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc1045d0 0x55fadc100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48318/0 (socket says 172.21.15.5:48318) 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843b7fe640 1 -- 172.21.15.5:0/3553083531 learned_addr learned my addr 172.21.15.5:0/3553083531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843bfff640 1 --2- 172.21.15.5:0/3553083531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc10f370 0x55fadc107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843affd640 1 -- 172.21.15.5:0/3553083531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc10f370 msgr2=0x55fadc107120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843affd640 1 --2- 172.21.15.5:0/3553083531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc10f370 0x55fadc107120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843affd640 1 -- 172.21.15.5:0/3553083531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc1045d0 msgr2=0x55fadc100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843affd640 1 --2- 172.21.15.5:0/3553083531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc1045d0 0x55fadc100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843affd640 1 -- 172.21.15.5:0/3553083531 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fadc10f090 con 0x55fadc105b40 2024-09-11T22:35:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843b7fe640 1 --2- 172.21.15.5:0/3553083531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc1045d0 0x55fadc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.199+0000 7f843affd640 1 --2- 172.21.15.5:0/3553083531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc105b40 0x55fadc11b310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55fadc10d5c0 tx=0x7f843002a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b286faa0ad4a2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f84217fa640 1 -- 172.21.15.5:0/3553083531 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430052030 con 0x55fadc105b40 2024-09-11T22:35:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f84217fa640 1 -- 172.21.15.5:0/3553083531 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84300038c0 con 0x55fadc105b40 2024-09-11T22:35:14.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f84217fa640 1 -- 172.21.15.5:0/3553083531 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430035070 con 0x55fadc105b40 2024-09-11T22:35:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f8443fe4740 1 -- 172.21.15.5:0/3553083531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc105b40 msgr2=0x55fadc11b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f8443fe4740 1 --2- 172.21.15.5:0/3553083531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc105b40 0x55fadc11b310 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55fadc10d5c0 tx=0x7f843002a480 comp rx=0 tx=0).stop 2024-09-11T22:35:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f8443fe4740 1 -- 172.21.15.5:0/3553083531 shutdown_connections 2024-09-11T22:35:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f8443fe4740 1 --2- 172.21.15.5:0/3553083531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc10f370 0x55fadc107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f8443fe4740 1 --2- 172.21.15.5:0/3553083531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc105b40 0x55fadc11b310 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f8443fe4740 1 --2- 172.21.15.5:0/3553083531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc1045d0 0x55fadc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f8443fe4740 1 -- 172.21.15.5:0/3553083531 >> 172.21.15.5:0/3553083531 conn(0x55fadc0cd310 msgr2=0x55fadc100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f8443fe4740 1 -- 172.21.15.5:0/3553083531 shutdown_connections 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.200+0000 7f8443fe4740 1 -- 172.21.15.5:0/3553083531 wait complete. 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f8443fe4740 1 -- start start 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f8443fe4740 1 Processor -- start 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 0x55fadc117810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc117d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f8443fe4740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 0x55fadc108c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f8443fe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1b1110 con 0x55fadc105b40 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f8443fe4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1b0a40 con 0x55fadc1045d0 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f8443fe4740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f843c1adc70 con 0x55fadc10f370 2024-09-11T22:35:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f8443fe4740 10 client.0 _ll_register_callbacks cb 0x7f8430009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f843affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc117d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.201+0000 7f843b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 0x55fadc117810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 0x55fadc108c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 0x55fadc108c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36542/0 (socket says 172.21.15.5:36542) 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 0x55fadc117810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57210/0 (socket says 172.21.15.5:57210) 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc117d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48324/0 (socket says 172.21.15.5:48324) 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843bfff640 1 -- 172.21.15.5:0/2673844580 learned_addr learned my addr 172.21.15.5:0/2673844580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843bfff640 1 -- 172.21.15.5:0/2673844580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 msgr2=0x55fadc117810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843bfff640 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 0x55fadc117810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843bfff640 1 -- 172.21.15.5:0/2673844580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 msgr2=0x55fadc117d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843bfff640 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc117d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843bfff640 1 -- 172.21.15.5:0/2673844580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fadc109440 con 0x55fadc10f370 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f843bfff640 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 0x55fadc108c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f843400ead0 tx=0x7f8434010450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=631f998a52b5adcf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f841affd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f841affd640 1 -- 172.21.15.5:0/2673844580 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434038030 con 0x55fadc10f370 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f841affd640 1 -- 172.21.15.5:0/2673844580 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f843402e040 con 0x55fadc10f370 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f8443fe4740 1 -- 172.21.15.5:0/2673844580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84300038c0 con 0x55fadc10f370 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f841affd640 1 -- 172.21.15.5:0/2673844580 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8434010dc0 con 0x55fadc10f370 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f8443fe4740 10 client.28525 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.202+0000 7f8443fe4740 1 -- 172.21.15.5:0/2673844580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55fadc1899f0 con 0x55fadc10f370 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f841affd640 1 -- 172.21.15.5:0/2673844580 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84300038c0 con 0x55fadc10f370 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f841affd640 1 -- 172.21.15.5:0/2673844580 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f843400f040 con 0x55fadc10f370 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f8443fe4740 10 client.28525 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f8443fe4740 10 client.28525 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f8443fe4740 20 client.28525 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55fadc189e30 0x55fadc18c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f8443fe4740 4 client.28525 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f8443fe4740 1 -- 172.21.15.5:0/2673844580 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fadc071d70 con 0x55fadc189e30 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f843b7fe640 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55fadc189e30 0x55fadc18c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f843b7fe640 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55fadc189e30 0x55fadc18c220 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a66d76cf1f72fcbd server_cookie=26c6def079922e18 in_seq=0 out_seq=0 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.203+0000 7f841affd640 10 client.28525 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f841affd640 1 -- 172.21.15.5:0/2673844580 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fadc071d70 con 0x55fadc189e30 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f841affd640 10 client.28525 handle_command_reply: tid=0 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 client.28525 shutdown 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 -- 172.21.15.5:0/2673844580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 msgr2=0x55fadc108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 0x55fadc108c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f843400ead0 tx=0x7f8434010450 comp rx=0 tx=0).stop 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 -- 172.21.15.5:0/2673844580 shutdown_connections 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55fadc189e30 0x55fadc18c220 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fadc10f370 0x55fadc108c10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fadc105b40 0x55fadc117d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f841affd640 0 client.28525 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:35:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 --2- 172.21.15.5:0/2673844580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fadc1045d0 0x55fadc117810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 -- 172.21.15.5:0/2673844580 >> 172.21.15.5:0/2673844580 conn(0x55fadc0cd310 msgr2=0x55fadc10d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 -- 172.21.15.5:0/2673844580 shutdown_connections 2024-09-11T22:35:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 1 -- 172.21.15.5:0/2673844580 wait complete. 2024-09-11T22:35:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.204+0000 7f8443fe4740 20 client.28525 trim_cache size 0 max 16384 2024-09-11T22:35:14.221 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 -- 172.21.15.5:0/3052495407 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84240a2cf0 msgr2=0x7f84240a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84240a2cf0 0x7f84240a51b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f843401f000 tx=0x7f8434022000 comp rx=0 tx=0).stop 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 -- 172.21.15.5:0/3052495407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f843c100e20 msgr2=0x7f843c0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f843c100e20 0x7f843c0fcc80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f842c00e9f0 tx=0x7f842c00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 -- 172.21.15.5:0/3052495407 shutdown_connections 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f843c102ef0 0x7f843c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f84240a2cf0 0x7f84240a51b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f843c102570 0x7f843c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 --2- 172.21.15.5:0/3052495407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f843c100e20 0x7f843c0fcc80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 -- 172.21.15.5:0/3052495407 >> 172.21.15.5:0/3052495407 conn(0x7f843c0fc7d0 msgr2=0x7f843c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 -- 172.21.15.5:0/3052495407 shutdown_connections 2024-09-11T22:35:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:14.205+0000 7f8441c0b640 1 -- 172.21.15.5:0/3052495407 wait complete. 2024-09-11T22:35:14.349 INFO:tasks.workunit.client.0.smithi005.stdout: 1 962845 7.11 MB/sec execute 572 sec latency 21.190 ms 2024-09-11T22:35:15.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:35:15.349 INFO:tasks.workunit.client.0.smithi005.stdout: 1 964220 7.11 MB/sec execute 573 sec latency 18.816 ms 2024-09-11T22:35:15.482 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:15.764 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:15 smithi005 ceph-mon[30943]: pgmap v416: 97 pgs: 97 active+clean; 101 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-09-11T22:35:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:15 smithi159 ceph-mon[35534]: pgmap v416: 97 pgs: 97 active+clean; 101 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-09-11T22:35:15.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:15 smithi172 ceph-mon[35461]: pgmap v416: 97 pgs: 97 active+clean; 101 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.9 MiB/s wr, 271 op/s 2024-09-11T22:35:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:15.999+0000 7fcc8dda8640 1 -- 172.21.15.5:0/1053400481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc88102790 msgr2=0x7fcc881067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:15.999+0000 7fcc8dda8640 1 --2- 172.21.15.5:0/1053400481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc88102790 0x7fcc881067e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009900 tx=0x7fcc7002a410 comp rx=0 tx=0).stop 2024-09-11T22:35:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 -- 172.21.15.5:0/1053400481 shutdown_connections 2024-09-11T22:35:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 --2- 172.21.15.5:0/1053400481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc88106f10 0x7fcc8810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 --2- 172.21.15.5:0/1053400481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc88102790 0x7fcc881067e0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 --2- 172.21.15.5:0/1053400481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc88101de0 0x7fcc881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 -- 172.21.15.5:0/1053400481 >> 172.21.15.5:0/1053400481 conn(0x7fcc880fd8e0 msgr2=0x7fcc880ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 -- 172.21.15.5:0/1053400481 shutdown_connections 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 -- 172.21.15.5:0/1053400481 wait complete. 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 Processor -- start 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 -- start start 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc88101de0 0x7fcc8819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc88102790 0x7fcc8819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc88106f10 0x7fcc8819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88111f70 con 0x7fcc88106f10 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcc88111df0 con 0x7fcc88102790 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.000+0000 7fcc8dda8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc881120f0 con 0x7fcc88101de0 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.001+0000 7fcc87fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc88106f10 0x7fcc8819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.001+0000 7fcc87fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc88106f10 0x7fcc8819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48346/0 (socket says 172.21.15.5:48346) 2024-09-11T22:35:16.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.001+0000 7fcc87fff640 1 -- 172.21.15.5:0/2771015746 learned_addr learned my addr 172.21.15.5:0/2771015746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:16.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.001+0000 7fcc7ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc88102790 0x7fcc8819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.001+0000 7fcc877fe640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc88101de0 0x7fcc8819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc87fff640 1 -- 172.21.15.5:0/2771015746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc88101de0 msgr2=0x7fcc8819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc87fff640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc88101de0 0x7fcc8819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc87fff640 1 -- 172.21.15.5:0/2771015746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc88102790 msgr2=0x7fcc8819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc87fff640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc88102790 0x7fcc8819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc87fff640 1 -- 172.21.15.5:0/2771015746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc8819f680 con 0x7fcc88106f10 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc877fe640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc88101de0 0x7fcc8819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc87fff640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc88106f10 0x7fcc8819efa0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000eb10 tx=0x7fcc8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc857fa640 1 -- 172.21.15.5:0/2771015746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80004b20 con 0x7fcc88106f10 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc8dda8640 1 -- 172.21.15.5:0/2771015746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc8819f970 con 0x7fcc88106f10 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc857fa640 1 -- 172.21.15.5:0/2771015746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc80004cc0 con 0x7fcc88106f10 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc857fa640 1 -- 172.21.15.5:0/2771015746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80010760 con 0x7fcc88106f10 2024-09-11T22:35:16.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.002+0000 7fcc8dda8640 1 -- 172.21.15.5:0/2771015746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc881a7250 con 0x7fcc88106f10 2024-09-11T22:35:16.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.005+0000 7fcc90181740 1 Processor -- start 2024-09-11T22:35:16.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.005+0000 7fcc857fa640 1 -- 172.21.15.5:0/2771015746 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fcc80010900 con 0x7fcc88106f10 2024-09-11T22:35:16.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.005+0000 7fcc90181740 1 -- start start 2024-09-11T22:35:16.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.005+0000 7fcc90181740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6db40 0x556ba3e6c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.005+0000 7fcc90181740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e6ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.005+0000 7fcc90181740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e79760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc90181740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ba3dd22f0 con 0x556ba3e77370 2024-09-11T22:35:16.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc90181740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556ba3d79cf0 con 0x556ba3e6c5d0 2024-09-11T22:35:16.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc90181740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556ba3dce2b0 con 0x556ba3e6db40 2024-09-11T22:35:16.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e79760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e79760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48360/0 (socket says 172.21.15.5:48360) 2024-09-11T22:35:16.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc7ffff640 1 -- 172.21.15.5:0/2650623881 learned_addr learned my addr 172.21.15.5:0/2650623881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc877fe640 1 --2- 172.21.15.5:0/2650623881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e6ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc7ffff640 1 --2- 172.21.15.5:0/2650623881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6db40 0x556ba3e6c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc877fe640 1 -- 172.21.15.5:0/2650623881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6db40 msgr2=0x556ba3e6c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc877fe640 1 --2- 172.21.15.5:0/2650623881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6db40 0x556ba3e6c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc877fe640 1 -- 172.21.15.5:0/2650623881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 msgr2=0x556ba3e79760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc877fe640 1 --2- 172.21.15.5:0/2650623881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e79760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc877fe640 1 -- 172.21.15.5:0/2650623881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba3d5e3a0 con 0x556ba3e6c5d0 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.006+0000 7fcc857fa640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc740a2cf0 0x7fcc740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.007+0000 7fcc877fe640 1 --2- 172.21.15.5:0/2650623881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e6ca10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009c30 tx=0x7fcc780117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38b2d33f65cb4c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.007+0000 7fcc857fa640 1 -- 172.21.15.5:0/2771015746 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7fcc800cfb10 con 0x7fcc88106f10 2024-09-11T22:35:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.007+0000 7fcc7ffff640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc740a2cf0 0x7fcc740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.007+0000 7fcc7dffb640 1 -- 172.21.15.5:0/2650623881 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78005020 con 0x556ba3e6c5d0 2024-09-11T22:35:16.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.007+0000 7fcc7dffb640 1 -- 172.21.15.5:0/2650623881 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc780047a0 con 0x556ba3e6c5d0 2024-09-11T22:35:16.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.007+0000 7fcc7dffb640 1 -- 172.21.15.5:0/2650623881 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc780126b0 con 0x556ba3e6c5d0 2024-09-11T22:35:16.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 -- 172.21.15.5:0/2650623881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 msgr2=0x556ba3e6ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:16.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc7ffff640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc740a2cf0 0x7fcc740a51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005ec0 tx=0x7fcc70041040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:16.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 --2- 172.21.15.5:0/2650623881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e6ca10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009c30 tx=0x7fcc780117b0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 -- 172.21.15.5:0/2650623881 shutdown_connections 2024-09-11T22:35:16.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 --2- 172.21.15.5:0/2650623881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e79760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 --2- 172.21.15.5:0/2650623881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e6ca10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 --2- 172.21.15.5:0/2650623881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6db40 0x556ba3e6c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 -- 172.21.15.5:0/2650623881 >> 172.21.15.5:0/2650623881 conn(0x556ba3e35310 msgr2=0x556ba3e68c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 -- 172.21.15.5:0/2650623881 shutdown_connections 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 -- 172.21.15.5:0/2650623881 wait complete. 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 -- start start 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.008+0000 7fcc90181740 1 Processor -- start 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc90181740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6c5d0 0x556ba3e76cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc90181740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e73f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc90181740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc90181740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ba3e7a0c0 con 0x556ba3e77370 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc90181740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556ba3d8d060 con 0x556ba3e6db40 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc90181740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556ba3e7a240 con 0x556ba3e6c5d0 2024-09-11T22:35:16.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc90181740 10 client.0 _ll_register_callbacks cb 0x556ba3ae08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc7ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e73f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc7ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e73f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57238/0 (socket says 172.21.15.5:57238) 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc7ffff640 1 -- 172.21.15.5:0/187308851 learned_addr learned my addr 172.21.15.5:0/187308851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc87fff640 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc877fe640 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6c5d0 0x556ba3e76cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc7ffff640 1 -- 172.21.15.5:0/187308851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6c5d0 msgr2=0x556ba3e76cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc7ffff640 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6c5d0 0x556ba3e76cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc7ffff640 1 -- 172.21.15.5:0/187308851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 msgr2=0x556ba3e744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc7ffff640 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.009+0000 7fcc7ffff640 1 -- 172.21.15.5:0/187308851 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba3b433c0 con 0x556ba3e6db40 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.010+0000 7fcc7ffff640 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e73f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009230 tx=0x7fcc7002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20af34b71a9f86b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.010+0000 7fcc66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.010+0000 7fcc66ffd640 1 -- 172.21.15.5:0/187308851 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc70046070 con 0x556ba3e6db40 2024-09-11T22:35:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.010+0000 7fcc90181740 1 -- 172.21.15.5:0/187308851 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc780047a0 con 0x556ba3e6db40 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.010+0000 7fcc66ffd640 1 -- 172.21.15.5:0/187308851 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7fcc700037f0 con 0x556ba3e6db40 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.010+0000 7fcc66ffd640 1 -- 172.21.15.5:0/187308851 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fcc700039f0 con 0x556ba3e6db40 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.010+0000 7fcc90181740 10 client.28691 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.010+0000 7fcc90181740 1 -- 172.21.15.5:0/187308851 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556ba3e85b00 con 0x556ba3e6db40 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.010+0000 7fcc66ffd640 1 -- 172.21.15.5:0/187308851 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc780047a0 con 0x556ba3e6db40 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.011+0000 7fcc66ffd640 1 -- 172.21.15.5:0/187308851 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fcc7003dc60 con 0x556ba3e6db40 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.011+0000 7fcc90181740 10 client.28691 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.011+0000 7fcc90181740 10 client.28691 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.011+0000 7fcc90181740 20 client.28691 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.011+0000 7fcc90181740 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556ba3e86f10 0x556ba3e89370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.011+0000 7fcc90181740 4 client.28691 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.011+0000 7fcc90181740 1 -- 172.21.15.5:0/187308851 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ba3ddf860 con 0x556ba3e86f10 2024-09-11T22:35:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.011+0000 7fcc877fe640 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556ba3e86f10 0x556ba3e89370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.011+0000 7fcc877fe640 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556ba3e86f10 0x556ba3e89370 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61871ee255de3f9c server_cookie=9be9d45f3946ad94 in_seq=0 out_seq=0 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.011+0000 7fcc66ffd640 10 client.28691 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc66ffd640 1 -- 172.21.15.5:0/187308851 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ba3ddf860 con 0x556ba3e86f10 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc66ffd640 10 client.28691 handle_command_reply: tid=0 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 client.28691 shutdown 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 -- 172.21.15.5:0/187308851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 msgr2=0x556ba3e73f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e73f60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009230 tx=0x7fcc7002a410 comp rx=0 tx=0).stop 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 -- 172.21.15.5:0/187308851 shutdown_connections 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556ba3e86f10 0x556ba3e89370 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e73f60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc66ffd640 0 client.28691 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:35:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 --2- 172.21.15.5:0/187308851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6c5d0 0x556ba3e76cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 -- 172.21.15.5:0/187308851 >> 172.21.15.5:0/187308851 conn(0x556ba3e35310 msgr2=0x556ba3e6e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 -- 172.21.15.5:0/187308851 shutdown_connections 2024-09-11T22:35:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.012+0000 7fcc90181740 1 -- 172.21.15.5:0/187308851 wait complete. 2024-09-11T22:35:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.013+0000 7fcc90181740 20 client.28691 trim_cache size 0 max 16384 2024-09-11T22:35:16.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.027+0000 7fcc90181740 1 Processor -- start 2024-09-11T22:35:16.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.027+0000 7fcc90181740 1 -- start start 2024-09-11T22:35:16.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.027+0000 7fcc90181740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e83360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.027+0000 7fcc90181740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6db40 0x556ba3e78760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.027+0000 7fcc90181740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e78ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.027+0000 7fcc90181740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc881afce0 con 0x556ba3e77370 2024-09-11T22:35:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.027+0000 7fcc90181740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcc881b1850 con 0x556ba3e6c5d0 2024-09-11T22:35:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.027+0000 7fcc90181740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc881b03b0 con 0x556ba3e6db40 2024-09-11T22:35:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc877fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e83360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc877fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e83360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57242/0 (socket says 172.21.15.5:57242) 2024-09-11T22:35:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc877fe640 1 -- 172.21.15.5:0/2001046441 learned_addr learned my addr 172.21.15.5:0/2001046441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:16.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc87fff640 1 --2- 172.21.15.5:0/2001046441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e78ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc7ffff640 1 --2- 172.21.15.5:0/2001046441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6db40 0x556ba3e78760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc877fe640 1 -- 172.21.15.5:0/2001046441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6db40 msgr2=0x556ba3e78760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc877fe640 1 --2- 172.21.15.5:0/2001046441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6db40 0x556ba3e78760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc877fe640 1 -- 172.21.15.5:0/2001046441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 msgr2=0x556ba3e78ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc877fe640 1 --2- 172.21.15.5:0/2001046441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e78ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc877fe640 1 -- 172.21.15.5:0/2001046441 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba3e68ed0 con 0x556ba3e6c5d0 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.028+0000 7fcc877fe640 1 --2- 172.21.15.5:0/2001046441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e83360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009790 tx=0x7fcc78005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce947f47b411c58e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.029+0000 7fcc7d7fa640 1 -- 172.21.15.5:0/2001046441 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78004660 con 0x556ba3e6c5d0 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.029+0000 7fcc7d7fa640 1 -- 172.21.15.5:0/2001046441 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7802c860 con 0x556ba3e6c5d0 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.029+0000 7fcc7d7fa640 1 -- 172.21.15.5:0/2001046441 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78012990 con 0x556ba3e6c5d0 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.029+0000 7fcc90181740 1 -- 172.21.15.5:0/2001046441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 msgr2=0x556ba3e83360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.029+0000 7fcc90181740 1 --2- 172.21.15.5:0/2001046441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e83360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fcc78009790 tx=0x7fcc78005f90 comp rx=0 tx=0).stop 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.029+0000 7fcc90181740 1 -- 172.21.15.5:0/2001046441 shutdown_connections 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.029+0000 7fcc90181740 1 --2- 172.21.15.5:0/2001046441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e77370 0x556ba3e78ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.029+0000 7fcc90181740 1 --2- 172.21.15.5:0/2001046441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e6db40 0x556ba3e78760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.029+0000 7fcc90181740 1 --2- 172.21.15.5:0/2001046441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6c5d0 0x556ba3e83360 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.030+0000 7fcc90181740 1 -- 172.21.15.5:0/2001046441 >> 172.21.15.5:0/2001046441 conn(0x556ba3e35310 msgr2=0x556ba3e73f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.030+0000 7fcc90181740 1 -- 172.21.15.5:0/2001046441 shutdown_connections 2024-09-11T22:35:16.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.030+0000 7fcc90181740 1 -- 172.21.15.5:0/2001046441 wait complete. 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.030+0000 7fcc90181740 1 -- start start 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.030+0000 7fcc90181740 1 Processor -- start 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc90181740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e6c5d0 0x556ba3e7f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc90181740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e7fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc90181740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e77370 0x556ba3e7a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc7ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e7fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc7ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e7fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57246/0 (socket says 172.21.15.5:57246) 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e6c5d0 0x556ba3e7f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc7ffff640 1 -- 172.21.15.5:0/3247453946 learned_addr learned my addr 172.21.15.5:0/3247453946 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc881b1180 con 0x556ba3e6c5d0 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcc881b0ab0 con 0x556ba3e6db40 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc881adce0 con 0x556ba3e77370 2024-09-11T22:35:16.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc90181740 10 client.0 _ll_register_callbacks cb 0x7fcc70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc87fff640 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e77370 0x556ba3e7a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc7ffff640 1 -- 172.21.15.5:0/3247453946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e77370 msgr2=0x556ba3e7a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc7ffff640 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e77370 0x556ba3e7a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc7ffff640 1 -- 172.21.15.5:0/3247453946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e6c5d0 msgr2=0x556ba3e7f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc7ffff640 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e6c5d0 0x556ba3e7f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc7ffff640 1 -- 172.21.15.5:0/3247453946 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba3e791e0 con 0x556ba3e6db40 2024-09-11T22:35:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc87fff640 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e77370 0x556ba3e7a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.031+0000 7fcc877fe640 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e6c5d0 0x556ba3e7f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:16.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.032+0000 7fcc7ffff640 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e7fa20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcc7003c640 tx=0x7fcc70031860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f3df3573b708cae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:16.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.032+0000 7fcc66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.032+0000 7fcc66ffd640 1 -- 172.21.15.5:0/3247453946 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc70061030 con 0x556ba3e6db40 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.032+0000 7fcc66ffd640 1 -- 172.21.15.5:0/3247453946 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7fcc700035f0 con 0x556ba3e6db40 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.032+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc7802c860 con 0x556ba3e6db40 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.032+0000 7fcc66ffd640 1 -- 172.21.15.5:0/3247453946 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fcc70046070 con 0x556ba3e6db40 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.032+0000 7fcc90181740 10 client.28697 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.032+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556ba3ef1ff0 con 0x556ba3e6db40 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc66ffd640 1 -- 172.21.15.5:0/3247453946 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7802c860 con 0x556ba3e6db40 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc66ffd640 1 -- 172.21.15.5:0/3247453946 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fcc7003d450 con 0x556ba3e6db40 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc90181740 10 client.28697 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc90181740 10 client.28697 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc90181740 20 client.28697 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc90181740 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556ba3ef2740 0x556ba3ef4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:16.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc90181740 4 client.28697 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ba3dd9d70 con 0x556ba3ef2740 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc877fe640 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556ba3ef2740 0x556ba3ef4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc66ffd640 10 client.28697 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.033+0000 7fcc877fe640 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556ba3ef2740 0x556ba3ef4b60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df7124a3fc77f917 server_cookie=6c7ba60b21b757b3 in_seq=0 out_seq=0 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc66ffd640 1 -- 172.21.15.5:0/3247453946 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ba3dd9d70 con 0x556ba3ef2740 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc66ffd640 10 client.28697 handle_command_reply: tid=0 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc90181740 1 client.28697 shutdown 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 msgr2=0x556ba3e7fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc90181740 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e7fa20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcc7003c640 tx=0x7fcc70031860 comp rx=0 tx=0).stop 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 shutdown_connections 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc90181740 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x556ba3ef2740 0x556ba3ef4b60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc90181740 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ba3e77370 0x556ba3e7a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc90181740 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ba3e6db40 0x556ba3e7fa20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc90181740 1 --2- 172.21.15.5:0/3247453946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ba3e6c5d0 0x556ba3e7f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 >> 172.21.15.5:0/3247453946 conn(0x556ba3e35310 msgr2=0x556ba3e757c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc66ffd640 0 client.28697 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:35:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.034+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 shutdown_connections 2024-09-11T22:35:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.035+0000 7fcc90181740 1 -- 172.21.15.5:0/3247453946 wait complete. 2024-09-11T22:35:16.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.035+0000 7fcc90181740 20 client.28697 trim_cache size 0 max 16384 2024-09-11T22:35:16.040 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:35:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.035+0000 7fcc8dda8640 1 -- 172.21.15.5:0/2771015746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc740a2cf0 msgr2=0x7fcc740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.035+0000 7fcc8dda8640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc740a2cf0 0x7fcc740a51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005ec0 tx=0x7fcc70041040 comp rx=0 tx=0).stop 2024-09-11T22:35:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.035+0000 7fcc8dda8640 1 -- 172.21.15.5:0/2771015746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc88106f10 msgr2=0x7fcc8819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.035+0000 7fcc8dda8640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc88106f10 0x7fcc8819efa0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000eb10 tx=0x7fcc8000c2d0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.035+0000 7fcc8dda8640 1 -- 172.21.15.5:0/2771015746 shutdown_connections 2024-09-11T22:35:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.035+0000 7fcc8dda8640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc88106f10 0x7fcc8819efa0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.036+0000 7fcc8dda8640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc740a2cf0 0x7fcc740a51b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.036+0000 7fcc8dda8640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc88102790 0x7fcc8819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.036+0000 7fcc8dda8640 1 --2- 172.21.15.5:0/2771015746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc88101de0 0x7fcc8819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.036+0000 7fcc8dda8640 1 -- 172.21.15.5:0/2771015746 >> 172.21.15.5:0/2771015746 conn(0x7fcc880fd8e0 msgr2=0x7fcc880febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.036+0000 7fcc8dda8640 1 -- 172.21.15.5:0/2771015746 shutdown_connections 2024-09-11T22:35:16.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:16.036+0000 7fcc8dda8640 1 -- 172.21.15.5:0/2771015746 wait complete. 2024-09-11T22:35:16.349 INFO:tasks.workunit.client.0.smithi005.stdout: 1 966384 7.12 MB/sec execute 574 sec latency 55.543 ms 2024-09-11T22:35:16.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.219+0000 7f795375f640 1 -- 172.21.15.5:0/2555885017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794c102ca0 msgr2=0x7f794c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.219+0000 7f795375f640 1 --2- 172.21.15.5:0/2555885017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794c102ca0 0x7f794c103080 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f7948009900 tx=0x7f794802a410 comp rx=0 tx=0).stop 2024-09-11T22:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.219+0000 7f795375f640 1 -- 172.21.15.5:0/2555885017 shutdown_connections 2024-09-11T22:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.219+0000 7f795375f640 1 --2- 172.21.15.5:0/2555885017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f794c10c0d0 0x7f794c10e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.219+0000 7f795375f640 1 --2- 172.21.15.5:0/2555885017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f794c1035c0 0x7f794c10baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.219+0000 7f795375f640 1 --2- 172.21.15.5:0/2555885017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794c102ca0 0x7f794c103080 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.219+0000 7f795375f640 1 -- 172.21.15.5:0/2555885017 >> 172.21.15.5:0/2555885017 conn(0x7f794c0fc6b0 msgr2=0x7f794c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.219+0000 7f795375f640 1 -- 172.21.15.5:0/2555885017 shutdown_connections 2024-09-11T22:35:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.219+0000 7f795375f640 1 -- 172.21.15.5:0/2555885017 wait complete. 2024-09-11T22:35:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.219+0000 7f795375f640 1 Processor -- start 2024-09-11T22:35:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f795375f640 1 -- start start 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f795375f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f794c102ca0 0x7f794c1008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f795375f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794c1035c0 0x7f794c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f795375f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f794c10c0d0 0x7f794c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f795375f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f794c111ed0 con 0x7f794c1035c0 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f795375f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f794c111d50 con 0x7f794c102ca0 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f795375f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f794c112050 con 0x7f794c10c0d0 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f7950cfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794c1035c0 0x7f794c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f7950cfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794c1035c0 0x7f794c100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48392/0 (socket says 172.21.15.5:48392) 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f7950cfa640 1 -- 172.21.15.5:0/3118327585 learned_addr learned my addr 172.21.15.5:0/3118327585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f79514fb640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f794c102ca0 0x7f794c1008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f7951cfc640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f794c10c0d0 0x7f794c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f7950cfa640 1 -- 172.21.15.5:0/3118327585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f794c10c0d0 msgr2=0x7f794c101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f7950cfa640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f794c10c0d0 0x7f794c101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f7950cfa640 1 -- 172.21.15.5:0/3118327585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f794c102ca0 msgr2=0x7f794c1008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f7950cfa640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f794c102ca0 0x7f794c1008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f7950cfa640 1 -- 172.21.15.5:0/3118327585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794c1a67e0 con 0x7f794c1035c0 2024-09-11T22:35:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.220+0000 7f7950cfa640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794c1035c0 0x7f794c100e30 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e9c0 tx=0x7f793c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.221+0000 7f79427fc640 1 -- 172.21.15.5:0/3118327585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c00cd80 con 0x7f794c1035c0 2024-09-11T22:35:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.221+0000 7f79427fc640 1 -- 172.21.15.5:0/3118327585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793c0035f0 con 0x7f794c1035c0 2024-09-11T22:35:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.221+0000 7f79427fc640 1 -- 172.21.15.5:0/3118327585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c010690 con 0x7f794c1035c0 2024-09-11T22:35:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.221+0000 7f795375f640 1 -- 172.21.15.5:0/3118327585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794c1a6ad0 con 0x7f794c1035c0 2024-09-11T22:35:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.221+0000 7f795375f640 1 -- 172.21.15.5:0/3118327585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794c1a7020 con 0x7f794c1035c0 2024-09-11T22:35:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.223+0000 7f79565d7740 1 Processor -- start 2024-09-11T22:35:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.223+0000 7f79565d7740 1 -- start start 2024-09-11T22:35:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f79565d7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 0x560ef9d22ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f79565d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f79565d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d2dcb0 0x560ef9d300a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f79565d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560ef9c872f0 con 0x560ef9d23410 2024-09-11T22:35:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f79565d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560ef9c2f270 con 0x560ef9d2dcb0 2024-09-11T22:35:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f79565d7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560ef9c832b0 con 0x560ef9d22af0 2024-09-11T22:35:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f79427fc640 1 -- 172.21.15.5:0/3118327585 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f793c0037a0 con 0x7f794c1035c0 2024-09-11T22:35:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f7951cfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 0x560ef9d22ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f7951cfc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 0x560ef9d22ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36634/0 (socket says 172.21.15.5:36634) 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f7951cfc640 1 -- 172.21.15.5:0/1097661478 learned_addr learned my addr 172.21.15.5:0/1097661478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f7950cfa640 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d2dcb0 0x560ef9d300a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f79514fb640 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f7951cfc640 1 -- 172.21.15.5:0/1097661478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d2dcb0 msgr2=0x560ef9d300a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f7951cfc640 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d2dcb0 0x560ef9d300a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f7951cfc640 1 -- 172.21.15.5:0/1097661478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 msgr2=0x560ef9d2d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f7951cfc640 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f7951cfc640 1 -- 172.21.15.5:0/1097661478 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef9c2e340 con 0x560ef9d22af0 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f7950cfa640 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d2dcb0 0x560ef9d300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.224+0000 7f79514fb640 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.225+0000 7f7951cfc640 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 0x560ef9d22ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7944007ec0 tx=0x7f794400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f68c7c45a8378d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.225+0000 7f79427fc640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79380a2da0 0x7f79380a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.225+0000 7f79377fe640 1 -- 172.21.15.5:0/1097661478 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944010840 con 0x560ef9d22af0 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.225+0000 7f79377fe640 1 -- 172.21.15.5:0/1097661478 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79440183f0 con 0x560ef9d22af0 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.225+0000 7f79377fe640 1 -- 172.21.15.5:0/1097661478 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944017600 con 0x560ef9d22af0 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.225+0000 7f79514fb640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79380a2da0 0x7f79380a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.225+0000 7f79427fc640 1 -- 172.21.15.5:0/3118327585 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f793c0cf880 con 0x7f794c1035c0 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 -- 172.21.15.5:0/1097661478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 msgr2=0x560ef9d22ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79514fb640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79380a2da0 0x7f79380a5260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7948009230 tx=0x7f79480058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 0x560ef9d22ed0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7944007ec0 tx=0x7f794400f790 comp rx=0 tx=0).stop 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 -- 172.21.15.5:0/1097661478 shutdown_connections 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d2dcb0 0x560ef9d300a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 --2- 172.21.15.5:0/1097661478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 0x560ef9d22ed0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 -- 172.21.15.5:0/1097661478 >> 172.21.15.5:0/1097661478 conn(0x560ef9d1f060 msgr2=0x560ef9d1f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 -- 172.21.15.5:0/1097661478 shutdown_connections 2024-09-11T22:35:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 -- 172.21.15.5:0/1097661478 wait complete. 2024-09-11T22:35:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 -- start start 2024-09-11T22:35:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.226+0000 7f79565d7740 1 Processor -- start 2024-09-11T22:35:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f79565d7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 0x560ef9d2abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f79565d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d23410 0x560ef9d2b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f79565d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 0x560ef9d2b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f79565d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560ef9d30880 con 0x560ef9d2dcb0 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f79565d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560ef9c10290 con 0x560ef9d23410 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f79565d7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560ef9d30a00 con 0x560ef9d22af0 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f79565d7740 10 client.0 _ll_register_callbacks cb 0x560ef9c41140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7951cfc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 0x560ef9d2b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7950cfa640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d23410 0x560ef9d2b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7951cfc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 0x560ef9d2b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48412/0 (socket says 172.21.15.5:48412) 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7951cfc640 1 -- 172.21.15.5:0/406332165 learned_addr learned my addr 172.21.15.5:0/406332165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f79514fb640 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 0x560ef9d2abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7951cfc640 1 -- 172.21.15.5:0/406332165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 msgr2=0x560ef9d2abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7951cfc640 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 0x560ef9d2abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7951cfc640 1 -- 172.21.15.5:0/406332165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d23410 msgr2=0x560ef9d2b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7951cfc640 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d23410 0x560ef9d2b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7951cfc640 1 -- 172.21.15.5:0/406332165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef999a210 con 0x560ef9d2dcb0 2024-09-11T22:35:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7950cfa640 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d23410 0x560ef9d2b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7951cfc640 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 0x560ef9d2b630 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7944018e70 tx=0x7f7944005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ab087c14cb410a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.227+0000 7f7934ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f7934ff9640 1 -- 172.21.15.5:0/406332165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944026020 con 0x560ef9d2dcb0 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f79565d7740 1 -- 172.21.15.5:0/406332165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79440183f0 con 0x560ef9d2dcb0 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f7934ff9640 1 -- 172.21.15.5:0/406332165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f7944018830 con 0x560ef9d2dcb0 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f7934ff9640 1 -- 172.21.15.5:0/406332165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7944031050 con 0x560ef9d2dcb0 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f79565d7740 10 client.19515 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f79565d7740 1 -- 172.21.15.5:0/406332165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560ef9a66620 con 0x560ef9d2dcb0 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f7934ff9640 1 -- 172.21.15.5:0/406332165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79440183f0 con 0x560ef9d2dcb0 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f7934ff9640 1 -- 172.21.15.5:0/406332165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f7944004040 con 0x560ef9d2dcb0 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f79565d7740 10 client.19515 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f79565d7740 10 client.19515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f79565d7740 20 client.19515 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.228+0000 7f79565d7740 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560ef9d3dbd0 0x560ef9d3ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.229+0000 7f79565d7740 4 client.19515 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.229+0000 7f79565d7740 1 -- 172.21.15.5:0/406332165 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ef9c94a20 con 0x560ef9d3dbd0 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.229+0000 7f79514fb640 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560ef9d3dbd0 0x560ef9d3ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.229+0000 7f7934ff9640 10 client.19515 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.229+0000 7f79514fb640 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560ef9d3dbd0 0x560ef9d3ffc0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452f64aec039b381 server_cookie=3625732350bf2d9a in_seq=0 out_seq=0 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f7934ff9640 1 -- 172.21.15.5:0/406332165 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ef9c94a20 con 0x560ef9d3dbd0 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f7934ff9640 10 client.19515 handle_command_reply: tid=0 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f79565d7740 1 client.19515 shutdown 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f79565d7740 1 -- 172.21.15.5:0/406332165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 msgr2=0x560ef9d2b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f79565d7740 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 0x560ef9d2b630 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7944018e70 tx=0x7f7944005a50 comp rx=0 tx=0).stop 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f79565d7740 1 -- 172.21.15.5:0/406332165 shutdown_connections 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f79565d7740 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560ef9d3dbd0 0x560ef9d3ffc0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f79565d7740 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 0x560ef9d2b630 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f79565d7740 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d23410 0x560ef9d2b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f7934ff9640 0 client.19515 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f79565d7740 1 --2- 172.21.15.5:0/406332165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d22af0 0x560ef9d2abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.230+0000 7f79565d7740 1 -- 172.21.15.5:0/406332165 >> 172.21.15.5:0/406332165 conn(0x560ef9d1f060 msgr2=0x560ef9d24dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.231+0000 7f79565d7740 1 -- 172.21.15.5:0/406332165 shutdown_connections 2024-09-11T22:35:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.231+0000 7f79565d7740 1 -- 172.21.15.5:0/406332165 wait complete. 2024-09-11T22:35:17.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.231+0000 7f79565d7740 20 client.19515 trim_cache size 0 max 16384 2024-09-11T22:35:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f79565d7740 1 Processor -- start 2024-09-11T22:35:17.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f79565d7740 1 -- start start 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f79565d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 0x560ef9d2cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f79565d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f79565d7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d2dcb0 0x560ef9d45590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f79565d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f794c1afc40 con 0x560ef9d23410 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f79565d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f794c1b17b0 con 0x560ef9d22af0 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f79565d7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f794c1b0310 con 0x560ef9d2dcb0 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f7950cfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f7950cfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48422/0 (socket says 172.21.15.5:48422) 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f7950cfa640 1 -- 172.21.15.5:0/4073727488 learned_addr learned my addr 172.21.15.5:0/4073727488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f7951cfc640 1 --2- 172.21.15.5:0/4073727488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d2dcb0 0x560ef9d45590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f79514fb640 1 --2- 172.21.15.5:0/4073727488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 0x560ef9d2cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f7950cfa640 1 -- 172.21.15.5:0/4073727488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d2dcb0 msgr2=0x560ef9d45590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f7950cfa640 1 --2- 172.21.15.5:0/4073727488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d2dcb0 0x560ef9d45590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.250+0000 7f7950cfa640 1 -- 172.21.15.5:0/4073727488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 msgr2=0x560ef9d2cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.251+0000 7f7950cfa640 1 --2- 172.21.15.5:0/4073727488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 0x560ef9d2cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.251+0000 7f7950cfa640 1 -- 172.21.15.5:0/4073727488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef9d1ee90 con 0x560ef9d23410 2024-09-11T22:35:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.251+0000 7f7950cfa640 1 --2- 172.21.15.5:0/4073727488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2b460 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f793c098190 tx=0x7f793c004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447ff3d8c6c592f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.251+0000 7f7936ffd640 1 -- 172.21.15.5:0/4073727488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c09c430 con 0x560ef9d23410 2024-09-11T22:35:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.251+0000 7f7936ffd640 1 -- 172.21.15.5:0/4073727488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793c09bd30 con 0x560ef9d23410 2024-09-11T22:35:17.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.251+0000 7f7936ffd640 1 -- 172.21.15.5:0/4073727488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c09b3e0 con 0x560ef9d23410 2024-09-11T22:35:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 -- 172.21.15.5:0/4073727488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 msgr2=0x560ef9d2b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 --2- 172.21.15.5:0/4073727488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2b460 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f793c098190 tx=0x7f793c004700 comp rx=0 tx=0).stop 2024-09-11T22:35:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 -- 172.21.15.5:0/4073727488 shutdown_connections 2024-09-11T22:35:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 --2- 172.21.15.5:0/4073727488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d2dcb0 0x560ef9d45590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 --2- 172.21.15.5:0/4073727488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d23410 0x560ef9d2b460 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 --2- 172.21.15.5:0/4073727488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 0x560ef9d2cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 -- 172.21.15.5:0/4073727488 >> 172.21.15.5:0/4073727488 conn(0x560ef9d1f060 msgr2=0x560ef9d1e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 -- 172.21.15.5:0/4073727488 shutdown_connections 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 -- 172.21.15.5:0/4073727488 wait complete. 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 -- start start 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.252+0000 7f79565d7740 1 Processor -- start 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f79565d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 0x560ef9d31ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f79565d7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d23410 0x560ef9d31ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f79565d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 0x560ef9d32530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f79565d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f794c1b10e0 con 0x560ef9d2dcb0 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f79565d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f794c1b0a10 con 0x560ef9d22af0 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f79565d7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f794c1adc40 con 0x560ef9d23410 2024-09-11T22:35:17.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f79565d7740 10 client.0 _ll_register_callbacks cb 0x7f7944007d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f79514fb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 0x560ef9d31ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f7950cfa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d23410 0x560ef9d31ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f79514fb640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 0x560ef9d31ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57286/0 (socket says 172.21.15.5:57286) 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f79514fb640 1 -- 172.21.15.5:0/2228528587 learned_addr learned my addr 172.21.15.5:0/2228528587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f7951cfc640 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 0x560ef9d32530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f7950cfa640 1 -- 172.21.15.5:0/2228528587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 msgr2=0x560ef9d31ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f7950cfa640 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 0x560ef9d31ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f7950cfa640 1 -- 172.21.15.5:0/2228528587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 msgr2=0x560ef9d32530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f7950cfa640 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 0x560ef9d32530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.253+0000 7f7950cfa640 1 -- 172.21.15.5:0/2228528587 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef9da7630 con 0x560ef9d23410 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.254+0000 7f7950cfa640 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d23410 0x560ef9d31ff0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f793c003dd0 tx=0x7f793c003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5bd3c1b3e77e800 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.254+0000 7f7934ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.254+0000 7f7934ff9640 1 -- 172.21.15.5:0/2228528587 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c0a49a0 con 0x560ef9d23410 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.254+0000 7f7934ff9640 1 -- 172.21.15.5:0/2228528587 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f793c09b390 con 0x560ef9d23410 2024-09-11T22:35:17.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.254+0000 7f79565d7740 1 -- 172.21.15.5:0/2228528587 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f793c09bd30 con 0x560ef9d23410 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.254+0000 7f7934ff9640 1 -- 172.21.15.5:0/2228528587 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f793c09c430 con 0x560ef9d23410 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.254+0000 7f79565d7740 10 client.28555 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.254+0000 7f79565d7740 1 -- 172.21.15.5:0/2228528587 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560ef9da8950 con 0x560ef9d23410 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.254+0000 7f7934ff9640 1 -- 172.21.15.5:0/2228528587 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f793c09bd30 con 0x560ef9d23410 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.255+0000 7f7934ff9640 1 -- 172.21.15.5:0/2228528587 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f793c0b2030 con 0x560ef9d23410 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.255+0000 7f79565d7740 10 client.28555 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.255+0000 7f79565d7740 10 client.28555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.255+0000 7f79565d7740 20 client.28555 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.255+0000 7f79565d7740 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560ef9da9080 0x560ef9dab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.255+0000 7f79565d7740 4 client.28555 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.255+0000 7f79565d7740 1 -- 172.21.15.5:0/2228528587 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ef9c8ed70 con 0x560ef9da9080 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.255+0000 7f79514fb640 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560ef9da9080 0x560ef9dab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.256+0000 7f79514fb640 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560ef9da9080 0x560ef9dab4a0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620e096106dd05dd server_cookie=371c68f377ebacb7 in_seq=0 out_seq=0 2024-09-11T22:35:17.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.256+0000 7f7934ff9640 10 client.28555 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:17.350 INFO:tasks.workunit.client.0.smithi005.stdout: 1 967532 7.12 MB/sec execute 575 sec latency 60.879 ms 2024-09-11T22:35:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f7934ff9640 1 -- 172.21.15.5:0/2228528587 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ef9c8ed70 con 0x560ef9da9080 2024-09-11T22:35:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f7934ff9640 10 client.28555 handle_command_reply: tid=0 2024-09-11T22:35:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 client.28555 shutdown 2024-09-11T22:35:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 -- 172.21.15.5:0/2228528587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d23410 msgr2=0x560ef9d31ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d23410 0x560ef9d31ff0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f793c003dd0 tx=0x7f793c003d40 comp rx=0 tx=0).stop 2024-09-11T22:35:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 -- 172.21.15.5:0/2228528587 shutdown_connections 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560ef9da9080 0x560ef9dab4a0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ef9d2dcb0 0x560ef9d32530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560ef9d23410 0x560ef9d31ff0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f7934ff9640 0 client.28555 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 --2- 172.21.15.5:0/2228528587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560ef9d22af0 0x560ef9d31ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 -- 172.21.15.5:0/2228528587 >> 172.21.15.5:0/2228528587 conn(0x560ef9d1f060 msgr2=0x560ef9d38b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 -- 172.21.15.5:0/2228528587 shutdown_connections 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 1 -- 172.21.15.5:0/2228528587 wait complete. 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.397+0000 7f79565d7740 20 client.28555 trim_cache size 0 max 16384 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:35:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:35:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:35:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:35:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 -- 172.21.15.5:0/3118327585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79380a2da0 msgr2=0x7f79380a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79380a2da0 0x7f79380a5260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7948009230 tx=0x7f79480058d0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 -- 172.21.15.5:0/3118327585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794c1035c0 msgr2=0x7f794c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794c1035c0 0x7f794c100e30 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f793c00e9c0 tx=0x7f793c00ee80 comp rx=0 tx=0).stop 2024-09-11T22:35:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 -- 172.21.15.5:0/3118327585 shutdown_connections 2024-09-11T22:35:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f794c10c0d0 0x7f794c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f794c1035c0 0x7f794c100e30 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f79380a2da0 0x7f79380a5260 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 --2- 172.21.15.5:0/3118327585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f794c102ca0 0x7f794c1008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 -- 172.21.15.5:0/3118327585 >> 172.21.15.5:0/3118327585 conn(0x7f794c0fc6b0 msgr2=0x7f794c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 -- 172.21.15.5:0/3118327585 shutdown_connections 2024-09-11T22:35:17.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:17.398+0000 7f795375f640 1 -- 172.21.15.5:0/3118327585 wait complete. 2024-09-11T22:35:17.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:17 smithi005 ceph-mon[30943]: pgmap v417: 97 pgs: 97 active+clean; 101 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2024-09-11T22:35:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:17 smithi159 ceph-mon[35534]: pgmap v417: 97 pgs: 97 active+clean; 101 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2024-09-11T22:35:17.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:17 smithi172 ceph-mon[35461]: pgmap v417: 97 pgs: 97 active+clean; 101 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 232 op/s 2024-09-11T22:35:18.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.979+0000 7fe3e0933640 1 -- 172.21.15.5:0/995972934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3dc102790 msgr2=0x7fe3dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:18.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.979+0000 7fe3e0933640 1 --2- 172.21.15.5:0/995972934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3dc102790 0x7fe3dc1067e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009900 tx=0x7fe3d002a410 comp rx=0 tx=0).stop 2024-09-11T22:35:18.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.979+0000 7fe3e0933640 1 -- 172.21.15.5:0/995972934 shutdown_connections 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.979+0000 7fe3e0933640 1 --2- 172.21.15.5:0/995972934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc106f10 0x7fe3dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.979+0000 7fe3e0933640 1 --2- 172.21.15.5:0/995972934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3dc102790 0x7fe3dc1067e0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.979+0000 7fe3e0933640 1 --2- 172.21.15.5:0/995972934 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc101de0 0x7fe3dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.979+0000 7fe3e0933640 1 -- 172.21.15.5:0/995972934 >> 172.21.15.5:0/995972934 conn(0x7fe3dc0fd8c0 msgr2=0x7fe3dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.979+0000 7fe3e0933640 1 -- 172.21.15.5:0/995972934 shutdown_connections 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.979+0000 7fe3e0933640 1 -- 172.21.15.5:0/995972934 wait complete. 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.980+0000 7fe3e0933640 1 Processor -- start 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.980+0000 7fe3e0933640 1 -- start start 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.980+0000 7fe3e0933640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc101de0 0x7fe3dc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.980+0000 7fe3e0933640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc102790 0x7fe3dc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.980+0000 7fe3e0933640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3dc106f10 0x7fe3dc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.980+0000 7fe3e0933640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc111f70 con 0x7fe3dc106f10 2024-09-11T22:35:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.980+0000 7fe3e0933640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc111df0 con 0x7fe3dc102790 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.980+0000 7fe3e0933640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1120f0 con 0x7fe3dc101de0 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3da59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc101de0 0x7fe3dc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3dad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3dc106f10 0x7fe3dc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3da59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc101de0 0x7fe3dc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36672/0 (socket says 172.21.15.5:36672) 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3da59c640 1 -- 172.21.15.5:0/341386835 learned_addr learned my addr 172.21.15.5:0/341386835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc102790 0x7fe3dc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3da59c640 1 -- 172.21.15.5:0/341386835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc102790 msgr2=0x7fe3dc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3da59c640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc102790 0x7fe3dc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3da59c640 1 -- 172.21.15.5:0/341386835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3dc106f10 msgr2=0x7fe3dc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3da59c640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3dc106f10 0x7fe3dc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3da59c640 1 -- 172.21.15.5:0/341386835 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3dc1a6970 con 0x7fe3dc101de0 2024-09-11T22:35:18.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3da59c640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc101de0 0x7fe3dc0744f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00e9f0 tx=0x7fe3cc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.981+0000 7fe3cb7fe640 1 -- 172.21.15.5:0/341386835 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc00cd80 con 0x7fe3dc101de0 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.982+0000 7fe3e0933640 1 -- 172.21.15.5:0/341386835 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3dc1a6c60 con 0x7fe3dc101de0 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.982+0000 7fe3cb7fe640 1 -- 172.21.15.5:0/341386835 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc005500 con 0x7fe3dc101de0 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.982+0000 7fe3cb7fe640 1 -- 172.21.15.5:0/341386835 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc010690 con 0x7fe3dc101de0 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.982+0000 7fe3e0933640 1 -- 172.21.15.5:0/341386835 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3dc1a7170 con 0x7fe3dc101de0 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.984+0000 7fe3cb7fe640 1 -- 172.21.15.5:0/341386835 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe3cc0046c0 con 0x7fe3dc101de0 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.984+0000 7fe3e2d0c740 1 Processor -- start 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3e2d0c740 1 -- start start 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3cb7fe640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3c40a2d70 0x7fe3c40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0db40 0x55bdb3c0c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c0ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c17370 0x55bdb3c19760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bdb3b722f0 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bdb3b19cf0 con 0x55bdb3c0db40 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bdb3b6e2b0 con 0x55bdb3c17370 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3cb7fe640 1 -- 172.21.15.5:0/341386835 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7fe3cc0cf8f0 con 0x7fe3dc101de0 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3dad9d640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3c40a2d70 0x7fe3c40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3da59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0db40 0x55bdb3c0c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3da59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0db40 0x55bdb3c0c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57316/0 (socket says 172.21.15.5:57316) 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3da59c640 1 -- 172.21.15.5:0/2003066629 learned_addr learned my addr 172.21.15.5:0/2003066629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/2003066629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c0ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3da59c640 1 -- 172.21.15.5:0/2003066629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c17370 msgr2=0x55bdb3c19760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3da59c640 1 --2- 172.21.15.5:0/2003066629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c17370 0x55bdb3c19760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.985+0000 7fe3da59c640 1 -- 172.21.15.5:0/2003066629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 msgr2=0x55bdb3c0ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.986+0000 7fe3dad9d640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3c40a2d70 0x7fe3c40a5230 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005fd0 tx=0x7fe3d4009500 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.986+0000 7fe3da59c640 1 --2- 172.21.15.5:0/2003066629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c0ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.986+0000 7fe3da59c640 1 -- 172.21.15.5:0/2003066629 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb3afe3a0 con 0x55bdb3c0db40 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.986+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/2003066629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c0ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.986+0000 7fe3da59c640 1 --2- 172.21.15.5:0/2003066629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0db40 0x55bdb3c0c090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc098710 tx=0x7fe3cc098110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e35c384debf40c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.986+0000 7fe3c8ff9640 1 -- 172.21.15.5:0/2003066629 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc09bd40 con 0x55bdb3c0db40 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.986+0000 7fe3c8ff9640 1 -- 172.21.15.5:0/2003066629 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc0a00f0 con 0x55bdb3c0db40 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.986+0000 7fe3c8ff9640 1 -- 172.21.15.5:0/2003066629 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0a4520 con 0x55bdb3c0db40 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.987+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/2003066629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0db40 msgr2=0x55bdb3c0c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.987+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/2003066629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0db40 0x55bdb3c0c090 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc098710 tx=0x7fe3cc098110 comp rx=0 tx=0).stop 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.987+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/2003066629 shutdown_connections 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.987+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/2003066629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c17370 0x55bdb3c19760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.987+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/2003066629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c0ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.987+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/2003066629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0db40 0x55bdb3c0c090 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.987+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/2003066629 >> 172.21.15.5:0/2003066629 conn(0x55bdb3bd5310 msgr2=0x55bdb3c08c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:18.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.987+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/2003066629 shutdown_connections 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.987+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/2003066629 wait complete. 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.987+0000 7fe3e2d0c740 1 -- start start 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3e2d0c740 1 Processor -- start 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c16e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 0x55bdb3c14840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bdb3c1a070 con 0x55bdb3c0db40 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bdb3b2d060 con 0x55bdb3c17370 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bdb3c1a1f0 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3e2d0c740 10 client.0 _ll_register_callbacks cb 0x55bdb38808c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48484/0 (socket says 172.21.15.5:48484) 2024-09-11T22:35:18.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3dad9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 0x55bdb3c14840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3d9d9b640 1 -- 172.21.15.5:0/59293891 learned_addr learned my addr 172.21.15.5:0/59293891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3da59c640 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c16e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3d9d9b640 1 -- 172.21.15.5:0/59293891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0c5d0 msgr2=0x55bdb3c16e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c16e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3d9d9b640 1 -- 172.21.15.5:0/59293891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 msgr2=0x55bdb3c14840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 0x55bdb3c14840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.988+0000 7fe3d9d9b640 1 -- 172.21.15.5:0/59293891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb38e33c0 con 0x55bdb3c0db40 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3dad9d640 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 0x55bdb3c14840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3da59c640 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c16e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c140b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009900 tx=0x7fe3d00317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ebc3254503f919 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/59293891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0035040 con 0x55bdb3c0db40 2024-09-11T22:35:18.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/59293891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7fe3d0004240 con 0x55bdb3c0db40 2024-09-11T22:35:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/59293891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3cc0a00f0 con 0x55bdb3c0db40 2024-09-11T22:35:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/59293891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe3d0004440 con 0x55bdb3c0db40 2024-09-11T22:35:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3e2d0c740 10 client.19533 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/59293891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55bdb3c25c10 con 0x55bdb3c0db40 2024-09-11T22:35:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.989+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/59293891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc0a00f0 con 0x55bdb3c0db40 2024-09-11T22:35:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.990+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/59293891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fe3d0004bf0 con 0x55bdb3c0db40 2024-09-11T22:35:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.990+0000 7fe3e2d0c740 10 client.19533 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.990+0000 7fe3e2d0c740 10 client.19533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.990+0000 7fe3e2d0c740 20 client.19533 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.990+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bdb3c27020 0x55bdb3c29440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.990+0000 7fe3e2d0c740 4 client.19533 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.990+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/59293891 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdb3b7f860 con 0x55bdb3c27020 2024-09-11T22:35:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.990+0000 7fe3da59c640 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bdb3c27020 0x55bdb3c29440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.990+0000 7fe3c1ffb640 10 client.19533 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.990+0000 7fe3da59c640 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bdb3c27020 0x55bdb3c29440 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44219115def33d32 server_cookie=ae0520fc6d5f7e2e in_seq=0 out_seq=0 2024-09-11T22:35:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/59293891 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdb3b7f860 con 0x55bdb3c27020 2024-09-11T22:35:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3c1ffb640 10 client.19533 handle_command_reply: tid=0 2024-09-11T22:35:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 client.19533 shutdown 2024-09-11T22:35:18.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/59293891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 msgr2=0x55bdb3c140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c140b0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009900 tx=0x7fe3d00317e0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/59293891 shutdown_connections 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bdb3c27020 0x55bdb3c29440 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 0x55bdb3c14840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c140b0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/59293891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c16e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3c1ffb640 0 client.19533 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/59293891 >> 172.21.15.5:0/59293891 conn(0x55bdb3bd5310 msgr2=0x55bdb3c0e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/59293891 shutdown_connections 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.991+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/59293891 wait complete. 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:17.992+0000 7fe3e2d0c740 20 client.19533 trim_cache size 0 max 16384 2024-09-11T22:35:18.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3e2d0c740 1 Processor -- start 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3e2d0c740 1 -- start start 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c08d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c12f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c17370 0x55bdb3c2ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1afbc0 con 0x55bdb3c0db40 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b1730 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b0290 con 0x55bdb3c17370 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c12f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c12f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48492/0 (socket says 172.21.15.5:48492) 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3d9d9b640 1 -- 172.21.15.5:0/2270317487 learned_addr learned my addr 172.21.15.5:0/2270317487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3da59c640 1 --2- 172.21.15.5:0/2270317487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c08d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.006+0000 7fe3dad9d640 1 --2- 172.21.15.5:0/2270317487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c17370 0x55bdb3c2ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.007+0000 7fe3d9d9b640 1 -- 172.21.15.5:0/2270317487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c17370 msgr2=0x55bdb3c2ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:18.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.007+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/2270317487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c17370 0x55bdb3c2ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.007+0000 7fe3d9d9b640 1 -- 172.21.15.5:0/2270317487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0c5d0 msgr2=0x55bdb3c08d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.007+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/2270317487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c08d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.007+0000 7fe3d9d9b640 1 -- 172.21.15.5:0/2270317487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb3c12600 con 0x55bdb3c0db40 2024-09-11T22:35:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.007+0000 7fe3dad9d640 1 --2- 172.21.15.5:0/2270317487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c17370 0x55bdb3c2ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.007+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/2270317487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c12f60 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0005940 tx=0x7fe3d00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45554242abc7ac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.007+0000 7fe3c3fff640 1 -- 172.21.15.5:0/2270317487 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0035030 con 0x55bdb3c0db40 2024-09-11T22:35:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.007+0000 7fe3c3fff640 1 -- 172.21.15.5:0/2270317487 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d0005510 con 0x55bdb3c0db40 2024-09-11T22:35:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.007+0000 7fe3c3fff640 1 -- 172.21.15.5:0/2270317487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0004170 con 0x55bdb3c0db40 2024-09-11T22:35:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.008+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/2270317487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 msgr2=0x55bdb3c12f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.008+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/2270317487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c12f60 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0005940 tx=0x7fe3d00045e0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.008+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/2270317487 shutdown_connections 2024-09-11T22:35:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.008+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/2270317487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c17370 0x55bdb3c2ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.008+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/2270317487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0db40 0x55bdb3c12f60 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.008+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/2270317487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c08d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.008+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/2270317487 >> 172.21.15.5:0/2270317487 conn(0x55bdb3bd5310 msgr2=0x55bdb3c12850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.008+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/2270317487 shutdown_connections 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.008+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/2270317487 wait complete. 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3e2d0c740 1 -- start start 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3e2d0c740 1 Processor -- start 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c1f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0db40 0x55bdb3c10b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3e2d0c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 0x55bdb3c110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b1060 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1b0990 con 0x55bdb3c17370 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3e2d0c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3dc1adbc0 con 0x55bdb3c0db40 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3e2d0c740 10 client.0 _ll_register_callbacks cb 0x7fe3d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c1f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c1f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48506/0 (socket says 172.21.15.5:48506) 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3da59c640 1 -- 172.21.15.5:0/4161811749 learned_addr learned my addr 172.21.15.5:0/4161811749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.009+0000 7fe3dad9d640 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 0x55bdb3c110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0db40 0x55bdb3c10b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3da59c640 1 -- 172.21.15.5:0/4161811749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0db40 msgr2=0x55bdb3c10b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3da59c640 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0db40 0x55bdb3c10b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3da59c640 1 -- 172.21.15.5:0/4161811749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 msgr2=0x55bdb3c110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3da59c640 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 0x55bdb3c110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3da59c640 1 -- 172.21.15.5:0/4161811749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb3c116b0 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0db40 0x55bdb3c10b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3da59c640 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c1f970 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc09ba40 tx=0x7fe3cc0056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b9fa779ef05296 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/4161811749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0a00f0 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/4161811749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7fe3cc0983b0 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/4161811749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d0005510 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/4161811749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe3cc0a4520 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3e2d0c740 10 client.19539 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.010+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/4161811749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55bdb3c91b00 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.011+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/4161811749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc0a46c0 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.011+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/4161811749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7fe3cc09b190 con 0x55bdb3c0c5d0 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.011+0000 7fe3e2d0c740 10 client.19539 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.011+0000 7fe3e2d0c740 10 client.19539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.011+0000 7fe3e2d0c740 20 client.19539 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.011+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bdb3c91de0 0x55bdb3c941d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.011+0000 7fe3e2d0c740 4 client.19539 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.011+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/4161811749 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe3d0005510 con 0x55bdb3c91de0 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.011+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bdb3c91de0 0x55bdb3c941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.012+0000 7fe3d9d9b640 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bdb3c91de0 0x55bdb3c941d0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87125e9d71bc481f server_cookie=ff5e5e791ba353b4 in_seq=0 out_seq=0 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.012+0000 7fe3c1ffb640 10 client.19539 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.013+0000 7fe3c1ffb640 1 -- 172.21.15.5:0/4161811749 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe3d0005510 con 0x55bdb3c91de0 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.013+0000 7fe3c1ffb640 10 client.19539 handle_command_reply: tid=0 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.013+0000 7fe3e2d0c740 1 client.19539 shutdown 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.013+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/4161811749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 msgr2=0x55bdb3c1f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.013+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c1f970 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc09ba40 tx=0x7fe3cc0056f0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.014+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/4161811749 shutdown_connections 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.014+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bdb3c91de0 0x55bdb3c941d0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.014+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bdb3c17370 0x55bdb3c110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.014+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bdb3c0db40 0x55bdb3c10b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.014+0000 7fe3c1ffb640 0 client.19539 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.014+0000 7fe3e2d0c740 1 --2- 172.21.15.5:0/4161811749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bdb3c0c5d0 0x55bdb3c1f970 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.014+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/4161811749 >> 172.21.15.5:0/4161811749 conn(0x55bdb3bd5310 msgr2=0x55bdb3c0e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.014+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/4161811749 shutdown_connections 2024-09-11T22:35:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.014+0000 7fe3e2d0c740 1 -- 172.21.15.5:0/4161811749 wait complete. 2024-09-11T22:35:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.014+0000 7fe3e2d0c740 20 client.19539 trim_cache size 0 max 16384 2024-09-11T22:35:18.029 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:35:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 -- 172.21.15.5:0/341386835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3c40a2d70 msgr2=0x7fe3c40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3c40a2d70 0x7fe3c40a5230 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005fd0 tx=0x7fe3d4009500 comp rx=0 tx=0).stop 2024-09-11T22:35:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 -- 172.21.15.5:0/341386835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc101de0 msgr2=0x7fe3dc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc101de0 0x7fe3dc0744f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00e9f0 tx=0x7fe3cc00edf0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 -- 172.21.15.5:0/341386835 shutdown_connections 2024-09-11T22:35:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3dc106f10 0x7fe3dc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3c40a2d70 0x7fe3c40a5230 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3dc102790 0x7fe3dc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 --2- 172.21.15.5:0/341386835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc101de0 0x7fe3dc0744f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 -- 172.21.15.5:0/341386835 >> 172.21.15.5:0/341386835 conn(0x7fe3dc0fd8c0 msgr2=0x7fe3dc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 -- 172.21.15.5:0/341386835 shutdown_connections 2024-09-11T22:35:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:18.015+0000 7fe3e0933640 1 -- 172.21.15.5:0/341386835 wait complete. 2024-09-11T22:35:18.350 INFO:tasks.workunit.client.0.smithi005.stdout: 1 969753 7.12 MB/sec execute 576 sec latency 96.991 ms 2024-09-11T22:35:19.350 INFO:tasks.workunit.client.0.smithi005.stdout: 1 970905 7.12 MB/sec execute 577 sec latency 13.879 ms 2024-09-11T22:35:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:19 smithi005 ceph-mon[30943]: pgmap v418: 97 pgs: 97 active+clean; 111 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-09-11T22:35:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:19 smithi159 ceph-mon[35534]: pgmap v418: 97 pgs: 97 active+clean; 111 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-09-11T22:35:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:19 smithi172 ceph-mon[35461]: pgmap v418: 97 pgs: 97 active+clean; 111 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-09-11T22:35:19.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:35:19.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:35:20.144 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:20.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 973263 7.13 MB/sec execute 578 sec latency 17.663 ms 2024-09-11T22:35:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.688+0000 7f1daafd3640 1 -- 172.21.15.5:0/3640186966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1da4102570 msgr2=0x7f1da41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.688+0000 7f1daafd3640 1 --2- 172.21.15.5:0/3640186966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1da4102570 0x7f1da41029b0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d9802a480 comp rx=0 tx=0).stop 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 -- 172.21.15.5:0/3640186966 shutdown_connections 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 --2- 172.21.15.5:0/3640186966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da4102ef0 0x7f1da410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 --2- 172.21.15.5:0/3640186966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1da4102570 0x7f1da41029b0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 --2- 172.21.15.5:0/3640186966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da4100e20 0x7f1da4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 -- 172.21.15.5:0/3640186966 >> 172.21.15.5:0/3640186966 conn(0x7f1da40fc7d0 msgr2=0x7f1da40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 -- 172.21.15.5:0/3640186966 shutdown_connections 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 -- 172.21.15.5:0/3640186966 wait complete. 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 Processor -- start 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 -- start start 2024-09-11T22:35:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1da4100e20 0x7f1da419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da4102570 0x7f1da419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da4102ef0 0x7f1da419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4110f80 con 0x7f1da4100e20 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4110e00 con 0x7f1da4102ef0 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.689+0000 7f1daafd3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4111100 con 0x7f1da4102570 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da8d6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1da4100e20 0x7f1da419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da4102570 0x7f1da419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da8d6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1da4100e20 0x7f1da419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48518/0 (socket says 172.21.15.5:48518) 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da4102570 0x7f1da419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36718/0 (socket says 172.21.15.5:36718) 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da8d6f640 1 -- 172.21.15.5:0/2207128936 learned_addr learned my addr 172.21.15.5:0/2207128936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da9570640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da4102ef0 0x7f1da419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da8d6f640 1 -- 172.21.15.5:0/2207128936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da4102570 msgr2=0x7f1da419ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da8d6f640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da4102570 0x7f1da419ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da8d6f640 1 -- 172.21.15.5:0/2207128936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da4102ef0 msgr2=0x7f1da419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da8d6f640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da4102ef0 0x7f1da419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da8d6f640 1 -- 172.21.15.5:0/2207128936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da419f6f0 con 0x7f1da4100e20 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da3fff640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da4102570 0x7f1da419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da8d6f640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1da4100e20 0x7f1da419a740 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00ebc0 tx=0x7f1d9c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da1ffb640 1 -- 172.21.15.5:0/2207128936 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c010630 con 0x7f1da4100e20 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da1ffb640 1 -- 172.21.15.5:0/2207128936 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c010cb0 con 0x7f1da4100e20 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1daafd3640 1 -- 172.21.15.5:0/2207128936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da419f9e0 con 0x7f1da4100e20 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.690+0000 7f1da1ffb640 1 -- 172.21.15.5:0/2207128936 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c018bd0 con 0x7f1da4100e20 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.691+0000 7f1daafd3640 1 -- 172.21.15.5:0/2207128936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da41a72c0 con 0x7f1da4100e20 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.693+0000 7f1da1ffb640 1 -- 172.21.15.5:0/2207128936 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1d9c0107d0 con 0x7f1da4100e20 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.693+0000 7f1dad3ac740 1 Processor -- start 2024-09-11T22:35:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.693+0000 7f1dad3ac740 1 -- start start 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a077f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077f2480 0x562a077fc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a07757280 con 0x562a077fcd40 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a076feb10 con 0x562a077f1b60 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562a07753240 con 0x562a077f2480 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da8d6f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077f2480 0x562a077fc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da9570640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a077f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ff130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48530/0 (socket says 172.21.15.5:48530) 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da9570640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a077f1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57386/0 (socket says 172.21.15.5:57386) 2024-09-11T22:35:20.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da3fff640 1 -- 172.21.15.5:0/3602362262 learned_addr learned my addr 172.21.15.5:0/3602362262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da1ffb640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d8c0a2d70 0x7f1d8c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da9570640 1 -- 172.21.15.5:0/3602362262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077f2480 msgr2=0x562a077fc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da9570640 1 --2- 172.21.15.5:0/3602362262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077f2480 0x562a077fc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da9570640 1 -- 172.21.15.5:0/3602362262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 msgr2=0x562a077ff130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da9570640 1 --2- 172.21.15.5:0/3602362262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ff130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da9570640 1 -- 172.21.15.5:0/3602362262 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0770b1f0 con 0x562a077f1b60 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da3fff640 1 --2- 172.21.15.5:0/3602362262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da1ffb640 1 -- 172.21.15.5:0/2207128936 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f1d9c0cfa60 con 0x7f1da4100e20 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da9570640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d8c0a2d70 0x7f1d8c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.694+0000 7f1da9570640 1 --2- 172.21.15.5:0/3602362262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a077f1f40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009c20 tx=0x7f1d94011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198573202a7d44ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.695+0000 7f1da9570640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d8c0a2d70 0x7f1d8c0a5230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d94023000 tx=0x7f1d94024000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.695+0000 7f1d8affd640 1 -- 172.21.15.5:0/3602362262 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94017070 con 0x562a077f1b60 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.695+0000 7f1d8affd640 1 -- 172.21.15.5:0/3602362262 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9401c910 con 0x562a077f1b60 2024-09-11T22:35:20.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.695+0000 7f1d8affd640 1 -- 172.21.15.5:0/3602362262 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9400f780 con 0x562a077f1b60 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.695+0000 7f1dad3ac740 1 -- 172.21.15.5:0/3602362262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 msgr2=0x562a077f1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.695+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/3602362262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a077f1f40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d94009c20 tx=0x7f1d94011810 comp rx=0 tx=0).stop 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.695+0000 7f1dad3ac740 1 -- 172.21.15.5:0/3602362262 shutdown_connections 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.695+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/3602362262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.695+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/3602362262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077f2480 0x562a077fc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/3602362262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a077f1f40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 -- 172.21.15.5:0/3602362262 >> 172.21.15.5:0/3602362262 conn(0x562a077ee0f0 msgr2=0x562a077ee4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 -- 172.21.15.5:0/3602362262 shutdown_connections 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 -- 172.21.15.5:0/3602362262 wait complete. 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 -- start start 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 Processor -- start 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f1b60 0x562a077f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f2480 0x562a077fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077fcd40 0x562a077fa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a077ff910 con 0x562a077f1b60 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a07711f10 con 0x562a077f2480 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.696+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562a077ffa90 con 0x562a077fcd40 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1dad3ac740 10 client.0 _ll_register_callbacks cb 0x562a076daa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da8d6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f1b60 0x562a077f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da8d6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f1b60 0x562a077f9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48546/0 (socket says 172.21.15.5:48546) 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da8d6f640 1 -- 172.21.15.5:0/2856325403 learned_addr learned my addr 172.21.15.5:0/2856325403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da3fff640 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f2480 0x562a077fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da9570640 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077fcd40 0x562a077fa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da8d6f640 1 -- 172.21.15.5:0/2856325403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077fcd40 msgr2=0x562a077fa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da8d6f640 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077fcd40 0x562a077fa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da8d6f640 1 -- 172.21.15.5:0/2856325403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f2480 msgr2=0x562a077fa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da8d6f640 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f2480 0x562a077fa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da8d6f640 1 -- 172.21.15.5:0/2856325403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a074c86a0 con 0x562a077f1b60 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da3fff640 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f2480 0x562a077fa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1da8d6f640 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f1b60 0x562a077f9ca0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c098990 tx=0x7f1d9c0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e06f2a0fc2c4f0e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1d83fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1d83fff640 1 -- 172.21.15.5:0/2856325403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0d6060 con 0x562a077f1b60 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1dad3ac740 1 -- 172.21.15.5:0/2856325403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d9401c910 con 0x562a077f1b60 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1d83fff640 1 -- 172.21.15.5:0/2856325403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f1d9c0bab50 con 0x562a077f1b60 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1d83fff640 1 -- 172.21.15.5:0/2856325403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1d9c010e60 con 0x562a077f1b60 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1dad3ac740 10 client.19551 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.697+0000 7f1dad3ac740 1 -- 172.21.15.5:0/2856325403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562a0780b5e0 con 0x562a077f1b60 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.698+0000 7f1d83fff640 1 -- 172.21.15.5:0/2856325403 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9401c910 con 0x562a077f1b60 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.698+0000 7f1d83fff640 1 -- 172.21.15.5:0/2856325403 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f1d9c096b30 con 0x562a077f1b60 2024-09-11T22:35:20.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.698+0000 7f1dad3ac740 10 client.19551 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.698+0000 7f1dad3ac740 10 client.19551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.698+0000 7f1dad3ac740 20 client.19551 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.698+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a0780caa0 0x562a0780ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.698+0000 7f1dad3ac740 4 client.19551 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.698+0000 7f1dad3ac740 1 -- 172.21.15.5:0/2856325403 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a077647f0 con 0x562a0780caa0 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.698+0000 7f1da3fff640 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a0780caa0 0x562a0780ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.699+0000 7f1d83fff640 10 client.19551 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.699+0000 7f1da3fff640 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a0780caa0 0x562a0780ef10 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5c652dae7bd807c server_cookie=4c39a64808c23d99 in_seq=0 out_seq=0 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1d83fff640 1 -- 172.21.15.5:0/2856325403 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a077647f0 con 0x562a0780caa0 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1d83fff640 10 client.19551 handle_command_reply: tid=0 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 client.19551 shutdown 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 -- 172.21.15.5:0/2856325403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f1b60 msgr2=0x562a077f9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f1b60 0x562a077f9ca0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c098990 tx=0x7f1d9c0099b0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 -- 172.21.15.5:0/2856325403 shutdown_connections 2024-09-11T22:35:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a0780caa0 0x562a0780ef10 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077fcd40 0x562a077fa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f2480 0x562a077fa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/2856325403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f1b60 0x562a077f9ca0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 -- 172.21.15.5:0/2856325403 >> 172.21.15.5:0/2856325403 conn(0x562a077ee0f0 msgr2=0x562a077ed270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1d83fff640 0 client.19551 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 -- 172.21.15.5:0/2856325403 shutdown_connections 2024-09-11T22:35:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 1 -- 172.21.15.5:0/2856325403 wait complete. 2024-09-11T22:35:20.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.700+0000 7f1dad3ac740 20 client.19551 trim_cache size 0 max 16384 2024-09-11T22:35:20.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1dad3ac740 1 Processor -- start 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1dad3ac740 1 -- start start 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a077f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077f2480 0x562a077f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ed160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41afc20 con 0x562a077fcd40 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b1790 con 0x562a077f1b60 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b02f0 con 0x562a077f2480 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1da9570640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ed160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077f2480 0x562a077f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1da9570640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ed160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48548/0 (socket says 172.21.15.5:48548) 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1da8d6f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a077f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1da9570640 1 -- 172.21.15.5:0/725823648 learned_addr learned my addr 172.21.15.5:0/725823648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1da9570640 1 -- 172.21.15.5:0/725823648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077f2480 msgr2=0x562a077f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1da9570640 1 --2- 172.21.15.5:0/725823648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077f2480 0x562a077f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1da9570640 1 -- 172.21.15.5:0/725823648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 msgr2=0x562a077f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1da9570640 1 --2- 172.21.15.5:0/725823648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a077f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.718+0000 7f1da9570640 1 -- 172.21.15.5:0/725823648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a077fc570 con 0x562a077fcd40 2024-09-11T22:35:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.719+0000 7f1da9570640 1 --2- 172.21.15.5:0/725823648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ed160 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f1d940040c0 tx=0x7f1d94004050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3378b001a32e934 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.719+0000 7f1d8a7fc640 1 -- 172.21.15.5:0/725823648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94017040 con 0x562a077fcd40 2024-09-11T22:35:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.719+0000 7f1d8a7fc640 1 -- 172.21.15.5:0/725823648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d94036070 con 0x562a077fcd40 2024-09-11T22:35:20.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.719+0000 7f1d8a7fc640 1 -- 172.21.15.5:0/725823648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d94004bb0 con 0x562a077fcd40 2024-09-11T22:35:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.719+0000 7f1dad3ac740 1 -- 172.21.15.5:0/725823648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 msgr2=0x562a077ed160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.719+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/725823648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ed160 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f1d940040c0 tx=0x7f1d94004050 comp rx=0 tx=0).stop 2024-09-11T22:35:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.720+0000 7f1dad3ac740 1 -- 172.21.15.5:0/725823648 shutdown_connections 2024-09-11T22:35:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.720+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/725823648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077fcd40 0x562a077ed160 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.720+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/725823648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077f2480 0x562a077f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.720+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/725823648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a077f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.720+0000 7f1dad3ac740 1 -- 172.21.15.5:0/725823648 >> 172.21.15.5:0/725823648 conn(0x562a077ee0f0 msgr2=0x562a077eca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.720+0000 7f1dad3ac740 1 -- 172.21.15.5:0/725823648 shutdown_connections 2024-09-11T22:35:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.720+0000 7f1dad3ac740 1 -- 172.21.15.5:0/725823648 wait complete. 2024-09-11T22:35:20.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.720+0000 7f1dad3ac740 1 -- start start 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.720+0000 7f1dad3ac740 1 Processor -- start 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a0786fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f2480 0x562a07870070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1dad3ac740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077fcd40 0x562a07800280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b10c0 con 0x562a077f2480 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41b09f0 con 0x562a077f1b60 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1dad3ac740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1da41adc20 con 0x562a077fcd40 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1dad3ac740 10 client.0 _ll_register_callbacks cb 0x7f1d9c00eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f2480 0x562a07870070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f2480 0x562a07870070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48554/0 (socket says 172.21.15.5:48554) 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da8d6f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a0786fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da3fff640 1 -- 172.21.15.5:0/1343869013 learned_addr learned my addr 172.21.15.5:0/1343869013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:20.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da9570640 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077fcd40 0x562a07800280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da3fff640 1 -- 172.21.15.5:0/1343869013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077fcd40 msgr2=0x562a07800280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da3fff640 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077fcd40 0x562a07800280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da3fff640 1 -- 172.21.15.5:0/1343869013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 msgr2=0x562a0786fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da3fff640 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a0786fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da3fff640 1 -- 172.21.15.5:0/1343869013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a07800a80 con 0x562a077f2480 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1da3fff640 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f2480 0x562a07870070 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f1d9802a9a0 tx=0x7f1d98003640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d49ba13d1c09ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1d83fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1d83fff640 1 -- 172.21.15.5:0/1343869013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98035040 con 0x562a077f2480 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.721+0000 7f1d83fff640 1 -- 172.21.15.5:0/1343869013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f1d98003c80 con 0x562a077f2480 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1dad3ac740 1 -- 172.21.15.5:0/1343869013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d94036070 con 0x562a077f2480 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1d83fff640 1 -- 172.21.15.5:0/1343869013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1d98056050 con 0x562a077f2480 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1dad3ac740 10 client.19557 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1dad3ac740 1 -- 172.21.15.5:0/1343869013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562a078773a0 con 0x562a077f2480 2024-09-11T22:35:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1d83fff640 1 -- 172.21.15.5:0/1343869013 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d94036070 con 0x562a077f2480 2024-09-11T22:35:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1d83fff640 1 -- 172.21.15.5:0/1343869013 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f1d98046560 con 0x562a077f2480 2024-09-11T22:35:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1dad3ac740 10 client.19557 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1dad3ac740 10 client.19557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1dad3ac740 20 client.19557 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a07877b50 0x562a07879f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1dad3ac740 4 client.19557 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:35:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1dad3ac740 1 -- 172.21.15.5:0/1343869013 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562a0775ed00 con 0x562a07877b50 2024-09-11T22:35:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.722+0000 7f1da8d6f640 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a07877b50 0x562a07879f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:20.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.723+0000 7f1d83fff640 10 client.19557 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:20.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.723+0000 7f1da8d6f640 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a07877b50 0x562a07879f70 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562c20077d2677a8 server_cookie=cfeb933ea674f154 in_seq=0 out_seq=0 2024-09-11T22:35:20.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1d83fff640 1 -- 172.21.15.5:0/1343869013 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a0775ed00 con 0x562a07877b50 2024-09-11T22:35:20.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1d83fff640 10 client.19557 handle_command_reply: tid=0 2024-09-11T22:35:20.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 client.19557 shutdown 2024-09-11T22:35:20.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 -- 172.21.15.5:0/1343869013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f2480 msgr2=0x562a07870070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:20.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f2480 0x562a07870070 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f1d9802a9a0 tx=0x7f1d98003640 comp rx=0 tx=0).stop 2024-09-11T22:35:20.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 -- 172.21.15.5:0/1343869013 shutdown_connections 2024-09-11T22:35:20.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562a07877b50 0x562a07879f70 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562a077fcd40 0x562a07800280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a077f2480 0x562a07870070 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 --2- 172.21.15.5:0/1343869013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a077f1b60 0x562a0786fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 -- 172.21.15.5:0/1343869013 >> 172.21.15.5:0/1343869013 conn(0x562a077ee0f0 msgr2=0x562a077fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1d83fff640 0 client.19557 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 -- 172.21.15.5:0/1343869013 shutdown_connections 2024-09-11T22:35:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.724+0000 7f1dad3ac740 1 -- 172.21.15.5:0/1343869013 wait complete. 2024-09-11T22:35:20.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1dad3ac740 20 client.19557 trim_cache size 0 max 16384 2024-09-11T22:35:20.730 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:35:20.730 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:35:20.730 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "673ed8d8-a53c-42e9-80f0-85b8c52dc7df", 2024-09-11T22:35:20.730 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:35:20.731 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:35:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 -- 172.21.15.5:0/2207128936 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d8c0a2d70 msgr2=0x7f1d8c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d8c0a2d70 0x7f1d8c0a5230 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1d94023000 tx=0x7f1d94024000 comp rx=0 tx=0).stop 2024-09-11T22:35:20.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 -- 172.21.15.5:0/2207128936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1da4100e20 msgr2=0x7f1da419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1da4100e20 0x7f1da419a740 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c00ebc0 tx=0x7f1d9c00c6a0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 -- 172.21.15.5:0/2207128936 shutdown_connections 2024-09-11T22:35:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1d8c0a2d70 0x7f1d8c0a5230 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1da4102ef0 0x7f1da419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1da4102570 0x7f1da419ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 --2- 172.21.15.5:0/2207128936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1da4100e20 0x7f1da419a740 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 -- 172.21.15.5:0/2207128936 >> 172.21.15.5:0/2207128936 conn(0x7f1da40fc7d0 msgr2=0x7f1da406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 -- 172.21.15.5:0/2207128936 shutdown_connections 2024-09-11T22:35:20.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:20.725+0000 7f1daafd3640 1 -- 172.21.15.5:0/2207128936 wait complete. 2024-09-11T22:35:21.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 974258 7.13 MB/sec execute 579 sec latency 26.145 ms 2024-09-11T22:35:21.544 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:35:21.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:21 smithi005 ceph-mon[30943]: pgmap v419: 97 pgs: 97 active+clean; 111 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-09-11T22:35:21.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:21 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:35:21.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:21 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:35:21.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:21 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:35:21.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:21 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:35:21.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:21 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:35:21.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:21 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:35:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:21 smithi159 ceph-mon[35534]: pgmap v419: 97 pgs: 97 active+clean; 111 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-09-11T22:35:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:21 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:35:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:21 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:35:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:21 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:35:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:21 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:35:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:21 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:35:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:21 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:35:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:21 smithi172 ceph-mon[35461]: pgmap v419: 97 pgs: 97 active+clean; 111 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 172 op/s 2024-09-11T22:35:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:21 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:35:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:21 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:35:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:21 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:35:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:21 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:35:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:21 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:35:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:21 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.311+0000 7f21279f4640 1 -- 172.21.15.5:0/1065843144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120102d90 msgr2=0x7f2120103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.311+0000 7f21279f4640 1 --2- 172.21.15.5:0/1065843144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120102d90 0x7f2120103170 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f211c009930 tx=0x7f211c02a420 comp rx=0 tx=0).stop 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.311+0000 7f21279f4640 1 -- 172.21.15.5:0/1065843144 shutdown_connections 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.311+0000 7f21279f4640 1 --2- 172.21.15.5:0/1065843144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120108290 0x7f212010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.311+0000 7f21279f4640 1 --2- 172.21.15.5:0/1065843144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2120103740 0x7f2120107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.311+0000 7f21279f4640 1 --2- 172.21.15.5:0/1065843144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120102d90 0x7f2120103170 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.311+0000 7f21279f4640 1 -- 172.21.15.5:0/1065843144 >> 172.21.15.5:0/1065843144 conn(0x7f21200fc7d0 msgr2=0x7f21200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.311+0000 7f21279f4640 1 -- 172.21.15.5:0/1065843144 shutdown_connections 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.311+0000 7f21279f4640 1 -- 172.21.15.5:0/1065843144 wait complete. 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f21279f4640 1 Processor -- start 2024-09-11T22:35:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f21279f4640 1 -- start start 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f21279f4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2120102d90 0x7f2120198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f21279f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120103740 0x7f2120198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f21279f4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120108290 0x7f212019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f21279f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21201120d0 con 0x7f2120103740 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f21279f4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2120111f50 con 0x7f2120108290 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f21279f4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2120112250 con 0x7f2120102d90 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2125790640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2120102d90 0x7f2120198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2124f8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120103740 0x7f2120198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2125790640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2120102d90 0x7f2120198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:39240/0 (socket says 172.21.15.5:39240) 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2125790640 1 -- 172.21.15.5:0/394868803 learned_addr learned my addr 172.21.15.5:0/394868803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2125f91640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120108290 0x7f212019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2124f8f640 1 -- 172.21.15.5:0/394868803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2120102d90 msgr2=0x7f2120198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2124f8f640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2120102d90 0x7f2120198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2124f8f640 1 -- 172.21.15.5:0/394868803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120108290 msgr2=0x7f212019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2124f8f640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120108290 0x7f212019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2124f8f640 1 -- 172.21.15.5:0/394868803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212019d5f0 con 0x7f2120103740 2024-09-11T22:35:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.312+0000 7f2125f91640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120108290 0x7f212019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.313+0000 7f2124f8f640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120103740 0x7f2120198b80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f211000b3c0 tx=0x7f211000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.313+0000 7f21167fc640 1 -- 172.21.15.5:0/394868803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110011830 con 0x7f2120103740 2024-09-11T22:35:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.313+0000 7f21167fc640 1 -- 172.21.15.5:0/394868803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2110016070 con 0x7f2120103740 2024-09-11T22:35:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.313+0000 7f21279f4640 1 -- 172.21.15.5:0/394868803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212019d8e0 con 0x7f2120103740 2024-09-11T22:35:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.313+0000 7f21167fc640 1 -- 172.21.15.5:0/394868803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211000f2e0 con 0x7f2120103740 2024-09-11T22:35:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.313+0000 7f21279f4640 1 -- 172.21.15.5:0/394868803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21201a51c0 con 0x7f2120103740 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.316+0000 7f21167fc640 1 -- 172.21.15.5:0/394868803 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f211001b460 con 0x7f2120103740 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2129dcd740 1 Processor -- start 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2129dcd740 1 -- start start 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 0x55692da4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4eca0 0x55692da5d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da5da90 0x55692da5fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2129dcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55692d9b42e0 con 0x55692da4e380 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2129dcd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55692d95bce0 con 0x55692da4eca0 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2129dcd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55692d9b02a0 con 0x55692da5da90 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f21167fc640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f210c0a2d70 0x7f210c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2125790640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4eca0 0x55692da5d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2125790640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4eca0 0x55692da5d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52690/0 (socket says 172.21.15.5:52690) 2024-09-11T22:35:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2125f91640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f210c0a2d70 0x7f210c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2125790640 1 -- 172.21.15.5:0/1950671922 learned_addr learned my addr 172.21.15.5:0/1950671922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f21167fc640 1 -- 172.21.15.5:0/394868803 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f21100ce860 con 0x7f2120103740 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2125f91640 1 --2- 172.21.15.5:0/1950671922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 0x55692da4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.317+0000 7f2124f8f640 1 --2- 172.21.15.5:0/1950671922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da5da90 0x55692da5fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.318+0000 7f2125790640 1 -- 172.21.15.5:0/1950671922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da5da90 msgr2=0x55692da5fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.318+0000 7f2125790640 1 --2- 172.21.15.5:0/1950671922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da5da90 0x55692da5fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.318+0000 7f2125790640 1 -- 172.21.15.5:0/1950671922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 msgr2=0x55692da4e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.318+0000 7f2125790640 1 --2- 172.21.15.5:0/1950671922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 0x55692da4e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.318+0000 7f2125790640 1 -- 172.21.15.5:0/1950671922 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692d940440 con 0x55692da4eca0 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.318+0000 7f2125790640 1 --2- 172.21.15.5:0/1950671922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4eca0 0x55692da5d550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f211c006010 tx=0x7f211c009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22876ff0496ed217 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.318+0000 7f2125f91640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f210c0a2d70 0x7f210c0a5230 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2118005fd0 tx=0x7f2118011720 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.318+0000 7f210b7fe640 1 -- 172.21.15.5:0/1950671922 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c040070 con 0x55692da4eca0 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.318+0000 7f210b7fe640 1 -- 172.21.15.5:0/1950671922 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c005020 con 0x55692da4eca0 2024-09-11T22:35:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.318+0000 7f210b7fe640 1 -- 172.21.15.5:0/1950671922 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c035040 con 0x55692da4eca0 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 -- 172.21.15.5:0/1950671922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4eca0 msgr2=0x55692da5d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 --2- 172.21.15.5:0/1950671922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4eca0 0x55692da5d550 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f211c006010 tx=0x7f211c009680 comp rx=0 tx=0).stop 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 -- 172.21.15.5:0/1950671922 shutdown_connections 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 --2- 172.21.15.5:0/1950671922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da5da90 0x55692da5fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 --2- 172.21.15.5:0/1950671922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4eca0 0x55692da5d550 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 --2- 172.21.15.5:0/1950671922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 0x55692da4e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 -- 172.21.15.5:0/1950671922 >> 172.21.15.5:0/1950671922 conn(0x55692da4aa30 msgr2=0x55692da4ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 -- 172.21.15.5:0/1950671922 shutdown_connections 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 -- 172.21.15.5:0/1950671922 wait complete. 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 -- start start 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.319+0000 7f2129dcd740 1 Processor -- start 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4e380 0x55692da5d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da5a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 0x55692da5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2129dcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55692da60660 con 0x55692da4eca0 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2129dcd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55692d96f050 con 0x55692da5da90 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2129dcd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55692da607e0 con 0x55692da4e380 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2129dcd740 10 client.0 _ll_register_callbacks cb 0x55692d9af160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2124f8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da5a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2125f91640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 0x55692da5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2124f8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da5a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34564/0 (socket says 172.21.15.5:34564) 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2124f8f640 1 -- 172.21.15.5:0/3930705697 learned_addr learned my addr 172.21.15.5:0/3930705697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2125790640 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4e380 0x55692da5d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2124f8f640 1 -- 172.21.15.5:0/3930705697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4e380 msgr2=0x55692da5d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2124f8f640 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4e380 0x55692da5d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2124f8f640 1 -- 172.21.15.5:0/3930705697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 msgr2=0x55692da5ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2124f8f640 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 0x55692da5ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2124f8f640 1 -- 172.21.15.5:0/3930705697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692d7253c0 con 0x55692da4eca0 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2124f8f640 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da5a650 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f21100cc100 tx=0x7f211009b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b032d0c0ef85dcb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.320+0000 7f2108ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2108ff9640 1 -- 172.21.15.5:0/3930705697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21100a3a90 con 0x55692da4eca0 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2108ff9640 1 -- 172.21.15.5:0/3930705697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f2110010510 con 0x55692da4eca0 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2129dcd740 1 -- 172.21.15.5:0/3930705697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f211c005020 con 0x55692da4eca0 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2108ff9640 1 -- 172.21.15.5:0/3930705697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f21100bc250 con 0x55692da4eca0 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2129dcd740 10 client.19569 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2129dcd740 1 -- 172.21.15.5:0/3930705697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55692da6c080 con 0x55692da4eca0 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2108ff9640 1 -- 172.21.15.5:0/3930705697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c005020 con 0x55692da4eca0 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2108ff9640 1 -- 172.21.15.5:0/3930705697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f21100c3f30 con 0x55692da4eca0 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2129dcd740 10 client.19569 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2129dcd740 10 client.19569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2129dcd740 20 client.19569 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2129dcd740 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55692da6d530 0x55692da6f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2129dcd740 4 client.19569 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2129dcd740 1 -- 172.21.15.5:0/3930705697 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55692d9c1850 con 0x55692da6d530 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.321+0000 7f2125790640 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55692da6d530 0x55692da6f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.322+0000 7f2125790640 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55692da6d530 0x55692da6f960 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9258281dea055454 server_cookie=37fcb83b8154706d in_seq=0 out_seq=0 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.322+0000 7f2108ff9640 10 client.19569 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2108ff9640 1 -- 172.21.15.5:0/3930705697 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55692d9c1850 con 0x55692da6d530 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2108ff9640 10 client.19569 handle_command_reply: tid=0 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 client.19569 shutdown 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 -- 172.21.15.5:0/3930705697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 msgr2=0x55692da5a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da5a650 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f21100cc100 tx=0x7f211009b5e0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 -- 172.21.15.5:0/3930705697 shutdown_connections 2024-09-11T22:35:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55692da6d530 0x55692da6f960 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2108ff9640 0 client.19569 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 0x55692da5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da5a650 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 --2- 172.21.15.5:0/3930705697 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4e380 0x55692da5d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 -- 172.21.15.5:0/3930705697 >> 172.21.15.5:0/3930705697 conn(0x55692da4aa30 msgr2=0x55692da50790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 -- 172.21.15.5:0/3930705697 shutdown_connections 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 1 -- 172.21.15.5:0/3930705697 wait complete. 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.323+0000 7f2129dcd740 20 client.19569 trim_cache size 0 max 16384 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.342+0000 7f2129dcd740 1 Processor -- start 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2129dcd740 1 -- start start 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 0x55692da69900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4eca0 0x55692da75230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 0x55692da75770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2129dcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21201ab8f0 con 0x55692da4e380 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2129dcd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f21201ad460 con 0x55692da5da90 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2129dcd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21201abfc0 con 0x55692da4eca0 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2125790640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 0x55692da69900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2124f8f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4eca0 0x55692da75230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2125790640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 0x55692da69900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34580/0 (socket says 172.21.15.5:34580) 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2125790640 1 -- 172.21.15.5:0/2747611984 learned_addr learned my addr 172.21.15.5:0/2747611984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2125f91640 1 --2- 172.21.15.5:0/2747611984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 0x55692da75770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2124f8f640 1 -- 172.21.15.5:0/2747611984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 msgr2=0x55692da75770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2124f8f640 1 --2- 172.21.15.5:0/2747611984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 0x55692da75770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2124f8f640 1 -- 172.21.15.5:0/2747611984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 msgr2=0x55692da69900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2124f8f640 1 --2- 172.21.15.5:0/2747611984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 0x55692da69900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2124f8f640 1 -- 172.21.15.5:0/2747611984 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692da59600 con 0x55692da4eca0 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2125790640 1 --2- 172.21.15.5:0/2747611984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 0x55692da69900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2125f91640 1 --2- 172.21.15.5:0/2747611984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 0x55692da75770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.343+0000 7f2124f8f640 1 --2- 172.21.15.5:0/2747611984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4eca0 0x55692da75230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f21100977f0 tx=0x7f21100046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41987d637e3f7325 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.344+0000 7f210affd640 1 -- 172.21.15.5:0/2747611984 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21100a3a90 con 0x55692da4eca0 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.344+0000 7f210affd640 1 -- 172.21.15.5:0/2747611984 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2110099120 con 0x55692da4eca0 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.344+0000 7f210affd640 1 -- 172.21.15.5:0/2747611984 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211009a960 con 0x55692da4eca0 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.344+0000 7f2129dcd740 1 -- 172.21.15.5:0/2747611984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4eca0 msgr2=0x55692da75230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.344+0000 7f2129dcd740 1 --2- 172.21.15.5:0/2747611984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4eca0 0x55692da75230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f21100977f0 tx=0x7f21100046c0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.345+0000 7f2129dcd740 1 -- 172.21.15.5:0/2747611984 shutdown_connections 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.345+0000 7f2129dcd740 1 --2- 172.21.15.5:0/2747611984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da5da90 0x55692da75770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.345+0000 7f2129dcd740 1 --2- 172.21.15.5:0/2747611984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da4eca0 0x55692da75230 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.345+0000 7f2129dcd740 1 --2- 172.21.15.5:0/2747611984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4e380 0x55692da69900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.345+0000 7f2129dcd740 1 -- 172.21.15.5:0/2747611984 >> 172.21.15.5:0/2747611984 conn(0x55692da4aa30 msgr2=0x55692da55d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.345+0000 7f2129dcd740 1 -- 172.21.15.5:0/2747611984 shutdown_connections 2024-09-11T22:35:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.345+0000 7f2129dcd740 1 -- 172.21.15.5:0/2747611984 wait complete. 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.345+0000 7f2129dcd740 1 -- start start 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.345+0000 7f2129dcd740 1 Processor -- start 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4e380 0x55692da66300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da61630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2129dcd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da5da90 0x55692da61c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2129dcd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21201acd90 con 0x55692da4eca0 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2129dcd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f21201ac6c0 con 0x55692da4e380 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2129dcd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f21201a98f0 con 0x55692da5da90 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2129dcd740 10 client.0 _ll_register_callbacks cb 0x7f211009a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2125790640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4e380 0x55692da66300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2125790640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4e380 0x55692da66300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52722/0 (socket says 172.21.15.5:52722) 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2125790640 1 -- 172.21.15.5:0/2146723181 learned_addr learned my addr 172.21.15.5:0/2146723181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2124f8f640 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da61630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2125f91640 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da5da90 0x55692da61c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2125790640 1 -- 172.21.15.5:0/2146723181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da5da90 msgr2=0x55692da61c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2125790640 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da5da90 0x55692da61c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2125790640 1 -- 172.21.15.5:0/2146723181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 msgr2=0x55692da61630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2125790640 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da61630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2125790640 1 -- 172.21.15.5:0/2146723181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692da5d6c0 con 0x55692da4e380 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2124f8f640 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da61630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2125790640 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4e380 0x55692da66300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f211c009900 tx=0x7f211c0314e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d5328d90806825 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.346+0000 7f2108ff9640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2108ff9640 1 -- 172.21.15.5:0/2146723181 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c040040 con 0x55692da4e380 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2129dcd740 1 -- 172.21.15.5:0/2146723181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2110099120 con 0x55692da4e380 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2108ff9640 1 -- 172.21.15.5:0/2146723181 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f211c005450 con 0x55692da4e380 2024-09-11T22:35:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2108ff9640 1 -- 172.21.15.5:0/2146723181 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f211c035040 con 0x55692da4e380 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2129dcd740 10 client.28769 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2129dcd740 1 -- 172.21.15.5:0/2146723181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55692dad81e0 con 0x55692da4e380 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2108ff9640 1 -- 172.21.15.5:0/2146723181 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2110099120 con 0x55692da4e380 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2108ff9640 1 -- 172.21.15.5:0/2146723181 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f211c03c4d0 con 0x55692da4e380 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2129dcd740 10 client.28769 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2129dcd740 10 client.28769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2129dcd740 20 client.28769 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2129dcd740 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55692da6d530 0x55692dada550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2129dcd740 4 client.28769 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.347+0000 7f2129dcd740 1 -- 172.21.15.5:0/2146723181 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55692d9bbd60 con 0x55692da6d530 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.348+0000 7f2124f8f640 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55692da6d530 0x55692dada550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.348+0000 7f2124f8f640 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55692da6d530 0x55692dada550 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a844f6bff21f71 server_cookie=e8e325f78e1beb0 in_seq=0 out_seq=0 2024-09-11T22:35:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.348+0000 7f2108ff9640 10 client.28769 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.348+0000 7f2108ff9640 1 -- 172.21.15.5:0/2146723181 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55692d9bbd60 con 0x55692da6d530 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.348+0000 7f2108ff9640 10 client.28769 handle_command_reply: tid=0 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.348+0000 7f2129dcd740 1 client.28769 shutdown 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 1 -- 172.21.15.5:0/2146723181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4e380 msgr2=0x55692da66300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4e380 0x55692da66300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f211c009900 tx=0x7f211c0314e0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 1 -- 172.21.15.5:0/2146723181 shutdown_connections 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55692da6d530 0x55692dada550 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55692da5da90 0x55692da61c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692da4eca0 0x55692da61630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 1 --2- 172.21.15.5:0/2146723181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55692da4e380 0x55692da66300 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 1 -- 172.21.15.5:0/2146723181 >> 172.21.15.5:0/2146723181 conn(0x55692da4aa30 msgr2=0x55692da5beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2108ff9640 0 client.28769 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 1 -- 172.21.15.5:0/2146723181 shutdown_connections 2024-09-11T22:35:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 1 -- 172.21.15.5:0/2146723181 wait complete. 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.349+0000 7f2129dcd740 20 client.28769 trim_cache size 0 max 16384 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "673ed8d8-a53c-42e9-80f0-85b8c52dc7df": { 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "673ed8d8-a53c-42e9-80f0-85b8c52dc7df", 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:22.366 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:35:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 -- 172.21.15.5:0/394868803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f210c0a2d70 msgr2=0x7f210c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f210c0a2d70 0x7f210c0a5230 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2118005fd0 tx=0x7f2118011720 comp rx=0 tx=0).stop 2024-09-11T22:35:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 -- 172.21.15.5:0/394868803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120103740 msgr2=0x7f2120198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120103740 0x7f2120198b80 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f211000b3c0 tx=0x7f211000b880 comp rx=0 tx=0).stop 2024-09-11T22:35:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 -- 172.21.15.5:0/394868803 shutdown_connections 2024-09-11T22:35:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f210c0a2d70 0x7f210c0a5230 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2120108290 0x7f212019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120103740 0x7f2120198b80 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.368 INFO:tasks.workunit.client.0.smithi005.stdout: 1 976439 7.14 MB/sec execute 580 sec latency 61.447 ms 2024-09-11T22:35:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 --2- 172.21.15.5:0/394868803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2120102d90 0x7f2120198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 -- 172.21.15.5:0/394868803 >> 172.21.15.5:0/394868803 conn(0x7f21200fc7d0 msgr2=0x7f2120105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 -- 172.21.15.5:0/394868803 shutdown_connections 2024-09-11T22:35:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:22.350+0000 7f21279f4640 1 -- 172.21.15.5:0/394868803 wait complete. 2024-09-11T22:35:23.132 INFO:tasks.cephfs.filesystem:scrub status for tag:673ed8d8-a53c-42e9-80f0-85b8c52dc7df - {'path': '/', 'tag': '673ed8d8-a53c-42e9-80f0-85b8c52dc7df', 'options': 'recursive,force'} 2024-09-11T22:35:23.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 977768 7.14 MB/sec execute 581 sec latency 19.238 ms 2024-09-11T22:35:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:23 smithi005 ceph-mon[30943]: pgmap v420: 97 pgs: 97 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2024-09-11T22:35:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:23 smithi159 ceph-mon[35534]: pgmap v420: 97 pgs: 97 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2024-09-11T22:35:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:23 smithi172 ceph-mon[35461]: pgmap v420: 97 pgs: 97 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 268 op/s 2024-09-11T22:35:24.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 980190 7.15 MB/sec execute 582 sec latency 22.186 ms 2024-09-11T22:35:25.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 981316 7.15 MB/sec execute 583 sec latency 22.076 ms 2024-09-11T22:35:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:25 smithi005 ceph-mon[30943]: pgmap v421: 97 pgs: 97 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-09-11T22:35:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:25 smithi159 ceph-mon[35534]: pgmap v421: 97 pgs: 97 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-09-11T22:35:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:25 smithi172 ceph-mon[35461]: pgmap v421: 97 pgs: 97 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-09-11T22:35:26.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 983474 7.15 MB/sec execute 584 sec latency 91.230 ms 2024-09-11T22:35:27.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 984697 7.15 MB/sec execute 585 sec latency 20.196 ms 2024-09-11T22:35:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:27 smithi005 ceph-mon[30943]: pgmap v422: 97 pgs: 97 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-09-11T22:35:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:27 smithi159 ceph-mon[35534]: pgmap v422: 97 pgs: 97 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-09-11T22:35:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:27 smithi172 ceph-mon[35461]: pgmap v422: 97 pgs: 97 active+clean; 106 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-09-11T22:35:28.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 986857 7.16 MB/sec execute 586 sec latency 22.418 ms 2024-09-11T22:35:29.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 988107 7.16 MB/sec execute 587 sec latency 16.211 ms 2024-09-11T22:35:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:29 smithi005 ceph-mon[30943]: pgmap v423: 97 pgs: 97 active+clean; 112 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.3 MiB/s wr, 292 op/s 2024-09-11T22:35:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:29 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:35:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:29 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:35:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:29 smithi159 ceph-mon[35534]: pgmap v423: 97 pgs: 97 active+clean; 112 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.3 MiB/s wr, 292 op/s 2024-09-11T22:35:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:29 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:35:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:29 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:35:29.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:29 smithi172 ceph-mon[35461]: pgmap v423: 97 pgs: 97 active+clean; 112 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.3 MiB/s wr, 292 op/s 2024-09-11T22:35:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:29 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:35:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:29 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:35:30.352 INFO:tasks.workunit.client.0.smithi005.stdout: 1 990288 7.16 MB/sec execute 588 sec latency 21.735 ms 2024-09-11T22:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.492+0000 7f009ff78640 1 -- 172.21.15.5:0/492430678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0098107b70 msgr2=0x7f0098107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.492+0000 7f009ff78640 1 --2- 172.21.15.5:0/492430678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0098107b70 0x7f0098107f50 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0094009930 tx=0x7f009402a420 comp rx=0 tx=0).stop 2024-09-11T22:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.492+0000 7f009ff78640 1 -- 172.21.15.5:0/492430678 shutdown_connections 2024-09-11T22:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.492+0000 7f009ff78640 1 --2- 172.21.15.5:0/492430678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0098102470 0x7f009810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.492+0000 7f009ff78640 1 --2- 172.21.15.5:0/492430678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0098101ad0 0x7f0098101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.492+0000 7f009ff78640 1 --2- 172.21.15.5:0/492430678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0098107b70 0x7f0098107f50 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.492+0000 7f009ff78640 1 -- 172.21.15.5:0/492430678 >> 172.21.15.5:0/492430678 conn(0x7f00980fd7d0 msgr2=0x7f00980ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.492+0000 7f009ff78640 1 -- 172.21.15.5:0/492430678 shutdown_connections 2024-09-11T22:35:30.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.492+0000 7f009ff78640 1 -- 172.21.15.5:0/492430678 wait complete. 2024-09-11T22:35:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009ff78640 1 Processor -- start 2024-09-11T22:35:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009ff78640 1 -- start start 2024-09-11T22:35:30.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009ff78640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0098101ad0 0x7f0098198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009ff78640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0098102470 0x7f00981988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009ff78640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0098107b70 0x7f009819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009ff78640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0098111f20 con 0x7f0098102470 2024-09-11T22:35:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009ff78640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0098111da0 con 0x7f0098107b70 2024-09-11T22:35:30.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009ff78640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00981120a0 con 0x7f0098101ad0 2024-09-11T22:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009d513640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0098102470 0x7f00981988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009d513640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0098102470 0x7f00981988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34590/0 (socket says 172.21.15.5:34590) 2024-09-11T22:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009d513640 1 -- 172.21.15.5:0/3334731664 learned_addr learned my addr 172.21.15.5:0/3334731664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009dd14640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0098101ad0 0x7f0098198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.493+0000 7f009e515640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0098107b70 0x7f009819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f009d513640 1 -- 172.21.15.5:0/3334731664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0098101ad0 msgr2=0x7f0098198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f009d513640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0098101ad0 0x7f0098198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f009d513640 1 -- 172.21.15.5:0/3334731664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0098107b70 msgr2=0x7f009819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f009d513640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0098107b70 0x7f009819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f009d513640 1 -- 172.21.15.5:0/3334731664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009819d360 con 0x7f0098102470 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f009e515640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0098107b70 0x7f009819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f009dd14640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0098101ad0 0x7f0098198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f009d513640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0098102470 0x7f00981988b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f008800ea60 tx=0x7f008800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f008effd640 1 -- 172.21.15.5:0/3334731664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008800cde0 con 0x7f0098102470 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f009ff78640 1 -- 172.21.15.5:0/3334731664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009819d650 con 0x7f0098102470 2024-09-11T22:35:30.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f008effd640 1 -- 172.21.15.5:0/3334731664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00880035f0 con 0x7f0098102470 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f008effd640 1 -- 172.21.15.5:0/3334731664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0088010670 con 0x7f0098102470 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.494+0000 7f009ff78640 1 -- 172.21.15.5:0/3334731664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00981a4ee0 con 0x7f0098102470 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.497+0000 7f00a2df1740 1 Processor -- start 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.497+0000 7f00a2df1740 1 -- start start 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c333dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c334310 0x56257c342b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c3454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f008effd640 1 -- 172.21.15.5:0/3334731664 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f00880037a0 con 0x7f0098102470 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f00a2df1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56257c2982f0 con 0x56257c3339f0 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f00a2df1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56257c240270 con 0x56257c3430c0 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f00a2df1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56257c2942b0 con 0x56257c334310 2024-09-11T22:35:30.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009e515640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c333dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009e515640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c333dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34600/0 (socket says 172.21.15.5:34600) 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009e515640 1 -- 172.21.15.5:0/132228457 learned_addr learned my addr 172.21.15.5:0/132228457 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009d513640 1 --2- 172.21.15.5:0/132228457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c3454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009dd14640 1 --2- 172.21.15.5:0/132228457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c334310 0x56257c342b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009e515640 1 -- 172.21.15.5:0/132228457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c334310 msgr2=0x56257c342b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009e515640 1 --2- 172.21.15.5:0/132228457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c334310 0x56257c342b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009e515640 1 -- 172.21.15.5:0/132228457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 msgr2=0x56257c3454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009e515640 1 --2- 172.21.15.5:0/132228457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c3454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009e515640 1 -- 172.21.15.5:0/132228457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257c23f340 con 0x56257c3339f0 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.498+0000 7f009e515640 1 --2- 172.21.15.5:0/132228457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c333dd0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f0090011790 tx=0x7f0090011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95862bf3e148f573 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.499+0000 7f0083fff640 1 -- 172.21.15.5:0/132228457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0090003c30 con 0x56257c3339f0 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.499+0000 7f0083fff640 1 -- 172.21.15.5:0/132228457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f009000fe80 con 0x56257c3339f0 2024-09-11T22:35:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.499+0000 7f0083fff640 1 -- 172.21.15.5:0/132228457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00900193e0 con 0x56257c3339f0 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.499+0000 7f008effd640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00840a2da0 0x7f00840a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.499+0000 7f009dd14640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00840a2da0 0x7f00840a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.499+0000 7f008effd640 1 -- 172.21.15.5:0/3334731664 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f00880cf760 con 0x7f0098102470 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.499+0000 7f00a2df1740 1 -- 172.21.15.5:0/132228457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 msgr2=0x56257c333dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.499+0000 7f00a2df1740 1 --2- 172.21.15.5:0/132228457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c333dd0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f0090011790 tx=0x7f0090011c50 comp rx=0 tx=0).stop 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.499+0000 7f009dd14640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00840a2da0 0x7f00840a5260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0094009900 tx=0x7f00940058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 -- 172.21.15.5:0/132228457 shutdown_connections 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 --2- 172.21.15.5:0/132228457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c3454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 --2- 172.21.15.5:0/132228457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c334310 0x56257c342b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 --2- 172.21.15.5:0/132228457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c333dd0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 -- 172.21.15.5:0/132228457 >> 172.21.15.5:0/132228457 conn(0x56257c330060 msgr2=0x56257c330440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 -- 172.21.15.5:0/132228457 shutdown_connections 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 -- 172.21.15.5:0/132228457 wait complete. 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 -- start start 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 Processor -- start 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c33ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c334310 0x56257c3404e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c340a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56257c345d20 con 0x56257c3339f0 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56257c221290 con 0x56257c3430c0 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.500+0000 7f00a2df1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56257c345ea0 con 0x56257c334310 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f00a2df1740 10 client.0 _ll_register_callbacks cb 0x56257c252140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009dd14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c33ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009dd14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c33ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34614/0 (socket says 172.21.15.5:34614) 2024-09-11T22:35:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009dd14640 1 -- 172.21.15.5:0/2360582274 learned_addr learned my addr 172.21.15.5:0/2360582274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009e515640 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c340a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009dd14640 1 -- 172.21.15.5:0/2360582274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c334310 msgr2=0x56257c3404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009dd14640 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c334310 0x56257c3404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009dd14640 1 -- 172.21.15.5:0/2360582274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 msgr2=0x56257c340a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009dd14640 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c340a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009dd14640 1 -- 172.21.15.5:0/2360582274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257bfab210 con 0x56257c3339f0 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009e515640 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c340a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f009dd14640 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c33ffa0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f0094009930 tx=0x7f0094009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cc57ea9e12378c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.501+0000 7f00817fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.502+0000 7f00817fa640 1 -- 172.21.15.5:0/2360582274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f009404d070 con 0x56257c3339f0 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.502+0000 7f00817fa640 1 -- 172.21.15.5:0/2360582274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f0094003cc0 con 0x56257c3339f0 2024-09-11T22:35:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.502+0000 7f00a2df1740 1 -- 172.21.15.5:0/2360582274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56257c2a5a20 con 0x56257c3339f0 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.502+0000 7f00817fa640 1 -- 172.21.15.5:0/2360582274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0094043070 con 0x56257c3339f0 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.502+0000 7f00a2df1740 10 client.19587 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.502+0000 7f00a2df1740 1 -- 172.21.15.5:0/2360582274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56257c077620 con 0x56257c3339f0 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.502+0000 7f00817fa640 1 -- 172.21.15.5:0/2360582274 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56257c2a5a20 con 0x56257c3339f0 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.502+0000 7f00817fa640 1 -- 172.21.15.5:0/2360582274 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f009403d450 con 0x56257c3339f0 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.502+0000 7f00a2df1740 10 client.19587 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.502+0000 7f00a2df1740 10 client.19587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.503+0000 7f00a2df1740 20 client.19587 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.503+0000 7f00a2df1740 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56257c353090 0x56257c355480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.503+0000 7f00a2df1740 4 client.19587 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.503+0000 7f00a2df1740 1 -- 172.21.15.5:0/2360582274 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56257c29fd70 con 0x56257c353090 2024-09-11T22:35:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.503+0000 7f009d513640 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56257c353090 0x56257c355480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.503+0000 7f00817fa640 10 client.19587 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.503+0000 7f009d513640 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56257c353090 0x56257c355480 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6052e050c0bfb3d1 server_cookie=2e00bc1aac78a1a5 in_seq=0 out_seq=0 2024-09-11T22:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00817fa640 1 -- 172.21.15.5:0/2360582274 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56257c29fd70 con 0x56257c353090 2024-09-11T22:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00817fa640 10 client.19587 handle_command_reply: tid=0 2024-09-11T22:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 client.19587 shutdown 2024-09-11T22:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 -- 172.21.15.5:0/2360582274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 msgr2=0x56257c33ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c33ffa0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f0094009930 tx=0x7f0094009260 comp rx=0 tx=0).stop 2024-09-11T22:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 -- 172.21.15.5:0/2360582274 shutdown_connections 2024-09-11T22:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56257c353090 0x56257c355480 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c340a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c334310 0x56257c3404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 --2- 172.21.15.5:0/2360582274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c3339f0 0x56257c33ffa0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00817fa640 0 client.19587 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 -- 172.21.15.5:0/2360582274 >> 172.21.15.5:0/2360582274 conn(0x56257c330060 msgr2=0x56257c335dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 -- 172.21.15.5:0/2360582274 shutdown_connections 2024-09-11T22:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.504+0000 7f00a2df1740 1 -- 172.21.15.5:0/2360582274 wait complete. 2024-09-11T22:35:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.505+0000 7f00a2df1740 20 client.19587 trim_cache size 0 max 16384 2024-09-11T22:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f00a2df1740 1 Processor -- start 2024-09-11T22:35:30.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f00a2df1740 1 -- start start 2024-09-11T22:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c3339f0 0x56257c335c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c35ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c35b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f00a2df1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00981ab740 con 0x56257c334310 2024-09-11T22:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f00a2df1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00981ad2b0 con 0x56257c3430c0 2024-09-11T22:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f00a2df1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00981abe10 con 0x56257c3339f0 2024-09-11T22:35:30.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f009d513640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c35ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f009d513640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c35ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34618/0 (socket says 172.21.15.5:34618) 2024-09-11T22:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f009d513640 1 -- 172.21.15.5:0/2437779841 learned_addr learned my addr 172.21.15.5:0/2437779841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.520+0000 7f009e515640 1 --2- 172.21.15.5:0/2437779841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c35b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.521+0000 7f009dd14640 1 --2- 172.21.15.5:0/2437779841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c3339f0 0x56257c335c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.521+0000 7f009d513640 1 -- 172.21.15.5:0/2437779841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c3339f0 msgr2=0x56257c335c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.521+0000 7f009d513640 1 --2- 172.21.15.5:0/2437779841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c3339f0 0x56257c335c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.521+0000 7f009d513640 1 -- 172.21.15.5:0/2437779841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 msgr2=0x56257c35b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.521+0000 7f009d513640 1 --2- 172.21.15.5:0/2437779841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c35b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.521+0000 7f009d513640 1 -- 172.21.15.5:0/2437779841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56257c32fe90 con 0x56257c334310 2024-09-11T22:35:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.521+0000 7f009d513640 1 --2- 172.21.15.5:0/2437779841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c35ad70 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f008800c760 tx=0x7f00880d1c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c087da572d2543 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:30.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.521+0000 7f00837fe640 1 -- 172.21.15.5:0/2437779841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008809b750 con 0x56257c334310 2024-09-11T22:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.521+0000 7f00837fe640 1 -- 172.21.15.5:0/2437779841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f008809c310 con 0x56257c334310 2024-09-11T22:35:30.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.521+0000 7f00837fe640 1 -- 172.21.15.5:0/2437779841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008809b570 con 0x56257c334310 2024-09-11T22:35:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 -- 172.21.15.5:0/2437779841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 msgr2=0x56257c35ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:30.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 --2- 172.21.15.5:0/2437779841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c35ad70 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f008800c760 tx=0x7f00880d1c70 comp rx=0 tx=0).stop 2024-09-11T22:35:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 -- 172.21.15.5:0/2437779841 shutdown_connections 2024-09-11T22:35:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 --2- 172.21.15.5:0/2437779841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c35b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 --2- 172.21.15.5:0/2437779841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c35ad70 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 --2- 172.21.15.5:0/2437779841 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c3339f0 0x56257c335c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 -- 172.21.15.5:0/2437779841 >> 172.21.15.5:0/2437779841 conn(0x56257c330060 msgr2=0x56257c335530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:30.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 -- 172.21.15.5:0/2437779841 shutdown_connections 2024-09-11T22:35:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 -- 172.21.15.5:0/2437779841 wait complete. 2024-09-11T22:35:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 -- start start 2024-09-11T22:35:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.522+0000 7f00a2df1740 1 Processor -- start 2024-09-11T22:35:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c3339f0 0x56257c3468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c346de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f00a2df1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c3b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f00a2df1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00981acbe0 con 0x56257c334310 2024-09-11T22:35:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f00a2df1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00981ac510 con 0x56257c3430c0 2024-09-11T22:35:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f00a2df1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00981a9760 con 0x56257c3339f0 2024-09-11T22:35:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f00a2df1740 10 client.0 _ll_register_callbacks cb 0x7f0094009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f009d513640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c346de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f009d513640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c346de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34620/0 (socket says 172.21.15.5:34620) 2024-09-11T22:35:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f009d513640 1 -- 172.21.15.5:0/3663486194 learned_addr learned my addr 172.21.15.5:0/3663486194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:30.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f009e515640 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c3b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f009dd14640 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c3339f0 0x56257c3468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f009d513640 1 -- 172.21.15.5:0/3663486194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c3339f0 msgr2=0x56257c3468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f009d513640 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c3339f0 0x56257c3468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f009d513640 1 -- 172.21.15.5:0/3663486194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 msgr2=0x56257c3b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f009d513640 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c3b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.523+0000 7f009d513640 1 -- 172.21.15.5:0/3663486194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56257c33ad50 con 0x56257c334310 2024-09-11T22:35:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f009d513640 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c346de0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f00880985e0 tx=0x7f00880cce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1698ca6ab7f94fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00817fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:30.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00817fa640 1 -- 172.21.15.5:0/3663486194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008809b750 con 0x56257c334310 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00817fa640 1 -- 172.21.15.5:0/3663486194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f0088003b30 con 0x56257c334310 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00a2df1740 1 -- 172.21.15.5:0/3663486194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f008809c310 con 0x56257c334310 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00817fa640 1 -- 172.21.15.5:0/3663486194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f008809bc10 con 0x56257c334310 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00a2df1740 10 client.19593 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00a2df1740 1 -- 172.21.15.5:0/3663486194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56257c3bd530 con 0x56257c334310 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00817fa640 1 -- 172.21.15.5:0/3663486194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f008809c310 con 0x56257c334310 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00817fa640 1 -- 172.21.15.5:0/3663486194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f00880d1ce0 con 0x56257c334310 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00a2df1740 10 client.19593 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00a2df1740 10 client.19593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00a2df1740 20 client.19593 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.524+0000 7f00a2df1740 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56257c3bdbd0 0x56257c3bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.525+0000 7f00a2df1740 4 client.19593 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:35:30.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.525+0000 7f00a2df1740 1 -- 172.21.15.5:0/3663486194 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56257c21c900 con 0x56257c3bdbd0 2024-09-11T22:35:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.525+0000 7f009dd14640 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56257c3bdbd0 0x56257c3bfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.525+0000 7f009dd14640 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56257c3bdbd0 0x56257c3bfff0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8063cdf7af417f48 server_cookie=8a9cf09ce0803e1c in_seq=0 out_seq=0 2024-09-11T22:35:30.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.525+0000 7f00817fa640 10 client.19593 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.650+0000 7f00817fa640 1 -- 172.21.15.5:0/3663486194 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56257c21c900 con 0x56257c3bdbd0 2024-09-11T22:35:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.650+0000 7f00817fa640 10 client.19593 handle_command_reply: tid=0 2024-09-11T22:35:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.650+0000 7f00a2df1740 1 client.19593 shutdown 2024-09-11T22:35:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.650+0000 7f00a2df1740 1 -- 172.21.15.5:0/3663486194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 msgr2=0x56257c346de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.650+0000 7f00a2df1740 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c346de0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f00880985e0 tx=0x7f00880cce20 comp rx=0 tx=0).stop 2024-09-11T22:35:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.650+0000 7f00a2df1740 1 -- 172.21.15.5:0/3663486194 shutdown_connections 2024-09-11T22:35:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.650+0000 7f00a2df1740 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56257c3bdbd0 0x56257c3bfff0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.651+0000 7f00817fa640 0 client.19593 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:30.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.651+0000 7f00a2df1740 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56257c3430c0 0x56257c3b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.651+0000 7f00a2df1740 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56257c334310 0x56257c346de0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.651+0000 7f00a2df1740 1 --2- 172.21.15.5:0/3663486194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56257c3339f0 0x56257c3468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.651+0000 7f00a2df1740 1 -- 172.21.15.5:0/3663486194 >> 172.21.15.5:0/3663486194 conn(0x56257c330060 msgr2=0x56257c335d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.651+0000 7f00a2df1740 1 -- 172.21.15.5:0/3663486194 shutdown_connections 2024-09-11T22:35:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.651+0000 7f00a2df1740 1 -- 172.21.15.5:0/3663486194 wait complete. 2024-09-11T22:35:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.651+0000 7f00a2df1740 20 client.19593 trim_cache size 0 max 16384 2024-09-11T22:35:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:35:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:35:30.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:35:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:35:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.652+0000 7f009ff78640 1 -- 172.21.15.5:0/3334731664 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00840a2da0 msgr2=0x7f00840a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:30.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.652+0000 7f009ff78640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00840a2da0 0x7f00840a5260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0094009900 tx=0x7f00940058d0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.652+0000 7f009ff78640 1 -- 172.21.15.5:0/3334731664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0098102470 msgr2=0x7f00981988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.652+0000 7f009ff78640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0098102470 0x7f00981988b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f008800ea60 tx=0x7f008800ef20 comp rx=0 tx=0).stop 2024-09-11T22:35:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.652+0000 7f009ff78640 1 -- 172.21.15.5:0/3334731664 shutdown_connections 2024-09-11T22:35:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.652+0000 7f009ff78640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00840a2da0 0x7f00840a5260 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.652+0000 7f009ff78640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0098107b70 0x7f009819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.652+0000 7f009ff78640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0098102470 0x7f00981988b0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.652+0000 7f009ff78640 1 --2- 172.21.15.5:0/3334731664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0098101ad0 0x7f0098198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.652+0000 7f009ff78640 1 -- 172.21.15.5:0/3334731664 >> 172.21.15.5:0/3334731664 conn(0x7f00980fd7d0 msgr2=0x7f00980fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.653+0000 7f009ff78640 1 -- 172.21.15.5:0/3334731664 shutdown_connections 2024-09-11T22:35:30.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:30.653+0000 7f009ff78640 1 -- 172.21.15.5:0/3334731664 wait complete. 2024-09-11T22:35:31.352 INFO:tasks.workunit.client.0.smithi005.stdout: 1 991469 7.17 MB/sec execute 589 sec latency 113.919 ms 2024-09-11T22:35:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:31 smithi005 ceph-mon[30943]: pgmap v424: 97 pgs: 97 active+clean; 112 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 222 op/s 2024-09-11T22:35:31.828 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:31 smithi159 ceph-mon[35534]: pgmap v424: 97 pgs: 97 active+clean; 112 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 222 op/s 2024-09-11T22:35:31.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:31 smithi172 ceph-mon[35461]: pgmap v424: 97 pgs: 97 active+clean; 112 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 222 op/s 2024-09-11T22:35:32.353 INFO:tasks.workunit.client.0.smithi005.stdout: 1 993372 7.17 MB/sec execute 590 sec latency 35.332 ms 2024-09-11T22:35:33.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 995162 7.18 MB/sec execute 591 sec latency 12.813 ms 2024-09-11T22:35:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:33 smithi005 ceph-mon[30943]: pgmap v425: 97 pgs: 97 active+clean; 104 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-09-11T22:35:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:33 smithi159 ceph-mon[35534]: pgmap v425: 97 pgs: 97 active+clean; 104 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-09-11T22:35:33.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:33 smithi172 ceph-mon[35461]: pgmap v425: 97 pgs: 97 active+clean; 104 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.1 MiB/s wr, 359 op/s 2024-09-11T22:35:34.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 997176 7.18 MB/sec execute 592 sec latency 24.978 ms 2024-09-11T22:35:35.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 998714 7.19 MB/sec execute 593 sec latency 18.149 ms 2024-09-11T22:35:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:35 smithi005 ceph-mon[30943]: pgmap v426: 97 pgs: 97 active+clean; 104 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-09-11T22:35:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:35 smithi159 ceph-mon[35534]: pgmap v426: 97 pgs: 97 active+clean; 104 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-09-11T22:35:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:35 smithi172 ceph-mon[35461]: pgmap v426: 97 pgs: 97 active+clean; 104 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-09-11T22:35:36.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1000824 7.19 MB/sec execute 594 sec latency 25.556 ms 2024-09-11T22:35:37.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1002307 7.19 MB/sec execute 595 sec latency 20.969 ms 2024-09-11T22:35:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:37 smithi005 ceph-mon[30943]: pgmap v427: 97 pgs: 97 active+clean; 104 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-09-11T22:35:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:37 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:35:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:37 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:35:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:37 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:35:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:37 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:35:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:37 smithi159 ceph-mon[35534]: pgmap v427: 97 pgs: 97 active+clean; 104 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-09-11T22:35:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:37 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:35:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:37 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:35:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:37 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:35:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:37 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:35:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:37 smithi172 ceph-mon[35461]: pgmap v427: 97 pgs: 97 active+clean; 104 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.4 MiB/s wr, 205 op/s 2024-09-11T22:35:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:37 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:35:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:37 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:35:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:37 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:35:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:37 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:35:38.351 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1004446 7.20 MB/sec execute 596 sec latency 22.233 ms 2024-09-11T22:35:39.352 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1005860 7.20 MB/sec execute 597 sec latency 60.035 ms 2024-09-11T22:35:39.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:39 smithi005 ceph-mon[30943]: pgmap v428: 97 pgs: 97 active+clean; 111 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-09-11T22:35:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:39 smithi159 ceph-mon[35534]: pgmap v428: 97 pgs: 97 active+clean; 111 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-09-11T22:35:39.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:39 smithi172 ceph-mon[35461]: pgmap v428: 97 pgs: 97 active+clean; 111 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.5 MiB/s wr, 288 op/s 2024-09-11T22:35:40.352 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1007855 7.20 MB/sec execute 598 sec latency 37.194 ms 2024-09-11T22:35:41.352 INFO:tasks.workunit.client.0.smithi005.stdout: 1 1009366 7.21 MB/sec execute 599 sec latency 19.925 ms 2024-09-11T22:35:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:41 smithi005 ceph-mon[30943]: pgmap v429: 97 pgs: 97 active+clean; 111 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-09-11T22:35:41.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:41 smithi159 ceph-mon[35534]: pgmap v429: 97 pgs: 97 active+clean; 111 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-09-11T22:35:41.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:41 smithi172 ceph-mon[35461]: pgmap v429: 97 pgs: 97 active+clean; 111 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 219 op/s 2024-09-11T22:35:42.352 INFO:tasks.workunit.client.0.smithi005.stdout: 1 cleanup 600 sec 2024-09-11T22:35:42.527 INFO:tasks.workunit.client.0.smithi005.stdout: 0 cleanup 600 sec 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: Operation Count AvgLat MaxLat 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: ---------------------------------------- 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: NTCreateX 137615 1.717 22425.632 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: Close 101090 0.003 0.141 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: Rename 5828 9.007 21530.991 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: Unlink 27788 2.789 18786.987 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: Deltree 4 73.152 85.716 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: Mkdir 2 0.007 0.007 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: Qpathinfo 124722 1.039 186.722 2024-09-11T22:35:42.528 INFO:tasks.workunit.client.0.smithi005.stdout: Qfileinfo 21877 0.001 0.043 2024-09-11T22:35:42.529 INFO:tasks.workunit.client.0.smithi005.stdout: Qfsinfo 22871 0.202 37.978 2024-09-11T22:35:42.529 INFO:tasks.workunit.client.0.smithi005.stdout: Sfileinfo 11211 1.691 18644.002 2024-09-11T22:35:42.529 INFO:tasks.workunit.client.0.smithi005.stdout: Find 48224 0.323 8971.448 2024-09-11T22:35:42.529 INFO:tasks.workunit.client.0.smithi005.stdout: WriteX 68717 0.439 26162.065 2024-09-11T22:35:42.529 INFO:tasks.workunit.client.0.smithi005.stdout: ReadX 215673 0.018 39.113 2024-09-11T22:35:42.529 INFO:tasks.workunit.client.0.smithi005.stdout: LockX 448 0.543 2.364 2024-09-11T22:35:42.529 INFO:tasks.workunit.client.0.smithi005.stdout: UnlockX 448 0.496 2.145 2024-09-11T22:35:42.529 INFO:tasks.workunit.client.0.smithi005.stdout: Flush 9645 2.915 123.698 2024-09-11T22:35:42.529 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-11T22:35:42.529 INFO:tasks.workunit.client.0.smithi005.stdout:Throughput 7.20971 MB/sec 1 clients 1 procs max_latency=26162.070 ms 2024-09-11T22:35:42.530 INFO:tasks.workunit.client.0.smithi005.stderr:+ dbench 10 2024-09-11T22:35:42.530 INFO:tasks.workunit.client.0.smithi005.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-11T22:35:42.530 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-11T22:35:42.530 INFO:tasks.workunit.client.0.smithi005.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-11T22:35:42.530 INFO:tasks.workunit.client.0.smithi005.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-11T22:35:42.629 INFO:tasks.workunit.client.0.smithi005.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-11T22:35:42.629 INFO:tasks.workunit.client.0.smithi005.stdout:releasing clients 2024-09-11T22:35:43.629 INFO:tasks.workunit.client.0.smithi005.stdout: 10 351 126.29 MB/sec warmup 1 sec latency 167.711 ms 2024-09-11T22:35:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.749+0000 7f8aacf2c640 1 -- 172.21.15.5:0/1445154009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8aa81035a0 msgr2=0x7f8aa810ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.749+0000 7f8aacf2c640 1 --2- 172.21.15.5:0/1445154009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8aa81035a0 0x7f8aa810ba80 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009900 tx=0x7f8a9c02a480 comp rx=0 tx=0).stop 2024-09-11T22:35:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.749+0000 7f8aacf2c640 1 -- 172.21.15.5:0/1445154009 shutdown_connections 2024-09-11T22:35:43.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.749+0000 7f8aacf2c640 1 --2- 172.21.15.5:0/1445154009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aa810c0d0 0x7f8aa810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.749+0000 7f8aacf2c640 1 --2- 172.21.15.5:0/1445154009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8aa81035a0 0x7f8aa810ba80 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.749+0000 7f8aacf2c640 1 --2- 172.21.15.5:0/1445154009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa8102c80 0x7f8aa8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.749+0000 7f8aacf2c640 1 -- 172.21.15.5:0/1445154009 >> 172.21.15.5:0/1445154009 conn(0x7f8aa8077990 msgr2=0x7f8aa80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.749+0000 7f8aacf2c640 1 -- 172.21.15.5:0/1445154009 shutdown_connections 2024-09-11T22:35:43.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.749+0000 7f8aacf2c640 1 -- 172.21.15.5:0/1445154009 wait complete. 2024-09-11T22:35:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aacf2c640 1 Processor -- start 2024-09-11T22:35:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aacf2c640 1 -- start start 2024-09-11T22:35:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aacf2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa8102c80 0x7f8aa8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aacf2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8aa81035a0 0x7f8aa81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aacf2c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aa810c0d0 0x7f8aa819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aacf2c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8111eb0 con 0x7f8aa81035a0 2024-09-11T22:35:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aacf2c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8111d30 con 0x7f8aa810c0d0 2024-09-11T22:35:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aacf2c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8112030 con 0x7f8aa8102c80 2024-09-11T22:35:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aa659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa8102c80 0x7f8aa8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aa659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa8102c80 0x7f8aa8198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59634/0 (socket says 172.21.15.5:59634) 2024-09-11T22:35:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aa659c640 1 -- 172.21.15.5:0/649939562 learned_addr learned my addr 172.21.15.5:0/649939562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8aa81035a0 0x7f8aa81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.750+0000 7f8aa6d9d640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aa810c0d0 0x7f8aa819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8aa659c640 1 -- 172.21.15.5:0/649939562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aa810c0d0 msgr2=0x7f8aa819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8aa659c640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aa810c0d0 0x7f8aa819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8aa659c640 1 -- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8aa81035a0 msgr2=0x7f8aa81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8aa659c640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8aa81035a0 0x7f8aa81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8aa659c640 1 -- 172.21.15.5:0/649939562 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa819d350 con 0x7f8aa8102c80 2024-09-11T22:35:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8aa81035a0 0x7f8aa81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8aa659c640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa8102c80 0x7f8aa8198360 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800d770 tx=0x7f8a9800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8a977fe640 1 -- 172.21.15.5:0/649939562 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98017070 con 0x7f8aa8102c80 2024-09-11T22:35:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8aacf2c640 1 -- 172.21.15.5:0/649939562 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa819d640 con 0x7f8aa8102c80 2024-09-11T22:35:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8a977fe640 1 -- 172.21.15.5:0/649939562 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a98005500 con 0x7f8aa8102c80 2024-09-11T22:35:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8a977fe640 1 -- 172.21.15.5:0/649939562 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98004e00 con 0x7f8aa8102c80 2024-09-11T22:35:43.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.751+0000 7f8aacf2c640 1 -- 172.21.15.5:0/649939562 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa81a4ed0 con 0x7f8aa8102c80 2024-09-11T22:35:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.753+0000 7f8a977fe640 1 -- 172.21.15.5:0/649939562 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8a98004fa0 con 0x7f8aa8102c80 2024-09-11T22:35:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8aafcbd740 1 Processor -- start 2024-09-11T22:35:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8aafcbd740 1 -- start start 2024-09-11T22:35:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8a977fe640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8a900a2da0 0x7f8a900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8aa6d9d640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8a900a2da0 0x7f8a900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 0x56117b9edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8a977fe640 1 -- 172.21.15.5:0/649939562 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f8a980cee90 con 0x7f8aa8102c80 2024-09-11T22:35:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 0x56117b9ff4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56117b9522f0 con 0x56117b9ee2d0 2024-09-11T22:35:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56117b8fa270 con 0x56117b9ed9b0 2024-09-11T22:35:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.754+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56117b94e2b0 con 0x56117b9fd0c0 2024-09-11T22:35:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 0x56117b9edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 0x56117b9ff4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 0x56117b9edd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51686/0 (socket says 172.21.15.5:51686) 2024-09-11T22:35:43.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 0x56117b9ff4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59636/0 (socket says 172.21.15.5:59636) 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa659c640 1 -- 172.21.15.5:0/2157491910 learned_addr learned my addr 172.21.15.5:0/2157491910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/2157491910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/2157491910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 msgr2=0x56117b9ff4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/2157491910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 0x56117b9ff4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/2157491910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 msgr2=0x56117b9edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/2157491910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 0x56117b9edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/2157491910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56117b8f9340 con 0x56117b9ee2d0 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa6d9d640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8a900a2da0 0x7f8a900a5260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0005fd0 tx=0x7f8aa0011b40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/2157491910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fcb80 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c005940 tx=0x7f8a9c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b253aab4dc8cbb6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8a94ff9640 1 -- 172.21.15.5:0/2157491910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c035040 con 0x56117b9ee2d0 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8a94ff9640 1 -- 172.21.15.5:0/2157491910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c031830 con 0x56117b9ee2d0 2024-09-11T22:35:43.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.755+0000 7f8a94ff9640 1 -- 172.21.15.5:0/2157491910 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c004750 con 0x56117b9ee2d0 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.756+0000 7f8aafcbd740 1 -- 172.21.15.5:0/2157491910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 msgr2=0x56117b9fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.756+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/2157491910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fcb80 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c005940 tx=0x7f8a9c005d50 comp rx=0 tx=0).stop 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.756+0000 7f8aafcbd740 1 -- 172.21.15.5:0/2157491910 shutdown_connections 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.756+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/2157491910 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 0x56117b9ff4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.756+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/2157491910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fcb80 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.756+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/2157491910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 0x56117b9edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.756+0000 7f8aafcbd740 1 -- 172.21.15.5:0/2157491910 >> 172.21.15.5:0/2157491910 conn(0x56117b9ea060 msgr2=0x56117b9ea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.756+0000 7f8aafcbd740 1 -- 172.21.15.5:0/2157491910 shutdown_connections 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.756+0000 7f8aafcbd740 1 -- 172.21.15.5:0/2157491910 wait complete. 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aafcbd740 1 -- start start 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aafcbd740 1 Processor -- start 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 0x56117b9fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 0x56117b9faac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56117b9ffc30 con 0x56117b9ee2d0 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56117b8db290 con 0x56117b9ed9b0 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56117b9ffdb0 con 0x56117b9fd0c0 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aafcbd740 10 client.0 _ll_register_callbacks cb 0x56117b90c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aa5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aa659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 0x56117b9fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aa6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 0x56117b9faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aa659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 0x56117b9fa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51698/0 (socket says 172.21.15.5:51698) 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.757+0000 7f8aa659c640 1 -- 172.21.15.5:0/595963201 learned_addr learned my addr 172.21.15.5:0/595963201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/595963201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 msgr2=0x56117b9faac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:43.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 0x56117b9faac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/595963201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 msgr2=0x56117b9fa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 0x56117b9fa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/595963201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56117b665210 con 0x56117b9ee2d0 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa580 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c02a4c0 tx=0x7f8a9c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769032db0fb6862a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8a8dffb640 1 -- 172.21.15.5:0/595963201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c035070 con 0x56117b9ee2d0 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8a8dffb640 1 -- 172.21.15.5:0/595963201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f8a9c031630 con 0x56117b9ee2d0 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8aafcbd740 1 -- 172.21.15.5:0/595963201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9c031830 con 0x56117b9ee2d0 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8a8dffb640 1 -- 172.21.15.5:0/595963201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8a9c0319e0 con 0x56117b9ee2d0 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8aafcbd740 10 client.19605 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8aafcbd740 1 -- 172.21.15.5:0/595963201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56117b731620 con 0x56117b9ee2d0 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.758+0000 7f8a8dffb640 1 -- 172.21.15.5:0/595963201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c031830 con 0x56117b9ee2d0 2024-09-11T22:35:43.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.759+0000 7f8a8dffb640 1 -- 172.21.15.5:0/595963201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f8a9c004750 con 0x56117b9ee2d0 2024-09-11T22:35:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.759+0000 7f8aafcbd740 10 client.19605 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.759+0000 7f8aafcbd740 10 client.19605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.759+0000 7f8aafcbd740 20 client.19605 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.759+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56117ba0ced0 0x56117ba0f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.759+0000 7f8aafcbd740 4 client.19605 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.759+0000 7f8aafcbd740 1 -- 172.21.15.5:0/595963201 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56117b95fa20 con 0x56117ba0ced0 2024-09-11T22:35:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.759+0000 7f8aa659c640 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56117ba0ced0 0x56117ba0f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.759+0000 7f8aa659c640 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56117ba0ced0 0x56117ba0f2c0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15669fcd3370b182 server_cookie=c2bea12fd79b0a1 in_seq=0 out_seq=0 2024-09-11T22:35:43.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.759+0000 7f8a8dffb640 10 client.19605 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8a8dffb640 1 -- 172.21.15.5:0/595963201 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56117b95fa20 con 0x56117ba0ced0 2024-09-11T22:35:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8a8dffb640 10 client.19605 handle_command_reply: tid=0 2024-09-11T22:35:43.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8aafcbd740 1 client.19605 shutdown 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8aafcbd740 1 -- 172.21.15.5:0/595963201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 msgr2=0x56117b9fa580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa580 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c02a4c0 tx=0x7f8a9c005740 comp rx=0 tx=0).stop 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8aafcbd740 1 -- 172.21.15.5:0/595963201 shutdown_connections 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56117ba0ced0 0x56117ba0f2c0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9fd0c0 0x56117b9faac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa580 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8a8dffb640 0 client.19605 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/595963201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9ed9b0 0x56117b9fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8aafcbd740 1 -- 172.21.15.5:0/595963201 >> 172.21.15.5:0/595963201 conn(0x56117b9ea060 msgr2=0x56117b9efd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.762+0000 7f8aafcbd740 1 -- 172.21.15.5:0/595963201 shutdown_connections 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.763+0000 7f8aafcbd740 1 -- 172.21.15.5:0/595963201 wait complete. 2024-09-11T22:35:43.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.763+0000 7f8aafcbd740 20 client.19605 trim_cache size 0 max 16384 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.778+0000 7f8aafcbd740 1 Processor -- start 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.778+0000 7f8aafcbd740 1 -- start start 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.778+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 0x56117b9fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.778+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.778+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9fd0c0 0x56117ba14890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.778+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81ab700 con 0x56117b9ee2d0 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81ad270 con 0x56117b9fd0c0 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81abdd0 con 0x56117b9ed9b0 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 0x56117b9fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52210/0 (socket says 172.21.15.5:52210) 2024-09-11T22:35:43.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/1471282195 learned_addr learned my addr 172.21.15.5:0/1471282195 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa6d9d640 1 --2- 172.21.15.5:0/1471282195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9fd0c0 0x56117ba14890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/1471282195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 msgr2=0x56117b9fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/1471282195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 0x56117b9fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/1471282195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9fd0c0 msgr2=0x56117ba14890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/1471282195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9fd0c0 0x56117ba14890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/1471282195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56117ba09a10 con 0x56117b9ee2d0 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa659c640 1 --2- 172.21.15.5:0/1471282195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 0x56117b9fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/1471282195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa8f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c000c00 tx=0x7f8a9c02ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2562bea6561ab7a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8a8ffff640 1 -- 172.21.15.5:0/1471282195 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c035040 con 0x56117b9ee2d0 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8a8ffff640 1 -- 172.21.15.5:0/1471282195 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c005090 con 0x56117b9ee2d0 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.779+0000 7f8a8ffff640 1 -- 172.21.15.5:0/1471282195 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0044a0 con 0x56117b9ee2d0 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.780+0000 7f8aafcbd740 1 -- 172.21.15.5:0/1471282195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 msgr2=0x56117b9fa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.780+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/1471282195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa8f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c000c00 tx=0x7f8a9c02ac00 comp rx=0 tx=0).stop 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.780+0000 7f8aafcbd740 1 -- 172.21.15.5:0/1471282195 shutdown_connections 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.780+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/1471282195 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9fd0c0 0x56117ba14890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.780+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/1471282195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117b9fa8f0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.780+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/1471282195 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 0x56117b9fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.780+0000 7f8aafcbd740 1 -- 172.21.15.5:0/1471282195 >> 172.21.15.5:0/1471282195 conn(0x56117b9ea060 msgr2=0x56117b9e91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.780+0000 7f8aafcbd740 1 -- 172.21.15.5:0/1471282195 shutdown_connections 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.780+0000 7f8aafcbd740 1 -- 172.21.15.5:0/1471282195 wait complete. 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.780+0000 7f8aafcbd740 1 -- start start 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aafcbd740 1 Processor -- start 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 0x56117ba00eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117ba013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aafcbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9fd0c0 0x56117ba01930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81acba0 con 0x56117b9ee2d0 2024-09-11T22:35:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81ac4d0 con 0x56117b9fd0c0 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aafcbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81a9700 con 0x56117b9ed9b0 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aafcbd740 10 client.0 _ll_register_callbacks cb 0x7f8a9c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117ba013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117ba013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52220/0 (socket says 172.21.15.5:52220) 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/3706502531 learned_addr learned my addr 172.21.15.5:0/3706502531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa6d9d640 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9fd0c0 0x56117ba01930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa659c640 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 0x56117ba00eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/3706502531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 msgr2=0x56117ba00eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 0x56117ba00eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/3706502531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9fd0c0 msgr2=0x56117ba01930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9fd0c0 0x56117ba01930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa5d9b640 1 -- 172.21.15.5:0/3706502531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56117ba76930 con 0x56117b9ee2d0 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa659c640 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 0x56117ba00eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:43.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8aa5d9b640 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117ba013f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c02a9a0 tx=0x7f8a9c0044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98c2b3adca7409 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.781+0000 7f8a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8a8dffb640 1 -- 172.21.15.5:0/3706502531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c035070 con 0x56117b9ee2d0 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8a8dffb640 1 -- 172.21.15.5:0/3706502531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f8a9c0048d0 con 0x56117b9ee2d0 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8aafcbd740 1 -- 172.21.15.5:0/3706502531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9c005090 con 0x56117b9ee2d0 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8a8dffb640 1 -- 172.21.15.5:0/3706502531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8a9c005240 con 0x56117b9ee2d0 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8aafcbd740 10 client.19611 fetch_fsmap learned FSMap version 48 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8aafcbd740 1 -- 172.21.15.5:0/3706502531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56117ba77c20 con 0x56117b9ee2d0 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8a8dffb640 1 -- 172.21.15.5:0/3706502531 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c005090 con 0x56117b9ee2d0 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8a8dffb640 1 -- 172.21.15.5:0/3706502531 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 48) ==== 8366+0+0 (secure 0 0 0) 0x7f8a9c0624f0 con 0x56117b9ee2d0 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8aafcbd740 10 client.19611 fetch_fsmap finished waiting for FSMap version 48 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8aafcbd740 10 client.19611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8aafcbd740 20 client.19611 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56117ba782e0 0x56117ba7a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8aafcbd740 4 client.19611 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:35:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.782+0000 7f8aafcbd740 1 -- 172.21.15.5:0/3706502531 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56117b959d70 con 0x56117ba782e0 2024-09-11T22:35:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.783+0000 7f8aa659c640 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56117ba782e0 0x56117ba7a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.783+0000 7f8aa659c640 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56117ba782e0 0x56117ba7a700 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c5dd45bf6e3947 server_cookie=7fa16b4afe922fd7 in_seq=0 out_seq=0 2024-09-11T22:35:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:43.783+0000 7f8a8dffb640 10 client.19611 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:43.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:43 smithi005 ceph-mon[30943]: pgmap v430: 97 pgs: 97 active+clean; 118 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2024-09-11T22:35:43.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:43 smithi159 ceph-mon[35534]: pgmap v430: 97 pgs: 97 active+clean; 118 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2024-09-11T22:35:43.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:43 smithi172 ceph-mon[35461]: pgmap v430: 97 pgs: 97 active+clean; 118 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.0 MiB/s wr, 312 op/s 2024-09-11T22:35:44.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8a8dffb640 1 -- 172.21.15.5:0/3706502531 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56117b959d70 con 0x56117ba782e0 2024-09-11T22:35:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8a8dffb640 10 client.19611 handle_command_reply: tid=0 2024-09-11T22:35:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8aafcbd740 1 client.19611 shutdown 2024-09-11T22:35:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8aafcbd740 1 -- 172.21.15.5:0/3706502531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 msgr2=0x56117ba013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117ba013f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c02a9a0 tx=0x7f8a9c0044f0 comp rx=0 tx=0).stop 2024-09-11T22:35:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8aafcbd740 1 -- 172.21.15.5:0/3706502531 shutdown_connections 2024-09-11T22:35:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56117ba782e0 0x56117ba7a700 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56117b9fd0c0 0x56117ba01930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56117b9ee2d0 0x56117ba013f0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8aafcbd740 1 --2- 172.21.15.5:0/3706502531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56117b9ed9b0 0x56117ba00eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8aafcbd740 1 -- 172.21.15.5:0/3706502531 >> 172.21.15.5:0/3706502531 conn(0x56117b9ea060 msgr2=0x56117b9e89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.134+0000 7f8a8dffb640 0 client.19611 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aafcbd740 1 -- 172.21.15.5:0/3706502531 shutdown_connections 2024-09-11T22:35:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aafcbd740 1 -- 172.21.15.5:0/3706502531 wait complete. 2024-09-11T22:35:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aafcbd740 20 client.19611 trim_cache size 0 max 16384 2024-09-11T22:35:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:35:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:35:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:35:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:35:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 -- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8a900a2da0 msgr2=0x7f8a900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8a900a2da0 0x7f8a900a5260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0005fd0 tx=0x7f8aa0011b40 comp rx=0 tx=0).stop 2024-09-11T22:35:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 -- 172.21.15.5:0/649939562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa8102c80 msgr2=0x7f8aa8198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa8102c80 0x7f8aa8198360 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800d770 tx=0x7f8a9800dc30 comp rx=0 tx=0).stop 2024-09-11T22:35:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 -- 172.21.15.5:0/649939562 shutdown_connections 2024-09-11T22:35:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8a900a2da0 0x7f8a900a5260 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8aa810c0d0 0x7f8aa819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8aa81035a0 0x7f8aa81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 --2- 172.21.15.5:0/649939562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa8102c80 0x7f8aa8198360 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 -- 172.21.15.5:0/649939562 >> 172.21.15.5:0/649939562 conn(0x7f8aa8077990 msgr2=0x7f8aa80fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.135+0000 7f8aacf2c640 1 -- 172.21.15.5:0/649939562 shutdown_connections 2024-09-11T22:35:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:44.136+0000 7f8aacf2c640 1 -- 172.21.15.5:0/649939562 wait complete. 2024-09-11T22:35:44.629 INFO:tasks.workunit.client.0.smithi005.stdout: 10 561 94.25 MB/sec warmup 2 sec latency 300.367 ms 2024-09-11T22:35:45.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1162 74.10 MB/sec warmup 3 sec latency 521.659 ms 2024-09-11T22:35:45.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:45 smithi005 ceph-mon[30943]: pgmap v431: 97 pgs: 97 active+clean; 118 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-11T22:35:45.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:45 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:35:45.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:45 smithi159 ceph-mon[35534]: pgmap v431: 97 pgs: 97 active+clean; 118 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-11T22:35:45.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:45 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:35:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:45 smithi172 ceph-mon[35461]: pgmap v431: 97 pgs: 97 active+clean; 118 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-11T22:35:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:45 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:35:46.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2006 65.75 MB/sec warmup 4 sec latency 1521.970 ms 2024-09-11T22:35:47.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2721 59.82 MB/sec warmup 5 sec latency 1684.982 ms 2024-09-11T22:35:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:47 smithi005 ceph-mon[30943]: pgmap v432: 97 pgs: 97 active+clean; 118 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-11T22:35:47.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:47 smithi159 ceph-mon[35534]: pgmap v432: 97 pgs: 97 active+clean; 118 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-11T22:35:47.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:47 smithi172 ceph-mon[35461]: pgmap v432: 97 pgs: 97 active+clean; 118 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-11T22:35:48.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3478 60.72 MB/sec warmup 6 sec latency 172.485 ms 2024-09-11T22:35:49.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3910 56.43 MB/sec warmup 7 sec latency 176.378 ms 2024-09-11T22:35:49.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:49 smithi005 ceph-mon[30943]: pgmap v433: 97 pgs: 97 active+clean; 122 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 472 op/s 2024-09-11T22:35:49.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:49 smithi159 ceph-mon[35534]: pgmap v433: 97 pgs: 97 active+clean; 122 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 472 op/s 2024-09-11T22:35:49.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:49 smithi172 ceph-mon[35461]: pgmap v433: 97 pgs: 97 active+clean; 122 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 472 op/s 2024-09-11T22:35:50.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4237 51.50 MB/sec warmup 8 sec latency 260.235 ms 2024-09-11T22:35:50.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:50 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:35:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:50 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:35:50.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:50 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:35:51.539 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T22:35:51.540 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-11T22:35:51.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:35:51.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4566 46.97 MB/sec warmup 9 sec latency 81.454 ms 2024-09-11T22:35:51.815 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:51.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:51 smithi005 ceph-mon[30943]: pgmap v434: 97 pgs: 97 active+clean; 122 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 389 op/s 2024-09-11T22:35:51.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:51 smithi159 ceph-mon[35534]: pgmap v434: 97 pgs: 97 active+clean; 122 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 389 op/s 2024-09-11T22:35:51.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:51 smithi172 ceph-mon[35461]: pgmap v434: 97 pgs: 97 active+clean; 122 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 389 op/s 2024-09-11T22:35:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.549+0000 7f2c6741a640 1 -- 172.21.15.5:0/55984510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c601008d0 msgr2=0x7f2c60100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.549+0000 7f2c6741a640 1 --2- 172.21.15.5:0/55984510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c601008d0 0x7f2c60100d30 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f2c50009930 tx=0x7f2c5002a420 comp rx=0 tx=0).stop 2024-09-11T22:35:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.549+0000 7f2c6741a640 1 -- 172.21.15.5:0/55984510 shutdown_connections 2024-09-11T22:35:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.549+0000 7f2c6741a640 1 --2- 172.21.15.5:0/55984510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c60101270 0x7f2c6010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.549+0000 7f2c6741a640 1 --2- 172.21.15.5:0/55984510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c601008d0 0x7f2c60100d30 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.549+0000 7f2c6741a640 1 --2- 172.21.15.5:0/55984510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c601068d0 0x7f2c60106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.549+0000 7f2c6741a640 1 -- 172.21.15.5:0/55984510 >> 172.21.15.5:0/55984510 conn(0x7f2c600fc610 msgr2=0x7f2c600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.549+0000 7f2c6741a640 1 -- 172.21.15.5:0/55984510 shutdown_connections 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.549+0000 7f2c6741a640 1 -- 172.21.15.5:0/55984510 wait complete. 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c6741a640 1 Processor -- start 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c6741a640 1 -- start start 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c6741a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c601008d0 0x7f2c6019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c6741a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c60101270 0x7f2c6019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c6741a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c601068d0 0x7f2c6019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c6741a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60110cd0 con 0x7f2c601068d0 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c6741a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60110b50 con 0x7f2c601008d0 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c6741a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60110e50 con 0x7f2c60101270 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c66418640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c601008d0 0x7f2c6019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c65c17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c60101270 0x7f2c6019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c66418640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c601008d0 0x7f2c6019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49004/0 (socket says 172.21.15.5:49004) 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c66418640 1 -- 172.21.15.5:0/4066500539 learned_addr learned my addr 172.21.15.5:0/4066500539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c65c17640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c60101270 0x7f2c6019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:40394/0 (socket says 172.21.15.5:40394) 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.550+0000 7f2c66c19640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c601068d0 0x7f2c6019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c65c17640 1 -- 172.21.15.5:0/4066500539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c601008d0 msgr2=0x7f2c6019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c65c17640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c601008d0 0x7f2c6019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c65c17640 1 -- 172.21.15.5:0/4066500539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c601068d0 msgr2=0x7f2c6019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c65c17640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c601068d0 0x7f2c6019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c65c17640 1 -- 172.21.15.5:0/4066500539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6019f500 con 0x7f2c60101270 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c66418640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c601008d0 0x7f2c6019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c65c17640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c60101270 0x7f2c6019aa90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2c50005940 tx=0x7f2c50031b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c577fe640 1 -- 172.21.15.5:0/4066500539 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5003f070 con 0x7f2c60101270 2024-09-11T22:35:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c6741a640 1 -- 172.21.15.5:0/4066500539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6019f790 con 0x7f2c60101270 2024-09-11T22:35:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c577fe640 1 -- 172.21.15.5:0/4066500539 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c50003ad0 con 0x7f2c60101270 2024-09-11T22:35:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c577fe640 1 -- 172.21.15.5:0/4066500539 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c50035070 con 0x7f2c60101270 2024-09-11T22:35:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.551+0000 7f2c6741a640 1 -- 172.21.15.5:0/4066500539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c601a70d0 con 0x7f2c60101270 2024-09-11T22:35:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.552+0000 7f2c6741a640 1 -- 172.21.15.5:0/4066500539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c60102010 con 0x7f2c60101270 2024-09-11T22:35:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.554+0000 7f2c577fe640 1 -- 172.21.15.5:0/4066500539 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2c500035f0 con 0x7f2c60101270 2024-09-11T22:35:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.554+0000 7f2c577fe640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c4c0a2cf0 0x7f2c4c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.555+0000 7f2c66418640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c4c0a2cf0 0x7f2c4c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.555+0000 7f2c577fe640 1 -- 172.21.15.5:0/4066500539 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f2c500efbc0 con 0x7f2c60101270 2024-09-11T22:35:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.555+0000 7f2c66418640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c4c0a2cf0 0x7f2c4c0a51b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c011490 tx=0x7f2c5c005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:52.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.560+0000 7f2c577fe640 1 -- 172.21.15.5:0/4066500539 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c500bc0c0 con 0x7f2c60101270 2024-09-11T22:35:52.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5020 43.98 MB/sec warmup 10 sec latency 219.928 ms 2024-09-11T22:35:52.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.839+0000 7f2c6741a640 1 -- 172.21.15.5:0/4066500539 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c60100d30 con 0x7f2c60101270 2024-09-11T22:35:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.840+0000 7f2c577fe640 1 -- 172.21.15.5:0/4066500539 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11975 (secure 0 0 0) 0x7f2c60100d30 con 0x7f2c60101270 2024-09-11T22:35:52.841 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:35:52.841 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"btime":"2024-09-11T22:35:49:548013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:35:48.842139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:35:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 50 2024-09-11T22:35:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 -- 172.21.15.5:0/4066500539 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c4c0a2cf0 msgr2=0x7f2c4c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c4c0a2cf0 0x7f2c4c0a51b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c011490 tx=0x7f2c5c005f90 comp rx=0 tx=0).stop 2024-09-11T22:35:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 -- 172.21.15.5:0/4066500539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c60101270 msgr2=0x7f2c6019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c60101270 0x7f2c6019aa90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2c50005940 tx=0x7f2c50031b50 comp rx=0 tx=0).stop 2024-09-11T22:35:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 -- 172.21.15.5:0/4066500539 shutdown_connections 2024-09-11T22:35:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c601068d0 0x7f2c6019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c60101270 0x7f2c6019aa90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2c4c0a2cf0 0x7f2c4c0a51b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 --2- 172.21.15.5:0/4066500539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2c601008d0 0x7f2c6019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 -- 172.21.15.5:0/4066500539 >> 172.21.15.5:0/4066500539 conn(0x7f2c600fc610 msgr2=0x7f2c60069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.842+0000 7f2c6741a640 1 -- 172.21.15.5:0/4066500539 shutdown_connections 2024-09-11T22:35:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:52.843+0000 7f2c6741a640 1 -- 172.21.15.5:0/4066500539 wait complete. 2024-09-11T22:35:53.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:35:53.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5713 43.75 MB/sec warmup 11 sec latency 154.110 ms 2024-09-11T22:35:53.675 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-11T22:35:53.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-11T22:35:53.742 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:53.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:53 smithi005 ceph-mon[30943]: pgmap v435: 97 pgs: 97 active+clean; 132 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-09-11T22:35:53.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:53 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4066500539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:35:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:53 smithi159 ceph-mon[35534]: pgmap v435: 97 pgs: 97 active+clean; 132 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-09-11T22:35:53.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:53 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4066500539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:35:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:53 smithi172 ceph-mon[35461]: pgmap v435: 97 pgs: 97 active+clean; 132 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.1 MiB/s wr, 516 op/s 2024-09-11T22:35:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:53 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4066500539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:35:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.272+0000 7f0b53de1640 1 -- 172.21.15.5:0/2462350601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b4c100e20 msgr2=0x7f0b4c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.272+0000 7f0b53de1640 1 --2- 172.21.15.5:0/2462350601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b4c100e20 0x7f0b4c101200 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009900 tx=0x7f0b4002a480 comp rx=0 tx=0).stop 2024-09-11T22:35:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.272+0000 7f0b53de1640 1 -- 172.21.15.5:0/2462350601 shutdown_connections 2024-09-11T22:35:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.272+0000 7f0b53de1640 1 --2- 172.21.15.5:0/2462350601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c102ef0 0x7f0b4c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.272+0000 7f0b53de1640 1 --2- 172.21.15.5:0/2462350601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b4c102570 0x7f0b4c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.272+0000 7f0b53de1640 1 --2- 172.21.15.5:0/2462350601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b4c100e20 0x7f0b4c101200 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.272+0000 7f0b53de1640 1 -- 172.21.15.5:0/2462350601 >> 172.21.15.5:0/2462350601 conn(0x7f0b4c0fc7d0 msgr2=0x7f0b4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.272+0000 7f0b53de1640 1 -- 172.21.15.5:0/2462350601 shutdown_connections 2024-09-11T22:35:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.272+0000 7f0b53de1640 1 -- 172.21.15.5:0/2462350601 wait complete. 2024-09-11T22:35:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b53de1640 1 Processor -- start 2024-09-11T22:35:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b53de1640 1 -- start start 2024-09-11T22:35:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b53de1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b4c100e20 0x7f0b4c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b53de1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c102570 0x7f0b4c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b53de1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b4c102ef0 0x7f0b4c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b53de1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c110f80 con 0x7f0b4c102ef0 2024-09-11T22:35:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b53de1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c110e00 con 0x7f0b4c100e20 2024-09-11T22:35:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b53de1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c111100 con 0x7f0b4c102570 2024-09-11T22:35:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b5237e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b4c102ef0 0x7f0b4c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b51b7d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b4c100e20 0x7f0b4c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b51b7d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b4c100e20 0x7f0b4c198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49024/0 (socket says 172.21.15.5:49024) 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b5137c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c102570 0x7f0b4c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.273+0000 7f0b51b7d640 1 -- 172.21.15.5:0/4110455855 learned_addr learned my addr 172.21.15.5:0/4110455855 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b5237e640 1 -- 172.21.15.5:0/4110455855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c102570 msgr2=0x7f0b4c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b5237e640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c102570 0x7f0b4c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b5237e640 1 -- 172.21.15.5:0/4110455855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b4c100e20 msgr2=0x7f0b4c198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b5237e640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b4c100e20 0x7f0b4c198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b5237e640 1 -- 172.21.15.5:0/4110455855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4c19d4b0 con 0x7f0b4c102ef0 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b51b7d640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b4c100e20 0x7f0b4c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b5137c640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c102570 0x7f0b4c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b5237e640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b4c102ef0 0x7f0b4c19cdd0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400ec50 tx=0x7f0b4400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b3effd640 1 -- 172.21.15.5:0/4110455855 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44003b70 con 0x7f0b4c102ef0 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b3effd640 1 -- 172.21.15.5:0/4110455855 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b44003d10 con 0x7f0b4c102ef0 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b53de1640 1 -- 172.21.15.5:0/4110455855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4c19d7a0 con 0x7f0b4c102ef0 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.274+0000 7f0b3effd640 1 -- 172.21.15.5:0/4110455855 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44010890 con 0x7f0b4c102ef0 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.275+0000 7f0b53de1640 1 -- 172.21.15.5:0/4110455855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b4c1a5080 con 0x7f0b4c102ef0 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.278+0000 7f0b561ba740 1 Processor -- start 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.278+0000 7f0b3effd640 1 -- 172.21.15.5:0/4110455855 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0b44003fc0 con 0x7f0b4c102ef0 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.278+0000 7f0b561ba740 1 -- start start 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b544a0 0x564103b54880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b54dc0 0x564103b5f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b61a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b561ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564103aba2e0 con 0x564103b5f680 2024-09-11T22:35:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b561ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564103a61ce0 con 0x564103b54dc0 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b561ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564103ab62a0 con 0x564103b544a0 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b5137c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b544a0 0x564103b54880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b51b7d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b54dc0 0x564103b5f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b5137c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b61a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b51b7d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b54dc0 0x564103b5f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49036/0 (socket says 172.21.15.5:49036) 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b51b7d640 1 -- 172.21.15.5:0/6412177 learned_addr learned my addr 172.21.15.5:0/6412177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b5137c640 1 -- 172.21.15.5:0/6412177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b544a0 msgr2=0x564103b54880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b5137c640 1 --2- 172.21.15.5:0/6412177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b544a0 0x564103b54880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b5137c640 1 -- 172.21.15.5:0/6412177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b54dc0 msgr2=0x564103b5f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b5137c640 1 --2- 172.21.15.5:0/6412177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b54dc0 0x564103b5f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b3effd640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0b380a2d70 0x7f0b380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b5137c640 1 -- 172.21.15.5:0/6412177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564103a46440 con 0x564103b5f680 2024-09-11T22:35:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b51b7d640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0b380a2d70 0x7f0b380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b3effd640 1 -- 172.21.15.5:0/4110455855 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f0b440cf9e0 con 0x7f0b4c102ef0 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.279+0000 7f0b5137c640 1 --2- 172.21.15.5:0/6412177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b61a70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0fff70 tx=0x7f0b48005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eee5c4b4a23ae7b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b37fff640 1 -- 172.21.15.5:0/6412177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48016070 con 0x564103b5f680 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b37fff640 1 -- 172.21.15.5:0/6412177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b4800fe50 con 0x564103b5f680 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b37fff640 1 -- 172.21.15.5:0/6412177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48019040 con 0x564103b5f680 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b561ba740 1 -- 172.21.15.5:0/6412177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 msgr2=0x564103b61a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b561ba740 1 --2- 172.21.15.5:0/6412177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b61a70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c0fff70 tx=0x7f0b48005f90 comp rx=0 tx=0).stop 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b51b7d640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0b380a2d70 0x7f0b380a5230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0b400096c0 tx=0x7f0b40005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b561ba740 1 -- 172.21.15.5:0/6412177 shutdown_connections 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b561ba740 1 --2- 172.21.15.5:0/6412177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b61a70 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b561ba740 1 --2- 172.21.15.5:0/6412177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b54dc0 0x564103b5f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b561ba740 1 --2- 172.21.15.5:0/6412177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b544a0 0x564103b54880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.280+0000 7f0b561ba740 1 -- 172.21.15.5:0/6412177 >> 172.21.15.5:0/6412177 conn(0x564103b50a30 msgr2=0x564103b50e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.281+0000 7f0b561ba740 1 -- 172.21.15.5:0/6412177 shutdown_connections 2024-09-11T22:35:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.281+0000 7f0b561ba740 1 -- 172.21.15.5:0/6412177 wait complete. 2024-09-11T22:35:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.281+0000 7f0b561ba740 1 -- start start 2024-09-11T22:35:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.281+0000 7f0b561ba740 1 Processor -- start 2024-09-11T22:35:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.281+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b544a0 0x564103b5f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.281+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b5c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.281+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b5f680 0x564103b5c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.281+0000 7f0b561ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564103b62250 con 0x564103b54dc0 2024-09-11T22:35:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b561ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564103a75050 con 0x564103b5f680 2024-09-11T22:35:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b561ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564103b623d0 con 0x564103b544a0 2024-09-11T22:35:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b561ba740 10 client.0 _ll_register_callbacks cb 0x564103ab5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b5137c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b5c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b5137c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b5c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53706/0 (socket says 172.21.15.5:53706) 2024-09-11T22:35:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b5137c640 1 -- 172.21.15.5:0/1051369502 learned_addr learned my addr 172.21.15.5:0/1051369502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b51b7d640 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b544a0 0x564103b5f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b5237e640 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b5f680 0x564103b5c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b5137c640 1 -- 172.21.15.5:0/1051369502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b544a0 msgr2=0x564103b5f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b5137c640 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b544a0 0x564103b5f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b5137c640 1 -- 172.21.15.5:0/1051369502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b5f680 msgr2=0x564103b5c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b5137c640 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b5f680 0x564103b5c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b5137c640 1 -- 172.21.15.5:0/1051369502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56410382b3c0 con 0x564103b54dc0 2024-09-11T22:35:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b5137c640 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b5c2b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0b480053a0 tx=0x7f0b480096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f148cc14ff7d86b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b357fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b357fa640 1 -- 172.21.15.5:0/1051369502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48016070 con 0x564103b54dc0 2024-09-11T22:35:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b357fa640 1 -- 172.21.15.5:0/1051369502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f0b480046c0 con 0x564103b54dc0 2024-09-11T22:35:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.282+0000 7f0b561ba740 1 -- 172.21.15.5:0/1051369502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b4800fe50 con 0x564103b54dc0 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b357fa640 1 -- 172.21.15.5:0/1051369502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0b48010930 con 0x564103b54dc0 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b561ba740 10 client.19629 fetch_fsmap learned FSMap version 50 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b561ba740 1 -- 172.21.15.5:0/1051369502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564103b6dc70 con 0x564103b54dc0 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b357fa640 1 -- 172.21.15.5:0/1051369502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b4800fe50 con 0x564103b54dc0 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b357fa640 1 -- 172.21.15.5:0/1051369502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f0b480289b0 con 0x564103b54dc0 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b561ba740 10 client.19629 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b561ba740 10 client.19629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b561ba740 20 client.19629 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b561ba740 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564103b6f070 0x564103b71460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b561ba740 4 client.19629 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b561ba740 1 -- 172.21.15.5:0/1051369502 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564103ac7850 con 0x564103b6f070 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.283+0000 7f0b51b7d640 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564103b6f070 0x564103b71460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.284+0000 7f0b51b7d640 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564103b6f070 0x564103b71460 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e57291e5f3eec47 server_cookie=48c2bc4aaea5de44 in_seq=0 out_seq=0 2024-09-11T22:35:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.284+0000 7f0b357fa640 10 client.19629 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.285+0000 7f0b357fa640 1 -- 172.21.15.5:0/1051369502 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564103ac7850 con 0x564103b6f070 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.285+0000 7f0b357fa640 10 client.19629 handle_command_reply: tid=0 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 client.19629 shutdown 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 -- 172.21.15.5:0/1051369502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 msgr2=0x564103b5c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b5c2b0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0b480053a0 tx=0x7f0b480096c0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 -- 172.21.15.5:0/1051369502 shutdown_connections 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564103b6f070 0x564103b71460 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b357fa640 0 client.19629 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b5f680 0x564103b5c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b5c2b0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 --2- 172.21.15.5:0/1051369502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b544a0 0x564103b5f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 -- 172.21.15.5:0/1051369502 >> 172.21.15.5:0/1051369502 conn(0x564103b50a30 msgr2=0x564103b56710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 -- 172.21.15.5:0/1051369502 shutdown_connections 2024-09-11T22:35:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 1 -- 172.21.15.5:0/1051369502 wait complete. 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.286+0000 7f0b561ba740 20 client.19629 trim_cache size 0 max 16384 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.305+0000 7f0b561ba740 1 Processor -- start 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.305+0000 7f0b561ba740 1 -- start start 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 0x564103b71460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b6b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b5f680 0x564103b5bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b561ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1afd00 con 0x564103b54dc0 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b561ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1b1930 con 0x564103b544a0 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b561ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1b03d0 con 0x564103b5f680 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b5137c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b6b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b5137c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b6b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53714/0 (socket says 172.21.15.5:53714) 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b5137c640 1 -- 172.21.15.5:0/471390950 learned_addr learned my addr 172.21.15.5:0/471390950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b51b7d640 1 --2- 172.21.15.5:0/471390950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 0x564103b71460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b5237e640 1 --2- 172.21.15.5:0/471390950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b5f680 0x564103b5bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b5137c640 1 -- 172.21.15.5:0/471390950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b5f680 msgr2=0x564103b5bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b5137c640 1 --2- 172.21.15.5:0/471390950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b5f680 0x564103b5bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b5137c640 1 -- 172.21.15.5:0/471390950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 msgr2=0x564103b71460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b5137c640 1 --2- 172.21.15.5:0/471390950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 0x564103b71460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b5137c640 1 -- 172.21.15.5:0/471390950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564103b54bf0 con 0x564103b54dc0 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b5137c640 1 --2- 172.21.15.5:0/471390950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b6b4e0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0b48010ad0 tx=0x7f0b48010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e221de25b27b216e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.306+0000 7f0b377fe640 1 -- 172.21.15.5:0/471390950 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48033070 con 0x564103b54dc0 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b377fe640 1 -- 172.21.15.5:0/471390950 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b480104b0 con 0x564103b54dc0 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b377fe640 1 -- 172.21.15.5:0/471390950 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48016040 con 0x564103b54dc0 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b561ba740 1 -- 172.21.15.5:0/471390950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 msgr2=0x564103b6b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b561ba740 1 --2- 172.21.15.5:0/471390950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b6b4e0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0b48010ad0 tx=0x7f0b48010b90 comp rx=0 tx=0).stop 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b561ba740 1 -- 172.21.15.5:0/471390950 shutdown_connections 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b561ba740 1 --2- 172.21.15.5:0/471390950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b5f680 0x564103b5bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b561ba740 1 --2- 172.21.15.5:0/471390950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b54dc0 0x564103b6b4e0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b561ba740 1 --2- 172.21.15.5:0/471390950 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 0x564103b71460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b561ba740 1 -- 172.21.15.5:0/471390950 >> 172.21.15.5:0/471390950 conn(0x564103b50a30 msgr2=0x564103b50e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b561ba740 1 -- 172.21.15.5:0/471390950 shutdown_connections 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.307+0000 7f0b561ba740 1 -- 172.21.15.5:0/471390950 wait complete. 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b561ba740 1 -- start start 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b561ba740 1 Processor -- start 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 0x564103b67a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b54dc0 0x564103b67f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b561ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b63010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b561ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1b1260 con 0x564103b5f680 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b561ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1b0ad0 con 0x564103b544a0 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b561ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c1add20 con 0x564103b54dc0 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b561ba740 10 client.0 _ll_register_callbacks cb 0x7f0b440093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b5237e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b63010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b51b7d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 0x564103b67a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b51b7d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 0x564103b67a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49054/0 (socket says 172.21.15.5:49054) 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b5237e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b63010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53730/0 (socket says 172.21.15.5:53730) 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b51b7d640 1 -- 172.21.15.5:0/529620811 learned_addr learned my addr 172.21.15.5:0/529620811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.308+0000 7f0b5137c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b54dc0 0x564103b67f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b5237e640 1 -- 172.21.15.5:0/529620811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b54dc0 msgr2=0x564103b67f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b5237e640 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b54dc0 0x564103b67f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b5237e640 1 -- 172.21.15.5:0/529620811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 msgr2=0x564103b67a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b5237e640 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 0x564103b67a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b5237e640 1 -- 172.21.15.5:0/529620811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564103b5f370 con 0x564103b5f680 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b51b7d640 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 0x564103b67a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b5237e640 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b63010 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f0b440cd0c0 tx=0x7f0b4409b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2315cc8c4fc65a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b357fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b357fa640 1 -- 172.21.15.5:0/529620811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4409be90 con 0x564103b5f680 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b357fa640 1 -- 172.21.15.5:0/529620811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f0b440a4670 con 0x564103b5f680 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b561ba740 1 -- 172.21.15.5:0/529620811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b480104b0 con 0x564103b5f680 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b357fa640 1 -- 172.21.15.5:0/529620811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0b440a0240 con 0x564103b5f680 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b561ba740 10 client.19635 fetch_fsmap learned FSMap version 50 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b561ba740 1 -- 172.21.15.5:0/529620811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564103bd9550 con 0x564103b5f680 2024-09-11T22:35:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.309+0000 7f0b357fa640 1 -- 172.21.15.5:0/529620811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b480104b0 con 0x564103b5f680 2024-09-11T22:35:54.430 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:35:54.430 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:35:54.430 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:35:54.430 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.310+0000 7f0b357fa640 1 -- 172.21.15.5:0/529620811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f0b44004bd0 con 0x564103b5f680 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.310+0000 7f0b561ba740 10 client.19635 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.310+0000 7f0b561ba740 10 client.19635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.310+0000 7f0b561ba740 20 client.19635 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.310+0000 7f0b561ba740 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564103b6f070 0x564103bdbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.310+0000 7f0b561ba740 4 client.19635 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.310+0000 7f0b561ba740 1 -- 172.21.15.5:0/529620811 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564103ac1d60 con 0x564103b6f070 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.310+0000 7f0b51b7d640 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564103b6f070 0x564103bdbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.310+0000 7f0b51b7d640 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564103b6f070 0x564103bdbd40 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54326576207e1312 server_cookie=453ef8ba3d7c6164 in_seq=0 out_seq=0 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.310+0000 7f0b357fa640 10 client.19635 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.312+0000 7f0b357fa640 1 -- 172.21.15.5:0/529620811 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564103ac1d60 con 0x564103b6f070 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.312+0000 7f0b357fa640 10 client.19635 handle_command_reply: tid=0 2024-09-11T22:35:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.312+0000 7f0b561ba740 1 client.19635 shutdown 2024-09-11T22:35:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 1 -- 172.21.15.5:0/529620811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 msgr2=0x564103b63010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b63010 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f0b440cd0c0 tx=0x7f0b4409b7f0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 1 -- 172.21.15.5:0/529620811 shutdown_connections 2024-09-11T22:35:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564103b6f070 0x564103bdbd40 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564103b5f680 0x564103b63010 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b357fa640 0 client.19635 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564103b54dc0 0x564103b67f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 1 --2- 172.21.15.5:0/529620811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564103b544a0 0x564103b67a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 1 -- 172.21.15.5:0/529620811 >> 172.21.15.5:0/529620811 conn(0x564103b50a30 msgr2=0x564103b57900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 1 -- 172.21.15.5:0/529620811 shutdown_connections 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 1 -- 172.21.15.5:0/529620811 wait complete. 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b561ba740 20 client.19635 trim_cache size 0 max 16384 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b53de1640 1 -- 172.21.15.5:0/4110455855 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0b380a2d70 msgr2=0x7f0b380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b53de1640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0b380a2d70 0x7f0b380a5230 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0b400096c0 tx=0x7f0b40005a20 comp rx=0 tx=0).stop 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b53de1640 1 -- 172.21.15.5:0/4110455855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b4c102ef0 msgr2=0x7f0b4c19cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b53de1640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b4c102ef0 0x7f0b4c19cdd0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400ec50 tx=0x7f0b4400c6a0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b53de1640 1 -- 172.21.15.5:0/4110455855 shutdown_connections 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b53de1640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b4c102ef0 0x7f0b4c19cdd0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b53de1640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b4c102570 0x7f0b4c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b53de1640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0b380a2d70 0x7f0b380a5230 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b53de1640 1 --2- 172.21.15.5:0/4110455855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b4c100e20 0x7f0b4c198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.313+0000 7f0b53de1640 1 -- 172.21.15.5:0/4110455855 >> 172.21.15.5:0/4110455855 conn(0x7f0b4c0fc7d0 msgr2=0x7f0b4c06a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.314+0000 7f0b53de1640 1 -- 172.21.15.5:0/4110455855 shutdown_connections 2024-09-11T22:35:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:54.314+0000 7f0b53de1640 1 -- 172.21.15.5:0/4110455855 wait complete. 2024-09-11T22:35:54.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 6412 41.82 MB/sec warmup 12 sec latency 158.323 ms 2024-09-11T22:35:54.727 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:55.535 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:35:55.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:35:55.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7047 42.00 MB/sec warmup 13 sec latency 207.620 ms 2024-09-11T22:35:55.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:55 smithi005 ceph-mon[30943]: pgmap v436: 97 pgs: 97 active+clean; 132 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-09-11T22:35:55.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.803+0000 7f9ed2b94640 1 -- 172.21.15.5:0/2886234235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc10c160 msgr2=0x7f9ecc10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.803+0000 7f9ed2b94640 1 --2- 172.21.15.5:0/2886234235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc10c160 0x7f9ecc10e600 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009ca0 tx=0x7f9ec402a950 comp rx=0 tx=0).stop 2024-09-11T22:35:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.803+0000 7f9ed2b94640 1 -- 172.21.15.5:0/2886234235 shutdown_connections 2024-09-11T22:35:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.803+0000 7f9ed2b94640 1 --2- 172.21.15.5:0/2886234235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc10c160 0x7f9ecc10e600 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.803+0000 7f9ed2b94640 1 --2- 172.21.15.5:0/2886234235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ecc1036a0 0x7f9ecc10bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.803+0000 7f9ed2b94640 1 --2- 172.21.15.5:0/2886234235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ecc102d80 0x7f9ecc103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.803+0000 7f9ed2b94640 1 -- 172.21.15.5:0/2886234235 >> 172.21.15.5:0/2886234235 conn(0x7f9ecc0fc7a0 msgr2=0x7f9ecc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.803+0000 7f9ed2b94640 1 -- 172.21.15.5:0/2886234235 shutdown_connections 2024-09-11T22:35:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.803+0000 7f9ed2b94640 1 -- 172.21.15.5:0/2886234235 wait complete. 2024-09-11T22:35:55.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.804+0000 7f9ed2b94640 1 Processor -- start 2024-09-11T22:35:55.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.804+0000 7f9ed2b94640 1 -- start start 2024-09-11T22:35:55.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed2b94640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ecc102d80 0x7f9ecc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed2b94640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ecc1036a0 0x7f9ecc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed2b94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc10c160 0x7f9ecc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed2b94640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc111f60 con 0x7f9ecc10c160 2024-09-11T22:35:55.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed2b94640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc111de0 con 0x7f9ecc1036a0 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed2b94640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1120e0 con 0x7f9ecc102d80 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed0930640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ecc102d80 0x7f9ecc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed0930640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ecc102d80 0x7f9ecc1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:40466/0 (socket says 172.21.15.5:40466) 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed0930640 1 -- 172.21.15.5:0/1575029835 learned_addr learned my addr 172.21.15.5:0/1575029835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed1131640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc10c160 0x7f9ecc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ecbfff640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ecc1036a0 0x7f9ecc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed0930640 1 -- 172.21.15.5:0/1575029835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ecc1036a0 msgr2=0x7f9ecc198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed0930640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ecc1036a0 0x7f9ecc198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed0930640 1 -- 172.21.15.5:0/1575029835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc10c160 msgr2=0x7f9ecc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed0930640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc10c160 0x7f9ecc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.805+0000 7f9ed0930640 1 -- 172.21.15.5:0/1575029835 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc19d370 con 0x7f9ecc102d80 2024-09-11T22:35:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.806+0000 7f9ed0930640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ecc102d80 0x7f9ecc1983c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000e9f0 tx=0x7f9ec000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:55.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.806+0000 7f9ec9ffb640 1 -- 172.21.15.5:0/1575029835 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000cd80 con 0x7f9ecc102d80 2024-09-11T22:35:55.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.806+0000 7f9ec9ffb640 1 -- 172.21.15.5:0/1575029835 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec0005500 con 0x7f9ecc102d80 2024-09-11T22:35:55.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.806+0000 7f9ed2b94640 1 -- 172.21.15.5:0/1575029835 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc19d660 con 0x7f9ecc102d80 2024-09-11T22:35:55.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.806+0000 7f9ec9ffb640 1 -- 172.21.15.5:0/1575029835 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0010690 con 0x7f9ecc102d80 2024-09-11T22:35:55.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.806+0000 7f9ed2b94640 1 -- 172.21.15.5:0/1575029835 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc100910 con 0x7f9ecc102d80 2024-09-11T22:35:55.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.809+0000 7f9ec9ffb640 1 -- 172.21.15.5:0/1575029835 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9ec0004f60 con 0x7f9ecc102d80 2024-09-11T22:35:55.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.810+0000 7f9ec9ffb640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ebc0a2e40 0x7f9ebc0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.810+0000 7f9ec9ffb640 1 -- 172.21.15.5:0/1575029835 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f9ec00cf7f0 con 0x7f9ecc102d80 2024-09-11T22:35:55.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.810+0000 7f9ecbfff640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ebc0a2e40 0x7f9ebc0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.810+0000 7f9ed4f6d740 1 Processor -- start 2024-09-11T22:35:55.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.810+0000 7f9ed4f6d740 1 -- start start 2024-09-11T22:35:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76bf70 0x564a8f76c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f776c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f779540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a8f6c92a0 con 0x564a8f76c890 2024-09-11T22:35:55.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564a8f670b10 con 0x564a8f76bf70 2024-09-11T22:35:55.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a8f6c5260 con 0x564a8f777150 2024-09-11T22:35:55.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ecbfff640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ebc0a2e40 0x7f9ebc0a5300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc0fed30 tx=0x7f9eb8005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:55.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ecbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f776c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed1131640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f779540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ecbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f776c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53760/0 (socket says 172.21.15.5:53760) 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ecbfff640 1 -- 172.21.15.5:0/3544536541 learned_addr learned my addr 172.21.15.5:0/3544536541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed1131640 1 -- 172.21.15.5:0/3544536541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76bf70 msgr2=0x564a8f76c350 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed1131640 1 --2- 172.21.15.5:0/3544536541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76bf70 0x564a8f76c350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed1131640 1 -- 172.21.15.5:0/3544536541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 msgr2=0x564a8f776c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed1131640 1 --2- 172.21.15.5:0/3544536541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f776c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.811+0000 7f9ed1131640 1 -- 172.21.15.5:0/3544536541 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a8f43ab90 con 0x564a8f777150 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.812+0000 7f9ed1131640 1 --2- 172.21.15.5:0/3544536541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f779540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4005cc0 tx=0x7f9ec40093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a715b047657c43bf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.812+0000 7f9eb6ffd640 1 -- 172.21.15.5:0/3544536541 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4048070 con 0x564a8f777150 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.812+0000 7f9eb6ffd640 1 -- 172.21.15.5:0/3544536541 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec402f940 con 0x564a8f777150 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.812+0000 7f9eb6ffd640 1 -- 172.21.15.5:0/3544536541 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4034040 con 0x564a8f777150 2024-09-11T22:35:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.813+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3544536541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 msgr2=0x564a8f779540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.813+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/3544536541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f779540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4005cc0 tx=0x7f9ec40093e0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.813+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3544536541 shutdown_connections 2024-09-11T22:35:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.813+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/3544536541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f779540 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.813+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/3544536541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f776c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.813+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/3544536541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76bf70 0x564a8f76c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.813+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3544536541 >> 172.21.15.5:0/3544536541 conn(0x564a8f768500 msgr2=0x564a8f7688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.813+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3544536541 shutdown_connections 2024-09-11T22:35:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.813+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3544536541 wait complete. 2024-09-11T22:35:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.814+0000 7f9ed4f6d740 1 -- start start 2024-09-11T22:35:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.814+0000 7f9ed4f6d740 1 Processor -- start 2024-09-11T22:35:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.814+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76bf70 0x564a8f774050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.814+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f774590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.814+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f774ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a8f779d70 con 0x564a8f76c890 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564a8f6516e0 con 0x564a8f76bf70 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a8f779ef0 con 0x564a8f777150 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed1131640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f774ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed1131640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f774ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:40492/0 (socket says 172.21.15.5:40492) 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed1131640 1 -- 172.21.15.5:0/2157552014 learned_addr learned my addr 172.21.15.5:0/2157552014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed4f6d740 10 client.0 _ll_register_callbacks cb 0x564a8f675a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ecbfff640 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f774590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed1131640 1 -- 172.21.15.5:0/2157552014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76bf70 msgr2=0x564a8f774050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed1131640 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76bf70 0x564a8f774050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed1131640 1 -- 172.21.15.5:0/2157552014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 msgr2=0x564a8f774590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed1131640 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f774590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed1131640 1 -- 172.21.15.5:0/2157552014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a8f783290 con 0x564a8f777150 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ecbfff640 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f774590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9ed1131640 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f774ad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009bf0 tx=0x7f9ec4009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae9eb485b08c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.815+0000 7f9eabfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.816+0000 7f9eabfff640 1 -- 172.21.15.5:0/2157552014 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4050060 con 0x564a8f777150 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.816+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/2157552014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec402f940 con 0x564a8f777150 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.816+0000 7f9eabfff640 1 -- 172.21.15.5:0/2157552014 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f9ec4030440 con 0x564a8f777150 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.816+0000 7f9eabfff640 1 -- 172.21.15.5:0/2157552014 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9ec4048070 con 0x564a8f777150 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.816+0000 7f9ed4f6d740 10 client.28672 fetch_fsmap learned FSMap version 50 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.816+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/2157552014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564a8f779d70 con 0x564a8f777150 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.816+0000 7f9eabfff640 1 -- 172.21.15.5:0/2157552014 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec402f940 con 0x564a8f777150 2024-09-11T22:35:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.816+0000 7f9eabfff640 1 -- 172.21.15.5:0/2157552014 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f9ec403a5d0 con 0x564a8f777150 2024-09-11T22:35:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.816+0000 7f9ed4f6d740 10 client.28672 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:35:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.816+0000 7f9ed4f6d740 10 client.28672 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:35:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.817+0000 7f9ed4f6d740 20 client.28672 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.817+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a8f786dd0 0x564a8f7891c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.817+0000 7f9ed4f6d740 4 client.28672 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.817+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/2157552014 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a8f6d69d0 con 0x564a8f786dd0 2024-09-11T22:35:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.817+0000 7f9ed0930640 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a8f786dd0 0x564a8f7891c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.817+0000 7f9eabfff640 10 client.28672 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.818+0000 7f9ed0930640 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a8f786dd0 0x564a8f7891c0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b088d4a710069d server_cookie=c435bf6cb881349d in_seq=0 out_seq=0 2024-09-11T22:35:55.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9eabfff640 1 -- 172.21.15.5:0/2157552014 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a8f6d69d0 con 0x564a8f786dd0 2024-09-11T22:35:55.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9eabfff640 10 client.28672 handle_command_reply: tid=0 2024-09-11T22:35:55.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 client.28672 shutdown 2024-09-11T22:35:55.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/2157552014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 msgr2=0x564a8f774ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:55.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f774ad0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009bf0 tx=0x7f9ec4009960 comp rx=0 tx=0).stop 2024-09-11T22:35:55.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/2157552014 shutdown_connections 2024-09-11T22:35:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a8f786dd0 0x564a8f7891c0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f774ad0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f774590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/2157552014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76bf70 0x564a8f774050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/2157552014 >> 172.21.15.5:0/2157552014 conn(0x564a8f768500 msgr2=0x564a8f75ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9eabfff640 0 client.28672 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/2157552014 shutdown_connections 2024-09-11T22:35:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.821+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/2157552014 wait complete. 2024-09-11T22:35:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.822+0000 7f9ed4f6d740 20 client.28672 trim_cache size 0 max 16384 2024-09-11T22:35:55.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:55 smithi159 ceph-mon[35534]: pgmap v436: 97 pgs: 97 active+clean; 132 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-09-11T22:35:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.851+0000 7f9ed4f6d740 1 Processor -- start 2024-09-11T22:35:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed4f6d740 1 -- start start 2024-09-11T22:35:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f76bf70 0x564a8f75ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f78ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f777150 0x564a8f78ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1ab6d0 con 0x564a8f76c890 2024-09-11T22:35:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1ad300 con 0x564a8f777150 2024-09-11T22:35:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1abda0 con 0x564a8f76bf70 2024-09-11T22:35:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ecbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f78ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed0930640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f76bf70 0x564a8f75ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed0930640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f76bf70 0x564a8f75ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:40498/0 (socket says 172.21.15.5:40498) 2024-09-11T22:35:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ecbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f78ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53774/0 (socket says 172.21.15.5:53774) 2024-09-11T22:35:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed0930640 1 -- 172.21.15.5:0/910955864 learned_addr learned my addr 172.21.15.5:0/910955864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ed1131640 1 --2- 172.21.15.5:0/910955864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f777150 0x564a8f78ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ecbfff640 1 -- 172.21.15.5:0/910955864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f76bf70 msgr2=0x564a8f75ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.852+0000 7f9ecbfff640 1 --2- 172.21.15.5:0/910955864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f76bf70 0x564a8f75ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.853+0000 7f9ecbfff640 1 -- 172.21.15.5:0/910955864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f777150 msgr2=0x564a8f78ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.853+0000 7f9ecbfff640 1 --2- 172.21.15.5:0/910955864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f777150 0x564a8f78ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.853+0000 7f9ecbfff640 1 -- 172.21.15.5:0/910955864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a8f7897f0 con 0x564a8f76c890 2024-09-11T22:35:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.853+0000 7f9ecbfff640 1 --2- 172.21.15.5:0/910955864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f78ea20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x564a8f7738e0 tx=0x7f9eb8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad3fd9a1c0e6fde server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.853+0000 7f9eb67fc640 1 -- 172.21.15.5:0/910955864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8005220 con 0x564a8f76c890 2024-09-11T22:35:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.853+0000 7f9eb67fc640 1 -- 172.21.15.5:0/910955864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb80047a0 con 0x564a8f76c890 2024-09-11T22:35:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.853+0000 7f9eb67fc640 1 -- 172.21.15.5:0/910955864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8003a90 con 0x564a8f76c890 2024-09-11T22:35:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.854+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/910955864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 msgr2=0x564a8f78ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.854+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/910955864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f78ea20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x564a8f7738e0 tx=0x7f9eb8014000 comp rx=0 tx=0).stop 2024-09-11T22:35:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.854+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/910955864 shutdown_connections 2024-09-11T22:35:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.854+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/910955864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f777150 0x564a8f78ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.854+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/910955864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76c890 0x564a8f78ea20 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.854+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/910955864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f76bf70 0x564a8f75ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.854+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/910955864 >> 172.21.15.5:0/910955864 conn(0x564a8f768500 msgr2=0x564a8f7731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.854+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/910955864 shutdown_connections 2024-09-11T22:35:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.854+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/910955864 wait complete. 2024-09-11T22:35:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.855+0000 7f9ed4f6d740 1 -- start start 2024-09-11T22:35:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.855+0000 7f9ed4f6d740 1 Processor -- start 2024-09-11T22:35:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76bf70 0x564a8f77b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76c890 0x564a8f77b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed4f6d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f77bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1acc30 con 0x564a8f76bf70 2024-09-11T22:35:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1ac4a0 con 0x564a8f76c890 2024-09-11T22:35:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed4f6d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1a96d0 con 0x564a8f777150 2024-09-11T22:35:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed4f6d740 10 client.0 _ll_register_callbacks cb 0x7f9ec4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed0930640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76bf70 0x564a8f77b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed0930640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76bf70 0x564a8f77b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53776/0 (socket says 172.21.15.5:53776) 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ecbfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76c890 0x564a8f77b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed0930640 1 -- 172.21.15.5:0/3842052346 learned_addr learned my addr 172.21.15.5:0/3842052346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed1131640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f77bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed1131640 1 -- 172.21.15.5:0/3842052346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76c890 msgr2=0x564a8f77b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed1131640 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76c890 0x564a8f77b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed1131640 1 -- 172.21.15.5:0/3842052346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76bf70 msgr2=0x564a8f77b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed1131640 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76bf70 0x564a8f77b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed1131640 1 -- 172.21.15.5:0/3842052346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a8f768330 con 0x564a8f777150 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.856+0000 7f9ed0930640 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76bf70 0x564a8f77b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.857+0000 7f9ecbfff640 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76c890 0x564a8f77b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.857+0000 7f9ed1131640 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f77bba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4006010 tx=0x7f9ec40536a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8354c956d98bc8c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.857+0000 7f9eabfff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:35:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.857+0000 7f9eabfff640 1 -- 172.21.15.5:0/3842052346 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4061060 con 0x564a8f777150 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.857+0000 7f9eabfff640 1 -- 172.21.15.5:0/3842052346 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f9ec4053aa0 con 0x564a8f777150 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.857+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3842052346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9eb80047a0 con 0x564a8f777150 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.857+0000 7f9eabfff640 1 -- 172.21.15.5:0/3842052346 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9ec4050040 con 0x564a8f777150 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.857+0000 7f9ed4f6d740 10 client.28678 fetch_fsmap learned FSMap version 50 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.857+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3842052346 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564a8f7f1d70 con 0x564a8f777150 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.858+0000 7f9eabfff640 1 -- 172.21.15.5:0/3842052346 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb80047a0 con 0x564a8f777150 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.858+0000 7f9eabfff640 1 -- 172.21.15.5:0/3842052346 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f9ec4064950 con 0x564a8f777150 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.858+0000 7f9ed4f6d740 10 client.28678 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.858+0000 7f9ed4f6d740 10 client.28678 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.858+0000 7f9ed4f6d740 20 client.28678 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.858+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a8f7f2560 0x564a8f7f4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.858+0000 7f9ed4f6d740 4 client.28678 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-11T22:35:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.858+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3842052346 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x564a8f6d0d20 con 0x564a8f7f2560 2024-09-11T22:35:55.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.858+0000 7f9ed0930640 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a8f7f2560 0x564a8f7f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:55.862 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:55 smithi172 ceph-mon[35461]: pgmap v436: 97 pgs: 97 active+clean; 132 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-09-11T22:35:55.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.859+0000 7f9ed0930640 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a8f7f2560 0x564a8f7f4980 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5620779902973c2b server_cookie=ac93e69311c09db4 in_seq=0 out_seq=0 2024-09-11T22:35:55.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:55.859+0000 7f9eabfff640 10 client.28678 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:55.888 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:56.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.546+0000 7f3c4dab7640 1 -- 172.21.15.5:0/969444800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481036e0 msgr2=0x7f3c48107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.546+0000 7f3c4dab7640 1 --2- 172.21.15.5:0/969444800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481036e0 0x7f3c48107b90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009930 tx=0x7f3c3c02a420 comp rx=0 tx=0).stop 2024-09-11T22:35:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.546+0000 7f3c4dab7640 1 -- 172.21.15.5:0/969444800 shutdown_connections 2024-09-11T22:35:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.546+0000 7f3c4dab7640 1 --2- 172.21.15.5:0/969444800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c481082a0 0x7f3c4810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.546+0000 7f3c4dab7640 1 --2- 172.21.15.5:0/969444800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481036e0 0x7f3c48107b90 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.546+0000 7f3c4dab7640 1 --2- 172.21.15.5:0/969444800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c48102d30 0x7f3c48103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.546+0000 7f3c4dab7640 1 -- 172.21.15.5:0/969444800 >> 172.21.15.5:0/969444800 conn(0x7f3c480fc7d0 msgr2=0x7f3c480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.546+0000 7f3c4dab7640 1 -- 172.21.15.5:0/969444800 shutdown_connections 2024-09-11T22:35:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.546+0000 7f3c4dab7640 1 -- 172.21.15.5:0/969444800 wait complete. 2024-09-11T22:35:56.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.547+0000 7f3c4dab7640 1 Processor -- start 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.547+0000 7f3c4dab7640 1 -- start start 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.547+0000 7f3c4dab7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48102d30 0x7f3c481985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.547+0000 7f3c4dab7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c481036e0 0x7f3c48198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.547+0000 7f3c4dab7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c481082a0 0x7f3c4819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.547+0000 7f3c4dab7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481120a0 con 0x7f3c48102d30 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.547+0000 7f3c4dab7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48111f20 con 0x7f3c481036e0 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.547+0000 7f3c4dab7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48112220 con 0x7f3c481082a0 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.547+0000 7f3c477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48102d30 0x7f3c481985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c46ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c481036e0 0x7f3c48198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c481082a0 0x7f3c4819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48102d30 0x7f3c481985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53796/0 (socket says 172.21.15.5:53796) 2024-09-11T22:35:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c477fe640 1 -- 172.21.15.5:0/689967597 learned_addr learned my addr 172.21.15.5:0/689967597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c477fe640 1 -- 172.21.15.5:0/689967597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c481082a0 msgr2=0x7f3c4819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c477fe640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c481082a0 0x7f3c4819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c477fe640 1 -- 172.21.15.5:0/689967597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c481036e0 msgr2=0x7f3c48198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c477fe640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c481036e0 0x7f3c48198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c477fe640 1 -- 172.21.15.5:0/689967597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4819d570 con 0x7f3c48102d30 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c477fe640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48102d30 0x7f3c481985c0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800e9f0 tx=0x7f3c3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c44ff9640 1 -- 172.21.15.5:0/689967597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800cd80 con 0x7f3c48102d30 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c4dab7640 1 -- 172.21.15.5:0/689967597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4819d860 con 0x7f3c48102d30 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c44ff9640 1 -- 172.21.15.5:0/689967597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c38010910 con 0x7f3c48102d30 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c44ff9640 1 -- 172.21.15.5:0/689967597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c380188f0 con 0x7f3c48102d30 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.548+0000 7f3c4dab7640 1 -- 172.21.15.5:0/689967597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c481a5140 con 0x7f3c48102d30 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.549+0000 7f3c4dab7640 1 -- 172.21.15.5:0/689967597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c48103110 con 0x7f3c48102d30 2024-09-11T22:35:56.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.551+0000 7f3c44ff9640 1 -- 172.21.15.5:0/689967597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3c38010ac0 con 0x7f3c48102d30 2024-09-11T22:35:56.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.551+0000 7f3c44ff9640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3c300a2cf0 0x7f3c300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:56.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.552+0000 7f3c46ffd640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3c300a2cf0 0x7f3c300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:56.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.552+0000 7f3c44ff9640 1 -- 172.21.15.5:0/689967597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f3c380cfc50 con 0x7f3c48102d30 2024-09-11T22:35:56.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.552+0000 7f3c46ffd640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3c300a2cf0 0x7f3c300a51b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005940 tx=0x7f3c3c0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:56.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.556+0000 7f3c44ff9640 1 -- 172.21.15.5:0/689967597 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c3809c1b0 con 0x7f3c48102d30 2024-09-11T22:35:56.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7233 40.12 MB/sec warmup 14 sec latency 694.565 ms 2024-09-11T22:35:56.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.811+0000 7f3c4dab7640 1 -- 172.21.15.5:0/689967597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c4810a2b0 con 0x7f3c48102d30 2024-09-11T22:35:56.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.812+0000 7f3c44ff9640 1 -- 172.21.15.5:0/689967597 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11975 (secure 0 0 0) 0x7f3c380a04b0 con 0x7f3c48102d30 2024-09-11T22:35:56.813 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:35:56.814 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"btime":"2024-09-11T22:35:49:548013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:35:48.842139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:35:56.814 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 50 2024-09-11T22:35:56.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 -- 172.21.15.5:0/689967597 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3c300a2cf0 msgr2=0x7f3c300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:56.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3c300a2cf0 0x7f3c300a51b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c005940 tx=0x7f3c3c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 -- 172.21.15.5:0/689967597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48102d30 msgr2=0x7f3c481985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:56.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48102d30 0x7f3c481985c0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800e9f0 tx=0x7f3c3800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 -- 172.21.15.5:0/689967597 shutdown_connections 2024-09-11T22:35:56.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c481082a0 0x7f3c4819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3c300a2cf0 0x7f3c300a51b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c481036e0 0x7f3c48198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 --2- 172.21.15.5:0/689967597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c48102d30 0x7f3c481985c0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:56.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 -- 172.21.15.5:0/689967597 >> 172.21.15.5:0/689967597 conn(0x7f3c480fc7d0 msgr2=0x7f3c48105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:56.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 -- 172.21.15.5:0/689967597 shutdown_connections 2024-09-11T22:35:56.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:56.815+0000 7f3c4dab7640 1 -- 172.21.15.5:0/689967597 wait complete. 2024-09-11T22:35:57.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.238+0000 7fd95744c640 1 -- 172.21.15.5:0/3646323653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd95010c110 msgr2=0x7fd95010e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.238+0000 7fd95744c640 1 --2- 172.21.15.5:0/3646323653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd95010c110 0x7fd95010e590 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fd948009ca0 tx=0x7fd94802a950 comp rx=0 tx=0).stop 2024-09-11T22:35:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.238+0000 7fd95744c640 1 -- 172.21.15.5:0/3646323653 shutdown_connections 2024-09-11T22:35:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.238+0000 7fd95744c640 1 --2- 172.21.15.5:0/3646323653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd95010c110 0x7fd95010e590 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.238+0000 7fd95744c640 1 --2- 172.21.15.5:0/3646323653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd950103660 0x7fd95010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.238+0000 7fd95744c640 1 --2- 172.21.15.5:0/3646323653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd950102d40 0x7fd950103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.238+0000 7fd95744c640 1 -- 172.21.15.5:0/3646323653 >> 172.21.15.5:0/3646323653 conn(0x7fd9500fc6b0 msgr2=0x7fd9500fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.239+0000 7fd95744c640 1 -- 172.21.15.5:0/3646323653 shutdown_connections 2024-09-11T22:35:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.239+0000 7fd95744c640 1 -- 172.21.15.5:0/3646323653 wait complete. 2024-09-11T22:35:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.239+0000 7fd95744c640 1 Processor -- start 2024-09-11T22:35:57.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.239+0000 7fd95744c640 1 -- start start 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.239+0000 7fd95744c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd950102d40 0x7fd950100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.239+0000 7fd95744c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd950103660 0x7fd950100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd95744c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd95010c110 0x7fd950101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd95744c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd950111ef0 con 0x7fd95010c110 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd95744c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd950111d70 con 0x7fd950102d40 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd95744c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd950112070 con 0x7fd950103660 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9559e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd95010c110 0x7fd950101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9559e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd95010c110 0x7fd950101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53818/0 (socket says 172.21.15.5:53818) 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9559e9640 1 -- 172.21.15.5:0/731813027 learned_addr learned my addr 172.21.15.5:0/731813027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9551e8640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd950102d40 0x7fd950100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9549e7640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd950103660 0x7fd950100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9559e9640 1 -- 172.21.15.5:0/731813027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd950103660 msgr2=0x7fd950100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9559e9640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd950103660 0x7fd950100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9559e9640 1 -- 172.21.15.5:0/731813027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd950102d40 msgr2=0x7fd950100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9559e9640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd950102d40 0x7fd950100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9559e9640 1 -- 172.21.15.5:0/731813027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9501a67e0 con 0x7fd95010c110 2024-09-11T22:35:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9549e7640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd950103660 0x7fd950100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:35:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9559e9640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd95010c110 0x7fd950101410 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd948004af0 tx=0x7fd94802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9467fc640 1 -- 172.21.15.5:0/731813027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948034040 con 0x7fd95010c110 2024-09-11T22:35:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9467fc640 1 -- 172.21.15.5:0/731813027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd948030920 con 0x7fd95010c110 2024-09-11T22:35:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.240+0000 7fd9467fc640 1 -- 172.21.15.5:0/731813027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948004820 con 0x7fd95010c110 2024-09-11T22:35:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.241+0000 7fd95744c640 1 -- 172.21.15.5:0/731813027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9501a6a70 con 0x7fd95010c110 2024-09-11T22:35:57.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.241+0000 7fd95744c640 1 -- 172.21.15.5:0/731813027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9501a6f10 con 0x7fd95010c110 2024-09-11T22:35:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.243+0000 7fd9467fc640 1 -- 172.21.15.5:0/731813027 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd948037020 con 0x7fd95010c110 2024-09-11T22:35:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd95a1e3740 1 Processor -- start 2024-09-11T22:35:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd95a1e3740 1 -- start start 2024-09-11T22:35:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e87a10 0x564a87e87df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87e9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e9b4c0 0x564a87e9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a87dec2f0 con 0x564a87e88330 2024-09-11T22:35:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564a87d94270 con 0x564a87e9b4c0 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a87de82b0 con 0x564a87e87a10 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9551e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87e9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9551e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87e9af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53832/0 (socket says 172.21.15.5:53832) 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9549e7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e9b4c0 0x564a87e9d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9551e8640 1 -- 172.21.15.5:0/3905703657 learned_addr learned my addr 172.21.15.5:0/3905703657 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9549e7640 1 --2- 172.21.15.5:0/3905703657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e87a10 0x564a87e87df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9467fc640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd93c0a2da0 0x7fd93c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9549e7640 1 -- 172.21.15.5:0/3905703657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e87a10 msgr2=0x564a87e87df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9549e7640 1 --2- 172.21.15.5:0/3905703657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e87a10 0x564a87e87df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9549e7640 1 -- 172.21.15.5:0/3905703657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 msgr2=0x564a87e9af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9549e7640 1 --2- 172.21.15.5:0/3905703657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87e9af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.244+0000 7fd9549e7640 1 -- 172.21.15.5:0/3905703657 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a87d93340 con 0x564a87e9b4c0 2024-09-11T22:35:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.245+0000 7fd9551e8640 1 --2- 172.21.15.5:0/3905703657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87e9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.245+0000 7fd9559e9640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd93c0a2da0 0x7fd93c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.245+0000 7fd9467fc640 1 -- 172.21.15.5:0/731813027 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7fd9480f0330 con 0x7fd95010c110 2024-09-11T22:35:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.245+0000 7fd9549e7640 1 --2- 172.21.15.5:0/3905703657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e9b4c0 0x564a87e9d8b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd940012770 tx=0x7fd940009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd25707a5564021e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.245+0000 7fd93b7fe640 1 -- 172.21.15.5:0/3905703657 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940012cf0 con 0x564a87e9b4c0 2024-09-11T22:35:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.245+0000 7fd9559e9640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd93c0a2da0 0x7fd93c0a5260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd9480edbd0 tx=0x7fd948041c60 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.245+0000 7fd93b7fe640 1 -- 172.21.15.5:0/3905703657 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9400107d0 con 0x564a87e9b4c0 2024-09-11T22:35:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.245+0000 7fd93b7fe640 1 -- 172.21.15.5:0/3905703657 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940003860 con 0x564a87e9b4c0 2024-09-11T22:35:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.246+0000 7fd95a1e3740 1 -- 172.21.15.5:0/3905703657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e9b4c0 msgr2=0x564a87e9d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.246+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/3905703657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e9b4c0 0x564a87e9d8b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd940012770 tx=0x7fd940009600 comp rx=0 tx=0).stop 2024-09-11T22:35:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.246+0000 7fd95a1e3740 1 -- 172.21.15.5:0/3905703657 shutdown_connections 2024-09-11T22:35:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.246+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/3905703657 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e9b4c0 0x564a87e9d8b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.246+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/3905703657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87e9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.246+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/3905703657 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e87a10 0x564a87e87df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.246+0000 7fd95a1e3740 1 -- 172.21.15.5:0/3905703657 >> 172.21.15.5:0/3905703657 conn(0x564a87e84060 msgr2=0x564a87e84440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.246+0000 7fd95a1e3740 1 -- 172.21.15.5:0/3905703657 shutdown_connections 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.246+0000 7fd95a1e3740 1 -- 172.21.15.5:0/3905703657 wait complete. 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.246+0000 7fd95a1e3740 1 -- start start 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd95a1e3740 1 Processor -- start 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 0x564a87e98410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e88330 0x564a87e98950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e9b4c0 0x564a87e98e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a87e9e090 con 0x564a87e9b4c0 2024-09-11T22:35:57.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564a87d75290 con 0x564a87e87a10 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a87e9e210 con 0x564a87e88330 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd95a1e3740 10 client.0 _ll_register_callbacks cb 0x564a87da6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd9549e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e9b4c0 0x564a87e98e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd9549e7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 0x564a87e98410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd9549e7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 0x564a87e98410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49196/0 (socket says 172.21.15.5:49196) 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd9549e7640 1 -- 172.21.15.5:0/1071568542 learned_addr learned my addr 172.21.15.5:0/1071568542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.247+0000 7fd9551e8640 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e88330 0x564a87e98950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd9549e7640 1 -- 172.21.15.5:0/1071568542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e88330 msgr2=0x564a87e98950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd9549e7640 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e88330 0x564a87e98950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd9549e7640 1 -- 172.21.15.5:0/1071568542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e9b4c0 msgr2=0x564a87e98e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd9549e7640 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e9b4c0 0x564a87e98e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd9549e7640 1 -- 172.21.15.5:0/1071568542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a87aff210 con 0x564a87e87a10 2024-09-11T22:35:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd9549e7640 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 0x564a87e98410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd940010df0 tx=0x7fd940005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a803ad1672a54589 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:35:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd938ff9640 1 -- 172.21.15.5:0/1071568542 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940025050 con 0x564a87e87a10 2024-09-11T22:35:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1071568542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9400107d0 con 0x564a87e87a10 2024-09-11T22:35:57.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd938ff9640 1 -- 172.21.15.5:0/1071568542 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7fd940005020 con 0x564a87e87a10 2024-09-11T22:35:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd938ff9640 1 -- 172.21.15.5:0/1071568542 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd940034050 con 0x564a87e87a10 2024-09-11T22:35:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd95a1e3740 10 client.28859 fetch_fsmap learned FSMap version 50 2024-09-11T22:35:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1071568542 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564a87bcb620 con 0x564a87e87a10 2024-09-11T22:35:57.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.248+0000 7fd938ff9640 1 -- 172.21.15.5:0/1071568542 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9400107d0 con 0x564a87e87a10 2024-09-11T22:35:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.249+0000 7fd938ff9640 1 -- 172.21.15.5:0/1071568542 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fd940004cf0 con 0x564a87e87a10 2024-09-11T22:35:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.249+0000 7fd95a1e3740 10 client.28859 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:35:57.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.249+0000 7fd95a1e3740 10 client.28859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.249+0000 7fd95a1e3740 20 client.28859 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.249+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a87eab300 0x564a87ead6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.249+0000 7fd95a1e3740 4 client.28859 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.249+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1071568542 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a87df9a20 con 0x564a87eab300 2024-09-11T22:35:57.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.249+0000 7fd9551e8640 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a87eab300 0x564a87ead6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.250+0000 7fd938ff9640 10 client.28859 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.250+0000 7fd9551e8640 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a87eab300 0x564a87ead6f0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8b9f9e9f0df1a0 server_cookie=7084245424cbfcf in_seq=0 out_seq=0 2024-09-11T22:35:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd938ff9640 1 -- 172.21.15.5:0/1071568542 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a87df9a20 con 0x564a87eab300 2024-09-11T22:35:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd938ff9640 10 client.28859 handle_command_reply: tid=0 2024-09-11T22:35:57.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 client.28859 shutdown 2024-09-11T22:35:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1071568542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 msgr2=0x564a87e98410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 0x564a87e98410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd940010df0 tx=0x7fd940005f90 comp rx=0 tx=0).stop 2024-09-11T22:35:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1071568542 shutdown_connections 2024-09-11T22:35:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a87eab300 0x564a87ead6f0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e9b4c0 0x564a87e98e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e88330 0x564a87e98950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1071568542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 0x564a87e98410 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd938ff9640 0 client.28859 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1071568542 >> 172.21.15.5:0/1071568542 conn(0x564a87e84060 msgr2=0x564a87e89d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:57.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1071568542 shutdown_connections 2024-09-11T22:35:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1071568542 wait complete. 2024-09-11T22:35:57.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.251+0000 7fd95a1e3740 20 client.28859 trim_cache size 0 max 16384 2024-09-11T22:35:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd95a1e3740 1 Processor -- start 2024-09-11T22:35:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd95a1e3740 1 -- start start 2024-09-11T22:35:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 0x564a87eb2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87eb3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 0x564a87e95650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501ab700 con 0x564a87e88330 2024-09-11T22:35:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501ad330 con 0x564a87e87a10 2024-09-11T22:35:57.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501abdd0 con 0x564a87e9b4c0 2024-09-11T22:35:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd9551e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87eb3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd9551e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87eb3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53854/0 (socket says 172.21.15.5:53854) 2024-09-11T22:35:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd9551e8640 1 -- 172.21.15.5:0/210221818 learned_addr learned my addr 172.21.15.5:0/210221818 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd9549e7640 1 --2- 172.21.15.5:0/210221818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 0x564a87e95650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.268+0000 7fd9549e7640 1 --2- 172.21.15.5:0/210221818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 0x564a87eb2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.269+0000 7fd9551e8640 1 -- 172.21.15.5:0/210221818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 msgr2=0x564a87e95650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.269+0000 7fd9551e8640 1 --2- 172.21.15.5:0/210221818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 0x564a87e95650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.269+0000 7fd9551e8640 1 -- 172.21.15.5:0/210221818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 msgr2=0x564a87eb2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.269+0000 7fd9551e8640 1 --2- 172.21.15.5:0/210221818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 0x564a87eb2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.269+0000 7fd9551e8640 1 -- 172.21.15.5:0/210221818 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a87e88160 con 0x564a87e88330 2024-09-11T22:35:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.269+0000 7fd9549e7640 1 --2- 172.21.15.5:0/210221818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 0x564a87e95650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.269+0000 7fd9551e8640 1 --2- 172.21.15.5:0/210221818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87eb3430 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd94c005850 tx=0x7fd94c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b89f1eaecce9e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.269+0000 7fd93affd640 1 -- 172.21.15.5:0/210221818 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c005020 con 0x564a87e88330 2024-09-11T22:35:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.269+0000 7fd93affd640 1 -- 172.21.15.5:0/210221818 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd94c0047a0 con 0x564a87e88330 2024-09-11T22:35:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.269+0000 7fd93affd640 1 -- 172.21.15.5:0/210221818 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c0116a0 con 0x564a87e88330 2024-09-11T22:35:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 -- 172.21.15.5:0/210221818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 msgr2=0x564a87eb3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/210221818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87eb3430 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd94c005850 tx=0x7fd94c00b390 comp rx=0 tx=0).stop 2024-09-11T22:35:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 -- 172.21.15.5:0/210221818 shutdown_connections 2024-09-11T22:35:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/210221818 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 0x564a87e95650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/210221818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e88330 0x564a87eb3430 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/210221818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e87a10 0x564a87eb2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 -- 172.21.15.5:0/210221818 >> 172.21.15.5:0/210221818 conn(0x564a87e84060 msgr2=0x564a87e9a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 -- 172.21.15.5:0/210221818 shutdown_connections 2024-09-11T22:35:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 -- 172.21.15.5:0/210221818 wait complete. 2024-09-11T22:35:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 -- start start 2024-09-11T22:35:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.270+0000 7fd95a1e3740 1 Processor -- start 2024-09-11T22:35:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e87a10 0x564a87ea3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e88330 0x564a87e9ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd95a1e3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 0x564a87e9f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501acc60 con 0x564a87e87a10 2024-09-11T22:35:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501ac4d0 con 0x564a87e88330 2024-09-11T22:35:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd95a1e3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501a9720 con 0x564a87e9b4c0 2024-09-11T22:35:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd95a1e3740 10 client.0 _ll_register_callbacks cb 0x7fd94c004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9549e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e87a10 0x564a87ea3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9549e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e87a10 0x564a87ea3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53858/0 (socket says 172.21.15.5:53858) 2024-09-11T22:35:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9549e7640 1 -- 172.21.15.5:0/1504692269 learned_addr learned my addr 172.21.15.5:0/1504692269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9551e8640 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e88330 0x564a87e9ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9549e7640 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 0x564a87e9f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9549e7640 1 -- 172.21.15.5:0/1504692269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 msgr2=0x564a87e9f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9549e7640 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 0x564a87e9f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9549e7640 1 -- 172.21.15.5:0/1504692269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e88330 msgr2=0x564a87e9ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9549e7640 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e88330 0x564a87e9ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9549e7640 1 -- 172.21.15.5:0/1504692269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a87e9fc60 con 0x564a87e87a10 2024-09-11T22:35:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9551e8640 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e88330 0x564a87e9ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:35:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd9549e7640 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e87a10 0x564a87ea3bb0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd940003970 tx=0x7fd940010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f517e05610bf1ed server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.271+0000 7fd938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.272+0000 7fd938ff9640 1 -- 172.21.15.5:0/1504692269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940025040 con 0x564a87e87a10 2024-09-11T22:35:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.272+0000 7fd938ff9640 1 -- 172.21.15.5:0/1504692269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7fd940003cf0 con 0x564a87e87a10 2024-09-11T22:35:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.272+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1504692269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd94c0047a0 con 0x564a87e87a10 2024-09-11T22:35:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.272+0000 7fd938ff9640 1 -- 172.21.15.5:0/1504692269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fd940028040 con 0x564a87e87a10 2024-09-11T22:35:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.272+0000 7fd95a1e3740 10 client.19677 fetch_fsmap learned FSMap version 50 2024-09-11T22:35:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.272+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1504692269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564a87f15ea0 con 0x564a87e87a10 2024-09-11T22:35:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.272+0000 7fd938ff9640 1 -- 172.21.15.5:0/1504692269 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd94c0047a0 con 0x564a87e87a10 2024-09-11T22:35:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.272+0000 7fd938ff9640 1 -- 172.21.15.5:0/1504692269 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fd94002e9a0 con 0x564a87e87a10 2024-09-11T22:35:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.272+0000 7fd95a1e3740 10 client.19677 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:35:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.272+0000 7fd95a1e3740 10 client.19677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:35:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.273+0000 7fd95a1e3740 20 client.19677 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:57.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.273+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a87f16710 0x564a87f18b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.273+0000 7fd95a1e3740 4 client.19677 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:35:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.273+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1504692269 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a87df3d70 con 0x564a87f16710 2024-09-11T22:35:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.273+0000 7fd9551e8640 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a87f16710 0x564a87f18b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.273+0000 7fd9551e8640 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a87f16710 0x564a87f18b30 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921cfa0710543c71 server_cookie=e29df43755a4777c in_seq=0 out_seq=0 2024-09-11T22:35:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.273+0000 7fd938ff9640 10 client.19677 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:57.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:35:57.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9eabfff640 1 -- 172.21.15.5:0/3842052346 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564a8f6d0d20 con 0x564a8f7f2560 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9eabfff640 10 client.28678 handle_command_reply: tid=0 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9ed4f6d740 1 client.28678 shutdown 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3842052346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 msgr2=0x564a8f77bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f77bba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4006010 tx=0x7f9ec40536a0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3842052346 shutdown_connections 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a8f7f2560 0x564a8f7f4980 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a8f777150 0x564a8f77bba0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a8f76c890 0x564a8f77b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9eabfff640 0 client.28678 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9ed4f6d740 1 --2- 172.21.15.5:0/3842052346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a8f76bf70 0x564a8f77b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.480+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3842052346 >> 172.21.15.5:0/3842052346 conn(0x564a8f768500 msgr2=0x564a8f7760d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3842052346 shutdown_connections 2024-09-11T22:35:57.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed4f6d740 1 -- 172.21.15.5:0/3842052346 wait complete. 2024-09-11T22:35:57.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed4f6d740 20 client.28678 trim_cache size 0 max 16384 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 50, 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 50, 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout: "91bd8d7f": { 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 50, 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:35:57.484 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:57.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:57.486 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:35:57.486 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:35:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 -- 172.21.15.5:0/1575029835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ebc0a2e40 msgr2=0x7f9ebc0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ebc0a2e40 0x7f9ebc0a5300 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc0fed30 tx=0x7f9eb8005f90 comp rx=0 tx=0).stop 2024-09-11T22:35:57.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 -- 172.21.15.5:0/1575029835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ecc102d80 msgr2=0x7f9ecc1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ecc102d80 0x7f9ecc1983c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000e9f0 tx=0x7f9ec000edf0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 -- 172.21.15.5:0/1575029835 shutdown_connections 2024-09-11T22:35:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc10c160 0x7f9ecc19cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ebc0a2e40 0x7f9ebc0a5300 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ecc1036a0 0x7f9ecc198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 --2- 172.21.15.5:0/1575029835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ecc102d80 0x7f9ecc1983c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 -- 172.21.15.5:0/1575029835 >> 172.21.15.5:0/1575029835 conn(0x7f9ecc0fc7a0 msgr2=0x7f9ecc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 -- 172.21.15.5:0/1575029835 shutdown_connections 2024-09-11T22:35:57.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:57.481+0000 7f9ed2b94640 1 -- 172.21.15.5:0/1575029835 wait complete. 2024-09-11T22:35:57.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 37.45 MB/sec warmup 15 sec latency 1694.604 ms 2024-09-11T22:35:57.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:57 smithi005 ceph-mon[30943]: pgmap v437: 97 pgs: 97 active+clean; 132 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-09-11T22:35:57.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:57 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/689967597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:35:57.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:57 smithi159 ceph-mon[35534]: pgmap v437: 97 pgs: 97 active+clean; 132 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-09-11T22:35:57.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:57 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/689967597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:35:57.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:57 smithi172 ceph-mon[35461]: pgmap v437: 97 pgs: 97 active+clean; 132 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 423 op/s 2024-09-11T22:35:57.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:57 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/689967597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:35:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.963+0000 7fd938ff9640 1 -- 172.21.15.5:0/1504692269 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a87df3d70 con 0x564a87f16710 2024-09-11T22:35:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.963+0000 7fd938ff9640 10 client.19677 handle_command_reply: tid=0 2024-09-11T22:35:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 client.19677 shutdown 2024-09-11T22:35:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1504692269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e87a10 msgr2=0x564a87ea3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e87a10 0x564a87ea3bb0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd940003970 tx=0x7fd940010f90 comp rx=0 tx=0).stop 2024-09-11T22:35:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1504692269 shutdown_connections 2024-09-11T22:35:57.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a87f16710 0x564a87f18b30 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a87e9b4c0 0x564a87e9f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a87e88330 0x564a87e9ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd938ff9640 0 client.19677 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 --2- 172.21.15.5:0/1504692269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a87e87a10 0x564a87ea3bb0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1504692269 >> 172.21.15.5:0/1504692269 conn(0x564a87e84060 msgr2=0x564a87e829a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1504692269 shutdown_connections 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 1 -- 172.21.15.5:0/1504692269 wait complete. 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.964+0000 7fd95a1e3740 20 client.19677 trim_cache size 0 max 16384 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:35:57.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:35:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:35:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 -- 172.21.15.5:0/731813027 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd93c0a2da0 msgr2=0x7fd93c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd93c0a2da0 0x7fd93c0a5260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd9480edbd0 tx=0x7fd948041c60 comp rx=0 tx=0).stop 2024-09-11T22:35:57.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 -- 172.21.15.5:0/731813027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd95010c110 msgr2=0x7fd950101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd95010c110 0x7fd950101410 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd948004af0 tx=0x7fd94802f5d0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 -- 172.21.15.5:0/731813027 shutdown_connections 2024-09-11T22:35:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd95010c110 0x7fd950101410 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd950103660 0x7fd950100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd93c0a2da0 0x7fd93c0a5260 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 --2- 172.21.15.5:0/731813027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd950102d40 0x7fd950100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 -- 172.21.15.5:0/731813027 >> 172.21.15.5:0/731813027 conn(0x7fd9500fc6b0 msgr2=0x7fd9500fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 -- 172.21.15.5:0/731813027 shutdown_connections 2024-09-11T22:35:57.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:35:57.965+0000 7fd95744c640 1 -- 172.21.15.5:0/731813027 wait complete. 2024-09-11T22:35:58.609 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '91bd8d7f', quiesce took 7.1 seconds. Will release after: 22.9 2024-09-11T22:35:58.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 35.11 MB/sec warmup 16 sec latency 2694.657 ms 2024-09-11T22:35:58.683 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:35:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.199+0000 7f4df5854640 1 -- 172.21.15.5:0/854039059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4df0106f20 msgr2=0x7f4df010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.199+0000 7f4df5854640 1 --2- 172.21.15.5:0/854039059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4df0106f20 0x7f4df010eaa0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009ca0 tx=0x7f4de802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.199+0000 7f4df5854640 1 -- 172.21.15.5:0/854039059 shutdown_connections 2024-09-11T22:35:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.199+0000 7f4df5854640 1 --2- 172.21.15.5:0/854039059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4df0106f20 0x7f4df010eaa0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.199+0000 7f4df5854640 1 --2- 172.21.15.5:0/854039059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4df01027a0 0x7f4df01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.199+0000 7f4df5854640 1 --2- 172.21.15.5:0/854039059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df0101df0 0x7f4df01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.199+0000 7f4df5854640 1 -- 172.21.15.5:0/854039059 >> 172.21.15.5:0/854039059 conn(0x7f4df00fd8f0 msgr2=0x7f4df00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.199+0000 7f4df5854640 1 -- 172.21.15.5:0/854039059 shutdown_connections 2024-09-11T22:35:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.199+0000 7f4df5854640 1 -- 172.21.15.5:0/854039059 wait complete. 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4df5854640 1 Processor -- start 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4df5854640 1 -- start start 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4df5854640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df0101df0 0x7f4df01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4df5854640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4df01027a0 0x7f4df0198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4df5854640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4df0106f20 0x7f4df019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4df5854640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4df0111f80 con 0x7f4df0106f20 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4df5854640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4df0111e00 con 0x7f4df0101df0 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4df5854640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4df0112100 con 0x7f4df01027a0 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4def7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4df0106f20 0x7f4df019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4def7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4df0106f20 0x7f4df019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53884/0 (socket says 172.21.15.5:53884) 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4def7fe640 1 -- 172.21.15.5:0/1785906132 learned_addr learned my addr 172.21.15.5:0/1785906132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4deeffd640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df0101df0 0x7f4df01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4df01027a0 0x7f4df0198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4def7fe640 1 -- 172.21.15.5:0/1785906132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4df01027a0 msgr2=0x7f4df0198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4def7fe640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4df01027a0 0x7f4df0198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4def7fe640 1 -- 172.21.15.5:0/1785906132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df0101df0 msgr2=0x7f4df01983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4def7fe640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df0101df0 0x7f4df01983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4def7fe640 1 -- 172.21.15.5:0/1785906132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df019d390 con 0x7f4df0106f20 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.200+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4df01027a0 0x7f4df0198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.201+0000 7f4def7fe640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4df0106f20 0x7f4df019ccb0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f4de8031a20 tx=0x7f4de8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.201+0000 7f4df4852640 1 -- 172.21.15.5:0/1785906132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de8003c30 con 0x7f4df0106f20 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.201+0000 7f4df5854640 1 -- 172.21.15.5:0/1785906132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df019d620 con 0x7f4df0106f20 2024-09-11T22:35:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.201+0000 7f4df4852640 1 -- 172.21.15.5:0/1785906132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de8003dd0 con 0x7f4df0106f20 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.201+0000 7f4df4852640 1 -- 172.21.15.5:0/1785906132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de802fcf0 con 0x7f4df0106f20 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.201+0000 7f4df5854640 1 -- 172.21.15.5:0/1785906132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df01a4f60 con 0x7f4df0106f20 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.203+0000 7f4df4852640 1 -- 172.21.15.5:0/1785906132 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4de8004050 con 0x7f4df0106f20 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4df7c2d740 1 Processor -- start 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4df7c2d740 1 -- start start 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac5b40 0x5586dfac4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac45d0 0x5586dfac4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfacf370 0x5586dfad1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5586dfa2a2f0 con 0x5586dfac5b40 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586df9d1cf0 con 0x5586dfac45d0 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5586dfa262b0 con 0x5586dfacf370 2024-09-11T22:35:59.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4df4852640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4dd80a2cf0 0x7f4dd80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4dee7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac5b40 0x5586dfac4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4dee7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac5b40 0x5586dfac4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53886/0 (socket says 172.21.15.5:53886) 2024-09-11T22:35:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4dee7fc640 1 -- 172.21.15.5:0/3480133249 learned_addr learned my addr 172.21.15.5:0/3480133249 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4deeffd640 1 --2- 172.21.15.5:0/3480133249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac45d0 0x5586dfac4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4def7fe640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4dd80a2cf0 0x7f4dd80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4df4852640 1 -- 172.21.15.5:0/1785906132 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f4de80f0a10 con 0x7f4df0106f20 2024-09-11T22:35:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/3480133249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfacf370 0x5586dfad1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4dee7fc640 1 -- 172.21.15.5:0/3480133249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfacf370 msgr2=0x5586dfad1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/3480133249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfacf370 0x5586dfad1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4dee7fc640 1 -- 172.21.15.5:0/3480133249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac45d0 msgr2=0x5586dfac4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/3480133249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac45d0 0x5586dfac4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.204+0000 7f4dee7fc640 1 -- 172.21.15.5:0/3480133249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586df9b63a0 con 0x5586dfac5b40 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.205+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/3480133249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac5b40 0x5586dfac4090 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009f70 tx=0x7f4de4009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c79d917f565a04 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.205+0000 7f4def7fe640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4dd80a2cf0 0x7f4dd80a51b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4de80bcf90 tx=0x7f4de80dd040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.205+0000 7f4dd5ffb640 1 -- 172.21.15.5:0/3480133249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de40150f0 con 0x5586dfac5b40 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.205+0000 7f4dd5ffb640 1 -- 172.21.15.5:0/3480133249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de4005500 con 0x5586dfac5b40 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.205+0000 7f4dd5ffb640 1 -- 172.21.15.5:0/3480133249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de40134c0 con 0x5586dfac5b40 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.205+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3480133249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac5b40 msgr2=0x5586dfac4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.205+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/3480133249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac5b40 0x5586dfac4090 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009f70 tx=0x7f4de4009590 comp rx=0 tx=0).stop 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.206+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3480133249 shutdown_connections 2024-09-11T22:35:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.206+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/3480133249 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfacf370 0x5586dfad1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.206+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/3480133249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac45d0 0x5586dfac4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.206+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/3480133249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac5b40 0x5586dfac4090 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.206+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3480133249 >> 172.21.15.5:0/3480133249 conn(0x5586dfa8d310 msgr2=0x5586dfac0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.206+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3480133249 shutdown_connections 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.206+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3480133249 wait complete. 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.206+0000 7f4df7c2d740 1 -- start start 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.206+0000 7f4df7c2d740 1 Processor -- start 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac45d0 0x5586dfacece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac5b40 0x5586dfacbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfacc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5586dfad2010 con 0x5586dfacf370 2024-09-11T22:35:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586df9e5060 con 0x5586dfac5b40 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5586dfad2190 con 0x5586dfac45d0 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4df7c2d740 10 client.0 _ll_register_callbacks cb 0x5586df7388c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dee7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfacc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dee7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfacc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53900/0 (socket says 172.21.15.5:53900) 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dee7fc640 1 -- 172.21.15.5:0/3346236866 learned_addr learned my addr 172.21.15.5:0/3346236866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4deeffd640 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac5b40 0x5586dfacbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac45d0 0x5586dfacece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dee7fc640 1 -- 172.21.15.5:0/3346236866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac45d0 msgr2=0x5586dfacece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac45d0 0x5586dfacece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dee7fc640 1 -- 172.21.15.5:0/3346236866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac5b40 msgr2=0x5586dfacbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac5b40 0x5586dfacbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dee7fc640 1 -- 172.21.15.5:0/3346236866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586df79b3c0 con 0x5586dfacf370 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfacc4f0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4de40049d0 tx=0x7f4de4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af416e112461a1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dceffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dceffd640 1 -- 172.21.15.5:0/3346236866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4004e80 con 0x5586dfacf370 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4dceffd640 1 -- 172.21.15.5:0/3346236866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f4de4005020 con 0x5586dfacf370 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.207+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3346236866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4de4005500 con 0x5586dfacf370 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4dceffd640 1 -- 172.21.15.5:0/3346236866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f4de4005220 con 0x5586dfacf370 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4df7c2d740 10 client.19689 fetch_fsmap learned FSMap version 50 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3346236866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5586dfadda20 con 0x5586dfacf370 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4dceffd640 1 -- 172.21.15.5:0/3346236866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de4005500 con 0x5586dfacf370 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4dceffd640 1 -- 172.21.15.5:0/3346236866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f4de400c070 con 0x5586dfacf370 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4df7c2d740 10 client.19689 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4df7c2d740 10 client.19689 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:35:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4df7c2d740 20 client.19689 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5586dfadee20 0x5586dfae1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4df7c2d740 4 client.19689 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3346236866 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586dfa37860 con 0x5586dfadee20 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.208+0000 7f4deeffd640 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5586dfadee20 0x5586dfae1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.209+0000 7f4dceffd640 10 client.19689 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.209+0000 7f4deeffd640 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5586dfadee20 0x5586dfae1210 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96dfc60c9f0a0b6a server_cookie=2c22016da89022b8 in_seq=0 out_seq=0 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.209+0000 7f4dceffd640 1 -- 172.21.15.5:0/3346236866 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5586dfa37860 con 0x5586dfadee20 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.209+0000 7f4dceffd640 10 client.19689 handle_command_reply: tid=0 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.209+0000 7f4df7c2d740 1 client.19689 shutdown 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3346236866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 msgr2=0x5586dfacc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfacc4f0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f4de40049d0 tx=0x7f4de4004ad0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3346236866 shutdown_connections 2024-09-11T22:35:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5586dfadee20 0x5586dfae1210 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfacc4f0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac5b40 0x5586dfacbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4dceffd640 0 client.19689 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:35:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/3346236866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac45d0 0x5586dfacece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3346236866 >> 172.21.15.5:0/3346236866 conn(0x5586dfa8d310 msgr2=0x5586dfac6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3346236866 shutdown_connections 2024-09-11T22:35:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 1 -- 172.21.15.5:0/3346236866 wait complete. 2024-09-11T22:35:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.210+0000 7f4df7c2d740 20 client.19689 trim_cache size 0 max 16384 2024-09-11T22:35:59.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.223+0000 7f4df7c2d740 1 Processor -- start 2024-09-11T22:35:59.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.223+0000 7f4df7c2d740 1 -- start start 2024-09-11T22:35:59.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.223+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac45d0 0x5586dfadb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.223+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac5b40 0x5586dfad7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.223+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfacf370 0x5586dfad7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.223+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4df01adad0 con 0x5586dfac45d0 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.223+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4df01af700 con 0x5586dfacf370 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.223+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4df01ae1a0 con 0x5586dfac5b40 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dee7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac45d0 0x5586dfadb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dee7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac45d0 0x5586dfadb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53908/0 (socket says 172.21.15.5:53908) 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dee7fc640 1 -- 172.21.15.5:0/2641374075 learned_addr learned my addr 172.21.15.5:0/2641374075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dee7fc640 1 -- 172.21.15.5:0/2641374075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac5b40 msgr2=0x5586dfad7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/2641374075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac5b40 0x5586dfad7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dee7fc640 1 -- 172.21.15.5:0/2641374075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfacf370 msgr2=0x5586dfad7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/2641374075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfacf370 0x5586dfad7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dee7fc640 1 -- 172.21.15.5:0/2641374075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586dfad2150 con 0x5586dfac45d0 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/2641374075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac45d0 0x5586dfadb2b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009860 tx=0x7f4de4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff84d71ad6add1f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dd57fa640 1 -- 172.21.15.5:0/2641374075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4015e60 con 0x5586dfac45d0 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dd57fa640 1 -- 172.21.15.5:0/2641374075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de4026040 con 0x5586dfac45d0 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.224+0000 7f4dd57fa640 1 -- 172.21.15.5:0/2641374075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de40035f0 con 0x5586dfac45d0 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2641374075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac45d0 msgr2=0x5586dfadb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/2641374075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac45d0 0x5586dfadb2b0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f4de4009860 tx=0x7f4de4003cd0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2641374075 shutdown_connections 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/2641374075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfacf370 0x5586dfad7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/2641374075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac5b40 0x5586dfad7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/2641374075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfac45d0 0x5586dfadb2b0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2641374075 >> 172.21.15.5:0/2641374075 conn(0x5586dfa8d310 msgr2=0x5586dfacb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2641374075 shutdown_connections 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2641374075 wait complete. 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 -- start start 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.225+0000 7f4df7c2d740 1 Processor -- start 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac45d0 0x5586dfac8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac5b40 0x5586dfac8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4df7c2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfac8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4df01af030 con 0x5586dfacf370 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4df01ae8a0 con 0x5586dfac5b40 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4df7c2d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4df01abad0 con 0x5586dfac45d0 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4df7c2d740 10 client.0 _ll_register_callbacks cb 0x7f4de000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dee7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfac8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dee7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfac8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53916/0 (socket says 172.21.15.5:53916) 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dee7fc640 1 -- 172.21.15.5:0/2535111346 learned_addr learned my addr 172.21.15.5:0/2535111346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:35:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4deeffd640 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac5b40 0x5586dfac8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac45d0 0x5586dfac8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dee7fc640 1 -- 172.21.15.5:0/2535111346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac45d0 msgr2=0x5586dfac8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac45d0 0x5586dfac8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dee7fc640 1 -- 172.21.15.5:0/2535111346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac5b40 msgr2=0x5586dfac8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac5b40 0x5586dfac8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dee7fc640 1 -- 172.21.15.5:0/2535111346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586dfac93c0 con 0x5586dfacf370 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dee7fc640 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfac8c80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4de4015360 tx=0x7f4de4017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6043e19e9014fe4e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.226+0000 7f4dceffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4dceffd640 1 -- 172.21.15.5:0/2535111346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de4009320 con 0x5586dfacf370 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4dceffd640 1 -- 172.21.15.5:0/2535111346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f4de401b070 con 0x5586dfacf370 2024-09-11T22:35:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2535111346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4de4026040 con 0x5586dfacf370 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4dceffd640 1 -- 172.21.15.5:0/2535111346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f4de4004680 con 0x5586dfacf370 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4df7c2d740 10 client.19695 fetch_fsmap learned FSMap version 50 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2535111346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5586dfb4dd10 con 0x5586dfacf370 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4dceffd640 1 -- 172.21.15.5:0/2535111346 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4de4026040 con 0x5586dfacf370 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4dceffd640 1 -- 172.21.15.5:0/2535111346 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f4de400c070 con 0x5586dfacf370 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4df7c2d740 10 client.19695 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4df7c2d740 10 client.19695 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4df7c2d740 20 client.19695 populate_metadata read hostname 'smithi005' 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5586dfadee20 0x5586dfb50380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4df7c2d740 4 client.19695 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2535111346 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586dfa31d70 con 0x5586dfadee20 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.227+0000 7f4deeffd640 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5586dfadee20 0x5586dfb50380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:35:59.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4dceffd640 10 client.19695 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4deeffd640 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5586dfadee20 0x5586dfb50380 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72e5642e98947f0a server_cookie=991fc9ef87885657 in_seq=0 out_seq=0 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4dceffd640 1 -- 172.21.15.5:0/2535111346 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586dfa31d70 con 0x5586dfadee20 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4dceffd640 10 client.19695 handle_command_reply: tid=0 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 client.19695 shutdown 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2535111346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 msgr2=0x5586dfac8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfac8c80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f4de4015360 tx=0x7f4de4017040 comp rx=0 tx=0).stop 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2535111346 shutdown_connections 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5586dfadee20 0x5586dfb50380 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4dceffd640 0 client.19695 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5586dfacf370 0x5586dfac8c80 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5586dfac5b40 0x5586dfac8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 --2- 172.21.15.5:0/2535111346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5586dfac45d0 0x5586dfac8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2535111346 >> 172.21.15.5:0/2535111346 conn(0x5586dfa8d310 msgr2=0x5586dfacd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:59.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2535111346 shutdown_connections 2024-09-11T22:35:59.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 1 -- 172.21.15.5:0/2535111346 wait complete. 2024-09-11T22:35:59.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.228+0000 7f4df7c2d740 20 client.19695 trim_cache size 0 max 16384 2024-09-11T22:35:59.243 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:35:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 -- 172.21.15.5:0/1785906132 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4dd80a2cf0 msgr2=0x7f4dd80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4dd80a2cf0 0x7f4dd80a51b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4de80bcf90 tx=0x7f4de80dd040 comp rx=0 tx=0).stop 2024-09-11T22:35:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 -- 172.21.15.5:0/1785906132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4df0106f20 msgr2=0x7f4df019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:35:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4df0106f20 0x7f4df019ccb0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f4de8031a20 tx=0x7f4de8031a50 comp rx=0 tx=0).stop 2024-09-11T22:35:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 -- 172.21.15.5:0/1785906132 shutdown_connections 2024-09-11T22:35:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4df0106f20 0x7f4df019ccb0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4df01027a0 0x7f4df0198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4dd80a2cf0 0x7f4dd80a51b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 --2- 172.21.15.5:0/1785906132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4df0101df0 0x7f4df01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:35:59.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 -- 172.21.15.5:0/1785906132 >> 172.21.15.5:0/1785906132 conn(0x7f4df00fd8f0 msgr2=0x7f4df00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:35:59.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 -- 172.21.15.5:0/1785906132 shutdown_connections 2024-09-11T22:35:59.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:35:59.229+0000 7f4df5854640 1 -- 172.21.15.5:0/1785906132 wait complete. 2024-09-11T22:35:59.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 33.04 MB/sec warmup 17 sec latency 3694.773 ms 2024-09-11T22:35:59.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:35:59 smithi005 ceph-mon[30943]: pgmap v438: 97 pgs: 97 active+clean; 211 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 618 op/s 2024-09-11T22:35:59.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:35:59 smithi159 ceph-mon[35534]: pgmap v438: 97 pgs: 97 active+clean; 211 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 618 op/s 2024-09-11T22:35:59.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:35:59 smithi172 ceph-mon[35461]: pgmap v438: 97 pgs: 97 active+clean; 211 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 618 op/s 2024-09-11T22:36:00.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:36:00.457 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:00.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 31.21 MB/sec warmup 18 sec latency 4694.862 ms 2024-09-11T22:36:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.021+0000 7f1f98f96640 1 -- 172.21.15.5:0/756834151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f94106a80 msgr2=0x7f1f94106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.021+0000 7f1f98f96640 1 --2- 172.21.15.5:0/756834151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f94106a80 0x7f1f94106e60 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009930 tx=0x7f1f8402a420 comp rx=0 tx=0).stop 2024-09-11T22:36:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.022+0000 7f1f98f96640 1 -- 172.21.15.5:0/756834151 shutdown_connections 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.022+0000 7f1f98f96640 1 --2- 172.21.15.5:0/756834151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94101420 0x7f1f9410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.022+0000 7f1f98f96640 1 --2- 172.21.15.5:0/756834151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f94100a80 0x7f1f94100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.022+0000 7f1f98f96640 1 --2- 172.21.15.5:0/756834151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f94106a80 0x7f1f94106e60 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.022+0000 7f1f98f96640 1 -- 172.21.15.5:0/756834151 >> 172.21.15.5:0/756834151 conn(0x7f1f940fc7a0 msgr2=0x7f1f940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.022+0000 7f1f98f96640 1 -- 172.21.15.5:0/756834151 shutdown_connections 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.022+0000 7f1f98f96640 1 -- 172.21.15.5:0/756834151 wait complete. 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.022+0000 7f1f98f96640 1 Processor -- start 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f98f96640 1 -- start start 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f98f96640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f94100a80 0x7f1f94199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f98f96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f94101420 0x7f1f94199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f98f96640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94106a80 0x7f1f941a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f98f96640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94110e80 con 0x7f1f94101420 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f98f96640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94110d00 con 0x7f1f94106a80 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f98f96640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1f94111000 con 0x7f1f94100a80 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f91d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f94101420 0x7f1f94199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f91d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f94101420 0x7f1f94199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53932/0 (socket says 172.21.15.5:53932) 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f91d9b640 1 -- 172.21.15.5:0/2106525771 learned_addr learned my addr 172.21.15.5:0/2106525771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f92d9d640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94106a80 0x7f1f941a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f9259c640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f94100a80 0x7f1f94199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f91d9b640 1 -- 172.21.15.5:0/2106525771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f94100a80 msgr2=0x7f1f94199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f91d9b640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f94100a80 0x7f1f94199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f91d9b640 1 -- 172.21.15.5:0/2106525771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94106a80 msgr2=0x7f1f941a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f91d9b640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94106a80 0x7f1f941a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.023+0000 7f1f91d9b640 1 -- 172.21.15.5:0/2106525771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f941a47f0 con 0x7f1f94101420 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.024+0000 7f1f91d9b640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f94101420 0x7f1f94199770 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800d770 tx=0x7f1f8800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.024+0000 7f1f837fe640 1 -- 172.21.15.5:0/2106525771 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88017070 con 0x7f1f94101420 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.024+0000 7f1f98f96640 1 -- 172.21.15.5:0/2106525771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f941a49c0 con 0x7f1f94101420 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.024+0000 7f1f837fe640 1 -- 172.21.15.5:0/2106525771 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f88005500 con 0x7f1f94101420 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.024+0000 7f1f837fe640 1 -- 172.21.15.5:0/2106525771 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88004e00 con 0x7f1f94101420 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.024+0000 7f1f98f96640 1 -- 172.21.15.5:0/2106525771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f941a4eb0 con 0x7f1f94101420 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.026+0000 7f1f837fe640 1 -- 172.21.15.5:0/2106525771 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1f88004fa0 con 0x7f1f94101420 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.026+0000 7f1f9b36f740 1 Processor -- start 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f9b36f740 1 -- start start 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f837fe640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1f7c0a2cf0 0x7f1f7c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb679090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb679a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb684370 0x55d1fb686760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1fb5df2f0 con 0x55d1fb6795d0 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d1fb586cf0 con 0x55d1fb67ab40 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d1fb5db2b0 con 0x55d1fb684370 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f837fe640 1 -- 172.21.15.5:0/2106525771 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f1f880ceff0 con 0x7f1f94101420 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f92d9d640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1f7c0a2cf0 0x7f1f7c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f91d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb679a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f91d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb679a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53946/0 (socket says 172.21.15.5:53946) 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f91d9b640 1 -- 172.21.15.5:0/1965006971 learned_addr learned my addr 172.21.15.5:0/1965006971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f9259c640 1 --2- 172.21.15.5:0/1965006971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb679090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f92d9d640 1 --2- 172.21.15.5:0/1965006971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb684370 0x55d1fb686760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f91d9b640 1 -- 172.21.15.5:0/1965006971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb684370 msgr2=0x55d1fb686760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f91d9b640 1 --2- 172.21.15.5:0/1965006971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb684370 0x55d1fb686760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f91d9b640 1 -- 172.21.15.5:0/1965006971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 msgr2=0x55d1fb679090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f91d9b640 1 --2- 172.21.15.5:0/1965006971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb679090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.027+0000 7f1f91d9b640 1 -- 172.21.15.5:0/1965006971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1fb56b3a0 con 0x55d1fb6795d0 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.028+0000 7f1f92d9d640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1f7c0a2cf0 0x7f1f7c0a51b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c012590 tx=0x7f1f8c012990 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.028+0000 7f1f91d9b640 1 --2- 172.21.15.5:0/1965006971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb679a10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f1f88097900 tx=0x7f1f8809ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6661a65992357a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.028+0000 7f1f80ff9640 1 -- 172.21.15.5:0/1965006971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f880046c0 con 0x55d1fb6795d0 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.028+0000 7f1f80ff9640 1 -- 172.21.15.5:0/1965006971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8809bba0 con 0x55d1fb6795d0 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.028+0000 7f1f80ff9640 1 -- 172.21.15.5:0/1965006971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f88013700 con 0x55d1fb6795d0 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 -- 172.21.15.5:0/1965006971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 msgr2=0x55d1fb679a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/1965006971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb679a10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f1f88097900 tx=0x7f1f8809ae00 comp rx=0 tx=0).stop 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 -- 172.21.15.5:0/1965006971 shutdown_connections 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/1965006971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb684370 0x55d1fb686760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/1965006971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb679a10 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/1965006971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb679090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 -- 172.21.15.5:0/1965006971 >> 172.21.15.5:0/1965006971 conn(0x55d1fb642310 msgr2=0x55d1fb675c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 -- 172.21.15.5:0/1965006971 shutdown_connections 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 -- 172.21.15.5:0/1965006971 wait complete. 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 -- start start 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 Processor -- start 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb6795d0 0x55d1fb683dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb67ab40 0x55d1fb681080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb681810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1fb686fb0 con 0x55d1fb684370 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d1fb59a060 con 0x55d1fb6795d0 2024-09-11T22:36:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.029+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d1fb687130 con 0x55d1fb67ab40 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f9b36f740 10 client.0 _ll_register_callbacks cb 0x55d1fb2ed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f92d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb681810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f91d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb67ab40 0x55d1fb681080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f92d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb681810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53950/0 (socket says 172.21.15.5:53950) 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f91d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb67ab40 0x55d1fb681080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:40672/0 (socket says 172.21.15.5:40672) 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f92d9d640 1 -- 172.21.15.5:0/2370524492 learned_addr learned my addr 172.21.15.5:0/2370524492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f9259c640 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb6795d0 0x55d1fb683dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f92d9d640 1 -- 172.21.15.5:0/2370524492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb67ab40 msgr2=0x55d1fb681080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f92d9d640 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb67ab40 0x55d1fb681080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f92d9d640 1 -- 172.21.15.5:0/2370524492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb6795d0 msgr2=0x55d1fb683dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f92d9d640 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb6795d0 0x55d1fb683dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f92d9d640 1 -- 172.21.15.5:0/2370524492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1fb3503c0 con 0x55d1fb684370 2024-09-11T22:36:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f92d9d640 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb681810 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c010d00 tx=0x7f1f8c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b655dbd2a4510a91 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f79ffb640 1 -- 172.21.15.5:0/2370524492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c023030 con 0x55d1fb684370 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f79ffb640 1 -- 172.21.15.5:0/2370524492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f1f8c01f420 con 0x55d1fb684370 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2370524492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f8809bba0 con 0x55d1fb684370 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f79ffb640 1 -- 172.21.15.5:0/2370524492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1f8c00de60 con 0x55d1fb684370 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.030+0000 7f1f9b36f740 10 client.19707 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.031+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2370524492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d1fb692ad0 con 0x55d1fb684370 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.031+0000 7f1f79ffb640 1 -- 172.21.15.5:0/2370524492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8809bba0 con 0x55d1fb684370 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.031+0000 7f1f79ffb640 1 -- 172.21.15.5:0/2370524492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f1f8c02f6d0 con 0x55d1fb684370 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.031+0000 7f1f9b36f740 10 client.19707 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.031+0000 7f1f9b36f740 10 client.19707 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.031+0000 7f1f9b36f740 20 client.19707 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.031+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55d1fb693ee0 0x55d1fb696330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.031+0000 7f1f9b36f740 4 client.19707 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.031+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2370524492 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1fb5ec860 con 0x55d1fb693ee0 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.031+0000 7f1f9259c640 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55d1fb693ee0 0x55d1fb696330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.032+0000 7f1f9259c640 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55d1fb693ee0 0x55d1fb696330 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7637287ae5aa660f server_cookie=df2cc88c6d955386 in_seq=0 out_seq=0 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.032+0000 7f1f79ffb640 10 client.19707 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.032+0000 7f1f79ffb640 1 -- 172.21.15.5:0/2370524492 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1fb5ec860 con 0x55d1fb693ee0 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.032+0000 7f1f79ffb640 10 client.19707 handle_command_reply: tid=0 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.032+0000 7f1f9b36f740 1 client.19707 shutdown 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.032+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2370524492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 msgr2=0x55d1fb681810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.032+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb681810 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c010d00 tx=0x7f1f8c004ab0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.033+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2370524492 shutdown_connections 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.033+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb681810 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.033+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb67ab40 0x55d1fb681080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.033+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55d1fb693ee0 0x55d1fb696330 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.033+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/2370524492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb6795d0 0x55d1fb683dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.033+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2370524492 >> 172.21.15.5:0/2370524492 conn(0x55d1fb642310 msgr2=0x55d1fb67b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.033+0000 7f1f79ffb640 0 client.19707 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.033+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2370524492 shutdown_connections 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.033+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2370524492 wait complete. 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.033+0000 7f1f9b36f740 20 client.19707 trim_cache size 0 max 16384 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.046+0000 7f1f9b36f740 1 Processor -- start 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.046+0000 7f1f9b36f740 1 -- start start 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.046+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb6795d0 0x55d1fb675e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb67c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb67c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f941ada90 con 0x55d1fb684370 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1f941af6c0 con 0x55d1fb67ab40 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1f941ae160 con 0x55d1fb6795d0 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f92d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb67c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f92d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb67c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53952/0 (socket says 172.21.15.5:53952) 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f92d9d640 1 -- 172.21.15.5:0/2608758057 learned_addr learned my addr 172.21.15.5:0/2608758057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f91d9b640 1 --2- 172.21.15.5:0/2608758057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb67c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f9259c640 1 --2- 172.21.15.5:0/2608758057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb6795d0 0x55d1fb675e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f92d9d640 1 -- 172.21.15.5:0/2608758057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb6795d0 msgr2=0x55d1fb675e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f92d9d640 1 --2- 172.21.15.5:0/2608758057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb6795d0 0x55d1fb675e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f92d9d640 1 -- 172.21.15.5:0/2608758057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 msgr2=0x55d1fb67c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f92d9d640 1 --2- 172.21.15.5:0/2608758057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb67c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f92d9d640 1 -- 172.21.15.5:0/2608758057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1fb690250 con 0x55d1fb684370 2024-09-11T22:36:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f92d9d640 1 --2- 172.21.15.5:0/2608758057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb67c660 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c02fac0 tx=0x7f1f8c00abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53bf451d15e00eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f7bfff640 1 -- 172.21.15.5:0/2608758057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c034020 con 0x55d1fb684370 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f7bfff640 1 -- 172.21.15.5:0/2608758057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8c02f8b0 con 0x55d1fb684370 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.047+0000 7f1f7bfff640 1 -- 172.21.15.5:0/2608758057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c023030 con 0x55d1fb684370 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.048+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2608758057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 msgr2=0x55d1fb67c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.048+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/2608758057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb67c660 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c02fac0 tx=0x7f1f8c00abf0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.048+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2608758057 shutdown_connections 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.048+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/2608758057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb684370 0x55d1fb67c660 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.048+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/2608758057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb67c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.048+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/2608758057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb6795d0 0x55d1fb675e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.048+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2608758057 >> 172.21.15.5:0/2608758057 conn(0x55d1fb642310 msgr2=0x55d1fb675700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.048+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2608758057 shutdown_connections 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.048+0000 7f1f9b36f740 1 -- 172.21.15.5:0/2608758057 wait complete. 2024-09-11T22:36:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9b36f740 1 -- start start 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9b36f740 1 Processor -- start 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb68c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb68cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9b36f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb684370 0x55d1fb687a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f941aeff0 con 0x55d1fb6795d0 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1f941ae860 con 0x55d1fb67ab40 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9b36f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1f941aba90 con 0x55d1fb684370 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9b36f740 10 client.0 _ll_register_callbacks cb 0x7f1f8c0109d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb68c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb68c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53954/0 (socket says 172.21.15.5:53954) 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9259c640 1 -- 172.21.15.5:0/3553531228 learned_addr learned my addr 172.21.15.5:0/3553531228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9259c640 1 -- 172.21.15.5:0/3553531228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb684370 msgr2=0x55d1fb687a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9259c640 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb684370 0x55d1fb687a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9259c640 1 -- 172.21.15.5:0/3553531228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 msgr2=0x55d1fb68cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9259c640 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb68cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.049+0000 7f1f9259c640 1 -- 172.21.15.5:0/3553531228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1fb6882c0 con 0x55d1fb6795d0 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f9259c640 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb68c6d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f1f84000c00 tx=0x7f1f84004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94ccae2dc08a079 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f79ffb640 1 -- 172.21.15.5:0/3553531228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84035040 con 0x55d1fb6795d0 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f79ffb640 1 -- 172.21.15.5:0/3553531228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f1f84005040 con 0x55d1fb6795d0 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f9b36f740 1 -- 172.21.15.5:0/3553531228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1fb5e6d70 con 0x55d1fb6795d0 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f79ffb640 1 -- 172.21.15.5:0/3553531228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1f84005240 con 0x55d1fb6795d0 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f9b36f740 10 client.19713 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f9b36f740 1 -- 172.21.15.5:0/3553531228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55d1fb6fe840 con 0x55d1fb6795d0 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f79ffb640 1 -- 172.21.15.5:0/3553531228 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d1fb5e6d70 con 0x55d1fb6795d0 2024-09-11T22:36:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f79ffb640 1 -- 172.21.15.5:0/3553531228 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f1f84003680 con 0x55d1fb6795d0 2024-09-11T22:36:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f9b36f740 10 client.19713 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f9b36f740 10 client.19713 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:36:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f9b36f740 20 client.19713 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.050+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55d1fb6fef90 0x55d1fb701430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.051+0000 7f1f9b36f740 4 client.19713 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:36:01.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.051+0000 7f1f9b36f740 1 -- 172.21.15.5:0/3553531228 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d1fb563170 con 0x55d1fb6fef90 2024-09-11T22:36:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.051+0000 7f1f91d9b640 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55d1fb6fef90 0x55d1fb701430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.053+0000 7f1f91d9b640 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55d1fb6fef90 0x55d1fb701430 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=54017b9e8ab13415 server_cookie=9c5c1b245485f1fb in_seq=0 out_seq=0 2024-09-11T22:36:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.053+0000 7f1f79ffb640 10 client.19713 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:36:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.053+0000 7f1f79ffb640 1 -- 172.21.15.5:0/3553531228 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d1fb563170 con 0x55d1fb6fef90 2024-09-11T22:36:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.053+0000 7f1f79ffb640 10 client.19713 handle_command_reply: tid=0 2024-09-11T22:36:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.053+0000 7f1f9b36f740 1 client.19713 shutdown 2024-09-11T22:36:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.053+0000 7f1f9b36f740 1 -- 172.21.15.5:0/3553531228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 msgr2=0x55d1fb68c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.053+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb68c6d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f1f84000c00 tx=0x7f1f84004130 comp rx=0 tx=0).stop 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.053+0000 7f1f9b36f740 1 -- 172.21.15.5:0/3553531228 shutdown_connections 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.053+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d1fb684370 0x55d1fb687a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.053+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55d1fb6fef90 0x55d1fb701430 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1fb67ab40 0x55d1fb68cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f79ffb640 0 client.19713 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f9b36f740 1 --2- 172.21.15.5:0/3553531228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1fb6795d0 0x55d1fb68c6d0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f9b36f740 1 -- 172.21.15.5:0/3553531228 >> 172.21.15.5:0/3553531228 conn(0x55d1fb642310 msgr2=0x55d1fb6744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f9b36f740 1 -- 172.21.15.5:0/3553531228 shutdown_connections 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f9b36f740 1 -- 172.21.15.5:0/3553531228 wait complete. 2024-09-11T22:36:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f9b36f740 20 client.19713 trim_cache size 0 max 16384 2024-09-11T22:36:01.069 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:36:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 -- 172.21.15.5:0/2106525771 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1f7c0a2cf0 msgr2=0x7f1f7c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1f7c0a2cf0 0x7f1f7c0a51b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c012590 tx=0x7f1f8c012990 comp rx=0 tx=0).stop 2024-09-11T22:36:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 -- 172.21.15.5:0/2106525771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f94101420 msgr2=0x7f1f94199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f94101420 0x7f1f94199770 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800d770 tx=0x7f1f8800dc30 comp rx=0 tx=0).stop 2024-09-11T22:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 -- 172.21.15.5:0/2106525771 shutdown_connections 2024-09-11T22:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1f7c0a2cf0 0x7f1f7c0a51b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1f94106a80 0x7f1f941a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f94101420 0x7f1f94199770 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 --2- 172.21.15.5:0/2106525771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1f94100a80 0x7f1f94199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 -- 172.21.15.5:0/2106525771 >> 172.21.15.5:0/2106525771 conn(0x7f1f940fc7a0 msgr2=0x7f1f940fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 -- 172.21.15.5:0/2106525771 shutdown_connections 2024-09-11T22:36:01.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:01.054+0000 7f1f98f96640 1 -- 172.21.15.5:0/2106525771 wait complete. 2024-09-11T22:36:01.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 29.56 MB/sec warmup 19 sec latency 5694.965 ms 2024-09-11T22:36:01.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:01 smithi005 ceph-mon[30943]: pgmap v439: 97 pgs: 97 active+clean; 211 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.8 MiB/s wr, 321 op/s 2024-09-11T22:36:01.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:01 smithi159 ceph-mon[35534]: pgmap v439: 97 pgs: 97 active+clean; 211 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.8 MiB/s wr, 321 op/s 2024-09-11T22:36:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:01 smithi172 ceph-mon[35461]: pgmap v439: 97 pgs: 97 active+clean; 211 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.8 MiB/s wr, 321 op/s 2024-09-11T22:36:01.903 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:36:02.161 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:02.630 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 28.09 MB/sec warmup 20 sec latency 6695.007 ms 2024-09-11T22:36:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.713+0000 7f58dd26c640 1 -- 172.21.15.5:0/3836648394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d8102570 msgr2=0x7f58d81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.713+0000 7f58dd26c640 1 --2- 172.21.15.5:0/3836648394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d8102570 0x7f58d81029b0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009900 tx=0x7f58cc02a480 comp rx=0 tx=0).stop 2024-09-11T22:36:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.713+0000 7f58dd26c640 1 -- 172.21.15.5:0/3836648394 shutdown_connections 2024-09-11T22:36:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.713+0000 7f58dd26c640 1 --2- 172.21.15.5:0/3836648394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d8102ef0 0x7f58d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.713+0000 7f58dd26c640 1 --2- 172.21.15.5:0/3836648394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d8102570 0x7f58d81029b0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.713+0000 7f58dd26c640 1 --2- 172.21.15.5:0/3836648394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8100e20 0x7f58d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.713+0000 7f58dd26c640 1 -- 172.21.15.5:0/3836648394 >> 172.21.15.5:0/3836648394 conn(0x7f58d80fc7d0 msgr2=0x7f58d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.713+0000 7f58dd26c640 1 -- 172.21.15.5:0/3836648394 shutdown_connections 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.713+0000 7f58dd26c640 1 -- 172.21.15.5:0/3836648394 wait complete. 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.713+0000 7f58dd26c640 1 Processor -- start 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58dd26c640 1 -- start start 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58dd26c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d8100e20 0x7f58d8198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58dd26c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8102570 0x7f58d8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58dd26c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d8102ef0 0x7f58d819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58dd26c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58d8110f50 con 0x7f58d8100e20 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58dd26c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58d8110dd0 con 0x7f58d8102570 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58dd26c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81110d0 con 0x7f58d8102ef0 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d8100e20 0x7f58d8198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d8100e20 0x7f58d8198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38372/0 (socket says 172.21.15.5:38372) 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d6ffd640 1 -- 172.21.15.5:0/4026543646 learned_addr learned my addr 172.21.15.5:0/4026543646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d77fe640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d8102ef0 0x7f58d819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d67fc640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8102570 0x7f58d8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d6ffd640 1 -- 172.21.15.5:0/4026543646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d8102ef0 msgr2=0x7f58d819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d8102ef0 0x7f58d819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d6ffd640 1 -- 172.21.15.5:0/4026543646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8102570 msgr2=0x7f58d8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8102570 0x7f58d8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d6ffd640 1 -- 172.21.15.5:0/4026543646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d819d500 con 0x7f58d8100e20 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.714+0000 7f58d67fc640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8102570 0x7f58d8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.715+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d8100e20 0x7f58d8198550 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f58c800e9f0 tx=0x7f58c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.715+0000 7f58bffff640 1 -- 172.21.15.5:0/4026543646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c800cd80 con 0x7f58d8100e20 2024-09-11T22:36:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.715+0000 7f58bffff640 1 -- 172.21.15.5:0/4026543646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c8010910 con 0x7f58d8100e20 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.715+0000 7f58dd26c640 1 -- 172.21.15.5:0/4026543646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d819d7f0 con 0x7f58d8100e20 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.715+0000 7f58bffff640 1 -- 172.21.15.5:0/4026543646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8018990 con 0x7f58d8100e20 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.715+0000 7f58dd26c640 1 -- 172.21.15.5:0/4026543646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d81a50d0 con 0x7f58d8100e20 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.717+0000 7f58bffff640 1 -- 172.21.15.5:0/4026543646 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f58c8010ac0 con 0x7f58d8100e20 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.717+0000 7f58df645740 1 Processor -- start 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.717+0000 7f58df645740 1 -- start start 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58df645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fef50 0x562abb5fd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58df645740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fd9e0 0x562abb5fde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58df645740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb60ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58df645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562abb55f2f0 con 0x562abb5fef50 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58df645740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562abb506cf0 con 0x562abb608780 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58df645740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562abb55b2b0 con 0x562abb5fd9e0 2024-09-11T22:36:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fef50 0x562abb5fd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58bffff640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58c00a2cf0 0x7f58c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fef50 0x562abb5fd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38380/0 (socket says 172.21.15.5:38380) 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d77fe640 1 -- 172.21.15.5:0/2999571996 learned_addr learned my addr 172.21.15.5:0/2999571996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d77fe640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58c00a2cf0 0x7f58c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/2999571996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fd9e0 0x562abb5fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d67fc640 1 --2- 172.21.15.5:0/2999571996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb60ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58bffff640 1 -- 172.21.15.5:0/4026543646 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f58c80cf880 con 0x7f58d8100e20 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d77fe640 1 -- 172.21.15.5:0/2999571996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fd9e0 msgr2=0x562abb5fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d77fe640 1 --2- 172.21.15.5:0/2999571996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fd9e0 0x562abb5fde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d77fe640 1 -- 172.21.15.5:0/2999571996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 msgr2=0x562abb60ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d77fe640 1 --2- 172.21.15.5:0/2999571996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb60ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d77fe640 1 -- 172.21.15.5:0/2999571996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abb4eb3a0 con 0x562abb5fef50 2024-09-11T22:36:02.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.718+0000 7f58d77fe640 1 --2- 172.21.15.5:0/2999571996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fef50 0x562abb5fd4a0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f58d0014a60 tx=0x7f58d0014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a192b0144483d4b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58bd7fa640 1 -- 172.21.15.5:0/2999571996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d001ec50 con 0x562abb5fef50 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58bd7fa640 1 -- 172.21.15.5:0/2999571996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58d0023070 con 0x562abb5fef50 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58bd7fa640 1 -- 172.21.15.5:0/2999571996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d001dd90 con 0x562abb5fef50 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58df645740 1 -- 172.21.15.5:0/2999571996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fef50 msgr2=0x562abb5fd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58d77fe640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58c00a2cf0 0x7f58c00a51b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f58d0022000 tx=0x7f58d001e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58df645740 1 --2- 172.21.15.5:0/2999571996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fef50 0x562abb5fd4a0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f58d0014a60 tx=0x7f58d0014f20 comp rx=0 tx=0).stop 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58df645740 1 -- 172.21.15.5:0/2999571996 shutdown_connections 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58df645740 1 --2- 172.21.15.5:0/2999571996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb60ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58df645740 1 --2- 172.21.15.5:0/2999571996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fd9e0 0x562abb5fde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58df645740 1 --2- 172.21.15.5:0/2999571996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fef50 0x562abb5fd4a0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58df645740 1 -- 172.21.15.5:0/2999571996 >> 172.21.15.5:0/2999571996 conn(0x562abb5c2310 msgr2=0x562abb5fa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:02.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58df645740 1 -- 172.21.15.5:0/2999571996 shutdown_connections 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.719+0000 7f58df645740 1 -- 172.21.15.5:0/2999571996 wait complete. 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58df645740 1 -- start start 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58df645740 1 Processor -- start 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58df645740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb5fd9e0 0x562abb6081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58df645740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb605450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58df645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb608780 0x562abb605a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58df645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562abb60b480 con 0x562abb608780 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58df645740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562abb51a060 con 0x562abb5fd9e0 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58df645740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562abb60b600 con 0x562abb5fef50 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58df645740 10 client.0 _ll_register_callbacks cb 0x562abb26d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb608780 0x562abb605a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb608780 0x562abb605a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38384/0 (socket says 172.21.15.5:38384) 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58d77fe640 1 -- 172.21.15.5:0/2567406177 learned_addr learned my addr 172.21.15.5:0/2567406177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb5fd9e0 0x562abb6081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.720+0000 7f58d67fc640 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb605450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58d77fe640 1 -- 172.21.15.5:0/2567406177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 msgr2=0x562abb605450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58d77fe640 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb605450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58d77fe640 1 -- 172.21.15.5:0/2567406177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb5fd9e0 msgr2=0x562abb6081a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58d77fe640 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb5fd9e0 0x562abb6081a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58d77fe640 1 -- 172.21.15.5:0/2567406177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abb2d03c0 con 0x562abb608780 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb5fd9e0 0x562abb6081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58d77fe640 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb608780 0x562abb605a60 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f58d001da40 tx=0x7f58d00128d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487ec04dd10c8fdc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58baff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58baff5640 1 -- 172.21.15.5:0/2567406177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0036060 con 0x562abb608780 2024-09-11T22:36:02.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58baff5640 1 -- 172.21.15.5:0/2567406177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f58d001db10 con 0x562abb608780 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58df645740 1 -- 172.21.15.5:0/2567406177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d0023070 con 0x562abb608780 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58baff5640 1 -- 172.21.15.5:0/2567406177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f58d001ed10 con 0x562abb608780 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58df645740 10 client.19725 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.721+0000 7f58df645740 1 -- 172.21.15.5:0/2567406177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562abb616f50 con 0x562abb608780 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.722+0000 7f58baff5640 1 -- 172.21.15.5:0/2567406177 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58d0023070 con 0x562abb608780 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.722+0000 7f58baff5640 1 -- 172.21.15.5:0/2567406177 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f58d000f040 con 0x562abb608780 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.722+0000 7f58df645740 10 client.19725 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.722+0000 7f58df645740 10 client.19725 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.722+0000 7f58df645740 20 client.19725 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.722+0000 7f58df645740 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562abb618360 0x562abb61a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.722+0000 7f58df645740 4 client.19725 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.722+0000 7f58df645740 1 -- 172.21.15.5:0/2567406177 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562abb56c860 con 0x562abb618360 2024-09-11T22:36:02.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.722+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562abb618360 0x562abb61a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.723+0000 7f58baff5640 10 client.19725 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.723+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562abb618360 0x562abb61a750 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f4d331cad7a998 server_cookie=65a880751c72b157 in_seq=0 out_seq=0 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58baff5640 1 -- 172.21.15.5:0/2567406177 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562abb56c860 con 0x562abb618360 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58baff5640 10 client.19725 handle_command_reply: tid=0 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 client.19725 shutdown 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 -- 172.21.15.5:0/2567406177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb608780 msgr2=0x562abb605a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb608780 0x562abb605a60 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f58d001da40 tx=0x7f58d00128d0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 -- 172.21.15.5:0/2567406177 shutdown_connections 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562abb618360 0x562abb61a750 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb608780 0x562abb605a60 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb605450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 --2- 172.21.15.5:0/2567406177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb5fd9e0 0x562abb6081a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 -- 172.21.15.5:0/2567406177 >> 172.21.15.5:0/2567406177 conn(0x562abb5c2310 msgr2=0x562abb5ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:02.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58baff5640 0 client.19725 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:36:02.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 -- 172.21.15.5:0/2567406177 shutdown_connections 2024-09-11T22:36:02.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 1 -- 172.21.15.5:0/2567406177 wait complete. 2024-09-11T22:36:02.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.724+0000 7f58df645740 20 client.19725 trim_cache size 0 max 16384 2024-09-11T22:36:02.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.737+0000 7f58df645740 1 Processor -- start 2024-09-11T22:36:02.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.737+0000 7f58df645740 1 -- start start 2024-09-11T22:36:02.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.737+0000 7f58df645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb614720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.737+0000 7f58df645740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb600530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.737+0000 7f58df645740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb600a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.737+0000 7f58df645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81ab870 con 0x562abb5fd9e0 2024-09-11T22:36:02.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58df645740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81ad4a0 con 0x562abb608780 2024-09-11T22:36:02.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58df645740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81abf40 con 0x562abb5fef50 2024-09-11T22:36:02.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb614720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb614720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38396/0 (socket says 172.21.15.5:38396) 2024-09-11T22:36:02.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d6ffd640 1 -- 172.21.15.5:0/2230430916 learned_addr learned my addr 172.21.15.5:0/2230430916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:02.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d67fc640 1 --2- 172.21.15.5:0/2230430916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb600530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d6ffd640 1 -- 172.21.15.5:0/2230430916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 msgr2=0x562abb600530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:02.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/2230430916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb600530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d77fe640 1 --2- 172.21.15.5:0/2230430916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb600a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d6ffd640 1 -- 172.21.15.5:0/2230430916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 msgr2=0x562abb600a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:02.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/2230430916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb600a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d6ffd640 1 -- 172.21.15.5:0/2230430916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562abb6039a0 con 0x562abb5fd9e0 2024-09-11T22:36:02.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/2230430916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb614720 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f58c8098700 tx=0x7f58c8013700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17cf80a010cd6853 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:02.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58bcff9640 1 -- 172.21.15.5:0/2230430916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80a49a0 con 0x562abb5fd9e0 2024-09-11T22:36:02.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58bcff9640 1 -- 172.21.15.5:0/2230430916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c809a030 con 0x562abb5fd9e0 2024-09-11T22:36:02.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.738+0000 7f58bcff9640 1 -- 172.21.15.5:0/2230430916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8018b30 con 0x562abb5fd9e0 2024-09-11T22:36:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 -- 172.21.15.5:0/2230430916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 msgr2=0x562abb614720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:02.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 --2- 172.21.15.5:0/2230430916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb614720 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f58c8098700 tx=0x7f58c8013700 comp rx=0 tx=0).stop 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 -- 172.21.15.5:0/2230430916 shutdown_connections 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 --2- 172.21.15.5:0/2230430916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb600a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 --2- 172.21.15.5:0/2230430916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb600530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 --2- 172.21.15.5:0/2230430916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb614720 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 -- 172.21.15.5:0/2230430916 >> 172.21.15.5:0/2230430916 conn(0x562abb5c2310 msgr2=0x562abb6074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 -- 172.21.15.5:0/2230430916 shutdown_connections 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 -- 172.21.15.5:0/2230430916 wait complete. 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 -- start start 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.739+0000 7f58df645740 1 Processor -- start 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58df645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb610c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58df645740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb611190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58df645740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb602030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58df645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81acdd0 con 0x562abb5fd9e0 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58df645740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81ac640 con 0x562abb608780 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58df645740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f58d81a9870 con 0x562abb5fef50 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58df645740 10 client.0 _ll_register_callbacks cb 0x7f58d0014860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb610c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb610c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38412/0 (socket says 172.21.15.5:38412) 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d6ffd640 1 -- 172.21.15.5:0/777152590 learned_addr learned my addr 172.21.15.5:0/777152590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d67fc640 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb611190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d77fe640 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb602030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d6ffd640 1 -- 172.21.15.5:0/777152590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 msgr2=0x562abb611190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb611190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d6ffd640 1 -- 172.21.15.5:0/777152590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 msgr2=0x562abb602030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:02.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb602030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d6ffd640 1 -- 172.21.15.5:0/777152590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562abb602830 con 0x562abb5fd9e0 2024-09-11T22:36:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58d6ffd640 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb610c50 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f58c80a4470 tx=0x7f58c8005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffff9e7dce022326 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58baff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58baff5640 1 -- 172.21.15.5:0/777152590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80b8020 con 0x562abb5fd9e0 2024-09-11T22:36:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58baff5640 1 -- 172.21.15.5:0/777152590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8629+0+0 (secure 0 0 0) 0x7f58c8018b30 con 0x562abb5fd9e0 2024-09-11T22:36:02.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.740+0000 7f58df645740 1 -- 172.21.15.5:0/777152590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c809a030 con 0x562abb5fd9e0 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58baff5640 1 -- 172.21.15.5:0/777152590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f58c809b180 con 0x562abb5fd9e0 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58df645740 10 client.19731 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58df645740 1 -- 172.21.15.5:0/777152590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x562abb6829f0 con 0x562abb5fd9e0 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58baff5640 1 -- 172.21.15.5:0/777152590 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c809a030 con 0x562abb5fd9e0 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58baff5640 1 -- 172.21.15.5:0/777152590 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f58c80b2070 con 0x562abb5fd9e0 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58df645740 10 client.19731 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58df645740 10 client.19731 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58df645740 20 client.19731 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58df645740 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562abb683060 0x562abb685480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58df645740 4 client.19731 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58df645740 1 -- 172.21.15.5:0/777152590 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562abb566d70 con 0x562abb683060 2024-09-11T22:36:02.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.741+0000 7f58d67fc640 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562abb683060 0x562abb685480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58d67fc640 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562abb683060 0x562abb685480 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa6fe03924181025 server_cookie=1461efd38208e432 in_seq=0 out_seq=0 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58baff5640 10 client.19731 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58baff5640 1 -- 172.21.15.5:0/777152590 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562abb566d70 con 0x562abb683060 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58baff5640 10 client.19731 handle_command_reply: tid=0 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 client.19731 shutdown 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 -- 172.21.15.5:0/777152590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 msgr2=0x562abb610c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb610c50 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f58c80a4470 tx=0x7f58c8005bf0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 -- 172.21.15.5:0/777152590 shutdown_connections 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x562abb683060 0x562abb685480 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562abb608780 0x562abb602030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58baff5640 0 client.19731 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:36:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562abb5fef50 0x562abb611190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 --2- 172.21.15.5:0/777152590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562abb5fd9e0 0x562abb610c50 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 -- 172.21.15.5:0/777152590 >> 172.21.15.5:0/777152590 conn(0x562abb5c2310 msgr2=0x562abb5f44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 -- 172.21.15.5:0/777152590 shutdown_connections 2024-09-11T22:36:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 1 -- 172.21.15.5:0/777152590 wait complete. 2024-09-11T22:36:02.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.742+0000 7f58df645740 20 client.19731 trim_cache size 0 max 16384 2024-09-11T22:36:02.750 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:36:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 -- 172.21.15.5:0/4026543646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58c00a2cf0 msgr2=0x7f58c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58c00a2cf0 0x7f58c00a51b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f58d0022000 tx=0x7f58d001e000 comp rx=0 tx=0).stop 2024-09-11T22:36:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 -- 172.21.15.5:0/4026543646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d8100e20 msgr2=0x7f58d8198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d8100e20 0x7f58d8198550 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f58c800e9f0 tx=0x7f58c800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 -- 172.21.15.5:0/4026543646 shutdown_connections 2024-09-11T22:36:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d8102ef0 0x7f58d819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58c00a2cf0 0x7f58c00a51b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58d8102570 0x7f58d8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 --2- 172.21.15.5:0/4026543646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58d8100e20 0x7f58d8198550 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:02.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 -- 172.21.15.5:0/4026543646 >> 172.21.15.5:0/4026543646 conn(0x7f58d80fc7d0 msgr2=0x7f58d806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:02.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 -- 172.21.15.5:0/4026543646 shutdown_connections 2024-09-11T22:36:02.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:02.743+0000 7f58dd26c640 1 -- 172.21.15.5:0/4026543646 wait complete. 2024-09-11T22:36:03.631 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 26.75 MB/sec warmup 21 sec latency 7695.042 ms 2024-09-11T22:36:03.719 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:03 smithi005 ceph-mon[30943]: pgmap v440: 97 pgs: 97 active+clean; 207 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 593 op/s 2024-09-11T22:36:03.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:36:03.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:03 smithi159 ceph-mon[35534]: pgmap v440: 97 pgs: 97 active+clean; 207 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 593 op/s 2024-09-11T22:36:03.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:03 smithi172 ceph-mon[35461]: pgmap v440: 97 pgs: 97 active+clean; 207 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 593 op/s 2024-09-11T22:36:04.025 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:04.631 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 25.53 MB/sec warmup 22 sec latency 8695.083 ms 2024-09-11T22:36:04.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.642+0000 7f4100796640 1 -- 172.21.15.5:0/3208844681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40f81008c0 msgr2=0x7f40f8100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:04.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.642+0000 7f4100796640 1 --2- 172.21.15.5:0/3208844681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40f81008c0 0x7f40f8100ca0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f40f4009860 tx=0x7f40f402a340 comp rx=0 tx=0).stop 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.643+0000 7f4100796640 1 -- 172.21.15.5:0/3208844681 shutdown_connections 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.643+0000 7f4100796640 1 --2- 172.21.15.5:0/3208844681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40f8105ec0 0x7f40f8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.643+0000 7f4100796640 1 --2- 172.21.15.5:0/3208844681 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40f81011e0 0x7f40f8105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.643+0000 7f4100796640 1 --2- 172.21.15.5:0/3208844681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40f81008c0 0x7f40f8100ca0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.643+0000 7f4100796640 1 -- 172.21.15.5:0/3208844681 >> 172.21.15.5:0/3208844681 conn(0x7f40f80fc7d0 msgr2=0x7f40f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.644+0000 7f4100796640 1 -- 172.21.15.5:0/3208844681 shutdown_connections 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.645+0000 7f4100796640 1 -- 172.21.15.5:0/3208844681 wait complete. 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.645+0000 7f4100796640 1 Processor -- start 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.645+0000 7f4100796640 1 -- start start 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.646+0000 7f4100796640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40f81008c0 0x7f40f81070b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.646+0000 7f4100796640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40f81011e0 0x7f40f81075f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.646+0000 7f4100796640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40f8105ec0 0x7f40f810b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.646+0000 7f4100796640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40f806c660 con 0x7f40f8105ec0 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.646+0000 7f4100796640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f40f806c4e0 con 0x7f40f81008c0 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.646+0000 7f4100796640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40f806c7e0 con 0x7f40f81011e0 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.647+0000 7f40fe532640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40f81008c0 0x7f40f81070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.647+0000 7f40fdd31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40f81011e0 0x7f40f81075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.647+0000 7f40fe532640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40f81008c0 0x7f40f81070b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50222/0 (socket says 172.21.15.5:50222) 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.647+0000 7f40fe532640 1 -- 172.21.15.5:0/3756303391 learned_addr learned my addr 172.21.15.5:0/3756303391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.647+0000 7f40fed33640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40f8105ec0 0x7f40f810b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.648+0000 7f40fe532640 1 -- 172.21.15.5:0/3756303391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40f81011e0 msgr2=0x7f40f81075f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.648+0000 7f40fe532640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40f81011e0 0x7f40f81075f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.648+0000 7f40fe532640 1 -- 172.21.15.5:0/3756303391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40f8105ec0 msgr2=0x7f40f810b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:04.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.648+0000 7f40fe532640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40f8105ec0 0x7f40f810b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.648+0000 7f40fe532640 1 -- 172.21.15.5:0/3756303391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f810b7f0 con 0x7f40f81008c0 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.649+0000 7f40fe532640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40f81008c0 0x7f40f81070b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f40f4005d00 tx=0x7f40f4033d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.649+0000 7f40ef7fe640 1 -- 172.21.15.5:0/3756303391 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f403f020 con 0x7f40f81008c0 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.650+0000 7f40ef7fe640 1 -- 172.21.15.5:0/3756303391 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f4030780 con 0x7f40f81008c0 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.650+0000 7f40ef7fe640 1 -- 172.21.15.5:0/3756303391 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f4037040 con 0x7f40f81008c0 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.650+0000 7f4100796640 1 -- 172.21.15.5:0/3756303391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f810ba80 con 0x7f40f81008c0 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.650+0000 7f4100796640 1 -- 172.21.15.5:0/3756303391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f810be50 con 0x7f40f81008c0 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.654+0000 7f4102b6f740 1 Processor -- start 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.656+0000 7f4102b6f740 1 -- start start 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.656+0000 7f40ef7fe640 1 -- 172.21.15.5:0/3756303391 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f40f4042020 con 0x7f40f81008c0 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.656+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 0x55e05b121090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.656+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b121a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.656+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b12e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.656+0000 7f4102b6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e05b0872f0 con 0x55e05b1215d0 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.656+0000 7f4102b6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e05b02ecf0 con 0x55e05b12c370 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.656+0000 7f4102b6f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e05b0832b0 con 0x55e05b122b40 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.658+0000 7f40ef7fe640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f40e40a2f00 0x7f40e40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.658+0000 7f40fe532640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b121a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.658+0000 7f40fe532640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b121a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38438/0 (socket says 172.21.15.5:38438) 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.658+0000 7f40fe532640 1 -- 172.21.15.5:0/1611635157 learned_addr learned my addr 172.21.15.5:0/1611635157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.659+0000 7f40fed33640 1 --2- 172.21.15.5:0/1611635157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 0x55e05b121090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.659+0000 7f40fed33640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f40e40a2f00 0x7f40e40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.659+0000 7f40ef7fe640 1 -- 172.21.15.5:0/3756303391 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f40f40ef1f0 con 0x7f40f81008c0 2024-09-11T22:36:04.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.660+0000 7f40fed33640 1 -- 172.21.15.5:0/1611635157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 msgr2=0x55e05b12e760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.660+0000 7f40fed33640 1 --2- 172.21.15.5:0/1611635157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b12e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.660+0000 7f40fed33640 1 -- 172.21.15.5:0/1611635157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 msgr2=0x55e05b121a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.660+0000 7f40fed33640 1 --2- 172.21.15.5:0/1611635157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b121a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.660+0000 7f40fed33640 1 -- 172.21.15.5:0/1611635157 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05b0133a0 con 0x55e05b122b40 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.660+0000 7f40fe532640 1 --2- 172.21.15.5:0/1611635157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b121a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.660+0000 7f40fed33640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f40e40a2f00 0x7f40e40a53c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005fd0 tx=0x7f40f0009a60 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.660+0000 7f40fed33640 1 --2- 172.21.15.5:0/1611635157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 0x55e05b121090 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009d60 tx=0x7f40f001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba633f42aa4a462 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.661+0000 7f40ecff9640 1 -- 172.21.15.5:0/1611635157 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0024070 con 0x55e05b122b40 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.661+0000 7f40ecff9640 1 -- 172.21.15.5:0/1611635157 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f0003e50 con 0x55e05b122b40 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.662+0000 7f40ecff9640 1 -- 172.21.15.5:0/1611635157 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0004290 con 0x55e05b122b40 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.662+0000 7f4102b6f740 1 -- 172.21.15.5:0/1611635157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 msgr2=0x55e05b121090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.662+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1611635157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 0x55e05b121090 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009d60 tx=0x7f40f001c000 comp rx=0 tx=0).stop 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.662+0000 7f4102b6f740 1 -- 172.21.15.5:0/1611635157 shutdown_connections 2024-09-11T22:36:04.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.663+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1611635157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b12e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.663+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1611635157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b121a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.663+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1611635157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 0x55e05b121090 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.663+0000 7f4102b6f740 1 -- 172.21.15.5:0/1611635157 >> 172.21.15.5:0/1611635157 conn(0x55e05b0ea310 msgr2=0x55e05b11dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.663+0000 7f4102b6f740 1 -- 172.21.15.5:0/1611635157 shutdown_connections 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.663+0000 7f4102b6f740 1 -- 172.21.15.5:0/1611635157 wait complete. 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.664+0000 7f4102b6f740 1 -- start start 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.664+0000 7f4102b6f740 1 Processor -- start 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.665+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b12bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.665+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b122b40 0x55e05b128f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.665+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b12c370 0x55e05b129460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.665+0000 7f4102b6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e05b12f060 con 0x55e05b1215d0 2024-09-11T22:36:04.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.665+0000 7f4102b6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e05b042060 con 0x55e05b122b40 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.665+0000 7f4102b6f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e05b12f1e0 con 0x55e05b12c370 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.665+0000 7f4102b6f740 10 client.0 _ll_register_callbacks cb 0x55e05ad958c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.665+0000 7f40fe532640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b12bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.665+0000 7f40fe532640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b12bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38454/0 (socket says 172.21.15.5:38454) 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.665+0000 7f40fe532640 1 -- 172.21.15.5:0/2669233650 learned_addr learned my addr 172.21.15.5:0/2669233650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.666+0000 7f40fe532640 1 -- 172.21.15.5:0/2669233650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b12c370 msgr2=0x55e05b129460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.666+0000 7f40fe532640 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b12c370 0x55e05b129460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.666+0000 7f40fe532640 1 -- 172.21.15.5:0/2669233650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b122b40 msgr2=0x55e05b128f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.666+0000 7f40fe532640 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b122b40 0x55e05b128f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.666+0000 7f40fe532640 1 -- 172.21.15.5:0/2669233650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05adf83c0 con 0x55e05b1215d0 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.666+0000 7f40fe532640 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b12bc70 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f40f40b80f0 tx=0x7f40f40036b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215a6d2869ffab12 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.667+0000 7f40e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:04.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.667+0000 7f40e1ffb640 1 -- 172.21.15.5:0/2669233650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40bb080 con 0x55e05b1215d0 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.667+0000 7f40e1ffb640 1 -- 172.21.15.5:0/2669233650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f40f4003cc0 con 0x55e05b1215d0 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.668+0000 7f4102b6f740 1 -- 172.21.15.5:0/2669233650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f0003e50 con 0x55e05b1215d0 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.668+0000 7f40e1ffb640 1 -- 172.21.15.5:0/2669233650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f40f40bbe20 con 0x55e05b1215d0 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.668+0000 7f4102b6f740 10 client.19743 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.668+0000 7f4102b6f740 1 -- 172.21.15.5:0/2669233650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e05b13aa60 con 0x55e05b1215d0 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.669+0000 7f40e1ffb640 1 -- 172.21.15.5:0/2669233650 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f0003e50 con 0x55e05b1215d0 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.669+0000 7f40e1ffb640 1 -- 172.21.15.5:0/2669233650 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f40f403a030 con 0x55e05b1215d0 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.669+0000 7f4102b6f740 10 client.19743 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.669+0000 7f4102b6f740 10 client.19743 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.669+0000 7f4102b6f740 20 client.19743 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.669+0000 7f4102b6f740 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55e05b13bf50 0x55e05b13e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.669+0000 7f4102b6f740 4 client.19743 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:04.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.669+0000 7f4102b6f740 1 -- 172.21.15.5:0/2669233650 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e05b094860 con 0x55e05b13bf50 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.669+0000 7f40fdd31640 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55e05b13bf50 0x55e05b13e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.672+0000 7f40e1ffb640 10 client.19743 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.672+0000 7f40fdd31640 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55e05b13bf50 0x55e05b13e410 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15170c50e40f34 server_cookie=5dffbb3e88bec43b in_seq=0 out_seq=0 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.674+0000 7f40e1ffb640 1 -- 172.21.15.5:0/2669233650 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e05b094860 con 0x55e05b13bf50 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.674+0000 7f40e1ffb640 10 client.19743 handle_command_reply: tid=0 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.674+0000 7f4102b6f740 1 client.19743 shutdown 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.678+0000 7f4102b6f740 1 -- 172.21.15.5:0/2669233650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 msgr2=0x55e05b12bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.678+0000 7f4102b6f740 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b12bc70 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f40f40b80f0 tx=0x7f40f40036b0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.679+0000 7f4102b6f740 1 -- 172.21.15.5:0/2669233650 shutdown_connections 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.679+0000 7f4102b6f740 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55e05b13bf50 0x55e05b13e410 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.679+0000 7f4102b6f740 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b12c370 0x55e05b129460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.679+0000 7f4102b6f740 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b122b40 0x55e05b128f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.679+0000 7f4102b6f740 1 --2- 172.21.15.5:0/2669233650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b12bc70 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.679+0000 7f4102b6f740 1 -- 172.21.15.5:0/2669233650 >> 172.21.15.5:0/2669233650 conn(0x55e05b0ea310 msgr2=0x55e05b123930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.679+0000 7f40e1ffb640 0 client.19743 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.680+0000 7f4102b6f740 1 -- 172.21.15.5:0/2669233650 shutdown_connections 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.680+0000 7f4102b6f740 1 -- 172.21.15.5:0/2669233650 wait complete. 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.681+0000 7f4102b6f740 20 client.19743 trim_cache size 0 max 16384 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.707+0000 7f4102b6f740 1 Processor -- start 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.707+0000 7f4102b6f740 1 -- start start 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.708+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b11de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.708+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 0x55e05b1383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.708+0000 7f40fe532640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b11de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.708+0000 7f40fe532640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b11de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38460/0 (socket says 172.21.15.5:38460) 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.708+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b143d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.709+0000 7f40fdd31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 0x55e05b1383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.709+0000 7f40fdd31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 0x55e05b1383e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59156/0 (socket says 172.21.15.5:59156) 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.709+0000 7f40fe532640 1 -- 172.21.15.5:0/1692205785 learned_addr learned my addr 172.21.15.5:0/1692205785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.709+0000 7f4102b6f740 1 -- 172.21.15.5:0/1692205785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc00eb00 con 0x55e05b1215d0 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.709+0000 7f4102b6f740 1 -- 172.21.15.5:0/1692205785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc010730 con 0x55e05b12c370 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.709+0000 7f4102b6f740 1 -- 172.21.15.5:0/1692205785 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc00f1d0 con 0x55e05b122b40 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.710+0000 7f40fed33640 1 --2- 172.21.15.5:0/1692205785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.710+0000 7f40fe532640 1 -- 172.21.15.5:0/1692205785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 msgr2=0x55e05b1383e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.710+0000 7f40fe532640 1 --2- 172.21.15.5:0/1692205785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 0x55e05b1383e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.710+0000 7f40fe532640 1 -- 172.21.15.5:0/1692205785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 msgr2=0x55e05b143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.710+0000 7f40fe532640 1 --2- 172.21.15.5:0/1692205785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b143d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.710+0000 7f40fe532640 1 -- 172.21.15.5:0/1692205785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05b11e960 con 0x55e05b1215d0 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.711+0000 7f40fe532640 1 --2- 172.21.15.5:0/1692205785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b11de10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f40f40c3e60 tx=0x7f40f40b9c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5847c9524233b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:04.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.711+0000 7f40e3fff640 1 -- 172.21.15.5:0/1692205785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40bb720 con 0x55e05b1215d0 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.711+0000 7f40e3fff640 1 -- 172.21.15.5:0/1692205785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f40c4390 con 0x55e05b1215d0 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.711+0000 7f40e3fff640 1 -- 172.21.15.5:0/1692205785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40babf0 con 0x55e05b1215d0 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.712+0000 7f4102b6f740 1 -- 172.21.15.5:0/1692205785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 msgr2=0x55e05b11de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.712+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1692205785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b11de10 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f40f40c3e60 tx=0x7f40f40b9c30 comp rx=0 tx=0).stop 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.713+0000 7f4102b6f740 1 -- 172.21.15.5:0/1692205785 shutdown_connections 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.713+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1692205785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.713+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1692205785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b122b40 0x55e05b1383e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.713+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1692205785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b1215d0 0x55e05b11de10 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.713+0000 7f4102b6f740 1 -- 172.21.15.5:0/1692205785 >> 172.21.15.5:0/1692205785 conn(0x55e05b0ea310 msgr2=0x55e05b11d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.713+0000 7f4102b6f740 1 -- 172.21.15.5:0/1692205785 shutdown_connections 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.714+0000 7f4102b6f740 1 -- 172.21.15.5:0/1692205785 wait complete. 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.715+0000 7f4102b6f740 1 -- start start 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.715+0000 7f4102b6f740 1 Processor -- start 2024-09-11T22:36:04.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.716+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b1215d0 0x55e05b134b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.716+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b122b40 0x55e05b125ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.716+0000 7f4102b6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b126020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.716+0000 7f40fdd31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b122b40 0x55e05b125ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.717+0000 7f40fdd31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b122b40 0x55e05b125ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38466/0 (socket says 172.21.15.5:38466) 2024-09-11T22:36:04.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.717+0000 7f4102b6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc010060 con 0x55e05b122b40 2024-09-11T22:36:04.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.717+0000 7f40fdd31640 1 -- 172.21.15.5:0/1889074394 learned_addr learned my addr 172.21.15.5:0/1889074394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:04.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.717+0000 7f40fed33640 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b126020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.717+0000 7f4102b6f740 1 -- 172.21.15.5:0/1889074394 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc00f8d0 con 0x55e05b12c370 2024-09-11T22:36:04.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.717+0000 7f4102b6f740 1 -- 172.21.15.5:0/1889074394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc00cb00 con 0x55e05b1215d0 2024-09-11T22:36:04.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.717+0000 7f4102b6f740 10 client.0 _ll_register_callbacks cb 0x7f40f4005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:04.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.718+0000 7f40fe532640 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b1215d0 0x55e05b134b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.722+0000 7f40fe532640 1 -- 172.21.15.5:0/1889074394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 msgr2=0x55e05b126020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:04.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.722+0000 7f40fe532640 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b126020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.722+0000 7f40fe532640 1 -- 172.21.15.5:0/1889074394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b122b40 msgr2=0x55e05b125ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:04.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.722+0000 7f40fe532640 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b122b40 0x55e05b125ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.722+0000 7f40fe532640 1 -- 172.21.15.5:0/1889074394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05b1266c0 con 0x55e05b1215d0 2024-09-11T22:36:04.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.722+0000 7f40fed33640 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b126020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:04.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.722+0000 7f40fdd31640 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b122b40 0x55e05b125ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:04.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.722+0000 7f40fe532640 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b1215d0 0x55e05b134b20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f40f4006010 tx=0x7f40f40c4220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a407e3b773d16efd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:04.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.722+0000 7f40e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:36:04.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.724+0000 7f40e1ffb640 1 -- 172.21.15.5:0/1889074394 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f40bb720 con 0x55e05b1215d0 2024-09-11T22:36:04.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.724+0000 7f40e1ffb640 1 -- 172.21.15.5:0/1889074394 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f40f40b7470 con 0x55e05b1215d0 2024-09-11T22:36:04.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.724+0000 7f4102b6f740 1 -- 172.21.15.5:0/1889074394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f40c4390 con 0x55e05b1215d0 2024-09-11T22:36:04.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.725+0000 7f40e1ffb640 1 -- 172.21.15.5:0/1889074394 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f40f40bfad0 con 0x55e05b1215d0 2024-09-11T22:36:04.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.726+0000 7f4102b6f740 10 client.28759 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:04.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.726+0000 7f4102b6f740 1 -- 172.21.15.5:0/1889074394 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e05b1266c0 con 0x55e05b1215d0 2024-09-11T22:36:04.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.726+0000 7f40e1ffb640 1 -- 172.21.15.5:0/1889074394 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f40b7670 con 0x55e05b1215d0 2024-09-11T22:36:04.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.727+0000 7f40e1ffb640 1 -- 172.21.15.5:0/1889074394 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f40f40bca50 con 0x55e05b1215d0 2024-09-11T22:36:04.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.728+0000 7f4102b6f740 10 client.28759 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:04.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.728+0000 7f4102b6f740 10 client.28759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:36:04.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.728+0000 7f4102b6f740 20 client.28759 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:04.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.728+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55e05b1a6e10 0x55e05b1a9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:04.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.728+0000 7f4102b6f740 4 client.28759 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:36:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.728+0000 7f4102b6f740 1 -- 172.21.15.5:0/1889074394 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f40f40c4390 con 0x55e05b1a6e10 2024-09-11T22:36:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.730+0000 7f40fdd31640 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55e05b1a6e10 0x55e05b1a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.731+0000 7f40fdd31640 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55e05b1a6e10 0x55e05b1a9230 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae6a44ea4bfeab62 server_cookie=42131514dd4edfcc in_seq=0 out_seq=0 2024-09-11T22:36:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.731+0000 7f40e1ffb640 10 client.28759 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:36:04.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f40e1ffb640 1 -- 172.21.15.5:0/1889074394 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f40f40c4390 con 0x55e05b1a6e10 2024-09-11T22:36:04.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f40e1ffb640 10 client.28759 handle_command_reply: tid=0 2024-09-11T22:36:04.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f4102b6f740 1 client.28759 shutdown 2024-09-11T22:36:04.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f4102b6f740 1 -- 172.21.15.5:0/1889074394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b1215d0 msgr2=0x55e05b134b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:04.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b1215d0 0x55e05b134b20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f40f4006010 tx=0x7f40f40c4220 comp rx=0 tx=0).stop 2024-09-11T22:36:04.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f4102b6f740 1 -- 172.21.15.5:0/1889074394 shutdown_connections 2024-09-11T22:36:04.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55e05b1a6e10 0x55e05b1a9230 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e05b12c370 0x55e05b126020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05b122b40 0x55e05b125ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f4102b6f740 1 --2- 172.21.15.5:0/1889074394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e05b1215d0 0x55e05b134b20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f4102b6f740 1 -- 172.21.15.5:0/1889074394 >> 172.21.15.5:0/1889074394 conn(0x55e05b0ea310 msgr2=0x55e05b12afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.732+0000 7f40e1ffb640 0 client.28759 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:36:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.734+0000 7f4102b6f740 1 -- 172.21.15.5:0/1889074394 shutdown_connections 2024-09-11T22:36:04.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.735+0000 7f4102b6f740 1 -- 172.21.15.5:0/1889074394 wait complete. 2024-09-11T22:36:04.744 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:36:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.735+0000 7f4102b6f740 20 client.28759 trim_cache size 0 max 16384 2024-09-11T22:36:04.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.737+0000 7f4100796640 1 -- 172.21.15.5:0/3756303391 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f40e40a2f00 msgr2=0x7f40e40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.737+0000 7f4100796640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f40e40a2f00 0x7f40e40a53c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f40f0005fd0 tx=0x7f40f0009a60 comp rx=0 tx=0).stop 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.737+0000 7f4100796640 1 -- 172.21.15.5:0/3756303391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40f81008c0 msgr2=0x7f40f81070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.737+0000 7f4100796640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40f81008c0 0x7f40f81070b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f40f4005d00 tx=0x7f40f4033d80 comp rx=0 tx=0).stop 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.738+0000 7f4100796640 1 -- 172.21.15.5:0/3756303391 shutdown_connections 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.738+0000 7f4100796640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40f8105ec0 0x7f40f810b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.738+0000 7f4100796640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40f81011e0 0x7f40f81075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.738+0000 7f4100796640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f40e40a2f00 0x7f40e40a53c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.738+0000 7f4100796640 1 --2- 172.21.15.5:0/3756303391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40f81008c0 0x7f40f81070b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.738+0000 7f4100796640 1 -- 172.21.15.5:0/3756303391 >> 172.21.15.5:0/3756303391 conn(0x7f40f80fc7d0 msgr2=0x7f40f80fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.738+0000 7f4100796640 1 -- 172.21.15.5:0/3756303391 shutdown_connections 2024-09-11T22:36:04.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:04.739+0000 7f4100796640 1 -- 172.21.15.5:0/3756303391 wait complete. 2024-09-11T22:36:05.583 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:36:05.631 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 24.42 MB/sec warmup 23 sec latency 9695.515 ms 2024-09-11T22:36:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:05 smithi005 ceph-mon[30943]: pgmap v441: 97 pgs: 97 active+clean; 207 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 9.6 MiB/s wr, 466 op/s 2024-09-11T22:36:05.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:05 smithi005 ceph-mon[30943]: osdmap e108: 12 total, 12 up, 12 in 2024-09-11T22:36:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:05 smithi172 ceph-mon[35461]: pgmap v441: 97 pgs: 97 active+clean; 207 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 9.6 MiB/s wr, 466 op/s 2024-09-11T22:36:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:05 smithi172 ceph-mon[35461]: osdmap e108: 12 total, 12 up, 12 in 2024-09-11T22:36:05.945 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:05 smithi159 ceph-mon[35534]: pgmap v441: 97 pgs: 97 active+clean; 207 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 910 KiB/s rd, 9.6 MiB/s wr, 466 op/s 2024-09-11T22:36:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:05 smithi159 ceph-mon[35534]: osdmap e108: 12 total, 12 up, 12 in 2024-09-11T22:36:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 -- 172.21.15.5:0/127388386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b01036e0 msgr2=0x7f39b0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 --2- 172.21.15.5:0/127388386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b01036e0 0x7f39b0107b90 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009930 tx=0x7f39a402a420 comp rx=0 tx=0).stop 2024-09-11T22:36:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 -- 172.21.15.5:0/127388386 shutdown_connections 2024-09-11T22:36:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 --2- 172.21.15.5:0/127388386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b01082a0 0x7f39b010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 --2- 172.21.15.5:0/127388386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b01036e0 0x7f39b0107b90 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 --2- 172.21.15.5:0/127388386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39b0102d30 0x7f39b0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 -- 172.21.15.5:0/127388386 >> 172.21.15.5:0/127388386 conn(0x7f39b00fc7d0 msgr2=0x7f39b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 -- 172.21.15.5:0/127388386 shutdown_connections 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 -- 172.21.15.5:0/127388386 wait complete. 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 Processor -- start 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.487+0000 7f39b69c8640 1 -- start start 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39b69c8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b0102d30 0x7f39b01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39b69c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b01036e0 0x7f39b0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39b69c8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39b01082a0 0x7f39b019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39b69c8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39b0112060 con 0x7f39b01036e0 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39b69c8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39b0111ee0 con 0x7f39b01082a0 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39b69c8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01121e0 con 0x7f39b0102d30 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b01036e0 0x7f39b0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b01036e0 0x7f39b0198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38486/0 (socket says 172.21.15.5:38486) 2024-09-11T22:36:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39af7fe640 1 -- 172.21.15.5:0/1914228357 learned_addr learned my addr 172.21.15.5:0/1914228357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39af7fe640 1 -- 172.21.15.5:0/1914228357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b0102d30 msgr2=0x7f39b01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39af7fe640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b0102d30 0x7f39b01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39af7fe640 1 -- 172.21.15.5:0/1914228357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39b01082a0 msgr2=0x7f39b019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39af7fe640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39b01082a0 0x7f39b019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39af7fe640 1 -- 172.21.15.5:0/1914228357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b019d570 con 0x7f39b01036e0 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39af7fe640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b01036e0 0x7f39b0198b00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f39a4005940 tx=0x7f39a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39ad7fa640 1 -- 172.21.15.5:0/1914228357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a403f030 con 0x7f39b01036e0 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39ad7fa640 1 -- 172.21.15.5:0/1914228357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a4003ce0 con 0x7f39b01036e0 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.488+0000 7f39ad7fa640 1 -- 172.21.15.5:0/1914228357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4035070 con 0x7f39b01036e0 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.489+0000 7f39b69c8640 1 -- 172.21.15.5:0/1914228357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b019d800 con 0x7f39b01036e0 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.489+0000 7f39b69c8640 1 -- 172.21.15.5:0/1914228357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b01a5140 con 0x7f39b01036e0 2024-09-11T22:36:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39ad7fa640 1 -- 172.21.15.5:0/1914228357 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f39a4042030 con 0x7f39b01036e0 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39b8da1740 1 Processor -- start 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39b8da1740 1 -- start start 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 0x558c85301090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c853015d0 0x558c85301a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c8530e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39b8da1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558c852672f0 con 0x558c8530c370 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39b8da1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558c8520ecf0 con 0x558c85302b40 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39b8da1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558c852632b0 con 0x558c853015d0 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39ad7fa640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39980a2d70 0x7f39980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.491+0000 7f39af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c8530e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c8530e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38500/0 (socket says 172.21.15.5:38500) 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39af7fe640 1 -- 172.21.15.5:0/1521058531 learned_addr learned my addr 172.21.15.5:0/1521058531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39b4f65640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39980a2d70 0x7f39980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39affff640 1 --2- 172.21.15.5:0/1521058531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c853015d0 0x558c85301a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39ad7fa640 1 -- 172.21.15.5:0/1914228357 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f39a40efae0 con 0x7f39b01036e0 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39b4f65640 1 --2- 172.21.15.5:0/1521058531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 0x558c85301090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39af7fe640 1 -- 172.21.15.5:0/1521058531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c853015d0 msgr2=0x558c85301a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39af7fe640 1 --2- 172.21.15.5:0/1521058531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c853015d0 0x558c85301a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39af7fe640 1 -- 172.21.15.5:0/1521058531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 msgr2=0x558c85301090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39af7fe640 1 --2- 172.21.15.5:0/1521058531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 0x558c85301090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39af7fe640 1 -- 172.21.15.5:0/1521058531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c851f33a0 con 0x558c8530c370 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39b4f65640 1 --2- 172.21.15.5:0/1521058531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 0x558c85301090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39af7fe640 1 --2- 172.21.15.5:0/1521058531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c8530e760 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f39a40bbef0 tx=0x7f39a403b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649c9562b5b6456b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39b4f65640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39980a2d70 0x7f39980a5230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f39a8013640 tx=0x7f39a8009200 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39967fc640 1 -- 172.21.15.5:0/1521058531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4049070 con 0x558c8530c370 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39967fc640 1 -- 172.21.15.5:0/1521058531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a40bbf60 con 0x558c8530c370 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.492+0000 7f39967fc640 1 -- 172.21.15.5:0/1521058531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40ed570 con 0x558c8530c370 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.493+0000 7f39b8da1740 1 -- 172.21.15.5:0/1521058531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 msgr2=0x558c8530e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.493+0000 7f39b8da1740 1 --2- 172.21.15.5:0/1521058531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c8530e760 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f39a40bbef0 tx=0x7f39a403b720 comp rx=0 tx=0).stop 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.493+0000 7f39b8da1740 1 -- 172.21.15.5:0/1521058531 shutdown_connections 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.493+0000 7f39b8da1740 1 --2- 172.21.15.5:0/1521058531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c8530e760 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.493+0000 7f39b8da1740 1 --2- 172.21.15.5:0/1521058531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c853015d0 0x558c85301a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.493+0000 7f39b8da1740 1 --2- 172.21.15.5:0/1521058531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 0x558c85301090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.493+0000 7f39b8da1740 1 -- 172.21.15.5:0/1521058531 >> 172.21.15.5:0/1521058531 conn(0x558c852ca310 msgr2=0x558c852fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.493+0000 7f39b8da1740 1 -- 172.21.15.5:0/1521058531 shutdown_connections 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.493+0000 7f39b8da1740 1 -- 172.21.15.5:0/1521058531 wait complete. 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.493+0000 7f39b8da1740 1 -- start start 2024-09-11T22:36:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b8da1740 1 Processor -- start 2024-09-11T22:36:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c853015d0 0x558c8530bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 0x558c85308f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c853094d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b8da1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558c8530ee30 con 0x558c8530c370 2024-09-11T22:36:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b8da1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558c85222060 con 0x558c85302b40 2024-09-11T22:36:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b8da1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558c8530efb0 con 0x558c853015d0 2024-09-11T22:36:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b8da1740 10 client.0 _ll_register_callbacks cb 0x558c84f758c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b4f65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c853094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39af7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 0x558c85308f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b4f65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c853094d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38508/0 (socket says 172.21.15.5:38508) 2024-09-11T22:36:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b4f65640 1 -- 172.21.15.5:0/3994592327 learned_addr learned my addr 172.21.15.5:0/3994592327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39affff640 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c853015d0 0x558c8530bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b4f65640 1 -- 172.21.15.5:0/3994592327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c853015d0 msgr2=0x558c8530bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b4f65640 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c853015d0 0x558c8530bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b4f65640 1 -- 172.21.15.5:0/3994592327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 msgr2=0x558c85308f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.494+0000 7f39b4f65640 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 0x558c85308f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f39b4f65640 1 -- 172.21.15.5:0/3994592327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c84fd83c0 con 0x558c8530c370 2024-09-11T22:36:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f39af7fe640 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 0x558c85308f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f39b4f65640 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c853094d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f39a800f680 tx=0x7f39a8005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66df1a27e71cf62 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f398f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:06.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f398f7fe640 1 -- 172.21.15.5:0/3994592327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8023070 con 0x558c8530c370 2024-09-11T22:36:06.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f39b8da1740 1 -- 172.21.15.5:0/3994592327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39a40bbf60 con 0x558c8530c370 2024-09-11T22:36:06.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f398f7fe640 1 -- 172.21.15.5:0/3994592327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f39a80109d0 con 0x558c8530c370 2024-09-11T22:36:06.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f398f7fe640 1 -- 172.21.15.5:0/3994592327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f39a8010bd0 con 0x558c8530c370 2024-09-11T22:36:06.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f39b8da1740 10 client.19761 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:06.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f39b8da1740 1 -- 172.21.15.5:0/3994592327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558c8531a780 con 0x558c8530c370 2024-09-11T22:36:06.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.495+0000 7f398f7fe640 1 -- 172.21.15.5:0/3994592327 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a40bbf60 con 0x558c8530c370 2024-09-11T22:36:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.496+0000 7f398f7fe640 1 -- 172.21.15.5:0/3994592327 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f39a800f040 con 0x558c8530c370 2024-09-11T22:36:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.496+0000 7f39b8da1740 10 client.19761 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.496+0000 7f39b8da1740 10 client.19761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.496+0000 7f39b8da1740 20 client.19761 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.496+0000 7f39b8da1740 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558c8531bbe0 0x558c8531e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.496+0000 7f39b8da1740 4 client.19761 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.496+0000 7f39b8da1740 1 -- 172.21.15.5:0/3994592327 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c85274860 con 0x558c8531bbe0 2024-09-11T22:36:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.496+0000 7f39affff640 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558c8531bbe0 0x558c8531e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f398f7fe640 10 client.19761 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:06.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f39affff640 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558c8531bbe0 0x558c8531e040 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc9bc9277d2394d server_cookie=20a37b6f176c1fdd in_seq=0 out_seq=0 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f398f7fe640 1 -- 172.21.15.5:0/3994592327 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c85274860 con 0x558c8531bbe0 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f398f7fe640 10 client.19761 handle_command_reply: tid=0 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f39b8da1740 1 client.19761 shutdown 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f39b8da1740 1 -- 172.21.15.5:0/3994592327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 msgr2=0x558c853094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f39b8da1740 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c853094d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f39a800f680 tx=0x7f39a8005c40 comp rx=0 tx=0).stop 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f39b8da1740 1 -- 172.21.15.5:0/3994592327 shutdown_connections 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f39b8da1740 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558c8531bbe0 0x558c8531e040 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f39b8da1740 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c8530c370 0x558c853094d0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f39b8da1740 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c85302b40 0x558c85308f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f39b8da1740 1 --2- 172.21.15.5:0/3994592327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c853015d0 0x558c8530bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f39b8da1740 1 -- 172.21.15.5:0/3994592327 >> 172.21.15.5:0/3994592327 conn(0x558c852ca310 msgr2=0x558c85303990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.497+0000 7f398f7fe640 0 client.19761 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:06.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.498+0000 7f39b8da1740 1 -- 172.21.15.5:0/3994592327 shutdown_connections 2024-09-11T22:36:06.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.498+0000 7f39b8da1740 1 -- 172.21.15.5:0/3994592327 wait complete. 2024-09-11T22:36:06.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.498+0000 7f39b8da1740 20 client.19761 trim_cache size 0 max 16384 2024-09-11T22:36:06.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39b8da1740 1 Processor -- start 2024-09-11T22:36:06.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39b8da1740 1 -- start start 2024-09-11T22:36:06.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c853015d0 0x558c8530aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c85304180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c8530c370 0x558c853046c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39b8da1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01ab9b0 con 0x558c85302b40 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39b8da1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01ad5e0 con 0x558c853015d0 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39b8da1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01ac080 con 0x558c8530c370 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c85304180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c85304180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38524/0 (socket says 172.21.15.5:38524) 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39af7fe640 1 -- 172.21.15.5:0/2129026623 learned_addr learned my addr 172.21.15.5:0/2129026623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39affff640 1 --2- 172.21.15.5:0/2129026623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c853015d0 0x558c8530aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39b4f65640 1 --2- 172.21.15.5:0/2129026623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c8530c370 0x558c853046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39af7fe640 1 -- 172.21.15.5:0/2129026623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c8530c370 msgr2=0x558c853046c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39af7fe640 1 --2- 172.21.15.5:0/2129026623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c8530c370 0x558c853046c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39af7fe640 1 -- 172.21.15.5:0/2129026623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c853015d0 msgr2=0x558c8530aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39af7fe640 1 --2- 172.21.15.5:0/2129026623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c853015d0 0x558c8530aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39af7fe640 1 -- 172.21.15.5:0/2129026623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8530b8e0 con 0x558c85302b40 2024-09-11T22:36:06.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.511+0000 7f39af7fe640 1 --2- 172.21.15.5:0/2129026623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c85304180 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f39a40b83c0 tx=0x7f39a40eff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59fff329fda07b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f3995ffb640 1 -- 172.21.15.5:0/2129026623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4049040 con 0x558c85302b40 2024-09-11T22:36:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f3995ffb640 1 -- 172.21.15.5:0/2129026623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a40c4bd0 con 0x558c85302b40 2024-09-11T22:36:06.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f3995ffb640 1 -- 172.21.15.5:0/2129026623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a40b85d0 con 0x558c85302b40 2024-09-11T22:36:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f39b8da1740 1 -- 172.21.15.5:0/2129026623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 msgr2=0x558c85304180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f39b8da1740 1 --2- 172.21.15.5:0/2129026623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c85304180 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f39a40b83c0 tx=0x7f39a40eff90 comp rx=0 tx=0).stop 2024-09-11T22:36:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f39b8da1740 1 -- 172.21.15.5:0/2129026623 shutdown_connections 2024-09-11T22:36:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f39b8da1740 1 --2- 172.21.15.5:0/2129026623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c8530c370 0x558c853046c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f39b8da1740 1 --2- 172.21.15.5:0/2129026623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c85304180 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f39b8da1740 1 --2- 172.21.15.5:0/2129026623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c853015d0 0x558c8530aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f39b8da1740 1 -- 172.21.15.5:0/2129026623 >> 172.21.15.5:0/2129026623 conn(0x558c852ca310 msgr2=0x558c8530a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f39b8da1740 1 -- 172.21.15.5:0/2129026623 shutdown_connections 2024-09-11T22:36:06.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.512+0000 7f39b8da1740 1 -- 172.21.15.5:0/2129026623 wait complete. 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39b8da1740 1 -- start start 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39b8da1740 1 Processor -- start 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c853015d0 0x558c8530f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c8530fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39b8da1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c8530c370 0x558c8537e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39b8da1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01acf10 con 0x558c85302b40 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39b8da1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01ac780 con 0x558c853015d0 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39b8da1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01a99b0 con 0x558c8530c370 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39b8da1740 10 client.0 _ll_register_callbacks cb 0x7f39a8009f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c8530fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c8530fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38536/0 (socket says 172.21.15.5:38536) 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39af7fe640 1 -- 172.21.15.5:0/2837172370 learned_addr learned my addr 172.21.15.5:0/2837172370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:06.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39af7fe640 1 -- 172.21.15.5:0/2837172370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c8530c370 msgr2=0x558c8537e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39af7fe640 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c8530c370 0x558c8537e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39af7fe640 1 -- 172.21.15.5:0/2837172370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c853015d0 msgr2=0x558c8530f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39af7fe640 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c853015d0 0x558c8530f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.513+0000 7f39af7fe640 1 -- 172.21.15.5:0/2837172370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8537ef40 con 0x558c85302b40 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f39af7fe640 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c8530fc20 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f39a40dc0b0 tx=0x7f39a402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06816cda1c471a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f398f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f398f7fe640 1 -- 172.21.15.5:0/2837172370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a4049040 con 0x558c85302b40 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f398f7fe640 1 -- 172.21.15.5:0/2837172370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f39a403bc50 con 0x558c85302b40 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f39b8da1740 1 -- 172.21.15.5:0/2837172370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39a40c4bd0 con 0x558c85302b40 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f398f7fe640 1 -- 172.21.15.5:0/2837172370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f39a403be50 con 0x558c85302b40 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f39b8da1740 10 client.19767 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f39b8da1740 1 -- 172.21.15.5:0/2837172370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558c853862b0 con 0x558c85302b40 2024-09-11T22:36:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f398f7fe640 1 -- 172.21.15.5:0/2837172370 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39a40c4bd0 con 0x558c85302b40 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f398f7fe640 1 -- 172.21.15.5:0/2837172370 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f39a40bb3b0 con 0x558c85302b40 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f39b8da1740 10 client.19767 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f39b8da1740 10 client.19767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f39b8da1740 20 client.19767 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f39b8da1740 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558c85386740 0x558c85388b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f39b8da1740 4 client.19767 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.514+0000 7f39b8da1740 1 -- 172.21.15.5:0/2837172370 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c8526ed70 con 0x558c85386740 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.515+0000 7f39affff640 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558c85386740 0x558c85388b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.515+0000 7f39affff640 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558c85386740 0x558c85388b60 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54fa50e7a7c3372e server_cookie=83215b13f941e1d5 in_seq=0 out_seq=0 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.515+0000 7f398f7fe640 10 client.19767 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.515+0000 7f398f7fe640 1 -- 172.21.15.5:0/2837172370 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c8526ed70 con 0x558c85386740 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.515+0000 7f398f7fe640 10 client.19767 handle_command_reply: tid=0 2024-09-11T22:36:06.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.515+0000 7f39b8da1740 1 client.19767 shutdown 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 1 -- 172.21.15.5:0/2837172370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 msgr2=0x558c8530fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c8530fc20 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f39a40dc0b0 tx=0x7f39a402a420 comp rx=0 tx=0).stop 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 1 -- 172.21.15.5:0/2837172370 shutdown_connections 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558c85386740 0x558c85388b60 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558c8530c370 0x558c8537e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c85302b40 0x558c8530fc20 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 1 --2- 172.21.15.5:0/2837172370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558c853015d0 0x558c8530f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 1 -- 172.21.15.5:0/2837172370 >> 172.21.15.5:0/2837172370 conn(0x558c852ca310 msgr2=0x558c852fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f398f7fe640 0 client.19767 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 1 -- 172.21.15.5:0/2837172370 shutdown_connections 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 1 -- 172.21.15.5:0/2837172370 wait complete. 2024-09-11T22:36:06.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b8da1740 20 client.19767 trim_cache size 0 max 16384 2024-09-11T22:36:06.522 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:36:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b69c8640 1 -- 172.21.15.5:0/1914228357 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39980a2d70 msgr2=0x7f39980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b69c8640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39980a2d70 0x7f39980a5230 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f39a8013640 tx=0x7f39a8009200 comp rx=0 tx=0).stop 2024-09-11T22:36:06.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b69c8640 1 -- 172.21.15.5:0/1914228357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b01036e0 msgr2=0x7f39b0198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.516+0000 7f39b69c8640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b01036e0 0x7f39b0198b00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f39a4005940 tx=0x7f39a40042f0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.517+0000 7f39b69c8640 1 -- 172.21.15.5:0/1914228357 shutdown_connections 2024-09-11T22:36:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.517+0000 7f39b69c8640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f39980a2d70 0x7f39980a5230 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.517+0000 7f39b69c8640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39b01082a0 0x7f39b019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.517+0000 7f39b69c8640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b01036e0 0x7f39b0198b00 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.517+0000 7f39b69c8640 1 --2- 172.21.15.5:0/1914228357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b0102d30 0x7f39b01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.517+0000 7f39b69c8640 1 -- 172.21.15.5:0/1914228357 >> 172.21.15.5:0/1914228357 conn(0x7f39b00fc7d0 msgr2=0x7f39b0106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.517+0000 7f39b69c8640 1 -- 172.21.15.5:0/1914228357 shutdown_connections 2024-09-11T22:36:06.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:06.517+0000 7f39b69c8640 1 -- 172.21.15.5:0/1914228357 wait complete. 2024-09-11T22:36:06.631 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 23.40 MB/sec warmup 24 sec latency 10695.597 ms 2024-09-11T22:36:07.631 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 22.47 MB/sec warmup 25 sec latency 11695.690 ms 2024-09-11T22:36:07.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:07 smithi005 ceph-mon[30943]: pgmap v443: 97 pgs: 97 active+clean; 207 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 12 MiB/s wr, 559 op/s 2024-09-11T22:36:07.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:07 smithi172 ceph-mon[35461]: pgmap v443: 97 pgs: 97 active+clean; 207 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 12 MiB/s wr, 559 op/s 2024-09-11T22:36:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:07 smithi159 ceph-mon[35534]: pgmap v443: 97 pgs: 97 active+clean; 207 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 12 MiB/s wr, 559 op/s 2024-09-11T22:36:08.383 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:36:08.383 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:36:08.631 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 21.60 MB/sec warmup 26 sec latency 12695.735 ms 2024-09-11T22:36:08.654 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.191+0000 7f6237599640 1 -- 172.21.15.5:0/609334097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6230101de0 msgr2=0x7f62301021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.191+0000 7f6237599640 1 --2- 172.21.15.5:0/609334097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6230101de0 0x7f62301021c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f622c009900 tx=0x7f622c02a410 comp rx=0 tx=0).stop 2024-09-11T22:36:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.192+0000 7f6237599640 1 -- 172.21.15.5:0/609334097 shutdown_connections 2024-09-11T22:36:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.192+0000 7f6237599640 1 --2- 172.21.15.5:0/609334097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6230106f10 0x7f623010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.192+0000 7f6237599640 1 --2- 172.21.15.5:0/609334097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6230102790 0x7f62301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.192+0000 7f6237599640 1 --2- 172.21.15.5:0/609334097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6230101de0 0x7f62301021c0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.192+0000 7f6237599640 1 -- 172.21.15.5:0/609334097 >> 172.21.15.5:0/609334097 conn(0x7f62300fd8c0 msgr2=0x7f62300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.192+0000 7f6237599640 1 -- 172.21.15.5:0/609334097 shutdown_connections 2024-09-11T22:36:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.192+0000 7f6237599640 1 -- 172.21.15.5:0/609334097 wait complete. 2024-09-11T22:36:09.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.192+0000 7f6237599640 1 Processor -- start 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.192+0000 7f6237599640 1 -- start start 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6237599640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6230101de0 0x7f6230198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6237599640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6230102790 0x7f6230198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6237599640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6230106f10 0x7f623019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6237599640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6230111f30 con 0x7f6230106f10 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6237599640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6230111db0 con 0x7f6230101de0 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6237599640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62301120b0 con 0x7f6230102790 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6235b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6230106f10 0x7f623019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6235b36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6230106f10 0x7f623019cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38544/0 (socket says 172.21.15.5:38544) 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6235b36640 1 -- 172.21.15.5:0/1487606694 learned_addr learned my addr 172.21.15.5:0/1487606694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6234b34640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6230102790 0x7f6230198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6235335640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6230101de0 0x7f6230198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6235b36640 1 -- 172.21.15.5:0/1487606694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6230102790 msgr2=0x7f6230198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6235b36640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6230102790 0x7f6230198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6235b36640 1 -- 172.21.15.5:0/1487606694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6230101de0 msgr2=0x7f6230198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6235b36640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6230101de0 0x7f6230198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6235b36640 1 -- 172.21.15.5:0/1487606694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623019d4c0 con 0x7f6230106f10 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6235b36640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6230106f10 0x7f623019cde0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f622800eb10 tx=0x7f622800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f62267fc640 1 -- 172.21.15.5:0/1487606694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228004b20 con 0x7f6230106f10 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f62267fc640 1 -- 172.21.15.5:0/1487606694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6228004cc0 con 0x7f6230106f10 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6237599640 1 -- 172.21.15.5:0/1487606694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623019d7b0 con 0x7f6230106f10 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f62267fc640 1 -- 172.21.15.5:0/1487606694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6228010760 con 0x7f6230106f10 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.193+0000 7f6237599640 1 -- 172.21.15.5:0/1487606694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62301a5090 con 0x7f6230106f10 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f62267fc640 1 -- 172.21.15.5:0/1487606694 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6228005020 con 0x7f6230106f10 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6239972740 1 Processor -- start 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6239972740 1 -- start start 2024-09-11T22:36:09.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6239972740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3ba60 0x557a9ed3be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6239972740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3c380 0x557a9ed4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6239972740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed4b150 0x557a9ed4d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6239972740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557a9eca1280 con 0x557a9ed4b150 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6239972740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557a9ec48b10 con 0x557a9ed3ba60 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6239972740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557a9ec9d240 con 0x557a9ed3c380 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6234b34640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed4b150 0x557a9ed4d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6234b34640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3ba60 0x557a9ed3be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6234b34640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3ba60 0x557a9ed3be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50308/0 (socket says 172.21.15.5:50308) 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6234b34640 1 -- 172.21.15.5:0/2676404815 learned_addr learned my addr 172.21.15.5:0/2676404815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6235335640 1 --2- 172.21.15.5:0/2676404815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3c380 0x557a9ed4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6234b34640 1 -- 172.21.15.5:0/2676404815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3c380 msgr2=0x557a9ed4ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6234b34640 1 --2- 172.21.15.5:0/2676404815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3c380 0x557a9ed4ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6234b34640 1 -- 172.21.15.5:0/2676404815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed4b150 msgr2=0x557a9ed4d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:09.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f6234b34640 1 --2- 172.21.15.5:0/2676404815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed4b150 0x557a9ed4d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.197+0000 7f62267fc640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f621c0a2d70 0x7f621c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.198+0000 7f6234b34640 1 -- 172.21.15.5:0/2676404815 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9ec551f0 con 0x557a9ed3ba60 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.198+0000 7f6235335640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f621c0a2d70 0x7f621c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.198+0000 7f62267fc640 1 -- 172.21.15.5:0/1487606694 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f62280cfb40 con 0x7f6230106f10 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.198+0000 7f6234b34640 1 --2- 172.21.15.5:0/2676404815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3ba60 0x557a9ed3be40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6220014a40 tx=0x7f62200046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd539a5d822b46eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.198+0000 7f621b7fe640 1 -- 172.21.15.5:0/2676404815 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200142d0 con 0x557a9ed3ba60 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.198+0000 7f6235335640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f621c0a2d70 0x7f621c0a5230 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557a9ed3e750 tx=0x7f622c005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.198+0000 7f621b7fe640 1 -- 172.21.15.5:0/2676404815 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6220011d80 con 0x557a9ed3ba60 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.198+0000 7f621b7fe640 1 -- 172.21.15.5:0/2676404815 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200047a0 con 0x557a9ed3ba60 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 -- 172.21.15.5:0/2676404815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3ba60 msgr2=0x557a9ed3be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 --2- 172.21.15.5:0/2676404815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3ba60 0x557a9ed3be40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6220014a40 tx=0x7f62200046c0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 -- 172.21.15.5:0/2676404815 shutdown_connections 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 --2- 172.21.15.5:0/2676404815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed4b150 0x557a9ed4d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 --2- 172.21.15.5:0/2676404815 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3c380 0x557a9ed4ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 --2- 172.21.15.5:0/2676404815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3ba60 0x557a9ed3be40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 -- 172.21.15.5:0/2676404815 >> 172.21.15.5:0/2676404815 conn(0x557a9ed380f0 msgr2=0x557a9ed384d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 -- 172.21.15.5:0/2676404815 shutdown_connections 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 -- 172.21.15.5:0/2676404815 wait complete. 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 -- start start 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.199+0000 7f6239972740 1 Processor -- start 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6239972740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed47ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6239972740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3c380 0x557a9ed48530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6239972740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed4b150 0x557a9ed48a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6239972740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557a9ed4dd20 con 0x557a9ed3ba60 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6239972740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557a9ec5bf10 con 0x557a9ed4b150 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6239972740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557a9ed4dea0 con 0x557a9ed3c380 2024-09-11T22:36:09.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6239972740 10 client.0 _ll_register_callbacks cb 0x557a9ec24a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6235335640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed47ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6235335640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed47ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38558/0 (socket says 172.21.15.5:38558) 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6235335640 1 -- 172.21.15.5:0/61807517 learned_addr learned my addr 172.21.15.5:0/61807517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6234b34640 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3c380 0x557a9ed48530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6235b36640 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed4b150 0x557a9ed48a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6235335640 1 -- 172.21.15.5:0/61807517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3c380 msgr2=0x557a9ed48530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6235335640 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3c380 0x557a9ed48530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6235335640 1 -- 172.21.15.5:0/61807517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed4b150 msgr2=0x557a9ed48a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6235335640 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed4b150 0x557a9ed48a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.200+0000 7f6235335640 1 -- 172.21.15.5:0/61807517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9ea126a0 con 0x557a9ed3ba60 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6235335640 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed47ff0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f622c009900 tx=0x7f622c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fa607c2fc1fdd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6218ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:09.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6218ff9640 1 -- 172.21.15.5:0/61807517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c04b070 con 0x557a9ed3ba60 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6218ff9640 1 -- 172.21.15.5:0/61807517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f622c004250 con 0x557a9ed3ba60 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6239972740 1 -- 172.21.15.5:0/61807517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6220011d80 con 0x557a9ed3ba60 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6218ff9640 1 -- 172.21.15.5:0/61807517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f622c057050 con 0x557a9ed3ba60 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6239972740 10 client.19779 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6239972740 1 -- 172.21.15.5:0/61807517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557a9ed59930 con 0x557a9ed3ba60 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6218ff9640 1 -- 172.21.15.5:0/61807517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6220011d80 con 0x557a9ed3ba60 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6218ff9640 1 -- 172.21.15.5:0/61807517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f622c047430 con 0x557a9ed3ba60 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6239972740 10 client.19779 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6239972740 10 client.19779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6239972740 20 client.19779 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6239972740 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a9ed5adc0 0x557a9ed5d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6239972740 4 client.19779 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.201+0000 7f6239972740 1 -- 172.21.15.5:0/61807517 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a9ecae7f0 con 0x557a9ed5adc0 2024-09-11T22:36:09.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.202+0000 7f6234b34640 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a9ed5adc0 0x557a9ed5d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.202+0000 7f6234b34640 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a9ed5adc0 0x557a9ed5d200 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea7b484e5da6d4d9 server_cookie=e15baffa0f918590 in_seq=0 out_seq=0 2024-09-11T22:36:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.202+0000 7f6218ff9640 10 client.19779 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6218ff9640 1 -- 172.21.15.5:0/61807517 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a9ecae7f0 con 0x557a9ed5adc0 2024-09-11T22:36:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6218ff9640 10 client.19779 handle_command_reply: tid=0 2024-09-11T22:36:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 client.19779 shutdown 2024-09-11T22:36:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 -- 172.21.15.5:0/61807517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 msgr2=0x557a9ed47ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed47ff0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f622c009900 tx=0x7f622c02a410 comp rx=0 tx=0).stop 2024-09-11T22:36:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 -- 172.21.15.5:0/61807517 shutdown_connections 2024-09-11T22:36:09.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a9ed5adc0 0x557a9ed5d200 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed4b150 0x557a9ed48a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3c380 0x557a9ed48530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 --2- 172.21.15.5:0/61807517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed47ff0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 -- 172.21.15.5:0/61807517 >> 172.21.15.5:0/61807517 conn(0x557a9ed380f0 msgr2=0x557a9ed3de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6218ff9640 0 client.19779 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 -- 172.21.15.5:0/61807517 shutdown_connections 2024-09-11T22:36:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 1 -- 172.21.15.5:0/61807517 wait complete. 2024-09-11T22:36:09.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.203+0000 7f6239972740 20 client.19779 trim_cache size 0 max 16384 2024-09-11T22:36:09.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6239972740 1 Processor -- start 2024-09-11T22:36:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6239972740 1 -- start start 2024-09-11T22:36:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6239972740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6239972740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3c380 0x557a9ed49e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6239972740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed4b150 0x557a9ed62ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6239972740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62301ab9c0 con 0x557a9ed3ba60 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6239972740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62301ad5f0 con 0x557a9ed3c380 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6239972740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62301ac090 con 0x557a9ed4b150 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6235335640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6235335640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38574/0 (socket says 172.21.15.5:38574) 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6235335640 1 -- 172.21.15.5:0/1904371889 learned_addr learned my addr 172.21.15.5:0/1904371889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6234b34640 1 --2- 172.21.15.5:0/1904371889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3c380 0x557a9ed49e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6235b36640 1 --2- 172.21.15.5:0/1904371889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed4b150 0x557a9ed62ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6235335640 1 -- 172.21.15.5:0/1904371889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed4b150 msgr2=0x557a9ed62ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6235335640 1 --2- 172.21.15.5:0/1904371889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed4b150 0x557a9ed62ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6235335640 1 -- 172.21.15.5:0/1904371889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3c380 msgr2=0x557a9ed49e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:09.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6235335640 1 --2- 172.21.15.5:0/1904371889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3c380 0x557a9ed49e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6235335640 1 -- 172.21.15.5:0/1904371889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9ed4de90 con 0x557a9ed3ba60 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.221+0000 7f6234b34640 1 --2- 172.21.15.5:0/1904371889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3c380 0x557a9ed49e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f6235335640 1 --2- 172.21.15.5:0/1904371889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed488c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f622c02a930 tx=0x7f622c032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b121bd02f77beec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f621affd640 1 -- 172.21.15.5:0/1904371889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c04b040 con 0x557a9ed3ba60 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f621affd640 1 -- 172.21.15.5:0/1904371889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f622c004250 con 0x557a9ed3ba60 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f621affd640 1 -- 172.21.15.5:0/1904371889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c035050 con 0x557a9ed3ba60 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f6239972740 1 -- 172.21.15.5:0/1904371889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 msgr2=0x557a9ed488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f6239972740 1 --2- 172.21.15.5:0/1904371889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed488c0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f622c02a930 tx=0x7f622c032c30 comp rx=0 tx=0).stop 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f6239972740 1 -- 172.21.15.5:0/1904371889 shutdown_connections 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f6239972740 1 --2- 172.21.15.5:0/1904371889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed4b150 0x557a9ed62ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f6239972740 1 --2- 172.21.15.5:0/1904371889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed3c380 0x557a9ed49e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f6239972740 1 --2- 172.21.15.5:0/1904371889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3ba60 0x557a9ed488c0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.222+0000 7f6239972740 1 -- 172.21.15.5:0/1904371889 >> 172.21.15.5:0/1904371889 conn(0x557a9ed380f0 msgr2=0x557a9ed49740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:09.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.223+0000 7f6239972740 1 -- 172.21.15.5:0/1904371889 shutdown_connections 2024-09-11T22:36:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.223+0000 7f6239972740 1 -- 172.21.15.5:0/1904371889 wait complete. 2024-09-11T22:36:09.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.223+0000 7f6239972740 1 -- start start 2024-09-11T22:36:09.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.223+0000 7f6239972740 1 Processor -- start 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.223+0000 7f6239972740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3ba60 0x557a9ed53300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.223+0000 7f6239972740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3c380 0x557a9ed53840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.223+0000 7f6239972740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed4b150 0x557a9ed4e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.223+0000 7f6239972740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62301acf20 con 0x557a9ed3c380 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.223+0000 7f6239972740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62301ac790 con 0x557a9ed4b150 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.223+0000 7f6239972740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f62301a99c0 con 0x557a9ed3ba60 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6239972740 10 client.0 _ll_register_callbacks cb 0x7f622c03f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6234b34640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3c380 0x557a9ed53840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6234b34640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3c380 0x557a9ed53840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38578/0 (socket says 172.21.15.5:38578) 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6234b34640 1 -- 172.21.15.5:0/3125341433 learned_addr learned my addr 172.21.15.5:0/3125341433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:09.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6235335640 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3ba60 0x557a9ed53300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6235b36640 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed4b150 0x557a9ed4e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6234b34640 1 -- 172.21.15.5:0/3125341433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3ba60 msgr2=0x557a9ed53300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6234b34640 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3ba60 0x557a9ed53300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6234b34640 1 -- 172.21.15.5:0/3125341433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed4b150 msgr2=0x557a9ed4e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6234b34640 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed4b150 0x557a9ed4e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6234b34640 1 -- 172.21.15.5:0/3125341433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9ed4eef0 con 0x557a9ed3c380 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6234b34640 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3c380 0x557a9ed53840 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f62200149e0 tx=0x7f6220005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4137900048c0e97e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6218ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6218ff9640 1 -- 172.21.15.5:0/3125341433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200140c0 con 0x557a9ed3c380 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6218ff9640 1 -- 172.21.15.5:0/3125341433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f6220014260 con 0x557a9ed3c380 2024-09-11T22:36:09.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6239972740 1 -- 172.21.15.5:0/3125341433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f622c004250 con 0x557a9ed3c380 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.224+0000 7f6218ff9640 1 -- 172.21.15.5:0/3125341433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f6220038050 con 0x557a9ed3c380 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6239972740 10 client.19785 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6239972740 1 -- 172.21.15.5:0/3125341433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x557a9edc55f0 con 0x557a9ed3c380 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6218ff9640 1 -- 172.21.15.5:0/3125341433 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f622c004250 con 0x557a9ed3c380 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6218ff9640 1 -- 172.21.15.5:0/3125341433 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f6220029070 con 0x557a9ed3c380 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6239972740 10 client.19785 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6239972740 10 client.19785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6239972740 20 client.19785 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6239972740 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a9edc5ae0 0x557a9edc7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6239972740 4 client.19785 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6239972740 1 -- 172.21.15.5:0/3125341433 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557a9eca8d00 con 0x557a9edc5ae0 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.225+0000 7f6235335640 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a9edc5ae0 0x557a9edc7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:09.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.226+0000 7f6218ff9640 10 client.19785 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.226+0000 7f6235335640 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a9edc5ae0 0x557a9edc7f00 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a16b20b87c3c0d server_cookie=aae47ef653075197 in_seq=0 out_seq=0 2024-09-11T22:36:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6218ff9640 1 -- 172.21.15.5:0/3125341433 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a9eca8d00 con 0x557a9edc5ae0 2024-09-11T22:36:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6218ff9640 10 client.19785 handle_command_reply: tid=0 2024-09-11T22:36:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 client.19785 shutdown 2024-09-11T22:36:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 -- 172.21.15.5:0/3125341433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3c380 msgr2=0x557a9ed53840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3c380 0x557a9ed53840 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f62200149e0 tx=0x7f6220005b20 comp rx=0 tx=0).stop 2024-09-11T22:36:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 -- 172.21.15.5:0/3125341433 shutdown_connections 2024-09-11T22:36:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557a9edc5ae0 0x557a9edc7f00 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557a9ed4b150 0x557a9ed4e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557a9ed3c380 0x557a9ed53840 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 --2- 172.21.15.5:0/3125341433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9ed3ba60 0x557a9ed53300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 -- 172.21.15.5:0/3125341433 >> 172.21.15.5:0/3125341433 conn(0x557a9ed380f0 msgr2=0x557a9ed3ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6218ff9640 0 client.19785 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 -- 172.21.15.5:0/3125341433 shutdown_connections 2024-09-11T22:36:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 1 -- 172.21.15.5:0/3125341433 wait complete. 2024-09-11T22:36:09.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.227+0000 7f6239972740 20 client.19785 trim_cache size 0 max 16384 2024-09-11T22:36:09.235 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:36:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:36:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "09ce1aea-442b-4b7d-9017-7157c7f1fdf4", 2024-09-11T22:36:09.235 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:36:09.235 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:36:09.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 -- 172.21.15.5:0/1487606694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f621c0a2d70 msgr2=0x7f621c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f621c0a2d70 0x7f621c0a5230 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557a9ed3e750 tx=0x7f622c005a20 comp rx=0 tx=0).stop 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 -- 172.21.15.5:0/1487606694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6230106f10 msgr2=0x7f623019cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6230106f10 0x7f623019cde0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f622800eb10 tx=0x7f622800c2d0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 -- 172.21.15.5:0/1487606694 shutdown_connections 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6230106f10 0x7f623019cde0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6230102790 0x7f6230198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f621c0a2d70 0x7f621c0a5230 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 --2- 172.21.15.5:0/1487606694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6230101de0 0x7f6230198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 -- 172.21.15.5:0/1487606694 >> 172.21.15.5:0/1487606694 conn(0x7f62300fd8c0 msgr2=0x7f62300fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 -- 172.21.15.5:0/1487606694 shutdown_connections 2024-09-11T22:36:09.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:09.228+0000 7f6237599640 1 -- 172.21.15.5:0/1487606694 wait complete. 2024-09-11T22:36:09.579 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:36:09 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:36:09.285+0000 7f54cdab7640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000221 [...16,head] /volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO/ auth{1=1,2=1,3=1,4=1} v208456 ap=3 RANDEPHEMERALPIN f(v5955 m2024-09-11T22:35:46.861726+0000 3=3+0) n(v70 rc2024-09-11T22:35:46.874725+0000 b780288 4=3+1) old_inodes=6 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1332 nref=3)) caps={24841=pAsLsXsFs/pAsLsXsFsx@12040} mcw={1=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 importing=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x556c7fef9b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000221:[<0x10000000009/WORDPRO v2606>,<0x10000000008/~dmtmp v158>,<0x10000000007/client0 v20>,<0x10000000006/clients v10>,<0x10000000005/tmp v10>,<0x100000001fb/client.0 v14>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v30>,<0x10000000001/sv_1 v53>,<0x10000000000/qa v43>,<0x1/volumes v42>]//[]","memoryvalue":"(2)0x10000000221:[<0x10000000009/WORDPRO v208456>,<0x10000000008/~dmtmp v13168>,<0x10000000007/client0 v1849>,<0x10000000006/clients v1140>,<0x10000000005/tmp v1116>,<0x100000001fb/client.0 v1122>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v1128>,<0x10000000001/sv_1 v1132>,<0x10000000000/qa v1097>,<0x1/volumes v1079>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T22:35:46.824726+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-09-11T22:35:46.874725+0000 b780288 4=3+1)","memory_value.dirstat":"f(v5955 m2024-09-11T22:35:46.861726+0000 3=3+0)","memory_value.rstat":"n(v70 rc2024-09-11T22:35:46.874725+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-11T22:36:09.631 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 20.80 MB/sec warmup 27 sec latency 13695.820 ms 2024-09-11T22:36:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:09 smithi172 ceph-mon[35461]: pgmap v444: 97 pgs: 97 active+clean; 207 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-09-11T22:36:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:09 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:09 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:09 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:09.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:09 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:09.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:09 smithi005 ceph-mon[30943]: pgmap v444: 97 pgs: 97 active+clean; 207 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-09-11T22:36:09.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:09 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:09.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:09 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:09.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:09 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:09.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:09 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:09 smithi159 ceph-mon[35534]: pgmap v444: 97 pgs: 97 active+clean; 207 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-09-11T22:36:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:09 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:09 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:09 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:09 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:36:10.631 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 20.06 MB/sec warmup 28 sec latency 14695.846 ms 2024-09-11T22:36:10.663 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:36:10.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:10.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:10 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:36:10.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:10.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:10.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:10.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:10.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:10.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:36:10.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:10.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:10.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:10 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.056+0000 7f25108f1640 1 -- 172.21.15.5:0/226276219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c1069c0 msgr2=0x7f250c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.056+0000 7f25108f1640 1 --2- 172.21.15.5:0/226276219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c1069c0 0x7f250c106da0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009cd0 tx=0x7f24fc02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.056+0000 7f25108f1640 1 -- 172.21.15.5:0/226276219 shutdown_connections 2024-09-11T22:36:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.056+0000 7f25108f1640 1 --2- 172.21.15.5:0/226276219 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f250c101350 0x7f250c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.056+0000 7f25108f1640 1 --2- 172.21.15.5:0/226276219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f250c1009b0 0x7f250c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.056+0000 7f25108f1640 1 --2- 172.21.15.5:0/226276219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c1069c0 0x7f250c106da0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.056+0000 7f25108f1640 1 -- 172.21.15.5:0/226276219 >> 172.21.15.5:0/226276219 conn(0x7f250c077990 msgr2=0x7f250c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.056+0000 7f25108f1640 1 -- 172.21.15.5:0/226276219 shutdown_connections 2024-09-11T22:36:11.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.056+0000 7f25108f1640 1 -- 172.21.15.5:0/226276219 wait complete. 2024-09-11T22:36:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f25108f1640 1 Processor -- start 2024-09-11T22:36:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f25108f1640 1 -- start start 2024-09-11T22:36:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f25108f1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f250c1009b0 0x7f250c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f25108f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c101350 0x7f250c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f25108f1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f250c1069c0 0x7f250c1a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f25108f1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f250c110e00 con 0x7f250c101350 2024-09-11T22:36:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f25108f1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f250c110c80 con 0x7f250c1069c0 2024-09-11T22:36:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f25108f1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f250c110f80 con 0x7f250c1009b0 2024-09-11T22:36:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f2509d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c101350 0x7f250c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f2509d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c101350 0x7f250c199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38590/0 (socket says 172.21.15.5:38590) 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f2509d9b640 1 -- 172.21.15.5:0/3831313531 learned_addr learned my addr 172.21.15.5:0/3831313531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f250ad9d640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f250c1069c0 0x7f250c1a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f250a59c640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f250c1009b0 0x7f250c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f2509d9b640 1 -- 172.21.15.5:0/3831313531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f250c1009b0 msgr2=0x7f250c199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f2509d9b640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f250c1009b0 0x7f250c199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f2509d9b640 1 -- 172.21.15.5:0/3831313531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f250c1069c0 msgr2=0x7f250c1a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f2509d9b640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f250c1069c0 0x7f250c1a41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f2509d9b640 1 -- 172.21.15.5:0/3831313531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c1a4710 con 0x7f250c101350 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.057+0000 7f250a59c640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f250c1009b0 0x7f250c199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.058+0000 7f2509d9b640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c101350 0x7f250c199690 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f250000d950 tx=0x7f250000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.058+0000 7f24fb7fe640 1 -- 172.21.15.5:0/3831313531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500017070 con 0x7f250c101350 2024-09-11T22:36:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.058+0000 7f25108f1640 1 -- 172.21.15.5:0/3831313531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c1a48e0 con 0x7f250c101350 2024-09-11T22:36:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.058+0000 7f24fb7fe640 1 -- 172.21.15.5:0/3831313531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2500009c30 con 0x7f250c101350 2024-09-11T22:36:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.058+0000 7f24fb7fe640 1 -- 172.21.15.5:0/3831313531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250000f6b0 con 0x7f250c101350 2024-09-11T22:36:11.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.058+0000 7f25108f1640 1 -- 172.21.15.5:0/3831313531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c1a4e30 con 0x7f250c101350 2024-09-11T22:36:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.060+0000 7f24fb7fe640 1 -- 172.21.15.5:0/3831313531 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f25000109d0 con 0x7f250c101350 2024-09-11T22:36:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.060+0000 7f251368e740 1 Processor -- start 2024-09-11T22:36:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f251368e740 1 -- start start 2024-09-11T22:36:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f251368e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22eaf0 0x564a3f22eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f251368e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22f410 0x564a3f239770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f251368e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f23c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f251368e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a3f1932f0 con 0x564a3f22eaf0 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f251368e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564a3f13b270 con 0x564a3f239cb0 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f251368e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a3f18f2b0 con 0x564a3f22f410 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f24fb7fe640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24f40a2da0 0x7f24f40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f250ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22eaf0 0x564a3f22eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f250ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22eaf0 0x564a3f22eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38594/0 (socket says 172.21.15.5:38594) 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f250ad9d640 1 -- 172.21.15.5:0/4204025966 learned_addr learned my addr 172.21.15.5:0/4204025966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f250ad9d640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24f40a2da0 0x7f24f40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f2509d9b640 1 --2- 172.21.15.5:0/4204025966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f23c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f24fb7fe640 1 -- 172.21.15.5:0/3831313531 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f25000cf1e0 con 0x7f250c101350 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f250a59c640 1 --2- 172.21.15.5:0/4204025966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22f410 0x564a3f239770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f250ad9d640 1 -- 172.21.15.5:0/4204025966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22f410 msgr2=0x564a3f239770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f250ad9d640 1 --2- 172.21.15.5:0/4204025966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22f410 0x564a3f239770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f250ad9d640 1 -- 172.21.15.5:0/4204025966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 msgr2=0x564a3f23c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f250ad9d640 1 --2- 172.21.15.5:0/4204025966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f23c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.061+0000 7f250ad9d640 1 -- 172.21.15.5:0/4204025966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a3f13a340 con 0x564a3f22eaf0 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.062+0000 7f250ad9d640 1 --2- 172.21.15.5:0/4204025966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22eaf0 0x564a3f22eed0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f2504011a00 tx=0x7f2504011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caeb1e577880ec6d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.062+0000 7f250ad9d640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24f40a2da0 0x7f24f40a5260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f250400f790 tx=0x7f250401e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.062+0000 7f24f8ff9640 1 -- 172.21.15.5:0/4204025966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504004230 con 0x564a3f22eaf0 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.062+0000 7f24f8ff9640 1 -- 172.21.15.5:0/4204025966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2504011560 con 0x564a3f22eaf0 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.062+0000 7f24f8ff9640 1 -- 172.21.15.5:0/4204025966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25040038e0 con 0x564a3f22eaf0 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 -- 172.21.15.5:0/4204025966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22eaf0 msgr2=0x564a3f22eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 --2- 172.21.15.5:0/4204025966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22eaf0 0x564a3f22eed0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f2504011a00 tx=0x7f2504011ec0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 -- 172.21.15.5:0/4204025966 shutdown_connections 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 --2- 172.21.15.5:0/4204025966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f23c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 --2- 172.21.15.5:0/4204025966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22f410 0x564a3f239770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 --2- 172.21.15.5:0/4204025966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22eaf0 0x564a3f22eed0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 -- 172.21.15.5:0/4204025966 >> 172.21.15.5:0/4204025966 conn(0x564a3f22b060 msgr2=0x564a3f22b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 -- 172.21.15.5:0/4204025966 shutdown_connections 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 -- 172.21.15.5:0/4204025966 wait complete. 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 -- start start 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 Processor -- start 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f22eaf0 0x564a3f236bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f237100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f239cb0 0x564a3f237640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564a3f23c880 con 0x564a3f22f410 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564a3f11c290 con 0x564a3f22eaf0 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.063+0000 7f251368e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564a3f23ca00 con 0x564a3f239cb0 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f251368e740 10 client.0 _ll_register_callbacks cb 0x564a3f14d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f2509d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f237100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f2509d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f237100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38608/0 (socket says 172.21.15.5:38608) 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f2509d9b640 1 -- 172.21.15.5:0/2767891328 learned_addr learned my addr 172.21.15.5:0/2767891328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f250ad9d640 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f239cb0 0x564a3f237640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f250a59c640 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f22eaf0 0x564a3f236bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f2509d9b640 1 -- 172.21.15.5:0/2767891328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f239cb0 msgr2=0x564a3f237640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f2509d9b640 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f239cb0 0x564a3f237640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f2509d9b640 1 -- 172.21.15.5:0/2767891328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f22eaf0 msgr2=0x564a3f236bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f2509d9b640 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f22eaf0 0x564a3f236bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f2509d9b640 1 -- 172.21.15.5:0/2767891328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3eea6210 con 0x564a3f22f410 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f2509d9b640 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f237100 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f25000cc890 tx=0x7f2500010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5a44e0a1582263 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f24f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f24f1ffb640 1 -- 172.21.15.5:0/2767891328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500020040 con 0x564a3f22f410 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f251368e740 1 -- 172.21.15.5:0/2767891328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2504011560 con 0x564a3f22f410 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f24f1ffb640 1 -- 172.21.15.5:0/2767891328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f2500084b40 con 0x564a3f22f410 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f24f1ffb640 1 -- 172.21.15.5:0/2767891328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f25000cccb0 con 0x564a3f22f410 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f251368e740 10 client.19797 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:11.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.064+0000 7f251368e740 1 -- 172.21.15.5:0/2767891328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564a3ef72620 con 0x564a3f22f410 2024-09-11T22:36:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f24f1ffb640 1 -- 172.21.15.5:0/2767891328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2504011560 con 0x564a3f22f410 2024-09-11T22:36:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f24f1ffb640 1 -- 172.21.15.5:0/2767891328 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f250009aa50 con 0x564a3f22f410 2024-09-11T22:36:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f251368e740 10 client.19797 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f251368e740 10 client.19797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f251368e740 20 client.19797 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f251368e740 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3f249c70 0x564a3f24c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f251368e740 4 client.19797 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f251368e740 1 -- 172.21.15.5:0/2767891328 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a3f1a0a20 con 0x564a3f249c70 2024-09-11T22:36:11.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f250a59c640 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3f249c70 0x564a3f24c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f250a59c640 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3f249c70 0x564a3f24c0b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab2b0e2482c7d04 server_cookie=26d63297d5ced16d in_seq=0 out_seq=0 2024-09-11T22:36:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.065+0000 7f24f1ffb640 10 client.19797 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f24f1ffb640 1 -- 172.21.15.5:0/2767891328 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a3f1a0a20 con 0x564a3f249c70 2024-09-11T22:36:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f24f1ffb640 10 client.19797 handle_command_reply: tid=0 2024-09-11T22:36:11.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f251368e740 1 client.19797 shutdown 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f251368e740 1 -- 172.21.15.5:0/2767891328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 msgr2=0x564a3f237100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f251368e740 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f237100 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f25000cc890 tx=0x7f2500010d70 comp rx=0 tx=0).stop 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f251368e740 1 -- 172.21.15.5:0/2767891328 shutdown_connections 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f251368e740 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3f249c70 0x564a3f24c0b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f251368e740 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f239cb0 0x564a3f237640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f251368e740 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f237100 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f24f1ffb640 0 client.19797 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f251368e740 1 --2- 172.21.15.5:0/2767891328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f22eaf0 0x564a3f236bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f251368e740 1 -- 172.21.15.5:0/2767891328 >> 172.21.15.5:0/2767891328 conn(0x564a3f22b060 msgr2=0x564a3f22a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.066+0000 7f251368e740 1 -- 172.21.15.5:0/2767891328 shutdown_connections 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.067+0000 7f251368e740 1 -- 172.21.15.5:0/2767891328 wait complete. 2024-09-11T22:36:11.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.067+0000 7f251368e740 20 client.19797 trim_cache size 0 max 16384 2024-09-11T22:36:11.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:36:11.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:11.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:11.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:11.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:11.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:11.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:11.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:11.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:11.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:11.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:11.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:36:11.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:11.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:11.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:10 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f251368e740 1 Processor -- start 2024-09-11T22:36:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f251368e740 1 -- start start 2024-09-11T22:36:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f251368e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22eaf0 0x564a3f237470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f251368e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2519c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f251368e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f251f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f251368e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1adab0 con 0x564a3f22f410 2024-09-11T22:36:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f251368e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1af6e0 con 0x564a3f239cb0 2024-09-11T22:36:11.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f251368e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1ae180 con 0x564a3f22eaf0 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f2509d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f2509d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38614/0 (socket says 172.21.15.5:38614) 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.081+0000 7f2509d9b640 1 -- 172.21.15.5:0/344985650 learned_addr learned my addr 172.21.15.5:0/344985650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f250a59c640 1 --2- 172.21.15.5:0/344985650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22eaf0 0x564a3f237470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f250ad9d640 1 --2- 172.21.15.5:0/344985650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f251f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f2509d9b640 1 -- 172.21.15.5:0/344985650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22eaf0 msgr2=0x564a3f237470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f2509d9b640 1 --2- 172.21.15.5:0/344985650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22eaf0 0x564a3f237470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f2509d9b640 1 -- 172.21.15.5:0/344985650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 msgr2=0x564a3f251f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f2509d9b640 1 --2- 172.21.15.5:0/344985650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f251f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f2509d9b640 1 -- 172.21.15.5:0/344985650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a3f23c9d0 con 0x564a3f22f410 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f2509d9b640 1 --2- 172.21.15.5:0/344985650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2519c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f2500009610 tx=0x7f2500005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ddb0e01b991b52 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f24f3fff640 1 -- 172.21.15.5:0/344985650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500097040 con 0x564a3f22f410 2024-09-11T22:36:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f24f3fff640 1 -- 172.21.15.5:0/344985650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2500009de0 con 0x564a3f22f410 2024-09-11T22:36:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f24f3fff640 1 -- 172.21.15.5:0/344985650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250009aa50 con 0x564a3f22f410 2024-09-11T22:36:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f251368e740 1 -- 172.21.15.5:0/344985650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 msgr2=0x564a3f2519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.082+0000 7f251368e740 1 --2- 172.21.15.5:0/344985650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2519c0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f2500009610 tx=0x7f2500005880 comp rx=0 tx=0).stop 2024-09-11T22:36:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 -- 172.21.15.5:0/344985650 shutdown_connections 2024-09-11T22:36:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 --2- 172.21.15.5:0/344985650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f251f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 --2- 172.21.15.5:0/344985650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2519c0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 --2- 172.21.15.5:0/344985650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22eaf0 0x564a3f237470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 -- 172.21.15.5:0/344985650 >> 172.21.15.5:0/344985650 conn(0x564a3f22b060 msgr2=0x564a3f238c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 -- 172.21.15.5:0/344985650 shutdown_connections 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 -- 172.21.15.5:0/344985650 wait complete. 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 -- start start 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 Processor -- start 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22eaf0 0x564a3f2422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f233d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1af010 con 0x564a3f22f410 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1ae880 con 0x564a3f239cb0 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1abab0 con 0x564a3f22eaf0 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f251368e740 10 client.0 _ll_register_callbacks cb 0x7f25000860b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:11.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f2509d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f2509d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38624/0 (socket says 172.21.15.5:38624) 2024-09-11T22:36:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f2509d9b640 1 -- 172.21.15.5:0/3504278944 learned_addr learned my addr 172.21.15.5:0/3504278944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.083+0000 7f250a59c640 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22eaf0 0x564a3f2422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f250ad9d640 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f233d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f2509d9b640 1 -- 172.21.15.5:0/3504278944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22eaf0 msgr2=0x564a3f2422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f2509d9b640 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22eaf0 0x564a3f2422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f2509d9b640 1 -- 172.21.15.5:0/3504278944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 msgr2=0x564a3f233d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f2509d9b640 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f233d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f2509d9b640 1 -- 172.21.15.5:0/3504278944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a3f234470 con 0x564a3f22f410 2024-09-11T22:36:11.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f250ad9d640 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f233d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f2509d9b640 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2337f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f2500084db0 tx=0x7f2500010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0b858deaaf03bd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f24f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f24f1ffb640 1 -- 172.21.15.5:0/3504278944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250009b600 con 0x564a3f22f410 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f24f1ffb640 1 -- 172.21.15.5:0/3504278944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f25000973a0 con 0x564a3f22f410 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f251368e740 1 -- 172.21.15.5:0/3504278944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2500009de0 con 0x564a3f22f410 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f24f1ffb640 1 -- 172.21.15.5:0/3504278944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2500097040 con 0x564a3f22f410 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f251368e740 10 client.19803 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f251368e740 1 -- 172.21.15.5:0/3504278944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x564a3f2b48b0 con 0x564a3f22f410 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f24f1ffb640 1 -- 172.21.15.5:0/3504278944 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2500009de0 con 0x564a3f22f410 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f24f1ffb640 1 -- 172.21.15.5:0/3504278944 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f2500010460 con 0x564a3f22f410 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f251368e740 10 client.19803 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:11.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f251368e740 10 client.19803 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f251368e740 20 client.19803 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f251368e740 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3f2b4de0 0x564a3f2b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f251368e740 4 client.19803 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:36:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.084+0000 7f251368e740 1 -- 172.21.15.5:0/3504278944 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a3f19ad70 con 0x564a3f2b4de0 2024-09-11T22:36:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.085+0000 7f250a59c640 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3f2b4de0 0x564a3f2b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.085+0000 7f250a59c640 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3f2b4de0 0x564a3f2b7200 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f494ef6eecb07f server_cookie=cdeb39de649ef987 in_seq=0 out_seq=0 2024-09-11T22:36:11.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.085+0000 7f24f1ffb640 10 client.19803 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:11.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f24f1ffb640 1 -- 172.21.15.5:0/3504278944 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a3f19ad70 con 0x564a3f2b4de0 2024-09-11T22:36:11.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f24f1ffb640 10 client.19803 handle_command_reply: tid=0 2024-09-11T22:36:11.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 client.19803 shutdown 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 -- 172.21.15.5:0/3504278944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 msgr2=0x564a3f2337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2337f0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f2500084db0 tx=0x7f2500010cc0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 -- 172.21.15.5:0/3504278944 shutdown_connections 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x564a3f2b4de0 0x564a3f2b7200 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564a3f239cb0 0x564a3f233d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3f22f410 0x564a3f2337f0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 --2- 172.21.15.5:0/3504278944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564a3f22eaf0 0x564a3f2422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 -- 172.21.15.5:0/3504278944 >> 172.21.15.5:0/3504278944 conn(0x564a3f22b060 msgr2=0x564a3f2299a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f24f1ffb640 0 client.19803 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 -- 172.21.15.5:0/3504278944 shutdown_connections 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 1 -- 172.21.15.5:0/3504278944 wait complete. 2024-09-11T22:36:11.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.141+0000 7f251368e740 20 client.19803 trim_cache size 0 max 16384 2024-09-11T22:36:11.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:36:11.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:36:11.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:36:11.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:36:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 -- 172.21.15.5:0/3831313531 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24f40a2da0 msgr2=0x7f24f40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24f40a2da0 0x7f24f40a5260 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f250400f790 tx=0x7f250401e000 comp rx=0 tx=0).stop 2024-09-11T22:36:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 -- 172.21.15.5:0/3831313531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c101350 msgr2=0x7f250c199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c101350 0x7f250c199690 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f250000d950 tx=0x7f250000de10 comp rx=0 tx=0).stop 2024-09-11T22:36:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 -- 172.21.15.5:0/3831313531 shutdown_connections 2024-09-11T22:36:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24f40a2da0 0x7f24f40a5260 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f250c1069c0 0x7f250c1a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f250c101350 0x7f250c199690 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 --2- 172.21.15.5:0/3831313531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f250c1009b0 0x7f250c199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 -- 172.21.15.5:0/3831313531 >> 172.21.15.5:0/3831313531 conn(0x7f250c077990 msgr2=0x7f250c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 -- 172.21.15.5:0/3831313531 shutdown_connections 2024-09-11T22:36:11.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:11.142+0000 7f25108f1640 1 -- 172.21.15.5:0/3831313531 wait complete. 2024-09-11T22:36:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.220+0000 7f6fb7b59640 1 -- 172.21.15.5:0/896814496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fb0106f10 msgr2=0x7f6fb010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.220+0000 7f6fb7b59640 1 --2- 172.21.15.5:0/896814496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fb0106f10 0x7f6fb010ea90 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009ca0 tx=0x7f6fa002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.220+0000 7f6fb7b59640 1 -- 172.21.15.5:0/896814496 shutdown_connections 2024-09-11T22:36:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.220+0000 7f6fb7b59640 1 --2- 172.21.15.5:0/896814496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fb0106f10 0x7f6fb010ea90 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.220+0000 7f6fb7b59640 1 --2- 172.21.15.5:0/896814496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fb0102790 0x7f6fb01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.220+0000 7f6fb7b59640 1 --2- 172.21.15.5:0/896814496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb0101de0 0x7f6fb01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.220+0000 7f6fb7b59640 1 -- 172.21.15.5:0/896814496 >> 172.21.15.5:0/896814496 conn(0x7f6fb00fd8c0 msgr2=0x7f6fb00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.220+0000 7f6fb7b59640 1 -- 172.21.15.5:0/896814496 shutdown_connections 2024-09-11T22:36:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.220+0000 7f6fb7b59640 1 -- 172.21.15.5:0/896814496 wait complete. 2024-09-11T22:36:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb7b59640 1 Processor -- start 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb7b59640 1 -- start start 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb7b59640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb0101de0 0x7f6fb019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb7b59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fb0102790 0x7f6fb019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb7b59640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fb0106f10 0x7f6fb019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb7b59640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb0111f90 con 0x7f6fb0102790 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb7b59640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb0111e10 con 0x7f6fb0101de0 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb7b59640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb0112110 con 0x7f6fb0106f10 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb50f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fb0102790 0x7f6fb019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb50f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fb0102790 0x7f6fb019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38650/0 (socket says 172.21.15.5:38650) 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb50f4640 1 -- 172.21.15.5:0/3954098715 learned_addr learned my addr 172.21.15.5:0/3954098715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb0101de0 0x7f6fb019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb60f6640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fb0106f10 0x7f6fb019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb50f4640 1 -- 172.21.15.5:0/3954098715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fb0106f10 msgr2=0x7f6fb019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fb0106f10 0x7f6fb019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb50f4640 1 -- 172.21.15.5:0/3954098715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb0101de0 msgr2=0x7f6fb019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb0101de0 0x7f6fb019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.221+0000 7f6fb50f4640 1 -- 172.21.15.5:0/3954098715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb019f610 con 0x7f6fb0102790 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.222+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fb0102790 0x7f6fb019aba0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400b4a0 tx=0x7f6fa400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.222+0000 7f6faeffd640 1 -- 172.21.15.5:0/3954098715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4014070 con 0x7f6fb0102790 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.222+0000 7f6fb7b59640 1 -- 172.21.15.5:0/3954098715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb019f900 con 0x7f6fb0102790 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.222+0000 7f6faeffd640 1 -- 172.21.15.5:0/3954098715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa4010af0 con 0x7f6fb0102790 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.222+0000 7f6faeffd640 1 -- 172.21.15.5:0/3954098715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4017d10 con 0x7f6fb0102790 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.222+0000 7f6fb7b59640 1 -- 172.21.15.5:0/3954098715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb01a71e0 con 0x7f6fb0102790 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.224+0000 7f6faeffd640 1 -- 172.21.15.5:0/3954098715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6fa4017070 con 0x7f6fb0102790 2024-09-11T22:36:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb9f32740 1 Processor -- start 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb9f32740 1 -- start start 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e67880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e72140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e72680 0x5588a1e74a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5588a1dcd2e0 con 0x5588a1e67dc0 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5588a1d74ce0 con 0x5588a1e72680 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5588a1dc92a0 con 0x5588a1e674a0 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6faeffd640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f980a2cf0 0x7f6f980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb58f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e72140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb58f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e72140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38656/0 (socket says 172.21.15.5:38656) 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb58f5640 1 -- 172.21.15.5:0/2465776403 learned_addr learned my addr 172.21.15.5:0/2465776403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/2465776403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e72680 0x5588a1e74a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb60f6640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f980a2cf0 0x7f6f980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.225+0000 7f6fb60f6640 1 --2- 172.21.15.5:0/2465776403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e67880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6faeffd640 1 -- 172.21.15.5:0/3954098715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f6fa40ce810 con 0x7f6fb0102790 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6fb58f5640 1 -- 172.21.15.5:0/2465776403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 msgr2=0x5588a1e67880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/2465776403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e67880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6fb58f5640 1 -- 172.21.15.5:0/2465776403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e72680 msgr2=0x5588a1e74a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/2465776403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e72680 0x5588a1e74a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6fb58f5640 1 -- 172.21.15.5:0/2465776403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a1d59440 con 0x5588a1e67dc0 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/2465776403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e72680 0x5588a1e74a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/2465776403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e72140 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800ac20 tx=0x7f6fa800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce762d8327769522 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6fb60f6640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f980a2cf0 0x7f6f980a51b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fa002f540 tx=0x7f6fa002f310 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6f97fff640 1 -- 172.21.15.5:0/2465776403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa801d070 con 0x5588a1e67dc0 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6f97fff640 1 -- 172.21.15.5:0/2465776403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa8018b20 con 0x5588a1e67dc0 2024-09-11T22:36:11.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.226+0000 7f6f97fff640 1 -- 172.21.15.5:0/2465776403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8010b90 con 0x5588a1e67dc0 2024-09-11T22:36:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.227+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2465776403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 msgr2=0x5588a1e72140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.227+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/2465776403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e72140 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800ac20 tx=0x7f6fa800f790 comp rx=0 tx=0).stop 2024-09-11T22:36:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.227+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2465776403 shutdown_connections 2024-09-11T22:36:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.227+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/2465776403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e72680 0x5588a1e74a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.227+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/2465776403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e72140 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.227+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/2465776403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e67880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.227+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2465776403 >> 172.21.15.5:0/2465776403 conn(0x5588a1e63a30 msgr2=0x5588a1e63e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.227+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2465776403 shutdown_connections 2024-09-11T22:36:11.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.227+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2465776403 wait complete. 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.227+0000 7f6fb9f32740 1 -- start start 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb9f32740 1 Processor -- start 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e674a0 0x5588a1e71f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 0x5588a1e6f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e72680 0x5588a1e6f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5588a1e75250 con 0x5588a1e674a0 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5588a1d88050 con 0x5588a1e67dc0 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5588a1e753d0 con 0x5588a1e72680 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb9f32740 10 client.0 _ll_register_callbacks cb 0x5588a1dc8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb58f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e674a0 0x5588a1e71f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb58f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e674a0 0x5588a1e71f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38668/0 (socket says 172.21.15.5:38668) 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb58f5640 1 -- 172.21.15.5:0/2015444450 learned_addr learned my addr 172.21.15.5:0/2015444450 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 0x5588a1e6f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.228+0000 7f6fb60f6640 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e72680 0x5588a1e6f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6fb58f5640 1 -- 172.21.15.5:0/2015444450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e72680 msgr2=0x5588a1e6f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e72680 0x5588a1e6f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6fb58f5640 1 -- 172.21.15.5:0/2015444450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 msgr2=0x5588a1e6f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 0x5588a1e6f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6fb58f5640 1 -- 172.21.15.5:0/2015444450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a1b3e3c0 con 0x5588a1e674a0 2024-09-11T22:36:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 0x5588a1e6f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e674a0 0x5588a1e71f60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8010690 tx=0x7f6fa800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7e4aa186b38d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6f957fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6f957fa640 1 -- 172.21.15.5:0/2015444450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa801d050 con 0x5588a1e674a0 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6f957fa640 1 -- 172.21.15.5:0/2015444450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f6fa8018790 con 0x5588a1e674a0 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2015444450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa8018b20 con 0x5588a1e674a0 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6f957fa640 1 -- 172.21.15.5:0/2015444450 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f6fa8031050 con 0x5588a1e674a0 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6fb9f32740 10 client.19815 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2015444450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5588a1e80bc0 con 0x5588a1e674a0 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6f957fa640 1 -- 172.21.15.5:0/2015444450 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa8018b20 con 0x5588a1e674a0 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.229+0000 7f6f957fa640 1 -- 172.21.15.5:0/2015444450 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f6fa80046c0 con 0x5588a1e674a0 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.230+0000 7f6fb9f32740 10 client.19815 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.230+0000 7f6fb9f32740 10 client.19815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.230+0000 7f6fb9f32740 20 client.19815 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.230+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5588a1e82430 0x5588a1e848f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.230+0000 7f6fb9f32740 4 client.19815 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:11.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.230+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2015444450 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588a1dda850 con 0x5588a1e82430 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.230+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5588a1e82430 0x5588a1e848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.230+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5588a1e82430 0x5588a1e848f0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1efafa6162d7bb1 server_cookie=a481539c3b2db506 in_seq=0 out_seq=0 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.230+0000 7f6f957fa640 10 client.19815 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6f957fa640 1 -- 172.21.15.5:0/2015444450 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588a1dda850 con 0x5588a1e82430 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6f957fa640 10 client.19815 handle_command_reply: tid=0 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 client.19815 shutdown 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2015444450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e674a0 msgr2=0x5588a1e71f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e674a0 0x5588a1e71f60 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8010690 tx=0x7f6fa800af90 comp rx=0 tx=0).stop 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2015444450 shutdown_connections 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5588a1e82430 0x5588a1e848f0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e72680 0x5588a1e6f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 0x5588a1e6f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/2015444450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e674a0 0x5588a1e71f60 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2015444450 >> 172.21.15.5:0/2015444450 conn(0x5588a1e63a30 msgr2=0x5588a1e69710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6f957fa640 0 client.19815 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2015444450 shutdown_connections 2024-09-11T22:36:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.231+0000 7f6fb9f32740 1 -- 172.21.15.5:0/2015444450 wait complete. 2024-09-11T22:36:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.232+0000 7f6fb9f32740 20 client.19815 trim_cache size 0 max 16384 2024-09-11T22:36:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb9f32740 1 Processor -- start 2024-09-11T22:36:11.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb9f32740 1 -- start start 2024-09-11T22:36:11.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e63a30 0x5588a1e7e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e8a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e8a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01afe40 con 0x5588a1e67dc0 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01b1a70 con 0x5588a1e63a30 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01b0510 con 0x5588a1e674a0 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb60f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e8a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb60f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e8a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38678/0 (socket says 172.21.15.5:38678) 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb60f6640 1 -- 172.21.15.5:0/923347161 learned_addr learned my addr 172.21.15.5:0/923347161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.249+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/923347161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e63a30 0x5588a1e7e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/923347161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e8a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6fb60f6640 1 -- 172.21.15.5:0/923347161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 msgr2=0x5588a1e8a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6fb60f6640 1 --2- 172.21.15.5:0/923347161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e8a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6fb60f6640 1 -- 172.21.15.5:0/923347161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e63a30 msgr2=0x5588a1e7e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6fb60f6640 1 --2- 172.21.15.5:0/923347161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e63a30 0x5588a1e7e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6fb60f6640 1 -- 172.21.15.5:0/923347161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a1e75360 con 0x5588a1e67dc0 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/923347161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e8a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6fb60f6640 1 --2- 172.21.15.5:0/923347161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e8a740 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009ca0 tx=0x7f6fa003f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8226eeb88b6bbbe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6f977fe640 1 -- 172.21.15.5:0/923347161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa002f810 con 0x5588a1e67dc0 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6f977fe640 1 -- 172.21.15.5:0/923347161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa0036040 con 0x5588a1e67dc0 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.250+0000 7f6f977fe640 1 -- 172.21.15.5:0/923347161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0049450 con 0x5588a1e67dc0 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 -- 172.21.15.5:0/923347161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 msgr2=0x5588a1e8a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/923347161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e8a740 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0009ca0 tx=0x7f6fa003f070 comp rx=0 tx=0).stop 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 -- 172.21.15.5:0/923347161 shutdown_connections 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/923347161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e67dc0 0x5588a1e8a740 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/923347161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e8a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/923347161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e63a30 0x5588a1e7e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 -- 172.21.15.5:0/923347161 >> 172.21.15.5:0/923347161 conn(0x5588a1e6f8b0 msgr2=0x5588a1e6dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 -- 172.21.15.5:0/923347161 shutdown_connections 2024-09-11T22:36:11.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 -- 172.21.15.5:0/923347161 wait complete. 2024-09-11T22:36:11.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 -- start start 2024-09-11T22:36:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.251+0000 7f6fb9f32740 1 Processor -- start 2024-09-11T22:36:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e63a30 0x5588a1e75640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb9f32740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 0x5588a1e760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01b13a0 con 0x5588a1e63a30 2024-09-11T22:36:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01b0c10 con 0x5588a1e67dc0 2024-09-11T22:36:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb9f32740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb01ade40 con 0x5588a1e674a0 2024-09-11T22:36:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb9f32740 10 client.0 _ll_register_callbacks cb 0x7f6fa801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb58f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e63a30 0x5588a1e75640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb58f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e63a30 0x5588a1e75640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38682/0 (socket says 172.21.15.5:38682) 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb58f5640 1 -- 172.21.15.5:0/3856960019 learned_addr learned my addr 172.21.15.5:0/3856960019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb60f6640 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 0x5588a1e760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb58f5640 1 -- 172.21.15.5:0/3856960019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 msgr2=0x5588a1e75b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e75b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb58f5640 1 -- 172.21.15.5:0/3856960019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 msgr2=0x5588a1e760c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 0x5588a1e760c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb58f5640 1 -- 172.21.15.5:0/3856960019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a1eeb410 con 0x5588a1e63a30 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e75b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.252+0000 7f6fb60f6640 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 0x5588a1e760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6fb58f5640 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e63a30 0x5588a1e75640 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800a6f0 tx=0x7f6fa8004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131abf35b5943cdc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6f957fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:11.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6f957fa640 1 -- 172.21.15.5:0/3856960019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa801d070 con 0x5588a1e63a30 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6f957fa640 1 -- 172.21.15.5:0/3856960019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8673+0+0 (secure 0 0 0) 0x7f6fa80189a0 con 0x5588a1e63a30 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6fb9f32740 1 -- 172.21.15.5:0/3856960019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fa0036040 con 0x5588a1e63a30 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6f957fa640 1 -- 172.21.15.5:0/3856960019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f6fa8045050 con 0x5588a1e63a30 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6fb9f32740 10 client.19821 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6fb9f32740 1 -- 172.21.15.5:0/3856960019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5588a1eec690 con 0x5588a1e63a30 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6f957fa640 1 -- 172.21.15.5:0/3856960019 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fa0036040 con 0x5588a1e63a30 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6f957fa640 1 -- 172.21.15.5:0/3856960019 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f6fa8018ba0 con 0x5588a1e63a30 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6fb9f32740 10 client.19821 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6fb9f32740 10 client.19821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6fb9f32740 20 client.19821 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5588a1e72680 0x5588a1eeee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6fb9f32740 4 client.19821 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.253+0000 7f6fb9f32740 1 -- 172.21.15.5:0/3856960019 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5588a1dd4d60 con 0x5588a1e72680 2024-09-11T22:36:11.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.254+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5588a1e72680 0x5588a1eeee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.254+0000 7f6fb50f4640 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5588a1e72680 0x5588a1eeee80 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72b37ca100d4777 server_cookie=d97c705c8b7b3fd0 in_seq=0 out_seq=0 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.254+0000 7f6f957fa640 10 client.19821 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.254+0000 7f6f957fa640 1 -- 172.21.15.5:0/3856960019 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5588a1dd4d60 con 0x5588a1e72680 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.254+0000 7f6f957fa640 10 client.19821 handle_command_reply: tid=0 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 client.19821 shutdown 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 -- 172.21.15.5:0/3856960019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e63a30 msgr2=0x5588a1e75640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e63a30 0x5588a1e75640 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f6fa800a6f0 tx=0x7f6fa8004870 comp rx=0 tx=0).stop 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 -- 172.21.15.5:0/3856960019 shutdown_connections 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5588a1e72680 0x5588a1eeee80 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5588a1e67dc0 0x5588a1e760c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5588a1e674a0 0x5588a1e75b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6f957fa640 0 client.19821 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:11.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 --2- 172.21.15.5:0/3856960019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588a1e63a30 0x5588a1e75640 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 -- 172.21.15.5:0/3856960019 >> 172.21.15.5:0/3856960019 conn(0x5588a1e6f8b0 msgr2=0x5588a1e62b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 -- 172.21.15.5:0/3856960019 shutdown_connections 2024-09-11T22:36:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 1 -- 172.21.15.5:0/3856960019 wait complete. 2024-09-11T22:36:11.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb9f32740 20 client.19821 trim_cache size 0 max 16384 2024-09-11T22:36:11.261 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:36:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:36:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:36:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "09ce1aea-442b-4b7d-9017-7157c7f1fdf4": { 2024-09-11T22:36:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:36:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "09ce1aea-442b-4b7d-9017-7157c7f1fdf4", 2024-09-11T22:36:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:36:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:11.262 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:36:11.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb7b59640 1 -- 172.21.15.5:0/3954098715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f980a2cf0 msgr2=0x7f6f980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb7b59640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f980a2cf0 0x7f6f980a51b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fa002f540 tx=0x7f6fa002f310 comp rx=0 tx=0).stop 2024-09-11T22:36:11.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb7b59640 1 -- 172.21.15.5:0/3954098715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fb0102790 msgr2=0x7f6fb019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:11.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.255+0000 7f6fb7b59640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fb0102790 0x7f6fb019aba0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400b4a0 tx=0x7f6fa400b960 comp rx=0 tx=0).stop 2024-09-11T22:36:11.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.256+0000 7f6fb7b59640 1 -- 172.21.15.5:0/3954098715 shutdown_connections 2024-09-11T22:36:11.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.256+0000 7f6fb7b59640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fb0106f10 0x7f6fb019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.256+0000 7f6fb7b59640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fb0102790 0x7f6fb019aba0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.256+0000 7f6fb7b59640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f980a2cf0 0x7f6f980a51b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.256+0000 7f6fb7b59640 1 --2- 172.21.15.5:0/3954098715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6fb0101de0 0x7f6fb019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:11.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.256+0000 7f6fb7b59640 1 -- 172.21.15.5:0/3954098715 >> 172.21.15.5:0/3954098715 conn(0x7f6fb00fd8c0 msgr2=0x7f6fb00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:11.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.256+0000 7f6fb7b59640 1 -- 172.21.15.5:0/3954098715 shutdown_connections 2024-09-11T22:36:11.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:11.256+0000 7f6fb7b59640 1 -- 172.21.15.5:0/3954098715 wait complete. 2024-09-11T22:36:11.631 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 19.37 MB/sec warmup 29 sec latency 15695.922 ms 2024-09-11T22:36:11.867 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:11 smithi172 ceph-mon[35461]: pgmap v445: 97 pgs: 97 active+clean; 207 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-09-11T22:36:11.957 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:11 smithi005 ceph-mon[30943]: pgmap v445: 97 pgs: 97 active+clean; 207 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-09-11T22:36:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:11 smithi159 ceph-mon[35534]: pgmap v445: 97 pgs: 97 active+clean; 207 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-09-11T22:36:12.273 INFO:tasks.cephfs.filesystem:scrub status for tag:09ce1aea-442b-4b7d-9017-7157c7f1fdf4 - {'path': '/', 'tag': '09ce1aea-442b-4b7d-9017-7157c7f1fdf4', 'options': 'recursive,force'} 2024-09-11T22:36:12.632 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 18.72 MB/sec warmup 30 sec latency 16696.046 ms 2024-09-11T22:36:13.632 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 18.12 MB/sec warmup 31 sec latency 17696.168 ms 2024-09-11T22:36:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:13 smithi005 ceph-mon[30943]: pgmap v446: 97 pgs: 97 active+clean; 198 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-09-11T22:36:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:13 smithi005 ceph-mon[30943]: osdmap e109: 12 total, 12 up, 12 in 2024-09-11T22:36:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:13 smithi159 ceph-mon[35534]: pgmap v446: 97 pgs: 97 active+clean; 198 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-09-11T22:36:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:13 smithi159 ceph-mon[35534]: osdmap e109: 12 total, 12 up, 12 in 2024-09-11T22:36:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:13 smithi172 ceph-mon[35461]: pgmap v446: 97 pgs: 97 active+clean; 198 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-09-11T22:36:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:13 smithi172 ceph-mon[35461]: osdmap e109: 12 total, 12 up, 12 in 2024-09-11T22:36:14.632 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 17.55 MB/sec warmup 32 sec latency 18696.236 ms 2024-09-11T22:36:15.632 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 17.02 MB/sec warmup 33 sec latency 19696.570 ms 2024-09-11T22:36:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:15 smithi005 ceph-mon[30943]: pgmap v448: 97 pgs: 97 active+clean; 198 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-11T22:36:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:15 smithi159 ceph-mon[35534]: pgmap v448: 97 pgs: 97 active+clean; 198 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-11T22:36:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:15 smithi172 ceph-mon[35461]: pgmap v448: 97 pgs: 97 active+clean; 198 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-09-11T22:36:16.632 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 16.52 MB/sec warmup 34 sec latency 20696.677 ms 2024-09-11T22:36:17.632 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 16.05 MB/sec warmup 35 sec latency 21696.784 ms 2024-09-11T22:36:17.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:17 smithi005 ceph-mon[30943]: pgmap v449: 97 pgs: 97 active+clean; 198 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-09-11T22:36:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:17 smithi159 ceph-mon[35534]: pgmap v449: 97 pgs: 97 active+clean; 198 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-09-11T22:36:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:17 smithi172 ceph-mon[35461]: pgmap v449: 97 pgs: 97 active+clean; 198 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-09-11T22:36:18.632 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 15.60 MB/sec warmup 36 sec latency 22696.914 ms 2024-09-11T22:36:19.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 15.18 MB/sec warmup 37 sec latency 23697.041 ms 2024-09-11T22:36:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:19 smithi005 ceph-mon[30943]: pgmap v450: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 12 MiB/s wr, 193 op/s 2024-09-11T22:36:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:19 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:36:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:19 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:36:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:19 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:36:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:19 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:36:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:19 smithi159 ceph-mon[35534]: pgmap v450: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 12 MiB/s wr, 193 op/s 2024-09-11T22:36:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:19 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:36:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:19 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:36:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:19 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:36:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:19 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:36:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:19 smithi172 ceph-mon[35461]: pgmap v450: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 12 MiB/s wr, 193 op/s 2024-09-11T22:36:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:19 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:36:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:19 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:36:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:19 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:36:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:19 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:36:20.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 14.78 MB/sec warmup 38 sec latency 24697.159 ms 2024-09-11T22:36:20.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:20 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:36:20.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:20 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:36:20.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:20 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:36:21.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:36:21.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 14.40 MB/sec warmup 39 sec latency 25697.211 ms 2024-09-11T22:36:21.786 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:21 smithi005 ceph-mon[30943]: pgmap v451: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 12 MiB/s wr, 193 op/s 2024-09-11T22:36:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:21 smithi159 ceph-mon[35534]: pgmap v451: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 12 MiB/s wr, 193 op/s 2024-09-11T22:36:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:21 smithi172 ceph-mon[35461]: pgmap v451: 97 pgs: 97 active+clean; 307 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 12 MiB/s wr, 193 op/s 2024-09-11T22:36:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.288+0000 7f24ecc26640 1 -- 172.21.15.5:0/815387058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8101420 msgr2=0x7f24e810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.288+0000 7f24ecc26640 1 --2- 172.21.15.5:0/815387058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8101420 0x7f24e810d950 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009930 tx=0x7f24e002a420 comp rx=0 tx=0).stop 2024-09-11T22:36:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.288+0000 7f24ecc26640 1 -- 172.21.15.5:0/815387058 shutdown_connections 2024-09-11T22:36:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.288+0000 7f24ecc26640 1 --2- 172.21.15.5:0/815387058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8101420 0x7f24e810d950 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.288+0000 7f24ecc26640 1 --2- 172.21.15.5:0/815387058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e8100a80 0x7f24e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.288+0000 7f24ecc26640 1 --2- 172.21.15.5:0/815387058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24e8106a80 0x7f24e8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.288+0000 7f24ecc26640 1 -- 172.21.15.5:0/815387058 >> 172.21.15.5:0/815387058 conn(0x7f24e80fc7a0 msgr2=0x7f24e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.288+0000 7f24ecc26640 1 -- 172.21.15.5:0/815387058 shutdown_connections 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.288+0000 7f24ecc26640 1 -- 172.21.15.5:0/815387058 wait complete. 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24ecc26640 1 Processor -- start 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24ecc26640 1 -- start start 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24ecc26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8100a80 0x7f24e819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24ecc26640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24e8101420 0x7f24e819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24ecc26640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e8106a80 0x7f24e819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24ecc26640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8110ea0 con 0x7f24e8100a80 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24ecc26640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8110d20 con 0x7f24e8101420 2024-09-11T22:36:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24ecc26640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f24e8111020 con 0x7f24e8106a80 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24e659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8100a80 0x7f24e819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24e659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8100a80 0x7f24e819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51732/0 (socket says 172.21.15.5:51732) 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.289+0000 7f24e659c640 1 -- 172.21.15.5:0/204603087 learned_addr learned my addr 172.21.15.5:0/204603087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24e6d9d640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e8106a80 0x7f24e819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24e5d9b640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24e8101420 0x7f24e819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24e659c640 1 -- 172.21.15.5:0/204603087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e8106a80 msgr2=0x7f24e819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24e659c640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e8106a80 0x7f24e819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24e659c640 1 -- 172.21.15.5:0/204603087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24e8101420 msgr2=0x7f24e819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24e659c640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24e8101420 0x7f24e819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24e659c640 1 -- 172.21.15.5:0/204603087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24e819f5f0 con 0x7f24e8100a80 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24e5d9b640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24e8101420 0x7f24e819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24e659c640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8100a80 0x7f24e819a640 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f24d800eab0 tx=0x7f24d800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24df7fe640 1 -- 172.21.15.5:0/204603087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d800ce40 con 0x7f24e8100a80 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24df7fe640 1 -- 172.21.15.5:0/204603087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24d8011950 con 0x7f24e8100a80 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24ecc26640 1 -- 172.21.15.5:0/204603087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24e819f8e0 con 0x7f24e8100a80 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24df7fe640 1 -- 172.21.15.5:0/204603087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d8010ae0 con 0x7f24e8100a80 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.290+0000 7f24ecc26640 1 -- 172.21.15.5:0/204603087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24e81a71c0 con 0x7f24e8100a80 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.291+0000 7f24ecc26640 1 -- 172.21.15.5:0/204603087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24e8100ee0 con 0x7f24e8100a80 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.292+0000 7f24df7fe640 1 -- 172.21.15.5:0/204603087 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f24d8011b00 con 0x7f24e8100a80 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.293+0000 7f24df7fe640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24d40a2cf0 0x7f24d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.293+0000 7f24e5d9b640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24d40a2cf0 0x7f24d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.293+0000 7f24df7fe640 1 -- 172.21.15.5:0/204603087 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f24d80cfc10 con 0x7f24e8100a80 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.294+0000 7f24e5d9b640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24d40a2cf0 0x7f24d40a51b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24e810d740 tx=0x7f24d0008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:22.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.297+0000 7f24df7fe640 1 -- 172.21.15.5:0/204603087 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24d8016030 con 0x7f24e8100a80 2024-09-11T22:36:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.509+0000 7f24ecc26640 1 -- 172.21.15.5:0/204603087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24e819b9a0 con 0x7f24e8100a80 2024-09-11T22:36:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.510+0000 7f24df7fe640 1 -- 172.21.15.5:0/204603087 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11975 (secure 0 0 0) 0x7f24d809c760 con 0x7f24e8100a80 2024-09-11T22:36:22.512 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:36:22.512 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"btime":"2024-09-11T22:35:49:548013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:35:48.842139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:36:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 50 2024-09-11T22:36:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 -- 172.21.15.5:0/204603087 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24d40a2cf0 msgr2=0x7f24d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24d40a2cf0 0x7f24d40a51b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24e810d740 tx=0x7f24d0008040 comp rx=0 tx=0).stop 2024-09-11T22:36:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 -- 172.21.15.5:0/204603087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8100a80 msgr2=0x7f24e819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8100a80 0x7f24e819a640 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f24d800eab0 tx=0x7f24d800ef70 comp rx=0 tx=0).stop 2024-09-11T22:36:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 -- 172.21.15.5:0/204603087 shutdown_connections 2024-09-11T22:36:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24e8106a80 0x7f24e819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f24d40a2cf0 0x7f24d40a51b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24e8101420 0x7f24e819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 --2- 172.21.15.5:0/204603087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24e8100a80 0x7f24e819a640 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 -- 172.21.15.5:0/204603087 >> 172.21.15.5:0/204603087 conn(0x7f24e80fc7a0 msgr2=0x7f24e810c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 -- 172.21.15.5:0/204603087 shutdown_connections 2024-09-11T22:36:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:22.512+0000 7f24ecc26640 1 -- 172.21.15.5:0/204603087 wait complete. 2024-09-11T22:36:22.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 14.04 MB/sec warmup 40 sec latency 26697.264 ms 2024-09-11T22:36:22.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:22 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/204603087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:22.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:22 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/204603087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:22.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:22 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/204603087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:23.314 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 91bd8d7f --release --await' 2024-09-11T22:36:23.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 91bd8d7f --release --await 2024-09-11T22:36:23.580 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:23.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7236 13.70 MB/sec warmup 41 sec latency 27697.373 ms 2024-09-11T22:36:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:23 smithi005 ceph-mon[30943]: pgmap v452: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 15 MiB/s wr, 123 op/s 2024-09-11T22:36:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:23 smithi159 ceph-mon[35534]: pgmap v452: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 15 MiB/s wr, 123 op/s 2024-09-11T22:36:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:23 smithi172 ceph-mon[35461]: pgmap v452: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 15 MiB/s wr, 123 op/s 2024-09-11T22:36:24.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.137+0000 7f0ef3adf640 1 -- 172.21.15.5:0/1468608926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eec101410 msgr2=0x7f0eec10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.137+0000 7f0ef3adf640 1 --2- 172.21.15.5:0/1468608926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eec101410 0x7f0eec10d940 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009cd0 tx=0x7f0ee402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.138+0000 7f0ef3adf640 1 -- 172.21.15.5:0/1468608926 shutdown_connections 2024-09-11T22:36:24.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.138+0000 7f0ef3adf640 1 --2- 172.21.15.5:0/1468608926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eec101410 0x7f0eec10d940 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.138+0000 7f0ef3adf640 1 --2- 172.21.15.5:0/1468608926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eec100a70 0x7f0eec100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.138+0000 7f0ef3adf640 1 --2- 172.21.15.5:0/1468608926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eec106a70 0x7f0eec106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.138+0000 7f0ef3adf640 1 -- 172.21.15.5:0/1468608926 >> 172.21.15.5:0/1468608926 conn(0x7f0eec0fc790 msgr2=0x7f0eec0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.138+0000 7f0ef3adf640 1 -- 172.21.15.5:0/1468608926 shutdown_connections 2024-09-11T22:36:24.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.138+0000 7f0ef3adf640 1 -- 172.21.15.5:0/1468608926 wait complete. 2024-09-11T22:36:24.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.139+0000 7f0ef3adf640 1 Processor -- start 2024-09-11T22:36:24.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.139+0000 7f0ef3adf640 1 -- start start 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.139+0000 7f0ef3adf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eec100a70 0x7f0eec19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.139+0000 7f0ef3adf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eec101410 0x7f0eec19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.139+0000 7f0ef3adf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eec106a70 0x7f0eec19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.139+0000 7f0ef3adf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec10fdc0 con 0x7f0eec106a70 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.139+0000 7f0ef3adf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec10fc40 con 0x7f0eec100a70 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.139+0000 7f0ef3adf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec10ff40 con 0x7f0eec101410 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef207c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eec106a70 0x7f0eec19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef107a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eec101410 0x7f0eec19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef187b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eec100a70 0x7f0eec19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef107a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eec101410 0x7f0eec19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:48374/0 (socket says 172.21.15.5:48374) 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef107a640 1 -- 172.21.15.5:0/327874559 learned_addr learned my addr 172.21.15.5:0/327874559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:24.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef207c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eec106a70 0x7f0eec19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51752/0 (socket says 172.21.15.5:51752) 2024-09-11T22:36:24.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef107a640 1 -- 172.21.15.5:0/327874559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eec100a70 msgr2=0x7f0eec19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef107a640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eec100a70 0x7f0eec19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef107a640 1 -- 172.21.15.5:0/327874559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eec106a70 msgr2=0x7f0eec19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef107a640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eec106a70 0x7f0eec19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef107a640 1 -- 172.21.15.5:0/327874559 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eec19f5c0 con 0x7f0eec101410 2024-09-11T22:36:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ef107a640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eec101410 0x7f0eec19ab50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0097d0 tx=0x7f0edc009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.140+0000 7f0ee2ffd640 1 -- 172.21.15.5:0/327874559 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00eab0 con 0x7f0eec101410 2024-09-11T22:36:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.141+0000 7f0ef3adf640 1 -- 172.21.15.5:0/327874559 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eec19f8b0 con 0x7f0eec101410 2024-09-11T22:36:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.141+0000 7f0ee2ffd640 1 -- 172.21.15.5:0/327874559 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0edc019070 con 0x7f0eec101410 2024-09-11T22:36:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.141+0000 7f0ee2ffd640 1 -- 172.21.15.5:0/327874559 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc013360 con 0x7f0eec101410 2024-09-11T22:36:24.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.141+0000 7f0ef3adf640 1 -- 172.21.15.5:0/327874559 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eec195f60 con 0x7f0eec101410 2024-09-11T22:36:24.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.144+0000 7f0ee2ffd640 1 -- 172.21.15.5:0/327874559 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0edc013070 con 0x7f0eec101410 2024-09-11T22:36:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.144+0000 7f0ef5eb8740 1 Processor -- start 2024-09-11T22:36:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.144+0000 7f0ef5eb8740 1 -- start start 2024-09-11T22:36:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4dab440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 0x55e0c4db5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ee2ffd640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ed80a2cf0 0x7f0ed80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4db8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e0c4d10620 con 0x55e0c4dab060 2024-09-11T22:36:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e0c4cb7e50 con 0x55e0c4dab980 2024-09-11T22:36:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e0c4d0c5e0 con 0x55e0c4db6240 2024-09-11T22:36:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef207c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4dab440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef187b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 0x55e0c4db5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ee2ffd640 1 -- 172.21.15.5:0/327874559 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f0edc0cef00 con 0x7f0eec101410 2024-09-11T22:36:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef207c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4dab440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51762/0 (socket says 172.21.15.5:51762) 2024-09-11T22:36:24.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef187b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 0x55e0c4db5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51612/0 (socket says 172.21.15.5:51612) 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef207c640 1 -- 172.21.15.5:0/1203132960 learned_addr learned my addr 172.21.15.5:0/1203132960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef107a640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ed80a2cf0 0x7f0ed80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef207c640 1 --2- 172.21.15.5:0/1203132960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4db8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef207c640 1 -- 172.21.15.5:0/1203132960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 msgr2=0x55e0c4db8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef207c640 1 --2- 172.21.15.5:0/1203132960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4db8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef207c640 1 -- 172.21.15.5:0/1203132960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 msgr2=0x55e0c4db5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef207c640 1 --2- 172.21.15.5:0/1203132960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 0x55e0c4db5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.145+0000 7f0ef207c640 1 -- 172.21.15.5:0/1203132960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c4ca8d40 con 0x55e0c4dab060 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.146+0000 7f0ef207c640 1 --2- 172.21.15.5:0/1203132960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4dab440 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009430 tx=0x7f0ee4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2543670a43718b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.146+0000 7f0ef107a640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ed80a2cf0 0x7f0ed80a51b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0cc690 tx=0x7f0edc096fa0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.146+0000 7f0ed7fff640 1 -- 172.21.15.5:0/1203132960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4031670 con 0x55e0c4dab060 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.146+0000 7f0ed7fff640 1 -- 172.21.15.5:0/1203132960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee4036070 con 0x55e0c4dab060 2024-09-11T22:36:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.146+0000 7f0ed7fff640 1 -- 172.21.15.5:0/1203132960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4004270 con 0x55e0c4dab060 2024-09-11T22:36:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1203132960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 msgr2=0x55e0c4dab440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/1203132960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4dab440 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009430 tx=0x7f0ee4009360 comp rx=0 tx=0).stop 2024-09-11T22:36:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1203132960 shutdown_connections 2024-09-11T22:36:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/1203132960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4db8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/1203132960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 0x55e0c4db5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/1203132960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4dab440 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1203132960 >> 172.21.15.5:0/1203132960 conn(0x55e0c4da75f0 msgr2=0x55e0c4da79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1203132960 shutdown_connections 2024-09-11T22:36:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1203132960 wait complete. 2024-09-11T22:36:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 -- start start 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.147+0000 7f0ef5eb8740 1 Processor -- start 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab060 0x55e0c4db3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab980 0x55e0c4db36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4db3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e0c4db8e10 con 0x55e0c4dab980 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e0c4c98f10 con 0x55e0c4dab060 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e0c4db8f90 con 0x55e0c4db6240 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef5eb8740 10 client.0 _ll_register_callbacks cb 0x55e0c4cbcdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef207c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab060 0x55e0c4db3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef207c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab060 0x55e0c4db3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51624/0 (socket says 172.21.15.5:51624) 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef207c640 1 -- 172.21.15.5:0/745643551 learned_addr learned my addr 172.21.15.5:0/745643551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef187b640 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab980 0x55e0c4db36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef207c640 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4db3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef207c640 1 -- 172.21.15.5:0/745643551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 msgr2=0x55e0c4db3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef207c640 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4db3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef207c640 1 -- 172.21.15.5:0/745643551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab980 msgr2=0x55e0c4db36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef207c640 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab980 0x55e0c4db36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef207c640 1 -- 172.21.15.5:0/745643551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c4a1f580 con 0x55e0c4dab060 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.148+0000 7f0ef187b640 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab980 0x55e0c4db36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ef207c640 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab060 0x55e0c4db3180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009ca0 tx=0x7f0ee4003ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6588613da33044ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ed57fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ed57fa640 1 -- 172.21.15.5:0/745643551 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4050030 con 0x55e0c4dab060 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/745643551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee4036070 con 0x55e0c4dab060 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ed57fa640 1 -- 172.21.15.5:0/745643551 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f0ee4031b50 con 0x55e0c4dab060 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ed57fa640 1 -- 172.21.15.5:0/745643551 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0ee4031d50 con 0x55e0c4dab060 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ef5eb8740 10 client.28988 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/745643551 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e0c4ae9960 con 0x55e0c4dab060 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ed57fa640 1 -- 172.21.15.5:0/745643551 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee4036070 con 0x55e0c4dab060 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ed57fa640 1 -- 172.21.15.5:0/745643551 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f0ee4030070 con 0x55e0c4dab060 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ef5eb8740 10 client.28988 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ef5eb8740 10 client.28988 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.149+0000 7f0ef5eb8740 20 client.28988 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.150+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e0c4dc6000 0x55e0c4dc8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.150+0000 7f0ef5eb8740 4 client.28988 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.150+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/745643551 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0c4d1db90 con 0x55e0c4dc6000 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.150+0000 7f0ef187b640 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e0c4dc6000 0x55e0c4dc8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.150+0000 7f0ed57fa640 10 client.28988 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.150+0000 7f0ef187b640 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e0c4dc6000 0x55e0c4dc8420 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c22e87a670e07fa7 server_cookie=4004ae8666aa4cff in_seq=0 out_seq=0 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ed57fa640 1 -- 172.21.15.5:0/745643551 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0c4d1db90 con 0x55e0c4dc6000 2024-09-11T22:36:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ed57fa640 10 client.28988 handle_command_reply: tid=0 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 client.28988 shutdown 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/745643551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab060 msgr2=0x55e0c4db3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab060 0x55e0c4db3180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009ca0 tx=0x7f0ee4003ee0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/745643551 shutdown_connections 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e0c4dc6000 0x55e0c4dc8420 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4db3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab980 0x55e0c4db36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/745643551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab060 0x55e0c4db3180 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/745643551 >> 172.21.15.5:0/745643551 conn(0x55e0c4da75f0 msgr2=0x55e0c4dad320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ed57fa640 0 client.28988 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/745643551 shutdown_connections 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/745643551 wait complete. 2024-09-11T22:36:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.151+0000 7f0ef5eb8740 20 client.28988 trim_cache size 0 max 16384 2024-09-11T22:36:24.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.168+0000 7f0ef5eb8740 1 Processor -- start 2024-09-11T22:36:24.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.168+0000 7f0ef5eb8740 1 -- start start 2024-09-11T22:36:24.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4da6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4dab980 0x55e0c4dcdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4db6240 0x55e0c4dce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1ab7b0 con 0x55e0c4dab060 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1ad3e0 con 0x55e0c4db6240 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1abe80 con 0x55e0c4dab980 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef207c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4da6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef207c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4da6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51778/0 (socket says 172.21.15.5:51778) 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef207c640 1 -- 172.21.15.5:0/2281425978 learned_addr learned my addr 172.21.15.5:0/2281425978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef187b640 1 --2- 172.21.15.5:0/2281425978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4dab980 0x55e0c4dcdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef207c640 1 --2- 172.21.15.5:0/2281425978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4db6240 0x55e0c4dce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef207c640 1 -- 172.21.15.5:0/2281425978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4dab980 msgr2=0x55e0c4dcdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef207c640 1 --2- 172.21.15.5:0/2281425978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4dab980 0x55e0c4dcdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef207c640 1 -- 172.21.15.5:0/2281425978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4db6240 msgr2=0x55e0c4dce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef207c640 1 --2- 172.21.15.5:0/2281425978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4db6240 0x55e0c4dce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef207c640 1 -- 172.21.15.5:0/2281425978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c4da8280 con 0x55e0c4dab060 2024-09-11T22:36:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.169+0000 7f0ef207c640 1 --2- 172.21.15.5:0/2281425978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4da6640 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4030360 tx=0x7f0ee4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9317bf312bb5b996 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.170+0000 7f0ed77fe640 1 -- 172.21.15.5:0/2281425978 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4050040 con 0x55e0c4dab060 2024-09-11T22:36:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.170+0000 7f0ed77fe640 1 -- 172.21.15.5:0/2281425978 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee4036040 con 0x55e0c4dab060 2024-09-11T22:36:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.170+0000 7f0ed77fe640 1 -- 172.21.15.5:0/2281425978 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4030720 con 0x55e0c4dab060 2024-09-11T22:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.170+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/2281425978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 msgr2=0x55e0c4da6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.170+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/2281425978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4da6640 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4030360 tx=0x7f0ee4000f40 comp rx=0 tx=0).stop 2024-09-11T22:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.170+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/2281425978 shutdown_connections 2024-09-11T22:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.170+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/2281425978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4db6240 0x55e0c4dce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.170+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/2281425978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4dab980 0x55e0c4dcdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.170+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/2281425978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4da6640 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.170+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/2281425978 >> 172.21.15.5:0/2281425978 conn(0x55e0c4da75f0 msgr2=0x55e0c4da5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/2281425978 shutdown_connections 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/2281425978 wait complete. 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 1 -- start start 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 1 Processor -- start 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4db9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 0x55e0c4dba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4dba920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1acd10 con 0x55e0c4dab060 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1ac580 con 0x55e0c4dab980 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0eec1a9810 con 0x55e0c4db6240 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef5eb8740 10 client.0 _ll_register_callbacks cb 0x7f0ee4003dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.171+0000 7f0ef207c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4db9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef207c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4db9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51782/0 (socket says 172.21.15.5:51782) 2024-09-11T22:36:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef207c640 1 -- 172.21.15.5:0/1941309802 learned_addr learned my addr 172.21.15.5:0/1941309802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef187b640 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 0x55e0c4dba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef207c640 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4dba920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef207c640 1 -- 172.21.15.5:0/1941309802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 msgr2=0x55e0c4dba920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef207c640 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4dba920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef207c640 1 -- 172.21.15.5:0/1941309802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 msgr2=0x55e0c4dba3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef207c640 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 0x55e0c4dba3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef207c640 1 -- 172.21.15.5:0/1941309802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c4e2fcc0 con 0x55e0c4dab060 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef207c640 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4db9ea0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4003da0 tx=0x7f0ee403d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd0285d18d44535 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ed57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ed57fa640 1 -- 172.21.15.5:0/1941309802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4050030 con 0x55e0c4dab060 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ed57fa640 1 -- 172.21.15.5:0/1941309802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f0ee402e070 con 0x55e0c4dab060 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1941309802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee4036040 con 0x55e0c4dab060 2024-09-11T22:36:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.172+0000 7f0ed57fa640 1 -- 172.21.15.5:0/1941309802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0ee4061070 con 0x55e0c4dab060 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ef5eb8740 10 client.19845 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1941309802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e0c4e30f40 con 0x55e0c4dab060 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ed57fa640 1 -- 172.21.15.5:0/1941309802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ee4036040 con 0x55e0c4dab060 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ed57fa640 1 -- 172.21.15.5:0/1941309802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f0ee4030040 con 0x55e0c4dab060 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ef5eb8740 10 client.19845 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ef5eb8740 10 client.19845 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ef5eb8740 20 client.19845 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e0c4e31670 0x55e0c4e33a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ef5eb8740 4 client.19845 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "91bd8d7f", "release": true, "await": true}] 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1941309802 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "91bd8d7f", "release": true, "await": true}) -- 0x55e0c4d180a0 con 0x55e0c4e31670 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.173+0000 7f0ef187b640 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e0c4e31670 0x55e0c4e33a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.174+0000 7f0ed57fa640 10 client.19845 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.174+0000 7f0ef187b640 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e0c4e31670 0x55e0c4e33a90 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a4f15ec9316c5b0 server_cookie=1cd2b151a5a496ec in_seq=0 out_seq=0 2024-09-11T22:36:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.236+0000 7fb0b55c4640 1 -- 172.21.15.5:0/3317772930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0102470 msgr2=0x7fb0b010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.236+0000 7fb0b55c4640 1 --2- 172.21.15.5:0/3317772930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0102470 0x7fb0b010e9a0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009cd0 tx=0x7fb0a802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.236+0000 7fb0b55c4640 1 -- 172.21.15.5:0/3317772930 shutdown_connections 2024-09-11T22:36:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.236+0000 7fb0b55c4640 1 --2- 172.21.15.5:0/3317772930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0102470 0x7fb0b010e9a0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.236+0000 7fb0b55c4640 1 --2- 172.21.15.5:0/3317772930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b0101ad0 0x7fb0b0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.236+0000 7fb0b55c4640 1 --2- 172.21.15.5:0/3317772930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b0107b70 0x7fb0b0107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.236+0000 7fb0b55c4640 1 -- 172.21.15.5:0/3317772930 >> 172.21.15.5:0/3317772930 conn(0x7fb0b00fd7d0 msgr2=0x7fb0b00ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.237+0000 7fb0b55c4640 1 -- 172.21.15.5:0/3317772930 shutdown_connections 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.237+0000 7fb0b55c4640 1 -- 172.21.15.5:0/3317772930 wait complete. 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.237+0000 7fb0b55c4640 1 Processor -- start 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.237+0000 7fb0b55c4640 1 -- start start 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.237+0000 7fb0b55c4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b0101ad0 0x7fb0b019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.237+0000 7fb0b55c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0102470 0x7fb0b019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.237+0000 7fb0b55c4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b0107b70 0x7fb0b019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.237+0000 7fb0b55c4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0075c00 con 0x7fb0b0102470 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.237+0000 7fb0b55c4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0075a80 con 0x7fb0b0107b70 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.237+0000 7fb0b55c4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b0075d80 con 0x7fb0b0101ad0 2024-09-11T22:36:24.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0102470 0x7fb0b019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0102470 0x7fb0b019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51794/0 (socket says 172.21.15.5:51794) 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0a7fff640 1 -- 172.21.15.5:0/2499960698 learned_addr learned my addr 172.21.15.5:0/2499960698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b0107b70 0x7fb0b019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0aeffd640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b0101ad0 0x7fb0b019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0a7fff640 1 -- 172.21.15.5:0/2499960698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b0101ad0 msgr2=0x7fb0b019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0a7fff640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b0101ad0 0x7fb0b019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0a7fff640 1 -- 172.21.15.5:0/2499960698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b0107b70 msgr2=0x7fb0b019ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0a7fff640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b0107b70 0x7fb0b019ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0a7fff640 1 -- 172.21.15.5:0/2499960698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b019f550 con 0x7fb0b0102470 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0a7fff640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0102470 0x7fb0b019aaa0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb09800d770 tx=0x7fb09800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0acff9640 1 -- 172.21.15.5:0/2499960698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098017070 con 0x7fb0b0102470 2024-09-11T22:36:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0b55c4640 1 -- 172.21.15.5:0/2499960698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b019f840 con 0x7fb0b0102470 2024-09-11T22:36:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0acff9640 1 -- 172.21.15.5:0/2499960698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb098005500 con 0x7fb0b0102470 2024-09-11T22:36:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0acff9640 1 -- 172.21.15.5:0/2499960698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098004e00 con 0x7fb0b0102470 2024-09-11T22:36:24.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.238+0000 7fb0b55c4640 1 -- 172.21.15.5:0/2499960698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b01a70d0 con 0x7fb0b0102470 2024-09-11T22:36:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.241+0000 7fb0b843b740 1 Processor -- start 2024-09-11T22:36:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0b843b740 1 -- start start 2024-09-11T22:36:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4daf0 0x556994d4ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d58770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 0x556994d5b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0acff9640 1 -- 172.21.15.5:0/2499960698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb098004fa0 con 0x7fb0b0102470 2024-09-11T22:36:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0b843b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556994cb22f0 con 0x556994d4daf0 2024-09-11T22:36:24.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0b843b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556994c5a270 con 0x556994d4e410 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0b843b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556994cae2b0 con 0x556994d58cb0 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4daf0 0x556994d4ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4daf0 0x556994d4ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51808/0 (socket says 172.21.15.5:51808) 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0af7fe640 1 -- 172.21.15.5:0/3409279736 learned_addr learned my addr 172.21.15.5:0/3409279736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0aeffd640 1 --2- 172.21.15.5:0/3409279736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d58770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0a7fff640 1 --2- 172.21.15.5:0/3409279736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 0x556994d5b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0af7fe640 1 -- 172.21.15.5:0/3409279736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 msgr2=0x556994d5b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/3409279736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 0x556994d5b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0af7fe640 1 -- 172.21.15.5:0/3409279736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 msgr2=0x556994d58770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/3409279736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d58770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0af7fe640 1 -- 172.21.15.5:0/3409279736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556994c59340 con 0x556994d4daf0 2024-09-11T22:36:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.242+0000 7fb0a7fff640 1 --2- 172.21.15.5:0/3409279736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 0x556994d5b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.243+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/3409279736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4daf0 0x556994d4ded0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009cd0 tx=0x7fb0a8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe3735973c99669 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.243+0000 7fb0acff9640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb09c0a2da0 0x7fb09c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.243+0000 7fb0a57fa640 1 -- 172.21.15.5:0/3409279736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8048070 con 0x556994d4daf0 2024-09-11T22:36:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.243+0000 7fb0a57fa640 1 -- 172.21.15.5:0/3409279736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a8036070 con 0x556994d4daf0 2024-09-11T22:36:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.243+0000 7fb0a57fa640 1 -- 172.21.15.5:0/3409279736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004470 con 0x556994d4daf0 2024-09-11T22:36:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.243+0000 7fb0aeffd640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb09c0a2da0 0x7fb09c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.243+0000 7fb0acff9640 1 -- 172.21.15.5:0/2499960698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fb0980cef40 con 0x7fb0b0102470 2024-09-11T22:36:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 -- 172.21.15.5:0/3409279736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4daf0 msgr2=0x556994d4ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0aeffd640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb09c0a2da0 0x7fb09c0a5260 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x556994d4aec0 tx=0x7fb0a0005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 --2- 172.21.15.5:0/3409279736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4daf0 0x556994d4ded0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009cd0 tx=0x7fb0a8009a20 comp rx=0 tx=0).stop 2024-09-11T22:36:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 -- 172.21.15.5:0/3409279736 shutdown_connections 2024-09-11T22:36:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 --2- 172.21.15.5:0/3409279736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 0x556994d5b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 --2- 172.21.15.5:0/3409279736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d58770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 --2- 172.21.15.5:0/3409279736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4daf0 0x556994d4ded0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 -- 172.21.15.5:0/3409279736 >> 172.21.15.5:0/3409279736 conn(0x556994d4a060 msgr2=0x556994d4a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 -- 172.21.15.5:0/3409279736 shutdown_connections 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 -- 172.21.15.5:0/3409279736 wait complete. 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 -- start start 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.244+0000 7fb0b843b740 1 Processor -- start 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d4daf0 0x556994d55b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d565e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0b843b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556994d5b8d0 con 0x556994d58cb0 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0b843b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556994c3b290 con 0x556994d4e410 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0b843b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556994d5ba50 con 0x556994d4daf0 2024-09-11T22:36:24.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0b843b740 10 client.0 _ll_register_callbacks cb 0x556994c6c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0aeffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d4daf0 0x556994d55b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51810/0 (socket says 172.21.15.5:51810) 2024-09-11T22:36:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0af7fe640 1 -- 172.21.15.5:0/609010627 learned_addr learned my addr 172.21.15.5:0/609010627 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0af7fe640 1 -- 172.21.15.5:0/609010627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d4daf0 msgr2=0x556994d55b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d4daf0 0x556994d55b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0af7fe640 1 -- 172.21.15.5:0/609010627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 msgr2=0x556994d560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0af7fe640 1 -- 172.21.15.5:0/609010627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569949c5210 con 0x556994d58cb0 2024-09-11T22:36:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d565e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fb0a802ae00 tx=0x7fb0a802b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c8657a8ddd6911 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.245+0000 7fb0967fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0967fc640 1 -- 172.21.15.5:0/609010627 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8048040 con 0x556994d58cb0 2024-09-11T22:36:24.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0967fc640 1 -- 172.21.15.5:0/609010627 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fb0a8031ac0 con 0x556994d58cb0 2024-09-11T22:36:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0b843b740 1 -- 172.21.15.5:0/609010627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a8036070 con 0x556994d58cb0 2024-09-11T22:36:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0967fc640 1 -- 172.21.15.5:0/609010627 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb0a8031cc0 con 0x556994d58cb0 2024-09-11T22:36:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0b843b740 10 client.19857 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0b843b740 1 -- 172.21.15.5:0/609010627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556994a91620 con 0x556994d58cb0 2024-09-11T22:36:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0967fc640 1 -- 172.21.15.5:0/609010627 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a8036070 con 0x556994d58cb0 2024-09-11T22:36:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0967fc640 1 -- 172.21.15.5:0/609010627 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fb0a8042030 con 0x556994d58cb0 2024-09-11T22:36:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0b843b740 10 client.19857 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0b843b740 10 client.19857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0b843b740 20 client.19857 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:24.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0b843b740 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556994d68aa0 0x556994d6ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0b843b740 4 client.19857 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.246+0000 7fb0b843b740 1 -- 172.21.15.5:0/609010627 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556994cbfa20 con 0x556994d68aa0 2024-09-11T22:36:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.247+0000 7fb0aeffd640 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556994d68aa0 0x556994d6ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.247+0000 7fb0967fc640 10 client.19857 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.247+0000 7fb0aeffd640 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556994d68aa0 0x556994d6ae90 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e203d3e94ab0ce5 server_cookie=ea77d4479847b5cf in_seq=0 out_seq=0 2024-09-11T22:36:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0967fc640 1 -- 172.21.15.5:0/609010627 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556994cbfa20 con 0x556994d68aa0 2024-09-11T22:36:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0967fc640 10 client.19857 handle_command_reply: tid=0 2024-09-11T22:36:24.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 client.19857 shutdown 2024-09-11T22:36:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 -- 172.21.15.5:0/609010627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 msgr2=0x556994d565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d565e0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fb0a802ae00 tx=0x7fb0a802b020 comp rx=0 tx=0).stop 2024-09-11T22:36:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 -- 172.21.15.5:0/609010627 shutdown_connections 2024-09-11T22:36:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556994d68aa0 0x556994d6ae90 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d565e0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0967fc640 0 client.19857 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 --2- 172.21.15.5:0/609010627 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d4daf0 0x556994d55b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 -- 172.21.15.5:0/609010627 >> 172.21.15.5:0/609010627 conn(0x556994d4a060 msgr2=0x556994d4fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 -- 172.21.15.5:0/609010627 shutdown_connections 2024-09-11T22:36:24.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 1 -- 172.21.15.5:0/609010627 wait complete. 2024-09-11T22:36:24.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.248+0000 7fb0b843b740 20 client.19857 trim_cache size 0 max 16384 2024-09-11T22:36:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ed57fa640 1 -- 172.21.15.5:0/1941309802 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e0c4d180a0 con 0x55e0c4e31670 2024-09-11T22:36:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ed57fa640 10 client.19845 handle_command_reply: tid=0 2024-09-11T22:36:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 client.19845 shutdown 2024-09-11T22:36:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1941309802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 msgr2=0x55e0c4db9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4db9ea0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4003da0 tx=0x7f0ee403d040 comp rx=0 tx=0).stop 2024-09-11T22:36:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1941309802 shutdown_connections 2024-09-11T22:36:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e0c4e31670 0x55e0c4e33a90 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e0c4db6240 0x55e0c4dba920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e0c4dab980 0x55e0c4dba3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ed57fa640 0 client.19845 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 --2- 172.21.15.5:0/1941309802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c4dab060 0x55e0c4db9ea0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1941309802 >> 172.21.15.5:0/1941309802 conn(0x55e0c4da75f0 msgr2=0x55e0c4da6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1941309802 shutdown_connections 2024-09-11T22:36:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.249+0000 7f0ef5eb8740 1 -- 172.21.15.5:0/1941309802 wait complete. 2024-09-11T22:36:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef5eb8740 20 client.19845 trim_cache size 0 max 16384 2024-09-11T22:36:24.260 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:36:24.260 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 50, 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 52, 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "91bd8d7f": { 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 52, 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:36:24.261 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:24.262 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:24.263 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:36:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 -- 172.21.15.5:0/327874559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ed80a2cf0 msgr2=0x7f0ed80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ed80a2cf0 0x7f0ed80a51b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0cc690 tx=0x7f0edc096fa0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 -- 172.21.15.5:0/327874559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eec101410 msgr2=0x7f0eec19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eec101410 0x7f0eec19ab50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0097d0 tx=0x7f0edc009c90 comp rx=0 tx=0).stop 2024-09-11T22:36:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 -- 172.21.15.5:0/327874559 shutdown_connections 2024-09-11T22:36:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0eec106a70 0x7f0eec19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0eec101410 0x7f0eec19ab50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f0ed80a2cf0 0x7f0ed80a51b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 --2- 172.21.15.5:0/327874559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0eec100a70 0x7f0eec19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 -- 172.21.15.5:0/327874559 >> 172.21.15.5:0/327874559 conn(0x7f0eec0fc790 msgr2=0x7f0eec0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 -- 172.21.15.5:0/327874559 shutdown_connections 2024-09-11T22:36:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:24.250+0000 7f0ef3adf640 1 -- 172.21.15.5:0/327874559 wait complete. 2024-09-11T22:36:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.268+0000 7fb0b843b740 1 Processor -- start 2024-09-11T22:36:24.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.268+0000 7fb0b843b740 1 -- start start 2024-09-11T22:36:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d4daf0 0x556994d54410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d704f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0b843b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01afb60 con 0x556994d58cb0 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0b843b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01b1790 con 0x556994d4e410 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0b843b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01b0230 con 0x556994d4daf0 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d704f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d704f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51818/0 (socket says 172.21.15.5:51818) 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0af7fe640 1 -- 172.21.15.5:0/3126864378 learned_addr learned my addr 172.21.15.5:0/3126864378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0aeffd640 1 --2- 172.21.15.5:0/3126864378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d4daf0 0x556994d54410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0af7fe640 1 -- 172.21.15.5:0/3126864378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d4daf0 msgr2=0x556994d54410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/3126864378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d4daf0 0x556994d54410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0af7fe640 1 -- 172.21.15.5:0/3126864378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 msgr2=0x556994d563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/3126864378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0af7fe640 1 -- 172.21.15.5:0/3126864378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556994d4e240 con 0x556994d58cb0 2024-09-11T22:36:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.269+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/3126864378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d704f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80317f0 tx=0x7fb0a8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8464551d9d3088 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.270+0000 7fb0a4ff9640 1 -- 172.21.15.5:0/3126864378 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8048040 con 0x556994d58cb0 2024-09-11T22:36:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.270+0000 7fb0a4ff9640 1 -- 172.21.15.5:0/3126864378 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a8036040 con 0x556994d58cb0 2024-09-11T22:36:24.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.270+0000 7fb0a4ff9640 1 -- 172.21.15.5:0/3126864378 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8031b20 con 0x556994d58cb0 2024-09-11T22:36:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.270+0000 7fb0b843b740 1 -- 172.21.15.5:0/3126864378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 msgr2=0x556994d704f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.270+0000 7fb0b843b740 1 --2- 172.21.15.5:0/3126864378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d704f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80317f0 tx=0x7fb0a8009a20 comp rx=0 tx=0).stop 2024-09-11T22:36:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.270+0000 7fb0b843b740 1 -- 172.21.15.5:0/3126864378 shutdown_connections 2024-09-11T22:36:24.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.270+0000 7fb0b843b740 1 --2- 172.21.15.5:0/3126864378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d58cb0 0x556994d704f0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.270+0000 7fb0b843b740 1 --2- 172.21.15.5:0/3126864378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4e410 0x556994d563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.270+0000 7fb0b843b740 1 --2- 172.21.15.5:0/3126864378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d4daf0 0x556994d54410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.270+0000 7fb0b843b740 1 -- 172.21.15.5:0/3126864378 >> 172.21.15.5:0/3126864378 conn(0x556994d4a060 msgr2=0x556994d57be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.271+0000 7fb0b843b740 1 -- 172.21.15.5:0/3126864378 shutdown_connections 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.271+0000 7fb0b843b740 1 -- 172.21.15.5:0/3126864378 wait complete. 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.271+0000 7fb0b843b740 1 -- start start 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.271+0000 7fb0b843b740 1 Processor -- start 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.271+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4daf0 0x556994d5c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.271+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4e410 0x556994d5c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0b843b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 0x556994dd9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4e410 0x556994d5c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0aeffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4daf0 0x556994d5c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0b843b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01b10c0 con 0x556994d4e410 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0b843b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01b0930 con 0x556994d4daf0 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0aeffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4daf0 0x556994d5c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51696/0 (socket says 172.21.15.5:51696) 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0b843b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b01adb60 con 0x556994d58cb0 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0aeffd640 1 -- 172.21.15.5:0/617726946 learned_addr learned my addr 172.21.15.5:0/617726946 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0b843b740 10 client.0 _ll_register_callbacks cb 0x7fb0a8046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0af7fe640 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 0x556994dd9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0aeffd640 1 -- 172.21.15.5:0/617726946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 msgr2=0x556994dd9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0aeffd640 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 0x556994dd9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0aeffd640 1 -- 172.21.15.5:0/617726946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4e410 msgr2=0x556994d5c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0aeffd640 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4e410 0x556994d5c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0aeffd640 1 -- 172.21.15.5:0/617726946 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556994dda2d0 con 0x556994d4daf0 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0a7fff640 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4e410 0x556994d5c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0aeffd640 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4daf0 0x556994d5c220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x556994d6b490 tx=0x7fb0a00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=404b966143d0f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.272+0000 7fb0967fc640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.273+0000 7fb0967fc640 1 -- 172.21.15.5:0/617726946 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a001fb40 con 0x556994d4daf0 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.273+0000 7fb0967fc640 1 -- 172.21.15.5:0/617726946 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fb0a001fce0 con 0x556994d4daf0 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.273+0000 7fb0b843b740 1 -- 172.21.15.5:0/617726946 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a8036040 con 0x556994d4daf0 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.273+0000 7fb0967fc640 1 -- 172.21.15.5:0/617726946 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb0a002b420 con 0x556994d4daf0 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.273+0000 7fb0b843b740 10 client.29009 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:24.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.273+0000 7fb0b843b740 1 -- 172.21.15.5:0/617726946 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556994ddb600 con 0x556994d4daf0 2024-09-11T22:36:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.273+0000 7fb0967fc640 1 -- 172.21.15.5:0/617726946 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a002baa0 con 0x556994d4daf0 2024-09-11T22:36:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.274+0000 7fb0967fc640 1 -- 172.21.15.5:0/617726946 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fb0a0033060 con 0x556994d4daf0 2024-09-11T22:36:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.274+0000 7fb0b843b740 10 client.29009 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.274+0000 7fb0b843b740 10 client.29009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.274+0000 7fb0b843b740 20 client.29009 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.274+0000 7fb0b843b740 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556994ddbe70 0x556994dde290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.274+0000 7fb0b843b740 4 client.29009 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:36:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.274+0000 7fb0b843b740 1 -- 172.21.15.5:0/617726946 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fb0a8036040 con 0x556994ddbe70 2024-09-11T22:36:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.274+0000 7fb0a7fff640 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556994ddbe70 0x556994dde290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:24.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.274+0000 7fb0a7fff640 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556994ddbe70 0x556994dde290 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276ace5b0c78e309 server_cookie=57c9e11dae8ecd6 in_seq=0 out_seq=0 2024-09-11T22:36:24.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.274+0000 7fb0967fc640 10 client.29009 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.435+0000 7fb0967fc640 1 -- 172.21.15.5:0/617726946 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fb0a8036040 con 0x556994ddbe70 2024-09-11T22:36:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.435+0000 7fb0967fc640 10 client.29009 handle_command_reply: tid=0 2024-09-11T22:36:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.435+0000 7fb0b843b740 1 client.29009 shutdown 2024-09-11T22:36:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.435+0000 7fb0b843b740 1 -- 172.21.15.5:0/617726946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4daf0 msgr2=0x556994d5c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.435+0000 7fb0b843b740 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4daf0 0x556994d5c220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x556994d6b490 tx=0x7fb0a00046c0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b843b740 1 -- 172.21.15.5:0/617726946 shutdown_connections 2024-09-11T22:36:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b843b740 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556994ddbe70 0x556994dde290 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b843b740 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556994d58cb0 0x556994dd9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b843b740 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556994d4e410 0x556994d5c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b843b740 1 --2- 172.21.15.5:0/617726946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556994d4daf0 0x556994d5c220 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0967fc640 0 client.29009 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b843b740 1 -- 172.21.15.5:0/617726946 >> 172.21.15.5:0/617726946 conn(0x556994d4a060 msgr2=0x556994d489a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b843b740 1 -- 172.21.15.5:0/617726946 shutdown_connections 2024-09-11T22:36:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b843b740 1 -- 172.21.15.5:0/617726946 wait complete. 2024-09-11T22:36:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b843b740 20 client.29009 trim_cache size 0 max 16384 2024-09-11T22:36:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:36:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:36:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:36:24.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:36:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b55c4640 1 -- 172.21.15.5:0/2499960698 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb09c0a2da0 msgr2=0x7fb09c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b55c4640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb09c0a2da0 0x7fb09c0a5260 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x556994d4aec0 tx=0x7fb0a0005f90 comp rx=0 tx=0).stop 2024-09-11T22:36:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b55c4640 1 -- 172.21.15.5:0/2499960698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0102470 msgr2=0x7fb0b019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b55c4640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0102470 0x7fb0b019aaa0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb09800d770 tx=0x7fb09800dc30 comp rx=0 tx=0).stop 2024-09-11T22:36:24.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b55c4640 1 -- 172.21.15.5:0/2499960698 shutdown_connections 2024-09-11T22:36:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b55c4640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb09c0a2da0 0x7fb09c0a5260 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b55c4640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb0b0107b70 0x7fb0b019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b55c4640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0102470 0x7fb0b019aaa0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b55c4640 1 --2- 172.21.15.5:0/2499960698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0b0101ad0 0x7fb0b019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.436+0000 7fb0b55c4640 1 -- 172.21.15.5:0/2499960698 >> 172.21.15.5:0/2499960698 conn(0x7fb0b00fd7d0 msgr2=0x7fb0b00fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.437+0000 7fb0b55c4640 1 -- 172.21.15.5:0/2499960698 shutdown_connections 2024-09-11T22:36:24.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:24.437+0000 7fb0b55c4640 1 -- 172.21.15.5:0/2499960698 wait complete. 2024-09-11T22:36:24.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7442 13.77 MB/sec warmup 42 sec latency 28360.432 ms 2024-09-11T22:36:25.147 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '91bd8d7f', total seconds elapsed: 33.6 2024-09-11T22:36:25.147 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-11T22:36:25.148 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:36:25.533 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:25.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7816 14.08 MB/sec warmup 43 sec latency 160.955 ms 2024-09-11T22:36:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:25 smithi005 ceph-mon[30943]: pgmap v453: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 12 MiB/s wr, 105 op/s 2024-09-11T22:36:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:25 smithi159 ceph-mon[35534]: pgmap v453: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 12 MiB/s wr, 105 op/s 2024-09-11T22:36:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:25 smithi172 ceph-mon[35461]: pgmap v453: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 12 MiB/s wr, 105 op/s 2024-09-11T22:36:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.100+0000 7f254f360640 1 -- 172.21.15.5:0/2877283596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25481036e0 msgr2=0x7f2548107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.100+0000 7f254f360640 1 --2- 172.21.15.5:0/2877283596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25481036e0 0x7f2548107b90 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f2544009890 tx=0x7f254402a380 comp rx=0 tx=0).stop 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.100+0000 7f254f360640 1 -- 172.21.15.5:0/2877283596 shutdown_connections 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.100+0000 7f254f360640 1 --2- 172.21.15.5:0/2877283596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25481082a0 0x7f254810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.100+0000 7f254f360640 1 --2- 172.21.15.5:0/2877283596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25481036e0 0x7f2548107b90 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.100+0000 7f254f360640 1 --2- 172.21.15.5:0/2877283596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2548102d30 0x7f2548103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.100+0000 7f254f360640 1 -- 172.21.15.5:0/2877283596 >> 172.21.15.5:0/2877283596 conn(0x7f25480fc7d0 msgr2=0x7f25480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.100+0000 7f254f360640 1 -- 172.21.15.5:0/2877283596 shutdown_connections 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.100+0000 7f254f360640 1 -- 172.21.15.5:0/2877283596 wait complete. 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.101+0000 7f254f360640 1 Processor -- start 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.101+0000 7f254f360640 1 -- start start 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254f360640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548102d30 0x7f2548100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254f360640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25481036e0 0x7f2548100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254f360640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25481082a0 0x7f2548101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254f360640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25481120a0 con 0x7f2548102d30 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254f360640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2548111f20 con 0x7f25481036e0 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254f360640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2548112220 con 0x7f25481082a0 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254d0fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548102d30 0x7f2548100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254d0fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548102d30 0x7f2548100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51844/0 (socket says 172.21.15.5:51844) 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254d0fc640 1 -- 172.21.15.5:0/1148474154 learned_addr learned my addr 172.21.15.5:0/1148474154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254c8fb640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25481036e0 0x7f2548100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254d8fd640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25481082a0 0x7f2548101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254d0fc640 1 -- 172.21.15.5:0/1148474154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25481082a0 msgr2=0x7f2548101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254d0fc640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25481082a0 0x7f2548101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254d0fc640 1 -- 172.21.15.5:0/1148474154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25481036e0 msgr2=0x7f2548100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254d0fc640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25481036e0 0x7f2548100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254d0fc640 1 -- 172.21.15.5:0/1148474154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25481a6a80 con 0x7f2548102d30 2024-09-11T22:36:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.102+0000 7f254c8fb640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25481036e0 0x7f2548100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.103+0000 7f254d0fc640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548102d30 0x7f2548100990 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f253800e9c0 tx=0x7f253800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.103+0000 7f253e7fc640 1 -- 172.21.15.5:0/1148474154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f253800ccc0 con 0x7f2548102d30 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.103+0000 7f253e7fc640 1 -- 172.21.15.5:0/1148474154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2538005500 con 0x7f2548102d30 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.103+0000 7f254f360640 1 -- 172.21.15.5:0/1148474154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25481a6d70 con 0x7f2548102d30 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.104+0000 7f254f360640 1 -- 172.21.15.5:0/1148474154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25481a7270 con 0x7f2548102d30 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.104+0000 7f253e7fc640 1 -- 172.21.15.5:0/1148474154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2538010430 con 0x7f2548102d30 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.106+0000 7f254f360640 1 -- 172.21.15.5:0/1148474154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254806a850 con 0x7f2548102d30 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.107+0000 7f253e7fc640 1 -- 172.21.15.5:0/1148474154 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2538004660 con 0x7f2548102d30 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.108+0000 7f253e7fc640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2bd0 0x7f25340a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.108+0000 7f253e7fc640 1 -- 172.21.15.5:0/1148474154 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f25380cf770 con 0x7f2548102d30 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.108+0000 7f254c8fb640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2bd0 0x7f25340a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.109+0000 7f254c8fb640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2bd0 0x7f25340a5090 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f25440057c0 tx=0x7f25440056d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.115+0000 7f253e7fc640 1 -- 172.21.15.5:0/1148474154 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f253809c2c0 con 0x7f2548102d30 2024-09-11T22:36:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.409+0000 7f254f360640 1 -- 172.21.15.5:0/1148474154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2548103140 con 0x7f2548102d30 2024-09-11T22:36:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.410+0000 7f253e7fc640 1 -- 172.21.15.5:0/1148474154 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11975 (secure 0 0 0) 0x7f2548103140 con 0x7f2548102d30 2024-09-11T22:36:26.412 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:36:26.412 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"btime":"2024-09-11T22:35:49:548013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:35:48.842139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:36:26.413 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 50 2024-09-11T22:36:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.412+0000 7f254f360640 1 -- 172.21.15.5:0/1148474154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2bd0 msgr2=0x7f25340a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.412+0000 7f254f360640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2bd0 0x7f25340a5090 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f25440057c0 tx=0x7f25440056d0 comp rx=0 tx=0).stop 2024-09-11T22:36:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.412+0000 7f254f360640 1 -- 172.21.15.5:0/1148474154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548102d30 msgr2=0x7f2548100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.412+0000 7f254f360640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548102d30 0x7f2548100990 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f253800e9c0 tx=0x7f253800ee80 comp rx=0 tx=0).stop 2024-09-11T22:36:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.413+0000 7f254f360640 1 -- 172.21.15.5:0/1148474154 shutdown_connections 2024-09-11T22:36:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.413+0000 7f254f360640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25481082a0 0x7f2548101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.413+0000 7f254f360640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f25340a2bd0 0x7f25340a5090 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.413+0000 7f254f360640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25481036e0 0x7f2548100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:26.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.413+0000 7f254f360640 1 --2- 172.21.15.5:0/1148474154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2548102d30 0x7f2548100990 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:26.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.413+0000 7f254f360640 1 -- 172.21.15.5:0/1148474154 >> 172.21.15.5:0/1148474154 conn(0x7f25480fc7d0 msgr2=0x7f2548105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:26.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.413+0000 7f254f360640 1 -- 172.21.15.5:0/1148474154 shutdown_connections 2024-09-11T22:36:26.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:26.413+0000 7f254f360640 1 -- 172.21.15.5:0/1148474154 wait complete. 2024-09-11T22:36:26.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8146 14.20 MB/sec warmup 44 sec latency 225.891 ms 2024-09-11T22:36:26.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:26 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1148474154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:26.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:26 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1148474154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:26.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:26 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1148474154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:27.310 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-11T22:36:27.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-11T22:36:27.587 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:28.062 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:27 smithi172 ceph-mon[35461]: pgmap v454: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 102 op/s 2024-09-11T22:36:28.062 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:27 smithi159 ceph-mon[35534]: pgmap v454: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 102 op/s 2024-09-11T22:36:28.062 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8536 14.29 MB/sec warmup 45 sec latency 51.131 ms 2024-09-11T22:36:28.063 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:27 smithi005 ceph-mon[30943]: pgmap v454: 97 pgs: 97 active+clean; 336 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 12 MiB/s wr, 102 op/s 2024-09-11T22:36:28.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.112+0000 7ff950c69640 1 -- 172.21.15.5:0/559985487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c100100 msgr2=0x7ff94c100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:28.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.112+0000 7ff950c69640 1 --2- 172.21.15.5:0/559985487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c100100 0x7ff94c100560 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff940009900 tx=0x7ff94002a410 comp rx=0 tx=0).stop 2024-09-11T22:36:28.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.112+0000 7ff950c69640 1 -- 172.21.15.5:0/559985487 shutdown_connections 2024-09-11T22:36:28.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.112+0000 7ff950c69640 1 --2- 172.21.15.5:0/559985487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94c100aa0 0x7ff94c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.112+0000 7ff950c69640 1 --2- 172.21.15.5:0/559985487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c100100 0x7ff94c100560 secure :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7ff940009900 tx=0x7ff94002a410 comp rx=0 tx=0).stop 2024-09-11T22:36:28.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.112+0000 7ff950c69640 1 --2- 172.21.15.5:0/559985487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff94c106b80 0x7ff94c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.112+0000 7ff950c69640 1 -- 172.21.15.5:0/559985487 >> 172.21.15.5:0/559985487 conn(0x7ff94c077230 msgr2=0x7ff94c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:28.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.114+0000 7ff950c69640 1 -- 172.21.15.5:0/559985487 shutdown_connections 2024-09-11T22:36:28.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.114+0000 7ff950c69640 1 -- 172.21.15.5:0/559985487 wait complete. 2024-09-11T22:36:28.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.115+0000 7ff950c69640 1 Processor -- start 2024-09-11T22:36:28.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff950c69640 1 -- start start 2024-09-11T22:36:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff950c69640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94c100aa0 0x7ff94c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff950c69640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff94c106b80 0x7ff94c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff950c69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c19a700 0x7ff94c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff94a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94c100aa0 0x7ff94c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff94a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94c100aa0 0x7ff94c1a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:48480/0 (socket says 172.21.15.5:48480) 2024-09-11T22:36:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff950c69640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c112060 con 0x7ff94c19a700 2024-09-11T22:36:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff950c69640 1 -- 172.21.15.5:0/1510314045 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c111ee0 con 0x7ff94c106b80 2024-09-11T22:36:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff94a59c640 1 -- 172.21.15.5:0/1510314045 learned_addr learned my addr 172.21.15.5:0/1510314045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff949d9b640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff94c106b80 0x7ff94c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff950c69640 1 -- 172.21.15.5:0/1510314045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1121e0 con 0x7ff94c100aa0 2024-09-11T22:36:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.116+0000 7ff94ad9d640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c19a700 0x7ff94c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.117+0000 7ff94a59c640 1 -- 172.21.15.5:0/1510314045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff94c106b80 msgr2=0x7ff94c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:28.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.117+0000 7ff94a59c640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff94c106b80 0x7ff94c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.117+0000 7ff94a59c640 1 -- 172.21.15.5:0/1510314045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c19a700 msgr2=0x7ff94c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.117+0000 7ff94a59c640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c19a700 0x7ff94c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.117+0000 7ff94a59c640 1 -- 172.21.15.5:0/1510314045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c19b2b0 con 0x7ff94c100aa0 2024-09-11T22:36:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.117+0000 7ff94a59c640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94c100aa0 0x7ff94c1a0630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00d950 tx=0x7ff93c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.117+0000 7ff93b7fe640 1 -- 172.21.15.5:0/1510314045 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c017030 con 0x7ff94c100aa0 2024-09-11T22:36:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.117+0000 7ff93b7fe640 1 -- 172.21.15.5:0/1510314045 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93c004020 con 0x7ff94c100aa0 2024-09-11T22:36:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.117+0000 7ff950c69640 1 -- 172.21.15.5:0/1510314045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c1a73e0 con 0x7ff94c100aa0 2024-09-11T22:36:28.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.118+0000 7ff950c69640 1 -- 172.21.15.5:0/1510314045 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c1a7780 con 0x7ff94c100aa0 2024-09-11T22:36:28.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.118+0000 7ff93b7fe640 1 -- 172.21.15.5:0/1510314045 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0047a0 con 0x7ff94c100aa0 2024-09-11T22:36:28.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.121+0000 7ff93b7fe640 1 -- 172.21.15.5:0/1510314045 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff93c0041d0 con 0x7ff94c100aa0 2024-09-11T22:36:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.121+0000 7ff953042740 1 Processor -- start 2024-09-11T22:36:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.122+0000 7ff953042740 1 -- start start 2024-09-11T22:36:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff93b7fe640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9340a2d70 0x7ff9340a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff953042740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f7db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff953042740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f7e080 0x563394f88400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff953042740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f88940 0x563394f8ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff953042740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563394ee32a0 con 0x563394f7e080 2024-09-11T22:36:28.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff953042740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563394e8ab10 con 0x563394f7d760 2024-09-11T22:36:28.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff953042740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563394edf260 con 0x563394f88940 2024-09-11T22:36:28.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff94a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f7db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff94a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f7db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51734/0 (socket says 172.21.15.5:51734) 2024-09-11T22:36:28.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff94a59c640 1 -- 172.21.15.5:0/307644102 learned_addr learned my addr 172.21.15.5:0/307644102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.123+0000 7ff949d9b640 1 --2- 172.21.15.5:0/307644102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f7e080 0x563394f88400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.124+0000 7ff94a59c640 1 -- 172.21.15.5:0/307644102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f88940 msgr2=0x563394f8ad30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:36:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.124+0000 7ff94a59c640 1 --2- 172.21.15.5:0/307644102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f88940 0x563394f8ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.124+0000 7ff94a59c640 1 -- 172.21.15.5:0/307644102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f7e080 msgr2=0x563394f88400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.124+0000 7ff94a59c640 1 --2- 172.21.15.5:0/307644102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f7e080 0x563394f88400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.124+0000 7ff94a59c640 1 -- 172.21.15.5:0/307644102 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563394c54b90 con 0x563394f7d760 2024-09-11T22:36:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.124+0000 7ff93b7fe640 1 -- 172.21.15.5:0/1510314045 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7ff93c0cede0 con 0x7ff94c100aa0 2024-09-11T22:36:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.124+0000 7ff94a59c640 1 --2- 172.21.15.5:0/307644102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f7db40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff93c097690 tx=0x7ff93c021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a709bbd543f10319 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.124+0000 7ff94ad9d640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9340a2d70 0x7ff9340a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.124+0000 7ff938ff9640 1 -- 172.21.15.5:0/307644102 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c09b930 con 0x563394f7d760 2024-09-11T22:36:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.125+0000 7ff938ff9640 1 -- 172.21.15.5:0/307644102 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93c09b230 con 0x563394f7d760 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.125+0000 7ff938ff9640 1 -- 172.21.15.5:0/307644102 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0071c0 con 0x563394f7d760 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 -- 172.21.15.5:0/307644102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 msgr2=0x563394f7db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff94ad9d640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9340a2d70 0x7ff9340a5230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x563394f7b070 tx=0x7ff9440096c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 --2- 172.21.15.5:0/307644102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f7db40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff93c097690 tx=0x7ff93c021000 comp rx=0 tx=0).stop 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 -- 172.21.15.5:0/307644102 shutdown_connections 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 --2- 172.21.15.5:0/307644102 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f88940 0x563394f8ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 --2- 172.21.15.5:0/307644102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f7e080 0x563394f88400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 --2- 172.21.15.5:0/307644102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f7db40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 -- 172.21.15.5:0/307644102 >> 172.21.15.5:0/307644102 conn(0x563394f79cf0 msgr2=0x563394f7a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 -- 172.21.15.5:0/307644102 shutdown_connections 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 -- 172.21.15.5:0/307644102 wait complete. 2024-09-11T22:36:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 -- start start 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.126+0000 7ff953042740 1 Processor -- start 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.127+0000 7ff953042740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 0x563394f857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.127+0000 7ff953042740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f85d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.127+0000 7ff953042740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f88940 0x563394f86260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.127+0000 7ff94a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 0x563394f857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.127+0000 7ff94a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 0x563394f857e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:48504/0 (socket says 172.21.15.5:48504) 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.127+0000 7ff94a59c640 1 -- 172.21.15.5:0/123493943 learned_addr learned my addr 172.21.15.5:0/123493943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.127+0000 7ff949d9b640 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f85d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.127+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563394f8b510 con 0x563394f88940 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.128+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563394e6b6e0 con 0x563394f7e080 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.128+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563394f8b690 con 0x563394f7d760 2024-09-11T22:36:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.128+0000 7ff953042740 10 client.0 _ll_register_callbacks cb 0x563394e8fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.128+0000 7ff94ad9d640 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f88940 0x563394f86260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.128+0000 7ff949d9b640 1 -- 172.21.15.5:0/123493943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 msgr2=0x563394f857e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.128+0000 7ff949d9b640 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 0x563394f857e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.128+0000 7ff949d9b640 1 -- 172.21.15.5:0/123493943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f88940 msgr2=0x563394f86260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.128+0000 7ff949d9b640 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f88940 0x563394f86260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.128+0000 7ff949d9b640 1 -- 172.21.15.5:0/123493943 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563394f9d1e0 con 0x563394f7e080 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.129+0000 7ff949d9b640 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f85d20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff940008000 tx=0x7ff940005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614c6422375403e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.129+0000 7ff931ffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.129+0000 7ff931ffb640 1 -- 172.21.15.5:0/123493943 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940035040 con 0x563394f7e080 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.129+0000 7ff931ffb640 1 -- 172.21.15.5:0/123493943 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7ff9400048c0 con 0x563394f7e080 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.129+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c09b230 con 0x563394f7e080 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.129+0000 7ff931ffb640 1 -- 172.21.15.5:0/123493943 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff940005d50 con 0x563394f7e080 2024-09-11T22:36:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.130+0000 7ff953042740 10 client.29027 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.130+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x563394f9fb30 con 0x563394f7e080 2024-09-11T22:36:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.130+0000 7ff931ffb640 1 -- 172.21.15.5:0/123493943 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93c09b230 con 0x563394f7e080 2024-09-11T22:36:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.131+0000 7ff931ffb640 1 -- 172.21.15.5:0/123493943 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7ff940031c00 con 0x563394f7e080 2024-09-11T22:36:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.131+0000 7ff953042740 10 client.29027 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.131+0000 7ff953042740 10 client.29027 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:36:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.131+0000 7ff953042740 20 client.29027 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.131+0000 7ff953042740 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563394fa0fb0 0x563394fa3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.131+0000 7ff953042740 4 client.29027 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.131+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563394ef09d0 con 0x563394fa0fb0 2024-09-11T22:36:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.132+0000 7ff94a59c640 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563394fa0fb0 0x563394fa3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.132+0000 7ff94a59c640 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563394fa0fb0 0x563394fa3430 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3c570f37fe2067a server_cookie=e3d96b5b1ecc5528 in_seq=0 out_seq=0 2024-09-11T22:36:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.132+0000 7ff931ffb640 10 client.29027 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.137+0000 7ff931ffb640 1 -- 172.21.15.5:0/123493943 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563394ef09d0 con 0x563394fa0fb0 2024-09-11T22:36:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.137+0000 7ff931ffb640 10 client.29027 handle_command_reply: tid=0 2024-09-11T22:36:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.137+0000 7ff953042740 1 client.29027 shutdown 2024-09-11T22:36:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 msgr2=0x563394f85d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff953042740 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f85d20 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff940008000 tx=0x7ff940005270 comp rx=0 tx=0).stop 2024-09-11T22:36:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 shutdown_connections 2024-09-11T22:36:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff953042740 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x563394fa0fb0 0x563394fa3430 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff953042740 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f88940 0x563394f86260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff953042740 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f85d20 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff931ffb640 0 client.29027 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff953042740 1 --2- 172.21.15.5:0/123493943 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 0x563394f857e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 >> 172.21.15.5:0/123493943 conn(0x563394f79cf0 msgr2=0x563394f7fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 shutdown_connections 2024-09-11T22:36:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.138+0000 7ff953042740 1 -- 172.21.15.5:0/123493943 wait complete. 2024-09-11T22:36:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.139+0000 7ff953042740 20 client.29027 trim_cache size 0 max 16384 2024-09-11T22:36:28.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.169+0000 7ff953042740 1 Processor -- start 2024-09-11T22:36:28.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.169+0000 7ff953042740 1 -- start start 2024-09-11T22:36:28.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff953042740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 0x563394f801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff953042740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f84720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff953042740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f88940 0x563394fa8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff953042740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1abed0 con 0x563394f88940 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff953042740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1adb00 con 0x563394f7e080 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff953042740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ac5a0 con 0x563394f7d760 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff94ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f88940 0x563394fa8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff949d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f84720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff949d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f84720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51746/0 (socket says 172.21.15.5:51746) 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff94a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 0x563394f801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.170+0000 7ff949d9b640 1 -- 172.21.15.5:0/72148688 learned_addr learned my addr 172.21.15.5:0/72148688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.171+0000 7ff94a59c640 1 -- 172.21.15.5:0/72148688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 msgr2=0x563394f84720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.171+0000 7ff94a59c640 1 --2- 172.21.15.5:0/72148688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f84720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.171+0000 7ff94a59c640 1 -- 172.21.15.5:0/72148688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f88940 msgr2=0x563394fa8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:28.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.171+0000 7ff94a59c640 1 --2- 172.21.15.5:0/72148688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f88940 0x563394fa8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.171+0000 7ff94a59c640 1 -- 172.21.15.5:0/72148688 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563394f79b20 con 0x563394f7d760 2024-09-11T22:36:28.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.171+0000 7ff949d9b640 1 --2- 172.21.15.5:0/72148688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f84720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:28.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.171+0000 7ff94a59c640 1 --2- 172.21.15.5:0/72148688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 0x563394f801c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0cf0d0 tx=0x7ff93c09b0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb3cc2743715a705 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.171+0000 7ff933fff640 1 -- 172.21.15.5:0/72148688 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0974d0 con 0x563394f7d760 2024-09-11T22:36:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.171+0000 7ff933fff640 1 -- 172.21.15.5:0/72148688 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff93c09b230 con 0x563394f7d760 2024-09-11T22:36:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.171+0000 7ff933fff640 1 -- 172.21.15.5:0/72148688 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0cc170 con 0x563394f7d760 2024-09-11T22:36:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.172+0000 7ff953042740 1 -- 172.21.15.5:0/72148688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 msgr2=0x563394f801c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.172+0000 7ff953042740 1 --2- 172.21.15.5:0/72148688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 0x563394f801c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0cf0d0 tx=0x7ff93c09b0f0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.172+0000 7ff953042740 1 -- 172.21.15.5:0/72148688 shutdown_connections 2024-09-11T22:36:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.172+0000 7ff953042740 1 --2- 172.21.15.5:0/72148688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f88940 0x563394fa8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.172+0000 7ff953042740 1 --2- 172.21.15.5:0/72148688 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7e080 0x563394f84720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.172+0000 7ff953042740 1 --2- 172.21.15.5:0/72148688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f7d760 0x563394f801c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.172+0000 7ff953042740 1 -- 172.21.15.5:0/72148688 >> 172.21.15.5:0/72148688 conn(0x563394f79cf0 msgr2=0x563394f87860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.173+0000 7ff953042740 1 -- 172.21.15.5:0/72148688 shutdown_connections 2024-09-11T22:36:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.173+0000 7ff953042740 1 -- 172.21.15.5:0/72148688 wait complete. 2024-09-11T22:36:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.173+0000 7ff953042740 1 -- start start 2024-09-11T22:36:28.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.173+0000 7ff953042740 1 Processor -- start 2024-09-11T22:36:28.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff953042740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff953042740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f7e080 0x563394f95830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff953042740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f88940 0x56339500ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff953042740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ad430 con 0x563394f7e080 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff953042740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1acca0 con 0x563394f7d760 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff953042740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1a9f10 con 0x563394f88940 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff953042740 10 client.0 _ll_register_callbacks cb 0x7ff940004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff94a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff94a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51756/0 (socket says 172.21.15.5:51756) 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff94a59c640 1 -- 172.21.15.5:0/2332458676 learned_addr learned my addr 172.21.15.5:0/2332458676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff94ad9d640 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f88940 0x56339500ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.174+0000 7ff949d9b640 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f7e080 0x563394f95830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff94ad9d640 1 -- 172.21.15.5:0/2332458676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 msgr2=0x563394f952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff94ad9d640 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff94ad9d640 1 -- 172.21.15.5:0/2332458676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f7e080 msgr2=0x563394f95830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff94ad9d640 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f7e080 0x563394f95830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff94ad9d640 1 -- 172.21.15.5:0/2332458676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563394f75260 con 0x563394f88940 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff94a59c640 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff94ad9d640 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f88940 0x56339500ad60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x563394f7afc0 tx=0x7ff944004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41541578e94c27f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff931ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff931ffb640 1 -- 172.21.15.5:0/2332458676 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94402d060 con 0x563394f88940 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff931ffb640 1 -- 172.21.15.5:0/2332458676 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7ff94401f840 con 0x563394f88940 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff953042740 1 -- 172.21.15.5:0/2332458676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c09b230 con 0x563394f88940 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff931ffb640 1 -- 172.21.15.5:0/2332458676 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff94401fa40 con 0x563394f88940 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff953042740 10 client.28867 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.175+0000 7ff953042740 1 -- 172.21.15.5:0/2332458676 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56339500c2d0 con 0x563394f88940 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.176+0000 7ff931ffb640 1 -- 172.21.15.5:0/2332458676 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94401fbe0 con 0x563394f88940 2024-09-11T22:36:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.176+0000 7ff931ffb640 1 -- 172.21.15.5:0/2332458676 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7ff94401d3b0 con 0x563394f88940 2024-09-11T22:36:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.176+0000 7ff953042740 10 client.28867 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.176+0000 7ff953042740 10 client.28867 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:36:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.176+0000 7ff953042740 20 client.28867 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.176+0000 7ff953042740 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56339500cb30 0x56339500ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.176+0000 7ff953042740 4 client.28867 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-11T22:36:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.176+0000 7ff953042740 1 -- 172.21.15.5:0/2332458676 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7ff93c09b230 con 0x56339500cb30 2024-09-11T22:36:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.176+0000 7ff94a59c640 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56339500cb30 0x56339500ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.177+0000 7ff931ffb640 10 client.28867 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.177+0000 7ff94a59c640 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56339500cb30 0x56339500ef50 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f292193ae4c127 server_cookie=d5965fd530855fcb in_seq=0 out_seq=0 2024-09-11T22:36:28.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8693 14.10 MB/sec warmup 46 sec latency 439.043 ms 2024-09-11T22:36:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.798+0000 7ff931ffb640 1 -- 172.21.15.5:0/2332458676 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7ff93c09b230 con 0x56339500cb30 2024-09-11T22:36:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff931ffb640 10 client.28867 handle_command_reply: tid=0 2024-09-11T22:36:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 client.28867 shutdown 2024-09-11T22:36:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 -- 172.21.15.5:0/2332458676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f88940 msgr2=0x56339500ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f88940 0x56339500ad60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x563394f7afc0 tx=0x7ff944004ab0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 -- 172.21.15.5:0/2332458676 shutdown_connections 2024-09-11T22:36:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56339500cb30 0x56339500ef50 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563394f88940 0x56339500ad60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563394f7e080 0x563394f95830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 --2- 172.21.15.5:0/2332458676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563394f7d760 0x563394f952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff931ffb640 0 client.28867 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 -- 172.21.15.5:0/2332458676 >> 172.21.15.5:0/2332458676 conn(0x563394f79cf0 msgr2=0x563394f78650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 -- 172.21.15.5:0/2332458676 shutdown_connections 2024-09-11T22:36:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 1 -- 172.21.15.5:0/2332458676 wait complete. 2024-09-11T22:36:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.799+0000 7ff953042740 20 client.28867 trim_cache size 0 max 16384 2024-09-11T22:36:28.802 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:36:28.802 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 50, 2024-09-11T22:36:28.802 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:36:28.802 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 54, 2024-09-11T22:36:28.802 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:36:28.802 INFO:teuthology.orchestra.run.smithi005.stdout: "25352b1d": { 2024-09-11T22:36:28.802 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 54, 2024-09-11T22:36:28.802 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:36:28.802 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:36:28.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:28.803 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:28.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:28.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:28.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:28.804 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 -- 172.21.15.5:0/1510314045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9340a2d70 msgr2=0x7ff9340a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9340a2d70 0x7ff9340a5230 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x563394f7b070 tx=0x7ff9440096c0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 -- 172.21.15.5:0/1510314045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94c100aa0 msgr2=0x7ff94c1a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94c100aa0 0x7ff94c1a0630 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00d950 tx=0x7ff93c00de10 comp rx=0 tx=0).stop 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 -- 172.21.15.5:0/1510314045 shutdown_connections 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff94c19a700 0x7ff94c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9340a2d70 0x7ff9340a5230 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff94c106b80 0x7ff94c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 --2- 172.21.15.5:0/1510314045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff94c100aa0 0x7ff94c1a0630 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 -- 172.21.15.5:0/1510314045 >> 172.21.15.5:0/1510314045 conn(0x7ff94c077230 msgr2=0x7ff94c105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 -- 172.21.15.5:0/1510314045 shutdown_connections 2024-09-11T22:36:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:28.800+0000 7ff950c69640 1 -- 172.21.15.5:0/1510314045 wait complete. 2024-09-11T22:36:29.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 13.80 MB/sec warmup 47 sec latency 1439.139 ms 2024-09-11T22:36:29.740 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '25352b1d', quiesce took 4.6 seconds. Will release after: 25.4 2024-09-11T22:36:29.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:29 smithi005 ceph-mon[30943]: pgmap v455: 97 pgs: 97 active+clean; 320 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 311 op/s 2024-09-11T22:36:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:29 smithi159 ceph-mon[35534]: pgmap v455: 97 pgs: 97 active+clean; 320 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 311 op/s 2024-09-11T22:36:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:29 smithi172 ceph-mon[35461]: pgmap v455: 97 pgs: 97 active+clean; 320 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 14 MiB/s wr, 311 op/s 2024-09-11T22:36:30.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 13.51 MB/sec warmup 48 sec latency 2439.262 ms 2024-09-11T22:36:31.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 13.24 MB/sec warmup 49 sec latency 3439.359 ms 2024-09-11T22:36:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:31 smithi005 ceph-mon[30943]: pgmap v456: 97 pgs: 97 active+clean; 320 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 267 op/s 2024-09-11T22:36:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:31 smithi159 ceph-mon[35534]: pgmap v456: 97 pgs: 97 active+clean; 320 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 267 op/s 2024-09-11T22:36:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:31 smithi172 ceph-mon[35461]: pgmap v456: 97 pgs: 97 active+clean; 320 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 267 op/s 2024-09-11T22:36:32.633 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 12.97 MB/sec warmup 50 sec latency 4439.485 ms 2024-09-11T22:36:33.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 12.72 MB/sec warmup 51 sec latency 5439.608 ms 2024-09-11T22:36:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:33 smithi005 ceph-mon[30943]: pgmap v457: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.2 MiB/s wr, 331 op/s 2024-09-11T22:36:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:33 smithi159 ceph-mon[35534]: pgmap v457: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.2 MiB/s wr, 331 op/s 2024-09-11T22:36:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:33 smithi172 ceph-mon[35461]: pgmap v457: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.2 MiB/s wr, 331 op/s 2024-09-11T22:36:34.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 12.47 MB/sec warmup 52 sec latency 6439.703 ms 2024-09-11T22:36:35.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 12.24 MB/sec warmup 53 sec latency 7439.812 ms 2024-09-11T22:36:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:35 smithi005 ceph-mon[30943]: pgmap v458: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-11T22:36:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:35 smithi159 ceph-mon[35534]: pgmap v458: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-11T22:36:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:35 smithi172 ceph-mon[35461]: pgmap v458: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-11T22:36:36.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 12.01 MB/sec warmup 54 sec latency 8439.925 ms 2024-09-11T22:36:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.530+0000 7f9d2c682640 1 -- 172.21.15.5:0/3122033766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d241035a0 msgr2=0x7f9d2410ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.530+0000 7f9d2c682640 1 --2- 172.21.15.5:0/3122033766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d241035a0 0x7f9d2410ba80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009900 tx=0x7f9d1402a480 comp rx=0 tx=0).stop 2024-09-11T22:36:37.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.530+0000 7f9d2c682640 1 -- 172.21.15.5:0/3122033766 shutdown_connections 2024-09-11T22:36:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.530+0000 7f9d2c682640 1 --2- 172.21.15.5:0/3122033766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d2410c0d0 0x7f9d2410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.530+0000 7f9d2c682640 1 --2- 172.21.15.5:0/3122033766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d241035a0 0x7f9d2410ba80 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.530+0000 7f9d2c682640 1 --2- 172.21.15.5:0/3122033766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d24102c80 0x7f9d24103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.530+0000 7f9d2c682640 1 -- 172.21.15.5:0/3122033766 >> 172.21.15.5:0/3122033766 conn(0x7f9d24077990 msgr2=0x7f9d240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.530+0000 7f9d2c682640 1 -- 172.21.15.5:0/3122033766 shutdown_connections 2024-09-11T22:36:37.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.530+0000 7f9d2c682640 1 -- 172.21.15.5:0/3122033766 wait complete. 2024-09-11T22:36:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.531+0000 7f9d2c682640 1 Processor -- start 2024-09-11T22:36:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.531+0000 7f9d2c682640 1 -- start start 2024-09-11T22:36:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.531+0000 7f9d2c682640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d24102c80 0x7f9d24198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.531+0000 7f9d2c682640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d241035a0 0x7f9d241988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.531+0000 7f9d2c682640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2410c0d0 0x7f9d2419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.531+0000 7f9d2c682640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24111eb0 con 0x7f9d2410c0d0 2024-09-11T22:36:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.531+0000 7f9d2c682640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24111d30 con 0x7f9d241035a0 2024-09-11T22:36:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.531+0000 7f9d2c682640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d24112030 con 0x7f9d24102c80 2024-09-11T22:36:37.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2a41e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d24102c80 0x7f9d24198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2a41e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d24102c80 0x7f9d24198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:34522/0 (socket says 172.21.15.5:34522) 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2a41e640 1 -- 172.21.15.5:0/704639091 learned_addr learned my addr 172.21.15.5:0/704639091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2ac1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2410c0d0 0x7f9d2419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d241035a0 0x7f9d241988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2ac1f640 1 -- 172.21.15.5:0/704639091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d24102c80 msgr2=0x7f9d24198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2ac1f640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d24102c80 0x7f9d24198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2ac1f640 1 -- 172.21.15.5:0/704639091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d241035a0 msgr2=0x7f9d241988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2ac1f640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d241035a0 0x7f9d241988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2ac1f640 1 -- 172.21.15.5:0/704639091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2419d350 con 0x7f9d2410c0d0 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2ac1f640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2410c0d0 0x7f9d2419cc20 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009fd0 tx=0x7f9d1c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d1b7fe640 1 -- 172.21.15.5:0/704639091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00ce20 con 0x7f9d2410c0d0 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2c682640 1 -- 172.21.15.5:0/704639091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2419d640 con 0x7f9d2410c0d0 2024-09-11T22:36:37.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d1b7fe640 1 -- 172.21.15.5:0/704639091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d1c005020 con 0x7f9d2410c0d0 2024-09-11T22:36:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d1b7fe640 1 -- 172.21.15.5:0/704639091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c010700 con 0x7f9d2410c0d0 2024-09-11T22:36:37.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.532+0000 7f9d2c682640 1 -- 172.21.15.5:0/704639091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d241a4ed0 con 0x7f9d2410c0d0 2024-09-11T22:36:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.534+0000 7f9d1b7fe640 1 -- 172.21.15.5:0/704639091 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9d1c005540 con 0x7f9d2410c0d0 2024-09-11T22:36:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.535+0000 7f9d2f422740 1 Processor -- start 2024-09-11T22:36:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.535+0000 7f9d2f422740 1 -- start start 2024-09-11T22:36:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.535+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d63f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.535+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f3410 0x5600d63fd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.535+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d64000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.535+0000 7f9d2f422740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5600d63572f0 con 0x5600d63f2af0 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.535+0000 7f9d2f422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600d62ff270 con 0x5600d63fdcb0 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.535+0000 7f9d2f422740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5600d63532b0 con 0x5600d63f3410 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.535+0000 7f9d1b7fe640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d100a2da0 0x7f9d100a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.535+0000 7f9d29c1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d63f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d2a41e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f3410 0x5600d63fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d29c1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d63f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32914/0 (socket says 172.21.15.5:32914) 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d29c1d640 1 -- 172.21.15.5:0/1271596985 learned_addr learned my addr 172.21.15.5:0/1271596985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/1271596985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d64000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d2ac1f640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d100a2da0 0x7f9d100a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d1b7fe640 1 -- 172.21.15.5:0/704639091 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f9d1c0cfac0 con 0x7f9d2410c0d0 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d2a41e640 1 -- 172.21.15.5:0/1271596985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 msgr2=0x5600d64000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/1271596985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d64000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d2a41e640 1 -- 172.21.15.5:0/1271596985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 msgr2=0x5600d63f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/1271596985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d63f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d2a41e640 1 -- 172.21.15.5:0/1271596985 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d62fe340 con 0x5600d63f3410 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/1271596985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f3410 0x5600d63fd770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9d20004510 tx=0x7f9d2000bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f4d35ebeef742f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d2ac1f640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d100a2da0 0x7f9d100a5260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0cd7a0 tx=0x7f9d1c06df30 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d18ff9640 1 -- 172.21.15.5:0/1271596985 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2000be00 con 0x5600d63f3410 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d18ff9640 1 -- 172.21.15.5:0/1271596985 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d20005500 con 0x5600d63f3410 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.536+0000 7f9d18ff9640 1 -- 172.21.15.5:0/1271596985 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d200038e0 con 0x5600d63f3410 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.537+0000 7f9d2f422740 1 -- 172.21.15.5:0/1271596985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f3410 msgr2=0x5600d63fd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:37.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.537+0000 7f9d2f422740 1 --2- 172.21.15.5:0/1271596985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f3410 0x5600d63fd770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9d20004510 tx=0x7f9d2000bbb0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.537+0000 7f9d2f422740 1 -- 172.21.15.5:0/1271596985 shutdown_connections 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.537+0000 7f9d2f422740 1 --2- 172.21.15.5:0/1271596985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d64000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.537+0000 7f9d2f422740 1 --2- 172.21.15.5:0/1271596985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f3410 0x5600d63fd770 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.537+0000 7f9d2f422740 1 --2- 172.21.15.5:0/1271596985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d63f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.537+0000 7f9d2f422740 1 -- 172.21.15.5:0/1271596985 >> 172.21.15.5:0/1271596985 conn(0x5600d63ef060 msgr2=0x5600d63ef440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.537+0000 7f9d2f422740 1 -- 172.21.15.5:0/1271596985 shutdown_connections 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.537+0000 7f9d2f422740 1 -- 172.21.15.5:0/1271596985 wait complete. 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.537+0000 7f9d2f422740 1 -- start start 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2f422740 1 Processor -- start 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 0x5600d63fab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d63fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2f422740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5600d6400880 con 0x5600d63f3410 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2f422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5600d62e0290 con 0x5600d63fdcb0 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2f422740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5600d6400a00 con 0x5600d63f2af0 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2f422740 10 client.0 _ll_register_callbacks cb 0x5600d6311140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2a41e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2a41e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32922/0 (socket says 172.21.15.5:32922) 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2a41e640 1 -- 172.21.15.5:0/41931553 learned_addr learned my addr 172.21.15.5:0/41931553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 0x5600d63fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d63fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2a41e640 1 -- 172.21.15.5:0/41931553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 msgr2=0x5600d63fab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 0x5600d63fab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2a41e640 1 -- 172.21.15.5:0/41931553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 msgr2=0x5600d63fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d63fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d2a41e640 1 -- 172.21.15.5:0/41931553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d606a210 con 0x5600d63f3410 2024-09-11T22:36:37.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.538+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 0x5600d63fab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb0a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000c010 tx=0x7f9d20004ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfd7e0f8ec91fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d0dffb640 1 -- 172.21.15.5:0/41931553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d200048d0 con 0x5600d63f3410 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d0dffb640 1 -- 172.21.15.5:0/41931553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f9d20018040 con 0x5600d63f3410 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d2f422740 1 -- 172.21.15.5:0/41931553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d20005500 con 0x5600d63f3410 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d0dffb640 1 -- 172.21.15.5:0/41931553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9d20004f10 con 0x5600d63f3410 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d2f422740 10 client.19899 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d2f422740 1 -- 172.21.15.5:0/41931553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5600d6136620 con 0x5600d63f3410 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d0dffb640 1 -- 172.21.15.5:0/41931553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d20005500 con 0x5600d63f3410 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d0dffb640 1 -- 172.21.15.5:0/41931553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f9d2001c470 con 0x5600d63f3410 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d2f422740 10 client.19899 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.539+0000 7f9d2f422740 10 client.19899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:37.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.540+0000 7f9d2f422740 20 client.19899 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.540+0000 7f9d2f422740 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600d640db90 0x5600d640ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.540+0000 7f9d2f422740 4 client.19899 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.540+0000 7f9d2f422740 1 -- 172.21.15.5:0/41931553 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600d6364a20 con 0x5600d640db90 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.540+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600d640db90 0x5600d640ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.540+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600d640db90 0x5600d640ff80 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa7a014ad5fc9aa server_cookie=2b72d383d16e1e60 in_seq=0 out_seq=0 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.540+0000 7f9d0dffb640 10 client.19899 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d0dffb640 1 -- 172.21.15.5:0/41931553 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600d6364a20 con 0x5600d640db90 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d0dffb640 10 client.19899 handle_command_reply: tid=0 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 client.19899 shutdown 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 -- 172.21.15.5:0/41931553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 msgr2=0x5600d63fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb0a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9d2000c010 tx=0x7f9d20004ca0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 -- 172.21.15.5:0/41931553 shutdown_connections 2024-09-11T22:36:37.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600d640db90 0x5600d640ff80 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d63fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb0a0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 --2- 172.21.15.5:0/41931553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 0x5600d63fab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 -- 172.21.15.5:0/41931553 >> 172.21.15.5:0/41931553 conn(0x5600d63ef060 msgr2=0x5600d63f4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d0dffb640 0 client.19899 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 -- 172.21.15.5:0/41931553 shutdown_connections 2024-09-11T22:36:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 1 -- 172.21.15.5:0/41931553 wait complete. 2024-09-11T22:36:37.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.541+0000 7f9d2f422740 20 client.19899 trim_cache size 0 max 16384 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2f422740 1 Processor -- start 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2f422740 1 -- start start 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 0x5600d63fcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d6415610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2f422740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241ab790 con 0x5600d63f3410 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2f422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241ad3c0 con 0x5600d63fdcb0 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2f422740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241abe60 con 0x5600d63f2af0 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2a41e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2a41e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32928/0 (socket says 172.21.15.5:32928) 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2a41e640 1 -- 172.21.15.5:0/3992382114 learned_addr learned my addr 172.21.15.5:0/3992382114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:37.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/3992382114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 0x5600d63fcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/3992382114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d6415610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2a41e640 1 -- 172.21.15.5:0/3992382114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 msgr2=0x5600d63fcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/3992382114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 0x5600d63fcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2a41e640 1 -- 172.21.15.5:0/3992382114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 msgr2=0x5600d6415610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/3992382114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d6415610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d2a41e640 1 -- 172.21.15.5:0/3992382114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600d63eee90 con 0x5600d63f3410 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.558+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/3992382114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 0x5600d63fcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.559+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/3992382114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb410 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f9d20000c00 tx=0x7f9d20004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aea33e0804033d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.559+0000 7f9d0ffff640 1 -- 172.21.15.5:0/3992382114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d200041b0 con 0x5600d63f3410 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.559+0000 7f9d0ffff640 1 -- 172.21.15.5:0/3992382114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d200048d0 con 0x5600d63f3410 2024-09-11T22:36:37.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.559+0000 7f9d0ffff640 1 -- 172.21.15.5:0/3992382114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20003920 con 0x5600d63f3410 2024-09-11T22:36:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 -- 172.21.15.5:0/3992382114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 msgr2=0x5600d63fb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 --2- 172.21.15.5:0/3992382114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb410 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f9d20000c00 tx=0x7f9d20004600 comp rx=0 tx=0).stop 2024-09-11T22:36:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 -- 172.21.15.5:0/3992382114 shutdown_connections 2024-09-11T22:36:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 --2- 172.21.15.5:0/3992382114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63fdcb0 0x5600d6415610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 --2- 172.21.15.5:0/3992382114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f3410 0x5600d63fb410 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 --2- 172.21.15.5:0/3992382114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63f2af0 0x5600d63fcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 -- 172.21.15.5:0/3992382114 >> 172.21.15.5:0/3992382114 conn(0x5600d63ef060 msgr2=0x5600d63fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 -- 172.21.15.5:0/3992382114 shutdown_connections 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 -- 172.21.15.5:0/3992382114 wait complete. 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 -- start start 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.560+0000 7f9d2f422740 1 Processor -- start 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d6405cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63f3410 0x5600d6406200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d2f422740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63fdcb0 0x5600d63f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d2f422740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241accf0 con 0x5600d63f2af0 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d2f422740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241ac560 con 0x5600d63f3410 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d2f422740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9d241a9790 con 0x5600d63fdcb0 2024-09-11T22:36:37.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d2f422740 10 client.0 _ll_register_callbacks cb 0x7f9d1c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d29c1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d6405cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d29c1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d6405cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32932/0 (socket says 172.21.15.5:32932) 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d29c1d640 1 -- 172.21.15.5:0/1089620349 learned_addr learned my addr 172.21.15.5:0/1089620349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63fdcb0 0x5600d63f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63f3410 0x5600d6406200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d29c1d640 1 -- 172.21.15.5:0/1089620349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63fdcb0 msgr2=0x5600d63f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63fdcb0 0x5600d63f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d29c1d640 1 -- 172.21.15.5:0/1089620349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63f3410 msgr2=0x5600d6406200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63f3410 0x5600d6406200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d29c1d640 1 -- 172.21.15.5:0/1089620349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600d63f8a00 con 0x5600d63f2af0 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d29c1d640 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d6405cc0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009230 tx=0x7f9d200056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73627f08417bd08f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:37.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.561+0000 7f9d0dffb640 1 -- 172.21.15.5:0/1089620349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14051040 con 0x5600d63f2af0 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d2f422740 1 -- 172.21.15.5:0/1089620349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d200048d0 con 0x5600d63f2af0 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d0dffb640 1 -- 172.21.15.5:0/1089620349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f9d140050f0 con 0x5600d63f2af0 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d0dffb640 1 -- 172.21.15.5:0/1089620349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9d14035040 con 0x5600d63f2af0 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d2f422740 10 client.19905 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d2f422740 1 -- 172.21.15.5:0/1089620349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5600d6477ab0 con 0x5600d63f2af0 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d0dffb640 1 -- 172.21.15.5:0/1089620349 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d200048d0 con 0x5600d63f2af0 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d0dffb640 1 -- 172.21.15.5:0/1089620349 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f9d140035f0 con 0x5600d63f2af0 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d2f422740 10 client.19905 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d2f422740 10 client.19905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d2f422740 20 client.19905 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d2f422740 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600d6478270 0x5600d647a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d2f422740 4 client.19905 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:36:37.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d2f422740 1 -- 172.21.15.5:0/1089620349 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600d635ed70 con 0x5600d6478270 2024-09-11T22:36:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.562+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600d6478270 0x5600d647a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.563+0000 7f9d2a41e640 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600d6478270 0x5600d647a690 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dae6cfe1e31c2e5b server_cookie=3efe48c8d4c0a904 in_seq=0 out_seq=0 2024-09-11T22:36:37.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.563+0000 7f9d0dffb640 10 client.19905 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:37.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 11.79 MB/sec warmup 55 sec latency 9440.211 ms 2024-09-11T22:36:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.686+0000 7f9d0dffb640 1 -- 172.21.15.5:0/1089620349 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600d635ed70 con 0x5600d6478270 2024-09-11T22:36:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.686+0000 7f9d0dffb640 10 client.19905 handle_command_reply: tid=0 2024-09-11T22:36:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.686+0000 7f9d2f422740 1 client.19905 shutdown 2024-09-11T22:36:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 1 -- 172.21.15.5:0/1089620349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 msgr2=0x5600d6405cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d6405cc0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009230 tx=0x7f9d200056b0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 1 -- 172.21.15.5:0/1089620349 shutdown_connections 2024-09-11T22:36:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5600d6478270 0x5600d647a690 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5600d63fdcb0 0x5600d63f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d0dffb640 0 client.19905 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5600d63f3410 0x5600d6406200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 1 --2- 172.21.15.5:0/1089620349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600d63f2af0 0x5600d6405cc0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 1 -- 172.21.15.5:0/1089620349 >> 172.21.15.5:0/1089620349 conn(0x5600d63ef060 msgr2=0x5600d63ee1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 1 -- 172.21.15.5:0/1089620349 shutdown_connections 2024-09-11T22:36:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 1 -- 172.21.15.5:0/1089620349 wait complete. 2024-09-11T22:36:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.687+0000 7f9d2f422740 20 client.19905 trim_cache size 0 max 16384 2024-09-11T22:36:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:36:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:36:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:36:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:36:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 -- 172.21.15.5:0/704639091 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d100a2da0 msgr2=0x7f9d100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d100a2da0 0x7f9d100a5260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0cd7a0 tx=0x7f9d1c06df30 comp rx=0 tx=0).stop 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 -- 172.21.15.5:0/704639091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2410c0d0 msgr2=0x7f9d2419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2410c0d0 0x7f9d2419cc20 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c009fd0 tx=0x7f9d1c00ef10 comp rx=0 tx=0).stop 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 -- 172.21.15.5:0/704639091 shutdown_connections 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2410c0d0 0x7f9d2419cc20 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9d100a2da0 0x7f9d100a5260 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d241035a0 0x7f9d241988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 --2- 172.21.15.5:0/704639091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d24102c80 0x7f9d24198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 -- 172.21.15.5:0/704639091 >> 172.21.15.5:0/704639091 conn(0x7f9d24077990 msgr2=0x7f9d240fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 -- 172.21.15.5:0/704639091 shutdown_connections 2024-09-11T22:36:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:37.688+0000 7f9d2c682640 1 -- 172.21.15.5:0/704639091 wait complete. 2024-09-11T22:36:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:37 smithi159 ceph-mon[35534]: pgmap v459: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-11T22:36:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:37 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:36:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:37 smithi172 ceph-mon[35461]: pgmap v459: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-11T22:36:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:37 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:36:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:37 smithi005 ceph-mon[30943]: pgmap v459: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 272 op/s 2024-09-11T22:36:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:37 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:36:38.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 11.58 MB/sec warmup 56 sec latency 10440.327 ms 2024-09-11T22:36:38.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:38 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:36:38.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:38 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:36:38.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:38 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:36:38.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:38 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:36:38.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:38 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:36:38.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:38 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:36:39.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:38 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:36:39.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:38 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:36:39.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:38 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:36:39.634 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 11.38 MB/sec warmup 57 sec latency 11440.427 ms 2024-09-11T22:36:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:39 smithi005 ceph-mon[30943]: pgmap v460: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-09-11T22:36:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:39 smithi159 ceph-mon[35534]: pgmap v460: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-09-11T22:36:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:39 smithi172 ceph-mon[35461]: pgmap v460: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-09-11T22:36:40.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 11.18 MB/sec warmup 58 sec latency 12440.517 ms 2024-09-11T22:36:41.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 10.99 MB/sec warmup 59 sec latency 13440.650 ms 2024-09-11T22:36:42.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:41 smithi005 ceph-mon[30943]: pgmap v461: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-09-11T22:36:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:41 smithi159 ceph-mon[35534]: pgmap v461: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-09-11T22:36:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:41 smithi172 ceph-mon[35461]: pgmap v461: 97 pgs: 97 active+clean; 331 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-09-11T22:36:42.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:36:42.546 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:42.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 10.81 MB/sec warmup 60 sec latency 14440.724 ms 2024-09-11T22:36:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.295+0000 7fb4bba88640 1 -- 172.21.15.5:0/793185626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4b4106a90 msgr2=0x7fb4b4106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:43.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.295+0000 7fb4bba88640 1 --2- 172.21.15.5:0/793185626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4b4106a90 0x7fb4b4106e70 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009930 tx=0x7fb4b002a420 comp rx=0 tx=0).stop 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.295+0000 7fb4bba88640 1 -- 172.21.15.5:0/793185626 shutdown_connections 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.295+0000 7fb4bba88640 1 --2- 172.21.15.5:0/793185626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4b4101430 0x7fb4b410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.295+0000 7fb4bba88640 1 --2- 172.21.15.5:0/793185626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b4100a90 0x7fb4b4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.295+0000 7fb4bba88640 1 --2- 172.21.15.5:0/793185626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4b4106a90 0x7fb4b4106e70 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.295+0000 7fb4bba88640 1 -- 172.21.15.5:0/793185626 >> 172.21.15.5:0/793185626 conn(0x7fb4b40fc7d0 msgr2=0x7fb4b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.295+0000 7fb4bba88640 1 -- 172.21.15.5:0/793185626 shutdown_connections 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.295+0000 7fb4bba88640 1 -- 172.21.15.5:0/793185626 wait complete. 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4bba88640 1 Processor -- start 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4bba88640 1 -- start start 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4bba88640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b4100a90 0x7fb4b4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4bba88640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4b4101430 0x7fb4b4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4bba88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4b4106a90 0x7fb4b419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4bba88640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4110e90 con 0x7fb4b4106a90 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4bba88640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4110d10 con 0x7fb4b4100a90 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4bba88640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4111010 con 0x7fb4b4101430 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4ba025640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4b4106a90 0x7fb4b419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4ba025640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4b4106a90 0x7fb4b419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60882/0 (socket says 172.21.15.5:60882) 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4ba025640 1 -- 172.21.15.5:0/3589153717 learned_addr learned my addr 172.21.15.5:0/3589153717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.296+0000 7fb4b9023640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4b4101430 0x7fb4b4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4b9824640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b4100a90 0x7fb4b4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4ba025640 1 -- 172.21.15.5:0/3589153717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4b4101430 msgr2=0x7fb4b4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4ba025640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4b4101430 0x7fb4b4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4ba025640 1 -- 172.21.15.5:0/3589153717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b4100a90 msgr2=0x7fb4b4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4ba025640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b4100a90 0x7fb4b4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4ba025640 1 -- 172.21.15.5:0/3589153717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b419d3e0 con 0x7fb4b4106a90 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4ba025640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4b4106a90 0x7fb4b419cd00 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00eaa0 tx=0x7fb4ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:43.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4aaffd640 1 -- 172.21.15.5:0/3589153717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00cdb0 con 0x7fb4b4106a90 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4aaffd640 1 -- 172.21.15.5:0/3589153717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4ac005020 con 0x7fb4b4106a90 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4bba88640 1 -- 172.21.15.5:0/3589153717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b419d6d0 con 0x7fb4b4106a90 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4aaffd640 1 -- 172.21.15.5:0/3589153717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac0116b0 con 0x7fb4b4106a90 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.297+0000 7fb4bba88640 1 -- 172.21.15.5:0/3589153717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b40fcd90 con 0x7fb4b4106a90 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.300+0000 7fb4aaffd640 1 -- 172.21.15.5:0/3589153717 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb4ac005560 con 0x7fb4b4106a90 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.300+0000 7fb4bde61740 1 Processor -- start 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.300+0000 7fb4bde61740 1 -- start start 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.300+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 0x55e4eb11c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.300+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb127140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb129a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4bde61740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4eb0822e0 con 0x55e4eb11cdc0 2024-09-11T22:36:43.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4bde61740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e4eb029ce0 con 0x55e4eb11c4a0 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4bde61740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e4eb07e2a0 con 0x55e4eb127680 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9824640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb127140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9824640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb127140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60886/0 (socket says 172.21.15.5:60886) 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9824640 1 -- 172.21.15.5:0/136814017 learned_addr learned my addr 172.21.15.5:0/136814017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9023640 1 --2- 172.21.15.5:0/136814017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb129a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9023640 1 --2- 172.21.15.5:0/136814017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 0x55e4eb11c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4aaffd640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4a00a2cf0 0x7fb4a00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9824640 1 -- 172.21.15.5:0/136814017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 msgr2=0x55e4eb129a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9824640 1 --2- 172.21.15.5:0/136814017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb129a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9824640 1 -- 172.21.15.5:0/136814017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 msgr2=0x55e4eb11c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9824640 1 --2- 172.21.15.5:0/136814017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 0x55e4eb11c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9824640 1 -- 172.21.15.5:0/136814017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4eb00e440 con 0x55e4eb11cdc0 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9023640 1 --2- 172.21.15.5:0/136814017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb129a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4aaffd640 1 -- 172.21.15.5:0/3589153717 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fb4ac0cf700 con 0x7fb4b4106a90 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4ba025640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4a00a2cf0 0x7fb4a00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.301+0000 7fb4b9824640 1 --2- 172.21.15.5:0/136814017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb127140 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb4b002a940 tx=0x7fb4b0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7bf3a12780838b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.302+0000 7fb49ffff640 1 -- 172.21.15.5:0/136814017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b003f070 con 0x55e4eb11cdc0 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.302+0000 7fb49ffff640 1 -- 172.21.15.5:0/136814017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b0004030 con 0x55e4eb11cdc0 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.302+0000 7fb49ffff640 1 -- 172.21.15.5:0/136814017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0035070 con 0x55e4eb11cdc0 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.302+0000 7fb4bde61740 1 -- 172.21.15.5:0/136814017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 msgr2=0x55e4eb127140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.302+0000 7fb4bde61740 1 --2- 172.21.15.5:0/136814017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb127140 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb4b002a940 tx=0x7fb4b0005e80 comp rx=0 tx=0).stop 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.302+0000 7fb4bde61740 1 -- 172.21.15.5:0/136814017 shutdown_connections 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.302+0000 7fb4bde61740 1 --2- 172.21.15.5:0/136814017 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb129a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.302+0000 7fb4bde61740 1 --2- 172.21.15.5:0/136814017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb127140 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.302+0000 7fb4bde61740 1 --2- 172.21.15.5:0/136814017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 0x55e4eb11c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.303+0000 7fb4bde61740 1 -- 172.21.15.5:0/136814017 >> 172.21.15.5:0/136814017 conn(0x55e4eb118a30 msgr2=0x55e4eb118e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:43.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.303+0000 7fb4ba025640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4a00a2cf0 0x7fb4a00a51b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac005910 tx=0x7fb4ac009270 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.303+0000 7fb4bde61740 1 -- 172.21.15.5:0/136814017 shutdown_connections 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.303+0000 7fb4bde61740 1 -- 172.21.15.5:0/136814017 wait complete. 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.303+0000 7fb4bde61740 1 -- start start 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.303+0000 7fb4bde61740 1 Processor -- start 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb11c4a0 0x55e4eb126f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11cdc0 0x55e4eb124220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb127680 0x55e4eb124760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4bde61740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4eb12a250 con 0x55e4eb127680 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4bde61740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e4eb03d050 con 0x55e4eb11cdc0 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4bde61740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e4eb12a3d0 con 0x55e4eb11c4a0 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4bde61740 10 client.0 _ll_register_callbacks cb 0x55e4eb07d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4b9023640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb127680 0x55e4eb124760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4b9023640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb127680 0x55e4eb124760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60894/0 (socket says 172.21.15.5:60894) 2024-09-11T22:36:43.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4b9023640 1 -- 172.21.15.5:0/2322388511 learned_addr learned my addr 172.21.15.5:0/2322388511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4b9824640 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11cdc0 0x55e4eb124220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4b9023640 1 -- 172.21.15.5:0/2322388511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb11c4a0 msgr2=0x55e4eb126f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4b9023640 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb11c4a0 0x55e4eb126f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4b9023640 1 -- 172.21.15.5:0/2322388511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11cdc0 msgr2=0x55e4eb124220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4b9023640 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11cdc0 0x55e4eb124220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.304+0000 7fb4b9023640 1 -- 172.21.15.5:0/2322388511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4eadf33c0 con 0x55e4eb127680 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.305+0000 7fb4b9023640 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb127680 0x55e4eb124760 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400be30 tx=0x7fb4a4012a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564013edb8b9ebde server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.305+0000 7fb49d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.305+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2322388511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a400f8b0 con 0x55e4eb127680 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.305+0000 7fb4bde61740 1 -- 172.21.15.5:0/2322388511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b0004030 con 0x55e4eb127680 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.305+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2322388511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fb4a400fa50 con 0x55e4eb127680 2024-09-11T22:36:43.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.305+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2322388511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb4a402a050 con 0x55e4eb127680 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.305+0000 7fb4bde61740 10 client.19917 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.305+0000 7fb4bde61740 1 -- 172.21.15.5:0/2322388511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e4eb135c70 con 0x55e4eb127680 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.305+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2322388511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b0004030 con 0x55e4eb127680 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.305+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2322388511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fb4a401b070 con 0x55e4eb127680 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.306+0000 7fb4bde61740 10 client.19917 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.306+0000 7fb4bde61740 10 client.19917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.306+0000 7fb4bde61740 20 client.19917 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.306+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4eb137100 0x55e4eb139540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.306+0000 7fb4bde61740 4 client.19917 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.306+0000 7fb4bde61740 1 -- 172.21.15.5:0/2322388511 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4eb08f850 con 0x55e4eb137100 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.306+0000 7fb4b9824640 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4eb137100 0x55e4eb139540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.306+0000 7fb4b9824640 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4eb137100 0x55e4eb139540 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b2abdea99b505b server_cookie=f6728666d7e55db4 in_seq=0 out_seq=0 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.306+0000 7fb49d7fa640 10 client.19917 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2322388511 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4eb08f850 con 0x55e4eb137100 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb49d7fa640 10 client.19917 handle_command_reply: tid=0 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 client.19917 shutdown 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 -- 172.21.15.5:0/2322388511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb127680 msgr2=0x55e4eb124760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb127680 0x55e4eb124760 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400be30 tx=0x7fb4a4012a40 comp rx=0 tx=0).stop 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 -- 172.21.15.5:0/2322388511 shutdown_connections 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4eb137100 0x55e4eb139540 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb127680 0x55e4eb124760 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11cdc0 0x55e4eb124220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2322388511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb11c4a0 0x55e4eb126f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb49d7fa640 0 client.19917 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 -- 172.21.15.5:0/2322388511 >> 172.21.15.5:0/2322388511 conn(0x55e4eb118a30 msgr2=0x55e4eb117b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 -- 172.21.15.5:0/2322388511 shutdown_connections 2024-09-11T22:36:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.307+0000 7fb4bde61740 1 -- 172.21.15.5:0/2322388511 wait complete. 2024-09-11T22:36:43.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.308+0000 7fb4bde61740 20 client.19917 trim_cache size 0 max 16384 2024-09-11T22:36:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.325+0000 7fb4bde61740 1 Processor -- start 2024-09-11T22:36:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.325+0000 7fb4bde61740 1 -- start start 2024-09-11T22:36:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.325+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 0x55e4eb1334b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.325+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb13ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.325+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb13f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.325+0000 7fb4bde61740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41afd00 con 0x55e4eb11cdc0 2024-09-11T22:36:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.325+0000 7fb4bde61740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41b1930 con 0x55e4eb11c4a0 2024-09-11T22:36:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.325+0000 7fb4bde61740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41b03d0 con 0x55e4eb127680 2024-09-11T22:36:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9023640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 0x55e4eb1334b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9824640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb13ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9023640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 0x55e4eb1334b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51584/0 (socket says 172.21.15.5:51584) 2024-09-11T22:36:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9023640 1 -- 172.21.15.5:0/1727736332 learned_addr learned my addr 172.21.15.5:0/1727736332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:43.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9023640 1 --2- 172.21.15.5:0/1727736332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb13f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9824640 1 -- 172.21.15.5:0/1727736332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 msgr2=0x55e4eb13f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9824640 1 --2- 172.21.15.5:0/1727736332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb13f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9824640 1 -- 172.21.15.5:0/1727736332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 msgr2=0x55e4eb1334b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9824640 1 --2- 172.21.15.5:0/1727736332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 0x55e4eb1334b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9824640 1 -- 172.21.15.5:0/1727736332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4eb11da10 con 0x55e4eb11cdc0 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9023640 1 --2- 172.21.15.5:0/1727736332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 0x55e4eb1334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb4b9824640 1 --2- 172.21.15.5:0/1727736332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb13ee10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0006010 tx=0x7fb4b002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d4ecc1eeb71285 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb49f7fe640 1 -- 172.21.15.5:0/1727736332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0035050 con 0x55e4eb11cdc0 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.326+0000 7fb49f7fe640 1 -- 172.21.15.5:0/1727736332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4b0004040 con 0x55e4eb11cdc0 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.327+0000 7fb49f7fe640 1 -- 172.21.15.5:0/1727736332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b003f040 con 0x55e4eb11cdc0 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.327+0000 7fb4bde61740 1 -- 172.21.15.5:0/1727736332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 msgr2=0x55e4eb13ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:43.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.327+0000 7fb4bde61740 1 --2- 172.21.15.5:0/1727736332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb13ee10 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0006010 tx=0x7fb4b002a420 comp rx=0 tx=0).stop 2024-09-11T22:36:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.327+0000 7fb4bde61740 1 -- 172.21.15.5:0/1727736332 shutdown_connections 2024-09-11T22:36:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.327+0000 7fb4bde61740 1 --2- 172.21.15.5:0/1727736332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb13f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.327+0000 7fb4bde61740 1 --2- 172.21.15.5:0/1727736332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11cdc0 0x55e4eb13ee10 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.327+0000 7fb4bde61740 1 --2- 172.21.15.5:0/1727736332 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11c4a0 0x55e4eb1334b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.327+0000 7fb4bde61740 1 -- 172.21.15.5:0/1727736332 >> 172.21.15.5:0/1727736332 conn(0x55e4eb118a30 msgr2=0x55e4eb122af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.328+0000 7fb4bde61740 1 -- 172.21.15.5:0/1727736332 shutdown_connections 2024-09-11T22:36:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.328+0000 7fb4bde61740 1 -- 172.21.15.5:0/1727736332 wait complete. 2024-09-11T22:36:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.328+0000 7fb4bde61740 1 -- start start 2024-09-11T22:36:43.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.328+0000 7fb4bde61740 1 Processor -- start 2024-09-11T22:36:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11c4a0 0x55e4eb19d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11cdc0 0x55e4eb12f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4bde61740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb12fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4bde61740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41b1260 con 0x55e4eb11c4a0 2024-09-11T22:36:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4bde61740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41b0ad0 con 0x55e4eb11cdc0 2024-09-11T22:36:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4bde61740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b41add00 con 0x55e4eb127680 2024-09-11T22:36:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4b9023640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11c4a0 0x55e4eb19d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4b9023640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11c4a0 0x55e4eb19d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60910/0 (socket says 172.21.15.5:60910) 2024-09-11T22:36:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4bde61740 10 client.0 _ll_register_callbacks cb 0x7fb4a400bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:43.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4b9023640 1 -- 172.21.15.5:0/2786820151 learned_addr learned my addr 172.21.15.5:0/2786820151 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4b9824640 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11cdc0 0x55e4eb12f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4b9023640 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb12fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4b9023640 1 -- 172.21.15.5:0/2786820151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 msgr2=0x55e4eb12fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4b9023640 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb12fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4b9023640 1 -- 172.21.15.5:0/2786820151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11cdc0 msgr2=0x55e4eb12f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4b9023640 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11cdc0 0x55e4eb12f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.329+0000 7fb4b9023640 1 -- 172.21.15.5:0/2786820151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4eb19ac20 con 0x55e4eb11c4a0 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.330+0000 7fb4b9023640 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11c4a0 0x55e4eb19d850 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400be30 tx=0x7fb4a4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd532b33892e2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.330+0000 7fb49d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.330+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2786820151 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4036030 con 0x55e4eb11c4a0 2024-09-11T22:36:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.330+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2786820151 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fb4a4004220 con 0x55e4eb11c4a0 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.330+0000 7fb4bde61740 1 -- 172.21.15.5:0/2786820151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b0004040 con 0x55e4eb11c4a0 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.330+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2786820151 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fb4a4043050 con 0x55e4eb11c4a0 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.330+0000 7fb4bde61740 10 client.19923 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.330+0000 7fb4bde61740 1 -- 172.21.15.5:0/2786820151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e4eb1aa950 con 0x55e4eb11c4a0 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.330+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2786820151 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a4021070 con 0x55e4eb11c4a0 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.330+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2786820151 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fb4a401b040 con 0x55e4eb11c4a0 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.331+0000 7fb4bde61740 10 client.19923 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.331+0000 7fb4bde61740 10 client.19923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.331+0000 7fb4bde61740 20 client.19923 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.331+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4eb1ab160 0x55e4eb1ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.331+0000 7fb4bde61740 4 client.19923 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.331+0000 7fb4bde61740 1 -- 172.21.15.5:0/2786820151 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb4b0004040 con 0x55e4eb1ab160 2024-09-11T22:36:43.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.331+0000 7fb4b9824640 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4eb1ab160 0x55e4eb1ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.331+0000 7fb49d7fa640 10 client.19923 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.331+0000 7fb4b9824640 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4eb1ab160 0x55e4eb1ad580 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=242dab1753c0e20e server_cookie=9450e32843a1c6e3 in_seq=0 out_seq=0 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb49d7fa640 1 -- 172.21.15.5:0/2786820151 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fb4b0004040 con 0x55e4eb1ab160 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb49d7fa640 10 client.19923 handle_command_reply: tid=0 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 client.19923 shutdown 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 -- 172.21.15.5:0/2786820151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11c4a0 msgr2=0x55e4eb19d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11c4a0 0x55e4eb19d850 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fb4a400be30 tx=0x7fb4a4014040 comp rx=0 tx=0).stop 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 -- 172.21.15.5:0/2786820151 shutdown_connections 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4eb1ab160 0x55e4eb1ad580 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4eb127680 0x55e4eb12fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb49d7fa640 0 client.19923 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:43.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4eb11cdc0 0x55e4eb12f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 --2- 172.21.15.5:0/2786820151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4eb11c4a0 0x55e4eb19d850 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 -- 172.21.15.5:0/2786820151 >> 172.21.15.5:0/2786820151 conn(0x55e4eb118a30 msgr2=0x55e4eb125a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 -- 172.21.15.5:0/2786820151 shutdown_connections 2024-09-11T22:36:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 1 -- 172.21.15.5:0/2786820151 wait complete. 2024-09-11T22:36:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.332+0000 7fb4bde61740 20 client.19923 trim_cache size 0 max 16384 2024-09-11T22:36:43.337 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:36:43.337 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:36:43.337 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:36:43.337 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:36:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 -- 172.21.15.5:0/3589153717 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4a00a2cf0 msgr2=0x7fb4a00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4a00a2cf0 0x7fb4a00a51b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac005910 tx=0x7fb4ac009270 comp rx=0 tx=0).stop 2024-09-11T22:36:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 -- 172.21.15.5:0/3589153717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4b4106a90 msgr2=0x7fb4b419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4b4106a90 0x7fb4b419cd00 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00eaa0 tx=0x7fb4ac00ef60 comp rx=0 tx=0).stop 2024-09-11T22:36:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 -- 172.21.15.5:0/3589153717 shutdown_connections 2024-09-11T22:36:43.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4b4106a90 0x7fb4b419cd00 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4b4101430 0x7fb4b4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb4a00a2cf0 0x7fb4a00a51b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 --2- 172.21.15.5:0/3589153717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb4b4100a90 0x7fb4b4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:43.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 -- 172.21.15.5:0/3589153717 >> 172.21.15.5:0/3589153717 conn(0x7fb4b40fc7d0 msgr2=0x7fb4b406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:43.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 -- 172.21.15.5:0/3589153717 shutdown_connections 2024-09-11T22:36:43.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:43.333+0000 7fb4bba88640 1 -- 172.21.15.5:0/3589153717 wait complete. 2024-09-11T22:36:43.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 10.63 MB/sec warmup 61 sec latency 15440.789 ms 2024-09-11T22:36:44.006 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:43 smithi005 ceph-mon[30943]: pgmap v462: 97 pgs: 97 active+clean; 332 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-11T22:36:44.034 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:36:44.034 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:36:44.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:43 smithi159 ceph-mon[35534]: pgmap v462: 97 pgs: 97 active+clean; 332 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-11T22:36:44.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:43 smithi172 ceph-mon[35461]: pgmap v462: 97 pgs: 97 active+clean; 332 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-11T22:36:44.295 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:44.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 10.46 MB/sec warmup 62 sec latency 16440.838 ms 2024-09-11T22:36:44.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.844+0000 7f3c0723a640 1 -- 172.21.15.5:0/947178922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c00102790 msgr2=0x7f3c001067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:44.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.844+0000 7f3c0723a640 1 --2- 172.21.15.5:0/947178922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c00102790 0x7f3c001067e0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009900 tx=0x7f3bec02a410 comp rx=0 tx=0).stop 2024-09-11T22:36:44.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.844+0000 7f3c0723a640 1 -- 172.21.15.5:0/947178922 shutdown_connections 2024-09-11T22:36:44.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.844+0000 7f3c0723a640 1 --2- 172.21.15.5:0/947178922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c00106f10 0x7f3c0010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:44.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.844+0000 7f3c0723a640 1 --2- 172.21.15.5:0/947178922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c00102790 0x7f3c001067e0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:44.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.844+0000 7f3c0723a640 1 --2- 172.21.15.5:0/947178922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c00101de0 0x7f3c001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:44.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.844+0000 7f3c0723a640 1 -- 172.21.15.5:0/947178922 >> 172.21.15.5:0/947178922 conn(0x7f3c000fd8c0 msgr2=0x7f3c000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:44.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.844+0000 7f3c0723a640 1 -- 172.21.15.5:0/947178922 shutdown_connections 2024-09-11T22:36:44.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.844+0000 7f3c0723a640 1 -- 172.21.15.5:0/947178922 wait complete. 2024-09-11T22:36:44.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c0723a640 1 Processor -- start 2024-09-11T22:36:44.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c0723a640 1 -- start start 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c0723a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c00101de0 0x7f3c001985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c0723a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c00102790 0x7f3c00198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c0723a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c00106f10 0x7f3c0019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c0723a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00112010 con 0x7f3c00101de0 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c0723a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00111e90 con 0x7f3c00102790 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c0723a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00112190 con 0x7f3c00106f10 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c04fd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c00101de0 0x7f3c001985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c04fd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c00101de0 0x7f3c001985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60918/0 (socket says 172.21.15.5:60918) 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c04fd6640 1 -- 172.21.15.5:0/1671921089 learned_addr learned my addr 172.21.15.5:0/1671921089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3bfffff640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c00102790 0x7f3c00198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c057d7640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c00106f10 0x7f3c0019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:44.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c04fd6640 1 -- 172.21.15.5:0/1671921089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c00106f10 msgr2=0x7f3c0019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c04fd6640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c00106f10 0x7f3c0019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c04fd6640 1 -- 172.21.15.5:0/1671921089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c00102790 msgr2=0x7f3c00198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.845+0000 7f3c04fd6640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c00102790 0x7f3c00198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.846+0000 7f3c04fd6640 1 -- 172.21.15.5:0/1671921089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0019d560 con 0x7f3c00101de0 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.846+0000 7f3bfffff640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c00102790 0x7f3c00198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.846+0000 7f3c04fd6640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c00101de0 0x7f3c001985b0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400b4a0 tx=0x7f3bf400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.846+0000 7f3bfdffb640 1 -- 172.21.15.5:0/1671921089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4014070 con 0x7f3c00101de0 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.846+0000 7f3bfdffb640 1 -- 172.21.15.5:0/1671921089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf4010af0 con 0x7f3c00101de0 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.846+0000 7f3c0723a640 1 -- 172.21.15.5:0/1671921089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0019d850 con 0x7f3c00101de0 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.846+0000 7f3bfdffb640 1 -- 172.21.15.5:0/1671921089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4017d10 con 0x7f3c00101de0 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.846+0000 7f3c0723a640 1 -- 172.21.15.5:0/1671921089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c001a5130 con 0x7f3c00101de0 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.847+0000 7f3c0723a640 1 -- 172.21.15.5:0/1671921089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c00102240 con 0x7f3c00101de0 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.848+0000 7f3bfdffb640 1 -- 172.21.15.5:0/1671921089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3bf4017070 con 0x7f3c00101de0 2024-09-11T22:36:44.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.849+0000 7f3bfdffb640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3bf00a2d70 0x7f3bf00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.849+0000 7f3bfdffb640 1 -- 172.21.15.5:0/1671921089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f3bf40ce750 con 0x7f3c00101de0 2024-09-11T22:36:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.849+0000 7f3bfffff640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3bf00a2d70 0x7f3bf00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.849+0000 7f3bfffff640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3bf00a2d70 0x7f3bf00a5230 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3c000fdd50 tx=0x7f3bec005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:44.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:44.853+0000 7f3bfdffb640 1 -- 172.21.15.5:0/1671921089 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3bf409b410 con 0x7f3c00101de0 2024-09-11T22:36:45.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.064+0000 7f3c0723a640 1 -- 172.21.15.5:0/1671921089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c0010a150 con 0x7f3c00101de0 2024-09-11T22:36:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.065+0000 7f3bfdffb640 1 -- 172.21.15.5:0/1671921089 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11975 (secure 0 0 0) 0x7f3bf409ad10 con 0x7f3c00101de0 2024-09-11T22:36:45.066 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:36:45.066 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"btime":"2024-09-11T22:35:49:548013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:35:48.842139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:36:45.067 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 50 2024-09-11T22:36:45.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.067+0000 7f3c0723a640 1 -- 172.21.15.5:0/1671921089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3bf00a2d70 msgr2=0x7f3bf00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.067+0000 7f3c0723a640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3bf00a2d70 0x7f3bf00a5230 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3c000fdd50 tx=0x7f3bec005a20 comp rx=0 tx=0).stop 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.067+0000 7f3c0723a640 1 -- 172.21.15.5:0/1671921089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c00101de0 msgr2=0x7f3c001985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.067+0000 7f3c0723a640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c00101de0 0x7f3c001985b0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400b4a0 tx=0x7f3bf400b960 comp rx=0 tx=0).stop 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.067+0000 7f3c0723a640 1 -- 172.21.15.5:0/1671921089 shutdown_connections 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.067+0000 7f3c0723a640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c00106f10 0x7f3c0019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.067+0000 7f3c0723a640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3bf00a2d70 0x7f3bf00a5230 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.067+0000 7f3c0723a640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3c00102790 0x7f3c00198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.067+0000 7f3c0723a640 1 --2- 172.21.15.5:0/1671921089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c00101de0 0x7f3c001985b0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.067+0000 7f3c0723a640 1 -- 172.21.15.5:0/1671921089 >> 172.21.15.5:0/1671921089 conn(0x7f3c000fd8c0 msgr2=0x7f3c000fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.068+0000 7f3c0723a640 1 -- 172.21.15.5:0/1671921089 shutdown_connections 2024-09-11T22:36:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:45.068+0000 7f3c0723a640 1 -- 172.21.15.5:0/1671921089 wait complete. 2024-09-11T22:36:45.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 10.30 MB/sec warmup 63 sec latency 17440.924 ms 2024-09-11T22:36:45.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:36:46.020 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:46.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:45 smithi005 ceph-mon[30943]: pgmap v463: 97 pgs: 97 active+clean; 332 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 843 KiB/s wr, 77 op/s 2024-09-11T22:36:46.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:45 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1671921089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:46.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:45 smithi159 ceph-mon[35534]: pgmap v463: 97 pgs: 97 active+clean; 332 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 843 KiB/s wr, 77 op/s 2024-09-11T22:36:46.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:45 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1671921089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:46.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:45 smithi172 ceph-mon[35461]: pgmap v463: 97 pgs: 97 active+clean; 332 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 843 KiB/s wr, 77 op/s 2024-09-11T22:36:46.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:45 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1671921089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:46.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 10.13 MB/sec warmup 64 sec latency 18440.992 ms 2024-09-11T22:36:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.658+0000 7f56c4346640 1 -- 172.21.15.5:0/1163668555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc101400 msgr2=0x7f56bc10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.658+0000 7f56c4346640 1 --2- 172.21.15.5:0/1163668555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc101400 0x7f56bc10d930 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009cd0 tx=0x7f56b402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.658+0000 7f56c4346640 1 -- 172.21.15.5:0/1163668555 shutdown_connections 2024-09-11T22:36:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.658+0000 7f56c4346640 1 --2- 172.21.15.5:0/1163668555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc101400 0x7f56bc10d930 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.658+0000 7f56c4346640 1 --2- 172.21.15.5:0/1163668555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc100a60 0x7f56bc100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.658+0000 7f56c4346640 1 --2- 172.21.15.5:0/1163668555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56bc106a60 0x7f56bc106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.658+0000 7f56c4346640 1 -- 172.21.15.5:0/1163668555 >> 172.21.15.5:0/1163668555 conn(0x7f56bc0fc780 msgr2=0x7f56bc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.658+0000 7f56c4346640 1 -- 172.21.15.5:0/1163668555 shutdown_connections 2024-09-11T22:36:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.658+0000 7f56c4346640 1 -- 172.21.15.5:0/1163668555 wait complete. 2024-09-11T22:36:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c4346640 1 Processor -- start 2024-09-11T22:36:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c4346640 1 -- start start 2024-09-11T22:36:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c4346640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc100a60 0x7f56bc1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c4346640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc101400 0x7f56bc199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c4346640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56bc106a60 0x7f56bc1a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c4346640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc110e60 con 0x7f56bc101400 2024-09-11T22:36:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c4346640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc110ce0 con 0x7f56bc106a60 2024-09-11T22:36:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c4346640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc110fe0 con 0x7f56bc100a60 2024-09-11T22:36:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c18e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc101400 0x7f56bc199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c18e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc101400 0x7f56bc199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60932/0 (socket says 172.21.15.5:60932) 2024-09-11T22:36:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c18e1640 1 -- 172.21.15.5:0/251996902 learned_addr learned my addr 172.21.15.5:0/251996902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c20e2640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc100a60 0x7f56bc1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c28e3640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56bc106a60 0x7f56bc1a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c18e1640 1 -- 172.21.15.5:0/251996902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc100a60 msgr2=0x7f56bc1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c18e1640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc100a60 0x7f56bc1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c18e1640 1 -- 172.21.15.5:0/251996902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56bc106a60 msgr2=0x7f56bc1a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c18e1640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56bc106a60 0x7f56bc1a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c18e1640 1 -- 172.21.15.5:0/251996902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc1a4790 con 0x7f56bc101400 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.659+0000 7f56c28e3640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56bc106a60 0x7f56bc1a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.660+0000 7f56c18e1640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc101400 0x7f56bc199710 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00d770 tx=0x7f56ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.660+0000 7f56b37fe640 1 -- 172.21.15.5:0/251996902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac017070 con 0x7f56bc101400 2024-09-11T22:36:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.660+0000 7f56c4346640 1 -- 172.21.15.5:0/251996902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc1a4960 con 0x7f56bc101400 2024-09-11T22:36:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.660+0000 7f56b37fe640 1 -- 172.21.15.5:0/251996902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ac005500 con 0x7f56bc101400 2024-09-11T22:36:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.660+0000 7f56b37fe640 1 -- 172.21.15.5:0/251996902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac004e00 con 0x7f56bc101400 2024-09-11T22:36:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.660+0000 7f56c4346640 1 -- 172.21.15.5:0/251996902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc1a4e50 con 0x7f56bc101400 2024-09-11T22:36:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.662+0000 7f56b37fe640 1 -- 172.21.15.5:0/251996902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f56ac004fa0 con 0x7f56bc101400 2024-09-11T22:36:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c671f740 1 Processor -- start 2024-09-11T22:36:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c671f740 1 -- start start 2024-09-11T22:36:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c671f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc1b40 0x558743dc0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c671f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dc05d0 0x558743dc0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c671f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dcb370 0x558743dcd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c671f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558743d262f0 con 0x558743dcb370 2024-09-11T22:36:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c671f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558743ccdcf0 con 0x558743dc1b40 2024-09-11T22:36:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c671f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558743d222b0 con 0x558743dc05d0 2024-09-11T22:36:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c18e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dcb370 0x558743dcd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c18e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dcb370 0x558743dcd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60948/0 (socket says 172.21.15.5:60948) 2024-09-11T22:36:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56b37fe640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56a80a2cf0 0x7f56a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c18e1640 1 -- 172.21.15.5:0/3334495012 learned_addr learned my addr 172.21.15.5:0/3334495012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.663+0000 7f56c20e2640 1 --2- 172.21.15.5:0/3334495012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dc05d0 0x558743dc0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56c28e3640 1 --2- 172.21.15.5:0/3334495012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc1b40 0x558743dc0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56c28e3640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56a80a2cf0 0x7f56a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56b37fe640 1 -- 172.21.15.5:0/251996902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f56ac0ceff0 con 0x7f56bc101400 2024-09-11T22:36:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56c20e2640 1 -- 172.21.15.5:0/3334495012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc1b40 msgr2=0x558743dc0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56c20e2640 1 --2- 172.21.15.5:0/3334495012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc1b40 0x558743dc0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56c20e2640 1 -- 172.21.15.5:0/3334495012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dcb370 msgr2=0x558743dcd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56c20e2640 1 --2- 172.21.15.5:0/3334495012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dcb370 0x558743dcd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56c20e2640 1 -- 172.21.15.5:0/3334495012 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558743cb23a0 con 0x558743dc05d0 2024-09-11T22:36:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56c18e1640 1 --2- 172.21.15.5:0/3334495012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dcb370 0x558743dcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56c20e2640 1 --2- 172.21.15.5:0/3334495012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dc05d0 0x558743dc0a10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f56b800cb20 tx=0x7f56b800e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9d53b4c7523d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56c28e3640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56a80a2cf0 0x7f56a80a51b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009a60 tx=0x7f56b40095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56b0dea640 1 -- 172.21.15.5:0/3334495012 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8014070 con 0x558743dc05d0 2024-09-11T22:36:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56b0dea640 1 -- 172.21.15.5:0/3334495012 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b8005020 con 0x558743dc05d0 2024-09-11T22:36:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.664+0000 7f56b0dea640 1 -- 172.21.15.5:0/3334495012 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8004d00 con 0x558743dc05d0 2024-09-11T22:36:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.665+0000 7f56c671f740 1 -- 172.21.15.5:0/3334495012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dc05d0 msgr2=0x558743dc0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.665+0000 7f56c671f740 1 --2- 172.21.15.5:0/3334495012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dc05d0 0x558743dc0a10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f56b800cb20 tx=0x7f56b800e350 comp rx=0 tx=0).stop 2024-09-11T22:36:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.665+0000 7f56c671f740 1 -- 172.21.15.5:0/3334495012 shutdown_connections 2024-09-11T22:36:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.665+0000 7f56c671f740 1 --2- 172.21.15.5:0/3334495012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dcb370 0x558743dcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.665+0000 7f56c671f740 1 --2- 172.21.15.5:0/3334495012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dc05d0 0x558743dc0a10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.665+0000 7f56c671f740 1 --2- 172.21.15.5:0/3334495012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc1b40 0x558743dc0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.665+0000 7f56c671f740 1 -- 172.21.15.5:0/3334495012 >> 172.21.15.5:0/3334495012 conn(0x558743d89310 msgr2=0x558743dbcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.665+0000 7f56c671f740 1 -- 172.21.15.5:0/3334495012 shutdown_connections 2024-09-11T22:36:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.665+0000 7f56c671f740 1 -- 172.21.15.5:0/3334495012 wait complete. 2024-09-11T22:36:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c671f740 1 -- start start 2024-09-11T22:36:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c671f740 1 Processor -- start 2024-09-11T22:36:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c671f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dcad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c671f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c671f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dc8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c671f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558743dce070 con 0x558743dc1b40 2024-09-11T22:36:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c671f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558743ce1060 con 0x558743dc05d0 2024-09-11T22:36:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c671f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558743dce1f0 con 0x558743dcb370 2024-09-11T22:36:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c671f740 10 client.0 _ll_register_callbacks cb 0x558743a348c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c18e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c18e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60952/0 (socket says 172.21.15.5:60952) 2024-09-11T22:36:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c18e1640 1 -- 172.21.15.5:0/1215285386 learned_addr learned my addr 172.21.15.5:0/1215285386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c28e3640 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dc8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.666+0000 7f56c20e2640 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dcad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56c18e1640 1 -- 172.21.15.5:0/1215285386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 msgr2=0x558743dc8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56c18e1640 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dc8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56c18e1640 1 -- 172.21.15.5:0/1215285386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 msgr2=0x558743dcad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56c18e1640 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dcad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56c18e1640 1 -- 172.21.15.5:0/1215285386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558743a973c0 con 0x558743dc1b40 2024-09-11T22:36:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56c18e1640 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc7fb0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f56ac09b430 tx=0x7f56ac09b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760b4187e1cb4aa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1215285386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac020040 con 0x558743dc1b40 2024-09-11T22:36:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56c671f740 1 -- 172.21.15.5:0/1215285386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b8005020 con 0x558743dc1b40 2024-09-11T22:36:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1215285386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f56ac0cc950 con 0x558743dc1b40 2024-09-11T22:36:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1215285386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f56ac09b4a0 con 0x558743dc1b40 2024-09-11T22:36:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56c671f740 10 client.19941 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56c671f740 1 -- 172.21.15.5:0/1215285386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558743dd9ad0 con 0x558743dc1b40 2024-09-11T22:36:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1215285386 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b8005020 con 0x558743dc1b40 2024-09-11T22:36:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.667+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1215285386 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f56ac09a8f0 con 0x558743dc1b40 2024-09-11T22:36:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.668+0000 7f56c671f740 10 client.19941 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.668+0000 7f56c671f740 10 client.19941 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:36:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.668+0000 7f56c671f740 20 client.19941 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.668+0000 7f56c671f740 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558743ddaee0 0x558743ddd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.668+0000 7f56c671f740 4 client.19941 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.668+0000 7f56c671f740 1 -- 172.21.15.5:0/1215285386 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558743d33860 con 0x558743ddaee0 2024-09-11T22:36:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.668+0000 7f56c20e2640 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558743ddaee0 0x558743ddd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.668+0000 7f56a5ffb640 10 client.19941 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:36:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.668+0000 7f56c20e2640 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558743ddaee0 0x558743ddd2d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ad25280610604ff server_cookie=d80a04a49348f324 in_seq=0 out_seq=0 2024-09-11T22:36:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1215285386 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558743d33860 con 0x558743ddaee0 2024-09-11T22:36:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56a5ffb640 10 client.19941 handle_command_reply: tid=0 2024-09-11T22:36:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56c671f740 1 client.19941 shutdown 2024-09-11T22:36:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56c671f740 1 -- 172.21.15.5:0/1215285386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 msgr2=0x558743dc7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56c671f740 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc7fb0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f56ac09b430 tx=0x7f56ac09b0a0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56c671f740 1 -- 172.21.15.5:0/1215285386 shutdown_connections 2024-09-11T22:36:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56c671f740 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558743ddaee0 0x558743ddd2d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56c671f740 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dc8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56c671f740 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc7fb0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56a5ffb640 0 client.19941 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:36:46.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56c671f740 1 --2- 172.21.15.5:0/1215285386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dcad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56c671f740 1 -- 172.21.15.5:0/1215285386 >> 172.21.15.5:0/1215285386 conn(0x558743d89310 msgr2=0x558743dc2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:46.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.669+0000 7f56c671f740 1 -- 172.21.15.5:0/1215285386 shutdown_connections 2024-09-11T22:36:46.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.670+0000 7f56c671f740 1 -- 172.21.15.5:0/1215285386 wait complete. 2024-09-11T22:36:46.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.670+0000 7f56c671f740 20 client.19941 trim_cache size 0 max 16384 2024-09-11T22:36:46.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.682+0000 7f56c671f740 1 Processor -- start 2024-09-11T22:36:46.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c671f740 1 -- start start 2024-09-11T22:36:46.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c671f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dbbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c671f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c671f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dc3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c671f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1adaa0 con 0x558743dc1b40 2024-09-11T22:36:46.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c671f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1af6d0 con 0x558743dc05d0 2024-09-11T22:36:46.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c671f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1ae170 con 0x558743dcb370 2024-09-11T22:36:46.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c18e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c18e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60964/0 (socket says 172.21.15.5:60964) 2024-09-11T22:36:46.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c18e1640 1 -- 172.21.15.5:0/2916729617 learned_addr learned my addr 172.21.15.5:0/2916729617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:46.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c20e2640 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dbbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c18e1640 1 -- 172.21.15.5:0/2916729617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 msgr2=0x558743dc3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:46.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c28e3640 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dc3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c18e1640 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dc3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c18e1640 1 -- 172.21.15.5:0/2916729617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 msgr2=0x558743dbbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:46.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c18e1640 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dbbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c18e1640 1 -- 172.21.15.5:0/2916729617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558743dcb070 con 0x558743dc1b40 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c28e3640 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dc3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c20e2640 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dbbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.683+0000 7f56c18e1640 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc3120 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0cccd0 tx=0x7f56ac0999b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b2edd2696cd6e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56a7fff640 1 -- 172.21.15.5:0/2916729617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac020070 con 0x558743dc1b40 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56a7fff640 1 -- 172.21.15.5:0/2916729617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ac0a4110 con 0x558743dc1b40 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56a7fff640 1 -- 172.21.15.5:0/2916729617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0cca70 con 0x558743dc1b40 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56c671f740 1 -- 172.21.15.5:0/2916729617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 msgr2=0x558743dc3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56c671f740 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc3120 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0cccd0 tx=0x7f56ac0999b0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56c671f740 1 -- 172.21.15.5:0/2916729617 shutdown_connections 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56c671f740 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dc3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56c671f740 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dc3120 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56c671f740 1 --2- 172.21.15.5:0/2916729617 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dbbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56c671f740 1 -- 172.21.15.5:0/2916729617 >> 172.21.15.5:0/2916729617 conn(0x558743d89310 msgr2=0x558743dbb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56c671f740 1 -- 172.21.15.5:0/2916729617 shutdown_connections 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56c671f740 1 -- 172.21.15.5:0/2916729617 wait complete. 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.684+0000 7f56c671f740 1 -- start start 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c671f740 1 Processor -- start 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c671f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dcf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c671f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dcf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c671f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dcfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c671f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1af000 con 0x558743dc1b40 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c671f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1ae870 con 0x558743dc05d0 2024-09-11T22:36:46.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c671f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1abaa0 con 0x558743dcb370 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c671f740 10 client.0 _ll_register_callbacks cb 0x7f56ac00df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c18e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dcf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c18e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dcf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60974/0 (socket says 172.21.15.5:60974) 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c18e1640 1 -- 172.21.15.5:0/1565607269 learned_addr learned my addr 172.21.15.5:0/1565607269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c20e2640 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dcf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c18e1640 1 -- 172.21.15.5:0/1565607269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 msgr2=0x558743dcfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c28e3640 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dcfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c18e1640 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dcfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c18e1640 1 -- 172.21.15.5:0/1565607269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 msgr2=0x558743dcf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c18e1640 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dcf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c18e1640 1 -- 172.21.15.5:0/1565607269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558743e49000 con 0x558743dc1b40 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56c18e1640 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dcf7c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f56ac005760 tx=0x7f56ac09b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e223943293226832 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:46.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.685+0000 7f56a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1565607269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac020030 con 0x558743dc1b40 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1565607269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f56ac0d2360 con 0x558743dc1b40 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56c671f740 1 -- 172.21.15.5:0/1565607269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56ac0a4110 con 0x558743dc1b40 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1565607269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f56ac09ab00 con 0x558743dc1b40 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56c671f740 10 client.19947 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56c671f740 1 -- 172.21.15.5:0/1565607269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558743e4a250 con 0x558743dc1b40 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1565607269 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ac0a4110 con 0x558743dc1b40 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1565607269 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f56ac0a09e0 con 0x558743dc1b40 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56c671f740 10 client.19947 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56c671f740 10 client.19947 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56c671f740 20 client.19947 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56c671f740 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558743e4a660 0x558743e4ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56c671f740 4 client.19947 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56c671f740 1 -- 172.21.15.5:0/1565607269 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558743d2dd70 con 0x558743e4a660 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.686+0000 7f56c20e2640 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558743e4a660 0x558743e4ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c20e2640 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558743e4a660 0x558743e4ca80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=258a215708fdd1f4 server_cookie=2407db3b6f75384f in_seq=0 out_seq=0 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56a5ffb640 10 client.19947 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56a5ffb640 1 -- 172.21.15.5:0/1565607269 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558743d2dd70 con 0x558743e4a660 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56a5ffb640 10 client.19947 handle_command_reply: tid=0 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 client.19947 shutdown 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 -- 172.21.15.5:0/1565607269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 msgr2=0x558743dcf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dcf7c0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f56ac005760 tx=0x7f56ac09b0a0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 -- 172.21.15.5:0/1565607269 shutdown_connections 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558743e4a660 0x558743e4ca80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558743dcb370 0x558743dcfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558743dc1b40 0x558743dcf7c0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56a5ffb640 0 client.19947 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:36:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 --2- 172.21.15.5:0/1565607269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558743dc05d0 0x558743dcf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 -- 172.21.15.5:0/1565607269 >> 172.21.15.5:0/1565607269 conn(0x558743d89310 msgr2=0x558743dca030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 -- 172.21.15.5:0/1565607269 shutdown_connections 2024-09-11T22:36:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 1 -- 172.21.15.5:0/1565607269 wait complete. 2024-09-11T22:36:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.687+0000 7f56c671f740 20 client.19947 trim_cache size 0 max 16384 2024-09-11T22:36:46.862 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 -- 172.21.15.5:0/251996902 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56a80a2cf0 msgr2=0x7f56a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56a80a2cf0 0x7f56a80a51b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009a60 tx=0x7f56b40095c0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 -- 172.21.15.5:0/251996902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc101400 msgr2=0x7f56bc199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc101400 0x7f56bc199710 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f56ac00d770 tx=0x7f56ac00dc30 comp rx=0 tx=0).stop 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 -- 172.21.15.5:0/251996902 shutdown_connections 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56a80a2cf0 0x7f56a80a51b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56bc106a60 0x7f56bc1a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc101400 0x7f56bc199710 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 --2- 172.21.15.5:0/251996902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56bc100a60 0x7f56bc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 -- 172.21.15.5:0/251996902 >> 172.21.15.5:0/251996902 conn(0x7f56bc0fc780 msgr2=0x7f56bc0fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 -- 172.21.15.5:0/251996902 shutdown_connections 2024-09-11T22:36:46.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:46.688+0000 7f56c4346640 1 -- 172.21.15.5:0/251996902 wait complete. 2024-09-11T22:36:47.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:36:47.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 9.98 MB/sec warmup 65 sec latency 19441.039 ms 2024-09-11T22:36:47.893 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:48.014 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:47 smithi005 ceph-mon[30943]: pgmap v464: 97 pgs: 97 active+clean; 332 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 843 KiB/s wr, 77 op/s 2024-09-11T22:36:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:47 smithi159 ceph-mon[35534]: pgmap v464: 97 pgs: 97 active+clean; 332 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 843 KiB/s wr, 77 op/s 2024-09-11T22:36:48.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:47 smithi172 ceph-mon[35461]: pgmap v464: 97 pgs: 97 active+clean; 332 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 843 KiB/s wr, 77 op/s 2024-09-11T22:36:48.636 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 9.83 MB/sec warmup 66 sec latency 20441.125 ms 2024-09-11T22:36:48.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.526+0000 7fa4f0f22640 1 -- 172.21.15.5:0/3452296643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102dd0 msgr2=0x7fa4ec1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:48.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.526+0000 7fa4f0f22640 1 --2- 172.21.15.5:0/3452296643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102dd0 0x7fa4ec1031b0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009930 tx=0x7fa4e402a420 comp rx=0 tx=0).stop 2024-09-11T22:36:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.526+0000 7fa4f0f22640 1 -- 172.21.15.5:0/3452296643 shutdown_connections 2024-09-11T22:36:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.526+0000 7fa4f0f22640 1 --2- 172.21.15.5:0/3452296643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec108300 0x7fa4ec10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.526+0000 7fa4f0f22640 1 --2- 172.21.15.5:0/3452296643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec103780 0x7fa4ec107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.526+0000 7fa4f0f22640 1 --2- 172.21.15.5:0/3452296643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102dd0 0x7fa4ec1031b0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.526+0000 7fa4f0f22640 1 -- 172.21.15.5:0/3452296643 >> 172.21.15.5:0/3452296643 conn(0x7fa4ec0fc7d0 msgr2=0x7fa4ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.526+0000 7fa4f0f22640 1 -- 172.21.15.5:0/3452296643 shutdown_connections 2024-09-11T22:36:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.526+0000 7fa4f0f22640 1 -- 172.21.15.5:0/3452296643 wait complete. 2024-09-11T22:36:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4f0f22640 1 Processor -- start 2024-09-11T22:36:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4f0f22640 1 -- start start 2024-09-11T22:36:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4f0f22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102dd0 0x7fa4ec198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4f0f22640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec103780 0x7fa4ec198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4f0f22640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec108300 0x7fa4ec19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4f0f22640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec112110 con 0x7fa4ec102dd0 2024-09-11T22:36:48.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4f0f22640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec111f90 con 0x7fa4ec108300 2024-09-11T22:36:48.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4f0f22640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec112290 con 0x7fa4ec103780 2024-09-11T22:36:48.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102dd0 0x7fa4ec198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4ea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102dd0 0x7fa4ec198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60994/0 (socket says 172.21.15.5:60994) 2024-09-11T22:36:48.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4ea59c640 1 -- 172.21.15.5:0/1435318145 learned_addr learned my addr 172.21.15.5:0/1435318145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:48.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec108300 0x7fa4ec19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.527+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec103780 0x7fa4ec198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4ea59c640 1 -- 172.21.15.5:0/1435318145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec103780 msgr2=0x7fa4ec198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec103780 0x7fa4ec198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4ea59c640 1 -- 172.21.15.5:0/1435318145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec108300 msgr2=0x7fa4ec19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec108300 0x7fa4ec19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4ea59c640 1 -- 172.21.15.5:0/1435318145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4ec19d5f0 con 0x7fa4ec102dd0 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec103780 0x7fa4ec198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102dd0 0x7fa4ec198640 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009230 tx=0x7fa4e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4e37fe640 1 -- 172.21.15.5:0/1435318145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e403f030 con 0x7fa4ec102dd0 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4e37fe640 1 -- 172.21.15.5:0/1435318145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e4003ad0 con 0x7fa4ec102dd0 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4f0f22640 1 -- 172.21.15.5:0/1435318145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4ec19d880 con 0x7fa4ec102dd0 2024-09-11T22:36:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4e37fe640 1 -- 172.21.15.5:0/1435318145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4035070 con 0x7fa4ec102dd0 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.528+0000 7fa4f0f22640 1 -- 172.21.15.5:0/1435318145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4ec1a51c0 con 0x7fa4ec102dd0 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.530+0000 7fa4e37fe640 1 -- 172.21.15.5:0/1435318145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa4e4042030 con 0x7fa4ec102dd0 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.530+0000 7fa4f32fb740 1 Processor -- start 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4f32fb740 1 -- start start 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1a6b40 0x55a32a1a5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a32a10b2f0 con 0x55a32a1b0370 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a32a0b2cf0 con 0x55a32a1a6b40 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a32a1072b0 con 0x55a32a1a55d0 2024-09-11T22:36:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4e37fe640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d80a2cf0 0x7fa4d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32768/0 (socket says 172.21.15.5:32768) 2024-09-11T22:36:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/1964859578 learned_addr learned my addr 172.21.15.5:0/1964859578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/1964859578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1a6b40 0x55a32a1a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d80a2cf0 0x7fa4d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4e37fe640 1 -- 172.21.15.5:0/1435318145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fa4e40efa30 con 0x7fa4ec102dd0 2024-09-11T22:36:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.531+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/1964859578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/1964859578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 msgr2=0x55a32a1a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:48.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/1964859578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/1964859578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1a6b40 msgr2=0x55a32a1a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/1964859578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1a6b40 0x55a32a1a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/1964859578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32a0973a0 con 0x55a32a1b0370 2024-09-11T22:36:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/1964859578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/1964859578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1b2760 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400bcd0 tx=0x7fa4d400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bedf998d9bf8fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d80a2cf0 0x7fa4d80a51b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009900 tx=0x7fa4dc009650 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4e0ff9640 1 -- 172.21.15.5:0/1964859578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4015070 con 0x55a32a1b0370 2024-09-11T22:36:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4e0ff9640 1 -- 172.21.15.5:0/1964859578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d401a940 con 0x55a32a1b0370 2024-09-11T22:36:48.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.532+0000 7fa4e0ff9640 1 -- 172.21.15.5:0/1964859578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4011a60 con 0x55a32a1b0370 2024-09-11T22:36:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1964859578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 msgr2=0x55a32a1b2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/1964859578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1b2760 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400bcd0 tx=0x7fa4d400f730 comp rx=0 tx=0).stop 2024-09-11T22:36:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1964859578 shutdown_connections 2024-09-11T22:36:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/1964859578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1b2760 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/1964859578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/1964859578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1a6b40 0x55a32a1a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1964859578 >> 172.21.15.5:0/1964859578 conn(0x55a32a16e310 msgr2=0x55a32a1a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:48.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1964859578 shutdown_connections 2024-09-11T22:36:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1964859578 wait complete. 2024-09-11T22:36:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 -- start start 2024-09-11T22:36:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.533+0000 7fa4f32fb740 1 Processor -- start 2024-09-11T22:36:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1a55d0 0x55a32a1afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a6b40 0x55a32a1acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a32a1b3070 con 0x55a32a1b0370 2024-09-11T22:36:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a32a0c6060 con 0x55a32a1a55d0 2024-09-11T22:36:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a32a1b31f0 con 0x55a32a1a6b40 2024-09-11T22:36:48.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4f32fb740 10 client.0 _ll_register_callbacks cb 0x55a329e198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32778/0 (socket says 172.21.15.5:32778) 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4ead9d640 1 -- 172.21.15.5:0/4132214901 learned_addr learned my addr 172.21.15.5:0/4132214901 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1a55d0 0x55a32a1afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a6b40 0x55a32a1acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4ead9d640 1 -- 172.21.15.5:0/4132214901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a6b40 msgr2=0x55a32a1acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a6b40 0x55a32a1acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4ead9d640 1 -- 172.21.15.5:0/4132214901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1a55d0 msgr2=0x55a32a1afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1a55d0 0x55a32a1afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4ead9d640 1 -- 172.21.15.5:0/4132214901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a329e7c3c0 con 0x55a32a1b0370 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1ad5d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc017e50 tx=0x7fa4dc004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66ff295a581fbc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:48.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/4132214901 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc014040 con 0x55a32a1b0370 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/4132214901 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fa4dc01bd90 con 0x55a32a1b0370 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.534+0000 7fa4f32fb740 1 -- 172.21.15.5:0/4132214901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d401a940 con 0x55a32a1b0370 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/4132214901 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa4dc00be30 con 0x55a32a1b0370 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4f32fb740 10 client.19959 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4f32fb740 1 -- 172.21.15.5:0/4132214901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a32a1bead0 con 0x55a32a1b0370 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/4132214901 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d401a940 con 0x55a32a1b0370 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/4132214901 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fa4dc02d460 con 0x55a32a1b0370 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4f32fb740 10 client.19959 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4f32fb740 10 client.19959 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4f32fb740 20 client.19959 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:48.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a32a1bff90 0x55a32a1c2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4f32fb740 4 client.19959 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4f32fb740 1 -- 172.21.15.5:0/4132214901 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a32a118860 con 0x55a32a1bff90 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.535+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a32a1bff90 0x55a32a1c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.536+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a32a1bff90 0x55a32a1c2430 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c718b6a832a02146 server_cookie=4c0f5817d685138e in_seq=0 out_seq=0 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.536+0000 7fa4d1ffb640 10 client.19959 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/4132214901 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a32a118860 con 0x55a32a1bff90 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4d1ffb640 10 client.19959 handle_command_reply: tid=0 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 client.19959 shutdown 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 -- 172.21.15.5:0/4132214901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 msgr2=0x55a32a1ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1ad5d0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc017e50 tx=0x7fa4dc004b50 comp rx=0 tx=0).stop 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 -- 172.21.15.5:0/4132214901 shutdown_connections 2024-09-11T22:36:48.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1b0370 0x55a32a1ad5d0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a6b40 0x55a32a1acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a32a1bff90 0x55a32a1c2430 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/4132214901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1a55d0 0x55a32a1afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 -- 172.21.15.5:0/4132214901 >> 172.21.15.5:0/4132214901 conn(0x55a32a16e310 msgr2=0x55a32a1a7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4d1ffb640 0 client.19959 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 -- 172.21.15.5:0/4132214901 shutdown_connections 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 1 -- 172.21.15.5:0/4132214901 wait complete. 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.537+0000 7fa4f32fb740 20 client.19959 trim_cache size 0 max 16384 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4f32fb740 1 Processor -- start 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4f32fb740 1 -- start start 2024-09-11T22:36:48.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1b0370 0x55a32a1b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1adb40 con 0x55a32a1a6b40 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1af770 con 0x55a32a1b0370 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1ae210 con 0x55a32a1a55d0 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32794/0 (socket says 172.21.15.5:32794) 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.550+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/2968695070 learned_addr learned my addr 172.21.15.5:0/2968695070 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/2968695070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 msgr2=0x55a32a1bc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1bc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/2968695070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1b0370 msgr2=0x55a32a1b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1b0370 0x55a32a1b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1b0370 0x55a32a1b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/2968695070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a32a1a1ed0 con 0x55a32a1a6b40 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1bc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4ead9d640 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1b0370 0x55a32a1b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b1760 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400bcd0 tx=0x7fa4d400b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb8f762c811fc71 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4d3fff640 1 -- 172.21.15.5:0/2968695070 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4028070 con 0x55a32a1a6b40 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4d3fff640 1 -- 172.21.15.5:0/2968695070 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d400b7e0 con 0x55a32a1a6b40 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.551+0000 7fa4d3fff640 1 -- 172.21.15.5:0/2968695070 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4015040 con 0x55a32a1a6b40 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.552+0000 7fa4f32fb740 1 -- 172.21.15.5:0/2968695070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 msgr2=0x55a32a1b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.552+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b1760 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400bcd0 tx=0x7fa4d400b3a0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.552+0000 7fa4f32fb740 1 -- 172.21.15.5:0/2968695070 shutdown_connections 2024-09-11T22:36:48.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.552+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1b0370 0x55a32a1b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.552+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b1760 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.552+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/2968695070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1bc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.552+0000 7fa4f32fb740 1 -- 172.21.15.5:0/2968695070 >> 172.21.15.5:0/2968695070 conn(0x55a32a16e310 msgr2=0x55a32a1acfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.552+0000 7fa4f32fb740 1 -- 172.21.15.5:0/2968695070 shutdown_connections 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.552+0000 7fa4f32fb740 1 -- 172.21.15.5:0/2968695070 wait complete. 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.552+0000 7fa4f32fb740 1 -- start start 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4f32fb740 1 Processor -- start 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1aab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4f32fb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1b0370 0x55a32a1b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1af0a0 con 0x55a32a1a6b40 2024-09-11T22:36:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1ae910 con 0x55a32a1b0370 2024-09-11T22:36:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4f32fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec1abb40 con 0x55a32a1a55d0 2024-09-11T22:36:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4f32fb740 10 client.0 _ll_register_callbacks cb 0x7fa4dc009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32810/0 (socket says 172.21.15.5:32810) 2024-09-11T22:36:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/1653174738 learned_addr learned my addr 172.21.15.5:0/1653174738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/1653174738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 msgr2=0x55a32a1aab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/1653174738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1b0370 msgr2=0x55a32a1b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:48.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1b0370 0x55a32a1b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4e9d9b640 1 -- 172.21.15.5:0/1653174738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32a1c7820 con 0x55a32a1a6b40 2024-09-11T22:36:48.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4e9d9b640 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b8480 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400f770 tx=0x7fa4d400b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5bab37a520fc35 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:48.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.553+0000 7fa4d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:48.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/1653174738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4015070 con 0x55a32a1a6b40 2024-09-11T22:36:48.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/1653174738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fa4d4005240 con 0x55a32a1a6b40 2024-09-11T22:36:48.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1653174738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d400b7e0 con 0x55a32a1a6b40 2024-09-11T22:36:48.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/1653174738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa4d4039050 con 0x55a32a1a6b40 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4f32fb740 10 client.19965 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1653174738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a32a22f2b0 con 0x55a32a1a6b40 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/1653174738 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d400b7e0 con 0x55a32a1a6b40 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/1653174738 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fa4d401a460 con 0x55a32a1a6b40 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4f32fb740 10 client.19965 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4f32fb740 10 client.19965 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4f32fb740 20 client.19965 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a32a22f7e0 0x55a32a231c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4f32fb740 4 client.19965 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.554+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1653174738 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a32a112d70 con 0x55a32a22f7e0 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a32a22f7e0 0x55a32a231c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:48.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4d1ffb640 10 client.19965 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:36:48.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4ea59c640 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a32a22f7e0 0x55a32a231c40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eaca4fd0655686be server_cookie=bc41ccbd96cd3738 in_seq=0 out_seq=0 2024-09-11T22:36:48.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4d1ffb640 1 -- 172.21.15.5:0/1653174738 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a32a112d70 con 0x55a32a22f7e0 2024-09-11T22:36:48.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4d1ffb640 10 client.19965 handle_command_reply: tid=0 2024-09-11T22:36:48.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 client.19965 shutdown 2024-09-11T22:36:48.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1653174738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 msgr2=0x55a32a1b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:48.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b8480 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400f770 tx=0x7fa4d400b680 comp rx=0 tx=0).stop 2024-09-11T22:36:48.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1653174738 shutdown_connections 2024-09-11T22:36:48.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55a32a22f7e0 0x55a32a231c40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a32a1b0370 0x55a32a1b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4d1ffb640 0 client.19965 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:36:48.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a32a1a6b40 0x55a32a1b8480 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 --2- 172.21.15.5:0/1653174738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a32a1a55d0 0x55a32a1aab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1653174738 >> 172.21.15.5:0/1653174738 conn(0x55a32a16e310 msgr2=0x55a32a1ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:48.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1653174738 shutdown_connections 2024-09-11T22:36:48.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.555+0000 7fa4f32fb740 1 -- 172.21.15.5:0/1653174738 wait complete. 2024-09-11T22:36:48.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f32fb740 20 client.19965 trim_cache size 0 max 16384 2024-09-11T22:36:48.717 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:36:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 -- 172.21.15.5:0/1435318145 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d80a2cf0 msgr2=0x7fa4d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d80a2cf0 0x7fa4d80a51b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009900 tx=0x7fa4dc009650 comp rx=0 tx=0).stop 2024-09-11T22:36:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 -- 172.21.15.5:0/1435318145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102dd0 msgr2=0x7fa4ec198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102dd0 0x7fa4ec198640 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009230 tx=0x7fa4e4004270 comp rx=0 tx=0).stop 2024-09-11T22:36:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 -- 172.21.15.5:0/1435318145 shutdown_connections 2024-09-11T22:36:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4d80a2cf0 0x7fa4d80a51b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4ec108300 0x7fa4ec19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4ec103780 0x7fa4ec198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 --2- 172.21.15.5:0/1435318145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4ec102dd0 0x7fa4ec198640 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:48.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 -- 172.21.15.5:0/1435318145 >> 172.21.15.5:0/1435318145 conn(0x7fa4ec0fc7d0 msgr2=0x7fa4ec105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 -- 172.21.15.5:0/1435318145 shutdown_connections 2024-09-11T22:36:48.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:48.556+0000 7fa4f0f22640 1 -- 172.21.15.5:0/1435318145 wait complete. 2024-09-11T22:36:49.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:36:49.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 9.68 MB/sec warmup 67 sec latency 21441.176 ms 2024-09-11T22:36:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:50.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:49 smithi005 ceph-mon[30943]: pgmap v465: 97 pgs: 97 active+clean; 333 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 881 KiB/s wr, 84 op/s 2024-09-11T22:36:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:49 smithi159 ceph-mon[35534]: pgmap v465: 97 pgs: 97 active+clean; 333 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 881 KiB/s wr, 84 op/s 2024-09-11T22:36:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:49 smithi172 ceph-mon[35461]: pgmap v465: 97 pgs: 97 active+clean; 333 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 881 KiB/s wr, 84 op/s 2024-09-11T22:36:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.296+0000 7f64d9793640 1 -- 172.21.15.5:0/279115922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64d41036e0 msgr2=0x7f64d4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.296+0000 7f64d9793640 1 --2- 172.21.15.5:0/279115922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64d41036e0 0x7f64d4107b90 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009930 tx=0x7f64c802a420 comp rx=0 tx=0).stop 2024-09-11T22:36:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.297+0000 7f64d9793640 1 -- 172.21.15.5:0/279115922 shutdown_connections 2024-09-11T22:36:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.297+0000 7f64d9793640 1 --2- 172.21.15.5:0/279115922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64d41082a0 0x7f64d410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.297+0000 7f64d9793640 1 --2- 172.21.15.5:0/279115922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64d41036e0 0x7f64d4107b90 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.297+0000 7f64d9793640 1 --2- 172.21.15.5:0/279115922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d4102d30 0x7f64d4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.297+0000 7f64d9793640 1 -- 172.21.15.5:0/279115922 >> 172.21.15.5:0/279115922 conn(0x7f64d40fc7d0 msgr2=0x7f64d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.297+0000 7f64d9793640 1 -- 172.21.15.5:0/279115922 shutdown_connections 2024-09-11T22:36:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.297+0000 7f64d9793640 1 -- 172.21.15.5:0/279115922 wait complete. 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.297+0000 7f64d9793640 1 Processor -- start 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.297+0000 7f64d9793640 1 -- start start 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d9793640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64d4102d30 0x7f64d4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d9793640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64d41036e0 0x7f64d4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d9793640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d41082a0 0x7f64d419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d9793640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4112060 con 0x7f64d41036e0 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d9793640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64d4111ee0 con 0x7f64d4102d30 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d9793640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41121e0 con 0x7f64d41082a0 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64d41036e0 0x7f64d4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64d41036e0 0x7f64d4198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32822/0 (socket says 172.21.15.5:32822) 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d27fc640 1 -- 172.21.15.5:0/1420266483 learned_addr learned my addr 172.21.15.5:0/1420266483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d37fe640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d41082a0 0x7f64d419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d2ffd640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64d4102d30 0x7f64d4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d27fc640 1 -- 172.21.15.5:0/1420266483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d41082a0 msgr2=0x7f64d419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d27fc640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d41082a0 0x7f64d419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d27fc640 1 -- 172.21.15.5:0/1420266483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64d4102d30 msgr2=0x7f64d4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d27fc640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64d4102d30 0x7f64d4198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d27fc640 1 -- 172.21.15.5:0/1420266483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d419d520 con 0x7f64d41036e0 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64d27fc640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64d41036e0 0x7f64d4198ab0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f64c8005940 tx=0x7f64c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64bbfff640 1 -- 172.21.15.5:0/1420266483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c803f030 con 0x7f64d41036e0 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64bbfff640 1 -- 172.21.15.5:0/1420266483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64c8003ce0 con 0x7f64d41036e0 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.298+0000 7f64bbfff640 1 -- 172.21.15.5:0/1420266483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8035070 con 0x7f64d41036e0 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.299+0000 7f64d9793640 1 -- 172.21.15.5:0/1420266483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d419d7b0 con 0x7f64d41036e0 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.299+0000 7f64d9793640 1 -- 172.21.15.5:0/1420266483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d41a50f0 con 0x7f64d41036e0 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.301+0000 7f64bbfff640 1 -- 172.21.15.5:0/1420266483 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f64c8042030 con 0x7f64d41036e0 2024-09-11T22:36:50.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.301+0000 7f64dbb6c740 1 Processor -- start 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.301+0000 7f64dbb6c740 1 -- start start 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.301+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086229f50 0x5610862284a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.301+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610862289e0 0x561086228e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.301+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086233780 0x561086235b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56108618a2f0 con 0x561086233780 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561086131cf0 con 0x561086229f50 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5610861862b0 con 0x5610862289e0 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64bbfff640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64bc0a2cf0 0x7f64bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086233780 0x561086235b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086233780 0x561086235b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32824/0 (socket says 172.21.15.5:32824) 2024-09-11T22:36:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d27fc640 1 -- 172.21.15.5:0/3542275704 learned_addr learned my addr 172.21.15.5:0/3542275704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d37fe640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64bc0a2cf0 0x7f64bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d2ffd640 1 --2- 172.21.15.5:0/3542275704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610862289e0 0x561086228e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d37fe640 1 --2- 172.21.15.5:0/3542275704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086229f50 0x5610862284a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64bbfff640 1 -- 172.21.15.5:0/1420266483 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f64c80efb90 con 0x7f64d41036e0 2024-09-11T22:36:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d27fc640 1 -- 172.21.15.5:0/3542275704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610862289e0 msgr2=0x561086228e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d27fc640 1 --2- 172.21.15.5:0/3542275704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610862289e0 0x561086228e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d27fc640 1 -- 172.21.15.5:0/3542275704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086229f50 msgr2=0x5610862284a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d27fc640 1 --2- 172.21.15.5:0/3542275704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086229f50 0x5610862284a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d27fc640 1 -- 172.21.15.5:0/3542275704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610861163a0 con 0x561086233780 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d37fe640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64bc0a2cf0 0x7f64bc0a51b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f64cc005d50 tx=0x7f64cc00e640 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64d27fc640 1 --2- 172.21.15.5:0/3542275704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086233780 0x561086235b70 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f64c80b83f0 tx=0x7f64c8047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3881a7ab2e47f756 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.302+0000 7f64b97fa640 1 -- 172.21.15.5:0/3542275704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8049070 con 0x561086233780 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64b97fa640 1 -- 172.21.15.5:0/3542275704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64c80bbfe0 con 0x561086233780 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64b97fa640 1 -- 172.21.15.5:0/3542275704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c80b8650 con 0x561086233780 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64dbb6c740 1 -- 172.21.15.5:0/3542275704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086233780 msgr2=0x561086235b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/3542275704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086233780 0x561086235b70 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f64c80b83f0 tx=0x7f64c8047000 comp rx=0 tx=0).stop 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64dbb6c740 1 -- 172.21.15.5:0/3542275704 shutdown_connections 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/3542275704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086233780 0x561086235b70 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/3542275704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610862289e0 0x561086228e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/3542275704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086229f50 0x5610862284a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64dbb6c740 1 -- 172.21.15.5:0/3542275704 >> 172.21.15.5:0/3542275704 conn(0x5610861ed310 msgr2=0x561086225070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64dbb6c740 1 -- 172.21.15.5:0/3542275704 shutdown_connections 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.303+0000 7f64dbb6c740 1 -- 172.21.15.5:0/3542275704 wait complete. 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64dbb6c740 1 -- start start 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64dbb6c740 1 Processor -- start 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610862289e0 0x561086233170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x561086230420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561086233780 0x561086230a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561086236240 con 0x561086229f50 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561086145060 con 0x5610862289e0 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5610862363c0 con 0x561086233780 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64dbb6c740 10 client.0 _ll_register_callbacks cb 0x561085e988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:50.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x561086230420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x561086230420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32834/0 (socket says 172.21.15.5:32834) 2024-09-11T22:36:50.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64d27fc640 1 -- 172.21.15.5:0/240276054 learned_addr learned my addr 172.21.15.5:0/240276054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:50.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64d2ffd640 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610862289e0 0x561086233170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64d37fe640 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561086233780 0x561086230a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64d27fc640 1 -- 172.21.15.5:0/240276054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561086233780 msgr2=0x561086230a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64d27fc640 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561086233780 0x561086230a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64d27fc640 1 -- 172.21.15.5:0/240276054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610862289e0 msgr2=0x561086233170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.304+0000 7f64d27fc640 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610862289e0 0x561086233170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64d27fc640 1 -- 172.21.15.5:0/240276054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561085efb3c0 con 0x561086229f50 2024-09-11T22:36:50.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64d2ffd640 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610862289e0 0x561086233170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64d27fc640 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x561086230420 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f64c803d040 tx=0x7f64c80044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67e4713bee344b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64b6ff5640 1 -- 172.21.15.5:0/240276054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8049070 con 0x561086229f50 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64b6ff5640 1 -- 172.21.15.5:0/240276054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f64c80b8650 con 0x561086229f50 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64dbb6c740 1 -- 172.21.15.5:0/240276054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c80bbfe0 con 0x561086229f50 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64b6ff5640 1 -- 172.21.15.5:0/240276054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f64c80c0390 con 0x561086229f50 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64dbb6c740 10 client.19977 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64dbb6c740 1 -- 172.21.15.5:0/240276054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x561086236240 con 0x561086229f50 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64b6ff5640 1 -- 172.21.15.5:0/240276054 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64c80bbfe0 con 0x561086229f50 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64b6ff5640 1 -- 172.21.15.5:0/240276054 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f64c803bc50 con 0x561086229f50 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64dbb6c740 10 client.19977 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64dbb6c740 10 client.19977 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64dbb6c740 20 client.19977 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x561086242f50 0x561086245360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64dbb6c740 4 client.19977 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.305+0000 7f64dbb6c740 1 -- 172.21.15.5:0/240276054 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561086197860 con 0x561086242f50 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.306+0000 7f64d2ffd640 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x561086242f50 0x561086245360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.306+0000 7f64b6ff5640 10 client.19977 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.306+0000 7f64d2ffd640 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x561086242f50 0x561086245360 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=114412dad36691 server_cookie=94196e1952b34354 in_seq=0 out_seq=0 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64b6ff5640 1 -- 172.21.15.5:0/240276054 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561086197860 con 0x561086242f50 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64b6ff5640 10 client.19977 handle_command_reply: tid=0 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 client.19977 shutdown 2024-09-11T22:36:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 -- 172.21.15.5:0/240276054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 msgr2=0x561086230420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x561086230420 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f64c803d040 tx=0x7f64c80044e0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 -- 172.21.15.5:0/240276054 shutdown_connections 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x561086242f50 0x561086245360 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561086233780 0x561086230a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x561086230420 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64b6ff5640 0 client.19977 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/240276054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5610862289e0 0x561086233170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 -- 172.21.15.5:0/240276054 >> 172.21.15.5:0/240276054 conn(0x5610861ed310 msgr2=0x56108622ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 -- 172.21.15.5:0/240276054 shutdown_connections 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.307+0000 7f64dbb6c740 1 -- 172.21.15.5:0/240276054 wait complete. 2024-09-11T22:36:50.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.308+0000 7f64dbb6c740 20 client.19977 trim_cache size 0 max 16384 2024-09-11T22:36:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.320+0000 7f64dbb6c740 1 Processor -- start 2024-09-11T22:36:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64dbb6c740 1 -- start start 2024-09-11T22:36:50.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610862289e0 0x56108622b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x56108622ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 0x56108624ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ab830 con 0x561086229f50 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ad460 con 0x561086233780 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41abf00 con 0x5610862289e0 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x56108622ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x56108622ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32840/0 (socket says 172.21.15.5:32840) 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d27fc640 1 -- 172.21.15.5:0/3002124820 learned_addr learned my addr 172.21.15.5:0/3002124820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d2ffd640 1 --2- 172.21.15.5:0/3002124820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610862289e0 0x56108622b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d27fc640 1 -- 172.21.15.5:0/3002124820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610862289e0 msgr2=0x56108622b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d27fc640 1 --2- 172.21.15.5:0/3002124820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610862289e0 0x56108622b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d27fc640 1 -- 172.21.15.5:0/3002124820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 msgr2=0x56108624ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d37fe640 1 --2- 172.21.15.5:0/3002124820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 0x56108624ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d27fc640 1 --2- 172.21.15.5:0/3002124820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 0x56108624ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d27fc640 1 -- 172.21.15.5:0/3002124820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561086232d90 con 0x561086229f50 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.321+0000 7f64d37fe640 1 --2- 172.21.15.5:0/3002124820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 0x56108624ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64d27fc640 1 --2- 172.21.15.5:0/3002124820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x56108622ba70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f64c80bbf70 tx=0x7f64c802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da0b37b8c4d185c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64b8ff9640 1 -- 172.21.15.5:0/3002124820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8049070 con 0x561086229f50 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64b8ff9640 1 -- 172.21.15.5:0/3002124820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64c80bbfe0 con 0x561086229f50 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64b8ff9640 1 -- 172.21.15.5:0/3002124820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c803b460 con 0x561086229f50 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64dbb6c740 1 -- 172.21.15.5:0/3002124820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 msgr2=0x56108622ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/3002124820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x56108622ba70 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f64c80bbf70 tx=0x7f64c802a420 comp rx=0 tx=0).stop 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64dbb6c740 1 -- 172.21.15.5:0/3002124820 shutdown_connections 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/3002124820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 0x56108624ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/3002124820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561086229f50 0x56108622ba70 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/3002124820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5610862289e0 0x56108622b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.322+0000 7f64dbb6c740 1 -- 172.21.15.5:0/3002124820 >> 172.21.15.5:0/3002124820 conn(0x5610861ed310 msgr2=0x5610862324a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 1 -- 172.21.15.5:0/3002124820 shutdown_connections 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 1 -- 172.21.15.5:0/3002124820 wait complete. 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 1 -- start start 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 1 Processor -- start 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610862289e0 0x561086237a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561086229f50 0x561086237f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 0x5610862acc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41acd90 con 0x5610862289e0 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41ac600 con 0x561086233780 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64d41a9830 con 0x561086229f50 2024-09-11T22:36:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64dbb6c740 10 client.0 _ll_register_callbacks cb 0x7f64c40075a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610862289e0 0x561086237a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610862289e0 0x561086237a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32848/0 (socket says 172.21.15.5:32848) 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.323+0000 7f64d2ffd640 1 -- 172.21.15.5:0/556551456 learned_addr learned my addr 172.21.15.5:0/556551456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64d27fc640 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561086229f50 0x561086237f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64d37fe640 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 0x5610862acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64d2ffd640 1 -- 172.21.15.5:0/556551456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561086229f50 msgr2=0x561086237f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64d2ffd640 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561086229f50 0x561086237f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64d2ffd640 1 -- 172.21.15.5:0/556551456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 msgr2=0x5610862acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64d2ffd640 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 0x5610862acc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64d2ffd640 1 -- 172.21.15.5:0/556551456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610862ad140 con 0x5610862289e0 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64d2ffd640 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610862289e0 0x561086237a00 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f64c4000f80 tx=0x7f64c4005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f390ee7491ee5726 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64b6ff5640 1 -- 172.21.15.5:0/556551456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c40128c0 con 0x5610862289e0 2024-09-11T22:36:50.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64b6ff5640 1 -- 172.21.15.5:0/556551456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f64c4012a60 con 0x5610862289e0 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64dbb6c740 1 -- 172.21.15.5:0/556551456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64c80bbfe0 con 0x5610862289e0 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64b6ff5640 1 -- 172.21.15.5:0/556551456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f64c4012c60 con 0x5610862289e0 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64dbb6c740 10 client.19983 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.324+0000 7f64dbb6c740 1 -- 172.21.15.5:0/556551456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5610862ae310 con 0x5610862289e0 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.325+0000 7f64b6ff5640 1 -- 172.21.15.5:0/556551456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64c80bbfe0 con 0x5610862289e0 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.325+0000 7f64b6ff5640 1 -- 172.21.15.5:0/556551456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f64c402b420 con 0x5610862289e0 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.325+0000 7f64dbb6c740 10 client.19983 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.325+0000 7f64dbb6c740 10 client.19983 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.325+0000 7f64dbb6c740 20 client.19983 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.325+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x561086242f50 0x5610862b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.325+0000 7f64dbb6c740 4 client.19983 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.325+0000 7f64dbb6c740 1 -- 172.21.15.5:0/556551456 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561086191d70 con 0x561086242f50 2024-09-11T22:36:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.325+0000 7f64d27fc640 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x561086242f50 0x5610862b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64d27fc640 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x561086242f50 0x5610862b0ad0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acd11fc477b77ac2 server_cookie=a62c697f42d5335a in_seq=0 out_seq=0 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64b6ff5640 10 client.19983 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64b6ff5640 1 -- 172.21.15.5:0/556551456 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561086191d70 con 0x561086242f50 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64b6ff5640 10 client.19983 handle_command_reply: tid=0 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 client.19983 shutdown 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 -- 172.21.15.5:0/556551456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610862289e0 msgr2=0x561086237a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610862289e0 0x561086237a00 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f64c4000f80 tx=0x7f64c4005410 comp rx=0 tx=0).stop 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 -- 172.21.15.5:0/556551456 shutdown_connections 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x561086242f50 0x5610862b0ad0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561086233780 0x5610862acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561086229f50 0x561086237f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 --2- 172.21.15.5:0/556551456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5610862289e0 0x561086237a00 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64b6ff5640 0 client.19983 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:36:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 -- 172.21.15.5:0/556551456 >> 172.21.15.5:0/556551456 conn(0x5610861ed310 msgr2=0x56108621f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 -- 172.21.15.5:0/556551456 shutdown_connections 2024-09-11T22:36:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 1 -- 172.21.15.5:0/556551456 wait complete. 2024-09-11T22:36:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.326+0000 7f64dbb6c740 20 client.19983 trim_cache size 0 max 16384 2024-09-11T22:36:50.341 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:36:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 -- 172.21.15.5:0/1420266483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64bc0a2cf0 msgr2=0x7f64bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64bc0a2cf0 0x7f64bc0a51b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f64cc005d50 tx=0x7f64cc00e640 comp rx=0 tx=0).stop 2024-09-11T22:36:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 -- 172.21.15.5:0/1420266483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64d41036e0 msgr2=0x7f64d4198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64d41036e0 0x7f64d4198ab0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f64c8005940 tx=0x7f64c80042f0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 -- 172.21.15.5:0/1420266483 shutdown_connections 2024-09-11T22:36:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64d41082a0 0x7f64d419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64d41036e0 0x7f64d4198ab0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64bc0a2cf0 0x7f64bc0a51b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 --2- 172.21.15.5:0/1420266483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64d4102d30 0x7f64d4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 -- 172.21.15.5:0/1420266483 >> 172.21.15.5:0/1420266483 conn(0x7f64d40fc7d0 msgr2=0x7f64d4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 -- 172.21.15.5:0/1420266483 shutdown_connections 2024-09-11T22:36:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:50.327+0000 7f64d9793640 1 -- 172.21.15.5:0/1420266483 wait complete. 2024-09-11T22:36:50.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 9.54 MB/sec warmup 68 sec latency 22441.272 ms 2024-09-11T22:36:50.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.783+0000 7f5b9ac42640 1 -- 172.21.15.5:0/3931551486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b94101390 msgr2=0x7f5b9410d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.783+0000 7f5b9ac42640 1 --2- 172.21.15.5:0/3931551486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b94101390 0x7f5b9410d8c0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009cd0 tx=0x7f5b8c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.783+0000 7f5b9ac42640 1 -- 172.21.15.5:0/3931551486 shutdown_connections 2024-09-11T22:36:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.783+0000 7f5b9ac42640 1 --2- 172.21.15.5:0/3931551486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b94101390 0x7f5b9410d8c0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.783+0000 7f5b9ac42640 1 --2- 172.21.15.5:0/3931551486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b941009f0 0x7f5b94100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.783+0000 7f5b9ac42640 1 --2- 172.21.15.5:0/3931551486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b94106a00 0x7f5b94106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.783+0000 7f5b9ac42640 1 -- 172.21.15.5:0/3931551486 >> 172.21.15.5:0/3931551486 conn(0x7f5b940fc6b0 msgr2=0x7f5b940fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.783+0000 7f5b9ac42640 1 -- 172.21.15.5:0/3931551486 shutdown_connections 2024-09-11T22:36:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.783+0000 7f5b9ac42640 1 -- 172.21.15.5:0/3931551486 wait complete. 2024-09-11T22:36:50.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.784+0000 7f5b9ac42640 1 Processor -- start 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.784+0000 7f5b9ac42640 1 -- start start 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.784+0000 7f5b9ac42640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b941009f0 0x7f5b94198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.784+0000 7f5b9ac42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b94101390 0x7f5b94198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.784+0000 7f5b9ac42640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b94106a00 0x7f5b9419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.784+0000 7f5b9ac42640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94111f80 con 0x7f5b94101390 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.784+0000 7f5b9ac42640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94111e00 con 0x7f5b941009f0 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.784+0000 7f5b9ac42640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b94112100 con 0x7f5b94106a00 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.784+0000 7f5b8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b94101390 0x7f5b94198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b94101390 0x7f5b94198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32866/0 (socket says 172.21.15.5:32866) 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b8bfff640 1 -- 172.21.15.5:0/34691335 learned_addr learned my addr 172.21.15.5:0/34691335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b989de640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b941009f0 0x7f5b94198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b991df640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b94106a00 0x7f5b9419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b8bfff640 1 -- 172.21.15.5:0/34691335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b94106a00 msgr2=0x7f5b9419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b8bfff640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b94106a00 0x7f5b9419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b8bfff640 1 -- 172.21.15.5:0/34691335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b941009f0 msgr2=0x7f5b94198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b8bfff640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b941009f0 0x7f5b94198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b8bfff640 1 -- 172.21.15.5:0/34691335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9419d320 con 0x7f5b94101390 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b8bfff640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b94101390 0x7f5b94198870 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400d950 tx=0x7f5b8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b89ffb640 1 -- 172.21.15.5:0/34691335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84017070 con 0x7f5b94101390 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b89ffb640 1 -- 172.21.15.5:0/34691335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b84009c30 con 0x7f5b94101390 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b9ac42640 1 -- 172.21.15.5:0/34691335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9419d610 con 0x7f5b94101390 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b89ffb640 1 -- 172.21.15.5:0/34691335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8400f6b0 con 0x7f5b94101390 2024-09-11T22:36:50.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.785+0000 7f5b9ac42640 1 -- 172.21.15.5:0/34691335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b941a4ea0 con 0x7f5b94101390 2024-09-11T22:36:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.788+0000 7f5b89ffb640 1 -- 172.21.15.5:0/34691335 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f5b840109d0 con 0x7f5b94101390 2024-09-11T22:36:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.788+0000 7f5b9d9d1740 1 Processor -- start 2024-09-11T22:36:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b9d9d1740 1 -- start start 2024-09-11T22:36:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf86ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf87410 0x5614ccf91770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf91cb0 0x5614ccf940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5614cceeb2f0 con 0x5614ccf86af0 2024-09-11T22:36:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5614cce93270 con 0x5614ccf91cb0 2024-09-11T22:36:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5614ccee72b0 con 0x5614ccf87410 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b991df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf86ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b89ffb640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5b7c0a2e20 0x7f5b7c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b991df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf86ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32878/0 (socket says 172.21.15.5:32878) 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b991df640 1 -- 172.21.15.5:0/848013165 learned_addr learned my addr 172.21.15.5:0/848013165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b991df640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5b7c0a2e20 0x7f5b7c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b89ffb640 1 -- 172.21.15.5:0/34691335 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f5b840cf0b0 con 0x7f5b94101390 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b8bfff640 1 --2- 172.21.15.5:0/848013165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf91cb0 0x5614ccf940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b989de640 1 --2- 172.21.15.5:0/848013165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf87410 0x5614ccf91770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b991df640 1 -- 172.21.15.5:0/848013165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf87410 msgr2=0x5614ccf91770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b991df640 1 --2- 172.21.15.5:0/848013165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf87410 0x5614ccf91770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b991df640 1 -- 172.21.15.5:0/848013165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf91cb0 msgr2=0x5614ccf940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b991df640 1 --2- 172.21.15.5:0/848013165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf91cb0 0x5614ccf940a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b991df640 1 -- 172.21.15.5:0/848013165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614cce92340 con 0x5614ccf86af0 2024-09-11T22:36:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.789+0000 7f5b989de640 1 --2- 172.21.15.5:0/848013165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf87410 0x5614ccf91770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.790+0000 7f5b991df640 1 --2- 172.21.15.5:0/848013165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf86ed0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c005fe0 tx=0x7f5b8c031c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fd52490855cc51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.790+0000 7f5b7affd640 1 -- 172.21.15.5:0/848013165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c004b40 con 0x5614ccf86af0 2024-09-11T22:36:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.790+0000 7f5b7affd640 1 -- 172.21.15.5:0/848013165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b8c036040 con 0x5614ccf86af0 2024-09-11T22:36:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.790+0000 7f5b7affd640 1 -- 172.21.15.5:0/848013165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c030800 con 0x5614ccf86af0 2024-09-11T22:36:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.790+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/848013165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 msgr2=0x5614ccf86ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.790+0000 7f5b991df640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5b7c0a2e20 0x7f5b7c0a52e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c04d000 tx=0x7f5b8c04b000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.790+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/848013165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf86ed0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c005fe0 tx=0x7f5b8c031c00 comp rx=0 tx=0).stop 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.790+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/848013165 shutdown_connections 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.791+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/848013165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf91cb0 0x5614ccf940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.791+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/848013165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf87410 0x5614ccf91770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.791+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/848013165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf86ed0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.791+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/848013165 >> 172.21.15.5:0/848013165 conn(0x5614ccf83060 msgr2=0x5614ccf83440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.791+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/848013165 shutdown_connections 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.791+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/848013165 wait complete. 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.791+0000 7f5b9d9d1740 1 -- start start 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.791+0000 7f5b9d9d1740 1 Processor -- start 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.791+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.791+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf87410 0x5614ccf8f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 0x5614ccf8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5614ccf94880 con 0x5614ccf86af0 2024-09-11T22:36:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5614cce74290 con 0x5614ccf87410 2024-09-11T22:36:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5614ccf94a00 con 0x5614ccf91cb0 2024-09-11T22:36:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b9d9d1740 10 client.0 _ll_register_callbacks cb 0x5614ccea5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b989de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b989de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf8eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32896/0 (socket says 172.21.15.5:32896) 2024-09-11T22:36:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b989de640 1 -- 172.21.15.5:0/497835984 learned_addr learned my addr 172.21.15.5:0/497835984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b8bfff640 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf87410 0x5614ccf8f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b991df640 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 0x5614ccf8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b989de640 1 -- 172.21.15.5:0/497835984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 msgr2=0x5614ccf8f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b989de640 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 0x5614ccf8f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b989de640 1 -- 172.21.15.5:0/497835984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf87410 msgr2=0x5614ccf8f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b989de640 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf87410 0x5614ccf8f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b989de640 1 -- 172.21.15.5:0/497835984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ccbfe210 con 0x5614ccf86af0 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b989de640 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf8eb00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f5b900116c0 tx=0x7f5b90011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a630533d4492fe4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b73fff640 1 -- 172.21.15.5:0/497835984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9001f060 con 0x5614ccf86af0 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b73fff640 1 -- 172.21.15.5:0/497835984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f5b9000fae0 con 0x5614ccf86af0 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/497835984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b8c036040 con 0x5614ccf86af0 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b73fff640 1 -- 172.21.15.5:0/497835984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f5b9000fce0 con 0x5614ccf86af0 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b9d9d1740 10 client.19995 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.792+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/497835984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5614cccca620 con 0x5614ccf86af0 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.793+0000 7f5b73fff640 1 -- 172.21.15.5:0/497835984 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b8c036040 con 0x5614ccf86af0 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.793+0000 7f5b73fff640 1 -- 172.21.15.5:0/497835984 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f5b90022540 con 0x5614ccf86af0 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.793+0000 7f5b9d9d1740 10 client.19995 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.793+0000 7f5b9d9d1740 10 client.19995 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.793+0000 7f5b9d9d1740 20 client.19995 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.793+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5614ccfa1c50 0x5614ccfa40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.793+0000 7f5b9d9d1740 4 client.19995 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.793+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/497835984 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614ccef8a20 con 0x5614ccfa1c50 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.793+0000 7f5b8bfff640 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5614ccfa1c50 0x5614ccfa40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.794+0000 7f5b8bfff640 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5614ccfa1c50 0x5614ccfa40a0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff5d45528b9a140 server_cookie=e5b79e93dd3d0ee7 in_seq=0 out_seq=0 2024-09-11T22:36:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.794+0000 7f5b73fff640 10 client.19995 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.794+0000 7f5b73fff640 1 -- 172.21.15.5:0/497835984 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614ccef8a20 con 0x5614ccfa1c50 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.794+0000 7f5b73fff640 10 client.19995 handle_command_reply: tid=0 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.794+0000 7f5b9d9d1740 1 client.19995 shutdown 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/497835984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 msgr2=0x5614ccf8eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf8eb00 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f5b900116c0 tx=0x7f5b90011b80 comp rx=0 tx=0).stop 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/497835984 shutdown_connections 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5614ccfa1c50 0x5614ccfa40a0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 0x5614ccf8f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf87410 0x5614ccf8f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b73fff640 0 client.19995 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/497835984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf86af0 0x5614ccf8eb00 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/497835984 >> 172.21.15.5:0/497835984 conn(0x5614ccf83060 msgr2=0x5614ccf821c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/497835984 shutdown_connections 2024-09-11T22:36:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/497835984 wait complete. 2024-09-11T22:36:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.795+0000 7f5b9d9d1740 20 client.19995 trim_cache size 0 max 16384 2024-09-11T22:36:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.809+0000 7f5b9d9d1740 1 Processor -- start 2024-09-11T22:36:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.809+0000 7f5b9d9d1740 1 -- start start 2024-09-11T22:36:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.809+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf86af0 0x5614ccf90a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.809+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf87410 0x5614ccfa99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.809+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 0x5614ccfa9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.809+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941ad9b0 con 0x5614ccf87410 2024-09-11T22:36:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.809+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941af5e0 con 0x5614ccf86af0 2024-09-11T22:36:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.809+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941ae080 con 0x5614ccf91cb0 2024-09-11T22:36:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf87410 0x5614ccfa99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf87410 0x5614ccfa99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32910/0 (socket says 172.21.15.5:32910) 2024-09-11T22:36:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b8bfff640 1 -- 172.21.15.5:0/182844804 learned_addr learned my addr 172.21.15.5:0/182844804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b8bfff640 1 -- 172.21.15.5:0/182844804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 msgr2=0x5614ccfa9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b991df640 1 --2- 172.21.15.5:0/182844804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 0x5614ccfa9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b989de640 1 --2- 172.21.15.5:0/182844804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf86af0 0x5614ccf90a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b8bfff640 1 --2- 172.21.15.5:0/182844804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 0x5614ccfa9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b8bfff640 1 -- 172.21.15.5:0/182844804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf86af0 msgr2=0x5614ccf90a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b8bfff640 1 --2- 172.21.15.5:0/182844804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf86af0 0x5614ccf90a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b8bfff640 1 -- 172.21.15.5:0/182844804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614ccf82e90 con 0x5614ccf87410 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b991df640 1 --2- 172.21.15.5:0/182844804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 0x5614ccfa9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b8bfff640 1 --2- 172.21.15.5:0/182844804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf87410 0x5614ccfa99b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5b8409b490 tx=0x7f5b840977e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b70d7828ce76734 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b7a7fc640 1 -- 172.21.15.5:0/182844804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84020040 con 0x5614ccf87410 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b7a7fc640 1 -- 172.21.15.5:0/182844804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b840a4170 con 0x5614ccf87410 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.810+0000 7f5b7a7fc640 1 -- 172.21.15.5:0/182844804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84097d00 con 0x5614ccf87410 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/182844804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf87410 msgr2=0x5614ccfa99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/182844804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf87410 0x5614ccfa99b0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f5b8409b490 tx=0x7f5b840977e0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/182844804 shutdown_connections 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/182844804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf91cb0 0x5614ccfa9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/182844804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf87410 0x5614ccfa99b0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/182844804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf86af0 0x5614ccf90a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/182844804 >> 172.21.15.5:0/182844804 conn(0x5614ccf83060 msgr2=0x5614ccf90360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/182844804 shutdown_connections 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/182844804 wait complete. 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 -- start start 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.811+0000 7f5b9d9d1740 1 Processor -- start 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf86af0 0x5614ccf94e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf87410 0x5614ccf95350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b9d9d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf91cb0 0x5614cd00a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941aef10 con 0x5614ccf91cb0 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941ae780 con 0x5614ccf87410 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b9d9d1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5b941ab9b0 con 0x5614ccf86af0 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b9d9d1740 10 client.0 _ll_register_callbacks cb 0x7f5b9000a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b991df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf91cb0 0x5614cd00a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b991df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf91cb0 0x5614cd00a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32920/0 (socket says 172.21.15.5:32920) 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b991df640 1 -- 172.21.15.5:0/2609704846 learned_addr learned my addr 172.21.15.5:0/2609704846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b989de640 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf86af0 0x5614ccf94e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b991df640 1 -- 172.21.15.5:0/2609704846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf86af0 msgr2=0x5614ccf94e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b991df640 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf86af0 0x5614ccf94e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b991df640 1 -- 172.21.15.5:0/2609704846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf87410 msgr2=0x5614ccf95350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b991df640 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf87410 0x5614ccf95350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b991df640 1 -- 172.21.15.5:0/2609704846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614ccf84110 con 0x5614ccf91cb0 2024-09-11T22:36:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b989de640 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf86af0 0x5614ccf94e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b991df640 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf91cb0 0x5614cd00a970 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009540 tx=0x7f5b8c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ef375bf67b7250 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b73fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b73fff640 1 -- 172.21.15.5:0/2609704846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c061060 con 0x5614ccf91cb0 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b73fff640 1 -- 172.21.15.5:0/2609704846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f5b8c02e040 con 0x5614ccf91cb0 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.812+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/2609704846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b840a4170 con 0x5614ccf91cb0 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b73fff640 1 -- 172.21.15.5:0/2609704846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f5b8c004d50 con 0x5614ccf91cb0 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b9d9d1740 10 client.20001 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/2609704846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5614cd00be60 con 0x5614ccf91cb0 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b73fff640 1 -- 172.21.15.5:0/2609704846 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b840a4170 con 0x5614ccf91cb0 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b73fff640 1 -- 172.21.15.5:0/2609704846 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f5b8c030070 con 0x5614ccf91cb0 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b9d9d1740 10 client.20001 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b9d9d1740 10 client.20001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b9d9d1740 20 client.20001 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5614cd00c140 0x5614cd00e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b9d9d1740 4 client.20001 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:36:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/2609704846 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614ccef2d70 con 0x5614cd00c140 2024-09-11T22:36:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.813+0000 7f5b989de640 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5614cd00c140 0x5614cd00e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.814+0000 7f5b73fff640 10 client.20001 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.814+0000 7f5b989de640 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5614cd00c140 0x5614cd00e560 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7eff606f6457ce3 server_cookie=8426955971964b78 in_seq=0 out_seq=0 2024-09-11T22:36:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.862+0000 7f5b73fff640 1 -- 172.21.15.5:0/2609704846 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614ccef2d70 con 0x5614cd00c140 2024-09-11T22:36:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.862+0000 7f5b73fff640 10 client.20001 handle_command_reply: tid=0 2024-09-11T22:36:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.862+0000 7f5b9d9d1740 1 client.20001 shutdown 2024-09-11T22:36:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/2609704846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf91cb0 msgr2=0x5614cd00a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf91cb0 0x5614cd00a970 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009540 tx=0x7f5b8c009570 comp rx=0 tx=0).stop 2024-09-11T22:36:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/2609704846 shutdown_connections 2024-09-11T22:36:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5614cd00c140 0x5614cd00e560 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614ccf91cb0 0x5614cd00a970 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5614ccf87410 0x5614ccf95350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 1 --2- 172.21.15.5:0/2609704846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5614ccf86af0 0x5614ccf94e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b73fff640 0 client.20001 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/2609704846 >> 172.21.15.5:0/2609704846 conn(0x5614ccf83060 msgr2=0x5614ccf88550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/2609704846 shutdown_connections 2024-09-11T22:36:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 1 -- 172.21.15.5:0/2609704846 wait complete. 2024-09-11T22:36:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.863+0000 7f5b9d9d1740 20 client.20001 trim_cache size 0 max 16384 2024-09-11T22:36:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:36:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:36:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:36:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:36:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 -- 172.21.15.5:0/34691335 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5b7c0a2e20 msgr2=0x7f5b7c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5b7c0a2e20 0x7f5b7c0a52e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c04d000 tx=0x7f5b8c04b000 comp rx=0 tx=0).stop 2024-09-11T22:36:50.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 -- 172.21.15.5:0/34691335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b94101390 msgr2=0x7f5b94198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b94101390 0x7f5b94198870 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400d950 tx=0x7f5b8400de10 comp rx=0 tx=0).stop 2024-09-11T22:36:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 -- 172.21.15.5:0/34691335 shutdown_connections 2024-09-11T22:36:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b94106a00 0x7f5b9419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b94101390 0x7f5b94198870 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5b7c0a2e20 0x7f5b7c0a52e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 --2- 172.21.15.5:0/34691335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b941009f0 0x7f5b94198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 -- 172.21.15.5:0/34691335 >> 172.21.15.5:0/34691335 conn(0x7f5b940fc6b0 msgr2=0x7f5b940fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 -- 172.21.15.5:0/34691335 shutdown_connections 2024-09-11T22:36:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:36:50.864+0000 7f5b9ac42640 1 -- 172.21.15.5:0/34691335 wait complete. 2024-09-11T22:36:50.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:36:51.250 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:51.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 9.40 MB/sec warmup 69 sec latency 23441.364 ms 2024-09-11T22:36:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.773+0000 7fc54c8e8640 1 -- 172.21.15.5:0/3728894709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548102570 msgr2=0x7fc5481029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.773+0000 7fc54c8e8640 1 --2- 172.21.15.5:0/3728894709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548102570 0x7fc5481029b0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fc530009900 tx=0x7fc53002a480 comp rx=0 tx=0).stop 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.773+0000 7fc54c8e8640 1 -- 172.21.15.5:0/3728894709 shutdown_connections 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.773+0000 7fc54c8e8640 1 --2- 172.21.15.5:0/3728894709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc548102ef0 0x7fc54810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.773+0000 7fc54c8e8640 1 --2- 172.21.15.5:0/3728894709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548102570 0x7fc5481029b0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.773+0000 7fc54c8e8640 1 --2- 172.21.15.5:0/3728894709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc548100e20 0x7fc548101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.773+0000 7fc54c8e8640 1 -- 172.21.15.5:0/3728894709 >> 172.21.15.5:0/3728894709 conn(0x7fc5480fc7d0 msgr2=0x7fc5480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.773+0000 7fc54c8e8640 1 -- 172.21.15.5:0/3728894709 shutdown_connections 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.773+0000 7fc54c8e8640 1 -- 172.21.15.5:0/3728894709 wait complete. 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc54c8e8640 1 Processor -- start 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc54c8e8640 1 -- start start 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc54c8e8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc548100e20 0x7fc5480fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc54c8e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548102570 0x7fc5480fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc54c8e8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc548102ef0 0x7fc5480fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc54c8e8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc548110fc0 con 0x7fc548102570 2024-09-11T22:36:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc54c8e8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc548110e40 con 0x7fc548102ef0 2024-09-11T22:36:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc54c8e8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc548111140 con 0x7fc548100e20 2024-09-11T22:36:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc53ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548102570 0x7fc5480fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc53ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548102570 0x7fc5480fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55290/0 (socket says 172.21.15.5:55290) 2024-09-11T22:36:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc53ffff640 1 -- 172.21.15.5:0/2490695807 learned_addr learned my addr 172.21.15.5:0/2490695807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc546d9d640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc548102ef0 0x7fc5480fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc54659c640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc548100e20 0x7fc5480fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc53ffff640 1 -- 172.21.15.5:0/2490695807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc548100e20 msgr2=0x7fc5480fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc53ffff640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc548100e20 0x7fc5480fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc53ffff640 1 -- 172.21.15.5:0/2490695807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc548102ef0 msgr2=0x7fc5480fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc53ffff640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc548102ef0 0x7fc5480fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.774+0000 7fc53ffff640 1 -- 172.21.15.5:0/2490695807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5481a6ad0 con 0x7fc548102570 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.775+0000 7fc53ffff640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548102570 0x7fc5480fd1c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fc530006010 tx=0x7fc530003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.775+0000 7fc53f7fe640 1 -- 172.21.15.5:0/2490695807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53003f070 con 0x7fc548102570 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.775+0000 7fc54c8e8640 1 -- 172.21.15.5:0/2490695807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5481a6d60 con 0x7fc548102570 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.775+0000 7fc53f7fe640 1 -- 172.21.15.5:0/2490695807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc530031e30 con 0x7fc548102570 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.775+0000 7fc53f7fe640 1 -- 172.21.15.5:0/2490695807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530035030 con 0x7fc548102570 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.775+0000 7fc54c8e8640 1 -- 172.21.15.5:0/2490695807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5481a71e0 con 0x7fc548102570 2024-09-11T22:36:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.777+0000 7fc54ecc1740 1 Processor -- start 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.777+0000 7fc53f7fe640 1 -- 172.21.15.5:0/2490695807 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc530042070 con 0x7fc548102570 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54ecc1740 1 -- start start 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355bb40 0x5584b355a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355a5d0 0x5584b355aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b3565370 0x5584b3567760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5584b34c02f0 con 0x5584b355a5d0 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5584b3467cf0 con 0x5584b3565370 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5584b34bc2b0 con 0x5584b355bb40 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355a5d0 0x5584b355aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355a5d0 0x5584b355aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55304/0 (socket says 172.21.15.5:55304) 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54659c640 1 -- 172.21.15.5:0/912486190 learned_addr learned my addr 172.21.15.5:0/912486190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc53ffff640 1 --2- 172.21.15.5:0/912486190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b3565370 0x5584b3567760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc546d9d640 1 --2- 172.21.15.5:0/912486190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355bb40 0x5584b355a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54659c640 1 -- 172.21.15.5:0/912486190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355bb40 msgr2=0x5584b355a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54659c640 1 --2- 172.21.15.5:0/912486190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355bb40 0x5584b355a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54659c640 1 -- 172.21.15.5:0/912486190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b3565370 msgr2=0x5584b3567760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54659c640 1 --2- 172.21.15.5:0/912486190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b3565370 0x5584b3567760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc54659c640 1 -- 172.21.15.5:0/912486190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b344c3a0 con 0x5584b355a5d0 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc53f7fe640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5340a2cf0 0x7fc5340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.778+0000 7fc546d9d640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5340a2cf0 0x7fc5340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.779+0000 7fc54659c640 1 --2- 172.21.15.5:0/912486190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355a5d0 0x5584b355aa10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc538009a70 tx=0x7fc538009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04e61cd92b72ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.779+0000 7fc53f7fe640 1 -- 172.21.15.5:0/2490695807 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fc5300efd00 con 0x7fc548102570 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.779+0000 7fc53cff9640 1 -- 172.21.15.5:0/912486190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc538017070 con 0x5584b355a5d0 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.779+0000 7fc53cff9640 1 -- 172.21.15.5:0/912486190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc538013470 con 0x5584b355a5d0 2024-09-11T22:36:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.779+0000 7fc53cff9640 1 -- 172.21.15.5:0/912486190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380125f0 con 0x5584b355a5d0 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.779+0000 7fc54ecc1740 1 -- 172.21.15.5:0/912486190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355a5d0 msgr2=0x5584b355aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.779+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/912486190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355a5d0 0x5584b355aa10 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc538009a70 tx=0x7fc538009f30 comp rx=0 tx=0).stop 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.779+0000 7fc546d9d640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5340a2cf0 0x7fc5340a51b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc54000aec0 tx=0x7fc54000a4d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 -- 172.21.15.5:0/912486190 shutdown_connections 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/912486190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b3565370 0x5584b3567760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/912486190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355a5d0 0x5584b355aa10 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/912486190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355bb40 0x5584b355a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 -- 172.21.15.5:0/912486190 >> 172.21.15.5:0/912486190 conn(0x5584b3523310 msgr2=0x5584b3556c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 -- 172.21.15.5:0/912486190 shutdown_connections 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 -- 172.21.15.5:0/912486190 wait complete. 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 -- start start 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 Processor -- start 2024-09-11T22:36:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 0x5584b3564d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b3561fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b3565370 0x5584b3562530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5584b3568070 con 0x5584b355bb40 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5584b347b060 con 0x5584b355a5d0 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5584b35681f0 con 0x5584b3565370 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.780+0000 7fc54ecc1740 10 client.0 _ll_register_callbacks cb 0x5584b31ce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc53ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b3561fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc54659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 0x5584b3564d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc546d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b3565370 0x5584b3562530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc54659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 0x5584b3564d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57590/0 (socket says 172.21.15.5:57590) 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc54659c640 1 -- 172.21.15.5:0/605181871 learned_addr learned my addr 172.21.15.5:0/605181871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:51.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc53ffff640 1 -- 172.21.15.5:0/605181871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b3565370 msgr2=0x5584b3562530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc53ffff640 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b3565370 0x5584b3562530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc53ffff640 1 -- 172.21.15.5:0/605181871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 msgr2=0x5584b3564d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc53ffff640 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 0x5584b3564d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc53ffff640 1 -- 172.21.15.5:0/605181871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b32313c0 con 0x5584b355bb40 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc54659c640 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 0x5584b3564d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc53ffff640 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b3561fb0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fc5300ed3e0 tx=0x7fc530009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fa0e6de7ad398c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc52dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc52dffb640 1 -- 172.21.15.5:0/605181871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530049070 con 0x5584b355bb40 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc52dffb640 1 -- 172.21.15.5:0/605181871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fc5300c0230 con 0x5584b355bb40 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc54ecc1740 1 -- 172.21.15.5:0/605181871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc538013470 con 0x5584b355bb40 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc52dffb640 1 -- 172.21.15.5:0/605181871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc5300bbcf0 con 0x5584b355bb40 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc54ecc1740 10 client.20013 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:51.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.781+0000 7fc54ecc1740 1 -- 172.21.15.5:0/605181871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5584b3573aa0 con 0x5584b355bb40 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.782+0000 7fc52dffb640 1 -- 172.21.15.5:0/605181871 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc538013470 con 0x5584b355bb40 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.782+0000 7fc52dffb640 1 -- 172.21.15.5:0/605181871 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fc5300b8820 con 0x5584b355bb40 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.782+0000 7fc54ecc1740 10 client.20013 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.782+0000 7fc54ecc1740 10 client.20013 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.782+0000 7fc54ecc1740 20 client.20013 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.782+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5584b3574f50 0x5584b3577390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.782+0000 7fc54ecc1740 4 client.20013 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.782+0000 7fc54ecc1740 1 -- 172.21.15.5:0/605181871 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584b34cd860 con 0x5584b3574f50 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.782+0000 7fc54659c640 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5584b3574f50 0x5584b3577390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.783+0000 7fc54659c640 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5584b3574f50 0x5584b3577390 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=180c6916cd5d7ff8 server_cookie=495a9e63133fc375 in_seq=0 out_seq=0 2024-09-11T22:36:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.783+0000 7fc52dffb640 10 client.20013 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.783+0000 7fc52dffb640 1 -- 172.21.15.5:0/605181871 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584b34cd860 con 0x5584b3574f50 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.783+0000 7fc52dffb640 10 client.20013 handle_command_reply: tid=0 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.783+0000 7fc54ecc1740 1 client.20013 shutdown 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.783+0000 7fc54ecc1740 1 -- 172.21.15.5:0/605181871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 msgr2=0x5584b3561fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.783+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b3561fb0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fc5300ed3e0 tx=0x7fc530009620 comp rx=0 tx=0).stop 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.783+0000 7fc54ecc1740 1 -- 172.21.15.5:0/605181871 shutdown_connections 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.783+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5584b3574f50 0x5584b3577390 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.783+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b3565370 0x5584b3562530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.784+0000 7fc52dffb640 0 client.20013 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.784+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b3561fb0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.784+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/605181871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 0x5584b3564d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.784+0000 7fc54ecc1740 1 -- 172.21.15.5:0/605181871 >> 172.21.15.5:0/605181871 conn(0x5584b3523310 msgr2=0x5584b355c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.784+0000 7fc54ecc1740 1 -- 172.21.15.5:0/605181871 shutdown_connections 2024-09-11T22:36:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.784+0000 7fc54ecc1740 1 -- 172.21.15.5:0/605181871 wait complete. 2024-09-11T22:36:51.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.784+0000 7fc54ecc1740 20 client.20013 trim_cache size 0 max 16384 2024-09-11T22:36:51.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc54ecc1740 1 Processor -- start 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc54ecc1740 1 -- start start 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355a5d0 0x5584b3556df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355bb40 0x5584b3571310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b3565370 0x5584b357cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481afd40 con 0x5584b3565370 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481b1970 con 0x5584b355bb40 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481b0410 con 0x5584b355a5d0 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc546d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b3565370 0x5584b357cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc546d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b3565370 0x5584b357cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55308/0 (socket says 172.21.15.5:55308) 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc546d9d640 1 -- 172.21.15.5:0/2769111892 learned_addr learned my addr 172.21.15.5:0/2769111892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc53ffff640 1 --2- 172.21.15.5:0/2769111892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355bb40 0x5584b3571310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.797+0000 7fc54659c640 1 --2- 172.21.15.5:0/2769111892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355a5d0 0x5584b3556df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.798+0000 7fc546d9d640 1 -- 172.21.15.5:0/2769111892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355a5d0 msgr2=0x5584b3556df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.798+0000 7fc546d9d640 1 --2- 172.21.15.5:0/2769111892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355a5d0 0x5584b3556df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.798+0000 7fc546d9d640 1 -- 172.21.15.5:0/2769111892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355bb40 msgr2=0x5584b3571310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.798+0000 7fc546d9d640 1 --2- 172.21.15.5:0/2769111892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355bb40 0x5584b3571310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.798+0000 7fc546d9d640 1 -- 172.21.15.5:0/2769111892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584b35779c0 con 0x5584b3565370 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.798+0000 7fc54659c640 1 --2- 172.21.15.5:0/2769111892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355a5d0 0x5584b3556df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.798+0000 7fc546d9d640 1 --2- 172.21.15.5:0/2769111892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b3565370 0x5584b357cc50 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fc54001d630 tx=0x7fc540014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eef545549587f60 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.798+0000 7fc52ffff640 1 -- 172.21.15.5:0/2769111892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54001c6d0 con 0x5584b3565370 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.798+0000 7fc52ffff640 1 -- 172.21.15.5:0/2769111892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc54001cd50 con 0x5584b3565370 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.798+0000 7fc52ffff640 1 -- 172.21.15.5:0/2769111892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54000f430 con 0x5584b3565370 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 -- 172.21.15.5:0/2769111892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b3565370 msgr2=0x5584b357cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/2769111892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b3565370 0x5584b357cc50 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fc54001d630 tx=0x7fc540014000 comp rx=0 tx=0).stop 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 -- 172.21.15.5:0/2769111892 shutdown_connections 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/2769111892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b3565370 0x5584b357cc50 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/2769111892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355bb40 0x5584b3571310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/2769111892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b355a5d0 0x5584b3556df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 -- 172.21.15.5:0/2769111892 >> 172.21.15.5:0/2769111892 conn(0x5584b3523310 msgr2=0x5584b35566e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 -- 172.21.15.5:0/2769111892 shutdown_connections 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 -- 172.21.15.5:0/2769111892 wait complete. 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 -- start start 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 Processor -- start 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 0x5584b3569b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b356a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b3565370 0x5584b35d8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481b12a0 con 0x5584b355bb40 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481b0b10 con 0x5584b355a5d0 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc5481add40 con 0x5584b3565370 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc54ecc1740 10 client.0 _ll_register_callbacks cb 0x7fc530005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.799+0000 7fc53ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b356a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc53ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b356a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55322/0 (socket says 172.21.15.5:55322) 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc53ffff640 1 -- 172.21.15.5:0/3801654973 learned_addr learned my addr 172.21.15.5:0/3801654973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc546d9d640 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b3565370 0x5584b35d8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc54659c640 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 0x5584b3569b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc53ffff640 1 -- 172.21.15.5:0/3801654973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b3565370 msgr2=0x5584b35d8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc53ffff640 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b3565370 0x5584b35d8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc53ffff640 1 -- 172.21.15.5:0/3801654973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 msgr2=0x5584b3569b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc53ffff640 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 0x5584b3569b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc53ffff640 1 -- 172.21.15.5:0/3801654973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584b355d120 con 0x5584b355bb40 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc53ffff640 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b356a050 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fc53003fff0 tx=0x7fc5300b8a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9168d00faac673ad server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc52dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc52dffb640 1 -- 172.21.15.5:0/3801654973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc530049070 con 0x5584b355bb40 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc52dffb640 1 -- 172.21.15.5:0/3801654973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fc5300bc410 con 0x5584b355bb40 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc54ecc1740 1 -- 172.21.15.5:0/3801654973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc54001cd50 con 0x5584b355bb40 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc52dffb640 1 -- 172.21.15.5:0/3801654973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fc5300bbb10 con 0x5584b355bb40 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc54ecc1740 10 client.20019 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.800+0000 7fc54ecc1740 1 -- 172.21.15.5:0/3801654973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5584b3557960 con 0x5584b355bb40 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.801+0000 7fc52dffb640 1 -- 172.21.15.5:0/3801654973 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc54001cd50 con 0x5584b355bb40 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.801+0000 7fc52dffb640 1 -- 172.21.15.5:0/3801654973 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fc5300c4990 con 0x5584b355bb40 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.801+0000 7fc54ecc1740 10 client.20019 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.801+0000 7fc54ecc1740 10 client.20019 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.801+0000 7fc54ecc1740 20 client.20019 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.801+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5584b35e0700 0x5584b35e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.801+0000 7fc54ecc1740 4 client.20019 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.801+0000 7fc54ecc1740 1 -- 172.21.15.5:0/3801654973 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584b34c7d70 con 0x5584b35e0700 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.801+0000 7fc54659c640 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5584b35e0700 0x5584b35e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54659c640 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5584b35e0700 0x5584b35e2b20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35af78104f2d362 server_cookie=61dc59028b3faed9 in_seq=0 out_seq=0 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc52dffb640 10 client.20019 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc52dffb640 1 -- 172.21.15.5:0/3801654973 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584b34c7d70 con 0x5584b35e0700 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc52dffb640 10 client.20019 handle_command_reply: tid=0 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 client.20019 shutdown 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 -- 172.21.15.5:0/3801654973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 msgr2=0x5584b356a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b356a050 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fc53003fff0 tx=0x7fc5300b8a50 comp rx=0 tx=0).stop 2024-09-11T22:36:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 -- 172.21.15.5:0/3801654973 shutdown_connections 2024-09-11T22:36:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x5584b35e0700 0x5584b35e2b20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5584b3565370 0x5584b35d8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584b355bb40 0x5584b356a050 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 --2- 172.21.15.5:0/3801654973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5584b355a5d0 0x5584b3569b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc52dffb640 0 client.20019 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:36:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 -- 172.21.15.5:0/3801654973 >> 172.21.15.5:0/3801654973 conn(0x5584b3523310 msgr2=0x5584b35554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 -- 172.21.15.5:0/3801654973 shutdown_connections 2024-09-11T22:36:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 1 -- 172.21.15.5:0/3801654973 wait complete. 2024-09-11T22:36:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.802+0000 7fc54ecc1740 20 client.20019 trim_cache size 0 max 16384 2024-09-11T22:36:51.811 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:36:51.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 -- 172.21.15.5:0/2490695807 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5340a2cf0 msgr2=0x7fc5340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:51.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5340a2cf0 0x7fc5340a51b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc54000aec0 tx=0x7fc54000a4d0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 -- 172.21.15.5:0/2490695807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548102570 msgr2=0x7fc5480fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548102570 0x7fc5480fd1c0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fc530006010 tx=0x7fc530003c80 comp rx=0 tx=0).stop 2024-09-11T22:36:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 -- 172.21.15.5:0/2490695807 shutdown_connections 2024-09-11T22:36:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc5340a2cf0 0x7fc5340a51b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc548102ef0 0x7fc5480fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc548102570 0x7fc5480fd1c0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 --2- 172.21.15.5:0/2490695807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc548100e20 0x7fc5480fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 -- 172.21.15.5:0/2490695807 >> 172.21.15.5:0/2490695807 conn(0x7fc5480fc7d0 msgr2=0x7fc54806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 -- 172.21.15.5:0/2490695807 shutdown_connections 2024-09-11T22:36:51.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:51.803+0000 7fc54c8e8640 1 -- 172.21.15.5:0/2490695807 wait complete. 2024-09-11T22:36:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:51 smithi005 ceph-mon[30943]: pgmap v466: 97 pgs: 97 active+clean; 333 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 859 KiB/s wr, 73 op/s 2024-09-11T22:36:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:51 smithi159 ceph-mon[35534]: pgmap v466: 97 pgs: 97 active+clean; 333 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 859 KiB/s wr, 73 op/s 2024-09-11T22:36:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:51 smithi172 ceph-mon[35461]: pgmap v466: 97 pgs: 97 active+clean; 333 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 859 KiB/s wr, 73 op/s 2024-09-11T22:36:52.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:36:52.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 9.27 MB/sec warmup 70 sec latency 24441.413 ms 2024-09-11T22:36:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:53.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.316+0000 7ff9d4477640 1 -- 172.21.15.5:0/3849121495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9cc102d30 msgr2=0x7ff9cc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:53.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.316+0000 7ff9d4477640 1 --2- 172.21.15.5:0/3849121495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9cc102d30 0x7ff9cc103110 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009cd0 tx=0x7ff9bc02a910 comp rx=0 tx=0).stop 2024-09-11T22:36:53.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 -- 172.21.15.5:0/3849121495 shutdown_connections 2024-09-11T22:36:53.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 --2- 172.21.15.5:0/3849121495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9cc1082a0 0x7ff9cc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 --2- 172.21.15.5:0/3849121495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc1036e0 0x7ff9cc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 --2- 172.21.15.5:0/3849121495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9cc102d30 0x7ff9cc103110 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 -- 172.21.15.5:0/3849121495 >> 172.21.15.5:0/3849121495 conn(0x7ff9cc0fc7d0 msgr2=0x7ff9cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 -- 172.21.15.5:0/3849121495 shutdown_connections 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 -- 172.21.15.5:0/3849121495 wait complete. 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 Processor -- start 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 -- start start 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc102d30 0x7ff9cc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9cc1036e0 0x7ff9cc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9cc1082a0 0x7ff9cc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1120e0 con 0x7ff9cc1082a0 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc111f60 con 0x7ff9cc102d30 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.317+0000 7ff9d4477640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc112260 con 0x7ff9cc1036e0 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2a14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9cc1082a0 0x7ff9cc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2a14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9cc1082a0 0x7ff9cc19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55346/0 (socket says 172.21.15.5:55346) 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2a14640 1 -- 172.21.15.5:0/183920028 learned_addr learned my addr 172.21.15.5:0/183920028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9cc1036e0 0x7ff9cc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2213640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc102d30 0x7ff9cc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2a14640 1 -- 172.21.15.5:0/183920028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9cc1036e0 msgr2=0x7ff9cc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2a14640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9cc1036e0 0x7ff9cc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2a14640 1 -- 172.21.15.5:0/183920028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc102d30 msgr2=0x7ff9cc198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2a14640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc102d30 0x7ff9cc198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2a14640 1 -- 172.21.15.5:0/183920028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc19d520 con 0x7ff9cc1082a0 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2213640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc102d30 0x7ff9cc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d2a14640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9cc1082a0 0x7ff9cc19ce40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c970 tx=0x7ff9c400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9cb7fe640 1 -- 172.21.15.5:0/183920028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4007bb0 con 0x7ff9cc1082a0 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d4477640 1 -- 172.21.15.5:0/183920028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc19d810 con 0x7ff9cc1082a0 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9cb7fe640 1 -- 172.21.15.5:0/183920028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4007d50 con 0x7ff9cc1082a0 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9cb7fe640 1 -- 172.21.15.5:0/183920028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4018770 con 0x7ff9cc1082a0 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.318+0000 7ff9d4477640 1 -- 172.21.15.5:0/183920028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc1a50f0 con 0x7ff9cc1082a0 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.321+0000 7ff9d6850740 1 Processor -- start 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.321+0000 7ff9cb7fe640 1 -- 172.21.15.5:0/183920028 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff9c400f450 con 0x7ff9cc1082a0 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d6850740 1 -- start start 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c81407ab40 0x55c814079090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 0x55c814079a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c814084370 0x55c814086760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d6850740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c813fdf2f0 con 0x55c814084370 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d6850740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c813f86cf0 con 0x55c8140795d0 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d6850740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c813fdb2b0 con 0x55c81407ab40 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d1a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c814084370 0x55c814086760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d1a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c814084370 0x55c814086760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55356/0 (socket says 172.21.15.5:55356) 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d1a12640 1 -- 172.21.15.5:0/1844812423 learned_addr learned my addr 172.21.15.5:0/1844812423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d2213640 1 --2- 172.21.15.5:0/1844812423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 0x55c814079a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/1844812423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c81407ab40 0x55c814079090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.322+0000 7ff9cb7fe640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9b40a2cf0 0x7ff9b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9d1a12640 1 -- 172.21.15.5:0/1844812423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c81407ab40 msgr2=0x55c814079090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/1844812423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c81407ab40 0x55c814079090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9d1a12640 1 -- 172.21.15.5:0/1844812423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 msgr2=0x55c814079a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/1844812423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 0x55c814079a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9d1a12640 1 -- 172.21.15.5:0/1844812423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c813f6b3a0 con 0x55c814084370 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9d2a14640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9b40a2cf0 0x7ff9b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9cb7fe640 1 -- 172.21.15.5:0/183920028 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7ff9c40cf500 con 0x7ff9cc1082a0 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/1844812423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c814084370 0x55c814086760 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009ef0 tx=0x7ff9c0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cd328f9633cdf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9c8ff9640 1 -- 172.21.15.5:0/1844812423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0017470 con 0x55c814084370 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9c8ff9640 1 -- 172.21.15.5:0/1844812423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c001c070 con 0x55c814084370 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.323+0000 7ff9c8ff9640 1 -- 172.21.15.5:0/1844812423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0015960 con 0x55c814084370 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 -- 172.21.15.5:0/1844812423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c814084370 msgr2=0x55c814086760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d2a14640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9b40a2cf0 0x7ff9b40a51b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40cd1e0 tx=0x7ff9c406d970 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1844812423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c814084370 0x55c814086760 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009ef0 tx=0x7ff9c0009580 comp rx=0 tx=0).stop 2024-09-11T22:36:53.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 -- 172.21.15.5:0/1844812423 shutdown_connections 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1844812423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c814084370 0x55c814086760 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1844812423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 0x55c814079a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1844812423 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c81407ab40 0x55c814079090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 -- 172.21.15.5:0/1844812423 >> 172.21.15.5:0/1844812423 conn(0x55c814042310 msgr2=0x55c814075c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 -- 172.21.15.5:0/1844812423 shutdown_connections 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 -- 172.21.15.5:0/1844812423 wait complete. 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 -- start start 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.324+0000 7ff9d6850740 1 Processor -- start 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8140795d0 0x55c814083d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c814081020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c814084370 0x55c814081610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d6850740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c814087070 con 0x55c81407ab40 2024-09-11T22:36:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d6850740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c813f9a060 con 0x55c814084370 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d6850740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c8140871f0 con 0x55c8140795d0 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d6850740 10 client.0 _ll_register_callbacks cb 0x55c813ced8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d2213640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c814081020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d2213640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c814081020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55362/0 (socket says 172.21.15.5:55362) 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d2213640 1 -- 172.21.15.5:0/1057181108 learned_addr learned my addr 172.21.15.5:0/1057181108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d2213640 1 -- 172.21.15.5:0/1057181108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8140795d0 msgr2=0x55c814083d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d2213640 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8140795d0 0x55c814083d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c814084370 0x55c814081610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d2213640 1 -- 172.21.15.5:0/1057181108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c814084370 msgr2=0x55c814081610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d2213640 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c814084370 0x55c814081610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d2213640 1 -- 172.21.15.5:0/1057181108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c813d503c0 con 0x55c81407ab40 2024-09-11T22:36:53.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d2213640 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c814081020 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009cd0 tx=0x7ff9bc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c89c95001f30ab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/1057181108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc003e30 con 0x55c81407ab40 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/1057181108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7ff9bc031460 con 0x55c81407ab40 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.325+0000 7ff9d6850740 1 -- 172.21.15.5:0/1057181108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c001c070 con 0x55c81407ab40 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/1057181108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff9bc004b40 con 0x55c81407ab40 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9d6850740 10 client.20031 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9d6850740 1 -- 172.21.15.5:0/1057181108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55c814092b10 con 0x55c81407ab40 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/1057181108 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c001c070 con 0x55c81407ab40 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/1057181108 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7ff9bc0045c0 con 0x55c81407ab40 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9d6850740 10 client.20031 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9d6850740 10 client.20031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9d6850740 20 client.20031 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c814094090 0x55c814096510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9d6850740 4 client.20031 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9d6850740 1 -- 172.21.15.5:0/1057181108 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c813fec860 con 0x55c814094090 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.326+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c814094090 0x55c814096510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.327+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c814094090 0x55c814096510 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=461861523cda6633 server_cookie=ba64a97363d1e245 in_seq=0 out_seq=0 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.327+0000 7ff9b1ffb640 10 client.20031 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/1057181108 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c813fec860 con 0x55c814094090 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9b1ffb640 10 client.20031 handle_command_reply: tid=0 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 client.20031 shutdown 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 -- 172.21.15.5:0/1057181108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 msgr2=0x55c814081020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c814081020 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009cd0 tx=0x7ff9bc003970 comp rx=0 tx=0).stop 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 -- 172.21.15.5:0/1057181108 shutdown_connections 2024-09-11T22:36:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c814094090 0x55c814096510 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c814084370 0x55c814081610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c814081020 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9b1ffb640 0 client.20031 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1057181108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8140795d0 0x55c814083d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 -- 172.21.15.5:0/1057181108 >> 172.21.15.5:0/1057181108 conn(0x55c814042310 msgr2=0x55c81407b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 -- 172.21.15.5:0/1057181108 shutdown_connections 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 1 -- 172.21.15.5:0/1057181108 wait complete. 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.328+0000 7ff9d6850740 20 client.20031 trim_cache size 0 max 16384 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d6850740 1 Processor -- start 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d6850740 1 -- start start 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 0x55c814090310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c8140808e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c814084370 0x55c81409bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d6850740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1ada20 con 0x55c81407ab40 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d6850740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1af650 con 0x55c8140795d0 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d6850740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1ae0f0 con 0x55c814084370 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d2213640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c8140808e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d2213640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c8140808e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55370/0 (socket says 172.21.15.5:55370) 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d2213640 1 -- 172.21.15.5:0/1762501840 learned_addr learned my addr 172.21.15.5:0/1762501840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.341+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/1762501840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 0x55c814090310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9d2213640 1 -- 172.21.15.5:0/1762501840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c814084370 msgr2=0x55c81409bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9d2213640 1 --2- 172.21.15.5:0/1762501840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c814084370 0x55c81409bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9d2213640 1 -- 172.21.15.5:0/1762501840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 msgr2=0x55c814090310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9d2213640 1 --2- 172.21.15.5:0/1762501840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 0x55c814090310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9d2213640 1 -- 172.21.15.5:0/1762501840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c814075de0 con 0x55c81407ab40 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/1762501840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 0x55c814090310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9d2213640 1 --2- 172.21.15.5:0/1762501840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c8140808e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc02f830 tx=0x7ff9bc032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4acf87b75271333b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9b3fff640 1 -- 172.21.15.5:0/1762501840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc02f3d0 con 0x55c81407ab40 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9b3fff640 1 -- 172.21.15.5:0/1762501840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9bc036040 con 0x55c81407ab40 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9b3fff640 1 -- 172.21.15.5:0/1762501840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0305d0 con 0x55c81407ab40 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.342+0000 7ff9d6850740 1 -- 172.21.15.5:0/1762501840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 msgr2=0x55c8140808e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1762501840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c8140808e0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc02f830 tx=0x7ff9bc032f30 comp rx=0 tx=0).stop 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 -- 172.21.15.5:0/1762501840 shutdown_connections 2024-09-11T22:36:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1762501840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c814084370 0x55c81409bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1762501840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81407ab40 0x55c8140808e0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 --2- 172.21.15.5:0/1762501840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8140795d0 0x55c814090310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 -- 172.21.15.5:0/1762501840 >> 172.21.15.5:0/1762501840 conn(0x55c814042310 msgr2=0x55c814080000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 -- 172.21.15.5:0/1762501840 shutdown_connections 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 -- 172.21.15.5:0/1762501840 wait complete. 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 -- start start 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 Processor -- start 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8140795d0 0x55c81407e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c81407ab40 0x55c81407e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c814084370 0x55c81407ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1aef80 con 0x55c8140795d0 2024-09-11T22:36:53.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1ae7f0 con 0x55c81407ab40 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.343+0000 7ff9d6850740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1aba20 con 0x55c814084370 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d6850740 10 client.0 _ll_register_callbacks cb 0x7ff9bc02f320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d1a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8140795d0 0x55c81407e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d1a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8140795d0 0x55c81407e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55372/0 (socket says 172.21.15.5:55372) 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d1a12640 1 -- 172.21.15.5:0/4041759665 learned_addr learned my addr 172.21.15.5:0/4041759665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d2213640 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c81407ab40 0x55c81407e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d1a12640 1 -- 172.21.15.5:0/4041759665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c814084370 msgr2=0x55c81407ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c814084370 0x55c81407ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d1a12640 1 -- 172.21.15.5:0/4041759665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c81407ab40 msgr2=0x55c81407e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c81407ab40 0x55c81407e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d1a12640 1 -- 172.21.15.5:0/4041759665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8140fdde0 con 0x55c8140795d0 2024-09-11T22:36:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d2213640 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c81407ab40 0x55c81407e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d1a12640 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8140795d0 0x55c81407e050 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009ef0 tx=0x7ff9c0010fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3ea7d964032dcc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/4041759665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0005bd0 con 0x55c8140795d0 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d6850740 1 -- 172.21.15.5:0/4041759665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9bc036040 con 0x55c8140795d0 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/4041759665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7ff9c0005d70 con 0x55c8140795d0 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/4041759665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7ff9c0015690 con 0x55c8140795d0 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d6850740 10 client.20037 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.344+0000 7ff9d6850740 1 -- 172.21.15.5:0/4041759665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55c8140ff010 con 0x55c8140795d0 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/4041759665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9bc036040 con 0x55c8140795d0 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/4041759665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7ff9c00063a0 con 0x55c8140795d0 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9d6850740 10 client.20037 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9d6850740 10 client.20037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9d6850740 20 client.20037 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9d6850740 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8140ff500 0x55c814101920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9d6850740 4 client.20037 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9d6850740 1 -- 172.21.15.5:0/4041759665 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c813fe6d70 con 0x55c8140ff500 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9d2213640 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8140ff500 0x55c814101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9d2213640 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8140ff500 0x55c814101920 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e05754298a3784 server_cookie=227367028312b4dc in_seq=0 out_seq=0 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.345+0000 7ff9b1ffb640 10 client.20037 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9b1ffb640 1 -- 172.21.15.5:0/4041759665 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c813fe6d70 con 0x55c8140ff500 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9b1ffb640 10 client.20037 handle_command_reply: tid=0 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 client.20037 shutdown 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 -- 172.21.15.5:0/4041759665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8140795d0 msgr2=0x55c81407e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8140795d0 0x55c81407e050 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009ef0 tx=0x7ff9c0010fc0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 -- 172.21.15.5:0/4041759665 shutdown_connections 2024-09-11T22:36:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8140ff500 0x55c814101920 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9b1ffb640 0 client.20037 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c814084370 0x55c81407ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c81407ab40 0x55c81407e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 --2- 172.21.15.5:0/4041759665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8140795d0 0x55c81407e050 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 -- 172.21.15.5:0/4041759665 >> 172.21.15.5:0/4041759665 conn(0x55c814042310 msgr2=0x55c8140744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 -- 172.21.15.5:0/4041759665 shutdown_connections 2024-09-11T22:36:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 1 -- 172.21.15.5:0/4041759665 wait complete. 2024-09-11T22:36:53.379 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:36:53.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.346+0000 7ff9d6850740 20 client.20037 trim_cache size 0 max 16384 2024-09-11T22:36:53.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 -- 172.21.15.5:0/183920028 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9b40a2cf0 msgr2=0x7ff9b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:53.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9b40a2cf0 0x7ff9b40a51b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40cd1e0 tx=0x7ff9c406d970 comp rx=0 tx=0).stop 2024-09-11T22:36:53.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 -- 172.21.15.5:0/183920028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9cc1082a0 msgr2=0x7ff9cc19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9cc1082a0 0x7ff9cc19ce40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400c970 tx=0x7ff9c400cd70 comp rx=0 tx=0).stop 2024-09-11T22:36:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 -- 172.21.15.5:0/183920028 shutdown_connections 2024-09-11T22:36:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9cc1082a0 0x7ff9cc19ce40 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9cc1036e0 0x7ff9cc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9b40a2cf0 0x7ff9b40a51b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 --2- 172.21.15.5:0/183920028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9cc102d30 0x7ff9cc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 -- 172.21.15.5:0/183920028 >> 172.21.15.5:0/183920028 conn(0x7ff9cc0fc7d0 msgr2=0x7ff9cc105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 -- 172.21.15.5:0/183920028 shutdown_connections 2024-09-11T22:36:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:53.347+0000 7ff9d4477640 1 -- 172.21.15.5:0/183920028 wait complete. 2024-09-11T22:36:53.635 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 9.14 MB/sec warmup 71 sec latency 25441.512 ms 2024-09-11T22:36:54.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:53 smithi005 ceph-mon[30943]: pgmap v467: 97 pgs: 97 active+clean; 329 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 900 KiB/s wr, 99 op/s 2024-09-11T22:36:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:53 smithi159 ceph-mon[35534]: pgmap v467: 97 pgs: 97 active+clean; 329 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 900 KiB/s wr, 99 op/s 2024-09-11T22:36:54.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:53 smithi172 ceph-mon[35461]: pgmap v467: 97 pgs: 97 active+clean; 329 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 900 KiB/s wr, 99 op/s 2024-09-11T22:36:54.636 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 9.01 MB/sec warmup 72 sec latency 26441.617 ms 2024-09-11T22:36:55.062 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:36:55.062 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:36:55.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:36:55.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:55 smithi005 ceph-mon[30943]: pgmap v468: 97 pgs: 97 active+clean; 329 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 79 KiB/s wr, 33 op/s 2024-09-11T22:36:55.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:55 smithi159 ceph-mon[35534]: pgmap v468: 97 pgs: 97 active+clean; 329 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 79 KiB/s wr, 33 op/s 2024-09-11T22:36:55.346 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:55.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:55 smithi172 ceph-mon[35461]: pgmap v468: 97 pgs: 97 active+clean; 329 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 79 KiB/s wr, 33 op/s 2024-09-11T22:36:55.636 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 8.89 MB/sec warmup 73 sec latency 27441.677 ms 2024-09-11T22:36:55.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 -- 172.21.15.5:0/2442300352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8101de0 msgr2=0x7fbea81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:55.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 --2- 172.21.15.5:0/2442300352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8101de0 0x7fbea81021c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009ca0 tx=0x7fbea002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 -- 172.21.15.5:0/2442300352 shutdown_connections 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 --2- 172.21.15.5:0/2442300352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbea8106f10 0x7fbea810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 --2- 172.21.15.5:0/2442300352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8102790 0x7fbea81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 --2- 172.21.15.5:0/2442300352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8101de0 0x7fbea81021c0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 -- 172.21.15.5:0/2442300352 >> 172.21.15.5:0/2442300352 conn(0x7fbea80fd8c0 msgr2=0x7fbea80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 -- 172.21.15.5:0/2442300352 shutdown_connections 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 -- 172.21.15.5:0/2442300352 wait complete. 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 Processor -- start 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.856+0000 7fbeaf061640 1 -- start start 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeaf061640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8101de0 0x7fbea80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeaf061640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbea8102790 0x7fbea8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeaf061640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8106f10 0x7fbea8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeaf061640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8111f70 con 0x7fbea8101de0 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeaf061640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8111df0 con 0x7fbea8102790 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeaf061640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81120f0 con 0x7fbea8106f10 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeacdfd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8101de0 0x7fbea80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeacdfd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8101de0 0x7fbea80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55394/0 (socket says 172.21.15.5:55394) 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeacdfd640 1 -- 172.21.15.5:0/3895427624 learned_addr learned my addr 172.21.15.5:0/3895427624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbead5fe640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8106f10 0x7fbea8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeacdfd640 1 -- 172.21.15.5:0/3895427624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8106f10 msgr2=0x7fbea8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeacdfd640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8106f10 0x7fbea8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeacdfd640 1 -- 172.21.15.5:0/3895427624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbea8102790 msgr2=0x7fbea8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeacdfd640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbea8102790 0x7fbea8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeacdfd640 1 -- 172.21.15.5:0/3895427624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea81a6870 con 0x7fbea8101de0 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbeacdfd640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8101de0 0x7fbea80744f0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fbea002ade0 tx=0x7fbea00319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.857+0000 7fbea5ffb640 1 -- 172.21.15.5:0/3895427624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea002fa20 con 0x7fbea8101de0 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.858+0000 7fbea5ffb640 1 -- 172.21.15.5:0/3895427624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbea002fbc0 con 0x7fbea8101de0 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.858+0000 7fbeaf061640 1 -- 172.21.15.5:0/3895427624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea81a6b00 con 0x7fbea8101de0 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.858+0000 7fbea5ffb640 1 -- 172.21.15.5:0/3895427624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea00467b0 con 0x7fbea8101de0 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.858+0000 7fbeaf061640 1 -- 172.21.15.5:0/3895427624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea81a7040 con 0x7fbea8101de0 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.860+0000 7fbea5ffb640 1 -- 172.21.15.5:0/3895427624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbea0046950 con 0x7fbea8101de0 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.860+0000 7fbeb143a740 1 Processor -- start 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.860+0000 7fbeb143a740 1 -- start start 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 0x55777e4c2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4c5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbeb143a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55777e41d280 con 0x55777e4b7b60 2024-09-11T22:36:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbeb143a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55777e3c4b10 con 0x55777e4b8480 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbeb143a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55777e419240 con 0x55777e4c2d40 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbead5fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbead5fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4b7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55410/0 (socket says 172.21.15.5:55410) 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbead5fe640 1 -- 172.21.15.5:0/3154417109 learned_addr learned my addr 172.21.15.5:0/3154417109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbeacdfd640 1 --2- 172.21.15.5:0/3154417109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 0x55777e4c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbead5fe640 1 -- 172.21.15.5:0/3154417109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 msgr2=0x55777e4c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbead5fe640 1 --2- 172.21.15.5:0/3154417109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbead5fe640 1 -- 172.21.15.5:0/3154417109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 msgr2=0x55777e4c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbead5fe640 1 --2- 172.21.15.5:0/3154417109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 0x55777e4c2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbead5fe640 1 -- 172.21.15.5:0/3154417109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777e3d11f0 con 0x55777e4b7b60 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbeacdfd640 1 --2- 172.21.15.5:0/3154417109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 0x55777e4c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbead5fe640 1 --2- 172.21.15.5:0/3154417109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4b7f40 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000cb40 tx=0x7fbe9000e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b263ead9b211f38b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.861+0000 7fbea5ffb640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbe940a2cf0 0x7fbe940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbeacdfd640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbe940a2cf0 0x7fbe940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbe9effd640 1 -- 172.21.15.5:0/3154417109 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90005020 con 0x55777e4b7b60 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbe9effd640 1 -- 172.21.15.5:0/3154417109 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe900047a0 con 0x55777e4b7b60 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbe9effd640 1 -- 172.21.15.5:0/3154417109 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90010750 con 0x55777e4b7b60 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbea5ffb640 1 -- 172.21.15.5:0/3895427624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fbea00f0900 con 0x7fbea8101de0 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbeb143a740 1 -- 172.21.15.5:0/3154417109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 msgr2=0x55777e4b7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbeb143a740 1 --2- 172.21.15.5:0/3154417109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4b7f40 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000cb40 tx=0x7fbe9000e350 comp rx=0 tx=0).stop 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbeacdfd640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbe940a2cf0 0x7fbe940a51b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbea00ee1a0 tx=0x7fbea004f000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbeb143a740 1 -- 172.21.15.5:0/3154417109 shutdown_connections 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbeb143a740 1 --2- 172.21.15.5:0/3154417109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbeb143a740 1 --2- 172.21.15.5:0/3154417109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 0x55777e4c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbeb143a740 1 --2- 172.21.15.5:0/3154417109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4b7f40 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.862+0000 7fbeb143a740 1 -- 172.21.15.5:0/3154417109 >> 172.21.15.5:0/3154417109 conn(0x55777e4b40f0 msgr2=0x55777e4b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 1 -- 172.21.15.5:0/3154417109 shutdown_connections 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 1 -- 172.21.15.5:0/3154417109 wait complete. 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 1 -- start start 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 1 Processor -- start 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 0x55777e4bfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4c06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55777e4c5910 con 0x55777e4b8480 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55777e3d7f10 con 0x55777e4b7b60 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55777e4c5a90 con 0x55777e4c2d40 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbeb143a740 10 client.0 _ll_register_callbacks cb 0x55777e3a0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.863+0000 7fbea7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbea7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55422/0 (socket says 172.21.15.5:55422) 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbea7fff640 1 -- 172.21.15.5:0/3638337650 learned_addr learned my addr 172.21.15.5:0/3638337650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbea7fff640 1 -- 172.21.15.5:0/3638337650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 msgr2=0x55777e4c06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbea7fff640 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4c06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbea7fff640 1 -- 172.21.15.5:0/3638337650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 msgr2=0x55777e4bfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbea7fff640 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 0x55777e4bfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbea7fff640 1 -- 172.21.15.5:0/3638337650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777e18e6a0 con 0x55777e4b8480 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbea7fff640 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c0180 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800bbb0 tx=0x7fbe9800d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1646330fdc6c4b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbe87fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbe87fff640 1 -- 172.21.15.5:0/3638337650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98016070 con 0x55777e4b8480 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbeb143a740 1 -- 172.21.15.5:0/3638337650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe900047a0 con 0x55777e4b8480 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbe87fff640 1 -- 172.21.15.5:0/3638337650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fbe98010660 con 0x55777e4b8480 2024-09-11T22:36:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbe87fff640 1 -- 172.21.15.5:0/3638337650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fbe98022050 con 0x55777e4b8480 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.864+0000 7fbeb143a740 10 client.20049 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.865+0000 7fbeb143a740 1 -- 172.21.15.5:0/3638337650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55777e4d1510 con 0x55777e4b8480 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.865+0000 7fbe87fff640 1 -- 172.21.15.5:0/3638337650 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe900047a0 con 0x55777e4b8480 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.865+0000 7fbe87fff640 1 -- 172.21.15.5:0/3638337650 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fbe9800f760 con 0x55777e4b8480 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.865+0000 7fbeb143a740 10 client.20049 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.865+0000 7fbeb143a740 10 client.20049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.865+0000 7fbeb143a740 20 client.20049 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.865+0000 7fbeb143a740 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777e4d29e0 0x55777e4d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.865+0000 7fbeb143a740 4 client.20049 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.865+0000 7fbeb143a740 1 -- 172.21.15.5:0/3638337650 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55777e42a7f0 con 0x55777e4d29e0 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.865+0000 7fbead5fe640 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777e4d29e0 0x55777e4d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.866+0000 7fbead5fe640 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777e4d29e0 0x55777e4d4e60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53a053a43726b76e server_cookie=ca9128b655ce8887 in_seq=0 out_seq=0 2024-09-11T22:36:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.866+0000 7fbe87fff640 10 client.20049 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.866+0000 7fbe87fff640 1 -- 172.21.15.5:0/3638337650 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55777e42a7f0 con 0x55777e4d29e0 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.866+0000 7fbe87fff640 10 client.20049 handle_command_reply: tid=0 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 client.20049 shutdown 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 -- 172.21.15.5:0/3638337650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 msgr2=0x55777e4c0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c0180 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fbe9800bbb0 tx=0x7fbe9800d6b0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 -- 172.21.15.5:0/3638337650 shutdown_connections 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777e4d29e0 0x55777e4d4e60 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4c06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c0180 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 --2- 172.21.15.5:0/3638337650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 0x55777e4bfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 -- 172.21.15.5:0/3638337650 >> 172.21.15.5:0/3638337650 conn(0x55777e4b40f0 msgr2=0x55777e4b9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbe87fff640 0 client.20049 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 -- 172.21.15.5:0/3638337650 shutdown_connections 2024-09-11T22:36:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 1 -- 172.21.15.5:0/3638337650 wait complete. 2024-09-11T22:36:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.867+0000 7fbeb143a740 20 client.20049 trim_cache size 0 max 16384 2024-09-11T22:36:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:55.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.892+0000 7fbeb143a740 1 Processor -- start 2024-09-11T22:36:55.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbeb143a740 1 -- start start 2024-09-11T22:36:55.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4c1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 0x55777e4da770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4dacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbeb143a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81afba0 con 0x55777e4b7b60 2024-09-11T22:36:55.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbeb143a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81b17d0 con 0x55777e4b8480 2024-09-11T22:36:55.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbeb143a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81b0270 con 0x55777e4c2d40 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbead5fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4c1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbead5fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4c1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55432/0 (socket says 172.21.15.5:55432) 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbead5fe640 1 -- 172.21.15.5:0/2133165904 learned_addr learned my addr 172.21.15.5:0/2133165904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbea7fff640 1 --2- 172.21.15.5:0/2133165904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 0x55777e4da770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.893+0000 7fbead5fe640 1 --2- 172.21.15.5:0/2133165904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4dacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.894+0000 7fbead5fe640 1 -- 172.21.15.5:0/2133165904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 msgr2=0x55777e4dacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.894+0000 7fbead5fe640 1 --2- 172.21.15.5:0/2133165904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4dacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.894+0000 7fbead5fe640 1 -- 172.21.15.5:0/2133165904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 msgr2=0x55777e4da770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.894+0000 7fbead5fe640 1 --2- 172.21.15.5:0/2133165904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 0x55777e4da770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.894+0000 7fbead5fe640 1 -- 172.21.15.5:0/2133165904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777e4b82b0 con 0x55777e4b7b60 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.894+0000 7fbead5fe640 1 --2- 172.21.15.5:0/2133165904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4c1aa0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000e850 tx=0x7fbe90005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bdaa1bcae29bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:55.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.894+0000 7fbe9e7fc640 1 -- 172.21.15.5:0/2133165904 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe900037f0 con 0x55777e4b7b60 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.894+0000 7fbe9e7fc640 1 -- 172.21.15.5:0/2133165904 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe90005580 con 0x55777e4b7b60 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.894+0000 7fbe9e7fc640 1 -- 172.21.15.5:0/2133165904 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90010c20 con 0x55777e4b7b60 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.895+0000 7fbeb143a740 1 -- 172.21.15.5:0/2133165904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 msgr2=0x55777e4c1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.895+0000 7fbeb143a740 1 --2- 172.21.15.5:0/2133165904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4c1aa0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000e850 tx=0x7fbe90005740 comp rx=0 tx=0).stop 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.895+0000 7fbeb143a740 1 -- 172.21.15.5:0/2133165904 shutdown_connections 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.895+0000 7fbeb143a740 1 --2- 172.21.15.5:0/2133165904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4dacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.895+0000 7fbeb143a740 1 --2- 172.21.15.5:0/2133165904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b8480 0x55777e4da770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.895+0000 7fbeb143a740 1 --2- 172.21.15.5:0/2133165904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b7b60 0x55777e4c1aa0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.895+0000 7fbeb143a740 1 -- 172.21.15.5:0/2133165904 >> 172.21.15.5:0/2133165904 conn(0x55777e4b40f0 msgr2=0x55777e4c1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.895+0000 7fbeb143a740 1 -- 172.21.15.5:0/2133165904 shutdown_connections 2024-09-11T22:36:55.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.895+0000 7fbeb143a740 1 -- 172.21.15.5:0/2133165904 wait complete. 2024-09-11T22:36:55.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.895+0000 7fbeb143a740 1 -- start start 2024-09-11T22:36:55.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbeb143a740 1 Processor -- start 2024-09-11T22:36:55.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 0x55777e4cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbeb143a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbeb143a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81b1100 con 0x55777e4b8480 2024-09-11T22:36:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbeb143a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81b0970 con 0x55777e4b7b60 2024-09-11T22:36:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbeb143a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81adba0 con 0x55777e4c2d40 2024-09-11T22:36:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbeb143a740 10 client.0 _ll_register_callbacks cb 0x7fbe98005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbead5fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 0x55777e4cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbea7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbead5fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 0x55777e4cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57688/0 (socket says 172.21.15.5:57688) 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbead5fe640 1 -- 172.21.15.5:0/1928745407 learned_addr learned my addr 172.21.15.5:0/1928745407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.896+0000 7fbead5fe640 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbead5fe640 1 -- 172.21.15.5:0/1928745407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 msgr2=0x55777e4c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbead5fe640 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbead5fe640 1 -- 172.21.15.5:0/1928745407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 msgr2=0x55777e4c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbead5fe640 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbead5fe640 1 -- 172.21.15.5:0/1928745407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55777e4c8a40 con 0x55777e4b7b60 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbea7fff640 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbead5fe640 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 0x55777e4cb4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe90010900 tx=0x7fbe9000e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99852d200698d444 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbe87fff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:36:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbe87fff640 1 -- 172.21.15.5:0/1928745407 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9002d040 con 0x55777e4b7b60 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbeb143a740 1 -- 172.21.15.5:0/1928745407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55777e424d00 con 0x55777e4b7b60 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbe87fff640 1 -- 172.21.15.5:0/1928745407 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7fbe90010ba0 con 0x55777e4b7b60 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbe87fff640 1 -- 172.21.15.5:0/1928745407 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fbe90010da0 con 0x55777e4b7b60 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbeb143a740 10 client.29138 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.897+0000 7fbeb143a740 1 -- 172.21.15.5:0/1928745407 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55777e53e7c0 con 0x55777e4b7b60 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.898+0000 7fbe87fff640 1 -- 172.21.15.5:0/1928745407 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55777e424d00 con 0x55777e4b7b60 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.898+0000 7fbe87fff640 1 -- 172.21.15.5:0/1928745407 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7fbe90031070 con 0x55777e4b7b60 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.898+0000 7fbeb143a740 10 client.29138 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.898+0000 7fbeb143a740 10 client.29138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.898+0000 7fbeb143a740 20 client.29138 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.898+0000 7fbeb143a740 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777e53ee40 0x55777e541260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.898+0000 7fbeb143a740 4 client.29138 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.898+0000 7fbeb143a740 1 -- 172.21.15.5:0/1928745407 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55777e3a1110 con 0x55777e53ee40 2024-09-11T22:36:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.898+0000 7fbea7fff640 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777e53ee40 0x55777e541260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.899+0000 7fbe87fff640 10 client.29138 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.899+0000 7fbea7fff640 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777e53ee40 0x55777e541260 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8a609c5b7155cc server_cookie=e7a4034ee31e2194 in_seq=0 out_seq=0 2024-09-11T22:36:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbe87fff640 1 -- 172.21.15.5:0/1928745407 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55777e3a1110 con 0x55777e53ee40 2024-09-11T22:36:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbe87fff640 10 client.29138 handle_command_reply: tid=0 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbeb143a740 1 client.29138 shutdown 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbeb143a740 1 -- 172.21.15.5:0/1928745407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 msgr2=0x55777e4cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbeb143a740 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 0x55777e4cb4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbe90010900 tx=0x7fbe9000e350 comp rx=0 tx=0).stop 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbeb143a740 1 -- 172.21.15.5:0/1928745407 shutdown_connections 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbeb143a740 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55777e53ee40 0x55777e541260 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbeb143a740 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55777e4c2d40 0x55777e4c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbeb143a740 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55777e4b8480 0x55777e4c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbeb143a740 1 --2- 172.21.15.5:0/1928745407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55777e4b7b60 0x55777e4cb4c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbe87fff640 0 client.29138 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.900+0000 7fbeb143a740 1 -- 172.21.15.5:0/1928745407 >> 172.21.15.5:0/1928745407 conn(0x55777e4b40f0 msgr2=0x55777e4b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.901+0000 7fbeb143a740 1 -- 172.21.15.5:0/1928745407 shutdown_connections 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.901+0000 7fbeb143a740 1 -- 172.21.15.5:0/1928745407 wait complete. 2024-09-11T22:36:55.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.901+0000 7fbeb143a740 20 client.29138 trim_cache size 0 max 16384 2024-09-11T22:36:55.905 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:36:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:36:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "c113d33b-5b4d-480c-bbe9-a4c6b2d782fa", 2024-09-11T22:36:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:36:55.905 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:36:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 -- 172.21.15.5:0/3895427624 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbe940a2cf0 msgr2=0x7fbe940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbe940a2cf0 0x7fbe940a51b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbea00ee1a0 tx=0x7fbea004f000 comp rx=0 tx=0).stop 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 -- 172.21.15.5:0/3895427624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8101de0 msgr2=0x7fbea80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8101de0 0x7fbea80744f0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fbea002ade0 tx=0x7fbea00319f0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 -- 172.21.15.5:0/3895427624 shutdown_connections 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbea8106f10 0x7fbea8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbe940a2cf0 0x7fbe940a51b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbea8102790 0x7fbea8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 --2- 172.21.15.5:0/3895427624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea8101de0 0x7fbea80744f0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 -- 172.21.15.5:0/3895427624 >> 172.21.15.5:0/3895427624 conn(0x7fbea80fd8c0 msgr2=0x7fbea80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 -- 172.21.15.5:0/3895427624 shutdown_connections 2024-09-11T22:36:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:55.902+0000 7fbeaf061640 1 -- 172.21.15.5:0/3895427624 wait complete. 2024-09-11T22:36:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:36:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:56.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: Scrub error on inode 0x10000007c3c (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client7/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: Scrub error on inode 0x10000007c7d (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:56.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: Scrub error on inode 0x10000007c50 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client6/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:56 smithi005 ceph-mon[30943]: Scrub error on inode 0x10000007c4d (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:36:56.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:56.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:56.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:56.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: Scrub error on inode 0x10000007c3c (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client7/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: Scrub error on inode 0x10000007c7d (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:56.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: Scrub error on inode 0x10000007c50 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client6/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:56 smithi159 ceph-mon[35534]: Scrub error on inode 0x10000007c4d (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.330 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:36:55 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:36:55.932+0000 7f54cdab7640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000007c3c [...18,head] /volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client7/~dmtmp/WORD/ auth{1=1,2=1,3=1,4=1} v1809 ap=3 RANDEPHEMERALPIN f(v32 m2024-09-11T22:36:28.072857+0000 3=3+0) n(v4 rc2024-09-11T22:36:28.072857+0000 b780288 4=3+1) old_inodes=1 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:2010 nref=3)) caps={24841=pAsLsXsFs/pAsLsXsFsx@87} mcw={4=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x556c7f9ee000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000007c3c:[<0x10000007bf8/WORD v8>,<0x10000007bf4/~dmtmp v26>,<0x10000000007/client7 v1354>,<0x10000000006/clients v1098>,<0x10000000005/tmp v1076>,<0x100000001fb/client.0 v1082>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v1088>,<0x10000000001/sv_1 v1091>,<0x10000000000/qa v1057>,<0x1/volumes v1043>]//[]","memoryvalue":"(2)0x10000007c3c:[<0x10000007bf8/WORD v1809>,<0x10000007bf4/~dmtmp v172>,<0x10000000007/client7 v2111>,<0x10000000006/clients v1195>,<0x10000000005/tmp v1171>,<0x100000001fb/client.0 v1177>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v1183>,<0x10000000001/sv_1 v1186>,<0x10000000000/qa v1151>,<0x1/volumes v1132>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T22:35:53.451587+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-09-11T22:36:28.072857+0000 b780288 4=3+1)","memory_value.dirstat":"f(v32 m2024-09-11T22:36:28.072857+0000 3=3+0)","memory_value.rstat":"n(v4 rc2024-09-11T22:36:28.072857+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-11T22:36:56.331 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:36:55 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:36:55.936+0000 7f54cdab7640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000007c7d [...18,head] /volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client3/~dmtmp/COREL/ auth{1=1,2=1,3=1,4=1} v1845 ap=3 RANDEPHEMERALPIN f(v32 m2024-09-11T22:36:27.378872+0000 11=11+0) n(v4 rc2024-09-11T22:36:27.378872+0000 b2861056 12=11+1) old_inodes=1 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:2026 nref=3)) caps={24841=pAsLsXsFs/pAsLsXsFsx@116} mcw={2=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x556c819f6c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000007c7d:[<0x10000007be4/COREL v50>,<0x10000007be3/~dmtmp v26>,<0x10000000007/client3 v1290>,<0x10000000006/clients v1098>,<0x10000000005/tmp v1076>,<0x100000001fb/client.0 v1082>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v1088>,<0x10000000001/sv_1 v1091>,<0x10000000000/qa v1057>,<0x1/volumes v1043>]//[]","memoryvalue":"(2)0x10000007c7d:[<0x10000007be4/COREL v1845>,<0x10000007be3/~dmtmp v170>,<0x10000000007/client3 v2113>,<0x10000000006/clients v1195>,<0x10000000005/tmp v1171>,<0x100000001fb/client.0 v1177>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v1183>,<0x10000000001/sv_1 v1186>,<0x10000000000/qa v1151>,<0x1/volumes v1132>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T22:36:24.926924+0000 15=15+0)","ondisk_value.rstat":"n(v0 rc2024-09-11T22:36:27.378872+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v32 m2024-09-11T22:36:27.378872+0000 11=11+0)","memory_value.rstat":"n(v4 rc2024-09-11T22:36:27.378872+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-11T22:36:56.331 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:36:55 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:36:55.945+0000 7f54cdab7640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000007c50 [...18,head] /volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client6/~dmtmp/EXCEL/ auth{1=1,2=1,3=1,4=1} v1726 ap=3 RANDEPHEMERALPIN f(v46 m2024-09-11T22:36:28.094857+0000 2=2+0) n(v4 rc2024-09-11T22:36:28.094857+0000 b520192 3=2+1) old_inodes=1 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:2045 nref=3)) caps={24841=pAsLsXsFs/pAsLsXsFsx@110} mcw={4=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x556c7ff62100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000007c50:[<0x10000007bf2/EXCEL v14>,<0x10000007bf1/~dmtmp v26>,<0x10000000007/client6 v1344>,<0x10000000006/clients v1098>,<0x10000000005/tmp v1076>,<0x100000001fb/client.0 v1082>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v1088>,<0x10000000001/sv_1 v1091>,<0x10000000000/qa v1057>,<0x1/volumes v1043>]//[]","memoryvalue":"(2)0x10000007c50:[<0x10000007bf2/EXCEL v1726>,<0x10000007bf1/~dmtmp v198>,<0x10000000007/client6 v2107>,<0x10000000006/clients v1195>,<0x10000000005/tmp v1171>,<0x100000001fb/client.0 v1177>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v1183>,<0x10000000001/sv_1 v1186>,<0x10000000000/qa v1151>,<0x1/volumes v1132>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T22:35:55.533543+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-09-11T22:36:28.094857+0000 b520192 3=2+1)","memory_value.dirstat":"f(v46 m2024-09-11T22:36:28.094857+0000 2=2+0)","memory_value.rstat":"n(v4 rc2024-09-11T22:36:28.094857+0000 b520192 3=2+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-11T22:36:56.331 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:36:55 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:36:55.966+0000 7f54cdab7640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000007c4d [...18,head] /volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client1/~dmtmp/WORD/ auth{1=1,2=1,3=1,4=1} v1581 ap=3 RANDEPHEMERALPIN f(v31 m2024-09-11T22:35:53.769580+0000 5=5+0) n(v4 rc2024-09-11T22:35:53.769580+0000 b814326 6=5+1) old_inodes=1 (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:2073 nref=3)) caps={24841=pAsLsXsFs/pAsLsXsFsx@87} mcw={2=AsLsXsFs} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x556c7f9efb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000007c4d:[<0x10000007c03/WORD v8>,<0x10000007bfc/~dmtmp v26>,<0x10000000007/client1 v1364>,<0x10000000006/clients v1098>,<0x10000000005/tmp v1076>,<0x100000001fb/client.0 v1082>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v1088>,<0x10000000001/sv_1 v1091>,<0x10000000000/qa v1057>,<0x1/volumes v1043>]//[]","memoryvalue":"(2)0x10000007c4d:[<0x10000007c03/WORD v1581>,<0x10000007bfc/~dmtmp v245>,<0x10000000007/client1 v2101>,<0x10000000006/clients v1195>,<0x10000000005/tmp v1171>,<0x100000001fb/client.0 v1177>,<0x100000001fa/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3 v1183>,<0x10000000001/sv_1 v1186>,<0x10000000000/qa v1151>,<0x1/volumes v1132>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-11T22:35:53.719581+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-09-11T22:35:53.769580+0000 b814326 6=5+1)","memory_value.dirstat":"f(v31 m2024-09-11T22:35:53.769580+0000 5=5+0)","memory_value.rstat":"n(v4 rc2024-09-11T22:35:53.769580+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-11T22:36:56.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:36:56.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:56.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:56.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:56.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: Scrub error on inode 0x10000007c3c (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client7/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: Scrub error on inode 0x10000007c7d (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:56.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:56.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: Scrub error on inode 0x10000007c50 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client6/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.361 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:56 smithi172 ceph-mon[35461]: Scrub error on inode 0x10000007c4d (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-09-11T22:36:56.636 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 8.77 MB/sec warmup 74 sec latency 28441.758 ms 2024-09-11T22:36:56.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:36:57.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:57 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:57.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:57 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:57 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:57.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:57 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:57 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:57.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:57 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:57 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:36:57.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:57 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:57 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:57 smithi005 ceph-mon[30943]: pgmap v469: 97 pgs: 97 active+clean; 329 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 79 KiB/s wr, 33 op/s 2024-09-11T22:36:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.210+0000 7f85b1b18640 1 -- 172.21.15.5:0/2730459575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85ac102790 msgr2=0x7f85ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.210+0000 7f85b1b18640 1 --2- 172.21.15.5:0/2730459575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85ac102790 0x7f85ac1067e0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009900 tx=0x7f85a002a410 comp rx=0 tx=0).stop 2024-09-11T22:36:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.210+0000 7f85b1b18640 1 -- 172.21.15.5:0/2730459575 shutdown_connections 2024-09-11T22:36:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.210+0000 7f85b1b18640 1 --2- 172.21.15.5:0/2730459575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85ac106f10 0x7f85ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.210+0000 7f85b1b18640 1 --2- 172.21.15.5:0/2730459575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85ac102790 0x7f85ac1067e0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.210+0000 7f85b1b18640 1 --2- 172.21.15.5:0/2730459575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85ac101de0 0x7f85ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.210+0000 7f85b1b18640 1 -- 172.21.15.5:0/2730459575 >> 172.21.15.5:0/2730459575 conn(0x7f85ac0fd8c0 msgr2=0x7f85ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.210+0000 7f85b1b18640 1 -- 172.21.15.5:0/2730459575 shutdown_connections 2024-09-11T22:36:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.210+0000 7f85b1b18640 1 -- 172.21.15.5:0/2730459575 wait complete. 2024-09-11T22:36:57.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.211+0000 7f85b1b18640 1 Processor -- start 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.211+0000 7f85b1b18640 1 -- start start 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.211+0000 7f85b1b18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85ac101de0 0x7f85ac1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.211+0000 7f85b1b18640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85ac102790 0x7f85ac198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.211+0000 7f85b1b18640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85ac106f10 0x7f85ac19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.211+0000 7f85b1b18640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac111f70 con 0x7f85ac101de0 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.211+0000 7f85b1b18640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac111df0 con 0x7f85ac102790 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.211+0000 7f85b1b18640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85ac1120f0 con 0x7f85ac106f10 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.211+0000 7f85ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85ac101de0 0x7f85ac1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85ac101de0 0x7f85ac1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55458/0 (socket says 172.21.15.5:55458) 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85ab7fe640 1 -- 172.21.15.5:0/1233318465 learned_addr learned my addr 172.21.15.5:0/1233318465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85abfff640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85ac106f10 0x7f85ac19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85aaffd640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85ac102790 0x7f85ac198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:57.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85ab7fe640 1 -- 172.21.15.5:0/1233318465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85ac106f10 msgr2=0x7f85ac19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:57.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85ab7fe640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85ac106f10 0x7f85ac19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:57.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85ab7fe640 1 -- 172.21.15.5:0/1233318465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85ac102790 msgr2=0x7f85ac198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:57.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85ab7fe640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85ac102790 0x7f85ac198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:57.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85ab7fe640 1 -- 172.21.15.5:0/1233318465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ac19d480 con 0x7f85ac101de0 2024-09-11T22:36:57.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85ab7fe640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85ac101de0 0x7f85ac1984d0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f859c00d950 tx=0x7f859c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:57.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85a8ff9640 1 -- 172.21.15.5:0/1233318465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c017070 con 0x7f85ac101de0 2024-09-11T22:36:57.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85a8ff9640 1 -- 172.21.15.5:0/1233318465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c009c30 con 0x7f85ac101de0 2024-09-11T22:36:57.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85b1b18640 1 -- 172.21.15.5:0/1233318465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ac19d770 con 0x7f85ac101de0 2024-09-11T22:36:57.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85a8ff9640 1 -- 172.21.15.5:0/1233318465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c00f6b0 con 0x7f85ac101de0 2024-09-11T22:36:57.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.212+0000 7f85b1b18640 1 -- 172.21.15.5:0/1233318465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ac1a5050 con 0x7f85ac101de0 2024-09-11T22:36:57.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.213+0000 7f85b1b18640 1 -- 172.21.15.5:0/1233318465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ac1021c0 con 0x7f85ac101de0 2024-09-11T22:36:57.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.214+0000 7f85a8ff9640 1 -- 172.21.15.5:0/1233318465 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f859c0109d0 con 0x7f85ac101de0 2024-09-11T22:36:57.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.215+0000 7f85a8ff9640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85940a2d70 0x7f85940a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:57.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.215+0000 7f85a8ff9640 1 -- 172.21.15.5:0/1233318465 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f859c0cf160 con 0x7f85ac101de0 2024-09-11T22:36:57.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.215+0000 7f85aaffd640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85940a2d70 0x7f85940a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:57.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.216+0000 7f85aaffd640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85940a2d70 0x7f85940a5230 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0fdd70 tx=0x7f85a0005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:57.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.219+0000 7f85a8ff9640 1 -- 172.21.15.5:0/1233318465 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f859c009de0 con 0x7f85ac101de0 2024-09-11T22:36:57.275 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:57.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:57 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:57.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:57 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:57 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:57.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:57 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:57 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:57.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:57 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:57 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:36:57.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:57 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:57 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.330 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:57 smithi159 ceph-mon[35534]: pgmap v469: 97 pgs: 97 active+clean; 329 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 79 KiB/s wr, 33 op/s 2024-09-11T22:36:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:57 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:57 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:57 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:57 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:57 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:57 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:57 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:36:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:57 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:57 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:36:57.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:57 smithi172 ceph-mon[35461]: pgmap v469: 97 pgs: 97 active+clean; 329 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 79 KiB/s wr, 33 op/s 2024-09-11T22:36:57.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.437+0000 7f85b1b18640 1 -- 172.21.15.5:0/1233318465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85ac10a150 con 0x7f85ac101de0 2024-09-11T22:36:57.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.437+0000 7f85a8ff9640 1 -- 172.21.15.5:0/1233318465 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11975 (secure 0 0 0) 0x7f859c09bd10 con 0x7f85ac101de0 2024-09-11T22:36:57.438 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:36:57.438 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"btime":"2024-09-11T22:35:49:548013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:35:48.842139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:36:57.438 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 50 2024-09-11T22:36:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.439+0000 7f85b1b18640 1 -- 172.21.15.5:0/1233318465 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85940a2d70 msgr2=0x7f85940a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.439+0000 7f85b1b18640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85940a2d70 0x7f85940a5230 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0fdd70 tx=0x7f85a0005a20 comp rx=0 tx=0).stop 2024-09-11T22:36:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.439+0000 7f85b1b18640 1 -- 172.21.15.5:0/1233318465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85ac101de0 msgr2=0x7f85ac1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.439+0000 7f85b1b18640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85ac101de0 0x7f85ac1984d0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f859c00d950 tx=0x7f859c00de10 comp rx=0 tx=0).stop 2024-09-11T22:36:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.440+0000 7f85b1b18640 1 -- 172.21.15.5:0/1233318465 shutdown_connections 2024-09-11T22:36:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.440+0000 7f85b1b18640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85ac106f10 0x7f85ac19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.440+0000 7f85b1b18640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85940a2d70 0x7f85940a5230 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.440+0000 7f85b1b18640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85ac102790 0x7f85ac198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.440+0000 7f85b1b18640 1 --2- 172.21.15.5:0/1233318465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85ac101de0 0x7f85ac1984d0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.440+0000 7f85b1b18640 1 -- 172.21.15.5:0/1233318465 >> 172.21.15.5:0/1233318465 conn(0x7f85ac0fd8c0 msgr2=0x7f85ac0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.440+0000 7f85b1b18640 1 -- 172.21.15.5:0/1233318465 shutdown_connections 2024-09-11T22:36:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:57.440+0000 7f85b1b18640 1 -- 172.21.15.5:0/1233318465 wait complete. 2024-09-11T22:36:57.636 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 8.65 MB/sec warmup 75 sec latency 29441.838 ms 2024-09-11T22:36:58.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:58 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1233318465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:58.187 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 25352b1d --release --await' 2024-09-11T22:36:58.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 25352b1d --release --await 2024-09-11T22:36:58.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:58 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1233318465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:58.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:58 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1233318465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:36:58.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.393+0000 7f01a5d46640 1 -- 172.21.15.5:0/1487961622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01a0100e20 msgr2=0x7f01a0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:58.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.393+0000 7f01a5d46640 1 --2- 172.21.15.5:0/1487961622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01a0100e20 0x7f01a0101200 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f0190009900 tx=0x7f019002a480 comp rx=0 tx=0).stop 2024-09-11T22:36:58.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.393+0000 7f01a5d46640 1 -- 172.21.15.5:0/1487961622 shutdown_connections 2024-09-11T22:36:58.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.393+0000 7f01a5d46640 1 --2- 172.21.15.5:0/1487961622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a0102ef0 0x7f01a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.393+0000 7f01a5d46640 1 --2- 172.21.15.5:0/1487961622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01a0102570 0x7f01a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.393+0000 7f01a5d46640 1 --2- 172.21.15.5:0/1487961622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01a0100e20 0x7f01a0101200 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.393+0000 7f01a5d46640 1 -- 172.21.15.5:0/1487961622 >> 172.21.15.5:0/1487961622 conn(0x7f01a00fc7d0 msgr2=0x7f01a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.393+0000 7f01a5d46640 1 -- 172.21.15.5:0/1487961622 shutdown_connections 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.393+0000 7f01a5d46640 1 -- 172.21.15.5:0/1487961622 wait complete. 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f01a5d46640 1 Processor -- start 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f01a5d46640 1 -- start start 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f01a5d46640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a0100e20 0x7f01a010e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f01a5d46640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01a0102570 0x7f01a010e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f01a5d46640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01a0102ef0 0x7f01a010edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f01a5d46640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0112090 con 0x7f01a0102ef0 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f01a5d46640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0111f10 con 0x7f01a0102570 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f01a5d46640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0112210 con 0x7f01a0100e20 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f019ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01a0102ef0 0x7f01a010edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f019ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01a0102ef0 0x7f01a010edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55480/0 (socket says 172.21.15.5:55480) 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f019ffff640 1 -- 172.21.15.5:0/583089726 learned_addr learned my addr 172.21.15.5:0/583089726 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f019effd640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01a0102570 0x7f01a010e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f019f7fe640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a0100e20 0x7f01a010e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f019ffff640 1 -- 172.21.15.5:0/583089726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a0100e20 msgr2=0x7f01a010e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f019ffff640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a0100e20 0x7f01a010e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f019ffff640 1 -- 172.21.15.5:0/583089726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01a0102570 msgr2=0x7f01a010e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f019ffff640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01a0102570 0x7f01a010e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.394+0000 7f019ffff640 1 -- 172.21.15.5:0/583089726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a01a6a30 con 0x7f01a0102ef0 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.395+0000 7f019ffff640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01a0102ef0 0x7f01a010edd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f019800eaa0 tx=0x7f019800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.395+0000 7f019cff9640 1 -- 172.21.15.5:0/583089726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019800cdb0 con 0x7f01a0102ef0 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.395+0000 7f019cff9640 1 -- 172.21.15.5:0/583089726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0198005020 con 0x7f01a0102ef0 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.395+0000 7f01a5d46640 1 -- 172.21.15.5:0/583089726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a01a6d20 con 0x7f01a0102ef0 2024-09-11T22:36:58.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.395+0000 7f019cff9640 1 -- 172.21.15.5:0/583089726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01980116b0 con 0x7f01a0102ef0 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.395+0000 7f01a5d46640 1 -- 172.21.15.5:0/583089726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a01a7260 con 0x7f01a0102ef0 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.397+0000 7f01a811f740 1 Processor -- start 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.397+0000 7f019cff9640 1 -- 172.21.15.5:0/583089726 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f0198005560 con 0x7f01a0102ef0 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.397+0000 7f01a811f740 1 -- start start 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f01a811f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f261880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f01a811f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f26c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f01a811f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f26c680 0x55e90f26ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f01a811f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e90f1c72e0 con 0x55e90f2614a0 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f01a811f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e90f16ece0 con 0x55e90f26c680 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f01a811f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e90f1c32a0 con 0x55e90f261dc0 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f261880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f261880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55490/0 (socket says 172.21.15.5:55490) 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019effd640 1 -- 172.21.15.5:0/948013244 learned_addr learned my addr 172.21.15.5:0/948013244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:58.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019f7fe640 1 --2- 172.21.15.5:0/948013244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f26c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019effd640 1 --2- 172.21.15.5:0/948013244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f26c680 0x55e90f26ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019effd640 1 -- 172.21.15.5:0/948013244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 msgr2=0x55e90f26c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019cff9640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01880a2cf0 0x7f01880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019effd640 1 --2- 172.21.15.5:0/948013244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f26c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019effd640 1 -- 172.21.15.5:0/948013244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f26c680 msgr2=0x55e90f26ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019effd640 1 --2- 172.21.15.5:0/948013244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f26c680 0x55e90f26ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019effd640 1 -- 172.21.15.5:0/948013244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e90f153440 con 0x55e90f2614a0 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019f7fe640 1 --2- 172.21.15.5:0/948013244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f26c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019ffff640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01880a2cf0 0x7f01880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.398+0000 7f019effd640 1 --2- 172.21.15.5:0/948013244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f261880 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f01940058a0 tx=0x7f019400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91085deeb47e92c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f019cff9640 1 -- 172.21.15.5:0/583089726 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f01980cf5c0 con 0x7f01a0102ef0 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f0185ffb640 1 -- 172.21.15.5:0/948013244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194004030 con 0x55e90f2614a0 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f0185ffb640 1 -- 172.21.15.5:0/948013244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01940041d0 con 0x55e90f2614a0 2024-09-11T22:36:58.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f0185ffb640 1 -- 172.21.15.5:0/948013244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194010460 con 0x55e90f2614a0 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f01a811f740 1 -- 172.21.15.5:0/948013244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 msgr2=0x55e90f261880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f019ffff640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01880a2cf0 0x7f01880a51b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f019809bab0 tx=0x7f01980977c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f01a811f740 1 --2- 172.21.15.5:0/948013244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f261880 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f01940058a0 tx=0x7f019400b840 comp rx=0 tx=0).stop 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f01a811f740 1 -- 172.21.15.5:0/948013244 shutdown_connections 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f01a811f740 1 --2- 172.21.15.5:0/948013244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f26c680 0x55e90f26ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f01a811f740 1 --2- 172.21.15.5:0/948013244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f26c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f01a811f740 1 --2- 172.21.15.5:0/948013244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f261880 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.399+0000 7f01a811f740 1 -- 172.21.15.5:0/948013244 >> 172.21.15.5:0/948013244 conn(0x55e90f25da30 msgr2=0x55e90f25de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.400+0000 7f01a811f740 1 -- 172.21.15.5:0/948013244 shutdown_connections 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.400+0000 7f01a811f740 1 -- 172.21.15.5:0/948013244 wait complete. 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.400+0000 7f01a811f740 1 -- start start 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.400+0000 7f01a811f740 1 Processor -- start 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.400+0000 7f01a811f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f2614a0 0x55e90f26bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.400+0000 7f01a811f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f2691e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.400+0000 7f01a811f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f269720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.400+0000 7f01a811f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e90f26f250 con 0x55e90f26c680 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.400+0000 7f01a811f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e90f182050 con 0x55e90f2614a0 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.400+0000 7f01a811f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e90f26f3d0 con 0x55e90f261dc0 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f01a811f740 10 client.0 _ll_register_callbacks cb 0x55e90f1c2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f269720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f269720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55502/0 (socket says 172.21.15.5:55502) 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019effd640 1 -- 172.21.15.5:0/2360737666 learned_addr learned my addr 172.21.15.5:0/2360737666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019f7fe640 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f2691e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019effd640 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f2614a0 0x55e90f26bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019effd640 1 -- 172.21.15.5:0/2360737666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 msgr2=0x55e90f2691e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019effd640 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f2691e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019effd640 1 -- 172.21.15.5:0/2360737666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f2614a0 msgr2=0x55e90f26bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:58.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019effd640 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f2614a0 0x55e90f26bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019effd640 1 -- 172.21.15.5:0/2360737666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90ef383c0 con 0x55e90f26c680 2024-09-11T22:36:58.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f019effd640 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f269720 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f0194003c50 tx=0x7f019400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ab2ede047a95ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:58.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f017effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:58.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f017effd640 1 -- 172.21.15.5:0/2360737666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194028040 con 0x55e90f26c680 2024-09-11T22:36:58.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f017effd640 1 -- 172.21.15.5:0/2360737666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f0194003800 con 0x55e90f26c680 2024-09-11T22:36:58.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.401+0000 7f01a811f740 1 -- 172.21.15.5:0/2360737666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01940041d0 con 0x55e90f26c680 2024-09-11T22:36:58.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f017effd640 1 -- 172.21.15.5:0/2360737666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0194012210 con 0x55e90f26c680 2024-09-11T22:36:58.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f01a811f740 10 client.20073 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:58.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f01a811f740 1 -- 172.21.15.5:0/2360737666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e90f27aba0 con 0x55e90f26c680 2024-09-11T22:36:58.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f017effd640 1 -- 172.21.15.5:0/2360737666 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01940041d0 con 0x55e90f26c680 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f017effd640 1 -- 172.21.15.5:0/2360737666 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f0194010460 con 0x55e90f26c680 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f01a811f740 10 client.20073 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f01a811f740 10 client.20073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f01a811f740 20 client.20073 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f01a811f740 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e90f27bef0 0x55e90f27e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f01a811f740 4 client.20073 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f01a811f740 1 -- 172.21.15.5:0/2360737666 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e90f1d4850 con 0x55e90f27bef0 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.402+0000 7f019f7fe640 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e90f27bef0 0x55e90f27e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.403+0000 7f017effd640 10 client.20073 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.403+0000 7f019f7fe640 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e90f27bef0 0x55e90f27e2e0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b13276a68b504294 server_cookie=ccb28064834fed02 in_seq=0 out_seq=0 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f017effd640 1 -- 172.21.15.5:0/2360737666 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e90f1d4850 con 0x55e90f27bef0 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f017effd640 10 client.20073 handle_command_reply: tid=0 2024-09-11T22:36:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 client.20073 shutdown 2024-09-11T22:36:58.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 -- 172.21.15.5:0/2360737666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 msgr2=0x55e90f269720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:58.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f269720 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f0194003c50 tx=0x7f019400b5a0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 -- 172.21.15.5:0/2360737666 shutdown_connections 2024-09-11T22:36:58.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e90f27bef0 0x55e90f27e2e0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f269720 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f2691e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 --2- 172.21.15.5:0/2360737666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f2614a0 0x55e90f26bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 -- 172.21.15.5:0/2360737666 >> 172.21.15.5:0/2360737666 conn(0x55e90f25da30 msgr2=0x55e90f263760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:58.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f017effd640 0 client.20073 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:58.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 -- 172.21.15.5:0/2360737666 shutdown_connections 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 1 -- 172.21.15.5:0/2360737666 wait complete. 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.404+0000 7f01a811f740 20 client.20073 trim_cache size 0 max 16384 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.421+0000 7f01a811f740 1 Processor -- start 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.421+0000 7f01a811f740 1 -- start start 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f01a811f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f2784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f01a811f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f261dc0 0x55e90f2693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f01a811f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f26c680 0x55e90f2681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f01a811f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01afd30 con 0x55e90f2614a0 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f01a811f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b1960 con 0x55e90f261dc0 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f01a811f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b0400 con 0x55e90f26c680 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f2784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f2784e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55514/0 (socket says 172.21.15.5:55514) 2024-09-11T22:36:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019effd640 1 -- 172.21.15.5:0/1007308935 learned_addr learned my addr 172.21.15.5:0/1007308935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019f7fe640 1 --2- 172.21.15.5:0/1007308935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f261dc0 0x55e90f2693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019effd640 1 -- 172.21.15.5:0/1007308935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f26c680 msgr2=0x55e90f2681c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019effd640 1 --2- 172.21.15.5:0/1007308935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f26c680 0x55e90f2681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019effd640 1 -- 172.21.15.5:0/1007308935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f261dc0 msgr2=0x55e90f2693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019effd640 1 --2- 172.21.15.5:0/1007308935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f261dc0 0x55e90f2693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019effd640 1 -- 172.21.15.5:0/1007308935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e90f27e8c0 con 0x55e90f2614a0 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019f7fe640 1 --2- 172.21.15.5:0/1007308935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f261dc0 0x55e90f2693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.422+0000 7f019effd640 1 --2- 172.21.15.5:0/1007308935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f2784e0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f019400b7f0 tx=0x7f0194005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a509a683a7c003 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.423+0000 7f01857fa640 1 -- 172.21.15.5:0/1007308935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194023070 con 0x55e90f2614a0 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.423+0000 7f01857fa640 1 -- 172.21.15.5:0/1007308935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0194005850 con 0x55e90f2614a0 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.423+0000 7f01857fa640 1 -- 172.21.15.5:0/1007308935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194003990 con 0x55e90f2614a0 2024-09-11T22:36:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.423+0000 7f01a811f740 1 -- 172.21.15.5:0/1007308935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 msgr2=0x55e90f2784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.423+0000 7f01a811f740 1 --2- 172.21.15.5:0/1007308935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f2784e0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f019400b7f0 tx=0x7f0194005c50 comp rx=0 tx=0).stop 2024-09-11T22:36:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.423+0000 7f01a811f740 1 -- 172.21.15.5:0/1007308935 shutdown_connections 2024-09-11T22:36:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.423+0000 7f01a811f740 1 --2- 172.21.15.5:0/1007308935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f26c680 0x55e90f2681c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.423+0000 7f01a811f740 1 --2- 172.21.15.5:0/1007308935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f261dc0 0x55e90f2693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.423+0000 7f01a811f740 1 --2- 172.21.15.5:0/1007308935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f2614a0 0x55e90f2784e0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.423+0000 7f01a811f740 1 -- 172.21.15.5:0/1007308935 >> 172.21.15.5:0/1007308935 conn(0x55e90f25da30 msgr2=0x55e90f267ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.424+0000 7f01a811f740 1 -- 172.21.15.5:0/1007308935 shutdown_connections 2024-09-11T22:36:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.424+0000 7f01a811f740 1 -- 172.21.15.5:0/1007308935 wait complete. 2024-09-11T22:36:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.424+0000 7f01a811f740 1 -- start start 2024-09-11T22:36:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.424+0000 7f01a811f740 1 Processor -- start 2024-09-11T22:36:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.424+0000 7f01a811f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f2614a0 0x55e90f274d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.424+0000 7f01a811f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f270080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.424+0000 7f01a811f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f2705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.424+0000 7f01a811f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b1290 con 0x55e90f26c680 2024-09-11T22:36:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f01a811f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b0b00 con 0x55e90f2614a0 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f01a811f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01add30 con 0x55e90f261dc0 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f01a811f740 10 client.0 _ll_register_callbacks cb 0x7f0194012480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f2705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f2705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55520/0 (socket says 172.21.15.5:55520) 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019effd640 1 -- 172.21.15.5:0/2561673036 learned_addr learned my addr 172.21.15.5:0/2561673036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019f7fe640 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f270080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019effd640 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f2614a0 0x55e90f274d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019effd640 1 -- 172.21.15.5:0/2561673036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 msgr2=0x55e90f270080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019effd640 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f270080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019effd640 1 -- 172.21.15.5:0/2561673036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f2614a0 msgr2=0x55e90f274d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019effd640 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f2614a0 0x55e90f274d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019effd640 1 -- 172.21.15.5:0/2561673036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e90f25eae0 con 0x55e90f26c680 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f019effd640 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f2705c0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f01940108b0 tx=0x7f0194020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d74df455240de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:36:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f017effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f017effd640 1 -- 172.21.15.5:0/2561673036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194023030 con 0x55e90f26c680 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f017effd640 1 -- 172.21.15.5:0/2561673036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f0194010460 con 0x55e90f26c680 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.425+0000 7f01a811f740 1 -- 172.21.15.5:0/2561673036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0194005850 con 0x55e90f26c680 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f017effd640 1 -- 172.21.15.5:0/2561673036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0194012070 con 0x55e90f26c680 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f01a811f740 10 client.20079 fetch_fsmap learned FSMap version 50 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f01a811f740 1 -- 172.21.15.5:0/2561673036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e90f25d860 con 0x55e90f26c680 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f017effd640 1 -- 172.21.15.5:0/2561673036 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0194005850 con 0x55e90f26c680 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f017effd640 1 -- 172.21.15.5:0/2561673036 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f0194043960 con 0x55e90f26c680 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f01a811f740 10 client.20079 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f01a811f740 10 client.20079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f01a811f740 20 client.20079 populate_metadata read hostname 'smithi005' 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f01a811f740 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e90f2ef5b0 0x55e90f2f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f01a811f740 4 client.20079 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f01a811f740 1 -- 172.21.15.5:0/2561673036 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e90f269ab0 con 0x55e90f2ef5b0 2024-09-11T22:36:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.426+0000 7f019f7fe640 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e90f2ef5b0 0x55e90f2f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f019f7fe640 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e90f2ef5b0 0x55e90f2f19d0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd456d5e3d0ccebf server_cookie=e7abd754e6e07d58 in_seq=0 out_seq=0 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f017effd640 10 client.20079 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f017effd640 1 -- 172.21.15.5:0/2561673036 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e90f269ab0 con 0x55e90f2ef5b0 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f017effd640 10 client.20079 handle_command_reply: tid=0 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f01a811f740 1 client.20079 shutdown 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f01a811f740 1 -- 172.21.15.5:0/2561673036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 msgr2=0x55e90f2705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f01a811f740 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f2705c0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f01940108b0 tx=0x7f0194020040 comp rx=0 tx=0).stop 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f01a811f740 1 -- 172.21.15.5:0/2561673036 shutdown_connections 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f01a811f740 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e90f2ef5b0 0x55e90f2f19d0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f01a811f740 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e90f26c680 0x55e90f2705c0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f017effd640 0 client.20079 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:36:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.427+0000 7f01a811f740 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e90f261dc0 0x55e90f270080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.428+0000 7f01a811f740 1 --2- 172.21.15.5:0/2561673036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e90f2614a0 0x55e90f274d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.428+0000 7f01a811f740 1 -- 172.21.15.5:0/2561673036 >> 172.21.15.5:0/2561673036 conn(0x55e90f25da30 msgr2=0x55e90f26aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.428+0000 7f01a811f740 1 -- 172.21.15.5:0/2561673036 shutdown_connections 2024-09-11T22:36:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.428+0000 7f01a811f740 1 -- 172.21.15.5:0/2561673036 wait complete. 2024-09-11T22:36:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.428+0000 7f01a811f740 20 client.20079 trim_cache size 0 max 16384 2024-09-11T22:36:58.444 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:36:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:36:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:36:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "c113d33b-5b4d-480c-bbe9-a4c6b2d782fa": { 2024-09-11T22:36:58.444 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:36:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "c113d33b-5b4d-480c-bbe9-a4c6b2d782fa", 2024-09-11T22:36:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:36:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:58.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:36:58.445 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:36:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.428+0000 7f01a5d46640 1 -- 172.21.15.5:0/583089726 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01880a2cf0 msgr2=0x7f01880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.428+0000 7f01a5d46640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01880a2cf0 0x7f01880a51b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f019809bab0 tx=0x7f01980977c0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.428+0000 7f01a5d46640 1 -- 172.21.15.5:0/583089726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01a0102ef0 msgr2=0x7f01a010edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:36:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.428+0000 7f01a5d46640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01a0102ef0 0x7f01a010edd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f019800eaa0 tx=0x7f019800ef60 comp rx=0 tx=0).stop 2024-09-11T22:36:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.429+0000 7f01a5d46640 1 -- 172.21.15.5:0/583089726 shutdown_connections 2024-09-11T22:36:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.429+0000 7f01a5d46640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01a0102ef0 0x7f01a010edd0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.429+0000 7f01a5d46640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01880a2cf0 0x7f01880a51b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.429+0000 7f01a5d46640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01a0102570 0x7f01a010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.429+0000 7f01a5d46640 1 --2- 172.21.15.5:0/583089726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01a0100e20 0x7f01a010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:36:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.429+0000 7f01a5d46640 1 -- 172.21.15.5:0/583089726 >> 172.21.15.5:0/583089726 conn(0x7f01a00fc7d0 msgr2=0x7f01a00fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:36:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.429+0000 7f01a5d46640 1 -- 172.21.15.5:0/583089726 shutdown_connections 2024-09-11T22:36:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:58.429+0000 7f01a5d46640 1 -- 172.21.15.5:0/583089726 wait complete. 2024-09-11T22:36:58.636 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 8.53 MB/sec warmup 76 sec latency 30441.950 ms 2024-09-11T22:36:59.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:36:59 smithi005 ceph-mon[30943]: pgmap v470: 97 pgs: 97 active+clean; 335 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 563 KiB/s wr, 112 op/s 2024-09-11T22:36:59.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:36:59 smithi159 ceph-mon[35534]: pgmap v470: 97 pgs: 97 active+clean; 335 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 563 KiB/s wr, 112 op/s 2024-09-11T22:36:59.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:36:59 smithi172 ceph-mon[35461]: pgmap v470: 97 pgs: 97 active+clean; 335 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 563 KiB/s wr, 112 op/s 2024-09-11T22:36:59.377 INFO:tasks.cephfs.filesystem:scrub status for tag:c113d33b-5b4d-480c-bbe9-a4c6b2d782fa - {'path': '/', 'tag': 'c113d33b-5b4d-480c-bbe9-a4c6b2d782fa', 'options': 'recursive,force'} 2024-09-11T22:36:59.468 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:36:59.636 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8694 8.42 MB/sec warmup 77 sec latency 31442.041 ms 2024-09-11T22:37:00.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.982+0000 7f984bb73640 1 -- 172.21.15.5:0/1631258819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441008d0 msgr2=0x7f9844100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:00.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.982+0000 7f984bb73640 1 --2- 172.21.15.5:0/1631258819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441008d0 0x7f9844100cb0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f9840009900 tx=0x7f984002a410 comp rx=0 tx=0).stop 2024-09-11T22:37:00.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.982+0000 7f984bb73640 1 -- 172.21.15.5:0/1631258819 shutdown_connections 2024-09-11T22:37:00.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.982+0000 7f984bb73640 1 --2- 172.21.15.5:0/1631258819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f984410e480 0x7f9844110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.982+0000 7f984bb73640 1 --2- 172.21.15.5:0/1631258819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9844101280 0x7f984410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.982+0000 7f984bb73640 1 --2- 172.21.15.5:0/1631258819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441008d0 0x7f9844100cb0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.982+0000 7f984bb73640 1 -- 172.21.15.5:0/1631258819 >> 172.21.15.5:0/1631258819 conn(0x7f98440fc7a0 msgr2=0x7f98440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.982+0000 7f984bb73640 1 -- 172.21.15.5:0/1631258819 shutdown_connections 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.982+0000 7f984bb73640 1 -- 172.21.15.5:0/1631258819 wait complete. 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984bb73640 1 Processor -- start 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984bb73640 1 -- start start 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984bb73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441008d0 0x7f984419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984bb73640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9844101280 0x7f984419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984bb73640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f984410e480 0x7f984419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984bb73640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98441141b0 con 0x7f98441008d0 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984bb73640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9844114030 con 0x7f9844101280 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984bb73640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9844114330 con 0x7f984410e480 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984990f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441008d0 0x7f984419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984990f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441008d0 0x7f984419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55532/0 (socket says 172.21.15.5:55532) 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.983+0000 7f984990f640 1 -- 172.21.15.5:0/386342105 learned_addr learned my addr 172.21.15.5:0/386342105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984a110640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f984410e480 0x7f984419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984910e640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9844101280 0x7f984419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984990f640 1 -- 172.21.15.5:0/386342105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f984410e480 msgr2=0x7f984419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984990f640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f984410e480 0x7f984419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984990f640 1 -- 172.21.15.5:0/386342105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9844101280 msgr2=0x7f984419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984990f640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9844101280 0x7f984419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984990f640 1 -- 172.21.15.5:0/386342105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984419f640 con 0x7f98441008d0 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984a110640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f984410e480 0x7f984419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984990f640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441008d0 0x7f984419a690 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f9840006010 tx=0x7f98400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f983affd640 1 -- 172.21.15.5:0/386342105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984003f030 con 0x7f98441008d0 2024-09-11T22:37:00.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f983affd640 1 -- 172.21.15.5:0/386342105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9840003ce0 con 0x7f98441008d0 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984bb73640 1 -- 172.21.15.5:0/386342105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984419f8d0 con 0x7f98441008d0 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f983affd640 1 -- 172.21.15.5:0/386342105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840035070 con 0x7f98441008d0 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.984+0000 7f984bb73640 1 -- 172.21.15.5:0/386342105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a7210 con 0x7f98441008d0 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.986+0000 7f983affd640 1 -- 172.21.15.5:0/386342105 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9840042030 con 0x7f98441008d0 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984df4c740 1 Processor -- start 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984df4c740 1 -- start start 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984df4c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c323440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984df4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 0x55e20c32dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984df4c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c32e240 0x55e20c330630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984df4c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e20c288620 con 0x55e20c323060 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984df4c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e20c22fe50 con 0x55e20c323980 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984df4c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e20c2845e0 con 0x55e20c32e240 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984a110640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c323440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f983affd640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98300a2d70 0x7f98300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984a110640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c323440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55536/0 (socket says 172.21.15.5:55536) 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.987+0000 7f984a110640 1 -- 172.21.15.5:0/3299859231 learned_addr learned my addr 172.21.15.5:0/3299859231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984910e640 1 --2- 172.21.15.5:0/3299859231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c32e240 0x55e20c330630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984990f640 1 --2- 172.21.15.5:0/3299859231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 0x55e20c32dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f983affd640 1 -- 172.21.15.5:0/386342105 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f98400efb20 con 0x7f98441008d0 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984a110640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98300a2d70 0x7f98300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984a110640 1 -- 172.21.15.5:0/3299859231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c32e240 msgr2=0x55e20c330630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984a110640 1 --2- 172.21.15.5:0/3299859231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c32e240 0x55e20c330630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984a110640 1 -- 172.21.15.5:0/3299859231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 msgr2=0x55e20c32dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984a110640 1 --2- 172.21.15.5:0/3299859231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 0x55e20c32dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984a110640 1 -- 172.21.15.5:0/3299859231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20c220d40 con 0x55e20c323060 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984990f640 1 --2- 172.21.15.5:0/3299859231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 0x55e20c32dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984a110640 1 --2- 172.21.15.5:0/3299859231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c323440 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f983c014e50 tx=0x7f983c012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5392389ffc2e9b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f984a110640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98300a2d70 0x7f98300a5230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f983c012ba0 tx=0x7f983c00c000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f982ffff640 1 -- 172.21.15.5:0/3299859231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c00cbf0 con 0x55e20c323060 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f982ffff640 1 -- 172.21.15.5:0/3299859231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c014430 con 0x55e20c323060 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.988+0000 7f982ffff640 1 -- 172.21.15.5:0/3299859231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c0103e0 con 0x55e20c323060 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.989+0000 7f984df4c740 1 -- 172.21.15.5:0/3299859231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 msgr2=0x55e20c323440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.989+0000 7f984df4c740 1 --2- 172.21.15.5:0/3299859231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c323440 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f983c014e50 tx=0x7f983c012760 comp rx=0 tx=0).stop 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.989+0000 7f984df4c740 1 -- 172.21.15.5:0/3299859231 shutdown_connections 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.989+0000 7f984df4c740 1 --2- 172.21.15.5:0/3299859231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c32e240 0x55e20c330630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.989+0000 7f984df4c740 1 --2- 172.21.15.5:0/3299859231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 0x55e20c32dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.989+0000 7f984df4c740 1 --2- 172.21.15.5:0/3299859231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c323440 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.989+0000 7f984df4c740 1 -- 172.21.15.5:0/3299859231 >> 172.21.15.5:0/3299859231 conn(0x55e20c31f5f0 msgr2=0x55e20c31f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.989+0000 7f984df4c740 1 -- 172.21.15.5:0/3299859231 shutdown_connections 2024-09-11T22:37:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.989+0000 7f984df4c740 1 -- 172.21.15.5:0/3299859231 wait complete. 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.989+0000 7f984df4c740 1 -- start start 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984df4c740 1 Processor -- start 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984df4c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c323060 0x55e20c32b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984df4c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323980 0x55e20c32b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984df4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c32e240 0x55e20c32bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984df4c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e20c330e10 con 0x55e20c323980 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984df4c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e20c210f10 con 0x55e20c32e240 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984df4c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e20c330f90 con 0x55e20c323060 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984df4c740 10 client.0 _ll_register_callbacks cb 0x55e20c234db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984910e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323980 0x55e20c32b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984910e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323980 0x55e20c32b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55548/0 (socket says 172.21.15.5:55548) 2024-09-11T22:37:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984910e640 1 -- 172.21.15.5:0/2052367959 learned_addr learned my addr 172.21.15.5:0/2052367959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984990f640 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c323060 0x55e20c32b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984a110640 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c32e240 0x55e20c32bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984910e640 1 -- 172.21.15.5:0/2052367959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c323060 msgr2=0x55e20c32b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984910e640 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c323060 0x55e20c32b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984910e640 1 -- 172.21.15.5:0/2052367959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c32e240 msgr2=0x55e20c32bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984910e640 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c32e240 0x55e20c32bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.990+0000 7f984910e640 1 -- 172.21.15.5:0/2052367959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20bf97580 con 0x55e20c323980 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f984910e640 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323980 0x55e20c32b730 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f983400be40 tx=0x7f9834012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b37033a04914fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f982d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f982d7fa640 1 -- 172.21.15.5:0/2052367959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983401f060 con 0x55e20c323980 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f984df4c740 1 -- 172.21.15.5:0/2052367959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983c014430 con 0x55e20c323980 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f982d7fa640 1 -- 172.21.15.5:0/2052367959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f9834010960 con 0x55e20c323980 2024-09-11T22:37:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f982d7fa640 1 -- 172.21.15.5:0/2052367959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9834010b60 con 0x55e20c323980 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f984df4c740 10 client.20091 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f984df4c740 1 -- 172.21.15.5:0/2052367959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e20c061960 con 0x55e20c323980 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f982d7fa640 1 -- 172.21.15.5:0/2052367959 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983c014430 con 0x55e20c323980 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f982d7fa640 1 -- 172.21.15.5:0/2052367959 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f9834019030 con 0x55e20c323980 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f984df4c740 10 client.20091 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f984df4c740 10 client.20091 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f984df4c740 20 client.20091 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f984df4c740 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e20c33dee0 0x55e20c3402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f984df4c740 4 client.20091 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.991+0000 7f984df4c740 1 -- 172.21.15.5:0/2052367959 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e20c295b90 con 0x55e20c33dee0 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.992+0000 7f984990f640 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e20c33dee0 0x55e20c3402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.992+0000 7f984990f640 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e20c33dee0 0x55e20c3402f0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ada16fc84ce0a8c0 server_cookie=66192bbe64aa8fe4 in_seq=0 out_seq=0 2024-09-11T22:37:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.992+0000 7f982d7fa640 10 client.20091 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f982d7fa640 1 -- 172.21.15.5:0/2052367959 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e20c295b90 con 0x55e20c33dee0 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f982d7fa640 10 client.20091 handle_command_reply: tid=0 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 client.20091 shutdown 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 -- 172.21.15.5:0/2052367959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323980 msgr2=0x55e20c32b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323980 0x55e20c32b730 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f983400be40 tx=0x7f9834012820 comp rx=0 tx=0).stop 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 -- 172.21.15.5:0/2052367959 shutdown_connections 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e20c33dee0 0x55e20c3402f0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c32e240 0x55e20c32bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323980 0x55e20c32b730 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f982d7fa640 0 client.20091 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 --2- 172.21.15.5:0/2052367959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c323060 0x55e20c32b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 -- 172.21.15.5:0/2052367959 >> 172.21.15.5:0/2052367959 conn(0x55e20c31f5f0 msgr2=0x55e20c31e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 -- 172.21.15.5:0/2052367959 shutdown_connections 2024-09-11T22:37:00.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.993+0000 7f984df4c740 1 -- 172.21.15.5:0/2052367959 wait complete. 2024-09-11T22:37:00.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:36:59.994+0000 7f984df4c740 20 client.20091 trim_cache size 0 max 16384 2024-09-11T22:37:00.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984df4c740 1 Processor -- start 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984df4c740 1 -- start start 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984df4c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c345bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984df4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 0x55e20c346100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984df4c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c32e240 0x55e20c328450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984df4c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98441afc60 con 0x55e20c323060 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984df4c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f98441b1890 con 0x55e20c323980 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984df4c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98441b0330 con 0x55e20c32e240 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984990f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c345bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984990f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c345bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55562/0 (socket says 172.21.15.5:55562) 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984990f640 1 -- 172.21.15.5:0/884220169 learned_addr learned my addr 172.21.15.5:0/884220169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984a110640 1 --2- 172.21.15.5:0/884220169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c32e240 0x55e20c328450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.011+0000 7f984910e640 1 --2- 172.21.15.5:0/884220169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 0x55e20c346100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.012+0000 7f984990f640 1 -- 172.21.15.5:0/884220169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c32e240 msgr2=0x55e20c328450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.012+0000 7f984990f640 1 --2- 172.21.15.5:0/884220169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c32e240 0x55e20c328450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.012+0000 7f984990f640 1 -- 172.21.15.5:0/884220169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 msgr2=0x55e20c346100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.012+0000 7f984990f640 1 --2- 172.21.15.5:0/884220169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 0x55e20c346100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.012+0000 7f984990f640 1 -- 172.21.15.5:0/884220169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20c340970 con 0x55e20c323060 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.012+0000 7f984990f640 1 --2- 172.21.15.5:0/884220169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c345bc0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9840005ab0 tx=0x7f9840040fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c223198a143a5784 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.012+0000 7f982f7fe640 1 -- 172.21.15.5:0/884220169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840049040 con 0x55e20c323060 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.012+0000 7f982f7fe640 1 -- 172.21.15.5:0/884220169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f984003b580 con 0x55e20c323060 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.012+0000 7f982f7fe640 1 -- 172.21.15.5:0/884220169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98400c4d00 con 0x55e20c323060 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 -- 172.21.15.5:0/884220169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 msgr2=0x55e20c345bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 --2- 172.21.15.5:0/884220169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c345bc0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9840005ab0 tx=0x7f9840040fb0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 -- 172.21.15.5:0/884220169 shutdown_connections 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 --2- 172.21.15.5:0/884220169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c32e240 0x55e20c328450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 --2- 172.21.15.5:0/884220169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c323980 0x55e20c346100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 --2- 172.21.15.5:0/884220169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c345bc0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 -- 172.21.15.5:0/884220169 >> 172.21.15.5:0/884220169 conn(0x55e20c31f5f0 msgr2=0x55e20c329ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 -- 172.21.15.5:0/884220169 shutdown_connections 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 -- 172.21.15.5:0/884220169 wait complete. 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 -- start start 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 Processor -- start 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c3362a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c323980 0x55e20c3367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.013+0000 7f984df4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c32e240 0x55e20c332730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984df4c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98441b11c0 con 0x55e20c323060 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984df4c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f98441b0a30 con 0x55e20c32e240 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984df4c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f98441adc60 con 0x55e20c323980 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984df4c740 10 client.0 _ll_register_callbacks cb 0x7f983400ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984990f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c3362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984990f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c3362a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55574/0 (socket says 172.21.15.5:55574) 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984990f640 1 -- 172.21.15.5:0/4102907356 learned_addr learned my addr 172.21.15.5:0/4102907356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984910e640 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c323980 0x55e20c3367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984a110640 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c32e240 0x55e20c332730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984990f640 1 -- 172.21.15.5:0/4102907356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c323980 msgr2=0x55e20c3367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984990f640 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c323980 0x55e20c3367e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984990f640 1 -- 172.21.15.5:0/4102907356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c32e240 msgr2=0x55e20c332730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984990f640 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c32e240 0x55e20c332730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984990f640 1 -- 172.21.15.5:0/4102907356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20c332f60 con 0x55e20c323060 2024-09-11T22:37:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984990f640 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c3362a0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f98400ed800 tx=0x7f98400bbb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56f59fc0d3f51da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f982d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f982d7fa640 1 -- 172.21.15.5:0/4102907356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9840049040 con 0x55e20c323060 2024-09-11T22:37:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f982d7fa640 1 -- 172.21.15.5:0/4102907356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f98400c00e0 con 0x55e20c323060 2024-09-11T22:37:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.014+0000 7f984df4c740 1 -- 172.21.15.5:0/4102907356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e20c2900a0 con 0x55e20c323060 2024-09-11T22:37:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f982d7fa640 1 -- 172.21.15.5:0/4102907356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f984003bcb0 con 0x55e20c323060 2024-09-11T22:37:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f984df4c740 10 client.20097 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f984df4c740 1 -- 172.21.15.5:0/4102907356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e20c3a9640 con 0x55e20c323060 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f982d7fa640 1 -- 172.21.15.5:0/4102907356 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e20c2900a0 con 0x55e20c323060 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f982d7fa640 1 -- 172.21.15.5:0/4102907356 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f98400b8700 con 0x55e20c323060 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f984df4c740 10 client.20097 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f984df4c740 10 client.20097 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f984df4c740 20 client.20097 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f984df4c740 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e20c3a9cb0 0x55e20c3ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f984df4c740 4 client.20097 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "25352b1d", "release": true, "await": true}] 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f984df4c740 1 -- 172.21.15.5:0/4102907356 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "25352b1d", "release": true, "await": true}) -- 0x55e20c20c5a0 con 0x55e20c3a9cb0 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.015+0000 7f984910e640 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e20c3a9cb0 0x55e20c3ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.016+0000 7f984910e640 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e20c3a9cb0 0x55e20c3ac0d0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b82e739302a0377 server_cookie=cf4c5a7b7e62bf53 in_seq=0 out_seq=0 2024-09-11T22:37:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.016+0000 7f982d7fa640 10 client.20097 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:00.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f982d7fa640 1 -- 172.21.15.5:0/4102907356 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e20c20c5a0 con 0x55e20c3a9cb0 2024-09-11T22:37:00.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f982d7fa640 10 client.20097 handle_command_reply: tid=0 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 client.20097 shutdown 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 -- 172.21.15.5:0/4102907356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 msgr2=0x55e20c3362a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c3362a0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f98400ed800 tx=0x7f98400bbb20 comp rx=0 tx=0).stop 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 -- 172.21.15.5:0/4102907356 shutdown_connections 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e20c3a9cb0 0x55e20c3ac0d0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e20c32e240 0x55e20c332730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f982d7fa640 0 client.20097 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20c323980 0x55e20c3367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 --2- 172.21.15.5:0/4102907356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c323060 0x55e20c3362a0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 -- 172.21.15.5:0/4102907356 >> 172.21.15.5:0/4102907356 conn(0x55e20c31f5f0 msgr2=0x55e20c324ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 -- 172.21.15.5:0/4102907356 shutdown_connections 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 1 -- 172.21.15.5:0/4102907356 wait complete. 2024-09-11T22:37:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.092+0000 7f984df4c740 20 client.20097 trim_cache size 0 max 16384 2024-09-11T22:37:00.095 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:37:00.095 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 50, 2024-09-11T22:37:00.095 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:37:00.095 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 56, 2024-09-11T22:37:00.095 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:37:00.095 INFO:teuthology.orchestra.run.smithi005.stdout: "25352b1d": { 2024-09-11T22:37:00.095 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 56, 2024-09-11T22:37:00.095 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:37:00.095 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:37:00.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:37:00.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:37:00.097 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:37:00.097 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:37:00.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 -- 172.21.15.5:0/386342105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98300a2d70 msgr2=0x7f98300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:00.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98300a2d70 0x7f98300a5230 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f983c012ba0 tx=0x7f983c00c000 comp rx=0 tx=0).stop 2024-09-11T22:37:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 -- 172.21.15.5:0/386342105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441008d0 msgr2=0x7f984419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441008d0 0x7f984419a690 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f9840006010 tx=0x7f98400042f0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 -- 172.21.15.5:0/386342105 shutdown_connections 2024-09-11T22:37:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f984410e480 0x7f984419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f98300a2d70 0x7f98300a5230 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9844101280 0x7f984419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 --2- 172.21.15.5:0/386342105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98441008d0 0x7f984419a690 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 -- 172.21.15.5:0/386342105 >> 172.21.15.5:0/386342105 conn(0x7f98440fc7a0 msgr2=0x7f98440feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 -- 172.21.15.5:0/386342105 shutdown_connections 2024-09-11T22:37:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:00.093+0000 7f984bb73640 1 -- 172.21.15.5:0/386342105 wait complete. 2024-09-11T22:37:00.636 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8700 8.32 MB/sec warmup 78 sec latency 31896.904 ms 2024-09-11T22:37:00.793 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '25352b1d', total seconds elapsed: 35.6 2024-09-11T22:37:01.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:01 smithi005 ceph-mon[30943]: pgmap v471: 97 pgs: 97 active+clean; 335 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 524 KiB/s wr, 105 op/s 2024-09-11T22:37:01.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:01 smithi159 ceph-mon[35534]: pgmap v471: 97 pgs: 97 active+clean; 335 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 524 KiB/s wr, 105 op/s 2024-09-11T22:37:01.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:01 smithi172 ceph-mon[35461]: pgmap v471: 97 pgs: 97 active+clean; 335 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 524 KiB/s wr, 105 op/s 2024-09-11T22:37:01.636 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8700 8.21 MB/sec warmup 79 sec latency 1531.764 ms 2024-09-11T22:37:02.636 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8700 8.11 MB/sec warmup 80 sec latency 2531.893 ms 2024-09-11T22:37:03.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:03 smithi005 ceph-mon[30943]: pgmap v472: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-11T22:37:03.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:03 smithi159 ceph-mon[35534]: pgmap v472: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-11T22:37:03.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:03 smithi172 ceph-mon[35461]: pgmap v472: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-09-11T22:37:03.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8700 8.01 MB/sec warmup 81 sec latency 3532.009 ms 2024-09-11T22:37:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.985+0000 7f931f323640 1 -- 172.21.15.5:0/1713388780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9318103600 msgr2=0x7f931810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.985+0000 7f931f323640 1 --2- 172.21.15.5:0/1713388780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9318103600 0x7f931810bae0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f9308009900 tx=0x7f930802a480 comp rx=0 tx=0).stop 2024-09-11T22:37:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.985+0000 7f931f323640 1 -- 172.21.15.5:0/1713388780 shutdown_connections 2024-09-11T22:37:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.985+0000 7f931f323640 1 --2- 172.21.15.5:0/1713388780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931810c0b0 0x7f931810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.985+0000 7f931f323640 1 --2- 172.21.15.5:0/1713388780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9318103600 0x7f931810bae0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.985+0000 7f931f323640 1 --2- 172.21.15.5:0/1713388780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9318102ce0 0x7f93181030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.985+0000 7f931f323640 1 -- 172.21.15.5:0/1713388780 >> 172.21.15.5:0/1713388780 conn(0x7f93180fc6f0 msgr2=0x7f93180feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.985+0000 7f931f323640 1 -- 172.21.15.5:0/1713388780 shutdown_connections 2024-09-11T22:37:03.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.985+0000 7f931f323640 1 -- 172.21.15.5:0/1713388780 wait complete. 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.986+0000 7f931f323640 1 Processor -- start 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.986+0000 7f931f323640 1 -- start start 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931f323640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9318102ce0 0x7f93181983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931f323640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9318103600 0x7f9318198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931f323640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931810c0b0 0x7f931819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931f323640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9318111f20 con 0x7f9318102ce0 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931f323640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9318111da0 con 0x7f9318103600 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931f323640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93181120a0 con 0x7f931810c0b0 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931d0bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9318102ce0 0x7f93181983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931d0bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9318102ce0 0x7f93181983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46516/0 (socket says 172.21.15.5:46516) 2024-09-11T22:37:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931d0bf640 1 -- 172.21.15.5:0/3730186736 learned_addr learned my addr 172.21.15.5:0/3730186736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931c8be640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9318103600 0x7f9318198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931d8c0640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931810c0b0 0x7f931819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931d8c0640 1 -- 172.21.15.5:0/3730186736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9318103600 msgr2=0x7f9318198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931d8c0640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9318103600 0x7f9318198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931d8c0640 1 -- 172.21.15.5:0/3730186736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9318102ce0 msgr2=0x7f93181983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931d8c0640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9318102ce0 0x7f93181983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931d8c0640 1 -- 172.21.15.5:0/3730186736 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931819d3c0 con 0x7f931810c0b0 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.987+0000 7f931d0bf640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9318102ce0 0x7f93181983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.988+0000 7f931d8c0640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931810c0b0 0x7f931819cc90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9310009fd0 tx=0x7f931000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.988+0000 7f930e7fc640 1 -- 172.21.15.5:0/3730186736 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931000ce20 con 0x7f931810c0b0 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.988+0000 7f930e7fc640 1 -- 172.21.15.5:0/3730186736 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9310005020 con 0x7f931810c0b0 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.988+0000 7f931f323640 1 -- 172.21.15.5:0/3730186736 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931819d6b0 con 0x7f931810c0b0 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.988+0000 7f930e7fc640 1 -- 172.21.15.5:0/3730186736 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310010720 con 0x7f931810c0b0 2024-09-11T22:37:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.988+0000 7f931f323640 1 -- 172.21.15.5:0/3730186736 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9318100860 con 0x7f931810c0b0 2024-09-11T22:37:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.990+0000 7f930e7fc640 1 -- 172.21.15.5:0/3730186736 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9310005540 con 0x7f931810c0b0 2024-09-11T22:37:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f93220bb740 1 Processor -- start 2024-09-11T22:37:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f93220bb740 1 -- start start 2024-09-11T22:37:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f93220bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491dffaf0 0x556491dffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f93220bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 0x556491e0a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f93220bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 0x556491e0d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f93220bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556491d642f0 con 0x556491e0acb0 2024-09-11T22:37:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f93220bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556491d0c270 con 0x556491dffaf0 2024-09-11T22:37:04.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f93220bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556491d602b0 con 0x556491e00410 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931c8be640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491dffaf0 0x556491dffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931d0bf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 0x556491e0a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931c8be640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491dffaf0 0x556491dffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53998/0 (socket says 172.21.15.5:53998) 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931c8be640 1 -- 172.21.15.5:0/4007714448 learned_addr learned my addr 172.21.15.5:0/4007714448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931d0bf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 0x556491e0a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:52540/0 (socket says 172.21.15.5:52540) 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f930e7fc640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93040a2da0 0x7f93040a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931c8be640 1 --2- 172.21.15.5:0/4007714448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 0x556491e0d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931d8c0640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93040a2da0 0x7f93040a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931c8be640 1 -- 172.21.15.5:0/4007714448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 msgr2=0x556491e0a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931c8be640 1 --2- 172.21.15.5:0/4007714448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 0x556491e0a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931c8be640 1 -- 172.21.15.5:0/4007714448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 msgr2=0x556491e0d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931c8be640 1 --2- 172.21.15.5:0/4007714448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 0x556491e0d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931c8be640 1 -- 172.21.15.5:0/4007714448 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556491d0b340 con 0x556491dffaf0 2024-09-11T22:37:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f931d0bf640 1 --2- 172.21.15.5:0/4007714448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 0x556491e0a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.991+0000 7f930e7fc640 1 -- 172.21.15.5:0/3730186736 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f93100cfa70 con 0x7f931810c0b0 2024-09-11T22:37:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.992+0000 7f931c8be640 1 --2- 172.21.15.5:0/4007714448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491dffaf0 0x556491dffed0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9308005ec0 tx=0x7f9308005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cccf6c9358883fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.992+0000 7f931d8c0640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93040a2da0 0x7f93040a5260 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9310098360 tx=0x7f93100bc040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.992+0000 7f93037fe640 1 -- 172.21.15.5:0/4007714448 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308035030 con 0x556491dffaf0 2024-09-11T22:37:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.992+0000 7f93037fe640 1 -- 172.21.15.5:0/4007714448 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93080046c0 con 0x556491dffaf0 2024-09-11T22:37:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.992+0000 7f93037fe640 1 -- 172.21.15.5:0/4007714448 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308003850 con 0x556491dffaf0 2024-09-11T22:37:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 -- 172.21.15.5:0/4007714448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491dffaf0 msgr2=0x556491dffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 --2- 172.21.15.5:0/4007714448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491dffaf0 0x556491dffed0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9308005ec0 tx=0x7f9308005b50 comp rx=0 tx=0).stop 2024-09-11T22:37:04.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 -- 172.21.15.5:0/4007714448 shutdown_connections 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 --2- 172.21.15.5:0/4007714448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 0x556491e0d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 --2- 172.21.15.5:0/4007714448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 0x556491e0a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 --2- 172.21.15.5:0/4007714448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491dffaf0 0x556491dffed0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 -- 172.21.15.5:0/4007714448 >> 172.21.15.5:0/4007714448 conn(0x556491dfc060 msgr2=0x556491dfc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 -- 172.21.15.5:0/4007714448 shutdown_connections 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 -- 172.21.15.5:0/4007714448 wait complete. 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 -- start start 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.993+0000 7f93220bb740 1 Processor -- start 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f93220bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e07b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f93220bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 0x556491e080d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f93220bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e0acb0 0x556491e08610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f93220bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556491e0d880 con 0x556491dffaf0 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f93220bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556491ced290 con 0x556491e00410 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f93220bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556491e0da00 con 0x556491e0acb0 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f93220bb740 10 client.0 _ll_register_callbacks cb 0x556491d1e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931c8be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e07b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931c8be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e07b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46532/0 (socket says 172.21.15.5:46532) 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931c8be640 1 -- 172.21.15.5:0/1030606982 learned_addr learned my addr 172.21.15.5:0/1030606982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931d0bf640 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 0x556491e080d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931c8be640 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e0acb0 0x556491e08610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931c8be640 1 -- 172.21.15.5:0/1030606982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e0acb0 msgr2=0x556491e08610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931c8be640 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e0acb0 0x556491e08610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931c8be640 1 -- 172.21.15.5:0/1030606982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 msgr2=0x556491e080d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931c8be640 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 0x556491e080d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931c8be640 1 -- 172.21.15.5:0/1030606982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556491a77210 con 0x556491dffaf0 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f931c8be640 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e07b90 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f930802ac70 tx=0x7f930802ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1373d867e8fc29 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f9300ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f9300ff9640 1 -- 172.21.15.5:0/1030606982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308052060 con 0x556491dffaf0 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f9300ff9640 1 -- 172.21.15.5:0/1030606982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f9308005200 con 0x556491dffaf0 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.994+0000 7f93220bb740 1 -- 172.21.15.5:0/1030606982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93080046c0 con 0x556491dffaf0 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.995+0000 7f9300ff9640 1 -- 172.21.15.5:0/1030606982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9308035030 con 0x556491dffaf0 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.995+0000 7f93220bb740 10 client.20109 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.995+0000 7f93220bb740 1 -- 172.21.15.5:0/1030606982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556491b43620 con 0x556491dffaf0 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.995+0000 7f9300ff9640 1 -- 172.21.15.5:0/1030606982 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93080046c0 con 0x556491dffaf0 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.995+0000 7f9300ff9640 1 -- 172.21.15.5:0/1030606982 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f930804a040 con 0x556491dffaf0 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.995+0000 7f93220bb740 10 client.20109 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:04.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.995+0000 7f93220bb740 10 client.20109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.995+0000 7f93220bb740 20 client.20109 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.996+0000 7f93220bb740 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556491e1aa80 0x556491e1ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.996+0000 7f93220bb740 4 client.20109 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.996+0000 7f93220bb740 1 -- 172.21.15.5:0/1030606982 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556491d71a20 con 0x556491e1aa80 2024-09-11T22:37:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.996+0000 7f931d0bf640 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556491e1aa80 0x556491e1ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.996+0000 7f931d0bf640 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556491e1aa80 0x556491e1ce70 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2702fca64e26a54c server_cookie=7c81653eb438492b in_seq=0 out_seq=0 2024-09-11T22:37:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.996+0000 7f9300ff9640 10 client.20109 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f9300ff9640 1 -- 172.21.15.5:0/1030606982 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556491d71a20 con 0x556491e1aa80 2024-09-11T22:37:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f9300ff9640 10 client.20109 handle_command_reply: tid=0 2024-09-11T22:37:04.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f93220bb740 1 client.20109 shutdown 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f93220bb740 1 -- 172.21.15.5:0/1030606982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 msgr2=0x556491e07b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f93220bb740 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e07b90 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f930802ac70 tx=0x7f930802ac00 comp rx=0 tx=0).stop 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f93220bb740 1 -- 172.21.15.5:0/1030606982 shutdown_connections 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f93220bb740 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556491e1aa80 0x556491e1ce70 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f93220bb740 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e0acb0 0x556491e08610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f93220bb740 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 0x556491e080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f93220bb740 1 --2- 172.21.15.5:0/1030606982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e07b90 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f9300ff9640 0 client.20109 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f93220bb740 1 -- 172.21.15.5:0/1030606982 >> 172.21.15.5:0/1030606982 conn(0x556491dfc060 msgr2=0x556491e01d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.997+0000 7f93220bb740 1 -- 172.21.15.5:0/1030606982 shutdown_connections 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.998+0000 7f93220bb740 1 -- 172.21.15.5:0/1030606982 wait complete. 2024-09-11T22:37:04.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:03.998+0000 7f93220bb740 20 client.20109 trim_cache size 0 max 16384 2024-09-11T22:37:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f93220bb740 1 Processor -- start 2024-09-11T22:37:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f93220bb740 1 -- start start 2024-09-11T22:37:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f93220bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e08440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f93220bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 0x556491e1fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f93220bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e0acb0 0x556491e203d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f93220bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93181afbb0 con 0x556491dffaf0 2024-09-11T22:37:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f93220bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93181b17e0 con 0x556491e0acb0 2024-09-11T22:37:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f93220bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93181b0280 con 0x556491e00410 2024-09-11T22:37:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f931c8be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e08440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f931c8be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e08440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46546/0 (socket says 172.21.15.5:46546) 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.012+0000 7f931c8be640 1 -- 172.21.15.5:0/2134592355 learned_addr learned my addr 172.21.15.5:0/2134592355 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.013+0000 7f931d0bf640 1 --2- 172.21.15.5:0/2134592355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 0x556491e1fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.013+0000 7f931c8be640 1 -- 172.21.15.5:0/2134592355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 msgr2=0x556491e1fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.013+0000 7f931c8be640 1 --2- 172.21.15.5:0/2134592355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 0x556491e1fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.013+0000 7f931c8be640 1 -- 172.21.15.5:0/2134592355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e0acb0 msgr2=0x556491e203d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.013+0000 7f931c8be640 1 --2- 172.21.15.5:0/2134592355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e0acb0 0x556491e203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.013+0000 7f931c8be640 1 -- 172.21.15.5:0/2134592355 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556491dff920 con 0x556491dffaf0 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.013+0000 7f931c8be640 1 --2- 172.21.15.5:0/2134592355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e08440 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9308031550 tx=0x7f9308004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6efe58303f9b82 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.013+0000 7f9302ffd640 1 -- 172.21.15.5:0/2134592355 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308052070 con 0x556491dffaf0 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.013+0000 7f9302ffd640 1 -- 172.21.15.5:0/2134592355 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9308041070 con 0x556491dffaf0 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.013+0000 7f9302ffd640 1 -- 172.21.15.5:0/2134592355 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308004de0 con 0x556491dffaf0 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 -- 172.21.15.5:0/2134592355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 msgr2=0x556491e08440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 --2- 172.21.15.5:0/2134592355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e08440 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9308031550 tx=0x7f9308004210 comp rx=0 tx=0).stop 2024-09-11T22:37:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 -- 172.21.15.5:0/2134592355 shutdown_connections 2024-09-11T22:37:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 --2- 172.21.15.5:0/2134592355 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e0acb0 0x556491e203d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 --2- 172.21.15.5:0/2134592355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491e00410 0x556491e1fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 --2- 172.21.15.5:0/2134592355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491dffaf0 0x556491e08440 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 -- 172.21.15.5:0/2134592355 >> 172.21.15.5:0/2134592355 conn(0x556491dfc060 msgr2=0x556491e09c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 -- 172.21.15.5:0/2134592355 shutdown_connections 2024-09-11T22:37:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 -- 172.21.15.5:0/2134592355 wait complete. 2024-09-11T22:37:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 -- start start 2024-09-11T22:37:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.014+0000 7f93220bb740 1 Processor -- start 2024-09-11T22:37:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f93220bb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491dffaf0 0x556491e206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f93220bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 0x556491e1da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f93220bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 0x556491e1e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f93220bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93181b1110 con 0x556491e0acb0 2024-09-11T22:37:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f93220bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93181b0980 con 0x556491e00410 2024-09-11T22:37:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f93220bb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f93181adbb0 con 0x556491dffaf0 2024-09-11T22:37:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f93220bb740 10 client.0 _ll_register_callbacks cb 0x7f9308009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931c8be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 0x556491e1e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931c8be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 0x556491e1e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46560/0 (socket says 172.21.15.5:46560) 2024-09-11T22:37:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931c8be640 1 -- 172.21.15.5:0/3529645157 learned_addr learned my addr 172.21.15.5:0/3529645157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931d0bf640 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 0x556491e1da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931c8be640 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491dffaf0 0x556491e206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931d0bf640 1 -- 172.21.15.5:0/3529645157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491dffaf0 msgr2=0x556491e206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931d0bf640 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491dffaf0 0x556491e206f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931d0bf640 1 -- 172.21.15.5:0/3529645157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 msgr2=0x556491e1e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931d0bf640 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 0x556491e1e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931d0bf640 1 -- 172.21.15.5:0/3529645157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556491e1e880 con 0x556491e00410 2024-09-11T22:37:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931c8be640 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 0x556491e1e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f931d0bf640 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 0x556491e1da80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x556491d0c380 tx=0x7f9314013c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5b75ccf7b825dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f9300ff9640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:37:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f9300ff9640 1 -- 172.21.15.5:0/3529645157 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314010b00 con 0x556491e00410 2024-09-11T22:37:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f93220bb740 1 -- 172.21.15.5:0/3529645157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9308041070 con 0x556491e00410 2024-09-11T22:37:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.015+0000 7f9300ff9640 1 -- 172.21.15.5:0/3529645157 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8629+0+0 (secure 0 0 0) 0x7f9314010ca0 con 0x556491e00410 2024-09-11T22:37:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f9300ff9640 1 -- 172.21.15.5:0/3529645157 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9314012830 con 0x556491e00410 2024-09-11T22:37:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f93220bb740 10 client.29195 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f93220bb740 1 -- 172.21.15.5:0/3529645157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x556491e85560 con 0x556491e00410 2024-09-11T22:37:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f9300ff9640 1 -- 172.21.15.5:0/3529645157 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9308041070 con 0x556491e00410 2024-09-11T22:37:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f9300ff9640 1 -- 172.21.15.5:0/3529645157 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f9314027030 con 0x556491e00410 2024-09-11T22:37:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f93220bb740 10 client.29195 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f93220bb740 10 client.29195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f93220bb740 20 client.29195 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f93220bb740 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556491e85d20 0x556491e88140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f93220bb740 4 client.29195 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:37:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.016+0000 7f93220bb740 1 -- 172.21.15.5:0/3529645157 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556491d6bd70 con 0x556491e85d20 2024-09-11T22:37:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.017+0000 7f931c8be640 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556491e85d20 0x556491e88140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.017+0000 7f931c8be640 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556491e85d20 0x556491e88140 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e059996eb73c16 server_cookie=499d0b94a453a888 in_seq=0 out_seq=0 2024-09-11T22:37:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.017+0000 7f9300ff9640 10 client.29195 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f9300ff9640 1 -- 172.21.15.5:0/3529645157 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556491d6bd70 con 0x556491e85d20 2024-09-11T22:37:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f9300ff9640 10 client.29195 handle_command_reply: tid=0 2024-09-11T22:37:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 client.29195 shutdown 2024-09-11T22:37:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 -- 172.21.15.5:0/3529645157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 msgr2=0x556491e1da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 0x556491e1da80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x556491d0c380 tx=0x7f9314013c70 comp rx=0 tx=0).stop 2024-09-11T22:37:04.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 -- 172.21.15.5:0/3529645157 shutdown_connections 2024-09-11T22:37:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556491e85d20 0x556491e88140 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556491e0acb0 0x556491e1e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f9300ff9640 0 client.29195 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556491e00410 0x556491e1da80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 --2- 172.21.15.5:0/3529645157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556491dffaf0 0x556491e206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 -- 172.21.15.5:0/3529645157 >> 172.21.15.5:0/3529645157 conn(0x556491dfc060 msgr2=0x556491dfa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 -- 172.21.15.5:0/3529645157 shutdown_connections 2024-09-11T22:37:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 1 -- 172.21.15.5:0/3529645157 wait complete. 2024-09-11T22:37:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.074+0000 7f93220bb740 20 client.29195 trim_cache size 0 max 16384 2024-09-11T22:37:04.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:37:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:37:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:37:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:37:04.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 -- 172.21.15.5:0/3730186736 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93040a2da0 msgr2=0x7f93040a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93040a2da0 0x7f93040a5260 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9310098360 tx=0x7f93100bc040 comp rx=0 tx=0).stop 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 -- 172.21.15.5:0/3730186736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931810c0b0 msgr2=0x7f931819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931810c0b0 0x7f931819cc90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9310009fd0 tx=0x7f931000ef10 comp rx=0 tx=0).stop 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 -- 172.21.15.5:0/3730186736 shutdown_connections 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f931810c0b0 0x7f931819cc90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f93040a2da0 0x7f93040a5260 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9318103600 0x7f9318198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 --2- 172.21.15.5:0/3730186736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9318102ce0 0x7f93181983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 -- 172.21.15.5:0/3730186736 >> 172.21.15.5:0/3730186736 conn(0x7f93180fc6f0 msgr2=0x7f93180fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 -- 172.21.15.5:0/3730186736 shutdown_connections 2024-09-11T22:37:04.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:04.075+0000 7f931f323640 1 -- 172.21.15.5:0/3730186736 wait complete. 2024-09-11T22:37:04.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:04 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:37:04.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:04 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:37:04.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:04 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:37:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:04 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:37:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:04 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:37:04.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:04 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:37:04.613 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:04 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:37:04.613 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:04 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:37:04.613 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:04 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:37:04.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8728 7.92 MB/sec warmup 82 sec latency 4532.107 ms 2024-09-11T22:37:05.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:05 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:37:05.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:05 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:37:05.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:05 smithi005 ceph-mon[30943]: pgmap v473: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-11T22:37:05.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:05 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:37:05.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:05 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:37:05.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:05 smithi159 ceph-mon[35534]: pgmap v473: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-11T22:37:05.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:05 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:37:05.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:05 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:37:05.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:05 smithi172 ceph-mon[35461]: pgmap v473: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-11T22:37:05.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8728 7.82 MB/sec warmup 83 sec latency 5532.211 ms 2024-09-11T22:37:06.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8728 7.73 MB/sec warmup 84 sec latency 6532.321 ms 2024-09-11T22:37:07.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:07 smithi005 ceph-mon[30943]: pgmap v474: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-11T22:37:07.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:07 smithi159 ceph-mon[35534]: pgmap v474: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-11T22:37:07.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:07 smithi172 ceph-mon[35461]: pgmap v474: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-11T22:37:07.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8728 7.64 MB/sec warmup 85 sec latency 7532.426 ms 2024-09-11T22:37:08.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8728 7.55 MB/sec warmup 86 sec latency 8532.563 ms 2024-09-11T22:37:09.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:09 smithi005 ceph-mon[30943]: pgmap v475: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-11T22:37:09.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:09 smithi159 ceph-mon[35534]: pgmap v475: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-11T22:37:09.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:09 smithi172 ceph-mon[35461]: pgmap v475: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-11T22:37:09.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8995 7.55 MB/sec warmup 87 sec latency 8831.118 ms 2024-09-11T22:37:10.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 9560 7.79 MB/sec warmup 88 sec latency 172.202 ms 2024-09-11T22:37:11.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:11 smithi005 ceph-mon[30943]: pgmap v476: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 779 KiB/s wr, 78 op/s 2024-09-11T22:37:11.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:11 smithi159 ceph-mon[35534]: pgmap v476: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 779 KiB/s wr, 78 op/s 2024-09-11T22:37:11.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:11 smithi172 ceph-mon[35461]: pgmap v476: 97 pgs: 97 active+clean; 340 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 779 KiB/s wr, 78 op/s 2024-09-11T22:37:11.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10173 8.04 MB/sec warmup 89 sec latency 200.152 ms 2024-09-11T22:37:12.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10842 8.29 MB/sec warmup 90 sec latency 201.317 ms 2024-09-11T22:37:13.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:13 smithi005 ceph-mon[30943]: pgmap v477: 97 pgs: 97 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.1 MiB/s wr, 222 op/s 2024-09-11T22:37:13.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:13 smithi159 ceph-mon[35534]: pgmap v477: 97 pgs: 97 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.1 MiB/s wr, 222 op/s 2024-09-11T22:37:13.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:13 smithi172 ceph-mon[35461]: pgmap v477: 97 pgs: 97 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.1 MiB/s wr, 222 op/s 2024-09-11T22:37:13.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11488 8.78 MB/sec warmup 91 sec latency 151.310 ms 2024-09-11T22:37:14.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11869 8.98 MB/sec warmup 92 sec latency 73.011 ms 2024-09-11T22:37:15.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:15 smithi005 ceph-mon[30943]: pgmap v478: 97 pgs: 97 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-09-11T22:37:15.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:15 smithi005 ceph-mon[30943]: osdmap e110: 12 total, 12 up, 12 in 2024-09-11T22:37:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:15 smithi159 ceph-mon[35534]: pgmap v478: 97 pgs: 97 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-09-11T22:37:15.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:15 smithi159 ceph-mon[35534]: osdmap e110: 12 total, 12 up, 12 in 2024-09-11T22:37:15.609 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:15 smithi172 ceph-mon[35461]: pgmap v478: 97 pgs: 97 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.4 MiB/s wr, 200 op/s 2024-09-11T22:37:15.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:15 smithi172 ceph-mon[35461]: osdmap e110: 12 total, 12 up, 12 in 2024-09-11T22:37:15.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12070 9.01 MB/sec warmup 93 sec latency 180.414 ms 2024-09-11T22:37:16.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12378 9.05 MB/sec warmup 94 sec latency 67.519 ms 2024-09-11T22:37:17.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.191+0000 7f7ead32c640 1 -- 172.21.15.5:0/3196439717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea81048f0 msgr2=0x7f7ea8108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.191+0000 7f7ead32c640 1 --2- 172.21.15.5:0/3196439717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea81048f0 0x7f7ea8108930 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009900 tx=0x7f7e9002a410 comp rx=0 tx=0).stop 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.191+0000 7f7ead32c640 1 -- 172.21.15.5:0/3196439717 shutdown_connections 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.191+0000 7f7ead32c640 1 --2- 172.21.15.5:0/3196439717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ea8109090 0x7f7ea8110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.191+0000 7f7ead32c640 1 --2- 172.21.15.5:0/3196439717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea81048f0 0x7f7ea8108930 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.191+0000 7f7ead32c640 1 --2- 172.21.15.5:0/3196439717 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ea8103f40 0x7f7ea8104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.191+0000 7f7ead32c640 1 -- 172.21.15.5:0/3196439717 >> 172.21.15.5:0/3196439717 conn(0x7f7ea80ff9b0 msgr2=0x7f7ea8101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.191+0000 7f7ead32c640 1 -- 172.21.15.5:0/3196439717 shutdown_connections 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.191+0000 7f7ead32c640 1 -- 172.21.15.5:0/3196439717 wait complete. 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ead32c640 1 Processor -- start 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ead32c640 1 -- start start 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ead32c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8103f40 0x7f7ea819a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ead32c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ea8109090 0x7f7ea819aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ead32c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ea819ee40 0x7f7ea819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ead32c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8114090 con 0x7f7ea8103f40 2024-09-11T22:37:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ead32c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8113f10 con 0x7f7ea819ee40 2024-09-11T22:37:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ead32c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8114210 con 0x7f7ea8109090 2024-09-11T22:37:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8103f40 0x7f7ea819a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8103f40 0x7f7ea819a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38100/0 (socket says 172.21.15.5:38100) 2024-09-11T22:37:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/3666085082 learned_addr learned my addr 172.21.15.5:0/3666085082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7e9ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ea8109090 0x7f7ea819aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.192+0000 7f7ea77fe640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ea819ee40 0x7f7ea819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/3666085082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ea8109090 msgr2=0x7f7ea819aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ea8109090 0x7f7ea819aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/3666085082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ea819ee40 msgr2=0x7f7ea819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ea819ee40 0x7f7ea819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/3666085082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea819f930 con 0x7f7ea8103f40 2024-09-11T22:37:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8103f40 0x7f7ea819a580 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800e9f0 tx=0x7f7e9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ea4ff9640 1 -- 172.21.15.5:0/3666085082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800cd80 con 0x7f7ea8103f40 2024-09-11T22:37:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ead32c640 1 -- 172.21.15.5:0/3666085082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea819fc20 con 0x7f7ea8103f40 2024-09-11T22:37:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ead32c640 1 -- 172.21.15.5:0/3666085082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea81a7430 con 0x7f7ea8103f40 2024-09-11T22:37:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ea4ff9640 1 -- 172.21.15.5:0/3666085082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e98010910 con 0x7f7ea8103f40 2024-09-11T22:37:17.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.193+0000 7f7ea4ff9640 1 -- 172.21.15.5:0/3666085082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98026060 con 0x7f7ea8103f40 2024-09-11T22:37:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.195+0000 7f7ea4ff9640 1 -- 172.21.15.5:0/3666085082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7e98010ac0 con 0x7f7ea8103f40 2024-09-11T22:37:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.196+0000 7f7eb00c1740 1 Processor -- start 2024-09-11T22:37:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.196+0000 7f7eb00c1740 1 -- start start 2024-09-11T22:37:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.196+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a7419ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a741a410 0x55e2a7424770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a74270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e2a737e2f0 con 0x55e2a7419af0 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e2a7326270 con 0x55e2a7424cb0 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e2a737a2b0 con 0x55e2a741a410 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a7419ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a741a410 0x55e2a7424770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a7419ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38102/0 (socket says 172.21.15.5:38102) 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7e9ffff640 1 --2- 172.21.15.5:0/2410685260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a74270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7ea77fe640 1 -- 172.21.15.5:0/2410685260 learned_addr learned my addr 172.21.15.5:0/2410685260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7ea4ff9640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e940a2da0 0x7f7e940a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7ea77fe640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e940a2da0 0x7f7e940a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7ea4ff9640 1 -- 172.21.15.5:0/3666085082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8673+0+0 (secure 0 0 0) 0x7f7e980cf8c0 con 0x7f7ea8103f40 2024-09-11T22:37:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7e9ffff640 1 -- 172.21.15.5:0/2410685260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a741a410 msgr2=0x55e2a7424770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7e9ffff640 1 --2- 172.21.15.5:0/2410685260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a741a410 0x55e2a7424770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7e9ffff640 1 -- 172.21.15.5:0/2410685260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 msgr2=0x55e2a7419ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7e9ffff640 1 --2- 172.21.15.5:0/2410685260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a7419ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7e9ffff640 1 -- 172.21.15.5:0/2410685260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a7325340 con 0x55e2a7424cb0 2024-09-11T22:37:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7ea77fe640 1 --2- 172.21.15.5:0/2410685260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a7419ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.197+0000 7f7e9ffff640 1 --2- 172.21.15.5:0/2410685260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a74270a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7e90006010 tx=0x7f7e90005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a7df159faf3606 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.198+0000 7f7ea77fe640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e940a2da0 0x7f7e940a5260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00049d0 tx=0x7f7ea0004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.198+0000 7f7e9d7fa640 1 -- 172.21.15.5:0/2410685260 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90040070 con 0x55e2a7424cb0 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.198+0000 7f7e9d7fa640 1 -- 172.21.15.5:0/2410685260 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e90005500 con 0x55e2a7424cb0 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.198+0000 7f7e9d7fa640 1 -- 172.21.15.5:0/2410685260 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90035040 con 0x55e2a7424cb0 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.198+0000 7f7eb00c1740 1 -- 172.21.15.5:0/2410685260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 msgr2=0x55e2a74270a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.198+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/2410685260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a74270a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7e90006010 tx=0x7f7e90005bd0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.199+0000 7f7eb00c1740 1 -- 172.21.15.5:0/2410685260 shutdown_connections 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.199+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/2410685260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a74270a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.199+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/2410685260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a741a410 0x55e2a7424770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.199+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/2410685260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a7419ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.199+0000 7f7eb00c1740 1 -- 172.21.15.5:0/2410685260 >> 172.21.15.5:0/2410685260 conn(0x55e2a7416060 msgr2=0x55e2a7416440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.199+0000 7f7eb00c1740 1 -- 172.21.15.5:0/2410685260 shutdown_connections 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.199+0000 7f7eb00c1740 1 -- 172.21.15.5:0/2410685260 wait complete. 2024-09-11T22:37:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.199+0000 7f7eb00c1740 1 -- start start 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.199+0000 7f7eb00c1740 1 Processor -- start 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 0x55e2a7421b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a741a410 0x55e2a74220a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a74225e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e2a7427880 con 0x55e2a741a410 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e2a7307290 con 0x55e2a7424cb0 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a74225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7e9ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a741a410 0x55e2a74220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7e9ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a741a410 0x55e2a74220a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38110/0 (socket says 172.21.15.5:38110) 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7e9ffff640 1 -- 172.21.15.5:0/700727104 learned_addr learned my addr 172.21.15.5:0/700727104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 0x55e2a7421b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7eb00c1740 1 -- 172.21.15.5:0/700727104 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e2a7427a00 con 0x55e2a7419af0 2024-09-11T22:37:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7eb00c1740 10 client.0 _ll_register_callbacks cb 0x55e2a7338140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7ea77fe640 1 -- 172.21.15.5:0/700727104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 msgr2=0x55e2a7421b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7ea77fe640 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 0x55e2a7421b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7ea77fe640 1 -- 172.21.15.5:0/700727104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a741a410 msgr2=0x55e2a74220a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7ea77fe640 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a741a410 0x55e2a74220a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7ea77fe640 1 -- 172.21.15.5:0/700727104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a7091210 con 0x55e2a7424cb0 2024-09-11T22:37:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.200+0000 7f7e9ffff640 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a741a410 0x55e2a74220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.201+0000 7f7ea77fe640 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a74225e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0005c40 tx=0x7f7ea0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12b913d11ea40245 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.201+0000 7f7e8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:37:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.201+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/700727104 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0022450 con 0x55e2a7424cb0 2024-09-11T22:37:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.201+0000 7f7eb00c1740 1 -- 172.21.15.5:0/700727104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e90005500 con 0x55e2a7424cb0 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.201+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/700727104 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8673+0+0 (secure 0 0 0) 0x7f7ea00225f0 con 0x55e2a7424cb0 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.201+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/700727104 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7ea00227f0 con 0x55e2a7424cb0 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.201+0000 7f7eb00c1740 10 client.29207 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.201+0000 7f7eb00c1740 1 -- 172.21.15.5:0/700727104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e2a715d620 con 0x55e2a7424cb0 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.201+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/700727104 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e90005500 con 0x55e2a7424cb0 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.202+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/700727104 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f7ea001f040 con 0x55e2a7424cb0 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.202+0000 7f7eb00c1740 10 client.29207 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.202+0000 7f7eb00c1740 10 client.29207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.202+0000 7f7eb00c1740 20 client.29207 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.202+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e2a7434cc0 0x55e2a7437180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.202+0000 7f7eb00c1740 4 client.29207 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.202+0000 7f7eb00c1740 1 -- 172.21.15.5:0/700727104 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a738ba20 con 0x55e2a7434cc0 2024-09-11T22:37:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.202+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e2a7434cc0 0x55e2a7437180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.202+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e2a7434cc0 0x55e2a7437180 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a9de62b1e7faf3 server_cookie=1a1cdb3150820d3f in_seq=0 out_seq=0 2024-09-11T22:37:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.202+0000 7f7e8e7fc640 10 client.29207 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/700727104 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2a738ba20 con 0x55e2a7434cc0 2024-09-11T22:37:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7e8e7fc640 10 client.29207 handle_command_reply: tid=0 2024-09-11T22:37:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 client.29207 shutdown 2024-09-11T22:37:17.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 -- 172.21.15.5:0/700727104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 msgr2=0x55e2a74225e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a74225e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0005c40 tx=0x7f7ea0004a40 comp rx=0 tx=0).stop 2024-09-11T22:37:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 -- 172.21.15.5:0/700727104 shutdown_connections 2024-09-11T22:37:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e2a7434cc0 0x55e2a7437180 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a74225e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a741a410 0x55e2a74220a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/700727104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 0x55e2a7421b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7e8e7fc640 0 client.29207 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 -- 172.21.15.5:0/700727104 >> 172.21.15.5:0/700727104 conn(0x55e2a7416060 msgr2=0x55e2a741bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 -- 172.21.15.5:0/700727104 shutdown_connections 2024-09-11T22:37:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 1 -- 172.21.15.5:0/700727104 wait complete. 2024-09-11T22:37:17.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.206+0000 7f7eb00c1740 20 client.29207 trim_cache size 0 max 16384 2024-09-11T22:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.224+0000 7f7eb00c1740 1 Processor -- start 2024-09-11T22:37:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.224+0000 7f7eb00c1740 1 -- start start 2024-09-11T22:37:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a741c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a741a410 0x55e2a7420410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a7422410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea81afc80 con 0x55e2a7419af0 2024-09-11T22:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea81b18b0 con 0x55e2a7424cb0 2024-09-11T22:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea81b0350 con 0x55e2a741a410 2024-09-11T22:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a741c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a741c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38124/0 (socket says 172.21.15.5:38124) 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/970524549 learned_addr learned my addr 172.21.15.5:0/970524549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7e9ffff640 1 --2- 172.21.15.5:0/970524549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a741a410 0x55e2a7420410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7ea77fe640 1 --2- 172.21.15.5:0/970524549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a7422410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/970524549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a741a410 msgr2=0x55e2a7420410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/970524549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a741a410 0x55e2a7420410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/970524549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 msgr2=0x55e2a7422410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/970524549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a7422410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/970524549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a741a240 con 0x55e2a7419af0 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/970524549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a741c560 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f7e980987c0 tx=0x7f7e98097f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd841ab761df75b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.225+0000 7f7e9cff9640 1 -- 172.21.15.5:0/970524549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980d5040 con 0x55e2a7419af0 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.226+0000 7f7e9cff9640 1 -- 172.21.15.5:0/970524549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e980a00f0 con 0x55e2a7419af0 2024-09-11T22:37:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.226+0000 7f7e9cff9640 1 -- 172.21.15.5:0/970524549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e980cd1f0 con 0x55e2a7419af0 2024-09-11T22:37:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.226+0000 7f7eb00c1740 1 -- 172.21.15.5:0/970524549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 msgr2=0x55e2a741c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.226+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/970524549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a741c560 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f7e980987c0 tx=0x7f7e98097f70 comp rx=0 tx=0).stop 2024-09-11T22:37:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 -- 172.21.15.5:0/970524549 shutdown_connections 2024-09-11T22:37:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/970524549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a7424cb0 0x55e2a7422410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/970524549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a741a410 0x55e2a7420410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/970524549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7419af0 0x55e2a741c560 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 -- 172.21.15.5:0/970524549 >> 172.21.15.5:0/970524549 conn(0x55e2a7416060 msgr2=0x55e2a7423be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 -- 172.21.15.5:0/970524549 shutdown_connections 2024-09-11T22:37:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 -- 172.21.15.5:0/970524549 wait complete. 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 -- start start 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 Processor -- start 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 0x55e2a742d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a741a410 0x55e2a74284d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7424cb0 0x55e2a7428a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea81b11e0 con 0x55e2a7424cb0 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea81b0a50 con 0x55e2a741a410 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea81adc80 con 0x55e2a7419af0 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7eb00c1740 10 client.0 _ll_register_callbacks cb 0x7f7ea0013620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7424cb0 0x55e2a7428a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 0x55e2a742d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 0x55e2a742d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:43526/0 (socket says 172.21.15.5:43526) 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/501814906 learned_addr learned my addr 172.21.15.5:0/501814906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7ea77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7424cb0 0x55e2a7428a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38138/0 (socket says 172.21.15.5:38138) 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.227+0000 7f7e9ffff640 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a741a410 0x55e2a74284d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/501814906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a741a410 msgr2=0x55e2a74284d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a741a410 0x55e2a74284d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/501814906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7424cb0 msgr2=0x55e2a7428a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7424cb0 0x55e2a7428a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7ea6ffd640 1 -- 172.21.15.5:0/501814906 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a7428fd0 con 0x55e2a7419af0 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7ea77fe640 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7424cb0 0x55e2a7428a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7ea6ffd640 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 0x55e2a742d160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7e980cd020 tx=0x7f7e980d50b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11ea4f039429f58a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7e8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/501814906 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98097b40 con 0x55e2a7419af0 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/501814906 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8673+0+0 (secure 0 0 0) 0x7f7e980cd1f0 con 0x55e2a7419af0 2024-09-11T22:37:17.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7eb00c1740 1 -- 172.21.15.5:0/501814906 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e980a00f0 con 0x55e2a7419af0 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/501814906 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f7e980cd3f0 con 0x55e2a7419af0 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7eb00c1740 10 client.29053 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7eb00c1740 1 -- 172.21.15.5:0/501814906 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e2a749f7e0 con 0x55e2a7419af0 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.228+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/501814906 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e980a00f0 con 0x55e2a7419af0 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.229+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/501814906 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f7e980ba570 con 0x55e2a7419af0 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.229+0000 7f7eb00c1740 10 client.29053 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.229+0000 7f7eb00c1740 10 client.29053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.229+0000 7f7eb00c1740 20 client.29053 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.229+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e2a749ff60 0x55e2a74a2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.229+0000 7f7eb00c1740 4 client.29053 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.229+0000 7f7eb00c1740 1 -- 172.21.15.5:0/501814906 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2a7385d70 con 0x55e2a749ff60 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.229+0000 7f7e9ffff640 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e2a749ff60 0x55e2a74a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:17.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.229+0000 7f7e9ffff640 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e2a749ff60 0x55e2a74a2380 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=915b3869b23eaba3 server_cookie=bf232ab53f080a37 in_seq=0 out_seq=0 2024-09-11T22:37:17.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.229+0000 7f7e8e7fc640 10 client.29053 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:17.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:17 smithi005 ceph-mon[30943]: pgmap v480: 97 pgs: 97 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-09-11T22:37:17.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:17 smithi159 ceph-mon[35534]: pgmap v480: 97 pgs: 97 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-09-11T22:37:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.579+0000 7f7e8e7fc640 1 -- 172.21.15.5:0/501814906 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2a7385d70 con 0x55e2a749ff60 2024-09-11T22:37:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.579+0000 7f7e8e7fc640 10 client.29053 handle_command_reply: tid=0 2024-09-11T22:37:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.579+0000 7f7eb00c1740 1 client.29053 shutdown 2024-09-11T22:37:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 1 -- 172.21.15.5:0/501814906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 msgr2=0x55e2a742d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 0x55e2a742d160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7e980cd020 tx=0x7f7e980d50b0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 1 -- 172.21.15.5:0/501814906 shutdown_connections 2024-09-11T22:37:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e2a749ff60 0x55e2a74a2380 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2a7424cb0 0x55e2a7428a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7e8e7fc640 0 client.29053 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e2a741a410 0x55e2a74284d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 1 --2- 172.21.15.5:0/501814906 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e2a7419af0 0x55e2a742d160 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 1 -- 172.21.15.5:0/501814906 >> 172.21.15.5:0/501814906 conn(0x55e2a7416060 msgr2=0x55e2a74149a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 1 -- 172.21.15.5:0/501814906 shutdown_connections 2024-09-11T22:37:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 1 -- 172.21.15.5:0/501814906 wait complete. 2024-09-11T22:37:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.580+0000 7f7eb00c1740 20 client.29053 trim_cache size 0 max 16384 2024-09-11T22:37:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:37:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:37:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:37:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:37:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.581+0000 7f7ead32c640 1 -- 172.21.15.5:0/3666085082 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e940a2da0 msgr2=0x7f7e940a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.581+0000 7f7ead32c640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e940a2da0 0x7f7e940a5260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00049d0 tx=0x7f7ea0004ab0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.581+0000 7f7ead32c640 1 -- 172.21.15.5:0/3666085082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8103f40 msgr2=0x7f7ea819a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.581+0000 7f7ead32c640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8103f40 0x7f7ea819a580 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800e9f0 tx=0x7f7e9800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.582+0000 7f7ead32c640 1 -- 172.21.15.5:0/3666085082 shutdown_connections 2024-09-11T22:37:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.582+0000 7f7ead32c640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7e940a2da0 0x7f7e940a5260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.582+0000 7f7ead32c640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ea819ee40 0x7f7ea819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.582+0000 7f7ead32c640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ea8109090 0x7f7ea819aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.582+0000 7f7ead32c640 1 --2- 172.21.15.5:0/3666085082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8103f40 0x7f7ea819a580 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.582+0000 7f7ead32c640 1 -- 172.21.15.5:0/3666085082 >> 172.21.15.5:0/3666085082 conn(0x7f7ea80ff9b0 msgr2=0x7f7ea8100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.582+0000 7f7ead32c640 1 -- 172.21.15.5:0/3666085082 shutdown_connections 2024-09-11T22:37:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:17.582+0000 7f7ead32c640 1 -- 172.21.15.5:0/3666085082 wait complete. 2024-09-11T22:37:17.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:17 smithi172 ceph-mon[35461]: pgmap v480: 97 pgs: 97 active+clean; 354 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.7 MiB/s wr, 240 op/s 2024-09-11T22:37:17.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12647 9.07 MB/sec warmup 95 sec latency 263.211 ms 2024-09-11T22:37:18.637 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12974 9.08 MB/sec warmup 96 sec latency 219.999 ms 2024-09-11T22:37:19.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:19 smithi005 ceph-mon[30943]: pgmap v481: 97 pgs: 97 active+clean; 367 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-09-11T22:37:19.580 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:19 smithi159 ceph-mon[35534]: pgmap v481: 97 pgs: 97 active+clean; 367 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-09-11T22:37:19.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:19 smithi172 ceph-mon[35461]: pgmap v481: 97 pgs: 97 active+clean; 367 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-09-11T22:37:19.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 13592 9.34 MB/sec warmup 97 sec latency 179.919 ms 2024-09-11T22:37:20.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14255 9.53 MB/sec warmup 98 sec latency 170.854 ms 2024-09-11T22:37:21.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:21 smithi005 ceph-mon[30943]: pgmap v482: 97 pgs: 97 active+clean; 367 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-09-11T22:37:21.579 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:21 smithi159 ceph-mon[35534]: pgmap v482: 97 pgs: 97 active+clean; 367 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-09-11T22:37:21.610 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:21 smithi172 ceph-mon[35461]: pgmap v482: 97 pgs: 97 active+clean; 367 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-09-11T22:37:21.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14964 9.78 MB/sec warmup 99 sec latency 176.896 ms 2024-09-11T22:37:22.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15650 10.21 MB/sec warmup 100 sec latency 183.826 ms 2024-09-11T22:37:23.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16106 10.42 MB/sec warmup 101 sec latency 65.950 ms 2024-09-11T22:37:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:23 smithi005 ceph-mon[30943]: pgmap v483: 97 pgs: 97 active+clean; 368 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 495 op/s 2024-09-11T22:37:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:23 smithi005 ceph-mon[30943]: osdmap e111: 12 total, 12 up, 12 in 2024-09-11T22:37:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:23 smithi159 ceph-mon[35534]: pgmap v483: 97 pgs: 97 active+clean; 368 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 495 op/s 2024-09-11T22:37:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:23 smithi159 ceph-mon[35534]: osdmap e111: 12 total, 12 up, 12 in 2024-09-11T22:37:23.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:23 smithi172 ceph-mon[35461]: pgmap v483: 97 pgs: 97 active+clean; 368 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 495 op/s 2024-09-11T22:37:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:23 smithi172 ceph-mon[35461]: osdmap e111: 12 total, 12 up, 12 in 2024-09-11T22:37:24.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16398 10.46 MB/sec warmup 102 sec latency 376.356 ms 2024-09-11T22:37:25.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16755 10.57 MB/sec warmup 103 sec latency 170.393 ms 2024-09-11T22:37:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:25 smithi005 ceph-mon[30943]: pgmap v485: 97 pgs: 97 active+clean; 368 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 499 op/s 2024-09-11T22:37:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:25 smithi159 ceph-mon[35534]: pgmap v485: 97 pgs: 97 active+clean; 368 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 499 op/s 2024-09-11T22:37:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:25 smithi172 ceph-mon[35461]: pgmap v485: 97 pgs: 97 active+clean; 368 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 499 op/s 2024-09-11T22:37:26.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17035 10.53 MB/sec warmup 104 sec latency 204.651 ms 2024-09-11T22:37:27.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17532 10.61 MB/sec warmup 105 sec latency 205.503 ms 2024-09-11T22:37:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:27 smithi005 ceph-mon[30943]: pgmap v486: 97 pgs: 97 active+clean; 368 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 495 op/s 2024-09-11T22:37:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:27 smithi159 ceph-mon[35534]: pgmap v486: 97 pgs: 97 active+clean; 368 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 495 op/s 2024-09-11T22:37:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:27 smithi172 ceph-mon[35461]: pgmap v486: 97 pgs: 97 active+clean; 368 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 495 op/s 2024-09-11T22:37:28.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18204 10.85 MB/sec warmup 106 sec latency 152.792 ms 2024-09-11T22:37:29.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:37:29.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18832 11.05 MB/sec warmup 107 sec latency 126.768 ms 2024-09-11T22:37:29.692 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:37:29.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:29 smithi005 ceph-mon[30943]: pgmap v487: 97 pgs: 97 active+clean; 392 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.9 MiB/s wr, 514 op/s 2024-09-11T22:37:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:29 smithi159 ceph-mon[35534]: pgmap v487: 97 pgs: 97 active+clean; 392 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.9 MiB/s wr, 514 op/s 2024-09-11T22:37:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:29 smithi172 ceph-mon[35461]: pgmap v487: 97 pgs: 97 active+clean; 392 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.9 MiB/s wr, 514 op/s 2024-09-11T22:37:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.548+0000 7f9acd09b640 1 -- 172.21.15.5:0/1381041566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ac8100e20 msgr2=0x7f9ac8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.548+0000 7f9acd09b640 1 --2- 172.21.15.5:0/1381041566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ac8100e20 0x7f9ac8101200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009890 tx=0x7f9ab802a3f0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.549+0000 7f9acd09b640 1 -- 172.21.15.5:0/1381041566 shutdown_connections 2024-09-11T22:37:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.549+0000 7f9acd09b640 1 --2- 172.21.15.5:0/1381041566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ac8102ef0 0x7f9ac810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.549+0000 7f9acd09b640 1 --2- 172.21.15.5:0/1381041566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102570 0x7f9ac81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.549+0000 7f9acd09b640 1 --2- 172.21.15.5:0/1381041566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ac8100e20 0x7f9ac8101200 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.549+0000 7f9acd09b640 1 -- 172.21.15.5:0/1381041566 >> 172.21.15.5:0/1381041566 conn(0x7f9ac80fc7d0 msgr2=0x7f9ac80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.549+0000 7f9acd09b640 1 -- 172.21.15.5:0/1381041566 shutdown_connections 2024-09-11T22:37:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.549+0000 7f9acd09b640 1 -- 172.21.15.5:0/1381041566 wait complete. 2024-09-11T22:37:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.549+0000 7f9acd09b640 1 Processor -- start 2024-09-11T22:37:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9acd09b640 1 -- start start 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9acd09b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ac8100e20 0x7f9ac819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9acd09b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ac8102570 0x7f9ac819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9acd09b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102ef0 0x7f9ac819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9acd09b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac810fe90 con 0x7f9ac8102570 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9acd09b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac810fd10 con 0x7f9ac8100e20 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9acd09b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac8110010 con 0x7f9ac8102ef0 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ac8102570 0x7f9ac819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac6d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ac8100e20 0x7f9ac819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ac8102570 0x7f9ac819b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42998/0 (socket says 172.21.15.5:42998) 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac659c640 1 -- 172.21.15.5:0/3873333576 learned_addr learned my addr 172.21.15.5:0/3873333576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac6d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ac8100e20 0x7f9ac819b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33582/0 (socket says 172.21.15.5:33582) 2024-09-11T22:37:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac759e640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102ef0 0x7f9ac819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac6d9d640 1 -- 172.21.15.5:0/3873333576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102ef0 msgr2=0x7f9ac819f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102ef0 0x7f9ac819f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac6d9d640 1 -- 172.21.15.5:0/3873333576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ac8102570 msgr2=0x7f9ac819b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ac8102570 0x7f9ac819b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.550+0000 7f9ac6d9d640 1 -- 172.21.15.5:0/3873333576 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ac819f840 con 0x7f9ac8100e20 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.551+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ac8100e20 0x7f9ac819b1c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009780 tx=0x7f9ab8030430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.551+0000 7f9aaffff640 1 -- 172.21.15.5:0/3873333576 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab803f020 con 0x7f9ac8100e20 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.551+0000 7f9aaffff640 1 -- 172.21.15.5:0/3873333576 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab8004400 con 0x7f9ac8100e20 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.551+0000 7f9acd09b640 1 -- 172.21.15.5:0/3873333576 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ac819fad0 con 0x7f9ac8100e20 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.551+0000 7f9aaffff640 1 -- 172.21.15.5:0/3873333576 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8037040 con 0x7f9ac8100e20 2024-09-11T22:37:30.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.551+0000 7f9acd09b640 1 -- 172.21.15.5:0/3873333576 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ac81a73d0 con 0x7f9ac8100e20 2024-09-11T22:37:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.554+0000 7f9acf474740 1 Processor -- start 2024-09-11T22:37:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.554+0000 7f9acf474740 1 -- start start 2024-09-11T22:37:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.554+0000 7f9aaffff640 1 -- 172.21.15.5:0/3873333576 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9ab8042020 con 0x7f9ac8100e20 2024-09-11T22:37:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9acf474740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 0x5593de49d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9acf474740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4a8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9acf474740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 0x5593de4aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9acf474740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593de4032e0 con 0x5593de49d4a0 2024-09-11T22:37:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9acf474740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5593de3aace0 con 0x5593de49ddc0 2024-09-11T22:37:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9acf474740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5593de3ff2a0 con 0x5593de4a8680 2024-09-11T22:37:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9ac759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 0x5593de49d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9ac759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 0x5593de49d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43010/0 (socket says 172.21.15.5:43010) 2024-09-11T22:37:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9ac759e640 1 -- 172.21.15.5:0/2279975895 learned_addr learned my addr 172.21.15.5:0/2279975895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9ac659c640 1 --2- 172.21.15.5:0/2279975895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 0x5593de4aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9aaffff640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ab00a2d70 0x7f9ab00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9ac759e640 1 -- 172.21.15.5:0/2279975895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 msgr2=0x5593de4aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9ac759e640 1 --2- 172.21.15.5:0/2279975895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 0x5593de4aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9ac759e640 1 -- 172.21.15.5:0/2279975895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 msgr2=0x5593de4a8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9ac759e640 1 --2- 172.21.15.5:0/2279975895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9ac759e640 1 -- 172.21.15.5:0/2279975895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593de38f440 con 0x5593de49d4a0 2024-09-11T22:37:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9aaffff640 1 -- 172.21.15.5:0/3873333576 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f9ab80ef280 con 0x7f9ac8100e20 2024-09-11T22:37:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.555+0000 7f9ac759e640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ab00a2d70 0x7f9ab00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.556+0000 7f9ac759e640 1 --2- 172.21.15.5:0/2279975895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 0x5593de49d880 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00099f0 tx=0x7f9ac00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c85cd76b7270e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.556+0000 7f9aad7fa640 1 -- 172.21.15.5:0/2279975895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000d7e0 con 0x5593de49d4a0 2024-09-11T22:37:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.556+0000 7f9aad7fa640 1 -- 172.21.15.5:0/2279975895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac0018450 con 0x5593de49d4a0 2024-09-11T22:37:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.556+0000 7f9aad7fa640 1 -- 172.21.15.5:0/2279975895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000de60 con 0x5593de49d4a0 2024-09-11T22:37:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.556+0000 7f9ac759e640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ab00a2d70 0x7f9ab00a5230 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0020000 tx=0x7f9ac0021000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.557+0000 7f9acf474740 1 -- 172.21.15.5:0/2279975895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 msgr2=0x5593de49d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.557+0000 7f9acf474740 1 --2- 172.21.15.5:0/2279975895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 0x5593de49d880 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00099f0 tx=0x7f9ac00095c0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.557+0000 7f9acf474740 1 -- 172.21.15.5:0/2279975895 shutdown_connections 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.557+0000 7f9acf474740 1 --2- 172.21.15.5:0/2279975895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 0x5593de4aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.557+0000 7f9acf474740 1 --2- 172.21.15.5:0/2279975895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4a8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.557+0000 7f9acf474740 1 --2- 172.21.15.5:0/2279975895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 0x5593de49d880 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.557+0000 7f9acf474740 1 -- 172.21.15.5:0/2279975895 >> 172.21.15.5:0/2279975895 conn(0x5593de499a30 msgr2=0x5593de499e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.557+0000 7f9acf474740 1 -- 172.21.15.5:0/2279975895 shutdown_connections 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.557+0000 7f9acf474740 1 -- 172.21.15.5:0/2279975895 wait complete. 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.557+0000 7f9acf474740 1 -- start start 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9acf474740 1 Processor -- start 2024-09-11T22:37:30.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9acf474740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49d4a0 0x5593de4a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9acf474740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49ddc0 0x5593de4a5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9acf474740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 0x5593de4a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9acf474740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5593de4ab250 con 0x5593de4a8680 2024-09-11T22:37:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9acf474740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5593de3be050 con 0x5593de49d4a0 2024-09-11T22:37:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9acf474740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5593de4ab3d0 con 0x5593de49ddc0 2024-09-11T22:37:30.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9acf474740 10 client.0 _ll_register_callbacks cb 0x5593de3fe160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:30.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9ac759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 0x5593de4a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9ac759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 0x5593de4a57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43012/0 (socket says 172.21.15.5:43012) 2024-09-11T22:37:30.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9ac759e640 1 -- 172.21.15.5:0/2635335253 learned_addr learned my addr 172.21.15.5:0/2635335253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:30.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.558+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49d4a0 0x5593de4a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9ac659c640 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49ddc0 0x5593de4a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9ac759e640 1 -- 172.21.15.5:0/2635335253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49ddc0 msgr2=0x5593de4a5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9ac759e640 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49ddc0 0x5593de4a5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9ac759e640 1 -- 172.21.15.5:0/2635335253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49d4a0 msgr2=0x5593de4a7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9ac759e640 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49d4a0 0x5593de4a7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9ac759e640 1 -- 172.21.15.5:0/2635335253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593de1743c0 con 0x5593de4a8680 2024-09-11T22:37:30.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49d4a0 0x5593de4a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:30.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9ac759e640 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 0x5593de4a57e0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0018d90 tx=0x7f9ac0003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65d0158ee964ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9aaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:30.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9aaaff5640 1 -- 172.21.15.5:0/2635335253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0028040 con 0x5593de4a8680 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9aaaff5640 1 -- 172.21.15.5:0/2635335253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f9ac0004b90 con 0x5593de4a8680 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.559+0000 7f9acf474740 1 -- 172.21.15.5:0/2635335253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ac0018450 con 0x5593de4a8680 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.560+0000 7f9aaaff5640 1 -- 172.21.15.5:0/2635335253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9ac0004d90 con 0x5593de4a8680 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.560+0000 7f9acf474740 10 client.20145 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.560+0000 7f9acf474740 1 -- 172.21.15.5:0/2635335253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5593de4bf490 con 0x5593de4a8680 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.561+0000 7f9aaaff5640 1 -- 172.21.15.5:0/2635335253 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac0018450 con 0x5593de4a8680 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.561+0000 7f9aaaff5640 1 -- 172.21.15.5:0/2635335253 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f9ac000b040 con 0x5593de4a8680 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.561+0000 7f9acf474740 10 client.20145 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.561+0000 7f9acf474740 10 client.20145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.561+0000 7f9acf474740 20 client.20145 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:30.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.561+0000 7f9acf474740 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593de4c0950 0x5593de4c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.561+0000 7f9acf474740 4 client.20145 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:30.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.561+0000 7f9acf474740 1 -- 172.21.15.5:0/2635335253 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593de410850 con 0x5593de4c0950 2024-09-11T22:37:30.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.564+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593de4c0950 0x5593de4c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.564+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593de4c0950 0x5593de4c2d40 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21e9b36064939b0 server_cookie=508affa96902fcb6 in_seq=0 out_seq=0 2024-09-11T22:37:30.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.564+0000 7f9aaaff5640 10 client.20145 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:30.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.566+0000 7f9aaaff5640 1 -- 172.21.15.5:0/2635335253 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593de410850 con 0x5593de4c0950 2024-09-11T22:37:30.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.566+0000 7f9aaaff5640 10 client.20145 handle_command_reply: tid=0 2024-09-11T22:37:30.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.566+0000 7f9acf474740 1 client.20145 shutdown 2024-09-11T22:37:30.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 1 -- 172.21.15.5:0/2635335253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 msgr2=0x5593de4a57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 0x5593de4a57e0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0018d90 tx=0x7f9ac0003bf0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 1 -- 172.21.15.5:0/2635335253 shutdown_connections 2024-09-11T22:37:30.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593de4c0950 0x5593de4c2d40 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 0x5593de4a57e0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49ddc0 0x5593de4a5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 1 --2- 172.21.15.5:0/2635335253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49d4a0 0x5593de4a7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 1 -- 172.21.15.5:0/2635335253 >> 172.21.15.5:0/2635335253 conn(0x5593de499a30 msgr2=0x5593de49f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:30.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9aaaff5640 0 client.20145 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:30.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 1 -- 172.21.15.5:0/2635335253 shutdown_connections 2024-09-11T22:37:30.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 1 -- 172.21.15.5:0/2635335253 wait complete. 2024-09-11T22:37:30.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.567+0000 7f9acf474740 20 client.20145 trim_cache size 0 max 16384 2024-09-11T22:37:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.591+0000 7f9acf474740 1 Processor -- start 2024-09-11T22:37:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.591+0000 7f9acf474740 1 -- start start 2024-09-11T22:37:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9acf474740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49d4a0 0x5593de4bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9acf474740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9acf474740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 0x5593de4a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9acf474740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81aba30 con 0x5593de4a8680 2024-09-11T22:37:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9acf474740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81ad660 con 0x5593de49ddc0 2024-09-11T22:37:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9acf474740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81ac100 con 0x5593de49d4a0 2024-09-11T22:37:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9ac6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49d4a0 0x5593de4bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9ac659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9ac6d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49d4a0 0x5593de4bcda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46774/0 (socket says 172.21.15.5:46774) 2024-09-11T22:37:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9ac6d9d640 1 -- 172.21.15.5:0/813769997 learned_addr learned my addr 172.21.15.5:0/813769997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9ac6d9d640 1 -- 172.21.15.5:0/813769997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 msgr2=0x5593de4c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/813769997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9ac6d9d640 1 -- 172.21.15.5:0/813769997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 msgr2=0x5593de4a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/813769997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 0x5593de4a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9ac6d9d640 1 -- 172.21.15.5:0/813769997 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593de4a4230 con 0x5593de49d4a0 2024-09-11T22:37:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.592+0000 7f9ac659c640 1 --2- 172.21.15.5:0/813769997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.593+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/813769997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49d4a0 0x5593de4bcda0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8033fd0 tx=0x7f9ab8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf01bf57bb6e7a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.593+0000 7f9aacff9640 1 -- 172.21.15.5:0/813769997 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80bbf40 con 0x5593de49d4a0 2024-09-11T22:37:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.593+0000 7f9aacff9640 1 -- 172.21.15.5:0/813769997 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab80bb840 con 0x5593de49d4a0 2024-09-11T22:37:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.593+0000 7f9aacff9640 1 -- 172.21.15.5:0/813769997 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80a60b0 con 0x5593de49d4a0 2024-09-11T22:37:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.594+0000 7f9acf474740 1 -- 172.21.15.5:0/813769997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49d4a0 msgr2=0x5593de4bcda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.594+0000 7f9acf474740 1 --2- 172.21.15.5:0/813769997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49d4a0 0x5593de4bcda0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8033fd0 tx=0x7f9ab8005f90 comp rx=0 tx=0).stop 2024-09-11T22:37:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.594+0000 7f9acf474740 1 -- 172.21.15.5:0/813769997 shutdown_connections 2024-09-11T22:37:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.594+0000 7f9acf474740 1 --2- 172.21.15.5:0/813769997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de4a8680 0x5593de4a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.594+0000 7f9acf474740 1 --2- 172.21.15.5:0/813769997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.594+0000 7f9acf474740 1 --2- 172.21.15.5:0/813769997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de49d4a0 0x5593de4bcda0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.594+0000 7f9acf474740 1 -- 172.21.15.5:0/813769997 >> 172.21.15.5:0/813769997 conn(0x5593de499a30 msgr2=0x5593de4a00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.594+0000 7f9acf474740 1 -- 172.21.15.5:0/813769997 shutdown_connections 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.594+0000 7f9acf474740 1 -- 172.21.15.5:0/813769997 wait complete. 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.595+0000 7f9acf474740 1 -- start start 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.595+0000 7f9acf474740 1 Processor -- start 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.595+0000 7f9acf474740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 0x5593de4b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.595+0000 7f9acf474740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4b9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.595+0000 7f9acf474740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 0x5593de4b4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.595+0000 7f9acf474740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81acf90 con 0x5593de49d4a0 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.595+0000 7f9acf474740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81ac800 con 0x5593de49ddc0 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.595+0000 7f9acf474740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ac81a9a70 con 0x5593de4a8680 2024-09-11T22:37:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.595+0000 7f9acf474740 10 client.0 _ll_register_callbacks cb 0x7f9ac0009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 0x5593de4b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4b9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33616/0 (socket says 172.21.15.5:33616) 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac659c640 1 -- 172.21.15.5:0/1093112687 learned_addr learned my addr 172.21.15.5:0/1093112687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 0x5593de4b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac659c640 1 -- 172.21.15.5:0/1093112687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 msgr2=0x5593de4b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac659c640 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 0x5593de4b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac659c640 1 -- 172.21.15.5:0/1093112687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 msgr2=0x5593de4b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac659c640 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 0x5593de4b8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac659c640 1 -- 172.21.15.5:0/1093112687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593de4a8340 con 0x5593de49ddc0 2024-09-11T22:37:30.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac759e640 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 0x5593de4b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:30.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9ac659c640 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4b9430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5593de49adb0 tx=0x7f9abc010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3c7505895bddc64 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.596+0000 7f9aaaff5640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:37:30.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.597+0000 7f9aaaff5640 1 -- 172.21.15.5:0/1093112687 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00fd00 con 0x5593de49ddc0 2024-09-11T22:37:30.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.597+0000 7f9aaaff5640 1 -- 172.21.15.5:0/1093112687 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f9abc005020 con 0x5593de49ddc0 2024-09-11T22:37:30.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.597+0000 7f9acf474740 1 -- 172.21.15.5:0/1093112687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab80bb840 con 0x5593de49ddc0 2024-09-11T22:37:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.597+0000 7f9aaaff5640 1 -- 172.21.15.5:0/1093112687 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f9abc00fea0 con 0x5593de49ddc0 2024-09-11T22:37:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.598+0000 7f9acf474740 10 client.29228 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.598+0000 7f9acf474740 1 -- 172.21.15.5:0/1093112687 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5593de52ace0 con 0x5593de49ddc0 2024-09-11T22:37:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.598+0000 7f9aaaff5640 1 -- 172.21.15.5:0/1093112687 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab80bb840 con 0x5593de49ddc0 2024-09-11T22:37:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.598+0000 7f9aaaff5640 1 -- 172.21.15.5:0/1093112687 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f9abc01d410 con 0x5593de49ddc0 2024-09-11T22:37:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.598+0000 7f9acf474740 10 client.29228 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:30.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.599+0000 7f9acf474740 10 client.29228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.599+0000 7f9acf474740 20 client.29228 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.599+0000 7f9acf474740 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593de52b5d0 0x5593de52d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.599+0000 7f9acf474740 4 client.29228 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:37:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.599+0000 7f9acf474740 1 -- 172.21.15.5:0/1093112687 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593de40ad60 con 0x5593de52b5d0 2024-09-11T22:37:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.599+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593de52b5d0 0x5593de52d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.600+0000 7f9ac6d9d640 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593de52b5d0 0x5593de52d9f0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9736f264d6c83c server_cookie=3a0e459c15bfb1f1 in_seq=0 out_seq=0 2024-09-11T22:37:30.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.600+0000 7f9aaaff5640 10 client.29228 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:30.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9aaaff5640 1 -- 172.21.15.5:0/1093112687 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5593de40ad60 con 0x5593de52b5d0 2024-09-11T22:37:30.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9aaaff5640 10 client.29228 handle_command_reply: tid=0 2024-09-11T22:37:30.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9acf474740 1 client.29228 shutdown 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9acf474740 1 -- 172.21.15.5:0/1093112687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 msgr2=0x5593de4b9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9acf474740 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4b9430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5593de49adb0 tx=0x7f9abc010c10 comp rx=0 tx=0).stop 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9acf474740 1 -- 172.21.15.5:0/1093112687 shutdown_connections 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9acf474740 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5593de52b5d0 0x5593de52d9f0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9acf474740 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5593de4a8680 0x5593de4b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9aaaff5640 0 client.29228 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9acf474740 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5593de49ddc0 0x5593de4b9430 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.603+0000 7f9acf474740 1 --2- 172.21.15.5:0/1093112687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593de49d4a0 0x5593de4b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.604+0000 7f9acf474740 1 -- 172.21.15.5:0/1093112687 >> 172.21.15.5:0/1093112687 conn(0x5593de499a30 msgr2=0x5593de4a7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.604+0000 7f9acf474740 1 -- 172.21.15.5:0/1093112687 shutdown_connections 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.604+0000 7f9acf474740 1 -- 172.21.15.5:0/1093112687 wait complete. 2024-09-11T22:37:30.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.604+0000 7f9acf474740 20 client.29228 trim_cache size 0 max 16384 2024-09-11T22:37:30.608 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:37:30.608 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:37:30.608 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:37:30.608 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:37:30.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.604+0000 7f9acd09b640 1 -- 172.21.15.5:0/3873333576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ab00a2d70 msgr2=0x7f9ab00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.604+0000 7f9acd09b640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ab00a2d70 0x7f9ab00a5230 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0020000 tx=0x7f9ac0021000 comp rx=0 tx=0).stop 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.604+0000 7f9acd09b640 1 -- 172.21.15.5:0/3873333576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ac8100e20 msgr2=0x7f9ac819b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.605+0000 7f9acd09b640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ac8100e20 0x7f9ac819b1c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009780 tx=0x7f9ab8030430 comp rx=0 tx=0).stop 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.605+0000 7f9acd09b640 1 -- 172.21.15.5:0/3873333576 shutdown_connections 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.605+0000 7f9acd09b640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ac8102ef0 0x7f9ac819f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.605+0000 7f9acd09b640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ac8102570 0x7f9ac819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.605+0000 7f9acd09b640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9ab00a2d70 0x7f9ab00a5230 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.605+0000 7f9acd09b640 1 --2- 172.21.15.5:0/3873333576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ac8100e20 0x7f9ac819b1c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.605+0000 7f9acd09b640 1 -- 172.21.15.5:0/3873333576 >> 172.21.15.5:0/3873333576 conn(0x7f9ac80fc7d0 msgr2=0x7f9ac80fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.605+0000 7f9acd09b640 1 -- 172.21.15.5:0/3873333576 shutdown_connections 2024-09-11T22:37:30.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:30.605+0000 7f9acd09b640 1 -- 172.21.15.5:0/3873333576 wait complete. 2024-09-11T22:37:30.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 19544 11.35 MB/sec warmup 108 sec latency 129.197 ms 2024-09-11T22:37:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.691+0000 7eff9841e640 1 -- 172.21.15.5:0/1306255335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff9010c0f0 msgr2=0x7eff9010e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.691+0000 7eff9841e640 1 --2- 172.21.15.5:0/1306255335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff9010c0f0 0x7eff9010e560 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7eff88009ca0 tx=0x7eff8802a950 comp rx=0 tx=0).stop 2024-09-11T22:37:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.691+0000 7eff9841e640 1 -- 172.21.15.5:0/1306255335 shutdown_connections 2024-09-11T22:37:30.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.691+0000 7eff9841e640 1 --2- 172.21.15.5:0/1306255335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff9010c0f0 0x7eff9010e560 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.691+0000 7eff9841e640 1 --2- 172.21.15.5:0/1306255335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff90103640 0x7eff9010bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.691+0000 7eff9841e640 1 --2- 172.21.15.5:0/1306255335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff90102d20 0x7eff90103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.691+0000 7eff9841e640 1 -- 172.21.15.5:0/1306255335 >> 172.21.15.5:0/1306255335 conn(0x7eff90077990 msgr2=0x7eff900fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.691+0000 7eff9841e640 1 -- 172.21.15.5:0/1306255335 shutdown_connections 2024-09-11T22:37:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.691+0000 7eff9841e640 1 -- 172.21.15.5:0/1306255335 wait complete. 2024-09-11T22:37:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff9841e640 1 Processor -- start 2024-09-11T22:37:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff9841e640 1 -- start start 2024-09-11T22:37:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff9841e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff90102d20 0x7eff90198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff9841e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff90103640 0x7eff90198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff9841e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff9010c0f0 0x7eff9019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff9841e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff90111ea0 con 0x7eff9010c0f0 2024-09-11T22:37:30.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff9841e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7eff90111d20 con 0x7eff90103640 2024-09-11T22:37:30.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff9841e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7eff90112020 con 0x7eff90102d20 2024-09-11T22:37:30.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff969bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff9010c0f0 0x7eff9019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff969bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff9010c0f0 0x7eff9019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43056/0 (socket says 172.21.15.5:43056) 2024-09-11T22:37:30.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff969bb640 1 -- 172.21.15.5:0/1011911828 learned_addr learned my addr 172.21.15.5:0/1011911828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:30.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff959b9640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff90103640 0x7eff90198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff961ba640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff90102d20 0x7eff90198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff969bb640 1 -- 172.21.15.5:0/1011911828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff90102d20 msgr2=0x7eff90198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff969bb640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff90102d20 0x7eff90198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff969bb640 1 -- 172.21.15.5:0/1011911828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff90103640 msgr2=0x7eff90198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff969bb640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff90103640 0x7eff90198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.692+0000 7eff969bb640 1 -- 172.21.15.5:0/1011911828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9019d340 con 0x7eff9010c0f0 2024-09-11T22:37:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.693+0000 7eff969bb640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff9010c0f0 0x7eff9019cc10 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7eff88004af0 tx=0x7eff8802f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.693+0000 7eff877fe640 1 -- 172.21.15.5:0/1011911828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88034040 con 0x7eff9010c0f0 2024-09-11T22:37:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.693+0000 7eff9841e640 1 -- 172.21.15.5:0/1011911828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff9019d5d0 con 0x7eff9010c0f0 2024-09-11T22:37:30.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.693+0000 7eff877fe640 1 -- 172.21.15.5:0/1011911828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff88030920 con 0x7eff9010c0f0 2024-09-11T22:37:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.693+0000 7eff877fe640 1 -- 172.21.15.5:0/1011911828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88004670 con 0x7eff9010c0f0 2024-09-11T22:37:30.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.693+0000 7eff9841e640 1 -- 172.21.15.5:0/1011911828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff901a4ec0 con 0x7eff9010c0f0 2024-09-11T22:37:30.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.696+0000 7eff9adfd740 1 Processor -- start 2024-09-11T22:37:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.696+0000 7eff877fe640 1 -- 172.21.15.5:0/1011911828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7eff88037020 con 0x7eff9010c0f0 2024-09-11T22:37:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.696+0000 7eff9adfd740 1 -- start start 2024-09-11T22:37:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d6164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d616f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d61720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff9adfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a0d60c92f0 con 0x55a0d6165410 2024-09-11T22:37:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff9adfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a0d6071270 con 0x55a0d6164af0 2024-09-11T22:37:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff9adfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0d60c52b0 con 0x55a0d616fcb0 2024-09-11T22:37:30.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff959b9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d6164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff959b9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d6164ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33640/0 (socket says 172.21.15.5:33640) 2024-09-11T22:37:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff959b9640 1 -- 172.21.15.5:0/3905590452 learned_addr learned my addr 172.21.15.5:0/3905590452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff961ba640 1 --2- 172.21.15.5:0/3905590452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d616f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff959b9640 1 --2- 172.21.15.5:0/3905590452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d61720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff959b9640 1 -- 172.21.15.5:0/3905590452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 msgr2=0x55a0d61720a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff959b9640 1 --2- 172.21.15.5:0/3905590452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d61720a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff959b9640 1 -- 172.21.15.5:0/3905590452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 msgr2=0x55a0d616f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff959b9640 1 --2- 172.21.15.5:0/3905590452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d616f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff877fe640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7eff7c0a2e20 0x7eff7c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff959b9640 1 -- 172.21.15.5:0/3905590452 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d6070340 con 0x55a0d6164af0 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff969bb640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7eff7c0a2e20 0x7eff7c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.697+0000 7eff959b9640 1 --2- 172.21.15.5:0/3905590452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d6164ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7eff800058c0 tx=0x7eff80005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a80952aa58019cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.698+0000 7eff877fe640 1 -- 172.21.15.5:0/1011911828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7eff880f0080 con 0x7eff9010c0f0 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.698+0000 7eff84bfe640 1 -- 172.21.15.5:0/3905590452 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8000f7c0 con 0x55a0d6164af0 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.698+0000 7eff84bfe640 1 -- 172.21.15.5:0/3905590452 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff8000fe40 con 0x55a0d6164af0 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.698+0000 7eff84bfe640 1 -- 172.21.15.5:0/3905590452 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8000ee40 con 0x55a0d6164af0 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.699+0000 7eff969bb640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7eff7c0a2e20 0x7eff7c0a52e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff880edd60 tx=0x7eff8804e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.699+0000 7eff9adfd740 1 -- 172.21.15.5:0/3905590452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 msgr2=0x55a0d6164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.699+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3905590452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d6164ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7eff800058c0 tx=0x7eff80005850 comp rx=0 tx=0).stop 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.699+0000 7eff9adfd740 1 -- 172.21.15.5:0/3905590452 shutdown_connections 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.699+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3905590452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d61720a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.699+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3905590452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d616f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.699+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3905590452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d6164ed0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.699+0000 7eff9adfd740 1 -- 172.21.15.5:0/3905590452 >> 172.21.15.5:0/3905590452 conn(0x55a0d6161060 msgr2=0x55a0d6161440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.699+0000 7eff9adfd740 1 -- 172.21.15.5:0/3905590452 shutdown_connections 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.699+0000 7eff9adfd740 1 -- 172.21.15.5:0/3905590452 wait complete. 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.700+0000 7eff9adfd740 1 -- start start 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.700+0000 7eff9adfd740 1 Processor -- start 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.700+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d616cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.700+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d616d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.700+0000 7eff959b9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d616cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.700+0000 7eff959b9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d616cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33642/0 (socket says 172.21.15.5:33642) 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.700+0000 7eff959b9640 1 -- 172.21.15.5:0/3293481666 learned_addr learned my addr 172.21.15.5:0/3293481666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:30.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.700+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d616d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.700+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a0d6172880 con 0x55a0d6165410 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.700+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a0d6052290 con 0x55a0d6164af0 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a0d6172a00 con 0x55a0d616fcb0 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff9adfd740 10 client.0 _ll_register_callbacks cb 0x55a0d6083140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff961ba640 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d616d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff959b9640 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d616d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff959b9640 1 -- 172.21.15.5:0/3293481666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 msgr2=0x55a0d616d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff959b9640 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d616d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff959b9640 1 -- 172.21.15.5:0/3293481666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 msgr2=0x55a0d616d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff959b9640 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d616d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff959b9640 1 -- 172.21.15.5:0/3293481666 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d5ddc210 con 0x55a0d6164af0 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff961ba640 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d616d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff959b9640 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d616cc70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7eff80009c90 tx=0x7eff8000ea20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=948fcda807d0ce7e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff79ffb640 1 -- 172.21.15.5:0/3293481666 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80028030 con 0x55a0d6164af0 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff8000fe40 con 0x55a0d6164af0 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.701+0000 7eff79ffb640 1 -- 172.21.15.5:0/3293481666 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7eff80019040 con 0x55a0d6164af0 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff79ffb640 1 -- 172.21.15.5:0/3293481666 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7eff80034050 con 0x55a0d6164af0 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff9adfd740 10 client.29240 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a0d5ea8620 con 0x55a0d6164af0 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff79ffb640 1 -- 172.21.15.5:0/3293481666 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff8000fe40 con 0x55a0d6164af0 2024-09-11T22:37:30.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff79ffb640 1 -- 172.21.15.5:0/3293481666 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7eff8001d020 con 0x55a0d6164af0 2024-09-11T22:37:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff9adfd740 10 client.29240 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff9adfd740 10 client.29240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff9adfd740 20 client.29240 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0d6183f70 0x55a0d6186360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff9adfd740 4 client.29240 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0d60d6a20 con 0x55a0d6183f70 2024-09-11T22:37:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.702+0000 7eff961ba640 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0d6183f70 0x55a0d6186360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.703+0000 7eff79ffb640 10 client.29240 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:30.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.703+0000 7eff961ba640 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0d6183f70 0x55a0d6186360 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdd111effaa062fb server_cookie=dbf781113c9ab5ae in_seq=0 out_seq=0 2024-09-11T22:37:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.705+0000 7eff79ffb640 1 -- 172.21.15.5:0/3293481666 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0d60d6a20 con 0x55a0d6183f70 2024-09-11T22:37:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.705+0000 7eff79ffb640 10 client.29240 handle_command_reply: tid=0 2024-09-11T22:37:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.705+0000 7eff9adfd740 1 client.29240 shutdown 2024-09-11T22:37:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.705+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 msgr2=0x55a0d616cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.705+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d616cc70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7eff80009c90 tx=0x7eff8000ea20 comp rx=0 tx=0).stop 2024-09-11T22:37:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.705+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 shutdown_connections 2024-09-11T22:37:30.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.705+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0d6183f70 0x55a0d6186360 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.705+0000 7eff79ffb640 0 client.29240 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.705+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d616d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.706+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d616d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.706+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3293481666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6164af0 0x55a0d616cc70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.706+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 >> 172.21.15.5:0/3293481666 conn(0x55a0d6161060 msgr2=0x55a0d6166dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.706+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 shutdown_connections 2024-09-11T22:37:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.706+0000 7eff9adfd740 1 -- 172.21.15.5:0/3293481666 wait complete. 2024-09-11T22:37:30.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.706+0000 7eff9adfd740 20 client.29240 trim_cache size 0 max 16384 2024-09-11T22:37:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.724+0000 7eff9adfd740 1 Processor -- start 2024-09-11T22:37:30.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.724+0000 7eff9adfd740 1 -- start start 2024-09-11T22:37:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.724+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6164af0 0x55a0d61609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.724+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6165410 0x55a0d616d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.724+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d618bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.724+0000 7eff9adfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff901ab710 con 0x55a0d6164af0 2024-09-11T22:37:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.724+0000 7eff9adfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7eff901ad340 con 0x55a0d6165410 2024-09-11T22:37:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.724+0000 7eff9adfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7eff901abde0 con 0x55a0d616fcb0 2024-09-11T22:37:30.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff959b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6164af0 0x55a0d61609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff959b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6164af0 0x55a0d61609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43088/0 (socket says 172.21.15.5:43088) 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff959b9640 1 -- 172.21.15.5:0/1411120238 learned_addr learned my addr 172.21.15.5:0/1411120238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff961ba640 1 --2- 172.21.15.5:0/1411120238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6165410 0x55a0d616d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff959b9640 1 --2- 172.21.15.5:0/1411120238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d618bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff959b9640 1 -- 172.21.15.5:0/1411120238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 msgr2=0x55a0d618bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff959b9640 1 --2- 172.21.15.5:0/1411120238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d618bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff959b9640 1 -- 172.21.15.5:0/1411120238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6165410 msgr2=0x55a0d616d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff959b9640 1 --2- 172.21.15.5:0/1411120238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6165410 0x55a0d616d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff959b9640 1 -- 172.21.15.5:0/1411120238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d6161cf0 con 0x55a0d6164af0 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff961ba640 1 --2- 172.21.15.5:0/1411120238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6165410 0x55a0d616d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:30.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff959b9640 1 --2- 172.21.15.5:0/1411120238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6164af0 0x55a0d61609b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7eff80005730 tx=0x7eff80005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12d0ab04381320b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff7bfff640 1 -- 172.21.15.5:0/1411120238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80028070 con 0x55a0d6164af0 2024-09-11T22:37:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff7bfff640 1 -- 172.21.15.5:0/1411120238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff80004520 con 0x55a0d6164af0 2024-09-11T22:37:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.725+0000 7eff7bfff640 1 -- 172.21.15.5:0/1411120238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80011320 con 0x55a0d6164af0 2024-09-11T22:37:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.726+0000 7eff9adfd740 1 -- 172.21.15.5:0/1411120238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6164af0 msgr2=0x55a0d61609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:30.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.726+0000 7eff9adfd740 1 --2- 172.21.15.5:0/1411120238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6164af0 0x55a0d61609b0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7eff80005730 tx=0x7eff80005f10 comp rx=0 tx=0).stop 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.726+0000 7eff9adfd740 1 -- 172.21.15.5:0/1411120238 shutdown_connections 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.726+0000 7eff9adfd740 1 --2- 172.21.15.5:0/1411120238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d616fcb0 0x55a0d618bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.726+0000 7eff9adfd740 1 --2- 172.21.15.5:0/1411120238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d6165410 0x55a0d616d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.726+0000 7eff9adfd740 1 --2- 172.21.15.5:0/1411120238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6164af0 0x55a0d61609b0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.726+0000 7eff9adfd740 1 -- 172.21.15.5:0/1411120238 >> 172.21.15.5:0/1411120238 conn(0x55a0d6161060 msgr2=0x55a0d616ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.726+0000 7eff9adfd740 1 -- 172.21.15.5:0/1411120238 shutdown_connections 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.726+0000 7eff9adfd740 1 -- 172.21.15.5:0/1411120238 wait complete. 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.726+0000 7eff9adfd740 1 -- start start 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff9adfd740 1 Processor -- start 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d6164af0 0x55a0d61782b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d6173620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff9adfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d616fcb0 0x55a0d6173b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff9adfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff901acc70 con 0x55a0d6165410 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff9adfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7eff901ac4e0 con 0x55a0d616fcb0 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff9adfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7eff901a9730 con 0x55a0d6164af0 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff9adfd740 10 client.0 _ll_register_callbacks cb 0x7eff8c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff961ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d6173620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff961ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d6173620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43098/0 (socket says 172.21.15.5:43098) 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff961ba640 1 -- 172.21.15.5:0/3844271168 learned_addr learned my addr 172.21.15.5:0/3844271168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff959b9640 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d6164af0 0x55a0d61782b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff959b9640 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d616fcb0 0x55a0d6173b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff961ba640 1 -- 172.21.15.5:0/3844271168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d6164af0 msgr2=0x55a0d61782b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff961ba640 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d6164af0 0x55a0d61782b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff961ba640 1 -- 172.21.15.5:0/3844271168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d616fcb0 msgr2=0x55a0d6173b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:30.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff961ba640 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d616fcb0 0x55a0d6173b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff961ba640 1 -- 172.21.15.5:0/3844271168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d6174200 con 0x55a0d6165410 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff961ba640 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d6173620 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x55a0d616e2b0 tx=0x7eff8c0109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c87fa16545e7c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.727+0000 7eff79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff79ffb640 1 -- 172.21.15.5:0/3844271168 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c0035f0 con 0x55a0d6165410 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff79ffb640 1 -- 172.21.15.5:0/3844271168 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7eff8c003790 con 0x55a0d6165410 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff9adfd740 1 -- 172.21.15.5:0/3844271168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff80004520 con 0x55a0d6165410 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff79ffb640 1 -- 172.21.15.5:0/3844271168 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7eff8c003990 con 0x55a0d6165410 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff9adfd740 10 client.20166 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff9adfd740 1 -- 172.21.15.5:0/3844271168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55a0d61ee9d0 con 0x55a0d6165410 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff79ffb640 1 -- 172.21.15.5:0/3844271168 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff80004520 con 0x55a0d6165410 2024-09-11T22:37:30.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff79ffb640 1 -- 172.21.15.5:0/3844271168 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7eff8c02e3d0 con 0x55a0d6165410 2024-09-11T22:37:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff9adfd740 10 client.20166 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff9adfd740 10 client.20166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff9adfd740 20 client.20166 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0d6183f70 0x55a0d61f1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff9adfd740 4 client.20166 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:37:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.728+0000 7eff9adfd740 1 -- 172.21.15.5:0/3844271168 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0d60d0d70 con 0x55a0d6183f70 2024-09-11T22:37:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.729+0000 7eff959b9640 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0d6183f70 0x55a0d61f1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.729+0000 7eff79ffb640 10 client.20166 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:30.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:30.729+0000 7eff959b9640 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0d6183f70 0x55a0d61f1280 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f1bfc2a9022b00f server_cookie=5cd24fc5f39ec70b in_seq=0 out_seq=0 2024-09-11T22:37:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.191+0000 7eff79ffb640 1 -- 172.21.15.5:0/3844271168 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0d60d0d70 con 0x55a0d6183f70 2024-09-11T22:37:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.191+0000 7eff79ffb640 10 client.20166 handle_command_reply: tid=0 2024-09-11T22:37:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.191+0000 7eff9adfd740 1 client.20166 shutdown 2024-09-11T22:37:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 1 -- 172.21.15.5:0/3844271168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 msgr2=0x55a0d6173620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d6173620 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x55a0d616e2b0 tx=0x7eff8c0109b0 comp rx=0 tx=0).stop 2024-09-11T22:37:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 1 -- 172.21.15.5:0/3844271168 shutdown_connections 2024-09-11T22:37:31.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a0d6183f70 0x55a0d61f1280 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a0d616fcb0 0x55a0d6173b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a0d6165410 0x55a0d6173620 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff79ffb640 0 client.20166 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 1 --2- 172.21.15.5:0/3844271168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a0d6164af0 0x55a0d61782b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 1 -- 172.21.15.5:0/3844271168 >> 172.21.15.5:0/3844271168 conn(0x55a0d6161060 msgr2=0x55a0d617ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 1 -- 172.21.15.5:0/3844271168 shutdown_connections 2024-09-11T22:37:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 1 -- 172.21.15.5:0/3844271168 wait complete. 2024-09-11T22:37:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9adfd740 20 client.20166 trim_cache size 0 max 16384 2024-09-11T22:37:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:37:31.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:37:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:37:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:37:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9841e640 1 -- 172.21.15.5:0/1011911828 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7eff7c0a2e20 msgr2=0x7eff7c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9841e640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7eff7c0a2e20 0x7eff7c0a52e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff880edd60 tx=0x7eff8804e000 comp rx=0 tx=0).stop 2024-09-11T22:37:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9841e640 1 -- 172.21.15.5:0/1011911828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff9010c0f0 msgr2=0x7eff9019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.192+0000 7eff9841e640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff9010c0f0 0x7eff9019cc10 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7eff88004af0 tx=0x7eff8802f3a0 comp rx=0 tx=0).stop 2024-09-11T22:37:31.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.193+0000 7eff9841e640 1 -- 172.21.15.5:0/1011911828 shutdown_connections 2024-09-11T22:37:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.193+0000 7eff9841e640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff9010c0f0 0x7eff9019cc10 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.193+0000 7eff9841e640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7eff7c0a2e20 0x7eff7c0a52e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.193+0000 7eff9841e640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff90103640 0x7eff90198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.193+0000 7eff9841e640 1 --2- 172.21.15.5:0/1011911828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff90102d20 0x7eff90198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.193+0000 7eff9841e640 1 -- 172.21.15.5:0/1011911828 >> 172.21.15.5:0/1011911828 conn(0x7eff90077990 msgr2=0x7eff900fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.193+0000 7eff9841e640 1 -- 172.21.15.5:0/1011911828 shutdown_connections 2024-09-11T22:37:31.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:31.193+0000 7eff9841e640 1 -- 172.21.15.5:0/1011911828 wait complete. 2024-09-11T22:37:31.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 19880 11.46 MB/sec warmup 109 sec latency 241.683 ms 2024-09-11T22:37:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:31 smithi005 ceph-mon[30943]: pgmap v488: 97 pgs: 97 active+clean; 392 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.9 MiB/s wr, 514 op/s 2024-09-11T22:37:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:31 smithi159 ceph-mon[35534]: pgmap v488: 97 pgs: 97 active+clean; 392 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.9 MiB/s wr, 514 op/s 2024-09-11T22:37:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:31 smithi172 ceph-mon[35461]: pgmap v488: 97 pgs: 97 active+clean; 392 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.9 MiB/s wr, 514 op/s 2024-09-11T22:37:32.124 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:37:32.125 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:37:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:37:32.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 20375 11.65 MB/sec warmup 110 sec latency 163.162 ms 2024-09-11T22:37:33.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.140+0000 7f60bba3b640 1 -- 172.21.15.5:0/3449457579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60b4102790 msgr2=0x7f60b41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:33.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.140+0000 7f60bba3b640 1 --2- 172.21.15.5:0/3449457579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60b4102790 0x7f60b41067e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009900 tx=0x7f60a402a410 comp rx=0 tx=0).stop 2024-09-11T22:37:33.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.140+0000 7f60bba3b640 1 -- 172.21.15.5:0/3449457579 shutdown_connections 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.140+0000 7f60bba3b640 1 --2- 172.21.15.5:0/3449457579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60b4106f10 0x7f60b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.140+0000 7f60bba3b640 1 --2- 172.21.15.5:0/3449457579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60b4102790 0x7f60b41067e0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.140+0000 7f60bba3b640 1 --2- 172.21.15.5:0/3449457579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b4101de0 0x7f60b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.140+0000 7f60bba3b640 1 -- 172.21.15.5:0/3449457579 >> 172.21.15.5:0/3449457579 conn(0x7f60b40fd8c0 msgr2=0x7f60b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.141+0000 7f60bba3b640 1 -- 172.21.15.5:0/3449457579 shutdown_connections 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.141+0000 7f60bba3b640 1 -- 172.21.15.5:0/3449457579 wait complete. 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.141+0000 7f60bba3b640 1 Processor -- start 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.141+0000 7f60bba3b640 1 -- start start 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60bba3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60b4101de0 0x7f60b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60bba3b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60b4102790 0x7f60b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60bba3b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b4106f10 0x7f60b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60bba3b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4111f70 con 0x7f60b4101de0 2024-09-11T22:37:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60bba3b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f60b4111df0 con 0x7f60b4106f10 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60bba3b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f60b41120f0 con 0x7f60b4102790 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60b9fd8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b4106f10 0x7f60b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60b97d7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60b4101de0 0x7f60b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60b9fd8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b4106f10 0x7f60b4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46666/0 (socket says 172.21.15.5:46666) 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60b9fd8640 1 -- 172.21.15.5:0/1862262691 learned_addr learned my addr 172.21.15.5:0/1862262691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.142+0000 7f60b8fd6640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60b4102790 0x7f60b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60b9fd8640 1 -- 172.21.15.5:0/1862262691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60b4102790 msgr2=0x7f60b4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60b9fd8640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60b4102790 0x7f60b4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60b9fd8640 1 -- 172.21.15.5:0/1862262691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60b4101de0 msgr2=0x7f60b40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60b9fd8640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60b4101de0 0x7f60b40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60b9fd8640 1 -- 172.21.15.5:0/1862262691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60b41a6900 con 0x7f60b4106f10 2024-09-11T22:37:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60b97d7640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60b4101de0 0x7f60b40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60b9fd8640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b4106f10 0x7f60b4074f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00eb10 tx=0x7f60ac00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60aa7fc640 1 -- 172.21.15.5:0/1862262691 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac004b20 con 0x7f60b4106f10 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60aa7fc640 1 -- 172.21.15.5:0/1862262691 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60ac004cc0 con 0x7f60b4106f10 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60bba3b640 1 -- 172.21.15.5:0/1862262691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60b41a6bf0 con 0x7f60b4106f10 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.143+0000 7f60aa7fc640 1 -- 172.21.15.5:0/1862262691 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac010760 con 0x7f60b4106f10 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.144+0000 7f60bba3b640 1 -- 172.21.15.5:0/1862262691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60b41a7150 con 0x7f60b4106f10 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.145+0000 7f60bba3b640 1 -- 172.21.15.5:0/1862262691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60b41021c0 con 0x7f60b4106f10 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.147+0000 7f60aa7fc640 1 -- 172.21.15.5:0/1862262691 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f60ac010900 con 0x7f60b4106f10 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.148+0000 7f60aa7fc640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60a00a2d70 0x7f60a00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.148+0000 7f60b97d7640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60a00a2d70 0x7f60a00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.148+0000 7f60aa7fc640 1 -- 172.21.15.5:0/1862262691 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f60ac0cfeb0 con 0x7f60b4106f10 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.148+0000 7f60b97d7640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60a00a2d70 0x7f60a00a5230 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009fc0 tx=0x7f60b0005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.156+0000 7f60aa7fc640 1 -- 172.21.15.5:0/1862262691 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60ac09ca00 con 0x7f60b4106f10 2024-09-11T22:37:33.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.399+0000 7f60bba3b640 1 -- 172.21.15.5:0/1862262691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60b4075a20 con 0x7f60b4106f10 2024-09-11T22:37:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.400+0000 7f60aa7fc640 1 -- 172.21.15.5:0/1862262691 <== mon.2 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11975 (secure 0 0 0) 0x7f60b4075a20 con 0x7f60b4106f10 2024-09-11T22:37:33.401 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:37:33.401 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":50,"btime":"2024-09-11T22:35:49:548013+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:35:48.842139+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:37:33.402 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 50 2024-09-11T22:37:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 -- 172.21.15.5:0/1862262691 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60a00a2d70 msgr2=0x7f60a00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60a00a2d70 0x7f60a00a5230 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009fc0 tx=0x7f60b0005f90 comp rx=0 tx=0).stop 2024-09-11T22:37:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 -- 172.21.15.5:0/1862262691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b4106f10 msgr2=0x7f60b4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b4106f10 0x7f60b4074f70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00eb10 tx=0x7f60ac00c2d0 comp rx=0 tx=0).stop 2024-09-11T22:37:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 -- 172.21.15.5:0/1862262691 shutdown_connections 2024-09-11T22:37:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f60a00a2d70 0x7f60a00a5230 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f60b4106f10 0x7f60b4074f70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60b4102790 0x7f60b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 --2- 172.21.15.5:0/1862262691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f60b4101de0 0x7f60b40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 -- 172.21.15.5:0/1862262691 >> 172.21.15.5:0/1862262691 conn(0x7f60b40fd8c0 msgr2=0x7f60b40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:33.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 -- 172.21.15.5:0/1862262691 shutdown_connections 2024-09-11T22:37:33.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:33.403+0000 7f60bba3b640 1 -- 172.21.15.5:0/1862262691 wait complete. 2024-09-11T22:37:33.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 20743 11.74 MB/sec warmup 111 sec latency 184.733 ms 2024-09-11T22:37:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:33 smithi005 ceph-mon[30943]: pgmap v489: 97 pgs: 97 active+clean; 389 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.8 MiB/s wr, 587 op/s 2024-09-11T22:37:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:33 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1862262691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:37:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:33 smithi159 ceph-mon[35534]: pgmap v489: 97 pgs: 97 active+clean; 389 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.8 MiB/s wr, 587 op/s 2024-09-11T22:37:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:33 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1862262691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:37:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:33 smithi172 ceph-mon[35461]: pgmap v489: 97 pgs: 97 active+clean; 389 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.8 MiB/s wr, 587 op/s 2024-09-11T22:37:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:33 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1862262691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:37:34.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21015 11.71 MB/sec warmup 112 sec latency 227.298 ms 2024-09-11T22:37:34.657 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:37:35.176 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:37:35.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:35 smithi005 ceph-mon[30943]: pgmap v490: 97 pgs: 97 active+clean; 389 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.1 MiB/s wr, 504 op/s 2024-09-11T22:37:35.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21035 11.61 MB/sec warmup 113 sec latency 1083.346 ms 2024-09-11T22:37:35.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.717+0000 7f49d201c640 1 -- 172.21.15.5:0/343706274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc101dd0 msgr2=0x7f49cc1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.717+0000 7f49d201c640 1 --2- 172.21.15.5:0/343706274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc101dd0 0x7f49cc1021b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009ca0 tx=0x7f49c402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.717+0000 7f49d201c640 1 -- 172.21.15.5:0/343706274 shutdown_connections 2024-09-11T22:37:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.717+0000 7f49d201c640 1 --2- 172.21.15.5:0/343706274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc106f00 0x7f49cc10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.717+0000 7f49d201c640 1 --2- 172.21.15.5:0/343706274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49cc102780 0x7f49cc1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.717+0000 7f49d201c640 1 --2- 172.21.15.5:0/343706274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc101dd0 0x7f49cc1021b0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.717+0000 7f49d201c640 1 -- 172.21.15.5:0/343706274 >> 172.21.15.5:0/343706274 conn(0x7f49cc0fd8b0 msgr2=0x7f49cc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.717+0000 7f49d201c640 1 -- 172.21.15.5:0/343706274 shutdown_connections 2024-09-11T22:37:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.717+0000 7f49d201c640 1 -- 172.21.15.5:0/343706274 wait complete. 2024-09-11T22:37:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.718+0000 7f49d201c640 1 Processor -- start 2024-09-11T22:37:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.718+0000 7f49d201c640 1 -- start start 2024-09-11T22:37:35.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.719+0000 7f49d201c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc101dd0 0x7f49cc073a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.719+0000 7f49d201c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc102780 0x7f49cc06cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.719+0000 7f49d201c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49cc106f00 0x7f49cc06d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.719+0000 7f49d201c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc111f70 con 0x7f49cc101dd0 2024-09-11T22:37:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.719+0000 7f49d201c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc111df0 con 0x7f49cc106f00 2024-09-11T22:37:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.719+0000 7f49d201c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1120f0 con 0x7f49cc102780 2024-09-11T22:37:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.719+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc101dd0 0x7f49cc073a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.719+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc101dd0 0x7f49cc073a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39956/0 (socket says 172.21.15.5:39956) 2024-09-11T22:37:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.719+0000 7f49caffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc102780 0x7f49cc06cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.719+0000 7f49cb7fe640 1 -- 172.21.15.5:0/323451955 learned_addr learned my addr 172.21.15.5:0/323451955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49caffd640 1 -- 172.21.15.5:0/323451955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49cc106f00 msgr2=0x7f49cc06d000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:37:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49caffd640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49cc106f00 0x7f49cc06d000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49caffd640 1 -- 172.21.15.5:0/323451955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc101dd0 msgr2=0x7f49cc073a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49caffd640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc101dd0 0x7f49cc073a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49caffd640 1 -- 172.21.15.5:0/323451955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc06d5f0 con 0x7f49cc102780 2024-09-11T22:37:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49caffd640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc102780 0x7f49cc06cac0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e9f0 tx=0x7f49c000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49c8ff9640 1 -- 172.21.15.5:0/323451955 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c000ccc0 con 0x7f49cc102780 2024-09-11T22:37:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49c8ff9640 1 -- 172.21.15.5:0/323451955 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c0010910 con 0x7f49cc102780 2024-09-11T22:37:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49c8ff9640 1 -- 172.21.15.5:0/323451955 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0018960 con 0x7f49cc102780 2024-09-11T22:37:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49d201c640 1 -- 172.21.15.5:0/323451955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc06d8e0 con 0x7f49cc102780 2024-09-11T22:37:35.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.720+0000 7f49d201c640 1 -- 172.21.15.5:0/323451955 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc071880 con 0x7f49cc102780 2024-09-11T22:37:35.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49d43f5740 1 Processor -- start 2024-09-11T22:37:35.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49d43f5740 1 -- start start 2024-09-11T22:37:35.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 0x5613d56e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e55d0 0x5613d56e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56f0370 0x5613d56f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49d43f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5613d564b2f0 con 0x5613d56e55d0 2024-09-11T22:37:35.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49d43f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5613d55f2cf0 con 0x5613d56f0370 2024-09-11T22:37:35.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49d43f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5613d56472b0 con 0x5613d56e6b40 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e55d0 0x5613d56e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e55d0 0x5613d56e5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39968/0 (socket says 172.21.15.5:39968) 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49c8ff9640 1 -- 172.21.15.5:0/323451955 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f49c0010ac0 con 0x7f49cc102780 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49cb7fe640 1 -- 172.21.15.5:0/2118643793 learned_addr learned my addr 172.21.15.5:0/2118643793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49cbfff640 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 0x5613d56e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49caffd640 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56f0370 0x5613d56f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.724+0000 7f49cb7fe640 1 -- 172.21.15.5:0/2118643793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 msgr2=0x5613d56e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 0x5613d56e5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49cb7fe640 1 -- 172.21.15.5:0/2118643793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56f0370 msgr2=0x5613d56f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56f0370 0x5613d56f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49cb7fe640 1 -- 172.21.15.5:0/2118643793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d55d73a0 con 0x5613d56e55d0 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49cbfff640 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 0x5613d56e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:35.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49caffd640 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56f0370 0x5613d56f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:37:35.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e55d0 0x5613d56e5a10 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f49c402f530 tx=0x7f49c4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b91b04f6556f998 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:35.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49b1ffb640 1 -- 172.21.15.5:0/2118643793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4031db0 con 0x5613d56e55d0 2024-09-11T22:37:35.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49c8ff9640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49b40a2dc0 0x7f49b40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49b1ffb640 1 -- 172.21.15.5:0/2118643793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c4036030 con 0x5613d56e55d0 2024-09-11T22:37:35.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49b1ffb640 1 -- 172.21.15.5:0/2118643793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c4004240 con 0x5613d56e55d0 2024-09-11T22:37:35.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49cbfff640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49b40a2dc0 0x7f49b40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.725+0000 7f49c8ff9640 1 -- 172.21.15.5:0/323451955 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f49c00cf8e0 con 0x7f49cc102780 2024-09-11T22:37:35.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49d43f5740 1 -- 172.21.15.5:0/2118643793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e55d0 msgr2=0x5613d56e5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:35.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49cbfff640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49b40a2dc0 0x7f49b40a5280 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009d20 tx=0x7f49bc005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49d43f5740 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e55d0 0x5613d56e5a10 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f49c402f530 tx=0x7f49c4009800 comp rx=0 tx=0).stop 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49d43f5740 1 -- 172.21.15.5:0/2118643793 shutdown_connections 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49d43f5740 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56f0370 0x5613d56f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49d43f5740 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e55d0 0x5613d56e5a10 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49d43f5740 1 --2- 172.21.15.5:0/2118643793 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 0x5613d56e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49d43f5740 1 -- 172.21.15.5:0/2118643793 >> 172.21.15.5:0/2118643793 conn(0x5613d56ae310 msgr2=0x5613d56e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49d43f5740 1 -- 172.21.15.5:0/2118643793 shutdown_connections 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49d43f5740 1 -- 172.21.15.5:0/2118643793 wait complete. 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.726+0000 7f49d43f5740 1 -- start start 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49d43f5740 1 Processor -- start 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e55d0 0x5613d56efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e6b40 0x5613d56ecf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d56ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49d43f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5613d56f2e90 con 0x5613d56f0370 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49d43f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5613d5606060 con 0x5613d56e6b40 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49d43f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5613d56f3010 con 0x5613d56e55d0 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49d43f5740 10 client.0 _ll_register_callbacks cb 0x5613d53598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d56ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49caffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e6b40 0x5613d56ecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d56ed4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39970/0 (socket says 172.21.15.5:39970) 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49cbfff640 1 -- 172.21.15.5:0/532141672 learned_addr learned my addr 172.21.15.5:0/532141672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.727+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e55d0 0x5613d56efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49caffd640 1 -- 172.21.15.5:0/532141672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e55d0 msgr2=0x5613d56efcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49caffd640 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e55d0 0x5613d56efcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49caffd640 1 -- 172.21.15.5:0/532141672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 msgr2=0x5613d56ed4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49caffd640 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d56ed4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49caffd640 1 -- 172.21.15.5:0/532141672 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d53bc3c0 con 0x5613d56e6b40 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49cbfff640 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d56ed4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e55d0 0x5613d56efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49caffd640 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e6b40 0x5613d56ecf80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f49c00ccf60 tx=0x7f49c009b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=224b5d317a79eb49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49aaffd640 1 -- 172.21.15.5:0/532141672 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00cd3f0 con 0x5613d56e6b40 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.728+0000 7f49aaffd640 1 -- 172.21.15.5:0/532141672 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f49c009b380 con 0x5613d56e6b40 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49d43f5740 1 -- 172.21.15.5:0/532141672 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49c4036030 con 0x5613d56e6b40 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49aaffd640 1 -- 172.21.15.5:0/532141672 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f49c00a49a0 con 0x5613d56e6b40 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49d43f5740 10 client.29258 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49d43f5740 1 -- 172.21.15.5:0/532141672 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5613d56fe840 con 0x5613d56e6b40 2024-09-11T22:37:35.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49aaffd640 1 -- 172.21.15.5:0/532141672 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c4036030 con 0x5613d56e6b40 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49aaffd640 1 -- 172.21.15.5:0/532141672 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f49c009abf0 con 0x5613d56e6b40 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49d43f5740 10 client.29258 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49d43f5740 10 client.29258 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49d43f5740 20 client.29258 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49d43f5740 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5613d56ffd00 0x5613d57021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49d43f5740 4 client.29258 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49d43f5740 1 -- 172.21.15.5:0/532141672 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613d5658860 con 0x5613d56ffd00 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.729+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5613d56ffd00 0x5613d57021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.730+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5613d56ffd00 0x5613d57021c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4516ba2e9a09e55 server_cookie=e363639296b27b3a in_seq=0 out_seq=0 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.730+0000 7f49aaffd640 10 client.29258 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49aaffd640 1 -- 172.21.15.5:0/532141672 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613d5658860 con 0x5613d56ffd00 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49aaffd640 10 client.29258 handle_command_reply: tid=0 2024-09-11T22:37:35.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49d43f5740 1 client.29258 shutdown 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49d43f5740 1 -- 172.21.15.5:0/532141672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e6b40 msgr2=0x5613d56ecf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49d43f5740 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e6b40 0x5613d56ecf80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f49c00ccf60 tx=0x7f49c009b750 comp rx=0 tx=0).stop 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49d43f5740 1 -- 172.21.15.5:0/532141672 shutdown_connections 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49d43f5740 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5613d56ffd00 0x5613d57021c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49d43f5740 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d56ed4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49d43f5740 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e6b40 0x5613d56ecf80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49d43f5740 1 --2- 172.21.15.5:0/532141672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e55d0 0x5613d56efcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49d43f5740 1 -- 172.21.15.5:0/532141672 >> 172.21.15.5:0/532141672 conn(0x5613d56ae310 msgr2=0x5613d56e7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.731+0000 7f49aaffd640 0 client.29258 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.732+0000 7f49d43f5740 1 -- 172.21.15.5:0/532141672 shutdown_connections 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.732+0000 7f49d43f5740 1 -- 172.21.15.5:0/532141672 wait complete. 2024-09-11T22:37:35.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.732+0000 7f49d43f5740 20 client.29258 trim_cache size 0 max 16384 2024-09-11T22:37:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49d43f5740 1 Processor -- start 2024-09-11T22:37:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49d43f5740 1 -- start start 2024-09-11T22:37:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 0x5613d56fc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e6b40 0x5613d56ec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56f0370 0x5613d5701d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49d43f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1b85e0 con 0x5613d56e6b40 2024-09-11T22:37:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49d43f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1ba210 con 0x5613d56e55d0 2024-09-11T22:37:35.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49d43f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1b8cb0 con 0x5613d56f0370 2024-09-11T22:37:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49caffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e6b40 0x5613d56ec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 0x5613d56fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 0x5613d56fc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46696/0 (socket says 172.21.15.5:46696) 2024-09-11T22:37:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49cb7fe640 1 -- 172.21.15.5:0/377227653 learned_addr learned my addr 172.21.15.5:0/377227653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:35.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.746+0000 7f49cbfff640 1 --2- 172.21.15.5:0/377227653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56f0370 0x5613d5701d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.747+0000 7f49cb7fe640 1 -- 172.21.15.5:0/377227653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56f0370 msgr2=0x5613d5701d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.747+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/377227653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56f0370 0x5613d5701d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.747+0000 7f49cb7fe640 1 -- 172.21.15.5:0/377227653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e6b40 msgr2=0x5613d56ec840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:35.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.747+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/377227653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e6b40 0x5613d56ec840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.747+0000 7f49cb7fe640 1 -- 172.21.15.5:0/377227653 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d5702250 con 0x5613d56e55d0 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.747+0000 7f49caffd640 1 --2- 172.21.15.5:0/377227653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e6b40 0x5613d56ec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.747+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/377227653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 0x5613d56fc160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49c4031900 tx=0x7f49c4031890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83210bfdd0ee839 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.747+0000 7f49b17fa640 1 -- 172.21.15.5:0/377227653 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c40319a0 con 0x5613d56e55d0 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.747+0000 7f49b17fa640 1 -- 172.21.15.5:0/377227653 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c4036070 con 0x5613d56e55d0 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.747+0000 7f49b17fa640 1 -- 172.21.15.5:0/377227653 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c40042a0 con 0x5613d56e55d0 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.748+0000 7f49d43f5740 1 -- 172.21.15.5:0/377227653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 msgr2=0x5613d56fc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.748+0000 7f49d43f5740 1 --2- 172.21.15.5:0/377227653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 0x5613d56fc160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49c4031900 tx=0x7f49c4031890 comp rx=0 tx=0).stop 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.748+0000 7f49d43f5740 1 -- 172.21.15.5:0/377227653 shutdown_connections 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.748+0000 7f49d43f5740 1 --2- 172.21.15.5:0/377227653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56f0370 0x5613d5701d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.748+0000 7f49d43f5740 1 --2- 172.21.15.5:0/377227653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56e6b40 0x5613d56ec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.748+0000 7f49d43f5740 1 --2- 172.21.15.5:0/377227653 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 0x5613d56fc160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.748+0000 7f49d43f5740 1 -- 172.21.15.5:0/377227653 >> 172.21.15.5:0/377227653 conn(0x5613d56ae310 msgr2=0x5613d56dca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.748+0000 7f49d43f5740 1 -- 172.21.15.5:0/377227653 shutdown_connections 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.748+0000 7f49d43f5740 1 -- 172.21.15.5:0/377227653 wait complete. 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.748+0000 7f49d43f5740 1 -- start start 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49d43f5740 1 Processor -- start 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 0x5613d5706a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 0x5613d5706fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49d43f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d5707500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49d43f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1b9b40 con 0x5613d56f0370 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49d43f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1b93b0 con 0x5613d56e55d0 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49d43f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1b65e0 con 0x5613d56e6b40 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49d43f5740 10 client.0 _ll_register_callbacks cb 0x7f49c0010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d5707500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d5707500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39988/0 (socket says 172.21.15.5:39988) 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 0x5613d5706a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49cbfff640 1 -- 172.21.15.5:0/1519273132 learned_addr learned my addr 172.21.15.5:0/1519273132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49caffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 0x5613d5706fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49cbfff640 1 -- 172.21.15.5:0/1519273132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 msgr2=0x5613d5706fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49cbfff640 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 0x5613d5706fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49cbfff640 1 -- 172.21.15.5:0/1519273132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 msgr2=0x5613d5706a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49cbfff640 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 0x5613d5706a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.749+0000 7f49cbfff640 1 -- 172.21.15.5:0/1519273132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d56f72d0 con 0x5613d56f0370 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49cbfff640 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d5707500 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f49bc003d20 tx=0x7f49bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2facbe89302b74 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49aaffd640 1 -- 172.21.15.5:0/1519273132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc02d060 con 0x5613d56f0370 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49aaffd640 1 -- 172.21.15.5:0/1519273132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f49bc003d50 con 0x5613d56f0370 2024-09-11T22:37:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49d43f5740 1 -- 172.21.15.5:0/1519273132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49c4036070 con 0x5613d56f0370 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49aaffd640 1 -- 172.21.15.5:0/1519273132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f49bc0044a0 con 0x5613d56f0370 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49d43f5740 10 client.20190 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49d43f5740 1 -- 172.21.15.5:0/1519273132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5613d56f8610 con 0x5613d56f0370 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49aaffd640 1 -- 172.21.15.5:0/1519273132 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c4036070 con 0x5613d56f0370 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49aaffd640 1 -- 172.21.15.5:0/1519273132 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f49bc028410 con 0x5613d56f0370 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49d43f5740 10 client.20190 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49d43f5740 10 client.20190 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49d43f5740 20 client.20190 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49d43f5740 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5613d576aa60 0x5613d576ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49d43f5740 4 client.20190 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.750+0000 7f49d43f5740 1 -- 172.21.15.5:0/1519273132 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613d5652d70 con 0x5613d576aa60 2024-09-11T22:37:35.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.751+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5613d576aa60 0x5613d576ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.751+0000 7f49cb7fe640 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5613d576aa60 0x5613d576ce50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c66cd5a68ccd51a7 server_cookie=c9c4e08891d22069 in_seq=0 out_seq=0 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.751+0000 7f49aaffd640 10 client.20190 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49aaffd640 1 -- 172.21.15.5:0/1519273132 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613d5652d70 con 0x5613d576aa60 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49aaffd640 10 client.20190 handle_command_reply: tid=0 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49d43f5740 1 client.20190 shutdown 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49d43f5740 1 -- 172.21.15.5:0/1519273132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 msgr2=0x5613d5707500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49d43f5740 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d5707500 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f49bc003d20 tx=0x7f49bc004730 comp rx=0 tx=0).stop 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49d43f5740 1 -- 172.21.15.5:0/1519273132 shutdown_connections 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49d43f5740 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5613d576aa60 0x5613d576ce50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49d43f5740 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613d56f0370 0x5613d5707500 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49d43f5740 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5613d56e6b40 0x5613d5706fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49d43f5740 1 --2- 172.21.15.5:0/1519273132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613d56e55d0 0x5613d5706a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49aaffd640 0 client.20190 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:37:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.752+0000 7f49d43f5740 1 -- 172.21.15.5:0/1519273132 >> 172.21.15.5:0/1519273132 conn(0x5613d56ae310 msgr2=0x5613d56eb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d43f5740 1 -- 172.21.15.5:0/1519273132 shutdown_connections 2024-09-11T22:37:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d43f5740 1 -- 172.21.15.5:0/1519273132 wait complete. 2024-09-11T22:37:35.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d43f5740 20 client.20190 trim_cache size 0 max 16384 2024-09-11T22:37:35.758 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:37:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 -- 172.21.15.5:0/323451955 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49b40a2dc0 msgr2=0x7f49b40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49b40a2dc0 0x7f49b40a5280 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009d20 tx=0x7f49bc005f90 comp rx=0 tx=0).stop 2024-09-11T22:37:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 -- 172.21.15.5:0/323451955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc102780 msgr2=0x7f49cc06cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc102780 0x7f49cc06cac0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e9f0 tx=0x7f49c000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 -- 172.21.15.5:0/323451955 shutdown_connections 2024-09-11T22:37:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f49b40a2dc0 0x7f49b40a5280 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49cc106f00 0x7f49cc06d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc102780 0x7f49cc06cac0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 --2- 172.21.15.5:0/323451955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49cc101dd0 0x7f49cc073a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 -- 172.21.15.5:0/323451955 >> 172.21.15.5:0/323451955 conn(0x7f49cc0fd8b0 msgr2=0x7f49cc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 -- 172.21.15.5:0/323451955 shutdown_connections 2024-09-11T22:37:35.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:35.753+0000 7f49d201c640 1 -- 172.21.15.5:0/323451955 wait complete. 2024-09-11T22:37:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:35 smithi159 ceph-mon[35534]: pgmap v490: 97 pgs: 97 active+clean; 389 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.1 MiB/s wr, 504 op/s 2024-09-11T22:37:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:35 smithi172 ceph-mon[35461]: pgmap v490: 97 pgs: 97 active+clean; 389 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.1 MiB/s wr, 504 op/s 2024-09-11T22:37:36.612 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:37:36.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21377 11.62 MB/sec warmup 114 sec latency 1168.473 ms 2024-09-11T22:37:36.970 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:37:37.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.541+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1618356400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d10102570 msgr2=0x7f5d101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.541+0000 7f5d14f6e640 1 --2- 172.21.15.5:0/1618356400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d10102570 0x7f5d101029b0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009900 tx=0x7f5cf802a480 comp rx=0 tx=0).stop 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.541+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1618356400 shutdown_connections 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.541+0000 7f5d14f6e640 1 --2- 172.21.15.5:0/1618356400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d10102ef0 0x7f5d1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.541+0000 7f5d14f6e640 1 --2- 172.21.15.5:0/1618356400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d10102570 0x7f5d101029b0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.541+0000 7f5d14f6e640 1 --2- 172.21.15.5:0/1618356400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d10100e20 0x7f5d10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.541+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1618356400 >> 172.21.15.5:0/1618356400 conn(0x7f5d100fc7d0 msgr2=0x7f5d100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.541+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1618356400 shutdown_connections 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.541+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1618356400 wait complete. 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.542+0000 7f5d14f6e640 1 Processor -- start 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.542+0000 7f5d14f6e640 1 -- start start 2024-09-11T22:37:37.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d14f6e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d10100e20 0x7f5d10199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d14f6e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d10102570 0x7f5d10199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d14f6e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d10102ef0 0x7f5d101a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d14f6e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10110f80 con 0x7f5d10102ef0 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d14f6e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10110e00 con 0x7f5d10102570 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d14f6e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10111100 con 0x7f5d10100e20 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d0e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d10100e20 0x7f5d10199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d0e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d10100e20 0x7f5d10199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:41920/0 (socket says 172.21.15.5:41920) 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d0e59c640 1 -- 172.21.15.5:0/1678426205 learned_addr learned my addr 172.21.15.5:0/1678426205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d07fff640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d10102570 0x7f5d10199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d0ed9d640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d10102ef0 0x7f5d101a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d0ed9d640 1 -- 172.21.15.5:0/1678426205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d10100e20 msgr2=0x7f5d10199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:37.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d0ed9d640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d10100e20 0x7f5d10199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d0ed9d640 1 -- 172.21.15.5:0/1678426205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d10102570 msgr2=0x7f5d10199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d0ed9d640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d10102570 0x7f5d10199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.543+0000 7f5d0ed9d640 1 -- 172.21.15.5:0/1678426205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d101a4910 con 0x7f5d10102ef0 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.544+0000 7f5d0ed9d640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d10102ef0 0x7f5d101a43d0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800da10 tx=0x7f5d0800ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.544+0000 7f5d077fe640 1 -- 172.21.15.5:0/1678426205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800bd70 con 0x7f5d10102ef0 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.544+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1678426205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d10197140 con 0x7f5d10102ef0 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.544+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1678426205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d10197600 con 0x7f5d10102ef0 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.544+0000 7f5d077fe640 1 -- 172.21.15.5:0/1678426205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d08003e50 con 0x7f5d10102ef0 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.544+0000 7f5d077fe640 1 -- 172.21.15.5:0/1678426205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d080106a0 con 0x7f5d10102ef0 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.547+0000 7f5d17347740 1 Processor -- start 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.547+0000 7f5d077fe640 1 -- 172.21.15.5:0/1678426205 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f5d08010920 con 0x7f5d10102ef0 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d17347740 1 -- start start 2024-09-11T22:37:37.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d17347740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622ddb40 0x5641622dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d17347740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622dca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d17347740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 0x5641622e9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d17347740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641622422f0 con 0x5641622dc5d0 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d17347740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5641621e9cf0 con 0x5641622ddb40 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d17347740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56416223e2b0 con 0x5641622e7370 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 0x5641622e9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622dca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622ddb40 0x5641622dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622dca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40020/0 (socket says 172.21.15.5:40020) 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.548+0000 7f5d0e59c640 1 -- 172.21.15.5:0/2981434145 learned_addr learned my addr 172.21.15.5:0/2981434145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.549+0000 7f5d077fe640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5cfc0a2df0 0x7f5cfc0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.549+0000 7f5d0ed9d640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5cfc0a2df0 0x7f5cfc0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.549+0000 7f5d0e59c640 1 -- 172.21.15.5:0/2981434145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 msgr2=0x5641622e9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.549+0000 7f5d0e59c640 1 --2- 172.21.15.5:0/2981434145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 0x5641622e9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.549+0000 7f5d0e59c640 1 -- 172.21.15.5:0/2981434145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622ddb40 msgr2=0x5641622dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.549+0000 7f5d0e59c640 1 --2- 172.21.15.5:0/2981434145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622ddb40 0x5641622dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.549+0000 7f5d0e59c640 1 -- 172.21.15.5:0/2981434145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641621ce3a0 con 0x5641622dc5d0 2024-09-11T22:37:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.549+0000 7f5d077fe640 1 -- 172.21.15.5:0/1678426205 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f5d080cf820 con 0x7f5d10102ef0 2024-09-11T22:37:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.549+0000 7f5d0e59c640 1 --2- 172.21.15.5:0/2981434145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622dca10 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000bd90 tx=0x7f5d0000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f549ae1b9dae53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.549+0000 7f5d0ed9d640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5cfc0a2df0 0x7f5cfc0a52b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5d0809b970 tx=0x7f5d0806ece0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.550+0000 7f5d04ff9640 1 -- 172.21.15.5:0/2981434145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00015070 con 0x5641622dc5d0 2024-09-11T22:37:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.550+0000 7f5d04ff9640 1 -- 172.21.15.5:0/2981434145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d000035f0 con 0x5641622dc5d0 2024-09-11T22:37:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.550+0000 7f5d04ff9640 1 -- 172.21.15.5:0/2981434145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00011680 con 0x5641622dc5d0 2024-09-11T22:37:37.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.551+0000 7f5d17347740 1 -- 172.21.15.5:0/2981434145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 msgr2=0x5641622dca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:37.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.551+0000 7f5d17347740 1 --2- 172.21.15.5:0/2981434145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622dca10 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000bd90 tx=0x7f5d0000f730 comp rx=0 tx=0).stop 2024-09-11T22:37:37.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.551+0000 7f5d17347740 1 -- 172.21.15.5:0/2981434145 shutdown_connections 2024-09-11T22:37:37.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.551+0000 7f5d17347740 1 --2- 172.21.15.5:0/2981434145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 0x5641622e9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.551+0000 7f5d17347740 1 --2- 172.21.15.5:0/2981434145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622dca10 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.551+0000 7f5d17347740 1 --2- 172.21.15.5:0/2981434145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622ddb40 0x5641622dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.551+0000 7f5d17347740 1 -- 172.21.15.5:0/2981434145 >> 172.21.15.5:0/2981434145 conn(0x5641622a5310 msgr2=0x5641622d8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:37.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.552+0000 7f5d17347740 1 -- 172.21.15.5:0/2981434145 shutdown_connections 2024-09-11T22:37:37.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.552+0000 7f5d17347740 1 -- 172.21.15.5:0/2981434145 wait complete. 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.552+0000 7f5d17347740 1 -- start start 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.552+0000 7f5d17347740 1 Processor -- start 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d17347740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622dc5d0 0x5641622e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d17347740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622ddb40 0x5641622e3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d17347740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622e7370 0x5641622e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d17347740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641622ea070 con 0x5641622ddb40 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d17347740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5641621fd060 con 0x5641622e7370 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d17347740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5641622ea1f0 con 0x5641622dc5d0 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d17347740 10 client.0 _ll_register_callbacks cb 0x564161f508c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622dc5d0 0x5641622e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622dc5d0 0x5641622e6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:41936/0 (socket says 172.21.15.5:41936) 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d07fff640 1 -- 172.21.15.5:0/3010597235 learned_addr learned my addr 172.21.15.5:0/3010597235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d0e59c640 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622ddb40 0x5641622e3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d07fff640 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622e7370 0x5641622e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d07fff640 1 -- 172.21.15.5:0/3010597235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622e7370 msgr2=0x5641622e45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d07fff640 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622e7370 0x5641622e45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d07fff640 1 -- 172.21.15.5:0/3010597235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622ddb40 msgr2=0x5641622e3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d07fff640 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622ddb40 0x5641622e3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.553+0000 7f5d07fff640 1 -- 172.21.15.5:0/3010597235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564161fb33c0 con 0x5641622dc5d0 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.554+0000 7f5d07fff640 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622dc5d0 0x5641622e6d30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009900 tx=0x7f5cf802a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e045f02c9e24fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.554+0000 7f5cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.555+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/3010597235 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8035040 con 0x5641622dc5d0 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.555+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/3010597235 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f5cf80035f0 con 0x5641622dc5d0 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.555+0000 7f5d17347740 1 -- 172.21.15.5:0/3010597235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d000035f0 con 0x5641622dc5d0 2024-09-11T22:37:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.555+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/3010597235 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f5cf80037f0 con 0x5641622dc5d0 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.555+0000 7f5d17347740 10 client.29125 fetch_fsmap learned FSMap version 50 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.555+0000 7f5d17347740 1 -- 172.21.15.5:0/3010597235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5641622fe340 con 0x5641622dc5d0 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.556+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/3010597235 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf8003e70 con 0x5641622dc5d0 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.556+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/3010597235 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 50) ==== 8382+0+0 (secure 0 0 0) 0x7f5cf803c900 con 0x5641622dc5d0 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.556+0000 7f5d17347740 10 client.29125 fetch_fsmap finished waiting for FSMap version 50 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.556+0000 7f5d17347740 10 client.29125 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.556+0000 7f5d17347740 20 client.29125 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.556+0000 7f5d17347740 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5641622ff800 0x564162301bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.556+0000 7f5d17347740 4 client.29125 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.556+0000 7f5d17347740 1 -- 172.21.15.5:0/3010597235 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5d000035f0 con 0x5641622ff800 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.556+0000 7f5d0e59c640 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5641622ff800 0x564162301bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.557+0000 7f5d0e59c640 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5641622ff800 0x564162301bf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=acdb85be4ceab54 server_cookie=3646664d36b3f5e0 in_seq=0 out_seq=0 2024-09-11T22:37:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.557+0000 7f5cf5ffb640 10 client.29125 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:37:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/3010597235 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5d000035f0 con 0x5641622ff800 2024-09-11T22:37:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5cf5ffb640 10 client.29125 handle_command_reply: tid=0 2024-09-11T22:37:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 client.29125 shutdown 2024-09-11T22:37:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 -- 172.21.15.5:0/3010597235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622dc5d0 msgr2=0x5641622e6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622dc5d0 0x5641622e6d30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009900 tx=0x7f5cf802a480 comp rx=0 tx=0).stop 2024-09-11T22:37:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 -- 172.21.15.5:0/3010597235 shutdown_connections 2024-09-11T22:37:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5641622ff800 0x564162301bf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622e7370 0x5641622e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622ddb40 0x5641622e3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 --2- 172.21.15.5:0/3010597235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622dc5d0 0x5641622e6d30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 -- 172.21.15.5:0/3010597235 >> 172.21.15.5:0/3010597235 conn(0x5641622a5310 msgr2=0x5641622de930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5cf5ffb640 0 client.29125 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:37:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 -- 172.21.15.5:0/3010597235 shutdown_connections 2024-09-11T22:37:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.558+0000 7f5d17347740 1 -- 172.21.15.5:0/3010597235 wait complete. 2024-09-11T22:37:37.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.559+0000 7f5d17347740 20 client.29125 trim_cache size 0 max 16384 2024-09-11T22:37:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.574+0000 7f5d17347740 1 Processor -- start 2024-09-11T22:37:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.574+0000 7f5d17347740 1 -- start start 2024-09-11T22:37:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.574+0000 7f5d17347740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622dc5d0 0x5641622fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.574+0000 7f5d17347740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622ddb40 0x5641622dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.574+0000 7f5d17347740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622e7370 0x5641622e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.574+0000 7f5d17347740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf000ea10 con 0x5641622e7370 2024-09-11T22:37:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.574+0000 7f5d17347740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf0010640 con 0x5641622dc5d0 2024-09-11T22:37:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.574+0000 7f5d17347740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf000f0e0 con 0x5641622ddb40 2024-09-11T22:37:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622e7370 0x5641622e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622e7370 0x5641622e8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40042/0 (socket says 172.21.15.5:40042) 2024-09-11T22:37:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d0e59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622ddb40 0x5641622dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d07fff640 1 -- 172.21.15.5:0/2567527041 learned_addr learned my addr 172.21.15.5:0/2567527041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d07fff640 1 --2- 172.21.15.5:0/2567527041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622dc5d0 0x5641622fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d07fff640 1 -- 172.21.15.5:0/2567527041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622ddb40 msgr2=0x5641622dbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d07fff640 1 --2- 172.21.15.5:0/2567527041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622ddb40 0x5641622dbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d07fff640 1 -- 172.21.15.5:0/2567527041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622dc5d0 msgr2=0x5641622fbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d07fff640 1 --2- 172.21.15.5:0/2567527041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622dc5d0 0x5641622fbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d07fff640 1 -- 172.21.15.5:0/2567527041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641622e9f50 con 0x5641622e7370 2024-09-11T22:37:37.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.575+0000 7f5d07fff640 1 --2- 172.21.15.5:0/2567527041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622e7370 0x5641622e8760 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5cf802a9a0 tx=0x7f5cf8009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6e93b92b8f0204 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5cf7fff640 1 -- 172.21.15.5:0/2567527041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8035040 con 0x5641622e7370 2024-09-11T22:37:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5cf7fff640 1 -- 172.21.15.5:0/2567527041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf8032e80 con 0x5641622e7370 2024-09-11T22:37:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5cf7fff640 1 -- 172.21.15.5:0/2567527041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8004750 con 0x5641622e7370 2024-09-11T22:37:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5d17347740 1 -- 172.21.15.5:0/2567527041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622e7370 msgr2=0x5641622e8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5d17347740 1 --2- 172.21.15.5:0/2567527041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622e7370 0x5641622e8760 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5cf802a9a0 tx=0x7f5cf8009830 comp rx=0 tx=0).stop 2024-09-11T22:37:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5d17347740 1 -- 172.21.15.5:0/2567527041 shutdown_connections 2024-09-11T22:37:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5d17347740 1 --2- 172.21.15.5:0/2567527041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622e7370 0x5641622e8760 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5d17347740 1 --2- 172.21.15.5:0/2567527041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622ddb40 0x5641622dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5d17347740 1 --2- 172.21.15.5:0/2567527041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622dc5d0 0x5641622fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5d17347740 1 -- 172.21.15.5:0/2567527041 >> 172.21.15.5:0/2567527041 conn(0x5641622a5310 msgr2=0x5641622df2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.576+0000 7f5d17347740 1 -- 172.21.15.5:0/2567527041 shutdown_connections 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.577+0000 7f5d17347740 1 -- 172.21.15.5:0/2567527041 wait complete. 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.577+0000 7f5d17347740 1 -- start start 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.577+0000 7f5d17347740 1 Processor -- start 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.577+0000 7f5d17347740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622f8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.577+0000 7f5d17347740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622ddb40 0x564162306b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622f8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40050/0 (socket says 172.21.15.5:40050) 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d17347740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 0x5641623070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf000ff70 con 0x5641622dc5d0 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf000f7e0 con 0x5641622ddb40 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d07fff640 1 -- 172.21.15.5:0/1605473256 learned_addr learned my addr 172.21.15.5:0/1605473256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf000ca10 con 0x5641622e7370 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d17347740 10 client.0 _ll_register_callbacks cb 0x7f5cf8004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d07fff640 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 0x5641623070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d0e59c640 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622ddb40 0x564162306b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d07fff640 1 -- 172.21.15.5:0/1605473256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622ddb40 msgr2=0x564162306b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d07fff640 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622ddb40 0x564162306b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d07fff640 1 -- 172.21.15.5:0/1605473256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 msgr2=0x5641622f8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d07fff640 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622f8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d07fff640 1 -- 172.21.15.5:0/1605473256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564162307670 con 0x5641622e7370 2024-09-11T22:37:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5d07fff640 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 0x5641623070b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80059d0 tx=0x7f5cf80329d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11f7d27602558c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.578+0000 7f5cf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/1605473256 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8035070 con 0x5641622e7370 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/1605473256 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f5cf803c640 con 0x5641622e7370 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cf8032e80 con 0x5641622e7370 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/1605473256 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5cf806f050 con 0x5641622e7370 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5d17347740 10 client.29131 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5641622f6ab0 con 0x5641622e7370 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/1605473256 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf803c840 con 0x5641622e7370 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/1605473256 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f5cf804cbd0 con 0x5641622e7370 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5d17347740 10 client.29131 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5d17347740 10 client.29131 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5d17347740 20 client.29131 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5d17347740 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x56416236a8c0 0x56416236cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5d17347740 4 client.29131 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.579+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5cf8032e80 con 0x56416236a8c0 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.580+0000 7f5d0e59c640 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x56416236a8c0 0x56416236cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.580+0000 7f5d0e59c640 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x56416236a8c0 0x56416236cce0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d23f61adb5516ccd server_cookie=989a5363154af0b in_seq=0 out_seq=0 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.580+0000 7f5cf5ffb640 10 client.29131 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.580+0000 7f5cf5ffb640 1 -- 172.21.15.5:0/1605473256 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5cf8032e80 con 0x56416236a8c0 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.580+0000 7f5cf5ffb640 10 client.29131 handle_command_reply: tid=0 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.580+0000 7f5d17347740 1 client.29131 shutdown 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 msgr2=0x5641623070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 0x5641623070b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80059d0 tx=0x7f5cf80329d0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 shutdown_connections 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5641622e7370 0x5641623070b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x56416236a8c0 0x56416236cce0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5cf5ffb640 0 client.29131 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:37:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641622ddb40 0x564162306b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 1 --2- 172.21.15.5:0/1605473256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641622dc5d0 0x5641622f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 >> 172.21.15.5:0/1605473256 conn(0x5641622a5310 msgr2=0x5641622e2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 shutdown_connections 2024-09-11T22:37:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 1 -- 172.21.15.5:0/1605473256 wait complete. 2024-09-11T22:37:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.581+0000 7f5d17347740 20 client.29131 trim_cache size 0 max 16384 2024-09-11T22:37:37.587 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:37:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5cfc0a2df0 msgr2=0x7f5cfc0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5cfc0a2df0 0x7f5cfc0a52b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5d0809b970 tx=0x7f5d0806ece0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d10102ef0 msgr2=0x7f5d101a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d10102ef0 0x7f5d101a43d0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800da10 tx=0x7f5d0800ded0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1678426205 shutdown_connections 2024-09-11T22:37:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d10102ef0 0x7f5d101a43d0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5cfc0a2df0 0x7f5cfc0a52b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5d10102570 0x7f5d10199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 --2- 172.21.15.5:0/1678426205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d10100e20 0x7f5d10199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1678426205 >> 172.21.15.5:0/1678426205 conn(0x7f5d100fc7d0 msgr2=0x7f5d100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1678426205 shutdown_connections 2024-09-11T22:37:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:37.582+0000 7f5d14f6e640 1 -- 172.21.15.5:0/1678426205 wait complete. 2024-09-11T22:37:37.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21753 11.69 MB/sec warmup 115 sec latency 241.037 ms 2024-09-11T22:37:37.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:37 smithi005 ceph-mon[30943]: pgmap v491: 97 pgs: 97 active+clean; 389 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.0 MiB/s wr, 489 op/s 2024-09-11T22:37:37.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:37 smithi159 ceph-mon[35534]: pgmap v491: 97 pgs: 97 active+clean; 389 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.0 MiB/s wr, 489 op/s 2024-09-11T22:37:37.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:37 smithi172 ceph-mon[35461]: pgmap v491: 97 pgs: 97 active+clean; 389 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.0 MiB/s wr, 489 op/s 2024-09-11T22:37:38.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22297 11.78 MB/sec warmup 116 sec latency 192.332 ms 2024-09-11T22:37:38.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:38 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:37:38.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:38 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:37:38.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:38 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:37:38.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:38 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:37:38.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:38 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:37:38.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:38 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:37:38.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:38 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:37:38.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:38 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:37:38.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:38 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:37:38.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:38 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:37:38.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:38 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:37:38.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:38 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:37:38.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:38 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:37:38.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:38 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:37:38.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:38 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:37:38.871 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:37:39.205 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:37:39.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23084 12.02 MB/sec warmup 117 sec latency 77.711 ms 2024-09-11T22:37:39.653 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:39 smithi005 ceph-mon[30943]: pgmap v492: 97 pgs: 97 active+clean; 425 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 14 MiB/s wr, 855 op/s 2024-09-11T22:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.753+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/2608746912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0102570 msgr2=0x7f9eb01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.753+0000 7f9eb4cc3640 1 --2- 172.21.15.5:0/2608746912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0102570 0x7f9eb01029b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009900 tx=0x7f9e9802a480 comp rx=0 tx=0).stop 2024-09-11T22:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.753+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/2608746912 shutdown_connections 2024-09-11T22:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.753+0000 7f9eb4cc3640 1 --2- 172.21.15.5:0/2608746912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb0102ef0 0x7f9eb010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.753+0000 7f9eb4cc3640 1 --2- 172.21.15.5:0/2608746912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0102570 0x7f9eb01029b0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.753+0000 7f9eb4cc3640 1 --2- 172.21.15.5:0/2608746912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9eb0100e20 0x7f9eb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.753+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/2608746912 >> 172.21.15.5:0/2608746912 conn(0x7f9eb00fc7d0 msgr2=0x7f9eb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.753+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/2608746912 shutdown_connections 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.753+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/2608746912 wait complete. 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.754+0000 7f9eb4cc3640 1 Processor -- start 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.754+0000 7f9eb4cc3640 1 -- start start 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.754+0000 7f9eb4cc3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb0100e20 0x7f9eb019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eb4cc3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0102570 0x7f9eb019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eb4cc3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9eb0102ef0 0x7f9eb019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eb4cc3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0110f50 con 0x7f9eb0102570 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eb4cc3640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb0110dd0 con 0x7f9eb0102ef0 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eae59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb0100e20 0x7f9eb019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eb4cc3640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01110d0 con 0x7f9eb0100e20 2024-09-11T22:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eae59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb0100e20 0x7f9eb019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:41976/0 (socket says 172.21.15.5:41976) 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eae59c640 1 -- 172.21.15.5:0/4079776823 learned_addr learned my addr 172.21.15.5:0/4079776823 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9ea5d9b640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0102570 0x7f9eb019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eaed9d640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9eb0102ef0 0x7f9eb019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eae59c640 1 -- 172.21.15.5:0/4079776823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9eb0102ef0 msgr2=0x7f9eb019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eae59c640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9eb0102ef0 0x7f9eb019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eae59c640 1 -- 172.21.15.5:0/4079776823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0102570 msgr2=0x7f9eb019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eae59c640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0102570 0x7f9eb019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eae59c640 1 -- 172.21.15.5:0/4079776823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb019f700 con 0x7f9eb0100e20 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eaed9d640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9eb0102ef0 0x7f9eb019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9eae59c640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb0100e20 0x7f9eb019a750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e9f0 tx=0x7f9ea000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.755+0000 7f9ea7fff640 1 -- 172.21.15.5:0/4079776823 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000cd80 con 0x7f9eb0100e20 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.756+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/4079776823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eb019f9f0 con 0x7f9eb0100e20 2024-09-11T22:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.756+0000 7f9ea7fff640 1 -- 172.21.15.5:0/4079776823 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea0010910 con 0x7f9eb0100e20 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.756+0000 7f9ea7fff640 1 -- 172.21.15.5:0/4079776823 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00188f0 con 0x7f9eb0100e20 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.756+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/4079776823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eb01a72d0 con 0x7f9eb0100e20 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.758+0000 7f9ea7fff640 1 -- 172.21.15.5:0/4079776823 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9ea0010ac0 con 0x7f9eb0100e20 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9ea7fff640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9e9c0a2cf0 0x7f9e9c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eb709c740 1 Processor -- start 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9ea5d9b640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9e9c0a2cf0 0x7f9e9c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9ea7fff640 1 -- 172.21.15.5:0/4079776823 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f9ea00cf900 con 0x7f9eb0100e20 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eb709c740 1 -- start start 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01dab40 0x55b2f01d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01d95d0 0x55b2f01d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eb709c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2f013f2f0 con 0x55b2f01e4370 2024-09-11T22:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eb709c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b2f00e6cf0 con 0x55b2f01dab40 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eb709c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b2f013b2b0 con 0x55b2f01d95d0 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eaed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eaed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40062/0 (socket says 172.21.15.5:40062) 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eaed9d640 1 -- 172.21.15.5:0/2192916063 learned_addr learned my addr 172.21.15.5:0/2192916063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9eae59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01dab40 0x55b2f01d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.759+0000 7f9ea5d9b640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9e9c0a2cf0 0x7f9e9c0a51b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009230 tx=0x7f9e980058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.760+0000 7f9eaed9d640 1 -- 172.21.15.5:0/2192916063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01d95d0 msgr2=0x55b2f01d9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.760+0000 7f9eaed9d640 1 --2- 172.21.15.5:0/2192916063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01d95d0 0x55b2f01d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.760+0000 7f9eaed9d640 1 -- 172.21.15.5:0/2192916063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01dab40 msgr2=0x55b2f01d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.760+0000 7f9eaed9d640 1 --2- 172.21.15.5:0/2192916063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01dab40 0x55b2f01d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.760+0000 7f9eaed9d640 1 -- 172.21.15.5:0/2192916063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2f00cb3a0 con 0x55b2f01e4370 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.760+0000 7f9eaed9d640 1 --2- 172.21.15.5:0/2192916063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e6760 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009fd0 tx=0x7f9ea8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f0ee08a977cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.760+0000 7f9ea4d99640 1 -- 172.21.15.5:0/2192916063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8011700 con 0x55b2f01e4370 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.760+0000 7f9ea4d99640 1 -- 172.21.15.5:0/2192916063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea8011d80 con 0x55b2f01e4370 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.760+0000 7f9ea4d99640 1 -- 172.21.15.5:0/2192916063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea800e3c0 con 0x55b2f01e4370 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.761+0000 7f9eb709c740 1 -- 172.21.15.5:0/2192916063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 msgr2=0x55b2f01e6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.761+0000 7f9eb709c740 1 --2- 172.21.15.5:0/2192916063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e6760 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8009fd0 tx=0x7f9ea8010860 comp rx=0 tx=0).stop 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.761+0000 7f9eb709c740 1 -- 172.21.15.5:0/2192916063 shutdown_connections 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.761+0000 7f9eb709c740 1 --2- 172.21.15.5:0/2192916063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e6760 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.761+0000 7f9eb709c740 1 --2- 172.21.15.5:0/2192916063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01d95d0 0x55b2f01d9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.761+0000 7f9eb709c740 1 --2- 172.21.15.5:0/2192916063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01dab40 0x55b2f01d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.761+0000 7f9eb709c740 1 -- 172.21.15.5:0/2192916063 >> 172.21.15.5:0/2192916063 conn(0x55b2f01a2310 msgr2=0x55b2f01d5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.761+0000 7f9eb709c740 1 -- 172.21.15.5:0/2192916063 shutdown_connections 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.761+0000 7f9eb709c740 1 -- 172.21.15.5:0/2192916063 wait complete. 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.762+0000 7f9eb709c740 1 -- start start 2024-09-11T22:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.762+0000 7f9eb709c740 1 Processor -- start 2024-09-11T22:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.762+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01d95d0 0x55b2f01e3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.762+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 0x55b2f01e1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.762+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01e4370 0x55b2f01e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.762+0000 7f9eb709c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2f01e7070 con 0x55b2f01dab40 2024-09-11T22:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.762+0000 7f9eb709c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b2f00fa060 con 0x55b2f01d95d0 2024-09-11T22:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.762+0000 7f9eb709c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b2f01e71f0 con 0x55b2f01e4370 2024-09-11T22:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.762+0000 7f9ea5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 0x55b2f01e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.762+0000 7f9ea5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 0x55b2f01e1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40066/0 (socket says 172.21.15.5:40066) 2024-09-11T22:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9ea5d9b640 1 -- 172.21.15.5:0/1661562991 learned_addr learned my addr 172.21.15.5:0/1661562991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:39.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9eb709c740 10 client.0 _ll_register_callbacks cb 0x55b2efe4d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:39.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9eae59c640 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01d95d0 0x55b2f01e3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9eaed9d640 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01e4370 0x55b2f01e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9ea5d9b640 1 -- 172.21.15.5:0/1661562991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01e4370 msgr2=0x55b2f01e17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9ea5d9b640 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01e4370 0x55b2f01e17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9ea5d9b640 1 -- 172.21.15.5:0/1661562991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01d95d0 msgr2=0x55b2f01e3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9ea5d9b640 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01d95d0 0x55b2f01e3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9ea5d9b640 1 -- 172.21.15.5:0/1661562991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2efeb03c0 con 0x55b2f01dab40 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9ea5d9b640 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 0x55b2f01e1060 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9e9802a9a0 tx=0x7f9e98009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292b260ffaf3de27 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9e95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9e95ffb640 1 -- 172.21.15.5:0/1661562991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e98035050 con 0x55b2f01dab40 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.763+0000 7f9e95ffb640 1 -- 172.21.15.5:0/1661562991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f9e98031540 con 0x55b2f01dab40 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9eb709c740 1 -- 172.21.15.5:0/1661562991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea8011d80 con 0x55b2f01dab40 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9e95ffb640 1 -- 172.21.15.5:0/1661562991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9e98031740 con 0x55b2f01dab40 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9eb709c740 10 client.20220 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9eb709c740 1 -- 172.21.15.5:0/1661562991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b2f01f2b40 con 0x55b2f01dab40 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9e95ffb640 1 -- 172.21.15.5:0/1661562991 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea8011d80 con 0x55b2f01dab40 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9e95ffb640 1 -- 172.21.15.5:0/1661562991 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f9e9803caf0 con 0x55b2f01dab40 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9eb709c740 10 client.20220 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9eb709c740 10 client.20220 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9eb709c740 20 client.20220 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9eb709c740 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b2f01f41e0 0x55b2f01f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9eb709c740 4 client.20220 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.764+0000 7f9eb709c740 1 -- 172.21.15.5:0/1661562991 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2f014c860 con 0x55b2f01f41e0 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.765+0000 7f9eae59c640 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b2f01f41e0 0x55b2f01f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.765+0000 7f9e95ffb640 10 client.20220 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.765+0000 7f9eae59c640 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b2f01f41e0 0x55b2f01f65d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c665bfb96a88b41e server_cookie=f8955cc949abbfde in_seq=0 out_seq=0 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9e95ffb640 1 -- 172.21.15.5:0/1661562991 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2f014c860 con 0x55b2f01f41e0 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9e95ffb640 10 client.20220 handle_command_reply: tid=0 2024-09-11T22:37:39.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 client.20220 shutdown 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 -- 172.21.15.5:0/1661562991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 msgr2=0x55b2f01e1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 0x55b2f01e1060 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f9e9802a9a0 tx=0x7f9e98009260 comp rx=0 tx=0).stop 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 -- 172.21.15.5:0/1661562991 shutdown_connections 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b2f01f41e0 0x55b2f01f65d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01e4370 0x55b2f01e17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 0x55b2f01e1060 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9e95ffb640 0 client.20220 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 --2- 172.21.15.5:0/1661562991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01d95d0 0x55b2f01e3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 -- 172.21.15.5:0/1661562991 >> 172.21.15.5:0/1661562991 conn(0x55b2f01a2310 msgr2=0x55b2f01db960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 -- 172.21.15.5:0/1661562991 shutdown_connections 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 1 -- 172.21.15.5:0/1661562991 wait complete. 2024-09-11T22:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.766+0000 7f9eb709c740 20 client.20220 trim_cache size 0 max 16384 2024-09-11T22:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.781+0000 7f9eb709c740 1 Processor -- start 2024-09-11T22:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.781+0000 7f9eb709c740 1 -- start start 2024-09-11T22:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.781+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01d95d0 0x55b2f01e1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.781+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 0x55b2f01fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.781+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01e4370 0x55b2f01fc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.781+0000 7f9eb709c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01afcf0 con 0x55b2f01dab40 2024-09-11T22:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.781+0000 7f9eb709c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b18f0 con 0x55b2f01e4370 2024-09-11T22:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.781+0000 7f9eb709c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b03c0 con 0x55b2f01d95d0 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.781+0000 7f9ea5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 0x55b2f01fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.781+0000 7f9eaed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01e4370 0x55b2f01fc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9eaed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01e4370 0x55b2f01fc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46784/0 (socket says 172.21.15.5:46784) 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9eaed9d640 1 -- 172.21.15.5:0/3083435438 learned_addr learned my addr 172.21.15.5:0/3083435438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9eae59c640 1 --2- 172.21.15.5:0/3083435438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01d95d0 0x55b2f01e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9eae59c640 1 -- 172.21.15.5:0/3083435438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01e4370 msgr2=0x55b2f01fc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9eae59c640 1 --2- 172.21.15.5:0/3083435438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01e4370 0x55b2f01fc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9eae59c640 1 -- 172.21.15.5:0/3083435438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 msgr2=0x55b2f01fbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9eae59c640 1 --2- 172.21.15.5:0/3083435438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 0x55b2f01fbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9eae59c640 1 -- 172.21.15.5:0/3083435438 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2f01d6960 con 0x55b2f01d95d0 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9eae59c640 1 --2- 172.21.15.5:0/3083435438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01d95d0 0x55b2f01e1030 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00ccfe0 tx=0x7f9ea001a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4ade79f17cf0daa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9e97fff640 1 -- 172.21.15.5:0/3083435438 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0098420 con 0x55b2f01d95d0 2024-09-11T22:37:39.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9e97fff640 1 -- 172.21.15.5:0/3083435438 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea00cd260 con 0x55b2f01d95d0 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.782+0000 7f9e97fff640 1 -- 172.21.15.5:0/3083435438 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0097d60 con 0x55b2f01d95d0 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.783+0000 7f9eb709c740 1 -- 172.21.15.5:0/3083435438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01d95d0 msgr2=0x55b2f01e1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.783+0000 7f9eb709c740 1 --2- 172.21.15.5:0/3083435438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01d95d0 0x55b2f01e1030 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00ccfe0 tx=0x7f9ea001a040 comp rx=0 tx=0).stop 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.783+0000 7f9eb709c740 1 -- 172.21.15.5:0/3083435438 shutdown_connections 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.783+0000 7f9eb709c740 1 --2- 172.21.15.5:0/3083435438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01e4370 0x55b2f01fc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.783+0000 7f9eb709c740 1 --2- 172.21.15.5:0/3083435438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01dab40 0x55b2f01fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.783+0000 7f9eb709c740 1 --2- 172.21.15.5:0/3083435438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01d95d0 0x55b2f01e1030 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.783+0000 7f9eb709c740 1 -- 172.21.15.5:0/3083435438 >> 172.21.15.5:0/3083435438 conn(0x55b2f01a2310 msgr2=0x55b2f01e0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 1 -- 172.21.15.5:0/3083435438 shutdown_connections 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 1 -- 172.21.15.5:0/3083435438 wait complete. 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 1 -- start start 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 1 Processor -- start 2024-09-11T22:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01d95d0 0x55b2f01e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01dab40 0x55b2f01e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b1220 con 0x55b2f01e4370 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01b0a90 con 0x55b2f01d95d0 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9eb01adcf0 con 0x55b2f01dab40 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eb709c740 10 client.0 _ll_register_callbacks cb 0x7f9ea80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eaed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9ea5d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01dab40 0x55b2f01e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eaed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40074/0 (socket says 172.21.15.5:40074) 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eaed9d640 1 -- 172.21.15.5:0/2097033920 learned_addr learned my addr 172.21.15.5:0/2097033920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.784+0000 7f9eae59c640 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01d95d0 0x55b2f01e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.785+0000 7f9eaed9d640 1 -- 172.21.15.5:0/2097033920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01dab40 msgr2=0x55b2f01e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.785+0000 7f9eaed9d640 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01dab40 0x55b2f01e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.785+0000 7f9eaed9d640 1 -- 172.21.15.5:0/2097033920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01d95d0 msgr2=0x55b2f01e84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.785+0000 7f9eaed9d640 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01d95d0 0x55b2f01e84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.785+0000 7f9eaed9d640 1 -- 172.21.15.5:0/2097033920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2f025dfe0 con 0x55b2f01e4370 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.785+0000 7f9eaed9d640 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e8f60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8010d60 tx=0x7f9ea8010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f820c7fe7fde171 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.785+0000 7f9e95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.785+0000 7f9e95ffb640 1 -- 172.21.15.5:0/2097033920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea802b020 con 0x55b2f01e4370 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.785+0000 7f9e95ffb640 1 -- 172.21.15.5:0/2097033920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f9ea8011500 con 0x55b2f01e4370 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.785+0000 7f9eb709c740 1 -- 172.21.15.5:0/2097033920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ea00cd260 con 0x55b2f01e4370 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9e95ffb640 1 -- 172.21.15.5:0/2097033920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9ea8036050 con 0x55b2f01e4370 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9eb709c740 10 client.20226 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9eb709c740 1 -- 172.21.15.5:0/2097033920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55b2f025f2e0 con 0x55b2f01e4370 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9e95ffb640 1 -- 172.21.15.5:0/2097033920 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ea00cd260 con 0x55b2f01e4370 2024-09-11T22:37:39.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9e95ffb640 1 -- 172.21.15.5:0/2097033920 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f9ea80056b0 con 0x55b2f01e4370 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9eb709c740 10 client.20226 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9eb709c740 10 client.20226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9eb709c740 20 client.20226 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9eb709c740 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b2f025f730 0x55b2f0261b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9eb709c740 4 client.20226 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9eb709c740 1 -- 172.21.15.5:0/2097033920 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2f0146d70 con 0x55b2f025f730 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.786+0000 7f9eae59c640 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b2f025f730 0x55b2f0261b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9e95ffb640 10 client.20226 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eae59c640 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b2f025f730 0x55b2f0261b50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5615265234baaa3f server_cookie=4c47a4c6b608482 in_seq=0 out_seq=0 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9e95ffb640 1 -- 172.21.15.5:0/2097033920 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2f0146d70 con 0x55b2f025f730 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9e95ffb640 10 client.20226 handle_command_reply: tid=0 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 client.20226 shutdown 2024-09-11T22:37:39.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 -- 172.21.15.5:0/2097033920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 msgr2=0x55b2f01e8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e8f60 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8010d60 tx=0x7f9ea8010fd0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 -- 172.21.15.5:0/2097033920 shutdown_connections 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b2f025f730 0x55b2f0261b50 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9e95ffb640 0 client.20226 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b2f01e4370 0x55b2f01e8f60 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b2f01dab40 0x55b2f01e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 --2- 172.21.15.5:0/2097033920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b2f01d95d0 0x55b2f01e84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 -- 172.21.15.5:0/2097033920 >> 172.21.15.5:0/2097033920 conn(0x55b2f01a2310 msgr2=0x55b2f01db930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 -- 172.21.15.5:0/2097033920 shutdown_connections 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.787+0000 7f9eb709c740 1 -- 172.21.15.5:0/2097033920 wait complete. 2024-09-11T22:37:39.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb709c740 20 client.20226 trim_cache size 0 max 16384 2024-09-11T22:37:39.797 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:37:39.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/4079776823 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9e9c0a2cf0 msgr2=0x7f9e9c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:39.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb4cc3640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9e9c0a2cf0 0x7f9e9c0a51b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009230 tx=0x7f9e980058d0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/4079776823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb0100e20 msgr2=0x7f9eb019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:39.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb4cc3640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb0100e20 0x7f9eb019a750 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e9f0 tx=0x7f9ea000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/4079776823 shutdown_connections 2024-09-11T22:37:39.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb4cc3640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9e9c0a2cf0 0x7f9e9c0a51b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb4cc3640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9eb0102ef0 0x7f9eb019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb4cc3640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eb0102570 0x7f9eb019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb4cc3640 1 --2- 172.21.15.5:0/4079776823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb0100e20 0x7f9eb019a750 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:39.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.788+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/4079776823 >> 172.21.15.5:0/4079776823 conn(0x7f9eb00fc7d0 msgr2=0x7f9eb006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:39.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.789+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/4079776823 shutdown_connections 2024-09-11T22:37:39.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:39.789+0000 7f9eb4cc3640 1 -- 172.21.15.5:0/4079776823 wait complete. 2024-09-11T22:37:39.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:39 smithi159 ceph-mon[35534]: pgmap v492: 97 pgs: 97 active+clean; 425 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 14 MiB/s wr, 855 op/s 2024-09-11T22:37:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:39 smithi172 ceph-mon[35461]: pgmap v492: 97 pgs: 97 active+clean; 425 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 14 MiB/s wr, 855 op/s 2024-09-11T22:37:40.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23723 12.31 MB/sec warmup 118 sec latency 119.594 ms 2024-09-11T22:37:40.701 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:37:41.023 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:37:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.581+0000 7fb3cc500640 1 -- 172.21.15.5:0/2562446026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3c4106a80 msgr2=0x7fb3c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.581+0000 7fb3cc500640 1 --2- 172.21.15.5:0/2562446026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3c4106a80 0x7fb3c4106e60 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009930 tx=0x7fb3c002a420 comp rx=0 tx=0).stop 2024-09-11T22:37:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.581+0000 7fb3cc500640 1 -- 172.21.15.5:0/2562446026 shutdown_connections 2024-09-11T22:37:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.581+0000 7fb3cc500640 1 --2- 172.21.15.5:0/2562446026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3c4101420 0x7fb3c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.581+0000 7fb3cc500640 1 --2- 172.21.15.5:0/2562446026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c4100a80 0x7fb3c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.581+0000 7fb3cc500640 1 --2- 172.21.15.5:0/2562446026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3c4106a80 0x7fb3c4106e60 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.581+0000 7fb3cc500640 1 -- 172.21.15.5:0/2562446026 >> 172.21.15.5:0/2562446026 conn(0x7fb3c40fc7a0 msgr2=0x7fb3c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.581+0000 7fb3cc500640 1 -- 172.21.15.5:0/2562446026 shutdown_connections 2024-09-11T22:37:41.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.581+0000 7fb3cc500640 1 -- 172.21.15.5:0/2562446026 wait complete. 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3cc500640 1 Processor -- start 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3cc500640 1 -- start start 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3cc500640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3c4100a80 0x7fb3c4197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3cc500640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3c4101420 0x7fb3c4197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3cc500640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c4106a80 0x7fb3c4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3cc500640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4110e80 con 0x7fb3c4100a80 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3cc500640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4110d00 con 0x7fb3c4101420 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3cc500640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4111000 con 0x7fb3c4106a80 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3ca29c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3c4100a80 0x7fb3c4197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3ca29c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3c4100a80 0x7fb3c4197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43058/0 (socket says 172.21.15.5:43058) 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3ca29c640 1 -- 172.21.15.5:0/1228149870 learned_addr learned my addr 172.21.15.5:0/1228149870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3c9a9b640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3c4101420 0x7fb3c4197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.582+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c4106a80 0x7fb3c4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3ca29c640 1 -- 172.21.15.5:0/1228149870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c4106a80 msgr2=0x7fb3c4197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3ca29c640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c4106a80 0x7fb3c4197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3ca29c640 1 -- 172.21.15.5:0/1228149870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3c4101420 msgr2=0x7fb3c4197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3ca29c640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3c4101420 0x7fb3c4197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3ca29c640 1 -- 172.21.15.5:0/1228149870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c41a6970 con 0x7fb3c4100a80 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3ca29c640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3c4100a80 0x7fb3c4197100 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009230 tx=0x7fb3c0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3bb7fe640 1 -- 172.21.15.5:0/1228149870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c003f030 con 0x7fb3c4100a80 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3bb7fe640 1 -- 172.21.15.5:0/1228149870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c0004440 con 0x7fb3c4100a80 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3cc500640 1 -- 172.21.15.5:0/1228149870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c41a6c00 con 0x7fb3c4100a80 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3bb7fe640 1 -- 172.21.15.5:0/1228149870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0035070 con 0x7fb3c4100a80 2024-09-11T22:37:41.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.583+0000 7fb3cc500640 1 -- 172.21.15.5:0/1228149870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c41a7130 con 0x7fb3c4100a80 2024-09-11T22:37:41.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.586+0000 7fb3bb7fe640 1 -- 172.21.15.5:0/1228149870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb3c0042030 con 0x7fb3c4100a80 2024-09-11T22:37:41.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.586+0000 7fb3ce8d9740 1 Processor -- start 2024-09-11T22:37:41.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.586+0000 7fb3ce8d9740 1 -- start start 2024-09-11T22:37:41.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c51090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c51a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c5c370 0x55a468c5e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a468bb72f0 con 0x55a468c52b40 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a468b5ecf0 con 0x55a468c515d0 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a468bb32b0 con 0x55a468c5c370 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3ca29c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c51a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3caa9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c51090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3caa9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c51090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43066/0 (socket says 172.21.15.5:43066) 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3c9a9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c5c370 0x55a468c5e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3caa9d640 1 -- 172.21.15.5:0/1406379154 learned_addr learned my addr 172.21.15.5:0/1406379154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3caa9d640 1 -- 172.21.15.5:0/1406379154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c5c370 msgr2=0x55a468c5e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/1406379154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c5c370 0x55a468c5e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3caa9d640 1 -- 172.21.15.5:0/1406379154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 msgr2=0x55a468c51a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/1406379154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c51a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3caa9d640 1 -- 172.21.15.5:0/1406379154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a468b433a0 con 0x55a468c52b40 2024-09-11T22:37:41.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.587+0000 7fb3ca29c640 1 --2- 172.21.15.5:0/1406379154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c51a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.588+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/1406379154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c51090 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009f60 tx=0x7fb3bc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3baae631b6cc86 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.588+0000 7fb3bb7fe640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3b00a2d70 0x7fb3b00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.588+0000 7fb3b8ff9640 1 -- 172.21.15.5:0/1406379154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc01e070 con 0x55a468c52b40 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.588+0000 7fb3b8ff9640 1 -- 172.21.15.5:0/1406379154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3bc003c30 con 0x55a468c52b40 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.588+0000 7fb3ca29c640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3b00a2d70 0x7fb3b00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.588+0000 7fb3b8ff9640 1 -- 172.21.15.5:0/1406379154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc00fa20 con 0x55a468c52b40 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.589+0000 7fb3bb7fe640 1 -- 172.21.15.5:0/1228149870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7fb3c00ef760 con 0x7fb3c4100a80 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.589+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/1406379154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 msgr2=0x55a468c51090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.589+0000 7fb3ca29c640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3b00a2d70 0x7fb3b00a5230 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00ecef0 tx=0x7fb3c0009260 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.589+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/1406379154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c51090 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009f60 tx=0x7fb3bc009f90 comp rx=0 tx=0).stop 2024-09-11T22:37:41.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.589+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/1406379154 shutdown_connections 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.589+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/1406379154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c5c370 0x55a468c5e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.589+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/1406379154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c51a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.589+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/1406379154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c51090 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.589+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/1406379154 >> 172.21.15.5:0/1406379154 conn(0x55a468c1a310 msgr2=0x55a468c4dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/1406379154 shutdown_connections 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/1406379154 wait complete. 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 1 -- start start 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 1 Processor -- start 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c59040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c5c370 0x55a468c59650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a468c5f070 con 0x55a468c52b40 2024-09-11T22:37:41.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a468b72060 con 0x55a468c515d0 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a468c5f1f0 con 0x55a468c5c370 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.590+0000 7fb3ce8d9740 10 client.0 _ll_register_callbacks cb 0x55a4688c58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3caa9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3caa9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c5bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36156/0 (socket says 172.21.15.5:36156) 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3caa9d640 1 -- 172.21.15.5:0/4265118037 learned_addr learned my addr 172.21.15.5:0/4265118037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3c9a9b640 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c59040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c5c370 0x55a468c59650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3caa9d640 1 -- 172.21.15.5:0/4265118037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c5c370 msgr2=0x55a468c59650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c5c370 0x55a468c59650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3caa9d640 1 -- 172.21.15.5:0/4265118037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 msgr2=0x55a468c59040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c59040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3caa9d640 1 -- 172.21.15.5:0/4265118037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4689283c0 con 0x55a468c515d0 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c5bd90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc011d60 tx=0x7fb3bc00f5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c91c0809fa3acfab server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3adffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3adffb640 1 -- 172.21.15.5:0/4265118037 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc02d060 con 0x55a468c515d0 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/4265118037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3bc003c30 con 0x55a468c515d0 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3adffb640 1 -- 172.21.15.5:0/4265118037 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7fb3bc004460 con 0x55a468c515d0 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.591+0000 7fb3adffb640 1 -- 172.21.15.5:0/4265118037 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb3bc01e070 con 0x55a468c515d0 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3ce8d9740 10 client.29312 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/4265118037 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a468c6ab20 con 0x55a468c515d0 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3adffb640 1 -- 172.21.15.5:0/4265118037 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3bc003c30 con 0x55a468c515d0 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3adffb640 1 -- 172.21.15.5:0/4265118037 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7fb3bc019670 con 0x55a468c515d0 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3ce8d9740 10 client.29312 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3ce8d9740 10 client.29312 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:37:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3ce8d9740 20 client.29312 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:41.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55a468c6bed0 0x55a468c6e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3ce8d9740 4 client.29312 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:41.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/4265118037 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a468bc4860 con 0x55a468c6bed0 2024-09-11T22:37:41.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.592+0000 7fb3c9a9b640 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55a468c6bed0 0x55a468c6e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.593+0000 7fb3adffb640 10 client.29312 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:37:41.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.593+0000 7fb3c9a9b640 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55a468c6bed0 0x55a468c6e2c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f831c006b3a8e7bf server_cookie=4c1b812c130e2f62 in_seq=0 out_seq=0 2024-09-11T22:37:41.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.593+0000 7fb3adffb640 1 -- 172.21.15.5:0/4265118037 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a468bc4860 con 0x55a468c6bed0 2024-09-11T22:37:41.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.593+0000 7fb3adffb640 10 client.29312 handle_command_reply: tid=0 2024-09-11T22:37:41.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.593+0000 7fb3ce8d9740 1 client.29312 shutdown 2024-09-11T22:37:41.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/4265118037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 msgr2=0x55a468c5bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c5bd90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc011d60 tx=0x7fb3bc00f5c0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/4265118037 shutdown_connections 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55a468c6bed0 0x55a468c6e2c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3adffb640 0 client.29312 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c5c370 0x55a468c59650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c59040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/4265118037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c515d0 0x55a468c5bd90 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/4265118037 >> 172.21.15.5:0/4265118037 conn(0x55a468c1a310 msgr2=0x55a468c53930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/4265118037 shutdown_connections 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/4265118037 wait complete. 2024-09-11T22:37:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.594+0000 7fb3ce8d9740 20 client.29312 trim_cache size 0 max 16384 2024-09-11T22:37:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.609+0000 7fb3ce8d9740 1 Processor -- start 2024-09-11T22:37:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.609+0000 7fb3ce8d9740 1 -- start start 2024-09-11T22:37:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c4cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c54120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c5c370 0x55a468c54660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41afd60 con 0x55a468c52b40 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41b1960 con 0x55a468c5c370 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41b0430 con 0x55a468c515d0 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3c9a9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c54120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3caa9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c4cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3c9a9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c54120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43086/0 (socket says 172.21.15.5:43086) 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3caa9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c4cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45840/0 (socket says 172.21.15.5:45840) 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3c9a9b640 1 -- 172.21.15.5:0/554040895 learned_addr learned my addr 172.21.15.5:0/554040895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/554040895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c5c370 0x55a468c54660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3c9a9b640 1 -- 172.21.15.5:0/554040895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 msgr2=0x55a468c4cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3c9a9b640 1 --2- 172.21.15.5:0/554040895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c4cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3c9a9b640 1 -- 172.21.15.5:0/554040895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c5c370 msgr2=0x55a468c54660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3c9a9b640 1 --2- 172.21.15.5:0/554040895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c5c370 0x55a468c54660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3c9a9b640 1 -- 172.21.15.5:0/554040895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a468c4e960 con 0x55a468c52b40 2024-09-11T22:37:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.610+0000 7fb3c9a9b640 1 --2- 172.21.15.5:0/554040895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c54120 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x55a468c588d0 tx=0x7fb3b4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90bdb4dcea9bdac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.611+0000 7fb3affff640 1 -- 172.21.15.5:0/554040895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4022070 con 0x55a468c52b40 2024-09-11T22:37:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.611+0000 7fb3affff640 1 -- 172.21.15.5:0/554040895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b40047a0 con 0x55a468c52b40 2024-09-11T22:37:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.611+0000 7fb3affff640 1 -- 172.21.15.5:0/554040895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b401f820 con 0x55a468c52b40 2024-09-11T22:37:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.611+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/554040895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 msgr2=0x55a468c54120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.611+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/554040895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c54120 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x55a468c588d0 tx=0x7fb3b4005f90 comp rx=0 tx=0).stop 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.611+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/554040895 shutdown_connections 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.611+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/554040895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c5c370 0x55a468c54660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.611+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/554040895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c54120 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.611+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/554040895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c4cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.611+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/554040895 >> 172.21.15.5:0/554040895 conn(0x55a468c1a310 msgr2=0x55a468c4c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.612+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/554040895 shutdown_connections 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.612+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/554040895 wait complete. 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.612+0000 7fb3ce8d9740 1 -- start start 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.612+0000 7fb3ce8d9740 1 Processor -- start 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c64660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c64ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3ce8d9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c5c370 0x55a468c558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41b1290 con 0x55a468c52b40 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41b0b00 con 0x55a468c5c370 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3ce8d9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41add60 con 0x55a468c515d0 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3caa9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c64660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3c9a9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c64ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3caa9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c64660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45850/0 (socket says 172.21.15.5:45850) 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3caa9d640 1 -- 172.21.15.5:0/3096816678 learned_addr learned my addr 172.21.15.5:0/3096816678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3ce8d9740 10 client.0 _ll_register_callbacks cb 0x7fb3bc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c5c370 0x55a468c558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3caa9d640 1 -- 172.21.15.5:0/3096816678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c5c370 msgr2=0x55a468c558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:41.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c5c370 0x55a468c558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3caa9d640 1 -- 172.21.15.5:0/3096816678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 msgr2=0x55a468c64ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c64ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3caa9d640 1 -- 172.21.15.5:0/3096816678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a468c560e0 con 0x55a468c515d0 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3caa9d640 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c64660 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc019bb0 tx=0x7fb3bc014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe618fcb74d74fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.613+0000 7fb3adffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3adffb640 1 -- 172.21.15.5:0/3096816678 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc02d040 con 0x55a468c515d0 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/3096816678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b40047a0 con 0x55a468c515d0 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3adffb640 1 -- 172.21.15.5:0/3096816678 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7fb3bc019430 con 0x55a468c515d0 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3adffb640 1 -- 172.21.15.5:0/3096816678 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fb3bc019e20 con 0x55a468c515d0 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3ce8d9740 10 client.29164 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/3096816678 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55a468cd63e0 con 0x55a468c515d0 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3adffb640 1 -- 172.21.15.5:0/3096816678 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b40047a0 con 0x55a468c515d0 2024-09-11T22:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3adffb640 1 -- 172.21.15.5:0/3096816678 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7fb3bc0274c0 con 0x55a468c515d0 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3ce8d9740 10 client.29164 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3ce8d9740 10 client.29164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3ce8d9740 20 client.29164 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55a468c6bed0 0x55a468cd8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3ce8d9740 4 client.29164 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.614+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/3096816678 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a468bbed70 con 0x55a468c6bed0 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3c9a9b640 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55a468c6bed0 0x55a468cd8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3adffb640 10 client.29164 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3c9a9b640 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55a468c6bed0 0x55a468cd8a50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d1803e55aefb2c99 server_cookie=f7f3ef91801f086 in_seq=0 out_seq=0 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3adffb640 1 -- 172.21.15.5:0/3096816678 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a468bbed70 con 0x55a468c6bed0 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3adffb640 10 client.29164 handle_command_reply: tid=0 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3ce8d9740 1 client.29164 shutdown 2024-09-11T22:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/3096816678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 msgr2=0x55a468c64660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c64660 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc019bb0 tx=0x7fb3bc014040 comp rx=0 tx=0).stop 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/3096816678 shutdown_connections 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55a468c6bed0 0x55a468cd8a50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a468c5c370 0x55a468c558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a468c52b40 0x55a468c64ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3adffb640 0 client.29164 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3ce8d9740 1 --2- 172.21.15.5:0/3096816678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a468c515d0 0x55a468c64660 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.615+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/3096816678 >> 172.21.15.5:0/3096816678 conn(0x55a468c1a310 msgr2=0x55a468c4cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/3096816678 shutdown_connections 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3ce8d9740 1 -- 172.21.15.5:0/3096816678 wait complete. 2024-09-11T22:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3ce8d9740 20 client.29164 trim_cache size 0 max 16384 2024-09-11T22:37:41.622 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3cc500640 1 -- 172.21.15.5:0/1228149870 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3b00a2d70 msgr2=0x7fb3b00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3cc500640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3b00a2d70 0x7fb3b00a5230 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3c00ecef0 tx=0x7fb3c0009260 comp rx=0 tx=0).stop 2024-09-11T22:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3cc500640 1 -- 172.21.15.5:0/1228149870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3c4100a80 msgr2=0x7fb3c4197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3cc500640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3c4100a80 0x7fb3c4197100 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009230 tx=0x7fb3c0004040 comp rx=0 tx=0).stop 2024-09-11T22:37:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3cc500640 1 -- 172.21.15.5:0/1228149870 shutdown_connections 2024-09-11T22:37:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3cc500640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c4106a80 0x7fb3c4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3cc500640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb3b00a2d70 0x7fb3b00a5230 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3cc500640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb3c4101420 0x7fb3c4197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3cc500640 1 --2- 172.21.15.5:0/1228149870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3c4100a80 0x7fb3c4197100 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.616+0000 7fb3cc500640 1 -- 172.21.15.5:0/1228149870 >> 172.21.15.5:0/1228149870 conn(0x7fb3c40fc7a0 msgr2=0x7fb3c40fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.617+0000 7fb3cc500640 1 -- 172.21.15.5:0/1228149870 shutdown_connections 2024-09-11T22:37:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:41.617+0000 7fb3cc500640 1 -- 172.21.15.5:0/1228149870 wait complete. 2024-09-11T22:37:41.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 24241 12.47 MB/sec warmup 119 sec latency 224.342 ms 2024-09-11T22:37:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:41 smithi005 ceph-mon[30943]: pgmap v493: 97 pgs: 97 active+clean; 425 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 713 op/s 2024-09-11T22:37:41.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:41 smithi159 ceph-mon[35534]: pgmap v493: 97 pgs: 97 active+clean; 425 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 713 op/s 2024-09-11T22:37:41.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:41 smithi172 ceph-mon[35461]: pgmap v493: 97 pgs: 97 active+clean; 425 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 713 op/s 2024-09-11T22:37:42.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:37:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:37:43.609 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:43 smithi005 ceph-mon[30943]: pgmap v494: 97 pgs: 97 active+clean; 401 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 912 op/s 2024-09-11T22:37:43.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25002 13.03 MB/sec execute 1 sec latency 182.807 ms 2024-09-11T22:37:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.608+0000 7fd9c1432640 1 -- 172.21.15.5:0/1810449426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9bc102570 msgr2=0x7fd9bc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.608+0000 7fd9c1432640 1 --2- 172.21.15.5:0/1810449426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9bc102570 0x7fd9bc1029b0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009900 tx=0x7fd9b002a480 comp rx=0 tx=0).stop 2024-09-11T22:37:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.608+0000 7fd9c1432640 1 -- 172.21.15.5:0/1810449426 shutdown_connections 2024-09-11T22:37:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.608+0000 7fd9c1432640 1 --2- 172.21.15.5:0/1810449426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc102ef0 0x7fd9bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.608+0000 7fd9c1432640 1 --2- 172.21.15.5:0/1810449426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9bc102570 0x7fd9bc1029b0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.608+0000 7fd9c1432640 1 --2- 172.21.15.5:0/1810449426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9bc100e20 0x7fd9bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.608+0000 7fd9c1432640 1 -- 172.21.15.5:0/1810449426 >> 172.21.15.5:0/1810449426 conn(0x7fd9bc0fc7d0 msgr2=0x7fd9bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:43.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.608+0000 7fd9c1432640 1 -- 172.21.15.5:0/1810449426 shutdown_connections 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.608+0000 7fd9c1432640 1 -- 172.21.15.5:0/1810449426 wait complete. 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.609+0000 7fd9c1432640 1 Processor -- start 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.609+0000 7fd9c1432640 1 -- start start 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.609+0000 7fd9c1432640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9bc100e20 0x7fd9bc199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9c1432640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc102570 0x7fd9bc199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9c1432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9bc102ef0 0x7fd9bc1a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9c1432640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc110f80 con 0x7fd9bc102ef0 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9c1432640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc110e00 con 0x7fd9bc100e20 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9c1432640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc111100 con 0x7fd9bc102570 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9bc102ef0 0x7fd9bc1a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9baffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9bc100e20 0x7fd9bc199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9baffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9bc100e20 0x7fd9bc199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36216/0 (socket says 172.21.15.5:36216) 2024-09-11T22:37:43.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9baffd640 1 -- 172.21.15.5:0/2876194263 learned_addr learned my addr 172.21.15.5:0/2876194263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9bb7fe640 1 -- 172.21.15.5:0/2876194263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc102570 msgr2=0x7fd9bc199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9bb7fe640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc102570 0x7fd9bc199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9bb7fe640 1 -- 172.21.15.5:0/2876194263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9bc100e20 msgr2=0x7fd9bc199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9bb7fe640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9bc100e20 0x7fd9bc199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9bb7fe640 1 -- 172.21.15.5:0/2876194263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc1a48d0 con 0x7fd9bc102ef0 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9bb7fe640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9bc102ef0 0x7fd9bc1a4390 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009fd0 tx=0x7fd9b400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.610+0000 7fd9a3fff640 1 -- 172.21.15.5:0/2876194263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400ce20 con 0x7fd9bc102ef0 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.611+0000 7fd9c1432640 1 -- 172.21.15.5:0/2876194263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9bc1a4aa0 con 0x7fd9bc102ef0 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.611+0000 7fd9a3fff640 1 -- 172.21.15.5:0/2876194263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b4005020 con 0x7fd9bc102ef0 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.611+0000 7fd9a3fff640 1 -- 172.21.15.5:0/2876194263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4010720 con 0x7fd9bc102ef0 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.611+0000 7fd9c1432640 1 -- 172.21.15.5:0/2876194263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9bc1a4fe0 con 0x7fd9bc102ef0 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.613+0000 7fd9c380b740 1 Processor -- start 2024-09-11T22:37:43.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9a3fff640 1 -- 172.21.15.5:0/2876194263 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd9b4005540 con 0x7fd9bc102ef0 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9c380b740 1 -- start start 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d3d780 0x556323d3fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9c380b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556323c942f0 con 0x556323d2e5d0 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9c380b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556323c3bcf0 con 0x556323d2fb40 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9c380b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556323c902b0 con 0x556323d3d780 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9baffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9baffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43116/0 (socket says 172.21.15.5:43116) 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9baffd640 1 -- 172.21.15.5:0/272923975 learned_addr learned my addr 172.21.15.5:0/272923975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.614+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/272923975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/272923975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d3d780 0x556323d3fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9baffd640 1 -- 172.21.15.5:0/272923975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d3d780 msgr2=0x556323d3fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9baffd640 1 --2- 172.21.15.5:0/272923975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d3d780 0x556323d3fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9baffd640 1 -- 172.21.15.5:0/272923975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 msgr2=0x556323d2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9a3fff640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd9a40a2d70 0x7fd9a40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9baffd640 1 --2- 172.21.15.5:0/272923975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9baffd640 1 -- 172.21.15.5:0/272923975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556323c203a0 con 0x556323d2e5d0 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/272923975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9bb7fe640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd9a40a2d70 0x7fd9a40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9a3fff640 1 -- 172.21.15.5:0/2876194263 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7fd9b40cfa90 con 0x7fd9bc102ef0 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9baffd640 1 --2- 172.21.15.5:0/272923975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ea10 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00bda0 tx=0x7fd9ac0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcc62325f5707ec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9a17fa640 1 -- 172.21.15.5:0/272923975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac01c680 con 0x556323d2e5d0 2024-09-11T22:37:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.615+0000 7fd9a17fa640 1 -- 172.21.15.5:0/272923975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac011920 con 0x556323d2e5d0 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9a17fa640 1 -- 172.21.15.5:0/272923975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac01a930 con 0x556323d2e5d0 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9bb7fe640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd9a40a2d70 0x7fd9a40a5230 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40983a0 tx=0x7fd9b40bc040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9c380b740 1 -- 172.21.15.5:0/272923975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 msgr2=0x556323d2ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9c380b740 1 --2- 172.21.15.5:0/272923975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ea10 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00bda0 tx=0x7fd9ac0107a0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9c380b740 1 -- 172.21.15.5:0/272923975 shutdown_connections 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9c380b740 1 --2- 172.21.15.5:0/272923975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d3d780 0x556323d3fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9c380b740 1 --2- 172.21.15.5:0/272923975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ea10 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9c380b740 1 --2- 172.21.15.5:0/272923975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9c380b740 1 -- 172.21.15.5:0/272923975 >> 172.21.15.5:0/272923975 conn(0x556323cf7310 msgr2=0x556323d2ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9c380b740 1 -- 172.21.15.5:0/272923975 shutdown_connections 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.616+0000 7fd9c380b740 1 -- 172.21.15.5:0/272923975 wait complete. 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9c380b740 1 -- start start 2024-09-11T22:37:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9c380b740 1 Processor -- start 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2e5d0 0x556323d3d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2fb40 0x556323d3a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d3d780 0x556323d3a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9c380b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556323d40480 con 0x556323d2fb40 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9c380b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556323c4f060 con 0x556323d2e5d0 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9c380b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x556323d40600 con 0x556323d3d780 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9c380b740 10 client.0 _ll_register_callbacks cb 0x5563239a28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9baffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2fb40 0x556323d3a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9baffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2fb40 0x556323d3a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43128/0 (socket says 172.21.15.5:43128) 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9baffd640 1 -- 172.21.15.5:0/2666648582 learned_addr learned my addr 172.21.15.5:0/2666648582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2e5d0 0x556323d3d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d3d780 0x556323d3a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9baffd640 1 -- 172.21.15.5:0/2666648582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d3d780 msgr2=0x556323d3a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9baffd640 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d3d780 0x556323d3a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9baffd640 1 -- 172.21.15.5:0/2666648582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2e5d0 msgr2=0x556323d3d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9baffd640 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2e5d0 0x556323d3d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9baffd640 1 -- 172.21.15.5:0/2666648582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556323a053c0 con 0x556323d2fb40 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.617+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2e5d0 0x556323d3d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.618+0000 7fd9baffd640 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2fb40 0x556323d3a400 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac011010 tx=0x7fd9ac011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7b38de33b9c2b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.618+0000 7fd99eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.618+0000 7fd99eff5640 1 -- 172.21.15.5:0/2666648582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac02a060 con 0x556323d2fb40 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.618+0000 7fd99eff5640 1 -- 172.21.15.5:0/2666648582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7fd9ac011650 con 0x556323d2fb40 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.618+0000 7fd9c380b740 1 -- 172.21.15.5:0/2666648582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac011920 con 0x556323d2fb40 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.618+0000 7fd99eff5640 1 -- 172.21.15.5:0/2666648582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd9ac00b970 con 0x556323d2fb40 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.618+0000 7fd9c380b740 10 client.20256 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.618+0000 7fd9c380b740 1 -- 172.21.15.5:0/2666648582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x556323d4bf20 con 0x556323d2fb40 2024-09-11T22:37:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.618+0000 7fd99eff5640 1 -- 172.21.15.5:0/2666648582 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac011920 con 0x556323d2fb40 2024-09-11T22:37:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.618+0000 7fd99eff5640 1 -- 172.21.15.5:0/2666648582 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7fd9ac01a630 con 0x556323d2fb40 2024-09-11T22:37:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.619+0000 7fd9c380b740 10 client.20256 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.619+0000 7fd9c380b740 10 client.20256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.619+0000 7fd9c380b740 20 client.20256 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.619+0000 7fd9c380b740 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556323d4d270 0x556323d4f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.619+0000 7fd9c380b740 4 client.20256 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.619+0000 7fd9c380b740 1 -- 172.21.15.5:0/2666648582 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556323ca1860 con 0x556323d4d270 2024-09-11T22:37:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.662+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556323d4d270 0x556323d4f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.663+0000 7fd99eff5640 10 client.20256 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:43.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.663+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556323d4d270 0x556323d4f660 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b1c8df0f2e3b820 server_cookie=9faee644a8660cb in_seq=0 out_seq=0 2024-09-11T22:37:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd99eff5640 1 -- 172.21.15.5:0/2666648582 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556323ca1860 con 0x556323d4d270 2024-09-11T22:37:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd99eff5640 10 client.20256 handle_command_reply: tid=0 2024-09-11T22:37:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 client.20256 shutdown 2024-09-11T22:37:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 -- 172.21.15.5:0/2666648582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2fb40 msgr2=0x556323d3a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2fb40 0x556323d3a400 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac011010 tx=0x7fd9ac011d00 comp rx=0 tx=0).stop 2024-09-11T22:37:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 -- 172.21.15.5:0/2666648582 shutdown_connections 2024-09-11T22:37:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556323d4d270 0x556323d4f660 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd99eff5640 0 client.20256 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d3d780 0x556323d3a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2fb40 0x556323d3a400 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 --2- 172.21.15.5:0/2666648582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2e5d0 0x556323d3d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 -- 172.21.15.5:0/2666648582 >> 172.21.15.5:0/2666648582 conn(0x556323cf7310 msgr2=0x556323d30930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 -- 172.21.15.5:0/2666648582 shutdown_connections 2024-09-11T22:37:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 1 -- 172.21.15.5:0/2666648582 wait complete. 2024-09-11T22:37:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.667+0000 7fd9c380b740 20 client.20256 trim_cache size 0 max 16384 2024-09-11T22:37:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.681+0000 7fd9c380b740 1 Processor -- start 2024-09-11T22:37:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9c380b740 1 -- start start 2024-09-11T22:37:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2fb40 0x556323d49750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d3d780 0x556323d39cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9c380b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc1ab8d0 con 0x556323d2e5d0 2024-09-11T22:37:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9c380b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc1ad4d0 con 0x556323d3d780 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9c380b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc1abfa0 con 0x556323d2fb40 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9ba7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9baffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2fb40 0x556323d49750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9ba7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43132/0 (socket says 172.21.15.5:43132) 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9ba7fc640 1 -- 172.21.15.5:0/3035305784 learned_addr learned my addr 172.21.15.5:0/3035305784 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/3035305784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d3d780 0x556323d39cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9baffd640 1 -- 172.21.15.5:0/3035305784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d3d780 msgr2=0x556323d39cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9baffd640 1 --2- 172.21.15.5:0/3035305784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d3d780 0x556323d39cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9baffd640 1 -- 172.21.15.5:0/3035305784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 msgr2=0x556323d2ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9baffd640 1 --2- 172.21.15.5:0/3035305784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.682+0000 7fd9baffd640 1 -- 172.21.15.5:0/3035305784 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556323d2b960 con 0x556323d2fb40 2024-09-11T22:37:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.683+0000 7fd9baffd640 1 --2- 172.21.15.5:0/3035305784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2fb40 0x556323d49750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00b2f0 tx=0x7fd9ac01afc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1ef6db5745d6144 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.683+0000 7fd9a0ff9640 1 -- 172.21.15.5:0/3035305784 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac02a070 con 0x556323d2fb40 2024-09-11T22:37:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.683+0000 7fd9a0ff9640 1 -- 172.21.15.5:0/3035305784 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac01abd0 con 0x556323d2fb40 2024-09-11T22:37:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.683+0000 7fd9a0ff9640 1 -- 172.21.15.5:0/3035305784 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac005350 con 0x556323d2fb40 2024-09-11T22:37:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 -- 172.21.15.5:0/3035305784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2fb40 msgr2=0x556323d49750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 --2- 172.21.15.5:0/3035305784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2fb40 0x556323d49750 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00b2f0 tx=0x7fd9ac01afc0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 -- 172.21.15.5:0/3035305784 shutdown_connections 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 --2- 172.21.15.5:0/3035305784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d3d780 0x556323d39cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 --2- 172.21.15.5:0/3035305784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2fb40 0x556323d49750 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 --2- 172.21.15.5:0/3035305784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d2e5d0 0x556323d2ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 -- 172.21.15.5:0/3035305784 >> 172.21.15.5:0/3035305784 conn(0x556323cf7310 msgr2=0x556323d2a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 -- 172.21.15.5:0/3035305784 shutdown_connections 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 -- 172.21.15.5:0/3035305784 wait complete. 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 -- start start 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.684+0000 7fd9c380b740 1 Processor -- start 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2e5d0 0x556323d45b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d460b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9c380b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d3d780 0x556323d36e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9c380b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc1ace00 con 0x556323d3d780 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9c380b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc1ac670 con 0x556323d2fb40 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9c380b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc1a98d0 con 0x556323d2e5d0 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9c380b740 10 client.0 _ll_register_callbacks cb 0x7fd9ac00bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9ba7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d3d780 0x556323d36e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9baffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9baffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d460b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36254/0 (socket says 172.21.15.5:36254) 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9baffd640 1 -- 172.21.15.5:0/1171856275 learned_addr learned my addr 172.21.15.5:0/1171856275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9ba7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d3d780 0x556323d36e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43134/0 (socket says 172.21.15.5:43134) 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2e5d0 0x556323d45b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9baffd640 1 -- 172.21.15.5:0/1171856275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2e5d0 msgr2=0x556323d45b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9baffd640 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2e5d0 0x556323d45b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9baffd640 1 -- 172.21.15.5:0/1171856275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d3d780 msgr2=0x556323d36e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9baffd640 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d3d780 0x556323d36e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9baffd640 1 -- 172.21.15.5:0/1171856275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556323d2a510 con 0x556323d2fb40 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d3d780 0x556323d36e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd9baffd640 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d460b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac01adc0 tx=0x7fd9ac0119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1da4cd0fe6ce138 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.685+0000 7fd99eff5640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd99eff5640 1 -- 172.21.15.5:0/1171856275 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac03a060 con 0x556323d2fb40 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd9c380b740 1 -- 172.21.15.5:0/1171856275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac01abd0 con 0x556323d2fb40 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd99eff5640 1 -- 172.21.15.5:0/1171856275 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7fd9ac0052d0 con 0x556323d2fb40 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd99eff5640 1 -- 172.21.15.5:0/1171856275 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fd9ac02a040 con 0x556323d2fb40 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd9c380b740 10 client.29336 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd9c380b740 1 -- 172.21.15.5:0/1171856275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x556323db7700 con 0x556323d2fb40 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd99eff5640 1 -- 172.21.15.5:0/1171856275 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ac01abd0 con 0x556323d2fb40 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd99eff5640 1 -- 172.21.15.5:0/1171856275 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7fd9ac003ad0 con 0x556323d2fb40 2024-09-11T22:37:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd9c380b740 10 client.29336 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd9c380b740 10 client.29336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd9c380b740 20 client.29336 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd9c380b740 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556323d4d270 0x556323db9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd9c380b740 4 client.29336 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:37:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd9c380b740 1 -- 172.21.15.5:0/1171856275 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556323c9bd70 con 0x556323d4d270 2024-09-11T22:37:43.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.686+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556323d4d270 0x556323db9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.687+0000 7fd99eff5640 10 client.29336 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.687+0000 7fd9ba7fc640 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556323d4d270 0x556323db9ed0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cec4d0e87bdb53 server_cookie=6f419e33bcedcfc0 in_seq=0 out_seq=0 2024-09-11T22:37:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd99eff5640 1 -- 172.21.15.5:0/1171856275 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556323c9bd70 con 0x556323d4d270 2024-09-11T22:37:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd99eff5640 10 client.29336 handle_command_reply: tid=0 2024-09-11T22:37:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 client.29336 shutdown 2024-09-11T22:37:43.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 -- 172.21.15.5:0/1171856275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 msgr2=0x556323d460b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d460b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac01adc0 tx=0x7fd9ac0119e0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 -- 172.21.15.5:0/1171856275 shutdown_connections 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x556323d4d270 0x556323db9ed0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556323d3d780 0x556323d36e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556323d2fb40 0x556323d460b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd99eff5640 0 client.29336 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 --2- 172.21.15.5:0/1171856275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556323d2e5d0 0x556323d45b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 -- 172.21.15.5:0/1171856275 >> 172.21.15.5:0/1171856275 conn(0x556323cf7310 msgr2=0x556323d294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 -- 172.21.15.5:0/1171856275 shutdown_connections 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 1 -- 172.21.15.5:0/1171856275 wait complete. 2024-09-11T22:37:43.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.689+0000 7fd9c380b740 20 client.29336 trim_cache size 0 max 16384 2024-09-11T22:37:43.694 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:37:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 -- 172.21.15.5:0/2876194263 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd9a40a2d70 msgr2=0x7fd9a40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd9a40a2d70 0x7fd9a40a5230 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40983a0 tx=0x7fd9b40bc040 comp rx=0 tx=0).stop 2024-09-11T22:37:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 -- 172.21.15.5:0/2876194263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9bc102ef0 msgr2=0x7fd9bc1a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9bc102ef0 0x7fd9bc1a4390 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009fd0 tx=0x7fd9b400ef10 comp rx=0 tx=0).stop 2024-09-11T22:37:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 -- 172.21.15.5:0/2876194263 shutdown_connections 2024-09-11T22:37:43.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9bc102ef0 0x7fd9bc1a4390 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc102570 0x7fd9bc199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd9a40a2d70 0x7fd9a40a5230 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 --2- 172.21.15.5:0/2876194263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9bc100e20 0x7fd9bc199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 -- 172.21.15.5:0/2876194263 >> 172.21.15.5:0/2876194263 conn(0x7fd9bc0fc7d0 msgr2=0x7fd9bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 -- 172.21.15.5:0/2876194263 shutdown_connections 2024-09-11T22:37:43.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:43.690+0000 7fd9c1432640 1 -- 172.21.15.5:0/2876194263 wait complete. 2024-09-11T22:37:43.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:43 smithi159 ceph-mon[35534]: pgmap v494: 97 pgs: 97 active+clean; 401 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 912 op/s 2024-09-11T22:37:43.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:43 smithi172 ceph-mon[35461]: pgmap v494: 97 pgs: 97 active+clean; 401 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 912 op/s 2024-09-11T22:37:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.318+0000 7fa6afde4640 1 -- 172.21.15.5:0/4211708271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a8101350 msgr2=0x7fa6a810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.318+0000 7fa6afde4640 1 --2- 172.21.15.5:0/4211708271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a8101350 0x7fa6a810d880 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009cd0 tx=0x7fa6a002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.318+0000 7fa6afde4640 1 -- 172.21.15.5:0/4211708271 shutdown_connections 2024-09-11T22:37:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.318+0000 7fa6afde4640 1 --2- 172.21.15.5:0/4211708271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a8101350 0x7fa6a810d880 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.318+0000 7fa6afde4640 1 --2- 172.21.15.5:0/4211708271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a81009b0 0x7fa6a8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.318+0000 7fa6afde4640 1 --2- 172.21.15.5:0/4211708271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a81069c0 0x7fa6a8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.318+0000 7fa6afde4640 1 -- 172.21.15.5:0/4211708271 >> 172.21.15.5:0/4211708271 conn(0x7fa6a8077990 msgr2=0x7fa6a80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.318+0000 7fa6afde4640 1 -- 172.21.15.5:0/4211708271 shutdown_connections 2024-09-11T22:37:44.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.318+0000 7fa6afde4640 1 -- 172.21.15.5:0/4211708271 wait complete. 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.319+0000 7fa6afde4640 1 Processor -- start 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.319+0000 7fa6afde4640 1 -- start start 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.319+0000 7fa6afde4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a81009b0 0x7fa6a8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.319+0000 7fa6afde4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a8101350 0x7fa6a8198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.319+0000 7fa6afde4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a81069c0 0x7fa6a819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.319+0000 7fa6afde4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a8111f40 con 0x7fa6a8101350 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.319+0000 7fa6afde4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a8111dc0 con 0x7fa6a81009b0 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.319+0000 7fa6afde4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81120c0 con 0x7fa6a81069c0 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.319+0000 7fa6ad37f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a8101350 0x7fa6a8198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.319+0000 7fa6adb80640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a81009b0 0x7fa6a8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6ad37f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a8101350 0x7fa6a8198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43152/0 (socket says 172.21.15.5:43152) 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6ad37f640 1 -- 172.21.15.5:0/1693898122 learned_addr learned my addr 172.21.15.5:0/1693898122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6ae381640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a81069c0 0x7fa6a819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6adb80640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a81009b0 0x7fa6a8198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36264/0 (socket says 172.21.15.5:36264) 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6ad37f640 1 -- 172.21.15.5:0/1693898122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a81069c0 msgr2=0x7fa6a819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6ad37f640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a81069c0 0x7fa6a819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6ad37f640 1 -- 172.21.15.5:0/1693898122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a81009b0 msgr2=0x7fa6a8198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6ad37f640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a81009b0 0x7fa6a8198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6ad37f640 1 -- 172.21.15.5:0/1693898122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a819d300 con 0x7fa6a8101350 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6ad37f640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a8101350 0x7fa6a8198850 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa69800e9f0 tx=0x7fa69800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa69effd640 1 -- 172.21.15.5:0/1693898122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69800cd80 con 0x7fa6a8101350 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa69effd640 1 -- 172.21.15.5:0/1693898122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa698010910 con 0x7fa6a8101350 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa69effd640 1 -- 172.21.15.5:0/1693898122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698018990 con 0x7fa6a8101350 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6afde4640 1 -- 172.21.15.5:0/1693898122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a819d5f0 con 0x7fa6a8101350 2024-09-11T22:37:44.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.320+0000 7fa6afde4640 1 -- 172.21.15.5:0/1693898122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a81a4e80 con 0x7fa6a8101350 2024-09-11T22:37:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.323+0000 7fa69effd640 1 -- 172.21.15.5:0/1693898122 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa698010ac0 con 0x7fa6a8101350 2024-09-11T22:37:44.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.324+0000 7fa6b2c56740 1 Processor -- start 2024-09-11T22:37:44.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.324+0000 7fa6b2c56740 1 -- start start 2024-09-11T22:37:44.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 0x55e1111aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa69effd640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6940a2da0 0x7fa6940a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111af410 0x55e1111b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111b9cb0 0x55e1111bc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e1111132f0 con 0x55e1111af410 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e1110bb270 con 0x55e1111aeaf0 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e11110f2b0 con 0x55e1111b9cb0 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6ae381640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 0x55e1111aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6ad37f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111af410 0x55e1111b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6ae381640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 0x55e1111aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36276/0 (socket says 172.21.15.5:36276) 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6ae381640 1 -- 172.21.15.5:0/2623606243 learned_addr learned my addr 172.21.15.5:0/2623606243 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6adb80640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6940a2da0 0x7fa6940a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6ae381640 1 --2- 172.21.15.5:0/2623606243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111b9cb0 0x55e1111bc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa69effd640 1 -- 172.21.15.5:0/1693898122 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7fa6980cf7b0 con 0x7fa6a8101350 2024-09-11T22:37:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6ae381640 1 -- 172.21.15.5:0/2623606243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111b9cb0 msgr2=0x55e1111bc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6ae381640 1 --2- 172.21.15.5:0/2623606243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111b9cb0 0x55e1111bc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6ae381640 1 -- 172.21.15.5:0/2623606243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111af410 msgr2=0x55e1111b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6ae381640 1 --2- 172.21.15.5:0/2623606243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111af410 0x55e1111b9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.325+0000 7fa6ae381640 1 -- 172.21.15.5:0/2623606243 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1110ba340 con 0x55e1111aeaf0 2024-09-11T22:37:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.326+0000 7fa6ae381640 1 --2- 172.21.15.5:0/2623606243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 0x55e1111aeed0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa6a002aed0 tx=0x7fa6a0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2523051ba7f03df server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.326+0000 7fa6adb80640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6940a2da0 0x7fa6940a5260 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009b30 tx=0x7fa6a4005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.326+0000 7fa693fff640 1 -- 172.21.15.5:0/2623606243 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0031ac0 con 0x55e1111aeaf0 2024-09-11T22:37:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.326+0000 7fa693fff640 1 -- 172.21.15.5:0/2623606243 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a0036040 con 0x55e1111aeaf0 2024-09-11T22:37:44.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.326+0000 7fa693fff640 1 -- 172.21.15.5:0/2623606243 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00045a0 con 0x55e1111aeaf0 2024-09-11T22:37:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2623606243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 msgr2=0x55e1111aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/2623606243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 0x55e1111aeed0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa6a002aed0 tx=0x7fa6a0004a40 comp rx=0 tx=0).stop 2024-09-11T22:37:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2623606243 shutdown_connections 2024-09-11T22:37:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/2623606243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111b9cb0 0x55e1111bc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/2623606243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111af410 0x55e1111b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/2623606243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 0x55e1111aeed0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2623606243 >> 172.21.15.5:0/2623606243 conn(0x55e1111ab060 msgr2=0x55e1111ab440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2623606243 shutdown_connections 2024-09-11T22:37:44.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2623606243 wait complete. 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 -- start start 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.327+0000 7fa6b2c56740 1 Processor -- start 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111aeaf0 0x55e1111b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 0x55e1111b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111b9cb0 0x55e1111b7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e1111bc880 con 0x55e1111aeaf0 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e11109c290 con 0x55e1111af410 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e1111bca00 con 0x55e1111b9cb0 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6adb80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111aeaf0 0x55e1111b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6adb80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111aeaf0 0x55e1111b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43166/0 (socket says 172.21.15.5:43166) 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6adb80640 1 -- 172.21.15.5:0/2193190071 learned_addr learned my addr 172.21.15.5:0/2193190071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6b2c56740 10 client.0 _ll_register_callbacks cb 0x55e1110cd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:44.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6ad37f640 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 0x55e1111b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6ae381640 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111b9cb0 0x55e1111b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6adb80640 1 -- 172.21.15.5:0/2193190071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111b9cb0 msgr2=0x55e1111b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6adb80640 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111b9cb0 0x55e1111b7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6adb80640 1 -- 172.21.15.5:0/2193190071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 msgr2=0x55e1111b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6adb80640 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 0x55e1111b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6adb80640 1 -- 172.21.15.5:0/2193190071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e110e26210 con 0x55e1111aeaf0 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6ad37f640 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 0x55e1111b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6adb80640 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111aeaf0 0x55e1111b6ac0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4004600 tx=0x7fa6a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2fb24dce0a42c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.328+0000 7fa6917fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.329+0000 7fa6917fa640 1 -- 172.21.15.5:0/2193190071 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a401e8f0 con 0x55e1111aeaf0 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.329+0000 7fa6917fa640 1 -- 172.21.15.5:0/2193190071 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7fa6a401ea90 con 0x55e1111aeaf0 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.329+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2193190071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a0036040 con 0x55e1111aeaf0 2024-09-11T22:37:44.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.329+0000 7fa6917fa640 1 -- 172.21.15.5:0/2193190071 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa6a401ec90 con 0x55e1111aeaf0 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.329+0000 7fa6b2c56740 10 client.20274 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.329+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2193190071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e110ef2620 con 0x55e1111aeaf0 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.329+0000 7fa6917fa640 1 -- 172.21.15.5:0/2193190071 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a0036040 con 0x55e1111aeaf0 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.329+0000 7fa6917fa640 1 -- 172.21.15.5:0/2193190071 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7fa6a40264d0 con 0x55e1111aeaf0 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.329+0000 7fa6b2c56740 10 client.20274 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.329+0000 7fa6b2c56740 10 client.20274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.330+0000 7fa6b2c56740 20 client.20274 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.330+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e1111c9bb0 0x55e1111cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.330+0000 7fa6b2c56740 4 client.20274 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.330+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2193190071 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e111120a20 con 0x55e1111c9bb0 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.330+0000 7fa6ad37f640 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e1111c9bb0 0x55e1111cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.330+0000 7fa6ad37f640 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e1111c9bb0 0x55e1111cc040 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac787dc58368c24 server_cookie=1196407b786e29e4 in_seq=0 out_seq=0 2024-09-11T22:37:44.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.330+0000 7fa6917fa640 10 client.20274 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.332+0000 7fa6917fa640 1 -- 172.21.15.5:0/2193190071 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e111120a20 con 0x55e1111c9bb0 2024-09-11T22:37:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.332+0000 7fa6917fa640 10 client.20274 handle_command_reply: tid=0 2024-09-11T22:37:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.332+0000 7fa6b2c56740 1 client.20274 shutdown 2024-09-11T22:37:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.332+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2193190071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111aeaf0 msgr2=0x55e1111b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.332+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111aeaf0 0x55e1111b6ac0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4004600 tx=0x7fa6a4004730 comp rx=0 tx=0).stop 2024-09-11T22:37:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.333+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2193190071 shutdown_connections 2024-09-11T22:37:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.333+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e1111c9bb0 0x55e1111cc040 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.333+0000 7fa6917fa640 0 client.20274 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.333+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111b9cb0 0x55e1111b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.333+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 0x55e1111b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.333+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/2193190071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111aeaf0 0x55e1111b6ac0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.333+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2193190071 >> 172.21.15.5:0/2193190071 conn(0x55e1111ab060 msgr2=0x55e1111b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.333+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2193190071 shutdown_connections 2024-09-11T22:37:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.333+0000 7fa6b2c56740 1 -- 172.21.15.5:0/2193190071 wait complete. 2024-09-11T22:37:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.333+0000 7fa6b2c56740 20 client.20274 trim_cache size 0 max 16384 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.349+0000 7fa6b2c56740 1 Processor -- start 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.349+0000 7fa6b2c56740 1 -- start start 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.349+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 0x55e1111b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.349+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111af410 0x55e1111b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.349+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111b7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.349+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81adb00 con 0x55e1111b9cb0 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.349+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81af700 con 0x55e1111aeaf0 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.349+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81ae1d0 con 0x55e1111af410 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6ae381640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6ae381640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111b7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43168/0 (socket says 172.21.15.5:43168) 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6ae381640 1 -- 172.21.15.5:0/1930207636 learned_addr learned my addr 172.21.15.5:0/1930207636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:44.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6ad37f640 1 --2- 172.21.15.5:0/1930207636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111af410 0x55e1111b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6adb80640 1 --2- 172.21.15.5:0/1930207636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 0x55e1111b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6ae381640 1 -- 172.21.15.5:0/1930207636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111af410 msgr2=0x55e1111b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6ae381640 1 --2- 172.21.15.5:0/1930207636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111af410 0x55e1111b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6ae381640 1 -- 172.21.15.5:0/1930207636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 msgr2=0x55e1111b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6ae381640 1 --2- 172.21.15.5:0/1930207636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 0x55e1111b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6ae381640 1 -- 172.21.15.5:0/1930207636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1111aae90 con 0x55e1111b9cb0 2024-09-11T22:37:44.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6ae381640 1 --2- 172.21.15.5:0/1930207636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111b7370 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0004290 tx=0x7fa6a002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf4d3a111134f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6937fe640 1 -- 172.21.15.5:0/1930207636 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0031c80 con 0x55e1111b9cb0 2024-09-11T22:37:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6937fe640 1 -- 172.21.15.5:0/1930207636 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a004f070 con 0x55e1111b9cb0 2024-09-11T22:37:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.350+0000 7fa6937fe640 1 -- 172.21.15.5:0/1930207636 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00305a0 con 0x55e1111b9cb0 2024-09-11T22:37:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.351+0000 7fa6b2c56740 1 -- 172.21.15.5:0/1930207636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 msgr2=0x55e1111b7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.351+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/1930207636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111b7370 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0004290 tx=0x7fa6a002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.351+0000 7fa6b2c56740 1 -- 172.21.15.5:0/1930207636 shutdown_connections 2024-09-11T22:37:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.351+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/1930207636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111b7370 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.351+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/1930207636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111af410 0x55e1111b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.351+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/1930207636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111aeaf0 0x55e1111b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.351+0000 7fa6b2c56740 1 -- 172.21.15.5:0/1930207636 >> 172.21.15.5:0/1930207636 conn(0x55e1111ab060 msgr2=0x55e1111b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.351+0000 7fa6b2c56740 1 -- 172.21.15.5:0/1930207636 shutdown_connections 2024-09-11T22:37:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.351+0000 7fa6b2c56740 1 -- 172.21.15.5:0/1930207636 wait complete. 2024-09-11T22:37:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6b2c56740 1 -- start start 2024-09-11T22:37:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6b2c56740 1 Processor -- start 2024-09-11T22:37:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111aeaf0 0x55e1111c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 0x55e1111c2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6b2c56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81af030 con 0x55e1111b9cb0 2024-09-11T22:37:44.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81ae8a0 con 0x55e1111af410 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6b2c56740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81abb00 con 0x55e1111aeaf0 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6b2c56740 10 client.0 _ll_register_callbacks cb 0x7fa6a4011be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6adb80640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111aeaf0 0x55e1111c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6ae381640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6ae381640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111bd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43182/0 (socket says 172.21.15.5:43182) 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.352+0000 7fa6adb80640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111aeaf0 0x55e1111c1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45974/0 (socket says 172.21.15.5:45974) 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6ae381640 1 -- 172.21.15.5:0/3016897968 learned_addr learned my addr 172.21.15.5:0/3016897968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6ad37f640 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 0x55e1111c2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6ae381640 1 -- 172.21.15.5:0/3016897968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111aeaf0 msgr2=0x55e1111c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6ae381640 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111aeaf0 0x55e1111c1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6ae381640 1 -- 172.21.15.5:0/3016897968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 msgr2=0x55e1111c2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:44.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6ae381640 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 0x55e1111c2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6ae381640 1 -- 172.21.15.5:0/3016897968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1111ae920 con 0x55e1111b9cb0 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6ad37f640 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 0x55e1111c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6ae381640 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111bd1c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0003e20 tx=0x7fa6a0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b74ba008aed59a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6917fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6917fa640 1 -- 172.21.15.5:0/3016897968 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0031c90 con 0x55e1111b9cb0 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6b2c56740 1 -- 172.21.15.5:0/3016897968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a004f070 con 0x55e1111b9cb0 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6917fa640 1 -- 172.21.15.5:0/3016897968 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7fa6a0004b40 con 0x55e1111b9cb0 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6917fa640 1 -- 172.21.15.5:0/3016897968 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7fa6a0004d40 con 0x55e1111b9cb0 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.353+0000 7fa6b2c56740 10 client.20280 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.354+0000 7fa6b2c56740 1 -- 172.21.15.5:0/3016897968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55e111234230 con 0x55e1111b9cb0 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.354+0000 7fa6917fa640 1 -- 172.21.15.5:0/3016897968 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a004f070 con 0x55e1111b9cb0 2024-09-11T22:37:44.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.354+0000 7fa6917fa640 1 -- 172.21.15.5:0/3016897968 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7fa6a0030070 con 0x55e1111b9cb0 2024-09-11T22:37:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.354+0000 7fa6b2c56740 10 client.20280 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.354+0000 7fa6b2c56740 10 client.20280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.354+0000 7fa6b2c56740 20 client.20280 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.354+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e1111c9bb0 0x55e111236500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.354+0000 7fa6b2c56740 4 client.20280 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:37:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.354+0000 7fa6b2c56740 1 -- 172.21.15.5:0/3016897968 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e11111ad70 con 0x55e1111c9bb0 2024-09-11T22:37:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.354+0000 7fa6adb80640 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e1111c9bb0 0x55e111236500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.355+0000 7fa6917fa640 10 client.20280 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:44.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.355+0000 7fa6adb80640 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e1111c9bb0 0x55e111236500 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b6a0b79b0892a3a server_cookie=19de608c45b1b44 in_seq=0 out_seq=0 2024-09-11T22:37:44.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25319 12.44 MB/sec execute 2 sec latency 188.348 ms 2024-09-11T22:37:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6917fa640 1 -- 172.21.15.5:0/3016897968 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e11111ad70 con 0x55e1111c9bb0 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6917fa640 10 client.20280 handle_command_reply: tid=0 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6b2c56740 1 client.20280 shutdown 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6b2c56740 1 -- 172.21.15.5:0/3016897968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 msgr2=0x55e1111bd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111bd1c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0003e20 tx=0x7fa6a0004170 comp rx=0 tx=0).stop 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6b2c56740 1 -- 172.21.15.5:0/3016897968 shutdown_connections 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e1111c9bb0 0x55e111236500 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1111b9cb0 0x55e1111bd1c0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e1111af410 0x55e1111c2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6b2c56740 1 --2- 172.21.15.5:0/3016897968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e1111aeaf0 0x55e1111c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6b2c56740 1 -- 172.21.15.5:0/3016897968 >> 172.21.15.5:0/3016897968 conn(0x55e1111ab060 msgr2=0x55e1111c4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6917fa640 0 client.20280 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.696+0000 7fa6b2c56740 1 -- 172.21.15.5:0/3016897968 shutdown_connections 2024-09-11T22:37:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.697+0000 7fa6b2c56740 1 -- 172.21.15.5:0/3016897968 wait complete. 2024-09-11T22:37:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.697+0000 7fa6b2c56740 20 client.20280 trim_cache size 0 max 16384 2024-09-11T22:37:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:37:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:37:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:37:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 -- 172.21.15.5:0/1693898122 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6940a2da0 msgr2=0x7fa6940a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6940a2da0 0x7fa6940a5260 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009b30 tx=0x7fa6a4005f90 comp rx=0 tx=0).stop 2024-09-11T22:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 -- 172.21.15.5:0/1693898122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a8101350 msgr2=0x7fa6a8198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a8101350 0x7fa6a8198850 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa69800e9f0 tx=0x7fa69800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 -- 172.21.15.5:0/1693898122 shutdown_connections 2024-09-11T22:37:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa6a81069c0 0x7fa6a819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6a8101350 0x7fa6a8198850 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa6940a2da0 0x7fa6940a5260 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 --2- 172.21.15.5:0/1693898122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6a81009b0 0x7fa6a8198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 -- 172.21.15.5:0/1693898122 >> 172.21.15.5:0/1693898122 conn(0x7fa6a8077990 msgr2=0x7fa6a8103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 -- 172.21.15.5:0/1693898122 shutdown_connections 2024-09-11T22:37:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:44.698+0000 7fa6afde4640 1 -- 172.21.15.5:0/1693898122 wait complete. 2024-09-11T22:37:45.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25800 15.35 MB/sec execute 3 sec latency 192.657 ms 2024-09-11T22:37:45.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:45 smithi159 ceph-mon[35534]: pgmap v495: 97 pgs: 97 active+clean; 401 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-09-11T22:37:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:45 smithi172 ceph-mon[35461]: pgmap v495: 97 pgs: 97 active+clean; 401 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-09-11T22:37:46.043 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:37:46.044 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:37:46.069 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:45 smithi005 ceph-mon[30943]: pgmap v495: 97 pgs: 97 active+clean; 401 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-09-11T22:37:46.346 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:37:46.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 26311 18.56 MB/sec execute 4 sec latency 158.315 ms 2024-09-11T22:37:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.916+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1162284816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7810c180 msgr2=0x7f6f7810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.916+0000 7f6f7cf39640 1 --2- 172.21.15.5:0/1162284816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7810c180 0x7f6f7810e600 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009ca0 tx=0x7f6f7002a950 comp rx=0 tx=0).stop 2024-09-11T22:37:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.916+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1162284816 shutdown_connections 2024-09-11T22:37:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.916+0000 7f6f7cf39640 1 --2- 172.21.15.5:0/1162284816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f7810c180 0x7f6f7810e600 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.916+0000 7f6f7cf39640 1 --2- 172.21.15.5:0/1162284816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f781036c0 0x7f6f7810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.916+0000 7f6f7cf39640 1 --2- 172.21.15.5:0/1162284816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f78102da0 0x7f6f78103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.916+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1162284816 >> 172.21.15.5:0/1162284816 conn(0x7f6f780fc7a0 msgr2=0x7f6f780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.916+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1162284816 shutdown_connections 2024-09-11T22:37:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.916+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1162284816 wait complete. 2024-09-11T22:37:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7cf39640 1 Processor -- start 2024-09-11T22:37:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7cf39640 1 -- start start 2024-09-11T22:37:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7cf39640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f78102da0 0x7f6f78100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7cf39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f781036c0 0x7f6f78100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7cf39640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f7810c180 0x7f6f78101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7cf39640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78111f60 con 0x7f6f781036c0 2024-09-11T22:37:46.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7cf39640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f78111de0 con 0x7f6f7810c180 2024-09-11T22:37:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7cf39640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6f781120e0 con 0x7f6f78102da0 2024-09-11T22:37:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f78102da0 0x7f6f78100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f76d9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f7810c180 0x7f6f78101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f78102da0 0x7f6f78100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:45998/0 (socket says 172.21.15.5:45998) 2024-09-11T22:37:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f75d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f781036c0 0x7f6f78100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.917+0000 7f6f7659c640 1 -- 172.21.15.5:0/1527991057 learned_addr learned my addr 172.21.15.5:0/1527991057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f76d9d640 1 -- 172.21.15.5:0/1527991057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f78102da0 msgr2=0x7f6f78100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f76d9d640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f78102da0 0x7f6f78100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f76d9d640 1 -- 172.21.15.5:0/1527991057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f781036c0 msgr2=0x7f6f78100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:46.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f76d9d640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f781036c0 0x7f6f78100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f76d9d640 1 -- 172.21.15.5:0/1527991057 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f781a6880 con 0x7f6f7810c180 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f76d9d640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f7810c180 0x7f6f78101480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6f70005cc0 tx=0x7f6f7002f3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f677fe640 1 -- 172.21.15.5:0/1527991057 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70034040 con 0x7f6f7810c180 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f677fe640 1 -- 172.21.15.5:0/1527991057 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f70030920 con 0x7f6f7810c180 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1527991057 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f781a6b10 con 0x7f6f7810c180 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f677fe640 1 -- 172.21.15.5:0/1527991057 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f700045b0 con 0x7f6f7810c180 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.918+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1527991057 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f781a6ff0 con 0x7f6f7810c180 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.921+0000 7f6f677fe640 1 -- 172.21.15.5:0/1527991057 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6f70037020 con 0x7f6f7810c180 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.921+0000 7f6f7f312740 1 Processor -- start 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.921+0000 7f6f7f312740 1 -- start start 2024-09-11T22:37:46.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3821f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d382c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d382cd40 0x5592d382f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7f312740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5592d3787280 con 0x5592d3821b60 2024-09-11T22:37:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7f312740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5592d372eb10 con 0x5592d3822480 2024-09-11T22:37:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7f312740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5592d3783240 con 0x5592d382cd40 2024-09-11T22:37:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f677fe640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f600a2cf0 0x7f6f600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f75d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3821f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f75d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3821f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43216/0 (socket says 172.21.15.5:43216) 2024-09-11T22:37:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f75d9b640 1 -- 172.21.15.5:0/1673344664 learned_addr learned my addr 172.21.15.5:0/1673344664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7659c640 1 --2- 172.21.15.5:0/1673344664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d382c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/1673344664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d382cd40 0x5592d382f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f76d9d640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f600a2cf0 0x7f6f600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7659c640 1 -- 172.21.15.5:0/1673344664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d382cd40 msgr2=0x5592d382f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7659c640 1 --2- 172.21.15.5:0/1673344664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d382cd40 0x5592d382f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7659c640 1 -- 172.21.15.5:0/1673344664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 msgr2=0x5592d3821f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f677fe640 1 -- 172.21.15.5:0/1527991057 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f6f700f00b0 con 0x7f6f7810c180 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7659c640 1 --2- 172.21.15.5:0/1673344664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3821f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f7659c640 1 -- 172.21.15.5:0/1673344664 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592d373b1f0 con 0x5592d3822480 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.922+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/1673344664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3821f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.923+0000 7f6f76d9d640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f600a2cf0 0x7f6f600a51b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6f700bc5a0 tx=0x7f6f7008f680 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.923+0000 7f6f7659c640 1 --2- 172.21.15.5:0/1673344664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d382c800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009c80 tx=0x7f6f68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7644a55fd079af4f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.923+0000 7f6f64ff9640 1 -- 172.21.15.5:0/1673344664 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68004250 con 0x5592d3822480 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.923+0000 7f6f64ff9640 1 -- 172.21.15.5:0/1673344664 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f68009e30 con 0x5592d3822480 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.923+0000 7f6f64ff9640 1 -- 172.21.15.5:0/1673344664 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68004dc0 con 0x5592d3822480 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.924+0000 7f6f7f312740 1 -- 172.21.15.5:0/1673344664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 msgr2=0x5592d382c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.924+0000 7f6f7f312740 1 --2- 172.21.15.5:0/1673344664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d382c800 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6f68009c80 tx=0x7f6f68011810 comp rx=0 tx=0).stop 2024-09-11T22:37:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.924+0000 7f6f7f312740 1 -- 172.21.15.5:0/1673344664 shutdown_connections 2024-09-11T22:37:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.924+0000 7f6f7f312740 1 --2- 172.21.15.5:0/1673344664 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d382cd40 0x5592d382f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.924+0000 7f6f7f312740 1 --2- 172.21.15.5:0/1673344664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d382c800 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.924+0000 7f6f7f312740 1 --2- 172.21.15.5:0/1673344664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3821f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.924+0000 7f6f7f312740 1 -- 172.21.15.5:0/1673344664 >> 172.21.15.5:0/1673344664 conn(0x5592d381e0f0 msgr2=0x5592d381e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.924+0000 7f6f7f312740 1 -- 172.21.15.5:0/1673344664 shutdown_connections 2024-09-11T22:37:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.924+0000 7f6f7f312740 1 -- 172.21.15.5:0/1673344664 wait complete. 2024-09-11T22:37:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f7f312740 1 -- start start 2024-09-11T22:37:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f7f312740 1 Processor -- start 2024-09-11T22:37:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3829b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d382a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d382cd40 0x5592d382a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f7f312740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5592d382f910 con 0x5592d3821b60 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f7f312740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5592d3741f10 con 0x5592d3822480 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f7f312740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5592d382fa90 con 0x5592d382cd40 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f7f312740 10 client.0 _ll_register_callbacks cb 0x5592d370aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f75d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3829b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f75d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3829b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43228/0 (socket says 172.21.15.5:43228) 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f75d9b640 1 -- 172.21.15.5:0/3020618531 learned_addr learned my addr 172.21.15.5:0/3020618531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f7659c640 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d382a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.925+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d382cd40 0x5592d382a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f7659c640 1 -- 172.21.15.5:0/3020618531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d382cd40 msgr2=0x5592d382a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f7659c640 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d382cd40 0x5592d382a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f7659c640 1 -- 172.21.15.5:0/3020618531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 msgr2=0x5592d3829b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f7659c640 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3829b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f7659c640 1 -- 172.21.15.5:0/3020618531 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592d34f86a0 con 0x5592d3822480 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3829b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f7659c640 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d382a0b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6f68011d10 tx=0x7f6f68005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daab6f786f9c531f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f5dffb640 1 -- 172.21.15.5:0/3020618531 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68029070 con 0x5592d3822480 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f5dffb640 1 -- 172.21.15.5:0/3020618531 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f6f680037f0 con 0x5592d3822480 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f7f312740 1 -- 172.21.15.5:0/3020618531 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f68009e30 con 0x5592d3822480 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.926+0000 7f6f5dffb640 1 -- 172.21.15.5:0/3020618531 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6f68035050 con 0x5592d3822480 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f7f312740 10 client.29366 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f7f312740 1 -- 172.21.15.5:0/3020618531 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5592d383b460 con 0x5592d3822480 2024-09-11T22:37:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f5dffb640 1 -- 172.21.15.5:0/3020618531 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f68009e30 con 0x5592d3822480 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f5dffb640 1 -- 172.21.15.5:0/3020618531 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f6f68005240 con 0x5592d3822480 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f7f312740 10 client.29366 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f7f312740 10 client.29366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f7f312740 20 client.29366 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f7f312740 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5592d383c7b0 0x5592d383eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f7f312740 4 client.29366 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f7f312740 1 -- 172.21.15.5:0/3020618531 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592d37947f0 con 0x5592d383c7b0 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.927+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5592d383c7b0 0x5592d383eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.928+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5592d383c7b0 0x5592d383eba0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=727d59e8bd3ff609 server_cookie=c70dc80f175ede01 in_seq=0 out_seq=0 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.928+0000 7f6f5dffb640 10 client.29366 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.931+0000 7f6f5dffb640 1 -- 172.21.15.5:0/3020618531 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592d37947f0 con 0x5592d383c7b0 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.931+0000 7f6f5dffb640 10 client.29366 handle_command_reply: tid=0 2024-09-11T22:37:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.931+0000 7f6f7f312740 1 client.29366 shutdown 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 1 -- 172.21.15.5:0/3020618531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 msgr2=0x5592d382a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d382a0b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6f68011d10 tx=0x7f6f68005d30 comp rx=0 tx=0).stop 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 1 -- 172.21.15.5:0/3020618531 shutdown_connections 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5592d383c7b0 0x5592d383eba0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d382cd40 0x5592d382a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d382a0b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 1 --2- 172.21.15.5:0/3020618531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3829b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f5dffb640 0 client.29366 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 1 -- 172.21.15.5:0/3020618531 >> 172.21.15.5:0/3020618531 conn(0x5592d381e0f0 msgr2=0x5592d381d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 1 -- 172.21.15.5:0/3020618531 shutdown_connections 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 1 -- 172.21.15.5:0/3020618531 wait complete. 2024-09-11T22:37:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.932+0000 7f6f7f312740 20 client.29366 trim_cache size 0 max 16384 2024-09-11T22:37:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.950+0000 7f6f7f312740 1 Processor -- start 2024-09-11T22:37:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.950+0000 7f6f7f312740 1 -- start start 2024-09-11T22:37:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.950+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d381e0f0 0x5592d38245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.950+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3844420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d3844960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f7f312740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f781afbd0 con 0x5592d3821b60 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f7f312740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f781b17d0 con 0x5592d3822480 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f7f312740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6f781b02a0 con 0x5592d381e0f0 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f7659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3844420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f75d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d381e0f0 0x5592d38245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f7659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3844420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43242/0 (socket says 172.21.15.5:43242) 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f7659c640 1 -- 172.21.15.5:0/2169953061 learned_addr learned my addr 172.21.15.5:0/2169953061 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/2169953061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d3844960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f75d9b640 1 -- 172.21.15.5:0/2169953061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 msgr2=0x5592d3844960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/2169953061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d3844960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f75d9b640 1 -- 172.21.15.5:0/2169953061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 msgr2=0x5592d3844420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/2169953061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3844420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f75d9b640 1 -- 172.21.15.5:0/2169953061 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592d381df20 con 0x5592d381e0f0 2024-09-11T22:37:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f7659c640 1 --2- 172.21.15.5:0/2169953061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3844420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/2169953061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d381e0f0 0x5592d38245c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0058c0 tx=0x7f6f6c012040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa101c64875e8a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.951+0000 7f6f5ffff640 1 -- 172.21.15.5:0/2169953061 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c005050 con 0x5592d381e0f0 2024-09-11T22:37:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f5ffff640 1 -- 172.21.15.5:0/2169953061 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c0046c0 con 0x5592d381e0f0 2024-09-11T22:37:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f5ffff640 1 -- 172.21.15.5:0/2169953061 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c010690 con 0x5592d381e0f0 2024-09-11T22:37:46.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f7f312740 1 -- 172.21.15.5:0/2169953061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d381e0f0 msgr2=0x5592d38245c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f7f312740 1 --2- 172.21.15.5:0/2169953061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d381e0f0 0x5592d38245c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0058c0 tx=0x7f6f6c012040 comp rx=0 tx=0).stop 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f7f312740 1 -- 172.21.15.5:0/2169953061 shutdown_connections 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f7f312740 1 --2- 172.21.15.5:0/2169953061 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d3822480 0x5592d3844960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f7f312740 1 --2- 172.21.15.5:0/2169953061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3844420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f7f312740 1 --2- 172.21.15.5:0/2169953061 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d381e0f0 0x5592d38245c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f7f312740 1 -- 172.21.15.5:0/2169953061 >> 172.21.15.5:0/2169953061 conn(0x5592d3825560 msgr2=0x5592d382bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f7f312740 1 -- 172.21.15.5:0/2169953061 shutdown_connections 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.952+0000 7f6f7f312740 1 -- 172.21.15.5:0/2169953061 wait complete. 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f7f312740 1 -- start start 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f7f312740 1 Processor -- start 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d381e0f0 0x5592d3830ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3831020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f7f312740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d3822480 0x5592d3831560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f7f312740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f781b1100 con 0x5592d3821b60 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f7f312740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f781b0970 con 0x5592d381e0f0 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f7f312740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6f781adbd0 con 0x5592d3822480 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f7f312740 10 client.0 _ll_register_callbacks cb 0x7f6f6c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f75d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d381e0f0 0x5592d3830ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f75d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d381e0f0 0x5592d3830ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36368/0 (socket says 172.21.15.5:36368) 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f75d9b640 1 -- 172.21.15.5:0/1011301077 learned_addr learned my addr 172.21.15.5:0/1011301077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f7659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3831020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.953+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d3822480 0x5592d3831560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f7659c640 1 -- 172.21.15.5:0/1011301077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d3822480 msgr2=0x5592d3831560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f7659c640 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d3822480 0x5592d3831560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f7659c640 1 -- 172.21.15.5:0/1011301077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d381e0f0 msgr2=0x5592d3830ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f7659c640 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d381e0f0 0x5592d3830ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f7659c640 1 -- 172.21.15.5:0/1011301077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592d38a64a0 con 0x5592d3821b60 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d381e0f0 0x5592d3830ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f7659c640 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3831020 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f6f68000c00 tx=0x7f6f68009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21f15874505d3b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f5dffb640 1 -- 172.21.15.5:0/1011301077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f6803e060 con 0x5592d3821b60 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f7f312740 1 -- 172.21.15.5:0/1011301077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f6c0046c0 con 0x5592d3821b60 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f5dffb640 1 -- 172.21.15.5:0/1011301077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f6f68023d10 con 0x5592d3821b60 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f5dffb640 1 -- 172.21.15.5:0/1011301077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f6f68029050 con 0x5592d3821b60 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f7f312740 10 client.20298 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.954+0000 7f6f7f312740 1 -- 172.21.15.5:0/1011301077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5592d38a77c0 con 0x5592d3821b60 2024-09-11T22:37:46.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f5dffb640 1 -- 172.21.15.5:0/1011301077 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f6c0046c0 con 0x5592d3821b60 2024-09-11T22:37:46.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f5dffb640 1 -- 172.21.15.5:0/1011301077 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f6f6802e440 con 0x5592d3821b60 2024-09-11T22:37:46.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f7f312740 10 client.20298 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:46.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f7f312740 10 client.20298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:46.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f7f312740 20 client.20298 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:46.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f7f312740 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5592d382cd40 0x5592d38a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:46.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f7f312740 4 client.20298 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:37:46.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f7f312740 1 -- 172.21.15.5:0/1011301077 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5592d381eca0 con 0x5592d382cd40 2024-09-11T22:37:46.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5592d382cd40 0x5592d38a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:46.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f75d9b640 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5592d382cd40 0x5592d38a9f90 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2db6517f82a4c2f server_cookie=99d0017f66e716ac in_seq=0 out_seq=0 2024-09-11T22:37:46.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.955+0000 7f6f5dffb640 10 client.20298 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:46.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.957+0000 7f6f5dffb640 1 -- 172.21.15.5:0/1011301077 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5592d381eca0 con 0x5592d382cd40 2024-09-11T22:37:46.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.957+0000 7f6f5dffb640 10 client.20298 handle_command_reply: tid=0 2024-09-11T22:37:46.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.957+0000 7f6f7f312740 1 client.20298 shutdown 2024-09-11T22:37:46.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 1 -- 172.21.15.5:0/1011301077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 msgr2=0x5592d3831020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3831020 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f6f68000c00 tx=0x7f6f68009b90 comp rx=0 tx=0).stop 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 1 -- 172.21.15.5:0/1011301077 shutdown_connections 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5592d382cd40 0x5592d38a9f90 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592d3822480 0x5592d3831560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592d3821b60 0x5592d3831020 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 1 --2- 172.21.15.5:0/1011301077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5592d381e0f0 0x5592d3830ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 1 -- 172.21.15.5:0/1011301077 >> 172.21.15.5:0/1011301077 conn(0x5592d3825560 msgr2=0x5592d382b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f5dffb640 0 client.20298 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 1 -- 172.21.15.5:0/1011301077 shutdown_connections 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 1 -- 172.21.15.5:0/1011301077 wait complete. 2024-09-11T22:37:46.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7f312740 20 client.20298 trim_cache size 0 max 16384 2024-09-11T22:37:46.964 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:37:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:37:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "cd188486-93c0-4815-95f2-d78aa3749f9c", 2024-09-11T22:37:46.964 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:37:46.964 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:37:46.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1527991057 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f600a2cf0 msgr2=0x7f6f600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7cf39640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f600a2cf0 0x7f6f600a51b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6f700bc5a0 tx=0x7f6f7008f680 comp rx=0 tx=0).stop 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1527991057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f7810c180 msgr2=0x7f6f78101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7cf39640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f7810c180 0x7f6f78101480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6f70005cc0 tx=0x7f6f7002f3a0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1527991057 shutdown_connections 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.958+0000 7f6f7cf39640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6f600a2cf0 0x7f6f600a51b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.959+0000 7f6f7cf39640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f7810c180 0x7f6f78101480 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.959+0000 7f6f7cf39640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f781036c0 0x7f6f78100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.959+0000 7f6f7cf39640 1 --2- 172.21.15.5:0/1527991057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6f78102da0 0x7f6f78100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.959+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1527991057 >> 172.21.15.5:0/1527991057 conn(0x7f6f780fc7a0 msgr2=0x7f6f780fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.959+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1527991057 shutdown_connections 2024-09-11T22:37:46.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:46.959+0000 7f6f7cf39640 1 -- 172.21.15.5:0/1527991057 wait complete. 2024-09-11T22:37:47.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 26952 19.37 MB/sec execute 5 sec latency 77.962 ms 2024-09-11T22:37:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:47 smithi005 ceph-mon[30943]: pgmap v496: 97 pgs: 97 active+clean; 401 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-09-11T22:37:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:47 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:37:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:47 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:47 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:37:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:47 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:37:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:47 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:47 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:37:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:47 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:47.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:47 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:37:47.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:47 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:47 smithi172 ceph-mon[35461]: pgmap v496: 97 pgs: 97 active+clean; 401 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-09-11T22:37:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:47 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:37:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:47 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:47 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:37:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:47 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:37:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:47 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:47 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:37:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:47 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:47 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:37:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:47 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:47.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:37:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:47 smithi159 ceph-mon[35534]: pgmap v496: 97 pgs: 97 active+clean; 401 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 12 MiB/s wr, 564 op/s 2024-09-11T22:37:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:47 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:37:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:47 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:47 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:37:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:47 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:37:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:47 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:47 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:37:48.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:47 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:47 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:37:48.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:47 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:37:48.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:48 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:37:48.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:48 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:48 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:37:48.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:48 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:48 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:37:48.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:48 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:48 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:37:48.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:48 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:48 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.638 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27594 24.53 MB/sec execute 6 sec latency 174.480 ms 2024-09-11T22:37:48.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:48 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:37:48.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:48 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:48 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:37:48.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:48 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:48 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:37:48.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:48 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:48 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:37:48.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:48 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:48 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.892+0000 7f3017cb5640 1 -- 172.21.15.5:0/3370118709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3010102570 msgr2=0x7f30101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.892+0000 7f3017cb5640 1 --2- 172.21.15.5:0/3370118709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3010102570 0x7f30101029b0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f3000009900 tx=0x7f300002a480 comp rx=0 tx=0).stop 2024-09-11T22:37:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.892+0000 7f3017cb5640 1 -- 172.21.15.5:0/3370118709 shutdown_connections 2024-09-11T22:37:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.892+0000 7f3017cb5640 1 --2- 172.21.15.5:0/3370118709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3010102ef0 0x7f301010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.892+0000 7f3017cb5640 1 --2- 172.21.15.5:0/3370118709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3010102570 0x7f30101029b0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.892+0000 7f3017cb5640 1 --2- 172.21.15.5:0/3370118709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3010100e20 0x7f3010101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.892+0000 7f3017cb5640 1 -- 172.21.15.5:0/3370118709 >> 172.21.15.5:0/3370118709 conn(0x7f30100fc7d0 msgr2=0x7f30100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:48.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.892+0000 7f3017cb5640 1 -- 172.21.15.5:0/3370118709 shutdown_connections 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.892+0000 7f3017cb5640 1 -- 172.21.15.5:0/3370118709 wait complete. 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3017cb5640 1 Processor -- start 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3017cb5640 1 -- start start 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3017cb5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3010100e20 0x7f30100fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3017cb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3010102570 0x7f30100fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3017cb5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3010102ef0 0x7f30100fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3017cb5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3010110f80 con 0x7f3010102570 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3017cb5640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3010110e00 con 0x7f3010102ef0 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3017cb5640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3010111100 con 0x7f3010100e20 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3015250640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3010102570 0x7f30100fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3015a51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3010100e20 0x7f30100fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3015250640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3010102570 0x7f30100fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43258/0 (socket says 172.21.15.5:43258) 2024-09-11T22:37:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3015250640 1 -- 172.21.15.5:0/2325420581 learned_addr learned my addr 172.21.15.5:0/2325420581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.893+0000 7f3015a51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3010100e20 0x7f30100fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:46042/0 (socket says 172.21.15.5:46042) 2024-09-11T22:37:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3016252640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3010102ef0 0x7f30100fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3015a51640 1 -- 172.21.15.5:0/2325420581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3010102ef0 msgr2=0x7f30100fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3015a51640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3010102ef0 0x7f30100fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3015a51640 1 -- 172.21.15.5:0/2325420581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3010102570 msgr2=0x7f30100fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3015a51640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3010102570 0x7f30100fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3015a51640 1 -- 172.21.15.5:0/2325420581 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30101a6a20 con 0x7f3010100e20 2024-09-11T22:37:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3015250640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3010102570 0x7f30100fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3015a51640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3010100e20 0x7f30100fcc80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f300c00e9f0 tx=0x7f300c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3006ffd640 1 -- 172.21.15.5:0/2325420581 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c00ccc0 con 0x7f3010100e20 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3006ffd640 1 -- 172.21.15.5:0/2325420581 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300c010910 con 0x7f3010100e20 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3017cb5640 1 -- 172.21.15.5:0/2325420581 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30101a6d10 con 0x7f3010100e20 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3006ffd640 1 -- 172.21.15.5:0/2325420581 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c018960 con 0x7f3010100e20 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.894+0000 7f3017cb5640 1 -- 172.21.15.5:0/2325420581 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30101a71f0 con 0x7f3010100e20 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f3006ffd640 1 -- 172.21.15.5:0/2325420581 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f300c010ac0 con 0x7f3010100e20 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f301a08e740 1 Processor -- start 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f301a08e740 1 -- start start 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f301a08e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a851880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f301a08e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a851dc0 0x555e9a85c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f301a08e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 0x555e9a85ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f301a08e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e9a7b72e0 con 0x555e9a8514a0 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f301a08e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555e9a75ece0 con 0x555e9a85c680 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f301a08e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e9a7b32a0 con 0x555e9a851dc0 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f3016252640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a851880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f3016252640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a851880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43264/0 (socket says 172.21.15.5:43264) 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f3016252640 1 -- 172.21.15.5:0/671232840 learned_addr learned my addr 172.21.15.5:0/671232840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f3015a51640 1 --2- 172.21.15.5:0/671232840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a851dc0 0x555e9a85c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.897+0000 7f3015250640 1 --2- 172.21.15.5:0/671232840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 0x555e9a85ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f3006ffd640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2ffc0a2cf0 0x7f2ffc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f3016252640 1 -- 172.21.15.5:0/671232840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a851dc0 msgr2=0x555e9a85c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f3016252640 1 --2- 172.21.15.5:0/671232840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a851dc0 0x555e9a85c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f3016252640 1 -- 172.21.15.5:0/671232840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 msgr2=0x555e9a85ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:48.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f3016252640 1 --2- 172.21.15.5:0/671232840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 0x555e9a85ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f3016252640 1 -- 172.21.15.5:0/671232840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e9a743440 con 0x555e9a8514a0 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f3015250640 1 --2- 172.21.15.5:0/671232840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 0x555e9a85ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f3006ffd640 1 -- 172.21.15.5:0/2325420581 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f300c0cf890 con 0x7f3010100e20 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f3016252640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2ffc0a2cf0 0x7f2ffc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f3016252640 1 --2- 172.21.15.5:0/671232840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a851880 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3008011590 tx=0x7f3008011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c99b07e29c3e43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f2ffbfff640 1 -- 172.21.15.5:0/671232840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3008017050 con 0x555e9a8514a0 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f2ffbfff640 1 -- 172.21.15.5:0/671232840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300801bb90 con 0x555e9a8514a0 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.898+0000 7f2ffbfff640 1 -- 172.21.15.5:0/671232840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300800e3b0 con 0x555e9a8514a0 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.899+0000 7f3016252640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2ffc0a2cf0 0x7f2ffc0a51b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3008022000 tx=0x7f3008023000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.899+0000 7f301a08e740 1 -- 172.21.15.5:0/671232840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 msgr2=0x555e9a851880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:48.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.899+0000 7f301a08e740 1 --2- 172.21.15.5:0/671232840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a851880 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f3008011590 tx=0x7f3008011a50 comp rx=0 tx=0).stop 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.899+0000 7f301a08e740 1 -- 172.21.15.5:0/671232840 shutdown_connections 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.899+0000 7f301a08e740 1 --2- 172.21.15.5:0/671232840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 0x555e9a85ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.899+0000 7f301a08e740 1 --2- 172.21.15.5:0/671232840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a851dc0 0x555e9a85c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.899+0000 7f301a08e740 1 --2- 172.21.15.5:0/671232840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a851880 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.899+0000 7f301a08e740 1 -- 172.21.15.5:0/671232840 >> 172.21.15.5:0/671232840 conn(0x555e9a84da30 msgr2=0x555e9a84de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.899+0000 7f301a08e740 1 -- 172.21.15.5:0/671232840 shutdown_connections 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.899+0000 7f301a08e740 1 -- 172.21.15.5:0/671232840 wait complete. 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f301a08e740 1 -- start start 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f301a08e740 1 Processor -- start 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f301a08e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a85bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f301a08e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a851dc0 0x555e9a859210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f301a08e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a85c680 0x555e9a859750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f301a08e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e9a85f250 con 0x555e9a8514a0 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f301a08e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555e9a772050 con 0x555e9a851dc0 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f301a08e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e9a85f3d0 con 0x555e9a85c680 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f301a08e740 10 client.0 _ll_register_callbacks cb 0x555e9a7b2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f3015a51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a85bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f3015250640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a851dc0 0x555e9a859210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f3016252640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a85c680 0x555e9a859750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f3015a51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a85bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43276/0 (socket says 172.21.15.5:43276) 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f3015a51640 1 -- 172.21.15.5:0/3822593846 learned_addr learned my addr 172.21.15.5:0/3822593846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.900+0000 7f3015250640 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a851dc0 0x555e9a859210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36420/0 (socket says 172.21.15.5:36420) 2024-09-11T22:37:48.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f3015a51640 1 -- 172.21.15.5:0/3822593846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a85c680 msgr2=0x555e9a859750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f3015a51640 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a85c680 0x555e9a859750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f3015a51640 1 -- 172.21.15.5:0/3822593846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a851dc0 msgr2=0x555e9a859210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f3015a51640 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a851dc0 0x555e9a859210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f3015a51640 1 -- 172.21.15.5:0/3822593846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e9a5283c0 con 0x555e9a8514a0 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f3015250640 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a851dc0 0x555e9a859210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f3016252640 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a85c680 0x555e9a859750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f3015a51640 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a85bf60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f300c00e9c0 tx=0x7f300c010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db052022eef0a444 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f2ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f2ff97fa640 1 -- 172.21.15.5:0/3822593846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c09b6f0 con 0x555e9a8514a0 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f2ff97fa640 1 -- 172.21.15.5:0/3822593846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f300c09b3e0 con 0x555e9a8514a0 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f301a08e740 1 -- 172.21.15.5:0/3822593846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f300801bb90 con 0x555e9a8514a0 2024-09-11T22:37:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f2ff97fa640 1 -- 172.21.15.5:0/3822593846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f300c0cd3a0 con 0x555e9a8514a0 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f301a08e740 10 client.20310 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.901+0000 7f301a08e740 1 -- 172.21.15.5:0/3822593846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x555e9a86abc0 con 0x555e9a8514a0 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f2ff97fa640 1 -- 172.21.15.5:0/3822593846 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f300801bb90 con 0x555e9a8514a0 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f2ff97fa640 1 -- 172.21.15.5:0/3822593846 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f300c097ae0 con 0x555e9a8514a0 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f301a08e740 10 client.20310 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f301a08e740 10 client.20310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f301a08e740 20 client.20310 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f301a08e740 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e9a86bfd0 0x555e9a86e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f301a08e740 4 client.20310 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f301a08e740 1 -- 172.21.15.5:0/3822593846 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e9a7c4850 con 0x555e9a86bfd0 2024-09-11T22:37:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f3015250640 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e9a86bfd0 0x555e9a86e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f2ff97fa640 10 client.20310 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.902+0000 7f3015250640 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e9a86bfd0 0x555e9a86e490 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc71ea50c400816 server_cookie=34baa876c22a50c9 in_seq=0 out_seq=0 2024-09-11T22:37:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f2ff97fa640 1 -- 172.21.15.5:0/3822593846 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e9a7c4850 con 0x555e9a86bfd0 2024-09-11T22:37:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f2ff97fa640 10 client.20310 handle_command_reply: tid=0 2024-09-11T22:37:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 client.20310 shutdown 2024-09-11T22:37:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 -- 172.21.15.5:0/3822593846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 msgr2=0x555e9a85bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a85bf60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f300c00e9c0 tx=0x7f300c010ee0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 -- 172.21.15.5:0/3822593846 shutdown_connections 2024-09-11T22:37:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e9a86bfd0 0x555e9a86e490 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a85c680 0x555e9a859750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a851dc0 0x555e9a859210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 --2- 172.21.15.5:0/3822593846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a8514a0 0x555e9a85bf60 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f2ff97fa640 0 client.20310 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 -- 172.21.15.5:0/3822593846 >> 172.21.15.5:0/3822593846 conn(0x555e9a84da30 msgr2=0x555e9a84cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 -- 172.21.15.5:0/3822593846 shutdown_connections 2024-09-11T22:37:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 1 -- 172.21.15.5:0/3822593846 wait complete. 2024-09-11T22:37:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.904+0000 7f301a08e740 20 client.20310 trim_cache size 0 max 16384 2024-09-11T22:37:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f301a08e740 1 Processor -- start 2024-09-11T22:37:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f301a08e740 1 -- start start 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f301a08e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a8514a0 0x555e9a8684b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f301a08e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a851dc0 0x555e9a873db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f301a08e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 0x555e9a8742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f301a08e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30101afdc0 con 0x555e9a851dc0 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f301a08e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30101b19c0 con 0x555e9a85c680 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f301a08e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30101b0490 con 0x555e9a8514a0 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f3015250640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a851dc0 0x555e9a873db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f3015a51640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a8514a0 0x555e9a8684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f3015250640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a851dc0 0x555e9a873db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43280/0 (socket says 172.21.15.5:43280) 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f3015250640 1 -- 172.21.15.5:0/2777773104 learned_addr learned my addr 172.21.15.5:0/2777773104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.924+0000 7f3016252640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 0x555e9a8742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f3015250640 1 -- 172.21.15.5:0/2777773104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a8514a0 msgr2=0x555e9a8684b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f3015250640 1 --2- 172.21.15.5:0/2777773104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a8514a0 0x555e9a8684b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f3015250640 1 -- 172.21.15.5:0/2777773104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 msgr2=0x555e9a8742f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f3015250640 1 --2- 172.21.15.5:0/2777773104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 0x555e9a8742f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f3015250640 1 -- 172.21.15.5:0/2777773104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e9a84d4d0 con 0x555e9a851dc0 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f3016252640 1 --2- 172.21.15.5:0/2777773104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 0x555e9a8742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f3015a51640 1 --2- 172.21.15.5:0/2777773104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a8514a0 0x555e9a8684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f3015250640 1 --2- 172.21.15.5:0/2777773104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a851dc0 0x555e9a873db0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f3000004650 tx=0x7f3000005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3e9fa77fd05059 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f2ffb7fe640 1 -- 172.21.15.5:0/2777773104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000035040 con 0x555e9a851dc0 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f2ffb7fe640 1 -- 172.21.15.5:0/2777773104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30000054b0 con 0x555e9a851dc0 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.925+0000 7f2ffb7fe640 1 -- 172.21.15.5:0/2777773104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3000003be0 con 0x555e9a851dc0 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.926+0000 7f301a08e740 1 -- 172.21.15.5:0/2777773104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a851dc0 msgr2=0x555e9a873db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.926+0000 7f301a08e740 1 --2- 172.21.15.5:0/2777773104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a851dc0 0x555e9a873db0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f3000004650 tx=0x7f3000005180 comp rx=0 tx=0).stop 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.926+0000 7f301a08e740 1 -- 172.21.15.5:0/2777773104 shutdown_connections 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.926+0000 7f301a08e740 1 --2- 172.21.15.5:0/2777773104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a85c680 0x555e9a8742f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.926+0000 7f301a08e740 1 --2- 172.21.15.5:0/2777773104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a851dc0 0x555e9a873db0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.926+0000 7f301a08e740 1 --2- 172.21.15.5:0/2777773104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a8514a0 0x555e9a8684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.926+0000 7f301a08e740 1 -- 172.21.15.5:0/2777773104 >> 172.21.15.5:0/2777773104 conn(0x555e9a84da30 msgr2=0x555e9a857ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.926+0000 7f301a08e740 1 -- 172.21.15.5:0/2777773104 shutdown_connections 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.927+0000 7f301a08e740 1 -- 172.21.15.5:0/2777773104 wait complete. 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.927+0000 7f301a08e740 1 -- start start 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.927+0000 7f301a08e740 1 Processor -- start 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.927+0000 7f301a08e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a8514a0 0x555e9a864af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.927+0000 7f301a08e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a851dc0 0x555e9a85fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.927+0000 7f301a08e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a85c680 0x555e9a860360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.927+0000 7f301a08e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30101b12f0 con 0x555e9a85c680 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.927+0000 7f301a08e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30101b0b60 con 0x555e9a8514a0 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.927+0000 7f301a08e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f30101addc0 con 0x555e9a851dc0 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.927+0000 7f301a08e740 10 client.0 _ll_register_callbacks cb 0x7f300c00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3015a51640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a8514a0 0x555e9a864af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3015250640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a851dc0 0x555e9a85fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3016252640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a85c680 0x555e9a860360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3015a51640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a8514a0 0x555e9a864af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36436/0 (socket says 172.21.15.5:36436) 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3015a51640 1 -- 172.21.15.5:0/1929595048 learned_addr learned my addr 172.21.15.5:0/1929595048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3015a51640 1 -- 172.21.15.5:0/1929595048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a851dc0 msgr2=0x555e9a85fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3015a51640 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a851dc0 0x555e9a85fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3015a51640 1 -- 172.21.15.5:0/1929595048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a85c680 msgr2=0x555e9a860360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3015a51640 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a85c680 0x555e9a860360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3015a51640 1 -- 172.21.15.5:0/1929595048 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e9a860920 con 0x555e9a8514a0 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f2ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f3015a51640 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a8514a0 0x555e9a864af0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f300c098760 tx=0x7f300c09b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfe50ad2ecc2903 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f2ff97fa640 1 -- 172.21.15.5:0/1929595048 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c09bd90 con 0x555e9a8514a0 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f301a08e740 1 -- 172.21.15.5:0/1929595048 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30000054b0 con 0x555e9a8514a0 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.928+0000 7f2ff97fa640 1 -- 172.21.15.5:0/1929595048 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f300c018b00 con 0x555e9a8514a0 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f2ff97fa640 1 -- 172.21.15.5:0/1929595048 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f300c0a4950 con 0x555e9a8514a0 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f301a08e740 10 client.29390 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f301a08e740 1 -- 172.21.15.5:0/1929595048 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x555e9a8d6b00 con 0x555e9a8514a0 2024-09-11T22:37:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f2ff97fa640 1 -- 172.21.15.5:0/1929595048 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30000054b0 con 0x555e9a8514a0 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f2ff97fa640 1 -- 172.21.15.5:0/1929595048 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f300c07ac40 con 0x555e9a8514a0 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f301a08e740 10 client.29390 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f301a08e740 10 client.29390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f301a08e740 20 client.29390 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f301a08e740 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e9a8d7300 0x555e9a8d9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f301a08e740 4 client.29390 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.929+0000 7f301a08e740 1 -- 172.21.15.5:0/1929595048 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e9a859ae0 con 0x555e9a8d7300 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.930+0000 7f3015250640 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e9a8d7300 0x555e9a8d9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.930+0000 7f2ff97fa640 10 client.29390 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.930+0000 7f3015250640 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e9a8d7300 0x555e9a8d9720 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963ff43a2878ee13 server_cookie=31091621d2f4816d in_seq=0 out_seq=0 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.931+0000 7f2ff97fa640 1 -- 172.21.15.5:0/1929595048 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555e9a859ae0 con 0x555e9a8d7300 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.931+0000 7f2ff97fa640 10 client.29390 handle_command_reply: tid=0 2024-09-11T22:37:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.931+0000 7f301a08e740 1 client.29390 shutdown 2024-09-11T22:37:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.931+0000 7f301a08e740 1 -- 172.21.15.5:0/1929595048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a8514a0 msgr2=0x555e9a864af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f301a08e740 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a8514a0 0x555e9a864af0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f300c098760 tx=0x7f300c09b410 comp rx=0 tx=0).stop 2024-09-11T22:37:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f301a08e740 1 -- 172.21.15.5:0/1929595048 shutdown_connections 2024-09-11T22:37:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f301a08e740 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e9a8d7300 0x555e9a8d9720 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f301a08e740 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e9a85c680 0x555e9a860360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f301a08e740 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e9a851dc0 0x555e9a85fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f2ff97fa640 0 client.29390 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f301a08e740 1 --2- 172.21.15.5:0/1929595048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e9a8514a0 0x555e9a864af0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f301a08e740 1 -- 172.21.15.5:0/1929595048 >> 172.21.15.5:0/1929595048 conn(0x555e9a84da30 msgr2=0x555e9a84c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f301a08e740 1 -- 172.21.15.5:0/1929595048 shutdown_connections 2024-09-11T22:37:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f301a08e740 1 -- 172.21.15.5:0/1929595048 wait complete. 2024-09-11T22:37:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f301a08e740 20 client.29390 trim_cache size 0 max 16384 2024-09-11T22:37:48.945 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:37:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:37:48.945 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:37:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "cd188486-93c0-4815-95f2-d78aa3749f9c": { 2024-09-11T22:37:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:37:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "cd188486-93c0-4815-95f2-d78aa3749f9c", 2024-09-11T22:37:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:37:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:37:48.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:37:48.946 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:37:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f3017cb5640 1 -- 172.21.15.5:0/2325420581 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2ffc0a2cf0 msgr2=0x7f2ffc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f3017cb5640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2ffc0a2cf0 0x7f2ffc0a51b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3008022000 tx=0x7f3008023000 comp rx=0 tx=0).stop 2024-09-11T22:37:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f3017cb5640 1 -- 172.21.15.5:0/2325420581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3010100e20 msgr2=0x7f30100fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f3017cb5640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3010100e20 0x7f30100fcc80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f300c00e9f0 tx=0x7f300c00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f3017cb5640 1 -- 172.21.15.5:0/2325420581 shutdown_connections 2024-09-11T22:37:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f3017cb5640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2ffc0a2cf0 0x7f2ffc0a51b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f3017cb5640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3010102ef0 0x7f30100fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f3017cb5640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3010102570 0x7f30100fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f3017cb5640 1 --2- 172.21.15.5:0/2325420581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3010100e20 0x7f30100fcc80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:48.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.932+0000 7f3017cb5640 1 -- 172.21.15.5:0/2325420581 >> 172.21.15.5:0/2325420581 conn(0x7f30100fc7d0 msgr2=0x7f301006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:48.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.933+0000 7f3017cb5640 1 -- 172.21.15.5:0/2325420581 shutdown_connections 2024-09-11T22:37:48.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:37:48.933+0000 7f3017cb5640 1 -- 172.21.15.5:0/2325420581 wait complete. 2024-09-11T22:37:49.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:48 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:37:49.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:48 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:49.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:48 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:37:49.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:48 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:49.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:48 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:37:49.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:48 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:49.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:48 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:37:49.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:48 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:49.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:48 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:37:49.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28050 24.15 MB/sec execute 7 sec latency 274.752 ms 2024-09-11T22:37:49.770 INFO:tasks.cephfs.filesystem:scrub status for tag:cd188486-93c0-4815-95f2-d78aa3749f9c - {'path': '/', 'tag': 'cd188486-93c0-4815-95f2-d78aa3749f9c', 'options': 'recursive,force'} 2024-09-11T22:37:49.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:49 smithi172 ceph-mon[35461]: pgmap v497: 97 pgs: 97 active+clean; 392 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 845 op/s 2024-09-11T22:37:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:49 smithi005 ceph-mon[30943]: pgmap v497: 97 pgs: 97 active+clean; 392 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 845 op/s 2024-09-11T22:37:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:49 smithi159 ceph-mon[35534]: pgmap v497: 97 pgs: 97 active+clean; 392 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 845 op/s 2024-09-11T22:37:50.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28555 25.09 MB/sec execute 8 sec latency 199.000 ms 2024-09-11T22:37:51.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28958 24.88 MB/sec execute 9 sec latency 177.051 ms 2024-09-11T22:37:51.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:51 smithi172 ceph-mon[35461]: pgmap v498: 97 pgs: 97 active+clean; 392 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.1 MiB/s wr, 479 op/s 2024-09-11T22:37:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:51 smithi005 ceph-mon[30943]: pgmap v498: 97 pgs: 97 active+clean; 392 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.1 MiB/s wr, 479 op/s 2024-09-11T22:37:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:51 smithi159 ceph-mon[35534]: pgmap v498: 97 pgs: 97 active+clean; 392 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.1 MiB/s wr, 479 op/s 2024-09-11T22:37:52.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29396 24.10 MB/sec execute 10 sec latency 167.844 ms 2024-09-11T22:37:53.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29880 24.20 MB/sec execute 11 sec latency 154.550 ms 2024-09-11T22:37:53.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:53 smithi172 ceph-mon[35461]: pgmap v499: 97 pgs: 97 active+clean; 406 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 646 op/s 2024-09-11T22:37:54.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:53 smithi005 ceph-mon[30943]: pgmap v499: 97 pgs: 97 active+clean; 406 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 646 op/s 2024-09-11T22:37:54.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:53 smithi159 ceph-mon[35534]: pgmap v499: 97 pgs: 97 active+clean; 406 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.5 MiB/s wr, 646 op/s 2024-09-11T22:37:54.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 30407 24.37 MB/sec execute 12 sec latency 62.552 ms 2024-09-11T22:37:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:54 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:37:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:54 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:37:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:54 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:37:54.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:54 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:37:54.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:54 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:37:54.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:54 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:37:54.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:54 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:37:54.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:54 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:37:55.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:54 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:37:55.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:54 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:37:55.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:54 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:37:55.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:54 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:37:55.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31148 25.08 MB/sec execute 13 sec latency 166.977 ms 2024-09-11T22:37:55.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:55 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:37:55.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:55 smithi172 ceph-mon[35461]: pgmap v500: 97 pgs: 97 active+clean; 406 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.0 MiB/s wr, 447 op/s 2024-09-11T22:37:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:55 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:37:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:55 smithi005 ceph-mon[30943]: pgmap v500: 97 pgs: 97 active+clean; 406 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.0 MiB/s wr, 447 op/s 2024-09-11T22:37:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:55 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:37:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:55 smithi159 ceph-mon[35534]: pgmap v500: 97 pgs: 97 active+clean; 406 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.0 MiB/s wr, 447 op/s 2024-09-11T22:37:56.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31757 26.86 MB/sec execute 14 sec latency 151.281 ms 2024-09-11T22:37:57.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32322 26.98 MB/sec execute 15 sec latency 177.900 ms 2024-09-11T22:37:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.795+0000 7f814caa6640 1 -- 172.21.15.5:0/1696715190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81481069c0 msgr2=0x7f8148106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.795+0000 7f814caa6640 1 --2- 172.21.15.5:0/1696715190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81481069c0 0x7f8148106da0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f8138009930 tx=0x7f813802a420 comp rx=0 tx=0).stop 2024-09-11T22:37:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.795+0000 7f814caa6640 1 -- 172.21.15.5:0/1696715190 shutdown_connections 2024-09-11T22:37:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.795+0000 7f814caa6640 1 --2- 172.21.15.5:0/1696715190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8148101350 0x7f814810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.795+0000 7f814caa6640 1 --2- 172.21.15.5:0/1696715190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81481009b0 0x7f8148100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.795+0000 7f814caa6640 1 --2- 172.21.15.5:0/1696715190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81481069c0 0x7f8148106da0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.795+0000 7f814caa6640 1 -- 172.21.15.5:0/1696715190 >> 172.21.15.5:0/1696715190 conn(0x7f8148077990 msgr2=0x7f81480fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.795+0000 7f814caa6640 1 -- 172.21.15.5:0/1696715190 shutdown_connections 2024-09-11T22:37:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.795+0000 7f814caa6640 1 -- 172.21.15.5:0/1696715190 wait complete. 2024-09-11T22:37:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.795+0000 7f814caa6640 1 Processor -- start 2024-09-11T22:37:57.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814caa6640 1 -- start start 2024-09-11T22:37:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814caa6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81481009b0 0x7f814819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814caa6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8148101350 0x7f814819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814caa6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81481069c0 0x7f814819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814caa6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8148110e50 con 0x7f81481009b0 2024-09-11T22:37:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814caa6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8148110cd0 con 0x7f8148101350 2024-09-11T22:37:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814caa6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8148110fd0 con 0x7f81481069c0 2024-09-11T22:37:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81481009b0 0x7f814819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81481009b0 0x7f814819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53834/0 (socket says 172.21.15.5:53834) 2024-09-11T22:37:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814659c640 1 -- 172.21.15.5:0/3224443428 learned_addr learned my addr 172.21.15.5:0/3224443428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f813dd9b640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8148101350 0x7f814819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f8146d9d640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81481069c0 0x7f814819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f8146d9d640 1 -- 172.21.15.5:0/3224443428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8148101350 msgr2=0x7f814819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f8146d9d640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8148101350 0x7f814819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f8146d9d640 1 -- 172.21.15.5:0/3224443428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81481009b0 msgr2=0x7f814819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f8146d9d640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81481009b0 0x7f814819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f8146d9d640 1 -- 172.21.15.5:0/3224443428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814819f520 con 0x7f81481069c0 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.796+0000 7f814659c640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81481009b0 0x7f814819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.797+0000 7f8146d9d640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81481069c0 0x7f814819edf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8140009fd0 tx=0x7f814000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.797+0000 7f813ffff640 1 -- 172.21.15.5:0/3224443428 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814000ce20 con 0x7f81481069c0 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.797+0000 7f813ffff640 1 -- 172.21.15.5:0/3224443428 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8140005020 con 0x7f81481069c0 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.797+0000 7f814caa6640 1 -- 172.21.15.5:0/3224443428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814819f810 con 0x7f81481069c0 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.797+0000 7f813ffff640 1 -- 172.21.15.5:0/3224443428 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140010720 con 0x7f81481069c0 2024-09-11T22:37:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.797+0000 7f814caa6640 1 -- 172.21.15.5:0/3224443428 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81481a70a0 con 0x7f81481069c0 2024-09-11T22:37:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.799+0000 7f813ffff640 1 -- 172.21.15.5:0/3224443428 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8140005540 con 0x7f81481069c0 2024-09-11T22:37:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.799+0000 7f814f83a740 1 Processor -- start 2024-09-11T22:37:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f814f83a740 1 -- start start 2024-09-11T22:37:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f814f83a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145dfaf0 0x5627145dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f814f83a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145ea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f814f83a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145eacb0 0x5627145ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f814f83a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5627145442f0 con 0x5627145dfaf0 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f814f83a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5627144ec270 con 0x5627145eacb0 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f814f83a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5627145402b0 con 0x5627145e0410 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f813dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145dfaf0 0x5627145dfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f814659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f813ffff640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81340a2da0 0x7f81340a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f813dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145dfaf0 0x5627145dfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53836/0 (socket says 172.21.15.5:53836) 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f814659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145ea770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:55144/0 (socket says 172.21.15.5:55144) 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f813dd9b640 1 -- 172.21.15.5:0/1080986661 learned_addr learned my addr 172.21.15.5:0/1080986661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1080986661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145eacb0 0x5627145ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f8146d9d640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81340a2da0 0x7f81340a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.800+0000 7f813ffff640 1 -- 172.21.15.5:0/3224443428 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f81400cfb30 con 0x7f81481069c0 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.801+0000 7f813dd9b640 1 -- 172.21.15.5:0/1080986661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 msgr2=0x5627145ea770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.801+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1080986661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145ea770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.801+0000 7f813dd9b640 1 -- 172.21.15.5:0/1080986661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145eacb0 msgr2=0x5627145ed0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.801+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1080986661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145eacb0 0x5627145ed0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.801+0000 7f813dd9b640 1 -- 172.21.15.5:0/1080986661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627144eb340 con 0x5627145dfaf0 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.801+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1080986661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145dfaf0 0x5627145dfed0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f8130013630 tx=0x7f8130005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d87de8dd02ac80e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.801+0000 7f8146d9d640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81340a2da0 0x7f81340a5260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f81400cd2c0 tx=0x7f81400bc040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.801+0000 7f813cd99640 1 -- 172.21.15.5:0/1080986661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813000f500 con 0x5627145dfaf0 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.801+0000 7f813cd99640 1 -- 172.21.15.5:0/1080986661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813000fb80 con 0x5627145dfaf0 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.801+0000 7f813cd99640 1 -- 172.21.15.5:0/1080986661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130010b00 con 0x5627145dfaf0 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.802+0000 7f814f83a740 1 -- 172.21.15.5:0/1080986661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145dfaf0 msgr2=0x5627145dfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:57.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.802+0000 7f814f83a740 1 --2- 172.21.15.5:0/1080986661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145dfaf0 0x5627145dfed0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f8130013630 tx=0x7f8130005f90 comp rx=0 tx=0).stop 2024-09-11T22:37:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.802+0000 7f814f83a740 1 -- 172.21.15.5:0/1080986661 shutdown_connections 2024-09-11T22:37:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.802+0000 7f814f83a740 1 --2- 172.21.15.5:0/1080986661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145eacb0 0x5627145ed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.802+0000 7f814f83a740 1 --2- 172.21.15.5:0/1080986661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145ea770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.802+0000 7f814f83a740 1 --2- 172.21.15.5:0/1080986661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145dfaf0 0x5627145dfed0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.802+0000 7f814f83a740 1 -- 172.21.15.5:0/1080986661 >> 172.21.15.5:0/1080986661 conn(0x5627145dc060 msgr2=0x5627145dc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.802+0000 7f814f83a740 1 -- 172.21.15.5:0/1080986661 shutdown_connections 2024-09-11T22:37:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.802+0000 7f814f83a740 1 -- 172.21.15.5:0/1080986661 wait complete. 2024-09-11T22:37:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814f83a740 1 -- start start 2024-09-11T22:37:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814f83a740 1 Processor -- start 2024-09-11T22:37:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814f83a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 0x5627145e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814f83a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145e8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814f83a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x5627145e8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814f83a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5627145ed880 con 0x5627145eacb0 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814f83a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5627144cd290 con 0x5627145dfaf0 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814f83a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5627145eda00 con 0x5627145e0410 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814f83a740 10 client.0 _ll_register_callbacks cb 0x5627144fe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f813dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x5627145e8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145e8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:55160/0 (socket says 172.21.15.5:55160) 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814659c640 1 -- 172.21.15.5:0/1573467095 learned_addr learned my addr 172.21.15.5:0/1573467095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 0x5627145e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f813dd9b640 1 -- 172.21.15.5:0/1573467095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 msgr2=0x5627145e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f813dd9b640 1 -- 172.21.15.5:0/1573467095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 msgr2=0x5627145e7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 0x5627145e7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f813dd9b640 1 -- 172.21.15.5:0/1573467095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562714257210 con 0x5627145eacb0 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.803+0000 7f814659c640 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x5627145e8540 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f8130009730 tx=0x7f81300106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdb0f23ee1280cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f812dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f812dffb640 1 -- 172.21.15.5:0/1573467095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130004260 con 0x5627145eacb0 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f812dffb640 1 -- 172.21.15.5:0/1573467095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f8130004400 con 0x5627145eacb0 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f814f83a740 1 -- 172.21.15.5:0/1573467095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f813000fb80 con 0x5627145eacb0 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f812dffb640 1 -- 172.21.15.5:0/1573467095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8130033050 con 0x5627145eacb0 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f814f83a740 10 client.20328 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f814f83a740 1 -- 172.21.15.5:0/1573467095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x562714323620 con 0x5627145eacb0 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f812dffb640 1 -- 172.21.15.5:0/1573467095 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813000fb80 con 0x5627145eacb0 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f812dffb640 1 -- 172.21.15.5:0/1573467095 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f81300035f0 con 0x5627145eacb0 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f814f83a740 10 client.20328 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f814f83a740 10 client.20328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f814f83a740 20 client.20328 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f814f83a740 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5627145fa9e0 0x5627145fce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f814f83a740 4 client.20328 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.804+0000 7f814f83a740 1 -- 172.21.15.5:0/1573467095 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562714551a20 con 0x5627145fa9e0 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.805+0000 7f814659c640 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5627145fa9e0 0x5627145fce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.805+0000 7f812dffb640 10 client.20328 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.805+0000 7f814659c640 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5627145fa9e0 0x5627145fce00 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53ac50785cfe48c2 server_cookie=c3a5757438f1a8b4 in_seq=0 out_seq=0 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f812dffb640 1 -- 172.21.15.5:0/1573467095 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562714551a20 con 0x5627145fa9e0 2024-09-11T22:37:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f812dffb640 10 client.20328 handle_command_reply: tid=0 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f814f83a740 1 client.20328 shutdown 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f814f83a740 1 -- 172.21.15.5:0/1573467095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 msgr2=0x5627145e8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f814f83a740 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x5627145e8540 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f8130009730 tx=0x7f81300106e0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f814f83a740 1 -- 172.21.15.5:0/1573467095 shutdown_connections 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f814f83a740 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5627145fa9e0 0x5627145fce00 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f814f83a740 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x5627145e8540 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f814f83a740 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145e0410 0x5627145e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f814f83a740 1 --2- 172.21.15.5:0/1573467095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 0x5627145e7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f812dffb640 0 client.20328 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.806+0000 7f814f83a740 1 -- 172.21.15.5:0/1573467095 >> 172.21.15.5:0/1573467095 conn(0x5627145dc060 msgr2=0x5627145e1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.807+0000 7f814f83a740 1 -- 172.21.15.5:0/1573467095 shutdown_connections 2024-09-11T22:37:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.807+0000 7f814f83a740 1 -- 172.21.15.5:0/1573467095 wait complete. 2024-09-11T22:37:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.807+0000 7f814f83a740 20 client.20328 trim_cache size 0 max 16384 2024-09-11T22:37:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.822+0000 7f814f83a740 1 Processor -- start 2024-09-11T22:37:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.822+0000 7f814f83a740 1 -- start start 2024-09-11T22:37:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f814f83a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145dfaf0 0x5627145e8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f814f83a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145e0410 0x562714602760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f814f83a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x562714602ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f814f83a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81481afb70 con 0x5627145eacb0 2024-09-11T22:37:57.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f814f83a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81481b1770 con 0x5627145e0410 2024-09-11T22:37:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f814f83a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81481b0240 con 0x5627145dfaf0 2024-09-11T22:37:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f813dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x562714602ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f814659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145e0410 0x562714602760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f813dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x562714602ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53852/0 (socket says 172.21.15.5:53852) 2024-09-11T22:37:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f814659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145e0410 0x562714602760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52512/0 (socket says 172.21.15.5:52512) 2024-09-11T22:37:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f813dd9b640 1 -- 172.21.15.5:0/2784355900 learned_addr learned my addr 172.21.15.5:0/2784355900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f813dd9b640 1 --2- 172.21.15.5:0/2784355900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145dfaf0 0x5627145e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f813dd9b640 1 -- 172.21.15.5:0/2784355900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145dfaf0 msgr2=0x5627145e8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f813dd9b640 1 --2- 172.21.15.5:0/2784355900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145dfaf0 0x5627145e8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f813dd9b640 1 -- 172.21.15.5:0/2784355900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145e0410 msgr2=0x562714602760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f813dd9b640 1 --2- 172.21.15.5:0/2784355900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145e0410 0x562714602760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f813dd9b640 1 -- 172.21.15.5:0/2784355900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627145fd480 con 0x5627145eacb0 2024-09-11T22:37:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f814659c640 1 --2- 172.21.15.5:0/2784355900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145e0410 0x562714602760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:37:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f813dd9b640 1 --2- 172.21.15.5:0/2784355900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x562714602ca0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f81300098e0 tx=0x7f8130005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af974472a9c321e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.823+0000 7f812ffff640 1 -- 172.21.15.5:0/2784355900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130009a80 con 0x5627145eacb0 2024-09-11T22:37:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f812ffff640 1 -- 172.21.15.5:0/2784355900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8130010430 con 0x5627145eacb0 2024-09-11T22:37:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f812ffff640 1 -- 172.21.15.5:0/2784355900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813000f760 con 0x5627145eacb0 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f814f83a740 1 -- 172.21.15.5:0/2784355900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 msgr2=0x562714602ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f814f83a740 1 --2- 172.21.15.5:0/2784355900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x562714602ca0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f81300098e0 tx=0x7f8130005970 comp rx=0 tx=0).stop 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f814f83a740 1 -- 172.21.15.5:0/2784355900 shutdown_connections 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f814f83a740 1 --2- 172.21.15.5:0/2784355900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145eacb0 0x562714602ca0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f814f83a740 1 --2- 172.21.15.5:0/2784355900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145e0410 0x562714602760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f814f83a740 1 --2- 172.21.15.5:0/2784355900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145dfaf0 0x5627145e8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f814f83a740 1 -- 172.21.15.5:0/2784355900 >> 172.21.15.5:0/2784355900 conn(0x5627145dc060 msgr2=0x5627145e6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f814f83a740 1 -- 172.21.15.5:0/2784355900 shutdown_connections 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.824+0000 7f814f83a740 1 -- 172.21.15.5:0/2784355900 wait complete. 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814f83a740 1 -- start start 2024-09-11T22:37:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814f83a740 1 Processor -- start 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814f83a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 0x5627145edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814f83a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145e0410 0x5627145ee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814f83a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145eacb0 0x562714663720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814f83a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81481b10a0 con 0x5627145e0410 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814f83a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81481b0910 con 0x5627145dfaf0 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814f83a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81481adb70 con 0x5627145eacb0 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814f83a740 10 client.0 _ll_register_callbacks cb 0x7f81400093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145e0410 0x5627145ee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145e0410 0x5627145ee220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53856/0 (socket says 172.21.15.5:53856) 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814659c640 1 -- 172.21.15.5:0/1269295740 learned_addr learned my addr 172.21.15.5:0/1269295740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:37:57.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 0x5627145edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145eacb0 0x562714663720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814659c640 1 -- 172.21.15.5:0/1269295740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145eacb0 msgr2=0x562714663720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814659c640 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145eacb0 0x562714663720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814659c640 1 -- 172.21.15.5:0/1269295740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 msgr2=0x5627145edce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814659c640 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 0x5627145edce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f814659c640 1 -- 172.21.15.5:0/1269295740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627145dbe90 con 0x5627145e0410 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.825+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 0x5627145edce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f814659c640 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145e0410 0x5627145ee220 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x5627145e7330 tx=0x7f81380051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfdcba1a093e3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f812dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f812dffb640 1 -- 172.21.15.5:0/1269295740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138035040 con 0x5627145e0410 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f814f83a740 1 -- 172.21.15.5:0/1269295740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8130010430 con 0x5627145e0410 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f812dffb640 1 -- 172.21.15.5:0/1269295740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 8629+0+0 (secure 0 0 0) 0x7f8138004760 con 0x5627145e0410 2024-09-11T22:37:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f812dffb640 1 -- 172.21.15.5:0/1269295740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f8138004960 con 0x5627145e0410 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f814f83a740 10 client.20334 fetch_fsmap learned FSMap version 51 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f814f83a740 1 -- 172.21.15.5:0/1269295740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x562714664c10 con 0x5627145e0410 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f812dffb640 1 -- 172.21.15.5:0/1269295740 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8130010430 con 0x5627145e0410 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f812dffb640 1 -- 172.21.15.5:0/1269295740 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f81380035f0 con 0x5627145e0410 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f814f83a740 10 client.20334 fetch_fsmap finished waiting for FSMap version 51 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f814f83a740 10 client.20334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f814f83a740 20 client.20334 populate_metadata read hostname 'smithi005' 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.826+0000 7f814f83a740 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562714665480 0x5627146678a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.827+0000 7f814f83a740 4 client.20334 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.827+0000 7f814f83a740 1 -- 172.21.15.5:0/1269295740 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56271454bd70 con 0x562714665480 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.827+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562714665480 0x5627146678a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.827+0000 7f812dffb640 10 client.20334 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:57.827+0000 7f813dd9b640 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562714665480 0x5627146678a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e9b912a74421d6d server_cookie=2f81b66b9bd8719a in_seq=0 out_seq=0 2024-09-11T22:37:57.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:57 smithi172 ceph-mon[35461]: pgmap v501: 97 pgs: 97 active+clean; 406 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.0 MiB/s wr, 447 op/s 2024-09-11T22:37:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:57 smithi005 ceph-mon[30943]: pgmap v501: 97 pgs: 97 active+clean; 406 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.0 MiB/s wr, 447 op/s 2024-09-11T22:37:58.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:57 smithi159 ceph-mon[35534]: pgmap v501: 97 pgs: 97 active+clean; 406 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.0 MiB/s wr, 447 op/s 2024-09-11T22:37:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.261+0000 7f812dffb640 1 -- 172.21.15.5:0/1269295740 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56271454bd70 con 0x562714665480 2024-09-11T22:37:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.261+0000 7f812dffb640 10 client.20334 handle_command_reply: tid=0 2024-09-11T22:37:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.261+0000 7f814f83a740 1 client.20334 shutdown 2024-09-11T22:37:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 1 -- 172.21.15.5:0/1269295740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145e0410 msgr2=0x5627145ee220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145e0410 0x5627145ee220 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x5627145e7330 tx=0x7f81380051e0 comp rx=0 tx=0).stop 2024-09-11T22:37:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 1 -- 172.21.15.5:0/1269295740 shutdown_connections 2024-09-11T22:37:58.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562714665480 0x5627146678a0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5627145eacb0 0x562714663720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627145e0410 0x5627145ee220 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 1 --2- 172.21.15.5:0/1269295740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627145dfaf0 0x5627145edce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f812dffb640 0 client.20334 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:37:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 1 -- 172.21.15.5:0/1269295740 >> 172.21.15.5:0/1269295740 conn(0x5627145dc060 msgr2=0x5627145e9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 1 -- 172.21.15.5:0/1269295740 shutdown_connections 2024-09-11T22:37:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 1 -- 172.21.15.5:0/1269295740 wait complete. 2024-09-11T22:37:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.262+0000 7f814f83a740 20 client.20334 trim_cache size 0 max 16384 2024-09-11T22:37:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:37:58.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:37:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:37:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:37:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 -- 172.21.15.5:0/3224443428 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81340a2da0 msgr2=0x7f81340a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81340a2da0 0x7f81340a5260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f81400cd2c0 tx=0x7f81400bc040 comp rx=0 tx=0).stop 2024-09-11T22:37:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 -- 172.21.15.5:0/3224443428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81481069c0 msgr2=0x7f814819edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:37:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81481069c0 0x7f814819edf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8140009fd0 tx=0x7f814000ef10 comp rx=0 tx=0).stop 2024-09-11T22:37:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 -- 172.21.15.5:0/3224443428 shutdown_connections 2024-09-11T22:37:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81481069c0 0x7f814819edf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81340a2da0 0x7f81340a5260 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8148101350 0x7f814819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 --2- 172.21.15.5:0/3224443428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81481009b0 0x7f814819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:37:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 -- 172.21.15.5:0/3224443428 >> 172.21.15.5:0/3224443428 conn(0x7f8148077990 msgr2=0x7f81480698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:37:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 -- 172.21.15.5:0/3224443428 shutdown_connections 2024-09-11T22:37:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:37:58.263+0000 7f814caa6640 1 -- 172.21.15.5:0/3224443428 wait complete. 2024-09-11T22:37:58.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32626 26.23 MB/sec execute 16 sec latency 328.238 ms 2024-09-11T22:37:59.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32963 25.98 MB/sec execute 17 sec latency 211.426 ms 2024-09-11T22:37:59.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:37:59 smithi172 ceph-mon[35461]: pgmap v502: 97 pgs: 97 active+clean; 423 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 602 op/s 2024-09-11T22:38:00.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:37:59 smithi005 ceph-mon[30943]: pgmap v502: 97 pgs: 97 active+clean; 423 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 602 op/s 2024-09-11T22:38:00.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:37:59 smithi159 ceph-mon[35534]: pgmap v502: 97 pgs: 97 active+clean; 423 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.7 MiB/s wr, 602 op/s 2024-09-11T22:38:00.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33449 25.73 MB/sec execute 18 sec latency 64.286 ms 2024-09-11T22:38:01.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33773 25.07 MB/sec execute 19 sec latency 257.024 ms 2024-09-11T22:38:01.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:01 smithi172 ceph-mon[35461]: pgmap v503: 97 pgs: 97 active+clean; 423 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-09-11T22:38:01.881 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:01 smithi159 ceph-mon[35534]: pgmap v503: 97 pgs: 97 active+clean; 423 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-09-11T22:38:02.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:01 smithi005 ceph-mon[30943]: pgmap v503: 97 pgs: 97 active+clean; 423 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 322 op/s 2024-09-11T22:38:02.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34148 24.85 MB/sec execute 20 sec latency 139.277 ms 2024-09-11T22:38:03.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34602 24.76 MB/sec execute 21 sec latency 249.214 ms 2024-09-11T22:38:03.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:03 smithi172 ceph-mon[35461]: pgmap v504: 97 pgs: 97 active+clean; 417 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.8 MiB/s wr, 723 op/s 2024-09-11T22:38:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:03 smithi005 ceph-mon[30943]: pgmap v504: 97 pgs: 97 active+clean; 417 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.8 MiB/s wr, 723 op/s 2024-09-11T22:38:04.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:03 smithi159 ceph-mon[35534]: pgmap v504: 97 pgs: 97 active+clean; 417 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 4.8 MiB/s wr, 723 op/s 2024-09-11T22:38:04.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35209 24.95 MB/sec execute 22 sec latency 164.110 ms 2024-09-11T22:38:05.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35444 24.49 MB/sec execute 23 sec latency 214.999 ms 2024-09-11T22:38:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:05 smithi172 ceph-mon[35461]: pgmap v505: 97 pgs: 97 active+clean; 417 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 556 op/s 2024-09-11T22:38:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:05 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:38:05.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:05 smithi172 ceph-mon[35461]: osdmap e112: 12 total, 12 up, 12 in 2024-09-11T22:38:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:05 smithi005 ceph-mon[30943]: pgmap v505: 97 pgs: 97 active+clean; 417 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 556 op/s 2024-09-11T22:38:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:05 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:38:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:05 smithi005 ceph-mon[30943]: osdmap e112: 12 total, 12 up, 12 in 2024-09-11T22:38:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:05 smithi159 ceph-mon[35534]: pgmap v505: 97 pgs: 97 active+clean; 417 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 556 op/s 2024-09-11T22:38:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:05 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:38:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:05 smithi159 ceph-mon[35534]: osdmap e112: 12 total, 12 up, 12 in 2024-09-11T22:38:06.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35716 24.13 MB/sec execute 24 sec latency 276.206 ms 2024-09-11T22:38:07.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36260 24.44 MB/sec execute 25 sec latency 226.360 ms 2024-09-11T22:38:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:07 smithi005 ceph-mon[30943]: pgmap v507: 97 pgs: 97 active+clean; 417 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 667 op/s 2024-09-11T22:38:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:07 smithi159 ceph-mon[35534]: pgmap v507: 97 pgs: 97 active+clean; 417 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 667 op/s 2024-09-11T22:38:08.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:07 smithi172 ceph-mon[35461]: pgmap v507: 97 pgs: 97 active+clean; 417 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.1 MiB/s wr, 667 op/s 2024-09-11T22:38:08.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36746 24.63 MB/sec execute 26 sec latency 194.397 ms 2024-09-11T22:38:09.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37080 24.50 MB/sec execute 27 sec latency 254.644 ms 2024-09-11T22:38:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:09 smithi005 ceph-mon[30943]: pgmap v508: 97 pgs: 97 active+clean; 429 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.4 MiB/s wr, 774 op/s 2024-09-11T22:38:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:09 smithi159 ceph-mon[35534]: pgmap v508: 97 pgs: 97 active+clean; 429 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.4 MiB/s wr, 774 op/s 2024-09-11T22:38:10.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:09 smithi172 ceph-mon[35461]: pgmap v508: 97 pgs: 97 active+clean; 429 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.4 MiB/s wr, 774 op/s 2024-09-11T22:38:10.639 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37406 24.15 MB/sec execute 28 sec latency 197.869 ms 2024-09-11T22:38:11.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:10 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:38:11.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:10 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:38:11.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:10 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:38:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.368+0000 7faf1e59f640 1 -- 172.21.15.5:0/1673126122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18106800 msgr2=0x7faf18106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.368+0000 7faf1e59f640 1 --2- 172.21.15.5:0/1673126122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18106800 0x7faf18106be0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7faf14009930 tx=0x7faf1402a420 comp rx=0 tx=0).stop 2024-09-11T22:38:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.368+0000 7faf1e59f640 1 -- 172.21.15.5:0/1673126122 shutdown_connections 2024-09-11T22:38:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.368+0000 7faf1e59f640 1 --2- 172.21.15.5:0/1673126122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf18101190 0x7faf1810d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.368+0000 7faf1e59f640 1 --2- 172.21.15.5:0/1673126122 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf181007f0 0x7faf18100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.368+0000 7faf1e59f640 1 --2- 172.21.15.5:0/1673126122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18106800 0x7faf18106be0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.368+0000 7faf1e59f640 1 -- 172.21.15.5:0/1673126122 >> 172.21.15.5:0/1673126122 conn(0x7faf180fc4f0 msgr2=0x7faf180fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.368+0000 7faf1e59f640 1 -- 172.21.15.5:0/1673126122 shutdown_connections 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.368+0000 7faf1e59f640 1 -- 172.21.15.5:0/1673126122 wait complete. 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.368+0000 7faf1e59f640 1 Processor -- start 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1e59f640 1 -- start start 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1e59f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf181007f0 0x7faf18198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1e59f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf18101190 0x7faf18198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1e59f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18106800 0x7faf1819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1e59f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf18111d20 con 0x7faf18106800 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1e59f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faf18111ba0 con 0x7faf181007f0 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1e59f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faf18111ea0 con 0x7faf18101190 2024-09-11T22:38:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1dd9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18106800 0x7faf1819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1d59d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf181007f0 0x7faf18198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1cd9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf18101190 0x7faf18198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1d59d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf181007f0 0x7faf18198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58970/0 (socket says 172.21.15.5:58970) 2024-09-11T22:38:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1dd9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18106800 0x7faf1819cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40328/0 (socket says 172.21.15.5:40328) 2024-09-11T22:38:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1d59d640 1 -- 172.21.15.5:0/2428171234 learned_addr learned my addr 172.21.15.5:0/2428171234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1dd9e640 1 -- 172.21.15.5:0/2428171234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf18101190 msgr2=0x7faf18198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1dd9e640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf18101190 0x7faf18198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1dd9e640 1 -- 172.21.15.5:0/2428171234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf181007f0 msgr2=0x7faf18198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1dd9e640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf181007f0 0x7faf18198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.369+0000 7faf1dd9e640 1 -- 172.21.15.5:0/2428171234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1819d240 con 0x7faf18106800 2024-09-11T22:38:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.370+0000 7faf1dd9e640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18106800 0x7faf1819cb10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7faf10009fd0 tx=0x7faf1000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.370+0000 7faf0e7fc640 1 -- 172.21.15.5:0/2428171234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf1000ce20 con 0x7faf18106800 2024-09-11T22:38:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.370+0000 7faf0e7fc640 1 -- 172.21.15.5:0/2428171234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf10005020 con 0x7faf18106800 2024-09-11T22:38:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.370+0000 7faf1e59f640 1 -- 172.21.15.5:0/2428171234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1819d530 con 0x7faf18106800 2024-09-11T22:38:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.370+0000 7faf0e7fc640 1 -- 172.21.15.5:0/2428171234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10010720 con 0x7faf18106800 2024-09-11T22:38:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.370+0000 7faf1e59f640 1 -- 172.21.15.5:0/2428171234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf181a4dc0 con 0x7faf18106800 2024-09-11T22:38:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.372+0000 7faf0e7fc640 1 -- 172.21.15.5:0/2428171234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7faf10005540 con 0x7faf18106800 2024-09-11T22:38:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.372+0000 7faf22675740 1 Processor -- start 2024-09-11T22:38:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf22675740 1 -- start start 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf22675740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 0x55fe2552e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf22675740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25538940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf22675740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25538e80 0x55fe2553b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf22675740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe254922f0 con 0x55fe25538e80 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf22675740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fe2543a270 con 0x55fe2552dcc0 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf22675740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fe2548e2b0 con 0x55fe2552e5e0 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf0e7fc640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faf040a2cf0 0x7faf040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf1cd9c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 0x55fe2552e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf1cd9c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 0x55fe2552e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58982/0 (socket says 172.21.15.5:58982) 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf1cd9c640 1 -- 172.21.15.5:0/148199080 learned_addr learned my addr 172.21.15.5:0/148199080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/148199080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25538e80 0x55fe2553b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf1d59d640 1 --2- 172.21.15.5:0/148199080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25538940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf1dd9e640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faf040a2cf0 0x7faf040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf0e7fc640 1 -- 172.21.15.5:0/2428171234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8673+0+0 (secure 0 0 0) 0x7faf100cfb60 con 0x7faf18106800 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf1d59d640 1 -- 172.21.15.5:0/148199080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 msgr2=0x55fe2552e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf1d59d640 1 --2- 172.21.15.5:0/148199080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 0x55fe2552e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf1d59d640 1 -- 172.21.15.5:0/148199080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25538e80 msgr2=0x55fe2553b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.373+0000 7faf1d59d640 1 --2- 172.21.15.5:0/148199080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25538e80 0x55fe2553b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.374+0000 7faf1d59d640 1 -- 172.21.15.5:0/148199080 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe25439340 con 0x55fe2552e5e0 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.374+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/148199080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25538e80 0x55fe2553b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.374+0000 7faf1dd9e640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faf040a2cf0 0x7faf040a51b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faf1001a040 tx=0x7faf10076040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.374+0000 7faf1d59d640 1 --2- 172.21.15.5:0/148199080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25538940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faf14009900 tx=0x7faf14005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8954a5c4834dd0b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.374+0000 7faf037fe640 1 -- 172.21.15.5:0/148199080 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14040070 con 0x55fe2552e5e0 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.374+0000 7faf037fe640 1 -- 172.21.15.5:0/148199080 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf140318f0 con 0x55fe2552e5e0 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.374+0000 7faf037fe640 1 -- 172.21.15.5:0/148199080 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14035040 con 0x55fe2552e5e0 2024-09-11T22:38:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 -- 172.21.15.5:0/148199080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 msgr2=0x55fe25538940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 --2- 172.21.15.5:0/148199080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25538940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7faf14009900 tx=0x7faf14005dc0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 -- 172.21.15.5:0/148199080 shutdown_connections 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 --2- 172.21.15.5:0/148199080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe25538e80 0x55fe2553b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 --2- 172.21.15.5:0/148199080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25538940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 --2- 172.21.15.5:0/148199080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 0x55fe2552e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 -- 172.21.15.5:0/148199080 >> 172.21.15.5:0/148199080 conn(0x55fe2552a230 msgr2=0x55fe2552a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 -- 172.21.15.5:0/148199080 shutdown_connections 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 -- 172.21.15.5:0/148199080 wait complete. 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 -- start start 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.375+0000 7faf22675740 1 Processor -- start 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf22675740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25535d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf22675740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25536260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf22675740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe25538e80 0x55fe255367a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf22675740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe2553ba50 con 0x55fe2552dcc0 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf22675740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fe2541b290 con 0x55fe25538e80 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf22675740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55fe2553bbd0 con 0x55fe2552e5e0 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf22675740 10 client.0 _ll_register_callbacks cb 0x55fe2544c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1cd9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25535d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1cd9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25535d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40350/0 (socket says 172.21.15.5:40350) 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1cd9c640 1 -- 172.21.15.5:0/29920891 learned_addr learned my addr 172.21.15.5:0/29920891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1d59d640 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25536260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe25538e80 0x55fe255367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1cd9c640 1 -- 172.21.15.5:0/29920891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 msgr2=0x55fe25536260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25536260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1cd9c640 1 -- 172.21.15.5:0/29920891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe25538e80 msgr2=0x55fe255367a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe25538e80 0x55fe255367a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1cd9c640 1 -- 172.21.15.5:0/29920891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe251a5210 con 0x55fe2552dcc0 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25535d20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7faf08004420 tx=0x7faf08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738cedda84754f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.376+0000 7faf00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf00ff9640 1 -- 172.21.15.5:0/29920891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08005020 con 0x55fe2552dcc0 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf00ff9640 1 -- 172.21.15.5:0/29920891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8673+0+0 (secure 0 0 0) 0x7faf080051c0 con 0x55fe2552dcc0 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf22675740 1 -- 172.21.15.5:0/29920891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf140318f0 con 0x55fe2552dcc0 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf00ff9640 1 -- 172.21.15.5:0/29920891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7faf080053c0 con 0x55fe2552dcc0 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf22675740 10 client.20346 fetch_fsmap learned FSMap version 53 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf22675740 1 -- 172.21.15.5:0/29920891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55fe25271620 con 0x55fe2552dcc0 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf00ff9640 1 -- 172.21.15.5:0/29920891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf140318f0 con 0x55fe2552dcc0 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf00ff9640 1 -- 172.21.15.5:0/29920891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) ==== 8366+0+0 (secure 0 0 0) 0x7faf08021620 con 0x55fe2552dcc0 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf22675740 10 client.20346 fetch_fsmap finished waiting for FSMap version 53 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf22675740 10 client.20346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf22675740 20 client.20346 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf22675740 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fe25548df0 0x55fe2554b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf22675740 4 client.20346 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.377+0000 7faf22675740 1 -- 172.21.15.5:0/29920891 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe2549fa20 con 0x55fe25548df0 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.378+0000 7faf1d59d640 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fe25548df0 0x55fe2554b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.378+0000 7faf1d59d640 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fe25548df0 0x55fe2554b1e0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=800fb3320d3eb15b server_cookie=e6f97c5c6ec5074c in_seq=0 out_seq=0 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.378+0000 7faf00ff9640 10 client.20346 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf00ff9640 1 -- 172.21.15.5:0/29920891 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe2549fa20 con 0x55fe25548df0 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf00ff9640 10 client.20346 handle_command_reply: tid=0 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 client.20346 shutdown 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 -- 172.21.15.5:0/29920891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 msgr2=0x55fe25535d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25535d20 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7faf08004420 tx=0x7faf08004730 comp rx=0 tx=0).stop 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 -- 172.21.15.5:0/29920891 shutdown_connections 2024-09-11T22:38:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fe25548df0 0x55fe2554b1e0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe25538e80 0x55fe255367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25536260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf00ff9640 0 client.20346 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 --2- 172.21.15.5:0/29920891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25535d20 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 -- 172.21.15.5:0/29920891 >> 172.21.15.5:0/29920891 conn(0x55fe2552a230 msgr2=0x55fe2552ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 -- 172.21.15.5:0/29920891 shutdown_connections 2024-09-11T22:38:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 1 -- 172.21.15.5:0/29920891 wait complete. 2024-09-11T22:38:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.380+0000 7faf22675740 20 client.20346 trim_cache size 0 max 16384 2024-09-11T22:38:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf22675740 1 Processor -- start 2024-09-11T22:38:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf22675740 1 -- start start 2024-09-11T22:38:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf22675740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25530730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf22675740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25536250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf22675740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe25538e80 0x55fe25536790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf22675740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf181ad940 con 0x55fe2552dcc0 2024-09-11T22:38:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf22675740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faf181af4b0 con 0x55fe25538e80 2024-09-11T22:38:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf22675740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faf181ae010 con 0x55fe2552e5e0 2024-09-11T22:38:11.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf1cd9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25530730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf1cd9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25530730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40352/0 (socket says 172.21.15.5:40352) 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf1cd9c640 1 -- 172.21.15.5:0/1945497701 learned_addr learned my addr 172.21.15.5:0/1945497701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/1945497701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe25538e80 0x55fe25536790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf1cd9c640 1 -- 172.21.15.5:0/1945497701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 msgr2=0x55fe25536250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.396+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/1945497701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25536250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.397+0000 7faf1cd9c640 1 -- 172.21.15.5:0/1945497701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe25538e80 msgr2=0x55fe25536790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.397+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/1945497701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe25538e80 0x55fe25536790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.397+0000 7faf1cd9c640 1 -- 172.21.15.5:0/1945497701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe2554b550 con 0x55fe2552dcc0 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.397+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/1945497701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25530730 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7faf08001eb0 tx=0x7faf08009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1230ac702a7a500 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.397+0000 7faf02ffd640 1 -- 172.21.15.5:0/1945497701 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08003ab0 con 0x55fe2552dcc0 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.397+0000 7faf02ffd640 1 -- 172.21.15.5:0/1945497701 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf08011e50 con 0x55fe2552dcc0 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.397+0000 7faf02ffd640 1 -- 172.21.15.5:0/1945497701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf08028c00 con 0x55fe2552dcc0 2024-09-11T22:38:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 -- 172.21.15.5:0/1945497701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 msgr2=0x55fe25530730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 --2- 172.21.15.5:0/1945497701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25530730 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7faf08001eb0 tx=0x7faf08009fd0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 -- 172.21.15.5:0/1945497701 shutdown_connections 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 --2- 172.21.15.5:0/1945497701 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe25538e80 0x55fe25536790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 --2- 172.21.15.5:0/1945497701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe2552e5e0 0x55fe25536250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 --2- 172.21.15.5:0/1945497701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552dcc0 0x55fe25530730 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 -- 172.21.15.5:0/1945497701 >> 172.21.15.5:0/1945497701 conn(0x55fe2552a230 msgr2=0x55fe25537da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 -- 172.21.15.5:0/1945497701 shutdown_connections 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 -- 172.21.15.5:0/1945497701 wait complete. 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 -- start start 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.398+0000 7faf22675740 1 Processor -- start 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf22675740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 0x55fe25532e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf22675740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552e5e0 0x55fe25533380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf22675740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe25538e80 0x55fe255338c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf22675740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf181aede0 con 0x55fe2552e5e0 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf22675740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faf181ae6e0 con 0x55fe2552dcc0 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf22675740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faf181ab940 con 0x55fe25538e80 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf22675740 10 client.0 _ll_register_callbacks cb 0x7faf08009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf1d59d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552e5e0 0x55fe25533380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf1cd9c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 0x55fe25532e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf1d59d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552e5e0 0x55fe25533380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40362/0 (socket says 172.21.15.5:40362) 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf1d59d640 1 -- 172.21.15.5:0/1894387405 learned_addr learned my addr 172.21.15.5:0/1894387405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf1d59d640 1 -- 172.21.15.5:0/1894387405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe25538e80 msgr2=0x55fe255338c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf1d59d640 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe25538e80 0x55fe255338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf1d59d640 1 -- 172.21.15.5:0/1894387405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 msgr2=0x55fe25532e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf1d59d640 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 0x55fe25532e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf1d59d640 1 -- 172.21.15.5:0/1894387405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe255b2ba0 con 0x55fe2552e5e0 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf1d59d640 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552e5e0 0x55fe25533380 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7faf14009a50 tx=0x7faf14005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142adb9e42c5eda8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf00ff9640 1 -- 172.21.15.5:0/1894387405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf14040040 con 0x55fe2552e5e0 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf00ff9640 1 -- 172.21.15.5:0/1894387405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8673+0+0 (secure 0 0 0) 0x7faf14004030 con 0x55fe2552e5e0 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.399+0000 7faf22675740 1 -- 172.21.15.5:0/1894387405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf08011e50 con 0x55fe2552e5e0 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf00ff9640 1 -- 172.21.15.5:0/1894387405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7faf14035040 con 0x55fe2552e5e0 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf22675740 10 client.20352 fetch_fsmap learned FSMap version 53 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf22675740 1 -- 172.21.15.5:0/1894387405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55fe255b3e50 con 0x55fe2552e5e0 2024-09-11T22:38:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf00ff9640 1 -- 172.21.15.5:0/1894387405 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf08011e50 con 0x55fe2552e5e0 2024-09-11T22:38:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf00ff9640 1 -- 172.21.15.5:0/1894387405 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) ==== 8366+0+0 (secure 0 0 0) 0x7faf14003cb0 con 0x55fe2552e5e0 2024-09-11T22:38:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf22675740 10 client.20352 fetch_fsmap finished waiting for FSMap version 53 2024-09-11T22:38:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf22675740 10 client.20352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf22675740 20 client.20352 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf22675740 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fe255b41f0 0x55fe255b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf22675740 4 client.20352 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:38:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.400+0000 7faf22675740 1 -- 172.21.15.5:0/1894387405 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe25499d70 con 0x55fe255b41f0 2024-09-11T22:38:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.401+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fe255b41f0 0x55fe255b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.401+0000 7faf1cd9c640 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fe255b41f0 0x55fe255b65e0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a320b8ebe618ab17 server_cookie=c1a0d4ed45762a9c in_seq=0 out_seq=0 2024-09-11T22:38:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:11.401+0000 7faf00ff9640 10 client.20352 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:11.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37773 23.94 MB/sec execute 29 sec latency 265.422 ms 2024-09-11T22:38:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:11 smithi005 ceph-mon[30943]: pgmap v509: 97 pgs: 97 active+clean; 429 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.4 MiB/s wr, 774 op/s 2024-09-11T22:38:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:11 smithi159 ceph-mon[35534]: pgmap v509: 97 pgs: 97 active+clean; 429 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.4 MiB/s wr, 774 op/s 2024-09-11T22:38:12.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:11 smithi172 ceph-mon[35461]: pgmap v509: 97 pgs: 97 active+clean; 429 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.4 MiB/s wr, 774 op/s 2024-09-11T22:38:12.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38112 23.86 MB/sec execute 30 sec latency 261.328 ms 2024-09-11T22:38:13.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38616 23.73 MB/sec execute 31 sec latency 178.201 ms 2024-09-11T22:38:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:13 smithi005 ceph-mon[30943]: pgmap v510: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 705 op/s 2024-09-11T22:38:13.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:13 smithi005 ceph-mon[30943]: osdmap e113: 12 total, 12 up, 12 in 2024-09-11T22:38:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:13 smithi159 ceph-mon[35534]: pgmap v510: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 705 op/s 2024-09-11T22:38:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:13 smithi159 ceph-mon[35534]: osdmap e113: 12 total, 12 up, 12 in 2024-09-11T22:38:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:13 smithi172 ceph-mon[35461]: pgmap v510: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 705 op/s 2024-09-11T22:38:13.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:13 smithi172 ceph-mon[35461]: osdmap e113: 12 total, 12 up, 12 in 2024-09-11T22:38:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.020+0000 7faf00ff9640 1 -- 172.21.15.5:0/1894387405 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe25499d70 con 0x55fe255b41f0 2024-09-11T22:38:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.020+0000 7faf00ff9640 10 client.20352 handle_command_reply: tid=0 2024-09-11T22:38:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 client.20352 shutdown 2024-09-11T22:38:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 -- 172.21.15.5:0/1894387405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552e5e0 msgr2=0x55fe25533380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552e5e0 0x55fe25533380 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7faf14009a50 tx=0x7faf14005850 comp rx=0 tx=0).stop 2024-09-11T22:38:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 -- 172.21.15.5:0/1894387405 shutdown_connections 2024-09-11T22:38:14.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55fe255b41f0 0x55fe255b65e0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fe25538e80 0x55fe255338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe2552e5e0 0x55fe25533380 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf00ff9640 0 client.20352 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 --2- 172.21.15.5:0/1894387405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe2552dcc0 0x55fe25532e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 -- 172.21.15.5:0/1894387405 >> 172.21.15.5:0/1894387405 conn(0x55fe2552a230 msgr2=0x55fe25528a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 -- 172.21.15.5:0/1894387405 shutdown_connections 2024-09-11T22:38:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 1 -- 172.21.15.5:0/1894387405 wait complete. 2024-09-11T22:38:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.021+0000 7faf22675740 20 client.20352 trim_cache size 0 max 16384 2024-09-11T22:38:14.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:38:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:38:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:38:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:38:14.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 -- 172.21.15.5:0/2428171234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faf040a2cf0 msgr2=0x7faf040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faf040a2cf0 0x7faf040a51b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faf1001a040 tx=0x7faf10076040 comp rx=0 tx=0).stop 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 -- 172.21.15.5:0/2428171234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18106800 msgr2=0x7faf1819cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18106800 0x7faf1819cb10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7faf10009fd0 tx=0x7faf1000ef10 comp rx=0 tx=0).stop 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 -- 172.21.15.5:0/2428171234 shutdown_connections 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18106800 0x7faf1819cb10 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf18101190 0x7faf18198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faf040a2cf0 0x7faf040a51b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 --2- 172.21.15.5:0/2428171234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faf181007f0 0x7faf18198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 -- 172.21.15.5:0/2428171234 >> 172.21.15.5:0/2428171234 conn(0x7faf180fc4f0 msgr2=0x7faf180fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 -- 172.21.15.5:0/2428171234 shutdown_connections 2024-09-11T22:38:14.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:14.022+0000 7faf1e59f640 1 -- 172.21.15.5:0/2428171234 wait complete. 2024-09-11T22:38:14.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39132 23.86 MB/sec execute 32 sec latency 304.033 ms 2024-09-11T22:38:15.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39652 23.93 MB/sec execute 33 sec latency 180.901 ms 2024-09-11T22:38:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:15 smithi005 ceph-mon[30943]: pgmap v512: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.6 MiB/s wr, 734 op/s 2024-09-11T22:38:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:15 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:38:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:15 smithi159 ceph-mon[35534]: pgmap v512: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.6 MiB/s wr, 734 op/s 2024-09-11T22:38:15.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:15 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:38:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:15 smithi172 ceph-mon[35461]: pgmap v512: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 7.6 MiB/s wr, 734 op/s 2024-09-11T22:38:15.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:15 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:38:16.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40091 24.24 MB/sec execute 34 sec latency 357.830 ms 2024-09-11T22:38:17.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40639 24.51 MB/sec execute 35 sec latency 215.394 ms 2024-09-11T22:38:17.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:17 smithi005 ceph-mon[30943]: pgmap v513: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 705 op/s 2024-09-11T22:38:17.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:17 smithi159 ceph-mon[35534]: pgmap v513: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 705 op/s 2024-09-11T22:38:17.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:17 smithi172 ceph-mon[35461]: pgmap v513: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 7.3 MiB/s wr, 705 op/s 2024-09-11T22:38:18.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41006 24.24 MB/sec execute 36 sec latency 57.895 ms 2024-09-11T22:38:19.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41435 24.11 MB/sec execute 37 sec latency 106.761 ms 2024-09-11T22:38:19.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:38:19.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:19 smithi005 ceph-mon[30943]: pgmap v514: 97 pgs: 97 active+clean; 454 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 690 op/s 2024-09-11T22:38:19.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:19 smithi159 ceph-mon[35534]: pgmap v514: 97 pgs: 97 active+clean; 454 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 690 op/s 2024-09-11T22:38:19.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:19 smithi172 ceph-mon[35461]: pgmap v514: 97 pgs: 97 active+clean; 454 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 690 op/s 2024-09-11T22:38:20.056 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:20.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41888 24.19 MB/sec execute 38 sec latency 162.165 ms 2024-09-11T22:38:20.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.932+0000 7f7bdda9c640 1 -- 172.21.15.5:0/3158217075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd810ae80 msgr2=0x7f7bd8113370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.932+0000 7f7bdda9c640 1 --2- 172.21.15.5:0/3158217075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd810ae80 0x7f7bd8113370 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009900 tx=0x7f7bcc02a3f0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.932+0000 7f7bdda9c640 1 -- 172.21.15.5:0/3158217075 shutdown_connections 2024-09-11T22:38:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.932+0000 7f7bdda9c640 1 --2- 172.21.15.5:0/3158217075 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd8113940 0x7f7bd8115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.932+0000 7f7bdda9c640 1 --2- 172.21.15.5:0/3158217075 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd810ae80 0x7f7bd8113370 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.932+0000 7f7bdda9c640 1 --2- 172.21.15.5:0/3158217075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd810a560 0x7f7bd810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.932+0000 7f7bdda9c640 1 -- 172.21.15.5:0/3158217075 >> 172.21.15.5:0/3158217075 conn(0x7f7bd806e140 msgr2=0x7f7bd806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.933+0000 7f7bdda9c640 1 -- 172.21.15.5:0/3158217075 shutdown_connections 2024-09-11T22:38:20.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.933+0000 7f7bdda9c640 1 -- 172.21.15.5:0/3158217075 wait complete. 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.934+0000 7f7bdda9c640 1 Processor -- start 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.934+0000 7f7bdda9c640 1 -- start start 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.934+0000 7f7bdda9c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd810a560 0x7f7bd819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.934+0000 7f7bdda9c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd810ae80 0x7f7bd819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.934+0000 7f7bdda9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd8113940 0x7f7bd81a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bdda9c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd8119660 con 0x7f7bd8113940 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bdda9c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd81194e0 con 0x7f7bd810a560 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bdda9c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd81197e0 con 0x7f7bd810ae80 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd810a560 0x7f7bd819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd810ae80 0x7f7bd819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd810a560 0x7f7bd819f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59028/0 (socket says 172.21.15.5:59028) 2024-09-11T22:38:20.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd77fe640 1 -- 172.21.15.5:0/2003912015 learned_addr learned my addr 172.21.15.5:0/2003912015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:20.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd8113940 0x7f7bd81a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd77fe640 1 -- 172.21.15.5:0/2003912015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd810ae80 msgr2=0x7f7bd819fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:20.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd810ae80 0x7f7bd819fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd77fe640 1 -- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd8113940 msgr2=0x7f7bd81a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:20.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd8113940 0x7f7bd81a48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd77fe640 1 -- 172.21.15.5:0/2003912015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd81a4de0 con 0x7f7bd810a560 2024-09-11T22:38:20.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.935+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd8113940 0x7f7bd81a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:20.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.936+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd810a560 0x7f7bd819f850 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800ea60 tx=0x7f7bc800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:20.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.936+0000 7f7bd4ff9640 1 -- 172.21.15.5:0/2003912015 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc800cde0 con 0x7f7bd810a560 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.936+0000 7f7bdda9c640 1 -- 172.21.15.5:0/2003912015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd81a50d0 con 0x7f7bd810a560 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.936+0000 7f7bd4ff9640 1 -- 172.21.15.5:0/2003912015 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc80035f0 con 0x7f7bd810a560 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.936+0000 7f7bd4ff9640 1 -- 172.21.15.5:0/2003912015 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8010670 con 0x7f7bd810a560 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.937+0000 7f7bdda9c640 1 -- 172.21.15.5:0/2003912015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd81a5630 con 0x7f7bd810a560 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.939+0000 7f7bd4ff9640 1 -- 172.21.15.5:0/2003912015 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f7bc80037a0 con 0x7f7bd810a560 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.940+0000 7f7bdfe75740 1 Processor -- start 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.940+0000 7f7bdfe75740 1 -- start start 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.940+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac314a0 0x56008ac31880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.940+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.940+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac3c680 0x56008ac3ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.940+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56008ab972e0 con 0x56008ac3c680 2024-09-11T22:38:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.940+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56008ab3ece0 con 0x56008ac314a0 2024-09-11T22:38:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.940+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56008ab932a0 con 0x56008ac31dc0 2024-09-11T22:38:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.940+0000 7f7bd4ff9640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7bc00a2d70 0x7f7bc00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac314a0 0x56008ac31880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac314a0 0x56008ac31880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59036/0 (socket says 172.21.15.5:59036) 2024-09-11T22:38:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3942776546 learned_addr learned my addr 172.21.15.5:0/3942776546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd6ffd640 1 --2- 172.21.15.5:0/3942776546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac3c680 0x56008ac3ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7bc00a2d70 0x7f7bc00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd4ff9640 1 -- 172.21.15.5:0/2003912015 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f7bc80cf760 con 0x7f7bd810a560 2024-09-11T22:38:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/3942776546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3942776546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 msgr2=0x56008ac3c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/3942776546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac3c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3942776546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac3c680 msgr2=0x56008ac3ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/3942776546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac3c680 0x56008ac3ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3942776546 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56008ab23440 con 0x56008ac314a0 2024-09-11T22:38:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/3942776546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac314a0 0x56008ac31880 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0010470 tx=0x7f7bd0010930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aa1c06f383b6e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:20.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.941+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7bc00a2d70 0x7f7bc00a5230 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0010e30 tx=0x7f7bd0017000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bbdffb640 1 -- 172.21.15.5:0/3942776546 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0012900 con 0x56008ac314a0 2024-09-11T22:38:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bbdffb640 1 -- 172.21.15.5:0/3942776546 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bd0012bd0 con 0x56008ac314a0 2024-09-11T22:38:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bbdffb640 1 -- 172.21.15.5:0/3942776546 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd001e3f0 con 0x56008ac314a0 2024-09-11T22:38:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3942776546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac314a0 msgr2=0x56008ac31880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/3942776546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac314a0 0x56008ac31880 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0010470 tx=0x7f7bd0010930 comp rx=0 tx=0).stop 2024-09-11T22:38:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3942776546 shutdown_connections 2024-09-11T22:38:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/3942776546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac3c680 0x56008ac3ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/3942776546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac3c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/3942776546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac314a0 0x56008ac31880 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3942776546 >> 172.21.15.5:0/3942776546 conn(0x56008ac2da30 msgr2=0x56008ac2de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3942776546 shutdown_connections 2024-09-11T22:38:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.942+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3942776546 wait complete. 2024-09-11T22:38:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.943+0000 7f7bdfe75740 1 -- start start 2024-09-11T22:38:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.943+0000 7f7bdfe75740 1 Processor -- start 2024-09-11T22:38:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac3bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac39270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56008ac3f250 con 0x56008ac314a0 2024-09-11T22:38:20.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56008ab52050 con 0x56008ac3c680 2024-09-11T22:38:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56008ac3f3d0 con 0x56008ac31dc0 2024-09-11T22:38:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bdfe75740 10 client.0 _ll_register_callbacks cb 0x56008ab92160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac39270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd6ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac39270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:32888/0 (socket says 172.21.15.5:32888) 2024-09-11T22:38:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/630640983 learned_addr learned my addr 172.21.15.5:0/630640983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac3bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd7fff640 1 -- 172.21.15.5:0/630640983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 msgr2=0x56008ac39270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:20.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac39270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd7fff640 1 -- 172.21.15.5:0/630640983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 msgr2=0x56008ac3bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac3bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd7fff640 1 -- 172.21.15.5:0/630640983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008a9083c0 con 0x56008ac3c680 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd6ffd640 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac39270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.944+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac3bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.945+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac397b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000ecd0 tx=0x7f7bd0012a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b0fb5d81cb4071 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.945+0000 7f7bb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.945+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/630640983 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd0004ba0 con 0x56008ac3c680 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.945+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/630640983 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f7bd0004d40 con 0x56008ac3c680 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.945+0000 7f7bdfe75740 1 -- 172.21.15.5:0/630640983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bd0012bd0 con 0x56008ac3c680 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.945+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/630640983 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7bd0028400 con 0x56008ac3c680 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.945+0000 7f7bdfe75740 10 client.29438 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:20.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.945+0000 7f7bdfe75740 1 -- 172.21.15.5:0/630640983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56008ac4ac00 con 0x56008ac3c680 2024-09-11T22:38:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.946+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/630640983 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bd0012bd0 con 0x56008ac3c680 2024-09-11T22:38:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.946+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/630640983 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f7bd000f040 con 0x56008ac3c680 2024-09-11T22:38:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.946+0000 7f7bdfe75740 10 client.29438 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.946+0000 7f7bdfe75740 10 client.29438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.946+0000 7f7bdfe75740 20 client.29438 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.946+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56008ac4c010 0x56008ac4e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.946+0000 7f7bdfe75740 4 client.29438 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.946+0000 7f7bdfe75740 1 -- 172.21.15.5:0/630640983 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56008aba4850 con 0x56008ac4c010 2024-09-11T22:38:20.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.947+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56008ac4c010 0x56008ac4e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.947+0000 7f7bb6ffd640 10 client.29438 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.948+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56008ac4c010 0x56008ac4e400 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f93332df3bdb756 server_cookie=3d5fc72dc09cdf8d in_seq=0 out_seq=0 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.949+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/630640983 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56008aba4850 con 0x56008ac4c010 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.949+0000 7f7bb6ffd640 10 client.29438 handle_command_reply: tid=0 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.949+0000 7f7bdfe75740 1 client.29438 shutdown 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.949+0000 7f7bdfe75740 1 -- 172.21.15.5:0/630640983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 msgr2=0x56008ac397b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.949+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac397b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7bd000ecd0 tx=0x7f7bd0012a40 comp rx=0 tx=0).stop 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.950+0000 7f7bdfe75740 1 -- 172.21.15.5:0/630640983 shutdown_connections 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.950+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56008ac4c010 0x56008ac4e400 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.950+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac397b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.950+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac39270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.950+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/630640983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac3bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.950+0000 7f7bb6ffd640 0 client.29438 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.950+0000 7f7bdfe75740 1 -- 172.21.15.5:0/630640983 >> 172.21.15.5:0/630640983 conn(0x56008ac2da30 msgr2=0x56008ac33760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.950+0000 7f7bdfe75740 1 -- 172.21.15.5:0/630640983 shutdown_connections 2024-09-11T22:38:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.950+0000 7f7bdfe75740 1 -- 172.21.15.5:0/630640983 wait complete. 2024-09-11T22:38:20.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.950+0000 7f7bdfe75740 20 client.29438 trim_cache size 0 max 16384 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.968+0000 7f7bdfe75740 1 Processor -- start 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.968+0000 7f7bdfe75740 1 -- start start 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.968+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac314a0 0x56008ac484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.968+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac31dc0 0x56008ac3b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.968+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac53be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.968+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd81bc980 con 0x56008ac31dc0 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.968+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd81be4c0 con 0x56008ac3c680 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.968+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd81bd050 con 0x56008ac314a0 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac31dc0 0x56008ac3b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac31dc0 0x56008ac3b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40396/0 (socket says 172.21.15.5:40396) 2024-09-11T22:38:20.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/1894706583 learned_addr learned my addr 172.21.15.5:0/1894706583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/1894706583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac314a0 0x56008ac484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/1894706583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac53be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/1894706583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac314a0 msgr2=0x56008ac484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd6ffd640 1 --2- 172.21.15.5:0/1894706583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac314a0 0x56008ac484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/1894706583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 msgr2=0x56008ac53be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd6ffd640 1 --2- 172.21.15.5:0/1894706583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac53be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd6ffd640 1 -- 172.21.15.5:0/1894706583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56008ac4e770 con 0x56008ac31dc0 2024-09-11T22:38:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/1894706583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac53be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:20.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bd6ffd640 1 --2- 172.21.15.5:0/1894706583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac31dc0 0x56008ac3b1e0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc005940 tx=0x7f7bcc031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a8ab9c2a96391c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bbd7fa640 1 -- 172.21.15.5:0/1894706583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc035040 con 0x56008ac31dc0 2024-09-11T22:38:20.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bbd7fa640 1 -- 172.21.15.5:0/1894706583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bcc004030 con 0x56008ac31dc0 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.969+0000 7f7bbd7fa640 1 -- 172.21.15.5:0/1894706583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc004750 con 0x56008ac31dc0 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.970+0000 7f7bdfe75740 1 -- 172.21.15.5:0/1894706583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac31dc0 msgr2=0x56008ac3b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.970+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/1894706583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac31dc0 0x56008ac3b1e0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc005940 tx=0x7f7bcc031420 comp rx=0 tx=0).stop 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.970+0000 7f7bdfe75740 1 -- 172.21.15.5:0/1894706583 shutdown_connections 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.970+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/1894706583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac53be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.970+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/1894706583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac31dc0 0x56008ac3b1e0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.970+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/1894706583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac314a0 0x56008ac484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.970+0000 7f7bdfe75740 1 -- 172.21.15.5:0/1894706583 >> 172.21.15.5:0/1894706583 conn(0x56008ac2da30 msgr2=0x56008ac3aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.970+0000 7f7bdfe75740 1 -- 172.21.15.5:0/1894706583 shutdown_connections 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.970+0000 7f7bdfe75740 1 -- 172.21.15.5:0/1894706583 wait complete. 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bdfe75740 1 -- start start 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bdfe75740 1 Processor -- start 2024-09-11T22:38:20.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac44d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac40060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bdfe75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd81bddf0 con 0x56008ac314a0 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd81bd720 con 0x56008ac3c680 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bdfe75740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd81ba980 con 0x56008ac31dc0 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bdfe75740 10 client.0 _ll_register_callbacks cb 0x7f7bd0012ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bd77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac44d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bd7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bd7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59062/0 (socket says 172.21.15.5:59062) 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bd77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac44d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40408/0 (socket says 172.21.15.5:40408) 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3964146365 learned_addr learned my addr 172.21.15.5:0/3964146365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:20.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.971+0000 7f7bd6ffd640 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac40060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3964146365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 msgr2=0x56008ac40060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac40060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3964146365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 msgr2=0x56008ac44d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac44d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bd7fff640 1 -- 172.21.15.5:0/3964146365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008ac40da0 con 0x56008ac3c680 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac44d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bd7fff640 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac405a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00057b0 tx=0x7f7bd00126f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a878e80111ae4d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/3964146365 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd003d080 con 0x56008ac3c680 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3964146365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bcc004030 con 0x56008ac3c680 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/3964146365 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f7bd0010070 con 0x56008ac3c680 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/3964146365 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7bd0005bd0 con 0x56008ac3c680 2024-09-11T22:38:20.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bdfe75740 10 client.29444 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.972+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3964146365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56008acb6b90 con 0x56008ac3c680 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/3964146365 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bcc004030 con 0x56008ac3c680 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/3964146365 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f7bd0005000 con 0x56008ac3c680 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bdfe75740 10 client.29444 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bdfe75740 10 client.29444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bdfe75740 20 client.29444 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56008acb7200 0x56008acb95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bdfe75740 4 client.29444 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3964146365 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56008ab9ed60 con 0x56008acb7200 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56008acb7200 0x56008acb95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bd77fe640 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56008acb7200 0x56008acb95f0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e64ec212e033014 server_cookie=9eb77751c319e20d in_seq=0 out_seq=0 2024-09-11T22:38:20.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.973+0000 7f7bb6ffd640 10 client.29444 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bb6ffd640 1 -- 172.21.15.5:0/3964146365 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56008ab9ed60 con 0x56008acb7200 2024-09-11T22:38:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bb6ffd640 10 client.29444 handle_command_reply: tid=0 2024-09-11T22:38:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bdfe75740 1 client.29444 shutdown 2024-09-11T22:38:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3964146365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 msgr2=0x56008ac405a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac405a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7bd00057b0 tx=0x7f7bd00126f0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3964146365 shutdown_connections 2024-09-11T22:38:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56008acb7200 0x56008acb95f0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56008ac3c680 0x56008ac405a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56008ac31dc0 0x56008ac40060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bb6ffd640 0 client.29444 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bdfe75740 1 --2- 172.21.15.5:0/3964146365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56008ac314a0 0x56008ac44d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.975+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3964146365 >> 172.21.15.5:0/3964146365 conn(0x56008ac2da30 msgr2=0x56008ac3b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3964146365 shutdown_connections 2024-09-11T22:38:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdfe75740 1 -- 172.21.15.5:0/3964146365 wait complete. 2024-09-11T22:38:20.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdfe75740 20 client.29444 trim_cache size 0 max 16384 2024-09-11T22:38:20.981 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:38:20.981 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:38:20.981 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:38:20.981 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:38:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdda9c640 1 -- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7bc00a2d70 msgr2=0x7f7bc00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdda9c640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7bc00a2d70 0x7f7bc00a5230 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0010e30 tx=0x7f7bd0017000 comp rx=0 tx=0).stop 2024-09-11T22:38:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdda9c640 1 -- 172.21.15.5:0/2003912015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd810a560 msgr2=0x7f7bd819f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdda9c640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd810a560 0x7f7bd819f850 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800ea60 tx=0x7f7bc800ef20 comp rx=0 tx=0).stop 2024-09-11T22:38:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdda9c640 1 -- 172.21.15.5:0/2003912015 shutdown_connections 2024-09-11T22:38:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdda9c640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bd8113940 0x7f7bd81a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdda9c640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd810ae80 0x7f7bd819fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdda9c640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f7bc00a2d70 0x7f7bc00a5230 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdda9c640 1 --2- 172.21.15.5:0/2003912015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7bd810a560 0x7f7bd819f850 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.976+0000 7f7bdda9c640 1 -- 172.21.15.5:0/2003912015 >> 172.21.15.5:0/2003912015 conn(0x7f7bd806e140 msgr2=0x7f7bd8070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.977+0000 7f7bdda9c640 1 -- 172.21.15.5:0/2003912015 shutdown_connections 2024-09-11T22:38:20.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:20.977+0000 7f7bdda9c640 1 -- 172.21.15.5:0/2003912015 wait complete. 2024-09-11T22:38:21.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 42125 23.86 MB/sec execute 39 sec latency 261.618 ms 2024-09-11T22:38:21.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:21 smithi005 ceph-mon[30943]: pgmap v515: 97 pgs: 97 active+clean; 454 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 690 op/s 2024-09-11T22:38:21.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:21 smithi159 ceph-mon[35534]: pgmap v515: 97 pgs: 97 active+clean; 454 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 690 op/s 2024-09-11T22:38:21.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:21 smithi172 ceph-mon[35461]: pgmap v515: 97 pgs: 97 active+clean; 454 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.9 MiB/s wr, 690 op/s 2024-09-11T22:38:21.958 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:38:21.959 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:38:22.239 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:22.640 INFO:tasks.workunit.client.0.smithi005.stdout: 10 42625 23.85 MB/sec execute 40 sec latency 191.567 ms 2024-09-11T22:38:22.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.775+0000 7fab01c28640 1 -- 172.21.15.5:0/1800976637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faafc102790 msgr2=0x7faafc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.775+0000 7fab01c28640 1 --2- 172.21.15.5:0/1800976637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faafc102790 0x7faafc1067e0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009900 tx=0x7faaf002a410 comp rx=0 tx=0).stop 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.775+0000 7fab01c28640 1 -- 172.21.15.5:0/1800976637 shutdown_connections 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.775+0000 7fab01c28640 1 --2- 172.21.15.5:0/1800976637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faafc106f10 0x7faafc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.775+0000 7fab01c28640 1 --2- 172.21.15.5:0/1800976637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faafc102790 0x7faafc1067e0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.775+0000 7fab01c28640 1 --2- 172.21.15.5:0/1800976637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faafc101de0 0x7faafc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.775+0000 7fab01c28640 1 -- 172.21.15.5:0/1800976637 >> 172.21.15.5:0/1800976637 conn(0x7faafc0fd8c0 msgr2=0x7faafc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.775+0000 7fab01c28640 1 -- 172.21.15.5:0/1800976637 shutdown_connections 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.775+0000 7fab01c28640 1 -- 172.21.15.5:0/1800976637 wait complete. 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.776+0000 7fab01c28640 1 Processor -- start 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.776+0000 7fab01c28640 1 -- start start 2024-09-11T22:38:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.776+0000 7fab01c28640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faafc101de0 0x7faafc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.776+0000 7fab01c28640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faafc102790 0x7faafc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.776+0000 7fab01c28640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faafc106f10 0x7faafc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.776+0000 7fab01c28640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faafc111f70 con 0x7faafc106f10 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.776+0000 7fab01c28640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faafc111df0 con 0x7faafc102790 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.776+0000 7fab01c28640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1120f0 con 0x7faafc101de0 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faafc101de0 0x7faafc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faafc106f10 0x7faafc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafaffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faafc102790 0x7faafc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faafc101de0 0x7faafc198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:34906/0 (socket says 172.21.15.5:34906) 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafb7fe640 1 -- 172.21.15.5:0/3242827400 learned_addr learned my addr 172.21.15.5:0/3242827400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafb7fe640 1 -- 172.21.15.5:0/3242827400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faafc102790 msgr2=0x7faafc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafb7fe640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faafc102790 0x7faafc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafb7fe640 1 -- 172.21.15.5:0/3242827400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faafc106f10 msgr2=0x7faafc19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafb7fe640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faafc106f10 0x7faafc19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafb7fe640 1 -- 172.21.15.5:0/3242827400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc19d440 con 0x7faafc101de0 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafbfff640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faafc106f10 0x7faafc19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faafb7fe640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faafc101de0 0x7faafc198490 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faaec00e9f0 tx=0x7faaec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faaf8ff9640 1 -- 172.21.15.5:0/3242827400 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec00cd80 con 0x7faafc101de0 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7fab01c28640 1 -- 172.21.15.5:0/3242827400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc19d730 con 0x7faafc101de0 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faaf8ff9640 1 -- 172.21.15.5:0/3242827400 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaec010910 con 0x7faafc101de0 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7faaf8ff9640 1 -- 172.21.15.5:0/3242827400 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec0188f0 con 0x7faafc101de0 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.777+0000 7fab01c28640 1 -- 172.21.15.5:0/3242827400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc1a5010 con 0x7faafc101de0 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.778+0000 7fab01c28640 1 -- 172.21.15.5:0/3242827400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faafc1021c0 con 0x7faafc101de0 2024-09-11T22:38:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.780+0000 7faaf8ff9640 1 -- 172.21.15.5:0/3242827400 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7faaec010ac0 con 0x7faafc101de0 2024-09-11T22:38:22.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.780+0000 7faaf8ff9640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faae40a2cf0 0x7faae40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:22.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.780+0000 7faafaffd640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faae40a2cf0 0x7faae40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:22.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.781+0000 7faaf8ff9640 1 -- 172.21.15.5:0/3242827400 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7faaec0cfc50 con 0x7faafc101de0 2024-09-11T22:38:22.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.781+0000 7faafaffd640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faae40a2cf0 0x7faae40a51b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009230 tx=0x7faaf00058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:22.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:22.784+0000 7faaf8ff9640 1 -- 172.21.15.5:0/3242827400 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faaec09c1b0 con 0x7faafc101de0 2024-09-11T22:38:23.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.017+0000 7fab01c28640 1 -- 172.21.15.5:0/3242827400 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faafc10a0f0 con 0x7faafc101de0 2024-09-11T22:38:23.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.018+0000 7faaf8ff9640 1 -- 172.21.15.5:0/3242827400 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11967 (secure 0 0 0) 0x7faaec0a04b0 con 0x7faafc101de0 2024-09-11T22:38:23.019 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:38:23.019 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"btime":"2024-09-11T22:38:14:548820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:38:13.865471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:38:23.019 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2024-09-11T22:38:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 -- 172.21.15.5:0/3242827400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faae40a2cf0 msgr2=0x7faae40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faae40a2cf0 0x7faae40a51b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009230 tx=0x7faaf00058d0 comp rx=0 tx=0).stop 2024-09-11T22:38:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 -- 172.21.15.5:0/3242827400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faafc101de0 msgr2=0x7faafc198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faafc101de0 0x7faafc198490 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faaec00e9f0 tx=0x7faaec00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:38:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 -- 172.21.15.5:0/3242827400 shutdown_connections 2024-09-11T22:38:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faafc106f10 0x7faafc19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faae40a2cf0 0x7faae40a51b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faafc102790 0x7faafc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 --2- 172.21.15.5:0/3242827400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faafc101de0 0x7faafc198490 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 -- 172.21.15.5:0/3242827400 >> 172.21.15.5:0/3242827400 conn(0x7faafc0fd8c0 msgr2=0x7faafc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 -- 172.21.15.5:0/3242827400 shutdown_connections 2024-09-11T22:38:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:23.020+0000 7fab01c28640 1 -- 172.21.15.5:0/3242827400 wait complete. 2024-09-11T22:38:23.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43202 24.02 MB/sec execute 41 sec latency 137.166 ms 2024-09-11T22:38:23.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:23 smithi005 ceph-mon[30943]: pgmap v516: 97 pgs: 97 active+clean; 467 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 519 op/s 2024-09-11T22:38:23.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:23 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3242827400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:38:23.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:38:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:23 smithi159 ceph-mon[35534]: pgmap v516: 97 pgs: 97 active+clean; 467 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 519 op/s 2024-09-11T22:38:23.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:23 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3242827400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:38:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:23 smithi172 ceph-mon[35461]: pgmap v516: 97 pgs: 97 active+clean; 467 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.2 MiB/s wr, 519 op/s 2024-09-11T22:38:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:23 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3242827400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:38:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:24.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.607+0000 7fa4fdffb640 1 -- 172.21.15.5:0/905498722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00319b0 con 0x7fa500101df0 2024-09-11T22:38:24.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.608+0000 7fa505440640 1 -- 172.21.15.5:0/905498722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa500101df0 msgr2=0x7fa5001021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:24.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.608+0000 7fa505440640 1 --2- 172.21.15.5:0/905498722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa500101df0 0x7fa5001021d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009900 tx=0x7fa4f002a410 comp rx=0 tx=0).stop 2024-09-11T22:38:24.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.608+0000 7fa505440640 1 -- 172.21.15.5:0/905498722 shutdown_connections 2024-09-11T22:38:24.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.608+0000 7fa505440640 1 --2- 172.21.15.5:0/905498722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa500106f20 0x7fa50010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.608+0000 7fa505440640 1 --2- 172.21.15.5:0/905498722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5001027a0 0x7fa5001067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.608+0000 7fa505440640 1 --2- 172.21.15.5:0/905498722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa500101df0 0x7fa5001021d0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.608+0000 7fa505440640 1 -- 172.21.15.5:0/905498722 >> 172.21.15.5:0/905498722 conn(0x7fa5000fd8f0 msgr2=0x7fa5000ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:24.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.608+0000 7fa505440640 1 -- 172.21.15.5:0/905498722 shutdown_connections 2024-09-11T22:38:24.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.608+0000 7fa505440640 1 -- 172.21.15.5:0/905498722 wait complete. 2024-09-11T22:38:24.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.609+0000 7fa505440640 1 Processor -- start 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.609+0000 7fa505440640 1 -- start start 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.609+0000 7fa505440640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa500101df0 0x7fa5000744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.609+0000 7fa505440640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5001027a0 0x7fa500074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.609+0000 7fa505440640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa500106f20 0x7fa500074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.609+0000 7fa505440640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa500111f40 con 0x7fa5001027a0 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.609+0000 7fa505440640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa500111dc0 con 0x7fa500106f20 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.609+0000 7fa505440640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001120c0 con 0x7fa500101df0 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.609+0000 7fa4fe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5001027a0 0x7fa500074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.609+0000 7fa4fe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5001027a0 0x7fa500074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54948/0 (socket says 172.21.15.5:54948) 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.610+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1730816448 learned_addr learned my addr 172.21.15.5:0/1730816448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.610+0000 7fa4feffd640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa500101df0 0x7fa5000744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.610+0000 7fa4ff7fe640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa500106f20 0x7fa500074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.610+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1730816448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa500101df0 msgr2=0x7fa5000744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.610+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa500101df0 0x7fa5000744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.610+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1730816448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa500106f20 msgr2=0x7fa500074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.610+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa500106f20 0x7fa500074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.610+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1730816448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5001a68c0 con 0x7fa5001027a0 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.610+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5001027a0 0x7fa500074a30 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400e9f0 tx=0x7fa4f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.610+0000 7fa4e7fff640 1 -- 172.21.15.5:0/1730816448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f400cd80 con 0x7fa5001027a0 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.611+0000 7fa505440640 1 -- 172.21.15.5:0/1730816448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5001a6bb0 con 0x7fa5001027a0 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.611+0000 7fa505440640 1 -- 172.21.15.5:0/1730816448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5001a7140 con 0x7fa5001027a0 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.611+0000 7fa4e7fff640 1 -- 172.21.15.5:0/1730816448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4010910 con 0x7fa5001027a0 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.611+0000 7fa4e7fff640 1 -- 172.21.15.5:0/1730816448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40188f0 con 0x7fa5001027a0 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.613+0000 7fa507819740 1 Processor -- start 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa507819740 1 -- start start 2024-09-11T22:38:24.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4e7fff640 1 -- 172.21.15.5:0/1730816448 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa4f4018bd0 con 0x7fa5001027a0 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa507819740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f9ab40 0x5596b1f99090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa507819740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f995d0 0x5596b1f99a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa507819740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1fa6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa507819740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596b1eff2f0 con 0x5596b1f995d0 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa507819740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5596b1ea6cf0 con 0x5596b1f9ab40 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa507819740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5596b1efb2b0 con 0x5596b1fa4370 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f995d0 0x5596b1f99a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4ff7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f9ab40 0x5596b1f99090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4ff7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f9ab40 0x5596b1f99090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59726/0 (socket says 172.21.15.5:59726) 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4feffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f995d0 0x5596b1f99a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54956/0 (socket says 172.21.15.5:54956) 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4ff7fe640 1 -- 172.21.15.5:0/931441881 learned_addr learned my addr 172.21.15.5:0/931441881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/931441881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1fa6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4ff7fe640 1 -- 172.21.15.5:0/931441881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 msgr2=0x5596b1fa6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4ff7fe640 1 --2- 172.21.15.5:0/931441881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1fa6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4ff7fe640 1 -- 172.21.15.5:0/931441881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f995d0 msgr2=0x5596b1f99a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.614+0000 7fa4ff7fe640 1 --2- 172.21.15.5:0/931441881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f995d0 0x5596b1f99a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.615+0000 7fa4ff7fe640 1 -- 172.21.15.5:0/931441881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596b1e8b3a0 con 0x5596b1f9ab40 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.615+0000 7fa4feffd640 1 --2- 172.21.15.5:0/931441881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f995d0 0x5596b1f99a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.615+0000 7fa4e7fff640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4e80a2d70 0x7fa4e80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.615+0000 7fa4ff7fe640 1 --2- 172.21.15.5:0/931441881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f9ab40 0x5596b1f99090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009f50 tx=0x7fa4f80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15f59f8509edae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.615+0000 7fa4feffd640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4e80a2d70 0x7fa4e80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.615+0000 7fa4e57fa640 1 -- 172.21.15.5:0/931441881 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8004d20 con 0x5596b1f9ab40 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.615+0000 7fa4e7fff640 1 -- 172.21.15.5:0/1730816448 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7fa4f40cf700 con 0x7fa5001027a0 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.615+0000 7fa4e57fa640 1 -- 172.21.15.5:0/931441881 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8004050 con 0x5596b1f9ab40 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.615+0000 7fa4e57fa640 1 -- 172.21.15.5:0/931441881 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8005280 con 0x5596b1f9ab40 2024-09-11T22:38:24.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 -- 172.21.15.5:0/931441881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f9ab40 msgr2=0x5596b1f99090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 --2- 172.21.15.5:0/931441881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f9ab40 0x5596b1f99090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009f50 tx=0x7fa4f80118d0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 -- 172.21.15.5:0/931441881 shutdown_connections 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 --2- 172.21.15.5:0/931441881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1fa6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 --2- 172.21.15.5:0/931441881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f995d0 0x5596b1f99a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 --2- 172.21.15.5:0/931441881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f9ab40 0x5596b1f99090 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 -- 172.21.15.5:0/931441881 >> 172.21.15.5:0/931441881 conn(0x5596b1f62310 msgr2=0x5596b1f95c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 -- 172.21.15.5:0/931441881 shutdown_connections 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa4feffd640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4e80a2d70 0x7fa4e80a5230 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009230 tx=0x7fa4f00058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 -- 172.21.15.5:0/931441881 wait complete. 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 -- start start 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 Processor -- start 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.616+0000 7fa507819740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 0x5596b1fa3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa507819740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1fa0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa507819740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1fa1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa507819740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596b1fa7070 con 0x5596b1f9ab40 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa507819740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5596b1eba060 con 0x5596b1f995d0 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa507819740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5596b1fa71f0 con 0x5596b1fa4370 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa507819740 10 client.0 _ll_register_callbacks cb 0x5596b1c0d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4fe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1fa0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4feffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 0x5596b1fa3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4fe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1fa0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54964/0 (socket says 172.21.15.5:54964) 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1629630672 learned_addr learned my addr 172.21.15.5:0/1629630672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4ff7fe640 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1fa1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1629630672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 msgr2=0x5596b1fa1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1fa1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1629630672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 msgr2=0x5596b1fa3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:24.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 0x5596b1fa3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1629630672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596b1c703c0 con 0x5596b1f9ab40 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1fa0f20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40ccde0 tx=0x7fa4f409b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a03332e684ab2f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1629630672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40cd210 con 0x5596b1f9ab40 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1629630672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7fa4f409b250 con 0x5596b1f9ab40 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa507819740 1 -- 172.21.15.5:0/1629630672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f8004050 con 0x5596b1f9ab40 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1629630672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa4f40a47c0 con 0x5596b1f9ab40 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.617+0000 7fa507819740 10 client.20388 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa507819740 1 -- 172.21.15.5:0/1629630672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5596b1fb2a80 con 0x5596b1f9ab40 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1629630672 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8004050 con 0x5596b1f9ab40 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1629630672 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fa4f40d2ce0 con 0x5596b1f9ab40 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa507819740 10 client.20388 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa507819740 10 client.20388 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:38:24.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa507819740 20 client.20388 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa507819740 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5596b1fb3f40 0x5596b1fb63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa507819740 4 client.20388 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa507819740 1 -- 172.21.15.5:0/1629630672 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596b1f0c860 con 0x5596b1fb3f40 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa4feffd640 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5596b1fb3f40 0x5596b1fb63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa4feffd640 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5596b1fb3f40 0x5596b1fb63a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b080e69afb46bc0b server_cookie=2f13a44828bf0866 in_seq=0 out_seq=0 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.618+0000 7fa4e2ff5640 10 client.20388 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1629630672 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596b1f0c860 con 0x5596b1fb3f40 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa4e2ff5640 10 client.20388 handle_command_reply: tid=0 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 client.20388 shutdown 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 -- 172.21.15.5:0/1629630672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 msgr2=0x5596b1fa0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1fa0f20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40ccde0 tx=0x7fa4f409b830 comp rx=0 tx=0).stop 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 -- 172.21.15.5:0/1629630672 shutdown_connections 2024-09-11T22:38:24.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5596b1fb3f40 0x5596b1fb63a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1fa1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1fa0f20 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 --2- 172.21.15.5:0/1629630672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 0x5596b1fa3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa4e2ff5640 0 client.20388 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:38:24.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 -- 172.21.15.5:0/1629630672 >> 172.21.15.5:0/1629630672 conn(0x5596b1f62310 msgr2=0x5596b1f9b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:24.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 -- 172.21.15.5:0/1629630672 shutdown_connections 2024-09-11T22:38:24.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 1 -- 172.21.15.5:0/1629630672 wait complete. 2024-09-11T22:38:24.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.620+0000 7fa507819740 20 client.20388 trim_cache size 0 max 16384 2024-09-11T22:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.636+0000 7fa507819740 1 Processor -- start 2024-09-11T22:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.636+0000 7fa507819740 1 -- start start 2024-09-11T22:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.636+0000 7fa507819740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1f995d0 0x5596b1fb0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.636+0000 7fa507819740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1f9c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.636+0000 7fa507819740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1fa4370 0x5596b1f9c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.636+0000 7fa507819740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec00e910 con 0x5596b1f9ab40 2024-09-11T22:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.636+0000 7fa507819740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec010450 con 0x5596b1fa4370 2024-09-11T22:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.636+0000 7fa507819740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec00efe0 con 0x5596b1f995d0 2024-09-11T22:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.636+0000 7fa4fe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1f9c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4fe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1f9c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54966/0 (socket says 172.21.15.5:54966) 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1377371709 learned_addr learned my addr 172.21.15.5:0/1377371709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4feffd640 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1f995d0 0x5596b1fb0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4ff7fe640 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1fa4370 0x5596b1f9c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1377371709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1f995d0 msgr2=0x5596b1fb0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1f995d0 0x5596b1fb0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1377371709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1fa4370 msgr2=0x5596b1f9c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1fa4370 0x5596b1f9c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1377371709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596b1f9f470 con 0x5596b1f9ab40 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4feffd640 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1f995d0 0x5596b1fb0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4ff7fe640 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1fa4370 0x5596b1f9c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1f9c120 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40a4290 tx=0x7fa4f4010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839fdc30d9e4ca13 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4e4ff9640 1 -- 172.21.15.5:0/1377371709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40cd210 con 0x5596b1f9ab40 2024-09-11T22:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4e4ff9640 1 -- 172.21.15.5:0/1377371709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f40b8070 con 0x5596b1f9ab40 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.637+0000 7fa4e4ff9640 1 -- 172.21.15.5:0/1377371709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f409b340 con 0x5596b1f9ab40 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.638+0000 7fa507819740 1 -- 172.21.15.5:0/1377371709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 msgr2=0x5596b1f9c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.638+0000 7fa507819740 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1f9c120 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40a4290 tx=0x7fa4f4010bd0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.638+0000 7fa507819740 1 -- 172.21.15.5:0/1377371709 shutdown_connections 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.638+0000 7fa507819740 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1fa4370 0x5596b1f9c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.638+0000 7fa507819740 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1f9c120 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.638+0000 7fa507819740 1 --2- 172.21.15.5:0/1377371709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1f995d0 0x5596b1fb0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.638+0000 7fa507819740 1 -- 172.21.15.5:0/1377371709 >> 172.21.15.5:0/1377371709 conn(0x5596b1f62310 msgr2=0x5596b1fa0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.638+0000 7fa507819740 1 -- 172.21.15.5:0/1377371709 shutdown_connections 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.638+0000 7fa507819740 1 -- 172.21.15.5:0/1377371709 wait complete. 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa507819740 1 -- start start 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa507819740 1 Processor -- start 2024-09-11T22:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa507819740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 0x5596b1fac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa507819740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1facd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa507819740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1f9db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa507819740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec00fd80 con 0x5596b1f9ab40 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa507819740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec00f6b0 con 0x5596b1f995d0 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa507819740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa4ec00c910 con 0x5596b1fa4370 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa507819740 10 client.0 _ll_register_callbacks cb 0x7fa4f400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa4fe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1facd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa4feffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 0x5596b1fac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa4fe7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1facd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54980/0 (socket says 172.21.15.5:54980) 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa4feffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 0x5596b1fac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59752/0 (socket says 172.21.15.5:59752) 2024-09-11T22:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.639+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1443176217 learned_addr learned my addr 172.21.15.5:0/1443176217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:24.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43692 23.95 MB/sec execute 42 sec latency 138.676 ms 2024-09-11T22:38:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4ff7fe640 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1f9db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1443176217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 msgr2=0x5596b1f9db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1f9db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1443176217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 msgr2=0x5596b1fac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 0x5596b1fac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4fe7fc640 1 -- 172.21.15.5:0/1443176217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596b1f9e300 con 0x5596b1f9ab40 2024-09-11T22:38:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4feffd640 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 0x5596b1fac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4fe7fc640 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1facd90 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fa4f409bb90 tx=0x7fa4f40106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac72553e5aae5da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1443176217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40cd210 con 0x5596b1f9ab40 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1443176217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7fa4f409aac0 con 0x5596b1f9ab40 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.640+0000 7fa507819740 1 -- 172.21.15.5:0/1443176217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f40b8070 con 0x5596b1f9ab40 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1443176217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa4f40ba570 con 0x5596b1f9ab40 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa507819740 10 client.20394 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa507819740 1 -- 172.21.15.5:0/1443176217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5596b201e880 con 0x5596b1f9ab40 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1443176217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f409bc00 con 0x5596b1f9ab40 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1443176217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fa4f40d2ce0 con 0x5596b1f9ab40 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa507819740 10 client.20394 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa507819740 10 client.20394 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa507819740 20 client.20394 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa507819740 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5596b201ec70 0x5596b2021090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa507819740 4 client.20394 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.641+0000 7fa507819740 1 -- 172.21.15.5:0/1443176217 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa4f40b8070 con 0x5596b201ec70 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.642+0000 7fa4feffd640 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5596b201ec70 0x5596b2021090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.642+0000 7fa4e2ff5640 10 client.20394 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.642+0000 7fa4feffd640 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5596b201ec70 0x5596b2021090 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=66e67d6f30d1c0e1 server_cookie=cdc896b8a11f5e67 in_seq=0 out_seq=0 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.642+0000 7fa4e2ff5640 1 -- 172.21.15.5:0/1443176217 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa4f40b8070 con 0x5596b201ec70 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.642+0000 7fa4e2ff5640 10 client.20394 handle_command_reply: tid=0 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.642+0000 7fa507819740 1 client.20394 shutdown 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa507819740 1 -- 172.21.15.5:0/1443176217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 msgr2=0x5596b1facd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa507819740 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1facd90 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fa4f409bb90 tx=0x7fa4f40106e0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa507819740 1 -- 172.21.15.5:0/1443176217 shutdown_connections 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa507819740 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x5596b201ec70 0x5596b2021090 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa507819740 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596b1fa4370 0x5596b1f9db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa4e2ff5640 0 client.20394 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:38:24.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa507819740 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596b1f9ab40 0x5596b1facd90 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa507819740 1 --2- 172.21.15.5:0/1443176217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5596b1f995d0 0x5596b1fac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa507819740 1 -- 172.21.15.5:0/1443176217 >> 172.21.15.5:0/1443176217 conn(0x5596b1f62310 msgr2=0x5596b1fa2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:24.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa507819740 1 -- 172.21.15.5:0/1443176217 shutdown_connections 2024-09-11T22:38:24.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.643+0000 7fa507819740 1 -- 172.21.15.5:0/1443176217 wait complete. 2024-09-11T22:38:24.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.644+0000 7fa507819740 20 client.20394 trim_cache size 0 max 16384 2024-09-11T22:38:24.653 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.644+0000 7fa505440640 1 -- 172.21.15.5:0/1730816448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4e80a2d70 msgr2=0x7fa4e80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.644+0000 7fa505440640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4e80a2d70 0x7fa4e80a5230 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009230 tx=0x7fa4f00058d0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.644+0000 7fa505440640 1 -- 172.21.15.5:0/1730816448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5001027a0 msgr2=0x7fa500074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.644+0000 7fa505440640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5001027a0 0x7fa500074a30 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400e9f0 tx=0x7fa4f400eeb0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.645+0000 7fa505440640 1 -- 172.21.15.5:0/1730816448 shutdown_connections 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.645+0000 7fa505440640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa4e80a2d70 0x7fa4e80a5230 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.645+0000 7fa505440640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa500106f20 0x7fa500074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.645+0000 7fa505440640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5001027a0 0x7fa500074a30 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.645+0000 7fa505440640 1 --2- 172.21.15.5:0/1730816448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa500101df0 0x7fa5000744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.645+0000 7fa505440640 1 -- 172.21.15.5:0/1730816448 >> 172.21.15.5:0/1730816448 conn(0x7fa5000fd8f0 msgr2=0x7fa5000fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:24.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.645+0000 7fa505440640 1 -- 172.21.15.5:0/1730816448 shutdown_connections 2024-09-11T22:38:24.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:24.645+0000 7fa505440640 1 -- 172.21.15.5:0/1730816448 wait complete. 2024-09-11T22:38:25.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:38:25.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44273 24.36 MB/sec execute 43 sec latency 181.721 ms 2024-09-11T22:38:25.673 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:25.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:25 smithi159 ceph-mon[35534]: pgmap v517: 97 pgs: 97 active+clean; 467 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.9 MiB/s wr, 445 op/s 2024-09-11T22:38:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:25 smithi172 ceph-mon[35461]: pgmap v517: 97 pgs: 97 active+clean; 467 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.9 MiB/s wr, 445 op/s 2024-09-11T22:38:26.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:25 smithi005 ceph-mon[30943]: pgmap v517: 97 pgs: 97 active+clean; 467 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 7.9 MiB/s wr, 445 op/s 2024-09-11T22:38:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.622+0000 7f6d1145e640 1 -- 172.21.15.5:0/3378633349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d0c106f10 msgr2=0x7f6d0c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.622+0000 7f6d1145e640 1 --2- 172.21.15.5:0/3378633349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d0c106f10 0x7f6d0c10ea90 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009ca0 tx=0x7f6d0402a8c0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.622+0000 7f6d1145e640 1 -- 172.21.15.5:0/3378633349 shutdown_connections 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.622+0000 7f6d1145e640 1 --2- 172.21.15.5:0/3378633349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d0c106f10 0x7f6d0c10ea90 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.622+0000 7f6d1145e640 1 --2- 172.21.15.5:0/3378633349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d0c102790 0x7f6d0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.622+0000 7f6d1145e640 1 --2- 172.21.15.5:0/3378633349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d0c101de0 0x7f6d0c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.622+0000 7f6d1145e640 1 -- 172.21.15.5:0/3378633349 >> 172.21.15.5:0/3378633349 conn(0x7f6d0c0fd880 msgr2=0x7f6d0c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.622+0000 7f6d1145e640 1 -- 172.21.15.5:0/3378633349 shutdown_connections 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.622+0000 7f6d1145e640 1 -- 172.21.15.5:0/3378633349 wait complete. 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.623+0000 7f6d1145e640 1 Processor -- start 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.623+0000 7f6d1145e640 1 -- start start 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.623+0000 7f6d1145e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d0c101de0 0x7f6d0c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.623+0000 7f6d1145e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d0c102790 0x7f6d0c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.623+0000 7f6d1145e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d0c106f10 0x7f6d0c19cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.623+0000 7f6d1145e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c111fc0 con 0x7f6d0c101de0 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.623+0000 7f6d1145e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c111e40 con 0x7f6d0c106f10 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.623+0000 7f6d1145e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c112140 con 0x7f6d0c102790 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.623+0000 7f6d0affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d0c101de0 0x7f6d0c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d0affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d0c101de0 0x7f6d0c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54992/0 (socket says 172.21.15.5:54992) 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d0affd640 1 -- 172.21.15.5:0/2491819751 learned_addr learned my addr 172.21.15.5:0/2491819751 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d0a7fc640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d0c102790 0x7f6d0c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d0b7fe640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d0c106f10 0x7f6d0c19cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d0affd640 1 -- 172.21.15.5:0/2491819751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d0c102790 msgr2=0x7f6d0c1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d0affd640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d0c102790 0x7f6d0c1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d0affd640 1 -- 172.21.15.5:0/2491819751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d0c106f10 msgr2=0x7f6d0c19cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:26.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d0affd640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d0c106f10 0x7f6d0c19cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d0affd640 1 -- 172.21.15.5:0/2491819751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0c19d320 con 0x7f6d0c101de0 2024-09-11T22:38:26.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d0affd640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d0c101de0 0x7f6d0c198370 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00e9f0 tx=0x7f6cfc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:26.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6cf3fff640 1 -- 172.21.15.5:0/2491819751 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00cd80 con 0x7f6d0c101de0 2024-09-11T22:38:26.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6cf3fff640 1 -- 172.21.15.5:0/2491819751 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cfc010910 con 0x7f6d0c101de0 2024-09-11T22:38:26.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d1145e640 1 -- 172.21.15.5:0/2491819751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0c19d610 con 0x7f6d0c101de0 2024-09-11T22:38:26.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6cf3fff640 1 -- 172.21.15.5:0/2491819751 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc018990 con 0x7f6d0c101de0 2024-09-11T22:38:26.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.624+0000 7f6d1145e640 1 -- 172.21.15.5:0/2491819751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0c1a4ef0 con 0x7f6d0c101de0 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.627+0000 7f6cf3fff640 1 -- 172.21.15.5:0/2491819751 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6cfc010ac0 con 0x7f6d0c101de0 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.627+0000 7f6d13837740 1 Processor -- start 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.627+0000 7f6d13837740 1 -- start start 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d13837740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac5f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d13837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d13837740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1ac6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d13837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ee1abc52f0 con 0x55ee1ac5f5d0 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d13837740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ee1ab6ccf0 con 0x55ee1ac60b40 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d13837740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ee1abc12b0 con 0x55ee1ac6a370 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d0affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d0b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac5f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d0a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1ac6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d0b7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac5f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59774/0 (socket says 172.21.15.5:59774) 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6cf3fff640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6cf40a2d70 0x7f6cf40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d0b7fe640 1 -- 172.21.15.5:0/2863579177 learned_addr learned my addr 172.21.15.5:0/2863579177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6cf3fff640 1 -- 172.21.15.5:0/2491819751 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f6cfc0cf7d0 con 0x7f6d0c101de0 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d0b7fe640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6cf40a2d70 0x7f6cf40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d0a7fc640 1 -- 172.21.15.5:0/2863579177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 msgr2=0x55ee1ac5f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d0a7fc640 1 --2- 172.21.15.5:0/2863579177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac5f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.628+0000 7f6d0a7fc640 1 -- 172.21.15.5:0/2863579177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac5f5d0 msgr2=0x55ee1ac5fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.629+0000 7f6d0a7fc640 1 --2- 172.21.15.5:0/2863579177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac5fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.629+0000 7f6d0a7fc640 1 -- 172.21.15.5:0/2863579177 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee1ab513a0 con 0x55ee1ac6a370 2024-09-11T22:38:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.629+0000 7f6d0affd640 1 --2- 172.21.15.5:0/2863579177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.629+0000 7f6d0a7fc640 1 --2- 172.21.15.5:0/2863579177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1ac6c760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009bf0 tx=0x7f6d00011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da9ddc67796e1d27 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.629+0000 7f6cf17fa640 1 -- 172.21.15.5:0/2863579177 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00004050 con 0x55ee1ac6a370 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.629+0000 7f6d0b7fe640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6cf40a2d70 0x7f6cf40a5230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009430 tx=0x7f6d04005f50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.629+0000 7f6cf17fa640 1 -- 172.21.15.5:0/2863579177 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d00005500 con 0x55ee1ac6a370 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.629+0000 7f6cf17fa640 1 -- 172.21.15.5:0/2863579177 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00004d40 con 0x55ee1ac6a370 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.630+0000 7f6d13837740 1 -- 172.21.15.5:0/2863579177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 msgr2=0x55ee1ac6c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.630+0000 7f6d13837740 1 --2- 172.21.15.5:0/2863579177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1ac6c760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009bf0 tx=0x7f6d00011810 comp rx=0 tx=0).stop 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.630+0000 7f6d13837740 1 -- 172.21.15.5:0/2863579177 shutdown_connections 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.630+0000 7f6d13837740 1 --2- 172.21.15.5:0/2863579177 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1ac6c760 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.630+0000 7f6d13837740 1 --2- 172.21.15.5:0/2863579177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac5fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.630+0000 7f6d13837740 1 --2- 172.21.15.5:0/2863579177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac5f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.630+0000 7f6d13837740 1 -- 172.21.15.5:0/2863579177 >> 172.21.15.5:0/2863579177 conn(0x55ee1ac28310 msgr2=0x55ee1ac5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:26.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.630+0000 7f6d13837740 1 -- 172.21.15.5:0/2863579177 shutdown_connections 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.630+0000 7f6d13837740 1 -- 172.21.15.5:0/2863579177 wait complete. 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.630+0000 7f6d13837740 1 -- start start 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d13837740 1 Processor -- start 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d13837740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac69c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d13837740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac66f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d13837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 0x55ee1ac67450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d13837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ee1ac71480 con 0x55ee1ac6a370 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d13837740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ee1ab80060 con 0x55ee1ac60b40 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d13837740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ee1ac71600 con 0x55ee1ac5f5d0 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d13837740 10 client.0 _ll_register_callbacks cb 0x55ee1a8d38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 0x55ee1ac67450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 0x55ee1ac67450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55000/0 (socket says 172.21.15.5:55000) 2024-09-11T22:38:26.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d0b7fe640 1 -- 172.21.15.5:0/2864728698 learned_addr learned my addr 172.21.15.5:0/2864728698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d0affd640 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac69c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.631+0000 7f6d0a7fc640 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac66f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6d0b7fe640 1 -- 172.21.15.5:0/2864728698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 msgr2=0x55ee1ac69c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6d0b7fe640 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac69c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6d0b7fe640 1 -- 172.21.15.5:0/2864728698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 msgr2=0x55ee1ac66f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6d0b7fe640 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac66f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6d0b7fe640 1 -- 172.21.15.5:0/2864728698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee1a9363c0 con 0x55ee1ac6a370 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6d0a7fc640 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac66f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6d0b7fe640 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 0x55ee1ac67450 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f6d04033040 tx=0x7f6d04045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e6c9e74e0aed42 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6ceeff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6ceeff5640 1 -- 172.21.15.5:0/2864728698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04051020 con 0x55ee1ac6a370 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6ceeff5640 1 -- 172.21.15.5:0/2864728698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f6d0402fd70 con 0x55ee1ac6a370 2024-09-11T22:38:26.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6d13837740 1 -- 172.21.15.5:0/2864728698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d00005500 con 0x55ee1ac6a370 2024-09-11T22:38:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.632+0000 7f6ceeff5640 1 -- 172.21.15.5:0/2864728698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6d0405c050 con 0x55ee1ac6a370 2024-09-11T22:38:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.633+0000 7f6d13837740 10 client.20406 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.633+0000 7f6d13837740 1 -- 172.21.15.5:0/2864728698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ee1ac7ce30 con 0x55ee1ac6a370 2024-09-11T22:38:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.633+0000 7f6ceeff5640 1 -- 172.21.15.5:0/2864728698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d00005500 con 0x55ee1ac6a370 2024-09-11T22:38:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.633+0000 7f6ceeff5640 1 -- 172.21.15.5:0/2864728698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f6d0404d510 con 0x55ee1ac6a370 2024-09-11T22:38:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.633+0000 7f6d13837740 10 client.20406 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.633+0000 7f6d13837740 10 client.20406 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:38:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.633+0000 7f6d13837740 20 client.20406 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.633+0000 7f6d13837740 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55ee1ac7e2f0 0x55ee1ac80760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.633+0000 7f6d13837740 4 client.20406 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.633+0000 7f6d13837740 1 -- 172.21.15.5:0/2864728698 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee1abd2860 con 0x55ee1ac7e2f0 2024-09-11T22:38:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.634+0000 7f6d0affd640 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55ee1ac7e2f0 0x55ee1ac80760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.634+0000 7f6d0affd640 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55ee1ac7e2f0 0x55ee1ac80760 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9705af71906e31c3 server_cookie=e1d814bfdb17b7a6 in_seq=0 out_seq=0 2024-09-11T22:38:26.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.634+0000 7f6ceeff5640 10 client.20406 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.635+0000 7f6ceeff5640 1 -- 172.21.15.5:0/2864728698 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee1abd2860 con 0x55ee1ac7e2f0 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.635+0000 7f6ceeff5640 10 client.20406 handle_command_reply: tid=0 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.635+0000 7f6d13837740 1 client.20406 shutdown 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 1 -- 172.21.15.5:0/2864728698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 msgr2=0x55ee1ac67450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 0x55ee1ac67450 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f6d04033040 tx=0x7f6d04045000 comp rx=0 tx=0).stop 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 1 -- 172.21.15.5:0/2864728698 shutdown_connections 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 0x55ee1ac67450 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55ee1ac7e2f0 0x55ee1ac80760 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac66f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 1 --2- 172.21.15.5:0/2864728698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac69c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6ceeff5640 0 client.20406 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 1 -- 172.21.15.5:0/2864728698 >> 172.21.15.5:0/2864728698 conn(0x55ee1ac28310 msgr2=0x55ee1ac61930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 1 -- 172.21.15.5:0/2864728698 shutdown_connections 2024-09-11T22:38:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 1 -- 172.21.15.5:0/2864728698 wait complete. 2024-09-11T22:38:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.636+0000 7f6d13837740 20 client.20406 trim_cache size 0 max 16384 2024-09-11T22:38:26.644 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44648 24.32 MB/sec execute 44 sec latency 372.338 ms 2024-09-11T22:38:26.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.654+0000 7f6d13837740 1 Processor -- start 2024-09-11T22:38:26.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.654+0000 7f6d13837740 1 -- start start 2024-09-11T22:38:26.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.654+0000 7f6d13837740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac7a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d13837740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac5abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d13837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 0x55ee1ac86080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d0affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac7a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d0affd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac7a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:35014/0 (socket says 172.21.15.5:35014) 2024-09-11T22:38:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d0a7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac5abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d0a7fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac5abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59800/0 (socket says 172.21.15.5:59800) 2024-09-11T22:38:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d0affd640 1 -- 172.21.15.5:0/2786563656 learned_addr learned my addr 172.21.15.5:0/2786563656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d13837740 1 -- 172.21.15.5:0/2786563656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1ab8b0 con 0x55ee1ac6a370 2024-09-11T22:38:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d13837740 1 -- 172.21.15.5:0/2786563656 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1ad3f0 con 0x55ee1ac60b40 2024-09-11T22:38:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d13837740 1 -- 172.21.15.5:0/2786563656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1abf80 con 0x55ee1ac5f5d0 2024-09-11T22:38:26.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d0affd640 1 -- 172.21.15.5:0/2786563656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 msgr2=0x55ee1ac5abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d0affd640 1 --2- 172.21.15.5:0/2786563656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac5abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d0affd640 1 -- 172.21.15.5:0/2786563656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 msgr2=0x55ee1ac86080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d0affd640 1 --2- 172.21.15.5:0/2786563656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 0x55ee1ac86080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.655+0000 7f6d0affd640 1 -- 172.21.15.5:0/2786563656 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee1ac5c960 con 0x55ee1ac5f5d0 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.656+0000 7f6d0affd640 1 --2- 172.21.15.5:0/2786563656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac7a720 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0ccf10 tx=0x7f6cfc0d1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32fe40edd8b94667 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.656+0000 7f6cf0ff9640 1 -- 172.21.15.5:0/2786563656 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0a48f0 con 0x55ee1ac5f5d0 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.656+0000 7f6cf0ff9640 1 -- 172.21.15.5:0/2786563656 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cfc099f80 con 0x55ee1ac5f5d0 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.656+0000 7f6cf0ff9640 1 -- 172.21.15.5:0/2786563656 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc09b0d0 con 0x55ee1ac5f5d0 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.657+0000 7f6d13837740 1 -- 172.21.15.5:0/2786563656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 msgr2=0x55ee1ac7a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.657+0000 7f6d13837740 1 --2- 172.21.15.5:0/2786563656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac7a720 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc0ccf10 tx=0x7f6cfc0d1040 comp rx=0 tx=0).stop 2024-09-11T22:38:26.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.657+0000 7f6d13837740 1 -- 172.21.15.5:0/2786563656 shutdown_connections 2024-09-11T22:38:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.657+0000 7f6d13837740 1 --2- 172.21.15.5:0/2786563656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac6a370 0x55ee1ac86080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.657+0000 7f6d13837740 1 --2- 172.21.15.5:0/2786563656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac60b40 0x55ee1ac5abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.657+0000 7f6d13837740 1 --2- 172.21.15.5:0/2786563656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac7a720 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.657+0000 7f6d13837740 1 -- 172.21.15.5:0/2786563656 >> 172.21.15.5:0/2786563656 conn(0x55ee1ac28310 msgr2=0x55ee1ac5a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:26.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.658+0000 7f6d13837740 1 -- 172.21.15.5:0/2786563656 shutdown_connections 2024-09-11T22:38:26.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.658+0000 7f6d13837740 1 -- 172.21.15.5:0/2786563656 wait complete. 2024-09-11T22:38:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.658+0000 7f6d13837740 1 -- start start 2024-09-11T22:38:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.658+0000 7f6d13837740 1 Processor -- start 2024-09-11T22:38:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d13837740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d13837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac60b40 0x55ee1ac72ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d13837740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1acf9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d13837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1acd20 con 0x55ee1ac60b40 2024-09-11T22:38:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d0b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1acf9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d0b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1acf9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:35028/0 (socket says 172.21.15.5:35028) 2024-09-11T22:38:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d0b7fe640 1 -- 172.21.15.5:0/1235270833 learned_addr learned my addr 172.21.15.5:0/1235270833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:26.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d13837740 1 -- 172.21.15.5:0/1235270833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1ac650 con 0x55ee1ac5f5d0 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d13837740 1 -- 172.21.15.5:0/1235270833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0c1a98b0 con 0x55ee1ac6a370 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d0a7fc640 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac60b40 0x55ee1ac72ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d0affd640 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.659+0000 7f6d13837740 10 client.0 _ll_register_callbacks cb 0x7f6d00005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.660+0000 7f6d0b7fe640 1 -- 172.21.15.5:0/1235270833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac5f5d0 msgr2=0x55ee1ac729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.660+0000 7f6d0b7fe640 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.660+0000 7f6d0b7fe640 1 -- 172.21.15.5:0/1235270833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac60b40 msgr2=0x55ee1ac72ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.660+0000 7f6d0b7fe640 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac60b40 0x55ee1ac72ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.660+0000 7f6d0b7fe640 1 -- 172.21.15.5:0/1235270833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee1acf95a0 con 0x55ee1ac6a370 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.660+0000 7f6d0b7fe640 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1acf9060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6d0402ade0 tx=0x7f6d04009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=268ad22aadad2a9e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.660+0000 7f6ceeff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:38:26.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.660+0000 7f6ceeff5640 1 -- 172.21.15.5:0/1235270833 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d04051070 con 0x55ee1ac6a370 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.661+0000 7f6d13837740 1 -- 172.21.15.5:0/1235270833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cfc099f80 con 0x55ee1ac6a370 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.661+0000 7f6ceeff5640 1 -- 172.21.15.5:0/1235270833 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f6d0404d480 con 0x55ee1ac6a370 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.661+0000 7f6ceeff5640 1 -- 172.21.15.5:0/1235270833 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6d04009b20 con 0x55ee1ac6a370 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.661+0000 7f6d13837740 10 client.29323 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.661+0000 7f6d13837740 1 -- 172.21.15.5:0/1235270833 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ee1acfa6c0 con 0x55ee1ac6a370 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.661+0000 7f6ceeff5640 1 -- 172.21.15.5:0/1235270833 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cfc099f80 con 0x55ee1ac6a370 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.661+0000 7f6ceeff5640 1 -- 172.21.15.5:0/1235270833 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f6d04068050 con 0x55ee1ac6a370 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.662+0000 7f6d13837740 10 client.29323 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.662+0000 7f6d13837740 10 client.29323 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.662+0000 7f6d13837740 20 client.29323 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.662+0000 7f6d13837740 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55ee1acfadd0 0x55ee1acfd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.662+0000 7f6d13837740 4 client.29323 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:38:26.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.662+0000 7f6d13837740 1 -- 172.21.15.5:0/1235270833 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee1abccd70 con 0x55ee1acfadd0 2024-09-11T22:38:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.662+0000 7f6d0affd640 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55ee1acfadd0 0x55ee1acfd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.662+0000 7f6d0affd640 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55ee1acfadd0 0x55ee1acfd1f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ceb299f9a46e9321 server_cookie=81971a74c83993a5 in_seq=0 out_seq=0 2024-09-11T22:38:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.662+0000 7f6ceeff5640 10 client.29323 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:38:26.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6ceeff5640 1 -- 172.21.15.5:0/1235270833 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee1abccd70 con 0x55ee1acfadd0 2024-09-11T22:38:26.670 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:38:26.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6ceeff5640 10 client.29323 handle_command_reply: tid=0 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 client.29323 shutdown 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 -- 172.21.15.5:0/1235270833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 msgr2=0x55ee1acf9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1acf9060 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6d0402ade0 tx=0x7f6d04009540 comp rx=0 tx=0).stop 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 -- 172.21.15.5:0/1235270833 shutdown_connections 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ee1ac6a370 0x55ee1acf9060 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ee1ac60b40 0x55ee1ac72ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55ee1acfadd0 0x55ee1acfd1f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6ceeff5640 0 client.29323 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 --2- 172.21.15.5:0/1235270833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ee1ac5f5d0 0x55ee1ac729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 -- 172.21.15.5:0/1235270833 >> 172.21.15.5:0/1235270833 conn(0x55ee1ac28310 msgr2=0x55ee1ac5ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 -- 172.21.15.5:0/1235270833 shutdown_connections 2024-09-11T22:38:26.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 1 -- 172.21.15.5:0/1235270833 wait complete. 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.663+0000 7f6d13837740 20 client.29323 trim_cache size 0 max 16384 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 -- 172.21.15.5:0/2491819751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6cf40a2d70 msgr2=0x7f6cf40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6cf40a2d70 0x7f6cf40a5230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6d04009430 tx=0x7f6d04005f50 comp rx=0 tx=0).stop 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 -- 172.21.15.5:0/2491819751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d0c101de0 msgr2=0x7f6d0c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d0c101de0 0x7f6d0c198370 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00e9f0 tx=0x7f6cfc00eeb0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 -- 172.21.15.5:0/2491819751 shutdown_connections 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6cf40a2d70 0x7f6cf40a5230 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d0c106f10 0x7f6d0c19cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d0c102790 0x7f6d0c1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 --2- 172.21.15.5:0/2491819751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d0c101de0 0x7f6d0c198370 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 -- 172.21.15.5:0/2491819751 >> 172.21.15.5:0/2491819751 conn(0x7f6d0c0fd880 msgr2=0x7f6d0c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 -- 172.21.15.5:0/2491819751 shutdown_connections 2024-09-11T22:38:26.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:26.664+0000 7f6d1145e640 1 -- 172.21.15.5:0/2491819751 wait complete. 2024-09-11T22:38:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.120+0000 7f4653331640 1 -- 172.21.15.5:0/127660388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f464c102cc0 msgr2=0x7f464c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.120+0000 7f4653331640 1 --2- 172.21.15.5:0/127660388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f464c102cc0 0x7f464c1030a0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f4648009900 tx=0x7f464802a480 comp rx=0 tx=0).stop 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.120+0000 7f4653331640 1 -- 172.21.15.5:0/127660388 shutdown_connections 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.120+0000 7f4653331640 1 --2- 172.21.15.5:0/127660388 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464c10c110 0x7f464c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.120+0000 7f4653331640 1 --2- 172.21.15.5:0/127660388 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f464c1035e0 0x7f464c10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.120+0000 7f4653331640 1 --2- 172.21.15.5:0/127660388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f464c102cc0 0x7f464c1030a0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.120+0000 7f4653331640 1 -- 172.21.15.5:0/127660388 >> 172.21.15.5:0/127660388 conn(0x7f464c0fc6b0 msgr2=0x7f464c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.121+0000 7f4653331640 1 -- 172.21.15.5:0/127660388 shutdown_connections 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.121+0000 7f4653331640 1 -- 172.21.15.5:0/127660388 wait complete. 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.121+0000 7f4653331640 1 Processor -- start 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.121+0000 7f4653331640 1 -- start start 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f4653331640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464c102cc0 0x7f464c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f4653331640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f464c1035e0 0x7f464c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f4653331640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f464c10c110 0x7f464c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f4653331640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f464c111f40 con 0x7f464c10c110 2024-09-11T22:38:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f4653331640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f464c111dc0 con 0x7f464c1035e0 2024-09-11T22:38:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f4653331640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f464c1120c0 con 0x7f464c102cc0 2024-09-11T22:38:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46518ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f464c10c110 0x7f464c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46510cd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464c102cc0 0x7f464c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46518ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f464c10c110 0x7f464c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55046/0 (socket says 172.21.15.5:55046) 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46518ce640 1 -- 172.21.15.5:0/1522476583 learned_addr learned my addr 172.21.15.5:0/1522476583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46508cc640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f464c1035e0 0x7f464c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46518ce640 1 -- 172.21.15.5:0/1522476583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464c102cc0 msgr2=0x7f464c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46518ce640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464c102cc0 0x7f464c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46518ce640 1 -- 172.21.15.5:0/1522476583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f464c1035e0 msgr2=0x7f464c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46518ce640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f464c1035e0 0x7f464c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46518ce640 1 -- 172.21.15.5:0/1522476583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c19d340 con 0x7f464c10c110 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.122+0000 7f46518ce640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f464c10c110 0x7f464c19cc10 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f464400eb10 tx=0x7f464400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.123+0000 7f46427fc640 1 -- 172.21.15.5:0/1522476583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644004b20 con 0x7f464c10c110 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.123+0000 7f46427fc640 1 -- 172.21.15.5:0/1522476583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4644004cc0 con 0x7f464c10c110 2024-09-11T22:38:27.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.123+0000 7f4653331640 1 -- 172.21.15.5:0/1522476583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464c19d630 con 0x7f464c10c110 2024-09-11T22:38:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.123+0000 7f46427fc640 1 -- 172.21.15.5:0/1522476583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644010760 con 0x7f464c10c110 2024-09-11T22:38:27.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.123+0000 7f4653331640 1 -- 172.21.15.5:0/1522476583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464c1a4ec0 con 0x7f464c10c110 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.125+0000 7f46427fc640 1 -- 172.21.15.5:0/1522476583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4644005020 con 0x7f464c10c110 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46560cd740 1 Processor -- start 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46560cd740 1 -- start start 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46427fc640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f46380a2e20 0x7f46380a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46560cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb447d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46560cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4482d0 0x55f3fb456b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46560cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb4594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46560cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3fb3ac2f0 con 0x55f3fb4570c0 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46560cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f3fb354270 con 0x55f3fb4482d0 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46560cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f3fb3a82b0 con 0x55f3fb4479b0 2024-09-11T22:38:27.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46508cc640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f46380a2e20 0x7f46380a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46518ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb4594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46518ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb4594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55062/0 (socket says 172.21.15.5:55062) 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.126+0000 7f46518ce640 1 -- 172.21.15.5:0/3098500891 learned_addr learned my addr 172.21.15.5:0/3098500891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46508cc640 1 --2- 172.21.15.5:0/3098500891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4482d0 0x55f3fb456b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46427fc640 1 -- 172.21.15.5:0/1522476583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f46440cf9b0 con 0x7f464c10c110 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46510cd640 1 --2- 172.21.15.5:0/3098500891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb447d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46518ce640 1 -- 172.21.15.5:0/3098500891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 msgr2=0x55f3fb447d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46518ce640 1 --2- 172.21.15.5:0/3098500891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb447d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46518ce640 1 -- 172.21.15.5:0/3098500891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4482d0 msgr2=0x55f3fb456b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46518ce640 1 --2- 172.21.15.5:0/3098500891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4482d0 0x55f3fb456b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46518ce640 1 -- 172.21.15.5:0/3098500891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3fb353340 con 0x55f3fb4570c0 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46508cc640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f46380a2e20 0x7f46380a52e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f464c100240 tx=0x7f463c009aa0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:27.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46518ce640 1 --2- 172.21.15.5:0/3098500891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb4594b0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f4644015040 tx=0x7f464400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ce310798b4c4a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46377fe640 1 -- 172.21.15.5:0/3098500891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46440b9070 con 0x55f3fb4570c0 2024-09-11T22:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46377fe640 1 -- 172.21.15.5:0/3098500891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464409c500 con 0x55f3fb4570c0 2024-09-11T22:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.127+0000 7f46377fe640 1 -- 172.21.15.5:0/3098500891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4644003970 con 0x55f3fb4570c0 2024-09-11T22:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.128+0000 7f46560cd740 1 -- 172.21.15.5:0/3098500891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 msgr2=0x55f3fb4594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.128+0000 7f46560cd740 1 --2- 172.21.15.5:0/3098500891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb4594b0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f4644015040 tx=0x7f464400efd0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.128+0000 7f46560cd740 1 -- 172.21.15.5:0/3098500891 shutdown_connections 2024-09-11T22:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.128+0000 7f46560cd740 1 --2- 172.21.15.5:0/3098500891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb4594b0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.128+0000 7f46560cd740 1 --2- 172.21.15.5:0/3098500891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4482d0 0x55f3fb456b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.128+0000 7f46560cd740 1 --2- 172.21.15.5:0/3098500891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb447d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.128+0000 7f46560cd740 1 -- 172.21.15.5:0/3098500891 >> 172.21.15.5:0/3098500891 conn(0x55f3fb444060 msgr2=0x55f3fb444440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.128+0000 7f46560cd740 1 -- 172.21.15.5:0/3098500891 shutdown_connections 2024-09-11T22:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.128+0000 7f46560cd740 1 -- 172.21.15.5:0/3098500891 wait complete. 2024-09-11T22:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46560cd740 1 -- start start 2024-09-11T22:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46560cd740 1 Processor -- start 2024-09-11T22:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46560cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4479b0 0x55f3fb454000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46560cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4482d0 0x55f3fb454540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46560cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb454a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46560cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3fb459c90 con 0x55f3fb4570c0 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46560cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f3fb335290 con 0x55f3fb4479b0 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46560cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f3fb459e10 con 0x55f3fb4482d0 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46560cd740 10 client.0 _ll_register_callbacks cb 0x55f3fb366140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46518ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb454a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46508cc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4482d0 0x55f3fb454540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46510cd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4479b0 0x55f3fb454000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46518ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb454a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55066/0 (socket says 172.21.15.5:55066) 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.129+0000 7f46518ce640 1 -- 172.21.15.5:0/3193690481 learned_addr learned my addr 172.21.15.5:0/3193690481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f46508cc640 1 -- 172.21.15.5:0/3193690481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4479b0 msgr2=0x55f3fb454000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f46508cc640 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4479b0 0x55f3fb454000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f46508cc640 1 -- 172.21.15.5:0/3193690481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 msgr2=0x55f3fb454a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f46508cc640 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb454a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f46508cc640 1 -- 172.21.15.5:0/3193690481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3fb0bf210 con 0x55f3fb4482d0 2024-09-11T22:38:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f46518ce640 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb454a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f46508cc640 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4482d0 0x55f3fb454540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f463c011d60 tx=0x7f463c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bce69ed89df74ea server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f4634ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:38:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f4634ff9640 1 -- 172.21.15.5:0/3193690481 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c011d90 con 0x55f3fb4482d0 2024-09-11T22:38:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f46560cd740 1 -- 172.21.15.5:0/3193690481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f464409c500 con 0x55f3fb4482d0 2024-09-11T22:38:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f4634ff9640 1 -- 172.21.15.5:0/3193690481 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f463c003a70 con 0x55f3fb4482d0 2024-09-11T22:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f4634ff9640 1 -- 172.21.15.5:0/3193690481 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f463c003c70 con 0x55f3fb4482d0 2024-09-11T22:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f46560cd740 10 client.29335 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.130+0000 7f46560cd740 1 -- 172.21.15.5:0/3193690481 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f3fb18b620 con 0x55f3fb4482d0 2024-09-11T22:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.131+0000 7f4634ff9640 1 -- 172.21.15.5:0/3193690481 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f464409c500 con 0x55f3fb4482d0 2024-09-11T22:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.131+0000 7f4634ff9640 1 -- 172.21.15.5:0/3193690481 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f463c020050 con 0x55f3fb4482d0 2024-09-11T22:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.131+0000 7f46560cd740 10 client.29335 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.131+0000 7f46560cd740 10 client.29335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.131+0000 7f46560cd740 20 client.29335 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.131+0000 7f46560cd740 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3fb467130 0x55f3fb469520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.131+0000 7f46560cd740 4 client.29335 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.131+0000 7f46560cd740 1 -- 172.21.15.5:0/3193690481 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3fb3b9a20 con 0x55f3fb467130 2024-09-11T22:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.131+0000 7f46510cd640 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3fb467130 0x55f3fb469520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.132+0000 7f46510cd640 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3fb467130 0x55f3fb469520 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1400517ff208a89 server_cookie=e3bd7393c6eb9ee0 in_seq=0 out_seq=0 2024-09-11T22:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.132+0000 7f4634ff9640 10 client.29335 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.135+0000 7f4634ff9640 1 -- 172.21.15.5:0/3193690481 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3fb3b9a20 con 0x55f3fb467130 2024-09-11T22:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.135+0000 7f4634ff9640 10 client.29335 handle_command_reply: tid=0 2024-09-11T22:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.135+0000 7f46560cd740 1 client.29335 shutdown 2024-09-11T22:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 1 -- 172.21.15.5:0/3193690481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4482d0 msgr2=0x55f3fb454540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4482d0 0x55f3fb454540 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f463c011d60 tx=0x7f463c004730 comp rx=0 tx=0).stop 2024-09-11T22:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 1 -- 172.21.15.5:0/3193690481 shutdown_connections 2024-09-11T22:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3fb467130 0x55f3fb469520 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb454a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4482d0 0x55f3fb454540 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f4634ff9640 0 client.29335 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 1 --2- 172.21.15.5:0/3193690481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4479b0 0x55f3fb454000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 1 -- 172.21.15.5:0/3193690481 >> 172.21.15.5:0/3193690481 conn(0x55f3fb444060 msgr2=0x55f3fb4431c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 1 -- 172.21.15.5:0/3193690481 shutdown_connections 2024-09-11T22:38:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 1 -- 172.21.15.5:0/3193690481 wait complete. 2024-09-11T22:38:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.136+0000 7f46560cd740 20 client.29335 trim_cache size 0 max 16384 2024-09-11T22:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.152+0000 7f46560cd740 1 Processor -- start 2024-09-11T22:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.152+0000 7f46560cd740 1 -- start start 2024-09-11T22:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.152+0000 7f46560cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb46ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.152+0000 7f46560cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4482d0 0x55f3fb46f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.152+0000 7f46560cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4570c0 0x55f3fb451280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.152+0000 7f46560cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f464c1ab780 con 0x55f3fb4482d0 2024-09-11T22:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.152+0000 7f46560cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f464c1ad2c0 con 0x55f3fb4570c0 2024-09-11T22:38:27.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.152+0000 7f46560cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f464c1abe50 con 0x55f3fb4479b0 2024-09-11T22:38:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46508cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4482d0 0x55f3fb46f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46510cd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb46ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46508cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4482d0 0x55f3fb46f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55068/0 (socket says 172.21.15.5:55068) 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46508cc640 1 -- 172.21.15.5:0/1857131749 learned_addr learned my addr 172.21.15.5:0/1857131749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46518ce640 1 --2- 172.21.15.5:0/1857131749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4570c0 0x55f3fb451280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46508cc640 1 -- 172.21.15.5:0/1857131749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 msgr2=0x55f3fb46ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46508cc640 1 --2- 172.21.15.5:0/1857131749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb46ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46508cc640 1 -- 172.21.15.5:0/1857131749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4570c0 msgr2=0x55f3fb451280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46508cc640 1 --2- 172.21.15.5:0/1857131749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4570c0 0x55f3fb451280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46508cc640 1 -- 172.21.15.5:0/1857131749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3fb452fb0 con 0x55f3fb4482d0 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f46508cc640 1 --2- 172.21.15.5:0/1857131749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4482d0 0x55f3fb46f3c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f463c0117b0 tx=0x7f463c009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a136f49fba4e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f4636ffd640 1 -- 172.21.15.5:0/1857131749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c003e10 con 0x55f3fb4482d0 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f4636ffd640 1 -- 172.21.15.5:0/1857131749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f463c013610 con 0x55f3fb4482d0 2024-09-11T22:38:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.153+0000 7f4636ffd640 1 -- 172.21.15.5:0/1857131749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c02abc0 con 0x55f3fb4482d0 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.154+0000 7f46560cd740 1 -- 172.21.15.5:0/1857131749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4482d0 msgr2=0x55f3fb46f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.154+0000 7f46560cd740 1 --2- 172.21.15.5:0/1857131749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4482d0 0x55f3fb46f3c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f463c0117b0 tx=0x7f463c009cf0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.154+0000 7f46560cd740 1 -- 172.21.15.5:0/1857131749 shutdown_connections 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.154+0000 7f46560cd740 1 --2- 172.21.15.5:0/1857131749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4570c0 0x55f3fb451280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.154+0000 7f46560cd740 1 --2- 172.21.15.5:0/1857131749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4482d0 0x55f3fb46f3c0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.154+0000 7f46560cd740 1 --2- 172.21.15.5:0/1857131749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb46ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.154+0000 7f46560cd740 1 -- 172.21.15.5:0/1857131749 >> 172.21.15.5:0/1857131749 conn(0x55f3fb444060 msgr2=0x55f3fb456080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.154+0000 7f46560cd740 1 -- 172.21.15.5:0/1857131749 shutdown_connections 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.154+0000 7f46560cd740 1 -- 172.21.15.5:0/1857131749 wait complete. 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46560cd740 1 -- start start 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46560cd740 1 Processor -- start 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46560cd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb4511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46560cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4482d0 0x55f3fb451700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46560cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb451c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46560cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f464c1acbf0 con 0x55f3fb4570c0 2024-09-11T22:38:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46560cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f464c1ac520 con 0x55f3fb4482d0 2024-09-11T22:38:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46560cd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f464c1a9780 con 0x55f3fb4479b0 2024-09-11T22:38:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46560cd740 10 client.0 _ll_register_callbacks cb 0x7f463c009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46510cd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb4511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46508cc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4482d0 0x55f3fb451700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46510cd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb4511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:35096/0 (socket says 172.21.15.5:35096) 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46510cd640 1 -- 172.21.15.5:0/2636445810 learned_addr learned my addr 172.21.15.5:0/2636445810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.155+0000 7f46518ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb451c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f46510cd640 1 -- 172.21.15.5:0/2636445810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4482d0 msgr2=0x55f3fb451700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f46510cd640 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4482d0 0x55f3fb451700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f46510cd640 1 -- 172.21.15.5:0/2636445810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 msgr2=0x55f3fb451c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f46510cd640 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb451c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f46510cd640 1 -- 172.21.15.5:0/2636445810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3fb459820 con 0x55f3fb4479b0 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f46510cd640 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb4511c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4648031550 tx=0x7f46480042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f39aa60e0db0f60f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f4634ff9640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:38:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f4634ff9640 1 -- 172.21.15.5:0/2636445810 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4648035040 con 0x55f3fb4479b0 2024-09-11T22:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f4634ff9640 1 -- 172.21.15.5:0/2636445810 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f4648005450 con 0x55f3fb4479b0 2024-09-11T22:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f46560cd740 1 -- 172.21.15.5:0/2636445810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f463c013610 con 0x55f3fb4479b0 2024-09-11T22:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.156+0000 7f4634ff9640 1 -- 172.21.15.5:0/2636445810 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f4648005650 con 0x55f3fb4479b0 2024-09-11T22:38:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f46560cd740 10 client.29341 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f46560cd740 1 -- 172.21.15.5:0/2636445810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f3fb4d2130 con 0x55f3fb4479b0 2024-09-11T22:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f4634ff9640 1 -- 172.21.15.5:0/2636445810 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f463c013610 con 0x55f3fb4479b0 2024-09-11T22:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f4634ff9640 1 -- 172.21.15.5:0/2636445810 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f46480318b0 con 0x55f3fb4479b0 2024-09-11T22:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f46560cd740 10 client.29341 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f46560cd740 10 client.29341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f46560cd740 20 client.29341 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f46560cd740 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3fb4d2890 0x55f3fb4d4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f46560cd740 4 client.29341 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:38:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f46560cd740 1 -- 172.21.15.5:0/2636445810 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3fb3b3d70 con 0x55f3fb4d2890 2024-09-11T22:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.157+0000 7f46508cc640 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3fb4d2890 0x55f3fb4d4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.158+0000 7f4634ff9640 10 client.29341 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.158+0000 7f46508cc640 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3fb4d2890 0x55f3fb4d4cb0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a89bdbd27f0dfa server_cookie=83b5abc4896b86f2 in_seq=0 out_seq=0 2024-09-11T22:38:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.630+0000 7f4634ff9640 1 -- 172.21.15.5:0/2636445810 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3fb3b3d70 con 0x55f3fb4d2890 2024-09-11T22:38:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.630+0000 7f4634ff9640 10 client.29341 handle_command_reply: tid=0 2024-09-11T22:38:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.630+0000 7f46560cd740 1 client.29341 shutdown 2024-09-11T22:38:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 1 -- 172.21.15.5:0/2636445810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 msgr2=0x55f3fb4511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb4511c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4648031550 tx=0x7f46480042f0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 1 -- 172.21.15.5:0/2636445810 shutdown_connections 2024-09-11T22:38:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f3fb4d2890 0x55f3fb4d4cb0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f4634ff9640 0 client.29341 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3fb4570c0 0x55f3fb451c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3fb4482d0 0x55f3fb451700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 1 --2- 172.21.15.5:0/2636445810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3fb4479b0 0x55f3fb4511c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 1 -- 172.21.15.5:0/2636445810 >> 172.21.15.5:0/2636445810 conn(0x55f3fb444060 msgr2=0x55f3fb453fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 1 -- 172.21.15.5:0/2636445810 shutdown_connections 2024-09-11T22:38:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 1 -- 172.21.15.5:0/2636445810 wait complete. 2024-09-11T22:38:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.631+0000 7f46560cd740 20 client.29341 trim_cache size 0 max 16384 2024-09-11T22:38:27.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:38:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:38:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:38:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:38:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 -- 172.21.15.5:0/1522476583 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f46380a2e20 msgr2=0x7f46380a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f46380a2e20 0x7f46380a52e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f464c100240 tx=0x7f463c009aa0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 -- 172.21.15.5:0/1522476583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f464c10c110 msgr2=0x7f464c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:27.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f464c10c110 0x7f464c19cc10 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f464400eb10 tx=0x7f464400c2d0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 -- 172.21.15.5:0/1522476583 shutdown_connections 2024-09-11T22:38:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f464c10c110 0x7f464c19cc10 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f46380a2e20 0x7f46380a52e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f464c1035e0 0x7f464c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 --2- 172.21.15.5:0/1522476583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f464c102cc0 0x7f464c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 -- 172.21.15.5:0/1522476583 >> 172.21.15.5:0/1522476583 conn(0x7f464c0fc6b0 msgr2=0x7f464c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 -- 172.21.15.5:0/1522476583 shutdown_connections 2024-09-11T22:38:27.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:27.632+0000 7f4653331640 1 -- 172.21.15.5:0/1522476583 wait complete. 2024-09-11T22:38:27.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44936 24.08 MB/sec execute 45 sec latency 434.404 ms 2024-09-11T22:38:27.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:27 smithi005 ceph-mon[30943]: pgmap v518: 97 pgs: 97 active+clean; 467 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.7 MiB/s wr, 432 op/s 2024-09-11T22:38:27.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:27 smithi159 ceph-mon[35534]: pgmap v518: 97 pgs: 97 active+clean; 467 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.7 MiB/s wr, 432 op/s 2024-09-11T22:38:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:27 smithi172 ceph-mon[35461]: pgmap v518: 97 pgs: 97 active+clean; 467 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.7 MiB/s wr, 432 op/s 2024-09-11T22:38:27.925 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:38:28.220 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:28.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45346 23.94 MB/sec execute 46 sec latency 188.882 ms 2024-09-11T22:38:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.729+0000 7f95c3d61640 1 -- 172.21.15.5:0/3507146422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95bc102790 msgr2=0x7f95bc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.729+0000 7f95c3d61640 1 --2- 172.21.15.5:0/3507146422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95bc102790 0x7f95bc1067e0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009900 tx=0x7f95ac02a410 comp rx=0 tx=0).stop 2024-09-11T22:38:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.729+0000 7f95c3d61640 1 -- 172.21.15.5:0/3507146422 shutdown_connections 2024-09-11T22:38:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.729+0000 7f95c3d61640 1 --2- 172.21.15.5:0/3507146422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95bc106f10 0x7f95bc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.729+0000 7f95c3d61640 1 --2- 172.21.15.5:0/3507146422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95bc102790 0x7f95bc1067e0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.729+0000 7f95c3d61640 1 --2- 172.21.15.5:0/3507146422 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95bc101de0 0x7f95bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.729+0000 7f95c3d61640 1 -- 172.21.15.5:0/3507146422 >> 172.21.15.5:0/3507146422 conn(0x7f95bc0fd8e0 msgr2=0x7f95bc0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.729+0000 7f95c3d61640 1 -- 172.21.15.5:0/3507146422 shutdown_connections 2024-09-11T22:38:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.729+0000 7f95c3d61640 1 -- 172.21.15.5:0/3507146422 wait complete. 2024-09-11T22:38:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.730+0000 7f95c3d61640 1 Processor -- start 2024-09-11T22:38:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.730+0000 7f95c3d61640 1 -- start start 2024-09-11T22:38:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.730+0000 7f95c3d61640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95bc101de0 0x7f95bc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.730+0000 7f95c3d61640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95bc102790 0x7f95bc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.730+0000 7f95c3d61640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95bc106f10 0x7f95bc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.730+0000 7f95c3d61640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc111f70 con 0x7f95bc102790 2024-09-11T22:38:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.730+0000 7f95c3d61640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc111df0 con 0x7f95bc101de0 2024-09-11T22:38:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.730+0000 7f95c3d61640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1120f0 con 0x7f95bc106f10 2024-09-11T22:38:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c12fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95bc102790 0x7f95bc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c1afd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95bc101de0 0x7f95bc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c1afd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95bc101de0 0x7f95bc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59884/0 (socket says 172.21.15.5:59884) 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c1afd640 1 -- 172.21.15.5:0/2438736972 learned_addr learned my addr 172.21.15.5:0/2438736972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c12fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95bc102790 0x7f95bc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55086/0 (socket says 172.21.15.5:55086) 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c22fe640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95bc106f10 0x7f95bc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c1afd640 1 -- 172.21.15.5:0/2438736972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95bc106f10 msgr2=0x7f95bc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c1afd640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95bc106f10 0x7f95bc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c1afd640 1 -- 172.21.15.5:0/2438736972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95bc102790 msgr2=0x7f95bc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c1afd640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95bc102790 0x7f95bc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c1afd640 1 -- 172.21.15.5:0/2438736972 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95bc1a68f0 con 0x7f95bc101de0 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c12fc640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95bc102790 0x7f95bc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c22fe640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95bc106f10 0x7f95bc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.731+0000 7f95c1afd640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95bc101de0 0x7f95bc0744f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f95b800e9f0 tx=0x7f95b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.732+0000 7f95b2ffd640 1 -- 172.21.15.5:0/2438736972 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b800cd80 con 0x7f95bc101de0 2024-09-11T22:38:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.732+0000 7f95b2ffd640 1 -- 172.21.15.5:0/2438736972 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b8010910 con 0x7f95bc101de0 2024-09-11T22:38:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.732+0000 7f95b2ffd640 1 -- 172.21.15.5:0/2438736972 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8018990 con 0x7f95bc101de0 2024-09-11T22:38:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.732+0000 7f95c3d61640 1 -- 172.21.15.5:0/2438736972 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95bc1a6be0 con 0x7f95bc101de0 2024-09-11T22:38:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.732+0000 7f95c3d61640 1 -- 172.21.15.5:0/2438736972 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95bc1a7140 con 0x7f95bc101de0 2024-09-11T22:38:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.734+0000 7f95c613a740 1 Processor -- start 2024-09-11T22:38:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.734+0000 7f95c613a740 1 -- start start 2024-09-11T22:38:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c613a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf1698b40 0x55caf1697090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c613a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf1697a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c613a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf16a2370 0x55caf16a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c613a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55caf15fd2f0 con 0x55caf1698b40 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c613a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55caf15a4cf0 con 0x55caf16a2370 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c613a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55caf15f92b0 con 0x55caf16975d0 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c22fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf1698b40 0x55caf1697090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c22fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf1698b40 0x55caf1697090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55100/0 (socket says 172.21.15.5:55100) 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c22fe640 1 -- 172.21.15.5:0/1169356144 learned_addr learned my addr 172.21.15.5:0/1169356144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c12fc640 1 --2- 172.21.15.5:0/1169356144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf16a2370 0x55caf16a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95b2ffd640 1 -- 172.21.15.5:0/2438736972 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f95b8010ac0 con 0x7f95bc101de0 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c22fe640 1 -- 172.21.15.5:0/1169356144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 msgr2=0x55caf1697a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c22fe640 1 --2- 172.21.15.5:0/1169356144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf1697a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c22fe640 1 -- 172.21.15.5:0/1169356144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf16a2370 msgr2=0x55caf16a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c22fe640 1 --2- 172.21.15.5:0/1169356144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf16a2370 0x55caf16a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c22fe640 1 -- 172.21.15.5:0/1169356144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf15893a0 con 0x55caf1698b40 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.735+0000 7f95c22fe640 1 --2- 172.21.15.5:0/1169356144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf1698b40 0x55caf1697090 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f95b4011790 tx=0x7f95b4011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81df23f6ba24516 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95a7fff640 1 -- 172.21.15.5:0/1169356144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b4003c30 con 0x55caf1698b40 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95a7fff640 1 -- 172.21.15.5:0/1169356144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95b400fe80 con 0x55caf1698b40 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95a7fff640 1 -- 172.21.15.5:0/1169356144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b40193e0 con 0x55caf1698b40 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95b2ffd640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95a80a2d70 0x7f95a80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95c1afd640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95a80a2d70 0x7f95a80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95b2ffd640 1 -- 172.21.15.5:0/2438736972 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f95b80cf730 con 0x7f95bc101de0 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95c613a740 1 -- 172.21.15.5:0/1169356144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf1698b40 msgr2=0x55caf1697090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95c613a740 1 --2- 172.21.15.5:0/1169356144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf1698b40 0x55caf1697090 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f95b4011790 tx=0x7f95b4011c50 comp rx=0 tx=0).stop 2024-09-11T22:38:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95c613a740 1 -- 172.21.15.5:0/1169356144 shutdown_connections 2024-09-11T22:38:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95c613a740 1 --2- 172.21.15.5:0/1169356144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf16a2370 0x55caf16a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95c613a740 1 --2- 172.21.15.5:0/1169356144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf1697a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95c613a740 1 --2- 172.21.15.5:0/1169356144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf1698b40 0x55caf1697090 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.736+0000 7f95c613a740 1 -- 172.21.15.5:0/1169356144 >> 172.21.15.5:0/1169356144 conn(0x55caf1660310 msgr2=0x55caf1693c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.737+0000 7f95c613a740 1 -- 172.21.15.5:0/1169356144 shutdown_connections 2024-09-11T22:38:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.737+0000 7f95c1afd640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95a80a2d70 0x7f95a80a5230 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55caf1694ac0 tx=0x7f95b806ac80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.737+0000 7f95c613a740 1 -- 172.21.15.5:0/1169356144 wait complete. 2024-09-11T22:38:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.737+0000 7f95c613a740 1 -- start start 2024-09-11T22:38:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.737+0000 7f95c613a740 1 Processor -- start 2024-09-11T22:38:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c613a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16975d0 0x55caf16a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c613a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf169f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c613a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16a2370 0x55caf169f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c613a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55caf16a5070 con 0x55caf16975d0 2024-09-11T22:38:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c613a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55caf15b8060 con 0x55caf1698b40 2024-09-11T22:38:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c613a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55caf16a51f0 con 0x55caf16a2370 2024-09-11T22:38:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c613a740 10 client.0 _ll_register_callbacks cb 0x55caf130b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c12fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf169f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c22fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16975d0 0x55caf16a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c12fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf169f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59904/0 (socket says 172.21.15.5:59904) 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c12fc640 1 -- 172.21.15.5:0/793252899 learned_addr learned my addr 172.21.15.5:0/793252899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c22fe640 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16a2370 0x55caf169f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c12fc640 1 -- 172.21.15.5:0/793252899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16a2370 msgr2=0x55caf169f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c12fc640 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16a2370 0x55caf169f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c12fc640 1 -- 172.21.15.5:0/793252899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16975d0 msgr2=0x55caf16a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c12fc640 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16975d0 0x55caf16a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c12fc640 1 -- 172.21.15.5:0/793252899 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf136e3c0 con 0x55caf1698b40 2024-09-11T22:38:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.738+0000 7f95c22fe640 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16975d0 0x55caf16a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.739+0000 7f95c12fc640 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf169f050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009900 tx=0x7f95ac0317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843bc2accde6de8b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.739+0000 7f95a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:38:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.739+0000 7f95a57fa640 1 -- 172.21.15.5:0/793252899 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac035040 con 0x55caf1698b40 2024-09-11T22:38:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.739+0000 7f95a57fa640 1 -- 172.21.15.5:0/793252899 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f95ac004450 con 0x55caf1698b40 2024-09-11T22:38:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.739+0000 7f95c613a740 1 -- 172.21.15.5:0/793252899 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55caf160a860 con 0x55caf1698b40 2024-09-11T22:38:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.739+0000 7f95a57fa640 1 -- 172.21.15.5:0/793252899 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f95ac005020 con 0x55caf1698b40 2024-09-11T22:38:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.739+0000 7f95c613a740 10 client.29516 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.739+0000 7f95c613a740 1 -- 172.21.15.5:0/793252899 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55caf16b0b50 con 0x55caf1698b40 2024-09-11T22:38:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.740+0000 7f95a57fa640 1 -- 172.21.15.5:0/793252899 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ac0047a0 con 0x55caf1698b40 2024-09-11T22:38:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.740+0000 7f95a57fa640 1 -- 172.21.15.5:0/793252899 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f95ac0035f0 con 0x55caf1698b40 2024-09-11T22:38:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.740+0000 7f95c613a740 10 client.29516 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.740+0000 7f95c613a740 10 client.29516 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:38:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.740+0000 7f95c613a740 20 client.29516 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.740+0000 7f95c613a740 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55caf16b2010 0x55caf16b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.740+0000 7f95c613a740 4 client.29516 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.740+0000 7f95c613a740 1 -- 172.21.15.5:0/793252899 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caf160a860 con 0x55caf16b2010 2024-09-11T22:38:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.740+0000 7f95c22fe640 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55caf16b2010 0x55caf16b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.741+0000 7f95a57fa640 10 client.29516 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:38:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.741+0000 7f95c22fe640 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55caf16b2010 0x55caf16b4400 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f993418c7f1faa40 server_cookie=2d00c3e76678c8e1 in_seq=0 out_seq=0 2024-09-11T22:38:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.741+0000 7f95a57fa640 1 -- 172.21.15.5:0/793252899 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55caf160a860 con 0x55caf16b2010 2024-09-11T22:38:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.741+0000 7f95a57fa640 10 client.29516 handle_command_reply: tid=0 2024-09-11T22:38:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.741+0000 7f95c613a740 1 client.29516 shutdown 2024-09-11T22:38:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 1 -- 172.21.15.5:0/793252899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 msgr2=0x55caf169f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf169f050 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009900 tx=0x7f95ac0317e0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 1 -- 172.21.15.5:0/793252899 shutdown_connections 2024-09-11T22:38:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55caf16b2010 0x55caf16b4400 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16a2370 0x55caf169f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf169f050 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 1 --2- 172.21.15.5:0/793252899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16975d0 0x55caf16a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 1 -- 172.21.15.5:0/793252899 >> 172.21.15.5:0/793252899 conn(0x55caf1660310 msgr2=0x55caf1699960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95a57fa640 0 client.29516 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:38:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 1 -- 172.21.15.5:0/793252899 shutdown_connections 2024-09-11T22:38:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 1 -- 172.21.15.5:0/793252899 wait complete. 2024-09-11T22:38:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.742+0000 7f95c613a740 20 client.29516 trim_cache size 0 max 16384 2024-09-11T22:38:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.757+0000 7f95c613a740 1 Processor -- start 2024-09-11T22:38:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c613a740 1 -- start start 2024-09-11T22:38:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c613a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf169df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c613a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c613a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16a2370 0x55caf169a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c613a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1afc90 con 0x55caf16a2370 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c613a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1b17d0 con 0x55caf1698b40 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c613a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1b0360 con 0x55caf16975d0 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c12fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c22fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf169df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c12fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59906/0 (socket says 172.21.15.5:59906) 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c12fc640 1 -- 172.21.15.5:0/1616120272 learned_addr learned my addr 172.21.15.5:0/1616120272 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c22fe640 1 --2- 172.21.15.5:0/1616120272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16a2370 0x55caf169a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c12fc640 1 -- 172.21.15.5:0/1616120272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 msgr2=0x55caf169df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c12fc640 1 --2- 172.21.15.5:0/1616120272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf169df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c12fc640 1 -- 172.21.15.5:0/1616120272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16a2370 msgr2=0x55caf169a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c12fc640 1 --2- 172.21.15.5:0/1616120272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16a2370 0x55caf169a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.758+0000 7f95c12fc640 1 -- 172.21.15.5:0/1616120272 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caf1697d80 con 0x55caf1698b40 2024-09-11T22:38:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.759+0000 7f95c12fc640 1 --2- 172.21.15.5:0/1616120272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a0fc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f95ac031490 tx=0x7f95ac02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df8e20e159ec5b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.759+0000 7f95a77fe640 1 -- 172.21.15.5:0/1616120272 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac035030 con 0x55caf1698b40 2024-09-11T22:38:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.759+0000 7f95a77fe640 1 -- 172.21.15.5:0/1616120272 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ac003c30 con 0x55caf1698b40 2024-09-11T22:38:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.759+0000 7f95a77fe640 1 -- 172.21.15.5:0/1616120272 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac0046c0 con 0x55caf1698b40 2024-09-11T22:38:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.760+0000 7f95c613a740 1 -- 172.21.15.5:0/1616120272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 msgr2=0x55caf16a0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.760+0000 7f95c613a740 1 --2- 172.21.15.5:0/1616120272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a0fc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f95ac031490 tx=0x7f95ac02a410 comp rx=0 tx=0).stop 2024-09-11T22:38:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.760+0000 7f95c613a740 1 -- 172.21.15.5:0/1616120272 shutdown_connections 2024-09-11T22:38:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.760+0000 7f95c613a740 1 --2- 172.21.15.5:0/1616120272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16a2370 0x55caf169a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.760+0000 7f95c613a740 1 --2- 172.21.15.5:0/1616120272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a0fc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.760+0000 7f95c613a740 1 --2- 172.21.15.5:0/1616120272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf169df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.760+0000 7f95c613a740 1 -- 172.21.15.5:0/1616120272 >> 172.21.15.5:0/1616120272 conn(0x55caf1660310 msgr2=0x55caf16a08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.760+0000 7f95c613a740 1 -- 172.21.15.5:0/1616120272 shutdown_connections 2024-09-11T22:38:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.760+0000 7f95c613a740 1 -- 172.21.15.5:0/1616120272 wait complete. 2024-09-11T22:38:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c613a740 1 -- start start 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c613a740 1 Processor -- start 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c613a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf16aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c613a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c613a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16a2370 0x55caf16a6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c613a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1b1100 con 0x55caf16a2370 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c613a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1b0a30 con 0x55caf1698b40 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c613a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f95bc1adc70 con 0x55caf16975d0 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c613a740 10 client.0 _ll_register_callbacks cb 0x7f95ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c12fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c22fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf16aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c12fc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59914/0 (socket says 172.21.15.5:59914) 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.761+0000 7f95c12fc640 1 -- 172.21.15.5:0/387625479 learned_addr learned my addr 172.21.15.5:0/387625479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95c22fe640 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16a2370 0x55caf16a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95c12fc640 1 -- 172.21.15.5:0/387625479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 msgr2=0x55caf16aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95c12fc640 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf16aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95c12fc640 1 -- 172.21.15.5:0/387625479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16a2370 msgr2=0x55caf16a6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95c12fc640 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16a2370 0x55caf16a6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95c12fc640 1 -- 172.21.15.5:0/387625479 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caf16a6660 con 0x55caf1698b40 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95c12fc640 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a5be0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f95ac033000 tx=0x7f95ac005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b680d06aac0c3fa server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95a57fa640 1 -- 172.21.15.5:0/387625479 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ac035070 con 0x55caf1698b40 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95c613a740 1 -- 172.21.15.5:0/387625479 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95ac003c30 con 0x55caf1698b40 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95a57fa640 1 -- 172.21.15.5:0/387625479 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f95ac031a90 con 0x55caf1698b40 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95a57fa640 1 -- 172.21.15.5:0/387625479 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f95ac031c90 con 0x55caf1698b40 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95c613a740 10 client.29522 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:28.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.762+0000 7f95c613a740 1 -- 172.21.15.5:0/387625479 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55caf171c970 con 0x55caf1698b40 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.763+0000 7f95a57fa640 1 -- 172.21.15.5:0/387625479 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95ac003c30 con 0x55caf1698b40 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.763+0000 7f95a57fa640 1 -- 172.21.15.5:0/387625479 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f95ac004b40 con 0x55caf1698b40 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.763+0000 7f95c613a740 10 client.29522 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.763+0000 7f95c613a740 10 client.29522 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.763+0000 7f95c613a740 20 client.29522 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.763+0000 7f95c613a740 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55caf171cff0 0x55caf171f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.763+0000 7f95c613a740 4 client.29522 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.763+0000 7f95c613a740 1 -- 172.21.15.5:0/387625479 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55caf1604d70 con 0x55caf171cff0 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.763+0000 7f95c22fe640 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55caf171cff0 0x55caf171f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95a57fa640 10 client.29522 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c22fe640 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55caf171cff0 0x55caf171f410 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f397c38312f95fc server_cookie=867aecdc46bf6114 in_seq=0 out_seq=0 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95a57fa640 1 -- 172.21.15.5:0/387625479 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55caf1604d70 con 0x55caf171cff0 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95a57fa640 10 client.29522 handle_command_reply: tid=0 2024-09-11T22:38:28.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 client.29522 shutdown 2024-09-11T22:38:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 -- 172.21.15.5:0/387625479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 msgr2=0x55caf16a5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a5be0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f95ac033000 tx=0x7f95ac005b50 comp rx=0 tx=0).stop 2024-09-11T22:38:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 -- 172.21.15.5:0/387625479 shutdown_connections 2024-09-11T22:38:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55caf171cff0 0x55caf171f410 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55caf16a2370 0x55caf16a6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55caf1698b40 0x55caf16a5be0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 --2- 172.21.15.5:0/387625479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55caf16975d0 0x55caf16aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95a57fa640 0 client.29522 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:38:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 -- 172.21.15.5:0/387625479 >> 172.21.15.5:0/387625479 conn(0x55caf1660310 msgr2=0x55caf16a10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 -- 172.21.15.5:0/387625479 shutdown_connections 2024-09-11T22:38:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.764+0000 7f95c613a740 1 -- 172.21.15.5:0/387625479 wait complete. 2024-09-11T22:38:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.765+0000 7f95c613a740 20 client.29522 trim_cache size 0 max 16384 2024-09-11T22:38:28.771 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:38:28.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.765+0000 7f95c3d61640 1 -- 172.21.15.5:0/2438736972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95a80a2d70 msgr2=0x7f95a80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.765+0000 7f95c3d61640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95a80a2d70 0x7f95a80a5230 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55caf1694ac0 tx=0x7f95b806ac80 comp rx=0 tx=0).stop 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.765+0000 7f95c3d61640 1 -- 172.21.15.5:0/2438736972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95bc101de0 msgr2=0x7f95bc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.765+0000 7f95c3d61640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95bc101de0 0x7f95bc0744f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f95b800e9f0 tx=0x7f95b800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.765+0000 7f95c3d61640 1 -- 172.21.15.5:0/2438736972 shutdown_connections 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.765+0000 7f95c3d61640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95bc106f10 0x7f95bc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.765+0000 7f95c3d61640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95bc102790 0x7f95bc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.765+0000 7f95c3d61640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95a80a2d70 0x7f95a80a5230 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.765+0000 7f95c3d61640 1 --2- 172.21.15.5:0/2438736972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95bc101de0 0x7f95bc0744f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.766+0000 7f95c3d61640 1 -- 172.21.15.5:0/2438736972 >> 172.21.15.5:0/2438736972 conn(0x7f95bc0fd8e0 msgr2=0x7f95bc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.766+0000 7f95c3d61640 1 -- 172.21.15.5:0/2438736972 shutdown_connections 2024-09-11T22:38:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:28.766+0000 7f95c3d61640 1 -- 172.21.15.5:0/2438736972 wait complete. 2024-09-11T22:38:29.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45859 24.02 MB/sec execute 47 sec latency 183.247 ms 2024-09-11T22:38:29.827 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:38:29.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:29 smithi159 ceph-mon[35534]: pgmap v519: 97 pgs: 97 active+clean; 479 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 640 op/s 2024-09-11T22:38:29.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:29 smithi005 ceph-mon[30943]: pgmap v519: 97 pgs: 97 active+clean; 479 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 640 op/s 2024-09-11T22:38:29.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:29 smithi172 ceph-mon[35461]: pgmap v519: 97 pgs: 97 active+clean; 479 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 640 op/s 2024-09-11T22:38:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.623+0000 7f67e0eb6640 1 -- 172.21.15.5:0/3709925606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67dc101de0 msgr2=0x7f67dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.623+0000 7f67e0eb6640 1 --2- 172.21.15.5:0/3709925606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67dc101de0 0x7f67dc1021c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009900 tx=0x7f67cc02a410 comp rx=0 tx=0).stop 2024-09-11T22:38:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.623+0000 7f67e0eb6640 1 -- 172.21.15.5:0/3709925606 shutdown_connections 2024-09-11T22:38:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.623+0000 7f67e0eb6640 1 --2- 172.21.15.5:0/3709925606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67dc106f10 0x7f67dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.623+0000 7f67e0eb6640 1 --2- 172.21.15.5:0/3709925606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67dc102790 0x7f67dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.623+0000 7f67e0eb6640 1 --2- 172.21.15.5:0/3709925606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67dc101de0 0x7f67dc1021c0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.623+0000 7f67e0eb6640 1 -- 172.21.15.5:0/3709925606 >> 172.21.15.5:0/3709925606 conn(0x7f67dc0fd8e0 msgr2=0x7f67dc0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.623+0000 7f67e0eb6640 1 -- 172.21.15.5:0/3709925606 shutdown_connections 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.623+0000 7f67e0eb6640 1 -- 172.21.15.5:0/3709925606 wait complete. 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.624+0000 7f67e0eb6640 1 Processor -- start 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.624+0000 7f67e0eb6640 1 -- start start 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.624+0000 7f67e0eb6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67dc101de0 0x7f67dc198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.624+0000 7f67e0eb6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67dc102790 0x7f67dc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67e0eb6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67dc106f10 0x7f67dc19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67e0eb6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc111f70 con 0x7f67dc101de0 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67e0eb6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc111df0 con 0x7f67dc106f10 2024-09-11T22:38:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67e0eb6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1120f0 con 0x7f67dc102790 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67d9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67dc102790 0x7f67dc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67d9d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67dc102790 0x7f67dc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:35174/0 (socket says 172.21.15.5:35174) 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67dc101de0 0x7f67dc198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67d9d9b640 1 -- 172.21.15.5:0/2971939356 learned_addr learned my addr 172.21.15.5:0/2971939356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67dad9d640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67dc106f10 0x7f67dc19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67d9d9b640 1 -- 172.21.15.5:0/2971939356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67dc106f10 msgr2=0x7f67dc19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67d9d9b640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67dc106f10 0x7f67dc19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67d9d9b640 1 -- 172.21.15.5:0/2971939356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67dc101de0 msgr2=0x7f67dc198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67d9d9b640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67dc101de0 0x7f67dc198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67d9d9b640 1 -- 172.21.15.5:0/2971939356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc19d400 con 0x7f67dc102790 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.625+0000 7f67da59c640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67dc101de0 0x7f67dc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.626+0000 7f67d9d9b640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67dc102790 0x7f67dc198990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e9f0 tx=0x7f67d000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:30.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.626+0000 7f67cb7fe640 1 -- 172.21.15.5:0/2971939356 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d000cd80 con 0x7f67dc102790 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.626+0000 7f67cb7fe640 1 -- 172.21.15.5:0/2971939356 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d0005500 con 0x7f67dc102790 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.626+0000 7f67e0eb6640 1 -- 172.21.15.5:0/2971939356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc19d6f0 con 0x7f67dc102790 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.626+0000 7f67cb7fe640 1 -- 172.21.15.5:0/2971939356 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0010690 con 0x7f67dc102790 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.626+0000 7f67e0eb6640 1 -- 172.21.15.5:0/2971939356 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc1a4fd0 con 0x7f67dc102790 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.629+0000 7f67cb7fe640 1 -- 172.21.15.5:0/2971939356 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f67d00046c0 con 0x7f67dc102790 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.629+0000 7f67e328f740 1 Processor -- start 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67e328f740 1 -- start start 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67e328f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 0x555ea532e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67e328f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532e5d0 0x555ea532ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67e328f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 0x555ea533fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67e328f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555ea52942f0 con 0x555ea532e5d0 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67e328f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555ea523bcf0 con 0x555ea532fb40 2024-09-11T22:38:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67e328f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555ea52902b0 con 0x555ea533d780 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67cb7fe640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67c40a2cf0 0x7f67c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532e5d0 0x555ea532ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67da59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532e5d0 0x555ea532ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55160/0 (socket says 172.21.15.5:55160) 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67da59c640 1 -- 172.21.15.5:0/1387965752 learned_addr learned my addr 172.21.15.5:0/1387965752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67dad9d640 1 --2- 172.21.15.5:0/1387965752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 0x555ea532e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67dad9d640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67c40a2cf0 0x7f67c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67d9d9b640 1 --2- 172.21.15.5:0/1387965752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 0x555ea533fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.630+0000 7f67cb7fe640 1 -- 172.21.15.5:0/2971939356 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f67d00cf840 con 0x7f67dc102790 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.631+0000 7f67dad9d640 1 -- 172.21.15.5:0/1387965752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 msgr2=0x555ea533fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.631+0000 7f67dad9d640 1 --2- 172.21.15.5:0/1387965752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 0x555ea533fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.631+0000 7f67dad9d640 1 -- 172.21.15.5:0/1387965752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532e5d0 msgr2=0x555ea532ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.631+0000 7f67dad9d640 1 --2- 172.21.15.5:0/1387965752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532e5d0 0x555ea532ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.631+0000 7f67dad9d640 1 -- 172.21.15.5:0/1387965752 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea52203a0 con 0x555ea532fb40 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.631+0000 7f67dad9d640 1 --2- 172.21.15.5:0/1387965752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 0x555ea532e090 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f67d4014e40 tx=0x7f67d40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd341fbf9d8dd57 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.631+0000 7f67dad9d640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67c40a2cf0 0x7f67c40a51b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f67d4011cb0 tx=0x7f67d4009000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.631+0000 7f67c8ff9640 1 -- 172.21.15.5:0/1387965752 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d4007b30 con 0x555ea532fb40 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.631+0000 7f67c8ff9640 1 -- 172.21.15.5:0/1387965752 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d4020920 con 0x555ea532fb40 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.631+0000 7f67c8ff9640 1 -- 172.21.15.5:0/1387965752 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d400f7c0 con 0x555ea532fb40 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.632+0000 7f67e328f740 1 -- 172.21.15.5:0/1387965752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 msgr2=0x555ea532e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.632+0000 7f67e328f740 1 --2- 172.21.15.5:0/1387965752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 0x555ea532e090 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f67d4014e40 tx=0x7f67d40117b0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.632+0000 7f67e328f740 1 -- 172.21.15.5:0/1387965752 shutdown_connections 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.632+0000 7f67e328f740 1 --2- 172.21.15.5:0/1387965752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 0x555ea533fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.632+0000 7f67e328f740 1 --2- 172.21.15.5:0/1387965752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532e5d0 0x555ea532ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.632+0000 7f67e328f740 1 --2- 172.21.15.5:0/1387965752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 0x555ea532e090 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.632+0000 7f67e328f740 1 -- 172.21.15.5:0/1387965752 >> 172.21.15.5:0/1387965752 conn(0x555ea52f7310 msgr2=0x555ea532ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.632+0000 7f67e328f740 1 -- 172.21.15.5:0/1387965752 shutdown_connections 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.632+0000 7f67e328f740 1 -- 172.21.15.5:0/1387965752 wait complete. 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67e328f740 1 -- start start 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67e328f740 1 Processor -- start 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67e328f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532e5d0 0x555ea533a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67e328f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 0x555ea533aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67e328f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 0x555ea533af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67e328f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555ea5340420 con 0x555ea532fb40 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67e328f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555ea524f060 con 0x555ea532e5d0 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67e328f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555ea53405a0 con 0x555ea533d780 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67e328f740 10 client.0 _ll_register_callbacks cb 0x555ea4fa28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67dad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 0x555ea533af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.633+0000 7f67da59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532e5d0 0x555ea533a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67dad9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 0x555ea533af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:35194/0 (socket says 172.21.15.5:35194) 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67da59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532e5d0 0x555ea533a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59948/0 (socket says 172.21.15.5:59948) 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67dad9d640 1 -- 172.21.15.5:0/3033884156 learned_addr learned my addr 172.21.15.5:0/3033884156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67dad9d640 1 -- 172.21.15.5:0/3033884156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532e5d0 msgr2=0x555ea533a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67dad9d640 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532e5d0 0x555ea533a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67dad9d640 1 -- 172.21.15.5:0/3033884156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 msgr2=0x555ea533aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67dad9d640 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 0x555ea533aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67dad9d640 1 -- 172.21.15.5:0/3033884156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea50053c0 con 0x555ea533d780 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67dad9d640 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 0x555ea533af80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f67d4020040 tx=0x7f67d4020750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4887063e0f6a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67c1ffb640 1 -- 172.21.15.5:0/3033884156 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d402d030 con 0x555ea533d780 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.634+0000 7f67c1ffb640 1 -- 172.21.15.5:0/3033884156 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f67d4014260 con 0x555ea533d780 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67e328f740 1 -- 172.21.15.5:0/3033884156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d4020920 con 0x555ea533d780 2024-09-11T22:38:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67c1ffb640 1 -- 172.21.15.5:0/3033884156 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f67d4020b50 con 0x555ea533d780 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67e328f740 10 client.29365 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67e328f740 1 -- 172.21.15.5:0/3033884156 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x555ea534bf70 con 0x555ea533d780 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67c1ffb640 1 -- 172.21.15.5:0/3033884156 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d4020920 con 0x555ea533d780 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67c1ffb640 1 -- 172.21.15.5:0/3033884156 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f67d401d070 con 0x555ea533d780 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67e328f740 10 client.29365 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67e328f740 10 client.29365 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67e328f740 20 client.29365 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67e328f740 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555ea534d380 0x555ea534f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67e328f740 4 client.29365 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.635+0000 7f67e328f740 1 -- 172.21.15.5:0/3033884156 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ea52a1860 con 0x555ea534d380 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.636+0000 7f67da59c640 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555ea534d380 0x555ea534f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.636+0000 7f67c1ffb640 10 client.29365 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.636+0000 7f67da59c640 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555ea534d380 0x555ea534f770 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac4c31bb4b7bcefa server_cookie=c13c2f8ba7bd7786 in_seq=0 out_seq=0 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.637+0000 7f67c1ffb640 1 -- 172.21.15.5:0/3033884156 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ea52a1860 con 0x555ea534d380 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.637+0000 7f67c1ffb640 10 client.29365 handle_command_reply: tid=0 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.637+0000 7f67e328f740 1 client.29365 shutdown 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.637+0000 7f67e328f740 1 -- 172.21.15.5:0/3033884156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 msgr2=0x555ea533af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.637+0000 7f67e328f740 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 0x555ea533af80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f67d4020040 tx=0x7f67d4020750 comp rx=0 tx=0).stop 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.638+0000 7f67e328f740 1 -- 172.21.15.5:0/3033884156 shutdown_connections 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.638+0000 7f67e328f740 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555ea534d380 0x555ea534f770 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.638+0000 7f67e328f740 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea533d780 0x555ea533af80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.638+0000 7f67c1ffb640 0 client.29365 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.638+0000 7f67e328f740 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 0x555ea533aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.638+0000 7f67e328f740 1 --2- 172.21.15.5:0/3033884156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532e5d0 0x555ea533a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.638+0000 7f67e328f740 1 -- 172.21.15.5:0/3033884156 >> 172.21.15.5:0/3033884156 conn(0x555ea52f7310 msgr2=0x555ea5330930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.638+0000 7f67e328f740 1 -- 172.21.15.5:0/3033884156 shutdown_connections 2024-09-11T22:38:30.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.638+0000 7f67e328f740 1 -- 172.21.15.5:0/3033884156 wait complete. 2024-09-11T22:38:30.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.638+0000 7f67e328f740 20 client.29365 trim_cache size 0 max 16384 2024-09-11T22:38:30.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46291 24.04 MB/sec execute 48 sec latency 61.912 ms 2024-09-11T22:38:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.653+0000 7f67e328f740 1 Processor -- start 2024-09-11T22:38:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.653+0000 7f67e328f740 1 -- start start 2024-09-11T22:38:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.653+0000 7f67e328f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea5329500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.653+0000 7f67e328f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 0x555ea5354dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.653+0000 7f67e328f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea533d780 0x555ea5355310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.653+0000 7f67e328f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1ab830 con 0x555ea533d780 2024-09-11T22:38:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.653+0000 7f67e328f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1ad370 con 0x555ea532fb40 2024-09-11T22:38:30.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.653+0000 7f67e328f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1abf00 con 0x555ea532e5d0 2024-09-11T22:38:30.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67da59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea5329500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67da59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea5329500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:35206/0 (socket says 172.21.15.5:35206) 2024-09-11T22:38:30.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67da59c640 1 -- 172.21.15.5:0/1852648367 learned_addr learned my addr 172.21.15.5:0/1852648367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:30.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67d9d9b640 1 --2- 172.21.15.5:0/1852648367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 0x555ea5354dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67dad9d640 1 --2- 172.21.15.5:0/1852648367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea533d780 0x555ea5355310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67da59c640 1 -- 172.21.15.5:0/1852648367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 msgr2=0x555ea5354dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:30.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67da59c640 1 --2- 172.21.15.5:0/1852648367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 0x555ea5354dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67da59c640 1 -- 172.21.15.5:0/1852648367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea533d780 msgr2=0x555ea5355310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:30.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67da59c640 1 --2- 172.21.15.5:0/1852648367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea533d780 0x555ea5355310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67da59c640 1 -- 172.21.15.5:0/1852648367 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea532a3b0 con 0x555ea532e5d0 2024-09-11T22:38:30.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.654+0000 7f67da59c640 1 --2- 172.21.15.5:0/1852648367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea5329500 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009900 tx=0x7f67cc005360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72176827f51250d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:30.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.655+0000 7f67c3fff640 1 -- 172.21.15.5:0/1852648367 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc035040 con 0x555ea532e5d0 2024-09-11T22:38:30.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.655+0000 7f67c3fff640 1 -- 172.21.15.5:0/1852648367 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67cc031d10 con 0x555ea532e5d0 2024-09-11T22:38:30.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.655+0000 7f67c3fff640 1 -- 172.21.15.5:0/1852648367 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc004e60 con 0x555ea532e5d0 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.655+0000 7f67e328f740 1 -- 172.21.15.5:0/1852648367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 msgr2=0x555ea5329500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.655+0000 7f67e328f740 1 --2- 172.21.15.5:0/1852648367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea5329500 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009900 tx=0x7f67cc005360 comp rx=0 tx=0).stop 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.656+0000 7f67e328f740 1 -- 172.21.15.5:0/1852648367 shutdown_connections 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.656+0000 7f67e328f740 1 --2- 172.21.15.5:0/1852648367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea533d780 0x555ea5355310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.656+0000 7f67e328f740 1 --2- 172.21.15.5:0/1852648367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea532fb40 0x555ea5354dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.656+0000 7f67e328f740 1 --2- 172.21.15.5:0/1852648367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea5329500 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.656+0000 7f67e328f740 1 -- 172.21.15.5:0/1852648367 >> 172.21.15.5:0/1852648367 conn(0x555ea52f7310 msgr2=0x555ea533c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.656+0000 7f67e328f740 1 -- 172.21.15.5:0/1852648367 shutdown_connections 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.656+0000 7f67e328f740 1 -- 172.21.15.5:0/1852648367 wait complete. 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.656+0000 7f67e328f740 1 -- start start 2024-09-11T22:38:30.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.656+0000 7f67e328f740 1 Processor -- start 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67e328f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea53460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67e328f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 0x555ea5341410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67e328f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea533d780 0x555ea5341ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67e328f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1acca0 con 0x555ea532fb40 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67e328f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1ac5d0 con 0x555ea533d780 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67e328f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc1a9830 con 0x555ea532e5d0 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67e328f740 10 client.0 _ll_register_callbacks cb 0x7f67cc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 0x555ea5341410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67da59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea53460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67d9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 0x555ea5341410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55166/0 (socket says 172.21.15.5:55166) 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67d9d9b640 1 -- 172.21.15.5:0/2987682449 learned_addr learned my addr 172.21.15.5:0/2987682449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67dad9d640 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea533d780 0x555ea5341ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67d9d9b640 1 -- 172.21.15.5:0/2987682449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 msgr2=0x555ea53460e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67d9d9b640 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea53460e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67d9d9b640 1 -- 172.21.15.5:0/2987682449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea533d780 msgr2=0x555ea5341ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67d9d9b640 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea533d780 0x555ea5341ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67d9d9b640 1 -- 172.21.15.5:0/2987682449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea53b6da0 con 0x555ea532fb40 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67da59c640 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea53460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67d9d9b640 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 0x555ea5341410 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009380 tx=0x7f67d0097a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6d668f331b72cb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.657+0000 7f67c1ffb640 1 -- 172.21.15.5:0/2987682449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d009b650 con 0x555ea532fb40 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67c1ffb640 1 -- 172.21.15.5:0/2987682449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f67d00a44d0 con 0x555ea532fb40 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67e328f740 1 -- 172.21.15.5:0/2987682449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67cc031d10 con 0x555ea532fb40 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67c1ffb640 1 -- 172.21.15.5:0/2987682449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f67d009b470 con 0x555ea532fb40 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67e328f740 10 client.20460 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:30.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67e328f740 1 -- 172.21.15.5:0/2987682449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x555ea53b7ed0 con 0x555ea532fb40 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67c1ffb640 1 -- 172.21.15.5:0/2987682449 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67cc031d10 con 0x555ea532fb40 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67c1ffb640 1 -- 172.21.15.5:0/2987682449 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f67d009abb0 con 0x555ea532fb40 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67e328f740 10 client.20460 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67e328f740 10 client.20460 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67e328f740 20 client.20460 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67e328f740 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555ea534d380 0x555ea53ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67e328f740 4 client.20460 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67e328f740 1 -- 172.21.15.5:0/2987682449 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ea529bd70 con 0x555ea534d380 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.658+0000 7f67da59c640 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555ea534d380 0x555ea53ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67c1ffb640 10 client.20460 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67da59c640 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555ea534d380 0x555ea53ba690 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80f7cd141594d39d server_cookie=fe47541f00ae04ec in_seq=0 out_seq=0 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67c1ffb640 1 -- 172.21.15.5:0/2987682449 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ea529bd70 con 0x555ea534d380 2024-09-11T22:38:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67c1ffb640 10 client.20460 handle_command_reply: tid=0 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67e328f740 1 client.20460 shutdown 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67e328f740 1 -- 172.21.15.5:0/2987682449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 msgr2=0x555ea5341410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67e328f740 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 0x555ea5341410 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009380 tx=0x7f67d0097a40 comp rx=0 tx=0).stop 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67e328f740 1 -- 172.21.15.5:0/2987682449 shutdown_connections 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67e328f740 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x555ea534d380 0x555ea53ba690 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67e328f740 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555ea533d780 0x555ea5341ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67e328f740 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ea532fb40 0x555ea5341410 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67e328f740 1 --2- 172.21.15.5:0/2987682449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555ea532e5d0 0x555ea53460e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67e328f740 1 -- 172.21.15.5:0/2987682449 >> 172.21.15.5:0/2987682449 conn(0x555ea52f7310 msgr2=0x555ea5338ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.659+0000 7f67c1ffb640 0 client.20460 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.660+0000 7f67e328f740 1 -- 172.21.15.5:0/2987682449 shutdown_connections 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.660+0000 7f67e328f740 1 -- 172.21.15.5:0/2987682449 wait complete. 2024-09-11T22:38:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.660+0000 7f67e328f740 20 client.20460 trim_cache size 0 max 16384 2024-09-11T22:38:30.666 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:38:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 -- 172.21.15.5:0/2971939356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67c40a2cf0 msgr2=0x7f67c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67c40a2cf0 0x7f67c40a51b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f67d4011cb0 tx=0x7f67d4009000 comp rx=0 tx=0).stop 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 -- 172.21.15.5:0/2971939356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67dc102790 msgr2=0x7f67dc198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67dc102790 0x7f67dc198990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e9f0 tx=0x7f67d000edf0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 -- 172.21.15.5:0/2971939356 shutdown_connections 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67c40a2cf0 0x7f67c40a51b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f67dc106f10 0x7f67dc19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67dc102790 0x7f67dc198990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 --2- 172.21.15.5:0/2971939356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67dc101de0 0x7f67dc198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 -- 172.21.15.5:0/2971939356 >> 172.21.15.5:0/2971939356 conn(0x7f67dc0fd8e0 msgr2=0x7f67dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 -- 172.21.15.5:0/2971939356 shutdown_connections 2024-09-11T22:38:30.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:30.661+0000 7f67e0eb6640 1 -- 172.21.15.5:0/2971939356 wait complete. 2024-09-11T22:38:31.522 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:38:31.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46846 24.15 MB/sec execute 49 sec latency 143.348 ms 2024-09-11T22:38:31.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:31 smithi159 ceph-mon[35534]: pgmap v520: 97 pgs: 97 active+clean; 479 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.0 MiB/s wr, 407 op/s 2024-09-11T22:38:31.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:31 smithi172 ceph-mon[35461]: pgmap v520: 97 pgs: 97 active+clean; 479 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.0 MiB/s wr, 407 op/s 2024-09-11T22:38:31.875 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:32.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:31 smithi005 ceph-mon[30943]: pgmap v520: 97 pgs: 97 active+clean; 479 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.0 MiB/s wr, 407 op/s 2024-09-11T22:38:32.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.425+0000 7f570e231640 1 -- 172.21.15.5:0/3363138200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5708100e20 msgr2=0x7f5708101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:32.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.425+0000 7f570e231640 1 --2- 172.21.15.5:0/3363138200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5708100e20 0x7f5708101200 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009900 tx=0x7f56fc02a480 comp rx=0 tx=0).stop 2024-09-11T22:38:32.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.425+0000 7f570e231640 1 -- 172.21.15.5:0/3363138200 shutdown_connections 2024-09-11T22:38:32.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.425+0000 7f570e231640 1 --2- 172.21.15.5:0/3363138200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5708102ef0 0x7f570810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.425+0000 7f570e231640 1 --2- 172.21.15.5:0/3363138200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5708102570 0x7f57081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.425+0000 7f570e231640 1 --2- 172.21.15.5:0/3363138200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5708100e20 0x7f5708101200 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.425+0000 7f570e231640 1 -- 172.21.15.5:0/3363138200 >> 172.21.15.5:0/3363138200 conn(0x7f57080fc7d0 msgr2=0x7f57080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:32.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.425+0000 7f570e231640 1 -- 172.21.15.5:0/3363138200 shutdown_connections 2024-09-11T22:38:32.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.425+0000 7f570e231640 1 -- 172.21.15.5:0/3363138200 wait complete. 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f570e231640 1 Processor -- start 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f570e231640 1 -- start start 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f570e231640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5708100e20 0x7f57081973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f570e231640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5708102570 0x7f5708197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f570e231640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5708102ef0 0x7f570819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f570e231640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5708110f80 con 0x7f5708100e20 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f570e231640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5708110e00 con 0x7f5708102ef0 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f570e231640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5708111100 con 0x7f5708102570 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f57077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5708100e20 0x7f57081973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f5706ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5708102570 0x7f5708197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f57077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5708100e20 0x7f57081973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46120/0 (socket says 172.21.15.5:46120) 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f57077fe640 1 -- 172.21.15.5:0/4097800048 learned_addr learned my addr 172.21.15.5:0/4097800048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f5706ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5708102570 0x7f5708197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59902/0 (socket says 172.21.15.5:59902) 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.426+0000 7f5707fff640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5708102ef0 0x7f570819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f57077fe640 1 -- 172.21.15.5:0/4097800048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5708102570 msgr2=0x7f5708197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f57077fe640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5708102570 0x7f5708197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f57077fe640 1 -- 172.21.15.5:0/4097800048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5708102ef0 msgr2=0x7f570819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f57077fe640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5708102ef0 0x7f570819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f57077fe640 1 -- 172.21.15.5:0/4097800048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570819c370 con 0x7f5708100e20 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f5706ffd640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5708102570 0x7f5708197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f57077fe640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5708100e20 0x7f57081973c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f56fc006010 tx=0x7f56fc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f5704ff9640 1 -- 172.21.15.5:0/4097800048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc03f070 con 0x7f5708100e20 2024-09-11T22:38:32.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f570e231640 1 -- 172.21.15.5:0/4097800048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570819c600 con 0x7f5708100e20 2024-09-11T22:38:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f5704ff9640 1 -- 172.21.15.5:0/4097800048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56fc031c30 con 0x7f5708100e20 2024-09-11T22:38:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f5704ff9640 1 -- 172.21.15.5:0/4097800048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc035030 con 0x7f5708100e20 2024-09-11T22:38:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.427+0000 7f570e231640 1 -- 172.21.15.5:0/4097800048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5708069f70 con 0x7f5708100e20 2024-09-11T22:38:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.429+0000 7f571060a740 1 Processor -- start 2024-09-11T22:38:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.429+0000 7f5704ff9640 1 -- 172.21.15.5:0/4097800048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f56fc042070 con 0x7f5708100e20 2024-09-11T22:38:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f571060a740 1 -- start start 2024-09-11T22:38:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f571060a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cc2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f571060a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cc2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f571060a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 0x55b1f5cd3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f571060a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b1f5c282f0 con 0x55b1f5cc25d0 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f571060a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b1f5bcfcf0 con 0x55b1f5cd1780 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f571060a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b1f5c242b0 con 0x55b1f5cc3b40 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f57077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cc2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f57077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cc2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46128/0 (socket says 172.21.15.5:46128) 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f5707fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cc2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f57077fe640 1 -- 172.21.15.5:0/1070410767 learned_addr learned my addr 172.21.15.5:0/1070410767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f5706ffd640 1 --2- 172.21.15.5:0/1070410767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 0x55b1f5cd3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f5707fff640 1 -- 172.21.15.5:0/1070410767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 msgr2=0x55b1f5cd3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f5707fff640 1 --2- 172.21.15.5:0/1070410767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 0x55b1f5cd3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f5707fff640 1 -- 172.21.15.5:0/1070410767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 msgr2=0x55b1f5cc2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:32.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f5704ff9640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56f00a2d70 0x7f56f00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f5707fff640 1 --2- 172.21.15.5:0/1070410767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cc2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.430+0000 7f5707fff640 1 -- 172.21.15.5:0/1070410767 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f5bb43a0 con 0x55b1f5cc3b40 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.431+0000 7f5704ff9640 1 -- 172.21.15.5:0/4097800048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f56fc0efaa0 con 0x7f5708100e20 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.431+0000 7f5707fff640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56f00a2d70 0x7f56f00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.431+0000 7f5707fff640 1 --2- 172.21.15.5:0/1070410767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cc2090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5700009f00 tx=0x7f5700011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2f40dc47d9a3ca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.431+0000 7f56edffb640 1 -- 172.21.15.5:0/1070410767 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57000045c0 con 0x55b1f5cc3b40 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.431+0000 7f56edffb640 1 -- 172.21.15.5:0/1070410767 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5700004760 con 0x55b1f5cc3b40 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.431+0000 7f56edffb640 1 -- 172.21.15.5:0/1070410767 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570001c660 con 0x55b1f5cc3b40 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 -- 172.21.15.5:0/1070410767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 msgr2=0x55b1f5cc2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f5707fff640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56f00a2d70 0x7f56f00a5230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5700020000 tx=0x7f5700021000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:32.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 --2- 172.21.15.5:0/1070410767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cc2090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5700009f00 tx=0x7f5700011810 comp rx=0 tx=0).stop 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 -- 172.21.15.5:0/1070410767 shutdown_connections 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 --2- 172.21.15.5:0/1070410767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 0x55b1f5cd3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 --2- 172.21.15.5:0/1070410767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cc2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 --2- 172.21.15.5:0/1070410767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cc2090 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 -- 172.21.15.5:0/1070410767 >> 172.21.15.5:0/1070410767 conn(0x55b1f5c8b310 msgr2=0x55b1f5cbec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 -- 172.21.15.5:0/1070410767 shutdown_connections 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 -- 172.21.15.5:0/1070410767 wait complete. 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 -- start start 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.432+0000 7f571060a740 1 Processor -- start 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f571060a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f571060a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f571060a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 0x55b1f5ccea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f571060a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b1f5cd4480 con 0x55b1f5cc25d0 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f571060a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b1f5be3060 con 0x55b1f5cd1780 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f571060a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b1f5cd4600 con 0x55b1f5cc3b40 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f571060a740 10 client.0 _ll_register_callbacks cb 0x55b1f59368c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f57077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f57077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46138/0 (socket says 172.21.15.5:46138) 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f57077fe640 1 -- 172.21.15.5:0/3805533931 learned_addr learned my addr 172.21.15.5:0/3805533931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f5706ffd640 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f5707fff640 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 0x55b1f5ccea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f57077fe640 1 -- 172.21.15.5:0/3805533931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 msgr2=0x55b1f5cce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:32.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f57077fe640 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f57077fe640 1 -- 172.21.15.5:0/3805533931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 msgr2=0x55b1f5ccea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f57077fe640 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 0x55b1f5ccea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f57077fe640 1 -- 172.21.15.5:0/3805533931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f59993c0 con 0x55b1f5cc25d0 2024-09-11T22:38:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f5707fff640 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 0x55b1f5ccea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:38:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f57077fe640 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd1130 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0c4690 tx=0x7f56fc0f31e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd71a429e9e3e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.433+0000 7f56e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:38:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f56e6ffd640 1 -- 172.21.15.5:0/3805533931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56fc0f5060 con 0x55b1f5cc25d0 2024-09-11T22:38:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f56e6ffd640 1 -- 172.21.15.5:0/3805533931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f56fc0b8670 con 0x55b1f5cc25d0 2024-09-11T22:38:32.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f571060a740 1 -- 172.21.15.5:0/3805533931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5700004760 con 0x55b1f5cc25d0 2024-09-11T22:38:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f56e6ffd640 1 -- 172.21.15.5:0/3805533931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f56fc049070 con 0x55b1f5cc25d0 2024-09-11T22:38:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f571060a740 10 client.20472 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f571060a740 1 -- 172.21.15.5:0/3805533931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b1f5cdff10 con 0x55b1f5cc25d0 2024-09-11T22:38:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f56e6ffd640 1 -- 172.21.15.5:0/3805533931 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5700004760 con 0x55b1f5cc25d0 2024-09-11T22:38:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f56e6ffd640 1 -- 172.21.15.5:0/3805533931 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f56fc0c4730 con 0x55b1f5cc25d0 2024-09-11T22:38:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f571060a740 10 client.20472 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f571060a740 10 client.20472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f571060a740 20 client.20472 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f571060a740 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b1f5ce1370 0x55b1f5ce3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f571060a740 4 client.20472 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f571060a740 1 -- 172.21.15.5:0/3805533931 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1f5c35860 con 0x55b1f5ce1370 2024-09-11T22:38:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.434+0000 7f5706ffd640 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b1f5ce1370 0x55b1f5ce3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.435+0000 7f5706ffd640 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b1f5ce1370 0x55b1f5ce3800 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38bdf5c01bab0700 server_cookie=63e63cf2b1bf6450 in_seq=0 out_seq=0 2024-09-11T22:38:32.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.435+0000 7f56e6ffd640 10 client.20472 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:32.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f56e6ffd640 1 -- 172.21.15.5:0/3805533931 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1f5c35860 con 0x55b1f5ce1370 2024-09-11T22:38:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f56e6ffd640 10 client.20472 handle_command_reply: tid=0 2024-09-11T22:38:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f571060a740 1 client.20472 shutdown 2024-09-11T22:38:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f571060a740 1 -- 172.21.15.5:0/3805533931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 msgr2=0x55b1f5cd1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f571060a740 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd1130 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0c4690 tx=0x7f56fc0f31e0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f571060a740 1 -- 172.21.15.5:0/3805533931 shutdown_connections 2024-09-11T22:38:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f571060a740 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b1f5ce1370 0x55b1f5ce3800 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f56e6ffd640 0 client.20472 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f571060a740 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cd1780 0x55b1f5ccea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f571060a740 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f571060a740 1 --2- 172.21.15.5:0/3805533931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd1130 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.437+0000 7f571060a740 1 -- 172.21.15.5:0/3805533931 >> 172.21.15.5:0/3805533931 conn(0x55b1f5c8b310 msgr2=0x55b1f5cc4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.438+0000 7f571060a740 1 -- 172.21.15.5:0/3805533931 shutdown_connections 2024-09-11T22:38:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.438+0000 7f571060a740 1 -- 172.21.15.5:0/3805533931 wait complete. 2024-09-11T22:38:32.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.438+0000 7f571060a740 20 client.20472 trim_cache size 0 max 16384 2024-09-11T22:38:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.451+0000 7f571060a740 1 Processor -- start 2024-09-11T22:38:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.451+0000 7f571060a740 1 -- start start 2024-09-11T22:38:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f571060a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cbee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f571060a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cdd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f571060a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cd1780 0x55b1f5ce90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f571060a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57081ab800 con 0x55b1f5cd1780 2024-09-11T22:38:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f571060a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f57081ad340 con 0x55b1f5cc25d0 2024-09-11T22:38:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f571060a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f57081abed0 con 0x55b1f5cc3b40 2024-09-11T22:38:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f5707fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cd1780 0x55b1f5ce90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f57077fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cbee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f57077fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cbee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43452/0 (socket says 172.21.15.5:43452) 2024-09-11T22:38:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f5707fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cd1780 0x55b1f5ce90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46154/0 (socket says 172.21.15.5:46154) 2024-09-11T22:38:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f57077fe640 1 -- 172.21.15.5:0/3797071267 learned_addr learned my addr 172.21.15.5:0/3797071267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f5706ffd640 1 --2- 172.21.15.5:0/3797071267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cdd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f5707fff640 1 -- 172.21.15.5:0/3797071267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 msgr2=0x55b1f5cdd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f5707fff640 1 --2- 172.21.15.5:0/3797071267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cdd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f5707fff640 1 -- 172.21.15.5:0/3797071267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc25d0 msgr2=0x55b1f5cbee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f5707fff640 1 --2- 172.21.15.5:0/3797071267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cbee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f5707fff640 1 -- 172.21.15.5:0/3797071267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1f5ce3de0 con 0x55b1f5cd1780 2024-09-11T22:38:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f57077fe640 1 --2- 172.21.15.5:0/3797071267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cbee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f5706ffd640 1 --2- 172.21.15.5:0/3797071267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cdd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:32.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.452+0000 7f5707fff640 1 --2- 172.21.15.5:0/3797071267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cd1780 0x55b1f5ce90a0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5700004950 tx=0x7f5700027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c89e989f293df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.453+0000 7f56ed7fa640 1 -- 172.21.15.5:0/3797071267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57000045c0 con 0x55b1f5cd1780 2024-09-11T22:38:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.453+0000 7f56ed7fa640 1 -- 172.21.15.5:0/3797071267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5700004760 con 0x55b1f5cd1780 2024-09-11T22:38:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.453+0000 7f56ed7fa640 1 -- 172.21.15.5:0/3797071267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570001cb20 con 0x55b1f5cd1780 2024-09-11T22:38:32.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.454+0000 7f571060a740 1 -- 172.21.15.5:0/3797071267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cd1780 msgr2=0x55b1f5ce90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:32.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.454+0000 7f571060a740 1 --2- 172.21.15.5:0/3797071267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cd1780 0x55b1f5ce90a0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f5700004950 tx=0x7f5700027000 comp rx=0 tx=0).stop 2024-09-11T22:38:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.454+0000 7f571060a740 1 -- 172.21.15.5:0/3797071267 shutdown_connections 2024-09-11T22:38:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.454+0000 7f571060a740 1 --2- 172.21.15.5:0/3797071267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cd1780 0x55b1f5ce90a0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.454+0000 7f571060a740 1 --2- 172.21.15.5:0/3797071267 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cdd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.454+0000 7f571060a740 1 --2- 172.21.15.5:0/3797071267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cbee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.454+0000 7f571060a740 1 -- 172.21.15.5:0/3797071267 >> 172.21.15.5:0/3797071267 conn(0x55b1f5c8b310 msgr2=0x55b1f5cbe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.454+0000 7f571060a740 1 -- 172.21.15.5:0/3797071267 shutdown_connections 2024-09-11T22:38:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.454+0000 7f571060a740 1 -- 172.21.15.5:0/3797071267 wait complete. 2024-09-11T22:38:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.454+0000 7f571060a740 1 -- start start 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f571060a740 1 Processor -- start 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f571060a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f571060a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cd5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f571060a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cd1780 0x55b1f5cd6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f571060a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57081acc70 con 0x55b1f5cc25d0 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f571060a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f57081ac5a0 con 0x55b1f5cc3b40 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f571060a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f57081a9800 con 0x55b1f5cd1780 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f571060a740 10 client.0 _ll_register_callbacks cb 0x7f56fc005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f57077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f57077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46164/0 (socket says 172.21.15.5:46164) 2024-09-11T22:38:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f57077fe640 1 -- 172.21.15.5:0/1166051014 learned_addr learned my addr 172.21.15.5:0/1166051014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f5707fff640 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cd1780 0x55b1f5cd6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.455+0000 7f5706ffd640 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cd5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f5707fff640 1 -- 172.21.15.5:0/1166051014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc3b40 msgr2=0x55b1f5cd5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f5707fff640 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cd5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f5707fff640 1 -- 172.21.15.5:0/1166051014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 msgr2=0x55b1f5cd56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f5707fff640 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f5707fff640 1 -- 172.21.15.5:0/1166051014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1f5d4b120 con 0x55b1f5cd1780 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f5707fff640 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cd1780 0x55b1f5cd6120 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5700012fd0 tx=0x7f570001dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b76e6c548772a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f56e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f56e6ffd640 1 -- 172.21.15.5:0/1166051014 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570003e070 con 0x55b1f5cd1780 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f56e6ffd640 1 -- 172.21.15.5:0/1166051014 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f5700022040 con 0x55b1f5cd1780 2024-09-11T22:38:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f571060a740 1 -- 172.21.15.5:0/1166051014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5700004760 con 0x55b1f5cd1780 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.456+0000 7f56e6ffd640 1 -- 172.21.15.5:0/1166051014 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f57000045c0 con 0x55b1f5cd1780 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.457+0000 7f571060a740 10 client.29389 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.457+0000 7f571060a740 1 -- 172.21.15.5:0/1166051014 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b1f5d4c400 con 0x55b1f5cd1780 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.457+0000 7f56e6ffd640 1 -- 172.21.15.5:0/1166051014 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f570004a070 con 0x55b1f5cd1780 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.457+0000 7f56e6ffd640 1 -- 172.21.15.5:0/1166051014 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f570000f980 con 0x55b1f5cd1780 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.457+0000 7f571060a740 10 client.29389 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.457+0000 7f571060a740 10 client.29389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.457+0000 7f571060a740 20 client.29389 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.457+0000 7f571060a740 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b1f5ce1370 0x55b1f5d4e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.457+0000 7f571060a740 4 client.29389 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:38:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.457+0000 7f571060a740 1 -- 172.21.15.5:0/1166051014 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1f5c2fd70 con 0x55b1f5ce1370 2024-09-11T22:38:32.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.458+0000 7f57077fe640 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b1f5ce1370 0x55b1f5d4e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:32.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.458+0000 7f56e6ffd640 10 client.29389 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:32.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.458+0000 7f57077fe640 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b1f5ce1370 0x55b1f5d4e930 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51a2e6bbf567cc5 server_cookie=1dbd4f767862fbe9 in_seq=0 out_seq=0 2024-09-11T22:38:32.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.462+0000 7f56e6ffd640 1 -- 172.21.15.5:0/1166051014 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1f5c2fd70 con 0x55b1f5ce1370 2024-09-11T22:38:32.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.462+0000 7f56e6ffd640 10 client.29389 handle_command_reply: tid=0 2024-09-11T22:38:32.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.462+0000 7f571060a740 1 client.29389 shutdown 2024-09-11T22:38:32.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.462+0000 7f571060a740 1 -- 172.21.15.5:0/1166051014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cd1780 msgr2=0x55b1f5cd6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:32.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.462+0000 7f571060a740 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cd1780 0x55b1f5cd6120 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5700012fd0 tx=0x7f570001dd40 comp rx=0 tx=0).stop 2024-09-11T22:38:32.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.462+0000 7f571060a740 1 -- 172.21.15.5:0/1166051014 shutdown_connections 2024-09-11T22:38:32.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.462+0000 7f571060a740 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55b1f5ce1370 0x55b1f5d4e930 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.462+0000 7f571060a740 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b1f5cd1780 0x55b1f5cd6120 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f571060a740 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b1f5cc3b40 0x55b1f5cd5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f56e6ffd640 0 client.29389 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:32.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f571060a740 1 --2- 172.21.15.5:0/1166051014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1f5cc25d0 0x55b1f5cd56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f571060a740 1 -- 172.21.15.5:0/1166051014 >> 172.21.15.5:0/1166051014 conn(0x55b1f5c8b310 msgr2=0x55b1f5ccfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:32.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f571060a740 1 -- 172.21.15.5:0/1166051014 shutdown_connections 2024-09-11T22:38:32.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f571060a740 1 -- 172.21.15.5:0/1166051014 wait complete. 2024-09-11T22:38:32.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f571060a740 20 client.29389 trim_cache size 0 max 16384 2024-09-11T22:38:32.468 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:38:32.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f570e231640 1 -- 172.21.15.5:0/4097800048 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56f00a2d70 msgr2=0x7f56f00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f570e231640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56f00a2d70 0x7f56f00a5230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5700020000 tx=0x7f5700021000 comp rx=0 tx=0).stop 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f570e231640 1 -- 172.21.15.5:0/4097800048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5708100e20 msgr2=0x7f57081973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.463+0000 7f570e231640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5708100e20 0x7f57081973c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f56fc006010 tx=0x7f56fc003c10 comp rx=0 tx=0).stop 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.464+0000 7f570e231640 1 -- 172.21.15.5:0/4097800048 shutdown_connections 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.464+0000 7f570e231640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f56f00a2d70 0x7f56f00a5230 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.464+0000 7f570e231640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5708102ef0 0x7f570819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.464+0000 7f570e231640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5708102570 0x7f5708197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.464+0000 7f570e231640 1 --2- 172.21.15.5:0/4097800048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5708100e20 0x7f57081973c0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.464+0000 7f570e231640 1 -- 172.21.15.5:0/4097800048 >> 172.21.15.5:0/4097800048 conn(0x7f57080fc7d0 msgr2=0x7f57080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.464+0000 7f570e231640 1 -- 172.21.15.5:0/4097800048 shutdown_connections 2024-09-11T22:38:32.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:32.464+0000 7f570e231640 1 -- 172.21.15.5:0/4097800048 wait complete. 2024-09-11T22:38:32.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47414 24.29 MB/sec execute 50 sec latency 172.988 ms 2024-09-11T22:38:33.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47891 24.23 MB/sec execute 51 sec latency 188.636 ms 2024-09-11T22:38:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:33 smithi005 ceph-mon[30943]: pgmap v521: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 775 op/s 2024-09-11T22:38:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:33 smithi172 ceph-mon[35461]: pgmap v521: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 775 op/s 2024-09-11T22:38:34.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:33 smithi159 ceph-mon[35534]: pgmap v521: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 775 op/s 2024-09-11T22:38:34.422 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:38:34.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:38:34.641 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48513 24.62 MB/sec execute 52 sec latency 177.474 ms 2024-09-11T22:38:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:35.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48983 24.62 MB/sec execute 53 sec latency 178.181 ms 2024-09-11T22:38:35.832 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:35 smithi005 ceph-mon[30943]: pgmap v522: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-11T22:38:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.831+0000 7f1b47308640 1 -- 172.21.15.5:0/2297320970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40102d40 msgr2=0x7f1b40103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.831+0000 7f1b47308640 1 --2- 172.21.15.5:0/2297320970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40102d40 0x7f1b40103120 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009900 tx=0x7f1b3c02a480 comp rx=0 tx=0).stop 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.831+0000 7f1b47308640 1 -- 172.21.15.5:0/2297320970 shutdown_connections 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.831+0000 7f1b47308640 1 --2- 172.21.15.5:0/2297320970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4010c170 0x7f1b4010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.831+0000 7f1b47308640 1 --2- 172.21.15.5:0/2297320970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b40103660 0x7f1b4010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.831+0000 7f1b47308640 1 --2- 172.21.15.5:0/2297320970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40102d40 0x7f1b40103120 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.831+0000 7f1b47308640 1 -- 172.21.15.5:0/2297320970 >> 172.21.15.5:0/2297320970 conn(0x7f1b400fc7a0 msgr2=0x7f1b400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.831+0000 7f1b47308640 1 -- 172.21.15.5:0/2297320970 shutdown_connections 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.831+0000 7f1b47308640 1 -- 172.21.15.5:0/2297320970 wait complete. 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b47308640 1 Processor -- start 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b47308640 1 -- start start 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b47308640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b40102d40 0x7f1b40198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b47308640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40103660 0x7f1b40198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b47308640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4010c170 0x7f1b4019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b47308640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40111f70 con 0x7f1b40103660 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b47308640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40111df0 con 0x7f1b40102d40 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b47308640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401120f0 con 0x7f1b4010c170 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b448a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40103660 0x7f1b40198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b450a4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b40102d40 0x7f1b40198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b448a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40103660 0x7f1b40198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46182/0 (socket says 172.21.15.5:46182) 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b450a4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b40102d40 0x7f1b40198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43488/0 (socket says 172.21.15.5:43488) 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b448a3640 1 -- 172.21.15.5:0/2981086197 learned_addr learned my addr 172.21.15.5:0/2981086197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.832+0000 7f1b458a5640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4010c170 0x7f1b4019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b448a3640 1 -- 172.21.15.5:0/2981086197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4010c170 msgr2=0x7f1b4019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b448a3640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4010c170 0x7f1b4019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b448a3640 1 -- 172.21.15.5:0/2981086197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b40102d40 msgr2=0x7f1b40198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b448a3640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b40102d40 0x7f1b40198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b448a3640 1 -- 172.21.15.5:0/2981086197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b4019d400 con 0x7f1b40103660 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b458a5640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4010c170 0x7f1b4019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b450a4640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b40102d40 0x7f1b40198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b448a3640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40103660 0x7f1b40198990 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000e9f0 tx=0x7f1b3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b367fc640 1 -- 172.21.15.5:0/2981086197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000cd80 con 0x7f1b40103660 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b47308640 1 -- 172.21.15.5:0/2981086197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4019d6f0 con 0x7f1b40103660 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b367fc640 1 -- 172.21.15.5:0/2981086197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b300035f0 con 0x7f1b40103660 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b367fc640 1 -- 172.21.15.5:0/2981086197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30010670 con 0x7f1b40103660 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.833+0000 7f1b47308640 1 -- 172.21.15.5:0/2981086197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b401a4fd0 con 0x7f1b40103660 2024-09-11T22:38:35.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.836+0000 7f1b367fc640 1 -- 172.21.15.5:0/2981086197 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1b300037a0 con 0x7f1b40103660 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.836+0000 7f1b496e1740 1 Processor -- start 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b496e1740 1 -- start start 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedebf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedec480 0x55c5dedf6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedf6d40 0x55c5dedf9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b496e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c5ded51280 con 0x55c5dedec480 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b496e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c5decf8b10 con 0x55c5dedebb60 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b496e1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c5ded4d240 con 0x55c5dedf6d40 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b450a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedec480 0x55c5dedf6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b458a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b450a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedec480 0x55c5dedf6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46190/0 (socket says 172.21.15.5:46190) 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b458a5640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedebf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43504/0 (socket says 172.21.15.5:43504) 2024-09-11T22:38:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b450a4640 1 -- 172.21.15.5:0/2630024217 learned_addr learned my addr 172.21.15.5:0/2630024217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b448a3640 1 --2- 172.21.15.5:0/2630024217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedf6d40 0x55c5dedf9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b458a5640 1 -- 172.21.15.5:0/2630024217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedf6d40 msgr2=0x55c5dedf9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b458a5640 1 --2- 172.21.15.5:0/2630024217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedf6d40 0x55c5dedf9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b458a5640 1 -- 172.21.15.5:0/2630024217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedec480 msgr2=0x55c5dedf6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b458a5640 1 --2- 172.21.15.5:0/2630024217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedec480 0x55c5dedf6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b458a5640 1 -- 172.21.15.5:0/2630024217 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ded051f0 con 0x55c5dedebb60 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b450a4640 1 --2- 172.21.15.5:0/2630024217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedec480 0x55c5dedf6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.837+0000 7f1b367fc640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b2c0a2cf0 0x7f1b2c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.838+0000 7f1b458a5640 1 --2- 172.21.15.5:0/2630024217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedebf40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1b38005fd0 tx=0x7f1b38011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7bbff252fea68f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.838+0000 7f1b450a4640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b2c0a2cf0 0x7f1b2c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.838+0000 7f1b367fc640 1 -- 172.21.15.5:0/2981086197 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f1b300cf750 con 0x7f1b40103660 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.838+0000 7f1b2b7fe640 1 -- 172.21.15.5:0/2630024217 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38004d20 con 0x55c5dedebb60 2024-09-11T22:38:35.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.838+0000 7f1b2b7fe640 1 -- 172.21.15.5:0/2630024217 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3800f9a0 con 0x55c5dedebb60 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.838+0000 7f1b2b7fe640 1 -- 172.21.15.5:0/2630024217 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38004550 con 0x55c5dedebb60 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 -- 172.21.15.5:0/2630024217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 msgr2=0x55c5dedebf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b450a4640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b2c0a2cf0 0x7f1b2c0a51b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c000c00 tx=0x7f1b3c0058d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 --2- 172.21.15.5:0/2630024217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedebf40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1b38005fd0 tx=0x7f1b38011bf0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 -- 172.21.15.5:0/2630024217 shutdown_connections 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 --2- 172.21.15.5:0/2630024217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedf6d40 0x55c5dedf9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 --2- 172.21.15.5:0/2630024217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedec480 0x55c5dedf6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 --2- 172.21.15.5:0/2630024217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedebf40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 -- 172.21.15.5:0/2630024217 >> 172.21.15.5:0/2630024217 conn(0x55c5dede80f0 msgr2=0x55c5dede84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 -- 172.21.15.5:0/2630024217 shutdown_connections 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 -- 172.21.15.5:0/2630024217 wait complete. 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 -- start start 2024-09-11T22:38:35.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.839+0000 7f1b496e1740 1 Processor -- start 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedf3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 0x55c5dedf4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedf6d40 0x55c5dedf4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b496e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c5dedf9910 con 0x55c5dedf6d40 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b496e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c5ded0bf10 con 0x55c5dedebb60 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b496e1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c5dedf9a90 con 0x55c5dedec480 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b496e1740 10 client.0 _ll_register_callbacks cb 0x55c5decd4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b450a4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedf3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b450a4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedf3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43512/0 (socket says 172.21.15.5:43512) 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b450a4640 1 -- 172.21.15.5:0/1240789707 learned_addr learned my addr 172.21.15.5:0/1240789707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b458a5640 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedf6d40 0x55c5dedf4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b448a3640 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 0x55c5dedf4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b450a4640 1 -- 172.21.15.5:0/1240789707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 msgr2=0x55c5dedf4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b450a4640 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 0x55c5dedf4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b450a4640 1 -- 172.21.15.5:0/1240789707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedf6d40 msgr2=0x55c5dedf4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b450a4640 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedf6d40 0x55c5dedf4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b450a4640 1 -- 172.21.15.5:0/1240789707 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5deac26a0 con 0x55c5dedebb60 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b448a3640 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 0x55c5dedf4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b450a4640 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedf3bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0057c0 tx=0x7f1b3c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc7361ce04d261 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.840+0000 7f1b28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.841+0000 7f1b28ff9640 1 -- 172.21.15.5:0/1240789707 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c043070 con 0x55c5dedebb60 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.841+0000 7f1b28ff9640 1 -- 172.21.15.5:0/1240789707 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f1b3c005500 con 0x55c5dedebb60 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.841+0000 7f1b496e1740 1 -- 172.21.15.5:0/1240789707 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b3800f9a0 con 0x55c5dedebb60 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.841+0000 7f1b28ff9640 1 -- 172.21.15.5:0/1240789707 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1b3c035040 con 0x55c5dedebb60 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.841+0000 7f1b496e1740 10 client.29567 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.841+0000 7f1b496e1740 1 -- 172.21.15.5:0/1240789707 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c5dee054c0 con 0x55c5dedebb60 2024-09-11T22:38:35.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.841+0000 7f1b28ff9640 1 -- 172.21.15.5:0/1240789707 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3800f9a0 con 0x55c5dedebb60 2024-09-11T22:38:35.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.841+0000 7f1b28ff9640 1 -- 172.21.15.5:0/1240789707 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f1b3c003a90 con 0x55c5dedebb60 2024-09-11T22:38:35.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.842+0000 7f1b496e1740 10 client.29567 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:35.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.842+0000 7f1b496e1740 10 client.29567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:35.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.842+0000 7f1b496e1740 20 client.29567 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:35.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.842+0000 7f1b496e1740 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c5dee06980 0x55c5dee08db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.842+0000 7f1b496e1740 4 client.29567 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:35.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.842+0000 7f1b496e1740 1 -- 172.21.15.5:0/1240789707 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5ded5e7f0 con 0x55c5dee06980 2024-09-11T22:38:35.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.842+0000 7f1b448a3640 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c5dee06980 0x55c5dee08db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.842+0000 7f1b28ff9640 10 client.29567 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:35.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.842+0000 7f1b448a3640 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c5dee06980 0x55c5dee08db0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b88c36371bd6f8 server_cookie=2b65f847b9870afb in_seq=0 out_seq=0 2024-09-11T22:38:35.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b28ff9640 1 -- 172.21.15.5:0/1240789707 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5ded5e7f0 con 0x55c5dee06980 2024-09-11T22:38:35.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b28ff9640 10 client.29567 handle_command_reply: tid=0 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b496e1740 1 client.29567 shutdown 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b496e1740 1 -- 172.21.15.5:0/1240789707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 msgr2=0x55c5dedf3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b496e1740 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedf3bc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c0057c0 tx=0x7f1b3c009260 comp rx=0 tx=0).stop 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b496e1740 1 -- 172.21.15.5:0/1240789707 shutdown_connections 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b496e1740 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c5dee06980 0x55c5dee08db0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b496e1740 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedf6d40 0x55c5dedf4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b496e1740 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 0x55c5dedf4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b496e1740 1 --2- 172.21.15.5:0/1240789707 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedebb60 0x55c5dedf3bc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.845+0000 7f1b28ff9640 0 client.29567 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.846+0000 7f1b496e1740 1 -- 172.21.15.5:0/1240789707 >> 172.21.15.5:0/1240789707 conn(0x55c5dede80f0 msgr2=0x55c5dede7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.846+0000 7f1b496e1740 1 -- 172.21.15.5:0/1240789707 shutdown_connections 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.846+0000 7f1b496e1740 1 -- 172.21.15.5:0/1240789707 wait complete. 2024-09-11T22:38:35.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.846+0000 7f1b496e1740 20 client.29567 trim_cache size 0 max 16384 2024-09-11T22:38:35.859 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:35 smithi172 ceph-mon[35461]: pgmap v522: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-11T22:38:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.863+0000 7f1b496e1740 1 Processor -- start 2024-09-11T22:38:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.863+0000 7f1b496e1740 1 -- start start 2024-09-11T22:38:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.863+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dee0e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.863+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 0x55c5dee0ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.863+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedf6d40 0x55c5deded5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.863+0000 7f1b496e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401ad9c0 con 0x55c5dedebb60 2024-09-11T22:38:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.863+0000 7f1b496e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401af500 con 0x55c5dedf6d40 2024-09-11T22:38:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b496e1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401ae090 con 0x55c5dedec480 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b450a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dee0e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b450a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dee0e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46214/0 (socket says 172.21.15.5:46214) 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b450a4640 1 -- 172.21.15.5:0/96161358 learned_addr learned my addr 172.21.15.5:0/96161358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b448a3640 1 --2- 172.21.15.5:0/96161358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 0x55c5dee0ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b458a5640 1 --2- 172.21.15.5:0/96161358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedf6d40 0x55c5deded5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b450a4640 1 -- 172.21.15.5:0/96161358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 msgr2=0x55c5dee0ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b450a4640 1 --2- 172.21.15.5:0/96161358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 0x55c5dee0ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b450a4640 1 -- 172.21.15.5:0/96161358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedf6d40 msgr2=0x55c5deded5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b450a4640 1 --2- 172.21.15.5:0/96161358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedf6d40 0x55c5deded5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b450a4640 1 -- 172.21.15.5:0/96161358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5dede2680 con 0x55c5dedebb60 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b458a5640 1 --2- 172.21.15.5:0/96161358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedf6d40 0x55c5deded5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b450a4640 1 --2- 172.21.15.5:0/96161358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dee0e6d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009900 tx=0x7f1b3c031440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54b1a545631a114 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b2affd640 1 -- 172.21.15.5:0/96161358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c067070 con 0x55c5dedebb60 2024-09-11T22:38:35.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b2affd640 1 -- 172.21.15.5:0/96161358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3c05e040 con 0x55c5dedebb60 2024-09-11T22:38:35.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.864+0000 7f1b2affd640 1 -- 172.21.15.5:0/96161358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c05d070 con 0x55c5dedebb60 2024-09-11T22:38:35.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.865+0000 7f1b496e1740 1 -- 172.21.15.5:0/96161358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 msgr2=0x55c5dee0e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.865+0000 7f1b496e1740 1 --2- 172.21.15.5:0/96161358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dee0e6d0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009900 tx=0x7f1b3c031440 comp rx=0 tx=0).stop 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.865+0000 7f1b496e1740 1 -- 172.21.15.5:0/96161358 shutdown_connections 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.865+0000 7f1b496e1740 1 --2- 172.21.15.5:0/96161358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedf6d40 0x55c5deded5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.865+0000 7f1b496e1740 1 --2- 172.21.15.5:0/96161358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedec480 0x55c5dee0ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.865+0000 7f1b496e1740 1 --2- 172.21.15.5:0/96161358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dee0e6d0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.865+0000 7f1b496e1740 1 -- 172.21.15.5:0/96161358 >> 172.21.15.5:0/96161358 conn(0x55c5dede80f0 msgr2=0x55c5dedf2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.865+0000 7f1b496e1740 1 -- 172.21.15.5:0/96161358 shutdown_connections 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.865+0000 7f1b496e1740 1 -- 172.21.15.5:0/96161358 wait complete. 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b496e1740 1 -- start start 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b496e1740 1 Processor -- start 2024-09-11T22:38:35.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dedf0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedec480 0x55c5dedf1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b496e1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedf6d40 0x55c5dedf1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b496e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401aee30 con 0x55c5dedebb60 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b496e1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401ae760 con 0x55c5dedec480 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b496e1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401ab9c0 con 0x55c5dedf6d40 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b496e1740 10 client.0 _ll_register_callbacks cb 0x7f1b3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b450a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dedf0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b450a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dedf0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46224/0 (socket says 172.21.15.5:46224) 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b450a4640 1 -- 172.21.15.5:0/249699361 learned_addr learned my addr 172.21.15.5:0/249699361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b448a3640 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedec480 0x55c5dedf1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.866+0000 7f1b458a5640 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedf6d40 0x55c5dedf1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b450a4640 1 -- 172.21.15.5:0/249699361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedf6d40 msgr2=0x55c5dedf1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b450a4640 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedf6d40 0x55c5dedf1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b450a4640 1 -- 172.21.15.5:0/249699361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedec480 msgr2=0x55c5dedf1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b450a4640 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedec480 0x55c5dedf1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b450a4640 1 -- 172.21.15.5:0/249699361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5dee706f0 con 0x55c5dedebb60 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b450a4640 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dedf0c00 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c02a9a0 tx=0x7f1b3c005530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae11545d2f6172e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b28ff9640 1 -- 172.21.15.5:0/249699361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c067070 con 0x55c5dedebb60 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b28ff9640 1 -- 172.21.15.5:0/249699361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f1b3c003ac0 con 0x55c5dedebb60 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b496e1740 1 -- 172.21.15.5:0/249699361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b3c05e040 con 0x55c5dedebb60 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b28ff9640 1 -- 172.21.15.5:0/249699361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1b3c043040 con 0x55c5dedebb60 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.867+0000 7f1b496e1740 10 client.20496 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.868+0000 7f1b496e1740 1 -- 172.21.15.5:0/249699361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c5dee71a10 con 0x55c5dedebb60 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.868+0000 7f1b28ff9640 1 -- 172.21.15.5:0/249699361 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b3c05e040 con 0x55c5dedebb60 2024-09-11T22:38:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.868+0000 7f1b28ff9640 1 -- 172.21.15.5:0/249699361 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f1b3c03d930 con 0x55c5dedebb60 2024-09-11T22:38:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.868+0000 7f1b496e1740 10 client.20496 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.868+0000 7f1b496e1740 10 client.20496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.868+0000 7f1b496e1740 20 client.20496 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.868+0000 7f1b496e1740 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c5dee72070 0x55c5dee74460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.868+0000 7f1b496e1740 4 client.20496 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:38:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.868+0000 7f1b496e1740 1 -- 172.21.15.5:0/249699361 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c5ded58d00 con 0x55c5dee72070 2024-09-11T22:38:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.868+0000 7f1b448a3640 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c5dee72070 0x55c5dee74460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.869+0000 7f1b448a3640 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c5dee72070 0x55c5dee74460 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb4298df3586d96 server_cookie=58de6c0c0196c139 in_seq=0 out_seq=0 2024-09-11T22:38:35.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.869+0000 7f1b28ff9640 10 client.20496 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b28ff9640 1 -- 172.21.15.5:0/249699361 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c5ded58d00 con 0x55c5dee72070 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b28ff9640 10 client.20496 handle_command_reply: tid=0 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b496e1740 1 client.20496 shutdown 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b496e1740 1 -- 172.21.15.5:0/249699361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 msgr2=0x55c5dedf0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b496e1740 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dedf0c00 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c02a9a0 tx=0x7f1b3c005530 comp rx=0 tx=0).stop 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b496e1740 1 -- 172.21.15.5:0/249699361 shutdown_connections 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b496e1740 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c5dee72070 0x55c5dee74460 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b496e1740 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c5dedf6d40 0x55c5dedf1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b496e1740 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c5dedec480 0x55c5dedf1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b496e1740 1 --2- 172.21.15.5:0/249699361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5dedebb60 0x55c5dedf0c00 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b496e1740 1 -- 172.21.15.5:0/249699361 >> 172.21.15.5:0/249699361 conn(0x55c5dede80f0 msgr2=0x55c5deded5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.872+0000 7f1b28ff9640 0 client.20496 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:35.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b496e1740 1 -- 172.21.15.5:0/249699361 shutdown_connections 2024-09-11T22:38:35.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b496e1740 1 -- 172.21.15.5:0/249699361 wait complete. 2024-09-11T22:38:35.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b496e1740 20 client.20496 trim_cache size 0 max 16384 2024-09-11T22:38:35.876 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:38:35.876 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:38:35.876 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "624f9247-0649-47d1-bcf3-5be70d54595e", 2024-09-11T22:38:35.876 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:38:35.876 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:38:35.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 -- 172.21.15.5:0/2981086197 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b2c0a2cf0 msgr2=0x7f1b2c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:35.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b2c0a2cf0 0x7f1b2c0a51b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c000c00 tx=0x7f1b3c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 -- 172.21.15.5:0/2981086197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40103660 msgr2=0x7f1b40198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:35.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40103660 0x7f1b40198990 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000e9f0 tx=0x7f1b3000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 -- 172.21.15.5:0/2981086197 shutdown_connections 2024-09-11T22:38:35.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4010c170 0x7f1b4019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b40103660 0x7f1b40198990 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1b2c0a2cf0 0x7f1b2c0a51b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 --2- 172.21.15.5:0/2981086197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1b40102d40 0x7f1b40198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:35.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 -- 172.21.15.5:0/2981086197 >> 172.21.15.5:0/2981086197 conn(0x7f1b400fc7a0 msgr2=0x7f1b400fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:35.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.873+0000 7f1b47308640 1 -- 172.21.15.5:0/2981086197 shutdown_connections 2024-09-11T22:38:35.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:35.874+0000 7f1b47308640 1 -- 172.21.15.5:0/2981086197 wait complete. 2024-09-11T22:38:36.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:35 smithi159 ceph-mon[35534]: pgmap v522: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-11T22:38:36.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49310 24.36 MB/sec execute 54 sec latency 244.979 ms 2024-09-11T22:38:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:38:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:38:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:38:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:38:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:36.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:38:36.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:36.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:38:36.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:36.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:38:36.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:36 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.048 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:38:37.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:36 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:38:37.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:38:37.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:38:37.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:38:37.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:38:37.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:38:37.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:38:37.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:38:37.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:36 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:37.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49766 24.42 MB/sec execute 55 sec latency 77.135 ms 2024-09-11T22:38:37.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:37 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:38:37.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:37 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:37 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:38:37.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:37 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:37 smithi005 ceph-mon[30943]: pgmap v523: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-11T22:38:37.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:37 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:37 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:38:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:37 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:37 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:38:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:37 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:37 smithi172 ceph-mon[35461]: pgmap v523: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-11T22:38:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:37 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.878+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2319842494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f44102570 msgr2=0x7f2f441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.878+0000 7f2f4bbaa640 1 --2- 172.21.15.5:0/2319842494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f44102570 0x7f2f441029b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009900 tx=0x7f2f3402a480 comp rx=0 tx=0).stop 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.878+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2319842494 shutdown_connections 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.878+0000 7f2f4bbaa640 1 --2- 172.21.15.5:0/2319842494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f44102ef0 0x7f2f4410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.878+0000 7f2f4bbaa640 1 --2- 172.21.15.5:0/2319842494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f44102570 0x7f2f441029b0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.878+0000 7f2f4bbaa640 1 --2- 172.21.15.5:0/2319842494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44100e20 0x7f2f44101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.878+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2319842494 >> 172.21.15.5:0/2319842494 conn(0x7f2f440fc7d0 msgr2=0x7f2f440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.879+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2319842494 shutdown_connections 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.879+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2319842494 wait complete. 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.879+0000 7f2f4bbaa640 1 Processor -- start 2024-09-11T22:38:37.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.879+0000 7f2f4bbaa640 1 -- start start 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.879+0000 7f2f4bbaa640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44100e20 0x7f2f441984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.879+0000 7f2f4bbaa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f44102570 0x7f2f44198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.879+0000 7f2f4bbaa640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f44102ef0 0x7f2f4419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.879+0000 7f2f4bbaa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44110f80 con 0x7f2f44102570 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.879+0000 7f2f4bbaa640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44110e00 con 0x7f2f44100e20 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.879+0000 7f2f4bbaa640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2f44111100 con 0x7f2f44102ef0 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f49145640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f44102570 0x7f2f44198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f49145640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f44102570 0x7f2f44198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46240/0 (socket says 172.21.15.5:46240) 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f49145640 1 -- 172.21.15.5:0/2793866410 learned_addr learned my addr 172.21.15.5:0/2793866410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f49946640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44100e20 0x7f2f441984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f4a147640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f44102ef0 0x7f2f4419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f49145640 1 -- 172.21.15.5:0/2793866410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f44102ef0 msgr2=0x7f2f4419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:37.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f49145640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f44102ef0 0x7f2f4419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f49145640 1 -- 172.21.15.5:0/2793866410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44100e20 msgr2=0x7f2f441984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f49145640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44100e20 0x7f2f441984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f49145640 1 -- 172.21.15.5:0/2793866410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4419d480 con 0x7f2f44102570 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f4a147640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f44102ef0 0x7f2f4419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f49145640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f44102570 0x7f2f44198a10 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f2f34006010 tx=0x7f2f34003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f3affd640 1 -- 172.21.15.5:0/2793866410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3403f070 con 0x7f2f44102570 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f3affd640 1 -- 172.21.15.5:0/2793866410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f34031c30 con 0x7f2f44102570 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2793866410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4419d710 con 0x7f2f44102570 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f3affd640 1 -- 172.21.15.5:0/2793866410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f34035030 con 0x7f2f44102570 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.880+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2793866410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f441a5050 con 0x7f2f44102570 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.883+0000 7f2f3affd640 1 -- 172.21.15.5:0/2793866410 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2f34042070 con 0x7f2f44102570 2024-09-11T22:38:37.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.883+0000 7f2f4df83740 1 Processor -- start 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4df83740 1 -- start start 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf3f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3fdc0 0x55e4baf4a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf4a680 0x55e4baf4ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4df83740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4baea52e0 con 0x55e4baf4a680 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4df83740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e4bae4cce0 con 0x55e4baf3f4a0 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4df83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e4baea12a0 con 0x55e4baf3fdc0 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f49946640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3fdc0 0x55e4baf4a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4a147640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf3f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f49145640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf4a680 0x55e4baf4ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4a147640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf3f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43570/0 (socket says 172.21.15.5:43570) 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f49946640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3fdc0 0x55e4baf4a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60016/0 (socket says 172.21.15.5:60016) 2024-09-11T22:38:37.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4a147640 1 -- 172.21.15.5:0/101780107 learned_addr learned my addr 172.21.15.5:0/101780107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f3affd640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2f300a2cf0 0x7f2f300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4a147640 1 -- 172.21.15.5:0/101780107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3fdc0 msgr2=0x55e4baf4a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4a147640 1 --2- 172.21.15.5:0/101780107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3fdc0 0x55e4baf4a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4a147640 1 -- 172.21.15.5:0/101780107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf4a680 msgr2=0x55e4baf4ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4a147640 1 --2- 172.21.15.5:0/101780107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf4a680 0x55e4baf4ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.884+0000 7f2f4a147640 1 -- 172.21.15.5:0/101780107 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4bae31440 con 0x55e4baf3f4a0 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.885+0000 7f2f49946640 1 --2- 172.21.15.5:0/101780107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3fdc0 0x55e4baf4a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.885+0000 7f2f3affd640 1 -- 172.21.15.5:0/2793866410 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f2f340efb20 con 0x7f2f44102570 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.885+0000 7f2f4a147640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2f300a2cf0 0x7f2f300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.885+0000 7f2f4a147640 1 --2- 172.21.15.5:0/101780107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf3f880 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0117e0 tx=0x7f2f3c011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18df28a6624f4ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:37.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.885+0000 7f2f2ffff640 1 -- 172.21.15.5:0/101780107 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c020070 con 0x55e4baf3f4a0 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.885+0000 7f2f2ffff640 1 -- 172.21.15.5:0/101780107 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f3c010e50 con 0x55e4baf3f4a0 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.885+0000 7f2f2ffff640 1 -- 172.21.15.5:0/101780107 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00e3b0 con 0x55e4baf3f4a0 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4df83740 1 -- 172.21.15.5:0/101780107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 msgr2=0x55e4baf3f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4a147640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2f300a2cf0 0x7f2f300a51b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c011490 tx=0x7f2f3c01e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4df83740 1 --2- 172.21.15.5:0/101780107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf3f880 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0117e0 tx=0x7f2f3c011ca0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4df83740 1 -- 172.21.15.5:0/101780107 shutdown_connections 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4df83740 1 --2- 172.21.15.5:0/101780107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf4a680 0x55e4baf4ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4df83740 1 --2- 172.21.15.5:0/101780107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3fdc0 0x55e4baf4a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4df83740 1 --2- 172.21.15.5:0/101780107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf3f880 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4df83740 1 -- 172.21.15.5:0/101780107 >> 172.21.15.5:0/101780107 conn(0x55e4baf3ba30 msgr2=0x55e4baf3be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4df83740 1 -- 172.21.15.5:0/101780107 shutdown_connections 2024-09-11T22:38:37.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4df83740 1 -- 172.21.15.5:0/101780107 wait complete. 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.886+0000 7f2f4df83740 1 -- start start 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f4df83740 1 Processor -- start 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3f4a0 0x55e4baf49f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3fdc0 0x55e4baf47220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf4a680 0x55e4baf47760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f4df83740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4baf4d250 con 0x55e4baf4a680 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f4df83740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e4bae60050 con 0x55e4baf3fdc0 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f4df83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e4baf4d3d0 con 0x55e4baf3f4a0 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f4df83740 10 client.0 _ll_register_callbacks cb 0x55e4baea0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f49946640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3f4a0 0x55e4baf49f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f4a147640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf4a680 0x55e4baf47760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f49946640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3f4a0 0x55e4baf49f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60022/0 (socket says 172.21.15.5:60022) 2024-09-11T22:38:37.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.887+0000 7f2f49946640 1 -- 172.21.15.5:0/3837928317 learned_addr learned my addr 172.21.15.5:0/3837928317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:37.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f49946640 1 -- 172.21.15.5:0/3837928317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3fdc0 msgr2=0x55e4baf47220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:37.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f49946640 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3fdc0 0x55e4baf47220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f49946640 1 -- 172.21.15.5:0/3837928317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf4a680 msgr2=0x55e4baf47760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:37.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f49946640 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf4a680 0x55e4baf47760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f49946640 1 -- 172.21.15.5:0/3837928317 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4bac163c0 con 0x55e4baf3f4a0 2024-09-11T22:38:37.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f49946640 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3f4a0 0x55e4baf49f70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2f40012740 tx=0x7f2f40012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=188aeb5cae98fab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:37.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/3837928317 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4000f920 con 0x55e4baf3f4a0 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f4df83740 1 -- 172.21.15.5:0/3837928317 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f3c010e50 con 0x55e4baf3f4a0 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/3837928317 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f2f4000fac0 con 0x55e4baf3f4a0 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/3837928317 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2f4000fcc0 con 0x55e4baf3f4a0 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f4df83740 10 client.29416 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.888+0000 7f2f4df83740 1 -- 172.21.15.5:0/3837928317 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e4baf58c50 con 0x55e4baf3f4a0 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/3837928317 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f3c010e50 con 0x55e4baf3f4a0 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/3837928317 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f2f4001c420 con 0x55e4baf3f4a0 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f4df83740 10 client.29416 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f4df83740 10 client.29416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f4df83740 20 client.29416 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:37.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f4df83740 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4baf5a060 0x55e4baf5c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f4df83740 4 client.29416 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f4df83740 1 -- 172.21.15.5:0/3837928317 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4baeb2850 con 0x55e4baf5a060 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f49145640 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4baf5a060 0x55e4baf5c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f49145640 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4baf5a060 0x55e4baf5c4d0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9167aaa9144e308b server_cookie=464b56e6df4060d4 in_seq=0 out_seq=0 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.889+0000 7f2f2d7fa640 10 client.29416 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.893+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/3837928317 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4baeb2850 con 0x55e4baf5a060 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.893+0000 7f2f2d7fa640 10 client.29416 handle_command_reply: tid=0 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.893+0000 7f2f4df83740 1 client.29416 shutdown 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 1 -- 172.21.15.5:0/3837928317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3f4a0 msgr2=0x55e4baf49f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3f4a0 0x55e4baf49f70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2f40012740 tx=0x7f2f40012c00 comp rx=0 tx=0).stop 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 1 -- 172.21.15.5:0/3837928317 shutdown_connections 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4baf5a060 0x55e4baf5c4d0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf4a680 0x55e4baf47760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3fdc0 0x55e4baf47220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 1 --2- 172.21.15.5:0/3837928317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf3f4a0 0x55e4baf49f70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f2d7fa640 0 client.29416 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:37.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 1 -- 172.21.15.5:0/3837928317 >> 172.21.15.5:0/3837928317 conn(0x55e4baf3ba30 msgr2=0x55e4baf41760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:37.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 1 -- 172.21.15.5:0/3837928317 shutdown_connections 2024-09-11T22:38:37.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 1 -- 172.21.15.5:0/3837928317 wait complete. 2024-09-11T22:38:37.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.894+0000 7f2f4df83740 20 client.29416 trim_cache size 0 max 16384 2024-09-11T22:38:37.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f4df83740 1 Processor -- start 2024-09-11T22:38:37.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f4df83740 1 -- start start 2024-09-11T22:38:37.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3f4a0 0x55e4baf564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3fdc0 0x55e4baf3be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 0x55e4baf61da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f4df83740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441ab870 con 0x55e4baf3f4a0 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f4df83740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441ad3b0 con 0x55e4baf3fdc0 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f4df83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441abf40 con 0x55e4baf4a680 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f49946640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3f4a0 0x55e4baf564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f49946640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3f4a0 0x55e4baf564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46266/0 (socket says 172.21.15.5:46266) 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f49946640 1 -- 172.21.15.5:0/416927055 learned_addr learned my addr 172.21.15.5:0/416927055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f49145640 1 --2- 172.21.15.5:0/416927055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3fdc0 0x55e4baf3be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f4a147640 1 --2- 172.21.15.5:0/416927055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 0x55e4baf61da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f49946640 1 -- 172.21.15.5:0/416927055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 msgr2=0x55e4baf61da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f49946640 1 --2- 172.21.15.5:0/416927055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 0x55e4baf61da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f49946640 1 -- 172.21.15.5:0/416927055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3fdc0 msgr2=0x55e4baf3be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f49946640 1 --2- 172.21.15.5:0/416927055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3fdc0 0x55e4baf3be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.912+0000 7f2f49946640 1 -- 172.21.15.5:0/416927055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4baf461d0 con 0x55e4baf3f4a0 2024-09-11T22:38:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.913+0000 7f2f49946640 1 --2- 172.21.15.5:0/416927055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3f4a0 0x55e4baf564b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000c00 tx=0x7f2f40009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406890d9205a3264 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.913+0000 7f2f2f7fe640 1 -- 172.21.15.5:0/416927055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4001c420 con 0x55e4baf3f4a0 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.913+0000 7f2f2f7fe640 1 -- 172.21.15.5:0/416927055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f40012e50 con 0x55e4baf3f4a0 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.913+0000 7f2f2f7fe640 1 -- 172.21.15.5:0/416927055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40004620 con 0x55e4baf3f4a0 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 -- 172.21.15.5:0/416927055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3f4a0 msgr2=0x55e4baf564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 --2- 172.21.15.5:0/416927055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3f4a0 0x55e4baf564b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f2f40000c00 tx=0x7f2f40009ee0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 -- 172.21.15.5:0/416927055 shutdown_connections 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 --2- 172.21.15.5:0/416927055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 0x55e4baf61da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 --2- 172.21.15.5:0/416927055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3fdc0 0x55e4baf3be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 --2- 172.21.15.5:0/416927055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3f4a0 0x55e4baf564b0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 -- 172.21.15.5:0/416927055 >> 172.21.15.5:0/416927055 conn(0x55e4baf3ba30 msgr2=0x55e4baf42900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 -- 172.21.15.5:0/416927055 shutdown_connections 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 -- 172.21.15.5:0/416927055 wait complete. 2024-09-11T22:38:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 -- start start 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.914+0000 7f2f4df83740 1 Processor -- start 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3fdc0 0x55e4baf44730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f4df83740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 0x55e4baf44c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f4df83740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441acce0 con 0x55e4baf3fdc0 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f4df83740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441ac610 con 0x55e4baf3f4a0 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f4df83740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2f441a9870 con 0x55e4baf4a680 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f4df83740 10 client.0 _ll_register_callbacks cb 0x7f2f40009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f49145640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3fdc0 0x55e4baf44730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f49946640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f4a147640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 0x55e4baf44c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f49946640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43596/0 (socket says 172.21.15.5:43596) 2024-09-11T22:38:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f49946640 1 -- 172.21.15.5:0/210749996 learned_addr learned my addr 172.21.15.5:0/210749996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f4a147640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 0x55e4baf44c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60038/0 (socket says 172.21.15.5:60038) 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f49946640 1 -- 172.21.15.5:0/210749996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 msgr2=0x55e4baf44c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f49946640 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 0x55e4baf44c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f49946640 1 -- 172.21.15.5:0/210749996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3fdc0 msgr2=0x55e4baf44730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f49946640 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3fdc0 0x55e4baf44730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f49946640 1 -- 172.21.15.5:0/210749996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4baf4a2e0 con 0x55e4baf3f4a0 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.915+0000 7f2f4a147640 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 0x55e4baf44c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f49946640 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf441f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2f40005fd0 tx=0x7f2f4000fbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6677edb47d4fc507 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/210749996 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40031020 con 0x55e4baf3f4a0 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/210749996 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f2f400050b0 con 0x55e4baf3f4a0 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f4df83740 1 -- 172.21.15.5:0/210749996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f40012e50 con 0x55e4baf3f4a0 2024-09-11T22:38:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/210749996 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2f400052b0 con 0x55e4baf3f4a0 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f4df83740 10 client.29588 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f4df83740 1 -- 172.21.15.5:0/210749996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e4bafc4e10 con 0x55e4baf3f4a0 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/210749996 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f40012e50 con 0x55e4baf3f4a0 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/210749996 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f2f4002b020 con 0x55e4baf3f4a0 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f4df83740 10 client.29588 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f4df83740 10 client.29588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f4df83740 20 client.29588 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f4df83740 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4bafc52d0 0x55e4bafc76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f4df83740 4 client.29588 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.916+0000 7f2f4df83740 1 -- 172.21.15.5:0/210749996 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e4baeacd60 con 0x55e4bafc52d0 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.917+0000 7f2f49145640 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4bafc52d0 0x55e4bafc76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.917+0000 7f2f2d7fa640 10 client.29588 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.917+0000 7f2f49145640 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4bafc52d0 0x55e4bafc76c0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9fa1ce4df18e8dd server_cookie=5756295856119d44 in_seq=0 out_seq=0 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.922+0000 7f2f2d7fa640 1 -- 172.21.15.5:0/210749996 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e4baeacd60 con 0x55e4bafc52d0 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.922+0000 7f2f2d7fa640 10 client.29588 handle_command_reply: tid=0 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.922+0000 7f2f4df83740 1 client.29588 shutdown 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.922+0000 7f2f4df83740 1 -- 172.21.15.5:0/210749996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 msgr2=0x55e4baf441f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.922+0000 7f2f4df83740 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf441f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2f40005fd0 tx=0x7f2f4000fbd0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4df83740 1 -- 172.21.15.5:0/210749996 shutdown_connections 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4df83740 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e4bafc52d0 0x55e4bafc76c0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4df83740 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e4baf4a680 0x55e4baf44c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4df83740 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4baf3fdc0 0x55e4baf44730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4df83740 1 --2- 172.21.15.5:0/210749996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4baf3f4a0 0x55e4baf441f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f2d7fa640 0 client.29588 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4df83740 1 -- 172.21.15.5:0/210749996 >> 172.21.15.5:0/210749996 conn(0x55e4baf3ba30 msgr2=0x55e4baf3a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4df83740 1 -- 172.21.15.5:0/210749996 shutdown_connections 2024-09-11T22:38:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4df83740 1 -- 172.21.15.5:0/210749996 wait complete. 2024-09-11T22:38:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4df83740 20 client.29588 trim_cache size 0 max 16384 2024-09-11T22:38:37.926 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:38:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:38:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:38:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "624f9247-0649-47d1-bcf3-5be70d54595e": { 2024-09-11T22:38:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:38:37.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "624f9247-0649-47d1-bcf3-5be70d54595e", 2024-09-11T22:38:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:38:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:38:37.927 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:38:37.927 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:38:37.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2793866410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2f300a2cf0 msgr2=0x7f2f300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:37.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4bbaa640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2f300a2cf0 0x7f2f300a51b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c011490 tx=0x7f2f3c01e000 comp rx=0 tx=0).stop 2024-09-11T22:38:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2793866410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f44102570 msgr2=0x7f2f44198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4bbaa640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f44102570 0x7f2f44198a10 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f2f34006010 tx=0x7f2f34003c10 comp rx=0 tx=0).stop 2024-09-11T22:38:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2793866410 shutdown_connections 2024-09-11T22:38:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4bbaa640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f44102ef0 0x7f2f4419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4bbaa640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f44102570 0x7f2f44198a10 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.923+0000 7f2f4bbaa640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2f300a2cf0 0x7f2f300a51b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.924+0000 7f2f4bbaa640 1 --2- 172.21.15.5:0/2793866410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f44100e20 0x7f2f441984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.924+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2793866410 >> 172.21.15.5:0/2793866410 conn(0x7f2f440fc7d0 msgr2=0x7f2f4406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.924+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2793866410 shutdown_connections 2024-09-11T22:38:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:37.924+0000 7f2f4bbaa640 1 -- 172.21.15.5:0/2793866410 wait complete. 2024-09-11T22:38:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:37 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:38:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:37 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:37 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:38:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:37 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:37 smithi159 ceph-mon[35534]: pgmap v523: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-11T22:38:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:37 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:38:38.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 50175 24.39 MB/sec execute 56 sec latency 178.559 ms 2024-09-11T22:38:38.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:38 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:38:38.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:38 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:38:38.875 INFO:tasks.cephfs.filesystem:scrub status for tag:624f9247-0649-47d1-bcf3-5be70d54595e - {'path': '/', 'tag': '624f9247-0649-47d1-bcf3-5be70d54595e', 'options': 'recursive,force'} 2024-09-11T22:38:39.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:38 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:38:39.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 50652 24.35 MB/sec execute 57 sec latency 187.143 ms 2024-09-11T22:38:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:39 smithi172 ceph-mon[35461]: pgmap v524: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.9 MiB/s wr, 811 op/s 2024-09-11T22:38:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:39 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:38:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:39 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:38:39.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:39 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:38:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:39 smithi005 ceph-mon[30943]: pgmap v524: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.9 MiB/s wr, 811 op/s 2024-09-11T22:38:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:39 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:38:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:39 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:38:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:39 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:38:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:39 smithi159 ceph-mon[35534]: pgmap v524: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.9 MiB/s wr, 811 op/s 2024-09-11T22:38:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:39 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:38:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:39 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:38:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:39 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:38:40.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51207 24.48 MB/sec execute 58 sec latency 148.666 ms 2024-09-11T22:38:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.727+0000 7f83f1b59640 1 -- 172.21.15.5:0/4197445259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec1009b0 msgr2=0x7f83ec100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.727+0000 7f83f1b59640 1 --2- 172.21.15.5:0/4197445259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec1009b0 0x7f83ec100e10 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009930 tx=0x7f83e002a420 comp rx=0 tx=0).stop 2024-09-11T22:38:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.727+0000 7f83f1b59640 1 -- 172.21.15.5:0/4197445259 shutdown_connections 2024-09-11T22:38:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.727+0000 7f83f1b59640 1 --2- 172.21.15.5:0/4197445259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83ec101350 0x7f83ec10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.727+0000 7f83f1b59640 1 --2- 172.21.15.5:0/4197445259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec1009b0 0x7f83ec100e10 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.727+0000 7f83f1b59640 1 --2- 172.21.15.5:0/4197445259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec1069c0 0x7f83ec106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.727+0000 7f83f1b59640 1 -- 172.21.15.5:0/4197445259 >> 172.21.15.5:0/4197445259 conn(0x7f83ec077990 msgr2=0x7f83ec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.727+0000 7f83f1b59640 1 -- 172.21.15.5:0/4197445259 shutdown_connections 2024-09-11T22:38:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.727+0000 7f83f1b59640 1 -- 172.21.15.5:0/4197445259 wait complete. 2024-09-11T22:38:40.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.727+0000 7f83f1b59640 1 Processor -- start 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83f1b59640 1 -- start start 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83f1b59640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83ec1009b0 0x7f83ec199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83f1b59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec101350 0x7f83ec1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83f1b59640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec1069c0 0x7f83ec1a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83f1b59640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec110e50 con 0x7f83ec101350 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83f1b59640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec110cd0 con 0x7f83ec1009b0 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83f1b59640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec110fd0 con 0x7f83ec1069c0 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec101350 0x7f83ec1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec101350 0x7f83ec1997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46298/0 (socket says 172.21.15.5:46298) 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83eaffd640 1 -- 172.21.15.5:0/2219365161 learned_addr learned my addr 172.21.15.5:0/2219365161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83ec1009b0 0x7f83ec199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83ebfff640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec1069c0 0x7f83ec1a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83eaffd640 1 -- 172.21.15.5:0/2219365161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec1069c0 msgr2=0x7f83ec1a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.728+0000 7f83eaffd640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec1069c0 0x7f83ec1a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.729+0000 7f83eaffd640 1 -- 172.21.15.5:0/2219365161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83ec1009b0 msgr2=0x7f83ec199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.729+0000 7f83eaffd640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83ec1009b0 0x7f83ec199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.729+0000 7f83eaffd640 1 -- 172.21.15.5:0/2219365161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec1a4840 con 0x7f83ec101350 2024-09-11T22:38:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.729+0000 7f83eaffd640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec101350 0x7f83ec1997c0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f83e002a940 tx=0x7f83e0003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.729+0000 7f83e8ff9640 1 -- 172.21.15.5:0/2219365161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e003f070 con 0x7f83ec101350 2024-09-11T22:38:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.729+0000 7f83e8ff9640 1 -- 172.21.15.5:0/2219365161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e0031c90 con 0x7f83ec101350 2024-09-11T22:38:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.729+0000 7f83f1b59640 1 -- 172.21.15.5:0/2219365161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ec1a4a10 con 0x7f83ec101350 2024-09-11T22:38:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.729+0000 7f83e8ff9640 1 -- 172.21.15.5:0/2219365161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0035030 con 0x7f83ec101350 2024-09-11T22:38:40.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.729+0000 7f83f1b59640 1 -- 172.21.15.5:0/2219365161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ec1a4e70 con 0x7f83ec101350 2024-09-11T22:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.731+0000 7f83e8ff9640 1 -- 172.21.15.5:0/2219365161 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f83e0042070 con 0x7f83ec101350 2024-09-11T22:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83f49d0740 1 Processor -- start 2024-09-11T22:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83f49d0740 1 -- start start 2024-09-11T22:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be642ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 0x5618be64d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be6500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83f49d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618be5a72f0 con 0x5618be64dcb0 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83f49d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5618be54f270 con 0x5618be642af0 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83f49d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5618be5a32b0 con 0x5618be643410 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83ebfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be642ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be6500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83eb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 0x5618be64d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83ebfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be642ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43626/0 (socket says 172.21.15.5:43626) 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83ebfff640 1 -- 172.21.15.5:0/2660557314 learned_addr learned my addr 172.21.15.5:0/2660557314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83e8ff9640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83d40a2da0 0x7f83d40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.732+0000 7f83ebfff640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83d40a2da0 0x7f83d40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83eb7fe640 1 -- 172.21.15.5:0/2660557314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 msgr2=0x5618be642ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/2660557314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be642ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83eb7fe640 1 -- 172.21.15.5:0/2660557314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 msgr2=0x5618be6500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/2660557314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be6500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83eb7fe640 1 -- 172.21.15.5:0/2660557314 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618be54e340 con 0x5618be643410 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83eaffd640 1 --2- 172.21.15.5:0/2660557314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be6500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83e8ff9640 1 -- 172.21.15.5:0/2219365161 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f83e00efac0 con 0x7f83ec101350 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/2660557314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 0x5618be64d770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009c00 tx=0x7f83dc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d6e7aea8ae0ef server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83ebfff640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83d40a2da0 0x7f83d40a5260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009fd0 tx=0x7f83e4004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83d1ffb640 1 -- 172.21.15.5:0/2660557314 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc004050 con 0x5618be643410 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83d1ffb640 1 -- 172.21.15.5:0/2660557314 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83dc005500 con 0x5618be643410 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.733+0000 7f83d1ffb640 1 -- 172.21.15.5:0/2660557314 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc004e00 con 0x5618be643410 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.734+0000 7f83f49d0740 1 -- 172.21.15.5:0/2660557314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 msgr2=0x5618be64d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.734+0000 7f83f49d0740 1 --2- 172.21.15.5:0/2660557314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 0x5618be64d770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009c00 tx=0x7f83dc011810 comp rx=0 tx=0).stop 2024-09-11T22:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.734+0000 7f83f49d0740 1 -- 172.21.15.5:0/2660557314 shutdown_connections 2024-09-11T22:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.734+0000 7f83f49d0740 1 --2- 172.21.15.5:0/2660557314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be6500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.734+0000 7f83f49d0740 1 --2- 172.21.15.5:0/2660557314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 0x5618be64d770 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.734+0000 7f83f49d0740 1 --2- 172.21.15.5:0/2660557314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be642ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.734+0000 7f83f49d0740 1 -- 172.21.15.5:0/2660557314 >> 172.21.15.5:0/2660557314 conn(0x5618be63f060 msgr2=0x5618be63f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.734+0000 7f83f49d0740 1 -- 172.21.15.5:0/2660557314 shutdown_connections 2024-09-11T22:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.734+0000 7f83f49d0740 1 -- 172.21.15.5:0/2660557314 wait complete. 2024-09-11T22:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.734+0000 7f83f49d0740 1 -- start start 2024-09-11T22:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83f49d0740 1 Processor -- start 2024-09-11T22:38:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be64ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 0x5618be64b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be64b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83f49d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618be650880 con 0x5618be64dcb0 2024-09-11T22:38:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83f49d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5618be530290 con 0x5618be642af0 2024-09-11T22:38:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83f49d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5618be650a00 con 0x5618be643410 2024-09-11T22:38:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83f49d0740 10 client.0 _ll_register_callbacks cb 0x5618be561140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83ebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be64b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83eb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be64ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83eb7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be64ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43640/0 (socket says 172.21.15.5:43640) 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83ebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be64b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46316/0 (socket says 172.21.15.5:46316) 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83eb7fe640 1 -- 172.21.15.5:0/277545843 learned_addr learned my addr 172.21.15.5:0/277545843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83eaffd640 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 0x5618be64b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83eb7fe640 1 -- 172.21.15.5:0/277545843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 msgr2=0x5618be64b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 0x5618be64b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83eb7fe640 1 -- 172.21.15.5:0/277545843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 msgr2=0x5618be64b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be64b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.735+0000 7f83eb7fe640 1 -- 172.21.15.5:0/277545843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618be2ba210 con 0x5618be642af0 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.736+0000 7f83ebfff640 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be64b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.736+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be64ab20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009710 tx=0x7f83dc009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24098342c8eba0fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:40.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.736+0000 7f83caffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:38:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.736+0000 7f83caffd640 1 -- 172.21.15.5:0/277545843 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc029070 con 0x5618be642af0 2024-09-11T22:38:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.736+0000 7f83f49d0740 1 -- 172.21.15.5:0/277545843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83dc005500 con 0x5618be642af0 2024-09-11T22:38:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.736+0000 7f83caffd640 1 -- 172.21.15.5:0/277545843 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f83dc004d80 con 0x5618be642af0 2024-09-11T22:38:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.736+0000 7f83caffd640 1 -- 172.21.15.5:0/277545843 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f83dc035050 con 0x5618be642af0 2024-09-11T22:38:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.736+0000 7f83f49d0740 10 client.29600 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.736+0000 7f83f49d0740 1 -- 172.21.15.5:0/277545843 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5618be386620 con 0x5618be642af0 2024-09-11T22:38:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.736+0000 7f83caffd640 1 -- 172.21.15.5:0/277545843 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83dc005500 con 0x5618be642af0 2024-09-11T22:38:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.737+0000 7f83caffd640 1 -- 172.21.15.5:0/277545843 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f83dc023430 con 0x5618be642af0 2024-09-11T22:38:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.737+0000 7f83f49d0740 10 client.29600 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:40.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.737+0000 7f83f49d0740 10 client.29600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.737+0000 7f83f49d0740 20 client.29600 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.737+0000 7f83f49d0740 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618be65da70 0x5618be65fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.737+0000 7f83f49d0740 4 client.29600 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.737+0000 7f83f49d0740 1 -- 172.21.15.5:0/277545843 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618be5b4a20 con 0x5618be65da70 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.737+0000 7f83eaffd640 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618be65da70 0x5618be65fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.737+0000 7f83caffd640 10 client.29600 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.737+0000 7f83eaffd640 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618be65da70 0x5618be65fe60 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9b583817104122 server_cookie=2d814dd85236fc0f in_seq=0 out_seq=0 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.740+0000 7f83caffd640 1 -- 172.21.15.5:0/277545843 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618be5b4a20 con 0x5618be65da70 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.740+0000 7f83caffd640 10 client.29600 handle_command_reply: tid=0 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.740+0000 7f83f49d0740 1 client.29600 shutdown 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 1 -- 172.21.15.5:0/277545843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 msgr2=0x5618be64ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be64ab20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009710 tx=0x7f83dc009830 comp rx=0 tx=0).stop 2024-09-11T22:38:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 1 -- 172.21.15.5:0/277545843 shutdown_connections 2024-09-11T22:38:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618be65da70 0x5618be65fe60 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be64b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be643410 0x5618be64b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83caffd640 0 client.29600 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 1 --2- 172.21.15.5:0/277545843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be642af0 0x5618be64ab20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 1 -- 172.21.15.5:0/277545843 >> 172.21.15.5:0/277545843 conn(0x5618be63f060 msgr2=0x5618be644d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 1 -- 172.21.15.5:0/277545843 shutdown_connections 2024-09-11T22:38:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 1 -- 172.21.15.5:0/277545843 wait complete. 2024-09-11T22:38:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.741+0000 7f83f49d0740 20 client.29600 trim_cache size 0 max 16384 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83f49d0740 1 Processor -- start 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83f49d0740 1 -- start start 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 0x5618be64ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be643410 0x5618be64b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be665550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83f49d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec1ad990 con 0x5618be64dcb0 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83f49d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec1af4d0 con 0x5618be643410 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83f49d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec1ae060 con 0x5618be642af0 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83ebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be665550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83ebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be665550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46328/0 (socket says 172.21.15.5:46328) 2024-09-11T22:38:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83ebfff640 1 -- 172.21.15.5:0/3216016560 learned_addr learned my addr 172.21.15.5:0/3216016560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/3216016560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 0x5618be64ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.757+0000 7f83eaffd640 1 --2- 172.21.15.5:0/3216016560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be643410 0x5618be64b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.758+0000 7f83ebfff640 1 -- 172.21.15.5:0/3216016560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 msgr2=0x5618be64ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.758+0000 7f83ebfff640 1 --2- 172.21.15.5:0/3216016560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 0x5618be64ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.758+0000 7f83ebfff640 1 -- 172.21.15.5:0/3216016560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be643410 msgr2=0x5618be64b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.758+0000 7f83ebfff640 1 --2- 172.21.15.5:0/3216016560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be643410 0x5618be64b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.758+0000 7f83ebfff640 1 -- 172.21.15.5:0/3216016560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618be63ee90 con 0x5618be64dcb0 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.758+0000 7f83ebfff640 1 --2- 172.21.15.5:0/3216016560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be665550 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f83e4004260 tx=0x7f83e4009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a634f72f270270 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.758+0000 7f83d17fa640 1 -- 172.21.15.5:0/3216016560 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e40109b0 con 0x5618be64dcb0 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.758+0000 7f83d17fa640 1 -- 172.21.15.5:0/3216016560 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4010b50 con 0x5618be64dcb0 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.758+0000 7f83d17fa640 1 -- 172.21.15.5:0/3216016560 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4021640 con 0x5618be64dcb0 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 -- 172.21.15.5:0/3216016560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 msgr2=0x5618be665550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:40.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 --2- 172.21.15.5:0/3216016560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be665550 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f83e4004260 tx=0x7f83e4009be0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 -- 172.21.15.5:0/3216016560 shutdown_connections 2024-09-11T22:38:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 --2- 172.21.15.5:0/3216016560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be64dcb0 0x5618be665550 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 --2- 172.21.15.5:0/3216016560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be643410 0x5618be64b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 --2- 172.21.15.5:0/3216016560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 0x5618be64ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 -- 172.21.15.5:0/3216016560 >> 172.21.15.5:0/3216016560 conn(0x5618be63f060 msgr2=0x5618be64c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 -- 172.21.15.5:0/3216016560 shutdown_connections 2024-09-11T22:38:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 -- 172.21.15.5:0/3216016560 wait complete. 2024-09-11T22:38:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 -- start start 2024-09-11T22:38:40.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.759+0000 7f83f49d0740 1 Processor -- start 2024-09-11T22:38:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 0x5618be6560a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be643410 0x5618be6474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83f49d0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be64dcb0 0x5618be647a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83f49d0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec1aee00 con 0x5618be643410 2024-09-11T22:38:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83f49d0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec1ae730 con 0x5618be64dcb0 2024-09-11T22:38:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83f49d0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec1ab990 con 0x5618be642af0 2024-09-11T22:38:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83f49d0740 10 client.0 _ll_register_callbacks cb 0x7f83e00b80f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 0x5618be6560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be643410 0x5618be6474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83ebfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be64dcb0 0x5618be647a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eb7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 0x5618be6560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60098/0 (socket says 172.21.15.5:60098) 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eb7fe640 1 -- 172.21.15.5:0/3108072816 learned_addr learned my addr 172.21.15.5:0/3108072816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eb7fe640 1 -- 172.21.15.5:0/3108072816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be64dcb0 msgr2=0x5618be647a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be64dcb0 0x5618be647a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eb7fe640 1 -- 172.21.15.5:0/3108072816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be643410 msgr2=0x5618be6474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be643410 0x5618be6474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eb7fe640 1 -- 172.21.15.5:0/3108072816 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618be647f60 con 0x5618be642af0 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eaffd640 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be643410 0x5618be6474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83eb7fe640 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 0x5618be6560a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f83dc011850 tx=0x7f83dc004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb504e9cdd87eec server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.760+0000 7f83caffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83caffd640 1 -- 172.21.15.5:0/3108072816 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc03e060 con 0x5618be642af0 2024-09-11T22:38:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83caffd640 1 -- 172.21.15.5:0/3108072816 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f83dc0035f0 con 0x5618be642af0 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83f49d0740 1 -- 172.21.15.5:0/3108072816 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e4010b50 con 0x5618be642af0 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83caffd640 1 -- 172.21.15.5:0/3108072816 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f83dc029040 con 0x5618be642af0 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83f49d0740 10 client.29440 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83f49d0740 1 -- 172.21.15.5:0/3108072816 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5618be6c81d0 con 0x5618be642af0 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83caffd640 1 -- 172.21.15.5:0/3108072816 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4010b50 con 0x5618be642af0 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83caffd640 1 -- 172.21.15.5:0/3108072816 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f83dc02e670 con 0x5618be642af0 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83f49d0740 10 client.29440 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83f49d0740 10 client.29440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83f49d0740 20 client.29440 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83f49d0740 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618be6c8780 0x5618be6cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83f49d0740 4 client.29440 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83f49d0740 1 -- 172.21.15.5:0/3108072816 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5618be5aed70 con 0x5618be6c8780 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.761+0000 7f83eaffd640 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618be6c8780 0x5618be6cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.762+0000 7f83eaffd640 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618be6c8780 0x5618be6cab70 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74064b7ec2b14f1 server_cookie=4bc5e60eabef4003 in_seq=0 out_seq=0 2024-09-11T22:38:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:40.762+0000 7f83caffd640 10 client.29440 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.351+0000 7f83caffd640 1 -- 172.21.15.5:0/3108072816 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5618be5aed70 con 0x5618be6c8780 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.351+0000 7f83caffd640 10 client.29440 handle_command_reply: tid=0 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.351+0000 7f83f49d0740 1 client.29440 shutdown 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.351+0000 7f83f49d0740 1 -- 172.21.15.5:0/3108072816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 msgr2=0x5618be6560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.351+0000 7f83f49d0740 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 0x5618be6560a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f83dc011850 tx=0x7f83dc004360 comp rx=0 tx=0).stop 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.351+0000 7f83f49d0740 1 -- 172.21.15.5:0/3108072816 shutdown_connections 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.351+0000 7f83f49d0740 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5618be6c8780 0x5618be6cab70 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.351+0000 7f83f49d0740 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5618be64dcb0 0x5618be647a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.352+0000 7f83caffd640 0 client.29440 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.352+0000 7f83f49d0740 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618be643410 0x5618be6474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.352+0000 7f83f49d0740 1 --2- 172.21.15.5:0/3108072816 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5618be642af0 0x5618be6560a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.352+0000 7f83f49d0740 1 -- 172.21.15.5:0/3108072816 >> 172.21.15.5:0/3108072816 conn(0x5618be63f060 msgr2=0x5618be644550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.352+0000 7f83f49d0740 1 -- 172.21.15.5:0/3108072816 shutdown_connections 2024-09-11T22:38:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.352+0000 7f83f49d0740 1 -- 172.21.15.5:0/3108072816 wait complete. 2024-09-11T22:38:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.352+0000 7f83f49d0740 20 client.29440 trim_cache size 0 max 16384 2024-09-11T22:38:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:38:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:38:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:38:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:38:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 -- 172.21.15.5:0/2219365161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83d40a2da0 msgr2=0x7f83d40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83d40a2da0 0x7f83d40a5260 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009fd0 tx=0x7f83e4004ab0 comp rx=0 tx=0).stop 2024-09-11T22:38:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 -- 172.21.15.5:0/2219365161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec101350 msgr2=0x7f83ec1997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec101350 0x7f83ec1997c0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f83e002a940 tx=0x7f83e0003b20 comp rx=0 tx=0).stop 2024-09-11T22:38:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 -- 172.21.15.5:0/2219365161 shutdown_connections 2024-09-11T22:38:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83ec1069c0 0x7f83ec1a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec101350 0x7f83ec1997c0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83d40a2da0 0x7f83d40a5260 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 --2- 172.21.15.5:0/2219365161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f83ec1009b0 0x7f83ec199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 -- 172.21.15.5:0/2219365161 >> 172.21.15.5:0/2219365161 conn(0x7f83ec077990 msgr2=0x7f83ec1037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 -- 172.21.15.5:0/2219365161 shutdown_connections 2024-09-11T22:38:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:41.353+0000 7f83f1b59640 1 -- 172.21.15.5:0/2219365161 wait complete. 2024-09-11T22:38:41.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51381 24.18 MB/sec execute 59 sec latency 440.361 ms 2024-09-11T22:38:41.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:41 smithi172 ceph-mon[35461]: pgmap v525: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-09-11T22:38:42.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:41 smithi005 ceph-mon[30943]: pgmap v525: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-09-11T22:38:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:41 smithi159 ceph-mon[35534]: pgmap v525: 97 pgs: 97 active+clean; 477 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.4 MiB/s wr, 604 op/s 2024-09-11T22:38:42.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51924 24.17 MB/sec execute 60 sec latency 161.087 ms 2024-09-11T22:38:43.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 52577 24.52 MB/sec execute 61 sec latency 156.146 ms 2024-09-11T22:38:43.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:43 smithi172 ceph-mon[35461]: pgmap v526: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 952 op/s 2024-09-11T22:38:43.872 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:43 smithi005 ceph-mon[30943]: pgmap v526: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 952 op/s 2024-09-11T22:38:44.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:43 smithi159 ceph-mon[35534]: pgmap v526: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.6 MiB/s wr, 952 op/s 2024-09-11T22:38:44.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53075 24.57 MB/sec execute 62 sec latency 163.991 ms 2024-09-11T22:38:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:44 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:38:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:44 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:38:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:44 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:38:44.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:44 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:38:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:44 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:38:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:44 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:38:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:44 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:38:45.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:44 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:38:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:44 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:38:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:44 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:38:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:44 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:38:45.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:44 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:38:45.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53615 24.58 MB/sec execute 63 sec latency 180.455 ms 2024-09-11T22:38:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:45 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:38:45.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:45 smithi172 ceph-mon[35461]: pgmap v527: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 584 op/s 2024-09-11T22:38:46.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:45 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:38:46.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:45 smithi005 ceph-mon[30943]: pgmap v527: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 584 op/s 2024-09-11T22:38:46.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:45 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:38:46.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:45 smithi159 ceph-mon[35534]: pgmap v527: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 584 op/s 2024-09-11T22:38:46.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53911 24.45 MB/sec execute 64 sec latency 402.280 ms 2024-09-11T22:38:47.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54320 24.36 MB/sec execute 65 sec latency 221.547 ms 2024-09-11T22:38:47.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:47 smithi172 ceph-mon[35461]: pgmap v528: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 584 op/s 2024-09-11T22:38:48.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:47 smithi005 ceph-mon[30943]: pgmap v528: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 584 op/s 2024-09-11T22:38:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:47 smithi159 ceph-mon[35534]: pgmap v528: 97 pgs: 97 active+clean; 474 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 584 op/s 2024-09-11T22:38:48.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54838 24.44 MB/sec execute 66 sec latency 81.913 ms 2024-09-11T22:38:49.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55341 24.53 MB/sec execute 67 sec latency 124.094 ms 2024-09-11T22:38:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:49 smithi005 ceph-mon[30943]: pgmap v529: 97 pgs: 97 active+clean; 488 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.6 MiB/s wr, 775 op/s 2024-09-11T22:38:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:49 smithi159 ceph-mon[35534]: pgmap v529: 97 pgs: 97 active+clean; 488 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.6 MiB/s wr, 775 op/s 2024-09-11T22:38:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:49 smithi172 ceph-mon[35461]: pgmap v529: 97 pgs: 97 active+clean; 488 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.6 MiB/s wr, 775 op/s 2024-09-11T22:38:50.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55960 24.55 MB/sec execute 68 sec latency 165.064 ms 2024-09-11T22:38:51.642 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56405 24.55 MB/sec execute 69 sec latency 216.646 ms 2024-09-11T22:38:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:51 smithi005 ceph-mon[30943]: pgmap v530: 97 pgs: 97 active+clean; 488 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.0 MiB/s wr, 538 op/s 2024-09-11T22:38:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:51 smithi159 ceph-mon[35534]: pgmap v530: 97 pgs: 97 active+clean; 488 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.0 MiB/s wr, 538 op/s 2024-09-11T22:38:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:51 smithi172 ceph-mon[35461]: pgmap v530: 97 pgs: 97 active+clean; 488 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 7.0 MiB/s wr, 538 op/s 2024-09-11T22:38:52.223 INFO:tasks.quiescer.fs.[cephfs]:Total duration (48.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-11T22:38:52.223 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.25 2024-09-11T22:38:52.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:38:52.502 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:52.643 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56872 24.61 MB/sec execute 70 sec latency 249.882 ms 2024-09-11T22:38:53.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.156+0000 7fc636071640 1 -- 172.21.15.5:0/3208813223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc630102d20 msgr2=0x7fc630103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.156+0000 7fc636071640 1 --2- 172.21.15.5:0/3208813223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc630102d20 0x7fc630103100 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fc620009900 tx=0x7fc62002a480 comp rx=0 tx=0).stop 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.157+0000 7fc636071640 1 -- 172.21.15.5:0/3208813223 shutdown_connections 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.157+0000 7fc636071640 1 --2- 172.21.15.5:0/3208813223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc63010c150 0x7fc63010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.157+0000 7fc636071640 1 --2- 172.21.15.5:0/3208813223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc630103640 0x7fc63010bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.157+0000 7fc636071640 1 --2- 172.21.15.5:0/3208813223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc630102d20 0x7fc630103100 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.157+0000 7fc636071640 1 -- 172.21.15.5:0/3208813223 >> 172.21.15.5:0/3208813223 conn(0x7fc6300fc780 msgr2=0x7fc6300feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.157+0000 7fc636071640 1 -- 172.21.15.5:0/3208813223 shutdown_connections 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.157+0000 7fc636071640 1 -- 172.21.15.5:0/3208813223 wait complete. 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.157+0000 7fc636071640 1 Processor -- start 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc636071640 1 -- start start 2024-09-11T22:38:53.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc636071640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc630102d20 0x7fc630100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc636071640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc630103640 0x7fc630100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc636071640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc63010c150 0x7fc630101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc636071640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc630111f50 con 0x7fc630102d20 2024-09-11T22:38:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc636071640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc630111dd0 con 0x7fc630103640 2024-09-11T22:38:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc636071640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc6301120d0 con 0x7fc63010c150 2024-09-11T22:38:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc62f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc630102d20 0x7fc630100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc62f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc630102d20 0x7fc630100980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57804/0 (socket says 172.21.15.5:57804) 2024-09-11T22:38:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc62f7fe640 1 -- 172.21.15.5:0/442761134 learned_addr learned my addr 172.21.15.5:0/442761134 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:53.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc62ffff640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc63010c150 0x7fc630101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc62effd640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc630103640 0x7fc630100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc62f7fe640 1 -- 172.21.15.5:0/442761134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc63010c150 msgr2=0x7fc630101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc62f7fe640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc63010c150 0x7fc630101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc62f7fe640 1 -- 172.21.15.5:0/442761134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc630103640 msgr2=0x7fc630100ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc62f7fe640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc630103640 0x7fc630100ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.158+0000 7fc62f7fe640 1 -- 172.21.15.5:0/442761134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6301a68e0 con 0x7fc630102d20 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.159+0000 7fc62f7fe640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc630102d20 0x7fc630100980 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fc620006010 tx=0x7fc6200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.159+0000 7fc62cff9640 1 -- 172.21.15.5:0/442761134 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62003f030 con 0x7fc630102d20 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.159+0000 7fc636071640 1 -- 172.21.15.5:0/442761134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6301a6b70 con 0x7fc630102d20 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.159+0000 7fc636071640 1 -- 172.21.15.5:0/442761134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6301a70a0 con 0x7fc630102d20 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.159+0000 7fc62cff9640 1 -- 172.21.15.5:0/442761134 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc620003ce0 con 0x7fc630102d20 2024-09-11T22:38:53.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.159+0000 7fc62cff9640 1 -- 172.21.15.5:0/442761134 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc620049060 con 0x7fc630102d20 2024-09-11T22:38:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.160+0000 7fc636071640 1 -- 172.21.15.5:0/442761134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61c005290 con 0x7fc630102d20 2024-09-11T22:38:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.161+0000 7fc62cff9640 1 -- 172.21.15.5:0/442761134 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc620042030 con 0x7fc630102d20 2024-09-11T22:38:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.162+0000 7fc62cff9640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6180a2cf0 0x7fc6180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.162+0000 7fc62effd640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6180a2cf0 0x7fc6180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.162+0000 7fc62cff9640 1 -- 172.21.15.5:0/442761134 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7fc6200efbb0 con 0x7fc630102d20 2024-09-11T22:38:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.162+0000 7fc62effd640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6180a2cf0 0x7fc6180a51b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc630102410 tx=0x7fc624005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:53.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.165+0000 7fc62cff9640 1 -- 172.21.15.5:0/442761134 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6200bc110 con 0x7fc630102d20 2024-09-11T22:38:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.401+0000 7fc636071640 1 -- 172.21.15.5:0/442761134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc61c005580 con 0x7fc630102d20 2024-09-11T22:38:53.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.401+0000 7fc62cff9640 1 -- 172.21.15.5:0/442761134 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11967 (secure 0 0 0) 0x7fc6200c0410 con 0x7fc630102d20 2024-09-11T22:38:53.403 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:38:53.403 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"btime":"2024-09-11T22:38:14:548820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:38:13.865471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:38:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2024-09-11T22:38:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 -- 172.21.15.5:0/442761134 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6180a2cf0 msgr2=0x7fc6180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6180a2cf0 0x7fc6180a51b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc630102410 tx=0x7fc624005f90 comp rx=0 tx=0).stop 2024-09-11T22:38:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 -- 172.21.15.5:0/442761134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc630102d20 msgr2=0x7fc630100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc630102d20 0x7fc630100980 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fc620006010 tx=0x7fc6200042f0 comp rx=0 tx=0).stop 2024-09-11T22:38:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 -- 172.21.15.5:0/442761134 shutdown_connections 2024-09-11T22:38:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc63010c150 0x7fc630101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc6180a2cf0 0x7fc6180a51b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:53.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc630103640 0x7fc630100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:53.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 --2- 172.21.15.5:0/442761134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc630102d20 0x7fc630100980 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:53.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 -- 172.21.15.5:0/442761134 >> 172.21.15.5:0/442761134 conn(0x7fc6300fc780 msgr2=0x7fc6300fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:53.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 -- 172.21.15.5:0/442761134 shutdown_connections 2024-09-11T22:38:53.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:53.404+0000 7fc636071640 1 -- 172.21.15.5:0/442761134 wait complete. 2024-09-11T22:38:53.643 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57373 24.70 MB/sec execute 71 sec latency 188.163 ms 2024-09-11T22:38:53.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:53 smithi005 ceph-mon[30943]: pgmap v531: 97 pgs: 97 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 735 op/s 2024-09-11T22:38:53.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:53 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/442761134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:38:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:53 smithi159 ceph-mon[35534]: pgmap v531: 97 pgs: 97 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 735 op/s 2024-09-11T22:38:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:53 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/442761134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:38:54.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:53 smithi172 ceph-mon[35461]: pgmap v531: 97 pgs: 97 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 10 MiB/s wr, 735 op/s 2024-09-11T22:38:54.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:53 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/442761134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:38:54.245 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0' 2024-09-11T22:38:54.245 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0 2024-09-11T22:38:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.460+0000 7f729c40a640 1 -- 172.21.15.5:0/2743884992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7294101250 msgr2=0x7f7294107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.460+0000 7f729c40a640 1 --2- 172.21.15.5:0/2743884992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7294101250 0x7f7294107990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7284009900 tx=0x7f728402a400 comp rx=0 tx=0).stop 2024-09-11T22:38:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.461+0000 7f729c40a640 1 -- 172.21.15.5:0/2743884992 shutdown_connections 2024-09-11T22:38:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.461+0000 7f729c40a640 1 --2- 172.21.15.5:0/2743884992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72941080f0 0x7f729410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.461+0000 7f729c40a640 1 --2- 172.21.15.5:0/2743884992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7294101250 0x7f7294107990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.461+0000 7f729c40a640 1 --2- 172.21.15.5:0/2743884992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72941008a0 0x7f7294100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.461+0000 7f729c40a640 1 -- 172.21.15.5:0/2743884992 >> 172.21.15.5:0/2743884992 conn(0x7f7294075ef0 msgr2=0x7f7294076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.461+0000 7f729c40a640 1 -- 172.21.15.5:0/2743884992 shutdown_connections 2024-09-11T22:38:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.461+0000 7f729c40a640 1 -- 172.21.15.5:0/2743884992 wait complete. 2024-09-11T22:38:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.462+0000 7f729c40a640 1 Processor -- start 2024-09-11T22:38:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.462+0000 7f729c40a640 1 -- start start 2024-09-11T22:38:54.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f729c40a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72941008a0 0x7f7294198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f729c40a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72941080f0 0x7f7294198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f729a1a6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72941008a0 0x7f7294198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f729c40a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f729419cc00 0x7f729419d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f729a1a6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72941008a0 0x7f7294198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52490/0 (socket says 172.21.15.5:52490) 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f72999a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72941080f0 0x7f7294198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f72999a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72941080f0 0x7f7294198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57816/0 (socket says 172.21.15.5:57816) 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f729a9a7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f729419cc00 0x7f729419d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f729a1a6640 1 -- 172.21.15.5:0/3655906535 learned_addr learned my addr 172.21.15.5:0/3655906535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7294111f70 con 0x7f72941080f0 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7294111df0 con 0x7f72941008a0 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.463+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f72941120f0 con 0x7f729419cc00 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.464+0000 7f729a9a7640 1 -- 172.21.15.5:0/3655906535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72941008a0 msgr2=0x7f7294198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.464+0000 7f729a9a7640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72941008a0 0x7f7294198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.464+0000 7f729a9a7640 1 -- 172.21.15.5:0/3655906535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72941080f0 msgr2=0x7f7294198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.464+0000 7f729a9a7640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72941080f0 0x7f7294198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.464+0000 7f729a9a7640 1 -- 172.21.15.5:0/3655906535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729419d6f0 con 0x7f729419cc00 2024-09-11T22:38:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.464+0000 7f729a9a7640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f729419cc00 0x7f729419d0b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f728c009fd0 tx=0x7f728c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.464+0000 7f728b7fe640 1 -- 172.21.15.5:0/3655906535 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c00ce20 con 0x7f729419cc00 2024-09-11T22:38:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.464+0000 7f728b7fe640 1 -- 172.21.15.5:0/3655906535 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f728c005020 con 0x7f729419cc00 2024-09-11T22:38:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.465+0000 7f728b7fe640 1 -- 172.21.15.5:0/3655906535 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c010720 con 0x7f729419cc00 2024-09-11T22:38:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.465+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729419d9e0 con 0x7f729419cc00 2024-09-11T22:38:54.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.465+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72941a50f0 con 0x7f729419cc00 2024-09-11T22:38:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.468+0000 7f729f277740 1 Processor -- start 2024-09-11T22:38:54.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.469+0000 7f728b7fe640 1 -- 172.21.15.5:0/3655906535 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f728c0051d0 con 0x7f729419cc00 2024-09-11T22:38:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.469+0000 7f729f277740 1 -- start start 2024-09-11T22:38:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.469+0000 7f729f277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e2230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.469+0000 7f729f277740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e21e660 0x55bb9e21eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.469+0000 7f729f277740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 0x55bb9e22a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.469+0000 7f729f277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb9e1792f0 con 0x55bb9e222cc0 2024-09-11T22:38:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.469+0000 7f729f277740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bb9e121270 con 0x55bb9e21e660 2024-09-11T22:38:54.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.469+0000 7f729f277740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bb9e1752b0 con 0x55bb9e21f000 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.469+0000 7f72999a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e2230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f72999a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e2230a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57824/0 (socket says 172.21.15.5:57824) 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f72999a5640 1 -- 172.21.15.5:0/2107369949 learned_addr learned my addr 172.21.15.5:0/2107369949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f72999a5640 1 -- 172.21.15.5:0/2107369949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 msgr2=0x55bb9e22a9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f72999a5640 1 --2- 172.21.15.5:0/2107369949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 0x55bb9e22a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f72999a5640 1 -- 172.21.15.5:0/2107369949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e21e660 msgr2=0x55bb9e21eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f72999a5640 1 --2- 172.21.15.5:0/2107369949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e21e660 0x55bb9e21eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f728b7fe640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72800a2da0 0x7f72800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f72999a5640 1 -- 172.21.15.5:0/2107369949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9e120340 con 0x55bb9e222cc0 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f729a9a7640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72800a2da0 0x7f72800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f728b7fe640 1 -- 172.21.15.5:0/3655906535 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f728c0cfbc0 con 0x7f729419cc00 2024-09-11T22:38:54.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.470+0000 7f72999a5640 1 --2- 172.21.15.5:0/2107369949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e2230a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7284009900 tx=0x7f7284005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37678e9a021c8fb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.471+0000 7f7288bfe640 1 -- 172.21.15.5:0/2107369949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284035040 con 0x55bb9e222cc0 2024-09-11T22:38:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.471+0000 7f729a9a7640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72800a2da0 0x7f72800a5260 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f728c09bd10 tx=0x7f728c066c40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.471+0000 7f7288bfe640 1 -- 172.21.15.5:0/2107369949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284005520 con 0x55bb9e222cc0 2024-09-11T22:38:54.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.471+0000 7f7288bfe640 1 -- 172.21.15.5:0/2107369949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7284004e20 con 0x55bb9e222cc0 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.472+0000 7f729f277740 1 -- 172.21.15.5:0/2107369949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 msgr2=0x55bb9e2230a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.472+0000 7f729f277740 1 --2- 172.21.15.5:0/2107369949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e2230a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f7284009900 tx=0x7f7284005de0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.472+0000 7f729f277740 1 -- 172.21.15.5:0/2107369949 shutdown_connections 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.472+0000 7f729f277740 1 --2- 172.21.15.5:0/2107369949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 0x55bb9e22a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.472+0000 7f729f277740 1 --2- 172.21.15.5:0/2107369949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e21e660 0x55bb9e21eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.472+0000 7f729f277740 1 --2- 172.21.15.5:0/2107369949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e2230a0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.472+0000 7f729f277740 1 -- 172.21.15.5:0/2107369949 >> 172.21.15.5:0/2107369949 conn(0x55bb9e210b10 msgr2=0x55bb9e210ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.473+0000 7f729f277740 1 -- 172.21.15.5:0/2107369949 shutdown_connections 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.473+0000 7f729f277740 1 -- 172.21.15.5:0/2107369949 wait complete. 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.473+0000 7f729f277740 1 -- start start 2024-09-11T22:38:54.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.474+0000 7f729f277740 1 Processor -- start 2024-09-11T22:38:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.474+0000 7f729f277740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 0x55bb9e227e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.474+0000 7f729f277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 0x55bb9e2283d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.474+0000 7f729f277740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e222cc0 0x55bb9e228910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.474+0000 7f729f277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb9e22b050 con 0x55bb9e21f000 2024-09-11T22:38:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.474+0000 7f729f277740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bb9e102290 con 0x55bb9e222cc0 2024-09-11T22:38:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.474+0000 7f729f277740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55bb9e22b1d0 con 0x55bb9e21e660 2024-09-11T22:38:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.474+0000 7f729f277740 10 client.0 _ll_register_callbacks cb 0x55bb9e133140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f72999a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 0x55bb9e227e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f72999a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 0x55bb9e227e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:58214/0 (socket says 172.21.15.5:58214) 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f72999a5640 1 -- 172.21.15.5:0/1165375851 learned_addr learned my addr 172.21.15.5:0/1165375851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f72999a5640 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e222cc0 0x55bb9e228910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f729a1a6640 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 0x55bb9e2283d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f72999a5640 1 -- 172.21.15.5:0/1165375851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e222cc0 msgr2=0x55bb9e228910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f72999a5640 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e222cc0 0x55bb9e228910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f72999a5640 1 -- 172.21.15.5:0/1165375851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 msgr2=0x55bb9e2283d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f72999a5640 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 0x55bb9e2283d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f72999a5640 1 -- 172.21.15.5:0/1165375851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb9de8c210 con 0x55bb9e21e660 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f72999a5640 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 0x55bb9e227e90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7284006010 tx=0x7f7284004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d65fe43b8985d815 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f727dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.475+0000 7f727dffb640 1 -- 172.21.15.5:0/1165375851 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728404d040 con 0x55bb9e21e660 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f729f277740 1 -- 172.21.15.5:0/1165375851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7284005520 con 0x55bb9e21e660 2024-09-11T22:38:54.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f727dffb640 1 -- 172.21.15.5:0/1165375851 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f7284004100 con 0x55bb9e21e660 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f727dffb640 1 -- 172.21.15.5:0/1165375851 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7284004300 con 0x55bb9e21e660 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f729f277740 10 client.29455 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f729f277740 1 -- 172.21.15.5:0/1165375851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55bb9df58620 con 0x55bb9e21e660 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f727dffb640 1 -- 172.21.15.5:0/1165375851 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7284005520 con 0x55bb9e21e660 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f727dffb640 1 -- 172.21.15.5:0/1165375851 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f72840315b0 con 0x55bb9e21e660 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f729f277740 10 client.29455 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f729f277740 10 client.29455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f729f277740 20 client.29455 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f729f277740 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bb9e238380 0x55bb9e23a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f729f277740 4 client.29455 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f729f277740 1 -- 172.21.15.5:0/1165375851 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb9e186a20 con 0x55bb9e238380 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.476+0000 7f729a1a6640 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bb9e238380 0x55bb9e23a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.477+0000 7f729a1a6640 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bb9e238380 0x55bb9e23a770 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c8a660b124c378 server_cookie=505fc1c83bd8b4da in_seq=0 out_seq=0 2024-09-11T22:38:54.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.477+0000 7f727dffb640 10 client.29455 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f727dffb640 1 -- 172.21.15.5:0/1165375851 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb9e186a20 con 0x55bb9e238380 2024-09-11T22:38:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f727dffb640 10 client.29455 handle_command_reply: tid=0 2024-09-11T22:38:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 client.29455 shutdown 2024-09-11T22:38:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 -- 172.21.15.5:0/1165375851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 msgr2=0x55bb9e227e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 0x55bb9e227e90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7284006010 tx=0x7f7284004a00 comp rx=0 tx=0).stop 2024-09-11T22:38:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 -- 172.21.15.5:0/1165375851 shutdown_connections 2024-09-11T22:38:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bb9e238380 0x55bb9e23a770 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e222cc0 0x55bb9e228910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 0x55bb9e2283d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f727dffb640 0 client.29455 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 --2- 172.21.15.5:0/1165375851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 0x55bb9e227e90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 -- 172.21.15.5:0/1165375851 >> 172.21.15.5:0/1165375851 conn(0x55bb9e210b10 msgr2=0x55bb9e21de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 -- 172.21.15.5:0/1165375851 shutdown_connections 2024-09-11T22:38:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 1 -- 172.21.15.5:0/1165375851 wait complete. 2024-09-11T22:38:54.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.479+0000 7f729f277740 20 client.29455 trim_cache size 0 max 16384 2024-09-11T22:38:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729f277740 1 Processor -- start 2024-09-11T22:38:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729f277740 1 -- start start 2024-09-11T22:38:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729f277740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 0x55bb9e226740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729f277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 0x55bb9e23d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729f277740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e222cc0 0x55bb9e23dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729f277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72941ab6d0 con 0x55bb9e21f000 2024-09-11T22:38:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729f277740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72941ad210 con 0x55bb9e222cc0 2024-09-11T22:38:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729f277740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f72941abda0 con 0x55bb9e21e660 2024-09-11T22:38:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729a1a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 0x55bb9e23d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729a1a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 0x55bb9e23d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57832/0 (socket says 172.21.15.5:57832) 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f729a1a6640 1 -- 172.21.15.5:0/1105505949 learned_addr learned my addr 172.21.15.5:0/1105505949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f72999a5640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 0x55bb9e226740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.499+0000 7f72999a5640 1 --2- 172.21.15.5:0/1105505949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e222cc0 0x55bb9e23dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.500+0000 7f729a1a6640 1 -- 172.21.15.5:0/1105505949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 msgr2=0x55bb9e226740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.500+0000 7f729a1a6640 1 --2- 172.21.15.5:0/1105505949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 0x55bb9e226740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.500+0000 7f729a1a6640 1 -- 172.21.15.5:0/1105505949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e222cc0 msgr2=0x55bb9e23dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.500+0000 7f729a1a6640 1 --2- 172.21.15.5:0/1105505949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e222cc0 0x55bb9e23dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.500+0000 7f729a1a6640 1 -- 172.21.15.5:0/1105505949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb9e21f3e0 con 0x55bb9e21f000 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.500+0000 7f729a1a6640 1 --2- 172.21.15.5:0/1105505949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 0x55bb9e23d790 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7290009620 tx=0x7f72900108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb3921c4faf9014 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.500+0000 7f727ffff640 1 -- 172.21.15.5:0/1105505949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290020030 con 0x55bb9e21f000 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.500+0000 7f727ffff640 1 -- 172.21.15.5:0/1105505949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7290005500 con 0x55bb9e21f000 2024-09-11T22:38:54.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.500+0000 7f727ffff640 1 -- 172.21.15.5:0/1105505949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729001c6a0 con 0x55bb9e21f000 2024-09-11T22:38:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.501+0000 7f729f277740 1 -- 172.21.15.5:0/1105505949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 msgr2=0x55bb9e23d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.501+0000 7f729f277740 1 --2- 172.21.15.5:0/1105505949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 0x55bb9e23d790 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f7290009620 tx=0x7f72900108c0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.501+0000 7f729f277740 1 -- 172.21.15.5:0/1105505949 shutdown_connections 2024-09-11T22:38:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.501+0000 7f729f277740 1 --2- 172.21.15.5:0/1105505949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e222cc0 0x55bb9e23dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.501+0000 7f729f277740 1 --2- 172.21.15.5:0/1105505949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e21f000 0x55bb9e23d790 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.501+0000 7f729f277740 1 --2- 172.21.15.5:0/1105505949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21e660 0x55bb9e226740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.501+0000 7f729f277740 1 -- 172.21.15.5:0/1105505949 >> 172.21.15.5:0/1105505949 conn(0x55bb9e210b10 msgr2=0x55bb9e21de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.501+0000 7f729f277740 1 -- 172.21.15.5:0/1105505949 shutdown_connections 2024-09-11T22:38:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.501+0000 7f729f277740 1 -- 172.21.15.5:0/1105505949 wait complete. 2024-09-11T22:38:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.501+0000 7f729f277740 1 -- start start 2024-09-11T22:38:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.503+0000 7f729f277740 1 Processor -- start 2024-09-11T22:38:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.503+0000 7f729f277740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e21e660 0x55bb9e23b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.503+0000 7f729f277740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 0x55bb9e23b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.503+0000 7f729f277740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e23bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.503+0000 7f729f277740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72941acb40 con 0x55bb9e222cc0 2024-09-11T22:38:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.503+0000 7f729f277740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72941ac470 con 0x55bb9e21e660 2024-09-11T22:38:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.503+0000 7f729f277740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f72941a96d0 con 0x55bb9e21f000 2024-09-11T22:38:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.503+0000 7f729f277740 10 client.0 _ll_register_callbacks cb 0x7f7284009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.503+0000 7f729a1a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 0x55bb9e23b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.504+0000 7f729a1a6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 0x55bb9e23b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:58236/0 (socket says 172.21.15.5:58236) 2024-09-11T22:38:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.504+0000 7f729a1a6640 1 -- 172.21.15.5:0/486657911 learned_addr learned my addr 172.21.15.5:0/486657911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.504+0000 7f72999a5640 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e23bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.504+0000 7f72999a5640 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e21e660 0x55bb9e23b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.504+0000 7f729a1a6640 1 -- 172.21.15.5:0/486657911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e21e660 msgr2=0x55bb9e23b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.504+0000 7f729a1a6640 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e21e660 0x55bb9e23b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.504+0000 7f729a1a6640 1 -- 172.21.15.5:0/486657911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 msgr2=0x55bb9e23bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.504+0000 7f729a1a6640 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e23bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.504+0000 7f729a1a6640 1 -- 172.21.15.5:0/486657911 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb9e2a1cc0 con 0x55bb9e21f000 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.505+0000 7f72999a5640 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e23bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.505+0000 7f729a1a6640 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 0x55bb9e23b980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7290020000 tx=0x7f72900056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374d5df493720233 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.505+0000 7f727dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.505+0000 7f727dffb640 1 -- 172.21.15.5:0/486657911 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290029070 con 0x55bb9e21f000 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.506+0000 7f727dffb640 1 -- 172.21.15.5:0/486657911 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f7290004730 con 0x55bb9e21f000 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.506+0000 7f729f277740 1 -- 172.21.15.5:0/486657911 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7290005500 con 0x55bb9e21f000 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.506+0000 7f727dffb640 1 -- 172.21.15.5:0/486657911 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7290004950 con 0x55bb9e21f000 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.506+0000 7f729f277740 10 client.29461 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:54.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.506+0000 7f729f277740 1 -- 172.21.15.5:0/486657911 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55bb9e2a2ef0 con 0x55bb9e21f000 2024-09-11T22:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.506+0000 7f727dffb640 1 -- 172.21.15.5:0/486657911 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7290005500 con 0x55bb9e21f000 2024-09-11T22:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.507+0000 7f727dffb640 1 -- 172.21.15.5:0/486657911 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f72900254d0 con 0x55bb9e21f000 2024-09-11T22:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.507+0000 7f729f277740 10 client.29461 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.507+0000 7f729f277740 10 client.29461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.507+0000 7f729f277740 20 client.29461 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.507+0000 7f729f277740 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bb9e2a3630 0x55bb9e2a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.507+0000 7f729f277740 4 client.29461 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:38:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.507+0000 7f729f277740 1 -- 172.21.15.5:0/486657911 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb9e180d70 con 0x55bb9e2a3630 2024-09-11T22:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.507+0000 7f72999a5640 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bb9e2a3630 0x55bb9e2a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.508+0000 7f727dffb640 10 client.29461 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:54.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:54.508+0000 7f72999a5640 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bb9e2a3630 0x55bb9e2a5a50 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f47f6fc7e314e49 server_cookie=3b80230044f41ffa in_seq=0 out_seq=0 2024-09-11T22:38:54.553 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:38:54.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57795 24.68 MB/sec execute 72 sec latency 284.420 ms 2024-09-11T22:38:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.002+0000 7f727dffb640 1 -- 172.21.15.5:0/486657911 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb9e180d70 con 0x55bb9e2a3630 2024-09-11T22:38:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.002+0000 7f727dffb640 10 client.29461 handle_command_reply: tid=0 2024-09-11T22:38:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 client.29461 shutdown 2024-09-11T22:38:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 -- 172.21.15.5:0/486657911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 msgr2=0x55bb9e23b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 0x55bb9e23b980 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7290020000 tx=0x7f72900056b0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 -- 172.21.15.5:0/486657911 shutdown_connections 2024-09-11T22:38:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55bb9e2a3630 0x55bb9e2a5a50 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb9e222cc0 0x55bb9e23bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bb9e21f000 0x55bb9e23b980 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f727dffb640 0 client.29461 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 --2- 172.21.15.5:0/486657911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb9e21e660 0x55bb9e23b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 -- 172.21.15.5:0/486657911 >> 172.21.15.5:0/486657911 conn(0x55bb9e210b10 msgr2=0x55bb9e2334f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 -- 172.21.15.5:0/486657911 shutdown_connections 2024-09-11T22:38:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.003+0000 7f729f277740 1 -- 172.21.15.5:0/486657911 wait complete. 2024-09-11T22:38:55.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729f277740 20 client.29461 trim_cache size 0 max 16384 2024-09-11T22:38:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:38:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:38:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:38:55.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:38:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72800a2da0 msgr2=0x7f72800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729c40a640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72800a2da0 0x7f72800a5260 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f728c09bd10 tx=0x7f728c066c40 comp rx=0 tx=0).stop 2024-09-11T22:38:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f729419cc00 msgr2=0x7f729419d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729c40a640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f729419cc00 0x7f729419d0b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f728c009fd0 tx=0x7f728c00ef10 comp rx=0 tx=0).stop 2024-09-11T22:38:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 shutdown_connections 2024-09-11T22:38:55.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729c40a640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f729419cc00 0x7f729419d0b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729c40a640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72941080f0 0x7f7294198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729c40a640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f72800a2da0 0x7f72800a5260 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729c40a640 1 --2- 172.21.15.5:0/3655906535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72941008a0 0x7f7294198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.004+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 >> 172.21.15.5:0/3655906535 conn(0x7f7294075ef0 msgr2=0x7f72940ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.005+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 shutdown_connections 2024-09-11T22:38:55.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:38:55.005+0000 7f729c40a640 1 -- 172.21.15.5:0/3655906535 wait complete. 2024-09-11T22:38:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.123+0000 7fbcb537e640 1 -- 172.21.15.5:0/2841116640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcb0102570 msgr2=0x7fbcb01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.123+0000 7fbcb537e640 1 --2- 172.21.15.5:0/2841116640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcb0102570 0x7fbcb01029b0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009900 tx=0x7fbca402a480 comp rx=0 tx=0).stop 2024-09-11T22:38:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.123+0000 7fbcb537e640 1 -- 172.21.15.5:0/2841116640 shutdown_connections 2024-09-11T22:38:55.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.123+0000 7fbcb537e640 1 --2- 172.21.15.5:0/2841116640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb0102ef0 0x7fbcb010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.123+0000 7fbcb537e640 1 --2- 172.21.15.5:0/2841116640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcb0102570 0x7fbcb01029b0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.123+0000 7fbcb537e640 1 --2- 172.21.15.5:0/2841116640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb0100e20 0x7fbcb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.123+0000 7fbcb537e640 1 -- 172.21.15.5:0/2841116640 >> 172.21.15.5:0/2841116640 conn(0x7fbcb00fc7d0 msgr2=0x7fbcb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.123+0000 7fbcb537e640 1 -- 172.21.15.5:0/2841116640 shutdown_connections 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.123+0000 7fbcb537e640 1 -- 172.21.15.5:0/2841116640 wait complete. 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcb537e640 1 Processor -- start 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcb537e640 1 -- start start 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcb537e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb0100e20 0x7fbcb00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcb537e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcb0102570 0x7fbcb00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcb537e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb0102ef0 0x7fbcb00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcb537e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb0110f50 con 0x7fbcb0102570 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcb537e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb0110dd0 con 0x7fbcb0100e20 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcb537e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01110d0 con 0x7fbcb0102ef0 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcb0102570 0x7fbcb00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcaeffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb0100e20 0x7fbcb00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcb0102570 0x7fbcb00fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57852/0 (socket says 172.21.15.5:57852) 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcae7fc640 1 -- 172.21.15.5:0/1871817564 learned_addr learned my addr 172.21.15.5:0/1871817564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.124+0000 7fbcaf7fe640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb0102ef0 0x7fbcb00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbcae7fc640 1 -- 172.21.15.5:0/1871817564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb0102ef0 msgr2=0x7fbcb00fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb0102ef0 0x7fbcb00fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbcae7fc640 1 -- 172.21.15.5:0/1871817564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb0100e20 msgr2=0x7fbcb00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb0100e20 0x7fbcb00fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbcae7fc640 1 -- 172.21.15.5:0/1871817564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb01a6a60 con 0x7fbcb0102570 2024-09-11T22:38:55.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbcaf7fe640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb0102ef0 0x7fbcb00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcb0102570 0x7fbcb00fd1c0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fbca4006010 tx=0x7fbca4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbc97fff640 1 -- 172.21.15.5:0/1871817564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca403f070 con 0x7fbcb0102570 2024-09-11T22:38:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbcb537e640 1 -- 172.21.15.5:0/1871817564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb01a6cf0 con 0x7fbcb0102570 2024-09-11T22:38:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbc97fff640 1 -- 172.21.15.5:0/1871817564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca4031c30 con 0x7fbcb0102570 2024-09-11T22:38:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbc97fff640 1 -- 172.21.15.5:0/1871817564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4035030 con 0x7fbcb0102570 2024-09-11T22:38:55.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.125+0000 7fbcb537e640 1 -- 172.21.15.5:0/1871817564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb01a71c0 con 0x7fbcb0102570 2024-09-11T22:38:55.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.128+0000 7fbc97fff640 1 -- 172.21.15.5:0/1871817564 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbca4042070 con 0x7fbcb0102570 2024-09-11T22:38:55.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcb7757740 1 Processor -- start 2024-09-11T22:38:55.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcb7757740 1 -- start start 2024-09-11T22:38:55.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf05dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf0684f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf06ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcb7757740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4cefc2160 con 0x55f4cf05e170 2024-09-11T22:38:55.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcb7757740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4cef69720 con 0x55f4cf05d850 2024-09-11T22:38:55.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcb7757740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4cefbe120 con 0x55f4cf068a30 2024-09-11T22:38:55.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf0684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcaf7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf05dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcaf7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf05dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52556/0 (socket says 172.21.15.5:52556) 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf0684f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57860/0 (socket says 172.21.15.5:57860) 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcaf7fe640 1 -- 172.21.15.5:0/4268318357 learned_addr learned my addr 172.21.15.5:0/4268318357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/4268318357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf06ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbc97fff640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc980a2cf0 0x7fbc980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcaeffd640 1 -- 172.21.15.5:0/4268318357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 msgr2=0x55f4cf06ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcaeffd640 1 --2- 172.21.15.5:0/4268318357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf06ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcaeffd640 1 -- 172.21.15.5:0/4268318357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 msgr2=0x55f4cf05dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcaeffd640 1 --2- 172.21.15.5:0/4268318357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf05dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.129+0000 7fbcaeffd640 1 -- 172.21.15.5:0/4268318357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ced33ba0 con 0x55f4cf05e170 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.130+0000 7fbcaf7fe640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc980a2cf0 0x7fbc980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.130+0000 7fbc97fff640 1 -- 172.21.15.5:0/1871817564 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7fbca40efaf0 con 0x7fbcb0102570 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.130+0000 7fbcaeffd640 1 --2- 172.21.15.5:0/4268318357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf0684f0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009c20 tx=0x7fbca0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0477d9909f25b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.130+0000 7fbc957fa640 1 -- 172.21.15.5:0/4268318357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca001e070 con 0x55f4cf05e170 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.130+0000 7fbc957fa640 1 -- 172.21.15.5:0/4268318357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca0009dd0 con 0x55f4cf05e170 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.130+0000 7fbc957fa640 1 -- 172.21.15.5:0/4268318357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0018870 con 0x55f4cf05e170 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 -- 172.21.15.5:0/4268318357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 msgr2=0x55f4cf0684f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcaf7fe640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc980a2cf0 0x7fbc980a51b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009fd0 tx=0x7fbca8004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 --2- 172.21.15.5:0/4268318357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf0684f0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009c20 tx=0x7fbca0011810 comp rx=0 tx=0).stop 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 -- 172.21.15.5:0/4268318357 shutdown_connections 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 --2- 172.21.15.5:0/4268318357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf06ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 --2- 172.21.15.5:0/4268318357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf0684f0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 --2- 172.21.15.5:0/4268318357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf05dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 -- 172.21.15.5:0/4268318357 >> 172.21.15.5:0/4268318357 conn(0x55f4cf059de0 msgr2=0x55f4cf05a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:55.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 -- 172.21.15.5:0/4268318357 shutdown_connections 2024-09-11T22:38:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 -- 172.21.15.5:0/4268318357 wait complete. 2024-09-11T22:38:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 -- start start 2024-09-11T22:38:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.131+0000 7fbcb7757740 1 Processor -- start 2024-09-11T22:38:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf0658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf05e170 0x55f4cf065e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf068a30 0x55f4cf066360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcb7757740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4cf06b600 con 0x55f4cf068a30 2024-09-11T22:38:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcb7757740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4cef4a900 con 0x55f4cf05d850 2024-09-11T22:38:55.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcb7757740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55f4cf06b780 con 0x55f4cf05e170 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcb7757740 10 client.0 _ll_register_callbacks cb 0x55f4cef48780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcaeffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf0658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcae7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf05e170 0x55f4cf065e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcaeffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf0658e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52566/0 (socket says 172.21.15.5:52566) 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcaeffd640 1 -- 172.21.15.5:0/81346747 learned_addr learned my addr 172.21.15.5:0/81346747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcaf7fe640 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf068a30 0x55f4cf066360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcae7fc640 1 -- 172.21.15.5:0/81346747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 msgr2=0x55f4cf0658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf0658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcae7fc640 1 -- 172.21.15.5:0/81346747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf068a30 msgr2=0x55f4cf066360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf068a30 0x55f4cf066360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcae7fc640 1 -- 172.21.15.5:0/81346747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4cef8ac50 con 0x55f4cf05e170 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcaeffd640 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf0658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:55.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf05e170 0x55f4cf065e20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbca40ed1a0 tx=0x7fbca40f3230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dd6157dcce36955 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.132+0000 7fbc92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbc92ff5640 1 -- 172.21.15.5:0/81346747 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4049070 con 0x55f4cf05e170 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbc92ff5640 1 -- 172.21.15.5:0/81346747 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7fbca40ed450 con 0x55f4cf05e170 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbcb7757740 1 -- 172.21.15.5:0/81346747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbca0009dd0 con 0x55f4cf05e170 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbc92ff5640 1 -- 172.21.15.5:0/81346747 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fbca40bbe20 con 0x55f4cf05e170 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbcb7757740 10 client.29473 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbcb7757740 1 -- 172.21.15.5:0/81346747 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f4cf06b600 con 0x55f4cf05e170 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbc92ff5640 1 -- 172.21.15.5:0/81346747 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca4031de0 con 0x55f4cf05e170 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbc92ff5640 1 -- 172.21.15.5:0/81346747 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fbca40bc2b0 con 0x55f4cf05e170 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbcb7757740 10 client.29473 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbcb7757740 10 client.29473 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbcb7757740 20 client.29473 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:55.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbcb7757740 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f4cf078650 0x55f4cf07aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbcb7757740 4 client.29473 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:38:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbcb7757740 1 -- 172.21.15.5:0/81346747 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbca0009dd0 con 0x55f4cf078650 2024-09-11T22:38:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.133+0000 7fbcaeffd640 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f4cf078650 0x55f4cf07aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.134+0000 7fbc92ff5640 10 client.29473 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:55.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.134+0000 7fbcaeffd640 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f4cf078650 0x55f4cf07aa70 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d759cd27340d548 server_cookie=ca2aaa0bc55c69af in_seq=0 out_seq=0 2024-09-11T22:38:55.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbc92ff5640 1 -- 172.21.15.5:0/81346747 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fbca0009dd0 con 0x55f4cf078650 2024-09-11T22:38:55.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbc92ff5640 10 client.29473 handle_command_reply: tid=0 2024-09-11T22:38:55.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbcb7757740 1 client.29473 shutdown 2024-09-11T22:38:55.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbcb7757740 1 -- 172.21.15.5:0/81346747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf05e170 msgr2=0x55f4cf065e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:55.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbcb7757740 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf05e170 0x55f4cf065e20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbca40ed1a0 tx=0x7fbca40f3230 comp rx=0 tx=0).stop 2024-09-11T22:38:55.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbcb7757740 1 -- 172.21.15.5:0/81346747 shutdown_connections 2024-09-11T22:38:55.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbcb7757740 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f4cf078650 0x55f4cf07aa70 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbcb7757740 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf068a30 0x55f4cf066360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbcb7757740 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf05e170 0x55f4cf065e20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbcb7757740 1 --2- 172.21.15.5:0/81346747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf0658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbc92ff5640 0 client.29473 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:55.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.137+0000 7fbcb7757740 1 -- 172.21.15.5:0/81346747 >> 172.21.15.5:0/81346747 conn(0x55f4cf059de0 msgr2=0x55f4cf05fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:55.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.138+0000 7fbcb7757740 1 -- 172.21.15.5:0/81346747 shutdown_connections 2024-09-11T22:38:55.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.138+0000 7fbcb7757740 1 -- 172.21.15.5:0/81346747 wait complete. 2024-09-11T22:38:55.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.138+0000 7fbcb7757740 20 client.29473 trim_cache size 0 max 16384 2024-09-11T22:38:55.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcb7757740 1 Processor -- start 2024-09-11T22:38:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcb7757740 1 -- start start 2024-09-11T22:38:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf05a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf05f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf080360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcb7757740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01afcf0 con 0x55f4cf05e170 2024-09-11T22:38:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcb7757740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01b1830 con 0x55f4cf05d850 2024-09-11T22:38:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcb7757740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01b03c0 con 0x55f4cf068a30 2024-09-11T22:38:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf05f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcaeffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf05a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf05f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57878/0 (socket says 172.21.15.5:57878) 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcae7fc640 1 -- 172.21.15.5:0/329818417 learned_addr learned my addr 172.21.15.5:0/329818417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.158+0000 7fbcaf7fe640 1 --2- 172.21.15.5:0/329818417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf080360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbcaf7fe640 1 -- 172.21.15.5:0/329818417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 msgr2=0x55f4cf05a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbcaf7fe640 1 --2- 172.21.15.5:0/329818417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf05a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbcaf7fe640 1 -- 172.21.15.5:0/329818417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 msgr2=0x55f4cf05f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbcaf7fe640 1 --2- 172.21.15.5:0/329818417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf05f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbcaf7fe640 1 -- 172.21.15.5:0/329818417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4cf064890 con 0x55f4cf068a30 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/329818417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf05f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbcaeffd640 1 --2- 172.21.15.5:0/329818417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf05a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:55.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbcaf7fe640 1 --2- 172.21.15.5:0/329818417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf080360 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbca800f790 tx=0x7fbca801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7073ed4bd424391 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:55.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbc94ff9640 1 -- 172.21.15.5:0/329818417 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca800fc50 con 0x55f4cf068a30 2024-09-11T22:38:55.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbc94ff9640 1 -- 172.21.15.5:0/329818417 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca802d070 con 0x55f4cf068a30 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.159+0000 7fbc94ff9640 1 -- 172.21.15.5:0/329818417 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca800e610 con 0x55f4cf068a30 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.160+0000 7fbcb7757740 1 -- 172.21.15.5:0/329818417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 msgr2=0x55f4cf080360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.160+0000 7fbcb7757740 1 --2- 172.21.15.5:0/329818417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf080360 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbca800f790 tx=0x7fbca801a000 comp rx=0 tx=0).stop 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.160+0000 7fbcb7757740 1 -- 172.21.15.5:0/329818417 shutdown_connections 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.160+0000 7fbcb7757740 1 --2- 172.21.15.5:0/329818417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf080360 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.160+0000 7fbcb7757740 1 --2- 172.21.15.5:0/329818417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05e170 0x55f4cf05f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.160+0000 7fbcb7757740 1 --2- 172.21.15.5:0/329818417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05d850 0x55f4cf05a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.160+0000 7fbcb7757740 1 -- 172.21.15.5:0/329818417 >> 172.21.15.5:0/329818417 conn(0x55f4cf059de0 msgr2=0x55f4cf05f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.160+0000 7fbcb7757740 1 -- 172.21.15.5:0/329818417 shutdown_connections 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.160+0000 7fbcb7757740 1 -- 172.21.15.5:0/329818417 wait complete. 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.160+0000 7fbcb7757740 1 -- start start 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcb7757740 1 Processor -- start 2024-09-11T22:38:55.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05d850 0x55f4cf070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05e170 0x55f4cf06d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcb7757740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcb7757740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01b1160 con 0x55f4cf05d850 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcb7757740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01b0a90 con 0x55f4cf05e170 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcb7757740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb01adcf0 con 0x55f4cf068a30 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcb7757740 10 client.0 _ll_register_callbacks cb 0x7fbca40f3330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05d850 0x55f4cf070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05d850 0x55f4cf070ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57884/0 (socket says 172.21.15.5:57884) 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcaeffd640 1 -- 172.21.15.5:0/1475509401 learned_addr learned my addr 172.21.15.5:0/1475509401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05e170 0x55f4cf06d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.161+0000 7fbcaf7fe640 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbcae7fc640 1 -- 172.21.15.5:0/1475509401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 msgr2=0x55f4cf06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf06dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbcae7fc640 1 -- 172.21.15.5:0/1475509401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05d850 msgr2=0x55f4cf070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05d850 0x55f4cf070ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbcae7fc640 1 -- 172.21.15.5:0/1475509401 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4cf06e4a0 con 0x55f4cf05e170 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbcaeffd640 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05d850 0x55f4cf070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbcae7fc640 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05e170 0x55f4cf06d9a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbca40bbd50 tx=0x7fbca403b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f8b684294a6deca server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbc92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbc92ff5640 1 -- 172.21.15.5:0/1475509401 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4049040 con 0x55f4cf05e170 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbc92ff5640 1 -- 172.21.15.5:0/1475509401 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7fbca40c4850 con 0x55f4cf05e170 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbcb7757740 1 -- 172.21.15.5:0/1475509401 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbca802d070 con 0x55f4cf05e170 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbc92ff5640 1 -- 172.21.15.5:0/1475509401 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fbca40bb9b0 con 0x55f4cf05e170 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbcb7757740 10 client.29645 fetch_fsmap learned FSMap version 54 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.162+0000 7fbcb7757740 1 -- 172.21.15.5:0/1475509401 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f4cf06e4a0 con 0x55f4cf05e170 2024-09-11T22:38:55.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.163+0000 7fbc92ff5640 1 -- 172.21.15.5:0/1475509401 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbca802d070 con 0x55f4cf05e170 2024-09-11T22:38:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.163+0000 7fbc92ff5640 1 -- 172.21.15.5:0/1475509401 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fbca40b6bc0 con 0x55f4cf05e170 2024-09-11T22:38:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.163+0000 7fbcb7757740 10 client.29645 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:38:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.163+0000 7fbcb7757740 10 client.29645 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:38:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.163+0000 7fbcb7757740 20 client.29645 populate_metadata read hostname 'smithi005' 2024-09-11T22:38:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.163+0000 7fbcb7757740 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f4cf0e4520 0x55f4cf0e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:38:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.163+0000 7fbcb7757740 4 client.29645 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}] 2024-09-11T22:38:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.163+0000 7fbcb7757740 1 -- 172.21.15.5:0/1475509401 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}) -- 0x55f4cefcf890 con 0x55f4cf0e4520 2024-09-11T22:38:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.163+0000 7fbcaeffd640 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f4cf0e4520 0x55f4cf0e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:38:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.164+0000 7fbc92ff5640 10 client.29645 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:55.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:55.164+0000 7fbcaeffd640 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f4cf0e4520 0x55f4cf0e6940 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ab51c4be5d77c34 server_cookie=22fad787f8cd8c7e in_seq=0 out_seq=0 2024-09-11T22:38:55.644 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57903 24.41 MB/sec execute 73 sec latency 438.614 ms 2024-09-11T22:38:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:55 smithi005 ceph-mon[30943]: pgmap v532: 97 pgs: 97 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 388 op/s 2024-09-11T22:38:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:55 smithi159 ceph-mon[35534]: pgmap v532: 97 pgs: 97 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 388 op/s 2024-09-11T22:38:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:55 smithi172 ceph-mon[35461]: pgmap v532: 97 pgs: 97 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 388 op/s 2024-09-11T22:38:56.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.141+0000 7fbc92ff5640 1 -- 172.21.15.5:0/1475509401 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f4cefcf890 con 0x55f4cf0e4520 2024-09-11T22:38:56.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.141+0000 7fbc92ff5640 10 client.29645 handle_command_reply: tid=0 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.141+0000 7fbcb7757740 1 client.29645 shutdown 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 1 -- 172.21.15.5:0/1475509401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05e170 msgr2=0x55f4cf06d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05e170 0x55f4cf06d9a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbca40bbd50 tx=0x7fbca403b690 comp rx=0 tx=0).stop 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 1 -- 172.21.15.5:0/1475509401 shutdown_connections 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55f4cf0e4520 0x55f4cf0e6940 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f4cf068a30 0x55f4cf06dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4cf05e170 0x55f4cf06d9a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 1 --2- 172.21.15.5:0/1475509401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4cf05d850 0x55f4cf070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbc92ff5640 0 client.29645 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 1 -- 172.21.15.5:0/1475509401 >> 172.21.15.5:0/1475509401 conn(0x55f4cf059de0 msgr2=0x55f4cf05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:56.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 1 -- 172.21.15.5:0/1475509401 shutdown_connections 2024-09-11T22:38:56.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 1 -- 172.21.15.5:0/1475509401 wait complete. 2024-09-11T22:38:56.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb7757740 20 client.29645 trim_cache size 0 max 16384 2024-09-11T22:38:56.145 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:38:56.145 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 54, 2024-09-11T22:38:56.145 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:38:56.145 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 58, 2024-09-11T22:38:56.145 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:38:56.145 INFO:teuthology.orchestra.run.smithi005.stdout: "12260228": { 2024-09-11T22:38:56.145 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 58, 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.2, 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:38:56.146 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:38:56.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:38:56.147 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:38:56.147 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:38:56.147 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:38:56.147 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:38:56.147 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:38:56.147 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:38:56.147 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:38:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb537e640 1 -- 172.21.15.5:0/1871817564 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc980a2cf0 msgr2=0x7fbc980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb537e640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc980a2cf0 0x7fbc980a51b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009fd0 tx=0x7fbca8004a40 comp rx=0 tx=0).stop 2024-09-11T22:38:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb537e640 1 -- 172.21.15.5:0/1871817564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcb0102570 msgr2=0x7fbcb00fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:38:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb537e640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcb0102570 0x7fbcb00fd1c0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fbca4006010 tx=0x7fbca4003c10 comp rx=0 tx=0).stop 2024-09-11T22:38:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb537e640 1 -- 172.21.15.5:0/1871817564 shutdown_connections 2024-09-11T22:38:56.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb537e640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbcb0102ef0 0x7fbcb00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb537e640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcb0102570 0x7fbcb00fd1c0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb537e640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc980a2cf0 0x7fbc980a51b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb537e640 1 --2- 172.21.15.5:0/1871817564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcb0100e20 0x7fbcb00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:38:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.142+0000 7fbcb537e640 1 -- 172.21.15.5:0/1871817564 >> 172.21.15.5:0/1871817564 conn(0x7fbcb00fc7d0 msgr2=0x7fbcb006a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:38:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.143+0000 7fbcb537e640 1 -- 172.21.15.5:0/1871817564 shutdown_connections 2024-09-11T22:38:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:38:56.143+0000 7fbcb537e640 1 -- 172.21.15.5:0/1871817564 wait complete. 2024-09-11T22:38:56.644 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 24.08 MB/sec execute 74 sec latency 1431.476 ms 2024-09-11T22:38:57.273 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '12260228', quiesce took 5.0 seconds. Will release after: 19.25 2024-09-11T22:38:57.644 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 23.76 MB/sec execute 75 sec latency 2431.524 ms 2024-09-11T22:38:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:57 smithi005 ceph-mon[30943]: pgmap v533: 97 pgs: 97 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 388 op/s 2024-09-11T22:38:58.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:57 smithi159 ceph-mon[35534]: pgmap v533: 97 pgs: 97 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 388 op/s 2024-09-11T22:38:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:57 smithi172 ceph-mon[35461]: pgmap v533: 97 pgs: 97 active+clean; 503 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 388 op/s 2024-09-11T22:38:58.644 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 23.44 MB/sec execute 76 sec latency 3431.613 ms 2024-09-11T22:38:59.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 23.14 MB/sec execute 77 sec latency 4431.714 ms 2024-09-11T22:39:00.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:38:59 smithi005 ceph-mon[30943]: pgmap v534: 97 pgs: 97 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 765 op/s 2024-09-11T22:39:00.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:38:59 smithi159 ceph-mon[35534]: pgmap v534: 97 pgs: 97 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 765 op/s 2024-09-11T22:39:00.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:38:59 smithi172 ceph-mon[35461]: pgmap v534: 97 pgs: 97 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 765 op/s 2024-09-11T22:39:00.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 22.84 MB/sec execute 78 sec latency 5431.819 ms 2024-09-11T22:39:01.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 22.55 MB/sec execute 79 sec latency 6431.874 ms 2024-09-11T22:39:01.881 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:01 smithi159 ceph-mon[35534]: pgmap v535: 97 pgs: 97 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-11T22:39:02.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:01 smithi005 ceph-mon[30943]: pgmap v535: 97 pgs: 97 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-11T22:39:02.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:01 smithi172 ceph-mon[35461]: pgmap v535: 97 pgs: 97 active+clean; 500 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-11T22:39:02.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 22.27 MB/sec execute 80 sec latency 7431.978 ms 2024-09-11T22:39:03.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 22.00 MB/sec execute 81 sec latency 8432.100 ms 2024-09-11T22:39:03.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:03 smithi005 ceph-mon[30943]: pgmap v536: 97 pgs: 97 active+clean; 490 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-09-11T22:39:03.911 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:03 smithi172 ceph-mon[35461]: pgmap v536: 97 pgs: 97 active+clean; 490 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-09-11T22:39:04.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:03 smithi159 ceph-mon[35534]: pgmap v536: 97 pgs: 97 active+clean; 490 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.7 MiB/s wr, 658 op/s 2024-09-11T22:39:04.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 21.73 MB/sec execute 82 sec latency 9432.134 ms 2024-09-11T22:39:05.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 21.47 MB/sec execute 83 sec latency 10432.249 ms 2024-09-11T22:39:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:05 smithi005 ceph-mon[30943]: pgmap v537: 97 pgs: 97 active+clean; 490 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 461 op/s 2024-09-11T22:39:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:05 smithi159 ceph-mon[35534]: pgmap v537: 97 pgs: 97 active+clean; 490 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 461 op/s 2024-09-11T22:39:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:05 smithi172 ceph-mon[35461]: pgmap v537: 97 pgs: 97 active+clean; 490 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 461 op/s 2024-09-11T22:39:06.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 21.21 MB/sec execute 84 sec latency 11432.383 ms 2024-09-11T22:39:07.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 20.96 MB/sec execute 85 sec latency 12432.423 ms 2024-09-11T22:39:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:07 smithi005 ceph-mon[30943]: pgmap v538: 97 pgs: 97 active+clean; 490 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 461 op/s 2024-09-11T22:39:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:07 smithi159 ceph-mon[35534]: pgmap v538: 97 pgs: 97 active+clean; 490 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 461 op/s 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.101+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/3803480124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a44102d60 msgr2=0x7f5a44103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.101+0000 7f5a4a7f1640 1 --2- 172.21.15.5:0/3803480124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a44102d60 0x7f5a44103140 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009900 tx=0x7f5a3402a480 comp rx=0 tx=0).stop 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.101+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/3803480124 shutdown_connections 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.101+0000 7f5a4a7f1640 1 --2- 172.21.15.5:0/3803480124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a4410c0f0 0x7f5a4410e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.101+0000 7f5a4a7f1640 1 --2- 172.21.15.5:0/3803480124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a44103680 0x7f5a4410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.101+0000 7f5a4a7f1640 1 --2- 172.21.15.5:0/3803480124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a44102d60 0x7f5a44103140 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.101+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/3803480124 >> 172.21.15.5:0/3803480124 conn(0x7f5a440fc6f0 msgr2=0x7f5a440feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.101+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/3803480124 shutdown_connections 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.101+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/3803480124 wait complete. 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.102+0000 7f5a4a7f1640 1 Processor -- start 2024-09-11T22:39:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.102+0000 7f5a4a7f1640 1 -- start start 2024-09-11T22:39:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.102+0000 7f5a4a7f1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a44102d60 0x7f5a44198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.102+0000 7f5a4a7f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a44103680 0x7f5a44198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.102+0000 7f5a4a7f1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a4410c0f0 0x7f5a4419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.102+0000 7f5a4a7f1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44111ee0 con 0x7f5a44103680 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.102+0000 7f5a4a7f1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44111d60 con 0x7f5a4410c0f0 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.102+0000 7f5a4a7f1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5a44112060 con 0x7f5a44102d60 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a44103680 0x7f5a44198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a44103680 0x7f5a44198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47082/0 (socket says 172.21.15.5:47082) 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a437fe640 1 -- 172.21.15.5:0/479629952 learned_addr learned my addr 172.21.15.5:0/479629952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a43fff640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a44102d60 0x7f5a44198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a48d8e640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a4410c0f0 0x7f5a4419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a437fe640 1 -- 172.21.15.5:0/479629952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a44102d60 msgr2=0x7f5a44198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a437fe640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a44102d60 0x7f5a44198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a437fe640 1 -- 172.21.15.5:0/479629952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a4410c0f0 msgr2=0x7f5a4419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a437fe640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a4410c0f0 0x7f5a4419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a437fe640 1 -- 172.21.15.5:0/479629952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4419d320 con 0x7f5a44103680 2024-09-11T22:39:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a437fe640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a44103680 0x7f5a44198870 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c970 tx=0x7f5a3800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a417fa640 1 -- 172.21.15.5:0/479629952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38007bb0 con 0x7f5a44103680 2024-09-11T22:39:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a417fa640 1 -- 172.21.15.5:0/479629952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a38007d50 con 0x7f5a44103680 2024-09-11T22:39:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/479629952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4419d610 con 0x7f5a44103680 2024-09-11T22:39:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a417fa640 1 -- 172.21.15.5:0/479629952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800f450 con 0x7f5a44103680 2024-09-11T22:39:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.103+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/479629952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a441a4ea0 con 0x7f5a44103680 2024-09-11T22:39:08.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:07 smithi172 ceph-mon[35461]: pgmap v538: 97 pgs: 97 active+clean; 490 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 461 op/s 2024-09-11T22:39:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a417fa640 1 -- 172.21.15.5:0/479629952 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f5a38005020 con 0x7f5a44103680 2024-09-11T22:39:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a4d663740 1 Processor -- start 2024-09-11T22:39:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a4d663740 1 -- start start 2024-09-11T22:39:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7a97ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a4d663740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a3b79fc2f0 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a4d663740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a3b79a4270 con 0x55a3b7a97af0 2024-09-11T22:39:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a4d663740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a3b79f82b0 con 0x55a3b7a98410 2024-09-11T22:39:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a48d8e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7a97ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a48d8e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7a97ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55740/0 (socket says 172.21.15.5:55740) 2024-09-11T22:39:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a48d8e640 1 -- 172.21.15.5:0/1474543047 learned_addr learned my addr 172.21.15.5:0/1474543047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.106+0000 7f5a43fff640 1 --2- 172.21.15.5:0/1474543047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a417fa640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a2c0a2da0 0x7f5a2c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a48d8e640 1 -- 172.21.15.5:0/1474543047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 msgr2=0x55a3b7aa2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a48d8e640 1 --2- 172.21.15.5:0/1474543047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a48d8e640 1 -- 172.21.15.5:0/1474543047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 msgr2=0x55a3b7aa50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a48d8e640 1 --2- 172.21.15.5:0/1474543047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a48d8e640 1 -- 172.21.15.5:0/1474543047 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3b79a3340 con 0x55a3b7a97af0 2024-09-11T22:39:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a437fe640 1 --2- 172.21.15.5:0/1474543047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a48d8e640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a2c0a2da0 0x7f5a2c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a417fa640 1 -- 172.21.15.5:0/479629952 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f5a380cf2b0 con 0x7f5a44103680 2024-09-11T22:39:08.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a48d8e640 1 --2- 172.21.15.5:0/1474543047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7a97ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c010520 tx=0x7f5a3c0109e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cfd842edbed0472 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a2a7fc640 1 -- 172.21.15.5:0/1474543047 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c019070 con 0x55a3b7a97af0 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a2a7fc640 1 -- 172.21.15.5:0/1474543047 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c003e10 con 0x55a3b7a97af0 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.107+0000 7f5a2a7fc640 1 -- 172.21.15.5:0/1474543047 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c011740 con 0x55a3b7a97af0 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a4d663740 1 -- 172.21.15.5:0/1474543047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 msgr2=0x55a3b7a97ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a48d8e640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a2c0a2da0 0x7f5a2c0a5260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c021000 tx=0x7f5a3c01e000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1474543047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7a97ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c010520 tx=0x7f5a3c0109e0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a4d663740 1 -- 172.21.15.5:0/1474543047 shutdown_connections 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1474543047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1474543047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1474543047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7a97ed0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a4d663740 1 -- 172.21.15.5:0/1474543047 >> 172.21.15.5:0/1474543047 conn(0x55a3b7a94060 msgr2=0x55a3b7a94440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:08.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a4d663740 1 -- 172.21.15.5:0/1474543047 shutdown_connections 2024-09-11T22:39:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a4d663740 1 -- 172.21.15.5:0/1474543047 wait complete. 2024-09-11T22:39:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.108+0000 7f5a4d663740 1 -- start start 2024-09-11T22:39:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a4d663740 1 Processor -- start 2024-09-11T22:39:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7a9fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a98410 0x55a3b7aa0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a4d663740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a3b7aa5880 con 0x55a3b7a98410 2024-09-11T22:39:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a4d663740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a3b7985290 con 0x55a3b7a97af0 2024-09-11T22:39:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a4d663740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a3b7aa5a00 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a4d663740 10 client.0 _ll_register_callbacks cb 0x55a3b79b6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a98410 0x55a3b7aa0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a98410 0x55a3b7aa0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47100/0 (socket says 172.21.15.5:47100) 2024-09-11T22:39:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a437fe640 1 -- 172.21.15.5:0/1733497080 learned_addr learned my addr 172.21.15.5:0/1733497080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a43fff640 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7a9fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a48d8e640 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a437fe640 1 -- 172.21.15.5:0/1733497080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7aa2cb0 msgr2=0x55a3b7aa0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a437fe640 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a437fe640 1 -- 172.21.15.5:0/1733497080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 msgr2=0x55a3b7a9fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a437fe640 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7a9fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a437fe640 1 -- 172.21.15.5:0/1733497080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3b770f210 con 0x55a3b7a98410 2024-09-11T22:39:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a437fe640 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a98410 0x55a3b7aa0110 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f5a3809b460 tx=0x7f5a3800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9082c895b274db7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.109+0000 7f5a237fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a237fe640 1 -- 172.21.15.5:0/1733497080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3801d040 con 0x55a3b7a98410 2024-09-11T22:39:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a237fe640 1 -- 172.21.15.5:0/1733497080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f5a38097dd0 con 0x55a3b7a98410 2024-09-11T22:39:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a4d663740 1 -- 172.21.15.5:0/1733497080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a3c003e10 con 0x55a3b7a98410 2024-09-11T22:39:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a237fe640 1 -- 172.21.15.5:0/1733497080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5a3809b2a0 con 0x55a3b7a98410 2024-09-11T22:39:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a4d663740 10 client.20586 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a4d663740 1 -- 172.21.15.5:0/1733497080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a3b77db620 con 0x55a3b7a98410 2024-09-11T22:39:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a237fe640 1 -- 172.21.15.5:0/1733497080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a3c003e10 con 0x55a3b7a98410 2024-09-11T22:39:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a237fe640 1 -- 172.21.15.5:0/1733497080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f5a38005310 con 0x55a3b7a98410 2024-09-11T22:39:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a4d663740 10 client.20586 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:08.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a4d663740 10 client.20586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a4d663740 20 client.20586 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3b7ab2c50 0x55a3b7ab5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a4d663740 4 client.20586 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.110+0000 7f5a4d663740 1 -- 172.21.15.5:0/1733497080 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3b7a09a20 con 0x55a3b7ab2c50 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.111+0000 7f5a43fff640 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3b7ab2c50 0x55a3b7ab5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.111+0000 7f5a43fff640 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3b7ab2c50 0x55a3b7ab5040 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71886ca8210b1d43 server_cookie=3cce12a3071b8dc9 in_seq=0 out_seq=0 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.111+0000 7f5a237fe640 10 client.20586 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a237fe640 1 -- 172.21.15.5:0/1733497080 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3b7a09a20 con 0x55a3b7ab2c50 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a237fe640 10 client.20586 handle_command_reply: tid=0 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 client.20586 shutdown 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 -- 172.21.15.5:0/1733497080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a98410 msgr2=0x55a3b7aa0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a98410 0x55a3b7aa0110 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f5a3809b460 tx=0x7f5a3800cf60 comp rx=0 tx=0).stop 2024-09-11T22:39:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 -- 172.21.15.5:0/1733497080 shutdown_connections 2024-09-11T22:39:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3b7ab2c50 0x55a3b7ab5040 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a237fe640 0 client.20586 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a98410 0x55a3b7aa0110 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1733497080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7a9fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 -- 172.21.15.5:0/1733497080 >> 172.21.15.5:0/1733497080 conn(0x55a3b7a94060 msgr2=0x55a3b7a99d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 -- 172.21.15.5:0/1733497080 shutdown_connections 2024-09-11T22:39:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 1 -- 172.21.15.5:0/1733497080 wait complete. 2024-09-11T22:39:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.112+0000 7f5a4d663740 20 client.20586 trim_cache size 0 max 16384 2024-09-11T22:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.126+0000 7f5a4d663740 1 Processor -- start 2024-09-11T22:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.126+0000 7f5a4d663740 1 -- start start 2024-09-11T22:39:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.126+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a97af0 0x55a3b7a9a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.126+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.126+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.126+0000 7f5a4d663740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441ad970 con 0x55a3b7a97af0 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.126+0000 7f5a4d663740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441af4b0 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.126+0000 7f5a4d663740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441ae040 con 0x55a3b7a98410 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a97af0 0x55a3b7a9a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a97af0 0x55a3b7a9a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47116/0 (socket says 172.21.15.5:47116) 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a43fff640 1 -- 172.21.15.5:0/2938427730 learned_addr learned my addr 172.21.15.5:0/2938427730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a437fe640 1 --2- 172.21.15.5:0/2938427730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a43fff640 1 -- 172.21.15.5:0/2938427730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 msgr2=0x55a3b7aa0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a43fff640 1 --2- 172.21.15.5:0/2938427730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a43fff640 1 -- 172.21.15.5:0/2938427730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7aa2cb0 msgr2=0x55a3b7aba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a43fff640 1 --2- 172.21.15.5:0/2938427730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a43fff640 1 -- 172.21.15.5:0/2938427730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3b7a97870 con 0x55a3b7a97af0 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a43fff640 1 --2- 172.21.15.5:0/2938427730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a97af0 0x55a3b7a9a560 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5a3402a9a0 tx=0x7f5a340058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f9026b0e382248 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a29ffb640 1 -- 172.21.15.5:0/2938427730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34035040 con 0x55a3b7a97af0 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a29ffb640 1 -- 172.21.15.5:0/2938427730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a34031880 con 0x55a3b7a97af0 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.127+0000 7f5a29ffb640 1 -- 172.21.15.5:0/2938427730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34004a00 con 0x55a3b7a97af0 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 -- 172.21.15.5:0/2938427730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a97af0 msgr2=0x55a3b7a9a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 --2- 172.21.15.5:0/2938427730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a97af0 0x55a3b7a9a560 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5a3402a9a0 tx=0x7f5a340058d0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 -- 172.21.15.5:0/2938427730 shutdown_connections 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 --2- 172.21.15.5:0/2938427730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 --2- 172.21.15.5:0/2938427730 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 --2- 172.21.15.5:0/2938427730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7a97af0 0x55a3b7a9a560 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 -- 172.21.15.5:0/2938427730 >> 172.21.15.5:0/2938427730 conn(0x55a3b7a94060 msgr2=0x55a3b7aa1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 -- 172.21.15.5:0/2938427730 shutdown_connections 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 -- 172.21.15.5:0/2938427730 wait complete. 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 -- start start 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.128+0000 7f5a4d663740 1 Processor -- start 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7aab230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a4d663740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a4d663740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441aede0 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a4d663740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441ae710 con 0x55a3b7a97af0 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a4d663740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5a441ab970 con 0x55a3b7a98410 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a4d663740 10 client.0 _ll_register_callbacks cb 0x7f5a3800c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a48d8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a48d8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47118/0 (socket says 172.21.15.5:47118) 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a48d8e640 1 -- 172.21.15.5:0/1301817991 learned_addr learned my addr 172.21.15.5:0/1301817991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a43fff640 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7aab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a437fe640 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a48d8e640 1 -- 172.21.15.5:0/1301817991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 msgr2=0x55a3b7aa65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a48d8e640 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a48d8e640 1 -- 172.21.15.5:0/1301817991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 msgr2=0x55a3b7aab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a48d8e640 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7aab230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a48d8e640 1 -- 172.21.15.5:0/1301817991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3b7aa7120 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a437fe640 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a48d8e640 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa6ae0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0116e0 tx=0x7f5a3c011710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2086cd61e210e758 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a237fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.129+0000 7f5a237fe640 1 -- 172.21.15.5:0/1301817991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c035060 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a237fe640 1 -- 172.21.15.5:0/1301817991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8629+0+0 (secure 0 0 0) 0x7f5a3c023040 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a4d663740 1 -- 172.21.15.5:0/1301817991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a34031880 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a237fe640 1 -- 172.21.15.5:0/1301817991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c019050 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a4d663740 10 client.20592 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a4d663740 1 -- 172.21.15.5:0/1301817991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55a3b7b1d710 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a237fe640 1 -- 172.21.15.5:0/1301817991 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a34031880 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a237fe640 1 -- 172.21.15.5:0/1301817991 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f5a3c013a00 con 0x55a3b7aa2cb0 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a4d663740 10 client.20592 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a4d663740 10 client.20592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a4d663740 20 client.20592 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3b7b1dc80 0x55a3b7b200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a4d663740 4 client.20592 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:39:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a4d663740 1 -- 172.21.15.5:0/1301817991 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3b7a03d70 con 0x55a3b7b1dc80 2024-09-11T22:39:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.130+0000 7f5a43fff640 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3b7b1dc80 0x55a3b7b200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.131+0000 7f5a237fe640 10 client.20592 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.131+0000 7f5a43fff640 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3b7b1dc80 0x55a3b7b200a0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5409b8874469611 server_cookie=61cbb9a785122a in_seq=0 out_seq=0 2024-09-11T22:39:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a237fe640 1 -- 172.21.15.5:0/1301817991 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3b7a03d70 con 0x55a3b7b1dc80 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a237fe640 10 client.20592 handle_command_reply: tid=0 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 client.20592 shutdown 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 -- 172.21.15.5:0/1301817991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 msgr2=0x55a3b7aa6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa6ae0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0116e0 tx=0x7f5a3c011710 comp rx=0 tx=0).stop 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 -- 172.21.15.5:0/1301817991 shutdown_connections 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3b7b1dc80 0x55a3b7b200a0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3b7aa2cb0 0x55a3b7aa6ae0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3b7a98410 0x55a3b7aa65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 --2- 172.21.15.5:0/1301817991 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3b7a97af0 0x55a3b7aab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a237fe640 0 client.20592 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 -- 172.21.15.5:0/1301817991 >> 172.21.15.5:0/1301817991 conn(0x55a3b7a94060 msgr2=0x55a3b7aa1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 -- 172.21.15.5:0/1301817991 shutdown_connections 2024-09-11T22:39:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.319+0000 7f5a4d663740 1 -- 172.21.15.5:0/1301817991 wait complete. 2024-09-11T22:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.320+0000 7f5a4d663740 20 client.20592 trim_cache size 0 max 16384 2024-09-11T22:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:39:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:39:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.320+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/479629952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a2c0a2da0 msgr2=0x7f5a2c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.320+0000 7f5a4a7f1640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a2c0a2da0 0x7f5a2c0a5260 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c021000 tx=0x7f5a3c01e000 comp rx=0 tx=0).stop 2024-09-11T22:39:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.320+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/479629952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a44103680 msgr2=0x7f5a44198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.320+0000 7f5a4a7f1640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a44103680 0x7f5a44198870 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c970 tx=0x7f5a3800ce30 comp rx=0 tx=0).stop 2024-09-11T22:39:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.320+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/479629952 shutdown_connections 2024-09-11T22:39:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.320+0000 7f5a4a7f1640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5a2c0a2da0 0x7f5a2c0a5260 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.320+0000 7f5a4a7f1640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a4410c0f0 0x7f5a4419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.320+0000 7f5a4a7f1640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a44103680 0x7f5a44198870 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.320+0000 7f5a4a7f1640 1 --2- 172.21.15.5:0/479629952 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a44102d60 0x7f5a44198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.321+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/479629952 >> 172.21.15.5:0/479629952 conn(0x7f5a440fc6f0 msgr2=0x7f5a440fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.321+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/479629952 shutdown_connections 2024-09-11T22:39:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:08.321+0000 7f5a4a7f1640 1 -- 172.21.15.5:0/479629952 wait complete. 2024-09-11T22:39:08.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 20.72 MB/sec execute 86 sec latency 13432.461 ms 2024-09-11T22:39:08.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:39:09.134 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:09.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 20.48 MB/sec execute 87 sec latency 14432.514 ms 2024-09-11T22:39:10.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:09 smithi005 ceph-mon[30943]: pgmap v539: 97 pgs: 97 active+clean; 486 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 536 op/s 2024-09-11T22:39:10.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.044+0000 7f680cbe4640 1 -- 172.21.15.5:0/1897843587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6808114ab0 msgr2=0x7f6808116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:10.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.044+0000 7f680cbe4640 1 --2- 172.21.15.5:0/1897843587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6808114ab0 0x7f6808116ea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f67fc00fb10 tx=0x7f67fc032a00 comp rx=0 tx=0).stop 2024-09-11T22:39:10.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.044+0000 7f680cbe4640 1 -- 172.21.15.5:0/1897843587 shutdown_connections 2024-09-11T22:39:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.044+0000 7f680cbe4640 1 --2- 172.21.15.5:0/1897843587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6808114ab0 0x7f6808116ea0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.044+0000 7f680cbe4640 1 --2- 172.21.15.5:0/1897843587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808109b10 0x7f6808114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.044+0000 7f680cbe4640 1 --2- 172.21.15.5:0/1897843587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68081091f0 0x7f68081095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.044+0000 7f680cbe4640 1 -- 172.21.15.5:0/1897843587 >> 172.21.15.5:0/1897843587 conn(0x7f680806e140 msgr2=0x7f680806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:10.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.045+0000 7f680cbe4640 1 -- 172.21.15.5:0/1897843587 shutdown_connections 2024-09-11T22:39:10.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.046+0000 7f680cbe4640 1 -- 172.21.15.5:0/1897843587 wait complete. 2024-09-11T22:39:10.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.047+0000 7f680cbe4640 1 Processor -- start 2024-09-11T22:39:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.050+0000 7f680cbe4640 1 -- start start 2024-09-11T22:39:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.050+0000 7f680cbe4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68081091f0 0x7f680819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.050+0000 7f680cbe4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6808109b10 0x7f680819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.050+0000 7f680cbe4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808114ab0 0x7f68081a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.050+0000 7f680cbe4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68081185c0 con 0x7f6808109b10 2024-09-11T22:39:10.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.050+0000 7f680cbe4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6808118440 con 0x7f68081091f0 2024-09-11T22:39:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.050+0000 7f680cbe4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6808118740 con 0x7f6808114ab0 2024-09-11T22:39:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.051+0000 7f680659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68081091f0 0x7f680819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.051+0000 7f680659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68081091f0 0x7f680819eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55778/0 (socket says 172.21.15.5:55778) 2024-09-11T22:39:10.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.051+0000 7f680659c640 1 -- 172.21.15.5:0/3263535049 learned_addr learned my addr 172.21.15.5:0/3263535049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.051+0000 7f6805d9b640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6808109b10 0x7f680819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.052+0000 7f6806d9d640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808114ab0 0x7f68081a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.052+0000 7f6805d9b640 1 -- 172.21.15.5:0/3263535049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808114ab0 msgr2=0x7f68081a3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:10.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.052+0000 7f6805d9b640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808114ab0 0x7f68081a3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.052+0000 7f6805d9b640 1 -- 172.21.15.5:0/3263535049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68081091f0 msgr2=0x7f680819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:10.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.052+0000 7f6805d9b640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68081091f0 0x7f680819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.052+0000 7f6805d9b640 1 -- 172.21.15.5:0/3263535049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68081a3b30 con 0x7f6808109b10 2024-09-11T22:39:10.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.052+0000 7f6806d9d640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808114ab0 0x7f68081a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:10.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.052+0000 7f6805d9b640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6808109b10 0x7f680819f040 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f67f800b3f0 tx=0x7f67f800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:10.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.053+0000 7f67f77fe640 1 -- 172.21.15.5:0/3263535049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8015020 con 0x7f6808109b10 2024-09-11T22:39:10.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.053+0000 7f67f77fe640 1 -- 172.21.15.5:0/3263535049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f8011d10 con 0x7f6808109b10 2024-09-11T22:39:10.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.053+0000 7f67f77fe640 1 -- 172.21.15.5:0/3263535049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f800f2e0 con 0x7f6808109b10 2024-09-11T22:39:10.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.053+0000 7f680cbe4640 1 -- 172.21.15.5:0/3263535049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68081a3e20 con 0x7f6808109b10 2024-09-11T22:39:10.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.053+0000 7f680cbe4640 1 -- 172.21.15.5:0/3263535049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68081ab6b0 con 0x7f6808109b10 2024-09-11T22:39:10.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.058+0000 7f67f77fe640 1 -- 172.21.15.5:0/3263535049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f67f8018020 con 0x7f6808109b10 2024-09-11T22:39:10.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.059+0000 7f67f77fe640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67f00a2df0 0x7f67f00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.059+0000 7f67f77fe640 1 -- 172.21.15.5:0/3263535049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7f67f80cec50 con 0x7f6808109b10 2024-09-11T22:39:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.060+0000 7f680efbd740 1 Processor -- start 2024-09-11T22:39:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f680efbd740 1 -- start start 2024-09-11T22:39:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f680efbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5e4a0 0x5635adf5e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f680efbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf69140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f680efbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf69680 0x5635adf6ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f680efbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5635adec42e0 con 0x5635adf5e4a0 2024-09-11T22:39:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f680efbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5635ade6bce0 con 0x5635adf69680 2024-09-11T22:39:10.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f680659c640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67f00a2df0 0x7f67f00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f680efbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5635adec02a0 con 0x5635adf5edc0 2024-09-11T22:39:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f6805d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf69140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f6805d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf69140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47642/0 (socket says 172.21.15.5:47642) 2024-09-11T22:39:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.061+0000 7f6805d9b640 1 -- 172.21.15.5:0/2596552850 learned_addr learned my addr 172.21.15.5:0/2596552850 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.063+0000 7f680659c640 1 --2- 172.21.15.5:0/2596552850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5e4a0 0x5635adf5e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.063+0000 7f6806d9d640 1 --2- 172.21.15.5:0/2596552850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf69680 0x5635adf6ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.063+0000 7f6805d9b640 1 -- 172.21.15.5:0/2596552850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf69680 msgr2=0x5635adf6ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.063+0000 7f6805d9b640 1 --2- 172.21.15.5:0/2596552850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf69680 0x5635adf6ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.063+0000 7f6805d9b640 1 -- 172.21.15.5:0/2596552850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5e4a0 msgr2=0x5635adf5e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.063+0000 7f6805d9b640 1 --2- 172.21.15.5:0/2596552850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5e4a0 0x5635adf5e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.063+0000 7f6805d9b640 1 -- 172.21.15.5:0/2596552850 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ade50440 con 0x5635adf5edc0 2024-09-11T22:39:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.065+0000 7f6805d9b640 1 --2- 172.21.15.5:0/2596552850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf69140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f67f80975e0 tx=0x7f67f809aba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f37f6ed1188ec751 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.065+0000 7f680659c640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67f00a2df0 0x7f67f00a52b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f68000137f0 tx=0x7f6800013bf0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:10.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.065+0000 7f67f4ff9640 1 -- 172.21.15.5:0/2596552850 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f80b4070 con 0x5635adf5edc0 2024-09-11T22:39:10.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.065+0000 7f67f4ff9640 1 -- 172.21.15.5:0/2596552850 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f809f530 con 0x5635adf5edc0 2024-09-11T22:39:10.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.065+0000 7f67f4ff9640 1 -- 172.21.15.5:0/2596552850 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f809f170 con 0x5635adf5edc0 2024-09-11T22:39:10.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.067+0000 7f680efbd740 1 -- 172.21.15.5:0/2596552850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 msgr2=0x5635adf69140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:10.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.067+0000 7f680efbd740 1 --2- 172.21.15.5:0/2596552850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf69140 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f67f80975e0 tx=0x7f67f809aba0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.067+0000 7f680efbd740 1 -- 172.21.15.5:0/2596552850 shutdown_connections 2024-09-11T22:39:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.067+0000 7f680efbd740 1 --2- 172.21.15.5:0/2596552850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf69680 0x5635adf6ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.067+0000 7f680efbd740 1 --2- 172.21.15.5:0/2596552850 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf69140 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.067+0000 7f680efbd740 1 --2- 172.21.15.5:0/2596552850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5e4a0 0x5635adf5e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.067+0000 7f680efbd740 1 -- 172.21.15.5:0/2596552850 >> 172.21.15.5:0/2596552850 conn(0x5635adf5aa30 msgr2=0x5635adf5ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.069+0000 7f680efbd740 1 -- 172.21.15.5:0/2596552850 shutdown_connections 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.070+0000 7f680efbd740 1 -- 172.21.15.5:0/2596552850 wait complete. 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.071+0000 7f680efbd740 1 -- start start 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.071+0000 7f680efbd740 1 Processor -- start 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.071+0000 7f680efbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5e4a0 0x5635adf69060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.071+0000 7f680efbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5edc0 0x5635adf66310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.071+0000 7f680efbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 0x5635adf668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.071+0000 7f680efbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5635adf6c250 con 0x5635adf5e4a0 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.071+0000 7f680efbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5635ade7f050 con 0x5635adf5edc0 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.071+0000 7f680efbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5635adf6c3d0 con 0x5635adf69680 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f6805d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5edc0 0x5635adf66310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f6805d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5edc0 0x5635adf66310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55792/0 (socket says 172.21.15.5:55792) 2024-09-11T22:39:10.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f6805d9b640 1 -- 172.21.15.5:0/520136156 learned_addr learned my addr 172.21.15.5:0/520136156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f680efbd740 10 client.0 _ll_register_callbacks cb 0x5635adebf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f6806d9d640 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 0x5635adf668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f680659c640 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5e4a0 0x5635adf69060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f6805d9b640 1 -- 172.21.15.5:0/520136156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 msgr2=0x5635adf668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f6805d9b640 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 0x5635adf668e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f6805d9b640 1 -- 172.21.15.5:0/520136156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5e4a0 msgr2=0x5635adf69060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f6805d9b640 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5e4a0 0x5635adf69060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f6805d9b640 1 -- 172.21.15.5:0/520136156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635adc353c0 con 0x5635adf5edc0 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.072+0000 7f6805d9b640 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5edc0 0x5635adf66310 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67f80a38c0 tx=0x7f67f80107d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4492af7ae266bdf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.073+0000 7f680efbd740 1 -- 172.21.15.5:0/520136156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f809f530 con 0x5635adf5edc0 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.073+0000 7f67edffb640 10 client.29672 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.073+0000 7f67edffb640 1 -- 172.21.15.5:0/520136156 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f80b4040 con 0x5635adf5edc0 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.073+0000 7f67edffb640 1 -- 172.21.15.5:0/520136156 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7f67f80bb0e0 con 0x5635adf5edc0 2024-09-11T22:39:10.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.073+0000 7f67edffb640 1 -- 172.21.15.5:0/520136156 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f67f809f170 con 0x5635adf5edc0 2024-09-11T22:39:10.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.074+0000 7f680efbd740 10 client.29672 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.074+0000 7f680efbd740 1 -- 172.21.15.5:0/520136156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=54}) -- 0x5635adf6c250 con 0x5635adf5edc0 2024-09-11T22:39:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.074+0000 7f67edffb640 1 -- 172.21.15.5:0/520136156 <== mon.2 v2:172.21.15.159:3300/0 4 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f67f80cc520 con 0x5635adf5edc0 2024-09-11T22:39:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.074+0000 7f680efbd740 10 client.29672 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.074+0000 7f680efbd740 10 client.29672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.074+0000 7f680efbd740 20 client.29672 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.074+0000 7f680efbd740 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5635adf78ee0 0x5635adf7b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.075+0000 7f680efbd740 4 client.29672 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.075+0000 7f680efbd740 1 -- 172.21.15.5:0/520136156 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f67f809f530 con 0x5635adf78ee0 2024-09-11T22:39:10.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.078+0000 7f680659c640 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5635adf78ee0 0x5635adf7b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:09 smithi159 ceph-mon[35534]: pgmap v539: 97 pgs: 97 active+clean; 486 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 536 op/s 2024-09-11T22:39:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.091+0000 7f680659c640 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5635adf78ee0 0x5635adf7b320 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926becc60d5e24f8 server_cookie=ea068dc38073c733 in_seq=0 out_seq=0 2024-09-11T22:39:10.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.091+0000 7f67edffb640 10 client.29672 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.093+0000 7f67edffb640 1 -- 172.21.15.5:0/520136156 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f67f809f530 con 0x5635adf78ee0 2024-09-11T22:39:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.093+0000 7f67edffb640 10 client.29672 handle_command_reply: tid=0 2024-09-11T22:39:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.093+0000 7f680efbd740 1 client.29672 shutdown 2024-09-11T22:39:10.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.094+0000 7f680efbd740 1 -- 172.21.15.5:0/520136156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5edc0 msgr2=0x5635adf66310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:10.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.094+0000 7f680efbd740 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5edc0 0x5635adf66310 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f67f80a38c0 tx=0x7f67f80107d0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.094+0000 7f680efbd740 1 -- 172.21.15.5:0/520136156 shutdown_connections 2024-09-11T22:39:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.094+0000 7f680efbd740 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5635adf78ee0 0x5635adf7b320 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.094+0000 7f680efbd740 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 0x5635adf668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.094+0000 7f680efbd740 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5edc0 0x5635adf66310 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.094+0000 7f680efbd740 1 --2- 172.21.15.5:0/520136156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5e4a0 0x5635adf69060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.094+0000 7f680efbd740 1 -- 172.21.15.5:0/520136156 >> 172.21.15.5:0/520136156 conn(0x5635adf5aa30 msgr2=0x5635adf60760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.094+0000 7f67edffb640 0 client.29672 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.094+0000 7f680efbd740 1 -- 172.21.15.5:0/520136156 shutdown_connections 2024-09-11T22:39:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.095+0000 7f680efbd740 1 -- 172.21.15.5:0/520136156 wait complete. 2024-09-11T22:39:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.095+0000 7f680efbd740 20 client.29672 trim_cache size 0 max 16384 2024-09-11T22:39:10.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:09 smithi172 ceph-mon[35461]: pgmap v539: 97 pgs: 97 active+clean; 486 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 536 op/s 2024-09-11T22:39:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.133+0000 7f680efbd740 1 Processor -- start 2024-09-11T22:39:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.133+0000 7f680efbd740 1 -- start start 2024-09-11T22:39:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.133+0000 7f680efbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 0x5635adf754b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.133+0000 7f680efbd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf68280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.133+0000 7f680efbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf69680 0x5635adf80bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.134+0000 7f680efbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68081b4300 con 0x5635adf69680 2024-09-11T22:39:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.134+0000 7f680efbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f68081b5e40 con 0x5635adf5e4a0 2024-09-11T22:39:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.134+0000 7f680efbd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68081b49d0 con 0x5635adf5edc0 2024-09-11T22:39:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.134+0000 7f6805d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf68280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.134+0000 7f6805d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf68280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:47662/0 (socket says 172.21.15.5:47662) 2024-09-11T22:39:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.134+0000 7f6805d9b640 1 -- 172.21.15.5:0/490604294 learned_addr learned my addr 172.21.15.5:0/490604294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.135+0000 7f680659c640 1 --2- 172.21.15.5:0/490604294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 0x5635adf754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.135+0000 7f6805d9b640 1 -- 172.21.15.5:0/490604294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 msgr2=0x5635adf754b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.135+0000 7f6805d9b640 1 --2- 172.21.15.5:0/490604294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 0x5635adf754b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.135+0000 7f6805d9b640 1 -- 172.21.15.5:0/490604294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf69680 msgr2=0x5635adf80bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.135+0000 7f6805d9b640 1 --2- 172.21.15.5:0/490604294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf69680 0x5635adf80bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.135+0000 7f6805d9b640 1 -- 172.21.15.5:0/490604294 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635adf693d0 con 0x5635adf5edc0 2024-09-11T22:39:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.138+0000 7f6805d9b640 1 --2- 172.21.15.5:0/490604294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf68280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f67f8097590 tx=0x7f67f80bb000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2aa1e91b2d5159b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:10.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.142+0000 7f67effff640 1 -- 172.21.15.5:0/490604294 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f80b4040 con 0x5635adf5edc0 2024-09-11T22:39:10.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.142+0000 7f67effff640 1 -- 172.21.15.5:0/490604294 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f80bb0e0 con 0x5635adf5edc0 2024-09-11T22:39:10.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.142+0000 7f67effff640 1 -- 172.21.15.5:0/490604294 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8097130 con 0x5635adf5edc0 2024-09-11T22:39:10.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.143+0000 7f680efbd740 1 -- 172.21.15.5:0/490604294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 msgr2=0x5635adf68280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:10.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.144+0000 7f680efbd740 1 --2- 172.21.15.5:0/490604294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf68280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f67f8097590 tx=0x7f67f80bb000 comp rx=0 tx=0).stop 2024-09-11T22:39:10.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.145+0000 7f680efbd740 1 -- 172.21.15.5:0/490604294 shutdown_connections 2024-09-11T22:39:10.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.145+0000 7f680efbd740 1 --2- 172.21.15.5:0/490604294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf69680 0x5635adf80bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.145+0000 7f680efbd740 1 --2- 172.21.15.5:0/490604294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf5edc0 0x5635adf68280 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.145+0000 7f680efbd740 1 --2- 172.21.15.5:0/490604294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 0x5635adf754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.145+0000 7f680efbd740 1 -- 172.21.15.5:0/490604294 >> 172.21.15.5:0/490604294 conn(0x5635adf5aa30 msgr2=0x5635adf67b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.145+0000 7f680efbd740 1 -- 172.21.15.5:0/490604294 shutdown_connections 2024-09-11T22:39:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.145+0000 7f680efbd740 1 -- 172.21.15.5:0/490604294 wait complete. 2024-09-11T22:39:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.146+0000 7f680efbd740 1 -- start start 2024-09-11T22:39:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.146+0000 7f680efbd740 1 Processor -- start 2024-09-11T22:39:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.147+0000 7f680efbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 0x5635adf6cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.147+0000 7f680efbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5edc0 0x5635adf6d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.147+0000 7f680659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 0x5635adf6cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.147+0000 7f680659c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 0x5635adf6cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55822/0 (socket says 172.21.15.5:55822) 2024-09-11T22:39:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.147+0000 7f680659c640 1 -- 172.21.15.5:0/128123808 learned_addr learned my addr 172.21.15.5:0/128123808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.147+0000 7f680efbd740 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 0x5635adfdbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.148+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68081b5770 con 0x5635adf5edc0 2024-09-11T22:39:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.148+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f68081b50a0 con 0x5635adf5e4a0 2024-09-11T22:39:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.148+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f68081b2300 con 0x5635adf69680 2024-09-11T22:39:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.148+0000 7f6806d9d640 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 0x5635adfdbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.148+0000 7f680efbd740 10 client.0 _ll_register_callbacks cb 0x7f67f80b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.148+0000 7f6806d9d640 1 -- 172.21.15.5:0/128123808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 msgr2=0x5635adf6cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.148+0000 7f6806d9d640 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 0x5635adf6cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.148+0000 7f6806d9d640 1 -- 172.21.15.5:0/128123808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5edc0 msgr2=0x5635adf6d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.148+0000 7f6806d9d640 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5edc0 0x5635adf6d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.148+0000 7f6806d9d640 1 -- 172.21.15.5:0/128123808 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635adfdc1f0 con 0x5635adf69680 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.149+0000 7f6806d9d640 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 0x5635adfdbb80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009230 tx=0x7f67fc00e890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b9784bd02d18e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.149+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f80bb0e0 con 0x5635adf69680 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.150+0000 7f67edffb640 10 client.29509 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.150+0000 7f67edffb640 1 -- 172.21.15.5:0/128123808 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67fc03b030 con 0x5635adf69680 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.150+0000 7f67edffb640 1 -- 172.21.15.5:0/128123808 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7f67fc03c070 con 0x5635adf69680 2024-09-11T22:39:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.150+0000 7f67edffb640 1 -- 172.21.15.5:0/128123808 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f67fc005d50 con 0x5635adf69680 2024-09-11T22:39:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.150+0000 7f680efbd740 10 client.29509 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.150+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({fsmap=54}) -- 0x5635adfe34b0 con 0x5635adf69680 2024-09-11T22:39:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.152+0000 7f67edffb640 1 -- 172.21.15.5:0/128123808 <== mon.1 v2:172.21.15.172:3300/0 4 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f67fc00ec10 con 0x5635adf69680 2024-09-11T22:39:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.152+0000 7f680efbd740 10 client.29509 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.152+0000 7f680efbd740 10 client.29509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.152+0000 7f680efbd740 20 client.29509 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.152+0000 7f680efbd740 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5635adfe3a70 0x5635adfe5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.152+0000 7f680efbd740 4 client.29509 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:39:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.152+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f67f80bb0e0 con 0x5635adfe3a70 2024-09-11T22:39:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.152+0000 7f680659c640 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5635adfe3a70 0x5635adfe5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.155+0000 7f680659c640 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5635adfe3a70 0x5635adfe5ef0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2bd07858580d555 server_cookie=3254f71c07037905 in_seq=0 out_seq=0 2024-09-11T22:39:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.155+0000 7f67edffb640 10 client.29509 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.155+0000 7f67edffb640 1 -- 172.21.15.5:0/128123808 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f67f80bb0e0 con 0x5635adfe3a70 2024-09-11T22:39:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.155+0000 7f67edffb640 10 client.29509 handle_command_reply: tid=0 2024-09-11T22:39:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.155+0000 7f680efbd740 1 client.29509 shutdown 2024-09-11T22:39:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.157+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 msgr2=0x5635adfdbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.157+0000 7f680efbd740 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 0x5635adfdbb80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009230 tx=0x7f67fc00e890 comp rx=0 tx=0).stop 2024-09-11T22:39:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.159+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 shutdown_connections 2024-09-11T22:39:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.159+0000 7f680efbd740 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5635adfe3a70 0x5635adfe5ef0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.159+0000 7f680efbd740 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5635adf69680 0x5635adfdbb80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.159+0000 7f67edffb640 0 client.29509 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.159+0000 7f680efbd740 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635adf5edc0 0x5635adf6d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.159+0000 7f680efbd740 1 --2- 172.21.15.5:0/128123808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635adf5e4a0 0x5635adf6cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.159+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 >> 172.21.15.5:0/128123808 conn(0x5635adf5aa30 msgr2=0x5635adf59b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:10.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.162+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 shutdown_connections 2024-09-11T22:39:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.162+0000 7f680efbd740 1 -- 172.21.15.5:0/128123808 wait complete. 2024-09-11T22:39:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.162+0000 7f680efbd740 20 client.29509 trim_cache size 0 max 16384 2024-09-11T22:39:10.163 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:39:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:39:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:39:10.164 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:39:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.164+0000 7f680cbe4640 1 -- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67f00a2df0 msgr2=0x7f67f00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.164+0000 7f680cbe4640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67f00a2df0 0x7f67f00a52b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f68000137f0 tx=0x7f6800013bf0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.164+0000 7f680cbe4640 1 -- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6808109b10 msgr2=0x7f680819f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.164+0000 7f680cbe4640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6808109b10 0x7f680819f040 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f67f800b3f0 tx=0x7f67f800b8b0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.167+0000 7f680cbe4640 1 -- 172.21.15.5:0/3263535049 shutdown_connections 2024-09-11T22:39:10.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.167+0000 7f680cbe4640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808114ab0 0x7f68081a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.167+0000 7f680cbe4640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6808109b10 0x7f680819f040 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.167+0000 7f680cbe4640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f67f00a2df0 0x7f67f00a52b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.167+0000 7f680cbe4640 1 --2- 172.21.15.5:0/3263535049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68081091f0 0x7f680819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:10.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.167+0000 7f680cbe4640 1 -- 172.21.15.5:0/3263535049 >> 172.21.15.5:0/3263535049 conn(0x7f680806e140 msgr2=0x7f68080708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:10.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.168+0000 7f680cbe4640 1 -- 172.21.15.5:0/3263535049 shutdown_connections 2024-09-11T22:39:10.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:10.169+0000 7f680cbe4640 1 -- 172.21.15.5:0/3263535049 wait complete. 2024-09-11T22:39:10.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 20.25 MB/sec execute 88 sec latency 15432.564 ms 2024-09-11T22:39:11.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:10 smithi005 ceph-mon[30943]: osdmap e114: 12 total, 12 up, 12 in 2024-09-11T22:39:11.077 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:39:11.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:39:11.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:10 smithi159 ceph-mon[35534]: osdmap e114: 12 total, 12 up, 12 in 2024-09-11T22:39:11.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:10 smithi172 ceph-mon[35461]: osdmap e114: 12 total, 12 up, 12 in 2024-09-11T22:39:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:11.645 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 20.02 MB/sec execute 89 sec latency 16432.631 ms 2024-09-11T22:39:11.993 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:11 smithi005 ceph-mon[30943]: pgmap v541: 97 pgs: 97 active+clean; 486 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 3.7 MiB/s wr, 190 op/s 2024-09-11T22:39:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:11 smithi159 ceph-mon[35534]: pgmap v541: 97 pgs: 97 active+clean; 486 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 3.7 MiB/s wr, 190 op/s 2024-09-11T22:39:12.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:11 smithi172 ceph-mon[35461]: pgmap v541: 97 pgs: 97 active+clean; 486 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 3.7 MiB/s wr, 190 op/s 2024-09-11T22:39:12.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.116+0000 7fc321445640 1 -- 172.21.15.5:0/2833746844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31c102d30 msgr2=0x7fc31c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:12.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.116+0000 7fc321445640 1 --2- 172.21.15.5:0/2833746844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31c102d30 0x7fc31c103110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009860 tx=0x7fc30c02a340 comp rx=0 tx=0).stop 2024-09-11T22:39:12.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.117+0000 7fc321445640 1 -- 172.21.15.5:0/2833746844 shutdown_connections 2024-09-11T22:39:12.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.117+0000 7fc321445640 1 --2- 172.21.15.5:0/2833746844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc31c1082a0 0x7fc31c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.117+0000 7fc321445640 1 --2- 172.21.15.5:0/2833746844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc31c1036e0 0x7fc31c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.117+0000 7fc321445640 1 --2- 172.21.15.5:0/2833746844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31c102d30 0x7fc31c103110 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.117+0000 7fc321445640 1 -- 172.21.15.5:0/2833746844 >> 172.21.15.5:0/2833746844 conn(0x7fc31c0fc7d0 msgr2=0x7fc31c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:12.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.117+0000 7fc321445640 1 -- 172.21.15.5:0/2833746844 shutdown_connections 2024-09-11T22:39:12.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.117+0000 7fc321445640 1 -- 172.21.15.5:0/2833746844 wait complete. 2024-09-11T22:39:12.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.118+0000 7fc321445640 1 Processor -- start 2024-09-11T22:39:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.118+0000 7fc321445640 1 -- start start 2024-09-11T22:39:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc321445640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc31c102d30 0x7fc31c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc321445640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31c1036e0 0x7fc31c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc321445640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc31c1082a0 0x7fc31c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc321445640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c1120a0 con 0x7fc31c102d30 2024-09-11T22:39:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc321445640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c111f20 con 0x7fc31c1082a0 2024-09-11T22:39:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc321445640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc31c112220 con 0x7fc31c1036e0 2024-09-11T22:39:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc31a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31c1036e0 0x7fc31c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc31a7fc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31c1036e0 0x7fc31c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:37054/0 (socket says 172.21.15.5:37054) 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc31a7fc640 1 -- 172.21.15.5:0/1745823889 learned_addr learned my addr 172.21.15.5:0/1745823889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc31b7fe640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc31c1082a0 0x7fc31c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc31a7fc640 1 -- 172.21.15.5:0/1745823889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc31c1082a0 msgr2=0x7fc31c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc31a7fc640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc31c1082a0 0x7fc31c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc31a7fc640 1 -- 172.21.15.5:0/1745823889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc31c102d30 msgr2=0x7fc31c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc31a7fc640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc31c102d30 0x7fc31c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc31a7fc640 1 -- 172.21.15.5:0/1745823889 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc31c1a6a60 con 0x7fc31c1036e0 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.119+0000 7fc31a7fc640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31c1036e0 0x7fc31c100ed0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc31000e7e0 tx=0x7fc31000eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.120+0000 7fc303fff640 1 -- 172.21.15.5:0/1745823889 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31000c8b0 con 0x7fc31c1036e0 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.120+0000 7fc303fff640 1 -- 172.21.15.5:0/1745823889 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc310005020 con 0x7fc31c1036e0 2024-09-11T22:39:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.120+0000 7fc303fff640 1 -- 172.21.15.5:0/1745823889 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc310004b60 con 0x7fc31c1036e0 2024-09-11T22:39:12.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.120+0000 7fc321445640 1 -- 172.21.15.5:0/1745823889 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31c198360 con 0x7fc31c1036e0 2024-09-11T22:39:12.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.120+0000 7fc321445640 1 -- 172.21.15.5:0/1745823889 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31c198850 con 0x7fc31c1036e0 2024-09-11T22:39:12.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.121+0000 7fc321445640 1 -- 172.21.15.5:0/1745823889 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31c103110 con 0x7fc31c1036e0 2024-09-11T22:39:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.124+0000 7fc303fff640 1 -- 172.21.15.5:0/1745823889 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc3100051d0 con 0x7fc31c1036e0 2024-09-11T22:39:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.124+0000 7fc303fff640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc3040a2e40 0x7fc3040a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.124+0000 7fc303fff640 1 -- 172.21.15.5:0/1745823889 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7fc3100d8600 con 0x7fc31c1036e0 2024-09-11T22:39:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.124+0000 7fc31affd640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc3040a2e40 0x7fc3040a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:12.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.125+0000 7fc31affd640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc3040a2e40 0x7fc3040a5300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc30c02a860 tx=0x7fc30c0058b0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:12.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.128+0000 7fc303fff640 1 -- 172.21.15.5:0/1745823889 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc31002d030 con 0x7fc31c1036e0 2024-09-11T22:39:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.355+0000 7fc321445640 1 -- 172.21.15.5:0/1745823889 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc31c101ec0 con 0x7fc31c1036e0 2024-09-11T22:39:12.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.356+0000 7fc303fff640 1 -- 172.21.15.5:0/1745823889 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11967 (secure 0 0 0) 0x7fc31c101ec0 con 0x7fc31c1036e0 2024-09-11T22:39:12.358 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:39:12.358 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"btime":"2024-09-11T22:38:14:548820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:38:13.865471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:39:12.359 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2024-09-11T22:39:12.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 -- 172.21.15.5:0/1745823889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc3040a2e40 msgr2=0x7fc3040a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:12.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc3040a2e40 0x7fc3040a5300 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc30c02a860 tx=0x7fc30c0058b0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 -- 172.21.15.5:0/1745823889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31c1036e0 msgr2=0x7fc31c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:12.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31c1036e0 0x7fc31c100ed0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc31000e7e0 tx=0x7fc31000eca0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 -- 172.21.15.5:0/1745823889 shutdown_connections 2024-09-11T22:39:12.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc3040a2e40 0x7fc3040a5300 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc31c1082a0 0x7fc31c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31c1036e0 0x7fc31c100ed0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 --2- 172.21.15.5:0/1745823889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc31c102d30 0x7fc31c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 -- 172.21.15.5:0/1745823889 >> 172.21.15.5:0/1745823889 conn(0x7fc31c0fc7d0 msgr2=0x7fc31c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 -- 172.21.15.5:0/1745823889 shutdown_connections 2024-09-11T22:39:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:12.359+0000 7fc321445640 1 -- 172.21.15.5:0/1745823889 wait complete. 2024-09-11T22:39:12.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 19.80 MB/sec execute 90 sec latency 17432.673 ms 2024-09-11T22:39:13.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:12 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1745823889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:13.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:12 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1745823889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:13.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:12 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1745823889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:13.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:39:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:13.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 19.58 MB/sec execute 91 sec latency 18432.710 ms 2024-09-11T22:39:13.947 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:13 smithi005 ceph-mon[30943]: pgmap v542: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 484 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-11T22:39:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.049+0000 7fcc77696640 1 -- 172.21.15.5:0/1758299028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70102570 msgr2=0x7fcc701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.049+0000 7fcc77696640 1 --2- 172.21.15.5:0/1758299028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70102570 0x7fcc701029b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009900 tx=0x7fcc6c02a480 comp rx=0 tx=0).stop 2024-09-11T22:39:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.049+0000 7fcc77696640 1 -- 172.21.15.5:0/1758299028 shutdown_connections 2024-09-11T22:39:14.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.049+0000 7fcc77696640 1 --2- 172.21.15.5:0/1758299028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc70102ef0 0x7fcc7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.049+0000 7fcc77696640 1 --2- 172.21.15.5:0/1758299028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70102570 0x7fcc701029b0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.049+0000 7fcc77696640 1 --2- 172.21.15.5:0/1758299028 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc70100e20 0x7fcc70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.049+0000 7fcc77696640 1 -- 172.21.15.5:0/1758299028 >> 172.21.15.5:0/1758299028 conn(0x7fcc700fc7d0 msgr2=0x7fcc700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.050+0000 7fcc77696640 1 -- 172.21.15.5:0/1758299028 shutdown_connections 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.050+0000 7fcc77696640 1 -- 172.21.15.5:0/1758299028 wait complete. 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.050+0000 7fcc77696640 1 Processor -- start 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.050+0000 7fcc77696640 1 -- start start 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc77696640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100e20 0x7fcc7019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc77696640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc70102570 0x7fcc7019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc77696640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc70102ef0 0x7fcc7019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc77696640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70110f80 con 0x7fcc70100e20 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc77696640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70110e00 con 0x7fcc70102ef0 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc77696640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70111100 con 0x7fcc70102570 2024-09-11T22:39:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc75432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100e20 0x7fcc7019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc75432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100e20 0x7fcc7019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42562/0 (socket says 172.21.15.5:42562) 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc75432640 1 -- 172.21.15.5:0/23838215 learned_addr learned my addr 172.21.15.5:0/23838215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc75c33640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc70102ef0 0x7fcc7019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc74c31640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc70102570 0x7fcc7019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc75432640 1 -- 172.21.15.5:0/23838215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc70102570 msgr2=0x7fcc7019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc75432640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc70102570 0x7fcc7019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc75432640 1 -- 172.21.15.5:0/23838215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc70102ef0 msgr2=0x7fcc7019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc75432640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc70102ef0 0x7fcc7019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc75432640 1 -- 172.21.15.5:0/23838215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc7019f700 con 0x7fcc70100e20 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc74c31640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc70102570 0x7fcc7019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc75432640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100e20 0x7fcc7019a750 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000d950 tx=0x7fcc6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc667fc640 1 -- 172.21.15.5:0/23838215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60017070 con 0x7fcc70100e20 2024-09-11T22:39:14.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc77696640 1 -- 172.21.15.5:0/23838215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc7019f9f0 con 0x7fcc70100e20 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc667fc640 1 -- 172.21.15.5:0/23838215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc60009c30 con 0x7fcc70100e20 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.051+0000 7fcc667fc640 1 -- 172.21.15.5:0/23838215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6000f6b0 con 0x7fcc70100e20 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.052+0000 7fcc77696640 1 -- 172.21.15.5:0/23838215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc701a72d0 con 0x7fcc70100e20 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.054+0000 7fcc667fc640 1 -- 172.21.15.5:0/23838215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fcc600109d0 con 0x7fcc70100e20 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.054+0000 7fcc79a6f740 1 Processor -- start 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.054+0000 7fcc79a6f740 1 -- start start 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f04090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f045d0 0x55b232f04a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f0f370 0x55b232f11760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b232e6a2f0 con 0x55b232f0f370 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b232e11cf0 con 0x55b232f05b40 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b232e662b0 con 0x55b232f045d0 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc74c31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f0f370 0x55b232f11760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc667fc640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc5c0a2cf0 0x7fcc5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc74c31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f0f370 0x55b232f11760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42564/0 (socket says 172.21.15.5:42564) 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc74c31640 1 -- 172.21.15.5:0/472142719 learned_addr learned my addr 172.21.15.5:0/472142719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc75c33640 1 --2- 172.21.15.5:0/472142719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f04090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc75432640 1 --2- 172.21.15.5:0/472142719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f045d0 0x55b232f04a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc75c33640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc5c0a2cf0 0x7fcc5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc667fc640 1 -- 172.21.15.5:0/23838215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7fcc600cf1e0 con 0x7fcc70100e20 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc74c31640 1 -- 172.21.15.5:0/472142719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f045d0 msgr2=0x55b232f04a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc74c31640 1 --2- 172.21.15.5:0/472142719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f045d0 0x55b232f04a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc74c31640 1 -- 172.21.15.5:0/472142719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 msgr2=0x55b232f04090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc74c31640 1 --2- 172.21.15.5:0/472142719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f04090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc74c31640 1 -- 172.21.15.5:0/472142719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b232df63a0 con 0x55b232f0f370 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.055+0000 7fcc75c33640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc5c0a2cf0 0x7fcc5c0a51b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009fd0 tx=0x7fcc68009a90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:14.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.056+0000 7fcc74c31640 1 --2- 172.21.15.5:0/472142719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f0f370 0x55b232f11760 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009900 tx=0x7fcc6c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37df8e2d8c9e684 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.056+0000 7fcc5b7fe640 1 -- 172.21.15.5:0/472142719 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c035030 con 0x55b232f0f370 2024-09-11T22:39:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.056+0000 7fcc5b7fe640 1 -- 172.21.15.5:0/472142719 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc6c005520 con 0x55b232f0f370 2024-09-11T22:39:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.056+0000 7fcc5b7fe640 1 -- 172.21.15.5:0/472142719 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0037f0 con 0x55b232f0f370 2024-09-11T22:39:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.056+0000 7fcc79a6f740 1 -- 172.21.15.5:0/472142719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f0f370 msgr2=0x55b232f11760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.056+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/472142719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f0f370 0x55b232f11760 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c009900 tx=0x7fcc6c0040f0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.056+0000 7fcc79a6f740 1 -- 172.21.15.5:0/472142719 shutdown_connections 2024-09-11T22:39:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.056+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/472142719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f0f370 0x55b232f11760 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.056+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/472142719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f045d0 0x55b232f04a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.056+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/472142719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f04090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.057+0000 7fcc79a6f740 1 -- 172.21.15.5:0/472142719 >> 172.21.15.5:0/472142719 conn(0x55b232ecd310 msgr2=0x55b232f00c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.057+0000 7fcc79a6f740 1 -- 172.21.15.5:0/472142719 shutdown_connections 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.057+0000 7fcc79a6f740 1 -- 172.21.15.5:0/472142719 wait complete. 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.057+0000 7fcc79a6f740 1 -- start start 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.057+0000 7fcc79a6f740 1 Processor -- start 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.057+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f045d0 0x55b232f0ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f05b40 0x55b232f0bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f0c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b232f12010 con 0x55b232f05b40 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b232e25060 con 0x55b232f045d0 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b232f12190 con 0x55b232f0f370 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc79a6f740 10 client.0 _ll_register_callbacks cb 0x55b232b788c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc74c31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f05b40 0x55b232f0bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc74c31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f05b40 0x55b232f0bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42580/0 (socket says 172.21.15.5:42580) 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc74c31640 1 -- 172.21.15.5:0/3292377752 learned_addr learned my addr 172.21.15.5:0/3292377752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc75432640 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f045d0 0x55b232f0ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc75c33640 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f0c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc74c31640 1 -- 172.21.15.5:0/3292377752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 msgr2=0x55b232f0c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc74c31640 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f0c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc74c31640 1 -- 172.21.15.5:0/3292377752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f045d0 msgr2=0x55b232f0ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc74c31640 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f045d0 0x55b232f0ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc74c31640 1 -- 172.21.15.5:0/3292377752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b232bdb3c0 con 0x55b232f05b40 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc74c31640 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f05b40 0x55b232f0bfe0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c000c00 tx=0x7fcc6c004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f416c54035afcd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc58ff9640 1 -- 172.21.15.5:0/3292377752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c035070 con 0x55b232f05b40 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc58ff9640 1 -- 172.21.15.5:0/3292377752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7fcc6c004ff0 con 0x55b232f05b40 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.058+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3292377752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc6c005520 con 0x55b232f05b40 2024-09-11T22:39:14.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc58ff9640 1 -- 172.21.15.5:0/3292377752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fcc6c0051f0 con 0x55b232f05b40 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc79a6f740 10 client.20619 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3292377752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b232f1da70 con 0x55b232f05b40 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc58ff9640 1 -- 172.21.15.5:0/3292377752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc6c005520 con 0x55b232f05b40 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc58ff9640 1 -- 172.21.15.5:0/3292377752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fcc6c0316e0 con 0x55b232f05b40 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc79a6f740 10 client.20619 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc79a6f740 10 client.20619 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc79a6f740 20 client.20619 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55b232f1edc0 0x55b232f211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc79a6f740 4 client.20619 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3292377752 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b232e77860 con 0x55b232f1edc0 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.059+0000 7fcc75432640 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55b232f1edc0 0x55b232f211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc75432640 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55b232f1edc0 0x55b232f211b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=29c20cd8deab6fdc server_cookie=a143545591a0d3d2 in_seq=0 out_seq=0 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc58ff9640 10 client.20619 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc58ff9640 1 -- 172.21.15.5:0/3292377752 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b232e77860 con 0x55b232f1edc0 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc58ff9640 10 client.20619 handle_command_reply: tid=0 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc79a6f740 1 client.20619 shutdown 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3292377752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f05b40 msgr2=0x55b232f0bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f05b40 0x55b232f0bfe0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c000c00 tx=0x7fcc6c004510 comp rx=0 tx=0).stop 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3292377752 shutdown_connections 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55b232f1edc0 0x55b232f211b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f0c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f05b40 0x55b232f0bfe0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/3292377752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f045d0 0x55b232f0ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc58ff9640 0 client.20619 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:39:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.060+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3292377752 >> 172.21.15.5:0/3292377752 conn(0x55b232ecd310 msgr2=0x55b232f06960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.061+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3292377752 shutdown_connections 2024-09-11T22:39:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.061+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3292377752 wait complete. 2024-09-11T22:39:14.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.061+0000 7fcc79a6f740 20 client.20619 trim_cache size 0 max 16384 2024-09-11T22:39:14.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:13 smithi159 ceph-mon[35534]: pgmap v542: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 484 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-11T22:39:14.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc79a6f740 1 Processor -- start 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc79a6f740 1 -- start start 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f0df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f10ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701afcb0 con 0x55b232f045d0 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701b17f0 con 0x55b232f05b40 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701b0380 con 0x55b232f0f370 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc75432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f0df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc75432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f0df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42596/0 (socket says 172.21.15.5:42596) 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc75432640 1 -- 172.21.15.5:0/3385526354 learned_addr learned my addr 172.21.15.5:0/3385526354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc74c31640 1 --2- 172.21.15.5:0/3385526354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc75c33640 1 --2- 172.21.15.5:0/3385526354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f10ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc75432640 1 -- 172.21.15.5:0/3385526354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 msgr2=0x55b232f10ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc75432640 1 --2- 172.21.15.5:0/3385526354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f10ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc75432640 1 -- 172.21.15.5:0/3385526354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 msgr2=0x55b232f10760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc75432640 1 --2- 172.21.15.5:0/3385526354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f10760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc75432640 1 -- 172.21.15.5:0/3385526354 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b232f01df0 con 0x55b232f045d0 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.074+0000 7fcc74c31640 1 --2- 172.21.15.5:0/3385526354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f10760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc75432640 1 --2- 172.21.15.5:0/3385526354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f0df50 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fcc600cc890 tx=0x7fcc6000dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccc5d93c6f7e31a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc5affd640 1 -- 172.21.15.5:0/3385526354 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60020040 con 0x55b232f045d0 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc5affd640 1 -- 172.21.15.5:0/3385526354 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc6009bd00 con 0x55b232f045d0 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc5affd640 1 -- 172.21.15.5:0/3385526354 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60097040 con 0x55b232f045d0 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3385526354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 msgr2=0x55b232f0df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/3385526354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f0df50 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fcc600cc890 tx=0x7fcc6000dfa0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3385526354 shutdown_connections 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/3385526354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f10ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/3385526354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f10760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/3385526354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f0df50 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.075+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3385526354 >> 172.21.15.5:0/3385526354 conn(0x55b232ecd310 msgr2=0x55b232f0d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3385526354 shutdown_connections 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 1 -- 172.21.15.5:0/3385526354 wait complete. 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 1 -- start start 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 1 Processor -- start 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f17430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f17970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f123e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701b1120 con 0x55b232f045d0 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701b0a50 con 0x55b232f05b40 2024-09-11T22:39:14.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcc701adcb0 con 0x55b232f0f370 2024-09-11T22:39:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc79a6f740 10 client.0 _ll_register_callbacks cb 0x7fcc68009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.076+0000 7fcc75432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f17430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc75432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f17430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42608/0 (socket says 172.21.15.5:42608) 2024-09-11T22:39:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc75432640 1 -- 172.21.15.5:0/2630042264 learned_addr learned my addr 172.21.15.5:0/2630042264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc75c33640 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc74c31640 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f17970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc75432640 1 -- 172.21.15.5:0/2630042264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 msgr2=0x55b232f123e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc75432640 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f123e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc75432640 1 -- 172.21.15.5:0/2630042264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 msgr2=0x55b232f17970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc75432640 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f17970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc75432640 1 -- 172.21.15.5:0/2630042264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b232f129d0 con 0x55b232f045d0 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc75c33640 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc75432640 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f17430 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fcc6009beb0 tx=0x7fcc60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb9f2a6aaef0b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc58ff9640 1 -- 172.21.15.5:0/2630042264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc600b9060 con 0x55b232f045d0 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc58ff9640 1 -- 172.21.15.5:0/2630042264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7fcc600ccab0 con 0x55b232f045d0 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc79a6f740 1 -- 172.21.15.5:0/2630042264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc6009bd00 con 0x55b232f045d0 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc58ff9640 1 -- 172.21.15.5:0/2630042264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fcc600b3070 con 0x55b232f045d0 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc79a6f740 10 client.20625 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:14.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.077+0000 7fcc79a6f740 1 -- 172.21.15.5:0/2630042264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b232f8a030 con 0x55b232f045d0 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc58ff9640 1 -- 172.21.15.5:0/2630042264 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc6009bd00 con 0x55b232f045d0 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc58ff9640 1 -- 172.21.15.5:0/2630042264 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fcc6009aa50 con 0x55b232f045d0 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc79a6f740 10 client.20625 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc79a6f740 10 client.20625 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc79a6f740 20 client.20625 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55b232f8a6b0 0x55b232f8caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc79a6f740 4 client.20625 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc79a6f740 1 -- 172.21.15.5:0/2630042264 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b232e71d70 con 0x55b232f8a6b0 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc74c31640 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55b232f8a6b0 0x55b232f8caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc74c31640 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55b232f8a6b0 0x55b232f8caa0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4bcfbcd137acdbc6 server_cookie=493092790302243 in_seq=0 out_seq=0 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.078+0000 7fcc58ff9640 10 client.20625 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc58ff9640 1 -- 172.21.15.5:0/2630042264 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b232e71d70 con 0x55b232f8a6b0 2024-09-11T22:39:14.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc58ff9640 10 client.20625 handle_command_reply: tid=0 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 client.20625 shutdown 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 -- 172.21.15.5:0/2630042264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 msgr2=0x55b232f17430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f17430 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fcc6009beb0 tx=0x7fcc60005880 comp rx=0 tx=0).stop 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 -- 172.21.15.5:0/2630042264 shutdown_connections 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55b232f8a6b0 0x55b232f8caa0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b232f0f370 0x55b232f123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b232f05b40 0x55b232f17970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 --2- 172.21.15.5:0/2630042264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b232f045d0 0x55b232f17430 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 -- 172.21.15.5:0/2630042264 >> 172.21.15.5:0/2630042264 conn(0x55b232ecd310 msgr2=0x55b232f0e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc58ff9640 0 client.20625 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 -- 172.21.15.5:0/2630042264 shutdown_connections 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 1 -- 172.21.15.5:0/2630042264 wait complete. 2024-09-11T22:39:14.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.079+0000 7fcc79a6f740 20 client.20625 trim_cache size 0 max 16384 2024-09-11T22:39:14.094 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:39:14.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 -- 172.21.15.5:0/23838215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc5c0a2cf0 msgr2=0x7fcc5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc5c0a2cf0 0x7fcc5c0a51b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009fd0 tx=0x7fcc68009a90 comp rx=0 tx=0).stop 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 -- 172.21.15.5:0/23838215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100e20 msgr2=0x7fcc7019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100e20 0x7fcc7019a750 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000d950 tx=0x7fcc6000de10 comp rx=0 tx=0).stop 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 -- 172.21.15.5:0/23838215 shutdown_connections 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcc5c0a2cf0 0x7fcc5c0a51b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcc70102ef0 0x7fcc7019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc70102570 0x7fcc7019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 --2- 172.21.15.5:0/23838215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100e20 0x7fcc7019a750 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 -- 172.21.15.5:0/23838215 >> 172.21.15.5:0/23838215 conn(0x7fcc700fc7d0 msgr2=0x7fcc7006a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 -- 172.21.15.5:0/23838215 shutdown_connections 2024-09-11T22:39:14.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:14.080+0000 7fcc77696640 1 -- 172.21.15.5:0/23838215 wait complete. 2024-09-11T22:39:14.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:13 smithi172 ceph-mon[35461]: pgmap v542: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 484 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-11T22:39:14.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 19.37 MB/sec execute 92 sec latency 19432.738 ms 2024-09-11T22:39:15.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:39:15.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 19.16 MB/sec execute 93 sec latency 20432.767 ms 2024-09-11T22:39:15.672 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:15.951 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:15 smithi005 ceph-mon[30943]: pgmap v543: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 484 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-11T22:39:16.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:15 smithi159 ceph-mon[35534]: pgmap v543: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 484 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-11T22:39:16.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:15 smithi172 ceph-mon[35461]: pgmap v543: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 484 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-11T22:39:16.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.304+0000 7faedece1640 1 -- 172.21.15.5:0/1588325012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed8100e20 msgr2=0x7faed8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:16.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.304+0000 7faedece1640 1 --2- 172.21.15.5:0/1588325012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed8100e20 0x7faed8101200 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7faecc009900 tx=0x7faecc02a480 comp rx=0 tx=0).stop 2024-09-11T22:39:16.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.304+0000 7faedece1640 1 -- 172.21.15.5:0/1588325012 shutdown_connections 2024-09-11T22:39:16.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.304+0000 7faedece1640 1 --2- 172.21.15.5:0/1588325012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faed8102ef0 0x7faed810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.304+0000 7faedece1640 1 --2- 172.21.15.5:0/1588325012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faed8102570 0x7faed81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.304+0000 7faedece1640 1 --2- 172.21.15.5:0/1588325012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed8100e20 0x7faed8101200 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.304+0000 7faedece1640 1 -- 172.21.15.5:0/1588325012 >> 172.21.15.5:0/1588325012 conn(0x7faed80fc7d0 msgr2=0x7faed80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.304+0000 7faedece1640 1 -- 172.21.15.5:0/1588325012 shutdown_connections 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.304+0000 7faedece1640 1 -- 172.21.15.5:0/1588325012 wait complete. 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faedece1640 1 Processor -- start 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faedece1640 1 -- start start 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faedece1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faed8100e20 0x7faed8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faedece1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed8102570 0x7faed8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faedece1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faed8102ef0 0x7faed819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faedece1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faed8110fa0 con 0x7faed8102570 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faedece1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faed8110e20 con 0x7faed8102ef0 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faedece1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faed8111120 con 0x7faed8100e20 2024-09-11T22:39:16.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faed7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed8102570 0x7faed8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faed7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed8102570 0x7faed8198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42618/0 (socket says 172.21.15.5:42618) 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faed7fff640 1 -- 172.21.15.5:0/281387947 learned_addr learned my addr 172.21.15.5:0/281387947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faedd27e640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faed8102ef0 0x7faed819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faedca7d640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faed8100e20 0x7faed8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faed7fff640 1 -- 172.21.15.5:0/281387947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faed8100e20 msgr2=0x7faed8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faed7fff640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faed8100e20 0x7faed8198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faed7fff640 1 -- 172.21.15.5:0/281387947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faed8102ef0 msgr2=0x7faed819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.305+0000 7faed7fff640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faed8102ef0 0x7faed819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.306+0000 7faed7fff640 1 -- 172.21.15.5:0/281387947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed819d540 con 0x7faed8102570 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.306+0000 7faed7fff640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed8102570 0x7faed8198ad0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7faec400c920 tx=0x7faec400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.306+0000 7faed5ffb640 1 -- 172.21.15.5:0/281387947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4007c70 con 0x7faed8102570 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.306+0000 7faed5ffb640 1 -- 172.21.15.5:0/281387947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec4007e10 con 0x7faed8102570 2024-09-11T22:39:16.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.306+0000 7faedece1640 1 -- 172.21.15.5:0/281387947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed819d830 con 0x7faed8102570 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.306+0000 7faed5ffb640 1 -- 172.21.15.5:0/281387947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4018890 con 0x7faed8102570 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.306+0000 7faedece1640 1 -- 172.21.15.5:0/281387947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed81a5110 con 0x7faed8102570 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.308+0000 7faed5ffb640 1 -- 172.21.15.5:0/281387947 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7faec4018a70 con 0x7faed8102570 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faee10ba740 1 Processor -- start 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faed5ffb640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faec80a2cf0 0x7faec80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faee10ba740 1 -- start start 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faedd27e640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faec80a2cf0 0x7faec80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faee10ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 0x559ff8928090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faed5ffb640 1 -- 172.21.15.5:0/281387947 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7faec40cf640 con 0x7faed8102570 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faee10ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff89285d0 0x559ff8928a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faee10ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8935760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faee10ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ff888e2f0 con 0x559ff8933370 2024-09-11T22:39:16.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faee10ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559ff8835cf0 con 0x559ff8929b40 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.309+0000 7faee10ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559ff888a2b0 con 0x559ff89285d0 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedd27e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8935760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedd27e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8935760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42622/0 (socket says 172.21.15.5:42622) 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedd27e640 1 -- 172.21.15.5:0/551161231 learned_addr learned my addr 172.21.15.5:0/551161231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faed7fff640 1 --2- 172.21.15.5:0/551161231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff89285d0 0x559ff8928a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedca7d640 1 --2- 172.21.15.5:0/551161231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 0x559ff8928090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedd27e640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faec80a2cf0 0x7faec80a51b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faed0009390 tx=0x7faed0009650 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedd27e640 1 -- 172.21.15.5:0/551161231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff89285d0 msgr2=0x559ff8928a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedd27e640 1 --2- 172.21.15.5:0/551161231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff89285d0 0x559ff8928a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedd27e640 1 -- 172.21.15.5:0/551161231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 msgr2=0x559ff8928090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedd27e640 1 --2- 172.21.15.5:0/551161231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 0x559ff8928090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedd27e640 1 -- 172.21.15.5:0/551161231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ff881a3a0 con 0x559ff8933370 2024-09-11T22:39:16.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faedd27e640 1 --2- 172.21.15.5:0/551161231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8935760 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7faed00187c0 tx=0x7faed0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fb6f9278c88ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faec2ffd640 1 -- 172.21.15.5:0/551161231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0017b10 con 0x559ff8933370 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faec2ffd640 1 -- 172.21.15.5:0/551161231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faed0025070 con 0x559ff8933370 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.310+0000 7faec2ffd640 1 -- 172.21.15.5:0/551161231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed001f5b0 con 0x559ff8933370 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.311+0000 7faee10ba740 1 -- 172.21.15.5:0/551161231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 msgr2=0x559ff8935760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.311+0000 7faee10ba740 1 --2- 172.21.15.5:0/551161231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8935760 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7faed00187c0 tx=0x7faed0019000 comp rx=0 tx=0).stop 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.311+0000 7faee10ba740 1 -- 172.21.15.5:0/551161231 shutdown_connections 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.311+0000 7faee10ba740 1 --2- 172.21.15.5:0/551161231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8935760 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.311+0000 7faee10ba740 1 --2- 172.21.15.5:0/551161231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff89285d0 0x559ff8928a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.311+0000 7faee10ba740 1 --2- 172.21.15.5:0/551161231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 0x559ff8928090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.311+0000 7faee10ba740 1 -- 172.21.15.5:0/551161231 >> 172.21.15.5:0/551161231 conn(0x559ff88f1310 msgr2=0x559ff8924c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.311+0000 7faee10ba740 1 -- 172.21.15.5:0/551161231 shutdown_connections 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.311+0000 7faee10ba740 1 -- 172.21.15.5:0/551161231 wait complete. 2024-09-11T22:39:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faee10ba740 1 -- start start 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faee10ba740 1 Processor -- start 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faee10ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff89285d0 0x559ff8932d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faee10ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 0x559ff892ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faee10ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8933370 0x559ff8930530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faee10ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ff8936070 con 0x559ff89285d0 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faee10ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559ff8849060 con 0x559ff8929b40 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faee10ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559ff89361f0 con 0x559ff8933370 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faee10ba740 10 client.0 _ll_register_callbacks cb 0x559ff859c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faedca7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff89285d0 0x559ff8932d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faedca7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff89285d0 0x559ff8932d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42632/0 (socket says 172.21.15.5:42632) 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faedca7d640 1 -- 172.21.15.5:0/2142188390 learned_addr learned my addr 172.21.15.5:0/2142188390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faed7fff640 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 0x559ff892ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faedca7d640 1 -- 172.21.15.5:0/2142188390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8933370 msgr2=0x559ff8930530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faedd27e640 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8933370 0x559ff8930530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faedca7d640 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8933370 0x559ff8930530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faedca7d640 1 -- 172.21.15.5:0/2142188390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 msgr2=0x559ff892ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faedca7d640 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 0x559ff892ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faedca7d640 1 -- 172.21.15.5:0/2142188390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ff85ff3c0 con 0x559ff89285d0 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.312+0000 7faed7fff640 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 0x559ff892ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faedca7d640 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff89285d0 0x559ff8932d00 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7faecc009900 tx=0x7faecc0044f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f325811c7d4222 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faeb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faeb7fff640 1 -- 172.21.15.5:0/2142188390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc035040 con 0x559ff89285d0 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faeb7fff640 1 -- 172.21.15.5:0/2142188390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7faecc003ac0 con 0x559ff89285d0 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faee10ba740 1 -- 172.21.15.5:0/2142188390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faed0025070 con 0x559ff89285d0 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faeb7fff640 1 -- 172.21.15.5:0/2142188390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7faecc003cc0 con 0x559ff89285d0 2024-09-11T22:39:16.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faee10ba740 10 client.20637 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faee10ba740 1 -- 172.21.15.5:0/2142188390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559ff8941aa0 con 0x559ff89285d0 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faeb7fff640 1 -- 172.21.15.5:0/2142188390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faed0025070 con 0x559ff89285d0 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faeb7fff640 1 -- 172.21.15.5:0/2142188390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7faecc03d5e0 con 0x559ff89285d0 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faee10ba740 10 client.20637 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faee10ba740 10 client.20637 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faee10ba740 20 client.20637 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faee10ba740 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x559ff8942f60 0x559ff8945350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faee10ba740 4 client.20637 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.313+0000 7faee10ba740 1 -- 172.21.15.5:0/2142188390 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ff889b860 con 0x559ff8942f60 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.314+0000 7faed7fff640 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x559ff8942f60 0x559ff8945350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.314+0000 7faed7fff640 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x559ff8942f60 0x559ff8945350 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=31d439280098b3f9 server_cookie=ec915892425a14ff in_seq=0 out_seq=0 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.314+0000 7faeb7fff640 10 client.20637 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:39:16.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faeb7fff640 1 -- 172.21.15.5:0/2142188390 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ff889b860 con 0x559ff8942f60 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faeb7fff640 10 client.20637 handle_command_reply: tid=0 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 client.20637 shutdown 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 -- 172.21.15.5:0/2142188390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff89285d0 msgr2=0x559ff8932d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff89285d0 0x559ff8932d00 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7faecc009900 tx=0x7faecc0044f0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 -- 172.21.15.5:0/2142188390 shutdown_connections 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8933370 0x559ff8930530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x559ff8942f60 0x559ff8945350 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff8929b40 0x559ff892ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 --2- 172.21.15.5:0/2142188390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff89285d0 0x559ff8932d00 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 -- 172.21.15.5:0/2142188390 >> 172.21.15.5:0/2142188390 conn(0x559ff88f1310 msgr2=0x559ff892a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:16.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faeb7fff640 0 client.20637 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:39:16.345 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:39:16.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 -- 172.21.15.5:0/2142188390 shutdown_connections 2024-09-11T22:39:16.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 1 -- 172.21.15.5:0/2142188390 wait complete. 2024-09-11T22:39:16.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.315+0000 7faee10ba740 20 client.20637 trim_cache size 0 max 16384 2024-09-11T22:39:16.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.328+0000 7faee10ba740 1 Processor -- start 2024-09-11T22:39:16.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.328+0000 7faee10ba740 1 -- start start 2024-09-11T22:39:16.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faee10ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff89285d0 0x559ff893f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faee10ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8929b40 0x559ff892b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faee10ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8933370 0x559ff892b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faee10ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faed81adb30 con 0x559ff8929b40 2024-09-11T22:39:16.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faee10ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faed81af670 con 0x559ff89285d0 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faee10ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faed81ae200 con 0x559ff8933370 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faed7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8929b40 0x559ff892b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faed7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8929b40 0x559ff892b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42636/0 (socket says 172.21.15.5:42636) 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faed7fff640 1 -- 172.21.15.5:0/776516203 learned_addr learned my addr 172.21.15.5:0/776516203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faed7fff640 1 -- 172.21.15.5:0/776516203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8933370 msgr2=0x559ff892b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faed7fff640 1 --2- 172.21.15.5:0/776516203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8933370 0x559ff892b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faed7fff640 1 -- 172.21.15.5:0/776516203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff89285d0 msgr2=0x559ff893f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faed7fff640 1 --2- 172.21.15.5:0/776516203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff89285d0 0x559ff893f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faed7fff640 1 -- 172.21.15.5:0/776516203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ff8924ed0 con 0x559ff8929b40 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faed7fff640 1 --2- 172.21.15.5:0/776516203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8929b40 0x559ff892b120 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7faec409b7c0 tx=0x7faec4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c458a9a0a07183c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faec27fc640 1 -- 172.21.15.5:0/776516203 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec4018d60 con 0x559ff8929b40 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faec27fc640 1 -- 172.21.15.5:0/776516203 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec400fb30 con 0x559ff8929b40 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.329+0000 7faec27fc640 1 -- 172.21.15.5:0/776516203 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec40a42a0 con 0x559ff8929b40 2024-09-11T22:39:16.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.330+0000 7faee10ba740 1 -- 172.21.15.5:0/776516203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8929b40 msgr2=0x559ff892b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:16.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.330+0000 7faee10ba740 1 --2- 172.21.15.5:0/776516203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8929b40 0x559ff892b120 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7faec409b7c0 tx=0x7faec4000bc0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.330+0000 7faee10ba740 1 -- 172.21.15.5:0/776516203 shutdown_connections 2024-09-11T22:39:16.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.330+0000 7faee10ba740 1 --2- 172.21.15.5:0/776516203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8933370 0x559ff892b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.330+0000 7faee10ba740 1 --2- 172.21.15.5:0/776516203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8929b40 0x559ff892b120 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.330+0000 7faee10ba740 1 --2- 172.21.15.5:0/776516203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff89285d0 0x559ff893f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.330+0000 7faee10ba740 1 -- 172.21.15.5:0/776516203 >> 172.21.15.5:0/776516203 conn(0x559ff88f1310 msgr2=0x559ff892f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:16.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.330+0000 7faee10ba740 1 -- 172.21.15.5:0/776516203 shutdown_connections 2024-09-11T22:39:16.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.330+0000 7faee10ba740 1 -- 172.21.15.5:0/776516203 wait complete. 2024-09-11T22:39:16.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faee10ba740 1 -- start start 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faee10ba740 1 Processor -- start 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faee10ba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff89285d0 0x559ff893bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faee10ba740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8929b40 0x559ff8937030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faee10ba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8937620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faee10ba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faed81aefa0 con 0x559ff8933370 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faee10ba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faed81ae8d0 con 0x559ff89285d0 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faee10ba740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7faed81abb30 con 0x559ff8929b40 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faee10ba740 10 client.0 _ll_register_callbacks cb 0x7faecc004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faedd27e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8937620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faedd27e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8937620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42638/0 (socket says 172.21.15.5:42638) 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faedd27e640 1 -- 172.21.15.5:0/384408843 learned_addr learned my addr 172.21.15.5:0/384408843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faedca7d640 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff89285d0 0x559ff893bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faed7fff640 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8929b40 0x559ff8937030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faedd27e640 1 -- 172.21.15.5:0/384408843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8929b40 msgr2=0x559ff8937030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faedd27e640 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8929b40 0x559ff8937030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faedd27e640 1 -- 172.21.15.5:0/384408843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff89285d0 msgr2=0x559ff893bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faedd27e640 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff89285d0 0x559ff893bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faedd27e640 1 -- 172.21.15.5:0/384408843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ff8937e50 con 0x559ff8933370 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.331+0000 7faed7fff640 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8929b40 0x559ff8937030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faedd27e640 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8937620 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7faed0004a80 tx=0x7faed0009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ecd60373f9fd9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faeb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faeb7fff640 1 -- 172.21.15.5:0/384408843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed002d070 con 0x559ff8933370 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faeb7fff640 1 -- 172.21.15.5:0/384408843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7faed0020650 con 0x559ff8933370 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faee10ba740 1 -- 172.21.15.5:0/384408843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec400fb30 con 0x559ff8933370 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faeb7fff640 1 -- 172.21.15.5:0/384408843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7faed0020850 con 0x559ff8933370 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faee10ba740 10 client.20643 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:16.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faee10ba740 1 -- 172.21.15.5:0/384408843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x559ff89adcd0 con 0x559ff8933370 2024-09-11T22:39:16.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faeb7fff640 1 -- 172.21.15.5:0/384408843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec400fb30 con 0x559ff8933370 2024-09-11T22:39:16.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faeb7fff640 1 -- 172.21.15.5:0/384408843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7faed0017070 con 0x559ff8933370 2024-09-11T22:39:16.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faee10ba740 10 client.20643 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:16.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faee10ba740 10 client.20643 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:39:16.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faee10ba740 20 client.20643 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:16.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.332+0000 7faee10ba740 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x559ff89ae0c0 0x559ff89b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:16.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faee10ba740 4 client.20643 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:39:16.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faee10ba740 1 -- 172.21.15.5:0/384408843 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ff8895d70 con 0x559ff89ae0c0 2024-09-11T22:39:16.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faedca7d640 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x559ff89ae0c0 0x559ff89b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faeb7fff640 10 client.20643 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faedca7d640 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x559ff89ae0c0 0x559ff89b04e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45be77dba0654616 server_cookie=5c6dc1eee1fc58a in_seq=0 out_seq=0 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faeb7fff640 1 -- 172.21.15.5:0/384408843 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ff8895d70 con 0x559ff89ae0c0 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faeb7fff640 10 client.20643 handle_command_reply: tid=0 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faee10ba740 1 client.20643 shutdown 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faee10ba740 1 -- 172.21.15.5:0/384408843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 msgr2=0x559ff8937620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faee10ba740 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8937620 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7faed0004a80 tx=0x7faed0009990 comp rx=0 tx=0).stop 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faee10ba740 1 -- 172.21.15.5:0/384408843 shutdown_connections 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faee10ba740 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ff8933370 0x559ff8937620 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faee10ba740 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559ff8929b40 0x559ff8937030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faee10ba740 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x559ff89ae0c0 0x559ff89b04e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.333+0000 7faee10ba740 1 --2- 172.21.15.5:0/384408843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ff89285d0 0x559ff893bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faee10ba740 1 -- 172.21.15.5:0/384408843 >> 172.21.15.5:0/384408843 conn(0x559ff88f1310 msgr2=0x559ff8931810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faeb7fff640 0 client.20643 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faee10ba740 1 -- 172.21.15.5:0/384408843 shutdown_connections 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faee10ba740 1 -- 172.21.15.5:0/384408843 wait complete. 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faee10ba740 20 client.20643 trim_cache size 0 max 16384 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faedece1640 1 -- 172.21.15.5:0/281387947 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faec80a2cf0 msgr2=0x7faec80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faedece1640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faec80a2cf0 0x7faec80a51b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7faed0009390 tx=0x7faed0009650 comp rx=0 tx=0).stop 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faedece1640 1 -- 172.21.15.5:0/281387947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed8102570 msgr2=0x7faed8198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faedece1640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed8102570 0x7faed8198ad0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7faec400c920 tx=0x7faec400cde0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faedece1640 1 -- 172.21.15.5:0/281387947 shutdown_connections 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faedece1640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7faec80a2cf0 0x7faec80a51b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faedece1640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faed8102ef0 0x7faed819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faedece1640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faed8102570 0x7faed8198ad0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faedece1640 1 --2- 172.21.15.5:0/281387947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faed8100e20 0x7faed8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.334+0000 7faedece1640 1 -- 172.21.15.5:0/281387947 >> 172.21.15.5:0/281387947 conn(0x7faed80fc7d0 msgr2=0x7faed806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.335+0000 7faedece1640 1 -- 172.21.15.5:0/281387947 shutdown_connections 2024-09-11T22:39:16.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:16.335+0000 7faedece1640 1 -- 172.21.15.5:0/281387947 wait complete. 2024-09-11T22:39:16.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:39:16.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 18.96 MB/sec execute 94 sec latency 21432.819 ms 2024-09-11T22:39:17.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:39:17.160 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:17.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 18.76 MB/sec execute 95 sec latency 22432.849 ms 2024-09-11T22:39:17.705 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:17 smithi005 ceph-mon[30943]: pgmap v544: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 484 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-11T22:39:17.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.704+0000 7fdf7ae31640 1 -- 172.21.15.5:0/4108062878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf74108290 msgr2=0x7fdf7410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:17.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.704+0000 7fdf7ae31640 1 --2- 172.21.15.5:0/4108062878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf74108290 0x7fdf7410ec30 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009cd0 tx=0x7fdf6c02a8e0 comp rx=0 tx=0).stop 2024-09-11T22:39:17.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.704+0000 7fdf7ae31640 1 -- 172.21.15.5:0/4108062878 shutdown_connections 2024-09-11T22:39:17.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.704+0000 7fdf7ae31640 1 --2- 172.21.15.5:0/4108062878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf74108290 0x7fdf7410ec30 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:17.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.704+0000 7fdf7ae31640 1 --2- 172.21.15.5:0/4108062878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf74103740 0x7fdf74107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.704+0000 7fdf7ae31640 1 --2- 172.21.15.5:0/4108062878 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf74102d90 0x7fdf74103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.704+0000 7fdf7ae31640 1 -- 172.21.15.5:0/4108062878 >> 172.21.15.5:0/4108062878 conn(0x7fdf740fc7d0 msgr2=0x7fdf740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.704+0000 7fdf7ae31640 1 -- 172.21.15.5:0/4108062878 shutdown_connections 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.704+0000 7fdf7ae31640 1 -- 172.21.15.5:0/4108062878 wait complete. 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf7ae31640 1 Processor -- start 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf7ae31640 1 -- start start 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf7ae31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf74102d90 0x7fdf74198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf7ae31640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf74103740 0x7fdf74198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf7ae31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf74108290 0x7fdf7419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf7ae31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74112090 con 0x7fdf74108290 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf7ae31640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74111f10 con 0x7fdf74103740 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf7ae31640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdf74112210 con 0x7fdf74102d90 2024-09-11T22:39:17.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf793ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf74108290 0x7fdf7419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf793ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf74108290 0x7fdf7419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42644/0 (socket says 172.21.15.5:42644) 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf793ce640 1 -- 172.21.15.5:0/990809477 learned_addr learned my addr 172.21.15.5:0/990809477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.705+0000 7fdf78bcd640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf74102d90 0x7fdf74198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf73fff640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf74103740 0x7fdf74198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf793ce640 1 -- 172.21.15.5:0/990809477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf74102d90 msgr2=0x7fdf74198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf793ce640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf74102d90 0x7fdf74198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf793ce640 1 -- 172.21.15.5:0/990809477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf74103740 msgr2=0x7fdf74198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf793ce640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf74103740 0x7fdf74198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf793ce640 1 -- 172.21.15.5:0/990809477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7419d4d0 con 0x7fdf74108290 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf793ce640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf74108290 0x7fdf7419cdf0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005cc0 tx=0x7fdf6c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf71ffb640 1 -- 172.21.15.5:0/990809477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c004040 con 0x7fdf74108290 2024-09-11T22:39:17.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf7ae31640 1 -- 172.21.15.5:0/990809477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7419d760 con 0x7fdf74108290 2024-09-11T22:39:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf71ffb640 1 -- 172.21.15.5:0/990809477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf6c0046c0 con 0x7fdf74108290 2024-09-11T22:39:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf71ffb640 1 -- 172.21.15.5:0/990809477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c046480 con 0x7fdf74108290 2024-09-11T22:39:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.706+0000 7fdf7ae31640 1 -- 172.21.15.5:0/990809477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf741a50a0 con 0x7fdf74108290 2024-09-11T22:39:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.707+0000 7fdf7ae31640 1 -- 172.21.15.5:0/990809477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf74103170 con 0x7fdf74108290 2024-09-11T22:39:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.709+0000 7fdf71ffb640 1 -- 172.21.15.5:0/990809477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fdf6c0305a0 con 0x7fdf74108290 2024-09-11T22:39:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.710+0000 7fdf71ffb640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdf640a2cf0 0x7fdf640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.710+0000 7fdf71ffb640 1 -- 172.21.15.5:0/990809477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7fdf6c02e030 con 0x7fdf74108290 2024-09-11T22:39:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.710+0000 7fdf78bcd640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdf640a2cf0 0x7fdf640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.711+0000 7fdf78bcd640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdf640a2cf0 0x7fdf640a51b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800bbe0 tx=0x7fdf6800b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.713+0000 7fdf71ffb640 1 -- 172.21.15.5:0/990809477 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf6c04b060 con 0x7fdf74108290 2024-09-11T22:39:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:17.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.933+0000 7fdf7ae31640 1 -- 172.21.15.5:0/990809477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf74199880 con 0x7fdf74108290 2024-09-11T22:39:17.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.934+0000 7fdf71ffb640 1 -- 172.21.15.5:0/990809477 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11967 (secure 0 0 0) 0x7fdf74199880 con 0x7fdf74108290 2024-09-11T22:39:17.935 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:39:17.935 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"btime":"2024-09-11T22:38:14:548820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:38:13.865471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:39:17.935 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2024-09-11T22:39:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 -- 172.21.15.5:0/990809477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdf640a2cf0 msgr2=0x7fdf640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdf640a2cf0 0x7fdf640a51b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800bbe0 tx=0x7fdf6800b390 comp rx=0 tx=0).stop 2024-09-11T22:39:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 -- 172.21.15.5:0/990809477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf74108290 msgr2=0x7fdf7419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf74108290 0x7fdf7419cdf0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c005cc0 tx=0x7fdf6c003970 comp rx=0 tx=0).stop 2024-09-11T22:39:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 -- 172.21.15.5:0/990809477 shutdown_connections 2024-09-11T22:39:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf74108290 0x7fdf7419cdf0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdf640a2cf0 0x7fdf640a51b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:17.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf74103740 0x7fdf74198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:17.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 --2- 172.21.15.5:0/990809477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf74102d90 0x7fdf74198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:17.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 -- 172.21.15.5:0/990809477 >> 172.21.15.5:0/990809477 conn(0x7fdf740fc7d0 msgr2=0x7fdf74106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:17.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 -- 172.21.15.5:0/990809477 shutdown_connections 2024-09-11T22:39:17.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:17.936+0000 7fdf7ae31640 1 -- 172.21.15.5:0/990809477 wait complete. 2024-09-11T22:39:18.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:17 smithi159 ceph-mon[35534]: pgmap v544: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 484 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-11T22:39:18.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:17 smithi172 ceph-mon[35461]: pgmap v544: 97 pgs: 6 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 88 active+clean; 484 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-11T22:39:18.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 18.56 MB/sec execute 96 sec latency 23432.894 ms 2024-09-11T22:39:18.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:18 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/990809477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:19.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:18 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/990809477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:19.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:18 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/990809477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:19.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 18.37 MB/sec execute 97 sec latency 24432.936 ms 2024-09-11T22:39:19.763 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id 12260228 --release --await' 2024-09-11T22:39:19.764 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id 12260228 --release --await 2024-09-11T22:39:19.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.898+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/1355150026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a04102570 msgr2=0x7f8a041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.898+0000 7f8a0a4f9640 1 --2- 172.21.15.5:0/1355150026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a04102570 0x7f8a041029b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009900 tx=0x7f89f802a480 comp rx=0 tx=0).stop 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.898+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/1355150026 shutdown_connections 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.898+0000 7f8a0a4f9640 1 --2- 172.21.15.5:0/1355150026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a04102ef0 0x7f8a0410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.898+0000 7f8a0a4f9640 1 --2- 172.21.15.5:0/1355150026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a04102570 0x7f8a041029b0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.898+0000 7f8a0a4f9640 1 --2- 172.21.15.5:0/1355150026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a04100e20 0x7f8a04101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.898+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/1355150026 >> 172.21.15.5:0/1355150026 conn(0x7f8a040fc7d0 msgr2=0x7f8a040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.898+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/1355150026 shutdown_connections 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.898+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/1355150026 wait complete. 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.898+0000 7f8a0a4f9640 1 Processor -- start 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a0a4f9640 1 -- start start 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a0a4f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a04100e20 0x7f8a04198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a0a4f9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a04102570 0x7f8a041989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a0a4f9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a04102ef0 0x7f8a0419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a0a4f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a04110f50 con 0x7f8a04100e20 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a0a4f9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8a04110dd0 con 0x7f8a04102570 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a0a4f9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a041110d0 con 0x7f8a04102ef0 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a04100e20 0x7f8a04198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a04100e20 0x7f8a04198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42658/0 (socket says 172.21.15.5:42658) 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a03fff640 1 -- 172.21.15.5:0/2711697041 learned_addr learned my addr 172.21.15.5:0/2711697041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a037fe640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a04102570 0x7f8a041989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a08a96640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a04102ef0 0x7f8a0419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a03fff640 1 -- 172.21.15.5:0/2711697041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a04102ef0 msgr2=0x7f8a0419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a03fff640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a04102ef0 0x7f8a0419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a03fff640 1 -- 172.21.15.5:0/2711697041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a04102570 msgr2=0x7f8a041989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.899+0000 7f8a03fff640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a04102570 0x7f8a041989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.900+0000 7f8a03fff640 1 -- 172.21.15.5:0/2711697041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0419d440 con 0x7f8a04100e20 2024-09-11T22:39:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.900+0000 7f8a03fff640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a04100e20 0x7f8a04198490 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c920 tx=0x7f89f400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.900+0000 7f8a017fa640 1 -- 172.21.15.5:0/2711697041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4007c70 con 0x7f8a04100e20 2024-09-11T22:39:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.900+0000 7f8a017fa640 1 -- 172.21.15.5:0/2711697041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f4007e10 con 0x7f8a04100e20 2024-09-11T22:39:19.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.900+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/2711697041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0419d730 con 0x7f8a04100e20 2024-09-11T22:39:19.844 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:39:19.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.900+0000 7f8a017fa640 1 -- 172.21.15.5:0/2711697041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f4018890 con 0x7f8a04100e20 2024-09-11T22:39:19.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.900+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/2711697041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a041a5010 con 0x7f8a04100e20 2024-09-11T22:39:19.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.902+0000 7f8a017fa640 1 -- 172.21.15.5:0/2711697041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f89f4018a70 con 0x7f8a04100e20 2024-09-11T22:39:19.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a0c8d2740 1 Processor -- start 2024-09-11T22:39:19.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a0c8d2740 1 -- start start 2024-09-11T22:39:19.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc45090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 0x5576ccc52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576ccbab2f0 con 0x5576ccc46b40 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5576ccb52cf0 con 0x5576ccc50370 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5576ccba72b0 con 0x5576ccc455d0 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a08a96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc45090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a08a96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc45090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42660/0 (socket says 172.21.15.5:42660) 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a08a96640 1 -- 172.21.15.5:0/4041138340 learned_addr learned my addr 172.21.15.5:0/4041138340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a037fe640 1 --2- 172.21.15.5:0/4041138340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 0x5576ccc52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a03fff640 1 --2- 172.21.15.5:0/4041138340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.903+0000 7f8a017fa640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89ec0a2cf0 0x7f89ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f8a08a96640 1 -- 172.21.15.5:0/4041138340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 msgr2=0x5576ccc45a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:19.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f8a017fa640 1 -- 172.21.15.5:0/2711697041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7f89f40cf640 con 0x7f8a04100e20 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f8a08a96640 1 --2- 172.21.15.5:0/4041138340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc45a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f8a08a96640 1 -- 172.21.15.5:0/4041138340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 msgr2=0x5576ccc52760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f8a08a96640 1 --2- 172.21.15.5:0/4041138340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 0x5576ccc52760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f8a08a96640 1 -- 172.21.15.5:0/4041138340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ccb373a0 con 0x5576ccc46b40 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f8a037fe640 1 --2- 172.21.15.5:0/4041138340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 0x5576ccc52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f8a08a96640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89ec0a2cf0 0x7f89ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f8a08a96640 1 --2- 172.21.15.5:0/4041138340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc45090 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f89fc010420 tx=0x7f89fc0108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68135832fbbb4abe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f89ea7fc640 1 -- 172.21.15.5:0/4041138340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc021070 con 0x5576ccc46b40 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f89ea7fc640 1 -- 172.21.15.5:0/4041138340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89fc01b3c0 con 0x5576ccc46b40 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.904+0000 7f89ea7fc640 1 -- 172.21.15.5:0/4041138340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00c5c0 con 0x5576ccc46b40 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/4041138340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 msgr2=0x5576ccc45090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a08a96640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89ec0a2cf0 0x7f89ec0a51b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f89fc01e000 tx=0x7f89fc01c000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/4041138340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc45090 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f89fc010420 tx=0x7f89fc0108e0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/4041138340 shutdown_connections 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/4041138340 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 0x5576ccc52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/4041138340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/4041138340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc45090 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/4041138340 >> 172.21.15.5:0/4041138340 conn(0x5576ccc0e310 msgr2=0x5576ccc41c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/4041138340 shutdown_connections 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/4041138340 wait complete. 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.905+0000 7f8a0c8d2740 1 -- start start 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a0c8d2740 1 Processor -- start 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc4fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 0x5576ccc4d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc50370 0x5576ccc4d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576ccc57480 con 0x5576ccc50370 2024-09-11T22:39:19.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5576ccb66060 con 0x5576ccc46b40 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5576ccc57600 con 0x5576ccc455d0 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a0c8d2740 10 client.0 _ll_register_callbacks cb 0x5576cc8b98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a08a96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc50370 0x5576ccc4d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a08a96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc50370 0x5576ccc4d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42668/0 (socket says 172.21.15.5:42668) 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a08a96640 1 -- 172.21.15.5:0/351297763 learned_addr learned my addr 172.21.15.5:0/351297763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a037fe640 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 0x5576ccc4d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a03fff640 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc4fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a08a96640 1 -- 172.21.15.5:0/351297763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 msgr2=0x5576ccc4fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a08a96640 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc4fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a08a96640 1 -- 172.21.15.5:0/351297763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 msgr2=0x5576ccc4d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a08a96640 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 0x5576ccc4d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.906+0000 7f8a08a96640 1 -- 172.21.15.5:0/351297763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576cc91c3c0 con 0x5576ccc50370 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f8a03fff640 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc4fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f8a037fe640 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 0x5576ccc4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f8a08a96640 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc50370 0x5576ccc4d610 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f89fc013a20 tx=0x7f89fc01bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ce20942641c386 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f89e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f89e37fe640 1 -- 172.21.15.5:0/351297763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89fc033060 con 0x5576ccc50370 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f89e37fe640 1 -- 172.21.15.5:0/351297763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7f89fc01ccc0 con 0x5576ccc50370 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/351297763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89fc01b3c0 con 0x5576ccc50370 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f89e37fe640 1 -- 172.21.15.5:0/351297763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f89fc03a050 con 0x5576ccc50370 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f8a0c8d2740 10 client.20661 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/351297763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5576ccc62f20 con 0x5576ccc50370 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.907+0000 7f89e37fe640 1 -- 172.21.15.5:0/351297763 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89fc01b3c0 con 0x5576ccc50370 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.908+0000 7f89e37fe640 1 -- 172.21.15.5:0/351297763 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f89fc00f040 con 0x5576ccc50370 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.908+0000 7f8a0c8d2740 10 client.20661 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:19.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.908+0000 7f8a0c8d2740 10 client.20661 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.908+0000 7f8a0c8d2740 20 client.20661 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.908+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5576ccc642d0 0x5576ccc666c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.908+0000 7f8a0c8d2740 4 client.20661 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.908+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/351297763 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576ccbb8860 con 0x5576ccc642d0 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.908+0000 7f8a03fff640 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5576ccc642d0 0x5576ccc666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.908+0000 7f89e37fe640 10 client.20661 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.908+0000 7f8a03fff640 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5576ccc642d0 0x5576ccc666c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfd2c6cc6eab8247 server_cookie=e62d17aa1c289361 in_seq=0 out_seq=0 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f89e37fe640 1 -- 172.21.15.5:0/351297763 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576ccbb8860 con 0x5576ccc642d0 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f89e37fe640 10 client.20661 handle_command_reply: tid=0 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 client.20661 shutdown 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/351297763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc50370 msgr2=0x5576ccc4d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc50370 0x5576ccc4d610 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f89fc013a20 tx=0x7f89fc01bce0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/351297763 shutdown_connections 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5576ccc642d0 0x5576ccc666c0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc50370 0x5576ccc4d610 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 0x5576ccc4d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f89e37fe640 0 client.20661 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/351297763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc4fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/351297763 >> 172.21.15.5:0/351297763 conn(0x5576ccc0e310 msgr2=0x5576ccc47960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/351297763 shutdown_connections 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/351297763 wait complete. 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.909+0000 7f8a0c8d2740 20 client.20661 trim_cache size 0 max 16384 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.923+0000 7f8a0c8d2740 1 Processor -- start 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.923+0000 7f8a0c8d2740 1 -- start start 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.923+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc60720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.923+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc48120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.923+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 0x5576ccc48660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.923+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a041ab900 con 0x5576ccc46b40 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.923+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8a041ad440 con 0x5576ccc50370 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.923+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a041abfd0 con 0x5576ccc455d0 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc48120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc48120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42670/0 (socket says 172.21.15.5:42670) 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a037fe640 1 -- 172.21.15.5:0/233586463 learned_addr learned my addr 172.21.15.5:0/233586463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a08a96640 1 --2- 172.21.15.5:0/233586463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 0x5576ccc48660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a03fff640 1 --2- 172.21.15.5:0/233586463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc60720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a037fe640 1 -- 172.21.15.5:0/233586463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 msgr2=0x5576ccc60720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a037fe640 1 --2- 172.21.15.5:0/233586463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc60720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a037fe640 1 -- 172.21.15.5:0/233586463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 msgr2=0x5576ccc48660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a037fe640 1 --2- 172.21.15.5:0/233586463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 0x5576ccc48660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a037fe640 1 -- 172.21.15.5:0/233586463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576ccc4b570 con 0x5576ccc46b40 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f8a037fe640 1 --2- 172.21.15.5:0/233586463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc48120 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f89f802a9a0 tx=0x7f89f80043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec435780c16ef34 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f89e9ffb640 1 -- 172.21.15.5:0/233586463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8035040 con 0x5576ccc46b40 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f89e9ffb640 1 -- 172.21.15.5:0/233586463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f80044d0 con 0x5576ccc46b40 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.924+0000 7f89e9ffb640 1 -- 172.21.15.5:0/233586463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8031b70 con 0x5576ccc46b40 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.925+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/233586463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 msgr2=0x5576ccc48120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.925+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/233586463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc48120 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f89f802a9a0 tx=0x7f89f80043e0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.925+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/233586463 shutdown_connections 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.925+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/233586463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc50370 0x5576ccc48660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.925+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/233586463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc46b40 0x5576ccc48120 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.925+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/233586463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc455d0 0x5576ccc60720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.925+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/233586463 >> 172.21.15.5:0/233586463 conn(0x5576ccc0e310 msgr2=0x5576ccc4c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.925+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/233586463 shutdown_connections 2024-09-11T22:39:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.925+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/233586463 wait complete. 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.925+0000 7f8a0c8d2740 1 -- start start 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a0c8d2740 1 Processor -- start 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc455d0 0x5576ccc5cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 0x5576ccc58080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a0c8d2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc50370 0x5576ccc585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a041acd70 con 0x5576ccc455d0 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8a041ac6a0 con 0x5576ccc46b40 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a0c8d2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8a041a9900 con 0x5576ccc50370 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a0c8d2740 10 client.0 _ll_register_callbacks cb 0x7f89fc01f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc455d0 0x5576ccc5cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc455d0 0x5576ccc5cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42676/0 (socket says 172.21.15.5:42676) 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a037fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 0x5576ccc58080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a03fff640 1 -- 172.21.15.5:0/1745139752 learned_addr learned my addr 172.21.15.5:0/1745139752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:19.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a08a96640 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc50370 0x5576ccc585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.926+0000 7f8a03fff640 1 -- 172.21.15.5:0/1745139752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc50370 msgr2=0x5576ccc585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a03fff640 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc50370 0x5576ccc585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a03fff640 1 -- 172.21.15.5:0/1745139752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 msgr2=0x5576ccc58080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a03fff640 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 0x5576ccc58080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a03fff640 1 -- 172.21.15.5:0/1745139752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576ccc58b30 con 0x5576ccc455d0 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a037fe640 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 0x5576ccc58080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a08a96640 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc50370 0x5576ccc585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a03fff640 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc455d0 0x5576ccc5cd50 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f89f40cd2f0 tx=0x7f89f40d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb373c62a20daa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f89e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f89e37fe640 1 -- 172.21.15.5:0/1745139752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f409fe70 con 0x5576ccc455d0 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f89e37fe640 1 -- 172.21.15.5:0/1745139752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7f89f4018d60 con 0x5576ccc455d0 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/1745139752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89f80044d0 con 0x5576ccc455d0 2024-09-11T22:39:19.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f89e37fe640 1 -- 172.21.15.5:0/1745139752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f89f40a4730 con 0x5576ccc455d0 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a0c8d2740 10 client.20667 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/1745139752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5576ccccebc0 con 0x5576ccc455d0 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f89e37fe640 1 -- 172.21.15.5:0/1745139752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89f80044d0 con 0x5576ccc455d0 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f89e37fe640 1 -- 172.21.15.5:0/1745139752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f89f409bd20 con 0x5576ccc455d0 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a0c8d2740 10 client.20667 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a0c8d2740 10 client.20667 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a0c8d2740 20 client.20667 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5576ccc642d0 0x5576cccd1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a0c8d2740 4 client.20667 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.927+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/1745139752 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576ccbb2d70 con 0x5576ccc642d0 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.928+0000 7f8a037fe640 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5576ccc642d0 0x5576cccd1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.928+0000 7f89e37fe640 10 client.20667 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:39:19.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.928+0000 7f8a037fe640 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5576ccc642d0 0x5576cccd1250 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6785f136f7289155 server_cookie=14839ad24644cf0b in_seq=0 out_seq=0 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.928+0000 7f89e37fe640 1 -- 172.21.15.5:0/1745139752 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576ccbb2d70 con 0x5576ccc642d0 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.928+0000 7f89e37fe640 10 client.20667 handle_command_reply: tid=0 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.928+0000 7f8a0c8d2740 1 client.20667 shutdown 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/1745139752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc455d0 msgr2=0x5576ccc5cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc455d0 0x5576ccc5cd50 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f89f40cd2f0 tx=0x7f89f40d1040 comp rx=0 tx=0).stop 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/1745139752 shutdown_connections 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x5576ccc642d0 0x5576cccd1250 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5576ccc50370 0x5576ccc585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576ccc46b40 0x5576ccc58080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f89e37fe640 0 client.20667 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 1 --2- 172.21.15.5:0/1745139752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576ccc455d0 0x5576ccc5cd50 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/1745139752 >> 172.21.15.5:0/1745139752 conn(0x5576ccc0e310 msgr2=0x5576ccc4e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:19.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/1745139752 shutdown_connections 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 1 -- 172.21.15.5:0/1745139752 wait complete. 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0c8d2740 20 client.20667 trim_cache size 0 max 16384 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/2711697041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89ec0a2cf0 msgr2=0x7f89ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89ec0a2cf0 0x7f89ec0a51b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f89fc01e000 tx=0x7f89fc01c000 comp rx=0 tx=0).stop 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/2711697041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a04100e20 msgr2=0x7f8a04198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a04100e20 0x7f8a04198490 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f89f400c920 tx=0x7f89f400cde0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/2711697041 shutdown_connections 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a04102ef0 0x7f8a0419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89ec0a2cf0 0x7f89ec0a51b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a04102570 0x7f8a041989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 --2- 172.21.15.5:0/2711697041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a04100e20 0x7f8a04198490 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/2711697041 >> 172.21.15.5:0/2711697041 conn(0x7f8a040fc7d0 msgr2=0x7f8a0406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.929+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/2711697041 shutdown_connections 2024-09-11T22:39:19.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:18.930+0000 7f8a0a4f9640 1 -- 172.21.15.5:0/2711697041 wait complete. 2024-09-11T22:39:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:19 smithi005 ceph-mon[30943]: pgmap v545: 97 pgs: 97 active+clean; 485 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 4.3 MiB/s wr, 215 op/s 2024-09-11T22:39:20.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:19 smithi159 ceph-mon[35534]: pgmap v545: 97 pgs: 97 active+clean; 485 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 4.3 MiB/s wr, 215 op/s 2024-09-11T22:39:20.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:19 smithi172 ceph-mon[35461]: pgmap v545: 97 pgs: 97 active+clean; 485 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 4.3 MiB/s wr, 215 op/s 2024-09-11T22:39:20.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57908 18.18 MB/sec execute 98 sec latency 25433.055 ms 2024-09-11T22:39:20.732 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:39:20.758 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.266+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3992003427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d8101200 msgr2=0x7ff9d8101660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.266+0000 7ff9e04ec640 1 --2- 172.21.15.5:0/3992003427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d8101200 0x7ff9d8101660 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009900 tx=0x7ff9cc02a410 comp rx=0 tx=0).stop 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.266+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3992003427 shutdown_connections 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.266+0000 7ff9e04ec640 1 --2- 172.21.15.5:0/3992003427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d8101ba0 0x7ff9d810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.266+0000 7ff9e04ec640 1 --2- 172.21.15.5:0/3992003427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d8101200 0x7ff9d8101660 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.266+0000 7ff9e04ec640 1 --2- 172.21.15.5:0/3992003427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8105ad0 0x7ff9d8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.266+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3992003427 >> 172.21.15.5:0/3992003427 conn(0x7ff9d8077210 msgr2=0x7ff9d80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.266+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3992003427 shutdown_connections 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.266+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3992003427 wait complete. 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9e04ec640 1 Processor -- start 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9e04ec640 1 -- start start 2024-09-11T22:39:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9e04ec640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8101200 0x7ff9d8197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9e04ec640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d8101ba0 0x7ff9d81977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9e04ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d8105ad0 0x7ff9d819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9e04ec640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8110ef0 con 0x7ff9d8105ad0 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9e04ec640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8110d70 con 0x7ff9d8101200 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9e04ec640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8111070 con 0x7ff9d8101ba0 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9dea89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d8105ad0 0x7ff9d819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9dea89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d8105ad0 0x7ff9d819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42700/0 (socket says 172.21.15.5:42700) 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.267+0000 7ff9dea89640 1 -- 172.21.15.5:0/3424695127 learned_addr learned my addr 172.21.15.5:0/3424695127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9de288640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8101200 0x7ff9d8197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9dda87640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d8101ba0 0x7ff9d81977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9dea89640 1 -- 172.21.15.5:0/3424695127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d8101ba0 msgr2=0x7ff9d81977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9dea89640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d8101ba0 0x7ff9d81977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9dea89640 1 -- 172.21.15.5:0/3424695127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8101200 msgr2=0x7ff9d8197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9dea89640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8101200 0x7ff9d8197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9dea89640 1 -- 172.21.15.5:0/3424695127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d819c230 con 0x7ff9d8105ad0 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9dea89640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d8105ad0 0x7ff9d819bb50 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000c970 tx=0x7ff9d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9cb7fe640 1 -- 172.21.15.5:0/3424695127 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0004230 con 0x7ff9d8105ad0 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3424695127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d819c520 con 0x7ff9d8105ad0 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9cb7fe640 1 -- 172.21.15.5:0/3424695127 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d00043d0 con 0x7ff9d8105ad0 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9cb7fe640 1 -- 172.21.15.5:0/3424695127 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000f450 con 0x7ff9d8105ad0 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.268+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3424695127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d81a3e00 con 0x7ff9d8105ad0 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.270+0000 7ff9cb7fe640 1 -- 172.21.15.5:0/3424695127 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff9d00079b0 con 0x7ff9d8105ad0 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9e28c5740 1 Processor -- start 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9e28c5740 1 -- start start 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f59339650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59339b90 0x558f59343f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59344450 0x558f59346840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558f5929c620 con 0x558f59339270 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558f59243e50 con 0x558f59344450 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558f592985e0 con 0x558f59339b90 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9dda87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f59339650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9dda87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f59339650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38578/0 (socket says 172.21.15.5:38578) 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.271+0000 7ff9dda87640 1 -- 172.21.15.5:0/4186746536 learned_addr learned my addr 172.21.15.5:0/4186746536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9de288640 1 --2- 172.21.15.5:0/4186746536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59339b90 0x558f59343f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9cb7fe640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9c40a2cf0 0x7ff9c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9dea89640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9c40a2cf0 0x7ff9c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9dda87640 1 --2- 172.21.15.5:0/4186746536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59344450 0x558f59346840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9dda87640 1 -- 172.21.15.5:0/4186746536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59339b90 msgr2=0x558f59343f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9cb7fe640 1 -- 172.21.15.5:0/3424695127 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7ff9d00cf190 con 0x7ff9d8105ad0 2024-09-11T22:39:21.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9dda87640 1 --2- 172.21.15.5:0/4186746536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59339b90 0x558f59343f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9dda87640 1 -- 172.21.15.5:0/4186746536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59344450 msgr2=0x558f59346840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9dda87640 1 --2- 172.21.15.5:0/4186746536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59344450 0x558f59346840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9dda87640 1 -- 172.21.15.5:0/4186746536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f59234d40 con 0x558f59339270 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9dda87640 1 --2- 172.21.15.5:0/4186746536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f59339650 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc02a930 tx=0x7ff9cc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d489b02964f592ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9dea89640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9c40a2cf0 0x7ff9c40a51b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0098030 tx=0x7ff9d0086040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9c8ff9640 1 -- 172.21.15.5:0/4186746536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc035070 con 0x558f59339270 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9c8ff9640 1 -- 172.21.15.5:0/4186746536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc003d50 con 0x558f59339270 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.272+0000 7ff9c8ff9640 1 -- 172.21.15.5:0/4186746536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc03b460 con 0x558f59339270 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.273+0000 7ff9e28c5740 1 -- 172.21.15.5:0/4186746536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 msgr2=0x558f59339650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.273+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/4186746536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f59339650 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc02a930 tx=0x7ff9cc005c00 comp rx=0 tx=0).stop 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.273+0000 7ff9e28c5740 1 -- 172.21.15.5:0/4186746536 shutdown_connections 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.273+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/4186746536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59344450 0x558f59346840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.273+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/4186746536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59339b90 0x558f59343f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.273+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/4186746536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f59339650 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.273+0000 7ff9e28c5740 1 -- 172.21.15.5:0/4186746536 >> 172.21.15.5:0/4186746536 conn(0x558f59335800 msgr2=0x558f59335be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.273+0000 7ff9e28c5740 1 -- 172.21.15.5:0/4186746536 shutdown_connections 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.273+0000 7ff9e28c5740 1 -- 172.21.15.5:0/4186746536 wait complete. 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9e28c5740 1 -- start start 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9e28c5740 1 Processor -- start 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339270 0x558f593413a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339b90 0x558f593418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 0x558f59341e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558f59347020 con 0x558f59339b90 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558f59224f10 con 0x558f59339270 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558f593471a0 con 0x558f59344450 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9e28c5740 10 client.0 _ll_register_callbacks cb 0x558f59248db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:21.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9de288640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339b90 0x558f593418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9de288640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339b90 0x558f593418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38580/0 (socket says 172.21.15.5:38580) 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9de288640 1 -- 172.21.15.5:0/2707543077 learned_addr learned my addr 172.21.15.5:0/2707543077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9dda87640 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339270 0x558f593413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9de288640 1 -- 172.21.15.5:0/2707543077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 msgr2=0x558f59341e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9dda87640 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 0x558f59341e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9de288640 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 0x558f59341e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9de288640 1 -- 172.21.15.5:0/2707543077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339270 msgr2=0x558f593413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9de288640 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339270 0x558f593413a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9de288640 1 -- 172.21.15.5:0/2707543077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f58fab580 con 0x558f59339b90 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.274+0000 7ff9dda87640 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 0x558f59341e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9de288640 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339b90 0x558f593418e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4010c00 tx=0x7ff9d400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce156b1a8818b53 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2707543077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400e5c0 con 0x558f59339b90 2024-09-11T22:39:21.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2707543077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7ff9d400e760 con 0x558f59339b90 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2707543077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9cc003d50 con 0x558f59339b90 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2707543077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff9d400e960 con 0x558f59339b90 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9e28c5740 10 client.20679 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2707543077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558f59075960 con 0x558f59339b90 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2707543077 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc003d50 con 0x558f59339b90 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2707543077 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7ff9d40193c0 con 0x558f59339b90 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9e28c5740 10 client.20679 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9e28c5740 10 client.20679 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9e28c5740 20 client.20679 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558f59354080 0x558f593564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9e28c5740 4 client.20679 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.275+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2707543077 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f592a9b90 con 0x558f59354080 2024-09-11T22:39:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.276+0000 7ff9dda87640 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558f59354080 0x558f593564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.276+0000 7ff9c1ffb640 10 client.20679 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.276+0000 7ff9dda87640 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558f59354080 0x558f593564b0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ceb556668bbdb19 server_cookie=2bcfe65a8cf995e6 in_seq=0 out_seq=0 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2707543077 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558f592a9b90 con 0x558f59354080 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9c1ffb640 10 client.20679 handle_command_reply: tid=0 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 client.20679 shutdown 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2707543077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339b90 msgr2=0x558f593418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339b90 0x558f593418e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4010c00 tx=0x7ff9d400f7f0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2707543077 shutdown_connections 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558f59354080 0x558f593564b0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 0x558f59341e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339b90 0x558f593418e0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2707543077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339270 0x558f593413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9c1ffb640 0 client.20679 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:21.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2707543077 >> 172.21.15.5:0/2707543077 conn(0x558f59335800 msgr2=0x558f59332750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2707543077 shutdown_connections 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2707543077 wait complete. 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.277+0000 7ff9e28c5740 20 client.20679 trim_cache size 0 max 16384 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.294+0000 7ff9e28c5740 1 Processor -- start 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.294+0000 7ff9e28c5740 1 -- start start 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.294+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339270 0x558f59343310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.294+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59339b90 0x558f5935bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.294+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59344450 0x558f5935c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.294+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81ab840 con 0x558f59344450 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.294+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81ad380 con 0x558f59339270 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.294+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81abf10 con 0x558f59339b90 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9dda87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59344450 0x558f5935c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9dda87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59344450 0x558f5935c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38586/0 (socket says 172.21.15.5:38586) 2024-09-11T22:39:21.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9dda87640 1 -- 172.21.15.5:0/18627456 learned_addr learned my addr 172.21.15.5:0/18627456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9dda87640 1 --2- 172.21.15.5:0/18627456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339270 0x558f59343310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9de288640 1 --2- 172.21.15.5:0/18627456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59339b90 0x558f5935bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9dda87640 1 -- 172.21.15.5:0/18627456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59339b90 msgr2=0x558f5935bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9dda87640 1 --2- 172.21.15.5:0/18627456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59339b90 0x558f5935bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9dda87640 1 -- 172.21.15.5:0/18627456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339270 msgr2=0x558f59343310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9dda87640 1 --2- 172.21.15.5:0/18627456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339270 0x558f59343310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9dda87640 1 -- 172.21.15.5:0/18627456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f59356b30 con 0x558f59344450 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9dda87640 1 --2- 172.21.15.5:0/18627456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59344450 0x558f5935c300 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc031ae0 tx=0x7ff9cc004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e573e960a45308 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9c3fff640 1 -- 172.21.15.5:0/18627456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc035040 con 0x558f59344450 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9c3fff640 1 -- 172.21.15.5:0/18627456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc005500 con 0x558f59344450 2024-09-11T22:39:21.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.295+0000 7ff9c3fff640 1 -- 172.21.15.5:0/18627456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc004e40 con 0x558f59344450 2024-09-11T22:39:21.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.296+0000 7ff9e28c5740 1 -- 172.21.15.5:0/18627456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59344450 msgr2=0x558f5935c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.296+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/18627456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59344450 0x558f5935c300 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc031ae0 tx=0x7ff9cc004400 comp rx=0 tx=0).stop 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.296+0000 7ff9e28c5740 1 -- 172.21.15.5:0/18627456 shutdown_connections 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.296+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/18627456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59344450 0x558f5935c300 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.296+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/18627456 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59339b90 0x558f5935bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.296+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/18627456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339270 0x558f59343310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.296+0000 7ff9e28c5740 1 -- 172.21.15.5:0/18627456 >> 172.21.15.5:0/18627456 conn(0x558f59335800 msgr2=0x558f59342c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.296+0000 7ff9e28c5740 1 -- 172.21.15.5:0/18627456 shutdown_connections 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.296+0000 7ff9e28c5740 1 -- 172.21.15.5:0/18627456 wait complete. 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9e28c5740 1 -- start start 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9e28c5740 1 Processor -- start 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f5934c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339b90 0x558f593493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9e28c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 0x558f59349900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81accb0 con 0x558f59339270 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81ac5e0 con 0x558f59339b90 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9e28c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81a9840 con 0x558f59344450 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9e28c5740 10 client.0 _ll_register_callbacks cb 0x7ff9d4010810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9dda87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f5934c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9dda87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f5934c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38594/0 (socket says 172.21.15.5:38594) 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.297+0000 7ff9dda87640 1 -- 172.21.15.5:0/2870113149 learned_addr learned my addr 172.21.15.5:0/2870113149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9dda87640 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 0x558f59349900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9dda87640 1 -- 172.21.15.5:0/2870113149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 msgr2=0x558f59349900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9dda87640 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 0x558f59349900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9dda87640 1 -- 172.21.15.5:0/2870113149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339b90 msgr2=0x558f593493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9dda87640 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339b90 0x558f593493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9dda87640 1 -- 172.21.15.5:0/2870113149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f593368b0 con 0x558f59339270 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9dda87640 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f5934c950 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc056010 tx=0x7ff9cc004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b970737825a3cf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2870113149 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc035040 con 0x558f59339270 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2870113149 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7ff9cc0056b0 con 0x558f59339270 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.298+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2870113149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9cc005500 con 0x558f59339270 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2870113149 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7ff9cc0317c0 con 0x558f59339270 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9e28c5740 10 client.20685 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2870113149 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558f593bfa40 con 0x558f59339270 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2870113149 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9cc005500 con 0x558f59339270 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2870113149 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7ff9cc005080 con 0x558f59339270 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9e28c5740 10 client.20685 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9e28c5740 10 client.20685 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9e28c5740 20 client.20685 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:21.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558f593bfd70 0x558f593c2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9e28c5740 4 client.20685 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "12260228", "release": true, "await": true}] 2024-09-11T22:39:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.299+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2870113149 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "12260228", "release": true, "await": true}) -- 0x558f592a40a0 con 0x558f593bfd70 2024-09-11T22:39:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.300+0000 7ff9de288640 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558f593bfd70 0x558f593c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.300+0000 7ff9c1ffb640 10 client.20685 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:21.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.300+0000 7ff9de288640 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558f593bfd70 0x558f593c2160 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221db73091556bd0 server_cookie=ee8308b1ff57603e in_seq=0 out_seq=0 2024-09-11T22:39:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.377+0000 7ff9c1ffb640 1 -- 172.21.15.5:0/2870113149 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558f592a40a0 con 0x558f593bfd70 2024-09-11T22:39:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.377+0000 7ff9c1ffb640 10 client.20685 handle_command_reply: tid=0 2024-09-11T22:39:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 client.20685 shutdown 2024-09-11T22:39:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2870113149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 msgr2=0x558f5934c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f5934c950 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc056010 tx=0x7ff9cc004a20 comp rx=0 tx=0).stop 2024-09-11T22:39:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2870113149 shutdown_connections 2024-09-11T22:39:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x558f593bfd70 0x558f593c2160 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558f59344450 0x558f59349900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558f59339b90 0x558f593493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 --2- 172.21.15.5:0/2870113149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558f59339270 0x558f5934c950 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9c1ffb640 0 client.20685 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2870113149 >> 172.21.15.5:0/2870113149 conn(0x558f59335800 msgr2=0x558f5933b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2870113149 shutdown_connections 2024-09-11T22:39:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 1 -- 172.21.15.5:0/2870113149 wait complete. 2024-09-11T22:39:21.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.378+0000 7ff9e28c5740 20 client.20685 trim_cache size 0 max 16384 2024-09-11T22:39:21.380 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:39:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 54, 2024-09-11T22:39:21.380 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 60, 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "12260228": { 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 60, 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.2, 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:39:21.381 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:39:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:39:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:39:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:39:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:39:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:21.382 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:21.382 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:39:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3424695127 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9c40a2cf0 msgr2=0x7ff9c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9c40a2cf0 0x7ff9c40a51b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0098030 tx=0x7ff9d0086040 comp rx=0 tx=0).stop 2024-09-11T22:39:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3424695127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d8105ad0 msgr2=0x7ff9d819bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d8105ad0 0x7ff9d819bb50 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000c970 tx=0x7ff9d000ce30 comp rx=0 tx=0).stop 2024-09-11T22:39:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3424695127 shutdown_connections 2024-09-11T22:39:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d8105ad0 0x7ff9d819bb50 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9d8101ba0 0x7ff9d81977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9c40a2cf0 0x7ff9c40a51b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 --2- 172.21.15.5:0/3424695127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9d8101200 0x7ff9d8197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3424695127 >> 172.21.15.5:0/3424695127 conn(0x7ff9d8077210 msgr2=0x7ff9d8103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3424695127 shutdown_connections 2024-09-11T22:39:21.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:21.379+0000 7ff9e04ec640 1 -- 172.21.15.5:0/3424695127 wait complete. 2024-09-11T22:39:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 -- 172.21.15.5:0/3159236257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d2810c0d0 msgr2=0x7f6d2810e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 --2- 172.21.15.5:0/3159236257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d2810c0d0 0x7f6d2810e550 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009ca0 tx=0x7f6d2002a950 comp rx=0 tx=0).stop 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 -- 172.21.15.5:0/3159236257 shutdown_connections 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 --2- 172.21.15.5:0/3159236257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d2810c0d0 0x7f6d2810e550 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 --2- 172.21.15.5:0/3159236257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d281035a0 0x7f6d2810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 --2- 172.21.15.5:0/3159236257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d28102c80 0x7f6d28103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 -- 172.21.15.5:0/3159236257 >> 172.21.15.5:0/3159236257 conn(0x7f6d28077990 msgr2=0x7f6d280fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 -- 172.21.15.5:0/3159236257 shutdown_connections 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 -- 172.21.15.5:0/3159236257 wait complete. 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 Processor -- start 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.421+0000 7f6d30077640 1 -- start start 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d30077640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d28102c80 0x7f6d28198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d30077640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d281035a0 0x7f6d281988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d30077640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d2810c0d0 0x7f6d2819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d30077640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28111eb0 con 0x7f6d281035a0 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d30077640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28111d30 con 0x7f6d2810c0d0 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d30077640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28112030 con 0x7f6d28102c80 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2d612640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d281035a0 0x7f6d281988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2d612640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d281035a0 0x7f6d281988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38618/0 (socket says 172.21.15.5:38618) 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2d612640 1 -- 172.21.15.5:0/1655775696 learned_addr learned my addr 172.21.15.5:0/1655775696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2de13640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d28102c80 0x7f6d28198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2e614640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d2810c0d0 0x7f6d2819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2d612640 1 -- 172.21.15.5:0/1655775696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d28102c80 msgr2=0x7f6d28198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2d612640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d28102c80 0x7f6d28198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2d612640 1 -- 172.21.15.5:0/1655775696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d2810c0d0 msgr2=0x7f6d2819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2d612640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d2810c0d0 0x7f6d2819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2d612640 1 -- 172.21.15.5:0/1655775696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2819d370 con 0x7f6d281035a0 2024-09-11T22:39:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d2d612640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d281035a0 0x7f6d281988c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800d950 tx=0x7f6d1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d1effd640 1 -- 172.21.15.5:0/1655775696 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d18017030 con 0x7f6d281035a0 2024-09-11T22:39:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d1effd640 1 -- 172.21.15.5:0/1655775696 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d18004020 con 0x7f6d281035a0 2024-09-11T22:39:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d30077640 1 -- 172.21.15.5:0/1655775696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2819d660 con 0x7f6d281035a0 2024-09-11T22:39:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d1effd640 1 -- 172.21.15.5:0/1655775696 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d180047a0 con 0x7f6d281035a0 2024-09-11T22:39:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.422+0000 7f6d30077640 1 -- 172.21.15.5:0/1655775696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d281a4ef0 con 0x7f6d281035a0 2024-09-11T22:39:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.425+0000 7f6d32e0f740 1 Processor -- start 2024-09-11T22:39:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.425+0000 7f6d32e0f740 1 -- start start 2024-09-11T22:39:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.425+0000 7f6d1effd640 1 -- 172.21.15.5:0/1655775696 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6d180041d0 con 0x7f6d281035a0 2024-09-11T22:39:21.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 0x55e3e95cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95cc0c0 0x55e3e95ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3e95212f0 con 0x55e3e95bca10 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e3e94c9270 con 0x55e3e95bd330 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e3e951d2b0 con 0x55e3e95cc0c0 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2e614640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2e614640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95bcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38620/0 (socket says 172.21.15.5:38620) 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2e614640 1 -- 172.21.15.5:0/1957237609 learned_addr learned my addr 172.21.15.5:0/1957237609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2de13640 1 --2- 172.21.15.5:0/1957237609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 0x55e3e95cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2d612640 1 --2- 172.21.15.5:0/1957237609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95cc0c0 0x55e3e95ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2e614640 1 -- 172.21.15.5:0/1957237609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95cc0c0 msgr2=0x55e3e95ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2e614640 1 --2- 172.21.15.5:0/1957237609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95cc0c0 0x55e3e95ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2e614640 1 -- 172.21.15.5:0/1957237609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 msgr2=0x55e3e95cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2e614640 1 --2- 172.21.15.5:0/1957237609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 0x55e3e95cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2e614640 1 -- 172.21.15.5:0/1957237609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e94c8340 con 0x55e3e95bca10 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2de13640 1 --2- 172.21.15.5:0/1957237609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 0x55e3e95cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d2e614640 1 --2- 172.21.15.5:0/1957237609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95bcdf0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009ca0 tx=0x7f6d20009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c1b128f1a5ec51 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.426+0000 7f6d13fff640 1 -- 172.21.15.5:0/1957237609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20048070 con 0x55e3e95bca10 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d1effd640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d140a2da0 0x7f6d140a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d13fff640 1 -- 172.21.15.5:0/1957237609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d2002fa00 con 0x55e3e95bca10 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d13fff640 1 -- 172.21.15.5:0/1957237609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20034040 con 0x55e3e95bca10 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d2de13640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d140a2da0 0x7f6d140a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d1effd640 1 -- 172.21.15.5:0/1655775696 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7f6d180ced80 con 0x7f6d281035a0 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1957237609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 msgr2=0x55e3e95bcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/1957237609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95bcdf0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f6d20009ca0 tx=0x7f6d20009820 comp rx=0 tx=0).stop 2024-09-11T22:39:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d2de13640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d140a2da0 0x7f6d140a5260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6d24005fd0 tx=0x7f6d2400b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1957237609 shutdown_connections 2024-09-11T22:39:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/1957237609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95cc0c0 0x55e3e95ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/1957237609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 0x55e3e95cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/1957237609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95bcdf0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.427+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1957237609 >> 172.21.15.5:0/1957237609 conn(0x55e3e95b9060 msgr2=0x55e3e95b9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.428+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1957237609 shutdown_connections 2024-09-11T22:39:21.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.428+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1957237609 wait complete. 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.428+0000 7f6d32e0f740 1 -- start start 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.428+0000 7f6d32e0f740 1 Processor -- start 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bd330 0x55e3e95c9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 0x55e3e95c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3e95cec90 con 0x55e3e95bca10 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e3e94aa290 con 0x55e3e95cc0c0 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e3e95cee10 con 0x55e3e95bd330 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d32e0f740 10 client.0 _ll_register_callbacks cb 0x55e3e94db140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2de13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2de13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38630/0 (socket says 172.21.15.5:38630) 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2de13640 1 -- 172.21.15.5:0/1823262844 learned_addr learned my addr 172.21.15.5:0/1823262844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2e614640 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 0x55e3e95c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2d612640 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bd330 0x55e3e95c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2de13640 1 -- 172.21.15.5:0/1823262844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bd330 msgr2=0x55e3e95c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2de13640 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bd330 0x55e3e95c9530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2de13640 1 -- 172.21.15.5:0/1823262844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 msgr2=0x55e3e95c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2de13640 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 0x55e3e95c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2de13640 1 -- 172.21.15.5:0/1823262844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e9234210 con 0x55e3e95bca10 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2e614640 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 0x55e3e95c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d2de13640 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95c8ff0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f6d24003e30 tx=0x7f6d24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e567dada751d613 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d117fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d117fa640 1 -- 172.21.15.5:0/1823262844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24011650 con 0x55e3e95bca10 2024-09-11T22:39:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d117fa640 1 -- 172.21.15.5:0/1823262844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7f6d240117f0 con 0x55e3e95bca10 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.429+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1823262844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d2002fa00 con 0x55e3e95bca10 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d117fa640 1 -- 172.21.15.5:0/1823262844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6d240119f0 con 0x55e3e95bca10 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d32e0f740 10 client.20697 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1823262844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e3e9300620 con 0x55e3e95bca10 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d117fa640 1 -- 172.21.15.5:0/1823262844 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d2002fa00 con 0x55e3e95bca10 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d117fa640 1 -- 172.21.15.5:0/1823262844 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f6d24026070 con 0x55e3e95bca10 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d32e0f740 10 client.20697 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d32e0f740 10 client.20697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d32e0f740 20 client.20697 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e3e95dc0e0 0x55e3e95de4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d32e0f740 4 client.20697 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1823262844 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3e952ea20 con 0x55e3e95dc0e0 2024-09-11T22:39:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.430+0000 7f6d2d612640 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e3e95dc0e0 0x55e3e95de4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.431+0000 7f6d117fa640 10 client.20697 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.431+0000 7f6d2d612640 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e3e95dc0e0 0x55e3e95de4d0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b64d8dc8631d67 server_cookie=b652b57c06c9b20d in_seq=0 out_seq=0 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.432+0000 7f6d117fa640 1 -- 172.21.15.5:0/1823262844 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3e952ea20 con 0x55e3e95dc0e0 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.432+0000 7f6d117fa640 10 client.20697 handle_command_reply: tid=0 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.432+0000 7f6d32e0f740 1 client.20697 shutdown 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.432+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1823262844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 msgr2=0x55e3e95c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.432+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95c8ff0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f6d24003e30 tx=0x7f6d24004730 comp rx=0 tx=0).stop 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.433+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1823262844 shutdown_connections 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.433+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e3e95dc0e0 0x55e3e95de4d0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.433+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 0x55e3e95c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.433+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bd330 0x55e3e95c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.433+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/1823262844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95c8ff0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.433+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1823262844 >> 172.21.15.5:0/1823262844 conn(0x55e3e95b9060 msgr2=0x55e3e95bed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.433+0000 7f6d117fa640 0 client.20697 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.433+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1823262844 shutdown_connections 2024-09-11T22:39:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.433+0000 7f6d32e0f740 1 -- 172.21.15.5:0/1823262844 wait complete. 2024-09-11T22:39:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.433+0000 7f6d32e0f740 20 client.20697 trim_cache size 0 max 16384 2024-09-11T22:39:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.447+0000 7f6d32e0f740 1 Processor -- start 2024-09-11T22:39:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d32e0f740 1 -- start start 2024-09-11T22:39:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bd330 0x55e3e95e4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 0x55e3e95c6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281ab7d0 con 0x55e3e95bca10 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281ad310 con 0x55e3e95cc0c0 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281abea0 con 0x55e3e95bd330 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2de13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2de13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95e3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38632/0 (socket says 172.21.15.5:38632) 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2de13640 1 -- 172.21.15.5:0/4099172618 learned_addr learned my addr 172.21.15.5:0/4099172618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2e614640 1 --2- 172.21.15.5:0/4099172618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 0x55e3e95c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2d612640 1 --2- 172.21.15.5:0/4099172618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bd330 0x55e3e95e4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2de13640 1 -- 172.21.15.5:0/4099172618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bd330 msgr2=0x55e3e95e4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2de13640 1 --2- 172.21.15.5:0/4099172618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bd330 0x55e3e95e4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2de13640 1 -- 172.21.15.5:0/4099172618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 msgr2=0x55e3e95c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2de13640 1 --2- 172.21.15.5:0/4099172618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 0x55e3e95c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2de13640 1 -- 172.21.15.5:0/4099172618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3e95c7f80 con 0x55e3e95bca10 2024-09-11T22:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d2de13640 1 --2- 172.21.15.5:0/4099172618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95e3e30 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f6d24003e00 tx=0x7f6d240041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e2e59f45197348 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.448+0000 7f6d137fe640 1 -- 172.21.15.5:0/4099172618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24011650 con 0x55e3e95bca10 2024-09-11T22:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d137fe640 1 -- 172.21.15.5:0/4099172618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d240117f0 con 0x55e3e95bca10 2024-09-11T22:39:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d137fe640 1 -- 172.21.15.5:0/4099172618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d24004a00 con 0x55e3e95bca10 2024-09-11T22:39:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4099172618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 msgr2=0x55e3e95e3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/4099172618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95e3e30 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f6d24003e00 tx=0x7f6d240041d0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4099172618 shutdown_connections 2024-09-11T22:39:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/4099172618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95cc0c0 0x55e3e95c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/4099172618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bd330 0x55e3e95e4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/4099172618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95bca10 0x55e3e95e3e30 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4099172618 >> 172.21.15.5:0/4099172618 conn(0x55e3e95b9060 msgr2=0x55e3e95cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4099172618 shutdown_connections 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4099172618 wait complete. 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.449+0000 7f6d32e0f740 1 -- start start 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d32e0f740 1 Processor -- start 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bca10 0x55e3e95d40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 0x55e3e95d4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d32e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95cc0c0 0x55e3e95d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281acc40 con 0x55e3e95cc0c0 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281ac570 con 0x55e3e95bd330 2024-09-11T22:39:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d32e0f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6d281a97f0 con 0x55e3e95bca10 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d32e0f740 10 client.0 _ll_register_callbacks cb 0x7f6d2400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2e614640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95cc0c0 0x55e3e95d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2e614640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95cc0c0 0x55e3e95d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38638/0 (socket says 172.21.15.5:38638) 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2e614640 1 -- 172.21.15.5:0/4239696223 learned_addr learned my addr 172.21.15.5:0/4239696223 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2d612640 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 0x55e3e95d4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2de13640 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bca10 0x55e3e95d40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2e614640 1 -- 172.21.15.5:0/4239696223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bca10 msgr2=0x55e3e95d40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2e614640 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bca10 0x55e3e95d40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2e614640 1 -- 172.21.15.5:0/4239696223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 msgr2=0x55e3e95d4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2e614640 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 0x55e3e95d4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2e614640 1 -- 172.21.15.5:0/4239696223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3e95bd160 con 0x55e3e95cc0c0 2024-09-11T22:39:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.450+0000 7f6d2de13640 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bca10 0x55e3e95d40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d2e614640 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95cc0c0 0x55e3e95d10a0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f6d20003e20 tx=0x7f6d20003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949eafba29cd5dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d117fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d117fa640 1 -- 172.21.15.5:0/4239696223 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20050020 con 0x55e3e95cc0c0 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d117fa640 1 -- 172.21.15.5:0/4239696223 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8673+0+0 (secure 0 0 0) 0x7f6d2002fa00 con 0x55e3e95cc0c0 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4239696223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d240117f0 con 0x55e3e95cc0c0 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d117fa640 1 -- 172.21.15.5:0/4239696223 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6d2005b050 con 0x55e3e95cc0c0 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d32e0f740 10 client.20703 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4239696223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e3e96471c0 con 0x55e3e95cc0c0 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d117fa640 1 -- 172.21.15.5:0/4239696223 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d240117f0 con 0x55e3e95cc0c0 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d117fa640 1 -- 172.21.15.5:0/4239696223 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f6d200574b0 con 0x55e3e95cc0c0 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d32e0f740 10 client.20703 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d32e0f740 10 client.20703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d32e0f740 20 client.20703 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e3e9647520 0x55e3e9649910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d32e0f740 4 client.20703 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:39:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4239696223 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3e9528d70 con 0x55e3e9647520 2024-09-11T22:39:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.451+0000 7f6d2de13640 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e3e9647520 0x55e3e9649910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.452+0000 7f6d2de13640 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e3e9647520 0x55e3e9649910 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bab9f26d20d0e8e1 server_cookie=e651820dc9203ec9 in_seq=0 out_seq=0 2024-09-11T22:39:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.452+0000 7f6d117fa640 10 client.20703 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.579+0000 7f6d117fa640 1 -- 172.21.15.5:0/4239696223 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3e9528d70 con 0x55e3e9647520 2024-09-11T22:39:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.579+0000 7f6d117fa640 10 client.20703 handle_command_reply: tid=0 2024-09-11T22:39:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.579+0000 7f6d32e0f740 1 client.20703 shutdown 2024-09-11T22:39:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4239696223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95cc0c0 msgr2=0x55e3e95d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95cc0c0 0x55e3e95d10a0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f6d20003e20 tx=0x7f6d20003db0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4239696223 shutdown_connections 2024-09-11T22:39:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e3e9647520 0x55e3e9649910 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3e95cc0c0 0x55e3e95d10a0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d117fa640 0 client.20703 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e3e95bd330 0x55e3e95d4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 1 --2- 172.21.15.5:0/4239696223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e3e95bca10 0x55e3e95d40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4239696223 >> 172.21.15.5:0/4239696223 conn(0x55e3e95b9060 msgr2=0x55e3e95b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4239696223 shutdown_connections 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 1 -- 172.21.15.5:0/4239696223 wait complete. 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d32e0f740 20 client.20703 trim_cache size 0 max 16384 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:39:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:39:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d30077640 1 -- 172.21.15.5:0/1655775696 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d140a2da0 msgr2=0x7f6d140a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d30077640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d140a2da0 0x7f6d140a5260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6d24005fd0 tx=0x7f6d2400b390 comp rx=0 tx=0).stop 2024-09-11T22:39:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d30077640 1 -- 172.21.15.5:0/1655775696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d281035a0 msgr2=0x7f6d281988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d30077640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d281035a0 0x7f6d281988c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800d950 tx=0x7f6d1800de10 comp rx=0 tx=0).stop 2024-09-11T22:39:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d30077640 1 -- 172.21.15.5:0/1655775696 shutdown_connections 2024-09-11T22:39:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d30077640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6d140a2da0 0x7f6d140a5260 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.580+0000 7f6d30077640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6d2810c0d0 0x7f6d2819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.581+0000 7f6d30077640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d281035a0 0x7f6d281988c0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.581+0000 7f6d30077640 1 --2- 172.21.15.5:0/1655775696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d28102c80 0x7f6d28198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.581+0000 7f6d30077640 1 -- 172.21.15.5:0/1655775696 >> 172.21.15.5:0/1655775696 conn(0x7f6d28077990 msgr2=0x7f6d280fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.581+0000 7f6d30077640 1 -- 172.21.15.5:0/1655775696 shutdown_connections 2024-09-11T22:39:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:21.581+0000 7f6d30077640 1 -- 172.21.15.5:0/1655775696 wait complete. 2024-09-11T22:39:21.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57954 18.04 MB/sec execute 99 sec latency 26202.521 ms 2024-09-11T22:39:22.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:21 smithi005 ceph-mon[30943]: pgmap v546: 97 pgs: 97 active+clean; 485 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 4.1 MiB/s wr, 204 op/s 2024-09-11T22:39:22.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:21 smithi159 ceph-mon[35534]: pgmap v546: 97 pgs: 97 active+clean; 485 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 4.1 MiB/s wr, 204 op/s 2024-09-11T22:39:22.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:21 smithi172 ceph-mon[35461]: pgmap v546: 97 pgs: 97 active+clean; 485 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 4.1 MiB/s wr, 204 op/s 2024-09-11T22:39:22.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58265 18.04 MB/sec execute 100 sec latency 196.530 ms 2024-09-11T22:39:22.837 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '12260228', total seconds elapsed: 30.6 2024-09-11T22:39:22.837 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.25 2024-09-11T22:39:22.837 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:39:23.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.097+0000 7f852a5df640 1 -- 172.21.15.5:0/3868907109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f852410c2f0 msgr2=0x7f852410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:23.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.097+0000 7f852a5df640 1 --2- 172.21.15.5:0/3868907109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f852410c2f0 0x7f852410e790 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f851c009be0 tx=0x7f851c02a6c0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.097+0000 7f852a5df640 1 -- 172.21.15.5:0/3868907109 shutdown_connections 2024-09-11T22:39:23.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.097+0000 7f852a5df640 1 --2- 172.21.15.5:0/3868907109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f852410c2f0 0x7f852410e790 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.097+0000 7f852a5df640 1 --2- 172.21.15.5:0/3868907109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85241037a0 0x7f852410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.097+0000 7f852a5df640 1 --2- 172.21.15.5:0/3868907109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8524102df0 0x7f85241031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.097+0000 7f852a5df640 1 -- 172.21.15.5:0/3868907109 >> 172.21.15.5:0/3868907109 conn(0x7f85240fc7d0 msgr2=0x7f85240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:23.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.098+0000 7f852a5df640 1 -- 172.21.15.5:0/3868907109 shutdown_connections 2024-09-11T22:39:23.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.098+0000 7f852a5df640 1 -- 172.21.15.5:0/3868907109 wait complete. 2024-09-11T22:39:23.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.098+0000 7f852a5df640 1 Processor -- start 2024-09-11T22:39:23.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f852a5df640 1 -- start start 2024-09-11T22:39:23.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f852a5df640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8524102df0 0x7f8524196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f852a5df640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85241037a0 0x7f85241968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f852a5df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f852410c2f0 0x7f852419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f852a5df640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f852410fee0 con 0x7f852410c2f0 2024-09-11T22:39:23.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f852a5df640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f852410fd60 con 0x7f8524102df0 2024-09-11T22:39:23.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f852a5df640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8524110060 con 0x7f85241037a0 2024-09-11T22:39:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f851bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85241037a0 0x7f85241968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f8523fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8524102df0 0x7f8524196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f8528b7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f852410c2f0 0x7f852419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f851bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85241037a0 0x7f85241968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59006/0 (socket says 172.21.15.5:59006) 2024-09-11T22:39:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f8528b7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f852410c2f0 0x7f852419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38662/0 (socket says 172.21.15.5:38662) 2024-09-11T22:39:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f8523fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8524102df0 0x7f8524196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44442/0 (socket says 172.21.15.5:44442) 2024-09-11T22:39:23.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.099+0000 7f8528b7c640 1 -- 172.21.15.5:0/3470716374 learned_addr learned my addr 172.21.15.5:0/3470716374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.100+0000 7f8528b7c640 1 -- 172.21.15.5:0/3470716374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85241037a0 msgr2=0x7f85241968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.100+0000 7f8528b7c640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85241037a0 0x7f85241968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.100+0000 7f8528b7c640 1 -- 172.21.15.5:0/3470716374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8524102df0 msgr2=0x7f8524196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.100+0000 7f8528b7c640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8524102df0 0x7f8524196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.100+0000 7f8528b7c640 1 -- 172.21.15.5:0/3470716374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852419b330 con 0x7f852410c2f0 2024-09-11T22:39:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.100+0000 7f8528b7c640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f852410c2f0 0x7f852419ac50 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f851c009b30 tx=0x7f851c0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.100+0000 7f8521ffb640 1 -- 172.21.15.5:0/3470716374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c041020 con 0x7f852410c2f0 2024-09-11T22:39:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.100+0000 7f8521ffb640 1 -- 172.21.15.5:0/3470716374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f851c003e50 con 0x7f852410c2f0 2024-09-11T22:39:23.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.100+0000 7f8521ffb640 1 -- 172.21.15.5:0/3470716374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c036040 con 0x7f852410c2f0 2024-09-11T22:39:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.100+0000 7f852a5df640 1 -- 172.21.15.5:0/3470716374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852419b5c0 con 0x7f852410c2f0 2024-09-11T22:39:23.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.101+0000 7f852a5df640 1 -- 172.21.15.5:0/3470716374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852419ba50 con 0x7f852410c2f0 2024-09-11T22:39:23.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.104+0000 7f852c9b8740 1 Processor -- start 2024-09-11T22:39:23.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.104+0000 7f8521ffb640 1 -- 172.21.15.5:0/3470716374 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f851c044020 con 0x7f852410c2f0 2024-09-11T22:39:23.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f852c9b8740 1 -- start start 2024-09-11T22:39:23.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ad5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad55d0 0x55ae56ad5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ae0370 0x55ae56ae2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f852c9b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae56a3b2f0 con 0x55ae56ae0370 2024-09-11T22:39:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f852c9b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae569e2cf0 con 0x55ae56ad55d0 2024-09-11T22:39:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f852c9b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ae56a372b0 con 0x55ae56ad6b40 2024-09-11T22:39:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f851bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ad5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f8523fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad55d0 0x55ae56ad5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f851bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ad5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59022/0 (socket says 172.21.15.5:59022) 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f851bfff640 1 -- 172.21.15.5:0/3624935318 learned_addr learned my addr 172.21.15.5:0/3624935318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.105+0000 7f851bfff640 1 --2- 172.21.15.5:0/3624935318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ae0370 0x55ae56ae2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f851bfff640 1 -- 172.21.15.5:0/3624935318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 msgr2=0x55ae56ad5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f851bfff640 1 --2- 172.21.15.5:0/3624935318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ad5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f851bfff640 1 -- 172.21.15.5:0/3624935318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad55d0 msgr2=0x55ae56ad5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f851bfff640 1 --2- 172.21.15.5:0/3624935318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad55d0 0x55ae56ad5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f851bfff640 1 -- 172.21.15.5:0/3624935318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae569c73a0 con 0x55ae56ae0370 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f8523fff640 1 --2- 172.21.15.5:0/3624935318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad55d0 0x55ae56ad5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f851bfff640 1 --2- 172.21.15.5:0/3624935318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ae0370 0x55ae56ae2760 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f850c013780 tx=0x7f850c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a520237f2ec5e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f851a7fc640 1 -- 172.21.15.5:0/3624935318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c013050 con 0x55ae56ae0370 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f8521ffb640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85100a2e40 0x7f85100a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f851a7fc640 1 -- 172.21.15.5:0/3624935318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850c0131f0 con 0x55ae56ae0370 2024-09-11T22:39:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f851a7fc640 1 -- 172.21.15.5:0/3624935318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c0055d0 con 0x55ae56ae0370 2024-09-11T22:39:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f8521ffb640 1 -- 172.21.15.5:0/3470716374 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f851c0f0010 con 0x7f852410c2f0 2024-09-11T22:39:23.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.106+0000 7f8523fff640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85100a2e40 0x7f85100a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.107+0000 7f852c9b8740 1 -- 172.21.15.5:0/3624935318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ae0370 msgr2=0x55ae56ae2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.107+0000 7f852c9b8740 1 --2- 172.21.15.5:0/3624935318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ae0370 0x55ae56ae2760 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f850c013780 tx=0x7f850c009260 comp rx=0 tx=0).stop 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.107+0000 7f8523fff640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85100a2e40 0x7f85100a5300 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55ae56ad2ac0 tx=0x7f851400f040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.107+0000 7f852c9b8740 1 -- 172.21.15.5:0/3624935318 shutdown_connections 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.107+0000 7f852c9b8740 1 --2- 172.21.15.5:0/3624935318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ae0370 0x55ae56ae2760 secure :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f850c013780 tx=0x7f850c009260 comp rx=0 tx=0).stop 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.107+0000 7f852c9b8740 1 --2- 172.21.15.5:0/3624935318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad55d0 0x55ae56ad5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.107+0000 7f852c9b8740 1 --2- 172.21.15.5:0/3624935318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ad5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.107+0000 7f852c9b8740 1 -- 172.21.15.5:0/3624935318 >> 172.21.15.5:0/3624935318 conn(0x55ae56a9e310 msgr2=0x55ae56ad1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.108+0000 7f852c9b8740 1 -- 172.21.15.5:0/3624935318 shutdown_connections 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.108+0000 7f852c9b8740 1 -- 172.21.15.5:0/3624935318 wait complete. 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.108+0000 7f852c9b8740 1 -- start start 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.108+0000 7f852c9b8740 1 Processor -- start 2024-09-11T22:39:23.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56add200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad6b40 0x55ae56add740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56addc80 0x55ae56aec520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f852c9b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae56ae3070 con 0x55ae56ad55d0 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f852c9b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae569f6060 con 0x55ae56ad6b40 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f852c9b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ae56ae31f0 con 0x55ae56addc80 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f852c9b8740 10 client.0 _ll_register_callbacks cb 0x55ae567498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f8528b7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56addc80 0x55ae56aec520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f8528b7c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56addc80 0x55ae56aec520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59034/0 (socket says 172.21.15.5:59034) 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f8528b7c640 1 -- 172.21.15.5:0/1249592151 learned_addr learned my addr 172.21.15.5:0/1249592151 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f8523fff640 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56add200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.109+0000 7f851bfff640 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad6b40 0x55ae56add740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f8528b7c640 1 -- 172.21.15.5:0/1249592151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad6b40 msgr2=0x55ae56add740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f8528b7c640 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad6b40 0x55ae56add740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f8528b7c640 1 -- 172.21.15.5:0/1249592151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 msgr2=0x55ae56add200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f8528b7c640 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56add200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f8528b7c640 1 -- 172.21.15.5:0/1249592151 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae567ac3c0 con 0x55ae56addc80 2024-09-11T22:39:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f8523fff640 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56add200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:39:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f8528b7c640 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56addc80 0x55ae56aec520 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f851c0b8e90 tx=0x7f851c0bc0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9b29c8e93f964 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:23.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f85037fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:39:23.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f85037fe640 1 -- 172.21.15.5:0/1249592151 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c0bcbc0 con 0x55ae56addc80 2024-09-11T22:39:23.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f85037fe640 1 -- 172.21.15.5:0/1249592151 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f851c0c4ca0 con 0x55ae56addc80 2024-09-11T22:39:23.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.110+0000 7f852c9b8740 1 -- 172.21.15.5:0/1249592151 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850c0131f0 con 0x55ae56addc80 2024-09-11T22:39:23.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.111+0000 7f85037fe640 1 -- 172.21.15.5:0/1249592151 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f851c0c4f00 con 0x55ae56addc80 2024-09-11T22:39:23.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.111+0000 7f852c9b8740 10 client.29602 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.111+0000 7f852c9b8740 1 -- 172.21.15.5:0/1249592151 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ae56ae3070 con 0x55ae56addc80 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.111+0000 7f85037fe640 1 -- 172.21.15.5:0/1249592151 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f850c0131f0 con 0x55ae56addc80 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.111+0000 7f85037fe640 1 -- 172.21.15.5:0/1249592151 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f851c0d5200 con 0x55ae56addc80 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.111+0000 7f852c9b8740 10 client.29602 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.112+0000 7f852c9b8740 10 client.29602 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.112+0000 7f852c9b8740 20 client.29602 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.112+0000 7f852c9b8740 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ae56af0730 0x55ae56af2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.112+0000 7f852c9b8740 4 client.29602 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.112+0000 7f852c9b8740 1 -- 172.21.15.5:0/1249592151 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae56a48860 con 0x55ae56af0730 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.112+0000 7f8523fff640 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ae56af0730 0x55ae56af2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.113+0000 7f85037fe640 10 client.29602 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.113+0000 7f8523fff640 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ae56af0730 0x55ae56af2b60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60557a171630bab5 server_cookie=d38b965d94518b3 in_seq=0 out_seq=0 2024-09-11T22:39:23.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.113+0000 7f85037fe640 1 -- 172.21.15.5:0/1249592151 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae56a48860 con 0x55ae56af0730 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.113+0000 7f85037fe640 10 client.29602 handle_command_reply: tid=0 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.113+0000 7f852c9b8740 1 client.29602 shutdown 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.113+0000 7f852c9b8740 1 -- 172.21.15.5:0/1249592151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56addc80 msgr2=0x55ae56aec520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.113+0000 7f852c9b8740 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56addc80 0x55ae56aec520 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f851c0b8e90 tx=0x7f851c0bc0c0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.114+0000 7f852c9b8740 1 -- 172.21.15.5:0/1249592151 shutdown_connections 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.114+0000 7f852c9b8740 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ae56af0730 0x55ae56af2b60 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.114+0000 7f852c9b8740 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56addc80 0x55ae56aec520 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.114+0000 7f852c9b8740 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56ad6b40 0x55ae56add740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.114+0000 7f852c9b8740 1 --2- 172.21.15.5:0/1249592151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56add200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.114+0000 7f852c9b8740 1 -- 172.21.15.5:0/1249592151 >> 172.21.15.5:0/1249592151 conn(0x55ae56a9e310 msgr2=0x55ae56ad7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.114+0000 7f85037fe640 0 client.29602 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.114+0000 7f852c9b8740 1 -- 172.21.15.5:0/1249592151 shutdown_connections 2024-09-11T22:39:23.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.114+0000 7f852c9b8740 1 -- 172.21.15.5:0/1249592151 wait complete. 2024-09-11T22:39:23.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.115+0000 7f852c9b8740 20 client.29602 trim_cache size 0 max 16384 2024-09-11T22:39:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.130+0000 7f852c9b8740 1 Processor -- start 2024-09-11T22:39:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.130+0000 7f852c9b8740 1 -- start start 2024-09-11T22:39:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.130+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad55d0 0x55ae56adbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.130+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad6b40 0x55ae56ae1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.130+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 0x55ae56ae1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.130+0000 7f852c9b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85241ab780 con 0x55ae56ad6b40 2024-09-11T22:39:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.130+0000 7f852c9b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85241ad2c0 con 0x55ae56addc80 2024-09-11T22:39:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f852c9b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85241abe50 con 0x55ae56ad55d0 2024-09-11T22:39:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f851bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad6b40 0x55ae56ae1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f851bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad6b40 0x55ae56ae1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38684/0 (socket says 172.21.15.5:38684) 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f8528b7c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 0x55ae56ae1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f851bfff640 1 -- 172.21.15.5:0/2818972559 learned_addr learned my addr 172.21.15.5:0/2818972559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f8523fff640 1 --2- 172.21.15.5:0/2818972559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad55d0 0x55ae56adbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f8528b7c640 1 -- 172.21.15.5:0/2818972559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad55d0 msgr2=0x55ae56adbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f8528b7c640 1 --2- 172.21.15.5:0/2818972559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad55d0 0x55ae56adbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f8528b7c640 1 -- 172.21.15.5:0/2818972559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad6b40 msgr2=0x55ae56ae1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f8528b7c640 1 --2- 172.21.15.5:0/2818972559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad6b40 0x55ae56ae1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f8528b7c640 1 -- 172.21.15.5:0/2818972559 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae56af31b0 con 0x55ae56addc80 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f851bfff640 1 --2- 172.21.15.5:0/2818972559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad6b40 0x55ae56ae1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f8523fff640 1 --2- 172.21.15.5:0/2818972559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad55d0 0x55ae56adbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:23.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f8528b7c640 1 --2- 172.21.15.5:0/2818972559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 0x55ae56ae1ca0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f851c0bc1c0 tx=0x7f851c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8111e91579756f01 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.131+0000 7f8519ffb640 1 -- 172.21.15.5:0/2818972559 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c0bcbc0 con 0x55ae56addc80 2024-09-11T22:39:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f8519ffb640 1 -- 172.21.15.5:0/2818972559 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f851c0c0870 con 0x55ae56addc80 2024-09-11T22:39:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f8519ffb640 1 -- 172.21.15.5:0/2818972559 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c0bb910 con 0x55ae56addc80 2024-09-11T22:39:23.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f852c9b8740 1 -- 172.21.15.5:0/2818972559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 msgr2=0x55ae56ae1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f852c9b8740 1 --2- 172.21.15.5:0/2818972559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 0x55ae56ae1ca0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f851c0bc1c0 tx=0x7f851c009500 comp rx=0 tx=0).stop 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f852c9b8740 1 -- 172.21.15.5:0/2818972559 shutdown_connections 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f852c9b8740 1 --2- 172.21.15.5:0/2818972559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 0x55ae56ae1ca0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f852c9b8740 1 --2- 172.21.15.5:0/2818972559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad6b40 0x55ae56ae1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f852c9b8740 1 --2- 172.21.15.5:0/2818972559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad55d0 0x55ae56adbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f852c9b8740 1 -- 172.21.15.5:0/2818972559 >> 172.21.15.5:0/2818972559 conn(0x55ae56a9e310 msgr2=0x55ae56adf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f852c9b8740 1 -- 172.21.15.5:0/2818972559 shutdown_connections 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.132+0000 7f852c9b8740 1 -- 172.21.15.5:0/2818972559 wait complete. 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.133+0000 7f852c9b8740 1 -- start start 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.133+0000 7f852c9b8740 1 Processor -- start 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.133+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56ae8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.133+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ae89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.133+0000 7f852c9b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 0x55ae56ae1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.133+0000 7f852c9b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85241acbf0 con 0x55ae56ad55d0 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.133+0000 7f852c9b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85241ac520 con 0x55ae56addc80 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.133+0000 7f852c9b8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f85241a9780 con 0x55ae56ad6b40 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f852c9b8740 10 client.0 _ll_register_callbacks cb 0x7f851c031440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f8523fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56ae8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f851bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ae89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f8528b7c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 0x55ae56ae1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f851bfff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ae89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59064/0 (socket says 172.21.15.5:59064) 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f8523fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56ae8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38700/0 (socket says 172.21.15.5:38700) 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f851bfff640 1 -- 172.21.15.5:0/934877687 learned_addr learned my addr 172.21.15.5:0/934877687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f8523fff640 1 -- 172.21.15.5:0/934877687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 msgr2=0x55ae56ae89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:23.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f8523fff640 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ae89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f8523fff640 1 -- 172.21.15.5:0/934877687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 msgr2=0x55ae56ae1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f8523fff640 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 0x55ae56ae1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f8523fff640 1 -- 172.21.15.5:0/934877687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae56ad63a0 con 0x55ae56ad55d0 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f851bfff640 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ae89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f8528b7c640 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 0x55ae56ae1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f8523fff640 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56ae8480 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f851c0b81a0 tx=0x7f8514004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8871f64741f9dd06 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.134+0000 7f85037fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.135+0000 7f85037fe640 1 -- 172.21.15.5:0/934877687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514026cb0 con 0x55ae56ad55d0 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.135+0000 7f85037fe640 1 -- 172.21.15.5:0/934877687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f8514009040 con 0x55ae56ad55d0 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.135+0000 7f852c9b8740 1 -- 172.21.15.5:0/934877687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f851c0c0870 con 0x55ae56ad55d0 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.135+0000 7f85037fe640 1 -- 172.21.15.5:0/934877687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f8514026e50 con 0x55ae56ad55d0 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.135+0000 7f852c9b8740 10 client.20718 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:23.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.135+0000 7f852c9b8740 1 -- 172.21.15.5:0/934877687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55ae56b5b650 con 0x55ae56ad55d0 2024-09-11T22:39:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.135+0000 7f85037fe640 1 -- 172.21.15.5:0/934877687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85140035f0 con 0x55ae56ad55d0 2024-09-11T22:39:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.135+0000 7f85037fe640 1 -- 172.21.15.5:0/934877687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f8514023040 con 0x55ae56ad55d0 2024-09-11T22:39:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.135+0000 7f852c9b8740 10 client.20718 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f852c9b8740 10 client.20718 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:39:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f852c9b8740 20 client.20718 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f852c9b8740 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ae56b5be20 0x55ae56b5e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f852c9b8740 4 client.20718 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:39:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f852c9b8740 1 -- 172.21.15.5:0/934877687 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f851c0c0870 con 0x55ae56b5be20 2024-09-11T22:39:23.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f851bfff640 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ae56b5be20 0x55ae56b5e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f85037fe640 10 client.20718 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:39:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f851bfff640 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ae56b5be20 0x55ae56b5e240 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e20fc230e8f0ad89 server_cookie=d3e909dc816c1918 in_seq=0 out_seq=0 2024-09-11T22:39:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f85037fe640 1 -- 172.21.15.5:0/934877687 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f851c0c0870 con 0x55ae56b5be20 2024-09-11T22:39:23.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f85037fe640 10 client.20718 handle_command_reply: tid=0 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.136+0000 7f852c9b8740 1 client.20718 shutdown 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 1 -- 172.21.15.5:0/934877687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 msgr2=0x55ae56ae8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56ae8480 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f851c0b81a0 tx=0x7f8514004730 comp rx=0 tx=0).stop 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 1 -- 172.21.15.5:0/934877687 shutdown_connections 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55ae56b5be20 0x55ae56b5e240 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae56addc80 0x55ae56ae1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ae56ad6b40 0x55ae56ae89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 1 --2- 172.21.15.5:0/934877687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae56ad55d0 0x55ae56ae8480 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 1 -- 172.21.15.5:0/934877687 >> 172.21.15.5:0/934877687 conn(0x55ae56a9e310 msgr2=0x55ae56ad0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f85037fe640 0 client.20718 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 1 -- 172.21.15.5:0/934877687 shutdown_connections 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 1 -- 172.21.15.5:0/934877687 wait complete. 2024-09-11T22:39:23.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.137+0000 7f852c9b8740 20 client.20718 trim_cache size 0 max 16384 2024-09-11T22:39:23.143 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:39:23.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 -- 172.21.15.5:0/3470716374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85100a2e40 msgr2=0x7f85100a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:23.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85100a2e40 0x7f85100a5300 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55ae56ad2ac0 tx=0x7f851400f040 comp rx=0 tx=0).stop 2024-09-11T22:39:23.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 -- 172.21.15.5:0/3470716374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f852410c2f0 msgr2=0x7f852419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f852410c2f0 0x7f852419ac50 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f851c009b30 tx=0x7f851c0305d0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 -- 172.21.15.5:0/3470716374 shutdown_connections 2024-09-11T22:39:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f852410c2f0 0x7f852419ac50 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85241037a0 0x7f85241968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f85100a2e40 0x7f85100a5300 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 --2- 172.21.15.5:0/3470716374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8524102df0 0x7f8524196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 -- 172.21.15.5:0/3470716374 >> 172.21.15.5:0/3470716374 conn(0x7f85240fc7d0 msgr2=0x7f85240fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 -- 172.21.15.5:0/3470716374 shutdown_connections 2024-09-11T22:39:23.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:23.138+0000 7f852a5df640 1 -- 172.21.15.5:0/3470716374 wait complete. 2024-09-11T22:39:23.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58728 18.05 MB/sec execute 101 sec latency 139.371 ms 2024-09-11T22:39:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:23 smithi005 ceph-mon[30943]: pgmap v547: 97 pgs: 97 active+clean; 474 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-09-11T22:39:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:23 smithi005 ceph-mon[30943]: osdmap e115: 12 total, 12 up, 12 in 2024-09-11T22:39:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:23 smithi172 ceph-mon[35461]: pgmap v547: 97 pgs: 97 active+clean; 474 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-09-11T22:39:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:23 smithi172 ceph-mon[35461]: osdmap e115: 12 total, 12 up, 12 in 2024-09-11T22:39:23.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:39:24.005 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:23 smithi159 ceph-mon[35534]: pgmap v547: 97 pgs: 97 active+clean; 474 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-09-11T22:39:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:23 smithi159 ceph-mon[35534]: osdmap e115: 12 total, 12 up, 12 in 2024-09-11T22:39:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.547+0000 7f89dee7e640 1 -- 172.21.15.5:0/1985294041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d8102ef0 msgr2=0x7f89d810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.547+0000 7f89dee7e640 1 --2- 172.21.15.5:0/1985294041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d8102ef0 0x7f89d810da40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009ca0 tx=0x7f89d002a950 comp rx=0 tx=0).stop 2024-09-11T22:39:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.547+0000 7f89dee7e640 1 -- 172.21.15.5:0/1985294041 shutdown_connections 2024-09-11T22:39:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.547+0000 7f89dee7e640 1 --2- 172.21.15.5:0/1985294041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d8102ef0 0x7f89d810da40 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.547+0000 7f89dee7e640 1 --2- 172.21.15.5:0/1985294041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89d8102570 0x7f89d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.547+0000 7f89dee7e640 1 --2- 172.21.15.5:0/1985294041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89d8100e20 0x7f89d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.547+0000 7f89dee7e640 1 -- 172.21.15.5:0/1985294041 >> 172.21.15.5:0/1985294041 conn(0x7f89d80fc7d0 msgr2=0x7f89d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.547+0000 7f89dee7e640 1 -- 172.21.15.5:0/1985294041 shutdown_connections 2024-09-11T22:39:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.547+0000 7f89dee7e640 1 -- 172.21.15.5:0/1985294041 wait complete. 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dee7e640 1 Processor -- start 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dee7e640 1 -- start start 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dee7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d8100e20 0x7f89d81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dee7e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89d8102570 0x7f89d8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dee7e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89d8102ef0 0x7f89d819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dee7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8110f80 con 0x7f89d8100e20 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dee7e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8110e00 con 0x7f89d8102ef0 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dee7e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f89d8111100 con 0x7f89d8102570 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dcc1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d8100e20 0x7f89d81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dcc1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d8100e20 0x7f89d81984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38720/0 (socket says 172.21.15.5:38720) 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dcc1a640 1 -- 172.21.15.5:0/2555561295 learned_addr learned my addr 172.21.15.5:0/2555561295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89d7fff640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89d8102570 0x7f89d8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:24.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.548+0000 7f89dd41b640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89d8102ef0 0x7f89d819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89dcc1a640 1 -- 172.21.15.5:0/2555561295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89d8102570 msgr2=0x7f89d8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89dcc1a640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89d8102570 0x7f89d8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89dcc1a640 1 -- 172.21.15.5:0/2555561295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89d8102ef0 msgr2=0x7f89d819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89dcc1a640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89d8102ef0 0x7f89d819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89dcc1a640 1 -- 172.21.15.5:0/2555561295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d819d490 con 0x7f89d8100e20 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89dcc1a640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d8100e20 0x7f89d81984e0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f89c800e9f0 tx=0x7f89c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89d5ffb640 1 -- 172.21.15.5:0/2555561295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c800cd80 con 0x7f89d8100e20 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89dee7e640 1 -- 172.21.15.5:0/2555561295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d819d780 con 0x7f89d8100e20 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89d5ffb640 1 -- 172.21.15.5:0/2555561295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89c8005500 con 0x7f89d8100e20 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89d5ffb640 1 -- 172.21.15.5:0/2555561295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8010690 con 0x7f89d8100e20 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.549+0000 7f89dee7e640 1 -- 172.21.15.5:0/2555561295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d81a5060 con 0x7f89d8100e20 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.551+0000 7f89dee7e640 1 -- 172.21.15.5:0/2555561295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d8101200 con 0x7f89d8100e20 2024-09-11T22:39:24.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.552+0000 7f89d5ffb640 1 -- 172.21.15.5:0/2555561295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f89c80046c0 con 0x7f89d8100e20 2024-09-11T22:39:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.553+0000 7f89d5ffb640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89c40a2cf0 0x7f89c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.553+0000 7f89d5ffb640 1 -- 172.21.15.5:0/2555561295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f89c80cf8a0 con 0x7f89d8100e20 2024-09-11T22:39:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.553+0000 7f89d7fff640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89c40a2cf0 0x7f89c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.553+0000 7f89d7fff640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89c40a2cf0 0x7f89c40a51b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f89d81052b0 tx=0x7f89c0005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.558+0000 7f89d5ffb640 1 -- 172.21.15.5:0/2555561295 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89c809c450 con 0x7f89d8100e20 2024-09-11T22:39:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:24.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 59175 18.12 MB/sec execute 102 sec latency 184.550 ms 2024-09-11T22:39:24.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.864+0000 7f89dee7e640 1 -- 172.21.15.5:0/2555561295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89d810a0a0 con 0x7f89d8100e20 2024-09-11T22:39:24.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.865+0000 7f89d5ffb640 1 -- 172.21.15.5:0/2555561295 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11967 (secure 0 0 0) 0x7f89c809bd50 con 0x7f89d8100e20 2024-09-11T22:39:24.866 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:39:24.866 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"btime":"2024-09-11T22:38:14:548820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:38:13.865471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:39:24.867 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2024-09-11T22:39:24.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.868+0000 7f89dee7e640 1 -- 172.21.15.5:0/2555561295 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89c40a2cf0 msgr2=0x7f89c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.868+0000 7f89dee7e640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89c40a2cf0 0x7f89c40a51b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f89d81052b0 tx=0x7f89c0005f90 comp rx=0 tx=0).stop 2024-09-11T22:39:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.868+0000 7f89dee7e640 1 -- 172.21.15.5:0/2555561295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d8100e20 msgr2=0x7f89d81984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.868+0000 7f89dee7e640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d8100e20 0x7f89d81984e0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f89c800e9f0 tx=0x7f89c800edf0 comp rx=0 tx=0).stop 2024-09-11T22:39:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.868+0000 7f89dee7e640 1 -- 172.21.15.5:0/2555561295 shutdown_connections 2024-09-11T22:39:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.868+0000 7f89dee7e640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f89c40a2cf0 0x7f89c40a51b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:24.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.868+0000 7f89dee7e640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f89d8102ef0 0x7f89d819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:24.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.868+0000 7f89dee7e640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89d8102570 0x7f89d8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:24.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.868+0000 7f89dee7e640 1 --2- 172.21.15.5:0/2555561295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89d8100e20 0x7f89d81984e0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:24.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.868+0000 7f89dee7e640 1 -- 172.21.15.5:0/2555561295 >> 172.21.15.5:0/2555561295 conn(0x7f89d80fc7d0 msgr2=0x7f89d806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:24.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.869+0000 7f89dee7e640 1 -- 172.21.15.5:0/2555561295 shutdown_connections 2024-09-11T22:39:24.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:24.869+0000 7f89dee7e640 1 -- 172.21.15.5:0/2555561295 wait complete. 2024-09-11T22:39:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.371+0000 7fb15ec37640 1 -- 172.21.15.5:0/109493494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1581009d0 msgr2=0x7fb158100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.371+0000 7fb15ec37640 1 --2- 172.21.15.5:0/109493494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1581009d0 0x7fb158100db0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fb148009900 tx=0x7fb14802a410 comp rx=0 tx=0).stop 2024-09-11T22:39:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.371+0000 7fb15ec37640 1 -- 172.21.15.5:0/109493494 shutdown_connections 2024-09-11T22:39:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.371+0000 7fb15ec37640 1 --2- 172.21.15.5:0/109493494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb15810e4f0 0x7fb158110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.371+0000 7fb15ec37640 1 --2- 172.21.15.5:0/109493494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb158101380 0x7fb15810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.371+0000 7fb15ec37640 1 --2- 172.21.15.5:0/109493494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1581009d0 0x7fb158100db0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.371+0000 7fb15ec37640 1 -- 172.21.15.5:0/109493494 >> 172.21.15.5:0/109493494 conn(0x7fb1580fc7d0 msgr2=0x7fb1580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.371+0000 7fb15ec37640 1 -- 172.21.15.5:0/109493494 shutdown_connections 2024-09-11T22:39:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.371+0000 7fb15ec37640 1 -- 172.21.15.5:0/109493494 wait complete. 2024-09-11T22:39:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.372+0000 7fb15ec37640 1 Processor -- start 2024-09-11T22:39:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.372+0000 7fb15ec37640 1 -- start start 2024-09-11T22:39:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb15ec37640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1581009d0 0x7fb1581a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb15ec37640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb158101380 0x7fb1581a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb15ec37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb15810e4f0 0x7fb15819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb15ec37640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581142c0 con 0x7fb15810e4f0 2024-09-11T22:39:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb157fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb158101380 0x7fb1581a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb15ec37640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb158114140 con 0x7fb1581009d0 2024-09-11T22:39:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb15ec37640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb158114440 con 0x7fb158101380 2024-09-11T22:39:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb157fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb158101380 0x7fb1581a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59102/0 (socket says 172.21.15.5:59102) 2024-09-11T22:39:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb157fff640 1 -- 172.21.15.5:0/2697404217 learned_addr learned my addr 172.21.15.5:0/2697404217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb15d1d4640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb15810e4f0 0x7fb15819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb157fff640 1 -- 172.21.15.5:0/2697404217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1581009d0 msgr2=0x7fb1581a03f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:39:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb157fff640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1581009d0 0x7fb1581a03f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb157fff640 1 -- 172.21.15.5:0/2697404217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb15810e4f0 msgr2=0x7fb15819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb157fff640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb15810e4f0 0x7fb15819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.373+0000 7fb157fff640 1 -- 172.21.15.5:0/2697404217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb15819ace0 con 0x7fb158101380 2024-09-11T22:39:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.374+0000 7fb157fff640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb158101380 0x7fb1581a0930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00eb90 tx=0x7fb14c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.374+0000 7fb155ffb640 1 -- 172.21.15.5:0/2697404217 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c004d20 con 0x7fb158101380 2024-09-11T22:39:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.374+0000 7fb155ffb640 1 -- 172.21.15.5:0/2697404217 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14c00ee90 con 0x7fb158101380 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.374+0000 7fb155ffb640 1 -- 172.21.15.5:0/2697404217 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c010850 con 0x7fb158101380 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.374+0000 7fb15ec37640 1 -- 172.21.15.5:0/2697404217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb15819afd0 con 0x7fb158101380 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.374+0000 7fb15ec37640 1 -- 172.21.15.5:0/2697404217 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1581a72c0 con 0x7fb158101380 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.377+0000 7fb155ffb640 1 -- 172.21.15.5:0/2697404217 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb14c005020 con 0x7fb158101380 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.377+0000 7fb161010740 1 Processor -- start 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.377+0000 7fb161010740 1 -- start start 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.377+0000 7fb161010740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 0x5620361fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.377+0000 7fb161010740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361fd5d0 0x5620361fda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.377+0000 7fb161010740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562036208370 0x56203620a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.377+0000 7fb161010740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620361632f0 con 0x5620361fd5d0 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.377+0000 7fb161010740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56203610acf0 con 0x562036208370 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.377+0000 7fb161010740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56203615f2b0 con 0x5620361feb40 2024-09-11T22:39:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.377+0000 7fb15d1d4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 0x5620361fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb15d1d4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 0x5620361fd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59106/0 (socket says 172.21.15.5:59106) 2024-09-11T22:39:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb15d1d4640 1 -- 172.21.15.5:0/2886693783 learned_addr learned my addr 172.21.15.5:0/2886693783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/2886693783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361fd5d0 0x5620361fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb157fff640 1 --2- 172.21.15.5:0/2886693783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562036208370 0x56203620a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb155ffb640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1400a2cf0 0x7fb1400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb15c9d3640 1 -- 172.21.15.5:0/2886693783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 msgr2=0x5620361fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/2886693783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 0x5620361fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb15c9d3640 1 -- 172.21.15.5:0/2886693783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562036208370 msgr2=0x56203620a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/2886693783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562036208370 0x56203620a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb15c9d3640 1 -- 172.21.15.5:0/2886693783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620360ef3a0 con 0x5620361fd5d0 2024-09-11T22:39:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb15d1d4640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1400a2cf0 0x7fb1400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/2886693783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361fd5d0 0x5620361fda10 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fb14802a930 tx=0x7fb148009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fc67cb8414821f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.378+0000 7fb155ffb640 1 -- 172.21.15.5:0/2697404217 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fb14c0cfc30 con 0x7fb158101380 2024-09-11T22:39:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.379+0000 7fb13effd640 1 -- 172.21.15.5:0/2886693783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14803e030 con 0x5620361fd5d0 2024-09-11T22:39:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.379+0000 7fb13effd640 1 -- 172.21.15.5:0/2886693783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1480035f0 con 0x5620361fd5d0 2024-09-11T22:39:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.379+0000 7fb13effd640 1 -- 172.21.15.5:0/2886693783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148035040 con 0x5620361fd5d0 2024-09-11T22:39:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.379+0000 7fb161010740 1 -- 172.21.15.5:0/2886693783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361fd5d0 msgr2=0x5620361fda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.379+0000 7fb161010740 1 --2- 172.21.15.5:0/2886693783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361fd5d0 0x5620361fda10 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fb14802a930 tx=0x7fb148009590 comp rx=0 tx=0).stop 2024-09-11T22:39:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.379+0000 7fb15d1d4640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1400a2cf0 0x7fb1400a51b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb1500045e0 tx=0x7fb1500046c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.380+0000 7fb161010740 1 -- 172.21.15.5:0/2886693783 shutdown_connections 2024-09-11T22:39:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.380+0000 7fb161010740 1 --2- 172.21.15.5:0/2886693783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562036208370 0x56203620a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.380+0000 7fb161010740 1 --2- 172.21.15.5:0/2886693783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361fd5d0 0x5620361fda10 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.380+0000 7fb161010740 1 --2- 172.21.15.5:0/2886693783 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 0x5620361fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.380+0000 7fb161010740 1 -- 172.21.15.5:0/2886693783 >> 172.21.15.5:0/2886693783 conn(0x5620361c6310 msgr2=0x5620361f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.380+0000 7fb161010740 1 -- 172.21.15.5:0/2886693783 shutdown_connections 2024-09-11T22:39:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.380+0000 7fb161010740 1 -- 172.21.15.5:0/2886693783 wait complete. 2024-09-11T22:39:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.380+0000 7fb161010740 1 -- start start 2024-09-11T22:39:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.380+0000 7fb161010740 1 Processor -- start 2024-09-11T22:39:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb161010740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x562036207d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb161010740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361feb40 0x562036205040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb161010740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562036208370 0x562036205650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb161010740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56203620b010 con 0x5620361feb40 2024-09-11T22:39:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb161010740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56203611e060 con 0x562036208370 2024-09-11T22:39:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb161010740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56203620b190 con 0x5620361fd5d0 2024-09-11T22:39:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb161010740 10 client.0 _ll_register_callbacks cb 0x562035e718c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb157fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361feb40 0x562036205040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb15c9d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x562036207d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb157fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361feb40 0x562036205040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38762/0 (socket says 172.21.15.5:38762) 2024-09-11T22:39:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb157fff640 1 -- 172.21.15.5:0/4063960718 learned_addr learned my addr 172.21.15.5:0/4063960718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.381+0000 7fb15d1d4640 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562036208370 0x562036205650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb15c9d3640 1 -- 172.21.15.5:0/4063960718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562036208370 msgr2=0x562036205650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562036208370 0x562036205650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb15c9d3640 1 -- 172.21.15.5:0/4063960718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361feb40 msgr2=0x562036205040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361feb40 0x562036205040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb15c9d3640 1 -- 172.21.15.5:0/4063960718 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562035ed43c0 con 0x5620361fd5d0 2024-09-11T22:39:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x562036207d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb148006010 tx=0x7fb1480043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce4829650e0e0a47 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb137fff640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:39:25.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb137fff640 1 -- 172.21.15.5:0/4063960718 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148035070 con 0x5620361fd5d0 2024-09-11T22:39:25.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb161010740 1 -- 172.21.15.5:0/4063960718 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1480035f0 con 0x5620361fd5d0 2024-09-11T22:39:25.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb137fff640 1 -- 172.21.15.5:0/4063960718 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fb1480039a0 con 0x5620361fd5d0 2024-09-11T22:39:25.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.382+0000 7fb137fff640 1 -- 172.21.15.5:0/4063960718 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb14803e040 con 0x5620361fd5d0 2024-09-11T22:39:25.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb161010740 10 client.29626 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:25.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb161010740 1 -- 172.21.15.5:0/4063960718 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56203620b010 con 0x5620361fd5d0 2024-09-11T22:39:25.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb137fff640 1 -- 172.21.15.5:0/4063960718 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1480035f0 con 0x5620361fd5d0 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb137fff640 1 -- 172.21.15.5:0/4063960718 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fb148043410 con 0x5620361fd5d0 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb161010740 10 client.29626 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb161010740 10 client.29626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb161010740 20 client.29626 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb161010740 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562036217c60 0x56203621a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb161010740 4 client.29626 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb161010740 1 -- 172.21.15.5:0/4063960718 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562036170860 con 0x562036217c60 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.383+0000 7fb157fff640 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562036217c60 0x56203621a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.384+0000 7fb137fff640 10 client.29626 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.384+0000 7fb157fff640 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562036217c60 0x56203621a050 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3acf0d4af3dc48a server_cookie=4818a733f4b14195 in_seq=0 out_seq=0 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb137fff640 1 -- 172.21.15.5:0/4063960718 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562036170860 con 0x562036217c60 2024-09-11T22:39:25.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb137fff640 10 client.29626 handle_command_reply: tid=0 2024-09-11T22:39:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 client.29626 shutdown 2024-09-11T22:39:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 -- 172.21.15.5:0/4063960718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 msgr2=0x562036207d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x562036207d90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb148006010 tx=0x7fb1480043b0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 -- 172.21.15.5:0/4063960718 shutdown_connections 2024-09-11T22:39:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562036217c60 0x56203621a050 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562036208370 0x562036205650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620361feb40 0x562036205040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 --2- 172.21.15.5:0/4063960718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x562036207d90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 -- 172.21.15.5:0/4063960718 >> 172.21.15.5:0/4063960718 conn(0x5620361c6310 msgr2=0x5620361ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb137fff640 0 client.29626 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 -- 172.21.15.5:0/4063960718 shutdown_connections 2024-09-11T22:39:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.387+0000 7fb161010740 1 -- 172.21.15.5:0/4063960718 wait complete. 2024-09-11T22:39:25.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.388+0000 7fb161010740 20 client.29626 trim_cache size 0 max 16384 2024-09-11T22:39:25.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.406+0000 7fb161010740 1 Processor -- start 2024-09-11T22:39:25.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.406+0000 7fb161010740 1 -- start start 2024-09-11T22:39:25.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.406+0000 7fb161010740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x5620362142b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.406+0000 7fb161010740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361feb40 0x5620361f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.406+0000 7fb161010740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562036208370 0x562036209760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.406+0000 7fb161010740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581afc20 con 0x562036208370 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.406+0000 7fb161010740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b1760 con 0x5620361feb40 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.406+0000 7fb161010740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b02f0 con 0x5620361fd5d0 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb157fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361feb40 0x5620361f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb15c9d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x5620362142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb15c9d3640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x5620362142b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59130/0 (socket says 172.21.15.5:59130) 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb157fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361feb40 0x5620361f8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44552/0 (socket says 172.21.15.5:44552) 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb15c9d3640 1 -- 172.21.15.5:0/3007958941 learned_addr learned my addr 172.21.15.5:0/3007958941 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb15d1d4640 1 --2- 172.21.15.5:0/3007958941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562036208370 0x562036209760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb157fff640 1 -- 172.21.15.5:0/3007958941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 msgr2=0x5620362142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb157fff640 1 --2- 172.21.15.5:0/3007958941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x5620362142b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb157fff640 1 -- 172.21.15.5:0/3007958941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562036208370 msgr2=0x562036209760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb157fff640 1 --2- 172.21.15.5:0/3007958941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562036208370 0x562036209760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb157fff640 1 -- 172.21.15.5:0/3007958941 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562036203590 con 0x5620361feb40 2024-09-11T22:39:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/3007958941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x5620362142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb157fff640 1 --2- 172.21.15.5:0/3007958941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361feb40 0x5620361f8bf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0049d0 tx=0x7fb14c0bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad511308162f5748 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.407+0000 7fb13e7fc640 1 -- 172.21.15.5:0/3007958941 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c09bbc0 con 0x5620361feb40 2024-09-11T22:39:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb13e7fc640 1 -- 172.21.15.5:0/3007958941 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14c09c780 con 0x5620361feb40 2024-09-11T22:39:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb13e7fc640 1 -- 172.21.15.5:0/3007958941 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14c09b9e0 con 0x5620361feb40 2024-09-11T22:39:25.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb161010740 1 -- 172.21.15.5:0/3007958941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361feb40 msgr2=0x5620361f8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:25.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb161010740 1 --2- 172.21.15.5:0/3007958941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361feb40 0x5620361f8bf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0049d0 tx=0x7fb14c0bc040 comp rx=0 tx=0).stop 2024-09-11T22:39:25.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb161010740 1 -- 172.21.15.5:0/3007958941 shutdown_connections 2024-09-11T22:39:25.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb161010740 1 --2- 172.21.15.5:0/3007958941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562036208370 0x562036209760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb161010740 1 --2- 172.21.15.5:0/3007958941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361feb40 0x5620361f8bf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb161010740 1 --2- 172.21.15.5:0/3007958941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361fd5d0 0x5620362142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb161010740 1 -- 172.21.15.5:0/3007958941 >> 172.21.15.5:0/3007958941 conn(0x5620361c6310 msgr2=0x5620361f84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:25.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb161010740 1 -- 172.21.15.5:0/3007958941 shutdown_connections 2024-09-11T22:39:25.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.408+0000 7fb161010740 1 -- 172.21.15.5:0/3007958941 wait complete. 2024-09-11T22:39:25.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.409+0000 7fb161010740 1 -- start start 2024-09-11T22:39:25.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.409+0000 7fb161010740 1 Processor -- start 2024-09-11T22:39:25.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.409+0000 7fb161010740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361fd5d0 0x5620362028d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.409+0000 7fb161010740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 0x562036210420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.409+0000 7fb161010740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562036208370 0x562036210960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.409+0000 7fb161010740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b1090 con 0x562036208370 2024-09-11T22:39:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.409+0000 7fb161010740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581b09c0 con 0x5620361fd5d0 2024-09-11T22:39:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.409+0000 7fb161010740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb1581adc20 con 0x5620361feb40 2024-09-11T22:39:25.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.409+0000 7fb161010740 10 client.0 _ll_register_callbacks cb 0x7fb14c0d35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb157fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 0x562036210420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb15c9d3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361fd5d0 0x5620362028d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb15c9d3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361fd5d0 0x5620362028d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44564/0 (socket says 172.21.15.5:44564) 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb157fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 0x562036210420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59144/0 (socket says 172.21.15.5:59144) 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb15c9d3640 1 -- 172.21.15.5:0/1447513052 learned_addr learned my addr 172.21.15.5:0/1447513052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb15d1d4640 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562036208370 0x562036210960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb15c9d3640 1 -- 172.21.15.5:0/1447513052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 msgr2=0x562036210420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 0x562036210420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb15c9d3640 1 -- 172.21.15.5:0/1447513052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562036208370 msgr2=0x562036210960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562036208370 0x562036210960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb15c9d3640 1 -- 172.21.15.5:0/1447513052 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562036210ef0 con 0x5620361fd5d0 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb15c9d3640 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361fd5d0 0x5620362028d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb148043860 tx=0x7fb1480058c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f43bb1b4253446b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:25.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb137fff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb137fff640 1 -- 172.21.15.5:0/1447513052 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb148057070 con 0x5620361fd5d0 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.410+0000 7fb137fff640 1 -- 172.21.15.5:0/1447513052 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fb148043a50 con 0x5620361fd5d0 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.411+0000 7fb161010740 1 -- 172.21.15.5:0/1447513052 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb14c09c780 con 0x5620361fd5d0 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.411+0000 7fb137fff640 1 -- 172.21.15.5:0/1447513052 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb148063050 con 0x5620361fd5d0 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.411+0000 7fb161010740 10 client.29801 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.411+0000 7fb161010740 1 -- 172.21.15.5:0/1447513052 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562036282710 con 0x5620361fd5d0 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.411+0000 7fb137fff640 1 -- 172.21.15.5:0/1447513052 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb14c09c780 con 0x5620361fd5d0 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.411+0000 7fb137fff640 1 -- 172.21.15.5:0/1447513052 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fb148031920 con 0x5620361fd5d0 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.411+0000 7fb161010740 10 client.29801 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.411+0000 7fb161010740 10 client.29801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.411+0000 7fb161010740 20 client.29801 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.411+0000 7fb161010740 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562036282d40 0x562036285160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:25.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.412+0000 7fb161010740 4 client.29801 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.412+0000 7fb161010740 1 -- 172.21.15.5:0/1447513052 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56203616ad70 con 0x562036282d40 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.412+0000 7fb157fff640 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562036282d40 0x562036285160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.412+0000 7fb157fff640 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562036282d40 0x562036285160 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ca1d8de18fa4ef server_cookie=ab51f6c3f6fa5d94 in_seq=0 out_seq=0 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.412+0000 7fb137fff640 10 client.29801 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.415+0000 7fb137fff640 1 -- 172.21.15.5:0/1447513052 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56203616ad70 con 0x562036282d40 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.415+0000 7fb137fff640 10 client.29801 handle_command_reply: tid=0 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.415+0000 7fb161010740 1 client.29801 shutdown 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 1 -- 172.21.15.5:0/1447513052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361fd5d0 msgr2=0x5620362028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361fd5d0 0x5620362028d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb148043860 tx=0x7fb1480058c0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 1 -- 172.21.15.5:0/1447513052 shutdown_connections 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562036282d40 0x562036285160 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562036208370 0x562036210960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb137fff640 0 client.29801 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:25.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620361feb40 0x562036210420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 1 --2- 172.21.15.5:0/1447513052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620361fd5d0 0x5620362028d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 1 -- 172.21.15.5:0/1447513052 >> 172.21.15.5:0/1447513052 conn(0x5620361c6310 msgr2=0x5620361f8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:25.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 1 -- 172.21.15.5:0/1447513052 shutdown_connections 2024-09-11T22:39:25.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 1 -- 172.21.15.5:0/1447513052 wait complete. 2024-09-11T22:39:25.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.416+0000 7fb161010740 20 client.29801 trim_cache size 0 max 16384 2024-09-11T22:39:25.422 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:39:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 -- 172.21.15.5:0/2697404217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1400a2cf0 msgr2=0x7fb1400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1400a2cf0 0x7fb1400a51b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb1500045e0 tx=0x7fb1500046c0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 -- 172.21.15.5:0/2697404217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb158101380 msgr2=0x7fb1581a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb158101380 0x7fb1581a0930 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00eb90 tx=0x7fb14c00c6a0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 -- 172.21.15.5:0/2697404217 shutdown_connections 2024-09-11T22:39:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb15810e4f0 0x7fb15819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb158101380 0x7fb1581a0930 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb1400a2cf0 0x7fb1400a51b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 --2- 172.21.15.5:0/2697404217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1581009d0 0x7fb1581a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 -- 172.21.15.5:0/2697404217 >> 172.21.15.5:0/2697404217 conn(0x7fb1580fc7d0 msgr2=0x7fb1580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 -- 172.21.15.5:0/2697404217 shutdown_connections 2024-09-11T22:39:25.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:25.417+0000 7fb15ec37640 1 -- 172.21.15.5:0/2697404217 wait complete. 2024-09-11T22:39:25.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 59730 18.15 MB/sec execute 103 sec latency 166.990 ms 2024-09-11T22:39:25.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:25 smithi005 ceph-mon[30943]: pgmap v549: 97 pgs: 97 active+clean; 471 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 4.3 MiB/s wr, 227 op/s 2024-09-11T22:39:25.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:25 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2555561295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:25 smithi172 ceph-mon[35461]: pgmap v549: 97 pgs: 97 active+clean; 471 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 4.3 MiB/s wr, 227 op/s 2024-09-11T22:39:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:25 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2555561295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:25 smithi159 ceph-mon[35534]: pgmap v549: 97 pgs: 97 active+clean; 471 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 4.3 MiB/s wr, 227 op/s 2024-09-11T22:39:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:25 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2555561295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:26.252 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0' 2024-09-11T22:39:26.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db / --timeout 90 --expiration 144.25 --await-for 45.0 2024-09-11T22:39:26.578 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:26.646 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60336 18.35 MB/sec execute 104 sec latency 161.199 ms 2024-09-11T22:39:27.647 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60789 18.41 MB/sec execute 105 sec latency 221.201 ms 2024-09-11T22:39:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:27 smithi172 ceph-mon[35461]: pgmap v550: 97 pgs: 97 active+clean; 471 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 4.3 MiB/s wr, 227 op/s 2024-09-11T22:39:27.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:27 smithi005 ceph-mon[30943]: pgmap v550: 97 pgs: 97 active+clean; 471 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 4.3 MiB/s wr, 227 op/s 2024-09-11T22:39:28.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.007+0000 7f4355ac7640 1 -- 172.21.15.5:0/3999969710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350074620 msgr2=0x7f4350074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.007+0000 7f4355ac7640 1 --2- 172.21.15.5:0/3999969710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350074620 0x7f4350074a00 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f4340009860 tx=0x7f434002a3d0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.007+0000 7f4355ac7640 1 -- 172.21.15.5:0/3999969710 shutdown_connections 2024-09-11T22:39:28.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.007+0000 7f4355ac7640 1 --2- 172.21.15.5:0/3999969710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f435010e3d0 0x7f4350110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.007+0000 7f4355ac7640 1 --2- 172.21.15.5:0/3999969710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4350074f40 0x7f435010de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.007+0000 7f4355ac7640 1 --2- 172.21.15.5:0/3999969710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4350074620 0x7f4350074a00 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.007+0000 7f4355ac7640 1 -- 172.21.15.5:0/3999969710 >> 172.21.15.5:0/3999969710 conn(0x7f43500fd8b0 msgr2=0x7f43500ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.007+0000 7f4355ac7640 1 -- 172.21.15.5:0/3999969710 shutdown_connections 2024-09-11T22:39:28.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.007+0000 7f4355ac7640 1 -- 172.21.15.5:0/3999969710 wait complete. 2024-09-11T22:39:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.008+0000 7f4355ac7640 1 Processor -- start 2024-09-11T22:39:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.009+0000 7f4355ac7640 1 -- start start 2024-09-11T22:39:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.009+0000 7f4355ac7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4350074620 0x7f4350101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.009+0000 7f4355ac7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350074f40 0x7f4350102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.009+0000 7f4355ac7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435010e3d0 0x7f4350102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.009+0000 7f4355ac7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4350114220 con 0x7f435010e3d0 2024-09-11T22:39:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.009+0000 7f4355ac7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f43501140a0 con 0x7f4350074f40 2024-09-11T22:39:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.009+0000 7f4355ac7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43501143a0 con 0x7f4350074620 2024-09-11T22:39:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.009+0000 7f434ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435010e3d0 0x7f4350102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.009+0000 7f434ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435010e3d0 0x7f4350102570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38796/0 (socket says 172.21.15.5:38796) 2024-09-11T22:39:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.010+0000 7f434ffff640 1 -- 172.21.15.5:0/576848594 learned_addr learned my addr 172.21.15.5:0/576848594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.010+0000 7f434effd640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350074f40 0x7f4350102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.010+0000 7f434ffff640 1 -- 172.21.15.5:0/576848594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4350074620 msgr2=0x7f4350101af0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:39:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.010+0000 7f434ffff640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4350074620 0x7f4350101af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.010+0000 7f434ffff640 1 -- 172.21.15.5:0/576848594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350074f40 msgr2=0x7f4350102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.010+0000 7f434ffff640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350074f40 0x7f4350102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.010+0000 7f434ffff640 1 -- 172.21.15.5:0/576848594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435019a4a0 con 0x7f435010e3d0 2024-09-11T22:39:28.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.012+0000 7f434ffff640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435010e3d0 0x7f4350102570 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f4338009fd0 tx=0x7f433800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.012+0000 7f434cff9640 1 -- 172.21.15.5:0/576848594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338011830 con 0x7f435010e3d0 2024-09-11T22:39:28.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.013+0000 7f4355ac7640 1 -- 172.21.15.5:0/576848594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435019a790 con 0x7f435010e3d0 2024-09-11T22:39:28.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.013+0000 7f434cff9640 1 -- 172.21.15.5:0/576848594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4338016070 con 0x7f435010e3d0 2024-09-11T22:39:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.013+0000 7f434cff9640 1 -- 172.21.15.5:0/576848594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433800f2e0 con 0x7f435010e3d0 2024-09-11T22:39:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.013+0000 7f4355ac7640 1 -- 172.21.15.5:0/576848594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435019acd0 con 0x7f435010e3d0 2024-09-11T22:39:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.016+0000 7f434cff9640 1 -- 172.21.15.5:0/576848594 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4338010510 con 0x7f435010e3d0 2024-09-11T22:39:28.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f434cff9640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f433c0a2e40 0x7f433c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f4357ea0740 1 Processor -- start 2024-09-11T22:39:28.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f434cff9640 1 -- 172.21.15.5:0/576848594 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f43380ce9d0 con 0x7f435010e3d0 2024-09-11T22:39:28.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f434f7fe640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f433c0a2e40 0x7f433c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f4357ea0740 1 -- start start 2024-09-11T22:39:28.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac70c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac71170 0x56325ac7b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 0x56325ac7de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f4357ea0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56325abd5160 con 0x56325ac70850 2024-09-11T22:39:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f4357ea0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56325ab7c720 con 0x56325ac71170 2024-09-11T22:39:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.017+0000 7f4357ea0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56325abd1120 con 0x56325ac7ba30 2024-09-11T22:39:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.018+0000 7f434effd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac71170 0x56325ac7b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.018+0000 7f434f7fe640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f433c0a2e40 0x7f433c0a5300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f434002a8f0 tx=0x7f43400094e0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.018+0000 7f434effd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac71170 0x56325ac7b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44594/0 (socket says 172.21.15.5:44594) 2024-09-11T22:39:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.018+0000 7f434effd640 1 -- 172.21.15.5:0/4086840960 learned_addr learned my addr 172.21.15.5:0/4086840960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.018+0000 7f434f7fe640 1 --2- 172.21.15.5:0/4086840960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac70c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.018+0000 7f434ffff640 1 --2- 172.21.15.5:0/4086840960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 0x56325ac7de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.018+0000 7f434f7fe640 1 -- 172.21.15.5:0/4086840960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 msgr2=0x56325ac7de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.019+0000 7f434f7fe640 1 --2- 172.21.15.5:0/4086840960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 0x56325ac7de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.019+0000 7f434f7fe640 1 -- 172.21.15.5:0/4086840960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac71170 msgr2=0x56325ac7b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.019+0000 7f434f7fe640 1 --2- 172.21.15.5:0/4086840960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac71170 0x56325ac7b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.019+0000 7f434f7fe640 1 -- 172.21.15.5:0/4086840960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56325a946ba0 con 0x56325ac70850 2024-09-11T22:39:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.019+0000 7f434f7fe640 1 --2- 172.21.15.5:0/4086840960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac70c30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f4340009830 tx=0x7f434003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ac24195a3de0f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.019+0000 7f4345ffb640 1 -- 172.21.15.5:0/4086840960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340040070 con 0x56325ac70850 2024-09-11T22:39:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.019+0000 7f4345ffb640 1 -- 172.21.15.5:0/4086840960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43400046c0 con 0x56325ac70850 2024-09-11T22:39:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.020+0000 7f4345ffb640 1 -- 172.21.15.5:0/4086840960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434003c7a0 con 0x56325ac70850 2024-09-11T22:39:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.020+0000 7f4357ea0740 1 -- 172.21.15.5:0/4086840960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 msgr2=0x56325ac70c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.020+0000 7f4357ea0740 1 --2- 172.21.15.5:0/4086840960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac70c30 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f4340009830 tx=0x7f434003d000 comp rx=0 tx=0).stop 2024-09-11T22:39:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.020+0000 7f4357ea0740 1 -- 172.21.15.5:0/4086840960 shutdown_connections 2024-09-11T22:39:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.020+0000 7f4357ea0740 1 --2- 172.21.15.5:0/4086840960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 0x56325ac7de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.020+0000 7f4357ea0740 1 --2- 172.21.15.5:0/4086840960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac71170 0x56325ac7b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.020+0000 7f4357ea0740 1 --2- 172.21.15.5:0/4086840960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac70c30 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.020+0000 7f4357ea0740 1 -- 172.21.15.5:0/4086840960 >> 172.21.15.5:0/4086840960 conn(0x56325ac6cde0 msgr2=0x56325ac6d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.021+0000 7f4357ea0740 1 -- 172.21.15.5:0/4086840960 shutdown_connections 2024-09-11T22:39:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.021+0000 7f4357ea0740 1 -- 172.21.15.5:0/4086840960 wait complete. 2024-09-11T22:39:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.021+0000 7f4357ea0740 1 -- start start 2024-09-11T22:39:28.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.021+0000 7f4357ea0740 1 Processor -- start 2024-09-11T22:39:28.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac78850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac71170 0x56325ac78d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 0x56325ac792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f4357ea0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56325ac7e6a0 con 0x56325ac70850 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f4357ea0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56325ab5d900 con 0x56325ac7ba30 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f4357ea0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56325ac7e820 con 0x56325ac71170 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f4357ea0740 10 client.0 _ll_register_callbacks cb 0x56325ab5b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f434f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac78850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f434effd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac71170 0x56325ac78d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f434f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac78850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38812/0 (socket says 172.21.15.5:38812) 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.022+0000 7f434f7fe640 1 -- 172.21.15.5:0/2733160244 learned_addr learned my addr 172.21.15.5:0/2733160244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.023+0000 7f434ffff640 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 0x56325ac792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.023+0000 7f434f7fe640 1 -- 172.21.15.5:0/2733160244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac71170 msgr2=0x56325ac78d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.023+0000 7f434f7fe640 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac71170 0x56325ac78d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.023+0000 7f434f7fe640 1 -- 172.21.15.5:0/2733160244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 msgr2=0x56325ac792d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.023+0000 7f434f7fe640 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 0x56325ac792d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.023+0000 7f434f7fe640 1 -- 172.21.15.5:0/2733160244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56325ab9dc50 con 0x56325ac70850 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.023+0000 7f434f7fe640 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac78850 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f434002a410 tx=0x7f4340005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded123dbedd166a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.023+0000 7f432effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.023+0000 7f432effd640 1 -- 172.21.15.5:0/2733160244 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340040070 con 0x56325ac70850 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.024+0000 7f4357ea0740 1 -- 172.21.15.5:0/2733160244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43400046c0 con 0x56325ac70850 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.024+0000 7f432effd640 1 -- 172.21.15.5:0/2733160244 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f4340004870 con 0x56325ac70850 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.024+0000 7f432effd640 1 -- 172.21.15.5:0/2733160244 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f4340004a70 con 0x56325ac70850 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.024+0000 7f4357ea0740 10 client.20754 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.024+0000 7f4357ea0740 1 -- 172.21.15.5:0/2733160244 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56325ac8a2d0 con 0x56325ac70850 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.024+0000 7f432effd640 1 -- 172.21.15.5:0/2733160244 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43400046c0 con 0x56325ac70850 2024-09-11T22:39:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.025+0000 7f432effd640 1 -- 172.21.15.5:0/2733160244 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f434003ccd0 con 0x56325ac70850 2024-09-11T22:39:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.025+0000 7f4357ea0740 10 client.20754 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.025+0000 7f4357ea0740 10 client.20754 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:39:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.025+0000 7f4357ea0740 20 client.20754 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.025+0000 7f4357ea0740 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56325ac8b620 0x56325ac8da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.025+0000 7f4357ea0740 4 client.20754 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.025+0000 7f4357ea0740 1 -- 172.21.15.5:0/2733160244 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56325abe2890 con 0x56325ac8b620 2024-09-11T22:39:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.026+0000 7f434effd640 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56325ac8b620 0x56325ac8da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.026+0000 7f434effd640 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56325ac8b620 0x56325ac8da10 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23bfef2c0cdf4eb0 server_cookie=4e965fa62a6aef31 in_seq=0 out_seq=0 2024-09-11T22:39:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.026+0000 7f432effd640 10 client.20754 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:28.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f432effd640 1 -- 172.21.15.5:0/2733160244 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56325abe2890 con 0x56325ac8b620 2024-09-11T22:39:28.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f432effd640 10 client.20754 handle_command_reply: tid=0 2024-09-11T22:39:28.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 client.20754 shutdown 2024-09-11T22:39:28.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 -- 172.21.15.5:0/2733160244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 msgr2=0x56325ac78850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac78850 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f434002a410 tx=0x7f4340005340 comp rx=0 tx=0).stop 2024-09-11T22:39:28.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 -- 172.21.15.5:0/2733160244 shutdown_connections 2024-09-11T22:39:28.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56325ac8b620 0x56325ac8da10 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 0x56325ac792d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac71170 0x56325ac78d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 --2- 172.21.15.5:0/2733160244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac78850 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 -- 172.21.15.5:0/2733160244 >> 172.21.15.5:0/2733160244 conn(0x56325ac6cde0 msgr2=0x56325ac72b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f432effd640 0 client.20754 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:28.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 -- 172.21.15.5:0/2733160244 shutdown_connections 2024-09-11T22:39:28.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 1 -- 172.21.15.5:0/2733160244 wait complete. 2024-09-11T22:39:28.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.033+0000 7f4357ea0740 20 client.20754 trim_cache size 0 max 16384 2024-09-11T22:39:28.044 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:39:28.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:39:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f4357ea0740 1 Processor -- start 2024-09-11T22:39:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f4357ea0740 1 -- start start 2024-09-11T22:39:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac70850 0x56325ac77b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac71170 0x56325ac7a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 0x56325ac72a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f4357ea0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43501bc940 con 0x56325ac71170 2024-09-11T22:39:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f4357ea0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f43501be480 con 0x56325ac70850 2024-09-11T22:39:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f4357ea0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43501bd010 con 0x56325ac7ba30 2024-09-11T22:39:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f434effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac71170 0x56325ac7a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f434effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac71170 0x56325ac7a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38814/0 (socket says 172.21.15.5:38814) 2024-09-11T22:39:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f434ffff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 0x56325ac72a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f434effd640 1 -- 172.21.15.5:0/953903214 learned_addr learned my addr 172.21.15.5:0/953903214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f434f7fe640 1 --2- 172.21.15.5:0/953903214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac70850 0x56325ac77b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.052+0000 7f434effd640 1 -- 172.21.15.5:0/953903214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 msgr2=0x56325ac72a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.053+0000 7f434effd640 1 --2- 172.21.15.5:0/953903214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 0x56325ac72a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.053+0000 7f434effd640 1 -- 172.21.15.5:0/953903214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac70850 msgr2=0x56325ac77b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.053+0000 7f434effd640 1 --2- 172.21.15.5:0/953903214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac70850 0x56325ac77b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.053+0000 7f434effd640 1 -- 172.21.15.5:0/953903214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56325ac70fa0 con 0x56325ac71170 2024-09-11T22:39:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.053+0000 7f434ffff640 1 --2- 172.21.15.5:0/953903214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 0x56325ac72a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.053+0000 7f434effd640 1 --2- 172.21.15.5:0/953903214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac71170 0x56325ac7a720 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f4348005fd0 tx=0x7f43480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee81138c7e5452e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.053+0000 7f43457fa640 1 -- 172.21.15.5:0/953903214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4348003970 con 0x56325ac71170 2024-09-11T22:39:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.053+0000 7f43457fa640 1 -- 172.21.15.5:0/953903214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4348004050 con 0x56325ac71170 2024-09-11T22:39:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.053+0000 7f43457fa640 1 -- 172.21.15.5:0/953903214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434800e7b0 con 0x56325ac71170 2024-09-11T22:39:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.054+0000 7f4357ea0740 1 -- 172.21.15.5:0/953903214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac71170 msgr2=0x56325ac7a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.054+0000 7f4357ea0740 1 --2- 172.21.15.5:0/953903214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac71170 0x56325ac7a720 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f4348005fd0 tx=0x7f43480095c0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.054+0000 7f4357ea0740 1 -- 172.21.15.5:0/953903214 shutdown_connections 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.054+0000 7f4357ea0740 1 --2- 172.21.15.5:0/953903214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac7ba30 0x56325ac72a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.054+0000 7f4357ea0740 1 --2- 172.21.15.5:0/953903214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac71170 0x56325ac7a720 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.054+0000 7f4357ea0740 1 --2- 172.21.15.5:0/953903214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac70850 0x56325ac77b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.054+0000 7f4357ea0740 1 -- 172.21.15.5:0/953903214 >> 172.21.15.5:0/953903214 conn(0x56325ac6cde0 msgr2=0x56325ac722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.054+0000 7f4357ea0740 1 -- 172.21.15.5:0/953903214 shutdown_connections 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.054+0000 7f4357ea0740 1 -- 172.21.15.5:0/953903214 wait complete. 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f4357ea0740 1 -- start start 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f4357ea0740 1 Processor -- start 2024-09-11T22:39:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac83ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac71170 0x56325ac84010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f4357ea0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 0x56325ac7ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f4357ea0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43501bddb0 con 0x56325ac70850 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f4357ea0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f43501bd6e0 con 0x56325ac7ba30 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f4357ea0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f43501a1840 con 0x56325ac71170 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f4357ea0740 10 client.0 _ll_register_callbacks cb 0x7f4348009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f434f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac83ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f434ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 0x56325ac7ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f434f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac83ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38826/0 (socket says 172.21.15.5:38826) 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f434f7fe640 1 -- 172.21.15.5:0/1817065570 learned_addr learned my addr 172.21.15.5:0/1817065570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.055+0000 7f434effd640 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac71170 0x56325ac84010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f434ffff640 1 -- 172.21.15.5:0/1817065570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac71170 msgr2=0x56325ac84010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f434ffff640 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac71170 0x56325ac84010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f434ffff640 1 -- 172.21.15.5:0/1817065570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 msgr2=0x56325ac83ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f434ffff640 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac83ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f434ffff640 1 -- 172.21.15.5:0/1817065570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56325ac7efc0 con 0x56325ac7ba30 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f434f7fe640 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac83ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f434ffff640 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 0x56325ac7ea80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f43380cc1c0 tx=0x7f433800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e712ce896b4525c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f432effd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f432effd640 1 -- 172.21.15.5:0/1817065570 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43380a3c00 con 0x56325ac7ba30 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f4357ea0740 1 -- 172.21.15.5:0/1817065570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4348004050 con 0x56325ac7ba30 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.056+0000 7f432effd640 1 -- 172.21.15.5:0/1817065570 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f4338097600 con 0x56325ac7ba30 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f432effd640 1 -- 172.21.15.5:0/1817065570 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f43380a8050 con 0x56325ac7ba30 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f4357ea0740 10 client.29816 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f4357ea0740 1 -- 172.21.15.5:0/1817065570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56325acf6820 con 0x56325ac7ba30 2024-09-11T22:39:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f432effd640 1 -- 172.21.15.5:0/1817065570 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4348004050 con 0x56325ac7ba30 2024-09-11T22:39:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f432effd640 1 -- 172.21.15.5:0/1817065570 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f433809a470 con 0x56325ac7ba30 2024-09-11T22:39:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f4357ea0740 10 client.29816 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f4357ea0740 10 client.29816 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:39:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f4357ea0740 20 client.29816 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f4357ea0740 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56325acf7000 0x56325acf9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f4357ea0740 4 client.29816 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}] 2024-09-11T22:39:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f4357ea0740 1 -- 172.21.15.5:0/1817065570 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.25, "await_for": 45.0}) -- 0x56325abdcbe0 con 0x56325acf7000 2024-09-11T22:39:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.057+0000 7f434f7fe640 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56325acf7000 0x56325acf9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.058+0000 7f432effd640 10 client.29816 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.058+0000 7f434f7fe640 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56325acf7000 0x56325acf9420 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5444ae05812541d server_cookie=42358240051fb103 in_seq=0 out_seq=0 2024-09-11T22:39:28.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:27 smithi159 ceph-mon[35534]: pgmap v550: 97 pgs: 97 active+clean; 471 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 4.3 MiB/s wr, 227 op/s 2024-09-11T22:39:28.316 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:28.647 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60961 18.36 MB/sec execute 106 sec latency 662.184 ms 2024-09-11T22:39:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.791+0000 7fbc37622640 1 -- 172.21.15.5:0/3758530147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc30102d30 msgr2=0x7fbc30103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.791+0000 7fbc37622640 1 --2- 172.21.15.5:0/3758530147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc30102d30 0x7fbc30103110 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009930 tx=0x7fbc2c02a420 comp rx=0 tx=0).stop 2024-09-11T22:39:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.791+0000 7fbc37622640 1 -- 172.21.15.5:0/3758530147 shutdown_connections 2024-09-11T22:39:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.791+0000 7fbc37622640 1 --2- 172.21.15.5:0/3758530147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc301082a0 0x7fbc3010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.791+0000 7fbc37622640 1 --2- 172.21.15.5:0/3758530147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc301036e0 0x7fbc30107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.791+0000 7fbc37622640 1 --2- 172.21.15.5:0/3758530147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc30102d30 0x7fbc30103110 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.791+0000 7fbc37622640 1 -- 172.21.15.5:0/3758530147 >> 172.21.15.5:0/3758530147 conn(0x7fbc300fc7d0 msgr2=0x7fbc300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.791+0000 7fbc37622640 1 -- 172.21.15.5:0/3758530147 shutdown_connections 2024-09-11T22:39:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.791+0000 7fbc37622640 1 -- 172.21.15.5:0/3758530147 wait complete. 2024-09-11T22:39:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc37622640 1 Processor -- start 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc37622640 1 -- start start 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc37622640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc30102d30 0x7fbc30100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc37622640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc301036e0 0x7fbc30100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc37622640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc301082a0 0x7fbc30101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc37622640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301120a0 con 0x7fbc301036e0 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc37622640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc30111f20 con 0x7fbc301082a0 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc37622640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc30112220 con 0x7fbc30102d30 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc34bbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc301036e0 0x7fbc30100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc34bbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc301036e0 0x7fbc30100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38840/0 (socket says 172.21.15.5:38840) 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc34bbd640 1 -- 172.21.15.5:0/2182864156 learned_addr learned my addr 172.21.15.5:0/2182864156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.792+0000 7fbc35bbf640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc301082a0 0x7fbc30101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc353be640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc30102d30 0x7fbc30100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc34bbd640 1 -- 172.21.15.5:0/2182864156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc30102d30 msgr2=0x7fbc30100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc34bbd640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc30102d30 0x7fbc30100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc34bbd640 1 -- 172.21.15.5:0/2182864156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc301082a0 msgr2=0x7fbc30101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc34bbd640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc301082a0 0x7fbc30101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc34bbd640 1 -- 172.21.15.5:0/2182864156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc301a6a60 con 0x7fbc301036e0 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc34bbd640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc301036e0 0x7fbc30100ed0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000d950 tx=0x7fbc2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc267fc640 1 -- 172.21.15.5:0/2182864156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc20017070 con 0x7fbc301036e0 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc267fc640 1 -- 172.21.15.5:0/2182864156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc20009c30 con 0x7fbc301036e0 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc267fc640 1 -- 172.21.15.5:0/2182864156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200047a0 con 0x7fbc301036e0 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc37622640 1 -- 172.21.15.5:0/2182864156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc301a6d50 con 0x7fbc301036e0 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.793+0000 7fbc37622640 1 -- 172.21.15.5:0/2182864156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc301a7230 con 0x7fbc301036e0 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.796+0000 7fbc267fc640 1 -- 172.21.15.5:0/2182864156 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fbc20004020 con 0x7fbc301036e0 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.796+0000 7fbc399fb740 1 Processor -- start 2024-09-11T22:39:28.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.796+0000 7fbc399fb740 1 -- start start 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 0x5623e1b58f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b63800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b63d40 0x5623e1b66130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc399fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623e1abe280 con 0x5623e1b59480 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc399fb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5623e1a65b10 con 0x5623e1b58b60 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc399fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5623e1aba240 con 0x5623e1b63d40 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc353be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b63800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc353be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b63800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38850/0 (socket says 172.21.15.5:38850) 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc353be640 1 -- 172.21.15.5:0/1409964435 learned_addr learned my addr 172.21.15.5:0/1409964435 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc35bbf640 1 --2- 172.21.15.5:0/1409964435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 0x5623e1b58f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc34bbd640 1 --2- 172.21.15.5:0/1409964435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b63d40 0x5623e1b66130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc267fc640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc1c0a2cf0 0x7fbc1c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc353be640 1 -- 172.21.15.5:0/1409964435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b63d40 msgr2=0x5623e1b66130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc353be640 1 --2- 172.21.15.5:0/1409964435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b63d40 0x5623e1b66130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc353be640 1 -- 172.21.15.5:0/1409964435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 msgr2=0x5623e1b58f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc353be640 1 --2- 172.21.15.5:0/1409964435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 0x5623e1b58f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc353be640 1 -- 172.21.15.5:0/1409964435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e1a721f0 con 0x5623e1b59480 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc35bbf640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc1c0a2cf0 0x7fbc1c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.797+0000 7fbc267fc640 1 -- 172.21.15.5:0/2182864156 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fbc200cf080 con 0x7fbc301036e0 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.798+0000 7fbc353be640 1 --2- 172.21.15.5:0/1409964435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b63800 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c02a940 tx=0x7fbc2c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de19a9fed56c1505 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.798+0000 7fbc1b7fe640 1 -- 172.21.15.5:0/1409964435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c040070 con 0x5623e1b59480 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.798+0000 7fbc1b7fe640 1 -- 172.21.15.5:0/1409964435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c0318f0 con 0x5623e1b59480 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.798+0000 7fbc1b7fe640 1 -- 172.21.15.5:0/1409964435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c035040 con 0x5623e1b59480 2024-09-11T22:39:28.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 -- 172.21.15.5:0/1409964435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 msgr2=0x5623e1b63800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc35bbf640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc1c0a2cf0 0x7fbc1c0a51b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005c40 tx=0x7fbc28004ab0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 --2- 172.21.15.5:0/1409964435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b63800 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c02a940 tx=0x7fbc2c005c00 comp rx=0 tx=0).stop 2024-09-11T22:39:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 -- 172.21.15.5:0/1409964435 shutdown_connections 2024-09-11T22:39:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 --2- 172.21.15.5:0/1409964435 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b63d40 0x5623e1b66130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 --2- 172.21.15.5:0/1409964435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b63800 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 --2- 172.21.15.5:0/1409964435 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 0x5623e1b58f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 -- 172.21.15.5:0/1409964435 >> 172.21.15.5:0/1409964435 conn(0x5623e1b550f0 msgr2=0x5623e1b554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 -- 172.21.15.5:0/1409964435 shutdown_connections 2024-09-11T22:39:28.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 -- 172.21.15.5:0/1409964435 wait complete. 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 -- start start 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.799+0000 7fbc399fb740 1 Processor -- start 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b58b60 0x5623e1b637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b60b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b63d40 0x5623e1b612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc399fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623e1b66910 con 0x5623e1b59480 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc399fb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5623e1a78f10 con 0x5623e1b63d40 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc399fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5623e1b66a90 con 0x5623e1b58b60 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc399fb740 10 client.0 _ll_register_callbacks cb 0x5623e1a41a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc34bbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b60b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc34bbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b60b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38862/0 (socket says 172.21.15.5:38862) 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc34bbd640 1 -- 172.21.15.5:0/1938051871 learned_addr learned my addr 172.21.15.5:0/1938051871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:28.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc353be640 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b58b60 0x5623e1b637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc35bbf640 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b63d40 0x5623e1b612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc34bbd640 1 -- 172.21.15.5:0/1938051871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b58b60 msgr2=0x5623e1b637f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc34bbd640 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b58b60 0x5623e1b637f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc34bbd640 1 -- 172.21.15.5:0/1938051871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b63d40 msgr2=0x5623e1b612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc34bbd640 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b63d40 0x5623e1b612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc34bbd640 1 -- 172.21.15.5:0/1938051871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e182f6a0 con 0x5623e1b59480 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc34bbd640 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b60b40 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fbc20097f00 tx=0x7fbc20005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3169b2fb62e91a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc18ff9640 1 -- 172.21.15.5:0/1938051871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc200bb040 con 0x5623e1b59480 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc18ff9640 1 -- 172.21.15.5:0/1938051871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fbc20097ba0 con 0x5623e1b59480 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.800+0000 7fbc399fb740 1 -- 172.21.15.5:0/1938051871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2c0318f0 con 0x5623e1b59480 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc18ff9640 1 -- 172.21.15.5:0/1938051871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fbc2009b300 con 0x5623e1b59480 2024-09-11T22:39:28.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc399fb740 10 client.20772 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc399fb740 1 -- 172.21.15.5:0/1938051871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5623e1b724d0 con 0x5623e1b59480 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc18ff9640 1 -- 172.21.15.5:0/1938051871 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c0318f0 con 0x5623e1b59480 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc18ff9640 1 -- 172.21.15.5:0/1938051871 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fbc200a3d10 con 0x5623e1b59480 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc399fb740 10 client.20772 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc399fb740 10 client.20772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc399fb740 20 client.20772 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc399fb740 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5623e1b738e0 0x5623e1b75cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc399fb740 4 client.20772 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc399fb740 1 -- 172.21.15.5:0/1938051871 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623e1acb7f0 con 0x5623e1b738e0 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.801+0000 7fbc353be640 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5623e1b738e0 0x5623e1b75cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.802+0000 7fbc353be640 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5623e1b738e0 0x5623e1b75cf0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8dbd1c3ec213e8 server_cookie=166c046f417eaf62 in_seq=0 out_seq=0 2024-09-11T22:39:28.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.802+0000 7fbc18ff9640 10 client.20772 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:28.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.805+0000 7fbc18ff9640 1 -- 172.21.15.5:0/1938051871 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623e1acb7f0 con 0x5623e1b738e0 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.805+0000 7fbc18ff9640 10 client.20772 handle_command_reply: tid=0 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.805+0000 7fbc399fb740 1 client.20772 shutdown 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.805+0000 7fbc399fb740 1 -- 172.21.15.5:0/1938051871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 msgr2=0x5623e1b60b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.805+0000 7fbc399fb740 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b60b40 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fbc20097f00 tx=0x7fbc20005bd0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.806+0000 7fbc399fb740 1 -- 172.21.15.5:0/1938051871 shutdown_connections 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.806+0000 7fbc399fb740 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5623e1b738e0 0x5623e1b75cf0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.806+0000 7fbc399fb740 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b63d40 0x5623e1b612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.806+0000 7fbc399fb740 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b59480 0x5623e1b60b40 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.806+0000 7fbc399fb740 1 --2- 172.21.15.5:0/1938051871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b58b60 0x5623e1b637f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.806+0000 7fbc399fb740 1 -- 172.21.15.5:0/1938051871 >> 172.21.15.5:0/1938051871 conn(0x5623e1b550f0 msgr2=0x5623e1b5ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.806+0000 7fbc18ff9640 0 client.20772 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.806+0000 7fbc399fb740 1 -- 172.21.15.5:0/1938051871 shutdown_connections 2024-09-11T22:39:28.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.806+0000 7fbc399fb740 1 -- 172.21.15.5:0/1938051871 wait complete. 2024-09-11T22:39:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.806+0000 7fbc399fb740 20 client.20772 trim_cache size 0 max 16384 2024-09-11T22:39:28.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.823+0000 7fbc399fb740 1 Processor -- start 2024-09-11T22:39:28.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.823+0000 7fbc399fb740 1 -- start start 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b58b60 0x5623e1b55930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 0x5623e1b7b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b63d40 0x5623e1b7bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc399fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301afd20 con 0x5623e1b58b60 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc399fb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301b1860 con 0x5623e1b63d40 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc399fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301b03f0 con 0x5623e1b59480 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc353be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b58b60 0x5623e1b55930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc353be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b58b60 0x5623e1b55930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38878/0 (socket says 172.21.15.5:38878) 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc353be640 1 -- 172.21.15.5:0/4094030105 learned_addr learned my addr 172.21.15.5:0/4094030105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc34bbd640 1 --2- 172.21.15.5:0/4094030105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 0x5623e1b7b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc35bbf640 1 --2- 172.21.15.5:0/4094030105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b63d40 0x5623e1b7bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc353be640 1 -- 172.21.15.5:0/4094030105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 msgr2=0x5623e1b7b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc353be640 1 --2- 172.21.15.5:0/4094030105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 0x5623e1b7b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc353be640 1 -- 172.21.15.5:0/4094030105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b63d40 msgr2=0x5623e1b7bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc353be640 1 --2- 172.21.15.5:0/4094030105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b63d40 0x5623e1b7bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc353be640 1 -- 172.21.15.5:0/4094030105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623e1b54f20 con 0x5623e1b58b60 2024-09-11T22:39:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.824+0000 7fbc353be640 1 --2- 172.21.15.5:0/4094030105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b58b60 0x5623e1b55930 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009930 tx=0x7fbc2c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ab30887153417b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc1affd640 1 -- 172.21.15.5:0/4094030105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c040070 con 0x5623e1b58b60 2024-09-11T22:39:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc1affd640 1 -- 172.21.15.5:0/4094030105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c0318f0 con 0x5623e1b58b60 2024-09-11T22:39:28.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc1affd640 1 -- 172.21.15.5:0/4094030105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c035040 con 0x5623e1b58b60 2024-09-11T22:39:28.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc399fb740 1 -- 172.21.15.5:0/4094030105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b58b60 msgr2=0x5623e1b55930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc399fb740 1 --2- 172.21.15.5:0/4094030105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b58b60 0x5623e1b55930 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009930 tx=0x7fbc2c02a420 comp rx=0 tx=0).stop 2024-09-11T22:39:28.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc399fb740 1 -- 172.21.15.5:0/4094030105 shutdown_connections 2024-09-11T22:39:28.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc399fb740 1 --2- 172.21.15.5:0/4094030105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b63d40 0x5623e1b7bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc399fb740 1 --2- 172.21.15.5:0/4094030105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 0x5623e1b7b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc399fb740 1 --2- 172.21.15.5:0/4094030105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b58b60 0x5623e1b55930 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc399fb740 1 -- 172.21.15.5:0/4094030105 >> 172.21.15.5:0/4094030105 conn(0x5623e1b550f0 msgr2=0x5623e1b5f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc399fb740 1 -- 172.21.15.5:0/4094030105 shutdown_connections 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.825+0000 7fbc399fb740 1 -- 172.21.15.5:0/4094030105 wait complete. 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc399fb740 1 -- start start 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc399fb740 1 Processor -- start 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 0x5623e1b67a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 0x5623e1b67f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc399fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b63d40 0x5623e1b684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc399fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301b1190 con 0x5623e1b63d40 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc399fb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301b0ac0 con 0x5623e1b58b60 2024-09-11T22:39:28.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc399fb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301add20 con 0x5623e1b59480 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc399fb740 10 client.0 _ll_register_callbacks cb 0x7fbc200046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc353be640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 0x5623e1b67a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc35bbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b63d40 0x5623e1b684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc353be640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 0x5623e1b67a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44684/0 (socket says 172.21.15.5:44684) 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc353be640 1 -- 172.21.15.5:0/639681154 learned_addr learned my addr 172.21.15.5:0/639681154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.826+0000 7fbc34bbd640 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 0x5623e1b67f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc35bbf640 1 -- 172.21.15.5:0/639681154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 msgr2=0x5623e1b67f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc35bbf640 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 0x5623e1b67f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc35bbf640 1 -- 172.21.15.5:0/639681154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 msgr2=0x5623e1b67a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc35bbf640 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 0x5623e1b67a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc35bbf640 1 -- 172.21.15.5:0/639681154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623e1bdd560 con 0x5623e1b63d40 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc34bbd640 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 0x5623e1b67f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc35bbf640 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b63d40 0x5623e1b684d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005fd0 tx=0x7fbc28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147554af74409fde server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc18ff9640 1 -- 172.21.15.5:0/639681154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28004d20 con 0x5623e1b63d40 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc18ff9640 1 -- 172.21.15.5:0/639681154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fbc28005020 con 0x5623e1b63d40 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc399fb740 1 -- 172.21.15.5:0/639681154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2c0318f0 con 0x5623e1b63d40 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.827+0000 7fbc18ff9640 1 -- 172.21.15.5:0/639681154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fbc28004810 con 0x5623e1b63d40 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc399fb740 10 client.20778 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc399fb740 1 -- 172.21.15.5:0/639681154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5623e1bde820 con 0x5623e1b63d40 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc18ff9640 1 -- 172.21.15.5:0/639681154 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c0318f0 con 0x5623e1b63d40 2024-09-11T22:39:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc18ff9640 1 -- 172.21.15.5:0/639681154 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fbc2802a070 con 0x5623e1b63d40 2024-09-11T22:39:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc399fb740 10 client.20778 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc399fb740 10 client.20778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc399fb740 20 client.20778 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc399fb740 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5623e1bdeff0 0x5623e1be1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc399fb740 4 client.20778 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:39:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc399fb740 1 -- 172.21.15.5:0/639681154 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5623e1ac5d00 con 0x5623e1bdeff0 2024-09-11T22:39:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.828+0000 7fbc353be640 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5623e1bdeff0 0x5623e1be1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.829+0000 7fbc18ff9640 10 client.20778 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.829+0000 7fbc353be640 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5623e1bdeff0 0x5623e1be1410 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e18501aa4050329a server_cookie=518c32f6d09c7aa5 in_seq=0 out_seq=0 2024-09-11T22:39:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc18ff9640 1 -- 172.21.15.5:0/639681154 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5623e1ac5d00 con 0x5623e1bdeff0 2024-09-11T22:39:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc18ff9640 10 client.20778 handle_command_reply: tid=0 2024-09-11T22:39:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 client.20778 shutdown 2024-09-11T22:39:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 -- 172.21.15.5:0/639681154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b63d40 msgr2=0x5623e1b684d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b63d40 0x5623e1b684d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005fd0 tx=0x7fbc28004a40 comp rx=0 tx=0).stop 2024-09-11T22:39:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 -- 172.21.15.5:0/639681154 shutdown_connections 2024-09-11T22:39:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5623e1bdeff0 0x5623e1be1410 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623e1b63d40 0x5623e1b684d0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5623e1b59480 0x5623e1b67f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 --2- 172.21.15.5:0/639681154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5623e1b58b60 0x5623e1b67a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 -- 172.21.15.5:0/639681154 >> 172.21.15.5:0/639681154 conn(0x5623e1b550f0 msgr2=0x5623e1b53a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc18ff9640 0 client.20778 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 -- 172.21.15.5:0/639681154 shutdown_connections 2024-09-11T22:39:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 1 -- 172.21.15.5:0/639681154 wait complete. 2024-09-11T22:39:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.835+0000 7fbc399fb740 20 client.20778 trim_cache size 0 max 16384 2024-09-11T22:39:28.838 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:39:28.838 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:39:28.838 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e71f1359-d20a-4300-a448-d60ce83b7f3a", 2024-09-11T22:39:28.838 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:39:28.839 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:39:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.836+0000 7fbc37622640 1 -- 172.21.15.5:0/2182864156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc1c0a2cf0 msgr2=0x7fbc1c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.836+0000 7fbc37622640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc1c0a2cf0 0x7fbc1c0a51b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbc28005c40 tx=0x7fbc28004ab0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.836+0000 7fbc37622640 1 -- 172.21.15.5:0/2182864156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc301036e0 msgr2=0x7fbc30100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.836+0000 7fbc37622640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc301036e0 0x7fbc30100ed0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000d950 tx=0x7fbc2000de10 comp rx=0 tx=0).stop 2024-09-11T22:39:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.836+0000 7fbc37622640 1 -- 172.21.15.5:0/2182864156 shutdown_connections 2024-09-11T22:39:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.836+0000 7fbc37622640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fbc1c0a2cf0 0x7fbc1c0a51b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.836+0000 7fbc37622640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc301082a0 0x7fbc30101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.836+0000 7fbc37622640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc301036e0 0x7fbc30100ed0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.836+0000 7fbc37622640 1 --2- 172.21.15.5:0/2182864156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc30102d30 0x7fbc30100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.836+0000 7fbc37622640 1 -- 172.21.15.5:0/2182864156 >> 172.21.15.5:0/2182864156 conn(0x7fbc300fc7d0 msgr2=0x7fbc30105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.837+0000 7fbc37622640 1 -- 172.21.15.5:0/2182864156 shutdown_connections 2024-09-11T22:39:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.837+0000 7fbc37622640 1 -- 172.21.15.5:0/2182864156 wait complete. 2024-09-11T22:39:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f432effd640 1 -- 172.21.15.5:0/1817065570 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56325abdcbe0 con 0x56325acf7000 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f432effd640 10 client.29816 handle_command_reply: tid=0 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 client.29816 shutdown 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 -- 172.21.15.5:0/1817065570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 msgr2=0x56325ac7ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 0x56325ac7ea80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f43380cc1c0 tx=0x7f433800bf90 comp rx=0 tx=0).stop 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 -- 172.21.15.5:0/1817065570 shutdown_connections 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56325acf7000 0x56325acf9420 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56325ac7ba30 0x56325ac7ea80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56325ac71170 0x56325ac84010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f432effd640 0 client.29816 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 --2- 172.21.15.5:0/1817065570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325ac70850 0x56325ac83ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 -- 172.21.15.5:0/1817065570 >> 172.21.15.5:0/1817065570 conn(0x56325ac6cde0 msgr2=0x56325ac6bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 -- 172.21.15.5:0/1817065570 shutdown_connections 2024-09-11T22:39:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.904+0000 7f4357ea0740 1 -- 172.21.15.5:0/1817065570 wait complete. 2024-09-11T22:39:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4357ea0740 20 client.29816 trim_cache size 0 max 16384 2024-09-11T22:39:28.907 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:39:28.907 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 54, 2024-09-11T22:39:28.907 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:39:28.907 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 62, 2024-09-11T22:39:28.907 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:39:28.907 INFO:teuthology.orchestra.run.smithi005.stdout: "a49692cd": { 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 62, 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.2, 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:39:28.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-11T22:39:28.909 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:39:28.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:28.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:28.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:28.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:28.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:28.909 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 -- 172.21.15.5:0/576848594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f433c0a2e40 msgr2=0x7f433c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f433c0a2e40 0x7f433c0a5300 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f434002a8f0 tx=0x7f43400094e0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 -- 172.21.15.5:0/576848594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435010e3d0 msgr2=0x7f4350102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435010e3d0 0x7f4350102570 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f4338009fd0 tx=0x7f433800b8e0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 -- 172.21.15.5:0/576848594 shutdown_connections 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435010e3d0 0x7f4350102570 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f433c0a2e40 0x7f433c0a5300 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4350074f40 0x7f4350102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 --2- 172.21.15.5:0/576848594 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4350074620 0x7f4350101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 -- 172.21.15.5:0/576848594 >> 172.21.15.5:0/576848594 conn(0x7f43500fd8b0 msgr2=0x7f43500fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 -- 172.21.15.5:0/576848594 shutdown_connections 2024-09-11T22:39:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:28.905+0000 7f4355ac7640 1 -- 172.21.15.5:0/576848594 wait complete. 2024-09-11T22:39:29.647 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 18.19 MB/sec execute 107 sec latency 1551.468 ms 2024-09-11T22:39:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: pgmap v551: 97 pgs: 97 active+clean; 487 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 409 op/s 2024-09-11T22:39:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:39:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:39:29.861 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:29 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.866 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: pgmap v551: 97 pgs: 97 active+clean; 487 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 409 op/s 2024-09-11T22:39:29.866 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:39:29.866 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:39:29.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:29 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:29.869 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a49692cd', quiesce took 7.0 seconds. Will release after: 17.25 2024-09-11T22:39:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: pgmap v551: 97 pgs: 97 active+clean; 487 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 409 op/s 2024-09-11T22:39:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:39:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:39:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:39:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:39:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:39:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:39:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:39:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:29 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:39:30.636 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:30.647 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 18.02 MB/sec execute 108 sec latency 2551.517 ms 2024-09-11T22:39:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:30 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:39:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:30 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:30 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:39:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:30 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:30 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:30 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:39:30.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:30 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:30 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:39:30.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:30 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:30.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:30 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:30 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:39:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:30 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:30 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:39:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:30 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:31.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:30 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:39:31.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.132+0000 7f698ca8a640 1 -- 172.21.15.5:0/209492282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69881082a0 msgr2=0x7f698810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.132+0000 7f698ca8a640 1 --2- 172.21.15.5:0/209492282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69881082a0 0x7f698810ec40 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f6980009cd0 tx=0x7f698002a910 comp rx=0 tx=0).stop 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.132+0000 7f698ca8a640 1 -- 172.21.15.5:0/209492282 shutdown_connections 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.132+0000 7f698ca8a640 1 --2- 172.21.15.5:0/209492282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69881082a0 0x7f698810ec40 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.132+0000 7f698ca8a640 1 --2- 172.21.15.5:0/209492282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69881036e0 0x7f6988107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.132+0000 7f698ca8a640 1 --2- 172.21.15.5:0/209492282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6988102d30 0x7f6988103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.132+0000 7f698ca8a640 1 -- 172.21.15.5:0/209492282 >> 172.21.15.5:0/209492282 conn(0x7f69880fc7d0 msgr2=0x7f69880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.132+0000 7f698ca8a640 1 -- 172.21.15.5:0/209492282 shutdown_connections 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.132+0000 7f698ca8a640 1 -- 172.21.15.5:0/209492282 wait complete. 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.132+0000 7f698ca8a640 1 Processor -- start 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f698ca8a640 1 -- start start 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f698ca8a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6988102d30 0x7f6988100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f698ca8a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69881036e0 0x7f6988100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f698ca8a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69881082a0 0x7f6988101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f698ca8a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112100 con 0x7f69881082a0 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f698ca8a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6988111f80 con 0x7f69881036e0 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f698ca8a640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112280 con 0x7f6988102d30 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f6986d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69881082a0 0x7f6988101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f6986d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69881082a0 0x7f6988101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38906/0 (socket says 172.21.15.5:38906) 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f6986d9d640 1 -- 172.21.15.5:0/987748318 learned_addr learned my addr 172.21.15.5:0/987748318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f6985d9b640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69881036e0 0x7f6988100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f698659c640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6988102d30 0x7f6988100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f6986d9d640 1 -- 172.21.15.5:0/987748318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6988102d30 msgr2=0x7f6988100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f6986d9d640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6988102d30 0x7f6988100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f6986d9d640 1 -- 172.21.15.5:0/987748318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69881036e0 msgr2=0x7f6988100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f6986d9d640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69881036e0 0x7f6988100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f6986d9d640 1 -- 172.21.15.5:0/987748318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69881a6a60 con 0x7f69881082a0 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f698659c640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6988102d30 0x7f6988100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.133+0000 7f6986d9d640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69881082a0 0x7f6988101410 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6980005cc0 tx=0x7f6980031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.134+0000 7f69777fe640 1 -- 172.21.15.5:0/987748318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980031850 con 0x7f69881082a0 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.134+0000 7f69777fe640 1 -- 172.21.15.5:0/987748318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6980004520 con 0x7f69881082a0 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.134+0000 7f698ca8a640 1 -- 172.21.15.5:0/987748318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a6cf0 con 0x7f69881082a0 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.134+0000 7f69777fe640 1 -- 172.21.15.5:0/987748318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698002fe20 con 0x7f69881082a0 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.134+0000 7f698ca8a640 1 -- 172.21.15.5:0/987748318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a7230 con 0x7f69881082a0 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.136+0000 7f69777fe640 1 -- 172.21.15.5:0/987748318 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6980004040 con 0x7f69881082a0 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.136+0000 7f698ee63740 1 Processor -- start 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.136+0000 7f698ee63740 1 -- start start 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f698ee63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa56a4a0 0x55cdfa56a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f698ee63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 0x55cdfa575140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f698ee63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa575680 0x55cdfa577a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f698ee63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa4d02e0 con 0x55cdfa575680 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f698ee63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa477ce0 con 0x55cdfa56adc0 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f698ee63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa4cc2a0 con 0x55cdfa56a4a0 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f69777fe640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69700a2cf0 0x7f69700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6985d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa575680 0x55cdfa577a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6985d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa575680 0x55cdfa577a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38908/0 (socket says 172.21.15.5:38908) 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6986d9d640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69700a2cf0 0x7f69700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6985d9b640 1 -- 172.21.15.5:0/3202394038 learned_addr learned my addr 172.21.15.5:0/3202394038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f698659c640 1 --2- 172.21.15.5:0/3202394038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 0x55cdfa575140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f69777fe640 1 -- 172.21.15.5:0/987748318 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f69800f0ab0 con 0x7f69881082a0 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3202394038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa56a4a0 0x55cdfa56a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6985d9b640 1 -- 172.21.15.5:0/3202394038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa56a4a0 msgr2=0x55cdfa56a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:31.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3202394038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa56a4a0 0x55cdfa56a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6985d9b640 1 -- 172.21.15.5:0/3202394038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 msgr2=0x55cdfa575140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3202394038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 0x55cdfa575140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6985d9b640 1 -- 172.21.15.5:0/3202394038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdfa45c440 con 0x55cdfa575680 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f698659c640 1 --2- 172.21.15.5:0/3202394038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 0x55cdfa575140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.137+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3202394038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa575680 0x55cdfa577a70 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f697c00ad20 tx=0x7f697c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d0ebdd171899b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f6986d9d640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69700a2cf0 0x7f69700a51b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f69800b99e0 tx=0x7f698008efd0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f6974ff9640 1 -- 172.21.15.5:0/3202394038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c01c070 con 0x55cdfa575680 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f6974ff9640 1 -- 172.21.15.5:0/3202394038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697c00fdf0 con 0x55cdfa575680 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f6974ff9640 1 -- 172.21.15.5:0/3202394038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c0145f0 con 0x55cdfa575680 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f698ee63740 1 -- 172.21.15.5:0/3202394038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa575680 msgr2=0x55cdfa577a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f698ee63740 1 --2- 172.21.15.5:0/3202394038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa575680 0x55cdfa577a70 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f697c00ad20 tx=0x7f697c00a430 comp rx=0 tx=0).stop 2024-09-11T22:39:31.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f698ee63740 1 -- 172.21.15.5:0/3202394038 shutdown_connections 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f698ee63740 1 --2- 172.21.15.5:0/3202394038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa575680 0x55cdfa577a70 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f698ee63740 1 --2- 172.21.15.5:0/3202394038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 0x55cdfa575140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f698ee63740 1 --2- 172.21.15.5:0/3202394038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa56a4a0 0x55cdfa56a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.138+0000 7f698ee63740 1 -- 172.21.15.5:0/3202394038 >> 172.21.15.5:0/3202394038 conn(0x55cdfa566a30 msgr2=0x55cdfa566e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 1 -- 172.21.15.5:0/3202394038 shutdown_connections 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 1 -- 172.21.15.5:0/3202394038 wait complete. 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 1 -- start start 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 1 Processor -- start 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56a4a0 0x55cdfa575060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 0x55cdfa572310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa575680 0x55cdfa5728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa578250 con 0x55cdfa56a4a0 2024-09-11T22:39:31.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa48b050 con 0x55cdfa56adc0 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa5783d0 con 0x55cdfa575680 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f698ee63740 10 client.0 _ll_register_callbacks cb 0x55cdfa4cb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f6985d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56a4a0 0x55cdfa575060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f6985d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56a4a0 0x55cdfa575060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38920/0 (socket says 172.21.15.5:38920) 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.139+0000 7f6985d9b640 1 -- 172.21.15.5:0/3575403056 learned_addr learned my addr 172.21.15.5:0/3575403056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f698659c640 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 0x55cdfa572310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f6985d9b640 1 -- 172.21.15.5:0/3575403056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa575680 msgr2=0x55cdfa5728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa575680 0x55cdfa5728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f6985d9b640 1 -- 172.21.15.5:0/3575403056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 msgr2=0x55cdfa572310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 0x55cdfa572310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f6985d9b640 1 -- 172.21.15.5:0/3575403056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdfa2413c0 con 0x55cdfa56a4a0 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56a4a0 0x55cdfa575060 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f697c00acf0 tx=0x7f697c006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a5527c450203f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:31.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f696dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f696dffb640 1 -- 172.21.15.5:0/3575403056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c028060 con 0x55cdfa56a4a0 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f696dffb640 1 -- 172.21.15.5:0/3575403056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f697c0179f0 con 0x55cdfa56a4a0 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f698ee63740 1 -- 172.21.15.5:0/3575403056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f697c00fdf0 con 0x55cdfa56a4a0 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f696dffb640 1 -- 172.21.15.5:0/3575403056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f697c01c040 con 0x55cdfa56a4a0 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f698ee63740 10 client.20790 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f698ee63740 1 -- 172.21.15.5:0/3575403056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55cdfa583cc0 con 0x55cdfa56a4a0 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f696dffb640 1 -- 172.21.15.5:0/3575403056 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697c00fdf0 con 0x55cdfa56a4a0 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.140+0000 7f696dffb640 1 -- 172.21.15.5:0/3575403056 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f697c015040 con 0x55cdfa56a4a0 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.141+0000 7f698ee63740 10 client.20790 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.141+0000 7f698ee63740 10 client.20790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.141+0000 7f698ee63740 20 client.20790 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.141+0000 7f698ee63740 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cdfa5850d0 0x55cdfa5874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.141+0000 7f698ee63740 4 client.20790 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.141+0000 7f698ee63740 1 -- 172.21.15.5:0/3575403056 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdfa4dd850 con 0x55cdfa5850d0 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.141+0000 7f698659c640 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cdfa5850d0 0x55cdfa5874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.141+0000 7f698659c640 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cdfa5850d0 0x55cdfa5874c0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f78f2f9bd9aa3 server_cookie=3931781d58c48efe in_seq=0 out_seq=0 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.141+0000 7f696dffb640 10 client.20790 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f696dffb640 1 -- 172.21.15.5:0/3575403056 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cdfa4dd850 con 0x55cdfa5850d0 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f696dffb640 10 client.20790 handle_command_reply: tid=0 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 client.20790 shutdown 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 -- 172.21.15.5:0/3575403056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56a4a0 msgr2=0x55cdfa575060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56a4a0 0x55cdfa575060 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f697c00acf0 tx=0x7f697c006d80 comp rx=0 tx=0).stop 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 -- 172.21.15.5:0/3575403056 shutdown_connections 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cdfa5850d0 0x55cdfa5874c0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa575680 0x55cdfa5728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56adc0 0x55cdfa572310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f696dffb640 0 client.20790 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:31.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 --2- 172.21.15.5:0/3575403056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56a4a0 0x55cdfa575060 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 -- 172.21.15.5:0/3575403056 >> 172.21.15.5:0/3575403056 conn(0x55cdfa566a30 msgr2=0x55cdfa56c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:31.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 -- 172.21.15.5:0/3575403056 shutdown_connections 2024-09-11T22:39:31.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.142+0000 7f698ee63740 1 -- 172.21.15.5:0/3575403056 wait complete. 2024-09-11T22:39:31.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.143+0000 7f698ee63740 20 client.20790 trim_cache size 0 max 16384 2024-09-11T22:39:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.160+0000 7f698ee63740 1 Processor -- start 2024-09-11T22:39:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.160+0000 7f698ee63740 1 -- start start 2024-09-11T22:39:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.160+0000 7f698ee63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa56a4a0 0x55cdfa5814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.160+0000 7f698ee63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa58ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.160+0000 7f698ee63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa575680 0x55cdfa58cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.160+0000 7f698ee63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69881afc70 con 0x55cdfa56adc0 2024-09-11T22:39:31.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.160+0000 7f698ee63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69881b17b0 con 0x55cdfa575680 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.160+0000 7f698ee63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69881b0340 con 0x55cdfa56a4a0 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.160+0000 7f698659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa58ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f698659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa58ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38926/0 (socket says 172.21.15.5:38926) 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f698659c640 1 -- 172.21.15.5:0/971682641 learned_addr learned my addr 172.21.15.5:0/971682641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f6985d9b640 1 --2- 172.21.15.5:0/971682641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa56a4a0 0x55cdfa5814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f6985d9b640 1 --2- 172.21.15.5:0/971682641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa575680 0x55cdfa58cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f698659c640 1 -- 172.21.15.5:0/971682641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa56a4a0 msgr2=0x55cdfa5814e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f698659c640 1 --2- 172.21.15.5:0/971682641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa56a4a0 0x55cdfa5814e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f698659c640 1 -- 172.21.15.5:0/971682641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa575680 msgr2=0x55cdfa58cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f698659c640 1 --2- 172.21.15.5:0/971682641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa575680 0x55cdfa58cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f698659c640 1 -- 172.21.15.5:0/971682641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdfa570860 con 0x55cdfa56adc0 2024-09-11T22:39:31.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f6985d9b640 1 --2- 172.21.15.5:0/971682641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa575680 0x55cdfa58cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:31.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f698659c640 1 --2- 172.21.15.5:0/971682641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa58ca90 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f6978005fd0 tx=0x7f6978005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325f3de54c350bcf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:31.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f696ffff640 1 -- 172.21.15.5:0/971682641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978019330 con 0x55cdfa56adc0 2024-09-11T22:39:31.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f696ffff640 1 -- 172.21.15.5:0/971682641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69780199b0 con 0x55cdfa56adc0 2024-09-11T22:39:31.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.161+0000 7f696ffff640 1 -- 172.21.15.5:0/971682641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69780109f0 con 0x55cdfa56adc0 2024-09-11T22:39:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.162+0000 7f698ee63740 1 -- 172.21.15.5:0/971682641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 msgr2=0x55cdfa58ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:31.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.162+0000 7f698ee63740 1 --2- 172.21.15.5:0/971682641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa58ca90 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f6978005fd0 tx=0x7f6978005df0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.162+0000 7f698ee63740 1 -- 172.21.15.5:0/971682641 shutdown_connections 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.162+0000 7f698ee63740 1 --2- 172.21.15.5:0/971682641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa575680 0x55cdfa58cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.162+0000 7f698ee63740 1 --2- 172.21.15.5:0/971682641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa58ca90 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.162+0000 7f698ee63740 1 --2- 172.21.15.5:0/971682641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa56a4a0 0x55cdfa5814e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.162+0000 7f698ee63740 1 -- 172.21.15.5:0/971682641 >> 172.21.15.5:0/971682641 conn(0x55cdfa566a30 msgr2=0x55cdfa56d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.162+0000 7f698ee63740 1 -- 172.21.15.5:0/971682641 shutdown_connections 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.162+0000 7f698ee63740 1 -- 172.21.15.5:0/971682641 wait complete. 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.162+0000 7f698ee63740 1 -- start start 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698ee63740 1 Processor -- start 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698ee63740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56a4a0 0x55cdfa57d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698ee63740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa57db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698ee63740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa575680 0x55cdfa57a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698ee63740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69881b10e0 con 0x55cdfa56adc0 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698ee63740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f69881b0a10 con 0x55cdfa56a4a0 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698ee63740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f69881adc70 con 0x55cdfa575680 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698ee63740 10 client.0 _ll_register_callbacks cb 0x7f697c01a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa57db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa57db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38934/0 (socket says 172.21.15.5:38934) 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698659c640 1 -- 172.21.15.5:0/3248960578 learned_addr learned my addr 172.21.15.5:0/3248960578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56a4a0 0x55cdfa57d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa575680 0x55cdfa57a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698659c640 1 -- 172.21.15.5:0/3248960578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa575680 msgr2=0x55cdfa57a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698659c640 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa575680 0x55cdfa57a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698659c640 1 -- 172.21.15.5:0/3248960578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56a4a0 msgr2=0x55cdfa57d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698659c640 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56a4a0 0x55cdfa57d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.163+0000 7f698659c640 1 -- 172.21.15.5:0/3248960578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdfa57ab60 con 0x55cdfa56adc0 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.164+0000 7f698659c640 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa57db90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f697800d5b0 tx=0x7f697800d700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698d2d2b40edc3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.164+0000 7f696dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.164+0000 7f696dffb640 1 -- 172.21.15.5:0/3248960578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697802f060 con 0x55cdfa56adc0 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.164+0000 7f696dffb640 1 -- 172.21.15.5:0/3248960578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f6978019530 con 0x55cdfa56adc0 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.164+0000 7f698ee63740 1 -- 172.21.15.5:0/3248960578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69780199b0 con 0x55cdfa56adc0 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.164+0000 7f696dffb640 1 -- 172.21.15.5:0/3248960578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f6978019d70 con 0x55cdfa56adc0 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.164+0000 7f698ee63740 10 client.20796 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.164+0000 7f698ee63740 1 -- 172.21.15.5:0/3248960578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55cdfa5f01e0 con 0x55cdfa56adc0 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.164+0000 7f696dffb640 1 -- 172.21.15.5:0/3248960578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69780199b0 con 0x55cdfa56adc0 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.165+0000 7f696dffb640 1 -- 172.21.15.5:0/3248960578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f6978022540 con 0x55cdfa56adc0 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.165+0000 7f698ee63740 10 client.20796 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.165+0000 7f698ee63740 10 client.20796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.165+0000 7f698ee63740 20 client.20796 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.165+0000 7f698ee63740 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cdfa5f0820 0x55cdfa5f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.165+0000 7f698ee63740 4 client.20796 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.165+0000 7f698ee63740 1 -- 172.21.15.5:0/3248960578 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cdfa4d7d60 con 0x55cdfa5f0820 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.165+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cdfa5f0820 0x55cdfa5f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.165+0000 7f6985d9b640 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cdfa5f0820 0x55cdfa5f2c40 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b81dc64bf5170004 server_cookie=b0a7fbe18675f614 in_seq=0 out_seq=0 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.165+0000 7f696dffb640 10 client.20796 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f696dffb640 1 -- 172.21.15.5:0/3248960578 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cdfa4d7d60 con 0x55cdfa5f0820 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f696dffb640 10 client.20796 handle_command_reply: tid=0 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 client.20796 shutdown 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 -- 172.21.15.5:0/3248960578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 msgr2=0x55cdfa57db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa57db90 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f697800d5b0 tx=0x7f697800d700 comp rx=0 tx=0).stop 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 -- 172.21.15.5:0/3248960578 shutdown_connections 2024-09-11T22:39:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55cdfa5f0820 0x55cdfa5f2c40 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f696dffb640 0 client.20796 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdfa575680 0x55cdfa57a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdfa56adc0 0x55cdfa57db90 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 --2- 172.21.15.5:0/3248960578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cdfa56a4a0 0x55cdfa57d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 -- 172.21.15.5:0/3248960578 >> 172.21.15.5:0/3248960578 conn(0x55cdfa566a30 msgr2=0x55cdfa56d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 -- 172.21.15.5:0/3248960578 shutdown_connections 2024-09-11T22:39:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 1 -- 172.21.15.5:0/3248960578 wait complete. 2024-09-11T22:39:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.166+0000 7f698ee63740 20 client.20796 trim_cache size 0 max 16384 2024-09-11T22:39:31.173 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:39:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:39:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:39:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "e71f1359-d20a-4300-a448-d60ce83b7f3a": { 2024-09-11T22:39:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:39:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e71f1359-d20a-4300-a448-d60ce83b7f3a", 2024-09-11T22:39:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:39:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:31.174 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 -- 172.21.15.5:0/987748318 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69700a2cf0 msgr2=0x7f69700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69700a2cf0 0x7f69700a51b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f69800b99e0 tx=0x7f698008efd0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 -- 172.21.15.5:0/987748318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69881082a0 msgr2=0x7f6988101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69881082a0 0x7f6988101410 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6980005cc0 tx=0x7f6980031460 comp rx=0 tx=0).stop 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 -- 172.21.15.5:0/987748318 shutdown_connections 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69881082a0 0x7f6988101410 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f69700a2cf0 0x7f69700a51b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f69881036e0 0x7f6988100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 --2- 172.21.15.5:0/987748318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6988102d30 0x7f6988100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 -- 172.21.15.5:0/987748318 >> 172.21.15.5:0/987748318 conn(0x7f69880fc7d0 msgr2=0x7f6988105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 -- 172.21.15.5:0/987748318 shutdown_connections 2024-09-11T22:39:31.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:31.167+0000 7f698ca8a640 1 -- 172.21.15.5:0/987748318 wait complete. 2024-09-11T22:39:31.647 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 17.86 MB/sec execute 109 sec latency 3551.645 ms 2024-09-11T22:39:31.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:31 smithi172 ceph-mon[35461]: pgmap v552: 97 pgs: 97 active+clean; 487 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 409 op/s 2024-09-11T22:39:31.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:31 smithi005 ceph-mon[30943]: pgmap v552: 97 pgs: 97 active+clean; 487 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 409 op/s 2024-09-11T22:39:31.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:31 smithi159 ceph-mon[35534]: pgmap v552: 97 pgs: 97 active+clean; 487 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 409 op/s 2024-09-11T22:39:31.989 INFO:tasks.cephfs.filesystem:scrub status for tag:e71f1359-d20a-4300-a448-d60ce83b7f3a - {'path': '/', 'tag': 'e71f1359-d20a-4300-a448-d60ce83b7f3a', 'options': 'recursive,force'} 2024-09-11T22:39:32.647 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 17.69 MB/sec execute 110 sec latency 4551.771 ms 2024-09-11T22:39:33.647 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 17.53 MB/sec execute 111 sec latency 5551.878 ms 2024-09-11T22:39:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:33 smithi172 ceph-mon[35461]: pgmap v553: 97 pgs: 97 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.4 MiB/s wr, 420 op/s 2024-09-11T22:39:34.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:33 smithi005 ceph-mon[30943]: pgmap v553: 97 pgs: 97 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.4 MiB/s wr, 420 op/s 2024-09-11T22:39:34.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:33 smithi159 ceph-mon[35534]: pgmap v553: 97 pgs: 97 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.4 MiB/s wr, 420 op/s 2024-09-11T22:39:34.647 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 17.38 MB/sec execute 112 sec latency 6551.948 ms 2024-09-11T22:39:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.680+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/3899578586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad0101350 msgr2=0x7f3ad010d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.680+0000 7f3ad7b4b640 1 --2- 172.21.15.5:0/3899578586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad0101350 0x7f3ad010d880 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009cd0 tx=0x7f3ac802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.680+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/3899578586 shutdown_connections 2024-09-11T22:39:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.680+0000 7f3ad7b4b640 1 --2- 172.21.15.5:0/3899578586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad0101350 0x7f3ad010d880 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.680+0000 7f3ad7b4b640 1 --2- 172.21.15.5:0/3899578586 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ad01009b0 0x7f3ad0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.680+0000 7f3ad7b4b640 1 --2- 172.21.15.5:0/3899578586 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ad01069c0 0x7f3ad0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.680+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/3899578586 >> 172.21.15.5:0/3899578586 conn(0x7f3ad0077990 msgr2=0x7f3ad00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.681+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/3899578586 shutdown_connections 2024-09-11T22:39:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.681+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/3899578586 wait complete. 2024-09-11T22:39:34.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.681+0000 7f3ad7b4b640 1 Processor -- start 2024-09-11T22:39:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.681+0000 7f3ad7b4b640 1 -- start start 2024-09-11T22:39:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.681+0000 7f3ad7b4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad01009b0 0x7f3ad0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.681+0000 7f3ad7b4b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ad0101350 0x7f3ad0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad7b4b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ad01069c0 0x7f3ad019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad7b4b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0110e00 con 0x7f3ad01009b0 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad7b4b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0110c80 con 0x7f3ad0101350 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad7b4b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0110f80 con 0x7f3ad01069c0 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad58e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad01009b0 0x7f3ad0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad58e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad01009b0 0x7f3ad0198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51410/0 (socket says 172.21.15.5:51410) 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1262126710 learned_addr learned my addr 172.21.15.5:0/1262126710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ad01069c0 0x7f3ad019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ad0101350 0x7f3ad0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1262126710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ad01069c0 msgr2=0x7f3ad019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ad01069c0 0x7f3ad019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1262126710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ad0101350 msgr2=0x7f3ad0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ad0101350 0x7f3ad0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1262126710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad019d320 con 0x7f3ad01009b0 2024-09-11T22:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ad0101350 0x7f3ad0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad01009b0 0x7f3ad0198330 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00d770 tx=0x7f3acc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.682+0000 7f3ac6ffd640 1 -- 172.21.15.5:0/1262126710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc017070 con 0x7f3ad01009b0 2024-09-11T22:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.683+0000 7f3ac6ffd640 1 -- 172.21.15.5:0/1262126710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3acc005500 con 0x7f3ad01009b0 2024-09-11T22:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.683+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/1262126710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad019d610 con 0x7f3ad01009b0 2024-09-11T22:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.683+0000 7f3ac6ffd640 1 -- 172.21.15.5:0/1262126710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc004e00 con 0x7f3ad01009b0 2024-09-11T22:39:34.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.683+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/1262126710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a4ea0 con 0x7f3ad01009b0 2024-09-11T22:39:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.685+0000 7f3ac6ffd640 1 -- 172.21.15.5:0/1262126710 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3acc004fa0 con 0x7f3ad01009b0 2024-09-11T22:39:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.685+0000 7f3ada9c1740 1 Processor -- start 2024-09-11T22:39:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ada9c1740 1 -- start start 2024-09-11T22:39:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35f6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 0x557ff3601770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff3601cb0 0x557ff36040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557ff355b2f0 con 0x557ff35f6af0 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557ff3503270 con 0x557ff3601cb0 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557ff35572b0 con 0x557ff35f7410 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad60e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35f6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ac6ffd640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3abc0a2da0 0x7f3abc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad60e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35f6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51422/0 (socket says 172.21.15.5:51422) 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad60e8640 1 -- 172.21.15.5:0/3633763746 learned_addr learned my addr 172.21.15.5:0/3633763746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/3633763746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff3601cb0 0x557ff36040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3abc0a2da0 0x7f3abc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ac6ffd640 1 -- 172.21.15.5:0/1262126710 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f3acc0cee90 con 0x7f3ad01009b0 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/3633763746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 0x557ff3601770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad60e8640 1 -- 172.21.15.5:0/3633763746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 msgr2=0x557ff3601770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/3633763746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 0x557ff3601770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad60e8640 1 -- 172.21.15.5:0/3633763746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff3601cb0 msgr2=0x557ff36040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/3633763746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff3601cb0 0x557ff36040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad60e8640 1 -- 172.21.15.5:0/3633763746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ff3502340 con 0x557ff35f6af0 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.686+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/3633763746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 0x557ff3601770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.687+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/3633763746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35f6ed0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009430 tx=0x7f3ac8031b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9889b81294077a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.687+0000 7f3abbfff640 1 -- 172.21.15.5:0/3633763746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8004610 con 0x557ff35f6af0 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.687+0000 7f3abbfff640 1 -- 172.21.15.5:0/3633763746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac8036070 con 0x557ff35f6af0 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.687+0000 7f3abbfff640 1 -- 172.21.15.5:0/3633763746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac804a450 con 0x557ff35f6af0 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 -- 172.21.15.5:0/3633763746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 msgr2=0x557ff35f6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/3633763746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35f6ed0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009430 tx=0x7f3ac8031b00 comp rx=0 tx=0).stop 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3abc0a2da0 0x7f3abc0a5260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009a60 tx=0x7f3ac804c000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:34.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 -- 172.21.15.5:0/3633763746 shutdown_connections 2024-09-11T22:39:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/3633763746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff3601cb0 0x557ff36040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/3633763746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 0x557ff3601770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/3633763746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35f6ed0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 -- 172.21.15.5:0/3633763746 >> 172.21.15.5:0/3633763746 conn(0x557ff35f3060 msgr2=0x557ff35f3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 -- 172.21.15.5:0/3633763746 shutdown_connections 2024-09-11T22:39:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 -- 172.21.15.5:0/3633763746 wait complete. 2024-09-11T22:39:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 -- start start 2024-09-11T22:39:34.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.688+0000 7f3ada9c1740 1 Processor -- start 2024-09-11T22:39:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35feb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f7410 0x557ff35ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff3601cb0 0x557ff35ff610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557ff3604880 con 0x557ff35f6af0 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557ff34e4290 con 0x557ff35f7410 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557ff3604a00 con 0x557ff3601cb0 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ada9c1740 10 client.0 _ll_register_callbacks cb 0x557ff3515140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ad58e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ad58e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35feb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51432/0 (socket says 172.21.15.5:51432) 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1643474270 learned_addr learned my addr 172.21.15.5:0/1643474270 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f7410 0x557ff35ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff3601cb0 0x557ff35ff610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1643474270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff3601cb0 msgr2=0x557ff35ff610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff3601cb0 0x557ff35ff610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1643474270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f7410 msgr2=0x557ff35ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f7410 0x557ff35ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.689+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1643474270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ff326e210 con 0x557ff35f6af0 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35feb90 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0cc5d0 tx=0x7f3acc09ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd292f52c13f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ab97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1643474270 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc020040 con 0x557ff35f6af0 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1643474270 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f3acc0cc7f0 con 0x557ff35f6af0 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1643474270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac8036070 con 0x557ff35f6af0 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1643474270 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3acc09f6f0 con 0x557ff35f6af0 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ada9c1740 10 client.20808 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1643474270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557ff333a620 con 0x557ff35f6af0 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1643474270 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac8036070 con 0x557ff35f6af0 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1643474270 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f3acc09a790 con 0x557ff35f6af0 2024-09-11T22:39:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ada9c1740 10 client.20808 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ada9c1740 10 client.20808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ada9c1740 20 client.20808 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557ff3611c20 0x557ff3614030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ada9c1740 4 client.20808 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.690+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1643474270 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ff3568a20 con 0x557ff3611c20 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.691+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557ff3611c20 0x557ff3614030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.691+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557ff3611c20 0x557ff3614030 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8efb21bc56dd9432 server_cookie=822cb329809c5791 in_seq=0 out_seq=0 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.691+0000 7f3ab97fa640 10 client.20808 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1643474270 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ff3568a20 con 0x557ff3611c20 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ab97fa640 10 client.20808 handle_command_reply: tid=0 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ada9c1740 1 client.20808 shutdown 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1643474270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 msgr2=0x557ff35feb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35feb90 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0cc5d0 tx=0x7f3acc09ad60 comp rx=0 tx=0).stop 2024-09-11T22:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1643474270 shutdown_connections 2024-09-11T22:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557ff3611c20 0x557ff3614030 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff3601cb0 0x557ff35ff610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f7410 0x557ff35ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ab97fa640 0 client.20808 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.692+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1643474270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff35feb90 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.693+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1643474270 >> 172.21.15.5:0/1643474270 conn(0x557ff35f3060 msgr2=0x557ff35f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.693+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1643474270 shutdown_connections 2024-09-11T22:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.693+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1643474270 wait complete. 2024-09-11T22:39:34.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.693+0000 7f3ada9c1740 20 client.20808 trim_cache size 0 max 16384 2024-09-11T22:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ada9c1740 1 Processor -- start 2024-09-11T22:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ada9c1740 1 -- start start 2024-09-11T22:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f6af0 0x557ff35ff440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 0x557ff3600b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff3601cb0 0x557ff3619940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01ad9d0 con 0x557ff3601cb0 2024-09-11T22:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01af510 con 0x557ff35f6af0 2024-09-11T22:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01ae0a0 con 0x557ff35f7410 2024-09-11T22:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad60e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff3601cb0 0x557ff3619940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad60e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff3601cb0 0x557ff3619940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51446/0 (socket says 172.21.15.5:51446) 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad60e8640 1 -- 172.21.15.5:0/1555041275 learned_addr learned my addr 172.21.15.5:0/1555041275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 0x557ff3600b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f6af0 0x557ff35ff440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad60e8640 1 -- 172.21.15.5:0/1555041275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 msgr2=0x557ff3600b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 0x557ff3600b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad60e8640 1 -- 172.21.15.5:0/1555041275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f6af0 msgr2=0x557ff35ff440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f6af0 0x557ff35ff440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad60e8640 1 -- 172.21.15.5:0/1555041275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ff360ec80 con 0x557ff3601cb0 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f6af0 0x557ff35ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.708+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 0x557ff3600b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.709+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff3601cb0 0x557ff3619940 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8004a40 tx=0x7f3ac803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2c97e0ca9eda4d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.709+0000 7f3abb7fe640 1 -- 172.21.15.5:0/1555041275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8062070 con 0x557ff3601cb0 2024-09-11T22:39:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.709+0000 7f3abb7fe640 1 -- 172.21.15.5:0/1555041275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac8036040 con 0x557ff3601cb0 2024-09-11T22:39:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.709+0000 7f3abb7fe640 1 -- 172.21.15.5:0/1555041275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac804a820 con 0x557ff3601cb0 2024-09-11T22:39:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.709+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1555041275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff3601cb0 msgr2=0x557ff3619940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.709+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff3601cb0 0x557ff3619940 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8004a40 tx=0x7f3ac803f000 comp rx=0 tx=0).stop 2024-09-11T22:39:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.709+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1555041275 shutdown_connections 2024-09-11T22:39:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff3601cb0 0x557ff3619940 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff35f7410 0x557ff3600b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1555041275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f6af0 0x557ff35ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1555041275 >> 172.21.15.5:0/1555041275 conn(0x557ff35f3060 msgr2=0x557ff36003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1555041275 shutdown_connections 2024-09-11T22:39:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1555041275 wait complete. 2024-09-11T22:39:34.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 -- start start 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 Processor -- start 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff360a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f7410 0x557ff3605410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff3601cb0 0x557ff3605950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01aee40 con 0x557ff35f6af0 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01ae770 con 0x557ff35f7410 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01ab9d0 con 0x557ff3601cb0 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.710+0000 7f3ada9c1740 10 client.0 _ll_register_callbacks cb 0x7f3acc0056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad58e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff360a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad58e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff360a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51456/0 (socket says 172.21.15.5:51456) 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1814459744 learned_addr learned my addr 172.21.15.5:0/1814459744 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f7410 0x557ff3605410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad60e8640 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff3601cb0 0x557ff3605950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1814459744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff3601cb0 msgr2=0x557ff3605950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff3601cb0 0x557ff3605950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1814459744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f7410 msgr2=0x557ff3605410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f7410 0x557ff3605410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad58e7640 1 -- 172.21.15.5:0/1814459744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ff3605e90 con 0x557ff35f6af0 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ad58e7640 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff360a0f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3acc097d10 tx=0x7f3acc0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a33038f8da78e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ab97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1814459744 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc014030 con 0x557ff35f6af0 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1814459744 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f3acc0cc7f0 con 0x557ff35f6af0 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1814459744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac8036040 con 0x557ff35f6af0 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1814459744 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3acc020070 con 0x557ff35f6af0 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ada9c1740 10 client.20814 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.711+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1814459744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557ff36809e0 con 0x557ff35f6af0 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.712+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1814459744 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac8036040 con 0x557ff35f6af0 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.712+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1814459744 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f3acc09a200 con 0x557ff35f6af0 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.712+0000 7f3ada9c1740 10 client.20814 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.712+0000 7f3ada9c1740 10 client.20814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.712+0000 7f3ada9c1740 20 client.20814 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.712+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557ff3680fc0 0x557ff36833b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.712+0000 7f3ada9c1740 4 client.20814 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.712+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1814459744 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ff3562d70 con 0x557ff3680fc0 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.712+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557ff3680fc0 0x557ff36833b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.713+0000 7f3ab97fa640 10 client.20814 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.713+0000 7f3ad50e6640 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557ff3680fc0 0x557ff36833b0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8be75855d3413c8 server_cookie=77cd6f34c14c0774 in_seq=0 out_seq=0 2024-09-11T22:39:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.977+0000 7f3ab97fa640 1 -- 172.21.15.5:0/1814459744 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ff3562d70 con 0x557ff3680fc0 2024-09-11T22:39:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.977+0000 7f3ab97fa640 10 client.20814 handle_command_reply: tid=0 2024-09-11T22:39:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.977+0000 7f3ada9c1740 1 client.20814 shutdown 2024-09-11T22:39:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.977+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1814459744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 msgr2=0x557ff360a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.977+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff360a0f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f3acc097d10 tx=0x7f3acc0048a0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.978+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1814459744 shutdown_connections 2024-09-11T22:39:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.978+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557ff3680fc0 0x557ff36833b0 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.978+0000 7f3ab97fa640 0 client.20814 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.978+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557ff3601cb0 0x557ff3605950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.978+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ff35f7410 0x557ff3605410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.978+0000 7f3ada9c1740 1 --2- 172.21.15.5:0/1814459744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ff35f6af0 0x557ff360a0f0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.978+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1814459744 >> 172.21.15.5:0/1814459744 conn(0x557ff35f3060 msgr2=0x557ff35f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.978+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1814459744 shutdown_connections 2024-09-11T22:39:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.978+0000 7f3ada9c1740 1 -- 172.21.15.5:0/1814459744 wait complete. 2024-09-11T22:39:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.979+0000 7f3ada9c1740 20 client.20814 trim_cache size 0 max 16384 2024-09-11T22:39:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:39:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:39:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:39:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:39:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.979+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/1262126710 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3abc0a2da0 msgr2=0x7f3abc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.979+0000 7f3ad7b4b640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3abc0a2da0 0x7f3abc0a5260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009a60 tx=0x7f3ac804c000 comp rx=0 tx=0).stop 2024-09-11T22:39:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.979+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/1262126710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad01009b0 msgr2=0x7f3ad0198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.979+0000 7f3ad7b4b640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad01009b0 0x7f3ad0198330 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00d770 tx=0x7f3acc00dc30 comp rx=0 tx=0).stop 2024-09-11T22:39:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.980+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/1262126710 shutdown_connections 2024-09-11T22:39:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.980+0000 7f3ad7b4b640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ad01069c0 0x7f3ad019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.980+0000 7f3ad7b4b640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3abc0a2da0 0x7f3abc0a5260 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.980+0000 7f3ad7b4b640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3ad0101350 0x7f3ad0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.980+0000 7f3ad7b4b640 1 --2- 172.21.15.5:0/1262126710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ad01009b0 0x7f3ad0198330 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.980+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/1262126710 >> 172.21.15.5:0/1262126710 conn(0x7f3ad0077990 msgr2=0x7f3ad00699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.980+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/1262126710 shutdown_connections 2024-09-11T22:39:34.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:34.980+0000 7f3ad7b4b640 1 -- 172.21.15.5:0/1262126710 wait complete. 2024-09-11T22:39:35.647 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 17.22 MB/sec execute 113 sec latency 7552.038 ms 2024-09-11T22:39:35.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:35 smithi005 ceph-mon[30943]: pgmap v554: 97 pgs: 97 active+clean; 498 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.4 MiB/s wr, 379 op/s 2024-09-11T22:39:35.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:35 smithi172 ceph-mon[35461]: pgmap v554: 97 pgs: 97 active+clean; 498 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.4 MiB/s wr, 379 op/s 2024-09-11T22:39:36.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:35 smithi159 ceph-mon[35534]: pgmap v554: 97 pgs: 97 active+clean; 498 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.4 MiB/s wr, 379 op/s 2024-09-11T22:39:36.647 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 17.07 MB/sec execute 114 sec latency 8552.136 ms 2024-09-11T22:39:37.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 16.92 MB/sec execute 115 sec latency 9552.243 ms 2024-09-11T22:39:37.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:37 smithi172 ceph-mon[35461]: pgmap v555: 97 pgs: 97 active+clean; 498 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.2 MiB/s wr, 367 op/s 2024-09-11T22:39:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:37 smithi005 ceph-mon[30943]: pgmap v555: 97 pgs: 97 active+clean; 498 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.2 MiB/s wr, 367 op/s 2024-09-11T22:39:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:37 smithi159 ceph-mon[35534]: pgmap v555: 97 pgs: 97 active+clean; 498 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.2 MiB/s wr, 367 op/s 2024-09-11T22:39:38.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 16.78 MB/sec execute 116 sec latency 10552.367 ms 2024-09-11T22:39:39.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 16.63 MB/sec execute 117 sec latency 11552.427 ms 2024-09-11T22:39:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:39 smithi005 ceph-mon[30943]: pgmap v556: 97 pgs: 97 active+clean; 484 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.6 MiB/s wr, 482 op/s 2024-09-11T22:39:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:39:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:39:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:39:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:39:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:39 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:39:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:39 smithi159 ceph-mon[35534]: pgmap v556: 97 pgs: 97 active+clean; 484 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.6 MiB/s wr, 482 op/s 2024-09-11T22:39:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:39:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:39:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:39:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:39:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:39 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:39:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:39 smithi172 ceph-mon[35461]: pgmap v556: 97 pgs: 97 active+clean; 484 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.6 MiB/s wr, 482 op/s 2024-09-11T22:39:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:39:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:39:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:39:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:39:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:39 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:39:40.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 16.49 MB/sec execute 118 sec latency 12552.525 ms 2024-09-11T22:39:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:40 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:39:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:40 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:39:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:40 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:39:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:40 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:39:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:40 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:39:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:40 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:39:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:40 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:39:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:40 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:39:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:40 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:39:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:40 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:39:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:40 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:39:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:40 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:39:41.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 16.35 MB/sec execute 119 sec latency 13552.602 ms 2024-09-11T22:39:42.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:41 smithi005 ceph-mon[30943]: pgmap v557: 97 pgs: 97 active+clean; 484 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 265 op/s 2024-09-11T22:39:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:41 smithi159 ceph-mon[35534]: pgmap v557: 97 pgs: 97 active+clean; 484 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 265 op/s 2024-09-11T22:39:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:41 smithi172 ceph-mon[35461]: pgmap v557: 97 pgs: 97 active+clean; 484 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 265 op/s 2024-09-11T22:39:42.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 16.22 MB/sec execute 120 sec latency 14552.713 ms 2024-09-11T22:39:43.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 16.08 MB/sec execute 121 sec latency 15552.784 ms 2024-09-11T22:39:44.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:43 smithi005 ceph-mon[30943]: pgmap v558: 97 pgs: 97 active+clean; 488 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-09-11T22:39:44.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:43 smithi159 ceph-mon[35534]: pgmap v558: 97 pgs: 97 active+clean; 488 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-09-11T22:39:44.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:43 smithi172 ceph-mon[35461]: pgmap v558: 97 pgs: 97 active+clean; 488 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-09-11T22:39:44.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 15.95 MB/sec execute 122 sec latency 16552.884 ms 2024-09-11T22:39:45.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 15.82 MB/sec execute 123 sec latency 17552.952 ms 2024-09-11T22:39:46.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:45 smithi005 ceph-mon[30943]: pgmap v559: 97 pgs: 97 active+clean; 492 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-09-11T22:39:46.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:45 smithi159 ceph-mon[35534]: pgmap v559: 97 pgs: 97 active+clean; 492 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-09-11T22:39:46.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:45 smithi172 ceph-mon[35461]: pgmap v559: 97 pgs: 97 active+clean; 492 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 1.1 MiB/s wr, 176 op/s 2024-09-11T22:39:46.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 15.70 MB/sec execute 124 sec latency 18553.005 ms 2024-09-11T22:39:47.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:39:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:47.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 15.57 MB/sec execute 125 sec latency 19553.035 ms 2024-09-11T22:39:47.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:47 smithi005 ceph-mon[30943]: pgmap v560: 97 pgs: 97 active+clean; 492 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 951 KiB/s wr, 126 op/s 2024-09-11T22:39:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.890+0000 7fb137cf4640 1 -- 172.21.15.5:0/4125554500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb130102570 msgr2=0x7fb1301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.890+0000 7fb137cf4640 1 --2- 172.21.15.5:0/4125554500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb130102570 0x7fb1301029b0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fb120009900 tx=0x7fb12002a480 comp rx=0 tx=0).stop 2024-09-11T22:39:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.890+0000 7fb137cf4640 1 -- 172.21.15.5:0/4125554500 shutdown_connections 2024-09-11T22:39:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.890+0000 7fb137cf4640 1 --2- 172.21.15.5:0/4125554500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb130102ef0 0x7fb13010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.890+0000 7fb137cf4640 1 --2- 172.21.15.5:0/4125554500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb130102570 0x7fb1301029b0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.890+0000 7fb137cf4640 1 --2- 172.21.15.5:0/4125554500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb130100e20 0x7fb130101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.890+0000 7fb137cf4640 1 -- 172.21.15.5:0/4125554500 >> 172.21.15.5:0/4125554500 conn(0x7fb1300fc7d0 msgr2=0x7fb1300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.891+0000 7fb137cf4640 1 -- 172.21.15.5:0/4125554500 shutdown_connections 2024-09-11T22:39:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.891+0000 7fb137cf4640 1 -- 172.21.15.5:0/4125554500 wait complete. 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.891+0000 7fb137cf4640 1 Processor -- start 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.891+0000 7fb137cf4640 1 -- start start 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb137cf4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb130100e20 0x7fb1301970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb137cf4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb130102570 0x7fb130197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb137cf4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb130102ef0 0x7fb130197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb137cf4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb130110f80 con 0x7fb130102ef0 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb137cf4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb130110e00 con 0x7fb130102570 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb137cf4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb130111100 con 0x7fb130100e20 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb136291640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb130102ef0 0x7fb130197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb136291640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb130102ef0 0x7fb130197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35136/0 (socket says 172.21.15.5:35136) 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb136291640 1 -- 172.21.15.5:0/297223217 learned_addr learned my addr 172.21.15.5:0/297223217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:47.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb13528f640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb130102570 0x7fb130197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb135a90640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb130100e20 0x7fb1301970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb136291640 1 -- 172.21.15.5:0/297223217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb130100e20 msgr2=0x7fb1301970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb136291640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb130100e20 0x7fb1301970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb136291640 1 -- 172.21.15.5:0/297223217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb130102570 msgr2=0x7fb130197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb136291640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb130102570 0x7fb130197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb136291640 1 -- 172.21.15.5:0/297223217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1301a6950 con 0x7fb130102ef0 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb135a90640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb130100e20 0x7fb1301970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb136291640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb130102ef0 0x7fb130197b60 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fb12800eaa0 tx=0x7fb12800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb126ffd640 1 -- 172.21.15.5:0/297223217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12800cdb0 con 0x7fb130102ef0 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb126ffd640 1 -- 172.21.15.5:0/297223217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb128005020 con 0x7fb130102ef0 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb137cf4640 1 -- 172.21.15.5:0/297223217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1301a6c40 con 0x7fb130102ef0 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb126ffd640 1 -- 172.21.15.5:0/297223217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1280116b0 con 0x7fb130102ef0 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.892+0000 7fb137cf4640 1 -- 172.21.15.5:0/297223217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1301a7100 con 0x7fb130102ef0 2024-09-11T22:39:47.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.893+0000 7fb137cf4640 1 -- 172.21.15.5:0/297223217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb130101200 con 0x7fb130102ef0 2024-09-11T22:39:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.895+0000 7fb126ffd640 1 -- 172.21.15.5:0/297223217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb128005560 con 0x7fb130102ef0 2024-09-11T22:39:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.895+0000 7fb126ffd640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb11c0a2cf0 0x7fb11c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.895+0000 7fb135a90640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb11c0a2cf0 0x7fb11c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.895+0000 7fb126ffd640 1 -- 172.21.15.5:0/297223217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fb1280cfab0 con 0x7fb130102ef0 2024-09-11T22:39:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.896+0000 7fb135a90640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb11c0a2cf0 0x7fb11c0a51b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0058c0 tx=0x7fb12c005ec0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:47.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:47.899+0000 7fb126ffd640 1 -- 172.21.15.5:0/297223217 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb128016030 con 0x7fb130102ef0 2024-09-11T22:39:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 -- 172.21.15.5:0/3640111357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d01009b0 msgr2=0x7fe3d0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 --2- 172.21.15.5:0/3640111357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d01009b0 0x7fe3d0100e10 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009930 tx=0x7fe3c402a420 comp rx=0 tx=0).stop 2024-09-11T22:39:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 -- 172.21.15.5:0/3640111357 shutdown_connections 2024-09-11T22:39:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 --2- 172.21.15.5:0/3640111357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3d0101350 0x7fe3d010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 --2- 172.21.15.5:0/3640111357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d01009b0 0x7fe3d0100e10 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 --2- 172.21.15.5:0/3640111357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d01069c0 0x7fe3d0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 -- 172.21.15.5:0/3640111357 >> 172.21.15.5:0/3640111357 conn(0x7fe3d0077990 msgr2=0x7fe3d00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 -- 172.21.15.5:0/3640111357 shutdown_connections 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 -- 172.21.15.5:0/3640111357 wait complete. 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 Processor -- start 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.074+0000 7fe3d8576640 1 -- start start 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d8576640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d01009b0 0x7fe3d0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d8576640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3d0101350 0x7fe3d0197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d8576640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d01069c0 0x7fe3d019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d8576640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0110dc0 con 0x7fe3d01069c0 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d8576640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0110c40 con 0x7fe3d01009b0 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d8576640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d0110f40 con 0x7fe3d0101350 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6b13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d01069c0 0x7fe3d019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6b13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d01069c0 0x7fe3d019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35152/0 (socket says 172.21.15.5:35152) 2024-09-11T22:39:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6b13640 1 -- 172.21.15.5:0/3670829803 learned_addr learned my addr 172.21.15.5:0/3670829803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6312640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d01009b0 0x7fe3d0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3d0101350 0x7fe3d0197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6b13640 1 -- 172.21.15.5:0/3670829803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3d0101350 msgr2=0x7fe3d0197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6b13640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3d0101350 0x7fe3d0197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6b13640 1 -- 172.21.15.5:0/3670829803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d01009b0 msgr2=0x7fe3d0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6b13640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d01009b0 0x7fe3d0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6b13640 1 -- 172.21.15.5:0/3670829803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d019c240 con 0x7fe3d01069c0 2024-09-11T22:39:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6312640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d01009b0 0x7fe3d0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3d0101350 0x7fe3d0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:39:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.075+0000 7fe3d6b13640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d01069c0 0x7fe3d019bb10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800e970 tx=0x7fe3c800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.076+0000 7fe3cf7fe640 1 -- 172.21.15.5:0/3670829803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c800ccc0 con 0x7fe3d01069c0 2024-09-11T22:39:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.076+0000 7fe3cf7fe640 1 -- 172.21.15.5:0/3670829803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c800ce60 con 0x7fe3d01069c0 2024-09-11T22:39:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.076+0000 7fe3d8576640 1 -- 172.21.15.5:0/3670829803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d019c530 con 0x7fe3d01069c0 2024-09-11T22:39:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.076+0000 7fe3cf7fe640 1 -- 172.21.15.5:0/3670829803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8010490 con 0x7fe3d01069c0 2024-09-11T22:39:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.076+0000 7fe3d8576640 1 -- 172.21.15.5:0/3670829803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d01a3dc0 con 0x7fe3d01069c0 2024-09-11T22:39:48.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:47 smithi159 ceph-mon[35534]: pgmap v560: 97 pgs: 97 active+clean; 492 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 951 KiB/s wr, 126 op/s 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.078+0000 7fe3db3ed740 1 Processor -- start 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3cf7fe640 1 -- 172.21.15.5:0/3670829803 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe3c8005020 con 0x7fe3d01069c0 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3db3ed740 1 -- start start 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd0af0 0x560b3fdd0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 0x560b3fddb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdde0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560b3fd352f0 con 0x560b3fddbcb0 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560b3fcdd270 con 0x560b3fdd1410 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b3fd312b0 con 0x560b3fdd0af0 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3d5b11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdde0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3d5b11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdde0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35160/0 (socket says 172.21.15.5:35160) 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3d5b11640 1 -- 172.21.15.5:0/3952323104 learned_addr learned my addr 172.21.15.5:0/3952323104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3d6312640 1 --2- 172.21.15.5:0/3952323104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 0x560b3fddb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3cf7fe640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3b80a2e20 0x7fe3b80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.079+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/3952323104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd0af0 0x560b3fdd0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3d6b13640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3b80a2e20 0x7fe3b80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3d5b11640 1 -- 172.21.15.5:0/3952323104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd0af0 msgr2=0x560b3fdd0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/3952323104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd0af0 0x560b3fdd0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3d5b11640 1 -- 172.21.15.5:0/3952323104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 msgr2=0x560b3fddb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/3952323104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 0x560b3fddb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3d5b11640 1 -- 172.21.15.5:0/3952323104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b3fcdc340 con 0x560b3fddbcb0 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3cf7fe640 1 -- 172.21.15.5:0/3670829803 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fe3c80cf9b0 con 0x7fe3d01069c0 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/3952323104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdde0a0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4005e00 tx=0x7fe3c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4240e6ed5cef5210 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:48.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3ccbfe640 1 -- 172.21.15.5:0/3952323104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4035040 con 0x560b3fddbcb0 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3d6b13640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3b80a2e20 0x7fe3b80a52e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80cd140 tx=0x7fe3c806de20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3ccbfe640 1 -- 172.21.15.5:0/3952323104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c4032d30 con 0x560b3fddbcb0 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.080+0000 7fe3ccbfe640 1 -- 172.21.15.5:0/3952323104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40038e0 con 0x560b3fddbcb0 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.081+0000 7fe3db3ed740 1 -- 172.21.15.5:0/3952323104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 msgr2=0x560b3fdde0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.081+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/3952323104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdde0a0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4005e00 tx=0x7fe3c4000bc0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.081+0000 7fe3db3ed740 1 -- 172.21.15.5:0/3952323104 shutdown_connections 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.081+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/3952323104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdde0a0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.081+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/3952323104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 0x560b3fddb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.081+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/3952323104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd0af0 0x560b3fdd0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.081+0000 7fe3db3ed740 1 -- 172.21.15.5:0/3952323104 >> 172.21.15.5:0/3952323104 conn(0x560b3fdcd060 msgr2=0x560b3fdcd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:48.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.081+0000 7fe3db3ed740 1 -- 172.21.15.5:0/3952323104 shutdown_connections 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.081+0000 7fe3db3ed740 1 -- 172.21.15.5:0/3952323104 wait complete. 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.081+0000 7fe3db3ed740 1 -- start start 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3db3ed740 1 Processor -- start 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd0af0 0x560b3fdd8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd1410 0x560b3fdd90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fddbcb0 0x560b3fdd9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560b3fdde880 con 0x560b3fdd1410 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560b3fcbe290 con 0x560b3fddbcb0 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x560b3fddea00 con 0x560b3fdd0af0 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3db3ed740 10 client.0 _ll_register_callbacks cb 0x560b3fcef140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d6312640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd1410 0x560b3fdd90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d6312640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd1410 0x560b3fdd90f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35162/0 (socket says 172.21.15.5:35162) 2024-09-11T22:39:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d6312640 1 -- 172.21.15.5:0/2421409286 learned_addr learned my addr 172.21.15.5:0/2421409286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fddbcb0 0x560b3fdd9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d6312640 1 -- 172.21.15.5:0/2421409286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd0af0 msgr2=0x560b3fdd8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd0af0 0x560b3fdd8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d6312640 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd0af0 0x560b3fdd8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d6312640 1 -- 172.21.15.5:0/2421409286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fddbcb0 msgr2=0x560b3fdd9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d6312640 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fddbcb0 0x560b3fdd9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d6312640 1 -- 172.21.15.5:0/2421409286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b3fa48210 con 0x560b3fdd1410 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.082+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fddbcb0 0x560b3fdd9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3d6312640 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd1410 0x560b3fdd90f0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0003c00 tx=0x7fe3c0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ad5d70e4f04c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/2421409286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0004d20 con 0x560b3fdd1410 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/2421409286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fe3c0011710 con 0x560b3fdd1410 2024-09-11T22:39:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3db3ed740 1 -- 172.21.15.5:0/2421409286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c4032d30 con 0x560b3fdd1410 2024-09-11T22:39:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/2421409286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe3c0011910 con 0x560b3fdd1410 2024-09-11T22:39:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3db3ed740 10 client.20832 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3db3ed740 1 -- 172.21.15.5:0/2421409286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560b3fb14620 con 0x560b3fdd1410 2024-09-11T22:39:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/2421409286 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c4032d30 con 0x560b3fdd1410 2024-09-11T22:39:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/2421409286 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c0019030 con 0x560b3fdd1410 2024-09-11T22:39:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3db3ed740 10 client.20832 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3db3ed740 10 client.20832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3db3ed740 20 client.20832 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560b3fdebd00 0x560b3fdee0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3db3ed740 4 client.20832 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.083+0000 7fe3db3ed740 1 -- 172.21.15.5:0/2421409286 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b3fd42a20 con 0x560b3fdebd00 2024-09-11T22:39:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.084+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560b3fdebd00 0x560b3fdee0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.084+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560b3fdebd00 0x560b3fdee0f0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1514093d5a1baf server_cookie=7f0f0cee6df99304 in_seq=0 out_seq=0 2024-09-11T22:39:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.084+0000 7fe3b5ffb640 10 client.20832 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/2421409286 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b3fd42a20 con 0x560b3fdebd00 2024-09-11T22:39:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3b5ffb640 10 client.20832 handle_command_reply: tid=0 2024-09-11T22:39:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 client.20832 shutdown 2024-09-11T22:39:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 -- 172.21.15.5:0/2421409286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd1410 msgr2=0x560b3fdd90f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd1410 0x560b3fdd90f0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0003c00 tx=0x7fe3c0004410 comp rx=0 tx=0).stop 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 -- 172.21.15.5:0/2421409286 shutdown_connections 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560b3fdebd00 0x560b3fdee0f0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fddbcb0 0x560b3fdd9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd1410 0x560b3fdd90f0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3b5ffb640 0 client.20832 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/2421409286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd0af0 0x560b3fdd8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 -- 172.21.15.5:0/2421409286 >> 172.21.15.5:0/2421409286 conn(0x560b3fdcd060 msgr2=0x560b3fdd2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 -- 172.21.15.5:0/2421409286 shutdown_connections 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 1 -- 172.21.15.5:0/2421409286 wait complete. 2024-09-11T22:39:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.085+0000 7fe3db3ed740 20 client.20832 trim_cache size 0 max 16384 2024-09-11T22:39:48.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:47 smithi172 ceph-mon[35461]: pgmap v560: 97 pgs: 97 active+clean; 492 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 951 KiB/s wr, 126 op/s 2024-09-11T22:39:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.111+0000 7fe3db3ed740 1 Processor -- start 2024-09-11T22:39:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.111+0000 7fe3db3ed740 1 -- start start 2024-09-11T22:39:48.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.111+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd0af0 0x560b3fdd9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.111+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 0x560b3fdf39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.111+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fddbcb0 0x560b3fdf3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.111+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01ab870 con 0x560b3fdd0af0 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.111+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01ad3b0 con 0x560b3fdd1410 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.111+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01abf40 con 0x560b3fddbcb0 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d5b11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd0af0 0x560b3fdd9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d5b11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd0af0 0x560b3fdd9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35178/0 (socket says 172.21.15.5:35178) 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d5b11640 1 -- 172.21.15.5:0/912332894 learned_addr learned my addr 172.21.15.5:0/912332894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d6312640 1 --2- 172.21.15.5:0/912332894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 0x560b3fdf39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d5b11640 1 -- 172.21.15.5:0/912332894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fddbcb0 msgr2=0x560b3fdf3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/912332894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fddbcb0 0x560b3fdf3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d5b11640 1 -- 172.21.15.5:0/912332894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 msgr2=0x560b3fdf39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/912332894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 0x560b3fdf39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d5b11640 1 -- 172.21.15.5:0/912332894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b3fdd3a80 con 0x560b3fdd0af0 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d6312640 1 --2- 172.21.15.5:0/912332894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 0x560b3fdf39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/912332894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd0af0 0x560b3fdd9460 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4005e30 tx=0x7fe3c4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d40cc5fbd51aa6a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3b7fff640 1 -- 172.21.15.5:0/912332894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4035040 con 0x560b3fdd0af0 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3b7fff640 1 -- 172.21.15.5:0/912332894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c4032d30 con 0x560b3fdd0af0 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.112+0000 7fe3b7fff640 1 -- 172.21.15.5:0/912332894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4004920 con 0x560b3fdd0af0 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 -- 172.21.15.5:0/912332894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd0af0 msgr2=0x560b3fdd9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/912332894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd0af0 0x560b3fdd9460 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4005e30 tx=0x7fe3c4000bc0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 -- 172.21.15.5:0/912332894 shutdown_connections 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/912332894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fddbcb0 0x560b3fdf3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/912332894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd1410 0x560b3fdf39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/912332894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fdd0af0 0x560b3fdd9460 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 -- 172.21.15.5:0/912332894 >> 172.21.15.5:0/912332894 conn(0x560b3fdcd060 msgr2=0x560b3fddac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 -- 172.21.15.5:0/912332894 shutdown_connections 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 -- 172.21.15.5:0/912332894 wait complete. 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 -- start start 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.113+0000 7fe3db3ed740 1 Processor -- start 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd0af0 0x560b3fde4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd1410 0x560b3fdd59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3db3ed740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdd5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01acce0 con 0x560b3fddbcb0 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01ac610 con 0x560b3fdd0af0 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3db3ed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d01a9870 con 0x560b3fdd1410 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3db3ed740 10 client.0 _ll_register_callbacks cb 0x7fe3c0009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d5b11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdd5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d5b11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdd5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35186/0 (socket says 172.21.15.5:35186) 2024-09-11T22:39:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d5b11640 1 -- 172.21.15.5:0/1067574068 learned_addr learned my addr 172.21.15.5:0/1067574068 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d6312640 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd1410 0x560b3fdd59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd0af0 0x560b3fde4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d5b11640 1 -- 172.21.15.5:0/1067574068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd1410 msgr2=0x560b3fdd59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd1410 0x560b3fdd59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d5b11640 1 -- 172.21.15.5:0/1067574068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd0af0 msgr2=0x560b3fde4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd0af0 0x560b3fde4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d5b11640 1 -- 172.21.15.5:0/1067574068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b3fdd67f0 con 0x560b3fddbcb0 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3d5b11640 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdd5fc0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4033000 tx=0x7fe3c4031d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e3270552a4914f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/1067574068 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4035030 con 0x560b3fddbcb0 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/1067574068 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fe3c4004310 con 0x560b3fddbcb0 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.114+0000 7fe3db3ed740 1 -- 172.21.15.5:0/1067574068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c4032d30 con 0x560b3fddbcb0 2024-09-11T22:39:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/1067574068 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe3c4005140 con 0x560b3fddbcb0 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3db3ed740 10 client.20838 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3db3ed740 1 -- 172.21.15.5:0/1067574068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x560b3fe5ae30 con 0x560b3fddbcb0 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/1067574068 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c4032d30 con 0x560b3fddbcb0 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/1067574068 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c40035f0 con 0x560b3fddbcb0 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3db3ed740 10 client.20838 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3db3ed740 10 client.20838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3db3ed740 20 client.20838 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560b3fe5b0b0 0x560b3fe5d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3db3ed740 4 client.20838 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3db3ed740 1 -- 172.21.15.5:0/1067574068 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b3fd3cd70 con 0x560b3fe5b0b0 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.115+0000 7fe3d6312640 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560b3fe5b0b0 0x560b3fe5d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.116+0000 7fe3b5ffb640 10 client.20838 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.116+0000 7fe3d6312640 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560b3fe5b0b0 0x560b3fe5d4a0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a90c096408ccb7 server_cookie=9b7322d8da21545 in_seq=0 out_seq=0 2024-09-11T22:39:48.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.131+0000 7fb137cf4640 1 -- 172.21.15.5:0/297223217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb13006a560 con 0x7fb130102ef0 2024-09-11T22:39:48.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.132+0000 7fb126ffd640 1 -- 172.21.15.5:0/297223217 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11967 (secure 0 0 0) 0x7fb13006a560 con 0x7fb130102ef0 2024-09-11T22:39:48.133 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:39:48.133 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"btime":"2024-09-11T22:38:14:548820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:38:13.865471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:39:48.133 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2024-09-11T22:39:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.137+0000 7fb137cf4640 1 -- 172.21.15.5:0/297223217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb11c0a2cf0 msgr2=0x7fb11c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.137+0000 7fb137cf4640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb11c0a2cf0 0x7fb11c0a51b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0058c0 tx=0x7fb12c005ec0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.137+0000 7fb137cf4640 1 -- 172.21.15.5:0/297223217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb130102ef0 msgr2=0x7fb130197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.137+0000 7fb137cf4640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb130102ef0 0x7fb130197b60 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fb12800eaa0 tx=0x7fb12800ef60 comp rx=0 tx=0).stop 2024-09-11T22:39:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.140+0000 7fb137cf4640 1 -- 172.21.15.5:0/297223217 shutdown_connections 2024-09-11T22:39:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.140+0000 7fb137cf4640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb130102ef0 0x7fb130197b60 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.141+0000 7fb137cf4640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb11c0a2cf0 0x7fb11c0a51b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.141+0000 7fb137cf4640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb130102570 0x7fb130197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.141+0000 7fb137cf4640 1 --2- 172.21.15.5:0/297223217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb130100e20 0x7fb1301970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.141+0000 7fb137cf4640 1 -- 172.21.15.5:0/297223217 >> 172.21.15.5:0/297223217 conn(0x7fb1300fc7d0 msgr2=0x7fb1300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.144+0000 7fb137cf4640 1 -- 172.21.15.5:0/297223217 shutdown_connections 2024-09-11T22:39:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:48.144+0000 7fb137cf4640 1 -- 172.21.15.5:0/297223217 wait complete. 2024-09-11T22:39:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3b5ffb640 1 -- 172.21.15.5:0/1067574068 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b3fd3cd70 con 0x560b3fe5b0b0 2024-09-11T22:39:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3b5ffb640 10 client.20838 handle_command_reply: tid=0 2024-09-11T22:39:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3db3ed740 1 client.20838 shutdown 2024-09-11T22:39:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3db3ed740 1 -- 172.21.15.5:0/1067574068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 msgr2=0x560b3fdd5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdd5fc0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4033000 tx=0x7fe3c4031d30 comp rx=0 tx=0).stop 2024-09-11T22:39:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3db3ed740 1 -- 172.21.15.5:0/1067574068 shutdown_connections 2024-09-11T22:39:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x560b3fe5b0b0 0x560b3fe5d4a0 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b3fddbcb0 0x560b3fdd5fc0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3b5ffb640 0 client.20838 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b3fdd1410 0x560b3fdd59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3db3ed740 1 --2- 172.21.15.5:0/1067574068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560b3fdd0af0 0x560b3fde4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.182+0000 7fe3db3ed740 1 -- 172.21.15.5:0/1067574068 >> 172.21.15.5:0/1067574068 conn(0x560b3fdcd060 msgr2=0x560b3fdda410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3db3ed740 1 -- 172.21.15.5:0/1067574068 shutdown_connections 2024-09-11T22:39:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3db3ed740 1 -- 172.21.15.5:0/1067574068 wait complete. 2024-09-11T22:39:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3db3ed740 20 client.20838 trim_cache size 0 max 16384 2024-09-11T22:39:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:39:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:39:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:39:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:39:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 -- 172.21.15.5:0/3670829803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3b80a2e20 msgr2=0x7fe3b80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3b80a2e20 0x7fe3b80a52e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80cd140 tx=0x7fe3c806de20 comp rx=0 tx=0).stop 2024-09-11T22:39:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 -- 172.21.15.5:0/3670829803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d01069c0 msgr2=0x7fe3d019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d01069c0 0x7fe3d019bb10 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800e970 tx=0x7fe3c800ed70 comp rx=0 tx=0).stop 2024-09-11T22:39:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 -- 172.21.15.5:0/3670829803 shutdown_connections 2024-09-11T22:39:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d01069c0 0x7fe3d019bb10 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3d0101350 0x7fe3d0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe3b80a2e20 0x7fe3b80a52e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 --2- 172.21.15.5:0/3670829803 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3d01009b0 0x7fe3d0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 -- 172.21.15.5:0/3670829803 >> 172.21.15.5:0/3670829803 conn(0x7fe3d0077990 msgr2=0x7fe3d01037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 -- 172.21.15.5:0/3670829803 shutdown_connections 2024-09-11T22:39:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:39:48.183+0000 7fe3d8576640 1 -- 172.21.15.5:0/3670829803 wait complete. 2024-09-11T22:39:48.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 15.45 MB/sec execute 126 sec latency 20553.064 ms 2024-09-11T22:39:49.067 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:48 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/297223217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:49.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:48 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/297223217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:49.102 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --set-id a49692cd --release --await' 2024-09-11T22:39:49.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --set-id a49692cd --release --await 2024-09-11T22:39:49.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:48 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/297223217' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:39:49.367 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:39:49.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60965 15.32 MB/sec execute 127 sec latency 21553.094 ms 2024-09-11T22:39:49.763 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:49 smithi005 ceph-mon[30943]: pgmap v561: 97 pgs: 97 active+clean; 493 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-09-11T22:39:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.853+0000 7fdd68ca8640 1 -- 172.21.15.5:0/4251612083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6410c170 msgr2=0x7fdd6410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.853+0000 7fdd68ca8640 1 --2- 172.21.15.5:0/4251612083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6410c170 0x7fdd6410e610 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009ca0 tx=0x7fdd5c02a950 comp rx=0 tx=0).stop 2024-09-11T22:39:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.853+0000 7fdd68ca8640 1 -- 172.21.15.5:0/4251612083 shutdown_connections 2024-09-11T22:39:49.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.853+0000 7fdd68ca8640 1 --2- 172.21.15.5:0/4251612083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6410c170 0x7fdd6410e610 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.853+0000 7fdd68ca8640 1 --2- 172.21.15.5:0/4251612083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd64103660 0x7fdd6410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.853+0000 7fdd68ca8640 1 --2- 172.21.15.5:0/4251612083 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd64102d40 0x7fdd64103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.853+0000 7fdd68ca8640 1 -- 172.21.15.5:0/4251612083 >> 172.21.15.5:0/4251612083 conn(0x7fdd640fc7a0 msgr2=0x7fdd640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.853+0000 7fdd68ca8640 1 -- 172.21.15.5:0/4251612083 shutdown_connections 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.853+0000 7fdd68ca8640 1 -- 172.21.15.5:0/4251612083 wait complete. 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd68ca8640 1 Processor -- start 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd68ca8640 1 -- start start 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd68ca8640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd64102d40 0x7fdd641009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd68ca8640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd64103660 0x7fdd64100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd68ca8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6410c170 0x7fdd64101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd68ca8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64111f70 con 0x7fdd6410c170 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd68ca8640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64111df0 con 0x7fdd64102d40 2024-09-11T22:39:49.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd68ca8640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641120f0 con 0x7fdd64103660 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd62d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6410c170 0x7fdd64101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd62d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6410c170 0x7fdd64101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35200/0 (socket says 172.21.15.5:35200) 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd62d9d640 1 -- 172.21.15.5:0/3266648166 learned_addr learned my addr 172.21.15.5:0/3266648166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd64103660 0x7fdd64100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd6259c640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd64102d40 0x7fdd641009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd62d9d640 1 -- 172.21.15.5:0/3266648166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd64103660 msgr2=0x7fdd64100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd62d9d640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd64103660 0x7fdd64100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd62d9d640 1 -- 172.21.15.5:0/3266648166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd64102d40 msgr2=0x7fdd641009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd62d9d640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd64102d40 0x7fdd641009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.854+0000 7fdd62d9d640 1 -- 172.21.15.5:0/3266648166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd641a68b0 con 0x7fdd6410c170 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.855+0000 7fdd62d9d640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6410c170 0x7fdd64101420 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c02ae70 tx=0x7fdd5c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.855+0000 7fdd537fe640 1 -- 172.21.15.5:0/3266648166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c034040 con 0x7fdd6410c170 2024-09-11T22:39:49.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.855+0000 7fdd68ca8640 1 -- 172.21.15.5:0/3266648166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd641a6b40 con 0x7fdd6410c170 2024-09-11T22:39:49.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.855+0000 7fdd537fe640 1 -- 172.21.15.5:0/3266648166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd5c030920 con 0x7fdd6410c170 2024-09-11T22:39:49.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.855+0000 7fdd537fe640 1 -- 172.21.15.5:0/3266648166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c045480 con 0x7fdd6410c170 2024-09-11T22:39:49.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.855+0000 7fdd68ca8640 1 -- 172.21.15.5:0/3266648166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd641a7070 con 0x7fdd6410c170 2024-09-11T22:39:49.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.857+0000 7fdd537fe640 1 -- 172.21.15.5:0/3266648166 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fdd5c037020 con 0x7fdd6410c170 2024-09-11T22:39:49.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6b081740 1 Processor -- start 2024-09-11T22:39:49.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6b081740 1 -- start start 2024-09-11T22:39:49.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b86440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b90d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b93630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6b081740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c199aeb620 con 0x55c199b91240 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6b081740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c199a92e50 con 0x55c199b86980 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6b081740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c199ae75e0 con 0x55c199b86060 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd537fe640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd4c0a2cf0 0x7fdd4c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b90d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b90d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45192/0 (socket says 172.21.15.5:45192) 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd6259c640 1 -- 172.21.15.5:0/4201050549 learned_addr learned my addr 172.21.15.5:0/4201050549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.858+0000 7fdd61d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b93630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd62d9d640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd4c0a2cf0 0x7fdd4c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/4201050549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b86440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd6259c640 1 -- 172.21.15.5:0/4201050549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 msgr2=0x55c199b86440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd6259c640 1 --2- 172.21.15.5:0/4201050549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b86440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd6259c640 1 -- 172.21.15.5:0/4201050549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 msgr2=0x55c199b93630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd6259c640 1 --2- 172.21.15.5:0/4201050549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b93630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd6259c640 1 -- 172.21.15.5:0/4201050549 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c199a83d40 con 0x55c199b86980 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd537fe640 1 -- 172.21.15.5:0/3266648166 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fdd5c0f02b0 con 0x7fdd6410c170 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd6259c640 1 --2- 172.21.15.5:0/4201050549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b90d00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400bcf0 tx=0x7fdd5400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1873572716f08101 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd62d9d640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd4c0a2cf0 0x7fdd4c0a51b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0b8cd0 tx=0x7fdd5c0e7400 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd50ff9640 1 -- 172.21.15.5:0/4201050549 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54015070 con 0x55c199b86980 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd50ff9640 1 -- 172.21.15.5:0/4201050549 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd54011920 con 0x55c199b86980 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.859+0000 7fdd50ff9640 1 -- 172.21.15.5:0/4201050549 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54010930 con 0x55c199b86980 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.860+0000 7fdd6b081740 1 -- 172.21.15.5:0/4201050549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 msgr2=0x55c199b90d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:49.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.860+0000 7fdd6b081740 1 --2- 172.21.15.5:0/4201050549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b90d00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400bcf0 tx=0x7fdd5400f730 comp rx=0 tx=0).stop 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.860+0000 7fdd6b081740 1 -- 172.21.15.5:0/4201050549 shutdown_connections 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.860+0000 7fdd6b081740 1 --2- 172.21.15.5:0/4201050549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b93630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.860+0000 7fdd6b081740 1 --2- 172.21.15.5:0/4201050549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b90d00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.860+0000 7fdd6b081740 1 --2- 172.21.15.5:0/4201050549 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b86440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.860+0000 7fdd6b081740 1 -- 172.21.15.5:0/4201050549 >> 172.21.15.5:0/4201050549 conn(0x55c199b825f0 msgr2=0x55c199b829d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.860+0000 7fdd6b081740 1 -- 172.21.15.5:0/4201050549 shutdown_connections 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.860+0000 7fdd6b081740 1 -- 172.21.15.5:0/4201050549 wait complete. 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd6b081740 1 -- start start 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd6b081740 1 Processor -- start 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b8e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b8e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b8ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd6b081740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c199b93e10 con 0x55c199b91240 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd6b081740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c199a73f10 con 0x55c199b86980 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd6b081740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c199b93f90 con 0x55c199b86060 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd6b081740 10 client.0 _ll_register_callbacks cb 0x55c199a97db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd61d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b8ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd61d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b8ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35228/0 (socket says 172.21.15.5:35228) 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd61d9b640 1 -- 172.21.15.5:0/3481726259 learned_addr learned my addr 172.21.15.5:0/3481726259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd6259c640 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b8e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b8e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd61d9b640 1 -- 172.21.15.5:0/3481726259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 msgr2=0x55c199b8e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b8e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd61d9b640 1 -- 172.21.15.5:0/3481726259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 msgr2=0x55c199b8e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:49.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b8e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.861+0000 7fdd61d9b640 1 -- 172.21.15.5:0/3481726259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1997fa580 con 0x55c199b91240 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b8ebd0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fdd58012c90 tx=0x7fdd58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0d7e86c1a8cb40 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd49ffb640 1 -- 172.21.15.5:0/3481726259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5801b040 con 0x55c199b91240 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd49ffb640 1 -- 172.21.15.5:0/3481726259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fdd58004030 con 0x55c199b91240 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd6b081740 1 -- 172.21.15.5:0/3481726259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd54011920 con 0x55c199b91240 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd49ffb640 1 -- 172.21.15.5:0/3481726259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdd58010e60 con 0x55c199b91240 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd6b081740 10 client.20850 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd6b081740 1 -- 172.21.15.5:0/3481726259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c1998c4960 con 0x55c199b91240 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd49ffb640 1 -- 172.21.15.5:0/3481726259 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd54011920 con 0x55c199b91240 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd49ffb640 1 -- 172.21.15.5:0/3481726259 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fdd58022430 con 0x55c199b91240 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd6b081740 10 client.20850 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:49.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd6b081740 10 client.20850 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd6b081740 20 client.20850 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd6b081740 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c199ba0e70 0x55c199ba3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd6b081740 4 client.20850 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.862+0000 7fdd6b081740 1 -- 172.21.15.5:0/3481726259 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c199af8b90 con 0x55c199ba0e70 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.863+0000 7fdd6259c640 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c199ba0e70 0x55c199ba3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.863+0000 7fdd6259c640 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c199ba0e70 0x55c199ba3260 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b6647cea27c596 server_cookie=7be1760fae9817f1 in_seq=0 out_seq=0 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.863+0000 7fdd49ffb640 10 client.20850 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd49ffb640 1 -- 172.21.15.5:0/3481726259 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c199af8b90 con 0x55c199ba0e70 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd49ffb640 10 client.20850 handle_command_reply: tid=0 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 client.20850 shutdown 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 -- 172.21.15.5:0/3481726259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 msgr2=0x55c199b8ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:49.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b8ebd0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fdd58012c90 tx=0x7fdd58004730 comp rx=0 tx=0).stop 2024-09-11T22:39:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 -- 172.21.15.5:0/3481726259 shutdown_connections 2024-09-11T22:39:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c199ba0e70 0x55c199ba3260 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b8ebd0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd49ffb640 0 client.20850 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b8e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 --2- 172.21.15.5:0/3481726259 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b8e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 -- 172.21.15.5:0/3481726259 >> 172.21.15.5:0/3481726259 conn(0x55c199b825f0 msgr2=0x55c199b882d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 -- 172.21.15.5:0/3481726259 shutdown_connections 2024-09-11T22:39:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 1 -- 172.21.15.5:0/3481726259 wait complete. 2024-09-11T22:39:49.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.864+0000 7fdd6b081740 20 client.20850 trim_cache size 0 max 16384 2024-09-11T22:39:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.881+0000 7fdd6b081740 1 Processor -- start 2024-09-11T22:39:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.881+0000 7fdd6b081740 1 -- start start 2024-09-11T22:39:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.881+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b88ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.881+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b86980 0x55c199b90020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.881+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b91240 0x55c199ba8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.881+0000 7fdd6b081740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641afc10 con 0x55c199b86980 2024-09-11T22:39:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.881+0000 7fdd6b081740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641b1750 con 0x55c199b91240 2024-09-11T22:39:49.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.881+0000 7fdd6b081740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641b02e0 con 0x55c199b86060 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b86980 0x55c199b90020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b86980 0x55c199b90020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35238/0 (socket says 172.21.15.5:35238) 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd6259c640 1 -- 172.21.15.5:0/3939634065 learned_addr learned my addr 172.21.15.5:0/3939634065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/3939634065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b88ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd6259c640 1 -- 172.21.15.5:0/3939634065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 msgr2=0x55c199b88ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd6259c640 1 --2- 172.21.15.5:0/3939634065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b88ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd6259c640 1 -- 172.21.15.5:0/3939634065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b91240 msgr2=0x55c199ba8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd6259c640 1 --2- 172.21.15.5:0/3939634065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b91240 0x55c199ba8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd6259c640 1 -- 172.21.15.5:0/3939634065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c199b85e90 con 0x55c199b86980 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd6259c640 1 --2- 172.21.15.5:0/3939634065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b86980 0x55c199b90020 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x55c199b83450 tx=0x7fdd5400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dc009f91ea8519 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd4bfff640 1 -- 172.21.15.5:0/3939634065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54015050 con 0x55c199b86980 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd4bfff640 1 -- 172.21.15.5:0/3939634065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd54005520 con 0x55c199b86980 2024-09-11T22:39:49.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.882+0000 7fdd4bfff640 1 -- 172.21.15.5:0/3939634065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54010580 con 0x55c199b86980 2024-09-11T22:39:49.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 -- 172.21.15.5:0/3939634065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b86980 msgr2=0x55c199b90020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:49.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 --2- 172.21.15.5:0/3939634065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b86980 0x55c199b90020 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x55c199b83450 tx=0x7fdd5400b5e0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 -- 172.21.15.5:0/3939634065 shutdown_connections 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 --2- 172.21.15.5:0/3939634065 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b91240 0x55c199ba8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 --2- 172.21.15.5:0/3939634065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b86980 0x55c199b90020 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 --2- 172.21.15.5:0/3939634065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b88ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 -- 172.21.15.5:0/3939634065 >> 172.21.15.5:0/3939634065 conn(0x55c199b825f0 msgr2=0x55c199b81750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 -- 172.21.15.5:0/3939634065 shutdown_connections 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 -- 172.21.15.5:0/3939634065 wait complete. 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 -- start start 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.883+0000 7fdd6b081740 1 Processor -- start 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b997e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b95b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd6b081740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b96060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd6b081740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641b1080 con 0x55c199b91240 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd6b081740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641b09b0 con 0x55c199b86980 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd6b081740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641adc10 con 0x55c199b86060 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd6b081740 10 client.0 _ll_register_callbacks cb 0x7fdd580128a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd61d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b96060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd61d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b96060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35254/0 (socket says 172.21.15.5:35254) 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd61d9b640 1 -- 172.21.15.5:0/4278715551 learned_addr learned my addr 172.21.15.5:0/4278715551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd6259c640 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b95b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd61d9b640 1 -- 172.21.15.5:0/4278715551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 msgr2=0x55c199b997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd61d9b640 1 -- 172.21.15.5:0/4278715551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 msgr2=0x55c199b95b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b95b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd61d9b640 1 -- 172.21.15.5:0/4278715551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c199b96700 con 0x55c199b91240 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd61d9b640 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b96060 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800b560 tx=0x7fdd58016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86f19d48a58f57b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.884+0000 7fdd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd49ffb640 1 -- 172.21.15.5:0/4278715551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5802e070 con 0x55c199b91240 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd49ffb640 1 -- 172.21.15.5:0/4278715551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fdd58022a20 con 0x55c199b91240 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd6b081740 1 -- 172.21.15.5:0/4278715551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd54005520 con 0x55c199b91240 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd49ffb640 1 -- 172.21.15.5:0/4278715551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdd5803b050 con 0x55c199b91240 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd6b081740 10 client.20856 fetch_fsmap learned FSMap version 54 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd6b081740 1 -- 172.21.15.5:0/4278715551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55c199c0c9a0 con 0x55c199b91240 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd49ffb640 1 -- 172.21.15.5:0/4278715551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd54005520 con 0x55c199b91240 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd49ffb640 1 -- 172.21.15.5:0/4278715551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fdd5802aa50 con 0x55c199b91240 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd6b081740 10 client.20856 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd6b081740 10 client.20856 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd6b081740 20 client.20856 populate_metadata read hostname 'smithi005' 2024-09-11T22:39:49.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd6b081740 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c199ba0e70 0x55c199c0f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:39:49.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd6b081740 4 client.20856 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a49692cd", "release": true, "await": true}] 2024-09-11T22:39:49.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.885+0000 7fdd6b081740 1 -- 172.21.15.5:0/4278715551 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a49692cd", "release": true, "await": true}) -- 0x55c199af30a0 con 0x55c199ba0e70 2024-09-11T22:39:49.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.886+0000 7fdd6259c640 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c199ba0e70 0x55c199c0f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:39:49.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.886+0000 7fdd49ffb640 10 client.20856 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:49.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.886+0000 7fdd6259c640 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c199ba0e70 0x55c199c0f010 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11df992f1605a892 server_cookie=b245dfd749c0bed9 in_seq=0 out_seq=0 2024-09-11T22:39:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd49ffb640 1 -- 172.21.15.5:0/4278715551 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c199af30a0 con 0x55c199ba0e70 2024-09-11T22:39:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd49ffb640 10 client.20856 handle_command_reply: tid=0 2024-09-11T22:39:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd6b081740 1 client.20856 shutdown 2024-09-11T22:39:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd6b081740 1 -- 172.21.15.5:0/4278715551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 msgr2=0x55c199b96060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd6b081740 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b96060 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800b560 tx=0x7fdd58016040 comp rx=0 tx=0).stop 2024-09-11T22:39:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd6b081740 1 -- 172.21.15.5:0/4278715551 shutdown_connections 2024-09-11T22:39:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd6b081740 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c199ba0e70 0x55c199c0f010 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd6b081740 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c199b91240 0x55c199b96060 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd6b081740 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c199b86980 0x55c199b95b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd49ffb640 0 client.20856 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:39:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd6b081740 1 --2- 172.21.15.5:0/4278715551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c199b86060 0x55c199b997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd6b081740 1 -- 172.21.15.5:0/4278715551 >> 172.21.15.5:0/4278715551 conn(0x55c199b825f0 msgr2=0x55c199b80f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.972+0000 7fdd6b081740 1 -- 172.21.15.5:0/4278715551 shutdown_connections 2024-09-11T22:39:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd6b081740 1 -- 172.21.15.5:0/4278715551 wait complete. 2024-09-11T22:39:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd6b081740 20 client.20856 trim_cache size 0 max 16384 2024-09-11T22:39:49.976 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:39:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 54, 2024-09-11T22:39:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:39:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 64, 2024-09-11T22:39:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-11T22:39:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "a49692cd": { 2024-09-11T22:39:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 64, 2024-09-11T22:39:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-11T22:39:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:39:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:39:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:39:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:39:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-11T22:39:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.2, 2024-09-11T22:39:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-11T22:39:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-11T22:39:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-11T22:39:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-11T22:39:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-11T22:39:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-11T22:39:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:39:49.979 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:39:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 -- 172.21.15.5:0/3266648166 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd4c0a2cf0 msgr2=0x7fdd4c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd4c0a2cf0 0x7fdd4c0a51b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0b8cd0 tx=0x7fdd5c0e7400 comp rx=0 tx=0).stop 2024-09-11T22:39:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 -- 172.21.15.5:0/3266648166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6410c170 msgr2=0x7fdd64101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:39:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6410c170 0x7fdd64101420 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c02ae70 tx=0x7fdd5c02f5b0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 -- 172.21.15.5:0/3266648166 shutdown_connections 2024-09-11T22:39:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd6410c170 0x7fdd64101420 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd64103660 0x7fdd64100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fdd4c0a2cf0 0x7fdd4c0a51b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 --2- 172.21.15.5:0/3266648166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdd64102d40 0x7fdd641009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:39:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 -- 172.21.15.5:0/3266648166 >> 172.21.15.5:0/3266648166 conn(0x7fdd640fc7a0 msgr2=0x7fdd640fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:39:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.973+0000 7fdd68ca8640 1 -- 172.21.15.5:0/3266648166 shutdown_connections 2024-09-11T22:39:49.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:39:49.974+0000 7fdd68ca8640 1 -- 172.21.15.5:0/3266648166 wait complete. 2024-09-11T22:39:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:49 smithi159 ceph-mon[35534]: pgmap v561: 97 pgs: 97 active+clean; 493 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-09-11T22:39:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:49 smithi172 ceph-mon[35461]: pgmap v561: 97 pgs: 97 active+clean; 493 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-09-11T22:39:50.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61309 15.34 MB/sec execute 128 sec latency 21876.977 ms 2024-09-11T22:39:50.877 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a49692cd', total seconds elapsed: 28.0 2024-09-11T22:39:51.648 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61740 15.44 MB/sec execute 129 sec latency 179.565 ms 2024-09-11T22:39:52.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:51 smithi005 ceph-mon[30943]: pgmap v562: 97 pgs: 97 active+clean; 493 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 728 KiB/s wr, 17 op/s 2024-09-11T22:39:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:51 smithi159 ceph-mon[35534]: pgmap v562: 97 pgs: 97 active+clean; 493 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 728 KiB/s wr, 17 op/s 2024-09-11T22:39:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:51 smithi172 ceph-mon[35461]: pgmap v562: 97 pgs: 97 active+clean; 493 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 728 KiB/s wr, 17 op/s 2024-09-11T22:39:52.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 62157 15.45 MB/sec execute 130 sec latency 201.219 ms 2024-09-11T22:39:53.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 62648 15.55 MB/sec execute 131 sec latency 179.105 ms 2024-09-11T22:39:54.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:53 smithi005 ceph-mon[30943]: pgmap v563: 97 pgs: 97 active+clean; 498 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 67 op/s 2024-09-11T22:39:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:53 smithi159 ceph-mon[35534]: pgmap v563: 97 pgs: 97 active+clean; 498 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 67 op/s 2024-09-11T22:39:54.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:53 smithi172 ceph-mon[35461]: pgmap v563: 97 pgs: 97 active+clean; 498 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 67 op/s 2024-09-11T22:39:54.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63107 15.59 MB/sec execute 132 sec latency 192.583 ms 2024-09-11T22:39:55.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63608 15.67 MB/sec execute 133 sec latency 64.034 ms 2024-09-11T22:39:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:55 smithi005 ceph-mon[30943]: pgmap v564: 97 pgs: 97 active+clean; 501 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-11T22:39:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:55 smithi159 ceph-mon[35534]: pgmap v564: 97 pgs: 97 active+clean; 501 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-11T22:39:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:55 smithi172 ceph-mon[35461]: pgmap v564: 97 pgs: 97 active+clean; 501 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-11T22:39:56.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64257 15.82 MB/sec execute 134 sec latency 79.659 ms 2024-09-11T22:39:57.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64804 15.94 MB/sec execute 135 sec latency 174.060 ms 2024-09-11T22:39:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:57 smithi005 ceph-mon[30943]: pgmap v565: 97 pgs: 97 active+clean; 501 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 875 KiB/s wr, 96 op/s 2024-09-11T22:39:58.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:57 smithi159 ceph-mon[35534]: pgmap v565: 97 pgs: 97 active+clean; 501 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 875 KiB/s wr, 96 op/s 2024-09-11T22:39:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:57 smithi172 ceph-mon[35461]: pgmap v565: 97 pgs: 97 active+clean; 501 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 875 KiB/s wr, 96 op/s 2024-09-11T22:39:58.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65129 15.97 MB/sec execute 136 sec latency 346.279 ms 2024-09-11T22:39:59.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65475 16.02 MB/sec execute 137 sec latency 266.212 ms 2024-09-11T22:40:00.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:39:59 smithi005 ceph-mon[30943]: pgmap v566: 97 pgs: 97 active+clean; 520 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-09-11T22:40:00.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:39:59 smithi159 ceph-mon[35534]: pgmap v566: 97 pgs: 97 active+clean; 520 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-09-11T22:40:00.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:39:59 smithi172 ceph-mon[35461]: pgmap v566: 97 pgs: 97 active+clean; 520 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.1 MiB/s wr, 245 op/s 2024-09-11T22:40:00.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65698 15.99 MB/sec execute 138 sec latency 504.552 ms 2024-09-11T22:40:01.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:00 smithi005 ceph-mon[30943]: overall HEALTH_OK 2024-09-11T22:40:01.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:00 smithi159 ceph-mon[35534]: overall HEALTH_OK 2024-09-11T22:40:01.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:00 smithi172 ceph-mon[35461]: overall HEALTH_OK 2024-09-11T22:40:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.280+0000 7f2b27b31640 1 -- 172.21.15.5:0/2735158810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2010e480 msgr2=0x7f2b20110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.281+0000 7f2b27b31640 1 --2- 172.21.15.5:0/2735158810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2010e480 0x7f2b20110870 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009ca0 tx=0x7f2b1802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.281+0000 7f2b27b31640 1 -- 172.21.15.5:0/2735158810 shutdown_connections 2024-09-11T22:40:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.281+0000 7f2b27b31640 1 --2- 172.21.15.5:0/2735158810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b2010e480 0x7f2b20110870 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.281+0000 7f2b27b31640 1 --2- 172.21.15.5:0/2735158810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b20101280 0x7f2b2010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.281+0000 7f2b27b31640 1 --2- 172.21.15.5:0/2735158810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b201008d0 0x7f2b20100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.281+0000 7f2b27b31640 1 -- 172.21.15.5:0/2735158810 >> 172.21.15.5:0/2735158810 conn(0x7f2b200fc6b0 msgr2=0x7f2b200fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.281+0000 7f2b27b31640 1 -- 172.21.15.5:0/2735158810 shutdown_connections 2024-09-11T22:40:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.281+0000 7f2b27b31640 1 -- 172.21.15.5:0/2735158810 wait complete. 2024-09-11T22:40:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.281+0000 7f2b27b31640 1 Processor -- start 2024-09-11T22:40:01.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.281+0000 7f2b27b31640 1 -- start start 2024-09-11T22:40:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b27b31640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b201008d0 0x7f2b2019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b27b31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b20101280 0x7f2b2019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b27b31640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2010e480 0x7f2b2019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b27b31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201140c0 con 0x7f2b20101280 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b27b31640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20113f40 con 0x7f2b2010e480 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b27b31640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b20114240 con 0x7f2b201008d0 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b250cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b20101280 0x7f2b2019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b250cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b20101280 0x7f2b2019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38068/0 (socket says 172.21.15.5:38068) 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b250cc640 1 -- 172.21.15.5:0/1803437651 learned_addr learned my addr 172.21.15.5:0/1803437651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b258cd640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b201008d0 0x7f2b2019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b260ce640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2010e480 0x7f2b2019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b250cc640 1 -- 172.21.15.5:0/1803437651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b201008d0 msgr2=0x7f2b2019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b250cc640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b201008d0 0x7f2b2019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b250cc640 1 -- 172.21.15.5:0/1803437651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2010e480 msgr2=0x7f2b2019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b250cc640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2010e480 0x7f2b2019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b250cc640 1 -- 172.21.15.5:0/1803437651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b2019f5c0 con 0x7f2b20101280 2024-09-11T22:40:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.282+0000 7f2b250cc640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b20101280 0x7f2b2019ab10 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000c890 tx=0x7f2b1000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.283+0000 7f2b16ffd640 1 -- 172.21.15.5:0/1803437651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10004220 con 0x7f2b20101280 2024-09-11T22:40:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.283+0000 7f2b16ffd640 1 -- 172.21.15.5:0/1803437651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b100043c0 con 0x7f2b20101280 2024-09-11T22:40:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.283+0000 7f2b16ffd640 1 -- 172.21.15.5:0/1803437651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100049e0 con 0x7f2b20101280 2024-09-11T22:40:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.283+0000 7f2b27b31640 1 -- 172.21.15.5:0/1803437651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2019f8b0 con 0x7f2b20101280 2024-09-11T22:40:01.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.283+0000 7f2b27b31640 1 -- 172.21.15.5:0/1803437651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b201a7140 con 0x7f2b20101280 2024-09-11T22:40:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.285+0000 7f2b16ffd640 1 -- 172.21.15.5:0/1803437651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2b100079b0 con 0x7f2b20101280 2024-09-11T22:40:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.285+0000 7f2b2a8ca740 1 Processor -- start 2024-09-11T22:40:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b2a8ca740 1 -- start start 2024-09-11T22:40:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a79ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a7a410 0x55d3f9a84770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d3f99de2f0 con 0x55d3f9a79af0 2024-09-11T22:40:01.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d3f9986270 con 0x55d3f9a7a410 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d3f99da2b0 con 0x55d3f9a84cb0 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b260ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a79ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b260ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a79ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38070/0 (socket says 172.21.15.5:38070) 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b260ce640 1 -- 172.21.15.5:0/2593611029 learned_addr learned my addr 172.21.15.5:0/2593611029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b258cd640 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a7a410 0x55d3f9a84770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b250cc640 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b16ffd640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b0c0a2da0 0x7f2b0c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b258cd640 1 -- 172.21.15.5:0/2593611029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a84cb0 msgr2=0x55d3f9a870a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b258cd640 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a870a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b258cd640 1 -- 172.21.15.5:0/2593611029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 msgr2=0x55d3f9a79ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b258cd640 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a79ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b258cd640 1 -- 172.21.15.5:0/2593611029 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f9985340 con 0x55d3f9a7a410 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b250cc640 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.286+0000 7f2b260ce640 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a79ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.287+0000 7f2b260ce640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b0c0a2da0 0x7f2b0c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.287+0000 7f2b16ffd640 1 -- 172.21.15.5:0/1803437651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f2b100cf430 con 0x7f2b20101280 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.287+0000 7f2b258cd640 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a7a410 0x55d3f9a84770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005fd0 tx=0x7f2b1c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb1d979d7b51b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.287+0000 7f2b0bfff640 1 -- 172.21.15.5:0/2593611029 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c01c480 con 0x55d3f9a7a410 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.287+0000 7f2b260ce640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b0c0a2da0 0x7f2b0c0a5260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009a60 tx=0x7f2b180095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.287+0000 7f2b0bfff640 1 -- 172.21.15.5:0/2593611029 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1c01cb00 con 0x55d3f9a7a410 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.287+0000 7f2b0bfff640 1 -- 172.21.15.5:0/2593611029 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c011b80 con 0x55d3f9a7a410 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/2593611029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a7a410 msgr2=0x55d3f9a84770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a7a410 0x55d3f9a84770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c005fd0 tx=0x7f2b1c0107a0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/2593611029 shutdown_connections 2024-09-11T22:40:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a870a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a7a410 0x55d3f9a84770 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/2593611029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a79ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/2593611029 >> 172.21.15.5:0/2593611029 conn(0x55d3f9a76060 msgr2=0x55d3f9a76440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/2593611029 shutdown_connections 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/2593611029 wait complete. 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 -- start start 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.288+0000 7f2b2a8ca740 1 Processor -- start 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a81b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a7a410 0x55d3f9a820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a82600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d3f9a87880 con 0x55d3f9a79af0 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d3f9967290 con 0x55d3f9a84cb0 2024-09-11T22:40:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d3f9a87a00 con 0x55d3f9a7a410 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b2a8ca740 10 client.0 _ll_register_callbacks cb 0x55d3f9998140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b258cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a81b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b258cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a81b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38074/0 (socket says 172.21.15.5:38074) 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b258cd640 1 -- 172.21.15.5:0/1894202629 learned_addr learned my addr 172.21.15.5:0/1894202629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b250cc640 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a7a410 0x55d3f9a820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b260ce640 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a82600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b258cd640 1 -- 172.21.15.5:0/1894202629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a7a410 msgr2=0x55d3f9a820c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b258cd640 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a7a410 0x55d3f9a820c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b258cd640 1 -- 172.21.15.5:0/1894202629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a84cb0 msgr2=0x55d3f9a82600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b258cd640 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a82600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.289+0000 7f2b258cd640 1 -- 172.21.15.5:0/1894202629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f96f1210 con 0x55d3f9a79af0 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.290+0000 7f2b258cd640 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a81b80 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00bf80 tx=0x7f2b1c00bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d998261cc17f4b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.290+0000 7f2b097fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.290+0000 7f2b097fa640 1 -- 172.21.15.5:0/1894202629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c02a060 con 0x55d3f9a79af0 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.290+0000 7f2b097fa640 1 -- 172.21.15.5:0/1894202629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f2b1c01c750 con 0x55d3f9a79af0 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.290+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/1894202629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1c01cb00 con 0x55d3f9a79af0 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.290+0000 7f2b097fa640 1 -- 172.21.15.5:0/1894202629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2b1c01c950 con 0x55d3f9a79af0 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.290+0000 7f2b2a8ca740 10 client.20868 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.290+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/1894202629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d3f97bd620 con 0x55d3f9a79af0 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.290+0000 7f2b097fa640 1 -- 172.21.15.5:0/1894202629 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1c01cb00 con 0x55d3f9a79af0 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.290+0000 7f2b097fa640 1 -- 172.21.15.5:0/1894202629 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f2b1c004750 con 0x55d3f9a79af0 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.291+0000 7f2b2a8ca740 10 client.20868 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.291+0000 7f2b2a8ca740 10 client.20868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.291+0000 7f2b2a8ca740 20 client.20868 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.291+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3f9a94ba0 0x55d3f9a96f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.291+0000 7f2b2a8ca740 4 client.20868 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.291+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/1894202629 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3f99eba20 con 0x55d3f9a94ba0 2024-09-11T22:40:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.291+0000 7f2b250cc640 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3f9a94ba0 0x55d3f9a96f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.291+0000 7f2b097fa640 10 client.20868 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.291+0000 7f2b250cc640 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3f9a94ba0 0x55d3f9a96f90 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f75d3e3437cca88c server_cookie=6c3ef39400d8dc0f in_seq=0 out_seq=0 2024-09-11T22:40:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b097fa640 1 -- 172.21.15.5:0/1894202629 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3f99eba20 con 0x55d3f9a94ba0 2024-09-11T22:40:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b097fa640 10 client.20868 handle_command_reply: tid=0 2024-09-11T22:40:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 client.20868 shutdown 2024-09-11T22:40:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/1894202629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 msgr2=0x55d3f9a81b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a81b80 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00bf80 tx=0x7f2b1c00bfb0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/1894202629 shutdown_connections 2024-09-11T22:40:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3f9a94ba0 0x55d3f9a96f90 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a82600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a7a410 0x55d3f9a820c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/1894202629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a79af0 0x55d3f9a81b80 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/1894202629 >> 172.21.15.5:0/1894202629 conn(0x55d3f9a76060 msgr2=0x55d3f9a7bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b097fa640 0 client.20868 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/1894202629 shutdown_connections 2024-09-11T22:40:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/1894202629 wait complete. 2024-09-11T22:40:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.292+0000 7f2b2a8ca740 20 client.20868 trim_cache size 0 max 16384 2024-09-11T22:40:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b2a8ca740 1 Processor -- start 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b2a8ca740 1 -- start start 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a79af0 0x55d3f9a7bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a7a410 0x55d3f9a9c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a9ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201afc60 con 0x55d3f9a7a410 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201b17a0 con 0x55d3f9a84cb0 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201b0330 con 0x55d3f9a79af0 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b250cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a7a410 0x55d3f9a9c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b250cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a7a410 0x55d3f9a9c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38080/0 (socket says 172.21.15.5:38080) 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b250cc640 1 -- 172.21.15.5:0/2742365283 learned_addr learned my addr 172.21.15.5:0/2742365283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.307+0000 7f2b258cd640 1 --2- 172.21.15.5:0/2742365283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a79af0 0x55d3f9a7bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b250cc640 1 -- 172.21.15.5:0/2742365283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a79af0 msgr2=0x55d3f9a7bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b250cc640 1 --2- 172.21.15.5:0/2742365283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a79af0 0x55d3f9a7bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b250cc640 1 -- 172.21.15.5:0/2742365283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a84cb0 msgr2=0x55d3f9a9ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b250cc640 1 --2- 172.21.15.5:0/2742365283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a9ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b250cc640 1 -- 172.21.15.5:0/2742365283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f9a75e90 con 0x55d3f9a7a410 2024-09-11T22:40:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b250cc640 1 --2- 172.21.15.5:0/2742365283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a7a410 0x55d3f9a9c530 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f2b100cd110 tx=0x7f2b10086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5ba613ee7cfabe server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/2742365283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1001d040 con 0x55d3f9a7a410 2024-09-11T22:40:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/2742365283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1009b990 con 0x55d3f9a7a410 2024-09-11T22:40:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b0b7fe640 1 -- 172.21.15.5:0/2742365283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10098000 con 0x55d3f9a7a410 2024-09-11T22:40:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/2742365283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a7a410 msgr2=0x55d3f9a9c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/2742365283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a7a410 0x55d3f9a9c530 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f2b100cd110 tx=0x7f2b10086040 comp rx=0 tx=0).stop 2024-09-11T22:40:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/2742365283 shutdown_connections 2024-09-11T22:40:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/2742365283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a9ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/2742365283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a7a410 0x55d3f9a9c530 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/2742365283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a79af0 0x55d3f9a7bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.308+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/2742365283 >> 172.21.15.5:0/2742365283 conn(0x55d3f9a76060 msgr2=0x55d3f9a7b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/2742365283 shutdown_connections 2024-09-11T22:40:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/2742365283 wait complete. 2024-09-11T22:40:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 1 -- start start 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 1 Processor -- start 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a79af0 0x55d3f9af7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a7a410 0x55d3f9af7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a8a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201b10d0 con 0x55d3f9a84cb0 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201b0a00 con 0x55d3f9a79af0 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b201adc60 con 0x55d3f9a7a410 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b2a8ca740 10 client.0 _ll_register_callbacks cb 0x7f2b18005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.309+0000 7f2b260ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a8a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b260ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a8a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38082/0 (socket says 172.21.15.5:38082) 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b260ce640 1 -- 172.21.15.5:0/4184267140 learned_addr learned my addr 172.21.15.5:0/4184267140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b250cc640 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a7a410 0x55d3f9af7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b258cd640 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a79af0 0x55d3f9af7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b260ce640 1 -- 172.21.15.5:0/4184267140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a7a410 msgr2=0x55d3f9af7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b260ce640 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a7a410 0x55d3f9af7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b260ce640 1 -- 172.21.15.5:0/4184267140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a79af0 msgr2=0x55d3f9af7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b260ce640 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a79af0 0x55d3f9af7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b260ce640 1 -- 172.21.15.5:0/4184267140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f9a712d0 con 0x55d3f9a84cb0 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b258cd640 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a79af0 0x55d3f9af7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b260ce640 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a8a280 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009ca0 tx=0x7f2b18005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51d667045eff1d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b097fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b097fa640 1 -- 172.21.15.5:0/4184267140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1802f330 con 0x55d3f9a84cb0 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b097fa640 1 -- 172.21.15.5:0/4184267140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f2b18003970 con 0x55d3f9a84cb0 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/4184267140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1009b990 con 0x55d3f9a84cb0 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b097fa640 1 -- 172.21.15.5:0/4184267140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2b18003b70 con 0x55d3f9a84cb0 2024-09-11T22:40:01.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b2a8ca740 10 client.20874 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.310+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/4184267140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55d3f9a84710 con 0x55d3f9a84cb0 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b097fa640 1 -- 172.21.15.5:0/4184267140 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b1009b990 con 0x55d3f9a84cb0 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b097fa640 1 -- 172.21.15.5:0/4184267140 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f2b180306c0 con 0x55d3f9a84cb0 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b2a8ca740 10 client.20874 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b2a8ca740 10 client.20874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b2a8ca740 20 client.20874 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3f9aff450 0x55d3f9b01870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b2a8ca740 4 client.20874 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/4184267140 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3f99e5d70 con 0x55d3f9aff450 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b258cd640 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3f9aff450 0x55d3f9b01870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b258cd640 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3f9aff450 0x55d3f9b01870 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24f84c1fbef0e4b server_cookie=2f65a36f3603bf00 in_seq=0 out_seq=0 2024-09-11T22:40:01.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.311+0000 7f2b097fa640 10 client.20874 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.625+0000 7f2b097fa640 1 -- 172.21.15.5:0/4184267140 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3f99e5d70 con 0x55d3f9aff450 2024-09-11T22:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.625+0000 7f2b097fa640 10 client.20874 handle_command_reply: tid=0 2024-09-11T22:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.625+0000 7f2b2a8ca740 1 client.20874 shutdown 2024-09-11T22:40:01.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/4184267140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a84cb0 msgr2=0x55d3f9a8a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a8a280 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009ca0 tx=0x7f2b18005fa0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/4184267140 shutdown_connections 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d3f9aff450 0x55d3f9b01870 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3f9a84cb0 0x55d3f9a8a280 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d3f9a7a410 0x55d3f9af7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b097fa640 0 client.20874 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 1 --2- 172.21.15.5:0/4184267140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d3f9a79af0 0x55d3f9af7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/4184267140 >> 172.21.15.5:0/4184267140 conn(0x55d3f9a76060 msgr2=0x55d3f9a751c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/4184267140 shutdown_connections 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 1 -- 172.21.15.5:0/4184267140 wait complete. 2024-09-11T22:40:01.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.626+0000 7f2b2a8ca740 20 client.20874 trim_cache size 0 max 16384 2024-09-11T22:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:40:01.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 -- 172.21.15.5:0/1803437651 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b0c0a2da0 msgr2=0x7f2b0c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b0c0a2da0 0x7f2b0c0a5260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009a60 tx=0x7f2b180095c0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 -- 172.21.15.5:0/1803437651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b20101280 msgr2=0x7f2b2019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b20101280 0x7f2b2019ab10 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000c890 tx=0x7f2b1000cd50 comp rx=0 tx=0).stop 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 -- 172.21.15.5:0/1803437651 shutdown_connections 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2b0c0a2da0 0x7f2b0c0a5260 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b2010e480 0x7f2b2019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b20101280 0x7f2b2019ab10 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 --2- 172.21.15.5:0/1803437651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b201008d0 0x7f2b2019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 -- 172.21.15.5:0/1803437651 >> 172.21.15.5:0/1803437651 conn(0x7f2b200fc6b0 msgr2=0x7f2b200feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:01.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 -- 172.21.15.5:0/1803437651 shutdown_connections 2024-09-11T22:40:01.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:01.627+0000 7f2b27b31640 1 -- 172.21.15.5:0/1803437651 wait complete. 2024-09-11T22:40:01.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65698 15.88 MB/sec execute 139 sec latency 1504.625 ms 2024-09-11T22:40:01.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:01 smithi159 ceph-mon[35534]: pgmap v567: 97 pgs: 97 active+clean; 520 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-09-11T22:40:01.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:40:02.015 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:01 smithi005 ceph-mon[30943]: pgmap v567: 97 pgs: 97 active+clean; 520 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-09-11T22:40:02.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:01 smithi172 ceph-mon[35461]: pgmap v567: 97 pgs: 97 active+clean; 520 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.0 MiB/s wr, 238 op/s 2024-09-11T22:40:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:02.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65698 15.76 MB/sec execute 140 sec latency 2504.732 ms 2024-09-11T22:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.785+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1311991610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee68100a80 msgr2=0x7fee68100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.785+0000 7fee6c9f6640 1 --2- 172.21.15.5:0/1311991610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee68100a80 0x7fee68100ee0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009930 tx=0x7fee5c02a420 comp rx=0 tx=0).stop 2024-09-11T22:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.785+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1311991610 shutdown_connections 2024-09-11T22:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.785+0000 7fee6c9f6640 1 --2- 172.21.15.5:0/1311991610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee68101420 0x7fee6810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.785+0000 7fee6c9f6640 1 --2- 172.21.15.5:0/1311991610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee68100a80 0x7fee68100ee0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.785+0000 7fee6c9f6640 1 --2- 172.21.15.5:0/1311991610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68106a80 0x7fee68106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.785+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1311991610 >> 172.21.15.5:0/1311991610 conn(0x7fee680fc7a0 msgr2=0x7fee680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.785+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1311991610 shutdown_connections 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.785+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1311991610 wait complete. 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee6c9f6640 1 Processor -- start 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee6c9f6640 1 -- start start 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee6c9f6640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee68100a80 0x7fee681a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee6c9f6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68101420 0x7fee681a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee6c9f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee68106a80 0x7fee6819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee6c9f6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee68110f20 con 0x7fee68106a80 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee6c9f6640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee68110da0 con 0x7fee68101420 2024-09-11T22:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee6c9f6640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee681110a0 con 0x7fee68100a80 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee66d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee68106a80 0x7fee6819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee66d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee68106a80 0x7fee6819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38100/0 (socket says 172.21.15.5:38100) 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee66d9d640 1 -- 172.21.15.5:0/1364032569 learned_addr learned my addr 172.21.15.5:0/1364032569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee65d9b640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68101420 0x7fee681a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee6659c640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee68100a80 0x7fee681a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee66d9d640 1 -- 172.21.15.5:0/1364032569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee68100a80 msgr2=0x7fee681a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee66d9d640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee68100a80 0x7fee681a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee66d9d640 1 -- 172.21.15.5:0/1364032569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68101420 msgr2=0x7fee681a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee66d9d640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68101420 0x7fee681a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.786+0000 7fee66d9d640 1 -- 172.21.15.5:0/1364032569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6819ad20 con 0x7fee68106a80 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.787+0000 7fee66d9d640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee68106a80 0x7fee6819a5f0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fee6000b970 tx=0x7fee6000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.787+0000 7fee577fe640 1 -- 172.21.15.5:0/1364032569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee60019070 con 0x7fee68106a80 2024-09-11T22:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.787+0000 7fee577fe640 1 -- 172.21.15.5:0/1364032569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee6000cb40 con 0x7fee68106a80 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.787+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1364032569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6819b010 con 0x7fee68106a80 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.787+0000 7fee577fe640 1 -- 172.21.15.5:0/1364032569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee60007650 con 0x7fee68106a80 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.787+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1364032569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee681a7300 con 0x7fee68106a80 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.789+0000 7fee577fe640 1 -- 172.21.15.5:0/1364032569 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fee6001c070 con 0x7fee68106a80 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.789+0000 7fee6edcf740 1 Processor -- start 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.789+0000 7fee6edcf740 1 -- start start 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e2f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2fdc0 0x557579e3a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e3a680 0x557579e3ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee6edcf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557579d952e0 con 0x557579e3a680 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee6edcf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557579d3cce0 con 0x557579e2fdc0 2024-09-11T22:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee6edcf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557579d912a0 con 0x557579e2f4a0 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee65d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e3a680 0x557579e3ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee65d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e3a680 0x557579e3ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38108/0 (socket says 172.21.15.5:38108) 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee65d9b640 1 -- 172.21.15.5:0/2473699483 learned_addr learned my addr 172.21.15.5:0/2473699483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee6659c640 1 --2- 172.21.15.5:0/2473699483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2fdc0 0x557579e3a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee65d9b640 1 --2- 172.21.15.5:0/2473699483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e2f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee577fe640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fee500a2d70 0x7fee500a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee65d9b640 1 -- 172.21.15.5:0/2473699483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 msgr2=0x557579e2f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee65d9b640 1 --2- 172.21.15.5:0/2473699483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e2f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee65d9b640 1 -- 172.21.15.5:0/2473699483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2fdc0 msgr2=0x557579e3a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee65d9b640 1 --2- 172.21.15.5:0/2473699483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2fdc0 0x557579e3a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee65d9b640 1 -- 172.21.15.5:0/2473699483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557579d21440 con 0x557579e3a680 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.790+0000 7fee66d9d640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fee500a2d70 0x7fee500a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.791+0000 7fee577fe640 1 -- 172.21.15.5:0/1364032569 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fee600ce9e0 con 0x7fee68106a80 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.791+0000 7fee65d9b640 1 --2- 172.21.15.5:0/2473699483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e3a680 0x557579e3ca70 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fee5c005940 tx=0x7fee5c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48c0c925aee66d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.791+0000 7fee54ff9640 1 -- 172.21.15.5:0/2473699483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c043030 con 0x557579e3a680 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.791+0000 7fee66d9d640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fee500a2d70 0x7fee500a5230 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fee600cc280 tx=0x7fee6000e040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.791+0000 7fee54ff9640 1 -- 172.21.15.5:0/2473699483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5c031750 con 0x557579e3a680 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.791+0000 7fee54ff9640 1 -- 172.21.15.5:0/2473699483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5c035040 con 0x557579e3a680 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 -- 172.21.15.5:0/2473699483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e3a680 msgr2=0x557579e3ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 --2- 172.21.15.5:0/2473699483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e3a680 0x557579e3ca70 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fee5c005940 tx=0x7fee5c0058d0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 -- 172.21.15.5:0/2473699483 shutdown_connections 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 --2- 172.21.15.5:0/2473699483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e3a680 0x557579e3ca70 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 --2- 172.21.15.5:0/2473699483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2fdc0 0x557579e3a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 --2- 172.21.15.5:0/2473699483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e2f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 -- 172.21.15.5:0/2473699483 >> 172.21.15.5:0/2473699483 conn(0x557579e2ba30 msgr2=0x557579e2be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 -- 172.21.15.5:0/2473699483 shutdown_connections 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 -- 172.21.15.5:0/2473699483 wait complete. 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 -- start start 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.792+0000 7fee6edcf740 1 Processor -- start 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e37470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e379b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e37ef0 0x557579e466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6edcf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557579e3d250 con 0x557579e2fdc0 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6edcf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557579d50050 con 0x557579e37ef0 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6edcf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x557579e3d3d0 con 0x557579e2f4a0 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6edcf740 10 client.0 _ll_register_callbacks cb 0x557579d90160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e379b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38120/0 (socket says 172.21.15.5:38120) 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6659c640 1 -- 172.21.15.5:0/3385862015 learned_addr learned my addr 172.21.15.5:0/3385862015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee65d9b640 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e37ef0 0x557579e466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee65d9b640 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e37470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6659c640 1 -- 172.21.15.5:0/3385862015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 msgr2=0x557579e37470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6659c640 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e37470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6659c640 1 -- 172.21.15.5:0/3385862015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e37ef0 msgr2=0x557579e466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6659c640 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e37ef0 0x557579e466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6659c640 1 -- 172.21.15.5:0/3385862015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557579b063c0 con 0x557579e2fdc0 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee6659c640 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e379b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fee580104d0 tx=0x7fee58010990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84a19c6b09f12c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.793+0000 7fee4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee4dffb640 1 -- 172.21.15.5:0/3385862015 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5801a020 con 0x557579e2fdc0 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee4dffb640 1 -- 172.21.15.5:0/3385862015 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fee58016690 con 0x557579e2fdc0 2024-09-11T22:40:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee6edcf740 1 -- 172.21.15.5:0/3385862015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee5c031750 con 0x557579e2fdc0 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee4dffb640 1 -- 172.21.15.5:0/3385862015 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fee58025050 con 0x557579e2fdc0 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee6edcf740 10 client.20886 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee6edcf740 1 -- 172.21.15.5:0/3385862015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557579e492b0 con 0x557579e2fdc0 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee4dffb640 1 -- 172.21.15.5:0/3385862015 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee5c031750 con 0x557579e2fdc0 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee4dffb640 1 -- 172.21.15.5:0/3385862015 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fee58014b20 con 0x557579e2fdc0 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee6edcf740 10 client.20886 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee6edcf740 10 client.20886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee6edcf740 20 client.20886 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee6edcf740 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557579e4a810 0x557579e4cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee6edcf740 4 client.20886 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.794+0000 7fee6edcf740 1 -- 172.21.15.5:0/3385862015 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557579da2850 con 0x557579e4a810 2024-09-11T22:40:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.795+0000 7fee65d9b640 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557579e4a810 0x557579e4cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.795+0000 7fee65d9b640 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557579e4a810 0x557579e4cc50 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713f020b7a44363b server_cookie=2fd707d35d2a3202 in_seq=0 out_seq=0 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.795+0000 7fee4dffb640 10 client.20886 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee4dffb640 1 -- 172.21.15.5:0/3385862015 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557579da2850 con 0x557579e4a810 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee4dffb640 10 client.20886 handle_command_reply: tid=0 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 client.20886 shutdown 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 -- 172.21.15.5:0/3385862015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 msgr2=0x557579e379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e379b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fee580104d0 tx=0x7fee58010990 comp rx=0 tx=0).stop 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 -- 172.21.15.5:0/3385862015 shutdown_connections 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557579e4a810 0x557579e4cc50 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e37ef0 0x557579e466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e379b0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 --2- 172.21.15.5:0/3385862015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e37470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee4dffb640 0 client.20886 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:02.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 -- 172.21.15.5:0/3385862015 >> 172.21.15.5:0/3385862015 conn(0x557579e2ba30 msgr2=0x557579e31790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:02.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 -- 172.21.15.5:0/3385862015 shutdown_connections 2024-09-11T22:40:02.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 1 -- 172.21.15.5:0/3385862015 wait complete. 2024-09-11T22:40:02.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.796+0000 7fee6edcf740 20 client.20886 trim_cache size 0 max 16384 2024-09-11T22:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.813+0000 7fee6edcf740 1 Processor -- start 2024-09-11T22:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.813+0000 7fee6edcf740 1 -- start start 2024-09-11T22:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2f4a0 0x557579e464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e46a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e37ef0 0x557579e52520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6edcf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee681afca0 con 0x557579e2fdc0 2024-09-11T22:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6edcf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee681b17e0 con 0x557579e2f4a0 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6edcf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee681b0370 con 0x557579e37ef0 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e46a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e46a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38124/0 (socket says 172.21.15.5:38124) 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6659c640 1 -- 172.21.15.5:0/3028639673 learned_addr learned my addr 172.21.15.5:0/3028639673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee65d9b640 1 --2- 172.21.15.5:0/3028639673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2f4a0 0x557579e464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6659c640 1 -- 172.21.15.5:0/3028639673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e37ef0 msgr2=0x557579e52520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6659c640 1 --2- 172.21.15.5:0/3028639673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e37ef0 0x557579e52520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6659c640 1 -- 172.21.15.5:0/3028639673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2f4a0 msgr2=0x557579e464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6659c640 1 --2- 172.21.15.5:0/3028639673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2f4a0 0x557579e464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6659c640 1 -- 172.21.15.5:0/3028639673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557579e359c0 con 0x557579e2fdc0 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee65d9b640 1 --2- 172.21.15.5:0/3028639673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2f4a0 0x557579e464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee6659c640 1 --2- 172.21.15.5:0/3028639673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e46a20 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fee58000c00 tx=0x7fee58010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af718b39d114b49 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee4ffff640 1 -- 172.21.15.5:0/3028639673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5802d020 con 0x557579e2fdc0 2024-09-11T22:40:02.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee4ffff640 1 -- 172.21.15.5:0/3028639673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee58012e60 con 0x557579e2fdc0 2024-09-11T22:40:02.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.814+0000 7fee4ffff640 1 -- 172.21.15.5:0/3028639673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5801a050 con 0x557579e2fdc0 2024-09-11T22:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 -- 172.21.15.5:0/3028639673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 msgr2=0x557579e46a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 --2- 172.21.15.5:0/3028639673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e46a20 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fee58000c00 tx=0x7fee58010e90 comp rx=0 tx=0).stop 2024-09-11T22:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 -- 172.21.15.5:0/3028639673 shutdown_connections 2024-09-11T22:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 --2- 172.21.15.5:0/3028639673 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e37ef0 0x557579e52520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 --2- 172.21.15.5:0/3028639673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e46a20 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 --2- 172.21.15.5:0/3028639673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e2f4a0 0x557579e464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 -- 172.21.15.5:0/3028639673 >> 172.21.15.5:0/3028639673 conn(0x557579e2ba30 msgr2=0x557579e31790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 -- 172.21.15.5:0/3028639673 shutdown_connections 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 -- 172.21.15.5:0/3028639673 wait complete. 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 -- start start 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.815+0000 7fee6edcf740 1 Processor -- start 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e42f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6edcf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e37ef0 0x557579e3df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6edcf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee681b1110 con 0x557579e2fdc0 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6edcf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee681b0a40 con 0x557579e37ef0 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6edcf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fee681adca0 con 0x557579e2f4a0 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6edcf740 10 client.0 _ll_register_callbacks cb 0x7fee5800b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e42f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e42f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38138/0 (socket says 172.21.15.5:38138) 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6659c640 1 -- 172.21.15.5:0/1279353169 learned_addr learned my addr 172.21.15.5:0/1279353169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee65d9b640 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e37ef0 0x557579e3df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6659c640 1 -- 172.21.15.5:0/1279353169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 msgr2=0x557579e429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6659c640 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6659c640 1 -- 172.21.15.5:0/1279353169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e37ef0 msgr2=0x557579e3df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6659c640 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e37ef0 0x557579e3df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6659c640 1 -- 172.21.15.5:0/1279353169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557579e3e790 con 0x557579e2fdc0 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6659c640 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e42f20 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fee58006b50 tx=0x7fee58012790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297e7a1d370cd13e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee4dffb640 1 -- 172.21.15.5:0/1279353169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5801a070 con 0x557579e2fdc0 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee4dffb640 1 -- 172.21.15.5:0/1279353169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7fee58012b70 con 0x557579e2fdc0 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.816+0000 7fee6edcf740 1 -- 172.21.15.5:0/1279353169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee58012e60 con 0x557579e2fdc0 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee4dffb640 1 -- 172.21.15.5:0/1279353169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fee58040050 con 0x557579e2fdc0 2024-09-11T22:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee6edcf740 10 client.20892 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee6edcf740 1 -- 172.21.15.5:0/1279353169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x557579eb50d0 con 0x557579e2fdc0 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee4dffb640 1 -- 172.21.15.5:0/1279353169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee58012e60 con 0x557579e2fdc0 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee4dffb640 1 -- 172.21.15.5:0/1279353169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7fee58014a10 con 0x557579e2fdc0 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee6edcf740 10 client.20892 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee6edcf740 10 client.20892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee6edcf740 20 client.20892 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee6edcf740 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557579eb5730 0x557579eb7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee6edcf740 4 client.20892 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.817+0000 7fee6edcf740 1 -- 172.21.15.5:0/1279353169 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557579d9cd60 con 0x557579eb5730 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee65d9b640 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557579eb5730 0x557579eb7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee65d9b640 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557579eb5730 0x557579eb7b50 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=215de869b313d085 server_cookie=b395d50215525dfa in_seq=0 out_seq=0 2024-09-11T22:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee4dffb640 10 client.20892 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee4dffb640 1 -- 172.21.15.5:0/1279353169 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557579d9cd60 con 0x557579eb5730 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee4dffb640 10 client.20892 handle_command_reply: tid=0 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee6edcf740 1 client.20892 shutdown 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee6edcf740 1 -- 172.21.15.5:0/1279353169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 msgr2=0x557579e42f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee6edcf740 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e42f20 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fee58006b50 tx=0x7fee58012790 comp rx=0 tx=0).stop 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee6edcf740 1 -- 172.21.15.5:0/1279353169 shutdown_connections 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee6edcf740 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x557579eb5730 0x557579eb7b50 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee6edcf740 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557579e37ef0 0x557579e3df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee6edcf740 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557579e2fdc0 0x557579e42f20 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee4dffb640 0 client.20892 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee6edcf740 1 --2- 172.21.15.5:0/1279353169 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557579e2f4a0 0x557579e429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee6edcf740 1 -- 172.21.15.5:0/1279353169 >> 172.21.15.5:0/1279353169 conn(0x557579e2ba30 msgr2=0x557579e35c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.818+0000 7fee6edcf740 1 -- 172.21.15.5:0/1279353169 shutdown_connections 2024-09-11T22:40:02.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6edcf740 1 -- 172.21.15.5:0/1279353169 wait complete. 2024-09-11T22:40:02.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6edcf740 20 client.20892 trim_cache size 0 max 16384 2024-09-11T22:40:02.825 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:40:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:40:02.825 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:40:02.825 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1364032569 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fee500a2d70 msgr2=0x7fee500a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fee500a2d70 0x7fee500a5230 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fee600cc280 tx=0x7fee6000e040 comp rx=0 tx=0).stop 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1364032569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee68106a80 msgr2=0x7fee6819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee68106a80 0x7fee6819a5f0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fee6000b970 tx=0x7fee6000be30 comp rx=0 tx=0).stop 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1364032569 shutdown_connections 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee68106a80 0x7fee6819a5f0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fee500a2d70 0x7fee500a5230 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee68101420 0x7fee681a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 --2- 172.21.15.5:0/1364032569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fee68100a80 0x7fee681a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1364032569 >> 172.21.15.5:0/1364032569 conn(0x7fee680fc7a0 msgr2=0x7fee6806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1364032569 shutdown_connections 2024-09-11T22:40:02.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:02.819+0000 7fee6c9f6640 1 -- 172.21.15.5:0/1364032569 wait complete. 2024-09-11T22:40:03.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65698 15.65 MB/sec execute 141 sec latency 3504.765 ms 2024-09-11T22:40:03.678 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:03 smithi005 ceph-mon[30943]: pgmap v568: 97 pgs: 97 active+clean; 502 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.4 MiB/s wr, 373 op/s 2024-09-11T22:40:03.679 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:40:03.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:40:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:04.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:03 smithi159 ceph-mon[35534]: pgmap v568: 97 pgs: 97 active+clean; 502 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.4 MiB/s wr, 373 op/s 2024-09-11T22:40:04.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:03 smithi172 ceph-mon[35461]: pgmap v568: 97 pgs: 97 active+clean; 502 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.4 MiB/s wr, 373 op/s 2024-09-11T22:40:04.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65950 15.61 MB/sec execute 142 sec latency 3931.909 ms 2024-09-11T22:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.648+0000 7f1237d50640 1 -- 172.21.15.5:0/3777678121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230102570 msgr2=0x7f12301029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.649+0000 7f1237d50640 1 --2- 172.21.15.5:0/3777678121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230102570 0x7f12301029b0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f1220009900 tx=0x7f122002a480 comp rx=0 tx=0).stop 2024-09-11T22:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.649+0000 7f1237d50640 1 -- 172.21.15.5:0/3777678121 shutdown_connections 2024-09-11T22:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.649+0000 7f1237d50640 1 --2- 172.21.15.5:0/3777678121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1230102ef0 0x7f123010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.649+0000 7f1237d50640 1 --2- 172.21.15.5:0/3777678121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230102570 0x7f12301029b0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.649+0000 7f1237d50640 1 --2- 172.21.15.5:0/3777678121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1230100e20 0x7f1230101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.650+0000 7f1237d50640 1 -- 172.21.15.5:0/3777678121 >> 172.21.15.5:0/3777678121 conn(0x7f12300fc7d0 msgr2=0x7f12300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.650+0000 7f1237d50640 1 -- 172.21.15.5:0/3777678121 shutdown_connections 2024-09-11T22:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.650+0000 7f1237d50640 1 -- 172.21.15.5:0/3777678121 wait complete. 2024-09-11T22:40:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.652+0000 7f1237d50640 1 Processor -- start 2024-09-11T22:40:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.652+0000 7f1237d50640 1 -- start start 2024-09-11T22:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.652+0000 7f1237d50640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1230100e20 0x7f1230197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f1237d50640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1230102570 0x7f1230197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f1237d50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230102ef0 0x7f123019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f1237d50640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1230110f50 con 0x7f1230102ef0 2024-09-11T22:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f1237d50640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1230110dd0 con 0x7f1230100e20 2024-09-11T22:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f1237d50640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f12301110d0 con 0x7f1230102570 2024-09-11T22:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f12352eb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1230102570 0x7f1230197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f12352eb640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1230102570 0x7f1230197970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60872/0 (socket says 172.21.15.5:60872) 2024-09-11T22:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f12352eb640 1 -- 172.21.15.5:0/486395113 learned_addr learned my addr 172.21.15.5:0/486395113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f12352eb640 1 -- 172.21.15.5:0/486395113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1230100e20 msgr2=0x7f1230197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f12352eb640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1230100e20 0x7f1230197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f12352eb640 1 -- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230102ef0 msgr2=0x7f123019bd00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-11T22:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f12362ed640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230102ef0 0x7f123019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f12352eb640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230102ef0 0x7f123019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f12352eb640 1 -- 172.21.15.5:0/486395113 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123019c3e0 con 0x7f1230102570 2024-09-11T22:40:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.653+0000 7f12362ed640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230102ef0 0x7f123019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:40:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.654+0000 7f12352eb640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1230102570 0x7f1230197970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1220006010 tx=0x7f122003cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.654+0000 7f1226ffd640 1 -- 172.21.15.5:0/486395113 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220035030 con 0x7f1230102570 2024-09-11T22:40:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.654+0000 7f1226ffd640 1 -- 172.21.15.5:0/486395113 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1220031540 con 0x7f1230102570 2024-09-11T22:40:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.654+0000 7f1226ffd640 1 -- 172.21.15.5:0/486395113 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220044640 con 0x7f1230102570 2024-09-11T22:40:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.654+0000 7f1237d50640 1 -- 172.21.15.5:0/486395113 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123019c670 con 0x7f1230102570 2024-09-11T22:40:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.654+0000 7f1237d50640 1 -- 172.21.15.5:0/486395113 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123019cb00 con 0x7f1230102570 2024-09-11T22:40:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.656+0000 7f1237d50640 1 -- 172.21.15.5:0/486395113 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1230101200 con 0x7f1230102570 2024-09-11T22:40:04.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.657+0000 7f1226ffd640 1 -- 172.21.15.5:0/486395113 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1220031a80 con 0x7f1230102570 2024-09-11T22:40:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.658+0000 7f1226ffd640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f121c0a2d70 0x7f121c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.659+0000 7f1235aec640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f121c0a2d70 0x7f121c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:04.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.659+0000 7f1226ffd640 1 -- 172.21.15.5:0/486395113 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f12200f4050 con 0x7f1230102570 2024-09-11T22:40:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.659+0000 7f1235aec640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f121c0a2d70 0x7f121c0a5230 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f122c0058c0 tx=0x7f122c005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:04.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:04.662+0000 7f1226ffd640 1 -- 172.21.15.5:0/486395113 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12200c0ba0 con 0x7f1230102570 2024-09-11T22:40:05.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.019+0000 7f1237d50640 1 -- 172.21.15.5:0/486395113 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1230108b00 con 0x7f1230102570 2024-09-11T22:40:05.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.020+0000 7f1226ffd640 1 -- 172.21.15.5:0/486395113 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11967 (secure 0 0 0) 0x7f12200c04a0 con 0x7f1230102570 2024-09-11T22:40:05.021 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:40:05.021 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"btime":"2024-09-11T22:38:14:548820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:38:13.865471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24436,24452,24454,24473,24500]},"id":1}]} 2024-09-11T22:40:05.022 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2024-09-11T22:40:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.023+0000 7f1237d50640 1 -- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f121c0a2d70 msgr2=0x7f121c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.023+0000 7f1237d50640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f121c0a2d70 0x7f121c0a5230 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f122c0058c0 tx=0x7f122c005850 comp rx=0 tx=0).stop 2024-09-11T22:40:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.023+0000 7f1237d50640 1 -- 172.21.15.5:0/486395113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1230102570 msgr2=0x7f1230197970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:05.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.023+0000 7f1237d50640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1230102570 0x7f1230197970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1220006010 tx=0x7f122003cd10 comp rx=0 tx=0).stop 2024-09-11T22:40:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.023+0000 7f1237d50640 1 -- 172.21.15.5:0/486395113 shutdown_connections 2024-09-11T22:40:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.023+0000 7f1237d50640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230102ef0 0x7f123019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.023+0000 7f1237d50640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1230102570 0x7f1230197970 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.023+0000 7f1237d50640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f121c0a2d70 0x7f121c0a5230 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.023+0000 7f1237d50640 1 --2- 172.21.15.5:0/486395113 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1230100e20 0x7f1230197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.023+0000 7f1237d50640 1 -- 172.21.15.5:0/486395113 >> 172.21.15.5:0/486395113 conn(0x7f12300fc7d0 msgr2=0x7f12300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.024+0000 7f1237d50640 1 -- 172.21.15.5:0/486395113 shutdown_connections 2024-09-11T22:40:05.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:05.024+0000 7f1237d50640 1 -- 172.21.15.5:0/486395113 wait complete. 2024-09-11T22:40:05.649 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66186 15.57 MB/sec execute 143 sec latency 113.815 ms 2024-09-11T22:40:05.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:40:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:05 smithi005 ceph-mon[30943]: pgmap v569: 97 pgs: 97 active+clean; 500 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.5 MiB/s wr, 470 op/s 2024-09-11T22:40:06.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:05 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/486395113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:40:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:05 smithi159 ceph-mon[35534]: pgmap v569: 97 pgs: 97 active+clean; 500 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.5 MiB/s wr, 470 op/s 2024-09-11T22:40:06.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:05 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/486395113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:40:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:05 smithi172 ceph-mon[35461]: pgmap v569: 97 pgs: 97 active+clean; 500 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 5.5 MiB/s wr, 470 op/s 2024-09-11T22:40:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:05 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/486395113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:40:06.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66477 15.54 MB/sec execute 144 sec latency 71.674 ms 2024-09-11T22:40:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.612+0000 7f2b04c34640 1 -- 172.21.15.5:0/1677130009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b00106f50 msgr2=0x7f2b0010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.612+0000 7f2b04c34640 1 --2- 172.21.15.5:0/1677130009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b00106f50 0x7f2b0010eaf0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009cd0 tx=0x7f2af802a8e0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.612+0000 7f2b04c34640 1 -- 172.21.15.5:0/1677130009 shutdown_connections 2024-09-11T22:40:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.612+0000 7f2b04c34640 1 --2- 172.21.15.5:0/1677130009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b00106f50 0x7f2b0010eaf0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.612+0000 7f2b04c34640 1 --2- 172.21.15.5:0/1677130009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00100ca0 0x7f2b00106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.612+0000 7f2b04c34640 1 --2- 172.21.15.5:0/1677130009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b001002f0 0x7f2b001006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.612+0000 7f2b04c34640 1 -- 172.21.15.5:0/1677130009 >> 172.21.15.5:0/1677130009 conn(0x7f2b00077210 msgr2=0x7f2b000fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.612+0000 7f2b04c34640 1 -- 172.21.15.5:0/1677130009 shutdown_connections 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.612+0000 7f2b04c34640 1 -- 172.21.15.5:0/1677130009 wait complete. 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2b04c34640 1 Processor -- start 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2b04c34640 1 -- start start 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2b04c34640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b001002f0 0x7f2b001984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2b04c34640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00100ca0 0x7f2b001989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2b04c34640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b00106f50 0x7f2b0019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2b04c34640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00111fa0 con 0x7f2b00106f50 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2b04c34640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00111e20 con 0x7f2b001002f0 2024-09-11T22:40:06.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2b04c34640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00112120 con 0x7f2b00100ca0 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2afed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b00106f50 0x7f2b0019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2afe59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b001002f0 0x7f2b001984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2afed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b00106f50 0x7f2b0019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38176/0 (socket says 172.21.15.5:38176) 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2afed9d640 1 -- 172.21.15.5:0/1029340765 learned_addr learned my addr 172.21.15.5:0/1029340765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.613+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00100ca0 0x7f2b001989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2afe59c640 1 -- 172.21.15.5:0/1029340765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00100ca0 msgr2=0x7f2b001989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2afe59c640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00100ca0 0x7f2b001989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2afe59c640 1 -- 172.21.15.5:0/1029340765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b00106f50 msgr2=0x7f2b0019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2afe59c640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b00106f50 0x7f2b0019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2afe59c640 1 -- 172.21.15.5:0/1029340765 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0019d450 con 0x7f2b001002f0 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2afed9d640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b00106f50 0x7f2b0019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2afe59c640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b001002f0 0x7f2b001984a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2af000e9f0 tx=0x7f2af000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2af77fe640 1 -- 172.21.15.5:0/1029340765 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af000ccc0 con 0x7f2b001002f0 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2b04c34640 1 -- 172.21.15.5:0/1029340765 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0019d740 con 0x7f2b001002f0 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2af77fe640 1 -- 172.21.15.5:0/1029340765 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af0010910 con 0x7f2b001002f0 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2af77fe640 1 -- 172.21.15.5:0/1029340765 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0018960 con 0x7f2b001002f0 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.614+0000 7f2b04c34640 1 -- 172.21.15.5:0/1029340765 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b001a5020 con 0x7f2b001002f0 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2b0700d740 1 Processor -- start 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2b0700d740 1 -- start start 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2af77fe640 1 -- 172.21.15.5:0/1029340765 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2af0010ac0 con 0x7f2b001002f0 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b763b40 0x558e6b762090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b762a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b76d370 0x558e6b76f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2b0700d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e6b6c82f0 con 0x558e6b76d370 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2b0700d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558e6b66fcf0 con 0x558e6b7625d0 2024-09-11T22:40:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2b0700d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558e6b6c42b0 con 0x558e6b763b40 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2afdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b76d370 0x558e6b76f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2afed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b763b40 0x558e6b762090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2afed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b763b40 0x558e6b762090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60896/0 (socket says 172.21.15.5:60896) 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2afdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b76d370 0x558e6b76f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38192/0 (socket says 172.21.15.5:38192) 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2afed9d640 1 -- 172.21.15.5:0/1773770078 learned_addr learned my addr 172.21.15.5:0/1773770078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.617+0000 7f2afe59c640 1 --2- 172.21.15.5:0/1773770078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b762a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2afdd9b640 1 -- 172.21.15.5:0/1773770078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b763b40 msgr2=0x558e6b762090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/1773770078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b763b40 0x558e6b762090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2afdd9b640 1 -- 172.21.15.5:0/1773770078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 msgr2=0x558e6b762a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/1773770078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b762a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2afdd9b640 1 -- 172.21.15.5:0/1773770078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6b6543a0 con 0x558e6b76d370 2024-09-11T22:40:06.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2afed9d640 1 --2- 172.21.15.5:0/1773770078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b763b40 0x558e6b762090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2af77fe640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2aec0a2cf0 0x7f2aec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/1773770078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b76d370 0x558e6b76f760 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2b0006a100 tx=0x7f2ae8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c05d98a1821e5c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:06.713 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:40:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2afed9d640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2aec0a2cf0 0x7f2aec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2af77fe640 1 -- 172.21.15.5:0/1029340765 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f2af00cf8b0 con 0x7f2b001002f0 2024-09-11T22:40:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2af4ff9640 1 -- 172.21.15.5:0/1773770078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8004050 con 0x558e6b76d370 2024-09-11T22:40:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2af4ff9640 1 -- 172.21.15.5:0/1773770078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae8005500 con 0x558e6b76d370 2024-09-11T22:40:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.618+0000 7f2af4ff9640 1 -- 172.21.15.5:0/1773770078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8004d40 con 0x558e6b76d370 2024-09-11T22:40:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.619+0000 7f2b0700d740 1 -- 172.21.15.5:0/1773770078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b76d370 msgr2=0x558e6b76f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.619+0000 7f2b0700d740 1 --2- 172.21.15.5:0/1773770078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b76d370 0x558e6b76f760 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f2b0006a100 tx=0x7f2ae8011810 comp rx=0 tx=0).stop 2024-09-11T22:40:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.619+0000 7f2afed9d640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2aec0a2cf0 0x7f2aec0a51b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2af8031860 tx=0x7f2af8005c50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.619+0000 7f2b0700d740 1 -- 172.21.15.5:0/1773770078 shutdown_connections 2024-09-11T22:40:06.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.619+0000 7f2b0700d740 1 --2- 172.21.15.5:0/1773770078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b76d370 0x558e6b76f760 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.619+0000 7f2b0700d740 1 --2- 172.21.15.5:0/1773770078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b762a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.619+0000 7f2b0700d740 1 --2- 172.21.15.5:0/1773770078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b763b40 0x558e6b762090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.619+0000 7f2b0700d740 1 -- 172.21.15.5:0/1773770078 >> 172.21.15.5:0/1773770078 conn(0x558e6b72b310 msgr2=0x558e6b75ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.619+0000 7f2b0700d740 1 -- 172.21.15.5:0/1773770078 shutdown_connections 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.619+0000 7f2b0700d740 1 -- 172.21.15.5:0/1773770078 wait complete. 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2b0700d740 1 -- start start 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2b0700d740 1 Processor -- start 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b771740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b771c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b775df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2b0700d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e6b770070 con 0x558e6b763b40 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2b0700d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558e6b683060 con 0x558e6b7625d0 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2b0700d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558e6b7701f0 con 0x558e6b76d370 2024-09-11T22:40:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2b0700d740 10 client.0 _ll_register_callbacks cb 0x558e6b3d68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2afdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b771c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2afed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b775df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2afdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b771c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38200/0 (socket says 172.21.15.5:38200) 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2afdd9b640 1 -- 172.21.15.5:0/2265076306 learned_addr learned my addr 172.21.15.5:0/2265076306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2afe59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b771740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2afdd9b640 1 -- 172.21.15.5:0/2265076306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 msgr2=0x558e6b775df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b775df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.620+0000 7f2afdd9b640 1 -- 172.21.15.5:0/2265076306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 msgr2=0x558e6b771740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b771740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2afdd9b640 1 -- 172.21.15.5:0/2265076306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e6b4393c0 con 0x558e6b763b40 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b771c80 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8004520 tx=0x7f2ae8009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe7f24fc20e6d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2ae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:06.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/2265076306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8029070 con 0x558e6b763b40 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/2265076306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f2ae8003a00 con 0x558e6b763b40 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2b0700d740 1 -- 172.21.15.5:0/2265076306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae8005500 con 0x558e6b763b40 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/2265076306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2ae8035050 con 0x558e6b763b40 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2b0700d740 10 client.20907 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2b0700d740 1 -- 172.21.15.5:0/2265076306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558e6b77fef0 con 0x558e6b763b40 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/2265076306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae8005500 con 0x558e6b763b40 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.621+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/2265076306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f2ae80051d0 con 0x558e6b763b40 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.622+0000 7f2b0700d740 10 client.20907 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.622+0000 7f2b0700d740 10 client.20907 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.622+0000 7f2b0700d740 20 client.20907 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.622+0000 7f2b0700d740 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558e6b781240 0x558e6b783630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.622+0000 7f2b0700d740 4 client.20907 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:06.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.622+0000 7f2b0700d740 1 -- 172.21.15.5:0/2265076306 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e6b6d5860 con 0x558e6b781240 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.622+0000 7f2afe59c640 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558e6b781240 0x558e6b783630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.622+0000 7f2afe59c640 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558e6b781240 0x558e6b783630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=77cb6530b595164a server_cookie=39d67a28f883097a in_seq=0 out_seq=0 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.622+0000 7f2ae5ffb640 10 client.20907 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/2265076306 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e6b6d5860 con 0x558e6b781240 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2ae5ffb640 10 client.20907 handle_command_reply: tid=0 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 client.20907 shutdown 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 -- 172.21.15.5:0/2265076306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 msgr2=0x558e6b771c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b771c80 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8004520 tx=0x7f2ae8009fa0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 -- 172.21.15.5:0/2265076306 shutdown_connections 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558e6b781240 0x558e6b783630 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b775df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b771c80 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2ae5ffb640 0 client.20907 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 --2- 172.21.15.5:0/2265076306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b771740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 -- 172.21.15.5:0/2265076306 >> 172.21.15.5:0/2265076306 conn(0x558e6b72b310 msgr2=0x558e6b764960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 -- 172.21.15.5:0/2265076306 shutdown_connections 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 1 -- 172.21.15.5:0/2265076306 wait complete. 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.623+0000 7f2b0700d740 20 client.20907 trim_cache size 0 max 16384 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2b0700d740 1 Processor -- start 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2b0700d740 1 -- start start 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b775de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b75ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b788da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2b0700d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b001aba60 con 0x558e6b763b40 2024-09-11T22:40:06.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2b0700d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b001ad5a0 con 0x558e6b7625d0 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2b0700d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b001ac130 con 0x558e6b76d370 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2afdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b75ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2afdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b75ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38212/0 (socket says 172.21.15.5:38212) 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2afdd9b640 1 -- 172.21.15.5:0/3559209005 learned_addr learned my addr 172.21.15.5:0/3559209005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.638+0000 7f2afe59c640 1 --2- 172.21.15.5:0/3559209005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b775de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.639+0000 7f2afed9d640 1 --2- 172.21.15.5:0/3559209005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b788da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.639+0000 7f2afdd9b640 1 -- 172.21.15.5:0/3559209005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 msgr2=0x558e6b788da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.639+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/3559209005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b788da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.639+0000 7f2afdd9b640 1 -- 172.21.15.5:0/3559209005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 msgr2=0x558e6b775de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.639+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/3559209005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b775de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.639+0000 7f2afdd9b640 1 -- 172.21.15.5:0/3559209005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6b771ff0 con 0x558e6b763b40 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.639+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/3559209005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b75ee10 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8011850 tx=0x7f2ae80059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb638238169327f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:06.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.639+0000 7f2ae7fff640 1 -- 172.21.15.5:0/3559209005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8029070 con 0x558e6b763b40 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.639+0000 7f2ae7fff640 1 -- 172.21.15.5:0/3559209005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae8005520 con 0x558e6b763b40 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.639+0000 7f2ae7fff640 1 -- 172.21.15.5:0/3559209005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8003af0 con 0x558e6b763b40 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.640+0000 7f2b0700d740 1 -- 172.21.15.5:0/3559209005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 msgr2=0x558e6b75ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.640+0000 7f2b0700d740 1 --2- 172.21.15.5:0/3559209005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b75ee10 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8011850 tx=0x7f2ae80059b0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.640+0000 7f2b0700d740 1 -- 172.21.15.5:0/3559209005 shutdown_connections 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.640+0000 7f2b0700d740 1 --2- 172.21.15.5:0/3559209005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b788da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.640+0000 7f2b0700d740 1 --2- 172.21.15.5:0/3559209005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b763b40 0x558e6b75ee10 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.640+0000 7f2b0700d740 1 --2- 172.21.15.5:0/3559209005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b7625d0 0x558e6b775de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.640+0000 7f2b0700d740 1 -- 172.21.15.5:0/3559209005 >> 172.21.15.5:0/3559209005 conn(0x558e6b72b310 msgr2=0x558e6b75e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.640+0000 7f2b0700d740 1 -- 172.21.15.5:0/3559209005 shutdown_connections 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.641+0000 7f2b0700d740 1 -- 172.21.15.5:0/3559209005 wait complete. 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.641+0000 7f2b0700d740 1 -- start start 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.641+0000 7f2b0700d740 1 Processor -- start 2024-09-11T22:40:06.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.641+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b7625d0 0x558e6b77d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.641+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b763b40 0x558e6b77d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.641+0000 7f2b0700d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b76ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.641+0000 7f2b0700d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b001aced0 con 0x558e6b7625d0 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.641+0000 7f2b0700d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b001ac800 con 0x558e6b763b40 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.641+0000 7f2b0700d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2b001a9a60 con 0x558e6b76d370 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.641+0000 7f2b0700d740 10 client.0 _ll_register_callbacks cb 0x7f2af00d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b7625d0 0x558e6b77d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b76ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b7625d0 0x558e6b77d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38220/0 (socket says 172.21.15.5:38220) 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afe59c640 1 -- 172.21.15.5:0/1392447294 learned_addr learned my addr 172.21.15.5:0/1392447294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b763b40 0x558e6b77d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afe59c640 1 -- 172.21.15.5:0/1392447294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 msgr2=0x558e6b76ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:06.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afe59c640 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b76ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afe59c640 1 -- 172.21.15.5:0/1392447294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b763b40 msgr2=0x558e6b77d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afe59c640 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b763b40 0x558e6b77d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afe59c640 1 -- 172.21.15.5:0/1392447294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e6b76b580 con 0x558e6b7625d0 2024-09-11T22:40:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afed9d640 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b76ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b763b40 0x558e6b77d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:40:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2afe59c640 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b7625d0 0x558e6b77d320 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f2af0098780 tx=0x7f2af00d1d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b8ad1075f4ef44 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2ae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/1392447294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af009bdb0 con 0x558e6b7625d0 2024-09-11T22:40:06.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/1392447294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f2af00cd1b0 con 0x558e6b7625d0 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.642+0000 7f2b0700d740 1 -- 172.21.15.5:0/1392447294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae8005520 con 0x558e6b7625d0 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/1392447294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2af00cd3b0 con 0x558e6b7625d0 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2b0700d740 10 client.20913 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2b0700d740 1 -- 172.21.15.5:0/1392447294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x558e6b7eb8f0 con 0x558e6b7625d0 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/1392447294 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae8005520 con 0x558e6b7625d0 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/1392447294 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f2af0086040 con 0x558e6b7625d0 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2b0700d740 10 client.20913 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2b0700d740 10 client.20913 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2b0700d740 20 client.20913 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2b0700d740 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558e6b7ec040 0x558e6b7ee460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2b0700d740 4 client.20913 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.643+0000 7f2b0700d740 1 -- 172.21.15.5:0/1392447294 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e6b6cfd70 con 0x558e6b7ec040 2024-09-11T22:40:06.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.644+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558e6b7ec040 0x558e6b7ee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.644+0000 7f2ae5ffb640 10 client.20913 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.644+0000 7f2afdd9b640 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558e6b7ec040 0x558e6b7ee460 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d42a998f20ea07b server_cookie=609b9ff38fd7823a in_seq=0 out_seq=0 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.645+0000 7f2ae5ffb640 1 -- 172.21.15.5:0/1392447294 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e6b6cfd70 con 0x558e6b7ec040 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.645+0000 7f2ae5ffb640 10 client.20913 handle_command_reply: tid=0 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 client.20913 shutdown 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 -- 172.21.15.5:0/1392447294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b7625d0 msgr2=0x558e6b77d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b7625d0 0x558e6b77d320 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f2af0098780 tx=0x7f2af00d1d00 comp rx=0 tx=0).stop 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 -- 172.21.15.5:0/1392447294 shutdown_connections 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558e6b7ec040 0x558e6b7ee460 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e6b76d370 0x558e6b76ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e6b763b40 0x558e6b77d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 --2- 172.21.15.5:0/1392447294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e6b7625d0 0x558e6b77d320 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 -- 172.21.15.5:0/1392447294 >> 172.21.15.5:0/1392447294 conn(0x558e6b72b310 msgr2=0x558e6b75d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2ae5ffb640 0 client.20913 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 -- 172.21.15.5:0/1392447294 shutdown_connections 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 1 -- 172.21.15.5:0/1392447294 wait complete. 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.646+0000 7f2b0700d740 20 client.20913 trim_cache size 0 max 16384 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 -- 172.21.15.5:0/1029340765 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2aec0a2cf0 msgr2=0x7f2aec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2aec0a2cf0 0x7f2aec0a51b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2af8031860 tx=0x7f2af8005c50 comp rx=0 tx=0).stop 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 -- 172.21.15.5:0/1029340765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b001002f0 msgr2=0x7f2b001984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b001002f0 0x7f2b001984a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2af000e9f0 tx=0x7f2af000eeb0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 -- 172.21.15.5:0/1029340765 shutdown_connections 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b00106f50 0x7f2b0019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00100ca0 0x7f2b001989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2aec0a2cf0 0x7f2aec0a51b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 --2- 172.21.15.5:0/1029340765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b001002f0 0x7f2b001984a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 -- 172.21.15.5:0/1029340765 >> 172.21.15.5:0/1029340765 conn(0x7f2b00077210 msgr2=0x7f2b00105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 -- 172.21.15.5:0/1029340765 shutdown_connections 2024-09-11T22:40:06.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:06.647+0000 7f2b04c34640 1 -- 172.21.15.5:0/1029340765 wait complete. 2024-09-11T22:40:07.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67003 15.61 MB/sec execute 145 sec latency 473.111 ms 2024-09-11T22:40:08.034 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:07 smithi005 ceph-mon[30943]: pgmap v570: 97 pgs: 97 active+clean; 500 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 429 op/s 2024-09-11T22:40:08.062 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:40:08.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:07 smithi159 ceph-mon[35534]: pgmap v570: 97 pgs: 97 active+clean; 500 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 429 op/s 2024-09-11T22:40:08.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:07 smithi172 ceph-mon[35461]: pgmap v570: 97 pgs: 97 active+clean; 500 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 429 op/s 2024-09-11T22:40:08.334 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:08.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67478 15.68 MB/sec execute 146 sec latency 1473.206 ms 2024-09-11T22:40:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.863+0000 7f184d690640 1 -- 172.21.15.5:0/3649525922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1848106f10 msgr2=0x7f184810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.863+0000 7f184d690640 1 --2- 172.21.15.5:0/3649525922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1848106f10 0x7f184810ea90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f1840009ca0 tx=0x7f184002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.863+0000 7f184d690640 1 -- 172.21.15.5:0/3649525922 shutdown_connections 2024-09-11T22:40:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.863+0000 7f184d690640 1 --2- 172.21.15.5:0/3649525922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1848106f10 0x7f184810ea90 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.863+0000 7f184d690640 1 --2- 172.21.15.5:0/3649525922 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1848102790 0x7f18481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.863+0000 7f184d690640 1 --2- 172.21.15.5:0/3649525922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848101de0 0x7f18481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.863+0000 7f184d690640 1 -- 172.21.15.5:0/3649525922 >> 172.21.15.5:0/3649525922 conn(0x7f18480fd8c0 msgr2=0x7f18480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.863+0000 7f184d690640 1 -- 172.21.15.5:0/3649525922 shutdown_connections 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.863+0000 7f184d690640 1 -- 172.21.15.5:0/3649525922 wait complete. 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.864+0000 7f184d690640 1 Processor -- start 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.864+0000 7f184d690640 1 -- start start 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.864+0000 7f184d690640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1848101de0 0x7f18480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.864+0000 7f184d690640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1848102790 0x7f1848074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.864+0000 7f184d690640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848106f10 0x7f1848074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.864+0000 7f184d690640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1848111f70 con 0x7f1848102790 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.864+0000 7f184d690640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1848111df0 con 0x7f1848106f10 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.864+0000 7f184d690640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f18481120f0 con 0x7f1848101de0 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f1846ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1848101de0 0x7f18480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f18467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1848102790 0x7f1848074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f18477fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848106f10 0x7f1848074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f1846ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1848101de0 0x7f18480744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60930/0 (socket says 172.21.15.5:60930) 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f1846ffd640 1 -- 172.21.15.5:0/422991623 learned_addr learned my addr 172.21.15.5:0/422991623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f1846ffd640 1 -- 172.21.15.5:0/422991623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848106f10 msgr2=0x7f1848074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f1846ffd640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848106f10 0x7f1848074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f1846ffd640 1 -- 172.21.15.5:0/422991623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1848102790 msgr2=0x7f1848074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f1846ffd640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1848102790 0x7f1848074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f1846ffd640 1 -- 172.21.15.5:0/422991623 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18481a68b0 con 0x7f1848101de0 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f18467fc640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1848102790 0x7f1848074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f1846ffd640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1848101de0 0x7f18480744f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f183800d950 tx=0x7f183800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f182ffff640 1 -- 172.21.15.5:0/422991623 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1838017030 con 0x7f1848101de0 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f184d690640 1 -- 172.21.15.5:0/422991623 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18481a6ba0 con 0x7f1848101de0 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.865+0000 7f182ffff640 1 -- 172.21.15.5:0/422991623 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1838004020 con 0x7f1848101de0 2024-09-11T22:40:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.866+0000 7f182ffff640 1 -- 172.21.15.5:0/422991623 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18380047a0 con 0x7f1848101de0 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.866+0000 7f184d690640 1 -- 172.21.15.5:0/422991623 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18481a70e0 con 0x7f1848101de0 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.868+0000 7f182ffff640 1 -- 172.21.15.5:0/422991623 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f18380041d0 con 0x7f1848101de0 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.868+0000 7f184fa69740 1 Processor -- start 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.868+0000 7f184fa69740 1 -- start start 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.868+0000 7f184fa69740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef4c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.868+0000 7f184fa69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f184fa69740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef57370 0x55dddef59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f184fa69740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dddeeb22f0 con 0x55dddef4db40 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f184fa69740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dddee59cf0 con 0x55dddef57370 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f184fa69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dddeeae2b0 con 0x55dddef4c5d0 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f18477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef4c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f1846ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f18477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef4c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38244/0 (socket says 172.21.15.5:38244) 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f18477fe640 1 -- 172.21.15.5:0/4012914565 learned_addr learned my addr 172.21.15.5:0/4012914565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f18467fc640 1 --2- 172.21.15.5:0/4012914565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef57370 0x55dddef59760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f182ffff640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18300a2cf0 0x7f18300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f1846ffd640 1 -- 172.21.15.5:0/4012914565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef57370 msgr2=0x55dddef59760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f1846ffd640 1 --2- 172.21.15.5:0/4012914565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef57370 0x55dddef59760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f1846ffd640 1 -- 172.21.15.5:0/4012914565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 msgr2=0x55dddef4c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f1846ffd640 1 --2- 172.21.15.5:0/4012914565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef4c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f182ffff640 1 -- 172.21.15.5:0/422991623 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f18380cee50 con 0x7f1848101de0 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f1846ffd640 1 -- 172.21.15.5:0/4012914565 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dddee3e3a0 con 0x55dddef4c5d0 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f18477fe640 1 --2- 172.21.15.5:0/4012914565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef4c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f18467fc640 1 --2- 172.21.15.5:0/4012914565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef57370 0x55dddef59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.869+0000 7f18477fe640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18300a2cf0 0x7f18300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.870+0000 7f1846ffd640 1 --2- 172.21.15.5:0/4012914565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef4ca10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f18380cc4d0 tx=0x7f1838005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9059ae6b1e2f8b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.870+0000 7f182d7fa640 1 -- 172.21.15.5:0/4012914565 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183809ade0 con 0x55dddef4c5d0 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.870+0000 7f18477fe640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18300a2cf0 0x7f18300a51b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f184002f540 tx=0x7f18400095c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.870+0000 7f182d7fa640 1 -- 172.21.15.5:0/4012914565 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183809b9a0 con 0x55dddef4c5d0 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.870+0000 7f182d7fa640 1 -- 172.21.15.5:0/4012914565 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183809ac00 con 0x55dddef4c5d0 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.870+0000 7f184fa69740 1 -- 172.21.15.5:0/4012914565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 msgr2=0x55dddef4ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.870+0000 7f184fa69740 1 --2- 172.21.15.5:0/4012914565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef4ca10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f18380cc4d0 tx=0x7f1838005bf0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.871+0000 7f184fa69740 1 -- 172.21.15.5:0/4012914565 shutdown_connections 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.871+0000 7f184fa69740 1 --2- 172.21.15.5:0/4012914565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef57370 0x55dddef59760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.871+0000 7f184fa69740 1 --2- 172.21.15.5:0/4012914565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef4ca10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.871+0000 7f184fa69740 1 --2- 172.21.15.5:0/4012914565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef4c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.871+0000 7f184fa69740 1 -- 172.21.15.5:0/4012914565 >> 172.21.15.5:0/4012914565 conn(0x55dddef15310 msgr2=0x55dddef48c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.871+0000 7f184fa69740 1 -- 172.21.15.5:0/4012914565 shutdown_connections 2024-09-11T22:40:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.871+0000 7f184fa69740 1 -- 172.21.15.5:0/4012914565 wait complete. 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.871+0000 7f184fa69740 1 -- start start 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.871+0000 7f184fa69740 1 Processor -- start 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f184fa69740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4c5d0 0x55dddef56c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f184fa69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4db40 0x55dddef53f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f184fa69740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef54480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f184fa69740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dddef5a010 con 0x55dddef57370 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f184fa69740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dddee6d060 con 0x55dddef4c5d0 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f184fa69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55dddef5a190 con 0x55dddef4db40 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f184fa69740 10 client.0 _ll_register_callbacks cb 0x55dddebc08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f18477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef54480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f18477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef54480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38250/0 (socket says 172.21.15.5:38250) 2024-09-11T22:40:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f18477fe640 1 -- 172.21.15.5:0/1108763139 learned_addr learned my addr 172.21.15.5:0/1108763139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f18467fc640 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4db40 0x55dddef53f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f18477fe640 1 -- 172.21.15.5:0/1108763139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4db40 msgr2=0x55dddef53f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f18477fe640 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4db40 0x55dddef53f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f18477fe640 1 -- 172.21.15.5:0/1108763139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4c5d0 msgr2=0x55dddef56c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f18477fe640 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4c5d0 0x55dddef56c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.872+0000 7f18477fe640 1 -- 172.21.15.5:0/1108763139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dddec233c0 con 0x55dddef57370 2024-09-11T22:40:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.873+0000 7f18477fe640 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef54480 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f184002ade0 tx=0x7f184002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e210262b9d6c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.873+0000 7f182aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.873+0000 7f182aff5640 1 -- 172.21.15.5:0/1108763139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840004590 con 0x55dddef57370 2024-09-11T22:40:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.873+0000 7f182aff5640 1 -- 172.21.15.5:0/1108763139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f1840004730 con 0x55dddef57370 2024-09-11T22:40:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.873+0000 7f184fa69740 1 -- 172.21.15.5:0/1108763139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f183809b9a0 con 0x55dddef57370 2024-09-11T22:40:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.873+0000 7f182aff5640 1 -- 172.21.15.5:0/1108763139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f184002f610 con 0x55dddef57370 2024-09-11T22:40:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.873+0000 7f184fa69740 10 client.20925 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.873+0000 7f184fa69740 1 -- 172.21.15.5:0/1108763139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55dddef65970 con 0x55dddef57370 2024-09-11T22:40:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.873+0000 7f182aff5640 1 -- 172.21.15.5:0/1108763139 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183809b9a0 con 0x55dddef57370 2024-09-11T22:40:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.874+0000 7f182aff5640 1 -- 172.21.15.5:0/1108763139 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f1840051020 con 0x55dddef57370 2024-09-11T22:40:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.874+0000 7f184fa69740 10 client.20925 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.874+0000 7f184fa69740 10 client.20925 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.874+0000 7f184fa69740 20 client.20925 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.874+0000 7f184fa69740 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55dddef66d80 0x55dddef69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.874+0000 7f184fa69740 4 client.20925 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.874+0000 7f184fa69740 1 -- 172.21.15.5:0/1108763139 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dddeebf860 con 0x55dddef66d80 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.874+0000 7f1846ffd640 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55dddef66d80 0x55dddef69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.875+0000 7f1846ffd640 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55dddef66d80 0x55dddef69240 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b46ffa1575638e0 server_cookie=5971afd587d8768f in_seq=0 out_seq=0 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.875+0000 7f182aff5640 10 client.20925 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f182aff5640 1 -- 172.21.15.5:0/1108763139 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dddeebf860 con 0x55dddef66d80 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f182aff5640 10 client.20925 handle_command_reply: tid=0 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 client.20925 shutdown 2024-09-11T22:40:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 -- 172.21.15.5:0/1108763139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 msgr2=0x55dddef54480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:08.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef54480 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f184002ade0 tx=0x7f184002a8c0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 -- 172.21.15.5:0/1108763139 shutdown_connections 2024-09-11T22:40:08.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef54480 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4db40 0x55dddef53f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55dddef66d80 0x55dddef69240 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 --2- 172.21.15.5:0/1108763139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4c5d0 0x55dddef56c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f182aff5640 0 client.20925 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:40:08.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 -- 172.21.15.5:0/1108763139 >> 172.21.15.5:0/1108763139 conn(0x55dddef15310 msgr2=0x55dddef4e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:08.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 -- 172.21.15.5:0/1108763139 shutdown_connections 2024-09-11T22:40:08.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 1 -- 172.21.15.5:0/1108763139 wait complete. 2024-09-11T22:40:08.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.876+0000 7f184fa69740 20 client.20925 trim_cache size 0 max 16384 2024-09-11T22:40:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.890+0000 7f184fa69740 1 Processor -- start 2024-09-11T22:40:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f184fa69740 1 -- start start 2024-09-11T22:40:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f184fa69740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4c5d0 0x55dddef632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f184fa69740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef6eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f184fa69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef57370 0x55dddef6f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f184fa69740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18481afcf0 con 0x55dddef4db40 2024-09-11T22:40:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f184fa69740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f18481b1830 con 0x55dddef4c5d0 2024-09-11T22:40:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f184fa69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f18481b03c0 con 0x55dddef57370 2024-09-11T22:40:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f18467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef6eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f18467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef6eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38256/0 (socket says 172.21.15.5:38256) 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f18467fc640 1 -- 172.21.15.5:0/1176802967 learned_addr learned my addr 172.21.15.5:0/1176802967 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f1846ffd640 1 --2- 172.21.15.5:0/1176802967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4c5d0 0x55dddef632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f18477fe640 1 --2- 172.21.15.5:0/1176802967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef57370 0x55dddef6f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f18467fc640 1 -- 172.21.15.5:0/1176802967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef57370 msgr2=0x55dddef6f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f18467fc640 1 --2- 172.21.15.5:0/1176802967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef57370 0x55dddef6f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f18467fc640 1 -- 172.21.15.5:0/1176802967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4c5d0 msgr2=0x55dddef632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f18467fc640 1 --2- 172.21.15.5:0/1176802967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4c5d0 0x55dddef632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.891+0000 7f18467fc640 1 -- 172.21.15.5:0/1176802967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dddef49960 con 0x55dddef4db40 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.892+0000 7f18467fc640 1 --2- 172.21.15.5:0/1176802967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef6eb10 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f183c005fd0 tx=0x7f183c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd649c593b1b162 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.892+0000 7f182cff9640 1 -- 172.21.15.5:0/1176802967 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c003cc0 con 0x55dddef4db40 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.892+0000 7f182cff9640 1 -- 172.21.15.5:0/1176802967 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183c005500 con 0x55dddef4db40 2024-09-11T22:40:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.892+0000 7f182cff9640 1 -- 172.21.15.5:0/1176802967 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c01c6d0 con 0x55dddef4db40 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 -- 172.21.15.5:0/1176802967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 msgr2=0x55dddef6eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 --2- 172.21.15.5:0/1176802967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef6eb10 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f183c005fd0 tx=0x7f183c0035f0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 -- 172.21.15.5:0/1176802967 shutdown_connections 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 --2- 172.21.15.5:0/1176802967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef57370 0x55dddef6f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 --2- 172.21.15.5:0/1176802967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef4db40 0x55dddef6eb10 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 --2- 172.21.15.5:0/1176802967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4c5d0 0x55dddef632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 -- 172.21.15.5:0/1176802967 >> 172.21.15.5:0/1176802967 conn(0x55dddef15310 msgr2=0x55dddef53f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 -- 172.21.15.5:0/1176802967 shutdown_connections 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 -- 172.21.15.5:0/1176802967 wait complete. 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 -- start start 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.893+0000 7f184fa69740 1 Processor -- start 2024-09-11T22:40:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f184fa69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef5f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f184fa69740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4db40 0x55dddef5fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f184fa69740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef5bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f184fa69740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18481b1160 con 0x55dddef57370 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f184fa69740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f18481b0a90 con 0x55dddef4db40 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f184fa69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f18481adcf0 con 0x55dddef4c5d0 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f184fa69740 10 client.0 _ll_register_callbacks cb 0x7f1840009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f1846ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef5f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f18477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef5bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f1846ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef5f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:60968/0 (socket says 172.21.15.5:60968) 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f1846ffd640 1 -- 172.21.15.5:0/3138829849 learned_addr learned my addr 172.21.15.5:0/3138829849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f18467fc640 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4db40 0x55dddef5fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f1846ffd640 1 -- 172.21.15.5:0/3138829849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4db40 msgr2=0x55dddef5fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f1846ffd640 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4db40 0x55dddef5fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f1846ffd640 1 -- 172.21.15.5:0/3138829849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 msgr2=0x55dddef5bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f1846ffd640 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef5bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f1846ffd640 1 -- 172.21.15.5:0/3138829849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dddef4f120 con 0x55dddef4c5d0 2024-09-11T22:40:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f18467fc640 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4db40 0x55dddef5fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f18477fe640 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef5bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f1846ffd640 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef5f7b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f18380d2940 tx=0x7f18380cf0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b07ec827e51d4a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.894+0000 7f182aff5640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f182aff5640 1 -- 172.21.15.5:0/3138829849 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183809ac00 con 0x55dddef4c5d0 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f182aff5640 1 -- 172.21.15.5:0/3138829849 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 8629+0+0 (secure 0 0 0) 0x7f18380cc750 con 0x55dddef4c5d0 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f184fa69740 1 -- 172.21.15.5:0/3138829849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f183c005500 con 0x55dddef4c5d0 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f182aff5640 1 -- 172.21.15.5:0/3138829849 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f183809b9a0 con 0x55dddef4c5d0 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f184fa69740 10 client.29782 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f184fa69740 1 -- 172.21.15.5:0/3138829849 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55dddefd2550 con 0x55dddef4c5d0 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f182aff5640 1 -- 172.21.15.5:0/3138829849 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18380cc950 con 0x55dddef4c5d0 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f182aff5640 1 -- 172.21.15.5:0/3138829849 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f18380d2450 con 0x55dddef4c5d0 2024-09-11T22:40:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f184fa69740 10 client.29782 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f184fa69740 10 client.29782 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f184fa69740 20 client.29782 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f184fa69740 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55dddefd27e0 0x55dddefd4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f184fa69740 4 client.29782 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.895+0000 7f184fa69740 1 -- 172.21.15.5:0/3138829849 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f183c005500 con 0x55dddefd27e0 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.896+0000 7f18467fc640 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55dddefd27e0 0x55dddefd4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.896+0000 7f182aff5640 10 client.29782 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.896+0000 7f18467fc640 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55dddefd27e0 0x55dddefd4bd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d15c504ac1c8fdec server_cookie=4654fb18510f2bad in_seq=0 out_seq=0 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.896+0000 7f182aff5640 1 -- 172.21.15.5:0/3138829849 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f183c005500 con 0x55dddefd27e0 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.896+0000 7f182aff5640 10 client.29782 handle_command_reply: tid=0 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.896+0000 7f184fa69740 1 client.29782 shutdown 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.896+0000 7f184fa69740 1 -- 172.21.15.5:0/3138829849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 msgr2=0x55dddef5f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.896+0000 7f184fa69740 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef5f7b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f18380d2940 tx=0x7f18380cf0d0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184fa69740 1 -- 172.21.15.5:0/3138829849 shutdown_connections 2024-09-11T22:40:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184fa69740 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dddef57370 0x55dddef5bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184fa69740 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x55dddefd27e0 0x55dddefd4bd0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184fa69740 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dddef4db40 0x55dddef5fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f182aff5640 0 client.29782 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:40:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184fa69740 1 --2- 172.21.15.5:0/3138829849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dddef4c5d0 0x55dddef5f7b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184fa69740 1 -- 172.21.15.5:0/3138829849 >> 172.21.15.5:0/3138829849 conn(0x55dddef15310 msgr2=0x55dddef474e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184fa69740 1 -- 172.21.15.5:0/3138829849 shutdown_connections 2024-09-11T22:40:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184fa69740 1 -- 172.21.15.5:0/3138829849 wait complete. 2024-09-11T22:40:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184fa69740 20 client.29782 trim_cache size 0 max 16384 2024-09-11T22:40:08.909 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:40:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 -- 172.21.15.5:0/422991623 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18300a2cf0 msgr2=0x7f18300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18300a2cf0 0x7f18300a51b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f184002f540 tx=0x7f18400095c0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 -- 172.21.15.5:0/422991623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1848101de0 msgr2=0x7f18480744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1848101de0 0x7f18480744f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f183800d950 tx=0x7f183800de10 comp rx=0 tx=0).stop 2024-09-11T22:40:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 -- 172.21.15.5:0/422991623 shutdown_connections 2024-09-11T22:40:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f18300a2cf0 0x7f18300a51b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1848106f10 0x7f1848074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1848102790 0x7f1848074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 --2- 172.21.15.5:0/422991623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1848101de0 0x7f18480744f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 -- 172.21.15.5:0/422991623 >> 172.21.15.5:0/422991623 conn(0x7f18480fd8c0 msgr2=0x7f18480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.897+0000 7f184d690640 1 -- 172.21.15.5:0/422991623 shutdown_connections 2024-09-11T22:40:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:08.898+0000 7f184d690640 1 -- 172.21.15.5:0/422991623 wait complete. 2024-09-11T22:40:09.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68071 15.85 MB/sec execute 147 sec latency 2473.237 ms 2024-09-11T22:40:10.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:09 smithi005 ceph-mon[30943]: pgmap v571: 97 pgs: 97 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 695 op/s 2024-09-11T22:40:10.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:09 smithi159 ceph-mon[35534]: pgmap v571: 97 pgs: 97 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 695 op/s 2024-09-11T22:40:10.101 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:40:10.109 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:09 smithi172 ceph-mon[35461]: pgmap v571: 97 pgs: 97 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.1 MiB/s wr, 695 op/s 2024-09-11T22:40:10.560 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:10.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68380 15.84 MB/sec execute 148 sec latency 3473.792 ms 2024-09-11T22:40:11.028 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:10 smithi005 ceph-mon[30943]: osdmap e116: 12 total, 12 up, 12 in 2024-09-11T22:40:11.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:10 smithi159 ceph-mon[35534]: osdmap e116: 12 total, 12 up, 12 in 2024-09-11T22:40:11.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:10 smithi172 ceph-mon[35461]: osdmap e116: 12 total, 12 up, 12 in 2024-09-11T22:40:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.151+0000 7f2dfdd9b640 1 -- 172.21.15.5:0/287789125 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80307c0 con 0x7f2e0010e4f0 2024-09-11T22:40:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.152+0000 7f2e04e9d640 1 -- 172.21.15.5:0/287789125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e0010e4f0 msgr2=0x7f2e001108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.152+0000 7f2e04e9d640 1 --2- 172.21.15.5:0/287789125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e0010e4f0 0x7f2e001108e0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009ca0 tx=0x7f2df802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.152+0000 7f2e04e9d640 1 -- 172.21.15.5:0/287789125 shutdown_connections 2024-09-11T22:40:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.152+0000 7f2e04e9d640 1 --2- 172.21.15.5:0/287789125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e0010e4f0 0x7f2e001108e0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.152+0000 7f2e04e9d640 1 --2- 172.21.15.5:0/287789125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e001012f0 0x7f2e0010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.152+0000 7f2e04e9d640 1 --2- 172.21.15.5:0/287789125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e00100940 0x7f2e00100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.152+0000 7f2e04e9d640 1 -- 172.21.15.5:0/287789125 >> 172.21.15.5:0/287789125 conn(0x7f2e000fc7d0 msgr2=0x7f2e000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.153+0000 7f2e04e9d640 1 -- 172.21.15.5:0/287789125 shutdown_connections 2024-09-11T22:40:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.153+0000 7f2e04e9d640 1 -- 172.21.15.5:0/287789125 wait complete. 2024-09-11T22:40:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.154+0000 7f2e04e9d640 1 Processor -- start 2024-09-11T22:40:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.154+0000 7f2e04e9d640 1 -- start start 2024-09-11T22:40:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.155+0000 7f2e04e9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e00100940 0x7f2e0011aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.155+0000 7f2e04e9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e001012f0 0x7f2e0011af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.155+0000 7f2e04e9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e0010e4f0 0x7f2e00112e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.155+0000 7f2e04e9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00103eb0 con 0x7f2e001012f0 2024-09-11T22:40:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.155+0000 7f2e04e9d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00103d30 con 0x7f2e0010e4f0 2024-09-11T22:40:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.155+0000 7f2e04e9d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00104030 con 0x7f2e00100940 2024-09-11T22:40:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.155+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e001012f0 0x7f2e0011af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.155+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e001012f0 0x7f2e0011af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38286/0 (socket says 172.21.15.5:38286) 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.155+0000 7f2df7fff640 1 -- 172.21.15.5:0/2069741468 learned_addr learned my addr 172.21.15.5:0/2069741468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.156+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e00100940 0x7f2e0011aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.156+0000 7f2df7fff640 1 -- 172.21.15.5:0/2069741468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e00100940 msgr2=0x7f2e0011aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.156+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e00100940 0x7f2e0011aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.156+0000 7f2df7fff640 1 -- 172.21.15.5:0/2069741468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e0010e4f0 msgr2=0x7f2e00112e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.156+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e0010e4f0 0x7f2e00112e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.156+0000 7f2df7fff640 1 -- 172.21.15.5:0/2069741468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e00113610 con 0x7f2e001012f0 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.156+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e001012f0 0x7f2e0011af50 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e9f0 tx=0x7f2de800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.157+0000 7f2df77fe640 1 -- 172.21.15.5:0/2069741468 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de800cd80 con 0x7f2e001012f0 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.157+0000 7f2df77fe640 1 -- 172.21.15.5:0/2069741468 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8010910 con 0x7f2e001012f0 2024-09-11T22:40:11.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.157+0000 7f2df77fe640 1 -- 172.21.15.5:0/2069741468 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de80188f0 con 0x7f2e001012f0 2024-09-11T22:40:11.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.157+0000 7f2e04e9d640 1 -- 172.21.15.5:0/2069741468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e00113900 con 0x7f2e001012f0 2024-09-11T22:40:11.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.157+0000 7f2e04e9d640 1 -- 172.21.15.5:0/2069741468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e00108690 con 0x7f2e001012f0 2024-09-11T22:40:11.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.161+0000 7f2df77fe640 1 -- 172.21.15.5:0/2069741468 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2de8010ac0 con 0x7f2e001012f0 2024-09-11T22:40:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.163+0000 7f2df77fe640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2dec0a2ec0 0x7f2dec0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.164+0000 7f2df77fe640 1 -- 172.21.15.5:0/2069741468 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f2de80cf8d0 con 0x7f2e001012f0 2024-09-11T22:40:11.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.164+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2dec0a2ec0 0x7f2dec0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.165+0000 7f2e07276740 1 Processor -- start 2024-09-11T22:40:11.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.165+0000 7f2e07276740 1 -- start start 2024-09-11T22:40:11.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.165+0000 7f2e07276740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1eab40 0x55b0fd1e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.165+0000 7f2e07276740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.165+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2dec0a2ec0 0x7f2dec0a5380 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2df000a810 tx=0x7f2df0008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:11.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.166+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.166+0000 7f2e07276740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.166+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1e9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:32768/0 (socket says 172.21.15.5:32768) 2024-09-11T22:40:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.166+0000 7f2df7fff640 1 -- 172.21.15.5:0/996268607 learned_addr learned my addr 172.21.15.5:0/996268607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.166+0000 7f2e07276740 1 -- 172.21.15.5:0/996268607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b0fd14f2f0 con 0x55b0fd1f4370 2024-09-11T22:40:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.166+0000 7f2e07276740 1 -- 172.21.15.5:0/996268607 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b0fd0f6cf0 con 0x55b0fd1eab40 2024-09-11T22:40:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.166+0000 7f2e07276740 1 -- 172.21.15.5:0/996268607 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b0fd14b2b0 con 0x55b0fd1e95d0 2024-09-11T22:40:11.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.167+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/996268607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1eab40 0x55b0fd1e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.167+0000 7f2dfed9d640 1 --2- 172.21.15.5:0/996268607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.168+0000 7f2dfed9d640 1 -- 172.21.15.5:0/996268607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1e95d0 msgr2=0x55b0fd1e9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.168+0000 7f2dfed9d640 1 --2- 172.21.15.5:0/996268607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1e9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.168+0000 7f2dfed9d640 1 -- 172.21.15.5:0/996268607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1eab40 msgr2=0x55b0fd1e9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:11.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.168+0000 7f2dfed9d640 1 --2- 172.21.15.5:0/996268607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1eab40 0x55b0fd1e9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.168+0000 7f2dfed9d640 1 -- 172.21.15.5:0/996268607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fd0db3a0 con 0x55b0fd1f4370 2024-09-11T22:40:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.168+0000 7f2dfed9d640 1 --2- 172.21.15.5:0/996268607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f6760 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009ca0 tx=0x7f2df8031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937ddbdb89e542c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.169+0000 7f2df4ff9640 1 -- 172.21.15.5:0/996268607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8003970 con 0x55b0fd1f4370 2024-09-11T22:40:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.169+0000 7f2df4ff9640 1 -- 172.21.15.5:0/996268607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2df8036070 con 0x55b0fd1f4370 2024-09-11T22:40:11.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.169+0000 7f2df4ff9640 1 -- 172.21.15.5:0/996268607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8004500 con 0x55b0fd1f4370 2024-09-11T22:40:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.170+0000 7f2e07276740 1 -- 172.21.15.5:0/996268607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1f4370 msgr2=0x55b0fd1f6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:11.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.170+0000 7f2e07276740 1 --2- 172.21.15.5:0/996268607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f6760 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009ca0 tx=0x7f2df8031b20 comp rx=0 tx=0).stop 2024-09-11T22:40:11.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.172+0000 7f2e07276740 1 -- 172.21.15.5:0/996268607 shutdown_connections 2024-09-11T22:40:11.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.172+0000 7f2e07276740 1 --2- 172.21.15.5:0/996268607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f6760 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.172+0000 7f2e07276740 1 --2- 172.21.15.5:0/996268607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1e9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.172+0000 7f2e07276740 1 --2- 172.21.15.5:0/996268607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1eab40 0x55b0fd1e9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.173+0000 7f2e07276740 1 -- 172.21.15.5:0/996268607 >> 172.21.15.5:0/996268607 conn(0x55b0fd1b2310 msgr2=0x55b0fd1e5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:11.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.173+0000 7f2e07276740 1 -- 172.21.15.5:0/996268607 shutdown_connections 2024-09-11T22:40:11.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.173+0000 7f2e07276740 1 -- 172.21.15.5:0/996268607 wait complete. 2024-09-11T22:40:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.174+0000 7f2e07276740 1 -- start start 2024-09-11T22:40:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.174+0000 7f2e07276740 1 Processor -- start 2024-09-11T22:40:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.174+0000 7f2e07276740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.174+0000 7f2e07276740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1eab40 0x55b0fd1f1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.174+0000 7f2e07276740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.174+0000 7f2e07276740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b0fd1f7010 con 0x55b0fd1eab40 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.174+0000 7f2e07276740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b0fd10a060 con 0x55b0fd1e95d0 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.174+0000 7f2e07276740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b0fd1f7190 con 0x55b0fd1f4370 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.174+0000 7f2e07276740 10 client.0 _ll_register_callbacks cb 0x55b0fce5d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2dfed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2dfed9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:32772/0 (socket says 172.21.15.5:32772) 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2dfed9d640 1 -- 172.21.15.5:0/2210676829 learned_addr learned my addr 172.21.15.5:0/2210676829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1eab40 0x55b0fd1f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2df7fff640 1 -- 172.21.15.5:0/2210676829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1f4370 msgr2=0x55b0fd1f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2df7fff640 1 -- 172.21.15.5:0/2210676829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1e95d0 msgr2=0x55b0fd1f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2df7fff640 1 -- 172.21.15.5:0/2210676829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fcec03c0 con 0x55b0fd1eab40 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2dfed9d640 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1eab40 0x55b0fd1f1040 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f2de80a4430 tx=0x7f2de809c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce3e93a3a5e3f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.175+0000 7f2de5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.176+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2210676829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de80cd3b0 con 0x55b0fd1eab40 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.176+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2210676829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f2de809b340 con 0x55b0fd1eab40 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.176+0000 7f2e07276740 1 -- 172.21.15.5:0/2210676829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df8036070 con 0x55b0fd1eab40 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.176+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2210676829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2de80a4bb0 con 0x55b0fd1eab40 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.176+0000 7f2e07276740 10 client.20943 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.176+0000 7f2e07276740 1 -- 172.21.15.5:0/2210676829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b0fd202ae0 con 0x55b0fd1eab40 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.176+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2210676829 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8097a40 con 0x55b0fd1eab40 2024-09-11T22:40:11.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.177+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2210676829 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f2de8018a90 con 0x55b0fd1eab40 2024-09-11T22:40:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.177+0000 7f2e07276740 10 client.20943 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.177+0000 7f2e07276740 10 client.20943 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:40:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.177+0000 7f2e07276740 20 client.20943 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.177+0000 7f2e07276740 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0fd203ef0 0x55b0fd2062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.177+0000 7f2e07276740 4 client.20943 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.177+0000 7f2e07276740 1 -- 172.21.15.5:0/2210676829 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2df8036070 con 0x55b0fd203ef0 2024-09-11T22:40:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.177+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0fd203ef0 0x55b0fd2062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.177+0000 7f2de5ffb640 10 client.20943 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:40:11.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.177+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0fd203ef0 0x55b0fd2062e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e1c2083c0c50cf0d server_cookie=2be924d2ff3d4e55 in_seq=0 out_seq=0 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2210676829 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2df8036070 con 0x55b0fd203ef0 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2de5ffb640 10 client.20943 handle_command_reply: tid=0 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 client.20943 shutdown 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 -- 172.21.15.5:0/2210676829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1eab40 msgr2=0x55b0fd1f1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1eab40 0x55b0fd1f1040 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f2de80a4430 tx=0x7f2de809c530 comp rx=0 tx=0).stop 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 -- 172.21.15.5:0/2210676829 shutdown_connections 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0fd203ef0 0x55b0fd2062e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd1f4370 0x55b0fd1f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1eab40 0x55b0fd1f1040 secure :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f2de80a4430 tx=0x7f2de809c530 comp rx=0 tx=0).stop 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 --2- 172.21.15.5:0/2210676829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 -- 172.21.15.5:0/2210676829 >> 172.21.15.5:0/2210676829 conn(0x55b0fd1b2310 msgr2=0x55b0fd1eb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:11.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2de5ffb640 0 client.20943 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:40:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 -- 172.21.15.5:0/2210676829 shutdown_connections 2024-09-11T22:40:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 1 -- 172.21.15.5:0/2210676829 wait complete. 2024-09-11T22:40:11.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.179+0000 7f2e07276740 20 client.20943 trim_cache size 0 max 16384 2024-09-11T22:40:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.199+0000 7f2e07276740 1 Processor -- start 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.199+0000 7f2e07276740 1 -- start start 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.199+0000 7f2e07276740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1ec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.199+0000 7f2e07276740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 0x55b0fd1ec660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2e07276740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd20bb90 0x55b0fd20bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2e07276740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001c51e0 con 0x55b0fd1e95d0 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2e07276740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001c6d20 con 0x55b0fd1f4370 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2e07276740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001c58b0 con 0x55b0fd20bb90 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2dfe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1ec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2dfe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1ec120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38316/0 (socket says 172.21.15.5:38316) 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2dfe59c640 1 -- 172.21.15.5:0/2864702025 learned_addr learned my addr 172.21.15.5:0/2864702025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2dfed9d640 1 --2- 172.21.15.5:0/2864702025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd20bb90 0x55b0fd20bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2864702025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 0x55b0fd1ec660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2dfe59c640 1 -- 172.21.15.5:0/2864702025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd20bb90 msgr2=0x55b0fd20bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2864702025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd20bb90 0x55b0fd20bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2dfe59c640 1 -- 172.21.15.5:0/2864702025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 msgr2=0x55b0fd1ec660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2864702025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 0x55b0fd1ec660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.200+0000 7f2dfe59c640 1 -- 172.21.15.5:0/2864702025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fd1f4100 con 0x55b0fd1e95d0 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.201+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2864702025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1ec120 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f2df000a5f0 tx=0x7f2df0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2795c2f1140c5891 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.201+0000 7f2de7fff640 1 -- 172.21.15.5:0/2864702025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df002d070 con 0x55b0fd1e95d0 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.201+0000 7f2de7fff640 1 -- 172.21.15.5:0/2864702025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2df000c950 con 0x55b0fd1e95d0 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.201+0000 7f2de7fff640 1 -- 172.21.15.5:0/2864702025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0021a30 con 0x55b0fd1e95d0 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.202+0000 7f2e07276740 1 -- 172.21.15.5:0/2864702025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 msgr2=0x55b0fd1ec120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.202+0000 7f2e07276740 1 --2- 172.21.15.5:0/2864702025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1ec120 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f2df000a5f0 tx=0x7f2df0012000 comp rx=0 tx=0).stop 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.202+0000 7f2e07276740 1 -- 172.21.15.5:0/2864702025 shutdown_connections 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.202+0000 7f2e07276740 1 --2- 172.21.15.5:0/2864702025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd20bb90 0x55b0fd20bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.202+0000 7f2e07276740 1 --2- 172.21.15.5:0/2864702025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 0x55b0fd1ec660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.202+0000 7f2e07276740 1 --2- 172.21.15.5:0/2864702025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1ec120 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.202+0000 7f2e07276740 1 -- 172.21.15.5:0/2864702025 >> 172.21.15.5:0/2864702025 conn(0x55b0fd1b2310 msgr2=0x55b0fd1e4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:11.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.202+0000 7f2e07276740 1 -- 172.21.15.5:0/2864702025 shutdown_connections 2024-09-11T22:40:11.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.202+0000 7f2e07276740 1 -- 172.21.15.5:0/2864702025 wait complete. 2024-09-11T22:40:11.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.203+0000 7f2e07276740 1 -- start start 2024-09-11T22:40:11.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.203+0000 7f2e07276740 1 Processor -- start 2024-09-11T22:40:11.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.203+0000 7f2e07276740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1fc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.203+0000 7f2e07276740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 0x55b0fd1fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2e07276740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd20bb90 0x55b0fd1f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2dfe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1fc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2dfe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1fc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38322/0 (socket says 172.21.15.5:38322) 2024-09-11T22:40:11.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2dfe59c640 1 -- 172.21.15.5:0/2799571094 learned_addr learned my addr 172.21.15.5:0/2799571094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:11.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001c6650 con 0x55b0fd1e95d0 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001c5f80 con 0x55b0fd1f4370 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0010e0c0 con 0x55b0fd20bb90 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 0x55b0fd1fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2e07276740 10 client.0 _ll_register_callbacks cb 0x7f2de809b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2dfe59c640 1 -- 172.21.15.5:0/2799571094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd20bb90 msgr2=0x55b0fd1f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd20bb90 0x55b0fd1f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2dfe59c640 1 -- 172.21.15.5:0/2799571094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 msgr2=0x55b0fd1fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 0x55b0fd1fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2dfe59c640 1 -- 172.21.15.5:0/2799571094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fd1f99b0 con 0x55b0fd1e95d0 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 0x55b0fd1fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2dfe59c640 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1fc420 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f2df0008010 tx=0x7f2df00216a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47594260ccc5e12d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.204+0000 7f2de5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:11.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.205+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2799571094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df002d070 con 0x55b0fd1e95d0 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.205+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2799571094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f2df00218b0 con 0x55b0fd1e95d0 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.205+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df000c950 con 0x55b0fd1e95d0 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.205+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2799571094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2df0021ad0 con 0x55b0fd1e95d0 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.205+0000 7f2e07276740 10 client.20949 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.205+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b0fd26f3f0 con 0x55b0fd1e95d0 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.206+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2799571094 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2df0021c70 con 0x55b0fd1e95d0 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.206+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2799571094 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f2df00113d0 con 0x55b0fd1e95d0 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.206+0000 7f2e07276740 10 client.20949 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.206+0000 7f2e07276740 10 client.20949 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.206+0000 7f2e07276740 20 client.20949 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.206+0000 7f2e07276740 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0fd26f9b0 0x55b0fd271dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.206+0000 7f2e07276740 4 client.20949 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.206+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2df000c950 con 0x55b0fd26f9b0 2024-09-11T22:40:11.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.207+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0fd26f9b0 0x55b0fd271dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.207+0000 7f2de5ffb640 10 client.20949 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:40:11.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.207+0000 7f2df7fff640 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0fd26f9b0 0x55b0fd271dd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b2d3a88a007748b server_cookie=29b1296bb568a9a3 in_seq=0 out_seq=0 2024-09-11T22:40:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2de5ffb640 1 -- 172.21.15.5:0/2799571094 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2df000c950 con 0x55b0fd26f9b0 2024-09-11T22:40:11.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2de5ffb640 10 client.20949 handle_command_reply: tid=0 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2e07276740 1 client.20949 shutdown 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 msgr2=0x55b0fd1fc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2e07276740 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1fc420 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f2df0008010 tx=0x7f2df00216a0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 shutdown_connections 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2e07276740 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55b0fd26f9b0 0x55b0fd271dd0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2de5ffb640 0 client.20949 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2e07276740 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b0fd20bb90 0x55b0fd1f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2e07276740 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b0fd1f4370 0x55b0fd1fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2e07276740 1 --2- 172.21.15.5:0/2799571094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b0fd1e95d0 0x55b0fd1fc420 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.208+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 >> 172.21.15.5:0/2799571094 conn(0x55b0fd1b2310 msgr2=0x55b0fd1e4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.209+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 shutdown_connections 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.209+0000 7f2e07276740 1 -- 172.21.15.5:0/2799571094 wait complete. 2024-09-11T22:40:11.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.210+0000 7f2e07276740 20 client.20949 trim_cache size 0 max 16384 2024-09-11T22:40:11.214 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:40:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.211+0000 7f2e04e9d640 1 -- 172.21.15.5:0/2069741468 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2dec0a2ec0 msgr2=0x7f2dec0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.211+0000 7f2e04e9d640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2dec0a2ec0 0x7f2dec0a5380 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2df000a810 tx=0x7f2df0008040 comp rx=0 tx=0).stop 2024-09-11T22:40:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.211+0000 7f2e04e9d640 1 -- 172.21.15.5:0/2069741468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e001012f0 msgr2=0x7f2e0011af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.211+0000 7f2e04e9d640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e001012f0 0x7f2e0011af50 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e9f0 tx=0x7f2de800eeb0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.212+0000 7f2e04e9d640 1 -- 172.21.15.5:0/2069741468 shutdown_connections 2024-09-11T22:40:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.212+0000 7f2e04e9d640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2dec0a2ec0 0x7f2dec0a5380 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.212+0000 7f2e04e9d640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e0010e4f0 0x7f2e00112e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.212+0000 7f2e04e9d640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e001012f0 0x7f2e0011af50 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.212+0000 7f2e04e9d640 1 --2- 172.21.15.5:0/2069741468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e00100940 0x7f2e0011aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.213+0000 7f2e04e9d640 1 -- 172.21.15.5:0/2069741468 >> 172.21.15.5:0/2069741468 conn(0x7f2e000fc7d0 msgr2=0x7f2e000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.213+0000 7f2e04e9d640 1 -- 172.21.15.5:0/2069741468 shutdown_connections 2024-09-11T22:40:11.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:11.213+0000 7f2e04e9d640 1 -- 172.21.15.5:0/2069741468 wait complete. 2024-09-11T22:40:11.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68582 15.81 MB/sec execute 149 sec latency 4473.829 ms 2024-09-11T22:40:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:11 smithi005 ceph-mon[30943]: pgmap v573: 97 pgs: 97 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-09-11T22:40:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:11 smithi159 ceph-mon[35534]: pgmap v573: 97 pgs: 97 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-09-11T22:40:12.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:11 smithi172 ceph-mon[35461]: pgmap v573: 97 pgs: 97 active+clean; 513 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.8 MiB/s wr, 656 op/s 2024-09-11T22:40:12.188 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:40:12.582 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:12.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68820 15.83 MB/sec execute 150 sec latency 4834.328 ms 2024-09-11T22:40:13.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.352+0000 7f3aa03d2640 1 -- 172.21.15.5:0/3203275942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a98103760 msgr2=0x7f3a98107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:13.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.352+0000 7f3aa03d2640 1 --2- 172.21.15.5:0/3203275942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a98103760 0x7f3a98107ba0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c009930 tx=0x7f3a8c02a420 comp rx=0 tx=0).stop 2024-09-11T22:40:13.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.352+0000 7f3aa03d2640 1 -- 172.21.15.5:0/3203275942 shutdown_connections 2024-09-11T22:40:13.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.352+0000 7f3aa03d2640 1 --2- 172.21.15.5:0/3203275942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a981082e0 0x7f3a9810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.352+0000 7f3aa03d2640 1 --2- 172.21.15.5:0/3203275942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a98103760 0x7f3a98107ba0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.352+0000 7f3aa03d2640 1 --2- 172.21.15.5:0/3203275942 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a98102db0 0x7f3a98103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.352+0000 7f3aa03d2640 1 -- 172.21.15.5:0/3203275942 >> 172.21.15.5:0/3203275942 conn(0x7f3a980fc7d0 msgr2=0x7f3a980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:13.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.352+0000 7f3aa03d2640 1 -- 172.21.15.5:0/3203275942 shutdown_connections 2024-09-11T22:40:13.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.352+0000 7f3aa03d2640 1 -- 172.21.15.5:0/3203275942 wait complete. 2024-09-11T22:40:13.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.353+0000 7f3aa03d2640 1 Processor -- start 2024-09-11T22:40:13.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.353+0000 7f3aa03d2640 1 -- start start 2024-09-11T22:40:13.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.353+0000 7f3aa03d2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a98102db0 0x7f3a98100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.353+0000 7f3aa03d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a98103760 0x7f3a98100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.353+0000 7f3aa03d2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a981082e0 0x7f3a98101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.353+0000 7f3aa03d2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a98112070 con 0x7f3a98103760 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.353+0000 7f3aa03d2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a98111ef0 con 0x7f3a98102db0 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.353+0000 7f3aa03d2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981121f0 con 0x7f3a981082e0 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9d96d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a98103760 0x7f3a98100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9d96d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a98103760 0x7f3a98100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57442/0 (socket says 172.21.15.5:57442) 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9d96d640 1 -- 172.21.15.5:0/1933655992 learned_addr learned my addr 172.21.15.5:0/1933655992 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9e16e640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a98102db0 0x7f3a98100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9e96f640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a981082e0 0x7f3a98101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9d96d640 1 -- 172.21.15.5:0/1933655992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a981082e0 msgr2=0x7f3a98101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:13.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a981082e0 0x7f3a98101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9d96d640 1 -- 172.21.15.5:0/1933655992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a98102db0 msgr2=0x7f3a98100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a98102db0 0x7f3a98100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9d96d640 1 -- 172.21.15.5:0/1933655992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a981aaf90 con 0x7f3a98103760 2024-09-11T22:40:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a98103760 0x7f3a98100f50 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c005940 tx=0x7f3a8c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a977fe640 1 -- 172.21.15.5:0/1933655992 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c03f070 con 0x7f3a98103760 2024-09-11T22:40:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3a977fe640 1 -- 172.21.15.5:0/1933655992 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8c031c30 con 0x7f3a98103760 2024-09-11T22:40:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.354+0000 7f3aa03d2640 1 -- 172.21.15.5:0/1933655992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a981ab160 con 0x7f3a98103760 2024-09-11T22:40:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.355+0000 7f3a977fe640 1 -- 172.21.15.5:0/1933655992 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c035030 con 0x7f3a98103760 2024-09-11T22:40:13.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.355+0000 7f3aa03d2640 1 -- 172.21.15.5:0/1933655992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a981ab6a0 con 0x7f3a98103760 2024-09-11T22:40:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.357+0000 7f3a977fe640 1 -- 172.21.15.5:0/1933655992 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3a8c042070 con 0x7f3a98103760 2024-09-11T22:40:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.357+0000 7f3aa27ab740 1 Processor -- start 2024-09-11T22:40:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.357+0000 7f3aa27ab740 1 -- start start 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 0x564f376e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376e55d0 0x564f376e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376f0370 0x564f376f2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f3764b2f0 con 0x564f376f0370 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564f375f2cf0 con 0x564f376e55d0 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564f376472b0 con 0x564f376e6b40 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e96f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 0x564f376e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e16e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376e55d0 0x564f376e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e96f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 0x564f376e5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33566/0 (socket says 172.21.15.5:33566) 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e96f640 1 -- 172.21.15.5:0/1143135530 learned_addr learned my addr 172.21.15.5:0/1143135530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/1143135530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376f0370 0x564f376f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a977fe640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3a800a2cf0 0x7f3a800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e96f640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3a800a2cf0 0x7f3a800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e96f640 1 -- 172.21.15.5:0/1143135530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376e55d0 msgr2=0x564f376e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e96f640 1 --2- 172.21.15.5:0/1143135530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376e55d0 0x564f376e5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e96f640 1 -- 172.21.15.5:0/1143135530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376f0370 msgr2=0x564f376f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e96f640 1 --2- 172.21.15.5:0/1143135530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376f0370 0x564f376f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e96f640 1 -- 172.21.15.5:0/1143135530 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f375d73a0 con 0x564f376e6b40 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9e16e640 1 --2- 172.21.15.5:0/1143135530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376e55d0 0x564f376e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/1143135530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376f0370 0x564f376f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.358+0000 7f3a977fe640 1 -- 172.21.15.5:0/1933655992 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f3a8c0efb00 con 0x7f3a98103760 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.359+0000 7f3a9e96f640 1 --2- 172.21.15.5:0/1143135530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 0x564f376e5090 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3a88005c40 tx=0x7f3a88013f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a0d4125d02db08c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.359+0000 7f3a9e96f640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3a800a2cf0 0x7f3a800a51b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3a8801d000 tx=0x7f3a8801f000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.359+0000 7f3a94dea640 1 -- 172.21.15.5:0/1143135530 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8800fde0 con 0x564f376e6b40 2024-09-11T22:40:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.359+0000 7f3a94dea640 1 -- 172.21.15.5:0/1143135530 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a88022020 con 0x564f376e6b40 2024-09-11T22:40:13.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.359+0000 7f3a94dea640 1 -- 172.21.15.5:0/1143135530 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a88010690 con 0x564f376e6b40 2024-09-11T22:40:13.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.360+0000 7f3aa27ab740 1 -- 172.21.15.5:0/1143135530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 msgr2=0x564f376e5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:13.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.360+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/1143135530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 0x564f376e5090 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3a88005c40 tx=0x7f3a88013f50 comp rx=0 tx=0).stop 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.360+0000 7f3aa27ab740 1 -- 172.21.15.5:0/1143135530 shutdown_connections 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.360+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/1143135530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376f0370 0x564f376f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.360+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/1143135530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376e55d0 0x564f376e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.360+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/1143135530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 0x564f376e5090 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.360+0000 7f3aa27ab740 1 -- 172.21.15.5:0/1143135530 >> 172.21.15.5:0/1143135530 conn(0x564f376ae310 msgr2=0x564f376e1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.360+0000 7f3aa27ab740 1 -- 172.21.15.5:0/1143135530 shutdown_connections 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.360+0000 7f3aa27ab740 1 -- 172.21.15.5:0/1143135530 wait complete. 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.360+0000 7f3aa27ab740 1 -- start start 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3aa27ab740 1 Processor -- start 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 0x564f376efd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376ecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376ed5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f376f3010 con 0x564f376e6b40 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564f37606060 con 0x564f376f0370 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x564f376f3190 con 0x564f376e55d0 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3aa27ab740 10 client.0 _ll_register_callbacks cb 0x564f373598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9d96d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376ecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9e16e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 0x564f376efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9d96d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376ecff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57470/0 (socket says 172.21.15.5:57470) 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9e96f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376ed5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9d96d640 1 -- 172.21.15.5:0/314781438 learned_addr learned my addr 172.21.15.5:0/314781438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9d96d640 1 -- 172.21.15.5:0/314781438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 msgr2=0x564f376efd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 0x564f376efd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9d96d640 1 -- 172.21.15.5:0/314781438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 msgr2=0x564f376ed5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376ed5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9d96d640 1 -- 172.21.15.5:0/314781438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f373bc3c0 con 0x564f376e6b40 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.361+0000 7f3a9e16e640 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 0x564f376efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376ecff0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0ed260 tx=0x7f3a8c03b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66af3fb75c44c178 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3a7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3a7dffb640 1 -- 172.21.15.5:0/314781438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c049070 con 0x564f376e6b40 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3a7dffb640 1 -- 172.21.15.5:0/314781438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f3a8c0b86a0 con 0x564f376e6b40 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3aa27ab740 1 -- 172.21.15.5:0/314781438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a88022020 con 0x564f376e6b40 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3a7dffb640 1 -- 172.21.15.5:0/314781438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3a8c031de0 con 0x564f376e6b40 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3aa27ab740 10 client.20961 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3aa27ab740 1 -- 172.21.15.5:0/314781438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564f376fea80 con 0x564f376e6b40 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3a7dffb640 1 -- 172.21.15.5:0/314781438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8c0d6020 con 0x564f376e6b40 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3a7dffb640 1 -- 172.21.15.5:0/314781438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f3a8c0c4760 con 0x564f376e6b40 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3aa27ab740 10 client.20961 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.362+0000 7f3aa27ab740 10 client.20961 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:40:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.363+0000 7f3aa27ab740 20 client.20961 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.363+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x564f376fff10 0x564f37702320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.363+0000 7f3aa27ab740 4 client.20961 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.363+0000 7f3aa27ab740 1 -- 172.21.15.5:0/314781438 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f37658860 con 0x564f376fff10 2024-09-11T22:40:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.363+0000 7f3a9e16e640 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x564f376fff10 0x564f37702320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.363+0000 7f3a9e16e640 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x564f376fff10 0x564f37702320 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7cd3b3186a130b7 server_cookie=337e3ee2fcad0ebc in_seq=0 out_seq=0 2024-09-11T22:40:13.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.363+0000 7f3a7dffb640 10 client.20961 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:40:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.364+0000 7f3a7dffb640 1 -- 172.21.15.5:0/314781438 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f37658860 con 0x564f376fff10 2024-09-11T22:40:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.364+0000 7f3a7dffb640 10 client.20961 handle_command_reply: tid=0 2024-09-11T22:40:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.364+0000 7f3aa27ab740 1 client.20961 shutdown 2024-09-11T22:40:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.364+0000 7f3aa27ab740 1 -- 172.21.15.5:0/314781438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 msgr2=0x564f376ecff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.364+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376ecff0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0ed260 tx=0x7f3a8c03b6b0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.365+0000 7f3aa27ab740 1 -- 172.21.15.5:0/314781438 shutdown_connections 2024-09-11T22:40:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.365+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x564f376fff10 0x564f37702320 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.365+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376ed5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.365+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376ecff0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.365+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/314781438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 0x564f376efd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.365+0000 7f3aa27ab740 1 -- 172.21.15.5:0/314781438 >> 172.21.15.5:0/314781438 conn(0x564f376ae310 msgr2=0x564f376e7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.365+0000 7f3a7dffb640 0 client.20961 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:40:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.365+0000 7f3aa27ab740 1 -- 172.21.15.5:0/314781438 shutdown_connections 2024-09-11T22:40:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.365+0000 7f3aa27ab740 1 -- 172.21.15.5:0/314781438 wait complete. 2024-09-11T22:40:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.365+0000 7f3aa27ab740 20 client.20961 trim_cache size 0 max 16384 2024-09-11T22:40:13.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3aa27ab740 1 Processor -- start 2024-09-11T22:40:13.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3aa27ab740 1 -- start start 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e55d0 0x564f376eef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 0x564f376e8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376e8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981b4200 con 0x564f376e55d0 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981b5d40 con 0x564f376f0370 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981b48d0 con 0x564f376e6b40 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3a9e16e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e55d0 0x564f376eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3a9e16e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e55d0 0x564f376eef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57484/0 (socket says 172.21.15.5:57484) 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3a9e16e640 1 -- 172.21.15.5:0/2368629945 learned_addr learned my addr 172.21.15.5:0/2368629945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/2368629945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 0x564f376e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.380+0000 7f3a9e96f640 1 --2- 172.21.15.5:0/2368629945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.381+0000 7f3a9e16e640 1 -- 172.21.15.5:0/2368629945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 msgr2=0x564f376e8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.381+0000 7f3a9e16e640 1 --2- 172.21.15.5:0/2368629945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 0x564f376e8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.381+0000 7f3a9e16e640 1 -- 172.21.15.5:0/2368629945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 msgr2=0x564f376e8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.381+0000 7f3a9e16e640 1 --2- 172.21.15.5:0/2368629945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376e8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.381+0000 7f3a9e16e640 1 -- 172.21.15.5:0/2368629945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f376ef960 con 0x564f376e55d0 2024-09-11T22:40:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.381+0000 7f3a9e16e640 1 --2- 172.21.15.5:0/2368629945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e55d0 0x564f376eef60 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f3a900129b0 tx=0x7f3a90005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ce67bb76c0b0cf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.381+0000 7f3a7ffff640 1 -- 172.21.15.5:0/2368629945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90012310 con 0x564f376e55d0 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.381+0000 7f3a7ffff640 1 -- 172.21.15.5:0/2368629945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a9000f920 con 0x564f376e55d0 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.381+0000 7f3a7ffff640 1 -- 172.21.15.5:0/2368629945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90003a80 con 0x564f376e55d0 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2368629945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e55d0 msgr2=0x564f376eef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/2368629945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e55d0 0x564f376eef60 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f3a900129b0 tx=0x7f3a90005f90 comp rx=0 tx=0).stop 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2368629945 shutdown_connections 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/2368629945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376e8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/2368629945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e6b40 0x564f376e8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/2368629945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e55d0 0x564f376eef60 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2368629945 >> 172.21.15.5:0/2368629945 conn(0x564f376ae310 msgr2=0x564f376ee850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2368629945 shutdown_connections 2024-09-11T22:40:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2368629945 wait complete. 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 -- start start 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.382+0000 7f3aa27ab740 1 Processor -- start 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 0x564f376f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376e9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3aa27ab740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981b5670 con 0x564f376e6b40 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981b4fa0 con 0x564f376f0370 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3aa27ab740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3a981b2200 con 0x564f376e55d0 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3aa27ab740 10 client.0 _ll_register_callbacks cb 0x7f3a8c005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9e16e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 0x564f376f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9d96d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376e9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9d96d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376e9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57498/0 (socket says 172.21.15.5:57498) 2024-09-11T22:40:13.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9d96d640 1 -- 172.21.15.5:0/2575205708 learned_addr learned my addr 172.21.15.5:0/2575205708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9e16e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 0x564f376f8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33600/0 (socket says 172.21.15.5:33600) 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9e96f640 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9d96d640 1 -- 172.21.15.5:0/2575205708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 msgr2=0x564f376f8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 0x564f376f8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9d96d640 1 -- 172.21.15.5:0/2575205708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 msgr2=0x564f376ea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376ea210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9d96d640 1 -- 172.21.15.5:0/2575205708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f376ea9e0 con 0x564f376e6b40 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a9d96d640 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376e9cd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0ed7b0 tx=0x7f3a8c0b88d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06611759872de2f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.383+0000 7f3a7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3a7dffb640 1 -- 172.21.15.5:0/2575205708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c049040 con 0x564f376e6b40 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3a7dffb640 1 -- 172.21.15.5:0/2575205708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f3a8c0b6ba0 con 0x564f376e6b40 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2575205708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a9000f920 con 0x564f376e6b40 2024-09-11T22:40:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3a7dffb640 1 -- 172.21.15.5:0/2575205708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3a8c0c4bf0 con 0x564f376e6b40 2024-09-11T22:40:13.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3aa27ab740 10 client.20967 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:13.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2575205708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x564f3776aba0 con 0x564f376e6b40 2024-09-11T22:40:13.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3a7dffb640 1 -- 172.21.15.5:0/2575205708 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8c0ca070 con 0x564f376e6b40 2024-09-11T22:40:13.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3a7dffb640 1 -- 172.21.15.5:0/2575205708 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f3a8c0bb480 con 0x564f376e6b40 2024-09-11T22:40:13.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3aa27ab740 10 client.20967 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:13.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3aa27ab740 10 client.20967 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:40:13.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3aa27ab740 20 client.20967 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:13.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x564f3776b1c0 0x564f3776d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:13.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3aa27ab740 4 client.20967 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:40:13.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.384+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2575205708 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f37652d70 con 0x564f3776b1c0 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.385+0000 7f3a9e16e640 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x564f3776b1c0 0x564f3776d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.385+0000 7f3a9e16e640 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x564f3776b1c0 0x564f3776d5e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac784d06ea339720 server_cookie=d429126f5851ccd8 in_seq=0 out_seq=0 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.385+0000 7f3a7dffb640 10 client.20967 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.385+0000 7f3a7dffb640 1 -- 172.21.15.5:0/2575205708 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f37652d70 con 0x564f3776b1c0 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.385+0000 7f3a7dffb640 10 client.20967 handle_command_reply: tid=0 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.385+0000 7f3aa27ab740 1 client.20967 shutdown 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.385+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2575205708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 msgr2=0x564f376e9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.385+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376e9cd0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0ed7b0 tx=0x7f3a8c0b88d0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.385+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2575205708 shutdown_connections 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x564f3776b1c0 0x564f3776d5e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f376f0370 0x564f376ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f376e6b40 0x564f376e9cd0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa27ab740 1 --2- 172.21.15.5:0/2575205708 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564f376e55d0 0x564f376f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2575205708 >> 172.21.15.5:0/2575205708 conn(0x564f376ae310 msgr2=0x564f376ef070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:13.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3a7dffb640 0 client.20967 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:40:13.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2575205708 shutdown_connections 2024-09-11T22:40:13.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa27ab740 1 -- 172.21.15.5:0/2575205708 wait complete. 2024-09-11T22:40:13.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa27ab740 20 client.20967 trim_cache size 0 max 16384 2024-09-11T22:40:13.392 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:40:13.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa03d2640 1 -- 172.21.15.5:0/1933655992 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3a800a2cf0 msgr2=0x7f3a800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:13.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa03d2640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3a800a2cf0 0x7f3a800a51b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3a8801d000 tx=0x7f3a8801f000 comp rx=0 tx=0).stop 2024-09-11T22:40:13.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa03d2640 1 -- 172.21.15.5:0/1933655992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a98103760 msgr2=0x7f3a98100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:13.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.386+0000 7f3aa03d2640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a98103760 0x7f3a98100f50 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c005940 tx=0x7f3a8c003c10 comp rx=0 tx=0).stop 2024-09-11T22:40:13.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.387+0000 7f3aa03d2640 1 -- 172.21.15.5:0/1933655992 shutdown_connections 2024-09-11T22:40:13.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.387+0000 7f3aa03d2640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a981082e0 0x7f3a98101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.387+0000 7f3aa03d2640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a98103760 0x7f3a98100f50 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.387+0000 7f3aa03d2640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3a800a2cf0 0x7f3a800a51b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.387+0000 7f3aa03d2640 1 --2- 172.21.15.5:0/1933655992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3a98102db0 0x7f3a98100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:13.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.387+0000 7f3aa03d2640 1 -- 172.21.15.5:0/1933655992 >> 172.21.15.5:0/1933655992 conn(0x7f3a980fc7d0 msgr2=0x7f3a98105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:13.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.387+0000 7f3aa03d2640 1 -- 172.21.15.5:0/1933655992 shutdown_connections 2024-09-11T22:40:13.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:13.387+0000 7f3aa03d2640 1 -- 172.21.15.5:0/1933655992 wait complete. 2024-09-11T22:40:13.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69137 15.82 MB/sec execute 151 sec latency 2515.735 ms 2024-09-11T22:40:13.991 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:13 smithi005 ceph-mon[30943]: pgmap v574: 97 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 68 active+clean; 512 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 558 op/s 2024-09-11T22:40:14.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:13 smithi159 ceph-mon[35534]: pgmap v574: 97 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 68 active+clean; 512 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 558 op/s 2024-09-11T22:40:14.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:13 smithi172 ceph-mon[35461]: pgmap v574: 97 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 68 active+clean; 512 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.0 MiB/s wr, 558 op/s 2024-09-11T22:40:14.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:40:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:14.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69494 15.84 MB/sec execute 152 sec latency 3515.765 ms 2024-09-11T22:40:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.736+0000 7f719dffd640 1 -- 172.21.15.5:0/1054569130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7198112800 msgr2=0x7f7198114bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.736+0000 7f719dffd640 1 --2- 172.21.15.5:0/1054569130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7198112800 0x7f7198114bf0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f7190009ca0 tx=0x7f719002a950 comp rx=0 tx=0).stop 2024-09-11T22:40:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.736+0000 7f719dffd640 1 -- 172.21.15.5:0/1054569130 shutdown_connections 2024-09-11T22:40:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.736+0000 7f719dffd640 1 --2- 172.21.15.5:0/1054569130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7198112800 0x7f7198114bf0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.736+0000 7f719dffd640 1 --2- 172.21.15.5:0/1054569130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7198100490 0x7f71981122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.736+0000 7f719dffd640 1 --2- 172.21.15.5:0/1054569130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71980ffb70 0x7f71980fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.736+0000 7f719dffd640 1 -- 172.21.15.5:0/1054569130 >> 172.21.15.5:0/1054569130 conn(0x7f71980fba30 msgr2=0x7f71980fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.737+0000 7f719dffd640 1 -- 172.21.15.5:0/1054569130 shutdown_connections 2024-09-11T22:40:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.737+0000 7f719dffd640 1 -- 172.21.15.5:0/1054569130 wait complete. 2024-09-11T22:40:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.739+0000 7f719dffd640 1 Processor -- start 2024-09-11T22:40:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.739+0000 7f719dffd640 1 -- start start 2024-09-11T22:40:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.739+0000 7f719dffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71980ffb70 0x7f71980739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.739+0000 7f719dffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7198100490 0x7f719806da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.739+0000 7f719dffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7198112800 0x7f719806df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.739+0000 7f719dffd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f719806c660 con 0x7f71980ffb70 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.739+0000 7f719dffd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f719806c4e0 con 0x7f7198100490 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.739+0000 7f719dffd640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f719806c7e0 con 0x7f7198112800 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f71977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71980ffb70 0x7f71980739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f71977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71980ffb70 0x7f71980739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57510/0 (socket says 172.21.15.5:57510) 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f71977fe640 1 -- 172.21.15.5:0/4115665409 learned_addr learned my addr 172.21.15.5:0/4115665409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f7196ffd640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7198100490 0x7f719806da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f7197fff640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7198112800 0x7f719806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f7196ffd640 1 -- 172.21.15.5:0/4115665409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7198112800 msgr2=0x7f719806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:14.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f7196ffd640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7198112800 0x7f719806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f7196ffd640 1 -- 172.21.15.5:0/4115665409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71980ffb70 msgr2=0x7f71980739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f7196ffd640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71980ffb70 0x7f71980739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f7196ffd640 1 -- 172.21.15.5:0/4115665409 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719806e530 con 0x7f7198100490 2024-09-11T22:40:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.740+0000 7f7196ffd640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7198100490 0x7f719806da30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f718c00b3f0 tx=0x7f718c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.741+0000 7f7194ff9640 1 -- 172.21.15.5:0/4115665409 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c015020 con 0x7f7198100490 2024-09-11T22:40:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.741+0000 7f7194ff9640 1 -- 172.21.15.5:0/4115665409 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718c011d10 con 0x7f7198100490 2024-09-11T22:40:14.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.741+0000 7f7194ff9640 1 -- 172.21.15.5:0/4115665409 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c00f2e0 con 0x7f7198100490 2024-09-11T22:40:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.741+0000 7f719dffd640 1 -- 172.21.15.5:0/4115665409 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719806e820 con 0x7f7198100490 2024-09-11T22:40:14.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.741+0000 7f719dffd640 1 -- 172.21.15.5:0/4115665409 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71981b3b80 con 0x7f7198100490 2024-09-11T22:40:14.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.744+0000 7f71a0e80740 1 Processor -- start 2024-09-11T22:40:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f71a0e80740 1 -- start start 2024-09-11T22:40:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f7194ff9640 1 -- 172.21.15.5:0/4115665409 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f718c018020 con 0x7f7198100490 2024-09-11T22:40:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a71af0 0x55e629a71ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 0x55e629a7f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f71a0e80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e6299d62f0 con 0x55e629a7ccb0 2024-09-11T22:40:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f71a0e80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e62997e270 con 0x55e629a72410 2024-09-11T22:40:14.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f71a0e80740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e6299d22b0 con 0x55e629a71af0 2024-09-11T22:40:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f7196ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 0x55e629a7f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f7196ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 0x55e629a7f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57518/0 (socket says 172.21.15.5:57518) 2024-09-11T22:40:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f7196ffd640 1 -- 172.21.15.5:0/979904845 learned_addr learned my addr 172.21.15.5:0/979904845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:14.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.745+0000 7f71977fe640 1 --2- 172.21.15.5:0/979904845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.746+0000 7f7197fff640 1 --2- 172.21.15.5:0/979904845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a71af0 0x55e629a71ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.746+0000 7f71977fe640 1 -- 172.21.15.5:0/979904845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a71af0 msgr2=0x55e629a71ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.746+0000 7f71977fe640 1 --2- 172.21.15.5:0/979904845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a71af0 0x55e629a71ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.746+0000 7f71977fe640 1 -- 172.21.15.5:0/979904845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 msgr2=0x55e629a7f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:14.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.746+0000 7f71977fe640 1 --2- 172.21.15.5:0/979904845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 0x55e629a7f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.746+0000 7f71977fe640 1 -- 172.21.15.5:0/979904845 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e62997d340 con 0x55e629a72410 2024-09-11T22:40:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.746+0000 7f7194ff9640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f71800a2ef0 0x7f71800a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.746+0000 7f71977fe640 1 --2- 172.21.15.5:0/979904845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7c770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f718800bcb0 tx=0x7f71880107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9940b3036850f241 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.746+0000 7f7197fff640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f71800a2ef0 0x7f71800a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.746+0000 7f7194ff9640 1 -- 172.21.15.5:0/4115665409 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f718c0ceb40 con 0x7f7198100490 2024-09-11T22:40:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.747+0000 7f7197fff640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f71800a2ef0 0x7f71800a53b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7190009a60 tx=0x7f7190005d80 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.747+0000 7f717dffb640 1 -- 172.21.15.5:0/979904845 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718800be60 con 0x55e629a72410 2024-09-11T22:40:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.747+0000 7f717dffb640 1 -- 172.21.15.5:0/979904845 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71880047a0 con 0x55e629a72410 2024-09-11T22:40:14.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.747+0000 7f717dffb640 1 -- 172.21.15.5:0/979904845 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7188011800 con 0x55e629a72410 2024-09-11T22:40:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.748+0000 7f71a0e80740 1 -- 172.21.15.5:0/979904845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 msgr2=0x55e629a7c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.748+0000 7f71a0e80740 1 --2- 172.21.15.5:0/979904845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7c770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f718800bcb0 tx=0x7f71880107a0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.748+0000 7f71a0e80740 1 -- 172.21.15.5:0/979904845 shutdown_connections 2024-09-11T22:40:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.748+0000 7f71a0e80740 1 --2- 172.21.15.5:0/979904845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 0x55e629a7f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.748+0000 7f71a0e80740 1 --2- 172.21.15.5:0/979904845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7c770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.748+0000 7f71a0e80740 1 --2- 172.21.15.5:0/979904845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a71af0 0x55e629a71ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.748+0000 7f71a0e80740 1 -- 172.21.15.5:0/979904845 >> 172.21.15.5:0/979904845 conn(0x55e629a6e060 msgr2=0x55e629a6e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.748+0000 7f71a0e80740 1 -- 172.21.15.5:0/979904845 shutdown_connections 2024-09-11T22:40:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.748+0000 7f71a0e80740 1 -- 172.21.15.5:0/979904845 wait complete. 2024-09-11T22:40:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.748+0000 7f71a0e80740 1 -- start start 2024-09-11T22:40:14.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.749+0000 7f71a0e80740 1 Processor -- start 2024-09-11T22:40:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.750+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a71af0 0x55e629a79c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.750+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.750+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a7ccb0 0x55e629a7a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.750+0000 7f71a0e80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e629a7f880 con 0x55e629a71af0 2024-09-11T22:40:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.750+0000 7f71a0e80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e62995f290 con 0x55e629a72410 2024-09-11T22:40:14.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.750+0000 7f71a0e80740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55e629a7fa00 con 0x55e629a7ccb0 2024-09-11T22:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.750+0000 7f7196ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.750+0000 7f7196ffd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42224/0 (socket says 172.21.15.5:42224) 2024-09-11T22:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.750+0000 7f7196ffd640 1 -- 172.21.15.5:0/1225452692 learned_addr learned my addr 172.21.15.5:0/1225452692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.750+0000 7f71a0e80740 10 client.0 _ll_register_callbacks cb 0x55e629990140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.751+0000 7f7196ffd640 1 -- 172.21.15.5:0/1225452692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a7ccb0 msgr2=0x55e629a7a6e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.751+0000 7f7196ffd640 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a7ccb0 0x55e629a7a6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.751+0000 7f7196ffd640 1 -- 172.21.15.5:0/1225452692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a71af0 msgr2=0x55e629a79c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.751+0000 7f7196ffd640 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a71af0 0x55e629a79c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.751+0000 7f7196ffd640 1 -- 172.21.15.5:0/1225452692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6296e9210 con 0x55e629a72410 2024-09-11T22:40:14.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.751+0000 7f7196ffd640 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7a1a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f718c0a37b0 tx=0x7f718c097390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=678a21111cdaf45d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.752+0000 7f7176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.752+0000 7f7176ffd640 1 -- 172.21.15.5:0/1225452692 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c009200 con 0x55e629a72410 2024-09-11T22:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.752+0000 7f7176ffd640 1 -- 172.21.15.5:0/1225452692 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f718c0cc410 con 0x55e629a72410 2024-09-11T22:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.752+0000 7f71a0e80740 1 -- 172.21.15.5:0/1225452692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71880047a0 con 0x55e629a72410 2024-09-11T22:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.752+0000 7f7176ffd640 1 -- 172.21.15.5:0/1225452692 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f718c010eb0 con 0x55e629a72410 2024-09-11T22:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.752+0000 7f71a0e80740 10 client.30002 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.753+0000 7f71a0e80740 1 -- 172.21.15.5:0/1225452692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e6297b5620 con 0x55e629a72410 2024-09-11T22:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.753+0000 7f7176ffd640 1 -- 172.21.15.5:0/1225452692 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f718c09b770 con 0x55e629a72410 2024-09-11T22:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.753+0000 7f7176ffd640 1 -- 172.21.15.5:0/1225452692 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f718c0976e0 con 0x55e629a72410 2024-09-11T22:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.754+0000 7f71a0e80740 10 client.30002 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.754+0000 7f71a0e80740 10 client.30002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.754+0000 7f71a0e80740 20 client.30002 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.754+0000 7f71a0e80740 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e629a8cca0 0x55e629a8f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.754+0000 7f71a0e80740 4 client.30002 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:14.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.754+0000 7f71a0e80740 1 -- 172.21.15.5:0/1225452692 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f71880047a0 con 0x55e629a8cca0 2024-09-11T22:40:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.838+0000 7f71977fe640 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e629a8cca0 0x55e629a8f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.839+0000 7f7176ffd640 10 client.30002 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.839+0000 7f71977fe640 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e629a8cca0 0x55e629a8f110 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adede4a282c5fe12 server_cookie=741dac903fe3929f in_seq=0 out_seq=0 2024-09-11T22:40:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f7176ffd640 1 -- 172.21.15.5:0/1225452692 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f71880047a0 con 0x55e629a8cca0 2024-09-11T22:40:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f7176ffd640 10 client.30002 handle_command_reply: tid=0 2024-09-11T22:40:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 client.30002 shutdown 2024-09-11T22:40:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 -- 172.21.15.5:0/1225452692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 msgr2=0x55e629a7a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7a1a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f718c0a37b0 tx=0x7f718c097390 comp rx=0 tx=0).stop 2024-09-11T22:40:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 -- 172.21.15.5:0/1225452692 shutdown_connections 2024-09-11T22:40:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e629a8cca0 0x55e629a8f110 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a7ccb0 0x55e629a7a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a72410 0x55e629a7a1a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 --2- 172.21.15.5:0/1225452692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a71af0 0x55e629a79c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f7176ffd640 0 client.30002 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 -- 172.21.15.5:0/1225452692 >> 172.21.15.5:0/1225452692 conn(0x55e629a6e060 msgr2=0x55e629a73d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 -- 172.21.15.5:0/1225452692 shutdown_connections 2024-09-11T22:40:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.844+0000 7f71a0e80740 1 -- 172.21.15.5:0/1225452692 wait complete. 2024-09-11T22:40:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.845+0000 7f71a0e80740 20 client.30002 trim_cache size 0 max 16384 2024-09-11T22:40:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.864+0000 7f71a0e80740 1 Processor -- start 2024-09-11T22:40:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f71a0e80740 1 -- start start 2024-09-11T22:40:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a74560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a72410 0x55e629a7a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 0x55e629a94a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f71a0e80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71981bca50 con 0x55e629a7ccb0 2024-09-11T22:40:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f71a0e80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f71981be590 con 0x55e629a71af0 2024-09-11T22:40:14.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f71a0e80740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f71981bd120 con 0x55e629a72410 2024-09-11T22:40:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f71977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a74560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f7197fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 0x55e629a94a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f71977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a74560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42234/0 (socket says 172.21.15.5:42234) 2024-09-11T22:40:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f71977fe640 1 -- 172.21.15.5:0/752825378 learned_addr learned my addr 172.21.15.5:0/752825378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.865+0000 7f7196ffd640 1 --2- 172.21.15.5:0/752825378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a72410 0x55e629a7a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.866+0000 7f71977fe640 1 -- 172.21.15.5:0/752825378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a72410 msgr2=0x55e629a7a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.866+0000 7f71977fe640 1 --2- 172.21.15.5:0/752825378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a72410 0x55e629a7a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.866+0000 7f71977fe640 1 -- 172.21.15.5:0/752825378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 msgr2=0x55e629a94a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.866+0000 7f71977fe640 1 --2- 172.21.15.5:0/752825378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 0x55e629a94a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.866+0000 7f71977fe640 1 -- 172.21.15.5:0/752825378 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e629a71920 con 0x55e629a71af0 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.866+0000 7f7196ffd640 1 --2- 172.21.15.5:0/752825378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a72410 0x55e629a7a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.866+0000 7f71977fe640 1 --2- 172.21.15.5:0/752825378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a74560 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55e629a6eec0 tx=0x7f718800b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d16467dce854719 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.866+0000 7f717d7fa640 1 -- 172.21.15.5:0/752825378 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71880051c0 con 0x55e629a71af0 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.866+0000 7f717d7fa640 1 -- 172.21.15.5:0/752825378 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7188004b40 con 0x55e629a71af0 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.866+0000 7f717d7fa640 1 -- 172.21.15.5:0/752825378 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71880277f0 con 0x55e629a71af0 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.867+0000 7f71a0e80740 1 -- 172.21.15.5:0/752825378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 msgr2=0x55e629a74560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.867+0000 7f71a0e80740 1 --2- 172.21.15.5:0/752825378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a74560 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55e629a6eec0 tx=0x7f718800b9a0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.867+0000 7f71a0e80740 1 -- 172.21.15.5:0/752825378 shutdown_connections 2024-09-11T22:40:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.867+0000 7f71a0e80740 1 --2- 172.21.15.5:0/752825378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a7ccb0 0x55e629a94a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.867+0000 7f71a0e80740 1 --2- 172.21.15.5:0/752825378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a72410 0x55e629a7a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.867+0000 7f71a0e80740 1 --2- 172.21.15.5:0/752825378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a74560 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.867+0000 7f71a0e80740 1 -- 172.21.15.5:0/752825378 >> 172.21.15.5:0/752825378 conn(0x55e629a6e060 msgr2=0x55e629a7bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.867+0000 7f71a0e80740 1 -- 172.21.15.5:0/752825378 shutdown_connections 2024-09-11T22:40:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.867+0000 7f71a0e80740 1 -- 172.21.15.5:0/752825378 wait complete. 2024-09-11T22:40:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.867+0000 7f71a0e80740 1 -- start start 2024-09-11T22:40:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.868+0000 7f71a0e80740 1 Processor -- start 2024-09-11T22:40:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.868+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a81170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.868+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a72410 0x55e629a816b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.868+0000 7f71a0e80740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a7ccb0 0x55e629af69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.868+0000 7f71a0e80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71981bdec0 con 0x55e629a72410 2024-09-11T22:40:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.868+0000 7f71a0e80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f71981bd7f0 con 0x55e629a71af0 2024-09-11T22:40:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.868+0000 7f71a0e80740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f71981baa50 con 0x55e629a7ccb0 2024-09-11T22:40:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.868+0000 7f71a0e80740 10 client.0 _ll_register_callbacks cb 0x7f718c010380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.869+0000 7f71977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a81170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.869+0000 7f71977fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a81170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42242/0 (socket says 172.21.15.5:42242) 2024-09-11T22:40:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.869+0000 7f71977fe640 1 -- 172.21.15.5:0/3172732056 learned_addr learned my addr 172.21.15.5:0/3172732056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.869+0000 7f7197fff640 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a7ccb0 0x55e629af69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.869+0000 7f7197fff640 1 -- 172.21.15.5:0/3172732056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 msgr2=0x55e629a81170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.869+0000 7f7197fff640 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a81170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.870+0000 7f7197fff640 1 -- 172.21.15.5:0/3172732056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a72410 msgr2=0x55e629a816b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.870+0000 7f7197fff640 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a72410 0x55e629a816b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.870+0000 7f7197fff640 1 -- 172.21.15.5:0/3172732056 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e629a7aa50 con 0x55e629a7ccb0 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.870+0000 7f7197fff640 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a7ccb0 0x55e629af69f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7190005e30 tx=0x7f7190005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a9ad48f8bb368e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.870+0000 7f7176ffd640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.870+0000 7f7176ffd640 1 -- 172.21.15.5:0/3172732056 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190034040 con 0x55e629a7ccb0 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.870+0000 7f71a0e80740 1 -- 172.21.15.5:0/3172732056 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7188004b40 con 0x55e629a7ccb0 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.870+0000 7f7176ffd640 1 -- 172.21.15.5:0/3172732056 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f7190004250 con 0x55e629a7ccb0 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.870+0000 7f7176ffd640 1 -- 172.21.15.5:0/3172732056 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f7190004450 con 0x55e629a7ccb0 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.871+0000 7f71a0e80740 10 client.29824 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.871+0000 7f71a0e80740 1 -- 172.21.15.5:0/3172732056 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e629a7c7f0 con 0x55e629a7ccb0 2024-09-11T22:40:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.871+0000 7f7176ffd640 1 -- 172.21.15.5:0/3172732056 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7188004b40 con 0x55e629a7ccb0 2024-09-11T22:40:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.872+0000 7f7176ffd640 1 -- 172.21.15.5:0/3172732056 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f7190048430 con 0x55e629a7ccb0 2024-09-11T22:40:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.872+0000 7f71a0e80740 10 client.29824 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.872+0000 7f71a0e80740 10 client.29824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.872+0000 7f71a0e80740 20 client.29824 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.872+0000 7f71a0e80740 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e629af85e0 0x55e629afaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.872+0000 7f71a0e80740 4 client.29824 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:40:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.872+0000 7f71a0e80740 1 -- 172.21.15.5:0/3172732056 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6299e3a20 con 0x55e629af85e0 2024-09-11T22:40:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.872+0000 7f71977fe640 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e629af85e0 0x55e629afaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.872+0000 7f71977fe640 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e629af85e0 0x55e629afaa40 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4dfa51c1a26caff server_cookie=31ebed42289c1195 in_seq=0 out_seq=0 2024-09-11T22:40:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:14.872+0000 7f7176ffd640 10 client.29824 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.977+0000 7f278ae38640 1 -- 172.21.15.5:0/1108670181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27841008d0 msgr2=0x7f2784100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.977+0000 7f278ae38640 1 --2- 172.21.15.5:0/1108670181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27841008d0 0x7f2784100cb0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f2778009900 tx=0x7f277802a410 comp rx=0 tx=0).stop 2024-09-11T22:40:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.987+0000 7f278ae38640 1 -- 172.21.15.5:0/1108670181 shutdown_connections 2024-09-11T22:40:14.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.987+0000 7f278ae38640 1 --2- 172.21.15.5:0/1108670181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278410e480 0x7f2784110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.987+0000 7f278ae38640 1 --2- 172.21.15.5:0/1108670181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2784101280 0x7f278410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.987+0000 7f278ae38640 1 --2- 172.21.15.5:0/1108670181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27841008d0 0x7f2784100cb0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.987+0000 7f278ae38640 1 -- 172.21.15.5:0/1108670181 >> 172.21.15.5:0/1108670181 conn(0x7f27840fc7a0 msgr2=0x7f27840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.988+0000 7f278ae38640 1 -- 172.21.15.5:0/1108670181 shutdown_connections 2024-09-11T22:40:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.989+0000 7f278ae38640 1 -- 172.21.15.5:0/1108670181 wait complete. 2024-09-11T22:40:14.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.989+0000 7f278ae38640 1 Processor -- start 2024-09-11T22:40:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.992+0000 7f278ae38640 1 -- start start 2024-09-11T22:40:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.993+0000 7f278ae38640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27841008d0 0x7f2784102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.993+0000 7f278ae38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2784101280 0x7f27841030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.993+0000 7f278ae38640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278410e480 0x7f2784103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.993+0000 7f278ae38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27841141b0 con 0x7f2784101280 2024-09-11T22:40:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.994+0000 7f278ae38640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2784114030 con 0x7f278410e480 2024-09-11T22:40:14.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.994+0000 7f278ae38640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2784114330 con 0x7f27841008d0 2024-09-11T22:40:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.994+0000 7f2788bd4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27841008d0 0x7f2784102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.994+0000 7f2788bd4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27841008d0 0x7f2784102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33656/0 (socket says 172.21.15.5:33656) 2024-09-11T22:40:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.994+0000 7f2788bd4640 1 -- 172.21.15.5:0/23348012 learned_addr learned my addr 172.21.15.5:0/23348012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:14.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.995+0000 7f27893d5640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278410e480 0x7f2784103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.999+0000 7f27893d5640 1 -- 172.21.15.5:0/23348012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27841008d0 msgr2=0x7f2784102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.999+0000 7f27893d5640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27841008d0 0x7f2784102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.999+0000 7f27893d5640 1 -- 172.21.15.5:0/23348012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2784101280 msgr2=0x7f27841030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:15.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.999+0000 7f27893d5640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2784101280 0x7f27841030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:14.999+0000 7f27893d5640 1 -- 172.21.15.5:0/23348012 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2784072b30 con 0x7f278410e480 2024-09-11T22:40:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.000+0000 7f27893d5640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278410e480 0x7f2784103630 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2784102ab0 tx=0x7f277c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.000+0000 7f2781ffb640 1 -- 172.21.15.5:0/23348012 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c00cdf0 con 0x7f278410e480 2024-09-11T22:40:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.001+0000 7f2781ffb640 1 -- 172.21.15.5:0/23348012 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f277c005020 con 0x7f278410e480 2024-09-11T22:40:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.001+0000 7f2781ffb640 1 -- 172.21.15.5:0/23348012 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c010690 con 0x7f278410e480 2024-09-11T22:40:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.001+0000 7f278ae38640 1 -- 172.21.15.5:0/23348012 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2784072e20 con 0x7f278410e480 2024-09-11T22:40:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.001+0000 7f278ae38640 1 -- 172.21.15.5:0/23348012 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2784073310 con 0x7f278410e480 2024-09-11T22:40:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.007+0000 7f2781ffb640 1 -- 172.21.15.5:0/23348012 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f277c0054d0 con 0x7f278410e480 2024-09-11T22:40:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.008+0000 7f2781ffb640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27740a2ec0 0x7f27740a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.008+0000 7f2781ffb640 1 -- 172.21.15.5:0/23348012 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f277c0cf9f0 con 0x7f278410e480 2024-09-11T22:40:15.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.012+0000 7f2788bd4640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27740a2ec0 0x7f27740a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.018+0000 7f2788bd4640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27740a2ec0 0x7f27740a5380 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f27780096c0 tx=0x7f2778005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:15.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.019+0000 7f278d211740 1 Processor -- start 2024-09-11T22:40:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.020+0000 7f278d211740 1 -- start start 2024-09-11T22:40:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.020+0000 7f278d211740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f21b40 0x562181f20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.020+0000 7f278d211740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f20a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.020+0000 7f278d211740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f2b370 0x562181f2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.020+0000 7f278d211740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562181e862f0 con 0x562181f21b40 2024-09-11T22:40:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.020+0000 7f278d211740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562181e2dcf0 con 0x562181f2b370 2024-09-11T22:40:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.020+0000 7f278d211740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562181e822b0 con 0x562181f205d0 2024-09-11T22:40:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.020+0000 7f2788bd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f21b40 0x562181f20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.021+0000 7f2788bd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f21b40 0x562181f20090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57562/0 (socket says 172.21.15.5:57562) 2024-09-11T22:40:15.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.021+0000 7f2788bd4640 1 -- 172.21.15.5:0/1120437909 learned_addr learned my addr 172.21.15.5:0/1120437909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.022+0000 7f2788bd4640 1 -- 172.21.15.5:0/1120437909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 msgr2=0x562181f20a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:40:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.022+0000 7f2788bd4640 1 --2- 172.21.15.5:0/1120437909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f20a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.022+0000 7f2788bd4640 1 -- 172.21.15.5:0/1120437909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f2b370 msgr2=0x562181f2d760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:40:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.022+0000 7f2788bd4640 1 --2- 172.21.15.5:0/1120437909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f2b370 0x562181f2d760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.022+0000 7f2788bd4640 1 -- 172.21.15.5:0/1120437909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562181e123a0 con 0x562181f21b40 2024-09-11T22:40:15.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.025+0000 7f2788bd4640 1 --2- 172.21.15.5:0/1120437909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f21b40 0x562181f20090 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f2778009a20 tx=0x7f277803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4af09b12cb99ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.028+0000 7f276effd640 1 -- 172.21.15.5:0/1120437909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277804b070 con 0x562181f21b40 2024-09-11T22:40:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.028+0000 7f276effd640 1 -- 172.21.15.5:0/1120437909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27780317f0 con 0x562181f21b40 2024-09-11T22:40:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.028+0000 7f276effd640 1 -- 172.21.15.5:0/1120437909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778035050 con 0x562181f21b40 2024-09-11T22:40:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.029+0000 7f278d211740 1 -- 172.21.15.5:0/1120437909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f21b40 msgr2=0x562181f20090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:15.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.029+0000 7f278d211740 1 --2- 172.21.15.5:0/1120437909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f21b40 0x562181f20090 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f2778009a20 tx=0x7f277803c000 comp rx=0 tx=0).stop 2024-09-11T22:40:15.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.033+0000 7f278d211740 1 -- 172.21.15.5:0/1120437909 shutdown_connections 2024-09-11T22:40:15.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.033+0000 7f278d211740 1 --2- 172.21.15.5:0/1120437909 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f2b370 0x562181f2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.033+0000 7f278d211740 1 --2- 172.21.15.5:0/1120437909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f20a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.033+0000 7f278d211740 1 --2- 172.21.15.5:0/1120437909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f21b40 0x562181f20090 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.033+0000 7f278d211740 1 -- 172.21.15.5:0/1120437909 >> 172.21.15.5:0/1120437909 conn(0x562181ee9310 msgr2=0x562181f1cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:15.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.033+0000 7f278d211740 1 -- 172.21.15.5:0/1120437909 shutdown_connections 2024-09-11T22:40:15.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.033+0000 7f278d211740 1 -- 172.21.15.5:0/1120437909 wait complete. 2024-09-11T22:40:15.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.034+0000 7f278d211740 1 -- start start 2024-09-11T22:40:15.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.034+0000 7f278d211740 1 Processor -- start 2024-09-11T22:40:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.035+0000 7f278d211740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f2ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.035+0000 7f278d211740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.035+0000 7f278d211740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f2b370 0x562181f28840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.036+0000 7f2783fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.036+0000 7f2783fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f280b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42268/0 (socket says 172.21.15.5:42268) 2024-09-11T22:40:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.036+0000 7f27893d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f2b370 0x562181f28840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.036+0000 7f27893d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f2b370 0x562181f28840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57576/0 (socket says 172.21.15.5:57576) 2024-09-11T22:40:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.036+0000 7f2783fff640 1 -- 172.21.15.5:0/4166627382 learned_addr learned my addr 172.21.15.5:0/4166627382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.036+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562181f2e010 con 0x562181f2b370 2024-09-11T22:40:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.036+0000 7f2788bd4640 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f2ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.036+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562181e41060 con 0x562181f21b40 2024-09-11T22:40:15.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.037+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562181f2e190 con 0x562181f205d0 2024-09-11T22:40:15.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.037+0000 7f278d211740 10 client.0 _ll_register_callbacks cb 0x562181b948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.037+0000 7f2783fff640 1 -- 172.21.15.5:0/4166627382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 msgr2=0x562181f2ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.037+0000 7f2783fff640 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f2ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.037+0000 7f2783fff640 1 -- 172.21.15.5:0/4166627382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f2b370 msgr2=0x562181f28840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.037+0000 7f2783fff640 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f2b370 0x562181f28840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.037+0000 7f2783fff640 1 -- 172.21.15.5:0/4166627382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562181bf73c0 con 0x562181f21b40 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.038+0000 7f2788bd4640 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f2ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.038+0000 7f2783fff640 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f280b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f277000e600 tx=0x7f277000eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ebb99ab620bf9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.038+0000 7f2763fff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.038+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27780317f0 con 0x562181f21b40 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.039+0000 7f2763fff640 1 -- 172.21.15.5:0/4166627382 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770005220 con 0x562181f21b40 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.039+0000 7f2763fff640 1 -- 172.21.15.5:0/4166627382 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f27700053c0 con 0x562181f21b40 2024-09-11T22:40:15.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.039+0000 7f2763fff640 1 -- 172.21.15.5:0/4166627382 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f27700055c0 con 0x562181f21b40 2024-09-11T22:40:15.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.040+0000 7f278d211740 10 client.30017 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:15.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.041+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=54}) -- 0x562181f39b60 con 0x562181f21b40 2024-09-11T22:40:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.041+0000 7f2763fff640 1 -- 172.21.15.5:0/4166627382 <== mon.2 v2:172.21.15.159:3300/0 4 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f27700047a0 con 0x562181f21b40 2024-09-11T22:40:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.041+0000 7f278d211740 10 client.30017 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.041+0000 7f278d211740 10 client.30017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.041+0000 7f278d211740 20 client.30017 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.041+0000 7f278d211740 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562181f3af10 0x562181f3d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.042+0000 7f278d211740 4 client.30017 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.042+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562181e93860 con 0x562181f3af10 2024-09-11T22:40:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.045+0000 7f2788bd4640 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562181f3af10 0x562181f3d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.049+0000 7f2763fff640 10 client.30017 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.049+0000 7f2788bd4640 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562181f3af10 0x562181f3d300 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f138ce5fcc0a9152 server_cookie=72bd0917e64da6e0 in_seq=0 out_seq=0 2024-09-11T22:40:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.074+0000 7f2763fff640 1 -- 172.21.15.5:0/4166627382 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562181e93860 con 0x562181f3af10 2024-09-11T22:40:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.074+0000 7f2763fff640 10 client.30017 handle_command_reply: tid=0 2024-09-11T22:40:15.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.074+0000 7f278d211740 1 client.30017 shutdown 2024-09-11T22:40:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.078+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 msgr2=0x562181f280b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.078+0000 7f278d211740 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f280b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f277000e600 tx=0x7f277000eac0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.078+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 shutdown_connections 2024-09-11T22:40:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.078+0000 7f278d211740 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562181f3af10 0x562181f3d300 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.078+0000 7f278d211740 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f2b370 0x562181f28840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.078+0000 7f278d211740 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f280b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.078+0000 7f278d211740 1 --2- 172.21.15.5:0/4166627382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f2ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.078+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 >> 172.21.15.5:0/4166627382 conn(0x562181ee9310 msgr2=0x562181f22960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.078+0000 7f2763fff640 0 client.30017 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.078+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 shutdown_connections 2024-09-11T22:40:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.079+0000 7f278d211740 1 -- 172.21.15.5:0/4166627382 wait complete. 2024-09-11T22:40:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.079+0000 7f278d211740 20 client.30017 trim_cache size 0 max 16384 2024-09-11T22:40:15.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.104+0000 7f278d211740 1 Processor -- start 2024-09-11T22:40:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.104+0000 7f278d211740 1 -- start start 2024-09-11T22:40:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.104+0000 7f278d211740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f205d0 0x562181f2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.104+0000 7f278d211740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f2cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.104+0000 7f278d211740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f2b370 0x562181f23120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.104+0000 7f278d211740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b2200 con 0x562181f205d0 2024-09-11T22:40:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.104+0000 7f278d211740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27841c6d60 con 0x562181f21b40 2024-09-11T22:40:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.104+0000 7f278d211740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b28d0 con 0x562181f2b370 2024-09-11T22:40:15.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.105+0000 7f2788bd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f205d0 0x562181f2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.106+0000 7f2783fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f2cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.106+0000 7f2788bd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f205d0 0x562181f2c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57590/0 (socket says 172.21.15.5:57590) 2024-09-11T22:40:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.106+0000 7f2788bd4640 1 -- 172.21.15.5:0/2905701366 learned_addr learned my addr 172.21.15.5:0/2905701366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:15.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.106+0000 7f27893d5640 1 --2- 172.21.15.5:0/2905701366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f2b370 0x562181f23120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.109+0000 7f27893d5640 1 -- 172.21.15.5:0/2905701366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 msgr2=0x562181f2cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.109+0000 7f27893d5640 1 --2- 172.21.15.5:0/2905701366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f2cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.109+0000 7f27893d5640 1 -- 172.21.15.5:0/2905701366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f205d0 msgr2=0x562181f2c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.109+0000 7f27893d5640 1 --2- 172.21.15.5:0/2905701366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f205d0 0x562181f2c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.109+0000 7f27893d5640 1 -- 172.21.15.5:0/2905701366 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562181f2b170 con 0x562181f2b370 2024-09-11T22:40:15.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.109+0000 7f27893d5640 1 --2- 172.21.15.5:0/2905701366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f2b370 0x562181f23120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f277c0cd040 tx=0x7f277c09b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2da14fbabc6e2d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:15.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.113+0000 7f276e7fc640 1 -- 172.21.15.5:0/2905701366 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c09be10 con 0x562181f2b370 2024-09-11T22:40:15.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.113+0000 7f276e7fc640 1 -- 172.21.15.5:0/2905701366 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f277c0a01c0 con 0x562181f2b370 2024-09-11T22:40:15.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.113+0000 7f276e7fc640 1 -- 172.21.15.5:0/2905701366 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c0a4850 con 0x562181f2b370 2024-09-11T22:40:15.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.115+0000 7f278d211740 1 -- 172.21.15.5:0/2905701366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f2b370 msgr2=0x562181f23120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:15.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.116+0000 7f278d211740 1 --2- 172.21.15.5:0/2905701366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f2b370 0x562181f23120 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f277c0cd040 tx=0x7f277c09b770 comp rx=0 tx=0).stop 2024-09-11T22:40:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.118+0000 7f278d211740 1 -- 172.21.15.5:0/2905701366 shutdown_connections 2024-09-11T22:40:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.118+0000 7f278d211740 1 --2- 172.21.15.5:0/2905701366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f2b370 0x562181f23120 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.118+0000 7f278d211740 1 --2- 172.21.15.5:0/2905701366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f2cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.118+0000 7f278d211740 1 --2- 172.21.15.5:0/2905701366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f205d0 0x562181f2c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.118+0000 7f278d211740 1 -- 172.21.15.5:0/2905701366 >> 172.21.15.5:0/2905701366 conn(0x562181ee9310 msgr2=0x562181f270d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:15.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.123+0000 7f278d211740 1 -- 172.21.15.5:0/2905701366 shutdown_connections 2024-09-11T22:40:15.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.124+0000 7f278d211740 1 -- 172.21.15.5:0/2905701366 wait complete. 2024-09-11T22:40:15.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.124+0000 7f278d211740 1 -- start start 2024-09-11T22:40:15.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.124+0000 7f278d211740 1 Processor -- start 2024-09-11T22:40:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.125+0000 7f278d211740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f33440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.125+0000 7f278d211740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f33980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.125+0000 7f278d211740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f2b370 0x562181f24380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.125+0000 7f278d211740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27841c6690 con 0x562181f2b370 2024-09-11T22:40:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.125+0000 7f278d211740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27841c5fe0 con 0x562181f21b40 2024-09-11T22:40:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.125+0000 7f278d211740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f27841b0200 con 0x562181f205d0 2024-09-11T22:40:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.125+0000 7f278d211740 10 client.0 _ll_register_callbacks cb 0x7f27700095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.125+0000 7f2783fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f33980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.125+0000 7f2783fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f33980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42276/0 (socket says 172.21.15.5:42276) 2024-09-11T22:40:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.125+0000 7f2783fff640 1 -- 172.21.15.5:0/1700193520 learned_addr learned my addr 172.21.15.5:0/1700193520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.126+0000 7f2788bd4640 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f33440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.126+0000 7f2783fff640 1 -- 172.21.15.5:0/1700193520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 msgr2=0x562181f33440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.126+0000 7f2783fff640 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f33440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.126+0000 7f2783fff640 1 -- 172.21.15.5:0/1700193520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f2b370 msgr2=0x562181f24380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:40:15.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.126+0000 7f2783fff640 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f2b370 0x562181f24380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.126+0000 7f2783fff640 1 -- 172.21.15.5:0/1700193520 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562181f28bb0 con 0x562181f21b40 2024-09-11T22:40:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.126+0000 7f2783fff640 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f33980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f277000e5d0 tx=0x7f2770005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29af8d64e2423d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.126+0000 7f2763fff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:40:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.127+0000 7f2763fff640 1 -- 172.21.15.5:0/1700193520 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2770005080 con 0x562181f21b40 2024-09-11T22:40:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.127+0000 7f278d211740 1 -- 172.21.15.5:0/1700193520 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277c0a01c0 con 0x562181f21b40 2024-09-11T22:40:15.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.127+0000 7f2763fff640 1 -- 172.21.15.5:0/1700193520 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f2770005220 con 0x562181f21b40 2024-09-11T22:40:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.128+0000 7f2763fff640 1 -- 172.21.15.5:0/1700193520 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f2770036050 con 0x562181f21b40 2024-09-11T22:40:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.128+0000 7f278d211740 10 client.30023 fetch_fsmap learned FSMap version 54 2024-09-11T22:40:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.128+0000 7f278d211740 1 -- 172.21.15.5:0/1700193520 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562181f259f0 con 0x562181f21b40 2024-09-11T22:40:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.132+0000 7f2763fff640 1 -- 172.21.15.5:0/1700193520 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f277c0a01c0 con 0x562181f21b40 2024-09-11T22:40:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.132+0000 7f2763fff640 1 -- 172.21.15.5:0/1700193520 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 54) ==== 8362+0+0 (secure 0 0 0) 0x7f277001f6d0 con 0x562181f21b40 2024-09-11T22:40:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.133+0000 7f278d211740 10 client.30023 fetch_fsmap finished waiting for FSMap version 54 2024-09-11T22:40:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.133+0000 7f278d211740 10 client.30023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.133+0000 7f278d211740 20 client.30023 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.133+0000 7f278d211740 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562181fa6b40 0x562181fa8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.133+0000 7f278d211740 4 client.30023 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:40:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.133+0000 7f278d211740 1 -- 172.21.15.5:0/1700193520 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562181e8dd70 con 0x562181fa6b40 2024-09-11T22:40:15.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.136+0000 7f2788bd4640 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562181fa6b40 0x562181fa8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.140+0000 7f2788bd4640 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562181fa6b40 0x562181fa8f30 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f275195802ed71 server_cookie=ccfe7177f5a09773 in_seq=0 out_seq=0 2024-09-11T22:40:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.140+0000 7f2763fff640 10 client.30023 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.150+0000 7f2763fff640 1 -- 172.21.15.5:0/1700193520 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562181e8dd70 con 0x562181fa6b40 2024-09-11T22:40:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.150+0000 7f2763fff640 10 client.30023 handle_command_reply: tid=0 2024-09-11T22:40:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.151+0000 7f278d211740 1 client.30023 shutdown 2024-09-11T22:40:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.158+0000 7f278d211740 1 -- 172.21.15.5:0/1700193520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 msgr2=0x562181f33980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.158+0000 7f278d211740 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f33980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f277000e5d0 tx=0x7f2770005f90 comp rx=0 tx=0).stop 2024-09-11T22:40:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.159+0000 7f278d211740 1 -- 172.21.15.5:0/1700193520 shutdown_connections 2024-09-11T22:40:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.159+0000 7f278d211740 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x562181fa6b40 0x562181fa8f30 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.159+0000 7f278d211740 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562181f2b370 0x562181f24380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.159+0000 7f278d211740 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562181f21b40 0x562181f33980 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.159+0000 7f278d211740 1 --2- 172.21.15.5:0/1700193520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562181f205d0 0x562181f33440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.159+0000 7f2763fff640 0 client.30023 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.159+0000 7f278d211740 1 -- 172.21.15.5:0/1700193520 >> 172.21.15.5:0/1700193520 conn(0x562181ee9310 msgr2=0x562181f362c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:15.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.159+0000 7f278d211740 1 -- 172.21.15.5:0/1700193520 shutdown_connections 2024-09-11T22:40:15.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.159+0000 7f278d211740 1 -- 172.21.15.5:0/1700193520 wait complete. 2024-09-11T22:40:15.161 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.159+0000 7f278d211740 20 client.30023 trim_cache size 0 max 16384 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.160+0000 7f278ae38640 1 -- 172.21.15.5:0/23348012 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27740a2ec0 msgr2=0x7f27740a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27740a2ec0 0x7f27740a5380 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f27780096c0 tx=0x7f2778005a20 comp rx=0 tx=0).stop 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 -- 172.21.15.5:0/23348012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278410e480 msgr2=0x7f2784103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278410e480 0x7f2784103630 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2784102ab0 tx=0x7f277c00ef10 comp rx=0 tx=0).stop 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 -- 172.21.15.5:0/23348012 shutdown_connections 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f27740a2ec0 0x7f27740a5380 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f278410e480 0x7f2784103630 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2784101280 0x7f27841030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 --2- 172.21.15.5:0/23348012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27841008d0 0x7f2784102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 -- 172.21.15.5:0/23348012 >> 172.21.15.5:0/23348012 conn(0x7f27840fc7a0 msgr2=0x7f27840feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 -- 172.21.15.5:0/23348012 shutdown_connections 2024-09-11T22:40:15.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:15.161+0000 7f278ae38640 1 -- 172.21.15.5:0/23348012 wait complete. 2024-09-11T22:40:15.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69731 15.80 MB/sec execute 153 sec latency 4515.806 ms 2024-09-11T22:40:15.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:15 smithi005 ceph-mon[30943]: pgmap v575: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 511 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 447 op/s 2024-09-11T22:40:16.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:15 smithi159 ceph-mon[35534]: pgmap v575: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 511 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 447 op/s 2024-09-11T22:40:16.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:15 smithi172 ceph-mon[35461]: pgmap v575: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 511 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 447 op/s 2024-09-11T22:40:16.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70126 15.81 MB/sec execute 154 sec latency 5515.850 ms 2024-09-11T22:40:17.204 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:40:17.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:40:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:17.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70659 15.88 MB/sec execute 155 sec latency 6515.882 ms 2024-09-11T22:40:17.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:17 smithi005 ceph-mon[30943]: pgmap v576: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 511 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 447 op/s 2024-09-11T22:40:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.008+0000 7f6bbf140640 1 -- 172.21.15.5:0/1454045076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb8102d90 msgr2=0x7f6bb8103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:18.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.008+0000 7f6bbf140640 1 --2- 172.21.15.5:0/1454045076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb8102d90 0x7f6bb8103170 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009930 tx=0x7f6bac02a420 comp rx=0 tx=0).stop 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.008+0000 7f6bbf140640 1 -- 172.21.15.5:0/1454045076 shutdown_connections 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.008+0000 7f6bbf140640 1 --2- 172.21.15.5:0/1454045076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb8108290 0x7f6bb810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.008+0000 7f6bbf140640 1 --2- 172.21.15.5:0/1454045076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bb8103740 0x7f6bb8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.008+0000 7f6bbf140640 1 --2- 172.21.15.5:0/1454045076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb8102d90 0x7f6bb8103170 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.008+0000 7f6bbf140640 1 -- 172.21.15.5:0/1454045076 >> 172.21.15.5:0/1454045076 conn(0x7f6bb80fc7d0 msgr2=0x7f6bb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.008+0000 7f6bbf140640 1 -- 172.21.15.5:0/1454045076 shutdown_connections 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.008+0000 7f6bbf140640 1 -- 172.21.15.5:0/1454045076 wait complete. 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbf140640 1 Processor -- start 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbf140640 1 -- start start 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbf140640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb8102d90 0x7f6bb81009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbf140640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb8103740 0x7f6bb8100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbf140640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bb8108290 0x7f6bb8101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbf140640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8112090 con 0x7f6bb8102d90 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbf140640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8111f10 con 0x7f6bb8103740 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbf140640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8112210 con 0x7f6bb8108290 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbcedc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb8102d90 0x7f6bb81009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbcedc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb8102d90 0x7f6bb81009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57600/0 (socket says 172.21.15.5:57600) 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbcedc640 1 -- 172.21.15.5:0/3767859520 learned_addr learned my addr 172.21.15.5:0/3767859520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bb7fff640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb8103740 0x7f6bb8100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.009+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bb8108290 0x7f6bb8101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bbcedc640 1 -- 172.21.15.5:0/3767859520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bb8108290 msgr2=0x7f6bb8101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bbcedc640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bb8108290 0x7f6bb8101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bbcedc640 1 -- 172.21.15.5:0/3767859520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb8103740 msgr2=0x7f6bb8100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bbcedc640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb8103740 0x7f6bb8100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bbcedc640 1 -- 172.21.15.5:0/3767859520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb81a6a90 con 0x7f6bb8102d90 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bbcedc640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb8102d90 0x7f6bb81009f0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009230 tx=0x7f6bac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bb5ffb640 1 -- 172.21.15.5:0/3767859520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac03f030 con 0x7f6bb8102d90 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bb5ffb640 1 -- 172.21.15.5:0/3767859520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bac004440 con 0x7f6bb8102d90 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bbf140640 1 -- 172.21.15.5:0/3767859520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb81a6d20 con 0x7f6bb8102d90 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bb5ffb640 1 -- 172.21.15.5:0/3767859520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac035070 con 0x7f6bb8102d90 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.010+0000 7f6bbf140640 1 -- 172.21.15.5:0/3767859520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb81a71b0 con 0x7f6bb8102d90 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bc1519740 1 Processor -- start 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bc1519740 1 -- start start 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bb5ffb640 1 -- 172.21.15.5:0/3767859520 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6bac042030 con 0x7f6bb8102d90 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 0x55c8b6701f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b6702480 0x55c8b670c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b670cd40 0x55c8b670f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bc1519740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c8b6667280 con 0x55c8b670cd40 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bc1519740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c8b660eb10 con 0x55c8b6701b60 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bc1519740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c8b6663240 con 0x55c8b6702480 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bbd6dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 0x55c8b6701f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bbcedc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b6702480 0x55c8b670c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b670cd40 0x55c8b670f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bbd6dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 0x55c8b6701f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42308/0 (socket says 172.21.15.5:42308) 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b670cd40 0x55c8b670f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57610/0 (socket says 172.21.15.5:57610) 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.013+0000 7f6bbd6dd640 1 -- 172.21.15.5:0/4118736216 learned_addr learned my addr 172.21.15.5:0/4118736216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.014+0000 7f6bbd6dd640 1 -- 172.21.15.5:0/4118736216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b6702480 msgr2=0x55c8b670c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.014+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/4118736216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b6702480 0x55c8b670c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.014+0000 7f6bbd6dd640 1 -- 172.21.15.5:0/4118736216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b670cd40 msgr2=0x55c8b670f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.014+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/4118736216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b670cd40 0x55c8b670f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.014+0000 7f6bbd6dd640 1 -- 172.21.15.5:0/4118736216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b661b1f0 con 0x55c8b6701b60 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.014+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/4118736216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 0x55c8b6701f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009f50 tx=0x7f6bb00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c61051ab9a85b9ba server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.014+0000 7f6ba2ffd640 1 -- 172.21.15.5:0/4118736216 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0004b20 con 0x55c8b6701b60 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.014+0000 7f6ba2ffd640 1 -- 172.21.15.5:0/4118736216 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb0004cc0 con 0x55c8b6701b60 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.014+0000 7f6ba2ffd640 1 -- 172.21.15.5:0/4118736216 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0005020 con 0x55c8b6701b60 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.015+0000 7f6bb5ffb640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6ba80a2cf0 0x7f6ba80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.015+0000 7f6bbcedc640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6ba80a2cf0 0x7f6ba80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.015+0000 7f6bb5ffb640 1 -- 172.21.15.5:0/3767859520 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f6bac0ef7d0 con 0x7f6bb8102d90 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.015+0000 7f6bc1519740 1 -- 172.21.15.5:0/4118736216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 msgr2=0x55c8b6701f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.015+0000 7f6bc1519740 1 --2- 172.21.15.5:0/4118736216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 0x55c8b6701f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0009f50 tx=0x7f6bb00118d0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.016+0000 7f6bc1519740 1 -- 172.21.15.5:0/4118736216 shutdown_connections 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.016+0000 7f6bc1519740 1 --2- 172.21.15.5:0/4118736216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b670cd40 0x55c8b670f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.016+0000 7f6bc1519740 1 --2- 172.21.15.5:0/4118736216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b6702480 0x55c8b670c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.016+0000 7f6bc1519740 1 --2- 172.21.15.5:0/4118736216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 0x55c8b6701f40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.016+0000 7f6bc1519740 1 -- 172.21.15.5:0/4118736216 >> 172.21.15.5:0/4118736216 conn(0x55c8b66fe0f0 msgr2=0x55c8b66fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.016+0000 7f6bc1519740 1 -- 172.21.15.5:0/4118736216 shutdown_connections 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.016+0000 7f6bc1519740 1 -- 172.21.15.5:0/4118736216 wait complete. 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.016+0000 7f6bbcedc640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6ba80a2cf0 0x7f6ba80a51b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0ecf30 tx=0x7f6bac0f5550 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bc1519740 1 -- start start 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bc1519740 1 Processor -- start 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6701b60 0x55c8b670c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6702480 0x55c8b6709b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b670cd40 0x55c8b670a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bc1519740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c8b670f910 con 0x55c8b6701b60 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bc1519740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c8b6621f10 con 0x55c8b6702480 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bc1519740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55c8b670fa90 con 0x55c8b670cd40 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bbd6dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6701b60 0x55c8b670c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bc1519740 10 client.0 _ll_register_callbacks cb 0x55c8b65eaa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bb7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6702480 0x55c8b6709b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bbd6dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6701b60 0x55c8b670c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57624/0 (socket says 172.21.15.5:57624) 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bbd6dd640 1 -- 172.21.15.5:0/939195142 learned_addr learned my addr 172.21.15.5:0/939195142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.017+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b670cd40 0x55c8b670a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6bb7fff640 1 -- 172.21.15.5:0/939195142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b670cd40 msgr2=0x55c8b670a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6bb7fff640 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b670cd40 0x55c8b670a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6bb7fff640 1 -- 172.21.15.5:0/939195142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6701b60 msgr2=0x55c8b670c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6bb7fff640 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6701b60 0x55c8b670c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6bb7fff640 1 -- 172.21.15.5:0/939195142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b63d86a0 con 0x55c8b6702480 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6701b60 0x55c8b670c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6bb7fff640 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6702480 0x55c8b6709b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8102480 tx=0x7f6ba4012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10a311de41485889 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6b97fff640 1 -- 172.21.15.5:0/939195142 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4004020 con 0x55c8b6702480 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6bc1519740 1 -- 172.21.15.5:0/939195142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb0004cc0 con 0x55c8b6702480 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6b97fff640 1 -- 172.21.15.5:0/939195142 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f6ba40041c0 con 0x55c8b6702480 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6b97fff640 1 -- 172.21.15.5:0/939195142 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6ba40043c0 con 0x55c8b6702480 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6bc1519740 10 client.30035 fetch_fsmap learned FSMap version 55 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.018+0000 7f6bc1519740 1 -- 172.21.15.5:0/939195142 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c8b671b4a0 con 0x55c8b6702480 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6b97fff640 1 -- 172.21.15.5:0/939195142 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb0004cc0 con 0x55c8b6702480 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6b97fff640 1 -- 172.21.15.5:0/939195142 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 55) ==== 8366+0+0 (secure 0 0 0) 0x7f6ba4003b80 con 0x55c8b6702480 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6bc1519740 10 client.30035 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T22:40:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6bc1519740 10 client.30035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6bc1519740 20 client.30035 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6bc1519740 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8b671c960 0x55c8b671ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6bc1519740 4 client.30035 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6bc1519740 1 -- 172.21.15.5:0/939195142 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8b66747f0 con 0x55c8b671c960 2024-09-11T22:40:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8b671c960 0x55c8b671ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8b671c960 0x55c8b671ed50 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d182f790983ddd server_cookie=d32e2ca52c34bfcd in_seq=0 out_seq=0 2024-09-11T22:40:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.019+0000 7f6b97fff640 10 client.30035 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6b97fff640 1 -- 172.21.15.5:0/939195142 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8b66747f0 con 0x55c8b671c960 2024-09-11T22:40:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6b97fff640 10 client.30035 handle_command_reply: tid=0 2024-09-11T22:40:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6bc1519740 1 client.30035 shutdown 2024-09-11T22:40:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6bc1519740 1 -- 172.21.15.5:0/939195142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6702480 msgr2=0x55c8b6709b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6bc1519740 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6702480 0x55c8b6709b40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8102480 tx=0x7f6ba4012b60 comp rx=0 tx=0).stop 2024-09-11T22:40:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6bc1519740 1 -- 172.21.15.5:0/939195142 shutdown_connections 2024-09-11T22:40:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6bc1519740 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8b671c960 0x55c8b671ed50 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6bc1519740 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b670cd40 0x55c8b670a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6bc1519740 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6702480 0x55c8b6709b40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6bc1519740 1 --2- 172.21.15.5:0/939195142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6701b60 0x55c8b670c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6bc1519740 1 -- 172.21.15.5:0/939195142 >> 172.21.15.5:0/939195142 conn(0x55c8b66fe0f0 msgr2=0x55c8b6703dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6b97fff640 0 client.30035 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.022+0000 7f6bc1519740 1 -- 172.21.15.5:0/939195142 shutdown_connections 2024-09-11T22:40:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.023+0000 7f6bc1519740 1 -- 172.21.15.5:0/939195142 wait complete. 2024-09-11T22:40:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.023+0000 7f6bc1519740 20 client.30035 trim_cache size 0 max 16384 2024-09-11T22:40:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bc1519740 1 Processor -- start 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bc1519740 1 -- start start 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 0x55c8b67045c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b6724660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b670cd40 0x55c8b6724ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bc1519740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81afd70 con 0x55c8b6702480 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bc1519740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81b18e0 con 0x55c8b6701b60 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bc1519740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81b0440 con 0x55c8b670cd40 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b6724660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b6724660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57632/0 (socket says 172.21.15.5:57632) 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bb7fff640 1 -- 172.21.15.5:0/111778236 learned_addr learned my addr 172.21.15.5:0/111778236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/111778236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 0x55c8b67045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/111778236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b670cd40 0x55c8b6724ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bb7fff640 1 -- 172.21.15.5:0/111778236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b670cd40 msgr2=0x55c8b6724ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bb7fff640 1 --2- 172.21.15.5:0/111778236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b670cd40 0x55c8b6724ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bb7fff640 1 -- 172.21.15.5:0/111778236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 msgr2=0x55c8b67045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bb7fff640 1 --2- 172.21.15.5:0/111778236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 0x55c8b67045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.042+0000 7f6bb7fff640 1 -- 172.21.15.5:0/111778236 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b66f9830 con 0x55c8b6702480 2024-09-11T22:40:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.043+0000 7f6bb7fff640 1 --2- 172.21.15.5:0/111778236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b6724660 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009f20 tx=0x7f6ba4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f32e74b3113f6fc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.043+0000 7f6ba27fc640 1 -- 172.21.15.5:0/111778236 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4004020 con 0x55c8b6702480 2024-09-11T22:40:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.043+0000 7f6ba27fc640 1 -- 172.21.15.5:0/111778236 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba40041c0 con 0x55c8b6702480 2024-09-11T22:40:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.043+0000 7f6ba27fc640 1 -- 172.21.15.5:0/111778236 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4003d10 con 0x55c8b6702480 2024-09-11T22:40:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 -- 172.21.15.5:0/111778236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 msgr2=0x55c8b6724660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 --2- 172.21.15.5:0/111778236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b6724660 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009f20 tx=0x7f6ba4005800 comp rx=0 tx=0).stop 2024-09-11T22:40:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 -- 172.21.15.5:0/111778236 shutdown_connections 2024-09-11T22:40:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 --2- 172.21.15.5:0/111778236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b670cd40 0x55c8b6724ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 --2- 172.21.15.5:0/111778236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b6724660 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 --2- 172.21.15.5:0/111778236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b6701b60 0x55c8b67045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 -- 172.21.15.5:0/111778236 >> 172.21.15.5:0/111778236 conn(0x55c8b66fe0f0 msgr2=0x55c8b6708410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 -- 172.21.15.5:0/111778236 shutdown_connections 2024-09-11T22:40:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 -- 172.21.15.5:0/111778236 wait complete. 2024-09-11T22:40:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 -- start start 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.044+0000 7f6bc1519740 1 Processor -- start 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b6701b60 0x55c8b67153a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b670db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bc1519740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b670cd40 0x55c8b670e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bc1519740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81b1210 con 0x55c8b6702480 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bc1519740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81b0b40 con 0x55c8b670cd40 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bc1519740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb81add70 con 0x55c8b6701b60 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bc1519740 10 client.0 _ll_register_callbacks cb 0x7f6ba4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b670db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bbd6dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b670cd40 0x55c8b670e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bbd6dd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b670cd40 0x55c8b670e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42326/0 (socket says 172.21.15.5:42326) 2024-09-11T22:40:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b670db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57648/0 (socket says 172.21.15.5:57648) 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bbd6dd640 1 -- 172.21.15.5:0/2682520303 learned_addr learned my addr 172.21.15.5:0/2682520303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b6701b60 0x55c8b67153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bbd6dd640 1 -- 172.21.15.5:0/2682520303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b6701b60 msgr2=0x55c8b67153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b6701b60 0x55c8b67153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bbd6dd640 1 -- 172.21.15.5:0/2682520303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 msgr2=0x55c8b670db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b670db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bbd6dd640 1 -- 172.21.15.5:0/2682520303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b67022b0 con 0x55c8b670cd40 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bb7fff640 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b670db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6bbd6dd640 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b670cd40 0x55c8b670e0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0004590 tx=0x7f6bb00045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184cde95dc1f73f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.045+0000 7f6b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6b97fff640 1 -- 172.21.15.5:0/2682520303 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0004880 con 0x55c8b670cd40 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6b97fff640 1 -- 172.21.15.5:0/2682520303 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f6bb001f040 con 0x55c8b670cd40 2024-09-11T22:40:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6bc1519740 1 -- 172.21.15.5:0/2682520303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ba40041c0 con 0x55c8b670cd40 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6b97fff640 1 -- 172.21.15.5:0/2682520303 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6bb0003c40 con 0x55c8b670cd40 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6bc1519740 10 client.30041 fetch_fsmap learned FSMap version 55 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6bc1519740 1 -- 172.21.15.5:0/2682520303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55c8b6713c40 con 0x55c8b670cd40 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6b97fff640 1 -- 172.21.15.5:0/2682520303 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba40041c0 con 0x55c8b670cd40 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6b97fff640 1 -- 172.21.15.5:0/2682520303 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 55) ==== 8366+0+0 (secure 0 0 0) 0x7f6bb0028450 con 0x55c8b670cd40 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6bc1519740 10 client.30041 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6bc1519740 10 client.30041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6bc1519740 20 client.30041 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6bc1519740 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8b6787990 0x55c8b6789d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6bc1519740 4 client.30041 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.046+0000 7f6bc1519740 1 -- 172.21.15.5:0/2682520303 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c8b666ed00 con 0x55c8b6787990 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.047+0000 7f6bb7fff640 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8b6787990 0x55c8b6789d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.047+0000 7f6bb7fff640 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8b6787990 0x55c8b6789d80 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db73e586bdf7df9e server_cookie=553dbec7bc7adbd9 in_seq=0 out_seq=0 2024-09-11T22:40:18.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.047+0000 7f6b97fff640 10 client.30041 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.048+0000 7f6b97fff640 1 -- 172.21.15.5:0/2682520303 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c8b666ed00 con 0x55c8b6787990 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.048+0000 7f6b97fff640 10 client.30041 handle_command_reply: tid=0 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.048+0000 7f6bc1519740 1 client.30041 shutdown 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 1 -- 172.21.15.5:0/2682520303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b670cd40 msgr2=0x55c8b670e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b670cd40 0x55c8b670e0a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0004590 tx=0x7f6bb00045c0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 1 -- 172.21.15.5:0/2682520303 shutdown_connections 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55c8b6787990 0x55c8b6789d80 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c8b670cd40 0x55c8b670e0a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8b6702480 0x55c8b670db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 1 --2- 172.21.15.5:0/2682520303 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c8b6701b60 0x55c8b67153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 1 -- 172.21.15.5:0/2682520303 >> 172.21.15.5:0/2682520303 conn(0x55c8b66fe0f0 msgr2=0x55c8b66fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:18.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6b97fff640 0 client.30041 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:18.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 1 -- 172.21.15.5:0/2682520303 shutdown_connections 2024-09-11T22:40:18.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 1 -- 172.21.15.5:0/2682520303 wait complete. 2024-09-11T22:40:18.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.049+0000 7f6bc1519740 20 client.30041 trim_cache size 0 max 16384 2024-09-11T22:40:18.054 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:40:18.054 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:40:18.054 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "73d962af-2d60-44de-ac0f-e71dc5e87afa", 2024-09-11T22:40:18.054 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:40:18.054 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:40:18.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 -- 172.21.15.5:0/3767859520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6ba80a2cf0 msgr2=0x7f6ba80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:18.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6ba80a2cf0 0x7f6ba80a51b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0ecf30 tx=0x7f6bac0f5550 comp rx=0 tx=0).stop 2024-09-11T22:40:18.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 -- 172.21.15.5:0/3767859520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb8102d90 msgr2=0x7f6bb81009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:18.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb8102d90 0x7f6bb81009f0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009230 tx=0x7f6bac004040 comp rx=0 tx=0).stop 2024-09-11T22:40:18.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 -- 172.21.15.5:0/3767859520 shutdown_connections 2024-09-11T22:40:18.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6bb8108290 0x7f6bb8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6ba80a2cf0 0x7f6ba80a51b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bb8103740 0x7f6bb8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 --2- 172.21.15.5:0/3767859520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bb8102d90 0x7f6bb81009f0 secure :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009230 tx=0x7f6bac004040 comp rx=0 tx=0).stop 2024-09-11T22:40:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 -- 172.21.15.5:0/3767859520 >> 172.21.15.5:0/3767859520 conn(0x7f6bb80fc7d0 msgr2=0x7f6bb8105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 -- 172.21.15.5:0/3767859520 shutdown_connections 2024-09-11T22:40:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:18.050+0000 7f6bbf140640 1 -- 172.21.15.5:0/3767859520 wait complete. 2024-09-11T22:40:18.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:17 smithi159 ceph-mon[35534]: pgmap v576: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 511 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 447 op/s 2024-09-11T22:40:18.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:17 smithi172 ceph-mon[35461]: pgmap v576: 97 pgs: 13 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 54 active+clean; 511 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 447 op/s 2024-09-11T22:40:18.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71125 16.01 MB/sec execute 156 sec latency 7515.910 ms 2024-09-11T22:40:19.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:18 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:40:19.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:18 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:40:19.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:18 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:19.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:18 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:40:19.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:18 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:40:19.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:18 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:19.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:18 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:40:19.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:18 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:40:19.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:18 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f7176ffd640 1 -- 172.21.15.5:0/3172732056 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6299e3a20 con 0x55e629af85e0 2024-09-11T22:40:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f7176ffd640 10 client.29824 handle_command_reply: tid=0 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 client.29824 shutdown 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 -- 172.21.15.5:0/3172732056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a7ccb0 msgr2=0x55e629af69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a7ccb0 0x55e629af69f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7190005e30 tx=0x7f7190005fd0 comp rx=0 tx=0).stop 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 -- 172.21.15.5:0/3172732056 shutdown_connections 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55e629af85e0 0x55e629afaa40 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e629a7ccb0 0x55e629af69f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f7176ffd640 0 client.29824 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e629a72410 0x55e629a816b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 --2- 172.21.15.5:0/3172732056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e629a71af0 0x55e629a81170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 -- 172.21.15.5:0/3172732056 >> 172.21.15.5:0/3172732056 conn(0x55e629a6e060 msgr2=0x55e629a7b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 -- 172.21.15.5:0/3172732056 shutdown_connections 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 1 -- 172.21.15.5:0/3172732056 wait complete. 2024-09-11T22:40:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.239+0000 7f71a0e80740 20 client.29824 trim_cache size 0 max 16384 2024-09-11T22:40:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:40:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:40:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:40:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:40:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 -- 172.21.15.5:0/4115665409 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f71800a2ef0 msgr2=0x7f71800a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f71800a2ef0 0x7f71800a53b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7190009a60 tx=0x7f7190005d80 comp rx=0 tx=0).stop 2024-09-11T22:40:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 -- 172.21.15.5:0/4115665409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7198100490 msgr2=0x7f719806da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:19.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7198100490 0x7f719806da30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f718c00b3f0 tx=0x7f718c00b8b0 comp rx=0 tx=0).stop 2024-09-11T22:40:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 -- 172.21.15.5:0/4115665409 shutdown_connections 2024-09-11T22:40:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7198112800 0x7f719806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f71800a2ef0 0x7f71800a53b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7198100490 0x7f719806da30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 --2- 172.21.15.5:0/4115665409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71980ffb70 0x7f71980739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 -- 172.21.15.5:0/4115665409 >> 172.21.15.5:0/4115665409 conn(0x7f71980fba30 msgr2=0x7f7198110e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 -- 172.21.15.5:0/4115665409 shutdown_connections 2024-09-11T22:40:19.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:19.240+0000 7f719dffd640 1 -- 172.21.15.5:0/4115665409 wait complete. 2024-09-11T22:40:19.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:40:19.632 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:19.650 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71327 15.96 MB/sec execute 157 sec latency 7924.325 ms 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: pgmap v577: 97 pgs: 97 active+clean; 523 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.5 MiB/s wr, 420 op/s 2024-09-11T22:40:20.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:40:20.072 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:19 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:40:20.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:40:20.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:40:20.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:40:20.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:40:20.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: pgmap v577: 97 pgs: 97 active+clean; 523 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.5 MiB/s wr, 420 op/s 2024-09-11T22:40:20.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:40:20.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:19 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:40:20.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:40:20.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:40:20.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:40:20.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:40:20.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: pgmap v577: 97 pgs: 97 active+clean; 523 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.5 MiB/s wr, 420 op/s 2024-09-11T22:40:20.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:40:20.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:19 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.166+0000 7f4e2cd21640 1 -- 172.21.15.5:0/2942994847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e28102d40 msgr2=0x7f4e28103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.166+0000 7f4e2cd21640 1 --2- 172.21.15.5:0/2942994847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e28102d40 0x7f4e28103120 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009900 tx=0x7f4e1802a480 comp rx=0 tx=0).stop 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.166+0000 7f4e2cd21640 1 -- 172.21.15.5:0/2942994847 shutdown_connections 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.166+0000 7f4e2cd21640 1 --2- 172.21.15.5:0/2942994847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e2810c170 0x7f4e2810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.166+0000 7f4e2cd21640 1 --2- 172.21.15.5:0/2942994847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e28103660 0x7f4e2810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.166+0000 7f4e2cd21640 1 --2- 172.21.15.5:0/2942994847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e28102d40 0x7f4e28103120 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.166+0000 7f4e2cd21640 1 -- 172.21.15.5:0/2942994847 >> 172.21.15.5:0/2942994847 conn(0x7f4e280fc7a0 msgr2=0x7f4e280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.166+0000 7f4e2cd21640 1 -- 172.21.15.5:0/2942994847 shutdown_connections 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.166+0000 7f4e2cd21640 1 -- 172.21.15.5:0/2942994847 wait complete. 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.167+0000 7f4e2cd21640 1 Processor -- start 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.167+0000 7f4e2cd21640 1 -- start start 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e2cd21640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e28102d40 0x7f4e281984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e2cd21640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e28103660 0x7f4e28198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e2cd21640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2810c170 0x7f4e2819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e2cd21640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28111f70 con 0x7f4e2810c170 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e2cd21640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e28111df0 con 0x7f4e28102d40 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e2cd21640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281120f0 con 0x7f4e28103660 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e1dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e28103660 0x7f4e28198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2810c170 0x7f4e2819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2810c170 0x7f4e2819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57662/0 (socket says 172.21.15.5:57662) 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e26d9d640 1 -- 172.21.15.5:0/1065307291 learned_addr learned my addr 172.21.15.5:0/1065307291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e1dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e28103660 0x7f4e28198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33774/0 (socket says 172.21.15.5:33774) 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e2659c640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e28102d40 0x7f4e281984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e26d9d640 1 -- 172.21.15.5:0/1065307291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e28103660 msgr2=0x7f4e28198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e26d9d640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e28103660 0x7f4e28198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e26d9d640 1 -- 172.21.15.5:0/1065307291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e28102d40 msgr2=0x7f4e281984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:20.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e26d9d640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e28102d40 0x7f4e281984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.168+0000 7f4e26d9d640 1 -- 172.21.15.5:0/1065307291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2819d490 con 0x7f4e2810c170 2024-09-11T22:40:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.169+0000 7f4e26d9d640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2810c170 0x7f4e2819cdb0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000eaa0 tx=0x7f4e2000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.169+0000 7f4e1ffff640 1 -- 172.21.15.5:0/1065307291 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2000cdb0 con 0x7f4e2810c170 2024-09-11T22:40:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.169+0000 7f4e1ffff640 1 -- 172.21.15.5:0/1065307291 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e20005020 con 0x7f4e2810c170 2024-09-11T22:40:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.169+0000 7f4e2cd21640 1 -- 172.21.15.5:0/1065307291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2819d780 con 0x7f4e2810c170 2024-09-11T22:40:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.169+0000 7f4e1ffff640 1 -- 172.21.15.5:0/1065307291 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e200116b0 con 0x7f4e2810c170 2024-09-11T22:40:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.169+0000 7f4e2cd21640 1 -- 172.21.15.5:0/1065307291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e281a5060 con 0x7f4e2810c170 2024-09-11T22:40:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.171+0000 7f4e2f0fa740 1 Processor -- start 2024-09-11T22:40:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e1ffff640 1 -- 172.21.15.5:0/1065307291 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4e20005560 con 0x7f4e2810c170 2024-09-11T22:40:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e2f0fa740 1 -- start start 2024-09-11T22:40:20.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 0x555e3b7a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3ca0 0x555e3b7b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 0x555e3b7b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e3b7092e0 con 0x555e3b7b2a90 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555e3b6b0ce0 con 0x555e3b7a3ca0 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e3b7052a0 con 0x555e3b7a3380 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e1dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 0x555e3b7a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e1dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 0x555e3b7a3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:33790/0 (socket says 172.21.15.5:33790) 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/1547857063 learned_addr learned my addr 172.21.15.5:0/1547857063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.172+0000 7f4e2659c640 1 --2- 172.21.15.5:0/1547857063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3ca0 0x555e3b7b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1547857063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 0x555e3b7b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1ffff640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e140a2cf0 0x7f4e140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/1547857063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3ca0 msgr2=0x555e3b7b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1547857063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3ca0 0x555e3b7b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/1547857063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 msgr2=0x555e3b7b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1547857063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 0x555e3b7b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/1547857063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e3b695440 con 0x555e3b7a3380 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e2659c640 1 --2- 172.21.15.5:0/1547857063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3ca0 0x555e3b7b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e26d9d640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e140a2cf0 0x7f4e140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1ffff640 1 -- 172.21.15.5:0/1065307291 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f4e200cf620 con 0x7f4e2810c170 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1547857063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 0x555e3b7a3760 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x555e3b7a0b20 tx=0x7f4e100097c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f61cfc6e79915207 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e26d9d640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e140a2cf0 0x7f4e140a51b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4e20098520 tx=0x7f4e20096040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1cd99640 1 -- 172.21.15.5:0/1547857063 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000f7c0 con 0x555e3b7a3380 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1cd99640 1 -- 172.21.15.5:0/1547857063 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e1000fe40 con 0x555e3b7a3380 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.173+0000 7f4e1cd99640 1 -- 172.21.15.5:0/1547857063 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1001fdd0 con 0x555e3b7a3380 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.174+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1547857063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 msgr2=0x555e3b7a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.174+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1547857063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 0x555e3b7a3760 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x555e3b7a0b20 tx=0x7f4e100097c0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.174+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1547857063 shutdown_connections 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.174+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1547857063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 0x555e3b7b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.174+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1547857063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3ca0 0x555e3b7b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.174+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1547857063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 0x555e3b7a3760 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.174+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1547857063 >> 172.21.15.5:0/1547857063 conn(0x555e3b79fa30 msgr2=0x555e3b79fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.174+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1547857063 shutdown_connections 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.174+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1547857063 wait complete. 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2f0fa740 1 -- start start 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2f0fa740 1 Processor -- start 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 0x555e3b7b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3ca0 0x555e3b7af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7b2a90 0x555e3b7afd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e3b7b5660 con 0x555e3b7a3ca0 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555e3b6c4050 con 0x555e3b7b2a90 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x555e3b7b57e0 con 0x555e3b7a3380 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2f0fa740 10 client.0 _ll_register_callbacks cb 0x555e3b704160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3ca0 0x555e3b7af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e1dd9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 0x555e3b7b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3ca0 0x555e3b7af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57672/0 (socket says 172.21.15.5:57672) 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e2659c640 1 -- 172.21.15.5:0/1096796335 learned_addr learned my addr 172.21.15.5:0/1096796335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.175+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7b2a90 0x555e3b7afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e2659c640 1 -- 172.21.15.5:0/1096796335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 msgr2=0x555e3b7b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e2659c640 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 0x555e3b7b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e2659c640 1 -- 172.21.15.5:0/1096796335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7b2a90 msgr2=0x555e3b7afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e2659c640 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7b2a90 0x555e3b7afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e2659c640 1 -- 172.21.15.5:0/1096796335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e3b47a3c0 con 0x555e3b7a3ca0 2024-09-11T22:40:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e2659c640 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3ca0 0x555e3b7af730 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f4e1802a9a0 tx=0x7f4e180318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8fbbe5fab5cf0a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e0dffb640 1 -- 172.21.15.5:0/1096796335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18035040 con 0x555e3b7a3ca0 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1096796335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1000fe40 con 0x555e3b7a3ca0 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e0dffb640 1 -- 172.21.15.5:0/1096796335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f4e18004240 con 0x555e3b7a3ca0 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e0dffb640 1 -- 172.21.15.5:0/1096796335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4e18004440 con 0x555e3b7a3ca0 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e2f0fa740 10 client.21021 fetch_fsmap learned FSMap version 55 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.176+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1096796335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555e3b7c1150 con 0x555e3b7a3ca0 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.177+0000 7f4e0dffb640 1 -- 172.21.15.5:0/1096796335 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e1000fe40 con 0x555e3b7a3ca0 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.177+0000 7f4e0dffb640 1 -- 172.21.15.5:0/1096796335 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 55) ==== 8366+0+0 (secure 0 0 0) 0x7f4e180455f0 con 0x555e3b7a3ca0 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.177+0000 7f4e2f0fa740 10 client.21021 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.177+0000 7f4e2f0fa740 10 client.21021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.177+0000 7f4e2f0fa740 20 client.21021 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.177+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e3b7c2560 0x555e3b7c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.177+0000 7f4e2f0fa740 4 client.21021 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.177+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1096796335 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e3b716850 con 0x555e3b7c2560 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.177+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e3b7c2560 0x555e3b7c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.178+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e3b7c2560 0x555e3b7c4980 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3a6dda191e668c server_cookie=10d51af3ea64f973 in_seq=0 out_seq=0 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.178+0000 7f4e0dffb640 10 client.21021 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e0dffb640 1 -- 172.21.15.5:0/1096796335 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e3b716850 con 0x555e3b7c2560 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e0dffb640 10 client.21021 handle_command_reply: tid=0 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 client.21021 shutdown 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1096796335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3ca0 msgr2=0x555e3b7af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3ca0 0x555e3b7af730 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f4e1802a9a0 tx=0x7f4e180318d0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1096796335 shutdown_connections 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e3b7c2560 0x555e3b7c4980 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7b2a90 0x555e3b7afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3ca0 0x555e3b7af730 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e0dffb640 0 client.21021 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1096796335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3380 0x555e3b7b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1096796335 >> 172.21.15.5:0/1096796335 conn(0x555e3b79fa30 msgr2=0x555e3b7a5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1096796335 shutdown_connections 2024-09-11T22:40:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1096796335 wait complete. 2024-09-11T22:40:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.180+0000 7f4e2f0fa740 20 client.21021 trim_cache size 0 max 16384 2024-09-11T22:40:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.200+0000 7f4e2f0fa740 1 Processor -- start 2024-09-11T22:40:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.200+0000 7f4e2f0fa740 1 -- start start 2024-09-11T22:40:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3380 0x555e3b7be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 0x555e3b7ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 0x555e3b7b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281aba10 con 0x555e3b7b2a90 2024-09-11T22:40:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281ad580 con 0x555e3b7a3380 2024-09-11T22:40:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281ac0e0 con 0x555e3b7a3ca0 2024-09-11T22:40:20.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e1dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 0x555e3b7b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e1dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 0x555e3b7b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57678/0 (socket says 172.21.15.5:57678) 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/1146656151 learned_addr learned my addr 172.21.15.5:0/1146656151 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e2659c640 1 --2- 172.21.15.5:0/1146656151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 0x555e3b7ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1146656151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3380 0x555e3b7be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/1146656151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 msgr2=0x555e3b7ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1146656151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 0x555e3b7ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/1146656151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3380 msgr2=0x555e3b7be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1146656151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3380 0x555e3b7be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/1146656151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e3b7a4840 con 0x555e3b7b2a90 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e2659c640 1 --2- 172.21.15.5:0/1146656151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 0x555e3b7ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.201+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/1146656151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 0x555e3b7b16a0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f4e10001eb0 tx=0x7f4e10021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381b3ddea889bc50 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.202+0000 7f4e0ffff640 1 -- 172.21.15.5:0/1146656151 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10034040 con 0x555e3b7b2a90 2024-09-11T22:40:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.202+0000 7f4e0ffff640 1 -- 172.21.15.5:0/1146656151 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e1001f860 con 0x555e3b7b2a90 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.202+0000 7f4e0ffff640 1 -- 172.21.15.5:0/1146656151 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10003850 con 0x555e3b7b2a90 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.202+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1146656151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 msgr2=0x555e3b7b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.202+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1146656151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 0x555e3b7b16a0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f4e10001eb0 tx=0x7f4e10021040 comp rx=0 tx=0).stop 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.202+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1146656151 shutdown_connections 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.202+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1146656151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7b2a90 0x555e3b7b16a0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.202+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1146656151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 0x555e3b7ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.202+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/1146656151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7a3380 0x555e3b7be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.202+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1146656151 >> 172.21.15.5:0/1146656151 conn(0x555e3b79fa30 msgr2=0x555e3b7b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1146656151 shutdown_connections 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/1146656151 wait complete. 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 1 -- start start 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 1 Processor -- start 2024-09-11T22:40:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3380 0x555e3b7b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 0x555e3b7b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7b2a90 0x555e3b82c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281aceb0 con 0x555e3b7a3380 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281ac7e0 con 0x555e3b7b2a90 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4e281a9a10 con 0x555e3b7a3ca0 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2f0fa740 10 client.0 _ll_register_callbacks cb 0x7f4e18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e1dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3380 0x555e3b7b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e2659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 0x555e3b7b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e1dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3380 0x555e3b7b6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57688/0 (socket says 172.21.15.5:57688) 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.203+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/4057970452 learned_addr learned my addr 172.21.15.5:0/4057970452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7b2a90 0x555e3b82c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/4057970452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 msgr2=0x555e3b7b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:20.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 0x555e3b7b7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/4057970452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7b2a90 msgr2=0x555e3b82c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:20.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7b2a90 0x555e3b82c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e1dd9b640 1 -- 172.21.15.5:0/4057970452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e3b7b7960 con 0x555e3b7a3380 2024-09-11T22:40:20.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e1dd9b640 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3380 0x555e3b7b6b40 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f4e10011520 tx=0x7f4e10009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfaaf6e9a3c8938f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:20.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e0dffb640 1 -- 172.21.15.5:0/4057970452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10043060 con 0x555e3b7a3380 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e0dffb640 1 -- 172.21.15.5:0/4057970452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8673+0+0 (secure 0 0 0) 0x7f4e1001f660 con 0x555e3b7a3380 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/4057970452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1001f860 con 0x555e3b7a3380 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e0dffb640 1 -- 172.21.15.5:0/4057970452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4e10005110 con 0x555e3b7a3380 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e2f0fa740 10 client.21027 fetch_fsmap learned FSMap version 55 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.204+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/4057970452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x555e3b82d880 con 0x555e3b7a3380 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e0dffb640 1 -- 172.21.15.5:0/4057970452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e1001f860 con 0x555e3b7a3380 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e0dffb640 1 -- 172.21.15.5:0/4057970452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 55) ==== 8366+0+0 (secure 0 0 0) 0x7f4e1001d020 con 0x555e3b7a3380 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e2f0fa740 10 client.21027 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e2f0fa740 10 client.21027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e2f0fa740 20 client.21027 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e3b7c2560 0x555e3b82fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e2f0fa740 4 client.21027 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/4057970452 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e3b710d60 con 0x555e3b7c2560 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e2659c640 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e3b7c2560 0x555e3b82fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e2659c640 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e3b7c2560 0x555e3b82fd10 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d49277f5e5bcf9a5 server_cookie=2b5395aa4bbff0e1 in_seq=0 out_seq=0 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.205+0000 7f4e0dffb640 10 client.21027 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e0dffb640 1 -- 172.21.15.5:0/4057970452 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x555e3b710d60 con 0x555e3b7c2560 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e0dffb640 10 client.21027 handle_command_reply: tid=0 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 client.21027 shutdown 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/4057970452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3380 msgr2=0x555e3b7b6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3380 0x555e3b7b6b40 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f4e10011520 tx=0x7f4e10009910 comp rx=0 tx=0).stop 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/4057970452 shutdown_connections 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x555e3b7c2560 0x555e3b82fd10 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e3b7b2a90 0x555e3b82c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555e3b7a3ca0 0x555e3b7b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 --2- 172.21.15.5:0/4057970452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e3b7a3380 0x555e3b7b6b40 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e0dffb640 0 client.21027 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/4057970452 >> 172.21.15.5:0/4057970452 conn(0x555e3b79fa30 msgr2=0x555e3b7a56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/4057970452 shutdown_connections 2024-09-11T22:40:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 1 -- 172.21.15.5:0/4057970452 wait complete. 2024-09-11T22:40:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.207+0000 7f4e2f0fa740 20 client.21027 trim_cache size 0 max 16384 2024-09-11T22:40:20.214 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:40:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2024-09-11T22:40:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:40:20.214 INFO:teuthology.orchestra.run.smithi005.stdout: "73d962af-2d60-44de-ac0f-e71dc5e87afa": { 2024-09-11T22:40:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:40:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "73d962af-2d60-44de-ac0f-e71dc5e87afa", 2024-09-11T22:40:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:40:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:40:20.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:40:20.215 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 -- 172.21.15.5:0/1065307291 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e140a2cf0 msgr2=0x7f4e140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e140a2cf0 0x7f4e140a51b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4e20098520 tx=0x7f4e20096040 comp rx=0 tx=0).stop 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 -- 172.21.15.5:0/1065307291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2810c170 msgr2=0x7f4e2819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2810c170 0x7f4e2819cdb0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4e2000eaa0 tx=0x7f4e2000ef60 comp rx=0 tx=0).stop 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 -- 172.21.15.5:0/1065307291 shutdown_connections 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e2810c170 0x7f4e2819cdb0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e28103660 0x7f4e28198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f4e140a2cf0 0x7f4e140a51b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 --2- 172.21.15.5:0/1065307291 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e28102d40 0x7f4e281984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 -- 172.21.15.5:0/1065307291 >> 172.21.15.5:0/1065307291 conn(0x7f4e280fc7a0 msgr2=0x7f4e280fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 -- 172.21.15.5:0/1065307291 shutdown_connections 2024-09-11T22:40:20.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:20.208+0000 7f4e2cd21640 1 -- 172.21.15.5:0/1065307291 wait complete. 2024-09-11T22:40:20.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71636 15.94 MB/sec execute 158 sec latency 180.069 ms 2024-09-11T22:40:21.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:20 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:40:21.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:20 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:21.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:20 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:40:21.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:20 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:21.084 INFO:tasks.cephfs.filesystem:scrub status for tag:73d962af-2d60-44de-ac0f-e71dc5e87afa - {'path': '/', 'tag': '73d962af-2d60-44de-ac0f-e71dc5e87afa', 'options': 'recursive,force'} 2024-09-11T22:40:21.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:20 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:40:21.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:20 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:21.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72033 15.98 MB/sec execute 159 sec latency 189.873 ms 2024-09-11T22:40:22.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:21 smithi005 ceph-mon[30943]: pgmap v578: 97 pgs: 97 active+clean; 523 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.4 MiB/s wr, 399 op/s 2024-09-11T22:40:22.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:21 smithi159 ceph-mon[35534]: pgmap v578: 97 pgs: 97 active+clean; 523 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.4 MiB/s wr, 399 op/s 2024-09-11T22:40:22.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:21 smithi172 ceph-mon[35461]: pgmap v578: 97 pgs: 97 active+clean; 523 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.4 MiB/s wr, 399 op/s 2024-09-11T22:40:22.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72419 16.00 MB/sec execute 160 sec latency 163.958 ms 2024-09-11T22:40:23.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72915 16.01 MB/sec execute 161 sec latency 299.326 ms 2024-09-11T22:40:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:23 smithi005 ceph-mon[30943]: pgmap v579: 97 pgs: 97 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.2 MiB/s wr, 449 op/s 2024-09-11T22:40:23.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:23 smithi005 ceph-mon[30943]: osdmap e117: 12 total, 12 up, 12 in 2024-09-11T22:40:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:23 smithi172 ceph-mon[35461]: pgmap v579: 97 pgs: 97 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.2 MiB/s wr, 449 op/s 2024-09-11T22:40:23.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:23 smithi172 ceph-mon[35461]: osdmap e117: 12 total, 12 up, 12 in 2024-09-11T22:40:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:23 smithi159 ceph-mon[35534]: pgmap v579: 97 pgs: 97 active+clean; 532 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 4.2 MiB/s wr, 449 op/s 2024-09-11T22:40:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:23 smithi159 ceph-mon[35534]: osdmap e117: 12 total, 12 up, 12 in 2024-09-11T22:40:24.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 73364 16.06 MB/sec execute 162 sec latency 1299.366 ms 2024-09-11T22:40:25.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74062 16.25 MB/sec execute 163 sec latency 2299.414 ms 2024-09-11T22:40:25.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:25 smithi172 ceph-mon[35461]: pgmap v581: 97 pgs: 97 active+clean; 536 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 552 op/s 2024-09-11T22:40:26.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:25 smithi005 ceph-mon[30943]: pgmap v581: 97 pgs: 97 active+clean; 536 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 552 op/s 2024-09-11T22:40:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:25 smithi159 ceph-mon[35534]: pgmap v581: 97 pgs: 97 active+clean; 536 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 552 op/s 2024-09-11T22:40:26.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74451 16.34 MB/sec execute 164 sec latency 3299.488 ms 2024-09-11T22:40:27.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74802 16.38 MB/sec execute 165 sec latency 4299.531 ms 2024-09-11T22:40:27.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:27 smithi172 ceph-mon[35461]: pgmap v582: 97 pgs: 97 active+clean; 536 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 552 op/s 2024-09-11T22:40:28.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:27 smithi005 ceph-mon[30943]: pgmap v582: 97 pgs: 97 active+clean; 536 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 552 op/s 2024-09-11T22:40:28.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:27 smithi159 ceph-mon[35534]: pgmap v582: 97 pgs: 97 active+clean; 536 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 5.5 MiB/s wr, 552 op/s 2024-09-11T22:40:28.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75012 16.31 MB/sec execute 166 sec latency 5299.565 ms 2024-09-11T22:40:29.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75280 16.25 MB/sec execute 167 sec latency 6299.606 ms 2024-09-11T22:40:29.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:29 smithi172 ceph-mon[35461]: pgmap v583: 97 pgs: 97 active+clean; 556 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 453 op/s 2024-09-11T22:40:30.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:29 smithi005 ceph-mon[30943]: pgmap v583: 97 pgs: 97 active+clean; 556 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 453 op/s 2024-09-11T22:40:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:29 smithi159 ceph-mon[35534]: pgmap v583: 97 pgs: 97 active+clean; 556 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 453 op/s 2024-09-11T22:40:30.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75747 16.30 MB/sec execute 168 sec latency 7299.667 ms 2024-09-11T22:40:31.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76201 16.35 MB/sec execute 169 sec latency 8299.711 ms 2024-09-11T22:40:31.881 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:31 smithi159 ceph-mon[35534]: pgmap v584: 97 pgs: 97 active+clean; 556 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 453 op/s 2024-09-11T22:40:32.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:31 smithi005 ceph-mon[30943]: pgmap v584: 97 pgs: 97 active+clean; 556 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 453 op/s 2024-09-11T22:40:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:31 smithi172 ceph-mon[35461]: pgmap v584: 97 pgs: 97 active+clean; 556 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.6 MiB/s wr, 453 op/s 2024-09-11T22:40:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.335+0000 7f1360e65640 1 -- 172.21.15.5:0/2886914712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f135c1035a0 msgr2=0x7f135c10ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.335+0000 7f1360e65640 1 --2- 172.21.15.5:0/2886914712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f135c1035a0 0x7f135c10ba80 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f1344009900 tx=0x7f134402a480 comp rx=0 tx=0).stop 2024-09-11T22:40:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.335+0000 7f1360e65640 1 -- 172.21.15.5:0/2886914712 shutdown_connections 2024-09-11T22:40:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.335+0000 7f1360e65640 1 --2- 172.21.15.5:0/2886914712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f135c10c0d0 0x7f135c10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.335+0000 7f1360e65640 1 --2- 172.21.15.5:0/2886914712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f135c1035a0 0x7f135c10ba80 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.335+0000 7f1360e65640 1 --2- 172.21.15.5:0/2886914712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f135c102c80 0x7f135c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.335+0000 7f1360e65640 1 -- 172.21.15.5:0/2886914712 >> 172.21.15.5:0/2886914712 conn(0x7f135c077990 msgr2=0x7f135c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.335+0000 7f1360e65640 1 -- 172.21.15.5:0/2886914712 shutdown_connections 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.335+0000 7f1360e65640 1 -- 172.21.15.5:0/2886914712 wait complete. 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.336+0000 7f1360e65640 1 Processor -- start 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.336+0000 7f1360e65640 1 -- start start 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.336+0000 7f1360e65640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f135c102c80 0x7f135c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.336+0000 7f1360e65640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f135c1035a0 0x7f135c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.336+0000 7f1360e65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f135c10c0d0 0x7f135c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.336+0000 7f1360e65640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f135c111eb0 con 0x7f135c10c0d0 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.336+0000 7f1360e65640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f135c111d30 con 0x7f135c102c80 2024-09-11T22:40:32.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.336+0000 7f1360e65640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f135c112030 con 0x7f135c1035a0 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f135ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f135c10c0d0 0x7f135c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f135ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f135c10c0d0 0x7f135c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55354/0 (socket says 172.21.15.5:55354) 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f135a59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f135c102c80 0x7f135c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f135ad9d640 1 -- 172.21.15.5:0/3958941066 learned_addr learned my addr 172.21.15.5:0/3958941066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f1353fff640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f135c1035a0 0x7f135c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f135ad9d640 1 -- 172.21.15.5:0/3958941066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f135c1035a0 msgr2=0x7f135c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f135ad9d640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f135c1035a0 0x7f135c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f135ad9d640 1 -- 172.21.15.5:0/3958941066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f135c102c80 msgr2=0x7f135c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f135ad9d640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f135c102c80 0x7f135c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f135ad9d640 1 -- 172.21.15.5:0/3958941066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135c19d350 con 0x7f135c10c0d0 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f135ad9d640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f135c10c0d0 0x7f135c19cc20 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f135400efd0 tx=0x7f135400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:32.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f13537fe640 1 -- 172.21.15.5:0/3958941066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1354003b70 con 0x7f135c10c0d0 2024-09-11T22:40:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f13537fe640 1 -- 172.21.15.5:0/3958941066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1354003d10 con 0x7f135c10c0d0 2024-09-11T22:40:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f13537fe640 1 -- 172.21.15.5:0/3958941066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13540109f0 con 0x7f135c10c0d0 2024-09-11T22:40:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f1360e65640 1 -- 172.21.15.5:0/3958941066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135c19d640 con 0x7f135c10c0d0 2024-09-11T22:40:32.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.337+0000 7f1360e65640 1 -- 172.21.15.5:0/3958941066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135c1a4ed0 con 0x7f135c10c0d0 2024-09-11T22:40:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f13537fe640 1 -- 172.21.15.5:0/3958941066 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1354010b90 con 0x7f135c10c0d0 2024-09-11T22:40:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1363c07740 1 Processor -- start 2024-09-11T22:40:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1363c07740 1 -- start start 2024-09-11T22:40:32.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1363c07740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a00db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1363c07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1363c07740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1363c07740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5594d79652f0 con 0x5594d7a009d0 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1363c07740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5594d790d270 con 0x5594d7a100c0 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1363c07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594d79612b0 con 0x5594d7a012f0 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1353fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a00db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f135a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1353fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a00db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55356/0 (socket says 172.21.15.5:55356) 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f135a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36896/0 (socket says 172.21.15.5:36896) 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.340+0000 7f1353fff640 1 -- 172.21.15.5:0/2798249744 learned_addr learned my addr 172.21.15.5:0/2798249744 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f1353fff640 1 --2- 172.21.15.5:0/2798249744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f13537fe640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13480a2da0 0x7f13480a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f1353fff640 1 -- 172.21.15.5:0/2798249744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 msgr2=0x5594d7a0fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f1353fff640 1 --2- 172.21.15.5:0/2798249744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f1353fff640 1 -- 172.21.15.5:0/2798249744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 msgr2=0x5594d7a124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f1353fff640 1 --2- 172.21.15.5:0/2798249744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a124b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f1353fff640 1 -- 172.21.15.5:0/2798249744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594d790c340 con 0x5594d7a009d0 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f135ad9d640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13480a2da0 0x7f13480a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f13537fe640 1 -- 172.21.15.5:0/3958941066 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f13540cf8c0 con 0x7f135c10c0d0 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f1353fff640 1 --2- 172.21.15.5:0/2798249744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a00db0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f134402a9a0 tx=0x7f1344000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb92d40b3af6aff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f1350ff9640 1 -- 172.21.15.5:0/2798249744 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344044020 con 0x5594d7a009d0 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f1350ff9640 1 -- 172.21.15.5:0/2798249744 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13440035f0 con 0x5594d7a009d0 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.341+0000 7f1350ff9640 1 -- 172.21.15.5:0/2798249744 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344035040 con 0x5594d7a009d0 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 -- 172.21.15.5:0/2798249744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 msgr2=0x5594d7a00db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:32.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 --2- 172.21.15.5:0/2798249744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a00db0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f134402a9a0 tx=0x7f1344000bc0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f135ad9d640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13480a2da0 0x7f13480a5260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f13540cd050 tx=0x7f1354005b50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 -- 172.21.15.5:0/2798249744 shutdown_connections 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 --2- 172.21.15.5:0/2798249744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 --2- 172.21.15.5:0/2798249744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 --2- 172.21.15.5:0/2798249744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a00db0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 -- 172.21.15.5:0/2798249744 >> 172.21.15.5:0/2798249744 conn(0x5594d79fd060 msgr2=0x5594d79fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 -- 172.21.15.5:0/2798249744 shutdown_connections 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 -- 172.21.15.5:0/2798249744 wait complete. 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 -- start start 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.342+0000 7f1363c07740 1 Processor -- start 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1363c07740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a009d0 0x5594d7a0d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1363c07740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a012f0 0x5594d7a0d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1363c07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a100c0 0x5594d7a0db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1363c07740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5594d7a12c90 con 0x5594d7a012f0 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1363c07740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5594d78ee290 con 0x5594d7a009d0 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1363c07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5594d7a12e10 con 0x5594d7a100c0 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1363c07740 10 client.0 _ll_register_callbacks cb 0x5594d791f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f135a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a012f0 0x5594d7a0d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f135a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a012f0 0x5594d7a0d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55372/0 (socket says 172.21.15.5:55372) 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f135a59c640 1 -- 172.21.15.5:0/3905874912 learned_addr learned my addr 172.21.15.5:0/3905874912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1353fff640 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a009d0 0x5594d7a0d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1353fff640 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a100c0 0x5594d7a0db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f135a59c640 1 -- 172.21.15.5:0/3905874912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a100c0 msgr2=0x5594d7a0db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f135a59c640 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a100c0 0x5594d7a0db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f135a59c640 1 -- 172.21.15.5:0/3905874912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a009d0 msgr2=0x5594d7a0d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:32.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f135a59c640 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a009d0 0x5594d7a0d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f135a59c640 1 -- 172.21.15.5:0/3905874912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594d7678210 con 0x5594d7a012f0 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1353fff640 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a100c0 0x5594d7a0db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f135a59c640 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a012f0 0x5594d7a0d5d0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f134c012740 tx=0x7f134c012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7964b6cba0e0223 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.343+0000 7f1341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1341ffb640 1 -- 172.21.15.5:0/3905874912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c00f920 con 0x5594d7a012f0 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1341ffb640 1 -- 172.21.15.5:0/3905874912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f134c00fac0 con 0x5594d7a012f0 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1363c07740 1 -- 172.21.15.5:0/3905874912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13440035f0 con 0x5594d7a012f0 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1341ffb640 1 -- 172.21.15.5:0/3905874912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f134c00fcc0 con 0x5594d7a012f0 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1363c07740 10 client.21039 fetch_fsmap learned FSMap version 55 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1363c07740 1 -- 172.21.15.5:0/3905874912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5594d7744620 con 0x5594d7a012f0 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1341ffb640 1 -- 172.21.15.5:0/3905874912 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13440035f0 con 0x5594d7a012f0 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1341ffb640 1 -- 172.21.15.5:0/3905874912 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 55) ==== 8366+0+0 (secure 0 0 0) 0x7f134c023420 con 0x5594d7a012f0 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1363c07740 10 client.21039 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T22:40:32.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1363c07740 10 client.21039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1363c07740 20 client.21039 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1363c07740 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594d7a20120 0x5594d7a22570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1363c07740 4 client.21039 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1363c07740 1 -- 172.21.15.5:0/3905874912 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594d7972a20 con 0x5594d7a20120 2024-09-11T22:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.344+0000 7f1353fff640 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594d7a20120 0x5594d7a22570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.345+0000 7f1341ffb640 10 client.21039 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:32.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.345+0000 7f1353fff640 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594d7a20120 0x5594d7a22570 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d1aa6e1ad1d257 server_cookie=20629e345376fb8c in_seq=0 out_seq=0 2024-09-11T22:40:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1341ffb640 1 -- 172.21.15.5:0/3905874912 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594d7972a20 con 0x5594d7a20120 2024-09-11T22:40:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1341ffb640 10 client.21039 handle_command_reply: tid=0 2024-09-11T22:40:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 client.21039 shutdown 2024-09-11T22:40:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 -- 172.21.15.5:0/3905874912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a012f0 msgr2=0x5594d7a0d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a012f0 0x5594d7a0d5d0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f134c012740 tx=0x7f134c012c00 comp rx=0 tx=0).stop 2024-09-11T22:40:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 -- 172.21.15.5:0/3905874912 shutdown_connections 2024-09-11T22:40:32.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594d7a20120 0x5594d7a22570 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a100c0 0x5594d7a0db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1341ffb640 0 client.21039 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a012f0 0x5594d7a0d5d0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 --2- 172.21.15.5:0/3905874912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a009d0 0x5594d7a0d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 -- 172.21.15.5:0/3905874912 >> 172.21.15.5:0/3905874912 conn(0x5594d79fd060 msgr2=0x5594d7a02dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 -- 172.21.15.5:0/3905874912 shutdown_connections 2024-09-11T22:40:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 1 -- 172.21.15.5:0/3905874912 wait complete. 2024-09-11T22:40:32.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.347+0000 7f1363c07740 20 client.21039 trim_cache size 0 max 16384 2024-09-11T22:40:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1363c07740 1 Processor -- start 2024-09-11T22:40:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1363c07740 1 -- start start 2024-09-11T22:40:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1363c07740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a27e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1363c07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a283c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1363c07740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a0a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1363c07740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1ab6c0 con 0x5594d7a009d0 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1363c07740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1ad230 con 0x5594d7a100c0 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1363c07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1abd90 con 0x5594d7a012f0 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1353fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a27e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1353fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a27e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55374/0 (socket says 172.21.15.5:55374) 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1353fff640 1 -- 172.21.15.5:0/2198161074 learned_addr learned my addr 172.21.15.5:0/2198161074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f135a59c640 1 --2- 172.21.15.5:0/2198161074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a283c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1353fff640 1 --2- 172.21.15.5:0/2198161074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a0a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1353fff640 1 -- 172.21.15.5:0/2198161074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 msgr2=0x5594d7a283c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1353fff640 1 --2- 172.21.15.5:0/2198161074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a283c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1353fff640 1 -- 172.21.15.5:0/2198161074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 msgr2=0x5594d7a0a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1353fff640 1 --2- 172.21.15.5:0/2198161074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a0a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1353fff640 1 -- 172.21.15.5:0/2198161074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594d7a01120 con 0x5594d7a009d0 2024-09-11T22:40:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f135a59c640 1 --2- 172.21.15.5:0/2198161074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a283c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:40:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1353fff640 1 --2- 172.21.15.5:0/2198161074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a27e80 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f134402a4c0 tx=0x7f1344005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14244340e0e60a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.362+0000 7f1343fff640 1 -- 172.21.15.5:0/2198161074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344035050 con 0x5594d7a009d0 2024-09-11T22:40:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1343fff640 1 -- 172.21.15.5:0/2198161074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1344031a20 con 0x5594d7a009d0 2024-09-11T22:40:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1343fff640 1 -- 172.21.15.5:0/2198161074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344044040 con 0x5594d7a009d0 2024-09-11T22:40:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1363c07740 1 -- 172.21.15.5:0/2198161074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 msgr2=0x5594d7a27e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1363c07740 1 --2- 172.21.15.5:0/2198161074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a27e80 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f134402a4c0 tx=0x7f1344005a20 comp rx=0 tx=0).stop 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1363c07740 1 -- 172.21.15.5:0/2198161074 shutdown_connections 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1363c07740 1 --2- 172.21.15.5:0/2198161074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a0a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1363c07740 1 --2- 172.21.15.5:0/2198161074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a283c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1363c07740 1 --2- 172.21.15.5:0/2198161074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a27e80 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1363c07740 1 -- 172.21.15.5:0/2198161074 >> 172.21.15.5:0/2198161074 conn(0x5594d79fd060 msgr2=0x5594d7a0f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1363c07740 1 -- 172.21.15.5:0/2198161074 shutdown_connections 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.363+0000 7f1363c07740 1 -- 172.21.15.5:0/2198161074 wait complete. 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1363c07740 1 -- start start 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1363c07740 1 Processor -- start 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1363c07740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a0a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1363c07740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1363c07740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a0aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1363c07740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1acb60 con 0x5594d7a009d0 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1363c07740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1ac490 con 0x5594d7a100c0 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1363c07740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f135c1a96e0 con 0x5594d7a012f0 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1363c07740 10 client.0 _ll_register_callbacks cb 0x7f134c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1353fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a0a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f135a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f135a59c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:36932/0 (socket says 172.21.15.5:36932) 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f135a59c640 1 -- 172.21.15.5:0/2624805800 learned_addr learned my addr 172.21.15.5:0/2624805800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f1353fff640 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a0aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f135a59c640 1 -- 172.21.15.5:0/2624805800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 msgr2=0x5594d7a0aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f135a59c640 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a0aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f135a59c640 1 -- 172.21.15.5:0/2624805800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 msgr2=0x5594d7a0a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f135a59c640 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a0a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.364+0000 7f135a59c640 1 -- 172.21.15.5:0/2624805800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594d7a89e50 con 0x5594d7a012f0 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.365+0000 7f135a59c640 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0a550 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f134c0059f0 tx=0x7f134c00ff50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=558a711711e4d3fd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.365+0000 7f1341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.365+0000 7f1341ffb640 1 -- 172.21.15.5:0/2624805800 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c023cc0 con 0x5594d7a012f0 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.365+0000 7f1363c07740 1 -- 172.21.15.5:0/2624805800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1344031a20 con 0x5594d7a012f0 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.365+0000 7f1341ffb640 1 -- 172.21.15.5:0/2624805800 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f134c023420 con 0x5594d7a012f0 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.365+0000 7f1341ffb640 1 -- 172.21.15.5:0/2624805800 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f134c032050 con 0x5594d7a012f0 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.365+0000 7f1363c07740 10 client.29890 fetch_fsmap learned FSMap version 55 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.365+0000 7f1363c07740 1 -- 172.21.15.5:0/2624805800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5594d7a8b0f0 con 0x5594d7a012f0 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.365+0000 7f1341ffb640 1 -- 172.21.15.5:0/2624805800 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1344031a20 con 0x5594d7a012f0 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.365+0000 7f1341ffb640 1 -- 172.21.15.5:0/2624805800 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 55) ==== 8366+0+0 (secure 0 0 0) 0x7f134c003860 con 0x5594d7a012f0 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.366+0000 7f1363c07740 10 client.29890 fetch_fsmap finished waiting for FSMap version 55 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.366+0000 7f1363c07740 10 client.29890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:32.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.366+0000 7f1363c07740 20 client.29890 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.366+0000 7f1363c07740 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594d7a8b890 0x5594d7a8dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.366+0000 7f1363c07740 4 client.29890 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:40:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.366+0000 7f1363c07740 1 -- 172.21.15.5:0/2624805800 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594d796cd70 con 0x5594d7a8b890 2024-09-11T22:40:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.366+0000 7f1353fff640 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594d7a8b890 0x5594d7a8dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.366+0000 7f1341ffb640 10 client.29890 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:32.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.366+0000 7f1353fff640 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594d7a8b890 0x5594d7a8dcb0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef178fc12b63182c server_cookie=c63f26481edb44a7 in_seq=0 out_seq=0 2024-09-11T22:40:32.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76517 16.35 MB/sec execute 170 sec latency 8454.154 ms 2024-09-11T22:40:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.915+0000 7f1341ffb640 1 -- 172.21.15.5:0/2624805800 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594d796cd70 con 0x5594d7a8b890 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1341ffb640 10 client.29890 handle_command_reply: tid=0 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 client.29890 shutdown 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 -- 172.21.15.5:0/2624805800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 msgr2=0x5594d7a0a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0a550 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f134c0059f0 tx=0x7f134c00ff50 comp rx=0 tx=0).stop 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 -- 172.21.15.5:0/2624805800 shutdown_connections 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5594d7a8b890 0x5594d7a8dcb0 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5594d7a100c0 0x5594d7a0aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1341ffb640 0 client.29890 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5594d7a012f0 0x5594d7a0a550 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 --2- 172.21.15.5:0/2624805800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594d7a009d0 0x5594d7a0a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 -- 172.21.15.5:0/2624805800 >> 172.21.15.5:0/2624805800 conn(0x5594d79fd060 msgr2=0x5594d7a0d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 -- 172.21.15.5:0/2624805800 shutdown_connections 2024-09-11T22:40:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.916+0000 7f1363c07740 1 -- 172.21.15.5:0/2624805800 wait complete. 2024-09-11T22:40:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.917+0000 7f1363c07740 20 client.29890 trim_cache size 0 max 16384 2024-09-11T22:40:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:40:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:40:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:40:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:40:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.917+0000 7f1360e65640 1 -- 172.21.15.5:0/3958941066 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13480a2da0 msgr2=0x7f13480a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.917+0000 7f1360e65640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13480a2da0 0x7f13480a5260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f13540cd050 tx=0x7f1354005b50 comp rx=0 tx=0).stop 2024-09-11T22:40:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.917+0000 7f1360e65640 1 -- 172.21.15.5:0/3958941066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f135c10c0d0 msgr2=0x7f135c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.918+0000 7f1360e65640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f135c10c0d0 0x7f135c19cc20 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f135400efd0 tx=0x7f135400c6a0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.918+0000 7f1360e65640 1 -- 172.21.15.5:0/3958941066 shutdown_connections 2024-09-11T22:40:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.918+0000 7f1360e65640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f135c10c0d0 0x7f135c19cc20 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.918+0000 7f1360e65640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f135c1035a0 0x7f135c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.918+0000 7f1360e65640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f13480a2da0 0x7f13480a5260 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.918+0000 7f1360e65640 1 --2- 172.21.15.5:0/3958941066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f135c102c80 0x7f135c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.918+0000 7f1360e65640 1 -- 172.21.15.5:0/3958941066 >> 172.21.15.5:0/3958941066 conn(0x7f135c077990 msgr2=0x7f135c0fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.918+0000 7f1360e65640 1 -- 172.21.15.5:0/3958941066 shutdown_connections 2024-09-11T22:40:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:32.918+0000 7f1360e65640 1 -- 172.21.15.5:0/3958941066 wait complete. 2024-09-11T22:40:33.010 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:32 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:40:33.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:32 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:33.011 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:32 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:33.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:32 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:40:33.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:32 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:33.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:32 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:33.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:32 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:40:33.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:32 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:33.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:32 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:40:33.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76861 16.39 MB/sec execute 171 sec latency 520.961 ms 2024-09-11T22:40:34.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:33 smithi005 ceph-mon[30943]: pgmap v585: 97 pgs: 97 active+clean; 561 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.3 MiB/s wr, 424 op/s 2024-09-11T22:40:34.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:33 smithi159 ceph-mon[35534]: pgmap v585: 97 pgs: 97 active+clean; 561 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.3 MiB/s wr, 424 op/s 2024-09-11T22:40:34.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:33 smithi172 ceph-mon[35461]: pgmap v585: 97 pgs: 97 active+clean; 561 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 7.3 MiB/s wr, 424 op/s 2024-09-11T22:40:34.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77192 16.39 MB/sec execute 172 sec latency 230.017 ms 2024-09-11T22:40:35.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77547 16.40 MB/sec execute 173 sec latency 205.225 ms 2024-09-11T22:40:36.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:35 smithi005 ceph-mon[30943]: pgmap v586: 97 pgs: 97 active+clean; 573 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.9 MiB/s wr, 353 op/s 2024-09-11T22:40:36.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:35 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:40:36.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:35 smithi159 ceph-mon[35534]: pgmap v586: 97 pgs: 97 active+clean; 573 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.9 MiB/s wr, 353 op/s 2024-09-11T22:40:36.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:35 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:40:36.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:35 smithi172 ceph-mon[35461]: pgmap v586: 97 pgs: 97 active+clean; 573 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.9 MiB/s wr, 353 op/s 2024-09-11T22:40:36.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:35 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:40:36.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77975 16.42 MB/sec execute 174 sec latency 189.297 ms 2024-09-11T22:40:37.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78472 16.53 MB/sec execute 175 sec latency 181.163 ms 2024-09-11T22:40:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:37 smithi005 ceph-mon[30943]: pgmap v587: 97 pgs: 97 active+clean; 573 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.8 MiB/s wr, 342 op/s 2024-09-11T22:40:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:37 smithi159 ceph-mon[35534]: pgmap v587: 97 pgs: 97 active+clean; 573 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.8 MiB/s wr, 342 op/s 2024-09-11T22:40:38.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:37 smithi172 ceph-mon[35461]: pgmap v587: 97 pgs: 97 active+clean; 573 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.8 MiB/s wr, 342 op/s 2024-09-11T22:40:38.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78973 16.57 MB/sec execute 176 sec latency 168.517 ms 2024-09-11T22:40:39.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 79237 16.55 MB/sec execute 177 sec latency 300.136 ms 2024-09-11T22:40:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:39 smithi005 ceph-mon[30943]: pgmap v588: 97 pgs: 97 active+clean; 553 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.7 MiB/s wr, 660 op/s 2024-09-11T22:40:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:40:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:40:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:40:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:39 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:40:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:39 smithi159 ceph-mon[35534]: pgmap v588: 97 pgs: 97 active+clean; 553 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.7 MiB/s wr, 660 op/s 2024-09-11T22:40:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:40:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:40:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:40:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:39 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:40:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:39 smithi172 ceph-mon[35461]: pgmap v588: 97 pgs: 97 active+clean; 553 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.7 MiB/s wr, 660 op/s 2024-09-11T22:40:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:40:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:40:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:40:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:39 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:40:40.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 79572 16.58 MB/sec execute 178 sec latency 153.166 ms 2024-09-11T22:40:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:40 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:40:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:40 smithi005 ceph-mon[30943]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:40:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:40 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:40:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:40 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:40:41.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:40 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:40:41.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:40 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:40:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:40 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:40:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:40 smithi159 ceph-mon[35534]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:40:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:40 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:40:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:40 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:40:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:40 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:40:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:40 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:40:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:40 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:40:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:40 smithi172 ceph-mon[35461]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-11T22:40:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:40 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:40:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:40 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:40:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:40 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:40:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:40 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:40:41.651 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80010 16.64 MB/sec execute 179 sec latency 174.613 ms 2024-09-11T22:40:42.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:41 smithi005 ceph-mon[30943]: pgmap v589: 97 pgs: 97 active+clean; 553 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.4 MiB/s wr, 498 op/s 2024-09-11T22:40:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:41 smithi159 ceph-mon[35534]: pgmap v589: 97 pgs: 97 active+clean; 553 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.4 MiB/s wr, 498 op/s 2024-09-11T22:40:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:41 smithi172 ceph-mon[35461]: pgmap v589: 97 pgs: 97 active+clean; 553 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.4 MiB/s wr, 498 op/s 2024-09-11T22:40:42.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80354 16.64 MB/sec execute 180 sec latency 159.862 ms 2024-09-11T22:40:43.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80653 16.61 MB/sec execute 181 sec latency 172.279 ms 2024-09-11T22:40:44.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:43 smithi005 ceph-mon[30943]: pgmap v590: 97 pgs: 97 active+clean; 559 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.7 MiB/s wr, 573 op/s 2024-09-11T22:40:44.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:43 smithi159 ceph-mon[35534]: pgmap v590: 97 pgs: 97 active+clean; 559 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.7 MiB/s wr, 573 op/s 2024-09-11T22:40:44.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:43 smithi172 ceph-mon[35461]: pgmap v590: 97 pgs: 97 active+clean; 559 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.7 MiB/s wr, 573 op/s 2024-09-11T22:40:44.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80992 16.59 MB/sec execute 182 sec latency 357.515 ms 2024-09-11T22:40:45.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81385 16.62 MB/sec execute 183 sec latency 1074.263 ms 2024-09-11T22:40:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.018+0000 7fe9a50ba640 1 -- 172.21.15.5:0/631060119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a010c0b0 msgr2=0x7fe9a010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.018+0000 7fe9a50ba640 1 --2- 172.21.15.5:0/631060119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a010c0b0 0x7fe9a010e550 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fe998009ca0 tx=0x7fe99802a950 comp rx=0 tx=0).stop 2024-09-11T22:40:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.018+0000 7fe9a50ba640 1 -- 172.21.15.5:0/631060119 shutdown_connections 2024-09-11T22:40:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.018+0000 7fe9a50ba640 1 --2- 172.21.15.5:0/631060119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a010c0b0 0x7fe9a010e550 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.018+0000 7fe9a50ba640 1 --2- 172.21.15.5:0/631060119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a0103600 0x7fe9a010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.018+0000 7fe9a50ba640 1 --2- 172.21.15.5:0/631060119 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a0102ce0 0x7fe9a01030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.018+0000 7fe9a50ba640 1 -- 172.21.15.5:0/631060119 >> 172.21.15.5:0/631060119 conn(0x7fe9a0077990 msgr2=0x7fe9a00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.018+0000 7fe9a50ba640 1 -- 172.21.15.5:0/631060119 shutdown_connections 2024-09-11T22:40:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.018+0000 7fe9a50ba640 1 -- 172.21.15.5:0/631060119 wait complete. 2024-09-11T22:40:46.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.018+0000 7fe9a50ba640 1 Processor -- start 2024-09-11T22:40:46.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe9a50ba640 1 -- start start 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe9a50ba640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a0102ce0 0x7fe9a01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe9a50ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a0103600 0x7fe9a0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe9a50ba640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a010c0b0 0x7fe9a019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe9a50ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a0111e90 con 0x7fe9a0103600 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe9a50ba640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a0111d10 con 0x7fe9a0102ce0 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe9a50ba640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a0112010 con 0x7fe9a010c0b0 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe99e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a0103600 0x7fe9a0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe99e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a0103600 0x7fe9a0198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45442/0 (socket says 172.21.15.5:45442) 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe99e59c640 1 -- 172.21.15.5:0/16748398 learned_addr learned my addr 172.21.15.5:0/16748398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe99ed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a0102ce0 0x7fe9a01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe99f59e640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a010c0b0 0x7fe9a019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe99e59c640 1 -- 172.21.15.5:0/16748398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a010c0b0 msgr2=0x7fe9a019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe99e59c640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a010c0b0 0x7fe9a019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe99e59c640 1 -- 172.21.15.5:0/16748398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a0102ce0 msgr2=0x7fe9a01982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe99e59c640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a0102ce0 0x7fe9a01982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.019+0000 7fe99e59c640 1 -- 172.21.15.5:0/16748398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a019d2b0 con 0x7fe9a0103600 2024-09-11T22:40:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.020+0000 7fe99e59c640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a0103600 0x7fe9a0198800 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fe99400e9f0 tx=0x7fe99400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.020+0000 7fe987fff640 1 -- 172.21.15.5:0/16748398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99400cd80 con 0x7fe9a0103600 2024-09-11T22:40:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.020+0000 7fe9a50ba640 1 -- 172.21.15.5:0/16748398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a019d5a0 con 0x7fe9a0103600 2024-09-11T22:40:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.020+0000 7fe987fff640 1 -- 172.21.15.5:0/16748398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe994005500 con 0x7fe9a0103600 2024-09-11T22:40:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.020+0000 7fe9a50ba640 1 -- 172.21.15.5:0/16748398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a01a4e30 con 0x7fe9a0103600 2024-09-11T22:40:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.020+0000 7fe987fff640 1 -- 172.21.15.5:0/16748398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994021640 con 0x7fe9a0103600 2024-09-11T22:40:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.022+0000 7fe987fff640 1 -- 172.21.15.5:0/16748398 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe9940046c0 con 0x7fe9a0103600 2024-09-11T22:40:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe9a7e56740 1 Processor -- start 2024-09-11T22:40:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe987fff640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9880a2da0 0x7fe9880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe9a7e56740 1 -- start start 2024-09-11T22:40:46.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe99f59e640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9880a2da0 0x7fe9880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a9886db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe987fff640 1 -- 172.21.15.5:0/16748398 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7fe9940cf860 con 0x7fe9a0103600 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a9895b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a98984b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5644a97eb2f0 con 0x5644a98869d0 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5644a9793270 con 0x5644a98960c0 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5644a97e72b0 con 0x5644a98872f0 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe99ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a9886db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe99ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a9886db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45444/0 (socket says 172.21.15.5:45444) 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe99ed9d640 1 -- 172.21.15.5:0/1136400876 learned_addr learned my addr 172.21.15.5:0/1136400876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe99e59c640 1 --2- 172.21.15.5:0/1136400876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a9895b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.023+0000 7fe99f59e640 1 --2- 172.21.15.5:0/1136400876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a98984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe99e59c640 1 -- 172.21.15.5:0/1136400876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 msgr2=0x5644a98984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe99e59c640 1 --2- 172.21.15.5:0/1136400876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a98984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe99e59c640 1 -- 172.21.15.5:0/1136400876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 msgr2=0x5644a9886db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe99e59c640 1 --2- 172.21.15.5:0/1136400876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a9886db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe99e59c640 1 -- 172.21.15.5:0/1136400876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644a9792340 con 0x5644a98872f0 2024-09-11T22:40:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe99f59e640 1 --2- 172.21.15.5:0/1136400876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a98984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:40:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe99f59e640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9880a2da0 0x7fe9880a5260 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe998005e30 tx=0x7fe99802f940 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe99e59c640 1 --2- 172.21.15.5:0/1136400876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a9895b80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe9940ccf40 tx=0x7fe99409b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3e4c2f0cfe51681 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe9857fa640 1 -- 172.21.15.5:0/1136400876 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99409bd10 con 0x5644a98872f0 2024-09-11T22:40:46.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe9857fa640 1 -- 172.21.15.5:0/1136400876 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9940a00c0 con 0x5644a98872f0 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.024+0000 7fe9857fa640 1 -- 172.21.15.5:0/1136400876 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9940a4750 con 0x5644a98872f0 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 -- 172.21.15.5:0/1136400876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 msgr2=0x5644a9895b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/1136400876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a9895b80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe9940ccf40 tx=0x7fe99409b910 comp rx=0 tx=0).stop 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 -- 172.21.15.5:0/1136400876 shutdown_connections 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/1136400876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a98984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/1136400876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a9895b80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/1136400876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a9886db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 -- 172.21.15.5:0/1136400876 >> 172.21.15.5:0/1136400876 conn(0x5644a9883060 msgr2=0x5644a9883440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 -- 172.21.15.5:0/1136400876 shutdown_connections 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 -- 172.21.15.5:0/1136400876 wait complete. 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 -- start start 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.025+0000 7fe9a7e56740 1 Processor -- start 2024-09-11T22:40:46.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98869d0 0x5644a9893030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98872f0 0x5644a9893570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a9893ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5644a9898c30 con 0x5644a98872f0 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5644a9774290 con 0x5644a98960c0 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5644a9898db0 con 0x5644a98869d0 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe9a7e56740 10 client.0 _ll_register_callbacks cb 0x5644a97a5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98872f0 0x5644a9893570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98872f0 0x5644a9893570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45452/0 (socket says 172.21.15.5:45452) 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99e59c640 1 -- 172.21.15.5:0/3652842561 learned_addr learned my addr 172.21.15.5:0/3652842561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99ed9d640 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98869d0 0x5644a9893030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99f59e640 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a9893ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99e59c640 1 -- 172.21.15.5:0/3652842561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98869d0 msgr2=0x5644a9893030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99e59c640 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98869d0 0x5644a9893030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99e59c640 1 -- 172.21.15.5:0/3652842561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 msgr2=0x5644a9893ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99e59c640 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a9893ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99e59c640 1 -- 172.21.15.5:0/3652842561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644a94fe210 con 0x5644a98872f0 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe99e59c640 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98872f0 0x5644a9893570 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe9940ccff0 tx=0x7fe99409bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21baf0e0e94d284a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe982ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe982ff5640 1 -- 172.21.15.5:0/3652842561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9940a4700 con 0x5644a98872f0 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe982ff5640 1 -- 172.21.15.5:0/3652842561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7fe9940a4bd0 con 0x5644a98872f0 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.026+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3652842561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9940a00c0 con 0x5644a98872f0 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe982ff5640 1 -- 172.21.15.5:0/3652842561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe9940a4dd0 con 0x5644a98872f0 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe9a7e56740 10 client.21057 fetch_fsmap learned FSMap version 57 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3652842561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5644a95ca620 con 0x5644a98872f0 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe982ff5640 1 -- 172.21.15.5:0/3652842561 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9940a00c0 con 0x5644a98872f0 2024-09-11T22:40:46.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe982ff5640 1 -- 172.21.15.5:0/3652842561 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fe99409abd0 con 0x5644a98872f0 2024-09-11T22:40:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe9a7e56740 10 client.21057 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:40:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe9a7e56740 10 client.21057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe9a7e56740 20 client.21057 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5644a98a5eb0 0x5644a98a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe9a7e56740 4 client.21057 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3652842561 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644a97f8a20 con 0x5644a98a5eb0 2024-09-11T22:40:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.027+0000 7fe99ed9d640 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5644a98a5eb0 0x5644a98a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.028+0000 7fe99ed9d640 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5644a98a5eb0 0x5644a98a82a0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74791369a01a671 server_cookie=901a09f8faa3ac55 in_seq=0 out_seq=0 2024-09-11T22:40:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.028+0000 7fe982ff5640 10 client.21057 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe982ff5640 1 -- 172.21.15.5:0/3652842561 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644a97f8a20 con 0x5644a98a5eb0 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe982ff5640 10 client.21057 handle_command_reply: tid=0 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 client.21057 shutdown 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3652842561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98872f0 msgr2=0x5644a9893570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98872f0 0x5644a9893570 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fe9940ccff0 tx=0x7fe99409bde0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3652842561 shutdown_connections 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5644a98a5eb0 0x5644a98a82a0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a9893ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98872f0 0x5644a9893570 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe982ff5640 0 client.21057 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3652842561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98869d0 0x5644a9893030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3652842561 >> 172.21.15.5:0/3652842561 conn(0x5644a9883060 msgr2=0x5644a9888dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3652842561 shutdown_connections 2024-09-11T22:40:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3652842561 wait complete. 2024-09-11T22:40:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.030+0000 7fe9a7e56740 20 client.21057 trim_cache size 0 max 16384 2024-09-11T22:40:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.044+0000 7fe9a7e56740 1 Processor -- start 2024-09-11T22:40:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe9a7e56740 1 -- start start 2024-09-11T22:40:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a98ad930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a98ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a9890270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a01ab7c0 con 0x5644a98869d0 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a01ad3f0 con 0x5644a98960c0 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a01abe90 con 0x5644a98872f0 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a98ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a98ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45454/0 (socket says 172.21.15.5:45454) 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99ed9d640 1 -- 172.21.15.5:0/1928559473 learned_addr learned my addr 172.21.15.5:0/1928559473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99f59e640 1 --2- 172.21.15.5:0/1928559473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a9890270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99e59c640 1 --2- 172.21.15.5:0/1928559473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a98ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99ed9d640 1 -- 172.21.15.5:0/1928559473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 msgr2=0x5644a98ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99ed9d640 1 --2- 172.21.15.5:0/1928559473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a98ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99ed9d640 1 -- 172.21.15.5:0/1928559473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 msgr2=0x5644a9890270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99ed9d640 1 --2- 172.21.15.5:0/1928559473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a9890270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99ed9d640 1 -- 172.21.15.5:0/1928559473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644a9891fc0 con 0x5644a98869d0 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe99ed9d640 1 --2- 172.21.15.5:0/1928559473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a98ad930 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fe990009d70 tx=0x7fe990005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d715bf114d129cac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.045+0000 7fe984ff9640 1 -- 172.21.15.5:0/1928559473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990010430 con 0x5644a98869d0 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe984ff9640 1 -- 172.21.15.5:0/1928559473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe990010ab0 con 0x5644a98869d0 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe984ff9640 1 -- 172.21.15.5:0/1928559473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99001ea50 con 0x5644a98869d0 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe9a7e56740 1 -- 172.21.15.5:0/1928559473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 msgr2=0x5644a98ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/1928559473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a98ad930 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fe990009d70 tx=0x7fe990005f90 comp rx=0 tx=0).stop 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe9a7e56740 1 -- 172.21.15.5:0/1928559473 shutdown_connections 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/1928559473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98960c0 0x5644a9890270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/1928559473 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a98ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/1928559473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98869d0 0x5644a98ad930 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe9a7e56740 1 -- 172.21.15.5:0/1928559473 >> 172.21.15.5:0/1928559473 conn(0x5644a9883060 msgr2=0x5644a98950b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe9a7e56740 1 -- 172.21.15.5:0/1928559473 shutdown_connections 2024-09-11T22:40:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.046+0000 7fe9a7e56740 1 -- 172.21.15.5:0/1928559473 wait complete. 2024-09-11T22:40:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe9a7e56740 1 -- start start 2024-09-11T22:40:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe9a7e56740 1 Processor -- start 2024-09-11T22:40:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98869d0 0x5644a989a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a989aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe9a7e56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98960c0 0x5644a99088c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a01acd20 con 0x5644a98960c0 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a01ac590 con 0x5644a98869d0 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe9a7e56740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a01a97e0 con 0x5644a98872f0 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe9a7e56740 10 client.0 _ll_register_callbacks cb 0x7fe9900097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe99ed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98869d0 0x5644a989a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe99ed9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98869d0 0x5644a989a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37894/0 (socket says 172.21.15.5:37894) 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe99ed9d640 1 -- 172.21.15.5:0/3557929310 learned_addr learned my addr 172.21.15.5:0/3557929310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe99f59e640 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98960c0 0x5644a99088c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe99e59c640 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a989aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.047+0000 7fe99ed9d640 1 -- 172.21.15.5:0/3557929310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 msgr2=0x5644a989aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe99ed9d640 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a989aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe99ed9d640 1 -- 172.21.15.5:0/3557929310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98960c0 msgr2=0x5644a99088c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe99ed9d640 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98960c0 0x5644a99088c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:46.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe99ed9d640 1 -- 172.21.15.5:0/3557929310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644a99090f0 con 0x5644a98869d0 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe99f59e640 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98960c0 0x5644a99088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe99ed9d640 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98869d0 0x5644a989a990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe990009380 tx=0x7fe99001e480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d55b905f8763f89 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe982ff5640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe982ff5640 1 -- 172.21.15.5:0/3557929310 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990036060 con 0x5644a98869d0 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe982ff5640 1 -- 172.21.15.5:0/3557929310 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7fe9900043d0 con 0x5644a98869d0 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3557929310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe990010ab0 con 0x5644a98869d0 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe982ff5640 1 -- 172.21.15.5:0/3557929310 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe990010430 con 0x5644a98869d0 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe9a7e56740 10 client.30095 fetch_fsmap learned FSMap version 57 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.048+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3557929310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5644a9911450 con 0x5644a98869d0 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.049+0000 7fe982ff5640 1 -- 172.21.15.5:0/3557929310 <== mon.2 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe990010ab0 con 0x5644a98869d0 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.049+0000 7fe982ff5640 1 -- 172.21.15.5:0/3557929310 <== mon.2 v2:172.21.15.159:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fe990027420 con 0x5644a98869d0 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.049+0000 7fe9a7e56740 10 client.30095 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.049+0000 7fe9a7e56740 10 client.30095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:46.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.049+0000 7fe9a7e56740 20 client.30095 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.049+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5644a9911ac0 0x5644a9913f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.049+0000 7fe9a7e56740 4 client.30095 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.049+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3557929310 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644a97f2d70 con 0x5644a9911ac0 2024-09-11T22:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.049+0000 7fe99e59c640 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5644a9911ac0 0x5644a9913f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.050+0000 7fe982ff5640 10 client.30095 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:46.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:46.050+0000 7fe99e59c640 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5644a9911ac0 0x5644a9913f00 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a41ca0b096837a0c server_cookie=2529ec57320f7f91 in_seq=0 out_seq=0 2024-09-11T22:40:46.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:45 smithi005 ceph-mon[30943]: pgmap v591: 97 pgs: 97 active+clean; 568 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.3 MiB/s wr, 593 op/s 2024-09-11T22:40:46.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:45 smithi159 ceph-mon[35534]: pgmap v591: 97 pgs: 97 active+clean; 568 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.3 MiB/s wr, 593 op/s 2024-09-11T22:40:46.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:45 smithi172 ceph-mon[35461]: pgmap v591: 97 pgs: 97 active+clean; 568 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.3 MiB/s wr, 593 op/s 2024-09-11T22:40:46.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81676 16.63 MB/sec execute 184 sec latency 2074.306 ms 2024-09-11T22:40:47.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82102 16.63 MB/sec execute 185 sec latency 3074.335 ms 2024-09-11T22:40:48.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:47 smithi005 ceph-mon[30943]: pgmap v592: 97 pgs: 97 active+clean; 568 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-09-11T22:40:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:47 smithi159 ceph-mon[35534]: pgmap v592: 97 pgs: 97 active+clean; 568 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-09-11T22:40:48.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:47 smithi172 ceph-mon[35461]: pgmap v592: 97 pgs: 97 active+clean; 568 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 487 op/s 2024-09-11T22:40:48.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82558 16.71 MB/sec execute 186 sec latency 4074.367 ms 2024-09-11T22:40:49.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82956 16.73 MB/sec execute 187 sec latency 5074.402 ms 2024-09-11T22:40:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:49 smithi005 ceph-mon[30943]: pgmap v593: 97 pgs: 97 active+clean; 581 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.0 MiB/s wr, 797 op/s 2024-09-11T22:40:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:49 smithi159 ceph-mon[35534]: pgmap v593: 97 pgs: 97 active+clean; 581 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.0 MiB/s wr, 797 op/s 2024-09-11T22:40:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:49 smithi172 ceph-mon[35461]: pgmap v593: 97 pgs: 97 active+clean; 581 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.0 MiB/s wr, 797 op/s 2024-09-11T22:40:50.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83281 16.73 MB/sec execute 188 sec latency 6074.449 ms 2024-09-11T22:40:51.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:40:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:51.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83753 16.80 MB/sec execute 189 sec latency 7074.478 ms 2024-09-11T22:40:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:51 smithi005 ceph-mon[30943]: pgmap v594: 97 pgs: 97 active+clean; 581 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.1 MiB/s wr, 480 op/s 2024-09-11T22:40:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.930+0000 7f55c7a91640 1 -- 172.21.15.5:0/3661446838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55c0106c30 msgr2=0x7f55c0107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.930+0000 7f55c7a91640 1 --2- 172.21.15.5:0/3661446838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55c0106c30 0x7f55c0107010 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009860 tx=0x7f55bc02a340 comp rx=0 tx=0).stop 2024-09-11T22:40:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.930+0000 7f55c7a91640 1 -- 172.21.15.5:0/3661446838 shutdown_connections 2024-09-11T22:40:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.930+0000 7f55c7a91640 1 --2- 172.21.15.5:0/3661446838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55c0101b40 0x7f55c010da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.930+0000 7f55c7a91640 1 --2- 172.21.15.5:0/3661446838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c01011a0 0x7f55c0101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.930+0000 7f55c7a91640 1 --2- 172.21.15.5:0/3661446838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55c0106c30 0x7f55c0107010 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.930+0000 7f55c7a91640 1 -- 172.21.15.5:0/3661446838 >> 172.21.15.5:0/3661446838 conn(0x7f55c0077210 msgr2=0x7f55c00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.930+0000 7f55c7a91640 1 -- 172.21.15.5:0/3661446838 shutdown_connections 2024-09-11T22:40:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.930+0000 7f55c7a91640 1 -- 172.21.15.5:0/3661446838 wait complete. 2024-09-11T22:40:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.931+0000 7f55c7a91640 1 Processor -- start 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.931+0000 7f55c7a91640 1 -- start start 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.931+0000 7f55c7a91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c01011a0 0x7f55c0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.931+0000 7f55c7a91640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55c0101b40 0x7f55c01998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c582d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c01011a0 0x7f55c0199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c582d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c01011a0 0x7f55c0199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44826/0 (socket says 172.21.15.5:44826) 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c502c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55c0101b40 0x7f55c01998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.931+0000 7f55c7a91640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55c0106c30 0x7f55c01a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c502c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55c0101b40 0x7f55c01998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:58942/0 (socket says 172.21.15.5:58942) 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c582d640 1 -- 172.21.15.5:0/3564203873 learned_addr learned my addr 172.21.15.5:0/3564203873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55c0110fa0 con 0x7f55c01011a0 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f55c0110e20 con 0x7f55c0106c30 2024-09-11T22:40:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f55c0111120 con 0x7f55c0101b40 2024-09-11T22:40:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c602e640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55c0106c30 0x7f55c01a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c582d640 1 -- 172.21.15.5:0/3564203873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55c0101b40 msgr2=0x7f55c01998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c582d640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55c0101b40 0x7f55c01998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c582d640 1 -- 172.21.15.5:0/3564203873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55c0106c30 msgr2=0x7f55c01a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c582d640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55c0106c30 0x7f55c01a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.932+0000 7f55c582d640 1 -- 172.21.15.5:0/3564203873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c01a4930 con 0x7f55c01011a0 2024-09-11T22:40:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.933+0000 7f55c582d640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c01011a0 0x7f55c0199370 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f55bc035040 tx=0x7f55bc033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.933+0000 7f55b6ffd640 1 -- 172.21.15.5:0/3564203873 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc03f020 con 0x7f55c01011a0 2024-09-11T22:40:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.934+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c01a4b00 con 0x7f55c01011a0 2024-09-11T22:40:51.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.934+0000 7f55b6ffd640 1 -- 172.21.15.5:0/3564203873 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc032770 con 0x7f55c01011a0 2024-09-11T22:40:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.934+0000 7f55b6ffd640 1 -- 172.21.15.5:0/3564203873 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc037040 con 0x7f55c01011a0 2024-09-11T22:40:51.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.934+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c01a4ed0 con 0x7f55c01011a0 2024-09-11T22:40:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.937+0000 7f55c9e6a740 1 Processor -- start 2024-09-11T22:40:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.937+0000 7f55b6ffd640 1 -- 172.21.15.5:0/3564203873 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f55bc042020 con 0x7f55c01011a0 2024-09-11T22:40:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c9e6a740 1 -- start start 2024-09-11T22:40:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 0x5620a7ca0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca1000 0x5620a7c9b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620a7bf72e0 con 0x5620a7ca0650 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5620a7b9ece0 con 0x5620a7c9b670 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5620a7bf32a0 con 0x5620a7ca1000 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c502c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c582d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca1000 0x5620a7c9b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c502c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37748/0 (socket says 172.21.15.5:37748) 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c502c640 1 -- 172.21.15.5:0/2370375912 learned_addr learned my addr 172.21.15.5:0/2370375912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.938+0000 7f55c602e640 1 --2- 172.21.15.5:0/2370375912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 0x5620a7ca0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.939+0000 7f55c602e640 1 -- 172.21.15.5:0/2370375912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca1000 msgr2=0x5620a7c9b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.939+0000 7f55c602e640 1 --2- 172.21.15.5:0/2370375912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca1000 0x5620a7c9b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.939+0000 7f55c602e640 1 -- 172.21.15.5:0/2370375912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 msgr2=0x5620a7c9baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:51.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.939+0000 7f55c602e640 1 --2- 172.21.15.5:0/2370375912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.939+0000 7f55c602e640 1 -- 172.21.15.5:0/2370375912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a7b83440 con 0x5620a7ca0650 2024-09-11T22:40:51.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.939+0000 7f55c502c640 1 --2- 172.21.15.5:0/2370375912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:51.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.940+0000 7f55b6ffd640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55ac0a2cf0 0x7f55ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.940+0000 7f55b6ffd640 1 -- 172.21.15.5:0/3564203873 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f55bc0eef20 con 0x7f55c01011a0 2024-09-11T22:40:51.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.940+0000 7f55c582d640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55ac0a2cf0 0x7f55ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.940+0000 7f55c602e640 1 --2- 172.21.15.5:0/2370375912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 0x5620a7ca0a30 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009e50 tx=0x7f55b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a7bdc5f09b8e2f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:51.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.941+0000 7f55abfff640 1 -- 172.21.15.5:0/2370375912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8003b70 con 0x5620a7ca0650 2024-09-11T22:40:51.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.941+0000 7f55abfff640 1 -- 172.21.15.5:0/2370375912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8015070 con 0x5620a7ca0650 2024-09-11T22:40:51.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.941+0000 7f55abfff640 1 -- 172.21.15.5:0/2370375912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80117b0 con 0x5620a7ca0650 2024-09-11T22:40:51.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.942+0000 7f55c582d640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55ac0a2cf0 0x7f55ac0a51b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x5620a7ca13e0 tx=0x7f55bc0f29d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:51.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.942+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2370375912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 msgr2=0x5620a7ca0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:51.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.942+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2370375912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 0x5620a7ca0a30 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f55b8009e50 tx=0x7f55b80107a0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.942+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2370375912 shutdown_connections 2024-09-11T22:40:51.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.942+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2370375912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.942+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2370375912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca1000 0x5620a7c9b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.942+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2370375912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 0x5620a7ca0a30 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.942+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2370375912 >> 172.21.15.5:0/2370375912 conn(0x5620a7c8d4e0 msgr2=0x5620a7c8d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:51.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.943+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2370375912 shutdown_connections 2024-09-11T22:40:51.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.943+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2370375912 wait complete. 2024-09-11T22:40:51.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.943+0000 7f55c9e6a740 1 -- start start 2024-09-11T22:40:51.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.943+0000 7f55c9e6a740 1 Processor -- start 2024-09-11T22:40:51.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.944+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7ca6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.944+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca0650 0x5620a7ca41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.944+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 0x5620a7ca4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.944+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5620a7ca7a20 con 0x5620a7ca1000 2024-09-11T22:40:51.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5620a7bb2050 con 0x5620a7c9b670 2024-09-11T22:40:51.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c602e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7ca6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c602e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7ca6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37760/0 (socket says 172.21.15.5:37760) 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c602e640 1 -- 172.21.15.5:0/2954993110 learned_addr learned my addr 172.21.15.5:0/2954993110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2954993110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5620a7ca7ba0 con 0x5620a7ca0650 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c502c640 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca0650 0x5620a7ca41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c9e6a740 10 client.0 _ll_register_callbacks cb 0x5620a7bf2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c602e640 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 0x5620a7ca4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c502c640 1 -- 172.21.15.5:0/2954993110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 msgr2=0x5620a7ca6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c502c640 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7ca6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c502c640 1 -- 172.21.15.5:0/2954993110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 msgr2=0x5620a7ca4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c502c640 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 0x5620a7ca4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.945+0000 7f55c502c640 1 -- 172.21.15.5:0/2954993110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a79683c0 con 0x5620a7ca0650 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.946+0000 7f55c502c640 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca0650 0x5620a7ca41f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f55b0011670 tx=0x7f55b0011b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f58239a41b27635f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:51.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.946+0000 7f55a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:40:51.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.946+0000 7f55a97fa640 1 -- 172.21.15.5:0/2954993110 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b000fbf0 con 0x5620a7ca0650 2024-09-11T22:40:51.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.946+0000 7f55a97fa640 1 -- 172.21.15.5:0/2954993110 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f55b000fd90 con 0x5620a7ca0650 2024-09-11T22:40:51.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.946+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2954993110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b8015070 con 0x5620a7ca0650 2024-09-11T22:40:51.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.946+0000 7f55a97fa640 1 -- 172.21.15.5:0/2954993110 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f55b0027050 con 0x5620a7ca0650 2024-09-11T22:40:51.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.947+0000 7f55c9e6a740 10 client.29920 fetch_fsmap learned FSMap version 57 2024-09-11T22:40:51.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.947+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2954993110 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5620a7cb3580 con 0x5620a7ca0650 2024-09-11T22:40:51.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.947+0000 7f55a97fa640 1 -- 172.21.15.5:0/2954993110 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8015070 con 0x5620a7ca0650 2024-09-11T22:40:51.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.948+0000 7f55a97fa640 1 -- 172.21.15.5:0/2954993110 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f55b0022030 con 0x5620a7ca0650 2024-09-11T22:40:51.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.948+0000 7f55c9e6a740 10 client.29920 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:40:51.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.948+0000 7f55c9e6a740 10 client.29920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:51.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.948+0000 7f55c9e6a740 20 client.29920 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:51.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.948+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5620a7cb4b10 0x5620a7cb6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.948+0000 7f55c9e6a740 4 client.29920 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:51.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.948+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2954993110 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620a7c04850 con 0x5620a7cb4b10 2024-09-11T22:40:51.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.948+0000 7f55c602e640 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5620a7cb4b10 0x5620a7cb6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.949+0000 7f55a97fa640 10 client.29920 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:51.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.949+0000 7f55c602e640 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5620a7cb4b10 0x5620a7cb6f50 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f475ac0ce11c2ff server_cookie=a05511af60a65b3c in_seq=0 out_seq=0 2024-09-11T22:40:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.958+0000 7f55a97fa640 1 -- 172.21.15.5:0/2954993110 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620a7c04850 con 0x5620a7cb4b10 2024-09-11T22:40:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.958+0000 7f55a97fa640 10 client.29920 handle_command_reply: tid=0 2024-09-11T22:40:51.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.958+0000 7f55c9e6a740 1 client.29920 shutdown 2024-09-11T22:40:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.958+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2954993110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca0650 msgr2=0x5620a7ca41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.958+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca0650 0x5620a7ca41f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f55b0011670 tx=0x7f55b0011b30 comp rx=0 tx=0).stop 2024-09-11T22:40:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.958+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2954993110 shutdown_connections 2024-09-11T22:40:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.958+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5620a7cb4b10 0x5620a7cb6f50 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.959+0000 7f55a97fa640 0 client.29920 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.959+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 0x5620a7ca4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.959+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca0650 0x5620a7ca41f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.959+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2954993110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7ca6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.959+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2954993110 >> 172.21.15.5:0/2954993110 conn(0x5620a7c8d4e0 msgr2=0x5620a7c8cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:51.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.959+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2954993110 shutdown_connections 2024-09-11T22:40:51.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.959+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2954993110 wait complete. 2024-09-11T22:40:51.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.959+0000 7f55c9e6a740 20 client.29920 trim_cache size 0 max 16384 2024-09-11T22:40:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c9e6a740 1 Processor -- start 2024-09-11T22:40:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c9e6a740 1 -- start start 2024-09-11T22:40:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 0x5620a7cb0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c602e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca1000 0x5620a7c9fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01ab8c0 con 0x5620a7ca0650 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01ad4f0 con 0x5620a7c9b670 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01abf90 con 0x5620a7ca1000 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c602e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37766/0 (socket says 172.21.15.5:37766) 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.983+0000 7f55c602e640 1 -- 172.21.15.5:0/3648225327 learned_addr learned my addr 172.21.15.5:0/3648225327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.984+0000 7f55c502c640 1 --2- 172.21.15.5:0/3648225327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 0x5620a7cb0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.984+0000 7f55c602e640 1 --2- 172.21.15.5:0/3648225327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca1000 0x5620a7c9fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.984+0000 7f55c602e640 1 -- 172.21.15.5:0/3648225327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca1000 msgr2=0x5620a7c9fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.984+0000 7f55c602e640 1 --2- 172.21.15.5:0/3648225327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca1000 0x5620a7c9fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.984+0000 7f55c602e640 1 -- 172.21.15.5:0/3648225327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 msgr2=0x5620a7cb0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.984+0000 7f55c602e640 1 --2- 172.21.15.5:0/3648225327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 0x5620a7cb0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.984+0000 7f55c602e640 1 -- 172.21.15.5:0/3648225327 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620a7ca0da0 con 0x5620a7c9b670 2024-09-11T22:40:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.984+0000 7f55c502c640 1 --2- 172.21.15.5:0/3648225327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 0x5620a7cb0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:40:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.984+0000 7f55c602e640 1 --2- 172.21.15.5:0/3648225327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9b050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f55b80114a0 tx=0x7f55b8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d96f1ae434e42313 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.985+0000 7f55ab7fe640 1 -- 172.21.15.5:0/3648225327 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8025040 con 0x5620a7c9b670 2024-09-11T22:40:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.985+0000 7f55ab7fe640 1 -- 172.21.15.5:0/3648225327 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8015040 con 0x5620a7c9b670 2024-09-11T22:40:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.985+0000 7f55ab7fe640 1 -- 172.21.15.5:0/3648225327 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8005470 con 0x5620a7c9b670 2024-09-11T22:40:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.986+0000 7f55c9e6a740 1 -- 172.21.15.5:0/3648225327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 msgr2=0x5620a7c9b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.986+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/3648225327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9b050 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f55b80114a0 tx=0x7f55b8005fd0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.986+0000 7f55c9e6a740 1 -- 172.21.15.5:0/3648225327 shutdown_connections 2024-09-11T22:40:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.986+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/3648225327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7ca1000 0x5620a7c9fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.986+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/3648225327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca0650 0x5620a7cb0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.986+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/3648225327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7c9b670 0x5620a7c9b050 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.986+0000 7f55c9e6a740 1 -- 172.21.15.5:0/3648225327 >> 172.21.15.5:0/3648225327 conn(0x5620a7c8d4e0 msgr2=0x5620a7ca5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.986+0000 7f55c9e6a740 1 -- 172.21.15.5:0/3648225327 shutdown_connections 2024-09-11T22:40:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.986+0000 7f55c9e6a740 1 -- 172.21.15.5:0/3648225327 wait complete. 2024-09-11T22:40:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.987+0000 7f55c9e6a740 1 -- start start 2024-09-11T22:40:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.987+0000 7f55c9e6a740 1 Processor -- start 2024-09-11T22:40:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7c9b670 0x5620a7cbc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7ca0650 0x5620a7cbc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c9e6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 0x5620a7cbce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01ace20 con 0x5620a7ca1000 2024-09-11T22:40:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01ac690 con 0x5620a7ca0650 2024-09-11T22:40:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c9e6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01a98c0 con 0x5620a7c9b670 2024-09-11T22:40:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c9e6a740 10 client.0 _ll_register_callbacks cb 0x7f55b0009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c602e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 0x5620a7cbce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c602e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 0x5620a7cbce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44864/0 (socket says 172.21.15.5:44864) 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c602e640 1 -- 172.21.15.5:0/2821440511 learned_addr learned my addr 172.21.15.5:0/2821440511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c602e640 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7c9b670 0x5620a7cbc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c502c640 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7ca0650 0x5620a7cbc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c602e640 1 -- 172.21.15.5:0/2821440511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7c9b670 msgr2=0x5620a7cbc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c602e640 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7c9b670 0x5620a7cbc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c602e640 1 -- 172.21.15.5:0/2821440511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7ca0650 msgr2=0x5620a7cbc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c602e640 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7ca0650 0x5620a7cbc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.988+0000 7f55c602e640 1 -- 172.21.15.5:0/2821440511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620a7d1e790 con 0x5620a7ca1000 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.989+0000 7f55c602e640 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 0x5620a7cbce40 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f55b8000f80 tx=0x7f55b801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbaa20fcce11cdd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.989+0000 7f55a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.989+0000 7f55a97fa640 1 -- 172.21.15.5:0/2821440511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b8004db0 con 0x5620a7ca1000 2024-09-11T22:40:51.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.989+0000 7f55a97fa640 1 -- 172.21.15.5:0/2821440511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f55b8011880 con 0x5620a7ca1000 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.989+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2821440511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b8015040 con 0x5620a7ca1000 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.989+0000 7f55a97fa640 1 -- 172.21.15.5:0/2821440511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f55b8011a80 con 0x5620a7ca1000 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.989+0000 7f55c9e6a740 10 client.21081 fetch_fsmap learned FSMap version 57 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.989+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2821440511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5620a7d1fa60 con 0x5620a7ca1000 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.990+0000 7f55a97fa640 1 -- 172.21.15.5:0/2821440511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8015040 con 0x5620a7ca1000 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.990+0000 7f55a97fa640 1 -- 172.21.15.5:0/2821440511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f55b8035070 con 0x5620a7ca1000 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.990+0000 7f55c9e6a740 10 client.21081 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.990+0000 7f55c9e6a740 10 client.21081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.990+0000 7f55c9e6a740 20 client.21081 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.990+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5620a7cb4b10 0x5620a7d22050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.990+0000 7f55c9e6a740 4 client.21081 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.990+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2821440511 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620a7bfed60 con 0x5620a7cb4b10 2024-09-11T22:40:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.991+0000 7f55c502c640 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5620a7cb4b10 0x5620a7d22050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.991+0000 7f55a97fa640 10 client.21081 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.991+0000 7f55c502c640 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5620a7cb4b10 0x5620a7d22050 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bcd39a30dd17049 server_cookie=101b24230b56ad4a in_seq=0 out_seq=0 2024-09-11T22:40:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55a97fa640 1 -- 172.21.15.5:0/2821440511 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5620a7bfed60 con 0x5620a7cb4b10 2024-09-11T22:40:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55a97fa640 10 client.21081 handle_command_reply: tid=0 2024-09-11T22:40:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55c9e6a740 1 client.21081 shutdown 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2821440511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 msgr2=0x5620a7cbce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 0x5620a7cbce40 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f55b8000f80 tx=0x7f55b801e040 comp rx=0 tx=0).stop 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2821440511 shutdown_connections 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5620a7cb4b10 0x5620a7d22050 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620a7ca1000 0x5620a7cbce40 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5620a7ca0650 0x5620a7cbc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55c9e6a740 1 --2- 172.21.15.5:0/2821440511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5620a7c9b670 0x5620a7cbc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55a97fa640 0 client.21081 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.993+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2821440511 >> 172.21.15.5:0/2821440511 conn(0x5620a7c8d4e0 msgr2=0x5620a7ca6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.994+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2821440511 shutdown_connections 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.994+0000 7f55c9e6a740 1 -- 172.21.15.5:0/2821440511 wait complete. 2024-09-11T22:40:51.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.994+0000 7f55c9e6a740 20 client.21081 trim_cache size 0 max 16384 2024-09-11T22:40:51.998 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:40:51.998 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:40:51.998 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:40:51.998 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.995+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55ac0a2cf0 msgr2=0x7f55ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.995+0000 7f55c7a91640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55ac0a2cf0 0x7f55ac0a51b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x5620a7ca13e0 tx=0x7f55bc0f29d0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.995+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c01011a0 msgr2=0x7f55c0199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.995+0000 7f55c7a91640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c01011a0 0x7f55c0199370 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f55bc035040 tx=0x7f55bc033d90 comp rx=0 tx=0).stop 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.996+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 shutdown_connections 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.996+0000 7f55c7a91640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f55ac0a2cf0 0x7f55ac0a51b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.996+0000 7f55c7a91640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55c0106c30 0x7f55c01a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.996+0000 7f55c7a91640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55c0101b40 0x7f55c01998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.996+0000 7f55c7a91640 1 --2- 172.21.15.5:0/3564203873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c01011a0 0x7f55c0199370 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.996+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 >> 172.21.15.5:0/3564203873 conn(0x7f55c0077210 msgr2=0x7f55c00fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.997+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 shutdown_connections 2024-09-11T22:40:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:51.997+0000 7f55c7a91640 1 -- 172.21.15.5:0/3564203873 wait complete. 2024-09-11T22:40:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:51 smithi159 ceph-mon[35534]: pgmap v594: 97 pgs: 97 active+clean; 581 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.1 MiB/s wr, 480 op/s 2024-09-11T22:40:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:51 smithi172 ceph-mon[35461]: pgmap v594: 97 pgs: 97 active+clean; 581 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 6.1 MiB/s wr, 480 op/s 2024-09-11T22:40:52.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84007 16.78 MB/sec execute 190 sec latency 8074.518 ms 2024-09-11T22:40:52.842 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:40:52.842 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:40:53.153 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:53.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84457 16.79 MB/sec execute 191 sec latency 9074.563 ms 2024-09-11T22:40:53.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 -- 172.21.15.5:0/1866847933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e68101430 msgr2=0x7f8e6810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 --2- 172.21.15.5:0/1866847933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e68101430 0x7f8e6810d960 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f8e60009cd0 tx=0x7f8e6002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 -- 172.21.15.5:0/1866847933 shutdown_connections 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 --2- 172.21.15.5:0/1866847933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e68101430 0x7f8e6810d960 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 --2- 172.21.15.5:0/1866847933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e68100a90 0x7f8e68100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 --2- 172.21.15.5:0/1866847933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68106a90 0x7f8e68106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 -- 172.21.15.5:0/1866847933 >> 172.21.15.5:0/1866847933 conn(0x7f8e680fc7d0 msgr2=0x7f8e680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 -- 172.21.15.5:0/1866847933 shutdown_connections 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 -- 172.21.15.5:0/1866847933 wait complete. 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 Processor -- start 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.706+0000 7f8e70571640 1 -- start start 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e70571640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e68100a90 0x7f8e6819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e70571640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101430 0x7f8e6819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e70571640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e68106a90 0x7f8e68198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e70571640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68110e90 con 0x7f8e68100a90 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e70571640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68110d10 con 0x7f8e68101430 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e70571640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68111010 con 0x7f8e68106a90 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6e30d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e68100a90 0x7f8e6819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6e30d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e68100a90 0x7f8e6819d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44882/0 (socket says 172.21.15.5:44882) 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6e30d640 1 -- 172.21.15.5:0/794493517 learned_addr learned my addr 172.21.15.5:0/794493517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6db0c640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101430 0x7f8e6819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6eb0e640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e68106a90 0x7f8e68198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6e30d640 1 -- 172.21.15.5:0/794493517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e68106a90 msgr2=0x7f8e68198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6e30d640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e68106a90 0x7f8e68198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6e30d640 1 -- 172.21.15.5:0/794493517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101430 msgr2=0x7f8e6819d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6e30d640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101430 0x7f8e6819d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6e30d640 1 -- 172.21.15.5:0/794493517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e68198cb0 con 0x7f8e68100a90 2024-09-11T22:40:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6db0c640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101430 0x7f8e6819d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.707+0000 7f8e6e30d640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e68100a90 0x7f8e6819d010 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400ea60 tx=0x7f8e6400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.708+0000 7f8e5f7fe640 1 -- 172.21.15.5:0/794493517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6400cde0 con 0x7f8e68100a90 2024-09-11T22:40:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.708+0000 7f8e5f7fe640 1 -- 172.21.15.5:0/794493517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e640035f0 con 0x7f8e68100a90 2024-09-11T22:40:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.708+0000 7f8e70571640 1 -- 172.21.15.5:0/794493517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e68198fa0 con 0x7f8e68100a90 2024-09-11T22:40:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.708+0000 7f8e70571640 1 -- 172.21.15.5:0/794493517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e681a4f40 con 0x7f8e68100a90 2024-09-11T22:40:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.708+0000 7f8e5f7fe640 1 -- 172.21.15.5:0/794493517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64010670 con 0x7f8e68100a90 2024-09-11T22:40:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.709+0000 7f8e70571640 1 -- 172.21.15.5:0/794493517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e68100ef0 con 0x7f8e68100a90 2024-09-11T22:40:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.710+0000 7f8e5f7fe640 1 -- 172.21.15.5:0/794493517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8e640037a0 con 0x7f8e68100a90 2024-09-11T22:40:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.711+0000 7f8e5f7fe640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e540a2cf0 0x7f8e540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.711+0000 7f8e5f7fe640 1 -- 172.21.15.5:0/794493517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f8e640cf6b0 con 0x7f8e68100a90 2024-09-11T22:40:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.711+0000 7f8e6db0c640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e540a2cf0 0x7f8e540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.712+0000 7f8e6db0c640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e540a2cf0 0x7f8e540a51b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8e681995b0 tx=0x7f8e58005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:53.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.716+0000 7f8e5f7fe640 1 -- 172.21.15.5:0/794493517 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e6409c260 con 0x7f8e68100a90 2024-09-11T22:40:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.961+0000 7f8e70571640 1 -- 172.21.15.5:0/794493517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e681a51a0 con 0x7f8e68100a90 2024-09-11T22:40:53.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:53 smithi005 ceph-mon[30943]: pgmap v595: 97 pgs: 97 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.1 MiB/s wr, 595 op/s 2024-09-11T22:40:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.961+0000 7f8e5f7fe640 1 -- 172.21.15.5:0/794493517 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11975 (secure 0 0 0) 0x7f8e6409bb60 con 0x7f8e68100a90 2024-09-11T22:40:53.963 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:40:53.963 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":57,"btime":"2024-09-11T22:40:39:632093+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:40:38.909431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:40:53.963 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 57 2024-09-11T22:40:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.964+0000 7f8e70571640 1 -- 172.21.15.5:0/794493517 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e540a2cf0 msgr2=0x7f8e540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.964+0000 7f8e70571640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e540a2cf0 0x7f8e540a51b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8e681995b0 tx=0x7f8e58005f90 comp rx=0 tx=0).stop 2024-09-11T22:40:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.964+0000 7f8e70571640 1 -- 172.21.15.5:0/794493517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e68100a90 msgr2=0x7f8e6819d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.964+0000 7f8e70571640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e68100a90 0x7f8e6819d010 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400ea60 tx=0x7f8e6400ef20 comp rx=0 tx=0).stop 2024-09-11T22:40:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.964+0000 7f8e70571640 1 -- 172.21.15.5:0/794493517 shutdown_connections 2024-09-11T22:40:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.964+0000 7f8e70571640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8e68106a90 0x7f8e68198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.964+0000 7f8e70571640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f8e540a2cf0 0x7f8e540a51b0 secure :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8e681995b0 tx=0x7f8e58005f90 comp rx=0 tx=0).stop 2024-09-11T22:40:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.964+0000 7f8e70571640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8e68101430 0x7f8e6819d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.964+0000 7f8e70571640 1 --2- 172.21.15.5:0/794493517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e68100a90 0x7f8e6819d010 secure :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f8e6400ea60 tx=0x7f8e6400ef20 comp rx=0 tx=0).stop 2024-09-11T22:40:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.964+0000 7f8e70571640 1 -- 172.21.15.5:0/794493517 >> 172.21.15.5:0/794493517 conn(0x7f8e680fc7d0 msgr2=0x7f8e681045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.965+0000 7f8e70571640 1 -- 172.21.15.5:0/794493517 shutdown_connections 2024-09-11T22:40:53.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:53.966+0000 7f8e70571640 1 -- 172.21.15.5:0/794493517 wait complete. 2024-09-11T22:40:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe982ff5640 1 -- 172.21.15.5:0/3557929310 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644a97f2d70 con 0x5644a9911ac0 2024-09-11T22:40:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe982ff5640 10 client.30095 handle_command_reply: tid=0 2024-09-11T22:40:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 client.30095 shutdown 2024-09-11T22:40:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3557929310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98869d0 msgr2=0x5644a989a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98869d0 0x5644a989a990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe990009380 tx=0x7fe99001e480 comp rx=0 tx=0).stop 2024-09-11T22:40:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3557929310 shutdown_connections 2024-09-11T22:40:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5644a9911ac0 0x5644a9913f00 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644a98960c0 0x5644a99088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a98872f0 0x5644a989aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 --2- 172.21.15.5:0/3557929310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5644a98869d0 0x5644a989a990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3557929310 >> 172.21.15.5:0/3557929310 conn(0x5644a9883060 msgr2=0x5644a98819a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe982ff5640 0 client.30095 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:40:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3557929310 shutdown_connections 2024-09-11T22:40:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.023+0000 7fe9a7e56740 1 -- 172.21.15.5:0/3557929310 wait complete. 2024-09-11T22:40:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.024+0000 7fe9a7e56740 20 client.30095 trim_cache size 0 max 16384 2024-09-11T22:40:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:40:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:40:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:40:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:40:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.024+0000 7fe9a50ba640 1 -- 172.21.15.5:0/16748398 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9880a2da0 msgr2=0x7fe9880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.024+0000 7fe9a50ba640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9880a2da0 0x7fe9880a5260 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe998005e30 tx=0x7fe99802f940 comp rx=0 tx=0).stop 2024-09-11T22:40:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.024+0000 7fe9a50ba640 1 -- 172.21.15.5:0/16748398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a0103600 msgr2=0x7fe9a0198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.024+0000 7fe9a50ba640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a0103600 0x7fe9a0198800 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fe99400e9f0 tx=0x7fe99400edf0 comp rx=0 tx=0).stop 2024-09-11T22:40:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.025+0000 7fe9a50ba640 1 -- 172.21.15.5:0/16748398 shutdown_connections 2024-09-11T22:40:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.025+0000 7fe9a50ba640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9a010c0b0 0x7fe9a019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.025+0000 7fe9a50ba640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9a0103600 0x7fe9a0198800 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.025+0000 7fe9a50ba640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9880a2da0 0x7fe9880a5260 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.025+0000 7fe9a50ba640 1 --2- 172.21.15.5:0/16748398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9a0102ce0 0x7fe9a01982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.025+0000 7fe9a50ba640 1 -- 172.21.15.5:0/16748398 >> 172.21.15.5:0/16748398 conn(0x7fe9a0077990 msgr2=0x7fe9a00fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.025+0000 7fe9a50ba640 1 -- 172.21.15.5:0/16748398 shutdown_connections 2024-09-11T22:40:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:40:54.025+0000 7fe9a50ba640 1 -- 172.21.15.5:0/16748398 wait complete. 2024-09-11T22:40:54.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:53 smithi159 ceph-mon[35534]: pgmap v595: 97 pgs: 97 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.1 MiB/s wr, 595 op/s 2024-09-11T22:40:54.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:53 smithi172 ceph-mon[35461]: pgmap v595: 97 pgs: 97 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.1 MiB/s wr, 595 op/s 2024-09-11T22:40:54.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84710 16.78 MB/sec execute 192 sec latency 9582.870 ms 2024-09-11T22:40:54.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:40:54.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:54 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/794493517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:40:55.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:54 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/794493517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:40:55.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:54 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/794493517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:40:55.277 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:55.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85180 16.84 MB/sec execute 193 sec latency 213.830 ms 2024-09-11T22:40:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.800+0000 7fc2b749e640 1 -- 172.21.15.5:0/3500605703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b01027a0 msgr2=0x7fc2b01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.800+0000 7fc2b749e640 1 --2- 172.21.15.5:0/3500605703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b01027a0 0x7fc2b01067f0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009900 tx=0x7fc2a402a410 comp rx=0 tx=0).stop 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.800+0000 7fc2b749e640 1 -- 172.21.15.5:0/3500605703 shutdown_connections 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.800+0000 7fc2b749e640 1 --2- 172.21.15.5:0/3500605703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2b0106f20 0x7fc2b010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.800+0000 7fc2b749e640 1 --2- 172.21.15.5:0/3500605703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b01027a0 0x7fc2b01067f0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.800+0000 7fc2b749e640 1 --2- 172.21.15.5:0/3500605703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2b0101df0 0x7fc2b01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.800+0000 7fc2b749e640 1 -- 172.21.15.5:0/3500605703 >> 172.21.15.5:0/3500605703 conn(0x7fc2b00fd8f0 msgr2=0x7fc2b00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.800+0000 7fc2b749e640 1 -- 172.21.15.5:0/3500605703 shutdown_connections 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.800+0000 7fc2b749e640 1 -- 172.21.15.5:0/3500605703 wait complete. 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b749e640 1 Processor -- start 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b749e640 1 -- start start 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b749e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2b0101df0 0x7fc2b01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b749e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2b01027a0 0x7fc2b0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b749e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b0106f20 0x7fc2b019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b749e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0111f90 con 0x7fc2b0106f20 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b749e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0111e10 con 0x7fc2b0101df0 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b749e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0112110 con 0x7fc2b01027a0 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b5a3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b0106f20 0x7fc2b019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b523a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2b0101df0 0x7fc2b01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b4a39640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2b01027a0 0x7fc2b0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b5a3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b0106f20 0x7fc2b019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44900/0 (socket says 172.21.15.5:44900) 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b523a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2b0101df0 0x7fc2b01984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37810/0 (socket says 172.21.15.5:37810) 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b5a3b640 1 -- 172.21.15.5:0/3873735245 learned_addr learned my addr 172.21.15.5:0/3873735245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b5a3b640 1 -- 172.21.15.5:0/3873735245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2b01027a0 msgr2=0x7fc2b0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b5a3b640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2b01027a0 0x7fc2b0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b5a3b640 1 -- 172.21.15.5:0/3873735245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2b0101df0 msgr2=0x7fc2b01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b5a3b640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2b0101df0 0x7fc2b01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.801+0000 7fc2b5a3b640 1 -- 172.21.15.5:0/3873735245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b019d480 con 0x7fc2b0106f20 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.802+0000 7fc2b5a3b640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b0106f20 0x7fc2b019cda0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800eb10 tx=0x7fc2a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.802+0000 7fc2a27fc640 1 -- 172.21.15.5:0/3873735245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8004b20 con 0x7fc2b0106f20 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.802+0000 7fc2a27fc640 1 -- 172.21.15.5:0/3873735245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2a8004cc0 con 0x7fc2b0106f20 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.802+0000 7fc2b749e640 1 -- 172.21.15.5:0/3873735245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b019d770 con 0x7fc2b0106f20 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.802+0000 7fc2a27fc640 1 -- 172.21.15.5:0/3873735245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8010760 con 0x7fc2b0106f20 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.802+0000 7fc2b749e640 1 -- 172.21.15.5:0/3873735245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b01a5050 con 0x7fc2b0106f20 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.805+0000 7fc2b9877740 1 Processor -- start 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.805+0000 7fc2a27fc640 1 -- 172.21.15.5:0/3873735245 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fc2a8005020 con 0x7fc2b0106f20 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.805+0000 7fc2b9877740 1 -- start start 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 0x558592ef0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592ef0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592eff780 0x558592f01b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b9877740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558592e562f0 con 0x558592ef05d0 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b9877740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558592dfdcf0 con 0x558592eff780 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b9877740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558592e522b0 con 0x558592ef1b40 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b523a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592ef0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b523a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592ef0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44916/0 (socket says 172.21.15.5:44916) 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b523a640 1 -- 172.21.15.5:0/4089562867 learned_addr learned my addr 172.21.15.5:0/4089562867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b523a640 1 -- 172.21.15.5:0/4089562867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 msgr2=0x558592ef0090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2a27fc640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc29c0a2d70 0x7fc29c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b523a640 1 --2- 172.21.15.5:0/4089562867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 0x558592ef0090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b523a640 1 -- 172.21.15.5:0/4089562867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592eff780 msgr2=0x558592f01b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b523a640 1 --2- 172.21.15.5:0/4089562867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592eff780 0x558592f01b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b5a3b640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc29c0a2d70 0x7fc29c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.806+0000 7fc2b523a640 1 -- 172.21.15.5:0/4089562867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558592de23a0 con 0x558592ef05d0 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.807+0000 7fc2a27fc640 1 -- 172.21.15.5:0/3873735245 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7fc2a80cf970 con 0x7fc2b0106f20 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.807+0000 7fc2b523a640 1 --2- 172.21.15.5:0/4089562867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592ef0a10 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009cf0 tx=0x7fc2ac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503741b7769b3e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.807+0000 7fc29b7fe640 1 -- 172.21.15.5:0/4089562867 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac01e070 con 0x558592ef05d0 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.807+0000 7fc29b7fe640 1 -- 172.21.15.5:0/4089562867 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac009de0 con 0x558592ef05d0 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.807+0000 7fc29b7fe640 1 -- 172.21.15.5:0/4089562867 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0196b0 con 0x558592ef05d0 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 -- 172.21.15.5:0/4089562867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 msgr2=0x558592ef0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 --2- 172.21.15.5:0/4089562867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592ef0a10 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac009cf0 tx=0x7fc2ac011810 comp rx=0 tx=0).stop 2024-09-11T22:40:55.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b5a3b640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc29c0a2d70 0x7fc29c0a5230 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc2a809bb00 tx=0x7fc2a800efd0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 -- 172.21.15.5:0/4089562867 shutdown_connections 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 --2- 172.21.15.5:0/4089562867 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592eff780 0x558592f01b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 --2- 172.21.15.5:0/4089562867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592ef0a10 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 --2- 172.21.15.5:0/4089562867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 0x558592ef0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 -- 172.21.15.5:0/4089562867 >> 172.21.15.5:0/4089562867 conn(0x558592eb9310 msgr2=0x558592eecc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 -- 172.21.15.5:0/4089562867 shutdown_connections 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 -- 172.21.15.5:0/4089562867 wait complete. 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 -- start start 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.808+0000 7fc2b9877740 1 Processor -- start 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592eff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592efc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592eff780 0x558592efcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b9877740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558592f02480 con 0x558592ef1b40 2024-09-11T22:40:55.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b9877740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558592e11060 con 0x558592ef05d0 2024-09-11T22:40:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b9877740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x558592f02600 con 0x558592eff780 2024-09-11T22:40:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b9877740 10 client.0 _ll_register_callbacks cb 0x558592b648c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b523a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592efc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b4a39640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592eff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b523a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592efc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44928/0 (socket says 172.21.15.5:44928) 2024-09-11T22:40:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b523a640 1 -- 172.21.15.5:0/1528012446 learned_addr learned my addr 172.21.15.5:0/1528012446 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b4a39640 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592eff780 0x558592efcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b523a640 1 -- 172.21.15.5:0/1528012446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592eff780 msgr2=0x558592efcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.809+0000 7fc2b523a640 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592eff780 0x558592efcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc2b523a640 1 -- 172.21.15.5:0/1528012446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 msgr2=0x558592eff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc2b523a640 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592eff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc2b523a640 1 -- 172.21.15.5:0/1528012446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558592bc73c0 con 0x558592ef1b40 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc2b4a39640 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592eff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc2b523a640 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592efc4c0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac000c00 tx=0x7fc2ac005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0cb1fe78c11091 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc298ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc298ff9640 1 -- 172.21.15.5:0/1528012446 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac027060 con 0x558592ef1b40 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc2b9877740 1 -- 172.21.15.5:0/1528012446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ac009de0 con 0x558592ef1b40 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc298ff9640 1 -- 172.21.15.5:0/1528012446 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7fc2ac005390 con 0x558592ef1b40 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc298ff9640 1 -- 172.21.15.5:0/1528012446 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac01e040 con 0x558592ef1b40 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc2b9877740 10 client.21099 fetch_fsmap learned FSMap version 57 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc2b9877740 1 -- 172.21.15.5:0/1528012446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x558592f0dfd0 con 0x558592ef1b40 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.810+0000 7fc298ff9640 1 -- 172.21.15.5:0/1528012446 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac009de0 con 0x558592ef1b40 2024-09-11T22:40:55.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.811+0000 7fc298ff9640 1 -- 172.21.15.5:0/1528012446 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fc2ac021070 con 0x558592ef1b40 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.811+0000 7fc2b9877740 10 client.21099 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.811+0000 7fc2b9877740 10 client.21099 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.811+0000 7fc2b9877740 20 client.21099 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.811+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558592f0f320 0x558592f11710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.811+0000 7fc2b9877740 4 client.21099 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.811+0000 7fc2b9877740 1 -- 172.21.15.5:0/1528012446 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558592e63860 con 0x558592f0f320 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.811+0000 7fc2b4a39640 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558592f0f320 0x558592f11710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.811+0000 7fc2b4a39640 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558592f0f320 0x558592f11710 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=86d6dfdaeeac1c29 server_cookie=553d386ff794fe68 in_seq=0 out_seq=0 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.811+0000 7fc298ff9640 10 client.21099 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.812+0000 7fc298ff9640 1 -- 172.21.15.5:0/1528012446 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558592e63860 con 0x558592f0f320 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.812+0000 7fc298ff9640 10 client.21099 handle_command_reply: tid=0 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 client.21099 shutdown 2024-09-11T22:40:55.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 -- 172.21.15.5:0/1528012446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 msgr2=0x558592efc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592efc4c0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac000c00 tx=0x7fc2ac005270 comp rx=0 tx=0).stop 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 -- 172.21.15.5:0/1528012446 shutdown_connections 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558592f0f320 0x558592f11710 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592eff780 0x558592efcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592efc4c0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1528012446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592eff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 -- 172.21.15.5:0/1528012446 >> 172.21.15.5:0/1528012446 conn(0x558592eb9310 msgr2=0x558592ef2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc298ff9640 0 client.21099 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 -- 172.21.15.5:0/1528012446 shutdown_connections 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 1 -- 172.21.15.5:0/1528012446 wait complete. 2024-09-11T22:40:55.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.813+0000 7fc2b9877740 20 client.21099 trim_cache size 0 max 16384 2024-09-11T22:40:55.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.827+0000 7fc2b9877740 1 Processor -- start 2024-09-11T22:40:55.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.827+0000 7fc2b9877740 1 -- start start 2024-09-11T22:40:55.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.827+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592eece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 0x558592f16da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592eff780 0x558592f172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b9877740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01ab8d0 con 0x558592ef05d0 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b9877740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01ad500 con 0x558592eff780 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b9877740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01abfa0 con 0x558592ef1b40 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b523a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 0x558592f16da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b4a39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592eece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b523a640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 0x558592f16da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59048/0 (socket says 172.21.15.5:59048) 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b523a640 1 -- 172.21.15.5:0/2242442974 learned_addr learned my addr 172.21.15.5:0/2242442974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b4a39640 1 --2- 172.21.15.5:0/2242442974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592eff780 0x558592f172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b523a640 1 -- 172.21.15.5:0/2242442974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592eff780 msgr2=0x558592f172e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b523a640 1 --2- 172.21.15.5:0/2242442974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592eff780 0x558592f172e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b523a640 1 -- 172.21.15.5:0/2242442974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 msgr2=0x558592eece10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b523a640 1 --2- 172.21.15.5:0/2242442974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592eece10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b523a640 1 -- 172.21.15.5:0/2242442974 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558592f025d0 con 0x558592ef1b40 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b4a39640 1 --2- 172.21.15.5:0/2242442974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592eece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.828+0000 7fc2b523a640 1 --2- 172.21.15.5:0/2242442974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 0x558592f16da0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac02c040 tx=0x7fc2ac005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f909a8b4ee2334e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.829+0000 7fc29affd640 1 -- 172.21.15.5:0/2242442974 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac027070 con 0x558592ef1b40 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.829+0000 7fc29affd640 1 -- 172.21.15.5:0/2242442974 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac015030 con 0x558592ef1b40 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.829+0000 7fc29affd640 1 -- 172.21.15.5:0/2242442974 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0198b0 con 0x558592ef1b40 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.829+0000 7fc2b9877740 1 -- 172.21.15.5:0/2242442974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 msgr2=0x558592f16da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.829+0000 7fc2b9877740 1 --2- 172.21.15.5:0/2242442974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 0x558592f16da0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac02c040 tx=0x7fc2ac005f90 comp rx=0 tx=0).stop 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.829+0000 7fc2b9877740 1 -- 172.21.15.5:0/2242442974 shutdown_connections 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.829+0000 7fc2b9877740 1 --2- 172.21.15.5:0/2242442974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592eff780 0x558592f172e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.829+0000 7fc2b9877740 1 --2- 172.21.15.5:0/2242442974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592ef1b40 0x558592f16da0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.829+0000 7fc2b9877740 1 --2- 172.21.15.5:0/2242442974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef05d0 0x558592eece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.829+0000 7fc2b9877740 1 -- 172.21.15.5:0/2242442974 >> 172.21.15.5:0/2242442974 conn(0x558592eb9310 msgr2=0x558592eec700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.830+0000 7fc2b9877740 1 -- 172.21.15.5:0/2242442974 shutdown_connections 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.830+0000 7fc2b9877740 1 -- 172.21.15.5:0/2242442974 wait complete. 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.830+0000 7fc2b9877740 1 -- start start 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.830+0000 7fc2b9877740 1 Processor -- start 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592f08060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592f04860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b9877740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592eff780 0x558592f04da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b523a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592f04860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b4a39640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592f08060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b523a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592f04860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44956/0 (socket says 172.21.15.5:44956) 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b4a39640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592f08060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37836/0 (socket says 172.21.15.5:37836) 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b523a640 1 -- 172.21.15.5:0/1206026775 learned_addr learned my addr 172.21.15.5:0/1206026775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01ace30 con 0x558592ef1b40 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01ac6a0 con 0x558592ef05d0 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01a98d0 con 0x558592eff780 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.831+0000 7fc2b4a39640 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592eff780 0x558592f04da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b4a39640 1 -- 172.21.15.5:0/1206026775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592eff780 msgr2=0x558592f04da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b4a39640 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592eff780 0x558592f04da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b4a39640 1 -- 172.21.15.5:0/1206026775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 msgr2=0x558592f04860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b4a39640 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592f04860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b4a39640 1 -- 172.21.15.5:0/1206026775 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558592f054e0 con 0x558592ef05d0 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b523a640 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592f04860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b9877740 10 client.0 _ll_register_callbacks cb 0x7fc2ac01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ac015030 con 0x558592ef05d0 2024-09-11T22:40:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b4a39640 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592f08060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc2a402aa00 tx=0x7fc2a4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=632c17d73af3971e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc298ff9640 10 client.30131 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:40:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc298ff9640 1 -- 172.21.15.5:0/1206026775 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4035070 con 0x558592ef05d0 2024-09-11T22:40:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc298ff9640 1 -- 172.21.15.5:0/1206026775 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7fc2a4004780 con 0x558592ef05d0 2024-09-11T22:40:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc298ff9640 1 -- 172.21.15.5:0/1206026775 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fc2a4052040 con 0x558592ef05d0 2024-09-11T22:40:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b9877740 10 client.30131 fetch_fsmap learned FSMap version 57 2024-09-11T22:40:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.832+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=57}) -- 0x558592f7d250 con 0x558592ef05d0 2024-09-11T22:40:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.833+0000 7fc298ff9640 1 -- 172.21.15.5:0/1206026775 <== mon.2 v2:172.21.15.159:3300/0 4 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fc2a40049c0 con 0x558592ef05d0 2024-09-11T22:40:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.833+0000 7fc2b9877740 10 client.30131 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:40:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.833+0000 7fc2b9877740 10 client.30131 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:40:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.833+0000 7fc2b9877740 20 client.30131 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.833+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558592f7d960 0x558592f7fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.833+0000 7fc2b9877740 4 client.30131 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.833+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558592e5dd70 con 0x558592f7d960 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.833+0000 7fc2b523a640 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558592f7d960 0x558592f7fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.833+0000 7fc2b523a640 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558592f7d960 0x558592f7fe20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ed68939b23c6c176 server_cookie=6b336e564e559d10 in_seq=0 out_seq=0 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.833+0000 7fc298ff9640 10 client.30131 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc298ff9640 1 -- 172.21.15.5:0/1206026775 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558592e5dd70 con 0x558592f7d960 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc298ff9640 10 client.30131 handle_command_reply: tid=0 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 client.30131 shutdown 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 msgr2=0x558592f08060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592f08060 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc2a402aa00 tx=0x7fc2a4009260 comp rx=0 tx=0).stop 2024-09-11T22:40:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 shutdown_connections 2024-09-11T22:40:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x558592f7d960 0x558592f7fe20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558592eff780 0x558592f04da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558592ef1b40 0x558592f04860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 --2- 172.21.15.5:0/1206026775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558592ef05d0 0x558592f08060 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 >> 172.21.15.5:0/1206026775 conn(0x558592eb9310 msgr2=0x558592efb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc298ff9640 0 client.30131 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:40:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 shutdown_connections 2024-09-11T22:40:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 1 -- 172.21.15.5:0/1206026775 wait complete. 2024-09-11T22:40:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.835+0000 7fc2b9877740 20 client.30131 trim_cache size 0 max 16384 2024-09-11T22:40:55.848 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:40:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 -- 172.21.15.5:0/3873735245 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc29c0a2d70 msgr2=0x7fc29c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc29c0a2d70 0x7fc29c0a5230 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc2a809bb00 tx=0x7fc2a800efd0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 -- 172.21.15.5:0/3873735245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b0106f20 msgr2=0x7fc2b019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b0106f20 0x7fc2b019cda0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800eb10 tx=0x7fc2a800c2d0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 -- 172.21.15.5:0/3873735245 shutdown_connections 2024-09-11T22:40:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b0106f20 0x7fc2b019cda0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2b01027a0 0x7fc2b0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fc29c0a2d70 0x7fc29c0a5230 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 --2- 172.21.15.5:0/3873735245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2b0101df0 0x7fc2b01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 -- 172.21.15.5:0/3873735245 >> 172.21.15.5:0/3873735245 conn(0x7fc2b00fd8f0 msgr2=0x7fc2b00fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 -- 172.21.15.5:0/3873735245 shutdown_connections 2024-09-11T22:40:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:55.836+0000 7fc2b749e640 1 -- 172.21.15.5:0/3873735245 wait complete. 2024-09-11T22:40:56.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:55 smithi005 ceph-mon[30943]: pgmap v596: 97 pgs: 97 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.5 MiB/s wr, 568 op/s 2024-09-11T22:40:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:55 smithi159 ceph-mon[35534]: pgmap v596: 97 pgs: 97 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.5 MiB/s wr, 568 op/s 2024-09-11T22:40:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:55 smithi172 ceph-mon[35461]: pgmap v596: 97 pgs: 97 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.5 MiB/s wr, 568 op/s 2024-09-11T22:40:56.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85502 16.87 MB/sec execute 194 sec latency 168.400 ms 2024-09-11T22:40:56.681 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:40:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:57.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85740 16.82 MB/sec execute 195 sec latency 186.539 ms 2024-09-11T22:40:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.757+0000 7f048cabc640 1 -- 172.21.15.5:0/2595878339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0488106f10 msgr2=0x7f048810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.757+0000 7f048cabc640 1 --2- 172.21.15.5:0/2595878339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0488106f10 0x7f048810ea90 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f0480009900 tx=0x7f048002a410 comp rx=0 tx=0).stop 2024-09-11T22:40:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.757+0000 7f048cabc640 1 -- 172.21.15.5:0/2595878339 shutdown_connections 2024-09-11T22:40:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.758+0000 7f048cabc640 1 --2- 172.21.15.5:0/2595878339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0488106f10 0x7f048810ea90 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.758+0000 7f048cabc640 1 --2- 172.21.15.5:0/2595878339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0488102790 0x7f04881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.758+0000 7f048cabc640 1 --2- 172.21.15.5:0/2595878339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0488101de0 0x7f04881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.758+0000 7f048cabc640 1 -- 172.21.15.5:0/2595878339 >> 172.21.15.5:0/2595878339 conn(0x7f04880fd8c0 msgr2=0x7f04880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.758+0000 7f048cabc640 1 -- 172.21.15.5:0/2595878339 shutdown_connections 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.758+0000 7f048cabc640 1 -- 172.21.15.5:0/2595878339 wait complete. 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.758+0000 7f048cabc640 1 Processor -- start 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.758+0000 7f048cabc640 1 -- start start 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048cabc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0488101de0 0x7f0488198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048cabc640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0488102790 0x7f0488198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048cabc640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0488106f10 0x7f048819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048cabc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0488111fc0 con 0x7f0488101de0 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048cabc640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0488111e40 con 0x7f0488106f10 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048cabc640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0488112140 con 0x7f0488102790 2024-09-11T22:40:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0488101de0 0x7f0488198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0488101de0 0x7f0488198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44964/0 (socket says 172.21.15.5:44964) 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048659c640 1 -- 172.21.15.5:0/446659515 learned_addr learned my addr 172.21.15.5:0/446659515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f0485d9b640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0488102790 0x7f0488198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f0486d9d640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0488106f10 0x7f048819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048659c640 1 -- 172.21.15.5:0/446659515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0488102790 msgr2=0x7f0488198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048659c640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0488102790 0x7f0488198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048659c640 1 -- 172.21.15.5:0/446659515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0488106f10 msgr2=0x7f048819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048659c640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0488106f10 0x7f048819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048659c640 1 -- 172.21.15.5:0/446659515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048819d3b0 con 0x7f0488101de0 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f048659c640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0488101de0 0x7f0488198400 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f047800ebc0 tx=0x7f047800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.759+0000 7f04777fe640 1 -- 172.21.15.5:0/446659515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478010630 con 0x7f0488101de0 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.760+0000 7f04777fe640 1 -- 172.21.15.5:0/446659515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0478010cb0 con 0x7f0488101de0 2024-09-11T22:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.760+0000 7f048cabc640 1 -- 172.21.15.5:0/446659515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048819d6a0 con 0x7f0488101de0 2024-09-11T22:40:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.760+0000 7f04777fe640 1 -- 172.21.15.5:0/446659515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478018bd0 con 0x7f0488101de0 2024-09-11T22:40:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.760+0000 7f048cabc640 1 -- 172.21.15.5:0/446659515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04881a4f80 con 0x7f0488101de0 2024-09-11T22:40:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.762+0000 7f04777fe640 1 -- 172.21.15.5:0/446659515 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f04780107d0 con 0x7f0488101de0 2024-09-11T22:40:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.762+0000 7f048ee95740 1 Processor -- start 2024-09-11T22:40:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.762+0000 7f048ee95740 1 -- start start 2024-09-11T22:40:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f048ee95740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x56532852c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f048ee95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 0x56532852ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f048ee95740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565328537370 0x565328539760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f048ee95740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5653284922f0 con 0x56532852db40 2024-09-11T22:40:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f048ee95740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565328439cf0 con 0x565328537370 2024-09-11T22:40:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f048ee95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56532848e2b0 con 0x56532852c5d0 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f0486d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x56532852c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f048659c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 0x56532852ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f0486d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x56532852c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44978/0 (socket says 172.21.15.5:44978) 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f0486d9d640 1 -- 172.21.15.5:0/1246617515 learned_addr learned my addr 172.21.15.5:0/1246617515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f0485d9b640 1 --2- 172.21.15.5:0/1246617515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565328537370 0x565328539760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f0486d9d640 1 -- 172.21.15.5:0/1246617515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 msgr2=0x56532852ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f0486d9d640 1 --2- 172.21.15.5:0/1246617515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 0x56532852ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f0486d9d640 1 -- 172.21.15.5:0/1246617515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565328537370 msgr2=0x565328539760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f0486d9d640 1 --2- 172.21.15.5:0/1246617515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565328537370 0x565328539760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f0486d9d640 1 -- 172.21.15.5:0/1246617515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56532841e3a0 con 0x56532852db40 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f0485d9b640 1 --2- 172.21.15.5:0/1246617515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565328537370 0x565328539760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.763+0000 7f04777fe640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f04700a2cf0 0x7f04700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.764+0000 7f0486d9d640 1 --2- 172.21.15.5:0/1246617515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x56532852c090 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f048002aa00 tx=0x7f0480005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed066465c93cbb1a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.764+0000 7f048659c640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f04700a2cf0 0x7f04700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.764+0000 7f04777fe640 1 -- 172.21.15.5:0/446659515 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f04780cfae0 con 0x7f0488101de0 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.764+0000 7f0474ff9640 1 -- 172.21.15.5:0/1246617515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480040070 con 0x56532852db40 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.764+0000 7f0474ff9640 1 -- 172.21.15.5:0/1246617515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0480004300 con 0x56532852db40 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.764+0000 7f0474ff9640 1 -- 172.21.15.5:0/1246617515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480035040 con 0x56532852db40 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 -- 172.21.15.5:0/1246617515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 msgr2=0x56532852c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048659c640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f04700a2cf0 0x7f04700a51b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f04780cd270 tx=0x7f04780d3630 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 --2- 172.21.15.5:0/1246617515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x56532852c090 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f048002aa00 tx=0x7f0480005c00 comp rx=0 tx=0).stop 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 -- 172.21.15.5:0/1246617515 shutdown_connections 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 --2- 172.21.15.5:0/1246617515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565328537370 0x565328539760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 --2- 172.21.15.5:0/1246617515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 0x56532852ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 --2- 172.21.15.5:0/1246617515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x56532852c090 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 -- 172.21.15.5:0/1246617515 >> 172.21.15.5:0/1246617515 conn(0x5653284f5310 msgr2=0x565328528c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 -- 172.21.15.5:0/1246617515 shutdown_connections 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 -- 172.21.15.5:0/1246617515 wait complete. 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 -- start start 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.765+0000 7f048ee95740 1 Processor -- start 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f048ee95740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852c5d0 0x565328536d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f048ee95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852db40 0x565328534040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f048ee95740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 0x565328534650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f048ee95740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56532853a010 con 0x565328537370 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f048ee95740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56532844d060 con 0x56532852c5d0 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f048ee95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56532853a190 con 0x56532852db40 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f048ee95740 10 client.0 _ll_register_callbacks cb 0x5653281a08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0486d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 0x565328534650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0485d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852db40 0x565328534040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0485d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852db40 0x565328534040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59100/0 (socket says 172.21.15.5:59100) 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0486d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 0x565328534650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44984/0 (socket says 172.21.15.5:44984) 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0485d9b640 1 -- 172.21.15.5:0/1115603680 learned_addr learned my addr 172.21.15.5:0/1115603680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0486d9d640 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852c5d0 0x565328536d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0485d9b640 1 -- 172.21.15.5:0/1115603680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852c5d0 msgr2=0x565328536d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0485d9b640 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852c5d0 0x565328536d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0485d9b640 1 -- 172.21.15.5:0/1115603680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 msgr2=0x565328534650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0485d9b640 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 0x565328534650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0485d9b640 1 -- 172.21.15.5:0/1115603680 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653282033c0 con 0x56532852db40 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0486d9d640 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 0x565328534650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f0485d9b640 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852db40 0x565328534040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f047c012700 tx=0x7f047c012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56a96009b91c4062 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.766+0000 7f046dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f046dffb640 1 -- 172.21.15.5:0/1115603680 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c00f8c0 con 0x56532852db40 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f048ee95740 1 -- 172.21.15.5:0/1115603680 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0480004300 con 0x56532852db40 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f046dffb640 1 -- 172.21.15.5:0/1115603680 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f047c00fa60 con 0x56532852db40 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f046dffb640 1 -- 172.21.15.5:0/1115603680 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f047c00fc60 con 0x56532852db40 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f048ee95740 10 client.29959 fetch_fsmap learned FSMap version 57 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f048ee95740 1 -- 172.21.15.5:0/1115603680 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x565328545b40 con 0x56532852db40 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f046dffb640 1 -- 172.21.15.5:0/1115603680 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0480004300 con 0x56532852db40 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f046dffb640 1 -- 172.21.15.5:0/1115603680 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f047c010550 con 0x56532852db40 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f048ee95740 10 client.29959 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f048ee95740 10 client.29959 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f048ee95740 20 client.29959 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f048ee95740 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x565328547050 0x565328549470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f048ee95740 4 client.29959 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f048ee95740 1 -- 172.21.15.5:0/1115603680 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56532849f860 con 0x565328547050 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.767+0000 7f0486d9d640 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x565328547050 0x565328549470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.768+0000 7f046dffb640 10 client.29959 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.768+0000 7f0486d9d640 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x565328547050 0x565328549470 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=19af8a20406ceae5 server_cookie=2b78f9d4f8d0e676 in_seq=0 out_seq=0 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f046dffb640 1 -- 172.21.15.5:0/1115603680 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56532849f860 con 0x565328547050 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f046dffb640 10 client.29959 handle_command_reply: tid=0 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 client.29959 shutdown 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 -- 172.21.15.5:0/1115603680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852db40 msgr2=0x565328534040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852db40 0x565328534040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f047c012700 tx=0x7f047c012bc0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 -- 172.21.15.5:0/1115603680 shutdown_connections 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 0x565328534650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852db40 0x565328534040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x565328547050 0x565328549470 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 --2- 172.21.15.5:0/1115603680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852c5d0 0x565328536d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 -- 172.21.15.5:0/1115603680 >> 172.21.15.5:0/1115603680 conn(0x5653284f5310 msgr2=0x56532852e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f046dffb640 0 client.29959 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 -- 172.21.15.5:0/1115603680 shutdown_connections 2024-09-11T22:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 1 -- 172.21.15.5:0/1115603680 wait complete. 2024-09-11T22:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.769+0000 7f048ee95740 20 client.29959 trim_cache size 0 max 16384 2024-09-11T22:40:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.784+0000 7f048ee95740 1 Processor -- start 2024-09-11T22:40:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.784+0000 7f048ee95740 1 -- start start 2024-09-11T22:40:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.784+0000 7f048ee95740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852c5d0 0x5653285432b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.784+0000 7f048ee95740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x565328533020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.784+0000 7f048ee95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565328537370 0x56532854ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.784+0000 7f048ee95740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04881adac0 con 0x56532852db40 2024-09-11T22:40:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.784+0000 7f048ee95740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04881af6f0 con 0x56532852c5d0 2024-09-11T22:40:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.784+0000 7f048ee95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04881ae190 con 0x565328537370 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.784+0000 7f0485d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x565328533020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f0485d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x565328533020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44998/0 (socket says 172.21.15.5:44998) 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f0485d9b640 1 -- 172.21.15.5:0/3343962977 learned_addr learned my addr 172.21.15.5:0/3343962977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f0486d9d640 1 --2- 172.21.15.5:0/3343962977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852c5d0 0x5653285432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f0486d9d640 1 --2- 172.21.15.5:0/3343962977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565328537370 0x56532854ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f0485d9b640 1 -- 172.21.15.5:0/3343962977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565328537370 msgr2=0x56532854ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f0485d9b640 1 --2- 172.21.15.5:0/3343962977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565328537370 0x56532854ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f0485d9b640 1 -- 172.21.15.5:0/3343962977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852c5d0 msgr2=0x5653285432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f0485d9b640 1 --2- 172.21.15.5:0/3343962977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852c5d0 0x5653285432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f0485d9b640 1 -- 172.21.15.5:0/3343962977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565328529960 con 0x56532852db40 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f0485d9b640 1 --2- 172.21.15.5:0/3343962977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x565328533020 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f047c012420 tx=0x7f047c00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535ef270af830222 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f046ffff640 1 -- 172.21.15.5:0/3343962977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c010be0 con 0x56532852db40 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f046ffff640 1 -- 172.21.15.5:0/3343962977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f047c010d80 con 0x56532852db40 2024-09-11T22:40:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.785+0000 7f046ffff640 1 -- 172.21.15.5:0/3343962977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c0056d0 con 0x56532852db40 2024-09-11T22:40:57.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.786+0000 7f048ee95740 1 -- 172.21.15.5:0/3343962977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 msgr2=0x565328533020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:57.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.786+0000 7f048ee95740 1 --2- 172.21.15.5:0/3343962977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x565328533020 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f047c012420 tx=0x7f047c00ff90 comp rx=0 tx=0).stop 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.786+0000 7f048ee95740 1 -- 172.21.15.5:0/3343962977 shutdown_connections 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.786+0000 7f048ee95740 1 --2- 172.21.15.5:0/3343962977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x565328537370 0x56532854ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.786+0000 7f048ee95740 1 --2- 172.21.15.5:0/3343962977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56532852db40 0x565328533020 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.786+0000 7f048ee95740 1 --2- 172.21.15.5:0/3343962977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852c5d0 0x5653285432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.786+0000 7f048ee95740 1 -- 172.21.15.5:0/3343962977 >> 172.21.15.5:0/3343962977 conn(0x5653284f5310 msgr2=0x565328532910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.786+0000 7f048ee95740 1 -- 172.21.15.5:0/3343962977 shutdown_connections 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.786+0000 7f048ee95740 1 -- 172.21.15.5:0/3343962977 wait complete. 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.786+0000 7f048ee95740 1 -- start start 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f048ee95740 1 Processor -- start 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f048ee95740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 0x5653285a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f048ee95740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852db40 0x5653285aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f048ee95740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 0x56532853f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f048ee95740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04881af020 con 0x565328537370 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f048ee95740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04881ae890 con 0x56532852db40 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f048ee95740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f04881abac0 con 0x56532852c5d0 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f048ee95740 10 client.0 _ll_register_callbacks cb 0x7f047c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0486d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 0x5653285a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0485d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852db40 0x5653285aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0486d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 0x5653285a9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59128/0 (socket says 172.21.15.5:59128) 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0486d9d640 1 -- 172.21.15.5:0/3434910635 learned_addr learned my addr 172.21.15.5:0/3434910635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0485d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852db40 0x5653285aa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37880/0 (socket says 172.21.15.5:37880) 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0486d9d640 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 0x56532853f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0486d9d640 1 -- 172.21.15.5:0/3434910635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852db40 msgr2=0x5653285aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0486d9d640 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852db40 0x5653285aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0486d9d640 1 -- 172.21.15.5:0/3434910635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 msgr2=0x56532853f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0486d9d640 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 0x56532853f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0486d9d640 1 -- 172.21.15.5:0/3434910635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565328534fe0 con 0x56532852c5d0 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.787+0000 7f0485d9b640 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852db40 0x5653285aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.788+0000 7f0486d9d640 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 0x5653285a9e90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0480031520 tx=0x7f0480005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98abed4b9c22a278 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.788+0000 7f046dffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.788+0000 7f046dffb640 1 -- 172.21.15.5:0/3434910635 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0480059070 con 0x56532852c5d0 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.788+0000 7f048ee95740 1 -- 172.21.15.5:0/3434910635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f047c010d80 con 0x56532852c5d0 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.788+0000 7f046dffb640 1 -- 172.21.15.5:0/3434910635 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f0480032b70 con 0x56532852c5d0 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.788+0000 7f046dffb640 1 -- 172.21.15.5:0/3434910635 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f0480035040 con 0x56532852c5d0 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.788+0000 7f048ee95740 10 client.29965 fetch_fsmap learned FSMap version 57 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.788+0000 7f048ee95740 1 -- 172.21.15.5:0/3434910635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5653285b1500 con 0x56532852c5d0 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.788+0000 7f046dffb640 1 -- 172.21.15.5:0/3434910635 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f047c010d80 con 0x56532852c5d0 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.788+0000 7f046dffb640 1 -- 172.21.15.5:0/3434910635 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f0480004aa0 con 0x56532852c5d0 2024-09-11T22:40:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f048ee95740 10 client.29965 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f048ee95740 10 client.29965 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f048ee95740 20 client.29965 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f048ee95740 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5653285b17f0 0x5653285b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f048ee95740 4 client.29965 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f048ee95740 1 -- 172.21.15.5:0/3434910635 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565328499d70 con 0x5653285b17f0 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f0485d9b640 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5653285b17f0 0x5653285b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f046dffb640 10 client.29965 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f0485d9b640 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5653285b17f0 0x5653285b3be0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dec00d592a5c1d59 server_cookie=5a3aab6d1aaff94e in_seq=0 out_seq=0 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f046dffb640 1 -- 172.21.15.5:0/3434910635 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565328499d70 con 0x5653285b17f0 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f046dffb640 10 client.29965 handle_command_reply: tid=0 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.789+0000 7f048ee95740 1 client.29965 shutdown 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 1 -- 172.21.15.5:0/3434910635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 msgr2=0x5653285a9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 0x5653285a9e90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0480031520 tx=0x7f0480005c00 comp rx=0 tx=0).stop 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 1 -- 172.21.15.5:0/3434910635 shutdown_connections 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565328537370 0x56532853f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x5653285b17f0 0x5653285b3be0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56532852db40 0x5653285aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 1 --2- 172.21.15.5:0/3434910635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56532852c5d0 0x5653285a9e90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f046dffb640 0 client.29965 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 1 -- 172.21.15.5:0/3434910635 >> 172.21.15.5:0/3434910635 conn(0x5653284f5310 msgr2=0x5653285274e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 1 -- 172.21.15.5:0/3434910635 shutdown_connections 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 1 -- 172.21.15.5:0/3434910635 wait complete. 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048ee95740 20 client.29965 trim_cache size 0 max 16384 2024-09-11T22:40:57.796 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:40:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048cabc640 1 -- 172.21.15.5:0/446659515 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f04700a2cf0 msgr2=0x7f04700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048cabc640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f04700a2cf0 0x7f04700a51b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f04780cd270 tx=0x7f04780d3630 comp rx=0 tx=0).stop 2024-09-11T22:40:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048cabc640 1 -- 172.21.15.5:0/446659515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0488101de0 msgr2=0x7f0488198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048cabc640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0488101de0 0x7f0488198400 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f047800ebc0 tx=0x7f047800c6a0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.790+0000 7f048cabc640 1 -- 172.21.15.5:0/446659515 shutdown_connections 2024-09-11T22:40:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.791+0000 7f048cabc640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f04700a2cf0 0x7f04700a51b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.791+0000 7f048cabc640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0488106f10 0x7f048819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.791+0000 7f048cabc640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0488102790 0x7f0488198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.791+0000 7f048cabc640 1 --2- 172.21.15.5:0/446659515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0488101de0 0x7f0488198400 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.791+0000 7f048cabc640 1 -- 172.21.15.5:0/446659515 >> 172.21.15.5:0/446659515 conn(0x7f04880fd8c0 msgr2=0x7f04880fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.791+0000 7f048cabc640 1 -- 172.21.15.5:0/446659515 shutdown_connections 2024-09-11T22:40:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:57.791+0000 7f048cabc640 1 -- 172.21.15.5:0/446659515 wait complete. 2024-09-11T22:40:58.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:57 smithi005 ceph-mon[30943]: pgmap v597: 97 pgs: 97 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-09-11T22:40:58.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:57 smithi159 ceph-mon[35534]: pgmap v597: 97 pgs: 97 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-09-11T22:40:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:57 smithi172 ceph-mon[35461]: pgmap v597: 97 pgs: 97 active+clean; 594 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 5.9 MiB/s wr, 473 op/s 2024-09-11T22:40:58.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85998 16.77 MB/sec execute 196 sec latency 178.671 ms 2024-09-11T22:40:58.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:40:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:40:59.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.551+0000 7f01688c0640 1 -- 172.21.15.5:0/3009590200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0164102d30 msgr2=0x7f0164103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:59.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.551+0000 7f01688c0640 1 --2- 172.21.15.5:0/3009590200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0164102d30 0x7f0164103110 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0154009930 tx=0x7f015402a420 comp rx=0 tx=0).stop 2024-09-11T22:40:59.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.551+0000 7f01688c0640 1 -- 172.21.15.5:0/3009590200 shutdown_connections 2024-09-11T22:40:59.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.551+0000 7f01688c0640 1 --2- 172.21.15.5:0/3009590200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641082a0 0x7f016410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.551+0000 7f01688c0640 1 --2- 172.21.15.5:0/3009590200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01641036e0 0x7f0164107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.551+0000 7f01688c0640 1 --2- 172.21.15.5:0/3009590200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0164102d30 0x7f0164103110 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.551+0000 7f01688c0640 1 -- 172.21.15.5:0/3009590200 >> 172.21.15.5:0/3009590200 conn(0x7f01640fc7d0 msgr2=0x7f01640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:59.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.551+0000 7f01688c0640 1 -- 172.21.15.5:0/3009590200 shutdown_connections 2024-09-11T22:40:59.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.551+0000 7f01688c0640 1 -- 172.21.15.5:0/3009590200 wait complete. 2024-09-11T22:40:59.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f01688c0640 1 Processor -- start 2024-09-11T22:40:59.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f01688c0640 1 -- start start 2024-09-11T22:40:59.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f01688c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0164102d30 0x7f01641985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f01688c0640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01641036e0 0x7f0164198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f01688c0640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641082a0 0x7f016419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f01688c0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01641120a0 con 0x7f0164102d30 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f01688c0640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0164111f20 con 0x7f01641082a0 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f01688c0640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f0164112220 con 0x7f01641036e0 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f016259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0164102d30 0x7f01641985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f0161d9b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01641036e0 0x7f0164198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f016259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0164102d30 0x7f01641985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45030/0 (socket says 172.21.15.5:45030) 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f016259c640 1 -- 172.21.15.5:0/3128405302 learned_addr learned my addr 172.21.15.5:0/3128405302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f0162d9d640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641082a0 0x7f016419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f0161d9b640 1 -- 172.21.15.5:0/3128405302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641082a0 msgr2=0x7f016419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f0161d9b640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641082a0 0x7f016419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f0161d9b640 1 -- 172.21.15.5:0/3128405302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0164102d30 msgr2=0x7f01641985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f0161d9b640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0164102d30 0x7f01641985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.552+0000 7f0161d9b640 1 -- 172.21.15.5:0/3128405302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016419d550 con 0x7f01641036e0 2024-09-11T22:40:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.553+0000 7f0161d9b640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01641036e0 0x7f0164198ae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f01640697d0 tx=0x7f014c00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.553+0000 7f015b7fe640 1 -- 172.21.15.5:0/3128405302 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c00cc40 con 0x7f01641036e0 2024-09-11T22:40:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.553+0000 7f01688c0640 1 -- 172.21.15.5:0/3128405302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016419d840 con 0x7f01641036e0 2024-09-11T22:40:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.553+0000 7f015b7fe640 1 -- 172.21.15.5:0/3128405302 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f014c00cde0 con 0x7f01641036e0 2024-09-11T22:40:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.553+0000 7f015b7fe640 1 -- 172.21.15.5:0/3128405302 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f014c0188e0 con 0x7f01641036e0 2024-09-11T22:40:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.553+0000 7f01688c0640 1 -- 172.21.15.5:0/3128405302 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01641a5120 con 0x7f01641036e0 2024-09-11T22:40:59.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.556+0000 7f016ac99740 1 Processor -- start 2024-09-11T22:40:59.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.556+0000 7f015b7fe640 1 -- 172.21.15.5:0/3128405302 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f014c018a80 con 0x7f01641036e0 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.556+0000 7f016ac99740 1 -- start start 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f016ac99740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed11063b40 0x55ed11062090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f016ac99740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 0x55ed11062a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f016ac99740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1106d370 0x55ed1106f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f016ac99740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ed10fc82f0 con 0x55ed110625d0 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f016ac99740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ed10f6fcf0 con 0x55ed1106d370 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f016ac99740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ed10fc42b0 con 0x55ed11063b40 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f0162d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed11063b40 0x55ed11062090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f016259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 0x55ed11062a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f0162d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed11063b40 0x55ed11062090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59148/0 (socket says 172.21.15.5:59148) 2024-09-11T22:40:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f0162d9d640 1 -- 172.21.15.5:0/3300825945 learned_addr learned my addr 172.21.15.5:0/3300825945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f0161d9b640 1 --2- 172.21.15.5:0/3300825945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1106d370 0x55ed1106f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f0162d9d640 1 -- 172.21.15.5:0/3300825945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1106d370 msgr2=0x55ed1106f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f0162d9d640 1 --2- 172.21.15.5:0/3300825945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1106d370 0x55ed1106f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f0162d9d640 1 -- 172.21.15.5:0/3300825945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 msgr2=0x55ed11062a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f0162d9d640 1 --2- 172.21.15.5:0/3300825945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 0x55ed11062a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.557+0000 7f0162d9d640 1 -- 172.21.15.5:0/3300825945 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed10f543a0 con 0x55ed11063b40 2024-09-11T22:40:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.558+0000 7f0162d9d640 1 --2- 172.21.15.5:0/3300825945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed11063b40 0x55ed11062090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f015c009f40 tx=0x7f015c012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8255e09b9eec4e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.558+0000 7f015b7fe640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01500a2cf0 0x7f01500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.558+0000 7f0158ff9640 1 -- 172.21.15.5:0/3300825945 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c020070 con 0x55ed11063b40 2024-09-11T22:40:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.558+0000 7f0158ff9640 1 -- 172.21.15.5:0/3300825945 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f015c003e50 con 0x55ed11063b40 2024-09-11T22:40:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.558+0000 7f016259c640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01500a2cf0 0x7f01500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.558+0000 7f0158ff9640 1 -- 172.21.15.5:0/3300825945 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c004430 con 0x55ed11063b40 2024-09-11T22:40:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f015b7fe640 1 -- 172.21.15.5:0/3128405302 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f014c0cf820 con 0x7f01641036e0 2024-09-11T22:40:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 -- 172.21.15.5:0/3300825945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed11063b40 msgr2=0x55ed11062090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 --2- 172.21.15.5:0/3300825945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed11063b40 0x55ed11062090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f015c009f40 tx=0x7f015c012bf0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016259c640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01500a2cf0 0x7f01500a51b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55ed1105fac0 tx=0x7f0154009620 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 -- 172.21.15.5:0/3300825945 shutdown_connections 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 --2- 172.21.15.5:0/3300825945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed1106d370 0x55ed1106f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 --2- 172.21.15.5:0/3300825945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 0x55ed11062a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 --2- 172.21.15.5:0/3300825945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed11063b40 0x55ed11062090 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 -- 172.21.15.5:0/3300825945 >> 172.21.15.5:0/3300825945 conn(0x55ed1102b310 msgr2=0x55ed1105ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 -- 172.21.15.5:0/3300825945 shutdown_connections 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 -- 172.21.15.5:0/3300825945 wait complete. 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 -- start start 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.559+0000 7f016ac99740 1 Processor -- start 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f016ac99740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 0x55ed1106ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f016ac99740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed11069fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f016ac99740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed1106a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f016ac99740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ed11070110 con 0x55ed1106d370 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f016ac99740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ed10f83060 con 0x55ed11063b40 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f016ac99740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55ed11070290 con 0x55ed110625d0 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f016ac99740 10 client.0 _ll_register_callbacks cb 0x55ed10cd68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0162d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed1106a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0162d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed1106a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45048/0 (socket says 172.21.15.5:45048) 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0162d9d640 1 -- 172.21.15.5:0/2464920587 learned_addr learned my addr 172.21.15.5:0/2464920587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0161d9b640 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed11069fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f016259c640 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 0x55ed1106ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0162d9d640 1 -- 172.21.15.5:0/2464920587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 msgr2=0x55ed1106ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0162d9d640 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 0x55ed1106ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0162d9d640 1 -- 172.21.15.5:0/2464920587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 msgr2=0x55ed11069fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0162d9d640 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed11069fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0162d9d640 1 -- 172.21.15.5:0/2464920587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed10d393c0 con 0x55ed1106d370 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0161d9b640 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed11069fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0162d9d640 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed1106a520 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c6a0 tx=0x7f015c009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b274588122eb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.560+0000 7f0149ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.561+0000 7f0149ffb640 1 -- 172.21.15.5:0/2464920587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c020070 con 0x55ed1106d370 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.561+0000 7f016ac99740 1 -- 172.21.15.5:0/2464920587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f015c003e50 con 0x55ed1106d370 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.561+0000 7f0149ffb640 1 -- 172.21.15.5:0/2464920587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f015c004c40 con 0x55ed1106d370 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.561+0000 7f0149ffb640 1 -- 172.21.15.5:0/2464920587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f015c004e40 con 0x55ed1106d370 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.561+0000 7f016ac99740 10 client.21135 fetch_fsmap learned FSMap version 57 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.561+0000 7f016ac99740 1 -- 172.21.15.5:0/2464920587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55ed1107bb40 con 0x55ed1106d370 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.561+0000 7f0149ffb640 1 -- 172.21.15.5:0/2464920587 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f015c004610 con 0x55ed1106d370 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.562+0000 7f0149ffb640 1 -- 172.21.15.5:0/2464920587 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f015c01a070 con 0x55ed1106d370 2024-09-11T22:40:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.562+0000 7f016ac99740 10 client.21135 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.562+0000 7f016ac99740 10 client.21135 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.562+0000 7f016ac99740 20 client.21135 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.562+0000 7f016ac99740 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ed1107cf50 0x55ed1107f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.562+0000 7f016ac99740 4 client.21135 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.562+0000 7f016ac99740 1 -- 172.21.15.5:0/2464920587 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f015c003e50 con 0x55ed1107cf50 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.562+0000 7f016259c640 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ed1107cf50 0x55ed1107f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.563+0000 7f0149ffb640 10 client.21135 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.563+0000 7f016259c640 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ed1107cf50 0x55ed1107f340 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d7b3a47ac71bea4d server_cookie=4b11359d816a6ba0 in_seq=0 out_seq=0 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f0149ffb640 1 -- 172.21.15.5:0/2464920587 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f015c003e50 con 0x55ed1107cf50 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f0149ffb640 10 client.21135 handle_command_reply: tid=0 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 client.21135 shutdown 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 -- 172.21.15.5:0/2464920587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 msgr2=0x55ed1106a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed1106a520 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f015c00c6a0 tx=0x7f015c009c70 comp rx=0 tx=0).stop 2024-09-11T22:40:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 -- 172.21.15.5:0/2464920587 shutdown_connections 2024-09-11T22:40:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ed1107cf50 0x55ed1107f340 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed1106a520 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed11069fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 --2- 172.21.15.5:0/2464920587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 0x55ed1106ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 -- 172.21.15.5:0/2464920587 >> 172.21.15.5:0/2464920587 conn(0x55ed1102b310 msgr2=0x55ed11064960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f0149ffb640 0 client.21135 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:40:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 -- 172.21.15.5:0/2464920587 shutdown_connections 2024-09-11T22:40:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 1 -- 172.21.15.5:0/2464920587 wait complete. 2024-09-11T22:40:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.564+0000 7f016ac99740 20 client.21135 trim_cache size 0 max 16384 2024-09-11T22:40:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.577+0000 7f016ac99740 1 Processor -- start 2024-09-11T22:40:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.577+0000 7f016ac99740 1 -- start start 2024-09-11T22:40:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016ac99740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 0x55ed110793b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016ac99740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed11065120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016ac99740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed11065660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016ac99740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01641ab900 con 0x55ed1106d370 2024-09-11T22:40:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016ac99740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f01641ad530 con 0x55ed11063b40 2024-09-11T22:40:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016ac99740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01641abfd0 con 0x55ed110625d0 2024-09-11T22:40:59.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f0162d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed11065660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 0x55ed110793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016259c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 0x55ed110793b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59158/0 (socket says 172.21.15.5:59158) 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016259c640 1 -- 172.21.15.5:0/3895738892 learned_addr learned my addr 172.21.15.5:0/3895738892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f0162d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed11065660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45060/0 (socket says 172.21.15.5:45060) 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f0161d9b640 1 --2- 172.21.15.5:0/3895738892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed11065120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016259c640 1 -- 172.21.15.5:0/3895738892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 msgr2=0x55ed11065120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016259c640 1 --2- 172.21.15.5:0/3895738892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed11065120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016259c640 1 -- 172.21.15.5:0/3895738892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 msgr2=0x55ed11065660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016259c640 1 --2- 172.21.15.5:0/3895738892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed11065660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016259c640 1 -- 172.21.15.5:0/3895738892 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed11065ba0 con 0x55ed110625d0 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f0162d9d640 1 --2- 172.21.15.5:0/3895738892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed11065660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:59.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.578+0000 7f016259c640 1 --2- 172.21.15.5:0/3895738892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 0x55ed110793b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0154054fd0 tx=0x7f015403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a85d12a84db691 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f014bfff640 1 -- 172.21.15.5:0/3895738892 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0154035040 con 0x55ed110625d0 2024-09-11T22:40:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f014bfff640 1 -- 172.21.15.5:0/3895738892 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f015405a070 con 0x55ed110625d0 2024-09-11T22:40:59.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f014bfff640 1 -- 172.21.15.5:0/3895738892 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015403d450 con 0x55ed110625d0 2024-09-11T22:40:59.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f016ac99740 1 -- 172.21.15.5:0/3895738892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 msgr2=0x55ed110793b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:59.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f016ac99740 1 --2- 172.21.15.5:0/3895738892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 0x55ed110793b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0154054fd0 tx=0x7f015403f000 comp rx=0 tx=0).stop 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f016ac99740 1 -- 172.21.15.5:0/3895738892 shutdown_connections 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f016ac99740 1 --2- 172.21.15.5:0/3895738892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed1106d370 0x55ed11065660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f016ac99740 1 --2- 172.21.15.5:0/3895738892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed11065120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f016ac99740 1 --2- 172.21.15.5:0/3895738892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed110625d0 0x55ed110793b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f016ac99740 1 -- 172.21.15.5:0/3895738892 >> 172.21.15.5:0/3895738892 conn(0x55ed1102b310 msgr2=0x55ed1106c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.579+0000 7f016ac99740 1 -- 172.21.15.5:0/3895738892 shutdown_connections 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016ac99740 1 -- 172.21.15.5:0/3895738892 wait complete. 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016ac99740 1 -- start start 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016ac99740 1 Processor -- start 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016ac99740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 0x55ed11075bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016ac99740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed110704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016ac99740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1106d370 0x55ed11070a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016ac99740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01641ace60 con 0x55ed110625d0 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016ac99740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f01641ac6d0 con 0x55ed11063b40 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016ac99740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f01641a9900 con 0x55ed1106d370 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016ac99740 10 client.0 _ll_register_callbacks cb 0x7f015c01e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 0x55ed11075bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f0162d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1106d370 0x55ed11070a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f0161d9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed110704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f0162d9d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1106d370 0x55ed11070a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:59174/0 (socket says 172.21.15.5:59174) 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f016259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 0x55ed11075bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45076/0 (socket says 172.21.15.5:45076) 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.580+0000 7f0162d9d640 1 -- 172.21.15.5:0/83571726 learned_addr learned my addr 172.21.15.5:0/83571726 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0162d9d640 1 -- 172.21.15.5:0/83571726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 msgr2=0x55ed110704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0162d9d640 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed110704e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0162d9d640 1 -- 172.21.15.5:0/83571726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 msgr2=0x55ed11075bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0162d9d640 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 0x55ed11075bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0162d9d640 1 -- 172.21.15.5:0/83571726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed11071250 con 0x55ed1106d370 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0161d9b640 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed110704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0162d9d640 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1106d370 0x55ed11070a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f015c000f80 tx=0x7f015c004dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b57b8034a865fce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0149ffb640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0149ffb640 1 -- 172.21.15.5:0/83571726 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c020040 con 0x55ed1106d370 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0149ffb640 1 -- 172.21.15.5:0/83571726 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f015c004180 con 0x55ed1106d370 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f016ac99740 1 -- 172.21.15.5:0/83571726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f015405a070 con 0x55ed1106d370 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f0149ffb640 1 -- 172.21.15.5:0/83571726 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f015c004380 con 0x55ed1106d370 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f016ac99740 10 client.29983 fetch_fsmap learned FSMap version 57 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.581+0000 7f016ac99740 1 -- 172.21.15.5:0/83571726 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55ed110e6a70 con 0x55ed1106d370 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.582+0000 7f0149ffb640 1 -- 172.21.15.5:0/83571726 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f015405a070 con 0x55ed1106d370 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.582+0000 7f0149ffb640 1 -- 172.21.15.5:0/83571726 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f015c01a040 con 0x55ed1106d370 2024-09-11T22:40:59.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.582+0000 7f016ac99740 10 client.29983 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.582+0000 7f016ac99740 10 client.29983 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.582+0000 7f016ac99740 20 client.29983 populate_metadata read hostname 'smithi005' 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.582+0000 7f016ac99740 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ed110e7150 0x55ed110e9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.582+0000 7f016ac99740 4 client.29983 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.582+0000 7f016ac99740 1 -- 172.21.15.5:0/83571726 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed10fd5860 con 0x55ed110e7150 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.582+0000 7f016259c640 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ed110e7150 0x55ed110e9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f0149ffb640 10 client.29983 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016259c640 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ed110e7150 0x55ed110e9570 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d73d9afebe22b7fc server_cookie=cf5abe5ddfa8865b in_seq=0 out_seq=0 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f0149ffb640 1 -- 172.21.15.5:0/83571726 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed10fd5860 con 0x55ed110e7150 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f0149ffb640 10 client.29983 handle_command_reply: tid=0 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 client.29983 shutdown 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 -- 172.21.15.5:0/83571726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1106d370 msgr2=0x55ed11070a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:59.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1106d370 0x55ed11070a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f015c000f80 tx=0x7f015c004dd0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 -- 172.21.15.5:0/83571726 shutdown_connections 2024-09-11T22:40:59.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55ed110e7150 0x55ed110e9570 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ed1106d370 0x55ed11070a20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed11063b40 0x55ed110704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 --2- 172.21.15.5:0/83571726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed110625d0 0x55ed11075bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 -- 172.21.15.5:0/83571726 >> 172.21.15.5:0/83571726 conn(0x55ed1102b310 msgr2=0x55ed11068740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:59.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f0149ffb640 0 client.29983 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:40:59.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 -- 172.21.15.5:0/83571726 shutdown_connections 2024-09-11T22:40:59.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 1 -- 172.21.15.5:0/83571726 wait complete. 2024-09-11T22:40:59.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.583+0000 7f016ac99740 20 client.29983 trim_cache size 0 max 16384 2024-09-11T22:40:59.593 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:40:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 -- 172.21.15.5:0/3128405302 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01500a2cf0 msgr2=0x7f01500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01500a2cf0 0x7f01500a51b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55ed1105fac0 tx=0x7f0154009620 comp rx=0 tx=0).stop 2024-09-11T22:40:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 -- 172.21.15.5:0/3128405302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01641036e0 msgr2=0x7f0164198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:40:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01641036e0 0x7f0164198ae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f01640697d0 tx=0x7f014c00ee50 comp rx=0 tx=0).stop 2024-09-11T22:40:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 -- 172.21.15.5:0/3128405302 shutdown_connections 2024-09-11T22:40:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f01500a2cf0 0x7f01500a51b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f01641082a0 0x7f016419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01641036e0 0x7f0164198ae0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 --2- 172.21.15.5:0/3128405302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0164102d30 0x7f01641985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:40:59.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 -- 172.21.15.5:0/3128405302 >> 172.21.15.5:0/3128405302 conn(0x7f01640fc7d0 msgr2=0x7f0164105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:40:59.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 -- 172.21.15.5:0/3128405302 shutdown_connections 2024-09-11T22:40:59.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:40:59.584+0000 7f01688c0640 1 -- 172.21.15.5:0/3128405302 wait complete. 2024-09-11T22:40:59.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86367 16.79 MB/sec execute 197 sec latency 313.754 ms 2024-09-11T22:40:59.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:40:59 smithi005 ceph-mon[30943]: pgmap v598: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.2 MiB/s wr, 652 op/s 2024-09-11T22:41:00.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:40:59 smithi159 ceph-mon[35534]: pgmap v598: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.2 MiB/s wr, 652 op/s 2024-09-11T22:41:00.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:40:59 smithi172 ceph-mon[35461]: pgmap v598: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.2 MiB/s wr, 652 op/s 2024-09-11T22:41:00.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:41:00.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86612 16.76 MB/sec execute 198 sec latency 424.185 ms 2024-09-11T22:41:00.665 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.188+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/3917580358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0106f20 msgr2=0x7f3fb010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.188+0000 7f3fb4b2c640 1 --2- 172.21.15.5:0/3917580358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0106f20 0x7f3fb010eaa0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009ca0 tx=0x7f3fa802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.188+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/3917580358 shutdown_connections 2024-09-11T22:41:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.188+0000 7f3fb4b2c640 1 --2- 172.21.15.5:0/3917580358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0106f20 0x7f3fb010eaa0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.188+0000 7f3fb4b2c640 1 --2- 172.21.15.5:0/3917580358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb01027a0 0x7f3fb01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.188+0000 7f3fb4b2c640 1 --2- 172.21.15.5:0/3917580358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb0101df0 0x7f3fb01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.188+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/3917580358 >> 172.21.15.5:0/3917580358 conn(0x7f3fb00fd8f0 msgr2=0x7f3fb00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.188+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/3917580358 shutdown_connections 2024-09-11T22:41:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.188+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/3917580358 wait complete. 2024-09-11T22:41:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3fb4b2c640 1 Processor -- start 2024-09-11T22:41:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3fb4b2c640 1 -- start start 2024-09-11T22:41:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3fb4b2c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb0101df0 0x7f3fb019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3fb4b2c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb01027a0 0x7f3fb019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3fb4b2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0106f20 0x7f3fb019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3fb4b2c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0111f80 con 0x7f3fb0106f20 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3fb4b2c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0111e00 con 0x7f3fb01027a0 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3fb4b2c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb0112100 con 0x7f3fb0101df0 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3faed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0106f20 0x7f3fb019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3fadd9b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb01027a0 0x7f3fb019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3faed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0106f20 0x7f3fb019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45092/0 (socket says 172.21.15.5:45092) 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3faed9d640 1 -- 172.21.15.5:0/2882538019 learned_addr learned my addr 172.21.15.5:0/2882538019 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3fae59c640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb0101df0 0x7f3fb019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3faed9d640 1 -- 172.21.15.5:0/2882538019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb0101df0 msgr2=0x7f3fb019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3faed9d640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb0101df0 0x7f3fb019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3faed9d640 1 -- 172.21.15.5:0/2882538019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb01027a0 msgr2=0x7f3fb019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3faed9d640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb01027a0 0x7f3fb019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.189+0000 7f3faed9d640 1 -- 172.21.15.5:0/2882538019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb019f630 con 0x7f3fb0106f20 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.190+0000 7f3faed9d640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0106f20 0x7f3fb019ef50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8031a20 tx=0x7f3fa8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.190+0000 7f3f9f7fe640 1 -- 172.21.15.5:0/2882538019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8003c30 con 0x7f3fb0106f20 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.190+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/2882538019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb019f8c0 con 0x7f3fb0106f20 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.190+0000 7f3f9f7fe640 1 -- 172.21.15.5:0/2882538019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa8003dd0 con 0x7f3fb0106f20 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.190+0000 7f3f9f7fe640 1 -- 172.21.15.5:0/2882538019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa802fc40 con 0x7f3fb0106f20 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.190+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/2882538019 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb01a7200 con 0x7f3fb0106f20 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.192+0000 7f3f9f7fe640 1 -- 172.21.15.5:0/2882538019 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3fa8004050 con 0x7f3fb0106f20 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.192+0000 7f3fb6f05740 1 Processor -- start 2024-09-11T22:41:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fb6f05740 1 -- start start 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66db40 0x55b36d66c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66c5d0 0x55b36d66ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d679760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b36d5d22f0 con 0x55b36d677370 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b36d579cf0 con 0x55b36d66c5d0 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b36d5ce2b0 con 0x55b36d66db40 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fadd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d679760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fadd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d679760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45104/0 (socket says 172.21.15.5:45104) 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fadd9b640 1 -- 172.21.15.5:0/4089996898 learned_addr learned my addr 172.21.15.5:0/4089996898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fae59c640 1 --2- 172.21.15.5:0/4089996898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66c5d0 0x55b36d66ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/4089996898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66db40 0x55b36d66c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3f9f7fe640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3f980a2cf0 0x7f3f980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3faed9d640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3f980a2cf0 0x7f3f980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.193+0000 7f3fadd9b640 1 -- 172.21.15.5:0/4089996898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66db40 msgr2=0x55b36d66c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.194+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/4089996898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66db40 0x55b36d66c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.194+0000 7f3fadd9b640 1 -- 172.21.15.5:0/4089996898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66c5d0 msgr2=0x55b36d66ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.194+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/4089996898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66c5d0 0x55b36d66ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.194+0000 7f3fadd9b640 1 -- 172.21.15.5:0/4089996898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36d55e3a0 con 0x55b36d677370 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.194+0000 7f3f9f7fe640 1 -- 172.21.15.5:0/2882538019 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f3fa80f0630 con 0x7f3fb0106f20 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.194+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/4089996898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d679760 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4012740 tx=0x7f3fa400b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3252f907016b47f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.194+0000 7f3f9cff9640 1 -- 172.21.15.5:0/4089996898 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4012cc0 con 0x55b36d677370 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.194+0000 7f3faed9d640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3f980a2cf0 0x7f3f980a51b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80b91a0 tx=0x7f3fa802a8c0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.194+0000 7f3f9cff9640 1 -- 172.21.15.5:0/4089996898 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa4010c50 con 0x55b36d677370 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.194+0000 7f3f9cff9640 1 -- 172.21.15.5:0/4089996898 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa401eaf0 con 0x55b36d677370 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4089996898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 msgr2=0x55b36d679760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/4089996898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d679760 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4012740 tx=0x7f3fa400b700 comp rx=0 tx=0).stop 2024-09-11T22:41:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4089996898 shutdown_connections 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/4089996898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d679760 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/4089996898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66c5d0 0x55b36d66ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/4089996898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66db40 0x55b36d66c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4089996898 >> 172.21.15.5:0/4089996898 conn(0x55b36d635310 msgr2=0x55b36d668c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4089996898 shutdown_connections 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4089996898 wait complete. 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 -- start start 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.195+0000 7f3fb6f05740 1 Processor -- start 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d676d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66db40 0x55b36d674030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d674640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b36d67a070 con 0x55b36d677370 2024-09-11T22:41:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b36d58d060 con 0x55b36d66db40 2024-09-11T22:41:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55b36d67a1f0 con 0x55b36d66c5d0 2024-09-11T22:41:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fb6f05740 10 client.0 _ll_register_callbacks cb 0x55b36d2e08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fadd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d674640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fadd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d674640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45106/0 (socket says 172.21.15.5:45106) 2024-09-11T22:41:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fadd9b640 1 -- 172.21.15.5:0/3437561247 learned_addr learned my addr 172.21.15.5:0/3437561247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fae59c640 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66db40 0x55b36d674030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d676d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fadd9b640 1 -- 172.21.15.5:0/3437561247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 msgr2=0x55b36d676d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d676d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fadd9b640 1 -- 172.21.15.5:0/3437561247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66db40 msgr2=0x55b36d674030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66db40 0x55b36d674030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fadd9b640 1 -- 172.21.15.5:0/3437561247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36d3433c0 con 0x55b36d677370 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d674640 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400b440 tx=0x7f3fa4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a18369faef46924 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3f95ffb640 1 -- 172.21.15.5:0/3437561247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4024040 con 0x55b36d677370 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3f95ffb640 1 -- 172.21.15.5:0/3437561247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f3fa4010740 con 0x55b36d677370 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.196+0000 7f3fb6f05740 1 -- 172.21.15.5:0/3437561247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fa4010c50 con 0x55b36d677370 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3f95ffb640 1 -- 172.21.15.5:0/3437561247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f3fa4010940 con 0x55b36d677370 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3fb6f05740 10 client.21153 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3fb6f05740 1 -- 172.21.15.5:0/3437561247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55b36d685b40 con 0x55b36d677370 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3f95ffb640 1 -- 172.21.15.5:0/3437561247 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa4010c50 con 0x55b36d677370 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3f95ffb640 1 -- 172.21.15.5:0/3437561247 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f3fa402a030 con 0x55b36d677370 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3fb6f05740 10 client.21153 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3fb6f05740 10 client.21153 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3fb6f05740 20 client.21153 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55b36d686e90 0x55b36d689280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3fb6f05740 4 client.21153 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3fb6f05740 1 -- 172.21.15.5:0/3437561247 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b36d5df860 con 0x55b36d686e90 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.197+0000 7f3fae59c640 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55b36d686e90 0x55b36d689280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.198+0000 7f3f95ffb640 10 client.21153 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.198+0000 7f3fae59c640 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55b36d686e90 0x55b36d689280 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5801b82451f26b6 server_cookie=b0cda9cad958b83a in_seq=0 out_seq=0 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.198+0000 7f3f95ffb640 1 -- 172.21.15.5:0/3437561247 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b36d5df860 con 0x55b36d686e90 2024-09-11T22:41:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.198+0000 7f3f95ffb640 10 client.21153 handle_command_reply: tid=0 2024-09-11T22:41:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.198+0000 7f3fb6f05740 1 client.21153 shutdown 2024-09-11T22:41:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.198+0000 7f3fb6f05740 1 -- 172.21.15.5:0/3437561247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 msgr2=0x55b36d674640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.198+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d674640 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400b440 tx=0x7f3fa4014040 comp rx=0 tx=0).stop 2024-09-11T22:41:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.198+0000 7f3fb6f05740 1 -- 172.21.15.5:0/3437561247 shutdown_connections 2024-09-11T22:41:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.198+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55b36d686e90 0x55b36d689280 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.199+0000 7f3f95ffb640 0 client.21153 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:41:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.199+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d674640 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.199+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66db40 0x55b36d674030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.199+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/3437561247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d676d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.199+0000 7f3fb6f05740 1 -- 172.21.15.5:0/3437561247 >> 172.21.15.5:0/3437561247 conn(0x55b36d635310 msgr2=0x55b36d66e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:01.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.199+0000 7f3fb6f05740 1 -- 172.21.15.5:0/3437561247 shutdown_connections 2024-09-11T22:41:01.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.199+0000 7f3fb6f05740 1 -- 172.21.15.5:0/3437561247 wait complete. 2024-09-11T22:41:01.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.199+0000 7f3fb6f05740 20 client.21153 trim_cache size 0 max 16384 2024-09-11T22:41:01.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fb6f05740 1 Processor -- start 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fb6f05740 1 -- start start 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d66f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d66db40 0x55b36d66f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d677370 0x55b36d68e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01afcd0 con 0x55b36d66db40 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b1900 con 0x55b36d677370 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b03a0 con 0x55b36d66c5d0 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d66db40 0x55b36d66f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d66db40 0x55b36d66f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45116/0 (socket says 172.21.15.5:45116) 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fae59c640 1 -- 172.21.15.5:0/2872859026 learned_addr learned my addr 172.21.15.5:0/2872859026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/2872859026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d677370 0x55b36d68e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/2872859026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d66f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fae59c640 1 -- 172.21.15.5:0/2872859026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 msgr2=0x55b36d66f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fae59c640 1 --2- 172.21.15.5:0/2872859026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d66f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fae59c640 1 -- 172.21.15.5:0/2872859026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d677370 msgr2=0x55b36d68e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.212+0000 7f3fae59c640 1 --2- 172.21.15.5:0/2872859026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d677370 0x55b36d68e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.213+0000 7f3fae59c640 1 -- 172.21.15.5:0/2872859026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b36d66fba0 con 0x55b36d66db40 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.213+0000 7f3fae59c640 1 --2- 172.21.15.5:0/2872859026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d66db40 0x55b36d66f660 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x55b36d6738c0 tx=0x7f3fa0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed312a5eb90c471 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.213+0000 7f3f97fff640 1 -- 172.21.15.5:0/2872859026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0004250 con 0x55b36d66db40 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.213+0000 7f3f97fff640 1 -- 172.21.15.5:0/2872859026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa00043f0 con 0x55b36d66db40 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.213+0000 7f3f97fff640 1 -- 172.21.15.5:0/2872859026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00035f0 con 0x55b36d66db40 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.213+0000 7f3fb6f05740 1 -- 172.21.15.5:0/2872859026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d66db40 msgr2=0x55b36d66f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.213+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/2872859026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d66db40 0x55b36d66f660 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x55b36d6738c0 tx=0x7f3fa0005f90 comp rx=0 tx=0).stop 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 -- 172.21.15.5:0/2872859026 shutdown_connections 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/2872859026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d677370 0x55b36d68e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/2872859026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d66db40 0x55b36d66f660 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/2872859026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d66f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 -- 172.21.15.5:0/2872859026 >> 172.21.15.5:0/2872859026 conn(0x55b36d635310 msgr2=0x55b36d6760b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 -- 172.21.15.5:0/2872859026 shutdown_connections 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 -- 172.21.15.5:0/2872859026 wait complete. 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 -- start start 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 Processor -- start 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d67fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66db40 0x55b36d67ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d67b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b1230 con 0x55b36d677370 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01b0aa0 con 0x55b36d66db40 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb01adcd0 con 0x55b36d66c5d0 2024-09-11T22:41:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fb6f05740 10 client.0 _ll_register_callbacks cb 0x7f3fa0009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fadd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d67b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fadd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d67b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45124/0 (socket says 172.21.15.5:45124) 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fadd9b640 1 -- 172.21.15.5:0/4164439067 learned_addr learned my addr 172.21.15.5:0/4164439067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.214+0000 7f3fae59c640 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66db40 0x55b36d67ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d67fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3fadd9b640 1 -- 172.21.15.5:0/4164439067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 msgr2=0x55b36d67fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d67fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3fadd9b640 1 -- 172.21.15.5:0/4164439067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66db40 msgr2=0x55b36d67ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66db40 0x55b36d67ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3fadd9b640 1 -- 172.21.15.5:0/4164439067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b36d67b980 con 0x55b36d677370 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3fadd9b640 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d67b2e0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f3fa401e6d0 tx=0x7f3fa4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3c3b86a7e10fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3f95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3f95ffb640 1 -- 172.21.15.5:0/4164439067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4004220 con 0x55b36d677370 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3f95ffb640 1 -- 172.21.15.5:0/4164439067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f3fa40043c0 con 0x55b36d677370 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4164439067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fa00043f0 con 0x55b36d677370 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3f95ffb640 1 -- 172.21.15.5:0/4164439067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f3fa401e840 con 0x55b36d677370 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3fb6f05740 10 client.21159 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.215+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4164439067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55b36d6f1820 con 0x55b36d677370 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3f95ffb640 1 -- 172.21.15.5:0/4164439067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa00043f0 con 0x55b36d677370 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3f95ffb640 1 -- 172.21.15.5:0/4164439067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f3fa402a040 con 0x55b36d677370 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3fb6f05740 10 client.21159 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3fb6f05740 10 client.21159 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3fb6f05740 20 client.21159 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55b36d686e90 0x55b36d6f3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3fb6f05740 4 client.21159 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:41:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4164439067 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b36d5d9d70 con 0x55b36d686e90 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3fae59c640 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55b36d686e90 0x55b36d6f3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3fae59c640 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55b36d686e90 0x55b36d6f3f20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a55102173d7f1669 server_cookie=14fcf89b8457c6a1 in_seq=0 out_seq=0 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.216+0000 7f3f95ffb640 10 client.21159 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3f95ffb640 1 -- 172.21.15.5:0/4164439067 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b36d5d9d70 con 0x55b36d686e90 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3f95ffb640 10 client.21159 handle_command_reply: tid=0 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 client.21159 shutdown 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4164439067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 msgr2=0x55b36d67b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d67b2e0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f3fa401e6d0 tx=0x7f3fa4005850 comp rx=0 tx=0).stop 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4164439067 shutdown_connections 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x55b36d686e90 0x55b36d6f3f20 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b36d677370 0x55b36d67b2e0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b36d66db40 0x55b36d67ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3f95ffb640 0 client.21159 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:41:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 --2- 172.21.15.5:0/4164439067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b36d66c5d0 0x55b36d67fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4164439067 >> 172.21.15.5:0/4164439067 conn(0x55b36d635310 msgr2=0x55b36d6674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4164439067 shutdown_connections 2024-09-11T22:41:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 1 -- 172.21.15.5:0/4164439067 wait complete. 2024-09-11T22:41:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb6f05740 20 client.21159 trim_cache size 0 max 16384 2024-09-11T22:41:01.229 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:41:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/2882538019 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3f980a2cf0 msgr2=0x7f3f980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb4b2c640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3f980a2cf0 0x7f3f980a51b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80b91a0 tx=0x7f3fa802a8c0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/2882538019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0106f20 msgr2=0x7f3fb019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb4b2c640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0106f20 0x7f3fb019ef50 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8031a20 tx=0x7f3fa8031a50 comp rx=0 tx=0).stop 2024-09-11T22:41:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/2882538019 shutdown_connections 2024-09-11T22:41:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb4b2c640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fb0106f20 0x7f3fb019ef50 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb4b2c640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3f980a2cf0 0x7f3f980a51b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb4b2c640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3fb01027a0 0x7f3fb019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.217+0000 7f3fb4b2c640 1 --2- 172.21.15.5:0/2882538019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fb0101df0 0x7f3fb019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.218+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/2882538019 >> 172.21.15.5:0/2882538019 conn(0x7f3fb00fd8f0 msgr2=0x7f3fb00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.218+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/2882538019 shutdown_connections 2024-09-11T22:41:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:01.218+0000 7f3fb4b2c640 1 -- 172.21.15.5:0/2882538019 wait complete. 2024-09-11T22:41:01.652 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86612 16.67 MB/sec execute 199 sec latency 1424.291 ms 2024-09-11T22:41:02.074 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:01 smithi005 ceph-mon[30943]: pgmap v599: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.1 MiB/s wr, 342 op/s 2024-09-11T22:41:02.114 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:01 smithi172 ceph-mon[35461]: pgmap v599: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.1 MiB/s wr, 342 op/s 2024-09-11T22:41:02.115 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:01 smithi159 ceph-mon[35534]: pgmap v599: 97 pgs: 97 active+clean; 583 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 5.1 MiB/s wr, 342 op/s 2024-09-11T22:41:02.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:41:02.410 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:02.653 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86612 16.59 MB/sec execute 200 sec latency 2424.358 ms 2024-09-11T22:41:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.935+0000 7f38b157e640 1 -- 172.21.15.5:0/3168346157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38ac101e10 msgr2=0x7f38ac1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.935+0000 7f38b157e640 1 --2- 172.21.15.5:0/3168346157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38ac101e10 0x7f38ac1021f0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f389c009900 tx=0x7f389c02a410 comp rx=0 tx=0).stop 2024-09-11T22:41:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.935+0000 7f38b157e640 1 -- 172.21.15.5:0/3168346157 shutdown_connections 2024-09-11T22:41:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.935+0000 7f38b157e640 1 --2- 172.21.15.5:0/3168346157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38ac106f40 0x7f38ac10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.935+0000 7f38b157e640 1 --2- 172.21.15.5:0/3168346157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38ac1027c0 0x7f38ac106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.935+0000 7f38b157e640 1 --2- 172.21.15.5:0/3168346157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38ac101e10 0x7f38ac1021f0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.935+0000 7f38b157e640 1 -- 172.21.15.5:0/3168346157 >> 172.21.15.5:0/3168346157 conn(0x7f38ac0fd8d0 msgr2=0x7f38ac0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.935+0000 7f38b157e640 1 -- 172.21.15.5:0/3168346157 shutdown_connections 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.935+0000 7f38b157e640 1 -- 172.21.15.5:0/3168346157 wait complete. 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38b157e640 1 Processor -- start 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38b157e640 1 -- start start 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38b157e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38ac101e10 0x7f38ac198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38b157e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38ac1027c0 0x7f38ac1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38b157e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38ac106f40 0x7f38ac19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38b157e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac111fc0 con 0x7f38ac1027c0 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38b157e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac111e40 con 0x7f38ac106f40 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38b157e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac112140 con 0x7f38ac101e10 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38aa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38ac1027c0 0x7f38ac1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38aa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38ac1027c0 0x7f38ac1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36924/0 (socket says 172.21.15.5:36924) 2024-09-11T22:41:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38aa7fc640 1 -- 172.21.15.5:0/1493994013 learned_addr learned my addr 172.21.15.5:0/1493994013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.936+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38ac106f40 0x7f38ac19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f38aaffd640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38ac101e10 0x7f38ac198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f38aa7fc640 1 -- 172.21.15.5:0/1493994013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38ac101e10 msgr2=0x7f38ac198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38ac101e10 0x7f38ac198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f38aa7fc640 1 -- 172.21.15.5:0/1493994013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38ac106f40 msgr2=0x7f38ac19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38ac106f40 0x7f38ac19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f38aa7fc640 1 -- 172.21.15.5:0/1493994013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac19d430 con 0x7f38ac1027c0 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38ac1027c0 0x7f38ac1989c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f38a000d950 tx=0x7f38a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f3893fff640 1 -- 172.21.15.5:0/1493994013 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0017070 con 0x7f38ac1027c0 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f38b157e640 1 -- 172.21.15.5:0/1493994013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac19d720 con 0x7f38ac1027c0 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f3893fff640 1 -- 172.21.15.5:0/1493994013 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a0009c30 con 0x7f38ac1027c0 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f3893fff640 1 -- 172.21.15.5:0/1493994013 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a000f6b0 con 0x7f38ac1027c0 2024-09-11T22:41:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.937+0000 7f38b157e640 1 -- 172.21.15.5:0/1493994013 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac1a5000 con 0x7f38ac1027c0 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f3893fff640 1 -- 172.21.15.5:0/1493994013 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f38a00109d0 con 0x7f38ac1027c0 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38b3957740 1 Processor -- start 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38b3957740 1 -- start start 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38b3957740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e28090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38b3957740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e28a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38b3957740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e35760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38b3957740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6d8e2f0 con 0x55a3f6e29b40 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38b3957740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6d35cf0 con 0x55a3f6e33370 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38b3957740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6d8a2b0 con 0x55a3f6e285d0 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e28090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e28090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36932/0 (socket says 172.21.15.5:36932) 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.940+0000 7f38ab7fe640 1 -- 172.21.15.5:0/984426389 learned_addr learned my addr 172.21.15.5:0/984426389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f3893fff640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38940a2cf0 0x7f38940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38aaffd640 1 --2- 172.21.15.5:0/984426389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e28a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/984426389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e35760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38ab7fe640 1 -- 172.21.15.5:0/984426389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 msgr2=0x55a3f6e28a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/984426389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e28a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38ab7fe640 1 -- 172.21.15.5:0/984426389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 msgr2=0x55a3f6e35760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/984426389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e35760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38ab7fe640 1 -- 172.21.15.5:0/984426389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f6d1a3a0 con 0x55a3f6e29b40 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/984426389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e35760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f3893fff640 1 -- 172.21.15.5:0/1493994013 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f38a00cf1b0 con 0x7f38ac1027c0 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38940a2cf0 0x7f38940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/984426389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e28090 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f38a400a5d0 tx=0x7f38a400aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ad62ae899e79d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38917fa640 1 -- 172.21.15.5:0/984426389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a401a040 con 0x55a3f6e29b40 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38917fa640 1 -- 172.21.15.5:0/984426389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a4021ba0 con 0x55a3f6e29b40 2024-09-11T22:41:02.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.941+0000 7f38917fa640 1 -- 172.21.15.5:0/984426389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a401f350 con 0x55a3f6e29b40 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.942+0000 7f38b3957740 1 -- 172.21.15.5:0/984426389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 msgr2=0x55a3f6e28090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.942+0000 7f38b3957740 1 --2- 172.21.15.5:0/984426389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e28090 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f38a400a5d0 tx=0x7f38a400aa90 comp rx=0 tx=0).stop 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.942+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38940a2cf0 0x7f38940a51b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f38a4020000 tx=0x7f38a4021000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.942+0000 7f38b3957740 1 -- 172.21.15.5:0/984426389 shutdown_connections 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.942+0000 7f38b3957740 1 --2- 172.21.15.5:0/984426389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e35760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.942+0000 7f38b3957740 1 --2- 172.21.15.5:0/984426389 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e28a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.942+0000 7f38b3957740 1 --2- 172.21.15.5:0/984426389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e28090 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.942+0000 7f38b3957740 1 -- 172.21.15.5:0/984426389 >> 172.21.15.5:0/984426389 conn(0x55a3f6df1310 msgr2=0x55a3f6e24c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.942+0000 7f38b3957740 1 -- 172.21.15.5:0/984426389 shutdown_connections 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.942+0000 7f38b3957740 1 -- 172.21.15.5:0/984426389 wait complete. 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38b3957740 1 -- start start 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38b3957740 1 Processor -- start 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38b3957740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e32d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38b3957740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e30040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38b3957740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e30650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38b3957740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6e36070 con 0x55a3f6e29b40 2024-09-11T22:41:02.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38b3957740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6d49060 con 0x55a3f6e33370 2024-09-11T22:41:02.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38b3957740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55a3f6e361f0 con 0x55a3f6e285d0 2024-09-11T22:41:02.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38b3957740 10 client.0 _ll_register_callbacks cb 0x55a3f6a9c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38aa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e30040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38aa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e30040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36938/0 (socket says 172.21.15.5:36938) 2024-09-11T22:41:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38aa7fc640 1 -- 172.21.15.5:0/2800331963 learned_addr learned my addr 172.21.15.5:0/2800331963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e30650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.943+0000 7f38aaffd640 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e32d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f38aa7fc640 1 -- 172.21.15.5:0/2800331963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 msgr2=0x55a3f6e32d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e32d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f38aa7fc640 1 -- 172.21.15.5:0/2800331963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 msgr2=0x55a3f6e30650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e30650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f38aa7fc640 1 -- 172.21.15.5:0/2800331963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f6aff3c0 con 0x55a3f6e29b40 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e30040 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009610 tx=0x7f38a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88af47f734bbc6f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f388eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f388eff5640 1 -- 172.21.15.5:0/2800331963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a0020040 con 0x55a3f6e29b40 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f388eff5640 1 -- 172.21.15.5:0/2800331963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f38a00ccab0 con 0x55a3f6e29b40 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f38b3957740 1 -- 172.21.15.5:0/2800331963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a4021ba0 con 0x55a3f6e29b40 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f388eff5640 1 -- 172.21.15.5:0/2800331963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f38a00cccb0 con 0x55a3f6e29b40 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f38b3957740 10 client.21171 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f38b3957740 1 -- 172.21.15.5:0/2800331963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55a3f6e41ba0 con 0x55a3f6e29b40 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.944+0000 7f388eff5640 1 -- 172.21.15.5:0/2800331963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a4021ba0 con 0x55a3f6e29b40 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.945+0000 7f388eff5640 1 -- 172.21.15.5:0/2800331963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f38a00b4330 con 0x55a3f6e29b40 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.945+0000 7f38b3957740 10 client.21171 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.945+0000 7f38b3957740 10 client.21171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.945+0000 7f38b3957740 20 client.21171 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.945+0000 7f38b3957740 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3f6e430b0 0x55a3f6e45500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.945+0000 7f38b3957740 4 client.21171 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.945+0000 7f38b3957740 1 -- 172.21.15.5:0/2800331963 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3f6d9b860 con 0x55a3f6e430b0 2024-09-11T22:41:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.945+0000 7f38aaffd640 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3f6e430b0 0x55a3f6e45500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.945+0000 7f38aaffd640 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3f6e430b0 0x55a3f6e45500 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d33a6d3df54017 server_cookie=7a513bf5ed2bfcd2 in_seq=0 out_seq=0 2024-09-11T22:41:02.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.945+0000 7f388eff5640 10 client.21171 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f388eff5640 1 -- 172.21.15.5:0/2800331963 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3f6d9b860 con 0x55a3f6e430b0 2024-09-11T22:41:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f388eff5640 10 client.21171 handle_command_reply: tid=0 2024-09-11T22:41:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f38b3957740 1 client.21171 shutdown 2024-09-11T22:41:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f38b3957740 1 -- 172.21.15.5:0/2800331963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 msgr2=0x55a3f6e30040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f38b3957740 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e30040 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009610 tx=0x7f38a0005880 comp rx=0 tx=0).stop 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f38b3957740 1 -- 172.21.15.5:0/2800331963 shutdown_connections 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f38b3957740 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3f6e430b0 0x55a3f6e45500 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f38b3957740 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e30650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f388eff5640 0 client.21171 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f38b3957740 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e29b40 0x55a3f6e30040 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f38b3957740 1 --2- 172.21.15.5:0/2800331963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e32d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f38b3957740 1 -- 172.21.15.5:0/2800331963 >> 172.21.15.5:0/2800331963 conn(0x55a3f6df1310 msgr2=0x55a3f6e2a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.946+0000 7f38b3957740 1 -- 172.21.15.5:0/2800331963 shutdown_connections 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.947+0000 7f38b3957740 1 -- 172.21.15.5:0/2800331963 wait complete. 2024-09-11T22:41:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.947+0000 7f38b3957740 20 client.21171 trim_cache size 0 max 16384 2024-09-11T22:41:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38b3957740 1 Processor -- start 2024-09-11T22:41:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38b3957740 1 -- start start 2024-09-11T22:41:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38b3957740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e285d0 0x55a3f6e31fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38b3957740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e29b40 0x55a3f6e2b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38b3957740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38b3957740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1adaf0 con 0x55a3f6e285d0 2024-09-11T22:41:02.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38b3957740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1af720 con 0x55a3f6e33370 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38b3957740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1ae1c0 con 0x55a3f6e29b40 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e285d0 0x55a3f6e31fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e285d0 0x55a3f6e31fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36942/0 (socket says 172.21.15.5:36942) 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.960+0000 7f38aaffd640 1 -- 172.21.15.5:0/3911824078 learned_addr learned my addr 172.21.15.5:0/3911824078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/3911824078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e2b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/3911824078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e29b40 0x55a3f6e2b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f38aaffd640 1 -- 172.21.15.5:0/3911824078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e29b40 msgr2=0x55a3f6e2b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f38aaffd640 1 --2- 172.21.15.5:0/3911824078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e29b40 0x55a3f6e2b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f38aaffd640 1 -- 172.21.15.5:0/3911824078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 msgr2=0x55a3f6e2b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f38aaffd640 1 --2- 172.21.15.5:0/3911824078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e2b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f38aaffd640 1 -- 172.21.15.5:0/3911824078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3f6e330c0 con 0x55a3f6e285d0 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/3911824078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e29b40 0x55a3f6e2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:02.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f38aaffd640 1 --2- 172.21.15.5:0/3911824078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e285d0 0x55a3f6e31fb0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f389c02a930 tx=0x7f389c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e121a7a9cc89ec8f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:02.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f3890ff9640 1 -- 172.21.15.5:0/3911824078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c049070 con 0x55a3f6e285d0 2024-09-11T22:41:02.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f3890ff9640 1 -- 172.21.15.5:0/3911824078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c005520 con 0x55a3f6e285d0 2024-09-11T22:41:02.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.961+0000 7f3890ff9640 1 -- 172.21.15.5:0/3911824078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c035040 con 0x55a3f6e285d0 2024-09-11T22:41:02.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 -- 172.21.15.5:0/3911824078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e285d0 msgr2=0x55a3f6e31fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:02.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 --2- 172.21.15.5:0/3911824078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e285d0 0x55a3f6e31fb0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f389c02a930 tx=0x7f389c005dd0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 -- 172.21.15.5:0/3911824078 shutdown_connections 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 --2- 172.21.15.5:0/3911824078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e33370 0x55a3f6e2b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 --2- 172.21.15.5:0/3911824078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e29b40 0x55a3f6e2b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 --2- 172.21.15.5:0/3911824078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e285d0 0x55a3f6e31fb0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 -- 172.21.15.5:0/3911824078 >> 172.21.15.5:0/3911824078 conn(0x55a3f6df1310 msgr2=0x55a3f6e318a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 -- 172.21.15.5:0/3911824078 shutdown_connections 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 -- 172.21.15.5:0/3911824078 wait complete. 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 -- start start 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.962+0000 7f38b3957740 1 Processor -- start 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38b3957740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e2d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38b3957740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e29b40 0x55a3f6e2d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38b3957740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e33370 0x55a3f6e2dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38b3957740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1af050 con 0x55a3f6e33370 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38b3957740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1ae8c0 con 0x55a3f6e29b40 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38b3957740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1abaf0 con 0x55a3f6e285d0 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38b3957740 10 client.0 _ll_register_callbacks cb 0x7f38a000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e33370 0x55a3f6e2dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e33370 0x55a3f6e2dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36950/0 (socket says 172.21.15.5:36950) 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38ab7fe640 1 -- 172.21.15.5:0/35064269 learned_addr learned my addr 172.21.15.5:0/35064269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38aa7fc640 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e29b40 0x55a3f6e2d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38aaffd640 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e2d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38ab7fe640 1 -- 172.21.15.5:0/35064269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 msgr2=0x55a3f6e2d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e2d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38ab7fe640 1 -- 172.21.15.5:0/35064269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e29b40 msgr2=0x55a3f6e2d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:02.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e29b40 0x55a3f6e2d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38ab7fe640 1 -- 172.21.15.5:0/35064269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3f6eacde0 con 0x55a3f6e33370 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f38ab7fe640 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e33370 0x55a3f6e2dae0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f38a400a5d0 tx=0x7f38a401f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3cedefc35ef3f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f388eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f388eff5640 1 -- 172.21.15.5:0/35064269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a401a050 con 0x55a3f6e33370 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.963+0000 7f388eff5640 1 -- 172.21.15.5:0/35064269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f38a4021b20 con 0x55a3f6e33370 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f38b3957740 1 -- 172.21.15.5:0/35064269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f389c005520 con 0x55a3f6e33370 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f388eff5640 1 -- 172.21.15.5:0/35064269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f38a4009970 con 0x55a3f6e33370 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f38b3957740 10 client.21177 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f38b3957740 1 -- 172.21.15.5:0/35064269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55a3f6eae0c0 con 0x55a3f6e33370 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f388eff5640 1 -- 172.21.15.5:0/35064269 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f389c005520 con 0x55a3f6e33370 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f388eff5640 1 -- 172.21.15.5:0/35064269 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f38a4028040 con 0x55a3f6e33370 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f38b3957740 10 client.21177 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f38b3957740 10 client.21177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f38b3957740 20 client.21177 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f38b3957740 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3f6e430b0 0x55a3f6eb0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f38b3957740 4 client.21177 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f38b3957740 1 -- 172.21.15.5:0/35064269 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3f6d95d70 con 0x55a3f6e430b0 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.964+0000 7f38aaffd640 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3f6e430b0 0x55a3f6eb0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38aaffd640 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3f6e430b0 0x55a3f6eb0800 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a66033b5524be6e server_cookie=2e36bdee5f04a525 in_seq=0 out_seq=0 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f388eff5640 10 client.21177 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f388eff5640 1 -- 172.21.15.5:0/35064269 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3f6d95d70 con 0x55a3f6e430b0 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f388eff5640 10 client.21177 handle_command_reply: tid=0 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 client.21177 shutdown 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 -- 172.21.15.5:0/35064269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e33370 msgr2=0x55a3f6e2dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e33370 0x55a3f6e2dae0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f38a400a5d0 tx=0x7f38a401f350 comp rx=0 tx=0).stop 2024-09-11T22:41:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 -- 172.21.15.5:0/35064269 shutdown_connections 2024-09-11T22:41:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55a3f6e430b0 0x55a3f6eb0800 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a3f6e33370 0x55a3f6e2dae0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a3f6e29b40 0x55a3f6e2d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 --2- 172.21.15.5:0/35064269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a3f6e285d0 0x55a3f6e2d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 -- 172.21.15.5:0/35064269 >> 172.21.15.5:0/35064269 conn(0x55a3f6df1310 msgr2=0x55a3f6e320c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f388eff5640 0 client.21177 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 -- 172.21.15.5:0/35064269 shutdown_connections 2024-09-11T22:41:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 1 -- 172.21.15.5:0/35064269 wait complete. 2024-09-11T22:41:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.965+0000 7f38b3957740 20 client.21177 trim_cache size 0 max 16384 2024-09-11T22:41:02.972 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:41:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 -- 172.21.15.5:0/1493994013 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38940a2cf0 msgr2=0x7f38940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38940a2cf0 0x7f38940a51b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f38a4020000 tx=0x7f38a4021000 comp rx=0 tx=0).stop 2024-09-11T22:41:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 -- 172.21.15.5:0/1493994013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38ac1027c0 msgr2=0x7f38ac1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38ac1027c0 0x7f38ac1989c0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f38a000d950 tx=0x7f38a000de10 comp rx=0 tx=0).stop 2024-09-11T22:41:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 -- 172.21.15.5:0/1493994013 shutdown_connections 2024-09-11T22:41:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f38940a2cf0 0x7f38940a51b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38ac106f40 0x7f38ac19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38ac1027c0 0x7f38ac1989c0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 --2- 172.21.15.5:0/1493994013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38ac101e10 0x7f38ac198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 -- 172.21.15.5:0/1493994013 >> 172.21.15.5:0/1493994013 conn(0x7f38ac0fd8d0 msgr2=0x7f38ac0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 -- 172.21.15.5:0/1493994013 shutdown_connections 2024-09-11T22:41:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:02.966+0000 7f38b157e640 1 -- 172.21.15.5:0/1493994013 wait complete. 2024-09-11T22:41:03.653 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86612 16.51 MB/sec execute 201 sec latency 3424.394 ms 2024-09-11T22:41:03.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:03 smithi159 ceph-mon[35534]: pgmap v600: 97 pgs: 97 active+clean; 586 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 412 op/s 2024-09-11T22:41:04.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:03 smithi005 ceph-mon[30943]: pgmap v600: 97 pgs: 97 active+clean; 586 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 412 op/s 2024-09-11T22:41:04.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:03 smithi172 ceph-mon[35461]: pgmap v600: 97 pgs: 97 active+clean; 586 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.8 MiB/s wr, 412 op/s 2024-09-11T22:41:04.613 INFO:tasks.workunit.client.0.smithi005.stdout:[80191] write failed on handle 10053 (Resource temporarily unavailable) 2024-09-11T22:41:04.614 INFO:tasks.workunit.client.0.smithi005.stdout:Child failed with status 1 2024-09-11T22:41:04.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T22:41:04.749 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-11T22:41:04.749 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T22:41:05.038 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-11T22:41:05.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-11T22:41:05.084 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi005 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-11T22:41:05.295 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=2a3ab32de7034b2990d97f3e388cc52f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi005 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-11T22:41:05.300 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-11T22:41:06.375 INFO:tasks.quiescer:joining Quiescers 2024-09-11T22:41:06.375 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-11T22:41:06.376 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:41:06.393 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:05 smithi172 ceph-mon[35461]: pgmap v601: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 351 op/s 2024-09-11T22:41:06.394 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:05 smithi159 ceph-mon[35534]: pgmap v601: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 351 op/s 2024-09-11T22:41:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:06.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:05 smithi005 ceph-mon[30943]: pgmap v601: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.9 MiB/s wr, 351 op/s 2024-09-11T22:41:06.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.815+0000 7f347db39640 1 -- 172.21.15.5:0/3687138982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3478102570 msgr2=0x7f34781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.815+0000 7f347db39640 1 --2- 172.21.15.5:0/3687138982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3478102570 0x7f34781029b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f346c009900 tx=0x7f346c02a480 comp rx=0 tx=0).stop 2024-09-11T22:41:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.815+0000 7f347db39640 1 -- 172.21.15.5:0/3687138982 shutdown_connections 2024-09-11T22:41:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.815+0000 7f347db39640 1 --2- 172.21.15.5:0/3687138982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3478102ef0 0x7f347810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.815+0000 7f347db39640 1 --2- 172.21.15.5:0/3687138982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3478102570 0x7f34781029b0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.815+0000 7f347db39640 1 --2- 172.21.15.5:0/3687138982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478100e20 0x7f3478101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.815+0000 7f347db39640 1 -- 172.21.15.5:0/3687138982 >> 172.21.15.5:0/3687138982 conn(0x7f34780fc7d0 msgr2=0x7f34780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:06.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.815+0000 7f347db39640 1 -- 172.21.15.5:0/3687138982 shutdown_connections 2024-09-11T22:41:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.815+0000 7f347db39640 1 -- 172.21.15.5:0/3687138982 wait complete. 2024-09-11T22:41:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f347db39640 1 Processor -- start 2024-09-11T22:41:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f347db39640 1 -- start start 2024-09-11T22:41:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f347db39640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3478100e20 0x7f3478198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f347db39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3478102570 0x7f3478198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f347db39640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102ef0 0x7f347819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f347db39640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3478110f80 con 0x7f3478102570 2024-09-11T22:41:06.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f347db39640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3478110e00 con 0x7f3478102ef0 2024-09-11T22:41:06.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f347db39640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3478111100 con 0x7f3478100e20 2024-09-11T22:41:06.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f3476ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3478102570 0x7f3478198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f3476ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3478102570 0x7f3478198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36972/0 (socket says 172.21.15.5:36972) 2024-09-11T22:41:06.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f3476ffd640 1 -- 172.21.15.5:0/3016598105 learned_addr learned my addr 172.21.15.5:0/3016598105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:06.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f34777fe640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3478100e20 0x7f3478198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.816+0000 7f3477fff640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102ef0 0x7f347819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f3476ffd640 1 -- 172.21.15.5:0/3016598105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3478100e20 msgr2=0x7f3478198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:06.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f3476ffd640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3478100e20 0x7f3478198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f3476ffd640 1 -- 172.21.15.5:0/3016598105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102ef0 msgr2=0x7f347819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:06.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f3476ffd640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102ef0 0x7f347819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f3476ffd640 1 -- 172.21.15.5:0/3016598105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347819d4c0 con 0x7f3478102570 2024-09-11T22:41:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f34777fe640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3478100e20 0x7f3478198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f3476ffd640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3478102570 0x7f3478198a50 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f346c006010 tx=0x7f346c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f3474ff9640 1 -- 172.21.15.5:0/3016598105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c03f070 con 0x7f3478102570 2024-09-11T22:41:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f3474ff9640 1 -- 172.21.15.5:0/3016598105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f346c031c30 con 0x7f3478102570 2024-09-11T22:41:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f347db39640 1 -- 172.21.15.5:0/3016598105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347819d750 con 0x7f3478102570 2024-09-11T22:41:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f3474ff9640 1 -- 172.21.15.5:0/3016598105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c035030 con 0x7f3478102570 2024-09-11T22:41:06.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.817+0000 7f347db39640 1 -- 172.21.15.5:0/3016598105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34781a5090 con 0x7f3478102570 2024-09-11T22:41:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.819+0000 7f3474ff9640 1 -- 172.21.15.5:0/3016598105 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f346c042070 con 0x7f3478102570 2024-09-11T22:41:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f347ff12740 1 Processor -- start 2024-09-11T22:41:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f347ff12740 1 -- start start 2024-09-11T22:41:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f347ff12740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f356ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f347ff12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f357a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f347ff12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f357d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3474ff9640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f34600a2cf0 0x7f34600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f347ff12740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2f34d5280 con 0x55d2f356fb60 2024-09-11T22:41:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f347ff12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2f347cb10 con 0x55d2f3570480 2024-09-11T22:41:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f347ff12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d2f34d1240 con 0x55d2f357ad40 2024-09-11T22:41:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3477fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f356ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3477fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f356ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36984/0 (socket says 172.21.15.5:36984) 2024-09-11T22:41:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3477fff640 1 -- 172.21.15.5:0/1592175643 learned_addr learned my addr 172.21.15.5:0/1592175643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3477fff640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f34600a2cf0 0x7f34600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3476ffd640 1 --2- 172.21.15.5:0/1592175643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f357d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3474ff9640 1 -- 172.21.15.5:0/3016598105 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f346c0efc80 con 0x7f3478102570 2024-09-11T22:41:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f34777fe640 1 --2- 172.21.15.5:0/1592175643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f357a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3477fff640 1 -- 172.21.15.5:0/1592175643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 msgr2=0x55d2f357d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3477fff640 1 --2- 172.21.15.5:0/1592175643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f357d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3477fff640 1 -- 172.21.15.5:0/1592175643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 msgr2=0x55d2f357a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3477fff640 1 --2- 172.21.15.5:0/1592175643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f357a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f3477fff640 1 -- 172.21.15.5:0/1592175643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f34891f0 con 0x55d2f356fb60 2024-09-11T22:41:06.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.820+0000 7f34777fe640 1 --2- 172.21.15.5:0/1592175643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f357a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.821+0000 7f3477fff640 1 --2- 172.21.15.5:0/1592175643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f356ff40 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f34700094d0 tx=0x7f34700107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1728d3f09676821 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.821+0000 7f3477fff640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f34600a2cf0 0x7f34600a51b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3470005c40 tx=0x7f347001d000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.821+0000 7f345dffb640 1 -- 172.21.15.5:0/1592175643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3470012840 con 0x55d2f356fb60 2024-09-11T22:41:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.821+0000 7f345dffb640 1 -- 172.21.15.5:0/1592175643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3470003e50 con 0x55d2f356fb60 2024-09-11T22:41:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.821+0000 7f345dffb640 1 -- 172.21.15.5:0/1592175643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347000de00 con 0x55d2f356fb60 2024-09-11T22:41:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.821+0000 7f347ff12740 1 -- 172.21.15.5:0/1592175643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 msgr2=0x55d2f356ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.821+0000 7f347ff12740 1 --2- 172.21.15.5:0/1592175643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f356ff40 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f34700094d0 tx=0x7f34700107a0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 -- 172.21.15.5:0/1592175643 shutdown_connections 2024-09-11T22:41:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 --2- 172.21.15.5:0/1592175643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f357d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 --2- 172.21.15.5:0/1592175643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f357a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 --2- 172.21.15.5:0/1592175643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f356ff40 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 -- 172.21.15.5:0/1592175643 >> 172.21.15.5:0/1592175643 conn(0x55d2f356c0f0 msgr2=0x55d2f356c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 -- 172.21.15.5:0/1592175643 shutdown_connections 2024-09-11T22:41:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 -- 172.21.15.5:0/1592175643 wait complete. 2024-09-11T22:41:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 -- start start 2024-09-11T22:41:06.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 Processor -- start 2024-09-11T22:41:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3577ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f3578220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f3578760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2f357d910 con 0x55d2f356fb60 2024-09-11T22:41:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2f348ff10 con 0x55d2f3570480 2024-09-11T22:41:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.822+0000 7f347ff12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d2f357da90 con 0x55d2f357ad40 2024-09-11T22:41:06.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f347ff12740 10 client.0 _ll_register_callbacks cb 0x55d2f3458a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f34777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3577ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f34777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3577ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36990/0 (socket says 172.21.15.5:36990) 2024-09-11T22:41:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f3477fff640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f3578760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f34777fe640 1 -- 172.21.15.5:0/3434794364 learned_addr learned my addr 172.21.15.5:0/3434794364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f3476ffd640 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f3578220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f34777fe640 1 -- 172.21.15.5:0/3434794364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 msgr2=0x55d2f3578760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f34777fe640 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f3578760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f34777fe640 1 -- 172.21.15.5:0/3434794364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 msgr2=0x55d2f3578220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f34777fe640 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f3578220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f34777fe640 1 -- 172.21.15.5:0/3434794364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f32466a0 con 0x55d2f356fb60 2024-09-11T22:41:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f3476ffd640 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f3578220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f3477fff640 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f3578760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:41:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f34777fe640 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3577ce0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f3468013790 tx=0x7f3468013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60946f215fc7add9 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f3456ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:06.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f3456ffd640 1 -- 172.21.15.5:0/3434794364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468020060 con 0x55d2f356fb60 2024-09-11T22:41:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f3456ffd640 1 -- 172.21.15.5:0/3434794364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f3468011dd0 con 0x55d2f356fb60 2024-09-11T22:41:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f347ff12740 1 -- 172.21.15.5:0/3434794364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3470003e50 con 0x55d2f356fb60 2024-09-11T22:41:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f3456ffd640 1 -- 172.21.15.5:0/3434794364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f346800f450 con 0x55d2f356fb60 2024-09-11T22:41:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f347ff12740 10 client.21189 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.823+0000 7f347ff12740 1 -- 172.21.15.5:0/3434794364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d2f3589680 con 0x55d2f356fb60 2024-09-11T22:41:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.824+0000 7f3456ffd640 1 -- 172.21.15.5:0/3434794364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3470003e50 con 0x55d2f356fb60 2024-09-11T22:41:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.824+0000 7f3456ffd640 1 -- 172.21.15.5:0/3434794364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f346801a030 con 0x55d2f356fb60 2024-09-11T22:41:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.824+0000 7f347ff12740 10 client.21189 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.824+0000 7f347ff12740 10 client.21189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.824+0000 7f347ff12740 20 client.21189 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.824+0000 7f347ff12740 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d2f358ab40 0x55d2f358cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.824+0000 7f347ff12740 4 client.21189 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.824+0000 7f347ff12740 1 -- 172.21.15.5:0/3434794364 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2f34e27f0 con 0x55d2f358ab40 2024-09-11T22:41:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.824+0000 7f3476ffd640 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d2f358ab40 0x55d2f358cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f3456ffd640 10 client.21189 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f3476ffd640 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d2f358ab40 0x55d2f358cfc0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c9a61a3e4c05af server_cookie=ed23c91b5c44f9d5 in_seq=0 out_seq=0 2024-09-11T22:41:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f3456ffd640 1 -- 172.21.15.5:0/3434794364 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2f34e27f0 con 0x55d2f358ab40 2024-09-11T22:41:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f3456ffd640 10 client.21189 handle_command_reply: tid=0 2024-09-11T22:41:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f347ff12740 1 client.21189 shutdown 2024-09-11T22:41:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f347ff12740 1 -- 172.21.15.5:0/3434794364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 msgr2=0x55d2f3577ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f347ff12740 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3577ce0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f3468013790 tx=0x7f3468013c50 comp rx=0 tx=0).stop 2024-09-11T22:41:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f347ff12740 1 -- 172.21.15.5:0/3434794364 shutdown_connections 2024-09-11T22:41:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f347ff12740 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d2f358ab40 0x55d2f358cfc0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f347ff12740 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f3578760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.825+0000 7f347ff12740 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f3578220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.826+0000 7f347ff12740 1 --2- 172.21.15.5:0/3434794364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3577ce0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.826+0000 7f3456ffd640 0 client.21189 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.826+0000 7f347ff12740 1 -- 172.21.15.5:0/3434794364 >> 172.21.15.5:0/3434794364 conn(0x55d2f356c0f0 msgr2=0x55d2f356b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.826+0000 7f347ff12740 1 -- 172.21.15.5:0/3434794364 shutdown_connections 2024-09-11T22:41:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.826+0000 7f347ff12740 1 -- 172.21.15.5:0/3434794364 wait complete. 2024-09-11T22:41:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.826+0000 7f347ff12740 20 client.21189 trim_cache size 0 max 16384 2024-09-11T22:41:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.843+0000 7f347ff12740 1 Processor -- start 2024-09-11T22:41:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.843+0000 7f347ff12740 1 -- start start 2024-09-11T22:41:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.843+0000 7f347ff12740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3576c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.843+0000 7f347ff12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f35725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.843+0000 7f347ff12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f35928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.843+0000 7f347ff12740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34781ad9f0 con 0x55d2f356fb60 2024-09-11T22:41:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.843+0000 7f347ff12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34781af620 con 0x55d2f3570480 2024-09-11T22:41:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.843+0000 7f347ff12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34781ae0c0 con 0x55d2f357ad40 2024-09-11T22:41:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f34777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3576c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f34777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3576c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37002/0 (socket says 172.21.15.5:37002) 2024-09-11T22:41:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f34777fe640 1 -- 172.21.15.5:0/2183218578 learned_addr learned my addr 172.21.15.5:0/2183218578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f3477fff640 1 --2- 172.21.15.5:0/2183218578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f35928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f3476ffd640 1 --2- 172.21.15.5:0/2183218578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f35725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f34777fe640 1 -- 172.21.15.5:0/2183218578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 msgr2=0x55d2f35928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f34777fe640 1 --2- 172.21.15.5:0/2183218578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f35928e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f34777fe640 1 -- 172.21.15.5:0/2183218578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 msgr2=0x55d2f35725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f34777fe640 1 --2- 172.21.15.5:0/2183218578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f35725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f34777fe640 1 -- 172.21.15.5:0/2183218578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f358d640 con 0x55d2f356fb60 2024-09-11T22:41:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f34777fe640 1 --2- 172.21.15.5:0/2183218578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3576c20 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f34680059f0 tx=0x7f3468009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6480907b571a63e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f345d7fa640 1 -- 172.21.15.5:0/2183218578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468011dd0 con 0x55d2f356fb60 2024-09-11T22:41:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f345d7fa640 1 -- 172.21.15.5:0/2183218578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3468031070 con 0x55d2f356fb60 2024-09-11T22:41:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.844+0000 7f345d7fa640 1 -- 172.21.15.5:0/2183218578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468003850 con 0x55d2f356fb60 2024-09-11T22:41:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.845+0000 7f347ff12740 1 -- 172.21.15.5:0/2183218578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 msgr2=0x55d2f3576c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.845+0000 7f347ff12740 1 --2- 172.21.15.5:0/2183218578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3576c20 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f34680059f0 tx=0x7f3468009d60 comp rx=0 tx=0).stop 2024-09-11T22:41:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.845+0000 7f347ff12740 1 -- 172.21.15.5:0/2183218578 shutdown_connections 2024-09-11T22:41:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.845+0000 7f347ff12740 1 --2- 172.21.15.5:0/2183218578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f35928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.845+0000 7f347ff12740 1 --2- 172.21.15.5:0/2183218578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f3570480 0x55d2f35725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.845+0000 7f347ff12740 1 --2- 172.21.15.5:0/2183218578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f356fb60 0x55d2f3576c20 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.845+0000 7f347ff12740 1 -- 172.21.15.5:0/2183218578 >> 172.21.15.5:0/2183218578 conn(0x55d2f356c0f0 msgr2=0x55d2f3579c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.845+0000 7f347ff12740 1 -- 172.21.15.5:0/2183218578 shutdown_connections 2024-09-11T22:41:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.845+0000 7f347ff12740 1 -- 172.21.15.5:0/2183218578 wait complete. 2024-09-11T22:41:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.845+0000 7f347ff12740 1 -- start start 2024-09-11T22:41:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f347ff12740 1 Processor -- start 2024-09-11T22:41:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f347ff12740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f356fb60 0x55d2f357ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f347ff12740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f3570480 0x55d2f357f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f347ff12740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f357f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f347ff12740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34781aef50 con 0x55d2f3570480 2024-09-11T22:41:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f347ff12740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34781ae7c0 con 0x55d2f356fb60 2024-09-11T22:41:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f347ff12740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f34781ab9f0 con 0x55d2f357ad40 2024-09-11T22:41:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f347ff12740 10 client.0 _ll_register_callbacks cb 0x7f3468009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f3476ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f3570480 0x55d2f357f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f3476ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f3570480 0x55d2f357f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37004/0 (socket says 172.21.15.5:37004) 2024-09-11T22:41:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f3476ffd640 1 -- 172.21.15.5:0/3260396331 learned_addr learned my addr 172.21.15.5:0/3260396331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f34777fe640 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f356fb60 0x55d2f357ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f3477fff640 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f357f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f3476ffd640 1 -- 172.21.15.5:0/3260396331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 msgr2=0x55d2f357f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f3476ffd640 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f357f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f3476ffd640 1 -- 172.21.15.5:0/3260396331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f356fb60 msgr2=0x55d2f357ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f3476ffd640 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f356fb60 0x55d2f357ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f3476ffd640 1 -- 172.21.15.5:0/3260396331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f35f8e30 con 0x55d2f3570480 2024-09-11T22:41:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.846+0000 7f34777fe640 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f356fb60 0x55d2f357ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:41:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f3476ffd640 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f3570480 0x55d2f357f130 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f346c0b8b00 tx=0x7f346c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f558277107ae0776 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f3456ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f3456ffd640 1 -- 172.21.15.5:0/3260396331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c049040 con 0x55d2f3570480 2024-09-11T22:41:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f3456ffd640 1 -- 172.21.15.5:0/3260396331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f346c0dc0b0 con 0x55d2f3570480 2024-09-11T22:41:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f347ff12740 1 -- 172.21.15.5:0/3260396331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3468031070 con 0x55d2f3570480 2024-09-11T22:41:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f3456ffd640 1 -- 172.21.15.5:0/3260396331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f346c0bc130 con 0x55d2f3570480 2024-09-11T22:41:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f347ff12740 10 client.21195 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f347ff12740 1 -- 172.21.15.5:0/3260396331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d2f35fa190 con 0x55d2f3570480 2024-09-11T22:41:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f3456ffd640 1 -- 172.21.15.5:0/3260396331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3468031070 con 0x55d2f3570480 2024-09-11T22:41:06.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f3456ffd640 1 -- 172.21.15.5:0/3260396331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f346c0c0160 con 0x55d2f3570480 2024-09-11T22:41:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f347ff12740 10 client.21195 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.847+0000 7f347ff12740 10 client.21195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.848+0000 7f347ff12740 20 client.21195 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.848+0000 7f347ff12740 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d2f35fa980 0x55d2f35fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.848+0000 7f347ff12740 4 client.21195 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-11T22:41:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.848+0000 7f347ff12740 1 -- 172.21.15.5:0/3260396331 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d2f34dcd00 con 0x55d2f35fa980 2024-09-11T22:41:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.848+0000 7f34777fe640 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d2f35fa980 0x55d2f35fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.848+0000 7f34777fe640 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d2f35fa980 0x55d2f35fcda0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b26e4270ffe1cbe1 server_cookie=e83dc92fbfd33696 in_seq=0 out_seq=0 2024-09-11T22:41:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.848+0000 7f3456ffd640 10 client.21195 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f3456ffd640 1 -- 172.21.15.5:0/3260396331 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d2f34dcd00 con 0x55d2f35fa980 2024-09-11T22:41:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f3456ffd640 10 client.21195 handle_command_reply: tid=0 2024-09-11T22:41:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f347ff12740 1 client.21195 shutdown 2024-09-11T22:41:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f347ff12740 1 -- 172.21.15.5:0/3260396331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f3570480 msgr2=0x55d2f357f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f347ff12740 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f3570480 0x55d2f357f130 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f346c0b8b00 tx=0x7f346c0dc040 comp rx=0 tx=0).stop 2024-09-11T22:41:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f347ff12740 1 -- 172.21.15.5:0/3260396331 shutdown_connections 2024-09-11T22:41:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f347ff12740 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55d2f35fa980 0x55d2f35fcda0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f347ff12740 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d2f357ad40 0x55d2f357f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f347ff12740 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f3570480 0x55d2f357f130 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f3456ffd640 0 client.21195 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:06.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f347ff12740 1 --2- 172.21.15.5:0/3260396331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f356fb60 0x55d2f357ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f347ff12740 1 -- 172.21.15.5:0/3260396331 >> 172.21.15.5:0/3260396331 conn(0x55d2f356c0f0 msgr2=0x55d2f3579260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:06.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.849+0000 7f347ff12740 1 -- 172.21.15.5:0/3260396331 shutdown_connections 2024-09-11T22:41:06.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347ff12740 1 -- 172.21.15.5:0/3260396331 wait complete. 2024-09-11T22:41:06.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347ff12740 20 client.21195 trim_cache size 0 max 16384 2024-09-11T22:41:06.444 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:41:06.444 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-11T22:41:06.444 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "86475d5a-5ec2-418f-b266-7d3c2b341fe4", 2024-09-11T22:41:06.444 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-11T22:41:06.445 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:41:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 -- 172.21.15.5:0/3016598105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f34600a2cf0 msgr2=0x7f34600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f34600a2cf0 0x7f34600a51b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3470005c40 tx=0x7f347001d000 comp rx=0 tx=0).stop 2024-09-11T22:41:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 -- 172.21.15.5:0/3016598105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3478102570 msgr2=0x7f3478198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3478102570 0x7f3478198a50 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f346c006010 tx=0x7f346c003c10 comp rx=0 tx=0).stop 2024-09-11T22:41:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 -- 172.21.15.5:0/3016598105 shutdown_connections 2024-09-11T22:41:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f34600a2cf0 0x7f34600a51b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3478102ef0 0x7f347819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3478102570 0x7f3478198a50 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 --2- 172.21.15.5:0/3016598105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3478100e20 0x7f3478198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 -- 172.21.15.5:0/3016598105 >> 172.21.15.5:0/3016598105 conn(0x7f34780fc7d0 msgr2=0x7f347806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 -- 172.21.15.5:0/3016598105 shutdown_connections 2024-09-11T22:41:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:05.850+0000 7f347db39640 1 -- 172.21.15.5:0/3016598105 wait complete. 2024-09-11T22:41:06.899 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:41:06.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub queued for path: / 2024-09-11T22:41:06.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:06.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:41:06.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:41:06.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:06.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:41:06.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:06.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:41:06.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:06.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:41:06.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:06.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:41:06.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:06 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:06.961 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:07.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub queued for path: / 2024-09-11T22:41:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:41:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:41:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:41:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:41:07.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:41:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:41:07.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:06 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub queued for path: / 2024-09-11T22:41:07.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:41:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:41:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:41:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:41:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:41:07.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:41:07.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:06 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.119+0000 7f64a5d66640 1 -- 172.21.15.5:0/1015805130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64a0102c80 msgr2=0x7f64a0103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.119+0000 7f64a5d66640 1 --2- 172.21.15.5:0/1015805130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64a0102c80 0x7f64a0103060 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f6490009900 tx=0x7f649002a480 comp rx=0 tx=0).stop 2024-09-11T22:41:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.119+0000 7f64a5d66640 1 -- 172.21.15.5:0/1015805130 shutdown_connections 2024-09-11T22:41:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.119+0000 7f64a5d66640 1 --2- 172.21.15.5:0/1015805130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64a010c0d0 0x7f64a010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.119+0000 7f64a5d66640 1 --2- 172.21.15.5:0/1015805130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64a01035a0 0x7f64a010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.119+0000 7f64a5d66640 1 --2- 172.21.15.5:0/1015805130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64a0102c80 0x7f64a0103060 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.119+0000 7f64a5d66640 1 -- 172.21.15.5:0/1015805130 >> 172.21.15.5:0/1015805130 conn(0x7f64a0077990 msgr2=0x7f64a00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:07.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.119+0000 7f64a5d66640 1 -- 172.21.15.5:0/1015805130 shutdown_connections 2024-09-11T22:41:07.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.119+0000 7f64a5d66640 1 -- 172.21.15.5:0/1015805130 wait complete. 2024-09-11T22:41:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.120+0000 7f64a5d66640 1 Processor -- start 2024-09-11T22:41:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f64a5d66640 1 -- start start 2024-09-11T22:41:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f64a5d66640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64a0102c80 0x7f64a019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f64a5d66640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64a01035a0 0x7f64a019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f64a5d66640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64a010c0d0 0x7f64a019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f64a5d66640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0100b90 con 0x7f64a01035a0 2024-09-11T22:41:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f64a5d66640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0100a10 con 0x7f64a010c0d0 2024-09-11T22:41:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f64a5d66640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64a0100d10 con 0x7f64a0102c80 2024-09-11T22:41:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f649effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64a01035a0 0x7f64a019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f649effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64a01035a0 0x7f64a019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37012/0 (socket says 172.21.15.5:37012) 2024-09-11T22:41:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f649effd640 1 -- 172.21.15.5:0/2163344283 learned_addr learned my addr 172.21.15.5:0/2163344283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:07.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f649f7fe640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64a0102c80 0x7f64a019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.121+0000 7f649ffff640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64a010c0d0 0x7f64a019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.122+0000 7f649f7fe640 1 -- 172.21.15.5:0/2163344283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64a010c0d0 msgr2=0x7f64a019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.122+0000 7f649f7fe640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64a010c0d0 0x7f64a019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.122+0000 7f649f7fe640 1 -- 172.21.15.5:0/2163344283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64a01035a0 msgr2=0x7f64a019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.122+0000 7f649f7fe640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64a01035a0 0x7f64a019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.122+0000 7f649f7fe640 1 -- 172.21.15.5:0/2163344283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a019f5a0 con 0x7f64a0102c80 2024-09-11T22:41:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.122+0000 7f649effd640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64a01035a0 0x7f64a019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:41:07.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.122+0000 7f649ffff640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64a010c0d0 0x7f64a019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.122+0000 7f649f7fe640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64a0102c80 0x7f64a019ae20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6490005940 tx=0x7f6490004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.122+0000 7f649cff9640 1 -- 172.21.15.5:0/2163344283 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649003f030 con 0x7f64a0102c80 2024-09-11T22:41:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.122+0000 7f649cff9640 1 -- 172.21.15.5:0/2163344283 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6490003ad0 con 0x7f64a0102c80 2024-09-11T22:41:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.123+0000 7f649cff9640 1 -- 172.21.15.5:0/2163344283 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490035070 con 0x7f64a0102c80 2024-09-11T22:41:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.123+0000 7f64a5d66640 1 -- 172.21.15.5:0/2163344283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a019f830 con 0x7f64a0102c80 2024-09-11T22:41:07.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.123+0000 7f64a5d66640 1 -- 172.21.15.5:0/2163344283 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a01a7120 con 0x7f64a0102c80 2024-09-11T22:41:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.125+0000 7f649cff9640 1 -- 172.21.15.5:0/2163344283 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6490042030 con 0x7f64a0102c80 2024-09-11T22:41:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.126+0000 7f649cff9640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64880a2e20 0x7f64880a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.126+0000 7f649effd640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64880a2e20 0x7f64880a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.126+0000 7f649cff9640 1 -- 172.21.15.5:0/2163344283 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f64900ef4d0 con 0x7f64a0102c80 2024-09-11T22:41:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.126+0000 7f64a8bdb740 1 Processor -- start 2024-09-11T22:41:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f64a8bdb740 1 -- start start 2024-09-11T22:41:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649effd640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64880a2e20 0x7f64880a52e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f64a00ff460 tx=0x7f6494005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x562824932ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x5628249400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5628248972f0 con 0x562824933410 2024-09-11T22:41:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56282483f270 con 0x56282493dcb0 2024-09-11T22:41:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5628248932b0 con 0x562824932af0 2024-09-11T22:41:07.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37016/0 (socket says 172.21.15.5:37016) 2024-09-11T22:41:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649effd640 1 -- 172.21.15.5:0/2560561364 learned_addr learned my addr 172.21.15.5:0/2560561364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:07.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649f7fe640 1 --2- 172.21.15.5:0/2560561364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x562824932ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649ffff640 1 --2- 172.21.15.5:0/2560561364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x5628249400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649effd640 1 -- 172.21.15.5:0/2560561364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 msgr2=0x562824932ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649effd640 1 --2- 172.21.15.5:0/2560561364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x562824932ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649effd640 1 -- 172.21.15.5:0/2560561364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 msgr2=0x5628249400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649effd640 1 --2- 172.21.15.5:0/2560561364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x5628249400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649effd640 1 -- 172.21.15.5:0/2560561364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56282483e340 con 0x562824933410 2024-09-11T22:41:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.127+0000 7f649effd640 1 --2- 172.21.15.5:0/2560561364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493d770 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f649400f6f0 tx=0x7f6494014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573f5ff6a66e827e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.128+0000 7f6485ffb640 1 -- 172.21.15.5:0/2560561364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649402b070 con 0x562824933410 2024-09-11T22:41:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.128+0000 7f6485ffb640 1 -- 172.21.15.5:0/2560561364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f649401ccb0 con 0x562824933410 2024-09-11T22:41:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.128+0000 7f6485ffb640 1 -- 172.21.15.5:0/2560561364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649400e3b0 con 0x562824933410 2024-09-11T22:41:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 -- 172.21.15.5:0/2560561364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 msgr2=0x56282493d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/2560561364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493d770 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f649400f6f0 tx=0x7f6494014000 comp rx=0 tx=0).stop 2024-09-11T22:41:07.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 -- 172.21.15.5:0/2560561364 shutdown_connections 2024-09-11T22:41:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/2560561364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x5628249400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/2560561364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493d770 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/2560561364 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x562824932ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 -- 172.21.15.5:0/2560561364 >> 172.21.15.5:0/2560561364 conn(0x56282492f060 msgr2=0x56282492f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 -- 172.21.15.5:0/2560561364 shutdown_connections 2024-09-11T22:41:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 -- 172.21.15.5:0/2560561364 wait complete. 2024-09-11T22:41:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 -- start start 2024-09-11T22:41:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.129+0000 7f64a8bdb740 1 Processor -- start 2024-09-11T22:41:07.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x56282493abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x56282493b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562824940880 con 0x562824933410 2024-09-11T22:41:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562824820290 con 0x56282493dcb0 2024-09-11T22:41:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x562824940a00 con 0x562824932af0 2024-09-11T22:41:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f64a8bdb740 10 client.0 _ll_register_callbacks cb 0x562824851140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:07.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f649effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f649effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37026/0 (socket says 172.21.15.5:37026) 2024-09-11T22:41:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f649effd640 1 -- 172.21.15.5:0/763835452 learned_addr learned my addr 172.21.15.5:0/763835452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f649effd640 1 -- 172.21.15.5:0/763835452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 msgr2=0x56282493abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f649effd640 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x56282493abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f649effd640 1 -- 172.21.15.5:0/763835452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 msgr2=0x56282493b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f649effd640 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x56282493b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f649effd640 1 -- 172.21.15.5:0/763835452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628245aa210 con 0x562824933410 2024-09-11T22:41:07.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f649effd640 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493b120 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f6494029040 tx=0x7f6494026bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702d8d9c3ca37e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:07.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f647effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.130+0000 7f647effd640 1 -- 172.21.15.5:0/763835452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649402b030 con 0x562824933410 2024-09-11T22:41:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f64a8bdb740 1 -- 172.21.15.5:0/763835452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f649401ccb0 con 0x562824933410 2024-09-11T22:41:07.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f647effd640 1 -- 172.21.15.5:0/763835452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f649401c5d0 con 0x562824933410 2024-09-11T22:41:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f647effd640 1 -- 172.21.15.5:0/763835452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f649401ce60 con 0x562824933410 2024-09-11T22:41:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f64a8bdb740 10 client.21207 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f64a8bdb740 1 -- 172.21.15.5:0/763835452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x562824676620 con 0x562824933410 2024-09-11T22:41:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f647effd640 1 -- 172.21.15.5:0/763835452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f649401ccb0 con 0x562824933410 2024-09-11T22:41:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f647effd640 1 -- 172.21.15.5:0/763835452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f6494004830 con 0x562824933410 2024-09-11T22:41:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f64a8bdb740 10 client.21207 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:07.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f64a8bdb740 10 client.21207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f64a8bdb740 20 client.21207 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56282494dad0 0x56282494fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f64a8bdb740 4 client.21207 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.131+0000 7f64a8bdb740 1 -- 172.21.15.5:0/763835452 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628248a4a20 con 0x56282494dad0 2024-09-11T22:41:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.132+0000 7f649f7fe640 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56282494dad0 0x56282494fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.134+0000 7f649f7fe640 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56282494dad0 0x56282494fec0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3620a2ebd8ce970a server_cookie=3edc2c84d66eb7e in_seq=0 out_seq=0 2024-09-11T22:41:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.134+0000 7f647effd640 10 client.21207 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:07.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f647effd640 1 -- 172.21.15.5:0/763835452 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5628248a4a20 con 0x56282494dad0 2024-09-11T22:41:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f647effd640 10 client.21207 handle_command_reply: tid=0 2024-09-11T22:41:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 client.21207 shutdown 2024-09-11T22:41:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 -- 172.21.15.5:0/763835452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 msgr2=0x56282493b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493b120 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f6494029040 tx=0x7f6494026bc0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 -- 172.21.15.5:0/763835452 shutdown_connections 2024-09-11T22:41:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56282494dad0 0x56282494fec0 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x56282493b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824933410 0x56282493b120 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f647effd640 0 client.21207 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/763835452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x56282493abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 -- 172.21.15.5:0/763835452 >> 172.21.15.5:0/763835452 conn(0x56282492f060 msgr2=0x562824934d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 -- 172.21.15.5:0/763835452 shutdown_connections 2024-09-11T22:41:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 1 -- 172.21.15.5:0/763835452 wait complete. 2024-09-11T22:41:07.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.137+0000 7f64a8bdb740 20 client.21207 trim_cache size 0 max 16384 2024-09-11T22:41:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.159+0000 7f64a8bdb740 1 Processor -- start 2024-09-11T22:41:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.159+0000 7f64a8bdb740 1 -- start start 2024-09-11T22:41:07.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824932af0 0x56282493b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824933410 0x56282493cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x5628249557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01afcc0 con 0x562824932af0 2024-09-11T22:41:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01b18f0 con 0x56282493dcb0 2024-09-11T22:41:07.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01b0390 con 0x562824933410 2024-09-11T22:41:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824932af0 0x56282493b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824932af0 0x56282493b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37040/0 (socket says 172.21.15.5:37040) 2024-09-11T22:41:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649f7fe640 1 -- 172.21.15.5:0/2794387756 learned_addr learned my addr 172.21.15.5:0/2794387756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649effd640 1 --2- 172.21.15.5:0/2794387756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824933410 0x56282493cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649ffff640 1 --2- 172.21.15.5:0/2794387756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x5628249557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649f7fe640 1 -- 172.21.15.5:0/2794387756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824933410 msgr2=0x56282493cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649f7fe640 1 --2- 172.21.15.5:0/2794387756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824933410 0x56282493cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649f7fe640 1 -- 172.21.15.5:0/2794387756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 msgr2=0x5628249557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649f7fe640 1 --2- 172.21.15.5:0/2794387756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x5628249557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649f7fe640 1 -- 172.21.15.5:0/2794387756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562824940410 con 0x562824932af0 2024-09-11T22:41:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f649f7fe640 1 --2- 172.21.15.5:0/2794387756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824932af0 0x56282493b490 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f64900b8330 tx=0x7f64900a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da524d493970b3a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f64857fa640 1 -- 172.21.15.5:0/2794387756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490049040 con 0x562824932af0 2024-09-11T22:41:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f64857fa640 1 -- 172.21.15.5:0/2794387756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64900bc060 con 0x562824932af0 2024-09-11T22:41:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.160+0000 7f64857fa640 1 -- 172.21.15.5:0/2794387756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64900b7fd0 con 0x562824932af0 2024-09-11T22:41:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.161+0000 7f64a8bdb740 1 -- 172.21.15.5:0/2794387756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824932af0 msgr2=0x56282493b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.161+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/2794387756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824932af0 0x56282493b490 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f64900b8330 tx=0x7f64900a6040 comp rx=0 tx=0).stop 2024-09-11T22:41:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.161+0000 7f64a8bdb740 1 -- 172.21.15.5:0/2794387756 shutdown_connections 2024-09-11T22:41:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.161+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/2794387756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56282493dcb0 0x5628249557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.161+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/2794387756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824933410 0x56282493cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.161+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/2794387756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562824932af0 0x56282493b490 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.161+0000 7f64a8bdb740 1 -- 172.21.15.5:0/2794387756 >> 172.21.15.5:0/2794387756 conn(0x56282492f060 msgr2=0x56282493c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.161+0000 7f64a8bdb740 1 -- 172.21.15.5:0/2794387756 shutdown_connections 2024-09-11T22:41:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.161+0000 7f64a8bdb740 1 -- 172.21.15.5:0/2794387756 wait complete. 2024-09-11T22:41:07.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f64a8bdb740 1 -- start start 2024-09-11T22:41:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f64a8bdb740 1 Processor -- start 2024-09-11T22:41:07.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x562824946010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562824933410 0x562824946550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f64a8bdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56282493dcb0 0x562824937740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01b1220 con 0x56282493dcb0 2024-09-11T22:41:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01b0a90 con 0x562824933410 2024-09-11T22:41:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f64a8bdb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f64a01adcc0 con 0x562824932af0 2024-09-11T22:41:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f64a8bdb740 10 client.0 _ll_register_callbacks cb 0x7f6494011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:07.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f649ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56282493dcb0 0x562824937740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f649ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56282493dcb0 0x562824937740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37056/0 (socket says 172.21.15.5:37056) 2024-09-11T22:41:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f649ffff640 1 -- 172.21.15.5:0/1069951518 learned_addr learned my addr 172.21.15.5:0/1069951518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f649effd640 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562824933410 0x562824946550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.162+0000 7f649f7fe640 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x562824946010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f649ffff640 1 -- 172.21.15.5:0/1069951518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 msgr2=0x562824946010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f649ffff640 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x562824946010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f649ffff640 1 -- 172.21.15.5:0/1069951518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562824933410 msgr2=0x562824946550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f649ffff640 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562824933410 0x562824946550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f649ffff640 1 -- 172.21.15.5:0/1069951518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562824937f40 con 0x56282493dcb0 2024-09-11T22:41:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f649effd640 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562824933410 0x562824946550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:41:07.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f649ffff640 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56282493dcb0 0x562824937740 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f6498004f40 tx=0x7f64980054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a7b2f9fd105537 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f647effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f647effd640 1 -- 172.21.15.5:0/1069951518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498012a60 con 0x56282493dcb0 2024-09-11T22:41:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f64a8bdb740 1 -- 172.21.15.5:0/1069951518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64900bc060 con 0x56282493dcb0 2024-09-11T22:41:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f647effd640 1 -- 172.21.15.5:0/1069951518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f6498012c00 con 0x56282493dcb0 2024-09-11T22:41:07.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f647effd640 1 -- 172.21.15.5:0/1069951518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f6498012e00 con 0x56282493dcb0 2024-09-11T22:41:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f64a8bdb740 10 client.21213 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f64a8bdb740 1 -- 172.21.15.5:0/1069951518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5628249b8410 con 0x56282493dcb0 2024-09-11T22:41:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f647effd640 1 -- 172.21.15.5:0/1069951518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64900bc060 con 0x56282493dcb0 2024-09-11T22:41:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f647effd640 1 -- 172.21.15.5:0/1069951518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f6498019070 con 0x56282493dcb0 2024-09-11T22:41:07.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f64a8bdb740 10 client.21213 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f64a8bdb740 10 client.21213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f64a8bdb740 20 client.21213 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.163+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5628249b8ca0 0x5628249bb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.164+0000 7f64a8bdb740 4 client.21213 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-11T22:41:07.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.164+0000 7f64a8bdb740 1 -- 172.21.15.5:0/1069951518 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56282489ed70 con 0x5628249b8ca0 2024-09-11T22:41:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.164+0000 7f649f7fe640 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5628249b8ca0 0x5628249bb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.164+0000 7f649f7fe640 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5628249b8ca0 0x5628249bb0c0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60ba1d0f32d70ed server_cookie=c3b4473f52952e71 in_seq=0 out_seq=0 2024-09-11T22:41:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.164+0000 7f647effd640 10 client.21213 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:07.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.554+0000 7f9de8bf7640 1 -- 172.21.15.5:0/1870462258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc0094f0 msgr2=0x7f9ddc0a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.554+0000 7f9de8bf7640 1 --2- 172.21.15.5:0/1870462258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc0094f0 0x7f9ddc0a5ef0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009860 tx=0x7f9dd002a340 comp rx=0 tx=0).stop 2024-09-11T22:41:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.554+0000 7f9de8bf7640 1 -- 172.21.15.5:0/1870462258 shutdown_connections 2024-09-11T22:41:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.554+0000 7f9de8bf7640 1 --2- 172.21.15.5:0/1870462258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ddc0a6430 0x7f9ddc0a8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.554+0000 7f9de8bf7640 1 --2- 172.21.15.5:0/1870462258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc0094f0 0x7f9ddc0a5ef0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.554+0000 7f9de8bf7640 1 --2- 172.21.15.5:0/1870462258 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ddc008bd0 0x7f9ddc008fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.554+0000 7f9de8bf7640 1 -- 172.21.15.5:0/1870462258 >> 172.21.15.5:0/1870462258 conn(0x7f9ddc093490 msgr2=0x7f9ddc0958b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:07.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.554+0000 7f9de8bf7640 1 -- 172.21.15.5:0/1870462258 shutdown_connections 2024-09-11T22:41:07.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.554+0000 7f9de8bf7640 1 -- 172.21.15.5:0/1870462258 wait complete. 2024-09-11T22:41:07.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.554+0000 7f9de8bf7640 1 Processor -- start 2024-09-11T22:41:07.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de8bf7640 1 -- start start 2024-09-11T22:41:07.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de8bf7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ddc008bd0 0x7f9ddc132470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de8bf7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc0094f0 0x7f9ddc1329b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de8bf7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ddc0a6430 0x7f9ddc136d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de8bf7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0abf30 con 0x7f9ddc0a6430 2024-09-11T22:41:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de8bf7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0abdb0 con 0x7f9ddc008bd0 2024-09-11T22:41:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc0094f0 0x7f9ddc1329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de2ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc0094f0 0x7f9ddc1329b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:44232/0 (socket says 172.21.15.5:44232) 2024-09-11T22:41:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ddc0a6430 0x7f9ddc136d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de2ffd640 1 -- 172.21.15.5:0/3909498524 learned_addr learned my addr 172.21.15.5:0/3909498524 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc0ac0b0 con 0x7f9ddc0094f0 2024-09-11T22:41:07.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.555+0000 7f9de37fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ddc008bd0 0x7f9ddc132470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de2ffd640 1 -- 172.21.15.5:0/3909498524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ddc008bd0 msgr2=0x7f9ddc132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de2ffd640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ddc008bd0 0x7f9ddc132470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de2ffd640 1 -- 172.21.15.5:0/3909498524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ddc0a6430 msgr2=0x7f9ddc136d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de2ffd640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ddc0a6430 0x7f9ddc136d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de2ffd640 1 -- 172.21.15.5:0/3909498524 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc137420 con 0x7f9ddc0094f0 2024-09-11T22:41:07.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de2ffd640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc0094f0 0x7f9ddc1329b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0035040 tx=0x7f9dd0033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f647effd640 1 -- 172.21.15.5:0/1069951518 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56282489ed70 con 0x5628249b8ca0 2024-09-11T22:41:07.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f647effd640 10 client.21213 handle_command_reply: tid=0 2024-09-11T22:41:07.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.557+0000 7f64a8bdb740 1 client.21213 shutdown 2024-09-11T22:41:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de0ff9640 1 -- 172.21.15.5:0/3909498524 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd003f020 con 0x7f9ddc0094f0 2024-09-11T22:41:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc1376b0 con 0x7f9ddc0094f0 2024-09-11T22:41:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc13eff0 con 0x7f9ddc0094f0 2024-09-11T22:41:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de0ff9640 1 -- 172.21.15.5:0/3909498524 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dd0004240 con 0x7f9ddc0094f0 2024-09-11T22:41:07.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.556+0000 7f9de0ff9640 1 -- 172.21.15.5:0/3909498524 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0037040 con 0x7f9ddc0094f0 2024-09-11T22:41:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.558+0000 7f64a8bdb740 1 -- 172.21.15.5:0/1069951518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56282493dcb0 msgr2=0x562824937740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.558+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56282493dcb0 0x562824937740 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f6498004f40 tx=0x7f64980054d0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.558+0000 7f64a8bdb740 1 -- 172.21.15.5:0/1069951518 shutdown_connections 2024-09-11T22:41:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.558+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5628249b8ca0 0x5628249bb0c0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.558+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56282493dcb0 0x562824937740 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.558+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562824933410 0x562824946550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.558+0000 7f64a8bdb740 1 --2- 172.21.15.5:0/1069951518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562824932af0 0x562824946010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.558+0000 7f64a8bdb740 1 -- 172.21.15.5:0/1069951518 >> 172.21.15.5:0/1069951518 conn(0x56282492f060 msgr2=0x56282492e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:07.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.558+0000 7f647effd640 0 client.21213 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:07.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.558+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc8005290 con 0x7f9ddc0094f0 2024-09-11T22:41:07.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.560+0000 7f64a8bdb740 1 -- 172.21.15.5:0/1069951518 shutdown_connections 2024-09-11T22:41:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.560+0000 7f64a8bdb740 1 -- 172.21.15.5:0/1069951518 wait complete. 2024-09-11T22:41:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.561+0000 7f64a8bdb740 20 client.21213 trim_cache size 0 max 16384 2024-09-11T22:41:07.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.560+0000 7f9de0ff9640 1 -- 172.21.15.5:0/3909498524 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9dd0042020 con 0x7f9ddc0094f0 2024-09-11T22:41:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2024-09-11T22:41:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2024-09-11T22:41:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2024-09-11T22:41:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2024-09-11T22:41:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.561+0000 7f64a5d66640 1 -- 172.21.15.5:0/2163344283 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64880a2e20 msgr2=0x7f64880a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64880a2e20 0x7f64880a52e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f64a00ff460 tx=0x7f6494005f90 comp rx=0 tx=0).stop 2024-09-11T22:41:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 -- 172.21.15.5:0/2163344283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64a0102c80 msgr2=0x7f64a019ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64a0102c80 0x7f64a019ae20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6490005940 tx=0x7f6490004270 comp rx=0 tx=0).stop 2024-09-11T22:41:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.561+0000 7f9de0ff9640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9dd40a2e00 0x7f9dd40a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:07.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f9de0ff9640 1 -- 172.21.15.5:0/3909498524 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f9dd00ef850 con 0x7f9ddc0094f0 2024-09-11T22:41:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 -- 172.21.15.5:0/2163344283 shutdown_connections 2024-09-11T22:41:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f64880a2e20 0x7f64880a52e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f64a010c0d0 0x7f64a019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64a01035a0 0x7f64a019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 --2- 172.21.15.5:0/2163344283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64a0102c80 0x7f64a019ae20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 -- 172.21.15.5:0/2163344283 >> 172.21.15.5:0/2163344283 conn(0x7f64a0077990 msgr2=0x7f64a010d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 -- 172.21.15.5:0/2163344283 shutdown_connections 2024-09-11T22:41:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2024-09-11T22:41:07.562+0000 7f64a5d66640 1 -- 172.21.15.5:0/2163344283 wait complete. 2024-09-11T22:41:07.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.563+0000 7f9de37fe640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9dd40a2e00 0x7f9dd40a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:07.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.563+0000 7f9de37fe640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9dd40a2e00 0x7f9dd40a52c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80095f0 tx=0x7f9dd8005f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:07.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.566+0000 7f9de0ff9640 1 -- 172.21.15.5:0/3909498524 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9dd0049020 con 0x7f9ddc0094f0 2024-09-11T22:41:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:07.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.792+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9dc8005580 con 0x7f9ddc0094f0 2024-09-11T22:41:07.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:07 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:41:07.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:07 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:07 smithi005 ceph-mon[30943]: scrub summary: active paths [/] 2024-09-11T22:41:07.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:07 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:07 smithi005 ceph-mon[30943]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:07.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:07 smithi005 ceph-mon[30943]: pgmap v602: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 303 op/s 2024-09-11T22:41:07.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.793+0000 7f9de0ff9640 1 -- 172.21.15.5:0/3909498524 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11975 (secure 0 0 0) 0x7f9dd00bc530 con 0x7f9ddc0094f0 2024-09-11T22:41:07.796 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:41:07.796 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 57 2024-09-11T22:41:07.797 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":57,"btime":"2024-09-11T22:40:39:632093+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:40:38.909431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:41:07.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9dd40a2e00 msgr2=0x7f9dd40a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9dd40a2e00 0x7f9dd40a52c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80095f0 tx=0x7f9dd8005f90 comp rx=0 tx=0).stop 2024-09-11T22:41:07.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc0094f0 msgr2=0x7f9ddc1329b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:07.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc0094f0 0x7f9ddc1329b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0035040 tx=0x7f9dd0033d90 comp rx=0 tx=0).stop 2024-09-11T22:41:07.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 shutdown_connections 2024-09-11T22:41:07.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ddc0a6430 0x7f9ddc136d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ddc0094f0 0x7f9ddc1329b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f9dd40a2e00 0x7f9dd40a52c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 --2- 172.21.15.5:0/3909498524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ddc008bd0 0x7f9ddc132470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:07.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 >> 172.21.15.5:0/3909498524 conn(0x7f9ddc093490 msgr2=0x7f9ddc0a79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 shutdown_connections 2024-09-11T22:41:07.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:07.795+0000 7f9de8bf7640 1 -- 172.21.15.5:0/3909498524 wait complete. 2024-09-11T22:41:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:07 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:41:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:07 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:07 smithi159 ceph-mon[35534]: scrub summary: active paths [/] 2024-09-11T22:41:08.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:07 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:08.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:07 smithi159 ceph-mon[35534]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:08.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:07 smithi159 ceph-mon[35534]: pgmap v602: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 303 op/s 2024-09-11T22:41:08.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:07 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:41:08.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:07 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:08.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:07 smithi172 ceph-mon[35461]: scrub summary: active paths [/] 2024-09-11T22:41:08.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:07 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:08.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:07 smithi172 ceph-mon[35461]: scrub summary: idle+waiting paths [/] 2024-09-11T22:41:08.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:07 smithi172 ceph-mon[35461]: pgmap v602: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.1 MiB/s wr, 303 op/s 2024-09-11T22:41:08.541 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24500 quiesce db --cancel --all' 2024-09-11T22:41:08.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24500 quiesce db --cancel --all 2024-09-11T22:41:08.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.707+0000 7f73bdd02640 1 -- 172.21.15.5:0/739752271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b81037a0 msgr2=0x7f73b810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:08.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.707+0000 7f73bdd02640 1 --2- 172.21.15.5:0/739752271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b81037a0 0x7f73b810bc00 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009900 tx=0x7f73a002a410 comp rx=0 tx=0).stop 2024-09-11T22:41:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.707+0000 7f73bdd02640 1 -- 172.21.15.5:0/739752271 shutdown_connections 2024-09-11T22:41:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.707+0000 7f73bdd02640 1 --2- 172.21.15.5:0/739752271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b810c2f0 0x7f73b810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.707+0000 7f73bdd02640 1 --2- 172.21.15.5:0/739752271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b81037a0 0x7f73b810bc00 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.707+0000 7f73bdd02640 1 --2- 172.21.15.5:0/739752271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b8102df0 0x7f73b81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.707+0000 7f73bdd02640 1 -- 172.21.15.5:0/739752271 >> 172.21.15.5:0/739752271 conn(0x7f73b80fc7d0 msgr2=0x7f73b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.707+0000 7f73bdd02640 1 -- 172.21.15.5:0/739752271 shutdown_connections 2024-09-11T22:41:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.707+0000 7f73bdd02640 1 -- 172.21.15.5:0/739752271 wait complete. 2024-09-11T22:41:08.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.708+0000 7f73bdd02640 1 Processor -- start 2024-09-11T22:41:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.708+0000 7f73bdd02640 1 -- start start 2024-09-11T22:41:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.708+0000 7f73bdd02640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b8102df0 0x7f73b8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.708+0000 7f73bdd02640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b81037a0 0x7f73b8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.708+0000 7f73bdd02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810c2f0 0x7f73b819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73bdd02640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81120d0 con 0x7f73b810c2f0 2024-09-11T22:41:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73bdd02640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8111f50 con 0x7f73b81037a0 2024-09-11T22:41:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73bdd02640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8112250 con 0x7f73b8102df0 2024-09-11T22:41:08.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810c2f0 0x7f73b819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810c2f0 0x7f73b819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37082/0 (socket says 172.21.15.5:37082) 2024-09-11T22:41:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b7fff640 1 -- 172.21.15.5:0/2781927469 learned_addr learned my addr 172.21.15.5:0/2781927469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b77fe640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b8102df0 0x7f73b8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73aeffd640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b81037a0 0x7f73b8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b7fff640 1 -- 172.21.15.5:0/2781927469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b8102df0 msgr2=0x7f73b8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b7fff640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b8102df0 0x7f73b8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b7fff640 1 -- 172.21.15.5:0/2781927469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b81037a0 msgr2=0x7f73b8198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:08.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b7fff640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b81037a0 0x7f73b8198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b7fff640 1 -- 172.21.15.5:0/2781927469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b819d5c0 con 0x7f73b810c2f0 2024-09-11T22:41:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b7fff640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810c2f0 0x7f73b819cee0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f73b000eaa0 tx=0x7f73b000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.709+0000 7f73b57fa640 1 -- 172.21.15.5:0/2781927469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b000cdb0 con 0x7f73b810c2f0 2024-09-11T22:41:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.710+0000 7f73bdd02640 1 -- 172.21.15.5:0/2781927469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b819d8b0 con 0x7f73b810c2f0 2024-09-11T22:41:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.710+0000 7f73b57fa640 1 -- 172.21.15.5:0/2781927469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73b0005020 con 0x7f73b810c2f0 2024-09-11T22:41:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.710+0000 7f73b57fa640 1 -- 172.21.15.5:0/2781927469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b00116b0 con 0x7f73b810c2f0 2024-09-11T22:41:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.710+0000 7f73bdd02640 1 -- 172.21.15.5:0/2781927469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b81a5190 con 0x7f73b810c2f0 2024-09-11T22:41:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.712+0000 7f73b57fa640 1 -- 172.21.15.5:0/2781927469 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f73b0005560 con 0x7f73b810c2f0 2024-09-11T22:41:08.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.712+0000 7f73c00db740 1 Processor -- start 2024-09-11T22:41:08.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.712+0000 7f73c00db740 1 -- start start 2024-09-11T22:41:08.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.712+0000 7f73c00db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 0x5564d4765880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.712+0000 7f73c00db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4765dc0 0x5564d4770140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.712+0000 7f73c00db740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4770680 0x5564d4772a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.712+0000 7f73c00db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5564d46cb2e0 con 0x5564d4765dc0 2024-09-11T22:41:08.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.712+0000 7f73c00db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5564d4672ce0 con 0x5564d47654a0 2024-09-11T22:41:08.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.712+0000 7f73c00db740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5564d46c72a0 con 0x5564d4770680 2024-09-11T22:41:08.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4765dc0 0x5564d4770140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4765dc0 0x5564d4770140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37098/0 (socket says 172.21.15.5:37098) 2024-09-11T22:41:08.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73b77fe640 1 -- 172.21.15.5:0/820774640 learned_addr learned my addr 172.21.15.5:0/820774640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:08.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73aeffd640 1 --2- 172.21.15.5:0/820774640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 0x5564d4765880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73aeffd640 1 --2- 172.21.15.5:0/820774640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4770680 0x5564d4772a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73b77fe640 1 -- 172.21.15.5:0/820774640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4770680 msgr2=0x5564d4772a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73b77fe640 1 --2- 172.21.15.5:0/820774640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4770680 0x5564d4772a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73b77fe640 1 -- 172.21.15.5:0/820774640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 msgr2=0x5564d4765880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73b77fe640 1 --2- 172.21.15.5:0/820774640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 0x5564d4765880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73b77fe640 1 -- 172.21.15.5:0/820774640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d4657440 con 0x5564d4765dc0 2024-09-11T22:41:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73b77fe640 1 --2- 172.21.15.5:0/820774640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4765dc0 0x5564d4770140 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009cf0 tx=0x7f73a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1372a0714aa5b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73b57fa640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73a40a2cf0 0x7f73a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.713+0000 7f73adffb640 1 -- 172.21.15.5:0/820774640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8004230 con 0x5564d4765dc0 2024-09-11T22:41:08.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73adffb640 1 -- 172.21.15.5:0/820774640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a80043d0 con 0x5564d4765dc0 2024-09-11T22:41:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73adffb640 1 -- 172.21.15.5:0/820774640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a80198c0 con 0x5564d4765dc0 2024-09-11T22:41:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73b57fa640 1 -- 172.21.15.5:0/2781927469 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f73b00cf650 con 0x7f73b810c2f0 2024-09-11T22:41:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73aeffd640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73a40a2cf0 0x7f73a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73c00db740 1 -- 172.21.15.5:0/820774640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4765dc0 msgr2=0x5564d4770140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73aeffd640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73a40a2cf0 0x7f73a40a51b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005940 tx=0x7f73a0000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73c00db740 1 --2- 172.21.15.5:0/820774640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4765dc0 0x5564d4770140 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009cf0 tx=0x7f73a8011810 comp rx=0 tx=0).stop 2024-09-11T22:41:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73c00db740 1 -- 172.21.15.5:0/820774640 shutdown_connections 2024-09-11T22:41:08.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73c00db740 1 --2- 172.21.15.5:0/820774640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4770680 0x5564d4772a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73c00db740 1 --2- 172.21.15.5:0/820774640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4765dc0 0x5564d4770140 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73c00db740 1 --2- 172.21.15.5:0/820774640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 0x5564d4765880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.714+0000 7f73c00db740 1 -- 172.21.15.5:0/820774640 >> 172.21.15.5:0/820774640 conn(0x5564d4761a30 msgr2=0x5564d4761e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.715+0000 7f73c00db740 1 -- 172.21.15.5:0/820774640 shutdown_connections 2024-09-11T22:41:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.715+0000 7f73c00db740 1 -- 172.21.15.5:0/820774640 wait complete. 2024-09-11T22:41:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.715+0000 7f73c00db740 1 -- start start 2024-09-11T22:41:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.715+0000 7f73c00db740 1 Processor -- start 2024-09-11T22:41:08.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.715+0000 7f73c00db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d476ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.715+0000 7f73c00db740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d476d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.715+0000 7f73c00db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d4770680 0x5564d476d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73c00db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5564d4773250 con 0x5564d47654a0 2024-09-11T22:41:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73c00db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5564d4686050 con 0x5564d4770680 2024-09-11T22:41:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73c00db740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x5564d47733d0 con 0x5564d4765dc0 2024-09-11T22:41:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d476ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73c00db740 10 client.0 _ll_register_callbacks cb 0x5564d46c6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:08.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d476ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37108/0 (socket says 172.21.15.5:37108) 2024-09-11T22:41:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73b77fe640 1 -- 172.21.15.5:0/3641837180 learned_addr learned my addr 172.21.15.5:0/3641837180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73aeffd640 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d476d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73b7fff640 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d4770680 0x5564d476d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73b77fe640 1 -- 172.21.15.5:0/3641837180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 msgr2=0x5564d476d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73b77fe640 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d476d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73b77fe640 1 -- 172.21.15.5:0/3641837180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d4770680 msgr2=0x5564d476d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73b77fe640 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d4770680 0x5564d476d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73b77fe640 1 -- 172.21.15.5:0/3641837180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d443c3c0 con 0x5564d47654a0 2024-09-11T22:41:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73aeffd640 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d476d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f73b77fe640 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d476ff90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f73a8000c00 tx=0x7f73a80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b562af146e20e0ac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.716+0000 7f7396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f7396ffd640 1 -- 172.21.15.5:0/3641837180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a8027070 con 0x5564d47654a0 2024-09-11T22:41:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f7396ffd640 1 -- 172.21.15.5:0/3641837180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f73a8005250 con 0x5564d47654a0 2024-09-11T22:41:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f73c00db740 1 -- 172.21.15.5:0/3641837180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73a80043d0 con 0x5564d47654a0 2024-09-11T22:41:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f7396ffd640 1 -- 172.21.15.5:0/3641837180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f73a8033050 con 0x5564d47654a0 2024-09-11T22:41:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f73c00db740 10 client.21231 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f73c00db740 1 -- 172.21.15.5:0/3641837180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5564d477ebd0 con 0x5564d47654a0 2024-09-11T22:41:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f7396ffd640 1 -- 172.21.15.5:0/3641837180 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73a80043d0 con 0x5564d47654a0 2024-09-11T22:41:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f7396ffd640 1 -- 172.21.15.5:0/3641837180 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f73a80195c0 con 0x5564d47654a0 2024-09-11T22:41:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f73c00db740 10 client.21231 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f73c00db740 10 client.21231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f73c00db740 20 client.21231 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f73c00db740 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5564d477ff80 0x5564d4782370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f73c00db740 4 client.21231 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:08.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.717+0000 7f73c00db740 1 -- 172.21.15.5:0/3641837180 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564d46d8850 con 0x5564d477ff80 2024-09-11T22:41:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.718+0000 7f73aeffd640 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5564d477ff80 0x5564d4782370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.718+0000 7f73aeffd640 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5564d477ff80 0x5564d4782370 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81212e72ce5e5498 server_cookie=92e221bc2ca3984 in_seq=0 out_seq=0 2024-09-11T22:41:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.718+0000 7f7396ffd640 10 client.21231 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f7396ffd640 1 -- 172.21.15.5:0/3641837180 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564d46d8850 con 0x5564d477ff80 2024-09-11T22:41:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f7396ffd640 10 client.21231 handle_command_reply: tid=0 2024-09-11T22:41:08.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f73c00db740 1 client.21231 shutdown 2024-09-11T22:41:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f73c00db740 1 -- 172.21.15.5:0/3641837180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 msgr2=0x5564d476ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f73c00db740 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d476ff90 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f73a8000c00 tx=0x7f73a80038a0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f73c00db740 1 -- 172.21.15.5:0/3641837180 shutdown_connections 2024-09-11T22:41:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f73c00db740 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5564d477ff80 0x5564d4782370 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f73c00db740 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d4770680 0x5564d476d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f73c00db740 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d476d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f73c00db740 1 --2- 172.21.15.5:0/3641837180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d476ff90 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f73c00db740 1 -- 172.21.15.5:0/3641837180 >> 172.21.15.5:0/3641837180 conn(0x5564d4761a30 msgr2=0x5564d4767760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f7396ffd640 0 client.21231 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.719+0000 7f73c00db740 1 -- 172.21.15.5:0/3641837180 shutdown_connections 2024-09-11T22:41:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.720+0000 7f73c00db740 1 -- 172.21.15.5:0/3641837180 wait complete. 2024-09-11T22:41:08.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.720+0000 7f73c00db740 20 client.21231 trim_cache size 0 max 16384 2024-09-11T22:41:08.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73c00db740 1 Processor -- start 2024-09-11T22:41:08.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73c00db740 1 -- start start 2024-09-11T22:41:08.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73c00db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 0x5564d477c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73c00db740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d476cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73c00db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4770680 0x5564d4787bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73c00db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81adbb0 con 0x5564d4770680 2024-09-11T22:41:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73c00db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81af7e0 con 0x5564d47654a0 2024-09-11T22:41:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73c00db740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81ae280 con 0x5564d4765dc0 2024-09-11T22:41:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4770680 0x5564d4787bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4770680 0x5564d4787bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37114/0 (socket says 172.21.15.5:37114) 2024-09-11T22:41:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73b7fff640 1 -- 172.21.15.5:0/3192264429 learned_addr learned my addr 172.21.15.5:0/3192264429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73b77fe640 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 0x5564d477c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73aeffd640 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d476cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.737+0000 7f73b7fff640 1 -- 172.21.15.5:0/3192264429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 msgr2=0x5564d476cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.738+0000 7f73b7fff640 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d476cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.738+0000 7f73b7fff640 1 -- 172.21.15.5:0/3192264429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 msgr2=0x5564d477c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.738+0000 7f73b7fff640 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 0x5564d477c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.738+0000 7f73b7fff640 1 -- 172.21.15.5:0/3192264429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d476dac0 con 0x5564d4770680 2024-09-11T22:41:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.738+0000 7f73b77fe640 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 0x5564d477c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:41:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.738+0000 7f73aeffd640 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d476cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.738+0000 7f73b7fff640 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4770680 0x5564d4787bf0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f73b009b850 tx=0x7f73b009bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee0a38e40a34f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.738+0000 7f73ad7fa640 1 -- 172.21.15.5:0/3192264429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b001f070 con 0x5564d4770680 2024-09-11T22:41:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.738+0000 7f73ad7fa640 1 -- 172.21.15.5:0/3192264429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73b00b4070 con 0x5564d4770680 2024-09-11T22:41:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.738+0000 7f73ad7fa640 1 -- 172.21.15.5:0/3192264429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b00a44e0 con 0x5564d4770680 2024-09-11T22:41:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.739+0000 7f73c00db740 1 -- 172.21.15.5:0/3192264429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4770680 msgr2=0x5564d4787bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.739+0000 7f73c00db740 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4770680 0x5564d4787bf0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f73b009b850 tx=0x7f73b009bc40 comp rx=0 tx=0).stop 2024-09-11T22:41:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.739+0000 7f73c00db740 1 -- 172.21.15.5:0/3192264429 shutdown_connections 2024-09-11T22:41:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.739+0000 7f73c00db740 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d4770680 0x5564d4787bf0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.739+0000 7f73c00db740 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d476cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.739+0000 7f73c00db740 1 --2- 172.21.15.5:0/3192264429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d47654a0 0x5564d477c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.739+0000 7f73c00db740 1 -- 172.21.15.5:0/3192264429 >> 172.21.15.5:0/3192264429 conn(0x5564d4761a30 msgr2=0x5564d4767710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.739+0000 7f73c00db740 1 -- 172.21.15.5:0/3192264429 shutdown_connections 2024-09-11T22:41:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.739+0000 7f73c00db740 1 -- 172.21.15.5:0/3192264429 wait complete. 2024-09-11T22:41:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.739+0000 7f73c00db740 1 -- start start 2024-09-11T22:41:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73c00db740 1 Processor -- start 2024-09-11T22:41:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73c00db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d4778950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73c00db740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d4778e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73c00db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d4770680 0x5564d4773d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73c00db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81af110 con 0x5564d47654a0 2024-09-11T22:41:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73c00db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81ae980 con 0x5564d4770680 2024-09-11T22:41:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73c00db740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f73b81abbb0 con 0x5564d4765dc0 2024-09-11T22:41:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73c00db740 10 client.0 _ll_register_callbacks cb 0x7f73a8009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d4778950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d4778950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37116/0 (socket says 172.21.15.5:37116) 2024-09-11T22:41:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.740+0000 7f73b77fe640 1 -- 172.21.15.5:0/299515578 learned_addr learned my addr 172.21.15.5:0/299515578 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73aeffd640 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d4778e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73b7fff640 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d4770680 0x5564d4773d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73b77fe640 1 -- 172.21.15.5:0/299515578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 msgr2=0x5564d4778e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73b77fe640 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d4778e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73b77fe640 1 -- 172.21.15.5:0/299515578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d4770680 msgr2=0x5564d4773d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73b77fe640 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d4770680 0x5564d4773d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73b77fe640 1 -- 172.21.15.5:0/299515578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d4774560 con 0x5564d47654a0 2024-09-11T22:41:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73b77fe640 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d4778950 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009250 tx=0x7f73a8019e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171bce01e877be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f7396ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f7396ffd640 1 -- 172.21.15.5:0/299515578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a803b070 con 0x5564d47654a0 2024-09-11T22:41:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f7396ffd640 1 -- 172.21.15.5:0/299515578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8629+0+0 (secure 0 0 0) 0x7f73a8005250 con 0x5564d47654a0 2024-09-11T22:41:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73c00db740 1 -- 172.21.15.5:0/299515578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73b00b4070 con 0x5564d47654a0 2024-09-11T22:41:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f7396ffd640 1 -- 172.21.15.5:0/299515578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f73a8047050 con 0x5564d47654a0 2024-09-11T22:41:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73c00db740 10 client.21237 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.741+0000 7f73c00db740 1 -- 172.21.15.5:0/299515578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5564d47eeb30 con 0x5564d47654a0 2024-09-11T22:41:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.742+0000 7f7396ffd640 1 -- 172.21.15.5:0/299515578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73b00b4070 con 0x5564d47654a0 2024-09-11T22:41:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.742+0000 7f7396ffd640 1 -- 172.21.15.5:0/299515578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f73a8023460 con 0x5564d47654a0 2024-09-11T22:41:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.742+0000 7f73c00db740 10 client.21237 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.742+0000 7f73c00db740 10 client.21237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.742+0000 7f73c00db740 20 client.21237 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.742+0000 7f73c00db740 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5564d47ef2b0 0x5564d47f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.742+0000 7f73c00db740 4 client.21237 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:41:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.742+0000 7f73c00db740 1 -- 172.21.15.5:0/299515578 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564d46d2d60 con 0x5564d47ef2b0 2024-09-11T22:41:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.742+0000 7f73aeffd640 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5564d47ef2b0 0x5564d47f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f7396ffd640 10 client.21237 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73aeffd640 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5564d47ef2b0 0x5564d47f16d0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e86f49ca22fcfd2 server_cookie=aff97fa60a7399ae in_seq=0 out_seq=0 2024-09-11T22:41:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f7396ffd640 1 -- 172.21.15.5:0/299515578 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5564d46d2d60 con 0x5564d47ef2b0 2024-09-11T22:41:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f7396ffd640 10 client.21237 handle_command_reply: tid=0 2024-09-11T22:41:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 client.21237 shutdown 2024-09-11T22:41:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 -- 172.21.15.5:0/299515578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 msgr2=0x5564d4778950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d4778950 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f73a8009250 tx=0x7f73a8019e30 comp rx=0 tx=0).stop 2024-09-11T22:41:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 -- 172.21.15.5:0/299515578 shutdown_connections 2024-09-11T22:41:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x5564d47ef2b0 0x5564d47f16d0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564d4770680 0x5564d4773d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5564d4765dc0 0x5564d4778e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 --2- 172.21.15.5:0/299515578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564d47654a0 0x5564d4778950 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 -- 172.21.15.5:0/299515578 >> 172.21.15.5:0/299515578 conn(0x5564d4761a30 msgr2=0x5564d476eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f7396ffd640 0 client.21237 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 -- 172.21.15.5:0/299515578 shutdown_connections 2024-09-11T22:41:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.743+0000 7f73c00db740 1 -- 172.21.15.5:0/299515578 wait complete. 2024-09-11T22:41:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73c00db740 20 client.21237 trim_cache size 0 max 16384 2024-09-11T22:41:08.754 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:41:08.754 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-11T22:41:08.754 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-11T22:41:08.754 INFO:teuthology.orchestra.run.smithi005.stdout: "86475d5a-5ec2-418f-b266-7d3c2b341fe4": { 2024-09-11T22:41:08.754 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-11T22:41:08.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "86475d5a-5ec2-418f-b266-7d3c2b341fe4", 2024-09-11T22:41:08.755 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-11T22:41:08.755 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:08.755 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:08.755 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:41:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 -- 172.21.15.5:0/2781927469 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73a40a2cf0 msgr2=0x7f73a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73a40a2cf0 0x7f73a40a51b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f73a0005940 tx=0x7f73a0000bc0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 -- 172.21.15.5:0/2781927469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810c2f0 msgr2=0x7f73b819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810c2f0 0x7f73b819cee0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f73b000eaa0 tx=0x7f73b000ef60 comp rx=0 tx=0).stop 2024-09-11T22:41:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 -- 172.21.15.5:0/2781927469 shutdown_connections 2024-09-11T22:41:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73b810c2f0 0x7f73b819cee0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f73a40a2cf0 0x7f73a40a51b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73b81037a0 0x7f73b8198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 --2- 172.21.15.5:0/2781927469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73b8102df0 0x7f73b8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 -- 172.21.15.5:0/2781927469 >> 172.21.15.5:0/2781927469 conn(0x7f73b80fc7d0 msgr2=0x7f73b80fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 -- 172.21.15.5:0/2781927469 shutdown_connections 2024-09-11T22:41:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:08.744+0000 7f73bdd02640 1 -- 172.21.15.5:0/2781927469 wait complete. 2024-09-11T22:41:08.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:08 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3909498524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:41:09.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:08 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3909498524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:41:09.091 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:09.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:08 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3909498524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:41:09.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:09 smithi005 ceph-mon[30943]: pgmap v603: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.8 MiB/s wr, 458 op/s 2024-09-11T22:41:10.045 INFO:tasks.cephfs.filesystem:scrub status for tag:86475d5a-5ec2-418f-b266-7d3c2b341fe4 - {'path': '/', 'tag': '86475d5a-5ec2-418f-b266-7d3c2b341fe4', 'options': 'recursive,force'} 2024-09-11T22:41:10.084 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:09 smithi159 ceph-mon[35534]: pgmap v603: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.8 MiB/s wr, 458 op/s 2024-09-11T22:41:10.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:09 smithi172 ceph-mon[35461]: pgmap v603: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 6.8 MiB/s wr, 458 op/s 2024-09-11T22:41:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.306+0000 7f5fde5c9640 1 -- 172.21.15.5:0/2827194043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd810e890 msgr2=0x7f5fd810ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.306+0000 7f5fde5c9640 1 --2- 172.21.15.5:0/2827194043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd810e890 0x7f5fd810ed40 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009900 tx=0x7f5fcc02a410 comp rx=0 tx=0).stop 2024-09-11T22:41:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.306+0000 7f5fde5c9640 1 -- 172.21.15.5:0/2827194043 shutdown_connections 2024-09-11T22:41:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.306+0000 7f5fde5c9640 1 --2- 172.21.15.5:0/2827194043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd810e890 0x7f5fd810ed40 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.306+0000 7f5fde5c9640 1 --2- 172.21.15.5:0/2827194043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd8109b10 0x7f5fd810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.306+0000 7f5fde5c9640 1 --2- 172.21.15.5:0/2827194043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd81091f0 0x7f5fd81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.306+0000 7f5fde5c9640 1 -- 172.21.15.5:0/2827194043 >> 172.21.15.5:0/2827194043 conn(0x7f5fd806e2d0 msgr2=0x7f5fd806e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:10.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.306+0000 7f5fde5c9640 1 -- 172.21.15.5:0/2827194043 shutdown_connections 2024-09-11T22:41:10.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.307+0000 7f5fde5c9640 1 -- 172.21.15.5:0/2827194043 wait complete. 2024-09-11T22:41:10.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.307+0000 7f5fde5c9640 1 Processor -- start 2024-09-11T22:41:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.307+0000 7f5fde5c9640 1 -- start start 2024-09-11T22:41:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.308+0000 7f5fde5c9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd81091f0 0x7f5fd8115640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.308+0000 7f5fde5c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd8109b10 0x7f5fd8115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.308+0000 7f5fde5c9640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd810e890 0x7f5fd81160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.308+0000 7f5fde5c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd8112540 con 0x7f5fd8109b10 2024-09-11T22:41:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.308+0000 7f5fde5c9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd81123c0 con 0x7f5fd81091f0 2024-09-11T22:41:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.308+0000 7f5fde5c9640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd8118810 con 0x7f5fd810e890 2024-09-11T22:41:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.309+0000 7f5fd7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd81091f0 0x7f5fd8115640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.309+0000 7f5fd7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd81091f0 0x7f5fd8115640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43592/0 (socket says 172.21.15.5:43592) 2024-09-11T22:41:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.309+0000 7f5fd7fff640 1 -- 172.21.15.5:0/933755520 learned_addr learned my addr 172.21.15.5:0/933755520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.309+0000 7f5fd7fff640 1 -- 172.21.15.5:0/933755520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd810e890 msgr2=0x7f5fd81160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.309+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd810e890 0x7f5fd81160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.309+0000 7f5fd7fff640 1 -- 172.21.15.5:0/933755520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd8109b10 msgr2=0x7f5fd8115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.309+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd8109b10 0x7f5fd8115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.309+0000 7f5fd7fff640 1 -- 172.21.15.5:0/933755520 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd81b33d0 con 0x7f5fd81091f0 2024-09-11T22:41:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.310+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd81091f0 0x7f5fd8115640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800c950 tx=0x7f5fc800ce10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.310+0000 7f5fd57fa640 1 -- 172.21.15.5:0/933755520 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8007c70 con 0x7f5fd81091f0 2024-09-11T22:41:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.311+0000 7f5fde5c9640 1 -- 172.21.15.5:0/933755520 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd81b36c0 con 0x7f5fd81091f0 2024-09-11T22:41:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.311+0000 7f5fde5c9640 1 -- 172.21.15.5:0/933755520 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd81b3c50 con 0x7f5fd81091f0 2024-09-11T22:41:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.311+0000 7f5fd57fa640 1 -- 172.21.15.5:0/933755520 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc8007e10 con 0x7f5fd81091f0 2024-09-11T22:41:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.312+0000 7f5fd57fa640 1 -- 172.21.15.5:0/933755520 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8018960 con 0x7f5fd81091f0 2024-09-11T22:41:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.314+0000 7f5fe09a2740 1 Processor -- start 2024-09-11T22:41:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.314+0000 7f5fe09a2740 1 -- start start 2024-09-11T22:41:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.314+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 0x55da589a6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.314+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a7320 0x55da589b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.314+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1be0 0x55da589b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.315+0000 7f5fe09a2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55da5890b200 con 0x55da589b1be0 2024-09-11T22:41:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.315+0000 7f5fe09a2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55da588b1dc0 con 0x55da589a6a00 2024-09-11T22:41:10.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.315+0000 7f5fe09a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55da589071c0 con 0x55da589a7320 2024-09-11T22:41:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.315+0000 7f5fd77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1be0 0x55da589b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.315+0000 7f5fd77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1be0 0x55da589b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37134/0 (socket says 172.21.15.5:37134) 2024-09-11T22:41:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.315+0000 7f5fd77fe640 1 -- 172.21.15.5:0/199247347 learned_addr learned my addr 172.21.15.5:0/199247347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.315+0000 7f5fdcb66640 1 --2- 172.21.15.5:0/199247347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 0x55da589a6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.316+0000 7f5fd77fe640 1 -- 172.21.15.5:0/199247347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a7320 msgr2=0x55da589b16a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-11T22:41:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.316+0000 7f5fd77fe640 1 --2- 172.21.15.5:0/199247347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a7320 0x55da589b16a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.316+0000 7f5fd77fe640 1 -- 172.21.15.5:0/199247347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 msgr2=0x55da589a6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.316+0000 7f5fd77fe640 1 --2- 172.21.15.5:0/199247347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 0x55da589a6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.316+0000 7f5fd77fe640 1 -- 172.21.15.5:0/199247347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da5867c620 con 0x55da589b1be0 2024-09-11T22:41:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.317+0000 7f5fd57fa640 1 -- 172.21.15.5:0/933755520 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f5fc800f450 con 0x7f5fd81091f0 2024-09-11T22:41:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.318+0000 7f5fd77fe640 1 --2- 172.21.15.5:0/199247347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1be0 0x55da589b3fd0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009d20 tx=0x7f5fd000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e6fc3a71d990cd server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.318+0000 7f5fd57fa640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fc00a2dc0 0x7f5fc00a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.318+0000 7f5fd57fa640 1 -- 172.21.15.5:0/933755520 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8673+0+0 (secure 0 0 0) 0x7f5fc80cf480 con 0x7f5fd81091f0 2024-09-11T22:41:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.319+0000 7f5fbe7fc640 1 -- 172.21.15.5:0/199247347 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0016070 con 0x55da589b1be0 2024-09-11T22:41:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.319+0000 7f5fbe7fc640 1 -- 172.21.15.5:0/199247347 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd0005020 con 0x55da589b1be0 2024-09-11T22:41:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.320+0000 7f5fdcb66640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fc00a2dc0 0x7f5fc00a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.320+0000 7f5fe09a2740 1 -- 172.21.15.5:0/199247347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1be0 msgr2=0x55da589b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.320+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/199247347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1be0 0x55da589b3fd0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009d20 tx=0x7f5fd000e6c0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.320+0000 7f5fdcb66640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fc00a2dc0 0x7f5fc00a5280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc005c20 tx=0x7f5fcc005d50 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.320+0000 7f5fe09a2740 1 -- 172.21.15.5:0/199247347 shutdown_connections 2024-09-11T22:41:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.320+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/199247347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1be0 0x55da589b3fd0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.320+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/199247347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a7320 0x55da589b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.320+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/199247347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 0x55da589a6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.320+0000 7f5fe09a2740 1 -- 172.21.15.5:0/199247347 >> 172.21.15.5:0/199247347 conn(0x55da589a2f90 msgr2=0x55da589a3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.322+0000 7f5fe09a2740 1 -- 172.21.15.5:0/199247347 shutdown_connections 2024-09-11T22:41:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.322+0000 7f5fe09a2740 1 -- 172.21.15.5:0/199247347 wait complete. 2024-09-11T22:41:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.322+0000 7f5fe09a2740 1 -- start start 2024-09-11T22:41:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.323+0000 7f5fe09a2740 1 Processor -- start 2024-09-11T22:41:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.323+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a6a00 0x55da589b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.323+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 0x55da589b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.323+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 0x55da589b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.323+0000 7f5fe09a2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55da589b4800 con 0x55da589b1430 2024-09-11T22:41:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.323+0000 7f5fe09a2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55da588a5c50 con 0x55da589a7320 2024-09-11T22:41:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.323+0000 7f5fe09a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55da589b4980 con 0x55da589a6a00 2024-09-11T22:41:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.323+0000 7f5fe09a2740 10 client.0 _ll_register_callbacks cb 0x55da588b7ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.324+0000 7f5fdcb66640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 0x55da589b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.324+0000 7f5fdcb66640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 0x55da589b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37146/0 (socket says 172.21.15.5:37146) 2024-09-11T22:41:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.324+0000 7f5fdcb66640 1 -- 172.21.15.5:0/7669530 learned_addr learned my addr 172.21.15.5:0/7669530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.324+0000 7f5fd77fe640 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 0x55da589b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.324+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a6a00 0x55da589b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.324+0000 7f5fdcb66640 1 -- 172.21.15.5:0/7669530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a6a00 msgr2=0x55da589b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.324+0000 7f5fdcb66640 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a6a00 0x55da589b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.324+0000 7f5fdcb66640 1 -- 172.21.15.5:0/7669530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 msgr2=0x55da589b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.324+0000 7f5fdcb66640 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 0x55da589b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.324+0000 7f5fdcb66640 1 -- 172.21.15.5:0/7669530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da589bdd50 con 0x55da589b1430 2024-09-11T22:41:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.325+0000 7f5fdcb66640 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 0x55da589b18b0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009900 tx=0x7f5fcc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474a3c06182dbb75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.326+0000 7f5fb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.326+0000 7f5fb77fe640 1 -- 172.21.15.5:0/7669530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc04d020 con 0x55da589b1430 2024-09-11T22:41:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.326+0000 7f5fe09a2740 1 -- 172.21.15.5:0/7669530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55da58918770 con 0x55da589b1430 2024-09-11T22:41:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.326+0000 7f5fb77fe640 1 -- 172.21.15.5:0/7669530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8673+0+0 (secure 0 0 0) 0x7f5fcc0046c0 con 0x55da589b1430 2024-09-11T22:41:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.326+0000 7f5fb77fe640 1 -- 172.21.15.5:0/7669530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f5fcc058050 con 0x55da589b1430 2024-09-11T22:41:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.326+0000 7f5fe09a2740 10 client.21246 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.326+0000 7f5fe09a2740 1 -- 172.21.15.5:0/7669530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55da589c06b0 con 0x55da589b1430 2024-09-11T22:41:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.327+0000 7f5fb77fe640 1 -- 172.21.15.5:0/7669530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55da58918770 con 0x55da589b1430 2024-09-11T22:41:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.327+0000 7f5fb77fe640 1 -- 172.21.15.5:0/7669530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f5fcc032bc0 con 0x55da589b1430 2024-09-11T22:41:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.327+0000 7f5fe09a2740 10 client.21246 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.327+0000 7f5fe09a2740 10 client.21246 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:41:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.327+0000 7f5fe09a2740 20 client.21246 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.327+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55da589c1be0 0x55da589c4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.327+0000 7f5fe09a2740 4 client.21246 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.327+0000 7f5fe09a2740 1 -- 172.21.15.5:0/7669530 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da58912c80 con 0x55da589c1be0 2024-09-11T22:41:10.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.330+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55da589c1be0 0x55da589c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.331+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55da589c1be0 0x55da589c4060 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9b0b359b915263 server_cookie=93c1132c02a171d2 in_seq=0 out_seq=0 2024-09-11T22:41:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.331+0000 7f5fb77fe640 10 client.21246 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.332+0000 7f5fb77fe640 1 -- 172.21.15.5:0/7669530 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da58912c80 con 0x55da589c1be0 2024-09-11T22:41:10.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.332+0000 7f5fb77fe640 10 client.21246 handle_command_reply: tid=0 2024-09-11T22:41:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.333+0000 7f5fe09a2740 1 client.21246 shutdown 2024-09-11T22:41:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.333+0000 7f5fe09a2740 1 -- 172.21.15.5:0/7669530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 msgr2=0x55da589b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.333+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 0x55da589b18b0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009900 tx=0x7f5fcc005fd0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.336+0000 7f5fe09a2740 1 -- 172.21.15.5:0/7669530 shutdown_connections 2024-09-11T22:41:10.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.336+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55da589c1be0 0x55da589c4060 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.336+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 0x55da589b18b0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.336+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 0x55da589b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.336+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/7669530 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a6a00 0x55da589b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.336+0000 7f5fe09a2740 1 -- 172.21.15.5:0/7669530 >> 172.21.15.5:0/7669530 conn(0x55da589a2f90 msgr2=0x55da589a21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.336+0000 7f5fb77fe640 0 client.21246 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.336+0000 7f5fe09a2740 1 -- 172.21.15.5:0/7669530 shutdown_connections 2024-09-11T22:41:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.336+0000 7f5fe09a2740 1 -- 172.21.15.5:0/7669530 wait complete. 2024-09-11T22:41:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.337+0000 7f5fe09a2740 20 client.21246 trim_cache size 0 max 16384 2024-09-11T22:41:10.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.368+0000 7f5fe09a2740 1 Processor -- start 2024-09-11T22:41:10.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.368+0000 7f5fe09a2740 1 -- start start 2024-09-11T22:41:10.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.369+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 0x55da589afe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.369+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a7320 0x55da589b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.369+0000 7f5fd7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 0x55da589afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.369+0000 7f5fd7fff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 0x55da589afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43620/0 (socket says 172.21.15.5:43620) 2024-09-11T22:41:10.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fd7fff640 1 -- 172.21.15.5:0/280635860 learned_addr learned my addr 172.21.15.5:0/280635860 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:10.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fd77fe640 1 --2- 172.21.15.5:0/280635860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a7320 0x55da589b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/280635860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 0x55da589c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fe09a2740 1 -- 172.21.15.5:0/280635860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc400eb80 con 0x55da589b1430 2024-09-11T22:41:10.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fe09a2740 1 -- 172.21.15.5:0/280635860 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc40107b0 con 0x55da589a6a00 2024-09-11T22:41:10.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fe09a2740 1 -- 172.21.15.5:0/280635860 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc400f250 con 0x55da589a7320 2024-09-11T22:41:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fd7fff640 1 -- 172.21.15.5:0/280635860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a7320 msgr2=0x55da589b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:10.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/280635860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a7320 0x55da589b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fd7fff640 1 -- 172.21.15.5:0/280635860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 msgr2=0x55da589c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/280635860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 0x55da589c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.370+0000 7f5fd7fff640 1 -- 172.21.15.5:0/280635860 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da589a7c50 con 0x55da589a6a00 2024-09-11T22:41:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.371+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/280635860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 0x55da589afe70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8098270 tx=0x7f5fc8018470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c421b0e5ef9acf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.371+0000 7f5fbdffb640 1 -- 172.21.15.5:0/280635860 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8099940 con 0x55da589a6a00 2024-09-11T22:41:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.371+0000 7f5fbdffb640 1 -- 172.21.15.5:0/280635860 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc8099ba0 con 0x55da589a6a00 2024-09-11T22:41:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.371+0000 7f5fbdffb640 1 -- 172.21.15.5:0/280635860 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc809cbd0 con 0x55da589a6a00 2024-09-11T22:41:10.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.372+0000 7f5fe09a2740 1 -- 172.21.15.5:0/280635860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 msgr2=0x55da589afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.372+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/280635860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 0x55da589afe70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8098270 tx=0x7f5fc8018470 comp rx=0 tx=0).stop 2024-09-11T22:41:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.372+0000 7f5fe09a2740 1 -- 172.21.15.5:0/280635860 shutdown_connections 2024-09-11T22:41:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.373+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/280635860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589b1430 0x55da589c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.373+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/280635860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589a7320 0x55da589b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.373+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/280635860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a6a00 0x55da589afe70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.373+0000 7f5fe09a2740 1 -- 172.21.15.5:0/280635860 >> 172.21.15.5:0/280635860 conn(0x55da589a2f90 msgr2=0x55da589a21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:10.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.373+0000 7f5fe09a2740 1 -- 172.21.15.5:0/280635860 shutdown_connections 2024-09-11T22:41:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.373+0000 7f5fe09a2740 1 -- 172.21.15.5:0/280635860 wait complete. 2024-09-11T22:41:10.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.374+0000 7f5fe09a2740 1 -- start start 2024-09-11T22:41:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.374+0000 7f5fe09a2740 1 Processor -- start 2024-09-11T22:41:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.375+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589a6a00 0x55da589ba2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.375+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 0x55da589b7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.375+0000 7f5fe09a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589b1430 0x55da589b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.375+0000 7f5fd77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 0x55da589b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.375+0000 7f5fd77fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 0x55da589b7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43634/0 (socket says 172.21.15.5:43634) 2024-09-11T22:41:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.375+0000 7f5fd77fe640 1 -- 172.21.15.5:0/3991585873 learned_addr learned my addr 172.21.15.5:0/3991585873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.376+0000 7f5fdcb66640 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589b1430 0x55da589b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.376+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc40100e0 con 0x55da589a6a00 2024-09-11T22:41:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.376+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc400f950 con 0x55da589a7320 2024-09-11T22:41:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.376+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc400cb80 con 0x55da589b1430 2024-09-11T22:41:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.376+0000 7f5fe09a2740 10 client.0 _ll_register_callbacks cb 0x7f5fcc004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.376+0000 7f5fd77fe640 1 -- 172.21.15.5:0/3991585873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589b1430 msgr2=0x55da589b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:10.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.376+0000 7f5fd77fe640 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589b1430 0x55da589b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.376+0000 7f5fd77fe640 1 -- 172.21.15.5:0/3991585873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589a6a00 msgr2=0x55da589ba2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.376+0000 7f5fd77fe640 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589a6a00 0x55da589ba2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.376+0000 7f5fd77fe640 1 -- 172.21.15.5:0/3991585873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da589b8250 con 0x55da589a7320 2024-09-11T22:41:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.377+0000 7f5fd77fe640 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 0x55da589b7620 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009d20 tx=0x7f5fd0009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afc77f3b65c8fb32 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.377+0000 7f5fb77fe640 10 client.30275 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-11T22:41:10.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.377+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc8099ba0 con 0x55da589a7320 2024-09-11T22:41:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.379+0000 7f5fb77fe640 1 -- 172.21.15.5:0/3991585873 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0016040 con 0x55da589a7320 2024-09-11T22:41:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.379+0000 7f5fb77fe640 1 -- 172.21.15.5:0/3991585873 <== mon.2 v2:172.21.15.159:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8673+0+0 (secure 0 0 0) 0x7f5fd0004d20 con 0x55da589a7320 2024-09-11T22:41:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.379+0000 7f5fb77fe640 1 -- 172.21.15.5:0/3991585873 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f5fd000ee90 con 0x55da589a7320 2024-09-11T22:41:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.379+0000 7f5fe09a2740 10 client.30275 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:10.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.379+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=57}) -- 0x55da58a2c770 con 0x55da589a7320 2024-09-11T22:41:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.382+0000 7f5fb77fe640 1 -- 172.21.15.5:0/3991585873 <== mon.2 v2:172.21.15.159:3300/0 4 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f5fd000f040 con 0x55da589a7320 2024-09-11T22:41:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.383+0000 7f5fe09a2740 10 client.30275 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.383+0000 7f5fe09a2740 10 client.30275 resolve_mds: validated gid 24500 aka daemon mds.b 2024-09-11T22:41:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.383+0000 7f5fe09a2740 20 client.30275 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.383+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55da58a2cc00 0x55da58a2f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.383+0000 7f5fe09a2740 4 client.30275 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-11T22:41:10.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.383+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x7f5fc8099ba0 con 0x55da58a2cc00 2024-09-11T22:41:10.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.384+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55da58a2cc00 0x55da58a2f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:10.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.390+0000 7f5fd7fff640 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55da58a2cc00 0x55da58a2f020 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c554182912af51c0 server_cookie=ee2681300b40ae4b in_seq=0 out_seq=0 2024-09-11T22:41:10.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.390+0000 7f5fb77fe640 10 client.30275 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:10.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.391+0000 7f5fb77fe640 1 -- 172.21.15.5:0/3991585873 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x7f5fc8099ba0 con 0x55da58a2cc00 2024-09-11T22:41:10.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.391+0000 7f5fb77fe640 10 client.30275 handle_command_reply: tid=0 2024-09-11T22:41:10.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.391+0000 7f5fe09a2740 1 client.30275 shutdown 2024-09-11T22:41:10.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.392+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 msgr2=0x55da589b7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:10.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.392+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 0x55da589b7620 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009d20 tx=0x7f5fd0009a90 comp rx=0 tx=0).stop 2024-09-11T22:41:10.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.392+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 shutdown_connections 2024-09-11T22:41:10.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.392+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x55da58a2cc00 0x55da58a2f020 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.392+0000 7f5fb77fe640 0 client.30275 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:10.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.392+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55da589b1430 0x55da589b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.392+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55da589a7320 0x55da589b7620 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.392+0000 7f5fe09a2740 1 --2- 172.21.15.5:0/3991585873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da589a6a00 0x55da589ba2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.392+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 >> 172.21.15.5:0/3991585873 conn(0x55da589a2f90 msgr2=0x55da589b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:10.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.393+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 shutdown_connections 2024-09-11T22:41:10.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.393+0000 7f5fe09a2740 1 -- 172.21.15.5:0/3991585873 wait complete. 2024-09-11T22:41:10.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.393+0000 7f5fe09a2740 20 client.30275 trim_cache size 0 max 16384 2024-09-11T22:41:10.397 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:41:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 57, 2024-09-11T22:41:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24500, 2024-09-11T22:41:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 64, 2024-09-11T22:41:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": {} 2024-09-11T22:41:10.397 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:41:10.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.394+0000 7f5fde5c9640 1 -- 172.21.15.5:0/933755520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fc00a2dc0 msgr2=0x7f5fc00a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:10.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.394+0000 7f5fde5c9640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fc00a2dc0 0x7f5fc00a5280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc005c20 tx=0x7f5fcc005d50 comp rx=0 tx=0).stop 2024-09-11T22:41:10.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.394+0000 7f5fde5c9640 1 -- 172.21.15.5:0/933755520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd81091f0 msgr2=0x7f5fd8115640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:10.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.394+0000 7f5fde5c9640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd81091f0 0x7f5fd8115640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800c950 tx=0x7f5fc800ce10 comp rx=0 tx=0).stop 2024-09-11T22:41:10.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.394+0000 7f5fde5c9640 1 -- 172.21.15.5:0/933755520 shutdown_connections 2024-09-11T22:41:10.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.394+0000 7f5fde5c9640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd810e890 0x7f5fd81160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.394+0000 7f5fde5c9640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd8109b10 0x7f5fd8115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.394+0000 7f5fde5c9640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f5fc00a2dc0 0x7f5fc00a5280 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.394+0000 7f5fde5c9640 1 --2- 172.21.15.5:0/933755520 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5fd81091f0 0x7f5fd8115640 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.394+0000 7f5fde5c9640 1 -- 172.21.15.5:0/933755520 >> 172.21.15.5:0/933755520 conn(0x7f5fd806e2d0 msgr2=0x7f5fd811ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.395+0000 7f5fde5c9640 1 -- 172.21.15.5:0/933755520 shutdown_connections 2024-09-11T22:41:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:10.395+0000 7f5fde5c9640 1 -- 172.21.15.5:0/933755520 wait complete. 2024-09-11T22:41:11.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:10 smithi005 ceph-mon[30943]: osdmap e118: 12 total, 12 up, 12 in 2024-09-11T22:41:11.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:10 smithi159 ceph-mon[35534]: osdmap e118: 12 total, 12 up, 12 in 2024-09-11T22:41:11.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:10 smithi172 ceph-mon[35461]: osdmap e118: 12 total, 12 up, 12 in 2024-09-11T22:41:11.151 INFO:tasks.quiescer:done joining Quiescers 2024-09-11T22:41:11.151 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-11T22:41:11.162 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T22:41:11.174 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-11T22:41:11.200 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-11T22:41:11.200 INFO:teuthology.task.exec:Executing custom commands... 2024-09-11T22:41:11.200 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T22:41:11.200 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-11T22:41:11.279 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-11T22:41:11.335 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-11T22:41:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.520+0000 7f459912c640 1 -- 172.21.15.5:0/541894921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f459410ea10 msgr2=0x7f4594115430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.520+0000 7f459912c640 1 --2- 172.21.15.5:0/541894921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f459410ea10 0x7f4594115430 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f458400af00 tx=0x7f4584032790 comp rx=0 tx=0).stop 2024-09-11T22:41:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.524+0000 7f459912c640 1 -- 172.21.15.5:0/541894921 shutdown_connections 2024-09-11T22:41:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.524+0000 7f459912c640 1 --2- 172.21.15.5:0/541894921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f459410ea10 0x7f4594115430 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.524+0000 7f459912c640 1 --2- 172.21.15.5:0/541894921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4594109e10 0x7f459410e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.524+0000 7f459912c640 1 --2- 172.21.15.5:0/541894921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4594109460 0x7f4594109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.524+0000 7f459912c640 1 -- 172.21.15.5:0/541894921 >> 172.21.15.5:0/541894921 conn(0x7f459406d210 msgr2=0x7f459406d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.526+0000 7f459912c640 1 -- 172.21.15.5:0/541894921 shutdown_connections 2024-09-11T22:41:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.526+0000 7f459912c640 1 -- 172.21.15.5:0/541894921 wait complete. 2024-09-11T22:41:11.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.527+0000 7f459912c640 1 Processor -- start 2024-09-11T22:41:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.529+0000 7f459912c640 1 -- start start 2024-09-11T22:41:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.529+0000 7f459912c640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4594109460 0x7f45941b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.529+0000 7f459912c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f459410ea10 0x7f45941b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.529+0000 7f459912c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45941af6d0 0x7f45941afb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:11.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.529+0000 7f459912c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45941184e0 con 0x7f45941af6d0 2024-09-11T22:41:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.529+0000 7f459912c640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4594118360 con 0x7f459410ea10 2024-09-11T22:41:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.529+0000 7f459912c640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4594118660 con 0x7f4594109460 2024-09-11T22:41:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.529+0000 7f459259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f459410ea10 0x7f45941b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.529+0000 7f459259c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f459410ea10 0x7f45941b5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40672/0 (socket says 172.21.15.5:40672) 2024-09-11T22:41:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.530+0000 7f459259c640 1 -- 172.21.15.5:0/1288555629 learned_addr learned my addr 172.21.15.5:0/1288555629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:11.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.530+0000 7f459359e640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45941af6d0 0x7f45941afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.530+0000 7f459259c640 1 -- 172.21.15.5:0/1288555629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4594109460 msgr2=0x7f45941b5610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T22:41:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.530+0000 7f459259c640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4594109460 0x7f45941b5610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.530+0000 7f459259c640 1 -- 172.21.15.5:0/1288555629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45941af6d0 msgr2=0x7f45941afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.530+0000 7f459259c640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45941af6d0 0x7f45941afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.530+0000 7f459259c640 1 -- 172.21.15.5:0/1288555629 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45941b02d0 con 0x7f459410ea10 2024-09-11T22:41:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.531+0000 7f459259c640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f459410ea10 0x7f45941b5b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f458800b450 tx=0x7f458800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.534+0000 7f457bfff640 1 -- 172.21.15.5:0/1288555629 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588015020 con 0x7f459410ea10 2024-09-11T22:41:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.534+0000 7f459912c640 1 -- 172.21.15.5:0/1288555629 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45941bc3b0 con 0x7f459410ea10 2024-09-11T22:41:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.534+0000 7f459912c640 1 -- 172.21.15.5:0/1288555629 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45941bc7b0 con 0x7f459410ea10 2024-09-11T22:41:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.535+0000 7f457bfff640 1 -- 172.21.15.5:0/1288555629 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4588011d70 con 0x7f459410ea10 2024-09-11T22:41:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.536+0000 7f457bfff640 1 -- 172.21.15.5:0/1288555629 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458800f340 con 0x7f459410ea10 2024-09-11T22:41:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.539+0000 7f459912c640 1 -- 172.21.15.5:0/1288555629 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45800052c0 con 0x7f459410ea10 2024-09-11T22:41:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.540+0000 7f457bfff640 1 -- 172.21.15.5:0/1288555629 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4588018020 con 0x7f459410ea10 2024-09-11T22:41:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.541+0000 7f457bfff640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f457c0a2ea0 0x7f457c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.541+0000 7f457bfff640 1 -- 172.21.15.5:0/1288555629 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8673+0+0 (secure 0 0 0) 0x7f45880cee70 con 0x7f459410ea10 2024-09-11T22:41:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.543+0000 7f4592d9d640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f457c0a2ea0 0x7f457c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:11.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.547+0000 7f4592d9d640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f457c0a2ea0 0x7f457c0a5360 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f458c005fd0 tx=0x7f458c00d040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.550+0000 7f457bfff640 1 -- 172.21.15.5:0/1288555629 <== mon.2 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f458801f020 con 0x7f459410ea10 2024-09-11T22:41:11.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.940+0000 7f459912c640 1 -- 172.21.15.5:0/1288555629 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f4580000bf0 con 0x7f457c0a2ea0 2024-09-11T22:41:11.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.961+0000 7f457bfff640 1 -- 172.21.15.5:0/1288555629 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f4580000bf0 con 0x7f457c0a2ea0 2024-09-11T22:41:11.964 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-11T00:00:00", "created": "2024-09-11T22:22:26", "first": "2024-09-11T22:23:00", "last": "2024-09-11T22:41:00", "last_pruned": "2024-09-11T22:41:00", "created_count": 19, "pruned_count": 13, "active": true} 2024-09-11T22:41:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.965+0000 7f459912c640 1 -- 172.21.15.5:0/1288555629 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f457c0a2ea0 msgr2=0x7f457c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f457c0a2ea0 0x7f457c0a5360 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f458c005fd0 tx=0x7f458c00d040 comp rx=0 tx=0).stop 2024-09-11T22:41:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 -- 172.21.15.5:0/1288555629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f459410ea10 msgr2=0x7f45941b5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f459410ea10 0x7f45941b5b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f458800b450 tx=0x7f458800b910 comp rx=0 tx=0).stop 2024-09-11T22:41:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 -- 172.21.15.5:0/1288555629 shutdown_connections 2024-09-11T22:41:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45941af6d0 0x7f45941afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f457c0a2ea0 0x7f457c0a5360 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f459410ea10 0x7f45941b5b50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 --2- 172.21.15.5:0/1288555629 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4594109460 0x7f45941b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 -- 172.21.15.5:0/1288555629 >> 172.21.15.5:0/1288555629 conn(0x7f459406d210 msgr2=0x7f4594072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 -- 172.21.15.5:0/1288555629 shutdown_connections 2024-09-11T22:41:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:11.966+0000 7f459912c640 1 -- 172.21.15.5:0/1288555629 wait complete. 2024-09-11T22:41:11.986 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-11T22:41:11.987 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:11 smithi005 ceph-mon[30943]: pgmap v605: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 335 op/s 2024-09-11T22:41:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:11 smithi159 ceph-mon[35534]: pgmap v605: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 335 op/s 2024-09-11T22:41:12.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:11 smithi172 ceph-mon[35461]: pgmap v605: 97 pgs: 97 active+clean; 589 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 335 op/s 2024-09-11T22:41:12.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.174+0000 7fb061274640 1 -- 172.21.15.5:0/1409273907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb05c100a70 msgr2=0x7fb05c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:12.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.174+0000 7fb061274640 1 --2- 172.21.15.5:0/1409273907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb05c100a70 0x7fb05c100e50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009900 tx=0x7fb04c02a490 comp rx=0 tx=0).stop 2024-09-11T22:41:12.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.174+0000 7fb061274640 1 -- 172.21.15.5:0/1409273907 shutdown_connections 2024-09-11T22:41:12.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.174+0000 7fb061274640 1 --2- 172.21.15.5:0/1409273907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb05c105ea0 0x7fb05c106310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:12.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.174+0000 7fb061274640 1 --2- 172.21.15.5:0/1409273907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb05c101390 0x7fb05c105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:12.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.174+0000 7fb061274640 1 --2- 172.21.15.5:0/1409273907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb05c100a70 0x7fb05c100e50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:12.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.174+0000 7fb061274640 1 -- 172.21.15.5:0/1409273907 >> 172.21.15.5:0/1409273907 conn(0x7fb05c077990 msgr2=0x7fb05c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:12.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.175+0000 7fb061274640 1 -- 172.21.15.5:0/1409273907 shutdown_connections 2024-09-11T22:41:12.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.175+0000 7fb061274640 1 -- 172.21.15.5:0/1409273907 wait complete. 2024-09-11T22:41:12.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.176+0000 7fb061274640 1 Processor -- start 2024-09-11T22:41:12.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.176+0000 7fb061274640 1 -- start start 2024-09-11T22:41:12.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.176+0000 7fb061274640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb05c100a70 0x7fb05c073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:12.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.176+0000 7fb061274640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb05c101390 0x7fb05c06ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:12.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.176+0000 7fb061274640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb05c105ea0 0x7fb05c06cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:12.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.176+0000 7fb061274640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c118570 con 0x7fb05c101390 2024-09-11T22:41:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.176+0000 7fb061274640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1183f0 con 0x7fb05c100a70 2024-09-11T22:41:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.176+0000 7fb061274640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fb05c1186f0 con 0x7fb05c105ea0 2024-09-11T22:41:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.177+0000 7fb05b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb05c105ea0 0x7fb05c06cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.177+0000 7fb05b7fe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb05c105ea0 0x7fb05c06cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.5:56910/0 (socket says 172.21.15.5:56910) 2024-09-11T22:41:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.177+0000 7fb05b7fe640 1 -- 172.21.15.5:0/4254838957 learned_addr learned my addr 172.21.15.5:0/4254838957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.177+0000 7fb05b7fe640 1 -- 172.21.15.5:0/4254838957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb05c100a70 msgr2=0x7fb05c073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.177+0000 7fb05b7fe640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb05c100a70 0x7fb05c073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:12.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.177+0000 7fb05b7fe640 1 -- 172.21.15.5:0/4254838957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb05c101390 msgr2=0x7fb05c06ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:12.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.177+0000 7fb05b7fe640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb05c101390 0x7fb05c06ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:12.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.177+0000 7fb05b7fe640 1 -- 172.21.15.5:0/4254838957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05c06d5c0 con 0x7fb05c105ea0 2024-09-11T22:41:12.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.178+0000 7fb05b7fe640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb05c105ea0 0x7fb05c06cfd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb05400b970 tx=0x7fb05400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:12.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.178+0000 7fb053fff640 1 -- 172.21.15.5:0/4254838957 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb054019070 con 0x7fb05c105ea0 2024-09-11T22:41:12.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.178+0000 7fb061274640 1 -- 172.21.15.5:0/4254838957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05c06d8b0 con 0x7fb05c105ea0 2024-09-11T22:41:12.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.178+0000 7fb061274640 1 -- 172.21.15.5:0/4254838957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb05c0718b0 con 0x7fb05c105ea0 2024-09-11T22:41:12.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.178+0000 7fb053fff640 1 -- 172.21.15.5:0/4254838957 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb05400cb40 con 0x7fb05c105ea0 2024-09-11T22:41:12.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.179+0000 7fb053fff640 1 -- 172.21.15.5:0/4254838957 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540076a0 con 0x7fb05c105ea0 2024-09-11T22:41:12.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.183+0000 7fb061274640 1 -- 172.21.15.5:0/4254838957 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb03c005290 con 0x7fb05c105ea0 2024-09-11T22:41:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.186+0000 7fb053fff640 1 -- 172.21.15.5:0/4254838957 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fb05400ccf0 con 0x7fb05c105ea0 2024-09-11T22:41:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.187+0000 7fb053fff640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb0480a2d90 0x7fb0480a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.187+0000 7fb053fff640 1 -- 172.21.15.5:0/4254838957 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8673+0+0 (secure 0 0 0) 0x7fb0540ced40 con 0x7fb05c105ea0 2024-09-11T22:41:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.188+0000 7fb05affd640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb0480a2d90 0x7fb0480a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:12.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.188+0000 7fb05affd640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb0480a2d90 0x7fb0480a5250 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0096c0 tx=0x7fb04c000bc0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.191+0000 7fb053fff640 1 -- 172.21.15.5:0/4254838957 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb05409b970 con 0x7fb05c105ea0 2024-09-11T22:41:12.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.580+0000 7fb061274640 1 -- 172.21.15.5:0/4254838957 --> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb03c000bc0 con 0x7fb0480a2d90 2024-09-11T22:41:12.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.594+0000 7fb053fff640 1 -- 172.21.15.5:0/4254838957 <== mgr.15201 v2:172.21.15.5:6800/233962001 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb03c000bc0 con 0x7fb0480a2d90 2024-09-11T22:41:12.596 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-09-11T22:41:12.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.597+0000 7fb061274640 1 -- 172.21.15.5:0/4254838957 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb0480a2d90 msgr2=0x7fb0480a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:12.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.597+0000 7fb061274640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb0480a2d90 0x7fb0480a5250 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb04c0096c0 tx=0x7fb04c000bc0 comp rx=0 tx=0).stop 2024-09-11T22:41:12.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.597+0000 7fb061274640 1 -- 172.21.15.5:0/4254838957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb05c105ea0 msgr2=0x7fb05c06cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:12.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.598+0000 7fb061274640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb05c105ea0 0x7fb05c06cfd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb05400b970 tx=0x7fb05400be30 comp rx=0 tx=0).stop 2024-09-11T22:41:12.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.598+0000 7fb061274640 1 -- 172.21.15.5:0/4254838957 shutdown_connections 2024-09-11T22:41:12.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.598+0000 7fb061274640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb05c105ea0 0x7fb05c06cfd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:12.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.598+0000 7fb061274640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb05c101390 0x7fb05c06ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:12.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.598+0000 7fb061274640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fb0480a2d90 0x7fb0480a5250 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:12.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.598+0000 7fb061274640 1 --2- 172.21.15.5:0/4254838957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb05c100a70 0x7fb05c073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:12.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.598+0000 7fb061274640 1 -- 172.21.15.5:0/4254838957 >> 172.21.15.5:0/4254838957 conn(0x7fb05c077990 msgr2=0x7fb05c103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.598+0000 7fb061274640 1 -- 172.21.15.5:0/4254838957 shutdown_connections 2024-09-11T22:41:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:12.598+0000 7fb061274640 1 -- 172.21.15.5:0/4254838957 wait complete. 2024-09-11T22:41:12.616 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-11T22:41:12.628 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-11T22:41:13.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:12 smithi005 ceph-mon[30943]: from='client.30281 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:41:13.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:12 smithi159 ceph-mon[35534]: from='client.30281 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:41:13.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:12 smithi172 ceph-mon[35461]: from='client.30281 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:41:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:13 smithi159 ceph-mon[35534]: pgmap v606: 97 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 68 active+clean; 557 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 401 op/s 2024-09-11T22:41:13.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:13 smithi159 ceph-mon[35534]: from='client.30079 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:41:14.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:13 smithi005 ceph-mon[30943]: pgmap v606: 97 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 68 active+clean; 557 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 401 op/s 2024-09-11T22:41:14.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:13 smithi005 ceph-mon[30943]: from='client.30079 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:41:14.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:13 smithi172 ceph-mon[35461]: pgmap v606: 97 pgs: 8 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 68 active+clean; 557 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 401 op/s 2024-09-11T22:41:14.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:13 smithi172 ceph-mon[35461]: from='client.30079 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T22:41:15.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:14 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:41:15.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:14 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:41:15.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:14 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:41:15.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:14 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:41:15.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:14 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:41:15.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:14 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:41:15.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:14 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:41:15.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:14 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:41:15.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:14 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:41:15.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:14 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:41:15.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:14 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:41:15.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:14 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:41:16.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:15 smithi005 ceph-mon[30943]: scrub summary: idle 2024-09-11T22:41:16.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:15 smithi005 ceph-mon[30943]: pgmap v607: 97 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 61 active+clean; 551 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-09-11T22:41:16.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:15 smithi159 ceph-mon[35534]: scrub summary: idle 2024-09-11T22:41:16.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:15 smithi159 ceph-mon[35534]: pgmap v607: 97 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 61 active+clean; 551 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-09-11T22:41:16.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:15 smithi172 ceph-mon[35461]: scrub summary: idle 2024-09-11T22:41:16.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:15 smithi172 ceph-mon[35461]: pgmap v607: 97 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 61 active+clean; 551 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-09-11T22:41:18.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:17 smithi005 ceph-mon[30943]: pgmap v608: 97 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 61 active+clean; 551 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-09-11T22:41:18.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:17 smithi159 ceph-mon[35534]: pgmap v608: 97 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 61 active+clean; 551 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-09-11T22:41:18.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:17 smithi172 ceph-mon[35461]: pgmap v608: 97 pgs: 11 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 61 active+clean; 551 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-09-11T22:41:20.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:19 smithi005 ceph-mon[30943]: pgmap v609: 97 pgs: 97 active+clean; 553 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.8 MiB/s wr, 277 op/s 2024-09-11T22:41:20.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:19 smithi159 ceph-mon[35534]: pgmap v609: 97 pgs: 97 active+clean; 553 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.8 MiB/s wr, 277 op/s 2024-09-11T22:41:20.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:19 smithi172 ceph-mon[35461]: pgmap v609: 97 pgs: 97 active+clean; 553 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.8 MiB/s wr, 277 op/s 2024-09-11T22:41:22.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:21 smithi005 ceph-mon[30943]: pgmap v610: 97 pgs: 97 active+clean; 553 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 3.6 MiB/s wr, 262 op/s 2024-09-11T22:41:22.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:21 smithi159 ceph-mon[35534]: pgmap v610: 97 pgs: 97 active+clean; 553 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 3.6 MiB/s wr, 262 op/s 2024-09-11T22:41:22.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:21 smithi172 ceph-mon[35461]: pgmap v610: 97 pgs: 97 active+clean; 553 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 3.6 MiB/s wr, 262 op/s 2024-09-11T22:41:24.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:23 smithi005 ceph-mon[30943]: pgmap v611: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 4.0 MiB/s wr, 264 op/s 2024-09-11T22:41:24.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:23 smithi005 ceph-mon[30943]: osdmap e119: 12 total, 12 up, 12 in 2024-09-11T22:41:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:23 smithi159 ceph-mon[35534]: pgmap v611: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 4.0 MiB/s wr, 264 op/s 2024-09-11T22:41:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:23 smithi159 ceph-mon[35534]: osdmap e119: 12 total, 12 up, 12 in 2024-09-11T22:41:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:23 smithi172 ceph-mon[35461]: pgmap v611: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 4.0 MiB/s wr, 264 op/s 2024-09-11T22:41:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:23 smithi172 ceph-mon[35461]: osdmap e119: 12 total, 12 up, 12 in 2024-09-11T22:41:25.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:25 smithi005 ceph-mon[30943]: pgmap v613: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 3.3 MiB/s wr, 125 op/s 2024-09-11T22:41:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:25 smithi159 ceph-mon[35534]: pgmap v613: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 3.3 MiB/s wr, 125 op/s 2024-09-11T22:41:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:25 smithi172 ceph-mon[35461]: pgmap v613: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 3.3 MiB/s wr, 125 op/s 2024-09-11T22:41:28.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:27 smithi005 ceph-mon[30943]: pgmap v614: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 3.3 MiB/s wr, 125 op/s 2024-09-11T22:41:28.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:27 smithi159 ceph-mon[35534]: pgmap v614: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 3.3 MiB/s wr, 125 op/s 2024-09-11T22:41:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:27 smithi172 ceph-mon[35461]: pgmap v614: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 3.3 MiB/s wr, 125 op/s 2024-09-11T22:41:30.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:29 smithi005 ceph-mon[30943]: pgmap v615: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.5 MiB/s wr, 139 op/s 2024-09-11T22:41:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:29 smithi159 ceph-mon[35534]: pgmap v615: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.5 MiB/s wr, 139 op/s 2024-09-11T22:41:30.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:29 smithi172 ceph-mon[35461]: pgmap v615: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.5 MiB/s wr, 139 op/s 2024-09-11T22:41:31.882 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:31 smithi159 ceph-mon[35534]: pgmap v616: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.5 MiB/s wr, 139 op/s 2024-09-11T22:41:32.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:31 smithi005 ceph-mon[30943]: pgmap v616: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.5 MiB/s wr, 139 op/s 2024-09-11T22:41:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:31 smithi172 ceph-mon[35461]: pgmap v616: 97 pgs: 97 active+clean; 555 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.5 MiB/s wr, 139 op/s 2024-09-11T22:41:33.912 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:33 smithi172 ceph-mon[35461]: pgmap v617: 97 pgs: 97 active+clean; 547 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.3 MiB/s wr, 140 op/s 2024-09-11T22:41:34.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:33 smithi005 ceph-mon[30943]: pgmap v617: 97 pgs: 97 active+clean; 547 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.3 MiB/s wr, 140 op/s 2024-09-11T22:41:34.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:33 smithi159 ceph-mon[35534]: pgmap v617: 97 pgs: 97 active+clean; 547 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 3.3 MiB/s wr, 140 op/s 2024-09-11T22:41:36.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:35 smithi005 ceph-mon[30943]: pgmap v618: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 2.6 MiB/s wr, 121 op/s 2024-09-11T22:41:36.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:35 smithi159 ceph-mon[35534]: pgmap v618: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 2.6 MiB/s wr, 121 op/s 2024-09-11T22:41:36.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:35 smithi172 ceph-mon[35461]: pgmap v618: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 2.6 MiB/s wr, 121 op/s 2024-09-11T22:41:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:37 smithi005 ceph-mon[30943]: pgmap v619: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 2.5 MiB/s wr, 117 op/s 2024-09-11T22:41:38.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:37 smithi159 ceph-mon[35534]: pgmap v619: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 2.5 MiB/s wr, 117 op/s 2024-09-11T22:41:38.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:37 smithi172 ceph-mon[35461]: pgmap v619: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 2.5 MiB/s wr, 117 op/s 2024-09-11T22:41:40.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-11T22:41:40.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:39 smithi005 ceph-mon[30943]: pgmap v620: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 2.6 MiB/s wr, 128 op/s 2024-09-11T22:41:40.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:39 smithi159 ceph-mon[35534]: pgmap v620: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 2.6 MiB/s wr, 128 op/s 2024-09-11T22:41:40.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:39 smithi172 ceph-mon[35461]: pgmap v620: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 2.6 MiB/s wr, 128 op/s 2024-09-11T22:41:40.321 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:40.767 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:40 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:41:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.859+0000 7fe586411640 1 -- 172.21.15.5:0/289017941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe580101360 msgr2=0x7fe58010df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.859+0000 7fe586411640 1 --2- 172.21.15.5:0/289017941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe580101360 0x7fe58010df90 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fe574009900 tx=0x7fe57402a410 comp rx=0 tx=0).stop 2024-09-11T22:41:40.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.859+0000 7fe586411640 1 -- 172.21.15.5:0/289017941 shutdown_connections 2024-09-11T22:41:40.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.859+0000 7fe586411640 1 --2- 172.21.15.5:0/289017941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58010e4d0 0x7fe5801108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.859+0000 7fe586411640 1 --2- 172.21.15.5:0/289017941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe580101360 0x7fe58010df90 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.859+0000 7fe586411640 1 --2- 172.21.15.5:0/289017941 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe5801009b0 0x7fe580100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.859+0000 7fe586411640 1 -- 172.21.15.5:0/289017941 >> 172.21.15.5:0/289017941 conn(0x7fe5800fc7d0 msgr2=0x7fe5800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:40.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.859+0000 7fe586411640 1 -- 172.21.15.5:0/289017941 shutdown_connections 2024-09-11T22:41:40.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.859+0000 7fe586411640 1 -- 172.21.15.5:0/289017941 wait complete. 2024-09-11T22:41:40.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe586411640 1 Processor -- start 2024-09-11T22:41:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe586411640 1 -- start start 2024-09-11T22:41:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe586411640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801009b0 0x7fe58019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe586411640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe580101360 0x7fe58019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe586411640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58010e4d0 0x7fe58019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe586411640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe580114200 con 0x7fe5801009b0 2024-09-11T22:41:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe586411640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe580114080 con 0x7fe580101360 2024-09-11T22:41:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe586411640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe580114380 con 0x7fe58010e4d0 2024-09-11T22:41:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe57ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801009b0 0x7fe58019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe57f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe580101360 0x7fe58019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe57f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe580101360 0x7fe58019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48156/0 (socket says 172.21.15.5:48156) 2024-09-11T22:41:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe57ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801009b0 0x7fe58019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36034/0 (socket says 172.21.15.5:36034) 2024-09-11T22:41:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe57f7fe640 1 -- 172.21.15.5:0/2448588448 learned_addr learned my addr 172.21.15.5:0/2448588448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.860+0000 7fe5849ae640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58010e4d0 0x7fe58019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe57f7fe640 1 -- 172.21.15.5:0/2448588448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58010e4d0 msgr2=0x7fe58019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58010e4d0 0x7fe58019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe57f7fe640 1 -- 172.21.15.5:0/2448588448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801009b0 msgr2=0x7fe58019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801009b0 0x7fe58019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe57f7fe640 1 -- 172.21.15.5:0/2448588448 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe58019f6f0 con 0x7fe580101360 2024-09-11T22:41:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe5849ae640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58010e4d0 0x7fe58019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:41:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe580101360 0x7fe58019ac80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe574006010 tx=0x7fe5740042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe57d7fa640 1 -- 172.21.15.5:0/2448588448 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57403f030 con 0x7fe580101360 2024-09-11T22:41:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe57d7fa640 1 -- 172.21.15.5:0/2448588448 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe574003ce0 con 0x7fe580101360 2024-09-11T22:41:40.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe586411640 1 -- 172.21.15.5:0/2448588448 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe58019f980 con 0x7fe580101360 2024-09-11T22:41:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe57d7fa640 1 -- 172.21.15.5:0/2448588448 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574035070 con 0x7fe580101360 2024-09-11T22:41:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.861+0000 7fe586411640 1 -- 172.21.15.5:0/2448588448 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5801a72c0 con 0x7fe580101360 2024-09-11T22:41:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.864+0000 7fe57d7fa640 1 -- 172.21.15.5:0/2448588448 <== mon.2 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe574042030 con 0x7fe580101360 2024-09-11T22:41:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.864+0000 7fe5887ea740 1 Processor -- start 2024-09-11T22:41:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.864+0000 7fe5887ea740 1 -- start start 2024-09-11T22:41:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.864+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f03a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.864+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f044950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.864+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 0x559e9f047280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5887ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e9ef972e0 con 0x559e9f03a5d0 2024-09-11T22:41:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5887ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e9ef3ece0 con 0x559e9f039cb0 2024-09-11T22:41:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5887ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559e9ef932a0 con 0x559e9f044e90 2024-09-11T22:41:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe57d7fa640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5680a2cf0 0x7fe5680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5849ae640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f03a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe57ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f044950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5849ae640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f03a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48164/0 (socket says 172.21.15.5:48164) 2024-09-11T22:41:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe57ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f044950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36038/0 (socket says 172.21.15.5:36038) 2024-09-11T22:41:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5849ae640 1 -- 172.21.15.5:0/1003043648 learned_addr learned my addr 172.21.15.5:0/1003043648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/1003043648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 0x559e9f047280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe57d7fa640 1 -- 172.21.15.5:0/2448588448 <== mon.2 v2:172.21.15.159:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7fe5740efbb0 con 0x7fe580101360 2024-09-11T22:41:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5849ae640 1 -- 172.21.15.5:0/1003043648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 msgr2=0x559e9f047280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5849ae640 1 --2- 172.21.15.5:0/1003043648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 0x559e9f047280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5849ae640 1 -- 172.21.15.5:0/1003043648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 msgr2=0x559e9f044950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5849ae640 1 --2- 172.21.15.5:0/1003043648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f044950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5849ae640 1 -- 172.21.15.5:0/1003043648 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9ef23440 con 0x559e9f039cb0 2024-09-11T22:41:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe57ffff640 1 --2- 172.21.15.5:0/1003043648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f044950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.865+0000 7fe5849ae640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5680a2cf0 0x7fe5680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.866+0000 7fe5849ae640 1 --2- 172.21.15.5:0/1003043648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f03a090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe578009fd0 tx=0x7fe578013d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e6171ac255788c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.866+0000 7fe5667fc640 1 -- 172.21.15.5:0/1003043648 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5780128a0 con 0x559e9f039cb0 2024-09-11T22:41:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.866+0000 7fe5667fc640 1 -- 172.21.15.5:0/1003043648 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57801c420 con 0x559e9f039cb0 2024-09-11T22:41:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.866+0000 7fe5667fc640 1 -- 172.21.15.5:0/1003043648 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57800e5d0 con 0x559e9f039cb0 2024-09-11T22:41:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.866+0000 7fe5849ae640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5680a2cf0 0x7fe5680a51b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe578023000 tx=0x7fe578020000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.867+0000 7fe5887ea740 1 -- 172.21.15.5:0/1003043648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 msgr2=0x559e9f03a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.867+0000 7fe5887ea740 1 --2- 172.21.15.5:0/1003043648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f03a090 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe578009fd0 tx=0x7fe578013d30 comp rx=0 tx=0).stop 2024-09-11T22:41:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.867+0000 7fe5887ea740 1 -- 172.21.15.5:0/1003043648 shutdown_connections 2024-09-11T22:41:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.867+0000 7fe5887ea740 1 --2- 172.21.15.5:0/1003043648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 0x559e9f047280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.867+0000 7fe5887ea740 1 --2- 172.21.15.5:0/1003043648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f044950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.867+0000 7fe5887ea740 1 --2- 172.21.15.5:0/1003043648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f03a090 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.867+0000 7fe5887ea740 1 -- 172.21.15.5:0/1003043648 >> 172.21.15.5:0/1003043648 conn(0x559e9f036240 msgr2=0x559e9f036620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.867+0000 7fe5887ea740 1 -- 172.21.15.5:0/1003043648 shutdown_connections 2024-09-11T22:41:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.867+0000 7fe5887ea740 1 -- 172.21.15.5:0/1003043648 wait complete. 2024-09-11T22:41:40.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.868+0000 7fe5887ea740 1 -- start start 2024-09-11T22:41:40.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.868+0000 7fe5887ea740 1 Processor -- start 2024-09-11T22:41:40.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.868+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 0x559e9f0447c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.868+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f041a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.868+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f044e90 0x559e9f041fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.868+0000 7fe5887ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e9f047a60 con 0x559e9f03a5d0 2024-09-11T22:41:40.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.868+0000 7fe5887ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e9ef52050 con 0x559e9f044e90 2024-09-11T22:41:40.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.868+0000 7fe5887ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559e9f047be0 con 0x559e9f039cb0 2024-09-11T22:41:40.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.868+0000 7fe5887ea740 10 client.0 _ll_register_callbacks cb 0x559e9ef92160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:40.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f041a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:40.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f041a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36050/0 (socket says 172.21.15.5:36050) 2024-09-11T22:41:40.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57f7fe640 1 -- 172.21.15.5:0/3899866940 learned_addr learned my addr 172.21.15.5:0/3899866940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:40.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57ffff640 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 0x559e9f0447c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:40.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe5849ae640 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f044e90 0x559e9f041fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:40.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57f7fe640 1 -- 172.21.15.5:0/3899866940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 msgr2=0x559e9f0447c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:40.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 0x559e9f0447c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57f7fe640 1 -- 172.21.15.5:0/3899866940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f044e90 msgr2=0x559e9f041fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:40.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f044e90 0x559e9f041fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:40.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57f7fe640 1 -- 172.21.15.5:0/3899866940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9ed083c0 con 0x559e9f03a5d0 2024-09-11T22:41:40.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57ffff640 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 0x559e9f0447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe5849ae640 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f044e90 0x559e9f041fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f041a70 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fe5740ed340 tx=0x7fe57403b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494991f854f76b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe55f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe55f7fe640 1 -- 172.21.15.5:0/3899866940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574049070 con 0x559e9f03a5d0 2024-09-11T22:41:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe55f7fe640 1 -- 172.21.15.5:0/3899866940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7fe5740c0230 con 0x559e9f03a5d0 2024-09-11T22:41:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.869+0000 7fe5887ea740 1 -- 172.21.15.5:0/3899866940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57801c420 con 0x559e9f03a5d0 2024-09-11T22:41:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.870+0000 7fe55f7fe640 1 -- 172.21.15.5:0/3899866940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe5740bbb80 con 0x559e9f03a5d0 2024-09-11T22:41:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.870+0000 7fe5887ea740 10 client.21264 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.870+0000 7fe5887ea740 1 -- 172.21.15.5:0/3899866940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559e9f053410 con 0x559e9f03a5d0 2024-09-11T22:41:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.870+0000 7fe55f7fe640 1 -- 172.21.15.5:0/3899866940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57801c420 con 0x559e9f03a5d0 2024-09-11T22:41:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.870+0000 7fe55f7fe640 1 -- 172.21.15.5:0/3899866940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fe5740de110 con 0x559e9f03a5d0 2024-09-11T22:41:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.870+0000 7fe5887ea740 10 client.21264 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.870+0000 7fe5887ea740 10 client.21264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.870+0000 7fe5887ea740 20 client.21264 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.871+0000 7fe5887ea740 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e9f0547b0 0x559e9f056ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.871+0000 7fe5887ea740 4 client.21264 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:41.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.871+0000 7fe5887ea740 1 -- 172.21.15.5:0/3899866940 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e9efa4850 con 0x559e9f0547b0 2024-09-11T22:41:41.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.871+0000 7fe57ffff640 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e9f0547b0 0x559e9f056ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:41.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.872+0000 7fe55f7fe640 10 client.21264 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:41.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.872+0000 7fe57ffff640 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e9f0547b0 0x559e9f056ba0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e18cf9f1d6aa8dcd server_cookie=9ea495e21448a93f in_seq=0 out_seq=0 2024-09-11T22:41:41.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.872+0000 7fe55f7fe640 1 -- 172.21.15.5:0/3899866940 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e9efa4850 con 0x559e9f0547b0 2024-09-11T22:41:41.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.872+0000 7fe55f7fe640 10 client.21264 handle_command_reply: tid=0 2024-09-11T22:41:41.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.872+0000 7fe5887ea740 1 client.21264 shutdown 2024-09-11T22:41:41.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 1 -- 172.21.15.5:0/3899866940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 msgr2=0x559e9f041a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f041a70 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fe5740ed340 tx=0x7fe57403b5d0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 1 -- 172.21.15.5:0/3899866940 shutdown_connections 2024-09-11T22:41:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e9f0547b0 0x559e9f056ba0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f044e90 0x559e9f041fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f041a70 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 1 --2- 172.21.15.5:0/3899866940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 0x559e9f0447c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 1 -- 172.21.15.5:0/3899866940 >> 172.21.15.5:0/3899866940 conn(0x559e9f036240 msgr2=0x559e9f02cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe55f7fe640 0 client.21264 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 1 -- 172.21.15.5:0/3899866940 shutdown_connections 2024-09-11T22:41:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 1 -- 172.21.15.5:0/3899866940 wait complete. 2024-09-11T22:41:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.873+0000 7fe5887ea740 20 client.21264 trim_cache size 0 max 16384 2024-09-11T22:41:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.895+0000 7fe5887ea740 1 Processor -- start 2024-09-11T22:41:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.895+0000 7fe5887ea740 1 -- start start 2024-09-11T22:41:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 0x559e9f0439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f03a5d0 0x559e9f05c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f044e90 0x559e9f05c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe5887ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801ab920 con 0x559e9f044e90 2024-09-11T22:41:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe5887ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801ad550 con 0x559e9f03a5d0 2024-09-11T22:41:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe5887ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801abff0 con 0x559e9f039cb0 2024-09-11T22:41:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe5849ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f044e90 0x559e9f05c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe5849ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f044e90 0x559e9f05c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36054/0 (socket says 172.21.15.5:36054) 2024-09-11T22:41:41.003 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:41:41.004 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-11T22:41:41.004 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-11T22:41:41.004 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:41:41.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe5849ae640 1 -- 172.21.15.5:0/917450253 learned_addr learned my addr 172.21.15.5:0/917450253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:41.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe57f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f03a5d0 0x559e9f05c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:41.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe57ffff640 1 --2- 172.21.15.5:0/917450253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 0x559e9f0439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:41.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe57f7fe640 1 -- 172.21.15.5:0/917450253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 msgr2=0x559e9f0439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:41.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/917450253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 0x559e9f0439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe57f7fe640 1 -- 172.21.15.5:0/917450253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f044e90 msgr2=0x559e9f05c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:41.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/917450253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f044e90 0x559e9f05c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe57f7fe640 1 -- 172.21.15.5:0/917450253 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9f03ffc0 con 0x559e9f03a5d0 2024-09-11T22:41:41.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe57ffff640 1 --2- 172.21.15.5:0/917450253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 0x559e9f0439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:41:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.896+0000 7fe5849ae640 1 --2- 172.21.15.5:0/917450253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f044e90 0x559e9f05c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:41:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.897+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/917450253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f03a5d0 0x559e9f05c4b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe5740bbf20 tx=0x7fe574031a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb0f3f1cdfd2113 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.897+0000 7fe565ffb640 1 -- 172.21.15.5:0/917450253 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe574049030 con 0x559e9f03a5d0 2024-09-11T22:41:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.897+0000 7fe565ffb640 1 -- 172.21.15.5:0/917450253 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5740c4ac0 con 0x559e9f03a5d0 2024-09-11T22:41:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.897+0000 7fe565ffb640 1 -- 172.21.15.5:0/917450253 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5740de110 con 0x559e9f03a5d0 2024-09-11T22:41:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 -- 172.21.15.5:0/917450253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f03a5d0 msgr2=0x559e9f05c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 --2- 172.21.15.5:0/917450253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f03a5d0 0x559e9f05c4b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe5740bbf20 tx=0x7fe574031a60 comp rx=0 tx=0).stop 2024-09-11T22:41:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 -- 172.21.15.5:0/917450253 shutdown_connections 2024-09-11T22:41:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 --2- 172.21.15.5:0/917450253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f044e90 0x559e9f05c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 --2- 172.21.15.5:0/917450253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f03a5d0 0x559e9f05c4b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 --2- 172.21.15.5:0/917450253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f039cb0 0x559e9f0439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 -- 172.21.15.5:0/917450253 >> 172.21.15.5:0/917450253 conn(0x559e9f036240 msgr2=0x559e9f0432d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 -- 172.21.15.5:0/917450253 shutdown_connections 2024-09-11T22:41:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 -- 172.21.15.5:0/917450253 wait complete. 2024-09-11T22:41:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 -- start start 2024-09-11T22:41:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 Processor -- start 2024-09-11T22:41:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f04d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f04a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 0x559e9f04adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801ace80 con 0x559e9f03a5d0 2024-09-11T22:41:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801ac6f0 con 0x559e9f039cb0 2024-09-11T22:41:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.898+0000 7fe5887ea740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe5801a9940 con 0x559e9f044e90 2024-09-11T22:41:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe5887ea740 10 client.0 _ll_register_callbacks cb 0x7fe574009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe57f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f04a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe57ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f04d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe5849ae640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 0x559e9f04adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe57ffff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f04d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48190/0 (socket says 172.21.15.5:48190) 2024-09-11T22:41:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe57ffff640 1 -- 172.21.15.5:0/939535352 learned_addr learned my addr 172.21.15.5:0/939535352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe5849ae640 1 -- 172.21.15.5:0/939535352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 msgr2=0x559e9f04d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe5849ae640 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f04d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe5849ae640 1 -- 172.21.15.5:0/939535352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 msgr2=0x559e9f04a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe5849ae640 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f04a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe5849ae640 1 -- 172.21.15.5:0/939535352 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9f04b5a0 con 0x559e9f044e90 2024-09-11T22:41:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe57ffff640 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f04d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T22:41:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe5849ae640 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 0x559e9f04adc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe578000f80 tx=0x7fe5780122a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f20125f68ae9cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe55f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.172:3300/0 2024-09-11T22:41:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe55f7fe640 1 -- 172.21.15.5:0/939535352 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5780132c0 con 0x559e9f044e90 2024-09-11T22:41:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe55f7fe640 1 -- 172.21.15.5:0/939535352 <== mon.1 v2:172.21.15.172:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7fe578018050 con 0x559e9f044e90 2024-09-11T22:41:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.899+0000 7fe5887ea740 1 -- 172.21.15.5:0/939535352 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5740c4ac0 con 0x559e9f044e90 2024-09-11T22:41:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe55f7fe640 1 -- 172.21.15.5:0/939535352 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe578009480 con 0x559e9f044e90 2024-09-11T22:41:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe5887ea740 10 client.30097 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe5887ea740 1 -- 172.21.15.5:0/939535352 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559e9f0bf450 con 0x559e9f044e90 2024-09-11T22:41:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe55f7fe640 1 -- 172.21.15.5:0/939535352 <== mon.1 v2:172.21.15.172:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5740c4ac0 con 0x559e9f044e90 2024-09-11T22:41:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe55f7fe640 1 -- 172.21.15.5:0/939535352 <== mon.1 v2:172.21.15.172:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fe578027a70 con 0x559e9f044e90 2024-09-11T22:41:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe5887ea740 10 client.30097 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe5887ea740 10 client.30097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe5887ea740 20 client.30097 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe5887ea740 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e9f0bfa90 0x559e9f0c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe5887ea740 4 client.30097 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-11T22:41:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe5887ea740 1 -- 172.21.15.5:0/939535352 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e9ef9ed60 con 0x559e9f0bfa90 2024-09-11T22:41:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.900+0000 7fe57ffff640 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e9f0bfa90 0x559e9f0c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe55f7fe640 10 client.30097 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe57ffff640 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e9f0bfa90 0x559e9f0c1eb0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d3d12d554721ce server_cookie=ad82ade686c5f74c in_seq=0 out_seq=0 2024-09-11T22:41:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe55f7fe640 1 -- 172.21.15.5:0/939535352 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559e9ef9ed60 con 0x559e9f0bfa90 2024-09-11T22:41:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe55f7fe640 10 client.30097 handle_command_reply: tid=0 2024-09-11T22:41:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 client.30097 shutdown 2024-09-11T22:41:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 -- 172.21.15.5:0/939535352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 msgr2=0x559e9f04adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 0x559e9f04adc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe578000f80 tx=0x7fe5780122a0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 -- 172.21.15.5:0/939535352 shutdown_connections 2024-09-11T22:41:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x559e9f0bfa90 0x559e9f0c1eb0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559e9f044e90 0x559e9f04adc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e9f03a5d0 0x559e9f04a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe55f7fe640 0 client.30097 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 --2- 172.21.15.5:0/939535352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e9f039cb0 0x559e9f04d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 -- 172.21.15.5:0/939535352 >> 172.21.15.5:0/939535352 conn(0x559e9f036240 msgr2=0x559e9f043af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:41.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 -- 172.21.15.5:0/939535352 shutdown_connections 2024-09-11T22:41:41.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 1 -- 172.21.15.5:0/939535352 wait complete. 2024-09-11T22:41:41.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.901+0000 7fe5887ea740 20 client.30097 trim_cache size 0 max 16384 2024-09-11T22:41:41.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 -- 172.21.15.5:0/2448588448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5680a2cf0 msgr2=0x7fe5680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:41.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5680a2cf0 0x7fe5680a51b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe578023000 tx=0x7fe578020000 comp rx=0 tx=0).stop 2024-09-11T22:41:41.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 -- 172.21.15.5:0/2448588448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe580101360 msgr2=0x7fe58019ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:41.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe580101360 0x7fe58019ac80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe574006010 tx=0x7fe5740042f0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 -- 172.21.15.5:0/2448588448 shutdown_connections 2024-09-11T22:41:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe58010e4d0 0x7fe58019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe5680a2cf0 0x7fe5680a51b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe580101360 0x7fe58019ac80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 --2- 172.21.15.5:0/2448588448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801009b0 0x7fe58019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 -- 172.21.15.5:0/2448588448 >> 172.21.15.5:0/2448588448 conn(0x7fe5800fc7d0 msgr2=0x7fe5800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:41.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 -- 172.21.15.5:0/2448588448 shutdown_connections 2024-09-11T22:41:41.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:40.902+0000 7fe586411640 1 -- 172.21.15.5:0/2448588448 wait complete. 2024-09-11T22:41:41.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:40 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:41:41.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:40 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T22:41:41.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:41 smithi005 ceph-mon[30943]: pgmap v621: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-09-11T22:41:41.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:41 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:41:41.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:41 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:41:41.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:41 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:41:41.826 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-11T22:41:41.827 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:41:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:41 smithi159 ceph-mon[35534]: pgmap v621: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-09-11T22:41:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:41 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:41:42.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:41 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:41:42.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:41 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:41:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:41 smithi172 ceph-mon[35461]: pgmap v621: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.2 MiB/s wr, 74 op/s 2024-09-11T22:41:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:41 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T22:41:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:41 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T22:41:42.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:41 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' 2024-09-11T22:41:42.140 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.645+0000 7f1e60cc7640 1 -- 172.21.15.5:0/3402751562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102790 msgr2=0x7f1e5c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.645+0000 7f1e60cc7640 1 --2- 172.21.15.5:0/3402751562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102790 0x7f1e5c1067e0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f1e50009900 tx=0x7f1e5002a410 comp rx=0 tx=0).stop 2024-09-11T22:41:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.645+0000 7f1e60cc7640 1 -- 172.21.15.5:0/3402751562 shutdown_connections 2024-09-11T22:41:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.645+0000 7f1e60cc7640 1 --2- 172.21.15.5:0/3402751562 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e5c106f10 0x7f1e5c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.645+0000 7f1e60cc7640 1 --2- 172.21.15.5:0/3402751562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c102790 0x7f1e5c1067e0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.645+0000 7f1e60cc7640 1 --2- 172.21.15.5:0/3402751562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e5c101de0 0x7f1e5c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.645+0000 7f1e60cc7640 1 -- 172.21.15.5:0/3402751562 >> 172.21.15.5:0/3402751562 conn(0x7f1e5c0fd8c0 msgr2=0x7f1e5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.645+0000 7f1e60cc7640 1 -- 172.21.15.5:0/3402751562 shutdown_connections 2024-09-11T22:41:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.645+0000 7f1e60cc7640 1 -- 172.21.15.5:0/3402751562 wait complete. 2024-09-11T22:41:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e60cc7640 1 Processor -- start 2024-09-11T22:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e60cc7640 1 -- start start 2024-09-11T22:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e60cc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c101de0 0x7f1e5c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e60cc7640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e5c102790 0x7f1e5c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e60cc7640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e5c106f10 0x7f1e5c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e60cc7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c111f70 con 0x7f1e5c101de0 2024-09-11T22:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e60cc7640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c111df0 con 0x7f1e5c102790 2024-09-11T22:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e60cc7640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5c1120f0 con 0x7f1e5c106f10 2024-09-11T22:41:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e5a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c101de0 0x7f1e5c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e5a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c101de0 0x7f1e5c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57754/0 (socket says 172.21.15.5:57754) 2024-09-11T22:41:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e5a59c640 1 -- 172.21.15.5:0/188649176 learned_addr learned my addr 172.21.15.5:0/188649176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e59d9b640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e5c102790 0x7f1e5c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e5a59c640 1 -- 172.21.15.5:0/188649176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e5c106f10 msgr2=0x7f1e5c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e5a59c640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e5c106f10 0x7f1e5c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e5a59c640 1 -- 172.21.15.5:0/188649176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e5c102790 msgr2=0x7f1e5c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e5a59c640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e5c102790 0x7f1e5c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e5a59c640 1 -- 172.21.15.5:0/188649176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5c1a6890 con 0x7f1e5c101de0 2024-09-11T22:41:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.646+0000 7f1e5a59c640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c101de0 0x7f1e5c0744f0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00d770 tx=0x7f1e4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.647+0000 7f1e4b7fe640 1 -- 172.21.15.5:0/188649176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c017070 con 0x7f1e5c101de0 2024-09-11T22:41:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.647+0000 7f1e4b7fe640 1 -- 172.21.15.5:0/188649176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e4c005500 con 0x7f1e5c101de0 2024-09-11T22:41:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.647+0000 7f1e60cc7640 1 -- 172.21.15.5:0/188649176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5c1a6b80 con 0x7f1e5c101de0 2024-09-11T22:41:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.647+0000 7f1e4b7fe640 1 -- 172.21.15.5:0/188649176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c004e00 con 0x7f1e5c101de0 2024-09-11T22:41:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.647+0000 7f1e60cc7640 1 -- 172.21.15.5:0/188649176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5c1a70c0 con 0x7f1e5c101de0 2024-09-11T22:41:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.648+0000 7f1e60cc7640 1 -- 172.21.15.5:0/188649176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5c1021c0 con 0x7f1e5c101de0 2024-09-11T22:41:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.649+0000 7f1e4b7fe640 1 -- 172.21.15.5:0/188649176 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1e4c004fa0 con 0x7f1e5c101de0 2024-09-11T22:41:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.650+0000 7f1e4b7fe640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1e440a2d70 0x7f1e440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.650+0000 7f1e59d9b640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1e440a2d70 0x7f1e440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.650+0000 7f1e4b7fe640 1 -- 172.21.15.5:0/188649176 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f1e4c0cef80 con 0x7f1e5c101de0 2024-09-11T22:41:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.650+0000 7f1e59d9b640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1e440a2d70 0x7f1e440a5230 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c075f80 tx=0x7f1e50005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.656+0000 7f1e4b7fe640 1 -- 172.21.15.5:0/188649176 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1e4c09bb30 con 0x7f1e5c101de0 2024-09-11T22:41:42.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.868+0000 7f1e60cc7640 1 -- 172.21.15.5:0/188649176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e5c075a20 con 0x7f1e5c101de0 2024-09-11T22:41:42.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.868+0000 7f1e4b7fe640 1 -- 172.21.15.5:0/188649176 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11975 (secure 0 0 0) 0x7f1e4c09b430 con 0x7f1e5c101de0 2024-09-11T22:41:42.869 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:41:42.870 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":57,"btime":"2024-09-11T22:40:39:632093+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:40:38.909431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:41:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 57 2024-09-11T22:41:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 -- 172.21.15.5:0/188649176 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1e440a2d70 msgr2=0x7f1e440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1e440a2d70 0x7f1e440a5230 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c075f80 tx=0x7f1e50005a20 comp rx=0 tx=0).stop 2024-09-11T22:41:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 -- 172.21.15.5:0/188649176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c101de0 msgr2=0x7f1e5c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c101de0 0x7f1e5c0744f0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00d770 tx=0x7f1e4c00dc30 comp rx=0 tx=0).stop 2024-09-11T22:41:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 -- 172.21.15.5:0/188649176 shutdown_connections 2024-09-11T22:41:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e5c106f10 0x7f1e5c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1e440a2d70 0x7f1e440a5230 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1e5c102790 0x7f1e5c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 --2- 172.21.15.5:0/188649176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e5c101de0 0x7f1e5c0744f0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 -- 172.21.15.5:0/188649176 >> 172.21.15.5:0/188649176 conn(0x7f1e5c0fd8c0 msgr2=0x7f1e5c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 -- 172.21.15.5:0/188649176 shutdown_connections 2024-09-11T22:41:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:42.871+0000 7f1e60cc7640 1 -- 172.21.15.5:0/188649176 wait complete. 2024-09-11T22:41:43.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-11T22:41:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:44.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:43 smithi005 ceph-mon[30943]: pgmap v622: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.2 MiB/s wr, 75 op/s 2024-09-11T22:41:44.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:43 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/188649176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:41:44.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:43 smithi159 ceph-mon[35534]: pgmap v622: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.2 MiB/s wr, 75 op/s 2024-09-11T22:41:44.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:43 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/188649176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:41:44.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:43 smithi172 ceph-mon[35461]: pgmap v622: 97 pgs: 97 active+clean; 544 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1.2 MiB/s wr, 75 op/s 2024-09-11T22:41:44.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:43 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/188649176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:41:44.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.557+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/799782958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3acc103660 msgr2=0x7f3acc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.557+0000 7f3ad1f4f640 1 --2- 172.21.15.5:0/799782958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3acc103660 0x7f3acc10bb50 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009900 tx=0x7f3ac002a480 comp rx=0 tx=0).stop 2024-09-11T22:41:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.557+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/799782958 shutdown_connections 2024-09-11T22:41:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.557+0000 7f3ad1f4f640 1 --2- 172.21.15.5:0/799782958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3acc10c170 0x7f3acc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.557+0000 7f3ad1f4f640 1 --2- 172.21.15.5:0/799782958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3acc103660 0x7f3acc10bb50 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.557+0000 7f3ad1f4f640 1 --2- 172.21.15.5:0/799782958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3acc102d40 0x7f3acc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.557+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/799782958 >> 172.21.15.5:0/799782958 conn(0x7f3acc0fc7a0 msgr2=0x7f3acc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.557+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/799782958 shutdown_connections 2024-09-11T22:41:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.557+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/799782958 wait complete. 2024-09-11T22:41:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.558+0000 7f3ad1f4f640 1 Processor -- start 2024-09-11T22:41:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.558+0000 7f3ad1f4f640 1 -- start start 2024-09-11T22:41:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3ad1f4f640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3acc102d40 0x7f3acc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3ad1f4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3acc103660 0x7f3acc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3ad1f4f640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3acc10c170 0x7f3acc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3ad1f4f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc111f70 con 0x7f3acc103660 2024-09-11T22:41:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3ad1f4f640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc111df0 con 0x7f3acc102d40 2024-09-11T22:41:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3ad1f4f640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1120f0 con 0x7f3acc10c170 2024-09-11T22:41:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3acc103660 0x7f3acc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3acc103660 0x7f3acc198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57762/0 (socket says 172.21.15.5:57762) 2024-09-11T22:41:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acaffd640 1 -- 172.21.15.5:0/3445104178 learned_addr learned my addr 172.21.15.5:0/3445104178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3acc102d40 0x7f3acc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acbfff640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3acc10c170 0x7f3acc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acaffd640 1 -- 172.21.15.5:0/3445104178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3acc10c170 msgr2=0x7f3acc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acaffd640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3acc10c170 0x7f3acc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acaffd640 1 -- 172.21.15.5:0/3445104178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3acc102d40 msgr2=0x7f3acc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acaffd640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3acc102d40 0x7f3acc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acaffd640 1 -- 172.21.15.5:0/3445104178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3acc19d3e0 con 0x7f3acc103660 2024-09-11T22:41:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3acc102d40 0x7f3acc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3acaffd640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3acc103660 0x7f3acc198970 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0006010 tx=0x7f3ac00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.559+0000 7f3ac8ff9640 1 -- 172.21.15.5:0/3445104178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac003f030 con 0x7f3acc103660 2024-09-11T22:41:44.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.560+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/3445104178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3acc19d670 con 0x7f3acc103660 2024-09-11T22:41:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.560+0000 7f3ac8ff9640 1 -- 172.21.15.5:0/3445104178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac0003ce0 con 0x7f3acc103660 2024-09-11T22:41:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.560+0000 7f3ac8ff9640 1 -- 172.21.15.5:0/3445104178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0035070 con 0x7f3acc103660 2024-09-11T22:41:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.560+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/3445104178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3acc1a4fb0 con 0x7f3acc103660 2024-09-11T22:41:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.562+0000 7f3ac8ff9640 1 -- 172.21.15.5:0/3445104178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f3ac0042030 con 0x7f3acc103660 2024-09-11T22:41:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.562+0000 7f3ad4328740 1 Processor -- start 2024-09-11T22:41:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.562+0000 7f3ad4328740 1 -- start start 2024-09-11T22:41:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.562+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef15b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.562+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef15b5d0 0x55d4ef15ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef166370 0x55d4ef168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3ad4328740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef0c12f0 con 0x55d4ef166370 2024-09-11T22:41:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3ad4328740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef068cf0 con 0x55d4ef15b5d0 2024-09-11T22:41:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3ad4328740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef0bd2b0 con 0x55d4ef15cb40 2024-09-11T22:41:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef166370 0x55d4ef168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef166370 0x55d4ef168760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57766/0 (socket says 172.21.15.5:57766) 2024-09-11T22:41:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3ac8ff9640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ab40a2cf0 0x7f3ab40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acaffd640 1 -- 172.21.15.5:0/3924892504 learned_addr learned my addr 172.21.15.5:0/3924892504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/3924892504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef15b5d0 0x55d4ef15ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acbfff640 1 --2- 172.21.15.5:0/3924892504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acbfff640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ab40a2cf0 0x7f3ab40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3ac8ff9640 1 -- 172.21.15.5:0/3445104178 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f3ac00efbb0 con 0x7f3acc103660 2024-09-11T22:41:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acaffd640 1 -- 172.21.15.5:0/3924892504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 msgr2=0x55d4ef15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acaffd640 1 --2- 172.21.15.5:0/3924892504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef15b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acaffd640 1 -- 172.21.15.5:0/3924892504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef15b5d0 msgr2=0x55d4ef15ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:44.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acaffd640 1 --2- 172.21.15.5:0/3924892504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef15b5d0 0x55d4ef15ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acaffd640 1 -- 172.21.15.5:0/3924892504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4ef04d3a0 con 0x55d4ef166370 2024-09-11T22:41:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acaffd640 1 --2- 172.21.15.5:0/3924892504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef166370 0x55d4ef168760 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00ed1a0 tx=0x7f3ac002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5303fe108a2b3e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3ab1ffb640 1 -- 172.21.15.5:0/3924892504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0049070 con 0x55d4ef166370 2024-09-11T22:41:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.563+0000 7f3acbfff640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ab40a2cf0 0x7f3ab40a51b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005cc0 tx=0x7f3ac4004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ab1ffb640 1 -- 172.21.15.5:0/3924892504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac00bc7c0 con 0x55d4ef166370 2024-09-11T22:41:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ab1ffb640 1 -- 172.21.15.5:0/3924892504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00c0470 con 0x55d4ef166370 2024-09-11T22:41:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ad4328740 1 -- 172.21.15.5:0/3924892504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef166370 msgr2=0x55d4ef168760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ad4328740 1 --2- 172.21.15.5:0/3924892504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef166370 0x55d4ef168760 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00ed1a0 tx=0x7f3ac002a480 comp rx=0 tx=0).stop 2024-09-11T22:41:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ad4328740 1 -- 172.21.15.5:0/3924892504 shutdown_connections 2024-09-11T22:41:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ad4328740 1 --2- 172.21.15.5:0/3924892504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef166370 0x55d4ef168760 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ad4328740 1 --2- 172.21.15.5:0/3924892504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef15b5d0 0x55d4ef15ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ad4328740 1 --2- 172.21.15.5:0/3924892504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ad4328740 1 -- 172.21.15.5:0/3924892504 >> 172.21.15.5:0/3924892504 conn(0x55d4ef124310 msgr2=0x55d4ef157c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ad4328740 1 -- 172.21.15.5:0/3924892504 shutdown_connections 2024-09-11T22:41:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.564+0000 7f3ad4328740 1 -- 172.21.15.5:0/3924892504 wait complete. 2024-09-11T22:41:44.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3ad4328740 1 -- start start 2024-09-11T22:41:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3ad4328740 1 Processor -- start 2024-09-11T22:41:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef165dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef163080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 0x55d4ef163810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3ad4328740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef0c12f0 con 0x55d4ef15b5d0 2024-09-11T22:41:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3ad4328740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef07c060 con 0x55d4ef166370 2024-09-11T22:41:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3ad4328740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55d4ef168e30 con 0x55d4ef15cb40 2024-09-11T22:41:44.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3ad4328740 10 client.0 _ll_register_callbacks cb 0x55d4eedcf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3acb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef165dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3acb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef165dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57778/0 (socket says 172.21.15.5:57778) 2024-09-11T22:41:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3acb7fe640 1 -- 172.21.15.5:0/1571337216 learned_addr learned my addr 172.21.15.5:0/1571337216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.565+0000 7f3acbfff640 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 0x55d4ef163810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3acaffd640 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3acb7fe640 1 -- 172.21.15.5:0/1571337216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 msgr2=0x55d4ef163080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef163080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3acb7fe640 1 -- 172.21.15.5:0/1571337216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 msgr2=0x55d4ef163810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:44.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 0x55d4ef163810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3acb7fe640 1 -- 172.21.15.5:0/1571337216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4eee323c0 con 0x55d4ef15b5d0 2024-09-11T22:41:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3acaffd640 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef165dd0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00bf40 tx=0x7f3abc0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5828489c190bbdc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3aaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3aaaffd640 1 -- 172.21.15.5:0/1571337216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc01f060 con 0x55d4ef15b5d0 2024-09-11T22:41:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3aaaffd640 1 -- 172.21.15.5:0/1571337216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f3abc010960 con 0x55d4ef15b5d0 2024-09-11T22:41:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3ad4328740 1 -- 172.21.15.5:0/1571337216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac00bc7c0 con 0x55d4ef15b5d0 2024-09-11T22:41:44.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3aaaffd640 1 -- 172.21.15.5:0/1571337216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f3abc010b60 con 0x55d4ef15b5d0 2024-09-11T22:41:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3ad4328740 10 client.21288 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3ad4328740 1 -- 172.21.15.5:0/1571337216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d4ef174830 con 0x55d4ef15b5d0 2024-09-11T22:41:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.566+0000 7f3aaaffd640 1 -- 172.21.15.5:0/1571337216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac00bc7c0 con 0x55d4ef15b5d0 2024-09-11T22:41:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.567+0000 7f3aaaffd640 1 -- 172.21.15.5:0/1571337216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f3abc019030 con 0x55d4ef15b5d0 2024-09-11T22:41:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.567+0000 7f3ad4328740 10 client.21288 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.567+0000 7f3ad4328740 10 client.21288 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:41:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.567+0000 7f3ad4328740 20 client.21288 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:44.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.567+0000 7f3ad4328740 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55d4ef175d40 0x55d4ef1781b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.567+0000 7f3ad4328740 4 client.21288 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.567+0000 7f3ad4328740 1 -- 172.21.15.5:0/1571337216 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4ef0ce860 con 0x55d4ef175d40 2024-09-11T22:41:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.567+0000 7f3acaffd640 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55d4ef175d40 0x55d4ef1781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.567+0000 7f3acaffd640 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55d4ef175d40 0x55d4ef1781b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b0a6a072546e73f server_cookie=ceffe60b48b3c6a5 in_seq=0 out_seq=0 2024-09-11T22:41:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.567+0000 7f3aaaffd640 10 client.21288 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:41:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.568+0000 7f3aaaffd640 1 -- 172.21.15.5:0/1571337216 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4ef0ce860 con 0x55d4ef175d40 2024-09-11T22:41:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.568+0000 7f3aaaffd640 10 client.21288 handle_command_reply: tid=0 2024-09-11T22:41:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.568+0000 7f3ad4328740 1 client.21288 shutdown 2024-09-11T22:41:44.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.568+0000 7f3ad4328740 1 -- 172.21.15.5:0/1571337216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 msgr2=0x55d4ef165dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.568+0000 7f3ad4328740 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef165dd0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00bf40 tx=0x7f3abc0128e0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.569+0000 7f3ad4328740 1 -- 172.21.15.5:0/1571337216 shutdown_connections 2024-09-11T22:41:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.569+0000 7f3ad4328740 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55d4ef175d40 0x55d4ef1781b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.569+0000 7f3ad4328740 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 0x55d4ef163810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.569+0000 7f3ad4328740 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef163080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.569+0000 7f3ad4328740 1 --2- 172.21.15.5:0/1571337216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef165dd0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.569+0000 7f3aaaffd640 0 client.21288 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:41:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.569+0000 7f3ad4328740 1 -- 172.21.15.5:0/1571337216 >> 172.21.15.5:0/1571337216 conn(0x55d4ef124310 msgr2=0x55d4ef15d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.569+0000 7f3ad4328740 1 -- 172.21.15.5:0/1571337216 shutdown_connections 2024-09-11T22:41:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.569+0000 7f3ad4328740 1 -- 172.21.15.5:0/1571337216 wait complete. 2024-09-11T22:41:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.569+0000 7f3ad4328740 20 client.21288 trim_cache size 0 max 16384 2024-09-11T22:41:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3ad4328740 1 Processor -- start 2024-09-11T22:41:44.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3ad4328740 1 -- start start 2024-09-11T22:41:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef17da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef17dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 0x55d4ef1602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3ad4328740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1ada40 con 0x55d4ef15b5d0 2024-09-11T22:41:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3ad4328740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1af670 con 0x55d4ef166370 2024-09-11T22:41:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3ad4328740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1ae110 con 0x55d4ef15cb40 2024-09-11T22:41:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef17da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef17da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57792/0 (socket says 172.21.15.5:57792) 2024-09-11T22:41:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acb7fe640 1 -- 172.21.15.5:0/1753792564 learned_addr learned my addr 172.21.15.5:0/1753792564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acaffd640 1 --2- 172.21.15.5:0/1753792564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef17dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acbfff640 1 --2- 172.21.15.5:0/1753792564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 0x55d4ef1602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acb7fe640 1 -- 172.21.15.5:0/1753792564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 msgr2=0x55d4ef17dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/1753792564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef17dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acb7fe640 1 -- 172.21.15.5:0/1753792564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 msgr2=0x55d4ef1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/1753792564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 0x55d4ef1602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acb7fe640 1 -- 172.21.15.5:0/1753792564 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4ef15e120 con 0x55d4ef15b5d0 2024-09-11T22:41:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.582+0000 7f3acbfff640 1 --2- 172.21.15.5:0/1753792564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 0x55d4ef1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/1753792564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef17da80 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f3abc000c00 tx=0x7f3abc010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e31119d563310a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3ab17fa640 1 -- 172.21.15.5:0/1753792564 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc030020 con 0x55d4ef15b5d0 2024-09-11T22:41:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3ab17fa640 1 -- 172.21.15.5:0/1753792564 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3abc01a830 con 0x55d4ef15b5d0 2024-09-11T22:41:44.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3ab17fa640 1 -- 172.21.15.5:0/1753792564 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc005540 con 0x55d4ef15b5d0 2024-09-11T22:41:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3ad4328740 1 -- 172.21.15.5:0/1753792564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 msgr2=0x55d4ef17da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3ad4328740 1 --2- 172.21.15.5:0/1753792564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef17da80 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f3abc000c00 tx=0x7f3abc010d00 comp rx=0 tx=0).stop 2024-09-11T22:41:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3ad4328740 1 -- 172.21.15.5:0/1753792564 shutdown_connections 2024-09-11T22:41:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3ad4328740 1 --2- 172.21.15.5:0/1753792564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef166370 0x55d4ef1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3ad4328740 1 --2- 172.21.15.5:0/1753792564 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef15cb40 0x55d4ef17dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3ad4328740 1 --2- 172.21.15.5:0/1753792564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15b5d0 0x55d4ef17da80 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.583+0000 7f3ad4328740 1 -- 172.21.15.5:0/1753792564 >> 172.21.15.5:0/1753792564 conn(0x55d4ef124310 msgr2=0x55d4ef162170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:44.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 1 -- 172.21.15.5:0/1753792564 shutdown_connections 2024-09-11T22:41:44.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 1 -- 172.21.15.5:0/1753792564 wait complete. 2024-09-11T22:41:44.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 1 -- start start 2024-09-11T22:41:44.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 1 Processor -- start 2024-09-11T22:41:44.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef15b5d0 0x55d4ef169fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15cb40 0x55d4ef16a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef166370 0x55d4ef16aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1aefa0 con 0x55d4ef15cb40 2024-09-11T22:41:44.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1ae810 con 0x55d4ef15b5d0 2024-09-11T22:41:44.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f3acc1aba40 con 0x55d4ef166370 2024-09-11T22:41:44.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3ad4328740 10 client.0 _ll_register_callbacks cb 0x7f3abc00bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:44.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.584+0000 7f3acaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15cb40 0x55d4ef16a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3acaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15cb40 0x55d4ef16a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57796/0 (socket says 172.21.15.5:57796) 2024-09-11T22:41:44.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3acaffd640 1 -- 172.21.15.5:0/978920485 learned_addr learned my addr 172.21.15.5:0/978920485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3acbfff640 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef166370 0x55d4ef16aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef15b5d0 0x55d4ef169fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3acaffd640 1 -- 172.21.15.5:0/978920485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef166370 msgr2=0x55d4ef16aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3acaffd640 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef166370 0x55d4ef16aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3acaffd640 1 -- 172.21.15.5:0/978920485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef15b5d0 msgr2=0x55d4ef169fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3acaffd640 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef15b5d0 0x55d4ef169fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3acaffd640 1 -- 172.21.15.5:0/978920485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ef1e4000 con 0x55d4ef15cb40 2024-09-11T22:41:44.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3acaffd640 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15cb40 0x55d4ef16a4e0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00bbdc0 tx=0x7f3ac0031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92705dedd4c90c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3aaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3aaaffd640 1 -- 172.21.15.5:0/978920485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0049040 con 0x55d4ef15cb40 2024-09-11T22:41:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3aaaffd640 1 -- 172.21.15.5:0/978920485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f3ac00c0150 con 0x55d4ef15cb40 2024-09-11T22:41:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3ad4328740 1 -- 172.21.15.5:0/978920485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3abc01a830 con 0x55d4ef15cb40 2024-09-11T22:41:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3aaaffd640 1 -- 172.21.15.5:0/978920485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f3ac00bbc00 con 0x55d4ef15cb40 2024-09-11T22:41:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3ad4328740 10 client.21294 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.585+0000 7f3ad4328740 1 -- 172.21.15.5:0/978920485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55d4ef1e5230 con 0x55d4ef15cb40 2024-09-11T22:41:44.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.586+0000 7f3aaaffd640 1 -- 172.21.15.5:0/978920485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3abc01a830 con 0x55d4ef15cb40 2024-09-11T22:41:44.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.586+0000 7f3aaaffd640 1 -- 172.21.15.5:0/978920485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f3ac00ed4b0 con 0x55d4ef15cb40 2024-09-11T22:41:44.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.586+0000 7f3ad4328740 10 client.21294 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:44.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.586+0000 7f3ad4328740 10 client.21294 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.c 2024-09-11T22:41:44.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.586+0000 7f3ad4328740 20 client.21294 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:44.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.586+0000 7f3ad4328740 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55d4ef1e5850 0x55d4ef1e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:44.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.586+0000 7f3ad4328740 4 client.21294 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:41:44.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.586+0000 7f3ad4328740 1 -- 172.21.15.5:0/978920485 --> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4ef0c8d70 con 0x55d4ef1e5850 2024-09-11T22:41:44.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.586+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55d4ef1e5850 0x55d4ef1e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:44.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3acb7fe640 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55d4ef1e5850 0x55d4ef1e7c70 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=57a6a92f9ec196bc server_cookie=67b18fe8d5f94550 in_seq=0 out_seq=0 2024-09-11T22:41:44.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3aaaffd640 10 client.21294 ms_handle_connect on v2:172.21.15.172:6836/990138139 2024-09-11T22:41:44.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3aaaffd640 1 -- 172.21.15.5:0/978920485 <== mds.4 v2:172.21.15.172:6836/990138139 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4ef0c8d70 con 0x55d4ef1e5850 2024-09-11T22:41:44.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3aaaffd640 10 client.21294 handle_command_reply: tid=0 2024-09-11T22:41:44.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 client.21294 shutdown 2024-09-11T22:41:44.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 -- 172.21.15.5:0/978920485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15cb40 msgr2=0x55d4ef16a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:44.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15cb40 0x55d4ef16a4e0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00bbdc0 tx=0x7f3ac0031a60 comp rx=0 tx=0).stop 2024-09-11T22:41:44.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 -- 172.21.15.5:0/978920485 shutdown_connections 2024-09-11T22:41:44.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.172:6836/990138139,v1:172.21.15.172:6837/990138139] conn(0x55d4ef1e5850 0x55d4ef1e7c70 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d4ef166370 0x55d4ef16aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4ef15cb40 0x55d4ef16a4e0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3aaaffd640 0 client.21294 ms_handle_reset on v2:172.21.15.172:6836/990138139 2024-09-11T22:41:44.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 --2- 172.21.15.5:0/978920485 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d4ef15b5d0 0x55d4ef169fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 -- 172.21.15.5:0/978920485 >> 172.21.15.5:0/978920485 conn(0x55d4ef124310 msgr2=0x55d4ef15d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:44.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 -- 172.21.15.5:0/978920485 shutdown_connections 2024-09-11T22:41:44.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 1 -- 172.21.15.5:0/978920485 wait complete. 2024-09-11T22:41:44.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.587+0000 7f3ad4328740 20 client.21294 trim_cache size 0 max 16384 2024-09-11T22:41:44.614 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:41:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/3445104178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ab40a2cf0 msgr2=0x7f3ab40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ab40a2cf0 0x7f3ab40a51b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4005cc0 tx=0x7f3ac4004a40 comp rx=0 tx=0).stop 2024-09-11T22:41:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/3445104178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3acc103660 msgr2=0x7f3acc198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3acc103660 0x7f3acc198970 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0006010 tx=0x7f3ac00042f0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/3445104178 shutdown_connections 2024-09-11T22:41:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3acc10c170 0x7f3acc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3acc103660 0x7f3acc198970 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f3ab40a2cf0 0x7f3ab40a51b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 --2- 172.21.15.5:0/3445104178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3acc102d40 0x7f3acc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/3445104178 >> 172.21.15.5:0/3445104178 conn(0x7f3acc0fc7a0 msgr2=0x7f3acc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/3445104178 shutdown_connections 2024-09-11T22:41:44.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:44.588+0000 7f3ad1f4f640 1 -- 172.21.15.5:0/3445104178 wait complete. 2024-09-11T22:41:45.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-11T22:41:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:45.956 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:45 smithi005 ceph-mon[30943]: pgmap v623: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 591 KiB/s wr, 46 op/s 2024-09-11T22:41:46.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:45 smithi159 ceph-mon[35534]: pgmap v623: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 591 KiB/s wr, 46 op/s 2024-09-11T22:41:46.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:45 smithi172 ceph-mon[35461]: pgmap v623: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 591 KiB/s wr, 46 op/s 2024-09-11T22:41:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.186+0000 7fd05759e640 1 -- 172.21.15.5:0/2706243750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05810bff0 msgr2=0x7fd05810e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.186+0000 7fd05759e640 1 --2- 172.21.15.5:0/2706243750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05810bff0 0x7fd05810e460 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd050009ca0 tx=0x7fd05002a950 comp rx=0 tx=0).stop 2024-09-11T22:41:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.186+0000 7fd05759e640 1 -- 172.21.15.5:0/2706243750 shutdown_connections 2024-09-11T22:41:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.186+0000 7fd05759e640 1 --2- 172.21.15.5:0/2706243750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05810bff0 0x7fd05810e460 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.186+0000 7fd05759e640 1 --2- 172.21.15.5:0/2706243750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd058103530 0x7fd05810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.186+0000 7fd05759e640 1 --2- 172.21.15.5:0/2706243750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd058102c10 0x7fd058102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.186+0000 7fd05759e640 1 -- 172.21.15.5:0/2706243750 >> 172.21.15.5:0/2706243750 conn(0x7fd0580fc610 msgr2=0x7fd0580fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.186+0000 7fd05759e640 1 -- 172.21.15.5:0/2706243750 shutdown_connections 2024-09-11T22:41:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.186+0000 7fd05759e640 1 -- 172.21.15.5:0/2706243750 wait complete. 2024-09-11T22:41:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd05759e640 1 Processor -- start 2024-09-11T22:41:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd05759e640 1 -- start start 2024-09-11T22:41:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd05759e640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd058102c10 0x7fd0581982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd05759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd058103530 0x7fd0581987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd05759e640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd05810bff0 0x7fd05819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd05759e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd058111e10 con 0x7fd058103530 2024-09-11T22:41:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd05759e640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd058111c90 con 0x7fd05810bff0 2024-09-11T22:41:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd05759e640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd058111f90 con 0x7fd058102c10 2024-09-11T22:41:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd055d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd058103530 0x7fd0581987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd055d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd058103530 0x7fd0581987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57810/0 (socket says 172.21.15.5:57810) 2024-09-11T22:41:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd055d9b640 1 -- 172.21.15.5:0/1550651702 learned_addr learned my addr 172.21.15.5:0/1550651702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd05659c640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd058102c10 0x7fd0581982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd056d9d640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd05810bff0 0x7fd05819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd055d9b640 1 -- 172.21.15.5:0/1550651702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd058102c10 msgr2=0x7fd0581982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd055d9b640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd058102c10 0x7fd0581982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd055d9b640 1 -- 172.21.15.5:0/1550651702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd05810bff0 msgr2=0x7fd05819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd055d9b640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd05810bff0 0x7fd05819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd055d9b640 1 -- 172.21.15.5:0/1550651702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05819d260 con 0x7fd058103530 2024-09-11T22:41:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.187+0000 7fd056d9d640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd05810bff0 0x7fd05819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.188+0000 7fd055d9b640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd058103530 0x7fd0581987f0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00c890 tx=0x7fd04c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.188+0000 7fd0477fe640 1 -- 172.21.15.5:0/1550651702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c004220 con 0x7fd058103530 2024-09-11T22:41:46.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.188+0000 7fd05759e640 1 -- 172.21.15.5:0/1550651702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05819d550 con 0x7fd058103530 2024-09-11T22:41:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.188+0000 7fd0477fe640 1 -- 172.21.15.5:0/1550651702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd04c0043c0 con 0x7fd058103530 2024-09-11T22:41:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.188+0000 7fd05759e640 1 -- 172.21.15.5:0/1550651702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0581a4e30 con 0x7fd058103530 2024-09-11T22:41:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.188+0000 7fd0477fe640 1 -- 172.21.15.5:0/1550651702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c01d060 con 0x7fd058103530 2024-09-11T22:41:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.190+0000 7fd0477fe640 1 -- 172.21.15.5:0/1550651702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fd04c0079b0 con 0x7fd058103530 2024-09-11T22:41:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.190+0000 7fd05f1d8740 1 Processor -- start 2024-09-11T22:41:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd05f1d8740 1 -- start start 2024-09-11T22:41:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619b95a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b95fd0 0x563619b9a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619b9b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563619af12f0 con 0x563619b9ac20 2024-09-11T22:41:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563619a98cf0 con 0x563619b95620 2024-09-11T22:41:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563619aed2b0 con 0x563619b95fd0 2024-09-11T22:41:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd055d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619b9b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd055d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619b9b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57824/0 (socket says 172.21.15.5:57824) 2024-09-11T22:41:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd055d9b640 1 -- 172.21.15.5:0/2751272300 learned_addr learned my addr 172.21.15.5:0/2751272300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:46.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd0477fe640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0400a2cf0 0x7fd0400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd056d9d640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0400a2cf0 0x7fd0400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.191+0000 7fd056d9d640 1 --2- 172.21.15.5:0/2751272300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619b95a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd0477fe640 1 -- 172.21.15.5:0/1550651702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7fd04c0cf550 con 0x7fd058103530 2024-09-11T22:41:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd055d9b640 1 -- 172.21.15.5:0/2751272300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b95fd0 msgr2=0x563619b9a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd055d9b640 1 --2- 172.21.15.5:0/2751272300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b95fd0 0x563619b9a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd055d9b640 1 -- 172.21.15.5:0/2751272300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 msgr2=0x563619b95a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd055d9b640 1 --2- 172.21.15.5:0/2751272300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619b95a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd055d9b640 1 -- 172.21.15.5:0/2751272300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563619a7d3a0 con 0x563619b9ac20 2024-09-11T22:41:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd055d9b640 1 --2- 172.21.15.5:0/2751272300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619b9b0d0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fd04c06a280 tx=0x7fd04c005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15cb2647414fff2d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd056d9d640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0400a2cf0 0x7fd0400a51b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd050004af0 tx=0x7fd050004a40 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd044ff9640 1 -- 172.21.15.5:0/2751272300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c09b360 con 0x563619b9ac20 2024-09-11T22:41:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd044ff9640 1 -- 172.21.15.5:0/2751272300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd04c09c100 con 0x563619b9ac20 2024-09-11T22:41:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.192+0000 7fd044ff9640 1 -- 172.21.15.5:0/2751272300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c098070 con 0x563619b9ac20 2024-09-11T22:41:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2751272300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 msgr2=0x563619b9b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2751272300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619b9b0d0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fd04c06a280 tx=0x7fd04c005300 comp rx=0 tx=0).stop 2024-09-11T22:41:46.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2751272300 shutdown_connections 2024-09-11T22:41:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2751272300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619b9b0d0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2751272300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b95fd0 0x563619b9a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2751272300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619b95a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2751272300 >> 172.21.15.5:0/2751272300 conn(0x563619b54310 msgr2=0x563619b878c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2751272300 shutdown_connections 2024-09-11T22:41:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2751272300 wait complete. 2024-09-11T22:41:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 -- start start 2024-09-11T22:41:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.193+0000 7fd05f1d8740 1 Processor -- start 2024-09-11T22:41:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619ba10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95fd0 0x563619b9e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b9ac20 0x563619b9e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563619ba1a50 con 0x563619b95fd0 2024-09-11T22:41:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563619aac060 con 0x563619b95620 2024-09-11T22:41:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x563619ba1bd0 con 0x563619b9ac20 2024-09-11T22:41:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd05f1d8740 10 client.0 _ll_register_callbacks cb 0x5636197ff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd055d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95fd0 0x563619b9e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd055d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95fd0 0x563619b9e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57834/0 (socket says 172.21.15.5:57834) 2024-09-11T22:41:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd055d9b640 1 -- 172.21.15.5:0/2526642235 learned_addr learned my addr 172.21.15.5:0/2526642235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd05659c640 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619ba10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd055d9b640 1 -- 172.21.15.5:0/2526642235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b9ac20 msgr2=0x563619b9e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd055d9b640 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b9ac20 0x563619b9e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd055d9b640 1 -- 172.21.15.5:0/2526642235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 msgr2=0x563619ba10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd055d9b640 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619ba10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd055d9b640 1 -- 172.21.15.5:0/2526642235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636198623c0 con 0x563619b95fd0 2024-09-11T22:41:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd055d9b640 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95fd0 0x563619b9e350 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0a4140 tx=0x7fd04c0cf4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4d364c7121c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd03dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd03dffb640 1 -- 172.21.15.5:0/2526642235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c09fdb0 con 0x563619b95fd0 2024-09-11T22:41:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd03dffb640 1 -- 172.21.15.5:0/2526642235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7fd04c0a41e0 con 0x563619b95fd0 2024-09-11T22:41:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.194+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2526642235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd04c09c100 con 0x563619b95fd0 2024-09-11T22:41:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd03dffb640 1 -- 172.21.15.5:0/2526642235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fd04c0a4670 con 0x563619b95fd0 2024-09-11T22:41:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd05f1d8740 10 client.21306 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2526642235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x563619bad4a0 con 0x563619b95fd0 2024-09-11T22:41:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd03dffb640 1 -- 172.21.15.5:0/2526642235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd04c09c100 con 0x563619b95fd0 2024-09-11T22:41:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd03dffb640 1 -- 172.21.15.5:0/2526642235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fd04c097750 con 0x563619b95fd0 2024-09-11T22:41:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd05f1d8740 10 client.21306 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd05f1d8740 10 client.21306 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:41:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd05f1d8740 20 client.21306 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x563619bae7f0 0x563619bb0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd05f1d8740 4 client.21306 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2526642235 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563619afe860 con 0x563619bae7f0 2024-09-11T22:41:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.195+0000 7fd05659c640 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x563619bae7f0 0x563619bb0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.196+0000 7fd03dffb640 10 client.21306 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:41:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.196+0000 7fd05659c640 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x563619bae7f0 0x563619bb0be0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21938a2c345b1a75 server_cookie=47cc35ce409147ca in_seq=0 out_seq=0 2024-09-11T22:41:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd03dffb640 1 -- 172.21.15.5:0/2526642235 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563619afe860 con 0x563619bae7f0 2024-09-11T22:41:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd03dffb640 10 client.21306 handle_command_reply: tid=0 2024-09-11T22:41:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 client.21306 shutdown 2024-09-11T22:41:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2526642235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95fd0 msgr2=0x563619b9e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95fd0 0x563619b9e350 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0a4140 tx=0x7fd04c0cf4c0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2526642235 shutdown_connections 2024-09-11T22:41:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b9ac20 0x563619b9e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95fd0 0x563619b9e350 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x563619bae7f0 0x563619bb0be0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2526642235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619ba10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd03dffb640 0 client.21306 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:41:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2526642235 >> 172.21.15.5:0/2526642235 conn(0x563619b54310 msgr2=0x563619b94c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2526642235 shutdown_connections 2024-09-11T22:41:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2526642235 wait complete. 2024-09-11T22:41:46.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.197+0000 7fd05f1d8740 20 client.21306 trim_cache size 0 max 16384 2024-09-11T22:41:46.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd05f1d8740 1 Processor -- start 2024-09-11T22:41:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd05f1d8740 1 -- start start 2024-09-11T22:41:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619b9a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b95fd0 0x563619ba02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619bb3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581ab760 con 0x563619b9ac20 2024-09-11T22:41:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581ad390 con 0x563619b95620 2024-09-11T22:41:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581abe30 con 0x563619b95fd0 2024-09-11T22:41:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619bb3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd056d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619bb3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57850/0 (socket says 172.21.15.5:57850) 2024-09-11T22:41:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.210+0000 7fd056d9d640 1 -- 172.21.15.5:0/1308717875 learned_addr learned my addr 172.21.15.5:0/1308717875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd05659c640 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619b9a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd055d9b640 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b95fd0 0x563619ba02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd056d9d640 1 -- 172.21.15.5:0/1308717875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b95fd0 msgr2=0x563619ba02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd056d9d640 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b95fd0 0x563619ba02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd056d9d640 1 -- 172.21.15.5:0/1308717875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 msgr2=0x563619b9a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd056d9d640 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619b9a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd056d9d640 1 -- 172.21.15.5:0/1308717875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563619b87e00 con 0x563619b9ac20 2024-09-11T22:41:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd055d9b640 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b95fd0 0x563619ba02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd05659c640 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619b9a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd056d9d640 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619bb3c00 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fd050003c90 tx=0x7fd05003d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586317e3c1dff52b server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd03ffff640 1 -- 172.21.15.5:0/1308717875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050034040 con 0x563619b9ac20 2024-09-11T22:41:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd03ffff640 1 -- 172.21.15.5:0/1308717875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd050003cc0 con 0x563619b9ac20 2024-09-11T22:41:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.211+0000 7fd03ffff640 1 -- 172.21.15.5:0/1308717875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050030860 con 0x563619b9ac20 2024-09-11T22:41:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 -- 172.21.15.5:0/1308717875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 msgr2=0x563619bb3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619bb3c00 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fd050003c90 tx=0x7fd05003d040 comp rx=0 tx=0).stop 2024-09-11T22:41:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 -- 172.21.15.5:0/1308717875 shutdown_connections 2024-09-11T22:41:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b9ac20 0x563619bb3c00 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b95fd0 0x563619ba02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/1308717875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95620 0x563619b9a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 -- 172.21.15.5:0/1308717875 >> 172.21.15.5:0/1308717875 conn(0x563619b54310 msgr2=0x563619b9fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 -- 172.21.15.5:0/1308717875 shutdown_connections 2024-09-11T22:41:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 -- 172.21.15.5:0/1308717875 wait complete. 2024-09-11T22:41:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 -- start start 2024-09-11T22:41:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.212+0000 7fd05f1d8740 1 Processor -- start 2024-09-11T22:41:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95620 0x563619bb1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95fd0 0x563619bb2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05f1d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b9ac20 0x563619bb25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581accc0 con 0x563619b95620 2024-09-11T22:41:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581ac530 con 0x563619b95fd0 2024-09-11T22:41:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05f1d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581a9760 con 0x563619b9ac20 2024-09-11T22:41:46.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05f1d8740 10 client.0 _ll_register_callbacks cb 0x7fd04c0058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:46.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95620 0x563619bb1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95620 0x563619bb1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57856/0 (socket says 172.21.15.5:57856) 2024-09-11T22:41:46.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05659c640 1 -- 172.21.15.5:0/2197519399 learned_addr learned my addr 172.21.15.5:0/2197519399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:46.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd056d9d640 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b9ac20 0x563619bb25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd055d9b640 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95fd0 0x563619bb2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05659c640 1 -- 172.21.15.5:0/2197519399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b9ac20 msgr2=0x563619bb25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:46.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05659c640 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b9ac20 0x563619bb25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05659c640 1 -- 172.21.15.5:0/2197519399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95fd0 msgr2=0x563619bb2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05659c640 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95fd0 0x563619bb2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd05659c640 1 -- 172.21.15.5:0/2197519399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563619b9ec60 con 0x563619b95620 2024-09-11T22:41:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.213+0000 7fd055d9b640 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95fd0 0x563619bb2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd05659c640 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95620 0x563619bb1b20 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fd0480058c0 tx=0x7fd048012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4ae30a3dd72402 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd03dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd03dffb640 1 -- 172.21.15.5:0/2197519399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04801f020 con 0x563619b95620 2024-09-11T22:41:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd03dffb640 1 -- 172.21.15.5:0/2197519399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7fd048004440 con 0x563619b95620 2024-09-11T22:41:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2197519399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd050003cc0 con 0x563619b95620 2024-09-11T22:41:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd03dffb640 1 -- 172.21.15.5:0/2197519399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fd04802a050 con 0x563619b95620 2024-09-11T22:41:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd05f1d8740 10 client.21312 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2197519399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x563619c19350 con 0x563619b95620 2024-09-11T22:41:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd03dffb640 1 -- 172.21.15.5:0/2197519399 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd050003cc0 con 0x563619b95620 2024-09-11T22:41:46.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd03dffb640 1 -- 172.21.15.5:0/2197519399 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7fd04801b600 con 0x563619b95620 2024-09-11T22:41:46.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd05f1d8740 10 client.21312 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:46.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd05f1d8740 10 client.21312 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.e 2024-09-11T22:41:46.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.214+0000 7fd05f1d8740 20 client.21312 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:46.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.215+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x563619bae7f0 0x563619c1bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:46.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.215+0000 7fd05f1d8740 4 client.21312 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:41:46.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.215+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2197519399 --> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563619af8d70 con 0x563619bae7f0 2024-09-11T22:41:46.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.215+0000 7fd055d9b640 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x563619bae7f0 0x563619c1bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.215+0000 7fd055d9b640 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x563619bae7f0 0x563619c1bae0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=155822464504536d server_cookie=be318c8fd54321e8 in_seq=0 out_seq=0 2024-09-11T22:41:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.215+0000 7fd03dffb640 10 client.21312 ms_handle_connect on v2:172.21.15.159:6834/1262598700 2024-09-11T22:41:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.215+0000 7fd03dffb640 1 -- 172.21.15.5:0/2197519399 <== mds.3 v2:172.21.15.159:6834/1262598700 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563619af8d70 con 0x563619bae7f0 2024-09-11T22:41:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.215+0000 7fd03dffb640 10 client.21312 handle_command_reply: tid=0 2024-09-11T22:41:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.215+0000 7fd05f1d8740 1 client.21312 shutdown 2024-09-11T22:41:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2197519399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95620 msgr2=0x563619bb1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95620 0x563619bb1b20 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fd0480058c0 tx=0x7fd048012820 comp rx=0 tx=0).stop 2024-09-11T22:41:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2197519399 shutdown_connections 2024-09-11T22:41:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563619b9ac20 0x563619bb25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.159:6834/1262598700,v1:172.21.15.159:6835/1262598700] conn(0x563619bae7f0 0x563619c1bae0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563619b95fd0 0x563619bb2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 1 --2- 172.21.15.5:0/2197519399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563619b95620 0x563619bb1b20 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2197519399 >> 172.21.15.5:0/2197519399 conn(0x563619b54310 msgr2=0x563619b94c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd03dffb640 0 client.21312 ms_handle_reset on v2:172.21.15.159:6834/1262598700 2024-09-11T22:41:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2197519399 shutdown_connections 2024-09-11T22:41:46.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 1 -- 172.21.15.5:0/2197519399 wait complete. 2024-09-11T22:41:46.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05f1d8740 20 client.21312 trim_cache size 0 max 16384 2024-09-11T22:41:46.235 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:41:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05759e640 1 -- 172.21.15.5:0/1550651702 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0400a2cf0 msgr2=0x7fd0400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05759e640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0400a2cf0 0x7fd0400a51b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd050004af0 tx=0x7fd050004a40 comp rx=0 tx=0).stop 2024-09-11T22:41:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05759e640 1 -- 172.21.15.5:0/1550651702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd058103530 msgr2=0x7fd0581987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.216+0000 7fd05759e640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd058103530 0x7fd0581987f0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00c890 tx=0x7fd04c00cd50 comp rx=0 tx=0).stop 2024-09-11T22:41:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.217+0000 7fd05759e640 1 -- 172.21.15.5:0/1550651702 shutdown_connections 2024-09-11T22:41:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.217+0000 7fd05759e640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fd0400a2cf0 0x7fd0400a51b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.217+0000 7fd05759e640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd05810bff0 0x7fd05819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.217+0000 7fd05759e640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd058103530 0x7fd0581987f0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.217+0000 7fd05759e640 1 --2- 172.21.15.5:0/1550651702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd058102c10 0x7fd0581982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.217+0000 7fd05759e640 1 -- 172.21.15.5:0/1550651702 >> 172.21.15.5:0/1550651702 conn(0x7fd0580fc610 msgr2=0x7fd0580fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.217+0000 7fd05759e640 1 -- 172.21.15.5:0/1550651702 shutdown_connections 2024-09-11T22:41:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:46.217+0000 7fd05759e640 1 -- 172.21.15.5:0/1550651702 wait complete. 2024-09-11T22:41:47.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-11T22:41:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:47.738 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:47 smithi005 ceph-mon[30943]: pgmap v624: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 128 KiB/s wr, 17 op/s 2024-09-11T22:41:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.820+0000 7f8168c41640 1 -- 172.21.15.5:0/4012983385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8164102d40 msgr2=0x7f8164103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.820+0000 7f8168c41640 1 --2- 172.21.15.5:0/4012983385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8164102d40 0x7f8164103120 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f8154009900 tx=0x7f815402a480 comp rx=0 tx=0).stop 2024-09-11T22:41:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.820+0000 7f8168c41640 1 -- 172.21.15.5:0/4012983385 shutdown_connections 2024-09-11T22:41:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.820+0000 7f8168c41640 1 --2- 172.21.15.5:0/4012983385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f816410c170 0x7f816410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.820+0000 7f8168c41640 1 --2- 172.21.15.5:0/4012983385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8164103660 0x7f816410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.820+0000 7f8168c41640 1 --2- 172.21.15.5:0/4012983385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8164102d40 0x7f8164103120 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.820+0000 7f8168c41640 1 -- 172.21.15.5:0/4012983385 >> 172.21.15.5:0/4012983385 conn(0x7f81640fc7a0 msgr2=0x7f81640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.820+0000 7f8168c41640 1 -- 172.21.15.5:0/4012983385 shutdown_connections 2024-09-11T22:41:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.820+0000 7f8168c41640 1 -- 172.21.15.5:0/4012983385 wait complete. 2024-09-11T22:41:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.821+0000 7f8168c41640 1 Processor -- start 2024-09-11T22:41:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.821+0000 7f8168c41640 1 -- start start 2024-09-11T22:41:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.821+0000 7f8168c41640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8164102d40 0x7f8164198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.821+0000 7f8168c41640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8164103660 0x7f8164198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.821+0000 7f8168c41640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f816410c170 0x7f816419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.821+0000 7f8168c41640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8164111f70 con 0x7f816410c170 2024-09-11T22:41:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.821+0000 7f8168c41640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8164111df0 con 0x7f8164103660 2024-09-11T22:41:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.821+0000 7f8168c41640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81641120f0 con 0x7f8164102d40 2024-09-11T22:41:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8162d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f816410c170 0x7f816419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8162d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f816410c170 0x7f816419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57876/0 (socket says 172.21.15.5:57876) 2024-09-11T22:41:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8162d9d640 1 -- 172.21.15.5:0/685958045 learned_addr learned my addr 172.21.15.5:0/685958045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f816259c640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8164102d40 0x7f8164198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f815bfff640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8164103660 0x7f8164198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8162d9d640 1 -- 172.21.15.5:0/685958045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8164102d40 msgr2=0x7f8164198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8162d9d640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8164102d40 0x7f8164198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8162d9d640 1 -- 172.21.15.5:0/685958045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8164103660 msgr2=0x7f8164198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8162d9d640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8164103660 0x7f8164198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8162d9d640 1 -- 172.21.15.5:0/685958045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816419d3e0 con 0x7f816410c170 2024-09-11T22:41:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f815bfff640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8164103660 0x7f8164198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8162d9d640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f816410c170 0x7f816419cd00 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f815c00cad0 tx=0x7f815c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f815b7fe640 1 -- 172.21.15.5:0/685958045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c00ae20 con 0x7f816410c170 2024-09-11T22:41:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f815b7fe640 1 -- 172.21.15.5:0/685958045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f815c005020 con 0x7f816410c170 2024-09-11T22:41:47.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f815b7fe640 1 -- 172.21.15.5:0/685958045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815c00f6f0 con 0x7f816410c170 2024-09-11T22:41:47.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8168c41640 1 -- 172.21.15.5:0/685958045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816419d6d0 con 0x7f816410c170 2024-09-11T22:41:47.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.822+0000 7f8168c41640 1 -- 172.21.15.5:0/685958045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81641a4fb0 con 0x7f816410c170 2024-09-11T22:41:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f815b7fe640 1 -- 172.21.15.5:0/685958045 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f815c0051d0 con 0x7f816410c170 2024-09-11T22:41:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816b01a740 1 Processor -- start 2024-09-11T22:41:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816b01a740 1 -- start start 2024-09-11T22:41:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816b01a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6b0b40 0x55633a6af090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816b01a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816b01a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6be780 0x55633a6c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816b01a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55633a6152f0 con 0x55633a6af5d0 2024-09-11T22:41:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816b01a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55633a5bccf0 con 0x55633a6b0b40 2024-09-11T22:41:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816b01a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55633a6112b0 con 0x55633a6be780 2024-09-11T22:41:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57884/0 (socket says 172.21.15.5:57884) 2024-09-11T22:41:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.825+0000 7f816259c640 1 -- 172.21.15.5:0/70521132 learned_addr learned my addr 172.21.15.5:0/70521132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f815bfff640 1 --2- 172.21.15.5:0/70521132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6b0b40 0x55633a6af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f815bfff640 1 --2- 172.21.15.5:0/70521132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6be780 0x55633a6c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f816259c640 1 -- 172.21.15.5:0/70521132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6be780 msgr2=0x55633a6c0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f816259c640 1 --2- 172.21.15.5:0/70521132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6be780 0x55633a6c0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f816259c640 1 -- 172.21.15.5:0/70521132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6b0b40 msgr2=0x55633a6af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f816259c640 1 --2- 172.21.15.5:0/70521132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6b0b40 0x55633a6af090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f815b7fe640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81500a2cf0 0x7f81500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f816259c640 1 -- 172.21.15.5:0/70521132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55633a5a13a0 con 0x55633a6af5d0 2024-09-11T22:41:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f8162d9d640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81500a2cf0 0x7f81500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f816259c640 1 --2- 172.21.15.5:0/70521132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6afa10 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f8154033000 tx=0x7f8154032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ed440df66c40f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f815b7fe640 1 -- 172.21.15.5:0/685958045 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f815c0cf5a0 con 0x7f816410c170 2024-09-11T22:41:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f8158ff9640 1 -- 172.21.15.5:0/70521132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815403f070 con 0x55633a6af5d0 2024-09-11T22:41:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f8158ff9640 1 -- 172.21.15.5:0/70521132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81540317b0 con 0x55633a6af5d0 2024-09-11T22:41:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.826+0000 7f8158ff9640 1 -- 172.21.15.5:0/70521132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154035070 con 0x55633a6af5d0 2024-09-11T22:41:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.827+0000 7f816b01a740 1 -- 172.21.15.5:0/70521132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 msgr2=0x55633a6afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.827+0000 7f8162d9d640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81500a2cf0 0x7f81500a51b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f815c0ccd30 tx=0x7f815c084f90 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.827+0000 7f816b01a740 1 --2- 172.21.15.5:0/70521132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6afa10 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f8154033000 tx=0x7f8154032ce0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.827+0000 7f816b01a740 1 -- 172.21.15.5:0/70521132 shutdown_connections 2024-09-11T22:41:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.827+0000 7f816b01a740 1 --2- 172.21.15.5:0/70521132 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6be780 0x55633a6c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.827+0000 7f816b01a740 1 --2- 172.21.15.5:0/70521132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6afa10 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.827+0000 7f816b01a740 1 --2- 172.21.15.5:0/70521132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6b0b40 0x55633a6af090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.827+0000 7f816b01a740 1 -- 172.21.15.5:0/70521132 >> 172.21.15.5:0/70521132 conn(0x55633a678310 msgr2=0x55633a6abc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.827+0000 7f816b01a740 1 -- 172.21.15.5:0/70521132 shutdown_connections 2024-09-11T22:41:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.827+0000 7f816b01a740 1 -- 172.21.15.5:0/70521132 wait complete. 2024-09-11T22:41:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816b01a740 1 -- start start 2024-09-11T22:41:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816b01a740 1 Processor -- start 2024-09-11T22:41:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816b01a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6af5d0 0x55633a6be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816b01a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6b0b40 0x55633a6bb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816b01a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 0x55633a6bb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816b01a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55633a6c1420 con 0x55633a6b0b40 2024-09-11T22:41:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816b01a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55633a5d0060 con 0x55633a6be780 2024-09-11T22:41:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816b01a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x55633a6c15a0 con 0x55633a6af5d0 2024-09-11T22:41:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816b01a740 10 client.0 _ll_register_callbacks cb 0x55633a3238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6b0b40 0x55633a6bb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6b0b40 0x55633a6bb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57894/0 (socket says 172.21.15.5:57894) 2024-09-11T22:41:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816259c640 1 -- 172.21.15.5:0/4055290647 learned_addr learned my addr 172.21.15.5:0/4055290647 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f815bfff640 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 0x55633a6bb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816259c640 1 -- 172.21.15.5:0/4055290647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6af5d0 msgr2=0x55633a6be150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816259c640 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6af5d0 0x55633a6be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816259c640 1 -- 172.21.15.5:0/4055290647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 msgr2=0x55633a6bb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816259c640 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 0x55633a6bb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f816259c640 1 -- 172.21.15.5:0/4055290647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55633a3863c0 con 0x55633a6b0b40 2024-09-11T22:41:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.828+0000 7f815bfff640 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 0x55633a6bb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f816259c640 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6b0b40 0x55633a6bb400 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f8154005270 tx=0x7f8154032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66aacc27140508 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f8149ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f8149ffb640 1 -- 172.21.15.5:0/4055290647 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815404b060 con 0x55633a6b0b40 2024-09-11T22:41:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f8149ffb640 1 -- 172.21.15.5:0/4055290647 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f81540035f0 con 0x55633a6b0b40 2024-09-11T22:41:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f816b01a740 1 -- 172.21.15.5:0/4055290647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81540317b0 con 0x55633a6b0b40 2024-09-11T22:41:47.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f8149ffb640 1 -- 172.21.15.5:0/4055290647 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f815403f030 con 0x55633a6b0b40 2024-09-11T22:41:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f816b01a740 10 client.21324 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f816b01a740 1 -- 172.21.15.5:0/4055290647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55633a6cce90 con 0x55633a6b0b40 2024-09-11T22:41:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f8149ffb640 1 -- 172.21.15.5:0/4055290647 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81540317b0 con 0x55633a6b0b40 2024-09-11T22:41:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f8149ffb640 1 -- 172.21.15.5:0/4055290647 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f8154004aa0 con 0x55633a6b0b40 2024-09-11T22:41:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f816b01a740 10 client.21324 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f816b01a740 10 client.21324 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:41:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f816b01a740 20 client.21324 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f816b01a740 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55633a6ce1e0 0x55633a6d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f816b01a740 4 client.21324 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.829+0000 7f816b01a740 1 -- 172.21.15.5:0/4055290647 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55633a622860 con 0x55633a6ce1e0 2024-09-11T22:41:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.830+0000 7f815bfff640 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55633a6ce1e0 0x55633a6d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.830+0000 7f815bfff640 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55633a6ce1e0 0x55633a6d05d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7122bd167acee84c server_cookie=d5993268231735a6 in_seq=0 out_seq=0 2024-09-11T22:41:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.830+0000 7f8149ffb640 10 client.21324 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:41:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f8149ffb640 1 -- 172.21.15.5:0/4055290647 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55633a622860 con 0x55633a6ce1e0 2024-09-11T22:41:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f8149ffb640 10 client.21324 handle_command_reply: tid=0 2024-09-11T22:41:47.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 client.21324 shutdown 2024-09-11T22:41:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 -- 172.21.15.5:0/4055290647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6b0b40 msgr2=0x55633a6bb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6b0b40 0x55633a6bb400 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f8154005270 tx=0x7f8154032f00 comp rx=0 tx=0).stop 2024-09-11T22:41:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 -- 172.21.15.5:0/4055290647 shutdown_connections 2024-09-11T22:41:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55633a6ce1e0 0x55633a6d05d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f8149ffb640 0 client.21324 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:41:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 0x55633a6bb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6b0b40 0x55633a6bb400 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 --2- 172.21.15.5:0/4055290647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6af5d0 0x55633a6be150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 -- 172.21.15.5:0/4055290647 >> 172.21.15.5:0/4055290647 conn(0x55633a678310 msgr2=0x55633a6b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 -- 172.21.15.5:0/4055290647 shutdown_connections 2024-09-11T22:41:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 1 -- 172.21.15.5:0/4055290647 wait complete. 2024-09-11T22:41:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.831+0000 7f816b01a740 20 client.21324 trim_cache size 0 max 16384 2024-09-11T22:41:47.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.844+0000 7f816b01a740 1 Processor -- start 2024-09-11T22:41:47.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.844+0000 7f816b01a740 1 -- start start 2024-09-11T22:41:47.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f816b01a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6af5d0 0x55633a6d35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f816b01a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6b0b40 0x55633a6d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f816b01a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6be780 0x55633a6b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f816b01a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81641ab810 con 0x55633a6be780 2024-09-11T22:41:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f816b01a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81641ad440 con 0x55633a6b0b40 2024-09-11T22:41:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f816b01a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81641abee0 con 0x55633a6af5d0 2024-09-11T22:41:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f815bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6be780 0x55633a6b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f815bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6be780 0x55633a6b86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57896/0 (socket says 172.21.15.5:57896) 2024-09-11T22:41:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f815bfff640 1 -- 172.21.15.5:0/3407755737 learned_addr learned my addr 172.21.15.5:0/3407755737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:47.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f816259c640 1 --2- 172.21.15.5:0/3407755737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6b0b40 0x55633a6d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f815bfff640 1 -- 172.21.15.5:0/3407755737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6af5d0 msgr2=0x55633a6d35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f815bfff640 1 --2- 172.21.15.5:0/3407755737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6af5d0 0x55633a6d35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f815bfff640 1 -- 172.21.15.5:0/3407755737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6b0b40 msgr2=0x55633a6d3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f815bfff640 1 --2- 172.21.15.5:0/3407755737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6b0b40 0x55633a6d3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f815bfff640 1 -- 172.21.15.5:0/3407755737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55633a6b9990 con 0x55633a6be780 2024-09-11T22:41:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f815bfff640 1 --2- 172.21.15.5:0/3407755737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6be780 0x55633a6b86c0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f814c0058c0 tx=0x7f814c00bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb86ade09feb3396 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f814bfff640 1 -- 172.21.15.5:0/3407755737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c005220 con 0x55633a6be780 2024-09-11T22:41:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.845+0000 7f814bfff640 1 -- 172.21.15.5:0/3407755737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c004c80 con 0x55633a6be780 2024-09-11T22:41:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f814bfff640 1 -- 172.21.15.5:0/3407755737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c020920 con 0x55633a6be780 2024-09-11T22:41:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f816b01a740 1 -- 172.21.15.5:0/3407755737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6be780 msgr2=0x55633a6b86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f816b01a740 1 --2- 172.21.15.5:0/3407755737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6be780 0x55633a6b86c0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f814c0058c0 tx=0x7f814c00bb90 comp rx=0 tx=0).stop 2024-09-11T22:41:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f816b01a740 1 -- 172.21.15.5:0/3407755737 shutdown_connections 2024-09-11T22:41:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f816b01a740 1 --2- 172.21.15.5:0/3407755737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6be780 0x55633a6b86c0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f816b01a740 1 --2- 172.21.15.5:0/3407755737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6b0b40 0x55633a6d3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f816b01a740 1 --2- 172.21.15.5:0/3407755737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6af5d0 0x55633a6d35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f816b01a740 1 -- 172.21.15.5:0/3407755737 >> 172.21.15.5:0/3407755737 conn(0x55633a678310 msgr2=0x55633a6a6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f816b01a740 1 -- 172.21.15.5:0/3407755737 shutdown_connections 2024-09-11T22:41:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f816b01a740 1 -- 172.21.15.5:0/3407755737 wait complete. 2024-09-11T22:41:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.846+0000 7f816b01a740 1 -- start start 2024-09-11T22:41:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f816b01a740 1 Processor -- start 2024-09-11T22:41:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f816b01a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6d39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f816b01a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6b0b40 0x55633a6d3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f816b01a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 0x55633a6d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f816b01a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81641acd70 con 0x55633a6af5d0 2024-09-11T22:41:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f816b01a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81641ac5e0 con 0x55633a6be780 2024-09-11T22:41:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f816b01a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f81641a9810 con 0x55633a6b0b40 2024-09-11T22:41:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f816b01a740 10 client.0 _ll_register_callbacks cb 0x7f816406b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f815bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6d39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f815bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6d39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57906/0 (socket says 172.21.15.5:57906) 2024-09-11T22:41:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f815bfff640 1 -- 172.21.15.5:0/1527776181 learned_addr learned my addr 172.21.15.5:0/1527776181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f816259c640 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6b0b40 0x55633a6d3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f815bfff640 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 0x55633a6d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f815bfff640 1 -- 172.21.15.5:0/1527776181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6b0b40 msgr2=0x55633a6d3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f815bfff640 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6b0b40 0x55633a6d3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f815bfff640 1 -- 172.21.15.5:0/1527776181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 msgr2=0x55633a6d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f815bfff640 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 0x55633a6d0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f815bfff640 1 -- 172.21.15.5:0/1527776181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55633a6d17c0 con 0x55633a6af5d0 2024-09-11T22:41:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f815bfff640 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6d39f0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f814c004700 tx=0x7f814c020500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ba5dc78ddcf0ae server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.847+0000 7f8149ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f8149ffb640 1 -- 172.21.15.5:0/1527776181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c005410 con 0x55633a6af5d0 2024-09-11T22:41:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f8149ffb640 1 -- 172.21.15.5:0/1527776181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f814c0055b0 con 0x55633a6af5d0 2024-09-11T22:41:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f816b01a740 1 -- 172.21.15.5:0/1527776181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f814c004c80 con 0x55633a6af5d0 2024-09-11T22:41:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f8149ffb640 1 -- 172.21.15.5:0/1527776181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f814c004aa0 con 0x55633a6af5d0 2024-09-11T22:41:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f816b01a740 10 client.21330 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f816b01a740 1 -- 172.21.15.5:0/1527776181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55633a7387d0 con 0x55633a6af5d0 2024-09-11T22:41:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f8149ffb640 1 -- 172.21.15.5:0/1527776181 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c004c80 con 0x55633a6af5d0 2024-09-11T22:41:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f8149ffb640 1 -- 172.21.15.5:0/1527776181 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f814c027030 con 0x55633a6af5d0 2024-09-11T22:41:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f816b01a740 10 client.21330 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f816b01a740 10 client.21330 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-11T22:41:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f816b01a740 20 client.21330 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f816b01a740 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55633a738de0 0x55633a73b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f816b01a740 4 client.21330 mds_command: new command op to 24454 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:41:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.848+0000 7f816b01a740 1 -- 172.21.15.5:0/1527776181 --> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55633a61cd70 con 0x55633a738de0 2024-09-11T22:41:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816259c640 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55633a738de0 0x55633a73b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816259c640 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55633a738de0 0x55633a73b200 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da4540be0829614d server_cookie=8a0d61a9c0da50a5 in_seq=0 out_seq=0 2024-09-11T22:41:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f8149ffb640 10 client.21330 ms_handle_connect on v2:172.21.15.172:6838/1937249261 2024-09-11T22:41:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f8149ffb640 1 -- 172.21.15.5:0/1527776181 <== mds.2 v2:172.21.15.172:6838/1937249261 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55633a61cd70 con 0x55633a738de0 2024-09-11T22:41:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f8149ffb640 10 client.21330 handle_command_reply: tid=0 2024-09-11T22:41:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 client.21330 shutdown 2024-09-11T22:41:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 -- 172.21.15.5:0/1527776181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 msgr2=0x55633a6d39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6d39f0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f814c004700 tx=0x7f814c020500 comp rx=0 tx=0).stop 2024-09-11T22:41:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 -- 172.21.15.5:0/1527776181 shutdown_connections 2024-09-11T22:41:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.172:6838/1937249261,v1:172.21.15.172:6839/1937249261] conn(0x55633a738de0 0x55633a73b200 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f8149ffb640 0 client.21330 ms_handle_reset on v2:172.21.15.172:6838/1937249261 2024-09-11T22:41:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633a6be780 0x55633a6d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55633a6b0b40 0x55633a6d3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 --2- 172.21.15.5:0/1527776181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633a6af5d0 0x55633a6d39f0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 -- 172.21.15.5:0/1527776181 >> 172.21.15.5:0/1527776181 conn(0x55633a678310 msgr2=0x55633a6b9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 -- 172.21.15.5:0/1527776181 shutdown_connections 2024-09-11T22:41:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 1 -- 172.21.15.5:0/1527776181 wait complete. 2024-09-11T22:41:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.849+0000 7f816b01a740 20 client.21330 trim_cache size 0 max 16384 2024-09-11T22:41:47.861 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:41:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 -- 172.21.15.5:0/685958045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81500a2cf0 msgr2=0x7f81500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81500a2cf0 0x7f81500a51b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f815c0ccd30 tx=0x7f815c084f90 comp rx=0 tx=0).stop 2024-09-11T22:41:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 -- 172.21.15.5:0/685958045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f816410c170 msgr2=0x7f816419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f816410c170 0x7f816419cd00 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f815c00cad0 tx=0x7f815c00cf90 comp rx=0 tx=0).stop 2024-09-11T22:41:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 -- 172.21.15.5:0/685958045 shutdown_connections 2024-09-11T22:41:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f816410c170 0x7f816419cd00 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f81500a2cf0 0x7f81500a51b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8164103660 0x7f8164198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 --2- 172.21.15.5:0/685958045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8164102d40 0x7f8164198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 -- 172.21.15.5:0/685958045 >> 172.21.15.5:0/685958045 conn(0x7f81640fc7a0 msgr2=0x7f81640fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 -- 172.21.15.5:0/685958045 shutdown_connections 2024-09-11T22:41:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:47.850+0000 7f8168c41640 1 -- 172.21.15.5:0/685958045 wait complete. 2024-09-11T22:41:48.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:47 smithi159 ceph-mon[35534]: pgmap v624: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 128 KiB/s wr, 17 op/s 2024-09-11T22:41:48.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:47 smithi172 ceph-mon[35461]: pgmap v624: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 128 KiB/s wr, 17 op/s 2024-09-11T22:41:48.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-11T22:41:48.951 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.489+0000 7f00eb529640 1 -- 172.21.15.5:0/813371990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00e4100350 msgr2=0x7f00e4100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.489+0000 7f00eb529640 1 --2- 172.21.15.5:0/813371990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00e4100350 0x7f00e4100730 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009930 tx=0x7f00e002a420 comp rx=0 tx=0).stop 2024-09-11T22:41:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.489+0000 7f00eb529640 1 -- 172.21.15.5:0/813371990 shutdown_connections 2024-09-11T22:41:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.489+0000 7f00eb529640 1 --2- 172.21.15.5:0/813371990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e4106f50 0x7f00e410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.489+0000 7f00eb529640 1 --2- 172.21.15.5:0/813371990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4100d00 0x7f00e4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.489+0000 7f00eb529640 1 --2- 172.21.15.5:0/813371990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00e4100350 0x7f00e4100730 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.489+0000 7f00eb529640 1 -- 172.21.15.5:0/813371990 >> 172.21.15.5:0/813371990 conn(0x7f00e4077210 msgr2=0x7f00e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:49.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.489+0000 7f00eb529640 1 -- 172.21.15.5:0/813371990 shutdown_connections 2024-09-11T22:41:49.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.489+0000 7f00eb529640 1 -- 172.21.15.5:0/813371990 wait complete. 2024-09-11T22:41:49.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00eb529640 1 Processor -- start 2024-09-11T22:41:49.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00eb529640 1 -- start start 2024-09-11T22:41:49.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00eb529640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e4100350 0x7f00e41012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00eb529640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00e4100d00 0x7f00e4101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00eb529640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4106f50 0x7f00e4101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00eb529640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4112000 con 0x7f00e4100d00 2024-09-11T22:41:49.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00eb529640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4111e80 con 0x7f00e4100350 2024-09-11T22:41:49.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00eb529640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00e4112180 con 0x7f00e4106f50 2024-09-11T22:41:49.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00e8ac4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00e4100d00 0x7f00e4101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00e8ac4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00e4100d00 0x7f00e4101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57928/0 (socket says 172.21.15.5:57928) 2024-09-11T22:41:49.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00e8ac4640 1 -- 172.21.15.5:0/2663521646 learned_addr learned my addr 172.21.15.5:0/2663521646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:49.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.490+0000 7f00e92c5640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e4100350 0x7f00e41012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00e9ac6640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4106f50 0x7f00e4101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00e8ac4640 1 -- 172.21.15.5:0/2663521646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4106f50 msgr2=0x7f00e4101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4106f50 0x7f00e4101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00e8ac4640 1 -- 172.21.15.5:0/2663521646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e4100350 msgr2=0x7f00e41012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e4100350 0x7f00e41012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00e8ac4640 1 -- 172.21.15.5:0/2663521646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e41a6a00 con 0x7f00e4100d00 2024-09-11T22:41:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00e4100d00 0x7f00e4101810 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f00d400b450 tx=0x7f00d400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00da7fc640 1 -- 172.21.15.5:0/2663521646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4015020 con 0x7f00e4100d00 2024-09-11T22:41:49.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00eb529640 1 -- 172.21.15.5:0/2663521646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e41a6cf0 con 0x7f00e4100d00 2024-09-11T22:41:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00da7fc640 1 -- 172.21.15.5:0/2663521646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00d4011d70 con 0x7f00e4100d00 2024-09-11T22:41:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00da7fc640 1 -- 172.21.15.5:0/2663521646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d400f340 con 0x7f00e4100d00 2024-09-11T22:41:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.491+0000 7f00eb529640 1 -- 172.21.15.5:0/2663521646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e41a71d0 con 0x7f00e4100d00 2024-09-11T22:41:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.493+0000 7f00da7fc640 1 -- 172.21.15.5:0/2663521646 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f00d4018020 con 0x7f00e4100d00 2024-09-11T22:41:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00ed902740 1 Processor -- start 2024-09-11T22:41:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00ed902740 1 -- start start 2024-09-11T22:41:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00ed902740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00ed902740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75a5d0 0x559c0e75aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00ed902740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e765370 0x559c0e767760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00ed902740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559c0e6c02f0 con 0x559c0e765370 2024-09-11T22:41:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00ed902740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559c0e667cf0 con 0x559c0e75bb40 2024-09-11T22:41:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00ed902740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559c0e6bc2b0 con 0x559c0e75a5d0 2024-09-11T22:41:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00e8ac4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e765370 0x559c0e767760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00da7fc640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00d00a2cf0 0x7f00d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00e9ac6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00e9ac6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42358/0 (socket says 172.21.15.5:42358) 2024-09-11T22:41:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00e9ac6640 1 -- 172.21.15.5:0/546173410 learned_addr learned my addr 172.21.15.5:0/546173410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:49.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00e8ac4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e765370 0x559c0e767760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57942/0 (socket says 172.21.15.5:57942) 2024-09-11T22:41:49.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00e92c5640 1 --2- 172.21.15.5:0/546173410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75a5d0 0x559c0e75aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.494+0000 7f00e9ac6640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00d00a2cf0 0x7f00d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00e9ac6640 1 -- 172.21.15.5:0/546173410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75a5d0 msgr2=0x559c0e75aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:49.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00e9ac6640 1 --2- 172.21.15.5:0/546173410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75a5d0 0x559c0e75aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00e9ac6640 1 -- 172.21.15.5:0/546173410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e765370 msgr2=0x559c0e767760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:49.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00e9ac6640 1 --2- 172.21.15.5:0/546173410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e765370 0x559c0e767760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00e9ac6640 1 -- 172.21.15.5:0/546173410 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0e64c3a0 con 0x559c0e75bb40 2024-09-11T22:41:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00da7fc640 1 -- 172.21.15.5:0/2663521646 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f00d40cee30 con 0x7f00e4100d00 2024-09-11T22:41:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00e9ac6640 1 --2- 172.21.15.5:0/546173410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75a090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0145e0 tx=0x7f00dc014aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4fe09de307e2a9a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00cf7fe640 1 -- 172.21.15.5:0/546173410 <== mon.2 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc022070 con 0x559c0e75bb40 2024-09-11T22:41:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00e9ac6640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00d00a2cf0 0x7f00d00a51b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f00dc01f000 tx=0x7f00dc01b000 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00cf7fe640 1 -- 172.21.15.5:0/546173410 <== mon.2 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00dc01dc00 con 0x559c0e75bb40 2024-09-11T22:41:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.495+0000 7f00cf7fe640 1 -- 172.21.15.5:0/546173410 <== mon.2 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00c360 con 0x559c0e75bb40 2024-09-11T22:41:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 -- 172.21.15.5:0/546173410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 msgr2=0x559c0e75a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 --2- 172.21.15.5:0/546173410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75a090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0145e0 tx=0x7f00dc014aa0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 -- 172.21.15.5:0/546173410 shutdown_connections 2024-09-11T22:41:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 --2- 172.21.15.5:0/546173410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e765370 0x559c0e767760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 --2- 172.21.15.5:0/546173410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75a5d0 0x559c0e75aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 --2- 172.21.15.5:0/546173410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75a090 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 -- 172.21.15.5:0/546173410 >> 172.21.15.5:0/546173410 conn(0x559c0e723310 msgr2=0x559c0e756c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 -- 172.21.15.5:0/546173410 shutdown_connections 2024-09-11T22:41:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 -- 172.21.15.5:0/546173410 wait complete. 2024-09-11T22:41:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 -- start start 2024-09-11T22:41:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.496+0000 7f00ed902740 1 Processor -- start 2024-09-11T22:41:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00ed902740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e764d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00ed902740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e762030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00ed902740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 0x559c0e762640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00ed902740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559c0e768070 con 0x559c0e75a5d0 2024-09-11T22:41:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00ed902740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559c0e67b060 con 0x559c0e75bb40 2024-09-11T22:41:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00ed902740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x559c0e7681f0 con 0x559c0e765370 2024-09-11T22:41:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00ed902740 10 client.0 _ll_register_callbacks cb 0x559c0e3ce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e92c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e764d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e92c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e764d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57954/0 (socket says 172.21.15.5:57954) 2024-09-11T22:41:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e92c5640 1 -- 172.21.15.5:0/3563498662 learned_addr learned my addr 172.21.15.5:0/3563498662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e762030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e9ac6640 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 0x559c0e762640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e92c5640 1 -- 172.21.15.5:0/3563498662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 msgr2=0x559c0e762640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e92c5640 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 0x559c0e762640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e92c5640 1 -- 172.21.15.5:0/3563498662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 msgr2=0x559c0e762030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e92c5640 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e762030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e92c5640 1 -- 172.21.15.5:0/3563498662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0e4313c0 con 0x559c0e75a5d0 2024-09-11T22:41:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00e92c5640 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e764d80 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f00e002a940 tx=0x7f00e0031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f7bce249458eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.497+0000 7f00ccff9640 1 -- 172.21.15.5:0/3563498662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0035040 con 0x559c0e75a5d0 2024-09-11T22:41:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ccff9640 1 -- 172.21.15.5:0/3563498662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f00e0004240 con 0x559c0e75a5d0 2024-09-11T22:41:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ed902740 1 -- 172.21.15.5:0/3563498662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00dc01dc00 con 0x559c0e75a5d0 2024-09-11T22:41:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ccff9640 1 -- 172.21.15.5:0/3563498662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f00e0004440 con 0x559c0e75a5d0 2024-09-11T22:41:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ed902740 10 client.21342 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ed902740 1 -- 172.21.15.5:0/3563498662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559c0e773b40 con 0x559c0e75a5d0 2024-09-11T22:41:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ccff9640 1 -- 172.21.15.5:0/3563498662 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00dc01dc00 con 0x559c0e75a5d0 2024-09-11T22:41:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ccff9640 1 -- 172.21.15.5:0/3563498662 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f00e003c7a0 con 0x559c0e75a5d0 2024-09-11T22:41:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ed902740 10 client.21342 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ed902740 10 client.21342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:41:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ed902740 20 client.21342 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ed902740 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559c0e775050 0x559c0e777460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ed902740 4 client.21342 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00ed902740 1 -- 172.21.15.5:0/3563498662 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c0e6cd860 con 0x559c0e775050 2024-09-11T22:41:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.498+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559c0e775050 0x559c0e777460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.499+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559c0e775050 0x559c0e777460 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47c36a993e0a6b6f server_cookie=c8d764adc6cb4127 in_seq=0 out_seq=0 2024-09-11T22:41:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.499+0000 7f00ccff9640 10 client.21342 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:41:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ccff9640 1 -- 172.21.15.5:0/3563498662 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c0e6cd860 con 0x559c0e775050 2024-09-11T22:41:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ccff9640 10 client.21342 handle_command_reply: tid=0 2024-09-11T22:41:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 client.21342 shutdown 2024-09-11T22:41:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 -- 172.21.15.5:0/3563498662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 msgr2=0x559c0e764d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e764d80 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f00e002a940 tx=0x7f00e0031920 comp rx=0 tx=0).stop 2024-09-11T22:41:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 -- 172.21.15.5:0/3563498662 shutdown_connections 2024-09-11T22:41:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559c0e775050 0x559c0e777460 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 0x559c0e762640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e762030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 --2- 172.21.15.5:0/3563498662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e764d80 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 -- 172.21.15.5:0/3563498662 >> 172.21.15.5:0/3563498662 conn(0x559c0e723310 msgr2=0x559c0e75c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ccff9640 0 client.21342 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:41:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 -- 172.21.15.5:0/3563498662 shutdown_connections 2024-09-11T22:41:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 1 -- 172.21.15.5:0/3563498662 wait complete. 2024-09-11T22:41:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.500+0000 7f00ed902740 20 client.21342 trim_cache size 0 max 16384 2024-09-11T22:41:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00ed902740 1 Processor -- start 2024-09-11T22:41:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00ed902740 1 -- start start 2024-09-11T22:41:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00ed902740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e760900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00ed902740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00ed902740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 0x559c0e75d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00ed902740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41afde0 con 0x559c0e75a5d0 2024-09-11T22:41:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00ed902740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41b1a10 con 0x559c0e75bb40 2024-09-11T22:41:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00ed902740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41b04b0 con 0x559c0e765370 2024-09-11T22:41:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00e92c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e760900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00e92c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e760900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57970/0 (socket says 172.21.15.5:57970) 2024-09-11T22:41:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00e92c5640 1 -- 172.21.15.5:0/2905207274 learned_addr learned my addr 172.21.15.5:0/2905207274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.513+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00e9ac6640 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 0x559c0e75d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00e92c5640 1 -- 172.21.15.5:0/2905207274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 msgr2=0x559c0e75d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00e92c5640 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 0x559c0e75d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00e92c5640 1 -- 172.21.15.5:0/2905207274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 msgr2=0x559c0e75d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00e92c5640 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00e92c5640 1 -- 172.21.15.5:0/2905207274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0e760fe0 con 0x559c0e75a5d0 2024-09-11T22:41:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00e9ac6640 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 0x559c0e75d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00e92c5640 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e760900 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f00e0031bf0 tx=0x7f00e0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f92dde0c7ffaeac server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00ceffd640 1 -- 172.21.15.5:0/2905207274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0035070 con 0x559c0e75a5d0 2024-09-11T22:41:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00ceffd640 1 -- 172.21.15.5:0/2905207274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e0005030 con 0x559c0e75a5d0 2024-09-11T22:41:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.514+0000 7f00ceffd640 1 -- 172.21.15.5:0/2905207274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e00049d0 con 0x559c0e75a5d0 2024-09-11T22:41:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 -- 172.21.15.5:0/2905207274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 msgr2=0x559c0e760900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e760900 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f00e0031bf0 tx=0x7f00e0000bc0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 -- 172.21.15.5:0/2905207274 shutdown_connections 2024-09-11T22:41:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e765370 0x559c0e75d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e75bb40 0x559c0e75d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 --2- 172.21.15.5:0/2905207274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e760900 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 -- 172.21.15.5:0/2905207274 >> 172.21.15.5:0/2905207274 conn(0x559c0e723310 msgr2=0x559c0e755d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 -- 172.21.15.5:0/2905207274 shutdown_connections 2024-09-11T22:41:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 -- 172.21.15.5:0/2905207274 wait complete. 2024-09-11T22:41:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 -- start start 2024-09-11T22:41:49.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.515+0000 7f00ed902740 1 Processor -- start 2024-09-11T22:41:49.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00ed902740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e769390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00ed902740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75bb40 0x559c0e7698d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00ed902740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e765370 0x559c0e769e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00ed902740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41b1340 con 0x559c0e75a5d0 2024-09-11T22:41:49.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00ed902740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41b0bb0 con 0x559c0e765370 2024-09-11T22:41:49.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00ed902740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f00e41adde0 con 0x559c0e75bb40 2024-09-11T22:41:49.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00ed902740 10 client.0 _ll_register_callbacks cb 0x7f00e0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:49.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e92c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e769390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e92c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e769390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57974/0 (socket says 172.21.15.5:57974) 2024-09-11T22:41:49.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e92c5640 1 -- 172.21.15.5:0/4252112864 learned_addr learned my addr 172.21.15.5:0/4252112864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e9ac6640 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e765370 0x559c0e769e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75bb40 0x559c0e7698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e92c5640 1 -- 172.21.15.5:0/4252112864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75bb40 msgr2=0x559c0e7698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e92c5640 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75bb40 0x559c0e7698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e92c5640 1 -- 172.21.15.5:0/4252112864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e765370 msgr2=0x559c0e769e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e92c5640 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e765370 0x559c0e769e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e92c5640 1 -- 172.21.15.5:0/4252112864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0e7def00 con 0x559c0e75a5d0 2024-09-11T22:41:49.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75bb40 0x559c0e7698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00e92c5640 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e769390 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009830 tx=0x7f00e002aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c75d384454cb31e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.516+0000 7f00ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ccff9640 1 -- 172.21.15.5:0/4252112864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0035070 con 0x559c0e75a5d0 2024-09-11T22:41:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ccff9640 1 -- 172.21.15.5:0/4252112864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f00e003c440 con 0x559c0e75a5d0 2024-09-11T22:41:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ed902740 1 -- 172.21.15.5:0/4252112864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e0005030 con 0x559c0e75a5d0 2024-09-11T22:41:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ccff9640 1 -- 172.21.15.5:0/4252112864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f00e003c660 con 0x559c0e75a5d0 2024-09-11T22:41:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ed902740 10 client.21348 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ed902740 1 -- 172.21.15.5:0/4252112864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559c0e7e00f0 con 0x559c0e75a5d0 2024-09-11T22:41:49.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ccff9640 1 -- 172.21.15.5:0/4252112864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e0005030 con 0x559c0e75a5d0 2024-09-11T22:41:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ccff9640 1 -- 172.21.15.5:0/4252112864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f00e00456c0 con 0x559c0e75a5d0 2024-09-11T22:41:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ed902740 10 client.21348 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ed902740 10 client.21348 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-11T22:41:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ed902740 20 client.21348 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ed902740 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559c0e7e0700 0x559c0e7e2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ed902740 4 client.21348 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:41:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.517+0000 7f00ed902740 1 -- 172.21.15.5:0/4252112864 --> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c0e6c7d70 con 0x559c0e7e0700 2024-09-11T22:41:49.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559c0e7e0700 0x559c0e7e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00e8ac4640 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559c0e7e0700 0x559c0e7e2b20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb4c0b9aedef6f4c server_cookie=799787f9b064013c in_seq=0 out_seq=0 2024-09-11T22:41:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ccff9640 10 client.21348 ms_handle_connect on v2:172.21.15.159:6836/2315339117 2024-09-11T22:41:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ccff9640 1 -- 172.21.15.5:0/4252112864 <== mds.1 v2:172.21.15.159:6836/2315339117 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c0e6c7d70 con 0x559c0e7e0700 2024-09-11T22:41:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ccff9640 10 client.21348 handle_command_reply: tid=0 2024-09-11T22:41:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ed902740 1 client.21348 shutdown 2024-09-11T22:41:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ed902740 1 -- 172.21.15.5:0/4252112864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 msgr2=0x559c0e769390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ed902740 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e769390 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009830 tx=0x7f00e002aae0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ed902740 1 -- 172.21.15.5:0/4252112864 shutdown_connections 2024-09-11T22:41:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ed902740 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.159:6836/2315339117,v1:172.21.15.159:6837/2315339117] conn(0x559c0e7e0700 0x559c0e7e2b20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ed902740 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c0e765370 0x559c0e769e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ed902740 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559c0e75bb40 0x559c0e7698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ed902740 1 --2- 172.21.15.5:0/4252112864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c0e75a5d0 0x559c0e769390 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ccff9640 0 client.21348 ms_handle_reset on v2:172.21.15.159:6836/2315339117 2024-09-11T22:41:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.518+0000 7f00ed902740 1 -- 172.21.15.5:0/4252112864 >> 172.21.15.5:0/4252112864 conn(0x559c0e723310 msgr2=0x559c0e755d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00ed902740 1 -- 172.21.15.5:0/4252112864 shutdown_connections 2024-09-11T22:41:49.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00ed902740 1 -- 172.21.15.5:0/4252112864 wait complete. 2024-09-11T22:41:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00ed902740 20 client.21348 trim_cache size 0 max 16384 2024-09-11T22:41:49.546 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:41:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 -- 172.21.15.5:0/2663521646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00d00a2cf0 msgr2=0x7f00d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00d00a2cf0 0x7f00d00a51b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f00dc01f000 tx=0x7f00dc01b000 comp rx=0 tx=0).stop 2024-09-11T22:41:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 -- 172.21.15.5:0/2663521646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00e4100d00 msgr2=0x7f00e4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00e4100d00 0x7f00e4101810 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f00d400b450 tx=0x7f00d400b910 comp rx=0 tx=0).stop 2024-09-11T22:41:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 -- 172.21.15.5:0/2663521646 shutdown_connections 2024-09-11T22:41:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00e4106f50 0x7f00e4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00e4100d00 0x7f00e4101810 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f00d00a2cf0 0x7f00d00a51b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 --2- 172.21.15.5:0/2663521646 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00e4100350 0x7f00e41012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:49.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 -- 172.21.15.5:0/2663521646 >> 172.21.15.5:0/2663521646 conn(0x7f00e4077210 msgr2=0x7f00e41052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 -- 172.21.15.5:0/2663521646 shutdown_connections 2024-09-11T22:41:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:49.519+0000 7f00eb529640 1 -- 172.21.15.5:0/2663521646 wait complete. 2024-09-11T22:41:50.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:49 smithi005 ceph-mon[30943]: pgmap v625: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 154 KiB/s wr, 24 op/s 2024-09-11T22:41:50.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:49 smithi159 ceph-mon[35534]: pgmap v625: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 154 KiB/s wr, 24 op/s 2024-09-11T22:41:50.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:49 smithi172 ceph-mon[35461]: pgmap v625: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 154 KiB/s wr, 24 op/s 2024-09-11T22:41:50.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-11T22:41:50.681 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.201+0000 7f1fcd076640 1 -- 172.21.15.5:0/167840936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc810c180 msgr2=0x7f1fc810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.201+0000 7f1fcd076640 1 --2- 172.21.15.5:0/167840936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc810c180 0x7f1fc810e610 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009ca0 tx=0x7f1fc002a950 comp rx=0 tx=0).stop 2024-09-11T22:41:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.202+0000 7f1fcd076640 1 -- 172.21.15.5:0/167840936 shutdown_connections 2024-09-11T22:41:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.202+0000 7f1fcd076640 1 --2- 172.21.15.5:0/167840936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc810c180 0x7f1fc810e610 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.202+0000 7f1fcd076640 1 --2- 172.21.15.5:0/167840936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc8103700 0x7f1fc810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.202+0000 7f1fcd076640 1 --2- 172.21.15.5:0/167840936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fc8102de0 0x7f1fc81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.202+0000 7f1fcd076640 1 -- 172.21.15.5:0/167840936 >> 172.21.15.5:0/167840936 conn(0x7f1fc80fc7a0 msgr2=0x7f1fc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.202+0000 7f1fcd076640 1 -- 172.21.15.5:0/167840936 shutdown_connections 2024-09-11T22:41:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.202+0000 7f1fcd076640 1 -- 172.21.15.5:0/167840936 wait complete. 2024-09-11T22:41:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.202+0000 7f1fcd076640 1 Processor -- start 2024-09-11T22:41:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fcd076640 1 -- start start 2024-09-11T22:41:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fcd076640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc8102de0 0x7f1fc8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fcd076640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fc8103700 0x7f1fc8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fcd076640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc810c180 0x7f1fc819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fcd076640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8112030 con 0x7f1fc8102de0 2024-09-11T22:41:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fcd076640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc8111eb0 con 0x7f1fc810c180 2024-09-11T22:41:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fcd076640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81121b0 con 0x7f1fc8103700 2024-09-11T22:41:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc8102de0 0x7f1fc8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc8102de0 0x7f1fc8198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57988/0 (socket says 172.21.15.5:57988) 2024-09-11T22:41:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc6d9d640 1 -- 172.21.15.5:0/2874731949 learned_addr learned my addr 172.21.15.5:0/2874731949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc659c640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fc8103700 0x7f1fc8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc759e640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc810c180 0x7f1fc819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc6d9d640 1 -- 172.21.15.5:0/2874731949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fc8103700 msgr2=0x7f1fc8198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc6d9d640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fc8103700 0x7f1fc8198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc6d9d640 1 -- 172.21.15.5:0/2874731949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc810c180 msgr2=0x7f1fc819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc6d9d640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc810c180 0x7f1fc819ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc6d9d640 1 -- 172.21.15.5:0/2874731949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc819d4f0 con 0x7f1fc8102de0 2024-09-11T22:41:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc659c640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fc8103700 0x7f1fc8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.203+0000 7f1fc6d9d640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc8102de0 0x7f1fc8198540 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800b4a0 tx=0x7f1fb800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.204+0000 7f1faffff640 1 -- 172.21.15.5:0/2874731949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8014070 con 0x7f1fc8102de0 2024-09-11T22:41:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.204+0000 7f1fcd076640 1 -- 172.21.15.5:0/2874731949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc819d7e0 con 0x7f1fc8102de0 2024-09-11T22:41:51.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.204+0000 7f1faffff640 1 -- 172.21.15.5:0/2874731949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb8010af0 con 0x7f1fc8102de0 2024-09-11T22:41:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.204+0000 7f1faffff640 1 -- 172.21.15.5:0/2874731949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8017d10 con 0x7f1fc8102de0 2024-09-11T22:41:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.204+0000 7f1fcd076640 1 -- 172.21.15.5:0/2874731949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc81a50c0 con 0x7f1fc8102de0 2024-09-11T22:41:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.206+0000 7f1faffff640 1 -- 172.21.15.5:0/2874731949 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1fb8017070 con 0x7f1fc8102de0 2024-09-11T22:41:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.206+0000 7f1fcf44f740 1 Processor -- start 2024-09-11T22:41:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.206+0000 7f1fcf44f740 1 -- start start 2024-09-11T22:41:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7dc370 0x56554d7de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56554d7372f0 con 0x56554d7d2b40 2024-09-11T22:41:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56554d6decf0 con 0x56554d7dc370 2024-09-11T22:41:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56554d7332b0 con 0x56554d7d15d0 2024-09-11T22:41:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57998/0 (socket says 172.21.15.5:57998) 2024-09-11T22:41:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc759e640 1 -- 172.21.15.5:0/2670986282 learned_addr learned my addr 172.21.15.5:0/2670986282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc6d9d640 1 --2- 172.21.15.5:0/2670986282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc659c640 1 --2- 172.21.15.5:0/2670986282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7dc370 0x56554d7de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc759e640 1 -- 172.21.15.5:0/2670986282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 msgr2=0x56554d7d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc759e640 1 --2- 172.21.15.5:0/2670986282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc759e640 1 -- 172.21.15.5:0/2670986282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7dc370 msgr2=0x56554d7de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc759e640 1 --2- 172.21.15.5:0/2670986282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7dc370 0x56554d7de760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc759e640 1 -- 172.21.15.5:0/2670986282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554d6c33a0 con 0x56554d7d2b40 2024-09-11T22:41:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1faffff640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1fb00a2cf0 0x7f1fb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc6d9d640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1fb00a2cf0 0x7f1fb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1fc759e640 1 --2- 172.21.15.5:0/2670986282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d1090 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009930 tx=0x7f1fc0009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa95401daf1b8c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:51.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.207+0000 7f1faffff640 1 -- 172.21.15.5:0/2874731949 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f1fb80ce830 con 0x7f1fc8102de0 2024-09-11T22:41:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fad7fa640 1 -- 172.21.15.5:0/2670986282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0034040 con 0x56554d7d2b40 2024-09-11T22:41:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fad7fa640 1 -- 172.21.15.5:0/2670986282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc0003970 con 0x56554d7d2b40 2024-09-11T22:41:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fad7fa640 1 -- 172.21.15.5:0/2670986282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0004610 con 0x56554d7d2b40 2024-09-11T22:41:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fcf44f740 1 -- 172.21.15.5:0/2670986282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 msgr2=0x56554d7d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fc6d9d640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1fb00a2cf0 0x7f1fb00a51b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80cc0d0 tx=0x7f1fb80d4550 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/2670986282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d1090 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009930 tx=0x7f1fc0009960 comp rx=0 tx=0).stop 2024-09-11T22:41:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fcf44f740 1 -- 172.21.15.5:0/2670986282 shutdown_connections 2024-09-11T22:41:51.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/2670986282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7dc370 0x56554d7de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/2670986282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/2670986282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d1090 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fcf44f740 1 -- 172.21.15.5:0/2670986282 >> 172.21.15.5:0/2670986282 conn(0x56554d79a310 msgr2=0x56554d7cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:51.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fcf44f740 1 -- 172.21.15.5:0/2670986282 shutdown_connections 2024-09-11T22:41:51.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.208+0000 7f1fcf44f740 1 -- 172.21.15.5:0/2670986282 wait complete. 2024-09-11T22:41:51.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fcf44f740 1 -- start start 2024-09-11T22:41:51.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fcf44f740 1 Processor -- start 2024-09-11T22:41:51.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7dbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7d2b40 0x56554d7d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7dc370 0x56554d7d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56554d7df010 con 0x56554d7dc370 2024-09-11T22:41:51.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56554d6f2060 con 0x56554d7d2b40 2024-09-11T22:41:51.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x56554d7df190 con 0x56554d7d15d0 2024-09-11T22:41:51.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fcf44f740 10 client.0 _ll_register_callbacks cb 0x56554d4458c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:51.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fc759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7dc370 0x56554d7d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fc759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7dc370 0x56554d7d9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58014/0 (socket says 172.21.15.5:58014) 2024-09-11T22:41:51.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fc759e640 1 -- 172.21.15.5:0/3555141887 learned_addr learned my addr 172.21.15.5:0/3555141887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:51.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.209+0000 7f1fc759e640 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7dbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1fc759e640 1 -- 172.21.15.5:0/3555141887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 msgr2=0x56554d7dbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:51.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1fc759e640 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7dbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1fc759e640 1 -- 172.21.15.5:0/3555141887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7d2b40 msgr2=0x56554d7d8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:51.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1fc759e640 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7d2b40 0x56554d7d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1fc759e640 1 -- 172.21.15.5:0/3555141887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554d4a83c0 con 0x56554d7dc370 2024-09-11T22:41:51.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1fc759e640 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7dc370 0x56554d7d9600 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0005fa0 tx=0x7f1fc00041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d4a5f0086599c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:51.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1faaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:51.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1faaff5640 1 -- 172.21.15.5:0/3555141887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc0055060 con 0x56554d7dc370 2024-09-11T22:41:51.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1faaff5640 1 -- 172.21.15.5:0/3555141887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f1fc002f5b0 con 0x56554d7dc370 2024-09-11T22:41:51.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1fcf44f740 1 -- 172.21.15.5:0/3555141887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fc0003970 con 0x56554d7dc370 2024-09-11T22:41:51.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1faaff5640 1 -- 172.21.15.5:0/3555141887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1fc0004390 con 0x56554d7dc370 2024-09-11T22:41:51.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1fcf44f740 10 client.21360 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:51.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1fcf44f740 1 -- 172.21.15.5:0/3555141887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56554d7df010 con 0x56554d7dc370 2024-09-11T22:41:51.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.210+0000 7f1faaff5640 1 -- 172.21.15.5:0/3555141887 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc0003970 con 0x56554d7dc370 2024-09-11T22:41:51.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.211+0000 7f1faaff5640 1 -- 172.21.15.5:0/3555141887 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f1fc004f040 con 0x56554d7dc370 2024-09-11T22:41:51.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.211+0000 7f1fcf44f740 10 client.21360 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:51.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.211+0000 7f1fcf44f740 10 client.21360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:51.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.211+0000 7f1fcf44f740 20 client.21360 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:51.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.211+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56554d7ebc00 0x56554d7edff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.211+0000 7f1fcf44f740 4 client.21360 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T22:41:51.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.211+0000 7f1fcf44f740 1 -- 172.21.15.5:0/3555141887 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56554d744860 con 0x56554d7ebc00 2024-09-11T22:41:51.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.211+0000 7f1fc659c640 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56554d7ebc00 0x56554d7edff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.211+0000 7f1fc659c640 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56554d7ebc00 0x56554d7edff0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22b3238432f5770c server_cookie=665b7756e48373bd in_seq=0 out_seq=0 2024-09-11T22:41:51.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.211+0000 7f1faaff5640 10 client.21360 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:51.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1faaff5640 1 -- 172.21.15.5:0/3555141887 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56554d744860 con 0x56554d7ebc00 2024-09-11T22:41:51.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1faaff5640 10 client.21360 handle_command_reply: tid=0 2024-09-11T22:41:51.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 client.21360 shutdown 2024-09-11T22:41:51.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 -- 172.21.15.5:0/3555141887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7dc370 msgr2=0x56554d7d9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:51.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7dc370 0x56554d7d9600 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0005fa0 tx=0x7f1fc00041f0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 -- 172.21.15.5:0/3555141887 shutdown_connections 2024-09-11T22:41:51.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56554d7ebc00 0x56554d7edff0 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7dc370 0x56554d7d9600 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7d2b40 0x56554d7d8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/3555141887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7dbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1faaff5640 0 client.21360 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:51.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 -- 172.21.15.5:0/3555141887 >> 172.21.15.5:0/3555141887 conn(0x56554d79a310 msgr2=0x56554d7d3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:51.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 -- 172.21.15.5:0/3555141887 shutdown_connections 2024-09-11T22:41:51.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 1 -- 172.21.15.5:0/3555141887 wait complete. 2024-09-11T22:41:51.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.212+0000 7f1fcf44f740 20 client.21360 trim_cache size 0 max 16384 2024-09-11T22:41:51.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fcf44f740 1 Processor -- start 2024-09-11T22:41:51.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fcf44f740 1 -- start start 2024-09-11T22:41:51.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7d15d0 0x56554d7ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7dc370 0x56554d7e4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81adb90 con 0x56554d7d2b40 2024-09-11T22:41:51.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81af7c0 con 0x56554d7d15d0 2024-09-11T22:41:51.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81ae260 con 0x56554d7dc370 2024-09-11T22:41:51.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7e4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58020/0 (socket says 172.21.15.5:58020) 2024-09-11T22:41:51.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc659c640 1 -- 172.21.15.5:0/1474153375 learned_addr learned my addr 172.21.15.5:0/1474153375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:51.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc759e640 1 --2- 172.21.15.5:0/1474153375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7d15d0 0x56554d7ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc759e640 1 --2- 172.21.15.5:0/1474153375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7dc370 0x56554d7e4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc659c640 1 -- 172.21.15.5:0/1474153375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7dc370 msgr2=0x56554d7e4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:51.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc659c640 1 --2- 172.21.15.5:0/1474153375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7dc370 0x56554d7e4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc659c640 1 -- 172.21.15.5:0/1474153375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7d15d0 msgr2=0x56554d7ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:51.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc659c640 1 --2- 172.21.15.5:0/1474153375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7d15d0 0x56554d7ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc659c640 1 -- 172.21.15.5:0/1474153375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56554d7d7fa0 con 0x56554d7d2b40 2024-09-11T22:41:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc759e640 1 --2- 172.21.15.5:0/1474153375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7d15d0 0x56554d7ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.226+0000 7f1fc659c640 1 --2- 172.21.15.5:0/1474153375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7e4800 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc006fd0 tx=0x7f1fbc009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ea0d5da88115b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1facff9640 1 -- 172.21.15.5:0/1474153375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc00f040 con 0x56554d7d2b40 2024-09-11T22:41:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1facff9640 1 -- 172.21.15.5:0/1474153375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fbc022070 con 0x56554d7d2b40 2024-09-11T22:41:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1facff9640 1 -- 172.21.15.5:0/1474153375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc01d3f0 con 0x56554d7d2b40 2024-09-11T22:41:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1fcf44f740 1 -- 172.21.15.5:0/1474153375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 msgr2=0x56554d7e4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/1474153375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7e4800 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc006fd0 tx=0x7f1fbc009dd0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1fcf44f740 1 -- 172.21.15.5:0/1474153375 shutdown_connections 2024-09-11T22:41:51.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/1474153375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7dc370 0x56554d7e4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/1474153375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7e4800 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/1474153375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7d15d0 0x56554d7ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1fcf44f740 1 -- 172.21.15.5:0/1474153375 >> 172.21.15.5:0/1474153375 conn(0x56554d79a310 msgr2=0x56554d7cc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:51.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1fcf44f740 1 -- 172.21.15.5:0/1474153375 shutdown_connections 2024-09-11T22:41:51.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.227+0000 7f1fcf44f740 1 -- 172.21.15.5:0/1474153375 wait complete. 2024-09-11T22:41:51.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fcf44f740 1 -- start start 2024-09-11T22:41:51.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fcf44f740 1 Processor -- start 2024-09-11T22:41:51.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7d5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fcf44f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7dc370 0x56554d7d5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81af0f0 con 0x56554d7d2b40 2024-09-11T22:41:51.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81ae960 con 0x56554d7dc370 2024-09-11T22:41:51.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fcf44f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc81abb90 con 0x56554d7d15d0 2024-09-11T22:41:51.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fcf44f740 10 client.0 _ll_register_callbacks cb 0x7f1fb80d4420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T22:41:51.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fc659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fc659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58032/0 (socket says 172.21.15.5:58032) 2024-09-11T22:41:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fc659c640 1 -- 172.21.15.5:0/699472838 learned_addr learned my addr 172.21.15.5:0/699472838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fc759e640 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7d5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fc759e640 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7dc370 0x56554d7d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fc659c640 1 -- 172.21.15.5:0/699472838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 msgr2=0x56554d7d5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fc659c640 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7d5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fc659c640 1 -- 172.21.15.5:0/699472838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7dc370 msgr2=0x56554d7d5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T22:41:51.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.228+0000 7f1fc659c640 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7dc370 0x56554d7d5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.229+0000 7f1fc659c640 1 -- 172.21.15.5:0/699472838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56554d7d64e0 con 0x56554d7d2b40 2024-09-11T22:41:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.229+0000 7f1fc659c640 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d57a0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc004700 tx=0x7f1fbc016cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1240035e883aa930 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.229+0000 7f1faaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-11T22:41:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.229+0000 7f1faaff5640 1 -- 172.21.15.5:0/699472838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc02a020 con 0x56554d7d2b40 2024-09-11T22:41:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.229+0000 7f1faaff5640 1 -- 172.21.15.5:0/699472838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f1fbc0159f0 con 0x56554d7d2b40 2024-09-11T22:41:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.229+0000 7f1fcf44f740 1 -- 172.21.15.5:0/699472838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fbc022070 con 0x56554d7d2b40 2024-09-11T22:41:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.229+0000 7f1faaff5640 1 -- 172.21.15.5:0/699472838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1fbc015bf0 con 0x56554d7d2b40 2024-09-11T22:41:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.229+0000 7f1fcf44f740 10 client.21366 fetch_fsmap learned FSMap version 57 2024-09-11T22:41:51.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.229+0000 7f1fcf44f740 1 -- 172.21.15.5:0/699472838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56554d85ad10 con 0x56554d7d2b40 2024-09-11T22:41:51.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.229+0000 7f1faaff5640 1 -- 172.21.15.5:0/699472838 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fbc022070 con 0x56554d7d2b40 2024-09-11T22:41:51.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.230+0000 7f1faaff5640 1 -- 172.21.15.5:0/699472838 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) ==== 8382+0+0 (secure 0 0 0) 0x7f1fbc026820 con 0x56554d7d2b40 2024-09-11T22:41:51.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.230+0000 7f1fcf44f740 10 client.21366 fetch_fsmap finished waiting for FSMap version 57 2024-09-11T22:41:51.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.230+0000 7f1fcf44f740 10 client.21366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-11T22:41:51.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.230+0000 7f1fcf44f740 20 client.21366 populate_metadata read hostname 'smithi005' 2024-09-11T22:41:51.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.230+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56554d85b440 0x56554d85d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:51.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.230+0000 7f1fcf44f740 4 client.21366 mds_command: new command op to 24500 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-11T22:41:51.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.230+0000 7f1fcf44f740 1 -- 172.21.15.5:0/699472838 --> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56554d73ed70 con 0x56554d85b440 2024-09-11T22:41:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.230+0000 7f1fc759e640 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56554d85b440 0x56554d85d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.230+0000 7f1faaff5640 10 client.21366 ms_handle_connect on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.230+0000 7f1fc759e640 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56554d85b440 0x56554d85d860 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e99c10b46699c1 server_cookie=4f549359b5dabc57 in_seq=0 out_seq=0 2024-09-11T22:41:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1faaff5640 1 -- 172.21.15.5:0/699472838 <== mds.0 v2:172.21.15.159:6838/994288762 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56554d73ed70 con 0x56554d85b440 2024-09-11T22:41:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1faaff5640 10 client.21366 handle_command_reply: tid=0 2024-09-11T22:41:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 client.21366 shutdown 2024-09-11T22:41:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 -- 172.21.15.5:0/699472838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 msgr2=0x56554d7d57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:51.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d57a0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc004700 tx=0x7f1fbc016cd0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 -- 172.21.15.5:0/699472838 shutdown_connections 2024-09-11T22:41:51.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.159:6838/994288762,v1:172.21.15.159:6839/994288762] conn(0x56554d85b440 0x56554d85d860 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56554d7dc370 0x56554d7d5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56554d7d2b40 0x56554d7d57a0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 --2- 172.21.15.5:0/699472838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56554d7d15d0 0x56554d7d5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 -- 172.21.15.5:0/699472838 >> 172.21.15.5:0/699472838 conn(0x56554d79a310 msgr2=0x56554d7db070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:51.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1faaff5640 0 client.21366 ms_handle_reset on v2:172.21.15.159:6838/994288762 2024-09-11T22:41:51.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 -- 172.21.15.5:0/699472838 shutdown_connections 2024-09-11T22:41:51.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 1 -- 172.21.15.5:0/699472838 wait complete. 2024-09-11T22:41:51.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcf44f740 20 client.21366 trim_cache size 0 max 16384 2024-09-11T22:41:51.249 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-11T22:41:51.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcd076640 1 -- 172.21.15.5:0/2874731949 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1fb00a2cf0 msgr2=0x7f1fb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:51.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcd076640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1fb00a2cf0 0x7f1fb00a51b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1fb80cc0d0 tx=0x7f1fb80d4550 comp rx=0 tx=0).stop 2024-09-11T22:41:51.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcd076640 1 -- 172.21.15.5:0/2874731949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc8102de0 msgr2=0x7f1fc8198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:51.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.231+0000 7f1fcd076640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc8102de0 0x7f1fc8198540 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800b4a0 tx=0x7f1fb800b960 comp rx=0 tx=0).stop 2024-09-11T22:41:51.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.232+0000 7f1fcd076640 1 -- 172.21.15.5:0/2874731949 shutdown_connections 2024-09-11T22:41:51.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.232+0000 7f1fcd076640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1fb00a2cf0 0x7f1fb00a51b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.232+0000 7f1fcd076640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fc810c180 0x7f1fc819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.232+0000 7f1fcd076640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1fc8103700 0x7f1fc8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.232+0000 7f1fcd076640 1 --2- 172.21.15.5:0/2874731949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc8102de0 0x7f1fc8198540 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.232+0000 7f1fcd076640 1 -- 172.21.15.5:0/2874731949 >> 172.21.15.5:0/2874731949 conn(0x7f1fc80fc7a0 msgr2=0x7f1fc80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.232+0000 7f1fcd076640 1 -- 172.21.15.5:0/2874731949 shutdown_connections 2024-09-11T22:41:51.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:51.232+0000 7f1fcd076640 1 -- 172.21.15.5:0/2874731949 wait complete. 2024-09-11T22:41:51.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:51 smithi005 ceph-mon[30943]: pgmap v626: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-11T22:41:52.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:51 smithi159 ceph-mon[35534]: pgmap v626: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-11T22:41:52.083 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-11T22:41:52.089 INFO:tasks.fwd_scrub:done joining 2024-09-11T22:41:52.090 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-11T22:41:52.099 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T22:41:52.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:51 smithi172 ceph-mon[35461]: pgmap v626: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 KiB/s wr, 13 op/s 2024-09-11T22:41:52.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.209+0000 7fa25e29b640 1 -- 172.21.15.5:0/3541642362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258101360 msgr2=0x7fa25810d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.209+0000 7fa25e29b640 1 --2- 172.21.15.5:0/3541642362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258101360 0x7fa25810d890 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fa250009cd0 tx=0x7fa25002a8e0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.209+0000 7fa25e29b640 1 -- 172.21.15.5:0/3541642362 shutdown_connections 2024-09-11T22:41:52.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.209+0000 7fa25e29b640 1 --2- 172.21.15.5:0/3541642362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258101360 0x7fa25810d890 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.209+0000 7fa25e29b640 1 --2- 172.21.15.5:0/3541642362 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2581009c0 0x7fa258100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.209+0000 7fa25e29b640 1 --2- 172.21.15.5:0/3541642362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2581069d0 0x7fa258106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.209+0000 7fa25e29b640 1 -- 172.21.15.5:0/3541642362 >> 172.21.15.5:0/3541642362 conn(0x7fa2580fc6a0 msgr2=0x7fa2580feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:52.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.209+0000 7fa25e29b640 1 -- 172.21.15.5:0/3541642362 shutdown_connections 2024-09-11T22:41:52.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.209+0000 7fa25e29b640 1 -- 172.21.15.5:0/3541642362 wait complete. 2024-09-11T22:41:52.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.210+0000 7fa25e29b640 1 Processor -- start 2024-09-11T22:41:52.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.210+0000 7fa25e29b640 1 -- start start 2024-09-11T22:41:52.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.210+0000 7fa25e29b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2581009c0 0x7fa258197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.210+0000 7fa25e29b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa258101360 0x7fa2581978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.210+0000 7fa25e29b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2581069d0 0x7fa25819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.210+0000 7fa25e29b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa258110e10 con 0x7fa2581069d0 2024-09-11T22:41:52.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.210+0000 7fa25e29b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa258110c90 con 0x7fa258101360 2024-09-11T22:41:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.210+0000 7fa25e29b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fa258110f90 con 0x7fa2581009c0 2024-09-11T22:41:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.210+0000 7fa25c838640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2581069d0 0x7fa25819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa25c838640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2581069d0 0x7fa25819bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40440/0 (socket says 172.21.15.5:40440) 2024-09-11T22:41:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa25c838640 1 -- 172.21.15.5:0/1561070215 learned_addr learned my addr 172.21.15.5:0/1561070215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa257fff640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2581009c0 0x7fa258197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa2577fe640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa258101360 0x7fa2581978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa25c838640 1 -- 172.21.15.5:0/1561070215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2581009c0 msgr2=0x7fa258197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa25c838640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2581009c0 0x7fa258197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa25c838640 1 -- 172.21.15.5:0/1561070215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa258101360 msgr2=0x7fa2581978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa25c838640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa258101360 0x7fa2581978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa25c838640 1 -- 172.21.15.5:0/1561070215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25819c350 con 0x7fa2581069d0 2024-09-11T22:41:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa25c838640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2581069d0 0x7fa25819bc20 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fa250005cc0 tx=0x7fa250003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa2557fa640 1 -- 172.21.15.5:0/1561070215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250004040 con 0x7fa2581069d0 2024-09-11T22:41:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa2557fa640 1 -- 172.21.15.5:0/1561070215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2500046c0 con 0x7fa2581069d0 2024-09-11T22:41:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa25e29b640 1 -- 172.21.15.5:0/1561070215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25819c5e0 con 0x7fa2581069d0 2024-09-11T22:41:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa2557fa640 1 -- 172.21.15.5:0/1561070215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250046480 con 0x7fa2581069d0 2024-09-11T22:41:52.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.211+0000 7fa25e29b640 1 -- 172.21.15.5:0/1561070215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25806a3c0 con 0x7fa2581069d0 2024-09-11T22:41:52.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.212+0000 7fa25e29b640 1 -- 172.21.15.5:0/1561070215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa258100e20 con 0x7fa2581069d0 2024-09-11T22:41:52.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.213+0000 7fa2557fa640 1 -- 172.21.15.5:0/1561070215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fa2500305a0 con 0x7fa2581069d0 2024-09-11T22:41:52.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.214+0000 7fa2557fa640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2400a2d00 0x7fa2400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.214+0000 7fa257fff640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2400a2d00 0x7fa2400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.214+0000 7fa2557fa640 1 -- 172.21.15.5:0/1561070215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7fa25002e030 con 0x7fa2581069d0 2024-09-11T22:41:52.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.214+0000 7fa257fff640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2400a2d00 0x7fa2400a51c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa248005fd0 tx=0x7fa24800b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:52.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.218+0000 7fa2557fa640 1 -- 172.21.15.5:0/1561070215 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa25004b060 con 0x7fa2581069d0 2024-09-11T22:41:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.396+0000 7fa25e29b640 1 -- 172.21.15.5:0/1561070215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa25806ade0 con 0x7fa2581069d0 2024-09-11T22:41:52.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.398+0000 7fa2557fa640 1 -- 172.21.15.5:0/1561070215 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+23256 (secure 0 0 0) 0x7fa2500bd6b0 con 0x7fa2581069d0 2024-09-11T22:41:52.399 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:41:52.399 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":119,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:41:22.614777+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":118,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2380674497":"2024-09-12T22:22:15.968309+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/3591749770":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/1577283141":"2024-09-12T22:22:15.968309+0000","172.21.15.5:6801/3591749770":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1618823752":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/785116270":"2024-09-12T22:22:15.968309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":33,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:41:52.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 -- 172.21.15.5:0/1561070215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2400a2d00 msgr2=0x7fa2400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2400a2d00 0x7fa2400a51c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa248005fd0 tx=0x7fa24800b390 comp rx=0 tx=0).stop 2024-09-11T22:41:52.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 -- 172.21.15.5:0/1561070215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2581069d0 msgr2=0x7fa25819bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2581069d0 0x7fa25819bc20 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fa250005cc0 tx=0x7fa250003970 comp rx=0 tx=0).stop 2024-09-11T22:41:52.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 -- 172.21.15.5:0/1561070215 shutdown_connections 2024-09-11T22:41:52.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2581069d0 0x7fa25819bc20 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fa2400a2d00 0x7fa2400a51c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa258101360 0x7fa2581978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 --2- 172.21.15.5:0/1561070215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2581009c0 0x7fa258197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 -- 172.21.15.5:0/1561070215 >> 172.21.15.5:0/1561070215 conn(0x7fa2580fc6a0 msgr2=0x7fa2580fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 -- 172.21.15.5:0/1561070215 shutdown_connections 2024-09-11T22:41:52.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.400+0000 7fa25e29b640 1 -- 172.21.15.5:0/1561070215 wait complete. 2024-09-11T22:41:52.412 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T22:15:32.851002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T22:20:14.556626+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T22:20:17.154834+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 118, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T22:41:52.413 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T22:41:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 -- 172.21.15.5:0/3772853367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe948100980 msgr2=0x7fe948100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 --2- 172.21.15.5:0/3772853367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe948100980 0x7fe948100de0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009930 tx=0x7fe93c02a420 comp rx=0 tx=0).stop 2024-09-11T22:41:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 -- 172.21.15.5:0/3772853367 shutdown_connections 2024-09-11T22:41:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 --2- 172.21.15.5:0/3772853367 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe948101320 0x7fe94810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 --2- 172.21.15.5:0/3772853367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe948100980 0x7fe948100de0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 --2- 172.21.15.5:0/3772853367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe948106990 0x7fe948106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 -- 172.21.15.5:0/3772853367 >> 172.21.15.5:0/3772853367 conn(0x7fe9480779d0 msgr2=0x7fe9480fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 -- 172.21.15.5:0/3772853367 shutdown_connections 2024-09-11T22:41:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 -- 172.21.15.5:0/3772853367 wait complete. 2024-09-11T22:41:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 Processor -- start 2024-09-11T22:41:52.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.519+0000 7fe94d98d640 1 -- start start 2024-09-11T22:41:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe94d98d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe948100980 0x7fe9480fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe94d98d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe948101320 0x7fe9480fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe94d98d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe948106990 0x7fe9480fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe94d98d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe948110e20 con 0x7fe948100980 2024-09-11T22:41:52.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe94d98d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe948110ca0 con 0x7fe948106990 2024-09-11T22:41:52.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe94d98d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fe948110fa0 con 0x7fe948101320 2024-09-11T22:41:52.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe946ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe948100980 0x7fe9480fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe946ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe948100980 0x7fe9480fce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40458/0 (socket says 172.21.15.5:40458) 2024-09-11T22:41:52.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe946ffd640 1 -- 172.21.15.5:0/1883269689 learned_addr learned my addr 172.21.15.5:0/1883269689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:52.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe9477fe640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe948106990 0x7fe9480fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe9467fc640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe948101320 0x7fe9480fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe946ffd640 1 -- 172.21.15.5:0/1883269689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe948101320 msgr2=0x7fe9480fd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe946ffd640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe948101320 0x7fe9480fd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe946ffd640 1 -- 172.21.15.5:0/1883269689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe948106990 msgr2=0x7fe9480fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe946ffd640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe948106990 0x7fe9480fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe946ffd640 1 -- 172.21.15.5:0/1883269689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9481a68f0 con 0x7fe948100980 2024-09-11T22:41:52.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe9477fe640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe948106990 0x7fe9480fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:52.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe946ffd640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe948100980 0x7fe9480fce50 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fe93800d950 tx=0x7fe93800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe94c98b640 1 -- 172.21.15.5:0/1883269689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938017030 con 0x7fe948100980 2024-09-11T22:41:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe94d98d640 1 -- 172.21.15.5:0/1883269689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9481a6be0 con 0x7fe948100980 2024-09-11T22:41:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe94c98b640 1 -- 172.21.15.5:0/1883269689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe938004020 con 0x7fe948100980 2024-09-11T22:41:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.520+0000 7fe94c98b640 1 -- 172.21.15.5:0/1883269689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9380047a0 con 0x7fe948100980 2024-09-11T22:41:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.521+0000 7fe94d98d640 1 -- 172.21.15.5:0/1883269689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9481a7150 con 0x7fe948100980 2024-09-11T22:41:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.522+0000 7fe94d98d640 1 -- 172.21.15.5:0/1883269689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe948100de0 con 0x7fe948100980 2024-09-11T22:41:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.523+0000 7fe94c98b640 1 -- 172.21.15.5:0/1883269689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fe9380041d0 con 0x7fe948100980 2024-09-11T22:41:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.523+0000 7fe94c98b640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9300a2d00 0x7fe9300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.524+0000 7fe9467fc640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9300a2d00 0x7fe9300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.524+0000 7fe94c98b640 1 -- 172.21.15.5:0/1883269689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7fe9380cf510 con 0x7fe948100980 2024-09-11T22:41:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.524+0000 7fe9467fc640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9300a2d00 0x7fe9300a51c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe93c006010 tx=0x7fe93c005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.526+0000 7fe94c98b640 1 -- 172.21.15.5:0/1883269689 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe93809c0c0 con 0x7fe948100980 2024-09-11T22:41:52.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.710+0000 7fe94d98d640 1 -- 172.21.15.5:0/1883269689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe9481a7380 con 0x7fe948100980 2024-09-11T22:41:52.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.710+0000 7fe94c98b640 1 -- 172.21.15.5:0/1883269689 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7fe93809b9c0 con 0x7fe948100980 2024-09-11T22:41:52.711 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-11T22:41:52.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 -- 172.21.15.5:0/1883269689 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9300a2d00 msgr2=0x7fe9300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9300a2d00 0x7fe9300a51c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe93c006010 tx=0x7fe93c005a20 comp rx=0 tx=0).stop 2024-09-11T22:41:52.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 -- 172.21.15.5:0/1883269689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe948100980 msgr2=0x7fe9480fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe948100980 0x7fe9480fce50 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7fe93800d950 tx=0x7fe93800de10 comp rx=0 tx=0).stop 2024-09-11T22:41:52.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 -- 172.21.15.5:0/1883269689 shutdown_connections 2024-09-11T22:41:52.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fe9300a2d00 0x7fe9300a51c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe948106990 0x7fe9480fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe948101320 0x7fe9480fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 --2- 172.21.15.5:0/1883269689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe948100980 0x7fe9480fce50 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 -- 172.21.15.5:0/1883269689 >> 172.21.15.5:0/1883269689 conn(0x7fe9480779d0 msgr2=0x7fe9480694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 -- 172.21.15.5:0/1883269689 shutdown_connections 2024-09-11T22:41:52.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.713+0000 7fe94d98d640 1 -- 172.21.15.5:0/1883269689 wait complete. 2024-09-11T22:41:52.725 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T22:41:52.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.831+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/2150859226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88101a90 msgr2=0x7f2e88101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.831+0000 7f2e8fdbe640 1 --2- 172.21.15.5:0/2150859226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88101a90 0x7f2e88101ef0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009930 tx=0x7f2e8402a420 comp rx=0 tx=0).stop 2024-09-11T22:41:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.831+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/2150859226 shutdown_connections 2024-09-11T22:41:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.831+0000 7f2e8fdbe640 1 --2- 172.21.15.5:0/2150859226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e88102430 0x7f2e8810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.831+0000 7f2e8fdbe640 1 --2- 172.21.15.5:0/2150859226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88101a90 0x7f2e88101ef0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.831+0000 7f2e8fdbe640 1 --2- 172.21.15.5:0/2150859226 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e88107aa0 0x7f2e88107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.831+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/2150859226 >> 172.21.15.5:0/2150859226 conn(0x7f2e880fd7d0 msgr2=0x7f2e880ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.831+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/2150859226 shutdown_connections 2024-09-11T22:41:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.831+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/2150859226 wait complete. 2024-09-11T22:41:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8fdbe640 1 Processor -- start 2024-09-11T22:41:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8fdbe640 1 -- start start 2024-09-11T22:41:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8fdbe640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e88101a90 0x7f2e88198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8fdbe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e88102430 0x7f2e881989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8fdbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88107aa0 0x7f2e8819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8fdbe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88111ee0 con 0x7f2e88107aa0 2024-09-11T22:41:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8fdbe640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88111d60 con 0x7f2e88102430 2024-09-11T22:41:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8fdbe640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f2e88112060 con 0x7f2e88101a90 2024-09-11T22:41:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8e35b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88107aa0 0x7f2e8819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8e35b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88107aa0 0x7f2e8819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40470/0 (socket says 172.21.15.5:40470) 2024-09-11T22:41:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8e35b640 1 -- 172.21.15.5:0/4095225491 learned_addr learned my addr 172.21.15.5:0/4095225491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.832+0000 7f2e8d359640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e88102430 0x7f2e881989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e8db5a640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e88101a90 0x7f2e88198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e8e35b640 1 -- 172.21.15.5:0/4095225491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e88101a90 msgr2=0x7f2e88198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e8e35b640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e88101a90 0x7f2e88198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e8e35b640 1 -- 172.21.15.5:0/4095225491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e88102430 msgr2=0x7f2e881989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e8e35b640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e88102430 0x7f2e881989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e8e35b640 1 -- 172.21.15.5:0/4095225491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e8819d460 con 0x7f2e88107aa0 2024-09-11T22:41:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e8db5a640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e88101a90 0x7f2e88198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e8e35b640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88107aa0 0x7f2e8819cd30 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000eaa0 tx=0x7f2e8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e7effd640 1 -- 172.21.15.5:0/4095225491 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000cdb0 con 0x7f2e88107aa0 2024-09-11T22:41:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e7effd640 1 -- 172.21.15.5:0/4095225491 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e80005020 con 0x7f2e88107aa0 2024-09-11T22:41:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/4095225491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e8819d750 con 0x7f2e88107aa0 2024-09-11T22:41:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e7effd640 1 -- 172.21.15.5:0/4095225491 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e800116b0 con 0x7f2e88107aa0 2024-09-11T22:41:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.833+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/4095225491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e881a4fe0 con 0x7f2e88107aa0 2024-09-11T22:41:52.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.834+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/4095225491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e88101ef0 con 0x7f2e88107aa0 2024-09-11T22:41:52.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.836+0000 7f2e7effd640 1 -- 172.21.15.5:0/4095225491 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2e80005560 con 0x7f2e88107aa0 2024-09-11T22:41:52.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.837+0000 7f2e7effd640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2e740a2d80 0x7f2e740a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:52.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.837+0000 7f2e8db5a640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2e740a2d80 0x7f2e740a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:52.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.837+0000 7f2e7effd640 1 -- 172.21.15.5:0/4095225491 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f2e800cfa30 con 0x7f2e88107aa0 2024-09-11T22:41:52.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.837+0000 7f2e8db5a640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2e740a2d80 0x7f2e740a5240 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2e78005fd0 tx=0x7f2e780094d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:52.841+0000 7f2e7effd640 1 -- 172.21.15.5:0/4095225491 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2e80016030 con 0x7f2e88107aa0 2024-09-11T22:41:53.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.016+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/4095225491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2e881997c0 con 0x7f2e88107aa0 2024-09-11T22:41:53.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.016+0000 7f2e7effd640 1 -- 172.21.15.5:0/4095225491 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7f2e8009c660 con 0x7f2e88107aa0 2024-09-11T22:41:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-11T22:41:53.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/4095225491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2e740a2d80 msgr2=0x7f2e740a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:53.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2e740a2d80 0x7f2e740a5240 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2e78005fd0 tx=0x7f2e780094d0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/4095225491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88107aa0 msgr2=0x7f2e8819cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:53.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88107aa0 0x7f2e8819cd30 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000eaa0 tx=0x7f2e8000ef60 comp rx=0 tx=0).stop 2024-09-11T22:41:53.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/4095225491 shutdown_connections 2024-09-11T22:41:53.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e88107aa0 0x7f2e8819cd30 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f2e740a2d80 0x7f2e740a5240 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2e88102430 0x7f2e881989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 --2- 172.21.15.5:0/4095225491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e88101a90 0x7f2e88198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/4095225491 >> 172.21.15.5:0/4095225491 conn(0x7f2e880fd7d0 msgr2=0x7f2e880fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:53.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/4095225491 shutdown_connections 2024-09-11T22:41:53.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.018+0000 7f2e8fdbe640 1 -- 172.21.15.5:0/4095225491 wait complete. 2024-09-11T22:41:53.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:52 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1561070215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:41:53.030 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T22:41:53.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:52 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1561070215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:41:53.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:52 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1561070215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:41:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.135+0000 7f282168d640 1 -- 172.21.15.5:0/3775073862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101350 msgr2=0x7f281c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.135+0000 7f282168d640 1 --2- 172.21.15.5:0/3775073862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101350 0x7f281c10d880 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f2814009cd0 tx=0x7f281402a8e0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.135+0000 7f282168d640 1 -- 172.21.15.5:0/3775073862 shutdown_connections 2024-09-11T22:41:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.135+0000 7f282168d640 1 --2- 172.21.15.5:0/3775073862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101350 0x7f281c10d880 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.135+0000 7f282168d640 1 --2- 172.21.15.5:0/3775073862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f281c1009b0 0x7f281c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.135+0000 7f282168d640 1 --2- 172.21.15.5:0/3775073862 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f281c1069c0 0x7f281c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.135+0000 7f282168d640 1 -- 172.21.15.5:0/3775073862 >> 172.21.15.5:0/3775073862 conn(0x7f281c0fc690 msgr2=0x7f281c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.135+0000 7f282168d640 1 -- 172.21.15.5:0/3775073862 shutdown_connections 2024-09-11T22:41:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.135+0000 7f282168d640 1 -- 172.21.15.5:0/3775073862 wait complete. 2024-09-11T22:41:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f282168d640 1 Processor -- start 2024-09-11T22:41:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f282168d640 1 -- start start 2024-09-11T22:41:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f282168d640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f281c1009b0 0x7f281c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f282168d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101350 0x7f281c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f282168d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f281c1069c0 0x7f281c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f282168d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f281c111ef0 con 0x7f281c101350 2024-09-11T22:41:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f282168d640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f281c111d70 con 0x7f281c1069c0 2024-09-11T22:41:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f282168d640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f281c112070 con 0x7f281c1009b0 2024-09-11T22:41:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101350 0x7f281c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101350 0x7f281c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40484/0 (socket says 172.21.15.5:40484) 2024-09-11T22:41:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281a7fc640 1 -- 172.21.15.5:0/3735106584 learned_addr learned my addr 172.21.15.5:0/3735106584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281affd640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f281c1009b0 0x7f281c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281b7fe640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f281c1069c0 0x7f281c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281a7fc640 1 -- 172.21.15.5:0/3735106584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f281c1009b0 msgr2=0x7f281c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281a7fc640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f281c1009b0 0x7f281c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281a7fc640 1 -- 172.21.15.5:0/3735106584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f281c1069c0 msgr2=0x7f281c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281a7fc640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f281c1069c0 0x7f281c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281a7fc640 1 -- 172.21.15.5:0/3735106584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c19d3e0 con 0x7f281c101350 2024-09-11T22:41:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.136+0000 7f281affd640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f281c1009b0 0x7f281c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:41:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.137+0000 7f281a7fc640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101350 0x7f281c198930 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f281000c970 tx=0x7f281000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.137+0000 7f2803fff640 1 -- 172.21.15.5:0/3735106584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810007bb0 con 0x7f281c101350 2024-09-11T22:41:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.137+0000 7f282168d640 1 -- 172.21.15.5:0/3735106584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c19d6d0 con 0x7f281c101350 2024-09-11T22:41:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.137+0000 7f2803fff640 1 -- 172.21.15.5:0/3735106584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2810007d50 con 0x7f281c101350 2024-09-11T22:41:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.137+0000 7f2803fff640 1 -- 172.21.15.5:0/3735106584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281000f450 con 0x7f281c101350 2024-09-11T22:41:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.137+0000 7f282168d640 1 -- 172.21.15.5:0/3735106584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c1a4f60 con 0x7f281c101350 2024-09-11T22:41:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.138+0000 7f282168d640 1 -- 172.21.15.5:0/3735106584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281c100e10 con 0x7f281c101350 2024-09-11T22:41:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.139+0000 7f2803fff640 1 -- 172.21.15.5:0/3735106584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f2810005020 con 0x7f281c101350 2024-09-11T22:41:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.140+0000 7f2803fff640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f28040a2d00 0x7f28040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.140+0000 7f281affd640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f28040a2d00 0x7f28040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.140+0000 7f2803fff640 1 -- 172.21.15.5:0/3735106584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f28100cf3c0 con 0x7f281c101350 2024-09-11T22:41:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.140+0000 7f281affd640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f28040a2d00 0x7f28040a51c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f280c005fd0 tx=0x7f280c0094d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.144+0000 7f2803fff640 1 -- 172.21.15.5:0/3735106584 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2810014030 con 0x7f281c101350 2024-09-11T22:41:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.320+0000 7f282168d640 1 -- 172.21.15.5:0/3735106584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f281c02d7c0 con 0x7f281c101350 2024-09-11T22:41:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.320+0000 7f2803fff640 1 -- 172.21.15.5:0/3735106584 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7f281009bf10 con 0x7f281c101350 2024-09-11T22:41:53.322 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-09-11T22:41:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.322+0000 7f282168d640 1 -- 172.21.15.5:0/3735106584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f28040a2d00 msgr2=0x7f28040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.322+0000 7f282168d640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f28040a2d00 0x7f28040a51c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f280c005fd0 tx=0x7f280c0094d0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.322+0000 7f282168d640 1 -- 172.21.15.5:0/3735106584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101350 msgr2=0x7f281c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.322+0000 7f282168d640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101350 0x7f281c198930 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f281000c970 tx=0x7f281000ce30 comp rx=0 tx=0).stop 2024-09-11T22:41:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.323+0000 7f282168d640 1 -- 172.21.15.5:0/3735106584 shutdown_connections 2024-09-11T22:41:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.323+0000 7f282168d640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f28040a2d00 0x7f28040a51c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.323+0000 7f282168d640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f281c1069c0 0x7f281c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.323+0000 7f282168d640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f281c101350 0x7f281c198930 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.323+0000 7f282168d640 1 --2- 172.21.15.5:0/3735106584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f281c1009b0 0x7f281c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.323+0000 7f282168d640 1 -- 172.21.15.5:0/3735106584 >> 172.21.15.5:0/3735106584 conn(0x7f281c0fc690 msgr2=0x7f281c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.323+0000 7f282168d640 1 -- 172.21.15.5:0/3735106584 shutdown_connections 2024-09-11T22:41:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.323+0000 7f282168d640 1 -- 172.21.15.5:0/3735106584 wait complete. 2024-09-11T22:41:53.335 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T22:41:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.436+0000 7f4278035640 1 -- 172.21.15.5:0/624119196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270100980 msgr2=0x7f4270100de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.436+0000 7f4278035640 1 --2- 172.21.15.5:0/624119196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270100980 0x7f4270100de0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f4260009930 tx=0x7f426002a440 comp rx=0 tx=0).stop 2024-09-11T22:41:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 -- 172.21.15.5:0/624119196 shutdown_connections 2024-09-11T22:41:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 --2- 172.21.15.5:0/624119196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270101320 0x7f427010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 --2- 172.21.15.5:0/624119196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270100980 0x7f4270100de0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 --2- 172.21.15.5:0/624119196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4270106990 0x7f4270106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 -- 172.21.15.5:0/624119196 >> 172.21.15.5:0/624119196 conn(0x7f4270077a10 msgr2=0x7f42700fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 -- 172.21.15.5:0/624119196 shutdown_connections 2024-09-11T22:41:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 -- 172.21.15.5:0/624119196 wait complete. 2024-09-11T22:41:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 Processor -- start 2024-09-11T22:41:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 -- start start 2024-09-11T22:41:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4270100980 0x7f42700fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270101320 0x7f42700fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270106990 0x7f42700fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4270110df0 con 0x7f4270106990 2024-09-11T22:41:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4270110c70 con 0x7f4270100980 2024-09-11T22:41:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.437+0000 7f4278035640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f4270110f70 con 0x7f4270101320 2024-09-11T22:41:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42765d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270106990 0x7f42700fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42765d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270106990 0x7f42700fd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40502/0 (socket says 172.21.15.5:40502) 2024-09-11T22:41:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42765d2640 1 -- 172.21.15.5:0/4064000788 learned_addr learned my addr 172.21.15.5:0/4064000788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:41:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42755d0640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270101320 0x7f42700fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f4275dd1640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4270100980 0x7f42700fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42765d2640 1 -- 172.21.15.5:0/4064000788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270101320 msgr2=0x7f42700fd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42765d2640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270101320 0x7f42700fd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42765d2640 1 -- 172.21.15.5:0/4064000788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4270100980 msgr2=0x7f42700fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42765d2640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4270100980 0x7f42700fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42765d2640 1 -- 172.21.15.5:0/4064000788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42701a6990 con 0x7f4270106990 2024-09-11T22:41:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42755d0640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270101320 0x7f42700fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T22:41:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f42765d2640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270106990 0x7f42700fd8d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f426800eaa0 tx=0x7f426800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f4266ffd640 1 -- 172.21.15.5:0/4064000788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426800cdb0 con 0x7f4270106990 2024-09-11T22:41:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f4278035640 1 -- 172.21.15.5:0/4064000788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42701a6c80 con 0x7f4270106990 2024-09-11T22:41:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f4266ffd640 1 -- 172.21.15.5:0/4064000788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4268005020 con 0x7f4270106990 2024-09-11T22:41:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f4266ffd640 1 -- 172.21.15.5:0/4064000788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680116b0 con 0x7f4270106990 2024-09-11T22:41:53.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.438+0000 7f4278035640 1 -- 172.21.15.5:0/4064000788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42701a71a0 con 0x7f4270106990 2024-09-11T22:41:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.439+0000 7f4278035640 1 -- 172.21.15.5:0/4064000788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4270100de0 con 0x7f4270106990 2024-09-11T22:41:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.440+0000 7f4266ffd640 1 -- 172.21.15.5:0/4064000788 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f4268005560 con 0x7f4270106990 2024-09-11T22:41:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.441+0000 7f4266ffd640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2d00 0x7f425c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:41:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.441+0000 7f4266ffd640 1 -- 172.21.15.5:0/4064000788 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8629+0+0 (secure 0 0 0) 0x7f42680cfb80 con 0x7f4270106990 2024-09-11T22:41:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.441+0000 7f4275dd1640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2d00 0x7f425c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:41:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.442+0000 7f4275dd1640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2d00 0x7f425c0a51c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f426c006fd0 tx=0x7f426c008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:41:53.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.444+0000 7f4266ffd640 1 -- 172.21.15.5:0/4064000788 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4268016030 con 0x7f4270106990 2024-09-11T22:41:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.667+0000 7f4278035640 1 -- 172.21.15.5:0/4064000788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f42700fe380 con 0x7f4270106990 2024-09-11T22:41:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.674+0000 7f4266ffd640 1 -- 172.21.15.5:0/4064000788 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740888 (secure 0 0 0) 0x7f426809c6d0 con 0x7f4270106990 2024-09-11T22:41:53.686 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:41:53.687 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:41:53.687 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-11T22:41:53.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.687 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 15201, 2024-09-11T22:41:53.687 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2024-09-11T22:41:53.687 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2024-09-11T22:41:53.688 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-11T22:41:53.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:53.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-11T22:41:53.688 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-11T22:41:53.688 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 233962001 2024-09-11T22:41:53.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.688 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:53.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-11T22:41:53.689 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-11T22:41:53.689 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 233962001 2024-09-11T22:41:53.689 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:53.689 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:53.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.689 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/233962001", 2024-09-11T22:41:53.690 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2024-09-11T22:22:15.968516+0000", 2024-09-11T22:41:53.690 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540701547738038271, 2024-09-11T22:41:53.690 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2024-09-11T22:41:53.690 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2024-09-11T22:41:53.690 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:53.690 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24826, 2024-09-11T22:41:53.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2024-09-11T22:41:53.690 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-09-11T22:41:53.691 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-11T22:41:53.691 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:53.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-11T22:41:53.691 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:53.691 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:53.691 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:53.691 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:53.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:53.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:53.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:53.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.693 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.693 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T22:41:53.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.694 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:53.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:53.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.695 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.695 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:53.695 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:53.695 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:53.695 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:53.695 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:53.695 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:53.696 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.696 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:53.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:53.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.698 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:53.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:53.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.699 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.699 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:53.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.699 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:53.700 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:53.700 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:53.700 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:53.700 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:53.700 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:53.700 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.701 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:53.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:53.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.703 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-11T22:41:53.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-11T22:41:53.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-11T22:41:53.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.705 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-11T22:41:53.705 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-11T22:41:53.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.706 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-11T22:41:53.706 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.706 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.706 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-11T22:41:53.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.707 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-11T22:41:53.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-11T22:41:53.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-11T22:41:53.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.709 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-11T22:41:53.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-11T22:41:53.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.710 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.710 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.710 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-11T22:41:53.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.711 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-11T22:41:53.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-11T22:41:53.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-11T22:41:53.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.712 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-11T22:41:53.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.713 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-11T22:41:53.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-11T22:41:53.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.713 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-11T22:41:53.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.715 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.715 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-11T22:41:53.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-11T22:41:53.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T22:41:53.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.717 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-11T22:41:53.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-11T22:41:53.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-11T22:41:53.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.719 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:53.719 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:53.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.719 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:53.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-11T22:41:53.719 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:53.720 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:53.720 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:53.720 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-11T22:41:53.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-11T22:41:53.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T22:41:53.721 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.722 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-11T22:41:53.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-11T22:41:53.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-11T22:41:53.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T22:41:53.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:41:53.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.724 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-11T22:41:53.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-11T22:41:53.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:53.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:53.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-11T22:41:53.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-11T22:41:53.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T22:41:53.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:41:53.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.726 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-11T22:41:53.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-11T22:41:53.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:53.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-11T22:41:53.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:53.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-11T22:41:53.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T22:41:53.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.728 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-11T22:41:53.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-11T22:41:53.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T22:41:53.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T22:41:53.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T22:41:53.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.730 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-11T22:41:53.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-11T22:41:53.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:53.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-11T22:41:53.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-11T22:41:53.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-11T22:41:53.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-11T22:41:53.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T22:41:53.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.732 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-11T22:41:53.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-11T22:41:53.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-11T22:41:53.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T22:41:53.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:41:53.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.734 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-11T22:41:53.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-11T22:41:53.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:53.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:53.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-11T22:41:53.735 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-11T22:41:53.735 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T22:41:53.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:41:53.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.736 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:53.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:53.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.738 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:53.738 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:53.738 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:53.738 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:53.738 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:53.738 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:53.738 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.739 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:53.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:53.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.742 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:53.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:53.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:53.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.743 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:53.743 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:53.743 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:53.743 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:53.743 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:53.743 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:53.744 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.744 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:53.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:53.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.746 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-11T22:41:53.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-11T22:41:53.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:53.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:53.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T22:41:53.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.749 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-11T22:41:53.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-11T22:41:53.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-11T22:41:53.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.750 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-11T22:41:53.751 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-11T22:41:53.751 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-11T22:41:53.751 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-11T22:41:53.751 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-11T22:41:53.751 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-11T22:41:53.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.752 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-11T22:41:53.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-11T22:41:53.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T22:41:53.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.755 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:53.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:53.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:53.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:53.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T22:41:53.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.757 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-11T22:41:53.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-11T22:41:53.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:53.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:53.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T22:41:53.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T22:41:53.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.759 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-11T22:41:53.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-11T22:41:53.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:53.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T22:41:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-11T22:41:53.761 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:53.762 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:53.762 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:53.762 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-11T22:41:53.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-11T22:41:53.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:53.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-11T22:41:53.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.763 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T22:41:53.763 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.764 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-11T22:41:53.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-11T22:41:53.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:53.764 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-11T22:41:53.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T22:41:53.765 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.765 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.766 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-11T22:41:53.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-11T22:41:53.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-11T22:41:53.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T22:41:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-11T22:41:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-11T22:41:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.770 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T22:41:53.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T22:41:53.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.771 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-11T22:41:53.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-11T22:41:53.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:53.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:53.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T22:41:53.772 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.772 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-11T22:41:53.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T22:41:53.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:53.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-11T22:41:53.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T22:41:53.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.774 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-11T22:41:53.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-11T22:41:53.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.775 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.775 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.775 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.776 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-11T22:41:53.776 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-11T22:41:53.776 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-11T22:41:53.776 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T22:41:53.776 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.777 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-11T22:41:53.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-11T22:41:53.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T22:41:53.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.779 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-11T22:41:53.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-11T22:41:53.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T22:41:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.781 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-11T22:41:53.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-11T22:41:53.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.781 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T22:41:53.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.783 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-11T22:41:53.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-11T22:41:53.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T22:41:53.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:53.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.785 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-11T22:41:53.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-11T22:41:53.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T22:41:53.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-11T22:41:53.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.788 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-11T22:41:53.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-11T22:41:53.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T22:41:53.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.789 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-11T22:41:53.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.790 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.790 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-11T22:41:53.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-11T22:41:53.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T22:41:53.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:53.791 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.792 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-11T22:41:53.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-11T22:41:53.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T22:41:53.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-11T22:41:53.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.794 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-11T22:41:53.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-11T22:41:53.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T22:41:53.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-11T22:41:53.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.795 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.796 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-11T22:41:53.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-11T22:41:53.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T22:41:53.796 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-11T22:41:53.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.797 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-11T22:41:53.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-11T22:41:53.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T22:41:53.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-11T22:41:53.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.799 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-11T22:41:53.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-11T22:41:53.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T22:41:53.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-11T22:41:53.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.801 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-11T22:41:53.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-11T22:41:53.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T22:41:53.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-11T22:41:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-11T22:41:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-11T22:41:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T22:41:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-11T22:41:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-11T22:41:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T22:41:53.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-11T22:41:53.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.806 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-11T22:41:53.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-11T22:41:53.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T22:41:53.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:53.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.807 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-11T22:41:53.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-11T22:41:53.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T22:41:53.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.808 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-11T22:41:53.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.809 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-11T22:41:53.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-11T22:41:53.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.810 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T22:41:53.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-11T22:41:53.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.811 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-11T22:41:53.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-11T22:41:53.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T22:41:53.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-11T22:41:53.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.813 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-11T22:41:53.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-11T22:41:53.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T22:41:53.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.814 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-11T22:41:53.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-11T22:41:53.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:53.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:53.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.815 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T22:41:53.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-11T22:41:53.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T22:41:53.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-11T22:41:53.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T22:41:53.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-11T22:41:53.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-11T22:41:53.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-11T22:41:53.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T22:41:53.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.819 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-11T22:41:53.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-11T22:41:53.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:53.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-11T22:41:53.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-11T22:41:53.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.821 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-11T22:41:53.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-11T22:41:53.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T22:41:53.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.823 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-11T22:41:53.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-11T22:41:53.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:53.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:53.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.824 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-11T22:41:53.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.824 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-11T22:41:53.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-11T22:41:53.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.825 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T22:41:53.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T22:41:53.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.826 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-11T22:41:53.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-11T22:41:53.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:53.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:53.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-11T22:41:53.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.828 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-11T22:41:53.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-11T22:41:53.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T22:41:53.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.830 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-11T22:41:53.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-11T22:41:53.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T22:41:53.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.831 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:53.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:53.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.832 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:53.833 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:53.833 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:53.833 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:53.833 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:53.833 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:53.833 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.833 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.834 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-11T22:41:53.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-11T22:41:53.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.834 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T22:41:53.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.836 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:53.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:53.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T22:41:53.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.838 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:53.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:53.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:53.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.839 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:53.839 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:53.839 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:53.839 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:53.839 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:53.840 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:53.840 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.840 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:53.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:53.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.843 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T22:41:53.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T22:41:53.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T22:41:53.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.844 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T22:41:53.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T22:41:53.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-11T22:41:53.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T22:41:53.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-11T22:41:53.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-11T22:41:53.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.846 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:53.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-11T22:41:53.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-11T22:41:53.848 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-11T22:41:53.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:53.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T22:41:53.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.850 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-11T22:41:53.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-11T22:41:53.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.851 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-11T22:41:53.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.851 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-11T22:41:53.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-11T22:41:53.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-11T22:41:53.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.852 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-11T22:41:53.853 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-11T22:41:53.853 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T22:41:53.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.854 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.854 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-11T22:41:53.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-11T22:41:53.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-11T22:41:53.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.855 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.855 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T22:41:53.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-11T22:41:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-11T22:41:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T22:41:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T22:41:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.858 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-11T22:41:53.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-11T22:41:53.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T22:41:53.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.860 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-11T22:41:53.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-11T22:41:53.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.860 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T22:41:53.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.862 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-11T22:41:53.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-11T22:41:53.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T22:41:53.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.863 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-11T22:41:53.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-11T22:41:53.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T22:41:53.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.865 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-11T22:41:53.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-11T22:41:53.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.866 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.866 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.866 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T22:41:53.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.867 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-11T22:41:53.867 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-11T22:41:53.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-11T22:41:53.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-11T22:41:53.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.869 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-11T22:41:53.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-11T22:41:53.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T22:41:53.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.871 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-11T22:41:53.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-11T22:41:53.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T22:41:53.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.873 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-11T22:41:53.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-11T22:41:53.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.873 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T22:41:53.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.875 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-11T22:41:53.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-11T22:41:53.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T22:41:53.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.876 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-11T22:41:53.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-11T22:41:53.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T22:41:53.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.878 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-11T22:41:53.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-11T22:41:53.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T22:41:53.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.879 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:53.880 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:53.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.880 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:53.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-11T22:41:53.880 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:53.880 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:53.881 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:53.881 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:53.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:53.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.882 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:53.882 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:53.882 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:53.882 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:53.882 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:53.882 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:53.883 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.883 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:53.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:53.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.885 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:53.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:53.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:53.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.886 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:53.886 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:53.886 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:53.887 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:53.887 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:53.887 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:53.887 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.887 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.888 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:53.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:53.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.890 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-11T22:41:53.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-11T22:41:53.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:53.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-11T22:41:53.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T22:41:53.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.892 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-11T22:41:53.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-11T22:41:53.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:53.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-11T22:41:53.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T22:41:53.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:53.893 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:53.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.894 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:53.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-11T22:41:53.894 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:53.894 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:53.894 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:53.894 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T22:41:53.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T22:41:53.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:53.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.896 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T22:41:53.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T22:41:53.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.898 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.898 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T22:41:53.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T22:41:53.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.899 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.899 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.899 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.900 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-11T22:41:53.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T22:41:53.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.901 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.901 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T22:41:53.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T22:41:53.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.902 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.903 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T22:41:53.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T22:41:53.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.904 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.905 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T22:41:53.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T22:41:53.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.907 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T22:41:53.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T22:41:53.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.907 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.908 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T22:41:53.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T22:41:53.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.909 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.910 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T22:41:53.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T22:41:53.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.912 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T22:41:53.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T22:41:53.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.912 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.914 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T22:41:53.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T22:41:53.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.914 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.915 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.915 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.915 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.915 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.915 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T22:41:53.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T22:41:53.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.917 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T22:41:53.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T22:41:53.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.919 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T22:41:53.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T22:41:53.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-11T22:41:53.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.921 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T22:41:53.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T22:41:53.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.922 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-11T22:41:53.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-11T22:41:53.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.923 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.924 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T22:41:53.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T22:41:53.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-11T22:41:53.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.925 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.926 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.926 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.926 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T22:41:53.926 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T22:41:53.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.927 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.928 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T22:41:53.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T22:41:53.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.930 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T22:41:53.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T22:41:53.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.931 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.932 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T22:41:53.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T22:41:53.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.932 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.932 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.934 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T22:41:53.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T22:41:53.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.935 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.935 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T22:41:53.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T22:41:53.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.936 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.937 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T22:41:53.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T22:41:53.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.938 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.939 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T22:41:53.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T22:41:53.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.941 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T22:41:53.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T22:41:53.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.942 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.942 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T22:41:53.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T22:41:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.944 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T22:41:53.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T22:41:53.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.945 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.946 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T22:41:53.946 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T22:41:53.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.948 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T22:41:53.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T22:41:53.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.950 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T22:41:53.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T22:41:53.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.951 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T22:41:53.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T22:41:53.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T22:41:53.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.953 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.953 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T22:41:53.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T22:41:53.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:53.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.955 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T22:41:53.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T22:41:53.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-11T22:41:53.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.957 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T22:41:53.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T22:41:53.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-11T22:41:53.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.959 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T22:41:53.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T22:41:53.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.961 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T22:41:53.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T22:41:53.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-11T22:41:53.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.962 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T22:41:53.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T22:41:53.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.964 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T22:41:53.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T22:41:53.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:53.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.966 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T22:41:53.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T22:41:53.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:53.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.968 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T22:41:53.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T22:41:53.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:53.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.969 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.969 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.969 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.970 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T22:41:53.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T22:41:53.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.972 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-11T22:41:53.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-11T22:41:53.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.973 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-11T22:41:53.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-11T22:41:53.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.974 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.974 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.975 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-11T22:41:53.975 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-11T22:41:53.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.976 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-11T22:41:53.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.977 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-11T22:41:53.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-11T22:41:53.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.978 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-11T22:41:53.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.979 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-11T22:41:53.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-11T22:41:53.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.980 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:53.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:53.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.982 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:53.982 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:53.982 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:53.982 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:53.982 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:53.982 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:53.982 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.983 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:53.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:53.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.984 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.985 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:53.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:53.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:53.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:53.986 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:53.986 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:53.986 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:53.986 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:53.986 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:53.987 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:53.987 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:53.987 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.987 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.987 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.987 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:53.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:53.988 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.988 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:53.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.989 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.990 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-11T22:41:53.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-11T22:41:53.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.990 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:53.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-11T22:41:53.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.991 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-11T22:41:53.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T22:41:53.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:53.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.993 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-11T22:41:53.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-11T22:41:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-11T22:41:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.996 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-11T22:41:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-11T22:41:53.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-11T22:41:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:53.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-11T22:41:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-11T22:41:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:53.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-11T22:41:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-11T22:41:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-11T22:41:54.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-11T22:41:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-11T22:41:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-11T22:41:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-11T22:41:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-11T22:41:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-11T22:41:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-11T22:41:54.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-11T22:41:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-11T22:41:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-11T22:41:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-11T22:41:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-11T22:41:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-11T22:41:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-11T22:41:54.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-11T22:41:54.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.012 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.016 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.017 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-11T22:41:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-11T22:41:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-11T22:41:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T22:41:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-11T22:41:54.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-11T22:41:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-11T22:41:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T22:41:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-11T22:41:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-11T22:41:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-11T22:41:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-11T22:41:54.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-11T22:41:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-11T22:41:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-11T22:41:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T22:41:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-11T22:41:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-11T22:41:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T22:41:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T22:41:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-11T22:41:54.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-11T22:41:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-11T22:41:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T22:41:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-11T22:41:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.034 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.037 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-11T22:41:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-11T22:41:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-11T22:41:54.042 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.043 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-11T22:41:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-11T22:41:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-11T22:41:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:54.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-11T22:41:54.047 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-11T22:41:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-11T22:41:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-11T22:41:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-11T22:41:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-11T22:41:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T22:41:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-11T22:41:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-11T22:41:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.050 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-11T22:41:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T22:41:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-11T22:41:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-11T22:41:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-11T22:41:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.053 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-11T22:41:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-11T22:41:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T22:41:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.057 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.058 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.061 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-11T22:41:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-11T22:41:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-11T22:41:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.066 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-11T22:41:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-11T22:41:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-11T22:41:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-11T22:41:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-11T22:41:54.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-11T22:41:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-11T22:41:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:41:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.070 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-11T22:41:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-11T22:41:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-11T22:41:54.071 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T22:41:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-11T22:41:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-11T22:41:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-11T22:41:54.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-11T22:41:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-11T22:41:54.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-11T22:41:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:41:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-11T22:41:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.078 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.080 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.086 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.087 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-11T22:41:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.092 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.096 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.099 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-11T22:41:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.100 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-11T22:41:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-11T22:41:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-11T22:41:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T22:41:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-11T22:41:54.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-11T22:41:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-11T22:41:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.103 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T22:41:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.105 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.107 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.110 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.111 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-11T22:41:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.113 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-11T22:41:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-11T22:41:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-11T22:41:54.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-11T22:41:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T22:41:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.115 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.121 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-11T22:41:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-11T22:41:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-11T22:41:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T22:41:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-11T22:41:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-11T22:41:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-11T22:41:54.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-11T22:41:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-11T22:41:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-11T22:41:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-11T22:41:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T22:41:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-11T22:41:54.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-11T22:41:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-11T22:41:54.131 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-11T22:41:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-11T22:41:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-11T22:41:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T22:41:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T22:41:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-11T22:41:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.136 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.139 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.142 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.143 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-11T22:41:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.144 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.145 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.146 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.148 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.150 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.151 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-11T22:41:54.153 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.155 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.156 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.159 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.161 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-11T22:41:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-11T22:41:54.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-11T22:41:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T22:41:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.165 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.166 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.171 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.173 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-11T22:41:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-11T22:41:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-11T22:41:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-11T22:41:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-11T22:41:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.175 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-11T22:41:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-11T22:41:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.178 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.178 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.178 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.178 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.179 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.179 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.179 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.179 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.180 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.180 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.180 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.181 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.181 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.181 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.181 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.182 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.182 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.182 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.183 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.183 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.183 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.183 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.183 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.183 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.184 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.184 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.185 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-11T22:41:54.186 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.186 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.186 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.186 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.187 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.188 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.188 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.188 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.188 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.188 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.188 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.189 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.190 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.190 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.190 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.190 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.191 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.191 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.191 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.192 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.192 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.192 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.192 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.192 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.192 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.192 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.193 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.194 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.195 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.195 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-11T22:41:54.195 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.195 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.196 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.196 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.197 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.201 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-11T22:41:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-11T22:41:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-11T22:41:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-11T22:41:54.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-11T22:41:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T22:41:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-11T22:41:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-11T22:41:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-11T22:41:54.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T22:41:54.211 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-11T22:41:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-11T22:41:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.214 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.215 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.219 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.220 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-11T22:41:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-11T22:41:54.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-11T22:41:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-11T22:41:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-11T22:41:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-11T22:41:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.226 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-11T22:41:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-11T22:41:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.227 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-11T22:41:54.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-11T22:41:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T22:41:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T22:41:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.232 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.235 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.236 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.237 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-11T22:41:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-11T22:41:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T22:41:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T22:41:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-11T22:41:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-11T22:41:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-11T22:41:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-11T22:41:54.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.244 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-11T22:41:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-11T22:41:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-11T22:41:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T22:41:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-11T22:41:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-11T22:41:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-11T22:41:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T22:41:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-11T22:41:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-11T22:41:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-11T22:41:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-11T22:41:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-11T22:41:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-11T22:41:54.250 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-11T22:41:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-11T22:41:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-11T22:41:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-11T22:41:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-11T22:41:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-11T22:41:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-11T22:41:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.253 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-11T22:41:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.254 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.260 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-11T22:41:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-11T22:41:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-11T22:41:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T22:41:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-11T22:41:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-11T22:41:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-11T22:41:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-11T22:41:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-11T22:41:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-11T22:41:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-11T22:41:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-11T22:41:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-11T22:41:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-11T22:41:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-11T22:41:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-11T22:41:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-11T22:41:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-11T22:41:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-11T22:41:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-11T22:41:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T22:41:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-11T22:41:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.312 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.312 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.312 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.313 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.313 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.313 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.313 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.315 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-11T22:41:54.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-11T22:41:54.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.317 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-11T22:41:54.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-11T22:41:54.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-11T22:41:54.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.319 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-11T22:41:54.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-11T22:41:54.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.320 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-11T22:41:54.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-11T22:41:54.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.321 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.322 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-11T22:41:54.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-11T22:41:54.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:54.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.324 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-11T22:41:54.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-11T22:41:54.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.326 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-11T22:41:54.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-11T22:41:54.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.326 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.326 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.327 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-11T22:41:54.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-11T22:41:54.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.329 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-11T22:41:54.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-11T22:41:54.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:54.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-11T22:41:54.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.331 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-11T22:41:54.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-11T22:41:54.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.332 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.333 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-11T22:41:54.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-11T22:41:54.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.334 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-11T22:41:54.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-11T22:41:54.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-11T22:41:54.337 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.337 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.337 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.337 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-11T22:41:54.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-11T22:41:54.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.338 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.338 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.338 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T22:41:54.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.339 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-11T22:41:54.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-11T22:41:54.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-11T22:41:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.341 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.342 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.342 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.342 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.342 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.342 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.342 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.343 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.345 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.346 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.346 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.346 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.346 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.346 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.346 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.347 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.347 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.349 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.349 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.349 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-11T22:41:54.349 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.350 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.350 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.350 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.351 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.351 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.351 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.351 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.351 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.352 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.352 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.352 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.354 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.355 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.355 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.355 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.355 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.356 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.356 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.356 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.357 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.358 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.359 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.359 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-11T22:41:54.359 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.359 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.359 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.359 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.360 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.361 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.361 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.361 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.361 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.361 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.361 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.362 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.364 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.365 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.365 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.365 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.365 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.366 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.366 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.366 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.367 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.368 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.368 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.369 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-11T22:41:54.369 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.369 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.369 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.369 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-11T22:41:54.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-11T22:41:54.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T22:41:54.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.371 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:54.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:54.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-11T22:41:54.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.373 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.374 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.374 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.374 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.375 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.375 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.375 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.375 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.376 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.377 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.378 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.379 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.379 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.379 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.379 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.379 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.379 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.380 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.381 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.382 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-11T22:41:54.382 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.382 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.382 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.382 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-11T22:41:54.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-11T22:41:54.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T22:41:54.384 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.384 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-11T22:41:54.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-11T22:41:54.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T22:41:54.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.386 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-11T22:41:54.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-11T22:41:54.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.387 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T22:41:54.387 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.387 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-11T22:41:54.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-11T22:41:54.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T22:41:54.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.389 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-11T22:41:54.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-11T22:41:54.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T22:41:54.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.391 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-11T22:41:54.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-11T22:41:54.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.392 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.392 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.392 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.393 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-11T22:41:54.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-11T22:41:54.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-11T22:41:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-11T22:41:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T22:41:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-11T22:41:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-11T22:41:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.398 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:54.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:54.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-11T22:41:54.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-11T22:41:54.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.400 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-11T22:41:54.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-11T22:41:54.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-11T22:41:54.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.401 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-11T22:41:54.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-11T22:41:54.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.402 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.403 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-11T22:41:54.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-11T22:41:54.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-11T22:41:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-11T22:41:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-11T22:41:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-11T22:41:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-11T22:41:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-11T22:41:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T22:41:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.419 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.419 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-11T22:41:54.419 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.419 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.419 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.420 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.421 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.421 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.421 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.421 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.421 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.421 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.421 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.422 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.422 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.424 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.425 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.425 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.425 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.425 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.426 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.426 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.426 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.427 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.428 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.428 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.428 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.429 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-11T22:41:54.429 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.429 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.429 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.429 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.429 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.430 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.430 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.431 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.431 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.431 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.431 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.431 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.432 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.432 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.434 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.434 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.435 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.435 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.435 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.436 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.436 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.436 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.436 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.437 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-11T22:41:54.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-11T22:41:54.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-11T22:41:54.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T22:41:54.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.440 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-11T22:41:54.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-11T22:41:54.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-11T22:41:54.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.442 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-11T22:41:54.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-11T22:41:54.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:54.443 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T22:41:54.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.444 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-11T22:41:54.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T22:41:54.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.445 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T22:41:54.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.446 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-11T22:41:54.446 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.446 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.446 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.446 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-11T22:41:54.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-11T22:41:54.446 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:54.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-11T22:41:54.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.447 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.447 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.448 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-11T22:41:54.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-11T22:41:54.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.449 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.450 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:54.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:54.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:54.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.451 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.452 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.452 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.453 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.453 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.453 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.453 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.453 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.454 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.454 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.454 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.456 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.457 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.457 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.457 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.457 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.457 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.457 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.457 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.458 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.460 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-11T22:41:54.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-11T22:41:54.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.460 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.462 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-11T22:41:54.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-11T22:41:54.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-11T22:41:54.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.463 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-11T22:41:54.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-11T22:41:54.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T22:41:54.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.465 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:54.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.466 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24875, 2024-09-11T22:41:54.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2024-09-11T22:41:54.466 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-09-11T22:41:54.466 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-11T22:41:54.466 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-11T22:41:54.467 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.467 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.467 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.467 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:54.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:54.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T22:41:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.470 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.471 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.474 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.477 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-11T22:41:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-11T22:41:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.478 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-11T22:41:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-11T22:41:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-11T22:41:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-11T22:41:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-11T22:41:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-11T22:41:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-11T22:41:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.481 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.482 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-11T22:41:54.482 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.483 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-11T22:41:54.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-11T22:41:54.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.483 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.483 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.484 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-11T22:41:54.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.484 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-11T22:41:54.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-11T22:41:54.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.485 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-11T22:41:54.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-11T22:41:54.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.486 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-11T22:41:54.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-11T22:41:54.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-11T22:41:54.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.488 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-11T22:41:54.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-11T22:41:54.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T22:41:54.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.490 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-11T22:41:54.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-11T22:41:54.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.490 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.490 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-11T22:41:54.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-11T22:41:54.492 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.492 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.492 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.492 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-11T22:41:54.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-11T22:41:54.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.493 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T22:41:54.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.494 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-11T22:41:54.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-11T22:41:54.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-11T22:41:54.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T22:41:54.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:41:54.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.495 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-11T22:41:54.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-11T22:41:54.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:54.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:54.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-11T22:41:54.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-11T22:41:54.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T22:41:54.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:41:54.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.497 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-11T22:41:54.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-11T22:41:54.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:54.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-11T22:41:54.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:54.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-11T22:41:54.498 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T22:41:54.498 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.499 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-11T22:41:54.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-11T22:41:54.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T22:41:54.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T22:41:54.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T22:41:54.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.501 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-11T22:41:54.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-11T22:41:54.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:54.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-11T22:41:54.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-11T22:41:54.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-11T22:41:54.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-11T22:41:54.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T22:41:54.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.502 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-11T22:41:54.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-11T22:41:54.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-11T22:41:54.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T22:41:54.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:41:54.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.504 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-11T22:41:54.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-11T22:41:54.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:54.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:54.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-11T22:41:54.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-11T22:41:54.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T22:41:54.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:41:54.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.507 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.511 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-11T22:41:54.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-11T22:41:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T22:41:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-11T22:41:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-11T22:41:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-11T22:41:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-11T22:41:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-11T22:41:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-11T22:41:54.517 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-11T22:41:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-11T22:41:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-11T22:41:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.518 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-11T22:41:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-11T22:41:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T22:41:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:54.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T22:41:54.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-11T22:41:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-11T22:41:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T22:41:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T22:41:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-11T22:41:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-11T22:41:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:54.524 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T22:41:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-11T22:41:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-11T22:41:54.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-11T22:41:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-11T22:41:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T22:41:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-11T22:41:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-11T22:41:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-11T22:41:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T22:41:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-11T22:41:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-11T22:41:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-11T22:41:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T22:41:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-11T22:41:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-11T22:41:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T22:41:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T22:41:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-11T22:41:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-11T22:41:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T22:41:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.535 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-11T22:41:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T22:41:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-11T22:41:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.536 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T22:41:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-11T22:41:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-11T22:41:54.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-11T22:41:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-11T22:41:54.538 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-11T22:41:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T22:41:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-11T22:41:54.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-11T22:41:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T22:41:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-11T22:41:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-11T22:41:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T22:41:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-11T22:41:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-11T22:41:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T22:41:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.544 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-11T22:41:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-11T22:41:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T22:41:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-11T22:41:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-11T22:41:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T22:41:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-11T22:41:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-11T22:41:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-11T22:41:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T22:41:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-11T22:41:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.549 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-11T22:41:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-11T22:41:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T22:41:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-11T22:41:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-11T22:41:54.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T22:41:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-11T22:41:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-11T22:41:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-11T22:41:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T22:41:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-11T22:41:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-11T22:41:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-11T22:41:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T22:41:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.555 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-11T22:41:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.556 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-11T22:41:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-11T22:41:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T22:41:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-11T22:41:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-11T22:41:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-11T22:41:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T22:41:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-11T22:41:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-11T22:41:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-11T22:41:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T22:41:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-11T22:41:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.561 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-11T22:41:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-11T22:41:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T22:41:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-11T22:41:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-11T22:41:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T22:41:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-11T22:41:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-11T22:41:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-11T22:41:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T22:41:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-11T22:41:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-11T22:41:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T22:41:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-11T22:41:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-11T22:41:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-11T22:41:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T22:41:54.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-11T22:41:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.570 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-11T22:41:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-11T22:41:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T22:41:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-11T22:41:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-11T22:41:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-11T22:41:54.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T22:41:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-11T22:41:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-11T22:41:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T22:41:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.575 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-11T22:41:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T22:41:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-11T22:41:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T22:41:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.577 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-11T22:41:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-11T22:41:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-11T22:41:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T22:41:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-11T22:41:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-11T22:41:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-11T22:41:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-11T22:41:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-11T22:41:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-11T22:41:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T22:41:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-11T22:41:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-11T22:41:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-11T22:41:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-11T22:41:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-11T22:41:54.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T22:41:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T22:41:54.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-11T22:41:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-11T22:41:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-11T22:41:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.587 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-11T22:41:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-11T22:41:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T22:41:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.589 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-11T22:41:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-11T22:41:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T22:41:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.592 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.594 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-11T22:41:54.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-11T22:41:54.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T22:41:54.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.595 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.595 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.596 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.596 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.596 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T22:41:54.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.597 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.598 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.598 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.598 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.599 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.599 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.599 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.599 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.599 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.600 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.602 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T22:41:54.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T22:41:54.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T22:41:54.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.603 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.603 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T22:41:54.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T22:41:54.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.604 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-11T22:41:54.604 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.604 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.604 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T22:41:54.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-11T22:41:54.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-11T22:41:54.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:54.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.606 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-11T22:41:54.606 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-11T22:41:54.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-11T22:41:54.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:54.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T22:41:54.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.609 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-11T22:41:54.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-11T22:41:54.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-11T22:41:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-11T22:41:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-11T22:41:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-11T22:41:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-11T22:41:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-11T22:41:54.614 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T22:41:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-11T22:41:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-11T22:41:54.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-11T22:41:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T22:41:54.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-11T22:41:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-11T22:41:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T22:41:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T22:41:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-11T22:41:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-11T22:41:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T22:41:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-11T22:41:54.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-11T22:41:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T22:41:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-11T22:41:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-11T22:41:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T22:41:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.624 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.624 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-11T22:41:54.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-11T22:41:54.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T22:41:54.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.625 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-11T22:41:54.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-11T22:41:54.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.627 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T22:41:54.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.627 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-11T22:41:54.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-11T22:41:54.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.628 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-11T22:41:54.628 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.628 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.628 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-11T22:41:54.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.629 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-11T22:41:54.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-11T22:41:54.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T22:41:54.630 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.630 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.631 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-11T22:41:54.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-11T22:41:54.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.631 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.631 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T22:41:54.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.632 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-11T22:41:54.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-11T22:41:54.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T22:41:54.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.634 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-11T22:41:54.634 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-11T22:41:54.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.635 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T22:41:54.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.636 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-11T22:41:54.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-11T22:41:54.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.636 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T22:41:54.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.637 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.638 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-11T22:41:54.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-11T22:41:54.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T22:41:54.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-11T22:41:54.640 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.640 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.640 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.640 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.641 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.641 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.642 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.642 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.642 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.642 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.642 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.643 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.645 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.646 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.646 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.646 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.647 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.647 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.647 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.647 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.648 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.649 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-11T22:41:54.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-11T22:41:54.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-11T22:41:54.650 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.650 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.650 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T22:41:54.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.651 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-11T22:41:54.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-11T22:41:54.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-11T22:41:54.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.652 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.652 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T22:41:54.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.653 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.653 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-11T22:41:54.653 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.653 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.654 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.654 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T22:41:54.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T22:41:54.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:54.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.655 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.655 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.655 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.655 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T22:41:54.656 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T22:41:54.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.656 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.657 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T22:41:54.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T22:41:54.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.659 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-11T22:41:54.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T22:41:54.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.659 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.660 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.661 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T22:41:54.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T22:41:54.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.662 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.662 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.662 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T22:41:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T22:41:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T22:41:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T22:41:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.666 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.666 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T22:41:54.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T22:41:54.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.667 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.668 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T22:41:54.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T22:41:54.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.669 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.669 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T22:41:54.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T22:41:54.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.671 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T22:41:54.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T22:41:54.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.673 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T22:41:54.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T22:41:54.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.673 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.675 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T22:41:54.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T22:41:54.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.676 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T22:41:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T22:41:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T22:41:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T22:41:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-11T22:41:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T22:41:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T22:41:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-11T22:41:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-11T22:41:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.683 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T22:41:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T22:41:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-11T22:41:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T22:41:54.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T22:41:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.686 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T22:41:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T22:41:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T22:41:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T22:41:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T22:41:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T22:41:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.692 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T22:41:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T22:41:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T22:41:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T22:41:54.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T22:41:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T22:41:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T22:41:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T22:41:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.699 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T22:41:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T22:41:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T22:41:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T22:41:54.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T22:41:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T22:41:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T22:41:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T22:41:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.706 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T22:41:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T22:41:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T22:41:54.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T22:41:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T22:41:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T22:41:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T22:41:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T22:41:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T22:41:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T22:41:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T22:41:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-11T22:41:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T22:41:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T22:41:54.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-11T22:41:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T22:41:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T22:41:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T22:41:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T22:41:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-11T22:41:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T22:41:54.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T22:41:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T22:41:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T22:41:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T22:41:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T22:41:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:54.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.725 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T22:41:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T22:41:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T22:41:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T22:41:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-11T22:41:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-11T22:41:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-11T22:41:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-11T22:41:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-11T22:41:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-11T22:41:54.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-11T22:41:54.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-11T22:41:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-11T22:41:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-11T22:41:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-11T22:41:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-11T22:41:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.737 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.743 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-11T22:41:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-11T22:41:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-11T22:41:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-11T22:41:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T22:41:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.749 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-11T22:41:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-11T22:41:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-11T22:41:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-11T22:41:54.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-11T22:41:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-11T22:41:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-11T22:41:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-11T22:41:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-11T22:41:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-11T22:41:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-11T22:41:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-11T22:41:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-11T22:41:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-11T22:41:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.758 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.758 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-11T22:41:54.758 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.759 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-11T22:41:54.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-11T22:41:54.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-11T22:41:54.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-11T22:41:54.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-11T22:41:54.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.760 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-11T22:41:54.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-11T22:41:54.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.762 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-11T22:41:54.763 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.763 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.763 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-11T22:41:54.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-11T22:41:54.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-11T22:41:54.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.765 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.766 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.766 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.766 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.766 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.766 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.766 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.766 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.767 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.767 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.768 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.769 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.770 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.770 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.770 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.770 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.771 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.771 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.771 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.771 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.773 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-11T22:41:54.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-11T22:41:54.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-11T22:41:54.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.775 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T22:41:54.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.775 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-11T22:41:54.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-11T22:41:54.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-11T22:41:54.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T22:41:54.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.777 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-11T22:41:54.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-11T22:41:54.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-11T22:41:54.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-11T22:41:54.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.779 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-11T22:41:54.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-11T22:41:54.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.779 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-11T22:41:54.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T22:41:54.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.781 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-11T22:41:54.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-11T22:41:54.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.781 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.781 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.781 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T22:41:54.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.782 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:54.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:54.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:54.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T22:41:54.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.784 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-11T22:41:54.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-11T22:41:54.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-11T22:41:54.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T22:41:54.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.786 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.786 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-11T22:41:54.786 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.786 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.786 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.787 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.787 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.788 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.788 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.788 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.788 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.788 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.788 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.788 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.789 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.791 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.792 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.792 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.792 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.792 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.792 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.792 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.793 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.793 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.795 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.795 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-11T22:41:54.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-11T22:41:54.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-11T22:41:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.797 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-11T22:41:54.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-11T22:41:54.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-11T22:41:54.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.798 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-11T22:41:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-11T22:41:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-11T22:41:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-11T22:41:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-11T22:41:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-11T22:41:54.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T22:41:54.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.803 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-11T22:41:54.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-11T22:41:54.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-11T22:41:54.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T22:41:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-11T22:41:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-11T22:41:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-11T22:41:54.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.806 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:54.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:54.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-11T22:41:54.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-11T22:41:54.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T22:41:54.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.808 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.809 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.809 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.809 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.809 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.809 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.810 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.810 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.810 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.811 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.812 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.813 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.813 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.814 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.814 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.814 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.814 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.814 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.815 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.816 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-11T22:41:54.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-11T22:41:54.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-11T22:41:54.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.818 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-11T22:41:54.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-11T22:41:54.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.819 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-11T22:41:54.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-11T22:41:54.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.820 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.820 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-11T22:41:54.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-11T22:41:54.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-11T22:41:54.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.821 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:41:54.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.822 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-11T22:41:54.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-11T22:41:54.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:54.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:54.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-11T22:41:54.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T22:41:54.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.823 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-11T22:41:54.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-11T22:41:54.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.824 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-11T22:41:54.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.825 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-11T22:41:54.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-11T22:41:54.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-11T22:41:54.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:41:54.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-11T22:41:54.827 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.828 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.828 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.828 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.829 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.829 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.829 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.829 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.829 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.830 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.830 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.830 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.832 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.833 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.833 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.834 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.834 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.834 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.834 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.834 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.835 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-11T22:41:54.837 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.837 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.837 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.837 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.839 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.839 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.839 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.839 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.839 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.839 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.839 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.840 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.843 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-11T22:41:54.847 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.847 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.847 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.847 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-11T22:41:54.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-11T22:41:54.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-11T22:41:54.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T22:41:54.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.849 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-11T22:41:54.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-11T22:41:54.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:54.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-11T22:41:54.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T22:41:54.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.850 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.852 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.852 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.852 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.852 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.852 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.852 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.852 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.853 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.855 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.856 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.857 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.863 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.867 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.868 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.868 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.868 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.868 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.870 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-11T22:41:54.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-11T22:41:54.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.871 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.871 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-11T22:41:54.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T22:41:54.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.872 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-11T22:41:54.872 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-11T22:41:54.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-11T22:41:54.873 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.873 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.873 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-11T22:41:54.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.873 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-11T22:41:54.874 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-11T22:41:54.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.874 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-11T22:41:54.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.875 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T22:41:54.875 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.875 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-11T22:41:54.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-11T22:41:54.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-11T22:41:54.877 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.877 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.877 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-11T22:41:54.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-11T22:41:54.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.878 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-11T22:41:54.878 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.878 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.878 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T22:41:54.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T22:41:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-11T22:41:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.881 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.882 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.885 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.888 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-11T22:41:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.889 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.890 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.891 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.891 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.891 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.891 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.891 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.891 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.892 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.894 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.895 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.895 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.895 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.895 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.895 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.895 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.895 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.896 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-11T22:41:54.898 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.899 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.899 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.899 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.899 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.900 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.900 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.900 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.900 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.900 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.901 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.901 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.901 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.901 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.902 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.903 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.904 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.904 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.904 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.905 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.905 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.905 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.905 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.906 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.907 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.908 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-11T22:41:54.908 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.908 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.908 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.908 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-11T22:41:54.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-11T22:41:54.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.909 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.909 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T22:41:54.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.910 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.911 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.911 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.911 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.912 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.912 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.912 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.912 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.913 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.914 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.914 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.915 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.915 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.916 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.916 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.916 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.916 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.916 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.916 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.917 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.919 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-11T22:41:54.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-11T22:41:54.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.920 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-11T22:41:54.920 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-11T22:41:54.920 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-11T22:41:54.920 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-11T22:41:54.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.921 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.921 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-11T22:41:54.921 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.922 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.922 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.922 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.923 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.923 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.923 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.923 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.923 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.924 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.924 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.924 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.925 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.926 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.926 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.926 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.926 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.927 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.927 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.927 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.927 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.927 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.928 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.928 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.928 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.929 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.930 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.930 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.930 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-11T22:41:54.931 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.931 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.931 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.931 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.932 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.932 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.932 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.932 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.934 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.935 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.935 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.936 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.936 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.936 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.937 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.937 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.937 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.937 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.937 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.938 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.939 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-11T22:41:54.940 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.940 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.940 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.940 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.941 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.942 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.942 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.942 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.942 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.942 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.942 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.943 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.945 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.946 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.946 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.946 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.946 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.946 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.946 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.946 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.947 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.949 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:54.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:54.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:54.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.951 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-11T22:41:54.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-11T22:41:54.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:54.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-11T22:41:54.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-11T22:41:54.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-11T22:41:54.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T22:41:54.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.953 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-11T22:41:54.953 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.953 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.953 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.953 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-11T22:41:54.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-11T22:41:54.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.954 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T22:41:54.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-11T22:41:54.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-11T22:41:54.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.957 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.958 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.958 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.958 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.958 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.958 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.959 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.959 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.959 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.959 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.959 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.960 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.961 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.961 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.962 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.962 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.962 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.963 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.963 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.963 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.963 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.963 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.964 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-11T22:41:54.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-11T22:41:54.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-11T22:41:54.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.966 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-11T22:41:54.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.967 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:54.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:54.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:54.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:54.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-11T22:41:54.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.969 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.969 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-11T22:41:54.969 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.970 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.970 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.970 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-11T22:41:54.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-11T22:41:54.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.971 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.972 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-11T22:41:54.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-11T22:41:54.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:54.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T22:41:54.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T22:41:54.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.973 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.974 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.974 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.975 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.975 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.975 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.975 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.975 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.975 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.975 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.976 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.976 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:54.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:54.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.977 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.978 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:54.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:54.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.978 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.978 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.978 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:54.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.979 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.979 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.979 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.979 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.979 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.979 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:54.980 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.980 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:54.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:54.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:54.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:54.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.982 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-11T22:41:54.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-11T22:41:54.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.983 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.983 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.984 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.984 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T22:41:54.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T22:41:54.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.984 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-11T22:41:54.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.986 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-11T22:41:54.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-11T22:41:54.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:54.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-11T22:41:54.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.987 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.987 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.987 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.988 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-11T22:41:54.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-11T22:41:54.988 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.988 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-11T22:41:54.988 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T22:41:54.989 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.989 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-11T22:41:54.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-11T22:41:54.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.990 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-11T22:41:54.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T22:41:54.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.991 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-11T22:41:54.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-11T22:41:54.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:54.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-11T22:41:54.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-11T22:41:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-11T22:41:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-11T22:41:54.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-11T22:41:54.994 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-11T22:41:54.994 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:54.994 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.994 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.995 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.995 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-11T22:41:54.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-11T22:41:54.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:54.995 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-11T22:41:54.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-11T22:41:54.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-11T22:41:54.996 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:54.996 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:54.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:54.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:54.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:54.997 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.997 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:54.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:54.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:54.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-11T22:41:54.997 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:54.997 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:54.998 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:54.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:54.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:54.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:54.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:54.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:54.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:54.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:54.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:54.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:54.999 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:54.999 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:54.999 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:54.999 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:54.999 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:54.999 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.000 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.000 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.002 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.003 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.003 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.003 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.003 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.004 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.004 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.004 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.005 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.006 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-11T22:41:55.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-11T22:41:55.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:55.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.008 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-11T22:41:55.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T22:41:55.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.010 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-11T22:41:55.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-11T22:41:55.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.012 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-11T22:41:55.012 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.012 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.012 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-11T22:41:55.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-11T22:41:55.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.013 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.013 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.014 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-11T22:41:55.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-11T22:41:55.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.016 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.017 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.017 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.017 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.017 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.017 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.017 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.017 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.020 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.020 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.021 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.021 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.022 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.023 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.023 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.023 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.023 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.023 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.023 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.024 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.025 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.025 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.025 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.026 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.026 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-11T22:41:55.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-11T22:41:55.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.027 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.027 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.027 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.028 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-11T22:41:55.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-11T22:41:55.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.029 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.029 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-11T22:41:55.030 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.030 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.030 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.030 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.031 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.031 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.031 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.031 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.032 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.032 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.032 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.032 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.033 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.034 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.034 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.034 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.035 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.035 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.035 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.036 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.036 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.036 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.036 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.036 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.036 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.037 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.038 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-11T22:41:55.039 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.039 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.039 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.039 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.041 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.041 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.041 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.041 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.041 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.041 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.041 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.042 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.042 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.042 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.043 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.044 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.045 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.045 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.045 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.045 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.045 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.046 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.046 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.046 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.047 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.047 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.047 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.047 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.048 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-11T22:41:55.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-11T22:41:55.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-11T22:41:55.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T22:41:55.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.050 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.050 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-11T22:41:55.050 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.051 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.051 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.051 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.052 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.052 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.052 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.052 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.052 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.052 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.053 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.053 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.054 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.055 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.055 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.056 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.056 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.056 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.057 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.057 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.057 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.057 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.057 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.057 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.057 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.058 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.059 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-11T22:41:55.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-11T22:41:55.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.061 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.061 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-11T22:41:55.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-11T22:41:55.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-11T22:41:55.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.062 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.063 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-11T22:41:55.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-11T22:41:55.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.064 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.064 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.064 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-11T22:41:55.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-11T22:41:55.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.065 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.066 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.066 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-11T22:41:55.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-11T22:41:55.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:55.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.068 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-11T22:41:55.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-11T22:41:55.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.070 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-11T22:41:55.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-11T22:41:55.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.071 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.071 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-11T22:41:55.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-11T22:41:55.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.072 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.073 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-11T22:41:55.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-11T22:41:55.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:55.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-11T22:41:55.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-11T22:41:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-11T22:41:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.077 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-11T22:41:55.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-11T22:41:55.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.077 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.078 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-11T22:41:55.078 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-11T22:41:55.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-11T22:41:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-11T22:41:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-11T22:41:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T22:41:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-11T22:41:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-11T22:41:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.084 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-11T22:41:55.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.084 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.085 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.086 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.086 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.086 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.086 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.086 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.086 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.087 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.088 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.088 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.088 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.089 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.089 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.089 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.090 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.090 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.090 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.090 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.090 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.090 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.090 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.091 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.092 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.092 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.093 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-11T22:41:55.093 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.094 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.094 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.094 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.095 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.095 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.095 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.095 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.095 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.095 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.095 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.096 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.096 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.096 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.096 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.099 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.099 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.099 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.099 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.099 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.100 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.100 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.100 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.100 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.101 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.101 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.102 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.102 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.102 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-11T22:41:55.103 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.103 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.103 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.103 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.104 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.104 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.105 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.105 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.105 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.105 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.105 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.106 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.106 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.107 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.107 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.107 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.107 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.109 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.109 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.109 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.109 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.109 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.109 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.109 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.110 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.110 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.110 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.110 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.111 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.111 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-11T22:41:55.112 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.112 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.113 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.113 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-11T22:41:55.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-11T22:41:55.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T22:41:55.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.114 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:55.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:55.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-11T22:41:55.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.116 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.117 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.117 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.117 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.118 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.118 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.118 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.118 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.119 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.120 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.120 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.124 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.125 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.125 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-11T22:41:55.125 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.125 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.125 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.126 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-11T22:41:55.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-11T22:41:55.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T22:41:55.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.127 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-11T22:41:55.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-11T22:41:55.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.128 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.128 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T22:41:55.128 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.129 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-11T22:41:55.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-11T22:41:55.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T22:41:55.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.131 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-11T22:41:55.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-11T22:41:55.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.131 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.131 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.131 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T22:41:55.132 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.132 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-11T22:41:55.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-11T22:41:55.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.133 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T22:41:55.134 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.134 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.134 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-11T22:41:55.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-11T22:41:55.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.134 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.135 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.135 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.136 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.136 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-11T22:41:55.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-11T22:41:55.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.136 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.136 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.137 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.137 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.137 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.137 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.137 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-11T22:41:55.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-11T22:41:55.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.138 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T22:41:55.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.139 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-11T22:41:55.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-11T22:41:55.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.140 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.140 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.140 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.142 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-11T22:41:55.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-11T22:41:55.142 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.142 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.143 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-11T22:41:55.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-11T22:41:55.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-11T22:41:55.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.144 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.144 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.144 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-11T22:41:55.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-11T22:41:55.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.145 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.145 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.146 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.146 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-11T22:41:55.146 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-11T22:41:55.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.146 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.147 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.147 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.147 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.147 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.148 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.148 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.148 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.148 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.149 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.149 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.149 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.149 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.149 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.149 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.150 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.150 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.150 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.150 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.150 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.150 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.151 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.151 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.151 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.152 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.153 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.153 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.153 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.153 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.154 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.154 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.154 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.154 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.154 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.154 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.154 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.155 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.156 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-11T22:41:55.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-11T22:41:55.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.158 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-11T22:41:55.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-11T22:41:55.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.160 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-11T22:41:55.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-11T22:41:55.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T22:41:55.161 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.161 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.162 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-11T22:41:55.162 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.162 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.163 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.163 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.164 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.164 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.164 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.164 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.164 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.164 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.164 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.165 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.165 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.165 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.166 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.167 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.168 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.168 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.168 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.168 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.168 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.169 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.169 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.169 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.169 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.171 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.171 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-11T22:41:55.172 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.172 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.172 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.172 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.172 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.173 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.173 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.173 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.173 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.174 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.174 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.174 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.175 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.176 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.176 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.177 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.177 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.178 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.178 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.178 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.178 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.178 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.179 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.179 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.179 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.179 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.180 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.180 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.180 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-11T22:41:55.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-11T22:41:55.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-11T22:41:55.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.182 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T22:41:55.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.182 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-11T22:41:55.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-11T22:41:55.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.183 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-11T22:41:55.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.184 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-11T22:41:55.184 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-11T22:41:55.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:55.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T22:41:55.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.186 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-11T22:41:55.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T22:41:55.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.186 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T22:41:55.187 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.187 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.187 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-11T22:41:55.188 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.188 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.188 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.188 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-11T22:41:55.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-11T22:41:55.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:55.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-11T22:41:55.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.190 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-11T22:41:55.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-11T22:41:55.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.191 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.192 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:55.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:55.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:55.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.194 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.195 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.195 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.195 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.195 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.195 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.195 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.195 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.195 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.196 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.196 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.196 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.196 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.199 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-11T22:41:55.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-11T22:41:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-11T22:41:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-11T22:41:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-11T22:41:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-11T22:41:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-11T22:41:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T22:41:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.207 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2024-09-11T22:41:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-11T22:41:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2024-09-11T22:41:55.208 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2024-09-11T22:41:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2024-09-11T22:41:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "restful", 2024-09-11T22:41:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_schedule" 2024-09-11T22:41:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-11T22:41:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-11T22:41:55.209 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T22:41:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.211 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.213 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.217 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.218 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-11T22:41:55.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-11T22:41:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-11T22:41:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-11T22:41:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-11T22:41:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-11T22:41:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-11T22:41:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-11T22:41:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-11T22:41:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-11T22:41:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.225 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-11T22:41:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-11T22:41:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-11T22:41:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-11T22:41:55.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-11T22:41:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-11T22:41:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-11T22:41:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-11T22:41:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-11T22:41:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-11T22:41:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-11T22:41:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-11T22:41:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T22:41:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-11T22:41:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-11T22:41:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-11T22:41:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-11T22:41:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-11T22:41:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-11T22:41:55.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T22:41:55.236 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-11T22:41:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-11T22:41:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-11T22:41:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T22:41:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:41:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-11T22:41:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-11T22:41:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:55.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-11T22:41:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-11T22:41:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T22:41:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:41:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.240 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-11T22:41:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-11T22:41:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-11T22:41:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-11T22:41:55.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T22:41:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-11T22:41:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-11T22:41:55.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T22:41:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T22:41:55.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T22:41:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-11T22:41:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-11T22:41:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:55.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-11T22:41:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-11T22:41:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-11T22:41:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-11T22:41:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T22:41:55.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-11T22:41:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-11T22:41:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-11T22:41:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T22:41:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T22:41:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-11T22:41:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-11T22:41:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-11T22:41:55.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-11T22:41:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T22:41:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T22:41:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.250 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.255 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-11T22:41:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-11T22:41:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T22:41:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-11T22:41:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-11T22:41:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-11T22:41:55.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-11T22:41:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-11T22:41:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-11T22:41:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-11T22:41:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-11T22:41:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-11T22:41:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-11T22:41:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-11T22:41:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T22:41:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:55.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T22:41:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.265 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-11T22:41:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-11T22:41:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T22:41:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T22:41:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-11T22:41:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-11T22:41:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:55.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T22:41:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-11T22:41:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-11T22:41:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-11T22:41:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-11T22:41:55.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T22:41:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.271 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-11T22:41:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-11T22:41:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-11T22:41:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T22:41:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-11T22:41:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-11T22:41:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-11T22:41:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T22:41:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-11T22:41:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-11T22:41:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T22:41:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T22:41:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-11T22:41:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-11T22:41:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T22:41:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-11T22:41:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T22:41:55.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-11T22:41:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T22:41:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-11T22:41:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-11T22:41:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-11T22:41:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-11T22:41:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-11T22:41:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T22:41:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-11T22:41:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-11T22:41:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T22:41:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-11T22:41:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-11T22:41:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T22:41:55.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.285 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-11T22:41:55.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-11T22:41:55.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T22:41:55.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.287 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-11T22:41:55.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-11T22:41:55.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.287 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-11T22:41:55.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:55.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.289 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-11T22:41:55.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-11T22:41:55.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T22:41:55.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-11T22:41:55.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.291 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-11T22:41:55.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-11T22:41:55.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T22:41:55.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-11T22:41:55.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.292 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-11T22:41:55.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-11T22:41:55.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-11T22:41:55.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:55.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.294 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-11T22:41:55.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-11T22:41:55.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T22:41:55.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-11T22:41:55.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.296 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-11T22:41:55.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-11T22:41:55.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T22:41:55.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-11T22:41:55.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.298 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-11T22:41:55.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-11T22:41:55.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T22:41:55.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-11T22:41:55.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.299 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-11T22:41:55.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-11T22:41:55.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T22:41:55.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-11T22:41:55.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.301 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-11T22:41:55.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-11T22:41:55.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T22:41:55.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-11T22:41:55.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.303 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-11T22:41:55.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-11T22:41:55.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T22:41:55.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-11T22:41:55.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.304 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-11T22:41:55.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-11T22:41:55.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-11T22:41:55.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:55.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.306 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-11T22:41:55.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-11T22:41:55.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T22:41:55.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-11T22:41:55.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.308 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-11T22:41:55.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-11T22:41:55.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-11T22:41:55.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-11T22:41:55.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.310 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-11T22:41:55.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-11T22:41:55.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T22:41:55.310 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-11T22:41:55.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.311 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-11T22:41:55.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-11T22:41:55.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T22:41:55.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-11T22:41:55.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.313 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-11T22:41:55.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-11T22:41:55.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T22:41:55.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-11T22:41:55.314 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.315 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-11T22:41:55.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-11T22:41:55.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T22:41:55.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.316 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-11T22:41:55.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-11T22:41:55.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:55.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T22:41:55.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-11T22:41:55.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T22:41:55.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-11T22:41:55.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T22:41:55.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-11T22:41:55.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-11T22:41:55.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-11T22:41:55.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T22:41:55.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.321 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-11T22:41:55.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-11T22:41:55.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-11T22:41:55.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-11T22:41:55.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.323 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-11T22:41:55.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-11T22:41:55.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T22:41:55.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.325 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-11T22:41:55.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-11T22:41:55.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.326 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:55.326 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-11T22:41:55.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.327 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-11T22:41:55.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-11T22:41:55.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T22:41:55.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T22:41:55.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.328 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-11T22:41:55.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-11T22:41:55.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:55.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-11T22:41:55.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.330 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-11T22:41:55.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-11T22:41:55.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T22:41:55.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-11T22:41:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-11T22:41:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T22:41:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.335 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.335 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.335 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.335 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.335 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.335 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.335 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.336 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-11T22:41:55.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-11T22:41:55.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.337 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T22:41:55.337 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.338 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.338 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.338 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T22:41:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.339 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.341 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.341 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.341 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.341 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.341 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.341 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.341 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.342 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.344 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T22:41:55.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T22:41:55.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T22:41:55.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.345 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T22:41:55.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T22:41:55.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-11T22:41:55.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T22:41:55.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-11T22:41:55.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-11T22:41:55.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:55.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-11T22:41:55.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-11T22:41:55.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-11T22:41:55.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:55.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T22:41:55.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.351 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-11T22:41:55.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-11T22:41:55.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-11T22:41:55.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.352 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-11T22:41:55.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-11T22:41:55.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-11T22:41:55.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.353 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-11T22:41:55.353 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-11T22:41:55.354 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T22:41:55.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.354 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-11T22:41:55.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-11T22:41:55.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-11T22:41:55.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T22:41:55.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.356 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-11T22:41:55.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-11T22:41:55.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T22:41:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T22:41:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-11T22:41:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-11T22:41:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T22:41:55.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.359 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-11T22:41:55.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-11T22:41:55.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T22:41:55.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.361 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-11T22:41:55.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-11T22:41:55.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T22:41:55.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.363 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-11T22:41:55.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-11T22:41:55.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T22:41:55.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.364 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-11T22:41:55.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-11T22:41:55.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T22:41:55.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.366 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-11T22:41:55.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-11T22:41:55.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-11T22:41:55.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-11T22:41:55.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.368 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.368 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-11T22:41:55.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-11T22:41:55.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T22:41:55.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.370 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-11T22:41:55.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-11T22:41:55.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T22:41:55.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.371 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-11T22:41:55.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-11T22:41:55.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T22:41:55.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.373 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-11T22:41:55.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-11T22:41:55.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T22:41:55.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.375 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-11T22:41:55.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-11T22:41:55.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.375 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T22:41:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.376 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-11T22:41:55.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-11T22:41:55.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T22:41:55.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-11T22:41:55.379 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.379 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.379 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.379 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.380 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.380 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.380 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.380 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.381 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.381 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.381 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.382 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.382 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.382 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.382 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.383 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.384 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.384 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.385 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.385 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.385 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.385 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.385 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.386 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.387 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.387 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.387 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-11T22:41:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-11T22:41:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-11T22:41:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T22:41:55.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.389 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-11T22:41:55.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-11T22:41:55.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-11T22:41:55.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T22:41:55.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.393 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-11T22:41:55.393 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.393 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.393 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.393 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T22:41:55.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T22:41:55.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:55.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.395 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T22:41:55.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T22:41:55.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.396 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.397 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T22:41:55.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T22:41:55.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.399 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-11T22:41:55.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T22:41:55.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.400 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T22:41:55.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T22:41:55.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.402 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T22:41:55.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T22:41:55.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.402 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.404 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T22:41:55.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T22:41:55.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.405 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T22:41:55.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T22:41:55.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.407 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.407 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T22:41:55.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T22:41:55.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.408 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.409 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T22:41:55.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T22:41:55.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.410 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.410 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T22:41:55.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T22:41:55.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.412 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T22:41:55.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T22:41:55.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.414 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T22:41:55.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T22:41:55.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.416 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T22:41:55.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T22:41:55.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.417 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T22:41:55.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T22:41:55.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-11T22:41:55.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.419 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T22:41:55.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T22:41:55.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.421 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-11T22:41:55.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-11T22:41:55.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.422 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T22:41:55.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T22:41:55.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-11T22:41:55.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.424 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T22:41:55.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T22:41:55.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.425 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.425 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.426 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T22:41:55.426 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T22:41:55.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.428 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T22:41:55.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T22:41:55.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.429 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T22:41:55.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T22:41:55.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.431 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T22:41:55.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T22:41:55.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.433 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T22:41:55.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T22:41:55.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.434 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.435 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T22:41:55.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T22:41:55.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.436 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T22:41:55.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T22:41:55.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.437 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.438 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T22:41:55.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T22:41:55.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.440 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T22:41:55.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T22:41:55.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.442 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T22:41:55.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T22:41:55.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.443 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T22:41:55.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T22:41:55.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.445 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T22:41:55.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T22:41:55.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.446 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.447 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T22:41:55.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T22:41:55.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.449 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T22:41:55.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T22:41:55.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T22:41:55.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.450 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T22:41:55.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T22:41:55.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:55.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.452 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T22:41:55.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T22:41:55.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-11T22:41:55.453 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.453 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.453 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.454 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T22:41:55.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T22:41:55.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.454 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.454 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-11T22:41:55.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.455 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T22:41:55.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T22:41:55.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.457 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T22:41:55.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T22:41:55.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-11T22:41:55.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.459 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.459 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T22:41:55.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T22:41:55.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.460 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.460 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.460 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.460 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T22:41:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T22:41:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.462 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.462 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.462 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.462 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T22:41:55.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T22:41:55.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.463 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:55.463 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.463 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.464 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.464 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T22:41:55.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T22:41:55.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.465 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:55.465 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.465 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.465 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.466 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.466 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T22:41:55.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T22:41:55.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:55.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.467 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.468 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-11T22:41:55.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-11T22:41:55.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.469 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-11T22:41:55.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-11T22:41:55.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.471 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-11T22:41:55.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-11T22:41:55.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.472 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-11T22:41:55.472 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.473 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-11T22:41:55.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-11T22:41:55.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-11T22:41:55.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.474 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.474 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.474 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.474 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-11T22:41:55.475 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-11T22:41:55.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.476 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.477 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.477 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.478 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.478 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.478 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.478 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.478 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.479 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.479 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.479 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.481 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.481 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.482 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.482 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.482 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.482 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.482 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.482 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.482 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.482 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.483 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.484 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.484 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.485 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.485 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-11T22:41:55.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-11T22:41:55.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.485 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.486 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.486 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.486 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-11T22:41:55.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.486 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-11T22:41:55.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T22:41:55.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.487 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-11T22:41:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-11T22:41:55.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-11T22:41:55.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.490 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-11T22:41:55.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-11T22:41:55.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.490 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-11T22:41:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.491 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-11T22:41:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-11T22:41:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-11T22:41:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-11T22:41:55.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-11T22:41:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.495 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-11T22:41:55.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-11T22:41:55.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.495 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-11T22:41:55.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.496 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.496 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-11T22:41:55.496 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.496 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.497 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-11T22:41:55.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-11T22:41:55.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-11T22:41:55.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-11T22:41:55.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-11T22:41:55.498 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.498 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.498 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.499 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-11T22:41:55.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-11T22:41:55.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.499 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-11T22:41:55.501 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.501 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.501 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-11T22:41:55.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-11T22:41:55.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.502 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-11T22:41:55.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.503 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.503 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.503 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.504 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.504 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.504 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.504 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.504 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.505 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.505 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.505 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.505 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.506 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.507 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.509 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.509 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.509 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.509 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.509 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.509 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.509 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.510 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.512 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-11T22:41:55.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-11T22:41:55.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-11T22:41:55.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.513 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T22:41:55.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.513 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.513 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-11T22:41:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-11T22:41:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-11T22:41:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T22:41:55.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.515 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-11T22:41:55.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-11T22:41:55.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-11T22:41:55.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-11T22:41:55.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.517 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-11T22:41:55.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-11T22:41:55.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-11T22:41:55.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T22:41:55.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-11T22:41:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-11T22:41:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T22:41:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:55.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T22:41:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-11T22:41:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-11T22:41:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-11T22:41:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T22:41:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-11T22:41:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.524 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.526 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.530 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.532 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.533 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-11T22:41:55.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-11T22:41:55.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-11T22:41:55.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.535 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-11T22:41:55.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-11T22:41:55.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-11T22:41:55.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.537 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:55.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:55.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-11T22:41:55.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.539 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.539 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-11T22:41:55.540 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-11T22:41:55.540 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-11T22:41:55.540 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.540 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-11T22:41:55.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-11T22:41:55.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-11T22:41:55.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T22:41:55.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.542 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-11T22:41:55.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-11T22:41:55.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-11T22:41:55.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.543 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T22:41:55.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.543 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.544 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-11T22:41:55.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-11T22:41:55.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.544 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.544 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.544 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-11T22:41:55.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.545 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:55.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:55.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-11T22:41:55.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-11T22:41:55.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T22:41:55.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.547 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.548 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.548 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.548 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.549 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.549 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.549 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.549 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.550 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.551 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.552 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.553 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.553 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.553 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.553 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.553 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.553 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.554 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.555 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.555 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.556 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-11T22:41:55.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-11T22:41:55.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.557 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-11T22:41:55.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.557 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-11T22:41:55.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-11T22:41:55.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.558 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-11T22:41:55.558 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-11T22:41:55.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.559 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-11T22:41:55.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-11T22:41:55.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.559 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-11T22:41:55.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.560 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.560 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.560 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:41:55.560 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.561 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-11T22:41:55.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-11T22:41:55.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.561 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:55.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:55.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-11T22:41:55.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.562 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T22:41:55.562 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.562 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-11T22:41:55.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-11T22:41:55.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-11T22:41:55.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.564 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-11T22:41:55.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-11T22:41:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-11T22:41:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T22:41:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-11T22:41:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.567 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.567 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.567 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.567 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.567 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.568 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.568 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.568 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.568 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.568 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.569 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.569 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.569 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.570 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.571 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.571 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.572 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.572 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.572 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.572 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.572 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.573 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.573 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.573 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.574 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.575 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.575 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.575 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.575 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-11T22:41:55.576 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.576 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.576 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.576 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.577 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.577 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.577 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.577 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.577 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.577 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.577 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.578 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.578 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.578 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.578 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.578 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.579 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.580 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.581 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.581 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.582 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.582 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.582 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.582 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.582 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.582 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.584 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.584 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.585 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.585 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-11T22:41:55.585 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.585 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.585 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.585 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-11T22:41:55.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-11T22:41:55.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.586 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-11T22:41:55.586 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.586 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.586 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-11T22:41:55.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.587 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.587 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-11T22:41:55.587 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-11T22:41:55.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:55.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-11T22:41:55.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.588 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.588 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-11T22:41:55.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.589 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.590 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.590 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.590 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.590 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.590 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.591 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.591 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.591 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.591 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.593 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.594 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.595 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.595 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.595 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.595 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.595 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.595 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.596 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.597 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.598 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.598 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-11T22:41:55.598 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.598 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.598 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.599 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-11T22:41:55.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-11T22:41:55.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-11T22:41:55.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-11T22:41:55.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T22:41:55.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.600 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.601 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.601 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.601 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.602 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.602 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.602 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.602 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.602 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.603 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.603 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.604 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-11T22:41:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-11T22:41:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-11T22:41:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T22:41:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-11T22:41:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-11T22:41:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-11T22:41:55.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-11T22:41:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.612 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-11T22:41:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-11T22:41:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-11T22:41:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.613 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T22:41:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-11T22:41:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-11T22:41:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.614 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-11T22:41:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-11T22:41:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-11T22:41:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-11T22:41:55.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T22:41:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T22:41:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-11T22:41:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.619 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.621 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.621 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.622 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.622 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.623 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.623 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.624 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.624 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.624 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.624 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.624 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.624 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.624 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.624 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.625 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.626 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-11T22:41:55.627 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.628 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.628 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.628 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.629 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.629 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.629 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.629 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.629 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.630 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.630 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.630 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.630 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.630 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.631 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.631 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.632 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.632 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.633 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.634 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.634 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.634 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.634 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.634 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.636 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.636 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.636 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.638 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.638 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-11T22:41:55.638 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.639 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.639 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.639 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.640 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.640 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.640 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.640 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.640 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.640 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.640 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.641 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.641 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.641 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.641 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.642 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.642 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.643 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.644 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.645 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.645 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.645 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.645 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.645 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.645 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.646 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.648 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-11T22:41:55.648 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.648 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.648 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.648 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-11T22:41:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-11T22:41:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T22:41:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.651 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.652 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.652 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.652 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.652 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.653 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.653 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.654 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.655 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.655 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.656 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.656 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.656 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.656 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.656 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.656 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.656 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.657 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.659 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-11T22:41:55.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-11T22:41:55.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.659 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.659 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.660 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-11T22:41:55.660 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-11T22:41:55.660 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-11T22:41:55.660 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-11T22:41:55.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.661 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-11T22:41:55.661 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.661 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.662 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.662 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.663 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.663 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.663 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.663 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.663 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.663 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.664 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.664 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.665 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.666 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.666 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.667 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.667 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.667 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.667 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.667 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.668 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.668 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.668 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.668 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.668 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.669 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.669 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.669 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.669 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.670 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.671 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-11T22:41:55.671 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.671 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.671 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.671 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.672 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.674 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.677 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-11T22:41:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.680 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.682 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.685 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.690 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.691 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-11T22:41:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-11T22:41:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-11T22:41:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-11T22:41:55.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-11T22:41:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T22:41:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-11T22:41:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-11T22:41:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-11T22:41:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T22:41:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-11T22:41:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-11T22:41:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.703 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-11T22:41:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-11T22:41:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-11T22:41:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-11T22:41:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-11T22:41:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-11T22:41:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-11T22:41:55.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-11T22:41:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.709 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-11T22:41:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-11T22:41:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-11T22:41:55.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-11T22:41:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-11T22:41:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T22:41:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T22:41:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.715 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.719 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-11T22:41:55.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-11T22:41:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T22:41:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T22:41:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-11T22:41:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-11T22:41:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-11T22:41:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-11T22:41:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-11T22:41:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-11T22:41:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-11T22:41:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T22:41:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-11T22:41:55.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-11T22:41:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-11T22:41:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T22:41:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-11T22:41:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-11T22:41:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-11T22:41:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.733 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-11T22:41:55.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-11T22:41:55.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-11T22:41:55.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-11T22:41:55.734 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-11T22:41:55.734 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.735 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-11T22:41:55.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-11T22:41:55.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-11T22:41:55.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-11T22:41:55.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-11T22:41:55.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.737 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.737 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.737 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-11T22:41:55.737 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.737 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.737 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.738 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.739 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.739 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.739 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.739 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.739 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.739 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.739 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.739 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.740 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.742 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.743 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.743 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.743 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.743 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.743 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.744 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.744 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.744 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.745 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.746 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-11T22:41:55.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-11T22:41:55.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-11T22:41:55.747 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.748 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-11T22:41:55.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T22:41:55.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.749 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.750 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-11T22:41:55.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-11T22:41:55.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.751 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.752 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-11T22:41:55.752 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.752 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.752 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-11T22:41:55.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-11T22:41:55.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.754 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-11T22:41:55.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-11T22:41:55.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.755 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.755 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.756 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.757 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.757 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.757 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.757 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.758 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.758 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.758 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.758 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.759 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.760 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.761 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.762 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.762 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.762 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.762 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.762 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.762 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.763 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.765 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-11T22:41:55.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-11T22:41:55.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.766 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.766 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.766 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.766 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-11T22:41:55.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-11T22:41:55.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.767 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.768 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-11T22:41:55.769 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.769 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.769 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.769 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.770 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.770 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.770 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.770 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.771 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.771 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.771 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.771 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.773 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.774 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.774 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.775 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.775 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.775 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.775 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.775 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.775 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.776 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.776 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.777 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.777 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.777 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.778 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-11T22:41:55.778 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.778 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.778 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.778 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.779 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.780 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.780 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.780 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.780 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.780 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.780 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.780 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.781 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.781 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.784 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.784 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.784 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.784 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.784 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.784 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.784 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.785 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.787 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-11T22:41:55.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-11T22:41:55.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-11T22:41:55.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-11T22:41:55.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.789 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-11T22:41:55.789 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.789 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.789 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.789 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.790 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.791 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.791 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.791 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.791 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.791 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.791 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.791 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.792 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.794 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.795 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.795 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.795 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.795 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.795 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.795 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.795 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.796 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.798 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-11T22:41:55.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-11T22:41:55.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.800 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-11T22:41:55.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-11T22:41:55.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-11T22:41:55.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.801 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-11T22:41:55.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-11T22:41:55.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.803 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-11T22:41:55.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-11T22:41:55.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.805 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-11T22:41:55.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-11T22:41:55.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-11T22:41:55.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.806 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-11T22:41:55.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-11T22:41:55.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.808 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-11T22:41:55.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-11T22:41:55.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.809 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.810 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-11T22:41:55.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-11T22:41:55.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.810 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.812 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-11T22:41:55.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-11T22:41:55.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.812 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-11T22:41:55.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-11T22:41:55.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.813 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.813 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-11T22:41:55.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-11T22:41:55.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.814 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.815 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-11T22:41:55.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-11T22:41:55.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.817 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-11T22:41:55.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-11T22:41:55.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.819 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.819 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-11T22:41:55.819 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.819 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.819 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.819 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-11T22:41:55.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-11T22:41:55.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.820 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T22:41:55.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.821 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-11T22:41:55.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-11T22:41:55.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-11T22:41:55.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.823 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.824 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.824 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.824 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.824 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.824 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.825 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.825 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.825 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.827 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.828 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.828 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.829 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.829 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.829 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.829 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.829 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.829 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.830 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.832 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.832 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-11T22:41:55.832 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.832 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.832 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.833 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.834 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.834 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.834 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.834 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.834 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.834 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.834 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.835 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.836 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.837 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.838 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.838 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.838 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.838 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.838 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.839 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.839 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.839 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.840 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.840 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.840 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.840 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.841 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.841 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-11T22:41:55.842 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.842 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.842 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.842 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.843 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.843 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.843 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.844 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.844 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.844 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.844 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.845 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.846 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.847 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.847 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.848 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.848 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.848 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.848 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.848 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.849 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-11T22:41:55.851 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.851 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.851 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.851 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-11T22:41:55.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-11T22:41:55.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T22:41:55.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.853 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:55.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:55.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-11T22:41:55.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.855 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.856 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.856 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.856 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.856 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.857 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.857 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.857 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.857 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.859 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.860 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.860 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.860 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.860 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.860 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.861 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.861 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.861 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.861 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.861 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.862 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.864 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-11T22:41:55.864 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.864 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.864 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.864 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-11T22:41:55.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-11T22:41:55.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T22:41:55.866 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.866 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-11T22:41:55.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-11T22:41:55.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T22:41:55.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.868 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-11T22:41:55.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-11T22:41:55.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T22:41:55.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.869 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-11T22:41:55.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-11T22:41:55.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T22:41:55.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.871 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-11T22:41:55.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-11T22:41:55.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T22:41:55.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.873 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-11T22:41:55.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-11T22:41:55.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.875 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-11T22:41:55.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-11T22:41:55.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.876 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-11T22:41:55.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-11T22:41:55.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T22:41:55.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.878 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-11T22:41:55.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-11T22:41:55.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.880 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:55.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:55.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-11T22:41:55.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-11T22:41:55.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.881 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.881 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-11T22:41:55.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-11T22:41:55.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.882 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.882 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.882 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-11T22:41:55.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.883 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-11T22:41:55.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-11T22:41:55.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.885 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-11T22:41:55.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-11T22:41:55.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.887 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.887 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.888 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.888 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.888 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.888 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.888 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.888 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.888 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.889 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.891 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.892 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.892 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.892 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.892 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.892 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.893 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.893 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.893 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.895 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-11T22:41:55.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-11T22:41:55.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.896 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.896 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.897 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-11T22:41:55.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-11T22:41:55.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.898 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.898 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.898 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-11T22:41:55.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-11T22:41:55.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.899 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T22:41:55.899 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.899 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-11T22:41:55.901 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.901 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.901 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.901 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.902 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.902 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.902 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.902 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.902 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.903 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.903 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.903 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.903 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.906 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.906 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.907 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.907 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.907 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.907 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.907 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.908 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.909 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.909 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.910 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-11T22:41:55.910 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.910 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.910 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.910 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.912 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.912 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.912 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.912 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.912 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.912 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.912 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.913 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.914 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.915 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.915 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.916 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.916 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.916 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.916 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.916 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.917 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.917 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.917 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-11T22:41:55.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-11T22:41:55.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-11T22:41:55.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T22:41:55.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.921 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-11T22:41:55.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-11T22:41:55.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-11T22:41:55.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.923 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-11T22:41:55.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-11T22:41:55.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-11T22:41:55.923 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T22:41:55.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.924 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-11T22:41:55.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T22:41:55.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-11T22:41:55.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.925 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T22:41:55.926 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.926 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.926 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-11T22:41:55.927 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-11T22:41:55.927 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-11T22:41:55.927 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-11T22:41:55.927 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-11T22:41:55.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-11T22:41:55.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-11T22:41:55.927 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-11T22:41:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-11T22:41:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-11T22:41:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-11T22:41:55.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-11T22:41:55.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-11T22:41:55.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-11T22:41:55.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.932 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-11T22:41:55.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-11T22:41:55.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.933 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.933 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.934 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.934 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.934 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.934 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.934 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.935 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-11T22:41:55.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-11T22:41:55.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.937 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-11T22:41:55.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-11T22:41:55.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-11T22:41:55.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-11T22:41:55.938 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-11T22:41:55.938 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-11T22:41:55.938 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-11T22:41:55.938 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-11T22:41:55.938 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-11T22:41:55.938 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-11T22:41:55.938 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.939 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-11T22:41:55.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-11T22:41:55.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-11T22:41:55.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-11T22:41:55.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-11T22:41:55.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.941 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-11T22:41:55.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-11T22:41:55.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-11T22:41:55.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.942 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.943 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-11T22:41:55.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-11T22:41:55.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-11T22:41:55.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-11T22:41:55.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.944 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-11T22:41:55.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-11T22:41:55.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-11T22:41:55.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-11T22:41:55.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-11T22:41:55.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T22:41:55.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-11T22:41:55.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-11T22:41:55.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-11T22:41:55.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-11T22:41:55.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-11T22:41:55.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-11T22:41:55.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-11T22:41:55.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.946 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.946 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2024-09-11T22:41:55.947 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2024-09-11T22:41:55.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.947 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2024-09-11T22:41:55.947 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2024-09-11T22:41:55.947 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-11T22:41:55.947 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-11T22:41:55.947 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-11T22:41:55.947 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-11T22:41:55.948 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-11T22:41:55.948 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-11T22:41:55.948 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-11T22:41:55.948 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-11T22:41:55.948 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-11T22:41:55.948 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-11T22:41:55.948 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.948 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2024-09-11T22:41:55.949 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-11T22:41:55.949 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-11T22:41:55.949 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-11T22:41:55.949 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-11T22:41:55.949 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-11T22:41:55.949 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-11T22:41:55.949 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-11T22:41:55.949 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-11T22:41:55.950 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-11T22:41:55.950 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-11T22:41:55.950 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.950 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2024-09-11T22:41:55.950 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-11T22:41:55.950 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-11T22:41:55.950 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-11T22:41:55.950 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-11T22:41:55.950 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-11T22:41:55.951 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-11T22:41:55.951 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-11T22:41:55.951 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-11T22:41:55.951 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-11T22:41:55.951 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-11T22:41:55.951 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.951 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2024-09-11T22:41:55.951 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-11T22:41:55.952 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-11T22:41:55.952 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-11T22:41:55.952 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-11T22:41:55.952 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-11T22:41:55.952 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-11T22:41:55.952 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-11T22:41:55.952 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-11T22:41:55.952 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-11T22:41:55.953 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-11T22:41:55.953 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-11T22:41:55.953 INFO:teuthology.orchestra.run.smithi005.stdout: "squid": [ 2024-09-11T22:41:55.953 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-11T22:41:55.953 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-11T22:41:55.953 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-11T22:41:55.953 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-11T22:41:55.953 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-11T22:41:55.954 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-11T22:41:55.954 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-11T22:41:55.954 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-11T22:41:55.954 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-11T22:41:55.954 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-11T22:41:55.954 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:55.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.954 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 83, 2024-09-11T22:41:55.955 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2024-09-11T22:41:55.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-11T22:41:55.955 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-11T22:41:55.955 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-11T22:41:55.955 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-11T22:41:55.955 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4140511043 2024-09-11T22:41:55.955 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.956 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:55.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2024-09-11T22:41:55.956 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-11T22:41:55.956 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-11T22:41:55.956 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-11T22:41:55.957 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 325757539 2024-09-11T22:41:55.957 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.957 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:55.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-11T22:41:55.957 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-11T22:41:55.957 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-11T22:41:55.958 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-11T22:41:55.958 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2412109342 2024-09-11T22:41:55.958 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.958 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:55.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.958 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-11T22:41:55.959 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-11T22:41:55.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-11T22:41:55.959 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-11T22:41:55.959 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2532980049 2024-09-11T22:41:55.959 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.959 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:55.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-11T22:41:55.960 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-11T22:41:55.960 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-11T22:41:55.960 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-11T22:41:55.960 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 401272265 2024-09-11T22:41:55.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.961 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:55.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:55.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-11T22:41:55.961 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-11T22:41:55.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-11T22:41:55.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-11T22:41:55.961 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-11T22:41:55.962 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3781485743 2024-09-11T22:41:55.962 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.962 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:55.962 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:55.962 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-11T22:41:55.962 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:41:55.963 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:53 smithi172 ceph-mon[35461]: pgmap v627: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-11T22:41:55.964 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:53 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/1883269689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:41:55.964 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:53 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4095225491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:41:55.964 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:53 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3735106584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:41:55.964 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:53 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4064000788' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T22:41:55.964 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:53 smithi159 ceph-mon[35534]: pgmap v627: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-11T22:41:55.964 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:53 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/1883269689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:41:55.964 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:53 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4095225491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:41:55.965 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:53 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3735106584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:41:55.965 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:53 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4064000788' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T22:41:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 -- 172.21.15.5:0/4064000788 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2d00 msgr2=0x7f425c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2d00 0x7f425c0a51c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f426c006fd0 tx=0x7f426c008040 comp rx=0 tx=0).stop 2024-09-11T22:41:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 -- 172.21.15.5:0/4064000788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270106990 msgr2=0x7f42700fd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:41:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270106990 0x7f42700fd8d0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f426800eaa0 tx=0x7f426800ef60 comp rx=0 tx=0).stop 2024-09-11T22:41:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 -- 172.21.15.5:0/4064000788 shutdown_connections 2024-09-11T22:41:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4270106990 0x7f42700fd8d0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4270101320 0x7f42700fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f425c0a2d00 0x7f425c0a51c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 --2- 172.21.15.5:0/4064000788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4270100980 0x7f42700fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:41:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 -- 172.21.15.5:0/4064000788 >> 172.21.15.5:0/4064000788 conn(0x7f4270077a10 msgr2=0x7f4270069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:41:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 -- 172.21.15.5:0/4064000788 shutdown_connections 2024-09-11T22:41:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:41:53.680+0000 7f4278035640 1 -- 172.21.15.5:0/4064000788 wait complete. 2024-09-11T22:41:55.973 DEBUG:tasks.check_counter:Getting stats from a 2024-09-11T22:41:55.973 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-11T22:41:55.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:53 smithi005 ceph-mon[30943]: pgmap v627: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 15 op/s 2024-09-11T22:41:55.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:53 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/1883269689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T22:41:55.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:53 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4095225491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-11T22:41:55.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:53 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3735106584' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-11T22:41:55.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:53 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4064000788' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-11T22:41:56.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:55 smithi005 ceph-mon[30943]: pgmap v628: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 87 KiB/s wr, 18 op/s 2024-09-11T22:41:56.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:55 smithi159 ceph-mon[35534]: pgmap v628: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 87 KiB/s wr, 18 op/s 2024-09-11T22:41:56.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:55 smithi172 ceph-mon[35461]: pgmap v628: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 87 KiB/s wr, 18 op/s 2024-09-11T22:41:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:56.760 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:41:56.760 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:41:56.761 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1332, 2024-09-11T22:41:56.761 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1334, 2024-09-11T22:41:56.761 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 444517, 2024-09-11T22:41:56.761 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 423577, 2024-09-11T22:41:56.761 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-11T22:41:56.761 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-09-11T22:41:56.761 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.131838438, 2024-09-11T22:41:56.761 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.043626974, 2024-09-11T22:41:56.762 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.033616974, 2024-09-11T22:41:56.762 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.041084666, 2024-09-11T22:41:56.762 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:41:56.762 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1332, 2024-09-11T22:41:56.762 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.058520593, 2024-09-11T22:41:56.762 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000043934 2024-09-11T22:41:56.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.762 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:41:56.763 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.763 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.763 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.763 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:41:56.763 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:41:56.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.763 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:41:56.764 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1187, 2024-09-11T22:41:56.764 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1449, 2024-09-11T22:41:56.764 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 294570, 2024-09-11T22:41:56.764 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1012506, 2024-09-11T22:41:56.764 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 8, 2024-09-11T22:41:56.764 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2024-09-11T22:41:56.764 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.115712857, 2024-09-11T22:41:56.764 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.040905779, 2024-09-11T22:41:56.764 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.025881154, 2024-09-11T22:41:56.765 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.037657647, 2024-09-11T22:41:56.765 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:41:56.765 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1447, 2024-09-11T22:41:56.765 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.053116064, 2024-09-11T22:41:56.765 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000036707 2024-09-11T22:41:56.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.765 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:41:56.765 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.766 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.766 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.766 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-11T22:41:56.766 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 636880 2024-09-11T22:41:56.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.766 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:41:56.766 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1777, 2024-09-11T22:41:56.767 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1736, 2024-09-11T22:41:56.767 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 5270165, 2024-09-11T22:41:56.767 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 650226, 2024-09-11T22:41:56.767 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 3, 2024-09-11T22:41:56.767 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-09-11T22:41:56.767 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.162984024, 2024-09-11T22:41:56.767 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.056338223, 2024-09-11T22:41:56.767 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.078809116, 2024-09-11T22:41:56.767 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.030667166, 2024-09-11T22:41:56.768 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:41:56.768 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1735, 2024-09-11T22:41:56.768 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.062465972, 2024-09-11T22:41:56.768 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000036003 2024-09-11T22:41:56.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.768 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:41:56.768 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.768 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.769 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.769 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1719472, 2024-09-11T22:41:56.769 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 223152 2024-09-11T22:41:56.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.769 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-11T22:41:56.769 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-09-11T22:41:56.769 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-11T22:41:56.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.770 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-09-11T22:41:56.770 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-11T22:41:56.770 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-11T22:41:56.770 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 6390, 2024-09-11T22:41:56.770 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.163528021, 2024-09-11T22:41:56.770 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000025591 2024-09-11T22:41:56.770 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:56.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.771 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-09-11T22:41:56.771 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-11T22:41:56.771 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-11T22:41:56.771 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.771 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.771 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.771 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:56.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.772 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-09-11T22:41:56.772 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-09-11T22:41:56.772 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-09-11T22:41:56.772 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-09-11T22:41:56.772 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.772 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.773 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.773 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-09-11T22:41:56.773 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-09-11T22:41:56.773 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-09-11T22:41:56.773 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-09-11T22:41:56.773 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-09-11T22:41:56.773 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-09-11T22:41:56.774 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-09-11T22:41:56.774 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 302, 2024-09-11T22:41:56.774 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 0, 2024-09-11T22:41:56.774 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 302, 2024-09-11T22:41:56.774 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-09-11T22:41:56.774 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 302, 2024-09-11T22:41:56.774 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-09-11T22:41:56.774 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-09-11T22:41:56.775 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-09-11T22:41:56.775 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 3, 2024-09-11T22:41:56.775 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-09-11T22:41:56.775 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-09-11T22:41:56.775 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-09-11T22:41:56.775 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-09-11T22:41:56.775 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:41:56.775 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-09-11T22:41:56.775 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-09-11T22:41:56.776 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-09-11T22:41:56.776 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-09-11T22:41:56.776 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-09-11T22:41:56.776 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-09-11T22:41:56.776 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-09-11T22:41:56.776 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-09-11T22:41:56.776 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-09-11T22:41:56.776 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:41:56.777 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-09-11T22:41:56.777 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-09-11T22:41:56.777 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-09-11T22:41:56.777 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-09-11T22:41:56.777 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:41:56.777 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-09-11T22:41:56.777 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:41:56.777 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:41:56.778 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:41:56.778 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:41:56.778 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:41:56.778 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-09-11T22:41:56.778 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:41:56.778 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:41:56.778 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:41:56.778 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:41:56.778 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:41:56.779 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:41:56.779 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-09-11T22:41:56.779 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:41:56.779 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-09-11T22:41:56.779 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-09-11T22:41:56.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.779 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-09-11T22:41:56.779 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-09-11T22:41:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-09-11T22:41:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-09-11T22:41:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-09-11T22:41:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-09-11T22:41:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-09-11T22:41:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-09-11T22:41:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:41:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-09-11T22:41:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-09-11T22:41:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-09-11T22:41:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-09-11T22:41:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-09-11T22:41:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:41:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:41:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-09-11T22:41:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-09-11T22:41:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:41:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:41:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:41:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-09-11T22:41:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-09-11T22:41:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:41:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-09-11T22:41:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-09-11T22:41:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.783 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-09-11T22:41:56.783 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-09-11T22:41:56.783 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-09-11T22:41:56.783 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-09-11T22:41:56.783 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-09-11T22:41:56.783 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 2100, 2024-09-11T22:41:56.783 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-09-11T22:41:56.783 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-09-11T22:41:56.783 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-09-11T22:41:56.784 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-09-11T22:41:56.784 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-09-11T22:41:56.784 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 4, 2024-09-11T22:41:56.784 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 2, 2024-09-11T22:41:56.784 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-09-11T22:41:56.784 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-09-11T22:41:56.784 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-09-11T22:41:56.784 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 7394967, 2024-09-11T22:41:56.785 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 7394393, 2024-09-11T22:41:56.785 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-09-11T22:41:56.785 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.785 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.785 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.785 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 2100 2024-09-11T22:41:56.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.785 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-09-11T22:41:56.786 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 51, 2024-09-11T22:41:56.786 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 301, 2024-09-11T22:41:56.786 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 250, 2024-09-11T22:41:56.786 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 14, 2024-09-11T22:41:56.786 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 14, 2024-09-11T22:41:56.786 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 0, 2024-09-11T22:41:56.786 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 302, 2024-09-11T22:41:56.786 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 302, 2024-09-11T22:41:56.787 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 0, 2024-09-11T22:41:56.787 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-09-11T22:41:56.787 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-09-11T22:41:56.787 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-09-11T22:41:56.787 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 49184, 2024-09-11T22:41:56.787 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215328 2024-09-11T22:41:56.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.787 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-09-11T22:41:56.787 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-09-11T22:41:56.788 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-09-11T22:41:56.788 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-09-11T22:41:56.788 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-09-11T22:41:56.788 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-09-11T22:41:56.788 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-09-11T22:41:56.788 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.788 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.788 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.789 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-09-11T22:41:56.789 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.789 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.789 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.789 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-09-11T22:41:56.789 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.790 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.790 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.790 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-09-11T22:41:56.790 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.790 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.790 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.790 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-09-11T22:41:56.791 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.791 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.791 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.791 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-09-11T22:41:56.791 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.791 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.791 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.792 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-09-11T22:41:56.792 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.792 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.792 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.792 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-09-11T22:41:56.792 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.792 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.793 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.793 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-09-11T22:41:56.793 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.793 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.793 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.793 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-09-11T22:41:56.794 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.794 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.794 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.794 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-09-11T22:41:56.794 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.794 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.794 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.795 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-09-11T22:41:56.795 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.795 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.796 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.796 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-09-11T22:41:56.796 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.796 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.796 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.796 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-09-11T22:41:56.797 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.797 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.797 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.797 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-09-11T22:41:56.797 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.797 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.797 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.798 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-09-11T22:41:56.798 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.798 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.798 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.799 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-09-11T22:41:56.799 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.799 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.799 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.799 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-09-11T22:41:56.799 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.799 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.800 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.800 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-09-11T22:41:56.800 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.800 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.800 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.800 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-09-11T22:41:56.801 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.801 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.801 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.801 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-09-11T22:41:56.801 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.801 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.801 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.802 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-09-11T22:41:56.802 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.802 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.802 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.802 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-09-11T22:41:56.802 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.802 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.803 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.803 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-09-11T22:41:56.803 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.803 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.803 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.803 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-09-11T22:41:56.804 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.804 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.804 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.804 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-09-11T22:41:56.804 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.804 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.804 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.805 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-09-11T22:41:56.805 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.805 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.805 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.805 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-09-11T22:41:56.805 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.806 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.806 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.806 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-09-11T22:41:56.806 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.806 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.806 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.807 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:41:56.807 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:41:56.807 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-09-11T22:41:56.807 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.807 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.807 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.807 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:56.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.807 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-09-11T22:41:56.808 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 2, 2024-09-11T22:41:56.808 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 4, 2024-09-11T22:41:56.808 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 2, 2024-09-11T22:41:56.808 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 4, 2024-09-11T22:41:56.808 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-09-11T22:41:56.808 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-09-11T22:41:56.808 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-09-11T22:41:56.808 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 3355, 2024-09-11T22:41:56.809 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-09-11T22:41:56.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.809 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-11T22:41:56.809 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:41:56.809 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-11T22:41:56.809 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:41:56.809 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:41:56.809 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:41:56.810 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:41:56.810 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:41:56.810 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:41:56.810 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:41:56.810 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:41:56.810 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:41:56.810 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:41:56.810 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:41:56.811 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:41:56.811 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:41:56.811 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-11T22:41:56.811 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:41:56.811 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-11T22:41:56.811 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:41:56.811 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:41:56.811 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:41:56.811 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:41:56.812 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:41:56.812 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:41:56.812 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:41:56.812 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-11T22:41:56.812 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:41:56.812 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:41:56.812 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:41:56.813 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-11T22:41:56.813 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-11T22:41:56.813 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-11T22:41:56.813 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:41:56.813 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:41:56.813 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:41:56.813 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:41:56.813 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 191746, 2024-09-11T22:41:56.814 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 96, 2024-09-11T22:41:56.814 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:41:56.814 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-11T22:41:56.814 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-11T22:41:56.814 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-11T22:41:56.814 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:41:56.814 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-11T22:41:56.814 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:41:56.814 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-11T22:41:56.815 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 8512, 2024-09-11T22:41:56.815 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 187, 2024-09-11T22:41:56.815 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:41:56.815 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:41:56.815 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-11T22:41:56.815 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-11T22:41:56.815 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 361968, 2024-09-11T22:41:56.815 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2322, 2024-09-11T22:41:56.816 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-11T22:41:56.816 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-11T22:41:56.816 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-11T22:41:56.816 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-11T22:41:56.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.816 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-09-11T22:41:56.816 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-09-11T22:41:56.816 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-09-11T22:41:56.816 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 3856, 2024-09-11T22:41:56.817 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-09-11T22:41:56.817 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-09-11T22:41:56.817 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 3856, 2024-09-11T22:41:56.817 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-09-11T22:41:56.817 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3856, 2024-09-11T22:41:56.817 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 3.635922413, 2024-09-11T22:41:56.817 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000942925 2024-09-11T22:41:56.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.818 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-09-11T22:41:56.818 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-09-11T22:41:56.818 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3856, 2024-09-11T22:41:56.818 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 3857 2024-09-11T22:41:56.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.818 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 3856, 2024-09-11T22:41:56.818 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 3856, 2024-09-11T22:41:56.818 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-09-11T22:41:56.819 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-09-11T22:41:56.819 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-09-11T22:41:56.819 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 2461, 2024-09-11T22:41:56.819 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-09-11T22:41:56.819 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 1393, 2024-09-11T22:41:56.819 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-09-11T22:41:56.819 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-09-11T22:41:56.819 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-09-11T22:41:56.819 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-09-11T22:41:56.820 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-09-11T22:41:56.820 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-09-11T22:41:56.820 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-09-11T22:41:56.820 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-09-11T22:41:56.820 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-09-11T22:41:56.820 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-09-11T22:41:56.820 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-09-11T22:41:56.820 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-09-11T22:41:56.821 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:41:56.821 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-09-11T22:41:56.821 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:41:56.821 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-09-11T22:41:56.821 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-09-11T22:41:56.821 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-09-11T22:41:56.821 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:41:56.821 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-09-11T22:41:56.822 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:41:56.822 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-09-11T22:41:56.822 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-09-11T22:41:56.822 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-09-11T22:41:56.822 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-09-11T22:41:56.822 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-09-11T22:41:56.822 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-09-11T22:41:56.822 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-09-11T22:41:56.822 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-09-11T22:41:56.823 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-09-11T22:41:56.823 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-09-11T22:41:56.823 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-09-11T22:41:56.823 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-09-11T22:41:56.823 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-09-11T22:41:56.823 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-09-11T22:41:56.823 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-09-11T22:41:56.823 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-09-11T22:41:56.824 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-09-11T22:41:56.824 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 119, 2024-09-11T22:41:56.824 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-09-11T22:41:56.824 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 39, 2024-09-11T22:41:56.824 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 5, 2024-09-11T22:41:56.824 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 6, 2024-09-11T22:41:56.824 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 1, 2024-09-11T22:41:56.824 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-09-11T22:41:56.825 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-09-11T22:41:56.825 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-09-11T22:41:56.825 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-09-11T22:41:56.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.825 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-09-11T22:41:56.825 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-09-11T22:41:56.825 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:41:56.825 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-09-11T22:41:56.825 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-09-11T22:41:56.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.826 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-09-11T22:41:56.826 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-09-11T22:41:56.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.826 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-09-11T22:41:56.826 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-09-11T22:41:56.826 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:41:56.826 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-09-11T22:41:56.827 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-09-11T22:41:56.827 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-09-11T22:41:56.827 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-09-11T22:41:56.827 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-09-11T22:41:56.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.827 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:41:56.827 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:41:56.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-11T22:41:56.827 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:41:56.828 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 4292, 2024-09-11T22:41:56.828 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 5598045, 2024-09-11T22:41:56.828 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:41:56.828 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 4292, 2024-09-11T22:41:56.828 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-11T22:41:56.828 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-11T22:41:56.828 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 4292, 2024-09-11T22:41:56.828 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 5598045, 2024-09-11T22:41:56.829 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:41:56.829 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.829 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.829 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:56.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.829 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-09-11T22:41:56.829 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:41:56.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-11T22:41:56.830 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:41:56.830 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-11T22:41:56.830 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-11T22:41:56.830 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:41:56.830 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-11T22:41:56.830 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 3856, 2024-09-11T22:41:56.830 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 3200668, 2024-09-11T22:41:56.830 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 3856, 2024-09-11T22:41:56.831 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 3200668, 2024-09-11T22:41:56.831 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:41:56.831 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.831 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.831 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:56.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.832 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-09-11T22:41:56.832 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:41:56.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-09-11T22:41:56.832 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:41:56.832 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-11T22:41:56.832 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-11T22:41:56.832 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:41:56.833 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-11T22:41:56.833 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 3856, 2024-09-11T22:41:56.833 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 3856, 2024-09-11T22:41:56.833 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 3856, 2024-09-11T22:41:56.833 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 3856, 2024-09-11T22:41:56.833 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:41:56.833 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.833 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.833 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.834 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:56.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.834 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:41:56.834 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:41:56.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-11T22:41:56.834 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:41:56.834 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-11T22:41:56.834 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-11T22:41:56.835 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:41:56.835 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-11T22:41:56.835 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-11T22:41:56.835 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-11T22:41:56.835 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-11T22:41:56.835 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-11T22:41:56.835 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:41:56.835 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.836 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.836 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.836 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:56.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:56.836 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x558336898b40": { 2024-09-11T22:41:56.836 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:41:56.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-11T22:41:56.836 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:41:56.836 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-11T22:41:56.837 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-11T22:41:56.837 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:41:56.837 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-11T22:41:56.837 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-11T22:41:56.837 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-11T22:41:56.837 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-11T22:41:56.837 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-11T22:41:56.837 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:41:56.838 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:56.838 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:56.838 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:56.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:56.838 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:56.838 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:41:57.892 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-11T22:41:57.892 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-11T22:41:57.893 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-11T22:41:57.893 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-11T22:41:57.893 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-11T22:41:57.893 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:41:57.893 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:41:57.893 DEBUG:tasks.check_counter:Getting stats from d 2024-09-11T22:41:57.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-11T22:41:57.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:57 smithi005 ceph-mon[30943]: pgmap v629: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 12 op/s 2024-09-11T22:41:58.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:57 smithi159 ceph-mon[35534]: pgmap v629: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 12 op/s 2024-09-11T22:41:58.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:57 smithi172 ceph-mon[35461]: pgmap v629: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 48 KiB/s wr, 12 op/s 2024-09-11T22:41:58.201 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:41:58.724 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:41:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:41:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 365, 2024-09-11T22:41:58.724 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 355, 2024-09-11T22:41:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1484320, 2024-09-11T22:41:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 219280, 2024-09-11T22:41:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-11T22:41:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-11T22:41:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.052018827, 2024-09-11T22:41:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.022058710, 2024-09-11T22:41:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.040552256, 2024-09-11T22:41:58.725 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:41:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:41:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 354, 2024-09-11T22:41:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.015781660, 2024-09-11T22:41:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000044580 2024-09-11T22:41:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:41:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:58.726 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1484320, 2024-09-11T22:41:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 219280 2024-09-11T22:41:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:41:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 7, 2024-09-11T22:41:58.727 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 274, 2024-09-11T22:41:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3552, 2024-09-11T22:41:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 284624, 2024-09-11T22:41:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-11T22:41:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-11T22:41:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.021963501, 2024-09-11T22:41:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.014414967, 2024-09-11T22:41:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.004426799, 2024-09-11T22:41:58.728 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:41:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:41:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 272, 2024-09-11T22:41:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.017678266, 2024-09-11T22:41:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000064993 2024-09-11T22:41:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:41:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:58.729 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-09-11T22:41:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 284624 2024-09-11T22:41:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:41:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-09-11T22:41:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-09-11T22:41:58.730 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-09-11T22:41:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-09-11T22:41:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-11T22:41:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-11T22:41:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.000339609, 2024-09-11T22:41:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-09-11T22:41:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000239830, 2024-09-11T22:41:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:41:58.731 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:41:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:41:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:58.732 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:58.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:41:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:41:58.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-11T22:41:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-11T22:41:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-11T22:41:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-11T22:41:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:41:58.736 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-11T22:41:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:41:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:41:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:41:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:41:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:41:58.737 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:41:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:41:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:41:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:41:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:41:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:41:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:41:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:41:58.738 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-11T22:41:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:41:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-11T22:41:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:41:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:41:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:41:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:41:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:41:58.739 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:41:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:41:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-11T22:41:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:41:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:41:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:41:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-11T22:41:58.740 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-11T22:41:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-11T22:41:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:41:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:41:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:41:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:41:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148731, 2024-09-11T22:41:58.741 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 31, 2024-09-11T22:41:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:41:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-11T22:41:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-11T22:41:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-11T22:41:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:41:58.742 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-11T22:41:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:41:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-11T22:41:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-11T22:41:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-11T22:41:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:41:58.743 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:41:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-11T22:41:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-11T22:41:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-11T22:41:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-11T22:41:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-11T22:41:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-11T22:41:58.744 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-11T22:41:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-11T22:41:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:41:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:41:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:41:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-11T22:41:58.745 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:41:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 368, 2024-09-11T22:41:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1432794, 2024-09-11T22:41:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:41:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 368, 2024-09-11T22:41:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-11T22:41:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-11T22:41:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 368, 2024-09-11T22:41:58.746 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1432794, 2024-09-11T22:41:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:41:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:41:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:41:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:41:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:58.747 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:41:58.747 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:41:59.560 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-11T22:41:59.561 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-11T22:41:59.561 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-11T22:41:59.561 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-11T22:41:59.561 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-11T22:41:59.561 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-11T22:41:59.561 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-11T22:41:59.561 DEBUG:tasks.check_counter:Getting stats from g 2024-09-11T22:41:59.562 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-11T22:41:59.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:41:59 smithi005 ceph-mon[30943]: pgmap v630: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-09-11T22:41:59.869 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:42:00.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:41:59 smithi159 ceph-mon[35534]: pgmap v630: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-09-11T22:42:00.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:41:59 smithi172 ceph-mon[35461]: pgmap v630: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 59 KiB/s wr, 19 op/s 2024-09-11T22:42:00.417 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:42:00.418 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:42:00.418 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1686, 2024-09-11T22:42:00.418 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1685, 2024-09-11T22:42:00.418 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 59277916, 2024-09-11T22:42:00.418 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 535007, 2024-09-11T22:42:00.419 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 9, 2024-09-11T22:42:00.419 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 4, 2024-09-11T22:42:00.419 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.189813550, 2024-09-11T22:42:00.419 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.058298147, 2024-09-11T22:42:00.419 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.074060158, 2024-09-11T22:42:00.419 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.042380176, 2024-09-11T22:42:00.419 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:00.419 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1684, 2024-09-11T22:42:00.420 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.076396523, 2024-09-11T22:42:00.420 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000045366 2024-09-11T22:42:00.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.420 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:00.420 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.420 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.420 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.421 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-11T22:42:00.421 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:42:00.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.421 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:42:00.421 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2113, 2024-09-11T22:42:00.421 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2379, 2024-09-11T22:42:00.421 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 42480538, 2024-09-11T22:42:00.422 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1309755, 2024-09-11T22:42:00.422 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 9, 2024-09-11T22:42:00.422 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 7, 2024-09-11T22:42:00.422 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.166388976, 2024-09-11T22:42:00.422 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.046465578, 2024-09-11T22:42:00.422 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.061352908, 2024-09-11T22:42:00.422 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.046134607, 2024-09-11T22:42:00.422 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:00.423 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2376, 2024-09-11T22:42:00.423 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.073506106, 2024-09-11T22:42:00.423 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000030936 2024-09-11T22:42:00.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.423 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:00.423 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.423 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.423 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.424 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T22:42:00.424 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 638400 2024-09-11T22:42:00.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.424 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:42:00.424 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 772, 2024-09-11T22:42:00.424 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 732, 2024-09-11T22:42:00.424 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 53229061, 2024-09-11T22:42:00.424 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 331609, 2024-09-11T22:42:00.425 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-11T22:42:00.425 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 4, 2024-09-11T22:42:00.425 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.112698894, 2024-09-11T22:42:00.425 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.030007019, 2024-09-11T22:42:00.425 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.086244108, 2024-09-11T22:42:00.425 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.007448522, 2024-09-11T22:42:00.425 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:00.425 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 731, 2024-09-11T22:42:00.426 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.026342674, 2024-09-11T22:42:00.426 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000036036 2024-09-11T22:42:00.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.426 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:00.426 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.426 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.426 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.427 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1720016, 2024-09-11T22:42:00.427 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 224432 2024-09-11T22:42:00.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.427 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-11T22:42:00.427 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-09-11T22:42:00.427 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-11T22:42:00.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.428 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-09-11T22:42:00.428 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-11T22:42:00.428 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-11T22:42:00.428 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 6888, 2024-09-11T22:42:00.428 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.150880492, 2024-09-11T22:42:00.428 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000021904 2024-09-11T22:42:00.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:00.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.429 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-09-11T22:42:00.429 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-11T22:42:00.429 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-11T22:42:00.430 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.430 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.430 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:00.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.430 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-09-11T22:42:00.431 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-09-11T22:42:00.431 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-09-11T22:42:00.431 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-09-11T22:42:00.431 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.431 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.431 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.432 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-09-11T22:42:00.432 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-09-11T22:42:00.432 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-09-11T22:42:00.432 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-09-11T22:42:00.433 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-09-11T22:42:00.433 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-09-11T22:42:00.433 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-09-11T22:42:00.433 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 8986, 2024-09-11T22:42:00.433 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 1, 2024-09-11T22:42:00.433 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 8985, 2024-09-11T22:42:00.434 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-09-11T22:42:00.434 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 8985, 2024-09-11T22:42:00.434 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-09-11T22:42:00.434 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-09-11T22:42:00.434 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-09-11T22:42:00.435 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 5, 2024-09-11T22:42:00.435 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-09-11T22:42:00.435 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-09-11T22:42:00.435 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-09-11T22:42:00.435 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-09-11T22:42:00.435 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:42:00.435 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-09-11T22:42:00.435 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-09-11T22:42:00.436 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-09-11T22:42:00.436 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-09-11T22:42:00.436 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-09-11T22:42:00.436 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-09-11T22:42:00.436 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-09-11T22:42:00.436 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-09-11T22:42:00.436 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-09-11T22:42:00.436 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:42:00.437 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-09-11T22:42:00.437 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-09-11T22:42:00.437 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-09-11T22:42:00.437 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-09-11T22:42:00.437 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:42:00.437 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-09-11T22:42:00.438 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:42:00.438 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:42:00.438 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:42:00.438 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:42:00.438 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:42:00.438 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-09-11T22:42:00.438 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:42:00.438 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:42:00.439 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:42:00.439 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:42:00.439 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:42:00.439 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:42:00.439 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-09-11T22:42:00.439 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:42:00.439 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-09-11T22:42:00.439 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-09-11T22:42:00.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.440 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-09-11T22:42:00.440 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-09-11T22:42:00.440 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-09-11T22:42:00.440 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-09-11T22:42:00.440 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-09-11T22:42:00.440 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-09-11T22:42:00.440 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-09-11T22:42:00.441 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-09-11T22:42:00.441 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:42:00.441 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-09-11T22:42:00.441 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-09-11T22:42:00.441 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-09-11T22:42:00.441 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-09-11T22:42:00.441 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-09-11T22:42:00.441 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:42:00.442 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:42:00.442 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-09-11T22:42:00.442 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-09-11T22:42:00.442 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:42:00.442 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:42:00.442 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:42:00.442 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-09-11T22:42:00.442 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-09-11T22:42:00.443 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:42:00.443 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-09-11T22:42:00.443 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-09-11T22:42:00.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.443 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-09-11T22:42:00.443 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-09-11T22:42:00.443 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-09-11T22:42:00.443 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-09-11T22:42:00.444 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-09-11T22:42:00.444 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 41422, 2024-09-11T22:42:00.444 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-09-11T22:42:00.444 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-09-11T22:42:00.444 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-09-11T22:42:00.444 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-09-11T22:42:00.444 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-09-11T22:42:00.444 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 64, 2024-09-11T22:42:00.444 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 5, 2024-09-11T22:42:00.445 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-09-11T22:42:00.445 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-09-11T22:42:00.445 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-09-11T22:42:00.445 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 156279757, 2024-09-11T22:42:00.445 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 156279183, 2024-09-11T22:42:00.445 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-09-11T22:42:00.445 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.445 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.446 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.446 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 41422 2024-09-11T22:42:00.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.446 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-09-11T22:42:00.446 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 537, 2024-09-11T22:42:00.446 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 8888, 2024-09-11T22:42:00.446 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 8351, 2024-09-11T22:42:00.446 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 43, 2024-09-11T22:42:00.447 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 73, 2024-09-11T22:42:00.447 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 30, 2024-09-11T22:42:00.447 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 8986, 2024-09-11T22:42:00.447 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 8986, 2024-09-11T22:42:00.447 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 0, 2024-09-11T22:42:00.447 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-09-11T22:42:00.447 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-09-11T22:42:00.447 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-09-11T22:42:00.448 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 61240, 2024-09-11T22:42:00.449 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215328 2024-09-11T22:42:00.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.449 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-09-11T22:42:00.450 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-09-11T22:42:00.450 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-09-11T22:42:00.450 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-09-11T22:42:00.450 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-09-11T22:42:00.451 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-09-11T22:42:00.451 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-09-11T22:42:00.451 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.451 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.452 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.452 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-09-11T22:42:00.452 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.453 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.453 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.453 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-09-11T22:42:00.454 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.454 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.454 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.455 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-09-11T22:42:00.455 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.455 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.455 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.456 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-09-11T22:42:00.456 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.456 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.457 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.457 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-09-11T22:42:00.457 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.458 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.458 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.458 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-09-11T22:42:00.458 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.459 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.459 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.460 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-09-11T22:42:00.460 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.460 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.460 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.461 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-09-11T22:42:00.461 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.461 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.462 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.462 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-09-11T22:42:00.462 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.463 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.463 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.463 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-09-11T22:42:00.463 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.464 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.464 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.465 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-09-11T22:42:00.466 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.466 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.466 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.466 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-09-11T22:42:00.467 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.467 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.467 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.468 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-09-11T22:42:00.468 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.468 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.468 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.469 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-09-11T22:42:00.469 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.469 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.469 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.470 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-09-11T22:42:00.470 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.470 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.471 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.471 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-09-11T22:42:00.471 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.472 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.472 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.472 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-09-11T22:42:00.472 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.472 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.473 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.473 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-09-11T22:42:00.473 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.473 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.474 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.474 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-09-11T22:42:00.474 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.475 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.475 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.475 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-09-11T22:42:00.475 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.476 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.476 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.476 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-09-11T22:42:00.476 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.476 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.476 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.477 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-09-11T22:42:00.477 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.477 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.477 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.477 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-09-11T22:42:00.478 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.478 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.478 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.478 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-09-11T22:42:00.478 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.479 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.479 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.479 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-09-11T22:42:00.479 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.479 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.480 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.480 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-09-11T22:42:00.480 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.480 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.480 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.481 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-09-11T22:42:00.481 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.481 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.481 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.481 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-09-11T22:42:00.482 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.482 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.482 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.482 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:42:00.482 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:42:00.482 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-09-11T22:42:00.482 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.483 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.483 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.483 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:00.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.483 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-09-11T22:42:00.483 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 2, 2024-09-11T22:42:00.483 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 3, 2024-09-11T22:42:00.483 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 1, 2024-09-11T22:42:00.484 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 2, 2024-09-11T22:42:00.484 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-09-11T22:42:00.484 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-09-11T22:42:00.484 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-09-11T22:42:00.484 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 2269, 2024-09-11T22:42:00.484 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-09-11T22:42:00.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.485 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-11T22:42:00.485 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:42:00.485 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-11T22:42:00.485 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:42:00.485 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:42:00.486 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:42:00.486 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:42:00.486 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:42:00.487 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:42:00.487 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:42:00.487 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:42:00.487 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:42:00.487 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:42:00.488 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:42:00.488 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:42:00.488 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:42:00.488 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-11T22:42:00.488 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:42:00.489 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-11T22:42:00.489 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:42:00.489 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:42:00.489 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:42:00.490 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:42:00.490 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:42:00.490 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:42:00.490 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:42:00.490 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-11T22:42:00.491 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:42:00.491 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:42:00.491 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:42:00.491 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-11T22:42:00.492 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-11T22:42:00.492 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-11T22:42:00.492 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:42:00.492 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:42:00.492 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:42:00.493 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:42:00.493 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 261167, 2024-09-11T22:42:00.493 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 666, 2024-09-11T22:42:00.493 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:42:00.493 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-11T22:42:00.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-11T22:42:00.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-11T22:42:00.494 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:42:00.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-11T22:42:00.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:42:00.495 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-11T22:42:00.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 8512, 2024-09-11T22:42:00.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 187, 2024-09-11T22:42:00.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:42:00.496 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:42:00.497 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-11T22:42:00.497 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-11T22:42:00.497 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 7729624, 2024-09-11T22:42:00.497 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 49377, 2024-09-11T22:42:00.497 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-11T22:42:00.498 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-11T22:42:00.498 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-11T22:42:00.498 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-11T22:42:00.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.498 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-09-11T22:42:00.499 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-09-11T22:42:00.499 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-09-11T22:42:00.499 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 4129, 2024-09-11T22:42:00.499 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-09-11T22:42:00.499 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-09-11T22:42:00.500 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 4129, 2024-09-11T22:42:00.500 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-09-11T22:42:00.500 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 4129, 2024-09-11T22:42:00.500 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 3.780920013, 2024-09-11T22:42:00.500 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000915698 2024-09-11T22:42:00.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.501 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-09-11T22:42:00.501 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-09-11T22:42:00.501 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 4129, 2024-09-11T22:42:00.501 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 4130 2024-09-11T22:42:00.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.502 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 4129, 2024-09-11T22:42:00.502 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 4129, 2024-09-11T22:42:00.502 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-09-11T22:42:00.502 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-09-11T22:42:00.503 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-09-11T22:42:00.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 2562, 2024-09-11T22:42:00.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-09-11T22:42:00.503 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 1565, 2024-09-11T22:42:00.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-09-11T22:42:00.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-09-11T22:42:00.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-09-11T22:42:00.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-09-11T22:42:00.504 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-09-11T22:42:00.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-09-11T22:42:00.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-09-11T22:42:00.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-09-11T22:42:00.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-09-11T22:42:00.505 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-09-11T22:42:00.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-09-11T22:42:00.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-09-11T22:42:00.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:42:00.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-09-11T22:42:00.506 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:42:00.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-09-11T22:42:00.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-09-11T22:42:00.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-09-11T22:42:00.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:42:00.507 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-09-11T22:42:00.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:42:00.508 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-09-11T22:42:00.508 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-09-11T22:42:00.508 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-09-11T22:42:00.508 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-09-11T22:42:00.509 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-09-11T22:42:00.509 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-09-11T22:42:00.509 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-09-11T22:42:00.509 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-09-11T22:42:00.509 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-09-11T22:42:00.510 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-09-11T22:42:00.510 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-09-11T22:42:00.510 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-09-11T22:42:00.510 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-09-11T22:42:00.510 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-09-11T22:42:00.510 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-09-11T22:42:00.510 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-09-11T22:42:00.511 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-09-11T22:42:00.511 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 119, 2024-09-11T22:42:00.511 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-09-11T22:42:00.511 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 39, 2024-09-11T22:42:00.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 11, 2024-09-11T22:42:00.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 13, 2024-09-11T22:42:00.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 2, 2024-09-11T22:42:00.512 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-09-11T22:42:00.512 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-09-11T22:42:00.512 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-09-11T22:42:00.512 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-09-11T22:42:00.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.512 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-09-11T22:42:00.512 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-09-11T22:42:00.512 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:42:00.513 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-09-11T22:42:00.513 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-09-11T22:42:00.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.513 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-09-11T22:42:00.513 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-09-11T22:42:00.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.513 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-09-11T22:42:00.514 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-09-11T22:42:00.514 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:42:00.514 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-09-11T22:42:00.514 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-09-11T22:42:00.514 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-09-11T22:42:00.514 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-09-11T22:42:00.515 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-09-11T22:42:00.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.515 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:42:00.515 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:42:00.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-11T22:42:00.515 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:42:00.515 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 4567, 2024-09-11T22:42:00.516 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 154551454, 2024-09-11T22:42:00.516 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:00.516 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 4567, 2024-09-11T22:42:00.516 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-11T22:42:00.516 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-11T22:42:00.516 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 4567, 2024-09-11T22:42:00.516 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 154551454, 2024-09-11T22:42:00.516 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:42:00.517 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.517 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.517 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.517 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:00.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.517 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-09-11T22:42:00.517 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:42:00.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-11T22:42:00.518 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:42:00.518 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-11T22:42:00.518 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-11T22:42:00.518 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:00.518 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:00.518 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 4129, 2024-09-11T22:42:00.518 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 152106698, 2024-09-11T22:42:00.518 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 4129, 2024-09-11T22:42:00.518 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 152106698, 2024-09-11T22:42:00.519 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:42:00.519 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.519 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.519 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:00.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.519 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-09-11T22:42:00.519 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:42:00.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-09-11T22:42:00.520 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:42:00.520 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-11T22:42:00.520 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-11T22:42:00.520 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:00.520 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:00.520 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 4129, 2024-09-11T22:42:00.521 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 4129, 2024-09-11T22:42:00.521 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 4129, 2024-09-11T22:42:00.521 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 4129, 2024-09-11T22:42:00.521 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:42:00.521 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.521 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.521 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.522 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:00.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.522 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:42:00.522 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:42:00.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-11T22:42:00.522 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:42:00.522 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-11T22:42:00.522 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-11T22:42:00.523 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:00.523 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:00.523 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-11T22:42:00.523 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-11T22:42:00.523 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-11T22:42:00.523 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-11T22:42:00.523 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:42:00.523 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.523 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.524 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:00.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:00.524 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x5651df4acb40": { 2024-09-11T22:42:00.524 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:42:00.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-11T22:42:00.524 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:42:00.524 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-11T22:42:00.525 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-11T22:42:00.525 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:00.525 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:00.525 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-11T22:42:00.525 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-11T22:42:00.525 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-11T22:42:00.525 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-11T22:42:00.526 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:42:00.526 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:00.526 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:00.526 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:00.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:00.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:00.526 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:42:01.270 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-11T22:42:01.271 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-11T22:42:01.271 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-11T22:42:01.271 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-11T22:42:01.271 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-11T22:42:01.271 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:42:01.271 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:42:01.271 DEBUG:tasks.check_counter:Getting stats from j 2024-09-11T22:42:01.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-11T22:42:01.533 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:42:01.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:01 smithi005 ceph-mon[30943]: pgmap v631: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-11T22:42:02.047 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-11T22:42:02.047 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:42:02.047 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 366, 2024-09-11T22:42:02.047 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 356, 2024-09-11T22:42:02.048 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1484592, 2024-09-11T22:42:02.048 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 219936, 2024-09-11T22:42:02.048 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-11T22:42:02.048 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-11T22:42:02.048 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.051778991, 2024-09-11T22:42:02.048 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.020969919, 2024-09-11T22:42:02.048 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.041352193, 2024-09-11T22:42:02.048 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:42:02.049 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:02.049 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 355, 2024-09-11T22:42:02.049 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.017524916, 2024-09-11T22:42:02.049 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000049365 2024-09-11T22:42:02.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.049 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:02.049 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:02.049 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:02.050 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:02.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.050 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1484592, 2024-09-11T22:42:02.050 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 219936 2024-09-11T22:42:02.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.050 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:42:02.050 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 3, 2024-09-11T22:42:02.050 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 274, 2024-09-11T22:42:02.051 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 480, 2024-09-11T22:42:02.051 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 284528, 2024-09-11T22:42:02.051 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-11T22:42:02.051 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-11T22:42:02.051 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.021538672, 2024-09-11T22:42:02.051 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.015243256, 2024-09-11T22:42:02.051 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.003017158, 2024-09-11T22:42:02.051 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:42:02.052 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:02.052 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 271, 2024-09-11T22:42:02.052 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.014839698, 2024-09-11T22:42:02.052 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000054759 2024-09-11T22:42:02.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.052 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:02.052 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:02.052 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:02.053 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:02.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.053 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-11T22:42:02.053 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 284528 2024-09-11T22:42:02.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.053 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:42:02.053 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2024-09-11T22:42:02.053 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 4, 2024-09-11T22:42:02.054 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3072, 2024-09-11T22:42:02.054 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 448, 2024-09-11T22:42:02.054 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-11T22:42:02.054 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-11T22:42:02.054 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.002333138, 2024-09-11T22:42:02.054 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000031837, 2024-09-11T22:42:02.054 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002268402, 2024-09-11T22:42:02.054 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-11T22:42:02.054 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:02.055 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3, 2024-09-11T22:42:02.055 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.004085274, 2024-09-11T22:42:02.055 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.001361758 2024-09-11T22:42:02.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.055 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:02.055 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:02.055 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:02.055 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:02.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.056 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-11T22:42:02.056 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:42:02.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.056 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-11T22:42:02.056 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-11T22:42:02.056 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-11T22:42:02.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.057 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-11T22:42:02.057 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:42:02.057 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-11T22:42:02.057 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:42:02.057 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:42:02.057 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:42:02.057 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:42:02.057 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:42:02.058 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:42:02.058 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:42:02.058 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:42:02.058 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:42:02.058 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:42:02.058 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:42:02.058 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:42:02.058 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:42:02.058 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-11T22:42:02.059 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:42:02.059 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-11T22:42:02.059 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:42:02.059 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:42:02.059 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:42:02.059 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:42:02.059 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:42:02.059 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:42:02.060 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:42:02.060 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-11T22:42:02.060 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:42:02.060 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:42:02.060 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:42:02.060 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-11T22:42:02.060 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-11T22:42:02.061 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-11T22:42:02.061 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:42:02.061 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:42:02.061 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:42:02.061 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:42:02.061 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148744, 2024-09-11T22:42:02.061 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-09-11T22:42:02.061 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:42:02.062 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-11T22:42:02.062 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-11T22:42:02.062 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-11T22:42:02.062 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:42:02.062 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-11T22:42:02.062 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:42:02.062 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-11T22:42:02.062 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-11T22:42:02.063 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-11T22:42:02.063 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:42:02.063 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:42:02.063 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-11T22:42:02.063 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-11T22:42:02.063 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-11T22:42:02.063 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-11T22:42:02.063 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-11T22:42:02.064 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-11T22:42:02.064 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-11T22:42:02.064 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-11T22:42:02.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-11T22:42:02.064 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:42:02.064 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-11T22:42:02.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-11T22:42:02.064 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-11T22:42:02.065 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 369, 2024-09-11T22:42:02.065 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1432924, 2024-09-11T22:42:02.065 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:02.065 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 369, 2024-09-11T22:42:02.065 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-11T22:42:02.065 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-11T22:42:02.065 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 369, 2024-09-11T22:42:02.066 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1432924, 2024-09-11T22:42:02.066 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-11T22:42:02.066 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-11T22:42:02.066 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-11T22:42:02.066 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-11T22:42:02.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:02.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-11T22:42:02.066 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-11T22:42:02.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:01 smithi159 ceph-mon[35534]: pgmap v631: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-11T22:42:02.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:01 smithi172 ceph-mon[35461]: pgmap v631: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 12 op/s 2024-09-11T22:42:02.770 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-11T22:42:02.770 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-11T22:42:02.771 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-11T22:42:02.771 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-11T22:42:02.771 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-11T22:42:02.771 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-11T22:42:02.771 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-11T22:42:02.771 DEBUG:tasks.check_counter:Getting stats from b 2024-09-11T22:42:02.772 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-11T22:42:03.112 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:42:03.765 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:03 smithi159 ceph-mon[35534]: pgmap v632: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-11T22:42:04.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:03 smithi005 ceph-mon[30943]: pgmap v632: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-11T22:42:04.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:03 smithi172 ceph-mon[35461]: pgmap v632: 97 pgs: 97 active+clean; 545 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-09-11T22:42:04.116 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-11T22:42:04.117 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:42:04.117 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 498657, 2024-09-11T22:42:04.117 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 840184, 2024-09-11T22:42:04.118 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 99921841, 2024-09-11T22:42:04.118 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 713686415, 2024-09-11T22:42:04.118 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 27, 2024-09-11T22:42:04.118 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-09-11T22:42:04.119 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 36.628824218, 2024-09-11T22:42:04.119 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 16.949023560, 2024-09-11T22:42:04.119 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 19.915451924, 2024-09-11T22:42:04.119 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 1.971595425, 2024-09-11T22:42:04.120 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:04.120 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 840189, 2024-09-11T22:42:04.120 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 55.106224414, 2024-09-11T22:42:04.120 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000065587 2024-09-11T22:42:04.121 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.121 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:04.122 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1210585, 2024-09-11T22:42:04.122 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.233281025, 2024-09-11T22:42:04.122 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000192 2024-09-11T22:42:04.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.122 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-11T22:42:04.122 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 520848 2024-09-11T22:42:04.122 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.122 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:42:04.123 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 629909, 2024-09-11T22:42:04.123 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 883483, 2024-09-11T22:42:04.123 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 203217547, 2024-09-11T22:42:04.123 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 1124687123, 2024-09-11T22:42:04.123 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 30, 2024-09-11T22:42:04.123 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 8, 2024-09-11T22:42:04.123 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 34.252823223, 2024-09-11T22:42:04.123 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 14.667257410, 2024-09-11T22:42:04.123 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 21.161330992, 2024-09-11T22:42:04.124 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 2.659574257, 2024-09-11T22:42:04.124 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:04.124 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 883476, 2024-09-11T22:42:04.124 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 76.560984463, 2024-09-11T22:42:04.124 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000086658 2024-09-11T22:42:04.124 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.124 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:04.124 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1267674, 2024-09-11T22:42:04.125 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.206975545, 2024-09-11T22:42:04.125 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000163 2024-09-11T22:42:04.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.125 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1858160, 2024-09-11T22:42:04.125 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 299040 2024-09-11T22:42:04.125 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.125 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:42:04.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 423337, 2024-09-11T22:42:04.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 795847, 2024-09-11T22:42:04.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 76128099, 2024-09-11T22:42:04.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 640981854, 2024-09-11T22:42:04.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 410, 2024-09-11T22:42:04.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2024-09-11T22:42:04.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 32.988035201, 2024-09-11T22:42:04.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 16.226006750, 2024-09-11T22:42:04.126 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 17.040124494, 2024-09-11T22:42:04.127 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 1.608562597, 2024-09-11T22:42:04.127 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:04.127 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 795844, 2024-09-11T22:42:04.127 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 42.689942075, 2024-09-11T22:42:04.127 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000053641 2024-09-11T22:42:04.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:04.128 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1128229, 2024-09-11T22:42:04.128 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.208852291, 2024-09-11T22:42:04.128 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000185 2024-09-11T22:42:04.128 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 702368, 2024-09-11T22:42:04.128 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 339712 2024-09-11T22:42:04.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.129 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-11T22:42:04.129 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-11T22:42:04.129 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-11T22:42:04.129 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.129 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-11T22:42:04.129 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-11T22:42:04.129 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-11T22:42:04.129 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 152448, 2024-09-11T22:42:04.130 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 179.326449276, 2024-09-11T22:42:04.130 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001176312 2024-09-11T22:42:04.130 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:04.130 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.130 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-11T22:42:04.130 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-11T22:42:04.130 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-11T22:42:04.130 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 96083, 2024-09-11T22:42:04.131 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 15.705013730, 2024-09-11T22:42:04.131 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000163452 2024-09-11T22:42:04.131 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:04.131 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.131 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-11T22:42:04.131 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 411573, 2024-09-11T22:42:04.131 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 404813, 2024-09-11T22:42:04.131 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-11T22:42:04.131 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 404813, 2024-09-11T22:42:04.132 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3158.972494374, 2024-09-11T22:42:04.132 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.007803535 2024-09-11T22:42:04.132 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.132 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-11T22:42:04.132 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 6760, 2024-09-11T22:42:04.132 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 4, 2024-09-11T22:42:04.132 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 1, 2024-09-11T22:42:04.132 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 5845, 2024-09-11T22:42:04.133 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 45, 2024-09-11T22:42:04.133 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 33, 2024-09-11T22:42:04.133 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 7310, 2024-09-11T22:42:04.133 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 3926, 2024-09-11T22:42:04.133 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 3003, 2024-09-11T22:42:04.133 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 381, 2024-09-11T22:42:04.133 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 1702, 2024-09-11T22:42:04.133 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 156, 2024-09-11T22:42:04.133 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 471, 2024-09-11T22:42:04.134 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 471, 2024-09-11T22:42:04.134 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 34, 2024-09-11T22:42:04.134 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 599470, 2024-09-11T22:42:04.134 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 368663, 2024-09-11T22:42:04.134 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 6750, 2024-09-11T22:42:04.134 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 8, 2024-09-11T22:42:04.134 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:42:04.134 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-11T22:42:04.135 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 3902, 2024-09-11T22:42:04.135 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-11T22:42:04.135 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-11T22:42:04.135 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 10, 2024-09-11T22:42:04.135 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 139, 2024-09-11T22:42:04.135 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 5, 2024-09-11T22:42:04.135 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 43, 2024-09-11T22:42:04.135 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-11T22:42:04.136 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 6343, 2024-09-11T22:42:04.136 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-11T22:42:04.136 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 681, 2024-09-11T22:42:04.136 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 219018171, 2024-09-11T22:42:04.136 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 6, 2024-09-11T22:42:04.136 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 6343, 2024-09-11T22:42:04.136 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-11T22:42:04.136 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:42:04.136 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:42:04.137 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 1107, 2024-09-11T22:42:04.137 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 24, 2024-09-11T22:42:04.137 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 1083, 2024-09-11T22:42:04.137 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 5236, 2024-09-11T22:42:04.137 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 7, 2024-09-11T22:42:04.137 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 35095, 2024-09-11T22:42:04.137 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 200017, 2024-09-11T22:42:04.138 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:42:04.138 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:42:04.138 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:42:04.138 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 113164, 2024-09-11T22:42:04.138 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 43688, 2024-09-11T22:42:04.138 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 19154, 2024-09-11T22:42:04.139 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 275693 2024-09-11T22:42:04.139 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.139 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-11T22:42:04.139 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 427, 2024-09-11T22:42:04.139 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 18, 2024-09-11T22:42:04.139 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 12, 2024-09-11T22:42:04.139 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 14, 2024-09-11T22:42:04.139 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 1826, 2024-09-11T22:42:04.139 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-09-11T22:42:04.140 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-11T22:42:04.140 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:42:04.140 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 51854, 2024-09-11T22:42:04.140 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 51854, 2024-09-11T22:42:04.140 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-11T22:42:04.140 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-11T22:42:04.141 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-11T22:42:04.141 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:42:04.141 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:42:04.141 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-11T22:42:04.141 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-11T22:42:04.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 16, 2024-09-11T22:42:04.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 4007, 2024-09-11T22:42:04.141 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 24, 2024-09-11T22:42:04.142 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 10, 2024-09-11T22:42:04.142 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-11T22:42:04.142 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 78, 2024-09-11T22:42:04.142 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-11T22:42:04.142 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-11T22:42:04.142 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.142 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-11T22:42:04.142 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-11T22:42:04.142 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 263082, 2024-09-11T22:42:04.143 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 260854, 2024-09-11T22:42:04.143 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 260854, 2024-09-11T22:42:04.143 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 2228, 2024-09-11T22:42:04.143 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-11T22:42:04.143 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 6652, 2024-09-11T22:42:04.143 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 426, 2024-09-11T22:42:04.143 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 422, 2024-09-11T22:42:04.143 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 422, 2024-09-11T22:42:04.144 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 4, 2024-09-11T22:42:04.144 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 1, 2024-09-11T22:42:04.144 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-11T22:42:04.144 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 10, 2024-09-11T22:42:04.144 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 993080608, 2024-09-11T22:42:04.144 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 1000292224, 2024-09-11T22:42:04.144 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-09-11T22:42:04.144 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-11T22:42:04.144 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 80886, 2024-09-11T22:42:04.145 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 540.298742254, 2024-09-11T22:42:04.145 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.006679755 2024-09-11T22:42:04.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.145 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-09-11T22:42:04.145 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.145 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-11T22:42:04.145 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 4160, 2024-09-11T22:42:04.145 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 56167, 2024-09-11T22:42:04.146 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 52007, 2024-09-11T22:42:04.146 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 277, 2024-09-11T22:42:04.146 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 550, 2024-09-11T22:42:04.146 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 273, 2024-09-11T22:42:04.146 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 7310, 2024-09-11T22:42:04.146 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 128317, 2024-09-11T22:42:04.146 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 121007, 2024-09-11T22:42:04.146 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 471, 2024-09-11T22:42:04.147 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 58923, 2024-09-11T22:42:04.147 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 58452, 2024-09-11T22:42:04.147 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 125864, 2024-09-11T22:42:04.147 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 231712 2024-09-11T22:42:04.147 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.147 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-11T22:42:04.147 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 598677, 2024-09-11T22:42:04.147 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 13309, 2024-09-11T22:42:04.147 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 411573, 2024-09-11T22:42:04.148 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 6547, 2024-09-11T22:42:04.148 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 26629, 2024-09-11T22:42:04.148 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-11T22:42:04.148 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 52521, 2024-09-11T22:42:04.148 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 781.267668869, 2024-09-11T22:42:04.148 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.014875338 2024-09-11T22:42:04.148 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.148 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-11T22:42:04.149 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 10442, 2024-09-11T22:42:04.149 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 125.233883722, 2024-09-11T22:42:04.149 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.011993285 2024-09-11T22:42:04.149 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.149 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-11T22:42:04.149 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.149 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.149 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.150 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-11T22:42:04.150 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.150 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.150 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.150 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.150 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-11T22:42:04.150 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 124214, 2024-09-11T22:42:04.151 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 527.203170439, 2024-09-11T22:42:04.151 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004244313 2024-09-11T22:42:04.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.151 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-11T22:42:04.151 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.151 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.151 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.151 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.151 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-11T22:42:04.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.152 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.152 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.152 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-11T22:42:04.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.152 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.152 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.153 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-11T22:42:04.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.153 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.153 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.153 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.154 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-11T22:42:04.154 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.154 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.154 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.154 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.154 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-11T22:42:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 20, 2024-09-11T22:42:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.011416420, 2024-09-11T22:42:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000570821 2024-09-11T22:42:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-11T22:42:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 119, 2024-09-11T22:42:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.946735666, 2024-09-11T22:42:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.007955761 2024-09-11T22:42:04.155 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.156 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-11T22:42:04.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.156 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.156 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.156 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-11T22:42:04.156 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 20, 2024-09-11T22:42:04.156 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 7.687801648, 2024-09-11T22:42:04.157 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.384390082 2024-09-11T22:42:04.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.157 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-11T22:42:04.157 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 10, 2024-09-11T22:42:04.157 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.036591242, 2024-09-11T22:42:04.157 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003659124 2024-09-11T22:42:04.157 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.157 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-11T22:42:04.158 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 154634, 2024-09-11T22:42:04.158 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 811.438939951, 2024-09-11T22:42:04.158 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005247480 2024-09-11T22:42:04.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.158 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-11T22:42:04.158 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 10063, 2024-09-11T22:42:04.158 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 149.227411594, 2024-09-11T22:42:04.158 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.014829316 2024-09-11T22:42:04.158 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.159 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-11T22:42:04.159 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.159 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.159 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.159 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.159 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-11T22:42:04.159 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.159 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.160 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.160 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.160 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-11T22:42:04.160 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-09-11T22:42:04.160 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.009134457, 2024-09-11T22:42:04.160 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.009134457 2024-09-11T22:42:04.160 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.160 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-11T22:42:04.160 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 14, 2024-09-11T22:42:04.161 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 9.183620105, 2024-09-11T22:42:04.161 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.655972864 2024-09-11T22:42:04.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.161 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-11T22:42:04.161 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.161 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.161 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.161 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.162 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-11T22:42:04.162 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 217, 2024-09-11T22:42:04.162 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 110.818920027, 2024-09-11T22:42:04.162 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.510686267 2024-09-11T22:42:04.162 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.162 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-11T22:42:04.162 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.162 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.163 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.163 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-11T22:42:04.163 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 676, 2024-09-11T22:42:04.163 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 5.712394534, 2024-09-11T22:42:04.163 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.008450287 2024-09-11T22:42:04.163 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.163 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-11T22:42:04.163 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.164 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.164 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.164 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.164 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-11T22:42:04.164 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8, 2024-09-11T22:42:04.164 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.023560954, 2024-09-11T22:42:04.164 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002945119 2024-09-11T22:42:04.164 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.165 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-11T22:42:04.165 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.165 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.165 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.165 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.165 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-11T22:42:04.165 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 51854, 2024-09-11T22:42:04.165 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 630.171244746, 2024-09-11T22:42:04.166 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.012152799 2024-09-11T22:42:04.166 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.166 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:42:04.166 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:42:04.166 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-11T22:42:04.166 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.166 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.166 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.166 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:04.167 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.167 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-11T22:42:04.167 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 2, 2024-09-11T22:42:04.167 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 6, 2024-09-11T22:42:04.167 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 4, 2024-09-11T22:42:04.167 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 2, 2024-09-11T22:42:04.167 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-11T22:42:04.167 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 22403, 2024-09-11T22:42:04.168 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 11201, 2024-09-11T22:42:04.168 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 1166, 2024-09-11T22:42:04.168 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-11T22:42:04.168 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.168 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-11T22:42:04.168 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:42:04.168 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-11T22:42:04.168 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:42:04.169 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:42:04.169 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:42:04.169 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:42:04.169 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:42:04.169 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:42:04.169 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:42:04.169 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:42:04.169 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:42:04.169 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:42:04.170 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:42:04.170 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:42:04.170 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:42:04.170 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-11T22:42:04.170 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:42:04.170 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-11T22:42:04.170 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:42:04.170 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:42:04.171 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:42:04.171 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:42:04.171 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:42:04.171 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:42:04.171 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:42:04.171 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-11T22:42:04.171 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:42:04.171 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:42:04.171 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:42:04.172 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-11T22:42:04.172 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-11T22:42:04.172 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-11T22:42:04.172 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:42:04.172 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:42:04.172 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:42:04.172 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:42:04.172 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 384660, 2024-09-11T22:42:04.173 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 1834, 2024-09-11T22:42:04.173 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 616, 2024-09-11T22:42:04.173 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 7, 2024-09-11T22:42:04.173 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-11T22:42:04.173 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-11T22:42:04.173 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:42:04.173 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-11T22:42:04.173 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:42:04.173 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-11T22:42:04.174 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8512, 2024-09-11T22:42:04.174 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 187, 2024-09-11T22:42:04.174 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:42:04.174 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:42:04.174 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-11T22:42:04.174 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-11T22:42:04.174 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 15832996, 2024-09-11T22:42:04.174 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 138340, 2024-09-11T22:42:04.175 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-11T22:42:04.175 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-11T22:42:04.175 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-11T22:42:04.175 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-11T22:42:04.175 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.175 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-11T22:42:04.175 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-11T22:42:04.175 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-11T22:42:04.175 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 178205, 2024-09-11T22:42:04.176 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 1052939394, 2024-09-11T22:42:04.176 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 31, 2024-09-11T22:42:04.176 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 178205, 2024-09-11T22:42:04.176 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-11T22:42:04.176 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 178205, 2024-09-11T22:42:04.176 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3774.789064411, 2024-09-11T22:42:04.176 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.021182284 2024-09-11T22:42:04.176 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.177 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-11T22:42:04.177 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-11T22:42:04.177 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 178205, 2024-09-11T22:42:04.177 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 247353 2024-09-11T22:42:04.177 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.177 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 178205, 2024-09-11T22:42:04.177 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 7843, 2024-09-11T22:42:04.177 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 170362, 2024-09-11T22:42:04.177 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-11T22:42:04.178 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-11T22:42:04.178 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 5324, 2024-09-11T22:42:04.178 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 27055, 2024-09-11T22:42:04.178 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 1471, 2024-09-11T22:42:04.178 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 82333, 2024-09-11T22:42:04.178 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 1031, 2024-09-11T22:42:04.179 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-11T22:42:04.179 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-11T22:42:04.179 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-11T22:42:04.179 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 45, 2024-09-11T22:42:04.179 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 53057, 2024-09-11T22:42:04.179 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-11T22:42:04.179 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-11T22:42:04.179 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-11T22:42:04.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 6372, 2024-09-11T22:42:04.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 54110, 2024-09-11T22:42:04.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:42:04.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-11T22:42:04.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:42:04.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-11T22:42:04.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-11T22:42:04.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-11T22:42:04.180 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:42:04.181 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-11T22:42:04.181 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:42:04.181 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-11T22:42:04.181 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-11T22:42:04.181 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-11T22:42:04.181 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-11T22:42:04.181 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-11T22:42:04.181 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-11T22:42:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-11T22:42:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-11T22:42:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-11T22:42:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-11T22:42:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-11T22:42:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-11T22:42:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-11T22:42:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-11T22:42:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-11T22:42:04.183 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-11T22:42:04.183 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-11T22:42:04.183 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 119, 2024-09-11T22:42:04.183 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-11T22:42:04.183 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 39, 2024-09-11T22:42:04.183 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2024-09-11T22:42:04.183 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2024-09-11T22:42:04.183 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-09-11T22:42:04.184 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-11T22:42:04.184 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 12687, 2024-09-11T22:42:04.184 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 10, 2024-09-11T22:42:04.184 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 3858 2024-09-11T22:42:04.184 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.184 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-11T22:42:04.184 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-09-11T22:42:04.184 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:42:04.185 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-11T22:42:04.185 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-11T22:42:04.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.185 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-11T22:42:04.185 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 4067 2024-09-11T22:42:04.185 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.185 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-11T22:42:04.185 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-11T22:42:04.186 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T22:42:04.186 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-11T22:42:04.186 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 6, 2024-09-11T22:42:04.186 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 51854, 2024-09-11T22:42:04.186 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 51854, 2024-09-11T22:42:04.186 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-11T22:42:04.186 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.186 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:42:04.186 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:04.187 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-11T22:42:04.187 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:04.187 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 1551899, 2024-09-11T22:42:04.187 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 239483217, 2024-09-11T22:42:04.187 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:04.187 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 1551899, 2024-09-11T22:42:04.187 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:04.187 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:04.188 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1551899, 2024-09-11T22:42:04.188 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 239483217, 2024-09-11T22:42:04.188 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:04.188 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.188 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.188 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.188 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:04.188 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.189 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-11T22:42:04.189 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:04.189 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-11T22:42:04.189 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:04.189 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-11T22:42:04.189 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-11T22:42:04.189 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:04.189 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:04.190 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 178205, 2024-09-11T22:42:04.190 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 1058188844, 2024-09-11T22:42:04.190 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 178205, 2024-09-11T22:42:04.190 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1058188844, 2024-09-11T22:42:04.190 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:04.190 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.190 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.190 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.190 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:04.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.191 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-11T22:42:04.191 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:04.191 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-11T22:42:04.191 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:04.191 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-11T22:42:04.191 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-11T22:42:04.191 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 178205, 2024-09-11T22:42:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 178205, 2024-09-11T22:42:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 178205, 2024-09-11T22:42:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 178205, 2024-09-11T22:42:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:42:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-11T22:42:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 51854, 2024-09-11T22:42:04.193 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 5249518, 2024-09-11T22:42:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 51854, 2024-09-11T22:42:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1447, 2024-09-11T22:42:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 5249518, 2024-09-11T22:42:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:04.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x556c7f9b2960": { 2024-09-11T22:42:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 60, 2024-09-11T22:42:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-11T22:42:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:04.195 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 263082, 2024-09-11T22:42:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 996097920, 2024-09-11T22:42:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 263082, 2024-09-11T22:42:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 80886, 2024-09-11T22:42:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 996097860, 2024-09-11T22:42:04.196 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:04.197 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:04.197 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-11T22:42:05.175 INFO:tasks.check_counter:Daemon mds.b mds.exported=10 2024-09-11T22:42:05.176 INFO:tasks.check_counter:Daemon mds.b mds.imported=5 2024-09-11T22:42:05.176 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=427 2024-09-11T22:42:05.176 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=18 2024-09-11T22:42:05.176 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-11T22:42:05.177 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=20 2024-09-11T22:42:05.177 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=14 2024-09-11T22:42:05.177 DEBUG:tasks.check_counter:Getting stats from e 2024-09-11T22:42:05.177 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-11T22:42:05.431 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:42:05.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:05 smithi159 ceph-mon[35534]: pgmap v633: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 102 KiB/s wr, 32 op/s 2024-09-11T22:42:05.997 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-11T22:42:05.997 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:42:05.997 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 638, 2024-09-11T22:42:05.997 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 910, 2024-09-11T22:42:05.997 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 254571, 2024-09-11T22:42:05.998 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 1088004, 2024-09-11T22:42:05.998 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 34, 2024-09-11T22:42:05.998 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2024-09-11T22:42:05.998 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.067140971, 2024-09-11T22:42:05.998 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.022954001, 2024-09-11T22:42:05.998 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.026892498, 2024-09-11T22:42:05.998 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.010395832, 2024-09-11T22:42:05.998 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:05.999 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 908, 2024-09-11T22:42:05.999 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.058925348, 2024-09-11T22:42:05.999 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000064895 2024-09-11T22:42:05.999 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:05.999 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:05.999 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 487, 2024-09-11T22:42:05.999 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000082595, 2024-09-11T22:42:05.999 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000169 2024-09-11T22:42:06.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.000 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 2688, 2024-09-11T22:42:06.000 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 527088 2024-09-11T22:42:06.000 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.000 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:42:06.000 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 35324, 2024-09-11T22:42:06.000 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 18657, 2024-09-11T22:42:06.000 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 14185807, 2024-09-11T22:42:06.001 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 6740716, 2024-09-11T22:42:06.001 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 13, 2024-09-11T22:42:06.001 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2024-09-11T22:42:06.001 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 1.513736985, 2024-09-11T22:42:06.001 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.556278829, 2024-09-11T22:42:06.001 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 1.232226123, 2024-09-11T22:42:06.001 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.028239967, 2024-09-11T22:42:06.001 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:06.002 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 18656, 2024-09-11T22:42:06.002 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.422394676, 2024-09-11T22:42:06.002 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000022641 2024-09-11T22:42:06.002 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.002 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:06.002 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 50938, 2024-09-11T22:42:06.002 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.007194026, 2024-09-11T22:42:06.002 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000141 2024-09-11T22:42:06.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.003 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1745856, 2024-09-11T22:42:06.003 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 243264 2024-09-11T22:42:06.003 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.003 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:42:06.003 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 410469, 2024-09-11T22:42:06.003 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 224356, 2024-09-11T22:42:06.003 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 176731220, 2024-09-11T22:42:06.003 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 38081174, 2024-09-11T22:42:06.004 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 44, 2024-09-11T22:42:06.004 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2024-09-11T22:42:06.004 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 19.292970882, 2024-09-11T22:42:06.004 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 8.461217288, 2024-09-11T22:42:06.004 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 14.697515702, 2024-09-11T22:42:06.004 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.011337233, 2024-09-11T22:42:06.004 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:06.004 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 224354, 2024-09-11T22:42:06.005 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4.038713615, 2024-09-11T22:42:06.005 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000018001 2024-09-11T22:42:06.005 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.005 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:06.005 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 631879, 2024-09-11T22:42:06.005 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.083963474, 2024-09-11T22:42:06.006 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000132 2024-09-11T22:42:06.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.006 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 5888, 2024-09-11T22:42:06.006 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 131552 2024-09-11T22:42:06.006 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.006 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-11T22:42:06.006 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-11T22:42:06.006 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-11T22:42:06.007 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.007 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-11T22:42:06.007 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-11T22:42:06.007 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-11T22:42:06.007 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2013, 2024-09-11T22:42:06.007 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.703482955, 2024-09-11T22:42:06.007 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000349469 2024-09-11T22:42:06.007 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:06.008 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.008 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-11T22:42:06.008 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-11T22:42:06.008 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-11T22:42:06.008 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 757, 2024-09-11T22:42:06.008 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.031797167, 2024-09-11T22:42:06.008 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000042004 2024-09-11T22:42:06.008 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:06.009 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.009 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-11T22:42:06.009 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 32280, 2024-09-11T22:42:06.009 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 1407, 2024-09-11T22:42:06.009 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-11T22:42:06.009 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1407, 2024-09-11T22:42:06.010 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 28.980335451, 2024-09-11T22:42:06.010 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.020597253 2024-09-11T22:42:06.010 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.010 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-11T22:42:06.010 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 30873, 2024-09-11T22:42:06.010 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-11T22:42:06.010 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-11T22:42:06.011 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2024-09-11T22:42:06.011 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-09-11T22:42:06.011 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-09-11T22:42:06.011 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 174, 2024-09-11T22:42:06.011 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 66, 2024-09-11T22:42:06.011 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 108, 2024-09-11T22:42:06.011 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-09-11T22:42:06.011 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 80, 2024-09-11T22:42:06.012 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-09-11T22:42:06.012 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 6, 2024-09-11T22:42:06.012 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 6, 2024-09-11T22:42:06.012 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 4, 2024-09-11T22:42:06.012 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 35046, 2024-09-11T22:42:06.012 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 2224, 2024-09-11T22:42:06.012 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 30872, 2024-09-11T22:42:06.012 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 120, 2024-09-11T22:42:06.013 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:42:06.013 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-11T22:42:06.013 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 84, 2024-09-11T22:42:06.013 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-11T22:42:06.013 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-11T22:42:06.013 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-09-11T22:42:06.013 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-09-11T22:42:06.013 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 1, 2024-09-11T22:42:06.014 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 5, 2024-09-11T22:42:06.014 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-11T22:42:06.014 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 47, 2024-09-11T22:42:06.014 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-11T22:42:06.014 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 681, 2024-09-11T22:42:06.014 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 218855980, 2024-09-11T22:42:06.014 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 6, 2024-09-11T22:42:06.014 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 47, 2024-09-11T22:42:06.015 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-11T22:42:06.015 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:42:06.015 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:42:06.015 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:42:06.015 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:42:06.015 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:42:06.015 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 47, 2024-09-11T22:42:06.015 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:42:06.016 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 73, 2024-09-11T22:42:06.016 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 1082, 2024-09-11T22:42:06.016 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:42:06.016 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:42:06.016 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:42:06.016 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 1074, 2024-09-11T22:42:06.016 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 670, 2024-09-11T22:42:06.016 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 108, 2024-09-11T22:42:06.017 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 1384 2024-09-11T22:42:06.017 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.017 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-11T22:42:06.017 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 76, 2024-09-11T22:42:06.017 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 887, 2024-09-11T22:42:06.017 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 495, 2024-09-11T22:42:06.017 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 814, 2024-09-11T22:42:06.017 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 83, 2024-09-11T22:42:06.017 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-09-11T22:42:06.018 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-11T22:42:06.018 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:42:06.018 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 250, 2024-09-11T22:42:06.018 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 250, 2024-09-11T22:42:06.018 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-11T22:42:06.018 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-11T22:42:06.018 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-11T22:42:06.018 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:42:06.019 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:42:06.019 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-11T22:42:06.019 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-11T22:42:06.019 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 16, 2024-09-11T22:42:06.019 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 730, 2024-09-11T22:42:06.019 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:42:06.019 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-09-11T22:42:06.019 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-11T22:42:06.020 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:42:06.020 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-11T22:42:06.020 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-11T22:42:06.020 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.020 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-11T22:42:06.020 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-11T22:42:06.020 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 2104, 2024-09-11T22:42:06.021 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-09-11T22:42:06.021 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-09-11T22:42:06.021 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 2104, 2024-09-11T22:42:06.021 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-11T22:42:06.021 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-09-11T22:42:06.021 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 4, 2024-09-11T22:42:06.021 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-09-11T22:42:06.021 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-09-11T22:42:06.022 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 4, 2024-09-11T22:42:06.022 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 2, 2024-09-11T22:42:06.022 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-11T22:42:06.022 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-09-11T22:42:06.022 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-09-11T22:42:06.022 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 7402067, 2024-09-11T22:42:06.022 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-09-11T22:42:06.023 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-11T22:42:06.023 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 955, 2024-09-11T22:42:06.023 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.420009545, 2024-09-11T22:42:06.023 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003581161 2024-09-11T22:42:06.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.023 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-09-11T22:42:06.023 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.023 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-11T22:42:06.024 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 171, 2024-09-11T22:42:06.024 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 421, 2024-09-11T22:42:06.024 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 250, 2024-09-11T22:42:06.024 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 216, 2024-09-11T22:42:06.024 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 542, 2024-09-11T22:42:06.024 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 326, 2024-09-11T22:42:06.024 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 176, 2024-09-11T22:42:06.024 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 463, 2024-09-11T22:42:06.024 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 287, 2024-09-11T22:42:06.025 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 4, 2024-09-11T22:42:06.025 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 289, 2024-09-11T22:42:06.025 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 285, 2024-09-11T22:42:06.025 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 64672, 2024-09-11T22:42:06.025 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223520 2024-09-11T22:42:06.025 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.025 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-11T22:42:06.025 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 33763, 2024-09-11T22:42:06.026 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-09-11T22:42:06.026 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 32280, 2024-09-11T22:42:06.026 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 164, 2024-09-11T22:42:06.026 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 591, 2024-09-11T22:42:06.026 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-11T22:42:06.026 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 252, 2024-09-11T22:42:06.026 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.731476239, 2024-09-11T22:42:06.027 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.014807445 2024-09-11T22:42:06.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.027 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-11T22:42:06.027 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 24, 2024-09-11T22:42:06.027 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.029288269, 2024-09-11T22:42:06.027 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001220344 2024-09-11T22:42:06.027 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.027 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-11T22:42:06.028 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.028 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.028 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.028 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.028 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-11T22:42:06.028 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.028 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.028 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.029 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-11T22:42:06.029 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 415, 2024-09-11T22:42:06.029 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.549179926, 2024-09-11T22:42:06.029 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001323325 2024-09-11T22:42:06.029 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.029 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-11T22:42:06.029 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.030 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.030 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.030 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-11T22:42:06.030 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1, 2024-09-11T22:42:06.030 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000169041, 2024-09-11T22:42:06.030 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000169041 2024-09-11T22:42:06.030 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.031 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-11T22:42:06.031 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.031 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.031 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.031 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.031 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-11T22:42:06.031 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.031 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.032 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.032 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-11T22:42:06.032 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.032 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.032 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.032 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.032 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-11T22:42:06.033 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.033 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.034 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.034 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-11T22:42:06.034 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.034 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.034 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.034 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.034 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-11T22:42:06.035 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.035 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.035 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.035 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.035 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-11T22:42:06.035 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.035 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.035 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.036 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-11T22:42:06.036 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.036 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.036 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.036 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.036 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-11T22:42:06.037 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 374, 2024-09-11T22:42:06.037 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.213352005, 2024-09-11T22:42:06.037 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000570459 2024-09-11T22:42:06.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.037 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-11T22:42:06.037 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 84, 2024-09-11T22:42:06.037 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.350615955, 2024-09-11T22:42:06.037 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.016078761 2024-09-11T22:42:06.037 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.038 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-11T22:42:06.038 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.038 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.038 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.038 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.038 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-11T22:42:06.038 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.038 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.039 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.039 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-11T22:42:06.039 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.039 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.039 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.039 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.039 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-11T22:42:06.040 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.040 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.040 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.040 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.040 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-11T22:42:06.040 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.040 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.040 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.041 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-11T22:42:06.041 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 7, 2024-09-11T22:42:06.041 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 21.811714731, 2024-09-11T22:42:06.041 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 3.115959247 2024-09-11T22:42:06.041 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.041 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-11T22:42:06.041 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.042 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.042 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.042 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-11T22:42:06.042 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.042 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.042 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.042 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.043 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-11T22:42:06.043 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.043 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.043 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.043 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.044 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-11T22:42:06.044 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.044 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.044 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.044 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.044 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-11T22:42:06.044 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.044 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.045 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.045 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-11T22:42:06.045 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 250, 2024-09-11T22:42:06.045 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.294539285, 2024-09-11T22:42:06.045 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005178157 2024-09-11T22:42:06.045 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.045 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:42:06.046 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:42:06.046 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-11T22:42:06.046 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.046 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.046 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.046 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:06.046 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.046 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-11T22:42:06.047 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 2, 2024-09-11T22:42:06.047 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 4, 2024-09-11T22:42:06.047 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 2, 2024-09-11T22:42:06.047 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 2, 2024-09-11T22:42:06.047 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-11T22:42:06.047 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 438, 2024-09-11T22:42:06.047 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 219, 2024-09-11T22:42:06.047 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 1153, 2024-09-11T22:42:06.048 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-11T22:42:06.048 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.048 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-11T22:42:06.048 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:42:06.048 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-11T22:42:06.048 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:42:06.048 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:42:06.048 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:42:06.048 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:42:06.049 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:42:06.049 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:42:06.049 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:42:06.049 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:42:06.049 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:42:06.049 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:42:06.049 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:42:06.049 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:42:06.050 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:42:06.050 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-11T22:42:06.050 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:42:06.050 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-11T22:42:06.050 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:42:06.050 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:42:06.050 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:42:06.050 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:42:06.051 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:42:06.051 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:42:06.051 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:42:06.051 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-11T22:42:06.051 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:42:06.051 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:42:06.051 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:42:06.052 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-11T22:42:06.052 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-11T22:42:06.052 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-11T22:42:06.052 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:42:06.052 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:42:06.052 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:42:06.052 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:42:06.052 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 329544, 2024-09-11T22:42:06.052 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 248, 2024-09-11T22:42:06.053 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:42:06.053 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-11T22:42:06.053 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-11T22:42:06.053 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-11T22:42:06.053 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:42:06.053 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-11T22:42:06.053 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:42:06.053 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-11T22:42:06.054 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8512, 2024-09-11T22:42:06.054 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 187, 2024-09-11T22:42:06.054 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:42:06.054 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:42:06.054 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-11T22:42:06.054 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-11T22:42:06.054 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 825214, 2024-09-11T22:42:06.055 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 4963, 2024-09-11T22:42:06.055 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-11T22:42:06.055 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-11T22:42:06.055 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-11T22:42:06.055 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-11T22:42:06.055 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.055 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-11T22:42:06.055 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-11T22:42:06.056 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-11T22:42:06.056 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 1660, 2024-09-11T22:42:06.056 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 3282666, 2024-09-11T22:42:06.056 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 33, 2024-09-11T22:42:06.056 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 1660, 2024-09-11T22:42:06.056 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-11T22:42:06.056 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1660, 2024-09-11T22:42:06.057 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 6.869870906, 2024-09-11T22:42:06.057 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004138476 2024-09-11T22:42:06.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.057 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-11T22:42:06.057 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-11T22:42:06.057 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1660, 2024-09-11T22:42:06.057 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1824 2024-09-11T22:42:06.057 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.058 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 1660, 2024-09-11T22:42:06.058 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 136, 2024-09-11T22:42:06.058 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 1524, 2024-09-11T22:42:06.058 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-11T22:42:06.058 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-11T22:42:06.058 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-09-11T22:42:06.058 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2024-09-11T22:42:06.058 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 89, 2024-09-11T22:42:06.059 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 1044, 2024-09-11T22:42:06.059 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 84, 2024-09-11T22:42:06.059 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-11T22:42:06.059 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-11T22:42:06.059 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-11T22:42:06.059 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-09-11T22:42:06.059 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 262, 2024-09-11T22:42:06.059 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-11T22:42:06.060 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-11T22:42:06.060 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-11T22:42:06.060 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 47, 2024-09-11T22:42:06.060 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2024-09-11T22:42:06.060 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:42:06.060 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-11T22:42:06.060 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:42:06.061 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-11T22:42:06.061 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-11T22:42:06.061 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-11T22:42:06.061 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:42:06.061 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-11T22:42:06.061 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:42:06.061 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-11T22:42:06.061 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-11T22:42:06.062 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-11T22:42:06.062 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-11T22:42:06.062 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-11T22:42:06.062 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-11T22:42:06.062 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-11T22:42:06.062 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-11T22:42:06.062 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-11T22:42:06.062 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-11T22:42:06.063 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-11T22:42:06.063 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-11T22:42:06.063 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-11T22:42:06.063 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-11T22:42:06.063 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-11T22:42:06.063 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-11T22:42:06.063 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-11T22:42:06.063 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 119, 2024-09-11T22:42:06.063 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-11T22:42:06.064 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 39, 2024-09-11T22:42:06.064 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2024-09-11T22:42:06.064 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 17, 2024-09-11T22:42:06.064 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 5, 2024-09-11T22:42:06.064 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-11T22:42:06.064 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 189, 2024-09-11T22:42:06.064 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-09-11T22:42:06.065 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 79 2024-09-11T22:42:06.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.065 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-11T22:42:06.065 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-09-11T22:42:06.065 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:42:06.065 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-11T22:42:06.065 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-11T22:42:06.065 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.066 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-11T22:42:06.066 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 747 2024-09-11T22:42:06.066 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.066 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-11T22:42:06.066 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-11T22:42:06.066 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T22:42:06.066 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-11T22:42:06.066 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 6, 2024-09-11T22:42:06.067 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 250, 2024-09-11T22:42:06.067 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 250, 2024-09-11T22:42:06.067 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-11T22:42:06.067 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.067 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:42:06.067 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:06.067 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-11T22:42:06.067 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:06.068 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 446427, 2024-09-11T22:42:06.068 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 150964228, 2024-09-11T22:42:06.068 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:06.068 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 446427, 2024-09-11T22:42:06.068 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:06.068 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:06.068 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 446427, 2024-09-11T22:42:06.068 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 150964228, 2024-09-11T22:42:06.069 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:06.069 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.069 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.069 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.069 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:06.069 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.069 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-11T22:42:06.069 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:06.070 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-11T22:42:06.070 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:06.070 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-11T22:42:06.070 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-11T22:42:06.070 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:06.070 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:06.070 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 1660, 2024-09-11T22:42:06.070 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 3307996, 2024-09-11T22:42:06.071 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1660, 2024-09-11T22:42:06.071 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 3307996, 2024-09-11T22:42:06.071 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:06.071 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.071 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.071 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.071 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:06.071 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.072 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-11T22:42:06.072 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:06.072 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-11T22:42:06.072 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:06.072 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-11T22:42:06.072 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-11T22:42:06.072 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:06.072 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:06.073 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 1660, 2024-09-11T22:42:06.073 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 1660, 2024-09-11T22:42:06.073 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1660, 2024-09-11T22:42:06.073 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1660, 2024-09-11T22:42:06.073 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:06.073 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.073 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.073 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.074 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:06.074 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.074 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:42:06.074 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:06.074 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-11T22:42:06.074 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:06.074 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 250, 2024-09-11T22:42:06.074 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 25330, 2024-09-11T22:42:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 250, 2024-09-11T22:42:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 89, 2024-09-11T22:42:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 25330, 2024-09-11T22:42:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:06.075 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.076 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.076 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.076 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:06.076 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:06.076 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x5605b097e960": { 2024-09-11T22:42:06.076 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:06.076 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-11T22:42:06.076 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:06.077 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 2104, 2024-09-11T22:42:06.077 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 3207763, 2024-09-11T22:42:06.077 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:06.077 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 2104, 2024-09-11T22:42:06.077 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:06.077 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:06.077 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 955, 2024-09-11T22:42:06.077 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 3207763, 2024-09-11T22:42:06.078 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:06.078 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:06.078 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:06.078 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:06.078 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:06.078 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:06.078 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-11T22:42:06.079 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:05 smithi005 ceph-mon[30943]: pgmap v633: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 102 KiB/s wr, 32 op/s 2024-09-11T22:42:06.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:05 smithi172 ceph-mon[35461]: pgmap v633: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 102 KiB/s wr, 32 op/s 2024-09-11T22:42:06.863 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-11T22:42:06.864 INFO:tasks.check_counter:Daemon mds.e mds.imported=1 2024-09-11T22:42:06.864 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=76 2024-09-11T22:42:06.864 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=887 2024-09-11T22:42:06.864 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-11T22:42:06.864 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:42:06.864 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:42:06.864 DEBUG:tasks.check_counter:Getting stats from h 2024-09-11T22:42:06.865 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-11T22:42:07.113 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:42:07.621 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-11T22:42:07.621 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:42:07.622 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 427342, 2024-09-11T22:42:07.622 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 238801, 2024-09-11T22:42:07.622 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 183808380, 2024-09-11T22:42:07.622 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 68440804, 2024-09-11T22:42:07.622 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 11, 2024-09-11T22:42:07.622 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2024-09-11T22:42:07.622 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 19.964611825, 2024-09-11T22:42:07.623 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 8.734714955, 2024-09-11T22:42:07.623 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 14.866710815, 2024-09-11T22:42:07.623 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.165095795, 2024-09-11T22:42:07.623 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:07.623 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 238799, 2024-09-11T22:42:07.623 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4.611864070, 2024-09-11T22:42:07.623 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000019312 2024-09-11T22:42:07.623 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.624 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:07.624 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 650118, 2024-09-11T22:42:07.624 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.093308966, 2024-09-11T22:42:07.624 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000143 2024-09-11T22:42:07.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.624 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 928, 2024-09-11T22:42:07.624 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 976 2024-09-11T22:42:07.624 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.625 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:42:07.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 69541, 2024-09-11T22:42:07.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 91175, 2024-09-11T22:42:07.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 20947190, 2024-09-11T22:42:07.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 54464941, 2024-09-11T22:42:07.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 21, 2024-09-11T22:42:07.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2024-09-11T22:42:07.625 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 3.676550311, 2024-09-11T22:42:07.626 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 1.475693814, 2024-09-11T22:42:07.626 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 2.609851211, 2024-09-11T22:42:07.626 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.065779054, 2024-09-11T22:42:07.626 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:07.626 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 91172, 2024-09-11T22:42:07.626 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.515727142, 2024-09-11T22:42:07.626 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000016624 2024-09-11T22:42:07.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:07.627 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 148584, 2024-09-11T22:42:07.627 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.023153437, 2024-09-11T22:42:07.627 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000155 2024-09-11T22:42:07.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-11T22:42:07.627 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 128064 2024-09-11T22:42:07.627 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.628 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:42:07.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 5360, 2024-09-11T22:42:07.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 5557, 2024-09-11T22:42:07.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 3554247, 2024-09-11T22:42:07.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 15751582, 2024-09-11T22:42:07.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 53, 2024-09-11T22:42:07.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2024-09-11T22:42:07.628 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.407127367, 2024-09-11T22:42:07.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.150515924, 2024-09-11T22:42:07.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.151293403, 2024-09-11T22:42:07.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.102271005, 2024-09-11T22:42:07.629 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:07.629 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5557, 2024-09-11T22:42:07.629 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.176870842, 2024-09-11T22:42:07.629 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000031828 2024-09-11T22:42:07.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.630 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:07.630 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 961, 2024-09-11T22:42:07.630 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000158237, 2024-09-11T22:42:07.630 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000164 2024-09-11T22:42:07.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.630 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1758608, 2024-09-11T22:42:07.630 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 767968 2024-09-11T22:42:07.630 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.631 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-11T22:42:07.631 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-11T22:42:07.631 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-11T22:42:07.631 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.631 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-11T22:42:07.631 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-11T22:42:07.631 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-11T22:42:07.631 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 16268, 2024-09-11T22:42:07.632 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 7.873092388, 2024-09-11T22:42:07.632 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000483961 2024-09-11T22:42:07.632 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:07.632 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.632 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-11T22:42:07.632 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-11T22:42:07.632 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-11T22:42:07.632 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8813, 2024-09-11T22:42:07.633 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.405970539, 2024-09-11T22:42:07.633 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000046064 2024-09-11T22:42:07.633 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:07.633 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.633 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-11T22:42:07.633 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 56503, 2024-09-11T22:42:07.633 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 26122, 2024-09-11T22:42:07.633 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-11T22:42:07.633 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 26122, 2024-09-11T22:42:07.634 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 44.059201568, 2024-09-11T22:42:07.634 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001686670 2024-09-11T22:42:07.634 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.634 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-11T22:42:07.634 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 30381, 2024-09-11T22:42:07.634 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-11T22:42:07.634 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-11T22:42:07.634 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 21, 2024-09-11T22:42:07.635 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 1, 2024-09-11T22:42:07.635 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 1, 2024-09-11T22:42:07.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 250, 2024-09-11T22:42:07.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 128, 2024-09-11T22:42:07.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 112, 2024-09-11T22:42:07.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 10, 2024-09-11T22:42:07.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 67, 2024-09-11T22:42:07.635 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 48, 2024-09-11T22:42:07.636 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 1, 2024-09-11T22:42:07.636 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 1, 2024-09-11T22:42:07.636 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 9, 2024-09-11T22:42:07.636 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 94137, 2024-09-11T22:42:07.636 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 33848, 2024-09-11T22:42:07.636 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 30377, 2024-09-11T22:42:07.636 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 117, 2024-09-11T22:42:07.636 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:42:07.637 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-11T22:42:07.637 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 1465, 2024-09-11T22:42:07.637 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-11T22:42:07.637 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-11T22:42:07.637 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 5, 2024-09-11T22:42:07.637 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 43, 2024-09-11T22:42:07.637 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 4, 2024-09-11T22:42:07.637 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 53, 2024-09-11T22:42:07.638 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-11T22:42:07.638 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 114, 2024-09-11T22:42:07.638 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-11T22:42:07.638 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 681, 2024-09-11T22:42:07.638 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 218855980, 2024-09-11T22:42:07.638 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 6, 2024-09-11T22:42:07.638 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 114, 2024-09-11T22:42:07.638 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-11T22:42:07.639 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:42:07.639 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:42:07.639 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:42:07.639 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:42:07.639 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:42:07.639 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 114, 2024-09-11T22:42:07.640 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:42:07.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 367, 2024-09-11T22:42:07.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 16641, 2024-09-11T22:42:07.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:42:07.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:42:07.640 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:42:07.640 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 5761, 2024-09-11T22:42:07.640 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 2682, 2024-09-11T22:42:07.641 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 1208, 2024-09-11T22:42:07.641 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 26147 2024-09-11T22:42:07.641 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.641 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-11T22:42:07.641 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 451, 2024-09-11T22:42:07.641 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 644, 2024-09-11T22:42:07.641 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 375, 2024-09-11T22:42:07.641 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 560, 2024-09-11T22:42:07.642 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 468, 2024-09-11T22:42:07.642 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-09-11T22:42:07.642 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-11T22:42:07.642 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:42:07.642 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 4394, 2024-09-11T22:42:07.642 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 4394, 2024-09-11T22:42:07.642 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-11T22:42:07.642 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-11T22:42:07.643 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-11T22:42:07.643 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:42:07.643 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:42:07.643 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-11T22:42:07.643 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-11T22:42:07.643 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 16, 2024-09-11T22:42:07.643 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 794, 2024-09-11T22:42:07.643 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:42:07.644 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 13, 2024-09-11T22:42:07.644 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-11T22:42:07.644 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 2, 2024-09-11T22:42:07.644 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-11T22:42:07.644 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-11T22:42:07.644 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.644 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-11T22:42:07.644 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-11T22:42:07.645 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 25392, 2024-09-11T22:42:07.645 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-09-11T22:42:07.645 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-09-11T22:42:07.645 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 25392, 2024-09-11T22:42:07.645 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-11T22:42:07.645 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-09-11T22:42:07.645 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 32, 2024-09-11T22:42:07.645 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-09-11T22:42:07.646 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-09-11T22:42:07.646 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 32, 2024-09-11T22:42:07.646 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 4, 2024-09-11T22:42:07.646 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-11T22:42:07.646 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-09-11T22:42:07.646 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-09-11T22:42:07.647 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 45299905, 2024-09-11T22:42:07.647 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-09-11T22:42:07.647 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-11T22:42:07.647 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 7745, 2024-09-11T22:42:07.647 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 15.533069737, 2024-09-11T22:42:07.647 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002005560 2024-09-11T22:42:07.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.648 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-09-11T22:42:07.648 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.648 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-11T22:42:07.648 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 249, 2024-09-11T22:42:07.648 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 4690, 2024-09-11T22:42:07.648 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 4441, 2024-09-11T22:42:07.648 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 216, 2024-09-11T22:42:07.649 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 426, 2024-09-11T22:42:07.649 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 210, 2024-09-11T22:42:07.649 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 250, 2024-09-11T22:42:07.649 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 4936, 2024-09-11T22:42:07.649 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 4686, 2024-09-11T22:42:07.649 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 1, 2024-09-11T22:42:07.649 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 4547, 2024-09-11T22:42:07.650 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 4546, 2024-09-11T22:42:07.650 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 71528, 2024-09-11T22:42:07.650 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223520 2024-09-11T22:42:07.650 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.650 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-11T22:42:07.650 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 81768, 2024-09-11T22:42:07.650 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 26, 2024-09-11T22:42:07.650 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 56503, 2024-09-11T22:42:07.651 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 1491, 2024-09-11T22:42:07.651 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 10297, 2024-09-11T22:42:07.651 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-11T22:42:07.651 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4399, 2024-09-11T22:42:07.651 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 20.755462871, 2024-09-11T22:42:07.651 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.004718222 2024-09-11T22:42:07.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.652 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-11T22:42:07.652 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8, 2024-09-11T22:42:07.652 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.008370344, 2024-09-11T22:42:07.652 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001046293 2024-09-11T22:42:07.652 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.653 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-11T22:42:07.653 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.653 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.653 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.653 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.653 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-11T22:42:07.653 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.653 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.654 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.654 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-11T22:42:07.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6118, 2024-09-11T22:42:07.654 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 9.033005007, 2024-09-11T22:42:07.654 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001476463 2024-09-11T22:42:07.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.655 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-11T22:42:07.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.655 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.655 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.655 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-11T22:42:07.655 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.656 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.656 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.656 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.656 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-11T22:42:07.656 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.656 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.656 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.657 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-11T22:42:07.657 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.657 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.657 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.657 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.657 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-11T22:42:07.657 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.658 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.658 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.658 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.658 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-11T22:42:07.658 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.658 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.658 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.659 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-11T22:42:07.659 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.659 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.659 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.659 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.659 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-11T22:42:07.659 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.659 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.660 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.661 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.661 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-11T22:42:07.661 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.662 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.662 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.662 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.663 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-11T22:42:07.663 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.663 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.663 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.663 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.664 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-11T22:42:07.664 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 8568, 2024-09-11T22:42:07.664 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.687979734, 2024-09-11T22:42:07.664 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000430436 2024-09-11T22:42:07.664 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.664 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-11T22:42:07.664 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1463, 2024-09-11T22:42:07.664 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3.420162493, 2024-09-11T22:42:07.665 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002337773 2024-09-11T22:42:07.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.665 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-11T22:42:07.665 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.665 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.665 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.665 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.666 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-11T22:42:07.666 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.666 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.666 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.666 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.666 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-11T22:42:07.666 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.667 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.667 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.667 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.667 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-11T22:42:07.667 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.667 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.667 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.668 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.668 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-11T22:42:07.668 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.668 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.669 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.669 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.669 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-11T22:42:07.669 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.669 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.670 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.670 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-11T22:42:07.670 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.670 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.670 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.670 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.670 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-11T22:42:07.671 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1172, 2024-09-11T22:42:07.671 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.400472210, 2024-09-11T22:42:07.671 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000341699 2024-09-11T22:42:07.671 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.671 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-11T22:42:07.671 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.671 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.671 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.672 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-11T22:42:07.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.672 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.672 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.672 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-11T22:42:07.672 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.673 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.673 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.673 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.673 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-11T22:42:07.673 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 4394, 2024-09-11T22:42:07.673 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 6.753748909, 2024-09-11T22:42:07.674 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001537038 2024-09-11T22:42:07.674 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.674 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:42:07.674 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:42:07.674 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-11T22:42:07.674 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.674 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.674 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.675 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:07.675 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.675 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-11T22:42:07.675 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-09-11T22:42:07.675 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 3, 2024-09-11T22:42:07.675 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 2, 2024-09-11T22:42:07.675 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-09-11T22:42:07.676 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-11T22:42:07.676 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 456, 2024-09-11T22:42:07.676 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 456, 2024-09-11T22:42:07.676 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 1192, 2024-09-11T22:42:07.676 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-11T22:42:07.676 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.677 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-11T22:42:07.677 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:42:07.677 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-11T22:42:07.677 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:42:07.677 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:42:07.677 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:42:07.677 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:42:07.677 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:42:07.678 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:42:07.678 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:42:07.678 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:42:07.678 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:42:07.678 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:42:07.678 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:42:07.678 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:42:07.678 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:42:07.679 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-11T22:42:07.679 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:42:07.679 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-11T22:42:07.679 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:42:07.679 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:42:07.679 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:42:07.679 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:42:07.679 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:42:07.680 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:42:07.680 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:42:07.680 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-11T22:42:07.680 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:42:07.680 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:42:07.680 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:42:07.680 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-11T22:42:07.680 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-11T22:42:07.681 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-11T22:42:07.681 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:42:07.681 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:42:07.681 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:42:07.681 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:42:07.681 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 307035, 2024-09-11T22:42:07.681 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 310, 2024-09-11T22:42:07.681 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:42:07.682 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-11T22:42:07.682 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-11T22:42:07.682 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-11T22:42:07.682 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:42:07.682 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-11T22:42:07.682 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:42:07.682 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-11T22:42:07.682 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8512, 2024-09-11T22:42:07.683 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 187, 2024-09-11T22:42:07.683 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:42:07.683 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:42:07.683 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-11T22:42:07.683 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-11T22:42:07.683 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 1125649, 2024-09-11T22:42:07.683 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 7536, 2024-09-11T22:42:07.683 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-11T22:42:07.684 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-11T22:42:07.684 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-11T22:42:07.684 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-11T22:42:07.684 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.684 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-11T22:42:07.684 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-11T22:42:07.684 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-11T22:42:07.685 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 14206, 2024-09-11T22:42:07.685 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 42206836, 2024-09-11T22:42:07.685 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 28, 2024-09-11T22:42:07.685 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 14206, 2024-09-11T22:42:07.685 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-11T22:42:07.685 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 14206, 2024-09-11T22:42:07.685 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 32.434389947, 2024-09-11T22:42:07.686 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002283147 2024-09-11T22:42:07.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.686 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-11T22:42:07.686 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-11T22:42:07.686 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 14206, 2024-09-11T22:42:07.686 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 15055 2024-09-11T22:42:07.686 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.687 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 14206, 2024-09-11T22:42:07.687 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 592, 2024-09-11T22:42:07.687 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 13614, 2024-09-11T22:42:07.687 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-11T22:42:07.687 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-11T22:42:07.687 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-09-11T22:42:07.687 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2024-09-11T22:42:07.687 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 478, 2024-09-11T22:42:07.688 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 8223, 2024-09-11T22:42:07.688 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 323, 2024-09-11T22:42:07.688 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-11T22:42:07.688 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-11T22:42:07.688 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-11T22:42:07.688 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 1, 2024-09-11T22:42:07.688 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 4417, 2024-09-11T22:42:07.688 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-11T22:42:07.689 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-11T22:42:07.689 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-11T22:42:07.689 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 114, 2024-09-11T22:42:07.689 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2024-09-11T22:42:07.689 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:42:07.689 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-11T22:42:07.689 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:42:07.689 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-11T22:42:07.690 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-11T22:42:07.690 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-11T22:42:07.690 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:42:07.690 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-11T22:42:07.690 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:42:07.690 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-11T22:42:07.691 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-11T22:42:07.691 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-11T22:42:07.691 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-11T22:42:07.691 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-11T22:42:07.691 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-11T22:42:07.691 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-11T22:42:07.691 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-11T22:42:07.691 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-11T22:42:07.692 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-11T22:42:07.692 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-11T22:42:07.692 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-11T22:42:07.692 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-11T22:42:07.692 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-11T22:42:07.692 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-11T22:42:07.692 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-11T22:42:07.693 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-11T22:42:07.693 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 119, 2024-09-11T22:42:07.693 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-11T22:42:07.693 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 39, 2024-09-11T22:42:07.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2024-09-11T22:42:07.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2024-09-11T22:42:07.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-09-11T22:42:07.693 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-11T22:42:07.693 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 1175, 2024-09-11T22:42:07.694 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-09-11T22:42:07.694 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 294 2024-09-11T22:42:07.694 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.694 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-11T22:42:07.694 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-09-11T22:42:07.694 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:42:07.694 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-11T22:42:07.695 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-11T22:42:07.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.695 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-11T22:42:07.695 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 811 2024-09-11T22:42:07.695 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.695 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-11T22:42:07.695 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-11T22:42:07.695 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T22:42:07.696 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-11T22:42:07.696 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 6, 2024-09-11T22:42:07.696 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 4394, 2024-09-11T22:42:07.696 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 4394, 2024-09-11T22:42:07.696 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-11T22:42:07.696 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.696 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:42:07.696 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:07.697 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-11T22:42:07.697 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:07.697 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 502239, 2024-09-11T22:42:07.697 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 163079366, 2024-09-11T22:42:07.697 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:07.697 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 502239, 2024-09-11T22:42:07.697 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:07.698 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:07.698 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 502239, 2024-09-11T22:42:07.698 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 163079366, 2024-09-11T22:42:07.698 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:07.698 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.698 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.698 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.699 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:07.699 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.699 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-11T22:42:07.699 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:07.699 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-11T22:42:07.699 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:07.699 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-11T22:42:07.700 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-11T22:42:07.700 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:07.700 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:07.700 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 14206, 2024-09-11T22:42:07.700 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 42651374, 2024-09-11T22:42:07.700 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 14206, 2024-09-11T22:42:07.700 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 42651374, 2024-09-11T22:42:07.701 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:07.701 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.701 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.701 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.701 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:07.701 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.701 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-11T22:42:07.701 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:07.701 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-11T22:42:07.702 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:07.702 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-11T22:42:07.702 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-11T22:42:07.702 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:07.702 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:07.702 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 14206, 2024-09-11T22:42:07.702 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 14206, 2024-09-11T22:42:07.702 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 14206, 2024-09-11T22:42:07.703 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 14206, 2024-09-11T22:42:07.703 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:07.703 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.703 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.703 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.703 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:07.703 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.703 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:42:07.704 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:07.704 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-11T22:42:07.704 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:07.704 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 4394, 2024-09-11T22:42:07.704 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 444538, 2024-09-11T22:42:07.704 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:07.704 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 4394, 2024-09-11T22:42:07.704 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:07.705 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:07.705 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 478, 2024-09-11T22:42:07.705 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 444538, 2024-09-11T22:42:07.705 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:07.705 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.705 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.705 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.706 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:07.706 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:07.706 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x563bcbb48960": { 2024-09-11T22:42:07.706 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:07.706 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-11T22:42:07.706 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:07.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 25392, 2024-09-11T22:42:07.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 41105601, 2024-09-11T22:42:07.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:07.707 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 25392, 2024-09-11T22:42:07.707 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:07.707 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:07.707 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 7745, 2024-09-11T22:42:07.707 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 41105601, 2024-09-11T22:42:07.708 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:07.708 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:07.708 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:07.708 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:07.708 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:07.708 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:07.708 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-11T22:42:07.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:07 smithi159 ceph-mon[35534]: pgmap v634: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 81 KiB/s wr, 27 op/s 2024-09-11T22:42:08.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:07 smithi005 ceph-mon[30943]: pgmap v634: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 81 KiB/s wr, 27 op/s 2024-09-11T22:42:08.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:07 smithi172 ceph-mon[35461]: pgmap v634: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 81 KiB/s wr, 27 op/s 2024-09-11T22:42:08.480 INFO:tasks.check_counter:Daemon mds.h mds.exported=5 2024-09-11T22:42:08.481 INFO:tasks.check_counter:Daemon mds.h mds.imported=4 2024-09-11T22:42:08.481 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=451 2024-09-11T22:42:08.481 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=644 2024-09-11T22:42:08.481 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-11T22:42:08.481 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:42:08.481 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:42:08.481 DEBUG:tasks.check_counter:Getting stats from k 2024-09-11T22:42:08.482 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-11T22:42:08.732 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T22:42:09.234 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-11T22:42:09.235 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:42:09.235 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 418, 2024-09-11T22:42:09.235 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 420, 2024-09-11T22:42:09.235 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 109024119, 2024-09-11T22:42:09.235 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 132930, 2024-09-11T22:42:09.236 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 8, 2024-09-11T22:42:09.236 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-09-11T22:42:09.236 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.086194027, 2024-09-11T22:42:09.236 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.008641810, 2024-09-11T22:42:09.236 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.063982530, 2024-09-11T22:42:09.236 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.009962669, 2024-09-11T22:42:09.236 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:09.237 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 418, 2024-09-11T22:42:09.237 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.018341606, 2024-09-11T22:42:09.237 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000043879 2024-09-11T22:42:09.237 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.237 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:09.237 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.237 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.238 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.238 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:42:09.238 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:42:09.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.239 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:42:09.239 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 969, 2024-09-11T22:42:09.239 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1054, 2024-09-11T22:42:09.239 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 136815774, 2024-09-11T22:42:09.239 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 547781, 2024-09-11T22:42:09.239 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-09-11T22:42:09.240 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2024-09-11T22:42:09.240 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.149263278, 2024-09-11T22:42:09.240 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.028023253, 2024-09-11T22:42:09.240 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.091594805, 2024-09-11T22:42:09.240 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.022928483, 2024-09-11T22:42:09.240 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:09.240 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1053, 2024-09-11T22:42:09.240 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.059389893, 2024-09-11T22:42:09.241 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000056400 2024-09-11T22:42:09.241 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.241 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:09.241 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.241 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.241 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.242 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-11T22:42:09.242 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 241168 2024-09-11T22:42:09.242 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.242 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:42:09.242 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 801, 2024-09-11T22:42:09.242 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 791, 2024-09-11T22:42:09.242 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 189175681, 2024-09-11T22:42:09.243 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 282826, 2024-09-11T22:42:09.243 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 5, 2024-09-11T22:42:09.243 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-09-11T22:42:09.243 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.178721020, 2024-09-11T22:42:09.243 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.017834778, 2024-09-11T22:42:09.243 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.146506154, 2024-09-11T22:42:09.243 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.014722126, 2024-09-11T22:42:09.244 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:09.244 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 790, 2024-09-11T22:42:09.244 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.042794829, 2024-09-11T22:42:09.244 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000054170 2024-09-11T22:42:09.244 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.245 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:09.245 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.245 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.245 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.245 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.245 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 259168, 2024-09-11T22:42:09.245 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 74208 2024-09-11T22:42:09.246 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.246 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-11T22:42:09.246 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-11T22:42:09.246 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-11T22:42:09.247 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.247 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-11T22:42:09.247 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-11T22:42:09.247 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-11T22:42:09.247 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3126, 2024-09-11T22:42:09.248 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.309651679, 2024-09-11T22:42:09.248 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000099056 2024-09-11T22:42:09.248 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:09.248 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.248 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-11T22:42:09.248 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-11T22:42:09.248 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-11T22:42:09.248 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.249 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.249 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.249 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:09.249 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.249 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-11T22:42:09.249 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2024-09-11T22:42:09.249 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2024-09-11T22:42:09.249 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-11T22:42:09.249 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.250 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.250 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.250 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.250 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-11T22:42:09.250 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-09-11T22:42:09.251 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-11T22:42:09.251 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-11T22:42:09.251 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2024-09-11T22:42:09.251 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-09-11T22:42:09.251 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-09-11T22:42:09.251 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 26881, 2024-09-11T22:42:09.252 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 17774, 2024-09-11T22:42:09.252 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 9107, 2024-09-11T22:42:09.252 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-09-11T22:42:09.252 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 1491, 2024-09-11T22:42:09.252 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-09-11T22:42:09.252 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-09-11T22:42:09.252 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-09-11T22:42:09.252 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 13, 2024-09-11T22:42:09.253 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2024-09-11T22:42:09.253 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2024-09-11T22:42:09.253 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-09-11T22:42:09.253 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-09-11T22:42:09.253 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:42:09.253 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-11T22:42:09.253 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-09-11T22:42:09.253 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-11T22:42:09.253 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-11T22:42:09.254 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-09-11T22:42:09.254 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-09-11T22:42:09.254 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-09-11T22:42:09.254 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-09-11T22:42:09.254 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-11T22:42:09.254 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:42:09.254 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-11T22:42:09.254 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 681, 2024-09-11T22:42:09.255 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 219018171, 2024-09-11T22:42:09.255 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 6, 2024-09-11T22:42:09.255 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:42:09.255 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-11T22:42:09.255 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:42:09.255 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:42:09.255 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:42:09.255 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:42:09.256 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:42:09.256 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-09-11T22:42:09.256 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:42:09.256 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:42:09.256 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:42:09.256 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:42:09.256 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:42:09.256 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:42:09.257 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-09-11T22:42:09.257 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:42:09.257 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-09-11T22:42:09.257 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2024-09-11T22:42:09.257 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.257 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-11T22:42:09.257 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-09-11T22:42:09.257 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-09-11T22:42:09.257 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-09-11T22:42:09.258 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-09-11T22:42:09.258 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-09-11T22:42:09.258 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-09-11T22:42:09.258 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-11T22:42:09.258 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:42:09.258 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-09-11T22:42:09.258 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-09-11T22:42:09.258 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-11T22:42:09.259 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-11T22:42:09.259 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-11T22:42:09.259 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:42:09.259 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:42:09.259 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-11T22:42:09.259 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-11T22:42:09.259 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:42:09.259 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:42:09.260 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:42:09.260 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-09-11T22:42:09.260 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-11T22:42:09.260 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:42:09.260 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-11T22:42:09.260 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-11T22:42:09.260 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.260 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-11T22:42:09.261 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-11T22:42:09.261 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2024-09-11T22:42:09.261 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-09-11T22:42:09.261 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-09-11T22:42:09.261 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 119314, 2024-09-11T22:42:09.261 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-11T22:42:09.261 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-09-11T22:42:09.261 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2024-09-11T22:42:09.262 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-09-11T22:42:09.262 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-09-11T22:42:09.262 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 4, 2024-09-11T22:42:09.262 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 27, 2024-09-11T22:42:09.262 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-11T22:42:09.262 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-09-11T22:42:09.262 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 993080608, 2024-09-11T22:42:09.263 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 1000554655, 2024-09-11T22:42:09.263 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 1000547682, 2024-09-11T22:42:09.263 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-11T22:42:09.263 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.263 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.263 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.263 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.264 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 119314 2024-09-11T22:42:09.264 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.264 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-11T22:42:09.264 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 3481, 2024-09-11T22:42:09.264 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 26422, 2024-09-11T22:42:09.264 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 22941, 2024-09-11T22:42:09.265 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 257, 2024-09-11T22:42:09.265 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 402, 2024-09-11T22:42:09.265 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 145, 2024-09-11T22:42:09.265 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 26899, 2024-09-11T22:42:09.265 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 27026, 2024-09-11T22:42:09.265 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 127, 2024-09-11T22:42:09.265 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-09-11T22:42:09.265 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2024-09-11T22:42:09.265 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2024-09-11T22:42:09.266 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 85264, 2024-09-11T22:42:09.266 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 215328 2024-09-11T22:42:09.266 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.267 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-11T22:42:09.267 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2024-09-11T22:42:09.267 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-09-11T22:42:09.267 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2024-09-11T22:42:09.267 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2024-09-11T22:42:09.267 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-09-11T22:42:09.267 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-11T22:42:09.268 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.268 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.269 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.269 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-11T22:42:09.269 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.269 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.269 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.269 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.269 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-11T22:42:09.270 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.270 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.270 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.270 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.270 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-11T22:42:09.270 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.270 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.270 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.271 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-11T22:42:09.271 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.271 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.271 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.271 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.271 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-11T22:42:09.271 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.271 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.272 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.272 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-11T22:42:09.272 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.272 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.272 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.272 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.273 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-11T22:42:09.273 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.273 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.273 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.273 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.273 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-11T22:42:09.273 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.273 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.274 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.274 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-11T22:42:09.274 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.274 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.274 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.274 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.274 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-11T22:42:09.275 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.275 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.275 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.275 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.276 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-11T22:42:09.276 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.276 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.276 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.276 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.276 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-11T22:42:09.276 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.277 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.277 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.277 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-11T22:42:09.277 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.277 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.277 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.277 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.278 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-11T22:42:09.278 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.278 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.278 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.278 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.278 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-11T22:42:09.278 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.278 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.279 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.279 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-11T22:42:09.279 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.279 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.279 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.279 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.279 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-11T22:42:09.280 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.280 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.280 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.280 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.280 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-11T22:42:09.280 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.280 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.280 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.281 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-11T22:42:09.281 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.281 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.281 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.281 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.281 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-11T22:42:09.281 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.282 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.282 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.282 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-11T22:42:09.282 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.282 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.282 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.282 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.283 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-11T22:42:09.283 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.283 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.283 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.283 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.283 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-11T22:42:09.283 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.283 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.284 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.284 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-11T22:42:09.284 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.284 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.284 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.284 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.284 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-11T22:42:09.285 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.285 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.285 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.285 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.285 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-11T22:42:09.285 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.286 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.286 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.286 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.286 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-11T22:42:09.286 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.286 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-11T22:42:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.287 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:42:09.288 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:42:09.288 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-11T22:42:09.288 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.288 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.288 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.288 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:09.288 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.288 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-11T22:42:09.289 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 2, 2024-09-11T22:42:09.289 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 2, 2024-09-11T22:42:09.289 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 0, 2024-09-11T22:42:09.289 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 0, 2024-09-11T22:42:09.289 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-11T22:42:09.289 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2024-09-11T22:42:09.289 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2024-09-11T22:42:09.289 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 2515, 2024-09-11T22:42:09.290 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-11T22:42:09.290 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.290 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-11T22:42:09.290 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:42:09.290 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-11T22:42:09.290 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:42:09.290 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:42:09.290 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:42:09.291 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:42:09.291 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:42:09.291 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:42:09.291 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:42:09.291 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:42:09.291 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:42:09.291 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:42:09.291 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:42:09.292 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:42:09.292 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:42:09.292 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-11T22:42:09.292 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:42:09.292 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-11T22:42:09.292 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:42:09.292 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:42:09.292 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:42:09.292 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:42:09.293 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:42:09.293 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:42:09.293 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:42:09.293 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-11T22:42:09.293 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:42:09.293 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:42:09.293 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:42:09.294 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-11T22:42:09.294 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-11T22:42:09.294 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-11T22:42:09.294 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:42:09.294 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:42:09.294 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:42:09.294 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:42:09.294 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 381413, 2024-09-11T22:42:09.295 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 4170, 2024-09-11T22:42:09.295 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:42:09.295 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-11T22:42:09.295 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-11T22:42:09.295 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-11T22:42:09.295 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:42:09.295 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-11T22:42:09.296 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:42:09.296 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-11T22:42:09.296 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 8512, 2024-09-11T22:42:09.296 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 187, 2024-09-11T22:42:09.296 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:42:09.296 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:42:09.296 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-11T22:42:09.296 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-11T22:42:09.297 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 27869243, 2024-09-11T22:42:09.297 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 199874, 2024-09-11T22:42:09.297 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-11T22:42:09.297 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-11T22:42:09.297 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-11T22:42:09.297 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-11T22:42:09.297 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.297 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-11T22:42:09.297 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-11T22:42:09.298 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-11T22:42:09.298 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 2038, 2024-09-11T22:42:09.298 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 26, 2024-09-11T22:42:09.298 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 3, 2024-09-11T22:42:09.298 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 2038, 2024-09-11T22:42:09.298 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-11T22:42:09.298 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2038, 2024-09-11T22:42:09.298 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4.874908104, 2024-09-11T22:42:09.299 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002392005 2024-09-11T22:42:09.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.299 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-11T22:42:09.299 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-11T22:42:09.299 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2038, 2024-09-11T22:42:09.299 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2039 2024-09-11T22:42:09.299 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.299 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 2038, 2024-09-11T22:42:09.300 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 2038, 2024-09-11T22:42:09.300 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2024-09-11T22:42:09.300 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-11T22:42:09.300 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-11T22:42:09.300 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 1218, 2024-09-11T22:42:09.300 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2024-09-11T22:42:09.300 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 817, 2024-09-11T22:42:09.300 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2024-09-11T22:42:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2024-09-11T22:42:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-11T22:42:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-11T22:42:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-11T22:42:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-09-11T22:42:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2024-09-11T22:42:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-11T22:42:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-11T22:42:09.302 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-11T22:42:09.302 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 2, 2024-09-11T22:42:09.302 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2024-09-11T22:42:09.302 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:42:09.302 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-11T22:42:09.302 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:42:09.302 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-11T22:42:09.302 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-11T22:42:09.303 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-11T22:42:09.303 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:42:09.303 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-11T22:42:09.303 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:42:09.303 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-11T22:42:09.303 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-11T22:42:09.303 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-11T22:42:09.303 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-11T22:42:09.303 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-11T22:42:09.304 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-11T22:42:09.304 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-11T22:42:09.304 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-11T22:42:09.304 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-11T22:42:09.304 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-11T22:42:09.304 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-11T22:42:09.304 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-11T22:42:09.304 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-11T22:42:09.305 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-11T22:42:09.305 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-11T22:42:09.305 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-11T22:42:09.305 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-11T22:42:09.305 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 119, 2024-09-11T22:42:09.305 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-11T22:42:09.305 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 13, 2024-09-11T22:42:09.305 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 10, 2024-09-11T22:42:09.306 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 10, 2024-09-11T22:42:09.306 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-09-11T22:42:09.306 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-11T22:42:09.306 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2024-09-11T22:42:09.306 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2024-09-11T22:42:09.306 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2024-09-11T22:42:09.306 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.307 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-11T22:42:09.307 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2024-09-11T22:42:09.307 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:42:09.307 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-11T22:42:09.307 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-11T22:42:09.307 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.307 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-11T22:42:09.307 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-09-11T22:42:09.308 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.308 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-11T22:42:09.308 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-11T22:42:09.308 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:42:09.308 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-11T22:42:09.308 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-09-11T22:42:09.308 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-09-11T22:42:09.308 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-09-11T22:42:09.309 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-11T22:42:09.309 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.309 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:42:09.309 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:09.309 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-11T22:42:09.309 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:09.309 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 2184, 2024-09-11T22:42:09.309 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 434808603, 2024-09-11T22:42:09.310 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:09.310 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 2184, 2024-09-11T22:42:09.310 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:09.310 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:09.310 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2184, 2024-09-11T22:42:09.310 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 434808603, 2024-09-11T22:42:09.310 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:09.310 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.311 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.311 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.311 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:09.311 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.311 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-11T22:42:09.311 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:09.311 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-11T22:42:09.311 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:09.312 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-11T22:42:09.312 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-11T22:42:09.312 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:09.312 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:09.312 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 2038, 2024-09-11T22:42:09.312 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 434191809, 2024-09-11T22:42:09.312 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2038, 2024-09-11T22:42:09.313 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 434191809, 2024-09-11T22:42:09.313 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:09.313 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.313 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.313 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.313 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:09.313 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.313 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-11T22:42:09.314 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:09.314 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-11T22:42:09.314 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:09.314 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-11T22:42:09.314 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-11T22:42:09.314 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:09.314 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:09.314 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 2038, 2024-09-11T22:42:09.315 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 2038, 2024-09-11T22:42:09.315 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 2038, 2024-09-11T22:42:09.315 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 2038, 2024-09-11T22:42:09.315 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:09.315 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.315 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.315 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.315 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:09.316 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.316 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:42:09.316 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:09.316 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-11T22:42:09.316 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:09.316 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-11T22:42:09.316 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-11T22:42:09.316 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:09.317 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:09.317 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:09.317 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:09.317 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-09-11T22:42:09.317 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-09-11T22:42:09.317 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:09.317 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.317 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.317 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.318 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:09.318 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-11T22:42:09.318 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x5607de2226e0": { 2024-09-11T22:42:09.318 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-11T22:42:09.318 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-11T22:42:09.318 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-11T22:42:09.318 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-11T22:42:09.318 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-11T22:42:09.319 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:09.319 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:09.319 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-11T22:42:09.319 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-11T22:42:09.319 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-09-11T22:42:09.319 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-09-11T22:42:09.319 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-11T22:42:09.319 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-11T22:42:09.319 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-11T22:42:09.320 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-11T22:42:09.320 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:09.320 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-11T22:42:09.320 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-11T22:42:10.003 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:09 smithi159 ceph-mon[35534]: pgmap v635: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 108 KiB/s wr, 34 op/s 2024-09-11T22:42:10.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:09 smithi005 ceph-mon[30943]: pgmap v635: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 108 KiB/s wr, 34 op/s 2024-09-11T22:42:10.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:09 smithi172 ceph-mon[35461]: pgmap v635: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 108 KiB/s wr, 34 op/s 2024-09-11T22:42:10.164 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-11T22:42:10.165 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-11T22:42:10.165 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-11T22:42:10.165 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-11T22:42:10.165 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-09-11T22:42:10.165 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:42:10.165 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:42:10.165 DEBUG:tasks.check_counter:Getting stats from c 2024-09-11T22:42:10.166 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-11T22:42:10.597 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:42:11.065 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:10 smithi172 ceph-mon[35461]: osdmap e120: 12 total, 12 up, 12 in 2024-09-11T22:42:11.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:10 smithi005 ceph-mon[30943]: osdmap e120: 12 total, 12 up, 12 in 2024-09-11T22:42:11.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:10 smithi159 ceph-mon[35534]: osdmap e120: 12 total, 12 up, 12 in 2024-09-11T22:42:11.241 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-11T22:42:11.241 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:42:11.242 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 478112, 2024-09-11T22:42:11.242 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 342648, 2024-09-11T22:42:11.242 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 197234764, 2024-09-11T22:42:11.242 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 188129748, 2024-09-11T22:42:11.242 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 29, 2024-09-11T22:42:11.242 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 9, 2024-09-11T22:42:11.242 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 20.084835851, 2024-09-11T22:42:11.242 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 7.876647821, 2024-09-11T22:42:11.243 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 16.004960733, 2024-09-11T22:42:11.243 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.225942612, 2024-09-11T22:42:11.243 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:11.243 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 342646, 2024-09-11T22:42:11.243 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 7.067363426, 2024-09-11T22:42:11.243 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000020625 2024-09-11T22:42:11.243 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.243 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:11.244 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 791420, 2024-09-11T22:42:11.244 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.108062914, 2024-09-11T22:42:11.244 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000136 2024-09-11T22:42:11.244 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.244 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-11T22:42:11.244 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 0 2024-09-11T22:42:11.244 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.245 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:42:11.245 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 23293, 2024-09-11T22:42:11.245 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 40234, 2024-09-11T22:42:11.245 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 4413441, 2024-09-11T22:42:11.245 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 52405782, 2024-09-11T22:42:11.245 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 47, 2024-09-11T22:42:11.245 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-09-11T22:42:11.245 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 2.103255879, 2024-09-11T22:42:11.246 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 1.035139227, 2024-09-11T22:42:11.246 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.951593745, 2024-09-11T22:42:11.246 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.158426796, 2024-09-11T22:42:11.246 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:11.246 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 40231, 2024-09-11T22:42:11.246 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.983774923, 2024-09-11T22:42:11.246 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000024453 2024-09-11T22:42:11.246 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.246 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:11.247 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 50411, 2024-09-11T22:42:11.247 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.009216084, 2024-09-11T22:42:11.247 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000182 2024-09-11T22:42:11.247 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.248 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 6400, 2024-09-11T22:42:11.248 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 669248 2024-09-11T22:42:11.248 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.248 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:42:11.248 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 44347, 2024-09-11T22:42:11.248 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 77505, 2024-09-11T22:42:11.249 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 9940138, 2024-09-11T22:42:11.249 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 104022869, 2024-09-11T22:42:11.249 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 13, 2024-09-11T22:42:11.249 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 7, 2024-09-11T22:42:11.249 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 3.048349834, 2024-09-11T22:42:11.249 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 1.343455744, 2024-09-11T22:42:11.249 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 1.446714752, 2024-09-11T22:42:11.249 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.271411936, 2024-09-11T22:42:11.250 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:11.250 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 77505, 2024-09-11T22:42:11.250 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 1.831567782, 2024-09-11T22:42:11.250 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000023631 2024-09-11T22:42:11.250 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.250 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:11.250 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 100512, 2024-09-11T22:42:11.250 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.016088731, 2024-09-11T22:42:11.251 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000160 2024-09-11T22:42:11.251 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.251 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1740160, 2024-09-11T22:42:11.251 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 243648 2024-09-11T22:42:11.251 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.251 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-11T22:42:11.251 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-09-11T22:42:11.251 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-11T22:42:11.252 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.252 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-09-11T22:42:11.252 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-11T22:42:11.252 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-11T22:42:11.252 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 26554, 2024-09-11T22:42:11.252 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 22.725059402, 2024-09-11T22:42:11.252 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000855805 2024-09-11T22:42:11.252 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:11.253 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.253 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-09-11T22:42:11.253 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-11T22:42:11.253 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-11T22:42:11.253 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 16134, 2024-09-11T22:42:11.253 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 1.095488758, 2024-09-11T22:42:11.253 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000067899 2024-09-11T22:42:11.253 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:11.254 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.254 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-09-11T22:42:11.254 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 56880, 2024-09-11T22:42:11.254 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 56837, 2024-09-11T22:42:11.254 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-09-11T22:42:11.254 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 56837, 2024-09-11T22:42:11.254 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 266.205563206, 2024-09-11T22:42:11.254 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.004683666 2024-09-11T22:42:11.255 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.255 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-09-11T22:42:11.255 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 43, 2024-09-11T22:42:11.255 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-09-11T22:42:11.255 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-09-11T22:42:11.255 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 48, 2024-09-11T22:42:11.255 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 4, 2024-09-11T22:42:11.255 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 5, 2024-09-11T22:42:11.256 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 1088, 2024-09-11T22:42:11.256 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 373, 2024-09-11T22:42:11.256 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 715, 2024-09-11T22:42:11.256 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-09-11T22:42:11.256 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 555, 2024-09-11T22:42:11.256 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-09-11T22:42:11.256 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 37, 2024-09-11T22:42:11.257 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 37, 2024-09-11T22:42:11.257 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 5, 2024-09-11T22:42:11.257 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 95592, 2024-09-11T22:42:11.257 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 53286, 2024-09-11T22:42:11.257 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 40, 2024-09-11T22:42:11.257 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 106, 2024-09-11T22:42:11.257 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:42:11.257 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-09-11T22:42:11.258 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 862, 2024-09-11T22:42:11.258 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-09-11T22:42:11.258 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-09-11T22:42:11.258 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-09-11T22:42:11.258 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-09-11T22:42:11.258 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 3, 2024-09-11T22:42:11.258 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 67, 2024-09-11T22:42:11.258 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-09-11T22:42:11.259 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 600, 2024-09-11T22:42:11.259 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-09-11T22:42:11.259 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 681, 2024-09-11T22:42:11.259 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 218855980, 2024-09-11T22:42:11.259 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 6, 2024-09-11T22:42:11.259 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 600, 2024-09-11T22:42:11.259 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-09-11T22:42:11.260 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:42:11.260 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:42:11.260 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 63, 2024-09-11T22:42:11.260 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:42:11.260 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 63, 2024-09-11T22:42:11.260 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 537, 2024-09-11T22:42:11.260 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:42:11.260 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 1901, 2024-09-11T22:42:11.261 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 26438, 2024-09-11T22:42:11.261 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:42:11.261 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:42:11.261 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:42:11.261 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 15506, 2024-09-11T22:42:11.261 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 6785, 2024-09-11T22:42:11.261 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 3190, 2024-09-11T22:42:11.261 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 38616 2024-09-11T22:42:11.262 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.262 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-09-11T22:42:11.262 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 212, 2024-09-11T22:42:11.262 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 814, 2024-09-11T22:42:11.262 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 458, 2024-09-11T22:42:11.262 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 716, 2024-09-11T22:42:11.262 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 231, 2024-09-11T22:42:11.262 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-09-11T22:42:11.263 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-09-11T22:42:11.263 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:42:11.263 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 8351, 2024-09-11T22:42:11.263 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 8351, 2024-09-11T22:42:11.263 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-09-11T22:42:11.263 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-09-11T22:42:11.263 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-09-11T22:42:11.263 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:42:11.264 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:42:11.264 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-09-11T22:42:11.264 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-09-11T22:42:11.264 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 16, 2024-09-11T22:42:11.264 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 1020, 2024-09-11T22:42:11.264 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:42:11.264 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-09-11T22:42:11.264 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-09-11T22:42:11.265 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 11, 2024-09-11T22:42:11.265 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-09-11T22:42:11.265 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-09-11T22:42:11.265 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.265 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-09-11T22:42:11.265 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-09-11T22:42:11.265 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 41427, 2024-09-11T22:42:11.265 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-09-11T22:42:11.266 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-09-11T22:42:11.266 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 41427, 2024-09-11T22:42:11.266 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-09-11T22:42:11.266 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-09-11T22:42:11.266 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 64, 2024-09-11T22:42:11.266 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-09-11T22:42:11.266 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-09-11T22:42:11.266 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 64, 2024-09-11T22:42:11.267 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 5, 2024-09-11T22:42:11.267 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-09-11T22:42:11.267 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-09-11T22:42:11.267 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-09-11T22:42:11.267 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 156315121, 2024-09-11T22:42:11.267 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 0, 2024-09-11T22:42:11.267 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-09-11T22:42:11.267 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 13441, 2024-09-11T22:42:11.268 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 52.850469050, 2024-09-11T22:42:11.268 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.003932034 2024-09-11T22:42:11.268 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.268 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 0 2024-09-11T22:42:11.268 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.268 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-09-11T22:42:11.268 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 632, 2024-09-11T22:42:11.268 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 8983, 2024-09-11T22:42:11.269 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 8351, 2024-09-11T22:42:11.269 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 217, 2024-09-11T22:42:11.269 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 522, 2024-09-11T22:42:11.269 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 305, 2024-09-11T22:42:11.269 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 1088, 2024-09-11T22:42:11.269 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 17750, 2024-09-11T22:42:11.269 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 16662, 2024-09-11T22:42:11.269 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 20, 2024-09-11T22:42:11.270 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 9542, 2024-09-11T22:42:11.270 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 9522, 2024-09-11T22:42:11.270 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 71012, 2024-09-11T22:42:11.270 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 223520 2024-09-11T22:42:11.270 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.270 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-09-11T22:42:11.270 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 84351, 2024-09-11T22:42:11.270 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-09-11T22:42:11.271 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 56880, 2024-09-11T22:42:11.271 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 1212, 2024-09-11T22:42:11.271 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 1362, 2024-09-11T22:42:11.271 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-09-11T22:42:11.271 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 8354, 2024-09-11T22:42:11.271 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 113.486301210, 2024-09-11T22:42:11.271 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.013584666 2024-09-11T22:42:11.272 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.272 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-09-11T22:42:11.272 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 79, 2024-09-11T22:42:11.272 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.210585585, 2024-09-11T22:42:11.272 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.002665640 2024-09-11T22:42:11.272 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.272 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-09-11T22:42:11.272 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.273 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.273 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.273 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.273 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-09-11T22:42:11.273 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.273 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.273 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.273 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.274 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-09-11T22:42:11.274 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 20558, 2024-09-11T22:42:11.274 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 17.884504128, 2024-09-11T22:42:11.274 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000869953 2024-09-11T22:42:11.274 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.274 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-09-11T22:42:11.274 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.274 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.275 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.275 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.275 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-09-11T22:42:11.275 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.275 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.275 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.275 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.275 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-09-11T22:42:11.275 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.276 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.276 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.276 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.276 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-09-11T22:42:11.276 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.276 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.276 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.277 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.277 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-09-11T22:42:11.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.277 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.278 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.278 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-09-11T22:42:11.279 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.279 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.279 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.279 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.279 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-09-11T22:42:11.279 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.279 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.279 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.280 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.280 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-09-11T22:42:11.280 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.280 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.280 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.280 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.280 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-09-11T22:42:11.280 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.281 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.281 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.281 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.281 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-09-11T22:42:11.281 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.281 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.281 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.281 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.282 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-09-11T22:42:11.282 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 17323, 2024-09-11T22:42:11.282 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 70.724760186, 2024-09-11T22:42:11.282 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.004082708 2024-09-11T22:42:11.282 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.282 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-09-11T22:42:11.282 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1896, 2024-09-11T22:42:11.282 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 11.046357908, 2024-09-11T22:42:11.283 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.005826138 2024-09-11T22:42:11.283 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.283 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-09-11T22:42:11.283 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.283 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.283 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.283 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.284 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-09-11T22:42:11.284 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.284 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.284 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.284 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.284 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-09-11T22:42:11.284 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.284 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.285 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.285 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.285 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-09-11T22:42:11.285 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.285 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.285 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.285 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.285 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-09-11T22:42:11.286 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.286 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.286 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.286 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.286 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-09-11T22:42:11.286 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 24, 2024-09-11T22:42:11.286 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.162553475, 2024-09-11T22:42:11.286 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.006773061 2024-09-11T22:42:11.287 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.287 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-09-11T22:42:11.287 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.287 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.287 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.287 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.287 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-09-11T22:42:11.287 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 252, 2024-09-11T22:42:11.288 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.162058141, 2024-09-11T22:42:11.288 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000643087 2024-09-11T22:42:11.288 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.288 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-09-11T22:42:11.288 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.288 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.288 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.288 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.289 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-09-11T22:42:11.289 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.289 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.289 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.289 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.289 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-09-11T22:42:11.289 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.289 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.290 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.290 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.290 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-09-11T22:42:11.290 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 8351, 2024-09-11T22:42:11.290 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 52.528442573, 2024-09-11T22:42:11.290 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.006290078 2024-09-11T22:42:11.290 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.290 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:42:11.291 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:42:11.291 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-09-11T22:42:11.291 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.291 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.291 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.291 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:11.291 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.291 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-09-11T22:42:11.292 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-09-11T22:42:11.292 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 3, 2024-09-11T22:42:11.292 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 1, 2024-09-11T22:42:11.292 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 2, 2024-09-11T22:42:11.292 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-09-11T22:42:11.292 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 17338, 2024-09-11T22:42:11.292 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 8669, 2024-09-11T22:42:11.292 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 1101, 2024-09-11T22:42:11.293 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-09-11T22:42:11.293 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.293 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-11T22:42:11.293 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:42:11.293 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-11T22:42:11.293 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:42:11.293 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:42:11.294 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:42:11.294 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:42:11.294 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:42:11.294 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:42:11.294 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:42:11.294 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:42:11.294 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:42:11.294 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:42:11.295 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:42:11.295 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:42:11.295 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:42:11.295 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-11T22:42:11.295 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:42:11.295 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-11T22:42:11.295 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:42:11.295 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:42:11.295 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:42:11.296 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:42:11.296 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:42:11.296 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:42:11.296 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:42:11.296 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-11T22:42:11.296 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:42:11.296 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:42:11.297 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:42:11.297 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-11T22:42:11.297 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-11T22:42:11.297 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-11T22:42:11.297 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:42:11.297 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:42:11.297 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:42:11.297 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:42:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 422099, 2024-09-11T22:42:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 464, 2024-09-11T22:42:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 88, 2024-09-11T22:42:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 1, 2024-09-11T22:42:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-11T22:42:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-11T22:42:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:42:11.298 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-11T22:42:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:42:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-11T22:42:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8512, 2024-09-11T22:42:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 187, 2024-09-11T22:42:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:42:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:42:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-11T22:42:11.299 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-11T22:42:11.300 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 2648614, 2024-09-11T22:42:11.300 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 21251, 2024-09-11T22:42:11.300 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-11T22:42:11.300 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-11T22:42:11.300 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-11T22:42:11.300 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-11T22:42:11.300 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.300 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-09-11T22:42:11.301 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-09-11T22:42:11.301 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-09-11T22:42:11.301 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 23575, 2024-09-11T22:42:11.301 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 154175312, 2024-09-11T22:42:11.301 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 33, 2024-09-11T22:42:11.301 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 23575, 2024-09-11T22:42:11.301 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-09-11T22:42:11.301 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 23575, 2024-09-11T22:42:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 103.591413496, 2024-09-11T22:42:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.004394121 2024-09-11T22:42:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-09-11T22:42:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-09-11T22:42:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 23575, 2024-09-11T22:42:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 24017 2024-09-11T22:42:11.302 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 23575, 2024-09-11T22:42:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 912, 2024-09-11T22:42:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 22663, 2024-09-11T22:42:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-09-11T22:42:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-09-11T22:42:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 0, 2024-09-11T22:42:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 10, 2024-09-11T22:42:11.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 312, 2024-09-11T22:42:11.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 13753, 2024-09-11T22:42:11.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 183, 2024-09-11T22:42:11.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-09-11T22:42:11.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-09-11T22:42:11.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-09-11T22:42:11.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 4, 2024-09-11T22:42:11.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 8425, 2024-09-11T22:42:11.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-09-11T22:42:11.305 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-09-11T22:42:11.305 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-09-11T22:42:11.305 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 600, 2024-09-11T22:42:11.305 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 20, 2024-09-11T22:42:11.305 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:42:11.305 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-09-11T22:42:11.305 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:42:11.305 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-09-11T22:42:11.306 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-09-11T22:42:11.306 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-09-11T22:42:11.306 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:42:11.306 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-09-11T22:42:11.306 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:42:11.306 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-09-11T22:42:11.306 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-09-11T22:42:11.306 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-09-11T22:42:11.307 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-09-11T22:42:11.307 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-09-11T22:42:11.307 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-09-11T22:42:11.307 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-09-11T22:42:11.307 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-09-11T22:42:11.307 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-09-11T22:42:11.307 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-09-11T22:42:11.307 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-09-11T22:42:11.308 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-09-11T22:42:11.308 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-09-11T22:42:11.308 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-09-11T22:42:11.308 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-09-11T22:42:11.308 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-09-11T22:42:11.308 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-09-11T22:42:11.308 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 120, 2024-09-11T22:42:11.308 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-09-11T22:42:11.309 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 40, 2024-09-11T22:42:11.309 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 12, 2024-09-11T22:42:11.309 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 16, 2024-09-11T22:42:11.309 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 4, 2024-09-11T22:42:11.309 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-09-11T22:42:11.309 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 531, 2024-09-11T22:42:11.309 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 0, 2024-09-11T22:42:11.309 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 179 2024-09-11T22:42:11.310 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.310 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-09-11T22:42:11.310 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 1, 2024-09-11T22:42:11.310 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:42:11.310 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-09-11T22:42:11.310 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-09-11T22:42:11.310 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.310 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-09-11T22:42:11.311 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 1040 2024-09-11T22:42:11.311 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.311 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-09-11T22:42:11.311 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-09-11T22:42:11.311 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T22:42:11.311 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-09-11T22:42:11.311 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 6, 2024-09-11T22:42:11.311 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 8351, 2024-09-11T22:42:11.312 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 8351, 2024-09-11T22:42:11.312 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-09-11T22:42:11.312 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.312 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:42:11.312 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:11.312 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-11T22:42:11.312 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:11.312 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 545748, 2024-09-11T22:42:11.313 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 162442446, 2024-09-11T22:42:11.313 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:11.313 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 545748, 2024-09-11T22:42:11.313 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:11.313 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:11.313 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 545748, 2024-09-11T22:42:11.313 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 162442446, 2024-09-11T22:42:11.313 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:11.314 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.314 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.314 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.314 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:11.314 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.314 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-09-11T22:42:11.314 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:11.314 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-11T22:42:11.315 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:11.315 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:11.315 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:11.315 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:11.315 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:11.315 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 23575, 2024-09-11T22:42:11.315 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 155019907, 2024-09-11T22:42:11.315 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 23575, 2024-09-11T22:42:11.316 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 155019907, 2024-09-11T22:42:11.316 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:11.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.316 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.316 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:11.316 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.316 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-09-11T22:42:11.317 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:11.317 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-09-11T22:42:11.317 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:11.317 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:11.317 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:11.317 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:11.317 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:11.317 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 23575, 2024-09-11T22:42:11.318 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 23575, 2024-09-11T22:42:11.318 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 23575, 2024-09-11T22:42:11.318 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 23575, 2024-09-11T22:42:11.318 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:11.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.318 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.318 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:11.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.319 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:42:11.319 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:11.319 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-11T22:42:11.319 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:11.319 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 8351, 2024-09-11T22:42:11.319 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 844595, 2024-09-11T22:42:11.319 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:11.320 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 8351, 2024-09-11T22:42:11.320 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:11.320 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:11.320 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 312, 2024-09-11T22:42:11.320 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 844595, 2024-09-11T22:42:11.320 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:11.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.320 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.321 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:11.321 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:11.321 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x564401fe4960": { 2024-09-11T22:42:11.321 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:11.321 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-11T22:42:11.321 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:11.321 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 41427, 2024-09-11T22:42:11.322 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 152120817, 2024-09-11T22:42:11.322 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:11.322 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 41427, 2024-09-11T22:42:11.322 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:11.322 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:11.322 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 13441, 2024-09-11T22:42:11.322 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 152120817, 2024-09-11T22:42:11.322 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:11.323 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:11.323 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:11.323 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:11.323 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:11.323 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:11.323 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-11T22:42:11.831 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:11 smithi172 ceph-mon[35461]: pgmap v637: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 116 KiB/s wr, 34 op/s 2024-09-11T22:42:12.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:11 smithi005 ceph-mon[30943]: pgmap v637: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 116 KiB/s wr, 34 op/s 2024-09-11T22:42:12.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:11 smithi159 ceph-mon[35534]: pgmap v637: 97 pgs: 97 active+clean; 546 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 116 KiB/s wr, 34 op/s 2024-09-11T22:42:12.437 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-11T22:42:12.437 INFO:tasks.check_counter:Daemon mds.c mds.imported=3 2024-09-11T22:42:12.437 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=212 2024-09-11T22:42:12.438 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=814 2024-09-11T22:42:12.438 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-11T22:42:12.438 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:42:12.438 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:42:12.438 DEBUG:tasks.check_counter:Getting stats from f 2024-09-11T22:42:12.438 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-11T22:42:12.733 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:42:13.512 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-11T22:42:13.513 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:42:13.513 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 2153, 2024-09-11T22:42:13.513 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 2113, 2024-09-11T22:42:13.513 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 10651400, 2024-09-11T22:42:13.513 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 772546, 2024-09-11T22:42:13.513 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 6, 2024-09-11T22:42:13.514 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 2, 2024-09-11T22:42:13.514 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.219585104, 2024-09-11T22:42:13.514 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.069471614, 2024-09-11T22:42:13.514 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.097105233, 2024-09-11T22:42:13.514 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.052520353, 2024-09-11T22:42:13.514 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:13.514 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2112, 2024-09-11T22:42:13.514 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.088059085, 2024-09-11T22:42:13.515 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000041694 2024-09-11T22:42:13.515 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.515 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:13.515 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.515 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.515 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.515 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.515 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1722512, 2024-09-11T22:42:13.516 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 228448 2024-09-11T22:42:13.516 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.516 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:42:13.516 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1427, 2024-09-11T22:42:13.516 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1698, 2024-09-11T22:42:13.516 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 12105703, 2024-09-11T22:42:13.516 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 1097340, 2024-09-11T22:42:13.516 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 8, 2024-09-11T22:42:13.517 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 6, 2024-09-11T22:42:13.517 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.122334310, 2024-09-11T22:42:13.517 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.037297646, 2024-09-11T22:42:13.517 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.037747269, 2024-09-11T22:42:13.517 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.035982268, 2024-09-11T22:42:13.518 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:13.518 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1695, 2024-09-11T22:42:13.518 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.061808761, 2024-09-11T22:42:13.518 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000036465 2024-09-11T22:42:13.518 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.518 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:13.518 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.518 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.519 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.519 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.519 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T22:42:13.519 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 644128 2024-09-11T22:42:13.519 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.519 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:42:13.519 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1286, 2024-09-11T22:42:13.520 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1286, 2024-09-11T22:42:13.520 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 21300673, 2024-09-11T22:42:13.520 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 408131, 2024-09-11T22:42:13.520 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 6, 2024-09-11T22:42:13.521 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 4, 2024-09-11T22:42:13.521 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.095309479, 2024-09-11T22:42:13.521 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.029489083, 2024-09-11T22:42:13.521 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.032587278, 2024-09-11T22:42:13.521 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.026405819, 2024-09-11T22:42:13.521 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:13.521 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1285, 2024-09-11T22:42:13.522 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.042887674, 2024-09-11T22:42:13.522 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000033375 2024-09-11T22:42:13.522 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.522 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:13.522 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.522 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.523 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.523 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.523 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-11T22:42:13.523 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:42:13.523 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.523 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-11T22:42:13.523 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-09-11T22:42:13.524 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-11T22:42:13.524 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.524 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-09-11T22:42:13.524 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-11T22:42:13.524 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-11T22:42:13.525 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 7568, 2024-09-11T22:42:13.525 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.173805016, 2024-09-11T22:42:13.525 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000022965 2024-09-11T22:42:13.525 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:13.525 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.525 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-09-11T22:42:13.525 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-11T22:42:13.526 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-11T22:42:13.526 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.526 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.526 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.526 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:13.526 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.527 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-09-11T22:42:13.527 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 0, 2024-09-11T22:42:13.527 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 0, 2024-09-11T22:42:13.527 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-09-11T22:42:13.527 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.528 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.528 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.528 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.528 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-09-11T22:42:13.528 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 0, 2024-09-11T22:42:13.529 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-09-11T22:42:13.529 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-09-11T22:42:13.529 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 0, 2024-09-11T22:42:13.529 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-09-11T22:42:13.530 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-09-11T22:42:13.530 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 4534, 2024-09-11T22:42:13.530 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 18, 2024-09-11T22:42:13.530 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 4516, 2024-09-11T22:42:13.530 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-09-11T22:42:13.530 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 4509, 2024-09-11T22:42:13.531 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-09-11T22:42:13.531 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-09-11T22:42:13.531 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-09-11T22:42:13.531 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 7, 2024-09-11T22:42:13.531 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 0, 2024-09-11T22:42:13.531 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 0, 2024-09-11T22:42:13.531 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 0, 2024-09-11T22:42:13.532 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 0, 2024-09-11T22:42:13.532 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:42:13.532 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-09-11T22:42:13.532 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-09-11T22:42:13.532 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-09-11T22:42:13.532 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-09-11T22:42:13.532 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-09-11T22:42:13.532 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-09-11T22:42:13.533 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 0, 2024-09-11T22:42:13.533 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 0, 2024-09-11T22:42:13.533 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-09-11T22:42:13.533 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:42:13.533 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-09-11T22:42:13.534 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 0, 2024-09-11T22:42:13.534 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 0, 2024-09-11T22:42:13.534 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 0, 2024-09-11T22:42:13.534 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:42:13.534 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-09-11T22:42:13.534 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:42:13.534 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:42:13.535 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:42:13.535 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:42:13.535 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:42:13.535 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-09-11T22:42:13.535 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:42:13.535 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:42:13.535 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:42:13.536 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:42:13.536 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:42:13.536 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:42:13.536 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 0, 2024-09-11T22:42:13.536 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:42:13.536 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 0, 2024-09-11T22:42:13.537 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-09-11T22:42:13.537 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.537 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-09-11T22:42:13.537 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-09-11T22:42:13.537 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 0, 2024-09-11T22:42:13.537 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 0, 2024-09-11T22:42:13.537 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 0, 2024-09-11T22:42:13.537 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-09-11T22:42:13.538 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-09-11T22:42:13.538 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-09-11T22:42:13.538 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:42:13.538 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-09-11T22:42:13.538 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-09-11T22:42:13.538 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-09-11T22:42:13.538 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-09-11T22:42:13.538 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-09-11T22:42:13.539 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:42:13.539 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:42:13.539 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-09-11T22:42:13.539 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-09-11T22:42:13.539 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:42:13.539 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:42:13.539 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:42:13.539 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-09-11T22:42:13.540 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-09-11T22:42:13.540 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:42:13.540 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-09-11T22:42:13.540 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-09-11T22:42:13.540 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.540 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-09-11T22:42:13.540 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-09-11T22:42:13.541 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 0, 2024-09-11T22:42:13.541 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-09-11T22:42:13.541 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-09-11T22:42:13.541 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 25392, 2024-09-11T22:42:13.541 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-09-11T22:42:13.541 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-09-11T22:42:13.541 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 0, 2024-09-11T22:42:13.542 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-09-11T22:42:13.542 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-09-11T22:42:13.542 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 32, 2024-09-11T22:42:13.542 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 4, 2024-09-11T22:42:13.542 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-09-11T22:42:13.542 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-09-11T22:42:13.542 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-09-11T22:42:13.542 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 45299905, 2024-09-11T22:42:13.543 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 45297588, 2024-09-11T22:42:13.543 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-09-11T22:42:13.543 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.543 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.543 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.543 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.543 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 25392 2024-09-11T22:42:13.543 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.544 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-09-11T22:42:13.544 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 140, 2024-09-11T22:42:13.544 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 4573, 2024-09-11T22:42:13.544 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 4433, 2024-09-11T22:42:13.544 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 26, 2024-09-11T22:42:13.544 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 34, 2024-09-11T22:42:13.544 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 8, 2024-09-11T22:42:13.544 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 4534, 2024-09-11T22:42:13.544 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 4575, 2024-09-11T22:42:13.545 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 41, 2024-09-11T22:42:13.545 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-09-11T22:42:13.545 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 0, 2024-09-11T22:42:13.545 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 0, 2024-09-11T22:42:13.545 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 54452, 2024-09-11T22:42:13.545 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 215328 2024-09-11T22:42:13.545 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.545 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-09-11T22:42:13.546 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 0, 2024-09-11T22:42:13.546 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-09-11T22:42:13.546 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 0, 2024-09-11T22:42:13.546 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 0, 2024-09-11T22:42:13.546 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 0, 2024-09-11T22:42:13.546 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-09-11T22:42:13.546 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.546 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.547 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.547 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.547 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-09-11T22:42:13.547 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.547 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.547 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.547 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.547 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-09-11T22:42:13.547 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.548 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.548 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.548 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.548 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-09-11T22:42:13.548 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.548 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.548 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.548 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.549 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-09-11T22:42:13.549 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.549 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.549 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.549 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.549 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-09-11T22:42:13.549 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.549 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.550 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.550 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.550 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-09-11T22:42:13.550 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.550 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.550 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.550 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.550 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-09-11T22:42:13.551 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.551 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.551 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.551 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.551 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-09-11T22:42:13.551 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.551 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.552 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.552 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.552 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-09-11T22:42:13.552 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.552 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.552 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.552 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.552 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-09-11T22:42:13.552 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.553 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.553 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.553 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.553 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-09-11T22:42:13.554 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.554 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.554 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.554 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.554 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-09-11T22:42:13.554 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.554 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.555 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.555 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.555 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-09-11T22:42:13.555 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.555 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.555 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.555 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.555 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-09-11T22:42:13.556 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.556 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.556 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.556 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.556 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-09-11T22:42:13.556 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.556 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.556 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.557 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.557 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-09-11T22:42:13.557 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.557 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.557 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.558 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.558 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-09-11T22:42:13.558 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.558 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.558 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.558 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.559 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-09-11T22:42:13.559 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.559 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.559 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.560 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.560 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-09-11T22:42:13.560 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.560 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.561 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.561 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.561 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-09-11T22:42:13.561 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.562 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.562 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.562 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.562 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-09-11T22:42:13.563 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.563 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.563 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.563 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.564 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-09-11T22:42:13.564 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.564 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.564 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.565 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.565 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-09-11T22:42:13.565 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.565 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.566 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.566 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.566 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-09-11T22:42:13.566 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.567 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.567 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.567 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.567 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-09-11T22:42:13.568 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.568 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.568 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.568 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.569 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-09-11T22:42:13.569 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.569 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.569 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.570 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.570 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-09-11T22:42:13.570 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.570 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.571 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.571 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.571 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-09-11T22:42:13.571 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.571 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.572 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.572 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.572 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:42:13.572 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:42:13.572 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-09-11T22:42:13.572 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.572 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.572 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.573 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:13.573 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.573 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-09-11T22:42:13.573 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 1, 2024-09-11T22:42:13.573 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 3, 2024-09-11T22:42:13.573 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 2, 2024-09-11T22:42:13.573 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 3, 2024-09-11T22:42:13.573 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-09-11T22:42:13.574 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 0, 2024-09-11T22:42:13.574 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 0, 2024-09-11T22:42:13.574 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 5137, 2024-09-11T22:42:13.574 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-09-11T22:42:13.574 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.574 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-11T22:42:13.574 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:42:13.574 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-11T22:42:13.575 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:42:13.575 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:42:13.575 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:42:13.575 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:42:13.575 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:42:13.575 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:42:13.576 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:42:13.576 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:42:13.576 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:42:13.576 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:42:13.576 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:42:13.576 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:42:13.576 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:42:13.576 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-11T22:42:13.577 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:42:13.577 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-11T22:42:13.577 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:42:13.577 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:42:13.577 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:42:13.577 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:42:13.577 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:42:13.577 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:42:13.578 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:42:13.578 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-11T22:42:13.578 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:42:13.578 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:42:13.578 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:42:13.578 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-11T22:42:13.578 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-11T22:42:13.578 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-11T22:42:13.579 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:42:13.579 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:42:13.579 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:42:13.579 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:42:13.579 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 221364, 2024-09-11T22:42:13.579 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 195, 2024-09-11T22:42:13.579 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:42:13.579 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-09-11T22:42:13.580 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-11T22:42:13.580 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-11T22:42:13.580 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:42:13.580 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-11T22:42:13.580 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:42:13.580 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-11T22:42:13.580 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8632, 2024-09-11T22:42:13.581 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 190, 2024-09-11T22:42:13.581 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:42:13.581 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:42:13.581 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-11T22:42:13.581 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-11T22:42:13.581 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 3611657, 2024-09-11T22:42:13.581 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 21212, 2024-09-11T22:42:13.581 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-11T22:42:13.582 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-11T22:42:13.582 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-11T22:42:13.582 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-11T22:42:13.582 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.582 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-09-11T22:42:13.582 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-09-11T22:42:13.583 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-09-11T22:42:13.583 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 4418, 2024-09-11T22:42:13.583 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 21, 2024-09-11T22:42:13.583 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 3, 2024-09-11T22:42:13.583 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 4418, 2024-09-11T22:42:13.583 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-09-11T22:42:13.584 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4418, 2024-09-11T22:42:13.584 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3.082922501, 2024-09-11T22:42:13.584 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000697809 2024-09-11T22:42:13.584 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.584 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-09-11T22:42:13.584 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-09-11T22:42:13.584 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 4418, 2024-09-11T22:42:13.584 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 4419 2024-09-11T22:42:13.585 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.585 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 4418, 2024-09-11T22:42:13.585 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 4418, 2024-09-11T22:42:13.585 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 0, 2024-09-11T22:42:13.585 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-09-11T22:42:13.585 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-09-11T22:42:13.586 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 2493, 2024-09-11T22:42:13.586 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 0, 2024-09-11T22:42:13.586 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 1923, 2024-09-11T22:42:13.586 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 0, 2024-09-11T22:42:13.586 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 0, 2024-09-11T22:42:13.587 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-09-11T22:42:13.587 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-09-11T22:42:13.587 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-09-11T22:42:13.587 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-09-11T22:42:13.587 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 0, 2024-09-11T22:42:13.587 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-09-11T22:42:13.587 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-09-11T22:42:13.587 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-09-11T22:42:13.588 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 1, 2024-09-11T22:42:13.588 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 0, 2024-09-11T22:42:13.588 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:42:13.588 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-09-11T22:42:13.588 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:42:13.588 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-09-11T22:42:13.589 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-09-11T22:42:13.589 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-09-11T22:42:13.589 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:42:13.589 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-09-11T22:42:13.589 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:42:13.590 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-09-11T22:42:13.590 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-09-11T22:42:13.590 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-09-11T22:42:13.590 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-09-11T22:42:13.590 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-09-11T22:42:13.590 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-09-11T22:42:13.590 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-09-11T22:42:13.591 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-09-11T22:42:13.591 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-09-11T22:42:13.591 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-09-11T22:42:13.591 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-09-11T22:42:13.591 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-09-11T22:42:13.591 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-09-11T22:42:13.591 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-09-11T22:42:13.592 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-09-11T22:42:13.592 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-09-11T22:42:13.592 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-09-11T22:42:13.592 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 120, 2024-09-11T22:42:13.592 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-09-11T22:42:13.592 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 40, 2024-09-11T22:42:13.592 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 8, 2024-09-11T22:42:13.593 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 9, 2024-09-11T22:42:13.593 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 1, 2024-09-11T22:42:13.593 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-09-11T22:42:13.593 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 0, 2024-09-11T22:42:13.593 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 2, 2024-09-11T22:42:13.593 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 0 2024-09-11T22:42:13.593 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.594 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-09-11T22:42:13.594 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 0, 2024-09-11T22:42:13.594 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:42:13.594 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-09-11T22:42:13.594 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-09-11T22:42:13.595 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.595 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-09-11T22:42:13.595 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-09-11T22:42:13.595 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.595 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-09-11T22:42:13.595 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-09-11T22:42:13.595 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:42:13.595 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-09-11T22:42:13.595 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-09-11T22:42:13.596 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-09-11T22:42:13.596 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-09-11T22:42:13.596 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-09-11T22:42:13.596 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.597 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:42:13.597 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:13.597 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-11T22:42:13.597 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:13.597 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 4862, 2024-09-11T22:42:13.597 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 43594802, 2024-09-11T22:42:13.598 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:13.598 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 4862, 2024-09-11T22:42:13.598 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:13.598 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:13.598 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4862, 2024-09-11T22:42:13.598 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 43594802, 2024-09-11T22:42:13.598 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:13.599 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.599 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.599 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.599 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:13.599 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.599 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-09-11T22:42:13.600 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:13.600 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-11T22:42:13.600 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:13.600 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:13.600 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:13.600 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:13.601 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:13.601 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 4418, 2024-09-11T22:42:13.601 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 41105606, 2024-09-11T22:42:13.601 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4418, 2024-09-11T22:42:13.601 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 41105606, 2024-09-11T22:42:13.601 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:13.601 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.602 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.602 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.602 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:13.602 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.602 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-09-11T22:42:13.603 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:13.603 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-09-11T22:42:13.603 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:13.603 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:13.603 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:13.604 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:13.604 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:13.604 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 4418, 2024-09-11T22:42:13.604 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 4418, 2024-09-11T22:42:13.604 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4418, 2024-09-11T22:42:13.605 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 4418, 2024-09-11T22:42:13.605 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:13.605 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.605 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.606 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.606 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:13.606 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.606 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:42:13.606 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:13.607 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-11T22:42:13.607 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:13.607 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:13.607 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:13.607 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:13.608 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:13.608 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:13.608 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:13.608 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-11T22:42:13.608 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-11T22:42:13.608 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:13.608 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.608 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.609 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.609 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:13.609 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:13.609 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x559f6dd32b40": { 2024-09-11T22:42:13.609 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:13.609 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-11T22:42:13.609 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:13.610 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:13.610 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:13.610 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:13.610 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:13.611 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:13.611 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:13.611 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-11T22:42:13.612 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-11T22:42:13.612 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:13.612 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:13.613 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:13.613 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:13.613 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:13.613 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:13.614 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-11T22:42:14.073 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:13 smithi005 ceph-mon[30943]: pgmap v638: 97 pgs: 11 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 80 active+clean; 533 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 91 KiB/s wr, 17 op/s 2024-09-11T22:42:14.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:13 smithi159 ceph-mon[35534]: pgmap v638: 97 pgs: 11 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 80 active+clean; 533 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 91 KiB/s wr, 17 op/s 2024-09-11T22:42:14.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:13 smithi172 ceph-mon[35461]: pgmap v638: 97 pgs: 11 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 80 active+clean; 533 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 91 KiB/s wr, 17 op/s 2024-09-11T22:42:14.320 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-11T22:42:14.320 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-11T22:42:14.320 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-11T22:42:14.321 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-11T22:42:14.321 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-11T22:42:14.321 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:42:14.321 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:42:14.322 DEBUG:tasks.check_counter:Getting stats from i 2024-09-11T22:42:14.322 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-11T22:42:14.571 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:42:15.268 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-11T22:42:15.268 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:42:15.268 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 412533, 2024-09-11T22:42:15.268 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 228530, 2024-09-11T22:42:15.269 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 177335413, 2024-09-11T22:42:15.269 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 45147469, 2024-09-11T22:42:15.269 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 27, 2024-09-11T22:42:15.269 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 5, 2024-09-11T22:42:15.269 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 16.768683308, 2024-09-11T22:42:15.270 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 6.500789451, 2024-09-11T22:42:15.270 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 13.966148195, 2024-09-11T22:42:15.270 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.035991999, 2024-09-11T22:42:15.270 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:15.270 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 228531, 2024-09-11T22:42:15.270 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3.411146899, 2024-09-11T22:42:15.271 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000014926 2024-09-11T22:42:15.271 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.271 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:15.271 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 636310, 2024-09-11T22:42:15.271 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.078336937, 2024-09-11T22:42:15.271 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000123 2024-09-11T22:42:15.271 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.271 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-11T22:42:15.272 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:42:15.272 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.272 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:42:15.272 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 821, 2024-09-11T22:42:15.272 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 889, 2024-09-11T22:42:15.272 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 276461, 2024-09-11T22:42:15.272 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 1428854, 2024-09-11T22:42:15.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 46, 2024-09-11T22:42:15.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 7, 2024-09-11T22:42:15.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.071391428, 2024-09-11T22:42:15.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.022131343, 2024-09-11T22:42:15.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.032927543, 2024-09-11T22:42:15.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.013757386, 2024-09-11T22:42:15.273 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:15.273 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 886, 2024-09-11T22:42:15.274 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.037141802, 2024-09-11T22:42:15.274 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000041920 2024-09-11T22:42:15.274 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.274 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:15.274 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 515, 2024-09-11T22:42:15.274 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000084372, 2024-09-11T22:42:15.274 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000163 2024-09-11T22:42:15.274 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.275 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 5328, 2024-09-11T22:42:15.275 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 142464 2024-09-11T22:42:15.275 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.275 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:42:15.275 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 34893, 2024-09-11T22:42:15.275 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 18482, 2024-09-11T22:42:15.275 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 14137819, 2024-09-11T22:42:15.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 4018722, 2024-09-11T22:42:15.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 18, 2024-09-11T22:42:15.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 8, 2024-09-11T22:42:15.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 1.693770102, 2024-09-11T22:42:15.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.734875380, 2024-09-11T22:42:15.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 1.294851056, 2024-09-11T22:42:15.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.009599982, 2024-09-11T22:42:15.276 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:15.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 18479, 2024-09-11T22:42:15.277 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.312702456, 2024-09-11T22:42:15.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000016922 2024-09-11T22:42:15.277 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.277 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:15.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 51286, 2024-09-11T22:42:15.277 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.006716680, 2024-09-11T22:42:15.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000130 2024-09-11T22:42:15.277 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.278 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1749360, 2024-09-11T22:42:15.278 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 759904 2024-09-11T22:42:15.278 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.278 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-11T22:42:15.278 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-09-11T22:42:15.278 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-11T22:42:15.278 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.279 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-09-11T22:42:15.279 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-11T22:42:15.279 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-11T22:42:15.279 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1916, 2024-09-11T22:42:15.279 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.934860857, 2024-09-11T22:42:15.279 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000487923 2024-09-11T22:42:15.279 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:15.280 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.280 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-09-11T22:42:15.280 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-11T22:42:15.280 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-11T22:42:15.280 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1231, 2024-09-11T22:42:15.280 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.049062518, 2024-09-11T22:42:15.280 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000039855 2024-09-11T22:42:15.281 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:15.281 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.281 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-09-11T22:42:15.281 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 33225, 2024-09-11T22:42:15.281 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 2526, 2024-09-11T22:42:15.281 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-09-11T22:42:15.281 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 2526, 2024-09-11T22:42:15.281 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 44.278011824, 2024-09-11T22:42:15.282 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.017528904 2024-09-11T22:42:15.282 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.282 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-09-11T22:42:15.282 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 30699, 2024-09-11T22:42:15.282 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-09-11T22:42:15.282 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-09-11T22:42:15.282 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 11, 2024-09-11T22:42:15.282 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-09-11T22:42:15.283 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-09-11T22:42:15.283 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 229, 2024-09-11T22:42:15.283 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 68, 2024-09-11T22:42:15.283 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 161, 2024-09-11T22:42:15.283 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-09-11T22:42:15.283 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 131, 2024-09-11T22:42:15.283 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-09-11T22:42:15.284 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 21, 2024-09-11T22:42:15.284 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 21, 2024-09-11T22:42:15.284 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 5, 2024-09-11T22:42:15.284 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 38153, 2024-09-11T22:42:15.284 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 4079, 2024-09-11T22:42:15.284 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 30699, 2024-09-11T22:42:15.284 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 117, 2024-09-11T22:42:15.285 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:42:15.285 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-09-11T22:42:15.285 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 209, 2024-09-11T22:42:15.285 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-09-11T22:42:15.285 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-09-11T22:42:15.285 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-09-11T22:42:15.286 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-09-11T22:42:15.286 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 2, 2024-09-11T22:42:15.286 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 14, 2024-09-11T22:42:15.286 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-09-11T22:42:15.286 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 116, 2024-09-11T22:42:15.286 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-09-11T22:42:15.286 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 681, 2024-09-11T22:42:15.286 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 218855980, 2024-09-11T22:42:15.287 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 6, 2024-09-11T22:42:15.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 116, 2024-09-11T22:42:15.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-09-11T22:42:15.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:42:15.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:42:15.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:42:15.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:42:15.287 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:42:15.288 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 116, 2024-09-11T22:42:15.288 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:42:15.288 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 183, 2024-09-11T22:42:15.288 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 1950, 2024-09-11T22:42:15.288 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:42:15.288 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:42:15.288 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:42:15.289 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 787, 2024-09-11T22:42:15.289 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 261, 2024-09-11T22:42:15.289 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 213, 2024-09-11T22:42:15.289 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 2566 2024-09-11T22:42:15.289 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.289 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-09-11T22:42:15.289 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 210, 2024-09-11T22:42:15.289 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 809, 2024-09-11T22:42:15.290 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 456, 2024-09-11T22:42:15.290 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 726, 2024-09-11T22:42:15.290 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 222, 2024-09-11T22:42:15.290 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-09-11T22:42:15.290 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-09-11T22:42:15.290 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:42:15.290 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 427, 2024-09-11T22:42:15.291 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 427, 2024-09-11T22:42:15.291 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-09-11T22:42:15.291 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-09-11T22:42:15.291 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-09-11T22:42:15.291 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:42:15.291 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:42:15.291 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-09-11T22:42:15.292 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-09-11T22:42:15.292 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 16, 2024-09-11T22:42:15.292 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 763, 2024-09-11T22:42:15.292 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:42:15.292 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-09-11T22:42:15.292 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-09-11T22:42:15.292 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:42:15.292 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-09-11T22:42:15.293 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-09-11T22:42:15.293 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.293 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-09-11T22:42:15.293 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-09-11T22:42:15.293 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 2946, 2024-09-11T22:42:15.293 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-09-11T22:42:15.293 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-09-11T22:42:15.293 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 2946, 2024-09-11T22:42:15.294 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-09-11T22:42:15.294 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-09-11T22:42:15.294 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 5, 2024-09-11T22:42:15.294 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-09-11T22:42:15.294 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-09-11T22:42:15.294 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 5, 2024-09-11T22:42:15.294 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-09-11T22:42:15.294 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-09-11T22:42:15.295 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-09-11T22:42:15.295 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-09-11T22:42:15.295 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 8902710, 2024-09-11T22:42:15.295 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 0, 2024-09-11T22:42:15.295 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-09-11T22:42:15.295 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 842, 2024-09-11T22:42:15.295 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3.722982292, 2024-09-11T22:42:15.296 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.004421594 2024-09-11T22:42:15.296 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.296 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 0 2024-09-11T22:42:15.296 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.296 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-09-11T22:42:15.296 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 211, 2024-09-11T22:42:15.296 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 638, 2024-09-11T22:42:15.296 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 427, 2024-09-11T22:42:15.297 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 217, 2024-09-11T22:42:15.297 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 498, 2024-09-11T22:42:15.297 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 281, 2024-09-11T22:42:15.297 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 229, 2024-09-11T22:42:15.297 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 766, 2024-09-11T22:42:15.297 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 537, 2024-09-11T22:42:15.297 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 21, 2024-09-11T22:42:15.298 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 486, 2024-09-11T22:42:15.298 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 465, 2024-09-11T22:42:15.298 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 63100, 2024-09-11T22:42:15.298 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 223520 2024-09-11T22:42:15.298 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.298 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-09-11T22:42:15.298 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 36064, 2024-09-11T22:42:15.299 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-09-11T22:42:15.299 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 33225, 2024-09-11T22:42:15.299 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 38, 2024-09-11T22:42:15.299 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 1077, 2024-09-11T22:42:15.299 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-09-11T22:42:15.299 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 431, 2024-09-11T22:42:15.299 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 34.309961854, 2024-09-11T22:42:15.299 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.079605479 2024-09-11T22:42:15.300 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.300 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-09-11T22:42:15.300 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1, 2024-09-11T22:42:15.300 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.001383435, 2024-09-11T22:42:15.300 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001383435 2024-09-11T22:42:15.300 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.300 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-09-11T22:42:15.301 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.301 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.301 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.301 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.301 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-09-11T22:42:15.302 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.302 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.302 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.302 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.302 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-09-11T22:42:15.302 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 765, 2024-09-11T22:42:15.302 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.943780531, 2024-09-11T22:42:15.303 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.001233700 2024-09-11T22:42:15.303 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.303 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-09-11T22:42:15.303 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.303 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.303 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.303 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.303 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-09-11T22:42:15.304 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1, 2024-09-11T22:42:15.304 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000119837, 2024-09-11T22:42:15.304 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000119837 2024-09-11T22:42:15.304 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.304 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-09-11T22:42:15.304 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.304 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.305 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.305 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.305 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-09-11T22:42:15.305 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.305 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.305 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.305 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.306 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-09-11T22:42:15.306 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.306 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.306 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.306 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.306 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-09-11T22:42:15.307 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.307 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.308 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.308 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.308 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-09-11T22:42:15.309 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.309 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.309 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.309 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.309 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-09-11T22:42:15.309 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.309 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.310 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.310 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.310 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-09-11T22:42:15.310 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.310 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.310 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.310 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.310 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-09-11T22:42:15.311 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.311 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.311 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.311 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.311 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-09-11T22:42:15.311 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 689, 2024-09-11T22:42:15.311 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 4.157120698, 2024-09-11T22:42:15.312 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.006033556 2024-09-11T22:42:15.312 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.312 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-09-11T22:42:15.312 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 212, 2024-09-11T22:42:15.312 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 2.815000504, 2024-09-11T22:42:15.312 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.013278304 2024-09-11T22:42:15.312 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.312 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-09-11T22:42:15.313 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.313 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.313 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.313 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.313 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-09-11T22:42:15.313 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.314 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.314 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.314 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.314 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-09-11T22:42:15.314 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.314 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.314 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.314 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.315 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-09-11T22:42:15.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.315 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.315 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.315 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-09-11T22:42:15.315 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.315 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.316 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.316 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-09-11T22:42:15.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.316 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.316 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.317 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-09-11T22:42:15.317 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.317 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.317 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.317 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.317 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-09-11T22:42:15.317 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.318 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.318 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.318 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-09-11T22:42:15.318 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.318 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.319 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-09-11T22:42:15.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.319 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.319 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.320 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-09-11T22:42:15.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.320 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.320 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.320 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-09-11T22:42:15.320 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 427, 2024-09-11T22:42:15.321 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 2.050644965, 2024-09-11T22:42:15.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.004802447 2024-09-11T22:42:15.321 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.321 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:42:15.321 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:42:15.321 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-09-11T22:42:15.321 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.321 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.322 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:15.322 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.322 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-09-11T22:42:15.322 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-09-11T22:42:15.322 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 4, 2024-09-11T22:42:15.322 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 2, 2024-09-11T22:42:15.322 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 2, 2024-09-11T22:42:15.323 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-09-11T22:42:15.323 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 727, 2024-09-11T22:42:15.323 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 363, 2024-09-11T22:42:15.323 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 1164, 2024-09-11T22:42:15.323 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-09-11T22:42:15.323 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.323 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-11T22:42:15.323 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:42:15.324 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-11T22:42:15.324 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:42:15.324 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:42:15.324 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:42:15.324 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:42:15.324 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:42:15.324 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:42:15.325 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:42:15.325 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:42:15.325 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:42:15.325 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:42:15.325 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:42:15.325 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:42:15.325 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:42:15.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-11T22:42:15.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:42:15.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-11T22:42:15.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:42:15.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:42:15.326 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:42:15.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:42:15.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:42:15.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:42:15.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:42:15.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-11T22:42:15.327 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:42:15.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:42:15.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:42:15.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-11T22:42:15.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-11T22:42:15.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-11T22:42:15.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:42:15.328 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:42:15.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:42:15.329 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:42:15.329 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 309544, 2024-09-11T22:42:15.329 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 251, 2024-09-11T22:42:15.329 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:42:15.329 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-09-11T22:42:15.329 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-11T22:42:15.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-11T22:42:15.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:42:15.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-11T22:42:15.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:42:15.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-11T22:42:15.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8632, 2024-09-11T22:42:15.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 190, 2024-09-11T22:42:15.330 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:42:15.331 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:42:15.331 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-11T22:42:15.331 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-11T22:42:15.331 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 967090, 2024-09-11T22:42:15.331 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 6036, 2024-09-11T22:42:15.331 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-11T22:42:15.331 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-11T22:42:15.332 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-11T22:42:15.332 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-11T22:42:15.332 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.332 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-09-11T22:42:15.332 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-09-11T22:42:15.332 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-09-11T22:42:15.332 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 1939, 2024-09-11T22:42:15.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 4834517, 2024-09-11T22:42:15.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 32, 2024-09-11T22:42:15.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 1939, 2024-09-11T22:42:15.333 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-09-11T22:42:15.333 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1939, 2024-09-11T22:42:15.333 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 8.406790058, 2024-09-11T22:42:15.333 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.004335631 2024-09-11T22:42:15.334 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.334 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-09-11T22:42:15.334 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-09-11T22:42:15.334 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1939, 2024-09-11T22:42:15.334 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 2159 2024-09-11T22:42:15.334 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.334 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 1939, 2024-09-11T22:42:15.334 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 247, 2024-09-11T22:42:15.335 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 1692, 2024-09-11T22:42:15.335 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-09-11T22:42:15.335 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-09-11T22:42:15.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 0, 2024-09-11T22:42:15.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 10, 2024-09-11T22:42:15.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 131, 2024-09-11T22:42:15.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 973, 2024-09-11T22:42:15.335 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 110, 2024-09-11T22:42:15.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-09-11T22:42:15.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-09-11T22:42:15.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-09-11T22:42:15.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-09-11T22:42:15.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 440, 2024-09-11T22:42:15.336 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-09-11T22:42:15.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-09-11T22:42:15.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-09-11T22:42:15.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 116, 2024-09-11T22:42:15.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 20, 2024-09-11T22:42:15.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:42:15.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-09-11T22:42:15.337 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:42:15.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-09-11T22:42:15.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-09-11T22:42:15.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-09-11T22:42:15.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:42:15.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-09-11T22:42:15.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:42:15.338 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-09-11T22:42:15.339 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-09-11T22:42:15.339 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-09-11T22:42:15.339 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-09-11T22:42:15.339 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-09-11T22:42:15.339 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-09-11T22:42:15.339 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-09-11T22:42:15.339 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-09-11T22:42:15.340 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-09-11T22:42:15.340 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-09-11T22:42:15.340 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-09-11T22:42:15.340 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-09-11T22:42:15.340 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-09-11T22:42:15.340 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-09-11T22:42:15.341 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-09-11T22:42:15.341 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-09-11T22:42:15.341 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-09-11T22:42:15.341 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 120, 2024-09-11T22:42:15.341 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-09-11T22:42:15.341 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 40, 2024-09-11T22:42:15.341 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 12, 2024-09-11T22:42:15.341 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 17, 2024-09-11T22:42:15.342 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 5, 2024-09-11T22:42:15.342 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-09-11T22:42:15.342 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 278, 2024-09-11T22:42:15.342 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 0, 2024-09-11T22:42:15.342 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 81 2024-09-11T22:42:15.343 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.343 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-09-11T22:42:15.343 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 1, 2024-09-11T22:42:15.343 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:42:15.343 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-09-11T22:42:15.343 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-09-11T22:42:15.343 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.344 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-09-11T22:42:15.344 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 780 2024-09-11T22:42:15.344 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.344 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-09-11T22:42:15.344 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-09-11T22:42:15.344 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 6, 2024-09-11T22:42:15.344 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-09-11T22:42:15.345 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 6, 2024-09-11T22:42:15.345 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 427, 2024-09-11T22:42:15.345 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 427, 2024-09-11T22:42:15.345 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-09-11T22:42:15.345 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.345 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:42:15.345 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:15.345 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-11T22:42:15.346 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:15.346 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 448243, 2024-09-11T22:42:15.346 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 151379140, 2024-09-11T22:42:15.346 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:15.346 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 448243, 2024-09-11T22:42:15.346 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:15.346 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:15.347 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 448243, 2024-09-11T22:42:15.347 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 151379140, 2024-09-11T22:42:15.347 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:15.347 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.347 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.347 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.348 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:15.348 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.348 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-09-11T22:42:15.348 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:15.348 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-11T22:42:15.348 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:15.348 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:15.349 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:15.349 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:15.349 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:15.349 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 1939, 2024-09-11T22:42:15.349 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 4877732, 2024-09-11T22:42:15.349 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 1939, 2024-09-11T22:42:15.349 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 4877732, 2024-09-11T22:42:15.350 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:15.350 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.350 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.350 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.350 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:15.350 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.350 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-09-11T22:42:15.350 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:15.351 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-09-11T22:42:15.351 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:15.351 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:15.351 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:15.351 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:15.351 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:15.351 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 1939, 2024-09-11T22:42:15.352 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 1939, 2024-09-11T22:42:15.352 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 1939, 2024-09-11T22:42:15.352 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 1939, 2024-09-11T22:42:15.352 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:15.352 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.352 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.352 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.352 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:15.353 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.353 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:42:15.353 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:15.353 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-11T22:42:15.353 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:15.353 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 427, 2024-09-11T22:42:15.353 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 43215, 2024-09-11T22:42:15.353 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:15.354 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 427, 2024-09-11T22:42:15.354 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:15.354 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:15.354 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 131, 2024-09-11T22:42:15.354 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 43215, 2024-09-11T22:42:15.354 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:15.355 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.355 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.355 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.355 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:15.355 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:15.355 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x556d0cefc960": { 2024-09-11T22:42:15.355 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 1148, 2024-09-11T22:42:15.356 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-11T22:42:15.356 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:15.356 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 2946, 2024-09-11T22:42:15.356 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 4708406, 2024-09-11T22:42:15.356 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:15.356 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 2946, 2024-09-11T22:42:15.356 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:15.357 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:15.357 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 842, 2024-09-11T22:42:15.357 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 4707258, 2024-09-11T22:42:15.357 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:15.357 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:15.357 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:15.357 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:15.358 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:15.358 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:15.358 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-11T22:42:16.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:15 smithi005 ceph-mon[30943]: pgmap v639: 97 pgs: 11 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 78 active+clean; 526 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 133 KiB/s wr, 15 op/s 2024-09-11T22:42:16.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:15 smithi159 ceph-mon[35534]: pgmap v639: 97 pgs: 11 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 78 active+clean; 526 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 133 KiB/s wr, 15 op/s 2024-09-11T22:42:16.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:15 smithi172 ceph-mon[35461]: pgmap v639: 97 pgs: 11 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 78 active+clean; 526 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 133 KiB/s wr, 15 op/s 2024-09-11T22:42:16.325 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-11T22:42:16.325 INFO:tasks.check_counter:Daemon mds.i mds.imported=2 2024-09-11T22:42:16.325 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=210 2024-09-11T22:42:16.326 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=809 2024-09-11T22:42:16.326 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-11T22:42:16.326 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:42:16.326 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:42:16.326 DEBUG:tasks.check_counter:Getting stats from l 2024-09-11T22:42:16.326 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-11T22:42:16.572 INFO:teuthology.orchestra.run.smithi172.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.c/config 2024-09-11T22:42:17.216 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-09-11T22:42:17.216 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-0": { 2024-09-11T22:42:17.217 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1352, 2024-09-11T22:42:17.217 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1352, 2024-09-11T22:42:17.217 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 4549223, 2024-09-11T22:42:17.217 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 429176, 2024-09-11T22:42:17.217 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 7, 2024-09-11T22:42:17.217 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 2, 2024-09-11T22:42:17.217 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.110876019, 2024-09-11T22:42:17.218 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.032799070, 2024-09-11T22:42:17.218 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.034871134, 2024-09-11T22:42:17.218 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.032074133, 2024-09-11T22:42:17.218 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:17.218 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1351, 2024-09-11T22:42:17.218 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.053953030, 2024-09-11T22:42:17.218 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000039935 2024-09-11T22:42:17.218 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.219 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:17.219 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.219 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.219 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.219 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.219 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-11T22:42:17.219 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 448 2024-09-11T22:42:17.219 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.220 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-1": { 2024-09-11T22:42:17.220 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1220, 2024-09-11T22:42:17.220 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1495, 2024-09-11T22:42:17.220 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 814444, 2024-09-11T22:42:17.220 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 1033777, 2024-09-11T22:42:17.220 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 7, 2024-09-11T22:42:17.220 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 5, 2024-09-11T22:42:17.220 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.112444729, 2024-09-11T22:42:17.221 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.036924870, 2024-09-11T22:42:17.221 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.024985120, 2024-09-11T22:42:17.221 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.039337840, 2024-09-11T22:42:17.221 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:17.221 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1492, 2024-09-11T22:42:17.221 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.060018433, 2024-09-11T22:42:17.221 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000040226 2024-09-11T22:42:17.222 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.222 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:17.222 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.222 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.223 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.223 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.223 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-11T22:42:17.223 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 646448 2024-09-11T22:42:17.223 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.223 INFO:teuthology.orchestra.run.smithi172.stdout: "AsyncMessenger::Worker-2": { 2024-09-11T22:42:17.223 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_messages": 1833, 2024-09-11T22:42:17.224 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages": 1793, 2024-09-11T22:42:17.224 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_bytes": 2182188, 2024-09-11T22:42:17.224 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_bytes": 672396, 2024-09-11T22:42:17.224 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_created_connections": 4, 2024-09-11T22:42:17.224 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_active_connections": 3, 2024-09-11T22:42:17.224 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_total_time": 0.194223824, 2024-09-11T22:42:17.225 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_send_time": 0.066023338, 2024-09-11T22:42:17.225 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_recv_time": 0.085009766, 2024-09-11T22:42:17.225 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_running_fast_dispatch_time": 0.045034256, 2024-09-11T22:42:17.225 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_messages_queue_lat": { 2024-09-11T22:42:17.225 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 1792, 2024-09-11T22:42:17.225 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.080487041, 2024-09-11T22:42:17.225 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000044914 2024-09-11T22:42:17.225 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.226 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_handle_ack_lat": { 2024-09-11T22:42:17.226 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.226 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.226 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.226 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.226 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_recv_encrypted_bytes": 1723872, 2024-09-11T22:42:17.226 INFO:teuthology.orchestra.run.smithi172.stdout: "msgr_send_encrypted_bytes": 231648 2024-09-11T22:42:17.226 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.227 INFO:teuthology.orchestra.run.smithi172.stdout: "cct": { 2024-09-11T22:42:17.227 INFO:teuthology.orchestra.run.smithi172.stdout: "total_workers": 1, 2024-09-11T22:42:17.227 INFO:teuthology.orchestra.run.smithi172.stdout: "unhealthy_workers": 0 2024-09-11T22:42:17.227 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.227 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-MDSRank": { 2024-09-11T22:42:17.227 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-11T22:42:17.227 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-11T22:42:17.228 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 6564, 2024-09-11T22:42:17.228 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.173104944, 2024-09-11T22:42:17.228 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000026371 2024-09-11T22:42:17.228 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:17.228 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.228 INFO:teuthology.orchestra.run.smithi172.stdout: "finisher-PurgeQueue": { 2024-09-11T22:42:17.228 INFO:teuthology.orchestra.run.smithi172.stdout: "queue_len": 0, 2024-09-11T22:42:17.228 INFO:teuthology.orchestra.run.smithi172.stdout: "complete_latency": { 2024-09-11T22:42:17.229 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.229 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.229 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.229 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:17.229 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.229 INFO:teuthology.orchestra.run.smithi172.stdout: "mds": { 2024-09-11T22:42:17.229 INFO:teuthology.orchestra.run.smithi172.stdout: "request": 0, 2024-09-11T22:42:17.230 INFO:teuthology.orchestra.run.smithi172.stdout: "reply": 0, 2024-09-11T22:42:17.230 INFO:teuthology.orchestra.run.smithi172.stdout: "reply_latency": { 2024-09-11T22:42:17.230 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.230 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.230 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.230 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.230 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_reply": 0, 2024-09-11T22:42:17.231 INFO:teuthology.orchestra.run.smithi172.stdout: "forward": 0, 2024-09-11T22:42:17.231 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_complete": 0, 2024-09-11T22:42:17.231 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_fetch_keys": 0, 2024-09-11T22:42:17.231 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_commit": 0, 2024-09-11T22:42:17.231 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_split": 0, 2024-09-11T22:42:17.231 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_merge": 0, 2024-09-11T22:42:17.231 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes": 536, 2024-09-11T22:42:17.231 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_top": 0, 2024-09-11T22:42:17.232 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_bottom": 536, 2024-09-11T22:42:17.232 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pin_tail": 0, 2024-09-11T22:42:17.232 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_pinned": 536, 2024-09-11T22:42:17.232 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_expired": 0, 2024-09-11T22:42:17.232 INFO:teuthology.orchestra.run.smithi172.stdout: "inodes_with_caps": 0, 2024-09-11T22:42:17.232 INFO:teuthology.orchestra.run.smithi172.stdout: "caps": 0, 2024-09-11T22:42:17.232 INFO:teuthology.orchestra.run.smithi172.stdout: "subtrees": 4, 2024-09-11T22:42:17.233 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse": 0, 2024-09-11T22:42:17.233 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_hit": 0, 2024-09-11T22:42:17.233 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_forward": 0, 2024-09-11T22:42:17.233 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_discover": 0, 2024-09-11T22:42:17.233 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_dir_fetch": 0, 2024-09-11T22:42:17.233 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_remote_ino": 0, 2024-09-11T22:42:17.233 INFO:teuthology.orchestra.run.smithi172.stdout: "traverse_lock": 0, 2024-09-11T22:42:17.234 INFO:teuthology.orchestra.run.smithi172.stdout: "load_cent": 0, 2024-09-11T22:42:17.234 INFO:teuthology.orchestra.run.smithi172.stdout: "q": 0, 2024-09-11T22:42:17.234 INFO:teuthology.orchestra.run.smithi172.stdout: "exported": 0, 2024-09-11T22:42:17.234 INFO:teuthology.orchestra.run.smithi172.stdout: "exported_inodes": 0, 2024-09-11T22:42:17.234 INFO:teuthology.orchestra.run.smithi172.stdout: "imported": 0, 2024-09-11T22:42:17.234 INFO:teuthology.orchestra.run.smithi172.stdout: "imported_inodes": 0, 2024-09-11T22:42:17.234 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_dir_fetch": 0, 2024-09-11T22:42:17.234 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_backtrace_fetch": 0, 2024-09-11T22:42:17.235 INFO:teuthology.orchestra.run.smithi172.stdout: "openino_peer_discover": 0, 2024-09-11T22:42:17.235 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rfiles": 0, 2024-09-11T22:42:17.235 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rbytes": 0, 2024-09-11T22:42:17.235 INFO:teuthology.orchestra.run.smithi172.stdout: "root_rsnaps": 0, 2024-09-11T22:42:17.236 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_fetch": 0, 2024-09-11T22:42:17.236 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_set_tag": 0, 2024-09-11T22:42:17.236 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_backtrace_repaired": 0, 2024-09-11T22:42:17.236 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_inotable_repaired": 0, 2024-09-11T22:42:17.236 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_inodes": 0, 2024-09-11T22:42:17.236 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dir_base_inodes": 0, 2024-09-11T22:42:17.237 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_dirfrag_rstats": 0, 2024-09-11T22:42:17.237 INFO:teuthology.orchestra.run.smithi172.stdout: "scrub_file_inodes": 0, 2024-09-11T22:42:17.237 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_inode_file_caps": 0, 2024-09-11T22:42:17.237 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_revoke": 0, 2024-09-11T22:42:17.238 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_grant": 0, 2024-09-11T22:42:17.238 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_trunc": 0, 2024-09-11T22:42:17.238 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-11T22:42:17.238 INFO:teuthology.orchestra.run.smithi172.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-11T22:42:17.238 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps": 0, 2024-09-11T22:42:17.238 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_caps_dirty": 0, 2024-09-11T22:42:17.239 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_cap_release": 0, 2024-09-11T22:42:17.239 INFO:teuthology.orchestra.run.smithi172.stdout: "process_request_cap_release": 0 2024-09-11T22:42:17.239 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.239 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_cache": { 2024-09-11T22:42:17.239 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update": 0, 2024-09-11T22:42:17.239 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_update_receipt": 0, 2024-09-11T22:42:17.239 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_try_discover": 0, 2024-09-11T22:42:17.240 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_send_discover": 0, 2024-09-11T22:42:17.240 INFO:teuthology.orchestra.run.smithi172.stdout: "dir_handle_discover": 0, 2024-09-11T22:42:17.240 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays": 0, 2024-09-11T22:42:17.240 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_delayed": 0, 2024-09-11T22:42:17.240 INFO:teuthology.orchestra.run.smithi172.stdout: "num_strays_enqueuing": 0, 2024-09-11T22:42:17.241 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_created": 0, 2024-09-11T22:42:17.241 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_enqueued": 0, 2024-09-11T22:42:17.241 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_reintegrated": 0, 2024-09-11T22:42:17.241 INFO:teuthology.orchestra.run.smithi172.stdout: "strays_migrated": 0, 2024-09-11T22:42:17.241 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_processing": 0, 2024-09-11T22:42:17.242 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_enqueued": 0, 2024-09-11T22:42:17.242 INFO:teuthology.orchestra.run.smithi172.stdout: "num_recovering_prioritized": 0, 2024-09-11T22:42:17.242 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_started": 0, 2024-09-11T22:42:17.242 INFO:teuthology.orchestra.run.smithi172.stdout: "recovery_completed": 0, 2024-09-11T22:42:17.242 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_path": 0, 2024-09-11T22:42:17.243 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_quiesce_inode": 0, 2024-09-11T22:42:17.243 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_enqueue_scrub": 0, 2024-09-11T22:42:17.243 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_exportdir": 0, 2024-09-11T22:42:17.243 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_flush": 0, 2024-09-11T22:42:17.243 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragmentdir": 0, 2024-09-11T22:42:17.243 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_fragstats": 0, 2024-09-11T22:42:17.244 INFO:teuthology.orchestra.run.smithi172.stdout: "ireq_inodestats": 0 2024-09-11T22:42:17.244 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.244 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_log": { 2024-09-11T22:42:17.244 INFO:teuthology.orchestra.run.smithi172.stdout: "evlrg": 0, 2024-09-11T22:42:17.244 INFO:teuthology.orchestra.run.smithi172.stdout: "evadd": 0, 2024-09-11T22:42:17.244 INFO:teuthology.orchestra.run.smithi172.stdout: "evex": 0, 2024-09-11T22:42:17.245 INFO:teuthology.orchestra.run.smithi172.stdout: "evtrm": 0, 2024-09-11T22:42:17.245 INFO:teuthology.orchestra.run.smithi172.stdout: "ev": 2944, 2024-09-11T22:42:17.245 INFO:teuthology.orchestra.run.smithi172.stdout: "evexg": 0, 2024-09-11T22:42:17.245 INFO:teuthology.orchestra.run.smithi172.stdout: "evexd": 0, 2024-09-11T22:42:17.245 INFO:teuthology.orchestra.run.smithi172.stdout: "segadd": 0, 2024-09-11T22:42:17.246 INFO:teuthology.orchestra.run.smithi172.stdout: "segex": 0, 2024-09-11T22:42:17.246 INFO:teuthology.orchestra.run.smithi172.stdout: "segtrm": 0, 2024-09-11T22:42:17.246 INFO:teuthology.orchestra.run.smithi172.stdout: "seg": 5, 2024-09-11T22:42:17.246 INFO:teuthology.orchestra.run.smithi172.stdout: "segmjr": 2, 2024-09-11T22:42:17.246 INFO:teuthology.orchestra.run.smithi172.stdout: "segexg": 0, 2024-09-11T22:42:17.247 INFO:teuthology.orchestra.run.smithi172.stdout: "segexd": 0, 2024-09-11T22:42:17.247 INFO:teuthology.orchestra.run.smithi172.stdout: "expos": 4194304, 2024-09-11T22:42:17.247 INFO:teuthology.orchestra.run.smithi172.stdout: "wrpos": 8901562, 2024-09-11T22:42:17.247 INFO:teuthology.orchestra.run.smithi172.stdout: "rdpos": 8900442, 2024-09-11T22:42:17.247 INFO:teuthology.orchestra.run.smithi172.stdout: "jlat": { 2024-09-11T22:42:17.248 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.248 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.248 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.248 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.248 INFO:teuthology.orchestra.run.smithi172.stdout: "replayed": 2944 2024-09-11T22:42:17.248 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.248 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_mem": { 2024-09-11T22:42:17.249 INFO:teuthology.orchestra.run.smithi172.stdout: "ino": 100, 2024-09-11T22:42:17.249 INFO:teuthology.orchestra.run.smithi172.stdout: "ino+": 527, 2024-09-11T22:42:17.249 INFO:teuthology.orchestra.run.smithi172.stdout: "ino-": 427, 2024-09-11T22:42:17.249 INFO:teuthology.orchestra.run.smithi172.stdout: "dir": 19, 2024-09-11T22:42:17.250 INFO:teuthology.orchestra.run.smithi172.stdout: "dir+": 19, 2024-09-11T22:42:17.250 INFO:teuthology.orchestra.run.smithi172.stdout: "dir-": 0, 2024-09-11T22:42:17.251 INFO:teuthology.orchestra.run.smithi172.stdout: "dn": 536, 2024-09-11T22:42:17.251 INFO:teuthology.orchestra.run.smithi172.stdout: "dn+": 536, 2024-09-11T22:42:17.251 INFO:teuthology.orchestra.run.smithi172.stdout: "dn-": 0, 2024-09-11T22:42:17.252 INFO:teuthology.orchestra.run.smithi172.stdout: "cap": 0, 2024-09-11T22:42:17.252 INFO:teuthology.orchestra.run.smithi172.stdout: "cap+": 0, 2024-09-11T22:42:17.252 INFO:teuthology.orchestra.run.smithi172.stdout: "cap-": 0, 2024-09-11T22:42:17.252 INFO:teuthology.orchestra.run.smithi172.stdout: "rss": 50268, 2024-09-11T22:42:17.252 INFO:teuthology.orchestra.run.smithi172.stdout: "heap": 215328 2024-09-11T22:42:17.252 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.252 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_server": { 2024-09-11T22:42:17.253 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_client_request": 0, 2024-09-11T22:42:17.253 INFO:teuthology.orchestra.run.smithi172.stdout: "dispatch_server_request": 0, 2024-09-11T22:42:17.253 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_request": 0, 2024-09-11T22:42:17.253 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_client_session": 0, 2024-09-11T22:42:17.253 INFO:teuthology.orchestra.run.smithi172.stdout: "handle_peer_request": 0, 2024-09-11T22:42:17.254 INFO:teuthology.orchestra.run.smithi172.stdout: "req_create_latency": { 2024-09-11T22:42:17.254 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.254 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.254 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.254 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.254 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getattr_latency": { 2024-09-11T22:42:17.255 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.255 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.255 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.255 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.255 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getfilelock_latency": { 2024-09-11T22:42:17.255 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.255 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.255 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.256 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.256 INFO:teuthology.orchestra.run.smithi172.stdout: "req_link_latency": { 2024-09-11T22:42:17.256 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.256 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.256 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.256 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.256 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookup_latency": { 2024-09-11T22:42:17.256 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.257 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.257 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.257 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.257 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookuphash_latency": { 2024-09-11T22:42:17.257 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.257 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.257 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.258 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.258 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupino_latency": { 2024-09-11T22:42:17.258 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.258 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.258 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.258 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.258 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupname_latency": { 2024-09-11T22:42:17.258 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.258 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.259 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.259 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.259 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupparent_latency": { 2024-09-11T22:42:17.259 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.259 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.260 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.260 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.260 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lookupsnap_latency": { 2024-09-11T22:42:17.260 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.260 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.260 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.261 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.261 INFO:teuthology.orchestra.run.smithi172.stdout: "req_lssnap_latency": { 2024-09-11T22:42:17.261 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.261 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.261 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.261 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.261 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mkdir_latency": { 2024-09-11T22:42:17.262 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.262 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.262 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.262 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.263 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mknod_latency": { 2024-09-11T22:42:17.263 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.263 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.263 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.263 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.263 INFO:teuthology.orchestra.run.smithi172.stdout: "req_mksnap_latency": { 2024-09-11T22:42:17.263 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.264 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.264 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.264 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.264 INFO:teuthology.orchestra.run.smithi172.stdout: "req_open_latency": { 2024-09-11T22:42:17.264 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.265 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.265 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.265 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.265 INFO:teuthology.orchestra.run.smithi172.stdout: "req_readdir_latency": { 2024-09-11T22:42:17.265 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.265 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.266 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.266 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.266 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rename_latency": { 2024-09-11T22:42:17.267 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.267 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.267 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.267 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.268 INFO:teuthology.orchestra.run.smithi172.stdout: "req_renamesnap_latency": { 2024-09-11T22:42:17.268 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.268 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.268 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.269 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.269 INFO:teuthology.orchestra.run.smithi172.stdout: "req_snapdiff_latency": { 2024-09-11T22:42:17.269 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.269 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.269 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.270 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.270 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmdir_latency": { 2024-09-11T22:42:17.270 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.270 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.271 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.271 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.271 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmsnap_latency": { 2024-09-11T22:42:17.272 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.272 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.272 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.273 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.273 INFO:teuthology.orchestra.run.smithi172.stdout: "req_rmxattr_latency": { 2024-09-11T22:42:17.273 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.273 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.274 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.274 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.274 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setattr_latency": { 2024-09-11T22:42:17.274 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.274 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.275 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.275 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.275 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setdirlayout_latency": { 2024-09-11T22:42:17.275 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.276 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.276 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.276 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.277 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setfilelock_latency": { 2024-09-11T22:42:17.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.277 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.277 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.277 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setlayout_latency": { 2024-09-11T22:42:17.277 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.278 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.278 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.278 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.279 INFO:teuthology.orchestra.run.smithi172.stdout: "req_setxattr_latency": { 2024-09-11T22:42:17.279 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.279 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.280 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.280 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.281 INFO:teuthology.orchestra.run.smithi172.stdout: "req_symlink_latency": { 2024-09-11T22:42:17.281 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.281 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.282 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.282 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.282 INFO:teuthology.orchestra.run.smithi172.stdout: "req_unlink_latency": { 2024-09-11T22:42:17.282 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.283 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.283 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.283 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.283 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_revoke_eviction": 0, 2024-09-11T22:42:17.283 INFO:teuthology.orchestra.run.smithi172.stdout: "cap_acquisition_throttle": 0, 2024-09-11T22:42:17.284 INFO:teuthology.orchestra.run.smithi172.stdout: "req_getvxattr_latency": { 2024-09-11T22:42:17.284 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.285 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.285 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.285 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:17.285 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.285 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_sessions": { 2024-09-11T22:42:17.285 INFO:teuthology.orchestra.run.smithi172.stdout: "session_count": 2, 2024-09-11T22:42:17.285 INFO:teuthology.orchestra.run.smithi172.stdout: "session_add": 4, 2024-09-11T22:42:17.286 INFO:teuthology.orchestra.run.smithi172.stdout: "session_remove": 2, 2024-09-11T22:42:17.286 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_open": 4, 2024-09-11T22:42:17.286 INFO:teuthology.orchestra.run.smithi172.stdout: "sessions_stale": 0, 2024-09-11T22:42:17.286 INFO:teuthology.orchestra.run.smithi172.stdout: "total_load": 0, 2024-09-11T22:42:17.286 INFO:teuthology.orchestra.run.smithi172.stdout: "average_load": 0, 2024-09-11T22:42:17.286 INFO:teuthology.orchestra.run.smithi172.stdout: "avg_session_uptime": 3827, 2024-09-11T22:42:17.286 INFO:teuthology.orchestra.run.smithi172.stdout: "mdthresh_evicted": 0 2024-09-11T22:42:17.286 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.287 INFO:teuthology.orchestra.run.smithi172.stdout: "mempool": { 2024-09-11T22:42:17.287 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_bytes": 0, 2024-09-11T22:42:17.287 INFO:teuthology.orchestra.run.smithi172.stdout: "bloom_filter_items": 0, 2024-09-11T22:42:17.287 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_bytes": 0, 2024-09-11T22:42:17.287 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_alloc_items": 0, 2024-09-11T22:42:17.287 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_bytes": 0, 2024-09-11T22:42:17.287 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_data_items": 0, 2024-09-11T22:42:17.287 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-11T22:42:17.288 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_onode_items": 0, 2024-09-11T22:42:17.288 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-11T22:42:17.288 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_meta_items": 0, 2024-09-11T22:42:17.288 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_bytes": 0, 2024-09-11T22:42:17.288 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_other_items": 0, 2024-09-11T22:42:17.288 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-11T22:42:17.288 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_cache_buffer_items": 0, 2024-09-11T22:42:17.289 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_bytes": 0, 2024-09-11T22:42:17.289 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_extent_items": 0, 2024-09-11T22:42:17.289 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_bytes": 0, 2024-09-11T22:42:17.289 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_blob_items": 0, 2024-09-11T22:42:17.290 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-11T22:42:17.290 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_shared_blob_items": 0, 2024-09-11T22:42:17.290 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-11T22:42:17.290 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_inline_bl_items": 0, 2024-09-11T22:42:17.290 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_bytes": 0, 2024-09-11T22:42:17.290 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_fsck_items": 0, 2024-09-11T22:42:17.290 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_bytes": 0, 2024-09-11T22:42:17.290 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_txc_items": 0, 2024-09-11T22:42:17.291 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-11T22:42:17.291 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_deferred_items": 0, 2024-09-11T22:42:17.291 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_bytes": 0, 2024-09-11T22:42:17.291 INFO:teuthology.orchestra.run.smithi172.stdout: "bluestore_writing_items": 0, 2024-09-11T22:42:17.291 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_bytes": 0, 2024-09-11T22:42:17.291 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_items": 0, 2024-09-11T22:42:17.291 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_bytes": 0, 2024-09-11T22:42:17.292 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_reader_items": 0, 2024-09-11T22:42:17.292 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_bytes": 0, 2024-09-11T22:42:17.292 INFO:teuthology.orchestra.run.smithi172.stdout: "bluefs_file_writer_items": 0, 2024-09-11T22:42:17.292 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_bytes": 201617, 2024-09-11T22:42:17.292 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_anon_items": 147, 2024-09-11T22:42:17.292 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_bytes": 0, 2024-09-11T22:42:17.292 INFO:teuthology.orchestra.run.smithi172.stdout: "buffer_meta_items": 0, 2024-09-11T22:42:17.292 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_bytes": 0, 2024-09-11T22:42:17.293 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_items": 0, 2024-09-11T22:42:17.293 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_bytes": 0, 2024-09-11T22:42:17.293 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_mapbl_items": 0, 2024-09-11T22:42:17.293 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_bytes": 0, 2024-09-11T22:42:17.293 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_pglog_items": 0, 2024-09-11T22:42:17.293 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_bytes": 8632, 2024-09-11T22:42:17.294 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_items": 190, 2024-09-11T22:42:17.294 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_bytes": 0, 2024-09-11T22:42:17.294 INFO:teuthology.orchestra.run.smithi172.stdout: "osdmap_mapping_items": 0, 2024-09-11T22:42:17.294 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_bytes": 0, 2024-09-11T22:42:17.295 INFO:teuthology.orchestra.run.smithi172.stdout: "pgmap_items": 0, 2024-09-11T22:42:17.295 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_bytes": 650351, 2024-09-11T22:42:17.295 INFO:teuthology.orchestra.run.smithi172.stdout: "mds_co_items": 4344, 2024-09-11T22:42:17.295 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_bytes": 0, 2024-09-11T22:42:17.296 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_1_items": 0, 2024-09-11T22:42:17.296 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_bytes": 0, 2024-09-11T22:42:17.296 INFO:teuthology.orchestra.run.smithi172.stdout: "unittest_2_items": 0 2024-09-11T22:42:17.297 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.297 INFO:teuthology.orchestra.run.smithi172.stdout: "objecter": { 2024-09-11T22:42:17.297 INFO:teuthology.orchestra.run.smithi172.stdout: "op_active": 0, 2024-09-11T22:42:17.297 INFO:teuthology.orchestra.run.smithi172.stdout: "op_laggy": 0, 2024-09-11T22:42:17.297 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send": 3952, 2024-09-11T22:42:17.297 INFO:teuthology.orchestra.run.smithi172.stdout: "op_send_bytes": 21, 2024-09-11T22:42:17.298 INFO:teuthology.orchestra.run.smithi172.stdout: "op_resend": 3, 2024-09-11T22:42:17.298 INFO:teuthology.orchestra.run.smithi172.stdout: "op_reply": 3952, 2024-09-11T22:42:17.298 INFO:teuthology.orchestra.run.smithi172.stdout: "op_latency": { 2024-09-11T22:42:17.298 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3952, 2024-09-11T22:42:17.298 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 2.904927108, 2024-09-11T22:42:17.298 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000735052 2024-09-11T22:42:17.299 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.299 INFO:teuthology.orchestra.run.smithi172.stdout: "op_inflight": 0, 2024-09-11T22:42:17.299 INFO:teuthology.orchestra.run.smithi172.stdout: "oplen_avg": { 2024-09-11T22:42:17.299 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 3952, 2024-09-11T22:42:17.300 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 3953 2024-09-11T22:42:17.300 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.300 INFO:teuthology.orchestra.run.smithi172.stdout: "op": 3952, 2024-09-11T22:42:17.300 INFO:teuthology.orchestra.run.smithi172.stdout: "op_r": 3952, 2024-09-11T22:42:17.300 INFO:teuthology.orchestra.run.smithi172.stdout: "op_w": 0, 2024-09-11T22:42:17.300 INFO:teuthology.orchestra.run.smithi172.stdout: "op_rmw": 0, 2024-09-11T22:42:17.300 INFO:teuthology.orchestra.run.smithi172.stdout: "op_pg": 0, 2024-09-11T22:42:17.301 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_stat": 2502, 2024-09-11T22:42:17.301 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_create": 0, 2024-09-11T22:42:17.301 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_read": 1448, 2024-09-11T22:42:17.301 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_write": 0, 2024-09-11T22:42:17.302 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writefull": 0, 2024-09-11T22:42:17.302 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_writesame": 0, 2024-09-11T22:42:17.302 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_append": 0, 2024-09-11T22:42:17.302 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_zero": 0, 2024-09-11T22:42:17.302 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_truncate": 0, 2024-09-11T22:42:17.302 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_delete": 0, 2024-09-11T22:42:17.302 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_mapext": 0, 2024-09-11T22:42:17.302 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_sparse_read": 0, 2024-09-11T22:42:17.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_clonerange": 0, 2024-09-11T22:42:17.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_getxattr": 1, 2024-09-11T22:42:17.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_setxattr": 0, 2024-09-11T22:42:17.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_cmpxattr": 0, 2024-09-11T22:42:17.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_rmxattr": 0, 2024-09-11T22:42:17.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_resetxattrs": 0, 2024-09-11T22:42:17.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_call": 0, 2024-09-11T22:42:17.303 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_watch": 0, 2024-09-11T22:42:17.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_notify": 0, 2024-09-11T22:42:17.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_src_cmpxattr": 0, 2024-09-11T22:42:17.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls": 0, 2024-09-11T22:42:17.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_pgls_filter": 0, 2024-09-11T22:42:17.304 INFO:teuthology.orchestra.run.smithi172.stdout: "osdop_other": 0, 2024-09-11T22:42:17.304 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_active": 0, 2024-09-11T22:42:17.304 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_send": 0, 2024-09-11T22:42:17.304 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_resend": 0, 2024-09-11T22:42:17.305 INFO:teuthology.orchestra.run.smithi172.stdout: "linger_ping": 0, 2024-09-11T22:42:17.305 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_active": 0, 2024-09-11T22:42:17.305 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_send": 0, 2024-09-11T22:42:17.305 INFO:teuthology.orchestra.run.smithi172.stdout: "poolop_resend": 0, 2024-09-11T22:42:17.305 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_active": 0, 2024-09-11T22:42:17.306 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_send": 0, 2024-09-11T22:42:17.306 INFO:teuthology.orchestra.run.smithi172.stdout: "poolstat_resend": 0, 2024-09-11T22:42:17.306 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_active": 0, 2024-09-11T22:42:17.306 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_send": 0, 2024-09-11T22:42:17.306 INFO:teuthology.orchestra.run.smithi172.stdout: "statfs_resend": 0, 2024-09-11T22:42:17.306 INFO:teuthology.orchestra.run.smithi172.stdout: "command_active": 0, 2024-09-11T22:42:17.306 INFO:teuthology.orchestra.run.smithi172.stdout: "command_send": 0, 2024-09-11T22:42:17.306 INFO:teuthology.orchestra.run.smithi172.stdout: "command_resend": 0, 2024-09-11T22:42:17.307 INFO:teuthology.orchestra.run.smithi172.stdout: "map_epoch": 120, 2024-09-11T22:42:17.307 INFO:teuthology.orchestra.run.smithi172.stdout: "map_full": 0, 2024-09-11T22:42:17.307 INFO:teuthology.orchestra.run.smithi172.stdout: "map_inc": 40, 2024-09-11T22:42:17.307 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_sessions": 6, 2024-09-11T22:42:17.307 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_open": 7, 2024-09-11T22:42:17.307 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_session_close": 1, 2024-09-11T22:42:17.307 INFO:teuthology.orchestra.run.smithi172.stdout: "osd_laggy": 0, 2024-09-11T22:42:17.308 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_wr": 0, 2024-09-11T22:42:17.308 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_rd": 2, 2024-09-11T22:42:17.308 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_del": 0 2024-09-11T22:42:17.308 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.308 INFO:teuthology.orchestra.run.smithi172.stdout: "oft": { 2024-09-11T22:42:17.308 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_objs": 0, 2024-09-11T22:42:17.309 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_kv_pairs": 0, 2024-09-11T22:42:17.309 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_updates": 0, 2024-09-11T22:42:17.309 INFO:teuthology.orchestra.run.smithi172.stdout: "omap_total_removes": 0 2024-09-11T22:42:17.309 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.309 INFO:teuthology.orchestra.run.smithi172.stdout: "osd-slow-ops": { 2024-09-11T22:42:17.309 INFO:teuthology.orchestra.run.smithi172.stdout: "slow_ops_count": 0 2024-09-11T22:42:17.309 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.309 INFO:teuthology.orchestra.run.smithi172.stdout: "purge_queue": { 2024-09-11T22:42:17.310 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops": 0, 2024-09-11T22:42:17.310 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_ops_high_water": 0, 2024-09-11T22:42:17.310 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing": 0, 2024-09-11T22:42:17.310 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executing_high_water": 0, 2024-09-11T22:42:17.310 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed_ops": 0, 2024-09-11T22:42:17.311 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_executed": 0, 2024-09-11T22:42:17.311 INFO:teuthology.orchestra.run.smithi172.stdout: "pq_item_in_journal": 0 2024-09-11T22:42:17.311 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.311 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-11T22:42:17.311 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:17.311 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-11T22:42:17.311 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:17.311 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 4401, 2024-09-11T22:42:17.312 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 7124111, 2024-09-11T22:42:17.312 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:17.312 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 4401, 2024-09-11T22:42:17.312 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:17.312 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:17.312 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 4401, 2024-09-11T22:42:17.313 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 7124111, 2024-09-11T22:42:17.313 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:17.313 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.313 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.313 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.313 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:17.314 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.314 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_bytes": { 2024-09-11T22:42:17.314 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:17.314 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 104857600, 2024-09-11T22:42:17.314 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:17.314 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:17.314 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:17.315 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:17.315 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:17.315 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 3952, 2024-09-11T22:42:17.315 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 4707263, 2024-09-11T22:42:17.315 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 3952, 2024-09-11T22:42:17.315 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 4707263, 2024-09-11T22:42:17.315 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:17.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.316 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.316 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.316 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:17.316 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.317 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-objecter_ops": { 2024-09-11T22:42:17.317 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:17.317 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 1024, 2024-09-11T22:42:17.317 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:17.317 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:17.317 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:17.318 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:17.318 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:17.318 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 3952, 2024-09-11T22:42:17.318 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 3952, 2024-09-11T22:42:17.318 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 3952, 2024-09-11T22:42:17.318 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 3952, 2024-09-11T22:42:17.319 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:17.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.319 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.319 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.319 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:17.320 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.320 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle": { 2024-09-11T22:42:17.320 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:17.320 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-11T22:42:17.321 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:17.321 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:17.321 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:17.321 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:17.321 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:17.321 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:17.322 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:17.322 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-11T22:42:17.322 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-11T22:42:17.322 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:17.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.322 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.322 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.322 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:17.323 INFO:teuthology.orchestra.run.smithi172.stdout: }, 2024-09-11T22:42:17.323 INFO:teuthology.orchestra.run.smithi172.stdout: "throttle-write_buf_throttle-0x563905890b40": { 2024-09-11T22:42:17.323 INFO:teuthology.orchestra.run.smithi172.stdout: "val": 0, 2024-09-11T22:42:17.323 INFO:teuthology.orchestra.run.smithi172.stdout: "max": 3758096384, 2024-09-11T22:42:17.323 INFO:teuthology.orchestra.run.smithi172.stdout: "get_started": 0, 2024-09-11T22:42:17.323 INFO:teuthology.orchestra.run.smithi172.stdout: "get": 0, 2024-09-11T22:42:17.323 INFO:teuthology.orchestra.run.smithi172.stdout: "get_sum": 0, 2024-09-11T22:42:17.323 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_fail": 0, 2024-09-11T22:42:17.324 INFO:teuthology.orchestra.run.smithi172.stdout: "get_or_fail_success": 0, 2024-09-11T22:42:17.324 INFO:teuthology.orchestra.run.smithi172.stdout: "take": 0, 2024-09-11T22:42:17.324 INFO:teuthology.orchestra.run.smithi172.stdout: "take_sum": 0, 2024-09-11T22:42:17.324 INFO:teuthology.orchestra.run.smithi172.stdout: "put": 0, 2024-09-11T22:42:17.324 INFO:teuthology.orchestra.run.smithi172.stdout: "put_sum": 0, 2024-09-11T22:42:17.324 INFO:teuthology.orchestra.run.smithi172.stdout: "wait": { 2024-09-11T22:42:17.324 INFO:teuthology.orchestra.run.smithi172.stdout: "avgcount": 0, 2024-09-11T22:42:17.325 INFO:teuthology.orchestra.run.smithi172.stdout: "sum": 0.000000000, 2024-09-11T22:42:17.325 INFO:teuthology.orchestra.run.smithi172.stdout: "avgtime": 0.000000000 2024-09-11T22:42:17.325 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:17.325 INFO:teuthology.orchestra.run.smithi172.stdout: } 2024-09-11T22:42:17.325 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-09-11T22:42:17.933 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-11T22:42:17.933 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-11T22:42:17.933 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-11T22:42:17.934 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-11T22:42:17.934 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-11T22:42:17.934 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-11T22:42:17.934 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-11T22:42:17.934 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-11T22:42:17.944 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T22:42:17.945 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:42:17.945 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:42:17.967 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-11T22:42:17.967 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T22:42:18.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-11T22:42:18.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:17 smithi005 ceph-mon[30943]: pgmap v640: 97 pgs: 11 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 78 active+clean; 526 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 133 KiB/s wr, 15 op/s 2024-09-11T22:42:18.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:17 smithi159 ceph-mon[35534]: pgmap v640: 97 pgs: 11 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 78 active+clean; 526 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 133 KiB/s wr, 15 op/s 2024-09-11T22:42:18.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:17 smithi172 ceph-mon[35461]: pgmap v640: 97 pgs: 11 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 78 active+clean; 526 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 133 KiB/s wr, 15 op/s 2024-09-11T22:42:18.340 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:42:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.855+0000 7f6a5a26b640 1 -- 172.21.15.5:0/4110914765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a54102790 msgr2=0x7f6a541067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.855+0000 7f6a5a26b640 1 --2- 172.21.15.5:0/4110914765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a54102790 0x7f6a541067e0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f6a48009900 tx=0x7f6a4802a410 comp rx=0 tx=0).stop 2024-09-11T22:42:18.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.855+0000 7f6a5a26b640 1 -- 172.21.15.5:0/4110914765 shutdown_connections 2024-09-11T22:42:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.855+0000 7f6a5a26b640 1 --2- 172.21.15.5:0/4110914765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a54106f10 0x7f6a5410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.855+0000 7f6a5a26b640 1 --2- 172.21.15.5:0/4110914765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a54102790 0x7f6a541067e0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.855+0000 7f6a5a26b640 1 --2- 172.21.15.5:0/4110914765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a54101de0 0x7f6a541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.855+0000 7f6a5a26b640 1 -- 172.21.15.5:0/4110914765 >> 172.21.15.5:0/4110914765 conn(0x7f6a540fd8c0 msgr2=0x7f6a540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.855+0000 7f6a5a26b640 1 -- 172.21.15.5:0/4110914765 shutdown_connections 2024-09-11T22:42:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.855+0000 7f6a5a26b640 1 -- 172.21.15.5:0/4110914765 wait complete. 2024-09-11T22:42:18.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a5a26b640 1 Processor -- start 2024-09-11T22:42:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a5a26b640 1 -- start start 2024-09-11T22:42:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a5a26b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a54101de0 0x7f6a5419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a5a26b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a54102790 0x7f6a5419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a5a26b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a54106f10 0x7f6a5419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a5a26b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54111f30 con 0x7f6a54106f10 2024-09-11T22:42:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a5a26b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54111db0 con 0x7f6a54101de0 2024-09-11T22:42:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a5a26b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541120b0 con 0x7f6a54102790 2024-09-11T22:42:18.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a58808640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a54106f10 0x7f6a5419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:18.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a58808640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a54106f10 0x7f6a5419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45786/0 (socket says 172.21.15.5:45786) 2024-09-11T22:42:18.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.856+0000 7f6a58808640 1 -- 172.21.15.5:0/504459046 learned_addr learned my addr 172.21.15.5:0/504459046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:42:18.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a53fff640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a54101de0 0x7f6a5419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:18.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a537fe640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a54102790 0x7f6a5419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:18.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a58808640 1 -- 172.21.15.5:0/504459046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a54102790 msgr2=0x7f6a5419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:18.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a58808640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a54102790 0x7f6a5419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:18.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a58808640 1 -- 172.21.15.5:0/504459046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a54101de0 msgr2=0x7f6a5419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:18.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a58808640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a54101de0 0x7f6a5419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:18.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a58808640 1 -- 172.21.15.5:0/504459046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5419f600 con 0x7f6a54106f10 2024-09-11T22:42:18.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a58808640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a54106f10 0x7f6a5419ef20 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009fd0 tx=0x7f6a4c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:18.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a517fa640 1 -- 172.21.15.5:0/504459046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c017070 con 0x7f6a54106f10 2024-09-11T22:42:18.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a517fa640 1 -- 172.21.15.5:0/504459046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4c003e50 con 0x7f6a54106f10 2024-09-11T22:42:18.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a5a26b640 1 -- 172.21.15.5:0/504459046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5419f8f0 con 0x7f6a54106f10 2024-09-11T22:42:18.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a517fa640 1 -- 172.21.15.5:0/504459046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0106c0 con 0x7f6a54106f10 2024-09-11T22:42:18.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.857+0000 7f6a5a26b640 1 -- 172.21.15.5:0/504459046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a541a71d0 con 0x7f6a54106f10 2024-09-11T22:42:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.858+0000 7f6a5a26b640 1 -- 172.21.15.5:0/504459046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a541021c0 con 0x7f6a54106f10 2024-09-11T22:42:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.859+0000 7f6a517fa640 1 -- 172.21.15.5:0/504459046 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f6a4c003970 con 0x7f6a54106f10 2024-09-11T22:42:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.860+0000 7f6a517fa640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6a3c0a2cf0 0x7f6a3c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.860+0000 7f6a517fa640 1 -- 172.21.15.5:0/504459046 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8673+0+0 (secure 0 0 0) 0x7f6a4c0cf1b0 con 0x7f6a54106f10 2024-09-11T22:42:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.860+0000 7f6a53fff640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6a3c0a2cf0 0x7f6a3c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.861+0000 7f6a53fff640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6a3c0a2cf0 0x7f6a3c0a51b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400bbe0 tx=0x7f6a4400b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:18.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:18.864+0000 7f6a517fa640 1 -- 172.21.15.5:0/504459046 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a4c09b6e0 con 0x7f6a54106f10 2024-09-11T22:42:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.052+0000 7f6a5a26b640 1 -- 172.21.15.5:0/504459046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6a541a73a0 con 0x7f6a54106f10 2024-09-11T22:42:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.053+0000 7f6a517fa640 1 -- 172.21.15.5:0/504459046 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v120) ==== 81+0+927 (secure 0 0 0) 0x7f6a4c09f9e0 con 0x7f6a54106f10 2024-09-11T22:42:19.054 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2380674497 2024-09-12T22:22:15.968309+0000 2024-09-11T22:42:19.054 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1194241380 2024-09-12T22:12:52.569019+0000 2024-09-11T22:42:19.055 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3381830267 2024-09-12T22:12:52.569019+0000 2024-09-11T22:42:19.055 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/3591749770 2024-09-12T22:22:15.968309+0000 2024-09-11T22:42:19.055 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2381319073 2024-09-12T22:13:16.336721+0000 2024-09-11T22:42:19.055 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2378444675 2024-09-12T22:12:52.569019+0000 2024-09-11T22:42:19.055 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1194241380 2024-09-12T22:12:52.569019+0000 2024-09-11T22:42:19.055 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1577283141 2024-09-12T22:22:15.968309+0000 2024-09-11T22:42:19.055 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/3591749770 2024-09-12T22:22:15.968309+0000 2024-09-11T22:42:19.055 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1875833568 2024-09-12T22:12:52.569019+0000 2024-09-11T22:42:19.056 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/981960152 2024-09-12T22:13:16.336721+0000 2024-09-11T22:42:19.056 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/497465376 2024-09-12T22:13:16.336721+0000 2024-09-11T22:42:19.056 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/3503484776 2024-09-12T22:13:16.336721+0000 2024-09-11T22:42:19.056 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/3503484776 2024-09-12T22:13:16.336721+0000 2024-09-11T22:42:19.056 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1618823752 2024-09-12T22:22:15.968309+0000 2024-09-11T22:42:19.056 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/785116270 2024-09-12T22:22:15.968309+0000 2024-09-11T22:42:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:listed 16 entries 2024-09-11T22:42:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 -- 172.21.15.5:0/504459046 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6a3c0a2cf0 msgr2=0x7f6a3c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6a3c0a2cf0 0x7f6a3c0a51b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400bbe0 tx=0x7f6a4400b390 comp rx=0 tx=0).stop 2024-09-11T22:42:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 -- 172.21.15.5:0/504459046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a54106f10 msgr2=0x7f6a5419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a54106f10 0x7f6a5419ef20 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009fd0 tx=0x7f6a4c00b260 comp rx=0 tx=0).stop 2024-09-11T22:42:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 -- 172.21.15.5:0/504459046 shutdown_connections 2024-09-11T22:42:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a54106f10 0x7f6a5419ef20 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a54102790 0x7f6a5419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f6a3c0a2cf0 0x7f6a3c0a51b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 --2- 172.21.15.5:0/504459046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a54101de0 0x7f6a5419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 -- 172.21.15.5:0/504459046 >> 172.21.15.5:0/504459046 conn(0x7f6a540fd8c0 msgr2=0x7f6a540fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 -- 172.21.15.5:0/504459046 shutdown_connections 2024-09-11T22:42:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:19.055+0000 7f6a5a26b640 1 -- 172.21.15.5:0/504459046 wait complete. 2024-09-11T22:42:19.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:19 smithi005 ceph-mon[30943]: pgmap v641: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 14 op/s 2024-09-11T22:42:19.770 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:19 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/504459046' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:42:19.771 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T22:42:19.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:42:19.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T22:42:19.897 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T22:42:19.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:42:19.897 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T22:42:19.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:42:19.947 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-11T22:42:19.996 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T22:42:19.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:42:19.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T22:42:20.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T22:42:20.072 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-11T22:42:20.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:19 smithi159 ceph-mon[35534]: pgmap v641: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 14 op/s 2024-09-11T22:42:20.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:19 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/504459046' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:42:20.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:19 smithi172 ceph-mon[35461]: pgmap v641: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 117 KiB/s wr, 14 op/s 2024-09-11T22:42:20.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:19 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/504459046' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T22:42:20.168 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T22:42:20.178 INFO:tasks.cephadm:Teardown begin 2024-09-11T22:42:20.178 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T22:42:20.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:42:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:42:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.389+0000 7f1aab28b640 1 -- 172.21.15.5:0/649352615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa4102da0 msgr2=0x7f1aa4103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.389+0000 7f1aab28b640 1 --2- 172.21.15.5:0/649352615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa4102da0 0x7f1aa4103180 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009900 tx=0x7f1aa002a480 comp rx=0 tx=0).stop 2024-09-11T22:42:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.389+0000 7f1aab28b640 1 -- 172.21.15.5:0/649352615 shutdown_connections 2024-09-11T22:42:21.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.389+0000 7f1aab28b640 1 --2- 172.21.15.5:0/649352615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1aa410c180 0x7f1aa410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.389+0000 7f1aab28b640 1 --2- 172.21.15.5:0/649352615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1aa41036c0 0x7f1aa410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.389+0000 7f1aab28b640 1 --2- 172.21.15.5:0/649352615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa4102da0 0x7f1aa4103180 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.389+0000 7f1aab28b640 1 -- 172.21.15.5:0/649352615 >> 172.21.15.5:0/649352615 conn(0x7f1aa40fc7a0 msgr2=0x7f1aa40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.389+0000 7f1aab28b640 1 -- 172.21.15.5:0/649352615 shutdown_connections 2024-09-11T22:42:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.389+0000 7f1aab28b640 1 -- 172.21.15.5:0/649352615 wait complete. 2024-09-11T22:42:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.390+0000 7f1aab28b640 1 Processor -- start 2024-09-11T22:42:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.390+0000 7f1aab28b640 1 -- start start 2024-09-11T22:42:21.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.390+0000 7f1aab28b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1aa4102da0 0x7f1aa4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.390+0000 7f1aab28b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa41036c0 0x7f1aa4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.390+0000 7f1aab28b640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1aa410c180 0x7f1aa419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.390+0000 7f1aab28b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4111f60 con 0x7f1aa41036c0 2024-09-11T22:42:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.390+0000 7f1aab28b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa4111de0 con 0x7f1aa4102da0 2024-09-11T22:42:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.390+0000 7f1aab28b640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa41120e0 con 0x7f1aa410c180 2024-09-11T22:42:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa8826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa41036c0 0x7f1aa4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:21.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa8826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa41036c0 0x7f1aa4198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46654/0 (socket says 172.21.15.5:46654) 2024-09-11T22:42:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa8826640 1 -- 172.21.15.5:0/2929278772 learned_addr learned my addr 172.21.15.5:0/2929278772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:42:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa9828640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1aa410c180 0x7f1aa419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa9027640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1aa4102da0 0x7f1aa4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa8826640 1 -- 172.21.15.5:0/2929278772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1aa410c180 msgr2=0x7f1aa419cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa8826640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1aa410c180 0x7f1aa419cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa8826640 1 -- 172.21.15.5:0/2929278772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1aa4102da0 msgr2=0x7f1aa4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa8826640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1aa4102da0 0x7f1aa4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:21.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa8826640 1 -- 172.21.15.5:0/2929278772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa419d3f0 con 0x7f1aa41036c0 2024-09-11T22:42:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aa8826640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa41036c0 0x7f1aa4198980 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400d950 tx=0x7f1a9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1a9a7fc640 1 -- 172.21.15.5:0/2929278772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a94017070 con 0x7f1aa41036c0 2024-09-11T22:42:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1a9a7fc640 1 -- 172.21.15.5:0/2929278772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a94009c30 con 0x7f1aa41036c0 2024-09-11T22:42:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aab28b640 1 -- 172.21.15.5:0/2929278772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa419d6e0 con 0x7f1aa41036c0 2024-09-11T22:42:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1a9a7fc640 1 -- 172.21.15.5:0/2929278772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9400f6b0 con 0x7f1aa41036c0 2024-09-11T22:42:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.391+0000 7f1aab28b640 1 -- 172.21.15.5:0/2929278772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa41a4fc0 con 0x7f1aa41036c0 2024-09-11T22:42:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.392+0000 7f1aab28b640 1 -- 172.21.15.5:0/2929278772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa4103180 con 0x7f1aa41036c0 2024-09-11T22:42:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.394+0000 7f1a9a7fc640 1 -- 172.21.15.5:0/2929278772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f1a940109d0 con 0x7f1aa41036c0 2024-09-11T22:42:21.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.394+0000 7f1a9a7fc640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1a900a2d70 0x7f1a900a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:21.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.395+0000 7f1aa9027640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1a900a2d70 0x7f1a900a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:21.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.395+0000 7f1a9a7fc640 1 -- 172.21.15.5:0/2929278772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8673+0+0 (secure 0 0 0) 0x7f1a940cf0e0 con 0x7f1aa41036c0 2024-09-11T22:42:21.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.395+0000 7f1aa9027640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1a900a2d70 0x7f1a900a5230 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0005ab0 tx=0x7f1aa0005a20 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:21.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.399+0000 7f1a9a7fc640 1 -- 172.21.15.5:0/2929278772 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a94009de0 con 0x7f1aa41036c0 2024-09-11T22:42:21.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.618+0000 7f1aab28b640 1 -- 172.21.15.5:0/2929278772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1aa402d7c0 con 0x7f1aa41036c0 2024-09-11T22:42:21.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.619+0000 7f1a9a7fc640 1 -- 172.21.15.5:0/2929278772 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11975 (secure 0 0 0) 0x7f1a9409bc60 con 0x7f1aa41036c0 2024-09-11T22:42:21.620 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:42:21.620 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":57,"btime":"2024-09-11T22:40:39:632093+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:40:38.909431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:42:21.621 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 57 2024-09-11T22:42:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 -- 172.21.15.5:0/2929278772 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1a900a2d70 msgr2=0x7f1a900a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:21.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1a900a2d70 0x7f1a900a5230 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0005ab0 tx=0x7f1aa0005a20 comp rx=0 tx=0).stop 2024-09-11T22:42:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 -- 172.21.15.5:0/2929278772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa41036c0 msgr2=0x7f1aa4198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa41036c0 0x7f1aa4198980 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400d950 tx=0x7f1a9400de10 comp rx=0 tx=0).stop 2024-09-11T22:42:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 -- 172.21.15.5:0/2929278772 shutdown_connections 2024-09-11T22:42:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1aa410c180 0x7f1aa419cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1aa41036c0 0x7f1aa4198980 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f1a900a2d70 0x7f1a900a5230 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:21.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 --2- 172.21.15.5:0/2929278772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1aa4102da0 0x7f1aa4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 -- 172.21.15.5:0/2929278772 >> 172.21.15.5:0/2929278772 conn(0x7f1aa40fc7a0 msgr2=0x7f1aa40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 -- 172.21.15.5:0/2929278772 shutdown_connections 2024-09-11T22:42:21.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:21.621+0000 7f1aab28b640 1 -- 172.21.15.5:0/2929278772 wait complete. 2024-09-11T22:42:22.028 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:21 smithi005 ceph-mon[30943]: pgmap v642: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 13 op/s 2024-09-11T22:42:22.028 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:21 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2929278772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:42:22.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:21 smithi159 ceph-mon[35534]: pgmap v642: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 13 op/s 2024-09-11T22:42:22.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:21 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2929278772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:42:22.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:21 smithi172 ceph-mon[35461]: pgmap v642: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 110 KiB/s wr, 13 op/s 2024-09-11T22:42:22.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:21 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2929278772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:42:22.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-11T22:42:22.556 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:42:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.053+0000 7f581d3c4640 1 -- 172.21.15.5:0/1829976914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581810c170 msgr2=0x7f581810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.053+0000 7f581d3c4640 1 --2- 172.21.15.5:0/1829976914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581810c170 0x7f581810e610 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f5810009ca0 tx=0x7f581002a950 comp rx=0 tx=0).stop 2024-09-11T22:42:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.053+0000 7f581d3c4640 1 -- 172.21.15.5:0/1829976914 shutdown_connections 2024-09-11T22:42:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.053+0000 7f581d3c4640 1 --2- 172.21.15.5:0/1829976914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f581810c170 0x7f581810e610 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.053+0000 7f581d3c4640 1 --2- 172.21.15.5:0/1829976914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5818103660 0x7f581810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.053+0000 7f581d3c4640 1 --2- 172.21.15.5:0/1829976914 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5818102d40 0x7f5818103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.053+0000 7f581d3c4640 1 -- 172.21.15.5:0/1829976914 >> 172.21.15.5:0/1829976914 conn(0x7f58180fc7a0 msgr2=0x7f58180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.053+0000 7f581d3c4640 1 -- 172.21.15.5:0/1829976914 shutdown_connections 2024-09-11T22:42:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.053+0000 7f581d3c4640 1 -- 172.21.15.5:0/1829976914 wait complete. 2024-09-11T22:42:23.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.054+0000 7f581d3c4640 1 Processor -- start 2024-09-11T22:42:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.054+0000 7f581d3c4640 1 -- start start 2024-09-11T22:42:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.054+0000 7f581d3c4640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5818102d40 0x7f5818198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.054+0000 7f581d3c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5818103660 0x7f58181988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.054+0000 7f581d3c4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f581810c170 0x7f581819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.054+0000 7f581d3c4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5818111f50 con 0x7f5818103660 2024-09-11T22:42:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.054+0000 7f581d3c4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5818111dd0 con 0x7f581810c170 2024-09-11T22:42:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.054+0000 7f581d3c4640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f58181120d0 con 0x7f5818102d40 2024-09-11T22:42:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.054+0000 7f58167fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5818103660 0x7f58181988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:23.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f5816ffd640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5818102d40 0x7f5818198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:23.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f58167fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5818103660 0x7f58181988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46676/0 (socket says 172.21.15.5:46676) 2024-09-11T22:42:23.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f58167fc640 1 -- 172.21.15.5:0/4153861097 learned_addr learned my addr 172.21.15.5:0/4153861097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:42:23.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f58177fe640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f581810c170 0x7f581819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:23.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f58167fc640 1 -- 172.21.15.5:0/4153861097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5818102d40 msgr2=0x7f5818198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:23.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f58167fc640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5818102d40 0x7f5818198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:23.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f58167fc640 1 -- 172.21.15.5:0/4153861097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f581810c170 msgr2=0x7f581819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f58167fc640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f581810c170 0x7f581819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f58167fc640 1 -- 172.21.15.5:0/4153861097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581819d320 con 0x7f5818103660 2024-09-11T22:42:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f58177fe640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f581810c170 0x7f581819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T22:42:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f58167fc640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5818103660 0x7f58181988b0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f580c00d750 tx=0x7f580c00db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f57fffff640 1 -- 172.21.15.5:0/4153861097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c017070 con 0x7f5818103660 2024-09-11T22:42:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f581d3c4640 1 -- 172.21.15.5:0/4153861097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581819d610 con 0x7f5818103660 2024-09-11T22:42:23.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f57fffff640 1 -- 172.21.15.5:0/4153861097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580c009820 con 0x7f5818103660 2024-09-11T22:42:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f57fffff640 1 -- 172.21.15.5:0/4153861097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c004c60 con 0x7f5818103660 2024-09-11T22:42:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.055+0000 7f581d3c4640 1 -- 172.21.15.5:0/4153861097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181a4ef0 con 0x7f5818103660 2024-09-11T22:42:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.056+0000 7f581d3c4640 1 -- 172.21.15.5:0/4153861097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5818103120 con 0x7f5818103660 2024-09-11T22:42:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.057+0000 7f57fffff640 1 -- 172.21.15.5:0/4153861097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f580c005220 con 0x7f5818103660 2024-09-11T22:42:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.058+0000 7f57fffff640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58000a2cf0 0x7f58000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.058+0000 7f5816ffd640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58000a2cf0 0x7f58000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.058+0000 7f57fffff640 1 -- 172.21.15.5:0/4153861097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8629+0+0 (secure 0 0 0) 0x7f580c0cede0 con 0x7f5818103660 2024-09-11T22:42:23.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.059+0000 7f5816ffd640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58000a2cf0 0x7f58000a51b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f580800abe0 tx=0x7f580800a2d0 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:23.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.063+0000 7f57fffff640 1 -- 172.21.15.5:0/4153861097 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f580c09b990 con 0x7f5818103660 2024-09-11T22:42:23.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.274+0000 7f581d3c4640 1 -- 172.21.15.5:0/4153861097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58181a50c0 con 0x7f5818103660 2024-09-11T22:42:23.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.275+0000 7f57fffff640 1 -- 172.21.15.5:0/4153861097 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11975 (secure 0 0 0) 0x7f580c09b290 con 0x7f5818103660 2024-09-11T22:42:23.276 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:42:23.276 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":57,"btime":"2024-09-11T22:40:39:632093+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2780062479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2780062479},{"type":"v1","addr":"172.21.15.5:6835","nonce":2780062479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/118173744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":118173744},{"type":"v1","addr":"172.21.15.5:6837","nonce":118173744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-11T22:20:18.900781+0000","modified":"2024-09-11T22:40:38.909431+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24500,"mds_1":24473,"mds_2":24454,"mds_3":24452,"mds_4":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14583":{"gid":14583,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/2458034474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2458034474},{"type":"v1","addr":"172.21.15.5:6839","nonce":2458034474}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14595":{"gid":14595,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2232645952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2232645952},{"type":"v1","addr":"172.21.15.5:6841","nonce":2232645952}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24391":{"gid":24391,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6833/1186865242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1186865242},{"type":"v1","addr":"172.21.15.172:6833","nonce":1186865242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.172:6835/4149321843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":4149321843},{"type":"v1","addr":"172.21.15.172:6835","nonce":4149321843}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":4,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6837/990138139","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6836","nonce":990138139},{"type":"v1","addr":"172.21.15.172:6837","nonce":990138139}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"e","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.159:6835/1262598700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":1262598700},{"type":"v1","addr":"172.21.15.159:6835","nonce":1262598700}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24454":{"gid":24454,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.172:6839/1937249261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6838","nonce":1937249261},{"type":"v1","addr":"172.21.15.172:6839","nonce":1937249261}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.159:6837/2315339117","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":2315339117},{"type":"v1","addr":"172.21.15.159:6837","nonce":2315339117}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24500":{"gid":24500,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.159:6839/994288762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":994288762},{"type":"v1","addr":"172.21.15.159:6839","nonce":994288762}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28601":{"gid":28601,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/4216563934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":4216563934},{"type":"v1","addr":"172.21.15.159:6833","nonce":4216563934}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24500,"qdb_cluster":[24500,24473,24454,24452,24436]},"id":1}]} 2024-09-11T22:42:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 57 2024-09-11T22:42:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.277+0000 7f581d3c4640 1 -- 172.21.15.5:0/4153861097 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58000a2cf0 msgr2=0x7f58000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.277+0000 7f581d3c4640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58000a2cf0 0x7f58000a51b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f580800abe0 tx=0x7f580800a2d0 comp rx=0 tx=0).stop 2024-09-11T22:42:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.277+0000 7f581d3c4640 1 -- 172.21.15.5:0/4153861097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5818103660 msgr2=0x7f58181988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.277+0000 7f581d3c4640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5818103660 0x7f58181988b0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f580c00d750 tx=0x7f580c00db50 comp rx=0 tx=0).stop 2024-09-11T22:42:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.277+0000 7f581d3c4640 1 -- 172.21.15.5:0/4153861097 shutdown_connections 2024-09-11T22:42:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.277+0000 7f581d3c4640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f58000a2cf0 0x7f58000a51b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.277+0000 7f581d3c4640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f581810c170 0x7f581819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.277+0000 7f581d3c4640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5818103660 0x7f58181988b0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.277+0000 7f581d3c4640 1 --2- 172.21.15.5:0/4153861097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5818102d40 0x7f5818198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.277+0000 7f581d3c4640 1 -- 172.21.15.5:0/4153861097 >> 172.21.15.5:0/4153861097 conn(0x7f58180fc7a0 msgr2=0x7f58180fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.278+0000 7f581d3c4640 1 -- 172.21.15.5:0/4153861097 shutdown_connections 2024-09-11T22:42:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:23.278+0000 7f581d3c4640 1 -- 172.21.15.5:0/4153861097 wait complete. 2024-09-11T22:42:23.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:23 smithi005 ceph-mon[30943]: pgmap v643: 97 pgs: 97 active+clean; 525 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 13 op/s 2024-09-11T22:42:23.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:23 smithi005 ceph-mon[30943]: osdmap e121: 12 total, 12 up, 12 in 2024-09-11T22:42:23.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:23 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/4153861097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:42:24.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T22:42:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:23 smithi159 ceph-mon[35534]: pgmap v643: 97 pgs: 97 active+clean; 525 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 13 op/s 2024-09-11T22:42:24.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:23 smithi159 ceph-mon[35534]: osdmap e121: 12 total, 12 up, 12 in 2024-09-11T22:42:24.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:23 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/4153861097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:42:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:23 smithi172 ceph-mon[35461]: pgmap v643: 97 pgs: 97 active+clean; 525 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 97 KiB/s wr, 13 op/s 2024-09-11T22:42:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:23 smithi172 ceph-mon[35461]: osdmap e121: 12 total, 12 up, 12 in 2024-09-11T22:42:24.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:23 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/4153861097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-11T22:42:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:42:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.786+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/90874923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c0100cc0 msgr2=0x7ff9c0106800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.786+0000 7ff9c5fbf640 1 --2- 172.21.15.5:0/90874923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c0100cc0 0x7ff9c0106800 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009900 tx=0x7ff9b402a410 comp rx=0 tx=0).stop 2024-09-11T22:42:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.786+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/90874923 shutdown_connections 2024-09-11T22:42:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.786+0000 7ff9c5fbf640 1 --2- 172.21.15.5:0/90874923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c0106f30 0x7ff9c010eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.786+0000 7ff9c5fbf640 1 --2- 172.21.15.5:0/90874923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c0100cc0 0x7ff9c0106800 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.786+0000 7ff9c5fbf640 1 --2- 172.21.15.5:0/90874923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c0100310 0x7ff9c01006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.786+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/90874923 >> 172.21.15.5:0/90874923 conn(0x7ff9c0077210 msgr2=0x7ff9c00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.786+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/90874923 shutdown_connections 2024-09-11T22:42:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.786+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/90874923 wait complete. 2024-09-11T22:42:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.787+0000 7ff9c5fbf640 1 Processor -- start 2024-09-11T22:42:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.787+0000 7ff9c5fbf640 1 -- start start 2024-09-11T22:42:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.787+0000 7ff9c5fbf640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c0100310 0x7ff9c01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.787+0000 7ff9c5fbf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c0100cc0 0x7ff9c01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.787+0000 7ff9c5fbf640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c0106f30 0x7ff9c019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.787+0000 7ff9c5fbf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0111fd0 con 0x7ff9c0100cc0 2024-09-11T22:42:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.787+0000 7ff9c5fbf640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0111e50 con 0x7ff9c0100310 2024-09-11T22:42:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.787+0000 7ff9c5fbf640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0112150 con 0x7ff9c0106f30 2024-09-11T22:42:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c0100cc0 0x7ff9c01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c0100cc0 0x7ff9c01989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46702/0 (socket says 172.21.15.5:46702) 2024-09-11T22:42:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9beffd640 1 -- 172.21.15.5:0/2175223588 learned_addr learned my addr 172.21.15.5:0/2175223588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:42:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9bf7fe640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c0100310 0x7ff9c01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9beffd640 1 -- 172.21.15.5:0/2175223588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c0106f30 msgr2=0x7ff9c019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9beffd640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c0106f30 0x7ff9c019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9beffd640 1 -- 172.21.15.5:0/2175223588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c0100310 msgr2=0x7ff9c01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:24.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9beffd640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c0100310 0x7ff9c01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:24.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9beffd640 1 -- 172.21.15.5:0/2175223588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c019d460 con 0x7ff9c0100cc0 2024-09-11T22:42:24.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9beffd640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c0100cc0 0x7ff9c01989f0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4006010 tx=0x7ff9b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:24.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9bcff9640 1 -- 172.21.15.5:0/2175223588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b403f030 con 0x7ff9c0100cc0 2024-09-11T22:42:24.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9bcff9640 1 -- 172.21.15.5:0/2175223588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b4003ce0 con 0x7ff9c0100cc0 2024-09-11T22:42:24.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/2175223588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c019d6f0 con 0x7ff9c0100cc0 2024-09-11T22:42:24.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9bcff9640 1 -- 172.21.15.5:0/2175223588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4035070 con 0x7ff9c0100cc0 2024-09-11T22:42:24.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.788+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/2175223588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c01a5030 con 0x7ff9c0100cc0 2024-09-11T22:42:24.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.789+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/2175223588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c01033d0 con 0x7ff9c0100cc0 2024-09-11T22:42:24.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.791+0000 7ff9bcff9640 1 -- 172.21.15.5:0/2175223588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7ff9b4042030 con 0x7ff9c0100cc0 2024-09-11T22:42:24.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.791+0000 7ff9bcff9640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9a80a2cf0 0x7ff9a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:24.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.791+0000 7ff9bcff9640 1 -- 172.21.15.5:0/2175223588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8629+0+0 (secure 0 0 0) 0x7ff9b40efed0 con 0x7ff9c0100cc0 2024-09-11T22:42:24.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.791+0000 7ff9bf7fe640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9a80a2cf0 0x7ff9a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:24.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.792+0000 7ff9bf7fe640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9a80a2cf0 0x7ff9a80a51b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009890 tx=0x7ff9b0008040 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:24.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.794+0000 7ff9bcff9640 1 -- 172.21.15.5:0/2175223588 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9b40bcb40 con 0x7ff9c0100cc0 2024-09-11T22:42:24.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.972+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/2175223588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9c01006f0 con 0x7ff9c0100cc0 2024-09-11T22:42:24.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.974+0000 7ff9bcff9640 1 -- 172.21.15.5:0/2175223588 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+23256 (secure 0 0 0) 0x7ff9c01006f0 con 0x7ff9c0100cc0 2024-09-11T22:42:24.975 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-11T22:42:24.976 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":121,"fsid":"b99b16b2-708a-11ef-bcea-c7b262605968","created":"2024-09-11T22:12:23.446853+0000","modified":"2024-09-11T22:42:22.628409+0000","last_up_change":"2024-09-11T22:19:25.112218+0000","last_in_change":"2024-09-11T22:19:01.581480+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T22:15:32.851002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T22:20:14.556626+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T22:20:17.154834+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":120,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e8ab7175-48c6-4c65-82eb-620196b29548","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6803","nonce":26218432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6805","nonce":26218432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6809","nonce":26218432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":26218432},{"type":"v1","addr":"172.21.15.5:6807","nonce":26218432}]},"public_addr":"172.21.15.5:6803/26218432","cluster_addr":"172.21.15.5:6805/26218432","heartbeat_back_addr":"172.21.15.5:6809/26218432","heartbeat_front_addr":"172.21.15.5:6807/26218432","state":["exists","up"]},{"osd":1,"uuid":"8a379e10-b214-42db-8b31-399b39b6cac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6801","nonce":1131654571}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6803","nonce":1131654571}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6807","nonce":1131654571}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":1131654571},{"type":"v1","addr":"172.21.15.159:6805","nonce":1131654571}]},"public_addr":"172.21.15.159:6801/1131654571","cluster_addr":"172.21.15.159:6803/1131654571","heartbeat_back_addr":"172.21.15.159:6807/1131654571","heartbeat_front_addr":"172.21.15.159:6805/1131654571","state":["exists","up"]},{"osd":2,"uuid":"5d23d4d6-1f97-4d0e-9c99-ae865e04ca24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6801","nonce":34708812}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6803","nonce":34708812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6807","nonce":34708812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":34708812},{"type":"v1","addr":"172.21.15.172:6805","nonce":34708812}]},"public_addr":"172.21.15.172:6801/34708812","cluster_addr":"172.21.15.172:6803/34708812","heartbeat_back_addr":"172.21.15.172:6807/34708812","heartbeat_front_addr":"172.21.15.172:6805/34708812","state":["exists","up"]},{"osd":3,"uuid":"35db3b3e-4a85-4b80-9df0-581f33718744","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6811","nonce":708957835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6813","nonce":708957835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6817","nonce":708957835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":708957835},{"type":"v1","addr":"172.21.15.5:6815","nonce":708957835}]},"public_addr":"172.21.15.5:6811/708957835","cluster_addr":"172.21.15.5:6813/708957835","heartbeat_back_addr":"172.21.15.5:6817/708957835","heartbeat_front_addr":"172.21.15.5:6815/708957835","state":["exists","up"]},{"osd":4,"uuid":"810e9588-d06b-493e-be95-d78591ab283a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6809","nonce":4176432146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6811","nonce":4176432146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6815","nonce":4176432146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":4176432146},{"type":"v1","addr":"172.21.15.159:6813","nonce":4176432146}]},"public_addr":"172.21.15.159:6809/4176432146","cluster_addr":"172.21.15.159:6811/4176432146","heartbeat_back_addr":"172.21.15.159:6815/4176432146","heartbeat_front_addr":"172.21.15.159:6813/4176432146","state":["exists","up"]},{"osd":5,"uuid":"0e0bf89e-0062-4032-8aec-6079003315a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6809","nonce":1387739113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6811","nonce":1387739113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6815","nonce":1387739113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":1387739113},{"type":"v1","addr":"172.21.15.172:6813","nonce":1387739113}]},"public_addr":"172.21.15.172:6809/1387739113","cluster_addr":"172.21.15.172:6811/1387739113","heartbeat_back_addr":"172.21.15.172:6815/1387739113","heartbeat_front_addr":"172.21.15.172:6813/1387739113","state":["exists","up"]},{"osd":6,"uuid":"f11a15a6-a7a8-4a42-ba15-20867f32d7ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6819","nonce":2494841053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6821","nonce":2494841053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6825","nonce":2494841053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":2494841053},{"type":"v1","addr":"172.21.15.5:6823","nonce":2494841053}]},"public_addr":"172.21.15.5:6819/2494841053","cluster_addr":"172.21.15.5:6821/2494841053","heartbeat_back_addr":"172.21.15.5:6825/2494841053","heartbeat_front_addr":"172.21.15.5:6823/2494841053","state":["exists","up"]},{"osd":7,"uuid":"b7d70b5c-cb1b-4e19-b185-595a94380ee6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6817","nonce":3702567808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6819","nonce":3702567808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6823","nonce":3702567808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":3702567808},{"type":"v1","addr":"172.21.15.159:6821","nonce":3702567808}]},"public_addr":"172.21.15.159:6817/3702567808","cluster_addr":"172.21.15.159:6819/3702567808","heartbeat_back_addr":"172.21.15.159:6823/3702567808","heartbeat_front_addr":"172.21.15.159:6821/3702567808","state":["exists","up"]},{"osd":8,"uuid":"12a4fca4-4f15-41b1-8776-6722e1692b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6817","nonce":421514522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6818","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6819","nonce":421514522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6822","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6823","nonce":421514522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":421514522},{"type":"v1","addr":"172.21.15.172:6821","nonce":421514522}]},"public_addr":"172.21.15.172:6817/421514522","cluster_addr":"172.21.15.172:6819/421514522","heartbeat_back_addr":"172.21.15.172:6823/421514522","heartbeat_front_addr":"172.21.15.172:6821/421514522","state":["exists","up"]},{"osd":9,"uuid":"a7758f2e-e665-4964-a5cf-f5298315934b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6827","nonce":26202527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6829","nonce":26202527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6833","nonce":26202527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":26202527},{"type":"v1","addr":"172.21.15.5:6831","nonce":26202527}]},"public_addr":"172.21.15.5:6827/26202527","cluster_addr":"172.21.15.5:6829/26202527","heartbeat_back_addr":"172.21.15.5:6833/26202527","heartbeat_front_addr":"172.21.15.5:6831/26202527","state":["exists","up"]},{"osd":10,"uuid":"2e3cad92-29cf-43a6-84c0-4533de103ce8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6825","nonce":1772373268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6827","nonce":1772373268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6831","nonce":1772373268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":1772373268},{"type":"v1","addr":"172.21.15.159:6829","nonce":1772373268}]},"public_addr":"172.21.15.159:6825/1772373268","cluster_addr":"172.21.15.159:6827/1772373268","heartbeat_back_addr":"172.21.15.159:6831/1772373268","heartbeat_front_addr":"172.21.15.159:6829/1772373268","state":["exists","up"]},{"osd":11,"uuid":"684e017e-bc9b-4fc5-afd8-89a3199c5653","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6825","nonce":2463160464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6826","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6827","nonce":2463160464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6830","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6831","nonce":2463160464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":2463160464},{"type":"v1","addr":"172.21.15.172:6829","nonce":2463160464}]},"public_addr":"172.21.15.172:6825/2463160464","cluster_addr":"172.21.15.172:6827/2463160464","heartbeat_back_addr":"172.21.15.172:6831/2463160464","heartbeat_front_addr":"172.21.15.172:6829/2463160464","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:14:39.425603+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:04.410536+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:29.918736+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:15:56.584014+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:22.809714+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:16:48.785793+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:13.950006+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:17:39.837176+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:05.253077+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:32.167778+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:18:57.673781+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T22:19:23.738772+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2380674497":"2024-09-12T22:22:15.968309+0000","172.21.15.5:6801/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/3381830267":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/3591749770":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/2381319073":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/2378444675":"2024-09-12T22:12:52.569019+0000","172.21.15.5:6800/1194241380":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/1577283141":"2024-09-12T22:22:15.968309+0000","172.21.15.5:6801/3591749770":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/1875833568":"2024-09-12T22:12:52.569019+0000","172.21.15.5:0/981960152":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/497465376":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6800/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:6801/3503484776":"2024-09-12T22:13:16.336721+0000","172.21.15.5:0/1618823752":"2024-09-12T22:22:15.968309+0000","172.21.15.5:0/785116270":"2024-09-12T22:22:15.968309+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T22:42:24.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.976+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/2175223588 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9a80a2cf0 msgr2=0x7ff9a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:24.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.976+0000 7ff9c5fbf640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9a80a2cf0 0x7ff9a80a51b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009890 tx=0x7ff9b0008040 comp rx=0 tx=0).stop 2024-09-11T22:42:24.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.976+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/2175223588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c0100cc0 msgr2=0x7ff9c01989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:24.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.976+0000 7ff9c5fbf640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c0100cc0 0x7ff9c01989f0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4006010 tx=0x7ff9b40042f0 comp rx=0 tx=0).stop 2024-09-11T22:42:24.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.977+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/2175223588 shutdown_connections 2024-09-11T22:42:24.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.977+0000 7ff9c5fbf640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9c0106f30 0x7ff9c019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:24.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.977+0000 7ff9c5fbf640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c0100cc0 0x7ff9c01989f0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:24.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.977+0000 7ff9c5fbf640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7ff9a80a2cf0 0x7ff9a80a51b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:24.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.977+0000 7ff9c5fbf640 1 --2- 172.21.15.5:0/2175223588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9c0100310 0x7ff9c01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:24.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.977+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/2175223588 >> 172.21.15.5:0/2175223588 conn(0x7ff9c0077210 msgr2=0x7ff9c0105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:24.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.977+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/2175223588 shutdown_connections 2024-09-11T22:42:24.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:24.977+0000 7ff9c5fbf640 1 -- 172.21.15.5:0/2175223588 wait complete. 2024-09-11T22:42:25.660 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-11T22:42:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:42:26.038 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:25 smithi005 ceph-mon[30943]: pgmap v645: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-11T22:42:26.039 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:25 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2175223588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:42:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:25 smithi159 ceph-mon[35534]: pgmap v645: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-11T22:42:26.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:25 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2175223588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:42:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:25 smithi172 ceph-mon[35461]: pgmap v645: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-11T22:42:26.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:25 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2175223588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T22:42:26.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.420+0000 7f959fb55640 1 -- 172.21.15.5:0/3360190179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9598102ef0 msgr2=0x7f959810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.420+0000 7f959fb55640 1 --2- 172.21.15.5:0/3360190179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9598102ef0 0x7f959810da40 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f9590009ca0 tx=0x7f959002a950 comp rx=0 tx=0).stop 2024-09-11T22:42:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.420+0000 7f959fb55640 1 -- 172.21.15.5:0/3360190179 shutdown_connections 2024-09-11T22:42:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.420+0000 7f959fb55640 1 --2- 172.21.15.5:0/3360190179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9598102ef0 0x7f959810da40 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.420+0000 7f959fb55640 1 --2- 172.21.15.5:0/3360190179 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598102570 0x7f95981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.420+0000 7f959fb55640 1 --2- 172.21.15.5:0/3360190179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9598100e20 0x7f9598101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.420+0000 7f959fb55640 1 -- 172.21.15.5:0/3360190179 >> 172.21.15.5:0/3360190179 conn(0x7f95980fc7d0 msgr2=0x7f95980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.420+0000 7f959fb55640 1 -- 172.21.15.5:0/3360190179 shutdown_connections 2024-09-11T22:42:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.420+0000 7f959fb55640 1 -- 172.21.15.5:0/3360190179 wait complete. 2024-09-11T22:42:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959fb55640 1 Processor -- start 2024-09-11T22:42:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959fb55640 1 -- start start 2024-09-11T22:42:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959fb55640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9598100e20 0x7f95981992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959fb55640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598102570 0x7f9598199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959fb55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9598102ef0 0x7f95981a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959fb55640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9598110f80 con 0x7f9598102ef0 2024-09-11T22:42:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959fb55640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9598110e00 con 0x7f9598102570 2024-09-11T22:42:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959fb55640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f9598111100 con 0x7f9598100e20 2024-09-11T22:42:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959e0f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9598102ef0 0x7f95981a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959e0f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9598102ef0 0x7f95981a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46726/0 (socket says 172.21.15.5:46726) 2024-09-11T22:42:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959e0f2640 1 -- 172.21.15.5:0/249025928 learned_addr learned my addr 172.21.15.5:0/249025928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:42:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959d0f0640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598102570 0x7f9598199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959d8f1640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9598100e20 0x7f95981992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959e0f2640 1 -- 172.21.15.5:0/249025928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9598100e20 msgr2=0x7f95981992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959e0f2640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9598100e20 0x7f95981992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.421+0000 7f959e0f2640 1 -- 172.21.15.5:0/249025928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598102570 msgr2=0x7f9598199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.422+0000 7f959e0f2640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598102570 0x7f9598199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.422+0000 7f959e0f2640 1 -- 172.21.15.5:0/249025928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95981a4890 con 0x7f9598102ef0 2024-09-11T22:42:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.422+0000 7f959e0f2640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9598102ef0 0x7f95981a4350 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f959002ae70 tx=0x7f959002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.422+0000 7f958effd640 1 -- 172.21.15.5:0/249025928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590034040 con 0x7f9598102ef0 2024-09-11T22:42:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.422+0000 7f958effd640 1 -- 172.21.15.5:0/249025928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9590030920 con 0x7f9598102ef0 2024-09-11T22:42:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.422+0000 7f959fb55640 1 -- 172.21.15.5:0/249025928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95981a4a60 con 0x7f9598102ef0 2024-09-11T22:42:26.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.422+0000 7f958effd640 1 -- 172.21.15.5:0/249025928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590045480 con 0x7f9598102ef0 2024-09-11T22:42:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.422+0000 7f959fb55640 1 -- 172.21.15.5:0/249025928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95981a4f10 con 0x7f9598102ef0 2024-09-11T22:42:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.423+0000 7f959fb55640 1 -- 172.21.15.5:0/249025928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9598101200 con 0x7f9598102ef0 2024-09-11T22:42:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.425+0000 7f958effd640 1 -- 172.21.15.5:0/249025928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f9590037020 con 0x7f9598102ef0 2024-09-11T22:42:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.426+0000 7f958effd640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95840a2d70 0x7f95840a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.426+0000 7f959d8f1640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95840a2d70 0x7f95840a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.426+0000 7f958effd640 1 -- 172.21.15.5:0/249025928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8629+0+0 (secure 0 0 0) 0x7f95900f0350 con 0x7f9598102ef0 2024-09-11T22:42:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.426+0000 7f959d8f1640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95840a2d70 0x7f95840a5230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f959400bbe0 tx=0x7f959400b390 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:26.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.429+0000 7f958effd640 1 -- 172.21.15.5:0/249025928 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95900811b0 con 0x7f9598102ef0 2024-09-11T22:42:26.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.641+0000 7f959fb55640 1 -- 172.21.15.5:0/249025928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f95981a50e0 con 0x7f9598102ef0 2024-09-11T22:42:26.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.643+0000 7f958effd640 1 -- 172.21.15.5:0/249025928 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7f95900bcfb0 con 0x7f9598102ef0 2024-09-11T22:42:26.644 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T22:42:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.647+0000 7f959fb55640 1 -- 172.21.15.5:0/249025928 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95840a2d70 msgr2=0x7f95840a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.647+0000 7f959fb55640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95840a2d70 0x7f95840a5230 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f959400bbe0 tx=0x7f959400b390 comp rx=0 tx=0).stop 2024-09-11T22:42:26.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.647+0000 7f959fb55640 1 -- 172.21.15.5:0/249025928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9598102ef0 msgr2=0x7f95981a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.647+0000 7f959fb55640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9598102ef0 0x7f95981a4350 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f959002ae70 tx=0x7f959002f5d0 comp rx=0 tx=0).stop 2024-09-11T22:42:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.647+0000 7f959fb55640 1 -- 172.21.15.5:0/249025928 shutdown_connections 2024-09-11T22:42:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.647+0000 7f959fb55640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9598102ef0 0x7f95981a4350 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.647+0000 7f959fb55640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f95840a2d70 0x7f95840a5230 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.647+0000 7f959fb55640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9598102570 0x7f9598199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.647+0000 7f959fb55640 1 --2- 172.21.15.5:0/249025928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9598100e20 0x7f95981992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:26.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.647+0000 7f959fb55640 1 -- 172.21.15.5:0/249025928 >> 172.21.15.5:0/249025928 conn(0x7f95980fc7d0 msgr2=0x7f9598106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:26.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.648+0000 7f959fb55640 1 -- 172.21.15.5:0/249025928 shutdown_connections 2024-09-11T22:42:26.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:26.648+0000 7f959fb55640 1 -- 172.21.15.5:0/249025928 wait complete. 2024-09-11T22:42:27.071 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a[30939]: 2024-09-11T22:42:26.641+0000 7fe8e8023640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:42:27.071 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:42:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a[69478]: ignoring --setuser ceph since I am not root 2024-09-11T22:42:27.071 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:42:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a[69478]: ignoring --setgroup ceph since I am not root 2024-09-11T22:42:27.072 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:42:26 smithi005 ceph-mds[69483]: -- 172.21.15.5:0/2918669075 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e4e5d91c20 con 0x55e4e5d6f400 2024-09-11T22:42:27.072 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:42:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a[69478]: starting mds.a at 2024-09-11T22:42:27.072 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:42:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-g[69058]: ignoring --setuser ceph since I am not root 2024-09-11T22:42:27.073 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:42:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-g[69058]: ignoring --setgroup ceph since I am not root 2024-09-11T22:42:27.073 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:42:26 smithi005 ceph-mds[69065]: -- 172.21.15.5:0/3244682324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb49f51860 con 0x55cb49f2f400 2024-09-11T22:42:27.073 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:42:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-g[69058]: starting mds.g at 2024-09-11T22:42:27.079 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: ignoring --setuser ceph since I am not root 2024-09-11T22:42:27.079 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: ignoring --setgroup ceph since I am not root 2024-09-11T22:42:27.079 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-mds[65965]: -- 172.21.15.159:0/3559765552 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5573d3dddc20 con 0x5573d3dbb400 2024-09-11T22:42:27.080 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: starting mds.b at 2024-09-11T22:42:27.080 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e[65346]: ignoring --setuser ceph since I am not root 2024-09-11T22:42:27.080 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e[65346]: ignoring --setgroup ceph since I am not root 2024-09-11T22:42:27.080 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-mds[65350]: -- 172.21.15.159:0/207600021 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cc126cf860 con 0x55cc126ac800 2024-09-11T22:42:27.080 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e[65346]: starting mds.e at 2024-09-11T22:42:27.081 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h[65647]: ignoring --setuser ceph since I am not root 2024-09-11T22:42:27.081 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h[65647]: ignoring --setgroup ceph since I am not root 2024-09-11T22:42:27.081 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-mds[65651]: -- 172.21.15.159:0/820820760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557fc223b860 con 0x557fc2219000 2024-09-11T22:42:27.081 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h[65647]: starting mds.h at 2024-09-11T22:42:27.082 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setuser ceph since I am not root 2024-09-11T22:42:27.082 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: ignoring --setgroup ceph since I am not root 2024-09-11T22:42:27.082 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:42:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: starting mds.k at 2024-09-11T22:42:27.110 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i[66381]: ignoring --setuser ceph since I am not root 2024-09-11T22:42:27.110 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i[66381]: ignoring --setgroup ceph since I am not root 2024-09-11T22:42:27.110 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-mds[66386]: -- 172.21.15.172:0/3567917605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cf8b57860 con 0x560cf8b35000 2024-09-11T22:42:27.110 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i[66381]: starting mds.i at 2024-09-11T22:42:27.111 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l[65473]: ignoring --setuser ceph since I am not root 2024-09-11T22:42:27.111 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l[65473]: ignoring --setgroup ceph since I am not root 2024-09-11T22:42:27.111 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-mds[65477]: -- 172.21.15.172:0/2589882232 <== mon.1 v2:172.21.15.172:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0e6c99860 con 0x55e0e6c76800 2024-09-11T22:42:27.111 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l[65473]: starting mds.l at 2024-09-11T22:42:27.112 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c[66081]: ignoring --setuser ceph since I am not root 2024-09-11T22:42:27.112 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c[66081]: ignoring --setgroup ceph since I am not root 2024-09-11T22:42:27.112 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-mds[66085]: -- 172.21.15.172:0/2852976649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f6fd98c680 con 0x55f6fd960800 2024-09-11T22:42:27.112 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-mds[66085]: -- 172.21.15.172:0/2852976649 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f6fd983860 con 0x55f6fd960800 2024-09-11T22:42:27.112 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c[66081]: starting mds.c at 2024-09-11T22:42:27.112 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-f[65778]: ignoring --setuser ceph since I am not root 2024-09-11T22:42:27.113 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-f[65778]: ignoring --setgroup ceph since I am not root 2024-09-11T22:42:27.113 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-mds[65782]: -- 172.21.15.172:0/1130369566 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e58a6c5860 con 0x55e58a6a3400 2024-09-11T22:42:27.113 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:42:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-f[65778]: starting mds.f at 2024-09-11T22:42:27.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T22:42:27.620 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:42:27.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:27 smithi005 ceph-mon[30943]: pgmap v646: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-11T22:42:27.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:27 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/249025928' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T22:42:27.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:27 smithi005 ceph-mon[30943]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T22:42:27.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:27 smithi005 ceph-mon[30943]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:42:27.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:27 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/249025928' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T22:42:27.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:27 smithi005 ceph-mon[30943]: osdmap e122: 12 total, 12 up, 12 in 2024-09-11T22:42:27.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:27 smithi005 ceph-mon[30943]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T22:42:28.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:27 smithi159 ceph-mon[35534]: pgmap v646: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-11T22:42:28.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:27 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/249025928' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T22:42:28.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:27 smithi159 ceph-mon[35534]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T22:42:28.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:27 smithi159 ceph-mon[35534]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:42:28.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:27 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/249025928' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T22:42:28.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:27 smithi159 ceph-mon[35534]: osdmap e122: 12 total, 12 up, 12 in 2024-09-11T22:42:28.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:27 smithi159 ceph-mon[35534]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T22:42:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:27 smithi172 ceph-mon[35461]: pgmap v646: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-11T22:42:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:27 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/249025928' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-11T22:42:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:27 smithi172 ceph-mon[35461]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-11T22:42:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:27 smithi172 ceph-mon[35461]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T22:42:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:27 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/249025928' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-11T22:42:28.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:27 smithi172 ceph-mon[35461]: osdmap e122: 12 total, 12 up, 12 in 2024-09-11T22:42:28.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:27 smithi172 ceph-mon[35461]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-11T22:42:28.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 -- 172.21.15.5:0/2708680498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8360106a80 msgr2=0x7f8360106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 --2- 172.21.15.5:0/2708680498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8360106a80 0x7f8360106e60 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f8350009930 tx=0x7f835002a420 comp rx=0 tx=0).stop 2024-09-11T22:42:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 -- 172.21.15.5:0/2708680498 shutdown_connections 2024-09-11T22:42:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 --2- 172.21.15.5:0/2708680498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8360101420 0x7f836010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 --2- 172.21.15.5:0/2708680498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8360100a80 0x7f8360100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 --2- 172.21.15.5:0/2708680498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8360106a80 0x7f8360106e60 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 -- 172.21.15.5:0/2708680498 >> 172.21.15.5:0/2708680498 conn(0x7f83600fc7a0 msgr2=0x7f83600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 -- 172.21.15.5:0/2708680498 shutdown_connections 2024-09-11T22:42:28.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 -- 172.21.15.5:0/2708680498 wait complete. 2024-09-11T22:42:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 Processor -- start 2024-09-11T22:42:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.109+0000 7f8366446640 1 -- start start 2024-09-11T22:42:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f8366446640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8360100a80 0x7f83601985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f8366446640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8360101420 0x7f8360198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f8366446640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8360106a80 0x7f836019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f8366446640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8360112060 con 0x7f8360101420 2024-09-11T22:42:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f8366446640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8360111ee0 con 0x7f8360106a80 2024-09-11T22:42:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f8366446640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f83601121e0 con 0x7f8360100a80 2024-09-11T22:42:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f835f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8360101420 0x7f8360198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f835f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8360101420 0x7f8360198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46776/0 (socket says 172.21.15.5:46776) 2024-09-11T22:42:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f835f7fe640 1 -- 172.21.15.5:0/2368748584 learned_addr learned my addr 172.21.15.5:0/2368748584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:42:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f835ffff640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8360100a80 0x7f83601985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f83649e3640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8360106a80 0x7f836019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f835f7fe640 1 -- 172.21.15.5:0/2368748584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8360100a80 msgr2=0x7f83601985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f835f7fe640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8360100a80 0x7f83601985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f835f7fe640 1 -- 172.21.15.5:0/2368748584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8360106a80 msgr2=0x7f836019ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f835f7fe640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8360106a80 0x7f836019ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f835f7fe640 1 -- 172.21.15.5:0/2368748584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836019d590 con 0x7f8360101420 2024-09-11T22:42:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.110+0000 7f835f7fe640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8360101420 0x7f8360198b20 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f835400b4a0 tx=0x7f835400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.111+0000 7f835d7fa640 1 -- 172.21.15.5:0/2368748584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354014070 con 0x7f8360101420 2024-09-11T22:42:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.111+0000 7f835d7fa640 1 -- 172.21.15.5:0/2368748584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8354010af0 con 0x7f8360101420 2024-09-11T22:42:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.111+0000 7f835d7fa640 1 -- 172.21.15.5:0/2368748584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354017d10 con 0x7f8360101420 2024-09-11T22:42:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.111+0000 7f8366446640 1 -- 172.21.15.5:0/2368748584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f836019d880 con 0x7f8360101420 2024-09-11T22:42:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.111+0000 7f8366446640 1 -- 172.21.15.5:0/2368748584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83601a5160 con 0x7f8360101420 2024-09-11T22:42:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.112+0000 7f8366446640 1 -- 172.21.15.5:0/2368748584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8360100f60 con 0x7f8360101420 2024-09-11T22:42:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.113+0000 7f835d7fa640 1 -- 172.21.15.5:0/2368748584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f8354017070 con 0x7f8360101420 2024-09-11T22:42:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.113+0000 7f835d7fa640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83480a2cf0 0x7f83480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.114+0000 7f835ffff640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83480a2cf0 0x7f83480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.114+0000 7f835d7fa640 1 -- 172.21.15.5:0/2368748584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9015+0+0 (secure 0 0 0) 0x7f83540cea80 con 0x7f8360101420 2024-09-11T22:42:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.114+0000 7f835ffff640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83480a2cf0 0x7f83480a51b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8350000c00 tx=0x7f8350031490 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.118+0000 7f835d7fa640 1 -- 172.21.15.5:0/2368748584 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83540d0e20 con 0x7f8360101420 2024-09-11T22:42:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.330+0000 7f8366446640 1 -- 172.21.15.5:0/2368748584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f836002d7c0 con 0x7f8360101420 2024-09-11T22:42:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.645+0000 7f835d7fa640 1 -- 172.21.15.5:0/2368748584 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7f835409b560 con 0x7f8360101420 2024-09-11T22:42:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.647+0000 7f8366446640 1 -- 172.21.15.5:0/2368748584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83480a2cf0 msgr2=0x7f83480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.647+0000 7f8366446640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83480a2cf0 0x7f83480a51b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8350000c00 tx=0x7f8350031490 comp rx=0 tx=0).stop 2024-09-11T22:42:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.647+0000 7f8366446640 1 -- 172.21.15.5:0/2368748584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8360101420 msgr2=0x7f8360198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.647+0000 7f8366446640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8360101420 0x7f8360198b20 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f835400b4a0 tx=0x7f835400b960 comp rx=0 tx=0).stop 2024-09-11T22:42:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.648+0000 7f8366446640 1 -- 172.21.15.5:0/2368748584 shutdown_connections 2024-09-11T22:42:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.648+0000 7f8366446640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f83480a2cf0 0x7f83480a51b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.648+0000 7f8366446640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8360106a80 0x7f836019ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.648+0000 7f8366446640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8360101420 0x7f8360198b20 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.648+0000 7f8366446640 1 --2- 172.21.15.5:0/2368748584 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8360100a80 0x7f83601985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.648+0000 7f8366446640 1 -- 172.21.15.5:0/2368748584 >> 172.21.15.5:0/2368748584 conn(0x7f83600fc7a0 msgr2=0x7f83600fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.648+0000 7f8366446640 1 -- 172.21.15.5:0/2368748584 shutdown_connections 2024-09-11T22:42:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:28.648+0000 7f8366446640 1 -- 172.21.15.5:0/2368748584 wait complete. 2024-09-11T22:42:29.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.5:6838/1040232620,v1:172.21.15.5:6839/1040232620] up:boot 2024-09-11T22:42:29.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.5:6840/3869135503,v1:172.21.15.5:6841/3869135503] up:boot 2024-09-11T22:42:29.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.172:6833/2149035300,v1:172.21.15.172:6838/2149035300] up:boot 2024-09-11T22:42:29.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.172:6834/1178795209,v1:172.21.15.172:6837/1178795209] up:boot 2024-09-11T22:42:29.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.172:6836/1299099427,v1:172.21.15.172:6839/1299099427] up:boot 2024-09-11T22:42:29.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.172:6832/3964987436,v1:172.21.15.172:6835/3964987436] up:boot 2024-09-11T22:42:29.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6832/2623728376,v1:172.21.15.159:6834/2623728376] up:boot 2024-09-11T22:42:29.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6833/502890468,v1:172.21.15.159:6835/502890468] up:boot 2024-09-11T22:42:29.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6836/2819563144,v1:172.21.15.159:6838/2819563144] up:boot 2024-09-11T22:42:29.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: mds.? [v2:172.21.15.159:6837/1269000362,v1:172.21.15.159:6839/1269000362] up:boot 2024-09-11T22:42:29.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T22:42:29.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:42:29.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:42:29.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:42:29.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:42:29.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:42:29.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:42:29.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:42:29.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:42:29.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:42:29.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:42:29.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:28 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2368748584' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:42:29.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.5:6838/1040232620,v1:172.21.15.5:6839/1040232620] up:boot 2024-09-11T22:42:29.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.5:6840/3869135503,v1:172.21.15.5:6841/3869135503] up:boot 2024-09-11T22:42:29.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.172:6833/2149035300,v1:172.21.15.172:6838/2149035300] up:boot 2024-09-11T22:42:29.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.172:6834/1178795209,v1:172.21.15.172:6837/1178795209] up:boot 2024-09-11T22:42:29.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.172:6836/1299099427,v1:172.21.15.172:6839/1299099427] up:boot 2024-09-11T22:42:29.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.172:6832/3964987436,v1:172.21.15.172:6835/3964987436] up:boot 2024-09-11T22:42:29.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6832/2623728376,v1:172.21.15.159:6834/2623728376] up:boot 2024-09-11T22:42:29.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6833/502890468,v1:172.21.15.159:6835/502890468] up:boot 2024-09-11T22:42:29.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6836/2819563144,v1:172.21.15.159:6838/2819563144] up:boot 2024-09-11T22:42:29.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: mds.? [v2:172.21.15.159:6837/1269000362,v1:172.21.15.159:6839/1269000362] up:boot 2024-09-11T22:42:29.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T22:42:29.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:42:29.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:42:29.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:42:29.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:42:29.081 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:42:29.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:42:29.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:42:29.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:42:29.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:42:29.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:42:29.082 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:28 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2368748584' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:42:29.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.5:6838/1040232620,v1:172.21.15.5:6839/1040232620] up:boot 2024-09-11T22:42:29.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.5:6840/3869135503,v1:172.21.15.5:6841/3869135503] up:boot 2024-09-11T22:42:29.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.172:6833/2149035300,v1:172.21.15.172:6838/2149035300] up:boot 2024-09-11T22:42:29.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.172:6834/1178795209,v1:172.21.15.172:6837/1178795209] up:boot 2024-09-11T22:42:29.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.172:6836/1299099427,v1:172.21.15.172:6839/1299099427] up:boot 2024-09-11T22:42:29.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.172:6832/3964987436,v1:172.21.15.172:6835/3964987436] up:boot 2024-09-11T22:42:29.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6832/2623728376,v1:172.21.15.159:6834/2623728376] up:boot 2024-09-11T22:42:29.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6833/502890468,v1:172.21.15.159:6835/502890468] up:boot 2024-09-11T22:42:29.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6836/2819563144,v1:172.21.15.159:6838/2819563144] up:boot 2024-09-11T22:42:29.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: mds.? [v2:172.21.15.159:6837/1269000362,v1:172.21.15.159:6839/1269000362] up:boot 2024-09-11T22:42:29.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-11T22:42:29.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-11T22:42:29.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-11T22:42:29.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-11T22:42:29.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-11T22:42:29.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-11T22:42:29.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-11T22:42:29.112 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-11T22:42:29.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-11T22:42:29.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-11T22:42:29.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='mgr.15201 172.21.15.5:0/404288323' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-11T22:42:29.113 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:28 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2368748584' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T22:42:29.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T22:42:29.661 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:42:29.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:29 smithi005 ceph-mon[30943]: pgmap v648: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-09-11T22:42:29.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:29 smithi005 ceph-mon[30943]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T22:42:29.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:29 smithi005 ceph-mon[30943]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T22:42:29.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:29 smithi005 ceph-mon[30943]: Cluster is now healthy 2024-09-11T22:42:29.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:29 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/2368748584' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:42:29.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:29 smithi005 ceph-mon[30943]: fsmap 12 up:standby 2024-09-11T22:42:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:29 smithi159 ceph-mon[35534]: pgmap v648: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-09-11T22:42:30.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:29 smithi159 ceph-mon[35534]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T22:42:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:29 smithi159 ceph-mon[35534]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T22:42:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:29 smithi159 ceph-mon[35534]: Cluster is now healthy 2024-09-11T22:42:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:29 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/2368748584' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:42:30.080 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:29 smithi159 ceph-mon[35534]: fsmap 12 up:standby 2024-09-11T22:42:30.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:29 smithi172 ceph-mon[35461]: pgmap v648: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-09-11T22:42:30.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:29 smithi172 ceph-mon[35461]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-11T22:42:30.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:29 smithi172 ceph-mon[35461]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-11T22:42:30.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:29 smithi172 ceph-mon[35461]: Cluster is now healthy 2024-09-11T22:42:30.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:29 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/2368748584' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-11T22:42:30.111 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:29 smithi172 ceph-mon[35461]: fsmap 12 up:standby 2024-09-11T22:42:30.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.146+0000 7fcfe0be1640 1 -- 172.21.15.5:0/1505141812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfdc1011a0 msgr2=0x7fcfdc101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:30.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.146+0000 7fcfe0be1640 1 --2- 172.21.15.5:0/1505141812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfdc1011a0 0x7fcfdc101600 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009900 tx=0x7fcfd002a410 comp rx=0 tx=0).stop 2024-09-11T22:42:30.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.146+0000 7fcfe0be1640 1 -- 172.21.15.5:0/1505141812 shutdown_connections 2024-09-11T22:42:30.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.146+0000 7fcfe0be1640 1 --2- 172.21.15.5:0/1505141812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc101b40 0x7fcfdc10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.146+0000 7fcfe0be1640 1 --2- 172.21.15.5:0/1505141812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfdc1011a0 0x7fcfdc101600 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.146+0000 7fcfe0be1640 1 --2- 172.21.15.5:0/1505141812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc106c30 0x7fcfdc107010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.146+0000 7fcfe0be1640 1 -- 172.21.15.5:0/1505141812 >> 172.21.15.5:0/1505141812 conn(0x7fcfdc077210 msgr2=0x7fcfdc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.146+0000 7fcfe0be1640 1 -- 172.21.15.5:0/1505141812 shutdown_connections 2024-09-11T22:42:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.146+0000 7fcfe0be1640 1 -- 172.21.15.5:0/1505141812 wait complete. 2024-09-11T22:42:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfe0be1640 1 Processor -- start 2024-09-11T22:42:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfe0be1640 1 -- start start 2024-09-11T22:42:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfe0be1640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc1011a0 0x7fcfdc198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfe0be1640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc101b40 0x7fcfdc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfe0be1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfdc106c30 0x7fcfdc19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfe0be1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc110fa0 con 0x7fcfdc106c30 2024-09-11T22:42:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfe0be1640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc110e20 con 0x7fcfdc1011a0 2024-09-11T22:42:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfe0be1640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc111120 con 0x7fcfdc101b40 2024-09-11T22:42:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfdad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfdc106c30 0x7fcfdc19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfdad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfdc106c30 0x7fcfdc19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46796/0 (socket says 172.21.15.5:46796) 2024-09-11T22:42:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfdad9d640 1 -- 172.21.15.5:0/583780971 learned_addr learned my addr 172.21.15.5:0/583780971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:42:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfda59c640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc1011a0 0x7fcfdc198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.147+0000 7fcfd9d9b640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc101b40 0x7fcfdc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfdad9d640 1 -- 172.21.15.5:0/583780971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc101b40 msgr2=0x7fcfdc198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfdad9d640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc101b40 0x7fcfdc198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfdad9d640 1 -- 172.21.15.5:0/583780971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc1011a0 msgr2=0x7fcfdc198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfdad9d640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc1011a0 0x7fcfdc198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfdad9d640 1 -- 172.21.15.5:0/583780971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc19d4d0 con 0x7fcfdc106c30 2024-09-11T22:42:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfdad9d640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfdc106c30 0x7fcfdc19cdf0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400eaa0 tx=0x7fcfd400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfcb7fe640 1 -- 172.21.15.5:0/583780971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd400cdb0 con 0x7fcfdc106c30 2024-09-11T22:42:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfcb7fe640 1 -- 172.21.15.5:0/583780971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd4005020 con 0x7fcfdc106c30 2024-09-11T22:42:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfe0be1640 1 -- 172.21.15.5:0/583780971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc19d7c0 con 0x7fcfdc106c30 2024-09-11T22:42:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfcb7fe640 1 -- 172.21.15.5:0/583780971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd40116b0 con 0x7fcfdc106c30 2024-09-11T22:42:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.148+0000 7fcfe0be1640 1 -- 172.21.15.5:0/583780971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1a50a0 con 0x7fcfdc106c30 2024-09-11T22:42:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.149+0000 7fcfe0be1640 1 -- 172.21.15.5:0/583780971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc101600 con 0x7fcfdc106c30 2024-09-11T22:42:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.150+0000 7fcfcb7fe640 1 -- 172.21.15.5:0/583780971 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7fcfd4005560 con 0x7fcfdc106c30 2024-09-11T22:42:30.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.151+0000 7fcfcb7fe640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcfc40a2cf0 0x7fcfc40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:30.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.151+0000 7fcfda59c640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcfc40a2cf0 0x7fcfc40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:30.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.151+0000 7fcfcb7fe640 1 -- 172.21.15.5:0/583780971 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9015+0+0 (secure 0 0 0) 0x7fcfd40cf800 con 0x7fcfdc106c30 2024-09-11T22:42:30.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.151+0000 7fcfda59c640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcfc40a2cf0 0x7fcfc40a51b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc005d00 tx=0x7fcfcc00b410 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:30.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.155+0000 7fcfcb7fe640 1 -- 172.21.15.5:0/583780971 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcfd4016030 con 0x7fcfdc106c30 2024-09-11T22:42:30.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.332+0000 7fcfe0be1640 1 -- 172.21.15.5:0/583780971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcfdc10a0b0 con 0x7fcfdc106c30 2024-09-11T22:42:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.659+0000 7fcfcb7fe640 1 -- 172.21.15.5:0/583780971 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) ==== 181+0+0 (secure 0 0 0) 0x7fcfd409bbe0 con 0x7fcfdc106c30 2024-09-11T22:42:30.660 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-09-11T22:42:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.663+0000 7fcfe0be1640 1 -- 172.21.15.5:0/583780971 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcfc40a2cf0 msgr2=0x7fcfc40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.663+0000 7fcfe0be1640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcfc40a2cf0 0x7fcfc40a51b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc005d00 tx=0x7fcfcc00b410 comp rx=0 tx=0).stop 2024-09-11T22:42:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.663+0000 7fcfe0be1640 1 -- 172.21.15.5:0/583780971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfdc106c30 msgr2=0x7fcfdc19cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:30.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.663+0000 7fcfe0be1640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfdc106c30 0x7fcfdc19cdf0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400eaa0 tx=0x7fcfd400ef60 comp rx=0 tx=0).stop 2024-09-11T22:42:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.663+0000 7fcfe0be1640 1 -- 172.21.15.5:0/583780971 shutdown_connections 2024-09-11T22:42:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.663+0000 7fcfe0be1640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfdc106c30 0x7fcfdc19cdf0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.663+0000 7fcfe0be1640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc101b40 0x7fcfdc198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.663+0000 7fcfe0be1640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7fcfc40a2cf0 0x7fcfc40a51b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.663+0000 7fcfe0be1640 1 --2- 172.21.15.5:0/583780971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfdc1011a0 0x7fcfdc198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:30.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.663+0000 7fcfe0be1640 1 -- 172.21.15.5:0/583780971 >> 172.21.15.5:0/583780971 conn(0x7fcfdc077210 msgr2=0x7fcfdc100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.664+0000 7fcfe0be1640 1 -- 172.21.15.5:0/583780971 shutdown_connections 2024-09-11T22:42:30.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:30.664+0000 7fcfe0be1640 1 -- 172.21.15.5:0/583780971 wait complete. 2024-09-11T22:42:31.047 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:30 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/583780971' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:42:31.079 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:30 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/583780971' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:42:31.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:30 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/583780971' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:42:31.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid b99b16b2-708a-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T22:42:31.781 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T22:42:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:31 smithi005 ceph-mon[30943]: pgmap v649: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-11T22:42:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:31 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/583780971' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:42:31.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:31 smithi005 ceph-mon[30943]: osdmap e123: 12 total, 12 up, 12 in 2024-09-11T22:42:31.980 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:31 smithi159 ceph-mon[35534]: pgmap v649: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-11T22:42:31.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:31 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/583780971' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:42:31.981 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:31 smithi159 ceph-mon[35534]: osdmap e123: 12 total, 12 up, 12 in 2024-09-11T22:42:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:31 smithi172 ceph-mon[35461]: pgmap v649: 97 pgs: 97 active+clean; 525 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-11T22:42:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:31 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/583780971' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:42:32.110 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:31 smithi172 ceph-mon[35461]: osdmap e123: 12 total, 12 up, 12 in 2024-09-11T22:42:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.307+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3207224822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e81019e0 msgr2=0x7f94e8101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.307+0000 7f94ed1a2640 1 --2- 172.21.15.5:0/3207224822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e81019e0 0x7f94e8101e40 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009930 tx=0x7f94d002a420 comp rx=0 tx=0).stop 2024-09-11T22:42:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.307+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3207224822 shutdown_connections 2024-09-11T22:42:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.307+0000 7f94ed1a2640 1 --2- 172.21.15.5:0/3207224822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94e8102380 0x7f94e810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.307+0000 7f94ed1a2640 1 --2- 172.21.15.5:0/3207224822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e81019e0 0x7f94e8101e40 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.307+0000 7f94ed1a2640 1 --2- 172.21.15.5:0/3207224822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94e8107af0 0x7f94e8107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.307+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3207224822 >> 172.21.15.5:0/3207224822 conn(0x7f94e80fd700 msgr2=0x7f94e80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3207224822 shutdown_connections 2024-09-11T22:42:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3207224822 wait complete. 2024-09-11T22:42:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ed1a2640 1 Processor -- start 2024-09-11T22:42:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ed1a2640 1 -- start start 2024-09-11T22:42:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ed1a2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94e81019e0 0x7f94e8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ed1a2640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94e8102380 0x7f94e81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ed1a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e8107af0 0x7f94e819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ed1a2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94e8111ef0 con 0x7f94e8107af0 2024-09-11T22:42:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ed1a2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94e8111d70 con 0x7f94e81019e0 2024-09-11T22:42:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ed1a2640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 -- 0x7f94e8112070 con 0x7f94e8102380 2024-09-11T22:42:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.308+0000 7f94ec9a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e8107af0 0x7f94e819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94ec9a1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e8107af0 0x7f94e819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34352/0 (socket says 172.21.15.5:34352) 2024-09-11T22:42:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94ec9a1640 1 -- 172.21.15.5:0/3152521993 learned_addr learned my addr 172.21.15.5:0/3152521993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-11T22:42:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94dffff640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94e8102380 0x7f94e81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94e7fff640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94e81019e0 0x7f94e8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94ec9a1640 1 -- 172.21.15.5:0/3152521993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94e8102380 msgr2=0x7f94e81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94ec9a1640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94e8102380 0x7f94e81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94ec9a1640 1 -- 172.21.15.5:0/3152521993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94e81019e0 msgr2=0x7f94e8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94ec9a1640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94e81019e0 0x7f94e8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94ec9a1640 1 -- 172.21.15.5:0/3152521993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e819d340 con 0x7f94e8107af0 2024-09-11T22:42:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94ec9a1640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e8107af0 0x7f94e819cc60 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f94e000eaa0 tx=0x7f94e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94e5ffb640 1 -- 172.21.15.5:0/3152521993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e000cdb0 con 0x7f94e8107af0 2024-09-11T22:42:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3152521993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e819d630 con 0x7f94e8107af0 2024-09-11T22:42:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94e5ffb640 1 -- 172.21.15.5:0/3152521993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e0005020 con 0x7f94e8107af0 2024-09-11T22:42:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94e5ffb640 1 -- 172.21.15.5:0/3152521993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00116b0 con 0x7f94e8107af0 2024-09-11T22:42:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.309+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3152521993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e81a4f10 con 0x7f94e8107af0 2024-09-11T22:42:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.310+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3152521993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e8101e40 con 0x7f94e8107af0 2024-09-11T22:42:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.311+0000 7f94e5ffb640 1 -- 172.21.15.5:0/3152521993 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 137963+0+0 (secure 0 0 0) 0x7f94e0005560 con 0x7f94e8107af0 2024-09-11T22:42:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.312+0000 7f94e5ffb640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94d40a2cf0 0x7f94d40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T22:42:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.312+0000 7f94e7fff640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94d40a2cf0 0x7f94d40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T22:42:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.313+0000 7f94e5ffb640 1 -- 172.21.15.5:0/3152521993 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 8432+0+0 (secure 0 0 0) 0x7f94e00cfa20 con 0x7f94e8107af0 2024-09-11T22:42:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.313+0000 7f94e7fff640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94d40a2cf0 0x7f94d40a51b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f94d8005fd0 tx=0x7f94d8005850 comp rx=0 tx=0).ready entity=mgr.15201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T22:42:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.316+0000 7f94e5ffb640 1 -- 172.21.15.5:0/3152521993 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f94e0016030 con 0x7f94e8107af0 2024-09-11T22:42:32.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.493+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3152521993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f94e81996f0 con 0x7f94e8107af0 2024-09-11T22:42:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.665+0000 7f94e5ffb640 1 -- 172.21.15.5:0/3152521993 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) ==== 169+0+0 (secure 0 0 0) 0x7f94e81996f0 con 0x7f94e8107af0 2024-09-11T22:42:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-09-11T22:42:32.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.668+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3152521993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94d40a2cf0 msgr2=0x7f94d40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.668+0000 7f94ed1a2640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94d40a2cf0 0x7f94d40a51b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f94d8005fd0 tx=0x7f94d8005850 comp rx=0 tx=0).stop 2024-09-11T22:42:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.668+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3152521993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e8107af0 msgr2=0x7f94e819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T22:42:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.668+0000 7f94ed1a2640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e8107af0 0x7f94e819cc60 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f94e000eaa0 tx=0x7f94e000ef60 comp rx=0 tx=0).stop 2024-09-11T22:42:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.669+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3152521993 shutdown_connections 2024-09-11T22:42:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.669+0000 7f94ed1a2640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e8107af0 0x7f94e819cc60 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.669+0000 7f94ed1a2640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94e8102380 0x7f94e81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.669+0000 7f94ed1a2640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.5:6800/233962001,v1:172.21.15.5:6801/233962001] conn(0x7f94d40a2cf0 0x7f94d40a51b0 secure :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f94d8005fd0 tx=0x7f94d8005850 comp rx=0 tx=0).stop 2024-09-11T22:42:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.669+0000 7f94ed1a2640 1 --2- 172.21.15.5:0/3152521993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f94e81019e0 0x7f94e8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T22:42:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.669+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3152521993 >> 172.21.15.5:0/3152521993 conn(0x7f94e80fd700 msgr2=0x7f94e80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T22:42:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.670+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3152521993 shutdown_connections 2024-09-11T22:42:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-11T22:42:32.671+0000 7f94ed1a2640 1 -- 172.21.15.5:0/3152521993 wait complete. 2024-09-11T22:42:32.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:32 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3152521993' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:42:32.992 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:32 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3152521993' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:42:33.010 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:32 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3152521993' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-11T22:42:33.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:42:33.525 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:42:33.560 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:42:33.589 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T22:42:33.589 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:42:33.606 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:42:33.621 DEBUG:teuthology.orchestra.run.smithi172:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T22:42:33.644 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T22:42:33.644 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T22:42:33.644 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.a 2024-09-11T22:42:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:33 smithi005 ceph-mon[30943]: pgmap v651: 65 pgs: 65 active+clean; 319 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:42:33.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:33 smithi005 ceph-mon[30943]: from='client.? 172.21.15.5:0/3152521993' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:42:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:33 smithi005 ceph-mon[30943]: osdmap e124: 12 total, 12 up, 12 in 2024-09-11T22:42:33.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:33 smithi005 systemd[1]: Stopping Ceph mon.a for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:42:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:33 smithi159 ceph-mon[35534]: pgmap v651: 65 pgs: 65 active+clean; 319 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:42:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:33 smithi159 ceph-mon[35534]: from='client.? 172.21.15.5:0/3152521993' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:42:33.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:33 smithi159 ceph-mon[35534]: osdmap e124: 12 total, 12 up, 12 in 2024-09-11T22:42:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:33 smithi172 ceph-mon[35461]: pgmap v651: 65 pgs: 65 active+clean; 319 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-11T22:42:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:33 smithi172 ceph-mon[35461]: from='client.? 172.21.15.5:0/3152521993' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-11T22:42:33.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:33 smithi172 ceph-mon[35461]: osdmap e124: 12 total, 12 up, 12 in 2024-09-11T22:42:34.320 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:33 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a[30939]: 2024-09-11T22:42:33.963+0000 7fe8ed82e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:42:34.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:33 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a[30939]: 2024-09-11T22:42:33.963+0000 7fe8ed82e640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T22:42:34.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:34 smithi005 podman[131735]: 2024-09-11 22:42:34.035270526 +0000 UTC m=+0.193052424 container died 32a280f5e920c55b3b7da4dd51dedf00d06635c2600b39019fbde98ad374d00a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T22:42:34.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:34 smithi005 podman[131735]: 2024-09-11 22:42:34.464528929 +0000 UTC m=+0.622310826 container cleanup 32a280f5e920c55b3b7da4dd51dedf00d06635c2600b39019fbde98ad374d00a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-11T22:42:34.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:34 smithi005 bash[131735]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a 2024-09-11T22:42:35.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 11 22:42:34 smithi005 podman[131781]: 2024-09-11 22:42:34.724585496 +0000 UTC m=+0.692550882 container remove 32a280f5e920c55b3b7da4dd51dedf00d06635c2600b39019fbde98ad374d00a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-11T22:42:35.060 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T22:42:35.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:42:35.111 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T22:42:35.112 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-11T22:42:35.112 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.b 2024-09-11T22:42:35.515 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:35 smithi159 systemd[1]: Stopping Ceph mon.b for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:42:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-b[35530]: 2024-09-11T22:42:35.514+0000 7f41b0bff640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:42:35.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-b[35530]: 2024-09-11T22:42:35.514+0000 7f41b0bff640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T22:42:35.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:35 smithi159 podman[73805]: 2024-09-11 22:42:35.647110237 +0000 UTC m=+0.275075256 container died 991186ed1dcb4d3161fa497c776d3ca8b5a65ce3519d7ddad429258934ad65fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:42:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:36 smithi159 podman[73805]: 2024-09-11 22:42:36.03129246 +0000 UTC m=+0.659257480 container cleanup 991186ed1dcb4d3161fa497c776d3ca8b5a65ce3519d7ddad429258934ad65fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:42:36.329 INFO:journalctl@ceph.mon.b.smithi159.stdout:Sep 11 22:42:36 smithi159 bash[73805]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-b 2024-09-11T22:42:36.521 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T22:42:36.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:42:36.565 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-11T22:42:36.565 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-11T22:42:36.566 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.c 2024-09-11T22:42:36.860 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:36 smithi172 systemd[1]: Stopping Ceph mon.c for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:42:37.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:36 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-c[35457]: 2024-09-11T22:42:36.953+0000 7f31a5d77640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:42:37.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:36 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-c[35457]: 2024-09-11T22:42:36.953+0000 7f31a5d77640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T22:42:37.360 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:37 smithi172 podman[74021]: 2024-09-11 22:42:37.169688757 +0000 UTC m=+0.406743561 container died 548b0f3e09d5f7efd9cc3d8690917aaff30f629f1ca1bd19b0c0bfef99501913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-11T22:42:37.814 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:37 smithi172 podman[74021]: 2024-09-11 22:42:37.595881571 +0000 UTC m=+0.832936374 container cleanup 548b0f3e09d5f7efd9cc3d8690917aaff30f629f1ca1bd19b0c0bfef99501913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-c, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:42:37.815 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:37 smithi172 bash[74021]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-c 2024-09-11T22:42:38.067 INFO:journalctl@ceph.mon.c.smithi172.stdout:Sep 11 22:42:37 smithi172 podman[74033]: 2024-09-11 22:42:37.814679126 +0000 UTC m=+0.641277731 container remove 548b0f3e09d5f7efd9cc3d8690917aaff30f629f1ca1bd19b0c0bfef99501913 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:42:38.072 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T22:42:38.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:42:38.117 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-11T22:42:38.117 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T22:42:38.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.x 2024-09-11T22:42:38.450 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:42:38 smithi005 systemd[1]: Stopping Ceph mgr.x for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:42:38.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:42:38 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x[31169]: 2024-09-11T22:42:38.448+0000 7f6e837f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:42:38.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:42:38 smithi005 podman[132203]: 2024-09-11 22:42:38.480000807 +0000 UTC m=+0.158188893 container died 894bca1446d0ed668ea805ec0fdbde6cd05ce713329f0bd6a02014d4acfa0890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:42:39.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:42:39 smithi005 podman[132203]: 2024-09-11 22:42:39.03371609 +0000 UTC m=+0.711904165 container cleanup 894bca1446d0ed668ea805ec0fdbde6cd05ce713329f0bd6a02014d4acfa0890 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:42:39.070 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 11 22:42:39 smithi005 bash[132203]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-x 2024-09-11T22:42:39.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T22:42:39.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:42:39.861 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T22:42:39.861 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T22:42:39.861 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.y 2024-09-11T22:42:40.207 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:42:39 smithi159 systemd[1]: Stopping Ceph mgr.y for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:42:40.514 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:42:40 smithi159 podman[73986]: 2024-09-11 22:42:40.207060358 +0000 UTC m=+0.141393168 container died 51f3b0a7d5903c045a526fcacb1f47a216dadb6bb7a3405c19288e04f36d5e6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-11T22:42:40.829 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:42:40 smithi159 podman[73986]: 2024-09-11 22:42:40.587356243 +0000 UTC m=+0.521689109 container cleanup 51f3b0a7d5903c045a526fcacb1f47a216dadb6bb7a3405c19288e04f36d5e6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-11T22:42:40.829 INFO:journalctl@ceph.mgr.y.smithi159.stdout:Sep 11 22:42:40 smithi159 bash[73986]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-y 2024-09-11T22:42:41.059 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T22:42:41.095 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:42:41.095 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T22:42:41.095 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-11T22:42:41.096 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.z 2024-09-11T22:42:41.441 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:42:41 smithi172 systemd[1]: Stopping Ceph mgr.z for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:42:41.735 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:42:41 smithi172 podman[74143]: 2024-09-11 22:42:41.440582756 +0000 UTC m=+0.153017884 container died 8053c59d603345bde68ef28c84c7d2eac27f0698a9cd23fb14dbc36810baac40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T22:42:42.061 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:42:41 smithi172 podman[74143]: 2024-09-11 22:42:41.834679658 +0000 UTC m=+0.547114769 container cleanup 8053c59d603345bde68ef28c84c7d2eac27f0698a9cd23fb14dbc36810baac40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T22:42:42.061 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:42:41 smithi172 bash[74143]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z 2024-09-11T22:42:42.309 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-11T22:42:42.330 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:42:42 smithi172 podman[74155]: 2024-09-11 22:42:42.061212905 +0000 UTC m=+0.616332511 container remove 8053c59d603345bde68ef28c84c7d2eac27f0698a9cd23fb14dbc36810baac40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mgr-z, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T22:42:42.330 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:42:42 smithi172 systemd[1]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-11T22:42:42.331 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:42:42 smithi172 systemd[1]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-11T22:42:42.331 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:42:42 smithi172 systemd[1]: Stopped Ceph mgr.z for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:42:42.331 INFO:journalctl@ceph.mgr.z.smithi172.stdout:Sep 11 22:42:42 smithi172 systemd[1]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mgr.z.service: Consumed 11.783s CPU time. 2024-09-11T22:42:42.344 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:42:42.344 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-11T22:42:42.344 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T22:42:42.344 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.0 2024-09-11T22:42:42.657 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:42 smithi005 systemd[1]: Stopping Ceph osd.0 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:42:43.070 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0[41953]: 2024-09-11T22:42:42.655+0000 7fd3d7587640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:42:43.071 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0[41953]: 2024-09-11T22:42:42.655+0000 7fd3d7587640 -1 osd.0 124 *** Got signal Terminated *** 2024-09-11T22:42:43.071 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0[41953]: 2024-09-11T22:42:42.655+0000 7fd3d7587640 -1 osd.0 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:42:48.492 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:48 smithi005 podman[132321]: 2024-09-11 22:42:48.196377208 +0000 UTC m=+5.652450858 container died 81f7b02cb496ea3b5f0d9dc3d80adf03638bf81ed9597bb06d7fa46bc4ff278d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T22:42:48.820 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:48 smithi005 podman[132321]: 2024-09-11 22:42:48.690443143 +0000 UTC m=+6.146516792 container cleanup 81f7b02cb496ea3b5f0d9dc3d80adf03638bf81ed9597bb06d7fa46bc4ff278d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T22:42:48.820 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:48 smithi005 bash[132321]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0 2024-09-11T22:42:49.320 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:48 smithi005 podman[132337]: 2024-09-11 22:42:48.956022288 +0000 UTC m=+0.760271171 container remove 81f7b02cb496ea3b5f0d9dc3d80adf03638bf81ed9597bb06d7fa46bc4ff278d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-11T22:42:49.604 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:49 smithi005 podman[132431]: 2024-09-11 22:42:49.224257388 +0000 UTC m=+0.016178256 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:42:49.604 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:49 smithi005 podman[132431]: 2024-09-11 22:42:49.34325858 +0000 UTC m=+0.135179448 container create c5b05d886715b59d9c820d347f2b8cc9e9622a684591c048d25165cfecb1172e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T22:42:50.070 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:49 smithi005 podman[132431]: 2024-09-11 22:42:49.604250151 +0000 UTC m=+0.396171019 container init c5b05d886715b59d9c820d347f2b8cc9e9622a684591c048d25165cfecb1172e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-11T22:42:50.070 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:49 smithi005 podman[132431]: 2024-09-11 22:42:49.608085381 +0000 UTC m=+0.400006251 container start c5b05d886715b59d9c820d347f2b8cc9e9622a684591c048d25165cfecb1172e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T22:42:50.070 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:49 smithi005 podman[132431]: 2024-09-11 22:42:49.713018058 +0000 UTC m=+0.504938935 container attach c5b05d886715b59d9c820d347f2b8cc9e9622a684591c048d25165cfecb1172e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-11T22:42:50.071 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 11 22:42:49 smithi005 podman[132431]: 2024-09-11 22:42:49.799365592 +0000 UTC m=+0.591286467 container died c5b05d886715b59d9c820d347f2b8cc9e9622a684591c048d25165cfecb1172e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-09-11T22:42:50.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T22:42:50.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:42:50.567 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T22:42:50.568 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T22:42:50.568 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.1 2024-09-11T22:42:50.884 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:50 smithi159 systemd[1]: Stopping Ceph osd.1 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:42:51.329 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:50 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1[40011]: 2024-09-11T22:42:50.882+0000 7f8c9f45f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:42:51.329 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:50 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1[40011]: 2024-09-11T22:42:50.882+0000 7f8c9f45f640 -1 osd.1 124 *** Got signal Terminated *** 2024-09-11T22:42:51.330 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:50 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1[40011]: 2024-09-11T22:42:50.882+0000 7f8c9f45f640 -1 osd.1 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:42:56.751 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:56 smithi159 podman[74114]: 2024-09-11 22:42:56.451594846 +0000 UTC m=+5.688826179 container died 8a4b9ef873509b4e60b1fa4c96cbfb7436480e2c33ff0e786fdf0e7ec735de88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T22:42:57.079 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:56 smithi159 podman[74114]: 2024-09-11 22:42:56.834238039 +0000 UTC m=+6.071469420 container cleanup 8a4b9ef873509b4e60b1fa4c96cbfb7436480e2c33ff0e786fdf0e7ec735de88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:42:57.079 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:56 smithi159 bash[74114]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1 2024-09-11T22:42:57.419 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:57 smithi159 podman[74130]: 2024-09-11 22:42:57.110725714 +0000 UTC m=+0.661025522 container remove 8a4b9ef873509b4e60b1fa4c96cbfb7436480e2c33ff0e786fdf0e7ec735de88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T22:42:57.706 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:57 smithi159 podman[74218]: 2024-09-11 22:42:57.321420889 +0000 UTC m=+0.015745425 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:42:57.706 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:57 smithi159 podman[74218]: 2024-09-11 22:42:57.453304155 +0000 UTC m=+0.147628691 container create 31f323310d22371dbb073844cea9cc6a74423358746cb06c50b20e467daf86d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T22:42:58.079 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:57 smithi159 podman[74218]: 2024-09-11 22:42:57.706323 +0000 UTC m=+0.400647537 container init 31f323310d22371dbb073844cea9cc6a74423358746cb06c50b20e467daf86d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-11T22:42:58.080 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:57 smithi159 podman[74218]: 2024-09-11 22:42:57.709973063 +0000 UTC m=+0.404297600 container start 31f323310d22371dbb073844cea9cc6a74423358746cb06c50b20e467daf86d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-11T22:42:58.080 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:57 smithi159 podman[74218]: 2024-09-11 22:42:57.814600747 +0000 UTC m=+0.508925283 container attach 31f323310d22371dbb073844cea9cc6a74423358746cb06c50b20e467daf86d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T22:42:58.372 INFO:journalctl@ceph.osd.1.smithi159.stdout:Sep 11 22:42:58 smithi159 podman[74218]: 2024-09-11 22:42:58.095755782 +0000 UTC m=+0.790080322 container died 31f323310d22371dbb073844cea9cc6a74423358746cb06c50b20e467daf86d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:42:58.806 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T22:42:58.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:42:58.844 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T22:42:58.845 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T22:42:58.845 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.2 2024-09-11T22:42:59.306 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:42:58 smithi172 systemd[1]: Stopping Ceph osd.2 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:42:59.610 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:42:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2[40430]: 2024-09-11T22:42:59.305+0000 7ff2787e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:42:59.610 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:42:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2[40430]: 2024-09-11T22:42:59.305+0000 7ff2787e3640 -1 osd.2 124 *** Got signal Terminated *** 2024-09-11T22:42:59.611 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:42:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2[40430]: 2024-09-11T22:42:59.305+0000 7ff2787e3640 -1 osd.2 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:43:05.073 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:43:04 smithi172 podman[74269]: 2024-09-11 22:43:04.796472612 +0000 UTC m=+5.752074001 container died ea23f0bb2ec9c1128dd76ee011e37c6971d4e170a1f2d13221333058c2013cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T22:43:05.360 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:43:05 smithi172 podman[74269]: 2024-09-11 22:43:05.197150166 +0000 UTC m=+6.152751561 container cleanup ea23f0bb2ec9c1128dd76ee011e37c6971d4e170a1f2d13221333058c2013cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T22:43:05.360 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:43:05 smithi172 bash[74269]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2 2024-09-11T22:43:05.810 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:43:05 smithi172 podman[74284]: 2024-09-11 22:43:05.457186251 +0000 UTC m=+0.661212487 container remove ea23f0bb2ec9c1128dd76ee011e37c6971d4e170a1f2d13221333058c2013cf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:43:06.084 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:43:05 smithi172 podman[74373]: 2024-09-11 22:43:05.712360878 +0000 UTC m=+0.015610188 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:43:06.084 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:43:05 smithi172 podman[74373]: 2024-09-11 22:43:05.82474448 +0000 UTC m=+0.127993784 container create 481fedafb63f3b4c97729533efa8d2c061bc895e44134e4b0e11514513372d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:43:06.360 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:43:06 smithi172 podman[74373]: 2024-09-11 22:43:06.077781826 +0000 UTC m=+0.381031130 container init 481fedafb63f3b4c97729533efa8d2c061bc895e44134e4b0e11514513372d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-11T22:43:06.361 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:43:06 smithi172 podman[74373]: 2024-09-11 22:43:06.081065902 +0000 UTC m=+0.384315206 container start 481fedafb63f3b4c97729533efa8d2c061bc895e44134e4b0e11514513372d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T22:43:06.361 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:43:06 smithi172 podman[74373]: 2024-09-11 22:43:06.177807417 +0000 UTC m=+0.481056741 container attach 481fedafb63f3b4c97729533efa8d2c061bc895e44134e4b0e11514513372d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:43:06.361 INFO:journalctl@ceph.osd.2.smithi172.stdout:Sep 11 22:43:06 smithi172 podman[74373]: 2024-09-11 22:43:06.267291747 +0000 UTC m=+0.570541053 container died 481fedafb63f3b4c97729533efa8d2c061bc895e44134e4b0e11514513372d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T22:43:06.999 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T22:43:07.038 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:43:07.038 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T22:43:07.038 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T22:43:07.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.3 2024-09-11T22:43:07.320 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:07 smithi005 systemd[1]: Stopping Ceph osd.3 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:43:07.820 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:07 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3[48410]: 2024-09-11T22:43:07.386+0000 7fbad38ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:43:07.821 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:07 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3[48410]: 2024-09-11T22:43:07.386+0000 7fbad38ef640 -1 osd.3 124 *** Got signal Terminated *** 2024-09-11T22:43:07.821 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:07 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3[48410]: 2024-09-11T22:43:07.386+0000 7fbad38ef640 -1 osd.3 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:43:10.570 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:10 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3[48410]: 2024-09-11T22:43:10.131+0000 7fbacbefd640 -1 osd.3 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.258572+0000 front 2024-09-11T22:42:45.258506+0000 (oldest deadline 2024-09-11T22:43:09.958728+0000) 2024-09-11T22:43:11.159 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:10 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:10.858+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:11.360 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:11.001+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:11.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:11.249+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:11.570 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3[48410]: 2024-09-11T22:43:11.158+0000 7fbacbefd640 -1 osd.3 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.258572+0000 front 2024-09-11T22:42:45.258506+0000 (oldest deadline 2024-09-11T22:43:09.958728+0000) 2024-09-11T22:43:12.208 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:11.902+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:12.209 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:11 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:11.824+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:12.360 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:12.050+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:12.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:12.199+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:12.570 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:12 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3[48410]: 2024-09-11T22:43:12.207+0000 7fbacbefd640 -1 osd.3 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.258572+0000 front 2024-09-11T22:42:45.258506+0000 (oldest deadline 2024-09-11T22:43:09.958728+0000) 2024-09-11T22:43:12.829 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:12 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:12.577+0000 7fcd3ee95640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:43.654940+0000 front 2024-09-11T22:42:43.654985+0000 (oldest deadline 2024-09-11T22:43:12.455307+0000) 2024-09-11T22:43:12.830 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:12 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:12.730+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:13.287 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:12 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:12.852+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:13.288 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:12 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:12.863+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:13.288 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:12 smithi005 podman[132533]: 2024-09-11 22:43:12.967043593 +0000 UTC m=+5.718993195 container died 2b1ff9a6a1a28a20987e72069e63cb8d5ccd8b1de09be35c9165481c2553bf46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:43:13.360 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:13.037+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:13.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:13.194+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:13.570 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:13 smithi005 podman[132533]: 2024-09-11 22:43:13.386351142 +0000 UTC m=+6.138300746 container cleanup 2b1ff9a6a1a28a20987e72069e63cb8d5ccd8b1de09be35c9165481c2553bf46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T22:43:13.570 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:13 smithi005 bash[132533]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3 2024-09-11T22:43:13.829 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:13 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:13.569+0000 7fcd3ee95640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:43.654940+0000 front 2024-09-11T22:42:43.654985+0000 (oldest deadline 2024-09-11T22:43:12.455307+0000) 2024-09-11T22:43:13.830 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:13 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:13.738+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:13.881 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:13 smithi005 podman[132550]: 2024-09-11 22:43:13.621290549 +0000 UTC m=+0.657692976 container remove 2b1ff9a6a1a28a20987e72069e63cb8d5ccd8b1de09be35c9165481c2553bf46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T22:43:14.219 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:14 smithi005 podman[132639]: 2024-09-11 22:43:13.969944433 +0000 UTC m=+0.016399254 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:43:14.219 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:14 smithi005 podman[132639]: 2024-09-11 22:43:14.105487593 +0000 UTC m=+0.151942398 container create 5e74e925e134f9c02bd8c598a6a5805629017102262f1d87667cac4f0340bd36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T22:43:14.220 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:13 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:13.880+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:14.220 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:13 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:13.881+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:14.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:14.217+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:14.360 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:14.046+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:14.541 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:14 smithi005 podman[132639]: 2024-09-11 22:43:14.341367046 +0000 UTC m=+0.387821855 container init 5e74e925e134f9c02bd8c598a6a5805629017102262f1d87667cac4f0340bd36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T22:43:14.541 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:14 smithi005 podman[132639]: 2024-09-11 22:43:14.34468651 +0000 UTC m=+0.391141319 container start 5e74e925e134f9c02bd8c598a6a5805629017102262f1d87667cac4f0340bd36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T22:43:14.541 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:14 smithi005 podman[132639]: 2024-09-11 22:43:14.441407127 +0000 UTC m=+0.487861950 container attach 5e74e925e134f9c02bd8c598a6a5805629017102262f1d87667cac4f0340bd36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-11T22:43:14.820 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 11 22:43:14 smithi005 podman[132639]: 2024-09-11 22:43:14.540987454 +0000 UTC m=+0.587442272 container died 5e74e925e134f9c02bd8c598a6a5805629017102262f1d87667cac4f0340bd36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-11T22:43:15.079 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:14.594+0000 7fcd3ee95640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:43.654940+0000 front 2024-09-11T22:42:43.654985+0000 (oldest deadline 2024-09-11T22:43:12.455307+0000) 2024-09-11T22:43:15.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:14 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:14.764+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:15.089 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:14.631+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:15.154 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:14.914+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:15.154 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:14 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:14.878+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:15.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T22:43:15.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:43:15.216 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T22:43:15.216 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T22:43:15.217 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.4 2024-09-11T22:43:15.360 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:15 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:15.088+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:15.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:15 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:15.208+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:15.539 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:15 smithi159 systemd[1]: Stopping Ceph osd.4 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:43:15.812 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:15.537+0000 7fcd4287f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:43:15.812 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:15.537+0000 7fcd4287f640 -1 osd.4 124 *** Got signal Terminated *** 2024-09-11T22:43:15.812 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:15.537+0000 7fcd4287f640 -1 osd.4 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:43:15.812 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:15.609+0000 7fcd3ee95640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:43.654940+0000 front 2024-09-11T22:42:43.654985+0000 (oldest deadline 2024-09-11T22:43:12.455307+0000) 2024-09-11T22:43:16.058 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:15 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:15.679+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:16.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:15 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:15.811+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:16.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:15.953+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:16.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:15 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:15.901+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:16.360 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:16.057+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:16.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:16.210+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:17.013 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:16.646+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:17.079 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:16 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:16.625+0000 7fcd3ee95640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:43.654940+0000 front 2024-09-11T22:42:43.654985+0000 (oldest deadline 2024-09-11T22:43:12.455307+0000) 2024-09-11T22:43:17.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:16 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:16.771+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:17.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:16.948+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:17.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:16 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:16.892+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:17.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:17 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:17.233+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:17.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:17 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:17.233+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:17.360 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:17 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:17.023+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:17.361 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:17 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:17.023+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:17.648 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:17 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:17.351+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:17.860 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:17 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:17.640+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:18.079 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:17 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:17.648+0000 7fcd3ee95640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:43.654940+0000 front 2024-09-11T22:42:43.654985+0000 (oldest deadline 2024-09-11T22:43:12.455307+0000) 2024-09-11T22:43:18.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:17 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:17.803+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:18.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:17.995+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:18.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:17.995+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:18.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:17 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:17.866+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:18.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:18 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:18.266+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:18.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:18 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:18.266+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:18.360 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:18 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:18.017+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:18.360 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:18 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:18.017+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:18.678 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:18.390+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:18.982 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:18 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:18.680+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:19.079 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:18.678+0000 7fcd3ee95640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:43.654940+0000 front 2024-09-11T22:42:43.654985+0000 (oldest deadline 2024-09-11T22:43:12.455307+0000) 2024-09-11T22:43:19.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:18.767+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:19.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:18 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:18.767+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:19.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:18 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:18.981+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:19.287 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:18 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:18.981+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:19.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:19.026+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:19.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:19.026+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:19.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:18 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:18.883+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:19.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:19 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:19.286+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:19.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:19 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:19.286+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:19.626 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:19 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:19.348+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:20.031 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:19 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:19.717+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:20.031 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:19 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:19.717+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:20.079 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:19 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4[46741]: 2024-09-11T22:43:19.660+0000 7fcd3ee95640 -1 osd.4 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:43.654940+0000 front 2024-09-11T22:42:43.654985+0000 (oldest deadline 2024-09-11T22:43:12.455307+0000) 2024-09-11T22:43:20.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:19 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:19.777+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:20.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:19 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:19.777+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:20.302 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:20 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:20.030+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:20.302 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:20 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:20.030+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:20.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:19.916+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:20.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:20 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:20.052+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:20.321 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:20 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:20.052+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:20.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:20 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:20.390+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:20.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:20 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:20.301+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:20.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:20 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:20.301+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:21.049 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:20 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:20.766+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:21.049 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:20 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:20.766+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:21.080 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:20 smithi159 podman[74336]: 2024-09-11 22:43:20.92596169 +0000 UTC m=+5.508464177 container died 7566b6e1a33724dc7d48e2cb00df1032c28386187fc1ccda48f9d688e274f9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:43:21.081 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:20 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:20.823+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:21.081 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:20 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:20.823+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:21.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:21 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:21.091+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:21.321 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:21 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:21.091+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:21.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:20 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:20.890+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:21.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:20 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:20.890+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:21.327 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:21 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:21.047+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:21.327 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:21 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:21.047+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:21.573 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:21 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:21.376+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:21.573 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:21 smithi159 podman[74336]: 2024-09-11 22:43:21.296369723 +0000 UTC m=+5.878872200 container cleanup 7566b6e1a33724dc7d48e2cb00df1032c28386187fc1ccda48f9d688e274f9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T22:43:21.574 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:21 smithi159 bash[74336]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4 2024-09-11T22:43:21.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:21 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:21.326+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:21.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:21 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:21.326+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:21.829 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:21 smithi159 podman[74353]: 2024-09-11 22:43:21.573254749 +0000 UTC m=+0.641422935 container remove 7566b6e1a33724dc7d48e2cb00df1032c28386187fc1ccda48f9d688e274f9c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T22:43:21.829 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:21 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:21.778+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:21.830 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:21 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:21.778+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:22.110 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:22 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:22.012+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:22.110 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:22 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:22.012+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:22.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:21 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:21.806+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:22.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:21 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:21.806+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:22.151 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:21 smithi159 podman[74444]: 2024-09-11 22:43:21.787541522 +0000 UTC m=+0.016028747 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:43:22.152 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:21 smithi159 podman[74444]: 2024-09-11 22:43:21.915752882 +0000 UTC m=+0.144240090 container create bfc6af578e1275d019d68628134999a66b9534704b58730ceb7d6303fa2a3cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-11T22:43:22.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:22 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:22.094+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:22.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:22 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:22.094+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:22.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:21 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:21.896+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:22.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:21 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:21.896+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:22.579 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:22 smithi159 podman[74444]: 2024-09-11 22:43:22.151697569 +0000 UTC m=+0.380184784 container init bfc6af578e1275d019d68628134999a66b9534704b58730ceb7d6303fa2a3cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-11T22:43:22.580 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:22 smithi159 podman[74444]: 2024-09-11 22:43:22.155082998 +0000 UTC m=+0.383570208 container start bfc6af578e1275d019d68628134999a66b9534704b58730ceb7d6303fa2a3cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T22:43:22.580 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:22 smithi159 podman[74444]: 2024-09-11 22:43:22.260024247 +0000 UTC m=+0.488511468 container attach bfc6af578e1275d019d68628134999a66b9534704b58730ceb7d6303fa2a3cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True) 2024-09-11T22:43:22.580 INFO:journalctl@ceph.osd.4.smithi159.stdout:Sep 11 22:43:22 smithi159 podman[74444]: 2024-09-11 22:43:22.340889001 +0000 UTC m=+0.569376210 container died bfc6af578e1275d019d68628134999a66b9534704b58730ceb7d6303fa2a3cf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-11T22:43:22.580 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:22 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:22.337+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:22.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:22 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:22.321+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:22.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:22 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:22.321+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:23.006 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:22 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:22.751+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:23.006 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:22 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:22.751+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:23.026 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T22:43:23.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:43:23.065 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T22:43:23.065 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T22:43:23.066 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.5 2024-09-11T22:43:23.093 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:22 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:22.986+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:23.094 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:22 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:22.986+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:23.094 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:22 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:22.765+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:23.094 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:22 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:22.765+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:23.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:23 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:23.103+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:23.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:23 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:23.103+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:23.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:22 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:22.937+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:23.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:22 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:22.937+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:23.360 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:23 smithi172 systemd[1]: Stopping Ceph osd.5 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:43:23.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:23 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:23.296+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:23.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:23 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:23.296+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:23.713 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:23 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:23.338+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:23.734 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:23 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:23.393+0000 7f6e0ee35640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:43:23.734 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:23 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:23.393+0000 7f6e0ee35640 -1 osd.5 124 *** Got signal Terminated *** 2024-09-11T22:43:23.734 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:23 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:23.393+0000 7f6e0ee35640 -1 osd.5 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:43:24.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:23 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:23.712+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:24.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:23 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:23.712+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:24.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:23 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:23.733+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:24.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:23 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:23.733+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:24.111 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:23 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:23.976+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:24.111 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:23 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:23.976+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:24.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:24 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:24.095+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:24.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:24 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:24.095+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:24.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:23 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:23.963+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:24.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:23 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:23.963+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:24.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:24 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:24.285+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:24.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:24 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:24.285+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:24.744 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:24 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:24.345+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:25.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:24 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:24.744+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:25.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:24 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:24.744+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:25.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:24 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:24.773+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:25.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:24 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:24.773+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:25.110 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:24 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:24.970+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:25.111 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:24 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:24.970+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:25.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:25 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:25.108+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:25.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:25 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:25.108+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:25.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:24 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:24.950+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:25.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:24 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:24.950+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:25.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:25 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:25.259+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:25.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:25 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:25.259+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:25.758 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:25 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:25.365+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:25.758 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:25 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:25.365+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:26.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:25 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:25.757+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:26.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:25 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:25.757+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:26.113 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:25 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:25.737+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:26.113 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:25 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:25.737+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:26.114 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:25 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:25.737+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:26.114 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:25 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:25.975+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:26.114 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:25 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:25.975+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:26.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:26.103+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:26.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:26.103+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:26.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:25 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:25.942+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:26.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:25 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:25.942+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:26.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:26.326+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:26.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:26.326+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:26.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:26.217+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:26.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:26.217+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:26.967 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:26.700+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:26.968 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:26.700+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:26.968 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:26.700+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:27.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:26.803+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:27.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:26 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:26.803+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:27.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:27 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:27.132+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:27.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:27 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:27.132+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:27.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:26.982+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:27.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:26.982+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:27.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:26 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:26.982+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:27.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:27 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:27.215+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:27.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:27 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:27.215+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:27.361 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:26.966+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:27.361 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:26.966+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:27.361 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:26 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:26.966+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:01.806780+0000 front 2024-09-11T22:43:01.806748+0000 (oldest deadline 2024-09-11T22:43:26.506914+0000) 2024-09-11T22:43:27.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:27 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:27.304+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:27.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:27 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:27.304+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:28.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:27 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:27.770+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:28.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:27 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:27.770+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:28.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:27 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:27.770+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:28.110 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:27 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:27.928+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.302614+0000 front 2024-09-11T22:42:46.302667+0000 (oldest deadline 2024-09-11T22:43:10.402486+0000) 2024-09-11T22:43:28.110 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:27 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:27.928+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.505434+0000 front 2024-09-11T22:42:55.505270+0000 (oldest deadline 2024-09-11T22:43:16.605300+0000) 2024-09-11T22:43:28.111 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:27 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5[47323]: 2024-09-11T22:43:27.928+0000 7f6e07443640 -1 osd.5 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:01.806780+0000 front 2024-09-11T22:43:01.806748+0000 (oldest deadline 2024-09-11T22:43:26.506914+0000) 2024-09-11T22:43:28.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:27 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:27.735+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:28.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:27 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:27.735+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:28.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:27 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:27.735+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:28.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:28 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:28.174+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:28.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:28 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:28.174+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:28.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:28 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:28.174+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.221384+0000 front 2024-09-11T22:43:02.221416+0000 (oldest deadline 2024-09-11T22:43:27.521451+0000) 2024-09-11T22:43:28.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:27 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:27.991+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:28.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:27 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:27.991+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:28.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:27 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:27.991+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:28.581 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:28 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:28.293+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:28.582 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:28 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:28.293+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:28.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:28 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:28.217+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:28.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:28 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:28.217+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:28.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:28 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:28.217+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:29.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:28 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:28.742+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:29.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:28 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:28.742+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:29.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:28 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:28.742+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:29.110 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:28 smithi172 podman[74488]: 2024-09-11 22:43:28.971886632 +0000 UTC m=+5.708371862 container died 3af4b6f48dd958f4e981deb2a2a9591ccfaebad36a983a8b30a8ae9c535fcdbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-11T22:43:29.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:28 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:28.784+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:29.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:28 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:28.784+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:29.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:28 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:28.784+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:29.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:29 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:29.194+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:29.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:29 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:29.194+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:29.321 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:29 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:29.194+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.221384+0000 front 2024-09-11T22:43:02.221416+0000 (oldest deadline 2024-09-11T22:43:27.521451+0000) 2024-09-11T22:43:29.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:29 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:29.024+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:29.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:29 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:29.024+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:29.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:29 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:29.024+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:29.610 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:29 smithi172 podman[74488]: 2024-09-11 22:43:29.376331591 +0000 UTC m=+6.112816820 container cleanup 3af4b6f48dd958f4e981deb2a2a9591ccfaebad36a983a8b30a8ae9c535fcdbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:43:29.610 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:29 smithi172 bash[74488]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5 2024-09-11T22:43:29.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:29 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:29.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:29.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:29 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:29.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:29.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:29 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:29.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:29.758 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:29 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:29.334+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:29.758 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:29 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:29.334+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:29.990 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:29 smithi172 podman[74502]: 2024-09-11 22:43:29.636178302 +0000 UTC m=+0.656501646 container remove 3af4b6f48dd958f4e981deb2a2a9591ccfaebad36a983a8b30a8ae9c535fcdbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T22:43:29.991 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:29 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:29.770+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:29.991 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:29 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:29.770+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:29.991 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:29 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:29.770+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:30.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:29 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:29.757+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:30.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:29 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:29.757+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:30.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:29 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:29.757+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:30.249 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:29 smithi172 podman[74593]: 2024-09-11 22:43:29.893341153 +0000 UTC m=+0.016315603 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:43:30.250 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:30 smithi172 podman[74593]: 2024-09-11 22:43:30.020381244 +0000 UTC m=+0.143355672 container create d8f2f02a4220a8c8e746e4beac86d91262144534e4c74cd09a5e2148aa821c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:43:30.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:30 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:30.202+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:30.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:30 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:30.202+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:30.321 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:30 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:30.202+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.221384+0000 front 2024-09-11T22:43:02.221416+0000 (oldest deadline 2024-09-11T22:43:27.521451+0000) 2024-09-11T22:43:30.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:29 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:29.994+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:30.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:29 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:29.994+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:30.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:29 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:29.994+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:30.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:30 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:30.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:30.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:30 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:30.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:30.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:30 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:30.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:30.611 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:30 smithi172 podman[74593]: 2024-09-11 22:43:30.298519105 +0000 UTC m=+0.421493547 container init d8f2f02a4220a8c8e746e4beac86d91262144534e4c74cd09a5e2148aa821c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T22:43:30.611 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:30 smithi172 podman[74593]: 2024-09-11 22:43:30.301808481 +0000 UTC m=+0.424782914 container start d8f2f02a4220a8c8e746e4beac86d91262144534e4c74cd09a5e2148aa821c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T22:43:30.611 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:30 smithi172 podman[74593]: 2024-09-11 22:43:30.381605522 +0000 UTC m=+0.504579962 container attach d8f2f02a4220a8c8e746e4beac86d91262144534e4c74cd09a5e2148aa821c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T22:43:30.611 INFO:journalctl@ceph.osd.5.smithi172.stdout:Sep 11 22:43:30 smithi172 podman[74593]: 2024-09-11 22:43:30.492975816 +0000 UTC m=+0.615950245 container died d8f2f02a4220a8c8e746e4beac86d91262144534e4c74cd09a5e2148aa821c20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T22:43:30.724 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:30 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:30.364+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:30.724 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:30 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:30.364+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:31.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:30 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:30.724+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:31.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:30 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:30.724+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:31.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:30 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:30.724+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:31.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:30 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:30.748+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:31.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:30 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:30.748+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:31.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:30 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:30.748+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:31.188 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T22:43:31.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:43:31.269 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T22:43:31.270 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T22:43:31.270 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.6 2024-09-11T22:43:31.298 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:31.181+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:31.298 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:31.181+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:31.298 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:31.181+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.221384+0000 front 2024-09-11T22:43:02.221416+0000 (oldest deadline 2024-09-11T22:43:27.521451+0000) 2024-09-11T22:43:31.298 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:31.001+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:31.298 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:31.001+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:31.299 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:31.001+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:31.570 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:31 smithi005 systemd[1]: Stopping Ceph osd.6 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:43:31.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:31 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:31.200+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:31.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:31 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:31.200+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:31.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:31 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:31.200+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:31.760 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:31 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:31.329+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:31.760 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:31 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:31.329+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:31.958 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:31.613+0000 7fe594f78640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:43:31.958 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:31.613+0000 7fe594f78640 -1 osd.6 124 *** Got signal Terminated *** 2024-09-11T22:43:31.959 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:31.613+0000 7fe594f78640 -1 osd.6 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:43:32.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:31 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:31.759+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:32.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:31 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:31.759+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:32.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:31 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:31.759+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:32.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:31 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:31.766+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:32.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:31 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:31.766+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:32.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:31 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:31.766+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:32.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:32.204+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:32.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:32.204+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:32.321 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:32.204+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.221384+0000 front 2024-09-11T22:43:02.221416+0000 (oldest deadline 2024-09-11T22:43:27.521451+0000) 2024-09-11T22:43:32.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:31.957+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:32.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:31.957+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:32.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:31.957+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:32.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:32 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:32.287+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:32.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:32 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:32.287+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:32.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:32 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:32.216+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:32.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:32 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:32.216+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:32.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:32 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:32.216+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:33.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:32 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:32.797+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:33.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:32 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:32.797+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:33.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:32 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:32.797+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:33.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:32 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:32.791+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:33.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:32 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:32.791+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:33.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:32 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:32.791+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:33.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:33 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:33.174+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:33.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:33 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:33.174+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:33.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:33 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:33.174+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.221384+0000 front 2024-09-11T22:43:02.221416+0000 (oldest deadline 2024-09-11T22:43:27.521451+0000) 2024-09-11T22:43:33.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:32.956+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:33.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:32.956+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:33.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:32 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:32.956+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:33.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:33 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:33.304+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:33.580 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:33 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:33.304+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:33.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:33 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:33.261+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:33.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:33 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:33.261+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:33.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:33 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:33.261+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:34.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:33 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:33.811+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:34.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:33 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:33.811+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:34.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:33 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:33.811+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:34.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:33 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:33.747+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:34.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:33 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:33.747+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:34.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:33 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:33.747+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:34.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:34 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:34.164+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:34.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:34 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:34.164+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:34.321 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:34 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:34.164+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.221384+0000 front 2024-09-11T22:43:02.221416+0000 (oldest deadline 2024-09-11T22:43:27.521451+0000) 2024-09-11T22:43:34.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:33 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:33.932+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:34.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:33 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:33.932+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:34.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:33 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:33.932+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:34.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:34 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:34.279+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:34.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:34 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:34.279+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:34.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:34 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:34.301+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:34.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:34 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:34.301+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:34.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:34 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:34.301+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:35.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:34 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:34.780+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:35.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:34 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:34.780+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:35.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:34 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:34.780+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:35.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:34 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:34.780+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:35.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:34 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:34.709+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:35.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:34 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:34.710+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:35.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:34 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:34.710+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:35.175 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:34 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:34.886+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:35.175 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:34 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:34.886+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:35.176 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:34 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:34.886+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:35.384 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:35 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:35.342+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:35.385 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:35 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:35.342+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:35.385 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:35 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:35.342+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:35.570 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:35 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:35.174+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:35.570 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:35 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:35.174+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:35.571 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:35 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:35.174+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.221384+0000 front 2024-09-11T22:43:02.221416+0000 (oldest deadline 2024-09-11T22:43:27.521451+0000) 2024-09-11T22:43:35.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:35.262+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:35.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:35.262+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:35.580 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:35.262+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:10.355378+0000 front 2024-09-11T22:43:10.355423+0000 (oldest deadline 2024-09-11T22:43:35.055421+0000) 2024-09-11T22:43:36.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:35.819+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:36.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:35.819+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:36.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:35.819+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:36.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:35.819+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:36.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:35 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:35.738+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:36.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:35 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:35.738+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:36.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:35 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:35.738+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:36.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:35 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:35.738+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:36.213 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:35 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:35.841+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:36.214 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:35 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:35.841+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:36.214 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:35 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:35.841+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:36.214 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:35 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:35.841+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:36.570 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:36 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:36.212+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:45.318566+0000 front 2024-09-11T22:42:45.318405+0000 (oldest deadline 2024-09-11T22:43:10.018529+0000) 2024-09-11T22:43:36.570 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:36 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:36.212+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.319488+0000 front 2024-09-11T22:42:55.319638+0000 (oldest deadline 2024-09-11T22:43:17.019619+0000) 2024-09-11T22:43:36.570 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:36 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6[55335]: 2024-09-11T22:43:36.212+0000 7fe59158e640 -1 osd.6 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.221384+0000 front 2024-09-11T22:43:02.221416+0000 (oldest deadline 2024-09-11T22:43:27.521451+0000) 2024-09-11T22:43:36.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:36 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:36.278+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:36.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:36 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:36.278+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:36.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:36 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:36.278+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:10.355378+0000 front 2024-09-11T22:43:10.355423+0000 (oldest deadline 2024-09-11T22:43:35.055421+0000) 2024-09-11T22:43:36.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:36 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:36.298+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:36.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:36 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:36.298+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:36.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:36 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:36.298+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:37.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:36 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:36.817+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:37.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:36 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:36.817+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:37.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:36 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:36.817+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:37.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:36 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:36.817+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:37.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:36 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:36.743+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:37.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:36 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:36.743+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:37.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:36 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:36.743+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:37.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:36 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:36.743+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:37.320 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:37 smithi005 podman[132741]: 2024-09-11 22:43:37.127952319 +0000 UTC m=+5.659325718 container died e1fad748b3059c3f93c0e4b1cd485354dd08d39b4b4965295b83ae70e1c890b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T22:43:37.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:36 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:36.879+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:37.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:36 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:36.879+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:37.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:36 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:36.879+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:37.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:36 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:36.879+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:37.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:37 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:37.265+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:37.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:37 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:37.265+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:37.580 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:37 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:37.265+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:10.355378+0000 front 2024-09-11T22:43:10.355423+0000 (oldest deadline 2024-09-11T22:43:35.055421+0000) 2024-09-11T22:43:37.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:37 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:37.341+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:37.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:37 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:37.341+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:37.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:37 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:37.341+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:37.820 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:37 smithi005 podman[132741]: 2024-09-11 22:43:37.623496276 +0000 UTC m=+6.154869720 container cleanup e1fad748b3059c3f93c0e4b1cd485354dd08d39b4b4965295b83ae70e1c890b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:43:37.820 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:37 smithi005 bash[132741]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6 2024-09-11T22:43:38.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:37 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:37.776+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:38.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:37 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:37.776+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:38.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:37 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:37.776+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:38.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:37 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:37.776+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:38.202 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:37 smithi005 podman[132756]: 2024-09-11 22:43:37.866612643 +0000 UTC m=+0.732197090 container remove e1fad748b3059c3f93c0e4b1cd485354dd08d39b4b4965295b83ae70e1c890b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-11T22:43:38.203 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:37 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:37.855+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:38.203 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:37 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:37.855+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:38.203 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:37 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:37.855+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:38.203 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:37 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:37.855+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:38.278 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:37 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:37.863+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:38.279 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:37 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:37.863+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:38.279 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:37 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:37.863+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:38.279 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:37 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:37.863+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:38.469 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:38 smithi005 podman[132847]: 2024-09-11 22:43:38.105370893 +0000 UTC m=+0.016647541 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:43:38.470 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:38 smithi005 podman[132847]: 2024-09-11 22:43:38.238704801 +0000 UTC m=+0.149981490 container create 2822bc1670d5abdac76aa6575e0b7fee46eaa41bd2f32ffc181294a987e80b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T22:43:38.470 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:38 smithi005 podman[132847]: 2024-09-11 22:43:38.469712491 +0000 UTC m=+0.380989142 container init 2822bc1670d5abdac76aa6575e0b7fee46eaa41bd2f32ffc181294a987e80b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True) 2024-09-11T22:43:38.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:38 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:38.278+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:38.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:38 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:38.278+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:38.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:38 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:38.278+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:10.355378+0000 front 2024-09-11T22:43:10.355423+0000 (oldest deadline 2024-09-11T22:43:35.055421+0000) 2024-09-11T22:43:38.746 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:38 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:38.391+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:38.747 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:38 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:38.391+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:38.747 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:38 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:38.391+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:38.821 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:38 smithi005 podman[132847]: 2024-09-11 22:43:38.473287944 +0000 UTC m=+0.384564594 container start 2822bc1670d5abdac76aa6575e0b7fee46eaa41bd2f32ffc181294a987e80b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T22:43:38.821 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:38 smithi005 podman[132847]: 2024-09-11 22:43:38.561587595 +0000 UTC m=+0.472864261 container attach 2822bc1670d5abdac76aa6575e0b7fee46eaa41bd2f32ffc181294a987e80b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-11T22:43:38.822 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 11 22:43:38 smithi005 podman[132847]: 2024-09-11 22:43:38.668307917 +0000 UTC m=+0.579584565 container died 2822bc1670d5abdac76aa6575e0b7fee46eaa41bd2f32ffc181294a987e80b5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-11T22:43:39.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:38 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:38.745+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:39.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:38 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:38.745+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:39.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:38 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:38.745+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:39.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:38 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:38.745+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:39.264 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:38 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:38.867+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:39.264 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:38 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:38.867+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:39.264 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:38 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:38.867+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:39.264 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:38 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:38.867+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:39.315 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:38 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:38.884+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:39.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:38 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:38.884+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:39.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:38 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:38.884+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:39.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:38 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:38.884+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:39.347 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T22:43:39.432 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:43:39.432 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T22:43:39.433 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T22:43:39.433 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.7 2024-09-11T22:43:39.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:39.264+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:39.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:39.264+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:39.580 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:39.264+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:10.355378+0000 front 2024-09-11T22:43:10.355423+0000 (oldest deadline 2024-09-11T22:43:35.055421+0000) 2024-09-11T22:43:39.580 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:39 smithi159 systemd[1]: Stopping Ceph osd.7 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:43:39.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:39 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:39.356+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:39.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:39 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:39.356+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:39.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:39 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:39.356+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:40.079 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:39.766+0000 7f7253fcc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:43:40.079 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:39.766+0000 7f7253fcc640 -1 osd.7 124 *** Got signal Terminated *** 2024-09-11T22:43:40.080 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:39.766+0000 7f7253fcc640 -1 osd.7 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:43:40.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:39.834+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:40.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:39.834+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:40.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:39.834+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:40.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:39.834+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:40.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:39 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:39.704+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:40.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:39 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:39.704+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:40.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:39 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:39.704+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:40.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:39 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:39.704+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:40.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:39 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:39.856+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:40.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:39 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:39.856+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:40.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:39 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:39.856+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:40.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:39 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:39.856+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:40.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:40.277+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:40.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:40.277+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:40.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:40.277+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:10.355378+0000 front 2024-09-11T22:43:10.355423+0000 (oldest deadline 2024-09-11T22:43:35.055421+0000) 2024-09-11T22:43:40.751 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:40 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:40.364+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:40.751 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:40 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:40.364+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:40.751 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:40 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:40.364+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:41.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:40.869+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:41.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:40.869+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:41.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:40.869+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:41.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:40 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:40.869+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:41.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:40 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:40.750+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:41.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:40 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:40.750+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:41.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:40 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:40.750+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:41.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:40 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:40.750+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:41.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:40 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:40.844+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:41.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:40 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:40.844+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:41.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:40 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:40.844+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:41.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:40 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:40.844+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:41.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:41 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:41.311+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:41.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:41 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:41.311+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:41.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:41 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:41.311+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:10.355378+0000 front 2024-09-11T22:43:10.355423+0000 (oldest deadline 2024-09-11T22:43:35.055421+0000) 2024-09-11T22:43:41.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:41 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:41.355+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:41.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:41 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:41.355+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:41.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:41 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:41.355+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:42.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:41 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:41.755+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:42.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:41 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:41.755+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:42.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:41 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:41.755+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:42.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:41 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:41.755+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:42.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:41 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:41.838+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:42.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:41 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:41.838+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:42.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:41 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:41.838+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:42.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:41 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:41.838+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:42.321 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:41 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:41.841+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:42.321 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:41 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:41.841+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:42.321 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:41 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:41.841+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:42.322 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:41 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:41.841+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:42.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:42 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:42.320+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:42.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:42 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:42.320+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:42.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:42 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:42.320+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:10.355378+0000 front 2024-09-11T22:43:10.355423+0000 (oldest deadline 2024-09-11T22:43:35.055421+0000) 2024-09-11T22:43:42.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:42 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:42.343+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:42.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:42 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:42.343+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:42.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:42 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:42.343+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:42.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:42 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:42.343+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:43.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:42 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:42.784+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:43.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:42 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:42.784+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:43.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:42 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:42.784+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:43.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:42 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:42.784+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:43.287 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:42 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:42.869+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:43.287 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:42 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:42.869+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:43.287 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:42 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:42.869+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:43.287 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:42 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:42.869+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:43.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:42.846+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:43.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:42.846+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:43.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:42.846+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:43.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:42 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:42.846+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:43.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:43 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:43.286+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:43.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:43 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:43.286+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:43.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:43 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:43.286+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:10.355378+0000 front 2024-09-11T22:43:10.355423+0000 (oldest deadline 2024-09-11T22:43:35.055421+0000) 2024-09-11T22:43:43.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:43 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:43.323+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:43.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:43 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:43.323+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:43.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:43 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:43.323+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:43.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:43 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:43.323+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:44.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:43 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:43.762+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:44.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:43 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:43.762+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:44.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:43 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:43.762+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:44.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:43 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:43.762+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:44.315 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:43 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:43.910+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:44.315 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:43 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:43.910+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:44.315 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:43 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:43.910+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:44.316 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:43 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:43.910+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:44.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:43 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:43.838+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:44.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:43 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:43.838+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:44.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:43 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:43.838+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:44.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:43 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:43.838+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:44.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:44 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:44.314+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:55.451367+0000 front 2024-09-11T22:42:55.451401+0000 (oldest deadline 2024-09-11T22:43:16.551564+0000) 2024-09-11T22:43:44.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:44 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:44.314+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.953295+0000 front 2024-09-11T22:43:02.953216+0000 (oldest deadline 2024-09-11T22:43:25.253202+0000) 2024-09-11T22:43:44.579 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:44 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7[53654]: 2024-09-11T22:43:44.314+0000 7f724c5da640 -1 osd.7 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:10.355378+0000 front 2024-09-11T22:43:10.355423+0000 (oldest deadline 2024-09-11T22:43:35.055421+0000) 2024-09-11T22:43:44.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:44 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:44.301+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:44.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:44 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:44.301+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:44.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:44 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:44.301+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:44.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:44 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:44.301+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:45.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:44 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:44.729+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:45.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:44 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:44.729+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:45.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:44 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:44.729+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:45.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:44 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:44.729+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:45.148 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:44 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:44.865+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:45.149 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:44 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:44.865+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:45.149 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:44 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:44.865+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:45.149 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:44 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:44.865+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:45.149 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:44 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:44.865+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:45.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:44 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:44.878+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:45.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:44 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:44.878+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:45.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:44 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:44.878+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:45.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:44 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:44.878+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:45.467 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:45 smithi159 podman[74548]: 2024-09-11 22:43:45.148230837 +0000 UTC m=+5.503654338 container stop b13e34e6b59db0c48a4400501ae3cb305591d6a32488ef3211481d808e8f411c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T22:43:45.468 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:45 smithi159 podman[74548]: 2024-09-11 22:43:45.173989312 +0000 UTC m=+5.529412812 container died b13e34e6b59db0c48a4400501ae3cb305591d6a32488ef3211481d808e8f411c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:43:45.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:45 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:45.268+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:45.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:45 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:45.268+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:45.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:45 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:45.268+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:45.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:45 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:45.268+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:45.829 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:45 smithi159 podman[74548]: 2024-09-11 22:43:45.566187532 +0000 UTC m=+5.921611044 container cleanup b13e34e6b59db0c48a4400501ae3cb305591d6a32488ef3211481d808e8f411c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-11T22:43:45.829 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:45 smithi159 bash[74548]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7 2024-09-11T22:43:46.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:45 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:45.759+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:46.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:45 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:45.759+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:46.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:45 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:45.759+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:46.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:45 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:45.759+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:46.159 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:45 smithi159 podman[74574]: 2024-09-11 22:43:45.850967414 +0000 UTC m=+0.680330569 container remove b13e34e6b59db0c48a4400501ae3cb305591d6a32488ef3211481d808e8f411c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:43:46.159 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:45 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:45.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:46.159 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:45 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:45.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:46.159 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:45 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:45.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:46.160 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:45 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:45.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:46.160 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:45 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:45.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:46.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:45 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:45.908+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:46.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:45 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:45.908+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:46.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:45 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:45.908+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:46.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:45 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:45.908+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:46.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:45 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:45.908+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:46.421 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:46 smithi159 podman[74665]: 2024-09-11 22:43:46.061502332 +0000 UTC m=+0.015569468 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:43:46.421 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:46 smithi159 podman[74665]: 2024-09-11 22:43:46.160068384 +0000 UTC m=+0.114135522 container create e73c3271833bd94688432031088334d798040636bec9081bdb7bd117287aad0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T22:43:46.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:46 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:46.253+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:46.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:46 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:46.253+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:46.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:46 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:46.253+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:46.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:46 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:46.253+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:46.829 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:46 smithi159 podman[74665]: 2024-09-11 22:43:46.421283229 +0000 UTC m=+0.375350364 container init e73c3271833bd94688432031088334d798040636bec9081bdb7bd117287aad0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:43:46.830 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:46 smithi159 podman[74665]: 2024-09-11 22:43:46.425085184 +0000 UTC m=+0.379152322 container start e73c3271833bd94688432031088334d798040636bec9081bdb7bd117287aad0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T22:43:46.830 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:46 smithi159 podman[74665]: 2024-09-11 22:43:46.521390957 +0000 UTC m=+0.475458110 container attach e73c3271833bd94688432031088334d798040636bec9081bdb7bd117287aad0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T22:43:46.830 INFO:journalctl@ceph.osd.7.smithi159.stdout:Sep 11 22:43:46 smithi159 podman[74665]: 2024-09-11 22:43:46.6178225 +0000 UTC m=+0.571889649 container died e73c3271833bd94688432031088334d798040636bec9081bdb7bd117287aad0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T22:43:47.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:46 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:46.736+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:47.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:46 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:46.736+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:47.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:46 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:46.736+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:47.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:46 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:46.736+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:47.283 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:46 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:46.866+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:47.284 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:46 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:46.866+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:47.284 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:46 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:46.866+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:47.284 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:46 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:46.866+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:47.284 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:46 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:46.866+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:47.306 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T22:43:47.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:46 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:46.944+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:47.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:46 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:46.944+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:47.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:46 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:46.944+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:47.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:46 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:46.944+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:47.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:46 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:46.944+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:47.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:43:47.346 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T22:43:47.346 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-11T22:43:47.347 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.8 2024-09-11T22:43:47.610 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:47 smithi172 systemd[1]: Stopping Ceph osd.8 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:43:47.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:47.280+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:47.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:47.280+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:47.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:47.280+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:47.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:47.280+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:48.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:47.678+0000 7f939200a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:43:48.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:47.678+0000 7f939200a640 -1 osd.8 124 *** Got signal Terminated *** 2024-09-11T22:43:48.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:47.678+0000 7f939200a640 -1 osd.8 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:43:48.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:47.695+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:48.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:47.695+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:48.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:47.695+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:48.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:47 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:47.695+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:48.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:47.981+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:48.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:47.981+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:48.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:47.981+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:48.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:47.981+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:48.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:47 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:47.981+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:48.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:47 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:47.832+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:48.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:47 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:47.832+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:48.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:47 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:47.832+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:48.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:47 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:47.832+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:48.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:47 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:47.832+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:48.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:48 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:48.312+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:48.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:48 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:48.312+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:48.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:48 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:48.312+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:48.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:48 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:48.312+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:49.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:48 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:48.656+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:49.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:48 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:48.657+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:49.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:48 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:48.657+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:49.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:48 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:48.657+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:49.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:48 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:48.967+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:49.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:48 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:48.967+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:49.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:48 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:48.967+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:49.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:48 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:48.967+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:49.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:48 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:48.967+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:49.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:48 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:48.829+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:49.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:48 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:48.829+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:49.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:48 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:48.829+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:49.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:48 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:48.829+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:49.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:48 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:48.829+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:49.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:49 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:49.359+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:49.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:49 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:49.359+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:49.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:49 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:49.359+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:49.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:49 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:49.359+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:50.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:49 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:49.684+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:50.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:49 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:49.684+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:50.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:49 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:49.684+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:50.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:49 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:49.684+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:50.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:49 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:49.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:50.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:49 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:49.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:50.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:49 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:49.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:50.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:49 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:49.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:50.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:49 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:49.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:50.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:49 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:49.846+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:50.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:49 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:49.846+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:50.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:49 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:49.846+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:50.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:49 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:49.846+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:50.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:49 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:49.846+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:50.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:49 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:49.846+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:43:50.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:50 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:50.324+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:50.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:50 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:50.324+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:50.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:50 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:50.324+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:50.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:50 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:50.324+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:51.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:50 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:50.813+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:51.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:50 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:50.813+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:51.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:50 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:50.813+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:51.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:50 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:50.813+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:51.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:50 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:50.813+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:51.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:50 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:50.813+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:43:51.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:50 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:50.698+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:51.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:50 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:50.698+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:51.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:50 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:50.698+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:51.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:50 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:50.698+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:51.111 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:50 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:50.698+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.094333+0000 front 2024-09-11T22:43:27.094407+0000 (oldest deadline 2024-09-11T22:43:50.594432+0000) 2024-09-11T22:43:51.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:50 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:50.905+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:51.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:50 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:50.905+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:51.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:50 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:50.905+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:51.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:50 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:50.905+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:51.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:50 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:50.905+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:51.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:50 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:50.905+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:25.686605+0000 front 2024-09-11T22:43:25.686528+0000 (oldest deadline 2024-09-11T22:43:50.386646+0000) 2024-09-11T22:43:51.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:51 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:51.298+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:51.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:51 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:51.298+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:51.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:51 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:51.298+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:51.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:51 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:51.298+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:51.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:51 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:51.298+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:43:52.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:51 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:51.814+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:52.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:51 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:51.814+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:52.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:51 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:51.814+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:52.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:51 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:51.814+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:52.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:51 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:51.814+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:52.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:51 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:51.814+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:43:52.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:51 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:51.694+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:52.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:51 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:51.694+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:52.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:51 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:51.694+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:52.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:51 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:51.694+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:52.110 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:51 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:51.694+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.094333+0000 front 2024-09-11T22:43:27.094407+0000 (oldest deadline 2024-09-11T22:43:50.594432+0000) 2024-09-11T22:43:52.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:51.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:52.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:51.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:52.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:51.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:52.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:51.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:52.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:51.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:52.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:51 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:51.940+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:25.686605+0000 front 2024-09-11T22:43:25.686528+0000 (oldest deadline 2024-09-11T22:43:50.386646+0000) 2024-09-11T22:43:52.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:52.317+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:52.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:52.317+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:52.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:52.317+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:52.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:52.317+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:52.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:52.317+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:43:53.060 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:52.649+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:42.088562+0000 front 2024-09-11T22:42:42.088437+0000 (oldest deadline 2024-09-11T22:43:13.888969+0000) 2024-09-11T22:43:53.060 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:52.649+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.989917+0000 front 2024-09-11T22:42:54.990042+0000 (oldest deadline 2024-09-11T22:43:19.689974+0000) 2024-09-11T22:43:53.060 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:52.649+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.191034+0000 front 2024-09-11T22:43:03.191052+0000 (oldest deadline 2024-09-11T22:43:24.891158+0000) 2024-09-11T22:43:53.060 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:52.649+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:08.892578+0000 front 2024-09-11T22:43:08.892566+0000 (oldest deadline 2024-09-11T22:43:34.792465+0000) 2024-09-11T22:43:53.061 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8[54261]: 2024-09-11T22:43:52.649+0000 7f938e620640 -1 osd.8 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.094333+0000 front 2024-09-11T22:43:27.094407+0000 (oldest deadline 2024-09-11T22:43:50.594432+0000) 2024-09-11T22:43:53.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:52 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:52.805+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:53.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:52 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:52.805+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:53.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:52 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:52.805+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:53.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:52 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:52.805+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:53.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:52 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:52.805+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:53.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:52 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:52.805+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:43:53.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:52 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:52.973+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:53.324 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:52 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:52.973+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:53.324 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:52 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:52.973+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:53.324 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:52 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:52.973+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:53.324 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:52 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:52.973+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:53.324 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:52 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:52.973+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:25.686605+0000 front 2024-09-11T22:43:25.686528+0000 (oldest deadline 2024-09-11T22:43:50.386646+0000) 2024-09-11T22:43:53.355 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:53 smithi172 podman[74709]: 2024-09-11 22:43:53.060234764 +0000 UTC m=+5.513367422 container died c4ee0b999fd06c7653267920d73e435f280ffb02c28980b2be0aaf092e0574f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-11T22:43:53.356 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:53 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:53.292+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:53.356 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:53 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:53.292+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:53.356 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:53 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:53.292+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:53.356 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:53 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:53.292+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:53.356 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:53 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:53.292+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:43:53.610 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:53 smithi172 podman[74709]: 2024-09-11 22:43:53.453891713 +0000 UTC m=+5.907024373 container cleanup c4ee0b999fd06c7653267920d73e435f280ffb02c28980b2be0aaf092e0574f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:43:53.610 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:53 smithi172 bash[74709]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8 2024-09-11T22:43:54.013 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:53 smithi172 podman[74723]: 2024-09-11 22:43:53.697000421 +0000 UTC m=+0.632087338 container remove c4ee0b999fd06c7653267920d73e435f280ffb02c28980b2be0aaf092e0574f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-11T22:43:54.305 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:54 smithi172 podman[74813]: 2024-09-11 22:43:53.915846607 +0000 UTC m=+0.015625977 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:43:54.305 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:54 smithi172 podman[74813]: 2024-09-11 22:43:54.056180377 +0000 UTC m=+0.155959743 container create f4261e4bd283542add317a2aba392badc001c28cbf43a443c2ba453f9b337625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731) 2024-09-11T22:43:54.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:53 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:53.976+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:54.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:53 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:53.976+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:54.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:53 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:53.976+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:54.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:53 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:53.976+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:54.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:53 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:53.976+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:54.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:53 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:53.976+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:25.686605+0000 front 2024-09-11T22:43:25.686528+0000 (oldest deadline 2024-09-11T22:43:50.386646+0000) 2024-09-11T22:43:54.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:53 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:53.831+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:54.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:53 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:53.831+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:54.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:53 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:53.831+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:54.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:53 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:53.831+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:54.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:53 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:53.831+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:54.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:53 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:53.831+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:43:54.610 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:54 smithi172 podman[74813]: 2024-09-11 22:43:54.308483114 +0000 UTC m=+0.408262481 container init f4261e4bd283542add317a2aba392badc001c28cbf43a443c2ba453f9b337625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-11T22:43:54.611 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:54 smithi172 podman[74813]: 2024-09-11 22:43:54.311790669 +0000 UTC m=+0.411570031 container start f4261e4bd283542add317a2aba392badc001c28cbf43a443c2ba453f9b337625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-11T22:43:54.611 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:54 smithi172 podman[74813]: 2024-09-11 22:43:54.400213582 +0000 UTC m=+0.499992953 container attach f4261e4bd283542add317a2aba392badc001c28cbf43a443c2ba453f9b337625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T22:43:54.611 INFO:journalctl@ceph.osd.8.smithi172.stdout:Sep 11 22:43:54 smithi172 podman[74813]: 2024-09-11 22:43:54.50208283 +0000 UTC m=+0.601862194 container died f4261e4bd283542add317a2aba392badc001c28cbf43a443c2ba453f9b337625 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:43:54.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:54 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:54.304+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:54.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:54 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:54.304+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:54.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:54 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:54.304+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:54.612 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:54 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:54.304+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:54.612 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:54 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:54.304+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:43:55.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:54 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:54.787+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:55.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:54 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:54.787+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:55.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:54 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:54.787+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:55.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:54 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:54.787+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:55.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:54 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:54.787+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:55.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:54 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:54.787+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:43:55.169 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.8.service' 2024-09-11T22:43:55.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:43:55.209 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-11T22:43:55.209 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-11T22:43:55.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.9 2024-09-11T22:43:55.237 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:54 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:54.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:55.238 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:54 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:54.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:55.238 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:54 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:54.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:55.238 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:54 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:54.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:55.238 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:54 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:54.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:55.238 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:54 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:54.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:25.686605+0000 front 2024-09-11T22:43:25.686528+0000 (oldest deadline 2024-09-11T22:43:50.386646+0000) 2024-09-11T22:43:55.519 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:55 smithi005 systemd[1]: Stopping Ceph osd.9 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:43:55.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:55 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:55.345+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:55.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:55 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:55.345+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:55.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:55 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:55.345+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:55.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:55 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:55.345+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:55.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:55 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:55.345+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:43:55.820 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:55 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:55.517+0000 7f7d39452640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:43:55.820 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:55 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:55.517+0000 7f7d39452640 -1 osd.9 124 *** Got signal Terminated *** 2024-09-11T22:43:55.820 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:55 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:55.517+0000 7f7d39452640 -1 osd.9 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:43:56.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:55 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:55.800+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:56.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:55 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:55.800+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:56.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:55 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:55.800+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:56.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:55 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:55.800+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:56.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:55 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:55.800+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:56.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:55 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:55.800+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:43:56.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:55 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:55.926+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:56.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:55 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:55.926+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:56.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:55 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:55.926+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:56.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:55 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:55.926+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:56.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:55 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:55.926+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:56.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:55 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:55.926+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:25.686605+0000 front 2024-09-11T22:43:25.686528+0000 (oldest deadline 2024-09-11T22:43:50.386646+0000) 2024-09-11T22:43:56.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:56 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:56.297+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:56.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:56 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:56.297+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:56.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:56 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:56.297+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:56.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:56 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:56.297+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:56.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:56 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:56.297+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:43:57.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:56 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:56.758+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:57.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:56 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:56.758+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:57.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:56 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:56.758+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:57.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:56 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:56.758+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:57.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:56 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:56.758+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:57.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:56 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:56.758+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:43:57.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:56.962+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:57.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:56.962+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:57.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:56.962+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:57.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:56.962+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:57.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:56.962+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:57.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:56 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:56.963+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:25.686605+0000 front 2024-09-11T22:43:25.686528+0000 (oldest deadline 2024-09-11T22:43:50.386646+0000) 2024-09-11T22:43:57.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:57 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:57.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:57.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:57 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:57.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:57.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:57 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:57.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:57.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:57 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:57.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:57.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:57 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:57.248+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:43:58.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:57 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:57.789+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:58.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:57 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:57.789+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:58.079 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:57 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:57.789+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:58.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:57 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:57.789+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:58.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:57 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:57.789+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:58.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:57 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:57.789+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:43:58.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:57.964+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:58.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:57.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:58.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:57.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:58.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:57.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:58.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:57.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:58.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:57 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:57.965+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:25.686605+0000 front 2024-09-11T22:43:25.686528+0000 (oldest deadline 2024-09-11T22:43:50.386646+0000) 2024-09-11T22:43:58.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:58 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:58.204+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:58.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:58 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:58.204+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:58.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:58 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:58.204+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:58.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:58 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:58.204+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:58.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:58 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:58.204+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:43:59.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:58 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:58.953+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:43:59.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:58 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:58.953+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:43:59.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:58 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:58.953+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:43:59.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:58 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:58.953+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:43:59.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:58 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:58.953+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:43:59.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:58 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:58.953+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:25.686605+0000 front 2024-09-11T22:43:25.686528+0000 (oldest deadline 2024-09-11T22:43:50.386646+0000) 2024-09-11T22:43:59.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:58 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:58.835+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:43:59.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:58 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:58.835+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:43:59.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:58 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:58.835+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:43:59.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:58 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:58.835+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:43:59.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:58 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:58.835+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:43:59.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:58 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:58.835+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:43:59.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:58 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:58.835+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:33.681068+0000 front 2024-09-11T22:43:33.681096+0000 (oldest deadline 2024-09-11T22:43:58.380815+0000) 2024-09-11T22:43:59.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:59.156+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:43:59.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:59.156+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:43:59.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:59.156+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:43:59.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:59.156+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:43:59.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:43:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:43:59.156+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:00.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:59 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:59.924+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:47.278607+0000 front 2024-09-11T22:42:47.278633+0000 (oldest deadline 2024-09-11T22:43:11.378980+0000) 2024-09-11T22:44:00.320 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:59 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:59.924+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:54.880390+0000 front 2024-09-11T22:42:54.880301+0000 (oldest deadline 2024-09-11T22:43:20.081223+0000) 2024-09-11T22:44:00.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:59 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:59.924+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:00.081888+0000 front 2024-09-11T22:43:00.081865+0000 (oldest deadline 2024-09-11T22:43:25.981830+0000) 2024-09-11T22:44:00.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:59 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:59.924+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:11.783647+0000 front 2024-09-11T22:43:11.783617+0000 (oldest deadline 2024-09-11T22:43:35.283947+0000) 2024-09-11T22:44:00.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:59 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:59.924+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.386003+0000 front 2024-09-11T22:43:20.385947+0000 (oldest deadline 2024-09-11T22:43:45.685959+0000) 2024-09-11T22:44:00.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:59 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:59.924+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:25.686605+0000 front 2024-09-11T22:43:25.686528+0000 (oldest deadline 2024-09-11T22:43:50.386646+0000) 2024-09-11T22:44:00.321 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:43:59 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9[62317]: 2024-09-11T22:43:59.924+0000 7f7d35a68640 -1 osd.9 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:34.487704+0000 front 2024-09-11T22:43:34.487734+0000 (oldest deadline 2024-09-11T22:43:59.787738+0000) 2024-09-11T22:44:00.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:59 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:59.845+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:44:00.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:59 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:59.845+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:44:00.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:59 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:59.845+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:44:00.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:59 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:59.845+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:44:00.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:59 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:59.845+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:44:00.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:59 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:59.845+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:44:00.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:43:59 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:43:59.845+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:33.681068+0000 front 2024-09-11T22:43:33.681096+0000 (oldest deadline 2024-09-11T22:43:58.380815+0000) 2024-09-11T22:44:00.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:00 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:00.137+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:00.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:00 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:00.137+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:00.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:00 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:00.137+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:00.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:00 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:00.137+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:00.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:00 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:00.137+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:00.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:00 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:00.137+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:01.184 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:44:00 smithi005 podman[132951]: 2024-09-11 22:44:00.893394746 +0000 UTC m=+5.479797098 container died e826e430d0418541f3dfd542f04be3d654eeb21905485a7ba5948149f7a151ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T22:44:01.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:00 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:00.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:44:01.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:00 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:00.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:44:01.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:00 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:00.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:44:01.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:00 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:00.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:44:01.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:00 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:00.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:44:01.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:00 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:00.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:44:01.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:00 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:00.884+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:33.681068+0000 front 2024-09-11T22:43:33.681096+0000 (oldest deadline 2024-09-11T22:43:58.380815+0000) 2024-09-11T22:44:01.408 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:01 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:01.154+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:01.409 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:01 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:01.154+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:01.409 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:01 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:01.154+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:01.409 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:01 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:01.154+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:01.409 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:01 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:01.154+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:01.409 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:01 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:01.154+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:01.570 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:44:01 smithi005 podman[132951]: 2024-09-11 22:44:01.350396678 +0000 UTC m=+5.936799010 container cleanup e826e430d0418541f3dfd542f04be3d654eeb21905485a7ba5948149f7a151ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:44:01.570 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:44:01 smithi005 bash[132951]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9 2024-09-11T22:44:01.975 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:44:01 smithi005 podman[132967]: 2024-09-11 22:44:01.635142271 +0000 UTC m=+0.738151727 container remove e826e430d0418541f3dfd542f04be3d654eeb21905485a7ba5948149f7a151ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T22:44:02.238 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:44:01 smithi005 podman[133054]: 2024-09-11 22:44:01.878408691 +0000 UTC m=+0.016399688 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:44:02.238 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:44:01 smithi005 podman[133054]: 2024-09-11 22:44:01.985986872 +0000 UTC m=+0.123977860 container create 837ec99d7323dd49102b75a5705f4a14df28dcf3eebd930ce2c777df1f9a59f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-11T22:44:02.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:01 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:01.864+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:44:02.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:01 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:01.864+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:44:02.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:01 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:01.864+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:44:02.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:01 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:01.864+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:44:02.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:01 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:01.864+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:44:02.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:01 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:01.864+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:44:02.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:01 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:01.864+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:33.681068+0000 front 2024-09-11T22:43:33.681096+0000 (oldest deadline 2024-09-11T22:43:58.380815+0000) 2024-09-11T22:44:02.570 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:44:02 smithi005 podman[133054]: 2024-09-11 22:44:02.238541864 +0000 UTC m=+0.376532854 container init 837ec99d7323dd49102b75a5705f4a14df28dcf3eebd930ce2c777df1f9a59f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:44:02.571 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:44:02 smithi005 podman[133054]: 2024-09-11 22:44:02.242074713 +0000 UTC m=+0.380065701 container start 837ec99d7323dd49102b75a5705f4a14df28dcf3eebd930ce2c777df1f9a59f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-11T22:44:02.571 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:44:02 smithi005 podman[133054]: 2024-09-11 22:44:02.33028415 +0000 UTC m=+0.468275144 container attach 837ec99d7323dd49102b75a5705f4a14df28dcf3eebd930ce2c777df1f9a59f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T22:44:02.571 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 11 22:44:02 smithi005 podman[133054]: 2024-09-11 22:44:02.432909204 +0000 UTC m=+0.570900192 container died 837ec99d7323dd49102b75a5705f4a14df28dcf3eebd930ce2c777df1f9a59f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T22:44:02.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:02 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:02.159+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:02.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:02 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:02.159+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:02.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:02 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:02.159+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:02.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:02 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:02.159+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:02.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:02 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:02.159+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:02.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:02 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:02.159+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:03.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.9.service' 2024-09-11T22:44:03.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:03.153 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-11T22:44:03.153 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-11T22:44:03.153 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.10 2024-09-11T22:44:03.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:02 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:02.874+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:44:03.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:02 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:02.874+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:44:03.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:02 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:02.874+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:44:03.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:02 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:02.874+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:44:03.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:02 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:02.874+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:44:03.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:02 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:02.874+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:44:03.181 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:02 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:02.874+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:33.681068+0000 front 2024-09-11T22:43:33.681096+0000 (oldest deadline 2024-09-11T22:43:58.380815+0000) 2024-09-11T22:44:03.461 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 systemd[1]: Stopping Ceph osd.10 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:03.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:03 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:03.128+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:03.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:03 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:03.128+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:03.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:03 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:03.128+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:03.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:03 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:03.128+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:03.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:03 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:03.128+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:03.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:03 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:03.128+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:03.829 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:03.460+0000 7f250ca4f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:03.829 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:03.460+0000 7f250ca4f640 -1 osd.10 124 *** Got signal Terminated *** 2024-09-11T22:44:03.830 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:03.460+0000 7f250ca4f640 -1 osd.10 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:44:04.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:03.862+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:44:04.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:03.862+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:44:04.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:03.862+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:44:04.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:03.862+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:44:04.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:03.862+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:44:04.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:03.862+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:44:04.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:03 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:03.862+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:33.681068+0000 front 2024-09-11T22:43:33.681096+0000 (oldest deadline 2024-09-11T22:43:58.380815+0000) 2024-09-11T22:44:04.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:04 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:04.123+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:04.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:04 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:04.123+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:04.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:04 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:04.123+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:04.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:04 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:04.123+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:04.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:04 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:04.123+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:04.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:04 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:04.123+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:05.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:04 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:04.882+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:44:05.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:04 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:04.882+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:44:05.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:04 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:04.882+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:44:05.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:04 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:04.882+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:44:05.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:04 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:04.882+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:44:05.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:04 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:04.882+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:44:05.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:04 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:04.882+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:33.681068+0000 front 2024-09-11T22:43:33.681096+0000 (oldest deadline 2024-09-11T22:43:58.380815+0000) 2024-09-11T22:44:05.511 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:05 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:05.119+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:05.512 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:05 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:05.119+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:05.512 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:05 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:05.119+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:05.512 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:05 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:05.119+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:05.512 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:05 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:05.119+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:05.512 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:05 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:05.119+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:06.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:05 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:05.886+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:44:06.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:05 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:05.886+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:44:06.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:05 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:05.886+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:44:06.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:05 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:05.886+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:44:06.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:05 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:05.886+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:44:06.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:05 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:05.886+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:44:06.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:05 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:05.886+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:33.681068+0000 front 2024-09-11T22:43:33.681096+0000 (oldest deadline 2024-09-11T22:43:58.380815+0000) 2024-09-11T22:44:06.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:06 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:06.111+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:06.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:06 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:06.111+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:06.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:06 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:06.111+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:06.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:06 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:06.111+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:06.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:06 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:06.111+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:06.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:06 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:06.111+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:07.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:06 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:06.852+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:44:07.583 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:06 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:06.852+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:44:07.583 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:06 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:06.852+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:44:07.583 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:06 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:06.852+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:44:07.583 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:06 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:06.852+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:44:07.583 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:06 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:06.852+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:44:07.584 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:06 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:06.852+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:33.681068+0000 front 2024-09-11T22:43:33.681096+0000 (oldest deadline 2024-09-11T22:43:58.380815+0000) 2024-09-11T22:44:07.584 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:07 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:07.100+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:07.584 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:07 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:07.100+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:07.585 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:07 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:07.100+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:07.585 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:07 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:07.100+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:07.585 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:07 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:07.100+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:07.585 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:07 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:07.100+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:08.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:07 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:07.901+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.973152+0000 front 2024-09-11T22:42:46.973205+0000 (oldest deadline 2024-09-11T22:43:12.273225+0000) 2024-09-11T22:44:08.329 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:07 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:07.901+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:52.273828+0000 front 2024-09-11T22:42:52.273892+0000 (oldest deadline 2024-09-11T22:43:18.174075+0000) 2024-09-11T22:44:08.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:07 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:07.901+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:02.775769+0000 front 2024-09-11T22:43:02.775821+0000 (oldest deadline 2024-09-11T22:43:26.875765+0000) 2024-09-11T22:44:08.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:07 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:07.901+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-11T22:43:12.176925+0000 front 2024-09-11T22:43:12.177030+0000 (oldest deadline 2024-09-11T22:43:33.876866+0000) 2024-09-11T22:44:08.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:07 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:07.901+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:19.579208+0000 front 2024-09-11T22:43:19.579223+0000 (oldest deadline 2024-09-11T22:43:44.279351+0000) 2024-09-11T22:44:08.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:07 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:07.901+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:24.280082+0000 front 2024-09-11T22:43:24.280003+0000 (oldest deadline 2024-09-11T22:43:48.979809+0000) 2024-09-11T22:44:08.330 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:07 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10[60341]: 2024-09-11T22:44:07.901+0000 7f2509065640 -1 osd.10 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:33.681068+0000 front 2024-09-11T22:43:33.681096+0000 (oldest deadline 2024-09-11T22:43:58.380815+0000) 2024-09-11T22:44:08.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:08 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:08.078+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:08.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:08 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:08.078+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:08.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:08 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:08.078+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:08.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:08 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:08.078+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:08.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:08 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:08.078+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:08.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:08 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:08.078+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:08.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:08 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:08.078+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-09-11T22:43:42.126048+0000 front 2024-09-11T22:43:42.126113+0000 (oldest deadline 2024-09-11T22:44:07.425832+0000) 2024-09-11T22:44:09.080 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:08 smithi159 podman[74767]: 2024-09-11 22:44:08.815487274 +0000 UTC m=+5.468932659 container died 65829637b94005fc55ea86f81fee33488df1913c593db1e35c9af70d415fed71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-11T22:44:09.395 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:09 smithi159 podman[74767]: 2024-09-11 22:44:09.218607531 +0000 UTC m=+5.872052915 container cleanup 65829637b94005fc55ea86f81fee33488df1913c593db1e35c9af70d415fed71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-11T22:44:09.395 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:09 smithi159 bash[74767]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10 2024-09-11T22:44:09.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:09 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:09.125+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:09.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:09 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:09.125+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:09.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:09 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:09.125+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:09.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:09 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:09.125+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:09.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:09 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:09.125+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:09.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:09 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:09.125+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:09.611 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:09 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:09.125+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-09-11T22:43:42.126048+0000 front 2024-09-11T22:43:42.126113+0000 (oldest deadline 2024-09-11T22:44:07.425832+0000) 2024-09-11T22:44:09.829 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:09 smithi159 podman[74782]: 2024-09-11 22:44:09.394834342 +0000 UTC m=+0.571556404 container remove 65829637b94005fc55ea86f81fee33488df1913c593db1e35c9af70d415fed71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T22:44:10.173 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:09 smithi159 podman[74871]: 2024-09-11 22:44:09.772676045 +0000 UTC m=+0.015884822 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:44:10.173 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:09 smithi159 podman[74871]: 2024-09-11 22:44:09.878942209 +0000 UTC m=+0.122150965 container create e0e2b87aa33df9140c6bb6afe2ca907c3e309adca983aac3cce7686a621d86f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T22:44:10.173 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:10 smithi159 podman[74871]: 2024-09-11 22:44:10.089719198 +0000 UTC m=+0.332927955 container init e0e2b87aa33df9140c6bb6afe2ca907c3e309adca983aac3cce7686a621d86f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:44:10.173 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:10 smithi159 podman[74871]: 2024-09-11 22:44:10.092937399 +0000 UTC m=+0.336146157 container start e0e2b87aa33df9140c6bb6afe2ca907c3e309adca983aac3cce7686a621d86f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10-deactivate, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:44:10.174 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:10 smithi159 podman[74871]: 2024-09-11 22:44:10.172869162 +0000 UTC m=+0.416077930 container attach e0e2b87aa33df9140c6bb6afe2ca907c3e309adca983aac3cce7686a621d86f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T22:44:10.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:10.107+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:10.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:10.107+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:10.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:10.107+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:10.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:10.107+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:10.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:10.107+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:10.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:10.107+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:10.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:10 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:10.107+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-09-11T22:43:42.126048+0000 front 2024-09-11T22:43:42.126113+0000 (oldest deadline 2024-09-11T22:44:07.425832+0000) 2024-09-11T22:44:10.578 INFO:journalctl@ceph.osd.10.smithi159.stdout:Sep 11 22:44:10 smithi159 podman[74871]: 2024-09-11 22:44:10.275956803 +0000 UTC m=+0.519165567 container died e0e2b87aa33df9140c6bb6afe2ca907c3e309adca983aac3cce7686a621d86f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:44:10.877 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.10.service' 2024-09-11T22:44:10.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:10.959 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-11T22:44:10.959 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-11T22:44:10.959 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.11 2024-09-11T22:44:11.256 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 systemd[1]: Stopping Ceph osd.11 for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:11.257 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:11.067+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:11.257 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:11.067+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:11.257 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:11.067+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:11.257 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:11.067+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:11.257 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:11.067+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:11.257 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:11.067+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:11.257 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:11.067+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-09-11T22:43:42.126048+0000 front 2024-09-11T22:43:42.126113+0000 (oldest deadline 2024-09-11T22:44:07.425832+0000) 2024-09-11T22:44:11.258 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:11.255+0000 7f0f3536c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:11.258 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:11.255+0000 7f0f3536c640 -1 osd.11 124 *** Got signal Terminated *** 2024-09-11T22:44:11.258 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:11 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:11.255+0000 7f0f3536c640 -1 osd.11 124 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T22:44:12.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:12.109+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:12.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:12.109+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:12.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:12.109+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:12.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:12.109+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:12.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:12.109+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:12.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:12.109+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:12.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:12 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:12.109+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-09-11T22:43:42.126048+0000 front 2024-09-11T22:43:42.126113+0000 (oldest deadline 2024-09-11T22:44:07.425832+0000) 2024-09-11T22:44:13.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:13.060+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:13.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:13.060+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:13.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:13.060+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:13.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:13.060+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:13.362 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:13.060+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:13.362 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:13.060+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:13.362 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:13 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:13.060+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-09-11T22:43:42.126048+0000 front 2024-09-11T22:43:42.126113+0000 (oldest deadline 2024-09-11T22:44:07.425832+0000) 2024-09-11T22:44:14.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.034+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:14.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.034+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:14.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.034+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:14.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.034+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:14.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.034+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:14.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.034+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:14.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.034+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-09-11T22:43:42.126048+0000 front 2024-09-11T22:43:42.126113+0000 (oldest deadline 2024-09-11T22:44:07.425832+0000) 2024-09-11T22:44:14.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.034+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6820 osd.8 since back 2024-09-11T22:43:47.426237+0000 front 2024-09-11T22:43:47.426201+0000 (oldest deadline 2024-09-11T22:44:13.326258+0000) 2024-09-11T22:44:15.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.996+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:15.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.996+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:15.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.996+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:15.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.996+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:15.362 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.996+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:15.362 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.996+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:15.362 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.996+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-09-11T22:43:42.126048+0000 front 2024-09-11T22:43:42.126113+0000 (oldest deadline 2024-09-11T22:44:07.425832+0000) 2024-09-11T22:44:15.362 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:14 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:14.996+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6820 osd.8 since back 2024-09-11T22:43:47.426237+0000 front 2024-09-11T22:43:47.426201+0000 (oldest deadline 2024-09-11T22:44:13.326258+0000) 2024-09-11T22:44:16.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:16.022+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-11T22:42:46.614324+0000 front 2024-09-11T22:42:46.614197+0000 (oldest deadline 2024-09-11T22:43:10.714244+0000) 2024-09-11T22:44:16.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:16.022+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6804 osd.1 since back 2024-09-11T22:42:50.714783+0000 front 2024-09-11T22:42:50.714762+0000 (oldest deadline 2024-09-11T22:43:16.614954+0000) 2024-09-11T22:44:16.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:16.022+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6804 osd.2 since back 2024-09-11T22:43:03.016444+0000 front 2024-09-11T22:43:03.016468+0000 (oldest deadline 2024-09-11T22:43:27.716628+0000) 2024-09-11T22:44:16.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:16.022+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6812 osd.4 since back 2024-09-11T22:43:20.519682+0000 front 2024-09-11T22:43:20.519700+0000 (oldest deadline 2024-09-11T22:43:41.619681+0000) 2024-09-11T22:44:16.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:16.022+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6812 osd.5 since back 2024-09-11T22:43:27.222039+0000 front 2024-09-11T22:43:27.222071+0000 (oldest deadline 2024-09-11T22:43:50.722230+0000) 2024-09-11T22:44:16.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:16.022+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-11T22:43:35.824312+0000 front 2024-09-11T22:43:35.824363+0000 (oldest deadline 2024-09-11T22:43:59.924296+0000) 2024-09-11T22:44:16.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:16.022+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.159:6820 osd.7 since back 2024-09-11T22:43:42.126048+0000 front 2024-09-11T22:43:42.126113+0000 (oldest deadline 2024-09-11T22:44:07.425832+0000) 2024-09-11T22:44:16.361 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:16 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11[60828]: 2024-09-11T22:44:16.022+0000 7f0f31982640 -1 osd.11 124 heartbeat_check: no reply from 172.21.15.172:6820 osd.8 since back 2024-09-11T22:43:47.426237+0000 front 2024-09-11T22:43:47.426201+0000 (oldest deadline 2024-09-11T22:44:13.326258+0000) 2024-09-11T22:44:17.064 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:16 smithi172 podman[74930]: 2024-09-11 22:44:16.715176842 +0000 UTC m=+5.561634400 container died d404a7ea46a4889c93297795eb0ff2e21822ea2e4b383b6a6acbe2fe880a09b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T22:44:17.357 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:17 smithi172 podman[74930]: 2024-09-11 22:44:17.163773089 +0000 UTC m=+6.010230688 container cleanup d404a7ea46a4889c93297795eb0ff2e21822ea2e4b383b6a6acbe2fe880a09b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T22:44:17.357 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:17 smithi172 bash[74930]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11 2024-09-11T22:44:17.610 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:17 smithi172 podman[74947]: 2024-09-11 22:44:17.356914735 +0000 UTC m=+0.636367251 container remove d404a7ea46a4889c93297795eb0ff2e21822ea2e4b383b6a6acbe2fe880a09b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:44:18.101 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:17 smithi172 podman[75035]: 2024-09-11 22:44:17.678633593 +0000 UTC m=+0.016053515 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-11T22:44:18.102 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:17 smithi172 podman[75035]: 2024-09-11 22:44:17.782490096 +0000 UTC m=+0.119910009 container create 101dc71427531a72591ad9c5b18f433ef540b3f513b87cfc0a7f593203cfcccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-11T22:44:18.102 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:18 smithi172 podman[75035]: 2024-09-11 22:44:18.018043103 +0000 UTC m=+0.355463023 container init 101dc71427531a72591ad9c5b18f433ef540b3f513b87cfc0a7f593203cfcccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T22:44:18.102 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:18 smithi172 podman[75035]: 2024-09-11 22:44:18.021538626 +0000 UTC m=+0.358958541 container start 101dc71427531a72591ad9c5b18f433ef540b3f513b87cfc0a7f593203cfcccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-11T22:44:18.102 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:18 smithi172 podman[75035]: 2024-09-11 22:44:18.10124998 +0000 UTC m=+0.438669905 container attach 101dc71427531a72591ad9c5b18f433ef540b3f513b87cfc0a7f593203cfcccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-11T22:44:18.360 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:18 smithi172 podman[75035]: 2024-09-11 22:44:18.217239837 +0000 UTC m=+0.554659754 container died 101dc71427531a72591ad9c5b18f433ef540b3f513b87cfc0a7f593203cfcccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-11T22:44:18.823 INFO:journalctl@ceph.osd.11.smithi172.stdout:Sep 11 22:44:18 smithi172 podman[75058]: 2024-09-11 22:44:18.820925516 +0000 UTC m=+0.597800792 container remove 101dc71427531a72591ad9c5b18f433ef540b3f513b87cfc0a7f593203cfcccc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T22:44:18.842 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@osd.11.service' 2024-09-11T22:44:18.882 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:18.882 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-11T22:44:18.882 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-11T22:44:18.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.a 2024-09-11T22:44:19.186 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:44:18 smithi005 systemd[1]: Stopping Ceph mds.a for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:19.570 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:44:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a[69478]: 2024-09-11T22:44:19.184+0000 7f699c37f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:19.570 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:44:19 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a[69478]: 2024-09-11T22:44:19.184+0000 7f699c37f640 -1 mds.a *** got signal Terminated *** 2024-09-11T22:44:22.994 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:44:22 smithi005 podman[133157]: 2024-09-11 22:44:22.705298217 +0000 UTC m=+3.624896711 container died 539c65355df609f69253d53def6837ede4fb066b930ffd3f2ff38f2df11c6750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T22:44:23.544 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:44:23 smithi005 podman[133157]: 2024-09-11 22:44:23.293063798 +0000 UTC m=+4.212662293 container cleanup 539c65355df609f69253d53def6837ede4fb066b930ffd3f2ff38f2df11c6750 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-11T22:44:23.545 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 11 22:44:23 smithi005 bash[133157]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-a 2024-09-11T22:44:23.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.a.service' 2024-09-11T22:44:23.717 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:23.718 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-11T22:44:23.718 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-11T22:44:23.718 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.d 2024-09-11T22:44:24.070 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:44:23 smithi005 systemd[1]: Stopping Ceph mds.d for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:24.570 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:44:24 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-d[68179]: 2024-09-11T22:44:24.071+0000 7f1a71df6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:24.570 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:44:24 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-d[68179]: 2024-09-11T22:44:24.071+0000 7f1a71df6640 -1 mds.d *** got signal Terminated *** 2024-09-11T22:44:26.131 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:44:25 smithi005 podman[133277]: 2024-09-11 22:44:25.847656933 +0000 UTC m=+1.898052598 container died 816ef0aabc16e895a02e8b5edb0795eee92071ed5167923c772d893be98c2119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T22:44:26.473 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:44:26 smithi005 podman[133277]: 2024-09-11 22:44:26.296915211 +0000 UTC m=+2.347310876 container cleanup 816ef0aabc16e895a02e8b5edb0795eee92071ed5167923c772d893be98c2119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-11T22:44:26.474 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:44:26 smithi005 bash[133277]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-d 2024-09-11T22:44:26.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.d.service' 2024-09-11T22:44:26.731 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:44:26 smithi005 podman[133289]: 2024-09-11 22:44:26.473277215 +0000 UTC m=+0.618898313 container remove 816ef0aabc16e895a02e8b5edb0795eee92071ed5167923c772d893be98c2119 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:44:26.731 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:44:26 smithi005 systemd[1]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-11T22:44:26.731 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:44:26 smithi005 systemd[1]: Stopped Ceph mds.d for b99b16b2-708a-11ef-bcea-c7b262605968. 2024-09-11T22:44:26.731 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 11 22:44:26 smithi005 systemd[1]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.d.service: Consumed 1.171s CPU time. 2024-09-11T22:44:26.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:26.742 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-11T22:44:26.743 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-11T22:44:26.743 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.g 2024-09-11T22:44:27.068 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:44:26 smithi005 systemd[1]: Stopping Ceph mds.g for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:27.320 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:44:27 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-g[69058]: 2024-09-11T22:44:27.067+0000 7f61baa70640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:27.320 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:44:27 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-g[69058]: 2024-09-11T22:44:27.067+0000 7f61baa70640 -1 mds.g *** got signal Terminated *** 2024-09-11T22:44:30.994 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:44:30 smithi005 podman[133397]: 2024-09-11 22:44:30.712442016 +0000 UTC m=+3.746800904 container died 5fd3bb2fb78d330c822ba0455fdbe236c0f48b006cd1c8cf265134e7ca2aef21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T22:44:31.320 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:44:31 smithi005 podman[133397]: 2024-09-11 22:44:31.167548438 +0000 UTC m=+4.201907324 container cleanup 5fd3bb2fb78d330c822ba0455fdbe236c0f48b006cd1c8cf265134e7ca2aef21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-11T22:44:31.320 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 11 22:44:31 smithi005 bash[133397]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-g 2024-09-11T22:44:31.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.g.service' 2024-09-11T22:44:31.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:31.639 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-11T22:44:31.639 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-11T22:44:31.640 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.j 2024-09-11T22:44:31.998 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:44:31 smithi005 systemd[1]: Stopping Ceph mds.j for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:32.320 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:44:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-j[67771]: 2024-09-11T22:44:31.997+0000 7ff4a423e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:32.320 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:44:31 smithi005 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-j[67771]: 2024-09-11T22:44:31.997+0000 7ff4a423e640 -1 mds.j *** got signal Terminated *** 2024-09-11T22:44:34.481 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:44:34 smithi005 podman[133516]: 2024-09-11 22:44:34.213526821 +0000 UTC m=+2.343937474 container died fe9c99224f8332e91febe5f727b81fa1085815ef25f374667c8eddb5168af80d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-11T22:44:34.820 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:44:34 smithi005 podman[133516]: 2024-09-11 22:44:34.671282241 +0000 UTC m=+2.801692892 container cleanup fe9c99224f8332e91febe5f727b81fa1085815ef25f374667c8eddb5168af80d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-11T22:44:34.820 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 11 22:44:34 smithi005 bash[133516]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-j 2024-09-11T22:44:35.109 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.j.service' 2024-09-11T22:44:35.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:35.142 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-11T22:44:35.142 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-11T22:44:35.143 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.b 2024-09-11T22:44:35.438 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:44:35 smithi159 systemd[1]: Stopping Ceph mds.b for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:35.829 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:44:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:44:35.436+0000 7fbd69bf7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:35.829 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:44:35 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b[65961]: 2024-09-11T22:44:35.436+0000 7fbd69bf7640 -1 mds.b *** got signal Terminated *** 2024-09-11T22:44:39.162 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:44:38 smithi159 podman[74986]: 2024-09-11 22:44:38.835665857 +0000 UTC m=+3.497008114 container died 8d154520033b68ab5a20962a8765bf187b08305f10cbfe2cd4fe25dee7f1caad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-11T22:44:39.162 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:44:39 smithi159 podman[74986]: 2024-09-11 22:44:39.162065888 +0000 UTC m=+3.823408103 container cleanup 8d154520033b68ab5a20962a8765bf187b08305f10cbfe2cd4fe25dee7f1caad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T22:44:39.579 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:44:39 smithi159 bash[74986]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b 2024-09-11T22:44:39.579 INFO:journalctl@ceph.mds.b.smithi159.stdout:Sep 11 22:44:39 smithi159 podman[74998]: 2024-09-11 22:44:39.397155398 +0000 UTC m=+0.547552293 container remove 8d154520033b68ab5a20962a8765bf187b08305f10cbfe2cd4fe25dee7f1caad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T22:44:39.617 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.b.service' 2024-09-11T22:44:39.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:39.651 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-11T22:44:39.651 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-11T22:44:39.652 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.e 2024-09-11T22:44:39.967 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:44:39 smithi159 systemd[1]: Stopping Ceph mds.e for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:40.329 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:44:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e[65346]: 2024-09-11T22:44:39.965+0000 7f2355691640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:40.329 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:44:39 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e[65346]: 2024-09-11T22:44:39.965+0000 7f2355691640 -1 mds.e *** got signal Terminated *** 2024-09-11T22:44:43.079 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:44:42 smithi159 podman[75109]: 2024-09-11 22:44:42.829113463 +0000 UTC m=+2.961706072 container died da2b9fe291c225c4aceb11433592cbafdfd1f16877d4691cad0453f1a02c1562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-11T22:44:43.459 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:44:43 smithi159 podman[75109]: 2024-09-11 22:44:43.207870606 +0000 UTC m=+3.340463200 container cleanup da2b9fe291c225c4aceb11433592cbafdfd1f16877d4691cad0453f1a02c1562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T22:44:43.459 INFO:journalctl@ceph.mds.e.smithi159.stdout:Sep 11 22:44:43 smithi159 bash[75109]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-e 2024-09-11T22:44:43.632 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.e.service' 2024-09-11T22:44:43.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:43.669 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-11T22:44:43.669 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-11T22:44:43.669 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.h 2024-09-11T22:44:44.079 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:44:43 smithi159 systemd[1]: Stopping Ceph mds.h for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:44.079 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:44:43 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h[65647]: 2024-09-11T22:44:43.977+0000 7efc9e5b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:44.080 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:44:43 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h[65647]: 2024-09-11T22:44:43.977+0000 7efc9e5b0640 -1 mds.h *** got signal Terminated *** 2024-09-11T22:44:47.087 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:44:46 smithi159 podman[75242]: 2024-09-11 22:44:46.835926448 +0000 UTC m=+2.951408888 container died 5e3191587899464e967e540688d3ba7d1baca144eafd5362f50ed72355c3b52d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T22:44:47.496 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:44:47 smithi159 podman[75242]: 2024-09-11 22:44:47.204035327 +0000 UTC m=+3.319517767 container cleanup 5e3191587899464e967e540688d3ba7d1baca144eafd5362f50ed72355c3b52d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:44:47.496 INFO:journalctl@ceph.mds.h.smithi159.stdout:Sep 11 22:44:47 smithi159 bash[75242]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-h 2024-09-11T22:44:47.629 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.h.service' 2024-09-11T22:44:47.662 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:47.662 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-11T22:44:47.662 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-11T22:44:47.663 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.k 2024-09-11T22:44:47.829 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:44:47 smithi159 systemd[1]: Stopping Ceph mds.k for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:48.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:44:47 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: 2024-09-11T22:44:47.972+0000 7f0e0ea01640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:48.329 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:44:47 smithi159 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k[65030]: 2024-09-11T22:44:47.972+0000 7f0e0ea01640 -1 mds.k *** got signal Terminated *** 2024-09-11T22:44:51.134 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:44:50 smithi159 podman[75364]: 2024-09-11 22:44:50.831406001 +0000 UTC m=+2.949791054 container died 114d39de8ee6b497757e67c80715108f314c307f80f68204e13a64770afce78c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T22:44:51.466 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:44:51 smithi159 podman[75364]: 2024-09-11 22:44:51.257005888 +0000 UTC m=+3.375390938 container cleanup 114d39de8ee6b497757e67c80715108f314c307f80f68204e13a64770afce78c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T22:44:51.467 INFO:journalctl@ceph.mds.k.smithi159.stdout:Sep 11 22:44:51 smithi159 bash[75364]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-k 2024-09-11T22:44:51.687 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.k.service' 2024-09-11T22:44:51.719 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:51.720 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-11T22:44:51.720 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-11T22:44:51.720 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.c 2024-09-11T22:44:52.012 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:44:51 smithi172 systemd[1]: Stopping Ceph mds.c for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:52.267 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:44:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c[66081]: 2024-09-11T22:44:52.011+0000 7f000b1af640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:52.267 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:44:52 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c[66081]: 2024-09-11T22:44:52.011+0000 7f000b1af640 -1 mds.c *** got signal Terminated *** 2024-09-11T22:44:55.054 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:44:54 smithi172 podman[75136]: 2024-09-11 22:44:54.723839942 +0000 UTC m=+2.815652948 container died ee25c5c264a2a21f76f5d8d259723819c3846848dcf764f358058d2fb0365aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T22:44:55.360 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:44:55 smithi172 podman[75136]: 2024-09-11 22:44:55.161060457 +0000 UTC m=+3.252873457 container cleanup ee25c5c264a2a21f76f5d8d259723819c3846848dcf764f358058d2fb0365aca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T22:44:55.360 INFO:journalctl@ceph.mds.c.smithi172.stdout:Sep 11 22:44:55 smithi172 bash[75136]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-c 2024-09-11T22:44:55.583 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.c.service' 2024-09-11T22:44:55.619 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:55.619 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-11T22:44:55.619 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-11T22:44:55.619 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.f 2024-09-11T22:44:55.940 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:44:55 smithi172 systemd[1]: Stopping Ceph mds.f for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:56.360 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:44:55 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-f[65778]: 2024-09-11T22:44:55.939+0000 7f099481f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:56.360 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:44:55 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-f[65778]: 2024-09-11T22:44:55.939+0000 7f099481f640 -1 mds.f *** got signal Terminated *** 2024-09-11T22:44:59.015 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:44:58 smithi172 podman[75270]: 2024-09-11 22:44:58.716899555 +0000 UTC m=+2.877736492 container died e7e643923079b1d55494536cba22dcdaa5cbbfe0b7729e86b9e0e3ddaead4d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-11T22:44:59.349 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:44:59 smithi172 podman[75270]: 2024-09-11 22:44:59.114478823 +0000 UTC m=+3.275315763 container cleanup e7e643923079b1d55494536cba22dcdaa5cbbfe0b7729e86b9e0e3ddaead4d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T22:44:59.349 INFO:journalctl@ceph.mds.f.smithi172.stdout:Sep 11 22:44:59 smithi172 bash[75270]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-f 2024-09-11T22:44:59.481 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.f.service' 2024-09-11T22:44:59.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:44:59.515 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-11T22:44:59.515 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-11T22:44:59.515 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.i 2024-09-11T22:44:59.860 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:44:59 smithi172 systemd[1]: Stopping Ceph mds.i for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:44:59.860 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:44:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i[66381]: 2024-09-11T22:44:59.835+0000 7f0dd2b05640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:44:59.861 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:44:59 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i[66381]: 2024-09-11T22:44:59.835+0000 7f0dd2b05640 -1 mds.i *** got signal Terminated *** 2024-09-11T22:45:03.068 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:45:02 smithi172 podman[75391]: 2024-09-11 22:45:02.724487784 +0000 UTC m=+2.987095272 container died 98668205922bcfaed6877e51b1a293d7123e616b17b81764a64a8e88d8423cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-11T22:45:03.360 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:45:03 smithi172 podman[75391]: 2024-09-11 22:45:03.176434451 +0000 UTC m=+3.439041928 container cleanup 98668205922bcfaed6877e51b1a293d7123e616b17b81764a64a8e88d8423cbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T22:45:03.360 INFO:journalctl@ceph.mds.i.smithi172.stdout:Sep 11 22:45:03 smithi172 bash[75391]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-i 2024-09-11T22:45:03.630 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.i.service' 2024-09-11T22:45:03.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:45:03.663 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-11T22:45:03.663 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-11T22:45:03.664 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.l 2024-09-11T22:45:03.860 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:45:03 smithi172 systemd[1]: Stopping Ceph mds.l for b99b16b2-708a-11ef-bcea-c7b262605968... 2024-09-11T22:45:04.360 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:45:03 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l[65473]: 2024-09-11T22:45:03.980+0000 7f882a4b2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T22:45:04.360 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:45:03 smithi172 ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l[65473]: 2024-09-11T22:45:03.980+0000 7f882a4b2640 -1 mds.l *** got signal Terminated *** 2024-09-11T22:45:06.980 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:45:06 smithi172 podman[75512]: 2024-09-11 22:45:06.724755225 +0000 UTC m=+2.846226848 container died 8e270a63289f4365aecd50fea679e2a0f0da5ec696acb623f7b38f469413bf65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T22:45:07.356 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:45:07 smithi172 podman[75512]: 2024-09-11 22:45:07.080065323 +0000 UTC m=+3.201536948 container cleanup 8e270a63289f4365aecd50fea679e2a0f0da5ec696acb623f7b38f469413bf65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-11T22:45:07.357 INFO:journalctl@ceph.mds.l.smithi172.stdout:Sep 11 22:45:07 smithi172 bash[75512]: ceph-b99b16b2-708a-11ef-bcea-c7b262605968-mds-l 2024-09-11T22:45:07.503 DEBUG:teuthology.orchestra.run.smithi172:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b99b16b2-708a-11ef-bcea-c7b262605968@mds.l.service' 2024-09-11T22:45:07.535 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T22:45:07.536 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-11T22:45:07.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b99b16b2-708a-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T22:45:07.749 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:45:10.199 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b99b16b2-708a-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T22:45:10.404 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:45:12.938 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b99b16b2-708a-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T22:45:13.141 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T22:45:15.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:45:15.518 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:45:15.551 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T22:45:15.581 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T22:45:15.583 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415/remote/smithi005/crash 2024-09-11T22:45:15.584 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/crash -- . 2024-09-11T22:45:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T22:45:15.622 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2024-09-11T22:45:15.624 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415/remote/smithi159/crash 2024-09-11T22:45:15.625 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/crash -- . 2024-09-11T22:45:15.668 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T22:45:15.668 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2024-09-11T22:45:15.670 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415/remote/smithi172/crash 2024-09-11T22:45:15.671 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/crash -- . 2024-09-11T22:45:15.708 INFO:teuthology.orchestra.run.smithi172.stderr:tar: /var/lib/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T22:45:15.709 INFO:teuthology.orchestra.run.smithi172.stderr:tar: Error is not recoverable: exiting now 2024-09-11T22:45:15.710 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T22:45:15.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-11T22:45:15.756 INFO:teuthology.orchestra.run.smithi005.stdout:2024-09-11T22:26:28.818825+0000 mds.b (mds.0) 29 : cluster [WRN] Scrub error on inode 0x10000000221 (/volumes/qa/sv_1/b91a8a25-f780-4c42-95ee-94d1e2b1b5e3/client.0/tmp/clients/client0/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-09-11T22:45:15.757 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-11T22:45:15.757 INFO:tasks.cephadm:Compressing logs... 2024-09-11T22:45:15.757 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T22:45:15.760 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T22:45:15.763 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T22:45:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T22:45:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:: No such file or directory 2024-09-11T22:45:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-11T22:45:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T22:45:15.786 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T22:45:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-11T22:45:15.787 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T22:45:15.788 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T22:45:15.789 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-11T22:45:15.790 INFO:teuthology.orchestra.run.smithi005.stderr: 84.5% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T22:45:15.793 INFO:teuthology.orchestra.run.smithi172.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T22:45:15.793 INFO:teuthology.orchestra.run.smithi172.stderr:: No such file or directory 2024-09-11T22:45:15.793 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T22:45:15.793 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-11T22:45:15.793 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T22:45:15.793 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T22:45:15.794 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-11T22:45:15.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T22:45:15.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip 82.1% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T22:45:15.794 INFO:teuthology.orchestra.run.smithi172.stderr: -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T22:45:15.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.log 2024-09-11T22:45:15.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-11T22:45:15.794 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T22:45:15.795 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-11T22:45:15.796 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.l.log: 87.1% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T22:45:15.796 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-11T22:45:15.796 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mon.a.log: 90.2% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T22:45:15.796 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T22:45:15.797 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.log 2024-09-11T22:45:15.801 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T22:45:15.802 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.8% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T22:45:15.802 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-11T22:45:15.805 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’: No such file or directorygzip 2024-09-11T22:45:15.805 INFO:teuthology.orchestra.run.smithi159.stderr: -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-11T22:45:15.805 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T22:45:15.807 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T22:45:15.807 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T22:45:15.808 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-11T22:45:15.808 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-11T22:45:15.808 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T22:45:15.808 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-11T22:45:15.808 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-11T22:45:15.811 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-11T22:45:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.log: 87.1% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T22:45:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T22:45:15.812 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T22:45:15.813 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.log 2024-09-11T22:45:15.813 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-11T22:45:15.813 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T22:45:15.813 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-11T22:45:15.813 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T22:45:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-11T22:45:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-11T22:45:15.820 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mgr.z.log: 92.4% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-11T22:45:15.822 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-11T22:45:15.826 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T22:45:15.834 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T22:45:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T22:45:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.3.log: 94.6% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-11T22:45:15.849 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T22:45:15.881 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T22:45:15.966 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T22:45:16.007 INFO:teuthology.orchestra.run.smithi159.stderr: 82.2% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-11T22:45:16.056 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-11T22:45:16.069 INFO:teuthology.orchestra.run.smithi172.stderr: 94.3% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-11T22:45:16.071 INFO:teuthology.orchestra.run.smithi159.stderr: 90.4% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-11T22:45:16.074 INFO:teuthology.orchestra.run.smithi159.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T22:45:16.075 INFO:teuthology.orchestra.run.smithi159.stderr: 91.0% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T22:45:16.371 INFO:teuthology.orchestra.run.smithi005.stderr: 90.2% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T22:45:17.182 INFO:teuthology.orchestra.run.smithi159.stderr: 87.1% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-11T22:45:17.473 INFO:teuthology.orchestra.run.smithi172.stderr: 95.1% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-11T22:45:17.475 INFO:teuthology.orchestra.run.smithi172.stderr: 92.9% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T22:45:17.636 INFO:teuthology.orchestra.run.smithi005.stderr: 90.2% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T22:45:18.844 INFO:teuthology.orchestra.run.smithi005.stderr: 95.5% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-11T22:45:27.839 INFO:teuthology.orchestra.run.smithi172.stderr: 95.6% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-11T22:45:35.199 INFO:teuthology.orchestra.run.smithi159.stderr: 91.6% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T22:45:37.230 INFO:teuthology.orchestra.run.smithi005.stderr: 91.4% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-11T22:45:41.688 INFO:teuthology.orchestra.run.smithi005.stderr: 91.3% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T22:45:42.785 INFO:teuthology.orchestra.run.smithi005.stderr: 91.6% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T22:45:43.073 INFO:teuthology.orchestra.run.smithi172.stderr: 91.2% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-11T22:45:44.301 INFO:teuthology.orchestra.run.smithi172.stderr: 91.2% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-11T22:45:44.702 INFO:teuthology.orchestra.run.smithi005.stderr: 91.4% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T22:45:44.705 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-11T22:45:44.705 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m28.932s 2024-09-11T22:45:44.705 INFO:teuthology.orchestra.run.smithi005.stderr:user 1m45.404s 2024-09-11T22:45:44.705 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m3.151s 2024-09-11T22:45:50.258 INFO:teuthology.orchestra.run.smithi172.stderr: 91.6% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T22:45:51.393 INFO:teuthology.orchestra.run.smithi172.stderr: 91.1% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T22:46:16.440 INFO:teuthology.orchestra.run.smithi172.stderr: 93.8% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-11T22:46:16.441 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-09-11T22:46:16.441 INFO:teuthology.orchestra.run.smithi172.stderr:real 1m0.663s 2024-09-11T22:46:16.442 INFO:teuthology.orchestra.run.smithi172.stderr:user 3m15.589s 2024-09-11T22:46:16.442 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m5.935s 2024-09-11T22:50:01.036 INFO:teuthology.orchestra.run.smithi159.stderr: 95.7% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-11T22:50:23.443 INFO:teuthology.orchestra.run.smithi159.stderr: 94.9% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-11T22:50:26.977 INFO:teuthology.orchestra.run.smithi159.stderr: 91.3% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T22:51:24.848 INFO:teuthology.orchestra.run.smithi159.stderr: 91.6% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T22:51:58.496 INFO:teuthology.orchestra.run.smithi159.stderr: 91.3% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-11T22:53:25.112 INFO:teuthology.orchestra.run.smithi159.stderr: 91.4% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T22:54:26.464 INFO:teuthology.orchestra.run.smithi159.stderr: 94.6% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-11T22:59:43.966 INFO:teuthology.orchestra.run.smithi159.stderr: 93.4% -- replaced with /var/log/ceph/b99b16b2-708a-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-11T22:59:44.009 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-11T22:59:44.010 INFO:teuthology.orchestra.run.smithi159.stderr:real 14m28.231s 2024-09-11T22:59:44.010 INFO:teuthology.orchestra.run.smithi159.stderr:user 7m48.365s 2024-09-11T22:59:44.010 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m21.811s 2024-09-11T22:59:44.011 INFO:tasks.cephadm:Archiving logs... 2024-09-11T22:59:44.012 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415/remote/smithi005/log 2024-09-11T22:59:44.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:00:01.165 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415/remote/smithi159/log 2024-09-11T23:00:01.166 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:01:18.049 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415/remote/smithi172/log 2024-09-11T23:01:18.050 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T23:01:46.494 INFO:tasks.cephadm:Removing cluster... 2024-09-11T23:01:46.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b99b16b2-708a-11ef-bcea-c7b262605968 --force 2024-09-11T23:01:46.711 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T23:01:47.143 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b99b16b2-708a-11ef-bcea-c7b262605968 --force 2024-09-11T23:01:47.661 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T23:01:48.367 DEBUG:teuthology.orchestra.run.smithi172:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b99b16b2-708a-11ef-bcea-c7b262605968 --force 2024-09-11T23:01:48.571 INFO:teuthology.orchestra.run.smithi172.stdout:Deleting cluster with fsid: b99b16b2-708a-11ef-bcea-c7b262605968 2024-09-11T23:01:49.070 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T23:01:49.070 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:01:49.091 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:01:49.108 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T23:01:49.125 INFO:tasks.cephadm:Teardown complete 2024-09-11T23:01:49.126 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T23:01:49.137 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T23:01:49.137 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:01:49.140 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:01:49.151 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T23:01:49.289 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T23:01:49.301 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T23:01:49.301 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:01:49.304 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:01:49.307 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T23:01:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:01:49.320 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:01:49.321 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-09-11T23:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-09-11T23:01:49.324 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +1175us[+1183us] +/- 42ms 2024-09-11T23:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -1196us[-1196us] +/- 81ms 2024-09-11T23:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -1176us[-1168us] +/- 86ms 2024-09-11T23:01:49.325 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:01:49.326 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:01:49.326 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-09-11T23:01:49.326 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +1185us[+1199us] +/- 42ms 2024-09-11T23:01:49.326 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -1194us[-1194us] +/- 81ms 2024-09-11T23:01:49.327 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 -1194us[-1194us] +/- 86ms 2024-09-11T23:01:49.327 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:01:49.400 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T23:01:49.401 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-09-11T23:01:49.401 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv01.front.sepia.ceph.com 3 7 377 114 +1195us[+1182us] +/- 41ms 2024-09-11T23:01:49.401 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -1200us[-1200us] +/- 81ms 2024-09-11T23:01:49.401 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -1210us[-1210us] +/- 86ms 2024-09-11T23:01:49.401 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T23:01:49.417 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T23:01:49.438 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T23:01:49.438 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T23:01:49.462 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:01:49.561 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:01:49.675 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T23:01:49.746 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:01:49.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:01:49.779 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:01:49.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:01:49.809 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-11T23:01:49.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:01:49.839 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T23:01:49.859 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T23:01:49.894 INFO:teuthology.task.internal:Duration was 3401.235492 seconds 2024-09-11T23:01:49.895 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T23:01:49.949 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T23:01:49.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:01:49.953 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:01:49.955 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T23:01:49.995 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:01:49.995 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:01:50.012 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T23:01:50.402 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T23:01:50.402 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-09-11T23:01:50.402 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:01:50.426 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2024-09-11T23:01:50.426 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:01:50.457 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-09-11T23:01:50.458 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T23:01:50.490 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T23:01:50.491 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:01:50.493 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:01:50.500 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T23:01:50.558 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T23:01:50.558 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:01:50.563 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:01:50.567 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T23:01:52.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T23:01:52.439 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T23:01:52.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:01:52.465 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:01:52.499 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T23:01:52.529 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T23:01:52.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:01:52.546 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:01:52.549 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:01:52.566 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-09-11T23:01:52.586 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2024-09-11T23:01:52.597 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-09-11T23:01:52.818 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:01:52.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:01:52.834 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:01:52.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:01:52.854 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T23:01:52.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T23:01:52.878 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T23:01:52.889 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T23:01:52.895 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415/remote/smithi005 2024-09-11T23:01:52.895 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:01:52.975 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415/remote/smithi159 2024-09-11T23:01:52.976 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:01:53.017 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901415/remote/smithi172 2024-09-11T23:01:53.018 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T23:01:53.061 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T23:01:53.061 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:01:53.064 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:01:53.066 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T23:01:53.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T23:01:53.130 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T23:01:53.131 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T23:01:53.153 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T23:01:53.153 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:01:53.156 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:01:53.159 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T23:01:53.169 INFO:teuthology.orchestra.run.smithi005.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:01 /home/ubuntu/cephtest 2024-09-11T23:01:53.174 INFO:teuthology.orchestra.run.smithi159.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:01 /home/ubuntu/cephtest 2024-09-11T23:01:53.176 INFO:teuthology.orchestra.run.smithi172.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 23:01 /home/ubuntu/cephtest 2024-09-11T23:01:53.177 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T23:01:53.203 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T23:01:53.309 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 3401.2354917526245 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi005 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=2a3ab32de7034b2990d97f3e388cc52f status: fail success: false 2024-09-11T23:01:53.310 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T23:01:53.403 INFO:teuthology.run:FAIL